20180713DLday1课程笔记

Introduction

---

0

• Jian Tang
• tangjianpku@gmail.com

1

• History 1950-1970 logic rules; 1980-1990 knowledge acquisition; 2010

-. machine learning

• DeepLearning⊂MachineLearning⊂ArtificialIntelligence$DeepLearning\subset MachineLearning\subset ArtificialIntelligence$
• machine learning
  
  • use statistical techniques, “learn” with data
  • extract features automatically, instead of by domain experts
  • learn automatically, instead of explicit programming
• Big Data-Big Computation-Big Model : Why deep learning now
• usage
  
  • …

2Probability

• Bayes’ Theorem

  • p(Y|X)=p(X|Y)p(Y)p(X),p(X)=∑Yp(X|Y)p(Y)$p(Y|X)=\frac{p(X|Y)p(Y)}{p(X)},p(X)=\sum _{Y}p(X|Y)p(Y)$
  • posterior ∝$\propto$ likelihood * prior

• variables

  • E[f] := the average value of f(X) under the distribution p(x)
  • E[f]=∑xp(x)f(x)$E[f]=\sum _{x}p(x)f(x)$
  • V[f], cov[x, y]

• distributions

  • binomial distribution
  • Bin(m|N,μ)=(Nm)μm(1−μ)N−m$Bin(m|N,\mu )=(\genfrac{}{}{0ex}{}{N}{m}){\mu }^m(1-\mu {)}^{N-m}$
  • E[m]=Nμ,var[m]=Nμ(1−μ)$E[m]=N\mu ,var[m]=N\mu (1-\mu )$

• multinomial variables

  • x可以取k种值，x=(0,0,1,0,0,0)T$x=(0,0,1,0,0,0{)}^T$表示x取了六种中的第三种

  μ=(μ1,μ2,...,μk)T$\mu =({\mu }_1,{\mu }_2,...,{\mu }_k{)}^T$，对应x向量每个位置上为1的概率

  从而某个特定的x出现的概率 p(x|μ)=∏k=1Kμxkk$p(x|\mu )=\prod _{k=1}^K{\mu }_k^{x_k}$ (也就是μk${\mu }_k$)

  • E[x|mu]=∑xp(x|μ)x=(μ1,μ2,...,μk)T=μ$E[x|mu]=\sum _{x}p(x|\mu )x=({\mu }_1,{\mu }_2,...,{\mu }_k{)}^T=\mu$

  • maximum likelihood estimation

  μk=mkN,mk=∑Nxnk观察值的矩阵的每列和${\mu }_k=\frac{m_k}{N},m_k=\sum _N{x}_{nk}观察值的矩阵的每列和$

• gaussian univariate distribution正态分布

  • multivariate gaussian distribution
  • maximum likelihood estimation
  • mixture of gaussians-可以模拟其他各种分布

• gradient descent梯度下降

  • a way to minimize an object function J(θ)$J(\theta )$
  • η$\eta$: learning rate, which determines the size of the steps we take to reach a local minimum
  • update equation: θ=θ−η∗∇θJ(θ)$\theta =\theta -\eta \ast {\mathrm{\nabla }}_{\theta }J(\theta )$