机器学习第三课

Classification:不能用线性回归
Logistic Regression:0${\leq}h_{\theta}(x){\leq}1$≤hθ​(x)≤1
(a classification algortihm)

$h_{\theta}(x)=g({\theta}^Tx)$hθ​(x)=g(θTx)
g(z)=$\frac{1}{1+e^{-z}}$1+e−z1​

$h_{\theta}(x)=P(y=1|x;{\theta})$hθ​(x)=P(y=1∣x;θ)
x,$\theta$θ已知条件下y=1的概率

Suppose predict “y=1” if $h_{\theta}(x)\geq0.5$hθ​(x)≥0.5
predict “y=0” if $h_{\theta}(x)<0.5$hθ​(x)<0.5

Decision Boundary
$h_{\theta}(x)=g(\theta_0+\theta_1x_1+\theta_2x_2)$hθ​(x)=g(θ0​+θ1​x1​+θ2​x2​) $\theta$θ=$\begin{bmatrix} -3 \\ 1 \\ 1 \end{bmatrix}\quad$⎣⎡​−311​⎦⎤​
Predict “y=1” if -3+$x_1$x1​+$x_2\geq$x2​≥ 0

Non-linear decision boundaries

Logistic regression cost function
cost($h_{\theta}(x),y$hθ​(x),y)$=\left\{\begin{aligned}-log(h_{\theta}(x)) \qquad if \quad y=1\\-log(1-h_{\theta}(x)) \qquad if \quad y=0 \end{aligned} \right.$={−log(hθ​(x))ify=1−log(1−hθ​(x))ify=0​

$J(\theta)=\frac{1}{m}\sum_{i=1}^{m} cost(h_{\theta}(x^{(i)}),y^{(i)})$J(θ)=m1​i=1∑m​cost(hθ​(x(i)),y(i))
cost($h_{\theta}(x),y)$hθ​(x),y)=-ylog($h_{\theta}(x)$hθ​(x))-(1-y)log(1-$h_{\theta}(x)$hθ​(x))

$J(\theta)=-\frac{1}{m}[\sum_{i=1}^{m}y^{(i)}logh_\theta(x^{(i)})+(1-y^{(i)})log(1-h_\theta(x^{(i)}))]$J(θ)=−m1​[i=1∑m​y(i)loghθ​(x(i))+(1−y(i))log(1−hθ​(x(i)))]

Want mine ($J(\theta)$J(θ))
Repeat
{
$\theta_j=\theta_j-\alpha\frac{\partial}{\partial\theta_j}J(\theta) \quad$θj​=θj​−α∂θj​∂​J(θ)
}simultaneously update all $\theta_j$θj​

$\frac{\partial}{\partial\theta_j}J(\theta)=\frac{1}{m}(h_{\theta}(x^{(i)})-y^{(i)})x^{(i)}_j$∂θj​∂​J(θ)=m1​(hθ​(x(i))−y(i))xj(i)​

Optimization algorithms(没有细看)

多元分类

Train a logistic regression classier $h_\theta^{(i)}(x) for \quad \qquad$hθ(i)​(x)for each class i to predict the possibility that y=i
On a new input x,to make a prediction,pick the class i that maximizes Z $\mathop{max}\limits_{i}$imax​ $h_\theta^{(i)}(x)$hθ(i)​(x)

最后说一句，写博客真的好费时间(不是)
祝能有一个不错的五一