机器学习基石 - Nonlinear Transformation

机器学习基石下 (Machine Learning Foundations)—Mathematical Foundations
Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

Quadratic Hypotheses

• circular separable 圆圈可分

• feature transform

  

• perceptrons in Z$\mathcal{Z}$-space ⟺$⟺$ quadratic hypotheses in X$\mathcal{X}$-space

Nonlinear Transform

• The Nonlinear Transform Steps

  

• linear model A$\mathcal{A}$ , not just binary classification

Price of Nonlinear Transform

• Q次多项式转换

  

• d维数据，转换的Q次项有CdQ+d$C_{Q+d}^d$个

  • Q次项的数量级最大，小于Q次的项达不到这个量级
  • x1+x2+...+xd=Q$x_1+x_2+...+x_d=Q$，非负数解的个数
  • 变换后自由度变的很大

• careful about your brain’s ‘model complexity’

  human learning 加到模型上去，造成模型不错的错觉

Structured Hypothesis Sets

• structure: nested Hi${\mathcal{H}}_i$

  

• Structured Hypothesis Sets

  

• safe route

  

• linear model: simple, efficient, safe, and workable!