Regularization

Intro

Regularization is an additional part in loss function or optimiazation loss, which prevent the loss function from doing too well on training data.

Formulation

indeed, the regularation prefer a smaller W intuitively for constructing a simple model.

Intuition

chosing L1-norm result in a pluse weight,while chosing L2-norm result in a plain vector.

---

pics from: cs231n