Linear classifier
We could see that there is two pictures to show two different points. They are two types in the chart.
We also could use some function to describe the side of reds and blues.
A linear classifier tries to identify which class an example belongs to by making a decision based on the value of a linear combination of the features:
It also has some advantages and disadvantages:
Fitting the model
If we get this picture, we also see the two types of points in here and a clear line is divided it.
In this picture, h(x) > 0 for all training examples belonging to the class r = +1 and h(x) < 0 for all training examples belonging to the class r = −1 .
Fitting the model: 0-1 loss
But in sometimes, we don’t always get this standard situation. So, we need some method to control the result of fitting.
We could use this function to control the result.
That is function to add some importance on the examples.
Margin
We could see that pictures, there is a space consists of many lines to diveide the area in to two parts, whose function is that:
The distance of the sides of special area is that:
When the h(X) = 0, the distance of the closest two different type is :
So, we could change this distance into another type in math:
Non-linearly separable classes
If we can’t use line to devide two types in char, we will use the method of Non-linearly to decribe the side of two types.
Nonlinear SVMs
For the 3d aspect, if we could use the line to decribe the side of two types, we can also use the nonlinear SVMs to solve this problem.
For this model, we could also use some math knowledge to make it easy. Then, we could get the limit of this function.
Kernels
Kernel function is also a useful way to canlulate the liner product.
Some example to explain it: