# SVM

## SVM Introduction

The boundary separating the examples of different classes is called the decision boundary.

In SVM, a hyperplane is used to make the boundary to classify the feature **X** as the label Y=+1 or Y=-1. The hyperplane is expressed with **two parameters w,** b

$$
{\bf{wx}}-b = 0
$$

where the expression **wx** means w(1)x(1) + w(2)x(2) + . . . + w(D)x(D), and D is the number of dimensions of the feature vector x.

The feature input X can be classified as Y=+1 or Y=-1, by the condition of

$$
y = sign({\bf wx} − b)
$$

### Classification

Choose the following method if the classification is using

**Linear (hyperplace) Decision Boundary** of Hard margin classification

* **Maximal Margin Classifier**
  * Hard margin classification is very sensitive to outliers in the training data

\*\*Linear Decision Boundary \*\* of Soft margin classification

* **Support Vector Classifier (Linear)**
  * Allows misclassification
  * But it may not work for certain data without changing the dimensions.

**Non-Linear Decision Boundary**

* **Support Vector Machine Classifier (Non-Linear)**
  * Use kernel function to make it higher dimension.
  * polynomial kernel, Gaussian RBF kernel
  * Use _Kernel Trick_ to reduces the computation that transforms the data from low to high dimension

### Regression

* **Support Vector Regression**

#### Method Comparison

**Maximal Margin Classifier vs SVC**\*\*

![image-20220121200932005](https://3698175758-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2Fgit-blob-c50a9023c649f95c052c109b5bd7768efb7f99bf%2Fimage-20220121200932005.png?alt=media)

**SVC** vs **SVM**

![image-20220121201030122](https://3698175758-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2Fgit-blob-5c148371978d4a19f99b4b7110ca8753adeddc53%2Fimage-20220121201030122.png?alt=media)

![image-20220121203515848](https://3698175758-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2Fgit-blob-25ba23077caca53c6566d6e171c9adf0228c84ca%2Fimage-20220121203515848.png?alt=media)

![image-20220121203533786](https://3698175758-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2Fgit-blob-34d4a78c195241a8694ef038455b9e6b92f4fd04%2Fimage-20220121203533786.png?alt=media)

## Study plan

Study SVM in the following order:

1. SVM concept
2. Maximal margin classifier
3. Support vector classifier
4. Support vector machine classifier
5. support vector regression
