> For the complete documentation index, see [llms.txt](https://ykkim.gitbook.io/wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ykkim.gitbook.io/wiki/machine-learning/lecture-notes/svm-introduction.md).

# SVM

## SVM Introduction

The boundary separating the examples of different classes is called the decision boundary.

In SVM, a hyperplane is used to make the boundary to classify the feature **X** as the label Y=+1 or Y=-1. The hyperplane is expressed with **two parameters w,** b

$$
{\bf{wx}}-b = 0
$$

where the expression **wx** means w(1)x(1) + w(2)x(2) + . . . + w(D)x(D), and D is the number of dimensions of the feature vector x.

The feature input X can be classified as Y=+1 or Y=-1, by the condition of

$$
y = sign({\bf wx} − b)
$$

### Classification

Choose the following method if the classification is using

**Linear (hyperplace) Decision Boundary** of Hard margin classification

* **Maximal Margin Classifier**
  * Hard margin classification is very sensitive to outliers in the training data

\*\*Linear Decision Boundary \*\* of Soft margin classification

* **Support Vector Classifier (Linear)**
  * Allows misclassification
  * But it may not work for certain data without changing the dimensions.

**Non-Linear Decision Boundary**

* **Support Vector Machine Classifier (Non-Linear)**
  * Use kernel function to make it higher dimension.
  * polynomial kernel, Gaussian RBF kernel
  * Use *Kernel Trick* to reduces the computation that transforms the data from low to high dimension

### Regression

* **Support Vector Regression**

#### Method Comparison

**Maximal Margin Classifier vs SVC**\*\*

![image-20220121200932005](/files/eVipdazSrsfBBkv2wrB8)

**SVC** vs **SVM**

![image-20220121201030122](/files/KutGk9PkW1wECcMfNezq)

![image-20220121203515848](/files/51ePKAW7gnQt3qJKslNV)

![image-20220121203533786](/files/2vXFY7mpd3gdKvlR23x4)

## Study plan

Study SVM in the following order:

1. SVM concept
2. Maximal margin classifier
3. Support vector classifier
4. Support vector machine classifier
5. support vector regression


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ykkim.gitbook.io/wiki/machine-learning/lecture-notes/svm-introduction.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.