Linear Regression

Linear Regression

See Numerical Method Lecture Note

Cost Function

Optimization

1. Solve analytically:

• Least square solution using normal equation

• Don't need to iterate, and no need to design learning rate

• Slow if the dimension of data (n) is very large.

normal equation

2. Gradient Descent:

• can use batch gradient descent

• Idea: Make sure features are on a similar scale ( e.g ,. −1≤𝑥𝑖≤1): normalization

• Check the learning rate

• Need many iterations

• Works well even when n is large

Derive the Gradient of the cost function J

Polynomial Regression

Logistic Regression

The first thing to say is that logistic regression is not a regression, but a classification learning algorithm.