There are mainly two types of regularization techniques, which are given below:

• • Ridge Regression
• • Lasso Regression

Ridge Regression

• • Ridge regression is one of the types of linear regression in which a small amount of bias is introduced so that we can get better long-term predictions.
• • Ridge regression is a regularization technique, which is used to reduce the complexity of the model. It is also called as L2 regularization.
• • In this technique, the cost function is altered by adding the penalty term to it. The amount of bias added to the model is called Ridge Regression penalty. We can calculate it by multiplying with the lambda to the squared weight of each individual feature.
• • The equation for the cost function in ridge regression will be:

• • In the above equation, the penalty term regularizes the coefficients of the model, and hence ridge regression reduces the amplitudes of the coefficients that decreases the complexity of the model.
• • As we can see from the above equation, if the values of λ tend to zero, the equation becomes the cost function of the linear regression model. Hence, for the minimum value of λ, the model will resemble the linear regression model.
• • IA general linear or polynomial regression will fail if there is high collinearity between the independent variables, so to solve such problems, Ridge regression can be used.
• • It helps to solve the problems if we have more parameters than samples.

Lasso Regression:

• • Lasso regression is another regularization technique to reduce the complexity of the model. It stands for Least Absolute and Selection Operator.
• • It is similar to the Ridge Regression except that the penalty term contains only the absolute weights instead of a square of weights.
• • Since it takes absolute values, hence, it can shrink the slope to 0, whereas Ridge Regression can only shrink it near to 0.
• • It is also called as L1 regularization. The equation for the cost function of Lasso regression will be:

• • Some of the features in this technique are completely neglected for model valuation.
• • Hence, the Lasso regression can help us to reduce the overfitting in the model as well as the feature selection.

Key Difference between Ridge Regression and Lasso Regression