This section will present one of the most important concepts in quantitative finance and machine learning: linear regression. Regression tasks can either be used to predict quantities (which we call regression tasks) or to predict a category (think up or down; we call these tasks classification). Understanding the key aspects of regression gives you the foundation for all of all machine learning.
In this section, we will address our case study with our first modelling algorithm: logistic regression.
In this section, we will show how to deal with one of the main weaknesses with linear models: how to address non-linearity. GAMs take the first step by using splines to better fit the data, without increasing too much complexity.
While GAMs can do a great job addressing non-linearity, they are still limited when dealing with data that has complex relationships. Decision trees are a very powerful next step.
While decision tress are a huge step-up in addressing non-linearity, they do have many weaknesses. Random Forests address many of them.