Regularized linear models, and spectral clustering with eigenvalue decomposition

less than 1 minute read

Implemented linear, ridge, and lasso regression on quantitative data. Performed spectral clustering on graphical data.

1. Pre-processing

Cleaned the quantitative data and set up everything in the format: $y=\beta_{1}x_{1}+\beta_{2}x_{2}+\beta_{3}x_{3}+...+\beta_{n}x_{n}$

For the further explanation, we represent $\beta_{1}x_{1}+\beta_{2}x_{2}+\beta_{3}x_{3}+...+\beta_{n}x_{n}$ as $XB$

2. Linear regression

Result:

3. Ridge regression

Through ridge regression, the linear regression’s RHS is L2-regularized as $\hat{\beta^{ridge}}=argmin\lVert{y-XB}\rVert_{2}^{2}+\lambda\lVert{B}\rVert_{2}^{2}$

Result:

4. Lasso regression

Through ridge regression, the linear regression’s RHS is L1-regularized as $\hat{\beta^{ridge}}=argmin\lVert{y-XB}\rVert_{2}^{2}+\lambda\lVert{B}\rVert$

Result: The top features are as follows

5. Degree and laplacian matrix

Firstly, the graph dataset is represented in the Adjacency Matrix format, i.e., if node 1 is connected to node 2, $A_{12}=1$ , else, $A_{ij}=0$

Next, the degree matrix is computed. It is defined as the diagonal matrix $D=diag(d_{1},d_{2},d_{3},...,d_{n})$ corresponding to the graph that has the vertex $d_{i}$ in the $i^{th}$ position.