- Residuals in Linear Regression:
In linear regression, the goal is to model the relationship between independent variables (features) and a dependent variable (target) using a linear equation. The model makes predictions based on this equation. The difference between the actual observed values and the predicted values is called the “residual.”
- Purpose of Residual Plots:
Residual plots are used to assess the validity of several key assumptions made in linear regression:
– Linearity Assumption: Residual plots help visualize whether the relationship between the independent variables and the dependent variable is adequately captured by a linear model. In a good linear model, the residuals should exhibit a random scatter around zero.
– Homoscedasticity Assumption: Homoscedasticity means that the variance of the residuals is constant across all levels of the independent variables. Residual plots help check for homoscedasticity. In a homoscedastic model, the spread of residuals should be roughly uniform across the predicted values.
– Normality Assumption: The residuals should follow a normal distribution. Residual plots, particularly histograms of residuals, can help assess whether the residuals are approximately normally distributed.
- Components of the Code:
Here’s a breakdown of the code’s components and how they relate to the theory:
– Calculate Residuals: `residuals = model.resid` calculates the residuals for each observation by subtracting the predicted values from the actual observed values. These residuals represent the discrepancies between the model’s predictions and the actual data.
– Scatterplot of Residuals vs. Predicted Values: `plt.scatter(model.fittedvalues, residuals)` creates a scatterplot with the predicted values on the x-axis and residuals on the y-axis. This plot assesses the linearity assumption. A good linear model would have residuals scattered randomly around zero, indicating that the model captures the linear relationship well.
– Horizontal Red Line: `plt.axhline(0, color=’red’, linestyle=’–‘)` adds a horizontal dashed line at y=0. This line represents the ideal situation where residuals are zero for all predicted values.
– Histogram of Residuals: `plt.hist(residuals, bins=30, edgecolor=’k’)` creates a histogram of residuals to assess the normality assumption. In a normally distributed model, the histogram should resemble a bell-shaped curve.
- Interpretation:
– In the scatterplot, a random scatter of residuals around the zero line suggests that the linearity assumption is reasonable.
– In the histogram, a roughly bell-shaped curve indicates that the normality assumption is met.
– For homoscedasticity, examine the scatter of residuals in the scatterplot. Ideally, there should be no discernible pattern or funnel shape in the residuals as you move along the x-axis.
Residual plots provide valuable diagnostic information to understand how well your linear regression model aligns with the underlying assumptions. If the plots reveal significant departures from the assumptions, it may be necessary to revisit the model or consider transformations or other adjustments to improve its performance.