8. Use the Model for Estimation and Prediction - Yousef's Notes
8. Use the Model for Estimation and Prediction

8. Use the Model for Estimation and Prediction

#Key Metrics

Is the model adequate?

To determine if the model is adequate, that is, if there is a linear relationship (straight line) between the independent variable and the dependent variable โ†’ Inferences about the true slope ๐œท1. It is essential to have statistical evidence to prove that the value of ๐œท1 is not equal to zero for our model to be adequate.

Is the model useful?

To evaluate how useful our model is, we must pay attention to its accuracy and the degree of association between x and y in our sample โ†’ 2s, mean error, AIC/BIC and $r^2$

The lower the value of 2s and the higher the value of r2, the better our model will be.

#Prediction and Estimation

We have been talking about model creation and evaluation, now you may be wondering…. can I use this model to predict values? Of course, we can use the predict function to create a new column in our data set containing the sales values estimated by the model.

If we want to check that the errors are distributed in a normal way we must create a new column in the dataset with the residuals and then make a histogram with that column.

Obviously, this example is not very representative because the dataset is very small. That way, we cannot say that the errors follow a normal distribution with mean 0. The standard deviation is 0.469. Imagine that we have normality of the errors with a larger sample, this way we can assume most of the assumptions we needed to consider this linear regression model. np.std(residuals) 0.469

This column is generated by substituting the values of the regression equation of the model โ†’ ลท = -0.1 + 0.7x

However, these values do not take into account the error component of the model, they simply take into account the estimator, or the expected value of y according to the model.

To account for the error component, we will use the following.

If we are satisfied that a useful and adequate model has been found to describe the relationship between reaction time and percent of drug in the bloodstream, we are ready for using the model for estimation and prediction. The most common uses of a probabilistic model for making inferences can be divided into two categories:

  • Estimate the mean value of y, E(y) for a specific x. For our drug reaction example, we may want to estimate the mean response time for all people whose blood contains 4% of the drug.
  • Predict a new individual y value for a given x. That is, we may want to predict the reaction time for a specific person who possesses 4% of the drug in the bloodstream.

In the first case, we are attempting to estimate the mean value of y for a very large number of experiments at the given x value. In the second case, we are trying to predict the outcome of a single experiment at the given x value.

When we want to estimate the real value of a population parameter from sample data, what do we use?

To estimate the mean value of y for a very large number of experiments at the given x value โ†’ Confidence Interval for the Mean of y (E(y)) at some value of x

To predict the outcome of a single experiment at the given x value โ†’ Prediction Interval for an Individual New Value of y at some value of x

#Estimation Example

Refer to the simple linear regression on drug reaction. Find a 95% confidence interval for the mean reaction time when the concentration of the drug in the bloodstream is 4% (x = 4).

#Prediction Example

Refer to the simple linear regression on drug reaction. Predict the reaction time for the next performance of the experiment for a subject with a drug concentration of 4% (x = 4). Use a 95% prediction interval.

#Confidence Interval vs Prediction Interval

The main difference between these intervals is the error. While the confidence interval is used to estimate the value of E(y) from the sample information (at some value of x), the prediction interval is used to estimate y from the sample information (at some value of x). Recall that a probabilistic model has two components, the deterministic component and the random error.

y = ๐œท0 + ๐œท1x + ๐œบ , E(y) = ๐œท0 + ๐œท1x โ†’ y = E(y) + ๐œบ

So, in short, when we want to estimate y, we have to estimate E(y) first. When we make a confidence interval we have the error of the estimation of E(y), but when we make a prediction interval we have the error of the estimation of E(y) + the error of the estimation of y. To put it simple:

  • Confidence interval to estimate the mean value of y E(y) at some value of x -> ‘1 ERROR’.
  • Preiction interval for estimating the value of y from a new observation at some value of x -> ‘2 ERRORS’