0
$\begingroup$
x<-c(1,2,3) y<-x^1.1+x summary(lm(y~x+I(x^1.1))) 

I have this code in R but it just is for the sake of easier understanding of what I am trying to achieve. If you execute the code, it will show the correct estimates, but it won't show p-values (shows NA) for each estimate, thus I am unable to test the significance. How can this be explained in a mathematical way?

P.S. if I add one more value to each of the vectors, p-value appears again. However, if I add another variable, let's say

summary(lm(y~x+I(x^1.1)+I(x^1.2))) 

again p-values are all NAs

1 Answers 1

1

As you are finding out, when the number of your observations (n say) equals the number of parameters to be estimated (p say), you do not get any distributional results. I am sure (but not certain) that the many theorems for the linear model state somewhere that you need n>p. However the problem when $n=p$ can be seen more clearly by considering the t-statistics for testing the null hypothesis that $\beta_{0}$ say is zero (this could be the intercept for example). I believe (again I am not certain) that the t-statistics divides the estimate $\hat{\beta}_{0}$ by a quantity that is a function of the sum of squares residuals $\sum\nolimits_{i=1}^{n}(y_{i}-\hat{y}_{i})$ which in your case is zero since you have a perfect fit. This makes the denominator zero and so you cannot divide. Thus you cannot compute the test statistic. I think you will always get a perfect fit if $n=p$. In general in statistics there needs to be much more data than parameters for things to behave well.