Spurious Regression

Lots of things are non-stationary and these are much harder to deal with, but can typically be made stationary by differencing

Definition: A time series $\{X_t\}$ is integrated of order d, denoted $\{X_t\}\sim I(d)$, if $d$ is the smallest number such that $\{{\Delta }^d{X}_t\}$ is weakly stationary

Now the problem of spurious regression is as follows,

• Let $\{Y_t\}\sim I(1)$ and $\{X_t\}\sim I(1)$ be independent, for example two independent random walks
• What happens if we perform the regression $Y_t=\hat{\alpha }+\hat{\beta }{X}_t+{\widehat{ϵ}}_t$? We expect to get $\hat{\beta }\to 0$ and $R^2\to 0$, but instead they converge to random variables and our test statistic diverges

This is a big problem! We want to be able to judge if one variable explains another

How do we judge the order of integration of a time series?

For an $\text{AR}(1)$ process, we focus on testing $H_0:\varphi =1$ versus $H_1:|\varphi |<1$, which are the most practically relevant cases in economics

Definition: The Dickey-Fuller unit root test (DF test) makes use of the $t$-ratio $\text{DF}=\frac{\hat{\varphi }-1}{\text{SE}(\hat{\varphi })}$, where $\hat{\varphi }$ is the least squares estimate of $\varphi$ in the $\text{AR}(1)$ model

This does not follow a regular distribution

When $\{X_t\}$ is stationary, $\hat{\varphi }$ is $\sqrt{T}$ consistent, i.e. $\sqrt{T}(\hat{\varphi }-\varphi )$ is asymptotically normal

When $\{X_t\}$ is non-stationary, $\hat{\varphi }$ is still consistent, but $\sqrt{T}(\hat{\varphi }-\varphi )$ also converges to $0$! So we must multiply by a larger number, and we get the Dickey-Fuller distribution

Example:
$P(\text{DF}<-2.58)=0.01$
$P(\text{DF}<-1.95)=0.05$
$P(\text{DF}<-1.62)=0.10$

Suppose you get $\hat{\varphi }=0.975$ and $\text{SE}(\hat{\varphi })=0.01$

$t=\frac{0.975-1}{0.01}=-2.5$

So $H_0$ is rejected at $5\%$ and $10\%$ significant levels, which lets us conclude that the process is stationary

How do we test general $\text{AR}(p)$ models? We use the Augmented Dickey-Fuller test

Regress $\Delta X_t$ on $X_{t-1}$ and test if $\beta =0$

To start, let’s consider how his works on an $\text{AR}(1)$ model (which of course it covers)

$X_t=\varphi X_{t-1}+{ϵ}_t$
$X_t-X_{t-1}=\varphi X_{t-1}-X_{t-1}+{ϵ}_t$
$\Delta X_t=\beta X_{t-1}+{ϵ}_t$, where $\beta =\varphi -1=0$ corresponds to a unit root

So our test statistic is $\text{ADF}=\frac{\hat{\beta }}{\text{SE}(\hat{\beta })}$

Now, note that adding an intercept or any other trend requires a different simulated distribution to test against

Extending to $\text{AR}(p)$ amounts to rewriting the process as a regression with $\beta ={\sum }_{i=1}^p{\varphi }_i-1$ and testing $\hat{\beta }$,
$\Delta X_t=\beta X_{t-1}+{\varphi }_2^{\ast }\Delta X_{t-1}+\cdots +{\varphi }_p^{\ast }\Delta X_{t-p+1}+{ϵ}_t$, where ${\varphi }_j^{\ast }=-{\sum }_{i=j}^p{\varphi }_i$

We can keep differencing and testing until the test succeeds to figure out the order of the time series

Given sample data, how do we choose a value of $p$ for the ADF test?

We estimate the ADF regression for $p=1,\dots ,p_{\text{max}}$ with/without intercept/trend, storing the BIC values

Then we find the best model whose residuals tests negative for autocorrelation