Probability & Statistics · Lecture 4 of 5

Time series

When today depends on yesterday: measure change, model persistence, and know when a pattern misleads

1 Describe2 Model3 Interpret

Developed from Ian Crawford’s lecture material. Notes, slides and PDF: fatih.ai/probabilityandstatistics

Opening question

Two simulated series correlate at 0.71. Did the generator link them?

$X_t,\ Y_t$ two series, generated by a computer

$0.71$ sample correlation of their levels across the paired dates

?how each series was generated

Your answerVote now. Part 3 answers it.

AYes: 0.71 is too strong to be chance

BYes, if the sample is long

CThe coefficient alone cannot tell

Why start here

Prices, wages, money and output all drift over time. Before reading a link into a strong correlation between two such series, ask how each one moves on its own.

Lecture 4 · The route

Describe the data, model their memory, then interpret their patterns

Part 1

Order, units and change

What does a series record, and how do we measure its change without losing the dates and units?

time seriesautocorrelationdifferencegrowth factorlog change

Notes 4.1–4.2

Part 2

Models with memory

What must stay unchanged for the past to inform the future, and does a shock fade or last?

stationaritywhite noiserandom walkAR(1)forecast band

Notes 4.3–4.5

Part 3

Reversion and spurious correlation

When does movement towards the mean, or a strong correlation between two series, reveal a real mechanism?

mean reversionregression to the meanspurious regressioncointegration

Notes 4.6–4.7

By the end of this lecture you can

Compute differences, growth factors, average growth and log changes, keeping the units and dates.
Derive the mean $c/(1-\phi)$ , variance and autocorrelation $\phi^h$ of a stable AR(1).
Forecast an AR(1) with a 95% interval, and explain why its band levels off but a random walk’s does not.
Explain why regression to the mean, and strong correlations between random walks, need no mechanism.

Part 1 of 3

Order, units and change

What does a series record, and how do we measure its change without losing the dates and units?

Key terms

time seriesautocorrelationdifferencegrowth factorlog change

Notes 4.1–4.2 ↗

Why it matters

Economic statistics arrive as changes: quarter on quarter, year on year, annualised or in log points. Equal absolute and equal percentage changes answer different questions, and mis-compounding, such as multiplying a quarterly rate by four, misstates growth.

In this part

4.1Time series, processes and the layers of a plot
4.1Autocorrelation and the correlogram
▶Animation: why a 20% rise and a 20% fall do not cancel
4.2Differences, growth factors and log changes

4.1Order is information

The date belongs to the observation: read a series in layers

Definition

Time series and process

A time series records a variable in chronological order. Its probability model is a stochastic process $\{X_t\}\text{;}$ we observe one path $x_1,\ldots,x_T\text{.}$ At date $t\text{,}$ the value at $t+1$ is not yet known.

Trend	a sustained change in level
Seasonality	repeats at a known calendar frequency
Cycle	rises and falls with no fixed period
Break	a change in mean, slope or volatility at a date

Hypothetical construction: $X_t=100\exp(0.025t+0.16\sin(\pi t/2))$ . The growing level and seasonal amplitude come from this formula, not historical observations.

Hypothetical: $X_t=100\exp(0.025t+0.16\sin(\pi t/2))$

Check

Every December is high, and the December peak rises from year to year. Which two features are present?

Answer

Seasonality (the timing) and a trend (the change across years). They can coexist.

“Time series” refers to ordered data, “stochastic process” to their probability model. A linear trend adds about the same amount each period; an exponential trend multiplies by about the same factor. Sorting the rows, deleting the dates or filling a missing month with 0 destroys information: a missing month is not a zero.

Quarterly nominal income in pounds is not quarterly real income; a stock measured at month-end is not a monthly flow.

The decks’ series work as questions, not estimates: Canadian housing starts, 1959q1–2000q4, show seasonality with booms and busts; the US dollar–sterling rate has breaks marked in 1967 and 1971. A visible break does not identify its cause. The decomposition $X_t=m_t+s_t+U_t$ is a model, not an identity with unique pieces.

4.1Dependence across dates

Autocorrelation compares a series with its own past

Key formula Definition (1)

Autocorrelation at lag $h$

\rho_t(h)=\frac{\operatorname{Cov}(X_t,X_{t-h})}{\sqrt{\operatorname{Var}(X_t)\operatorname{Var}(X_{t-h})}}

In words $\operatorname{Corr}(X_t,X_{t-h})\text{:}$ how today moves, linearly, with the value $h$ periods earlier. Write $\rho(h)$ when it depends on the lag alone.

Needs finite, positive variances.

Result reading a sample correlogram

Lag $h$ leaves only $T-h$ pairs.
Slow decay can mean persistence, or a trending level.
Peaks at lags 4, 8, 12 suggest quarterly seasonality.

Common mistake

Small sample autocorrelations do not prove independence, nor rule out nonlinear predictability.

Check

Monthly sales are sorted from low to high, the dates deleted and one month is missing. Does a lag-one correlation still measure monthly persistence?

Answer

No: sorted neighbours are close by construction, not in time, and a missing month is not a zero. Restore the calendar, then form lag pairs.

Final frame: three bars drawn to scale at t = 0, 1, 2, with a dashed reference line at 100. Bar 0 is 100; bar 1 is 120, its top 20 shaded green; bar 2 is 96, with the removed 24 shown as a red block beside it, taller than the green 20. Under the bars, the factors ×1.20 and ×0.80, and below them the equal factors ×0.9798 and ×0.9798 of a dashed yellow steady path from 100 to 96. Beside it: start X₀ = 100; 100 × 1.20 = 120, the rise is 20% of 100, +20; 120 × 0.80 = 96, the fall is 20% of 120, −24; overall 96/100 − 1 = −4%; (1 + ḡ)² = 0.96, so ḡ = √0.96 − 1 ≈ −2.02%, not the simple average (20% − 20%)/2 = 0.

4.2Measuring change

Differences subtract, growth factors multiply, logs add

Definition

Differences

$\Delta X_t=X_t-X_{t-1}\text{:}$ an absolute change, in units of $X$ per period. $\Delta_4X_t=X_t-X_{t-4}$ compares matching quarters; it is not $\Delta^4X_t\text{.}$

Key formula Definition (2)

Growth factors compound

(1+\bar g)^h=\frac{X_t}{X_{t-h}}

In words with rate $g_t=X_t/X_{t-1}-1\text{,}$ the growth factor $1+g_t$ multiplies the level; $\bar g$ compounds to the observed ratio.

Needs positive levels.

Key formula Identity (3)

Log change

r_t=\Delta\log X_t=\log(1+g_t)

In words the log of the growth factor; log changes add over time. Close to $g_t$ only for small rates: 0.20 is growth of $e^{0.20}-1\approx22.14\%$ .

Needs $X_t>0\text{;}$ natural logarithms.

Worked example compounding

Up 20%, down 20%: $(1+\bar g)^2=0.96\text{,}$ so $\bar g\approx-2.02\%$ per period, not 0. And 2% a quarter is $(1.02)^4-1\approx8.243\%$ a year, not 8%.

Three different questions: by how much (a difference), by what factor (growth), and in log points (a log change). Check: an index goes from 100 to 121 in two quarters; the equivalent quarterly rate is $\sqrt{1.21}-1=10\%\text{,}$ and the average log change $\log(1.21)/2\approx0.09531$ is close to 0.10 but not equal. The lag operator $LX_t=X_{t-1}$ gives $\Delta=1-L$ and $\Delta_4=1-L^4\text{.}$ $T$ observations give $T-1$ first differences and $T-4$ seasonal differences: the losses are part of the definition.

Logs turn an exponential trend into a linear one, which still moves with time: they do not remove a trend. Adding a constant so that negative values can be logged changes ratios and the economic interpretation. $\log X_t-\log X_{t-1}$ is a log ratio, not $\log(X_t-X_{t-1})\text{.}$

Part 2 of 3

Models with memory

What must stay unchanged for the past to inform the future, and does a shock fade or last?

Key terms

stationaritywhite noiserandom walkAR(1)forecast band

Notes 4.3–4.5 ↗

So far a series is one path of a process; autocorrelation measures how it depends on its own past.

Why it matters

Any forecast of unemployment, earnings or an exchange rate rests on a model of persistence. Whether a shock is temporary or permanent decides where the forecast goes, and whether its uncertainty stays bounded.

In this part

4.3Weak stationarity: what must stay unchanged
4.4White noise and the random walk
4.3Choosing a transformation from the model
4.4The AR(1) and its special cases
4.5Mean, variance and autocorrelation of a stable AR(1)
4.5AR(1) forecasts and their uncertainty
▶Animation: the forecast returns towards the mean inside a band that levels off

4.3What must stay unchanged

Stationarity: the same probability structure at every date

Key formula Definition (4)

Weak stationarity

\begin{aligned}E[X_t]&=\mu\\ \operatorname{Var}(X_t)&=\gamma(0)<\infty\\ \operatorname{Cov}(X_t,X_{t-h})&=\gamma(h)\end{aligned}

In words the same mean and finite variance at every date, and a covariance that depends only on the gap $h\text{.}$ Then $\rho(h)=\gamma(h)/\gamma(0)$ .

Needs finite second moments; it restricts moments, not the whole distribution.

Stationarity lets one stretch of history inform another: the role that iid sampling played in Lecture 3.

Common mistake

Stationary is not flat-looking: long runs above the mean are allowed. Nor is zero autocorrelation enough: $X_t\sim N(0,t)$ is uncorrelated but not stationary.

Check

With $Z_t$ iid $N(0,1)\text{,}$ which of $3+Z_t\text{,}$ $t+Z_t$ and $\sqrt t\,Z_t$ is weakly stationary?

Answer

Only $3+Z_t\text{:}$ mean 3, variance 1. $t+Z_t$ has mean $t\text{;}$ $\sqrt t\,Z_t$ has variance $t\text{.}$

4.4Two building blocks

White noise is the building block; a random walk adds the shocks up

Definition

White noise

The innovation $\varepsilon_t$ is the new component at date $t\text{:}$ iid, mean 0, variance $\sigma^2>0\text{,}$ independent of the past. Unforecastable in mean, not in spread.

Key formula Result (5)

The random walk accumulates shocks

X_t=X_{t-1}+\varepsilon_t=x_0+\sum_{j=1}^{t}\varepsilon_j

In words the start plus every shock so far: $E[X_t]=x_0\text{,}$ but $\operatorname{Var}(X_t)=t\sigma^2$ grows, so it is not stationary.

Needs a fixed start $x_0$ and iid mean-zero shocks.

Result what a walk implies

Every shock carries into every later level: a random walk has no restoring force.
$\operatorname{Corr}(X_t,X_{t-h})=\sqrt{(t-h)/t}$ depends on the date: there is no stationary ACF.
The forecast of $X_{t+h}$ is $X_t\text{,}$ with error variance $h\sigma^2\text{.}$
With drift $d\text{,}$ the mean is $x_0+td$ and $\Delta X_t=d+\varepsilon_t\text{.}$

Common mistake

“It will come back, because its mean is constant.” The constant mean is an average across possible paths at one date; it does not pull a path back. The best forecast is today’s level.

4.3Detrend or difference

Detrend or difference? The model decides

Detrend a deterministic trend; difference a unit root
Model	Transformation	What is left
$X_t=a+bt+U_t$	subtract the trend	$U_t\text{,}$ stationary
$X_t=X_{t-1}+\varepsilon_t$	first difference	$\Delta X_t=\varepsilon_t$
$U_t\text{,}$ already white noise	a difference too many	lag-one correlation $-\tfrac12$

Result over-differencing

$V_t=U_t-U_{t-1}$ has variance $2\sigma^2$ and lag-one covariance $-\sigma^2\text{:}$ the transformation created the correlation.

Common mistake

Fitting a straight line to a random walk does not make what is left stationary.

Check

Can a log transformation remove the growing variance of a Gaussian random walk that takes negative values?

Answer

No: the log is not even defined at negative values. The model justifies the first difference.

4.4Autoregression

One recursion: shocks that vanish, last for ever, or fade

Key formula Definition (6)

The AR(1) model

X_t=c+\phi X_{t-1}+\varepsilon_t

In words an intercept, plus $\phi$ times yesterday, plus a new shock. An AR( $p$ ) adds lags up to $X_{t-p}\text{.}$

Needs white-noise shocks; a stable solution needs $|\phi|<1\text{.}$

Restriction	Model	A shock …
$c=0,\ \phi=0$	white noise	vanishes at once
$c=0,\ \phi=1$	random walk	lasts for ever
$c=d,\ \phi=1$	walk with drift	lasts, around a drifting mean
$\|\phi\|<1$	stable AR(1)	fades: $1,\ \phi,\ \phi^2,\ldots$

Check

Compare $X_t=2+X_{t-1}+\varepsilon_t$ with $Y_t=2+0.5Y_{t-1}+\varepsilon_t$ . Which has a stationary long-run mean?

Answer

Only $Y\text{:}$ $\mu=2+0.5\mu$ gives $\mu=4\text{.}$ $X$ is a walk with drift 2; its mean $x_0+2t$ keeps rising.

How long a shock lasts: a unit shock leaves $1,0.8,0.64,\ldots$ when $\phi=0.8\text{,}$ $1,-0.8,0.64,\ldots$ when $\phi=-0.8\text{,}$ and never decays when $\phi=1\text{.}$ A “causal solution” of an AR model is built from present and past shocks: a time-series construction, not a policy effect.

“Order” counts the included lags, not a polynomial trend in time. The decks write $\theta_0,\theta_1$ for $c,\phi$ and $e_t$ for $\varepsilon_t\text{.}$

For an AR( $p$ ), stability depends on the roots of $1-\phi_1z-\cdots-\phi_pz^p\text{,}$ which must lie outside the unit circle; checking that each coefficient is below one is not a test.

4.5The stationary AR(1)

A stable AR(1): the intercept is not the mean

Key formula Result (7)

Stationary mean and variance

\mu=\frac{c}{1-\phi},\qquad v=\frac{\sigma^2}{1-\phi^2}

In words expectations of the recursion give $\mu=c+\phi\mu\text{;}$ variances give $v=\phi^2v+\sigma^2\text{.}$

Needs $|\phi|<1\text{,}$ the stationary solution.

Key formula Result (8)

Autocorrelation of an AR(1)

\rho(h)=\phi^h\qquad(h\ge0)

In words the new shock is independent of the past, so $\gamma(h)=\phi\,\gamma(h-1)\text{;}$ and $\rho(1)=\phi\text{.}$

Exact $\rho(h)\text{:}$ $\phi=0.8$ solid, $\phi=-0.8$ dashed.

Check

$X_t=3+0.6X_{t-1}+\varepsilon_t$ with innovation variance 2: find $\mu\text{,}$ $v$ and $\rho(2)\text{.}$

Answer

$\mu=3/0.4=7.5\text{;}$ $v=2/0.64=3.125\text{;}$ $\rho(2)=0.36\text{.}$

Derivation: centre with $Y_t=X_t-\mu\text{;}$ then $\gamma(h)=\phi\operatorname{Cov}(Y_{t-1},Y_{t-h})+\operatorname{Cov}(\varepsilon_t,Y_{t-h})\text{,}$ and the last term is zero because $Y_{t-h}$ depends only on earlier shocks. With $c=2\text{,}$ $\phi=0.5\text{,}$ $\sigma^2=1\text{:}$ $\mu=4$ and $v=4/3\text{.}$

Persistence inflates the shock variance: as $|\phi|\to1$ the stationary variance grows, and at $\phi=1$ the division fails, so use the random walk. The ±0.8 curves share the same envelope.

The decks argue that $|\theta_1|<1$ because a variance cannot be negative; the notes assume $|\phi|<1$ for the stable solution and then derive its moments. A simulation started at a fixed value is a transient that only approaches these moments. A declining sample ACF does not prove an AR(1).

4.5Forecasting

Forecast the centre, then state the uncertainty

Key formula Result (9)

AR(1) forecast and error variance

\begin{aligned}\widehat X_{t+h\mid t}&=\mu+\phi^h(X_t-\mu)\\ V_h&=\sigma^2\,\frac{1-\phi^{2h}}{1-\phi^2}\end{aligned}

In words the mean plus $\phi^h$ times today’s deviation. The error is the sum of the $h$ unseen shocks, weighted $1,\phi,\ldots,\phi^{h-1}\text{.}$

Needs information through $t$ and known parameters; normal shocks for the interval $\pm1.96\sqrt{V_h}\text{.}$

Result as the horizon grows

The forecast tends to $\mu$ and $V_h$ to the stationary variance: the band levels off. For a random walk the forecast stays at $X_t$ and $h\sigma^2$ grows without limit.

Worked example a worked forecast

$\mu=10\text{,}$ $\phi=0.8\text{,}$ $\sigma^2=1\text{,}$ and today $X_t=15\text{.}$

\begin{aligned}\widehat X_{t+1\mid t}&=10+0.8(5)=14\\ \widehat X_{t+2\mid t}&=10+0.8^2(5)=13.2\\ V_2&=1+0.8^2=1.64\end{aligned}

95% interval: $13.2\pm1.96\sqrt{1.64}=[10.69,\ 15.71]\text{.}$

Why economists care

A forecast is a distribution: report its centre and its band. Estimating $\mu\text{,}$ $\phi$ and $\sigma^2$ from data adds further uncertainty.

Final frame: a time plot with the mean μ = 10 dashed. An illustrative past path rises to X_t = 15. From there the forecast centre falls towards 10 (14, 13.2, 12.56, …), inside a shaded 95% band that widens and then levels off between dashed limits 10 ± 3.27; two steps ahead, a yellow bracket marks [10.69, 15.71]. Three simulated green paths continue from 15, and one first rises to about 15.6. Beside it: μ = 10, φ = 0.8, σ² = 1, X_t = 15; X̂_{t+h|t} = 10 + 0.8^h(15 − 10) = 14, 13.2, 12.56, … → 10; V₂ = 1 + 0.8² = 1.64; 13.2 ± 1.96√1.64 = [10.69, 15.71]; V_h → 1/(1 − 0.8²) ≈ 2.78, so the band levels off. The expected deviation shrinks; a realised value can still rise.

Part 3 of 3

Reversion and spurious correlation

When does movement towards the mean, or a strong correlation between two series, reveal a real mechanism?

Key terms

mean reversionregression to the meanspurious regressioncointegration

Notes 4.6–4.7 ↗

So far a stable AR(1) forecast returns towards $\mu$ ; a random walk’s stays at today’s level while its variance grows.

Why it matters

Programmes that select extreme cases, such as tutoring the lowest scorers or treating road sites after unusually many accidents, can show before–after improvements with no effect. Regressing one trending level on another can give a high fit and a small p-value with no relationship at all.

In this part

4.6Mean reversion is a property of the expected path
▶Animation: why extreme first scores drift back on retest
4.6Regression to the mean, and what it means for programmes
4.7Independent random walks can correlate strongly
4.7Before trusting a correlation between two series

4.6Mean reversion

Mean reversion describes the expected path, not the next value

Definition

Mean reversion

In a stable AR(1) the expected deviation after $h$ periods, $\phi^h(X_t-\mu)\text{,}$ tends to zero. Next period’s expected change is

E[X_{t+1}-X_t\mid\mathcal F_t]=(\phi-1)(X_t-\mu)

which is negative when today is above the mean.

Worked example half-life

The expected deviation halves when $\phi^h=1/2\text{,}$ so $h_{1/2}=\log(1/2)/\log\phi\text{.}$ At $\phi=0.8$ that is about 3.11 periods.

Check

$\mu=0\text{,}$ $\phi=0.8\text{,}$ and today’s value is 5. Must the next value be below 5?

Answer

No. The forecast is 4, but $X_{t+1}=4+\varepsilon_{t+1}$ exceeds 5 whenever the shock exceeds 1. The expected change is $-1\text{;}$ the realised change is not restricted.

Common mistake

“It is far above its historical average, so it will fall.” That needs an identified restoring model. A random walk has a constant unconditional mean but no restoring forecast.

Final frame: a scatter of 300 simulated people, first score S₁ against retest score S₂, with a dashed no-change line S₂ = S₁ and a green conditional-mean line of slope 0.6. A strip of first scores near 30 is highlighted; a sideways yellow curve shows their retest distribution N(38, 8²), with an arrow up from 30 to 38 marked +8. A strip near 80 shows the curve N(68, 8²), with an arrow down from 80 to 68 marked −12. Beside it: S₁ = 50 + A + U₁ and S₂ = 50 + A + U₂, Var(A) = 60 persists and Var(U_i) = 40 is fresh; E[S₂ | S₁ = 30] = 50 + 0.6(30 − 50) = 38, +8 with no programme at all; E[S₂ | S₁ = 80] = 50 + 0.6(80 − 50) = 68; r = 60/(60 + 40) = 0.6, slope 0.6, not 1. Extreme first scores carry unusual noise; the retest draws fresh noise. A before–after gain is not a programme effect.

4.6Regression to the mean

Selecting extreme first scores also selects favourable noise

Key formula Result (10)

Regression to the mean

\begin{aligned}E[S_2\mid S_1=s]&=\mu+r\,(s-\mu)\\ r&=\frac{\tau^2}{\tau^2+\omega^2}\end{aligned}

In words a score is $\mu$ plus a lasting part (variance $\tau^2$ ) plus fresh noise (variance $\omega^2$ ); the retest keeps the share $r$ of the first deviation.

Needs the Gaussian model $S_k=\mu+A+U_k\text{,}$ all parts independent.

Result two extremes

Pure guessing: the top scorers’ expected retest is the mean. No new noise: no regression at all.

Why economists care

A school tutors its lowest scorers. With $\mu=50$ and $r=0.6$ , a first score of 30 retests at 38 on average, with no effect at all. A before–after gain mixes an effect, regression to the mean, a common trend and noise. Ask what the same selection gives without treatment (Lecture 5).

The same form as the AR(1) forecast, a different mechanism: a lasting component measured twice, and selection on the first measurement.

4.7Back to the opening question

Back to 0.71: independent random walks can correlate strongly

Worked example two independent walks

$X_t=X_{t-1}+\varepsilon_t$ and $Y_t=Y_{t-1}+\eta_t\text{,}$ both from 0, with independent $N(0,1)$ shocks. An early positive shock lifts $X$ for many later dates; an unrelated early shock may do the same for $Y\text{.}$ Across one realised sample, their levels then move together.

Neighbouring dates share shocks, so $T$ date pairs are not $T$ independent pieces of information.

Result the opening question

(c). The generator did not link the series. Nor does a longer sample help: levels correlations stay widely spread across independent pairs.

The notes’ lab: 200 prespecified pairs of independent walks, $T=100\text{.}$

Levels $(X_t,Y_t)$	$\|r\|>0.5$ in 85 of 200 pairs
Changes $(\Delta X_t,\Delta Y_t)$	$\|r\|>0.5$ in 0 of 200: independent iid pairs, correlation zero

4.7Carry the question through

A high coefficient does not choose its own explanation

Definition

Spurious regression

Misleading evidence of a relationship when unrelated persistent, nonstationary levels are regressed on one another, using inference designed for a different error structure.

Result four sources of a high coefficient

A common driver of both series.
Two deterministic trends, such as $X_t=t+\varepsilon_t$ and $Y_t=2t+\eta_t$ .
A search across many pairs that selects an extreme one.
Nonstationary levels, which align even when the processes are independent.

Common mistake

“The p-value is tiny, so the relation is real.” iid correlation tests and regression standard errors are wrong for random-walk levels; a small p-value from the wrong reference distribution is not reassuring.

Transform because the model says so. Differencing changes the question and can remove a real long-run relation: some nonstationary series have a stationary combination, called cointegration.

Review · How the ideas connect

Describe, model the memory, then interpret through the model

Part 1 · Describe

Part 2 · Model

Part 3 · Interpret

measure changecompare with its past

Change:

\Delta X_t\text{,}

growth (2), logs (3)

Ordered data: one path of

\{X_t\}

Autocorrelation

\rho(h)

(1)

h

alone if stationaryshocks feed it

\phi=1

|\phi|<1

is stationarycondition on

X_t

forecast

X_t\text{,}

band grows

White-noise shocks

\varepsilon_t

AR(1) (6)

X_t=c+\phi X_{t-1}+\varepsilon_t

Weak stationarity (4)

Random walk (5): variance

t\sigma^2

Forecast and band (9)

Stable:

\mu\text{,}

v

(7);

\phi^h

(8)

centre returns to

\mu

same form, other mechanismindependent walks align

Spurious correlation of levels

Mean reversion, in expectation

Regression to the mean (10)

Review · Summary sheet

What to take away: the terms and the ten formulas

Key terms

1 Order, units and change

Time series and process ordered data; the process $\{X_t\}$ is their probability model, and we observe one path.

Differences $\Delta X_t=X_t-X_{t-1}\text{;}$ $\Delta_4X_t$ compares matching quarters.

growth factor $1+g_t$ multiplies the level; rates compound, they do not add.

2 Models with memory

White noise the new shock $\varepsilon_t\text{:}$ iid, mean 0, variance $\sigma^2\text{.}$

random walk $X_t=X_{t-1}+\varepsilon_t\text{:}$ every shock is permanent and the variance $t\sigma^2$ grows.

forecast the expected $X_{t+h}$ given information at $t\text{;}$ $V_h$ is its error variance.

3 Reversion and spurious correlation

Mean reversion the expected deviation $\phi^h(X_t-\mu)$ shrinks to zero when $|\phi|<1\text{.}$

regression to the mean selecting extreme noisy scores selects favourable noise too, so retests move towards the mean.

Spurious regression misleading evidence of a link between persistent levels, from iid-style inference.

Key formulas

1 Order, units and change

(1)Autocorrelation at lag

h

Definition

\rho(h)=\operatorname{Corr}(X_t,X_{t-h})

finite variances; a function of

h

alone if stationary

(2)Growth factors compound Definition

(1+\bar g)^h=X_t/X_{t-h}

positive levels

(3)Log change Identity

r_t=\log(1+g_t)\text{,}

g_t=e^{r_t}-1

X_t>0\text{;}

natural logs

2 Models with memory

(4)Weak stationarity Definition

E[X_t]=\mu\text{,}

\operatorname{Cov}(X_t,X_{t-h})=\gamma(h)

finite variance

\gamma(0)

(5)The random walk accumulates shocks Result

X_t=x_0+\sum_{j\le t}\varepsilon_j\text{,}

\operatorname{Var}(X_t)=t\sigma^2

fixed start; iid mean-zero shocks

(6)The AR(1) model Definition

X_t=c+\phi X_{t-1}+\varepsilon_t

iid shocks; stable if

|\phi|<1

(7)Stationary mean and variance Result

\mu=c/(1-\phi)\text{,}

v=\sigma^2/(1-\phi^2)

|\phi|<1\text{;}

the stationary solution

(8)Autocorrelation of an AR(1) Result

\rho(h)=\phi^h\qquad(h\ge0)

the stationary AR(1)

(9)AR(1) forecast and error variance Result

\widehat X_{t+h\mid t}=\mu+\phi^h(X_t-\mu)\text{,}

V_h=\sigma^2\frac{1-\phi^{2h}}{1-\phi^2}

known parameters; normal shocks for

\pm1.96\sqrt{V_h}

3 Reversion and spurious correlation

(10)Regression to the mean Result

E[S_2\mid S_1=s]=\mu+r(s-\mu)

Gaussian model;

r=\tau^2/(\tau^2+\omega^2)

Review · Remember

Keep the order, model the memory, question the pattern

Result the opening question

(c). The series were independent random walks. Their levels share shocks across dates, so a correlation of 0.71 is unremarkable; their changes are independent pairs.

Remember

Keep the dates, units and information set: differences subtract, growth factors compound, logs add.
In $X_t=c+\phi X_{t-1}+\varepsilon_t\text{,}$ $|\phi|<1$ makes shocks fade and forecasts return to $\mu$ inside a band that levels off; $\phi=1$ makes them last.
Reversion and strong level correlations can arise with no mechanism: ask what the same model or selection gives without the claimed link.

Before next time

Notes 4.1–4.7 with the worked solutions.
The four labs: scales, the AR(1), retest scores and independent walks.
Rework the forecast with $\mu=20\text{,}$ $\phi=-0.5\text{,}$ $\sigma^2=4\text{,}$ $X_t=24\text{.}$

Next · Lecture 5

Causal inference. A before–after gain after selection, or a correlation between two series, is not an effect. Which comparison identifies one?