********************************************************************************
*  Empirical Exercise 7.2 - Earnings and Height: Joint Hypothesis Testing
*  Stock & Watson - Introduction to Econometrics (4th Edition)
*
*  Code: Fatih Kansoy - www.fatih.ai
*  Course: Financial Econometrics, SJTU Summer School 2025
********************************************************************************

clear all
set more off

* Set your working directory
* Set the working directory to the folder that holds the data before running,
* for example:  cd "path/to/data"

import delimited "Earnings_and_Height.csv", clear

********************************************************************************
* DATA OVERVIEW
********************************************************************************
* Chapter 7 Focus: Joint Hypothesis Testing (F-tests) and OVB with Education
*
* Case and Paxson (2008) hypothesis:
*   Height → correlated with cognitive ability (omitted) → affects Earnings
*   If true, controlling for education (proxy for cognitive ability) should
*   reduce the height coefficient
*
* Key Variables:
*   earnings: Annual earnings ($)
*   height: Height in inches
*   sex: 0 = Female, 1 = Male
*   educ: Years of education
*
* Education Dummy Variables (to be created):
*   LT_HS: Less than high school (educ < 12)
*   HS: High school diploma only (educ = 12)
*   Some_Col: Some college (12 < educ < 16)
*   College: Bachelor's degree or higher (educ >= 16) -- REFERENCE GROUP

describe earnings height sex educ
summarize earnings height sex educ

* Check education distribution
tab educ

********************************************************************************
* CREATE EDUCATION DUMMY VARIABLES
********************************************************************************
* These dummies partition the sample into 4 mutually exclusive education groups
* College will be the OMITTED reference category (to avoid dummy variable trap)

gen lt_hs = (educ < 12)
gen hs = (educ == 12)
gen some_col = (educ > 12 & educ < 16)
gen college = (educ >= 16)

label variable lt_hs "Less than High School (educ < 12)"
label variable hs "High School diploma (educ = 12)"
label variable some_col "Some College (12 < educ < 16)"
label variable college "College degree (educ >= 16)"

* Verify the dummies are mutually exclusive and exhaustive
gen educ_sum = lt_hs + hs + some_col + college
tab educ_sum

* Check distribution of education categories
tab lt_hs
tab hs
tab some_col
tab college

* Summary by gender
tab sex

********************************************************************************
* (a) OMITTED VARIABLE BIAS THEORY
********************************************************************************
* If cognitive ability affects both Height and Earnings:
*   - Corr(Height, CogAbility) > 0 (taller people tend to have higher ability)
*   - CogAbility → Earnings (higher ability leads to higher earnings)
*
* OVB Formula: Bias = β_CogAbility × [Cov(Height, CogAbility) / Var(Height)]
*
* Since both terms are positive → Bias > 0
* The simple regression OVERESTIMATES the true effect of height
*
* Education serves as a PROXY for cognitive ability:
* Higher ability → more schooling → education controls for ability

********************************************************************************
* (b) WOMEN ONLY ANALYSIS
********************************************************************************

*------------------------------------------------------------------------------
* (b) Regression (1): Simple regression - Earnings on Height (Women)
*------------------------------------------------------------------------------
regress earnings height if sex == 0

* Store height coefficient
scalar b_height_f1 = _b[height]
scalar se_height_f1 = _se[height]
scalar r2_f1 = e(r2)

*------------------------------------------------------------------------------
* (b) Regression (2): With Education Controls (Women)
*------------------------------------------------------------------------------
* Include LT_HS, HS, Some_Col; College is omitted (reference group)
* This avoids the dummy variable trap

regress earnings height lt_hs hs some_col if sex == 0

* Store results
scalar b_height_f2 = _b[height]
scalar se_height_f2 = _se[height]
scalar b_lt_hs_f = _b[lt_hs]
scalar b_hs_f = _b[hs]
scalar b_some_col_f = _b[some_col]
scalar r2_f2 = e(r2)

*------------------------------------------------------------------------------
* (b)(iii) Joint F-test: H₀: β_lt_hs = β_hs = β_some_col = 0
*------------------------------------------------------------------------------
* This tests whether education variables JOINTLY explain earnings
* Uses the F-statistic for testing multiple restrictions

test lt_hs hs some_col

* Store F-statistic and p-value
scalar F_stat_f = r(F)
scalar p_value_f = r(p)

********************************************************************************
* (c) MEN ONLY ANALYSIS
********************************************************************************

*------------------------------------------------------------------------------
* (c) Regression (1): Simple regression - Earnings on Height (Men)
*------------------------------------------------------------------------------
regress earnings height if sex == 1

* Store height coefficient
scalar b_height_m1 = _b[height]
scalar se_height_m1 = _se[height]
scalar r2_m1 = e(r2)

*------------------------------------------------------------------------------
* (c) Regression (2): With Education Controls (Men)
*------------------------------------------------------------------------------
regress earnings height lt_hs hs some_col if sex == 1

* Store results
scalar b_height_m2 = _b[height]
scalar se_height_m2 = _se[height]
scalar b_lt_hs_m = _b[lt_hs]
scalar b_hs_m = _b[hs]
scalar b_some_col_m = _b[some_col]
scalar r2_m2 = e(r2)

*------------------------------------------------------------------------------
* (c)(iii) Joint F-test: H₀: β_lt_hs = β_hs = β_some_col = 0
*------------------------------------------------------------------------------
test lt_hs hs some_col

* Store F-statistic and p-value
scalar F_stat_m = r(F)
scalar p_value_m = r(p)

********************************************************************************
* SUMMARY TABLES
********************************************************************************

di ""
di "=============================================================================="
di "E7.2 SUMMARY: HEIGHT COEFFICIENT WITH AND WITHOUT EDUCATION CONTROLS"
di "=============================================================================="
di ""
di "WOMEN (sex = 0, n = 9,974)"
di "-------------------------------------------------------------------------"
di "Regression          | β̂_height |   SE    |   R²   | Change from (1)"
di "--------------------|----------|---------|--------|------------------"
di "(1) Height only     | " %8.2f b_height_f1 " | " %7.2f se_height_f1 " | " %6.4f r2_f1 " |    --"
di "(2) + Education     | " %8.2f b_height_f2 " | " %7.2f se_height_f2 " | " %6.4f r2_f2 " | " %6.1f (b_height_f2 - b_height_f1) " (" %4.1f 100*(b_height_f2 - b_height_f1)/b_height_f1 "%)"
di "-------------------------------------------------------------------------"
di "Joint F-test (education): F = " %7.2f F_stat_f ", p-value = " %8.6f p_value_f
di ""
di "Education coefficients (relative to College):"
di "  LT_HS:    " %10.2f b_lt_hs_f
di "  HS:       " %10.2f b_hs_f
di "  Some_Col: " %10.2f b_some_col_f
di ""

di "MEN (sex = 1, n = 7,896)"
di "-------------------------------------------------------------------------"
di "Regression          | β̂_height |   SE    |   R²   | Change from (1)"
di "--------------------|----------|---------|--------|------------------"
di "(1) Height only     | " %8.2f b_height_m1 " | " %7.2f se_height_m1 " | " %6.4f r2_m1 " |    --"
di "(2) + Education     | " %8.2f b_height_m2 " | " %7.2f se_height_m2 " | " %6.4f r2_m2 " | " %6.1f (b_height_m2 - b_height_m1) " (" %4.1f 100*(b_height_m2 - b_height_m1)/b_height_m1 "%)"
di "-------------------------------------------------------------------------"
di "Joint F-test (education): F = " %7.2f F_stat_m ", p-value = " %8.6f p_value_m
di ""
di "Education coefficients (relative to College):"
di "  LT_HS:    " %10.2f b_lt_hs_m
di "  HS:       " %10.2f b_hs_m
di "  Some_Col: " %10.2f b_some_col_m
di "=============================================================================="

********************************************************************************
* KEY INTERPRETATIONS
********************************************************************************
*
* (b)(ii) Why is College omitted?
*   → To avoid the dummy variable trap (perfect multicollinearity)
*   → LT_HS + HS + Some_Col + College = 1 for all observations
*   → Including all four would make X'X singular
*
* (b)(iv) Why are education coefficients negative and ordered?
*   → Reference group is College (highest education)
*   → β_lt_hs < β_hs < β_some_col < 0 because:
*     - Less education → lower earnings relative to college graduates
*     - The ordering reflects the education-earnings gradient
*     - LT_HS has the most negative coefficient (largest earnings gap vs college)
*
* (b)(i) Does height coefficient change?
*   → If it decreases substantially, this is consistent with OVB theory
*   → Education "absorbs" some of what height was capturing (ability)

********************************************************************************
* END OF E7.2
********************************************************************************