Econometrics I Professor William Greene Stern School of Business Department of Economics 4-1/22 Part 4: Partial Regression and Correlation Econometrics I Part 4 – Partial Regression and Correlation 4-2/22 Part 4: Partial Regression and Correlation Frisch-Waugh (1933) Theorem Context: Model contains two sets of variables: X = [ (1,time) : (other variables)] = [X1 X2] Regression model: y = X11 + X22 + (population) = X1b1 + X2b2 + e (sample) Problem: Algebraic expression for the second set of least squares coefficients, b2 4-3/22 Part 4: Partial Regression and Correlation Partitioned Solution Direct manipulation of normal equations produces ( X X )b = X y X1 X1 X 1 X 2 X1 y X = [X 1 , X 2 ] so X X = and X y = X y X X X X 2 2 2 1 2 X X X 1 X 2 b1 X 1 y (X X)b = 1 1 = X 2 X1 X 2 X 2 b2 X 2 y X1 X1b1 X1 X 2b2 X1 y X 2 X1b1 X 2 X 2b2 X 2 y ==> X 2 X 2b2 X 2 y - X 2 X1b1 = X 2 (y - X 1b1 ) 4-4/22 Part 4: Partial Regression and Correlation Partitioned Solution Direct manipulation of normal equations produces b2 = (X2X2)-1X2(y - X1b1) What is this? Regression of (y - X1b1) on X2 If we knew b1, this is the solution for b2. Important result (perhaps not fundamental). Note the result if X2X1 = 0. 4-5/22 Part 4: Partial Regression and Correlation Partitioned Inverse Use of the partitioned inverse result produces a fundamental result: What is the southeast element in the inverse of the moment matrix? -1 X1'X1 X 'X 2 1 4-6/22 X1'X2 X2'X2 Part 4: Partial Regression and Correlation Partitioned Inverse The algebraic result is: [ ]-1(2,2) = {[X2’X2] - X2’X1(X1’X1)-1X1’X2}-1 = [X2’(I - X1(X1’X1)-1X1’)X2]-1 = [X2’M1X2]-1 Note the appearance of an “M” matrix. How do we interpret this result? 4-7/22 Part 4: Partial Regression and Correlation Frisch-Waugh Result Continuing the algebraic manipulation: b2 = [X2’M1X2]-1[X2’M1y]. This is Frisch and Waugh’s famous result - the “double residual regression.” How do we interpret this? A regression of residuals on residuals. Regrido Y em X1 – pego o resíduo que sobra e regrido esta parte no que sobra de X2 regredido em X1 (resíduo). Tudo de Y que não foi explicado por X1 mas pode ser explicado por X2. Tudo de X2 que não está sendo explicado por X1 4-8/22 Part 4: Partial Regression and Correlation 4-9/22 Part 4: Partial Regression and Correlation Important Implications M1 is idempotent. (This is a very useful result.) (i.e., X2M1’M1y = X2M1y) (Orthogonal regression) Suppose X1 and X2 are orthogonal; X1X2 = 0. What is M1X2? 4-10/22 Part 4: Partial Regression and Correlation Applying Frisch-Waugh Using gasoline data from Notes 3. X = [1, year, PG, Y], y = G as before. Full least squares regression of y on X. 4-11/22 Part 4: Partial Regression and Correlation Detrending the Variables - Pg 4-12/22 Part 4: Partial Regression and Correlation Regression of Detrended G on Detrended Pg and Detrended Y 4-13/22 Part 4: Partial Regression and Correlation Partial Regression Important terms in this context: Partialing out the effect of X1. Netting out the effect … “Partial regression coefficients.” To continue belaboring the point: Note the interpretation of partial regression as “net of the effect of …” Now, follow this through for the case in which X1 is just a constant term, column of ones. What are the residuals in a regression on a constant. What is M1? Note that this produces the result that we can do linear regression on data in mean deviation form. As inclinações numa regressão múltipla que contém o termo constante são obtidas por transformar os dados como desvios em relação à média e regredir a variável y na forma de desvio em todas variáveis explicativas também como desvios em relação às suas médias. 'Partial regression coefficients' are the same as 'multiple regression coefficients.' It follows from the Frisch-Waugh theorem. 4-14/22 Part 4: Partial Regression and Correlation Partial Correlation Working definition. Correlation between sets of residuals. Some results on computation: Based on the M matrices. Some important considerations: Partial correlations and coefficients can have signs and magnitudes that differ greatly from gross correlations and simple regression coefficients. Compare the simple (gross) correlation of G and PG with the partial correlation, net of the time effect. CALC;list;Cor(g,pg)$ Result = .7696572 4 .5 0 4 .0 0 3 .5 0 CALC;list;cor(gstar,pgstar)$ Result = -.6589938 PG 3 .0 0 2 .5 0 2 .0 0 1 .5 0 1 .0 0 .5 0 70 80 90 100 110 120 G 4-15/22 Part 4: Partial Regression and Correlation Partial Correlation 4-16/22 Part 4: Partial Regression and Correlation THE Application of Frisch-Waugh The Fixed Effects Model A regression model with a dummy variable for each individual in the sample, each observed Ti times. yi = Xi + diαi + εi, for each individual N columns y1 y2 yN X1 X 2 X N d1 0 0 d2 0 0 0 0 0 0 0 β ε α dN N may be thousands. I.e., the regression has thousands of variables (coefficients). β = [X, D] ε α = Zδ ε 4-17/22 Part 4: Partial Regression and Correlation Estimating the Fixed Effects Model The FEM is a linear regression model but with many independent variables b a Using b 4-18/22 1 X X X D X y DX DD Dy the Frisch-Waugh theorem 1 =[X MD X ] X MD y Part 4: Partial Regression and Correlation Fixed Effects Estimator (cont.) M1D 0 2 0 M D MD 0 0 0 0 (The dummy variables are orthogonal) N MD MDi I Ti di (didi ) 1 d = I Ti (1/Ti )did 1 - T1i - T1i 1 1 - Ti 1 - Ti = ... ... 1 - T1 Ti i 4-19/22 ... - T1i 1 ... - Ti ... ... ... 1 - T1i Part 4: Partial Regression and Correlation ‘Within’ Transformations XMD X = Ni=1 XiMDi X i , XMD y = Ni=1 XiMDi y i , XM y XiMDi X i i i D i k k,l Ti t=1 (x it,k -x i.,k )(x it,l -x i.,l ) Ti t=1 (x it,k -x i.,k )(y it -y i. ) Least Squares Dummy Variable Estimator b is obtained by ‘within’ groups least squares (group mean deviations) 4-20/22 Part 4: Partial Regression and Correlation