Econometric Analysis Of Cross Section And Panel Data

Getting Panel Data Models to Actually Work

Most people learn econometrics the wrong way. They start with the textbook formulas, memorize the fixed effects estimator, and then spend a week trying to make their data fit before they even run the first regression. I stopped doing that years ago. Now I look at the dataset first. The structure tells you what you're dealing with before any software can help. Cross section and panel data are often taught together because they share the same foundation, but they behave differently once your sample gets messy. Cross section gives you a snapshot. Panel data tracks the same units over time. The temptation is to treat them as interchangeable. That's where things fall apart.

The Econometric Analysis Of Cross Section And Panel Data In Practice

The core distinction comes down to how you handle unobserved heterogeneity. In cross section, omitted variable bias is your main enemy. You try to control for it with observables or instrumental variables. With panel data, you have within-unit variation over time, which means you can difference out time-invariant unobservables. That's the entire theoretical advantage. The practical reality is more complicated. I learned this the hard way on a project analyzing firm-level productivity with a panel spanning 2008 to 2021. Roughly twelve thousand firms, quarterly observations. Standard two-way fixed effects model. The results looked fine on paper. Then I checked the dynamic panel bias. My dependent variable had a lagged dependent variable, and with roughly twelve thousand units but only fourteen time periods, the within estimator was biased. Nickell bias, if you want the technical name. The coefficient on the lagged term was pulling everything else with it. The fix wasn't dramatic. I switched to Arellano-Bond GMM with Windmeijer-corrected standard errors. It took about ten minutes to code once you know the syntax. The estimates shifted noticeably from the fixed effects results, and the standard errors widened. Not a collapse, just a correction. Something most tutorials don't mention is that GMM works better here precisely because your T is small relative to N. That's the regime where it excels.

Here's what most people miss about panel data. You need to think about the time dimension first. If you have fewer than five time periods, the panel structure offers almost no advantage over pooled cross section. You're essentially running a fixed effects regression on noise. I've seen analysts push through with T=3 and still publish results. The numbers look clean until you check the within-transformed variance. It's barely different from zero in some cases. Another thing nobody warns you about is the assignment problem in difference-in-differences with panel data. When treatment timing varies across units, the standard two-way fixed effects estimator becomes negative-weighted. You can get a negative treatment effect even when every single unit responds positively. This isn't a rare edge case anymore. It's the default result once you have staggered adoption. The workaround is straightforward: use the Callaway and Sant'Anna estimator or the Sun and Abraham method. Both are implemented in standard statistical software. The output is more reliable than the classic event study design with year and group dummies.

Setting Up Your Analysis Properly

Start by reshaping your data into long format if it isn't already. Panel data software expects one row per unit per time period. Missing values in the time dimension create unbalanced panels, which is fine as long as you're not assuming balancedness in your estimator choice. Most modern estimators handle unbalanced panels without issue. Some don't. Check the documentation. For cross section work, the main concern is clustering. Standard errors that ignore within-group correlation will be wrong. If your data has a natural grouping structure like industry, region, or firm, cluster at that level. With twenty-five clusters or more, the correction is usually stable. Below that, you're in thin-cluster territory and need adjustments like the Cameron-Miller correction or wild bootstrap. I use the latter routinely now. It's computationally heavier but the coverage is noticeably better when your cluster count is low. The Hausman test between fixed and random effects is still widely used, but it's largely obsolete as a diagnostic. The test assumes that the unobserved effects are uncorrelated with your regressors under the null, which is exactly the assumption you're trying to test. It creates a circular logic problem. Instead, look at the within and between R-squared values. If they're close, random effects might be efficient. If they diverge, you likely have correlation between effects and regressors. The between estimator can help here too, though it has its own assumptions.

When you move to panel data, always test for cross-sectional dependence. The Pesaran CD test takes roughly three seconds to run. If your data shows cross-sectional dependence, which most macro and micro datasets do, standard clustered standard errors aren't enough. You need Driscoll-Kraay standard errors or the PMG estimator. These account for contemporaneous correlations across units that your clustering variable might not capture. I run the CD test before every panel regression now. It's become a default first step alongside checking for stationarity.

Common Mistakes That Waste Hours

The biggest time sink I see is people applying OLS to panel data without accounting for the hierarchical structure. They run a pooled regression, get confident results, and then discover later that their standard errors are wrong by an order of magnitude. The fix is simple once you know it's a problem. Group your data and use the appropriate estimator. But catching it late means redoing the entire analysis. Another mistake is treating time trends as deterministic when they should be stochastic. A linear time trend in your model assumes the trend doesn't change over the sample period. If your data spans a crisis or a structural shift, that assumption breaks. I include time-varying trends in my preferred specifications. It adds a few degrees of freedom but captures something real. When working with cross section data, people frequently ignore sample selection bias. If your data only includes certain firms or individuals who self-selected into the sample, your estimates are biased. The Heckman two-step correction is the standard approach, though it relies heavily on the exclusion restriction. A single instrument that affects selection but not the outcome equation is hard to find and even harder to defend. I've found that sensitivity analysis is more honest than forcing an identification assumption that doesn't hold.

Panel data has its own version of this problem with attrition. People drop out of longitudinal studies for reasons related to the outcome. That's not missing at random. That's missing not at random, which most standard software doesn't handle well. The best you can do is model the attrition process explicitly and adjust your weights accordingly. It adds complexity but the alternative is biased estimates that look precise.

Practical Workflow That Actually Saves Time

Here's what I do before running any substantive estimation. First, reshape and validate. Check for duplicate identifiers, impossible dates, and extreme outliers. I filter out observations more than five standard deviations from the group mean within each unit. This removes data entry errors without touching the real distribution. Second, generate summary statistics by group and time period. Third, run the diagnostic tests. Cross-sectional dependence, autocorrelation, unit roots if relevant. This usually takes twenty to thirty minutes depending on dataset size. Fourth, choose the estimator based on what the diagnostics tell you, not what the textbook says you should use. For implementation, Stata handles most panel estimations efficiently. The commands are consistent and well-documented. R with plm and AER packages works fine for cross section and basic panel work. Python's linearmodels package covers the essentials but the ecosystem is less mature. I recommend Stata if you're doing this regularly. The time savings from cleaner syntax and better diagnostics outweigh the licensing cost for most researchers. One final note on interpretation. Panel data gives you causal leverage that cross section cannot match, but only when your identifying assumptions hold. Fixed effects remove time-invariant omitted variables. That's a real advantage. But it doesn't remove time-varying confounders. If something changes within your units over time and correlates with both your treatment and outcome, your estimates are still biased. No amount of clustering or robust standard errors fixes that. You need a credible identification strategy, not just the right software command.

The difference between a useful panel analysis and a misleading one usually comes down to whether you checked your assumptions before looking at the coefficients. Most papers skip that part. I don't anymore. The extra time upfront prevents rework later.