gmm: A library for Z-Estimation

Functions to solve for parameters $\theta$ defined as solutions to moment conditions / estimating equations $$E[g(Z; \theta)] = 0$$

for some function $g(Z; \theta)$ of the data $Z$, which is typically a matrix of residuals (hence has expectation 0 in each column). Common examples include $g(\theta) = X(y - X\theta)$ for OLS and $g(\theta) = z(y - X\theta)$ for IV. This is a remarkably flexible approach to performing estimation and inference without making strong distributional assumptions about the data, and underpins many of the most widely-used estimators in econometrics.

Look in the notebooks/ directory for examples of how to use the library.

installation

pip install git+https://github.com/py-econometrics/gmm --extra-index-url https://download.pytorch.org/whl/cpu

where the --extra-index-url sources the torch dependency. Swap it for the GPU version if you have a CUDA-compatible GPU.

gmm.GMMEstimator

solves for a k-dimensional parameter $\theta$ are defined by solving the following optimization problem

$$ \hat{\theta} = \text{argmin}_{\theta} \left(n^{-1} \sum_i g(Z, \theta)' \right) \mathbf{W} \left(n^{-1} \sum_i g(Z, \theta) \right) $$

for a moment condition $g(\cdot)$ and a $m \times m$ weight matrix $\mathbf{W}$. For a just-identified problem (M = K), the choice of the weight matrix $\mathbf{W}$ does not matter. For over-identified problems (M > K), it does. Hansen(1982) covers the details and won him the Nobel prize.

Supports both scipy.optimize.minimize and pytorch.minimize to solve the GMM for just- and over-identified problems (with Identity or Optimal weight matrix) and computes HAC-robust standard errors. See OLS and IV examples in example.ipynb, and several maximum likelihood examples in maximum_likelihood.ipynb.

The scipy optimizer uses an analytic expression for the jacobian of linear moment conditions, while the pytorch.minimize version uses forward-mode autodiff and therefore supports both linear and non-linear moment conditions.

gmm.GELEstimator

Solves the same category of problem using generalized empirical likelihood (Exponential tilting by default, but also supports EL and CUE) by solving the following optimization problem

$$ \min_{\pi, \theta} I_{\lambda}(\iota / N, \pi) ; \text{subject to} ; \sum_{i} \pi_i g(Z; \theta) = 0 ; \text{ and } ; \sum_i \pi_i = 1 $$

where $I_\lambda(\iota/N, \pi)$ is a Cressie-Read power-divergence family discrepancy statistic. Intuitively, this approach solves for a weight vector $\pi$ that is minimally different from uniform weights $\iota/N$ while satisfying the moment condition $g(\cdot) = 0$. Different choices of $\lambda$ correspond to different divergence measures, with $\lambda = 0$ corresponding to the empirical likelihood and $\lambda = 1$ corresponding to the chi-squared distance, and $\lambda = -1$ coressponding to the Kullback-Liebler distance. This problem appears daunting since we have (seemingly unnecessarily) added a n-vector of weights $pi$ to the problem, but it turns out that this is a very powerful approach to solving for the parameter $\theta$ in a variety of settings. We use the saddle-point representation (Newey and Smith 2004) that concentrates out the probability weights $\pi$

$$ \min_{\theta \in \Theta} \sup_{\lambda \in \Lambda_n} \sum_i \rho (\lambda ' g(Z_i, \theta)) $$

where $\rho$ is a smooth scalar function that satisfies $\rho(0) = 0, \partial\rho(0)/\partial v = \partial^2 \rho(0) / \partial v^2 = -1$ (which have one-to-one mappings with the Cressie-Read family). We support the following $\rho$s:

• Empirical Likelihood: $\rho(v) = log(1 - v)$
• Exponential tilting: $\rho(v) = 1 - \exp(v)$
• Continuously Updated (CU) GMM: $\rho(v) = -(1/2)v^2 - v$

Todo / Planned feature updates

• Support numerical optimization via pytorch-minimize
• Support Empirical Likelihood and Generalized Empirical Likelihood
• Support bootstrapping influence functions ('fast bootstrap') for inference
• Support for autodiff-based gradients in GEL
• Refactor GMM estimators to accept a single data argument instead of separate X, y, z, etc.

References

• Microeconometrics [Chapter 6], Cameron and Trivedi

• Imbens, Generalized Method of Moments and Empirical Likelihood

• Guido Imbens' lectures

  • slides
  • notes

• Anatolyev and Gospodinov, Methods for Estimation and Inference in Modern Econometrics

• Owen, Empirical Likelihood

• Newey and Smith, Higher Order Properties of GMM and Generalized Empirical Likelihood Estimators

• here be gremlins: Newey and McFadden (1994)

• here be dragons :Van Der Vaart, Asymptotic Statistics