Pesendorfer and Schmidt-Dengler (2008)

These notes are based on the following article:

Pesendorfer, Martin and Philipp Schmidt-Dengler (2008). Asymptotic Least Squares Estimators for Dynamic Games. Review of Economic Studies 75, 901–928.

Presentation by Jason Blevins, Duke University Applied Microeconomics Reading Group, June 11, 2008.

Outline

• Considers the class of asymptotic least squares estimators for dynamic games.
• Estimation is based on equilibrium conditions.
• Discuss identification and provide sufficient conditions for exact identification.
• Characterize the efficient asymptotic least squares estimator.
• Several well-known estimators are members of this class.
• Monte Carlo experiments.

Framework

• Dynamic games in discrete time with $t=1,\dots ,\mathrm{\infty }$.
• $N$ players, $K+1$ actions, $L$ states per player, common discount factor $\beta$.
• States:
  • $s_{i,t}\in S_i=\{1,\dots ,L\}$
  • ${\epsilon }_{i,t}\sim F(\epsilon \mid s_{i,t},s_{-i,t})$ on ${ℝ}^K$.
  • Let $S=S_1\times \dots \times S_N$.
• The payoff shocks ${\epsilon }_{i,t}$ are private information, independent across players and time, and independent of the actions of other players.
• Actions $a_{i,t}\in A_i=\{0,1,\dots ,K\}$ are made simultaneously. Let $A=A_1\times \dots \times A_N$.
• State transitions follow some density $g(a_t,s_t,s_{t+1})$. Let $G$ denote the $m_a{m}_s\times m_s$ matrix of these probabilities where $m_s=\#S=L^N$ and $m_a=\#A=(K+1{)}^N$.
• Period payoffs are given by $${\pi }_i(a_t,s_t)+\sum _{k=1}^K{\epsilon }_{i,t,k}1\{a_{i,t}=k\}$$

Equilibrium Characterization

The continuation value net of payoff shocks under $a_i$ with beliefs ${\sigma }_i$ is $$u_i(a_i;{\sigma }_i,\theta )=\sum _{a_{-i}}{\sigma }_i(a_{-i}\mid s)[{\pi }_i(a_{-i},a_i,s)+\beta \sum _{s\prime }g(a_{-i},a_i,s,s\prime )V_i(s\prime ;{\sigma }_i)].$$ It is optimal to choose $a_i$ under the beliefs ${\sigma }_i$ if $$u_i(a_i;{\sigma }_i,\theta )+{\epsilon }_{i,a_i}\ge u_i(a_i\prime ;{\sigma }_i,\theta )+{\epsilon }_{i,a_i\prime }\forall a_i\prime \in A_i.$$

Ex ante, in expectation we have $$p(a_i\mid s,{\sigma }_i)={\Psi }_i(a_i,s,{\sigma }_i;\theta )=\int 1\{u_i(a_i;{\sigma }_i,\theta )-u_i(k;{\sigma }_i,\theta )\ge {\epsilon }_{i,k}-{\epsilon }_{i,a_i},k\ne a_i\}\mathrm{dF}.$$ In matrix notation we have a $(N\cdot K\cdot m_s)\times 1$ system $$p=\Psi (\sigma ;\theta ).$$

Equilibrium Properties

In equilibrium, beliefs are consistent and we have the fixed point problem $$label{\mathrm{fixed}}_{\mathrm{point}}p=\Psi (p;\theta ).$$ Thus, finding an equilibrium is a fixed point problem on $[0,1{]}^{N\cdot K\cdot m_s}$.

Proposition: In any Markov perfect equilibrium, the probability vector $p$ satisfies \eqref{fixed_point}. Conversely, any $p$ that satisfies \eqref{fixed_point} can be extended to a Markov perfect equilibrium.

Theorem: A Markov perfect equilibrium exists.

We have the same results under symmetric equilibria: existence and necessary and sufficient conditions. Symmetry reduces the number of equations in \eqref{fixed_point} and thus the computational complexity.

Identification

The model is identified if there exists a unique set of model primitives $({\Pi }_i,\dots ,{\Pi }_N,F,\beta ,g)$ that generate any particular set of choice and state transition probabilities.

• Time series data $\{a_t,s_t{\}}_{t=1}^T$.
• Suppose the data allow us to characterize $p(a\mid s)$ and $g(a,s,s\prime )$.
• Fix $\beta$ and $F$.
• There are $m_a\cdot m_s\cdot N$ remaining unknowns in $({\Pi }_1,\dots ,{\Pi }_N)$.

Proposition: Suppose $F$ and $\beta$ are given. Then at most $K\cdot m_s\cdot N$ parameters can be identified.

There are only $K\cdot m_s\cdot N$ equations in the equilibrium conditions but $m_a\cdot m_s\cdot N$ parameters. We need at least $(m_a\cdot m_s-K\cdot m_s)\cdot N$ restrictions in order to identify all parameters.

Identification: A Linear Representation

There is some ${\overline{\epsilon }}_i^{a_i}(s)$ that makes player $i$ indifferent between actions $a_i$ and $0$: $$\begin{array}{rl}& \sum _{a_{-i}\in A_{-i}}p(a_{-i}\mid s)[{\pi }_i(a_{-i},a_i,s)+\beta \sum _{s\prime \in S}g(a_{-i},a_i,s,s\prime )V_i(s\prime ;p)]+{\overline{\epsilon }}_i^{a_i}(s)\\ & =\sum _{a_{-i}\in A_{-i}}p(a_{-i}\mid s)[{\pi }_i(a_{-i},0,s)+\beta \sum _{s\prime \in S}g(a_{-i},0,s,s\prime )V_i(s\prime ;p)]\end{array}$$

From before, $V_i({\sigma }_i)=[I-\beta {\sigma }_{i}G{]}^{-1}[{\sigma }_i{\Pi }_i+D_i({\sigma }_i)]$. Thus, we have a linear system of equations for player $i$: $$X_i(p,g,\beta ){\Pi }_i+Y_i(p,g,\beta )=0$$ where $X_i$ is a $(K\cdot m_s)\times (m_a\cdot m_s)$ matrix and $Y_i$ is a $(K\cdot m_s)\times 1$ vector, both of which depend on the choice probabilities, transition probabilities, and $\beta$.

Identification: Linear Restrictions

Consider player $i$. Let $R_i$ be a $(m_a\cdot m_s-K\cdot m_s)\times (m_a\cdot m_s)$ matrix of restrictions and let $r_i$ be a $(m_a\cdot m_s-K\cdot m_s)\times 1$-dimensional vector such that $R_i{\Pi }_i=r_i$.

We can now form an augmented linear system of $m_a\cdot m_s$ equations in $m_a\cdot m_s$ unknowns (hence the order condition is satisfied): $$\left[\begin{array}{c}{X}_i\\ R_i\end{array}\right]{\Pi }_i+\left[\begin{array}{c}{Y}_i\\ r_i\end{array}\right]={\overline{X}}_i{\Pi }_i+{\overline{Y}}_i=0.$$

Proposition: Consider any player $i$ and suppose that $F$ and $\beta$ are given. If $rank({\overline{X}}_i)=m_a\cdot m_s$, then ${\Pi }_i$ is exactly identified.

Example: Consider the following restrictions: $$\begin{array}{rlrl}{\pi }_i(a_i,a_{-i},s_i,s_{-i})& ={\pi }_i(a_i,a_{-i},s_i,s_{-i}\prime )& & \forall a\in A,(s_i,s_{-i})\in S,(s_i,s_{-i}\prime )\in S\\ {\pi }_i(0,a_{-i},s_i)& =r_i(a_{-i},s_i)& & \forall a_{-i}\in A_{-i},s_i\in S_i\end{array}$$ The first is an exclusion restriction while the second is an exogeneity restriction (e.g., payoffs for inactive firms are known to be zero). If $L\ge K+1$, then these restrictions ensure identification (provided that the rank condition holds).

Asymptotic Least Squares Estimators

Let $\theta =({\theta }_{\pi },{\theta }_F,\beta ,{\theta }_g)\in \Theta \subset {ℝ}^q$ be the parameters of interest.

There are also $H\le (N\cdot K\cdot m_s)+(m_a\cdot m_s\cdot m_s)$ auxiliary parameters $p(\theta )$ and $g(\theta )$, related to $\theta$ through the $N\cdot K\cdot m_s$ equations $$label{\mathrm{estimating}}_{\mathrm{equations}}h(p,g,\theta )=p-\Psi (p,g,\theta )=0.$$

Asymptotic least squares estimators (Gourieroux and Monfort, 1995, Section 9.1) proceed in two steps:

1. Estimate the auxiliary parameters $p$ and $g$.
2. Estimate the parameters of interest using weighted least squares using \eqref{estimating_equations} as estimating equations.

Asymptotic Least Squares Estimators

Assume that consistent and asymptotically normal estimators of $p$ and $g$ are available such that as $T\to \mathrm{\infty }$, $$\begin{array}{c}({\hat{p}}_T,{\hat{g}}_T)⟶(p({\theta }_0),g({\theta }_0))a.s.,\\ \sqrt{T}[({\hat{p}}_T,{\hat{g}}_T)-(p({\theta }_0),g({\theta }_0))]\stackrel{d}{⟶}Normal(0,\Sigma ({\theta }_0)).\end{array}$$

The estimation principle involves choosing $\theta$ in order to satisfy the constraints $$h({\hat{p}}_T,{\hat{g}}_T,\theta )={\hat{p}}_T-\Psi ({\hat{p}}_T,{\hat{g}}_T,\theta )=0.$$

Let $W_T$ be a symmetric positive-definite weight matrix of dimension $(N\cdot K\cdot m_s)\times (N\times K\times m_s)$. The asymptotic least squares estimator corresponding to $W_T$ is defined as $${\tilde{\theta }}_T(W_T)=\arg \underset{\theta }{\min }[{\hat{p}}_T-\Psi ({\hat{p}}_T,{\hat{g}}_T,\theta ){]}^{\top }{W}_T[{\hat{p}}_T-\Psi ({\hat{p}}_T,{\hat{g}}_T,\theta )].$$

Asymptotic Least Squares Estimators: Assumptions

1. $\Theta$ is a compact set.
2. ${\theta }_0$ lies in the interior of $\Theta$.
3. As $T\to \mathrm{\infty }$, $W_T\to W_0$ a.s. where $W_0$ is a non-stochastic positive definite matrix.
4. $\theta$ satisfies $${[p({\theta }_0)-\Psi (p({\theta }_0),g({\theta }_0),\theta )]}^{\top }{W}_o[p({\theta }_0)-\Psi (p({\theta }_0),g({\theta }_0),\theta )]=0$$ implies that $\theta ={\theta }_0$.
5. The functions $\pi$, $g$, and $F$ are twice continuously differentiable in $\theta$.
6. The matrix ${[{\nabla }_{\theta }\Psi (p({\theta }_0),g({\theta }_0),{\theta }_0)]}^{\top }{W}_o[{\nabla }_{\theta }\Psi (p({\theta }_0),g({\theta }_0),{\theta }_0)]$ is nonsingular.

Asymptotic Least Squares Estimators: Properties

Proposition: Given the assumptions above the asymptotic least squares estimator ${\tilde{\theta }}_T(W_T)$ exists, ${\tilde{\theta }}_T(W_T)\stackrel{a.s.}{\to }{\theta }_0$, and as $T\to 0$, $$\sqrt{T}({\tilde{\theta }}_T(W_T)-{\theta }_0)\stackrel{d}{\to }Normal(0,\Omega ({\theta }_0))$$ where $$\Omega ({\theta }_0)={\left({\nabla }_{\theta }{\Psi }^{\top }{W}_0{\nabla }_{{\theta }^{\top }}\right)}^{-1}{\nabla }_{\theta }{\Psi }^{\top }{W}_0[\left(\begin{array}{cc}I& 0\end{array}\right)-{\nabla }_{(p,g{)}^{\top }}\Psi ]\Sigma \cdot {[\left(\begin{array}{cc}I& 0\end{array}\right)-{\nabla }_{(p,g{)}^{\top }}\Psi ]}^{\top }{W}_0{\nabla }_{{\theta }^{\top }}\Psi {\left({\nabla }_{\theta }{\Psi }^{\top }{W}_0{\nabla }_{{\theta }^{\top }}\right)}^{-1}$$ where $0$ is the $(N\cdot K\cdot m_s)\times (m_a\cdot m_s\cdot m_s)$ zero matrix and the various matrices are evaluated at ${\theta }_0$, $p({\theta }_0)$, and $g({\theta }_0)$.

Efficient Asymptotic Least Squares

Proposition: Under the maintained assumptions, the best asymptotic least squares estimators exist. They correspond to sequences of matrices $W_T^{\ast }$ converging to $$W_0^{\ast }={([(\begin{array}{cc}I& 0\end{array})-{\nabla }_{(p,g)\prime }\Psi ]\Sigma [(\begin{array}{cc}I& 0\end{array})-{\nabla }_{(p,g)\prime }\Psi {]}^{\top })}^{-1}.$$ Their asymptotic covariance matrices are $${\left({\nabla }_{\theta }{\Psi }^{\top }{([(\begin{array}{cc}I& 0\end{array})-{\nabla }_{(p,g)\prime }\Psi ]\Sigma [(\begin{array}{cc}I& 0\end{array})-{\nabla }_{(p,g)\prime }\Psi {]}^{\top })}^{-1}{\nabla }_{{\theta }^{\top }}\Psi \right)}^{-1}$$

Here, $0$ denotes a $(N\cdot K\cdot m_s)\times (m_a\cdot m_s\cdot m_s)$ matrix of zeros.

Asymptotic Least Squares: Moment Estimator

The moment estimator proposed by Hotz and Miller (1993) is an asymptotic least squares estimator with a particular weight matrix.

Let $T_{\mathrm{is}}$ denote the set of observations for individual $i$ in state $s$ and let ${\alpha }_{\mathrm{is}}=({\alpha }_1,\dots ,{\alpha }_K)$ be a vector of indicators for each choice (with zero omitted).

The moment condition is $$E[Z\otimes ({\alpha }_{\mathrm{is}}-{\Psi }_{\mathrm{is}}({\hat{p}}_T,{\hat{g}}_T,\theta ))]=0$$ where $Z$ is a $J\times 1$-dimensional vector of instruments.

Suppose $Z_t=Z_{\mathrm{is}}$. Then the corresponding sample analog becomes $$\frac{1}{\mathrm{NT}}\sum _{\stackrel{1\le i\le N}{s\in S}}\sum _{t\in T_{\mathrm{is}}}{Z}_t\otimes ({\alpha }_t-{\Psi }_{\mathrm{is}}({\hat{p}}_T,{\hat{g}}_T,\theta ))=\frac{1}{\mathrm{NT}}\sum _{\stackrel{1\le i\le N}{s\in S}}{n}_{\mathrm{is}}[Z_{\mathrm{is}}\otimes ({\hat{p}}_{\mathrm{is}}-{\Psi }_{\mathrm{is}}({\hat{p}}_T,{\hat{g}}_T,\theta ))].$$

Thus, the moment estimator in this case is an asymptotic least squares estimator with estimating equation $\hat{p}-\Psi ({\hat{p}}_T,{\hat{g}}_T,\theta )=0$.

Asymptotic Least Squares: Pseudo Maximum Likelihood

The pseudo maximum likelihood estimator of Aguirregabiria and Mira (2002, 2007) is also an asymptotic least squares estimator.

The partial pseudo log-likelihood, conditional on estimates ${\hat{g}}_T$ is $$\ell =\sum _{s\in S}\sum _{i=1}^N\sum _{k\in A_i}{n}_{\mathrm{kis}}\ln {\Psi }_{\mathrm{kis}}({\hat{p}}_T,{\hat{g}}_T,\theta ).$$

The first order condition is $$\frac{\partial \ell }{\partial \theta }=\left({\nabla }_{\theta }{\Psi }^{\top }\right){\Sigma }_p^{-1}(\Psi )[\hat{p}-\Psi ({\hat{p}}_T,{\hat{g}}_T,\theta )]$$ where ${\Sigma }_p^{-1}(\Psi )$ is the inverse covariance matrix of the choice probabilities.

This is equivalent to the first order condition of the asymptotic least squares estimator with weight matrix $W_T^{\mathrm{ml}}\stackrel{p}{\to }{\Sigma }_p^{-1}$.

Monte Carlo Study

• Compare LS-E, PML, LS-I, and k-PML.
• A simple two player, two action, two state, game with five equilibria.
• Three equilibria are used for experiments with various sample sizes.
• LS-E estimator performs best overall (in eight of 12 experiments).
• LS-E performs poorly with the smallest sample size ($T=100$).
• PML ranks second (by MSE) in seven of 12 specifications.
• PML performs better than LS-E for $T=100$ and worse for larger sample sizes. This may be because the covariance matrix of $({\hat{p}}_T,{\hat{g}}_T)$ is estimated better than the efficient weight matrix for small $T$.
• PML may be less computationally burdensome for large state spaces.