We propose a nonlinear infinite moving average as an alternative to the standard state space policy function for solving nonlinear DSGE models. Perturbation of the nonlinear moving average policy function provides a direct mapping from a history of innovations to endogenous variables, decomposes the contributions from individual orders of uncertainty and nonlinearity, and enables familiar impulse response analysis in nonlinear settings. When the linear approximation is saddle stable and free of unit roots, higher order terms are likewise saddle stable and first order corrections for uncertainty are zero. We derive the third order approximation explicitly, examine the accuracy of the method using Euler equation tests, and compare with state space approximations.