We introduce a new class of models that has both stochastic volatility and moving average errors, where the conditional mean has a state space representation. Having a moving average component, however, means that the errors in the measurement equation are no longer serially independent, and estimation becomes more difficult. We develop a posterior simulator that builds upon recent advances in precision-based algorithms for estimating these new models. In an empirical application involving U.S. inflation we find that these moving average stochastic volatility models provide better in sample fitness and out-of sample forecast performance than the standard variants with only stochastic volatility.