Posterior Predictive Checks
Posterior predictive checks can let us inspect what the model suggests for our target variable vs. what actually is the case6
Lots to play with
Error: Type 'x' is not a valid ppc type. Valid types are:
'bars', 'bars_grouped', 'boxplot', 'data', 'dens', 'dens_overlay', 'ecdf_overlay', 'error_binned', 'error_hist', 'error_hist_grouped', 'error_scatter', 'error_scatter_avg', 'error_scatter_avg_vs_x', 'freqpoly', 'freqpoly_grouped', 'hist', 'intervals', 'intervals_data', 'intervals_grouped', 'loo_intervals', 'loo_pit', 'loo_pit_overlay', 'loo_pit_qq', 'loo_ribbon', 'ribbon', 'ribbon_data', 'ribbon_grouped', 'rootogram', 'scatter', 'scatter_avg', 'scatter_avg_grouped', 'stat', 'stat_2d', 'stat_freqpoly_grouped', 'stat_grouped', 'violin_grouped'How well did we capture the mean, or some quantile?
q75 <- function(y) quantile(y, 0.75)
pp_check(attendance_brms, type='stat', stat='q75', nsamples = 100)
What can we say about the predictive error?
The Poisson’s underlying assumption of the mean equaling the variance rarely holds with typical data. One way to handle overdispersion in count models is to move to something like negative binomial or other approaches. Interestingly, for Poisson models we can have a random effect per observation7 to model additional variance. In this case, our pp_check suggests a much better result.
attendance_brms_add_re = update(attendance_brms, . ~ . + (1|id),
newdata = attendance)
pp_check(attendance_brms_add_re)
For more on this see Ben Bolker’s demonstration with lme4.
The posterior predictive distribution is the distribution of the outcome variable implied by a model after using the observed data y (a vector of outcome values), and typically predictors X, to update our beliefs about the unknown parameters θ in the model. For each draw of the parameters θ from the posterior distribution p(θ | y, X) we generate an entire vector of outcomes.↩
This essentially changes the model to incorporate a Poisson log-normal distribution. As Montesinos-López et al. (2017) describe: > The Poisson component of the Poisson-lognormal distribution accommodates integer inputs (or outputs) to describe the actual number of counts observed within a single unit or sample, while the lognormal component of the distribution describes the overdispersion in the Poisson rate parameter…↩