Stan

Probabilistic programming language

HMC/NUTS

Compared to others:

  • Fast convergence
  • No conjugacy required
  • warm-up vs. burn-in1
  • less autocorrelation
  • faster for more complex models

Why use?

  • Fit very complex models
  • Better approaches for model diagnostics
  • Natural interval estimates for any statistic that comes out of a model

data {                      // Data block
  int<lower=1> N;           // Sample size
  int<lower=1> K;           // Dimension of model matrix
  matrix[N, K] X;           // Model Matrix
  vector[N] y;              // Target variable
}

parameters {                // Parameters block
  vector[K] beta;           // Coefficients
  real<lower=0> sigma;      // Error scale
}

model {                     // Model block
  vector[N] mu;
  mu = X * beta;            // Linear predictor
  
  // priors
  beta ~ normal(0, 10);
  sigma ~ cauchy(0, 5);     
  
  // likelihood
  y ~ normal(mu, sigma);
}

  1. Gelman et al. (2013) : In the simulation literature (including earlier editions of this book), the warm-up period is called burn-in, a term we now avoid because we feel it draws a misleading analogy to industrial processes in which products are stressed in order to reveal defects. We prefer the term ‘warm-up’ to describe the early phase of the simulations in which the sequences get closer to the mass of the distribution. (see also, this discussion at Gelman’s blog)↩