• Model Estimation by Example
  • Introduction
  • Models
  • Linear Regression
    • Data Setup
    • Functions
    • Estimation
    • Comparison
    • Python
    • Source
  • Logistic Regression
    • Data Setup
    • Functions
    • Estimation
    • Comparison
    • Python
    • Source
  • Mixed Models
    • One-factor Mixed Model
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Source
    • Two-factor Mixed Model
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Source
    • Mixed Model via ML
      • Introduction
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Link with penalized regression
      • Source
  • Probit & Bivariate Probit
    • Standard Probit
      • Function
      • Examples
    • Bivariate Probit
      • Example
    • Source
  • Heckman Selection
    • Data Setup
    • Two step approach
      • Step 1: Probit Model
      • Step 2: Estimate via Linear Regression
    • Maximum Likelihood
      • Comparison
    • Source
  • Marginal Structural Model
    • Data Setup
    • Function
    • Estimation
    • Comparison
    • Source
  • Tobit Regression
    • Censoring with an Upper Limit
      • Data Setup
      • Function
      • Estimation
      • Comparison
    • Censoring with a Lower Limit
      • Comparison
    • Source
  • Cox Survival
    • Standard Proportional Hazards
      • Data Setup
      • Function
      • Estimation
      • Comparison
    • Time-varying coefficients
      • Data Setup
      • Function
      • Estimation
      • Comparison
    • Stratified Cox Model
      • Data Setup
      • Function
      • Estimation
      • Comparison
    • Source
  • Hurdle Model
    • Poisson
      • Data Setup
      • Function
      • Estimation
      • Comparison
    • Negative Binomial
      • Function
      • Estimation
      • Comparison
    • Source
  • Zero-Inflated Model
    • Poisson
      • Data Setup
      • Function
      • Estimation
      • Comparison
    • Negative Binomial
      • Function
      • Estimation
      • Comparison
    • Supplemental Example
    • Source
  • Naive Bayes
    • Initialization
    • Comparison
    • Source
  • Multinomial Regression
    • Standard (Categorical) Model
      • Data Setup
      • Function
      • Comparison
    • Alternative specific and constant variables
      • Comparison
    • Source
  • Ordinal Regression
    • Data
    • Function
    • Estimation
    • Comparison
    • Source
  • Markov Model
    • Data Setup
      • Data Functions
      • Two states Demo
      • Three states demo
    • Function
    • Estimation
    • Comparison
    • Source
  • Hidden Markov Model
    • Data Setup
    • Function
    • Estimation
    • Supplemental demo
    • Source
  • Quantile Regression
    • Data Setup
    • Function
    • Estimation
      • Other quantiles
    • Comparison
    • Visualize
    • Python
    • Source
  • Cubic Spline Model
    • Data Setup
    • Functions
    • Example 1
    • Example 2
    • Source
  • Gaussian Processes
    • Noise-Free Demonstration
      • Data Setup
      • Functions
      • Visualize the prior distribution
      • Generate the posterior predictive distribution
      • Visualize the Posterior Predictive Distribution
    • Noisy Demonstration
      • Data Setup
      • Functions
      • Visualize the prior distribution
      • Generate the posterior predictive distribution
      • Visualize the Posterior Predictive Distribution
    • Source
  • Neural Network
    • Example 1
      • Python
      • R
    • Example 2
      • Python
      • R
      • Comparison
    • Example 3
      • Python
      • R
      • Comparison
    • Source
  • Extreme Learning Machine
    • Data Setup
    • Function
    • Estimation
    • Comparison
    • Supplemental Example
    • Source
  • Reproducing Kernel Hilbert Space Regression
    • Data Setup
    • Functions
      • Function to find the inverse of a matrix
      • Reproducing Kernel
      • Gram matrix
      • Ridge regression
    • Estimation
    • Comparison
    • Example: Cubic Spline
      • Data Setup
      • Functions
      • Estimation
      • Comparison
    • Source
      • Current Supplemental Code
      • Original Supplemental Code
  • Confirmatory Factor Analysis
    • Data Setup
    • Functions
    • Estimation
      • Raw
      • Standardized
    • Comparison
      • Mplus
    • Source
  • Bayesian
  • Introduction to Bayesian Methods
  • Basics
    • Prior, likelihood, & posterior distributions
    • Prior
    • Likelihood
    • Posterior
    • Posterior predictive
    • Source
  • Bayesian t-test
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Visualization
    • Source
  • Bayesian Linear Regression
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Visualize
    • Source
  • Bayesian Beta Regression
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Visualization
    • Source
  • Bayesian Mixed Model
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Visualize
    • Source
  • Bayesian Multilevel Mediation
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Visualization
    • Source
  • Bayesian IRT
    • One Parameter IRT
      • Data Setup
      • Model Code
      • Estimation
      • Comparison
    • Two Parameter IRT
      • Model Code
      • Estimation
      • Comparison
    • Three Parameter IRT
    • Four Parameter IRT
    • Source
  • Bayesian CFA
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Source
  • Bayesian Nonparametric Models
    • Chinese Restaurant Process
    • Indian Buffet Process
    • Source
  • Bayesian Stochastic Volatility Model
    • Data Setup
    • Model Code
    • Estimation
    • Results
    • Visualization
    • Source
  • Bayesian Multinomial Models
    • Data Setup
    • Model Code
    • Estimation
    • Comparison
    • Adding Complexity
    • Source
  • Variational Bayes Regression
    • Data Setup
    • Function
    • Estimation
    • Comparison
    • Visualization
    • Supplemental Example
    • Source
  • Topic Model
    • Data Setup
    • Function
    • Estimation
    • Comparison
    • Source
  • Estimation
  • Maximum Likelihood
    • Example
    • Linear Model
    • Source
  • Penalized Maximum Likelihood
    • Introduction
      • Data Setup
      • Functions
      • Estimation
      • Comparison
      • Source
    • L1 (lasso) regularization
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Source
    • L2 (ridge) regularization
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Source
  • Newton and IRLS
    • Data Setup
    • Functions
      • Newton’s Method
      • IRLS
    • Estimation
    • Comparison
    • Source
  • Nelder-Mead
    • Functions
      • First Version
      • Example
      • A Regression Model
    • Second Version
      • Example
      • A Regression Model
    • Source
  • Expectation-Maximization
    • Mixture Model
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Visualization
      • Supplemental Example
      • Source
    • Multivariate Mixture Model
      • Function
      • Example 1: Old Faithful
      • Example 2: Iris
      • Source
    • Probit Model
      • Data Setup
      • Probit via Maximum Likelihood
      • EM for Latent Variable Approach to Probit
      • Source
    • PCA
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Visualize
      • Source
    • Probabilistic PCA
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Visualize
      • Missing Data Example
      • Source
    • State Space Model
      • Data Setup
      • Function
      • Estimation
      • Visualization
      • Source
  • Gradient Descent
    • Data Setup
    • Function
    • Estimation
    • Comparison
    • Source
  • Stochastic Gradient Descent
    • Data Setup
    • Function
    • Estimation
    • Comparison
    • Visualize Estimates
    • Data Set Shift
      • Data Setup
      • Estimation
      • Comparison
      • Visualize Estimates
    • SGD Variants
      • Data Setup
      • Function
      • Estimation
      • Comparison
      • Visualize Estimates
    • Source
  • Metropolis Hastings
    • Data Setup
    • Functions
    • Estimation
    • Comparison
    • Source
  • Hamiltonian Monte Carlo
    • Data Setup
    • Functions
    • Estimation
    • Comparison
    • Source
  • Other
  • Supplemental
    • Other Languages
    • Python Demos
      • Linear Regression
      • Logistic Regression
      • Quantile Regression
      • Stochastic Gradient Descent
      • Nelder-Mead
      • HMM
    • Julia Demos
      • Mixed Models
    • Matlab Demos
      • Mixed Models
      • Gaussian Processes
  • MC logo

Model Estimation by Example

Model Estimation by Example

Demonstrations with R

Michael Clark
m-clark.github.io