Models by Example

Roll your own to understand more.

regression
machine learning
Author
Affiliation
Published

November 30, 2020

Keywords

R, Python, Julia, Matlab, algorithms, bayesian, neural network, linear regression, lasso, pca, expectation maximization, splines, logistic regression, gradient descent, rkhs, ridge regression, cox, hamiltonian monte carlo, mixed models, penalized regression, maximum likelihood estimation, tobit, multinomial, ordinal, zero inflated, marginal structural models

New Book

I’ve completed a new bookdown document, Models by Example, that converts most of the code from my Miscellaneous R repo. I initially just wanted to update the code, but decided to use a more formal approach to make it cleaner and more accessible. It’s mostly complete, though may be added to on rare occasion, and further cleaned as I find annoying bits here and there. Each topic contains ‘by-hand’ demonstration, such that you can see conceptually how a model is estimated, or technique employed. This can help those that want to dive a little deeper to get a peek behind the curtain of the functions and packages they use, hopefully empowering them to go further with such models.

Topics covered include the following, and I plan to post a sample chapter soon.

Models
  • Linear Regression
  • Logistic Regression
  • One-factor Mixed Model
  • Two-factor Mixed Model
  • Mixed Model via ML
  • Probit & Bivariate Probit
  • Heckman Selection
  • Marginal Structural Model
  • Tobit
  • Cox Survival
  • Hurdle Model
  • Zero-Inflated Model
  • Naive Bayes
  • Multinomial
  • Ordinal
  • Markov Model
  • Hidden Markov Model
  • Quantile Regression
  • Cubic Spline Model
  • Gaussian Processes
  • Neural Net
  • Extreme Learning Machine
  • Reproducing Kernel Hilbert Space Regression
  • Confirmatory Factor Analysis
Bayesian
  • Basics
  • Bayesian t-test
  • Bayesian Linear Regression
  • Bayesian Beta Regression
  • Bayesian Mixed Model
  • Bayesian Multilevel Mediation
  • Bayesian IRT
  • Bayesian CFA
  • Bayesian Nonparametric Models
  • Bayesian Stochastic Volatility Model
  • Bayesian Multinomial Models
  • Variational Bayes Regression
  • Topic Model
Estimation
  • Maximum Likelihood
  • Penalized Maximum Likelihood
  • L1 (lasso) regularization
  • L2 (ridge) regularization
  • Newton and IRLS
  • Nelder Mead
  • Expectation-Maximization
  • Gradient Descent
  • Stochastic Gradient Descent
  • Metropolis Hastings
  • Hamiltonian Monte Carlo

Reuse

Citation

BibTeX citation:
@online{clark2020,
  author = {Clark, Michael},
  title = {Models by {Example}},
  date = {2020-11-30},
  url = {https://m-clark.github.io/posts/2020-11-30-models-by-example/},
  langid = {en}
}
For attribution, please cite this work as:
Clark, Michael. 2020. “Models by Example.” November 30. https://m-clark.github.io/posts/2020-11-30-models-by-example/.