In Cooperation with:

American Society for Quality Statistics Division

American Statistical Association

Bernoulli Society for Mathematical Statistics and Probability

Institute of Mathematical Statistics

International Biometric Society

International Chinese Statistical Association

International Society for Bayesian Analysis

International Statistical Institute

Royal Statistical Society

Statistical Society of Canada / Société statistique du Canada

Linear Mixed Model

ref

Linear Mixed Model

Geert Molenberghs

I-BioStat, Universiteit Hasselt & Katholieke Universiteit Leuven, Belgium.

In many studies, data are collected hierarchically. Not always do such data follow balanced, multivariate designs. For example, in repeated measurements may be taken at almost arbitrary time points, resulting in an extremely large number of time points at which only one or only a few measurements have been taken. Many of the parametric covariance models described so far may then contain too many parameters to make them useful in practice, while other, more parsimonious, models may be based on assumptions which are too simplistic to be realistic. A general, and very flexible, class of parametric models for continuous longitudinal data is formulated as follows:

$\displaystyle \mbox{\boldmath$y_i$}$ $\displaystyle \vert$ $\displaystyle \mbox{\boldmath$b_i$}$	$\displaystyle \sim$	$\displaystyle N(X_i$ $\displaystyle \mbox{\boldmath$\beta$}$ $\displaystyle + Z_i$ $\displaystyle \mbox{\boldmath$b_i$}$ $\displaystyle , \Sigma_i),$	(1)
$\displaystyle \mbox{\boldmath$b_i$}$	$\displaystyle \sim$	$\displaystyle N($ $\displaystyle \mbox{\boldmath$0$}$ $\displaystyle , D),$	(2)

where

and

are $(n_i \times p)$ and $(n_i \times q)$ dimensional matrices of known covariates, $\beta$ is a -dimensional vector of regression parameters, called the fixed effects, is a general $(q \times q)$ covariance matrix, and $\Sigma_i$ is a $(n_i \times n_i)$ covariance matrix which depends on only through its dimension , i.e., the set of unknown parameters in $\Sigma_i$ will not depend upon . Finally, is a vector of subject-specific or random effects.

The above model can be interpreted as a linear regression model for the vector of repeated measurements for each unit separately, where some of the regression parameters are specific (random effects, ), while others are not (fixed effects, $\beta$ ). The distributional assumptions in (2) with respect to the random effects can be motivated as follows. First, E implies that the mean of still equals $\beta$ , such that the fixed effects in the random-effects model (1) can also be interpreted marginally. Not only do they reflect the effect of changing covariates within specific units, they also measure the marginal effect in the population of changing the same covariates. Second, the normality assumption immediately implies that, marginally, also follows a normal distribution with mean vector $\beta$ and with covariance matrix $V_i = Z_i D Z_i' + \Sigma_i$ .

Note that the random effects in (1) implicitly imply the marginal covariance matrix of to be of the very specific form $V_i=Z_i D Z_i' + \Sigma_i$ . Let us consider two examples under the assumption of conditional independence, i.e., assuming $\Sigma_i=\sigma^2 I_{n_i}$ . First, consider the case where the random effects are univariate and represent unit-specific intercepts. This corresponds to covariates which are -dimensional vectors containing only ones.

The marginal model implied by expressions (1) and (2) is

$\displaystyle \mbox{\boldmath$y_i$}$

$\displaystyle \sim$

$\displaystyle N(X_i$ $\displaystyle \mbox{\boldmath$\beta$}$ $\displaystyle , V_i), \quad V_i = Z_i D Z_i' + \Sigma_i$

which can be viewed as a multivariate linear regression model, with a very particular parameterization of the covariance matrix .

With respect to the estimation of unit-specific parameters , the posterior distribution of given the observed data can be shown to be (multivariate) normal with mean vector equal to $D Z_i' V_i^{-1}($ $\alpha$ $\beta$ . Replacing $\beta$ and $\alpha$ by their maximum likelihood estimates, we obtain the so-called empirical Bayes estimates $\widehat{\mbox{\boldmath $b_i$}}$ for the . A key property of these EB estimates is shrinkage, which is best illustrated by considering the prediction $\widehat{\mbox{\boldmath $y_i$}} \equiv X_i \widehat{\mbox{\boldmath $\beta$}} + Z_i \widehat{\mbox{\boldmath $b_i$}}$ of the th profile. It can easily be shown that

$\displaystyle \widehat{\mbox{\boldmath$y_i$}}$

$\displaystyle =$

$\displaystyle \Sigma_i V_i^{-1} X_i \widehat{\mbox{\boldmath$\beta$}} \ + \left(I_{n_i} - \Sigma_i V_i^{-1} \right) \mbox{\boldmath$y_i$},$

which can be interpreted as a weighted average of the population-averaged profile $X_i \widehat{\mbox{\boldmath $\beta$}}$ and the observed data , with weights $\Sigma_i V_i^{-1}$ and $I_{n_i} - \Sigma_i V_i^{-1}$ , respectively. Note that the ``numerator'' of $\Sigma_i V_i^{-1}$ represents within-unit variability and the ``denominator'' is the overall covariance matrix . Hence, much weight will be given to the overall average profile if the within-unit variability is large in comparison to the between-unit variability (modeled by the random effects), whereas much weight will be given to the observed data if the opposite is true. This phenomenon is referred to as shrinkage toward the average profile $X_i \widehat{\mbox{\boldmath $\beta$}}$ . An immediate consequence of shrinkage is that the EB estimates show less variability than actually present in the random-effects distribution, i.e., for any linear combination $\lambda$ of the random effects,

var $\displaystyle ($ $\displaystyle \mbox{\boldmath$\lambda$}$ $\displaystyle ' \widehat{\mbox{\boldmath$b_i$}})$

$\displaystyle \leq$

var $\displaystyle ($ $\displaystyle \mbox{\boldmath$\lambda$}$ $\displaystyle '$ $\displaystyle \mbox{\boldmath$b_i$}$ $\displaystyle ) =$ $\displaystyle \mbox{\boldmath$\lambda$}$ $\displaystyle ' D$ $\displaystyle \mbox{\boldmath$\lambda$}$ $\displaystyle .$

References

Fitzmaurice, G.M., Davidian, M., Verbeke, G., and Molenberghs, G.(2009). Longitudinal Data Analysis. Handbook. Hoboken, NJ: John Wiley & Sons.

Fitzmaurice, G.M., Laird, N.M., and Ware, J.H. (2004). Applied Longitudinal Analysis. New York: John Wiley & Sons.

Henderson, C.R. (1984) Applications of Linear Models in Animal Breeding. Guelph, Canada: University of Guelph Press.

Verbeke, G. and Molenberghs, G. (2000) Linear Mixed Models for Longitudinal Data. New York: Springer.

Acknowledgment

Based on an article from Lovric, Miodrag (2011), International Encyclopedia of Statistical Science. Heidelberg: Springer Science +Business Media, LLC

Linear Mixed Model is owned by Geert Molenberghs.

Note

NOTE ON THE PROPER CITATION OF THIS ARTICLE: The proper citation for this article is Geert Molenberghs, "Linear Mixed Model", In: Lovric, Miodrag (Editor), (2010). International Encyclopedia of Statistical Science. Heidelberg: Springer Science +Business Media, LLC, reprinted and freely available at StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies.

How to Cite This Entry

Geert Molenberghs. "Linear Mixed Model" (version 6). StatProb: The Encyclopedia Sponsored by Statistics and Probability Societies. Freely available at http://statprob.com/encyclopedia/LinearMixedModel.html

Classification

AMS MSC:

62-07 (Statistics :: Data analysis)

Pending Errata and Addenda

1. Linear Mixed Model by acarriquiry on 2010-09-22 16:34:30
[ View all 1 ]

Discussion

No messages.

Username:
Password:

Register
I've forgotten my login details

Login

In Cooperation with:

Linear Mixed Model

References

Acknowledgment

Note

How to Cite This Entry

Classification

Pending Errata and Addenda

Discussion