How to use linear mixed effect model to test two sets of data are significantly different?

Hi,

I am learning how to use linear mixed effect model to test two set of datas.

Suppose I have my table like this, where data1 and data2 are what I want to compare and see if they are different. 'sampleID' records from which object the data is collected.

'data1' 'data2' 'SampleID'

0.516889740895640 0.495026736720585 '3'

0.568247158919234 0.553939998754281 '3'

0.384527694682852 0.639242823546553 '2'

0.243338708824316 0.477584777544589 '2'

0.388269268112705 0.772324689922884 '1'

0.346430703280194 0.752582761153593 '1'

0.563529866967876 0.847575046711875 '1'

I believe sampleID will have random effects on both data1 and data2, therefore I am considering using mixed effect model instead of t test to run statistical checks.

To the best of my understanding, I wrote the follow code. However, I don't know how can I get the p value which I can get from a normal t test. Can anyone let me know how to interpret the results?

lme = fitlme(tb,'data1~data2+(data2|SampleID)+(data1|SampleID)')

The output is:

lme =

Linear mixed-effects model fit by ML

Model information:

Number of observations 7

Fixed effects coefficients 2

Random effects coefficients 12

Covariance parameters 7

Formula:

data1 ~ 1 + data2 + (1 + data2 | SampleID) + (1 + data1 | SampleID)

Model fit statistics:

AIC BIC LogLikelihood Deviance

-111.09 -111.58 64.546 -129.09

Fixed effects coefficients (95% CIs):

Name Estimate SE tStat DF pValue Lower Upper

{'(Intercept)'} 0.53456 1.1255e-07 4.7494e+06 5 7.8541e-33 0.53456 0.53456

{'data2' } -2.6199e-14 1.6541e-07 -1.5839e-07 5 1 -4.2519e-07 4.2519e-07

Random effects covariance parameters (95% CIs):

Group: Eye (3 Levels)

Name1 Name2 Type Estimate Lower Upper

{'(Intercept)'} {'(Intercept)'} {'std' } 3.8333e-15 NaN NaN

{'data2' } {'(Intercept)'} {'corr'} -0.98044 NaN NaN

{'data2' } {'data2' } {'std' } 5.7463e-15 NaN NaN

Group: Eye (3 Levels)

Name1 Name2 Type Estimate Lower Upper

{'(Intercept)'} {'(Intercept)'} {'std' } 0.34995 NaN NaN

{'data1' } {'(Intercept)'} {'corr'} -1 NaN NaN

{'data1' } {'data1' } {'std' } 0.65465 NaN NaN

Group: Error

Name Estimate Lower Upper

{'Res Std'} 4.8154e-08 NaN NaN

Then I checked the tutorial, there is a case which is very similar to what I want:

Test if there is any difference between the evening and morning shifts.

pVal = coefTest(lme,[0 1 -1])

pVal = 3.6147e-04

This small p-value indicates that the performance of the operators are not the same in the morning and the evening shifts.

However, I am having difficulty understanding what is [0 1 -1] means.

Can anybody help? Thank you.

Answer 1

[Paras Gupta]

Hi Fengting,

I understand that you want to use the linear mixed effect model between two sets of data - 'data1’ and ‘data2’, with random effects from a third grouping variable ‘SampleID’ and find the p-value of the statistical test.

lme = fitlme(tb,'data1~data2+(data2|SampleID)+(data1|SampleID)') 

The formula used for model specification in the above code allows for the possibility that the relationship between ‘data2’ and ‘data1’ may vary across different levels of ‘SampleID’.

The provided output of the created model object for the sample table shows the estimates of the fixed-effect coefficients for ‘(Intercept)’ and ‘data2’. The ‘(Intercept)’ represents the average value of data1 when data2 is zero. It can be seen that,

• The coefficient for ‘(Intercept)’ has an estimated value of 0.53456 with a standard error of 1.1255e-07. The p-value indicates that it is statistically significant (p < 0.001).
• The coefficient for data2 is estimated to be -2.6199e-14 with a standard error of 1.6541e-07. However, the non-significant p-value (p = 1) for data2 suggests that there is no significant linear relationship between data2 and data1 in the model.

The second argument provided in the coefTest function is the contrast matrix. The contrast matrix H is used to specify a linear combination of the fixed-effects coefficients for which you want to test the hypothesis. Each row of the contrast matrix represents a separate hypothesis. It tests the null hypothesis that H0: Hβ = 0, where β is the fixed-effects vector.

pVal = coefTest(lme,[0 1]) 

Therefore, the output of the above statement is the p-value of the fixed-effects coefficient for ‘data2’.

You can refer to the following documentation for more information.

• https://www.mathworks.com/help/stats/linearmixedmodel.html
• https://www.mathworks.com/help/stats/fitlme.html
• https://www.mathworks.com/help/stats/linearmixedmodel.coeftest.html

Hope this helps.