Why do mattest and ttest2 produce two different p-values?

I was looking for a function to perform a two sample t-test for an array of samples and noticed that I could do each set individually with ttest2 or do all at once with mattest. mattest appears to me to just be a function that will process each row all at once and I thought that was the only difference. However, I noticed that when I collected p-values each way they had different values. Why is this? Is there a bigger difference between these two function than I had thought? Thank you for your response.

Answer 1

[Siddharth Sundar]

I understand that you want to know why the functions mattest and ttest2 behave differently for the same input.

Both functions accept the two primary inputs as either vectors or matrices. In your case, you indicated that you wanted to work on matrices, in which case, they are required to have the same number of columns. There are a couple of differences in the standard syntax for mattest and ttest2 functions that contribute to this difference in behavior for the same inputs.

Firstly, mattest performs the test assuming the two samples have unknown and unequal variances unless the VarTypeValue argument is specifically set to 'equal'. On the other hand, ttest2 conducts a test using the assumption that the two samples are from normal distributions with unknown but equal variances.

Secondly, unless the 'Dim' Name-Value pair in ttest2 is explicitly set to 2 (to test the row means), it assumes it to be the first nonsingleton dimension.

Here is an example using both functions to generate the same p-values:

x = rand(2,5);
y = rand(2,5);

% Using the mattest function
p1 = mattest(x,y)

% Using the ttest2 function with the necessary Name-Value pairs
[~,p2,~,~] = ttest2(x,y,'vartype','unequal','dim',2)