Multcompare for friedman structure

Hi there,

Does anyone know what kind of pairwise test the multcompare function perform, when the input stats structure comes from a friedman test? On the description, only the t-test is mentioned, however this is not relevant to friedman tests which are usually employed under non-normality assumptions.

Multiple-comparison results also look a bit funny for friedman tests. For example, although a significant factor is discovered during the friedman test (p<1e-24) no significant differences come across during the multiple comparisons, -when it is obvious from the data that there are some-.

I've had a look at the multcompare.m source code, but couldn't figure what pairwise test is performed, although it seems to me that this is the same regardless of the stats.source.

Any help would be appreciated.

Agamemnon

[Ernesto Soares]

I have the exact same problem. Have you fixed it? How? Thanks! Ernesto

Answer 1

[Jeremy Kemmerer]

Hi Agamemnon,

The function ‘multcompare’ should accept a ‘stats’ structure from the function ‘friedman’ without any problems. While it seems strange that you would see such a low p-value from the ‘friedman’ test but see no significant differences pair-wise, I believe this does happen in practice. Does the graph produced by ‘multcompare’ show groups that are far apart?

Also, by default, MATLAB uses a ‘tukey-kramer’ critical value for data sets generated from ‘friedman.’ This critical value can be changed by changing the ‘ctype’ parameter, although other values may be more conservative.

[Agamemnon Krasoulis]

Thanks for your answer Jeremy.

It does accept a stats structure from 'friedman'. The groups are shown not to be far part -although this graph is not easy interpretable. When I have boxplots the groups are very far part but in the transformed median plot they are not.

I 've changed the correction procedure to Bonferroni. I have also tried with the 'lsd' method which should return significant values as there are no corrections whatsoever, but this didn't work either.

I suspect there is a 'bug' in the multcompare function which performs t-tests regardless of the type of test. If you look at the description of the function, and concretely at the description of the 'lsd' correction procedure, it specifically says that this just performs ordinary t-tests. However, the nature of the test should depend on the source test. If the source test is ANOVA, t-test is the way to go. If the source test is Friedman, then non-parametric (e.g. Wilcoxon ranksum) tests should be used in my humble opinion. Anyone from Mathworks to verify this?

Now I have written custom code which performs individual ranksum tests and also corrects for multiple comparisons.

[gbernardi]

I'm in the same situation described here. Has there been any update?

[Christian Tieber]

Same problem here. No updates yet?