As we can clearly see, the overall significance level is much higher than the nominal significance level of 5% associated with each one of the tests. If we want the probability of declaring a significant finding in the trial, when no such declaration should have been made, to be at most 5%, we must make the significance level of each separate test much smaller. This is a typical property of multiple testing. When our experiment involves scanning over many possibilities, and when an extreme value of the individual test statistics is an indication of departure from the null hypothesis, then a larger threshold should be used compared to the threshold that would have been used had there been only a single test.
Compare the distribution of \Zs\ for a fixed value of s to the distribution of maxt \Zt\. (See Fig. 5.2.)
Was this article helpful?