Application of false discovery rate procedure to pairwise comparisons of refractive index of glass fragments

This study was undertaken to apply a new method of controlling type I error when performing pairwise comparisons. The Benjamini and Hochberg false discovery rate (FDR) controlling procedures have proved to be very powerful tools in solving many practical problems but have not yet been applied to pairwise comparisons of refractive index of glass samples. Students t-test and Welch test (unequal variance Student's t-test) were applied to all possible pairwise comparisons. The comparisons were made on the basis of refractive index values of 72 glass fragments from different car windows and 69 different fragments from one windscreen. The type I error was controlled by the use of Benjamini and Hochberg false discovery rate (FDR) controlling procedure. To illustrate the importance of controlling type I error when using pairwise comparisons, results of pairwise comparisons with the FDR controlling procedure were compared to those comparisons made without any controlling procedure. Significantly fewer false negative results (false rejection of H0) were found during the use of FDR procedure in the comparison of refractive indices from the same windscreen than when comparing without controlling type I error. The results of application of FDR method were also compared to the results of other post-hoc tests, such as Tukey HSD test and Bonfferoni test. The FDR method has higher power than Bonferroni method and Tukey HSD method and control errors better than comparing without controlling type I error. The method of choice, for pairwise comparison of glass fragments on the basis of refractive index is Welch test with FDR adjustment. The method gave type I errors at level about 5.3% and type II error at about 3.7%.