Recalibration of p-values for Multiple Testing Problems in Genomics

Abstract

Statistical practice has shown a preference for conservative statistical tests in situations where exact control of the Type I error rate is difficult, likely due to a belief that the consequences of falsely rejecting the null hypothesis are usually greater than the opposing Type II error of failing to reject an incorrect null. Conservative tests, however, prevent us from detecting some of the real differences between two groups and, as a consequence, reduce statistical power. For example, a conservative RNA-seq differential expression test may not allow us to detect some differentially expressed genes, even when the sample size is sufficient for a non-conservative test.

Here, we present techniques for recalibrating the p-values of conservative statistical tests with the aim of increasing the number of true-positive findings at each given p-value threshold. As a secondary benefit, these techniques may allow us to reduce the sample size required to achieve a desired level of power, leading to reduced experimental costs.