The Benjamini-Hochberg procedure is a method to change the significance value when doing multiple hypothesis testing. The explanation to this is that if you’re doing a lot of hypothesis tests on a single dataset, you’re bound to find something. If the type I error is \(\alpha\) in one test, in \(k\) different tests, the probability of rejecting at least 1 null hypothesis when it’s true becomes \(1-(1-\alpha)^k\).

Implementing this procedure in R is quite straightforward. Suppose again you have \(H_0,…,H_k\) null hypothesis with \(p_0,…,p_k\) p-values.

## Benjamini–Hochberg procedure (BH step-up procedure) BHproc <- function(pvalues,alpha){ sorted <- sort(pvalues) m <- length(pvalues) k = 1 while(p[k]<=k/m*alpha){ k = k+1 } cat("Declared significant: ",k, " first tests. Note p-values were ordered") }