KYCG testEnrichmentSEA function appears to give biased p-values that are stronger for positive estimates than negative ones.

[jdreyf]

We applied testEnrichmentSEA(z, databases = db, platform = "MM285", precise = TRUE) using sesame (v.1.24.0) for different databases where z is a vector of z-scores and found that positive estimates per DB in the output were given smaller p-values than negative estimates. So we tried testEnrichmentSEA(-1 * z, databases = db, platform = "MM285", precise = TRUE) to test the method, since I think it should be symmetric about positive vs. negative estimates. We found that the estimates did flip sign, but the p-values, instead of staying the same, were again biased toward being smaller for positive estimates. So this appears to be a bug. We looked at the code and didn't find what caused the bug, though.

I also see that the devel branch no longer has the KYCG. I wonder if the database groups will still be kept somewhere, since we can still test the DB groups with other methods, e.g. Limma's pre-ranked Camera method, which is quick because it doesn't do any permutations.

Thanks for all your work on Sesame!
JD

[jdreyf]

It looks like KYCG has moved to https://github.com/zhou-lab/knowYourCG.

I created the test myself, which can be put into the tests/testthat directory but currently fails:

test_that("testEnrichmentSEA works", {
df <- rowData(sesameDataGet('MM285.tissueSignature'))
query <- setNames(df$delta_beta, nm=df$Probe_ID)
suppressWarnings(res0 <- testEnrichmentSEA(query, "chromHMM", platform="MM285") |>
dplyr::arrange(dbname))
suppressWarnings(res1 <- testEnrichmentSEA(-1*query, "chromHMM", platform="MM285") |>
dplyr::arrange(dbname))
expect_equal(res0$estimate, -res1$estimate) # ok
expect_equal(res0$p.value, res1$p.value) # fails
})