Question about how FDR is calculated

[mooreryan]

Hey I have a couple questions about the way FDR is calculated....

Using rho statistic

The first involves rho: (see

propr/R/2-proprCall.R

Line 238 in b85112e

FDR[cut, "randcounts"] <- FDR[cut, "randcounts"] + sum(pkt > FDR[cut, "cutoff"])

for the relevant line).

For the FDR calculation for rho, shouldn't the code be counting any negative rho value that is below the cutoff as well as any positive rho that is above the cutoff?

E.g., something like rho of -0.7 would "pass" a cutoff of -0.5 but fail a cutoff of 0.5, and 0.7 rho value would "pass"' a cutoff of 0.5 but "fail" a cutoff of -0.5?

As it stands now, rho=0.7 would pass the cutoff for both 0.5 and -0.5. And rho=-0.7 would fail the cutoff for -0.5 as well as for 0.5.

Granted, negative proportionality can be weird to interpret, but for calculating FDR for rho properly, shouldn't the sign be taken into account as mentioned above?

Using phi statistic

My other question involves FDR calculation with the phi statistic.

First, the result that is being summed here

propr/R/2-proprCall.R

Line 240 in b85112e

FDR[cut, "randcounts"] <- FDR[cut, "randcounts"] + sum(pkt < FDR[cut, "cutoff"])

comes from the object@results$propr data here

propr/R/2-proprCall.R

Line 231 in b85112e

pkt <- pr.k@results$propr

Now, that data is one directional (eg 2 vs 1 is there but 1 vs 2 is not there). However, phi is non-symmetric and from data that I've looked at, phi(taxa1, taxa2) can be quite different than phi(taxa2, taxa1).

So I'm thinking that FDR calculations for phi should include t1 vs t2 and t2 vs t1 for all taxa pairs rather than just the one direction it calculates now.

What do you think?

[tpq]

Hello Ryan!, thanks for your challenging inquiry. I'll share my thoughts below...

For rho -- The way that I think about the FDR for rho is a one-sided test where we are interested in large positive values. (Part of this is because I've come to believe that negative proportionality can't be trusted.) As such, only big rho values are "positive" findings. I consider everything else a negative. Does this make sense?

For phi -- I agree with your reasoning. Though, I'd prefer to encourage analysts to use the symmetric version of phi ("phi_s" or "phs" instead). This is a good point though...I think I will add a warning here when using metric = "phi".

Happy to discuss further. It's essential that this procedure works correctly lest the scientific literature gets polluted with even more false positives!!

[mooreryan]

Only a year and a half later :) ... but yes I agree with both your points...negative proportionality is weird/confusing/often misleading, and phi_s is easier to interpret than phi.