tag:blogger.com,1999:blog-987850932434001559.post5609641236484159250..comments2023-01-26T12:54:16.768+01:00Comments on The 20% Statistician: Justify Your Alpha by Minimizing or Balancing Error RatesDaniel Lakenshttp://www.blogger.com/profile/18143834258497875354noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-987850932434001559.post-1108881644459718352019-07-20T01:37:33.167+02:002019-07-20T01:37:33.167+02:00Hey Daniel, great post - thanks for sharing! I hav...Hey Daniel, great post - thanks for sharing! I have a couple suggestions for improvement and a question:<br />1) Thought you might like to know your first line of R-script for your function is missing double quotes.<br />res = optimal_alpha(power_function = [ADD_DOUBLE_QUOTES_HERE]pwr.t.test(d=0.5, n=100, sig.level = x, type='two.sample', alternative='two.sided')$power")<br /><br />2) For some reason, the balance function produces incorrect total error rates. For example, the following produces a res$tot = 8.888209e-08 but a res$alpha + res$beta = 0.9967886.<br />res = optimal_alpha(power_function = "pwr.t.test(d=0.001, n=30000, sig.level = x, type='two.sample', alternative='two.sided')$power", error = "balance")<br />res$alpha<br />res$beta<br />res$tot<br />res$beta + res$alpha<br /><br />3) You mention "If you collect large amounts of data, you should really consider lowering your alpha level." I'm not sure if I follow entirely. Assuming a sample size of 10000 where Cohen's d = 0.2, then adjusting the alpha from 0.5 to something smaller such as .0000000000000000005 has no impact on power, right? I'm probably missing something here, so I'd love to hear your thoughts.Anonymoushttps://www.blogger.com/profile/14576444348427501628noreply@blogger.com