tag:blogger.com,1999:blog-987850932434001559.post9099819490920201929..comments2024-11-11T20:33:55.028+01:00Comments on The 20% Statistician: Justify Your Alpha by Decreasing Alpha Levels as a Function of the Sample SizeDaniel Lakenshttp://www.blogger.com/profile/18143834258497875354noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-987850932434001559.post-17984756573008569432023-09-10T21:37:56.366+02:002023-09-10T21:37:56.366+02:00Thanks Daniel. What do I do when the sample size i...Thanks Daniel. What do I do when the sample size is 20,000 (as when using data from AP Votecast)?Anonymoushttps://www.blogger.com/profile/04768013683767362765noreply@blogger.comtag:blogger.com,1999:blog-987850932434001559.post-80821085515631438842021-09-09T19:13:04.024+02:002021-09-09T19:13:04.024+02:00This is excellent. I was wondering how to justify ...This is excellent. I was wondering how to justify a smaller alpha level in my analysis of data using 100000+ records. Thank you!Anonymoushttps://www.blogger.com/profile/18433171966396716090noreply@blogger.comtag:blogger.com,1999:blog-987850932434001559.post-47945549444676608522021-07-31T09:02:26.692+02:002021-07-31T09:02:26.692+02:00This comment has been removed by a blog administrator.Sajjad Ahmedhttps://www.blogger.com/profile/08974601470220195378noreply@blogger.comtag:blogger.com,1999:blog-987850932434001559.post-80025694200887734322019-05-14T20:28:46.107+02:002019-05-14T20:28:46.107+02:00Hi Daniel, how this standarization of P-values bas...Hi Daniel, how this standarization of P-values based on sample size can be coupled to the multiple-testing adjustment by Bonferroni or BH? Dodgerhttps://www.blogger.com/profile/00995569797695061139noreply@blogger.comtag:blogger.com,1999:blog-987850932434001559.post-46935573936434022722018-12-07T15:59:28.123+01:002018-12-07T15:59:28.123+01:00As the blog explains, this is about solving a prob...As the blog explains, this is about solving a problem with large N - so not intended to be used to increase the alpha for smaller N. Standardization for 100 is a pretty random choice - for these N's, there is no substantial mismatch yet, according to Good. He mentions it is just a useful thing to all use - but feel free to use another number. Or devise another scaling. Daniel Lakenshttps://www.blogger.com/profile/18143834258497875354noreply@blogger.comtag:blogger.com,1999:blog-987850932434001559.post-76644472876885543582018-12-05T21:56:48.744+01:002018-12-05T21:56:48.744+01:00Hi Daniel, I couldn't help thinking about your...Hi Daniel, I couldn't help thinking about your idea of scaling alpha by the square root of the sample size divided by the constant 100. I completely fail to understand you choice of constant, which obviously assigns a false positive rate higher than the traditional criterion of p < 0.05 to independent frequentist null hypothesis tests with a sample size below that arbitrary constant. Wouldn't you prefer an adaptive false positive rate that starts with the traditional criterion or any other initial probability and decreases with sample size, for example alpha = alpha/log(n) or alpha = alpha/n^(1/3) ?<br /><br />Best,<br />Martin DietzAnonymoushttps://www.blogger.com/profile/06915342196835755035noreply@blogger.com