I didn't know about Bem himself, but the replication crisis has hit the social sciences like a magnitude 9 earthquake. Here is this guy who says, "I have proven the impossible with standard, even boring, methodology", and says to everyone else, "Please, replicate my results!" And a decade later, this is still going on, turning brother against brother, with some field-fundamental results being disproven in replication. It's even spread a little bit outside social sciences, into the physical and biological sciences.
The problems summarize to:
* p-hacking: choosing your findings after the fact of the experiment to find an effect that looks statistically significant. Reminder that if you measure 20 random things, most likely one of them is above the 95% confidence level, because maths. Adding more data/subjects, and then stopping once you have the result you want also qualifies.
* no replication: there's no funding to replicate other people's results. Boring! It all goes to novel science. And no one gets publication or social credit for replication. So in theory science is self-correcting but most of the time no one bothers to do this step.
* data not shared: This is my data from my grant dollars, why should I let you make discoveries with it? Especially ones that discredit me! Special award goes to those making up their data, but this can also cover up honest mistakes made by researchers as well. Thankfully this is the easiest one to solve; journals, grant agencies, and universities have all made policies to address this.
The problems summarize to:
* p-hacking: choosing your findings after the fact of the experiment to find an effect that looks statistically significant. Reminder that if you measure 20 random things, most likely one of them is above the 95% confidence level, because maths. Adding more data/subjects, and then stopping once you have the result you want also qualifies.
* no replication: there's no funding to replicate other people's results. Boring! It all goes to novel science. And no one gets publication or social credit for replication. So in theory science is self-correcting but most of the time no one bothers to do this step.
* data not shared: This is my data from my grant dollars, why should I let you make discoveries with it? Especially ones that discredit me! Special award goes to those making up their data, but this can also cover up honest mistakes made by researchers as well. Thankfully this is the easiest one to solve; journals, grant agencies, and universities have all made policies to address this.
"Kitto daijoubu da yo." - Sakura Kinomoto