Have you heard about the replication crisis? The term generally refers to the current situation in psychology – but also other research domains – that most findings cannot be replicated. This means that if the exact same study is done again, different results occur. How can that be?
The Psychology Domain
You may be familiar with studies but if you are not, let me tell you they are not perfect. Most studies conducted do not rely on a representative sample of subjects, most often the subjects are students who have to participate in studies in order to get required credits to finish their degree.
The theoretical process of research should be that a study is done and redone a couple of times before it is accepted as proven. The problem is that researchers are not particularly interested in redoing studies that others already did and even if they do, they might not get published. The reason is the “statistical significance”.
In Psychology, something is “statistically significant” when the results are to 95 % not random. Not statistically significant results very rarely get published. At first, this seems logical: If you didn’t find anything, there is nothing to be published. But in reality, it can be that you redid a study and found insignificant results, disproving the original study. Those results are also very unlikely to get published.
If you didn’t get that last part, don’t worry. John Oliver from Last Week Tonight did the hard work for me and explains in a hilarious way how the science publication process works and what its flaws are:
To sum up, the pressure to publicize as many papers as possible with as many new findings as possible deters many researchers from doing replication studies. At the same time, getting a representative sample for every study is too costly, which also weakens the external validity of the study, no matter how good the design is.