Part of the Department of Psychology Colloquium Series on the Big Problems Being Solved with Psychology.
There has been a great deal of interest in the reproducibility of scientific results in recent years. However, results can be highly reproducible but wrong (J-shaped curves in epidemiology, in particular – sadly a little bit of alcohol probably isn’t good for you). We therefore need more than replication if we want to know whether our inferences are correct – we need triangulation. In other words, we need to approach the same question with different methods that have different biases etc. Various causal inference methods exist in epidemiology – genetically informed methods, negative controls, cross contextual comparisons etc. – but they are often not applied systematically together. In addition, to avoid triangulation becoming an exercise in cherry picking we need to prospectively register our triangulation plans, and think carefully about what results would support specific inferences. More generally, we need to think about how we can avoid researcher biases shaping the results we obtain, particularly when we’re working with secondary data rather than collecting primary data.