A new preprint was recently published by Thomas Klebel and Vincent Traag on the topic of causality in science studies. Partially funded by TIER2, it hopes to help more researchers in science studies consider causality explicitly. The publication focuses on structural causal models, which have a convenient graphical representation, allowing researchers to make their causal assumptions and findings transparent, thereby fostering further discussion.

To make the main concepts more accessible, the authors provide three case studies based on a simulated model of Open Science. One case study illustrated particularly well how a lack of causal thinking can lead to incorrect interpretations. When looking at research published in journals, the authors observe that articles with open data are less likely to be reproducible which is in contrast with the known positive effect of open data on reproducibility. The explanation is that journals are more likely to publish research that has open data, but also that is more rigorous. If published research has no open data, it tends to be more rigorous, otherwise it would not be published at all. Research that is more rigorous tends to be more reproducible, and in the analysed case, this effect is stronger than open data. So, published research without open data is more rigorous which in turn is more reproducible. The key insight here is that the negative observed association does not reflect the positive causal effect of open data on reproducibility. Therefore, drawing the conclusion that open data decreases reproducibility would be completely wrong and could lead to further negative impacts, especially if policy recommendations are based on it.

Read the full preprint to get insight on this and the other case studies here or find it in TIER2’s library. This preprint is part of the PathOS project, and will also form an introductory chapter to an Open Science Indicator Handbook.