Loading Events
This event has passed.

Devezer will present, “Theoretical Insights on Replication: A Miracle or a Crutch?” at 11:30 am

Lunch will be provided at 1:00 pm

Buzbas will present a follow-up, “Three Proposals for Pushing Forward the Theory of Results Reproducibility” at 2:00 pm

See details below!

Presentation Title: Theoretical Insights on Replication: A Miracle or a Crutch?

Time & Location: Friday, December 1st, 11:30am; WIRB 3000 (lunch to follow at 1:00pm)


Replication experiments purport to independently validate claims from previous research or provide some diagnostic evidence about their reliability. In practice, this value of replication experiments is often taken for granted. Our research shows that in replications, practice often does not live up to theory. Most replication experiments have a design different from the original rendering their results uninterpretable. These results can be driven by the true data generating mechanism, issues present in the original experiment, discrepancies between the original and the replication experiment new issues introduced in the replication experiment, or combinations of any of these factors. The answers we are looking for with regard to the true state of nature require a rigorous and meticulous investigative process of eliminating errors and singling out elementary or pure cases. In this talk, I will provide an overview of our theoretical approaches and results to date, building up toward a discussion of why most experiments are not replication-ready and what it would take for replication results to be clearly interpretable. I will chart an alternative path, where replication experiments play a critical yet a much more limited role in the scientific process than many scientists appear to envision today.


Berna Devezer isa metascience theorist and modeler, with background in behavioral experimentation and statistics. She has a broad interest in scientific theory, philosophy of science, and interdisciplinary metaresearch. Her current work is focused on theoretical and statistical aspects of metascience, use of statistical theory, mathematical modeling, and simulations to investigate:

  • How scientific processes of accumulating knowledge work
  • How sciences progress
  • What is the role of reproducibility of scientific findings in advancing knowledge.

Presentation Title: Three Proposals for Pushing Forward the Theory of Results Reproducibility

Time & Location: Friday, December 1st, 2:00pm; WIRB 3000


In this talk, we describe three fundamental theoretical problems to move forward in understanding results reproducibility in sciences. We believe that these problems recur or are on the horizon, and they constitute major challenges in understanding reproducibility in mathematical, philosophical, and applied contexts. We propose some ideas toward overcoming these problems to motivate discussion, and obtaining interdisciplinary input. The problems are as follows: 1) Experiments condition on background knowledge about the system being modeled. Background knowledge affects the modeling assumptions and the results of a study. There is no clear theory of how to capture background knowledge and it is challenging to pinpoint mathematically and philosophically. Classical probability experiments make their straightforward assumptions and move on to technical solutions. Applied scientists may not be aware of these assumptions, and often not equipped with tools to incorporate these assumptions in their studies. We propose the use of simulation studies to partially alleviate this problem. 2) What is a theoretically sound way to capture the elusive concept of results reproducibility? It does not seem to apply to different fields in the same way (e.g., physics vs. psychology). For a sound overarching theory valid to sciences across fields, we seem to be lacking an understanding about the nature of replication studies in different fields. We categorize replication studies into three types mathematically and philosophically, and discuss their aims, frequencies, and relevance in assessing results reproducibility. 3) Issues related to results reproducibility have been recognized by a few fields due to direct consequences of non-reproduced results on their fundamental theories. However, other fields in which the models, methods, and results do not pass rigorous scrutiny seem to be oblivious to these issues. Assuming that eventually a collective awareness of these issues will arise, a major problem is: How can we describe and measure the concept of results reproducibility across scientific fields. We define a novel probability distribution up to the task, and describe its properties.


Erkan Buzbas is a professor of Statistics at the University of Idaho, and a statistician and mathematical modeler.
His background is in stochastic modeling of population-level phenomena, and development of computational statistical methods to perform inference under uncertainty. He has a broad interest in stochastic modeling
of complex systems and statistical theory.

His current work is focused on:

  • Mathematical and statistical aspects of metascience. He uses  statistical theory, mathematical modeling,
    and simulations to investigate:

      • How scientific processes of accumulating knowledge work
      • How sciences progress
      • What is the role of reproducibility of scientific findings in advancing knowledge.


Upcoming Events

Event Videos

Most of our events are recorded & videos are posted onto our YouTube channel. Watch some of our most popular videos in this playlist.

Visit the Rotman Institute YouTube Channel

Past Events

Browse our past events sorted by philosophical area.

Conferences & Workshops

With outreach a central part of our mission, we host a number of public events.

Learn about our interdisciplinary panel discussions, public talks, and an annual lecture series co-sponsored with the London Public Library and the Department of Philosophy here at Western University.

View Rotman Public Events

Share This, Choose Your Platform!