The Seven Deadly Selection Biases

Date: 

Monday, March 5, 2018, 1:00pm to 2:30pm

Location: 

Waterhouse Room, Gordon Hall, 1st. Fl., 25 Shattuck Street
REGISTER
This event is part of the Research, Rigor & Reproducibility Series
Speaker: Xiao-Li Meng, PhD, Dean, Graduate School of Arts and Sciences, and the Whipple V.N. Jone Professor of Statistics, Harvard University (on sabbatical 2017-2018 academic year) (full bio here)
Description: This talk provides a statistical perspective on the roles the seven S’s (sins?) play in increasing the amount of irreproducible research, in medical and life sciences and beyond: 
  1. Selections in hypotheses (e.g., subgroup analysis);
  2. Selections in data (e.g., deleting “outliers” or only using “complete cases”);
  3. Selections in methodologies (e.g., for goodness of fit);
  4. Selections in due diligence and debugging (e.g., triple checking only when the outcome seems undesirable);
  5. Selections in publication (e.g., only when p-value <0.05);
  6. Selections in reporting/summary (e.g., suppressing caveats);
  7. Selections in understanding and interpretation (e.g., our preference for deterministic, “common sense” interpretation).

The Big Data Paradox and Simpson’s Paradox will be used to demonstrate that the problem of irreproducible research is getting BIGGER with Big Data. A cocktail treatment approach together with a selfish/blowfish test is suggested to combat this problem.