The Filedrawer Problem: A Resource
Something brought to our attention by the Cheiron Forum (Cheiron = The International society for the History of the Behavioral & Social Sciences) –
The Open Science Framework – an open collaboration of scientists “to increase the alignment between scientific values and scientific practices” – recently announced its “Reproducibility Project,” a collaboration intended to estimate the reproducibility of a sample of studies from the psychological sciences. You can read about this project here.
A related resource which may interest h-madness readers is PsychFileDrawer, an archive of replication attempts in experimental psychology. The site explains its archive and the famous “file drawer problem” this way:
The “file drawer problem” (a term coined in 1979 by Robert Rosenthal, a member of our Advisory Board) refers to the bias introduced into the scientific literature by selective publication–chiefly by a tendency to publish positive results but not to publish negative or nonconfirmatory results. Awareness and concern about the file drawer problem seem to be growing explosively at the current time (early 2012). The pages below provide a fairly comprehensive list of recent discussions of this problem organized into different categories of publications–ranging from popular articles about the extent of the problem in many fields, to technical articles asking how failures to reject the null hypothesis should be analyzed and presented.