The replication crisis in economics – and how we might answer it

On September 8, 2017 the ZBW Leibniz Information Center for Economics hosted the workshop “Replications in Empirical Economics – Ways out of the Crisis” at the Annual Conference of the Verein für Socialpolitik in Vienna, Austria. Thirty participants and four speakers engaged in lively and stimulating discussions about replications and the publication of replications in Economics.

This is a cross-post of the weblog of the replication network. Read the rest of this entry »

Against the replication crisis: New international journal encourages replication studies

Read the rest of this entry »

Statistical software: its use and popularity in Economics

by Christina Kläre & Timo Borst

During a four weeks project at ZBW’s Department for Information Systems and Publishing Technologies, we collected some publicly available information about statistical software packages being used in research in Economics. This work is inspired by a constantly updated blog post from Robert A. Muenchen, who examined information sources like job announcements, scientific articles, reports from IT companies, questionnaires, sales statistics from software textbooks, blogposts, forums, polls measuring popularity of programming languages, sales and download figures, or the frequency of software releases from software vendors. By means of these sources, we conducted the following data collections. Read the rest of this entry »

German Research Foundation (DFG) publishes Statement on Replicability

The German Research Foundation (DFG) has currently released a statement on the replicability of research results.

Interestingly (at least for me), the five-pager first starts with a broader definition of what replicable research is NOT.

Of course, replication is a very important method for testing empirical knowledge claims based on experimental and quantitative research in medicine, the natural, life, engineering, social and behavioural sciences, as well as the humanities.

But, according to DFG, there are limitations:

• Replicability is not a universal criterion for scientific knowledge.
• Ascertaining the replicability or non-replicability of a scientific result is itself a scientific result. As such, it is not final but subject to methodological scepticism and further investigation.
• Non-replicability is not a universal proof by falsification.
• Non-replicability is not a universal indicator of poor science.

‘Well, an unorthodox starting point for a paper on reproducible research‘ – so, at least, were my thoughts when I read the first page of the statement. Wouldn’t it be more common to first depict the important aspects of reproducible research and to suggest measures to support it, instead of rowing back at the beginning of such a statement? Read the rest of this entry »

Note on “research parasites” published in SCIENCE

Benedikt Fecher and Gert G. Wagner, both also affiliated with the EDaWaX-project, have just published a note in a recent issue of sience under the title “A research symbiont”.

The background for their note is the editorial of D. Longo and J. Drazen – two editors of the New England Journal of Medicine – published in January 2016. In the editorial, Longo and Drazen critically assessed the concept of data sharing in medicine. Their main concern is that a “new class of research person will emerge” that uses data, which were gathered by other researchers, for their own original research questions. The authors, although indirectly, later referred to this class of researcher as “research parasites”.

Fecher and Wagner argue contrarily: For them, the two editors of the New England Journal of Medicine miss the core of the scientific paradigm when writing that researchers may “even use the [open] data to try to disprove what the original investigators had posited.” Instead, Fecher and Wagner argue that data users are research symbionts and benefit the research ecosystem: Using research data to try to disprove a result is good scientific practice, especially in light of the replication crisis (not only) in the social sciences.

A longer version of the Fecher’s and Wagner’s note is available open access (working paper).

Picture: “share-computer-key-260″ by Emilio Quintana on flickr.com. License: CC BY-NC-SA 2.0