The replication crisis in economics – and how we might answer it

Posted: September 21st, 2017 | Author: | Filed under: Conference, Workshop | Tags: , | No Comments »

On September 8, 2017 the ZBW Leibniz Information Center for Economics hosted the workshop “Replications in Empirical Economics – Ways out of the Crisis” at the Annual Conference of the Verein für Socialpolitik in Vienna, Austria. Thirty participants and four speakers engaged in lively and stimulating discussions about replications and the publication of replications in Economics.

This is a cross-post of the weblog of the replication network. Read the rest of this entry »


Against the replication crisis: New international journal encourages replication studies

Posted: August 1st, 2017 | Author: | Filed under: Data Sharing, journals, Projects | Tags: , , | Comments Off on Against the replication crisis: New international journal encourages replication studies
 Replications are pivotal for the credibility of empirical economics. Only findings which are robust and replicable can be generalized and can serve as evidence based advice to economic policy. But, already in 1983 Edward Leamer stated (p. 37):

Read the rest of this entry »


New Working Paper published: ‘The Economics of Replication’

Posted: February 14th, 2017 | Author: | Filed under: Data Sharing, EDaWaX, journals | Tags: , | Comments Off on New Working Paper published: ‘The Economics of Replication’

Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff and Gert G. Wagner have published a new working paper on the ‘economics of replication’.

In the abstract of the paper, the researchers state:

“Replication studies are considered a hallmark of good scientific practice. Yet they are treated among researchers as an ideal to be professed but not practiced. To provide incentives and favorable boundary conditions for replication practice, the main stakeholders need to be aware of what drives replication. Here we investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that from 1974 to 2014 less than 0.1% of publications in the top-50 economics journals were replications. We do not find empirical support that mandatory data disclosure policies or the availability of data or code have a significant effect on the incidence of replication. The mere provision of data repositories may be ineffective, unless accompanied by appropriate incentives. However, we find that higher-impact articles and articles by authors from leading institutions are more likely to be subject of published replication studies whereas the replication probability is lower for articles published in higher-ranked journals.”

The paper is available here.

Read the rest of this entry »


Dutch research funder grants 3 million Euros for replication studies

Posted: August 2nd, 2016 | Author: | Filed under: Data Sharing, found on the net | Tags: , | Comments Off on Dutch research funder grants 3 million Euros for replication studies

350_incentive_6912846727_48f07c8899_zThe Netherlands Organisation for Scientific Research (NWO), one of the biggest science-funding bodies in the Netherlands, is making 3 million euros available for a Replication Studies pilot programme. In this programme, scientists will be able to repeat research that has been carried out by others. The pilot focuses on replicating studies that have a large impact on science, government policy or the public debate – namely on the area of the social sciences and health research and healthcare innovation.

The pilot programme Replication Studies focuses on two types of research: On the one hand it focusses on replications with existing data: the datasets from the original study are reanalysed. On the other hand, it focusses on reproductions (with new data): a data collection is put together, which is subjected to the same research protocol as in the original study (read more about the different definitions of reproduction and replication). Read the rest of this entry »


New Working Paper: “Perceptions and Practices of Replication by Social and Behavioral Scientists”

Posted: April 21st, 2016 | Author: | Filed under: Data Sharing, Report | Tags: , , | Comments Off on New Working Paper: “Perceptions and Practices of Replication by Social and Behavioral Scientists”

300 CoverOne of our project partners has just released a publication that deals with the replication crises in economics and the social sciences.

In the abstract the three autors state:

We live in a time of increasing publication rates and specialization of scientific disciplines. More and more, the research community is facing the challenge of assuring the quality of research and maintaining trust in the scientific enterprise. Replication studies are necessary to detect erroneous research. Thus, the replicability of research is considered a hallmark of good scientific practice and it has lately become a key concern for research communities and science policy makers alike.

In their discussion paper Fecher, Fräßdorf and Wagner analyze perceptions and practices regarding replication studies in the social and behavioral sciences. Their analyses are based on a survey of almost 300 researchers.

Read the rest of this entry »


Note on “research parasites” published in SCIENCE

Posted: April 5th, 2016 | Author: | Filed under: Data Sharing, EDaWaX, Opinion | Tags: | Comments Off on Note on “research parasites” published in SCIENCE

300_ Emilio Quintana_share-computer-key-260_flickr_comBenedikt Fecher and Gert G. Wagner, both also affiliated with the EDaWaX-project, have just published a note in a recent issue of sience under the title “A research symbiont”.

The background for their note is the editorial of D. Longo and J. Drazen – two editors of the New England Journal of Medicine – published in January 2016. In the editorial, Longo and Drazen critically assessed the concept of data sharing in medicine. Their main concern is that a “new class of research person will emerge” that uses data, which were gathered by other researchers, for their own original research questions. The authors, although indirectly, later referred to this class of researcher as “research parasites”.

Fecher and Wagner argue contrarily: For them, the two editors of the New England Journal of Medicine miss the core of the scientific paradigm when writing that researchers may “even use the [open] data to try to disprove what the original investigators had posited.” Instead, Fecher and Wagner argue that data users are research symbionts and benefit the research ecosystem: Using research data to try to disprove a result is good scientific practice, especially in light of the replication crisis (not only) in the social sciences.

A longer version of the Fecher’s and Wagner’s note is available open access (working paper).

Picture: “share-computer-key-260″ by Emilio Quintana on flickr.com. License: CC BY-NC-SA 2.0


Contemporary, useful and subject-based: The replication network

Posted: October 23rd, 2015 | Author: | Filed under: found on the net | Tags: , | Comments Off on Contemporary, useful and subject-based: The replication network

3d social network_by_stockmonkeys.com_cc_by_3.0

Today I would like to introduce the replication network (TRN) to our readers, a network whose purpose is “to encourage economists and their journals to publish replications.” This is all along in line with the purpose of our own project.

The website of the replication network serves as a channel of communication to both update scholars about the state of replications in economics and to establish a network for the sharing  of information and ideas among economists. It offers important information on the possibility to publish replication studies in economics journals and provides lists of publications dealing with the topic of replications in economic research. Also a list of published replication studies is available. Read the rest of this entry »


Goettingen University: Project fosters Replication in Economics

Posted: June 3rd, 2014 | Author: | Filed under: Projects | Tags: , | 1 Comment »

300_med_miracleThere are many good reasons why we should replicate scientific findings. In his article “Open Access Economics Journals and the Market for Reproducible Economic Research“, the economist B.D. McCullough (2009) lists some of the reasons why replicable research is crucial for science:

„[…]replication ensures that the method used to produce the results is known. Whether the results are correct or not is another matter, but unless everyone knows how the results were produced, their correctness cannot be assessed. Replicable research is subject to the scientific principle of verification; non-replicable research cannot be verified. Second, and more importantly, replicable research speeds scientific progress. We are all familiar with Newton’s quote, ‘If I have seen a little further it is by standing on the shoulders of giants.’ […] Third, researchers will have an incentive to avoid sloppiness. […] Fourth, the incidence of fraud will decrease.“ (p.118)

More recently, the case of the US-economists Rogoff and Reinhart clearly illustrated the need for replications to be much more common in science and scientific education. But… Read the rest of this entry »