New Paper asks: “Do journals journals enforce their data policies?”

I am happy to announce that our paper “Journals in Economic Sciences: Paying Lip Service to Reproducible Research?” finally has been released.

The article, published in the latest issue of the IASSIST Quarterly, regards the practice of 37 journals in economics and business administration when it comes to the enforcement of their data policies.

For this purpose, we checked 599 articles which have been published in two issues of these journals. We chose the issues 1/2013 and 1/2014.

First, we estimated the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we checked the journal data archives and supplemental information section of each of these articles for the availability of replication files.

For a reduced sub-sample of 245 data-based articles, we checked in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies and to calculate the ‘compliance rate’ for each journal in our sample. Read the rest of this entry »

Statistical software: its use and popularity in Economics

by Christina Kläre & Timo Borst

During a four weeks project at ZBW’s Department for Information Systems and Publishing Technologies, we collected some publicly available information about statistical software packages being used in research in Economics. This work is inspired by a constantly updated blog post from Robert A. Muenchen, who examined information sources like job announcements, scientific articles, reports from IT companies, questionnaires, sales statistics from software textbooks, blogposts, forums, polls measuring popularity of programming languages, sales and download figures, or the frequency of software releases from software vendors. By means of these sources, we conducted the following data collections. Read the rest of this entry »

German Research Foundation (DFG) publishes Statement on Replicability

The German Research Foundation (DFG) has currently released a statement on the replicability of research results.

Interestingly (at least for me), the five-pager first starts with a broader definition of what replicable research is NOT.

Of course, replication is a very important method for testing empirical knowledge claims based on experimental and quantitative research in medicine, the natural, life, engineering, social and behavioural sciences, as well as the humanities.

But, according to DFG, there are limitations:

• Replicability is not a universal criterion for scientific knowledge.
• Ascertaining the replicability or non-replicability of a scientific result is itself a scientific result. As such, it is not final but subject to methodological scepticism and further investigation.
• Non-replicability is not a universal proof by falsification.
• Non-replicability is not a universal indicator of poor science.

‘Well, an unorthodox starting point for a paper on reproducible research‘ – so, at least, were my thoughts when I read the first page of the statement. Wouldn’t it be more common to first depict the important aspects of reproducible research and to suggest measures to support it, instead of rowing back at the beginning of such a statement? Read the rest of this entry »

Managing research data using the ZBW Journal Data Archive

Yesterday, a new blog post has been published on ZBW Mediatalk in which I describe the history and the background of the EDaWaX-project that led to the development of the ZBW Journal Data Archive. The Journal Data Archive (JDA) is a service for editorial offices of journals in economics and partially also in the social sciences. Currently this service is free of charge and already two renowned journals are using our services productively.

The  JDA offers direct linking between published research papers and the underlying data, that has been used to generate the findings of an article. The workflow is time-saving and easy to handle: Authors of a journal’s article submit their replication files to the system and describe these files with additional metadata. For us it was quite important to find a useful balance between the efforts a user has to invest to generate these additional metadata and the needs of our information systems. For the development of our metadata schema, we followed the motto ‘as much as necessary, as little as possible.’ Read the rest of this entry »

New Guest Blog for TRN

In a Guest Blog for the Replication Network, I wrote a little bit about the history of the EDaWaX project and listed some of the project’s outputs. In addition -and possibly even more important- I also outlined the current and future developments: In particular I dwelled on the question what remains after the project completed its second funding term in autumn.

One answer is the ZBW Journal Data Archive. The technological core of the project has become part of ZBW’s infrastructure. Currently, we are still in the process of enhancing technical functionalities and layout of the application. In the next weeks I will report on detail on the application. Read the rest of this entry »

ZBW Journal Data Archive – Deposit License (EN)

[Download the Deposit License in pdf-format]

I. Description
With the ZBW Journal Data Archive ZBW provides a technical platform for the storage, documentation and dissemination of publication-related research data (hereafter “digital objects”) from articles published in social sciences’ and economics’ journals.
The primary objective of this service is to support the replicability of published economic research and to promote the sharing and accessibility of these data and related materials (such as e.g. the documentation of the data and corresponding program codes).
For this purpose, authors are invited to submit the replication files of their article to the ZBW Journal Data Archive and to describe them with structured metadata. Prior to be able to submit such digital objects to the platform, authors have to be registered by a participating journal. After completing a submission, these digital objects receive a persistent identifier (DOI) in accordance with the requirements of the particular journal. Thereby, the replication files get uniquely referenceable and quotable.
ZBW supports the freest possible access to publicly funded research data. Read the rest of this entry »

EDaWaX: Final spurt

Slowly but steady, the EDaWaX project draws to a close. For me a good point in time to outline some of the final steps of our project.

First of all, we are very much engaged in transforming EDaWaX from a project to a regular service for scholarly journals in economic sciences. Therefore we still have a lot of work to do: There are still some issues to solve on the level of software engineering. Currently we are also in the process of improving some of the application’s features and to update the documentation. Nevertheless the software for journals in economics and business research already is available. Starting in the next months we will offer our service (a publication-related data archive) free of charge for editorial offices.

Second, we will present some rescent results of the project at national and international conferences, in roadshows and on annual meetings of learned societies in economics and business studies.

This program starts in two weeks in Munich, where we will disucss the topic of managing research data with the editors of journals in business research and present our software solution at the annual meeting of the German Association for Business Research (VHB), the German learned society for business studies. Read the rest of this entry »

Note on “research parasites” published in SCIENCE

Benedikt Fecher and Gert G. Wagner, both also affiliated with the EDaWaX-project, have just published a note in a recent issue of sience under the title “A research symbiont”.

The background for their note is the editorial of D. Longo and J. Drazen – two editors of the New England Journal of Medicine – published in January 2016. In the editorial, Longo and Drazen critically assessed the concept of data sharing in medicine. Their main concern is that a “new class of research person will emerge” that uses data, which were gathered by other researchers, for their own original research questions. The authors, although indirectly, later referred to this class of researcher as “research parasites”.

Fecher and Wagner argue contrarily: For them, the two editors of the New England Journal of Medicine miss the core of the scientific paradigm when writing that researchers may “even use the [open] data to try to disprove what the original investigators had posited.” Instead, Fecher and Wagner argue that data users are research symbionts and benefit the research ecosystem: Using research data to try to disprove a result is good scientific practice, especially in light of the replication crisis (not only) in the social sciences.

A longer version of the Fecher’s and Wagner’s note is available open access (working paper).

Picture: “share-computer-key-260″ by Emilio Quintana on flickr.com. License: CC BY-NC-SA 2.0

User Manual: ZBW Journal Data Archive (English Version)

Dear Author,

First of all, many thanks for using our application! Within this document you will find useful notes for handling our application. In case of questions or ambiguities, this documentation is intended you support you in the process of uploading your replication files. In addition, we also provide guidance on creating metadata for these resources, which is an important task: By providing us with rich metadata, it is quite simple to add these datasets to your personal list of your scientific outputs. Also your replication files can be added to the scientific output of your institution.

We will successively expand the documentation and also add some short ‚best-practice‘ examples for ‚typical‘ submissions to economics journals.

Furthermore, we are continually enhancing the application. In particular, we aim to significantly reduce the efforts for describing each file with additional metadata.

If there are additional questions, which are not covered by this documentation, please feel free to contact the product manager of the ZBW Journal Data Archive, Mr. Sven Vlaeminck (E-Mail: journaldata [at] zbw [dot] eu; Phone: +49(0)40-428 344 15).

Please note: the most recent documentation is available on the webpage of the JDA .

Below, you will find an outdated version, but I leave it here for purposes of documentation.

[gview file="http://www.edawax.de/wp-content/uploads/2017/02/User-Manual-EN-02-02-2017-V1.2.pdf"]

Merry Christmas – and a happy new year!

The year draws to a close – therefore the EDaWaX-project team wants to thank our partners, funders and readers for a successful year!

Again, 2015 has been a year full of thrilling events and developments within our project. This year’s focal point primarily was on the advancement of our CKAN-based application. In the first quarter of 2016 we will present more information on the future of the EDaWaX-application and in which direction the project will move after the end of the funding period in May 2016.

Since the start of the second funding period, we implemented a lot of new functionalities. For instance, it is possible to register DataCite-DOIs (digital object identifiers) for submitted research data directly from the application’s frontend. We also slightly improved CKAN’s user and right management to make it more suitable for an utilisation in the context of journal’s data archives. One result is that authors and editorial offices now receive e-mail notifications on recent activities within the data archive.

But we also changed some things based on feedback from the community: For example, we completely overhauled the metadata fields and the web frontend for uploading data. Now it is much catchier for researchers to submit their data to our application.

Read the rest of this entry »

New working paper: “Is Economics Research Replicable?”

Andrew Chang and Phillip Li, two researchers working at the Board of Governors of the Federal Reserve System and at the Office of the Comptroller of the Currency/U.S. Department of the Treasury, attempt to replicate 67 papers published in 13 well-regarded economics journals (American Economic Journal: Economic Policy, American Economic Journal: Macroeconomics, American Economic Review, American Economic Review: Papers and Proceedings (P&P), Canadian Journal of Economics, Econometrica, Economic Journal, Journal of Applied Econometrics, Journal of Political Economy, Review of Economic Dynamics, Review of Economic Studies, Review of Economics and Statistics, and Quarterly Journal of Economics), using author-provided replication files that include both data and code.

Some journals in the sample of Chang and Li require data and code replication files, and other journals do not require such files. Aside from 6 papers that use confidential data, they obtain data and code replication files for at least 29 of 35 papers (83%) that are required to provide such files as a condition of publication, compared to 11 of 26 papers (42%) that are not required to provide data and code replication files. Read the rest of this entry »

EDaWaX: New research paper published

We are happpy to announce that our new peer-reviewed research paper “Data Policies and Data Archives: A New Paradigm for Academic Publishing in Economic Sciences?” has just been published in the Conference Proceedings of the 19th International Conference on Electronic Publishing (ELPUB15). The paper summarises some results of a broader analyses on data policies of journals in economics and business studies.

In our paper we summarise the findings of an empirical study in which a sample of 346 journals in economics and business studies were examined. We regard both the extent and the quality of journals’ data policies, which should facilitate replications of published empirical research. The paper presents some characteristics of journals equipped with data policies and gives some recommendations for suitable data policies in economics and business sciences journals. In addition, we also evaluate the journals’ data archives to roughly estimate whether these journals really enforce data availability. Our key finding is that we are currently not able to determine a new publishing paradigm for journals in economic sciences.

I would like to thank the reviewers and the organisers of the conference for their useful comments! The full paper is available here.

Read the rest of this entry »

Busy months before summer holidays

It is vacation season and like many others, also some of the people working for the EDaWaX-project enjoy their well-earned summer holidays. Before I also leave the office, I would like to jump on the chance to report on the progress of our project for the last 6 months or so.

It has been a quite busy time for most of us and we achieved progress in many of our work packages. The main focus has been on advancing the technical work packages, but we were also able to finalise two work packages of the project’s analyses phase.

Let me go into detail: Read the rest of this entry »

Article: The Economics of Reproducibility in Preclinical Research

In 2012, an estimated 114.8 billion $ in the US were spent on life sciences research. Roughly half of it is spent on preclinical research, with government sources providing the majority of funding – approximatly 38 billion US$.

Now, three researchers calculate the costs of irreproducible research in preclinical research near 28 billion $ – only for the United States alone. That is the conclusion of a study published in PLoS Biology a few days ago.

In the opinion of the study’s authors, the giant amount of 28 billion $ accumulates, because low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development.

Read the rest of this entry »

German Council of Science and Humanities publishes Recommendations on Scientific Integrity

Yesterday, the German council of Science and Humanities (Wissenschaftsrat) published a 54-page position paper on scientific integrity (unfortunately in German only). In a press release the scientific body stated:

“Prosperity and development of modern societies today depend to a large extent on the quality and progress of scientific knowledge. Therefore it is important that the public can trust in the ethical and quality standards of the scientific community. Cases of fraud and misconduct – particularly major fraud and plagiarism scandals – undermine that trust. In response to such cases of fraud and loss of confidence many scientific organisations have published rules and guidelines dedicated to a Good Scientific Practice in the last few years. The German council of Science and Humanities now has adopted a position paper and recommendations on scientific integrity.” Read the rest of this entry »