Journal content

Journal of Internet Medical Research | Rigor

picture: Source: freepik; Copyright: pch.vector; License: under JMIR license
see After

Credit: under JMIR license

JMIR Publications recently published Establishing institutional scores with the Rigor and Transparency Index: a large-scale analysis of the quality of scientific reports in the Journal of Internet Medical Research (JMIR), who indicated that improving stringency and transparency measures should lead to improvements in reproducibility in the scientific literature, but evaluating transparency measures tends to be very difficult if done manually by examiners.

Video interview with the authors of this article:

The overall objective of this study is to establish a measure of the quality of scientific reporting that can be used across institutions and countries, as well as to highlight the need for high quality reporting to ensure reproducibility within biomedicine. , using manuscripts of the reproducibility project: cancer biology.

The authors discuss an improvement to the previously introduced version Rigor and Transparency Index (RTI), which attempts to automatically assess the rigor and transparency of journals, institutions, and countries using manuscripts graded on criteria found in reproducibility guidelines (e.g., NIH, MDAR, TO ARRIVE).

Use the work of the reproducibility project: cancer biologythe authors were able to determine that the replication studies scored significantly higher than the original papers, which the project said all required additional information from the authors to begin replication efforts.

Unfortunately, the RTI metrics for journals, institutions, and countries all currently score below the replication study average. If they take the RTI from these replication studies as a target for future manuscripts, further work will be needed to ensure that the average manuscript contains enough information for replication attempts.

Dr. Anita Bandrowski of University of California San Diego said, “Reproducibility in research is necessary for scientific progress. However, over the past decade numerous reports of irreproducible research have highlighted a persistent problem, which is proving both troublesome and costly. “

“Reproducibility in research is necessary for scientific progress. However, over the past decade numerous reports of irreproducible research have highlighted a persistent problem, which is proving both troublesome and costly. “

To encourage reproducibility, many organizations and scientific journals have adopted the Promoting Transparency and Openness guidelines, which focus on establishing best practices at the level of individual journals.

Along the same lines, the publisher Material design, analysis and reporting is a multidisciplinary research framework designed to improve the transparency of reporting on life science research at the level of individual manuscripts.

This framework provides a consistent, minimal checklist whose criteria were used, in part, to create the first RTI, a measure of journal quality focused on research methodologies and reporting transparency.

Specifically, the authors present here the latest version of the RTI, which represents the average SciScore on a subset of articles, and demonstrate how it can be used to assess transparency of reporting within research institutions.

While we cannot simply describe all items scoring a “2” as non-reproducible and all items scoring an “8” as reproducible, as many domains and their subsequent best practices exist, we can assert that higher scores are associated with more methodological details and as these are probably easier to use to attempt replication.



Full Text
Free Altmetric Report

JMIR Publications is a leading, born-digital, open-access publisher of more than 30 scholarly journals and other innovative science communication products that focus on the intersection of health and technology. Its flagship newspaper, the Journal of Internet Medical Researchis the world’s leading digital health journal in terms of content and visibility, and it is the largest journal in the field of medical informatics.

To learn more about JMIR publications, please visit or contact us via:






The head office – 130 Queens Quay East, Unit 1100 Toronto, ON, M5A 0P6 Canada

If you would like to learn more about promotional opportunities, please contact us at [email protected]

The content of this communication is licensed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided that the original work, published by JMIR Publications, is properly cited. JMIR Publications is a registered trademark of JMIR Publications.

About SciScore

SciScore is a scientific content checker/validation tool that checks common rigor criteria (NIH, MDAR, ARRIVE) and research resources (antibodies, cell lines, organisms). These guidelines can be checked by the editorial staff, but the process is tedious and requires a great deal of effort from a trained professional, so the checklists are applied only in the best-resourced journals. SciScore uses text mining techniques to get the job done in minutes, providing a report to editors, reviewers or authors on which criteria has and has not been covered. Additionally, it provides a numerical score, which allows publishers to assess the percentage of criteria met or not met at a glance.

Contact the researchers: Anita Bandrowski | [email protected]

Contact media/publishers: Martijn Roelandse | [email protected]

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.