A system for assessing the quality of scientific materials consisting in publishing articles submitted to a scientific journal so that anyone can participate in the review.
The traditional system is for an article to be evaluated by a recognized scientist in a given field while maintaining the requirements for anonymizing the author's personal data. The review is usually carried out by two recognized scientists in a given field, less often by only one. If a reviewed article is approved, it is published by the journal. This system of reviewing articles is intended to maintain the integrity and impartiality of the process of evaluating scientific work.
The practice of the traditional system was often criticized for various reasons, including the lack of transparency of the reviewing system, as well as the selectivity of reviewers selected by the publisher of a given journal, or the prolonged duration of a scientific review without a justified reason. Open scientific reviews were created in opposition to the traditional system of reviewing scientific materials.
The main features of an open review procedure are:
(according to the Foster Open Science) the following points:
- all the people involved in the review process, including e.g. external Internet users, know their personal details
- all review reports are available for viewing
- external users can participate in the review process at any stage
- preprints become generally available to users
- the final version of the article is still available for commenting
- discussion participants mutually encourage other people to work with each other
- a commonly available open source software is used for reviews
The journal "Atmospheric Chemistry and Physics" published by the European Geophysical Union is a repeatedly cited example of a journal operating in the Open Peer Review mode. The journal has developed a multi-stage open review process. A diagram of how this journal is reviewed is presented below. This is an example of a specific policy of open review - a real way of putting an idea into practice.
The explanation of the numbers from the above mentioned diagram is described below:
- Submitting the article for review
- Initial evaluation, e.g. whether the article is consistent with the subject of the journal
- Removal of technical and stylistic errors
- Open discussion of the article
- Responding to the discussion, authors' answers and creating the final version of the article
- Submitting the version of the article after the review
- Assessment whether the article requires additional consultations
- Publication or rejection of the article
The pros Open Scientific Reviews:
(according to the Nature Neuroscience America journal)
- greater restraint in judgment and refinement of reviews by professional reviewers
- possibility to identify a conflict of interest when performing reviews
- possibility to refer to the reviewer's comments and appeal against the decision to reject the article. No ethical allegation of "kangaroo court"
The cons of Open Scientific Reviews:
(according to the Nature Neuroscience America journal)
- the decision to publish an article is often made by journal editors rather than reviewers
- lack of secrecy can make it very difficult to find people willing to review in specialized fields of science, and much more difficult in science in general
- the phenomenon of nepotism when evaluating articles is not necessarily eliminated