SourceCodePeerReview: Difference between revisions

Revision as of 11:12, 26 May 2022

Peer Review for Model Codes

For model codes, there are at least three general (and not mutually exclusive) types of peer review:

review of the model code itself by a journal or scholarly organization
review of an article that describes the concepts and algorithms used by a numerical model
ongoing review of code as an intrinsic part of the development process (for team- and community-based codes that involve multiple developers and contributors).

1. Software Journals and Peer-Reviewed Community Code Libraries

There is a growing list of journals that review and publish research software, either as their sole mission or as one possible category of contribution. For example:

The Journal of Open Source Software (JOSS), which describes itself as “a developer friendly, open access journal for research software packages.”

JOSS software publications are fully citable objects with a digital object identifier (DOI). JOSS is a good choice when you want to publish a version of the software itself, as opposed to a written description of the concepts or algorithms underlying it.

Several professional and scholarly organizations offer reviews of software that is included in an open-source community collection. For example:

ROpenSci (R packages)
PyOpenSci (Python packages)
CoMSESnet (Agent-based models)

Publications that Describe Software, Models, and Algorithms

Several geoscience journals specialize in papers that describe research software, algorithms, and other computational research tools and concepts. Examples include:

Geoscientific Model Development
Computers and Geosciences
Environmental Modelling and Software.

These journals tend to focus on an article as the primary reviewed product, but they welcome articles that describe particular software packages.

Ongoing Peer Review during Development

Code review is a powerful quality-control tool, and for that reason review tools are now routinely built into collaborative code-development platforms like GitHub. These days, it is common for research software teams to use peer review as a regular part of the software development process. For example, at the CSDMS Integration Facility, additions and changes to software tools are initially developed on a separate branch, and that branch is only merged into the main code base once it has been reviewed and approved by at least one other person. The same goes for community contributions to CSDMS tools. (In fact, reviewing code pull requests is a great way to contribute to CSDMS!). Many of the larger, team-built codes used in geoscience benefit from some type of ongoing internal peer review. Unlike software papers, this ongoing review process does not by itself result in a citable product, and in that sense the labor may be invisible to the academic credit system. Nonetheless, we highly recommend using a review process for your own code, even if it is just a matter of asking a friend or colleague to take a look and give feedback. One advantage is that the road to formal publication in a software-oriented journal is likely to be easier if you code has already passed through an informal peer review process during development.

@@ Line 1: / Line 1: @@
 '''Peer Review for Model Codes'''
+For model codes, there are at least three general (and not mutually exclusive) types of peer review:
+# review of the model code itself by a journal or scholarly organization
+# review of an article that describes the concepts and algorithms used by a numerical model
+# ongoing review of code as an intrinsic part of the development process (for team- and community-based codes that involve multiple developers and contributors).
-You have worked hard to develop your model code, and you have generously shared it with the world through the CSDMS Model Repository. Here are some ways you can be properly credited for your important contributions to research software.
-A new rocket design explodes shortly after takeoff. A Mars orbiter misses its target and burns up in the atmosphere. A series of papers in computational biochemistry are suddenly retracted. What do these events have in common? Bugs in scientific software.
+'''1. Software Journals and Peer-Reviewed Community Code Libraries'''
-Software today is fundamental to nearly all science. Concerns about quality, reproducibility, and efficiency have prompted calls for greater transparency and reliability. At the same time, there has been growing awareness that software is a vital and first-class research product that deserves recognition and credit. One way to promote both quality control and proper attribution is the same mechanism we rely on for scientific results generally: peer review.
+There is a growing list of journals that review and publish research software, either as their sole mission or as one possible category of contribution. For example:
+*  The Journal of Open Source Software (JOSS), which describes itself as “a developer friendly, open access journal for research software packages.”
+JOSS software publications are fully citable objects with a digital object identifier (DOI). JOSS is a good choice when you want to publish a version of the software itself, as opposed to a written description of the concepts or algorithms underlying it.
-For model codes, there are at least three general (and not mutually exclusive) types of peer review: review of the model code itself by a journal or scholarly organization, review of an article that describes the concepts and algorithms used by a numerical model, and (for team- and community-based codes that involve multiple developers and contributors), ongoing review of code as an intrinsic part of the development process.
+Several professional and scholarly organizations offer reviews of software that is included in an open-source community collection. For example:
+* ROpenSci (R packages)
+* PyOpenSci (Python packages)
-'''Software Journals and Peer-Reviewed Community Libraries'''
+* CoMSESnet (Agent-based models)
-There is a growing list of journals that review and publish research software, either as their sole mission or as one possible category of contribution. One example is the Journal of Open Source Software (JOSS), which describes itself as “a developer friendly, open access journal for research software packages.” JOSS reviewers are asked to evaluate software packages based on several criteria, including demonstrated need for the software, the level of scholarly effort required to produce it, documentation, installation, and functionality. A JOSS publication includes a brief description of the software and why it is needed, alongside a version of the code. JOSS software publications are fully citable objects with a digital object identifier (DOI). JOSS is a good choice when you want to publish a version of the software itself, as opposed to a written description of the concepts or algorithms underlying it.
-Several professional and scholarly organizations offer reviews of software that is included in an open-source community collection. For example, the organizations ROpenSci and PyOpenSci provide curated suites of open-source packages in R and Python, respectively. In order to be included in these collections, a code must undergo an iterative peer review process that examines quality, fit, documentation, and clarity. Similarly, for agent-based models and similar types of model, the Network for Computational Modeling in Social and Ecological Sciences (CoMSESnet) offers model authors the option of submitting codes for peer review. Codes in the CoMSES Model Library that have passed review are flagged with an identifying badge.
 '''Publications that Describe Software, Models, and Algorithms'''
-Several geoscience journals specialize in papers that describe research software, algorithms, and other computational research tools and concepts. Some examples include Geoscientific Model Development, Computers and Geosciences, and Environmental Modelling and Software. These journals tend to focus on an article as the primary reviewed product, but they welcome articles that describe particular software packages. They can be a great choice when the main goal is to describe a new concept, algorithm, or capability that your software implements. There are also broader-scope journals such as the Journal of Open Research Software, which publishes “Software Metapapers”.
+Several geoscience journals specialize in papers that describe research software, algorithms, and other computational research tools and concepts. Examples include:
+* Geoscientific Model Development
+* Computers and Geosciences
+* Environmental Modelling and Software.
+These journals tend to focus on an article as the primary reviewed product, but they welcome articles that describe particular software packages.