Peer Review for Model Codes

For model codes, there are at least three general (and not mutually exclusive) types of peer review:

  1. review of the model code itself by a journal or scholarly organization
  2. review of an article that describes the concepts and algorithms used by a numerical model
  3. ongoing review of code as an intrinsic part of the development process (for team- and community-based codes that involve multiple developers and contributors).

1. Software Journals and Peer-Reviewed Community Code Libraries

There is a growing list of journals that review and publish research software, either as their sole mission or as one possible category of contribution. For example:

  • The Journal of Open Source Software (JOSS), which describes itself as “a developer friendly, open access journal for research software packages.”

JOSS software publications are fully citable objects with a digital object identifier (DOI). JOSS is a good choice when you want to publish a version of the software itself, as opposed to a written description of the concepts or algorithms underlying it. CSDMS considers code peer reviewed when there is a JOSS publictions associated with the code. See also the CSDMS model repository.

Several professional and scholarly organizations offer reviews of software that is included in an open-source community collection. For example:

2. Publications that Describe Software, Models, and Algorithms

Several geoscience journals specialize in papers that describe research software, algorithms, and other computational research tools and concepts. Examples include:

These journals tend to focus on an article as the primary reviewed product, but they welcome articles that describe particular software packages.

3. Ongoing Peer Review during Development

Code review is a powerful quality-control tool, and for that reason review tools are now routinely built into collaborative code-development platforms like GitHub. These days, it is common for research software teams to use peer review as a regular part of the software development process. For example, at the CSDMS Integration Facility, additions and changes to software tools are initially developed on a separate branch, and that branch is only merged into the main code base once it has been reviewed and approved by at least one other person. The same goes for community contributions to CSDMS tools. (In fact, reviewing code pull requests is a great way to contribute to CSDMS!). Many of the larger, team-built codes used in geoscience benefit from some type of ongoing internal peer review. Unlike software papers, this ongoing review process does not by itself result in a citable product, and in that sense the labor may be invisible to the academic credit system. Nonetheless, we highly recommend using a review process for your own code, even if it is just a matter of asking a friend or colleague to take a look and give feedback. One advantage is that the road to formal publication in a software-oriented journal is likely to be easier if you code has already passed through an informal peer review process during development.