Web APIs

From CSDMS

Introduction how to use CSDMS website APIs

APIs

API or Application Programming Interface allows applications to send and receive data to each other. This is done through a set of subroutine definitions, protocols, and tools. In practice, when using programmable APIs the request (or ‘call’) is simply a URL and the response is often in JSON format. CSDMS has enabled API functionality for its website and associated databases.The API of the CSDMS website provide direct, high-level access to the data contained in the MediaWiki databases. The CSDMS website API entry point is:

mediawiki/api.php


At the entry point more information can be found among others on Parameters and their actions and format. Here you find the various actions available for the databases that are operational on the CSDMS website (Cargo and Semantic MediaWiki (SMW)).

Playground

A sandbox is setup to test out simple CSDMS web API functionality through a GUI. This :

Special:ApiSandbox

Most often though, APIs are applied not through a GUI but in a coding environment, where URLs are constructed to, for example, query a web database.

Implemented databases

On top of the main MediaWiki database, CSDMS has implemented two other web databases, Cargo and Semantic MediaWiki (SMW), which have their own API parameters and actions. The SMW and Cargo API modules allow to select and retrieve, i.e. read annotated information stored on the CSDMS wiki. If information has to be written to the wiki then the regular MediaWiki API-modules available for editing must be used. This is to ensure that the corresponding wiki page is updated the same way as when one would edit the website. And this makes sure that no information accessible via a wiki page revision is lost.
In the examples below we focus on how to retrieve information from the CSDMS repositories, which is unique for each of the different implemented databases.

Examples

Regular MediaWiki API

The CSDMS website can be queried using the API. For example, the website can be asked to provide the current time by construction the following URL:

API URL: https://csdms.colorado.edu/mediawiki/api.php?action=query&curtimestamp=1
When you click on the above link, you'll see it returns something like: { "batchcomplete": "", "curtimestamp": "2018-05-08T17:06:46Z" }

The action=query sends a requests to the server, which is followed by a parameter, in this case curtimestamp.

A somewhat more complex API request would be to find out who all made contributions to a certain page. For example, who made edits to the landscape evolution model CHILD? This can be queried by:

API URL: https://csdms.colorado.edu/mediawiki/api.php?action=query&prop=contributors&titles=Model:CHILD&format=json
(click on link to see results)

Here the URL contains elements to define a server request action=query for the page property contributors prop=contributors, for the CHILD model page titles=Model:CHILD, and the requested information is provided in a json format format=json.

Semantic MediaWiki (SMW) API

CSDMS is hosting some of its repositories in the SMW database, which can be queried using the askargs statement. The module supports three parameters in un-serialized form specific to the SMW #ask function:

  1. "conditions": The query conditions, i.e. the requirements for a subject (page or subobject) to be included
  2. "printouts": The query printouts, i.e, the properties to show per subject (page or subobject)
  3. "parameters": The query parameters, i.e. all non-condition and non-printout arguments
Limit and offset

As in regular inline (ask) queries, both "limit" and "offset" are specified in the parameter list, e.g. to get the first 10 results: &parameters=limit%3D10.

The API result contains a "query-continue-offset" key, which can be used to fetch additional results: &parameters=offset%3D10|limit%3D10. If there is no "query-continue-offset" key in the result, the end of the result set was reached.

Query examples using SMW
Description Query URL examples
From the entire Model repo, show 3 models created by Tucker and print the output in a json format https://csdms.colorado.edu/mediawiki/api.php?action=askargs &conditions=Model:+|Last_name::Tucker &parameters=limit=3&format=json
From the entire Model repo, show only those models programmed in python, and print the URL directing to the CSDMS source code, or to other, outside the CSDMS repository code storage repos. All in jsonfm format.
(Notice that limit is set to very large to make sure all results are shown)
https://csdms.colorado.edu/mediawiki/api.php?action=askargs &conditions=Model:+|Programming_language::Python &printouts=Source_csdms_web_address|Source_web_address &parameters=limit=10000&format=jsonfm


Cargo API

The Cargo database API is setup slightly different than for example the SMW described above. For querying Cargo the action cargoquery can be used. Cargoquery can hold the following parameters:

  • limit: A limit on the number of results returned. No more than 500 (5,000 for bots) allowed. Default: 50
  • tables: The Cargo database table or tables on which to search. Available cargo tables can be found here
  • fields: The table field(s) to retrieve
  • where: The conditions for the query, corresponding to an SQL WHERE clause
  • join_on: Conditions for joining multiple tables, corresponding to an SQL JOIN ON clause
  • group_by: Field(s) on which to group results, corresponding to an SQL GROUP BY clause
  • having: Conditions for grouped values, corresponding to an SQL HAVING clause
  • order_by: The order of results, corresponding to an SQL ORDER BY clause
  • offset: The query offset. The value must be no less than 0. Default: 0
Query examples using Cargo
Description Query URL examples
From the entire model reference repo, show the total citations per publication https://csdms.colorado.edu/mediawiki/api.php? action=cargoquery &tables=RefsInt1 &fields=_pageName=RefsInt1,PublicationCitationsNr &limit=5000
From the entire model reference repo, show the title of the references of the HydroTrend model that are entered manually https://csdms.colorado.edu/mediawiki/api.php? action=cargoquery &tables=RefsInt2,BibEntryCargo &join_on=RefsInt2._pageName=BibEntryCargo._pageName &fields=Title,PublicationMultipleModelsCargo &where=PublicationMultipleModelsCargo=%22HydroTrend%22 &limit=5000

More examples on how to query databases using Cargo are described here: https://www.mediawiki.org/wiki/Extension:Cargo/Querying_data

Repositories and their parameters

CSDMS stores its website data in various repositories. Each repository has its own set of query conditions and query printouts. The table below shows available condition and printouts parameters

Database & available parameters Format
Model metadata SMW
Model related publications Cargo
Data repository SMW
Model movies SMW