JupyterHub: Difference between revisions
m (Add maintenance section) |
m (Improve text) |
||
| (19 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
{{PageTitle| | {{PageTitle|EarthscapeHub}} | ||
<div class=AutoScaleImage>[[ File: | <div class=AutoScaleImage>[[ File:oes-logo.png | What is OpenEarthscape? | 400 px | right | link=https://csdms.colorado.edu/wiki/OpenEarthscape ]]<div> | ||
<!-- | |||
<div class=AutoScaleImage>[[ File:Jupyterhub-logo.png | What is JupyterHub? | 400 px | right | link=https://jupyter.org/hub ]]<div> | |||
--> | |||
Through the NSF-funded [[OpenEarthscape]] project, | Through the NSF-funded [[OpenEarthscape]] project, | ||
CSDMS | CSDMS operates three [https://jupyter.org/hub JupyterHubs]: | ||
# [https://jupyter.openearthscape.org jupyter.openearthscape.org] | # [https://jupyter.openearthscape.org jupyter.openearthscape.org] | ||
# [https://lab.openearthscape.org lab.openearthscape.org] | # [https://lab.openearthscape.org lab.openearthscape.org] | ||
# [https://frontier.openearthscape.org frontier.openearthscape.org] | |||
Together, they form ''EarthscapeHub''. | |||
Each Hub is hosted on the commercial cloud through the NSF [https://www.cloudbank.org/ CloudBank] program. | |||
Each is free and available for use by all CSDMS members, whether you're an undergraduate, grad student, postdoc, researcher, or faculty. | |||
What can you do with | What can you do with EarthscapeHub? Here are a few ideas: | ||
* Prototype and test Python code for a model | * Prototype and test Python code for a model | ||
| Line 22: | Line 28: | ||
* Run a lab from the [[Labs portal|CSDMS EKT repository]] | * Run a lab from the [[Labs portal|CSDMS EKT repository]] | ||
* Highlight your research in a notebook and share it with others in the community | * Highlight your research in a notebook and share it with others in the community | ||
* Explore a parameter space with multiple simultaneous model runs | |||
* Access a Linux OS and shell in the cloud | * Access a Linux OS and shell in the cloud | ||
The roles of these | The roles of these JupyterHubs---''jupyter'', ''lab'', and ''frontier''---as well as sign up and sign in instructions for each, are given in the sections below. | ||
= jupyter = | = jupyter = | ||
This is the primary JupyterHub for use by all CSDMS members. | This is the primary JupyterHub for use by all CSDMS members. | ||
It's a place to work | It's a place to work and experiment. | ||
It includes CSDMS | It includes [[Workbench|CSDMS Workbench]] elements such as [[Landlab]], [[PyMT|pymt]], and [[DataComponents|data components]]. | ||
Labs from the [[Labs portal|CSDMS EKT repository]] can run on this Hub. | Labs from the [[Labs portal|CSDMS EKT repository]] can be run on this Hub. | ||
Use this Hub to design, prototype, and test model code before production runs on the ''frontier'' Hub. | |||
This Hub uses GitHub as a login service. | This Hub uses GitHub as a login service. | ||
| Line 42: | Line 50: | ||
This JupyterHub is used for teaching; e.g., university classes or interactive workshops at a conference. | This JupyterHub is used for teaching; e.g., university classes or interactive workshops at a conference. | ||
It's used for teaching and collaborative work in [[ESPIn]]. | |||
Like ''jupyter'', it's a place to work, experiment, and learn. | Like ''jupyter'', it's a place to work, experiment, and learn. | ||
It includes CSDMS | It includes [[Workbench|CSDMS Workbench]] elements such as [[Landlab]], [[PyMT|pymt]], and [[DataComponents|data components]]. | ||
Labs from the [[Labs portal|CSDMS EKT repository]] can also run on this Hub. | Labs from the [[Labs portal|CSDMS EKT repository]] can also be run on this Hub. | ||
A username and password are required to login to this Hub. | A username (typically an institutional email address) and password are required to login to this Hub. | ||
If you'd like to teach a class or lead a workshop using | If you'd like to teach a class or lead a workshop using ''lab'', | ||
please contact us through the [https://csdms.github.io/help-desk CSDMS Help Desk], | please contact us through the [https://csdms.github.io/help-desk CSDMS Help Desk], | ||
or email us at [mailto:csdms@colorado.edu csdms@colorado.edu]. | or email us at [mailto:csdms@colorado.edu csdms@colorado.edu]. | ||
| Line 54: | Line 63: | ||
Note that the login for this Hub is separate from the login for the ''jupyter'' Hub. | Note that the login for this Hub is separate from the login for the ''jupyter'' Hub. | ||
Non-instructor logins that are inactive for more than a year will be removed. | Non-instructor logins that are inactive for more than a year will be removed. | ||
= frontier = | |||
On ''frontier'', work with a CSDMS RSE to set up a powerful computational resource in the cloud, | |||
occupying a middle ground between a personal computer and an HPC. | |||
This Hub is ideal, for example, for exploring a parameter space with multiple simultaneous model runs. | |||
Access to ''frontier'' is limited to a few users at a time, | |||
allowing them to take maximum advantage of its compute power. | |||
If you have an idea for a project that you'd like to run on ''frontier'', | |||
please contact us through the [https://github.com/csdms/help-desk/issues/new?assignees=mdpiper&labels=jupyterhub&template=access-frontier-hub.md&title=Access+the+frontier+JupyterHub CSDMS Help Desk]. | |||
We can help you get set up. | |||
Bear in mind you may have to think about a [https://hpc-wiki.info/hpc/Scaling scaling plan] for your project; | |||
e.g., how many processors, how much memory, how much run time, how much disk space will you use? | |||
= common features = | = common features = | ||
The ''jupyter'' and '' | The ''jupyter'', ''lab'', and ''frontier'' Hubs share some common features described below. | ||
== user interface == | == user interface == | ||
The default user interface on | The default user interface on all Hubs is the flexible and configurable [https://jupyter.org/ JupyterLab]. | ||
If you haven't used JupyterLab before, there's ample [https://jupyterlab.readthedocs.io documentation]. | If you haven't used JupyterLab before, there's ample [https://jupyterlab.readthedocs.io documentation]. | ||
== default files == | == default files == | ||
On each Hub, a welcome notebook and pair of directories are included by default in your home directory. | |||
The directories are: | The directories are: | ||
| Line 77: | Line 98: | ||
If you'd like to add files to the ''data'' directory, | If you'd like to add files to the ''data'' directory, | ||
please contact us through the [https://csdms.github.io/help-desk CSDMS Help Desk]. | please contact us through the [https://csdms.github.io/help-desk CSDMS Help Desk]. | ||
We've also created a brief tutorial on [[JupyterHubUploadDownload|uploading and downloading files]] to/from a JupyterHub. | |||
== software == | == software == | ||
| Line 87: | Line 109: | ||
including [[Landlab]] and [[PyMT|pymt]], | including [[Landlab]] and [[PyMT|pymt]], | ||
select the ''CSDMS'' kernel. | select the ''CSDMS'' kernel. | ||
The core packages installed in the ''CSDMS'' kernel are listed in [https://github.com/csdms/jupyterhub-management/blob/main/environments/csdms.yaml this conda environment file]. | |||
On the ''jupyter'' and ''frontier'' Hub instances, | |||
there's also a kernel called ''Develop'' that contains a host of development tools for writing software, | |||
including the GNU toolchain, CMake, conda compilers, and all the dependencies needed to build a Landlab component. | |||
The core packages installed in the ''Develop'' kernel are listed in [https://github.com/csdms/jupyterhub-management/blob/main/environments/develop.yaml this conda environment file]. | |||
It's like having your own Linux development machine in the cloud! | |||
If you'd like new software installed on a Hub (e.g., for a class or a workshop), | If you'd like new software installed on a Hub (e.g., for a class or a workshop), | ||
| Line 93: | Line 122: | ||
= system maintenance = | = system maintenance = | ||
The ''lab'' and '' | The ''lab'', ''jupyter'', and ''frontier'' Hubs will be offline for maintenance and software updates a few days per year, | ||
usually at the beginning of June and the end of December. | usually at the beginning of June and the end of December. | ||
The Hubs *may* also be offline for maintenance on Sundays between 9:30 and 10:00 AM Mountain Time. | |||
If people are using the Hubs around these times, | If people are using the Hubs around these times, | ||
we'll work with them to minimize disruption. | we'll work with them to minimize disruption. | ||
If you encounter a problem using EarthscapeHub, | |||
please contact us through the [https://csdms.github.io/help-desk CSDMS Help Desk]. | |||
Or, if you're comfortable with GitHub, | |||
log an issue directly in our [https://github.com/csdms/jupyterhub-management Hub management repository]. | |||
Latest revision as of 12:20, 9 November 2023
Through the NSF-funded OpenEarthscape project,
CSDMS operates three JupyterHubs:
Together, they form EarthscapeHub. Each Hub is hosted on the commercial cloud through the NSF CloudBank program. Each is free and available for use by all CSDMS members, whether you're an undergraduate, grad student, postdoc, researcher, or faculty.
What can you do with EarthscapeHub? Here are a few ideas:
- Prototype and test Python code for a model
- Run a model through Landlab or pymt
- Create a notebook for a journal article
- Teach a class at a university
- Lead an interactive workshop at a conference
- Work on a course assignment
- Run a lab from the CSDMS EKT repository
- Highlight your research in a notebook and share it with others in the community
- Explore a parameter space with multiple simultaneous model runs
- Access a Linux OS and shell in the cloud
The roles of these JupyterHubs---jupyter, lab, and frontier---as well as sign up and sign in instructions for each, are given in the sections below.
jupyter
This is the primary JupyterHub for use by all CSDMS members. It's a place to work and experiment. It includes CSDMS Workbench elements such as Landlab, pymt, and data components. Labs from the CSDMS EKT repository can be run on this Hub. Use this Hub to design, prototype, and test model code before production runs on the frontier Hub.
This Hub uses GitHub as a login service. Please see these instructions to sign up and sign in.
Note that the login for this Hub is separate from the login for the lab Hub. There is no expiration date for a login on this Hub.
lab
This JupyterHub is used for teaching; e.g., university classes or interactive workshops at a conference. It's used for teaching and collaborative work in ESPIn. Like jupyter, it's a place to work, experiment, and learn. It includes CSDMS Workbench elements such as Landlab, pymt, and data components. Labs from the CSDMS EKT repository can also be run on this Hub.
A username (typically an institutional email address) and password are required to login to this Hub. If you'd like to teach a class or lead a workshop using lab, please contact us through the CSDMS Help Desk, or email us at csdms@colorado.edu. We can help install software and create logins for the participants in your class or workshop.
Note that the login for this Hub is separate from the login for the jupyter Hub. Non-instructor logins that are inactive for more than a year will be removed.
frontier
On frontier, work with a CSDMS RSE to set up a powerful computational resource in the cloud, occupying a middle ground between a personal computer and an HPC. This Hub is ideal, for example, for exploring a parameter space with multiple simultaneous model runs. Access to frontier is limited to a few users at a time, allowing them to take maximum advantage of its compute power.
If you have an idea for a project that you'd like to run on frontier, please contact us through the CSDMS Help Desk. We can help you get set up. Bear in mind you may have to think about a scaling plan for your project; e.g., how many processors, how much memory, how much run time, how much disk space will you use?
common features
The jupyter, lab, and frontier Hubs share some common features described below.
user interface
The default user interface on all Hubs is the flexible and configurable JupyterLab. If you haven't used JupyterLab before, there's ample documentation.
default files
On each Hub, a welcome notebook and pair of directories are included by default in your home directory. The directories are:
- data: A read-only directory for storing and sharing data
- scratch: A read-write directory for sharing code, notebooks, or other files with other users
Files that remain in scratch longer than 90 days may be deleted. If you'd like to add files to the data directory, please contact us through the CSDMS Help Desk. We've also created a brief tutorial on uploading and downloading files to/from a JupyterHub.
software
These Hubs use Jupyter kernels mapped to conda environments to group software. The default kernel is called Python 3. It contains a minimal set of software packages. To explore with CSDMS models and tools, including Landlab and pymt, select the CSDMS kernel. The core packages installed in the CSDMS kernel are listed in this conda environment file.
On the jupyter and frontier Hub instances, there's also a kernel called Develop that contains a host of development tools for writing software, including the GNU toolchain, CMake, conda compilers, and all the dependencies needed to build a Landlab component. The core packages installed in the Develop kernel are listed in this conda environment file. It's like having your own Linux development machine in the cloud!
If you'd like new software installed on a Hub (e.g., for a class or a workshop), please contact us through the CSDMS Help Desk.
system maintenance
The lab, jupyter, and frontier Hubs will be offline for maintenance and software updates a few days per year, usually at the beginning of June and the end of December. The Hubs *may* also be offline for maintenance on Sundays between 9:30 and 10:00 AM Mountain Time. If people are using the Hubs around these times, we'll work with them to minimize disruption.
If you encounter a problem using EarthscapeHub, please contact us through the CSDMS Help Desk. Or, if you're comfortable with GitHub, log an issue directly in our Hub management repository.
