AnalystExecutorBlanca

From CSDMS
wmt-analyst executor on blanca

Instructions for installing and configuring a WMT executor on blanca.


Set build environment on blanca

Login to summit.

ssh mapi8461@login.rc.colorado.edu

Load the correct slurm for blanca.

module load slurm/blanca

Login to a compute node.

sinteractive

Build everything on the compute node.

Set install and conda directories

The install directory for this executor is /work/csdms/wmt/analyst.

install_dir=/work/csdms/wmt/analyst
mkdir -p $install_dir

Make sure read and execute bits are set on this directory.

chmod 0775 $install_dir

An Anaconda distribution is located at /work/csdms/anaconda3. This distribution is used by all WMT executors on blanca. Store this location and add it to your path.

conda_dir=/work/csdms/anaconda3
PATH=$conda_dir/bin:$PATH

WMT executors are installed as environments within this distribution.

Install the CSDMS software stack

Load blanca's `git` module.

module load git

Clone the `wmt-executor-config-files` repo.

mkdir -p $install_dir/opt && cd $install_dir/opt
git clone https://github.com/mdpiper/wmt-executor-config-files
cd wmt-executor-config-files

Install the CSDMS software stack, including the `babelizer` for components that need to be built from source, in a new environment.

conda env create -f wmt-analyst.yml

The stack will be installed in $conda_dir/envs/wmt-analyst.

Note that the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT (see discussion here). Also note I used a tagged version of PyMT because there have been a lot of changes to it since I built the conda recipes for the permamodel models.

Install executor software

Install the `wmt-exe` package from source.

mkdir -p $install_dir/opt && cd $install_dir/opt
git clone https://github.com/csdms/wmt-exe
cd wmt-exe
python setup.py develop

Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.

work_dir='/rc_scratch/$USER/wmt/analyst'  # note single quotes
python setup.py configure --wmt-prefix=$install_dir --launch-dir=$work_dir --exec-dir=$work_dir
ln -s "$(realpath wmt.cfg)" $conda_dir/envs/wmt-analyst/etc

Check that `$USER` didn't get expanded in the file. Lines 10-11 should be:

exec_dir = /rc_scratch/$USER/wmt/analyst
launch_dir = /rc_scratch/$USER/wmt/analyst

Note that we're using /rc_scratch for the launch and execution directories instead of the default ~/.wmt. Also note that we needed an SbatchLauncher class for wmt-exe because blanca uses Slurm instead of Torque for job control.

Install and test CSDMS components

Each section below describes how to install and test a particular CSDMS component.

Currently installed components:

  1. BRaKE
  2. CHILD
  3. CMIP
  4. CRUAKTemp
  5. FrostNumberGeoModel
  6. FrostNumberModel
  7. Hydrotrend
  8. KuGeoModel
  9. KuModel
  10. PyDeltaRCM
  11. Sedflux3D

Note that when running IPython remotely it's helpful to set

export MPLBACKEND=Agg

Note: 2019-02-11

I deleted the wmt-analyst environment and reinstalled it with Hydrotrend as the only component. It works. (It did not work when I also installed the permamodel components.)