AnalystExecutorBlanca
Instructions for installing and configuring a WMT executor on blanca.
Set build environment on blanca
Login to summit.
ssh mapi8461@login.rc.colorado.edu
Load the correct slurm for blanca.
module load slurm/blanca
Login to a compute node.
sinteractive
Build everything on the compute node.
Set install and conda directories
The install directory for this executor is /work/csdms/wmt/analyst.
install_dir=/work/csdms/wmt/analyst mkdir -p $install_dir
Make sure read and execute bits are set on this directory.
chmod 0775 $install_dir
An Anaconda distribution is located at /work/csdms/anaconda3. This distribution is used by all WMT executors on blanca. Store this location and add it to your path.
conda_dir=/work/csdms/anaconda3 PATH=$conda_dir/bin:$PATH
WMT executors are installed as environments within this distribution.
Install the CSDMS software stack
Load blanca's `git` module.
module load git
Clone the `wmt-executor-config-files` repo.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/mdpiper/wmt-executor-config-files cd wmt-executor-config-files
Install the CSDMS software stack, including the `babelizer` for components that need to be built from source, in a new environment.
conda env create -f wmt-analyst.yml
The stack will be installed in $conda_dir/envs/wmt-analyst.
Note that the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT (see discussion here). Also note I used a tagged version of PyMT because there have been a lot of changes to it since I built the conda recipes for the permamodel models.
Install executor software
Install the `wmt-exe` package from source.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/csdms/wmt-exe cd wmt-exe python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.
work_dir='/rc_scratch/$USER/wmt/analyst' # note single quotes python setup.py configure --wmt-prefix=$install_dir --launch-dir=$work_dir --exec-dir=$work_dir ln -s "$(realpath wmt.cfg)" $conda_dir/envs/wmt-analyst/etc
Check that `$USER` didn't get expanded in the file. Lines 10-11 should be:
exec_dir = /rc_scratch/$USER/wmt/analyst launch_dir = /rc_scratch/$USER/wmt/analyst
Note that we're using /rc_scratch for the launch and execution directories instead of the default ~/.wmt. Also note that we needed an SbatchLauncher class for wmt-exe because blanca uses Slurm instead of Torque for job control.
Install and test CSDMS components
Each section below describes how to install and test a particular CSDMS component.
Currently installed components:
- BRaKE
- CHILD
- CMIP
- CRUAKTemp
- FrostNumberGeoModel
- FrostNumberModel
- Hydrotrend
- KuGeoModel
- KuModel
- PyDeltaRCM
- Sedflux3D
Note that when running IPython remotely it's helpful to set
export MPLBACKEND=Agg
Note: 2019-02-11
I deleted the wmt-analyst environment and reinstalled it with Hydrotrend as the only component. It works. (It did not work when I also installed the permamodel components.)