AnalystExecutorSiwenna

wmt-analyst executor on siwenna

Instructions for installing and configuring a WMT executor on siwenna.

--Mpiper (talk) 15:40, 8 October 2018 (MDT)

Set install and conda directories

The install directory for this executor is /home/csdms/wmt/analyst.

install_dir=/home/csdms/wmt/analyst
mkdir -p $install_dir

Make sure read and execute bits are set on this directory.

chmod 0775 $install_dir

Install Python

Install a Python distribution to be used locally by WMT. We like to use Miniconda.

cd $install_dir
curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -o miniconda.sh
bash ./miniconda.sh -f -b -p $(pwd)/conda
export PATH=$(pwd)/conda/bin:$PATH

If working with an existing Miniconda install, be sure to update everything before continuing:

conda update conda
conda update --all

Install the CSDMS software stack

Clone the `wmt-executor-config-files` repo.

mkdir -p $install_dir/opt && cd $install_dir/opt
git clone https://github.com/mdpiper/wmt-executor-config-files
cd wmt-executor-config-files

Install the CSDMS software stack, including the `babelizer` for components that need to be built from source, in a new environment.

conda install --file wmt-analyst.txt -c csdms-stack -c defaults -c conda-forge

Note that the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT (see discussion here). Also note I used a tagged version of PyMT because there have been a lot of changes to it since I built the conda recipes for the permamodel models.

Install executor software

Install the `wmt-exe` package from source.

mkdir -p $install_dir/opt && cd $install_dir/opt
git clone https://github.com/csdms/wmt-exe
cd wmt-exe
python setup.py develop

Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.

work_dir='/scratch/$USER/wmt/analyst'  # note single quotes
python setup.py configure --wmt-prefix=$install_dir --launch-dir=$work_dir --exec-dir=$work_dir
ln -s "$(realpath wmt.cfg)" $conda_dir/envs/wmt-analyst/etc

Note that we're using /scratch for the launch and execution directories instead of the default ~/.wmt.

Install and test CSDMS components

Each section below describes how to install and test a particular CSDMS component.

Currently installed components:

1. BRaKE
2. CHILD
3. CMIP
4. CRUAKTemp
5. FrostNumberGeoModel
6. FrostNumberModel
7. Hydrotrend
8. KuGeoModel
9. KuModel
10. PyDeltaRCM
11. Sedflux3D

Update (2018-10-08): CHILD is not working, so I've pulled it.

Note that when running IPython remotely it's helpful to set

export MPLBACKEND=Agg