AnalystExecutorSiwenna
Instructions for installing and configuring a WMT executor on siwenna.
--Mpiper (talk) 15:40, 8 October 2018 (MDT)
Set install and conda directories
The install directory for this executor is /home/csdms/wmt/analyst.
install_dir=/home/csdms/wmt/analyst mkdir -p $install_dir
Make sure read and execute bits are set on this directory.
chmod 0775 $install_dir
Install Python
Install a Python distribution to be used locally by WMT. We like to use Miniconda.
cd $install_dir curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -o miniconda.sh bash ./miniconda.sh -f -b -p $(pwd)/conda export PATH=$(pwd)/conda/bin:$PATH
If working with an existing Miniconda install, be sure to update everything before continuing:
conda update conda conda update --all
Install the CSDMS software stack
Clone the `wmt-executor-config-files` repo.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/mdpiper/wmt-executor-config-files cd wmt-executor-config-files
Install the CSDMS software stack, including the `babelizer` for components that need to be built from source, in a new environment.
conda install --file wmt-analyst.txt -c csdms-stack -c defaults -c conda-forge
Note that the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT (see discussion here). Also note I used a tagged version of PyMT because there have been a lot of changes to it since I built the conda recipes for the permamodel models.
Install executor software
Install the `wmt-exe` package from source.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/csdms/wmt-exe cd wmt-exe python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.
work_dir='/scratch/$USER/wmt/analyst' # note single quotes python setup.py configure --wmt-prefix=$install_dir --launch-dir=$work_dir --exec-dir=$work_dir ln -s "$(realpath wmt.cfg)" $conda_dir/envs/wmt-analyst/etc
Check that `$USER` didn't get expanded in the file. Lines 10-11 should be:
exec_dir = /scratch/$USER/wmt/analyst launch_dir = /scratch/$USER/wmt/analyst
Note that we're using /scratch for the launch and execution directories instead of the default ~/.wmt.
Install and test CSDMS components
Each section below describes how to install and test a particular CSDMS component.
Currently installed components:
- BRaKE
- CHILD
- CMIP
- CRUAKTemp
- FrostNumberGeoModel
- FrostNumberModel
- Hydrotrend
- KuGeoModel
- KuModel
- PyDeltaRCM
- Sedflux3D
Update (2018-10-08): CHILD is not working, so I've pulled it.
Note that when running IPython remotely it's helpful to set
export MPLBACKEND=Agg