PermafrostExecutorBlanca

wmt-permafrost executor on blanca

Instructions for installing and configuring a WMT executor on blanca.

--Mpiper (talk) 16:48, 19 March 2018 (MDT)

Set build environment on blanca

Login to summit.

ssh mapi8461@login.rc.colorado.edu

Get the corrent slurm for blanca.

module load slurm/blanca

Login to a compute node.

sinteractive

Build everything on the compute node.

Set install directory

The install directory for this executor is /work/csdms/wmt/permafrost.

install_dir=/work/csdms/wmt/permafrost
mkdir -p $install_dir

Make sure read and execute bits are set on this directory.

chmod 0775 $install_dir

Install Python

Install a Python distribution to be used locally by WMT. We like to use Miniconda.

cd $install_dir
curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -o miniconda.sh
bash ./miniconda.sh -f -b -p $(pwd)/conda
export PATH=$(pwd)/conda/bin:$PATH

If working with an existing Miniconda install, be sure to update everything before continuing:

conda update conda
conda update --all

Install the CSDMS software stack

Install the base of the CSDMS software stack, including the `babelizer` for components that need to be built from source.

Note that the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT (see discussion here).

conda install cca-tools babelizer netcdf-fortran=4.4.4=5 -c csdms-stack -c defaults -c conda-forge

Optionally install IPython for testing.

conda install ipython

Note that when running IPython remotely on siwenna, it's helpful to set

export MPLBACKEND=Agg

Install executor software

Load blanca's `git` module.

module load git

Install the `wmt-exe` package from source.

mkdir -p $install_dir/opt && cd $install_dir/opt
git clone https://github.com/csdms/wmt-exe
cd wmt-exe
python setup.py develop

Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.

python setup.py configure --wmt-prefix=$install_dir --launch-dir='/rc_scratch/$USER/wmt/permafrost' --exec-dir='/rc_scratch/$USER/wmt/permafrost'
#ln -s "$(realpath wmt.cfg)" $install_dir/conda/etc # "realpath" not installed on blanca :(
cd $install_dir/conda/etc
ln -s $install_dir/opt/wmt-exe/wmt.cfg

Note that we're using rc_scratch for the launch and execution directories instead of the default ~/.wmt. Also note that we needed an SbatchLauncher class for wmt-exe because blanca uses Slurm instead of Torque for job control.

Install and test CSDMS components

Each section below describes how to install and test a particular CSDMS component.

Currently installed components:

1. CMIP
2. CRUAKTemp
3. FrostNumberGeoModel
4. FrostNumberModel
5. KuGeoModel
6. KuModel

Permamodel

Install the permamodel components from the Bakery using the `csdms-permamodel` metapackage.

conda install csdms-permamodel -c csdms-stack

This metapackage contains

• csdms-permamodel-ku
• csdms-permamodel-frostnumber
• csdms-permamodel-kugeo
• csdms-permamodel-frostnumbergeo
• csdms-cruaktemp

Test a component (here, FnM) by starting a Python session in $install_dir/test in an interactive Slurm session and executing the setup and IRF methods.

from pymt.components import FrostNumberModel

comp = FrostNumberModel()
comp.get_component_name()
# args = comp.setup('.')
# comp.initialize(*args)
comp.setup('.')
comp.initialize('frostnumber_model.cfg')
comp.get_start_time()
comp.get_end_time()
comp.get_current_time()
comp.update()
comp.finalize()

Check the ouput of the run

$ cat frostnumber_output.dat
Year: 2000  output=('0.396', '-1.000', '-1.000')

CMIP

Install the CMIP component through its conda recipe.

conda install csdms-cmip -c csdms-stack -c defaults -c conda-forge

Download the sample config file and test the CMIP component by starting a Python session in $install_dir/test and executing the setup and IRF methods.

from pymt.components import CMIP

comp = CMIP()
comp.get_component_name()
comp.initialize('sample_cmip-blanca.cfg')
comp.get_start_time()
comp.get_end_time()
comp.get_current_time()
comp.update()
comp.get_current_time()
comp.update_until(comp.get_end_time())
comp.get_current_time()
comp.finalize()