PermafrostExecutorBlanca
Instructions for installing and configuring a WMT executor on blanca.
--Mpiper (talk) 16:48, 19 March 2018 (MDT)
Set build environment on blanca
Login to summit.
ssh mapi8461@login.rc.colorado.edu
Get the corrent slurm for blanca.
module load slurm/blanca
Login to a compute node.
sinteractive
Build everything on the compute node.
Set install directory
The install directory for this executor is /work/csdms/wmt/permafrost.
install_dir=/work/csdms/wmt/permafrost mkdir -p $install_dir
Make sure read and execute bits are set on this directory.
chmod 0775 $install_dir
Install Python
Install a Python distribution to be used locally by WMT. We like to use Miniconda.
cd $install_dir curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -o miniconda.sh bash ./miniconda.sh -f -b -p $(pwd)/conda export PATH=$(pwd)/conda/bin:$PATH
If working with an existing Miniconda install, be sure to update everything before continuing:
conda update conda conda update --all
Install the CSDMS software stack
Install the base of the CSDMS software stack, including the `babelizer` for components that need to be built from source.
Note that the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT (see discussion here).
Also note I used a tagged version of PyMT because there have been a lot of changes to it since I built the conda recipes for the permamodel models.
conda install cca-tools babelizer pymt==0.2 netcdf-fortran=4.4.4=5 -c csdms-stack -c defaults -c conda-forge
Alternately, use the requirements file listed here.
Optionally install IPython for testing.
conda install ipython
Note that when running IPython remotely on siwenna, it's helpful to set
export MPLBACKEND=Agg
Install executor software
Load blanca's `git` module.
module load git
Install the `wmt-exe` package from source.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/csdms/wmt-exe cd wmt-exe python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.
work_dir="/rc_scratch/$USER/wmt/permafrost" python setup.py configure --wmt-prefix=$install_dir --launch-dir=$work_dir --exec-dir=$work_dir #ln -s "$(realpath wmt.cfg)" $install_dir/conda/etc # "realpath" not installed on blanca :( cd $install_dir/conda/etc ln -s $install_dir/opt/wmt-exe/wmt.cfg
Check that `$USER` didn't get expanded in the file. Lines 10-11 should be:
exec_dir = /rc_scratch/$USER/wmt/permafrost launch_dir = /rc_scratch/$USER/wmt/permafrost
Note that we're using /rc_scratch for the launch and execution directories instead of the default ~/.wmt. Also note that we needed an SbatchLauncher class for wmt-exe because blanca uses Slurm instead of Torque for job control.
Install and test CSDMS components
Each section below describes how to install and test a particular CSDMS component.
Currently installed components:
- CMIP
- CRUAKTemp
- FrostNumberGeoModel
- FrostNumberModel
- KuGeoModel
- KuModel
Permamodel
Install the permamodel components from the Bakery using the `csdms-permamodel` metapackage.
conda install csdms-permamodel -c csdms-stack
This metapackage contains
- csdms-permamodel-ku
- csdms-permamodel-frostnumber
- csdms-permamodel-kugeo
- csdms-permamodel-frostnumbergeo
- csdms-cruaktemp
Test a component (here, FnM) by starting a Python session in $install_dir/test in an interactive Slurm session and executing the setup and IRF methods.
from pymt.components import FrostNumberModel
comp = FrostNumberModel()
comp.get_component_name()
# args = comp.setup('.')
# comp.initialize(*args)
comp.setup('.')
comp.initialize('frostnumber_model.cfg')
comp.get_start_time()
comp.get_end_time()
comp.get_current_time()
comp.update()
comp.finalize()
Check the ouput of the run
$ cat frostnumber_output.dat
Year: 2000 output=('0.396', '-1.000', '-1.000')
CMIP
Install the CMIP component through its conda recipe.
conda install csdms-cmip -c csdms-stack -c defaults -c conda-forge
Download the sample config file and test the CMIP component by starting a Python session in $install_dir/test and executing the setup and IRF methods.
from pymt.components import CMIP
comp = CMIP()
comp.get_component_name()
comp.initialize('sample_cmip-blanca.cfg')
comp.get_start_time()
comp.get_end_time()
comp.get_current_time()
comp.update()
comp.get_current_time()
comp.update_until(comp.get_end_time())
comp.get_current_time()
comp.finalize()
