TestingExecutorSiwenna
Instructions for installing and configuring a WMT executor on siwenna.
--Mpiper (talk) 12:33, 22 February 2018 (MST)
Set install directory
The install directory for this executor is /home/csdms/wmt/_testing.
install_dir=/home/csdms/wmt/_testing mkdir -p $install_dir
Install Python
Install a Python distribution to be used locally by WMT. We like to use Miniconda.
cd $install_dir curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -o miniconda.sh bash ./miniconda.sh -f -b -p $(pwd)/conda export PATH=$(pwd)/conda/bin:$PATH
If working with an existing Miniconda install, be sure to update everything before continuing:
conda update conda conda update --all
Install the CSDMS software stack
Using the csdms-stack conda channel (the Bakery) install the CSDMS software stack, including several pre-built components, with the `csdms-stack` metapackage.
conda install csdms-stack netcdf-fortran=4.4.4=5 -c csdms-stack -c defaults -c conda-forge
This metapackage currently includes
- pymt
- cca-tools
- csdms-child
- csdms-sedflux-3d
- csdms-hydrotrend
- csdms-permamodel-ku
- csdms-permamodel-frostnumber
- csdms-permamodel-kugeo
- csdms-permamodel-frostnumbergeo
- csdms-cruaktemp
- csdms-brake
- csdms-pydeltarcm
As of 2018-03-20, wind back the `netcdf-fortran` package so that HDF5 1.8.18 is used. There are problems with HDF5 1.10.1 and ESMF, used in PyMT. Hopefully, this will be a temporary change.
Optionally install the `babelizer`, in case a component needs to be built from source.
conda install -c csdms-stack babelizer
Optionally install IPython for testing.
conda install ipython
Note that when running IPython remotely on siwenna, it's helpful to set
export MPLBACKEND=Agg
Install executor software
Install the `wmt-exe` package from source.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/csdms/wmt-exe cd wmt-exe python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.
python setup.py configure --wmt-prefix=$install_dir --launch-dir='/scratch/$USER/wmt' --exec-dir='/scratch/$USER/wmt' ln -s "$(realpath wmt.cfg)" $install_dir/conda/etc
Here I've configured the executor to do stuff in the /scratch directory instead of the default ~/.wmt.
Install and test CSDMS components
Each section below describes how to install and test a particular CSDMS component.
Consider setting the PYMT_DEBUG environment variable.
export PYMT_DEBUG=True
It's really helpful for debugging.
Note that the ILAMB component from the Bakery won't work because it's built against `pymt=0.2`; in particular, my BMI for ILAMB is incompatible with the `model_metadata` package in newer versions of PyMT.
Currently installed components:
- BRaKE
- Child (no wmt-metadata)
- CMIP
- CRUAKTemp
- FrostNumberGeoModel
- FrostNumberModel
- Hydrotrend
- KuGeoModel
- KuModel
- PyDeltaRCM
- Sedflux3D (no wmt-metadata)
HydroTrend
Hydrotrend is included in the `csdms-stack` metapackage. Test it by starting a Python session in $install_dir/test and executing the setup and IRF methods.
from pymt.components import Hydrotrend comp = Hydrotrend() comp.get_component_name() # args = comp.setup('.') # comp.initialize(*args) comp.setup('.') comp.initialize(None) comp.get_start_time() comp.get_end_time() comp.get_current_time() comp.update() comp.finalize()
Permamodel and CRUAKTemp
Permamodel and CRUAKTemp are included in the `csdms-stack` metapackage. Test a component by starting a Python session in $install_dir/test and executing the setup and IRF methods.
from pymt.components import FrostNumberModel comp = FrostNumberModel() comp.get_component_name() # args = comp.setup('.') # comp.initialize(*args) comp.setup('.') comp.initialize('frostnumber_model.cfg') comp.get_start_time() comp.get_end_time() comp.get_current_time() comp.update() comp.finalize()
CMIP
Download and install the `cmip` software from source.
cd $install_dir/opt git clone https://github.com/permamodel/cmip cd cmip python setup.py develop
Babelize the CMIP tool.
cd $install_dir mkdir -p build && cd build bmi-babelize $install_dir/opt/cmip --prefix=$install_dir/conda
Download the sample config file and test the babelized CMIP component by starting a Python session in $install_dir/test and executing the setup and IRF methods.
from pymt.components import CMIP comp = CMIP() comp.get_component_name() comp.initialize('sample_cmip-siwenna.cfg') comp.get_start_time() comp.get_end_time() comp.get_current_time() comp.update() comp.get_current_time() comp.update_until(comp.get_end_time()) comp.get_current_time() comp.finalize()