PermafrostExecutorBlanca: Difference between revisions
(Initial sections) |
No edit summary |
||
Line 17: | Line 17: | ||
{{:Shorter_Miniconda_install_instructions}} | {{:Shorter_Miniconda_install_instructions}} | ||
==Install the CSDMS software stack== | |||
A limiting factor in setting up this executor | |||
is the componentized version of ILAMB can only be run with `pymt=0.2`; | |||
in particular, | |||
the BMI I wrote for ILAMB is incompatible | |||
with the `model_metadata` package in newer versions of PyMT. | |||
Further, | |||
as of 2018-03-20, | |||
the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used | |||
because of problems with HDF5 1.10.1 and ESMF, used in PyMT. | |||
To ensure all the packages meet the same dependencies | |||
(looking at you, numpy), | |||
simultaneously install `cca-tools`, `netcdf-fortran`, `pymt=0.2` and `csdms-ilamb` | |||
using the ''csdms-stack'' conda channel (the [https://anaconda.org/csdms-stack/ Bakery]), | |||
conda install cca-tools netcdf-fortran=4.4.4=5 pymt=0.2 csdms-ilamb -c csdms-stack -c defaults -c conda-forge | |||
Install the `babelizer` for components that need to be built from source. | |||
conda install -c csdms-stack babelizer | |||
Optionally install IPython for testing. | |||
conda install ipython | |||
Note that when running IPython remotely on '''''siwenna''''', | |||
it's helpful to set | |||
export MPLBACKEND=Agg | |||
===Install executor software=== | |||
Install the `wmt-exe` package from source. | |||
mkdir -p $install_dir/opt && cd $install_dir/opt | |||
git clone https://github.com/csdms/wmt-exe | |||
cd wmt-exe | |||
python setup.py develop | |||
Create a site configuration file that describes the executor and symlink it to the executor's '''etc/''' diectory. | |||
python setup.py configure --wmt-prefix=$install_dir --launch-dir='/scratch/$USER/wmt/permafrost' --exec-dir='/scratch/$USER/wmt/permafrost' | |||
ln -s "$(realpath wmt.cfg)" $install_dir/conda/etc | |||
Here I've configured the executor to do stuff in the '''/scratch''' directory instead of the default '''~/.wmt'''. |
Revision as of 16:40, 23 March 2018
Instructions for installing and configuring a WMT executor on blanca.
--Mpiper (talk) 16:48, 19 March 2018 (MDT)
Set install directory
The install directory for this executor is /projects/mapi8461/wmt/permafrost.
install_dir=/projects/mapi8461/wmt/permafrost mkdir -p $install_dir
Install Python
Install a Python distribution to be used locally by WMT. We like to use Miniconda.
cd $install_dir curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -o miniconda.sh bash ./miniconda.sh -f -b -p $(pwd)/conda export PATH=$(pwd)/conda/bin:$PATH
If working with an existing Miniconda install, be sure to update everything before continuing:
conda update conda conda update --all
Install the CSDMS software stack
A limiting factor in setting up this executor is the componentized version of ILAMB can only be run with `pymt=0.2`; in particular, the BMI I wrote for ILAMB is incompatible with the `model_metadata` package in newer versions of PyMT. Further, as of 2018-03-20, the `netcdf-fortran` package needs to be wound back so that HDF5 1.8.18 is used because of problems with HDF5 1.10.1 and ESMF, used in PyMT. To ensure all the packages meet the same dependencies (looking at you, numpy), simultaneously install `cca-tools`, `netcdf-fortran`, `pymt=0.2` and `csdms-ilamb` using the csdms-stack conda channel (the Bakery),
conda install cca-tools netcdf-fortran=4.4.4=5 pymt=0.2 csdms-ilamb -c csdms-stack -c defaults -c conda-forge
Install the `babelizer` for components that need to be built from source.
conda install -c csdms-stack babelizer
Optionally install IPython for testing.
conda install ipython
Note that when running IPython remotely on siwenna, it's helpful to set
export MPLBACKEND=Agg
Install executor software
Install the `wmt-exe` package from source.
mkdir -p $install_dir/opt && cd $install_dir/opt git clone https://github.com/csdms/wmt-exe cd wmt-exe python setup.py develop
Create a site configuration file that describes the executor and symlink it to the executor's etc/ diectory.
python setup.py configure --wmt-prefix=$install_dir --launch-dir='/scratch/$USER/wmt/permafrost' --exec-dir='/scratch/$USER/wmt/permafrost' ln -s "$(realpath wmt.cfg)" $install_dir/conda/etc
Here I've configured the executor to do stuff in the /scratch directory instead of the default ~/.wmt.