HPCC guidelines
HPCC guidelines
Access methods
You have several options for accessing the CSDMS cluster. You can use:
- Secure SHell (SSH) for direct access
- Transferring files via SSH scp or sftp
- To run selected models you can use the CSDMS Web Modeling Tool
SSH - Secure SHell
SSH encrypts transmissions between remote computers and secure computers. Use SSH for local and remote access to beach.colorado.edu.
To use SSH for remote computing on beach, you must have SSH software installed on your local computer. Your SSH client must be compatible with that installed on beach. If you do not have SSH installed on your local computer, see our SSH product page for names and sources of some available SSH software.
When you are familiar with SSH and have a client installed on your local computer, you will be able to SSH and/or scp to beach.
There are both free and commercial versions of ssh available for many platforms. For Unix/Linux-based systems (including Mac OS X) the most common ssh software is OpenSSH. OpenSSH is free and can be downloaded as either source or binaries. Microsoft Windows users also have several choices for ssh client support, including:
Transferring files
Using secure file transfer protocol (sftp)
sftp is an interactive file transfer program similar to FTP. sftp performs all operations over an encrypted SSH transport. It may also use many features of SSH, such as public key authentication and compression. sftp connects and logs into the specified host, then enters an interactive command mode.
From within sftp, all of the normal ftp commands are present. sftp does not support anonymous logins. A user account is required on the remote end.
For usage information, see the man page for sftp on the computer from which you are issuing the command.
Copying files using secure copy (scp)
The secure copy command (scp) works with secure shell (SSH).
For usage information, see the man page for scp on the computer from which you are issuing the command.
- Copy a file from beach to your local computer
> scp your_username@beach.colorado.edu:foo.txt /some/local/directory
- Copy a file from your local computer to beach
> scp bar.txt your_username@beach.colorado.edu:/some/remote/directory
- Copy a directory from beach to your local computer
> scp -r your_username@beach.colorado.edu:foo /some/local/directory/bar
Copying files using rsync
rsync is an open source utility that provides fast incremental file transfer that also works with ssh.
- Copy a file from a compute node to beach
> rsync -av -e ssh --progress --stats --bwlimit=1024 host:/dir/file.ext /data/dir
Environment modules
The environment modules utility is available on beach for modifying your environment to find alternate compilers and software installed on beach.
To use modules you will need to add the following line to your startup rc file. If you are a C-like shell (csh, tcsh) user, add the following to your .cshrc file after all other path-setting commands,
source /contrib/Modules/3.2.6/init/csh
If you are a Korn-like shell (sh, ksh, bash, etc.) user, add the following line to your .profile file,
. /contrib/Modules/3.2.6/init/ksh
After this setup is executed upon login, you can use the module command.
- To see which modules are in force:
> module list
- To load a new module (the intel compilers for example):
> module load intel
- To show all available modulefiles:
> module avail
- For help:
> module help
Please see our Environment Modules page for a more complete description.
Running jobs on beach
Beach uses Torque as it's scheduling system. Please see the Torque website for a complete users' guide, or our cheat sheet for Torque usage on beach.
Note that jobs must be submitted to the cluster through Torque's qsub command. Jobs not submitted in this way, or jobs that are run on the head node will be unceremoniously killed.
File storage
Although we do not yet actively enforce disk quotas, we ask that users limit their disk usage in the following manner:
- /home: We ask users to limit their disk usage in their home directories to 5GB. All files under /home are backed up to a tape storage device.
- /data: If users require more storage, they can request space on /data. Data held here should be regarded as temporary as it is not backed up. Valuable data should be either moved to /home (if it isn't too large) or, better yet, downloaded to a user's local storage device.
Backups
The backup schedule for /home is the following:
- Full backup: First Saturday of every month
- Incremental backup: Sunday-Friday
- Differentials: Saturday
Depending on disk usage, and data compressibility our backup retention is between 3 and 4 months.