|
|
| Line 1: |
Line 1: |
| = HPCC guidelines =
| | Please see the |
| == Access methods ==
| | [[HPC]] |
| | | page for the latest information on this topic. |
| You have several options for accessing the CSDMS cluster. You can use:
| |
| * Secure SHell (SSH) for direct access
| |
| * Transferring files via SSH scp or sftp
| |
| * To run selected models you can use the [https://csdms.colorado.edu/wiki/WMT_information| CSDMS Web Modeling Tool ]
| |
| | |
| === SSH - Secure SHell ===
| |
| | |
| [http://en.wikipedia.org/wiki/Secure_Shell SSH] encrypts transmissions between remote computers and secure computers. Use SSH for local and remote access to beach.colorado.edu.
| |
| | |
| To use SSH for remote computing on beach, you must have SSH software installed on your local computer. Your SSH client must be compatible with that installed on beach. If you do not have SSH installed on your local computer, see our SSH product page for names and sources of some available SSH software.
| |
| | |
| When you are familiar with SSH and have a client installed on your local computer, you will be able to SSH and/or scp to beach.
| |
| | |
| There are both free and commercial versions of ssh available for many platforms. For Unix/Linux-based systems (including Mac OS X) the most common ssh software is OpenSSH. OpenSSH is free and can be downloaded as either source or binaries. Microsoft Windows users also have several choices for ssh client support, including:
| |
| * [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]
| |
| * [http://www.cryptzone.com/products/agmindterm/ MindTerm], and
| |
| * [https://www2.stat.duke.edu/comp/security/install-ttssh.html TTSSH]
| |
| | |
| == Transferring files==
| |
| === Using secure file transfer protocol (sftp)===
| |
| | |
| ''sftp'' is an interactive file transfer program similar to [http://en.wikipedia.org/wiki/File_Transfer_Protocol FTP]. ''sftp'' performs all operations over an encrypted [http://en.wikipedia.org/wiki/Secure_Shell SSH] transport. It may also use many features of SSH, such as public key authentication and compression. ''sftp'' connects and logs into the specified host, then enters an interactive command mode.
| |
| | |
| From within ''sftp'', all of the normal ftp commands are present. ''sftp'' does not support anonymous logins. A user account is required on the remote end.
| |
| | |
| For usage information, see the [http://www.openbsd.org/cgi-bin/man.cgi?query=sftp&sektion=1 man page] for ''sftp'' on the computer from which you are issuing the command.
| |
| | |
| === Copying files using secure copy (scp) ===
| |
| | |
| The secure copy command (''scp'') works with secure shell (SSH).
| |
| | |
| For usage information, see the [http://www.openbsd.org/cgi-bin/man.cgi?query=scp&sektion=1 man page] for ''scp'' on the computer from which you are issuing the command.
| |
| | |
| :'''Copy a file from beach to your local computer'''
| |
| :: <syntaxhighlight lang=bash>
| |
| > scp your_username@beach.colorado.edu:foo.txt /some/local/directory
| |
| </syntaxhighlight>
| |
| :'''Copy a file from your local computer to beach'''
| |
| :: <syntaxhighlight lang=bash>
| |
| > scp bar.txt your_username@beach.colorado.edu:/some/remote/directory
| |
| </syntaxhighlight>
| |
| :'''Copy a directory from beach to your local computer'''
| |
| :: <syntaxhighlight lang=bash>
| |
| > scp -r your_username@beach.colorado.edu:foo /some/local/directory/bar
| |
| </syntaxhighlight>
| |
| | |
| === Copying files using rsync ===
| |
| [http://rsync.samba.org/ rsync] is an open source utility that provides fast incremental file transfer that also works with ssh.
| |
| | |
| : '''Copy a file from a compute node to beach'''
| |
| ::<syntaxhighlight lang=bash>
| |
| > rsync -av -e ssh --progress --stats --bwlimit=1024 host:/dir/file.ext /data/dir
| |
| </syntaxhighlight>
| |
| | |
| == Environment modules ==
| |
| The [http://modules.sourceforge.net/ environment modules] utility is available on beach for modifying your environment to find alternate compilers and software installed on beach.
| |
| | |
| To use modules you will need to add the following line to your startup rc file. If you are a C-like shell (csh, tcsh) user, add the following to your .cshrc file after all other path-setting commands,
| |
| <syntaxhighlight lang=bash>
| |
| source /contrib/Modules/3.2.6/init/csh
| |
| </syntaxhighlight>
| |
| | |
| If you are a Korn-like shell (sh, ksh, bash, etc.) user, add the following line to your .profile file,
| |
| <syntaxhighlight lang=bash>
| |
| . /contrib/Modules/3.2.6/init/ksh
| |
| </syntaxhighlight>
| |
| After this setup is executed upon login, you can use the module command.
| |
| | |
| :'''To see which modules are in force''':
| |
| ::<syntaxhighlight lang=bash>
| |
| > module list
| |
| </syntaxhighlight>
| |
| :'''To load a new module (the intel compilers for example)''':
| |
| ::<syntaxhighlight lang=bash>
| |
| > module load intel
| |
| </syntaxhighlight>
| |
| | |
| :'''To show all available modulefiles:'''
| |
| ::<syntaxhighlight lang=bash>
| |
| > module avail
| |
| </syntaxhighlight>
| |
| :'''For help:'''
| |
| ::<syntaxhighlight lang=bash>
| |
| > module help
| |
| </syntaxhighlight>
| |
| | |
| Please see our [[HPCC Env Modules|Environment Modules]] page for a more complete description.
| |
| | |
| == Running jobs on beach ==
| |
| Beach uses [http://www.adaptivecomputing.com/products/open-source/torque/ Torque] as it's scheduling system. Please see the [http://docs.adaptivecomputing.com/torque/4-2-7/Content/topics/12-appendices/torqueQuickStartGuide.htm Torque website] for a complete users' guide, or our cheat sheet for [[HPCC_usage_rules|Torque usage on beach]].
| |
| | |
| Note that jobs must be submitted to the cluster through Torque's qsub command. Jobs not submitted in this way, or jobs that are run on the head node will be unceremoniously killed.
| |
| | |
| == File storage ==
| |
| Although we do not yet actively enforce disk quotas, we ask that users limit their disk usage in the following manner:
| |
| * '''/home''': We ask users to limit their disk usage in their home directories to '''5GB'''. All files under /home are backed up to a tape storage device.
| |
| * '''/data''': If users require more storage, they can request space on /data. Data held here should be regarded as temporary as it is not backed up. Valuable data should be either moved to /home (if it isn't too large) or, better yet, downloaded to a user's local storage device.
| |
| | |
| == Backups ==
| |
| The backup schedule for /home is the following:
| |
| * Full backup: First Saturday of every month
| |
| * Incremental backup: Sunday-Friday
| |
| * Differentials: Saturday
| |
| Depending on disk usage, and data compressibility our backup retention is between '''3 and 4 months'''.
| |
