Help:HPCC Torque Queues: Difference between revisions

From CSDMS
Huttone (talk | contribs)
oauth-consumer, Administrators
1,010 edits
Create a page that describes the job queues on the HPCC
 
m Add very important image caption
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Job Queues on the CSDMS HPCC =
= Job Queues on the CSDMS HPCC =
[[File:Eiffel_Tower_Line_(Medium).jpg | right  | 300px |  Photo by Flickr user gadl used under a Creative Commons license | thumb | ''The queues on beach are much shorter.'']]


We have defined a set of queues that you can submit your jobs to on beach.  Although you can submit your job to a queue directly, generally your job will be routed to an appropriate queue, given the resources you request.  As such, it is important that you specify what resources you will need.  Of particular importance in the amount of memory that your job will use and the amount of time you estimate your job will take to run.
Please note that some of these queues have maximum runtimes associated with them.  This means that '''your job will be killed''' if it runs over the queue's time limit.
== Cheat Sheet ==
Job queues that you can submit to directly,
{|
! align=left width=75 | Queue
! align=left width=125 | Max Run Time
! align=left width=125 | Max Memory
! align=left width=125 | Max Nodes
! align=left | Description
|-
| debug
| 2 hours
| 16 GB
| 54
| Short jobs that might be used for testing
|-
| vip
| 1 day
| 16 GB
| 54
| Important jobs that need to get done ''now''
|}
Queues that you job will be routed to based on resource needs that you specify,
{|
{|
! align=left width=200 | Queue
! align=left width=75 | Queue
! align=left width=125 | Max Run Time
! align=left width=125 | Max Memory
! align=left width=125 | Max Nodes
! align=left | Description
! align=left | Description
|-
|-
| Default
| default
| 4 days
| 16 GB
| 54
| Jobs that should be done within a few days
| Jobs that should be done within a few days
|-
|-
| Fat
| long
| Memory intensive jobs
| unlimited
|-
| 16 GB
| Long
| 54
| Jobs that will run for a long time
| Jobs that will run for a long time
|-
|-
| VIP
| himem
| Important jobs that need to get done ''now''
| unlimited
|-
| 32 GB
| Debug
| 8
| Short jobs that might be used for testing
| Jobs that need to use memory-rich nodes
|}
|}
== Queue Status ==
Use <tt>qstat</tt> to get an up-to-date summary of the queues.
<syntaxhighlight lang="bash" lines="0">
> qstat -q
Queue            Memory CPU Time Walltime Node  Run Que Lm  State
---------------- ------ -------- -------- ----  --- --- --  -----
debug              --  02:00:00    --      --    2  0 --  E R
vip                --  24:00:00    --      --    0  0 --  E R
ocean-owner        --      --      --      --    0  0 --  E R
route              --      --      --      --    0  0 --  E R
himem              --      --      --      --    7  0 --  E R
long              15gb    --      --      --    1  0 12  E R
default            15gb 96:00:00 96:00:00  --    0  0 --  E R
total              --      --      --      --    0  0 --  E R
wrf                --      --      --      --    0  0 --  E R
ocean              --      --      --      --    0  0 --  E R
wrf-owner          --      --      --      --    0  0 --  E R
                                              ----- -----
                                                  10    0
</syntaxhighlight>

Latest revision as of 16:02, 4 November 2015

Job Queues on the CSDMS HPCC

The queues on beach are much shorter.

We have defined a set of queues that you can submit your jobs to on beach. Although you can submit your job to a queue directly, generally your job will be routed to an appropriate queue, given the resources you request. As such, it is important that you specify what resources you will need. Of particular importance in the amount of memory that your job will use and the amount of time you estimate your job will take to run.

Please note that some of these queues have maximum runtimes associated with them. This means that your job will be killed if it runs over the queue's time limit.

Cheat Sheet

Job queues that you can submit to directly,

Queue Max Run Time Max Memory Max Nodes Description
debug 2 hours 16 GB 54 Short jobs that might be used for testing
vip 1 day 16 GB 54 Important jobs that need to get done now

Queues that you job will be routed to based on resource needs that you specify,

Queue Max Run Time Max Memory Max Nodes Description
default 4 days 16 GB 54 Jobs that should be done within a few days
long unlimited 16 GB 54 Jobs that will run for a long time
himem unlimited 32 GB 8 Jobs that need to use memory-rich nodes

Queue Status

Use qstat to get an up-to-date summary of the queues. 

> qstat -q
Queue            Memory CPU Time Walltime Node  Run Que Lm  State
---------------- ------ -------- -------- ----  --- --- --  -----
debug              --   02:00:00    --      --    2   0 --   E R
vip                --   24:00:00    --      --    0   0 --   E R
ocean-owner        --      --       --      --    0   0 --   E R
route              --      --       --      --    0   0 --   E R
himem              --      --       --      --    7   0 --   E R
long               15gb    --       --      --    1   0 12   E R
default            15gb 96:00:00 96:00:00   --    0   0 --   E R
total              --      --       --      --    0   0 --   E R
wrf                --      --       --      --    0   0 --   E R
ocean              --      --       --      --    0   0 --   E R
wrf-owner          --      --       --      --    0   0 --   E R
                                               ----- -----
                                                  10     0
Retrieved from "https://csdms.colorado.edu/csdms_wiki/index.php?title=Help:HPCC_Torque_Queues&oldid=98092"