This Quick Reference sheet is designed for people already familiar with the HPCC system at MSU. It has been specifically designed to focus on settings specific to MSU HPCC and help users who use many different HPC systems keep track of the differences. Users also find the following guide useful to print out and put on their desk:
HPCC Related URLs
Regular SSH Login
Eval-Node regular Login
Remote Desktop Gateway (from campus-only)
From gateway (ssh gateway.hpcc.msu.edu) you can log into a developer node using (ssh "Node Name"):
|dev-intel14||20||256GB||Large memory intel14 node|
|dev-intel14-k20||20||128GB||Two Nvidia K20 GPUs|
|dev-intel14-phi||20||128GB||Two Xeon Phi accelerators|
|dev-intel16||28||128GB||Two 2.4Ghz 14-core Intel Xeon E5-2680v4|
|dev-intel16-k80||28||256GB||Intel16 node with 4 Nvidia Tesla K80 GPUs|
The following hardware is available on the hpcc main cluster.
|intel14||220||20 (plus accelarators)||4400||250GB||450GB||css-###, csm-###, csn-###, csp-###|
|intel14-xl||5||48-96||288||1 TB - 6 TB||800 GB||qml-###|
128 GB (x290)
256 GB (x24)
512 GB (x6)
19968 stream processors
User home directory
Research home directory
Scratch space for fast shared file access.
Temporary local scratch space allocated to each job.
General Script options:
Common Scheduler Commands:
Submit a submission script to the scheduler.
showq -u $USER
Show the jobs of the current user.
Delete a job from the queue.
qdel $(qselect -u username)
Delete all jobs of a particular user from the queue.
See the current system utilization status.
checkjob -v <jobid>
Check the details of a particular job.
showstart -e all <jobid>
Show estimated start times for a job.
Common Module Commands:
Show currently available modules.
List currently loaded modules.
module show <module name>
Show what is changed by a module.
module unload <module name>
Unload a currently loaded module.
module load <module name>
Load an available module.
Commonly used PBS environment variables
Name of the computer currently running the script.This should be one of the nodes listed in the PBS_NODEFILE
User Name (NetID). Useful if you would like to dynamically generate a directory on some scratch space.
Directory where the qsub command was executed. Useful with the cd (change directory) command to change your current directory to your working directory.
Local temporary disk storage unique to each node and each job. This directory is automatically created at the begining of the job and deleted at the end of the job.
Job ID number given to this job. This number is used by many of the job monitoring programs such as qstat, showstart, and dque.
Name of the job. This can be set using the -N option in the PBS script (or from the command line). The default job name is the name of the PBS script.
Name of the file that contains a list of the HOSTS provided for the job.
Array ID numbers for jobs submitted with the -t flag. For example a job submitted with #PBS -t 1-8 will run eight identical copies of the shell script. The value of the PBS_ARRAYID will be a an integer between 1 and 8.
Used with pbsdsh to determine the task number of each processor. For more information see http://www.ep.ph.bham.ac.uk/general/support/torquepbsdsh.html.
Original PBS path. Used with pbsdsh.
Number of Processors Per Node requested by the current job (useful for hybrid code).
|PBS_NP||Total number of cores requested (Nodes*PPN)|
|PBS_O_HOST||Current Host of PBS job|
|PBS_NUM_NODES||Number of requested nodes|