With funding from the Blavatnik Institute, HMS IT has significantly increased its graphical processing unit (GPU) infrastructure resources. GPUs accelerate image analysis as well as supporting rapidly growing Data Science application needs. Importantly, these resources enable the Blavatnik Center for Computational Biomedicine mission.

We have installed 71 new GPU cards, comprised of the following:

• 47 RTX 8000 single precision cards
• 24 Tesla V100S double precision cards

These cards are interconnected via 40 Gigabit per second (Gbps) networking to reduce data latency, and also connected to a 1 Pebibyte, all-flash scratch filesystem dedicated to the new GPU nodes.

We gratefully acknowledge Dell Technologies, which provided 28 of the 71 cards.

At this time, the new GPUs are available only for labs with a primary or secondary appointment in a pre-clinical HMS department. Most affiliate labs will therefore be limited to using the existing gpu and gpu_requeue partitions that utilize our existing GPU infrastructure. If your PI has an appointment in a pre-clinical HMS department but you see an error message saying you cannot submit to the gpu_quad partition, please contact us. Each user can use up to 15 TiB of storage on the GPU scratch filesystem. Like the existing O2 scratch filesystem, files not accessed for 30 days will be deleted.

To run jobs on these new cards:

• submit your jobs to the gpu_quad partition instead of the standard gpu partition. 
• If your jobs require GPU double-precision, add the following option to your normal sbatch/srun command, which will exclude the RTX8000s and use only V100s’. This is only required for double-precision jobs:
  • --constraint=gpu_doublep

For more information, please refer to:

• GPUs on O2 wiki page.

If you have any questions or concerns, please email rchelp@hms.harvard.edu