GPU Access¶

Normalized Graphics Processor Units (NGUs) include all of the infrastructure (memory, network, rack space, cooling) necessary for GPU-accelerated computation. Each NGU is equivalent to 1 GPU presently, however newer GPUs such as the A100s may require more than 1 NGU to access in the future.

Researchers can add NGUs to their allocations by filling out the Purchase Form or requesting a Trial Allocation.

Open On Demand Access¶

To access GPUs using Open-On-Demand, please check the form for your application. If your application supports multiple GPU types, choose the GPU partition and specify number of GPUs and type:

To request access to one GPU (of any type, use this gres string):
- gpu:1
To request multiple GPUs (of any type, use this gres string where n is the number of GPUs you need):
- gpu:n
To request a specific type of GPU, use this gres string (requesting GeForce GPUs in this example):
- gpu:geforce:1
To request an A100 GPU, use this gres string:
- gpu:a100:1

GPU-enabled Services¶

Types of GPUs are listed below. Two partitions contain GPUs - the hwgui partition for visualization and the gpu partition for general computation.

Hardware Accelerated GUI¶

GPUs in these servers are used to accelerate rendering for graphical applications. These servers are in the SLURM "hwgui" partition. To do this:

Select the 'hwgui' partition for an Open-On-Demand console or desktop session. See Open OnDemand for details on using OOD.
load the appropriate environment module and launch the application in question.

GPU Assisted Computation¶

A number of high performance applications installed on HiPerGator implement GPU-accelerated computing functions via CUDA to achieve significant speed-up over CPU calculations. These servers are in the SLURM "gpu" partition (--partition=gpu).

Hardware Specifications for the GPU Partition¶

We have the following types of NVIDIA GPU nodes available in the "gpu" partition:

GPU Specs	GeForce 1080Ti	GeForce 2080Ti	GeForce 2080Ti	Quadro RTX 6000 SLI	NVIDIA A100 NVSWITCH
Host Quantity	1	32	38	6	140
Host Architecture	Intel Haswell	Intel Skylake	Intel Cascade Lake	Intel Cascade Lake	AMD EPYC ROME
Host Memory	128 GB	187 GB	187 GB	187 GB	2 TB
Host Interconnect	FDR IB	EDR IB	EDR IB	EDR IB	HDR IB
CPUs per Host	28	32	32	32	128
CPUS per Socket	14	16	16	16	16
GPUs per Host	2	8	8	8	8
CPUs per GPU	14	4	4	4	16
Memory per GPU	11GB	11GB	11GB	23GB	80GB
SLURM Feature	n/a	2080ti	2080ti	rtx6000	a100
GRES GPU type	geforce	geforce	geforce	quadro	a100
Technical Ref	Specifications	Specifications	Specifications	Specifications	Specifications

For a list of additional node features, see the Available Node Features page.

To select a specific type of GPU within a partition please use either a SLURM constraint (e.g. --constraint=rtx6000) or a GRES with the needed GPU type (--gres or --gpu=a100:1).

Compiling CUDA Enabled Programs¶

The most direct way to develop a custom GPU accelerated algorithm is with the CUDA programming, please refer to the Nvidia CUDA Toolkit page. The current CUDA environment is cuda/11. However, C++ or Python packages numba and PyCuda are other ways to program GPU algorithms.

Slurm and GPU Use¶

View instructions for using GPUs and scheduling GPU jobs with SLURM at Slurm and GPU Use