Temporary Directories

Temporary Directory Guidance Changed Jun 29, 2026

A Slurm update on Jun 29, 2026 changed how users should use the local disk on HiPerGator.

Users should now use $TMPDIR or the /tmp directory directly.

Job Temporary Directory

When a SLURM job starts, the scheduler creates a custom temporary directory for the job on the compute node's fast, flash-based local hard drive. This directory also works great for jobs that need to use or generate many small files, as the main UFIT-RC parallel filesystem (Blue storage) is optimized for large file streaming and is less suitable for small files.

The job's temporary directory is owned by the user running the job. At the end of the job, the temporary directory is automatically removed unless the job fails, in which case it's temporarily retained for troubleshooting.

This directory is accessible within the job via the $TMPDIR environment variable, or directly at /tmp or /var/tmp.

You can use the ${TMPDIR} variable as a filesystem path in the shell script portion of the job script as an argument for any commands you run in a job.

You can also use the ${TMPDIR} variable in shell commands at the beginning of the job to copy data to the temporary job directory (staging out) and copy results back into /blue at the end of the job script.

Default Temporary Path

Many applications and programming languages use the $TMPDIR environment variable, if available, as the default temporary directory path.

If a personal Apptainer or Singularity container is used, make sure that the $APPTAINERENV_TMPDIR or $SINGULARITYENV_TMPDIR variable is set within the job to export the local scratch location into the Singularity container.

FAQ about temporary directories

What about the older /local/scratch/$JOB_ID directories?

Under the hood, those directories are where the new /tmp and /var/tmp reside, but users can no longer access them directly at the older path. Simply use /tmp, $TMPDIR, or /var/tmp. Linux namespaces are used to map those directories to the local flash drive.

If I SSH into a server where I have multiple jobs, I only see one job's /tmp folder. How can I see others?

HiPerGator uses Slurm's pam_slurm_adopt feature so that when a user SSHs into a compute node, their session is "adopted" into the user's running job on that node. This prevents users from using resources (CPUs, RAM, and GPUs) beyond those allocated to them and impacting other users. If the user has multiple jobs on a node, the highest job ID is selected to adopt into.

To attach to a specific job, and have access to that job's temporary folders, you can use the srun command (replacing YOUR_JOBID with the job id of the job you want to connect to.):

srun --pty --overlap --jobid YOUR_JOBID bash