Disk Space Management
Introduction
To ensure adequate storage resources for all VHPC users, we've implemented a combination of filesystem quotas and automated cleanup scripts to help manage VHPC storage resources.
Disk Space
Home Directory
Your home directory will be small and meant for storing only necessary configuration files. All users will have a quota on storage space in their home directory (see below for details). At the soft limit, you will receive a warning. At the hard limit, you can only delete or move files until you are below the soft limit. The home directory is the only user space which is backed up.
| Quota Type | Soft Limit | Hard Limit |
|---|---|---|
| Disk Space | 10 GB | 12 GB |
Work Folder
This is the location for large files used in currently running processes. This is a shared resource among all users of the cluster. Leaving files, even briefly, that are not actively needed limits the ability of others to use the cluster.
Because of the size of the files sometimes required for HPC computations, there are no quotas on this filesystem. We ask that you be cognizant and considerate of others in your usage.
This space is optimized for speed and should be treated as volatile storage. As such, there are no redundancies in this file system. Additionally, this filesystem is not backed up, and there are no options for file recovery.
Data needed for calculations should be uploaded (copied) directly to this folder. Results should be downloaded as soon as possible once calculations are completed.
Stage Folder
The stage file system is a place to stage large files which will be needed for jobs. Files should be uploaded here and then later copied to the work folder. As a shared resource, users should be cognizant of their usage. This space is not backed up.
There is automated cleanup of this space, similar to the work folder. In situations where space is running low, you may be required to remove files. You will be notified via email. If overall available space is less than 10%, account holders will be asked to remove files within two days. Should available space fall below 5%, system administrators may be forced to delete files.
Automatic Cleaning Scripts
Filesystem usage on the Work and Stage filesystems will be enforced by automated cleaning scripts. There will be a soft limit where you will receive warnings about stale files. Then there is a hard limit at which point files will be deleted. Emails will be sent daily notifying you of any file older than the soft limit. If files pass the hard limit, they will be deleted without further notice. The list of files in violation will be in your home directory in a file named WORK.TO.BE.DELETED.txt.
Filesystem |
Soft Limit (Days) |
Hard Limit (Days) |
|---|---|---|
| work | 30 | 44 |
| stage | 90 | 104 |
Users may request a 30-day extension by opening a ticket with the Service Desk. After two consecutive extensions, arrangements will need to be discussed with Information Technology staff.
If overall available space on either file system is less than 10%, system administrators will reach out to users to accelerate file removal. In critical situations, when available space drops below 5%, the system administrators may be forced to remove files earlier to maintain system integrity.