Project-Centric Data Storage

Project directories provide members of a project with a common place to store code, data, and other files related to their project.

Project Home Directories (NFS)

Area Path Type Permissions Quota Backups Purged Retention On Compute Nodes
Project Home /ccs/proj/[projid] NFS 770 50 GB Yes No 90 days Read-only

Projects are provided with a Project Home storage area in the NFS-mounted filesystem. This area is intended for storage of data, code, and other files that are of interest to all members of a project. Since Project Home is an NFS-mounted filesystem, its performance will not be as high as other filesystems.

Project Home Path

Project Home area is accessible at /ccs/proj/abc123 (where abc123 is your project ID).

Project Home Quotas

To check your project’s current usage, run df -h /ccs/proj/abc123 (where abc123 is your project ID). Quotas are enforced on project home directories. The current limit is shown in the table above.

Project Home Permissions

The default permissions for project home directories are 0770 (full access to the user and group). The directory is owned by root and the group includes the project’s group members. All members of a project should also be members of that group-specific project. For example, all members of project “ABC123” should be members of the “abc123” UNIX group.

Project Home Backups

If you accidentally delete files from your project home directory (/ccs/proj/[projid]), you may be able to retrieve them. Online backups are performed at regular intervals. Hourly backups for the past 24 hours, daily backups for the last 7 days, and once-weekly backups are available. It is possible that the deleted files are available in one of those backups. The backup directories are named hourly.*, daily.*, and weekly.* where * is the date/time stamp of backup creation. For example, hourly.2020-01-01-0905 is an hourly backup made on January 1st, 2020 at 9:05 AM.

The backups are accessed via the .snapshot subdirectory. Note that ls alone (or even ls -a) will not show the .snapshot subdirectory exists, though ls .snapshot will show its contents. The .snapshot feature is available in any subdirectory of your project home directory and will show the online backups available for that subdirectory.

To retrieve a backup, simply copy it into your desired destination with the cp command.

Project Work Areas

Three Project Work Areas to Facilitate Collaboration

To facilitate collaboration among researchers, the OLCF provides (3) distinct types of project-centric work storage areas: Member Work directories, Project Work directories, and World Work directories. Each directory should be used for storing files generated by computationally-intensive HPC jobs related to a project.

Area Path Type Permissions Quota Backups Purged Retention On Compute Nodes
Member Work /gpfs/alpine/[projid]/scratch/[userid] Spectrum Scale 700 [1] 50 TB No 90 days N/A [2] Yes
Project Work /gpfs/alpine/[projid]/proj-shared Spectrum Scale 770 50 TB No 90 days N/A [2] Yes
World Work /gpfs/alpine/[projid]/world-shared Spectrum Scale 775 50 TB No 90 days N/A [2] Yes

Footnotes

[1]Permissions on Member Work directories can be controlled to an extent by project members. By default, only the project member has any accesses, but accesses can be granted to other project members by setting group permissions accordingly on the Member Work directory. The parent directory of the Member Work directory prevents accesses by “UNIX-others” and cannot be changed (security measures).
[2](1, 2, 3) Retention is not applicable as files will follow purge cycle.

On Summit, Rhea and the DTNs, additional paths to the various project-centric work areas are available via the following symbolic links and/or environment variables:

  • Member Work Directory: /gpfs/alpine/scratch/[userid]/[projid] or $MEMBERWORK/[projid]
  • Project Work Directory: /gpfs/alpine/proj-shared/[projid] or $PROJWORK/[projid]
  • World Work Directory: /gpfs/alpine/world-shared/[projid] or $WORLDWORK/[projid]

The difference between the three lies in the accessibility of the data to project members and to researchers outside of the project. Member Work directories are accessible only by an individual project member by default. Project Work directories are accessible by all project members. World Work directories are readable by any user on the system.

Permissions

UNIX Permissions on each project-centric work storage area differ according to the area’s intended collaborative use. Under this setup, the process of sharing data with other researchers amounts to simply ensuring that the data resides in the proper work directory.

  • Member Work Directory: 700
  • Project Work Directory: 770
  • World Work Directory: 775

For example, if you have data that must be restricted only to yourself, keep them in your Member Work directory for that project (and leave the default permissions unchanged). If you have data that you intend to share with researchers within your project, keep them in the project’s Project Work directory. If you have data that you intend to share with researchers outside of a project, keep them in the project’s World Work directory.

Backups

Member Work, Project Work, and World Work directories are not backed up. Project members are responsible for backing up these files, either to Project Archive areas (HPSS) or to an off-site location.

Project Archive Directories

Projects are also allocated project-specific archival space on the High Performance Storage System (HPSS). The default quota is shown on the table below. If a higher quota is needed, contact the User Assistance Center.

Three Project Archive Areas Facilitae Collaboration on Archival Data

To facilitate collaboration among researchers, the OLCF provides (3) distinct types of project-centric archival storage areas: Member Archive directories, Project Archive directories, and World Archive directories. These directories should be used for storage of data not immediately needed in either the Project Home (NFS) areas or Project Work (Alpine) areas and to serve as a location to store backup copies of project-related files.

Area Path Type Permissions Quota Backups Purged Retention On Compute Nodes
Member Archive /hpss/prod/[projid]/users/$USER HPSS 700 100 TB No No 90 days No
Project Archive /hpss/prod/[projid]/proj-shared HPSS 770 100 TB No No 90 days No
World Archive /hpss/prod/[projid]/world-shared HPSS 775 100 TB No No 90 days No

As with the three project work areas, the difference between these three areas lies in the accessibility of data to project members and to researchers outside of the project. Member Archive directories are accessible only by an individual project member by default, Project Archive directories are accessible by all project members, and World Archive directories are readable by any user on the system.

Permissions

UNIX Permissions on each project-centric archive storage area differ according to the area’s intended collaborative use. Under this setup, the process of sharing data with other researchers amounts to simply ensuring that the data resides in the proper archive directory.

  • Member Archive Directory: 700
  • Project Archive Directory: 770
  • World Archive Directory: 775

For example, if you have data that must be restricted only to yourself, keep them in your Member Archive directory for that project (and leave the default permissions unchanged). If you have data that you intend to share with researchers within your project, keep them in the project’s Project Archive directory. If you have data that you intend to share with researchers outside of a project, keep them in the project’s World Archive directory.

Project Archive Access

Project Archive directories may only be accessed via utilities called HSI and HTAR. For more information on using HSI or HTAR, see the HPSS Data Archival System section.