Project-Centric Data Storage¶
Project directories provide members of a project with a common place to store code, data, and other files related to their project.
Project Home Directories (NFS)¶
Projects are provided with a Project Home storage area in the NFS-mounted filesystem. This area is intended for storage of data, code, and other files that are of interest to all members of a project. Since Project Home is an NFS-mounted filesystem, its performance will not be as high as other filesystems.
Project Home Path¶
Project Home area is accessible at
abc123 is your project ID).
Project Home Quotas¶
To check your project’s current usage, run
df -h /ccs/proj/abc123
abc123 is your project ID). Quotas are enforced on project
home directories. The current limit is shown in the table above.
Project Home Permissions¶
The default permissions for project home directories are
access to the user and group). The directory is owned by root and the
group includes the project’s group members. All members of a project
should also be members of that group-specific project. For example, all
members of project “ABC123” should be members of the “abc123” UNIX
Three Project Work Areas to Facilitate Collaboration¶
To facilitate collaboration among researchers, the OLCF provides (3) distinct types of project-centric work storage areas: Member Work directories, Project Work directories, and World Work directories. Each directory should be used for storing files generated by computationally-intensive HPC jobs related to a project.
||Spectrum Scale||700||no||90 days||50TB|
||Spectrum Scale||770||no||90 days||50TB|
||Spectrum Scale||775||no||90 days||50TB|
The difference between the three lies in the accessibility of the data to project members and to researchers outside of the project. Member Work directories are accessible only by an individual project member by default. Project Work directories are accessible by all project members. World Work directories are readable by any user on the system.
UNIX Permissions on each project-centric work storage area differ according to the area’s intended collaborative use. Under this setup, the process of sharing data with other researchers amounts to simply ensuring that the data resides in the proper work directory.
- Member Work Directory:
- Project Work Directory:
- World Work Directory:
For example, if you have data that must be restricted only to yourself, keep them in your Member Work directory for that project (and leave the default permissions unchanged). If you have data that you intend to share with researchers within your project, keep them in the project’s Project Work directory. If you have data that you intend to share with researchers outside of a project, keep them in the project’s World Work directory.
Member Work, Project Work, and World Work directories are not backed up. Project members are responsible for backing up these files, either to Project Archive areas (HPSS) or to an off-site location.
Project Archive Directories¶
Projects are also allocated project-specific archival space on the High Performance Storage System (HPSS). The default quota is shown on the table above. If a higher quota is needed, contact the User Assistance Center.
The Project Archive space on HPSS is intended for storage of data not immediately needed in either Project Home (NFS) areas nor Project Work (Alpine) areas, and to serve as a location to store backup copies of project-related files.
Project Archive Path¶
The project archive directories are located at
pjt000 is your Project ID).