Skip to main content

Research Project Space (RPS)

Description​

Research Project Space (RPS) volumes provide working space for sharing data and code amongst project or lab members. RPS directories are built on the same parallel file system (GPFS) like HPC Scratch. They are mounted on the cluster Compute Nodes, and thus they can be accessed by running jobs. RPS directories are backed up and there is no old file purging policy. These features of RPS simplify the management of data in the HPC environment as users of the HPC Cluster can store their data and code on RPS directories and they do not need to move data between the HPC Scratch and the HPC Archive file systems.

note
  • Due to limitations of the underlying parallel file system, the total number of RPS volumes that can be created is limited.
  • There is an annual cost associated with RPS.
  • The disk space and inode usage in RPS directories do not count towards quota limits in other HPC file systems (Home, Scratch, and Archive).

Calculating RPS Costs​

The PI should estimate the cost of the RPS volume by taking into account storage size and number of inodes (files). The cost is calculated annually. Costs are divided into the total space, in terabytes, and the number of inodes, in blocks of 200,000.

  • 1 TB of Storage Cost: $100
  • 200,000 inodes Cost: $100

An initial RPS volume request must include both storage space and inodes. Modifications of existing RPS volumes can include just Storage or just inode adjustments.

An initial request includes 1TB and 200,000 inodes for an annual cost of $200.

Example RPS Requests​

Requests can include more storage or files, as needed, such as 1TB and 400,000 inodes or 2TB and 200,000 inodes. Both of the previous examples would cost $300, since they are requesting an incremental increase of storage or inodes, respectively.

This would be the breakdown of the examples listed above:

  • 1 TB ($100) + 400,000 inodes ($200) = $300
  • 2 TB ($200) + 200,000 inodes ($100) = $300

Submitting an RPS volume Request or Modification​

Step 1: Decide the size (in TB) and number of inodes (files) that is needed for one year​

The minimum size of an RPS request (to create a new RPS volume or extend an existing one) is 1TB of space.

If this is a new/first request, you must purchase both storage and inodes. A typical request includes 200,000 inodes per TB of storage.

tip

Before submitting an RPS request (request for a new RPS volume or extending the size of an existing volume) PIs should estimate the growth of their data (in terms of storage space and number of files) during the entire year, rather than submitting a request based on their data storage needs at the time of the request.

Step 2: Determine the cost of the request​

Determine the total annual cost of the request and the contact info of the School/Department/Center finance person.

Step 3: Verify that the project PI has a valid HPC user account​

The PI administers the top level RTS directory and grants access to other users. Thus the PI must have a valid HPC user account at the time of request. Please note that the HPC user account of NYU faculty never expires and thus does not need to be renewed every year. If the PI does not have an HPC account, please request one here.

Step 4: The PI submits the request to the HPC team via email​

warning

Only PIs can submit RPS requests.

Submit your RPS request by contacting the HPC team via email (hpc@nyu.edu).
Please include in the request:

  • size in TB
  • number of inodes
  • contact information of the School/Department/Center finance person

The HPC team will review the request and will contact the PI with any questions. If the request is approved, the HPC team will create (or adjust) the RPS volume with the PI's HPC user account as the owner of the RPS directory. An invoice will be generated by the IT finance team.

Current HPC RPS Stakeholders​

HPC RPS Stakeholders

FAQs​

Data Retention and Backups​

  • How long can I keep the lab data in RPS?
    • For as long as the lab pays for the RPS resources. Even if the current HPC cluster retires, the RPS volumes will be transferred to the next cluster
  • How can I find out how much of the storage and inodes have I used in my lab RPS volume
  • What kind of backups are provided?
    • Backups are done once a day (daily incremental). Backups are kept for 30 days. This means that if something was deleted more than 30 days ago, it won't be in the back ups and thus it won't be recoverable.
  • Where are backups stored?
    • RPS backups are stored on public cloud (AWS S3 Storage buckets).

Billing and Payments​

  • What happens if I do not pay my bill?
    • If the invoice is not paid for more than 60 days, the lab RPS directory will be 'tar'-ed and copied to an archival area. If 60 more days pass and the invoice is still not paid the tar files will be deleted.
  • Can I pay for RPS using a credit card?
    • Unfortunately we're unable to process credit card payments
  • Can I pay for multiple years instead of paying every year?
    • Yes, we can arrange for multiyear agreement