Research space is created upon request from a MSU principal investigator (PI) for his or her research group. The initial quota is 50GB and 1,000,000 on number of files. PI can increase to 1TB on space size for free. Additional space above 1TB may be purchased by completing Large Quota Increase Requet form, where you can find the annual fee. This space is associated with your group name and located at /mnt/research/[group name] by default. It is accessible to all users who have been added to the group and convenient for sharing files in the research directory. Please set your umask and the file permissions appropriately so the group members can access files and directories in the space. See the section Instructions for using research space below for more details.
Same as home directories, all research directories are also stored in the IBM GPFS under /mnt/ufs18. Files are backed up automatically (except saved in
nodr space). To access any file backup, please submit a ticket and let us know the paths to the files or the directory with the time frame you would like them restored.
To learn about the space quota and usage in your research directory, please check Space quota section in Home Space page. If you would like to have more than 1 million files in your research space, please refer to Limit on number of files section in Home Space page. Please also read the following sections for how to use your research space.
Currently our home file system check quota function might cause a users directory to have an incorrect quota. If you see this please open a ticket and we will work with you to temporarily increase your quota. We continue to work with our vendor to correct this issue.
Instructions for using research space
In order for group users able to access the files and directories in their research space, it is important to read and follow the instructions below.
All folders in research space need to have the group ownership same as the group name and the set-group-ID (setgid) bit. By default, your research directory (e.g. /mnt/research/<GroupName>) is set with the right group ownership and set-group-ID (setgid) bit:
The letter "
s" in the permissions
drwxrws---of the directory is the setgid which makes new files and sub-directories created inside to inherit its group ownership rather than the primary group of the user. If the settings of a sub-directory are not right, there could be a "Disk quota exceeded" problem when create, copy or transfer files under it. (See next section: "Quota setting on research space" for more explanations.) Users can run the following two commands to ensure all files and directories in your research space have the right settings:
If you get permission issues to work on some files or directories, their owners also need to run the commands above. PI of the research space can also submit a ticket to request the above corrections.
- Do not preserve group ownership when copying files to your research space. Ideally, with the setting of set-group-ID bit, all files and directories in the research space should have the right group ownership. However, if you copy files from other places and preserve ownership (such as using "
cp -p" or "
mv" command), the group ownership might not be kept the same as the research space.
rsynccommand with the option
--chmod=Dg+sto transfer files from a local computer to your research space. For example, use the command:
to transfer a directory
TestDirin your local computer to
/mnt/research/<GroupName>/in HPCC. This will automatically make all copied directories with set-group-ID bit.
0007or any lower value:
In this way, your created files or directories are readable, writable and executable to all group users. Since you might need to do this every time in a new session, it is a good idea to put the command in the file
~/.bashrcso it is automatically executed at the beginning of every session. Alternatively, you can run the following powertools command once to add the line
umask 0002in the
Quota setting on research space
The quota setting on an HPCC research space is based on the group ownership of the files. Any files with the group ownership the same as the research space are followed by the
quota command. However, any files (larger than 8 MB) with a group ownership different from the research space are not allowed to exist.
For this reason, although the results of
quota command indicate that you have not exceeded your quota, users may still get an error message such as “failed to ... ... Disk quota exceeded” while creating, copying or writing a file to their research space. To resolve this "Disk quota exceeded" problem, users may do the following:
Make sure the directory to which files are being copied has the same group ownership as the research space and has the set-group-ID bit.
For example, if you get the error message when trying to transfer files from your local computer to a directory
Drctryin your research space
/mnt/research/Group. Use the
ls -ldcommand to check for differences in the group ownership and access permissions of the directory
Drctryand the research space
In the example above, the group ownership of the directory
Drctry is Prmry. This isdifferent from group ownership,
Group,of the research space
/mnt/research/Group. Additionally, the permissions,
rwxrwx---of the directory
Drctrydoes not have the set-group-ID bit
rwxrws---as the research space. To change these two attributes, the owner of the directory,
UserName, can run the following commands:
Once the settings are corrected, the file transfer can proceed to the directory:
(Further instruction about file permission can be reviewed from the wiki page File Permissions on HPCC.)
If the file already exists, its group ownership needs to be changed to the group of the research space.
For example, you try running a command to copy, transfer or write a file
footo a directory
Drctryof your research space. However, a file with the name
fooalready exists in the directory
Drctry. In order for this command to work,
fooin the directory
Drctrymust have the same group ownership as the research space. Otherwise, the owner of the file can use the
chgrpcommand mentioned above to correct the group ownership. Alternatively, you can first rename or remove
fooin the directory
If the file is going to be created, the user's primary group may need to be set to the group of the research space.
Users can use
newgrpcommand to reset their primary group temporarily. For more information, please refer to Change Primary Group page.
ACL for GPFS
If you are using access control list (ACL), you will need to update them to NFSv4 ACLs. You will need to use the mmgetfacl, mmputacl, and mmeditacl commands. Please refer to the GPFS Commands page for more details.