Page tree
Skip to end of metadata
Go to start of metadata

      Research space is created upon request from a MSU principal investigator (PI) for his or her research group. The initial quota is 50GB and 1,000,000 on number of files. PI can increase to 1TB on space size for free. Additional space above 1TB may be purchased by completing Large Quota Increase Requet form, where you can find the annual fee. This space is associated with your group name and located at /mnt/research/[group name] by default. It is accessible to all users who have been added to the group and convenient for sharing files in the research directory. Please set your umask and the file permissions appropriately so the group members can access files and directories in the space. See the section Instructions for using research space below for more details.

      Same as home directories, all research directories are also stored in the IBM GPFS under /mnt/ufs18. Files are backed up automatically (except saved in nodr space). To access any file backup, please submit a ticket and let us know the paths to the files or the directory with the time frame you would like them restored.

      To learn about the space quota and usage in your research directory, please check Space quota section in Home Space page. If you would like to have more than 1 million files in your research space, please refer to Limit on number of files section in Home Space page. Please also read the following sections for how to use your research space.

Currently our home file system check quota function might cause a users directory to have an incorrect quota.  If you see this please open a ticket and we will work with you to temporarily increase your quota.  We continue to work with our vendor to correct this issue.

Instructions for using research space


      In order for group users able to access the files and directories in their research space, it is important to read and follow the instructions below.  

  1. All folders in research space need to have the group ownership same as the group name and the set-group-ID (setgid) bit. By default, your research directory (e.g. /mnt/research/<GroupName>) is set with the right group ownership and set-group-ID (setgid) bit:

    $ ls -ld /mnt/research/<GroupName>/
    drwxrws--- 9 ProjInvst <GroupName> 8192 Jul 22 08:38 /mnt/research/<GroupName>/

    The letter "s" in the permissions drwxrws--- of the directory is the setgid which makes new files and sub-directories created inside to inherit its group ownership rather than the primary group of the user. If the settings of a sub-directory are not right, there could be a "Disk quota exceeded" problem when create, copy or transfer files under it. (See next section: "Quota setting on research space" for more explanations.) Users can run the following two commands to ensure all files and directories in your research space have the right settings:

    find /mnt/research/<GroupName>/ -not -group <GroupName> -print0 |xargs -0 chgrp <GroupName>
    find /mnt/research/<GroupName>/ -type d -print0 |xargs -0 chmod g+s

    If you get permission issues to work on some files or directories, their owners also need to run the commands above. PI of the research space can also submit a ticket to request the above corrections.

  2. Do not preserve group ownership when copying files to your research space. Ideally, with the setting of set-group-ID bit, all files and directories in the research space should have the right group ownership. However, if you copy files from other places and preserve ownership (such as using "cp -p" or "mv" command), the group ownership might not be kept the same as the research space.
  3. Please use rsync command with the option --chmod=Dg+s to transfer files from a local computer to your research space. For example, use the command:

    rsync -avz TestDir --chmod=Dg+s <username>@rsync.hpcc.msu.edu:/mnt/research/<GroupName>/

    to transfer a directory TestDir in your local computer to /mnt/research/<GroupName>/ in HPCC. This will automatically make all copied directories with set-group-ID bit.

  4. Please set umask 0007 or any lower value:

    $ umask 0007

    In this way, your created files or directories are readable, writable and executable to all group users. Since you might need to do this every time in a new session, it is a good idea to put the command in the file ~/.bashrc so it is automatically executed at the beginning of every session. Alternatively, you can run the following powertools command once to add the line  umask 0002  in the  ~/.bashrc file:

    $ umask_in_bash                 # powertools command

Quota setting on research space


      The quota setting on an HPCC research space is based on the group ownership of the files. Any files with the group ownership the same as the research space are followed by the quota command. However, any files (larger than 8 MB) with a group ownership different from the research space are not allowed to exist.

     For this reason, although the results of quota command indicate that you have not exceeded your quota, users may still get an error message such as “failed to ... ... Disk quota exceeded” while creating, copying or writing a file to their research space. To resolve this "Disk quota exceeded" problem, users may do the following:

  • Make sure the directory to which files are being copied has the same group ownership as the research space and has the set-group-ID bit.

    For example, if you get the error message when trying to transfer files from your local computer to a directory Drctry in your research space /mnt/research/Group. Use the ls -ld command to check for differences in the group ownership and access permissions of the directory Drctry and the research space Group :

    $ ls -ld /mnt/research/Group/Drctry /mnt/research/Group
    drwxrwx--- 2 UserName Prmry 5464 Feb 27 11:34 /mnt/research/Group/Drctry
    drwxrws--- 9 UserName Group 8192 Jul 10 15:34 /mnt/research/Group
    

    In the example above, the group ownership of the directory Drctry is Prmry. This is different from group ownership, Group, of the research space /mnt/research/Group. Additionally, the permissions, rwxrwx--- of the directory Drctry does not have the set-group-ID bit rwxrws--- as the research space. To change these two attributes, the owner of the directory, UserName, can run the following commands:

    $ chgrp -R Group /mnt/research/Group/Drctry     # Change the group ownership to Group
    $ chmod g+s /mnt/research/Group/Drctry          # Set up set-group-ID bit

    Once the settings are corrected, the file transfer can proceed to the directory:

    $ ls -ld /mnt/research/Group/Drctry
    drwxrws--- 2 UserName Group 5464 Feb 27 11:34 /mnt/research/Group/Drctry
    

    (Further instruction about file permission can be reviewed from the wiki page File Permissions on HPCC.)
       

  • If the file already exists, its group ownership needs to be changed to the group of the research space.

    For example, you try running a command to copy, transfer or write a file foo to a directory Drctry of your research space. However, a file with the name foo already exists in the directory Drctry. In order for this command to work, foo in the directory Drctry must have the same group ownership as the research space. Otherwise, the owner of the file can use the chgrp command mentioned above to correct the group ownership. Alternatively, you can first rename or remove foo in the directory Drctry.
          

  • If the file is going to be created, the user's primary group may need to be set to the group of the research space.

    Users can use newgrp command to reset their primary group temporarily. For more information, please refer to Change Primary Group page.

ACL for GPFS


      If you are using access control list (ACL), you will need to update them to NFSv4 ACLs. You will need to use the mmgetfacl, mmputacl, and mmeditacl commands. Please refer to the GPFS Commands page for more details.



  • No labels