Page tree
Skip to end of metadata
Go to start of metadata

      Each user account is given a home space for personal file storage. By default, It is set only accessible to the user and located at /mnt/home/$USER for every node, where $USER is the environment variable of the user's login name. It is often referred as “home directory” since this is the beginning directory after login of any HPCC node.

      Every home space starts with a 50 GB limit for file storage space and can not contain more than 1 million files. You can request and increase this up to 1TB by completing Quota Increase Request form. Storage space greater than 1TB is available for an annual fee paid through a MSU financial account. Users can find the fee and submit their request by Large Quota Increase Request form. To find out the quota and used space of your home directory, please check Space quota section. If you would like to store more than 1 million files in your home space, please refer to the following section Limit on number of files.

      All home directories are stored in the IBM General Parallel File System (GPFS) mounted on /mnt/ufs18. It is automatically backed-up except files saved in nodr space. To restore any file from backup, please submit a ticket and let us know the paths to the files or the directory with the time frame you would like them restored. To learn more information, please check the wiki page of ufs18 file system.

      For the system security and user data privacy, we recommend that users do NOT open HOME directory access permission to others. When you report an issue about files saved in HOME, please attach them to your message for reference. ICER staffs do not access any files or directories in your home directory.

Incorrect Used Space

Currently our home file system check quota function will sometimes cause a user's directory over the quota due to incorrect calculation of used space .  If you see this please open a ticket and we will work with you to temporarily increase your quota.  We continue to work with our vendor to correct this issue.

Space quota

      The only way to get quota information of home space is to run the command quota :

$ quota
Home Directory:             Space        Space        Space        Space        Files        Files        Files        Files
                            Quota        Used         Available    % Used       Quota        Used         Available    % Used
/mnt/home/UserName          1024G        1213G        -198G        118%         1048576      540389       508187       52%

Research Groups:            Space        Space        Space        Space        Files        Files        Files        Files
                            Quota        Used         Available    % Used       Quota        Used         Available    % Used
ResearchSpace1              2048G        1778G        270G         87%          2097152      432525       1664627      21%
ResearchSpace2              1024G        126G         898G         12%          1048576      11755        1036821      1%

Temporary Filesystems:

/mnt/scratch (/mnt/gs18)    Space Quota  Space Used   Space Free   Space % Used Filess Quota Files Used   Files Free   Files % Used
                            51200G         64G          51136G          0%          1048576      5724         1042852      1%

/mnt/ffs17                  Used          Quota
                            35.66GiB      100.00GiB

/mnt/ls15 (legacy scratch)  Inodes Used   Quota         Free
                            125721        1000000       874279

where all file spaces accessible to the user are listed, including home, research, scratch and ffs17. In each space, the information of quota, usage and availablility on space size and number of files can be found. If "Free" or "Available" is a negative value (such as "Space Available" in home directory of the above example), the usage is over the quota. Please remove, transfer or compress some files so the used space or the file count can be lower than the "Quota" value. Since GPFS uses a different compression algorithm, you may notice higher space size after files are copied to the ufs18 file system.

Actual disk usage (du) different from quota results

      The new file system has a smallest file block size of 64k. This means that files between 2K and 64K will occupy 64K of space. This causes space usage inflated greatly for users with large amounts of small files. One suggested solution would be to compress many small files into one large file (larger than 64K). in this way, the number of files can also be reduced. If you still have any difficulty, a temporarily larger quota can be requested if your quota is at 1T with many small files.

Limit on number of files

      Besides the quota on the size of space, users are also limited to 1 million files in their home or research directory. We need to set this limit because, with a great number of files, the file system will spend too much time on back-up to be able to function normally. If possible, users can compress many files into one to reduce the file number.

      If users do not wish to have the limit, they can request to have part of their home (or research space) moved under nodr space (/mnt/ufs18/nodr/ or /mnt/ufs18/nodr/research/) where there is no limit on the file count yet no back-up on the files either. Users will be responsible for their own back-up in the nodr space.

      By default, one half of the space quota is assigned to the requested nodr space. The quota of the original space under /mnt/home/ (or /mnt/research/) is then donwsized to its half so the total space quota remains the same. A different size of nodr space can also be assigned according to user's request. Once this space is created, the path and the quota information can be found by the quota command mentioned above.


  • No labels