Files and folders
Linux has a single directory 'tree', separated by slash, the top is the 'root'. All additional disks are connected on /mnt ('mounted'). In Linux, there is no concept of driver letters.
tree is a recursive directory listing program which prints a depth indented listing of files. With no arguments, tree lists the files in the current directory. You can change the depth with -L argument.
The example shows the structure of the 'linux_tutorial' directory which we used for an exercise.
tree command does not show hidden files/directory by default. To see hidden files/directories, use -a argument.
Linux has a method to keep files private and safe. When you type ls -a you will get a lot of information of your directory including files such as
The first letter shows a type of a file.
-: normal file
l: symbolic link
Next nine characters shows permission; first three is for you, next three is for your group members, and last three is for a whole world. Let's see the first file in the picture which shows 'd rwxr-xr-x'.
The first character is 'd', and therefore, even the name is 'hello.txt', it is not a file, but a directory. Next nine charcters represent the settings for the three sets of permissions.
- The first three characters show the permissions for the user who owns the file (user permissions).
- The middle three characters show the permissions for members of the file’s group (group permissions).
- The last three characters show the permissions for anyone not in the first two categories (other permissions).
The letters represent:
- r: Read permissions. The file can be opened, and its content viewed.
- w: Write permissions. The file can be edited, modified, and deleted.
- x: Execute permissions. If the file is a script or a program, it can be run (executed).
In our screenshot, the first three characters are 'rwx' which means you(owner) can read, write/modify or execute (it means you can go inside the directory here). Next three characters are 'r_x' which means your group members can read, or execute, but can not modify the contents. Last three characters are 'r_x' which means everyone else can read, and execute.
- ---------: it means no permissions have been granted at all.
- rwxrwxrwx: it means full permissions have been granted to everyone.
Permissions can be set with chmod command, and ownership set with chown. You can only change these for files you are the owner of. To use chmod, we need to tell it 'who' we are setting permissions for, 'what' change we are making, 'which' permissions we are setting.
The “who” values can be:
- u: User - the owner of the file.
- g: Group - members of the group the file belongs to.
- o: Others - people not governed by the u and g permissions.
- a: All - all of the above.
If none of these are used, chmod behaves as if “a” had been used.
The “what” values can be:
- –: Minus sign. Removes the permission.
- +: Plus sign. Grants the permission. The permission is added to the existing permissions. If you want to have this permission and only this permission set, use the = option, described below.
- =: Equals sign. Set a permission and remove others.
The “which ” values can be:
- r: The read permission.
- w: The write permission.
- x: The execute permission.
You can also use a three three-digit numbers (total nine) to provide the permission with chmod. The leftmost digit represents the permissions for the owner. The middle digit represents the permissions for the group members. The rightmost digit represents the permissions for the others. x is 1, w is 2, and r is 4. and the some of these three numbers are the permission.
The digits you can use and what they represent are listed here:
- 0: (000) No permission.
- 1: (001) Execute permission.
- 2: (010) Write permission.
- 3: (011) Write and execute permissions.
- 4: (100) Read permission.
- 5: (101) Read and execute permissions.
- 6: (110) Read and write permissions.
- 7: (111) Read, write, and execute permissions.
The tables shows a summary of the chmod command.
|u - user||+ add||r - read (4)|
|g - group|
|w - write (2)|
|o - others||= set exactly||x - execute (1)|
|a - all|
chmod u=rx, og=r my_file1.txt : set the user has read, and executable permissions; group/other have read permission only for my_file1.txt
chmod 544 my_file1.txt: same as chmod u=rx, og=r my_file1.txt.
chmod 750 my_file1.txt: set the user has read, write, and executable permissions; group has read/executable permission; others no permission.
The cat (short for “concatenate“) command is one of the most frequently used command in Linux. cat command allows us to create single or multiple files, view contain of file, concatenate files and redirect output in terminal or files.
- To display contents of a file: cat filename
- To view contents of multiple files: cat file1 file2
- To create a file: cat >file2.txt
If file content is large, and won't fit in a terminal screen, you can use more and less with cat command such as (we are using pipes |, which will be covered later)
- cat file2.txt |more
- cat file2.txt |less
more, less, and most (most has more features than more and less commands) are commands to open a given file for interactive reading.
Redirection is a feature such that when you execute a command with it, you can change the standard input/output devices. The standard input (stdin) device is the keyboard, and the standard output (stdout) device is the screen. With redirection, stdin/stdout can be changed.
>: Output redirection (overwriting)
eg. ls -la > list.txt (ls -la results save as list.txt instead of standard output (screen))
>>: Output redirection (appending)
eg. ls -l >> list.txt (ls -l result will be appended at the end of the list.txt. If there is no list.txt, it works as ls -la > list.txt)
<: Input redirection
>&: writing the output from one file to the input of another file.
eg. matlab > outfile 2>&1 : send stdout and stderr to 'outfile'. Here 1 and 2 are file descriptors. File descriptor 1 is the standard output (stdout), and 2 is the standard error (stderr). > is redirection, and & indicates that what follows and precedes is a file descriptor and not a filename.
Wildcards are symbols or special characters that represent other characters. You can use them with any command such as ls/rm/cp etc.
*: anything or nothing
?: single character
[ ]: any character or range of characters
[! ]: inverse the match of [ ]
ls *.txt # list all txt files
ls *-?.txt # list all files with ‘-‘ and with one cha in front of ‘.txt’
ls [0-9]*.txt # list all files start with a number.
ls [A-Z]*.txt # list all files start with a capital letter? [A-Z] can be different based on LC_COLLATE value. For further discussion, check here. In HPC at MSU, the default of [A-Z] is a, A, b, B, c, C, ....y, Y, z, Z, which is standard collations (en_US).
Try ls [[:lower:]].txt; ls [[:upper:]].txt; ls [[:lower:][:upper:]].txt
ls [!a-Z]*.txt # list txt files that don’t begin with any letter.
For more informaiton, refer to https://wiki.hpcc.msu.edu/display/ITH/Regular+Expressions
Linux includes a built in manual for nearly all commands. Type ‘man’ followed by the commands.
e.g. man man
To navigate the man pages use the arrow keys to scroll up and down or use the enter key to advance al ine, space bar to advance a page, letter u to go back a page. Use the q key to quit out of the display.
The manual pages often include these sections:
- Name: a one line desctiption of what it does
- Synopsis: basic syntax for the command line.
- Description: describes the program’s functionalities.
- Options: lists commnand line options available for this program.
- Example: examples of some of the options available.
- See Also: list of related commands.
Note that options can be with single dash ‘-‘ or double dash ‘—’
wc, grep, sort, head, tail
wc: count words, lines or characters
eg. who | wc -l # number of users logged in
grep: find patterns in files or data, returns just matching lines
eg. who |grep $USER
sort: given a list of items, sort in various ways
eg. who | sort
head: list only top n lines of file
who > who.txt; head who.txt
tail: list only last n lines of file
zip, unzip, tar
tar: create (tar -c) or extract (tar -x) ‘tape archive’ file.
Using the man pages, find out what the default number of lines that head and tail will display, and how to limit those to just one line
Can you tar all files and folders in workshop folder? Question? Use man.
Shell maintains and you can set ‘variables’ that the shell uses for configuration and in your script. Variables start with $, and can be seen with echo $VARNAME
Explore common variables with the echo command and list what they are $HOME, $USER, $SHELL, $PATH.
For more information, please refer to