Linux / UNIX Command Fundamentals and the File System

Linux / UNIX Commands

So you want to start using the powerful Linux / UNIX command-line interface, but you don't know where to start. Here's a suggested set of things to know how to do. But first, just a quick note on nomenclature to avoid unneeded confusion — UNIX is a general family of operating systems of which Linux is just one example. Learn the commands once, and you know them everywhere. Everything on this page should be true for any vaguely UNIX-like operating system. This includes:

Commercial examples like Solaris, HP-UX, IRIX, AIX, Tru64, and so on
Any distribution of Linux
All other free UNIX-family operating systems like FreeBSD, OpenBSD, and NetBSD.
Apple OS X

Your Goal

Here is a quick inventory of what you need to learn in order to be a reasonably capable Linux command-line user. For the most part, these are lists of commands. You need to understand how to run them with appropriate parameters and options, and how to interpret their output. All these and more are explained further down this page.

Navigate the file system: cd, pwd

Understand fundamental directories: ., .., ~, ~joe

Manipulate directories: mkdir, rmdir

Understand information about files and directories: ls, ls -a, ls -l
In the ls -l output, be able to interpret the permission mask. Also notice the owner, group, size in bytes, and modification timestamp, although those are very obvious.

Use file name wildcards: *, ?, [abc], [a-f]

Copy, move (or rename), and remove files: cp, mv, rm

Read file contents: cat, more, less

Compare file contents: diff, cmp

Redirect input and output streams: >, <, >>, |

Extract lines containing specified strings from files or output streams: grep

Change file permissions: chmod

Find files based on name or other attributes: find

Edit text files: You can start with the very simple nano or pico, but eventually you need to learn vi, or really its replacement, the much improved vim.

Read the manual pages: man

There's much more to learn, but the above list is a good start. Someone can't honestly claim to be a Linux/Unix command-prompt user without knowing all of the above,

If I Had To Interview People

Let's say there's a job posting for a position where you will be writing software in a Linux environment. You will be collaborating with a group of people on the project. Here are some things I would expect a new hire to be able to do or answer with no problem:

1: What is the list of all directories on the system with "log" anywhere in their name? Since you don't know the root password, and the needed search will create a large number of unavoidable error messages, throw away the error output.

2: Same thing, except either "log" or "Log".

3: In the directory /var/log, what are the last 20 lines in the most recently modified file? Half credit if this takes two commands and human intervention.

4: In the file /var/log/messages, how many times does the string apache appear?

5: In the file /var/log/messages, display the last line containing the string apache.

6: Your file ~/myfile is owned by you and group coders. Leaving all other existing permissions alone, change permissions so that others (not you, not group coders) have none of read, write, or execute permission.

7: Your file ~/otherfile is owned by you and group coders. Set its permissions so that:

You can read, write, and execute it.
Group coders members can read and execute it, but they cannot write it.
Everyone else can read it, but not write or execute it.

8: You are suddenly worried about insider threats. Change permissions to remove all access for all users other than you for your home directory and all of its contents. You have thousands of files.

9: Without modifying its content, change the modification timestamp of the file current-file in your home directory to be right now.

10: The file /tmp/list contains lines that look like this:
.M......C    /boot
SM5....T.    /boot/grub.conf
.M.....T.    /etc/passwd
S.5....T.    /var/lib/unbound/root.key
.....UG..    /var/run/avahi-daemon
.......T.    /boot/kernel-5.4.42-v7.1.el7.img
You want to extract just those 9-character patterns at the beginning of each line. That is, the output should look like this, with just the first 9 characters from each line:
.M......C
SM5....T.
.M.....T.
S.5....T.
.....UG..
.......T.

11: Run commandA and save its output in /tmp/output, and then run commandB and append its output to that same file. Don't destroy the output of commandA!

12: The file /etc/passwd contains one line per user, with fields specifying various attributes. One field is their preferred command shell. Extract the list of users who use the /bin/bash shell, sorted into alphabetical order.

13: What is the list of all files owned by the user julie named *.pptx?

14: Run the command complex and save all of its output, both standard output and standard error, in the file /tmp/all-output.

15: How do you save your changes and exit your vi (or, better yet, vim) session?

16: How do you abort your vi (or vim) session, exiting without saving changes?

Answers

1:
$ find / -name '*log*' 2> /dev/null

2:
$ find / -name '*[Ll]og*' 2> /dev/null

3:
$ tail -20 $(ls -t /var/log/* | head -1)
or
$ tail -20 $(ls -tr /var/log/* | tail -1)

Or run:
$ ls -t /var/log | head
or
$ ls -tr /var/log | tail
and examine the output to find the newest file. Then use tail on that file.

4:
$ grep -c /var/log/messages
or, less elegantly,
$ grep /var/log/messages | wc -l

5:
$ grep apache /var/log/messages | tail -1

6:
$ chmod o-rwx ~/myfile

7:
$ chmod 754 ~/otherfile

8:
$ chmod -R go-rwx ~

9:
$ touch ~/current-file

10:
$ cut -c1-9 /tmp/list
That does what the question asks, the first nine characters from each line. If the entire file looks like the example, you could get the same output from either of the following. Personally, casual use of awk or regular expressions would impress me more than fussing about the distinction between "first 9 characters" and "first white-space-delimited token".
$ awk '{print $1}' /tmp/list
or
$ sed 's/ .*//' /tmp/list

11:
$ commandA > /tmp/output
$ commandB >> /tmp/output

12:
$ grep /bin/bash /etc/passwd | sort
Extra credit for just the user names themselves:
$ grep /bin/bash /etc/passwd | sed 's/:.*//' | sort
or
$ grep /bin/bash /etc/passwd | awk -F: '{print $1}' | sort
Hire them immediately if they suggest:
$ awk -F: '/\/bin\/bash/ {print $1}' /etc/passwd | sort

13:
$ find / -user julie -name '*.pptx' 2> /dev/null

14:
$ complex > /tmp/all-output 2>&1

15:
Press <Escape> and then
ZZ
or
:wq
or
:w
:q

16:
Press <Escape> and then
:q!

Discussing how to exit vi. I would assume that the aliens use Emacs.

Getting Started and Learning More

This page is just a starting point, it mentions some of the most fundamental commands and provides examples of some simple and commonly needed tasks. The examples are just a glimpse of what they can accomplish, some of the commands have so many options that the overwhelming variety of possible command-line parameters could be considered a bug, as the command may be too complex to fully memorize.

This isn't really a problem! UNIX is designed around tools which each do one very specific thing well. Learn the main purposes of some core commands, and remember that you can always refresh your memory or learn more about options with the on-line manual pages.

The O'Reilly book Unix in a Nutshell is a good introductory text. Yes, there is also a Linux in a Nutshell title, but it uses a lot of its content showing you how to use fairly intuitive graphical interfaces. Unix in a Nutshell is much better for learning the command-line environment and powerful tools.

Amazon
ASIN: 0596100299

Amazon
ASIN: 1593273894

Another good introduction is The Linux Command Line: A Complete Introduction, which is available as a free download here.

Let's say that you want to remove a file but you you do not remember the precise command. You can ask for a list of commands related to the topic. Ask for those one-line summaries of commands containing the string remove, where -k specifies by keyword:

$ man -k remove

You will see a lot of output, including commands to remove directories, to remove users, and arcane things for system administrators such as removing logical volume groups and named semaphores. But as you look down the alphabetical list of output you will see the line I have highlighted here:

[...  lots of output  ...]
removef              (1)  - remove a file from software database
remque [insque]      (3)  - insert/remove an item from a queue
rm                   (1)  - remove files or directories
rmdir                (1)  - remove empty directories
rmvq                 (8)  - remove a message from a queue
[... lots more output ...]

So you see that you probably need the command rm but you want to read a little about it. You need to verify that it's really the correct command and to see how to use it and possibly how to specify some optional behavior. Ask to see its detailed manual page:

$ man rm

Let's get started! Skilled Windows users will see that many features of the Windows command line interface were modeled directly on the UNIX command line interface — some things work exactly the same, and others are analogies.

File Name Wildcards

This is probably easiest to understand as a list of examples. You would use these literally on the command line, typing something like:
$ ls blah.*
or:
$ cp blah.[gjp]* backup-directory
You will see examples of these types of uses in later sections of this page.

* matches any string, including nothing at all.

foo* matches foo, foot, and football.

UNIX is stricter than Windows about matching. A Windows user would expect this command to remove all the files:
$ rm *.*
However, it does what you ask. It only removes files with a "." in their name.
$ ls Index.html image.jpg logo.gif notes $ rm *.* $ ls notes
The file notes has no "." in its name, so it was not removed.

? matches any single character.

foo? matches food and foot, but not football.

[abc...] matches one of any character in the list.

foo[ldt] matches fool, food, and foot.

You can specify ranges. foo[a-c] matches fooa, foob, and fooc.

foo[a-cX-Z] matches fooa, foob, fooc, fooX, fooY, and fooZ.

foo[a-cmx-z] matches fooa, foob, fooc, foom, foox, fooy, and fooz.

Be careful! foo[d,t]ball matches foodball and football, but it also matches foo,ball because that comma is treated as just another character.

[^...] matches any single character not in the list.

foo[^ldt] matches everything foo? would match except for fool, food, and foot.

Brace expansion is a powerful shortcut that can compress a series of similar commands into just one. Instead of typing this:
$ mkdir /home/cromwell/project/subdir1
$ mkdir /home/cromwell/project/subdir2
$ mkdir /home/cromwell/project/subdir3
you could simply type this:
$ mkdir /home/cromwell/project/{subdir1,subdir2,subdir3}
or, even shorter:
$ mkdir /home/cromwell/project/subdir{1,2,3}

Very Fundamental Directories: `.` and `..` and `~`

The directory named ".", that is, just one dot, is your current working directory. What is it really? It's wherever you are at the moment. As Buckaroo Banzai said, "No matter where you go, there you are."

Related to that, the directory named ".." is the parent of your current working directory. If your current working directory is /home/cromwell then:
. is /home/cromwell and
.. is /home.

There is just one exception — when you are at the root of the file system there is no parent. That is, when your current working directory is / then you can't go up to its parent.

Actually, there is a directory named ".." in the root of the file system, but it points to the same location as it has the same i-node. If you don't believe me, verify this for yourself:
$ ls -ldi /.*

The special directory name ~ refers to your home directory, while ~julie would be the home directory of the user julie. Home directories are specified in the file /etc/passwd, but you can simply use ~ and the shell automatically looks it up.

Running Commands and Your PATH

You do not need to be in any particular directory to run a program, so long as either of the following is true.

1: The program is in a directory listed in your PATH environment variable, or

2: You specify the full path to the program. This could be an absolute path:
$ /usr/bin/vim
or a relative path. In this example, mydir is a subdirectory of your current working directory and it contains a program named myprogram:
$ mydir/myprogram
That relative path could start with ".", to mean "Run the program that's right here, not a program by that name installed in a standard system binary area":
$ ./myprogram

You can see your current PATH environment variable setting with the following command. You will see that the components are separated by colons:

$ echo $PATH

When you just type a command, the shell scans through the elements of your PATH looking for the first match. The first match wins — when the shell finds an executable program by that name in a PATH component, it runs that program. If no match is found, an error is reported. The precise error depends on your command shell, but it will be something like:
foo: Command not found

A reasonable PATH value for users includes the following directories. Note that X might be installed under X11R6 as shown here, or just X11, or even just X. You will have to investigate and think just a little:
/bin
/usr/bin
/usr/X11R6/bin
/usr/local/bin

A reasonable PATH value for root includes the above directories and adds the following:
/sbin
/usr/sbin
/usr/local/sbin

Notice that "." should never be in the path for root as that would allow a malicious user to lay a trap. That means that root needs to specify the path to a command in the local directory, for example, ./programname.

You can run anything from anywhere if you provide the details. For example, let's say that your Apache web server saves its logs in a bunch of files in the directory /var/www/logs and you want to process them with some program named /var/www/tools/log-analyzer, a program that reads the files listed as its command-line parameters. You want to run that analysis and save the output in a file named result in your home directory. All you have to do is this:

$ /var/www/tools/log-analyzer /var/www/logs/* > ~/result

Redirecting I/O — Writing To and Reading From Arbitrary Files

Again, this is probably easiest to understand by example, once you realize that any process has at least three standard I/O streams: input, output, and error. They are sometimes called stdin, stdout, and stderr, especially by C/C++ programmers.

The standard output stream is general output created by the process — its contents may be the main intended result of the program. The standard error stream is available for a second parallel stream of output intended for use in reporting errors. As you will see below, they can be sent to the same place, which could be the terminal screen where you started the program, or to a file, or into another program as its standard input. Or, the two output streams could be sent to different places. It's up to you, Unix gives you fine control!

Command Syntax	What Happens
`$ myprogram`	Run the program `myprogram` and allow the standard output and standard error streams to print on the terminal screen. If `myprogram` expects to read some data from its standard input stream, it will just wait patiently until you type something because standard input, by default, is connected to the keyboard of the terminal where you ran this command. You didn't specify a full path to `myprogram` so it needs to be stored somewhere in your PATH.
`$ myprogram < datafile`	Just like above, except that `myprogram` is now reading its standard input from the file named `datafile`.
`$ myprogram > saveit`	Run the program `myprogram` and send its standard output stream into a file named `saveit.` If the file `saveit` already existed, this overwrites it! If the program prints any error messages, they will appear on the terminal screen as you only redirected standard output, not standard error.
`$ myprogram >> saveit`	Run the program `myprogram` and append its standard output stream to the end of the file named `saveit.` Any data in `saveit` is preserved as the new output is appended to the end of the file. Again, any error messages will appear on the terminal screen.
`$ myprogram > saveit 2> oops`	Send the standard output of `myprogram` to `saveit` and standard error (stream #2) to `oops.`
`$ myprogram > saveit 2>&1`	Send the standard output (I/O stream #1) to `saveit`, and send standard error (stream #2) to the same place as stream #1. That is, save both standard output and standard error to the file `saveit`.
`$ myprogram \| otherprogram`	Build a pipeline — the standard output of `myprogram` is used as the standard input of `otherprogram`.
`$ prog1 \| prog2 \| prog3`	Build a bigger pipeline — the standard output of `prog1` is used as the standard input of `prog2`, and its standard output becomes the standard input of `prog3`.

Editing Text Files With `vi`

Now you're ready to run some programs! But those programs will need data, and the programs may be simple shell scripts that you create. This means that you need to edit simple text files.

This means that you really need to learn some standard Unix text editor, either vi or emacs. Sorry, that's just the way that it is.

Note that vi may really be vim, which stands for "Vi Improved", on your system, or else both vi and vim may be available. If you use a variety of Unix systems, you might want to get into the habit of at least attempting to run vim and falling back to the slightly less friendly vi if necessary.

You can find a good on-line tutorial of vi at YoLinux.com, it goes deeper than this page does. Wikibooks has a book on-line, "Learning the vi editor". Other useful pages include vim.org and thomer.com.

You may also have the vimtutor program installed on your system, or you may be able to add it.

Amazon
ASIN: 059652983X

Crucial things to know about vi (and vim) include:

It has two modes: command and text insertion. When you're in text insertion mode, pressing keys just inserts those characters into the file at the current location of the cursor. When you're in command mode, pressing keys causes commands to happen.

Be careful, as always it is a bad idea to execute random commands. Typing away when you think you're inserting text but those characters are being interpreted as commands can have strange results at best, disastrous at worse.

To go from text insertion mode to command mode, just press the <Escape> key.

Now, when you're in command mode, there are many commands, but these are the vital ones plus a few more especially useful ones. The vital ones are those with green backgrounds. The good news is that you only need to know a handful of commands to be fairly useful. You will want to learn more so you can use the editor much more efficiently:

Cursor Navigation	Meaning
`h j k l`	Move one character to the left, town, up, and right, respectively. You must know how to navigate using these four keys. Some of you may have noticed that the arrow keys and the <Page Up> and <Page Down> keys work — at least they work some of the time, on some keyboards, if your environment fully understands and can handle your terminal emulator. Do not count on those special keys, know how to navigate with keys `h,` `j,` `k,` and `l`!
`^F ^B`	Move forward (control-F) and back (control-B) by one full screen.
`^U ^D`	Move up (control-U) and down (control-D) by one-half screen.
`w b`	Move forward one word (w) or back one word (b).
`e`	Move to the end of the current word.
`0 $`	Move to the beginning (0, zero) or end ($) of the current line.
`{ }`	Move to just before ({) or just after (}) the current paragraph or block of code.
`H K`	Move up to the first line (H) or down to the last line (L) currently visible on the screen. Notice how upper-case `H` and `L` are somewhat analogous to lower-case `h` and `l`. This is often the case with `vi/vim` commands.
Searching & Jumping	Explanation
`/some-string<Enter>`	When you type the `/` the cursor jumps down to a search and command line below the page of text shown on the screen. Then you type the string you want to search for, `some-string` in the example here. Then, when you press `<Enter>` the cursor will jump to the next instance of `some-string`, if it appears in the file.
`n N`	Pressing `n` jumps you to the next match for the most recent search string. `N` searches in the opposite direction. A message in the status line at the bottom of the screen will announce when the search wraps around from the end of the file to the beginning or vice-versa.
``	Two back-quotes in a row jump the cursor back to where it was before the last search jump.
`123G<Enter>`	This would make the cursor jump to the first text on line number 123.
`^G`	Pressing `<Ctrl-G>` announces the current file name and line number of the cursor position in the status line at the bottom of the screen. You can configure `vi` to always display the word and line counts and positions in the status bar. This lets you check it manually.
Undoing & Re-doing	Meaning
`.`	Repeat the last deletion, addition, or modification at the current cursor position.
`u`	Undo the last deletion, addition, or modification.
Entering & Leaving Insertion Mode	Meaning
`i a`	Insert new text before the cursor position (i), or add new text after the cursor position (a). As with all commands that put you into insertion mode, the editor will insert everything you type until you leave insertion mode and get back into command mode.
`I A`	Insert new text at the beginning of the current line (I), or add new text to the end of the current line (A).
`o O`	Open a new line below the current line (o), or before the current line (O).
`<Escape>`	Press the `<Escape>` key to leave insertion mode.
Deletion	Meaning
`x`	Delete the character currently under the cursor.
`dw`	Delete one word.
`dd`	Delete the current line.
`D`	Delete from the cursor through the end of this line.
And more...	Combine `d` (for "delete") with any motion character above to mean "delete from the current cursor position through the specified motion." That's really what `dw` means, but you can do things like delete the next half-screen with `d^U` or delete through the rest of the paragraph or block of code with `d}.`
Modification	Meaning
`r`	Replace the current character. Press `r` to issue this command, then press the new character. You will see the character change in place.
`R`	Replace from the cursor position through the end of the current line. Notice that this command and all the others after simple `r` throw you into text insertion mode. You can use `R` to replace the rest of this line with a few more words plus hundreds of lines of new text.
`cw`	Change the word. More precisely, change from the current cursor through the rest of this word.
`cc`	Change the current line. Compare `cc` for "change this line" to `dd` for "delete this line".
`C`	Change from the cursor through the end of this line. Compare `C` for "change the rest of this line" to `D` for "delete the rest of this line".
And more...	Just as with deletion, you can combine `c` (for "change") with any motion character above to mean "change from the current cursor position through the specified motion."
Copy & Paste	Meaning
Background...	You may know it as "copy & paste" because of some recent upstart programs that made up new terminology as they went along. But `vi` dates from at least the early 1980s, and for over a decade it was using the terms "yank & push" to refer to yanking some data off the screen and into a buffer, and then pushing it back out somewhere else.
`yy`	Yank the entire line into the buffer.
`p`	Push the data back out after the cursor position. If it's an entire line or more, it will appear below the current line. If it's a single character through a partial line, it will appear after the cursor position on this same line.
`P`	Just like `p` except it pushes it before the current cursor position, not after.
And more...	This should come as no surprise, but you can combine `y` with a motion command character to yank a word (`yw`), yank the rest of the line (`y$`), yank the rest of the current paragraph or block of code (`y}`), and so on. Another important thing is that whatever you last deleted with one of the deletion character commands is in the buffer, so you can move a block from here to there by deleting it with one command, moving to the new position, and pushing it back out.
`ed`-style Commands	Meaning
Background...	Pressing ":" puts you into single-command mode, with the cursor jumped down to a ":" prompt below the editing area. This lets you describe changes to be made through the entire file, or through a range of lines. One somewhat (but not terribly!) complex command can do the work of a huge sequence of moves and changes.
`:1,22s/oldstring/newstring`	On lines 1 through 22, do a substitution replacing the first instance of `oldstring` on each line with `newstring`.
`:1,$s/oldstring/newstring` `:%s/oldstring/newstring`	The same thing, except on all lines. Note that you can specify the line range as `1,$`, "from line #1 to the end", or the short-cut `%`.
`:1,$s/oldstring/newstring/g` `:%s/oldstring/newstring/g`	The same thing, except make it a global change — change every instance of `oldstring` on each line to `newstring`.
`:%s@/bin@/usr/bin@g`	This shows how to handle search or replacement strings containing "`/`" — use a different delimiter character. This would change every instance of "`/bin`" to "`/usr/bin`".
Saving (or not) and Exiting	Meaning
`:w`	Write out the file as it exists now in the editor buffer, but stay within the editor session.
`:q`	Quit the editor session. If you have made changes that haven't been written, maybe you want to write them out first. But if you made some changes you didn't want to make...
`:q!`	Quit without writing the file. You started making changes but you weren't happy with the way the editor session was going.
`:wq ZZ`	Write the file to disk and quit the editor session. Classes usually teach `:wq` as the combination of `:w` and `:q`. Eventually you learn that `ZZ` does the same thing.
`:w!`	Write the file to disk, even though you don't have write permission, and stay within the editor session. This will only work if you own the file or you're `root`. (You need to have the permission to change the permissions to grant yourself write access, make the write, and then change the permissions back.)
`:wq!`	As above, but also quit.

Listing Files and Directories

The Swiss Army Chainsaw is the ls command. It has many options, to the point that the BUGS section of the GNU manual page has at times referred to the large number of options and the inability of users to accurately remember and use most of them as a bug. It would properly be a design flaw, but there's a point in there somewhere.

Useful applications of ls include:

Command	Meaning
`ls`	List the names of the files, or at least those with names not starting with a "." character. Remember that directories are just a special type of file. With no arguments, `ls` reports on the current directory. With one or more arguments, `ls` reports for each of those. If any argument is a directory, `ls` reports on its contents, not on the directory itself.
`ls -a`	Add `-a` to report on all of the files, even those with names starting with a "." character.
`ls -d`	If any argument passed on the command line happens to be a directory, then report just that directory itself and not its contents.
`ls -l`	Produce a long listing with details as described below.
`ls -F`	"Decorate" some of the reported file names with single characters indicating the file type, including: `/ =` directory `* =` executable `@ =` symbolic link `= =` socket Note that the GNU version of `ls`, which might be simply `/bin/ls` on your system or perhaps something like `gls` or `colorls` on a BSD system, can also use color (if the terminal emulator supports it) to indicate data types for plain old files.
`ls -R`	Produce a recursive listing, listing the specified file(s) and then their contents for any directories in the list, and their contents, and so on.
`ls -t`	Sort the output in time order, newest to oldest.
`ls -r`	Reverse the order of the output. So, to see the files in reverse time order, oldest to newest, use: `lt -tr`
Combinations	If you like lots of details, you could issue a command like the following for a recursive (R) list of all (a) files, even dot-named ones, in the current directory, decorating the file names to distinguish directories, files, symbolic links, sockets, and so on, use: `ls -RaF` The order does not matter so you could have used any of these to get the same result: `ls -aFR` or `ls -aRF` or `ls -FaR` or `ls -FRa` or `ls -RFa`

Now, to read that long listing:

$ ls -lFd [Dp]* /tmp/file1OjOkm /dev/ttyS0 /dev/sda1 /usr/bin/passwd /usr/bin/man
brw-rw----.  1 root     disk      8,  1 Apr 04 20:43 /dev/sda1
crw-rw----.  1 cromwell uucp      4, 64 Apr 04 20:43 /dev/ttyS0
srwxrwxr-x.  1 cromwell wheel         0 Apr 06 11:41 /tmp/file1OjOkm=
-rwxr-sr-x.  1 root     mail      47360 Dec 14 17:55 /usr/bin/mail*
-r-s--x--x.  1 root     shadow    22104 Dec 14 13:56 /usr/bin/passwd*
drwxr-x---. 12 cromwell wheel    126976 Apr 07 14:14 Documents/
lrwxrwxrwx.  1 cromwell cromwell     18 Jan 28 14:40 pete@ -> purdue.jpeg
-rw-r--r--.  1 cromwell wheel    314943 Jan 22 12:16 purdue.jpeg

^\________/  ^    ^          ^      ^   \__________/     ^
|    |       |    |          |      |        |           |
|    |       |    |          |      |        |       File name
|    |       |  User       Group    |    Time stamp
|    |       |                      |
|    |       |                   Size in bytes for regular files.
|    |       |                   Major,minor device numbers for
|    |       |                   device special files.
|    |       |
|    |   Number of links pointing to this in the file system.  "Documents" is
|    |   a directory with 11 subdirectories.  There is one link in the current
|    |   directory pointing to it by the name "Documents" plus one more link
|    |   in each of those 11 subdirectories pointing to it by the name "..".
|    |
| Permission mask:  1st three for the user r = read
|                   2nd three for group    w = write
|                   3nd three for others   x = execute for files
|                                              search for directories
|                                          s = setuid/setgid, set user ID and
|                                              set group ID
|                                          - = permission not granted
| If there is a final "." on the permission mask, as seen here, it means that
| additional NSA Security-Enhanced Linux ACL (Access Control List) restrictions
| apply.  See the SELinux ACLs with:  ls --context
| Similarly, on BSD and Solaris a "@" or "+" character indicates extended ACLs.
| See them with:  getfacl
|
|  So, "Documents" has permissions rwxr-x---.  cromwell  wheel
|    Subject to unexpected restrictions by the extended SELinux ACLs:
|    Owner cromwell can read, write and execute (search)
|    Group wheel members can read and execute (search)
|    Others have no permissions
|
|  "/usr/bin/passwd" has permissions r-s--x--x.  root  shadow:
|    Numeric mode 4511.
|    Subject to unexpected restrictions by the extended SELinux ACLs:
|    Anyone can execute it, but that one process will have the effective user
|    ID of "root" so it can read and write /etc/shadow.
|
|  "/usr/bin/mail" has permissions rwxr-sr-x.  root  mail
|    Numeric mode 2555.
|    Subject to unexpected restrictions by the extended SELinux ACLs:
|    Anyone can execute it, but that one process will have the effective group
|    ID of "mail" so it can manipulate the directory /var/spool/mail.
|
File type:  -  =  regular file
            d  =  directory
            l  =  symbolic link
            s  =  socket
            b  =  block (buffered) special device
            c  =  character (raw) special device

Manipulating Files and Directories

Command	Meaning
`cp foo bar`	Copy file `foo` to `bar`. If `bar` is a directory, this creates the file `bar/foo`. Otherwise, `bar` will be a file that is a copy of `foo`.
`mv foo bar`	Move file `foo` to `bar`. If `bar` is a directory, this creates the file `bar/foo`. Otherwise, `foo` will have its name changed to `bar`.
`mkdir foo`	Create a directory named `foo`.
`rmdir foo`	Remove the directory named `foo`. This fails with an error report if `foo` is not empty.
`touch foo`	If `foo` exists, change its modification timestamp. If `foo` does not exist, create it as an empty file.
`ln -s foo bar`	Create a symbolic link named `bar` pointing to `foo`.
`rm foo`	Remove `foo`.
`rm -r foo`	Recursively remove `foo` and its contents.
`rm -rf foo`	Recursively remove `foo` and its contents and force it to happen with no error or warning messages. Be careful using this one!

Examining File Contents

Command	Meaning
`more foo`	View the contents of `foo` one screen at a time. Press `<Spacebar>` to move forward one screen, `b` to move back one screen, and `q` to quit. Bonus: Press `v` to jump into a `vi` session at that point in the file.
`less foo` `my-program \| less`	The command `less` is like `more` except it's more capable. Those GNU folks love their puns and self-references.... One of the most useful differences is that `less` can back up when reading a stream, while `more` can only back up when reading files.
`cat foo`	Print all the contents of `foo` to the screen.
`cmp foo bar`	Compare the files `foo` and `bar`, answering the question "Do `foo` and `bar` have identical contents?"
`diff foo bar`	Show the differences between the contents of `foo` and `bar`.
`grep blah foo`	Print just those lines of `foo` containing the literal string `blah`...
`grep -i blah foo`	... except ignore the case of the letters in `blah`
`grep -w blah foo`	... only when `blah` appears as an isolated word
`grep -iw blah foo`	... ignoring case and only when `blah` appears as an isolated word
`grep -v blah foo`	... except reversing the sense of the search, only the lines that do not contain the string `blah`
`egrep 'blah\|fnord' foo`	Notice that this uses `egrep`, not `grep`, "e" for "extended", to search for those lines that contain either the literal string `blah` or the literal string `fnord`.
`egrep -ivw 'blah\|fnord' foo`	Output only those lines that contain neither the isolated word `blah` nor the isolated word `fnord`, ignoring upper versus lower case. The options can appear in any order, `-ivw`, `-iwv`, `-viw`, `-vwi`, `-wiv`, or `-wvi`.

Changing File Permissions

Permissions can be specified in octal. Yes, octal, base 8. It will eventually become second nature, but to the uninitiated this is like Cypher in The Matrix when he waves his hand at the screens of cascading code and says he no longer sees the character patterns but instead what they represent. "All I see now is blonde, brunette, redhead." Only six patterns are really useful:

  rwx  ->  111  =  7
  rw-  ->  110  =  6
  r-x  ->  101  =  5
  r--  ->  100  =  4
  --x  ->  001  =  1
  ---  ->  000  =  0

Once you learn this, you will see rwxr-x--- and read it as 750. Really. And you will immediately understand it as "Full access for the owner, all but writing for the group, nothing else."

Permissions can be specified explicitly in octal, as above, or symbolically:

                [ugo][+-=][rwxs]
                  ^    ^    ^
                  |    |    |
  +---------------+    |    +-------+
  |                    |            |
  |                    |            |
u = user (owner)     + = add      r = read
g = group            - = remove   w = write
o = other (world)    = = equal    x = execute (or search, for directories)
                                  X = search (only if it's a directory)
                                  s = set-UID or set-GID

So, with all that background we're ready for examples:

Command	Meaning
`chmod 760 something`	Change the permissions of the file `something` to mode 760, `rwxrw----`.
`chmod o+r something`	Add the permission for others to read the file `something`, but leave the other permissions alone.
`chmod -R o+r something`	Apply that change recursively to the directory `something`, its contents, and so on.
`chmod -R go=rX something`	Recursively give the group and others permission to read, and if it's a directory, also permission to search, and take away their permission to write, within the directory `something`, its contents, and so on. Leave the user permissions as they are.

Who Am I and Where Am I?

Command	Meaning
`id`	List your credentials: your user ID, your primary group ID, and any other groups to which you belong.
`pwd`	Print the current working directory.
`echo ~`	If you were to run the command `cd` with no argument, so you change to your home directory, where would that be?

Command History

`history`	Print your command history. The first column is the command number in the sequence. In some shells like `tcsh`, the second column shows the time at which that command was run.
`!!`	Re-run the last command in your history.
`!395`	Re-run command #395 in your history.
`!vi`	Re-run the most recent command starting with "vi". This doesn't have to be the command `vi` itself, just the most recent command starting with those two letters. It could be `vi` or `vim` or `view` or `vimtutor` or `visudo` or ...
`!v:p`	You are not certain, but you think that the most recent command starting with "v" might be the one you want. The ":p" means only print what would be run, do not actually run it.
`!v:s/jpg/conf`	Let's say that the last time you edited a file with `vi` or `vim` you ran this command: `vim /some/long/path/to/whatever.jpg` when you really should have run this: `vim /some/long/path/to/whatever.conf` This history modifier means "Run the most recent command starting with 'v' except change the first instance of 'jpg' to 'conf'."
`^jpg^conf`	This does the above replacement of "jpg" for "conf" but applies to the most recent command only.
`!c:gs/123/124`	Let's say that you recently ran this command: `cp /path/to/dscf0123.jpg ~/Pictures/image-0123.jpg` Now you want to do the same thing for the next sequentially numbered image file name. This history modifier means "Run the most recent command starting with 'c' except change every instance of '123' to '124'."

Identifying and Controlling Processes

Command	Meaning
`ps axuw`	List many details about all running processes. On Solaris you will need to use the full path to the much more useful BSD version of the command, so you don't get the fairly lame SVR4 one in `/bin`: `/usr/ucb/ps axuw`
`top`	This lists the processes using the most of the CPU cycles, by default in decreasing order of CPU use percentage. See your local manual page, as `top` formats its output differently on different Unix implementations. There is usually a way to change the column by which the output is sorted, typically with the characters `<` and `>`, or the left-arrow and right-arrow keys, if they work correctly. There will probably be some way to change the update timing and specify which processes are displayed in terms of PID and/or user. The `htop` package provides the same information in a color-coded display, with alternative ways of changing the sortings. The `glances` package presents CPU, memory, and storage and network I/O in continuously updated tables.
`kill -HUP 1234`	Send a HUP signal to the process with PID 1234. Many daemons interpret a HUP signal to mean "Keep running, but re-read your configuration file."
`kill -TERM 1234`	Send a TERM signal to the process with PID 1234. A well-written program will use this opportunity to shut itself down cleanly.
`kill -KILL 1234`	You tried a reasonably polite and gentle TERM signal but that did not work. Use a bigger hammer that won't shut it down cleanly, but it will kill the process.
`pkill -HUP named`	Like `kill`, but you can specify processes by names without looking up their numeric PID.

Network Commands

Network
Commands

For far more details and explanations of how to read some sample output, see my page with network commands for various operating systems. Here is a short version. Some of these will require you to have /sbin and maybe /usr/sbin in your PATH. Why? Because with a lot of other commands in those directories, they are administrator and power user tasks but not the sort of the thing the average user does. See the above section on PATH for details on this.

Command	Meaning
`hostname`	Show the hostname, probably the fully-qualified domain name. That is, `www.cromwell-intl.com` instead of just `www`.
`ip addr`	Show the IP address(es) and subnet mask for each interface. Warning — This used to be done with the `ifconfig` command. That command cannot be trusted to work as expected on a modern system!
`ip route`	Show the routing table. Warning — This used to be done with the `netstat -r` command. That command cannot be trusted to work as expected on a modern system!
`cat /etc/resolv.conf`	Show the DNS configuration: DNS domain and DNS name server(s).

Controlling System State

For Linux with systemd, meaning most any Linux distribution after 2013–2014:

Command	Meaning
`systemctl poweroff`	Shut Linux down cleanly, halt the OS, and power off the system.
`systemctl reboot`	Reboot Linux cleanly.
`systemctl rescue`	Take the system to rescue mode, which means being `root` on the console. Depending on the configuration, you may need to type the `root` password.

For UNIX family in general and Linux before systemd:

Command	Meaning
`init 0` `halt` `shutdown -h -t 0 now`	Shut UNIX down cleanly and halt the OS. You may be able to turn off the power, but to accomplish that you may need to use: `halt -p` Note that in some forms of UNIX (e.g., Solaris) these may behave rather differently and `halt` may be inappropriate.
`init 6` `reboot` `shutdown -r -t 0 now`	Shut UNIX down cleanly and reboot. Again note that in some forms of UNIX (e.g., Solaris) these may behave rather differently.
`init 1`	Go to single-user mode: maintenance mode, `root` on the console. On BSD you will need to use: `kill -s TERM 1`

Managing Users and Groups

All these commands must be run as root:

Command	Meaning
`useradd -m julie`	Create a user named `julie` and create the home directory `/home/julie` based on the template in `/etc/skel`.
`userdel julie`	Delete the user `julie`, the home directory and files will be left on the system.
`passwd julie`	Assign a password to `julie`.
`groupadd -g 100 staff`	Create a group named `staff` with group ID 100.

Here are the files defining users:

File	Defines
`/etc/passwd`	This specifies most of the details about a user. The file must be world-readable so any user could use `~username` and have that resolved to a full path. The fields are: login UID primary GID real name (or "GECOS field", for historical reasons) home directory command interpreter shell
`/etc/shadow`	This specifies the password and related details about a user. Therefore it must be readable only by `root` and the operating system itself. The fields are: login hashed password and salt account and password aging controls
`/etc/groups`	The groups are defined here, one line per group. If a user belongs to more than one group, their primary group is defined in `/etc/passwd` and the other groups here. Let's say that `/etc/passwd` contains: `cromwell:567:101:....` and `/etc/group` contains: `wheel::10:cromwell` `users::101:` So, the user `cromwell` has a primary group ID of 101, meaning `users`, and also belongs to the group `10` or `wheel`. So, when that user creates a new file it will belong to the group `users` by default, and the user can optionally change that file to group `wheel` by running a command shown below.

Changing File Ownership and Group

With one exception, chgrp, all these commands must be run as root:

Command	Meaning
`chown julie bigfile`	Change the owner of the file `bigfile` to the user `julie`.
`chgrp wheel bigfile`	Change the group of the file `bigfile` to the group `wheel`. The owner of the file can do this as long as they belong to the target group.
`chown root:sys bigfile`	Change the owner of the file `bigfile` to the user `root`, and the group `bigfile` to `wheel`.
`chown -R root:sys data`	Apply that change recursively to the directory `data` and its contents.

Finding Files and Directories

Most people seem to find the find command the most confusing, with a manual page that may not be terribly helpful. Here is an attempt to present some useful examples.

Command	Meaning
`find /home -name trouble`	Find, anywhere under the `/home` directory, all files named `trouble`.
`find /home -name 'trouble*'`	Find, anywhere under the `/home` directory, all files with names starting with `trouble`. Notice that you have to hide the wild card character from the shell so it does not interpret it in terms of the files in your current working directory. Used as seen here, the wildcard is passed to `find` so it can do the search you want.
`find / -name 'trouble*'`	The same search, except look through the entire file system.
`find /tmp /var -name 'trouble*'`	The same search, except look under both `/tmp` and `/var` but nowhere else.
`find /home -type l`	Do any users have symbolic links under their home directories?
`for DIR in /home/*` `> do` `> echo $DIR $(find $DIR -type f \| wc -l)` `> done`	What are the user logins, and how many files does each one own? That command within backquotes, `$(...)`, is executed first. Its output, a number in this case, is substituted into the outer command, the `echo`.
`find /home -type l -name 'missing'`	Do any users have symbolic links with names containing the string "`missing`" under their home directories?

Do any users own a file containing the string "mystery", ignoring case?

find /home -type f -exec grep -i mystery {} /dev/null \;

Courage Wolf uses /usr/bin/vim. You're no longer in grade school. You don't need a cute animated doggy.

The Microsoft animated puppy is helpful if you are 4 years old and intimidated by technology.

Notice the bizarre syntax of find -exec — the command you want to execute for every file matching the search criteria (in this case, under /home and a regular file) must appear between -exec and the delimiter formed by an escaped semicolon. The command will be run once for every file found, with {} replaced by the name of that file.

The example here always passes a second file name argument, /dev/null. The grep command can read that empty file in no time at all, but the presence of a second file name means that any output will be prefaced with the file name where the string was found. Compare the output of the two grep commands in this sequence:

echo foo > /tmp/bar
$ grep foo /tmp/bar
foo
$ grep foo /tmp/bar /dev/null
/tmp/bar:foo

There is usually a way to tell your grep command to include that file name even with a single file name on the command line, but the precise way to do that varies from one implementation to the other, and it isn't always possible at all.

Where The Pieces Go

This is hard to predict in detail, but the following table provides generally useful guidance.

See my page on file system design for performance and security for more detail, and suggestions of how to design a partitioning scheme.

Directory	Contents
`/etc`	Most of the system configuration goes in `/etc/` and its subdirectories.
`/etc/rc* /etc/init.d`	Boot scripts, on older systems. Systemd replaces all the boot scripts with compiled binaries, and completely changes the way the system is started once the kernel has been loaded and found the root file system. Click here for the details of Linux booting. Systemd components go in `/lib/systemd`.
`/home`	User home directories
`/bin /usr/bin /usr/local/bin`	Programs generally useful to users
`/sbin /usr/sbin /usr/local/sbin`	Programs useful to boot and maintain the system, but not frequently used by users. However, some of the commands mentioned on this page, `ifconfig` and `ip` are prominent examples, are located in one of these.
`/usr/ucb`	On Solaris only, this is where the good and true and proper BSD versions of tools exist, as opposed to the less useful SVR4 versions in `/bin` or wherever. The most prominent example is `ps` but there are others where the UCB behavior may be more useful.
`/var`	All sorts of vital things that users never notice. Log files are in `/var/log` and/or `/var/adm`, mail and printing may use `/var/spool` or similar, and so on.
`/lib /usr/lib /lib64 /usr/lib64 /usr/local/lib`	Shared libraries critical to most of the executables on the system. Seriously. Do not mess these up.
`/dev`	Device-special files mostly ignored by users except for `null`, `zero`, `random`, `urandom`, and maybe a few others.
`/boot`	On a Linux system, the kernel itself and some support files are stored in here.
`/proc`	A user-accessible file system, or at least what appears to be a file system. Really it's a collection of kernel data structures. On Linux and some BSD versions, this is directly useful. On Solaris and most other commercial Unixes, this is really useful only for interpretation by `ps` and debuggers. That is, if you consider running a debugger on a running kernel "useful".
`/usr/share`	Application configuration files, collections of error messages and their translations into various languages, manual pages under `/usr/share/man`, further documentation under `/usr/share/doc`, and who knows what else your vendor or distribution assembler has stuffed into this area.