IT 244: Introduction to Linux/Unix

IT 244: Introduction to Linux/Unix
Class 11

Microphone

Graded Quiz

You can connect to Gradescope to take weekly graded quiz today during the last 15 minutes of the class.

Once you start the quiz you have 15 minutes to finish it.

You can only take this quiz today.

There is not makeup for the weekly quiz because Gradescope does not permit it.

Readings

If you have the textbook you should read chapter 5, The Shell.

Homework 6

I have posted homework 6 here.

It is due this coming Sunday at 11:59 PM.

Midterm

The Midterm exam for this course will be held on Tuesday, March 19th.

That is the first Tuesday after Spring Break

It will be given in this room.

It will consist of 25 questions like those on the quizzes.

60% of the questions will come from the Ungraded Class Quizzes.

The other 40% of points will be questions that I create specially for this exam.

For these questions you will have to know

Absolute and relative pathnames
The PATH system variable
Access permissions
Redirection & and pipes
grep
Utilities

The last class before the exam, Thursday, March 7th, will be a review session.

You will only be responsible for the material in the Class Notes for that class on the exam.

You will find the Midterm review Class Notes here.

If for some reason you cannot take the exam on the date mentioned above you must contact me to make alternate arrangements.

The Midterm is given on paper.

I scan each exam paper and upload the scans to Gradescope.

I score the exam on Gradescope.

You will get an email from Gradescope with your score when I am done.

The Midterm is a closed book exam.

You are not allowed to any resource, other than what is in your head, while taking the exam.

Cheating on the exam will result in a score of 0 and will be reported to the Administration.

Remember your Oath of Honesty.

To prevent cheating, certain rules will be enforced during the exam.

Remember, the Midterm and Final determine 50% of your grade.

Questions

Are there any questions before I begin?

Tips and Examples

Studying for the Midterm with Flashcards

60% of the point on the Midterm come from Class Quiz Questions
You can use the flashcards you create for these questions when studying for the Midterm
But not all Class Quiz Questions will appear on the Midterm
If the flashcard question covers a topic not in the Midterm Review ...
you do not have to study it for the exam
You should remove these flashcards from your collection

Getting Your First IT Job

Students sometimes come to me asking for advice on how to get an IT job
I have not had a job in industry for many years
So I usually refer them to Career Services
They provide a web site called Handshake where companies can find interns
In IT, as in many fields, your first job is the stepping stone to your career in the field
But it can be hard to find an internship or or a job when you have no IT experience
If this is a situation you face, there are things you can do
If you have a job, check with your current employer
They must have computers somewhere and maybe you can help keeping them running
If they have an IT Department ask if there is something you can do for them in your free time
You can do something similar with local organization like a church, temple, mosque or youth group
They probably use a computer for some of the work they do or perhaps they need a web page
Volunteer to do some IT work for them in return for a letter talking about the work you did
You can cite this work in your resume
Another place to find volunteer opportunities is Volunteer Match
They have virtual opportunities that in many different areas
Or perhaps you can find some open source project that needs help
The Free Software Foundations is based in Boston and often needs volunteers
Many people who are looking to hire people say they are having a hard time ...
finding people who take their work seriously
Whenever you get a job, no matter how menial ...
be sure to do your best ...
and take the work seriously
Employers don't want people who don't give a damn
The economy goes through cycles and sometimes jobs are hard to find
Just get in the habit constantly looking for opportunities
But above all don't give up

Keep In Contact

When solving a problem, one of your best resources ...
is the people you work with
Sometimes you get you get too wrapped up in a problem ...
and can't see something that is obvious to others
Other times you need a function of technique ...
that you have never used before
Nobody in IT knows everything
Of course you could always Google for an answer
But someone you work with might be able to explain it ...
saving you hours of searching
All of us who work in IT are critically dependent on our peers
This is why I urge all of you to keep in touch with those you meet in your classes ...
or on the job
Most of the jobs I have had in my life ...
have come from leads from people I know
Make sure you have the email or text address of everyone you study or work with ...
and keep in regular contact
Maybe go out for a beer with them one a month
You won't regret it

Viewing Directory Permissions

Running ls -l on a directory will show the permission of everything inside that directory

$ ls -l tmp
total 8
-rw-r--r-- 2 ghoffman faculty 22 Jun 19 14:15 lines.txt
-rw-r--r-- 2 ghoffman faculty 22 Jun 19 14:15 test.txt

What if you wanted to see the permission on the directory itself?
You have two options

You can run ls -l on the parent directory

$ ls -l ~
total 80
drwxr-xr-x  4 ghoffman grad       4096 Oct 15  2016 bin
drwxr-xr-x  6 ghoffman faculty    4096 Jan 20 14:44 code
drwxr-xr-x  6 ghoffman faculty    4096 Sep  9  2016 course_files
...
drwxr--r--  2 ghoffman faculty    4096 Jun 19 14:15 tmp

Or you can run ls -ld on the directory itself

$ ls -ld tmp
drwxr--r-- 2 ghoffman faculty 4096 Jun 19 14:15 tmp

The -d option tells ls to show information on the directory
Not the things inside the directory

Never Use . . in an Absolute Pathname

Sometimes when scoring a homework script I'll come across a pathname like this
```
/courses/it244/ghoffman/../..
```
This is a valid pathname but it's stupid
Any pathname starting with / is an absolute pathname
That means it will work whatever directory you are in
. . means your parent directory
. . should only be used in a relative pathname
When you use it in an absolute pathname it merely goes back to a previous directory
So the pathname above first goes down to /courses/it244/ghoffman
And then goes up two directories to /courses/it244
This is just a longer way of writing the absolute pathname /courses/it244
You should never write anything on the command line that is longer than you need it to be
Longer command lines increases the chance of an error

Write Permissions on a Directory

Write permission on a directory only applies to what is inside the directory
It does not apply to the directory itself
If you have write permission on a directory you can
- Create a file or directory inside the directory
- Delete a file or directory inside the directory
- Rename a file or directory inside the directory
To delete or change the name of the directory you must have write permission on the parent directory

Using the `ls` Command

You don't have to be in a directory to run ls on it
If you run ls without an argument it lists your current directory
But if you give ls a directory as an argument it will list the contents of that directory
```
$ ls ~ghoffman/course_files/
cs115_files  it244_files  it341_files
```
So you can run ls on any directory in the file system if you have read permission on that directory

The `chmod` Chant

All of you need to memorize the following

read    write    execute
4       2        1
owner   group    everyone

This is all the information you need to use chomd with the numeric format

Using . When Copying a File

Sometimes, when copying a file to their current directory students will use a long pathname

They do something like this

$ pwd
/home/it244gh/it244/work

$cp ../test.txt ~/it244/work

They could have saved themselves some typing by using .
. stands for your current directory
Wherever you are in the filesystem
In the above example, . would have made things much easier
```
$cp ../test.txt .
```

Review

The root Account

On every Unix or Linux system there is a special account named root
root can access any file or run any program
root is an administrator account
It is used for system configuration and maintenance
Even a system administrator should not log in as root
Instead he or she should use a regular Unix account
And use sudo when running a command that needs root privileges
sudo allows a user to run a command that normally only root can run
When you run sudo it asks you for your password
Not the password of the root account
In order to run sudo you must be on the sudoers list
Only root or someone running sudo can change this list

Directory Access Permissions

The Unix access permissions work a little differently for directories than they do for files
Read and write permissions for a directory are similar to those for a file
Read permission allows you to list the names of things inside that directory using ls
Write permission allows you to create, delete or rename the things inside the directory
Write permission on a directory does not allow you to change the contents of a file in that directory
If you have write permission on a directory you can change what's inside it
But you cannot rename or delete the directory itself
To do that you have write permission on its parent directory
Execute permission on a directory allows you to cd into that directory

Links

Links are like shortcuts on Windows
Or aliases on a Mac
Links are pointers to files or directories
Each of you has an entry in your home directory called it244

In my home directory, I have such a link

$ ls -l it244
lrwxrwxrwx 1 ghoffman faculty 39 May 30 14:08 it244 -> /courses/it244/f23/ghoffman/ghoffman

This is a link to/courses/it244/f23/ghoffman/ghoffman
If you cd into this location and use pwd you will see
```
$ cd it244

$ pwd
/home/ghoffman/it244
```
This path reflects the route you took to get here
But it is not the real pathname of the directory
You can only see your true location if you use pwd with the -P (note the capitalization) option
```
$ pwd -P
/courses/it244/f23/ghoffman/ghoffman
```

The Two Types of Links

There are two types of links
- Hard links
- Symbolic, or soft, links
Hard links are older
A hard link is like a duplicate file name
Hard links can only point to files not directories
You can only have a hard link to a file
If that file is on the same hard disk partition as the link
Symbolic links are much more flexible
A symbolic link can point to a file or directory anywhere in the filesystem
Deleting a symbolic link does not delete the file or directory it points to

`ln`

To create a symbolic or soft link, use ln with the -s option

ln takes two arguments, a pathname and the name for the link

$ pwd
/home/it244gh

$ ln -s ~ghoffman/examples_it244 examples

$ ls -l examples
lrwxrwxrwx 1 it244gh libuuid 28 2012-09-17 17:53 examples -> /home/ghoffman/examples_it244

The second argument is optional
You can use either an absolute or relative pathname when creating a link

Setuid and Setgid Permissions

Sometimes a program needs to read or modify a file to do the work it was designed to do
The passwd command is used to change the password for an account
To do this it must change to the file /etc/shadow
But /etc/shadow is owned by the root account
No other account can change it
To deal with situations like this, two special permissions were created
- setuid
- setgid
Both permissions are set on programs
But they affect the files that the program acts upon
setuid permission allows anyone who runs the program to do what the owner can do
They can modify the any file the owner can modify
Let's say the owner of a executable file is also the owner of another file
If another user ran the executable file and it did not have setuid permission
They could not change that other file
But if the executable file has the setuid permission, they could change it
If you run a program with setuid permission, you can do anything the owner can do
passwd is owned by root
passwd needs to change /etc/shadow, which is also owned by root
But passwd has setuid permission
So when you run it you can change /etc/shadow
setgid permission applies to the group
Anyone who runs the program can do whatever the group can do
setuid and setgid permissions are only given to executable files and scripts
But while the permission is set on the executable files it works on other files
setuid permission appears as an s in the owner's executable column
setgid permission appears as an s in the group's executable column
Since setuid and setgid permission apply only to executable files there is no ambiguity in replacing x with s

Attendance

New Material

Running a Unix Command

You run a Unix command by typing the name of the command on the command line
The name of the command is really the name of the executable file that contains the code for the command
It turns out that there is another way you can run a Unix command
You can use a pathname for the executable file
To run that command
We can use which to find the absolute pathname of ls
```
$ which ls
/usr/bin/ls
```

I can run ls using this absolute pathname

$ /usr/bin/ls
bin      data          groups     lib32       media     proc  snap     swap.img  users
boot     dev           home       lib64       mnt       root  sources  sys       usr
cdrom    etc           home.ORIG  libx32      nobackup  run   spool    tmp       var
courses  etc.ORIG.tar  lib        lost+found  opt       sbin  srv      tools

I can also use a relative pathname

$ ../../usr/bin/ls /
bin	 data	       groups	  lib32       media	proc  snap     swap.img  users
boot	 dev	       home	  lib64       mnt	root  sources  sys	 usr
cdrom	 etc	       home.ORIG  libx32      nobackup	run   spool    tmp	 var
courses  etc.ORIG.tar  lib	  lost+found  opt	sbin  srv      tools

So there are two way you can run a Unix command
- Using the name of the executable file for the command
- Using a pathname for the executable file

Syntax of the Command Line

The syntax of the command line is straightforward
First comes the command
Then, perhaps, some options
Finally some arguments

The command is executed when you hit the Enter key

COMMAND  [OPTIONS]  [ARG_1]  [ARG_2]  ...  [ARG_N]

The brackets indicate that the contents are optional
Commands vary in the number of arguments they accept
Some accept none
Others require a specific number of arguments
Still others accept a variable number of arguments
Arguments must be separated by one or more spaces

Command Options

Many commands have options
Options modify the behavior of the command
Options are usually preceded by one or two dashes, -
GNU programs frequently have options that are preceded by two dashes, --
The options in GNU programs are usually words
The options in other Unix programs are usually a single letter
When a command uses a single dash, -, before an option
You can combine single letter options after the dash
An example of this is ls -ltr
This means run ls
- To get a long listing
- Sorted by modification date and time
- In reverse order
Options using two dashes, --, cannot usually be combined
In this case, each option must be written separately and preceded by two dashes
Sometimes the option can have it's own argument
When this happens, the argument is usually separated from the option by spaces
```
gcc  -o  prog  prog.c
```
Utilities that report the size of files usually do so in bytes
This works well with small files
But with large files, a size in bytes can be hard to read
Such utilities often have a -h, or --human-readable, option
With this option, the file size will be displayed in kilobytes, megabytes or gigabytes, as appropriate

df (disk free) shows the amount of space on the various filesystems

$ df 
Filesystem                       1K-blocks     Used Available Use% Mounted on
/dev/sda1                         12253360  6027788   5580096  52% /
none                                     4        0         4   0% /sys/fs/cgroup
udev                               2013316        4   2013312   1% /dev
tmpfs                               404836      560    404276   1% /run
none                                  5120        0      5120   0% /run/lock
none                               2024172        0   2024172   0% /run/shm
none                                102400        0    102400   0% /run/user
blade66:/disk/sd0g/courses/it244   8260768  2615904   5562240  32% /courses/it244
blade82:/disk/sd0f/home/jdu        8260768  8171264      6912 100% /home/jdu
mx1:/disk/sd1e/spool/mail          4129312  3403648    684384  84% /spool/mail
blade82:/disk/sd1h/home/ghoffman  78805984 32988544  45029408  43% /home/ghoffman

When used with the -h option df produces more readable output

$ df -h
Filesystem                        Size  Used Avail Use% Mounted on
/dev/sda1                          12G  5.8G  5.4G  52% /
none                              4.0K     0  4.0K   0% /sys/fs/cgroup
udev                              2.0G  4.0K  2.0G   1% /dev
tmpfs                             396M  568K  395M   1% /run
none                              5.0M     0  5.0M   0% /run/lock
none                              2.0G     0  2.0G   0% /run/shm
none                              100M     0  100M   0% /run/user
blade66:/disk/sd0g/courses/it244  7.9G  2.5G  5.4G  32% /courses/it244
blade82:/disk/sd0f/home/jdu       7.9G  7.8G  6.8M 100% /home/jdu
mx1:/disk/sd1e/spool/mail         4.0G  3.3G  669M  84% /spool/mail
blade82:/disk/sd1h/home/ghoffman   76G   32G   43G  43% /home/ghoffman

The -h also works with ls

If I run ls -l on a directory the file size will be given in bytes

$ ls -l
total 1236
-rw-rw-r-- 1 ghoffman faculty 26211 Jun 12 09:23 01_class_notes_it244.html
-rw-rw-r-- 1 ghoffman faculty 26138 Jun 21 15:25 02_class_notes_it244.html
-rw-rw-r-- 1 ghoffman faculty 29395 Jun  6 11:21 03_class_notes_it244.html
...

But if I use ls -lh I get a much more readable result

$ ls -lh
total 1.3M
-rw-rw-r-- 1 ghoffman faculty  26K Jun 12 09:23 01_class_notes_it244.html
-rw-rw-r-- 1 ghoffman faculty  26K Jun 21 15:25 02_class_notes_it244.html
-rw-rw-r-- 1 ghoffman faculty  29K Jun  6 11:21 03_class_notes_it244.html

Many commands display a help message when run with the --help option

$ mkdir --help
Usage: mkdir [OPTION]... DIRECTORY...
Create the DIRECTORY(ies), if they do not already exist.

Mandatory arguments to long options are mandatory for short options too.
  -m, --mode=MODE   set file mode (as in chmod), not a=rwx - umask
  -p, --parents     no error if existing, make parent directories as needed
  -v, --verbose     print a message for each created directory
  -Z, --context=CTX  set the SELinux security context of each created
                      directory to CTX
      --help     display this help and exit
      --version  output version information and exit

Report mkdir bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
For complete documentation, run: info coreutils 'mkdir invocation'

All GNU utilities accept this option

Device Drivers

All operations that need to access any hardware must do so through the kernel
The kernel controls access to
- RAM (short term memory)
- Disc (long term memory)
- CPU
How does the kernel know how to talk to new pieces of hardware?
Like a printer?
The answer is software known as a device driver
Every manufacturer of a device that works with a computer creates a device driver for it
The device driver lets the hardware and the operating system communicate

tty

In the very early days of Unix people used a machine called a teletype
In place of a screen and keyboard
It consisted of a keyboard and a printer
Here is a picture

A user would type a command on the keyboard
The output of the command would be printed on a continuous flow of paper
The name of this device was soon shortened to tty
The printer part of these devices was soon replaced with a video monitor
Any device that allows you to send text to a computer and see the output is called a terminal
But Unix still refers to it as tty

The Unix tty Device Driver

A terminal is a device connected to the computer
So it needs a device driver
The device driver for the terminal is called tty
It is built into the kernel
Otherwise you would not be able to talk to the machine
The tty device driver allows your keyboard to talk to the kernel
And for the kernel to send output to your screen ...
as you type on the command line
The characters you type are examined by the tty device driver
It does different things with different characters
Most of the time, it places the character in a buffer
A buffer is a space in memory that holds data for later processing
But the tty device driver responds differently to certain special characters
When the character you type is the backspace
It erases the previous character from the buffer
When the character is Control U something different happens
tty erases the buffer from the current insertion point to the beginning of the line
tty is responsible for all command line editing
When the tty gets a newline character it passes the contents of the buffer to the shell ...
so the shell can run the command
Newline is the character you get from hitting Enter on a Windows machine
Or Return on a Mac

Parsing the Command Line

The shell now takes the contents of the buffer
And breaks it up into tokens
Tokens are the strings of text separated by spaces
This action is called parsing
Parsing is the act of making a list of all the strings on the command line
Without the spaces
Next, the shell looks for the name of the command
Usually, the command name is the first string on the command line
The command can be specified by a simple filename
```
ls
```
Or by using a pathname to the executable file
```
/bin/ls
```

The PATH System Variable

When you run a program using a pathname at the terminal
The shell knows where to find the executable file

So we can run any command using the pathname of the executable file

$ /usr/bin/ls /
bin      data          groups     lib32       media     proc  snap     swap.img  users
boot     dev           home       lib64       mnt       root  sources  sys       usr
cdrom    etc           home.ORIG  libx32      nobackup  run   spool    tmp       var
courses  etc.ORIG.tar  lib        lost+found  opt       sbin  srv      tools

But when we run ls we use
```
ls
```
Not
```
/usr/bin/ls
```
How can the shell know where to find the executable file?
Programs are executable files that can be stored anywhere in the filesystem
So how does the shell find the correct file?
The shell checks a system variable called PATH
PATH contains a list of directories to search for an executable file
It checks each directory for a file with the right name
It checks each directory in order and stops when it finds the first match
It stores the locations in memory, so it won't have to look more than once

The hash command will show you this list

$ hash
hits    command
   1    /usr/bin/which
   1    /bin/ls

PATH always has a default value which is created when the system is installed

Here is the default value on our system

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/oracle/12.1/client64/bin

The absolute pathname of each directory is separated from the next by a colon, :
If the shell can't find the file it prints an error message
You will also get an error message if you don't have permission to run the file
You can modify the PATH variable in your own Unix environment
We'll see how to do this in a few classes

Running a Program in the Current Directory

How do you run a program if it is in your current directory?
You might think you simply enter the name of the program
But this is not the case
Whenever your run a command by simply typing its name Unix checks PATH
Unless your current directory is listed in PATH, Unix will not find the file
You could fix this by adding . to your PATH variable
But this would be a very bad idea
Why?
Consider the following situation
Some malevolent user adds a malware version of ls to a directory that contains some interesting content
He advertises the content to everyone
You go to the directory and run ls
Since . is in your PATH, Unix uses the local malware version of ls
This program runs the real ls but also loads into memory a keylogger
The keylogger records everything you type
Including your passwords
So putting . in your PATH is a very bad idea
Then how do we run programs in the current directory?
By using a pathname, not a filename
You do this by putting . / in front of the filename
Like this
```
./PROGRAM_NAME
```
. represents your current directory
And / separates the directory name from the rest of the pathname
So this is a relative pathname
If I wanted to run cat on the file memo.txt I could enter
```
cat  ./memo.txt
```
Which is the same as typing
```
cat  memo.txt
```

To run the executable Bash script foo.sh

$ ls -l foo.sh
-rwxrwxr-x 1 ghoffman grad 16 Oct  1 15:49 foo.sh

I would type
```
$ ./foo.sh 
Foo to you
```
Notice that I did not have to run bash to run this script
That's because the script file has execute permission set

Running the Command Entered on the Command Line

Once the shell knows where to find the program, it can ask the kernel to run it
The kernel does this by creating a process for that program
A process is a running program
And it is given its own chunk of RAM
Only the kernel can create a process
Each process needs resources to do its job
The most important is the space in RAM
Only the running program can use this space
This prevents one program from interfering with another
The kernel gives the process other system resources
Such as access to certain files
The kernel also gives the program the list of tokens from the command line
- The name used to call the program
- The options used
- The arguments used
The shell does not check the options or arguments
The shell has no idea what options or arguments are appropriate to a given program
The program has the responsibility to check the options and arguments for correctness
If the program gets the wrong options or arguments
The program has to produce the error message
While the program is running, the shell waits for it to finish
It does this by going into an inactive state known as "sleep"
When the program finishes it must tell the shell that it is done
It does this by sending the shell an exit status
The exit status is an integer that must be 0 or greater
An exit status of 0 means the program finished without error
Any exit status greater than zero indicates an error
A program can issue different exit status values for different types of errors
You can see the exit status of the last program run

By looking at the value of the system variable ?

$ cat foo
cat: foo: No such file or directory

$ echo $?
1

The exit status is 1, meaning the command failed
Notice that I had to put a dollar sign, $, before the variable name
That's how you get the value of a variable
When the shell gets an exit status it returns to an active state
It prints a prompt to the screen
And waits for the next command

Setgid Permission on Directories

The setgid permission works differently on directories
Every Unix account has a default group assigned to it
Whenever you create a file or directory this default group will be assigned to it
You can change this group afterwards
But a new file or directory is always assigned the default group when it is created
The default group for all your Unix accounts is ugrad
But my default group is faculty

If I go to a directory and create a file it will be assigned the faculty group

$ touch foo.txt

total 0
-rw-r--r-- 1 ghoffman faculty 0 Feb 27 17:30 foo.txt

However if I go to my ex directory and create a directory for today's Class Exercise
```
mkdir  ex10
```
it will be assigned to a different group
```
$ ls -ld ex10
drwxr-sr-x 2 ghoffman it244-1G 4096 Jun 18 08:26 ex10
```
Any file or directory I create inside ex10
Will also be assigned to this group
Look closely at the permissions assigned to this new directory
```
drwxr-sr-x 2 ghoffman it244-1G 4096 Jun 18 08:26 ex10
```
Notice that the directory has setgid permission
setgid permission works differently on a directory
Any file or directory I create in this directory will have the same group
And every directory created there will have setgid permission
Since ex10 is assigned the group it244-1G
Every file or directory inside it will have the same group
The group it244-1G is a grading group
It only contains my username and that of the Class Assistant

Look at the permission of the it244 class directory

drwxrwsr-x 12 ghoffman it244-1G 4096 May 30 14:07 /courses/it244/f23/ghoffman

This directory contain all your IT 244 directories
Notice the setgid permission and group is it244-1G
Since by default every file and directory has read permission set
I can read all your files
And copy them

Tips and Examples

Review

New Material

Graded Quiz

Readings

Homework 6

Midterm

Questions

Tips and Examples

Studying for the Midterm with Flashcards

Getting Your First IT Job

Keep In Contact

Viewing Directory Permissions

Never Use . . in an Absolute Pathname

Write Permissions on a Directory

Using the ls Command

The chmod Chant

Using . When Copying a File

Review

The root Account

Directory Access Permissions

Links

The Two Types of Links

ln

Setuid and Setgid Permissions

Attendance

New Material

Running a Unix Command

Syntax of the Command Line

Command Options

Device Drivers

tty

The Unix tty Device Driver

Parsing the Command Line

The PATH System Variable

Running a Program in the Current Directory

Running the Command Entered on the Command Line

Setgid Permission on Directories

Class Exercise

Class Quiz

Using the `ls` Command

The `chmod` Chant

`ln`