More Shell

Redirecting Input and Output

• A running program is called a process
• By default, every process has three connections to the outside world:
  • Standard input (stdin): connected to the keyboard
  • Standard output (stdout): connected to the screen
  • Standard error (stderr): also connected to the screen
    • Used for error messages

  • Figure 5.1: The Shell Running a Program

• You can tell the shell to connect standard input and standard output to files instead
  • command < inputFile reads from inputFile instead of from the keyboard
    • Don't need to use this very often, because most Unix commands let you specify the input file (or files) as command-line arguments
  • command > outputFile writes to outputFile instead of to the screen
    • Only “normal” output goes to the file, not error messages
  • command < inputFile > outputFile does both

  • Figure 5.2: Redirecting Standard Input and Output

• Example: save number of lines in all text files to words.len

  • $ wc -w *.txt > words.len
    

  • Nothing appears, because output is being sent to the file words.len

  • $ ls -t
    

    words.len       venus.txt       earth.txt
    

  • $ cat words.len
    

      4   9  69 earth.txt
      4   9  69 venus.txt
      3  12  62 words.len
     11  30 200 total
    

• Try typing cat > junk.txt
  • No input file specified, so cat reads from the keyboard
  • Output sent to a file
  • Voila: the world's dumbest text editor
• When you're done, use rm junk.txt to get rid of the file
  • Don't type rm * unless you're really, really sure that's what you want to do…
• Comment on this slide

Pipes

• Suppose you want to use the output of one program as the input of another
  • E.g., use wc -w *.* to count the words in some files, then sort -n to sort numerically
• Option 1: send output of first command to a temporary file, then read from that file

  • wc *.txt > temp
    sort -n < temp
    

• Option 2: use a pipe to connect the two programs
  • Written as "|"
  • Tells the operating system to send what the first program writes to its stdout to the second program's stdin

  • wc -w *.* | sort -n
    

      9 earth.txt
      9 venus.txt
     12 words.len
     30 total
    

  • Figure 5.3: Pipes

  • More convenient (and much less error prone) than temporary files
• Can chain any number of commands together
  • And combine with input and output redirection

  • wc *.txt | sort -n | head -5 > shortest.files
    

• Any program that reads from standard input and writes to standard output can use redirection and pipes
  • Programs that do this are often called filters
  • If you make your programs work like filters, you can easily combine them with others
  • A combinatorial explosion of goodness
• Comment on this slide

Environment Variables

• Like any other program, the shell has variables
• Since they define a user's environment, they are usually called environment variables
  • Usually all upper case
• Type set at the command prompt to get a listing:

  • $ set
    

    ANT_HOME=C:/apache-ant-1.6.2
    BASH=/usr/bin/bash
    COLUMNS=80
    COMPUTERNAME=ISHAD
    HISTFILESIZE=500
    

• Get a particular variable's value by putting a "$" in front of its name
  • E.g., the shell replaces "$HOME" with the current user's home directory
  • Often use the echo command to print this out

    • $ echo $HOME
      

      /home/gvwilson
      

    • Question: why must you type echo $HOME, and not just $HOME?
• To set or reset a variable's value temporarily, use this:

  • $ export VARNAME=value
    

  • Only affects the current shell (and programs run from it)
• To set a variable's value automatically when you log in, set it in ~/.bashrc
  • Remember, "~" is a shortcut meaning “your home directory”
    • For me, right now, ~/.bashrc is /home/gvwilson/.bashrc
• Important environment variables

  • Name	Typical Value	Notes
    HOME	/home/gvwilson	The current user's home directory
    HOMEDRIVE	C:	The current user's home drive (Windows only)
    HOSTNAME	"ishad"	This computer's name
    HOSTTYPE	"i686"	What kind of computer this is
    OS	"Windows_NT"	What operating system is running
    PATH	"/home/gvwilson/bin:/usr/local/bin:/usr/bin:/bin:/Python24/"	Where to look for programs
    PWD	/home/gvwilson/swc/lec	Present working directory (sometimes CWD, for current working directory)
    SHELL	/bin/bash	What shell is being run
    TEMP	/tmp	Where to store temporary files
    USER	"gvwilson"	The current user's ID

    Table 5.1: Important Environment Variables

• Comment on this slide

How the Shell Finds Programs

• The most important of these variables is PATH
  • The search path that tells the shell where to look for programs
  • When you type a command like tabulate, the shell:
    • Splits $PATH on colons to get a list of directories
    • Looks for the program in each directory, in left-to-right order
    • Runs the first one that it finds
• Example
  • PATH is /home/gvwilson/bin:/usr/local/bin:/usr/bin:/bin:/Python24
  • Both /usr/local/bin/tabulate and /home/gvwilson/bin/tabulate exist
  • /home/gvwilson/bin/tabulate will be run when you type tabulate at the command prompt
  • Can run the other one by specifying the path, instead of just the command name
• Warning: it is common to include . in your path
  • This allows you to run a program in the current directory just by typing whatever, instead of ./whatever
  • But it also means you can never be quite sure what program a command will invoke
    • Though you can use the command which program_name, which will tell you
• Common entries in PATH include:
  • /bin, /usr/bin: core tools like ls
    • Note: the word “bin” comes from “binary”, which is geekspeak for “a compiled program”
  • /usr/local/bin: optional (but common) tools, like the gcc
  • $HOME/bin: tools you have built for yourself
    • Remember, $HOME means “the user's home directory”
    • So this is equivalent to ~/bin
• Cygwin does things a little differently
  • Uses the notation /cygdrive/c/somewhere instead of Windows' c:/somewhere
    • The colon in c:/somewhere would clash with the colons in the PATH variable
  • By default, Cygwin treats c:/cygwin as the root of its file system
    • So /home/aturing is a synonym for c:/cygwin/home/aturing
    • Yes, it can be confusing, but remember: we're trying to run one operating system's tools on top of another
• Comment on this slide

Basic Tools

man	Documentation for commands.
cat	Concatenate and display text files.
cd	Change working directory.
chmod	Change file and directory permissions.
clear	Clear the screen.
cp	Copy files and directories.
date	Display the current date and time.
diff	Show differences between two text files.
echo	Print arguments.
env	Show environment variables.
head	Display the first few lines of a file.
ls	List files and directories.
mkdir	Make directories.
more	Page through a text file.
mv	Move (rename) files and directories.
od	Display the bytes in a file.
passwd	Change your password.
pwd	Print current working directory.
rm	Remove files.
rmdir	Remove directories.
sort	Sort lines.
tail	Display the last few lines of a file.
uniq	Remove duplicate lines.
wc	Count lines, words, and characters in a file.
which	locate a command

Table 5.2: Basic Command-Line Tools

• Comment on this slide

Ownership and Permission: Unix

• On Unix, every user belongs to one or more groups
  • The groups command will show you which ones you are in
• Every file is owned by a particular user and a particular group
  • Owner can assign different read, write, and execute permissions to user, group, and others
  • Read: can look at contents, but not modify them
  • Write: can modify contents
  • Execute: can run the file (e.g., it's a program)
• ls -l will show all of this information
  • (Along with the file's size and a few other things)
  • Permissions displayed as three rwx triples
  • “Missing” permissions shown by "-"
  • Example: rw-rw-r-- means “user and group can read and write; everyone else can read; no one can execute”
• Change permissions using chmod
  • Example: chmod u+x something.exe gives the user execute permission to something.exe
  • Example: chmod o-r notes.txt takes away the world's read permission for notes.txt
• Permissions mean something a little different for directories
  • Execute permission means you can “go into” a directory, but does not mean you can read its contents
  • So if a directory called tools has permission rwx--x--x (i.e., owner can do anything, but everyone else only has execute permission), then:
    • If someone other than the owner does ls tools, permission is denied
    • But if there's a useful program called tools/findanswers, other users can still run it
• Comment on this slide

Ownership and Permission: Windows

• Of course, it all works differently on Windows
  • Not better or worse, just differently
• Windows XP uses access control lists (ACLs)
  • Instead of describing users as “file owner, group member, or something else”, ACLs let you specify exactly what any particular user, or set of users, can do to a file, directory, device, etc.
  • Older versions of Windows (such as Windows 95 and Windows 2000) are fundamentally insecure, and shouldn't be used
• Cygwin does its best to make the Windows model look like Unix's
• If you trip over the differences, please consult a system administrator
• Comment on this slide

More Advanced Tools

du	Print the disk space used by files and directories.
find	Find files that match a pattern.
grep	Print lines matching a pattern.
gunzip	Uncompress a file.
gzip	Compress a file.
lpr	Send a file to a printer.
lprm	Remove a print job from a printer's queue.
lpq	Check the status of a printer's queue.
ps	Display running processes.
tar	Archive files.
which	Find the path to a program.
who	See who is logged in.
xargs	Execute a command for each line of input.

Table 5.3: Advanced Command-Line Tools

• Comment on this slide

Exercises

Exercise 5.1:

You're worried your data files can be read by your nemesis, Dr. Evil. How would you check whether or not he can, and if necessary change permissions so only you can read or write the files?