UNIX is an interactive system designed
to handle multiple processes and multiple users at the same time. It was
designed by programmers, for programmers, to use in an environment in which the
majority of the users are relatively sophisticated and are engaged in (often
quite complex) software development projects. In many cases, a large number of
programmers are actively cooperating to produce a single system, so UNIX has
extensive facilities to allow people to work together and share information in
controlled ways. The model of a group of experienced programmers working
together closely to produce advanced software is obviously very different from
the personal computer model of a single beginner working alone with a word
processor, and this difference is reflected throughout UNIX from start to
finish.
What
is it that good programmers want in a system? To start with, most like their
systems to be simple, elegant, and consistent. For example, at the lowest
level, a file should just be a collection of bytes. Having different classes of
files for sequential access, random access, keyed access, remote access, etc.
(as mainframes do) just gets in the way. Similarly, if the command
ls
A*
means list all the
files beginning with “A” then the command
rm
A*
should mean remove all the
files beginning with “A” and not remove the one file whose name consists of an
“A” and an asterisk. This characteristic is sometimes called the principle
of least surprise.
A UNIX system can be regarded as a
kind of pyramid, as illustrated in Fig. 10-1. At the bottom is the hardware,
consisting of the CPU, memory, disks, terminals, and other devices. Running on
the bare hardware is the UNIX operating system. Its function is to control the
hardware and provide a system call interface to all the programs. These system
calls allow user programs to create and manage processes, files, and other
resources.
Programs make system calls by putting
the arguments in registers (or sometimes, on the stack), and issuing trap
instructions to switch from user mode to kernel mode to start up UNIX, Since
there is no way to write a trap instruction in C, a library is provided, with
one procedure per system call. These procedures are written in assembly
language, but can be called from C. Each one first puts its arguments in the
proper place, then executes the trap instruction. Thus to execute the read system call, a C program can call the read library
procedure. As an aside, it is the library interface, and not the system call
interface, that is specified by POSIX. In other words, POSIX tells which
library procedures a conformant system must supply, what their parameters are,
what they must do, and what results they must return. It does not even mention
the actual system calls.
In addition to the operating system
and system call library, all versions of UNIX supply a large number of standard
programs, some of which are specified by the POSIX 1003.2 standard, and some of
which differ between UNIX versions. These include the command processor
(shell), compilers, editors, text processing programs, and file manipulation
utilities. It is these programs that a user at a terminal invokes.
Thus we can speak of three different
interfaces to UNIX: the true system call interface, the library interface, and
the interface formed by the set of standard utility programs. While the latter
is what the casual user thinks of as “UNIX,” in fact, it has almost nothing to
do with the operating system itself and can easily be replaced.
Some versions of UNIX, for example,
have replaced this keyboard-oriented user interface with a mouse-oriented
graphical user interface, without changing the operating system itself at all.
It is precisely this flexibility that makes UNIX so popular and has allowed it
to survive numerous changes in the underlying technology so well.
The UNIX Shell
Many UNIX systems
have a graphical user interface of the kind made popular by the Macintosh and
later Windows. However, real programmers still prefer a command line interface,
called the shell. It is much faster to use, more powerful, easily
extensible, and does not give the user RSI from having to use a mouse all the
time. Below we will briefly describe the Bourne shell (sh). Since then,
many new shells have been written (ksh, bash, etc.). Although
UNIX fully supports a graphical environment (X Windows), even in this world
many programmers simply make multiple console windows and act as if they have
half a dozen ASCII terminals each running the shell.
When the shell starts
up, it initializes itself, then types a prompt character, often a percent
or dollar sign, on the screen and waits for the user to type a command line.
When the user types a
command line, the shell extracts the first word from it, assumes it is the name
of a program to be run, searches for this program, and if it finds it runs the
program. The shell then suspends itself until the program terminates, at which
time it tries to read the next command. What is important here is simply the
observation that the shell is an ordinary user program. All it needs is the
ability to read from and write to the terminal, and the power to execute other
programs
It is possible to put
a list of shell commands in a file and then start a shell with this file as
standard input. The (second) shell just processes them in order, the same as it
would with commands typed on the keyboard. Files containing shell commands are
called shell scripts. Shell scripts may assign values to shell variables
and then read them later. They may also have parameters, and use if, for, while,
and case constructs. Thus a shell script is really a
program written in shell language. The Berkeley C shell is an alternative shell
that has been designed to make shell scripts (and the command language in
general) look like C programs in many respects. Since the shell is just another
user program, various other people have written and distributed a variety of
other shells.
In Fig. 10-1 we saw the overall
structure of a UNIX system. Now let us zoom in and look more closely at the
kernel before examining the various parts. Showing the kernel structure is
slightly tricky since there are many different versions of UNIX, but although
the diagram of Fig. 10-3 describes 4.4BSD, it also applies to many other
versions with perhaps small changes here and there.
The bottom layer of the kernel
consists of the device drivers plus process dispatching. All UNIX drivers are
classified as either character device drivers or block device drivers, with the
main difference that seeks are allowed on block devices and not on character
devices. Technically, network devices are character devices, but they are
handled so differently that it is probably clearer to separate them, as has
been done in the figure. Process dispatching occurs when an interrupt happens.
The low-level code here stops the running process, saves its state in the
kernel process table, and starts the appropriate driver. Process dispatching
also happens when the kernel is finished and it is time to start up a user
process again. Dispatching code is in assembler and is quite distinct from
scheduling.
Above the bottom level, the code is
different in each of the four “columns” of Fig. 10-3. At the left, we have the
character devices. There are two ways they are used. Some programs, such as
visual editors like vi and emacs, want every key stroke as it is
hit. Raw terminal (tty) I/O makes this possible. Other software, such as the
shell (sh), is line oriented and allows users to edit the current line
before hitting ENTER to send it to the program. This software uses cooked mode
and line disciplines.
Networking software is often modular,
with different devices and protocols supported. The layer above the network
drivers handles a kind of routing function, making sure the right packet goes
to the right device or protocol handler. Most UNIX systems contain the full
functionality of an Internet router within the kernel, although the performance
is less than that of a hardware router, but this code predated modern hardware
routers. Above the router code is the actual protocol stack, always including
IP and TCP, but sometimes additional protocols as well. Overlaying all the
network is the socket interface, which allows programs to create sockets for
particular networks and protocols, getting back a file descriptor for each
socket to use later.
On top of the disk drivers are the
file system’s buffer cache and the page cache. In early UNIX systems, the
buffer cache was a fixed chunk of memory, with the rest of memory for user
pages. In many modern UNIX systems, there is no longer a fixed boundary, and
any page of memory can be grabbed for either function, depending on what is
needed more.
On top of the buffer cache come the
file systems. Most UNIX systems support multiple file systems, including the Berkeley fast file
system, log-structured file system, and various System V file systems. All of
these file systems share the same buffer cache. On top of the file systems come
file naming, directory management, hard link and symbolic link management, and
other file system properties that are the same for all file systems.
On top of the page cache is the
virtual memory system. All the paging logic is here, such as the page
replacement algorithm. On top of it is the code for mapping files onto virtual
memory and the high-level page fault management code. This is the code that
figures out what to do when a page fault occurs. It first checks if the memory
reference is valid, and if so, where the needed page is located and how it can
be obtained.
The last column deals with process
management. Above the dispatcher is the process scheduler, which chooses which
process to run next. If threads are managed in the kernel, thread management is
also here, although threads are managed in user space on some UNIX systems.
Above the scheduler comes the code for processing signals and sending them to
the correct destination, as well as the process creation and termination code.
The top layer is the interface into
the system. On the left is the system call interface. All system calls come
here and are directed to one of the lower modules, depending on the nature of
the call. On right part of the top layer is the entrance for traps and
interrupts, including signals, page faults, processor exceptions of all kinds,
and I/O interrupts.
