Operating Systems

System Level Programming

Week 5
Rob Pooley

rjp

System Level Progra

What is the OS?

The operating system provides a layer
between the programmer and the underlying
system architecture, giving higher level
instructions ("system calls"), and insulating
the programmer from having to think about
low level issues.
The OS enables processor resources and
memory to be shared among many programs
running at the same time.
rjp

System Level Progra

 We'll focus on three things:

Memory: How is the programmer insulated from
having to think about real physical memory
limitations?
I/O: How does the OS support the programmer in
managing I/O to various peripheral devices?
Processes: How can several programs
(processes) run at the same time (without the
programmer having to worry about how).

rjp

System Level Progra

Virtual Memory
Computer memory consists of
primary memory - RAM (random access memory)
various forms of secondary (disk) storage.

RAM is so called because you can access the

contents of a memory location directly if you know the
location.
So, ideally we want our program and data in RAM.
But what happens if we want to run a lot of large
programs, and there is not enough RAM?
rjp

System Level Progra

 The solution (now almost universally adopted) is to use

"virtual memory".
Virtual memory separates the concepts of address
space and actual memory locations.
We can see how this works by considering computers
with a very small amount of memory lets say 4096
bytes.
If the processor used 2 byte integers to hold addresses
it could in principle refer to 65536 (2 ** 16) locations.
But only 4096 of these could actually be locations in
physical RAM.
If a program or programs requires more than these
4096 bytes to hold its code and data segments, some
of this must be copied to disk.

rjp

System Level Progra

 With virtual memory the OS is responsible for

managing a mapping between
the address space used in the programs (i.e., the addresses
used in the stack etc) and
the actual physical RAM (and disk).

The computer can look at the RAM for areas that

have not been used recently and copy them onto
hard disk.
This frees up space for new programs to run.
As this mapping and copying is done automatically
within the operating system the programmer (high
level or assembly) can use as much of the full
address space as will fit onto the disc and not be
limited to the physical RAM.

rjp

System Level Progra

2 pages mapped into memory

Memory has a
free page
Data needed from
backing store

rjp

System Level Progra

Third page swapped into memory

A page fault

Physical memory full

rjp

System Level Progra

Another page needed from disc

A page fault

Physical memory full

More data needed
from backing store
rjp

System Level Progra

One page written back onto disc

Physical memory has

a free page

rjp

System Level Progra

10

New page fetched into memory

Physical memory
full again

rjp

Need to write back

before fetching
System Level Progra
11

Input and Output

Input and output refers not just to the stuff that
you read in and write out when you run your
program, but also to interaction with peripheral
devices (printers etc).
Fortunately we can use the same model for
both: writing something out on the user's
screen, writing to a file, and writing to a printer
all are based on the same principles.
The operating system handles the low level
details of how devices are actually written to.
rjp

System Level Progra

12

Files and I/O Devices

Input and output is based on the notion of a "file".
But a file doesnt just refer to something you store on
your hard disk, but to any sequence of bytes that can be
written to an i/o device.
So we can write a file to a printer (or any other output device,
such as an audio device).
We can read from our floppy drive, hard drive, CD player etc.

The programmer does not have to be concerned with

how to "activate" these devices and locate files.
The physical organisation of files on (say) the hard disk
is quite separate from the logical file structure that the
programmer or user uses to refer to them.

rjp

System Level Progra

13

I/O system calls

The operating system provides system calls for
opening, reading, writing and closing files.
When you open a file in "C" (or Java) you will be
invoking the appropriate operating system call.
Opening a file involves locating it, and bringing into
memory information necessary to access it.
Reading it involves accessing the file and the first
part of the data it contains.
As we will usually not be reading all the data in the
buffer at once, a pointer is updated indicating which
byte should be read next.
rjp

System Level Progra

14

Buffering
In practice I/O is usually buffered.
We don't read/write directly to the physical device,
but to a temporary storage area (the buffer).
The OS handles the interface between this buffer and
the actual physical device, transforming the data as
required and reading/writing fixed sized chunks for
efficiency.
This is referred to as stream I/O.
The user/programmer can usually ignore this, except
abnormal cases where the buffer written to fails to be
flushed/written out.
rjp

System Level Progra

15

Buffering

rjp

System Level Progra

16

Directories
Although the hard bit of file handling is left to the
operating system, the user/programmer needs a way
to refer to files (names)
to keep them organised (folders or directories).

Directories provide
a logical structure for users to keep their files organised,
a structure suitable for adding security instructions to prevent
unauthorised use
you can change permissions on a directory so that only you
(or your "group") can read or write to files within it.

The operating system provides system calls for

managing this structure (e.g., creating a directory;
moving a file into it) and altering permissions.

rjp

System Level Progra

17

I/O devices
The directory structure also provides a means
for specifying input/output devices.
On Unix the "/dev" directory contains files (try
ls /dev) but the files there correspond to
devices, not to files on the hard disk.
If you write to these files the necessary device
driver will be invoked by the operating system this is a bit of software that knows how to start
up, read and write to this particular device.
rjp

System Level Progra

18

 You can get more information about devices simply

using ls -l (ie, a detailed file listing):
pele% ls -l /dev/tty
crw-rw-rw- 1 root root 5, 0 Aug 22 09:34 /dev/tty
pele% ls -l /dev/fd0
brw-rw-rw- 1 ceebde1 floppy 2, 0 May 5 1998 /dev/fd0
pele% ls -l /dev/lp0
crw-rw---- 1 root daemon 6, 0 May 5 1998 /dev/lp0
pele% ls -l /dev/audio
crw-rw-rw- 1 ceebde1 bin 14, 4 May 5 1998 /dev/audio

In the first example, the very first letter (c) indicates

that it is a character type device.
The numbers (5, 2, 6, 14) are part of what is used to
indicate the device driver to use.
rjp

System Level Progra

19

 You can write to these devices just as you would

write to files.
But you have to know what sort of data it will make
sense to write.
Printers may expect postscript files; audio devices will
expect particular audio data formats.
Normally your floppy (or CD writer) is used to hold
files of particular types, not just streams of
characters.
We will therefore generally restrict examples to
writing to the terminal (the device /dev/tty) or to
the hard disk.
rjp

System Level Progra

20

Unix I/O
Normally Unix programs read input from what is
called the "standard input stream" and write to the
"standard output stream".
By default, standard input will be the keyboard, and
standard output the terminal (screen). that is, the
place to read and write stuff if no other file is
specified.
By default the standard input and output is your
terminal - or the device /dev/tty.
But it is possible (and easy) to redirect standard input
and output, so that any device or file is used for i/o.
rjp

System Level Progra

21

Redirecting I/O
To redirect standard output the ">" symbol is used.
If we use the command "ls" it outputs to the standard
output, the terminal. But we can redirect this to a file:
%pele ls > myfile
Try this, and look at the contents of the file. If you
then want to add more on to the end of the file we can
use ">>":
%pele date >> myfile
This will stick todays date at the end of your file, after
your directory listing

rjp

System Level Progra

22

 To append redirected output, rather than

overwriting the file use >>
To redirect standard input use the "<" symbol.
If a program usually takes input from the
terminal, you can get it to take the input from
a file.
You could, for example, mail the contents of a
file to someone using:
mail alison < myfile
rjp

System Level Progra

23

Redirecting other devices

In general either < or > can be preceded by the
number of a file descriptor, forcing redirection of
that device.
cat fred >jim 2>alice
Redirects stdout to jim and stderr to alice
stdin is number 0 and stdout is number 1, but
these can usually be omitted
rjp

System Level Progra

24

Pipes
You can also arrange for the input for one program to
come from another program. This allows us to string
together a sequence of simple commands.
Pipes ("|") are used for this. They take the output stream
of one program and connect it to the input stream of
another.
Suppose we want our friends to know what's in our
directory. We could use:
ls | mail fred
The output of "ls" becomes the input of "mail" and a mail
message is sent containing the directory listing. Try using
this approach to mail yourself todays date.

rjp

System Level Progra

25