Overview of Storage and Indexing: Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

Overview of Storage and
Indexing
Chapter 8
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke
Data on External Storage
Disks: Can retrieve random page at fixed cost

But reading several consecutive pages is much
cheaper than reading them in random order
record
Tapes: Can only read pages in sequence

Cheaper than disks; used for archival storage
File organization: Method of arranging a file of

records on external storage.
Buffer Manager: stages pages from external

storage to main memory buffer pool. File and
index layers make calls to the buffer manager.
File/Inde
x
Record
Record
Record id (rid) is sufficient to physically locate record

Indexes are data structures that allow us to find the
record ids of records with given values in index
search key fields
key
ID
Buffer
Manage
r
Storage
Manag
er
2
Alternative File Organizations

Many alternatives exist, each ideal for
some situations, and not so good in
others:
Heap (random order) files: Suitable when

typical access is a file scan retrieving all
records.
Sorted Files: Best if records must be

retrieved in some order, or only a `range
of records is needed.
Indexes: Data structures to organize

records via trees or hashing.
Like sorted files, they speed up searches for a

subset of records, based on values in certain
(search key) fields
Updates are much faster than in sorted files.
Indexes Search Key

An index on a file speeds
up selections on the
search key fields for the
index.
Search
Key
A
Any subset of the fields of

a relation can be the
search key for an index on
the relation.
Search key is not the

same as key (minimal set
of fields that uniquely
identify a record in a
relation).
In
de
x
on
AB
2
3
4
5
6
7
Indexes Data Entries

An index contains a collection of
data entries, and supports efficient
retrieval of all data entries k* with
a given key value k.
To locate (one or more) data records
with search key value k
Search the index using k to find the
desired data entry k*
The data entry k* contains information
A
to locate (one or more) data records
1
A
with search key value k
data
2
Searc
h key
entry
*Record
ID
Search Key
A data
record
4
5
6
7
Alternatives for Data Entry k* in

Index
Key
Three alternatives:
1. Actual data record (with search
Indexing
technique
key value k)
2. <k, rid> pair, where rid is the
record id of data record with
search key value k
3. <k, rid-list> pair, where rid-list
is a list of rids of data records
Choice of alternative for data entries is
with search key k
Data
entries
Data Records
orthogonal to the indexing technique used to

locate data entries with a given key value k.
Examples of indexing techniques: B+ trees, hash-based
structures
Typically, index contains auxiliary information that
directs searches to the desired data entries
Alternatives for Data Entries

(Contd.)
Alternative 1:
If this is used, index structure is a

file organization for data records
(instead of a Heap file or sorted
file).
At most one index on a given

collection of data records can use
Alternative 1. (Otherwise, data
records are duplicated, leading to
redundant storage and potential
inconsistency.)
If data records are very large, # of

pages containing data records is
high. Implies size of auxiliary
information in the index is also
large, typically.
Key
DR
DR
DR DR
Auxiliary
informati
on
DR
DR DR DR
DR
Data records
7
B-tree
B-tree can be used to implement Alternative 1
Data
records
(instead of
data
entries)
stored in
tree node
The tree is
relatively large
Alternatives for Data Entries

(Contd.)
Alternatives 2: Data entries <k,
rid>, typically much smaller
than data records. So, better
than Alternative 1 with large
data records, especially if search
keys are small. (Portion of index
structure used to direct search,
which depends on size of data
entries, is much smaller than
Variable
with Alternative 1.)
sized data
entries
Key
Smaller
than
Alternative
1
Data
entries
Heap
file
Data Records
Alternative
3: Data entries <k, list-rid>, more

compact than Alternative 2, but leads to
variable sized data entries even if search keys
Database
Systems
3ed, R. Ramakrishnan and J. Gehrke
areManagement
of fixed
length.
Index Classification
Primary vs. secondary: If search key contains

primary key, then called primary index (alternative
1 is usually used to avoid one more I/O to bring in the
matching data record).
Unique index: Search key contains a candidate key.
Clustered vs. unclustered: If order of data

records is the same as, or `close to, order of data
of retrieving
entries, then called Cost
clustered
index.data records
through index varies greatly based on

whether index is clustered or not!
10
Clustered Index
Suppose that Alternative (2) is used for data entries,
and that the data records are stored in a Heap file.
To build clustered index, first sort the Heap file (with

some free space on each page for future inserts).
Overflow pages may be needed for inserts. (Thus, order
of data records is `close to, but not identical to, the sort
order.)
Index entries
direct search for
data entries
CLUSTERED
Data entries
(Index File)
(Data file)
Data Records
Heap
file
11
Clustered vs Unclustered
Alternative
1 implies clustered; in
practice, clustered also implies
Alternative 1 (since sorted files are
rare).
file can be clustered on at most

one search key (more in next page)
12
Only One Clustered Index
<k, rid>
Clustered
Sorted
according
to SSN
Data file
sorted
according to
SSN
SSN
Ag
e
Incom
e
Phone
56312132
5
32126454
18
57236167
2
40754931
24
67856327
6
32196593
32
69839425
0
40734598
76
72035732
0
40775890
92
73470586
2
32145510
23
80943562
0
40776523
64
A file can have only one clustered index & alternative

90342955
40712454
1
often
used
4
36 J. Gehrke
Database Management Systems
3ed, R. Ramakrishnan and
13
Hash-based Index
Record
IDs
pointing
to data
records
0
2
key
h(key) = ID
of hash
bucket
Overf
ow
page
An entry may
be a record, a
<k, rid>, or a
<k, list_rid>.
N-1
e.g., h(key) = key

Primary
mod N
bucket pages
The mod function
computes the
remainder of the
division key N
14
Hash-Based Indexes
Good for equality selections.
Index is a collection of buckets.
Bucket = primary page plus zero or more overflow
pages.
Hashing function h: h is applied to the search key

fields of r. h(r) = ID of bucket in which record r
belongs.
Hash on the key fields to determine the bucket(s)
Scan the data entries in these buckets to find the
matching <key, rid>
Use rid to locate the record r
If Alternative (1) is used, the buckets contain the data

records (instead of <key, rid> or <key, rid-list> pairs)
15
B+ Tree Indexes: Non-leaf

Page
Non-leaf
Pages
Leaf
Pages
Leaf pages contain data entries, and are chained

(prev & next)
Non-leaf pages contain index entries and direct
16
Example B+ Tree
Root
17
Entries <= 17
5
2*
3*
Entries > 17
27
13
5*
7* 8*
14* 16*
29*?
22* 24*
30
27* 29*
33* 34* 38* 39*
Find 28*?
All > 15* and < 30*
Insert/delete: Find data entry in leaf, then

change it. Need to adjust parent sometimes.
And change sometimes bubbles up the tree
17
Cost Model for Our Analysis

We ignore CPU costs, for
simplicity:
Measuring number of page I/Os

ignores gains of pre-fetching a
sequence of pages; thus, even
I/O cost is only approximated.
Average-case analysis; based on
several simplistic assumptions.
D
R
Number
of data
pages
Average
time to
read/wri
te disk
page
Number of
records per
Good enough to show the overall trends!
page
18
Review
20
Number
of levels
log24
22
N leaf nodes with fanout F logFN

19
Cost Computation
Fano
ut is
F
Heigh
t is
logF
B
B leaf
pages
The I/O cost for finding a particular range of 10

records:
Clustered Index: D(logFB + 1) /* 10 records fit
D is time to
Time to
write a
on read
one or
page
descend the
disk page
tree
20
Cost Computation
1
more
I/O to
read
the
data
recor
d

records:
1 more I/O to
on one page
read the
corresponding
data record
21
Cost Computation

records:
on one page
Cost of retrieving data
Fetch
records
through index
Unclustered
Index:
D(log
B + 10)
/* 10
10Fdata
varies greatly based on
pages
whether
indexscattered
is clustered
records
or not!
Database
Management Systems 3ed, R. Ramakrishnan and J. Gehrke
over different pages
22
Comparing File Organizations

1. Heap files (random order; insert at eof)
2. Sorted files, sorted on <age, sal>
3. Clustered B+ tree file, Alternative (1),
search key <age, sal>
4. Heap file with unclustered B+ tree index
on search key <age, sal>
5. Heap file with unclustered hash index on
search key <age, sal>
23
Operations to Compare
Range selection
Insert a record
Delete a record
Delete
Insert
Equality search
Selection
Scan
Scan: Fetch all records from

disk
Search
24
Assumptions in Our Analysis
Heap Files:
Sorted Files:
Equality selection on key; exactly one match.

Files compacted after deletions.
Indexes:
Alt (2), (3): data entry size = 10% size of
record
Hash: No overflow buckets.
80% page occupancy File size = 1.25 data

size (next page)
Tree: 67% occupancy (this is typical).
Implies file size = 1.5 data size (next page)
25
Assumptions in Our Analysis
Hash: No overflow buckets.

80% page occupancy File size = 1.25
data size
Dat
a
siz
e
400 records
File
siz
e
100
records
Larg
er
File
size
100
records
100
records
100
records
100% occupancy use four pages

Free
spac
e
80
80
80
80
80
records records records records records
80% occupancy
use 25% more pages
A disk page
Tree: 67% occupancy (this is typical).

Implies file size = 1.5 data size
26
Cost of Operations
D
Several assumptions
underlie these (rough)
estimates!
Heap
Scan
Equali
ty
BD
0.5BD
Sorted
Clustere
d
Uncluste
red Tree
Index
Range
D
Delet
e
Time to read
Search +
BD
2D
or write disk
D
Write the page
page
Number of
Heap files (not
records perR
sorted; insert at
page
eof)
Insert
back after the

update
of
B Number
data pages
Uncluste
red Hash
Index
27
Cost of Operations
D
Several assumptions
estimates!
Heap
Sorted
Clustere
d
Uncluste
red Tree
Index
Scan
Equal
ity
Range
Insert
Delet
e
BD
0.5BD
BD
2D
Search +
D
BD
Dlog2B
Sorted files,
D(log2B + #
matching
pages)
sorted
on
sal>
Search + Fetch
Search
&+
rewrite
BD
<age,BD the
latter
half of the
file after
adding
the new
record
Uncluste
28
Cost of
Operations
Several assumptions

estimates!
Heap
Sorted
Clustere
d
Uncluste
red Tree
Index
Scan
Equali
ty
Range
Insert
Delet
e
BD
0.5BD
BD
2D
Search +
D
D(log2B + #
Search +
BD
Search +
BD
BD
Dlog2B
matching
pages)
Clustered B+ tree file,

Alternative (1), search key
<age,sal>
Uncluste
29
Cost of
Operations
Several assumptions

estimates!
67% page
occupancy,
Heap
50%
more
pages to scan
Sorted
Clustere
d
Insert
Delet
e
2D
Search +
D
Search +
BD
Search +
BD
Clustered B+D(log
tree
file,
F(1.5B)
DlogF(1.5
+
+
1.5BD
Alternative
(1), search
keySearch
D
B)
# matching
<age,sal>
pages)
1 write to
insert
the
Search
new +
D
record
Scan
Equali
ty
R
Range
Height
Number
of the
of pages
BD tree 0.5BD as leaf BD
nodes
BD
Dlog2B
D(log2B + #
matching
pages)
Uncluste
Database
red Tree
30
Cost of Operations
D
Several assumptions
estimates!
Heap
Sorted
Clustere
d
Uncluste
red Tree
Index
Scan
Equal
ity
Range
Insert
Delet
e
BD
0.5BD
BD
2D
Search +
D
D(log2B + #
&+
Search + Fetch
Search
Dlog2B
BD
matching
rewrite
BD
BD
pages)
Sorted files, sorted
on <age,
the latter
half of the
sal>
file after
adding
the new
record
Uncluste
31
Cost of
Operations
Several assumptions

estimates!
67% page
occupancy,
Heap
50%
more
pages to scan
Sorted
Clustere
d
Insert
Delet
e
2D
Search +
D
Search +
BD
Search +
BD
Clustered B+D(log
tree
file,
F(1.5B)
DlogF(1.5
+
+
1.5BD
Alternative
(1), search
keySearch
D
B)
# matching
<age,sal>
pages)
1 write to
insert
the
Search
+
new
D
record
Scan
Equali
ty
R
Range
Height
Number
of the
of pages
BD tree 0.5BD as leaf BD
nodes
BD
Dlog2B
D(log2B + #
matching
pages)
Uncluste
Database
red Tree
32
Cost of Operations
Heap File /w Unclustered B+ tree
SCAN (to obtain data records in sorting order)
Scan the leaf level of the index 1

For each data entry in a leaf node, fetch the
corresponding data record from the heap file 2
SCAN COST:
0.1(1.5B)D + BRD = BD(R+0.15)

R
Cost of scanning the leaf nodes
Each page is 67% occupied # data pages is
B
Data
size
1.5B
Data entry is only 10% the size of data record
# leaf pages is 0.1(1.5B)
Cost of scanning the leaf pages is 0.1(1.5B)D
Cost of fetching the data records

Number of data record is BR Cost of retrieving
all data records is BRD
0.1(1.5) B
pages
B pages
33
Cost of Operations
Heap File /w Unclustered
B+ tree
EQUALITY SEACH
Search for the matching data entry in the index
Fetch the corresponding data record from the data
file
SEARCH COST:
Cost of searching the index
# leaf pages is 0.1(1.5B) tree height is logF(0.15B)
Descending the index tree visits logF(0.15B) pages
Cost of finding the matching data entry is DlogF(0.15B)

Fetching the corresponding data records incurs one
more I/O, or 1D
Total search cost: DlogF(0.15B) + 1D = D(1+ logF(0.1
34
Cost of Operations
Equality
slide)
Search (from last

D(1+ logF(0.15B))
Fetching
the
matching
record
Search
the B+
tree
Range
Selection
D(# matches + logF(0.15B))
Fetching each
match in the
range incurs one
I/O
Search
the B+
tree
35
Cost of Operations
INSERT
Insert the new record in the heap file
Insert the corresponding data entry in the B+ tree
INSERT COST:
Cost of inserting the new record

Inserting the new record incurs two I/Os:
2D
Cost of inserting the data entry in the B+ tree

# leaf pages is 0.1(1.5B) tree height is logF(0.15B)
Descending the index tree visits logF(0.15B) pages
Cost of finding the target leaf page is DlogF(0.15B)
Updating target leaf page incurs one more I/O: 1D + DlogF(0.15B)
Total insert cost:
D+DlogF(0.15B) + 2D = D(3 +
Database Management Systems 3ed,
Ramakrishnan and J. Gehrke
logR. (0.15B))
36
Cost of Operations
Insert
(from last
slide)
Delete
D(3 + logF(0.15B))
1 I/O to
insert the
data entry
+ 2 I/Os to
insert the
new record
Search
the B+
tree
D(3 + logF(0.15B)) = 2D + S
1 I/O to
delete the
data entry
+ 2 I/Os to
delete the
data record
Search
the B+
tree
1 I/O to write
back the dataentry page
and another I/O
to write back
the data-record
page
37
Cost of Operations
D
Several assumptions
estimates!
Equal
Delet
Range
Insert
ity
e
+
Heap
file
with
unclustered
B
tree
Search +
BD
0.5BD
BD
2D
D
index on search key <age,sal>
1 I/O to insert the
Scan
Heap
Sorted
Clustered
BD
1.5BD
Dlog2B
DlogF1.5
B
D(log2B + #
matches)
data entry
2 I/Os +
Search
+ +
Search
toBD
insert the data
BD
D(logF1.5B +
# matching
pages)
Search +
D
record
Search +
D
1 I/Os to write
Cost of
Unclustered
D(1
+
D(logEach
Uncluster
matchD(3 +back the datascanning
F0.15B
+1
entry Search
page and
one I/O BD(R+0.1
per
requires
an
logF0.15
logF0.15B
data
entries
ed Tree
+
#
I/O to write
record 5)
2Dback
I/O
is 0.1(1.5B)D
Index
B)
)
matches)
the data-record38
page
Cost of Operations (1)

Heap File /w Unclustered Hash Index
SCAN (to obtain data records in hash order)
Fetch the hash buckets1
For each data entry in a hash bucket, fetch2the
corresponding data record from the heap file
1
2
39

SCAN (to obtain data records in hash order)
Fetch the hash buckets
For each data entry in a hash bucket, fetch the
corresponding data record from the heap file
SCAN COST:
0.125BD + BRD = BD(R+0.125)
Cost of scanning the hash buckets
Each page is 80% occupied # data pages is 1.25B
Data entry is only 10% the size of data record #
index pages (i.e., # hash buckets) is 0.1(1.25B) =
0.125B
Cost of scanning the data entry is 0.125BD

Since number of data record is BR, cost of retrieving all
data records is BRD (i.e., 1 I/O per record)
40

Range Selection (Hash structure cannot help)
Scan the hash buckets
For each hash bucket, fetch the data record from
the heap file if the corresponding data entry is
within the range
41

Range Selection (Hash structure cannot help)
Scan the hash buckets
For each hash bucket, fetch the data record from the
heap file if the corresponding data entry is within the
range
SCAN COST:
0.125BD + (# matches) D = D(0.125B
Cost of scanning the hash buckets

Each page is 80% occupied # data pages is 1.25B
Data entry is only 10% the size of data record # index
pages (i.e., # hash buckets) is 0.1(1.25B) = 0.125B
Cost of scanning the data entry is 0.125BD
Cost of fetching the data records:

(# matches)D
42
Cost of Operations
D
Several assumptions
estimates!
Scan
Heap
Heap
file
Sorted
Clustere
d
Equal
ity
R
Range
Insert
2D to
update
BD unclustered
0.5BD
BD
2D
with
hash
the
index file
index
D(log2B + # +
Search
2D to +
Dlog2B
BD
BD
matches)
update
Cost of
Hash
the data
scanning
structure
Dlog
1.5B
+
F
file
data entries
cannot
DlogF1.5
Search +
1.5BD is
help
# matching
D
B
1.25(0.1B)D
pages
B
Delet
e
Search
2D to +
update
D
the
Search
+
index file
+ BD
2D to
update
the data
Search
+
file
D
D(1 +
D(3 +
D(logF0.15B
Search +
logF0.15
logF0.15B
+#
2D
B)
)
matches)
Uncluste
red Tree
Index
BD(R+0.1
5)
43
Cost of Operations
D
Several assumptions
estimates!
Heap
Sorted
Clustere
d
Scan
Equal
ity
Range
Insert
Delet
e
BD
0.5BD
BD
2D
Search +
D
Search +
BD
Search +
BD
Search +
D
Search +
D
BD
Dlog2B
1.5BD
DlogF1.5
B
D(log2B + #
matching
pages)
DlogF1.5B +
# matching
pages
D(1 +
D(3 +
D(logF0.15B
Uncluste
BD(R+0.1
Search +
logF0.15
log
0.15B
red Tree
+#
F
5)
2D
Database
Management
Systems
3ed,
R.
Ramakrishnan
and
J.
Gehrke
Index
B)
)
matches)
44
Understanding the Workload
For each query in the workload:
Which relations does it access?
Which attributes are retrieved?
Which attributes are involved in selection/join

conditions? How selective are these conditions
likely to be?
For each update in the workload:
The type of update (INSERT/DELETE/UPDATE), and

the attributes that are affected.
45
Choice of Indexes
What indexes should we create?
Which relations should have indexes? What

field(s) should be the search key? Should we
build several indexes?
For each index, what kind of an index should

it be?
Clustered? Hash/tree?
46
Choice of Indexes (Contd.)
One approach: Consider the most important

queries in turn. Consider the best plan using the
current indexes, and see if a better plan is
possible with an additional index. If so, create it.
Obviously, this implies that we must understand how
a DBMS evaluates queries and creates query
evaluation plans!
For now, we discuss simple 1-table queries.
Before creating an index, must also consider the

impact on updates in the workload!
Trade-off: Indexes can make queries go faster,

updates slower. Require disk space, too.
47
Index Selection Guidelines

(1)
Attributes in WHERE clause are
candidates for index keys.
Exact match condition suggests hash
index.
SELECT
E.dno
FROM
Employees E
WHERE E.num = 568429543
Range query suggests tree index.

Clustering is especially useful for range
queries; can also help on equality queries ifDept.
123 has
there are many duplicates.
Employee
s older
than 40
SELECT
E.dno
FROM
Employees E
WHERE E.age > 40
SELECTE.name
FROM Employees E
WHEREE.dno=123
many
employ
ees
48

(2)
Multi-attribute search keys should be

considered when a WHERE clause contains
several conditions
SELECT
FROM
WHERE
E.name
Employees E
E.age1 = 20 AND E.sal
= 50000
2
An index on <age,sal> would be better than

an index on age or an index on sal
Reason: The qualifying data entries are
grouped together
49

(3)
Indexes can sometimes enable index-only

evaluation for important queries.
Example: use leaf pages of index on age
to compute the average age
Try to choose indexes that benefit as

many queries as possible.
Since only one index can be clustered per

relation, choose it based on important
queries that would benefit the most from
clustering.
50
Examples of Clustered Indexes

(1)
B+ tree index on
E.age can be used
to get qualifying
tuples
What is the selectivity of
SELECT
FROM
WHERE
E.dno
Emp E
E.age>30
the condition ?
If most employees are older than 30, a

sequential scan of the relation would do
asonly
well 10% are older
almost
What if
than 30 ?
Unclustered index: Performing

one I/O per qualifying tuple could
be more expensive than a
sequential scan
On
age
51
Examples of Clustered
Indexes (2)
Using E.age index
may be costly
Retrieve
tuples with
E.age > 25
Sort the tuples on
dno, and count
number of tuples for
each dno
Good to
know
DBMS
internal
On
age
SELECT E.dno, COUNT (*

FROM Emp E
WHERE E.age>25
GROUP BY E.dno
Clustered E.dno index
may be better !
For
each dno, count

the tuples with E.age >
25
No need to sort !
52
Examples of Clustered Indexes

(3)
SELECT
FROM
WHERE
If many employees collect

E.dno
stamps, a clustered index
Emp E
on E.hobby is helpful
E.hobby=Stamps
An unclustered index on
E.eid is good enough for
the second query since
no two employees have
the same E.eid.
If this query is important,

we should consider this
index
SELECT
FROM
WHERE
E.dno
Emp E
E.eid=32816945
53
Indexes with Composite Search

Keys
Composite Search
Keys: Search on a
combination of fields.
Equality query: Every

field value is equal to a
constant value, e.g. wrt
<sal,age> index:
age=20 and sal =75
Range query: Some field

value is not a constant,
e.g., age =20; or age=20
and sal > 10
Data entries in index

sorted by search key to
support range queries.
Examples of composite key

Dataindexes
entrie
s
11,80
11
12,10
12
12,20
13,75
<age, sal>
10,12
20,12
75,13
name age sal

bob
12 10
cal
11
joe
12 20
sue
13 75
12
13
<age>
80
10
Data records
sorted by name
80,11
<sal, age>
Data entries in index

sorted by <sal,age>
20
75
80
<sal>
Data entries
sorted by <sal>
54
Composite Search Keys
Order of attributes is important for range queries
key<age,sal> key<sal,age>
To retrieve Emp records:

If condition is: age=30 AND 3000<sal<5000:
Clustered
<age,sal> index much better than

<sal,age> index!
Reason: The qualifying data entries are grouped
together in <age,sal> index, but not in <sal,age>
index
If condition is: 20<age<30 AND 3000<sal<5000:
Clustered
tree index on <age,sal> or <sal,age> is
best.
Composite indexes are larger, updated more often
55
Database Example
We use this database for the following query examples
Dep
tdno
budge
mgr
t
Em
pei
dn ag pho
sal
d
o e ne
Foreign key
56
Index-Only Plans (1)

A number of queries
can be answered
without retrieving any
tuples from one or
more of the relations
involved if a suitable
index is available.
No need to
perform
join
operation
Index
on
dno
SELECT
SELECT
FROM
FROM
WHERE
WHERE
Find mangers of
D.mgr
departments with
Dept D, Emp
E one
at least
employee
D.dno=E.dno
If <E.dno> index is available,

no need to retrieve
Employees tuples, i.e., scan
the E.dno data entries and
pick up the corresponding
Em
Dept tuples
pei
dn ag pho
sal
d
o e ne
57

Em
pei
dn ag pho
sal
d
o e ne
SELECT
SELECT
FROM
FROM
GROUP
GROUP BY
BY
Index
on dno
E.dno,
E.dno, COUNT(*)
COUNT(*)
Emp
Emp E
E
E.dno
E.dno
If <E.dno> index is
available, need only
scan the data entries
and count employees
for each dno
58

SELECT
E.dno, MIN(E.sal)
FROM
Emp E
GROUP BY E.dno
Index
on dno
& sal
Emp
If <E.dno,E.sal> tree
index is available, need
only scan the data
entries and compute
MIN(E.sal) for each dno
ei
dn ag pho
sal
d
o e ne
59

Emp
ei
dn ag pho
sal
d
o e ne
If <E.age,E.sal> or
<E.sal,E.age> tree index
is available, the average
salary can be computed
using only data entries in
the index
Compute
average
SELECT
SELECT AVG(E.sal)
AVG(E.sal)
salary of
FROM
FROM Emp
Emp E
E
young
WHERE
E.age=25
AND
WHERE E.age=25
AND
executives
E.sal
E.sal BETWEEN
BETWEEN 300000
300000 AND
AND 50000
5000
60

Index-only plans are possible
if the key is <dno,age> or
we have a tree index with
key <age,dno>
SELECT
E.dno,
SELECT
E.dno,
COUNT (*)
(*)
COUNT
FROM
Emp E
E
FROM
Emp
Using <dno,age> index
WHERE
E.age=30
WHERE
E.age=30
GROUP BY
BY E.dno
E.dno
GROUP
Scan all data entries
Do not
not
Do
For each dno, count
scan all
all
scan
ages
ages
number of tuples with
Using
<age,dno> index
age=30
Better !!
Better
Use index find first data entry /w age = 30
Scan data entries with age = 30, and count
number of tuples for each dno (the
departments are arranged continuously
for age=30)
61

Index only plans are possible
Using the <age,dno>
index
For data entries with age >
30, sort them on dno
Scan sorted data entries and
count number of data entries
for each dno
Using <dno,age> index

Scan data entries
For each dno, count number
of data entries with age > 30
SELECT
SELECT
COUNT (*)
(*)
COUNT
FROM
FROM
WHERE
WHERE
GROUP BY
BY
GROUP
E.dno,
E.dno,
Emp E
E
Emp
E.age>30
E.age>30
E.dno
E.dno
Note: Sorting is not

necessary if the
department counters
can fit in memory
No
sorting.
Better !
62
Summary
Many alternative file organizations exist, each

appropriate in some situation.
If selection queries are frequent, sorting the file

or building an index is important.
Hash-based indexes only good for equality search.
Sorted files and tree-based indexes best for range

search; also good for equality search. (Files rarely
kept sorted in practice; B+ tree index is better.)
Index is a collection of data entries (<key, rid>

pairs or <key, rid-list> pairs) plus a way to
quickly find entries (using hashing or a search
tree) with given key values.
63
Summary (Contd.)
Data entries can be actual data records,

<key, rid> pairs, or <key, rid-list> pairs.
Choice orthogonal to indexing technique used

to locate data entries with a given key value.
Can have several indexes on a given file of

data records, each with a different search
key.
Indexes can be classified as clustered vs.

unclustered, and primary vs. secondary.
Differences have important consequences
for utility/performance.
64
Summary (Contd.)
Understanding
the nature of the workload for

the application, and the performance goals, is
essential to developing a good design.
What are the important queries and updates? What

attributes/relations are involved?
Indexes
must be chosen to speed up important

queries (and perhaps some updates!).
Index maintenance overhead on updates to key fields.

Choose indexes that can help many queries, if possible.
Build indexes to support index-only strategies.
Clustering is an important decision; only one index on a
given relation can be clustered!
Order of fields in composite index key can be important.
65

Overview of Storage and Indexing: Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Overview of Storage and Indexing: Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1

Uploaded by

Copyright:

Available Formats

Overview of Storage and

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Data on External Storage

Disks: Can retrieve random page at fixed cost

Tapes: Can only read pages in sequence

File organization: Method of arranging a file of

Buffer Manager: stages pages from external

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Record id (rid) is sufficient to physically locate record

Alternative File Organizations

Heap (random order) files: Suitable when

Sorted Files: Best if records must be

Indexes: Data structures to organize

Like sorted files, they speed up searches for a

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Indexes Search Key

Any subset of the fields of

Search key is not the

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Indexes Data Entries

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Alternatives for Data Entry k* in

orthogonal to the indexing technique used to

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Alternatives for Data Entries

If this is used, index structure is a

At most one index on a given

If data records are very large, # of

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Alternatives for Data Entries

3: Data entries <k, list-rid>, more

Primary vs. secondary: If search key contains

Unique index: Search key contains a candidate key.

Clustered vs. unclustered: If order of data

through index varies greatly based on

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

To build clustered index, first sort the Heap file (with

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

file can be clustered on at most

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Only One Clustered Index

A file can have only one clustered index & alternative

e.g., h(key) = key

Hashing function h: h is applied to the search key

If Alternative (1) is used, the buckets contain the data

B+ Tree Indexes: Non-leaf

Leaf pages contain data entries, and are chained

33* 34* 38* 39*

All > 15* and < 30*

Insert/delete: Find data entry in leaf, then

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Cost Model for Our Analysis

Measuring number of page I/Os

N leaf nodes with fanout F logFN

The I/O cost for finding a particular range of 10

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

The I/O cost for finding a particular range of 10

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

The I/O cost for finding a particular range of 10

Comparing File Organizations

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke

Scan: Fetch all records from

Assumptions in Our Analysis