CS3230: Design and Analysis of Algorithms

Fall 2014

CS3230 : Design and Analysis of Algorithms (Fall 2014)

Tutorial Set #5
[SOLUTION SKETCHES]
Helpful Hints Series: (On Amortized Analysis)
Read and understand this. Reflect on this after you finished each question in this tutorial.
When doing amortized analysis for a data structure that supports multiple operations (like
INSERT, DELETE, DELETEMIN, etc), you must give an amortized analysis for each and
every operation supported by the data structure.
In doing so, we will inevitably also analyze the worst-case time of each operation, in
addition to the amortized time of each operation. It is quite OK that for some operations,
the worst-case time and the amortized time are the same.
Of course, for some operations, the amortized time is much better than the worst-case time
because we are paying for the extra work using credits saved (accounting method) or
saved potential (potential method).
And this is precisely the reason we do amortized complexity analysis.
Learn to reflect on each operation and see where do we save up credits, and for what?
Often, this will help you develop the appropriate potential function.

R1. Exercises 17.1-1 of [CLRS] (Stack with MultiPUSH and MultiPOP)

If a MULTIPUSH operation were included in the set of stack operations, would the O(1)
bound on the amortized cost of stack operations continues to hold?
[First read [CLRS]-17.1on Stack operations, especially part on MULTIPOP operation.]
HINT: Consider a sequence of n/2 {MULTIPUSH(k, Items); MULTIPOP(k)} operations.
What is the actual cost this sequence of n operations (in the worst-case)?
Then actual time for this sequence of n operations will be (nk)
R2. Exercises 17.2-1 of [CLRS] (Bounded Stack with Backup)
A sequence of stack operations is performed on a stack whose size never exceeds k. After
every k operations, a copy of the entire stack is made for backup purposes. Show that the
cost of n operations, including copying the stack, is O(n) by assigning suitable amortized
costs to the various stack operations.
HINT: How to save up credit to pay for the expensive backup operations?
Idea: Each PUSH, POP, MULTIPOP operation must save up credit for the regular backup
operations. So, k operations will save up k credits, enough to pay for the backup operation.

Printed 9/30/14 10:34 AM

Page 1

CS3230: Design and Analysis of Algorithms

Fall 2014

S1. Modified from 17.2-3 of [CLRS] (Binary Counter with Reset)

In the example of incrementing a binary counter given in the lectures, suppose we wish not
only to increment a counter but also to reset the counter to zero (i.e., set all bits in it to 0).
We want to implement a counter as a bit vector so that any sequence of n
INCREMENT/RESET operations takes time O(n) on an initially zero counter.
(Hint: Keep a pointer to the most-significant, highest-order 1-bit.)
(a)
(b)

Give the algorithms for INCREMENT and RESET operations.

Using the accounting method, show that any sequence of n INCREMENT / RESET
operations take time O(n) on an initially zero counter.

SOLUTION SKETCH: Idea: This is similar to T5-R2. Define HP to be the highest-order

1-bit in the binary counter. When we reset, we do not go beyond HP. Our INCREMENT and
RESET operations need to update HP. (Note initially, HP = 1)
(a)

DIY. INCREMENT updates HP. RESET changes bits 0,1,, HP to 0 and update HP to -1.

(b)

We charge US$2 whenever a bit is set (changes from 0 to 1). In addition, we also charge
SG$2 per INCREMENT operation -- SG$1 to check and update HP where necessary, and
SG$1 to stick to that bit to pay for a subsequent RESET operation. So, every bit 0,1,2,HP
has a SG$1 to pay for a future RESET operation.
Amortized Analysis:
Cost of INCREMENT operation:
Changing bits from 1 to 0 (line 3) is free. Changing bit from 0 to 1 cost US$2.
Cost of checking and updating HP is SG$1 (S$1 is saved up).
Amortized cost per INCREMENT operation: US$2 + SG$2. (constant time).
Cost of RESET operation:
Changing bits 0, 1, 2,, HP to 0 is free since each bit has SG$1 stuck to it.
Resetting HP to -1 is SG$1.
Amortized cost per RESET operation: SG$1.
(constant time).
Cost of any sequence of n INCREMENT/RESET operations from zero counter is O(n) time.

Printed 9/30/14 10:34 AM

Page 2

CS3230: Design and Analysis of Algorithms

Fall 2014

[NOTE: This solution sketch is INTENTIONALLY made more detailed for the
benefit of those who are seeing amortized complexity analysis for the first time.]
D1. Exercises 17.3-6 of [CLRS] (Queue with Two Stacks)
Show how to implement a queue with two ordinary stacks so that the amortized cost of
each ENQUEUE and each DEQUEUE operation is O(1).
SOLUTION SKETCH:
Idea: Queue is FIFO structure, but we are given stack, which is a LIFO structure.
We assume that we already have the Stack data structure with PUSH and POP operations.
To build our Queue Q, (wow! sounds like qq.com) we use two stacks Stk1 and Stk2.
So, we ENQUEUE items into Stk1 (they store in LIFO format, TOP of Stk1 is last-in item).
When we DEQUEUE from our Queue, if Stk2 is empty, then we need the bottom item of
Stk1. So, we POP off items from Stk1 and PUSH into Stk2, until Stk1 is empty.
Then the items in Stk2 are in FIFO format. And to DEQUEUE, we just POP from Stk2.
Thus, the general procedure for DEQUEUE is as follows: if Stk2 is not empty, then POP
from Stk2. Else if Stk1 is not empty, we move items from Stk1 into Stk2 (as described
above), then POP from Stk2. If Stk1 is also empty, we have a Queue underflow error.
Amortized Analysis: We assign to each ENQUEUE operation $3. $1 is used to pay PUSH
into Stk1, and the remaining $2 is stuck with the item (in Stk1), to pay for subsequent
POP from Stk1 and PUSH into Stk2. Each item in Stk1 has $2, while items in Stk2 has
no credit.
And we assign each DEQUEUE operation $1 to pay for POP from Stk2. Note that if the
deq operation needs to move items from Stk1 into Stk2, that is paid for by the credits in
the Stk1 items. So, amortized cost is $1.
D2. (Exercise 17.1-2 of [CLRS]) (INCREMENT and DECREMENT)
Show that if a DECREMENT operation were included in the k-bit counter example, n
operations could cost as much as (nk) time. (Note: Give such a sequence of operations.)
SOLUTION SKETCH:
Idea: This is similar to T5-R1. If we have INCREMENT and DECREMENT operations, then
we may have a situation where the worst-case time occurs very often and it is not
possible to amortize out the worst-case time.
Now, lets look at the specifics.
The worst-case of INCREMENT occurs when the binary counter is at (1111) and the
operation cost is k flips. (After this, the counter becomes (0000), which is very good
scenario for the INCREMENT operation.
The worst-case of DECREMENT occurs when the binary counter is at (0000) and the
operation cost is k flips. After that the counter becomes (1111).
Thus, if we have a sequence of n/2 {DECREMENT, INCREMENT} operations, then each
operation cost is k flips (worst-case time). So, the total cost of n operations is nk flips.
Hence, total for this sequence of operations is (nk) time.

Printed 9/30/14 10:34 AM

Page 3

CS3230: Design and Analysis of Algorithms

Fall 2014

D3. Modified from Problem 17.3-3 of [CLRS]) Amortized Analysis of Binary Heap
Consider the usual binary min-heap data structure (from [CLRS]-C6), with INSERT and
DELETEMIN done in O(log n) worst-case time. Give an amortized analysis that shows that
INSERT has amortized time O(log n) and DELETEMIN has amortized time O(1).
SOLUTION SKETCH: Intuitive Idea: We know that actual cost of DELETEMIN is O(h)
in the worst-case, where h is the height of the tree (more precisely, h = lg n). Actually, it
makes 2h comparisons when we do the HEAPIFY operation during DELETEMIN.
Now, how can we make its amortized cost O(1) for DELETEMIN?
So, there must be savings from somewhere else and the only other operation is INSERT,
so we should save credit for DELETEMIN when doing INSERT.
Next, how much to save?
In the worst-case, HEAPIFY traces a path of length h from the root down to some leaf, and
makes 2 comparisons per level down this path. So, it takes 2h credit at least.
But, when we INSERT that leaf node, it also requires a path of length h in the worst-case.
So, we can save up the 2h credits when we insert. So, each node on that path will save an
additional 2 credits, to be used later for DELETEMIN operation.
Amortized Analysis: Accounting method;
It is not hard to show (by mathematical induction, for example) that in each node x, the
amount of credits saved is 2*size(Tx), where size(Tx) is the number of nodes in Tx, the
subtree root at x. Another way you can show it is via a table (like that in L6-Slide-33), but
the entries are heaps of different sizes 1, 2, 3, .
Then, we assign a cost of $(3h) for each INSERT operation, where h is the height of the
heap after INSERT operation. $(h) is used up during the insertion process. Then we stick $2
to each node in the insert path (of length h) all the way to the root, giving a total of $(2h).
Then, we during the DELETEMIN operation, we have sufficient credit to pay for HEAPIFY
procedure. During DELETEMIN, when we decrease the size of the heap by 1, we also
remove $(2h) credit which is sufficient to pay for the HEAPIFY procedure.
(Work out the rest of the details yourself, if you still dont get it all.)

Printed 9/30/14 10:34 AM

Page 4

CS3230: Design and Analysis of Algorithms

Fall 2014

D4* Alternative Dynamic Table

Suppose we can INSERT or DELETE an element into a dynamic table in constant time. In
order to ensure that our hash table is always big enough, without wasting a lot of memory,
we will use the following global rebuilding rules:

After an INSERT operation, if the table is more than fill, we allocate a new table
twice as big as current table, insert everything into the new table, and then free the
old table.

After a DELETE operation, if the table is less than full, we allocate a new table
half as big as our current table, insert everything into the new table, and then free
the old table.

Show that for any sequence of n operations (any combination of INSERT or DELETE
operations), the amortized time per operation is still a constant.
(Note: Do not use the potential method (it makes it much more difficult))
[* A little bit harder, but not too much harder. Can skip if there is not enough time.]
SOLUTION SKETCH:
First, make sure you understand how the INSERT and DELETE works for the dynamic table
covered in the lectures. (Read lecture notes L6-Slide-108-118 and [CLRS]-C17.4.2)
Intuitive Idea:
One simplification is to consider expansion-credits and contraction-credits separately.
Consider the situation before-and-after a table expansion. For good amortized costs, the
table should have $0 expansion-credit after table expansion. So, how to do that?
Let new table size after expansion be n.
Just after table expansion, the load-ratio = (#occupied)/(table-size), of the table is 3/8.
And the next table expansion will happen when the table gets to be 6/8 full.
So, each of the 3/8n new inserted items must save expansion-credits for itself-and-a-buddy.
So, for each INSERT operation, we add give $3 to be used as follows: $1 for insert the
item itself, $2 saved expansion-credits to move itself-and-a-buddy during expansion.
Now, consider the situation before-and-after a table contraction. For good amortized costs,
the table should have $0 contraction-credit after table contraction. So, how to do that?
Let new table size after contraction be n.
Just after table contraction, the load-ratio = (#occupied)/(table-size), of the table is 1/2.
The next table contraction happens when table becomes 1/4 full (after deleting n/4 items).
So, each of the n/4 deleted items must save contraction-credits for the n/4 remaining items.
So, for each DELETE operation, we add give $2 to be used as follows: $1 for delete the
item itself, $1 saved contraction-credit to move one-buddy during contraction.
Finally, you work out everything and make sure that we never run into a deficit.
(Note: The system will lose some credits. But, this shows O(1) time per operation. )

Printed 9/30/14 10:34 AM

Page 5