Professional Documents
Culture Documents
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
16 Acknowledgments
17 Index
1
Perhaps more than once
Stepanov, McJones Elements of Programming March 14, 2008 7 / 880
The choice of C++
2
C does not address the issues of memory hierarchies, instruction-level parallelism,
and thread-level parallelism; research is needed to incorporate these features into the
system programming language of the future
Stepanov, McJones Elements of Programming March 14, 2008 8 / 880
The relationship to C++ STL
The book and C++ STL are results of the same research
The book departs from it significantly by providing improved
interfaces in a number of places
The book does not attempt to provide a full set of library
interfaces or deal with all the issues a real library must address
3
Therefore all the constructors defined in the book should be considered to be
explicit
Stepanov, McJones Elements of Programming March 14, 2008 11 / 880
Our use of mathematics
1 Introduction
2 Foundations
Meaning
Objects
Relations on objects
Procedures
Assignment, destruction, and construction
Relationships between types
Concept Regular
Procedures as objects
Generalized procedures
Parameter passing
Regular functions
5 Orderings
6 Combining concepts
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
17 Index
4
Leonard Euler. Elements of Algebra. Third Edition, Longman, London, 1822.
Stepanov, McJones Elements of Programming March 14, 2008 22 / 880
Continuity and discreteness
Definition
An object is a set of memory locations and other resources together
with primitive procedures defined on it that provide its
computational basis
The state of an object is the addresses of its memory locations
together with the content of those locations at a particular point in
time
The interpretation of an object is a mapping from its state to an
entity
An interpretation maps states in such a way that procedures on the
object are consistent with the corresponding activities on entities
Definition
An object is in a well-formed state if there is an entity corresponding to it
Examples
Every state of a computer word is well-formed when interpreted
as an integer
A pointer containing an illegal address is not well-formed
An IEEE 754 floating point NaN (Not a Number) is not
well-formed when interpreted as a real number
Definition
A representation for an entity of a given nature is an object state
corresponding to it; the entity is representable if such a state exists
Definition
A representation is partial if not every entity is representable by an
object state; a representation is total if it is not partial
Example
The type int is a partial representation of integers
Definition
A representation is unique if there is at most one object state
corresponding to every set of particulars of the entity
Examples
A representation of a truth value as a byte that interprets zero as
false and nonzero as true is not unique
Signed-magnitude representation does not provide a unique
representation of zero
Leibniz’s law
x = y if, and only if, everything that may be said about any one of the
things x or y may also be said about the other a
a
Alfred Tarski, Introduction to Logic and to the Methodology of Science, Oxford, 1965
x = y ⇔ ∀P(P(x) ⇔ P(y))
2
Arithmetic: 4 = 21
Geometry: 4ABC = 4ACB
Algebra: (a + b)2 = a2 + 2ab + b2
Definition
Two objects are of the same genus if they represent entities with the
same nature under their corresponding interpretations
Examples
Polar and Cartesian representations of a complex number are of
the same genus
Two’s-complement and signed-magnitude representations of
integers are of the same genus
16-bit and 32-bit unsigned representations of natural numbers are
of the same genus
Definition
Two objects have the same type if they share the same procedures and
the same interpretation
The English word type derives from the Greek word túpoc (tupos):
“the impress made by the blow, what is formed, what leaves its
impress, the form-giving form, hence form generally as outline”5
The technical sense of tupos was “. . . the pattern in conformity to
which a thing must be made” 6
In programming, type is a fundamental idea preceding any notion
of type in programming languages
A type is a design for objects, as an engineering drawing is a
design for manufactured objects
5
Kittel and Friedrich. Theological Dictionary of the New Testament, Volume 8,
Eerdmans, 1972, page 246
6
Thayer and Smith. “Greek Lexicon entry for Tupos”. “The New Testament Greek
Lexicon”
Stepanov, McJones Elements of Programming March 14, 2008 37 / 880
Equality of objects
Definition
Two objects of the same genus are equal if their states represent entities
with equal essential particulars
Definition
A nonessential object is one whose value does not affect the
computation
Example
An object used only to instrument or tune the performance is
nonessential
Definition
The definition space for a procedure is that subset of the direct product
of its argument types to which the procedure is intended to be applied
Definition
The codomain for a procedure is the type it returns
Definition
The result space for a procedure is the set of values from its codomain
returned by the procedure for some inputs from its definition space
Definition
A procedure is partial if its definition space is a subset of the direct
product of the types of its arguments; it is total if they are equala
a
In mathematical usage, “partial” includes “total”; for example, total ordering
implies partial ordering
Definition
A procedure is terminating, semi-terminating, or non-terminating
depending on whether it always terminates, sometimes terminates, or
never terminates on its definition space
Definition
A procedure is stateful, indexed, or stateless depending on whether its
own state is mutable, immutable, or empty
Definition
A procedure is proper if it does not modify any essential objects in its
global state; otherwise it is improper
Definition
A function is a procedure that does not modify its arguments and
returns a newly constructed object called its result; a mutator is a
procedure whose main purpose is to modify its arguments
Definition
An assignment is a mutator taking two objects of the same genus whose
effect is to make the first object equal to the second without modifying
the second
Definition
A destructor is a mutator causing the cessation of an object’s existence
After a destructor has been invoked on an object, no procedure
can be applied to it and its former memory locations and
resources may be reused for other purposes
Definition
An object is in a partially-formed state if it can be assigned to or
destroyed
Definition
A constructor is a mutator transforming memory locations into an
object
Definition
A type constructor is a mechanism for creating a new type from one or
more existing types
Examples
In C++, T* is the type “pointer to T”, for any type T
In C++, struct{T0 , ..., Tn−1 } is an n-ary type constructor
In Chapter 10 we show how to construct types corresponding to
data structures such as lists and arrays
Definition
A type attribute is a mapping from a type to a value describing some
characteristic of the type
Examples
The size of an object in bytes
The alignment of an object
The number of members in a struct
Definition
If F is a procedure type, Arity(F) returns its number of arguments
Examples
Given “pointer to T ”, the value type T
The result type of the difference of two pointers of a given type
The type of the ith member of a struct type (counting from 0)
Definition
If F is a procedure type and i < Arity(F), ArgumentT ype(F, i)
returns the type of the ith argument (counting from 0)
If F is a procedure type, Codomain(F) returns the type of the
result
Definition
A concept is a predicate on types stated in terms of properties of
procedures, type attributes, and type functions defined on the types
Definition
A concept is said to be modeled by specific types, or the types model
the concept, if the properties of the concept are satisfied for those
types
To assert that a concept C is modeled by types t0 , . . . , tn−1 , we
write C(t0 , . . . , tn−1 )
Definition
Concept C 0 refines concept C if whenever C 0 is satisfied for a set of
types, C is also satisfied for those types a
We say C weakens C 0 if C 0 refines C
a
Technically, it is sufficient for C to hold for a subset of the types, possibly reordered
Definition
A type concept is a concept defined on one type
Examples
Integral type
Unsigned integral type
Modeled by: unsigned, unsigned long, unsigned char
Signed integral type
Modeled by: int, long, char
Pointer type
Modeled by: int*, char*
Sequence
Modeled by: std::vector<int>, std::list<int>
Bidirectional iterator
Modeled by: std::list<int>::iterator, int*
Procedure
ProperProcedure
Function
Definition
HomogeneousFunction(F) ⇒
Function(F)
Arity(F) > 0
For all i, j such that 0 6 i, j < Arity(F),
ArgumentT ype(F, i) = ArgumentT ype(F, j)
Definition
The type function Domain(F) is defined on any function type F
satisfying HomogeneousFunction(F):
9
In Chapter 12 we extend Regular with the notion of underlying type and its
affiliated types and functions
Stepanov, McJones Elements of Programming March 14, 2008 63 / 880
Equality for Regular
Definition
A generalized procedure is a procedure defined in terms of type
requirements
Each type requirement has a formal parameter name, introduced
via a typename clause following the template keyword
Requirements on the types are specified via the requires clause,
whose argument is an expression built up from actual types, formal
types, applications of type functions, type equality, concepts, and
their logical connectivesa
a
In this book we use only conjunction
10
In the future C++ is expected to include support for concepts with different syntax
Stepanov, McJones Elements of Programming March 14, 2008 72 / 880
Example of a generalized procedure
Definition
RegularFunction(F) ⇒
Function(F)
Regular(F)
For all f, g ∈ F such that f = g and for all 0 6 i < n:
Regular(ArgumentT ype(F, i))
For all xi , yi ∈ ArgumentT ype(F, i) such that xi = yi ,
where n = Arity(F)
Example
If f and g are regular, we can optimize f(g(x), g(x)) by evaluating g(x)
only once
11
In a binary computer we should in principle measure areas in bits, but unless we
used packed representations we deal in bytes
Stepanov, McJones Elements of Programming March 14, 2008 82 / 880
Storage efficiency
Definition
dlog2 56s(x)e
The byte storage efficiency of a fixed-area object x is area(x) , where
s(x) is the number of distinct states of x
Example
The byte storage efficiency of struct{bool} is 1
The byte storage efficiency of struct{bool,bool} is 1/2
The byte storage efficiency of struct{bool,bool,bool} is 1/3
The cost of a procedure call is the time required for its execution
The worst-case cost of a procedure call for arguments of given areas
is the maximum cost across all procedure calls with arguments of
these areas
The average cost of a procedure call for arguments of given areas is
the average cost across all procedure calls with arguments of these
areas
A procedure has uniform cost if the cost of a procedure call
depends only on the areas of its arguments
A procedure has fixed cost if the cost of a procedure call does not
depend on its arguments
There is a (possibly implicit) function, cost, that returns the cost
when applied to a procedure and set of arguments lying within
the definition space of that procedure
1 Introduction
2 Foundations
6 Combining concepts
9 Rotations
11 Coordinate structures
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Definition
Operation(Op) ⇒
RegularFunction(Op)
HomogeneousFunction(Op)
Codomain(Op) = Domain(Op)
Example
Unary abs : double → double
Binary + : double × double → double
Ternary multiply_add : double × double × double → double
Definition
Transformation(F) ⇒
Operation(F)
Arity(F) = 1
The predicate is_defined : F × Domain(F) → bool is defined
Example
Numbers Square root
Geometry Plane rotations; space translations
Combinatorics Reversal of a sequence
Definition
f0 (x) = x
fn+1 (x) = f(fn (x))
Definition
y is reachable from x under a transformation f if y = x or there is a z
reachable from x and f(z) = y
Definition
x is cyclic under f if f(x) is defined and x is reachable from f(x)
Definition
x is terminal under f if and only if f is not defined at x
Definitions
An orbit of x under a transformation f is the set of all elements
reachable from x under f
Lemma
An orbit does not contain both a cyclic and a terminal element
Lemma
An orbit contains at most one terminal element
Definitions
An orbit of x under f is:
infinite if it has no cyclic or terminal elements
terminating if it has a terminal element
circular if x is cyclic
ρ-shaped if x is not cyclic and its orbit contains a cyclic element
An orbit of x is finite if it is not infinite
terminating
circular
ρ-shaped
Definition
The orbit cycle is the set of cyclic elements in the orbit
Definition
The orbit handle is the complement of the orbit cycle with respect to the
orbit
Definition
The connection point is the first cyclic element
Definitions
The orbit size o of an orbit is the number of distinct elements in it
The handle size h of an orbit is the number of elements in the orbit
handle
The cycle size c of an orbit is the number of elements in the orbit
cycle
Lemma
o=h+c
If two cars, a fast car and a slow car, start along a path, the fast car
will catch up with the slow car if and only if there is a cycle
If there is no cycle, the fast car will reach the end of the path before
the slow car
If there is a cycle, by the time the slow car enters the cycle, the fast
car will already be there, and will catch up eventually
Definition
The collision point of a transformation f and a starting point x is the
unique y such that
y = fn (x) = f2n+1 (x)
and n > 0 is the smallest integer satisfying this condition
Definition
The distance type for a type T is an integer type a that allows us to
encode the maximum number of transformations from one
element of T into another
If a type occupies k bits, its distance type can be represented with
an unsigned integer type occupying k bits
This avoids an infinite tower of types
(It is difficult to have a count type that could count the number of
elements in any collection of type T , because that would require
an extra value)
The type function DistanceT ype(T ) returns the distance type of
T ; for the implementation in C++, see Appendix 2
a
We will discuss integer types briefly in the next chapter and in depth in Chapter 4
typedef DistanceType(Domain(F)) D;
Domain(F) z = x;
D n = D(0);
while (z != y) {
z = f(z);
n = n + D(1);
}
return n;
}
Exercise
Given the C++ function object
struct random_function
{
int operator()(int x)
{
srand(x);
return rand();
}
};
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Since its inception by the Arabs in the ninth century, algebra has
been viewed as the theory of operations on numbers irrespective
of their particular kinds (such as integers or rationals) or values
The common algebraic operations are addition, subtraction,
multiplication, and division
These operations possess fundamental properties such as
associativity of addition or distributivity of multiplication over
addition
12
Jean Dieudonné. Linear Algebra and Geometry. Herman, Paris, 1969, page 17
Stepanov, McJones Elements of Programming March 14, 2008 127 / 880
Grounding abstraction
Definition
BinaryOperation(Op) ⇒
Operation(Op)
Arity(Op) = 2
Definition
SemigroupOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b, c ∈ Domain(Op):
a2 = a ◦ a
a3 = a ◦ a ◦ a
a1 = a
an = a
| ◦ a ◦{z· · · ◦ a}
n
an ◦ am = an+m
Lemma
an ◦ am = am ◦ an = an+m (powers of the same element commute)
Lemma
(an )m = anm
Definition
An element x has finite order under a semigroup operation if there exist
integers 0 < n < m such that xn = xm
Definition
An element x is an idempotent element under a semigroup operation if
x = x2
Theorem
An element of finite order has an idempotent power a
a
Georg Ferdinand Frobenius. Über endliche Gruppen. Berlin 1895. In:
Sitzungesberichte der Königlich Preussischen Akademie der Wissenschaften zu Berlin.
Phys.-math. Classe 1895, pages 163-194
Definition
MonoidOperation(Op) ⇒
SemigroupOperation(Op)
identity_element : Op → Domain(Op) is defined
For all op ∈ Op, for all a ∈ Domain(Op), and
for e = identity_element(op):
op(a, e) = a = op(e, a)
Definition
GroupOperation(Op) ⇒
MonoidOperation(Op)
inverse_operation : Op → (Domain(Op) → Domain(Op)) is
defined
For all op ∈ Op, for all a ∈ Domain(Op), and for
g = inverse_operation(op):
a0 = e
an ◦ a0 = an+0 = an
a−n = (an )−1
an ◦ a−n = an−n = a0
Lemma
(a−1 )n = a−n
13
Gay Robins and Charles Shute. The Rhind Mathematical Papyrus, British Museum
Press, 1987
14
The oldest reference to Russian origin we have found appears in: Sir Thomas
Heath. A History of Greek Mathematics, Volume I, Clarendon Press, 1921, page 53.
Reprint: Dover, 1981. Heath writes: “I have been told that there is a method in use
to-day (some say in Russia, but I have not been able to verify this) . . . ” We have not
been able to verify this either.
Stepanov, McJones Elements of Programming March 14, 2008 144 / 880
Exploiting associativity
Exercise
Prove that power achieves this complexity
The total cost is the sum of the cost of the operations on the
exponent, the cost of the semigroup operation, and the cost of the
control structure
For all the models of Integer we are familiar with, the logarithmic
number of exponent operations performed by power will be faster
than the linear number of operations performed by slow_power,
except for very small n
For most models of the semigroup operation, power will be faster,
but there are exceptions
Reflection
While developing a component, it is often possible to discover
additional useful interfaces
Reflection
Compilers can perform code transformations when the semantics
of the operations are known
Currently this is only for built-in types
Someday we will be able to tell the compiler the semantics of our
operations
Definition
f0 = 0
f1 = 1
fn+2 = fn+1 + fn
Example
0, 1, 1, 2, 3, 5, 8, 13, . . .
1 1
F1 =
1 0
fn+1 fn
Fn =
fn fn−1
fn+1 + fn fn + fn−1 fn+2 fn+1
F1 Fn = = = Fn+1
fn+1 fn fn+1 fn
Fn = F1 F1 . . . F1 = Fn
| {z } 1
n
m n
Fm Fn = F1 F1 = Fm+n
1 = Fm+n
fm+1 fm
Fm =
fm fm−1
fn+1 fn
Fn =
fn fn−1
fm+1 fn+1 + fm fn fm+1 fn + fm fn−1
Fm Fn =
fm fn+1 + fm−1 fn fm fn + fm−1 fn−1
Reflection
Progress in mathematics often involves extending an operator to a new
domain
Example
+ on
natural numbers
integers
rationals
polynomials
matrices
Stepanov, McJones Elements of Programming March 14, 2008 190 / 880
Overloading should preserve semantics
Definition
CommutativeOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b ∈ Domain(Op),
op(a, b) = op(b, a)
Definition
CancellableOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b, c ∈ Domain(Op),
op(a, c) = op(b, c) ⇒ a = b
op(c, a) = op(c, b) ⇒ a = b
Definition
AdditiveSemigroup(T ) ⇒
The binary operation infix + is defined on T
SemigroupOperation(+)
CommutativeOperation(+) a
a
It would have been nice to require the cancellation law, but digital logic designers
use + for an operation that is not cancellable
Definition
MultiplicativeSemigroup(T ) ⇒
The binary operation infix × is defined on T
SemigroupOperation(×)
Definition
AdditiveMonoid(T ) ⇒
AdditiveSemigroup(T )
MonoidOperation(+)
The constant T (0) is defined
identity_element(+) = T (0)
Definition
MultiplicativeMonoid(T ) ⇒
MultiplicativeSemigroup(T )
MonoidOperation(×)
The constant T (1) is defined
identity_element(×) = T (1)
Definition
AdditiveGroup(T ) ⇒
AdditiveMonoid(T )
GroupOperation(+)
The unary operation negation (prefix −) is defined on T
inverse_operation(+) = negation
Definition
For an additive group, subtraction is defined as a − b = a + (−b)
Definition
MultiplicativeGroup(T ) ⇒
MultiplicativeMonoid(T )
GroupOperation(×)
The unary operation reciprocal is defined on T
inverse_operation(×) = reciprocal
Definition
For a multiplicative group, division is defined as
a/b = a × reciprocal(b)
Project
Our definition of the Fibonacci sequence starts from zero and goes up;
extend the definition and our code to work for negative indices a
a
Suggested by Oleg Zabluda
Charles M. Fiduccia.
An efficient formula for linear recurrences.
SIAM Journal of Computing, Volume 14, Number 1, February 1985,
page 106-112.
Generalizes the fibonacci algorithm to arbitrary linear recurrences.
Project
Create a library implementing Fiduccia’s algorithm
Donald E. Knuth.
Addition chains.
The Art of Computer Programming, Volume 2: Seminumerical
Algorithms, 3rd edition, Addison-Wesley, San Francisco, 1998,
pages 465-481.
Describes a way to do minimal-operation exponentiation using
addition chains.
Project
Implement a useful library doing power optimally for exponents
known at compile time
Project
Floating-point multiplication and addition are not associative, so
may give different results when they are used as the operation for
slow_power and power; establish whether slow_power or power
gives a more accurate result for:
1 Raising a floating-point number to an integral power
2 Multiplying a floating-point number by an integral factor
1 Introduction
2 Foundations
5 Orderings
Motivation
Relation; transitivity; ordering
Strict versus reflexive
Symmetric versus asymmetric
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
16 Acknowledgments
17 Index
Definition
Predicate(Op) ⇒
RegularFunction(Op)
Codomain(Op) = bool
Definition
Relation(Op) ⇒
Predicate(Op)
HomogeneousFunction(Op)
Arity(Op) = 2
(Other books use this term for more general non-unary predicates)
Definition
Ordering(Op) ⇒
Relation(Op)
For all op ∈ Op and for all a, b, c ∈ Domain(Op)
Examples
Equality
Equality of the first character
Reachability in an orbit
Divisibility
Definition
An ordering r is strict if ¬r(a, a)
An ordering r is reflexive if r(a, a)
An ordering r is weakly reflexive if r(a, b) ⇒ r(a, a) ∧ r(b, b)
Examples
Reflexive: all examples on the previous slide
Strict: proper factor
Definition
A relation r is symmetric if r(a, b) ⇒ r(b, a)
A relation r is asymmetric if r(a, b) ⇒ ¬r(b, a)
Lemma
A strict ordering is asymmetric
Lemma
A symmetric ordering is weakly reflexive
Examples
Symmetric: sibling
Asymmetric: parent
Definition
EquivalenceRelation(R) ⇒
Ordering(R)
Reflexive(R)
Symmetric(R)
Examples
Equality
Geometric congruence
a ≡ b (mod n)
Lemma
If r is an equivalence relation, a = b ⇒ r(a, b)
Definition
If T is a regular type and r ∈ R is an equivalence relation on T , a
function f ∈ F : T → T 0 is a key function for r if
r(x, x 0 ) ⇔ f(x) = f(x 0 )
Given the choice of a particular key function f, we say f(x) is a
canonical representation of the elements equivalent to x
Definition
StrictTotalOrdering(R) ⇒
Ordering(R)
For all r ∈ R and for all a, b ∈ Domain(R)
symmetric_complementr (a, b) ⇔ a = b
Lemma
A total ordering is strict
Lemma
A total ordering is asymmetric
Lemma
The trichotomy law holds: r(a, b) ∨ r(b, a) ∨ (a = b)
Lemma
Exactly one clause of the trichotomy law holds
r̄((x0 , . . . , xn ), (y0 , . . . , yn )) ≡
(∃k ∈ [0, n])(x0 = y0 ∧ . . . ∧ xk−1 = yk−1 ∧ rk (xk , yk ))
Lemma
r̄ is a strict total ordering
Definition
StrictWeakOrdering(R) ⇒
Ordering(R)
EquivalenceRelation(symmetric_complementR )
Lemma
StrictTotalOrdering(R) ⇒ StrictWeakOrdering(R)
Lemma
A weak ordering is strict
Lemma
A weak ordering is asymmetric
Lemma
The trichotomy law holds:
r(a, b) ∨ r(b, a) ∨ symmetric_complementr (a, b)
Lemma
Exactly one clause of the trichotomy law holds
Example
Weak ordering on ith component of a direct product
There are algorithms that deal with orderings that are not weak:
algorithms on partially-ordered sets
The most important is topological sort:
Donald Knuth.
Topological sorting.
The Art of Computer Programming, Volume 1, Addison-Wesley,
1997, pages 261-268.
Operations as simple as max and min do not make sense without
a weak order
Definition
The total ordering on a type that is consistent with algebraic operations
on it is called the natural total ordering
Example
Examples of consistent axioms for natural ordering:
Incrementable a < successor(a)
Incrementable a < b ⇒ successor(a) < successor(b)
Ordered group a < b ⇒ a + c < b + c
Ordered ring (a < b) ∧ (0 < c) ⇒ ca < cb
Definition
less<T> defines the default ordering for T
Definition
StrictTotallyOrdered(T ) ⇒
Regular(T )
The relation < is defined on T
StrictTotalOrdering(<)
Example
In an additive group, negation and subtraction constitute such a cluster
Lemma
Given an ordering, its complement, its converse, and the
complement of its converse are orderings
If an ordering is strict, its converse is also strict and its
complement and complement of converse are reflexive
If an ordering is reflexive, its converse is also reflexive and its
complement and complement of converse are strict
There exist C++ template function object classes complement,
converse, and complement_of_converse which, given a relation,
construct the corresponding derived relations
For every totally ordered type, we have <, >, 6, and >
We assure consistency by always defining < and relying on the
following templates for the others
Definition
A closed interval [a, b] is the set of all elements x such that a 6 x 6 b
An open interval (a, b) is the set of all elements x such that
a<x<b
A half-open on right interval [a, b) is the set of all elements x such
that a 6 x < b
A half-open on left interval (a, b] is the set of all elements x such that
a<x6b
A half-open interval is our short-hand for half-open on right
These definitions generalize to weak orderings
17
The C++ standard library uses min and other_max
Stepanov, McJones Elements of Programming March 14, 2008 245 / 880
A convention for order selection functions
For the order selection procedures in this section, there are four
useful versions
Since the parameters are being passed for selecting, versions taking
both references and const references are needed
For convenience, we want versions for totally ordered types (with
<) as well as for an explicitly-supplied ordering
From now on, we show only the non-const reference version with
an explict ordering; see Appendix 2 for the other versions
The code below shows how to obtain a version for a totally
ordered type given a version for an explicitly supplied ordering
Definition
An algorithm is stable if it respects the original order of equivalent
elements
Example
Stable sort
Multiple passes compose naturally
Sorting by first name and then by last name results in expected order
Stable partition
Even/odd partition of {1, 2, 3, 4, 5} results in {1, 3, 5, 2, 4}, not
{1, 5, 3, 4, 2} as produced by a fast partitioning algorithm
Stability sometimes increases the complexity
Stepanov, McJones Elements of Programming March 14, 2008 247 / 880
Increasing order
Definition
increasing
A sequence . . . , x, . . . , x 0 , . . . is in order with
strictly increasing
¬r(x 0 , x) a
respect to a strict weak ordering r if
r(x, x 0 )
a
While some would use the terms non-decreasing and increasing, we follow Nicolas
Bourbaki, Theory of Sets, III.1.5, page 138.
sort_2 is stable
It does the minimal amount of work since it does not swap
equivalent elements
min and max satisfy it, while other_min and other_max do not
Lemma
associativity minr (minr (a, b), c) = minr (a, minr (b, c))
associativity maxr (maxr (a, b), c) = maxr (a, maxr (b, c))
commutativity minr (a, b) = minr (b, a)
commutativity maxr (a, b) = maxr (b, a)
absorption minr (a, maxr (a, b)) = a
absorption maxr (minr (a, b), b) = b
idempotency minr (a, a) = a
idempotency maxr (a, a) = a
Lemma
associativity minr (minr (a, b), c) = minr (a, minr (b, c))
associativity maxr (maxr (a, b), c) = maxr (a, maxr (b, c))
commutativity symmetric_complementr (minr (a, b), minr (b, a))
commutativity symmetric_complementr (maxr (a, b), maxr (b, a))
Exercise
What are the absorption and idempotency laws for minr and maxr ?
There are more complicated cases than min and max such as
median of 3 and select 2nd of 4
Writing order selections is somewhat complicated and can be
helped by decomposition into simpler subproblems
Exercise
Define the appropriate stability property for median_3
Lemma
median_3 is stable
Exercise
Is select_2nd_4 stable?
Exercise
Implement select_3rd_4
Exercise
Show that median_5 is not stable
Exercise
Design a stable version of median_5
Exercise
Find an algorithm for median of 5 that does slightly fewer
comparisons on average
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
Combining algebraic structures and ordering
Remainder
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Definition
OrderedAdditiveSemigroup(T ) ⇒
AdditiveSemigroup(T )
StrictTotallyOrdered(T )
For all x, y, z ∈ T ,
x<y⇒x+z<y+z
Definition
OrderedAdditiveMonoid(T ) ⇒ Monoid(T ) ∧ OrderedAdditiveSemigroup(T )
Definition
OrderedAdditiveGroup(T ) ⇒ Group(T ) ∧ OrderedAdditiveMonoid(T )
Definition
OrderedCancellableMonoid(T ) ⇒
OrderedAdditiveMonoid(T )
The partial binary infix operator − is defined on T
For all a, b ∈ T
b 6 a ⇒ is_defined(−, a, b) ∧ (a − b) + b = a
Definition
ArchimedeanMonoid(T) ⇒
OrderedCancellableMonoid(T )
For all a and all b > 0 ∈ T , weak_remainder(a, b) terminates
Reflection
A concept can be defined with an axiom stating that a particular
procedure terminates
18
T.L. Heath, translator. The Works of Archimedes, Preface to Quadrature of the
Parabola. Dover, 2002, page 234
19
Sir Thomas Heath. A History of Greek Mathematics, Volume I. Dover, 1981, page 327
Stepanov, McJones Elements of Programming March 14, 2008 277 / 880
Examples of Archimedean monoids
The first comparison is not needed in the recursive calls and could
be eliminated by the introduction of a helper function
a − b < b serves as a guard to ensure b + b will not overflow
Exercise
Determine the exact counts of different operations
Definition
WeaklyHalvableMonoid(T ) ⇒
ArchimedeanMonoid(T )
The partial unary operation half_nonnegative is defined on T +
where T + = {x ∈ T |x > T (0)}
For all a, b ∈ T +
a = b + b ⇒ half_nonnegative(a) = b
20
Edsger W. Dijkstra attributes this algorithm to N.G. de Bruijn on page 13 of Notes
on Structured Programming, in: O.-J. Dahl, E. W. Dijkstra and C.A.R. Hoare.
Structured Programming. Academic Press, London and New York, 1972. The algorithm
also appears in: Niklaus Wirth. Systematic Programming: An Introduction.
Prentice-Hall, 1973, program 7.22, page 38.
Stepanov, McJones Elements of Programming March 14, 2008 284 / 880
Divisibility on an Archimedean monoid T
Definition
For a > T (0) and b > T (0), b divides a ⇔ remainder(a, b) = T (0)
Lemma
In an Archimedean monoid T with positive x, a, b
b divides a ⇒ b 6 a
b > a ∧ x divides a ∧ x divides b ⇒ x divides (b − a)
x divides a ∧ x divides b ⇒ x divides remainder(a, b)
Definition
DiscreteArchimedeanMonoid(T) ⇒
ArchimedeanMonoid(T )
(∃u ∈ T )(∀x ∈ T )x < u ⇒ ¬(T (0) < x)
Lemma
Every element of a discrete Archimedean monoid is divisible by the
unit
Definition
The greatest common divisor of a and b, denoted by gcd(a, b), is a
divisor of a and b that is divisible by any other divisor of a and b
Lemma
In an Archimedean monoid with positive x, a, b, the following hold
x divides a ∧ x divides b ⇒ x 6 gcd(a, b)
gcd(a, b) is unique
gcd(a, a) = a
Proof.
1 Each subtraction preserves the property of being divisible by any
divisor of the original a and b
2 If the variables become equal, they divide the original a and b
3 Therefore it returns a divisor of a and b that is divisible by any
other divisor
4 In other words, it returns the greatest common divisor
Lemma
It always terminates for integers and rationals
Lemma
It does not always terminate for reals
Definition
EuclideanMonoid(T ) ⇒
ArchimedeanMonoid(T )
For all a > T (0), b > T (0) ∈ T , subtractive_gcd(a, b) terminates
Lemma
Every discrete Archimedean monoid is Euclidean
Definition
CommutativeSemiring(T ) ⇒
AdditiveMonoid(T )
MultiplicativeSemigroup(T ) ∧ CommutativeOperation(×)
For any a, T (0) × a = T (0)
The constant T (1) is defined
For any a , T (0), T (1) × a = a
For all a, b, c ∈ T , a × (b + c) = a × b + a × c
Example
Nonnegative integers constitute a commutative semiring
Definition
Semimodule(T ) ⇒
AdditiveMonoid(T )
The type function Scalar(T ) is defined
CommutativeSemiring(Scalar(T ))
The binary infix function · : Scalar(T ) × T → T is defined
For all α, β ∈ Scalar(T ) and for all x, y ∈ T ,
α · (β · x) = (α × β) · x
(α + β) · x = α · x + β · x
α · (x + y) = α · x + α · y
Scalar(T )(1) · x = x
Example
The set {(a, b)} of two-dimensional vectors with nonnegative real
coefficients constitutes a semimodule over nonnegative integers
Polynomials with nonnegative integer coefficients constitute a
semimodule over nonnegative integers
Definition
QRSemimodule(T) ⇒
Semimodule(T)
The partial binary operation remainder is defined on T
The partial binary infix function
quotient : T × T → Scalar(T ) is defined
For all a ∈ T , b , T (0) ∈ T ,
a = quotient(a, b) · b + remainder(a, b)
Definition
EuclideanSemimodule(T) ⇒
QRSemimodule(T)
For all a, b ∈ T such that ¬(a = T (0) ∧ b = T (0)), gcd(a, b)
terminates
22
Technically, the traditional concept introduced by Noether and van der Waerden
is the Euclidean ring, but semirings suffice
Stepanov, McJones Elements of Programming March 14, 2008 307 / 880
Deriving a fast algorithm for quotient and remainder
23
For an excellent discussion of this and other problems, see: Raymond T. Boute.
The Euclidean Definition of the Functions div and mod. ACM Transactions on
Programming Languages and Systems, Volume 14, Number 2, April 1992, pages 127-144
24
This requirement is equivalent to the classical mathematical definition of
congruence: “If two numbers a and b have the same remainder r relative to the same
modulus k they will be called congruent relative to the modulus k (following Gauss).”
—P. G. L. Dirichlet, Lectures on Number Theory, American Mathematical Society, 1999
Stepanov, McJones Elements of Programming March 14, 2008 313 / 880
Adapting nonnegative remainder and quotient
The next two functions are adaptors that convert remainder and
quotient_remainder correctly defined for nonnegative inputs to
produce “correct” results for positive or negative inputs
Definition
DiscreteArchimedeanSemiring(T ) ⇒
CommutativeSemiring(T )
ArchimedeanMonoid(T )
For all a, b, c ∈ T , a < b ∧ 0 < c ⇒ a × c < b × c
For all a ∈ T , a < T (1) ⇒ ¬(T (0) < a)
Definition
NonnegativeDiscreteArchimedeanSemiring(T ) ⇒
NonnegativeDiscreteArchimedeanSemiring(T )
For all a ∈ T , T (0) 6 a
Definition
DiscreteArchimedeanRing(T ) ⇒
DiscreteArchimedeanSemiring(T )
AdditiveGroup(T )
Definition
Two types T1 and T2 are isomorphic if it is possible to write conversion
functions from T1 to T2 and from T2 to T1 that preserve the procedures
and their semantics
Definition
A concept is univalent if any types satisfying it are isomorphic
25
We follow Peano and include 0 in the natural numbers: Giuseppe Peano.
Formulario Mathematico, Edizioni Cremonese, Roma, 1960, page 27
Stepanov, McJones Elements of Programming March 14, 2008 320 / 880
Computer integer types
Definition
A bounded unsigned integer type, Un , where n = 8, 16, 32, 64, . . ., is an
unsigned integer type capable of representing a value in the interval
[0, 2n )
Definition
A bounded signed integer type, Sn , where n = 8, 16, 32, 64, . . ., is a
signed integer type capable of representing a value in the interval
[−2n−1 , 2n−1 )
sum_extended : Un × Un × U1 → U1 × Un
difference_extended : Un × Un × U1 → U1 × Un
product_extended : Un × Un → U2n
quotient_remainder_extended : U2n × Un → Un × Un
26
Donald E. Knuth. The Art of Computer Programming, Volume 1, Fascicle 1, MMIX : A
RISC Computer for the New Millenium, Addison-Wesley, 2005, pages 1-28
Stepanov, McJones Elements of Programming March 14, 2008 324 / 880
Conclusions
Donald Knuth.
The Greatest Common Divisor, and Division of Polynomials.
The Art of Computer Programming, Volume 2, 3rd edition, 1998,
Sections 4.5.2 and 4.6.1, pages 333-356 and 420-439.
Exhaustive coverage of both Euclidean and Stein (binary) gcd.
Pierre Samuel.
About Euclidean Rings.
Journal of Algebra, Volume 19, 1971, pages 282-301.
Exhaustive and relatively elementary treatment of Euclidean rings.
Project
Find the correct abstract setting for binary binary_gcd_nonnegative
(Stein domain)
Andre Weilert.
(1+ i)-ary GCD Computation in Z[i] as an Analogue of the Binary
GCD Algorithm.
J. Symbolic Computation (2000) 30, pages 605-617.
Ivan Bjerre Damgård and Gudmund Skovbjerg Frandsen.
Efficient algorithms for GCD and cubic residuosity in the ring of
Eisenstein integers.
Proceedings of the 14th International Symposium on Fundamentals of
Computation Theory, Lecture Notes in Computer Science 2751,
Springer-Verlag (2003), pages 109-117.
Saurabh Agarwal and Gudmund Skovbjerg Frandsen.
Binary GCD Like Algorithms for Some Complex Quadratic Rings.
ANTS 2004, pages 57-71.
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Intuition
Memory is a set of locations, each with an address and a value
Getting or setting the value given an address is “fast”
The association of a value with a location is changeable
Examples
The physical and virtual address spaces of a computer
The locations visited by an algorithm
The locations owned by a data structure
Definition
Readable(T ) ⇒
The type function ValueT ype(T ) is defined
Regular(ValueT ype(T ))
source : T → ValueT ype(T ) is defined a
a
source returns its result by value, by reference, or by constant reference
Definition
Writable(T ) ⇒
The type function ValueT ype(T ) is defined
Regular(ValueT ype(T ))
If x ∈ T and v ∈ ValueT ype(T ), then sink(x) ← v is defined
No other use of sink(x) can be justified by the concept Writable,
although other uses may be supported by a specific type modeling
Writable
For a particular state of an object x, only a single assignment to
sink(x) can be justified by the concept Writable; a specific type
might provide a protocol allowing subsequent assignments to
sink(x)
Definition
Mutable(T ) ⇒
Readable(T ) ∧ Writable(T )
For all x ∈ T , is_aliased(x, x)
Definition
Dereferenceable(T ) ⇒
Readable(T ) ∨ Writable(T ) ∨ Mutable(T )
Reflection
It is useful if dereferencing an object x that doesn’t refer to another
object returns x itself
Therefore we assume that unless otherwise defined,
ValueT ype(T ) = T and source and sink return the object to
which they are applied
Definition
Action(T ) ⇒
ProperProcedure(T )
Arity(T ) = 1
T takes its argument by reference
Definition
RegularAction(T ) ⇒
Action(T )
RegularFunction(transformation_from_actionT )
Definition
The amortized complexity of an operation is the complexity averaged
over a worst-case sequence of operations a
a
For an extensive treatment of amortized complexity, see: Robert Endre Tarjan.
Amortized Computational Complexity. SIAM Journal on Algebraic and Discrete
Methods, Volume 6, Number 2, April 1985, pages 306-318
Definition
AmortizedConstantTime(T ) ⇒
ProperProcedure(T )
The amortized time complexity of it is constant and “small”
Definition
Iterator(T ) ⇒
Regular(T )
The type function DistanceT ype(T ) is defined
Integer(DistanceT ype(T ))
The prefix operator ++ is defined on T
Action(++)
AmortizedConstantTime(++)
28
Our treatment of iterators departs significantly from that in the STL
Stepanov, McJones Elements of Programming March 14, 2008 348 / 880
Iterator protocols
Example
Binary search looks for the sequence of iterators whose values are
equal to a given value
This sequence is empty if there are no such values
The location of this empty sequence carries information: the values
in the sequence approximating the given value
Definition
A closed counted range Ji, nK, where n > 0 is an integer, denotes the
sequence of iterators {k|i k successorn−1 (i)}
Definition
A semi-open bounded range [i, j) denotes the sequence of iterators
{k|i k ≺ j}
Definition
A closed bounded range [i, j] denotes the sequence of iterators
{k|i k j}
Stepanov, McJones Elements of Programming March 14, 2008 357 / 880
Terminology for iterators in a range
Definition
An iterator f in a range [i, j) is called the first if f = i ∧ f , j
An iterator l in a range [i, j) is called the limit if l = j
An iterator k in a range [i, j) is called the lasta if successor(k) = j
Otherwise an iterator in the range is called not last
a
The terminology distinguishing limit and last was suggested by John Banning
Definition
The size of a counted range Ji, nM or Ji, nK is n
The size of a semi-open bounded range [i, j) is
distance(i, j, successor)
The size of a closed bounded range [i, j] is
distance(i, j, successor) + 1
Definition
An empty semi-open range is specified by Ji, 0M or [i, i) for some
iterator i
There are no empty closed ranges
Lemma
++ and successor are defined for every iterator in a semi-open range,
and for every iterator except the last in a closed range
Definition
A range r is readable, writable, or mutable if, correspondingly, source,
sink, or both of them are defined on all the iterators in the range
While the ends of both ranges are known to the caller, returning
the pair allows determining which range is smaller and where in
the larger range copying stopped
Definition
ForwardIterator(T ) ⇒
Iterator(T )
RegularAction(++)
Reflection
ForwardIterator allows
algorithms that maintain more than one iterator into a range
multipass algorithms
Definition
BidirectionalIterator(T ) ⇒
ForwardIterator(T )
The prefix operator -- and the corresponding transformation
predecessor are defined
RegularAction(--)
AmortizedConstantTime(--)
For any iterator k in a semi-open range [i, j):
k , j ⇒ predecessor(successor(k)) = k
k , i ⇒ successor(predecessor(k)) = k
Definition
IndexedIterator(T ) ⇒
ForwardIterator(T )
The mutating assignment += is defined on T × DistanceT ype(T )
- : T × T → DistanceT ype(T ) is defined
AmortizedConstantTime(+=)
AmortizedConstantTime(-)
Exercise
Write an appropriate set of axioms relating the operations to each other
int∗
int
Theorem
For any function defined on a range of random access iterators, there is
another function defined on indexed iterators with the same
asymptotic complexity
U ≡ DistanceT ype(I)
W ≡ sign_extendedU
Replace With
DifferenceType(I) W
i < j i - f < j - f
i += n when n < 0 i = f + ((i - f) - U(-n))
i -= n i += -n
i - j W(i - f) - W(j - f)
BI
It FI RI
II
Definition (axioms)
SegmentedIterator(T) ⇒
If
[f, l) is a range of segmented iterators
i ∈ [f, l)
w is a segment iterator in [begin(i), end(i))
then the following hold:
1 is_aliased(i, begin(i))
2 begin(i + w) = w
3 i + begin(i) = i
4 i + (begin(i) + 1) = i + 1
5 begin(i) + 1 , end(i) ⇒ begin(i) + 1 = begin(i + 1)
6 begin(i) , end(i)
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Definition
A (regular) function f is onto if for all y ∈ Codomain(F), there exists
x ∈ Domain(R) such that y = f(x)
Definition
A (regular) function f is one-to-one if for all x, x 0 ∈ Domain(F),
f(x) = f(x 0 ) ⇒ x = x 0
Definition
A permutation is a transformation on a finite domain that is one-to-one
and onto
p(0) = 5
p(1) = 2
p(2) = 4
p(3) = 3
p(4) = 1
p(5) = 0
Definition
A fixed point of a transformation is an element x such that f(x) = x
Definition
The identity permutation on a set S, identityS , maps each element of S to
itself; every element in S is a fixed point of identityS
Definition
If p and q are two permutations on a set S, the composition q ◦ p takes
x ∈ S to q(p(x))
Lemma
The composition of permutations is a permutation
Lemma
Composition of permutations is associative
Lemma
For every permutation p on a set S, there is an inverse permutation p−1
such that p−1 ◦ p = p ◦ p−1 = identityS
Lemma
Every group is a subgroup of a permutation group of its elements
where every permutation in the subgroup is generated by multiplying
all the elements by an individual element
Example
× 1 2 3 4
1 1 2 3 4
2 2 4 1 3
3 3 1 4 2
4 4 3 2 1
Multiplication mod 5:
Every row and column of the multiplication table is a permutation
Stepanov, McJones Elements of Programming March 14, 2008 414 / 880
Cycles in a permutation
Definition
A cycle is a circular orbit within a permutation
Definition
A trivial cycle is one with a cycle size of 1
The element in a trivial cycle is a fixed point
Lemma
Every element in a permutation of a finite set belongs to a unique cycle
p(0) = 5
p(1) = 2
p(2) = 4
p(3) = 3
p(4) = 1
p(5) = 0
p = (0 5)(1 2 4)(3)
Lemma
Any permutation of a set with n elements contains k 6 n disjoint cycles
Each cycle is itself a permutation of this set, called a cyclic permutation
Disjoint cyclic permutations commute
Every permutation can be represented as a product of its cycles
The inverse of a permutation is the product of the inverses of its cycles
Definition
A finite set S of size n is a set for which there exists a pair of functions
chooseS : [0, n) → S
indexS : S → [0, n)
satisfying
Definition
If p is a permutation on a finite set S of size n, there is a corresponding
index permutation p 0 on [0, n) defined as
Lemma
p(x) = chooseS (p 0 (indexS (x)))
Definition
A rearrangement is an algorithm that rearranges the elements of a
mutable range to satisfy a given postcondition
Definition
A rearrangement is stable if it respects the original order of the range to
the maximal extent possible while satisfying the postcondition
Examples
Stable index partition Move all the elements at even positions in the
sequence before the elements at odd positions,
keeping the original order in both groups
Stable partition Move all the elements with even values before
those with odd values, keeping the original
order in both groups
Stable sort Sort pairs of integers by their first components
so pairs with equal first components remain in
the original order
For the same problem, there are often different algorithms for
different iterator requirements
Examples
reverse_forward
reverse_bidirectional
reverse_indexed
Definition
Rearrangements as we have defined them are mutative
Definition
A rearrangement is copying if it sets a writable range to a rearranged
copy of a readable range in way that satisfies a given postcondition
A copying rearrangement could always be obtained by composing
copy with the corresponding mutative rearrangement
Often, however, there are faster algorithms that rearrange while
copying
Definition
A mutative algorithm is in place (or in situ) if it uses an amount of
additional space that is (poly-)logarithmic in the size of the input
Definition
A memory-adaptive algorithm uses as much additional space as it can
acquire to maximize performance
A small percentage of additional space, while theoretically
“linear,” can lead to a large performance improvement
Definition
An algorithm with buffer requires the caller to provide a buffer to be
used by the algorithm
Definition
Every rearrangement corresponds to two permutations of its
range
A to-permutation mapping an iterator i to the iterator pointing to the
destination of the element at i
A from-permutation mapping an iterator i to the iterator pointing to
the origin of the element moved to i
These two permutations are inverses of each other
Theorem
The minimum number of assignments required for a rearrangement is
n + cN − cT , where n is the number of elements, cN the number of
nontrivial cycles, and cT the number of trivial cycles
29
The idea for the proof was provided to us by John Wilkinson
Stepanov, McJones Elements of Programming March 14, 2008 434 / 880
do_cycle_to
Exercise
Implement do_cycle_to and compare the number of assignments it
performs to do_cycle_from
Definition
The permutation p on a finite set with n elements defined by an index
permutation p(i) = (n − 1) − i is called the reverse permutation of the set
Definition
The reverse rearrangement of a range is the rearrangement induced by
the reverse permutation
[c b a] d [g f e] [j i h] k [n m l]
[g f e d c b a] [n m l k j i h]
n m l k j i h g f e d c b a
Lemma
The reverse permutation on [0, n) is the only permutation satisfying
i < j ⇒ p(j) < p(i)
Lemma
Pblog nc
For a range of length n = i=0 ai 2i , where ai is the ith digit in the
binary representation of n, the number of assignments equals
3 Pblog nc
2 i=0 ai i2i
Exercise
Derive a formula for the number of assignments performed by
reverse_n_adaptive for given range and buffer sizes
Conjecture
There is no algorithm that reverses a forward iterator range with
polylogarithmic additional space and in linear time
Donald Knuth.
Section 1.2.5: Permutations and Factorials.
The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,
pages 45-50.
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
Rotation permutations
Least common multiple
Cycle structure of rotations
Interface for rotate
rotate for indexed iterators
Loop fusion
rotate for bidirectional iterators
rotate for forward iterators
Conclusions
Reading
Projects
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Definition
The permutation p on a finite set with n elements defined by an index
permutation p(i) = (i + k) mod n is called the k-rotation of the set
Lemma
The inverse of a k-rotation on an n-element set is an (n − k)-rotation
Definition
The least common multiple of integers a and b, written lcm(a, b), is the
smallest integer m such that both a and b divide it
Lemma
lcm(a, b) is a multiple of a and b that divides any other multiple of a
and b
Theorem
a · b = lcm(a, b) · gcd(a, b)
Proof.
Q i
If the prime decompositions of a and b are respectively pu i and
Q vi Q ui +vi Q max(ui ,vi )
pi , then a · b = p , lcm(a, b) = pi , and
Q min(ui ,vii)
gcd(a, b) = pi , and the result follows from the useful
identity x + y = max(x, y) + min(x, y)
Definition
An ordered set E is said to be a lattice if every subset consisting of two
elements of E has a least upper bound and a greatest lower bound in E
Example
The set of integers > 1, ordered by the relation “m divides n” between
m and n, is a lattice; the least upper bound of {m, n} is lcm(m, n), and
the greatest lower bound is gcd(m, n)
The length of the cycle is the smallest positive integer m such that
i = (i + mk) mod n
This is equivalent to mk mod n = 0, which shows the length of
the cycle to be independent of i
Since m is the smallest positive number such that mk mod n = 0,
it is obvious that lcm(k, n) = mk
lcm(k,n) kn n
m= k = gcd(k,n)k = gcd(k,n)
The number of cycles, therefore, is gcd(k, n)
30
Joseph Tighe suggests returning a pair, m and m 0 , in the order constituting a valid
range; while it is an interesting suggestion and preserves all the information, we do
not yet know of a compelling use of such interface
Stepanov, McJones Elements of Programming March 14, 2008 469 / 880
From-permutation for k-rotation in terms of f, m, l
Lemma
The k-rotation on [0, n) is the only permutation p such that
1 i < n − k ∧ n − k 6 j < n ⇒ p(j) < p(i)
2 i < j < n − k ∨ n − k 6 i < j ⇒ p(i) < p(j)
31
It was suggested to us by Raymond Lo and Wilson Ho
Stepanov, McJones Elements of Programming March 14, 2008 482 / 880
rotate for BidirectionalIterator
Lemma
The number of assignments is 3(bn/2c + bk/2c + b(n − k)/2c), which
gives us 3n when n and k are both even and 3(n − 2) in every other
case
m = m 0 swap_ranges is sufficient
m 0 < m after swap_ranges, [f, m 0 ) are in the final position; we
need to rotate [m 0 , l) around m
m < m 0 after swap_ranges, [f, m) are in the final position; we
need to rotate [m, l) around m 0
This algorithm was first published by:
David Gries and Harlan Mills.
Swapping Sections.
Technical Report 81-452, Department of Computer Science, Cornell
University, January 1981.
Lemma
The number of assignments is 3(n − gcd(n, k))
Donald Knuth.
Section 1.3.3: Permutations and Factorials.
The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,
pages 164-185.
In particular, see problems 34 and 35.
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Definition
A range [f, l) is increasing if for every non-last iterator m ∈ [f, l),
¬(source(successor(m)) < source(m))
A range [f, l) is increasing with respect to a strict weak ordering r if
for every non-last iterator m ∈ [f, l),
¬r(source(successor(m)), source(m))
A range [f, l) is key-increasing a with respect to a regular function
key and a strict weak ordering r if for every non-last iterator
m ∈ [f, l), ¬r(key(source(successor(m))), key(source(k)))
a
In database parlance, a key is a part of a record on which sorting is done
Definition
A range is partitioned if it is key-increasing with respect to a key
function that is a unary predicate
More generally, a range is k-partitioned if it is key-increasing with
respect to a key function that returns one of k distinct values for
some (compile-time) constant integer k > 0
Definition
A partition point in a range [f, l) partitioned by a unary predicate p is an
iterator i ∈ [f, l] such that ¬p(source(j)) for all j ∈ [f, i) and
p(source(k)) for all k ∈ [i, l)
Definition
In an increasing range [f, l), every value a of the value type of the
range determines two partition points, lower bound and upper bound,
defined by, respectively:
Pa (x) ≡ x < a
Pa0 (x) ≡ ¬(a < x)
Lemma
lower bound upper bound
A sequence with lower bound i and upper bound j for the value a
(note any of the three regions may be empty):
Lemma
Given any range and a unary predicate, some prefix of the range is
partitioned by the predicate
Then finding the first element not satisfying the predicate gives
the longest partitioned prefix
template <typename I, typename P>
requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_if_not(I f, I l, P p) {
while (f != l && p(source(f)))
++f;
return f;
}
Lemma
Choosing m in the middle of the range assures optimal worst-case
performance
32
Bernard Bolzano. Rein analytischer Beweis des Lehrsatzes, daß zwischen je zwey
Werthen, die ein entgegengesetztes Resultat gewähren, wenigstens eine reelle Wurzel der
Gleichung liege. 1817
33
A.-L. Cauchy. Cours D’Analyse de L’Ecole Royale Polytechnique. 1821
34
This book explores the connections between some simple algebraic concepts and
algorithms; we strongly believe it is also possible to define algorithms on analytic
concepts
Stepanov, McJones Elements of Programming March 14, 2008 518 / 880
bisection_interpolation
35
T.K. Lakshman suggested the interface
Stepanov, McJones Elements of Programming March 14, 2008 523 / 880
Properties of partition_copy 37
It is stable: the values of each of the two output ranges are in the
same relative order as in the input range
It is a copying rearrangement36
It is optimal in terms both of number of predicate applications and
assignments (both numbers are n, the size of the range)
36
Note that if the input range is defined by mutable forward iterators, it may be
supplied as one of the output ranges
37
A variation we use later, partition_copy_n, is given in Appendix 2
Stepanov, McJones Elements of Programming March 14, 2008 524 / 880
Semistable partition rearrangements
Definition
A partition rearrangement is semistable if the relative order of elements
not satisfying the predicate is preserved
38
In Chapter 12 we will explain why this is the right thing to do
Stepanov, McJones Elements of Programming March 14, 2008 525 / 880
semistable_partition39
39
The algorithm is due to Nico Lomuto, according to: Jon Bentley. Programming
Pearls. Communications of the ACM, Volume 27, Number 4, April 1984, pages 287-291
Stepanov, McJones Elements of Programming March 14, 2008 526 / 880
Useful lemmas
The annotations on the previous slide follow from some trivial but
useful lemmas
Lemma
After executing i = find_if(f, l, p), none(f, i, p) ∧ (i = l ∨ p(i))
After executing j = find_if_not(f, l, p), all(f, j, p) ∧ (j = l ∨ ¬p(j))
If p(source(i)) ∧ q(source(j)) before executing cycle_2(i, j), then
afterward q(source(i)) ∧ p(source(j))
none(f, l, p) ∧ ¬p(l) ⇒ none(f, l + 1, p)
all(f + 1, l, p) ∧ p(f) ⇒ all(f, l, p)
It is in place
Let n = l − f be the number of elements in the range and let w be
the number of elements not satisfying the predicate following the
partitioned prefix40
The predicate is applied n times
cycle_2 is performed w times
The number of iterator increments is n + w
It is totally inappropriate for use in quicksort since it does not
divide a range of equal values at the midpoint and that leads to
quadratic complexity in many practical situations
40
w = count_if_not(find_partitioned_prefix(f, l, p), l, p)
Stepanov, McJones Elements of Programming March 14, 2008 528 / 880
semistable_partition performs too many
assignments
Consider a range t, f, f, f, f, f, f, f
semistable_partition will perform seven calls of cycle_2, while
one suffices
It is in place
The number of times cycle_2 is performed, v, equals the number
of misplaced elements not satisfying the predicate, that is, the
number of elements not satisfying the predicate in the subrange
from the potential partition point to the end of the range41
The algorithm applies the predicate 2n times, therefore it is useful
when the cost of predicate application is insignificant compared to
the cost of cycle_2
This technique of computing the sizes of a set of buckets and then
moving the elements appears in radix sort42
With a bidirectional iterator, we can avoid precomputing the
bucket sizes
41
v = count_if_not(potential_partition_point(f, l, p))
42
See the use of this technique for in-place k-partition Exercise 5.2-13 in Donald
Knuth. The Art of Computer Programming. Volume 3, Addison-Wesley, 1998, page 80;
Solution, page 618
Stepanov, McJones Elements of Programming March 14, 2008 532 / 880
find_backward_if and find_backward_if_not
To indicate “not found”, we return f, which forces us to return
i + 1 if we find a satisfying element at position i
template <typename I, typename P>
requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_backward_if(I f, I l, P p) {
while (true) {
if (f == l) return f;
--l;
if (p(source(l))) return successor(l);
}
}
template <typename I, typename P>
requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_backward_if_not(I f, I l, P p) {
while (true) {
if (f == l) return f;
--l;
if (!p(source(l))) return successor(l);
}
}
Stepanov, McJones Elements of Programming March 14, 2008 533 / 880
unstable_bidirectional_partition
It is in place
The predicate is applied n times
cycle_2 is called v times (where v is as previously defined for
unstable_forward_partition)
The algorithm moves only those elements that need to be moved,
and puts two elements in their final destination with each call to
cycle_2
The total number of assignments, therefore, is 3v
43
It is possible to combine the sentinel and single-cycle techniques, producing
sentinel_single_cycle_partition
Stepanov, McJones Elements of Programming March 14, 2008 542 / 880
The origin of partition algorithms
44
C.A.R. Hoare. Quicksort. The Computer Journal, Volume 5, Number 1, 1962, pages
10-16. This paper, which is a serious contender for the best-ever computer science
paper, should not be confused with his earlier publication: C.A.R. Hoare. Algorithm
63, Partition; Algorithm 64, Quicksort; Algorithm 65, Find. Communications of the
ACM, Volume 4, Number 7, July 1961, pages 321-322. No one should attempt to
implement or teach quicksort without studying this paper.
Stepanov, McJones Elements of Programming March 14, 2008 543 / 880
Regularity of predicate for partition algorithms
Exercise
Design an algorithm implementing in-place uniform random
shuffle of a range of forward iterators (hint: recursively partition
the range with a non-regular coin-tossing predicate a )
Prove that the expected number of coin tosses is n log2 n
Prove that in-place uniform random shuffle of a range of forward
iterators cannot be done in linear time
a
The algorithm was discovered by Raymond Lo and Wilson Ho
Exercise
Determine the number of assignments for the adaptive version
Pa (x) ≡ x < a
Pa0 (x) ≡ ¬(a < x)
Exercise
Implement a function that returns both lower and upper bounds and
does fewer comparisons than the sum of the comparisons that would
be done by calling both lower_bound_n and upper_bound_na
a
A similar STL function is called equal_range
Definition
Merge is an operation that combines two increasing ranges into a single
increasing range that contains all the elements of both ranges and no
other elements
Definition
A merge is stable if the output range preserves the relative order of
equivalent elements both within each input range, and between the
first and second input range
Lemma
Stability establishes a unique ordering of the elements in the output
range
45
A mutating merge may or may not be in place depending on how much extra
memory it uses
46
We define node iterators in the next chapter
Stepanov, McJones Elements of Programming March 14, 2008 556 / 880
is_mergeable
Exercise
Implement this function that checks whether the range starting at o is
equal to the merge of the ranges [f0, l0) and [f1, l1)
template <typename I, typename R>
requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))
bool is_merged(I f0, I l0, I f1, I l1, I o, R r);
If ◦ denotes the merge operation and C(x, y) denotes the cost (the
number of assignments) to merge ranges x and y, then C
satisfies47 :
C(x, y) + C(x, z) + C(y, z)
C(x ◦ y, z) =
2
A consequence of this is:
47
In the next chapter we will define a Huffman semigroup, where the cost of the
semigroup operation satisfies this axiom
Stepanov, McJones Elements of Programming March 14, 2008 560 / 880
merge_copy_n
template <typename I0, typename I1, typename O, typename R>
requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&
Writable(O) && Iterator(O) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&
ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))
void merge_copy_n(I0 f0, DistanceType(I0) n0,
I1 f1, DistanceType(I1) n1, O o, R r) {
// Precondition: f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1, r)
// Precondition: the input ranges do not overlap the output range
typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;
while (true) {
if (n0 == N0(0)) { copy_n(f1, n1, o); return; }
if (n1 == N1(0)) { copy_n(f0, n0, o); return; }
if (r(source(f1), source(f0))) {
sink(o) = source(f1); ++f1; n1 = n1 - N1(1);
} else {
sink(o) = source(f0); ++f0; n0 = n0 - N0(1);
}
++o;
}
}
Stability requires the order of the if clauses is reversed from the one in merge_copy_n
Stepanov, McJones Elements of Programming March 14, 2008 562 / 880
Return value for copying merges
Exercise
Reimplement merge_copy_n and merge_copy_backward_n with the
correct return values
Lemma
The number of assignments is always n0 + n1 and the worst-case
number of comparisons is n0 + n1 − 1
Exercise
Determine the average number of comparisons
Definition
A key-sorting (or just sorting) algorithm is a rearrangement those
postcondition is that the range is key-increasing (or just
increasing) with respect to a given strict weak order
A key-sorting (or sorting) algorithm is stable if it does not change
the relative order of values that are equivalent under the
symmetric complement of the given strict weak order
dlog2 ne
X n
ndlog2 ne − ≈ ndlog2 ne − n
2i
i=1
Proof.
Consider j = lower_bound(m, l, source(i), r)
Lemma
1 Each call of rotate puts one element into its final position
2 The algorithm terminates with a sorted range
3 Each level of recursion reduces m = min(n0, n1) to no more than
bm/2c
4 There are at most blog2 (min(n0, n1))c + 1 recursive levels
48
Krzysztof Dudziński and Andrzej Dydek. On a Stable Minimum Storage Merging
Algorithm. Information Processing Letters, Volume 12, Number 1, February 1981, pages
5-8.
Stepanov, McJones Elements of Programming March 14, 2008 576 / 880
Complexity of merge_n_in_place: number of
assignments
Theorem
Let
kn be the maximum number of assignments required to rotate a
range of length n
a(n0 , n1 ) be the maximum number of assignments required for a
stable in-place merge of two ranges with lengths n0 and n1 , where
n0 6 n1
Then
a(n0 , n1 ) 6 k(n1 + dn0 /2e) log2 (2n0 ) 6 kn log2 n
Proof.
By induction on n0 .
n0 = 1 A single rotation of a range of length at most n1 + 1 accomplishes the merge, so
a(1, n1 ) = r(n1 + 1) 6 k(n1 + 1) = k(n1 + d1/2e) log2 (2).
n0 > 1 We have a(n0 , n1 ) 6 r(n1 + n0 − h0 ) + a(h0 , h1 ) + a(n0 − h0 − 1, n1 − h1 ). By
the induction hypothesis, a(h0 , h1 ) 6 k(h1 + dh0 /2e) log2 (2h0 ) and
It is easily verified that dx/2e + dy/2e 6 d(x + y + 1)/2e for any x and y. Thus
and
a(n0 , n1 ) 6 k(n1 + dn0 /2e)(log2 (n0 ) + 1) = k(n1 + dn0 /2e)(log2 (2n0 )).
49
We are indebted to John Wilkinson for this proof
Stepanov, McJones Elements of Programming March 14, 2008 578 / 880
Complexity of merge_n_in_place: number of
comparisons
Theorem
Let c(n0 , n1 ) be the maximum number of comparisons required for a
stable in-place merge of a range of length n0 with a range of length n1 ,
where n0 6 n1 ; then c(n0 , n1 ) 6 kn1
Exercise
Determine the bound k and prove the theorem
50
Donald Knuth ranked this problem as 47 out of 50 – just a notch below the Last
Fermat Theorem – in the first edition Volume 3 of his Art of Computer Programming
Stepanov, McJones Elements of Programming March 14, 2008 585 / 880
The origin of computer sorting and binary searching
51
Martin Campbell-Kelly and Michael R. Williams, editors. The Moore School
Lectures: Theory and Techniques for Design of Electronic Digital Computers. The
MIT Press and Tomash Publishers, 1985.
Stepanov, McJones Elements of Programming March 14, 2008 586 / 880
Conclusions
Mathematicians use definitions and theorems as building blocks
for ever more powerful results
This chapter shows how a difficult problem of stable sorting in
minimal space can be effectively solved using simple building
blocks
Copying (forward and backward)
Rotate (for various iterator categories)
Lower and upper bound
It provides specific examples of why our attention to interfaces
(arguments and return values) is important
It shows that the technique of memory adaptive algorithms,
which we first introduced in a rather artifical context (reverse for
forward iterators), leads to a simple, efficient version of an
important algorithm (stable sort)
Lower and upper bound, merge, and sort allow sequences to be
used efficiently as representations of sets and multisets
Stepanov, McJones Elements of Programming March 14, 2008 587 / 880
Project
Project
Design a benchmark comparing performance of
unstable_bidirectional_partition, single_cycle_partition,
sentinel_partition, and sentinel_single_cycle_partition for
different array sizes, element sizes, and distributions of elements
satisfying the predicate
Design a benchmark to analyze the performance of stable_sort_n
for different array sizes, element sizes, and buffer sizes
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
Coordinate structures
Linkable iterators
Linking operations
Link rearrangement
Reversing a linkable range
Examples of linkable iterators
Generalized rearrangements
Partition
12 Composite objects
14 Mathematical notation
16 Acknowledgments
17 Index
Definition
A concept is a coordinate structure if it consists of:
One or more coordinate types
For each coordinate type Ti , a value type ValueT ype(Ti )
For each coordinate type, dereferencing functions source and/or
sink providing “fast” readable, writable, or mutable access
Access via a coordinate value is as fast as via any other mechanism,
so there is no need to “cache” a low-level pointer or reference
For each coordinate type Ti , one or more traversal functions:
fi,j : Ti × · · · → Tk
Definition
ForwardLinker(S) ⇒
A type function IteratorT ype(S) is defined
ForwardIterator(IteratorT ype(S))
An object of type S is a proper mutator taking two iterators of the
affiliated type
For all set_link ∈ S, immediately after set_link(i, j) is performed,
successor(i) = j
Definition
BackwardLinker(S) ⇒
A type function IteratorT ype(S) is defined
BidirectionalIterator(IteratorT ype(S))
An object of type S is a proper mutator taking two iterators of the
affiliated type
For all set_link ∈ S, immediately after set_link(i, j) is performed,
i = predecessor(j)
Definition
BidirectionalLinker(S) ⇒
ForwardLinker(S)
BackwardLinker(S)
Definition
LinkableForwardIterator(I) ⇒
ForwardIterator(I)
For all i ∈ I,
source(i.p).forward_link = successor(i)
For all j ∈ I, immediately after sink(i.p).forward_link ← j,
successor(i) = j
Definition
LinkableBidirectionalIterator(I) ⇒
BidirectionalIterator(I) ∧ LinkableForwardIterator(I)
For all j ∈ I,
source(j.p).backward_link = predecessor(j)
For all i ∈ I, immediately after sink(j.p).backward_link ← i,
i = predecessor(j)
53
Undisciplined use of goto leads to code that is harder to understand and less
likely to be correct; disciplined use frequently improves clarity and efficiency.
Benchmarking is required to verify that versions with goto are faster on processors
with instruction level parallelism.
Stepanov, McJones Elements of Programming March 14, 2008 612 / 880
partition_linkable signature
Exercise
sort_linkable_n and merge_sort_n from Chapter 9 are very similar;
figure out how to unify them
Definition
WeakBifurcateCoordinate(T ) ⇒
Regular(T )
The type function WeightT ype(T ) is defined
Integer(WeightT ype(T ))
The unary predicates is_empty, has_left_successor and
has_right_successor are defined on T
The actions step_left and step_right, and the corresponding
transformations left_successor and right_successor, are defined
on T
AmortizedConstantTime(step_left) ∧
AmortizedConstantTime(step_right)
Definition
BifurcateCoordinate(T ) ⇒
WeakBifurcateCoordinate(T )
RegularAction(step_left) ∧ RegularAction(step_right)
Definition (signatures)
template <typename C>
requires(BidirectionalBifurcateCoordinate(C))
bool is_left_successor(C c)
{
// Precondition: has_predecessor(c)
C parent = predecessor(c);
return has_left_successor(parent) && left_successor(parent) == c;
}
Definition (axioms)
For any coordinate x ∈ T :
has_left_successor(x) ⇒ predecessor(left_successor(x)) = x
has_right_successor(x) ⇒ predecessor(right_successor(x)) = x
has_predecessor(x) ⇒
is_left_successor(x) ∨ is_right_successor(x)
Definition
The descendants of a bifurcate coordinate x are all the coordinates y
such that reachable(x, y)
Definition
The descendants of x are acyclic if for all y in the descendants of x,
¬left_reachable(y, y) ∧ ¬right_reachable(y, y)
Definition
The descendants of x are a tree if they are acyclic and for all y, z in the
descendants of x, ¬left_reachable(y, z) ∨ ¬right_reachable(y, z)
Lemma
The descendants of x are a finite tree if and only if they are a tree and
weight_recursive(x) terminates
Lemma
height(x) 6 weight(x)
Definition
LinkableBifurcateCoordinate(T ) ⇒
BifurcateCoordinate(T )
Proper mutators set_left_successor and set_right_successor
taking two iterators of type T are defined
Immediately after set_left_successor(i, j) is performed,
left_succesor(i) = j
Immediately after set_right_successor(i, j) is performed,
right_succesor(i) = j
Definition
LinkableBidirectionalBifurcateCoordinate(T ) ⇒
LinkableBifurcateCoordinate(T )
Immediately after set_left_successor(i, j) is performed,
has_left_successor(i) ⇒ predecessor(j) = i
Immediately after set_right_successor(i, j) is performed,
has_right_successor(i) ⇒ predecessor(j) = i
55
The exclusive-or technique was first applied to binary trees in: Laurent Siklóssy.
Fast and Read-only Algorithms for Traversing Trees Without an Auxiliary Stack.
Information Processing Letters, Volume 1, 1972, pages 149-152.
Stepanov, McJones Elements of Programming March 14, 2008 646 / 880
Similarity
Definition
Two sets S and S 0 of coordinates from the same coordinate type T of a
coordinate structure are similar under a given one-to-one
correspondence if any traversal function maps corresponding
coordinates to corresponding coordinates
Examples
Two ranges of iterators [f, l) and [f 0 , l 0 ) are similar if l − f = l 0 − f 0
Two trees are similar if both are empty or they have similar left
and right subtrees
Definition
Two sets S and S 0 of coordinates from the same coordinate type T of a
coordinate structure represent equivalent value sets under a given
one-to-one correspondence and a given equivalence relation if they are
similar and dereferencing corresponding coordinates gives equivalent
objects
Project 1
Design an adapter that converts traverse_step into an iterator, taking
into consideration the following issues:
What should the value type be?
What iterator concept should be modeled?
What should the limit iterator be to allow traversal of a subtree
rooted at an arbitrary coordinate?
Project 2
Extend ordering to an arbitrary coordinate structure by defining an
appropriate meta-algorithm
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
Composite objects, regular types, and containers
Equality on composite objects
Collocated types
Distributed types
Construction
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Definition
A type T belongs to a concept that is a composite object concept if:
Parts Objects of type T are made up of other objects called
parts
Subparts Subsidiary definition: x is a subpart of y if x is a part of y
or x is a subpart of a part of y
Definition
Connected T has an affiliated coordinate structure
Every object of type T has a distinguished starting
address
It is possible to reach every part of an object of type T
using traversals in the coordinate structure, beginning
at the starting address
Definition
Noncircular No object is a subpart of itself
Disjoint Subsidiary definition: Two objects are disjoint if they
have no subpart in common
If x and y are objects, either one is a subpart of the
other or they are disjoint
Definition
Complete destruction Destroying an object of type T causes the
destruction of every part of the object (and,
therefore, every subpart)
Example
A tuple: traversal via member access
An array: traversal via random access iterator
A binary tree: traversal via bifurcate coordinate
A complex number: traversal via member access
Definition
A container is a composite object whose interpretation is a collection of
the interpretations of the parts of the container
Example
A tuple
An array
A binary tree
Definition
Each container type defines a corresponding coordinate structure and
functions giving one or more specific coordinates to initiate traversals
or access
Example
Tuple: member access
Array: random-access iterators and begin and end
Binary tree: bifurcate coordinates and root
Definition
Two containers are equal if corresponding parts are equal
Example
Array equality: use lexicographic_equivalence with equality on
the element type
Binary tree equality: use bifircate_equivalence with equality on
the element type
Definition
Two composite objects are structurally equal if their states are equal
Examples
Complex numbers x + iy and x 0 + iy 0 are equal if x = x 0 ∧ y = y 0
0
Complex numbers reiϕ and r 0 eiϕ are equal if
r = r0 ∧ ϕ ≡ ϕ0 (mod 2π)
Example
Implementing Leibniz equality for priority queues requires comparing
successive values from both queues
This consumes the elements compared, requiring copies to be
made of the two queues
It requires n log n time
Definition
A collocated type is one all of whose parts are adjacent in memory
Example
Given types T0 , . . . , Tk−1 , we can construct the heterogeneous type
structT0 ,...,Tk−1 with useful special cases
pairT0 ,T1
tripleT0 ,T1 ,T2
If all the Ti are the same, we can define the homogeneous type
arrayk,T
56
In some situations C++ automatically generates elementwise constructors,
destructor, and assignment for struct types; equality and ordering are never
automatically generated. Constant-size arrays in C++ are not first-class citizens and
one must define a type such as the array_k in Appendix 2 to make them be regular.
Stepanov, McJones Elements of Programming March 14, 2008 676 / 880
Distributed types
Definition
A distributed type has parts that are not adjacent:
Data representation of the externally-visible behavior of the
type
Connectors traversal through all the data
Header the origin for all traversal
Padding memory locations included to maintain hardware
alignment or optimize cache behavior
Reserve memory locations reserved for future growth
Other locks, software caches, handles for system resources, etc.
Example: list
The functions begin and end, when applied to a list, each return a list
iterator
The function successor, when applied to a list iterator, returns another
list iterator; there is no function like successor defined on list
The destructor of the listT type is responsible for iterating through all
the elements, destroying each object of type T , and freeing the storage of
its containing list node
Destroying a list iterator has no impact on any list object
Definition
A primary object is one that is not a subpart of another object; an object
that is not primary is secondary
Example
***** Show code with global object, local object, and insert constructor
?????
Many languages follow the example of Lisp57 use the same type
for both a list as a container and a list iterator
This design decision means it is not possible for the programmer
to explicitly free a list since there might be other variables
containing list objects with overlapping storage
57
John McCarthy. Recursive Functions of Symbolic Expressions and Their
Computation by Machine, Part I. Communications of the ACM, Volume 3, Number 4,
1960, pages 184-195
Stepanov, McJones Elements of Programming March 14, 2008 682 / 880
remote
void* raw_allocate(size_t n)
{
return ::operator new(n);
}
void deallocate(void* p)
{
::operator delete(p);
}
Definition
The type function IteratorT ype(C) is defined on any container type C;
it returns a type such that Iterator(IteratorT ype(C))
Definition
A relational concept is a concept defined on two types
Definition
SameGenus(T , T 0 ) ⇒
The genus of T and the genus of T 0 are equal
Example
SameGenus(int, long)
Definition (axioms)
PartiallyCompatible(T , U) ⇒
Where = is defined, it is true if and only if its arguments have the
same interpretation
← depends on the type but not the value of its first argument;
where it is defined, it makes the first argument equal to the second
without modifying the second
Where the morphisms U(T ) and T (U) are defined, each constructs
a new element of the target type that has the same interpretation
as its argument
Lemma
PartiallyCompatible(T , U) ⇒ PartiallyCompatible(U, T )
Lemma
If PartiallyCompatible(T , U), PartiallyCompatible(U, V),
PartiallyCompatible(T , V), then (∀t ∈ T )(∀u ∈ U)(∀v ∈ V):
t=u⇒u=t
(t = u ∧ u = v) ⇒ t = v
t←u⇒t=u
T t(u) ⇒ t = u
Definition
Compatible(T , U) ⇒
PartiallyCompatible(T , U)
For all t ∈ T and for all u ∈ U,
is_representableU (t) ∧ is_representableT (u)
#if 0 // *****
While the ends of both ranges are known to the caller, returning
the pair allows determining which range is smaller and where in
the larger range copying stopped
Definition
For any type T , the underlying type U is an affiliated type satisfying:
T and U are compatible
References with value types T and U may be reinterpretively cast
into each other
Construction of type U and assignment to type U never throw an
exception
An object of type U may only be used to hold temporary values
while implementing a rearrangement of a range of T objects
A reference to an object of type U may be reinterpretively cast to a
reference to T and passed as a const T& parameter
We denote the underlying type of T as UnderlyingT ype(T )
Reflection
Disciplined violation of invariants to enhance performance is perfectly
legitimate
#endif // *****
...
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Definition
PartialSemigroupOperation(Op) ⇒
BinaryOperation(Op)
For all op ∈ Op and for all a, b, c, ∈ Domain(Op), if
is_defined(op, a, b) ∧ is_defined(op, b, c)
then
is_defined(op, op(a, b), c) ∧ is_defined(op, a, op(b, c)) ∧
op(op(a, b), c) = op(a, op(b, c))
Project
***** Write an iterative version of stable_partition_n and
stable_partition_n_adaptive from Chapter 9 using
reduce_balanced
***** Give hint that many z’s per op will be required ?????
***** Compare the performance of the corresponding recursive
and iterative versions
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
16 Acknowledgments
17 Index
Definition
If P and Q are propositions, then so are ¬P (read as “not P”), P ∨ Q (“P
or Q”), P ∧ Q (“P and Q”), P ⇒ Q (“P implies Q”), and P ⇔ Q (“P is
equivalent to Q”), with these truth tables:
Definition
If P is a proposition and x is a variable, then (∃x)P is a proposition
(read as "there exists x such that P")
Definition
If P is a proposition and x is a variable, then (∀x)P is a proposition
(read as “for all x, P”); (∀x)P ⇔ (¬(∃x)¬P)
1 Introduction
2 Foundations
5 Orderings
6 Combining concepts
9 Rotations
11 Coordinate structures
12 Composite objects
14 Mathematical notation
16 Acknowledgments
#define requires(...)
#define models(...)
template <>
struct distance_type<short>
{
typedef unsigned short type;
};
BI
It FI RI
II
***** Need const and non-const versions of begin and end ?????
***** What about subscript: a[i] ?????
template <int k, typename T>
requires(Regular(T))
struct array_k
{
typedef T* I; // IteratorT ype(arrayk,T )
T a[k];
array_k() { }
array_k(const array_k& x) { basic_copy_k_indexed<k, I, O>(&x.a, &a); }
friend void operator=(const array_k& x)
{ basic_copy_k_indexed<k, I, O>(&x.a, &a); }
friend size_t size(const array_k& x) { return size_t(k); }
friend I begin(array_k& x) { return &x.a[0]; }
friend I end(array_k& x) { return &x.a[k]; }
friend bool operator==(const array_k& x, const array_k& y)
{ return lexicographic_equality_k<k, I, I>(begin(x), begin(y)); }
friend bool operator<(const array_k& x, const array_k& y)
{ return lexicographic_ordering_k<k, I, I>(begin(x), begin(y)); }
};
default constructor
for Regular, 67
default ordering, 236
definition space of procedure, 41
Dereferenceable concept, 340
derived relations, 239
descendants
of bifurcate coordinate, 633
description, 18
destructor, 50
for Regular, 66
DifferenceT ype type function, 387
direct product (×), 805
discrete Archimedean monoid, 287
DiscreteArchimedeanMonoid concept, 287
DiscreteArchimedeanRing concept, 319
DiscreteArchimedeanSemiring concept, 317
discreteness, 317
distance type, 110
DistanceT ype type function, 348
distributivity
of semiring, 298
divisibility lattice of integers, 466
divisibility on an Archimedean monoid, 285
division
Egyptian, 279
division on multiplicative group, 201
Fibonacci
matrices, 185
numbers, 182
Fiduccia, Charles M., 206
finite set, 418
finite tree
descendents of bifurcate coordinate, 634
first in a range, 358
fixed area, 82
fixed cost, 84
fixed point, 411
for all (∀), 804
ForwardIterator concept, 371
ForwardLinker concept, 600
from-permutation, 431
function
one-to-one, 408
onto, 407
function (→), 805
Function concept, 60
function procedure, 47
idea, 18
idempotency
k-partition, 504
key function, 499, 569
key-increasing range, 499
key-sorting, 569
object, 24, 25
interpretation, 25
state, 25
one-to-one function, 408
onto function, 407
partial
procedure, 43
representation, 27
partial ordering
example, 231
topological sort defined on, 233
same genus, 35
SameGenus concept, 727
terminating procedure, 44
thing, 18
Tighe, Joseph, 469
time complexity
of rearrangement, 427, 434
to-permutation, 431
weak
usage convention, 232
WeakBifurcateCoordinate concept, 626
weakening of concept, 57
weakly reflexive ordering, 217
WeaklyHalvableMonoid concept, 283
WeightT ype type function, 626
well-formed object state, 26
with buffer algorithm, 426
worst-case cost, 84
Writable concept, 337
writable range, 362