COMS4236: Introduction to

Computational Complexity
Summer 2014
Mihalis Yannakakis
Lecture 1

Course Information
Lectures:
Monday, Wednesday 2:40-3:55
535 Mudd
Web site:
Courseworks
TA: Dimitris Paparas

Course Work
Homeworks, Final
Policies
Late Homework: 10% penalty per late day
Lowest hw does not count in grade
Collaboration policy
Grading: Homeworks 60%, Final 40%

Homework 0: Info about yourself due Monday 1/27

Textbook
Required:
Computational Complexity, by C. Papadimitriou
Other:
-Computational Complexity: A Modern Approach, by S. Arora,
B. Barak
-Introduction to the Theory of Computation, by M. Sipser
-Introduction to Automata Theory, Languages and
Computation, by Hopcroft, Motwani, Ullman
-Computers and Intractability: A Guide to the Theory of NPcompleteness, by Garey, Johnson

Problems, Algorithms, Models

Problem: What is to be computed

Model: Who computes it

Algorithm: How to compute it

Problems
Problem: What needs to be computed
Input

Output

Example: Graph Reachability problem

input = (graph G, nodes s,t)
output = Yes if G has path from s to t, No otherwise
Factoring problem
input = positive integer
output = list of its prime factors

Set of instances (possible inputs)

For each instance a set of acceptable (correct) outputs
Problem: Given an instance, compute an acceptable output

Algorithms, Models
Model of Computation
eg. sequential (von Neumann) computer
parallel computer
quantum computer
Boolean circuit
..

Algorithm: Method for solving the problem on the

model of computation

Resources & Complexity

Computational resources: Time, Space, Processors
(for parallel computation), Communication (for distributed
computation), Area (for a VLSI implementation), .

R-Complexity of an algorithm: amount of resource R

that it uses (R=time, space,)

R-Complexity of a problem: lowest complexity of an

algorithm that solves the problem wrt resource R of
interest (time, space, etc)

Goals of Complexity Theory

1. Characterize the complexity of problems
and classify them into complexity classes:
classes of problems that can be solved within
certain complexity bounds
- Problems are studied not only in isolation but
also as they relate to each other
Central notions of
- Reduction between problems
- Completeness for a class

Goals of Complexity Theory ctd.

2. Relations between
- resources (time, space etc.)
- complexity classes
- models of computation

Power and limitations of different

modes/paradigms of computation, eg.
nondeterminism, randomization, parallelism etc.

Laws of Computation

Focus of this course

Basic Resources: Time, Space mainly
Basic Models: Turing machines mainly (and
briefly RAM, Boolean combinational circuits)
Deterministic, nondeterministic, probabilistic
Problems: Various types: Decision, search,
optimization, games, counting problems

Graph Reachability
Input: Directed Graph G=(N,E), nodes s, t
Question: Is there a path in G from s to t?
i.e. Output= Yes (if there is a path) or No (if
there is no path)
Decision problem: output = Yes/No
Graph may be given by its adjacency matrix A or
by adjacency lists: a list Adj[u] for each u in N
that lists all nodes v to which u has an edge

Example
1

4
5

0
0
0
0
0

1
0
0
1
0

0
1
0
0
0

1
0
1
0
0

0
0
1
1
0

3
Graph

Adjacency Matrix
A[i,j]=1 iff ij

Node 1 can reach node 5

Node 4 cannot reach node 1

Graph Search
for each vN-{s} do {mark[v]=0}
mark[s]=1 ;
Q = {s}
while Q do
{ u= select and delete a node from Q
for each v N do
if (A[u,v]=1 and mark[v]=0) then
{mark[v]=1; Q=Q{v} }
}
If mark[t]=1 then print Yes else print No
If Q implemented as a queue (i.e. we delete from front, insert
into back), then Breadth-First-Search

Time Complexity
For every input graph with n nodes, the time complexity is
proportional to n: Time=O(n)
Some important points to note:
Complexity is expressed as a function of a parameter n
measuring the size of the input (here the number of
nodes)
Worst-case complexity: maximum over all inputs of size n
Expressed complexity in terms of its rate of growth:
asymptotic notation O(.) suppresses constant factors

Asymptotic Notation Review:

Theta, Big-Oh, Omega
Big-Oh: O( g (n)) { f (n) | constant c 0 and n0 s.t. n n0 :
(Order)
0 f ( n) c g ( n) }
Convention : We usually write f (n) O( g (n))
Caution: = here denotes membership, not equality

Example:
cg(n)

f(n)

5n O(n 2 )

but not vice-versa

n0

Asymptotic Notations:
Theta, Big-Oh, Omega
Omega:

( g (n)) { f (n) | constant c 0 and n0 s.t. n n0 :

c g ( n) f ( n) }
Convention : We usually write f (n) ( g (n))

f(n)

Example:
5n 2 (n)

c g(n)

n0

but not vice-versa

Asymptotic Notations:
Theta, Big-Oh, Omega
Theta:

( g (n)) { f (n) | constants c1 , c2 0 and n0 s.t. n n0 :

c1 g (n) f (n) c2 g (n) }
Convention : We usually write f (n) ( g (n))

c2 g (n)

f(n)

c1 g (n)

n0

Asymptotic Notations:
little-oh, little-omega
little-oh:

o( g (n)) { f (n) | constant c 0 n0 s.t. n n0 :

0 f ( n) c g ( n) }

little-omega: ( g (n)) { f (n) | constant c 0 n0 s.t. n n0 :

0 c g ( n) f ( n) }

f(n)=o(g(n) means that for large n, function f is smaller

than any constant fraction of g
f(n)=(g(n) means that for large n, function f is larger than
any constant multiple of g, i.e., g=o(f(n))
Example:

5n o(n 2 ),

5n 2 (n)

Asymptotic Notations Summary

Notation

Ratio f(n)/g(n) for large n

f (n) ( g (n))

f ( n) / g ( n)

f (n) ( g (n))

c f ( n) / g ( n)

f (n) ( g (n))

c1 f (n) / g (n) c 2

f (n) O( g (n))

f ( n) / g ( n) c

f (n) o( g (n))

f ( n) / g ( n) 0

Examples
For every polynomial p(n) of degree d with a positive
leading term, p(n)=(nd). For example, 4n-3n+7=(n)
Every constant is O(1), for example 1000=O(1)
logan = (logcn) for any two constant bases a, c,
For example log10n = (log2n)
Every constant power of logn is little-oh of every
10
constant root of n, for example (log n) o( 5 n )
Every power of n is little-oh of every exponential function
10
n
n

o
(1.1
)
of n, for example

Some common functions

log n log n log n
2

n n n log n n n
2

(polynomial)

2 3 n !
n

(exponential)