A Postmortem of the Last Decade
and
Some Directions for the Future
Vijay V. Vazirani
Georgia Tech
Although this may seem a paradox,
all exact science is dominated by
the idea of approximation.
Bertrand Russell (1872-1970)
Exact algorithms have been studied
intensively for over four decades,
and yet basic insights are still being obtained.
Since polynomial time solvability is the exception
rather than the rule, it is only reasonable
to expect the theory of approximation algorithms
to grow considerably over the years.
Beyond the list …

Unique Games Conjecture

Simpler proof of PCP Theorem

Online algorithms for AdWords problem
Beyond the list …

Unique Games Conjecture

Simpler proof of PCP Theorem

Online algorithms for AdWords problem

Integrality gaps vs approximability
Raghevendra, 2008: Assuming UGC,
for every constrained satisfaction problem:

Can achieve approximation factor
= integrality gap of “standard SDP”

NP-hard to approximate better.
Future Directions

Status of UGC

Raghavendra-type results for LP-relaxations

Randomized dual growth in
primal-dual algorithms
Approximability: sharp thresholds
For a natural problem:

Can achieve approximation factor  (n) in P.

If we can achieve  (n)  (n) in P
=> complexity-theoretic disaster
Conjecture

There is a natural problem
having sharp thresholds
1 (n)   2 (n)  ...   k (n)
w.r.t. time classes P T1 (n)  T2 (n)  ... Tk (n)
Group Steiner Tree Problem

Chekuri & Pal, 2005:
log
2
n factor algorithm in time
2^(2^(log n

O()
))
Halperin & Krauthgamer, 2003:
time = 2^(2^(log n o() ))
 subexponential algorithm for 3SAT
What lies at the core of
approximation algorithms?
What lies at the core of
approximation algorithms?
Combinatorial optimization!
Combinatorial optimization

Central problems have LP-relaxations
that always have integer optimal solutions!
ILP: Integral LP
Combinatorial optimization

Central problems have LP-relaxations
that always have integer optimal solutions!
ILP: Integral LP
i.e., it “behaves” like an IP!
Massive accident!
Cornerstone problems in P





Matching (general graph)
Network flow
Shortest paths
Minimum spanning tree
Minimum branching
Is combinatorial optimization
relevant today?

Why design combinatorial algorithms,
especially today that LP-solvers are so fast?
Combinatorial algorithms

Very rich theory

Gave field of algorithms some of its formative
and fundamental notions, e.g. P

Preferable in applications, since efficient
and malleable.
 Helped
spawn off algorithmic areas,
e.g., approximation algorithms and
parallel algorithms.
Combinatorial optimization studied:
Problems
admitting ILPs
Approximation algorithms studied:
Problems admitting
LP-relaxations with
bounded integrality gaps
Problems admitting
LP-relaxations with
bounded integrality gaps
Problems
admitting ILPs
Rational convex program

A nonlinear convex program that
always has a rational solution (if feasible),
using polynomially many bits,
if all parameters are rational.
Rational convex program


Always has a rational solution (if feasible)
using polynomially many bits,
if all parameters are rational.
i.e., it “behaves” like an LP!
Rational convex program

Always has a rational solution (if feasible)
using polynomially many bits,
if all parameters are rational.

i.e., it “behaves” like an LP!

Do they exist??
KKT optimality conditions
 f0 (x) 
y
i
i
yi  0
fi '(x) 
z
j
aj
j
for 1i  m
yi  0  fi (x)  0 for 1i  m
fi (x)  0
for 1 i  m
a x  bj
for 1 j  p
T
j
Possible RCPs
Pick fi ' s linear, and
f0 quadratic or logarithmic.
Quadratic RCPs
fo (x)  x P x  q x
T
T
convexity requires:
 f0 f  0, i.e., P f  0
2
Two opportunities for RCPs:

Program A: Combinatorial, polynomial time
(strongly poly.) algorithm

Program B: Polynomial time (strongly poly.)
algorithm, given LP-oracle.
Combinatorial Algorithms

Helgason, Kennington & Lall, 1980
 Single

constraint
Minoux, 1984
 Minimum

quadratic cost flow
Frank & Karzanov, 1992
 Closest
point from origin to bipartite perfect
matching polytope.

Karzanov & McCormick, 1997
 Any
totally unimodular matrix.
Ben-Tal & Nemirovski, 1999

Polyhedral approximation of second-order cone

Main technique: Solves any quadratic RCP
in polynomial time, given an LP-oracle.
Ben-Tal & Nemirovski, 1999

Polyhedral approximation of second-order cone

Main technique: Solves any quadratic RCP
in polynomial time, given an LP-oracle.

Strongly polynomial algorithm?
Logarithmic RCPs
f0 (x)    mi log (ui (x))
i
where mi  0 and ui (x) is linear in x.
Logarithmic RCPs
f0 (x)    mi log (ui (x))
i
where mi  0 and ui (x) is linear in x.

Rationality is the exception to the rule,
and needs to be established piece-meal.
Logarithmic RCPs
f0 (x)    mi log (ui (x))
i
where mi  0 and ui (x) is linear in x.

Optimal solutions to such RCPs capture
equilibria for various market models!
Arrow-Debreu Theorem, 1954

Celebrated theorem in Mathematical Economics

Established existence of market equilibrium under
very general conditions using a deep theorem from
topology - Kakutani fixed point theorem.
Arrow-Debreu Theorem, 1954

Celebrated theorem in Mathematical Economics

Established existence of market equilibrium under
very general conditions using a theorem from
topology - Kakutani fixed point theorem.

Highly non-constructive!
Arrow-Debreu Theorem, 1954

Celebrated theorem in Mathematical Economics

Established existence of market equilibrium under
very general conditions using a theorem from
topology - Kakutani fixed point theorem.

Continuous, quasiconcave,
satisfying non-satiation.
Complexity-theoretic question

For “reasonable” utility fns.,
can market equilibrium be computed in P?

If not, what is its complexity?
Short summary

So far, all markets
whose equilibria can be computed efficiently
admit convex or quasiconvex programs,
many of which are RCPs!
Combinatorial Algorithm for
Linear Case of Fisher’s Model

Devanur, Papadimitriou, Saberi & V., 2002
By extending primal-dual paradigm to setting of
convex programs & KKT conditions
Eisenberg-Gale Program, 1959
max  mi log vi
i
s.t.
i : vi   j u ij x ij
 x 1
ij : x  0
j :
ij
i
ij
Eisenberg-Gale Program, 1959
max  mi log vi
i
s.t.
i : vi   j u ij x ij
 x 1
ij : x  0
j :
ij
i
ij
prices pj
KKT conditions
1). j : p j  0
2). j : p j  0   i xij  1
3). i, j :
uij
pj

vi
m(i)
4). i, j : xij  0 
uij
pj

vi
m(i)


x
u
ij
ij
j
m(i)
Proof of rationality

Guess positive allocation variables (say k).

Substitute 1/pj by a new variable.

LP with (k + g) equations and
non-negativity constraint for each variable.
Auction for Google’s TV ads
N. Nisan et. al, 2009:

Used market equilibrium based approach.

Combinatorial algorithms for linear case
provided “inspiration”.
Long-standing open problem

Complexity of finding an equilibrium for
Fisher and Arrow-Debreu models under
separable, piecewise-linear, concave utilities?
Piecewise linear, concave
utility
Additively separable
over goods
amount of j
Long-standing open problem

Complexity of finding an equilibrium for
Fisher and Arrow-Debreu models under
separable, piecewise-linear, concave utilities?

Equilibrium is rational!
Markets with separable, plc utilities
are PPAD-complete

Chen, Dai, Du, Teng, 2009

Chen & Teng, 2009

V. & Yannakakis, 2009
Markets with separable, plc utilities
are PPAD-complete

Chen, Dai, Du, Teng, 2009

Chen & Teng, 2009

V. & Yannakakis, 2009
(Building on combinatorial insights from DPSV)
Theorem (V., 2002):
 Generalized linear Fisher market to
Spending constraint utilities.

Polynomial time algorithm for
computing equilibrium.
Is there a convex program for this model?

“We believe the answer to this question should
be ‘yes’. In our experience, non-trivial
polynomial time algorithms for problems are
rare and happen for a good reason – a deep
mathematical structure intimately connected to
the problem.”
Eisenberg-Gale Markets
Jain & V., 2007
Price disc.
Market
Goel & V.
EG[2] Markets
Chakrabarty, Devanur & V.
2008
Nash Bargaining
V., 2008
EG convex program
=
Spending constraint
market
V., 2005
Devanur’s program
V., 2010: Assuming perfect price
discrimination, can handle:

Continuously differentiable, quasiconcave
(non-separable) utilities, satisfying non-satiation.
V., 2010:

Continuously differentiable, quasiconcave
(non-separable) utilities, satisfying non-satiation.
Compare with Arrow-Debreu utilities!!
continuous, quasiconcave, satisfying non-satiation.

A new development

Orlin, 2009: Strongly polynomial algorithm
for Fisher’s linear case, using scaling.

Open: For rest
Are there other classes of RCPs?

Sturmfels & Uhler, 2009:
S f  0 n  n, sample covariance matrix
G  ([n], E) chordal graph
Then the following is an RCP:
min log det 
s.t.  ij  Sij (i, j)E or i  j
Eisenberg-Gale Markets
Jain & V., 2007
Price disc.
Market
Goel & V.
EG[2] Markets
Chakrabarty, Devanur & V.
2008
Nash Bargaining
V., 2008
EG convex program
=
Spending constraint
market
V., 2005
Devanur’s program
Building on

Karzanov & McCormick, 1997:
Combinatorial algorithm for min cost flow
under concave cost functions on edges.
Eisenberg-Gale Markets
Jain & V., 2007
Price disc.
Market
Goel & V.
EG[2] Markets
Chakrabarty, Devanur & V.
2008
Nash Bargaining
V., 2008
EG convex program
=
Spending constraint
market
V., 2005
Devanur’s program
Rational (combinatorial)
approximations
to convex programs
Problems
admitting RCPs
Eisenberg-Gale Markets
Jain & V., 2007
Price disc.
Market
Goel & V.
EG[2] Markets
Chakrabarty, Devanur & V.
2008
Nash Bargaining
V., 2008
EG convex program
=
Spending constraint
market
V., 2005
Devanur’s program
Rational (combinatorial)
approximations
to convex programs
Problems
admitting RCPs