Week 
Topics 
Notable Events 
1. Aug 26 
A Gentle
Introduction to Probability Theory, Probabilistic Method, and
Randomized Algorithms
 Reading: Chapers 1 to 4
 Lecture notes 1 in PDF.
Concepts: Discrete Probability Spaces, the Union Bound,
the Probabilistic Method.
Examples: Sperner's Lemma, Ramsey Numbers, ddisjunct Matrices
 Lecture notes 2 in PDF:
Conditional Probability, Independence,
Randomized Algorithms, Random Variables, Expectation and
its Linearity, Conditional Expectation, Law of Total Probability.
 Lecture notes 3 in PDF:
Concepts: (Co)Variance, Coupon Collector Problem,
Concentration/Tail Inequalities (Markov, Chebyshev,
Chernoff).
Examples: Probabilistic Packet Marking, Sampling and Estimation,
Randomized Routing on the Hypercube.
Concepts:
 Probability Spaces and Events
 Independence and Conditional Probability
 Random variables, Expectation, Linearity of
Expectation, Variance
 Conditional Distributions, Conditional Expectation
 Basic distributions: Bernoulli, Binomial, Geometric
 Derandomization with the method of conditional
probability
 Moments
 Tail bounds: Markov, Chebyshev, Chernoff
 Sample mean, median, sampling and estimation
Examples:
 Extremal
Combinatorics:
 Ramsey Numbers
 Sperner Lemma
 Randomized Algorithms:
 Randomized MinCut
 Randomized Quicksort Analsysis
 MaxE3SAT
 Networking:
 Probabilistic Packet Marking
 Permuation routing on the Hypercube
 Coding Theory
and Group Testing:
 Expander code
 Disjunct matrices

Tue Aug 26
 Homework 1 out. Elementary practice exercises on
discrete probabilities and the probabilistic method 
2. Sep 02 

3. Sep 09 
Tue, Sep 09
 Homework 1 due.

Homework 2 out

4. Sep 16 
The
Probabilistic Method
 Reading: Chaper 6
 Lecture notes 4 in PDF. Concepts: the union bound technique. Examples: "nice" tournaments, 2coloring of uniform hypergraphs, expander codes.
 Lecture notes 5 in PDF. Concepts: the argument from expectation.
Exampes: large cuts in graphs, linear combinations of unit vectors,
unbalancing lights.
 Lecture notes 6 in PDF. Concepts: the alteration technique.
Examples: Independent Sets, Dominating Sets, 2coloring of uniform
hypergraphs (revisited).
 Lecture notes 7 in PDF. Concepts: the second moment method, the
G(n,p) random graph model. Examples: distinct subset sums, 4clique
property
 Lecture notes 8 in PDF. Concepts: Lovasz local lemma. Examples: hypergraph coloring (again), kSAT, edgedisjoint paths.
Concepts
 Union bound
 Argument from expectation
 Alterations
 Second moment method; also the G(n,p) random graph model
 Lovasz local lemma
Examples
 Tournaments
 2coloring of uniform hypergraphs
 Expander codes
 Finding large cuts, Max SAT
 Linear combinations of unit vectors
 Unbalancing lights
 Dominating sets
 Independent Sets
 Distinct subset sums
 The G(n,p) random graph model and 4clique property
 Edgedisjoint paths
 kSAT


5. Sep 23 

6. Sep 30 
Tue Sep 30: No Class. Rosh Hashanah
Thu, Oct 02:
 Homework 3 due.

Homework 4 out

7. Oct 07 
Thu, Oct 09: No Class. Yom Kippur 
8. Oct 14 
Randomized Algorithms
 Reading: Chaper 7, 9, Part of Chapter 11
 Lecture notes 9 in PDF.
Concepts: linear programming (LP), integer linear programming (ILP),
duality, complementary slackness. Examples: maxflow, mincut, multiway
cut, set cover, vertex cover, maxSAT.
 Lecture notes 10 in PDF. Concepts: randomized rounding. Example: maxflowmincut theorem, multiway cut.
 Lecture notes 11 in PDF. Concepts: randomized rounding. Examples: weighted set cover.
 Lecture notes 12 in PDF. Concepts: randomized rounding, derandomization. Examples: satisfiability problems.
 Lecture notes 13 in PDF. Concepts: semidefinite programming,
randomized rounding. Example: maxcut. See also the following notes for more on Semidefinite programming and MaxCut.
 Laci Lovasz, "Semidefinite programs and combinatorial optimization", [ postscript ]
 Uriel Feige, "The use of semidefinite programming in approximation algorithms", [ ppt ]
 Uri Zwick, "Semidefinite programmingbased approximation algorithms", [ ppt ]
 M. Goemans, "MAXCUT, SDPbased 0.878approximation algorithm", [ pdf ]
 Prasad Raghavendra, "Optimal Algorithms and Inapproximability Results for Every CSP?", [ paper in pdf  presentation in ppt ]
 Lecture x in PDF: Introduction to data streams (if time allows! Well, it looks like we'll have to skip this, along with approximate sampling and counting.)
Concepts
 Approximation Algorithms for Combinatorial Optimization Problems
 Linear Programming, Integer Programming
 Semidefinite programming
 Randomized Rounding
 Basic Monte Carlo Method
 Complexity of Counting (#P, #PCompleteness)
 Approximate Counting
 "Equivalence" between Approximate Counting and
Uniform Sampling
 Random walks on graphs, Eigenvalue Connection, Expanders
 Saving Random Bits, Derandomization
Examples
 randomized rounding
 satisfiability problems
 covering problems
 cut problems
 randomized algorithm for s,tconnectivity in logspace
 randomized algorithm for 2sat
 saving random bits
 #DNF
 01Permanent
 Network Reliability
 (May be) Volume Estimation
 Data Streams:
 Frequency Moment Estimation
 Hot Item Estimation


9. Oct 21 

10. Oct 28 
Tue, Oct 28:
 Homework 3 due.

Homework 4 out

11. Nov 04 

12. Nov 11 
Computational Learning Theory
 Lecture 14 in PDF: Introduction to Computational Learning Theory
and the Probably Approximately Correct model.
 Lecture 15 in PDF: Occam's Razor, VC dimension, Sauer's Lemma
 A blog post from Tim Gowers on Sauer's Lemma using the dimensionargument
 Ming Li, John Tromp, Paul M. B. Vitányi: Sharpening Occam's razor. Inf. Process. Lett. 85(5): 267274 (2003).
 Anselm Blumer,
Andrzej Ehrenfeucht,
David Haussler,
Manfred K. Warmuth:
Occam's Razor.
Inf. Process. Lett. 24(6): 377380 (1987). [This is the original Occam's Razor paper]
 Introduction. PAC model and Occam's razor, Lecture Notes by Avrim Blum (CMU) from Spring 2007
 Lecture 16 in PDF: Dealing with Noise and Inconsistent Hypothesis
 See also lecture notes seven and eight from Bob Schapire's course at Princeton (Spring 08).
 Lecture 17 in PDF: Online Learning and Learning from Expert Advice
 Avrim Blum's survey

 A Mind Reader Game
 Yoav Freund and Robert E. Schapire, Adaptive Game Playing Using Multiplicative Weights, Games and Economics Behaviors, 29: 79103, 1999
 CesaBianchi, N., Freund, Y., Haussler, D., Helmbold, D. P., Schapire,
R. E., and Warmuth, M. K. 1997. How to use expert advice. J. ACM 44, 3 (May. 1997), 427485.
 Frans M. J. Willems,
Yuri M. Shtarkov,
Tjalling J. Tjalkens:
The contexttree weighting method: basic properties.
IEEE Transactions on Information Theory 41(3): 653664 (1995). [1996
Paper
Award of the IEEE Information Theory Society]
 Littlestone, N. 1988. Learning Quickly When Irrelevant Attributes Abound: A New LinearThreshold Algorithm. Mach. Learn. 2, 4 (Apr. 1988), 285318. [The Winnow paper]
 Lecture 18 in PDF: Boosting and Adaboost.
 We probably won't have time to get to Lecture 18. There are so many things to talk about, so little time.
 Lecture 19 in PDF: Support Vector Machines
 We certainly will not have time to do this. Wait for the next incarnation of the course.

Thu, Nov 13:
 Homework 4 due.

Homework 5 out 
13. Nov 18 

14. Nov 25 
Thu, Nov 27: No Class. Thanks Giving! 
15. Dec 02 
Thu, Dec 04:
 Homework 5 due.
Friday Dec 05 is the last day of
classes.

16. Dec 09 
DONE!

