• Reading: Chapers 1 to 4
• Lecture notes 1 in PDF. Concepts: Discrete Probability Spaces, the Union Bound, the Probabilistic Method. Examples: Sperner's Lemma, Ramsey Numbers, d-disjunct Matrices
• Lecture notes 2 in PDF: Conditional Probability, Independence, Randomized Algorithms, Random Variables, Expectation and its Linearity, Conditional Expectation, Law of Total Probability.
• Lecture notes 3 in PDF: Concepts: (Co)Variance, Coupon Collector Problem, Concentration/Tail Inequalities (Markov, Chebyshev, Chernoff). Examples: Probabilistic Packet Marking, Sampling and Estimation, Randomized Routing on the Hypercube.

• Probability Spaces and Events
• Independence and Conditional Probability
• Random variables, Expectation, Linearity of Expectation, Variance
• Conditional Distributions, Conditional Expectation
• Basic distributions: Bernoulli, Binomial, Geometric
• Derandomization with the method of conditional probability
• Moments
• Tail bounds: Markov, Chebyshev, Chernoff
• Sample mean, median, sampling and estimation

• Extremal Combinatorics: 
• Ramsey Numbers
• Sperner Lemma
• Randomized Algorithms:
• Randomized Min-Cut
• Randomized Quicksort Analsysis
• Max-E3SAT
• Networking:
  
• Probabilistic Packet Marking
• Permuation routing on the Hypercube
• Coding Theory and Group Testing:
• Expander code
• Disjunct matrices

• Reading: Chaper 6
• Lecture notes 4 in PDF. Concepts: the union bound technique. Examples: "nice" tournaments, 2-coloring of uniform hypergraphs, expander codes.
• Lecture notes 5 in PDF. Concepts: the argument from expectation. Exampes: large cuts in graphs, linear combinations of unit vectors, unbalancing lights.
• Lecture notes 6 in PDF. Concepts: the alteration technique. Examples: Independent Sets, Dominating Sets, 2-coloring of uniform hypergraphs (revisited).
• Lecture notes 7 in PDF. Concepts: the second moment method, the G(n,p) random graph model. Examples: distinct subset sums, 4-clique property
• Lecture notes 8 in PDF. Concepts: Lovasz local lemma. Examples: hypergraph coloring (again), k-SAT, edge-disjoint paths.

Concepts

• Union bound
  
• Argument from expectation
• Alterations
• Second moment method; also the G(n,p) random graph model
• Lovasz local lemma

• Tournaments
• 2-coloring of uniform hypergraphs
• Expander codes
• Finding large cuts, Max SAT
• Linear combinations of unit vectors
• Unbalancing lights
• Dominating sets
  
• Independent Sets
• Distinct subset sums
• The G(n,p) random graph model and 4-clique property
• Edge-disjoint paths
• k-SAT

Tue Sep 30: No Class. Rosh Hashanah

Thu, Oct 02:
- Homework 3 due.
- Homework 4 out

• Reading: Chaper 7, 9, Part of Chapter 11
• Lecture notes 9 in PDF. Concepts: linear programming (LP), integer linear programming (ILP), duality, complementary slackness. Examples: maxflow, mincut, multiway cut, set cover, vertex cover, max-SAT.
• Lecture notes 10 in PDF. Concepts: randomized rounding. Example: maxflow-mincut theorem, multiway cut.
• Lecture notes 11 in PDF. Concepts: randomized rounding. Examples: weighted set cover.
• Lecture notes 12 in PDF. Concepts: randomized rounding, derandomization. Examples: satisfiability problems.
  
• Lecture notes 13 in PDF. Concepts: semidefinite programming, randomized rounding. Example: max-cut. See also the following notes for more on Semidefinite programming and Max-Cut.
• Laci Lovasz, "Semidefinite programs and combinatorial optimization", [ postscript ]
• Uriel Feige, "The use of semidefinite programming in approximation algorithms", [ ppt ]
• Uri Zwick, "Semidefinite programming-based approximation algorithms", [ ppt ]
• M. Goemans, "MAXCUT, SDP-based 0.878-approximation algorithm", [ pdf ]
• Prasad Raghavendra, "Optimal Algorithms and Inapproximability Results for Every CSP?", [ paper in pdf | presentation in ppt ]
• Lecture x in PDF: Introduction to data streams (if time allows! Well, it looks like we'll have to skip this, along with approximate sampling and counting.)

Concepts

• Approximation Algorithms for Combinatorial Optimization Problems
• Linear Programming, Integer Programming
• Semidefinite programming
• Randomized Rounding
• Basic Monte Carlo Method
• Complexity of Counting (#P, #P-Completeness)
• Approximate Counting
• "Equivalence" between Approximate Counting and Uniform Sampling
• Random walks on graphs, Eigenvalue Connection, Expanders
• Saving Random Bits, Derandomization

• randomized rounding
• satisfiability problems
• covering problems
• cut problems
• randomized algorithm for s,t-connectivity in log-space
• randomized algorithm for 2-sat
• saving random bits
• #DNF
• 01-Permanent
• Network Reliability
• (May be) Volume Estimation
• Data Streams:
• Frequency Moment Estimation
• Hot Item Estimation

Tue, Oct 28:
- Homework 3 due.
- Homework 4 out

• Lecture 14 in PDF: Introduction to Computational Learning Theory and the Probably Approximately Correct model.
• David Aldous, Umesh V. Vazirani: A Markovian Extension of Valiant's Learning Model, Inf. Comput. 117(2): 181-186 (1995)
• Lecture 15 in PDF: Occam's Razor, VC dimension, Sauer's Lemma
• A blog post from Tim Gowers on Sauer's Lemma using the dimension-argument
• Ming Li, John Tromp, Paul M. B. Vitányi: Sharpening Occam's razor. Inf. Process. Lett. 85(5): 267-274 (2003).
• Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, Manfred K. Warmuth: Occam's Razor. Inf. Process. Lett. 24(6): 377-380 (1987). [This is the original Occam's Razor paper]
• Introduction. PAC model and Occam's razor, Lecture Notes by Avrim Blum (CMU) from Spring 2007
• Lecture 16 in PDF: Dealing with Noise and Inconsistent Hypothesis
• See also lecture notes seven and eight from Bob Schapire's course at Princeton (Spring 08).
• Lecture 17 in PDF: Online Learning and Learning from Expert Advice
• Avrim Blum's survey
• Blum, A. and Y. Mansour (2007) Learning,Regret Minimization, and Equilibria. In Algorithmic Game Theory (eds. N. Nisan, T. Roughgarden, E. Tardos, and V. Vazirani),Cambridge University Press.
• A Mind Reader Game
• Yoav Freund and Robert E. Schapire, Adaptive Game Playing Using Multiplicative Weights, Games and Economics Behaviors, 29: 79-103, 1999
• Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D. P., Schapire, R. E., and Warmuth, M. K. 1997. How to use expert advice. J. ACM 44, 3 (May. 1997), 427-485.
• Frans M. J. Willems, Yuri M. Shtarkov, Tjalling J. Tjalkens: The context-tree weighting method: basic properties. IEEE Transactions on Information Theory 41(3): 653-664 (1995). [1996 Paper Award of the IEEE Information Theory Society]
• Littlestone, N. 1988. Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm. Mach. Learn. 2, 4 (Apr. 1988), 285-318. [The Winnow paper]
• Lecture 18 in PDF: Boosting and Adaboost.
• We probably won't have time to get to Lecture 18. There are so many things to talk about, so little time.
• Lecture 19 in PDF: Support Vector Machines
• We certainly will not have time to do this. Wait for the next incarnation of the course.

Friday Dec 05 is the last day of classes.

DONE!