|1. Aug 26
Introduction to Probability Theory, Probabilistic Method, and
- Reading: Chapers 1 to 4
- Lecture notes 1 in PDF.
Concepts: Discrete Probability Spaces, the Union Bound,
the Probabilistic Method.
Examples: Sperner's Lemma, Ramsey Numbers, d-disjunct Matrices
- Lecture notes 2 in PDF:
Conditional Probability, Independence,
Randomized Algorithms, Random Variables, Expectation and
its Linearity, Conditional Expectation, Law of Total Probability.
- Lecture notes 3 in PDF:
Concepts: (Co)Variance, Coupon Collector Problem,
Concentration/Tail Inequalities (Markov, Chebyshev,
Examples: Probabilistic Packet Marking, Sampling and Estimation,
Randomized Routing on the Hypercube.
- Probability Spaces and Events
- Independence and Conditional Probability
- Random variables, Expectation, Linearity of
- Conditional Distributions, Conditional Expectation
- Basic distributions: Bernoulli, Binomial, Geometric
- Derandomization with the method of conditional
- Tail bounds: Markov, Chebyshev, Chernoff
- Sample mean, median, sampling and estimation
- Ramsey Numbers
- Sperner Lemma
- Randomized Algorithms:
- Randomized Min-Cut
- Randomized Quicksort Analsysis
- Probabilistic Packet Marking
- Permuation routing on the Hypercube
- Coding Theory
and Group Testing:
- Expander code
- Disjunct matrices
|Tue Aug 26
- Homework 1 out. Elementary practice exercises on
discrete probabilities and the probabilistic method
|2. Sep 02
|3. Sep 09
||Tue, Sep 09
- Homework 1 due.
Homework 2 out
|4. Sep 16
- Reading: Chaper 6
- Lecture notes 4 in PDF. Concepts: the union bound technique. Examples: "nice" tournaments, 2-coloring of uniform hypergraphs, expander codes.
- Lecture notes 5 in PDF. Concepts: the argument from expectation.
Exampes: large cuts in graphs, linear combinations of unit vectors,
- Lecture notes 6 in PDF. Concepts: the alteration technique.
Examples: Independent Sets, Dominating Sets, 2-coloring of uniform
- Lecture notes 7 in PDF. Concepts: the second moment method, the
G(n,p) random graph model. Examples: distinct subset sums, 4-clique
- Lecture notes 8 in PDF. Concepts: Lovasz local lemma. Examples: hypergraph coloring (again), k-SAT, edge-disjoint paths.
- Union bound
- Argument from expectation
- Second moment method; also the G(n,p) random graph model
- Lovasz local lemma
- 2-coloring of uniform hypergraphs
- Expander codes
- Finding large cuts, Max SAT
- Linear combinations of unit vectors
- Unbalancing lights
- Dominating sets
- Independent Sets
- Distinct subset sums
- The G(n,p) random graph model and 4-clique property
- Edge-disjoint paths
|5. Sep 23
|6. Sep 30
Tue Sep 30: No Class. Rosh Hashanah
Thu, Oct 02:
- Homework 3 due.
Homework 4 out
|7. Oct 07
||Thu, Oct 09: No Class. Yom Kippur
|8. Oct 14
- Reading: Chaper 7, 9, Part of Chapter 11
- Lecture notes 9 in PDF.
Concepts: linear programming (LP), integer linear programming (ILP),
duality, complementary slackness. Examples: maxflow, mincut, multiway
cut, set cover, vertex cover, max-SAT.
- Lecture notes 10 in PDF. Concepts: randomized rounding. Example: maxflow-mincut theorem, multiway cut.
- Lecture notes 11 in PDF. Concepts: randomized rounding. Examples: weighted set cover.
- Lecture notes 12 in PDF. Concepts: randomized rounding, derandomization. Examples: satisfiability problems.
- Lecture notes 13 in PDF. Concepts: semidefinite programming,
randomized rounding. Example: max-cut. See also the following notes for more on Semidefinite programming and Max-Cut.
- Laci Lovasz, "Semidefinite programs and combinatorial optimization", [ postscript ]
- Uriel Feige, "The use of semidefinite programming in approximation algorithms", [ ppt ]
- Uri Zwick, "Semidefinite programming-based approximation algorithms", [ ppt ]
- M. Goemans, "MAXCUT, SDP-based 0.878-approximation algorithm", [ pdf ]
- Prasad Raghavendra, "Optimal Algorithms and Inapproximability Results for Every CSP?", [ paper in pdf | presentation in ppt ]
- Lecture x in PDF: Introduction to data streams (if time allows! Well, it looks like we'll have to skip this, along with approximate sampling and counting.)
- Approximation Algorithms for Combinatorial Optimization Problems
- Linear Programming, Integer Programming
- Semidefinite programming
- Randomized Rounding
- Basic Monte Carlo Method
- Complexity of Counting (#P, #P-Completeness)
- Approximate Counting
- "Equivalence" between Approximate Counting and
- Random walks on graphs, Eigenvalue Connection, Expanders
- Saving Random Bits, Derandomization
- randomized rounding
- satisfiability problems
- covering problems
- cut problems
- randomized algorithm for s,t-connectivity in log-space
- randomized algorithm for 2-sat
- saving random bits
- Network Reliability
- (May be) Volume Estimation
- Data Streams:
- Frequency Moment Estimation
- Hot Item Estimation
|9. Oct 21
|10. Oct 28
Tue, Oct 28:
- Homework 3 due.
Homework 4 out
|11. Nov 04
|12. Nov 11
||Computational Learning Theory
- Lecture 14 in PDF: Introduction to Computational Learning Theory
and the Probably Approximately Correct model.
- Lecture 15 in PDF: Occam's Razor, VC dimension, Sauer's Lemma
- A blog post from Tim Gowers on Sauer's Lemma using the dimension-argument
- Ming Li, John Tromp, Paul M. B. Vitányi: Sharpening Occam's razor. Inf. Process. Lett. 85(5): 267-274 (2003).
- Anselm Blumer,
Manfred K. Warmuth:
Inf. Process. Lett. 24(6): 377-380 (1987). [This is the original Occam's Razor paper]
- Introduction. PAC model and Occam's razor, Lecture Notes by Avrim Blum (CMU) from Spring 2007
- Lecture 16 in PDF: Dealing with Noise and Inconsistent Hypothesis
- See also lecture notes seven and eight from Bob Schapire's course at Princeton (Spring 08).
- Lecture 17 in PDF: Online Learning and Learning from Expert Advice
- Avrim Blum's survey
- A Mind Reader Game
- Yoav Freund and Robert E. Schapire, Adaptive Game Playing Using Multiplicative Weights, Games and Economics Behaviors, 29: 79-103, 1999
- Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D. P., Schapire,
R. E., and Warmuth, M. K. 1997. How to use expert advice. J. ACM 44, 3 (May. 1997), 427-485.
- Frans M. J. Willems,
Yuri M. Shtarkov,
Tjalling J. Tjalkens:
The context-tree weighting method: basic properties.
IEEE Transactions on Information Theory 41(3): 653-664 (1995). [1996
Award of the IEEE Information Theory Society]
- Littlestone, N. 1988. Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm. Mach. Learn. 2, 4 (Apr. 1988), 285-318. [The Winnow paper]
- Lecture 18 in PDF: Boosting and Adaboost.
- We probably won't have time to get to Lecture 18. There are so many things to talk about, so little time.
- Lecture 19 in PDF: Support Vector Machines
- We certainly will not have time to do this. Wait for the next incarnation of the course.
|Thu, Nov 13:
- Homework 4 due.
Homework 5 out
|13. Nov 18
|14. Nov 25
||Thu, Nov 27: No Class. Thanks Giving!
|15. Dec 02
Thu, Dec 04:
- Homework 5 due.
Friday Dec 05 is the last day of
|16. Dec 09