CS380: Homework and Reading Assigments
This webpage serves as an outline of the topics covered in each class.
You will also find links to the handouts here.
Announcements (Homeworks, Midterms, Final, etc.)

Final will be held on Friday, 11/17/06, 10:15am12:15pm
in room 701610. Bring your lettersize handwritten "helpsheet".
We will have a review session
on Monday, 11/13/06, at 4pm, most probably in
70ICL5 (if the location changes, I'll post a notice on the lab door). Come
with questions.
The practice final is now available.
The practice final should give you an idea of the type of questions asked
at the final, and the level of difficulty. However, the real final might
deviate in topics and form from the practice final.
Here (pdf) or here (a funny format that as far as I know only
the Windows webbrowser can read but since it's smaller than the pdf, I decided to link to it as well)
is a practice final with solutions (keep in mind
that for many of the problems there are several correct answers,
e.g. your completely different automaton can correctly accept the same
language as my automaton).
The graduate theory class has their practice final available
here.
The list of topics to study for the final:

The focus will be on the topics discussed after Midterm2; however, you
should have a general understanding of all topics discussed in class.
For example, you should know that NFALambda is equivalent to (D)FA but
you will not be required to apply the subset sum construction.
Topics of the final exam:

Pushdown automata (Chapter 7.17.4, you will not be required to apply the construction from Chapter 7.4 but you should know that
it exists). Exercises 7.5, 7.13, 7.14.

Pumping lemma for contextfree languages (Chapter 8.1, you do not need to study Ogden's lemma).
Exercise 8.1

Closure properties of various language classes (regular, contextfree,
recursive, recursively enumerable, think also about closure properties
of P and NP  see "general advice" below),
equivalence of various computational models (variants of automata, grammars).
Scan the material covered in individual weeks (below) for references to the relevant Chapters in the book.
Check e.g. problem 2 on Midterm 2 and practice midterm 2,
problems 1 and 2 on the graduate practice final,
also review old homework assignments.

Turing machines (Chapters 9.19.5). E.g. problem 4 on the graduate practice final and
problem 4 on Homework 6 (if this problem is on the final, it will be
a simpler language than on the homework).

Chomsky hierachy. Given a language, which language classes does
the language belong in? Given a language, describe a device
accepting/generating the language. E.g. problem 4 on the practice final,
problem 4 on the graduate practice final, problem 1 on Midterm 1,
classify languages from Exercises 8.5, 8.8 (for each of these languages determine
if it is regular, contextfree, in P, in NP, recursive, or recursively enumerable).

Decidability (Chapter 11, except 11.4; Chapter 5.4; Chapter 8.3).
You will not be required to prove that
a problem is undecidable but you should intuitively understand
that a problem such as ACCEPTFINITE is more
difficult than ACCEPTLambda and thus undecidable. E.g. problem 2
on practice final, problems 2,3 on Homework 7.

Modeling. Define your own model based on a verbal description of the model's capabilities.
E.g. problem 1 on Homework 7, problem 5 on practice final.

P, NP, and NPcompleteness (Chapter 14). You should understand general concepts, e.g. what is required to prove
that a problem is in NP? In P? You will not be required to come up with a reduction on the final, however, you
should understand how reductions work (what does it mean to reduce a problem to another problem) and what implications
the reduction has in the given context. E.g. in class we proved that CNFSAT reduces to CLIQUE and CLIQUE reduces
to VERTEXCOVER. Since CNFSAT is NPcomplete, the other two problems must be NPcomplete as well. Check e.g. problem 1
on practice final, also Exercise 14.16.

General advice:
 Understand the solutions to all assigned homework problems and midterms.
 Solve both practice finals (in the graduate practice final,
you can leave the "Is NP closed under
complementation" box blank  the answer is "nobody knows").
 You do not need to study material that was not covered in class. Thus if I recommend a section in the book
to read, you may skip parts that have not been covered in class (such as Ogden's lemma). With the undecidability
of ACCEPTANCE and HALTING (Chapters 11.12)
I deviated from the book  if you understand your class notes, there is no need to try to understand the book's
notation.

Change in grading scale. I am relaxing the grading scale to: >88% A,
7788% B, 6677% C, 5566% D. It is possible that the grading scale will be
further adjusted but if it will, it will be only to your advantage (i.e. I
might decrease the 88% required for A to, say 87%, but I will definitely not
increase it to something more than 88%). Recall also that your better midterm counts for
20% of your final grade and the lower score midterm contributes 10% towards your final grade.
(MyCourses does not reflect this change in grading since it appears to be impossible to
set it up in such way. Thus, on MyCourses you'll see contribution of 15% for each midterm.)

Because of privacy issues, I cannot leave your graded homeworks in the tutoring center.
I placed the graded Hw5 in your folders  if you do not have a folder on the third floor of GCCIS and you
would like to see your homework before Monday, please email me.

Seventh homework is available, due Thursday 11/09/06, 11:00am (it will not be possible
to move this deadline to a later time).

Sixth homework is available, due Thursday 11/02/06.
Update: The deadline for HW6 has been extended until Friday 11/03/06,
11:59:59pm (slide your submission under my office door).

Fifth homework is available, due Thursday 10/26/06,
11:00am.
Erratum: In the question in problem 6, L was replaced with L^r. The
currently posted version is (I hope) correct.

Practice midterm 2 is now available.
(Note for those who downloaded the midterm on Monday evening: I added one more TRUE/FALSE
statement to Problem 2, as that part was worth 7 points and originally I had only 6 statements.
I also made the language in Problem 3 slightly more challenging  if you have already solved
the original problem, the correct solution is the same but the starting symbol is T.)
Here are the solutions and
this is a not very pretty picture solution of problem 4 (in
the second construction, red arcs correspond to transitions on 0 and blue arcs correspond to transitions on 1).
Note: I just (9:44pm, Wednesday) updated the picture solution, since I forgot to add a transition on 1 from
the initial to the original accepting state (this is the result of the lambdalambda1lambda path in the NFAlambda).
Thanks to Sam Chase for noticing this.

Midterm 2 will be held in class on Thursday, 10/19/06, 11:0011:50am.
The structure of the test will be similar to that of midterm 1. Study
Sections 4.2 through 6.6 (you may skip 6.3 as we did not cover that in class),
and the first half of Section 8.1 (pumping lemma and the examples
immediately following it  Example 8.18.4).
Here is the list of topics for midterm questions, along with relevant
exercises from the book. If the question asks for a standard construction
(discussed in the class/book), make sure to use that construction (the point
is to understand how the constructions work and that they work for
any input, e.g. we do not need to study what an NFA accepts to construct an
equivalent DFA).

Convert NFAlambda to NFA: 4.28, 4.29

Kleene's Theorem  create an NFAlambda for a language given by a regular
expression: 4.35 (review also 4.36)

Kleene's Theorem  give a regular expression representing the language given
by a DFA: 4.38 (do not panic: this construction is very time consuming 
if it appears on the midterm, the DFA will have only 2 states)

Minimize a DFA  5.16, 5.17, 5.19

Pumping lemma for regular languages: 5.23, 5.24 (one of the midterm questions
will be a proof of nonregularity by pumping lemma)

Given a language, construct a CFG generating it: 6.4, 6.9

Convert a CFG to an equivalent CFG in the Chomsky normal form: 6.39 (also
review 6.40)

Pumping lemma for contextfree languages: 8.1 (we will do more examples in
class next week)

Review closure properties, think about questions in Exercises 5.26, 5.27,
5.29, 5.30.

Fourth homework, due Thursday, 10/12/06, 11:00am.

Third homework, due Thursday, 10/05/06, 11:00am.
For some case of problem 1 you might find it useful
to use closure properties rather than
distinguishabilitytype proofs. The closure properties are not discussed in
the book (as far as I know), thus I am restating here a proof of nonregularity
by closure properties (done in class in the 4th week).
Claim: L1 = {x over the alphabet
{a,b}  number of a's in x is the same as number of b's}
is not regular.
Proof: We will prove this by contradiction, using the fact that the
language L2 = {a^i b^i  i>= 0} is not regular (we proved this in class by
distinguishability; it is also discussed in the book as Example 5.2,
using the equivalence classes terminology). Let L3 = {a^i b^j  i,j>=0}.
Notice that L1 intersection L3 equals L2. Suppose that L1 is regular. L3 is
clearly regular, described by the regular expression a^*b^*. Since the
intersection of two regular languages is regular, L1 intersection L3 must be
regular. But it is equal to L2 which we know is not regular. This is the
contradiction, thus L1 cannot be regular.
Recall that you cannot discuss the actual homework problems at the tutoring
center. However, you may find it useful to discuss some of the following
languages (and any others that you come across, e.g. in the exercises in
the book).
1. {0^i 1^j  i,j>=0, i!=j}
2. {0^i 1^i 0^j  i,j>=0}
3. {0^i 1^j 2^k  i,j,k>=0, i+j=k}
4. {0^i 1^j 0^i 1^j  i,j>=0}

Midterm 1 will be held in class on Thursday, 09/28/06, 1111:50am.
It will cover material discussed in weeks 13. The following exercises
represent the knowledge you should have gained during these weeks:
1.381.44, 1.47, 3.2, 3.4, 3.5, 3.8, 3.9, 3.11, 3.12,
3.18, 3.19, 3.20, 3.21, 3.23, 3.33, 4.4, 4.9a, 4.10. You should also have a
feeling for which languages are regular and which are not.
You are free to discuss with your classmates or at the tutoring center
the solutions to these exercises, past homework
problems, or any other issues you might have with the material in the book or
covered in classes.
Recall that you are allowed to prepare a single "helpsheet", a handwritten,
lettersized, single paged sheet with notes from class.
(No photocopies please.)

Second homework, due Monday, 09/25/06, 11:00am.
I am leaving town 09/21/06 shortly after the class, ask your questions before then.
I will answer email sporadically over the weekend, 09/2324, and I won't answer any
emails that come after 9pm on Sunday.

First homework, due Wednesday, 09/20/06, 11:00am. I will stop answering email on Tuesday, 09/19/06, at 4:30pm.
Week 1

Monday, Sept 4th

Tuesday, Sept 5th
 Covered in class: Section 2.4.2 (Recursive Definitions of Sets).
 Covered in class: Section 2.5 (Structural induction). In particular, Example 2.22 (language with more a's than b's).
 Skim Chapter 2.
 Prepare for Discrete Math Quiz, to be held on Thursday, September 7,
11:0011:30am, in class. There will be two problems.
The first will require basic discrete math reading/writing/reasoning skills.
The second will ask for a simple proof by mathematical induction.

Wednesday, Sept 6th
 Covered in class: handout (language with more a's than b's, part not covered in Example 2.22)

Thursday, Sept 7th
Week 2

Monday, Sept 11th

Covered in class: Exercises 1.38, 1.47(a), also 1.47(a) with intersection replaced by union (i.e. what is the relationship between L1.(L2 union L3) and L1.L2 union L1.L3 ?).

Do Exercise 1.47(bd). We will discuss it on Tuesday.

Briefly discussed in class: Regular languages (Chapter 3.1), to be continued on Tuesday.

Tuesday, Sept 12th

Covered in class: Regular languages and regular expressions (Chapter 3.1).

Wednesday, Sept 13th

Covered in class: Exercise 1.47(bd).

Covered in class: Introduction to finite automata (Chapter 3.2).

First homework, due next Wednesday, 09/20/06 at 11:00am.

Thursday, Sept 14th

Covered in class: Formal definition of finite automata (Chapter 3.3).

Covered in class: Operations on languages defined by finite automata (intersection, complement  Chapter 3.5).
Week 3

Monday, Sept 18th

Covered in class: Distinguishability (Chapter 3.4)

Second homework, due next Monday, 09/25/06 at 11:00am.

Tuesday, Sept 19th

Covered in class: Distinguishability revisited.

Covered in class: Operations on languages accepted by finite automata
(union, subtraction).

Briefly discussed in class: nondeterministic FA.

Wednesday, Sept 20th

Covered in class: Nondeterministic FA (Chapter 4.1).

Covered in class: Operations on languages accepted by NFAs (concatenation).

Thursday, Sept 21th

Covered in class: For every NFA there exists a DFA accepting the same language
(Chapter 4.1).
Week 4

Monday, Sept 25th

Covered in class: Recursive definitions revisited. A donothandin exercise:
For every string x over the alphabet {a,b} let n_a(x) denote the number of a's
in x. Give a recursive definition for n_a(x).

Covered in class: Closure properties of regular languages revisited. Closure
properties can be used to prove that certain languages are nonregular.
Think about closure properties of the class of nonregular languages.

Covered in class: Definition of NFAlambda, to be continued on Tuesday.

Midterm 1 will be held in class on Thursday, 09/28/06.

Tuesday, Sept 26th

Covered in class: NFAlambda, converting NFAlambda to NFA (Chapter 4.2).

Wednesday, Sept 27th

Review session for midterm 1.

Thursday, Sept 28th

Midterm 1. (The NFA from problem 3 is not included.) The tutoring center will be happy to discuss any questions you might have.

Homework 3, due Thursday, October 5th, 11:00am.
Week 5

Monday, Oct 2nd

Covered in class: Kleene's Theorem (Chapter 4.3), part 1 (constructing
NFAlambda for languages given by regular expressions). Started part 2
(constructing a regular expression for languages accepted by FA).

Change of rules for computing final grade: The lower midterm grade will
count 10% toward your final grade, the higher midterm grade will count 20%
towards your final grade. This rule cannot be implemented on mycourses, thus
there you will still see 15% contribution for each midterm. I will take the
new rule into account when computing the final grade. Moreover, your final
grade cannot be more than one letter grade better than the average of the two
best exam grades (out of the three exams: 2 midterms and 1 final).


Tuesday, Oct 3rd

Covered in class: Finished Kleene's Theorem, part 2.

Covered in class: Equivalence classes and MyhillNerode Theorem (Chapter 5.1).
Wednesday, Oct 4th

Covered in class: Minimizing finite automata (Chapter 5.2).
Thursday, Oct 5th

Covered in class: Pumping lemma for regular languages (Chapter 5.3).

Briefly discussed: Decision problems about regular languages (Chapter 5.4). We probably will not have time to go over this in more detail, please read the
relevant chapter in the book.

Fourth homework has been posted. It's due Thursday,
October 12th, at 11am.
Week 6

Monday, Oct 9th

Covered in class: contextfree grammars (Section 6.1).

Tuesday, Oct 10th

Covered in class: closure properties of contextfree grammars (Section 6.2)

Briefly discussed in class: derivation trees, ambiguity.

Wednesday, Oct 11th

Covered in class: Ambiguity (Section 6.46.5).

Covered in class: Nullable variables and Chomsky normal form (Section 6.6)

Thursday, Oct 12th

Covered in class: Pumping lemma for contextfree languages (Section 8.1, not
including Ogden's lemma)

Prepare for midterm 2, to be held on Thursday 10/19/06 11:0011:50am in
class. More info under "Homeworks and Midterms" at the top of this page.
Week 7

Monday, Oct 16th

Covered in class: Solutions to exercise 5.26 and Hw3, Problem 1.

Covered in class: More exercises using the pumping lemma for contextfree
languages (check Chapter 8.1).

Practice midterm 2 is now available. I'll
post the same midterm with solutions either tonight or tomorrow afternoon.

Tuesday, Oct 17th

Covered in class: The family of contextfree languages is not closed under
intersection and complement. (Chapter 8.2)

Wednesday, Oct 18th

Covered in class: Decision problems on contextfree languages. (Chapter 8.3)

In class we discussed an algorithm for deciding if a given CFG generates a finite language.
The idea was different than what is in the book, thus I'll try to summarize the main points here.
There are a some relevant exercises in the book (6.3638) and I'll use the terminology from
these exercises here.
First, we find all "live" variables, i.e. those that can lead to a string consisting of all terminals. (We
had an algorithm for these in class, the "circling" algorithm, see also Exercise 6.36.) We remove all nonlive variables from the grammar,
i.e. we discard all productions that contain a nonlive variable. If the starting symbol (S) is not live,
we know that the language is empty. Thus, let's assume that the starting symbol is live.
The overall idea is to check if there exists a variable A such that S >* uAv and A >* xAy where u,v,x,y
are strings of terminals and variables. If there exists such A, we can clearly generate an infinite number of
strings from the language and thus the language is infinite. If there does not exist such A, then
the language is finite (why?).
The next goal is decide if for a given variable A we can have S >* uAv (if A is "reachable" from S, see Exercise 6.37).
(How do you find "unreachable" variables?) Once we identify the "unreachable" variables, we discard them.
Now all variables are reachable from
the start symbol and all variables can lead to a string of terminals.
Finally, check if there exists a variable A such that A >* xAy where x,y are strings of terminals and variables.
(How do you check this?) If there is no such A, the language is finite. Otherwise, if there exists such A,
the language is infinite.

Here you can download a little program which
uses the construction from the proof of the Kleene's Theorem to
represent a language accepted by a (deterministic) finite automaton by a
regular expression. This is an input
corresponding to the Problem 2 from Homework 4, and here is the output (all intermediate steps are printed,
and there is a bunch of redundant parenthesis  I did not want to spend
more than 20 minutes coding it so you'll have to live with these
redundancies :)

Midterm 2 will be held in class tomorrow, Thursday, 10/19/06,
11:00am11:50am. If you want, bring your handwritten singlepage
cheatsheet!

Thursday, Oct 19th
Week 8

Monday, Oct 23th

Covered in class: Pushdown automata (Chapter 7.1 and 7.2).

Tuesday, Oct 24th

Covered in class: Deterministic pushdown automata (Chapter 7.3). We will
revisit Thm 7.1 (no DPDA can accept the language pal) on Wednesday.

Briefly discussed in class: The class of languages accepted by DPDA's
is closed under complement (see end of Chapter 8.2). The idea is to swap
accepting and nonaccepting states (just as for DFA's)
but first we have to make sure that the PDA finishes reading every input
(see Exercise 7.46).

Briefly discussed in class: Given a PDA A1 and a FA A2, it is possible to
construct a PDA A3 such that L(A3) = L(A1) intersection L(A2).
(We will soon recognize this statement as Thm 8.4. Notice also that
the same construction statement/construction holds if we replace PDA by DPDA.)

Wednesday, Oct 25th

Covered in class: CFG to PDF conversion (Chapter 7.4), closure properties
of (deterministic) contextfree languages revisited (Chapter 8.2).

We will skip Chapter 7.5 (PDF to CFG conversion). The bottom line of this
chapter is:
we can convert CFG to PDF, PDF to CFG, thus PDF's accept the class of
contextfree languages.

Thursday, Oct 26th

Covered in class: Turing machines (deterministic)  Chapter 9.1.

Homework 6 has been posted, due 11/02/06.
Week 9

Monday, Oct 30th

Covered in class: nondeterministic Turing machines (Chapter 9.5).

Tuesday, Oct 31th

Covered in class: variants of Turing machines (Chapter 9.4),
classes RE and Rec (Chapter 10.1).

Wednesday, Nov 1st

Covered in class: closure properties of RE and Rec.

Covered in class: undecidability of ACCEPT and HALT (Chapters 11.1 and
11.2).

Thursday, Nov 2nd

Covered in class: Undecidability continued, proving undecidability by
reduction (Chapter 11.3).

Covered in class: Post's Correspondence Problem (Chapter 11.5).
Week 10

Monday, Nov 6th

Covered in class: Solution to problem 4 on HW6.

Covered in class: Post Correspondence Problem revisited (Chapter 11.5), undecidable problems involving
contextfree languages (Chapter 11.6).

Tuesday, Nov 7th

Covered in class: Chomsky hierarchy, closure properties of Rec and RE revisited (Chapter 10.1),
introduction to P and NP (Chapter 14.1).

Wednesday, Nov 8th

Covered in class: P vs NP continued  PRIMALITY (Chapter 14.1), reduction from CNFSAT to CLIQUE (Chapter 14.2).

Thursday, Nov 9th

Covered in class: NPcompleteness and polynomialtime reductions defined formally (Chapter 14.2), statement of
Cook's Theorem (Chapter 14.3) and a reduction from CLIQUE to VERTEXCOVER (Chapter 14.4).