CS380: Homework and Reading Assigments

This webpage serves as an outline of the topics covered in each class. You will also find links to the handouts here.

Announcements (Homeworks, Midterms, Final, etc.)

• Final will be held on Friday, 11/17/06, 10:15am-12:15pm in room 70-1610. Bring your letter-size handwritten "help-sheet".
  
  We will have a review session on Monday, 11/13/06, at 4pm, most probably in 70-ICL5 (if the location changes, I'll post a notice on the lab door). Come with questions.
  
  The practice final is now available. The practice final should give you an idea of the type of questions asked at the final, and the level of difficulty. However, the real final might deviate in topics and form from the practice final.
  Here (pdf) or here (a funny format that as far as I know only the Windows web-browser can read but since it's smaller than the pdf, I decided to link to it as well) is a practice final with solutions (keep in mind that for many of the problems there are several correct answers, e.g. your completely different automaton can correctly accept the same language as my automaton). The graduate theory class has their practice final available here.
  
  The list of topics to study for the final:
  • The focus will be on the topics discussed after Midterm2; however, you should have a general understanding of all topics discussed in class. For example, you should know that NFA-Lambda is equivalent to (D)FA but you will not be required to apply the subset sum construction.
    Topics of the final exam:
    • Pushdown automata (Chapter 7.1-7.4, you will not be required to apply the construction from Chapter 7.4 but you should know that it exists). Exercises 7.5, 7.13, 7.14.
    • Pumping lemma for context-free languages (Chapter 8.1, you do not need to study Ogden's lemma). Exercise 8.1
    • Closure properties of various language classes (regular, context-free, recursive, recursively enumerable, think also about closure properties of P and NP - see "general advice" below), equivalence of various computational models (variants of automata, grammars). Scan the material covered in individual weeks (below) for references to the relevant Chapters in the book. Check e.g. problem 2 on Midterm 2 and practice midterm 2, problems 1 and 2 on the graduate practice final, also review old homework assignments.
    • Turing machines (Chapters 9.1-9.5). E.g. problem 4 on the graduate practice final and problem 4 on Homework 6 (if this problem is on the final, it will be a simpler language than on the homework).
    • Chomsky hierachy. Given a language, which language classes does the language belong in? Given a language, describe a device accepting/generating the language. E.g. problem 4 on the practice final, problem 4 on the graduate practice final, problem 1 on Midterm 1, classify languages from Exercises 8.5, 8.8 (for each of these languages determine if it is regular, context-free, in P, in NP, recursive, or recursively enumerable).
    • Decidability (Chapter 11, except 11.4; Chapter 5.4; Chapter 8.3). You will not be required to prove that a problem is undecidable but you should intuitively understand that a problem such as ACCEPT-FINITE is more difficult than ACCEPT-Lambda and thus undecidable. E.g. problem 2 on practice final, problems 2,3 on Homework 7.
    • Modeling. Define your own model based on a verbal description of the model's capabilities. E.g. problem 1 on Homework 7, problem 5 on practice final.
    • P, NP, and NP-completeness (Chapter 14). You should understand general concepts, e.g. what is required to prove that a problem is in NP? In P? You will not be required to come up with a reduction on the final, however, you should understand how reductions work (what does it mean to reduce a problem to another problem) and what implications the reduction has in the given context. E.g. in class we proved that CNF-SAT reduces to CLIQUE and CLIQUE reduces to VERTEX-COVER. Since CNF-SAT is NP-complete, the other two problems must be NP-complete as well. Check e.g. problem 1 on practice final, also Exercise 14.16.
  
  
  • General advice:
    • Understand the solutions to all assigned homework problems and midterms.
      
    • Solve both practice finals (in the graduate practice final, you can leave the "Is NP closed under complementation" box blank - the answer is "nobody knows").
      
    • You do not need to study material that was not covered in class. Thus if I recommend a section in the book to read, you may skip parts that have not been covered in class (such as Ogden's lemma). With the undecidability of ACCEPTANCE and HALTING (Chapters 11.1-2) I deviated from the book - if you understand your class notes, there is no need to try to understand the book's notation.


• Change in grading scale. I am relaxing the grading scale to: >88% A, 77-88% B, 66-77% C, 55-66% D. It is possible that the grading scale will be further adjusted but if it will, it will be only to your advantage (i.e. I might decrease the 88% required for A to, say 87%, but I will definitely not increase it to something more than 88%). Recall also that your better midterm counts for 20% of your final grade and the lower score midterm contributes 10% towards your final grade. (MyCourses does not reflect this change in grading since it appears to be impossible to set it up in such way. Thus, on MyCourses you'll see contribution of 15% for each midterm.)


• Because of privacy issues, I cannot leave your graded homeworks in the tutoring center. I placed the graded Hw5 in your folders - if you do not have a folder on the third floor of GCCIS and you would like to see your homework before Monday, please e-mail me.


• Seventh homework is available, due Thursday 11/09/06, 11:00am (it will not be possible to move this deadline to a later time).


• Sixth homework is available, due Thursday 11/02/06.
  Update: The deadline for HW6 has been extended until Friday 11/03/06, 11:59:59pm (slide your submission under my office door).


• Fifth homework is available, due Thursday 10/26/06, 11:00am.
  Erratum: In the question in problem 6, L was replaced with L^r. The currently posted version is (I hope) correct.


• Practice midterm 2 is now available.
  (Note for those who downloaded the midterm on Monday evening: I added one more TRUE/FALSE statement to Problem 2, as that part was worth 7 points and originally I had only 6 statements. I also made the language in Problem 3 slightly more challenging - if you have already solved the original problem, the correct solution is the same but the starting symbol is T.)
  Here are the solutions and this is a not very pretty picture solution of problem 4 (in the second construction, red arcs correspond to transitions on 0 and blue arcs correspond to transitions on 1).
  Note: I just (9:44pm, Wednesday) updated the picture solution, since I forgot to add a transition on 1 from the initial to the original accepting state (this is the result of the lambda-lambda-1-lambda path in the NFA-lambda). Thanks to Sam Chase for noticing this.


• Midterm 2 will be held in class on Thursday, 10/19/06, 11:00-11:50am.
  
  The structure of the test will be similar to that of midterm 1. Study Sections 4.2 through 6.6 (you may skip 6.3 as we did not cover that in class), and the first half of Section 8.1 (pumping lemma and the examples immediately following it - Example 8.1-8.4).
  
  Here is the list of topics for midterm questions, along with relevant exercises from the book. If the question asks for a standard construction (discussed in the class/book), make sure to use that construction (the point is to understand how the constructions work and that they work for any input, e.g. we do not need to study what an NFA accepts to construct an equivalent DFA).
  
  
  • Convert NFA-lambda to NFA: 4.28, 4.29
  • Kleene's Theorem - create an NFA-lambda for a language given by a regular expression: 4.35 (review also 4.36)
  • Kleene's Theorem - give a regular expression representing the language given by a DFA: 4.38 (do not panic: this construction is very time consuming - if it appears on the midterm, the DFA will have only 2 states)
  • Minimize a DFA - 5.16, 5.17, 5.19
  • Pumping lemma for regular languages: 5.23, 5.24 (one of the midterm questions will be a proof of nonregularity by pumping lemma)
  • Given a language, construct a CFG generating it: 6.4, 6.9
  • Convert a CFG to an equivalent CFG in the Chomsky normal form: 6.39 (also review 6.40)
  • Pumping lemma for context-free languages: 8.1 (we will do more examples in class next week)
  • Review closure properties, think about questions in Exercises 5.26, 5.27, 5.29, 5.30.


• Fourth homework, due Thursday, 10/12/06, 11:00am.


• Third homework, due Thursday, 10/05/06, 11:00am.
  For some case of problem 1 you might find it useful to use closure properties rather than distinguishability-type proofs. The closure properties are not discussed in the book (as far as I know), thus I am restating here a proof of nonregularity by closure properties (done in class in the 4th week).
  
  
  Claim: L1 = {x over the alphabet {a,b} | number of a's in x is the same as number of b's} is not regular.
  
  Proof: We will prove this by contradiction, using the fact that the language L2 = {a^i b^i | i>= 0} is not regular (we proved this in class by distinguishability; it is also discussed in the book as Example 5.2, using the equivalence classes terminology). Let L3 = {a^i b^j | i,j>=0}. Notice that L1 intersection L3 equals L2. Suppose that L1 is regular. L3 is clearly regular, described by the regular expression a^*b^*. Since the intersection of two regular languages is regular, L1 intersection L3 must be regular. But it is equal to L2 which we know is not regular. This is the contradiction, thus L1 cannot be regular.
  
  
  Recall that you cannot discuss the actual homework problems at the tutoring center. However, you may find it useful to discuss some of the following languages (and any others that you come across, e.g. in the exercises in the book).
  1. {0^i 1^j | i,j>=0, i!=j}
  2. {0^i 1^i 0^j | i,j>=0}
  3. {0^i 1^j 2^k | i,j,k>=0, i+j=k}
  4. {0^i 1^j 0^i 1^j | i,j>=0}
  
  
• Midterm 1 will be held in class on Thursday, 09/28/06, 11-11:50am.
  It will cover material discussed in weeks 1-3. The following exercises represent the knowledge you should have gained during these weeks: 1.38-1.44, 1.47, 3.2, 3.4, 3.5, 3.8, 3.9, 3.11, 3.12, 3.18, 3.19, 3.20, 3.21, 3.23, 3.33, 4.4, 4.9a, 4.10. You should also have a feeling for which languages are regular and which are not. You are free to discuss with your classmates or at the tutoring center the solutions to these exercises, past homework problems, or any other issues you might have with the material in the book or covered in classes. Recall that you are allowed to prepare a single "help-sheet", a hand-written, letter-sized, single paged sheet with notes from class. (No photocopies please.)
  
  
• Second homework, due Monday, 09/25/06, 11:00am. I am leaving town 09/21/06 shortly after the class, ask your questions before then. I will answer e-mail sporadically over the weekend, 09/23-24, and I won't answer any e-mails that come after 9pm on Sunday.
  
  
• First homework, due Wednesday, 09/20/06, 11:00am. I will stop answering e-mail on Tuesday, 09/19/06, at 4:30pm.
  
  

• Monday, Sept 4th
  • Buy the book.
  • Handout: Course logistics, topics, grading policy.
  • Covered in class: Section 1.5 (Languages).
  • Skim Chapter 1.


• Tuesday, Sept 5th
  • Covered in class: Section 2.4.2 (Recursive Definitions of Sets).
  • Covered in class: Section 2.5 (Structural induction). In particular, Example 2.22 (language with more a's than b's).
  • Skim Chapter 2.
  • Prepare for Discrete Math Quiz, to be held on Thursday, September 7, 11:00-11:30am, in class. There will be two problems. The first will require basic discrete math reading/writing/reasoning skills. The second will ask for a simple proof by mathematical induction.


• Wednesday, Sept 6th
  • Covered in class: handout (language with more a's than b's, part not covered in Example 2.22)


• Thursday, Sept 7th
  • Covered in class: discrete math quiz (typo corrected)
  • Study discrete math if you need it, make full use of the tutoring center.

• Monday, Sept 11th
  • Covered in class: Exercises 1.38, 1.47(a), also 1.47(a) with intersection replaced by union (i.e. what is the relationship between L1.(L2 union L3) and L1.L2 union L1.L3 ?).
  • Do Exercise 1.47(b-d). We will discuss it on Tuesday.
  • Briefly discussed in class: Regular languages (Chapter 3.1), to be continued on Tuesday.


• Tuesday, Sept 12th
  • Covered in class: Regular languages and regular expressions (Chapter 3.1).


• Wednesday, Sept 13th
  • Covered in class: Exercise 1.47(b-d).
  • Covered in class: Introduction to finite automata (Chapter 3.2).
  • First homework, due next Wednesday, 09/20/06 at 11:00am.


• Thursday, Sept 14th
  • Covered in class: Formal definition of finite automata (Chapter 3.3).
  • Covered in class: Operations on languages defined by finite automata (intersection, complement - Chapter 3.5).

• Monday, Sept 18th
  • Covered in class: Distinguishability (Chapter 3.4)
  • Second homework, due next Monday, 09/25/06 at 11:00am.


• Tuesday, Sept 19th
  • Covered in class: Distinguishability revisited.
  • Covered in class: Operations on languages accepted by finite automata (union, subtraction).
  • Briefly discussed in class: nondeterministic FA.


• Wednesday, Sept 20th
  • Covered in class: Nondeterministic FA (Chapter 4.1).
  • Covered in class: Operations on languages accepted by NFAs (concatenation).


• Thursday, Sept 21th
  • Covered in class: For every NFA there exists a DFA accepting the same language (Chapter 4.1).

• Monday, Sept 25th
  • Covered in class: Recursive definitions revisited. A do-not-hand-in exercise: For every string x over the alphabet {a,b} let n_a(x) denote the number of a's in x. Give a recursive definition for n_a(x).
  • Covered in class: Closure properties of regular languages revisited. Closure properties can be used to prove that certain languages are nonregular. Think about closure properties of the class of nonregular languages.
  • Covered in class: Definition of NFA-lambda, to be continued on Tuesday.
  • Midterm 1 will be held in class on Thursday, 09/28/06.


• Tuesday, Sept 26th
  • Covered in class: NFA-lambda, converting NFA-lambda to NFA (Chapter 4.2).


• Wednesday, Sept 27th
  • Review session for midterm 1.


• Thursday, Sept 28th
  • Midterm 1. (The NFA from problem 3 is not included.) The tutoring center will be happy to discuss any questions you might have.
  • Homework 3, due Thursday, October 5th, 11:00am.

• Monday, Oct 2nd
  • Covered in class: Kleene's Theorem (Chapter 4.3), part 1 (constructing NFA-lambda for languages given by regular expressions). Started part 2 (constructing a regular expression for languages accepted by FA).
  • Change of rules for computing final grade: The lower midterm grade will count 10% toward your final grade, the higher midterm grade will count 20% towards your final grade. This rule cannot be implemented on mycourses, thus there you will still see 15% contribution for each midterm. I will take the new rule into account when computing the final grade. Moreover, your final grade cannot be more than one letter grade better than the average of the two best exam grades (out of the three exams: 2 midterms and 1 final).


• Tuesday, Oct 3rd
  • Covered in class: Finished Kleene's Theorem, part 2.
  • Covered in class: Equivalence classes and Myhill-Nerode Theorem (Chapter 5.1).

Wednesday, Oct 4th
• Covered in class: Minimizing finite automata (Chapter 5.2).

Thursday, Oct 5th
• Covered in class: Pumping lemma for regular languages (Chapter 5.3).
• Briefly discussed: Decision problems about regular languages (Chapter 5.4). We probably will not have time to go over this in more detail, please read the relevant chapter in the book.
• Fourth homework has been posted. It's due Thursday, October 12th, at 11am.

• Monday, Oct 9th
  • Covered in class: context-free grammars (Section 6.1).


• Tuesday, Oct 10th
  • Covered in class: closure properties of context-free grammars (Section 6.2)
  • Briefly discussed in class: derivation trees, ambiguity.


• Wednesday, Oct 11th
  • Covered in class: Ambiguity (Section 6.4-6.5).
  • Covered in class: Nullable variables and Chomsky normal form (Section 6.6)


• Thursday, Oct 12th
  • Covered in class: Pumping lemma for context-free languages (Section 8.1, not including Ogden's lemma)
  • Prepare for midterm 2, to be held on Thursday 10/19/06 11:00-11:50am in class. More info under "Homeworks and Midterms" at the top of this page.

• Monday, Oct 16th
  • Covered in class: Solutions to exercise 5.26 and Hw3, Problem 1.
  • Covered in class: More exercises using the pumping lemma for context-free languages (check Chapter 8.1).
  • Practice midterm 2 is now available. I'll post the same midterm with solutions either tonight or tomorrow afternoon.


• Tuesday, Oct 17th
  • Covered in class: The family of context-free languages is not closed under intersection and complement. (Chapter 8.2)


• Wednesday, Oct 18th
  • Covered in class: Decision problems on context-free languages. (Chapter 8.3)
    
    
  • In class we discussed an algorithm for deciding if a given CFG generates a finite language. The idea was different than what is in the book, thus I'll try to summarize the main points here. There are a some relevant exercises in the book (6.36-38) and I'll use the terminology from these exercises here.
    
    First, we find all "live" variables, i.e. those that can lead to a string consisting of all terminals. (We had an algorithm for these in class, the "circling" algorithm, see also Exercise 6.36.) We remove all non-live variables from the grammar, i.e. we discard all productions that contain a non-live variable. If the starting symbol (S) is not live, we know that the language is empty. Thus, let's assume that the starting symbol is live.
    
    The overall idea is to check if there exists a variable A such that S ->* uAv and A ->* xAy where u,v,x,y are strings of terminals and variables. If there exists such A, we can clearly generate an infinite number of strings from the language and thus the language is infinite. If there does not exist such A, then the language is finite (why?).
    
    The next goal is decide if for a given variable A we can have S ->* uAv (if A is "reachable" from S, see Exercise 6.37). (How do you find "unreachable" variables?) Once we identify the "unreachable" variables, we discard them. Now all variables are reachable from the start symbol and all variables can lead to a string of terminals.
    
    Finally, check if there exists a variable A such that A ->* xAy where x,y are strings of terminals and variables. (How do you check this?) If there is no such A, the language is finite. Otherwise, if there exists such A, the language is infinite.
    
    
  • Here you can download a little program which uses the construction from the proof of the Kleene's Theorem to represent a language accepted by a (deterministic) finite automaton by a regular expression. This is an input corresponding to the Problem 2 from Homework 4, and here is the output (all intermediate steps are printed, and there is a bunch of redundant parenthesis - I did not want to spend more than 20 minutes coding it so you'll have to live with these redundancies :)
  
  
  • Midterm 2 will be held in class tomorrow, Thursday, 10/19/06, 11:00am-11:50am. If you want, bring your hand-written single-page cheat-sheet!


• Thursday, Oct 19th
  • Midterm 2.
  • Homework 5 has been posted, due 10/26/06, 11:00am.

• Monday, Oct 23th
  • Covered in class: Pushdown automata (Chapter 7.1 and 7.2).


• Tuesday, Oct 24th
  • Covered in class: Deterministic pushdown automata (Chapter 7.3). We will revisit Thm 7.1 (no DPDA can accept the language pal) on Wednesday.
  • Briefly discussed in class: The class of languages accepted by DPDA's is closed under complement (see end of Chapter 8.2). The idea is to swap accepting and nonaccepting states (just as for DFA's) but first we have to make sure that the PDA finishes reading every input (see Exercise 7.46).
  • Briefly discussed in class: Given a PDA A1 and a FA A2, it is possible to construct a PDA A3 such that L(A3) = L(A1) intersection L(A2).
    (We will soon recognize this statement as Thm 8.4. Notice also that the same construction statement/construction holds if we replace PDA by DPDA.)
  
  
• Wednesday, Oct 25th
  • Covered in class: CFG to PDF conversion (Chapter 7.4), closure properties of (deterministic) context-free languages revisited (Chapter 8.2).
  • We will skip Chapter 7.5 (PDF to CFG conversion). The bottom line of this chapter is: we can convert CFG to PDF, PDF to CFG, thus PDF's accept the class of context-free languages.


• Thursday, Oct 26th
  • Covered in class: Turing machines (deterministic) - Chapter 9.1.
  • Homework 6 has been posted, due 11/02/06.

Week 9

• Monday, Oct 30th
  • Covered in class: nondeterministic Turing machines (Chapter 9.5).


• Tuesday, Oct 31th
  • Covered in class: variants of Turing machines (Chapter 9.4), classes RE and Rec (Chapter 10.1).


• Wednesday, Nov 1st
  • Covered in class: closure properties of RE and Rec.
  • Covered in class: undecidability of ACCEPT and HALT (Chapters 11.1 and 11.2).


• Thursday, Nov 2nd
  • Covered in class: Undecidability continued, proving undecidability by reduction (Chapter 11.3).
  • Covered in class: Post's Correspondence Problem (Chapter 11.5).


Week 10

• Monday, Nov 6th
  • Covered in class: Solution to problem 4 on HW6.
  • Covered in class: Post Correspondence Problem revisited (Chapter 11.5), undecidable problems involving context-free languages (Chapter 11.6).


• Tuesday, Nov 7th
  • Covered in class: Chomsky hierarchy, closure properties of Rec and RE revisited (Chapter 10.1), introduction to P and NP (Chapter 14.1).


• Wednesday, Nov 8th
  • Covered in class: P vs NP continued - PRIMALITY (Chapter 14.1), reduction from CNF-SAT to CLIQUE (Chapter 14.2).


• Thursday, Nov 9th
  • Covered in class: NP-completeness and polynomial-time reductions defined formally (Chapter 14.2), statement of Cook's Theorem (Chapter 14.3) and a reduction from CLIQUE to VERTEX-COVER (Chapter 14.4).