形式语言与自动机_笔记整理(四)_可判定性与可计算性

• Decidability
  • The diagonalization method
    • Undecidability
    • Countable and Uncountable Sets
    • Non-RE Languages
  • The halting problem is undecidable
    • The Halting Problem
    • RE and co-RE
    • A natural non-RE Language
• Complexity
  • Time Complexity
    • Asymptotic Notation
  • Time Complexity Class
  • Polynomial Time Class P
  • Nondeterministic Polynomial Time Class NP
    • Poly-Time Verifiers
    • Examples of Languages in NP
      • CLIQUE is NP
    • NP in relation to P and EXP
• Reduce P NP NPC
  • Reductions
    • Example 1
    • Example 2
    • Definition of Reduction
    • Using Reductions
  • Rice Therom
  • Poly-Time Reductions
    • Poly-Time computable function
  • NP-Completeness
  • SAT
  • Complexity

Decidability

The diagonalization method

Undecidability

fig2

Countable and Uncountable Sets

• The natural numbers N={1,2,3,…} are countable.

• Definition: a set S is countable if it is finite, or it is infinite and there is a bijection f: N→S

• The positive rational numbers Q={m / n | m,n∈N} are countable.

• Theorem: The real numbers R are uncountable. (Cantor diagonalization)

Non-RE Languages

Theorem: there exist languages that are not Recursively Enumerable.
Proof outline:

• the set of all TMs is countable
• the set of all languages is uncountable
• the function L: {TM’s} → {languages}$ cannot be onto

Proof:

The halting problem is undecidable

The Halting Problem

• Definition:
  HALT = {<M,x> | TM M halts on input x}

• Theorem: HALT is not decidable (undecidable).

  • Proof:、
  • Suppose TM H decides HALT
    
    • if M halts on x, H accept
    • if M does not halt on x, H reject
  • Define new TM H′: on input <M>
    
    • if H accepts <M,<M>>, then loop
    • if H rejects <M,<M>>, then halt
  • Consider H′ on input <H′>:
    
    • if it halts, then H rejects <H′,<H′>>, which implies it cannot halt
    • if it loops, then H accepts <H′,<H′>>, which implies it must halt
  • Contradiction. Thus neither H nor H′ can exist

RE and co-RE

The complement of a RE language is called a co-RE language.
Theorem: a language L is decidable if and only if L is RE and L is co-RE.
Proof:
(⇒) we already know decidable implies RE

• If L is decidable, then complement of L is decidable by flipping accept/reject.
• So L is in co-RE.

(⇐) we have TM M that recognizes L, and TM M′ recognizes complement of L.

• On input x, simulate M, M′ in parallel.
• If M accepts, accept; if M′ accepts, reject.

A natural non-RE Language

• Theorem: the complement of HALT is not recursively enumerable.

• Proof:
  we know that HALT is RE
  suppose complement of HALT is RE
  then HALT is co-RE
  implies HALT is decidable.
  Contradiction.

Complexity

Analyze Algorithms
Worst-Case Analysis

Time Complexity

Measure Time Complexity

Asymptotic Notation

Asymptotic Notation Facts
- logarithmic: O(log n)
- logbn=(log2n)/(log2b)
- so logbn = O(log2n) for any constant b; therefore suppress base when write it.
- polynomial: O(nc) = nO(1)
also: cO(logn) = O(nc′)=nO(1)
- exponential: O(2nδ) for δ>0

Time Complexity Class

TIME(t(n)) = {L | there exists a TM M that decides L in time O(t(n))}

• Regular Languages,
• Context-Free Languages,
• Decidable Languages,
• RE Languages,
• co-RE languages

Multitape TMs
Theorem: Let t(n) satisfy t(n)≥n. Every t(n) multitape TM has an equivalent O(t(n)2) single-tape TM.

Polynomial Time Class P

Definition: P or polynomial-time is the class of languages that are decidable in polynomial time on a deterministic single-tape Turing Machine.
P=∪k≥1TIME(nk)

Any reasonable deterministic computational models are polynomially equivalent.

Example:
PATH = {<G,s,t> | G is a directed graph that has a directed path from s to t}
RELPRIME = {<x,y> | x and y are relatively prime}
ACFG = {<G,w> | G is a CFG that generates string w}

Nondeterministic Polynomial Time Class NP

Recall: Nondeterministic TM

Visualize computation of a NTM M as a tree.

Definition:

NTIME(t(n)) = {L | there exists a NTM M that decides L in time O(t(n))}

NP = ∪k≥1NTIME(nk)

Poly-Time Verifiers

Theorem: language L∈ NP iff it is expressible as:
L={x|y,|y| ≤ |x|k,<x,y>∈R}, where R is a language in P.
Poly-time TM MR deciding R is a verifier

Example:
HAMPATH={ <G,s,t> | G is a directed graph with a Hamiltonian path from s to t} is expressible as
HAMPATH={ <G,s,t> | p for which <<G,s,t>, p>∈ R},
R={ <<G,s,t>, p> | p is a Ham. path in G from s to t}

• p is a certificate to verify that <G,s,t> is in HAMPATH.
• R is decidable in poly-time.

Proof: construct an NTM N to decide CLIQUE in poly-time
N = “On input <G,k>, where G is a graph:
1. Nondeterministically select a subset c of k nodes of G.
2. Test whether G contains all edges connecting nodes in c.
3. If yes, accept; otherwise, reject.”

Examples of Languages in NP

A clique in an undirected graph is a subgraph, wherein every two nodes are connected.
CLIQUE = {<G,k> | graph G has a k-clique}

CLIQUE is NP

Proof: construct an NTM N to decide CLIQUE in poly-time

N = "On input <G,k>, where G is a graph:
1. Nondeterministically select a subset c of k nodes of G.
2. Test whether G contains all edges connecting nodes in c.
3. If yes, accept; otherwise, reject."

Alternative Proof:

CLIQUE is expressible as
CLIQUE = {<G,k>|c for which <<G,k>,c>∈ R},

R={<<G,k>,c>|c is a set of k nodes in G, and all the k nodes are connected in G}

R is decidable in poly-time

NP in relation to P and EXP

We do not know if either inclusion is proper.

Reduce P NP NPC

Reductions

A reduction is a way of converting one problem into another such that a solution to the second problem can be used to solve the first problem.

关于规约的方向，引用某高水平同学的解释，

“把考试题规约到作业题，作业题的解法就可以用来解决考试题。”

Reductions are one of the most important and widely used techniques in TCS, especially for proving problems hard.

Example 1

Try to prove undecidable:
ATM = {<M,w> : M accepts input w}

We know this language is undecidable:
HALT = {<M,w> : M halts on input w}

Idea:
Suppose ATM is decidable.
Show that we can use ATM to decide HALT.

• On input <M,w>
• Check if <M,w>∈ ATM
  
  • if yes, the M halts on w; ACCEPT
  • if no, then M either rejects w or it loops on w
• Construct M′ by swapping qaccept/qreject in M
• Check if <M′,w>∈ ATM
  
  • if yes, then M′ accepts w, so M rejects w; ACCEPT
  • if no, then M neither accepts nor rejects w; REJECT

Conclude HALT is decidable. Contradiction.

Example 2

Try to prove undecidable:
ETM = {<M> : L(M)=Ø}

Suppose ETM is decidable.
We showed how to use ETM to decide ATM.

• on input <M,w>
• construct M′ from description of M
  
  • on input x, if x ≠ w, then reject
  • else simulate M on x, and accept if M does.
• check if <M′>∈ETM
  
  • if no, M must accept w; ACCEPT
  • if yes, M cannot accept w; REJECT
• Conclude ATM is decidable. Contradiction.

Definition of Reduction

f:Σ∗→Σ∗ is computable if there exists a TM Mf such that on every w∈Σ∗, Mf halts on w with f(w) written on its tape.

A is mapping reducible to B, written A≤mB, if there is a computable function f such that for all w, there is w∈A<⇔f(w)∈B, where f is called the reduction of A to B.

“Yes maps to yes and no maps to no” means:
w∈A maps to f(w)∈B & w∈A maps to f(w)∈B

Using Reductions

Theorem: if A≤mB and B is decidable, then A is decidable.
Main use: Given language NEW, prove it is undecidable by showing OLD ≤m NEW, where OLD is known to be undecidable.

Theorem: if A ≤m B and B is RE, then A is RE
Main use: Given language NEW, prove it is not RE by showing OLD ≤m NEW, where OLD is known to be not RE.

Rice Therom

A TM property is a language P for which
if L(M1)=L(M2) then <M1>∈P iff <M2>∈P .

TM property P is nontrivial if

• there exists a TM M1 for which <M1>∈P, and
• there exists a TM M2 for which <M2>∉P.

Rice's Theorem: Every nontrivial TM property is undecidable.

Poly-Time Reductions

Function f should be poly-time computable.

Definition: f : Σ→ Σ is poly-time computable if for some g(n)=nO(1) there exists a g(n)-time TM Mf such that on every w∈Σ∗, Mf halts with f(w) on its tape.

Poly-Time computable function

Definition: A ≤P B (“A reduces to B”) if there is a poly-time computable function f such that for all w
w∈A⇔f(w)∈B

as before, condition equivalent to: YES maps to YES and NO maps to NO
as before, meaning is: B is at least as hard (or expressive) as A

Theorem: if A ≤P B and B∈P then A∈P.

Proof:
A poly-time algorithm for deciding A:

• on input w, compute f(w) in poly-time.
• run poly-time algorithm to decide if f(w)∈B
• if it says yes, output yes
• if it says no, output no

NP-Completeness

Definition:
A language B is NP-complete if it satisfies two conditions:

1. B is in NP, and
2. every A in NP is polynomial time reducible to B.

B is called NP-hard if we omit the first condition.

• language L is C-hard if every problem in C reduces to L
• language L is C-complete if L is C-hard and L is in C.

Theorem: If B is NP-complete and B∈P, then P=NP.

Theorem: If B is NP-complete and B≤PC for C in NP, then C is NP-complete.

Theorem: The following are equivalent.

1. P = NP.
2. Every NP-complete language is in P.
3. Some NP-complete language is in P.

SAT

A Boolean formula is satisfiable if some assignment of TRUE/FALSE to the variables makes the formula evaluate to TRUE.
SAT = {<φ> | φ is a satisfiable Boolean formula}

Theorem: SAT is NP-complete.
Proof:

• SAT is in NP
  
  • guess an assignment to the variables, check the assignment
• A ≤P SAT (for any A∈NP)
  
  • let M be a NTM that decides A in nk time. For any input string w, we construct a Boolean formula ϕM,w which is satisfiable iff M accepts w.

Example: 3SAT is in NP.

• 3SAT is a special case of SAT, and is therefore clearly in NP.

Complexity

Definition: the time complexity of a TM M is a function f:N→N, where f(n) is the maximum number of steps M uses on any input of length n.

Definition: the space complexity of a TM M is a function f:N→N, where f(n) is the maximum number of tape cells M scans on any input of length n.

Definition:
TIME(t(n)) = {L : there exists a TM M that decides L in time O(t(n))}

P = ∪k≥1 TIME(nk)
EXP = ∪k≥1 TIME(2nk)

Definition:
SPACE(t(n)) = {L : there exists a TM M that decides L in space O(t(n))}

PSPACE = ∪k≥1 SPACE(nk)

Definition:
NTIME(t(n)) = {L : there exists a NTM M that decides L in time O(t(n))}

NP = ∪k≥1 NTIME(nk)

Theorem:
P≠EXP
P⊂NP⊂PSPACE⊂EXP
Don't know if any of the containments are proper.

The Church-Turing Thesis
Everything we can compute on a physical computer
can be computed on a Turing Machine

The extended Church-Turing Thesis
Everything we can compute in time t(n) on a physical computer can be computed on a Turing Machine in time t(n)O(1) (polynomial slowdown)