Chapter 0

Introduction to the Theory of Computation

Main Goals of ITEC 420

1. Answer this Basic Question: What are the fundamental capabilities and limitations of computers?


2. Learn about the tools needed to answer this question.

Areas of Study

• Languages, Automata, Regular Expressions, and Grammars
• Mathematical tools and models of computation for reasoning about computers [Part One of the text]


• Computability Theory
• Theoretical limits on computing [Part Two]


• Complexity Theory
• Practical limits on computing [Part Three]

Languages and Models of Computation

• Languages
• A language is a set of strings
• We will define lots of languages
• We will define tools for defining languages


• Language Definition Tools (examples below):
1. Automata


2. Regular Expressions


3. Grammars


• Automata are used to model computation


• Languages are used to reason about results of computations

Automata

• Precise and simple model of an abstract computer
• What does abstract mean?


• Easy to reason about and analyze


• Aka Machines

Example Finite Automata

• Example 1





• Example 2

• What strings do these automata validate?


• Validation is a simple computation


• Remember the rules for computation with a FSM:
1. Start at start state (ie open-ended arrow)
2. Process the string making one transition for each character
• Must have transition in each state for each character (for now)
3. Finish when reach end of string
4. String is accepted (ie validated) if end in accept state (ie double circle)

Regular Expressions

• Another precise way of defining a language
• Examples:
• 1(0|1)*
  
• Letter (Letter | Digit)*

Grammars

• Another precise way of defining a language


• Examples:
• LeadingOneString → 1 | 1 Bitstring
• Bitstring → Bit | Bit Bitstring
• Bit → 0 | 1


• Ident → Letter | Letter LettersOrDigits
• LettersOrDigits → Letter | Digit | Letter LettersOrDigits | Digit LettersOrDigits
• ...

Back to Automata: Kinds of Automata

• We study three kinds of automata
1. Finite Automata (FA):
• No storage
• Limited computational power
• aka Finite State Automata (FSA)
• aka Finite State Machine (FSM)


2. Push Down Automata (PDA): FA + stack
• FA + stack


3. Turing Machine (TM): FA + tape
• FA + tape

Computing Power of Automata

• Models are increasingly powerful
• More powerful means able to define a larger set of languages


• Power of automata (in terms of programming languages):
• FA can recognize program tokens, but not structure


• PDA can recognize program structure, but can't do type checking, etc.


• TM can do complete compilation (though it would be hard to implement!)


• TM can handle any program
• Church-Turing Thesis

Church-Turing Thesis!

• The Thesis: Anything that can be computed can be computed on a Turing Machine!


• Can this be proved?
• Hint: What does "Anything that can be computed" mean?

Consequence of the Church-Turing Thesis!

• Is (your favorite language) more powerful than (my favorite language)?


• Answer this first: Can each implement a Turing Machine?


• One of the main concepts for you to understand from 420!

Computability Theory

• Fundamental Question: What problems can a computer NOT solve
• CT Thesis deals with "anything that can be computed"
• Now we consider what cannot be computed
• Precisely defined problems only (ie not what's for dinner)


• Example - Can we write function HaltCheck(P, I) to determine whether program P halts when run on input I


• Example: given a set of axioms and a set of proof rules (sufficient for arithmetic), can we find a proof for every true statement x


• Fundamental tool: Diagonalization

Diagonalization Example

• How to solve this problem: Which has more elements: naturals or even naturals?


• Two ways to tell if two sets have the same number of elements?
• Count
• 1:1 correspondence


• Back to the questions: which has more elements: naturals or even naturals?


• Now another question: Which has more elements: naturals or rationals?
• Table of all rationals: row index is numerator, column index is denom


• Now the really interesting question: Which has more elements: naturals or reals?
• Assume function f(n) lists all real numbers n:


• Proof by contradiction

Complexity Theory - Basic Question

• Answer this Basic Question: What are the practical limitations of computers?


• Consider hard problems that take too long to effectively compute
• An Easy Problem: Shortest path between 2 graph nodes
• A Hard Problem: Shortest path between 2 nodes that visits all nodes


• Easy: Encrypt a password
• Hard: Find key to decrypt an encrypted password


• Basic question: Is P = NP?
• Do (undiscovered) easy algorithms (ie in P) exist for certain hard problems (ie in NP, not known to be in P)
• Win $10⁶ and become famous
• Analysis uses concepts from Parts One and Two

Why Study Theory of Computation

• Lots of applications
• Programming language design and implementation
• String processing
• Cryptography
• Ability to recognize hard problems


• Broaden your understanding of computers


• Improve ability for abstraction


• Exposure to beautiful ideas


• Preparation for Graduate School