├── 04_lambda_calculus.md
├── 05_abstract_programming.md
├── 06_self_interpretation.md
├── 07_the_secd_abstract_machine.md
├── 08_memory_management_for_s_expressions.md
├── 10_lisp_variations_and_implementations.md
├── 13_logic_overview.md
├── 14_predicate_logic_and_the_first_inference_engine.md
├── 17_warren_abstract_machine.md
├── README.md
└── figures
    ├── 06_01_metainterpreters.png
    ├── 06_06_recursive_closures.jpg
    ├── 07_01_secd.jpg
    ├── 07_02_memory_model.jpg
    ├── 07_10_recursive_closures.jpg
    ├── 07_11_stack.jpg
    ├── 07_19_rap.jpg
    ├── 10_15_cadr_machine.jpg
    ├── 10_16_memory_word.jpg
    ├── 10_17_cdr_coding.jpg
    ├── 10_18_compact_lisp_machine.jpg
    ├── 10_22_register_windows.jpg
    ├── 10_23_spur_instructions.jpg
    ├── 13_01_logic_computing.jpg
    ├── 13_05_logic_system.jpg
    ├── 13_06_satisfying_interpretation.jpg
    ├── 13_10_connectives.jpg
    ├── 13_12_axiom_schemas.jpg
    ├── 13_13_truth_table.jpg
    ├── 14_02_standard_logic_forms.jpg
    ├── 14_04_conversion_propositional.jpg
    ├── 14_06_conversion_prenex.jpg
    ├── 14_09_horn_clauses.jpg
    ├── 14_10_combination_of_axioms.jpg
    ├── 14_12_herbrand_universe.jpg
    ├── 14_13_another_herbrand_universe.jpg
    ├── 14_14_herbrand_interpretations.jpg
    ├── 14_15_inference_engine.jpg
    ├── 17_01_prolog_execution.jpg
    ├── 17_02_translation.jpg
    ├── 17_03_structure.jpg
    ├── 17_04_memory.jpg
    ├── 17_05_objects_in_memory.jpg
    ├── 17_06_choice_point.jpg
    ├── 17_07_procedure_code_segment.jpg
    ├── 17_08_quick_sort.jpg
    ├── 17_09_indexing_instructions.jpg
    ├── 17_10_procedural_instructions.jpg
    ├── 17_11_choice_points.jpg
    ├── 17_12_get_instructions.jpg
    ├── 17_13_getv_instruction.jpg
    ├── 17_15_put_instructions.jpg
    ├── 17_16_unify_instructions.jpg
    ├── 17_17_complex_objects.jpg
    ├── 17_18_add_compilation.jpg
    ├── 17_19_builtin_instructions.jpg
    ├── 17_20_multiinstruction.jpg
    └── 17_22_code_for_append.jpg


/04_lambda_calculus.md:
--------------------------------------------------------------------------------
  1 | # Lambda Calculus
  2 | 
  3 | Lambda calculus is a mathematical language for describing arbitrary
  4 | *expressions* built from the application of functions to other expressions.
  5 | It is as powerful as any other notations for describing algorithms, including
  6 | any programming language.
  7 | 
  8 | 
  9 | Main operation of λ - calculus is the application of one subexpression
 10 | (function) to another (its single argument) by substitution of the argument into
 11 | the function's body. **Currying** is the normal mode of execution.
 12 | 
 13 | * functions have no names
 14 | * names given to things are the formal parameters of a function
 15 | * lexical scoping
 16 | 
 17 | ## Syntax
 18 | 
 19 | {{{
 20 | 	<identifier>  := a|b|c|d|e ...
 21 | 	<function>    := ( λ<identifier>"|"<expression>)
 22 | 	<application> := (<expression><expression>)
 23 | 	<expression>  := <identifier> | <function> | <application>
 24 | }}}
 25 | 
 26 | Examples:
 27 | 
 28 | {{{
 29 | 	identifier:  a
 30 | 	application: ((λx|(yx))a)
 31 | 	function:    (λx|(yx))
 32 | }}}
 33 | 
 34 | 
 35 | ## General Model Of Computation
 36 | 
 37 | In λ - Calculus, what an expression means is equivalent to what it can
 38 | reduce to after all function applications have been performed.
 39 | 
 40 | Interpretative semantic model:
 41 | 
 42 | 1. find all possible application subexpressions in the expression
 43 | 2. pick one where the function is neither a simple identifier nor an
 44 |    application expression, but a function expression of the form `(λx|E)`
 45 |    where `E` is arbitrary expression
 46 | 3. assume that the expression to the right of this function is expression `A`
 47 | 4. perform the substitution `[A/x]E` - identify all free occcurences of `x` in
 48 |    `E` and replace them by `A`
 49 | 5. replace the entrire application by the result of this substitution and loop
 50 |    back to step 1
 51 | 
 52 | Sample computation:
 53 | 
 54 | 	((λx|(xi))((λz|((λq|q)z))h)) # applying (λx|(xi)) to ((λz|((λq|q)z))h))
 55 | 	------------------------------- # ([((λz|((λq|q)z))h)/x](λx|(xi))) 
 56 | 	(((λz|((λq|q)z))h)i)			# applying (λq|q) to z ([z/q]q)
 57 | 	(((λz|z)h)i)					# applying (λz|z) to h ([h/z]z)
 58 | 	(hi)
 59 | 
 60 | 
 61 | ## Standard Simplifications ##
 62 | 
 63 | `(...((AB)C)...X)` is the same as `(ABC...X)`
 64 | 
 65 | `(AB)` is the same as `AB`, therefore `(λx|(λy|(λz|M)))` is the same as `(λx|λy|λz|M)`
 66 | 
 67 | `(λx|λy|λz|M)` is the same as `(λxyz|M)`
 68 | 
 69 | ## Identifiers ##
 70 | 
 71 | An ***identifier*** is used in λ - Calculus as a placeholder to indicate
 72 | where in a function's definition a real argument should be substituted, when
 73 | the function is applied. Each use of the identifier in the function body is
 74 | called an ***instance*** of that identifier. ***Scope*** of an identifier is
 75 | the region of the expression where instances of the identifier will always have
 76 | the same value - in λ - Calculus, the rules for scoping are the same as
 77 | in conventional statically scoped languages. An instance can be ***bound*** or
 78 | ***free*** depending on whether the identifier is in the argument list of the
 79 | function. belongs to the current scope.
 80 | 
 81 | To clarify a little bit more:
 82 | 1. free variables of `x` are just `x`
 83 | 2. free variables of `(λx|A)` are free variables of `A` with `x` removed
 84 | 3. free variables of `(AB)` are union of free variables of `A` and `B`
 85 | 
 86 | Examples:
 87 | 
 88 | 	(λx|x)		#has no free variables
 89 | 	(λx|xy)	#has one free variable - y 
 90 | 	(λx|xx)x	#has one free variable - x !but only last instance!
 91 | 
 92 | 
 93 | ## Substitution Rules ##
 94 | 
 95 | An expression of the form `(λx|E)A`, where `E` and `A` are arbitrary
 96 | expressions, the evaluation of this expression involves the ***substitution***
 97 | of the expression `A` for all appropriate *free* instances of the identifier
 98 | `x` in the expression `E`. Any bound instances of `x` and any other instances
 99 | are left unchanged. 
100 | 
101 | ### Renaming Rule ###
102 | 
103 | Consider an expression `(λx|(λy|B))A`. If `A` contains any free occurences of
104 | `y`, then a blind substitution into the body of the function will end up
105 | changing those instances of `y` in `A` from free to bound, radically changing
106 | the value of the expression. The solution is to change the formal parameter of
107 | the function and all free occurences of that symbol in the function's body.
108 | 
109 | We replace all `y` in B by `z` (or any not yet used name `[z/y]B`)
110 | 
111 | ### Conversion Rules, Reduction, Normal Order ###
112 | 
113 | Two expressions `A` and `B` are the same if there is a formal way of converting
114 | from to other via a series of reductions (`A >> B` means `A` reduces to `B`)
115 | 
116 | * ***alpha conversion*** corresponds to simple and safe renaming of formal
117 |   identifiers. Two expressions are the same if they differ only in symbol
118 |   names.
119 | * ***beta conversion*** matches normal λ - Calculus function
120 |   applications. Two expressions are the same if one represents the result of
121 |   performing some function application found in the other.
122 | * ***eta conversion*** corresponds to a simple optimization of a function
123 |   application that occurs quite often, it is basically a special case of the
124 |   beta conversion: `(λx|Ex)A >> EA`
125 | 
126 | When we can no longer apply beta or eta conversions, the expression is in
127 | ***normal order*** 
128 | 
129 | ### The Church-Rosses Theorem ###
130 | 
131 | In 1936 Alonzo Church and J. R. Rosser proved two important theorems.
132 | 
133 | ***Church-Rosses  Theorem I*** states that if an expression `A` through two
134 | different conversion sequences, can reduce to two different expressions `B` and
135 | `C`, there is always some other expression `D` such that both `B` and `C` can
136 | reduce to it. 
137 | 
138 | ***Church-Rosses Theorem II*** states that if an expression `A` reduces to `B`,
139 | and `B` is in normal order (therefore when `A` can be reduced to normal order),
140 | then we can get from `A` to `B` by doing the ***leftmost reduction*** at each
141 | step. This one is important, since it gives us a concrete algorithm for finding
142 | normal order of the expression.
143 | 
144 | 
145 | ### Order of Evaluation ###
146 | 
147 | ***Normal-order reduction*** follows CRT - it always locates the leftmost
148 | function involved in the application and substitues unchanged copies of
149 | arguments into the function body. No reductions are performed on the arguments
150 | until after.
151 | 
152 | ***Applicative-order reduction*** reduces the argument completely before
153 | function is applied on it.
154 | 
155 | Normal-order reduction terminates with normal-order expression, but possibly by
156 | evaluating the same expression multiple times. Applicative-order reduction is
157 | not guaranteed to stop when doing only leftmost application.
158 | 
159 | Example of normal order reduction:
160 | 
161 | 	(λx|(λw|(λy|wyw)b))((λx|xxx)(λx|xxx))((λz|z)a)
162 | 	(λw|(λy|wyw)b)((λz|z)a)
163 | 	(λy|((λz|z)a)y((λz|z)a))b
164 | 	((λz|z)a)b((λz|z)a)     #first redundant application
165 | 	(ab((λz|z)a))			  #second redundant application
166 | 	(aba)
167 | 
168 | Example of applicative order reduction:
169 | 
170 | 	(λx|(λw|(λy|wyw)b))((λx|xxx)(λx|xxx))((λz|z)a)
171 | 	#evaluation of ((λx|xxx)(λx|xxx))
172 | 	([(λx|xxx)/x](xxx))
173 | 	((λx|xxx)(λx|xxx)(λx|xxx))    #and will aparently continue forewer
174 | 
175 | ## Basic Arithmetic in λ ##
176 | 
177 | Mathematically, we have to define how integer 0 will look like, and then the
178 | function `s` which given an integer `k` will produce an expression for the
179 | integer `k+1`.
180 | 
181 | Because we are using λ - Calculus,  the integer 0 will be represented as a function: 
182 | 
183 | 	0 = (λsz|z)
184 | 
185 | The definition of the successor function is:
186 | 
187 | 	s(x) = (λxyz|y(xyz))
188 | 
189 | Some examples to hurt my brain more:
190 | 
191 | 	0 = (λsz|z)
192 | 	1 = (λxyz|y(xyz))(λsz|z)
193 | 	    (λyz|y((λsz|z)yz))
194 | 		(λyz|yz)
195 | 	2 = (λxyz|y(xyz))(λsz|sz)
196 | 	    (λyz|y((λsz|sz)yz))
197 | 		(λyz|y(yz))
198 | 	3 = (λxyz|y(xyz))(λsz|s(sz))
199 | 		(λyz|y((λsz|s(sz))yz))
200 | 		(λyz|y(y(yz)))
201 | 	...
202 | 
203 | ### Operations ###
204 | 
205 | ***Addition***:
206 | 
207 | 	(λwzyx|wy(zyx))
208 | 
209 | ***Multiplication***
210 | 
211 | 	(λwzy|w(zy))
212 | 
213 | Example 2 + 1:
214 | 
215 | 	(λwzyx|wy(zyx))(λsz|s(sz))(λsz|sz)
216 | 	(λyx|(λsz|s(sz))y((λsz|sz)yx))
217 | 	(λyx|y(y((λsz|sz)yx)))
218 | 	(λyx|y(y(yx)))
219 | 	3
220 | 
221 | Example 1 * 2:
222 | 
223 | 	(λwzy|w(zy))(λsz|sz)(λsz|s(sz))
224 | 	(λy|(λsz|sz)((λsz|s(sz))y))
225 | 	(λy|(λsz|sz)(λz|y(yz)))
226 | 	(λy|(λz|(λz|y(yz))z))
227 | 	(λy|(λz|y(yz)))
228 | 	2
229 | 
230 | Example 2 * 3
231 | 
232 | 	(λwzy|w(zy))(λsz|s(sz))(λsz|s(s(sz)))
233 | 	(λy|(λsz|s(sz))((λsz|s(s(sz)))y))
234 | 	(λy|(λsz|s(sz))(λz|y(y(yz))))
235 | 	(λy|(λz|(λz|y(y(yz)))((λz|y(y(yz)))z)))
236 | 	(λy|(λz|(λz|y(y(yz)))(y(y(yz)))))
237 | 	(λy|(λz|y(y(y(y(y(yz)))))))
238 | 	6	#uff :)
239 | 
240 | 
241 | ## Boolean Operations in λ - Calculus ##
242 | 
243 | 	true = T = (λxy|x)
244 | 	false = F = (λxy|y)
245 | 
246 | On contrary to integers, where the internal body matched our concepts of
247 | integers, these functions are used because of the way how they function when
248 | given real arguments. Consider `Q` and `R` are arbitrary expressions:
249 | 
250 | 	if P == T then PQR = T Q R = (λxy|x)QR = Q
251 | 	if P == F then PQR = F Q R = (λxy|y)QR = R
252 | 
253 | Other interesting functions:
254 | 
255 | 	not = (λw|wFT)
256 | 	and = (λwz|wzF)
257 | 	or  = (λwz|wTz)
258 | 	xor = (λwz|w(zFT)(zTF))
259 | 
260 | 	zero(x) = (λx|x F not F)
261 | 
262 | Examples: 
263 | 
264 | 	not T = (λw|w(λxy|y)(λxy|x))(λxy|x)
265 | 			(λxy|x)(λxy|y)(λxy|x)
266 | 			(λxy|y)
267 | 			F
268 | 
269 | 	and T F = (λwz|wzF)(λxy|x)(λxy|y)
270 | 			  ((λxy|x)(λxy|y)F)
271 | 			  (λxy|y)
272 | 			  F
273 | 
274 | 	zero(1) = (λx|x (λxy|y) (λw|w(λxy|y)(λxy|x)) (λxy|y))(λsz|sz)
275 | 			  ((λsz|sz) (λxy|y) (λw|w(λxy|y)(λxy|x)) (λxy|y))
276 | 			  ((λxy|y) (λw|w(λxy|y)(λxy|x)) (λxy|y))
277 | 			  (λxy|y)
278 | 			  F
279 | 
280 | 	zero(0) = (λx|x F not F)(λsz|z)
281 | 			  ((λsz|z) F not F)
282 | 			  (not F)
283 | 			  T
284 | 
285 | ## Recursion in λ - Calculus ##
286 | 
287 | Consider the application `RA`, where `R` is some recursively defined function
288 | and `A` is some argument expression. If `A` satisfies the ***basis test*** for
289 | `R`, then `RA` reduces to ***basis case***. If not, then `RA` reduces to some
290 | other expression of the form `...(RB)...` where `B` is some simpler expression.
291 | Making `R` recursive has a lot to do with making it ***repeat itself***.
292 | 
293 | How to do this self-repetition - try to evaluate following expression:
294 | 
295 | 	(λx|xx)(λx|xx)
296 | 
297 | The expression has the property that it does not change regardless of how many
298 | beta conversions are performed. Using this as a basis, for any lambda
299 | expression `R`:
300 | 
301 | 	((λx|R(xx))(λx|R(xx)))
302 | 	R((λx|R(xx))(λx|R(xx)))
303 | 	R(R((λx|R(xx))(λx|R(xx))))
304 | 	R(R(R((λx|R(xx))(λx|R(xx)))))
305 | 
306 | This allows us to compose an arbitrary function `R` on itself an infinite
307 | number of times. So to finally tell you the secret - here comes the ***Y
308 | combinator*** (also called fixed point combinator):
309 | 
310 | 	(λy|((λx|y(xx))(λx|(xx))))
311 | 
312 | What a recursion would it be without the factorial:
313 | 
314 | 	fact(n) = if zero(n) then 1 else n*fact(n-1)
315 | 
316 | λ - Calculus equivalent:
317 | 
318 | 	fact(n) = Y(λfn|zero n 1 (* n (f (- n 1))))
319 | 
320 | fact(4):
321 | 
322 | 	R = (λrn|zero n 1 (* n (r (- n 1))))
323 | 
324 | 	fact(4)
325 | 	Y R 4
326 | 	R (YR) 4
327 | 	(λn|zero n 1 (* n (YR (- n 1))))4
328 | 	zero 4 1 (* 4 (YR (- 4 1)))
329 | 	zero 4 1 (* 4 ((YR) 3))
330 | 	F 1 (* 4 ((YR) 3))
331 | 	(* 4 ((YR) 3))
332 | 	(* 4 (R (YR) 3))
333 | 	(* 4 ((λn|zero n 1 (* n (YR (- n 1))))3))
334 | 	(* 4 (zero 3 1 (* 3 (YR (- n 1)))))
335 | 	(* 4 (* 3 ((YR) 2)))
336 | 	(* 4 (* 3 (R (YR) 2)))
337 | 	(* 4 (* 3 ((λn|zero n 1 (* n (YR (- n 1))))2)))
338 | 	(* 4 (* 3 (zero 2 1 (* 2 (YR (- 2 1))))))
339 | 	(* 4 (* 3 (* 2 (R (YR) 1))))
340 | 	...
341 | 	(* 4 (* 3 (* 2 (* 1 1))))
342 | 	...
343 | 	24
344 | 


--------------------------------------------------------------------------------
/05_abstract_programming.md:
--------------------------------------------------------------------------------
  1 | # a formal basis for abstract programming #
  2 | 
  3 | to make &lambda; - calculus easier to use for humans, the notation called
  4 | ***abstract programming*** has been developed. it is much easier to read and
  5 | still completely convertible to pure &lambda; - calculus. 
  6 | 
  7 | ## syntax ##
  8 | 
  9 | 	<identifier>		:= <alpha-char>{<alpha-char>|<number>}*
 10 | 	<function-name> 	:= <identifier>
 11 | 	<constant>      	:= <number> | <boolean> | <char-string>
 12 | 	<expression>    	:= <constant> 
 13 | 							| <identifier> 
 14 | 							| (-><identifier> "|" <expression)
 15 | 							| (<expression>+)
 16 | 							| <function-name>(<expression>{,<expression>}*
 17 | 							| let <definition> in <body>
 18 | 							| letrec <definition> in <body>
 19 | 							| <body> where <definition>
 20 | 							| <body> whererec <definition>
 21 | 							| if <expression> 
 22 | 								then <expression> 
 23 | 								else <expression>
 24 | 							| ... # standard arithmetic expressions etc
 25 | 
 26 | 	<body>				:= <expression>
 27 | 	<definition>		:= <header> = <expression>
 28 | 							| <definition> {and <definition}*
 29 | 	<header>			:= <identifier>
 30 | 							| <function-name>(<identifier>{,<identifier>}*)
 31 | 	<abstract-program>	:= <expression>
 32 | 
 33 | example:
 34 | 
 35 | 	letrec fact = (->n|if n == 0 then 1 else n * fact(n - 1)) in fact(4)
 36 | 
 37 | 	fact(4) whererec fact(n) = if n == 0 then 1 else n * fact(n - 1)
 38 | 
 39 | 	letrec fact(n) = ((if n = 0 then 1 else z)
 40 | 						whererec z = n * fact(n - 1))
 41 | 		in (fact(4))
 42 | 
 43 | 
 44 | ## constants ##
 45 | 
 46 | semantically, a ***constant*** is any object whose name denotes its value
 47 | directly. in abstract programming we assume we have syntax for constants of
 48 | types ***boolean, integer*** and ***character string***. therefore any
 49 | occurences of `t` or `f` are taken as expressions `(->xy|x)` and `(->xy|y)`.
 50 | similarly numbers are translated to the form `(->sz|s^k z)`, negative numbers
 51 | can be expressed like `(->n |n - k)` where `k` is their positive equivalent.
 52 | character string will be ascii encoded.
 53 | 
 54 | ## function applications ##
 55 | 
 56 | two possibilities for function application exist:
 57 | 
 58 | 	f(a,b,c,d...) = (f a b c d ...)
 59 | 
 60 | where `f` is defined by enclosing `let`, `letrec` etc. expressions.
 61 | 
 62 | ## conditionals ##
 63 | 
 64 | small difference between abstract conditional and the conventional conditionals
 65 | in programming languages it, that conventional languages do not define what
 66 | happens when `p` (predicate) is not `t` or `f`. in abstract programming and
 67 | &lambda; - calculus, two expressions are accepted as arguments to whatever `p`
 68 | reduces to. also, `else` is not optional in abstract programming, the result
 69 | would be curried function, that causes havoc to further processing.
 70 | 
 71 | ## local definitions ##
 72 | 
 73 | abstract programming has a notation very akin to a cleaned-up combination of
 74 | macros and call-by-name. the simplest form of this notation is:
 75 | 
 76 | 	let <identifier> = <expression> in <body>
 77 | 
 78 | this whole expression is equivalent in value to a copy of the `<body>` where every
 79 | free occurence of the `<identifier>` is replaced by the `<expression>`.
 80 | expressed by &lambda; - calculus:
 81 | 
 82 | 	let x = a in e
 83 | 
 84 | is the same as:
 85 | 
 86 | 	(->x|e)a
 87 | 
 88 | 
 89 | ## recursive definitions ##
 90 | 
 91 | one limitation of the `let` and `where` expressions is that they do not permit
 92 | recursion in a function definition. a brute-force solution is to use **y
 93 | combinator** defined earlier. or one can use `letrec`. the major difference is
 94 | that any free occurence of the definition's identifier in the deinition's
 95 | expression is replaced by the expression itself. thus in:
 96 | 
 97 | 	letrec f(n) = if zero(n) then 1 else n*f(n-1) in f(4)
 98 | 
 99 | the free occurences of `f` in `f(n-1)` is replaced recursively by the whole
100 | expression `if zero(n) then 1 else n*f(n-1) in f(4)`. 
101 | 
102 | the conversion to the pure &lambda; - calculus is direct.  given `letrec f=a in
103 | e`, we form an application where the function is an anonymous function whose
104 | formal parameter is `f` and whose body is `e`. instead of using `a` as an
105 | argument, we create `(->f|a)` and use that as an argument to the function `y`.
106 | therefore pure &lambda; - calculus equivalent of:
107 | 
108 | 	letrec f = a in e
109 | 
110 | is
111 | 	
112 | 	(->f|e) (y (->f|a)) = (->f|e) ((->y|(->x|y(xx))(->x|y(xx))) (->f|a))
113 | 
114 | there are some subtle differences between multiple definitions in the ***and
115 | form*** of a `letrec` versus a `let`. in the `letrec` form, the expression in
116 | each anded definition has complete access to every other definition. the result
117 | is a set fo ***mutually recursive functions*** that are defined together. and
118 | this is challenging, we will need y1 and y2 combinators.
119 | 
120 | ## higher-order functions ##
121 | 
122 | are functions which accept other functions as an argument (we are ignoring
123 | functions accepting integer, string etc. arguments, althrough they are
124 | functions too).
125 | 
126 | ***map*** takes as its arguments some arbitrary function and a list of objects.
127 | the result is equally long list of objects returned by application of the
128 | function to the k-th element of the input list
129 | 
130 | 	map(f,x) = 	if null(x) then nil
131 | 				else cons(f(car(x)), map(f, cdr(x)))
132 | 
133 | 
134 | ***map2*** is identical, except that it takes 2 equally long lists and applies
135 | function to matching elements.
136 | 
137 | 	map2(f,x,y) = if null(x) then nill
138 | 					else cons(f(car(x),car(y)), map2(f, cdr(x), cdr(y)))
139 | 
140 | ***reduce*** takes a function, an object, and a list. the value returned is
141 | result of applying the function to the first element of the list and the result
142 | of applying ***reduce*** to the rest of the list. if the list is empty, an
143 | object (the second argument to the ***reduce***) is returned. 
144 | 
145 | 	reduce(f,t,x) = if null(x) then t
146 | 					else f(car(x), reduce(f,t, cdr(x)))
147 | 
148 | ***vector*** takes a list of functions and an object and returns the list of
149 | the applications of functions in the list to the object.
150 | 
151 | 	vector(o,x) = if null(x) then nil
152 | 					else cons(car(x)o,vector(o, cdr(x)))					
153 | 
154 | ***while*** takes two functions and an accumulating parameter. as long as the
155 | application of the first function to the accumulating parameter returns `T`,
156 | ***while*** recursively calls itself with the accumulating parameter modified
157 | by applying second function parameter to it. 
158 | 
159 | 	while(p,f,x) = if p(x) then while(p,f,f(x))
160 | 					else x
161 | 
162 | ***compose*** takes two functions and returns a new function which is the
163 | composition of the two.
164 | 
165 | 	compose(f,g) = (->x|f(g(x)))
166 | 
167 | 
168 | ## Exercises ##
169 | 
170 | 1. Write as an abstract program the predicate ***free***(x,e), which returns
171 |    true if the identifier occurs free in e. 
172 | 
173 | 		free(x,e) = if null(e) 
174 | 					then F
175 | 					elseif is-id(e)
176 | 					then if x = e
177 | 						 then T
178 | 						 else F
179 | 					elseif is-lambda(e) then 
180 | 							if get-lambda-arg(e) = x 
181 | 							then F
182 | 							else free(x, get-body(e))
183 | 					else or(free(x, get-function(e)),
184 | 							free(x, get-argument(e)))
185 | 
186 | 
187 | 2. Write an abstract definition for a function ***subs***(y,x,m) where `y` and
188 |    `m` are any valid s-expressions and x is an atom representing variable name.
189 |    The result is s-expression equivalent to `[y/x]m`.
190 | 
191 | 		subs(y,x,m) = if null(m)
192 | 					  then nil
193 | 					  elseif is-id(m)
194 | 					  then if m = x
195 | 						   then y
196 | 						   else m
197 | 					  elseif is-lambda(m)
198 | 					  then let a = get-lambda-arg(m)
199 | 						   and b = get-body(m)
200 | 						   and f = free(a,y)
201 | 						   in if a = x
202 | 						      then m
203 | 						      elseif f
204 | 							  then let z = new-id()
205 | 							       in create-function(z, subs(y,z,subs(z,y,get-body(m))))
206 | 							  else create-function(get-arg(m), subs(y,x,get-body(m)))
207 | 					  else create-application(subs(y,x,get-function(m)),
208 | 											  subs(y,x,get-argument(m)))
209 | 					
210 | 						
211 | 							     
212 | 
213 | 
214 | 
215 | 
216 | 
217 | 


--------------------------------------------------------------------------------
/06_self_interpretation.md:
--------------------------------------------------------------------------------
  1 | # Self - Interpretation #
  2 | 
  3 | &lambda; - Calculus is powerful enough to express any computable function. In
  4 | this chapter we will show, that &lambda; - Calculus is expressible as
  5 | a computable function. It is therefore possible to write an interpreter for
  6 | &lambda; - Calculus in &lambda; - Calculus.
  7 | 
  8 | We will write a function called ***eval***, that takes arbitrary &lambda; function
  9 | and evaluates it. Once having this ***eval*** function writen in machine
 10 | language, we can use it to write a more advanced ***eval*** on top of it, which
 11 | has more advanced features (abstract programming syntax, builtin functions, IO,
 12 | error handling etc). We can then continue and use new ***eval*** to build more
 13 | **evals** for even more powerful features. This way we can quickly implement
 14 | experimental langauges, and if the performance needs to be improved, a layer
 15 | can be reimplemented in native language. Layers above will execute faster
 16 | without the need to reimplement them.
 17 | 
 18 | ![Metainterpreters, languages, and compilers](figures/06_01_metainterpreters.png)
 19 | 
 20 | ## Abstract Interpreters ##
 21 | 
 22 | First interpreter interprets simple &lambda; - Calculus without any of the
 23 | *simplifications*.
 24 | 
 25 | syntax summary:
 26 | 
 27 | 	<expression> := <identifier> | <function> | <application>
 28 | 	<function>   := (-><identifier>'|'<expression)
 29 | 	<application := (<expression><expression)
 30 | 
 31 | Abstract syntax functions for ***eval***:
 32 | 
 33 | 	is-id(E)
 34 | 	is-function(E)
 35 | 	is-application(E)
 36 | 
 37 | 	get-function(E)			#get function from application E
 38 | 	get-argument(E)  		#get argument from application E
 39 | 	get-id(E)        		#get identifier from function E
 40 | 	get-body(E)      		#get body from function E
 41 | 
 42 | 	create-function(x,E)    #create (->x|E)
 43 | 	create-application(A,E) #create (AB)
 44 | 	new-id()				#return a guaranteed unique identifier symbol
 45 | 
 46 | Abstract interpreter:
 47 | 
 48 | 	eval(e) =				#evaluate expression e as far as we can
 49 | 		if is-id(e)
 50 | 		then e
 51 | 		else if is-function(e) 
 52 | 			 then create-function(get-id(e), eval(get-body(e)))
 53 | 			 else apply(get-function(e), get-argument(e))
 54 | 
 55 | 	apply(f,a) =			#normal order application
 56 | 		if is-id(f)
 57 | 		then create-application(f, eval(a))
 58 | 		else if is-application(f)
 59 | 			 then apply(eval(f),a)
 60 | 			 else eval(subs(a, get-id(f),get-body(f)))
 61 | 
 62 | 	apply(f,a) =			#applicative order application
 63 | 		if is-id(f)
 64 | 		then create-application(f, eval(a))
 65 | 		else let b = eval(a) in 
 66 | 				if is-application(f)
 67 | 				then apply(eval(f),a)
 68 | 				else eval(subs(a, get-id(f),get-body(f)))
 69 | 
 70 | 	subs(a,x,e) =			#substitute a for x in e
 71 | 		if is-id(e)
 72 | 		then if e = x 
 73 | 			 then a
 74 | 			 else e
 75 | 		else if is-application(e)
 76 | 			 then create-application(subs(a,x,get-function(e)),
 77 | 									 subs(a,x,get-argument))
 78 | 			 else let y = get-id(e)
 79 | 				  and c = get-body(e)
 80 | 				  in if y = x
 81 | 					 then e
 82 | 					 else let z = new-id() in
 83 | 							create-function(z, subs(a,x,subs(z,y,c)))
 84 | 
 85 | 
 86 | And now in human language. ***Eval*** has 3 sections, in first it leaves identifiers as
 87 | they are, in second it handles functions by by recreating them, but with their
 88 | body fully reduced. In third, it handles application by applying the function
 89 | to arguments (therefore passing it into ***apply*** function)
 90 | 
 91 | ***Apply*** handles reduction of applications. If `f` is identifier, the only
 92 | thing that can be reduced is the argument. If `f` is application, then the
 93 | function needs to be evaluated before applied to arguments. If  `f` is
 94 | function, it is applied to the argument `a`. There are two versions of
 95 | ***apply***, one for normal order evaluation, one for applicative order
 96 | evaluation. In the second, arguments are evaluated before evaluating function
 97 | and before applying the function on arguments.  
 98 | *Note: If `f` is an application in form `(xy)`, then the __eval__ will be caught in
 99 | an infinite loop. Fix is left as an exercise for the reader.*
100 | 
101 | ***Subs*** function performs the substitution process. For simplicity, the
102 | function always renames `y`, even when not necessary. Improving this is left as
103 | an exercise for the reader.
104 | 
105 | 
106 | ## Lambda Expressions as S-Expressions ##
107 | 
108 | 	<identifier> => <identifier>
109 | 	(<->identifier>|<l-expr>) => (lambda (<identifier>) <s-expr>)
110 | 	(<l-expr><l-expr>) => (<s-expr> <s-expr>)
111 | 
112 | Example:
113 | 
114 | 	(((->x|(->y|x))a)b) => (((lambda (x) (lambda (y) x)) a) b)
115 | 
116 | Converted ***subs*** function:
117 | 
118 | 	subs(a,x,e) =
119 | 		if atom(e)
120 | 		then if eq(e,x)
121 | 			 then a
122 | 			 else e
123 | 		else if not(is-function(e))
124 | 			 then list(subs(a,x,car(e)), subs(a,x,cadr(e)))
125 | 			 else let y = caadr(e)
126 | 				  and c = caddr(e)
127 | 				  in if y = x
128 | 					 then e
129 | 					 else let z = new-id()
130 | 						  in list(lambda, list(z), subs(a,x,subs(zyc)))
131 | 						  
132 | 
133 | Multiple arguments:
134 | 
135 | 	(->xy|y(xx))wz => ((lambda (x y) (y (x x))) w z)
136 | 
137 | Literal booleans, numbers and strings represent ***Constants***.
138 | 
139 | Common functions, such as `+, -, *, and, or, car, cdr, const`, are called
140 | ***built-in functions*** and ***eval*** or ***apply*** must be modified to
141 | include tests for and code to perform there functions.
142 | 
143 | Similarly to `lambda` we can add other convenient notations from abstract
144 | programming such as `let-in`, `letrec-in`, `if-then-else` etc. These are called
145 | ***special forms***. ***Special forms*** are not functions, it depends on
146 | ***eval*** how they will be handled. For example `if-then-else` if condition
147 | is truthy, then the *then* expression will be passed into the ***eval***, if
148 | not, te *else* expression will. Other example is `quote` special form, which
149 | will cause that its arguments will not be evaluated at all.
150 | 
151 | 
152 | ## An Expanded Interpreter ##
153 | 
154 | * Interpreted language is in S-Expression format
155 | * Functions may have multiple arguments
156 | * Support for built-in functions such as `+`, `car` ...
157 | * A mix of normal (default) and applicative (for built-in functions etc) order reduction
158 | * Support for special forms such as `let, letrec, if, cond` ...  
159 | 
160 | Code:
161 | 
162 | 	eval(e) =
163 | 		if atom(e)
164 | 		then e
165 | 		else let fcn = car(e)
166 | 			 and args = cdr(e)
167 | 			 in if atom(fcn)
168 | 			    then if fcn = quote then car(args)
169 | 				elseif member(fcn, builtins)
170 | 				then apply-builtin(fcn, args)
171 | 				elseif fcm = lambda
172 | 				then list(lambda, car(args), eval(cadr(args)))
173 | 				elseif fcn = if
174 | 				then if eval(car(args)) = T
175 | 					 then eval(cadr(args))
176 | 					 else eval(caddr(args))
177 | 				elseif fcn = cond then eval-cond(args)
178 | 				elseif (fcn = let) or (fcn = letrec)
179 | 				then let ids = car(args)
180 | 					 and vals = cadr(args)
181 | 					 and body = caddr(args)
182 | 					 in if fcn = let
183 | 					    then subs2(vals, ids, body)
184 | 						else #letrec supports only one definition at the moment
185 | 							apply(list(lambda, ids, body),
186 | 								  list(Y, list(lambda, ids, vals))
187 | 							where y = (lambda (y) ((lambda (x) (y (x x)))
188 | 							  					   (lambda (x) (y (x x)))))
189 | 				else apply(fcn, args)
190 | 			else apply(fcn, args)
191 | 
192 | 	apply(f,a) = #normal order evaluation
193 | 		if atom(f)
194 | 		then cons(f, map(eval, a))
195 | 		elseif car(f) = lambda
196 | 		then let normal = length(cadr(f))
197 | 			 and actual = length(a)
198 | 			 in if formal = actual
199 | 				then eval(subs2(a, cadr(f), caddr(f))
200 | 				else list(lambda,
201 | 						  drop(actual, cadr(f)),
202 | 						  eval(subs2(a, cadr(f), caddr(f))))
203 | 					 whererec drop(count, x) = 
204 | 								  if count = 0
205 | 								  then x
206 | 								  else drop(count - 1, cdr(x))
207 | 		else apply(f, a)
208 | 
209 | 	apply-builtin(f, a) = 
210 | 		let args = map(eval(a))
211 | 		in if f = car
212 | 		   then caar(args)
213 | 		   elseif f = cdr
214 | 		   then cdar(args)
215 | 		   elseif f = +
216 | 		   then car(args) + cadr(args)
217 | 		   elseif ...
218 | 
219 | 	eval-cond(a) = 
220 | 		if null(a) 
221 | 		then nil
222 | 		else let z = car(a) 
223 | 			 in if eval(car(z)) = T
224 | 			 then eval(cadr(z))
225 | 			 else eval-cond(cdr(z))
226 | 
227 | 	subs2(v,n,e) = #handles multiple arguments
228 | 		if atom(e)
229 | 		then lookup(e,n,v)
230 | 		elseif car(e) = lambda
231 | 		then let old = cadr(e) 
232 | 		     in let new = map(new-id, old)
233 | 			    in list(lambda, new, subs2(v,n,subs2(new,old,caddr(e))))
234 | 		else map((->z|subs2(v,n,z)),e)
235 | 
236 | 	lookup(z,n,v) = #find z in n and replace by value in v
237 | 		if null(n) or null(v)
238 | 		then z
239 | 		elseif car(n) = z then car(v)
240 | 		else lookup(z,cdr(n),cdr(v))
241 |       
242 |     
243 | This approach suffers from an inefficiency - more passes may be needed (up to
244 | two per argument, once to find all free occurences and once to do the
245 | substitutions. One alternative is simply to remember at the time of an
246 | application which identifiers are to get which values, and then do the
247 | substitution on an identifier by identifier basis when the function's body is
248 | scanned for the next application. These records will be stored in
249 | ***association list***, which is created at the time of application. We've
250 | saved some work, but we still have to rename inner function's binding
251 | variables. Solution will come, but remember the source of the problem because
252 | it will pop up later as the ***funarg problem***.
253 | 
254 | ## Closures ##
255 | 
256 | Most of the problems with renaming come from trying to improve the *efficiency*
257 | of a basic interpreter through a combination of applicative order reduction and
258 | the use of simple association lists in place of immediate substitutions.
259 | A better solution involves *packaging* an expression with its environment into
260 | a single unit, which can be passed around, but still unpackaged and evaluated
261 | when needed. Such a package is called ***closure***. 
262 | 
263 | Consider an application `(->x|E)A` (or the equivalent `let x=A in E`). Assuming
264 | for the time being that `A` has no free identifiers, let us suspend this
265 | evaluation just before the substitution takes place. We have an expression `E`
266 | and the ***environment*** `x=A` (or alist `((x.A))`). This combination is
267 | called ***closure***. 
268 | 
269 | 	<closure> := [<expression>,<environment>]
270 | 
271 | Basic closure eval:
272 | 
273 | 	eval(e, alist) =
274 | 		if is-identifier(e)
275 | 		then let z = assoc(e, alist)
276 | 			 in if null(z) then e else cdr(z)
277 | 		elseif is-closure(e)
278 | 		then let e1 = get-expression(e)
279 | 			 and alist1 = get-environment(e)
280 | 			 in eval(eval(e1, alist1),alis)
281 | 		else ...
282 | 
283 | 	apply(f, a) =
284 | 		...
285 | 		elseif is-function(f)
286 | 		then create-closure(get-body(f),
287 | 							create-alist(get-identifier(f), a))
288 | 		else ...
289 | 
290 | Example:
291 | 	
292 | 	let x = 3 and y = 2 in let x = x + y in x * y
293 | 	= eval([[x * y, ((x.x + y))], ((x.3)(y.2))], nil)
294 | 	= eval(eval([x * y, ((x.x + y))], ((x.3)(y.2))),nil)
295 | 	= eval(eval(eval(x * y, ((x.x + y))), ((x.3)(y.2))),nil)
296 | 	= eval(eval((x + y) * y, ((x.3)(y.2))),nil)
297 | 	= eval((3 + 2) * 2, nil)
298 | 	= 10
299 | 
300 | ***Closure*** is ***referentially transparent***, its value is the same
301 | whenever and wherever it is evaluated.
302 | 
303 | ## Recursive Closures ##
304 | 
305 | Above implementation handles nested applications, free variables, normal and
306 | applicative order reductions, but it does not handle recursion. Consider for
307 | example the differences between `let f = A in E` and `letrec f = A in E`. In
308 | the former, any free instances of `f` in `A` refer to `f`'s bound in
309 | expressions surrounding this one. In the latter, any free instances of `f` in
310 | `A` should refer to the whole value of `A` as is. Even worse, those references
311 | to `f` in the `A` being substituted for `f` in `A` must also be replaced, as
312 | must the references to `f` in that replacement. This will go forever.
313 | 
314 | What we want is:
315 | 
316 | 	[A/f]E = [([a/F]A)/F]E = [([([...)/f]A)f]A)/f]E
317 | 
318 | By using a closure to handle this is that it can delay any required
319 | substitution until they are absolutely needed, and then perform only a minimal
320 | amount of substitution necessary to satisfy the immediate evaluation. 
321 | 
322 | The problem with expressing this as a closure is getting an alist entry which
323 | has some sort of internal references to itself. We want a closure of the form
324 | `[E, ((f.'value for f'))]`. If we let a stand for the 'value for f', then
325 | 
326 | 	a = [A, ((f.a))]
327 | 
328 | ![Value ina a recursive closure](figures/06_06_recursive_closures.jpg)
329 | 
330 | If we implement such data structures as s-expressions, we find that the
331 | ***cdr***s of cells containing such contexts point back to the ***car***s of
332 | that cell or some earlier cell linked to it. This will be discussed in more
333 | detail for the SECD Machine.
334 | 
335 | ## Exercises ##
336 | 
337 | 1. Modify subs to rename variables only when necessary.
338 | 
339 | 		subs(y,x,m) = if null(m)
340 | 					  then nil
341 | 					  elseif is-id(m)
342 | 					  then if m = x
343 | 						   then y
344 | 						   else m
345 | 					  elseif is-lambda(m)
346 | 					  then let a = get-lambda-arg(m)
347 | 						   and b = get-body(m)
348 | 						   and f = free(a,y)
349 | 						   in if a = x
350 | 						      then m
351 | 						      elseif f
352 | 							  then let z = new-id()
353 | 							       in create-function(z, subs(y,z,subs(z,y,get-body(m))))
354 | 							  else create-function(get-arg(m), subs(y,x,get-body(m)))
355 | 					  else create-application(subs(y,x,get-function(m)),
356 | 											  subs(y,x,get-argument(m)))
357 | 
358 | 2. Assume following syntax for certain class of s-expressions:
359 | 
360 |         <logic-expr> := <id> | T | F
361 |         			| (and <logic-expr> <logic-expr>)
362 |         			| (or <logic-expr> <logic-expr)
363 |         			| (not <logic-expr>)
364 |         			| (let <id> <logic-expr> <logic-expr>)
365 | 
366 |   Define in abstract syntax a function that will evaluate such expressions,
367 |   assuming it is given as input an association list of all identifiers and their
368 |   current values. Show that it works for the expression `(let x F (let y T (and (not x) (or x y))))`:
369 | 	
370 | 		#using assoc
371 | 		logic-eval(e,a) =
372 | 			if null(e)
373 | 			then nil
374 | 			elseif is-id(e)
375 | 			then assoc(e,a)
376 | 			elseif is-atom(e)
377 | 			then e-to-boolean(e)
378 | 			elseif is-not-expr(e)
379 | 			then not(logic-eval(cadr(e)))
380 | 			elseif is-and-expr(e)
381 | 			then and(logic-eval(cadr(e),a), logic-eval(caddr(e),a))
382 | 			elseif is-or-expr(e)
383 | 			then or(logic-eval(cadr(e),a), logic-eval(caddr(e),a))
384 | 			else logic-eval(cadddr(e), 
385 | 							cons(cons(cadr(e), logic-eval(caddr(e),a)), a))
386 | 
387 | 	Evaluation of `(let x F (let y T (and (not x) (or x y))))`:
388 | 		
389 | 		logic-eval(*expr*, nil) ->
390 | 		logic-eval((let y T (and (not x) (or x y))), ((x F))) ->
391 | 		logic-eval((and (not x) (or x y)), ((y T) (x F))) ->
392 | 		and(logic-eval((not x), ((y T) (x F))),
393 | 		    logic-eval((or x y))) ->
394 | 		and(not(logic-eval(x, ((y T) (x F))),
395 | 			or(logic-eval(x, ((y T) (x F))),
396 | 			   logic-eval(y, ((y T) (x F)))))) ->
397 | 		and(not(F),or(F, T)) ->
398 | 		T
399 | 			
400 | 
401 | 
402 | 3. Modify closure evaluator to handle an extension to `cond` such that if the
403 |    last element of `cond` list has only one expression in it, and no prior
404 |    test passes, the value of this expression is returned.  
405 |    
406 | 		(cond ((= 1 2) F) ((> 4 5) F) ((+ 1 2))) 
407 | 
408 | 	would return 3.
409 | 
410 | 		eval-cond(a) =
411 | 			if null(a)
412 | 			then nil
413 | 			elseif and(null(cdr(a)), null(cadr(a)))
414 | 			then eval(caar(a))
415 | 			else let z = car(a)
416 | 				 in if eval(caar(z)) = T
417 | 					then eval(cadar(z))
418 | 					else eval-cond(cdr(z))
419 | 


--------------------------------------------------------------------------------
/07_the_secd_abstract_machine.md:
--------------------------------------------------------------------------------
  1 | # The SECD Abstract Machine #
  2 |  
  3 | So far we have described two simple functional languages based on &lambda;
  4 | - Calculus: abstract programming and a prefix s-expression equivalent. The
  5 | operations of these languages has been described in terms of interpreters for
  6 | such languages (written in themselves) - ***denotational semantics***.
  7 | 
  8 | An alternative approach is through ***interpretative semantics***, where we
  9 | define a simple ***abstract machine*** and combine this with a description of
 10 | how a compiler would take programs written in the language of interest and
 11 | generate matching code for the abstract machine.
 12 | 
 13 | We will define ***SECD Machine*** as an abstract machine with properties which
 14 | are well suited to functional languages, and we will give a simple compiler for
 15 | the prefix s-exression.
 16 | 
 17 | The SECD Machine was invented by Peter Landin (1963) as a sample target for the
 18 | first function-based langauge, LISP. 
 19 | ![Uses of the SECD instruction set architecture](figures/07_01_secd.jpg)
 20 | 
 21 | ## List Memory ##
 22 | 
 23 | The basic SECD memory model assumes a large collection of identically formatted
 24 | ***cells*** in a single memory pool. 
 25 | 
 26 | ![The SECD memory model](figures/07_02_memory_model.jpg)
 27 | 
 28 | The ***tag*** field in each cell describes how the rest of the cell should be
 29 | interpreted. For this chapter, we assume only two types: ***integers*** and
 30 | ***cons cells***. For integers, bytes after the tag represent the value, for
 31 | cons cells, the space after tag is split into two halves, the ***car field***
 32 | and ***cdr field***. Both of these fields contain pointers (addresses) to other
 33 | cells in the memory. A pointer value of 0 indicates ***nil pointer***. 
 34 | 
 35 | A register in the machine called ***freelist pointer*** (or ***F register***),
 36 | points to the first free cell in the memory. When a cell is allocated,
 37 | a value in the F register is incremented. Program starts with F initialized to
 38 | 0. What will happen when F runs over the top of available memory will be
 39 | explained later.
 40 | 
 41 | ## Basic Data Structures ##
 42 | 
 43 | There are five key kinds of data structures that the SECD Machine will build,
 44 | manipulate, and keep in memory:
 45 | 
 46 | * Arbitrary s-expressions for computed data
 47 | * Lists representing programs to be executed
 48 | * Stacks used by the program's instructions
 49 | * ***Value lists*** containing the arguments for uncompleted function
 50 |   applications
 51 | * Closures to represent unprocessed function applications
 52 | 
 53 | ### Programs as Lists ###
 54 | 
 55 | Programs for SECD Machine are lists. Each element of the list corresponds to an
 56 | individual instruction. A call to function involves saving where are we in the
 57 | current list and starting execution at the beginning of the list associated
 58 | with the called function. 
 59 | 
 60 | Individual elements of a program list are of two types: simple integers or
 61 | arbitrary lists. The former are equivalend to an ***opcode*** specifying some
 62 | basic instruction in the SECD Machine's architecture. The latter usually
 63 | represent alternative branches of program flow (for example `if-then-else`)
 64 | 
 65 | ### Stacks as Lists ###
 66 | 
 67 | Stacks are represented as lists, pushing on top of the stack is in fact consing
 68 | an object onto the list. Popping an element returns **caar** to get element's
 69 | value and a **cdr** to return a pointer to the rest of the list. 
 70 | 
 71 | A subtle but important difference between these stacks and the conventional
 72 | stacks build out of sequential memory is that whilst a pop following a push
 73 | does not overwrite the storage allocated by the element popped off. New cell is
 74 | allocated for push, and a cell holding popped value still exists after push.
 75 | There are cases when this is valuable feature, but most of the time this just
 76 | introduces garbage. Garbage collection will be addressed in future sections.
 77 | 
 78 | ### Value Lists ###
 79 | 
 80 | Given that the SECD Machine supports arbitrary s-expressions, it naturally
 81 | supports association lists. The instruction set supports directly a form of
 82 | alist that includes only the value half of each pair. The SECD form is then
 83 | a list of sublists where each sublist contains the argument values (and not the
 84 | names) for a particular function call that has been made but as yet is not
 85 | complete.
 86 | 
 87 | A simple compiler can eliminate the need for the identifier name by building an
 88 | analog at compile time and encoding the index into the appropriate SECD Machine
 89 | instruction. Index will be in the form `(i,j)` where `i` determines which
 90 | sublist of the alist is desired, and `j` determines which element of that
 91 | sublist is needed. 
 92 | 
 93 | ### Closures ###
 94 | 
 95 | To the SECD Machine a ***closure*** is the combination of the code for some
 96 | function and a value list.
 97 | 
 98 | ### Recursive Closures ###
 99 | 
100 | ![Value lists and closures for recursive expressions](figures/07_10_recursive_closures.jpg)
101 | 
102 | ## The SECD Machine Registers ##
103 | 
104 | The basic instruction set architecture of the SECD Machine consists of
105 | instructions that manipulate four main data structures found in memory:
106 | 
107 | * ***evaluation stack*** (or just simply ***stack***)
108 | * ***environment***
109 | * ***control list***
110 | * ***dump***
111 | 
112 | These are accompanied by the fifth - ***F register***.
113 | 
114 | ### The S Register ###
115 | 
116 | The S register is threated as a conventional evaluation stack for built-in
117 | functions, such as `+, -, *, /` and stack operations (`car,cdr,cons`).
118 | 
119 | ### The E Register ###
120 | 
121 | The ***environment register*** points to the current value list of function
122 | arguments. This list is referenced by the machine when a value for an argument
123 | is needed, augumented, when a new environment for a function application is
124 | constructed, and modified when previously created closure is unpacked. 
125 | 
126 | ### The C Register ###
127 | 
128 | The ***control register*** functions just like the ***program counter*** or
129 | ***instruction pointer***. It points to the memory cell that designates
130 | through its car the current instruction to execute. When an instruction is
131 | completed, the content of the C register is replaced by the content of the cdr
132 | field of the last memory cell used by the current instruction `C <- cdr(C)`.
133 | 
134 | ### The D Register ###
135 | 
136 | The ***dump register*** points to a list in memory called ***dump***. The
137 | purpose of the ***dump*** is to remember the state of a function application
138 | when a new application in that function's body is to be started. This is done
139 | by appending onto the dump three new cells which record in their cars the
140 | values of the S, E, and C registers before the application. When the
141 | application completes, popping the top of the dump restores those registers to
142 | their original values so that the original application can continue. This is
143 | similar to ***call-return*** sequence found in conventional machines for
144 | procedure or subprogram activation and return.
145 | 
146 | ## The Basic Instruction Set ##
147 | 
148 | ### Accessing Object Values ###
149 | 
150 | These instructions push values of objects onto the S stack.
151 | * ***NIL*** pushes a nil value
152 | * ***LDC*** pushes a *constant*. The constant is found as the next element on
153 |   the C list after the instruction (the **card** of the C list).
154 | * ***LD*** loads an element from current environment. The **cadr** of the
155 |   C list is a cell of the form `(i,j)`.
156 | 
157 | ### Built-in Function Applications  ###
158 | 
159 | These instructions handle build-in functions such as **car, cdr, cons, atom,
160 | add, sub** ... Proper number of arguments is popped from the S stack and
161 | the result is pushed on the S stack.
162 | 
163 | ![The S register and the stack](figures/07_11_stack.jpg)
164 | 
165 | ### Instructions for if-then-else ###
166 | 
167 | ***SEL*** instruction assumes that the top of the S is an integer either zero
168 | or nonzero. Following the ***SEL*** is the C list are two elements (**cadr**
169 | and **caddr** of the C list) both of which are lists of instructions. The last
170 | instruction in both lists is ***JOIN***. When executed, the ***SEL*** will push
171 | onto the D a pointer to the C list just beyond the second sublist (**cdddr** of
172 | the C).  The machine will then pop the S stack, test it, and replace C with
173 | **cadr** or **caddr** depending if the value was zero. The ***JOIN*** then
174 | resumes the original program by popping the top off the D and resetting C to
175 | point to it.  
176 | 
177 | ### Nonrecursive Program-Defined Functions ###
178 | 
179 | ***LDF*** (load function) is followed in the C list by an element pointing to
180 | a sublist containing the code representing some program-defined function. The
181 | last instruction of this subprogram list is ***RTN*** which functions similarly
182 | to ***JOIN***. 
183 | 
184 | When ***LDF*** is executed, it builds in a new cell a closure consisting of
185 | a pointer to the new function's code and a copy of the current E register. The
186 | closure is pushed on the S stack.
187 | 
188 | The closure is executed by ***AP*** (apply) instruction. Top of the S stack
189 | should contain a closure, and underneath it a list representing the argument
190 | values to be applied to the function. Executing the ***AP*** causes the
191 | **cddr** of S, the E, and the **cdr** of C to be pushed onto the dump. After
192 | this, the S register is set to nil, the C register is set to the beginning of
193 | the code specified by the closure, and the E register is set to the cons of the
194 | arguments (second element of the original S list) and the environment from
195 | closure (cdr of the closure cell).
196 | 
197 | ***RTN*** takes the top of the S as the value to return from the application.
198 | This value is consed to the old S from the dump, S register is set to point to
199 | this list. The E and C registers are restored directly from the dump and the
200 | calling function is restored. 
201 | 
202 | ### Recursive Program-Defined Functions ###
203 | 
204 | The ***DUM*** instruction conses onto the E list a new cell whose car is nil.
205 | This corresponds to a new argument list tat is initially empty (a dummy list).
206 | This will eventually be a pointer to the self-looping value list.
207 | 
208 | A ***DUM*** instruction is used just before the ***LDF*** instruction to build
209 | a recursive closure. This will make the environments stored in those closures
210 | point to a value list where the first element is this current *dummy* sublist.
211 | 
212 | The ***RAP*** (recursive apply) assumes that the top of the S stack looks like
213 | that for an ***AP*** (a closure representing a function to be executed, and an
214 | argument list). In this case the function in the closure is the expression that
215 | calls the recursively defined functions. The closure's environment is a pointer
216 | to the same list indicated by the current E register. This, in turn, should be
217 | a list where the first element was that built by the ***DUM***. Furthermore,
218 | the list of argument values should be a list of closures, one per recursively
219 | defined function, where the environments of these closures are also pointer to
220 | the same dummy cell. 
221 | 
222 | Execution of the ***RAP*** is identical to the ***AP*** except that the car of
223 | the cell pointed to by E (the dummy cell) is reset to point to the second
224 | argument of the S stack (the list of closures). Given that the closures in that
225 | argument also point to the dummy cell, the result is exactly the loop of
226 | pointers desired. 
227 | 
228 | Within the code called by the ***RAP***, any required calls to the i-th
229 | recursively defined function **fi** are initiated by a ***LD(1,i)*** followed
230 | by the ***AP***. The ***LD*** fetches the closure for the function from the
231 | environment, and ***AP*** unpacks it as before. The environment established by
232 | the closure is the same environment it came from, with the exception that the
233 | ***AP*** adds a list to the front representing the arguments to the function.
234 | Thus, a ***LD(2,i)*** from inside the code for the function **fi** will
235 | retrieve an identical copy of its closure, and another ***AP*** will thus start
236 | a recursive call to **fi** properly. 
237 | 
238 | Again, a ***RTN*** at the end of the expression unstacks the original S, E, and
239 | C values stacked by the ***RAP*** function(s).
240 | 
241 | ![Executing a ***RAP*** instruction](figures/07_19_rap.jpg)
242 | 
243 | ### Machine Control Instructions ###
244 | 
245 | * ***STOP*** stops the machine in its tracks.
246 | * ***READC*** and ***WRITEC*** perform simple input/output operations.
247 | 
248 | 
249 | ## Compiling Simple S-Expressions ##
250 | 
251 | `*<expr>` stands for SECD code compiled from the s-expression `<expr>`.
252 | `AB` stands for the result of appending list `B` to list `A` as in `(1 2)(3 4)
253 | = (1 2 3 4)`
254 | 
255 | The function ***compile*** has three arguments, the expression `e` being
256 | compiled, a namelist `n`, and an accumulating parameter `c`. The namelist
257 | represents the variables that would be available in the environment when the
258 | s-expression `e` is executed. The accumulating parameter represents already
259 | generated code. Initial call to compile an expression `e` would be:
260 | 
261 | 	compile(e, nil, (STOP))
262 | 
263 | Code sequences for s-expressions:
264 | 
265 | 	Syntax:  <number>
266 | 	Code:    (LDC <number>)
267 | 
268 | 	Syntax:  nil
269 | 	Code:    (NIL)
270 | 
271 | 	Syntax:  <identifier>
272 | 	Code:    (LD (i,j))
273 | 
274 | 	Syntax:  (<builtin> <expr1> ... <exprn>)
275 | 	Code:    (*<exprn> ... *<expr1> <builtin>)
276 | 	Example: (mpy (add x 1) 256) =>
277 | 			 (LDC 256 LDC 1 LD (1.1) ADD MPY)
278 | 			
279 | 
280 | 	Syntax:  (if <expr1> <expr2> <expr3>)
281 | 	Code:    *<expr1> SEL (*<expr2> JOIN) (*<expr3> JOIN) *<expr2>
282 | 	Example: (if (null x) 1 (car x)) =>
283 | 			 (LD (1.1) NULL SEL (LDC 1 JOIN) (LD (1.1) CAR JOIN))
284 | 
285 | 	Syntax:  (lambda (<id1> ... <idn>) <expr>)
286 | 	Code:    (LDF (*<expr> RTN))
287 | 	Example: (lambda (x y) (add x y)) =>
288 | 			 (LDF (LD (1.2) LD (1.1) ADD RTN))
289 | 
290 | 	Syntax:  (let (<id1> ... <idn>) (<expr1> ... <exprn>) <expr>)
291 | 	Code:    (NIL *<exprn> CONS ... *<expr1> CONS LDF *<expr> RTN AP)
292 | 	Example: (let (x y) (1 2) (+ x y)) =>
293 | 			 (NIL LDC 2 CONS LDC 1 CONS LDF (LD (1.2) LD (1.1) ADD RTN) AP)
294 | 			 
295 | 	Syntax:  (letrec (<id1> ... <idn>) (<expr1> ... <exprn>) <expr>)
296 | 	Code:    (DUM NIL *<exprn> CONS ... <expr1> CONS LDF (*<expr> RTN RAP))
297 | 	Example: (letrec (f) 
298 | 					 ((lambda (x m) 
299 | 						(if (null x) 
300 | 							m
301 | 							(f (cdr x) (+ m 1))))) 
302 | 					 (f (1 2 3) 0)) =>
303 | 			 (DUM NIL LDF (LD (1.1) SEL 
304 | 				(LD (1.2) JOIN) 
305 | 				(NIL LDC 1 LD (1.2) ADD 
306 | 					CONS LD (1.1) CDR 
307 | 					CONS LD (2.1) AP JOIN) RTN)
308 | 			 CONS LDF (NIL LDC 0 CONS LDC (1 2 3) CONS LD (1.1) AP RTN) RAP)
309 | 
310 | You still there? Ok, let's continue.
311 | 
312 | A compiler from s-expressions to SECD code:
313 | 
314 | 	compile(e,n,c) =
315 | 		if atom(e)
316 | 		then #a nil, number, or identifier
317 | 			if null(e)
318 | 			then cons(NIL, c)
319 | 			else let ij = index(e, n) 
320 | 				 in if null(ej)
321 | 				 then cons(LDC, cons(e, c))
322 | 				 else cons(LD, cons(ij, c))
323 | 		else let fcn = car(e)
324 | 			 and args = cdr(e)
325 | 			 in if atom(fcn)
326 | 				then #a builtin, lambda or special form
327 | 					if member(fcn, builtins)
328 | 					then compile-builtin(args, n, cons(fcn, c))
329 | 					elseif fcn = LAMBDA
330 | 					then compile-lambda(cadr(args), cons(car(args), n), c)
331 | 					elseif fcn = IF
332 | 					then compile-if(car(args), cadr(args), caddr(args), n, c)
333 | 					elseif fcn = LET or fcn = LETREC
334 | 					then let newn = cons(car(args), n)
335 | 						 and values = cadr(args)
336 | 						 and body = caddr(args)
337 | 						 in if fcn = let
338 | 							then cons(NIL, 
339 | 									  compile-app(values, n,
340 | 										  compile-lambda(body, newn, cons(AP, c))))
341 | 							else #letrec
342 | 								append((DUM NIL),
343 | 									compile-app(values, 
344 | 										newn,
345 | 										compile-lambda(body, newn, cons(RAP,
346 | 										c))))
347 | 					else #fcn must be a variable
348 | 						cons(NIL, 
349 | 							 compile-app(args, n, cons(
350 | 								LD, cons(index(fcn,n), cons(AP, c)))))
351 | 				else #an application with nested function
352 | 					cons(NIL, compile-app(
353 | 						args, n, compile(fcn, n, cons(AP, c))))
354 | 
355 | You still there? Seriously??
356 | 
357 | Auxiliary functions:
358 | 
359 | 	compile-builtin(args, n, c) =
360 | 		if null(args)
361 | 		then c
362 | 		else compile-builtin(cdr(args), n, compile(car(args), n, c))
363 | 
364 | 	compile-if(test, then, else, n, c) =
365 | 		compile(test, n, cons(SEL, cons(compile(then, n, cons(JOIN, nil)),
366 | 						 cons(compile(else, n, cons(JOIN, nil)), c))))
367 | 
368 | 	compile-lambda(body, n, c) =
369 | 		cons(LDF, cons(compile(body, n, cons(RTN, nil)), c))
370 | 
371 | 	compile-app(args, n, c) =
372 | 		if null(args)
373 | 		then c
374 | 		else compile-app(cdr(args), n,
375 | 					compile(car(args), n, cons(CONS, c)))
376 | 
377 | 	index(e,n) = index(e, n, 1)
378 | 
379 | 	index(e, n, i) = 
380 | 		if null(n) then nil
381 | 		else letrec indx2(e, n, j) =
382 | 						if null(n) then nil
383 | 						elseif car(n) = e then j
384 | 						else indx2(e, cdr(n), j + 1)
385 | 			 in let j = indx2(e, car(n), 1)
386 | 				in if null(j)
387 | 				then index(e, cdr(n), i + 1)
388 | 				else cons(i, j)
389 | 
390 | 
391 | ### Built-in Function Applications ###
392 | 
393 | The code generated for the application of ***built-in*** functions consists of
394 | the sequences of SECD code needed for each argument appended to the SECD
395 | instruction that performs the function.
396 | 
397 | ### Conditional Forms ###
398 | 
399 | The code consist of the instructions that evaluates the test expression,
400 | followed by ***SEL*** and two lists which correspond to *then* and *else*
401 | expressions. Both sublists end with ***JOIN***.
402 | 
403 | ### Lambda Function Definitions ###
404 | 
405 | The code compiled for ***lambda*** expression consists of a two element list,
406 | ***LDF*** instruction and the list consisting of the compiled form of the
407 | lambda expression's body terminated by a ***RTN***.
408 | 
409 | Executing such code sequence will push onto S a closure whose code pointer is
410 | pointing to the sublist following the ***LDF*** and whose embedded environment
411 | is the cell pointed to by E when the ***LDF*** is executed. When
412 | ***compile-lambda*** is called, namelist already contains lambda arguments.
413 | 
414 | ### Combined Function Definitions and Applications ###
415 | 
416 | `Let` and `letrec` expressions correspond to evaluating a series of
417 | expressions, associating them with identifiers, and then evaluating a new
418 | expression that references these identifiers. The compiled code must compute
419 | each of these expressions and then **cons** the results together into a list
420 | that can be passed as an argument to the lambda function implied by the `in`
421 | expression.
422 | 
423 | All `let` expressions are pushed onto S - these will serve as arguments to the
424 | lambda function, which is pushed next. Body of the lambda is made by `in`
425 | expression and terminated by ***RTN***. 
426 | 
427 | For the `let` expression, the final instruction compiled is an ***AP***. When
428 | executed, this instruction unpacks the closure, and starts the body, with
429 | computed values estabilished on the top of E.
430 | 
431 | The only difference for a `letrec` is that a ***RAP*** is used instead, and
432 | there is an extra instruction to begin the sequence, ***DUM***. 
433 | 
434 | ## Sample Compilation ##
435 | 
436 | Original abstract program to compile:
437 | 
438 | 	let x = 3 and one = 1 in
439 | 		letrec fact(n, m) =
440 | 			if (eq n 0) then m
441 | 			else fact(n - one, n * m)
442 | 		in fact(x, one)
443 | 
444 | As s-expression:
445 | 
446 | 	(let (x one) (3 1)
447 | 		(letrec (fact) 
448 | 				((lambda (n m) (if (= n 0)
449 | 								   m 
450 | 								   (fact (- n one) (* n m)))))
451 | 				(fact x one)))
452 | 
453 | And the compilation process:
454 | 
455 | 	compile((let...), nil, (STOP))
456 | 
457 | Step 1:
458 | 
459 | 	(NIL.compile-app((3 1), nil, 
460 | 		compile-lambda((letrec...), ((x one)), (AP STOP))))
461 | 
462 | Step 2:
463 | 	
464 | 	(NIL.compile-app((3 1), nil,
465 | 		(LDF.(compile((letrec...), ((x one)), (RTN)), (AP STOP)))))
466 | 
467 | Step 3:
468 | 
469 | 	(NIL.compile-app((3 1), nil,
470 | 		(LDF (DUM NIL.compile-app(((lambda ...)), 
471 | 								  ((fact) (x one)),
472 | 								  compile-lambda((fact x one), 
473 | 												 ((fact) (x one)),
474 | 												 (RAP RTN))
475 | 								  (AP STOP))))))
476 | 
477 | Step 4:
478 | 
479 | 	compile-lambda((fact x one), ((fact) (x one)), (RAP RTN))
480 | 
481 | 	=
482 | 
483 | 	(LDF (NIL LD (2.2) CONS LD (2.1) CONS LD (1.1) AP RTN) RAP RTN)
484 | 
485 | 	=
486 | 
487 | 	(AAAAA)
488 | 
489 | Step 5:
490 | 
491 | 	(NIL.compile-app((3 1), nil,
492 | 		(LDF (DUM NIL.compile-app(((lambda ...)), 
493 | 								  ((fact) (x one)),
494 | 								  (AAAAA))
495 | 			(AP STOP))))))
496 | 
497 | Step 6:
498 | 
499 | 	compile-app(((lambda ...)), ((fact) (x one)), (AAAAA))
500 | 
501 | 	=
502 | 
503 | 	compile-app((), ((fact) (x one)), 
504 | 		compile((lambda...), ((fact) (x one)), (CONS AAAAA)))
505 | 
506 | 	=
507 | 
508 | 	compile((lambda...), ((fact) (x one)), (CONS AAAAA))
509 | 
510 | 	=
511 | 
512 | 	(LDF.(compile((if...), ((m n) (fact) (x one)), (RTN)).(CONS AAAAA)))
513 | 
514 | 	=
515 | 		((m n) (fact) (x one)) => nmfactxone
516 | 	(LDF.(compile((= n 0), nmfactxone, 
517 | 		cons(SEL, 
518 | 			cons(compile((then), nmfactxone, (JOIN)), 
519 | 				cons(compile((else), nmfactxone, (JOIN)), (RTN))))).(CONS
520 | 				AAAAA)))
521 | 
522 | Step 7:
523 | 
524 | 	compile((then), nmfactxone, (JOIN))
525 | 
526 | 	=
527 | 
528 | 	compile(m, nmfactxone, (JOIN))
529 | 
530 | 	= (LD (1.1) JOIN) => THEN
531 | 
532 | Step 8:
533 | 
534 | 	compile((else), nmfactxone, (JOIN))
535 | 
536 | 	=
537 | 
538 | 	compile((fact ...), nmfactxone, (JOIN))
539 | 
540 | 	=
541 | 
542 | 	cons(NIL, compile-app(((- n one) (* n m)), nmfactxone,
543 | 		cons(LD, cons((2.1), cons(AP JOIN)))))
544 | 
545 | 	=
546 | 	
547 | 	cons(NIL, compile-app(((- n one) (* n m)), nmfactxone,
548 | 		(LD (2.1) AP JOIN)))
549 | 
550 | 	= 
551 | 
552 | 	(NIL LD (1.2) LD (1.1) MPY CONS 
553 | 		LD (1.2) LD (3.2) SUB CONS
554 | 			LD (2.1) AP JOIN) => ELSE
555 | 
556 | Step 9:
557 | 
558 | 	(LDF (NIL LDC 0 CONS LD (1.1) EQ SEL THEN ELSE RTN) CONS AAAAA) =>
559 | 	(BBBBB)
560 | 
561 | Step 10:
562 | 
563 | 	(NIL.compile-app((3 1), nil,
564 | 		(LDF.(DUM NIL.BBBBB.(AP STOP)))))
565 | 
566 | Step 11:
567 | 
568 | 	(NIL.compile-app((3 1), nil,
569 | 		(LDF (DUM NIL BBBBB) AP STOP)))
570 | 
571 | Step 12:
572 | 
573 | 	(LDF (DUM NIL BBBBB) AP STOP) => (CCCCC)
574 | 
575 | 	(NIL.compile-app((), nil
576 | 		compile((3 1), nil, CCCCC)))
577 | 
578 | 	=
579 | 
580 | 	(NIL.compile-app((1), nil,
581 | 		(LDC 3 CONS CCCCC)))
582 | 
583 | 	=
584 | 
585 | 	(NIL.compile-app((), nil
586 | 		(LDC 1 CONS LDC 3 CONS CCCCC)))
587 | 
588 | 	= (NIL LDC 1 CONS LDC 3 CONS CCCCC)
589 | 
590 | Step 13:
591 | 
592 | 	(NIL LDC 1 CONS LDC 3 CONS LDF
593 | 		(DUM NIL LDF
594 | 			(NIL LDC 0 CONS LD (1.1) EQ SEL 
595 | 				(LD (1.1) JOIN)
596 | 				(NIL LD (1.2) LD (1.1) MPY CONS
597 | 					LD (1.2) LD (3.2) SUB CONS
598 | 					LD (2.1) AP JOIN)
599 | 				RTN) 
600 | 		CONS LDF
601 | 			(NIL LD (2.2) CONS LD (2.1) CONS LD (1.1) AP RTN) 
602 | 		RAP RTN)
603 | 	AP STOP)
604 | 
605 | 
606 | ## The Funarg Problem ##
607 | 
608 | If we try to implement SECD Machine on conventional stacks (which are much more
609 | performant), all kinds of strange results can occur, when dealing with
610 | closures. This goes by the name ***funard problem***. On conventional stack,
611 | popping the value and then pushing other value just overwrites the memory area,
612 | it does not allocate more more space.
613 | 
614 | ***Funarg problem*** can occur whenever a function which produces result
615 | contains ***free variables***. 
616 | 
617 | Solutions are to use linked lists, separate heaps for storing environments,
618 | duplicate environment stacks, spaghetti stacks or cactus stacks, or doing
619 | explicit code copying and substituting argument values.
620 | 	
621 | ## Optimizations ##
622 | 
623 | ***Tail recursion*** occurs when the last thing a function does is call some
624 | other function. Instructions generated by the compiler are `(... AP RTN)`. When
625 | executed, `AP` pushes values for S, E, and C onto D. The `RTN` of the called
626 | function will pop these values, only to be replaced by the `RTN` in the current
627 | function. 
628 | 
629 | An extension to the SECD architecture is needed - `DAP` (***direct apply***)
630 | instruction. The instruction performs only the environment modification and
631 | does not push values onto D. Sequence `(... AP RTN)` would be replaced by `(...
632 | DAP)`.
633 | 
634 | Now very common sequence is:
635 | 
636 | 	(... LDF (...code... RTN) DAP)
637 | 
638 | A closure is created just to be taken apart by `DAP`. We introduce a new
639 | instruction - `AA` (***Add Arguments***) with following register transition:
640 | 
641 | 	AA: (v.s) e (AA.c) d => s (v.e) c d
642 | 
643 | And the above sequence could be replaced by:
644 | 
645 | 	(... AA ...code... RTN
646 | 
647 | 
648 | When the called function does not need reference to the current arguments,
649 | a new arguments does not have to be consed on E, the existing cells could be
650 | reused. A new instruction is to be added - `TRAP` (***Tail-Recursive Apply***),
651 | 
652 | 	TRAP: ((f.(e'.e''))v.s) e (TRAP c) d => NIL (v.e'') f d
653 | 
654 | or even better:
655 | 
656 | 	TRAP: ((f.e') v.s) e (TRAP c) d => NIL rplaca(e', v) f d
657 | 
658 | As before, ***rplaca*** replaces the car subfield. Now the compiler must verify
659 | that the arguments are not used, and there is one very common case when this is
660 | true - ***self-recursion***, where there is no way how to use old arguments.
661 | Further optimization along the lines of `AA` would be `MA` (***Modify
662 | Arguments***):
663 | 
664 | 	MA: (v.s) e (MA.c) d => rplaca(e,v) c d
665 | 
666 | This instruction replaces the first element of the current environment by the
667 | argument list currently on the top of the S.
668 | 
669 | 
670 | ## Examples ##
671 | 
672 | 1. Generate the SECD code for the following:
673 | 	1. `(let (x y) (1 2) (* x y))`
674 | 	2. `let f = (let y = 4 in (->x| y * x)) in f(3)`
675 | 	3. `letrec f=(->n | if n then 1 else f(n - 1) + f(n - 2)) in f(3)`
676 | 	4. The towers of Hanoi
677 | 
678 | 	Compilation of `(let (x y) (1 2) (* x y))`:
679 | 
680 | 		(NIL *2 CONS *1 CONS LDF (LD (1.2) LD (1.1) MUL RTN) AP) ->
681 | 		(NIL LDC 2 CONS LDC 1 CONS LDF (LD (1.2) LD (1.1) MUL RTN) AP)
682 | 
683 | 	Compilation of `let f = (let y = 4 in (->x| y * x)) in f(3)`:
684 | 
685 | 		*(LET (F) ((LET (Y) (4) (lambda (x) (* y x)))) (f 3)) ->
686 | 		(NIL *(LET (Y) (4) (lambda (x) (* y x))) CONS LDF (LDC 3 CONS LD(1.1) AP RTN) AP)->
687 | 
688 | 		*(LET (Y) (4) (lambda (x) (* y x))) ->
689 | 		(NIL LDC 4 CONS LDF (*(lambda (x) (* y x)) RTN) AP)
690 | 
691 | 		*(lambda (x) (* y x)) ->
692 | 		(LDF (LD (1.1) LD (2.1) MUL) RTN)
693 | 
694 | 		------>
695 | 		(NIL NIL LDC 4 CONS LDF (LDF (LD (1.1) LD (2.1) MUL) RTN) AP CONS LDF (LDC 3 CONS LD(1.1) AP RTN) AP)
696 | 
697 | 	Compilation of `letrec f=(->n | if n then 1 else f(n - 1) + f(n - 2)) in f(3)`:
698 | 
699 | 		->
700 | 		*(LETREC (f) (LAMBDA (n) (IF n 1 (+ (f (- n 1)) (f (- n 2))))) (f 3)) ->
701 | 		(DUM NIL *(LAMBDA (n) (IF n 1 (+ (f (- n 1)) (f (- n 2)))) RTN) 
702 | 			 CONS LDF (LDC 3 CONS LD(1.1) AP RTN) RAP)
703 | 	
704 | 		*(LAMBDA (n) (IF n 1 (+ (f (- n 1)) (f (- n 2)))) RTN) ->
705 | 		(LDF (*(IF n 1 (+ (f (- n 1)) (f (- n 2)))) RTN))
706 | 
707 | 		*(IF n 1 (+ (f (- n 1)) (f (- n 2)))) ->
708 | 		(LD (1.1) SEL (LDC 1) *(+ (f (- n 1)) (f (- n 2))))
709 | 
710 | 		*(+ (f (- n 1)) (f (- n 2))) ->
711 | 		(NIL LDC 2 LD (1.1) SUB CONS LDF (LD (2.1) RTN) AP 
712 | 		    NIL LDC 1 LD (1.1) SUB CONS LDF (LD (2.1) RTN) AP ADD)
713 | 
714 | 
715 | 		------->
716 | 		(DUM NIL LDF (LD (1.1) SEL 
717 | 							   (LDC 1) 
718 | 							   (NIL LDC 2 LD (1.1) SUB CONS LDF (LD (2.1) RTN) AP 
719 | 								NIL LDC 1 LD (1.1) SUB CONS LDF (LD (2.1) RTN) AP ADD)
720 | 					  RTN) 
721 | 			 CONS LDF (LDC 3 CONS LD(2.1) AP RTN) RAP)
722 | 
723 | 
724 | 	Tower of Hanoi:
725 | 
726 | 		toh(n) = (lambda (n) 
727 | 					(let (l r m b) 
728 | 						 (n 0 0 nil)
729 | 						 (letrec (h) 
730 | 								 (lambda (n f t o b) 
731 | 										 (if (eq n 1)
732 | 											 (append (cons f t) b)
733 | 											 (h (- n 1) 
734 | 												o 
735 | 												t 
736 | 												f 
737 | 												(append (cons f t) 
738 | 														(h (- n 1)
739 | 														   f
740 | 														   o
741 | 														   t
742 | 														   b)))))
743 | 								 (h n l r m b)))) ->
744 | 		(LDF (*AAAAA RTN) RTN)
745 | 			where AAAAA = 
746 | 					(let (l r m b) 
747 | 						 (n 0 0 nil)
748 | 						 (letrec (h) 
749 | 								 (lambda (n f t o b) 
750 | 										 (if (eq n 1)
751 | 											 (append (cons f t) b)
752 | 											 (h (- n 1) 
753 | 												o 
754 | 												t 
755 | 												f 
756 | 												(append (cons f t) 
757 | 														(h (- n 1)
758 | 														   f
759 | 														   o
760 | 														   t
761 | 														   b)))))
762 | 								 (h n l r m b))) ->
763 | 		*AAAAA ->
764 | 		(NIL NIL CONS LDC 0 CONS LDC 0 CONS LD (1.1) CONS LDF (*BBBBB RTN) AP)
765 | 			where BBBBB = (letrec (h) 
766 | 								 (lambda (n f t o b) 
767 | 										 (if (eq n 1)
768 | 											 (append (cons f t) b)
769 | 											 (h (- n 1) 
770 | 												o 
771 | 												t 
772 | 												f 
773 | 												(append (cons f t) 
774 | 														(h (- n 1)
775 | 														   f
776 | 														   o
777 | 														   t
778 | 														   b)))))
779 | 								 (h n l r m b))
780 | 
781 | 		*BBBBB ->
782 | 		(DUM NIL *CCCCC CONS LDF (LD (2.4) CONS LD (2.3) CONS LD (2.2) CONS
783 | 			LD (2.1) CONS LD AP))
784 | 			where CCCCC = (lambda (n f t o b) 
785 | 							 (if (eq n 1)
786 | 								 (append (cons f t) b)
787 | 								 (h (- n 1) 
788 | 									o 
789 | 									t 
790 | 									f 
791 | 									(append (cons f t) 
792 | 											(h (- n 1)
793 | 											   f
794 | 											   o
795 | 											   t
796 | 											   b)))))
797 | 
798 | 		*CCCCC ->
799 | 		(LDF (*DDDDD RTN))
800 | 			where DDDDD = (if (eq n 1)
801 | 								 (append (cons f t) b)
802 | 								 (h (- n 1) 
803 | 									o 
804 | 									t 
805 | 									f 
806 | 									(append (cons f t) 
807 | 											(h (- n 1)
808 | 											   f
809 | 											   o
810 | 											   t
811 | 											   b))))
812 | 
813 | 		*DDDDD ->
814 | 		(LDC 1 LD (1.1) EQ SEL 
815 | 			(LD (1.5) ld (1.3) LD (1.2) CONS APPEND)
816 | 			(LD (1.5) CONS LD (1.3) CONS LD (1.4) CONS LD (1.2) CONS LDC 1 LD
817 | 				(1.1) SUB CONS LD (2.1) AP CONS LD (1.2) CONS LD (1.3) CONS 
818 | 					LD (1.4) CONS LDC 1 LD (1.1) SUB CONS LD (2.1) AP))
819 | 
820 | 
821 | 		-------->
822 | 		(LDF (NIL NIL CONS LDC 0 CONS LDC 0 CONS LD (1.1) CONS 
823 | 				LDF (DUM NIL 
824 | 					LDF (LDC 1 LD (1.1) EQ SEL 
825 | 										   (LD (1.5) ld (1.3) LD (1.2) CONS APPEND JOIN)
826 | 										   (LD (1.5) CONS LD (1.3) CONS LD (1.4) CONS 
827 | 										    LD (1.2) CONS LDC 1 LD (1.1) SUB CONS 
828 | 											LD (2.1) AP CONS LD (1.2) CONS LD (1.3) CONS 
829 | 											LD (1.4) CONS LDC 1 LD (1.1) SUB CONS 
830 | 											LD (2.1) AP JOIN) 
831 | 					     RTN) CONS 
832 | 					LDF (LD (2.4) CONS LD (2.3) CONS LD (2.2) CONS
833 | 						 LD (2.1) CONS LD AP) 
834 | 				     RAP RTN) AP
835 | 			  RTN) RTN)
836 | 
837 | 
838 | 	Hopefully...
839 | 
840 | 2. If one were to implement the SECD memory model out of a conventional memory
841 |    with 32-bit words, how many bits could be allocated to integers, and how big
842 |    could the memory be (in cells) if:
843 | 	* one memory word was one cell = `31 bits for integers, 15 bits for address`
844 | 	* two memory words made up one cell = `63 bits for integers, 31 bits for address`
845 | 
846 | 3. Write an SECD program for:
847 | 
848 | 		append(x,y) = if atom(x)
849 | 					  then y
850 | 					  else car(x).append(cdr(x),y)
851 | 
852 | 		(LETREC (a) (lambda (x y) (if (atom x)
853 | 									  y
854 | 									  (cons (car x) (a (cdr x) y))))
855 | 					a)
856 | 
857 | 	1. How many new memory cells are used during the compilation of `append((1
858 | 	   2), (3 4))`? = `2`
859 | 	2. If you had 100 000 cells available, how big a list could you append to
860 | 	   (3 4)? = `99998 if not counting those 2 used by (3 4)`
861 | 
862 | 
863 | 
864 | 			 
865 | 
866 | 
867 | 
868 | 
869 | 


--------------------------------------------------------------------------------
/08_memory_management_for_s_expressions.md:
--------------------------------------------------------------------------------
  1 | # Memory Management for S-Expressions #
  2 | 
  3 | Many cells are allocated by the SECD programs, and there is no possibility to
  4 | reuse existing cells yet. We need a so called ***Garbage Collector***.
  5 | 
  6 | A cell can be in 3 states:
  7 | 
  8 | * Free
  9 | * Allocated
 10 | * Garbage
 11 | 
 12 | ## Allocation ##
 13 | 
 14 | 	allocate(f) =
 15 | 		if no-more-free-space(f)
 16 | 		then allocate(garbage-collect(f))
 17 | 		else get-next-free-cell(f)
 18 | 		where
 19 | 			no-more-free-space(f) = T if no free cells available
 20 | 			and garbage-collect(f) collects all garbage cells
 21 | 			and get-next-free-cell(f) get next available free cell
 22 | 
 23 | ### Free Cells in Consecutive Locations ###
 24 | 
 25 | This is the simplest approach. The F registers stores a memory address below
 26 | which all cells are already used and above which all free cells are present.
 27 | 
 28 | ### Free Cells Individually Marked ###
 29 | 
 30 | Each cell contains in its tag field an extra bit - the ***mark bit***,
 31 | which has two values - *free* and *allocated*. Initially all cells are marked
 32 | free, and are marked allocated when used. If some garbage collection mechanism
 33 | determines that the allocated cell is an fact garbage, it can set the flag to
 34 | free again and the cell will be used by allocation again.
 35 | 
 36 | Allocate function is more complicated now, it has to scan through memory
 37 | checking the mark bits.
 38 | 
 39 | ### Linked Free List ###
 40 | 
 41 | A garbage collector linked all free cells into a ***free list***. The
 42 | F register points to the first cell of the list. Allocation is made by getting
 43 | the car(F) and F is replaced by cdr(F).
 44 | 
 45 | ## Mark-Sweep Collection ##
 46 | 
 47 | The basic approach is to mark all cells as free and then trace through all
 48 | cells which are in use (all cells referenced by any of the SECD registers) and
 49 | mark them. 
 50 | 
 51 | The basic problem is that we are running out of memory, how can we afford to
 52 | scan all registers? Simple solution is to reserve a finite length
 53 | memory for a stack, and when the stack overflows, we can start over and seach
 54 | for black cells which have white children. Improvement would also be to
 55 | remember the smallest address that is 'forgotten'.
 56 | 
 57 | Entirely different technique called ***Deutsch-Schorr-Waite algorithm*** avoids
 58 | the stack altogether by reversing the pointers. An additional information is
 59 | needed in each cell - ***direction tag***.
 60 | 
 61 | 	mark(i, parent, c) = "color all cells accessible from i as c"
 62 | 		if mark-bit(i) = c
 63 | 		then backup(i, parent, c)
 64 | 		else let ix = color(i, c)
 65 | 			 in if tag(i) = terminal
 66 | 			 then backup(i, parent, c)
 67 | 			 else let child = car(i) 
 68 | 				  in mark(child, rplacdtag((rplaca(i, parent), A) c))
 69 | 
 70 | 	backup(child, parent, c) = "back up from child"
 71 | 		if null(parent) then c
 72 | 		else let dtag = direction-tag(parent)
 73 | 		     and pcar = car(parent)
 74 | 			 and pcdr = cdr(parent)
 75 | 			 in if dtag = A "see which parent field was reversed"
 76 | 				then "reset parent's car and then reverse parents cdr"
 77 | 					mark(pcdr, 
 78 | 						 rplactag(
 79 | 							rplacd(rplaca(parent, child), pcar), D), 
 80 | 						 c)
 81 | 				else "reset parent's cdr and back up"
 82 | 					backup(rplacd(parent, child), pcdr, c)
 83 | 
 84 | 
 85 | ## Parallel Mark-Sweep ##
 86 | 
 87 | Above solutions are so called *stop the world* algorithms. The execution has to
 88 | be stopped in order to collect garbage. An incremental variation of mark-sweep
 89 | exists. A third color has to be added - ***gray***. Simple marking algorithm is
 90 | now:
 91 | 
 92 | 	mark(i, c) = "color all cells accessible from i as c"
 93 | 		if mark-bits(i) = c "stop on marked cell"
 94 | 		then next-gray(0, c) "and start scan from 0"
 95 | 		elseif mark-bits(i) = gray
 96 | 		then mark(cdr(color(i, c)), c)
 97 | 		else let ix = color(i, c)
 98 | 			 in if tag(i) = terminal
 99 | 			 then next-gray(0, c)
100 | 			 else let iy = color(car(i), gray)
101 | 				  in mark(cdr(ix), c)
102 | 	where next-gray(i, c) =
103 | 		if i > top-of-memory
104 | 		then c
105 | 		elseif mark-bits(i) = gray
106 | 		then mark(i, c)
107 | 		else next-gray(i + 1, c)
108 | 
109 | * White - represents free or potentially free cells
110 | * Gray - represents cells touched by the gc, but their car or cdr has not yet
111 |   been touched
112 | * Black - represents cells that have been fully traced
113 | 
114 | *Note: mutator cannot arbitrarily change links from black cells.*
115 | 
116 | ## Reference Counts ##
117 | 
118 | Another solution is to keep a count of references pointing to the cell in its
119 | header. Whenever a new reference is created (e.g. using cons function), the count is
120 | incremented, whenever the cell is abandoned (e.g. rplaca function), the count is decremented.
121 | 
122 | It has its drawbacks:
123 | * inability to guarantee a bounded amount of time on each operation that
124 |   modifies the count
125 | * large number of bits required in the header
126 | * loops can cause instabilities
127 | 
128 | ## Compacting Collectors ##
129 | 
130 | Instead of having a linked list of free cells, a garbage collector actually
131 | moves the cells in the memory to ensure consecutive free space. ***Baker's
132 | algorithm*** is one of more often used, even in production systems.
133 | 
134 | The basic idea of the algorithm is to split the memory into two equally large
135 | halves - ***hemispaces***, and to use only one of them. The one currently in
136 | use is called ***fromspace*** and the other is called ***tospace***. When the
137 | fromspace is completely full, the living cells are evacuated into
138 | ***tospace***. When evacuation is finished, the spaces switch and old fromspace
139 | becomes new tospace. As a biproduct of evacuation, cells are nicely compacted.
140 | 
141 | Role of the F register of the SECD Machine takes ***CP*** (***Creation
142 | Pointer***). On the opposite end there is the ***EP*** (***Evacuation
143 | Pointer***). 
144 | 
145 | Evacuating a cell means moving its exact copy to tospace. All existing cells
146 | which point to this cells don't know the cell moved. This is solved by setting
147 | ***invisile pointer*** or ***reference pointer***. Any reference to this cell
148 | is invisibly forwarded into tospace. 
149 | 
150 | *Note: As I already know about Baker quite a bit, I skipped a lot of stuff from the
151 | book. If interested, read the fantastic paper **Uniprocessor garbage collection
152 | techniques** by Paul Wilson where you'll find pretty much everything you need
153 | to know about garbage collection.*
154 | 
155 | ### Region Collectors ###
156 | 
157 | Baker has issues:
158 | 
159 | * There is no distinction between very long-lived data (such as the root of the
160 |   environment) and very short-lived data (such as top of the stack). Both are
161 |   copied at each flip.
162 | * The nice performance characteristics fall apart when actual storage in use
163 |   nears or exceeds one-half of the total available storage.
164 | 
165 | Both issues can be solved by splitting the memory into ***regions***. A region
166 | is quite small, few pages of about 1K to 4K each. 
167 | 
168 | *Note: Again, check the paper by Paul WIlson*
169 | 
170 | A problem is, that there may be pointers from one region to another, and after
171 | freeing a condemned region such older regions may be left with a ***dangling
172 | pointer***. The most common solution to this is to append to each region an
173 | ***entry table*** which contains an entry for each forward pointer reference
174 | from an earlier region to this one. This entry includes an invisible pointer to
175 | the appropriate cell in the region and a backward pointer to the cell in the
176 | earlier region making the forward reference. The actual forward pointer in this
177 | latter cell is to the entry table entry, not the desired cell. 
178 | 
179 | ## Alternative List Representations ##
180 | 
181 | Ever since their invention, s-expressions have been a magnet for clever
182 | implementations other than the simple tag-car-cdr representations used to this
183 | point. These ideas fall into three categories:
184 | 
185 | * Methods to compact the amount of storage needed to represent a s-expression
186 | * Techniques for implementing tag bits
187 | * Approaches to list representation that avoid pointers altogether
188 | 
189 | ### List Compaction ###
190 | 
191 | In many cases, the space taken up by the car and cdr pointer exceeds the number
192 | of bits of the information pointed to. This was recognized early on, and
193 | techniques called ***car coding*** and ***cdr coding*** were developed.
194 | 
195 | ***Car coding*** comes from the observation, that most of the car cells point
196 | to constants. Significant savings are thus possible if the constant value
197 | replaces car pointer. This requires an extra tag bit to permit the machine to
198 | distinguish a car pointer from car constant. 
199 | 
200 | For ***cdr coding***, very often car pointer points to the next word in memory.
201 | Second often occurence is cdr pointing to nil. We need two bits to express
202 | this information. 
203 | 
204 | So up until now, each cell holds in its tag field:
205 | 
206 | * Memory management bits (2)
207 | * Indicators of the type of value stored in the value field
208 | * A representation of what the cdr of this field would look like if it were
209 |   a cons cell
210 | 
211 | 
212 | ### Tag Implementation Techniques ###
213 | 
214 | In short, we have 3 approaches:
215 | 
216 | * Store tags together with the data
217 | * Store tags on one place and data elsewhere
218 | * Throw away the tags completely and store different objects into different
219 |   areas of the memory (first 1mb of memory contains only integer constants,
220 |   then 1mb of floating point numbers etc.)
221 | 
222 | ### Pointer Avoidance ###
223 | 
224 | It is not true that the only way to implement s-expressions is with pointers.
225 | This section gives several interesting approaches based on explicit labelling.
226 | 
227 | * The root node of an s-expression is assigned the number 1
228 | * For each nonterminal, if its label is k:
229 | 	* the label for the node's car subnode is 2 * k
230 | 	* the label for the node's cdr subnode is 2 * k + 1
231 | 
232 | Now whole s-expression can be stored in an array.
233 | 
234 | Without the proof, the following is an algorithm that takes any composition of
235 | cars and cdrs and computes the binary form of such an index as follows:
236 | 
237 | 1. Start with a leading 1
238 | 2. For the sequences of **a**s and **d**s in `c{ad}+r` create a sequence of
239 |    1 for each d and 0 for each a.
240 | 3. Reverse this sequence
241 | 4. Append this sequence to the leading 1.
242 | 
243 | For example, the function car translates into binary number 10, or index 2.
244 | Cadaddr should return an element at index 58 (111010).
245 | 
246 | 
247 | 
248 | 
249 | 
250 | 


--------------------------------------------------------------------------------
/10_lisp_variations_and_implementations.md:
--------------------------------------------------------------------------------
  1 | # Lisp: Variations and Implementations #
  2 | 
  3 | A brief history of Lisp implementations:
  4 | 
  5 | * The original ***Lisp*** (late 1950s) at MIT
  6 | * ***LISP 1.5*** in the early 1960s as the first standard
  7 | * ***MacLISP*** late 60s, an MIT upgrade (see Pitman, 1983)
  8 | * ***InterLISP*** early 70s, a West Coast variant (see Teitelman, 1978)
  9 | * ***ZetaLISP*** and ***LISP Machine LISP*** late 70s, commercial variants of
 10 |   MacLISP
 11 | * ***SCHEME*** mid 70s, a majo LISP variant much closer to lambda calculus (see
 12 |   Spring and Friedman, 1990)
 13 | * ***Portable Standard LISP (PSL)*** early 80s, an efficient version of Lisp
 14 |   from the University of Utah that was largely written in itself and is easily
 15 |   transported to new computers (Griss, 1983)
 16 | * ***Franz LISP*** early 80s, another variant of MacLISP optimized to run in
 17 |   UNIX (AT&T) environments (see Wilensky 1984)
 18 | * ***Common LISP*** early 80s, standardized combination of many of the previous
 19 |   variants (see Steele, 1984)
 20 | * ***MultiLISP*** mid 80s, Scheme with explicit support for parallelism (see
 21 |   Halstead, 1985, 1986)
 22 | 
 23 | For the reader interested in trying his or her hand at actually implemting
 24 | a Lisp system, we recommend Allen (1978), Henderson (1980), and Henderson et
 25 | al. (1983).
 26 | 
 27 | ## The Original Lisp ##
 28 | 
 29 | The original Lisp was created by John McCarthy with &lambda; - Calculus in
 30 | mind. The basic computational model is that of expressions build by applying
 31 | functions to other expressions. Functions are themselves expressible in the
 32 | same format as any other data objects. Functions can be passed to and returned
 33 | by other functions.
 34 | 
 35 | Lisp is not a pure functional language. Operations that are integral to the
 36 | language permit different values to be bound to the same variable at different
 37 | times. The same symbol can have many different values at the same time, each of
 38 | which is accessed in a different way.
 39 | 
 40 | Functions are also not quite first class citizens in Lisp. While they can be
 41 | passed as arguments, doing so requires somewhat special coding and care must be
 42 | taken to avoid strange funarg problems. 
 43 | 
 44 | Expression evaluation is not in total agreement with our previous functional
 45 | model. Many Lisp forms include variants where a sequence of expressions are
 46 | evaluated ina a very specific sequential fashion. All but the last of these are
 47 | executed for their *side effects*.
 48 | 
 49 | ### Major Language Components ###
 50 | 
 51 | Lisp has a variety of atomic data types: *integers, floating-point numbers,
 52 | literals, character strings, booleans, big-nums*. Major data structures
 53 | supported by Lisp are s-expressions. But arrays are supported as well.
 54 | 
 55 | ### Variable Scopes and the Property List ###
 56 | 
 57 | So far we were working with ***statically scoped variables***. The expressions
 58 | that give them values can be determiined by a review of the static program
 59 | code. 
 60 | 
 61 | In List, a programmer can define certain identifiers to be ***dynamically
 62 | scoped***. The expression that bind values to them cannot be determined until
 63 | runtime. Such variables come in two forms. ***Global variables*** or ***special
 64 | variables***, which are visible to all expressions  whenever they are executed
 65 | in the program (unless they are masked by a local variable of the same name).
 66 | ***Program variables*** are dynamically allocated at the beginning  of
 67 | execution of certain special forms and then act like global variables until
 68 | that form completes. 
 69 | 
 70 | There are two standard ways of implementing binding for such variables. First
 71 | is via an association list (called ***deep binding***),  which is quite slow.
 72 | The second approach is called ***shalow binding*** and the main idea is to
 73 | associate some unique global storage with the variable. Today, deep binding
 74 | predominates. 
 75 | 
 76 | ### Built-in Pure Functions ###
 77 | 
 78 | There are many of them, interesting are:
 79 | 
 80 | * `(QUOTE E)` returns its argument as unevaluated s-expression
 81 | * `(SPECIAL (V1 ...Vn))` make V1 ... Vn special variables
 82 | * `(UNSPECIAL (V1 ... Vn)` releases associated storage
 83 | 
 84 | ### Modifying Values ###
 85 | 
 86 | All these functions have form of:
 87 | 
 88 | 	(<keyword> <address-expr> <value-expr>)
 89 | 
 90 | * `SET` sets a value into program variable
 91 | * `SETQ` the same as SET, but address-expr is not evaluated
 92 | * `CSET` sets a value into a global variable
 93 | * `CSETQ` 
 94 | * `rplaca`
 95 | * `rplacd`
 96 | * `nconc` non copying concatenate
 97 | 
 98 | ### Special Forms ###
 99 | 
100 | A ***special form*** is an expression whose function is a special keyword known
101 | to the system, but unlike the built-ins, order of argument evaluation is
102 | special.
103 | 
104 | * `(LAMBDA (<id>*) <body)` a function expression
105 | * `(DEFINE ((<id> <expr>)*))` global function definition
106 | * `(LABEL <id> <expr>)` global recursive function
107 | * `(AND <expr>+)` evaluate expressions until first F
108 | * `(OR <expr>+)` evaluate expressions until first T
109 | * `(COND (<test expr> <body>)*)` nested conditionals
110 | * `(PROG (<id>*) <progbody>)` sequential form execution
111 | * `(GO <id>)` branch inside of PROG
112 | * `(RETURN <expression>)` exit `<body>`
113 | 
114 | ### System Functions ###
115 | 
116 | LISP has visible to the programmer a variety of direct hooks into its internal
117 | operation. `EVAL` and `EVALQUOTE` both accept one argument which should be an
118 | s-expression and return the result of evaluating the expression. `EVALQUOTE`
119 | does not evaluate its argument before it evaluates it.
120 | 
121 | A Lisp compiler can be invoked using `(COMPILE (<id> <expr>)*)`. Now whenever
122 | such an identifier is found in the function position of an application, the
123 | compiled code will be invoked with its arguments evaluated.
124 | 
125 | ***Macros*** are functions which, when they appear as a function in an
126 | expression passed to the compiler, are executed by the compiler before the code
127 | is executed. Arguments to the macros are not evaluated. The s-expression
128 | resulting from executing macro is passed back to the compiler for the
129 | compilation.
130 | 
131 | ## Scheme - a Pure Lisp ##
132 | 
133 | Functions are truly first class citizens, identifiers are statically scoped,
134 | tail recursive expressions are truly optimizable, and a  great deal of
135 | attention has been given to a clean continuation mechanism.
136 | 
137 | ### Scheme Syntax ###
138 | 
139 | 	<basic-form> := <constant>
140 | 					| <identifier>
141 | 					| <LAMBDA <arg-list> <body>
142 | 					| (<basic-form> <basic-form>*)
143 | 					| (QUOTE <expression>)
144 | 					| (IF < basic-form> <basic-form> <basic-form>)
145 | 					| (begin <body>)
146 | 					| (SET! <identifier> <basic-form>)
147 | 
148 | 	<body> := <basic-form>+
149 | 	<arg-list> := (<identifier>*)
150 | 					| <identifier>
151 | 					| (<identifier>+.<identifier>)
152 | 
153 | ### Extended Syntax ###
154 | 
155 | The basic evaluator understands only the above syntax. To extend the spectrum
156 | of special forms seen by the programmer, Scheme employs a very elegant
157 | mechanism that is an outgrowth of the ***macro*** mechanism of the original
158 | LISP, but that is more fully integrated into both compilation and
159 | interpretation.
160 | 
161 | This mechanism has two parts, one records how a new special form should be
162 | translated into an expression and a function that does the actual translation
163 | into something the standard eval can handle.
164 | 
165 | The latter is called ***system syntax expander*** and consists of a database of
166 | ***pattern/expansion pairs*** and a function to search and use them. This
167 | function, ***expand***, takes a single unevaluated expression as an argument
168 | and looks recursively at any keywords found in the fnuction position of the
169 | expression itself or any nested subexpression. After the successfull expansion,
170 | the interpreter/compiler would be recalled on the expanded expression. 
171 | 
172 | ### Bindings ###
173 | 
174 | As with the original LISP, identifiers in Scheme appear in two kinds, lambda
175 | arguments and others. Lambda arguments are statically scoped. The other
176 | mechanism is the `DEFINE` form. When executed at the top level, the form
177 | defines ***global variables***. When executed inside of `LAMBDA` expression, it
178 | auguments the current environment defined by lambda to include space for the
179 | new variable and initializes it. After return from lambda, the binding is
180 | discarted.
181 | 
182 | The approach taken to handle global variables is to define a specially globally
183 | accessible ***fluid environment*** for ***fluid variables***:
184 | 
185 | 	(FLUID-LET ((<identifier> <expression>)*) <body>)
186 | 
187 | Any function executed in this body, even one wohose definition is not withing
188 | the `FLUID-LET`'s scope, can refer to a fluid variable and get the current
189 | value.
190 | 
191 | ### Suspending Evaluation ###
192 | 
193 | Clean treatment of function objects in Scheme permits inclusion of three
194 | different mechanisms for suspending and resuming evaluation: ***delay*** and
195 | ***force*** functions for supporting futures, and enhanced ***continuation***
196 | implementation that permits sophisticated co-routining, and an ***engine***
197 | mechanism that permits specification and control of multitasking of individual
198 | Scheme evaluations.
199 | 
200 | ***Engine*** is a device that contains a continuation for some expression
201 | which, when activated, runs that continuation for some specified period of
202 | time. The Programmer has total control over how long and what to do if the
203 | computation does not complete in that time. This permits elegant
204 | multiprogramming executives to be built directly in Scheme without resorting to
205 | any lower level programming languages. 
206 | 
207 | ### Implementation ###
208 | 
209 | Identification of a small core language and an integrated way of extending the
210 | language from that code means that optimization techniques can focus on
211 | a relatively small set of forms. For the most part the compiler can determine
212 | at compile time exactly what binding is associated with most symbols in the
213 | body of an expression. This permits a compiler to know exactly when a set of
214 | bindings is no longer useful. In turn, this means that much of a function's
215 | environment can be kep in an easily managed stack. ***Tail recursion*** can be
216 | implemented optimally by branches back to the beginning of a routine and by
217 | reusing the same set of memory cells for each set of argumentsin the sequence
218 | of calls. 
219 | 
220 | One of the first Scheme compilers was ***Rabbit*** (Steele, 1978). A good
221 | compiler for the core set of forms was written in MacLISP, which generated LISP
222 | code that could be compiled by the MacLISP compiler. This was then rewritten in
223 | Scheme with optimization features added to it. Entire compiler came to about 50
224 | pages of code. 
225 | 
226 | ## Common Lisp - a Modern Standard ##
227 | 
228 | The multiplicity of LISP dialects was a healthy aspect of its first 25 years of
229 | existence, as its primary users were academics. By the early 80s it had become
230 | apparent that an industrial strength LISP was needed. ***Common Lisp*** is the
231 | result.
232 | 
233 | It includes 7 types of numbers, with at least 7 more of complex numbers. Two
234 | types of characters, bits, symbols, random states, unreadable data objects,
235 | common round out the atomic types, structures, types and subtypes, keyword
236 | arguments, multiple results from functions, exceptions:
237 | 
238 | 	(CATCH <tag> <body>)
239 | 	(THROW <tag> <expression>)
240 | 
241 | It inherits static scoping, and special variables with scope similar to
242 | Scheme's fluid variables. It supports streams.
243 | 
244 | ***Evaluation*** is supported by `eval`, which evaluates the form in the
245 | current special environment and a nil static environment. A variation of this
246 | `(EVAL-WHEN {compile | load | eval} <form>)` specifies when the form should be
247 | evaluated. 
248 | 
249 | ## MultiLisp - a Parralel Lisp ##
250 | 
251 | ***MultiLisp*** is a Scheme extension designed with parallelism in mind. It
252 | adds several new forms. ***PCALL*** (***Parallel CALL***) has n + 1 arguments
253 | all of which the programmer is willing to have evaluated in any order. When all
254 | of them are computed, they are recombined as a normal MultiLisp expression and
255 | evaluated. The next form is ***DELAY***, which takes its argument, packs it in
256 | a closurelike structure, which can be passed around. When the value of the
257 | argument is needed, it is ***forced*** and resulting value is substitued for
258 | the closure.
259 | 
260 | ***FUTURE*** is a form similar to DELAY, but after its creation, processor can
261 | evaluate it in its free time.
262 | 
263 | ### Implementation ###
264 | 
265 | The first implementation was on the ***Concert*** machine at MIT. The MultiLisp
266 | programs were compiled into a SECD-like abstract machine ISA called
267 | ***MCODE***, which was then interpreted by an interpreter written as 3000 lines
268 | of C. 
269 | 
270 | MCODE programs manage data structures called ***tasks*** that are accessed by
271 | 3 pointers: a program pointer, stack pointer, and environment pointer. 
272 | 
273 | The FUTURE function creates a new task and leaves it accessible for any free
274 | processor. Deciding which task to run is done using an ***unfair scheduling
275 | policy***.
276 | 
277 | Garbage collection is distributed across all processors. Each has its own set
278 | of ***semispaces*** and employs a variant of Baker's algorithm. All processors
279 | synchronize their flip, and a ***lock bit*** is associated with cell to prevent
280 | it to be evacuated into multiple tospaces. 
281 | 
282 | A later version of MultiLisp called ***Butterfly LISP*** has been implemented
283 | for the ***Butterfly*** parralel processor (Steinberg 1986, Allen 1987).
284 | 
285 | ## The CADR Machine ##
286 | 
287 | The first machine build for Lisp specifically was ***CONS Machine*** at MIT 
288 | 1975 running MacLisp. CONS machine, althrough operational, had a variety of
289 | weaknesses. In 1978 a new version, ***CADR Machine*** replaced it at MIT. It
290 | represented an important point in computer architecture for specific languages
291 | history and drove the design of many sytems, both academic and commercial. 
292 | 
293 | Some of the specific characteristics of the CADR:
294 | 
295 | * Optimization for high-performance, single user interaction
296 | * LISP as the primary language for applications, interpreters, compilers,
297 |   operating system functions.
298 | * Data type checking support in hardware
299 | * Large memories (for the time)
300 | * Built-in memory allocation and garbage collection
301 | 
302 | ![CADR Machine organization](figures/10_15_cadr_machine.jpg)
303 | 
304 | Major hardware support was included for a ***push-down list***,
305 | or ***PDL buffer***, which would keep up to the top 1K entries of a stack very
306 | close to the main dataflow. This stack served many of the functions of the S,
307 | E and D stacks in the SECD Machine. Invoking new functions involved
308 | constructing ***frames*** ot control information on it that contained:
309 | 
310 | * a pointer backwards in the stack to the previous frame
311 | * cells for argument values
312 | * cells for local variables and constants
313 | * cells for special variable access
314 | 
315 | Operands for built-in functions invoked insie a function's body were
316 | push/popped from the stack space in front of the current frame. 
317 | 
318 | ### Memory Structure ###
319 | 
320 | Memory in the CADR machine consists of a single 16-million word virtual address
321 | space, with up to 1 million words of real memory. Thus, data and instruction
322 | addresses in their complete form take 24 bits.
323 | 
324 | The format of an individual memory cell reflects SECD influences. Each 32 bit
325 | long cell contains a tag, a cdr code, and a value field.
326 | 
327 | ![a CADR memory word](figures/10_16_memory_word.jpg)
328 | ![Sample CDR coding and symbol object](figures/10_17_cdr_coding.jpg)
329 | 
330 | Different types of objects are kep in different areas of the memory, not for
331 | ***BIBOP*** like tagging, but for symplifying garbage collection. ***Garbage
332 | collector*** is incremental Baker's algorithm. With every CONS some few cells
333 | are copied to another region. 
334 | 
335 | ### Program Forms ###
336 | 
337 | Programs for this machine can be represented in three forms - ***MacLISP***,
338 | ***microcode***, and ***macrocode***. Microcode provides most of the major
339 | system functions (virtual memory management, GC etc.). The middle level
340 | - macrocode - supports stacks much as the SECD Machine did, but in a format
341 | more resembling conventional instruction set. The general form of such
342 | instruction is
343 | 
344 | 	<operation>, <operand>, <destination>
345 | 
346 | where the operation is usually a very generic one (add, cons ...). Individual
347 | macroinstructions include a variety of wrinkles that improve performance (for
348 | example when creating a list of known length etc.).
349 | 
350 | ## CADR Derivatives ##
351 | 
352 | Althrough the CADR Machine was designed as a one-of-a-kind research machine, it
353 | spawned a series of commercial derivatives that were oprimized for LISP
354 | execution in a high performance, single-user workstation environment. 
355 | 
356 | ### The LMI Lambda LISP Processor ###
357 | 
358 | ***Lambda LISP Processors*** by Lambda Machines, Inc., were some of the first
359 | commercially available machines tailored for Lisp. They were fairly close to
360 | CADR based designs with following additions:
361 | 
362 | * a richer set of tag values
363 | * two memory word sizes, 32bit size that supports a 25 bit virtual address
364 |   space, and a 40 bit size that supports 32 bit address space.
365 | * larger PDL stack buffers up to 2K words
366 | * an internal 4K word A memory, which can be used by microcode for internal and
367 |   temporary results
368 | * a sectored 4K word cache in front of memory
369 | * a standardized bus, the ***Nubus***, to connect the back of the cache to
370 |   memory and I/O cards
371 | * larger (64bit) microinstruction word length
372 | * virtual microstore mechanism that permits up to 8K words of microprograms to
373 |   be pages ina as needed from a larger 64K word space on disk
374 | * Direct HW support for many arithmetic functions
375 | * a faster, 100ns clock speed
376 | * a motorola 68000-series microprocessor as a coprocessor for many operating
377 |   system and console functions
378 | 
379 | ### The Texas Instrucments Compact LISP Machine ###
380 | 
381 | The overall architecture of the machine resembles that of the Lambda machine,
382 | with the major difference being in the cache (a more modern, two-way set
383 | associative design), in support for garbage collection (8 status bits for each
384 | 8K-word region of memory), and in the use of page bits to extend the real
385 | memory address space to the full capacity of the Nubus, namely, 1 gigaword.
386 | 
387 | ![Compact LISP Machine](figures/10_18_compact_lisp_machine.jpg)
388 | 
389 | The processor is also pipelined, with the capability of initiating a new
390 | microinstruction each microcycle. A goal of 25ns per clock cycle offers
391 | performance substantially in excess of earlier machines.
392 | 
393 | ### The Symbolics 3600 ###
394 | 
395 | These machines range from cabinet-sized machines with multiple cards to support
396 | the basic CPU to modern custom VLSI designs that capture most of the same logic
397 | in a single chip (the ***Ivory Processor***).
398 | 
399 | Althrough built for ***ZetaLISP***, it also adapts well to Common Lisp, and to
400 | supporting both traditional languages like Fortran and Prolog.
401 | 
402 | Both the architecture of the machine and ZetaLISP permit separate processes to
403 | be created. The current state of each process is foverned by a ***stack
404 | group*** consisting of three stack pointers and their associated areas in
405 | memory. The ***Control Stack*** represents a combination of all four of SECD
406 | Machine registers. It contains a ***frame*** for each nontrivial function
407 | application that is still pending. There are two pointers - ***Stack Pointer***
408 | and ***Frame Pointer***. 
409 | 
410 | Function calls are handled similarly to modern standards, a frame is created,
411 | arguments evaluated one at a time, and return instruciton discards the frame
412 | and passes execution to the parent frame. The return values replace the
413 | arguments on the stack. The overhead for just the CALL and RETURN instructions
414 | was as little as 20 machine cycles.
415 | 
416 | The second stack is the ***binding stack*** which supports ***shallow
417 | binding***. When required, the current bindings of such variables are pushed
418 | onto this stack in 2-word pairs. The first of these is a pointer to the special
419 | variable's value cell, and the second is the value at the time of the push. The
420 | cdr-code field of these cells is used to designate such things as use of the
421 | variables in a closure. Completion of an expression that releases a set of
422 | special bindings causes this stack to be popped an appropriate number of times,
423 | with each pair of words giving an address of a cell to change and the value
424 | that it should resume. Information in the stack frame indicates how much of the
425 | binding stack should be unwound at each function return.
426 | 
427 | The final stack is the ***data stack***, which is akin to a heap in
428 | a conventional machine. 
429 | 
430 | 3600 had a sophisticated descendant of Baker's algorithm. Given the potential
431 | size of the memory space, 256M words, and its structure as a virtual store
432 | where perhaps one-thirtieth of all memory is in fast RAM and the rest is on the
433 | disk, it is impossible to conceive a mark-sweep type of algorithm. The space is
434 | simply too big. 
435 | 
436 | Several solutions are employed. First is to separate memory into ***areas***.
437 | The ***ephemeral area*** contains object whose lifetime is likely short. This
438 | area is scavenged and compacted frequently. Next is ***dynamic area*** with
439 | object living longer, but not for the indefinite time. Globals and special
440 | variables fall into this category. Collection occurs only when it has exceeded
441 | some predetermined capacity. Finally there is ***static area***, whose objects
442 | are assumed to exist forever. System code and tables for example. Garbage
443 | collection usually occurs only when user commands.
444 | 
445 | To prevent accidental collection of ephemeral objects requires knowing when any
446 | reference to such object is stored elsewhere. This was done by the hardware by
447 | monitoring each store into memory. If the value being stored is address and it
448 | points to the ephemeral area, a bit associated with the page containing the
449 | location being modified is set. Conceptually, collecting the ephemeral area
450 | requires scavenging all pages whose bit is set for references to that area. 
451 | 
452 | The actual process of scavenging and copying is also done to minimize future
453 | work. Tospace is incrementally increased as needed.
454 | 
455 | ## The SPUR RISC ##
456 | 
457 | One of the strongest trends in current computer architecture is towards
458 | ***Reduced Instruction Set Computers*** (***RISC***). In general RISCs have
459 | following characteristics:
460 | 
461 | * a small number of simple instructions
462 | * a large number of registers
463 | * most, if not all, access to memory is through just load and store (i.e. no
464 |   memory to register adds, etc.)
465 | * almost all operations like add, compare, etc., are register to register, with
466 |   the result going back into a register
467 | * in most cases new instructions can be started at a rate of one per machine
468 |   cycle
469 | * an overall architecture which made generating optimized machine code from
470 |   a high-level language easy
471 | 
472 | One of the first such RISCc was the ***Berkeley RISC-I***, shortly followed by
473 | ***RISC-II***, which included:
474 | 
475 | * ***register windows*** where registers accessible to the program are just
476 |   part of a larger stach of registers, and at each call to or return from
477 |   a subprogram, all or part of this set slides down and up.
478 | * ***delayed branches*** where one or more instructions immediately following
479 |   a branch will be executed even if the branch is taken.
480 | * an efficient ***trap mechanism*** that will suspend execution of a program
481 |   when certain circumstances have been detected, and start up some prespecified
482 |   routine in its place.
483 | 
484 | ![Register windows in the RISC-II](figures/10_22_register_windows.jpg)
485 | 
486 | The ***SPUR*** is a variant of RISC-II optimized for handling languages such as
487 | Lisp, Prolog, or Smalltalk. Each register and memory cell is 40 bits, with
488 | 6 bits for tags, and 2 holding ***generation number*** used by the GC. 
489 | 
490 | ![Partial SPUR instruction set](figures/10_23_spur_instructions.jpg)
491 | 
492 | The trap facilities are used to advantage by streamlining many instructions to
493 | work only for the *expected* cases, and trap to predefined routines to handle
494 | the atypical ones. The net effect of this is that most LISP functions compile
495 | very easily into very short sequences of SPUR code that execute ina very few
496 | machine cycles (CONS for example, compiles into 4 instruction sequence). Again,
497 | special conditions such as stack overflow are handled by setting up
498 | *inaccessible pages* at the ends of the allocated stack and letting a memory
499 | management trap handle the cases where something must be done. 
500 | 
501 | Projected LSIP benchmark performance for 150-ns SPUR indicates that with the
502 | exception of heavy floating-point benchmarks, it is as much as 4.9 times faster
503 | than a Symbolics 3600 (with a slightly slower cycle time) and more than 10
504 | times faster than a DEC VAX 8600 (which issues instruction twice as fast as the
505 | SPUR).
506 | 
507 | ## Benchmarking ##
508 | 
509 | Actual comparisons of these machines was written by Gabriel (1984):
510 | ***Performance and Evaluation of Lisp Systems***.
511 | 
512 | 
513 | 
514 | 
515 | 


--------------------------------------------------------------------------------
/13_logic_overview.md:
--------------------------------------------------------------------------------
  1 | # A Logic Overview #
  2 | 
  3 | The primary kind of statement with which logic computing deals is a ***fact***.
  4 | A ***fact*** is an expression that some object or set of objects satisfies some
  5 | specific relationship.
  6 | 
  7 | ![Logic computing](figures/13_01_logic_computing.jpg)
  8 | 
  9 | ## Formal Logic Systems ##
 10 | 
 11 | ![A logic system](figures/13_05_logic_system.jpg)
 12 | 
 13 | The ***sytax*** of a logic model revolves around ***logical expressions***
 14 | built up from a set of more basic ***symbols***. A subset of symbol strings
 15 | representing valid expressions is called ***well-formed formulas*** (or
 16 | ***wffs***). Individually, they are called ***statement, proposition,*** or
 17 | ***sentence***.
 18 | 
 19 | Typisal syntaxes include allowances for:
 20 | 
 21 | * ***constants*** (specific objects)
 22 | * ***functions*** applied to such objects to yield other objects
 23 | * ***predicates*** to test membership of tuples of objects in relations
 24 | * ***identifiers*** or ***logic variables*** stand for *unknown* objects
 25 | * other operations which combine these or place constraints on values that
 26 |   various variables may take on.
 27 | 
 28 | The ultimate purpose of a system of logic is to divide the universe of wffs
 29 | into pieces; those to be considered true, those that are false, and those about
 30 | which no absolute statement can be made.
 31 | 
 32 | In most real logic programming languages the user provides ***axioms***, rules
 33 | that are considered true. 
 34 | 
 35 | ### Rules of Inference ###
 36 | 
 37 | Axiom specification provides only part of the information needed. In a properly
 38 | written program, other parts of the language's semantics extend these
 39 | statements to cover other statemenets which fully define the true and false
 40 | sides of the wff set. This extension is done via ***inferencing***, which uses
 41 | specific patterns of known sets of wffs to predict or deduce the veracity of
 42 | other wffs. Each such pattern is called an ***interence rule***. 
 43 | 
 44 | The actual process of inferencing involves finding some inference rule `R`,
 45 | some subset `X` of wffs already in the true set, and some other wff `a`
 46 | such that `X + a = R`. Given our definition of `R` as a set of wffs, this means
 47 | that the wff `a` should also be considered true. We say that `a` is an
 48 | ***inference, logical consequence, conclusion*** or ***direct consequence***
 49 | from the set `X` using the inference rule `R`. 
 50 | 
 51 | Common inference rule is ***modus ponens***. This rule takes some wff `A` and
 52 | some other wff of the form `if A then B`, and infers that the wff `B` is also
 53 | true:
 54 | 
 55 | 	modus-ponens={(A, 'if A then B', B)|A and B any valid wffs}
 56 | 
 57 | A ***proof*** is sequence of wffs `a1...an` such taht eack `ak` is either an
 58 | axiom or direct consequence of some subset of the prior `aj`.
 59 | 
 60 | A ***theorem*** is a wff `a` which is a member of some proof sequence, usually
 61 | as the last wff. The notation `Γ⊢ a` indicates that `a` is a theorem in the
 62 | system of logic under discussion (with Γ standing for the input set of axioms
 63 | specified by the logic program).
 64 | 
 65 | ### Properties of Inference Rules ###
 66 | 
 67 | A set of inference rules which does not infer a theorem which is not in the
 68 | overall true set, is called ***sound***. On the other hand, we want the
 69 | inference rules to permit proofs for all true wffs that are derivable from te
 70 | axioms - such set is called ***complete***.
 71 | 
 72 | Slight variations of the theorem concept will also be of interest, for example
 73 | to find out if some wff would be true if we added some set of wffs. These are
 74 | called ***hypothesis, premissses***. If a proof sequence exists which includes
 75 | members of hypotheses  as axioms, then this sequence is called ***deduction***
 76 | and is noted `Γ, Y⊢ a`. It is roughly equivalent to `Γ⊢ (if Y then a)`
 77 | 
 78 | A ***consistent*** set of wffs does not permit derivation of contradiction. In
 79 | most cases this is important property of an axiom set and one the logic
 80 | programmer must strive to achieve.
 81 | 
 82 | ### Decision Procedures ###
 83 | 
 84 | Finding inferences and proof sequences is an important part of logic-based
 85 | computing. The programmer specifies a set of axioms and then asks questions
 86 | about other wffs and their validity. The system will then try various
 87 | combinations of inference rules in an attempt to find a proof sequence. We
 88 | would like some guarantees that the questions we ask are answerable in finite
 89 | time. We also expect that the search will be more efficient than trying all
 90 | possible combinations technique, which is called ***exhaustive search*** or
 91 | ***British Museum search***.
 92 | 
 93 | There are systems which are ***undecidable***, there is no algorithmic approach
 94 | for determining whether a particular wff is true or false. 
 95 | 
 96 | ### Interpretations ###
 97 | 
 98 | Another key part of the semantics of a logic expression is the specification of
 99 | an ***interpretation*** that maps a meaning to each symbol. This starts with
100 | a ***domain*** that defines the set of possible values or objects to be dealt
101 | with. Functions are assigned definitions (mappings) from some domains to
102 | others.  Symbols used as predicates map into tests on relations over these
103 | domains. 
104 | 
105 | The key point here is that there is often an infinite number of interpretations
106 | and combinations of interpretations which can be given the symbols used in wff
107 | or set of wffs. The prime definition is that a wff is ***true under an
108 | interpretation*** if the result of ***evaluating*** the wff under the
109 | interpretation is true. The specified interpretation ***satisfies*** the wff.
110 | 
111 | ![A satisfying interpretation](figures/13_06_satisfying_interpretation.jpg)
112 | 
113 | We say that two wffs are ***equivalent*** if they evaluate to the same
114 | true/false values for every possible interpretations.
115 | 
116 | ### The Deduction Theorem ###
117 | 
118 | The above definitions lead to a very important result called the ***deductioin
119 | theorem***, which will drive many of the inference engines. 
120 | 
121 | The wff `G` is a logical consequence of the wffs `A1...An` if and only if the
122 | wff:
123 | 
124 | 	if (A1 ∧ A2 ... ∧ An) then G
125 | 
126 | is valid (i.e. a tautology). 
127 | 
128 | The second form of the theorem states that the wff `G` is a logical
129 | consequence of the wffs `A1...An` if and only if the wff
130 | 
131 | 	A1 ∧ A2 ... ∧ An ∧ ¬ G
132 | 
133 | is unsatisfiable.
134 | 
135 | ## Propositional Logic ##
136 | 
137 | Perhaps the simplest system of logic that demonstrates most of the syntactic
138 | and semantic features mentioned is ***propositional logic***. In the syntax for
139 | this logic, a ***proposition, propositional letter*** or ***atom*** corresponds
140 | to a declarative sentence that may take on interpretations of true or false.
141 | There are no functions or nonboolean constants.
142 | 
143 | Example:
144 | 
145 | 	A = The car has gas.
146 | 	B = I can go to the store.
147 | 	C = I have money.
148 | 
149 | Construction of wffs from these letters involves combinations using ***logical
150 | connectives*** - functions over `{T,F}x{T,F}->{T,F}. 
151 | 
152 | 	<proposition> := A|B|C...
153 | 	<binary-connective> := ⇒ | ∧ | ∨ | ≡
154 | 	<unary-connective> := ¬
155 | 	<wff> := <proposition>
156 | 			 | (<unary-connective><wff>)
157 | 			 | (<wff> <binary-connective> <wff>)
158 | 
159 | ![Common logical connectives](figures/13_10_connectives.jpg)
160 | 
161 | ## A Simple Inference Engine ##
162 | 
163 | All real propositional logic programs have more than one axiom.
164 | 
165 | ### A Brute-Force Inference Engine ###
166 | 
167 | We can outline a simple ***truth table-driven inference engine*** for
168 | propositional logic.
169 | 
170 | Given set of axioms `X` and some new wff `G` (goal), we want to determine that
171 | it is a theorem in our system, namely, does `X⊢ G`. Since this is propositional
172 | logc, `G` is an expression involving propositional letters and connectives. The
173 | operation of the inference engine is:
174 | 
175 | 1. Pick an interpretation (assignment of T and F to letters) that has not been
176 |    tried before.
177 | 2. If all interpretations have been tried, we are done and G is a theorem
178 | 3. If the interpretation does not satisfy the original axiom set `X`, ignore it
179 |    and go back to 1.
180 | 4. Try the interpretation an `G` using the standard definitions of all
181 |    connectives.
182 | 5. If the result is F, quit, `G` is not a theorem.
183 | 6. If the result is T, go back to 1.
184 | 
185 | ![Axiom schemas in the propositional calculus](figures/13_12_axiom_schemas.jpg)
186 | 
187 | ### A Modus Ponens-Based Inference Engine ###
188 | 
189 | For large problems, the above is not usable (for 30 letters, full table would
190 | require over 1 billion interpretations). The use of inference rules in the
191 | engine drastically decreases this combiinatorial explosion by handling with one
192 | symbolic operation many common parts of the truth table. For proposition
193 | calculus, only one such rule is needed - ***modus ponens***. This rule states,
194 | that if one is given as valid the pair of wffs `x` and `x ⇒ y` (where `x` and
195 | `y` are arbitrary wffs), then the wff `y` is also valid. This engine requires
196 | that all wffs are written in form using only `not ¬` and `implies ⇒`.
197 | 
198 | Inference engine:
199 | 
200 | 1. (Decision Procedure) Pick two wffs from the axiom set, where one of these
201 |    wffs has ⇒ as its outermost connective.
202 | 2. (Application of Inference Rule) Verify that the antecedent part of the axiom
203 |    with the outermost ⇒ exactly matches the other axiom.
204 | 3. If no match occurs, go back to 1.
205 | 4. If a match occurs, compare the consequent with `G`
206 | 5. If the same, quit with answer yes.
207 | 6. If different, add the consequent to the axiom set, and go to 1.
208 | 
209 | Eventually, this process will generate all wffs which can be deduced from the
210 | original set. If one of them matches, the goal is theorem. 
211 | 
212 | ### Other Inference Rules ###
213 | 
214 | There are other inference rules with the same logical power. ***Modus
215 | tollens*** and ***resolution*** and a myrid of variations. ***Modus tollens***
216 | is like running modus ponnens backwards. Given a wff `¬y` and another wff `x
217 | ⇒ y`, we can infer the wff `¬x`. This is the same as stating `¬y,(x ⇒ y) ⊢ ¬x`.
218 | Again, modus tollens requires wffs expressed only with `⇒` and `¬`.
219 | 
220 | ***Resolution*** is a little different, since it is normally used when
221 | individual wffs are expressed in ***conjunctive normal form***, which is the
222 | `and ∧` of terms built only from `not ¬` and `or ∨`. It can be viewed as
223 | a *chaining rule*, which takes two wffs of the form `x ⇒ y` and `y ⇒ z` and
224 | infers `x ⇒ z`. In their conjunctive form, these wffs look like `(¬x ∨ y)` and
225 | `(¬ y ∨ z), with the resolution simply *combining* the two wffs and *cancelling
226 | out* the `y` from one and the matching `¬y` from the other.
227 | 
228 | ## A Sample Problem ##
229 | 
230 | Propositions:
231 | 
232 | 	A = The car has gas.
233 | 	B = I can go to the store.
234 | 	C = I have money.
235 | 	D = I have food.
236 | 	E = The sun is shining.
237 | 	F = I have an umbrella.
238 | 	G = Today I can go on a picnic.
239 | 
240 | Initial set of axioms:
241 | 
242 | 1. `A ⇒ B` #If the car has gas, then I can go to the store
243 | 2. `(B ∧ C) ⇒ D` #If I can go to the store and I have manoy, then I can buy food.
244 | 	> Note: when we translate this wff into a form using only ⇒ and ¬, there
245 | 	> are several possibilities e.g. `B ⇒ (C ⇒ D)` or `C ⇒ (B ⇒ D)`
246 | 3. `(D ∧ (E ∨ F)) ⇒ G` #If I have food and either the sun is shining or I have an umbrella, then today I can go on a picnic.
247 | 4. `A` #The car has gas
248 | 5. `C` #I have money
249 | 6. `¬E` #The sun is not shining
250 | 
251 | Stating that all these wffs are axioms is equivalent to assuming they are
252 | always evaluated to true. Since they are not tautologies in themselves, this is
253 | equivalent to *and*ing them all together into one big wff, and constraining the
254 | interpretations which we use to solve problems using them to ones that make
255 | this single wff evaluate to tru. For example, the fourth axiom constrains all
256 | interpretations to ones there the `A` is assigned true. Likewise, the axiom
257 | 1 excludes any interpretation in which `A` is true and `B` is false. Note that
258 | modus ponens would deduce from these two that `B` is also true. 
259 | 
260 | In structure, the first three of these program wffs are axioms that define
261 | ***if-then rules***, while the next three correspond to ***facts***. It is
262 | important to emhasize that this later set represents inherent tautologies, the
263 | truth of the others is due totally to the wishes of the programmer of the
264 | system.
265 | 
266 | Examples of the kind of wff questions that one might ask are:
267 | 
268 | * `D` #Is it true that I can buy food?
269 | * `F ⇒ G #Is it true that if I have an umbrella, then I can go on a picnic?
270 | 
271 | Again, asking a question in this system is equivalent to identifying a wff and
272 | asking if it is in the set of true wffs deducible from the axioms and inference
273 | rules. It is up to the decision procedure to mechanically find the proof
274 | sequence, if it exists. 
275 | 
276 | For propositional logic, the simplest possible proof procedure is to simply do
277 | a truth table analysis, i.e. look at all possible assignments of true/false to
278 | all the propositions used in the axiom set and gual and see if there is at
279 | least one interpretation (one row) that makes all the original axioms and the
280 | goal true at the same time. 
281 | 
282 | ![A proof procedure using truth tables](figures/13_13_truth_table.jpg)
283 | 
284 | Of the 128 possible interpretations only 4 satisfy all the axioms.
285 | 
286 | Another possible decision procedure involves beating these wffs (using an
287 | inference rule) against each other and any new wffs as long as new wffs are
288 | inferred. As each is inferred, it can be compared to the desired question.
289 | A match ends the computation. As an example, a proof sequence for `D` using
290 | only modus ponens:
291 | 
292 | * From `A` and `(A ⇒ B)` infer `B`
293 | * From `B` and `(B ⇒ (C ⇒ D))` infer `(C ⇒ D)`
294 | * From `C` and `(C ⇒ D) infer `D`
295 | 
296 | The general term for this kind of inference is ***forward chaining***, or
297 | ***antecedent reasoning*** since the system is expanding forward from known
298 | wffs to new ones.
299 | 
300 | There is an alternative decision procedure that involves picking a wff whose
301 | truth value is desired and working backwards.At each step, one tries to find
302 | a wff which, if one used modus ponens between it and some other wff, would
303 | infer the desired wff. This other wff then becomes the new question. This is
304 | ***backward chaining*** or ***consequence reasoning***.
305 | 


--------------------------------------------------------------------------------
/14_predicate_logic_and_the_first_inference_engine.md:
--------------------------------------------------------------------------------
  1 | # Predicate Logic and the First Inference Engine #
  2 | 
  3 | One of the problems with propositional calculus is its inability to make
  4 | *logical generalizations*. Sometimes we need to say `For any x, if x is
  5 | something, then x has a property`. ***Predicate-based logics*** allow us to do
  6 | this by replacing the statement letters by ***predicate expressions*** that
  7 | test whether or not certain combinations of objects, or tuples of objects, are
  8 | members of certain relations. 
  9 | 
 10 | The simplest and the most used system is ***first-order predicate*** calculus.
 11 | Besides predicates, it introduces ***constants, functions, variables*** and
 12 | ***quantifiers***. Constant identify specific objects (John, 3), functions
 13 | indicate how to map one or more objects specified as arguments into other
 14 | objects. Variables may take on values from the domain of objects expressible
 15 | from the functions and constants, and may be used freely in any place where
 16 | constants or functions are permitted. All three may be combined into ***logic
 17 | expressions***.
 18 | 
 19 | Quantifier takes a variable and a logical expression containing copies of that
 20 | variable and specifies how much of the relevant domain of objects needs to be
 21 | substituted for that variable to make complete expression true. Quantifiers
 22 | ***Bind*** identifiers and specify where values for those identifiers should be
 23 | substituted. The most common are `all objects` and `at least one object`.
 24 | 
 25 | Together, these additions make it possible to express statements such as `For
 26 | all x, if x is human, then x has a mother.`
 27 | 
 28 | ## Basic Syntax ##
 29 | 
 30 | 	<constant> := <symbol> | <number>
 31 | 	<functor> := <symbol>
 32 | 	<predicate-name> := <symbol>
 33 | 	<identifier> := <symbol>
 34 | 	<tuple> := (<term> {,<term>}*)
 35 | 	<term> := <constant> | <identifier> | <functor><tuple>
 36 | 	<atom> := <predicate-name><tuple>
 37 | 	<literal> := <atom> | ¬<atom>
 38 | 	<binary-connective> := ∧ | ∨ | ≡ | ⇒ | ⊕
 39 | 	<quantifier> := ∀ | ∃
 40 | 	<wff> := <literal> 
 41 | 			 | ¬<wff>
 42 | 			 | (<wff><binary-connective><wff>)
 43 | 			 | (<quantifier><identifier>'|'<wff>)
 44 | 
 45 | 
 46 | ### Atoms, Literals, and Predicates	###
 47 | 
 48 | ***Atom*** consists of the name of a relation prefixed to a tuple whose elements are
 49 | terms. The meaning is that under an interpretation the atom evaluates to true
 50 | if the object specified by the tuple is in the set defined by the relation.
 51 | 
 52 | 
 53 | ***Literal*** is an atom with an optional ¬ prefixed. 
 54 | 
 55 | While atoms and terms have identical syntax, they are used in different places.
 56 | Terms represent objects and appear as arguments to functions or atoms. Atoms
 57 | represent the most basic wff construction and may never appear as arguments. 
 58 | 
 59 | Atoms can be interpreted as boolean tests, with the predicate name reflecting
 60 | the type of test to be applied to the tuple. Perhaps the most useful such
 61 | predicate is ***equality***. 
 62 | 
 63 | ### Wffs and Quantifiers ###
 64 | 
 65 | A ***wff*** is a logical expression built up from atoms combined by the standard
 66 | logical connectives from propositional calculus plus quantifier symbols to
 67 | scope identifier bindings. Quantifiers are followed by a ***binding variable***
 68 | and a wff making up the quantifier's body. 
 69 | 
 70 | ***Universal quantifier ∀*** defines a wff which is true only if the sub-wff
 71 | making up its body remains true regardless of what values are given to the
 72 | binding variable. 
 73 | 
 74 | ***Existential quantifier ∃*** forms a wff which is true if there exists as
 75 | little as one object which if substituted for the free instances of the binding
 76 | variable in the body, make it true.
 77 | 
 78 | ## Basic Interpretations ##
 79 | 
 80 | Again, an ***interpretation*** is a description of what a wff means, the
 81 | ***semantics*** of the wff. For our purposes, we define an interpretation as
 82 | having two parts. First, the ***permanent*** part, is constant across all wffs
 83 | in a language. This includes primarily the definition of the logical
 84 | connectives and the quantifiers. Second are a programmer-defined mapping of
 85 | values to the various other symbols that make up a wff. Such interpretations
 86 | may wary from wff to wff or even somethimes among different parts of a large
 87 | wff. This includes:
 88 | 
 89 | * Constant symbols are assigned some specific actual values
 90 | * Function symbols are assigned some mappings, and terms computed as function
 91 |   applications from those mappings and the constant interpretations.
 92 | * Predicate symbols are assigned relations, with predicates given true/false
 93 |   values on the basis of whether or not their arguments form valid tuples in
 94 |   the relation. 
 95 | 
 96 | Now given an interpretation, we can ***evaluate*** an unquantified wff by the
 97 | following:
 98 | 
 99 | * Compute terms as the appropriate function applications.
100 | * See if the tuples formed by the evaluated terms are in the associated
101 |   relations.
102 | * If so, mark those atoms as true.
103 | * If not, mark the atoms as false.
104 | * Combinations of atoms joined via the logical connectives are interpreted
105 |   exactly as in propositional calculus.
106 | 
107 | Universal quantifier in from of a wff causes the above procedure to be repeated
108 | for every possible object. This means that if we have a wff such as
109 | `(∀x|P(x))`, it is totally equivalent to multiple copies of the body
110 | ***and***ed together, one per possible domain value. 
111 | 
112 | 	(∀x|P(x)) ≡ P(red) ∧ P(white) ∧ P(blue)
113 | 
114 | Existential quantifier is similar, only ***or***ed
115 | 
116 | 	(∃x|P(x)) ≡ P(red) ∨ P(white) ∨ P(blue)
117 | 
118 | ## Standard Expression Forms ##
119 | 
120 | ![Some standard logic forms for wff](figures/14_02_standard_logic_forms.jpg)
121 | 
122 | ### Form Conversions in Propositional Calculus ###
123 | 
124 | ![Conversion steps for propositional logic](figures/14_04_conversion_propositional.jpg)
125 | 
126 | ### Prenex Normal Form ###
127 | 
128 | ![Conversion steps for prenex logic](figures/14_06_conversion_prenex.jpg)
129 | 
130 | ### Skolem Standard Form ###
131 | 
132 | Next step in conversion process is elimination of all existential quantifiers
133 | in a wff in prenex normal form. The result is ***Skolem standard form***. The
134 | rules for conversion are a combination of prior procedures and several rather
135 | different new ones. Removal of the existential quantifiers requires invention
136 | of what are called ***Skolem constans*** and ***Skolem functions***. 
137 | 
138 | Consider a wff which leftmost quantifier is ∃. ***Skolemization*** removes
139 | a leading `∃x` and replaces all free occurences of `x` in the rest of the wff
140 | with a new constant symbol, which is not used anywhere else but is used solely
141 | in the wff to represent the *magic value* which has to exist in order for the
142 | wff to be true.
143 | 
144 | This new constant is called ***Skolem constant*** and its purpose is to name
145 | the value which satisfies the wff. Instead of saying `There exists an x such
146 | that...` we are saying that the object exists with some unambiguous name.
147 | 
148 | If the wff does not start with ∃, ut with ∀, the process is more complex. 
149 | 
150 | 	`∀x1...∀xk-1 ∃xk ... M
151 | 
152 | The meaning of this is that regardless of what values we pick for `x1..xk-1`,
153 | there is still some value we can give to `xk` to make the rest of the wff true.
154 | However, this value can vary with varying `x1...xk-1` values. To remove
155 | existential quantifier, we will do as before. We will create a special symbol
156 | to represent the value for `xk`, but it must be a function - ***Skolem
157 | function*** of `k-1` arguments. Assuming tha function name `gk`, we can delete
158 | `∃xk` by replacing all instances of `xk` with `gk(x1, ...xk-1)`.
159 | 
160 | Example:
161 | 
162 | 	∀x|∃y|(¬human(x) ∨ is-mother-of(y,x)
163 | 
164 | Choice of y depends on value of x. We will define skolem function
165 | `mother-of:D→D`
166 | 
167 | 	∀x|(¬human(x) ∨ (is-mother-of(mother-of(x), x))
168 | 
169 | ## Clausal Form ##
170 | 
171 | Pure ***clausal form*** for predicate calculus occurs when a wff is in the
172 | Skolem standard form, and the quantifier-free subwff on the right is in
173 | conjunctive normal form:
174 | 
175 | 	∀x1 | ∀x2... | ∀xn | (C1 ∧ C2 ∧ ... ∧ Cm)
176 | 
177 | where each `C` is quantifier-free disjuction of literals. What that means?
178 | 
179 | 	Ck = (¬q1 ∨ ... ∨ ¬qn ∨ p1 ∨ ... ∨ pm)
180 | 	
181 | Temporarily factoring out negations:
182 | 
183 | 	Ck = (¬(q1 ∧ q2 ∧ ... ∧ qn) ∨ (p1 ∨ p2 ∨ ... ∨ pm))
184 | 
185 | From the definition of `⇒` this is equivalent to:
186 | 
187 | 	Ck = ((q1 ∧ ... ∧ qn) ⇒ (p1 ∨ ... ∨ pm))
188 | 
189 | Such clause is called an ***if-then rule*** with `q`s representing the
190 | ***antecedent*** or ***conditions***, and `p`s representing ***consequent,
191 | conclusion,*** or ***result***.
192 | 
193 | ## Horn Clauses ##
194 | 
195 | A ***Horn clause*** is a clause C with the property that at most one atom in
196 | the clause is not negated. With this constraint there are exactly three forms
197 | a clause can take:
198 | 
199 | 1. Exactly one unnegated atiom and one or more negated ones. This matches an
200 |    if-then rule with exactly one positive literal in consequent and all
201 |    positive literals in antecedent. The rule reads as `if all of the antecedent
202 |    predicates q1 through qn are true, then so is p`.
203 | 2. Exactly one unnegated atom and no negated ones. This is often called an
204 |    ***assertion*** or ***fact***, since it must be true by itself in order for
205 |    the whole wff to be true.
206 | 3. No unnegated atoms and one or more negated atoms. This is called
207 |    ***question*** or ***goal clause***.
208 | 
209 | Now consider what happens if we build a logic system where all wffs must be
210 | expressed in ways that permit conversion to a pure Horn clause form. It is
211 | clear that not all wffs can be expressed in this form (e.g. `A ⇒ (B ∨ C)`).
212 | Even so, this kind of a restriction forms the basis for many real logic
213 | programming systems, such as ***Prolog***. There are several reasons including:
214 | 
215 | * Constraints can be placed on the wff syntax that guarantee that each wff
216 |   entered into the system converts into exactly one clause.
217 | * Reasonably complete and efficient decision procedures exist for axiom sets in
218 |   this form
219 | * FOr many practical problems, the limitations of one predicate in the
220 |   conclusion is not severe and can be worked around.
221 | * The three distinct forms (rule, fact, goal) seem to represent a clean match
222 |   to normal human thinking and problem solving activities.
223 | 
224 | ![Same sample Horn clauses](figures/14_09_horn_clauses.jpg)
225 | 
226 | ### The Empty Clause ###
227 | 
228 | The ***empty clause*** is written ∅.There is no possible interpretation that
229 | can make it true. The value of this clause is that if it is ever inferred, it's
230 | an indication that the original set of axioms is contradictory.
231 | 
232 | ### Combining Multiple Wffs ###
233 | 
234 | If we have a set of wffs, all true, then we can *and* them together without
235 | loss of validity. Further, rules 4 and 5 in the Prenex conversion proces then
236 | permit us to move all the universal quantifiers to the left of the wff bodies.
237 | This creates one giant conjunction.
238 | 
239 | ![Combination of multiple axioms](figures/14_10_combination_of_axioms.jpg)
240 | 
241 | One point that needs to be discussed is the potential requirement to
242 | ***rename*** variables in individual wff bodies when the quantifiers are moved
243 | up front.
244 | 
245 | ## Decidability Issues - The Herbrand Result ##
246 | 
247 | An algorithm which can determine whether or not inputs (wff or set of wffs)
248 | follow logicaly from axioms is ***decision method*** or ***decision
249 | procedure***. This algorithm forms a large part of the inference engine.
250 | Computationally, monitoring such a decision procedure (recording a proof
251 | sequence and the interpretations so involved) often represents the outputs
252 | desired from the program. 
253 | 
254 | Given the importance of a decision procedure, a critical question to ask is
255 | whether or not such a procedure exists for an arbitrary set of axioms. This
256 | question is called a ***decision problem***, and for propositional logic the
257 | answer is yes. For the first-order predicate logic, there is not such algorithm
258 | that works in guaranteed finite time. Predicate logic is ***undecidable***.
259 | 
260 | But we can in fact guarantee that in finite time we can check whether the wff
261 | is theorem, but if it's not, we will never find it and the program will most
262 | probably hang up in an unending loop.
263 | 
264 | Thus the decision problem for first-order logic is at best ***semidecidable***,
265 | and we will limit our search for inference engines to those that check that
266 | wffs are theorems and ignore what happens if they are not theorems. 
267 | 
268 | ### The Herbrand Universe ###
269 | 
270 | Just as with truth tables and propositional logic, the process of generating
271 | theorems in first order logic seems to go hand in hand with generating
272 | interpretations in some well ordered and structured fashion. Such an order
273 | exists in what is called a ***Herbrand interpretation***. For each axiom set
274 | this interpretation defines a special domain of objects, called the ***Herbrand
275 | universe***, to which any other domain could be mapped, and then assigns
276 | mappings to the function and predicate symbols. 
277 | 
278 | The Herbrand universe is a potentially infinite set `H` wich includes all
279 | objects expressible as ***terms*** in the system, and is constructed
280 | iteratively from a series of simpler sets `H0,H1...`. The original set `H0`
281 | represents all the constants we know to be present, namely, those designated by
282 | the constant symbols used in the axioms. If no constant symbols exist, we
283 | invent one. Then, from any set `Hk`, we create a new `Hk+1` by appending to
284 | `Hk` any term expressible as any one of the function symbols used in the axiom
285 | set applied to the appropriate number of objects from `Hk`. Duplicates are
286 | eliminated. This is repeated until no new objects are added, at which point the
287 | `H` corresponds to this final set.
288 | 
289 | This process essentially finds all objects either explicitly defined in the
290 | system or constructible by applying functions to already defined objects. 
291 | 
292 | ![A sample Herbrand universe](figures/14_12_herbrand_universe.jpg)
293 | 
294 | ![Another sample Herbrand universe](figures/14_13_another_herbrand_universe.jpg)
295 | 
296 | ### Herbrand Interpretations ###
297 | 
298 | Now the ***atom set*** or ***Herbrand base*** of a set of axioms is defined as
299 | the set of all predicate symbols applied to all possible tuples of elements
300 | from the Herbran universe. This if `is-father-of` is a predicate symbol found
301 | in the axiom set, then the atom set would include all expressions of the form
302 | `is-father-of(t)`, where `t` is a member of `H`. 
303 | 
304 | Each such combination of a predicate symbol and a tuple of objects from `H` is
305 | termed a ***ground atom***, or ***ground literal***. 
306 | 
307 | With these definitions we can describe ***Berbrand interpretation*** as any
308 | mapping that has the following characteristcs.
309 | 
310 | * Each constant symbol is mapped to some unique constant of the same name
311 | * Each function symbol `f` of arity n is mapped to a function from `Hn` to `H`,
312 |   where the tuple `(t1...tn)` is mapped to the unique object in `H` with name
313 |   `f(t1...tn)
314 | * Each element in the atom set is mapped to either T or F.
315 | 
316 | ![Some simple Herbrand predicate interpretations](figures/14_14_herbrand_interpretations.jpg)
317 | 
318 | ## The Herbrand Inference Engine ##
319 | 
320 | ***Herbrand's Theorem*** says that if a wff is unsatisfiable, we can determine
321 | this after analyzing only a finite number of possible substitutions from the
322 | Herbrand universe. This is most easily understood by building a wff according
323 | to the second form of the Deduction Theorem, and converting it into clausal
324 | from,`∀x1...|(C1 ∧ ...Cn)`, where each `Ck` is the *or* of some literals.  As
325 | mentioned earlier, this is equivalent to repeating the quantifier-free
326 | expression (`C1 ∧ ... Cn`) an infinite number of times, each time substituting
327 | a different set of objects for the variables. Each substitution leaves no
328 | variables behind, only connections of predicates applied  to constants or
329 | functions of constants. Such clauses are called ***ground instances*** of the
330 | original clause. Given that they are now variable-free, the resulting wff is
331 | equivalent to an expression in propositional logic.
332 | 
333 | Note that in this form, as soon as any one of the ground clauses becomes false,
334 | the whole wff is false. 
335 | 
336 | The Herbrand Theorem states that we need not look at the infinite conjunction
337 | of such ground clauses. After only a finite number of substitutions from the
338 | Herbrand Universe, we will end up with a finite number of clauses which are
339 | themselves inconsistent (false under all interpretations). Further, testing
340 | a finite number of ground clauses for inconsistency is feasible: Just as in
341 | propositional logic, we need only do a truth table analysis assuming that each
342 | distinct predicate and argument expression is a separate propositional letter.
343 | If the final column of such an analysis is all false, then that set of clauses
344 | is inconsistent under all possible interpretations, and we are done.
345 | 
346 | This process leads directly to a possible inference engine for verifying that
347 | a wff `G` is a theorem of a set of axioms. We start by combining the axioms
348 | with the negation of `G` and converting to clausal form. Then, starting with
349 | `H0`, we iterate through the Herbrand sets `Hk`. For each set we form all
350 | possible substitutions from the set into the variables in the clauses of the
351 | body, make the substitutions, and combine together in a large conjunction. This
352 | conjunction is then analyzed for unsatisfiability using any technique suitable
353 | for propositional calculus. If it is satisfiable, the process is repeated for
354 | the next bigger set. As soon as we find a case where it is unsatisfiable, we
355 | stop - the original wff is a theorem.
356 | 
357 | Within months of the appearance of Gilmore's original paper, a dramatically
358 | improved procedure was developed (Davis and Putnam, 1960). Their procedure used
359 | a clausal form of the expansion but with the following analysis rules:
360 | 
361 | 1. Delete all clauses that are known tautologies. These will never yield
362 |    a false, so they can be ignored.
363 | 2. If there is some unit clause `L` (i.e there is only one atom in it), delete
364 |    it and all clauses that include it. If the resulting wff is empty, stop
365 |    because the wff is satisfiable (go on to the next Herbrand set). If the
366 |    resulting wff is not empty, delete `¬L` from any clause that contains it. If
367 |    any of these deleted clauses are themselves unit clauses, stop; the wff is
368 |    unsatisfiable. 
369 | 3. If a literal appears ***pure***, that is, `L` appears but `¬L` is never
370 |    used, delete all clauses that contain it.
371 | 4. If the wff is of the form
372 | 
373 | 		(A1 ∨ L) ∧ ... ∧ (Am ∨ L) ∧ (B1 ∨ ¬L) ∧ ... ∧ (Bn ∨ ¬L) ∧ C
374 | 
375 | 	where all `A`s, `B`s, and `C`s are clauses free of either `L` or `¬L`, then
376 | 	split the wff into two:
377 | 	
378 | 		A1 ∧ ... ∧ Am ∧ C
379 | 		B1 ∧ ... ∧ Bn ∧ C
380 | 
381 | 	If both of those are themselves unsatisfiable, then so is the original wff.
382 | 
383 | If after taking this process as far as it will go there still is some wff left,
384 | then we must go on to the next Herbrand set. 
385 | 
386 | ![A Herbrand inference engine](figures/14_15_inference_engine.jpg)
387 | 
388 | ## Problems ##
389 | 
390 | 3. Transform the following into clausal form:
391 | 
392 | 	1. `((p ⇒ q) ∧ ¬p) ⇒ ¬q`
393 | 
394 | 			-> ¬((p ⇒ q) ∧ ¬p) ∨ ¬q
395 | 			-> ¬((¬p ∨ q) ∧ ¬p) ∨ ¬q
396 | 			-> ¬(¬p) ∨ ¬q
397 | 			-> p ∨ ¬q
398 | 
399 | 	2. `∃u|∃v|∀w|∀x|∃y|∀z|∃q|∀s|p(q,s,u,v,w,x,y,z)`
400 | 
401 | 			-> ∀w|∀x|∀z|∀s|p(g(w,x,z),s,a,b,w,x,f(w,x),z)
402 | 
403 | 
404 | 	3. `∀x|(P(a,x) ⇒ ∃y|Q(a,y,x))`
405 | 
406 | 			-> ∀x|(¬P(a,x) ∨ ∃y|Q(a,y,x))
407 | 			-> ∀x|∃y(¬P(a,x) ∨ Q(a,y,x))
408 | 			-> ∀x(¬P(a,x) ∨ Q(a,f(x),x))
409 | 
410 | 	4. `∀x|(P(a,x) ⇒ ∃x|Q(a,y,x))`
411 | 
412 | 			-> ∀x|(¬P(a,x) ∨ ∃x|Q(a,y,x))
413 | 			-> ∀x|∃z|(¬P(a,x) ∨ Q(a,y,z))
414 | 			-> ∀x|(¬P(a,x) ∨ Q(a,y,f(x)))
415 | 
416 | 	5. `∃y|((∀x|(P(x) ⇒ Q(f(x),y))) ⇒ ((∃y|P(y)) ⇒ (∃x|Q(x,y))))`
417 | 
418 | 			-> ∃y|((∀x|(P(x) ⇒ Q(f(x),y))) ⇒ ((∃y|P(y)) ⇒ (∃x|Q(x,y))))
419 | 			-> ((∀x|(P(x) ⇒ Q(f(x),z))) ⇒ ((∃y|P(y)) ⇒ (∃x|Q(x,z))))
420 | 			-> (¬(∀x|(P(x) ⇒ Q(f(x),z))) ∨ ((∃y|P(y)) ⇒ (∃x|Q(x,z))))
421 | 			-> (¬(∀x|(¬P(x) ∨ Q(f(x),z))) ∨ (¬(∃y|P(y)) ∨ (∃x|Q(x,z))))
422 | 			-> ((∀x|¬(¬P(x) ∨ Q(f(x),z))) ∨ (∀y|¬P(y) ∨ ∃x|Q(x,z)))
423 | 			-> ((∀x|P(x) ∧ ¬Q(f(x),z)) ∨ (∀y|¬P(y) ∨ ∃x|Q(x,z)))
424 | 			-> ((∀x|P(x) ∧ ¬Q(f(x),z)) ∨ ((∀y|¬P(y)) ∨ (∃x|Q(x,z))))
425 | 			-> ((∀x|(P(x) ∧ ¬Q(f(x),z))) ∨ (∀y|∃x|¬P(y) ∨ Q(x,z)))
426 | 			-> ((∀x|(P(x) ∧ ¬Q(f(x),z))) ∨ (∀y|∃a|¬P(y) ∨ Q(a,z)))
427 | 			-> ∀x|((P(x) ∧ ¬Q(f(x),z))) ∨ (∀y|¬P(y) ∨ Q(g(y),z))
428 | 			-> ∀x|∀y|(P(x) ∧ ¬Q(f(x),z)) ∨ (¬P(y) ∨ Q(g(y),z))
429 | 			-> ∀x|∀y|(P(x) ∨ (¬P(y) ∨ Q(g(y),z))) ∧ (¬Q(f(x),z) ∨ (¬P(y) ∨ Q(g(y),z)))
430 | 			-> ∀x|∀y|(P(x) ∨ ¬P(y) ∨ Q(g(y),z)) ∧ (¬Q(f(x),z) ∨ ¬P(y) ∨ Q(g(y),z))
431 | 
432 | 4. Assume that wffs in propositional logic have been converted to clausal form
433 |    and written as s-expression lists, where each element of the topmost list
434 |    corresponds to a clause and each clause is written as a list itself of
435 |    either `<proposition>` or `(NOT <proposition>)`. For example, `(¬d ∨ e ∨ f)
436 |    ∧ (h ∨ ¬f)` would be:
437 | 
438 | 		(((NOT d) e f) (h (NOT f)))
439 | 
440 |    Write an abstract function `unsat(wff, letters)`, where `letters` is a list
441 |    of the propositional letters in the wff, which determines whether or not
442 |    such a wff is unsatisfiable.
443 | 
444 | 		unsat(wff, letters) =
445 | 			if null(letters)
446 | 			then is-wff-false(wff)
447 | 			else let letter = car(letters)
448 | 			in and(is-wff-false(subs(wff, letter, T), cdr(letters)),
449 | 				   is-wff-false(subs(wff, letter, F), cdr(letters)))
450 | 		where is-wff-false(wff) =
451 | 				not(and((apply or car(wff)),
452 | 					is-wff-false(cdr(wff))))
453 | 		and subs(wff, letter, value) = 
454 | 				if atom(wff)
455 | 				then if wff = letter
456 | 					 then value
457 | 					 else wff
458 | 				else cons(subs(car(wff),letter,value),
459 | 						  subs(cdr(wff),letter,value))
460 | 
461 | 
462 | 
463 | 
464 | 
465 | 


--------------------------------------------------------------------------------
/17_warren_abstract_machine.md:
--------------------------------------------------------------------------------
  1 | # The Warren Abstract Machine #
  2 | 
  3 | WAM is to logical programming what SECD is to functional. Basic semantic model
  4 | of how a program carries on computation. WAM serves as the basis for most
  5 | high-performance implementations of ***Prolog*** and similar langauges. This
  6 | includes use of intermediate language for a compiler from Prolog to
  7 | conventional machines, as the basis for entirely new computer architectures
  8 | that support Prolog features directly, and as a starting point for
  9 | implementations of non-Prolog logic languages. Origins come from David H. D.
 10 | Warren's Ph.D. thesis (1977).
 11 | 
 12 | The general model of execution assumes that the argument registers mentioned
 13 | above contain the actual arguments for the current goal literal, and these
 14 | values are successively unified against the formal argument expression. When
 15 | a match occurs, new argument values are built for the first literal in that
 16 | clause's body, and the process is repeated for that goal. Success in solving
 17 | that goal causes code to be executed that builds the arguments for the next
 18 | goal on the right hand side. A linking mechanism keeps track of which literals
 19 | are left to be treated as goals, and in what order, after the current goal is
 20 | proven successfully. Saving copies of the argument and other registers in
 21 | a memory stack permits failure and backtracking operations to restart with
 22 | a previous goal as required.
 23 | 
 24 | ![Prolog execution using the WAM model](figures/17_01_prolog_execution.jpg)
 25 | 
 26 | ## Program Structure ##
 27 | 
 28 | For each of original Prolog statements there is a corresponding section of WAM
 29 | instructions which handles the head unification for that clause and the
 30 | sequencing through goals called our on the statement's right hand side.
 31 | 
 32 | ![Simplified Prolog-to-WAM translation](figures/17_02_translation.jpg)
 33 | 
 34 | All such code sections for clauses aving the same predicate name in their head
 35 | literal are chained directly together in the order in which the programmer
 36 | entered the original text. The chaining is via instructions at the beginning of
 37 | each section. This permits the computer to rapidly find the next clause to try
 38 | if one clause fails.
 39 | 
 40 | Together, each linkage of sections acts as a single ***procedure***, tailored
 41 | specifically to handle any goal whose predicate symbol matches that for its
 42 | internal clauses. The internals of the precedure step through the appropriate
 43 | statements in the expected Prolog order, with calls to other such prcedures as
 44 | needed as goals are processed. All such calls are to the entry code of
 45 | a procedure segment where the neccessary initialization is performed.
 46 | 
 47 | ### Code for One Clause ###
 48 | 
 49 | WIthin a procedure it is possible to link the individual code sections in
 50 | orders other than the simplistic way shown here. In many cases this can improve
 51 | performance by avoiding entirely clauses which are known beforehand not to work
 52 | for certain goals.
 53 | 
 54 | ### An Individual Code Section ###
 55 | 
 56 | ![General structure of clause code](figures/17_03_structure.jpg)
 57 | 
 58 | Starting out the segment is initialization code which sets up the machine's
 59 | major data structures to permit trying the clause. This includes saving
 60 | a pointer to the next clause with the same predicate name if backtracking
 61 | occurs out of this one.
 62 | 
 63 | Following this is code that checks that the formal argument expressions of the
 64 | clause are in fact inifiable with the actual arguments in the current goal.
 65 | These actual arguments are always found in the ***argument registers*** denoted
 66 | `A1` through `An`. Mismatch in any of these tests causes this code sequence to
 67 | be aborted and control transfered to the next appropriate code segment. This is
 68 | sometimes termed ***shallow backtracking***.
 69 | 
 70 | In the process of doing these unification checks, it may be neccessary to
 71 | assign values to formal variables in the clause. This is done by allocating
 72 | a memory location to each variable in the clause and storing a value as
 73 | appropriate during unification checks. The set of memory locations covering the
 74 | variables for a clause is called its ***environment*** and is kept on an
 75 | internal stack.
 76 | 
 77 | Once all arguments have unified successfully, control in the WAM program passes
 78 | to a series of instructions that mirror the body of the original statement.
 79 | There is a series of instructions for each goal, in the order in which the
 80 | goals were written. These instructions are of two types, first takes the
 81 | current substitutions for clause variables and creates the actual arguments (in
 82 | the argument registers). Second type actually performs the transfer of control
 83 | to the entry code for the goal's predicate. That code saves any machine state
 84 | information that might have to be reloaded if a return is necessary to the
 85 | current code, plus initialization for the new predicate's clauses.
 86 | 
 87 | Successful execution of the new code for some goal will eventually result in
 88 | control being passed back to the original code that called it, with the
 89 | original machine state largely restored.
 90 | 
 91 | A failure in the called code to find any matching clauses at all will cause
 92 | a ***backtrack*** into the caller's code to look for another alternative for
 93 | a prior goal. This is called a ***deep backtrack***.
 94 | 
 95 | Successful completion of the code segment for the last goal in a clause causes
 96 | return to the section's exit. This code does whatever storage housekeeping is
 97 | necessary before returning to the code that called the procedure in which this
 98 | section is embedded.
 99 | 
100 | The primary difference between conventional subroutine call and WAM execution
101 | sequence is that a return in the WAM does not free up the stack space allocated
102 | to the call. This is because of Prolog's backtrack mechanism, which may require
103 | restarting the procedure later if a failure is detected in some other clause.
104 | 
105 | ## Major Data Structures and State Registers ##
106 | 
107 | The WAM model matcher fairly directly a conventional von Neumann computer.
108 | 
109 | ![Major memory areas for the simplified WAM](figures/17_04_memory.jpg)
110 | 
111 | The first memory area is devoted to program code. Instructions are fetched from
112 | this area one at a time as indicated by the ***PC register***.
113 | 
114 | Successful completion of the code section for some clause means that a goal
115 | built by some other clause's right-hand size code has been successful, and the
116 | machine should *return* to that point in the right-hand side and resume
117 | execution. The ***continuation pointer*** or ***CP register*** points to this
118 | location. Typical WAM instruction calling a procedure sets the CP to one
119 | instruction after the call.
120 | 
121 | The next major data area is the ***heap***, which contains structures and
122 | lists built during the unification process. These objects are usually too big
123 | to fit into an argument register or single environment cell. The ***structure
124 | pointer*** or ***SP register*** steps through the components of such objects
125 | during unification. Storage here is allocated dynamically as needed, with
126 | pointers to them left where needed. The ***H register*** indicates the top of
127 | the allocated part of the heap.
128 | 
129 | The most important data structure is the ***stack***. It holds call/return and
130 | environment information for sequencing through the code segments corresponding
131 | to the clauses. The information for each attempt to solve a goal is called
132 | ***choice point***, with the ***B register*** (backtrack) pointing to the most recently
133 | created one and the ***E register*** (environment) pointing to the one created
134 | when the clause code currently pointed to by the PC was entered. The ***S
135 | register*** points to the current stack top from which new choice points will
136 | be built.
137 | 
138 | At any point in time the PC points into the code for some clause, and
139 | E register gives access to the current values for variables in that clause. The
140 | B register points to the most recently created choice point and may be equal to
141 | or greather than E. It is equal just as the code for the right-hand side is
142 | entered and is greather as goals in that clause's body are solved successully.
143 | 
144 | The ***trail*** is a stack of locations containing references to variables that
145 | have received values at some point during execution (e.g. are ***bound*** or
146 | ***instantiated***), and may have to be *unbound*. ***TR register*** points to
147 | the top of this area where new trails can be pushed.
148 | 
149 | ***PDL*** or ***push-down list*** is a small stack used by the unification
150 | instructions to save information during unification of complex objects. ***PDL
151 | register*** points to the top of this stack.
152 | 
153 | ## Memory Word Format ##
154 | 
155 | A cell has two parts, a ***tag*** and a ***value***. Tags are of following
156 | types:
157 | 
158 | * ***constant***
159 | * ***variable*** a logical variable that has not yet beed given a value by
160 |   unification.
161 | * ***list*** - identical to general s-expression
162 | * ***structure*** corresponds to a syntactic term involving a function symbol
163 |   and its arguments. It has an arity telling us how many cells are following
164 |   (arguments)
165 | * ***structure pointer*** indicates that the object in question is structure
166 |   starting at designated memory location.
167 | * ***reference*** is indirect pointer to some other cell. This is used to chain
168 |   objects together. In many ways it behaves like ***invisible pointer*** in
169 |   GC algorithms.
170 | 
171 | For simplicity, the notation `x#y` will denote the contents of some cell, where
172 | `x` is tag and `y` is value. A value `*` denotes the address of that cell.
173 | Therefore `var#*` stands for unbound variable.
174 | 
175 | ![Representation of objects in memory](figures/17_05_objects_in_memory.jpg)
176 | 
177 | ## Simplified Choice Point ##
178 | 
179 | The major data structure controlling program execution is the ***choice
180 | point*** - a set of contiguous locations on the ain stack. They closely
181 | resemble a ***frame***. It contains copies of the various machine registers
182 | needed to restart clause's code under various conditions.
183 | 
184 | At any point in a program's execution there is one choice point for each goal
185 | currently still active.
186 | 
187 | ![The choice point and environment](figures/17_06_choice_point.jpg)
188 | 
189 | Information found includes:
190 | 
191 | * a copy of the argument registers `A1...An`. This permits the argument
192 |   registers to be reloaded to their initial values if one clause fails after
193 |   changing some of them and a new clause is to be tried against the same goal.
194 | * where to return to if the goal represented by this choice point is solved
195 |   successfully. This is called the ***continuation***, and consists of:
196 | 
197 | 	* ***Backtrack Continuation Pointer or BCP*** entry. The
198 | 	instructionaddressed by this value is the beginning of the code for the
199 | 	next goal to solve in left-to-right order.
200 | 	* ***Backtrack Continuation Environment or BCE***
201 | 
202 | * The address of the code for the next clause to try if the current clause
203 |   fails - ***FA*** (for ***failure address***). This is the primary information
204 |   needed by the ***shallow backtrack*** process
205 | * The state of the main memory data structures at the time the choice was
206 |   built, namely, the top of the trail and heap, and the choice point if effect
207 |   before this one (***BTR (backtrack trail), BH (backtrack heap), BB (backtrack
208 |   B)***). This is primary information for ***deep backtrack*** if no clause
209 |   exists which satisfies the current goal.
210 | * The ***environment*** for the values to be bound to local variables.
211 | 
212 | The notation `<entry-name>[X]` refers to the contents of a specific entry in
213 | the choice point designated by `X` (usually `B` or `E`). Thus, `BTR[B]` is the
214 | memory cell labeled `BTR` in the choice point selected by `B`.
215 | 
216 | ## Simplified WAM Instruction Set ##
217 | 
218 | Five classes:
219 | 
220 | * ***Indexing instructions*** to control sequencing through the chain of code
221 |   sections associated with one procedure (one head predicate symbol)
222 | * ***Procedural instructions*** to control choice point and environment setup
223 |   and tranfer of control from one chain to another
224 | * ***Get instructions*** to verify that the formal arguments in a clause unify
225 |   with current actual arguments (as recorded in the argument registers), and to
226 |   record the appropriate unifying substitutions
227 | * ***Put instructions*** to load the argument registers for the next goal on
228 |   the right-hand side of some clause
229 | * ***Unify instructions*** to handle gets and puts of complex objects such as
230 |   lists and structures.
231 | 
232 | ![A generic WAM procedure code segment](figures/17_07_procedure_code_segment.jpg)
233 | 
234 | ![Simplified WAM program quick sort](figures/17_08_quick_sort.jpg)
235 | 
236 | In general, the instructions described below treat the WAM machine registers in
237 | a certain fashion:
238 | 
239 | * The B register always points to the topmost choice point on the stack
240 | * Once inside the code for some particular clause, the E register points to the
241 |   choice point that was created when the procedure containing that clause was
242 |   entered. The only time this may be the same as B is just as the code for
243 |   a particular clause is entered and before any goals on the right-hand side
244 |   are tried.
245 | * At the entry to the code segment for a predicate symbol, the CP register
246 |   contains the instruction address to return to if a clause is found in the new
247 |   procedure that unifies with the current goal, and has all of its right-hand
248 |   side goals fully satisfiable. The E register at this time points to the
249 |   environment needed to continue execution at CP.
250 | * Unless otherwise specified, each instruction increments the PC register.
251 | 
252 | ### Indexing Instructions ###
253 | 
254 | ![Simplified indexing instructions](figures/17_09_indexing_instructions.jpg)
255 | 
256 | ***Indexing instructions*** chain together and control the code sections for
257 | different clauses that have the same predicate symbol in their head.
258 | 
259 | The ***mark*** instruction is the first instruction encountered in the
260 | procedure. It builds a choice point. After execution, the B register is set to
261 | point to the new choice point.
262 | 
263 | The ***retry-me-else*** is the first instruction for each clause code section.
264 | It modifies the FA entry in B's choice point to indicate the start of the code
265 | section for the next possible clause with the same predicate symbol. This
266 | address is provided as an argument to the instruction. The address planted by
267 | the instruction is used if the clause corresponding to the code following it
268 | fails for any reason. Usually the first instruction at this address is another
269 | `retry-me-else`.
270 | 
271 | The ***backtrack*** instruction is used as the target of the retry for the last
272 | clause of a chain. If control reaches the backtrack, then none of the clauses
273 | satisfied current goal, and deep backtracking is necessary.
274 | 
275 | Fail sequence:
276 | 
277 | 1. reload the argument registers from B's choice point
278 | 2. reset the heap top to what it was when B's choice point was built
279 | 3. use B to recompute the top of the main stack
280 | 4. ***unwind*** the trail stack by popping off the entried until the TR
281 |    register reaches the value stored in B's choice point. For each entry popped
282 |    off, the memory location corresponding to that variable is reset to
283 |    a ****variable*** entry.
284 | 5 .Branch to the code specified by the FA entry in the restored choice point.
285 | This is the next possible clause which may satisfy the goal.
286 | 
287 | ### Procedural Instructions ###
288 | 
289 | ![Simplified procedural instructions](figures/17_10_procedural_instructions.jpg)
290 | 
291 | ***Procedural instructions*** handle the management of environments and the
292 | transfer of control between chains of clauses.
293 | 
294 | The ***allocate*** instruction is typically the first instruction of the code
295 | section. and allocates space for all the clause's variables (as indicated by
296 | its single argument). In the version shown here, this allocation is on the
297 | stack right after the current choice point. It also initializes all N locations
298 | in the environment to entries with tag ***variable*** and value equaling the
299 | address of its own memory location. Also, this instruction sets up the
300 | E register to point to the choice point where the new environment has just been
301 | created.
302 | 
303 | The ***call*** instruction is used just after loading the argument registers
304 | with argument values for a goal literal in the body of current clause. It saves
305 | the address of the next instruction in the CP register and branches off to the
306 | entry point of that clause code that corresponds to the predicate symbol in
307 | that goal. This is identical to a subroutine call.
308 | 
309 | The ***return*** instruction is the last instruction in a clause segment, and
310 | if executed, it indicates successful satisfaction of all goals in the body of
311 | the clause. Control is passed back to the continuation address with the
312 | caller's environment set, without deallocating any choice points or
313 | environments. Unlike conventional computers, this return does not pop anything.
314 | 
315 | ![Sample stack of choice points](figures/17_11_choice_points.jpg)
316 | 
317 | ### Get Instructions ###
318 | 
319 | ![Simplified get instructions](figures/17_12_get_instructions.jpg)
320 | 
321 | ![A get instruction for formal variables](figures/17_13_getv_instruction.jpg)
322 | 
323 | ***Get instructions*** perform the initial unification checks between the
324 | actual arguments (found in A registers) and the formal arguments in the head of
325 | a potentially applicable clause. They are used right after an ***allocate*** in
326 | the code section for that clause. At this point both B and E point to the same
327 | location in the same choice point.
328 | 
329 | There is typically one get per formal argument. The form of the get depends on
330 | the type of the formal argument.
331 | 
332 | When a get instruction has found that some object is being matched to
333 | a curently unbound variable, unification always works, with
334 | a ***substitution*** generated which records the binding of the object's value
335 | to the variable. In the WAM this substitution is recorded by writing into the
336 | memory cell asociated with the variable the tag and value of the object. This
337 | is fine if the rest of the head literal/goal unification goes through, but the
338 | machine must be able to 'unwrite' the substitution if a later get finds
339 | a contradiction.
340 | 
341 | The process of recording the information necessary to repeal the subsitution
342 | if necessary is termed ***trailing***. It consists of pushing onto ***trail
343 | stack*** the address of the variable being bound. To trail a variable, we
344 | simply push a copy of its memory cell to the trail stack.
345 | 
346 | The failure of a get instruction to find a match between its argument and the
347 | corresponding A register causes a ***shalow backtrack***.
348 | 
349 | #### Explicit Get Instructions ####
350 | 
351 | When a programmer writes:
352 | 
353 | 	P(11,(Pete.Mike),age(Pete,40)) := ...
354 | 
355 | the code representing this would use a ***get-constant*** instruction to check
356 | the first actual argument, a ***get-list*** instruction to check the second,
357 | adn a ***get-structure*** instruction to check the third.
358 | 
359 | ***Get-constant*** takes the A register, dereference it, if the result has
360 | a tag of ***constant***, then the value fields are compared. A match means that
361 | the unification is successful, and execution continues. A mismatch means that
362 | the actual and formal arguments are not unifiable and this clause cannot be
363 | used for the current goal. The failure sequence described above is then invoked
364 | to start up the code for some other potential clause.
365 | 
366 | If the result of the dereference has a tag of ***variable***, then that
367 | variable is trailed and a copy of the constant is stored into the variable's
368 | corresponding memory cell (wiping out the ***variable*** tag). This corresponds
369 | to a successful unification where a substitution is necessary.
370 | 
371 | ***Get-list*** exprects that actual argument is list or unbound variable. If it
372 | is a list, a later pair of unify instructions will check that the car and cdr
373 | of the actual list match what is expected by the formal arguments. To set up
374 | for this test, the get-list will set the ***mode flag*** status bit in the CPU
375 | to ***read mode*** and set the SP register to  point to the memory location
376 | containing the car of the actual goal's list.
377 | 
378 | If the tag of the actual argument is a ***variable***, then we still have
379 | a successul unification, but we must now generate a substitution for that
380 | variable where the variable's ew value is the formal list. In this case the
381 | get-list sets the tag of the variable cell to ***list*** and gives the cell's
382 | value field a copy of the current H register. The following unify instructions
383 | will build a copy of the desired list there. In addition, this instruction sets
384 | the ***mode flag*** to ***write mode***, telling these unify instructions to
385 | build such list.
386 | 
387 | The ***get-structure*** is similar. After dereferencing the actual argument,
388 | this instruction expects to see either a variable or a structure pointer. In
389 | the latter case, the value for the functor name and arity is extracted and
390 | compared to that stored in the instruction. A mismatch causes failure.
391 | 
392 | A match means that the actual and formal arguments have at least the same
393 | function symbol and the same number of arguments. Following unify instructions
394 | will check the components for matches. This is signalled by setting ***mode
395 | flag*** to ***read mode*** and SP to point to one memory cell beyond the actual
396 | argument's ***structure*** cell.
397 | 
398 | An actual argument that is an unbound variable causes that variable to be
399 | trailed and the corresponding cell overwritten by a tag of ***structure
400 | pointer*** and a value equaling the current H register value. The cell at
401 | memory[H] receives a tag of ***structure*** and a value equaling the functor/arity
402 | code from the get-structure instruction. H is incremented to indicate tha
403 | a cellhas been assigned, and the ***mode flag*** is set to ***write mode***.
404 | 
405 | #### Variable Get Instructions ####
406 | 
407 | The ***getv*** instruction handles the case where the formal argument is
408 | a clause variable. This is complex because the compiler cannot always know
409 | beforehand whether or not this clause variable might have a value at
410 | a particular point, or even what kind of value that might be.
411 | 
412 | As an example consider the case where the clause head is `p(x,x)` generating
413 | code:
414 | 
415 | 	getv 1,A1; Assume 1 is offset in environment for x
416 | 	getv 1,A2; See if first argument matches the second.
417 | 
418 | For the case where the goal is `p(2,2)`, the first getv binds 2 to x and the
419 | second one does simple constant to constant test.
420 | 
421 | Now consider a goal of the form:
422 | 
423 | 	p(g(h(3), (Pete.(a.Tim)),h(a)), g(b, (Pete.c), b))
424 | 
425 | In this goal p has two actual arguments both complex structures. The first getv
426 | recognizes that x is unbound and binds to it a structure representing
427 | `g(h(3),(Pete.(a.Tim)),h(a))`. The second getv must recognize that x is now
428 | bound to a structure which does have a matching functor and whose arguments can
429 | be unified by binding `h(3)` to `b`, `3` to `a`, and `(3.Tim)` to `c`. Further,
430 | several of the arguments are themselves complex objects requiring checks of
431 | their arguments.
432 | 
433 | Such process is potentially recursive, requiring some sort of internal stack to
434 | keep track of complex objects that are not yet bound. In the WAM this is the
435 | puprose of the ***PDL*** (or ***push-down list***). This stack is emptied at
436 | the start of each getv, and as complex structures are found that must be
437 | matched, a triple consisting of the starting addresses of the actual and formal
438 | objects and the number of consecutive cells to compare is pushed onto the PDL.
439 | 
440 | 
441 | ### Put Instructions ###
442 | 
443 | ![Simplified put instructions](figures/17_15_put_instructions.jpg)
444 | 
445 | ***Put instructions*** are used to load the A registers with the actual
446 | arguments to be passed on to predicates found in the body of a clause. They
447 | occur in bunches, one bunch per literal on the right-hand side, with one put in
448 | each bunch for each top-level arugment in the corresponding literal. For the
449 | most part heir operation consists of simply copying something and involves no
450 | possibilit of a fail or backtrack. They correspond to a load register
451 | instructions found in conventional ISAs.
452 | 
453 | For complex argument objects these instructions handle only the start of the
454 | object. The setting of the ***mode flag*** to ***write mode*** indicates to the
455 | unifys that objects are to be built.
456 | 
457 | ### Unify Instructions ###
458 | 
459 | ![Simplified unify instructions](figures/17_16_unify_instructions.jpg)
460 | 
461 | A sequence of ***Unify instructions*** are used afer get or put instructions to
462 | handle components of lists or structures. They operate in one of two modes
463 | signalled by the current value in the ***mode flag*** - ***read*** or
464 | ***write***. In read mode they simply attempt to unify the next component of
465 | the object (as pointed to by the SP register) with the variable or constant
466 | specified in the instruction. A successful match may cause variables to be
467 | trailed and bound as in get, and increments SP to point to the next component.
468 | Also as before, a mismatch causes a ***fail sequence*** to back the processor
469 | up to try the next clause. Only get instructions can set the read mode.
470 | 
471 | In write mode, these instructions copy the specified constant or variable to
472 | the object being built up on the heap. The initial get or put has earlier given
473 | either a register or a variable a reference to the start of this object. The
474 | SP register is not needed. Pushing on the heap increments the H register.
475 | 
476 | There are no unify-list and unify-structure instructions. A simpler approach to
477 | handling such situations exist:
478 | 
479 | 1. For each list or structure used as a component of a complex formal argument
480 |    in the head of a clause, allocate an extra local clause variable cell not to
481 |    be used anywhere else in the clause.
482 | 2. When the place in the code is reached where a unify-list or unify-structure
483 |    would be used, replace it by a unifyv with an argument that specifies the
484 |    new variable defined in step 1.
485 | 3. After completion of the top-level code for that formal argument, generate
486 |    a putv to load some unneeded argument register with the contents of one of
487 |    these new variables.
488 | 4. Follow this by a get-list or get-structure as appropriate against this
489 |    argument register.
490 | 5. Use unify instructions as above to complete the components of this new
491 |    structure.
492 | 
493 | ![Some complex objects and their WAM code](figures/17_17_complex_objects.jpg)
494 | 
495 | ## Prolog-to-WAM Compiler Overview ##
496 | 
497 | ![Compilation of the symbolic add procedure](figures/17_18_add_compilation.jpg)
498 | 
499 | There are two major sections to this compiler. First an outer loop that cycles
500 | through the clauses and chains them into procedures. Second is the compilation
501 | of a single clause into a code section for the above procedure chains.
502 | 
503 | ### Procedure-Level Compilation ###
504 | 
505 | The following is a high-level description of the steps that might be involved
506 | in cycling throuhg the clauses and linking sections of code together. It
507 | assumes we maintain a ***symbol table*** containing an entry for each symbol
508 | used as the predicate symbol of the head of some clause. At minimum, an entry
509 | in this table has:
510 | 
511 | * the name of a symbol
512 | * the memory address of the initial mark instruction for the symbol
513 | * the memory address of the last retry-me-else instruction compiled for that
514 |   symbol
515 | * space for a list of places in the program where this predicate symbol has
516 |   been referenced as a right-hand-side goal literal (where it shows up as the
517 |   argument to a call)
518 | * a flag indicating whether or not any code has been generated yet for the
519 |   predicate symbol.
520 | 
521 | We assume below that this table is built first, with all entries but the first
522 | initialized to appropriate nulls.
523 | 
524 | The program clauses are then processed in the order they were entered by the
525 | programmer using following algorithm:
526 | 
527 | 1. select the next unprocessed clause from the program and get the predicate
528 |    symbol used in the head literal.
529 | 2. if the symbol table entry for this symbol indicates that no code has been
530 |    generated for it yet:
531 | 	1. mark the entry as having had code generated
532 | 	2. record as the initial address the next available memory location
533 | 	3. compile into this location a mark instruction, followed by
534 | 	   a retry-me-else with the label field left empty
535 | 	4. save in the symbol table entry the address of the retry's label
536 | 3. if the symbol table entry indicated that some code has already been
537 |    generated:
538 | 	1. store into the memory word designated by the last retry field the
539 | 	   address of the next available word in memory
540 | 	2. compile into this location a retry-me-else instruction with the label
541 | 	   left empty.
542 |     3. save in the symbol table entry the address of the retry's label
543 | 4. generate a code section for the clause as described in next section
544 | 5. If there are more clauses in the program, go to 1
545 | 6. After compilling all clauses
546 | 	1. compile into the next available location a backtrack instruction,
547 | 	   remembering the address where it went
548 | 	2. for each symbol table entry, fix up the label field of the last retry
549 | 	   instruction to point to this backtrack
550 | 	3. for each symbol table entry, go through the list of addresses of call
551 | 	   instructions that used that symbol, adn write into those locations the
552 | 	   address of the mark instruction.
553 | 7. compile the query using a variant of the clause code generator (no head
554 |    unification code is needed)
555 | 8. the first instruction of the query's code is the program's starting point.
556 | 
557 | ### Clause-Level Compilation ###
558 | 
559 | This section assumes that a clause has been selected for compilation by the
560 | outer compiler routine, adn that all the appropriate interclause link addresses
561 | are set up correctly in the symbol table.
562 | 
563 | 1. calculate the number of local variables needed, including allowances for
564 |    temporaries used for complex objects buried inside other complex objects. If
565 |    the number is not zero, generate an allocate instruction. Also build a list
566 |    pairing local variable names and their offsets.
567 | 2. process the k-th formal argument of the head as follows (k=1,2...):
568 | 	1. if it is a constant, generate a get-constant instruction, encoding in
569 | 	   the value of the constant and Ak.
570 | 	2. if it is a variable, generate a getv instruction, encoding in the offset
571 | 	   to that variable from the above-computed pairings and Ak.
572 | 	3. if it is a list, generate a get-list instruction which references Ak.
573 | 	   Then for the car and cdr of this list generate either a unify-constant
574 | 	   or unifyv instruction, as appropriate. If either car or cdr is a complex
575 | 	   object, generate a unifyv instructionwhich refers to one of the
576 | 	   allocated temporaries.
577 | 	4. If it is a structure, generate a get-structure instruction, encoding in
578 | 	   the name of the functor and its arity. Then do exactly as described for
579 | 	   lists for each argument of this expression.
580 | 3. for each complex object that was a component of some other object:
581 | 	1. generate a putv instruction which refers to the allocated local variable
582 | 	   and to some argument register that is not needed any more.
583 | 	2. generate either a get-list or a get-structure instruction as appropriate
584 | 	   against this register.
585 | 	3. for each component of this object, generate code as was done above.
586 | 4. process the goal literals on the right-hand side one at a time, from left to
587 |    right.
588 | 	1. process the i-th top-level argument of the next literal as follows
589 | 	   (i=1,2,...):
590 | 		1. if it is a constant, generate a put-constant instruction, encoding
591 | 		   in the constant's value and Ak.
592 | 		2. if it is a variable, generate a putv instruction, encoding in Ak and
593 | 		   the offset to the variable from the pairings developed in the first
594 | 		   step.
595 | 		3. If it is a list or a structure all of whose components are
596 | 		   either variables or constants, generate either a put-list or
597 | 		   a put-structure as appropriate (with Ak encoded), and then generate
598 | 		   a unify-constant or unifyv for each argument.
599 | 		4. if it is a list or structure that includes as embedded list or
600 | 		   structure:
601 | 			1. take the most deeply nested such component
602 | 			2. select a currently unused argument register Au
603 | 			3. generate an instruction sequence as in the prior step (put-list
604 | 			   or put-structure), but target the result to Au.
605 | 			4. generate a getv instruction to place Au in a specially allocated
606 | 			   clause variable (as was done for nested objects in the clause
607 | 			   head)
608 | 			5. repeat the above process for the next most nested component,
609 | 			   except that for components that refer to nested structures that
610 | 			   have already been processed, use a unifyv iinstruction with the
611 | 			   offset of the clause variable into which they were compoled
612 | 			   earlier.
613 | 			6. mark the register Au as free again.
614 | 	2. generate a call instruction leaving the label field empty.
615 | 	3. link the address of this field into the symbol table entry for the
616 | 	   predicate being called.
617 | 5. complete the code by generating a return instruction
618 | 
619 | Note that the processing order for complex objects as they are built for
620 | right-hand side  goals is just the opposite of that for head unification,
621 | namely, inside out versus outside in. they do, however, use the same ide of
622 | temporarily saving a partially processed object in an extra clause variable
623 | until it is needed. Note also that one of the final steps in the outer loop
624 | uses information from the symbol table to fix up all the labels for the call
625 | instructions to point to the correct procedure entry points.
626 | 
627 | ## Supporting Built-ins ##
628 | 
629 | The WAM as described supports pure Prolog, without special built-in predicates
630 | that have side effects, such sa cut, I/O, ad the various predicates to read and
631 | modify the program dynamically.
632 | 
633 | ### Some Simple New Instructions ###
634 | 
635 | ![New WAM instructions to support built-ins](figures/17_19_builtin_instructions.jpg)
636 | 
637 | ***Fail*** invoked the failure sequence.
638 | 
639 | ***Escape*** permits a WAM machine to communicate with some other processor
640 | (perhaps one that is capable of arithmetic functions, IO...). The instruction
641 | simply takes the current argument registers, places theme somewhere the other
642 | processor can access them, signals the other processor and waits for
643 | a completion.
644 | 
645 | ***Switch-on-type*** specifies an argument register and four addresses to
646 | branch to. The argument is dereferenced and the tag is tested. Depending on the
647 | tag one of the four branch addresses are placed in the PC.
648 | 
649 | To understand ***cut*** consider a clause of the form:
650 | 
651 | 	p(...) := q1(...),...qn(...),!,r1(...),...,rm(...).
652 | 
653 | When converted into WAM cde, the code to support the cut operation should come
654 | immediately after the call qn instruction. Following this code should then be
655 | the normal puts in support of r1.
656 | 
657 | If the program execution ever reaches the cut code, B points to the most recent
658 | choice point build in support of qn, while E points to the earlier one
659 | established for p. The semantics of a cut dictate that any backtracks through
660 | it will always succeed, but will have the effect that any backtracks through it
661 | should result in a backtrack through the p choice point. It should be as if the
662 | choice points for q1 to qn never existed, and that this clause is the last one
663 | possible for the p predicate.
664 | 
665 | One way to do this is to have the cut code set B to poin to the same choice
666 | point as E does, and to load FA[E] with the address of some location known to
667 | hold a backtrack instruction. Now if the code for r1 backtracks, it will
668 | restart p's choice point, which will branch to a backtract instruction, which
669 | in turn will restart the prior choice point as desired.
670 | 
671 | ### Multiiinstruction Build-ins ###
672 | 
673 | ![Some multi-WAM instruction built-ins](figures/17_20_multiinstruction.jpg)
674 | 
675 | ***Disjunction*** `;` is worth discussion. What it does is build a separate
676 | choice point for the goals involved that will try the second if the first does
677 | not succeed, and so on.
678 | 
679 | ### Tough Predicates ###
680 | 
681 | Some of the Prolog built-in predicates represent a very tough challenge to
682 | implementation with WAM code. These include the predicates that read and
683 | modify the set of Prolog statements during the execution, such as ***clause,
684 | assert, retract***.
685 | 
686 | ## Detailed Example ##
687 | 
688 | ![Simplified WAM code for append](figures/17_22_code_for_append.jpg)
689 | 
690 | Any time the append predicate symbol shows up in a goal, control is transferred
691 | to the first instruction, the mark, with three arguments in argument registers
692 | A1, A2, A3. This instruction builds the initial choice point and drops down to
693 | the entry append1 for the first clause. The entry code here consists of
694 | a retry-me-else which designated append2 as the starting location for the next
695 | possible rule for append and allocates space for the single clause variable x.
696 | The body of the clause code consists of checking that the first argument is
697 | nil, and then that the second and third arguments are the same. The final
698 | instruction, return, returns control to the caller if this all worked.
699 | 
700 | The second clause is a little more complex. Here the first argument is
701 | supposed to be a list, so the unification code for the first argument starts
702 | with a get-list. If A1 is a list, this will set the SP register to poin to its
703 | car cell, and set the machine to rad mode. The following two unifyv
704 | instructions then try to unify the car of the actual list with H, and the cdr
705 | with L1. Both of these have no current value, so the net effect is a copying of
706 | the car and cdr of the actual list into these two variable locations in th
707 | environment.
708 | 
709 | If the actual argument in A1 is an unbound variable to begin with, a copy of
710 | the variable's cell is pushed to the trail, the cell itself is loaded with
711 | a tag of list and a value pointing to the top of the heap, and the machine is
712 | set to write mode. We will build for the variable a new list on the heap. The
713 | two unifyv instructions handle the specifications for the car and cdr of this
714 | list, and will create the appropriate entries on the heap.
715 | 
716 | Another getv and then get-list, unifyv, unifyv combination follows to process
717 | the other two arguments.
718 | 
719 | Following this code, three putvs create the new argument values needed to call
720 | append from the body. Note that copies of the original A registers are in the
721 | choice point where they get reloaded if a backtrack occurs.
722 | 
723 | The recursive call to append repeats this whole process over again with the new
724 | arguments.
725 | 
726 | Upon a successful return from one call, the return instruction will
727 | successfully return from either code sequence.
728 | 
729 | The final instruction is a backtrack, and is positioned as a new clause if the
730 | second one fails. This signals that there are no more rules for this goan and
731 | deep backtrack must occur.
732 | 
733 | ## Problems ##
734 | 
735 | 1. Convert the set of Prolog statements for each of the following predicate
736 |    symbols as found in the text into a sequence of simplified WAM instructions:
737 |    1. member(x,y) [use z in place of the anonymous variable]
738 | 
739 | 			#set membership
740 | 			member(x,(x._)).
741 | 			member(x,(_.y)) :- member(x,y)
742 | 
743 | 		Solution:
744 | 
745 | 			;at entry registers A1, A2 hold two arguments
746 | 			member:  mark
747 | 			member1: retry-me-else member2
748 | 					 allocate 3
749 | 					 getv y1, A1
750 | 					 get-list A2
751 | 					 unifyv y2
752 | 					 unifyv y3
753 | 					 getv y1, y2
754 | 					 return
755 | 			member2: retry-me-else quit
756 | 					 allocate 3
757 | 					 getv y1, A1
758 | 					 get-list A2
759 | 					 unifyv y2
760 | 					 unifyv y3
761 | 					 putv y1, A1
762 | 					 putv y3, A2
763 | 					 call member
764 | 					 return
765 | 			quit:    backtrack
766 | 
767 |    2. reverse(x,y)
768 | 
769 | 			#second argument is reversed list
770 | 			reverse(x,y) :- rev(x,(),y).
771 | 			rev(nil,y,y).
772 | 			rev((h.t),y,z) :- rev(t,(h.y),z)
773 | 
774 | 		Solution:
775 | 
776 | 			reverse:  mark
777 | 			reverse1: retry-me-else quit
778 | 					  allocate 2
779 | 					  getv y1, A1
780 | 					  getv y2, A2
781 | 					  putv y2, A3
782 | 					  put-constant nil, A2
783 | 					  call rev
784 | 					  return
785 | 			quit:	  backtrack
786 | 
787 | 			rev:	  mark
788 | 			rev1:	  retry-me-else rev2
789 | 					  allocate 1
790 | 					  get-constant nil, A1
791 | 					  getv y1, A2
792 | 					  getv y1, A3
793 | 					  return
794 | 			rev2:     retry-me-else quit2
795 | 					  allocate 4
796 | 					  get-list A1
797 | 					  unifyv y1
798 | 					  unifyv y2
799 | 					  getv y3, A2
800 | 					  getv y4, A3
801 | 					  putv y2, A1
802 | 					  put-list A2
803 | 					  unifyv y1
804 | 					  unifyv y3
805 | 					  putv y4, A3
806 | 					  call rev
807 | 					  return
808 | 			quit2:    backtrack
809 | 
810 | 
811 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Architecture of Symbolic Computers #
2 | 
3 | My notes from reading this amazing book by Peter M. Kogge. Shame it is out of
4 | print now.
5 | 


--------------------------------------------------------------------------------
/figures/06_01_metainterpreters.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/06_01_metainterpreters.png


--------------------------------------------------------------------------------
/figures/06_06_recursive_closures.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/06_06_recursive_closures.jpg


--------------------------------------------------------------------------------
/figures/07_01_secd.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/07_01_secd.jpg


--------------------------------------------------------------------------------
/figures/07_02_memory_model.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/07_02_memory_model.jpg


--------------------------------------------------------------------------------
/figures/07_10_recursive_closures.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/07_10_recursive_closures.jpg


--------------------------------------------------------------------------------
/figures/07_11_stack.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/07_11_stack.jpg


--------------------------------------------------------------------------------
/figures/07_19_rap.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/07_19_rap.jpg


--------------------------------------------------------------------------------
/figures/10_15_cadr_machine.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/10_15_cadr_machine.jpg


--------------------------------------------------------------------------------
/figures/10_16_memory_word.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/10_16_memory_word.jpg


--------------------------------------------------------------------------------
/figures/10_17_cdr_coding.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/10_17_cdr_coding.jpg


--------------------------------------------------------------------------------
/figures/10_18_compact_lisp_machine.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/10_18_compact_lisp_machine.jpg


--------------------------------------------------------------------------------
/figures/10_22_register_windows.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/10_22_register_windows.jpg


--------------------------------------------------------------------------------
/figures/10_23_spur_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/10_23_spur_instructions.jpg


--------------------------------------------------------------------------------
/figures/13_01_logic_computing.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/13_01_logic_computing.jpg


--------------------------------------------------------------------------------
/figures/13_05_logic_system.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/13_05_logic_system.jpg


--------------------------------------------------------------------------------
/figures/13_06_satisfying_interpretation.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/13_06_satisfying_interpretation.jpg


--------------------------------------------------------------------------------
/figures/13_10_connectives.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/13_10_connectives.jpg


--------------------------------------------------------------------------------
/figures/13_12_axiom_schemas.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/13_12_axiom_schemas.jpg


--------------------------------------------------------------------------------
/figures/13_13_truth_table.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/13_13_truth_table.jpg


--------------------------------------------------------------------------------
/figures/14_02_standard_logic_forms.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_02_standard_logic_forms.jpg


--------------------------------------------------------------------------------
/figures/14_04_conversion_propositional.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_04_conversion_propositional.jpg


--------------------------------------------------------------------------------
/figures/14_06_conversion_prenex.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_06_conversion_prenex.jpg


--------------------------------------------------------------------------------
/figures/14_09_horn_clauses.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_09_horn_clauses.jpg


--------------------------------------------------------------------------------
/figures/14_10_combination_of_axioms.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_10_combination_of_axioms.jpg


--------------------------------------------------------------------------------
/figures/14_12_herbrand_universe.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_12_herbrand_universe.jpg


--------------------------------------------------------------------------------
/figures/14_13_another_herbrand_universe.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_13_another_herbrand_universe.jpg


--------------------------------------------------------------------------------
/figures/14_14_herbrand_interpretations.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_14_herbrand_interpretations.jpg


--------------------------------------------------------------------------------
/figures/14_15_inference_engine.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/14_15_inference_engine.jpg


--------------------------------------------------------------------------------
/figures/17_01_prolog_execution.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_01_prolog_execution.jpg


--------------------------------------------------------------------------------
/figures/17_02_translation.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_02_translation.jpg


--------------------------------------------------------------------------------
/figures/17_03_structure.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_03_structure.jpg


--------------------------------------------------------------------------------
/figures/17_04_memory.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_04_memory.jpg


--------------------------------------------------------------------------------
/figures/17_05_objects_in_memory.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_05_objects_in_memory.jpg


--------------------------------------------------------------------------------
/figures/17_06_choice_point.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_06_choice_point.jpg


--------------------------------------------------------------------------------
/figures/17_07_procedure_code_segment.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_07_procedure_code_segment.jpg


--------------------------------------------------------------------------------
/figures/17_08_quick_sort.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_08_quick_sort.jpg


--------------------------------------------------------------------------------
/figures/17_09_indexing_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_09_indexing_instructions.jpg


--------------------------------------------------------------------------------
/figures/17_10_procedural_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_10_procedural_instructions.jpg


--------------------------------------------------------------------------------
/figures/17_11_choice_points.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_11_choice_points.jpg


--------------------------------------------------------------------------------
/figures/17_12_get_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_12_get_instructions.jpg


--------------------------------------------------------------------------------
/figures/17_13_getv_instruction.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_13_getv_instruction.jpg


--------------------------------------------------------------------------------
/figures/17_15_put_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_15_put_instructions.jpg


--------------------------------------------------------------------------------
/figures/17_16_unify_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_16_unify_instructions.jpg


--------------------------------------------------------------------------------
/figures/17_17_complex_objects.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_17_complex_objects.jpg


--------------------------------------------------------------------------------
/figures/17_18_add_compilation.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_18_add_compilation.jpg


--------------------------------------------------------------------------------
/figures/17_19_builtin_instructions.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_19_builtin_instructions.jpg


--------------------------------------------------------------------------------
/figures/17_20_multiinstruction.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_20_multiinstruction.jpg


--------------------------------------------------------------------------------
/figures/17_22_code_for_append.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hlopko/architecture_of_symbolic_computers_notes/0d034770b707ffa1e0412a6cc445425da902036a/figures/17_22_code_for_append.jpg


--------------------------------------------------------------------------------