├── graphs.pdf ├── .gitignore ├── README.md └── graphs.tex /graphs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jozefg/graph-models/HEAD/graphs.pdf -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.aux 2 | *.log 3 | auto/* 4 | *.pyg 5 | _minted*/* 6 | *.fls 7 | *.fdb_latexmk 8 | _region_.tex 9 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## graph models 2 | 3 | Really old notes from a talk Dana Scott gave at CMU. Just about 4 | P-omega, one of the simplest mathematical models for the lambda 5 | calculus. 6 | -------------------------------------------------------------------------------- /graphs.tex: -------------------------------------------------------------------------------- 1 | \documentclass{amsart} 2 | \usepackage{amsmath, amssymb, stmaryrd} 3 | \usepackage[parfill]{parskip} 4 | 5 | \newcommand{\set}[1]{\ensuremath{\mathbf{set}(#1)}} 6 | \newcommand{\unset}[1]{\ensuremath{\mathbf{set}^{-1}(#1)}} 7 | \newcommand{\enc}{\text{enc}} 8 | \newcommand{\dec}{\text{dec}} 9 | \newcommand{\bbrack}[1]{\ensuremath{\llbracket #1 \rrbracket}} 10 | 11 | \title{A Simple Model of The Lambda Calculus} 12 | \author{Danny Gratzer} 13 | 14 | \begin{document} 15 | \maketitle 16 | 17 | 18 | Hi folks. Today I'd like to talk about something that came up during 19 | the talks Dana Scott gave this week at CMU. He was discussing a new 20 | way of integrating randomness into models of the lambda calculus, but 21 | the model which he was building this all on was surprisingly simple 22 | and nice. In an attempt to consolidate my notes I'm writing down 23 | everything that I can remember about what are called "graph models" of 24 | the lambda calculus. Hopefully it'll be helpful to someone 25 | else. 26 | 27 | Since these are just my notes they come with a huge disclaimer, 28 | they're only correct to the best of my knowledge 29 | 30 | \section{Observations about Models of The Lambda Calculus} 31 | 32 | If we want to construct a model of the lambda calculus we need to 33 | construct some domain of discourse, $D$ with operations on it letting 34 | us model the critical rules of the lambda calculus. Specifically we 35 | need to be able to simulate one particularly troublesome rule. 36 | 37 | \[ 38 | (\lambda x.\ E)E' \equiv [E'/x]E 39 | \] 40 | 41 | What makes this rule such a bother is that it forces us construct some 42 | way to convert $d \in D$ into a function upon $D$. Furthermore, in 43 | order to properly interpret lambdas into the model we'll want to find 44 | some way to turn a function on $D$ back into a term on $D$. It 45 | doesn't necessarily need to be an isomorphism, but it certainly needs 46 | to be an embedding. That is, if we have $\enc : (D \to D) \to D$ and 47 | 48 | $\dec : D \to (D \to D)$ it really needs to be the case that 49 | $\dec \circ \enc = 1$. It would be even better though to be able to 50 | construct a model where these two collections are isomorphic, meaning 51 | that all functions on our domain are computable functions and all 52 | elements of our domain are computable functions. This is not as easy 53 | as one would hope though, if our domain has 2 distinct elements (it 54 | needs to in order to be a reasonable model) then 55 | $D \to D \cong \mathcal{P}(D)$. Then Cantor pops up and we know that 56 | we won't be able to just build some set so that $\dec \circ \enc = 1$. 57 | 58 | This circularity is going to come back up in pretty much any model you 59 | attempt to build and this won't be any exception. 60 | 61 | \section{Using Graphs} 62 | 63 | In order to construct our model we're going to think of lambda terms 64 | as graphs. They'll pair their input to it's output rendering the term 65 | into one huge stinking table. This is basically how a set theorist 66 | would describe their functions so it's not completely out of no 67 | where. If a lambda term doesn't terminate on a particular input that's 68 | fine, we'll just neglect to add an entry there. This matches our 69 | intuition that two functions are equivalent if they coterminate and 70 | are equivalent on all the cases where they terminate. 71 | 72 | So then we can think of how to define $\enc$ and $\dec$. 73 | You should think of $\enc$ as a function squashing a 74 | function into the appropriate look up table. 75 | 76 | \begin{align*} 77 | \enc(F) &= \{(m, n) \mid n = F(m)\}\\ 78 | \dec(d) &= m \mapsto 79 | \begin{cases} 80 | n & (m, n) \in d\\ 81 | \bot & \text{otherwise} 82 | \end{cases} 83 | \end{align*} 84 | 85 | There I just mean $\bot$ for undefined, it's notation not some 86 | specific element of our model. This looks so great and yet it's 87 | completely broken. The whole thing starts to unravel as soon as we 88 | ask the question "what are the entries of our tables?" Then we'll 89 | notice that our tables are built of.. tables, made of tables, made of 90 | tables, .. off and off and off to infinity. A little set theory is 91 | enough to convince us that this will lead to paradoxes. In particular 92 | it shouldn't be tricky to show that we couldn't actually realize a set 93 | of such lookup tables. 94 | 95 | So now we've reached the main technical challenge, this is the 96 | circularity I've hinted at before; the model we want to define is just 97 | too big to exist. In order to make sure that we don't make the same 98 | mistake twice let's get a little more serious about our math and fix 99 | some concrete set to work with. Specifically, let's look at the 100 | powerset of natural numbers. There's nothing special about the natural 101 | numbers other than them having an infinite number of them. 102 | 103 | Our clever idea here is that while we definitely can't encode tables 104 | of tables as sets of numbers, we can encode pairs of natural numbers 105 | as natural numbers. It's not so hard, just 106 | \[ 107 | (n, m) \triangleq 2^n(2m + 1) 108 | \] 109 | 110 | ("Fun" exercise, convince yourself that this is injective) 111 | 112 | Pairs are a good start, but we need more than that, we need a finite 113 | set of pairs. But looking to lisp, we can easily see we can encode 114 | finite sequences with our pairing operation as cons and nil is just 115 | encoded as 0 (our pairing operation never gives 0). And from sequences 116 | there's a simple but woefully non-injective correspondence between 117 | finite sets of pairs and sequence pairs. These can be mapped to 118 | sequences of natural numbers, which are just natural numbers. I'll 119 | use the notation \set{n} to indicate the finite set of natural numbers 120 | $n$ denotes by this process. 121 | 122 | So we've G\"odel numbered our way to being able to think of finite 123 | tables as numbers. So, since we're talking about 124 | $\mathcal{P}(\mathbb{N})$, that means we could regard each element 125 | really as an approximation of the structure we were talking about 126 | before where each time we go down a level in the table we go from 127 | infinitely entries at the top to finitely many entries, and then fewer 128 | and fewer until we're completely out. We're forced into a much smaller 129 | model than we had before but it actually exists which is a step up I 130 | suppose. 131 | 132 | Now we have to convince ourselves that we can work in such a 133 | restricted model. Let's define a few helpful concepts to help us show 134 | this. Let 135 | 136 | \[ 137 | X^* \triangleq \{n \mid \set{n} \subseteq X\} 138 | \] 139 | 140 | We then know that 141 | $X^* = \{\unset{F} \mid F \text{ finite} \wedge F \subseteq X\}$. We 142 | can see $X^*$ as a collection of approximations of $X$. It is after 143 | all just an encoding of a set of sets which all describe some finite 144 | fragment of $X$. It's not a terribly efficient way of describing $X$ 145 | because there's lots of duplication between approximations but having 146 | all the finite approximations of a set is enough to uniquely determine 147 | it (just union the approximations). 148 | 149 | What we do then is define $\dec$ and $\enc$ by piecing together all of 150 | the approximations of relevant sets. 151 | \begin{align*} 152 | \enc(F) &= \{(n, m) \mid m \in F(\set{n})^*\} \cup \{0\}\\ 153 | \dec(d) &= X \mapsto 154 | \bigcup \{\set{m} \mid \exists n \in X^*.\ (n, m) \in d\} 155 | \end{align*} 156 | 157 | So $\enc$ creates a table mapping a fragment of input to the 158 | appropriate fragment of output. We also tack on $\{0\}$ which is our 159 | G\"odel encoding of $\emptyset$ because ``completely undefined'' is 160 | always a valid approximation of a function, if a poor one. To reverse 161 | the process and get a function out of a lookup table we take our input 162 | and rip it apart into all its valid approximations and piece together 163 | what the lookup table gives us. 164 | 165 | The natural question to ask is whether these functions cohere with 166 | each other. Is it the case that $\dec \circ \enc = 1$ for example? 167 | The short answer is no for a variety of reasons. Remember that while 168 | we're encoding everything into a lookup table we can only ever encode 169 | finite entries, infinite sets can't have a G\"odel number. In order 170 | for us to be able prove that $\dec \circ \enc$ could \emph{ever} be an 171 | identity it needs to be possible to determine the behavior of our 172 | function from only the finite cases. 173 | 174 | This idea is a driving idea between a lot of Scott's work on lattice 175 | theoretic models. It stems from the observation that computation is 176 | smooth. In the more general sense we can state this property with the 177 | preservation of least upper bounds of directed sets. In our specific 178 | case we can use the more simplistic definition that $F$ is continuous 179 | if 180 | \[ 181 | F(X) = \bigcup \{F(Y) \mid Y \subseteq X \wedge Y \text{ finite}\} 182 | \] 183 | 184 | If we apply our trick with G\"odel numbers this is equivalent to 185 | \[ 186 | F(X) = \bigcup \{F(\set{n}) \mid n \in X^*\} 187 | \] 188 | 189 | It's not hard to prove that if $F$ is continuous then 190 | $\dec(\enc(F)) = F$. The reverse has some complications, while 191 | $\enc \circ \dec$ will ``morally'' work on the appropriate class of 192 | sets, it won't return the exact same set on the nose. It may return 193 | any of a number of tables which describe the same function in subtly 194 | different ways. 195 | 196 | Now finally, we can define a model of the lambda calculus which 197 | satisfies $\alpha$ and $\beta$ conversions. 198 | 199 | \begin{align*} 200 | \bbrack{x}(\rho) &= \rho(x)\\ 201 | \bbrack{M\ N}(\rho) &= \dec(\bbrack{M}(\rho))(\bbrack{N}(\rho))\\ 202 | \bbrack{\lambda x.\ E}(\rho) &= \enc(X \mapsto \bbrack{E}(\rho[x \mapsto X])) 203 | \end{align*} 204 | 205 | The unfortunate bit of this model is that it's not extensional. It's 206 | not the case that two lambda terms are equal if and only if they 207 | behave the same when applied. There are many different reasons for 208 | this, but the first and most obvious one is that two lambda terms are 209 | only on the nose equal if they are defined by the same lookup 210 | table. But any given function can be given infinitely many finitely 211 | sized look up tables! This means that even if $\dec$ claims the tables 212 | are the same function they may not be equal as elements of 213 | $\mathcal{P}(\mathbb{N})$. For many applications though this is just 214 | fine, in particular we can still analyze questions of computability 215 | and recursion theory, it's just not a good model to use for proving 216 | some kinds of metatheorems about the lambda calculus. 217 | 218 | \section{Conclusion} 219 | 220 | This wraps up my presentation of a simple graph model of the lambda 221 | calculus. The key insight into all of this is that computable 222 | functions are continuous. From there it's only natural to take 223 | advantage of the fact that continuous functions are easily encodable 224 | in countable amounts of information which allows us to construct 225 | $\enc$ and $\dec$ so that $\dec \circ \enc = 1$. 226 | 227 | Questions I'm left with include 228 | 229 | \begin{itemize} 230 | \item This notion of continuity can be tied back to what topology on 231 | $\mathcal{P}(\mathbb{N})$?\\ 232 | 233 | The answer to the question with a few years of hindsight is that, 234 | yes, it can be. In fact, the topology to which our notion of continuity 235 | corresponds is called the \emph{Scott topology} and it's precisely 236 | the topology generated by the basic opens of 237 | \[ 238 | O_F = \{X \subseteq \mathbb{N} \mid F\ \mathrm{finite} \mathrel{\wedge} F \subseteq X\} 239 | \] 240 | \item The loss of extensionality is a little disappointing, but it 241 | seems possible that it would be recoverable if we could quotient our 242 | set by $\dec$ or something similar. Is it really that easy?\\ 243 | 244 | The answer to this question is somewhat less obvious. I would hazard 245 | a guess that you can recover an extensional model by appropriate 246 | quotienting but the result is unlikely to be as useful since it 247 | won't contain the same natural mathematical structure. $D_\infty$ 248 | would be a much more compelling choice if extensionality is truly 249 | required to show the desired property. 250 | \end{itemize} 251 | \end{document} 252 | --------------------------------------------------------------------------------