├── 2024-12-09-poetry-to-prose.md ├── 2024-12-12-calculus-calculi.md ├── 2024-12-13-fame.md ├── 2024-12-14-logic.md ├── 2024-12-20-infinity.md ├── 2024-12-23-tweets-on-trees.md ├── 2024-12-27-linear-algebra.md ├── 2024-12-31-watchword.md ├── 2025-01-02-semantics.md ├── 2025-01-05-type-inference.md ├── 2025-01-06-diagonal.md ├── 2025-01-17-arrogance.md ├── 2025-01-25-engineering.md ├── 2025-02-03-POPL-PEPM.md ├── 2025-02-23-stuck-infer.md ├── 2025-03-04-polymorphism-without-paramters.md ├── 2025-03-17-unique_types.md ├── 2025-03-30-explosion.md └── README.md /2024-12-09-poetry-to-prose.md: -------------------------------------------------------------------------------- 1 | 2 | # Turning Poetry into Prose 3 | ## Barry Jay 4 | ### 2024-12-09 5 | 6 | Hi everyone! I'm going to write about tree calculus and intensional 7 | computation and overturning the standard model of computation etc. Of 8 | course, I've written papers and books about this stuff, have been 9 | chipping away for almost 25 years, but the dominant reaction has been 10 | that the work is cute, or crazy, depending on how much I attack the 11 | Church-Turing Thesis. Along the way, I've noticed that my writing 12 | style gets strong reactions (good and bad!). At first, I couldn't 13 | figure out why, but now think that its because I write poetry; those 14 | expecting prose get frustated. 15 | 16 | To illustrate, here is a concluding slide from my upcoming talk (PEPM, 17 | January 2025, draft paper is online). 18 | 19 | - Encodings are patches. 20 | - Theorems trump theses. 21 | - Programs are normal forms. 22 | - Programs are trees not numbers. 23 | - Tree types become function types. 24 | - Tricky programs are typed by hacking. 25 | 26 | Each sentence has a very simple structure, so it should be easy, 27 | right? But each sentence is loaded with meaning, and provocation - is 28 | that the right word? 29 | 30 | To digress, this blog is going to be in prose. An editor once told me 31 | that in prose, no sentence should require a second reading. I 32 | extrapolate that in poetry, a second reading should enrich the 33 | experience, open up new meanings. To avoid slipping into poetic mode, 34 | I am not going to revise anything I write here (except for spelling 35 | mistakes, perhaps a paragraph break). If I'm not happy with it later, 36 | I will write a new sentence. 37 | 38 | Let's take a look at the "verse" above. 39 | 40 | **Encodings are patches** 41 | 42 | This summarises the argument (or observation) 43 | that the traditional models of computation (the standard model?) 44 | cannot analyse their own programs, cannot analyse their own normal 45 | forms, without encodings, e.g. from lambda-calculus to the nautral 46 | numbers (and back). This leads to a damning criticism of the 47 | Church-Turing Thesis. Oh! No backsies! Let us rather say that it 48 | exposes a limitation of the Church-Turing Thesis, one that was 49 | inconsequential in 1936 but became a real limitation once compilers 50 | were in play. Notice how I used the work "limitation" twice there? Its 51 | a bit ugly, isn't it, but no going back! 52 | 53 | 54 | **Theorems trump theses** 55 | 56 | The argument for the Church-Turing Thesis, well, the original one, was 57 | that since all these different models compute the same numbers, and we 58 | haven't seen anything better, maybe that is all there is. My 59 | refutation of this conjecture (thanks, Popper) is that we also want to 60 | do program analysis. Now tree calculus (actually, "calculi", since 61 | there is more than one now) support program analysis in a way that 62 | none of the traditional models do. So the original argument is blown 63 | out of the water. The other argument I have heard is that everything 64 | can be implemented on a Turing machine; QED. However, these 65 | implementations require encodings, which takes us back to the previous 66 | point. Indeed, modern compilers have many, many layers of encoding, 67 | all of which create friction in the machinery. My hope is that tree 68 | calculus can eliminate many of these intermediate languages. 69 | 70 | **Programs are normal forms** 71 | 72 | In lambda-calculus and combinatory logic it is traditional to 73 | represent recursive functions as fixpoints that do not (in the 74 | interesting cases) have normal forms, and so are unstable with repect 75 | to computation. Thus, program analysis requires encodings to freeze 76 | the target. Although this is unavoidable for lambda-calculus, all 77 | function that are representable in combinatory logic can be 78 | represented by normal forms, so that encodings can be avoided. 79 | 80 | **Programs are trees not number** 81 | 82 | When programs are represented by natural numbers (tapes) then the 83 | program structure is encoded by arithmetic properties, e.g. prime 84 | factorisation of Goedel numbers. But we want to avoid encodings! By 85 | contrast, binary natural trees, (no labels) can 86 | represent program fragments by branches. One way to see this is to 87 | build a syntax tree, and then replace each label by a small natural 88 | tree. So the real challenge is to express the functionality of these 89 | trees through algebra. 90 | 91 | So far, this is the story of my book "Reflective Programs in Tree 92 | Calculus". The latest work "Typed Program Analysis with Encodings" 93 | shows how to type tree calculus. 94 | 95 | **Tree types become function types** 96 | 97 | Since program analysis can reveal all program structure, each program 98 | must have a unique type, a tree type, that exactly captures its 99 | structure. However, many different programs may have the same function 100 | type, so this is captured by subtyping. For example, the reduction rule 101 | 102 | $\Delta \Delta y z \Longrightarrow y$ 103 | 104 | yields the subtyping rule 105 | 106 | $Fork\ Leaf\ Y < Z \rightarrow Y$ 107 | 108 | **Tricky programs are typed by hacking** 109 | 110 | Traditionally, when complex behaviour threatens to break the type 111 | system, new syntax is introduced. For example, when a fixpoint 112 | combinator is represented by the self-application of some combinator 113 | omega which doesn't have a type, the solution has been to add a new 114 | operator $Y$ for fixpoints. Its behaviour is captured by a new reduction 115 | rule 116 | 117 | $Y f \Longrightarrow f (Y f)$ 118 | 119 | and its type is 120 | 121 | $(X \rightarrow X) \rightarrow X$. 122 | 123 | However, there is no need to hack the term language. It is enough to 124 | hack the type system, by adding a subtyping rule for the program type 125 | of omega. 126 | 127 | Ps. I failed to produce a blog with GitHub pages: it wouldn't display my post; 128 | I corrupted the files; it wouldn't let me delete them; I made them private; 129 | I'm not allowed to have two blogs; 130 | and now I can't even see the files to make them public again. 131 | So I'm just making a regular repository (sigh). 132 | You can make pull requests or write me direct on Barry.Jay8@gmail.com 133 | 134 | ### Comment [Joshua Seigler](joshua@seigler.net) 135 | 136 | Hi Barry, 137 | 138 | I came across your site treecalcul.us on HackerNews and it has stuck in my head! As a fan of Douglas Hofstadter's "Godel, Escher, Bach" I've had a little exposure to the idea of lambda calculus, and as a software engineer I enjoy esoteric programming languages. I intend to play with this very interesting thing you have made! I may attempt some old Advent of Code challenges using it. 139 | 140 | Anyway I am writing with a suggestion for your blog GitHub repository. In your post on 2024-12-09 (which I found quite instructive, thanks) you had some trouble getting your Latex syntax working. GitHub flavored markdown supports Latex math syntax inside $ fences. I don't know Latex syntax or I would have opened a PR. 141 | 142 | I hope this is helpful. 143 | 144 | Best regards, 145 | Joshua Seigler 146 | 147 | ### Reply 148 | 149 | Thanks, all sorted now. 150 | 151 | ### Comment [Chewy](bsky.chewxy.com) 152 | 153 | From the first post of your blog: 154 | 155 | ⁠At first, I couldn't figure out why, but now think that its because I write poetry; those expecting prose get frustated. 156 | 157 | That and the PEPM slide that you outline reminds me that I still keep with me a little post-it note from one of our first meetings. In it: 158 | 159 | ⁠Source code is a parse tree. 160 | ⁠A parse tree is but a rose tree. 161 | ⁠A rose tree is but a labelled binary tree. 162 | ⁠A labelled binary tree can be written as an unlabelled binary tree. 163 | ⁠𝔹 is the mathematical object which binary trees belong to. 164 | ⁠Instead of representing programs in ℕ, we can represent them in 𝔹 165 | 166 | Don't think anyone's put that concept up so ... elegantly since you told it to me 167 | 168 | ### Reply 169 | 170 | Wow, this was long before tree calculus. The missing step was to make these trees be functions, by adding some algebra. Some still haven’t grasped this point. 171 | 172 | ### Comment [Chewy](bsky.chewxy.com) 173 | 174 | Alan Kay once said something like "a change in viewpoint is worth 20 IQ points". Yeah, that was definitely well worth more than 20 IQ points. Made me look at programs much differently. I like the more conversational/prose style of your blog btw 175 | 176 | ### Reply 177 | 178 | Thx for the feedback. I’m not sure if I can change, or if I want to 🙃 The book would probably expand by a factor of two or three, with all of the attendant difficulties, especially with precision, in a hostile environment. The same sort of thing happens with highly polymorphic programs. They are shorter but harder to understand without a change of viewpoint. Similarly I could have made pattern-matching on trees into a primitive operation. 179 | 180 | When writing the book, I tried to write for advanced high school students, who have no preconceptions. I plan to use the blog for writing less formal prose. In my dreams, people post their questions for me to answer. Today, I plan to post on intensionality in machine learning 😀 181 | -------------------------------------------------------------------------------- /2024-12-12-calculus-calculi.md: -------------------------------------------------------------------------------- 1 | 2 | # Calculus or Calculi 3 | ## Barry Jay 4 | ### 2024-12-12 5 | 6 | According to the [Internet Encyclopedia of Philosophy](https://iep.utm.edu/lambda-calculi) the λ-calculus is in fact a 7 | family of formal systems, a family of calculi that all stem from the 8 | same root. They may support additional term forms or reduction rules, 9 | such as $\eta$-reduction, or give different accounts of substitution, 10 | as implicit or explicit, or be typed or not. In turn, types may be 11 | simple, quantified, dependent, subtyped, static or dynamic, explicit 12 | or implicit or something in between. The shared root is that 13 | computation is about functions, expressed as $\lambda$-abstractions. 14 | 15 | The same variability can be found in combinatory logic. There are five 16 | or three or two basic operators, which can be combined by application 17 | to produce functions without any need for binding. Indeed, it seems 18 | that any new model of computation is destined to spawn variants that 19 | stem from a common root. For example, pattern calculus is rooted in 20 | pattern-matching but there is more than one way to handle mismatches. 21 | 22 | Most recently, I developed tree calculus on the principal that 23 | computations are unlabelled trees, and the programs and values are the 24 | binary trees, but there is some freedom when it comes to choosing the 25 | reduction rules. They should maximise expressive power, be compact and 26 | be reasonable. 27 | 28 | In my book [Reflective Programs in Tree 29 | Calculus](https://github.com/barry-jay-personal/tree-calculus/blob/master/tree_book.pdf) 30 | I emphasised expressive power and compactness but reasoning is 31 | hard. That is, there are only three reduction rules, according to 32 | whether the first argument is a leaf, stem or fork, but the second 33 | rule does double duty, to duplicate arguments like the operator $S$ of 34 | combinatory logic, and to support program analysis. 35 | From this can be built *triage* for case analysis, using the rules 36 | - T {f0, f1, f2} $\Delta \longrightarrow$ f0 37 | - T {f0, f1, f2} $(\Delta x)\longrightarrow$ f1 $x$ 38 | - T {f0, f1, f2} $(\Delta x y)\longrightarrow$ f2 $x$ $y$ 39 | 40 | 41 | Well, everything worked just fine until it came to typing. Each triage 42 | is a large term, and the reductions above take many steps, each of 43 | which must be well typed. I slogged my way through that, but then the 44 | self-interpreters use triply-nested triage, which was overwhelming. 45 | 46 | After explaining all this to Johannes Bader in one of our Zoom 47 | meetings, he suggested that the triage reductions above should be 48 | rules, where triage is introduced by the following mnemonic: 49 | 50 | - $\Delta \Delta = K$ 51 | - $\Delta (\Delta x) = S x$ 52 | - $\Delta (\Delta w x) y = T$ { $w,x,y$ } 53 | 54 | This is more reasonable, but less compact than the original. There are 55 | five reduction rules instead of three, but each rule does a single 56 | job, and the typing is more straightfoward. I used this *triage 57 | calculus* in my paper [Typed Program Analysis without 58 | Encodings](https://github.com/barry-jay-personal/typed_tree_calculus/blob/main/typed_program_analysis.pdf) 59 | and Johannes used it in his [exploration of tree 60 | calculus](https://treecalcul.us/). 61 | 62 | So now we have the *original tree calculus* of the book and *triage 63 | calculus*. Who knows what other tree calculi will emerge in the 64 | future? 65 | -------------------------------------------------------------------------------- /2024-12-13-fame.md: -------------------------------------------------------------------------------- 1 | # 15 Minutes of Fame 2 | ## Barry jay 3 | ### 2024-12-13 4 | 5 | You are number one on 6 | [HackerNews](https://hn.algolia.com/?q=tree+calculus)! This message 7 | arrived as I was getting ready for bed but of course I had to follow the action. 8 | The comments were streaming through. Some friends were clarifying, 9 | referencing etc. I didn't say much. Views of [my website](https://github.com/barry-jay-personal) went from zero to three thousand 10 | overnight! And this for a book that went online four years ago! 11 | Perhaps the ice has finally broken! 12 | 13 | Well, steady on, Barry. Views are back down to a couple of hundred; 14 | you are already yesterday's news. However that plays out, I'd like to 15 | respond to all those comments here. Presumably, those readers that already 16 | care will find it but you can repost to HackerNews or elsewhere if you 17 | think it appropriate. 18 | 19 | I noticed four kinds of reaction. 20 | - Wow, cool! 21 | - I can't parse this: how does the BNF of the terms relate to those unlabelled trees? 22 | - There's nothing new here: it is a glorified 23 | $\lambda$-calculus/combinatory logic/S-expressions/syntax trees/Lisp. 24 | - This is so wrong! Is he a crackpot? 25 | 26 | Cool kids, I'll write a separate post, just for you. Here I want to address the criticisms. 27 | 28 | ### Syntax versus Semantics 29 | 30 | The BNF says that every term, every combination is either a leaf 31 | $\Delta$ or an application $MN$ of combinations. The semantics of 32 | $\Delta$ is a tree that is a mere leaf. The semantics of $MN$ 33 | combines the tree $T_1$ for $M$ with the tree $T_2$ for $N$ by 34 | attaching $T_2$ as the right-most branch of the root of $T_1$ to get a 35 | tree $T$. For example, if $T_1$ is a binary tree then $T$ is a ternary 36 | tree. There are lots of examples in the 37 | [book](https://github.com/barry-jay-personal/tree-calculus/blob/master/tree_book.pdf). 38 | 39 | ### Trivial or Wrong? 40 | 41 | Until now, tree calculus has never survived peer-review. Of three 42 | reviews, one would think it cute, one think it trivial and one think 43 | it wrong. Of course, it can't be trivially true and also wrong, so 44 | what is going on? Well, it would be wrong if Church's Thesis were true 45 | in any interesting way. Although it makes sense for numbers, Church's Thesis is 46 | false for general computation. I've written about this with Jose 47 | Vergara in the appendix of my book, and [blogged about 48 | it](https://github.com/barry-jay-personal/blog/blob/main/2024-12-09-poetry-to-prose.md) 49 | the other day. This is why some people think I'm a crackpot. But a theorem trumps a thesis. Nuff said for now. 50 | 51 | The final question is whether it adds expressive power compared to the 52 | systems mentioned above. The short answer is yes, since, for example, 53 | tree calculus can decide equality of programs while these other 54 | systems cannot. In my experience, people accept this point about 55 | $\lambda$-calculus and combinatory logic but struggle with syntax trees and Lisp. 56 | 57 | ### Syntax Trees 58 | Since unlabelled trees carry less information than 59 | syntax trees, haven't we given up some expressive power? Where's the payoff? 60 | 61 | There are several payoffs. First, syntax trees are artificial. They 62 | can be used to model a particular programming language but not 63 | computation in general. By contrast, the unlabelled trees are natural, 64 | like the numbers, and general enough to model any language. Syntax 65 | trees are to special purpose Turing machines as natural trees are to a 66 | universal Turing machine. 67 | 68 | Second, where evaluation rules are applied to syntax trees, full 69 | expressive power requires that the rules be [intensional]( 70 | https://www.sciencedirect.com/science/article/pii/S0304397519301227) 71 | in that the internal structure of arguments is sometimes examined but 72 | usually the rules are *extensional* just like those of 73 | $\lambda$-calculus and combinatory logic. Has anyone considered how to 74 | add intensionality to something more practical than combinatory logic? 75 | 76 | ### Lisp 77 | 78 | For this discussion, the key point about Lisp is the existence of 79 | quote and eval: quote embeds raw syntax into the calculus, delaying 80 | its conversion to an executable by eval until the environment has been 81 | determined. If the executables are thought of as Turing machines, then quotation 82 | imports the tape of some other machine. Of course, equality of tapes 83 | is a triviality but equality of machines makes no sense. This is still 84 | true after enriching by quotation. From the viewpoint of tree calculus, quote was a first step 85 | towards intensional programming. 86 | 87 | Have I overlooked anything? 88 | As always, send comments to Barry.Jay8@gmail.com and I will upload them. 89 | -------------------------------------------------------------------------------- /2024-12-14-logic.md: -------------------------------------------------------------------------------- 1 | # Logic Engineers Truth 2 | ## Barry Jay 3 | ### 2024-12-14 4 | 5 | Logic is engineering, not science. The same holds for category theory, 6 | and any type system that corresponds to a logic through the 7 | Curry-Howard Isomorphism. 8 | 9 | These are bold statements, to be sure. The usual definitions assert 10 | that engineers make useful stuff, while scientists discover things 11 | about nature. This is a pretty good description of the people 12 | involved, but doesn't really apply to theories. For me, engineering is 13 | *synthetic* while science is *analytic*. That is, engineering combines 14 | components to produce useful things. The utility is inferred from the 15 | known (or assumed) properties of the components. By contrast, science 16 | divides compounds into atoms, and atoms into particles, to determine 17 | their nature as precisely as possible. 18 | 19 | Now logic is the study of valid arguments, deciding whether a 20 | conclusion follows from some premises. The premises are the 21 | components, which are combined to form the conclusion. Logic doesn't 22 | care if the conclusion is useful or not, but it is synthetic, not 23 | analytic, which is the key point. It follows that any type system that 24 | corresponds to a logic is also engineering, not science. This is true 25 | of most type systems, especially when they are intended to provide 26 | interfaces for programs as components. 27 | 28 | 29 | The argument is even clearer for category theory, which can be thought 30 | of as the algebra of composition. Dont' ask about the nature of the 31 | arrows, as they are the premises. The remarkable thing about category 32 | theory is the claim that one form of combination, namely composition, 33 | is enough for all purposes. 34 | 35 | 36 | Actually, the argument extends to combinatory logic, too. One starts 37 | with a small collection of primitives, say $S,K$ and $I$, and combine 38 | them in exactly one way (application) to produce useful 39 | programs. Although we know a lot about these primitives, we don't know 40 | everything. For example, $I$ may be an atom (an operator) or a 41 | compound (e.g. $SKK$) but this analytic question doesn't matter for 42 | engineering purposes. 43 | 44 | To the extent that $\lambda$-calculus is equivalent to combinatory 45 | logic, it too is an engineering discipline. More precisly, although we 46 | can see all the internal structure of a 47 | $\lambda$-abstraction, it cannot be revealed within the calculus, 48 | except through application. 49 | 50 | It follows that logic, category theory, combinatory logic and 51 | $\lambda$-calculus may be good foundations for *computer engineering*, 52 | they cannot provide a foundation for *computer science*. To take but 53 | one example, complexity theory estimates the run-time of a program as 54 | a function of the size of its inputs, as determined by analysis. 55 | 56 | We can expand our vocabulary by aligning synthesis with extensionality 57 | and analysis with intensionality. Logic, category theory, combinatory 58 | logic and $\lambda$-calculus are all extensional, in that they focus 59 | on combination, not factorisation. By contrast, [pattern calculus](https://link.springer.com/book/10.1007/978-3-540-89185-7), 60 | [SF-calculus](https://www.cambridge.org/core/journals/journal-of-symbolic-logic/listing?q=a+combinatory+account+of+internal+structure&productId=56ED3BA46977662562A26BC04DD604FD&context=%2Fjournals%2Fjournal-of-symbolic-logic&type=journal&fts=yes) and [tree calculus](https://github.com/barry-jay-personal/tree-calculus) are also intensional. Pattern calculus 61 | can analyse data but not functions. $SF$-calculus can analyse 62 | arbitrary programs, but the choice of $S$ and $F$ is artificial. Tree 63 | calculus is more natural, and is complete for program analysis in the 64 | way that the extensional theories are complete for numerical analysis. 65 | 66 | All of the theories above are important: after all, engineering is 67 | important. All of them were developed to address specific 68 | questions. Predicate logic showed that existence is a property of 69 | predicates, not of things (the truth of "There is no golden mountain" 70 | does not require a witness). $\lambda$-calculus describes functions as 71 | finite rules, without requiring any infinite sets of input-output 72 | pairs. Category theory shows how much of mathematics can be engineered 73 | without analysis of fundamentals. 74 | 75 | That said, we are now in a position to work on the science, too. We 76 | have seen that tree calculus provides a scientific foundation for 77 | programming. For example, the size of a program is itself a 78 | program. How far can this analysis extend to the hardware of 79 | computing? Could it also provide a foundation for mathematics? Instead 80 | of sets, or the objects and arrows of a category, why not adopt binary 81 | natural trees as the underlying structure of mathematics? These trees 82 | are both data and functions; they are finite; and support 83 | computation. What more is required? How far can we get? Time will 84 | tell. 85 | -------------------------------------------------------------------------------- /2024-12-20-infinity.md: -------------------------------------------------------------------------------- 1 | # Taming Infinity 2 | ## Barry Jay 3 | ### 2024-12-20 4 | 5 | The usual foundations of mathematics are beset with infinities, 6 | despite the best efforts of the constructivists. In set theory, for 7 | example, the addition of natural numbers is an infinite set of ordered 8 | pairs, while a real number is an infinite sequence. One approach to 9 | taming these infinities is to work with rules or processes instead of 10 | sets. Then rule for addition can be described by a 11 | $\lambda$-abstraction while a real number is a process for producing its 12 | digits. However, the infinities re-appear in the 13 | semantics of $\lambda$-calculus. For example, the meaning of a 14 | $\lambda$-abstraction in domain theory is the limit of an infinite 15 | sequence of approximations. Again, categorical semantics looks 16 | promising but any grounding of it in domain theory or set theory 17 | re-introduces the infinities. 18 | 19 | Tree calculus seems to avoid these problems since functions and 20 | processes are represented by natural binary trees, which are finite. 21 | Of course, processes may not terminate, so computation may go on 22 | forever, but at each moment the computation is a finite tree, which is 23 | all that we can ask for. Programs are finite, even if computation time 24 | is unbounded. 25 | 26 | My book [Reflective Programs in Tree 27 | Calculus](https://github.com/barry-jay-personal/tree-calculus/tree_book.pdf) 28 | shows how to implement arithmetic but does not go on to consider real numbers, etc. 29 | so here is very first attempt, hot off the whiteboard. (Yes! I have a whiteoard at home, hanging on the door of my study. Usually the door is open, with the board facing my desk, but if the door is closed, then the board serves as fair warning.) 30 | 31 | Every program $r$ can be viewed as a real number between zero and one, expressed in binary digits, as follows. 32 | - If the application of $r$ to zero is not a boolean then the real number is zero. 33 | - If the application of $r$ to zero is a boolean then that is the first digit, with the rest produced by applying $r$ to one, etc. 34 | - If the application of $r$ to zero does not have a value then neither does $r$. 35 | 36 | Thus, if applications of $r$ take boolean values on arguments from 37 | zero to some $n$ but does not have a value on the successor of $n$ 38 | then $r$ is partially-defined, with only its first $n+1$ digits known. 39 | It should be possible to develop the usual arithmetic operations on 40 | the reals. I can also imagine defining the algebraic numbers as 41 | solutions to equations. Even transcendental numbers such as $\pi$ and 42 | $e$ should be doable, as their digits are defined by rules. What else? Let us know if you try this out! 43 | -------------------------------------------------------------------------------- /2024-12-23-tweets-on-trees.md: -------------------------------------------------------------------------------- 1 | # Tweets on Trees 2 | ## Barry Jay 3 | ### 2024-12-23 4 | 5 | Here are some tweets about tree calculus that I wrote in September, 2020. My goals was to summarise each chapter of the book in one tweet but I ended up with almost double that number. The text was illustrated with some cartoons and diagrams of trees from the book, but they aren't replicated here. 6 | 7 | I've just finished drafting my new book "Reflective Programs in Tree Calculus". 8 | 9 | Good computer science needs theorems (to show that bad things don’t happen) an implementation (to show that good things do happen) and a story (that ties it all together). 10 | 11 | At heart, computer science is the study of programs. An implementation is an interpreter or compiler, i.e a program that acts on programs. In a good story, all these programs have equal status, so interpreters can be self-applied. This requires a theory of reflective programs. 12 | 13 | Turing machines do reflection by encoding machines as numbers on a tape. But this computation is outside of the model, so Turing machines don't make a good theory of computation! Nor do natural numbers. 14 | 15 | Lambda calculi do most reflection by encoding terms as syntax trees. But this computation is outside of the model, so lambda calculus doesn't make a good theory of computation! Nor does combinatory logic. 16 | 17 | So reflective programming is hard, but no harder than operating on your own brain. 18 | 19 | Syntax trees support reflection but are artificial, so use natural trees, i.e. without labels, as in the frontispiece shown before. 20 | 21 | Let parsing convert syntax to natural trees. Meet Fir (a tree made of blank tape). 22 | 23 | In tree calculus, the programs, functions and data structures are exactly the binary trees, which are leaves, stems or forks. Other trees are computations. All these can be written as combinations of a single operator called Node, drawn as a small triangle. 24 | 25 | The rules (K) and (S) delete or copy the third argument, so tree calculus: supports all combinators; has macros for lambda-abstraction; and is Turing-complete. The rule (F) adds factorisation, as in SF-calculus. 26 | 27 | When recursion is expressed by fixpoints, the functions are traditionally not stable, without normal form, but in tree calculus they are. Any instability is caused by function application. Meet Shasta, the flower of recursion. 28 | 29 | Tree calculus also supports program analysis, such as a test for program equality. 30 | 31 | Such program analysis is not possible in traditional models, as Fir understands. 32 | 33 | A tree calculus program f can be tagged with a comment, type or other guidance t that can be recovered by applying a query to it, but behaves like f when it is applied: 34 | $tag (t,f) =\Delta(\Delta t)(\Delta(\Delta f)(\Delta\Delta(\Delta\Delta)))$. 35 | Tags could support just-in-time program analysis. 36 | 37 | Self-interpretation for branch-first (eager) evaluation and root-first (lazy) evaluation uses trees that are very small (<1000 nodes). Maybe self-awareness is not so complicated :) 38 | 39 | Combinatory logic is incomplete since equality is not definable. Also, it has no *meaningful translation* of tree calculus. 40 | 41 | VA-calculus supports variables and also lambda-abstractions that have environments, much as closures do. This replaces all the meta-theory, for substitution, variable renaming etc, with 7 simple equations. 42 | 43 | Although VA-calculus is incomplete (equality is not definable) there is a *meaningful translation* to it from tree calculus. Here is the translation of Node to a combination of V and A, given as a labelled tree. 44 | 45 | There is also a meaningful translation from VA-calculus to tree calculus. Here is the tree for A. 46 | 47 | SF-calculus supports divide-and-conquer algorithms, such as equality or pattern-matching, using combinations built from operators S and F. 48 | 49 | There are meaningful translations between SF-calculus and tree calculus. 50 | -------------------------------------------------------------------------------- /2024-12-27-linear-algebra.md: -------------------------------------------------------------------------------- 1 | # Machine Learning is Intensional 2 | ## Barry Jay 3 | ### 2024-12-25 4 | 5 | I want to get something off my chest. For a long time, I have been 6 | stretching our models of computation to support program analysis, so 7 | that we can write programs that reveal all the internal structure of 8 | other programs (or themselves!) or, if you prefer, the intensions of 9 | the programmer. This intensional approach goes beyond the expressive 10 | power of merely extensional models of computation, such as lambda 11 | calculus or combinatory logic, by supporting pattern-matching on program structure through triage. Okay, so triage 12 | is a bit new, a bit weird, and it takes more than a glance to appreciate. 13 | How can we relate it to more familiar things? 14 | 15 | Till now, I've taken a two-pronged approach. One is to develop generic 16 | queries, to provide a natural foundation for the familar database 17 | queries like SELECT. This is clear enough but the focus is on data analysis, not 18 | program analysis, which has already been done both in practice and in 19 | theory by [pattern calculus](). 20 | 21 | The other is to produce toy examples of program analyses, such as 22 | computing the size of a program, or deciding program 23 | equality, but these theoretical advances do not impress in 24 | practice. 25 | 26 | However, there is another domain in which program analysis interleaves with program execution, namely the linear algebra 27 | underpinning machine learning. Let me say straight up that this is not 28 | my field, so there will be no discussion of practicalities. Indeed, 29 | there are no theorems to share either. It's just an 30 | observation. 31 | 32 | A matrix is both a function and a data structure. It is both a 33 | function of vectors that answers queries and a two-dimensional array to be updated 34 | during learning. To freely interleave these activities requires an 35 | intensional programming language. I am curious to see how easily tree 36 | calculus can support this. Peraps the main challenge is that trees 37 | use one-and-a-half dimensions, between the 38 | one-dimensional numbers and the two-dimensional matrices. How might 39 | this affect performance? 40 | -------------------------------------------------------------------------------- /2024-12-31-watchword.md: -------------------------------------------------------------------------------- 1 | # My Word for 2025 2 | ## Barry Jay 3 | ### 2024-12-22 4 | 5 | It's time to choose my word of the year, my watchword. Something to focus on when I 6 | hesitate, to remind me about my goals. A word is better than a 7 | resolution. Not only is it shorter, but it's more robust. You make a 8 | resolution in January, break it in February and now you're busted for 9 | the year. You have let down your resolution. What a failure! But a 10 | watchword is an aspiration, a friendly reminder, there when you need it, 11 | without ever failing. 12 | 13 | In 2021, my watchword was 14 | 15 | ## Share 16 | 17 | When I finished writing [Reflective Programs in Tree Calculus](https://github.com/barry-jay-personal/tree-calculus/blob/master/tree_book.pdf) 18 | my publisher was unable to find an expert willing to write a 19 | review, and didn't know what to do. So I pulled out and 20 | self-published. But who would know it existed? So my word for the year was **share**. For 21 | example, I summarised each chapter in a 22 | [tweet](https://github.com/barry-jay-personal/blog/blob/main/2024-12-23-tweets-on-trees.md). 23 | 24 | 25 | 26 | In 2024, my watchword was 27 | 28 | ## Finish 29 | 30 | Ever since the book went online, I 31 | have been working on type systems for tree calculus, with the goal of typing a self-interpreter. 32 | I accumulated 33 | partial results by developing tree types, making them subtypes of 34 | function types, and typing generic queries. In mid 2023, I showed how to 35 | type a self-interpreter, but only in the presence of a Top type, which 36 | seemed like cheating. I fully expected to eliminate it by the end of 37 | that year but was still stuck. So my focus for 2024 was to **finish** 38 | something. If necessary, write up the ugly solution with Top 39 | types. Happily, this was avoided by introducing the *as-function* 40 | types, which activate when their argument becomes a function type. The 41 | same day as the verfication in Coq was completed, I saw the call for 42 | papers for [PEPM](https://popl25.sigplan.org/home/pepm-2025), wrote 43 | the [paper](https://github.com/barry-jay-personal/typed_tree_calculus/blob/main/typed_program_analysis.pdf) 44 | in ten days for presentation in January. Finished! 45 | 46 | This word helped keep me motivated through the year, but its true 47 | value would have emerged if the better proof had not emerged. It would 48 | have urged me to confront the issue, and made it easier to pursue the 49 | weaker result. 50 | 51 | Now it's time to choose a word for 2025. I still have a lifetime of 52 | research goals (implementation of heaps as trees, generic algebraic 53 | data types, complexity theory, foundations of mathematics, etc) but 54 | need to pace myself a little. Working for three years without anything 55 | to show for it was a little stressful. I kept reminding myself that 56 | there is no Plan B. That done, I will keep working, but also need 57 | to recharge the batteries. So my watchword for 2025 is 58 | 59 | ## Fun 60 | 61 | What will your watchword be? 62 | 63 | ### Comment James Eversole 64 | 65 | Hi Barry, 66 | 67 | I wanted to thank you directly for sharing your discoveries related to the Tree Calculi. I came across your work as a result of Johannes Bader's website and have spent the last 3 weeks captivated by the possibilities of intensionality in practice. It led to me writing a small language in Haskell for exploring Tree Calculus very similar to the one Johannes implemented on his site's playground. While I'm not treading any new ground there, this has been an immensely fun and satisfying project that I can't seem to stop thinking about. 68 | 69 | The sharing of your papers, Coq proofs, and blog posts are all deeply appreciated. I don't have a formal background in computer science or mathematics, so some of the insights in your blog posts were instrumental in me gaining an intuition for the Tree Calculus. Similarly helpful was being able to reference the Coq proofs when reading through Reflective Programs in Tree Calculus and Typed Program Analysis without Encodings. Again, thank you sincerely for making these resources easily and publicly available. 70 | 71 | I think your 2025 watchword "Fun" perfectly describes my feelings since discovering Tree Calculus a few weeks ago. It seems that my watchword will be "Learn" as I continue working towards a deeper understanding of both your research and the implications it has for practical programming. 72 | 73 | Have a happy new year! 74 | 75 | ### Reply 76 | 77 | Thank you so much for the kind words. 78 | I'm particularly pleased that you have been able to get to grips with this, despite lacking any formal background, as I wrote the book with you as my ideal reader. 79 | -------------------------------------------------------------------------------- /2025-01-02-semantics.md: -------------------------------------------------------------------------------- 1 | # Identify Syntax and Semantics! 2 | ## Barry Jay 3 | ### 2025-01-02 4 | 5 | The semantics of tree calculus is its syntax! 6 | 7 | At a glance, this seems to be a nonsense. If language is to describe 8 | the world then we must understand the relation between a word and its 9 | meaning, between its syntax and its semantics. For programming 10 | languages, the world is mathematical. Traditionally, programs 11 | describe functions as understood by their input-out relation. Two 12 | programs are equivalent if they have the same semantics. In this 13 | manner, an understanding of semantics can free us from the accidents 14 | of syntax. As well as programs, we can compare programming languages, 15 | by using translations of syntax that preserve semantics. 16 | 17 | However, this separation of syntax and semantics is a form of dualism, 18 | a separation of mind and matter. It breaks down when programs are used 19 | to analyse other programs, when programs are the things being 20 | described or computed, as in tree calculus. This requires a form of 21 | materialism. 22 | 23 | So the natural binary trees (those without labels) are both the syntax 24 | and the semantics of tree calculus! Certainly the natural trees are 25 | mathematical objects, no matter what foundation your mathematics 26 | adopts. Their identification with syntax is a bit of a stretch, since 27 | we must choose a symbol to represent a node, we still have brackets and 28 | we still read from left to right. The key point, however, is that 29 | equivalence of programs is decidable, indeed, trivial. No matter the 30 | syntactic details, two programs have the same semantics if and only if 31 | they are equal. By contrast, in all traditional models of 32 | computation, equivalence of programs is undecidable, because of the 33 | Halting Problem. 34 | 35 | These thoughts have rescued me from the temptation to build a 36 | categorical model of tree calculus. For the untyped calculus, the 37 | objects would be the natural numbers (representing tuples of binary 38 | trees), and the arrows from m to n would be n-tuples of binary trees. 39 | The resulting category should be cartesian-closed, in the traditional 40 | way. However, what would be the point of this dualism? The binary 41 | trees are easily understood, without adding a layer of formalism on 42 | top. Well, let me back up a little. There may well be some 43 | interesting theorem about tree calculus that is best expressed in the 44 | language of category theory, but until I know what it is, I won't have 45 | a target to aim for, in the way that self-evaluation has been a target 46 | until now. 47 | 48 | Having touched on dualism, let me share a thought about Richard 49 | Dawkin's selfish gene. The titular claim of his book is that evolution 50 | is driven by copying of genes, not organisms. He also says that a gene 51 | is a recipe not a blueprint. In more detail, DNA is a recipe for 52 | building another organism, not a blueprint that outlines the organism. 53 | However, 54 | 55 | **a genome is both recipe and a blueprint**. 56 | 57 | It is a recipe 58 | for making copies of itself. Perhaps this serves as a definition of 59 | life itself. Something is alive if, in the right environment, it is 60 | both a recipe and a blueprint. In this sense, Conway's Game of Life 61 | supports living things. Perhaps tree calculus does, too. 62 | -------------------------------------------------------------------------------- /2025-01-05-type-inference.md: -------------------------------------------------------------------------------- 1 | # Hack the Types not the Terms 2 | ## Barry Jay 3 | ### 2025-01-05 4 | 5 | Now that we can type a self-evaluator by $\forall X.\forall Y. (X \to Y) \to (X \to Y)$ 6 | let's consider *type inference*. That is, given a term $t$ in some type context $\Gamma$ find a type $T$ such that $\Gamma \vdash t : T$. 7 | What follows are my latest 8 | thoughts on the problem, not a solution. I'm a bit reluctant to post 9 | ideas that are half-baked, if not wrong, but that approach led to 10 | years of radio silence so here 'tis. 11 | 12 | Traditionally, type inference has 13 | required us to hack the term language in some way, e.g. by adding 14 | type decorations or let-terms, but my goal is to leave the term 15 | language unchanged, and merely hack the type system. To appreciate 16 | this, let us consider the self-application of the identity program $I$ 17 | in four variants of $\lambda$-calculus. 18 | 19 | In pure $\lambda$-calculus, $I$ is given by $\lambda x .x$. 20 | 21 | In simply-typed $\lambda$-calculus, each 22 | term has an explicit type, i.e. the type is part of the structure of 23 | the term. Thus, there is an identity function for each type $U$, 24 | namely 25 | $$I^U = (\lambda x^U. x^U)^{U\to U} .$$ 26 | The self-application of $I$ becomes $I^{U\to V}\ I^U$. 27 | 28 | In System F, the types $T$ also form a sort of $\lambda$-calculus 29 | 30 | $T ::= X \ | \ T\to T \ | \ \forall X. T$ 31 | 32 | where $\forall X. T$ binds the type variable $X$ in $T$. Now there is 33 | a parametrically-polymorphic identity function 34 | 35 | $I = \Lambda X. \lambda x^X. x^X$ 36 | 37 | but self-application requires that I be applied to its own type in 38 | 39 | $I\ (\forall X. X \to X)\ I$ 40 | 41 | which hacks the term language in new ways. Further, type inference for 42 | this must discover the type $\forall X. X \to X$ used in the type 43 | application. Unfortunately, as is well known, even type-checking for 44 | System F is undecidable, so that type inference is hopeless. 45 | 46 | Between simply-typed $\lambda$-calculus and System F lies the 47 | Hindley-Milner type system with its let-polymorphism. It limits type quantification by separating the *monotypes* $T$ which are like the simple types, from the *type schemes* which apply quantifiers to the monotypes, so that all quantifiers are on the outside. Now the 48 | self-application of $I$ becomes 49 | 50 | $let\ f = \lambda x. x\ in\ f\ f .$ 51 | 52 | This could be de-sugared to the application $(\lambda 53 | f. f\ f)(\lambda x.x)$ but then it is hard to determine the type of 54 | $f$ in $\lambda f. f f$. By providing its value $\lambda x. x$ first, 55 | we can infer the type of $f$ to be the principal type of $\lambda x.x$ 56 | namely $\forall X. X \to X$ and all is well. 57 | 58 | Still, each variant of $\lambda$-calculus requires its own hacks to 59 | the term-language, and these grow more elaborate as the subtlety of 60 | typing increases. 61 | 62 | The challenges are a little different in tree calculus. The 63 | application $(\lambda f. f\ f)(\lambda x.x)$ corresponds to 64 | $(SII)I$. Now $I$ has principal type 65 | 66 | $Id = Fork (Stem (Stem\ Leaf))(Stem\ Leaf))$ 67 | 68 | and the value of $SII$ has a principal type $Fork (Stem\ Id) Id$. So far, there is no need for quantifiers. Rather, the 69 | challenge is to handle the supertyping of the tree type to a function 70 | type. That is, find a type $V$ such that $$Fork (Stem Id) Id < 71 | Id \to V$$. More generally, for given types $T$ and $U$, find the most 72 | general $V$ such that $T