├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── coq_blog.rb ├── index.html.erb ├── posts ├── 2013-05-02 Cybele.md ├── 2014-11-24 Use OPAM for Coq.md ├── 2014-11-25 Make a Coq package.md ├── 2014-12-03 Why and how to write code compatible with many Coq versions.md ├── 2014-12-04 Pluto: a first concurrent web server in Gallina.md ├── 2014-12-18 Checking concurrent programs with symbolic simulations.md ├── 2015-01-16 A blog engine written and proven in Coq.md ├── 2015-02-17 A bench system for the Coq packages.md ├── 2015-02-20 Tutorial: a Hello World in Coq.md ├── 2015-03-04 Write a script in Coq.md ├── 2015-03-05 Formally verify a script in Coq.md ├── 2015-03-14 Concurrency with promises in Coq.md ├── 2015-03-16 Implementation of promises for Coq.md ├── 2015-07-21 Launch of the Coq.io website.md ├── 2015-07-23 Handle errors in Coq.md ├── 2015-07-25 Simple unit testing in Coq.md ├── 2018-07-24 Approximating GADTs in Flow.md ├── 2019-07-25 Continuous testing for Coq projects.md ├── 2019-09-03 Connecting the opam bench to Gitter.md ├── 2019-09-04 Fixing flaky makefiles in opam Coq packages.md ├── 2019-09-13 Multiple error messages in coq-of-ocaml.md ├── 2019-09-28 Importing mutually recursive types from OCaml to Coq.md ├── 2019-11-04 First-class modules in coq-of-ocaml.md ├── 2020-01-15 Formalization of the Tezos protocol's interface in Coq.md ├── 2020-02-17 Latest updates of coq-of-ocaml for the Tezos protocol.md ├── 2020-06-19 GADTs with type erasure in coq-of-ocaml.md ├── 2020-10-20 Improvements of coq-of-ocaml for functors and signatures.md ├── 2021-02-08 Removing existential types from modules in coq-of-ocaml.md └── 2021-02-22 Beginning of verification for the parsing of smart-contracts.md ├── rss.xml.erb ├── static ├── artifacts │ └── tezos-interface-in-coq │ │ ├── coqdoc.css │ │ └── v1_mli.html ├── favicon.png └── images │ ├── coq-of-ocaml-multiple-errors │ └── report.png │ ├── cybele_comparison.svg │ ├── cybele_compilation.svg │ ├── opam-bench-gitter │ └── report.png │ ├── pluto_runtime.svg │ └── travis-ci │ ├── build-report.png │ ├── error-logs.png │ └── pull-request.png ├── templates ├── footer.html.erb ├── header.html.erb └── post.html.erb └── wip.html.erb /.gitignore: -------------------------------------------------------------------------------- 1 | static/style.min.css 2 | blog/ 3 | 4 | *.gem 5 | *.rbc 6 | /.config 7 | /coverage/ 8 | /InstalledFiles 9 | /pkg/ 10 | /spec/reports/ 11 | /test/tmp/ 12 | /test/version_tmp/ 13 | /tmp/ 14 | 15 | ## Specific to RubyMotion: 16 | .dat* 17 | .repl_history 18 | build/ 19 | 20 | ## Documentation cache and generated files: 21 | /.yardoc/ 22 | /_yardoc/ 23 | /doc/ 24 | /rdoc/ 25 | 26 | ## Environment normalisation: 27 | /.bundle/ 28 | /lib/bundler/man/ 29 | 30 | # for a library or gem, you might want to ignore these files since the code is 31 | # intended to run in multiple environments; otherwise, check them in: 32 | # Gemfile.lock 33 | # .ruby-version 34 | # .ruby-gemset 35 | 36 | # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: 37 | .rvmrc 38 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Guillaume Claret 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: clean 2 | mkdir blog 3 | ln -rs static blog/static 4 | ruby coq_blog.rb 5 | 6 | watch: 7 | while inotifywait posts/*; do make; done 8 | 9 | clean: 10 | rm -Rf blog/ 11 | 12 | serve: 13 | @echo Starting on http://localhost:8000/ 14 | ruby -run -e httpd blog/ -p 8000 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Coq Blog 2 | A blog about Coq. Hosted on [coq-blog.clarus.me](http://coq-blog.clarus.me/). 3 | 4 | ## Use or fork to make your own blog 5 | Install the Markdown parser (you first need Ruby): 6 | 7 | gem install redcarpet 8 | 9 | Add my coq-ish theme: 10 | 11 | curl -L https://github.com/clarus/coq-red-css/releases/download/coq-blog.1.0.2/style.min.css >static/style.min.css 12 | 13 | If you want other themes, add a [Bootstrap](http://getbootstrap.com/) based CSS in `static/style.min.css`. A nice list is available on [Bootswatch](http://bootswatch.com/). 14 | 15 | Compile: 16 | 17 | make 18 | 19 | Compile each time a post is updated: 20 | 21 | make watch 22 | 23 | Preview the results on [localhost:8000](http://localhost:8000/): 24 | 25 | make serve 26 | 27 | ## WIP posts 28 | WIP posts are posts with no links from the index so that you can share them for review first. In order to mark a post as Work In Progress, add `wip` in its title. It will not appear in the list of posts, but still be generated. To get the list of WIP posts, go to `http://my-url/wip.html`. 29 | 30 | ## License 31 | Published under MIT License. This holds both for the posts and the blog engine. 32 | -------------------------------------------------------------------------------- /coq_blog.rb: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | require 'erb' 3 | require 'redcarpet' 4 | include ERB::Util 5 | 6 | class Blog 7 | attr_reader :title, :url, :disqus, :posts 8 | 9 | def initialize(title, url, disqus) 10 | @title, @url, @disqus = title, url, disqus 11 | @posts = Dir.glob("posts/*.md").map {|file_name| Post.new(file_name)} 12 | .sort_by {|post| post.date}.reverse 13 | end 14 | 15 | def public_posts 16 | @posts.select {|post| not post.wip?} 17 | end 18 | 19 | def wip_posts 20 | @posts.select {|post| post.wip?} 21 | end 22 | end 23 | 24 | class MarkdownRender < Redcarpet::Render::HTML 25 | include Redcarpet::Render::SmartyPants 26 | 27 | def image(link, title, alt_text) 28 | "\"#{h(alt_text)}\"
#{h(title)}
" 29 | end 30 | end 31 | 32 | class Post 33 | attr_reader :name, :date, :html, :url 34 | 35 | def initialize(file_name) 36 | if /\A(\d+)-(\d+)-(\d+)\s*(.*)\z/ === File.basename(file_name, ".md") then 37 | @date = Time.local($1, $2, $3) 38 | @name = $4 39 | else 40 | raise "The name #{file_name.inspect} should have the form \"yyyy-mm-dd title.md\"." 41 | end 42 | markdown = File.read(file_name, encoding: "UTF-8") 43 | @html = Redcarpet::Markdown.new(MarkdownRender).render(markdown) 44 | @url = "#{@name.gsub(/[^a-zA-Z0-9]+/, "-").downcase}.html" 45 | end 46 | 47 | def date_string 48 | @date.strftime("%B %e, %Y") 49 | end 50 | 51 | def wip? 52 | @name.downcase.include?("wip") 53 | end 54 | end 55 | 56 | def render_erb(file_name, binding) 57 | ERB.new(File.read(file_name, encoding: "UTF-8")).result(binding) 58 | end 59 | 60 | def header(blog, title) 61 | render_erb("templates/header.html.erb", binding) 62 | end 63 | 64 | def footer 65 | render_erb("templates/footer.html.erb", binding) 66 | end 67 | 68 | blog = Blog.new("Coq blog - Guillaume Claret", "http://coq-blog.clarus.me/", "coqblog") 69 | 70 | for page in ["index.html", "wip.html", "rss.xml"] do 71 | File.open("blog/#{page}", "w") do |f| 72 | f << render_erb("#{page}.erb", binding) 73 | end 74 | end 75 | 76 | for post in blog.posts do 77 | File.open("blog/#{post.url}", "w") do |f| 78 | f << render_erb("templates/post.html.erb", binding) 79 | end 80 | end 81 | -------------------------------------------------------------------------------- /index.html.erb: -------------------------------------------------------------------------------- 1 | <%= header(blog, h(blog.title)) %> 2 | 3 |
4 |
5 |
6 |

rooster

7 |
8 |
9 |

Welcome

10 |

I am Guillaume Claret, a former PhD student in computer science for the πr² team in Paris. Here I post some articles about things I am doing in Coq.

11 |
12 |
13 |
14 |
15 |
16 |
17 |

Posts <%= blog.public_posts.size %>

18 | 23 |
24 |
25 |
26 | 27 | <%= footer %> 28 | -------------------------------------------------------------------------------- /posts/2013-05-02 Cybele.md: -------------------------------------------------------------------------------- 1 | *This post is a re-post from the now defunct blog of [PPS](http://www.pps.univ-paris-diderot.fr/). The date of this post is the same as the original post.* 2 | 3 | The *proof by extraction* is a new proof technique, combining advantages of the proof by reflection and the proof by untrusted OCaml oracles. The idea is to add effects to the [Coq](https://coq.inria.fr/) language, so mutable variables, non-termination and exceptions are allowed. The execution is made efficient by extraction to [OCaml](http://ocaml.org/) which can pre-compute complex values. The results are then post-checked in Coq thanks to our theory of *simulable monads*. 4 | 5 | We provide [Cybele](http://cybele.gforge.inria.fr/), a Coq plugin to do proof by extraction. You can start looking at the [Get started](http://cybele.gforge.inria.fr/get_started.html) and [Examples](http://cybele.gforge.inria.fr/examples.html) sections. It is based on an original idea of [Yann Régis-Gianas](http://www.pps.univ-paris-diderot.fr/~yrg/), and we are developing it with the help of [Lourdes del Carmen González Huesca](http://www.pps.univ-paris-diderot.fr/~lgonzale/), and [Beta Ziliani](http://www.mpi-sws.org/~beta/). 6 | 7 | ## From proof by reflection to proof by extraction 8 | We consider the example of the equivalence decision between two terms given a list of known equivalences. Let us take a type `T` with `~` an equivalence relation and a list of hypothesis: 9 | 10 | * `H_1 : e_{i_1} ~ e_{j_1}` 11 | * ... 12 | * `H_n : e_{i_n} ~ e_{j_n}` 13 | 14 | Our goal is to prove that `e_i ~ e_j` using the reflexivity, symmetry and transitivity properties. On small instances we can obviously solve it by hand, or write a tactic to do it for us. We just need to apply the equivalence properties as many times as necessary. But on a bigger sample: 15 | 16 | * `H_{1,2} : e_1 ~ e_2` 17 | * ... 18 | * `H_{999,1000} : e_{999} ~ e_{1000}` 19 | 20 | a proof of `e_1 ~ e_{1000}` would consist of 998 applications of the transitivity rule. This is just huge. The solution of the proof by reflection technique is to replace a proof term by a *computation*. This computation has to be done by a proven correct decision procedure. 21 | 22 | The [union-find](http://en.wikipedia.org/wiki/Disjoint-set_data_structure) algorithm is an efficient way to solve the equivalence problem. Let us note `hs` the list of hypothesis indexes `[(i_1, j_1), ..., (i_n, j_n)]` and `(i, j)` the indexes of our goal `e_i ~ e_j`. Then a decision procedure in pseudo-code would look like: 23 | 24 | decide (hs, (i, j)) : bool := 25 | let a = ref [] in 26 | map (fun (i, j) -> union a i j) hs; 27 | let i2 = find a i in 28 | let j2 = find a j in 29 | i2 = j2 30 | 31 | We start with an empty array `a` representing the fact that equivalence classes are *a priori* singleton sets. We merge them calling the `union` procedure on each hypothesis. If the representatives `(i2, j2)` of `(i, j)` are equal then the property `e_i ~ e_j` holds. 32 | 33 | A nice way to show the soundness of `decide` is to add an invariant to the array `a` stating that if `i` and `j` are linked then `e_i ~ e_j`. This can be done elegantly in Coq, which natively supports mixed proofs and programs (see the [Program](http://coq.inria.fr/refman/Reference-Manual028.html) construct). At the end we just return a proof of `e_i ~ e_j` if `i2` equals `j2`. An harder thing is to encode the union-find in a purely functional way since it critically relies on a mutable array, and show the termination of the `union` and `find` procedures. 34 | 35 | We decided not to do it, and instead introduce a [monad](http://en.wikipedia.org/wiki/Monad_(functional_programming)) `M` representing side-effects, non-termination and exceptions. We obtain a solution close to the pseudo-code: 36 | 37 | Definition decide (known_eqs : equalities) (i j : T) : M (i ~ j). 38 | refine ( 39 | let! a := tmp_ref s 0 nil in 40 | do! List.iter (unify a) known_eqs in 41 | let! Pi2 := find a i in 42 | let (i2, Hii2) := Pi2 in 43 | let! Pj2 := find a j in 44 | let (j2, Hjj2) := Pj2 in 45 | if eq_dec i2 j2 then 46 | ret _ 47 | else 48 | error "decide: the terms are not equal"). 49 | ... (* some proof term *) 50 | Defined. 51 | 52 | On line 3, `let!` is a notation for the bind operator. Line 9, we test the equality of `i2` and `j2`. Line 10, `ret _` means we are returning a proof, namely the proof that `e_i ~ e_j`. This proof is delayed until line 13, where it is made in proof mode. It uses the properties `Hii2` and `Hjj2` given by the invariant on `a`, stating that `e_i ~ e_{i_2}` and `e_j ~ e_{j_2}`. 53 | 54 | This is very much in the style of the IO monad of [Haskell](http://www.haskell.org/). Ours provides the following primitives. 55 | 56 | #### Memory 57 | * ref : `forall i, T_i -> M (Ref.t T_i)` (memory allocation) 58 | * read : `Ref.t T -> M T` 59 | * write : `Ref.t T -> T -> M ()` 60 | 61 | #### Non-termination 62 | * fix : `((A -> B) -> A -> B) -> A -> B` 63 | * dependent_fix : `((forall x : A, B) -> forall x : A, B) -> forall x : A, B` 64 | 65 | #### Exceptions 66 | * raise : `string -> M T` 67 | * try_with : `(() -> M T) -> (string -> M T) -> M T` 68 | 69 | #### Printing 70 | * print : `T -> M ()` 71 | 72 | Obviously, this monad cannot be run natively in Coq. There is no run function of type `M A -> A` or `M A -> option A`. This is due to the general fixpoint operators which cannot be encoded in Coq since its typing rules enforce termination. Plus we want to run it as efficiently as possible. 73 | 74 | ## An hybrid back-end 75 | To run our monad we combine the Coq and OCaml back-ends: 76 | 77 | ![Compilation](static/images/cybele_compilation.svg "Compilation chain.") 78 | 79 | We use the [extraction](http://coq.inria.fr/refman/Reference-Manual027.html) mechanism of Coq to get an equivalent program of our decision procedure in OCaml. All the monadic operators are replaced by native OCaml operators, since this language supports all the effects of our monad. We run it on a specific problem instance `x`. It may not return a result in case of uncaught exception or infinite loop. If the execution is successful, we extract what we call a *prophecy*. This value is a guide which helps Coq to evaluate the monad efficiently with an extended run function of type: 80 | 81 | run : prophecy -> M A -> option A 82 | 83 | For the non-termination it contains the numbers of steps needed to evaluate a monadic value. It can also contain some pre-computed values in OCaml to save computational time in Coq. We provide a general mechanism to pass results from OCaml to Coq in the prophecy with the system of *input memory*. When you allocate a reference, you can mark it as an "input". If you do so, its final value after the OCaml run will be given as its initial value to Coq. Any value can be passed thought the prophecy as long as you can serialize it. It does not work for functions or proofs, but does for all common data-structures such as integers or lists. 84 | 85 | Notice that if you do not save results in the prophecy, computations are made twice: once in OCaml, then post-checked in Coq with the run function. This is the main limitation of our system, but it could be solved with a proven correct extraction mechanism. We could directly trust the execution made in OCaml and import the result in Coq. 86 | 87 | ## An hybrid programming model 88 | We pretend that our programming model combines nicely safety and efficiency. Safety because of the strong type system of Coq and the ability to prove invariants on our programs. Efficiency because costly computations are delayed to OCaml. It supersedes what can be done with proof by reflection in pure Coq or with an OCaml plugin generating proof certificates. In a way, [Cybele](http://cybele.gforge.inria.fr/) unifies these two techniques, and allow to combine both approach depending on the problem. 89 | 90 | ![Compilation](static/images/cybele_comparison.svg "Trade-off efficiency vs correctness.") 91 | 92 | As future work we plan to add more monadic operators, especially for concurrency. It would also be great to have a trusted extraction mechanism for Coq. 93 | -------------------------------------------------------------------------------- /posts/2014-11-24 Use OPAM for Coq.md: -------------------------------------------------------------------------------- 1 | [OPAM](http://opam.ocamlpro.com/) is a package manager edited by [OCamlPro](http://www.ocamlpro.com/). We will describe how to use it to handle [Coq](https://coq.inria.fr/) packages. 2 | 3 | ## Install OPAM 4 | Install OPAM by your preferred method. It is recommended to use the latest version (`1.2.2` as of May 7, 2015), which can be installed from the sources: 5 | 6 | curl -L https://github.com/ocaml/opam/releases/download/1.2.2/opam-full-1.2.2.tar.gz |tar -xz 7 | cd opam-full-1.2.2 8 | ./configure && make lib-ext && make 9 | sudo make install 10 | 11 | By default the OPAM packages are installed in `~/.opam`. You can also have many installation folders if you want many versions of Coq or packages. A practice I recommend is to have one installation folder per project or configuration. To configure OPAM in a fresh folder `opam`: 12 | 13 | mkdir opam 14 | opam init --root=opam # answer no to the question 15 | eval `opam config env --root=opam` # run this command for each shell session 16 | 17 | **Remark:** Some people also use the `switch` mechanism to handle many OPAM installations. 18 | 19 | ## Add the repositories 20 | To add the repository for the Coq packages: 21 | 22 | opam repo add coq-released https://coq.inria.fr/opam/released 23 | 24 | There is also a repository for the development versions. Use it at your own risks: 25 | 26 | opam repo add coq-extra-dev https://coq.inria.fr/opam/extra-dev 27 | 28 | To add the development versions of Coq: 29 | 30 | opam repo add coq-core-dev https://coq.inria.fr/opam/core-dev 31 | 32 | ## Install a package 33 | The Coq packages are in the namespace `coq-`. To list all of them: 34 | 35 | opam search coq- 36 | 37 | To install a package: 38 | 39 | opam install coq-io-hello-world 40 | 41 | To specify the version you want to install: 42 | 43 | opam install coq-io-hello-world.1.1.0 44 | 45 | If the package is slow to install (for instance, Coq itself), use the `-j` option to speed it up and `-v` to see the progress: 46 | 47 | opam install -j4 -v coq 48 | -------------------------------------------------------------------------------- /posts/2014-11-25 Make a Coq package.md: -------------------------------------------------------------------------------- 1 | We will show you a typical workflow to create and publish a Coq package with [OPAM](http://opam.ocamlpro.com/). This will allow you to share your Coq developments in a simple way and gain visibility. 2 | 3 | We assume you already know how to use OPAM to install Coq packages. If not, you can read this [tutorial](http://coq-blog.clarus.me/use-opam-for-coq.html). 4 | 5 | ## Create a project 6 | Go on [GitHub](https://github.com/) and make a new project, for example `that-super-proof`. Add a `LICENSE` file with your copyright if you want to make your package open-source (without a license a code is proprietary). I usually choose the [MIT](http://opensource.org/licenses/MIT) license, as one of the most permissive and popular: 7 | 8 | The MIT License (MIT) 9 | 10 | Copyright (c) 11 | 12 | Permission is hereby granted, free of charge, to any person obtaining a copy 13 | of this software and associated documentation files (the "Software"), to deal 14 | in the Software without restriction, including without limitation the rights 15 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 16 | copies of the Software, and to permit persons to whom the Software is 17 | furnished to do so, subject to the following conditions: 18 | 19 | The above copyright notice and this permission notice shall be included in 20 | all copies or substantial portions of the Software. 21 | 22 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 23 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 24 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 25 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 26 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 27 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 28 | THE SOFTWARE. 29 | 30 | Add a main file `All.v`: 31 | 32 | Theorem two : 1 + 1 = 2. 33 | now admit. 34 | Qed. 35 | 36 | You can now commit your work. 37 | 38 | ### The case of OCaml plugins 39 | If your project is a Coq plugin (containing [OCaml](https://ocaml.org/) files), you can get inspiration from the [Constructors](https://github.com/mattam82/Constructors) project of [Matthieu Sozeau](http://www.pps.univ-paris-diderot.fr/~sozeau/). This is an example of a simple OCaml plugin, including the branches `v8.4` and `v8.5` for compatibility with the different versions of Coq. 40 | 41 | ## Compile 42 | We will use `coq_makefile` to generate a Makefile. Create a file `Make`: 43 | 44 | -R . ThatSuperProof 45 | 46 | All.v 47 | 48 | and an executable file `configure.sh`: 49 | 50 | #!/bin/sh 51 | 52 | coq_makefile -f Make -o Makefile 53 | 54 | To compile your project, run: 55 | 56 | ./configure.sh 57 | make 58 | 59 | `coq_makefile` is clever and also generates an `install` rule, among over things. 60 | 61 | ## Publish 62 | To publish a new version you need to make a release. In the GitHub page of your project, go to the *releases* section and add a new release named `1.0.0`. People tend to use the [SemVer](http://semver.org/) convention for the version names, as `MAJOR.MINOR.PATCH`: 63 | 64 | * `MAJOR`: major changes 65 | * `MINOR`: minor changes 66 | * `PATCH`: bug fixes 67 | 68 | We do a pull-request to add our new package (see [pull-requests on GitHub](https://help.github.com/articles/using-pull-requests/) if you need help). Fork the [OPAM Coq repository](https://github.com/coq/opam-coq-archive) and add a folder `coq-that-super-proof/coq-that-super-proof.dev` in `released/packages/`. Package names must start with the `coq-` prefix. 69 | 70 | A package is described by three files: 71 | 72 | * `descr` 73 | 74 | An arithmetic library. 75 | 76 | * `opam` 77 | 78 | opam-version: "1.2" 79 | maintainer: "me@myself.ninja" 80 | homepage: "https://github.com/myself/that-super-proof" 81 | dev-repo: "https://github.com/myself/that-super-proof.git" 82 | bug-reports: "https://github.com/myself/that-super-proof/issues" 83 | authors: ["Me Myself"] 84 | license: "MIT" 85 | build: [ 86 | ["./configure.sh"] 87 | [make "-j%{jobs}%"] 88 | ] 89 | install: [ 90 | [make "install"] 91 | ] 92 | remove: ["rm" "-R" "%{lib}%/coq/user-contrib/ThatSuperProof"] 93 | depends: [ 94 | "coq" {>= "8.4.5" & < "8.5~"} 95 | ] 96 | 97 | The `< "8.5~"` is there to say that the Coq version must be lesser than `8.5.0`, `8.5.1`, ... as well as `8.5~beta1`, `8.5~beta2`, ... 98 | 99 | * `url` 100 | 101 | http: "https://github.com/myself/that-super-proof/archive/1.0.0.tar.gz" 102 | checksum: "da1da74c8f6c560b153ab8dc558cf29e" 103 | 104 | The MD5 checksum is mandatory, and can be obtained with: 105 | 106 | curl -L https://github.com/myself/that-super-proof/archive/1.0.0.tar.gz |md5sum 107 | 108 | You can test your own fork of the OPAM Coq repository using `opam repo add` on the folder `released` of your fork. Then, issue a pull-request with your new package. 109 | 110 | ## Use the bench 111 | There is a bench system available on [coq-bench.github.io](http://coq-bench.github.io/). We test all the packages for each version of Coq. We host this service to help you to check that your packages compile for each platform, even development ones. Compatibility across Coq versions is not necessary but allows you to reach more users. You can specify the Coq versions you depend upon in the `depends` field of your `opam` files. 112 | -------------------------------------------------------------------------------- /posts/2014-12-03 Why and how to write code compatible with many Coq versions.md: -------------------------------------------------------------------------------- 1 | You should always write your code to make it compatible with at least two consecutive Coq versions, if you wish to have larger people adoption. 2 | 3 | The pace of Coq releases is surprisingly slow for a project with that many users, unfortunately due to a lack of organization resources. However, the releases happen and usually introduce a lot of incompatibilities. At this point, users depending on your libraries may be forced to migrate because of other dependencies, or forced not to migrate. Playing nice with them means allowing them to have the choice. 4 | 5 | More important, a lot of people prefer to use the development version of Coq because of the slow release cycle. So your code should at least support the latest stable release and the current development version. 6 | 7 | ## Check your code 8 | [OPAM](http://opam.ocamlpro.com/) is the best way to test your code with different versions of Coq. You can read this [introduction](http://coq-blog.clarus.me/use-opam-for-coq.html) to learn more about how to use OPAM for Coq. 9 | 10 | Let us say we want to test our project `foo` with the Coq versions `8.4.5` and `dev`. We create two installs of Coq in `foo/opam.8.4.5` and `foo/opam.dev`. For the stable version: 11 | 12 | cd foo/ 13 | mkdir opam.8.4.5 # we create the empty directory to prevent a bug of OPAM 14 | opam init --root=opam.8.4.5 15 | eval `opam config --root=opam.8.4.5 env` 16 | opam install --jobs=4 coq.8.4.5 17 | 18 | In an other terminal, for the unstable Coq: 19 | 20 | mkdir opam.dev 21 | opam init --root=opam.dev 22 | eval `opam config --root=opam.dev env` 23 | opam repo add coq-core-dev https://coq.inria.fr/opam/core-dev 24 | opam install -j4 -v coq.dev 25 | 26 | You have now a different version of Coq in each terminal, and can test your code for two versions. If you have an old computer, and may be afraid of having many Coq installations for each project, remember you can always use cache mechanisms like the one provided by [Docker](https://www.docker.com/) to save disk space. 27 | 28 | We also provide a [coq-bench](http://coq-bench.github.io/) website, where OPAM packages are tested for different versions of Coq. This is another simple way to check your code compatibility if you have a package. 29 | 30 | ## Be clean and robust 31 | This is obvious but this must be emphasized: be clean and robust. Most Coq features outside the kernel can be considered as experimental, or do not have a clear semantics. This includes in particular the tactic language [LTac](https://coq.inria.fr/distrib/V8.4pl5/refman/Reference-Manual012.html). 32 | 33 | So try not to rely too much on advanced features, and make your proofs scripts robust (using [explicitly named variables](http://poleiro.info/posts/2013-11-17-automatic-naming-considered-harmful.html) or [bullets](http://poleiro.info/posts/2013-06-27-structuring-proofs-with-bullets.html) for example). 34 | 35 | ## Preprocess 36 | You can use preprocessing to solve breaking incompatibility changes. This helps to keep one code database, instead of splitting your developments with one branch per Coq version. 37 | 38 | Some people use the [CPP preprocessor](http://en.wikipedia.org/wiki/C_preprocessor), but it has a heavy syntax and is quite limited. Instead, I recommend to go for simpler and more powerful tools like [ERB](http://en.wikipedia.org/wiki/ERuby). Here is an example file `Test.v`: 39 | 40 | Definition proj (n : {n : nat & n >= 2}) : nat := 41 | match n with 42 | | existT n _ => n 43 | end. 44 | 45 | This code will not work with `coq.dev` as you need one more argument in the `match`: 46 | 47 | Definition proj (n : {n : nat & n >= 2}) : nat := 48 | match n with 49 | | existT _ n _ => n 50 | end. 51 | 52 | The solution is to make a `Test.v.erb` file, which will be preprocessed into `Test.v`: 53 | 54 | Definition proj (n : {n : nat & n >= 2}) : nat := 55 | match n with 56 | | existT <%= "_" unless version[0..2] == "8.4" %> n _ => n 57 | end. 58 | 59 | We add a `_` in the `match`, unless if the Coq version starts by `8.4`. Other common constructs are: 60 | 61 | <%= "bla" if version[0..2] == "8.4" %> 62 | <%= version[0..2] == "8.4" ? "bla" : "bli" %> 63 | 64 | To preprocess this `Test.v.erb` file, create a `pp.rb` file at the root of your directory: 65 | 66 | # Preprocessor. 67 | require 'erb' 68 | 69 | # The version of Coq. 70 | version = `coqc -v`.match(/version ([^(]*) \(/)[1] 71 | 72 | Dir.glob("*.v.erb") do |file_name| 73 | renderer = ERB.new(File.read(file_name, encoding: "UTF-8")) 74 | output_name = file_name[0..-5] 75 | File.open(output_name, "w") do |file| 76 | file << renderer.result() 77 | end 78 | puts "#{file_name} -> #{output_name}" 79 | end 80 | 81 | The command `ruby pp.rb` will then compile all the files matching `*.v.erb` to `*.v`. 82 | 83 | **Edit from the comments:** 84 | To integrate preprocessing with an IDE, you can use the [ProofGeneral's plugin](https://gist.github.com/cpitclaudel/2c75c8dc88b0e1c9a6e7) of [Clément Pit-Claudel](http://pit-claudel.fr/clement/). For other IDEs, the only solution is to edit the `.v` files in the IDE for one specific version of Coq, and do back-and-forths with the `.v.erb` files to add compatibility. 85 | -------------------------------------------------------------------------------- /posts/2014-12-04 Pluto: a first concurrent web server in Gallina.md: -------------------------------------------------------------------------------- 1 | [Pluto](https://github.com/coq-concurrency/pluto) is, to our knowledge, the first concurrent web server written in Gallina, the functional language of [Coq](https://coq.inria.fr/). 2 | 3 | It is a research project which aims to apply the pure and dependently typed [Coq](https://coq.inria.fr/) language to system programming, with inputs/outputs and fine grained concurrency in mind. This kind of programming is particularly error-prone and hard to test, due to non-determinism and interactions with the external world. Moreover, such programs (like servers and databases) often manipulate critical data. We try to develop new programming techniques with an extremist and purely functional approach, in the hope to lead to safer systems. 4 | 5 | For now [Pluto](https://github.com/coq-concurrency/pluto) can serve static websites, using event-based I/Os and lightweight threads to handle concurrent requests. 6 | 7 | ## Use 8 | The simplest way to install [Pluto](https://github.com/coq-concurrency/pluto) is to use [OPAM](http://opam.ocamlpro.com/) for Coq. See this [tutorial](http://coq-blog.clarus.me/use-opam-for-coq.html) for more informations. Add the stable and unstable repositories: 9 | 10 | opam repo add coq-released https://coq.inria.fr/opam/released 11 | opam repo add coq-extra-dev https://coq.inria.fr/opam/extra-dev 12 | 13 | Install Pluto: 14 | 15 | opam install --jobs=4 coq-concurrency-pluto 16 | 17 | Run it on some `html/` folder: 18 | 19 | pluto.native 8000 html/ 20 | 21 | Your website is now available on [localhost:8000](http://localhost:8000/). 22 | 23 | ## Architecture 24 | Coq is a pure language so it cannot directly express concurrency and I/Os. For that we use a Domain Specific Language (DSL) with new primitive constructs to describe impure computations. The architecture is implemented in the [coq-concurrency/system](https://github.com/coq-concurrency/system) project. 25 | 26 | ### Operators 27 | The `Read` and `Write` commands read or update atomically global references (shared by all the threads). `Ret` lifts a pure Coq expression, `Bind` sequences two computations. The `Send` constructor does an asynchronous call to the OS. It uses a handler with its own private memory (a lightweight thread), called each time an answer is sent to the request. The `Exit` command halts the program and stops all pending handlers. 28 | 29 | We decided to use fully asynchronous I/Os with lightweight threads for two reasons: 30 | 31 | * it is generally considered more efficient than the synchronous system-calls plus system-threads model (see the evolution from the [Apache](http://www.apache.org/) 1 multi-threaded server to mono-threaded event-driven systems like [Node.js](http://nodejs.org/)) 32 | * it corresponds more to what computers intrinsically are: the most primitive communication facilities on microprocessors are the [OUT instruction](http://x86.renejeschke.de/html/file_module_x86_id_222.html) and the [interruption mechanism](http://en.wikipedia.org/wiki/Interrupt). The [Direct Memory Access](http://en.wikipedia.org/wiki/Direct_memory_access) is a fastest solution in practice, but also relies on these primitives. Finally, this corresponds to the [Xen API](http://openmirage.org/wiki/xen-events) design, in the hope that some day Coq could be ported as an unikernel like OCaml with [MirageOS](http://www.openmirage.org/). 33 | 34 | ### Implementation 35 | The implementation of this DSL is two folds. In Coq, a `run` function gives an executable semantics of the computations. The existence of an executable semantics is an improvement over other works which only give unrealized axioms for I/Os effects (for example in Haskell or in Idris, see [IO.idr](https://github.com/idris-lang/Idris-dev/blob/master/libs/prelude/IO.idr)). 36 | 37 | We also compile Coq programs to OCaml using a customized version of the [extraction mechanism](http://www.pps.univ-paris-diderot.fr/~letouzey/download/letouzey_extr_cie08.pdf) of Coq. The impure operators are compiled to impure OCaml operators realizing the effects, like sending messages to the OS. 38 | 39 | The impure effects can be classified into three categories: 40 | 41 | * memory 42 | * exit 43 | * asynchronous calls 44 | 45 | The memory and exit effects are implemented, both in Coq and OCaml, by the monadic transformations of the state and exception monads (see [monadic transformations](http://gallium.inria.fr/~xleroy/mpri/progfunc/monads.2up.pdf)). 46 | 47 | In Coq, the asynchronous calls are represented by a monad reader and a monad writer. The system-calls are typed, and to each request type corresponds one answer type. The handlers are stored into a heap and must have a type compatible with their request. They have their own private memory (implemented as a state monad), and can be called an unbounded number of times (for example, with a web server, the socket listener will get as many inputs as connecting clients). A handler can also disconnect itself to be garbage collected. 48 | 49 | In OCaml, the handlers are stored in the heap extracted from the Coq implementation. Communication with the OS occurs through a unique bidirectional pipe. The `Send` method writes a message on the pipe. A single loop listens synchronously to the pipe, and dispatches the events to the handlers. All messages are serialized into strings. At the other end of the pipe sits a [proxy](https://github.com/coq-concurrency/proxy). The proxy parses the messages and translates them into real system-calls. It is implemented in OCaml using [Lwt](http://ocsigen.org/lwt/). Note that Lwt is not used as a lightweight threads library, but as an asynchronous API to Unix. For a more technical discussion about asynchronous system-calls in Unix, you can read this [paper](http://www.pps.univ-paris-diderot.fr/~jch/research/cpc-2012.pdf) of Kerneis and Chroboczek. 50 | 51 | ![Schema](static/images/pluto_runtime.svg "Implementation architecture") 52 | 53 | ### Correction 54 | There is no mechanism to prove properties about the effects yet (I/Os, shared memory, ...). However it is possible to certify the functional part of the server, or any interactive Coq application based on our runtime, using standard techniques in Coq (see for example the [Program extension](http://www.pps.univ-paris-diderot.fr/~sozeau/research/publications/Program-ing_Finger_Trees_in_Coq.pdf) of Matthieu Sozeau). 55 | 56 | We can also question the correctness of the compilation to OCaml, or whether the `run` function in Coq is faithful to the extracted code. We have no formal proof, but we designed the system with correctness in mind. In particular, we designed our DSL with a minimal "attack surface". The memory, the lightweight threads, the exit effect are compiled by monadic transformation in Coq and then using the standard extraction mechanism. For I/Os, there is no abstract `World` type which could lead to duplication in Coq. We just use a monad reader and monad writer, and messages are sent or read into a pipe. On the long term, we could dream of a formally proven Coq compiler and adapt it to our customized compilation of read/write effects. 57 | 58 | ## Code extracts 59 | This is the main function of the server, in [pluto/Pluto.v](https://github.com/coq-concurrency/pluto/blob/master/Pluto.v): 60 | 61 | Definition program (argv : list LString.t) : C.t [] unit := 62 | match argv with 63 | | [_; port; website_dir] => 64 | match LString.to_N 10 port with 65 | | None => print_usage tt 66 | | Some port_number => 67 | Time.get (fun time => 68 | let time := Moment.Print.rfc1123 @@ Moment.of_epoch @@ Z.of_N time in 69 | let welcome_message := LString.s "Pluto starting on " ++ website_dir ++ 70 | LString.s ", port " ++ port ++ 71 | LString.s ", on " ++ time ++ LString.s "." in 72 | Log.write welcome_message (fun _ => 73 | ServerSocket.bind port_number (fun client => 74 | match client with 75 | | None => 76 | Log.write (LString.s "Server socket failed.") (fun _ => C.Exit tt) 77 | | Some client => handle_client website_dir client 78 | end))) 79 | end 80 | | _ => print_usage tt 81 | end. 82 | 83 | The program is parametrized by the list of the command-line arguments. The return type is `C.t [] unit`, the type of computations returning `unit` with an empty shared memory (`[]` means that the list of memory cells is empty). 84 | 85 | We first check that there are two arguments, the first one being an integer (the port number). Else we print a usage message and exit. Then we request the current time to log it. The definition of `Time.get` is in [system/StdLib.v](https://github.com/coq-concurrency/system/blob/master/StdLib.v): 86 | 87 | (** Get the current time (the number of seconds since Unix epoch). *) 88 | Definition get {sig : Signature.t} (handler : N -> C.t sig unit) 89 | : C.t sig unit := 90 | C.Send Command.Time tt tt (fun _ time => 91 | do! handler time in 92 | C.Ret None). 93 | 94 | The function `get` is parametrized by a handler `N -> C.t sig unit` which is a computation receiving the time in seconds. The `handler` is called when the OS responds to the `C.Send` operator, calling the callback `fun _ time => do! handler time in C.Ret None`. This callback has no private memory (actually a private memory of type `unit` initialized to `tt`). The callback always returns `None` to disconnect itself after an answer (so that the handler is called at most once, even if the OS is bugged and responds several times to the `Command.Time` request). 95 | 96 | We log the current time in RFC 1123 format with a welcome message and bind to the server socket. The handler is listening while there are new clients connecting, and runs the `handle_client` method for each. This program may never terminate, while we are writing pure Coq without explicit non-termination monad. The non-termination effect is provided by the monad reader, which is compiled to OCaml as an infinite loop listening to the system pipe. 97 | 98 | ## Future work 99 | We learn many things writing a realistic example of a web server in Coq, especially in the way of implementing side effects. We also shown that it is possible to use Coq as a programming language to write interactive and concurrent softwares. 100 | 101 | Our main goal now is to extend our DSL of computations with a specification and a certification mechanism. We would like to write a specification of the non-purely functional part of our web server and prove its correctness, exploiting the unique ability of Coq to marry proofs and programs. 102 | 103 | Note: *Pluto is also the only planet discovered and undiscovered by the Americans. The [New Horizons](http://en.wikipedia.org/wiki/New_Horizons) space probe should give us more insights about this mysterious object.* 104 | -------------------------------------------------------------------------------- /posts/2014-12-18 Checking concurrent programs with symbolic simulations.md: -------------------------------------------------------------------------------- 1 | Concurrent programs with inputs/outputs are hard to test: they are non-deterministic and interact with the outside world. They are even harder to formally specify completely, and prove correct. We present (what we believe to be) a novel approach to check such programs doing symbolic simulations in [Coq](https://coq.inria.fr/). 2 | 3 | Symbolic simulations are like deterministic simulations one concrete data, using symbolic values to run and check the results of a *full set* of simulations. Here is a good talk about testing real systems by simulation: [Testing Distributed Systems with Deterministic Simulation](https://foundationdb.com/videos/testing-distributed-systems-with-deterministic-simulation). We use the Coq's pure programming language Gallina to describe the behavior of a program's environment, which can include the user, the operating system, other connected computers, ... The environment is responsible for two things: 4 | 5 | * feeding data to the program 6 | * checking that its answers verify some properties 7 | 8 | We hope this approach will provide a good compromise between full specifications and testing on single data instances. We will define a simple concurrent calculus and give some examples. 9 | 10 | All the following code extracts are in Coq and available on [coq-concurrency/system#simulation-callbacks](https://github.com/coq-concurrency/system/tree/simulation-callbacks). 11 | 12 | ## Calculus with handlers 13 | To allow simpler reasoning, we want our calculus to be somehow "deterministic". A first way is to remove shared mutable variables. Then the only source of non-determinism is the order of the input events. Having shared mutable states is often useful, typically to implement a data storage. Still, we will try to go as far as possible without mutable states. 14 | 15 | An second way to reduce non-determinism is to restrict how we are waiting for events and doing threads. We define a calculus with only asynchronous calls and forks (without joins): 16 | 17 | Inductive t : Type := 18 | | Ret : t 19 | | Par : t -> t -> t 20 | | Send : forall (command : Command.t), Command.request command -> 21 | (Command.answer command -> t) -> t. 22 | 23 | A computation can do three things: 24 | 25 | * `Ret`: do nothing 26 | * `Par c1 c2`: start `c1` and `c2` in parallel 27 | * `Send command request (fun answer => c)`: emit a request `request` of kind `command` with a handler for the answer 28 | 29 | This is the classical HelloWorld: 30 | 31 | Definition hello_world : C.t := 32 | do! Command.Write @ LString.s "Hello world!" in 33 | C.Ret. 34 | 35 | We use the notation `do! command @ request in e` for `Send command request (fun _ => e)`. 36 | 37 | Here is a simple echo program repeating one user input: 38 | 39 | Definition echo : C.t := 40 | let! message := Command.Read @ tt in 41 | do! Command.Write @ message in 42 | C.Ret. 43 | 44 | The `let!` is like the `do!` but naming the argument of the handler. 45 | 46 | These examples are sequential programs. Note that in practice they may not terminate, if the operating system never answers to the requests. There can be at most one answer per request. We use a fixpoint and the `Par` operator to handle several requests in parallel: 47 | 48 | Fixpoint echo_par (fuel : nat) : C.t := 49 | match fuel with 50 | | O => C.Ret 51 | | S fuel => C.Par echo (echo_par fuel) 52 | end. 53 | 54 | This program concurrently waits for `fuel` user inputs, prints them and stops. 55 | 56 | ## Simulations 57 | A simulation, or a run, is a co-program over a concurrent and interactive program. It answers to the requests of the program, playing the role of the environment. A simulation is defined by induction over the program's structure. This has two advantages: 58 | 59 | * by construction, a simulation must give exactly one answer per request 60 | * you can construct the simulation following the structure of the program 61 | 62 | The type of a simulation is: 63 | 64 | Inductive t : C.t -> Type := 65 | | Ret : t C.Ret 66 | | Par : forall {c1 c2 : C.t}, t c1 -> t c2 -> t (C.Par c1 c2) 67 | | Send : forall (command : Command.t) (request : Command.request command) 68 | (answer : Command.answer command) {handler : Command.answer command -> C.t}, 69 | t (handler answer) -> t (C.Send command request handler). 70 | 71 | For convenience, we use the tactic mode of Coq to build simulations. Doing so, we get a kind of interactive and symbolic debugger for our programs. This helps writing meaningful simulations. Here is the simulation of HelloWorld: 72 | 73 | Definition hello_world_run : Run.t hello_world. 74 | apply (Run.Send Command.Write (LString.s "Hello world!") tt). 75 | exact Run.Ret. 76 | Defined. 77 | 78 | This simulation checks that the program `hello_world` does exactly one thing: sending the message `"Hello world!"`. For the echo of one message: 79 | 80 | Definition run_echo (message : LString.t) : Run.t echo. 81 | apply (Run.Send Command.Read tt message). 82 | apply (Run.Send Command.Write message tt). 83 | exact Run.Ret. 84 | Defined. 85 | 86 | We check that, for any message entered by the user, the program will only print this message. Given a list of messages, we can construct a set of simulations for the concurrent echo: 87 | 88 | Fixpoint run_echo_par (messages : list LString.t) 89 | : Run.t (echo_par (List.length messages)). 90 | destruct messages as [|message messages]. 91 | - exact Run.Ret. 92 | - apply Run.Par. 93 | * exact (run_echo message). 94 | * exact (run_echo_par messages). 95 | Defined. 96 | 97 | This simulation is recursive to follow the shape of the simulated program. It reuses the simulation `run_echo` of `echo`. The simulation is defined for any order of interleaving of the message events. 98 | 99 | ## Time server 100 | In the file [Simulation.v](https://github.com/coq-concurrency/system/blob/simulation-callbacks/Simulation.v) we give an example of a simple time server. The clients can connect and get the current time of the server. Unlike for the previous examples, we do not cover all possible runs. Instead we give two sets of simulations: 101 | 102 | * a first one where the server socket cannot be bound 103 | * a second one where both the server and clients sockets never return an error 104 | 105 | We do not cover all the cases, for example the case of a client socket which cannot be written to. We believe this is both a weakness and a strength of this approach. Contrary to a full specification, not all the execution paths are tested. But simulations are simple to write, while covering more cases than traditional tests. 106 | 107 | ## Future work 108 | We want to extend this method to check more complex programs, like a database or a chat-server. For that we should first extend the expressiveness of the calculus, and so of the simulations. 109 | 110 | We also want to relate this method to the specifications plus proofs approach. A first way could be to have simulations which *by construction* cover all the execution paths. An other way could be to express properties over simulations to enforce a specification over any simulations. 111 | -------------------------------------------------------------------------------- /posts/2015-01-16 A blog engine written and proven in Coq.md: -------------------------------------------------------------------------------- 1 | I present ChickBlog ([sources](https://github.com/clarus/coq-chick-blog) on GitHub), a blog engine written and proven in [Coq](https://coq.inria.fr/). 2 | 3 | This is a demo blog engine where a user can login (no passwords), add, edit or delete posts. The code is written mostly in Coq, compiled to [OCaml](https://ocaml.org/) and linked to the [CoHTTP](https://github.com/mirage/ocaml-cohttp) library to handle the HTTP protocol. 4 | 5 | The aim of this project is to demonstrate that applications with I/Os can be written and specified naturally using the (new) concept of [symbolic simulations in Coq](http://coq-blog.clarus.me/checking-concurrent-programs-with-symbolic-simulations.html). 6 | 7 | ## Run 8 | Get the sources: 9 | 10 | git clone https://github.com/clarus/coq-chick-blog.git 11 | cd coq-chick-blog/ 12 | 13 | Add the Coq repositories with [OPAM](https://opam.ocaml.org/): 14 | 15 | opam repo add coq-released https://coq.inria.fr/opam/released 16 | 17 | Install the dependencies: 18 | 19 | opam install --jobs=4 coq-list-string coq-error-handlers coq-function-ninjas coq-moment 20 | opam install --jobs=4 lwt cohttp 21 | 22 | Download the CSS: 23 | 24 | curl -L https://github.com/clarus/coq-red-css/releases/download/coq-blog.1.0.2/style.min.css >extraction/static/style.min.css 25 | 26 | Compile: 27 | 28 | ./configure.sh && make 29 | cd extraction/ 30 | make 31 | 32 | Run on [localhost:8008](http://localhost:8008/): 33 | 34 | ./chickBlog.native 35 | 36 | ### Dockerfile 37 | A Dockerfile is provided to run ChickBlog in an isolated environment using [Docker](https://www.docker.com/): 38 | 39 | docker build --tag=chick-blog . 40 | docker run -ti -p 8008:8008 chick-blog 41 | 42 | ## Specification 43 | The blog is defined in `Main.v` as the function: 44 | 45 | Definition server (path : Path.t) (cookies : Cookies.t) : C.t Response.t. 46 | 47 | It handles an HTTP request and generate an answer using system calls to the file system. The type `C.t A` represents a computation doing I/O operations: 48 | 49 | Inductive t (A : Type) : Type := 50 | | Ret : forall (x : A), t A 51 | | Call : forall (command : Command.t), (Command.answer command -> t A) -> t A. 52 | 53 | A computation can either: 54 | 55 | * return a pure value of type `A` 56 | * call an external command and wait for its result 57 | 58 | The purity of Coq ensures that each request is answered exactly once in finite time. We specify the behavior of the server in `Spec.v`. 59 | 60 | ### Scenarios 61 | A scenario is a set of runs of the server. A type-checking scenario shows that the server behaves as expected in a certain use case. For example, we check that when we create, edit and view a post we get the same result as what we entered. You can think of a scenario as a unit test with universally quantified variables. 62 | 63 | Here is a simple check of the execution of the index page: 64 | 65 | (** The index page when the list of posts is available. *) 66 | Definition index_ok (cookies : Cookies.t) (post_headers : list Post.Header.t) 67 | : Run.t (Main.server Path.Index cookies). 68 | (* The handler asks the list of available posts. We return `post_headers`. *) 69 | apply (Call (Command.ListPosts _ ) (Some post_headers)). 70 | (* The handler terminates without other system calls. *) 71 | apply (Ret (Response.Index (Cookies.is_logged cookies) post_headers)). 72 | Defined. 73 | 74 | Given any `cookies` and `post_headers`, we execute the server handler on the page `Request.Path.Index`. The handler does exactly one system call, to which we answer `Some post_headers`, playing the role of the system. The final response of the server is then `Response.Public.Index post_headers`. Note that we do not need to execute `index_ok` on every instances of `cookies` and `post_headers`: since the type-system of Coq is supposed sound, it is enough to type-check `index_ok`. 75 | 76 | ### Privacy 77 | We check that, for any runs of a program, an unauthenticated user cannot access private pages (like edit) or modify the file system with system calls. 78 | 79 | *More comments on [Hacker News](https://news.ycombinator.com/item?id=9037115), [Reddit](https://www.reddit.com/r/programming/comments/2vn3xa/a_blog_engine_written_and_proven_in_coq/) and [Lobsters](https://lobste.rs/s/7ugonf/a_blog_engine_written_and_proven_in_coq/comments/4oholx).* 80 | -------------------------------------------------------------------------------- /posts/2015-02-17 A bench system for the Coq packages.md: -------------------------------------------------------------------------------- 1 | We will present the [bench system](http://coq-bench.github.io/) for the [Coq](https://coq.inria.fr/) packages. 2 | 3 | [OPAM](http://opam.ocamlpro.com/) is the package manager for Coq. The number of packages is growing, so we needed a bench system to automatically check that all packages are compiling. The bench system will help both: 4 | 5 | * packages developers, to check that their programs are compiling with correct dependency constraints; 6 | * Coq developers, to monitor the changes breaking compatibility. 7 | 8 | ## Use the bench 9 | The results of the bench are available on [coq-bench.github.io](http://coq-bench.github.io/), using the two installation strategies `clean` and `tree`. The results are presented in a colored table with the installation times for valid packages. The `best` column contains the best score obtained for each package. 10 | 11 | We check: 12 | 13 | * that packages are well-formed, using our [lint](https://github.com/coq-bench/run/blob/master/lint.rb) checker 14 | * the installation of dependencies (with a timeout of 5 minutes) 15 | * the installation of the package (with a timeout of 30 minutes) 16 | * the removing of the package (by comparing the list of files before installation and after removing) 17 | 18 | We record the duration of each operation and compute the installation size. 19 | 20 | Because your dependencies may evolve, it is good practice to regularly check the bench results for your packages. The `clean` strategy installs each package in a clean environment whereas the `tree` strategy installs as many packages as possible before to remove incompatible ones. In practice, the `tree` strategy yields more errors because packages are tested in an environment polluted by many other packages. 21 | 22 | ## Architecture 23 | The bench system is hosted on GitHub in the organization [coq-bench](https://github.com/coq-bench). There are four projects: 24 | 25 | * [run](https://github.com/coq-bench/run): run the benchmarks 26 | * [database](https://github.com/coq-bench/database): the backup of the benchmarks 27 | * [make-html](https://github.com/coq-bench/make-html): generate the web pages 28 | * [coq-bench.github.io](https://github.com/coq-bench/coq-bench.github.io): the website itself 29 | 30 | ### [run](https://github.com/coq-bench/run) 31 | This program runs the benchmarks and is written in Ruby. The entry-point is [long_run.rb](https://github.com/coq-bench/run/blob/master/long_run.rb). For each bench a clean [Docker](https://www.docker.com/) image is generated from the parametrized [Dockerfile.erb](https://github.com/coq-bench/run/blob/master/Dockerfile.erb). Then OCaml, OPAM and Coq are installed, and each package is tested. The packages can be from the stable repository or both the stable and unstable repositories. The results are then saved into the database. 32 | 33 | ### [database](https://github.com/coq-bench/database) 34 | The database is in the [CSV](http://en.wikipedia.org/wiki/Comma-separated_values) format. There is one file per bench and one row per package. The first row gives a legend for each of the 31 columns. 35 | 36 | ### [make-html](https://github.com/coq-bench/make-html) 37 | This program generates the static HTML pages from the CSV database and is written in Ruby. For example, to generate the pages of the `clean` results: 38 | 39 | ruby make_html.rb ../database/clean html/clean 40 | 41 | ### [coq-bench.github.io](https://github.com/coq-bench/coq-bench.github.io) 42 | These are the static HTML pages of the bench website. [GitHub](https://github.com/) provides us a nice and simple way to host web pages from a Git repository using [GitHub Pages](https://pages.github.com/). 43 | 44 | ## Strategies 45 | There are many possible strategies with respect to the installation order of the packages. The installation order is important because OPAM installs different dependencies in different contexts. 46 | 47 | We would like to optimize the installation order to reduce the total execution time of the bench, by always installing and testing the dependencies first, so that no packages are compiled twice. Unfortunately this is not possible. Here is a simple counter example. With the following list of packages: 48 | 49 | * `A.1.0.0` 50 | * `A.2.0.0` 51 | * `B.1.0.0` depending on `A` (any versions) 52 | * `C.1.0.0` depending on `A.1.0.0` and `B` 53 | * `C.2.0.0` depending on `A.2.0.0` and `B` 54 | 55 | we must compile `B` twice (once with `A.1.0.0` and once with `A.2.0.0`) to test both `C.1.0.0` and `C.2.0.0`. 56 | 57 | We provide the two following strategies. 58 | 59 | ### Clean 60 | This is the simplest strategy. We install each package in a fresh environment. This is the most robust and reproducible strategy. But this is not really optimal for packages with big dependencies because they are always reinstalled from scratch. 61 | 62 | ### Tree 63 | This is a more complex strategy. We install as many packages as possible until all new packages are incompatible with the current environment. The main source of incompatibility is the fact that we cannot install two packages with the same name but different version numbers. Once we are blocked, we roll-back until new packages are installable. 64 | 65 | This strategy is more clever but also more fragile. We use the branch mechanism of Git on the `.opam` folder to switch efficiently between OPAM states. At the end of the process, we obtain a tree of all the Git branches used to explore the packages space. For example, for the stable repository: 66 | 67 | * b99f693 coq-concurrency-pluto.1.0.0 68 | * a1ff2f1 coq-concurrency-system.1.0.0 69 | * 27c7b93 coq-moment.1.0.0 70 | * d950f53 coq-list-string.2.0.0 71 | | * 4817264 coq-flocq.2.2.0 72 | | | * 3d4105d coq-flocq.2.3.0 73 | | |/ 74 | | | * 544de40 coq-concurrency-proxy.1.0.0 75 | | | * 9a1989e coq-coqeal:refinements.0.9.1 76 | | | * 0bfd44f coq-fpmods.0.2.0 77 | | | * bda2af7 coq-coqeal:theory.0.9.1 78 | | | * 27aa320 coq-plouffe.1.0.0 79 | | | * 35961d5 coq-coquelicot.2.0.1 80 | | | * 89db498 coq-error-handlers.1.0.0 81 | | | * aa85f8c coq-flocq.2.4.0 82 | | |/ 83 | | * 48f33a4 coq-iterable.1.0.0 84 | | * 0a48735 coq-function-ninjas.1.0.0 85 | | * e2980d1 coq-list-plus.1.0.0 86 | | * 4122f1a coq-list-string.1.0.0 87 | |/ 88 | * 89705ad coq-math-classes.1.0.2 89 | * 5d3ec5d coq-math-comp.1.5.0 90 | * 470eefb coq-ssreflect.1.5.0 91 | * b0205c8 Initial files. 92 | 93 | ## Related work 94 | The OPAM for OCaml community did some work to obtain a bench system too. There are: 95 | 96 | * [OPAM Weather Service](http://ows.irill.org/): do not install the packages, only check the dependency constraints 97 | * [OPAM Bulk](http://www.recoil.org/~avsm/opam-bulk/): a future bench system 98 | * [OCamlot](https://github.com/ocamllabs/ocamlot): a bench system (abandoned?) 99 | -------------------------------------------------------------------------------- /posts/2015-02-20 Tutorial: a Hello World in Coq.md: -------------------------------------------------------------------------------- 1 | We will present the classic [Hello World](http://en.wikipedia.org/wiki/%22Hello,_world!%22_program) program in [Coq](https://coq.inria.fr/). We will explain how to compile, run, and certify interactive programs in Coq. We will use the library [Coq.io](http://coq.io/). 2 | 3 | The *Hello World* program exists in almost every languages, including [in White Space](http://en.wikipedia.org/wiki/List_of_Hello_world_program_examples#W). In Coq this is more complicated because the language is purely functional. This means that no effects can be done, in particular no inputs-outputs. This constraint is there to preserve the logical consistency of the system. However, we can still encode inputs-outputs by defining a [monad](http://en.wikipedia.org/wiki/Monad_%28functional_programming%29). This technique was popularized by the [Haskell](http://en.wikipedia.org/wiki/Haskell_%28programming_language%29) programming language. 4 | 5 | ## Hello World 6 | Using [OPAM for Coq](http://coq-blog.clarus.me/use-opam-for-coq.html), install the package [coq-io-system](https://github.com/coq-io/system): 7 | 8 | opam repo add coq-released https://coq.inria.fr/opam/released 9 | opam install coq-io-system 10 | 11 | You can now write the *Hello World*: 12 | 13 | Require Import Coq.Lists.List. 14 | Require Import Io.All. 15 | Require Import Io.System.All. 16 | Require Import ListString.All. 17 | 18 | Import ListNotations. 19 | Import C.Notations. 20 | 21 | (** The classic Hello World program. *) 22 | Definition hello_world (argv : list LString.t) : C.t System.effect unit := 23 | System.log (LString.s "Hello world!"). 24 | 25 | We load some libraries and write the `hello_world` program by calling the `System.log` function. The return type is `C.t System.effect unit`, which means this is a *impure* computation with *System* effects and returning a value of type `unit`. The command line arguments are given in the list of strings `argv`, but not used here. 26 | 27 | To compile the `hello_world` program we add: 28 | 29 | Definition main := Extraction.launch hello_world. 30 | Extraction "extraction/main" main. 31 | 32 | to generate an [OCaml](https://ocaml.org/) file `main.ml`. We compile and run this file: 33 | 34 | ocamlbuild main.native -use-ocamlfind -package io-system 35 | ./main.native 36 | 37 | This should display `Hello world!` on the terminal. 38 | 39 | ### Specification 40 | The specification of this program is very straightforward: *the program only prints "Hello world!" and quits* (here the specification is as long as the program itself, but this is not always the case). A simple way to write this specification is to describe an environment of the program which reacts to a single event: the printing of the message "Hello world!" on the terminal. You can express this environment as a program: 41 | 42 | (** The Hello World program only says hello. *) 43 | Definition hello_world_ok (argv : list LString.t) : Run.t (hello_world argv) tt. 44 | apply (Run.log_ok (LString.s "Hello world!")). 45 | Defined. 46 | 47 | The specification `hello_world_ok` is of type `Run.t (hello_world argv) tt`: this runs the program `hello_world` on an argument `argv` and returns the result `tt`. We just apply the function `Run.log_ok` of the [coq-io-system](https://github.com/coq-io/system) library which exactly describes an environment reacting to a single printing event. 48 | 49 | This specification needs *no* proofs because it is valid *by construction*. We do not need any SMT solver, model checker or manual proof. The specification is valid because it is well-typed. Of course this example is very simple, so we will see a slightly more complex one. 50 | 51 | ## What is your name? 52 | This program asks for your name and replies with your name: 53 | 54 | (** Ask for the user name and answer hello. *) 55 | Definition your_name (argv : list LString.t) : C.t System.effect unit := 56 | do! System.log (LString.s "What is your name?") in 57 | let! name := System.read_line in (* Ask the name. *) 58 | match name with 59 | | None => ret tt (* In case of error do nothing. *) 60 | | Some name => System.log (LString.s "Hello " ++ name ++ LString.s "!") 61 | end. 62 | 63 | We see here how to compose impure computations in sequence with the `do!` and `let!` keywords. The construct: 64 | 65 | let! x := e1 in e2 66 | 67 | executes `e1`, assigns the result to `x` and then executes `e2`. The `do!` is a syntactic sugar for a `let!` with an empty variable name. We use the `System.read_line` function which gets a new line on the standard input. If the `read_line` operation fails we returns the pure value `tt` using the `ret` operator, else we print the user name on the terminal. 68 | 69 | You can run this program as before by compilation to OCaml. 70 | 71 | ### Specification 72 | We have two [use cases](http://en.wikipedia.org/wiki/Use_case) for the `your_name` program: 73 | 74 | * when the user enters a name, 75 | * when the standard input is broken. 76 | 77 | For the first use case: 78 | 79 | (** The `your_name` program answers something when you give your name. *) 80 | Definition your_name_ok (argv : list LString.t) (name : LString.t) 81 | : Run.t (your_name argv) tt. 82 | apply (Run.Let (Run.log_ok _)). 83 | apply (Run.Let (Run.read_line_ok name)). 84 | apply (Run.log_ok _). 85 | Defined. 86 | 87 | The specification is parametrized by a `name` which could be any string. The environment does exactly three steps: 88 | 89 | * display something on the terminal (the underscore `_` means any value) 90 | * answer `name` to the event `read_line` 91 | * display something on the terminal 92 | 93 | The `Run.Let` command composes two steps. For a failing standard input: 94 | 95 | (** The `your_name` program does nothing more in case of error on stdin. *) 96 | Definition your_name_error (argv : list LString.t) : Run.t (your_name argv) tt. 97 | apply (Run.Let (Run.log_ok _)). 98 | apply (Run.Let Run.read_line_error). 99 | apply Run.Ret. 100 | Defined. 101 | 102 | we also give three steps: 103 | 104 | * display something on the terminal 105 | * answer by an error to the event `read_line` 106 | * terminate (the command `Run.Ret`) 107 | 108 | Again, we do not need to prove anything because the specifications are correct by construction. 109 | 110 | Next time we will continue with more tutorials, to help you build and certify realistic interactive applications in Coq. 111 | -------------------------------------------------------------------------------- /posts/2015-03-04 Write a script in Coq.md: -------------------------------------------------------------------------------- 1 | We will explain how to write scripts in [Coq](https://coq.inria.fr/) using the library [Coq.io](http://coq.io/) with the example of [repos2web](https://github.com/clarus/repos2web), a website generator. This generator parses an [OPAM](http://opam.ocaml.org/) repository with Coq packages and generates an HTML page. 2 | 3 | *There is now a newer OPAM website generator [opam-website](https://github.com/coq-io/opam-website). See the results on [coq.io/opam](http://coq.io/opam/).* 4 | 5 | ## Get started 6 | Install the [coq-io-system](https://github.com/coq-io/system) package with OPAM to enable the system effects. See [Use OPAM for Coq](http://coq-blog.clarus.me/use-opam-for-coq.html) to configure OPAM for Coq. 7 | 8 | opam repo add coq-released https://coq.inria.fr/opam/released 9 | opam install coq-io-system 10 | 11 | Create an empty Coq project by adding the following files in a fresh directory: 12 | 13 | * `configure.sh`, the configure script: 14 | 15 | #!/bin/sh 16 | 17 | coq_makefile -f Make -o Makefile 18 | 19 | * `Make`, the `coq_makefile` project file: 20 | 21 | -R src Repos2Web 22 | 23 | src/Main.v 24 | 25 | * `src/Main.v`, the main source file: 26 | 27 | Require Import Coq.Lists.List. 28 | Require Import Io.All. 29 | Require Import Io.System.All. 30 | Require Import ListString.All. 31 | 32 | Import ListNotations. 33 | Import C.Notations. 34 | 35 | (** The main function. *) 36 | Definition main (argv : list LString.t) : C.t System.effects unit := 37 | System.log (LString.s "test"). 38 | 39 | (** The extracted program. *) 40 | Definition repos2web := Extraction.run main. 41 | Extraction "extraction/repos2web" repos2web. 42 | 43 | * `extraction/Makefile`, the Makefile for the extracted program: 44 | 45 | build: 46 | ocamlbuild repos2web.native -use-ocamlfind -package io-system 47 | 48 | clean: 49 | ocamlbuild -clean 50 | 51 | Compile your Coq code to [OCaml](http://ocaml.org/): 52 | 53 | ./configure.sh 54 | make 55 | 56 | Compile and run the generated OCaml: 57 | 58 | cd extraction/ 59 | make 60 | ./repos2web.native 61 | 62 | This should print you the message `test` on the terminal! 63 | 64 | ## Parse the OPAM repository 65 | To write our script we need to understand the basis of how OPAM for Coq repositories are organized (for example in [opam-coq-archive](https://github.com/coq/opam-coq-archive)). All the packages are in the `packages` folders. There is one folder per package name, all prefixed by `coq-` because we are in the Coq namespace. In each package folder, there is one folder per version of the package with three files `descr`, `opam` and `url` to describe the package. 66 | 67 | packages/ 68 | coq-list-string/ 69 | coq-list-string.1.0.0/ 70 | descr 71 | opam 72 | url 73 | coq-list-string.2.0.0/ 74 | ... 75 | ... 76 | 77 | We define the data type of an OPAM repository in [src/Model.v](https://github.com/clarus/repos2web/blob/master/src/Model.v). In a first pass, we will generate an element of type `Packages.t`. This is a list of packages described by a name and a list of versions. In a second pass, we will generate an element of type `FullPackages.t`, by adding the description of each version and by computing each latest version using the (complex) [Debian ordering](https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version). 78 | 79 | ### First pass 80 | The first pass is described in [src/Main.v](https://github.com/clarus/repos2web/blob/master/src/Main.v) in the `Basic` module. The function `list_coq_files` lists the files/folders which are starting with the `coq-` prefix in a given `folder`: 81 | 82 | Definition list_coq_files (folder : LString.t) : C (option (list Name.t)) := 83 | let! folders := System.list_files folder in 84 | match folders with 85 | | None => 86 | do! log (LString.s "The folder " ++ folder ++ LString.s " cannot be listed.") in 87 | ret None 88 | | Some folders => ret (Some (Name.of_strings folders)) 89 | end. 90 | 91 | Let us precise the return type `C (option (list Name.t))`. We defined `C` as: 92 | 93 | Definition C := C.t System.effects. 94 | 95 | for convenience. This means than `C` is the type of computations doing interactions with the system. We need to use this special type because Coq is a purely functional language: without using the type `C`, functions cannot do inputs--outputs. 96 | 97 | The type `C` is parametrized by `option (list Name.t)`, the result type of the function. Thus, the result of `list_coq_files` can be either: 98 | 99 | * `None` in case of error, 100 | * some list of names in case of success. 101 | 102 | As described in [Tutorial: a Hello World in Coq](http://coq-blog.clarus.me/tutorial-a-hello-world-in-coq.html), we use the `let!` operator to combine computations and the `ret` operator to return a pure value. We interact with the system by calling the following functions: 103 | 104 | * `list_files : LString.t -> C (option (list LString.t))`: list the content of a folder 105 | * `log : LString.t -> C unit`: print a message on the terminal 106 | 107 | The complete list of system functions is available on [system API](http://clarus.github.io/doc/io-system/Io.System.System.html). 108 | 109 | We continue by defining more functions and conclude with: 110 | 111 | Definition get_packages (repository : LString.t) : C (option Packages.t) := 112 | let! names := list_coq_files repository in 113 | match names with 114 | | None => ret None 115 | | Some names => get_packages_of_names repository names 116 | end. 117 | 118 | to get the list of packages in a repository folder (or `None` in case of error). 119 | 120 | ### Second pass 121 | The second pass follows the same structure as the first one. The main trick is the function: 122 | 123 | (** Return the latest version, using Debian `dpkg` for comparison. *) 124 | Definition max_version (version1 version2 : Version.t) : C (option Version.t) := 125 | ... 126 | 127 | which uses the `dpkg` command line tool to compare two versions numbers according to the [Debian ordering](https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version). If you want to test it, the `dpkg` tool should be available on most Linux distribution, even on those which are not based on [Debian](https://www.debian.org/). 128 | 129 | ## Render the HTML 130 | We define the HTML rendering in [src/View.v](https://github.com/clarus/repos2web/blob/master/src/View.v). The last function is: 131 | 132 | Definition index (packages : FullPackages.t) : LString.t := 133 | header ++ title packages ++ table packages ++ footer. 134 | 135 | which pretty-prints a list of packages to HTML. This function is pure (no inputs--outputs), because the return type is not `C` of something but `LString.t`. There is nothing special about the pretty-printing, and we generate the page using the [Bootstrap](http://getbootstrap.com/) CSS framework to get a nice rendering. 136 | 137 | The final `main` function in [src/Main.v](https://github.com/clarus/repos2web/blob/master/src/Main.v) combines the parsing and the rendering to write the output file in `html/index.html`: 138 | 139 | Definition main (argv : list LString.t) : C unit := 140 | match argv with 141 | | [_; repository] => 142 | let repository := repository ++ LString.s "/packages" in 143 | let! packages := Basic.get_packages repository in 144 | match packages with 145 | | None => log (LString.s "The packages cannot be listed.") 146 | | Some packages => 147 | let! full_packages := Full.get_packages repository packages in 148 | let index_content := View.index full_packages in 149 | let index_name := LString.s "html/index.html" in 150 | let! is_success := System.write_file index_name index_content in 151 | if is_success then 152 | log (index_name ++ LString.s " generated.") 153 | else 154 | log (LString.s "Cannot generate " ++ index_name ++ LString.s ".") 155 | end 156 | | _ => log (LString.s "Exactly one argument expected (the repository folder).") 157 | end. 158 | 159 | We use the list of command line arguments `argv` to get the folder in which the OPAM repository is stored. You can add a Coq-ish theme downloading this Bootstrap CSS: 160 | 161 | curl -L https://github.com/clarus/coq-red-css/releases/download/coq-blog.1.0.2/style.min.css >html/style.min.css 162 | 163 | ## Next time 164 | We have seen how to write a script in Coq doing file manipulations. [Next time](http://coq-blog.clarus.me/formally-verify-a-script-in-coq.html) we will see how to specify this script and prove it correct, using a reasoning by [use cases](http://en.wikipedia.org/wiki/Use_case). 165 | -------------------------------------------------------------------------------- /posts/2015-03-05 Formally verify a script in Coq.md: -------------------------------------------------------------------------------- 1 | Last time, in [Write a script in Coq](http://coq-blog.clarus.me/write-a-script-in-coq.html), we explained how to implement scripts in [Coq](https://coq.inria.fr/) using [Coq.io](http://coq.io/) with the example of [repos2web](https://github.com/clarus/repos2web), a website generator for [OPAM](http://opam.ocaml.org/) repositories. We will see how to specify and prove correct this script. 2 | 3 | *There is now a newer OPAM website generator [opam-website](https://github.com/coq-io/opam-website). See the results on [coq.io/opam](http://coq.io/opam/).* 4 | 5 | ## Unit testing, revisited 6 | A common practice to check programs is to write [unit tests](http://en.wikipedia.org/wiki/Unit_testing). For each function, a test is written to execute it on some particular inputs, and to check that the results are what we expect. On more complex programs, we also need to simulate the inputs--outputs with the users, a database, the network, ... To solve this problem, programmers invented solutions like the [mock objects](http://en.wikipedia.org/wiki/Mock_object) which are basically implementations of fake execution environments. 7 | 8 | Unfortunately tests are not exhaustive since the set of program inputs is usually infinite. Methods like [random testing](http://en.wikipedia.org/wiki/Random_testing) can extend the range of tested configurations, but are always limited to a finite set of inputs. 9 | 10 | We will explain how to write exhaustive tests in Coq by writing *scenarios* which formalize the notion of [use cases](http://en.wikipedia.org/wiki/Use_case). The completeness of the scenarios is given us for free, thanks to the strict type-system of Coq: in our settings, a scenario is correct if it is well-typed so that there is no need to run it on its (infinite) set of parameters. 11 | 12 | To the best of our knowledge, Coq is the only language with formal verification of use cases. 13 | 14 | **Remark:** some people are used to specify programs with pre/post-conditions or invariants, rather than by [use cases](http://en.wikipedia.org/wiki/Use_case). We believe that these approaches are pertinent for algorithms but not for interactive programs, because (from our experience) these specifications are less natural than the specifications by use cases. This seems counter-intuitive because use-cases do not cover all the execution paths. But we have to remember that: 15 | 16 | * the Coq type system protects us from bad behaviors in all execution paths, 17 | * there are often hypothesis on the environment, so all execution paths may not be interesting, 18 | * the quality of a specification depends on its clarity and its similarities with the human intuition. 19 | 20 | ## Write the scenarios 21 | We write our scenarios in [src/Scenarios.v](https://github.com/clarus/repos2web/blob/master/src/Scenarios.v). There is at least one scenario per function, for the cases in which the environment has no bugs (no file system errors) and the repository architecture is well-formed. For example, for the function: 22 | 23 | Definition get_packages (repository : LString.t) : C (option Packages.t) := 24 | let! names := list_coq_files repository in 25 | match names with 26 | | None => ret None 27 | | Some names => get_packages_of_names repository names 28 | end. 29 | 30 | we write the following scenario: 31 | 32 | Definition get_packages_ok (repository : LString.t) (packages : Packages.t) 33 | : Run.t (get_packages repository) (Some packages). 34 | apply (Let (list_coq_files_ok _ (Packages.to_folders packages))). 35 | apply (get_packages_of_names_ok repository packages). 36 | Defined. 37 | 38 | ### What does this mean? 39 | 40 | This scenario is parametrized by any `repository` folder names and any lists of OPAM `packages`. It runs the function `get_packages` on `repository` in a "valid" environment and ensures that the result is `Some packages`, meaning that the repository was successfully parsed. 41 | 42 | The `get_package` function starts by a `let!` to call `list_coq_files`. We first reuse the scenario `list_coq_files_ok`. This describes a use case in which a list of folders starting by `coq-` is correctly listed: 43 | 44 | apply (Let (list_coq_files_ok _ (Packages.to_folders packages))). 45 | 46 | Since this scenario states that the result is `Some files`: 47 | 48 | Definition list_coq_files_ok (folder : LString.t) (files : list Name.t) 49 | : Run.t (list_coq_files folder) (Some files). 50 | 51 | we know that the function `get_packages` will call the function `get_packages_of_names`. We then apply its scenario: 52 | 53 | apply (get_packages_of_names_ok repository packages). 54 | 55 | Coq tells us there nothing more to do, so we conclude by: 56 | 57 | Defined. 58 | 59 | The Coq type-checker accepts our scenario which means it is valid: we do not need to run it on every `(repository, packages)` tuples. Even if we used the tactical mode to define the scenario, we did not write any proofs: the scenario is valid *by-construction*. However, in some cases, proofs are required to help the type-checker with non-trivial equalities. See for example the `list_coq_files_ok` scenario. Over 13 scenarios, only 2 required proofs to help the type-checker. 60 | 61 | ### What did we prove? 62 | We proved that, for any list of OPAM packages, given the "right answers" from the file system, the function `get_packages` will terminate without errors and will return `Some packages`. The "right answers" are defined by giving the "right answers" to `list_coq_files` and then giving the "right answers" to `get_packages_of_names`. 63 | 64 | ## Next time 65 | We have seen how to formally verify an interactive program in Coq. [Next time](http://coq-blog.clarus.me/concurrency-with-promises-in-coq.html) we will see how to optimize this program using concurrency. 66 | -------------------------------------------------------------------------------- /posts/2015-03-14 Concurrency with promises in Coq.md: -------------------------------------------------------------------------------- 1 | We will present two primitives to write concurrent (and interactive) programs in [Coq](http://coq.inria.fr/), using the concept of [promises](http://en.wikipedia.org/wiki/Futures_and_promises) in the library [Coq.io](http://coq.io/). We will also show how to formally specify programs written using promises. 2 | 3 | ## Primitives for concurrency 4 | To write an interactive application we quickly need a way to do non-blocking inputs--outputs operations. There are many approaches to do non-blocking operations or concurrency, for example: 5 | 6 | * blocking calls in system threads 7 | * a main event loop 8 | * callbacks in lightweight threads 9 | * [promises](http://en.wikipedia.org/wiki/Futures_and_promises) 10 | * the [actor model](http://en.wikipedia.org/wiki/Actor_model) 11 | 12 | We chose to use promises, in the style of the [Lwt library](http://ocsigen.org/lwt/) for [lightweight threads](http://en.wikipedia.org/wiki/Light-weight_process) in [OCaml](https://ocaml.org/). The promises can be implemented efficiently using lightweight threads, while being simpler to use than an event loop or callbacks. The actor model is another interesting approach which we will study later. 13 | 14 | ### Sequential computations 15 | We recall the definition of our interactive sequential computations (see the [coq-io](https://github.com/coq-io/io) package): 16 | 17 | Module C. 18 | Inductive t (E : Effects.t) : Type -> Type := 19 | | Ret : forall {A : Type} (x : A), t E A 20 | | Call : forall (command : Effects.command E), t E (Effects.answer E command) 21 | | Let : forall {A B : Type}, t E A -> (A -> t E B) -> t E B. 22 | End C. 23 | 24 | where the effects are defined as: 25 | 26 | Module Effects. 27 | Record t := New { 28 | command : Type; 29 | answer : command -> Type }. 30 | End Effects. 31 | 32 | The effects are a type of `command` and a type of `answer`, dependent on the values of the commands. A computation of type `C.t E A` is a computation returning a value of type `A` with the effects `E`. It can be of three forms: 33 | 34 | * `Ret x`, the pure value `x` 35 | * `Call command`, the call to a command 36 | * `Let x f`, the evaluation of `x`, followed by the evaluation of `f` applied to the result of `x` 37 | 38 | The `Let` operator is also called the *bind* in the terminology of [monads](http://en.wikipedia.org/wiki/Monad_%28functional_programming%29). The effects `E` typically represent external calls to the system. You can for example look at the [API](http://clarus.github.io/doc/io-system/Io.System.System.html) of the [coq-io-system](https://github.com/coq-io/system) library, which provides basic calls to manipulates files and the terminal. 39 | 40 | ### Join 41 | We add the `Join` operator to the computations: 42 | 43 | | Join : forall {A B : Type}, t E A -> t E B -> t E (A * B) 44 | 45 | The program `Join x y` runs the two computations `x` and `y` in parallel and returns the couple of their results. Since the two computations are launched concurrently, blocking calls made in `x` will not block calls in `y`, and reciprocally. This is the main operator we will use to write concurrent programs. 46 | 47 | Note that since our programs are pure, there are no shared states between `x` and `y`. We will see in a future post how to represent a shared state in a concurrent program in Coq. 48 | 49 | ### First 50 | The `First` operator is defined as: 51 | 52 | | First : forall {A B : Type}, t E A -> t E B -> t E (A + B) 53 | 54 | The program `First x y` runs the two computations `x` and `y` in parallel and returns the result of the first which terminated. The other may be canceled. 55 | 56 | This operator is dangerous because one of our computations can get canceled. The `First` primitive is mainly there to implement timeouts. If we want to run a computation `slow` which may not terminate or may take too much time, we can combine this computation with a timeout by writing something like: 57 | 58 | First slow (sleep 10) 59 | 60 | to make sure the program terminates after 10 seconds (we suppose that `sleep 10` is the computation which does nothing but terminating after 10 seconds). 61 | 62 | ## Specification of promises 63 | We specify our computations using [use-cases](http://en.wikipedia.org/wiki/Use_case) described by runs: 64 | 65 | Module Run. 66 | Inductive t : forall {E : Effects.t} {A : Type}, C.t E A -> A -> Type := 67 | | Ret : forall {E A} (x : A), t (C.Ret (E := E) x) x 68 | | Call : forall E (command : Effects.command E) (answer : Effects.answer E command), 69 | t (C.Call E command) answer 70 | | Let : forall {E A B} {c_x : C.t E B} {x : B} {c_f : B -> C.t E A} {y : A}, 71 | t c_x x -> t (c_f x) y -> t (C.Let c_x c_f) y. 72 | End Run. 73 | 74 | You can see an example of specification using runs in [Formally verify a script in Coq](http://coq-blog.clarus.me/formally-verify-a-script-in-coq.html). The main idea is that a universally quantified run can be viewed as the formal specification of a use-case. This kind of specification is verified just by typing, using lemma on equalities when the type-checker fails. 75 | 76 | A run can be a run of the computation: 77 | 78 | * `Ret x`, the pure value `x` returned by the computation 79 | * `Call command`, an answer to this command 80 | * `Let x f`, a run of `x` and a run of `f` applied to the result of the run of `x` 81 | 82 | We will extend our definition of runs with new cases for the primitives `Join` and `First`. 83 | 84 | ### Join 85 | We define the run of a `Join` by: 86 | 87 | | Join : forall {E A B} {c_x : C.t E A} {x : A} {c_y : C.t E B} {y : B}, 88 | t c_x x -> t c_y y -> t (C.Join c_x c_y) (x, y) 89 | 90 | This means that a run of `Join x y` is a couple of runs for `x` and `y`. Equivalently, a specification of `Join x y` is a couple of specifications for `x` and for `y`. 91 | 92 | *A priori*, the interactions of the threads `x` and `y` are not specified. This is up to the user to express these interactions in the specification, if there are some. 93 | 94 | ### First 95 | There are two ways to run a `First`: 96 | 97 | | Left : forall {E A B} {c_x : C.t E A} {x : A} {c_y : C.t E B}, 98 | t c_x x -> t (C.First c_x c_y) (inl x) 99 | | Right : forall {E A B} {c_x : C.t E A} {c_y : C.t E B} {y : B}, 100 | t c_y y -> t (C.First c_x c_y) (inr y) 101 | 102 | A run of `First x y` is either a run of `x` (the `Left` case) or a run of `y` (the `Right` case). Equivalently, a specification of `First x y` is a specification of `x` or a specification of `y`. 103 | 104 | We could ask: yes, but sometimes both `x` and `y` are actually executed! And if we choose the `Left` case for example, we should program `x` instead of `First x y` to start with. These are the same concerns we may have for the definition of the *logical disjunction* in Coq: 105 | 106 | Inductive or (A B : Prop) : Prop := 107 | | or_introl : A -> A \/ B 108 | | or_intror : B -> A \/ B 109 | 110 | where "A \/ B" := (or A B) : type_scope. 111 | 112 | and the reasons of these definitions are similar. 113 | 114 | ## Next time 115 | [Next time](http://coq-blog.clarus.me/implementation-of-promises-for-coq.html) we will see how to implement the primitives `Join` and `First`, to generate efficient and non-blocking interactive programs written in Coq. 116 | -------------------------------------------------------------------------------- /posts/2015-03-16 Implementation of promises for Coq.md: -------------------------------------------------------------------------------- 1 | [Last time](http://coq-blog.clarus.me/concurrency-with-promises-in-coq.html), we presented two primitives `Join` and `First` to write and specify concurrent programs with [Coq.io](http://coq.io/), using the concept of [promises](http://en.wikipedia.org/wiki/Futures_and_promises). We will explain how we implemented these primitives and give examples of concurrent programs. 2 | 3 | ## Implementation 4 | The promises are available in the package [coq-io](https://github.com/coq-io/io), starting from the version `2.1.0`. This package symbolically describes the primitive `Join` and `First`. They must be implemented for each set of effects. 5 | 6 | For example, in [coq-io-system](https://github.com/coq-io/system) (which defines effects to interact with the system), we compile all the external calls and the computation primitives to [Lwt](http://ocsigen.org/lwt/) in [coq-io/src/Extraction.v](https://github.com/coq-io/system/blob/master/src/Extraction.v): 7 | 8 | Fixpoint eval {A : Type} (x : C.t System.effects A) : Lwt.t A. 9 | destruct x as [A x | command | A B x f | A B x y | A B x y]. 10 | - exact (Lwt.ret x). 11 | - exact (eval_command command). 12 | - exact (Lwt.bind (eval _ x) (fun x => eval _ (f x))). 13 | - exact (Lwt.join (eval _ x) (eval _ y)). 14 | - exact ( 15 | Lwt.bind (Lwt.first (eval _ x) (eval _ y)) (fun s => 16 | Lwt.ret @@ Sum.to_coq s)). 17 | Defined. 18 | 19 | This code is written in proof mode, so that we do not care about the `match`/`with` parameters and compile with both Coq 8.4 and 8.5 patterns. The functions `join` and `first` are implemented in [coq-io-system-ocaml/ioSystem.ml](https://github.com/coq-io/system-ocaml/blob/master/ioSystem.ml) using Lwt primitives. The Lwt primitives needed some wrappers because their types are: 20 | 21 | Lwt.join : unit Lwt.t list -> unit Lwt.t 22 | Lwt.pick : 'a Lwt.t list -> 'a list Lwt.t 23 | 24 | where we needed: 25 | 26 | join : 'a Lwt.t -> 'b Lwt.t -> ('a * 'b) Lwt.t 27 | first : 'a Lwt.t -> 'b Lwt.t -> ('a, 'b) Sum.t Lwt.t 28 | 29 | ## Hello World 30 | You can see a simple example of concurrent program in [coq-hello-world](https://github.com/coq-io/hello-world): 31 | 32 | Definition concurrent_hello_world (argv : list LString.t) 33 | : C.t System.effects unit := 34 | let! _ : unit * unit := join 35 | (System.log (LString.s "Hello")) 36 | (System.log (LString.s "World")) in 37 | ret tt. 38 | 39 | You can compile this program by replacing the line: 40 | 41 | Definition main := Extraction.run hello_world. 42 | 43 | with: 44 | 45 | Definition main := Extraction.run concurrent_hello_world. 46 | 47 | and following the instructions given in the [README.md](https://github.com/coq-io/hello-world/blob/master/README.md). It will display either: 48 | 49 | Hello 50 | World 51 | 52 | or: 53 | 54 | World 55 | Hello 56 | 57 | The specification of this program stays that it does exactly one thing: displaying concurrently `Hello` and `World`: 58 | 59 | Definition concurrent_hello_world_ok (argv : list LString.t) 60 | : Run.t (concurrent_hello_world argv) tt. 61 | apply (Run.Let (Run.Join 62 | (Run.log_ok (LString.s "Hello")) 63 | (Run.log_ok (LString.s "World")))). 64 | apply Run.Ret. 65 | Defined. 66 | 67 | ## Larger example and drawbacks 68 | In the branch [#join](https://github.com/clarus/repos2web/tree/join) of [repos2web](https://github.com/clarus/repos2web), we modified our website generator for [OPAM](http://opam.ocamlpro.com/) repositories to read the packages descriptions concurrently. Our main modification is to use the `Join` operator in the recursive functions to make concurrent recursive calls: 69 | 70 | Fixpoint get_packages (repository : LString.t) (packages : Packages.t) 71 | : C FullPackages.t := 72 | match packages with 73 | | [] => ret [] 74 | | package :: packages => 75 | let! full_package_full_packages := join 76 | (get_package repository package) 77 | (get_packages repository packages) in 78 | let (full_package, full_packages) := full_package_full_packages in 79 | ret (full_package :: full_packages) 80 | end. 81 | 82 | The specifications are updated similarly. 83 | 84 | We tried to run this program on large generated repositories, to see if we gained performances. Unfortunately, after a couple of thousands of packages, we hit the limit of number of files we can open at the same time. We could have a finer control of the concurrency with, for example, a bounded pool of light-weight threads, or by handling the `EMFILE` exception ("too many open files") to retry again later. 85 | 86 | We feel that the specification of the Linux system API is quite complex, with many corner cases in practice (like this limit on the number of opened files). These corner cases are hard to be spotted by formal specifications, since the specification of Linux seems complex. Thus programming on top of Linux does not benefit as much as we could hope from formal specifications. 87 | 88 | Next, we would like to investigate more the [Xen](http://www.xenproject.org/) API which seems cleaner, completely event-driven and available in [OCaml](https://ocaml.org/) thanks to the [MirageOS](http://www.openmirage.org/) project. 89 | -------------------------------------------------------------------------------- /posts/2015-07-21 Launch of the Coq.io website.md: -------------------------------------------------------------------------------- 1 | [Coq.io](http://coq.io/) is a library for writing and proving concurrent applications with inputs--outputs in [Coq](https://coq.inria.fr/). In order to centralize the informations about it, I have setup a [website on http://coq.io/](http://coq.io/). I have been lucky to get this domain name, since this gives a direct reference to the [IO monad](https://wiki.haskell.org/IO_inside) of [Haskell](https://www.haskell.org/), which popularized the idea of clean imperative programming in a functional language. 2 | 3 | There is a [Getting started](http://coq.io/getting_started.html) page with some basic examples and an introduction to the technique of formal specification by use cases, and links to some reference documentation. The library is still evolving, with three upcoming improvements: 4 | 5 | * a type [`C.I.t`](https://github.com/coq-io/io/blob/master/src/C.v) for infinite (co-inductive) computations; 6 | * a type [`Trace.t`](https://github.com/coq-io/io/blob/master/src/Trace.v) for whose who want to separate the specifications from the proofs of use cases; 7 | * a [`Lwt.E`](https://github.com/coq-io/lwt) effect to add arbitrary [Lwt](http://ocsigen.org/lwt/) commands. 8 | -------------------------------------------------------------------------------- /posts/2015-07-23 Handle errors in Coq.md: -------------------------------------------------------------------------------- 1 | Many programming languages handle errors with an exceptions mechanism. There are no exceptions in [Coq](https://coq.inria.fr/), since this is a pure programming language. We mainly get two alternatives: 2 | 3 | 1. to extend Coq with an effect system for exceptions, implemented with monads or alike; 4 | 2. to use explicit sum types. 5 | 6 | Even if the first option seems more powerful, from my experience an effect system is too heavy for the gains it brings compared the use of sum types with combinators. Sum types are just *simple* and *ubiquitous*. This is in fact the way errors are handled in [Rust](http://blog.burntsushi.net/rust-error-handling/). 7 | 8 | There are two basic sum types: 9 | 10 | * `option A`: either `Some value` or `None` for an error 11 | * `A + E`: either `inl value` or `inr err` for an error 12 | 13 | ## The ErrorHandlers package 14 | Since there are no error combinators in the standard library, I wrote the [ErrorHandlers](https://github.com/clarus/coq-error-handlers) package: 15 | 16 | opam repo add coq-released https://coq.inria.fr/opam/released 17 | opam install -j4 coq-error-handlers 18 | 19 | It provides the basic combinators `bind`, `map` and `default`. The `bind` sequences two computations with errors: 20 | 21 | Require Import Coq.Lists.List. 22 | Require Import ErrorHandlers.All. 23 | 24 | Import ListNotations. 25 | 26 | Compute 27 | Option.bind (List.hd_error [2; 4; 5]) (fun n => 28 | Some (n + 1)). 29 | 30 | prints `Some 3`. 31 | 32 | The `map` is a particular case for when the second expression does not return errors: 33 | 34 | Compute Option.map (List.hd_error [2; 4; 5]) (fun n => n + 1). 35 | 36 | prints `Some 3`. 37 | 38 | The `default` replaces an error with a default value: 39 | 40 | Compute Option.default (Some 3) 0. 41 | Compute Option.default None 0. 42 | 43 | prints `3` and `0`. 44 | 45 | To summarize, here are the types of the combinators: 46 | 47 | * `Option.bind : forall {A B}, option A -> (A -> option B) -> option B` 48 | * `Option.map : forall {A B}, option A -> (A -> B) -> option B` 49 | * `Option.default : forall {A}, option A -> A -> A` 50 | * `Sum.bind : forall {E A B}, A + E -> (A -> B + E) -> B + E` 51 | * `Sum.map : forall {E A B}, A + E -> (A -> B) -> B + E` 52 | * `Sum.default : forall {E A}, A + E -> A -> A` 53 | 54 | There are no notations to keep things simple and non-intrusive. 55 | -------------------------------------------------------------------------------- /posts/2015-07-25 Simple unit testing in Coq.md: -------------------------------------------------------------------------------- 1 | Even if we can prove things in [Coq](http://coq.inria.fr/), [unit testing](https://en.wikipedia.org/wiki/Unit_testing) remains an extremely simple and powerful technique to verify what we write. Thanks to the dependent types, we can run unit tests in Coq at compile time by computing into the types. For example, in: 2 | 3 | Definition test_pred : pred 5 = 4 := 4 | eq_refl. 5 | 6 | we check that the predecessor of `5` is `4` by saying that they are logically equal. The proof of the equality is its only constructor `eq_refl`. We force the type-checker of Coq to reduce both `pred 5` and `4` to check that they are the same. 7 | 8 | Usually we want to check many values, what we can do with a list: 9 | 10 | Require Import Coq.Lists.List. 11 | 12 | Import ListNotations. 13 | 14 | Definition test_pred : 15 | List.map pred [0; 1; 2; 5] = [0; 0; 1; 4] := 16 | eq_refl. 17 | 18 | We map the `pred` function on the values `0`, `1`, `2`, `5` and check the results. 19 | 20 | ## The CUnit package 21 | For functions with several parameters, the trick of the `List.map` function does not work well because of the [currying](https://en.wikipedia.org/wiki/Currying). What need instead is a map over lists of tuples. I made the package [CUnit](https://github.com/clarus/coq-cunit) to provide such a map. Install it with [OPAM](use-opam-for-coq.html): 22 | 23 | opam repo add coq-released https://coq.inria.fr/opam/released 24 | opam install coq-cunit 25 | 26 | Now we can test the `plus` function: 27 | 28 | Require Import Coq.Lists.List. 29 | Require Import CUnit.All. 30 | 31 | Import ListNotations. 32 | 33 | Definition test_plus : List.map_pair plus 34 | [(0, 0); (0, 3); (4, 0); (4, 3)] = 35 | [0; 3; 4; 7] := 36 | eq_refl. 37 | 38 | The complete list of maps in the package [CUnit](https://github.com/clarus/coq-cunit) is the following: 39 | 40 | * `List.map_pair {A B C} (f : A -> B -> C) (l : list (A * B)) : list C` 41 | * `List.map_triple {A B C D} (f : A -> B -> C -> D) (l : list (A * B * C)) : list D` 42 | * `List.map_quad {A B C D E} (f : A -> B -> C -> D -> E) (l : list (A * B * C * D)) : list E` 43 | 44 | *Edit: this list of functions can be generalized to an arbitrary number of arguments. See the post [Currying using Canonical Structures](http://gallais.github.io/blog/canonical-structures-currying.html) of [Guillaume Allais](http://gallais.github.io/).* 45 | -------------------------------------------------------------------------------- /posts/2018-07-24 Approximating GADTs in Flow.md: -------------------------------------------------------------------------------- 1 | The [GADTs (Generalized algebraic data types)](https://en.wikipedia.org/wiki/Generalized_algebraic_data_type#Higher-order_abstract_syntax) are a generalization of sum types where the type parameter can change in each case. GADTs are not available in [Flow](https://flow.org/) but we show a technique to approximate them. We take inspiration from [Approximating GADTs in PureScript](http://code.slipthrough.net/2016/08/10/approximating-gadts-in-purescript/). 2 | 3 | ## Use case 4 | Let us say we want to implement an evaluator for simple arithmetic expressions. We define the type of expressions: 5 | 6 | type Expr = { 7 | type: 'I', 8 | value: number 9 | } | { 10 | type: 'B', 11 | value: boolean 12 | } | { 13 | type: 'Add', 14 | x: Expr, 15 | y: Expr 16 | } | { 17 | type: 'Mul', 18 | x: Expr, 19 | y: Expr 20 | } | { 21 | type: 'Eq', 22 | x: Expr, 23 | y: Expr 24 | }; 25 | 26 | and an evaluation function: 27 | 28 | function evaluate(expr: Expr): boolean | number { 29 | switch (expr.type) { 30 | case 'I': 31 | return expr.value; 32 | case 'B': 33 | return expr.value; 34 | case 'Add': 35 | return evaluate(expr.x) + evaluate(expr.y); 36 | case 'Mul': 37 | return evaluate(expr.x) * evaluate(expr.y); 38 | case 'Eq': 39 | return evaluate(expr.x) === evaluate(expr.y); 40 | default: 41 | return expr; 42 | } 43 | } 44 | 45 | but we get a Flow error: 46 | 47 | 32: return evaluate(expr.x) * evaluate(expr.y); 48 | ^ Cannot perform arithmetic operation because boolean [1] is not a number. 49 | References: 50 | 23: function evaluate(expr: Expr): boolean | number { 51 | ^ [1] 52 | 53 | This error occurs because Flow does not know if we multiply booleans or numbers since the return type of `evaluate` is `boolean | number`. Flow is right to warn us because the type `Expr` does not prevent to write expressions multiplying booleans: 54 | 55 | const foo: Expr = { 56 | type: 'Mul', 57 | x: {type: 'B', value: true}, 58 | y: {type: 'B', value: false} 59 | }; 60 | 61 | To force the multiplication to be over numeric expressions, we would like to distinguish expressions which evaluate to a `number` from expressions which evaluate to a `boolean`. 62 | 63 | ## GADTs to the rescue 64 | GADTs allow to parametrize the expression type `Expr` by the return type of the evaluation (either `boolean` or `number`). In a language with GADTs like [Haskell](https://www.haskell.org/) we can define a type `Expr a` (with `a` the type of the evaluation result) as follows: 65 | 66 | -- In Haskell: 67 | data Expr a where 68 | B :: Bool -> Expr Bool 69 | I :: Int -> Expr Int 70 | Add :: Expr Int -> Expr Int -> Expr Int 71 | Mult :: Expr Int -> Expr Int -> Expr Int 72 | Eq :: Expr Int -> Expr Int -> Expr Bool 73 | 74 | evaluate :: Expr a -> a 75 | evaluate expr = ... 76 | 77 | Unfortunately, Flow has no syntax to express this kind of types: 78 | 79 | type Expr = { 80 | type: 'I', // Where do we say this is Expr? 81 | value: number 82 | } | { 83 | type: 'B', // Where do we say this is Expr? 84 | value: boolean 85 | } | { 86 | ... 87 | 88 | ## Encoding in Flow 89 | We use a trick to encode GATDs in Flow. We express that in the case `type: 'I'` of an expression of type `Expr` the type `A` is actually a `number` by adding an _equality witness_ `_eq` between `A` and `number`: 90 | 91 | type Expr = { 92 | type: 'I', 93 | _eq: number => A, 94 | value: number 95 | } | { 96 | ... 97 | 98 | To make the equality witness `_eq` valid, we must define it with the identity function: 99 | 100 | function i(value: number): Expr { 101 | return {type: 'I', _eq: x => x, value}; 102 | } 103 | 104 | The full defintion: 105 | 106 | type Expr = { 107 | type: 'I', 108 | _eq: number => A, 109 | value: number 110 | } | { 111 | type: 'B', 112 | _eq: boolean => A, 113 | value: boolean 114 | } | { 115 | type: 'Add', 116 | _eq: number => A, 117 | x: Expr, 118 | y: Expr 119 | } | { 120 | type: 'Mul', 121 | _eq: number => A, 122 | x: Expr, 123 | y: Expr 124 | } | { 125 | type: 'Eq', 126 | _eq: boolean => A, 127 | x: Expr, 128 | y: Expr 129 | }; 130 | 131 | and all the wrappers: 132 | 133 | function i(value: number): Expr { 134 | return {type: 'I', _eq: x => x, value}; 135 | } 136 | 137 | function b(value: boolean): Expr { 138 | return {type: 'B', _eq: x => x, value}; 139 | } 140 | 141 | function add(x: Expr, y: Expr): Expr { 142 | return {type: 'Add', _eq: x => x, x, y}; 143 | } 144 | 145 | function mul(x: Expr, y: Expr): Expr { 146 | return {type: 'Mul', _eq: x => x, x, y}; 147 | } 148 | 149 | function eq(x: Expr, y: Expr): Expr { 150 | return {type: 'Eq', _eq: x => x, x, y}; 151 | } 152 | 153 | We can then define a well-typed `evaluate` function by applying the `_eq` witness to use the type equalities when needed: 154 | 155 | function evaluate(expr: Expr): A { 156 | switch (expr.type) { 157 | case 'I': 158 | return expr._eq(expr.value); 159 | case 'B': 160 | return expr._eq(expr.value); 161 | case 'Add': 162 | return expr._eq(evaluate(expr.x) + evaluate(expr.y)); 163 | case 'Mul': 164 | return expr._eq(evaluate(expr.x) * evaluate(expr.y)); 165 | case 'Eq': 166 | return expr._eq(evaluate(expr.x) === evaluate(expr.y)); 167 | default: 168 | return expr; 169 | } 170 | } 171 | 172 | const e1: Expr = add(i(2), i(4)); 173 | const e2: Expr = mul(i(2), i(3)); 174 | const e3: Expr = eq(e1, e2); 175 | console.log(evaluate(e1), evaluate(e2), evaluate(e3)); 176 | 177 | Note that `_eq` has no runtime value, is only here for typing and could be eliminated given a clever enough compiler / interpreter. 178 | 179 | ## Robustness 180 | We test the effectiveness of this encoding by introducing some errors in our code. In the definition of expressions: 181 | 182 | * mistake: 183 | 184 | const e1: Expr = add(i(2), i(4)); 185 | 186 | * Flow output: 187 | 188 | const e1: Expr = add(i(2), i(4)); 189 | ^ Cannot assign `add(...)` to `e1` because number [1] is incompatible with boolean [2] in type argument `A` [3]. 190 | 191 | In the definition of `evaluate`, if we return a `boolean` where we expect a `number`: 192 | 193 | * mistake: 194 | 195 | case 'Mul': 196 | return expr._eq(evaluate(expr.x) === evaluate(expr.y)); 197 | 198 | * Flow output: 199 | 200 | return expr._eq(evaluate(expr.x) === evaluate(expr.y)); 201 | ^ Cannot call `expr._eq` with `evaluate(...) === evaluate(...)` bound to the first parameter because boolean [1] is incompatible with number [2]. 202 | 203 | If we do not use `_eq`: 204 | 205 | * mistake: 206 | 207 | case 'Mul': 208 | return evaluate(expr.x) * evaluate(expr.y); 209 | 210 | * Flow output: 211 | 212 | return evaluate(expr.x) * evaluate(expr.y); 213 | ^ Cannot return `evaluate(...) * evaluate(...)` because number [1] is incompatible with `A` [2]. 214 | 215 | A weakness of this encoding is that we must enforce by hand that `_eq` is always `x => x`. 216 | 217 | ## Related 218 | The idea of using type equalities is taken from [Approximating GADTs in PureScript](http://code.slipthrough.net/2016/08/10/approximating-gadts-in-purescript/). In [PureScript](http://www.purescript.org/) the type system almost enforces that `_eq` is the identity (it could also be a non-terminating function). There is a [thread](https://github.com/facebook/flow/issues/1356) on GitHub issues discussing the addition of GADTs to Flow. 219 | -------------------------------------------------------------------------------- /posts/2019-07-25 Continuous testing for Coq projects.md: -------------------------------------------------------------------------------- 1 | Testing that a [Coq](https://coq.inria.fr/) project works with different Coq versions may be time consuming. This is necessary to deliver with confidence new project releases. [Travis CI](https://travis-ci.com/) is a service to run tests on [GitHub](https://github.com/) projects and is free for open-source code. We show a setup to run automated testing with Travis CI using pre-compiled Coq instances and [opam](https://opam.ocaml.org/). Thanks to that setup, we can automatically and quickly check new pull-requests and commits. 2 | 3 | This configuration is directly inspired by the [CI setup](https://github.com/coq-community/docker-coq/wiki/CI-setup) documentation written by [Erik Martin-Dorel](https://github.com/erikmd). 4 | 5 | ![Travis CI report on a pull-request](static/images/travis-ci/build-report.png "Travis CI report on a pull-request") 6 | 7 | ## Setting up Travis CI 8 | To show the setup [Travis CI](https://travis-ci.com/), we take the example of the [github.com/coq-io/system](https://github.com/coq-io/system) project. We need to create two files at the root of the project: 9 | 10 | * `coq-io-system.opam` to describe the dependencies; 11 | * `.travis.yml` to configure and activate Travis CI. 12 | 13 | ## Describing the dependencies 14 | We create an [opam](https://opam.ocaml.org/) file `coq-io-system.opam` with the following content: 15 | 16 | version: "dev" 17 | 18 | opam-version: "2.0" 19 | maintainer: "dev@clarus.me" 20 | homepage: "https://github.com/clarus/io-system" 21 | dev-repo: "git+https://github.com/clarus/io-system.git" 22 | bug-reports: "https://github.com/clarus/io-system/issues" 23 | authors: ["Guillaume Claret"] 24 | license: "MIT" 25 | build: [ 26 | ["./configure.sh"] 27 | [make "-j%{jobs}%"] 28 | ] 29 | install: [ 30 | [make "install"] 31 | ] 32 | depends: [ 33 | "ocaml" 34 | "coq" {>= "8.5"} 35 | "coq-function-ninjas" 36 | "coq-list-string" {>= "2.0.0"} 37 | "coq-io" {>= "4.0.0"} 38 | "coq-io-system-ocaml" {>= "2.3.0"} 39 | ] 40 | tags: [ 41 | "keyword:effects" 42 | "keyword:extraction" 43 | "logpath:Io/System" 44 | ] 45 | synopsis: "System effects for Coq" 46 | 47 | This file describes the dependencies of the project. It can list any other opam packages published on the [OCaml repository](https://opam.ocaml.org/) or the [Coq repository](https://github.com/coq/opam-coq-archive). We also give the build and install commands. To check that this file is correct: 48 | 49 | # linting to check for typos 50 | opam lint coq-io-system.opam 51 | # adding the Coq repository if not already done 52 | opam repo add coq-released https://coq.inria.fr/opam/released 53 | # installing the package 54 | opam pin add coq-io-system . --kind=path -y 55 | 56 | Later on, you may want to publish your package in opam. You can just take this file content and create a pull-request on the [Coq repository](https://github.com/coq/opam-coq-archive). Since this opam file is used for continuous testing, you can be pretty confident that your package is correct. 57 | 58 | ## Configuring Travis CI 59 | We add a [Travis CI](https://travis-ci.com/) file `.travis.yml` with the following content: 60 | 61 | dist: bionic 62 | language: generic 63 | 64 | services: 65 | - docker 66 | 67 | env: 68 | global: 69 | - PACKAGE_NAME="coq-io-system" 70 | matrix: 71 | - COQ_IMAGE="coqorg/coq:8.4" SHOULD_SUPPORT="false" 72 | - COQ_IMAGE="coqorg/coq:8.5" SHOULD_SUPPORT="true" 73 | - COQ_IMAGE="coqorg/coq:8.6" SHOULD_SUPPORT="true" 74 | - COQ_IMAGE="coqorg/coq:8.7" SHOULD_SUPPORT="true" 75 | - COQ_IMAGE="coqorg/coq:8.8" SHOULD_SUPPORT="true" 76 | - COQ_IMAGE="coqorg/coq:8.9" SHOULD_SUPPORT="true" 77 | - COQ_IMAGE="coqorg/coq:8.10" SHOULD_SUPPORT="true" 78 | 79 | install: | 80 | # Prepare the COQ container 81 | docker pull ${COQ_IMAGE} 82 | docker run -d -i --init --name=COQ -v ${TRAVIS_BUILD_DIR}:/home/project -w /home/project ${COQ_IMAGE} 83 | docker exec COQ /bin/bash --login -c " 84 | # This bash script is double-quoted to interpolate Travis CI env vars: 85 | echo \"Build triggered by ${TRAVIS_EVENT_TYPE}\" 86 | export PS4='+ \e[33;1m(\$0 @ line \$LINENO) \$\e[0m ' 87 | set -ex # -e = exit on failure; -x = trace for debug 88 | opam update 89 | opam pin add ${PACKAGE_NAME} . --kind=path --no-action -y 90 | opam config list; opam repo list; opam pin list; opam list 91 | " install 92 | 93 | script: 94 | - echo -e "${ANSI_YELLOW}Building...${ANSI_RESET}" && echo -en 'travis_fold:start:script\\r' 95 | - | 96 | docker exec COQ /bin/bash --login -c " 97 | export PS4='+ \e[33;1m(\$0 @ line \$LINENO) \$\e[0m ' 98 | set -ex 99 | sudo chown -R coq:coq /home/project 100 | # Check if the package is compatible with the current environment 101 | if [ "${SHOULD_SUPPORT}" = "true" ] || opam install ${PACKAGE_NAME} --show-action -y; then 102 | # First install the dependencies 103 | opam install ${PACKAGE_NAME} --deps-only -y 104 | opam list 105 | # Then install the package itself in verbose mode 106 | opam install ${PACKAGE_NAME} -v -y 107 | fi; 108 | " script 109 | - echo -en 'travis_fold:end:script\\r' 110 | 111 | after_script: 112 | - docker stop COQ # optional 113 | 114 | This YAML file explains to Travis CI what to do to check the project. We use the [coqorg/coq](https://hub.docker.com/r/coqorg/coq) Docker images. These images have pre-compiled versions of Coq with opam, what is crucial to speedup the tests. For each architecture from Coq `8.4` to Coq `8.9` we: 115 | 116 | * check if either: 117 | * the platform is supposed to be supported `if [ "${SHOULD_SUPPORT}" = "true" ]`, 118 | * or if opam considers the platform as compatible `opam install ${PACKAGE_NAME} --show-action -y`; 119 | * install the dependencies `opam install ${PACKAGE_NAME} --deps-only -y`; 120 | * install the project in verbose mode `opam install ${PACKAGE_NAME} -v -y`. 121 | 122 | ## Using Travis CI 123 | You may need to activate Travis CI for your project in the [settings page of Travis CI](https://travis-ci.com/account/repositories). Then for each pull-request or commit you will see a green or red mark precising if the project passed the tests, along with detailed logs when you click on it: 124 | 125 | ![Commits in a pull-request](static/images/travis-ci/pull-request.png "Commits in a pull-request") 126 | 127 | ![Error logs](static/images/travis-ci/error-logs.png "Error logs") 128 | 129 | Thanks for reading! 130 | -------------------------------------------------------------------------------- /posts/2019-09-03 Connecting the opam bench to Gitter.md: -------------------------------------------------------------------------------- 1 | The [Coq opam bench system](https://coq-bench.github.io/) detects bugs in [opam](https://opam.ocaml.org/) packages from the [Coq repository](https://github.com/coq/opam-coq-archive). However these bugs may be hard to follow. Indeed, they are in a table one needs to consult regularly. We present a system to post results in a [Gitter](https://gitter.im/) channel [coq/opam-bench-reports](https://gitter.im/coq/opam-bench-reports). Thanks to this channel, we can more easily track down the latest build failures. 2 | 3 | The idea of publishing results in Gitter was given by [Emilio Jesús Gallego Arias](https://www.cri.ensmp.fr/people/gallego/) in [coq/issues/10418](https://github.com/coq/coq/issues/10418). Improvements were added from suggestions from Emilio Jesús Gallego Arias and [Karl Palmskog](https://setoid.com/). 4 | 5 | ![Bench report in Gitter](static/images/opam-bench-gitter/report.png "Bench report in Gitter") 6 | 7 | ## Implementation 8 | The code to publish results in Gitter is at [github.com/coq-bench/make-html](https://github.com/coq-bench/make-html). The implementation is in [Ruby](https://www.ruby-lang.org/) and shares code with the bench website generator. We extract the results from a CSV database generated by the project [github.com/coq-bench/run](https://github.com/coq-bench/run) which tests the packages. 9 | 10 | To connect to Gitter, we use the [ruby-gitter](https://github.com/kristenmills/ruby-gitter) package. The code goes as follows: 11 | 12 | # Use the developer token from the Gitter documentation 13 | # (what is possible as very few messages are sent) 14 | client = Gitter::Client.new(token) 15 | 16 | # Find the room id 17 | room = client.rooms.find {|room| room.name == "coq/opam-bench-reports"} 18 | room_id = room.id 19 | 20 | # Construct the message with the results 21 | message = "..." 22 | 23 | # Actually send the message 24 | client.send_message(message, room_id) 25 | 26 | We use a dedicated [coq-opam-bench-gitter-bot](https://github.com/coq-opam-bench-gitter-bot) user to publish on Gitter and sign the messages. We retrieve the results of the past 72 hours, and run the code every 72 hours with a shell command: 27 | 28 | while true; do sleep 72h; date; git pull; ruby push_to_gitter.rb ../database gitter-token coq/opam-bench-reports 72; done 29 | 30 | Note the `git pull` of hot code swapping! This is useful to integrate small bench updates without breaking the 72 hours rythm. 31 | 32 | We either display the list of packages which failed, or a success message. 33 | 34 | ## Results 35 | We went from dozens of errors every 72 hours (when setting up the system) to just a few these days. Probably, this was mainly due to the bug corrections themselves but the Gitter reports helped. The Gitter reports also created a channel to talk about opam bugs. 36 | -------------------------------------------------------------------------------- /posts/2019-09-04 Fixing flaky makefiles in opam Coq packages.md: -------------------------------------------------------------------------------- 1 | In [opam](https://opam.ocaml.org/), some packages fail to install _sometimes_. This is a problem because these bugs are hard to reproduce. Yet the user may get the impression that some packages are broken. We show how we listed and then corrected these bugs. The [opam Coq repository](https://github.com/coq/opam-coq-archive) should now be more stable. 2 | 3 | ## Round 1 4 | At first we (I) did not know how to fix these installation bugs. They would typically result in OCaml errors such as: 5 | 6 | The following actions will be performed: 7 | - install coq-rational 8.6.0 8 | [...] 9 | - File "Rewrite/LeibnizRewrite/AC/aC.ml", line 1: 10 | - Error: Corrupted compiled interface 11 | 12 | in `coq-rational.8.6.0`. As we ran opam with the option `-j1` (one parallel build), this did not seem like a concurrency bug. We decided to add a [black-list file](https://github.com/coq-bench/make-html/blob/master/black_list.rb) to the [Coq opam bench](https://coq-bench.github.io/) to list out and hide these bugs. This was not a perfect solution, but at least it did not flood the results with non-reproducible errors. 13 | 14 | ## Round 2 15 | As noticed by [Karl Palmskog](https://setoid.com/), the opam builds were running in parallel despite the `-j1` option. As a proof, the presence in the installation traces of: 16 | 17 | make[2]: *** Waiting for unfinished jobs.... 18 | 19 | This was due to a bug in opam, [corrected in version `2.0.5`](https://github.com/ocaml/opam/blob/2.0.5/CHANGES#L10). Thus the fix was just to replace: 20 | 21 | [make "-j%{jobs}%"] 22 | 23 | by: 24 | 25 | [make] 26 | 27 | in each package definition with parallel build issues. 28 | 29 | These errors occurred in most of the packages using [coq_makefile](https://coq.inria.fr/refman/practical-tools/utilities.html#building-a-coq-project-with-coq-makefile) and mixing Coq and OCaml code. The versions `4.02` and `4.05` of OCaml were impacted, but the version `4.07` seems free of bugs. The most frequent error message is `Corrupted compiled interface`. Apparently this happens when both `byte` and `opt` compilations run in parallel, modifying `.cmi` files at the same time. The [Dune](https://dune.build/) build system may be a more robust alternative to `coq_makefile`, but I have no data about it. 30 | 31 | ## Result 32 | Most of the Coq packages with flaky makefiles are now corrected, and the respective maintainers contacted. We will continue to fix these bugs as they occur. 33 | -------------------------------------------------------------------------------- /posts/2019-09-13 Multiple error messages in coq-of-ocaml.md: -------------------------------------------------------------------------------- 1 | The [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml) compiler transforms [OCaml](https://ocaml.org/) programs to [Coq](https://coq.inria.fr/) ones. There usually are many errors in each file to import (the Coq language tends to be stricter than OCaml and we do not want to import code with too much encoding). Fixing errors may be the most time consuming part of an import. We present a system to display all the errors at once instead of one by one. We believe that it helps to get a quick idea of how difficult a file is to translate, while having a sense of progress when fixing the errors. 2 | 3 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 4 | 5 | ![Multiple errors report](static/images/coq-of-ocaml-multiple-errors/report.png "Multiple errors report") 6 | 7 | ## Example 8 | Take this perfectly valid OCaml program: 9 | 10 | let foo x = 11 | if x then 12 | assert false (* no assert in Coq *) 13 | else begin 14 | print_endline "bar"; (* no sequencing of side-effects in Coq *) 15 | true 16 | end 17 | 18 | We would like to report that both the `assert` and the sequencing of effects with `;` are not available in Coq. 19 | 20 | To import programs, `coq-of-ocaml` runs a single pass on the OCaml's typed abstract syntax tree. During the recursive exploration of the syntax tree, we would like to explore both branches of the `if` to report both errors. We need a way to accumulate errors along the way and not block at the first mistake. Since `coq-of-ocaml` has a single pass, we hope to get most of the errors doing so. 21 | 22 | ## An error monad 23 | To encapsulate the error handling mechanism, we define the following [free monad](https://stackoverflow.com/a/13388966/3873794) in OCaml (`coq-of-ocaml` is implemented in OCaml): 24 | 25 | module Command = struct 26 | type 'a t = 27 | | GetEnv : Env.t t 28 | | Raise : Error.Category.t * string -> 'a t 29 | end 30 | 31 | module Wrapper = struct 32 | type t = 33 | | SetEnv of Env.t 34 | | SetLoc of Loc.t 35 | end 36 | 37 | type 'a t = 38 | | All : 'a t * 'b t -> ('a * 'b) t 39 | | Bind : 'b t * ('b -> 'a t) -> 'a t 40 | | Command of 'a Command.t 41 | | Return of 'a 42 | | Wrapper of Wrapper.t * 'a t 43 | 44 | There are three main constructs to note: 45 | 46 | * `All` which combines the results of two computations, both of which may fail. It allows to accumulate errors in each branch of the syntax tree; 47 | * `Command`, especially the case `Raise` to create an error at the current code location; 48 | * `Wrapper`, especially the case `SetLoc` to set the current code location in (and only in) the following computation. 49 | 50 | We also have primitives `GetEnv` and `SetEnv` to manipulate the current OCaml environment from the AST. The `Bind` and `Return` are the standard monadic primitives. We chose a free-monad in order to isolate the definition of side-effects and, one day, import `coq-of-ocaml` to Coq. 51 | 52 | We rewrote `coq-of-ocaml` using the `All` primitive as much as possible (typically when two computations were not depending on each other). Since the current OCaml environment and location are handled by the monad, we cleaned the code to propagate their values. Having done that, retrieving the complete list of errors was mostly given for free. 53 | 54 | ## A nice output 55 | In order to present a nice output to the user, we decided to do the following: 56 | 57 | * add colors; 58 | * show the code extract where the error comes from, using a presentation similar to the JavaScript's [@babel/code-frame](https://babeljs.io/docs/en/next/babel-code-frame.html) (for now an error location is a line; we plan to precise the column and add multiline support); 59 | * clearly separate errors with a long line and some spaces; 60 | * add an error category (side-effect, dependency not found, ...). 61 | 62 | Some of these changes were inspired by the [Elm](https://elm-lang.org/) blog post on producing [Compiler Errors for Humans](https://elm-lang.org/news/compiler-errors-for-humans). 63 | -------------------------------------------------------------------------------- /posts/2019-09-28 Importing mutually recursive types from OCaml to Coq.md: -------------------------------------------------------------------------------- 1 | In order to make [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml) usable on a maximum of [OCaml](https://ocaml.org/) programs, we should handle mutually recursive types. We show how we import these types to [Coq](https://coq.inria.fr/) and the main differences between the two languages. As a result, more OCaml programming patterns should be supported by coq-of-ocaml. 2 | 3 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 4 | 5 | ## Example 6 | Take the following mutually recursive definition of a tree in OCaml: 7 | 8 | type 'a tree = Tree of 'a node list 9 | 10 | and 'a node = 11 | | Leaf of 'a leaf 12 | | Node of 'a tree 13 | 14 | and 'content leaf = string * 'content 15 | 16 | By applying [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml) on this code we get: 17 | 18 | Reserved Notation "'leaf". 19 | 20 | Inductive tree (a : Type) : Type := 21 | | Tree : (list (node a)) -> tree a 22 | 23 | with node (a : Type) : Type := 24 | | Leaf : ('leaf a) -> node a 25 | | Node : (tree a) -> node a 26 | 27 | where "'leaf" := (fun (content : Type) => string * content). 28 | 29 | Definition leaf := 'leaf. 30 | 31 | Arguments Tree {_}. 32 | Arguments Leaf {_}. 33 | Arguments Node {_}. 34 | 35 | We transform algebraic data types to Coq's inductive types. We use a notation to implement the type synonym `leaf`. This is a trick because Coq only supports inductive types in mutual definitions. See the [documentation on notations](https://coq.inria.fr/refman/user-extensions/syntax-extensions.html?highlight=notation#reserving-notations) for more information. We rename the `leaf` type parameter `'content` instead of `'a` to avoid a name collision in Coq. The type parameters of the constructors are set implicit with the command `Arguments` to keep the behavior of OCaml. 36 | 37 | ## General mechanism 38 | We handle `type ... and ...` definitions with algebraic data types (including GADTs, "Generalized Algebraic Data Types") and type synonyms. We do not handle abstract types. For records in mutual definitions, one can use a type synonym to a more generic record. For example: 39 | 40 | type expression = 41 | | Number of int 42 | | Operation of operation 43 | 44 | and operation = { 45 | name : string; 46 | parameters : expression list } 47 | 48 | can be rewritten as: 49 | 50 | type 'a operation_skeleton = { 51 | name : string; 52 | parameters : 'a list } 53 | 54 | type expression = 55 | | Number of int 56 | | Operation of operation 57 | 58 | and operation = expression operation_skeleton 59 | 60 | in order to be imported into Coq. 61 | 62 | In Coq, there is a distinction between type variables on the left (type parameters) and on the right of the `:` (type indices): 63 | 64 | Inductive t (A1 A2 ... : Type) : forall (B1 B2 ... : Type), Type := 65 | | Constr1 : ... -> t A1 A2 ... C1 C2 ... 66 | | ... 67 | 68 | The variables `Ai` do not behave the same as the variables `Bi`. Type parameters have the constraint of being the same for each constructor. When used, they simplify the typing of the pattern matching. Typically, in GADTs, type variables are type indices while in non-generalized algebraic data types they are type parameters. In mutually recursive inductive types there is one more constraint: the type parameters must be the same for each type. We consider a type variable to be a parameter if it appears with the same name in each OCaml constructor and in the type definition. For example: 69 | 70 | type ('a, 'b) arith = 71 | | Int : 'a * int -> ('a, int) arith 72 | | Eq : 'a * ('a, int) arith * ('a, int) arith -> ('a, bool) arith 73 | | Plus : 'a * ('a, int) arith * ('a, int) arith -> ('a, int) arith 74 | 75 | is imported to: 76 | 77 | Inductive arith (a : Type) : forall (b : Type), Type := 78 | | Int : a -> Z -> arith a Z 79 | | Eq : a -> (arith a Z) -> (arith a Z) -> arith a bool 80 | | Plus : a -> (arith a Z) -> (arith a Z) -> arith a Z. 81 | 82 | Arguments Int {_}. 83 | Arguments Eq {_}. 84 | Arguments Plus {_}. 85 | 86 | Here we transform `'a` to a type parameter and `'b` to a type index. 87 | 88 | ## Limitations 89 | Many OCaml programs define all types as mutually recursive by default. In Coq this is usually very difficult as: 90 | 91 | * only algebraic and synonym types can be mutual; 92 | * all type parameters must be the same; 93 | * the "strictly positive" constraint in Coq prevents some constructions (such as having a type as a type parameter for another); 94 | * the proofs or the definition of recursive functions on mutually recursive types is more complicated than with simple recursive types. 95 | 96 | In practice, I would recommend to find a way to avoid mutually recursive types when possible. For cases they are used, I hope this import mechanism to be useful. 97 | -------------------------------------------------------------------------------- /posts/2019-11-04 First-class modules in coq-of-ocaml.md: -------------------------------------------------------------------------------- 1 | For now, [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml) only supports plain modules used as namespaces (no functors). [First-class modules](https://caml.inria.fr/pub/docs/manual-ocaml/manual028.html) are an important construction to abstract code in [OCaml](https://ocaml.org/) as there are flexible and used heavily in some programs (including in [Tezos](https://tezos.com/)). We present our strategy to import first-class modules to dependent record in [Coq](https://coq.inria.fr/). We show that it works for the set example extracted from the Tezos source code. 2 | 3 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 4 | 5 | ## Strategy 6 | Here is an example of a first-class module in OCaml. We say that a type `t` is `Printable` when it provides a function `to_string` to convert it in `string`: 7 | 8 | module type Printable = sig 9 | type t 10 | val to_string : t -> string 11 | end 12 | 13 | Given a printable type, we can write a generic function to print it on the terminal: 14 | 15 | let print_on_the_terminal 16 | (type a) 17 | (module PrintableInstance : Printable with type t = a) 18 | (x : a) 19 | : unit = 20 | print_endline (PrintableInstance.to_string x) 21 | 22 | The module `Printable` encapsulates both a type `t` and some associated data `to_string`. We model that with a dependent record in Coq, that is to say a record mixing types and values: 23 | 24 | Module Printable. 25 | Record signature {t : Type} := { 26 | t := t; 27 | to_string : t -> string; 28 | }. 29 | Arguments signature : clear implicits. 30 | End Printable. 31 | 32 | Since the type `t` is not known at this point, we model it with a type parameter `t`. Later on, when someone uses the syntax `with type t = ...`, we call the `signature` with the value for `t`. We define a synonym field  `t := t` to have a uniform way to access to the fields of the module. We put the whole record into a module `Printable` to namespace the projections and prevent name collisions. 33 | 34 | We generate the following Coq code for the `print_on_the_terminal` function: 35 | 36 | Definition print_on_the_terminal {A : Type} 37 | (PrintableInstance : {_ : unit & Printable.signature A}) : A -> unit := 38 | let PrintableInstance := projT2 PrintableInstance in 39 | fun x => 40 | print_endline (PrintableInstance.(Printable.to_string) x). 41 | 42 | The `print_endline` function is an axiom, as side-effects are forbidden in Coq. The curly braces `{}` set implicit the type parameter `A`, because type variables are implicit in OCaml. The `PrintableInstance` value is a dependent pair, since it may contain a list of type values for the abstract types of the module. Here the only abstract type `t` is already filled with a value `A`, thus there are no type values (`_ : unit`). We open the dependent pair with `projT2`. We use the projection `Printable.to_string` to access to the `to_string` function of the module. 43 | 44 | We often use types with a first-class module in a boxed form, where we associate both a module type and its value: 45 | 46 | module type BoxedPrintable = sig 47 | module Printable : Printable 48 | val value : Printable.t 49 | end 50 | 51 | let print_boxed_printable (module BoxedPrintable : BoxedPrintable) : unit = 52 | print_endline (BoxedPrintable.Printable.to_string BoxedPrintable.value) 53 | 54 | In Coq, we proceed as for the previous module and propagate the abstract type `t` of the sub-module `Printable` to the signature of `BoxedPrintable`: 55 | 56 | Module BoxedPrintable. 57 | Record signature {Printable_t : Type} := { 58 | Printable : Printable.signature Printable_t; 59 | value : Printable.(Printable.t); 60 | }. 61 | Arguments signature : clear implicits. 62 | End BoxedPrintable. 63 | 64 | For the function `print_boxed_printable`, we use a parameter `Printable_t` in the dependent pair `BoxedPrintable` as the abstract type `t` is not known at this point. More generally, the abstract types are universally quantified in the definition of signatures and existentially quantified in values. 65 | 66 | Definition print_boxed_printable 67 | (BoxedPrintable : {Printable_t : _ & BoxedPrintable.signature Printable_t}) 68 | : unit := 69 | let BoxedPrintable := projT2 BoxedPrintable in 70 | print_endline 71 | (BoxedPrintable.(BoxedPrintable.Printable).(Printable.to_string) 72 | BoxedPrintable.(BoxedPrintable.value)). 73 | 74 | Note that the path to access the `to_string` function in Coq is more verbose and more explicit than in OCaml. 75 | 76 | ## What we support 77 | We support first-class modules with values, abstract types and type synonyms. We do not support first-class modules with other kinds of fields, such as the definition of new algebraic data types. We do not support functors (although we support first-class functions on first-class modules). 78 | 79 | A difficulty is to be able to distinguish between first-class modules and plain modules. This is necessary because we import first-class modules to dependent records and plain modules to Coq modules. For example, for projections, the syntax in OCaml is the same in both cases but different in Coq. When accessing a field of a module, we consider the module to be first-class if there exists a signature of the same shape. Once we found the name of the signature, we generate a call to the corresponding projection in Coq. If there are more than one signature corresponding to a module we generate an error. This can be the case because OCaml modules are not generative by default (we can make generative OCaml modules by [adding a `()` parameter](https://softwareengineering.stackexchange.com/questions/326304/what-is-the-difference-between-applicative-and-generative-modules-and-type-clas)). Our strategy to decide if a module is first-class is a heuristic, we may reconsider it later. 80 | 81 | ## Set example 82 | Here is the definition of sets as first-class modules, extracted from the [Tezos source code](https://gitlab.com/tezos/tezos/): 83 | 84 | module S = struct 85 | module type SET = sig 86 | type elt 87 | type t 88 | val empty: t 89 | val is_empty: t -> bool 90 | val mem: elt -> t -> bool 91 | val add: elt -> t -> t 92 | val remove: elt -> t -> t 93 | end 94 | end 95 | 96 | type 'a comparable_ty 97 | 98 | module type Boxed_set = sig 99 | type elt 100 | val elt_ty : elt comparable_ty 101 | module OPS : S.SET with type elt = elt 102 | val boxed : OPS.t 103 | val size : int 104 | end 105 | 106 | type 'elt set = (module Boxed_set with type elt = 'elt) 107 | 108 | let set_update 109 | : type a. a -> bool -> a set -> a set 110 | = fun v b (module Box) -> 111 | (module struct 112 | type elt = a 113 | let elt_ty = Box.elt_ty 114 | module OPS = Box.OPS 115 | let boxed = 116 | if b 117 | then Box.OPS.add v Box.boxed 118 | else Box.OPS.remove v Box.boxed 119 | let size = 120 | let mem = Box.OPS.mem v Box.boxed in 121 | if mem 122 | then if b then Box.size else Box.size - 1 123 | else if b then Box.size + 1 else Box.size 124 | end) 125 | 126 | We successfully generate the following valid Coq code: 127 | 128 | Module S. 129 | Module SET. 130 | Record signature {elt t : Type} := { 131 | elt := elt; 132 | t := t; 133 | empty : t; 134 | is_empty : t -> bool; 135 | mem : elt -> t -> bool; 136 | add : elt -> t -> t; 137 | remove : elt -> t -> t; 138 | }. 139 | Arguments signature : clear implicits. 140 | End SET. 141 | End S. 142 | 143 | Parameter comparable_ty : forall (a : Type), Type. 144 | 145 | Module Boxed_set. 146 | Record signature {elt OPS_t : Type} := { 147 | elt := elt; 148 | elt_ty : comparable_ty elt; 149 | OPS : S.SET.signature elt OPS_t; 150 | boxed : OPS.(S.SET.t); 151 | size : Z; 152 | }. 153 | Arguments signature : clear implicits. 154 | End Boxed_set. 155 | 156 | Definition set (elt : Type) := {OPS_t : _ & Boxed_set.signature elt OPS_t}. 157 | 158 | Definition set_update {a : Type} (v : a) (b : bool) (Box : set a) : set a := 159 | let Box := projT2 Box in 160 | existT _ _ 161 | {| 162 | Boxed_set.elt_ty := Box.(Boxed_set.elt_ty); 163 | Boxed_set.OPS := Box.(Boxed_set.OPS); 164 | Boxed_set.boxed := 165 | if b then 166 | Box.(Boxed_set.OPS).(S.SET.add) v Box.(Boxed_set.boxed) 167 | else 168 | Box.(Boxed_set.OPS).(S.SET.remove) v Box.(Boxed_set.boxed); 169 | Boxed_set.size := 170 | let mem := Box.(Boxed_set.OPS).(S.SET.mem) v Box.(Boxed_set.boxed) in 171 | if mem then 172 | if b then 173 | Box.(Boxed_set.size) 174 | else 175 | Z.sub Box.(Boxed_set.size) 1 176 | else 177 | if b then 178 | Z.add Box.(Boxed_set.size) 1 179 | else 180 | Box.(Boxed_set.size) 181 | |}. 182 | 183 | We use `existT _ _` to instantiate a dependent pair. We rely on the inference mechanism of Coq to fill the existential type variable `OPS_t` in this pair. 184 | 185 | ## Future work and opinions 186 | Things are not perfect yet. We still need to test the implementation more, debug, and add some features such as polymorphic abstract types. 187 | 188 | As a matter of taste, we prefer to import OCaml code to dependent records rather than functors. Indeed, we believe that records are safer than functors. The implementation of functors in the Coq kernel is complex as we have heard, while dependent records are already given by the dependent types. Moreover, Coq functors are generative while OCaml ones are not. We hope that we will not need Coq functors to import the code we wish to verify. 189 | -------------------------------------------------------------------------------- /posts/2020-01-15 Formalization of the Tezos protocol's interface in Coq.md: -------------------------------------------------------------------------------- 1 | The protocol of [Tezos](https://tezos.com/) is written in the [OCaml](https://ocaml.org/) language. It uses a [restricted OCaml interface](https://gitlab.com/tezos/tezos/tree/master/src/lib_protocol_environment/sigs/v1) of a few thousands lines of code to access to its primitives. Using the compiler [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml), we generate a [Coq formalization of most of the Tezos protocol's interface](static/artifacts/tezos-interface-in-coq/v1_mli.html) (the missing OCaml constructs are listed [here](https://clarus.github.io/coq-of-ocaml/examples/tezos-interface/)). We hope this work to be a first step to enable formal reasoning on the implementation of the Tezos protocol. In this blog post, we present what we added to `coq-of-ocaml` in order to support the Tezos interface. 2 | 3 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 4 | 5 | ## Polymorphic variants 6 | In OCaml, we can tag sum types with the backquote operator as follows: 7 | 8 | type json = 9 | [ `O of (string * json) list 10 | | `Bool of bool 11 | | `Float of float 12 | | `A of json list 13 | | `Null 14 | | `String of string ] 15 | 16 | There are no direct equivalents in [Coq](https://coq.inria.fr/) and we generate a warning message when we encounter tags. Still, we generate some code as a first approximation. For type definitions we generate a Coq inductive: 17 | 18 | Inductive json : Set := 19 | | Bool : bool -> json 20 | | Null : json 21 | | O : list (string * json) -> json 22 | | Float : Z -> json 23 | | String : string -> json 24 | | A : list json -> json. 25 | 26 | and for inlined variants: 27 | 28 | val fold : 29 | t -> 30 | key -> 31 | init:'a -> 32 | f:([`Key of key | `Dir of key] -> 'a -> 'a Lwt.t) -> 33 | 'a Lwt.t 34 | 35 | we generate a sum type with the tags as comments: 36 | 37 | Parameter fold : forall {a : Set}, 38 | t -> key -> a -> ((* `Dir *) key + (* `Key *) key -> a -> Lwt.t a) -> 39 | Lwt.t a. 40 | 41 | ## Types and values collisions 42 | In Coq, in contrast to OCaml, types and values live in the same namespace. Thus there are often name collisions in the generated code. Typically, this is due to the use of values having the same name as their type. We prevent that for the common cases (`list`, `string`, ...) by translating the value names with a `_value` suffix. For example, we convert: 43 | 44 | val string : string encoding 45 | 46 | to: 47 | 48 | Parameter __string_value : encoding string. 49 | 50 | ## Modules 51 | We added the import of the definitions of module types in the `.mli` files. The code to do so is the same as for `.ml` files. For example, we import: 52 | 53 | module type COMPARABLE = sig 54 | type t 55 | 56 | val compare : t -> t -> int 57 | end 58 | 59 | to: 60 | 61 | Module COMPARABLE. 62 | Record signature {t : Set} := { 63 | t := t; 64 | compare : t -> t -> Z; 65 | }. 66 | Arguments signature : clear implicits. 67 | End COMPARABLE. 68 | 69 | We handle the import of modules declared with a signature by unfolding the signature. We do so as we do not represent signatures in Coq unless for first-class modules (using a record type). For example, we import: 70 | 71 | module String : COMPARABLE with type t = string 72 | 73 | to: 74 | 75 | Module String. 76 | Definition t := string. 77 | 78 | Parameter compare : t -> t -> Z. 79 | End String. 80 | 81 | This idea is due to [Mehdi Bouaziz](https://fr.linkedin.com/in/mehdibouaziz). 82 | 83 | We also handle the `include` keyword by doing a similar unfolding. The unfolding of `include` significantly increases the size of the Coq code for the Tezos protocol's interface. Indeed, the number of generated lines for the interface was multiplied by two once we converted the `include` occurences. 84 | 85 | ## First-class modules 86 | We improved the detection of first-class modules. The detection of first-class modules is a challenge in `coq-of-ocaml`. Indeed, we need to distinguish between first-class and plain modules in order to generated either a dependent record or a Coq module. Moreover, there are no builtin ways to translate between a record and a module in Coq. 87 | 88 | When we access to the field of a module, we consider this module as a Coq record when it: 89 | 90 | * has a named signature (to have a corresponding named record definition); 91 | * is locally opened in an expression. 92 | 93 | We added the constraint of being locally opened in order to filter out some plain modules with a named signature. We may change this rule in the future and convert more plain modules to records by default, as this could factorize the generated code and help to handle functors. 94 | 95 | We added the support of first-class modules with polymorphic abstract types. For that, we mark the arity of the abstract types. For example, for a map signature in OCaml: 96 | 97 | module type MAP = sig 98 | type key 99 | type +'a t 100 | val empty : 'a t 101 | val is_empty : 'a t -> bool 102 | val mem : key -> 'a t -> bool 103 | end 104 | 105 | we generate: 106 | 107 | Module MAP. 108 | Record signature {key : Set} {t : Set -> Set} := { 109 | key := key; 110 | t := t; 111 | empty : forall {a : Set}, t a; 112 | is_empty : forall {a : Set}, t a -> bool; 113 | mem : forall {a : Set}, key -> t a -> bool; 114 | }. 115 | Arguments signature : clear implicits. 116 | End MAP. 117 | 118 | ## Records 119 | In Coq, the fields of a record are projection functions which live in the same namespace as the values. Thus Coq is more prone to name collisions for record fields than OCaml. We solve this issue by putting the record definitions into modules of the same name. For example, for the record: 120 | 121 | type descr = {name : string; descr : string option} 122 | 123 | we generate the following Coq code: 124 | 125 | Module descr. 126 | Record record := { 127 | name : string; 128 | descr : option string }. 129 | End descr. 130 | Definition descr := descr.record. 131 | 132 | We thus prevent a name collision between the record type `descr` and the record field `descr`. 133 | 134 | In subsequent accesses to the record fields, we prefix the projections by the record name. For example, to access to the `name` field of a record instance `r` of `descr`, we write in Coq: 135 | 136 | r.(descr.name) 137 | 138 | ## Pretty-printing 139 | We improved the pretty-printing for the types. In particular: 140 | 141 | * we limit the number of parenthesis by taking into account the precedence of the operators; 142 | * we use a more consistent indentation. 143 | 144 | As an example, we moved from the following generated Coq: 145 | 146 | | FLambda : Z -> (list ((Context.binder_annot Names.Name.t) * Constr.constr)) -> 147 | Constr.constr -> (Esubst.subs fconstr) -> fterm 148 | 149 | to: 150 | 151 | | FLambda : 152 | Z -> list (Context.binder_annot Names.Name.t * Constr.constr) -> 153 | Constr.constr -> Esubst.subs fconstr -> fterm 154 | 155 | ## Conclusion 156 | We generated around six thousands valid Coq lines. Some OCaml constructs were missing, such as the extensible types. We may not support them as we see no direct equivalents in Coq. There is also a lot of code duplication due to the repetitive inclusion of similar module signatures. We will try to find solutions to factorize the generated code. 157 | 158 | Finally, we will see if the code for the Tezos interface is usable when applying `coq-of-ocaml` to the protocol implementation. 159 | -------------------------------------------------------------------------------- /posts/2020-02-17 Latest updates of coq-of-ocaml for the Tezos protocol.md: -------------------------------------------------------------------------------- 1 | We recently made a lot of progress on the formalization in [Coq](https://coq.inria.fr/) of the [Tezos economic protocol](https://gitlab.com/tezos/tezos/-/tree/master/src/proto_alpha/lib_protocol), written in [OCaml](https://ocaml.org/), using [coq-of-ocaml](https://clarus.github.io/coq-of-ocaml/). We present here some of the new features of coq-of-ocaml and the results on the formalization of Tezos. 2 | 3 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 4 | 5 | ## Functors 6 | Previously, we were using dependent records to represent first-class modules. We now use dependent records whenever possible, even for non-first-class modules. This enables us to convert [OCaml functors](https://dev.realworldocaml.org/functors.html) to Coq functions over dependent records. As an example, we translate this OCaml program: 7 | 8 | (* Signature of the parameter of the functor *) 9 | module type Source = sig 10 | type t 11 | val x : t 12 | end 13 | 14 | (* Signature of the result of the functor *) 15 | module type Target = sig 16 | type t 17 | val y : t 18 | end 19 | 20 | (* Definition of the functor *) 21 | module F (X : Source) : Target with type t = X.t = struct 22 | type t = X.t 23 | let y = X.x 24 | end 25 | 26 | (* Application to the module [M] *) 27 | module M : Source = struct 28 | type t = int 29 | let x = 12 30 | end 31 | 32 | module N = F (M) 33 | 34 | to the following Coq code: 35 | 36 | Module Source. 37 | Record signature {t : Set} := { 38 | t := t; 39 | x : t; 40 | }. 41 | Arguments signature : clear implicits. 42 | End Source. 43 | 44 | Module Target. 45 | Record signature {t : Set} := { 46 | t := t; 47 | y : t; 48 | }. 49 | Arguments signature : clear implicits. 50 | End Target. 51 | 52 | Definition F := 53 | fun (X : {t : _ & Source.signature t}) => 54 | (let t := (|X|).(Source.t) in 55 | let y := (|X|).(Source.x) in 56 | existT (fun _ => _) tt 57 | {| 58 | Target.y := y 59 | |} : {_ : unit & Target.signature (|X|).(Source.t)}). 60 | 61 | Definition M := 62 | let t := Z in 63 | let x := 12 in 64 | existT _ _ 65 | {| 66 | Source.x := x 67 | |}. 68 | 69 | Definition N := 70 | F 71 | (existT _ _ 72 | {| 73 | Source.x := (|M|).(Source.x) 74 | |}). 75 | 76 | We note `(|M|)` the projection `projT2` to get the second component of an existential type `{x : A & P x}`. We represent signatures by records parametrized by their abstract types. We wrap these records with existential types for the abstract types which are not precised by a `with type t = ...` construct. Functors are then plain Coq functions over dependent records. For more details, see the documentation on the [module system](https://clarus.github.io/coq-of-ocaml/docs/module-system) in coq-of-ocaml. 77 | 78 | Our convention is to represent a module as a dependent record if there exists a name for its signature. In this case, we use its signature name for its record's type name. We consider the other modules as namespaces, and represent them with standard Coq modules. 79 | 80 | ## GADTs and existential types 81 | [GADTs](https://caml.inria.fr/pub/docs/manual-ocaml/manual033.html), an advanced form of algebraic datatypes, are used a lot in the Tezos protocol. They help to ensure safety properties, like the soundness of the type-checker of the smart-contracts. 82 | 83 | ### GADTs 84 | As we did not find a general way to convert GADTs to Coq, we chose to erase the type annotations and introduce unsafe casts. Here is an example of OCaml code with a GADT: 85 | 86 | type 'a int_or_string = 87 | | Int : int int_or_string 88 | | String : string int_or_string 89 | 90 | let to_string (type a) (kind : a int_or_string) (x : a) : string = 91 | match[@coq_match_gadt] kind, x with 92 | | Int, (x : int) -> string_of_int x 93 | | String, (x : string) -> x 94 | 95 | Note the `[@coq_match_gadt]` on the `match` and the type annotations on the variable `x` in the patterns. We import this code in Coq to: 96 | 97 | Reserved Notation "'int_or_string". 98 | 99 | Inductive int_or_string_gadt : Set := 100 | | Int : int_or_string_gadt 101 | | String : int_or_string_gadt 102 | 103 | where "'int_or_string" := (fun (_ : Set) => int_or_string_gadt). 104 | 105 | Definition int_or_string := 'int_or_string. 106 | 107 | Definition to_string {A : Set} (kind : int_or_string A) (x : A) : string := 108 | match (kind, x) with 109 | | (Int, _ as x) => 110 | let 'existT _ tt x := cast_exists (fun _ => Z) x in 111 | cast string (OCaml.Stdlib.string_of_int x) 112 | | (String, _ as x) => 113 | let 'existT _ tt x := cast_exists (fun _ => string) x in 114 | cast string x 115 | end. 116 | 117 | We convert the GADT `int_or_string` to an inductive `int_or_string_gadt` without annotations. We generate two axioms in each branch of the `match`, to cast: 118 | 119 | * the variables introduced by the pattern; 120 | * the result of the branch. 121 | 122 | The axiom `cast` is the equivalent in Coq of the OCaml's [Obj.magic](https://caml.inria.fr/pub/docs/manual-ocaml/libref/Obj.html) cast. It has the following signature: 123 | 124 | Axiom cast : forall {A : Set} (B : Set), A -> B. 125 | 126 | We say that: 127 | 128 | * `cast` behaves as the identity function when `A` is equal to `B`; 129 | * is undefined in other cases. 130 | 131 | To specify this behavior, we use the following axiom: 132 | 133 | Axiom cast_eval : forall {A : Set} {x : A}, cast A x = x. 134 | 135 | While doing proofs, we must use the axiom `cast_eval` to evaluate `cast` by proving that the types `A` and `B` are the same. Doing so, we also verify that the type unifications of the type-checker of OCaml are indeed correct. The `cast_exists` axiom is like `cast` with the ability to introduce some existential variables when needed. 136 | 137 | ### Existential variables 138 | Types with existential variables are a special case of GADTs, where the type parameters are the same for all the constructors. For example, to represent a value which can be converted to a `string`, we can use: 139 | 140 | type printable = Printable : 'a * ('a -> string) -> printable 141 | 142 | let printable_to_string (x : printable) : string = 143 | let Printable (value, print) = x in 144 | print value 145 | 146 | In this example, `'a` is an existential variable. Coq also support existential variables in algebraic types. Here is what we generate for this example: 147 | 148 | Inductive printable : Set := 149 | | Printable : forall {a : Set}, a -> (a -> string) -> printable. 150 | 151 | Definition printable_to_string (x : printable) : string := 152 | let 'Printable value print := x in 153 | let 'existT _ __Printable_'a [value, print] := 154 | existT 155 | (fun __Printable_'a : Set => 156 | [__Printable_'a ** (__Printable_'a -> string)]) _ [value, print] in 157 | print value. 158 | 159 | We do not need any axioms. The `let 'existT _ __Printable_'a [value, print] :=` block is there to rename the existential variables generated by Coq to the names of the OCaml compiler. We replace the forbidden symbol `$` by `__`, so that: 160 | 161 | $Printable_'a 162 | 163 | becomes: 164 | 165 | __Printable_'a 166 | 167 | Having well named existential variables helps to get: 168 | 169 | * cleaner error messages; 170 | * type annotations on sub-expressions using these existential variables. 171 | 172 | ## Documentation website 173 | We added a [website for coq-of-ocaml](https://clarus.github.io/coq-of-ocaml/) to have a central place with documentation. We used [Docusaurus](https://docusaurus.io/) to generate the website. We chose Docusaurus for the following reasons: 174 | 175 | * open-source; 176 | * good defaults; 177 | * generates static HTML; 178 | * we can write the documentation in Markdown. 179 | 180 | ## We converted most of the protocol 181 | At the time of writing, 57% of the Coq code generated by coq-of-ocaml for the Tezos protocol compiles. This includes the interpreter of the smart-contracts, and amounts to around 30.000 lines of valid Coq code. We still ignore many features, like the side-effects or the extensible types. The main missing files are the `*_services.ml` files and the type-checker of smart-contracts (6.000 lines of OCaml, the largest file of the protocol). 182 | -------------------------------------------------------------------------------- /posts/2020-10-20 Improvements of coq-of-ocaml for functors and signatures.md: -------------------------------------------------------------------------------- 1 | With [coq-of-ocaml](https://clarus.github.io/coq-of-ocaml/) we can translate many [OCaml](https://ocaml.org/) constructs to an equivalent in the [Coq](https://coq.inria.fr/) language. Based on the code which we encounter, we continue to update coq-of-ocaml to handle more OCaml programming patterns. In this post, we will show: 2 | 3 | * how we changed the representation of functors to have a clearer generated code; 4 | * how we handled the anonymous sub-signatures. 5 | 6 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 7 | 8 | ## Functors like plain modules 9 | Th module system of OCaml allows to represent modules parametrized by others. Such parametrized modules are called [functors](https://ocaml.org/releases/4.11/htmlman/moduleexamples.html#s:functors). We represent modules with a named signature by [records](https://coq.inria.fr/refman/language/core/records.html) in Coq. Then we encode functors by functions over dependent records. We like this representation because: 10 | 11 | * we can also encode [first-class modules](https://ocaml.org/releases/4.11/htmlman/firstclassmodules.html) (modules used in the context of values in OCaml); 12 | * it does not depend on the functor system of Coq, but only on dependent types. The less Coq features from the kernel we use, the safer we feel. 13 | 14 | An issue we had was expressing lemmas about items of a functor. For example, with the following OCaml code: 15 | 16 | module type Source = sig 17 | type t 18 | val x : t 19 | val id : 'a -> 'a 20 | end 21 | 22 | module type Target = sig 23 | type t 24 | val y : t 25 | end 26 | 27 | module F (X : Source) : Target with type t = X.t = struct 28 | type t = X.t 29 | let y = X.x 30 | end 31 | 32 | we would generate the following Coq code: 33 | 34 | Module Source. 35 | Record signature {t : Set} : Set := { 36 | t := t; 37 | x : t; 38 | }. 39 | End Source. 40 | 41 | Module Target. 42 | Record signature {t : Set} : Set := { 43 | t := t; 44 | y : t; 45 | }. 46 | End Target. 47 | 48 | Definition F := 49 | fun (X : {t : Set & Source.signature (t := t)}) => 50 | ((let t : Set := (|X|).(Source.t) in 51 | let y := (|X|).(Source.x) in 52 | existT (A := unit) (fun _ => _) tt 53 | {| 54 | Target.y := y 55 | |}) : {_ : unit & Target.signature (t := (|X|).(Source.t))}). 56 | 57 | There are two module types `Source` and `Target` which we represent by record definitions. We parametrize these records by their abstract types. We represent the functor `F` by a function from the type of record `Source` to the type of record `Target`. Here the whole definition of `F` is packed into a single function. We define the resulting record using local `let` declarations. We changed that to have top-level definitions for each item of the functor (`t` and `y`). By representing the functor parameters as a type-class, we now generate: 58 | 59 | Module F. 60 | Class FArgs := { 61 | X : {t : Set & Source.signature (t := t)}; 62 | }. 63 | 64 | Definition t `{FArgs} : Set := (|X|).(Source.t). 65 | 66 | Definition y `{FArgs} : (|X|).(Source.t) := (|X|).(Source.x). 67 | 68 | Definition functor `(FArgs) 69 | : {_ : unit & Target.signature (t := (|X|).(Source.t))} := 70 | existT (A := unit) (fun _ => _) tt 71 | {| 72 | Target.y := y 73 | |}. 74 | End F. 75 | Definition F X := F.functor {| F.X := X |}. 76 | 77 | The class `FArgs` contains the functor arguments. This class has one field per functor parameter (in this case only `X`). We give to each declaration an implicit parameter `` `{FArgs}`` so that the parameter `X` is always accessible. At the end of the module, we materialize the functor as a function `functor` from an instance of the class of parameters, to the record of the resulting module. Eventually, we define the functor `F` as a function taking parameters in order and convert them to an instance of the class `FArgs`. 78 | 79 | Remark: we do not use the [section mechanism](https://coq.inria.fr/refman/language/core/sections.html), because this would not compose. Indeed, we cannot create modules inside sections in the current version of Coq. Thus we can only represent flat functors with sections, and cannot represent functors with sub-modules. 80 | 81 | For modules represented by records we do the same, without the `FArgs` parameter. For example, we translate: 82 | 83 | module M : Source = struct 84 | type t = int 85 | let x = 12 86 | let id x = x 87 | end 88 | 89 | to: 90 | 91 | Module M. 92 | Definition t : Set := int. 93 | 94 | Definition x : int := 12. 95 | 96 | Definition id {A : Set} (x : A) : A := x. 97 | 98 | Definition module := 99 | existT (A := Set) _ t 100 | {| 101 | Source.x := x; 102 | Source.id _ := id 103 | |}. 104 | End M. 105 | Definition M := M.module. 106 | 107 | We hope that this presentation is cleaner on the Coq side. For example: 108 | 109 | * we can directly talk about individual items without referencing the whole resulting record; 110 | * we can talk about intermediate items which may not be exported at the end; 111 | * we can have plain sub-modules for large functors (which cannot be represented as records with the current system when there are no named signatures); 112 | * we can define new types as we would at top-level. 113 | 114 | ## Anonymous sub-signatures 115 | For large signatures, we tend to use sub-signatures in order to group items going together. For example, we can define in OCaml: 116 | 117 | module type T_encoding = sig 118 | type t 119 | 120 | val encoding : t list 121 | end 122 | 123 | module type Validator = sig 124 | module Ciphertext : sig 125 | include T_encoding 126 | 127 | val get_memo_size : t -> int 128 | end 129 | 130 | module CV : T_encoding 131 | 132 | type t = Ciphertext.t 133 | end 134 | 135 | Since we represent signatures by records, we cannot directly represent the sub-signature for `Ciphertext` in the signature `Validator`. Indeed, there are no notions of sub-records in Coq. Thus, we were generating the following error message: 136 | 137 | --- foo.ml:8:23 ------------------------------------------------------------ not_supported (1/1) --- 138 | 139 | 6 | 140 | 7 | module type Validator = sig 141 | > 8 | module Ciphertext : sig 142 | > 9 | include T_encoding 143 | > 10 | 144 | > 11 | val get_memo_size : t -> int 145 | > 12 | end 146 | 13 | 147 | 14 | module CV : T_encoding 148 | 15 | 149 | 150 | 151 | Anonymous definition of signatures is not handled 152 | 153 | Now we inline the sub-modules by prefixing the name of their fields, so that we generate a single flat record in Coq. For the OCaml code above, we generate: 154 | 155 | Module T_encoding. 156 | Record signature {t : Set} : Set := { 157 | t := t; 158 | encoding : list t; 159 | }. 160 | End T_encoding. 161 | 162 | Module Validator. 163 | Record signature {Ciphertext_t CV_t : Set} : Set := { 164 | Ciphertext_t := Ciphertext_t; 165 | Ciphertext_encoding : list Ciphertext_t; 166 | Ciphertext_get_memo_size : Ciphertext_t -> int; 167 | CV : T_encoding.signature (t := CV_t); 168 | t := Ciphertext_t; 169 | }. 170 | End Validator. 171 | 172 | We prefix all the fields of the sub-module `Ciphertext` by `Ciphertext_`. We propagate the naming of the types. For example the type `t` from `Ciphertext` is renamed as `Ciphertext_t` in all the subsequent expressions. For all other fields which involve this type `t`, we mark it as `Ciphertext_t`. We keep using nested records for sub-signatures with a name. For example, the sub-module `CV` is a field of type `T_encoding.signature`, which is itself a record. The two variables `Ciphertext_t` and `CV_t` are the two abstract types of the signature. As with flat signatures, we represent them using existential types when unknown. 173 | 174 | We also changed the way we translate the identifiers to reference items in signatures. For example, if we have a functor using the field `Ciphertext.get_memo_size`: 175 | 176 | module F (V : Validator) = struct 177 | let get = V.Ciphertext.get_memo_size 178 | end 179 | 180 | we now generate: 181 | 182 | Module F. 183 | Class FArgs := { 184 | V : 185 | {'[Ciphertext_t, CV_t] : [Set ** Set] & 186 | Validator.signature (Ciphertext_t := Ciphertext_t) (CV_t := CV_t)}; 187 | }. 188 | 189 | Definition get `{FArgs} : (|V|).(Validator.Ciphertext_t) -> int := 190 | (|V|).(Validator.Ciphertext_get_memo_size). 191 | End F. 192 | 193 | Here we represent `V.Ciphertext.get_memo_size` by `(|V|).(Validator.Ciphertext_get_memo_size)`. We use the notation to access record fields with `record.(field)`. We project the potential existential variables with the notation `(|...|)`. Here are the steps we followed to translate this identifier: 194 | 195 | * check if `V` has a known named signature (in this case we find the signature `Validator`); 196 | * check if `Ciphertext` has a known signature (we do not find one); 197 | * conclude that the `get_memo_size` field has probably been inlined in the signature definition; 198 | * generate a record access on `V` on the field `Ciphertext_get_memo_size`, which is the concatenation of `Ciphertext` and `get_memo_size` according to the way we prefix field names in inlining. 199 | 200 | The handling of anonymous sub-signatures is not completely mandatory. Indeed, we could also give a name to the sub-signatures in the OCaml code. However we encountered some cases were this was convenient, in order not to modify the OCaml code too much. 201 | 202 | ## Conclusion 203 | We have shown two new translation strategies which, we hope, will polish the experience of using [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml). This work was mainly directed by an experiment at handling some code from the [Tezos](https://tezos.com/) codebase. The aim was to verify some parts of the [sapling](https://blog.nomadic-labs.com/sapling-integration-in-tezos-tech-preview.html) project. This code is outside of the protocol, which is the part we are the most used to work with. Then, we met some new OCaml constructs and patterns. In particular, we encountered large functors with anonymous sub-signatures, plain sub-modules, and new type definitions. 204 | -------------------------------------------------------------------------------- /posts/2021-02-08 Removing existential types from modules in coq-of-ocaml.md: -------------------------------------------------------------------------------- 1 | The tool [coq-of-ocaml](https://github.com/clarus/coq-of-ocaml) translates [OCaml](https://ocaml.org/) programs to [Coq](https://coq.inria.fr/) programs using a shallow embedding. To translate the [modules and functors](https://caml.inria.fr/pub/docs/manual-ocaml/moduleexamples.html) of OCaml we use polymorphic [records](https://coq.inria.fr/refman/language/core/records.html) in Coq. With this representation, we are also able to translate [first-class modules](https://caml.inria.fr/pub/docs/manual-ocaml/firstclassmodules.html). 2 | 3 | We originally used existential types to represent abstract module types. This could be a source of complexity for the reasoning on the generated code. Indeed, existential types require to do frequent [projections](https://coq.inria.fr/library/Coq.Init.Specif.html#projT2) and [wrapping](https://coq.inria.fr/library/Coq.Init.Specif.html#existT) in Coq. In this blog post, we show how we removed the need of existential types in most non-first-class modules. 4 | 5 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 6 | 7 | ## Example 8 | We take the following OCaml code: 9 | 10 | module F (X : Source) : Target with type t2 = X.t = struct 11 | type t1 = string 12 | type t2 = X.t 13 | let y = X.x 14 | end 15 | 16 | module FM = F (M) 17 | 18 | We assume that: 19 | 20 | * the signature `Source` has one abstract type `t`; 21 | * the signature `Target` has two abstract types `t1` and `t2`, the type `t2` being explicitly specified as `X.t` in this example. 22 | 23 | ### Now 24 | We show how we now translate this OCaml example with the latest changes in coq-of-ocaml. In particular, we show how we avoid using existential types for the abstract module types. We start by using a [Coq type class](https://coq.inria.fr/refman/addendum/type-classes.html) to represent the functor parameters as explained in a [previous article](http://coq-blog.clarus.me/improvements-of-coq-of-ocaml-for-functors-and-signatures.html): 25 | 26 | Module F. 27 | Class FArgs {X_t : Set} := { 28 | X : Source (t := X_t); 29 | }. 30 | Arguments Build_FArgs {_}. 31 | 32 | We represent the abstract type `t` of `X` with a parameter `X_t` of the class `FArgs`. This type is implicit (notation `{}`) since we can infer it from the module `X`. We then define the various fields of the functor, including a special field `functor` which is the record representing the result of the functor: 33 | 34 | Definition t1 `{FArgs} : Set := string. 35 | 36 | Definition t2 `{FArgs} : Set := X.(Source.t). 37 | 38 | Definition y `{FArgs} : X.(Source.t) := X.(Source.x). 39 | 40 | Definition functor `{FArgs} := 41 | {| 42 | Target.y := y 43 | |}. 44 | End F. 45 | 46 | Finally, we wrap the functor `F.functor` into a function `F` without type classes: 47 | 48 | Definition F {X_t : Set} (X : Source (t := X_t)) 49 | : Target (t1 := _) (t2 := X.(Source.t)) := 50 | let '_ := F.Build_FArgs X in 51 | F.functor. 52 | 53 | We keep `X_t` as an implicit parameter. The type `X_t` is a parameter rather than an existential type. For the abstract type `t1`, we let Coq infer its value with: 54 | 55 | Target (t1 := _) (t2 := X.(Source.t)) 56 | 57 | Since Coq has access to the definition of `F`, it is able to guess the value of `t1`, and we do not need an existential type there. For the type `t2`, we directly give its value `X.(Source.t)` like in the OCaml source. We think it is better to always give an explicit type value when possible. Indeed, the expression inferred by Coq may be too large and have caused performance issues in some of our examples. 58 | 59 | To apply the functor `F` on `M` we simply do a function application: 60 | 61 | Definition FM := F M. 62 | 63 | Coq infers all the missing abstract type values for us. 64 | 65 | ### Before 66 | Before these recent changes, we were wrapping all the modules into one existential type for each abstract type. On this example, we would have generated the following Coq code: 67 | 68 | Module F. 69 | Class FArgs := { 70 | X : {t : Set & Source.signature (t := t)}; 71 | }. 72 | 73 | Definition t1 `{FArgs} : Set := string. 74 | 75 | Definition t2 `{FArgs} : Set := (|X|).(Source.t). 76 | 77 | Definition y `{FArgs} : (|X|).(Source.t) := (|X|).(Source.x). 78 | 79 | Definition functor `(FArgs) 80 | : {t1 : Set & Target.signature (t1 := t1) (t2 := (|X|).(Source.t))} := 81 | existT (A := Set) _ t1 82 | {| 83 | Target.y := y 84 | |}. 85 | End F. 86 | Definition F X := F.functor {| F.X := X |}. 87 | 88 | Definition FM := F (existT (A := Set) _ _ (|M|)). 89 | 90 | We wrap the abstract type `t` of `X` in the existential: 91 | 92 | {t : Set & Source.signature (t := t)} 93 | 94 | We wrap the abstract type `t1` in the existential: 95 | 96 | existT (A := Set) _ t1 {| ... |} 97 | 98 | To access the fields of each module we first project the existential types with the notation: 99 | 100 | Notation "(| M |)" := (projT2 M). 101 | 102 | To apply the functor `F` to `M` we also wrap `M` into an existential in order to be sure to have the correct number of existential types. We believe that the use of existential types associated with frequent projections or wrapping was a source of complexity for proofs made on the generated Coq code. 103 | 104 | ## Strategy 105 | We now describe the general strategy we use to remove the need of existential types in common use cases. 106 | 107 | ### Definitions 108 | When we define a module `M` with some abstract types in the signature, we let Coq infer their values. When we define a functor `F` parametrized by some modules: 109 | 110 | M_1, ..., M_n 111 | 112 | with the abstract types: 113 | 114 | M_1 : t_1_1, t_1_2, ... 115 | ... 116 | M_n : t_n_1, t_n_2, ... 117 | 118 | and returning a module of signature `S` with the abstract types: 119 | 120 | S : t_1, t_2, ... 121 | 122 | we push all the abstract types in front, so that the type of the function representing the functor `F` in Coq is: 123 | 124 | F : 125 | (* The abstract types could also have a higher arity such as Set -> Set *) 126 | forall {t_1_1 t_1_2 ... t_n_1 t_n_2 ... : Set}, 127 | forall (M1 : M_1_signature (t_1_1 := t_1_1) (t_1_2 := t_1_2) ...), 128 | ... 129 | forall (M_n : M_n_signature (t_n_1 := t_n_1) (t_n_2 := t_n_2) ...), 130 | S 131 | (t_1 := either _ if abstract or some type expression if specified) 132 | ... 133 | (t_n := ...) 134 | 135 | We only support functors whose parameters are all modules. Since we support modules containing functors, it is possible to wrap a functor into a module to pass it to another functor. However the functors in modules cannot return some abstract types (we had [universe level](http://adam.chlipala.net/cpdt/html/Universes.html) issues with that). We also expect the functors to be applied on all their parameters at once (no [currying](https://en.wikipedia.org/wiki/Currying)). Indeed, we need to infer all the abstract types of the parameters at once. 136 | 137 | An example of a functor type from the [Coq code generated from Tezos](https://gitlab.com/nomadic-labs/coq-tezos-of-ocaml) is the following: 138 | 139 | Definition Make_single_data_storage {C_t V_t : Set} 140 | (R : Storage_sigs.REGISTER) (C : Raw_context.T (t := C_t)) 141 | (N : Storage_sigs.NAME) (V : Storage_sigs.VALUE (t := V_t)) 142 | : Storage_sigs.Single_data_storage (t := C.(Raw_context.T.t)) 143 | (value := V.(Storage_sigs.VALUE.t)) := 144 | (* ... the definition *) 145 | 146 | ### Axioms 147 | We convert `.mli` files to lists of axioms. This raises an issue as we cannot infer the abstract types in a module or in the output module of a functor anymore. This inference is not possible because we do not have access to the definition of axioms. We solve this issue by adding one more axiom for each type to infer. 148 | 149 | For example, in OCaml the module [`Map`](https://caml.inria.fr/pub/docs/manual-ocaml/libref/Map.html) is declared as follows: 150 | 151 | module type S = sig 152 | (** The type of the map keys. *) 153 | type key 154 | 155 | (** The type of maps from type [key] to type ['a]. *) 156 | type +'a t 157 | 158 | (* ... *) 159 | end 160 | 161 | module Make: functor (Ord : OrderedType) -> S with type key = Ord.t 162 | 163 | The signature `S` has two abstract types `key` and `t`. The signature `OrderedType` has one abstract type `t`. We transform the declaration of the functor `Make` to the axioms: 164 | 165 | Parameter Make_t : 166 | forall {Ord_t : Set} (Ord : OrderedType (t := Ord_t)), Set -> Set. 167 | 168 | Parameter Make : 169 | forall {Ord_t : Set}, 170 | forall (Ord : OrderedType (t := Ord_t)), 171 | S (key := Ord.(OrderedType.t)) (t := Make_t Ord). 172 | 173 | Since the abstract type `t` of the signature `S` is unspecified, we introduce an additional axiom `Make_t` to describe its value. This axiom takes the same arguments as the functor `Make`. 174 | 175 | ### First-class modules 176 | We keep using existential types for the [first-class modules](https://caml.inria.fr/pub/docs/manual-ocaml/firstclassmodules.html). This is because first-class module values can appear at any position in a program (in a function, in a data structure, ...). Thus the methods above would not be enough to eliminate the need of existential types in all cases. 177 | 178 | We use the following strategy: 179 | 180 | * we add the existential types when we go from a module to a value; 181 | * we remove the existential types (by projection) when we go from a value to a module; 182 | * when we access to the field of a first-class module, we consider that the existential types have already been removed, since it has already been converted to a module. 183 | 184 | Here is an example of OCaml code with first-class modules, from the code of Tezos: 185 | 186 | type ('key, 'value) map = 187 | (module Boxed_map with type key = 'key and type value = 'value) 188 | 189 | let map_set : type a b. a -> b -> (a, b) map -> (a, b) map = 190 | fun k v (module Box) -> 191 | ( module struct 192 | type key = a 193 | 194 | type value = b 195 | 196 | let key_ty = Box.key_ty 197 | 198 | module OPS = Box.OPS 199 | 200 | let boxed = 201 | let (map, size) = Box.boxed in 202 | (Box.OPS.add k v map, if Box.OPS.mem k map then size else size + 1) 203 | end ) 204 | 205 | This implements a map type which is polymorphic in the type of the keys. One needs to give the comparison function for the keys when initializing an empty map. We translate this code to: 206 | 207 | Definition map (key value : Set) : Set := 208 | {OPS_t : Set -> Set @ 209 | Boxed_map (key := key) (value := value) (OPS_t := OPS_t)}. 210 | 211 | Definition map_set {a b : Set} (k : a) (v : b) (Box : map a b) 212 | : map a b := 213 | let 'existS _ _ Box := Box in 214 | existS (A := Set -> Set) _ _ 215 | (let key : Set := a in 216 | let value : Set := b in 217 | let key_ty := Box.(Boxed_map.key_ty) in 218 | let OPS := Box.(Boxed_map.OPS) in 219 | let boxed := 220 | let '(map, size) := Box.(Boxed_map.boxed) in 221 | ((Box.(Boxed_map.OPS).(S.MAP.add) k v map), 222 | (if Box.(Boxed_map.OPS).(S.MAP.mem) k map then 223 | size 224 | else 225 | size +i 1)) in 226 | {| 227 | Boxed_map.key_ty := key_ty; 228 | Boxed_map.OPS := OPS; 229 | Boxed_map.boxed := boxed 230 | |}). 231 | 232 | We use an existential type to define the type `map`. Note that we use an existential type in the sort `Set` with the [imprediative Set](https://github.com/coq/coq/wiki/Impredicative-Set) option of Coq enabled. This allows us to avoid any universe level issue by always staying in the sort `Set`. We define the existential in `Set` like the ones in `Prop` or `Type`: 233 | 234 | Inductive sigS (A : Type) (P : A -> Set) : Set := 235 | | existS : forall (x : A), P x -> sigS P. 236 | 237 | Notation "{ x : A @ P }" := (sigS (A := A) (fun x => P)) : type_scope. 238 | 239 | To open the first-class module value `Box` as a module we use the pattern: 240 | 241 | fun k v (module Box) -> 242 | 243 | in OCaml. We convert this pattern to an existential projection: 244 | 245 | let 'existS _ _ Box := Box in 246 | 247 | in Coq. Then to close the module and convert it back to a value, we wrap it into an existential with: 248 | 249 | existS (A := Set -> Set) _ _ (...) 250 | 251 | ## Conclusion 252 | We hope that the effort we made into removing existential types from the generated Coq code will help to do simpler proofs on OCaml programs using a lot of modules. 253 | -------------------------------------------------------------------------------- /posts/2021-02-22 Beginning of verification for the parsing of smart-contracts.md: -------------------------------------------------------------------------------- 1 | In this blog post, we will show how we [formally verify](https://en.wikipedia.org/wiki/Formal_verification) in [Coq](https://coq.inria.fr/) some properties about the parser of [smart-contracts](https://en.wikipedia.org/wiki/Smart_contract) for the crypto-currency [Tezos](https://tezos.com/). The [proofs of this formal verification](https://gitlab.com/nomadic-labs/coq-tezos-of-ocaml/-/tree/master/src/Proto_alpha/Proofs/Script_ir_translator) are hosted in our project [coq-tezos-of-ocaml](https://gitlab.com/nomadic-labs/coq-tezos-of-ocaml). 2 | 3 | To reason about the [implementation of the parser](https://gitlab.com/tezos/tezos/-/blob/master/src/proto_alpha/lib_protocol/script_ir_translator.ml), written in [OCaml](https://ocaml.org/), we use the tool [coq-of-ocaml](https://clarus.github.io/coq-of-ocaml/) to convert it automatically to Coq. We will talk about the tricks we used to get this conversion to work, in particular for the GADTs and the mutually recursive functions. We will also present the properties which we verified and how we did it. 4 | 5 | > We develop [coq-of-ocaml]((https://clarus.github.io/coq-of-ocaml/)) at [🐙 Nomadic Labs](https://www.nomadic-labs.com/) with the aim to formally verify OCaml programs, and in particular the implementation of the crypto-currency [Tezos](https://tezos.com/). If you want to use this tool for your own projects, please do not hesitate to look at our [website](https://clarus.github.io/coq-of-ocaml/) or [contact us](mailto:contact@nomadic-labs.com)! 6 | 7 | ## How do we convert the OCaml code to Coq 8 | The code which interests us is in the file [`script_ir_translator.ml`](https://gitlab.com/tezos/tezos/-/blob/master/src/proto_alpha/lib_protocol/script_ir_translator.ml). This is a long file, containing in particular the parser and type-checker of the [Michelson](https://wiki.tezosagora.org/files/language.html#michelson) language for smart-contracts. We are interested into the functions "parse something" and "unparse something" to show that there are compatible. We use the tool coq-of-ocaml to convert the code of this file. Since it depends on other files of the code of Tezos (for type definitions or primitives), we need to have the Coq definition of all its dependencies too. This is done in the project [coq-tezos-of-ocaml](https://gitlab.com/nomadic-labs/coq-tezos-of-ocaml). The translation of `script_ir_translator.ml` is in [src/Proto\_alpha/Script\_ir\_translator.v](https://gitlab.com/nomadic-labs/coq-tezos-of-ocaml/-/blob/master/src/Proto_alpha/Script_ir_translator.v). 9 | 10 | ### Mutually recursive functions 11 | Mutually recursive functions are a challenge in Coq because of the syntactic constraints to make sure that each function terminates. Indeed, termination is important for Coq because a non-terminating functions would make the whole system logically inconsistent. Moreover, OCaml programs may not follow the syntactic constraints for termination of Coq as the OCaml compiler does not check for termination. Thus we use several techniques to translate the OCaml code to Coq without modifying too much the source: 12 | 13 | * disabling if needed the termination checker of Coq with the [`Guard Checking`](https://coq.inria.fr/refman/proof-engine/vernacular-commands.html#coq:flag.Guard-Checking) flag; 14 | * introducing [OCaml attributes](https://clarus.github.io/coq-of-ocaml/docs/attributes) to guide the translation. 15 | 16 | There are two main attributes useful to translate recursive functions: 17 | 18 | * `@coq_struct "ident"` to specify the `{struct ident}` parameter of the [`Fixpoint`](https://coq.inria.fr/refman/language/core/inductive.html#coq:cmd.Fixpoint) command of Coq; 19 | * `@coq_mutual_as_notation` to force some mutual functions to be defined as notations. 20 | 21 | For example, let us we take the function `parse_ty` to parse types, defined in OCaml as: 22 | 23 | let rec parse_ty = 24 | fun ctxt 25 | ~legacy 26 | ~allow_lazy_storage 27 | ~allow_operation 28 | ~allow_contract 29 | ~allow_ticket 30 | node -> 31 | Gas.consume ctxt Typecheck_costs.parse_type_cycle 32 | >>? fun ctxt -> 33 | match node with 34 | | Prim (_loc, T_unit, [], _annot) -> 35 | ok (Ex_ty Unit_t, ctxt) 36 | | Prim (_loc, T_int, [], _annot) -> 37 | ok (Ex_ty Int_t, ctxt) 38 | | Prim (_loc, T_nat, [], _annot) -> 39 | ok (Ex_ty Nat_t, ctxt) 40 | | Prim (_loc, T_string, [], _annot) -> 41 | ok (Ex_ty String_t, ctxt) 42 | | Prim (_loc, T_bytes, [], _annot) -> 43 | ok (Ex_ty Bytes_t, ctxt) 44 | | Prim (_loc, T_mutez, [], _annot) -> 45 | ok (Ex_ty Mutez_t, ctxt) 46 | | Prim (_loc, T_bool, [], _annot) -> 47 | ok (Ex_ty Bool_t, ctxt) 48 | [...] 49 | 50 | and parse_parameter_ty = 51 | fun ctxt ~legacy -> 52 | parse_ty 53 | ctxt 54 | ~legacy 55 | ~allow_lazy_storage:true 56 | ~allow_operation:false 57 | ~allow_contract:true 58 | ~allow_ticket:true 59 | 60 | and parse_any_ty = 61 | [...] 62 | 63 | We have one main function `parse_ty` iterating over the parameter `node` and several auxiliary functions such as `parse_parameter_ty` which are exposed at top-level. We represent these auxiliary functions as notations in Coq, so that they are simpler to reason about compared to mutual fixpoints. We annotate the `parse_parameter_ty` function by the `@coq_mutual_as_notation` attribute so that we generate in Coq: 64 | 65 | Reserved Notation "'parse_parameter_ty". 66 | Reserved Notation "'parse_any_ty". 67 | [...] 68 | 69 | Fixpoint parse_ty 70 | (ctxt : Alpha_context.context) (legacy : bool) (allow_lazy_storage : bool) 71 | (allow_operation : bool) (allow_contract : bool) (allow_ticket : bool) 72 | (node : Alpha_context.Script.node) {struct node} 73 | : M? (ex_ty * Alpha_context.context) := 74 | let parse_parameter_ty := 'parse_parameter_ty in 75 | let parse_any_ty := 'parse_any_ty in 76 | [...] 77 | let? ctxt := Alpha_context.Gas.consume ctxt Typecheck_costs.parse_type_cycle 78 | in 79 | match 80 | (node, 81 | match node with 82 | | Micheline.Prim loc Michelson_v1_primitives.T_big_map args _annot => 83 | allow_lazy_storage 84 | | _ => false 85 | end, 86 | match node with 87 | | 88 | Micheline.Prim _loc Michelson_v1_primitives.T_sapling_state 89 | (cons memo_size []) _annot => allow_lazy_storage 90 | | _ => false 91 | end) with 92 | | (Micheline.Prim _loc Michelson_v1_primitives.T_unit [] _annot, _, _) => 93 | return? ((Ex_ty Script_typed_ir.Unit_t), ctxt) 94 | | (Micheline.Prim _loc Michelson_v1_primitives.T_int [] _annot, _, _) => 95 | return? ((Ex_ty Script_typed_ir.Int_t), ctxt) 96 | | (Micheline.Prim _loc Michelson_v1_primitives.T_nat [] _annot, _, _) => 97 | return? ((Ex_ty Script_typed_ir.Nat_t), ctxt) 98 | | (Micheline.Prim _loc Michelson_v1_primitives.T_string [] _annot, _, _) => 99 | return? ((Ex_ty Script_typed_ir.String_t), ctxt) 100 | [...] 101 | 102 | where "'parse_parameter_ty" := 103 | (fun (ctxt : Alpha_context.context) (legacy : bool) => 104 | parse_ty ctxt legacy true false true true) 105 | 106 | and "'parse_any_ty" := 107 | [...] 108 | 109 | Definition parse_parameter_ty := 'parse_parameter_ty. 110 | Definition parse_any_ty := 'parse_any_ty. 111 | [...] 112 | 113 | We define `parse_parameter_ty` as a notation `'parse_parameter_ty`. We introduce an alias `parse_parameter_ty` as a standard definition at the end, so that the code depending on `parse_parameter_ty` does not need to know that this is actually a notation. Using the notation, we consider `parse_parameter_ty` as a shorthand to call `parse_ty` rather than a whole new function. This can also simplifies our proofs as `parse_ty` is now a single recursive function. 114 | With the `@coq_struct` attribute, we specify that the parameter `node` is the one to recurse on. Even if the function `parse_ty` is not syntactically terminating for Coq (due to some flattening when parsing pair elements), it is important for the proofs to have a "resonable" `struct` parameter. This prevents the `simpl` tactic to diverge when doing proofs by symbolic evaluation. Here we choose the parameter `node` as it is different on each recursive call. 115 | 116 | 117 | ### GADTs 118 | Our current approach to translate [GADTs](https://caml.inria.fr/pub/docs/manual-ocaml/gadts.html) to Coq is to: 119 | 120 | * erase the type parameters; 121 | * use OCaml attributes to force the generation of dynamic casts in Coq when needed (these dynamic casts are axioms). 122 | 123 | Fortunately, for our experiment we did not need to use dynamic casts and the code generated without the type parameters for the GADTs was compiling just fine! 124 | 125 | ## The proofs 126 | All the proofs are accessible online on [coq-tezos-of-ocaml/src/Proto\_alpha/Proofs/Script\_ir\_translator](https://gitlab.com/nomadic-labs/coq-tezos-of-ocaml/-/tree/master/src/Proto_alpha/Proofs/Script_ir_translator). We wanted to verify the following property: 127 | 128 | forall term, parse (unparse term) = term 129 | 130 | for the terms of type `comparable_ty` and `ty`. For the type `ty`, we express this property as: 131 | 132 | Lemma parse_unparse_ty 133 | ctxt 134 | legacy 135 | allow_lazy_storage allow_operation allow_contract allow_ticket 136 | ty 137 | (H_ty 138 | : Script_typed_ir.Ty.is_valid 139 | legacy 140 | allow_lazy_storage allow_operation allow_contract allow_ticket 141 | ty = 142 | true 143 | ) 144 | : let unlimited_ctxt := Raw_context.with_unlimited_gas ctxt in 145 | (let? '(node, ctxt) := Script_ir_translator.unparse_ty unlimited_ctxt ty in 146 | Script_ir_translator.parse_ty 147 | ctxt 148 | legacy 149 | allow_lazy_storage allow_operation allow_contract allow_ticket 150 | node) = 151 | return? (Script_ir_translator.Ex_ty ty, unlimited_ctxt). 152 | 153 | The parameters `legacy`, `allow_lazy_storage`, `allow_operation`, `allow_contract`, `allow_ticket` are boolean flags of the parsing function. They are mainly there to allow or forbid some elements, such as the contracts or the tickets. The context `ctxt` represents the current state, and in particular contains a reference to the block-chain state. We replace it by: 154 | 155 | let unlimited_ctxt := Raw_context.with_unlimited_gas ctxt in 156 | 157 | which is the same context with an unlimited gas value. The gas is there to compute the execution cost for smart-contracts. By setting the gas as unlimited, we avoid having to reason about failures due to gas exhaustion, without too much loss of generality. The `unparse_ty` and `parse_ty` functions are in the error monad, whose basic operators are `return?` to return a success value and `let?` to bind two operations. 158 | 159 | We recursively express the pre-condition `Script_typed_ir.Ty.is_valid` on the type `ty`. This pre-condition depends on the same flags as the parsing function. It checks that the forbidden operations are not present, and that all the integers are in the expected intervals. 160 | 161 | We do the proof by induction on the value `ty`. The most complex case is the case of pairs because we flatten the pairs to lists of elements, so the induction is not direct. Apart from that case, most of the proof is dedicated to handling the error monad and unfolding the definitions. To get an idea, here is an extract of the Coq proof to handle the general case: 162 | 163 | destruct ty; unfold simple_unparse_ty; simpl; try reflexivity; 164 | repeat (rewrite simple_unparse_ty_eq; simpl); 165 | repeat (rewrite Comparable_ty.parse_unparse_comparable_ty; simpl); 166 | repeat (rewrite (parse_simple_unparse_ty ctxt); simpl); 167 | trivial; 168 | simpl in H_ty; try (rewrite Bool.andb_true_iff in H_ty; destruct H_ty); 169 | trivial. 170 | 171 | For each case, we try to do some symbolic evaluation with `simpl`, and apply the inductive hypothesis `parse_simple_unparse_ty` or lemma such as `parse_unparse_comparable_ty`. We also do some basic boolean manipulations with `andb_true_iff` for the pre-condition. 172 | 173 | Finally, we check that the pre-condition `Script_typed_ir.Ty.is_valid` is true for all the terms parsed by the function `parse_ty`. We express this property as follows: 174 | 175 | Lemma parse_is_valid 176 | ctxt 177 | legacy 178 | allow_lazy_storage allow_operation allow_contract allow_ticket 179 | node 180 | : let result := 181 | Script_ir_translator.parse_ty 182 | ctxt 183 | legacy 184 | allow_lazy_storage allow_operation allow_contract allow_ticket 185 | node in 186 | match result with 187 | | Pervasives.Ok (Script_ir_translator.Ex_ty ty, _) => 188 | Script_typed_ir.Ty.is_valid 189 | legacy 190 | allow_lazy_storage allow_operation allow_contract allow_ticket 191 | ty = 192 | true 193 | | _ => True 194 | end. 195 | 196 | The proof proceeds by induction on the parameter `node`. 197 | 198 | ## Conclusion 199 | We have seen that we can write basic formal proofs about the parser of smart-contracts of Tezos. We do that first by automatically translating the OCaml code to Coq, and then by doing the proofs in Coq. 200 | 201 | We will next focus on writing proofs on the parsing functions for the data of smart-contracts. These functions are slightly more involved than the parsing functions on the types, but follow the same structure. 202 | -------------------------------------------------------------------------------- /rss.xml.erb: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | <%= h blog.title %> 5 | 6 | <%= blog.url %> 7 | <%= Time.new.strftime("%a, %d %b %Y %H:%M:%S %z") %> 8 | 9 | <% blog.public_posts.each do |post| %> 10 | 11 | <%= h post.name %> 12 | <%= post.date.strftime("%a, %d %b %Y %H:%M:%S %z") %> 13 | <%= h post.html %> 14 | <%= "#{blog.url}#{u post.url}" %> 15 | <%= "#{blog.url}#{u post.url}" %> 16 | 17 | <% end %> 18 | 19 | 20 | -------------------------------------------------------------------------------- /static/artifacts/tezos-interface-in-coq/coqdoc.css: -------------------------------------------------------------------------------- 1 | body { padding: 0px 0px; 2 | margin: 0px 0px; 3 | background-color: white } 4 | 5 | #page { display: block; 6 | padding: 0px; 7 | margin: 0px; 8 | padding-bottom: 10px; } 9 | 10 | #header { display: block; 11 | position: relative; 12 | padding: 0; 13 | margin: 0; 14 | vertical-align: middle; 15 | border-bottom-style: solid; 16 | border-width: thin } 17 | 18 | #header h1 { padding: 0; 19 | margin: 0;} 20 | 21 | 22 | /* Contents */ 23 | 24 | #main{ display: block; 25 | padding: 10px; 26 | font-family: sans-serif; 27 | font-size: 100%; 28 | line-height: 100% } 29 | 30 | #main h1 { line-height: 95% } /* allow for multi-line headers */ 31 | 32 | #main a.idref:visited {color : #416DFF; text-decoration : none; } 33 | #main a.idref:link {color : #416DFF; text-decoration : none; } 34 | #main a.idref:hover {text-decoration : none; } 35 | #main a.idref:active {text-decoration : none; } 36 | 37 | #main a.modref:visited {color : #416DFF; text-decoration : none; } 38 | #main a.modref:link {color : #416DFF; text-decoration : none; } 39 | #main a.modref:hover {text-decoration : none; } 40 | #main a.modref:active {text-decoration : none; } 41 | 42 | #main .keyword { color : #cf1d1d } 43 | #main { color: black } 44 | 45 | .section { background-color: rgb(60%,60%,100%); 46 | padding-top: 13px; 47 | padding-bottom: 13px; 48 | padding-left: 3px; 49 | margin-top: 5px; 50 | margin-bottom: 5px; 51 | font-size : 175% } 52 | 53 | h2.section { background-color: rgb(80%,80%,100%); 54 | padding-left: 3px; 55 | padding-top: 12px; 56 | padding-bottom: 10px; 57 | font-size : 130% } 58 | 59 | h3.section { background-color: rgb(90%,90%,100%); 60 | padding-left: 3px; 61 | padding-top: 7px; 62 | padding-bottom: 7px; 63 | font-size : 115% } 64 | 65 | h4.section { 66 | /* 67 | background-color: rgb(80%,80%,80%); 68 | max-width: 20em; 69 | padding-left: 5px; 70 | padding-top: 5px; 71 | padding-bottom: 5px; 72 | */ 73 | background-color: white; 74 | padding-left: 0px; 75 | padding-top: 0px; 76 | padding-bottom: 0px; 77 | font-size : 100%; 78 | font-weight : bold; 79 | text-decoration : underline; 80 | } 81 | 82 | #main .doc { margin: 0px; 83 | font-family: sans-serif; 84 | font-size: 100%; 85 | line-height: 125%; 86 | max-width: 40em; 87 | color: black; 88 | padding: 10px; 89 | background-color: #90bdff } 90 | 91 | .inlinecode { 92 | display: inline; 93 | /* font-size: 125%; */ 94 | color: #666666; 95 | font-family: monospace } 96 | 97 | .doc .inlinecode { 98 | display: inline; 99 | font-size: 120%; 100 | color: rgb(30%,30%,70%); 101 | font-family: monospace } 102 | 103 | .doc .inlinecode .id { 104 | color: rgb(30%,30%,70%); 105 | } 106 | 107 | .inlinecodenm { 108 | display: inline; 109 | color: #444444; 110 | } 111 | 112 | .doc .code { 113 | display: inline; 114 | font-size: 120%; 115 | color: rgb(30%,30%,70%); 116 | font-family: monospace } 117 | 118 | .comment { 119 | display: inline; 120 | font-family: monospace; 121 | color: rgb(50%,50%,80%); 122 | } 123 | 124 | .code { 125 | display: block; 126 | /* padding-left: 15px; */ 127 | font-size: 110%; 128 | font-family: monospace; 129 | } 130 | 131 | table.infrule { 132 | border: 0px; 133 | margin-left: 50px; 134 | margin-top: 10px; 135 | margin-bottom: 10px; 136 | } 137 | 138 | td.infrule { 139 | font-family: monospace; 140 | text-align: center; 141 | /* color: rgb(35%,35%,70%); */ 142 | padding: 0px; 143 | line-height: 100%; 144 | } 145 | 146 | tr.infrulemiddle hr { 147 | margin: 1px 0 1px 0; 148 | } 149 | 150 | .infrulenamecol { 151 | color: rgb(60%,60%,60%); 152 | font-size: 80%; 153 | padding-left: 1em; 154 | padding-bottom: 0.1em 155 | } 156 | 157 | /* Pied de page */ 158 | 159 | #footer { font-size: 65%; 160 | font-family: sans-serif; } 161 | 162 | /* Identifiers: ) */ 163 | 164 | .id { display: inline; } 165 | 166 | .id[title="constructor"] { 167 | color: rgb(60%,0%,0%); 168 | } 169 | 170 | .id[title="var"] { 171 | color: rgb(40%,0%,40%); 172 | } 173 | 174 | .id[title="variable"] { 175 | color: rgb(40%,0%,40%); 176 | } 177 | 178 | .id[title="definition"] { 179 | color: rgb(0%,40%,0%); 180 | } 181 | 182 | .id[title="abbreviation"] { 183 | color: rgb(0%,40%,0%); 184 | } 185 | 186 | .id[title="lemma"] { 187 | color: rgb(0%,40%,0%); 188 | } 189 | 190 | .id[title="instance"] { 191 | color: rgb(0%,40%,0%); 192 | } 193 | 194 | .id[title="projection"] { 195 | color: rgb(0%,40%,0%); 196 | } 197 | 198 | .id[title="method"] { 199 | color: rgb(0%,40%,0%); 200 | } 201 | 202 | .id[title="inductive"] { 203 | color: rgb(0%,0%,80%); 204 | } 205 | 206 | .id[title="record"] { 207 | color: rgb(0%,0%,80%); 208 | } 209 | 210 | .id[title="class"] { 211 | color: rgb(0%,0%,80%); 212 | } 213 | 214 | .id[title="keyword"] { 215 | color : #cf1d1d; 216 | /* color: black; */ 217 | } 218 | 219 | /* Deprecated rules using the 'type' attribute of (not xhtml valid) */ 220 | 221 | .id[type="constructor"] { 222 | color: rgb(60%,0%,0%); 223 | } 224 | 225 | .id[type="var"] { 226 | color: rgb(40%,0%,40%); 227 | } 228 | 229 | .id[type="variable"] { 230 | color: rgb(40%,0%,40%); 231 | } 232 | 233 | .id[type="definition"] { 234 | color: rgb(0%,40%,0%); 235 | } 236 | 237 | .id[type="abbreviation"] { 238 | color: rgb(0%,40%,0%); 239 | } 240 | 241 | .id[type="lemma"] { 242 | color: rgb(0%,40%,0%); 243 | } 244 | 245 | .id[type="instance"] { 246 | color: rgb(0%,40%,0%); 247 | } 248 | 249 | .id[type="projection"] { 250 | color: rgb(0%,40%,0%); 251 | } 252 | 253 | .id[type="method"] { 254 | color: rgb(0%,40%,0%); 255 | } 256 | 257 | .id[type="inductive"] { 258 | color: rgb(0%,0%,80%); 259 | } 260 | 261 | .id[type="record"] { 262 | color: rgb(0%,0%,80%); 263 | } 264 | 265 | .id[type="class"] { 266 | color: rgb(0%,0%,80%); 267 | } 268 | 269 | .id[type="keyword"] { 270 | color : #cf1d1d; 271 | /* color: black; */ 272 | } 273 | 274 | .inlinecode .id { 275 | color: rgb(0%,0%,0%); 276 | } 277 | 278 | 279 | /* TOC */ 280 | 281 | #toc h2 { 282 | padding: 10px; 283 | background-color: rgb(60%,60%,100%); 284 | } 285 | 286 | #toc li { 287 | padding-bottom: 8px; 288 | } 289 | 290 | /* Index */ 291 | 292 | #index { 293 | margin: 0; 294 | padding: 0; 295 | width: 100%; 296 | } 297 | 298 | #index #frontispiece { 299 | margin: 1em auto; 300 | padding: 1em; 301 | width: 60%; 302 | } 303 | 304 | .booktitle { font-size : 140% } 305 | .authors { font-size : 90%; 306 | line-height: 115%; } 307 | .moreauthors { font-size : 60% } 308 | 309 | #index #entrance { 310 | text-align: center; 311 | } 312 | 313 | #index #entrance .spacer { 314 | margin: 0 30px 0 30px; 315 | } 316 | 317 | #index #footer { 318 | position: absolute; 319 | bottom: 0; 320 | } 321 | 322 | .paragraph { 323 | height: 0.75em; 324 | } 325 | 326 | ul.doclist { 327 | margin-top: 0em; 328 | margin-bottom: 0em; 329 | } 330 | -------------------------------------------------------------------------------- /static/favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clarus/coq-blog/40c15b7c5ccb3e9c8011065863955c11483b51a2/static/favicon.png -------------------------------------------------------------------------------- /static/images/coq-of-ocaml-multiple-errors/report.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clarus/coq-blog/40c15b7c5ccb3e9c8011065863955c11483b51a2/static/images/coq-of-ocaml-multiple-errors/report.png -------------------------------------------------------------------------------- /static/images/cybele_comparison.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 19 | 21 | 28 | 34 | 35 | 42 | 48 | 49 | 50 | 75 | 84 | 85 | 87 | 88 | 90 | image/svg+xml 91 | 93 | 94 | 95 | 96 | 97 | 102 | 110 | 113 | 120 | 127 | 128 | Efficiency 149 | Correctness 160 | 167 | LTac 178 | 185 | Proof by reflection 196 | 199 | 206 | OCaml plugin 217 | 218 | 225 | Proof by extraction 236 | 237 | 238 | -------------------------------------------------------------------------------- /static/images/opam-bench-gitter/report.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clarus/coq-blog/40c15b7c5ccb3e9c8011065863955c11483b51a2/static/images/opam-bench-gitter/report.png -------------------------------------------------------------------------------- /static/images/pluto_runtime.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 18 | 20 | 27 | 33 | 34 | 41 | 47 | 48 | 55 | 61 | 62 | 63 | 91 | 101 | 102 | 104 | 105 | 107 | image/svg+xml 108 | 110 | 111 | 112 | 113 | 114 | 119 | 127 | Pluto 138 | Coq 149 | 156 | run 167 | extraction 179 | pipe 190 | Proxy 201 | Pluto 212 | OCaml 223 | 230 | 236 | 244 | 247 | Unix 258 | 266 | 267 | 274 | OCaml 285 | 286 | 287 | -------------------------------------------------------------------------------- /static/images/travis-ci/build-report.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clarus/coq-blog/40c15b7c5ccb3e9c8011065863955c11483b51a2/static/images/travis-ci/build-report.png -------------------------------------------------------------------------------- /static/images/travis-ci/error-logs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clarus/coq-blog/40c15b7c5ccb3e9c8011065863955c11483b51a2/static/images/travis-ci/error-logs.png -------------------------------------------------------------------------------- /static/images/travis-ci/pull-request.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/clarus/coq-blog/40c15b7c5ccb3e9c8011065863955c11483b51a2/static/images/travis-ci/pull-request.png -------------------------------------------------------------------------------- /templates/footer.html.erb: -------------------------------------------------------------------------------- 1 | 2 |
3 |
4 |

5 | RSS feed 6 |

7 |

8 | Sources are on GitHub. © Guillaume Claret 9 |

10 |
11 | 12 | 13 | 19 | 20 | 21 | -------------------------------------------------------------------------------- /templates/header.html.erb: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | <%= title %> 7 | 8 | 9 | 10 | 11 | 12 | 13 | 17 | 18 | 24 | 25 | 26 |
27 | 32 |
33 | -------------------------------------------------------------------------------- /templates/post.html.erb: -------------------------------------------------------------------------------- 1 | <%= header(blog, h(post.name)) %> 2 | 3 |
4 |
5 |
6 | <% if post.wip? %> 7 | 10 | <% end %> 11 |

<%= h post.name %>

12 |

🐓 <%= post.date_string %>

13 | <%= post.html %> 14 | 15 |
16 | 25 | 26 | blog comments powered by Disqus 27 |
28 |
29 |
30 | 31 | <%= footer %> 32 | -------------------------------------------------------------------------------- /wip.html.erb: -------------------------------------------------------------------------------- 1 | <%= header(blog, h(blog.title)) %> 2 | 3 |
4 |
5 |
6 |

WIP articles

7 |
    8 | <% blog.wip_posts.each do |post| %> 9 |
  • <%= h post.name %> <%= post.date_string %>
    • 10 | <% end %> 11 |
12 |
13 |
14 |
15 | 16 | <%= footer %> 17 | --------------------------------------------------------------------------------