├── CNAME ├── .gitignore ├── _pages ├── try.md ├── 404.md ├── install.md ├── blog.html └── examples.md ├── _includes ├── .#open-source-logo.html ├── button.html ├── open-source-link.html ├── image.html ├── nav.html ├── toc.html ├── header.html ├── footer.html └── head.html ├── _layouts ├── docs.html ├── external_article.html ├── home.html ├── page.html ├── base.html └── blog.html ├── assets ├── imgs │ ├── JS_logo.png │ ├── OCaml_logo.png │ ├── dune-2.0.0.png │ ├── dune_logo.png │ ├── dune-describe.png │ ├── dune-lambdas.png │ ├── dune_favicon.png │ ├── dune_mobile_logo.png │ ├── github_logo_dark.png │ ├── js_open_source.png │ ├── dune-retreat-2020.jpeg │ └── jbuilder-deprecated.png ├── js │ └── scripts.js └── styles │ └── main.scss ├── _data ├── footer.yml ├── authors.yml └── nav.yml ├── _posts ├── 2018-09-06-dune-1-2-0.md ├── 2018-08-02-dune-1-1-0.md ├── 2018-12-19-virtual-libraries.md ├── 2019-01-30-second-step-deprecation.md ├── 2019-04-10-dune-1-9-0.md ├── 2020-03-11-dune-describe.md ├── 2021-03-05-dune-3-plan.md ├── 2019-01-03-configurator-constants.md ├── 2018-12-14-new-computation-model.md ├── 2019-07-16-dune-2-coming-soon.md └── 2020-02-26-dune-retreat-2020.md ├── _sass ├── _page.scss ├── _settings.scss ├── _blog.scss ├── _footer.scss ├── _docs.scss ├── _styles.scss ├── _header.scss └── _syntax.scss ├── Gemfile ├── _config.yml ├── README.md ├── index.md └── Gemfile.lock /CNAME: -------------------------------------------------------------------------------- 1 | dune.build -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | _site 2 | .bundle 3 | vendor 4 | -------------------------------------------------------------------------------- /_pages/try.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Try Dune 3 | --- 4 | -------------------------------------------------------------------------------- /_includes/.#open-source-logo.html: -------------------------------------------------------------------------------- 1 | ynagai@igm-qws-u12712a.28375:1520062067 -------------------------------------------------------------------------------- /_layouts/docs.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | category: docs 4 | --- 5 | {{ content }} 6 | -------------------------------------------------------------------------------- /_pages/404.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Page Not Found 3 | tagline: The requested page could not be found. 4 | --- 5 | -------------------------------------------------------------------------------- /assets/imgs/JS_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/JS_logo.png -------------------------------------------------------------------------------- /assets/imgs/OCaml_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/OCaml_logo.png -------------------------------------------------------------------------------- /assets/imgs/dune-2.0.0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune-2.0.0.png -------------------------------------------------------------------------------- /assets/imgs/dune_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune_logo.png -------------------------------------------------------------------------------- /assets/imgs/dune-describe.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune-describe.png -------------------------------------------------------------------------------- /assets/imgs/dune-lambdas.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune-lambdas.png -------------------------------------------------------------------------------- /assets/imgs/dune_favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune_favicon.png -------------------------------------------------------------------------------- /assets/imgs/dune_mobile_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune_mobile_logo.png -------------------------------------------------------------------------------- /assets/imgs/github_logo_dark.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/github_logo_dark.png -------------------------------------------------------------------------------- /assets/imgs/js_open_source.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/js_open_source.png -------------------------------------------------------------------------------- /assets/imgs/dune-retreat-2020.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/dune-retreat-2020.jpeg -------------------------------------------------------------------------------- /_layouts/external_article.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 | 5 | -------------------------------------------------------------------------------- /assets/imgs/jbuilder-deprecated.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Leonidas-from-XIV/dune-www/master/assets/imgs/jbuilder-deprecated.png -------------------------------------------------------------------------------- /_includes/button.html: -------------------------------------------------------------------------------- 1 | {% if include.url %}{% endif %}
{{ include.text }}
{% if include.url %}
{% endif %} 2 | -------------------------------------------------------------------------------- /_data/footer.yml: -------------------------------------------------------------------------------- 1 | # Main nav links 2 | - title: Documentation 3 | desc: Get started & learn more 4 | url: https://dune.readthedocs.io 5 | - title: Github 6 | desc: Browse source, file and discuss issues 7 | url: https://github.com/ocaml/dune 8 | -------------------------------------------------------------------------------- /_data/authors.yml: -------------------------------------------------------------------------------- 1 | - id: jeremiedimino 2 | name: Jeremie Dimino 3 | - id: rgrinb 4 | name: Rudi Grinberg 5 | - id: etiennemillon 6 | name: Etienne Millon 7 | - id: TheLortex 8 | name: Lucas Pluvinage 9 | - id: andreymokhov 10 | name: Andrey Mokhov 11 | -------------------------------------------------------------------------------- /_includes/open-source-link.html: -------------------------------------------------------------------------------- 1 | 7 | -------------------------------------------------------------------------------- /_data/nav.yml: -------------------------------------------------------------------------------- 1 | # Main nav links 2 | - title: Install 3 | url: /install 4 | - title: Examples 5 | url: /examples 6 | - title: Documentation 7 | url: https://dune.readthedocs.io 8 | - title: Discuss 9 | url: https://discuss.ocaml.org/tags/dune 10 | - title: Blog 11 | url: /blog 12 | -------------------------------------------------------------------------------- /_includes/image.html: -------------------------------------------------------------------------------- 1 | {% if include.url %}{% endif %}{% if include.url %}{% endif %} 2 | -------------------------------------------------------------------------------- /_layouts/home.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 | 8 | 9 |
10 | {{ content }} 11 |
12 | -------------------------------------------------------------------------------- /_layouts/page.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 |
5 | 9 | 10 |
11 | {{ content }} 12 |
13 |
14 | -------------------------------------------------------------------------------- /_pages/install.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Install Dune 3 | --- 4 | 5 | # Quickstart 6 | 7 | The recommended way to install dune is via the [opam package manager](https://opam.ocaml.org/): 8 | 9 | ```bash 10 | $ opam install dune 11 | ``` 12 | 13 | You can also build it manually with: 14 | 15 | ```bash 16 | $ make release 17 | $ make install 18 | ``` 19 | -------------------------------------------------------------------------------- /_posts/2018-09-06-dune-1-2-0.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Dune 1.2.0 3 | external_url: https://tarides.com/blog/2018-09-06-dune-1-2-0.html 4 | layout: external_article 5 | author: etiennemillon 6 | --- 7 | 8 | After a tiny but important patch release as 1.1.1, the dune team is thrilled to 9 | announce the release of dune 1.2.0! Here are some highlights of the new features 10 | in that version. 11 | -------------------------------------------------------------------------------- /_includes/nav.html: -------------------------------------------------------------------------------- 1 | 13 | -------------------------------------------------------------------------------- /_posts/2018-08-02-dune-1-1-0.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: What’s coming in Dune 1.1.0? 3 | external_url: http://rgrinberg.com/posts/dune-upcoming-1-1/ 4 | layout: external_article 5 | author: rgrinb 6 | --- 7 | 8 | Now that 1.0.0 has been released, the team took a break from dune affairs. 9 | However, now we’re back to work, and although we’re working at a more relaxed 10 | pace, we still have some interesting features in store. 11 | -------------------------------------------------------------------------------- /_posts/2018-12-19-virtual-libraries.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Virtual libraries 3 | external_url: http://rgrinberg.com/posts/virtual-libraries/ 4 | layout: external_article 5 | author: rgrinb 6 | --- 7 | 8 | Last week I finally managed to slog through the last remaining issues required 9 | to implement Virtual Libraries, and since it’s quite a milestone (slated for 10 | dune 1.7), I’d like to share my excitement for this feature. 11 | -------------------------------------------------------------------------------- /assets/js/scripts.js: -------------------------------------------------------------------------------- 1 | document.addEventListener("DOMContentLoaded", function() { 2 | 3 | var pageWrap = document.getElementsByClassName('page-wrap docs')[0]; 4 | if (pageWrap) { 5 | document.getElementById('nav-wrapper').addEventListener('click', toggleDocsNav); 6 | } 7 | 8 | }); 9 | 10 | function toggleDocsNav(e) { 11 | document.getElementsByClassName('page-wrap')[0].classList.toggle('open'); 12 | } 13 | -------------------------------------------------------------------------------- /assets/styles/main.scss: -------------------------------------------------------------------------------- 1 | --- 2 | # this ensures Jekyll reads the file to be transformed into CSS later 3 | --- 4 | /** Change below to each code library's theme colors **/ 5 | $theme-color: #884010; 6 | $header-bg: #FFFFFF; 7 | $button-border: #ffad1d; 8 | $footer-bg: #884010; 9 | $nav-border-top: #884040; 10 | 11 | @import "settings"; 12 | @import "header"; 13 | @import "footer"; 14 | @import "page"; 15 | @import "blog"; 16 | @import "syntax"; 17 | @import "docs"; 18 | 19 | @import "styles"; 20 | -------------------------------------------------------------------------------- /_layouts/base.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | {% include head.html %} 4 | 5 |
6 | {% include header.html %} 7 |
8 | {% if page.layout == 'docs' %}{% include toc.html %}{% endif %} 9 |
10 |
11 | {{ content }} 12 |
13 |
14 |
15 |
16 | {% include footer.html %} 17 | 18 | 19 | -------------------------------------------------------------------------------- /_sass/_page.scss: -------------------------------------------------------------------------------- 1 | main { 2 | .main-container { 3 | min-height: 250px; 4 | 5 | .page-header { 6 | text-align: center; 7 | max-width: 600px; 8 | margin: 40px auto; 9 | 10 | h1 { 11 | font-size: 48px; 12 | font-weight: bold; 13 | } 14 | 15 | small { 16 | font-size: 17px; 17 | } 18 | } 19 | 20 | p { 21 | line-height: 1.5em; 22 | margin-bottom: 15px; 23 | } 24 | 25 | ul { 26 | margin-bottom: 15px; 27 | } 28 | 29 | .flex-grid .col pre { 30 | width: auto; 31 | } 32 | } 33 | } 34 | -------------------------------------------------------------------------------- /_includes/toc.html: -------------------------------------------------------------------------------- 1 |
2 | 19 |
20 | -------------------------------------------------------------------------------- /_sass/_settings.scss: -------------------------------------------------------------------------------- 1 | /** Fonts **/ 2 | $main-font: 'Alright Sans LTV', 'Alright Sans', 'Helvetica Neue', Helvetica, Arial, sans-serif; 3 | $med-font: 'Alright Sans LTV Medium', 'Aliright Sans Medium', Helvetica, Arial, sans-serif; 4 | $light-font: 'Alright Sans LTV Light', 'Alright Sans Light', Helvetica, Arial, sans-serif; 5 | $code-font: 'Source Code Pro', Courier New, monospace; 6 | 7 | /** Colors **/ 8 | $black: #000000; 9 | $white: #ffffff; 10 | $dark-gray: #444444; 11 | $bg-gray: #ededed; 12 | 13 | 14 | /** Media query breakpoints **/ 15 | $lg-device: 1200px; 16 | $md-device: 992px; 17 | $sm-device: 768px; 18 | $mobile: 480px; 19 | -------------------------------------------------------------------------------- /_sass/_blog.scss: -------------------------------------------------------------------------------- 1 | .blog, 2 | article.blog-post { 3 | .post-header { 4 | h1 { 5 | margin-top: 6px; 6 | font-size: 20px; 7 | } 8 | h3 { 9 | font-size: 40px; 10 | } 11 | margin-bottom: 10px; 12 | } 13 | 14 | .post-content { 15 | margin-bottom: 40px; 16 | } 17 | 18 | .post-header-image { 19 | width: 100%; 20 | 21 | img { 22 | display: block; 23 | max-width: 80%; 24 | margin: 20px auto; 25 | } 26 | } 27 | 28 | .post-comments { 29 | margin-bottom: 20px; 30 | } 31 | } 32 | 33 | // Blog index page 34 | .blog .article-index article { 35 | margin-bottom: 40px; 36 | } 37 | 38 | // Individual blog post 39 | article.blog-post { 40 | margin: 40px 0; 41 | } 42 | -------------------------------------------------------------------------------- /_includes/header.html: -------------------------------------------------------------------------------- 1 | {% capture github_url %}{{ site.github_url }}{% endcapture %} 2 | {% capture logo %}{{ site.logo }}{% endcapture %} 3 | {% capture github_text %}Find {{ site.name }} on Github{% endcapture %} 4 | {% capture github_logo %}{{ site.github_logo }}{% endcapture %} 5 | 17 | -------------------------------------------------------------------------------- /_pages/blog.html: -------------------------------------------------------------------------------- 1 | --- 2 | title: Blog 3 | tagline: Read the latest updates! 4 | --- 5 |
6 | {% for post in site.posts %} 7 | 8 |
9 |
10 | {{ post.date | date: '%b %d, %Y' }} 11 | {% if post.author %} 12 | {% assign the_author = site.data.authors | where: 'id', post.author %} 13 | {% for author in the_author %}| By: {{ author.name }}{% endfor %} 14 | {% endif %} 15 |

{{ post.title }}

16 |
17 |

{{ post.content | strip_html | truncatewords: 25 }}

18 |
19 |
20 | {% endfor %} 21 |
22 | -------------------------------------------------------------------------------- /_includes/footer.html: -------------------------------------------------------------------------------- 1 | 25 | -------------------------------------------------------------------------------- /_includes/head.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | {{ page.title | default: site.title }} 4 | 5 | {% if page.description %}{% endif %} 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | {% if page.picture %} 15 | 16 | 17 | {% endif %} 18 | 19 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source "https://rubygems.org" 2 | 3 | # Hello! This is where you manage which Jekyll version is used to run. 4 | # When you want to use a different version, change it below, save the 5 | # file and run `bundle install`. Run Jekyll with `bundle exec`, like so: 6 | # 7 | # bundle exec jekyll serve 8 | # 9 | # This will help ensure the proper Jekyll version is running. 10 | # Happy Jekylling! 11 | # gem "jekyll", "~> 3.6.2" 12 | 13 | # This is the default theme for new Jekyll sites. You may change this to anything you like. 14 | # gem "slate" 15 | 16 | # If you want to use GitHub Pages, remove the "gem "jekyll"" above and 17 | # uncomment the line below. To upgrade, run `bundle update github-pages`. 18 | gem "github-pages", group: :jekyll_plugins 19 | 20 | # If you have any plugins, put them here! 21 | group :jekyll_plugins do 22 | gem "jekyll-feed", "~> 0.6" 23 | gem "jekyll-postfiles" 24 | end 25 | 26 | # Windows does not include zoneinfo files, so bundle the tzinfo-data gem 27 | gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby] 28 | 29 | gem "webrick", "~> 1.7" 30 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | name: Dune 2 | description: A composable build system for OCaml 3 | url: https://dune.build 4 | baseurl: / 5 | permalink: /:title 6 | 7 | port: 2020 8 | 9 | timezone: UTC 10 | encoding: utf-8 11 | media_folder: /assets/imgs 12 | 13 | logo: "dune_logo.png" 14 | favicon: "dune_favicon.png" 15 | github_url: "https://github.com/ocaml/dune" 16 | github_logo: "github_logo_dark.png" 17 | 18 | defaults: 19 | - scope: 20 | path: "_posts" 21 | type: "posts" 22 | values: 23 | layout: "post" 24 | - scope: 25 | path: _pages 26 | type: pages 27 | values: 28 | layout: page 29 | permalink: "/:title" 30 | 31 | collections: 32 | pages: 33 | output: true 34 | permalink: /:title 35 | docs: 36 | output: true 37 | category: "docs" 38 | permalink: /:collection/:title 39 | posts: 40 | output: true 41 | permalink: /blog/:title/ 42 | 43 | sass: 44 | sass_dir: _sass 45 | style: compress 46 | 47 | markdown: kramdown 48 | kramdown: 49 | input: GFM 50 | syntax_highlighter: rouge 51 | parse_block_html: true 52 | 53 | exclude: 54 | - CNAME 55 | - README.md 56 | - Gemfile 57 | - Gemfile.lock 58 | - vendor 59 | -------------------------------------------------------------------------------- /_layouts/blog.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 |
5 |
6 |

{{ page.title }}

7 | {{ page.date | date: '%b %d, %Y' }} 8 | {% if page.author %}{% assign the_author = site.data.authors | where: 'id', page.author %} 9 | {% for author in the_author %}| By: 10 | {{ author.name }} 11 | {% endfor %} 12 | {% endif %} 13 |
14 | 15 | {% if page.picture %} 16 |
17 | 18 |
19 | {% endif %} 20 | 21 |
22 | {{ content }} 23 |
24 | 25 | {% if page.discuss %} 26 |
27 | Did you enjoy this article? You can let us know what you think on 28 | the corresponding thread on 29 | discuss.ocaml.org. 30 |
31 | {% endif %} 32 | 33 | {% if page.tags.size > 0 %} 34 |

Tag{% if page.tags.size > 1 %}s{% endif %}: {{ page.tags | sort | join: ", " }}

35 | {% endif %} 36 |
37 | -------------------------------------------------------------------------------- /_sass/_footer.scss: -------------------------------------------------------------------------------- 1 | .site-footer { 2 | background-color: $footer-bg; 3 | padding: 20px 0; 4 | color: $white; 5 | z-index: 100; 6 | .col { 7 | h3 { 8 | font-family: $med-font; 9 | font-size: 32px; 10 | color: $white; 11 | } 12 | a { 13 | color: $white; 14 | font-weight: bold; 15 | } 16 | p { 17 | font-family: $light-font; 18 | } 19 | a, p { 20 | font-size: 19px; 21 | } 22 | ul { 23 | margin: 0; 24 | } 25 | ul li { 26 | list-style: none; 27 | } 28 | } 29 | 30 | .footer-logos { 31 | margin-top: 20px; 32 | text-align: center; 33 | img { 34 | margin: 15px; 35 | max-width: 130px; 36 | height: 29px; 37 | &:nth-child(2) { 38 | margin-right: 0; 39 | } 40 | } 41 | } 42 | } 43 | 44 | @media screen and (max-width: $sm-device) { 45 | .site-footer { 46 | .footer-logos { 47 | img:first-child { 48 | margin-left: 0; 49 | } 50 | } 51 | .col { 52 | margin-left: 0; 53 | margin-right: 0; 54 | h3 { 55 | font-size: 24px; 56 | } 57 | a, p { 58 | font-size: 17px; 59 | } 60 | } 61 | } 62 | } 63 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Welcome to GitHub Pages 2 | 3 | You can use the [editor on GitHub](https://github.com/ocaml/dune-www/edit/master/README.md) to maintain and preview the content for your website in Markdown files. 4 | 5 | Whenever you commit to this repository, GitHub Pages will run [Jekyll](https://jekyllrb.com/) to rebuild the pages in your site, from the content in your Markdown files. 6 | 7 | ### Markdown 8 | 9 | Markdown is a lightweight and easy-to-use syntax for styling your writing. It includes conventions for 10 | 11 | ```markdown 12 | Syntax highlighted code block 13 | 14 | # Header 1 15 | ## Header 2 16 | ### Header 3 17 | 18 | - Bulleted 19 | - List 20 | 21 | 1. Numbered 22 | 2. List 23 | 24 | **Bold** and _Italic_ and `Code` text 25 | 26 | [Link](url) and ![Image](src) 27 | ``` 28 | 29 | For more details see [GitHub Flavored Markdown](https://guides.github.com/features/mastering-markdown/). 30 | 31 | ### Jekyll Themes 32 | 33 | Your Pages site will use the layout and styles from the Jekyll theme you have selected in your [repository settings](https://github.com/ocaml/dune-www/settings). The name of this theme is saved in the Jekyll `_config.yml` configuration file. 34 | 35 | ### Support or Contact 36 | 37 | Having trouble with Pages? Check out our [documentation](https://help.github.com/categories/github-pages-basics/) or [contact support](https://github.com/contact) and we’ll help you sort it out. 38 | -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 |
5 | 6 | Dune is a build system for OCaml projects. 7 | Using it, you can build executables, libraries, run tests, and much more. 8 | 9 |
10 |
11 | #### Dune knows OCaml and its ecosystem 12 | Dune has rules that precisely capture how the OCaml toolchain works. 13 | It is able to interoperate with most of the existing tools like OPAM, merlin, 14 | reason, and `js_of_ocaml`. 15 | 16 | #### Dune is fast 17 | Dune works hard to do things once and in parallel, so both cold and incremental 18 | builds are way faster than traditional build systems. 19 | 20 | #### Dune is widely used 21 | Dune is used in both large projects and small libraries. 22 | About 40% of OPAM packages are built using Dune. 23 |
24 | 25 |
26 | #### `dune` 27 | ```scheme 28 | (executable 29 | (name hello_world) 30 | (libraries lwt.unix)) 31 | ``` 32 | 33 | #### `hello_world.ml` 34 | ```ocaml 35 | Lwt_main.run (Lwt_io.printf "Hello, world!\n") 36 | ``` 37 | 38 | #### Your shell 39 | 40 | ``` 41 | $ dune build hello_world.exe 42 | $ ./_build/default/hello_world.exe 43 | Hello, world! 44 | ``` 45 | 46 | *Note: Dune uniformly uses the `.exe` extension to build native executables, even on Unix where programs don't usually have a `.exe` extension.* 47 | 48 |
49 |
50 |
51 | -------------------------------------------------------------------------------- /_pages/examples.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Examples 3 | tagline: Dune files you can use for common cases 4 | --- 5 | 6 | The following examples come from the [dune quickstart], where more explanations 7 | can be found. 8 | 9 | [dune quickstart]: https://dune.readthedocs.io/en/latest/quick-start.html 10 | 11 | # Executable 12 | 13 | This will build `hello_world.ml` into an executable as 14 | `_build/default/hello_world.exe`. 15 | 16 | ```scheme 17 | (executable 18 | (name hello_world) 19 | (libraries lwt.unix)) 20 | ``` 21 | 22 | # Library 23 | 24 | ```scheme 25 | (library 26 | (name mylib) 27 | (libraries re lwt)) 28 | ``` 29 | 30 | The library will be composed of all the modules in the same directory. Outside 31 | of the library, module ``Foo`` will be accessible as ``Mylib.Foo``, unless you 32 | write an explicit ``mylib.ml`` file. 33 | 34 | This library can be made available as an opam package if you replace `(name 35 | mylib)` by `(public_name mylib)` and write a `mylib.opam` file. 36 | 37 | # Tests 38 | 39 | ```scheme 40 | (test 41 | (name my_test_program)) 42 | ``` 43 | 44 | With this, the tests can be run using `dune runtest`. 45 | 46 | It will run the test program (the main module is ``my_test_program.ml``) and 47 | error if it exits with a nonzero code. 48 | 49 | # Mix and match 50 | 51 | A typical project will have several `dune` files referring to each other. 52 | 53 | For example: 54 | 55 | ``` 56 | project 57 | ├── bin 58 | │   ├── dune 59 | │   └── mylib-demo.ml 60 | ├── lib 61 | │   ├── dune 62 | │   ├── mylib.ml 63 | │   ├── mylib.mli 64 | │   └── mylib.opam 65 | └── test 66 | ├── dune 67 | └── test_mylib.ml 68 | ``` 69 | 70 | - `bin/dune` defines an executable depending on `mylib` 71 | - `test/dune` defines a test executable depending on `mylib` 72 | 73 | Note that the `bin`, `lib`, `test` directory names are not meaningful to dune. 74 | -------------------------------------------------------------------------------- /_sass/_docs.scss: -------------------------------------------------------------------------------- 1 | .page-wrap.docs { 2 | main { 3 | display: flex; 4 | } 5 | .toc-wrapper { 6 | margin: 50px 30px 30px 4%; 7 | width: 275px; 8 | .toc { 9 | background: $white; 10 | padding: 20px; 11 | #nav-wrapper { 12 | position: relative; 13 | #hamburger { 14 | cursor: pointer; 15 | height: 50px; 16 | padding: 15px 20px; 17 | margin-bottom: 0; 18 | #hamburg-icon { 19 | display: none; 20 | margin-right: 10px; 21 | } 22 | } 23 | #nav-items { 24 | padding: 0 30px; 25 | } 26 | } 27 | ul li a { 28 | line-height: 1.5em; 29 | &.active { 30 | font-weight: bold; 31 | color: $theme-color; 32 | } 33 | } 34 | } 35 | } 36 | 37 | .main-container { 38 | max-width: 63%; 39 | flex: 1; 40 | } 41 | 42 | @media screen and (max-width: $sm-device) { 43 | // Mobile menu 44 | .toc-wrapper { 45 | background: $white; 46 | height: 40px; 47 | left: 0; 48 | right: 0; 49 | position: absolute; 50 | width: 100%; 51 | z-index: 100; 52 | margin: 0; 53 | .toc { 54 | padding: 0; 55 | border-bottom: 1px solid $theme-color; 56 | #nav-wrapper { 57 | #hamburger #hamburg-icon { 58 | display: inline-block; 59 | } 60 | #nav-items { 61 | display: none; 62 | } 63 | } 64 | } 65 | } 66 | .main-container { 67 | max-width: 100%; 68 | margin-top: 80px; 69 | } 70 | 71 | // Open mobile menu 72 | &.open .toc-wrapper { 73 | box-sizing: border-box; 74 | height: auto; 75 | overflow-y: auto; 76 | transition: .3s ease all; 77 | .toc #nav-wrapper #nav-items { 78 | display: block; 79 | padding-bottom: 20px; 80 | } 81 | } 82 | } 83 | } 84 | -------------------------------------------------------------------------------- /_sass/_styles.scss: -------------------------------------------------------------------------------- 1 | * { 2 | margin: 0; 3 | padding: 0; 4 | box-sizing: border-box; 5 | } 6 | 7 | html, body { 8 | height: 100%; 9 | font-family: $main-font; 10 | font-size: 100%; 11 | color: $dark-gray; 12 | text-rendering: optimizeLegibility; 13 | background-color: $bg-gray; 14 | } 15 | 16 | body { 17 | display: flex; 18 | flex-direction: column; 19 | scroll-behavior: smooth; 20 | } 21 | 22 | .page-wrap { 23 | flex: 1 0 auto; 24 | } 25 | 26 | .wrapper { 27 | margin: 0px auto; 28 | max-width: 900px; 29 | padding: 0 20px; 30 | } 31 | 32 | .flex-grid { 33 | display: flex; 34 | justify-content: space-between; 35 | align-items: stretch; 36 | } 37 | 38 | .flex-grid > * { 39 | margin: 10px; 40 | flex: 1 100%; 41 | } 42 | 43 | header a { 44 | text-decoration: none; 45 | } 46 | 47 | footer a { 48 | text-decoration: none; 49 | } 50 | 51 | a { 52 | color: $dark-gray; 53 | } 54 | 55 | p { 56 | font-size: 16px; 57 | } 58 | 59 | h1, h2, h3, h4, h5, h6 { 60 | margin-bottom: 6px; 61 | } 62 | 63 | main ul { 64 | margin-left: 20px; 65 | } 66 | 67 | .code-container { 68 | background-color: #f8f8f8; 69 | padding: 30px; 70 | } 71 | 72 | pre { 73 | font-family: $code-font; 74 | white-space: pre; 75 | margin-bottom: 20px; 76 | } 77 | 78 | .text-container { 79 | background: $white; 80 | padding: 30px; 81 | p:last-child { 82 | margin-bottom: 0; 83 | } 84 | } 85 | 86 | .section { 87 | margin: 50px 0; 88 | h1, h3 { 89 | text-align: center; 90 | } 91 | h1 { 92 | font-size: 48px; 93 | } 94 | h3 { 95 | font-size: 17px; 96 | font-weight: normal; 97 | margin-bottom: 20px; 98 | } 99 | } 100 | 101 | .button { 102 | display: inline-block; 103 | padding: 7px 12px 6px 12px; 104 | text-transform: uppercase; 105 | border: 2px solid $button-border; 106 | background: $header-bg; 107 | color: $black; 108 | font-size: 14px; 109 | font-weight: bold; 110 | margin: 0 2px; 111 | } 112 | 113 | @media screen and (max-width: $sm-device) { 114 | .flex-grid { 115 | display: block !important; 116 | } 117 | .flex-grid > * { 118 | margin: 10px 0; 119 | } 120 | article.blog-post .post-header h3, 121 | .section h1, 122 | main .main-container .page-header h1 { 123 | font-size: 30px; 124 | } 125 | } 126 | -------------------------------------------------------------------------------- /_sass/_header.scss: -------------------------------------------------------------------------------- 1 | header { 2 | text-align: center; 3 | padding-top: 50px; 4 | background-color: $header-bg; 5 | 6 | .wrapper { 7 | position: relative; 8 | } 9 | .header-logo { 10 | position: relative; 11 | margin-top: 20px; 12 | .logo { 13 | max-width: 380px; 14 | width: 100%; 15 | height: 100px; 16 | background-repeat: no-repeat; 17 | background-size: 100% 100%; 18 | margin: 0 auto; 19 | background-image: url(/assets/imgs/dune_logo.png); 20 | } 21 | } 22 | 23 | .open-source-link { 24 | position: absolute; 25 | top: -70px; 26 | cursor: pointer; 27 | .text { 28 | position: absolute; 29 | top: 20px; 30 | font-size: 14px; 31 | transition: .5s all ease; 32 | opacity: 0; 33 | width: 200px; 34 | } 35 | &:hover { 36 | .text { 37 | opacity: 1; 38 | -webkit-transform: translate(20px, 0); 39 | transform: translate(20px, 0); 40 | } 41 | } 42 | img { 43 | width: 50px; 44 | height: 50px; 45 | z-index: 99; 46 | } 47 | } 48 | 49 | .github-logo { 50 | max-width: 38px; 51 | position: relative; 52 | bottom: -15px; 53 | } 54 | 55 | .links-container { 56 | position: absolute; 57 | right: 20px; 58 | top: -70px; 59 | a { margin: 0 2px; } 60 | } 61 | 62 | .tagline { 63 | font-family: $light-font; 64 | margin-top: 20px; 65 | font-style: italic; 66 | font-size: 14px; 67 | color: $black; 68 | } 69 | 70 | nav { 71 | border-top: 1px solid $nav-border-top; 72 | border-bottom: 5px solid $theme-color; 73 | margin-top: 50px; 74 | padding: 8px 0; 75 | background-color: $white; 76 | ul { 77 | padding: 10px 0; 78 | li { 79 | list-style: none; 80 | display: inline-block; 81 | &.active a { 82 | border-bottom: 3px solid $theme-color; 83 | } 84 | a { 85 | padding-bottom: 6px; 86 | margin: 0 20px; 87 | font-size: 20px; 88 | text-transform: uppercase; 89 | color: $black; 90 | font-family: $med-font; 91 | } 92 | } 93 | } 94 | } 95 | } 96 | 97 | @media screen and (max-width: $sm-device) { 98 | header nav ul li a { 99 | margin: 0 8px; 100 | font-size: 14px; 101 | } 102 | header .header-logo .logo { 103 | background-image: url(/assets/imgs/dune_mobile_logo.png); 104 | height: 270px; 105 | width: 240px; 106 | } 107 | header .tagline { 108 | position: absolute; 109 | right: 0; 110 | left: 0; 111 | bottom: 0; 112 | } 113 | header .links-container span { 114 | display: none; 115 | } 116 | } 117 | -------------------------------------------------------------------------------- /_sass/_syntax.scss: -------------------------------------------------------------------------------- 1 | .highlighter-rouge { 2 | margin: 0; 3 | display: inline; 4 | } 5 | 6 | pre { 7 | background-color: #f8f8f8; 8 | padding: 6px 10px; 9 | border-radius: 2px; 10 | overflow-x:auto; 11 | } 12 | 13 | $olivegray: #999988; 14 | $palepink: #e3d2d2; 15 | $darkred: #a61717; 16 | $medgray: #999; 17 | 18 | .highlight { 19 | 20 | pre, 21 | .hll { 22 | @extend pre; 23 | } 24 | 25 | .c, 26 | .cm, 27 | .c1 { 28 | color: $olivegray; 29 | } 30 | 31 | .gr, 32 | .gt { 33 | color: $darkred; 34 | } 35 | 36 | .k, 37 | .o, 38 | .cp, 39 | .cs, 40 | .gs, 41 | .kc, 42 | .kd, 43 | .kn, 44 | .kp, 45 | .kr, 46 | .gu, 47 | .ne, 48 | .nf, 49 | .ow { 50 | font-weight: bold; 51 | } 52 | 53 | .cp, 54 | .cs, 55 | .gh { 56 | color: $medgray; 57 | } 58 | 59 | .gd, 60 | .gd .x, 61 | .gi, 62 | .gi .x { 63 | color: black; 64 | } 65 | 66 | .gd { 67 | background-color: $palepink; 68 | } 69 | 70 | .gd .x { 71 | background-color: #ffaaaa; 72 | } 73 | 74 | .gi { 75 | background-color: #ddffdd; 76 | 77 | .x { 78 | background-color: #aaffaa; 79 | } 80 | } 81 | 82 | .go { 83 | color: #888; 84 | } 85 | 86 | .gp { 87 | color: #555; 88 | } 89 | 90 | .gu { 91 | color: #800080; 92 | } 93 | 94 | .kt { 95 | color: #445588; 96 | font-weight: bold; 97 | } 98 | 99 | .m { 100 | color: #009999; 101 | } 102 | 103 | .s { 104 | color: #dd1144; 105 | } 106 | 107 | .n { 108 | color: #333; 109 | } 110 | 111 | .na { 112 | color: teal; 113 | } 114 | 115 | .nb { 116 | color: #0086b3; 117 | } 118 | 119 | .nc { 120 | color: #445588; 121 | font-weight: bold; 122 | } 123 | 124 | .no { 125 | color: teal; 126 | } 127 | 128 | .ni { 129 | color: purple; 130 | } 131 | 132 | .ne { 133 | color: #990000; 134 | } 135 | 136 | .nf { 137 | color: #990000; 138 | } 139 | 140 | .nn { 141 | color: #555555; 142 | } 143 | 144 | .nt { 145 | color: navy; 146 | } 147 | 148 | .nv { 149 | color: teal; 150 | } 151 | 152 | .w { 153 | color: #bbbbbb; 154 | } 155 | 156 | .mf { 157 | color: #009999; 158 | } 159 | 160 | .mh { 161 | color: #009999; 162 | } 163 | 164 | .mi { 165 | color: #009999; 166 | } 167 | 168 | .mo { 169 | color: #009999; 170 | } 171 | 172 | .sb { 173 | color: #dd1144; 174 | } 175 | 176 | .sc { 177 | color: #dd1144; 178 | } 179 | 180 | .sd { 181 | color: #dd1144; 182 | } 183 | 184 | .s2 { 185 | color: #dd1144; 186 | } 187 | 188 | .se { 189 | color: #dd1144; 190 | } 191 | 192 | .sh { 193 | color: #dd1144; 194 | } 195 | 196 | .si { 197 | color: #dd1144; 198 | } 199 | 200 | .sx { 201 | color: #dd1144; 202 | } 203 | 204 | .sr { 205 | color: #009926; 206 | } 207 | 208 | .s1 { 209 | color: #dd1144; 210 | } 211 | 212 | .ss { 213 | color: #990073; 214 | } 215 | 216 | .bp { 217 | color: $medgray; 218 | } 219 | 220 | .vc { 221 | color: teal; 222 | } 223 | 224 | .vg { 225 | color: teal; 226 | } 227 | 228 | .vi { 229 | color: teal; 230 | } 231 | 232 | .il { 233 | color: #009999; 234 | } 235 | 236 | .gc { 237 | color: #999; 238 | background-color: #EAF2F5; 239 | } 240 | } 241 | -------------------------------------------------------------------------------- /_posts/2019-01-30-second-step-deprecation.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: Second stage of the Jbuilder deprecation 4 | author: jeremiedimino 5 | tags: [ocaml, dune, jbuilder] 6 | picture: /assets/imgs/jbuilder-deprecated.png 7 | discuss: https://discuss.ocaml.org/t/second-stage-of-the-jbuilder-deprecation 8 | --- 9 | 10 | As [planned][migration], we are now entering the second stage of the 11 | Jbuilder deprecation. This means that the `jbuilder` binary still 12 | exists and work as before however it prints a warning on every 13 | startup. Additionally, `jbuild` files are still accepted but both 14 | `jbuilder` and `dune` display a warning when encountering them. This 15 | behavior is now in the development version of dune and will be part of 16 | the upcoming 1.7.0 release. 17 | 18 | The support for the `jbuilder` binary and `jbuild` files will be 19 | discontinued in July 2019. If you haven't switch your project to 20 | dune already, now is a good time to do it. 21 | 22 | To do that, you can either follow the [migration's guide][migration], 23 | or use the newly added `dune upgrade` command that does it for you. 24 | 25 | Using the automatic upgrader 26 | ---------------------------- 27 | 28 | The automatic upgrader is a new feature in dune 1.7.0. It automatically 29 | convert your project from jbuilder to dune. Using it is as simple as 30 | running the following command at the root of your project: 31 | 32 | ``` 33 | $ dune upgrade 34 | ``` 35 | 36 | Note that this command also work on workspaces composed of several 37 | projects. In this case, all projects in the workspace are converted. 38 | 39 | The upgrader crawls the workspace and automatically convert every 40 | `jbuild` and `jbuild-ignore` files it encounter to a `dune` file. It 41 | also creates `dune-project` files wherever appropriate. Note that the 42 | `dune` files are all pretty-printed, so the layout of your original 43 | `jbuild` will not be preserved. 44 | 45 | Additionally, `dune upgrade` also edits your `.opam` files as 46 | follow: 47 | 48 | - it updates the build instruction with the recommended calls to `dune` 49 | - it updates the dependencies on `jbuilder` by dependencies on `dune` 50 | 51 | Note that this is not a simple replacement of the word `jbuilder` by 52 | `dune`. In particular, the build instruction are replaced by the 53 | recommended ones. This is because in the early days of jbuilder, there 54 | were no recommended opam build instructions, and as a result some 55 | projects might still not be using the most appropriate 56 | instructions. You should review the edits performed by the upgrader to 57 | make sure you are happy with them. 58 | 59 | For the record, the recommended opam build instructions are the 60 | following ones: 61 | 62 | ``` 63 | build: [ 64 | ["dune" "subst"] {pinned} 65 | ["dune" "build" "-p" name "-j" jobs] 66 | ["dune" "runtest" "-p" name "-j" jobs] {with-test} 67 | ["dune" "build" "@doc" "-p" name "-j" jobs] {with-doc} 68 | ] 69 | ``` 70 | 71 | Interaction with git 72 | -------------------- 73 | 74 | The upgrader detects when it is running inside a git repository. In 75 | such cases, it issues git commands in such a way that running `git 76 | diff` just after `dune upgrade` will show you exactly what changed. If 77 | you do not like the result or notice a problem with the automatic 78 | upgrade, running `git reset --hard` will undo all the edits and remove 79 | the newly created `dune` and `dune-project` files. 80 | 81 | It is a good idea to run `dune build` before and after the upgrade to 82 | make sure everything works as expected. 83 | 84 | [migration]: https://dune.readthedocs.io/en/latest/migration.html 85 | -------------------------------------------------------------------------------- /_posts/2019-04-10-dune-1-9-0.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: Dune 1.9.0 4 | author: TheLortex 5 | tags: [ocaml, dune] 6 | --- 7 | 8 | # Dune 1.9.0 9 | 10 | The dune team is pleased to announce the dune 1.9.0 release which introduces the concept of library variants. Changes include: 11 | * Coloring in the watch mode ([#1956](https://github.com/ocaml/dune/pull/1956)) 12 | * `$ dune init` command to create or update project boilerplate ([#1448](https://github.com/ocaml/dune/pull/1448)) 13 | * Allow "." in c_names and cxx_names ([#2036](https://github.com/ocaml/dune/pull/2036)) 14 | * Experimental Coq support 15 | * Support for library variants and default implementations ([#1900](https://github.com/ocaml/dune/pull/1900)) 16 | 17 | ## Variants 18 | 19 | In dune 1.7.0, the concept of virtual library was introduced: https://dune.build/blog/virtual-libraries/. This feature allows to mark some abstract library as virtual, and then have several implementations for it. These implementations could be for multiple targets (`unix`, `xen`, `js`), using different algorithms, using C code or not. However each implementation in a project dependency tree had to be manually selected. Dune 1.9.0 introduces features for automatic selection of implementations. 20 | 21 | ### Library variants 22 | 23 | Variants is a tagging mechanism to select implementations on the final linking step. There's not much to add to make your implementation use variants. For example, you could decide to design a `bar_js` library which is the javascript implementation of `bar`, a virtual library. All you need to do is specificy a `js` tag using the `variant` option. 24 | ``` 25 | (library 26 | (name bar_js) 27 | (implements bar) 28 | (variant js)); <-- variant specification 29 | ``` 30 | 31 | Now any executable that depend on `bar` can automatically select the `bar_js` library variant using the `variants` option in the dune file. 32 | ``` 33 | (executable 34 | (name foo) 35 | (libraries bar baz) 36 | (variants js)); <-- variants selection 37 | ``` 38 | 39 | ### Common variants 40 | 41 | #### Language selection 42 | 43 | In your projects you might want to trade off speed for portability: 44 | * `ocaml`: pure OCaml 45 | * `c`: OCaml accelerated by C 46 | 47 | 48 | #### JavaScript backend 49 | 50 | * `js`: code aiming for a Node backend, using `Js_of_ocaml` 51 | 52 | 53 | ### Mirage backends 54 | 55 | The Mirage project ([mirage.io](https://mirage.io/)) will make extensive use of this feature in order to select the appropriate dependencies according to the selected backend. 56 | * `unix`: Unikernels as Unix applications, running on top of `mirage-unix` 57 | * `xen`: Xen backend, on top of `mirage-xen` 58 | * `freestanding`: Freestanding backend, on top of `mirage-solo5` 59 | 60 | ### Default implementation 61 | 62 | To facilitate the transition from normal libraries into virtuals ones, it's possible to specify an implementation that is selected by default. This default implementation is selected if no implementation is chosen after variant resolution. 63 | 64 | ``` 65 | (library 66 | (name bar) 67 | (virtual_modules hello) 68 | (default_implementation bar_unix)); <-- default implementation selection 69 | ``` 70 | 71 | 72 | ### Selection mechanism 73 | 74 | Implementation is done with respect to some priority rules: 75 | * manual selection of an implementation overrides everything 76 | * after that comes selection by variants 77 | * finally unimplemented virtual libraries can select their default implementation 78 | 79 | 80 | Libraries may depend on specific implementations but this is not recommended. In this case, several things can happen: 81 | * the implementation conflicts with a manually selected implementation: resolution fails. 82 | * the implementation overrides variants and default implementations: a cycle check is done and this either resolves or fails. 83 | 84 | ### Conclusion 85 | 86 | Variant libraries and default implementations are fully [documented here](https://dune.readthedocs.io/en/latest/variants.html). This feature improves the usability of virtual libraries. 87 | 88 | This [commit](https://github.com/dune-universe/mirage-entropy/commit/576d25d79e3117bba64355ae73597651cfd27631) shows the amount of changes needed to make a virtual library use variants. 89 | 90 | ## Coq support 91 | 92 | Dune now supports building Coq projects. To enable the experimental Coq 93 | extension, add `(using coq 0.1)` to your `dune-project` file. Then, 94 | you can use the `(coqlib ...)` stanza to declare Coq libraries. 95 | 96 | A typical `dune` file for a Coq project will look like: 97 | 98 | ``` 99 | (include_subdirs qualified) ; Use if your development is based on sub directories 100 | 101 | (coqlib 102 | (name Equations) ; Name of wrapper module 103 | (public_name equations.Equations) ; Generate an .install file 104 | (synopsis "Equations Plugin") ; Synopsis 105 | (libraries equations.plugin) ; ML dependencies (for plugins) 106 | (modules :standard \ IdDec) ; modules to build 107 | (flags -w -notation-override)) ; coqc flags 108 | ``` 109 | 110 | See the [documentation of the 111 | extension](https://dune.readthedocs.io/en/latest/coq.html) for more 112 | details. 113 | -------------------------------------------------------------------------------- /_posts/2020-03-11-dune-describe.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: Introducing dune describe 4 | author: jeremiedimino 5 | tags: [ocaml, dune] 6 | picture: /assets/imgs/dune-describe.png 7 | discuss: https://discuss.ocaml.org/t/introducing-dune-describe 8 | --- 9 | 10 | Dune 2.4.0 has a new `describe` command for extracting various 11 | information about a dune project or workspace in a stable and machine 12 | readable format. The goal is to allow third-party applications such as 13 | the trustworthy refactoring tool [rotor][rotor] to understand the 14 | topology of a dune workspace, locate various compilation artifacts and 15 | other things. 16 | 17 | At the time of writing this post, the query language accepted by this 18 | command and its output are not yet stabilised. We plan to stabilise it 19 | once someone wants to release a application that relies on `dune 20 | describe`. 21 | 22 | In the future, we also plan to add a nice human readable output 23 | allowing users to quickly understand what a dune project is about and 24 | what it has to offer. 25 | 26 | [rotor]: https://trustworthy-refactoring.gitlab.io/refactorer/ 27 | 28 | Usage 29 | ----- 30 | 31 | 32 | $ dune describe [--lang VERSION] [--format FORMAT] WHAT-TO-DESCRIBE 33 | 34 | Passing `--lang x.y` requests that dune behaves the same way as 35 | version `x.y` of the `dune` binary, no matter its current 36 | version. More precisely, the two promises we make are: 37 | 38 | - that `WHAT-TO-DESCRIBE` will be parsed in the same way as the 39 | version `x.y` of the `dune` binary 40 | - the output of this command will be the same as the version `x.y` of the 41 | `dune` binary 42 | 43 | Right now, the only version you can pass to `--lang` is 0.1. The "0." 44 | suffix means that the behavior is not yet stabilised and might change 45 | without notice in future minor releases or even bugfix releases of 46 | Dune. Once either rotor or another project is ready to make a stable 47 | release of a tool relying of `dune describe`, we'll stabilise this 48 | output. So if you plan on using this feature, please let us know! 49 | 50 | If you intend to parse the output of `dune describe` programmatically, 51 | then you should also pass `--format csexp`. This will cause `dune` to 52 | print the result as a [S-expression in canonical 53 | form][csexp-wiki]. You can then use the [csexp library][csexp-lib] to 54 | parse the output and use something like [ppx_sexp_conv][ppx_sexp_conv] 55 | to parse the result without hassle. 56 | 57 | [csexp-wiki]: https://en.wikipedia.org/wiki/Canonical_S-expressions 58 | [csexp-lib]: https://github.com/diml/csexp 59 | [ppx_sexp_conv]: https://github.com/janestreet/ppx_sexp_conv 60 | 61 | Example 62 | ------- 63 | 64 | ```ocaml 65 | # #require "csexp";; 66 | # #require "ppx_sexp_conv";; 67 | # #require "base";; 68 | # open Base;; 69 | # module Csexp = Csexp.Make(Sexp);; 70 | module Csexp : 71 | sig 72 | val parse_string : string -> (Sexp.t, int * string) result 73 | val parse_string_many : string -> (Sexp.t list, int * string) result 74 | val input : in_channel -> (Sexp.t, string) result 75 | val input_opt : in_channel -> (Sexp.t option, string) result 76 | val input_many : in_channel -> (Sexp.t list, string) result 77 | val serialised_length : Sexp.t -> int 78 | val to_string : Sexp.t -> string 79 | val to_buffer : Buffer.t -> Sexp.t -> unit 80 | val to_channel : out_channel -> Sexp.t -> unit 81 | end 82 | # type item = Library of library 83 | and library = 84 | { name : string 85 | ; uid : uid 86 | ; requires : uid list 87 | ; source_dir : string 88 | ; modules : module_ list 89 | } 90 | and uid = string 91 | and module_ = 92 | { name : string 93 | ; impl : string option 94 | ; intf : string option 95 | ; cmt : string option 96 | ; cmti : string option 97 | } 98 | [@@deriving of_sexp];; 99 | type item = Library of library 100 | and library = { 101 | name : string; 102 | uid : uid; 103 | requires : uid list; 104 | source_dir : string; 105 | modules : module_ list; 106 | } 107 | and uid = string 108 | and module_ = { 109 | name : string; 110 | impl : string option; 111 | intf : string option; 112 | cmt : string option; 113 | cmti : string option; 114 | } 115 | val item_of_sexp : Sexp.t -> item = 116 | val library_of_sexp : Sexp.t -> library = 117 | val uid_of_sexp : Sexp.t -> uid = 118 | val module__of_sexp : Sexp.t -> module_ = 119 | # Stdlib.Sys.command "dune describe --format csexp > /tmp/x";; 120 | - : int = 0 121 | # match Csexp.input (Stdlib.open_in "/tmp/x") with 122 | | Ok x -> [%of_sexp: item list] x 123 | | Error msg -> failwith msg;; 124 | - : item list = 125 | [Library 126 | {name = "csexp"; uid = "2ac224c04fa61d226241d5394bcfc444"; requires = []; 127 | source_dir = "_build/default/src"; 128 | modules = 129 | [{name = "Csexp"; impl = Some "_build/default/src/csexp.ml"; 130 | intf = Some "_build/default/src/csexp.mli"; 131 | cmt = Some "_build/default/src/.csexp.objs/byte/csexp.cmt"; 132 | cmti = Some "_build/default/src/.csexp.objs/byte/csexp.cmti"}]}; 133 | Library 134 | {name = "csexp_tests"; uid = "cae2600322b2b8a67c297c353294d54d"; 135 | requires = 136 | ["2ac224c04fa61d226241d5394bcfc444"; "a09aff04d7aac4201a8d17178505dc41"; 137 | "c31eace71b8050d2eaf1b2ac88be5a21"; "0004486e60e09317002ea3fc0f012c04"; 138 | "d785d8082259c31f19b0c9350988438d"]; 139 | source_dir = "_build/default/test"; 140 | modules = 141 | [{name = "Test"; impl = Some "_build/default/test/test.ml"; 142 | intf = None; 143 | cmt = 144 | Some 145 | "_build/default/test/.csexp_tests.objs/byte/csexp_tests__Test.cmt"; 146 | cmti = None}; 147 | {name = "Csexp_tests"; 148 | impl = Some "_build/default/test/csexp_tests.ml-gen"; 149 | intf = None; 150 | cmt = 151 | Some 152 | "_build/default/test/.csexp_tests.objs/byte/csexp_tests.cmt"; 153 | cmti = None}]}] 154 | ``` 155 | -------------------------------------------------------------------------------- /_posts/2021-03-05-dune-3-plan.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: Plan for Dune 3.0 4 | author: andreymokhov 5 | tags: [dune] 6 | discuss: https://discuss.ocaml.org/t/plan-for-dune-3-0/7414 7 | --- 8 | 9 | The Dune team has started planning the next major release and we would like to 10 | share and discuss the current plan with the community. We expect to release 11 | Dune 3 in August 2021. Here is the 12 | [corresponding discuss.ocaml.org page](https://discuss.ocaml.org/t/plan-for-dune-3-0/7414). 13 | 14 | As with previous major releases, we put a lot of effort in backward 15 | compatibility. The majority of Dune users should not notice any breaking changes 16 | unless they explicitly opt into the new behaviour by increasing the version of 17 | the Dune language to 3.0 in the `dune-project` file. However, some of the 18 | changes that we plan to make are not specific to the way Dune deals with 19 | individual projects, and so they cannot be versioned through `dune-project` 20 | files. This means that a small number of packages may be broken by upgrading 21 | from Dune 2 to Dune 3. We are happy to help the authors of the affected packages 22 | to fix the breakage. 23 | 24 | ## What's new 25 | 26 | Here are the main features we plan to include in Dune 3: 27 | 28 | * **Incremental file-watching mode.** Dune already supports a basic 29 | file-watching mode, activated by the `-w` flag, where the build is restarted 30 | whenever any source file changes. The current implementation is 31 | non-incremental: restarting really means restarting the build from scratch, 32 | which, for example, involves reparsing all `dune` files. On small-size 33 | projects, this is fast enough, but this clearly doesn't scale well to large 34 | monorepos. We are making the file-watching mode fully incremental, to avoid 35 | redoing the computations whose inputs didn't change since the previous build. 36 | 37 | * **Dune RPC.** We are working on adding an RPC server to Dune that will make it 38 | possible to control Dune programmatically (instead of using the command line), 39 | which is often desirable for integration with various other tools, such as 40 | editors. 41 | 42 | * **Better support for warnings.** Currently, if Dune executes a build command 43 | that reports a warning but doesn't fail, that warning will only be shown to 44 | the user on the first build. Indeed, since the command succeeded, there is no 45 | need to rerun it again in subsequent builds unless its inputs change, and so 46 | the warning may go unnoticed and unfixed. This is why Dune turns warnings into 47 | errors by default, which can be annoying. Dune users (and we too) often 48 | complain about this, so we are working on better support for warnings. 49 | 50 | ## What we plan to change 51 | 52 | We plan to make two changes that affect the semantics of `dune` files in a 53 | backward-incompatible way. 54 | 55 | * **Better scoping semantics.** Dune provides some project-level scoping 56 | functionality, for example, it is possible to mark a library as private to 57 | prevent leaking the implementation details outside of a project. The current 58 | implementation of scoping has wrong semantics for nested vendored projects: a 59 | public library in the inner vendored project escapes its scope and is globally 60 | visible. This loophole caused problems for users, and we are going to remove 61 | it. This will be a breaking change but nested vendored projects are not very 62 | common, so this change will only affect a small number of users and in a 63 | positive way. 64 | 65 | * **Disallow conditional cycles.** Dune language makes it possible to declare 66 | optional libraries that depend on each other, as well as libraries that enable 67 | each other via `enabled_if`. The semantics of such conditional cycles had 68 | never been formalised. Furthermore, their existence significantly complicates 69 | Dune's library resolution logic. We would like to simplify this part of Dune 70 | and plan to disallow conditional cycles. This is a breaking change but we are 71 | not aware of any packages that would actually be broken by this, so please get 72 | in touch if you have any concerns. 73 | 74 | ## What we plan to remove 75 | 76 | We plan to remove the features listed below. Please let us know if you will be 77 | affected and will need some help with the transition to Dune 3. 78 | 79 | * **Support for opam 1.** We believe that it is not used any more (it [was 80 | deprecated in 2017](https://opam.ocaml.org/blog/deprecating-opam-1-2-0/)), so 81 | we'd like to remove the associated code. 82 | 83 | * **The external-lib-deps subcommand.** This subcommand prints out an 84 | approximate set of external libraries that are required for building given 85 | targets, without running the build. While this feature is useful, over time 86 | the quality of approximation has degraded and the cost of maintenance has 87 | increased. 88 | 89 | * **Automatic creation/editing of dune-project files.** If there is no 90 | `dune-project` file, Dune interprets the `dune` file using the latest version 91 | of the Dune language, which changes over time. It is therefore a good practice 92 | to create a `dune-project` file to freeze the semantics of the `dune` file to a specific version of the language. Dune used to provide some support for creating and updating `dune-project` files automatically but in practice this feature caused a fair amount of confusion for users, so we are removing it. 93 | 94 | * **Calling ocamlfind at runtime.** When running Dune outside of an opam 95 | environment, Dune invokes `ocamlfind` if it is present to figure out where to 96 | look for installed libraries. This is bad for reproducibility and doesn't feel 97 | useful anymore, so we plan to stop doing that. Dune will however continue to 98 | use the `OCAMLPATH` variable, as well as the search paths hard-coded at build 99 | time by the optional `./configure` script. 100 | 101 | ## Feedback 102 | 103 | This plan is not set in stone. If you don't like it, please let us know and 104 | we will see how we can improve it. 105 | 106 | Note that the Dune team is currently working on a few other features. They are 107 | not going to be synchronised with the Dune 3 release, which is why we didn't 108 | mention them above. If you are waiting for them, don't worry: they are still 109 | coming but not necessarily in Dune 3. And don't hesitate to remind us about your 110 | favourite feature requests — this helps us to set priorities. 111 | -------------------------------------------------------------------------------- /_posts/2019-01-03-configurator-constants.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: How configurator reads C constants 4 | tags: configurator 5 | author: etiennemillon 6 | --- 7 | 8 | Dune comes with a library to query OS-specific information, called 9 | [`configurator`][configurator-doc]. It is able to evaluate C expressions and 10 | turn them into OCaml value. Surprisingly, it even works when compiling for a 11 | different architecture. How can it do that? 12 | 13 | ## A CD-ROM problem 14 | 15 | Let's take an old school example: suppose we want to eject a CD-ROM drive. On 16 | Linux, the way to do that is to open the device file such as `/dev/cdrom` and to 17 | call `ioctl(fd, CDROMEJECT, 0)` on it. The `CDROMEJECT` part is a constant 18 | defined in ``. 19 | 20 | To do the same in OCaml, it is possible to define a C function that calls 21 | `ioctl` directly. Or this can be done directly using [ctypes], but we need to 22 | know the value of the `CDROMEJECT` constant; `configurator` can be used to do 23 | that. 24 | 25 | ## Enter `configurator` 26 | 27 | How to use `configurator` in a dune project is a bit out of scope for this 28 | article, but at the core is a function `C_define.import` that can read the value 29 | of some C expressions, including macros. 30 | 31 | The following program uses `configurator` to fetch and display the value of the 32 | `CDROMEJECT` constant. 33 | 34 | ```ocaml 35 | let () = 36 | let open Configurator.V1 in 37 | main ~name:"c_test" (fun t -> 38 | let result = 39 | C_define.import t ~includes:["linux/cdrom.h"] [("CDROMEJECT", Int)] 40 | in 41 | match result with 42 | | [(_, Int n)] -> Printf.printf "%d\n" n 43 | | _ -> assert false ) 44 | ``` 45 | 46 | Note that just getting the constants could be done by parsing the header files 47 | themselves. But this also supports constant C expressions (such as `1 << 8`) and 48 | some C features such as `sizeof(int)`. 49 | 50 | So, how does it work? 51 | 52 | ## An almost correct solution 53 | 54 | It is certainly necessary to generate and compile some C to do this. A first 55 | version is to generate a short C program such as the following one. 56 | 57 | ```c 58 | #include 59 | #include 60 | 61 | int main(void) 62 | { 63 | printf("%d\n", CDROMEJECT); 64 | return 0; 65 | } 66 | ``` 67 | 68 | By running this program and parsing the output, configurator can get the 69 | correct value. 70 | 71 | Except that dune supports cross-compilation: [when compiling an unikernel for an 72 | ESP32 CPU][esp32], it could be handy to have the value of constants such as 73 | `ESP_ERR_WIFI_PASSWORD` that are only available using a foreign toolchain. But 74 | it is not possible to run ESP32 binaries on the host system. 75 | 76 | ## A better solution 77 | 78 | Since it is necessary to use a C compiler, but not to run a program, the 79 | solution is looking at the compiled code: 80 | 81 | - generate a C file containing the expressions to extract 82 | - build it using the target C compiler 83 | - parse the resulting binary 84 | 85 | This is what `configurator` does. Since parsing compiled code is difficult (and 86 | not all targets use the same binary format), the values are stored in constant 87 | strings, between known markers. 88 | 89 | Here is the generated C file. Note that unlike in the previous attempt, this is 90 | not a complete executable, just a file to be built with `-c`. 91 | 92 | ```c 93 | #include 94 | #include 95 | 96 | #define D0(x) ('0'+(x/1 )%10) 97 | #define D1(x) ('0'+(x/10 )%10), D0(x) 98 | #define D2(x) ('0'+(x/100 )%10), D1(x) 99 | #define D3(x) ('0'+(x/1000 )%10), D2(x) 100 | #define D4(x) ('0'+(x/10000 )%10), D3(x) 101 | #define D5(x) ('0'+(x/100000 )%10), D4(x) 102 | #define D6(x) ('0'+(x/1000000 )%10), D5(x) 103 | #define D7(x) ('0'+(x/10000000 )%10), D6(x) 104 | #define D8(x) ('0'+(x/100000000 )%10), D7(x) 105 | #define D9(x) ('0'+(x/1000000000)%10), D8(x) 106 | 107 | const char s0[] = { 108 | 'B', 'E', 'G', 'I', 'N', '-', '0', '-', 109 | D9((CDROMEJECT)), 110 | '-', 'E', 'N', 'D' 111 | }; 112 | ``` 113 | 114 | The `Dn(x)` macros seem daunting at first, but remember that we need a string 115 | constant, so it is necessary to convert the integer value to a list of 116 | characters. The comma operator ensures that the result will look like `'1', '2', 117 | '3', '4'` which will be inserted in the array initializer. 118 | 119 | After compiling this file, the string is visible directly in the binary: 120 | 121 | ``` 122 | 00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000 .ELF............ 123 | 00000010: 0100 3e00 0100 0000 0000 0000 0000 0000 ..>............. 124 | 00000020: 0000 0000 0000 0000 a801 0000 0000 0000 ................ 125 | 00000030: 0000 0000 4000 0000 0000 4000 0a00 0900 ....@.....@..... 126 | 00000040: 4245 4749 4e2d 302d 3030 3030 3032 3132 BEGIN-0-00000212 127 | 00000050: 3537 2d45 4e44 0047 4343 3a20 2844 6562 57-END.GCC: (Deb 128 | 00000060: 6961 6e20 382e 322e 302d 3133 2920 382e ian 8.2.0-13) 8. 129 | 00000070: 322e 3000 0000 0000 0000 0000 0000 0000 2.0............. 130 | 00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 131 | 00000090: 0100 0000 0400 f1ff 0000 0000 0000 0000 ................ 132 | ``` 133 | 134 | It is even possible to parse it using plain Unix tools. 135 | 136 | ``` 137 | % strings x.o | grep BEGIN 138 | BEGIN-0-0000021257-END 139 | ``` 140 | 141 | The actual `configurator` library will parse it using a very simple 142 | lexer. It uses the number just after `BEGIN` (`-0-` above) to distinguish 143 | between the different constants that have been requested. 144 | 145 | It also supports more types of bindings, such as strings. In this case, the 146 | string is directly inserted between `BEGIN-0-` and `-END`. 147 | 148 | ## Conclusion 149 | 150 | Binary file formats can seem tricky to parse, but for some cases this is the 151 | correct solution. In the context of dune when it is not always possible to 152 | execute the output binaries, this is the correct solution to extract information 153 | from the target system. 154 | 155 | As far as I know, this technique has been borrowed from [ctypes][ctypes] where 156 | [it had been implemented by @whitequark][ctypes-pr]. Thanks! 157 | 158 | [configurator-doc]: https://dune.readthedocs.io/en/latest/configurator.html 159 | [ctypes]: https://github.com/ocamllabs/ocaml-ctypes/ 160 | [ctypes-pr]: https://github.com/ocamllabs/ocaml-ctypes/pull/383 161 | [esp32]: https://www.lortex.org/esp32/ 162 | -------------------------------------------------------------------------------- /_posts/2018-12-14-new-computation-model.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: A new general computation model for Dune 4 | author: jeremiedimino 5 | tags: [ocaml, dune] 6 | picture: /assets/imgs/dune-lambdas.png 7 | discuss: https://discuss.ocaml.org/t/blog-post-a-new-general-computation-model-for-dune/3061 8 | --- 9 | 10 | Dune is fast. However, if you try to use Dune to develop in a big 11 | workspace such as the [OCaml platform repository][platform], you will 12 | notice that on every run, even if there is nothing to rebuild `dune 13 | build` will take a couple of seconds to provide an answer. 14 | 15 | This might not seem like a big deal, however the bigger your workspace 16 | is and the longer this time will be. Just imagine having a workspace 17 | composed of all the open source OCaml software that exist out 18 | there. With such a workspace, you could achieve massive refactorings 19 | in one go, such as fixing historical mistake of the stdlib. For 20 | instance, we could finally make `compare` return a proper `Ordering.t` 21 | rather than an `int`. 22 | 23 | On such a repository, it is likely that `dune build` would 24 | systematically spend tens of seconds on every invocation. This is way 25 | too long to wait for feedback. So what exactly is Dune doing during 26 | all this time, and how can we speed things up? This post explains what 27 | the issue is and how we are planning to solve it in Dune. 28 | 29 | How Dune works 30 | -------------- 31 | 32 | To understand where this time is spent, let's analyse how Dune works. 33 | Roughly, you can see Dune as a system composed of two sub-systems 34 | running concurrently: a build script generator and an executor. The 35 | job of the build script generator is to read the user written 36 | configuration files and analyse the environment in order to procude a 37 | build script that can assemble the final application. The executor 38 | reads the build script and schedule the execution of commands using as 39 | much parallelism as possible. When possible, the executor will not 40 | re-execute commands in order to speed up incremental builds. 41 | 42 | In some tools, the distinction between the two is very clear. For 43 | instance in systems using ninja, ninja is the executor and there is a 44 | tool on top to interpret the user configuration and produce a build 45 | script. In Dune however, we chose to have both sub-systems in the same 46 | executable. This gives us better control of the overall system and 47 | allows us to use feedback from the execution of commands in order to 48 | produce more of the build script, enabling us to provide even more 49 | advanced features to our users. 50 | 51 | For small to medium sized projects, the generation of the build script 52 | is so fast that the user cannot notice it. However, for bigger 53 | workspaces, the generation of the build script can take quite a bit of 54 | time. 55 | 56 | This doesn't seem right at all; if there is one thing a build system 57 | should be good at, it is avoiding recomputing the same thing over and 58 | over. So why can't we do the same for the generation of the build 59 | script? We could indeed go through an external command to generate the 60 | build script, and this computation would effecitvely be cached by the 61 | system. However, this is an absolutely terrible programming model. This 62 | brings us to the new computation model we just merged in Dune, which 63 | will allow to cache any kind of computations, not just the ones 64 | performed by external commands. 65 | 66 | Towards a generic computation model 67 | ----------------------------------- 68 | 69 | When I first started writting Dune, there is one thing that bothered 70 | me quite a lot: the execution of external commands was properly cached 71 | by the system, however all the computations performed by the build 72 | system itself while generating the build script were not. In some way, 73 | these were second class computations. However, when you consider the 74 | overall job being done by the build system, i.e. turning source files 75 | into libraries and executables, the fact that some computations are 76 | performed by the build system and some are performed by external tools 77 | such as the compiler is just an implementation detail, so they should 78 | just all be treated the same way. 79 | 80 | It is this consideration that led to the new computation model at the 81 | core of Dune. The main idea is to come up with a model that can 82 | represent both running normal OCaml code or external commands and 83 | share results at any points chosen by the developer. 84 | 85 | It took some time to polish this model, develop it and and merge it 86 | into Dune. To validate our ideas, we discussed them with Andrey 87 | Mokhov, one of the authors of the famous [Build system a la carte 88 | paper][bs-a-la-carte]. The initial implementation was written by Rudi 89 | Horn and it is now fully merged into Dune. 90 | 91 | Let's see what this model is about. 92 | 93 | A general memoization system 94 | ---------------------------- 95 | 96 | When you think about it, a build system is nothing more that a system 97 | that can memoize pure computations. Indeed, when all the dependencies 98 | and targets of a command are fully specified, then we effectively get 99 | a pure function in the mathemical sense. This is why it is safe to not 100 | re-execute a command when none of its dependencies has changed. 101 | 102 | The new computation model simply generalizes this idea to all 103 | computations, not just external commands. At the core of Dune, we now 104 | have a system that is able to memoize any OCaml functions, running an 105 | external command being just one way to perform a computation. The 106 | result of memoized functions can be reused immediately during a single 107 | build, and can also be reused between different builds as long as it 108 | is safe to do so, i.e. when none of the dependencies of the function 109 | changed. 110 | 111 | Finding the right API 112 | --------------------- 113 | 114 | One limitation when using external commands is that once the command 115 | is started there is no interaction between the build system and the 116 | external process. So one has to effecively specify all the inputs and 117 | outputs upfront. This is tedious, and in many cases we have to specify 118 | a superset of what we think the inputs will be in order to be on the 119 | safe side. 120 | 121 | For internal computations, we have a bit more freedom. In particular, 122 | instead of manually specifying all the inputs we can simply discover 123 | them as the function is running. 124 | 125 | In the end, here is a simplifed version of what the API looks like: 126 | 127 | ```ocaml 128 | val memoize 129 | : name:string 130 | -> doc:string 131 | -> 'a input_ops 132 | -> 'b output_ops 133 | -> ('a -> 'b Fiber.t) 134 | -> ('a -> 'b Fiber.t) 135 | ``` 136 | 137 | `input_ops` and `output_ops` provide operations that the memoization 138 | system needs, such as hashing and equality functions. `Fiber.t` is the 139 | concurrency monad we use inside Dune. 140 | 141 | The first time the function returned by `memoize` is called, the 142 | memoization system will execute the actual function and record all the 143 | observations it makes, such as what files it reads or what other 144 | memoized functions it calls. During subsequent builds, if the function 145 | is called again with the same argument, the system will check whether 146 | the previous observations are still valid, and if yes it will reuse 147 | the previous result rather than execute the function again. 148 | 149 | This gives a very powerful and easy to use tool to build system 150 | developers to share computations in a single build or between builds. 151 | 152 | There are many ways to implement Dune's job on top of this 153 | abstraction. We were recently brainstorming about this with other Dune 154 | developers and we have many ideas on how to proceed. I feel like just 155 | as video game developers who need time to understand and fully exploit 156 | the power of new video game systems, it will take us some time to 157 | truly understand all the power of this new system and get the most 158 | benefits out of it. Which means that it is a truly exciting time to be 159 | hacking on Dune! 160 | 161 | Going further: better debugging and profiling 162 | --------------------------------------------- 163 | 164 | Memoized functions are not only used for sharing computations. These 165 | functions are annotated with a name, documentation and ways to display 166 | the inputs/outputs. This means that we can see precise stack traces, 167 | which should be really useful for debugging. Such functions should 168 | also be a good entry point for profiling reports, which should help 169 | understand better where the time is spent in a given build and how we 170 | can improve it. 171 | 172 | Finally, Dune also provides access to these functions via the CLI, 173 | which should be really nice for debugging: 174 | 175 | ``` 176 | $ dune compute 177 | 178 | ``` 179 | 180 | All this should help improve the quality of Dune and continue to 181 | improve the experience for users of Dune. 182 | 183 | [platform]: https://github.com/avsm/platform 184 | [bs-a-la-carte]: https://www.microsoft.com/en-us/research/publication/build-systems-la-carte/ 185 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | activesupport (6.0.6.1) 5 | concurrent-ruby (~> 1.0, >= 1.0.2) 6 | i18n (>= 0.7, < 2) 7 | minitest (~> 5.1) 8 | tzinfo (~> 1.1) 9 | zeitwerk (~> 2.2, >= 2.2.2) 10 | addressable (2.8.0) 11 | public_suffix (>= 2.0.2, < 5.0) 12 | coffee-script (2.4.1) 13 | coffee-script-source 14 | execjs 15 | coffee-script-source (1.11.1) 16 | colorator (1.1.0) 17 | commonmarker (0.23.7) 18 | concurrent-ruby (1.2.0) 19 | dnsruby (1.61.9) 20 | simpleidn (~> 0.1) 21 | em-websocket (0.5.3) 22 | eventmachine (>= 0.12.9) 23 | http_parser.rb (~> 0) 24 | ethon (0.15.0) 25 | ffi (>= 1.15.0) 26 | eventmachine (1.2.7) 27 | execjs (2.8.1) 28 | faraday (1.10.0) 29 | faraday-em_http (~> 1.0) 30 | faraday-em_synchrony (~> 1.0) 31 | faraday-excon (~> 1.1) 32 | faraday-httpclient (~> 1.0) 33 | faraday-multipart (~> 1.0) 34 | faraday-net_http (~> 1.0) 35 | faraday-net_http_persistent (~> 1.0) 36 | faraday-patron (~> 1.0) 37 | faraday-rack (~> 1.0) 38 | faraday-retry (~> 1.0) 39 | ruby2_keywords (>= 0.0.4) 40 | faraday-em_http (1.0.0) 41 | faraday-em_synchrony (1.0.0) 42 | faraday-excon (1.1.0) 43 | faraday-httpclient (1.0.1) 44 | faraday-multipart (1.0.3) 45 | multipart-post (>= 1.2, < 3) 46 | faraday-net_http (1.0.1) 47 | faraday-net_http_persistent (1.2.0) 48 | faraday-patron (1.0.0) 49 | faraday-rack (1.0.0) 50 | faraday-retry (1.0.3) 51 | ffi (1.15.5) 52 | forwardable-extended (2.6.0) 53 | gemoji (3.0.1) 54 | github-pages (226) 55 | github-pages-health-check (= 1.17.9) 56 | jekyll (= 3.9.2) 57 | jekyll-avatar (= 0.7.0) 58 | jekyll-coffeescript (= 1.1.1) 59 | jekyll-commonmark-ghpages (= 0.2.0) 60 | jekyll-default-layout (= 0.1.4) 61 | jekyll-feed (= 0.15.1) 62 | jekyll-gist (= 1.5.0) 63 | jekyll-github-metadata (= 2.13.0) 64 | jekyll-include-cache (= 0.2.1) 65 | jekyll-mentions (= 1.6.0) 66 | jekyll-optional-front-matter (= 0.3.2) 67 | jekyll-paginate (= 1.1.0) 68 | jekyll-readme-index (= 0.3.0) 69 | jekyll-redirect-from (= 0.16.0) 70 | jekyll-relative-links (= 0.6.1) 71 | jekyll-remote-theme (= 0.4.3) 72 | jekyll-sass-converter (= 1.5.2) 73 | jekyll-seo-tag (= 2.8.0) 74 | jekyll-sitemap (= 1.4.0) 75 | jekyll-swiss (= 1.0.0) 76 | jekyll-theme-architect (= 0.2.0) 77 | jekyll-theme-cayman (= 0.2.0) 78 | jekyll-theme-dinky (= 0.2.0) 79 | jekyll-theme-hacker (= 0.2.0) 80 | jekyll-theme-leap-day (= 0.2.0) 81 | jekyll-theme-merlot (= 0.2.0) 82 | jekyll-theme-midnight (= 0.2.0) 83 | jekyll-theme-minimal (= 0.2.0) 84 | jekyll-theme-modernist (= 0.2.0) 85 | jekyll-theme-primer (= 0.6.0) 86 | jekyll-theme-slate (= 0.2.0) 87 | jekyll-theme-tactile (= 0.2.0) 88 | jekyll-theme-time-machine (= 0.2.0) 89 | jekyll-titles-from-headings (= 0.5.3) 90 | jemoji (= 0.12.0) 91 | kramdown (= 2.3.2) 92 | kramdown-parser-gfm (= 1.1.0) 93 | liquid (= 4.0.3) 94 | mercenary (~> 0.3) 95 | minima (= 2.5.1) 96 | nokogiri (>= 1.13.4, < 2.0) 97 | rouge (= 3.26.0) 98 | terminal-table (~> 1.4) 99 | github-pages-health-check (1.17.9) 100 | addressable (~> 2.3) 101 | dnsruby (~> 1.60) 102 | octokit (~> 4.0) 103 | public_suffix (>= 3.0, < 5.0) 104 | typhoeus (~> 1.3) 105 | html-pipeline (2.14.1) 106 | activesupport (>= 2) 107 | nokogiri (>= 1.4) 108 | http_parser.rb (0.8.0) 109 | i18n (0.9.5) 110 | concurrent-ruby (~> 1.0) 111 | jekyll (3.9.2) 112 | addressable (~> 2.4) 113 | colorator (~> 1.0) 114 | em-websocket (~> 0.5) 115 | i18n (~> 0.7) 116 | jekyll-sass-converter (~> 1.0) 117 | jekyll-watch (~> 2.0) 118 | kramdown (>= 1.17, < 3) 119 | liquid (~> 4.0) 120 | mercenary (~> 0.3.3) 121 | pathutil (~> 0.9) 122 | rouge (>= 1.7, < 4) 123 | safe_yaml (~> 1.0) 124 | jekyll-avatar (0.7.0) 125 | jekyll (>= 3.0, < 5.0) 126 | jekyll-coffeescript (1.1.1) 127 | coffee-script (~> 2.2) 128 | coffee-script-source (~> 1.11.1) 129 | jekyll-commonmark (1.4.0) 130 | commonmarker (~> 0.22) 131 | jekyll-commonmark-ghpages (0.2.0) 132 | commonmarker (~> 0.23.4) 133 | jekyll (~> 3.9.0) 134 | jekyll-commonmark (~> 1.4.0) 135 | rouge (>= 2.0, < 4.0) 136 | jekyll-default-layout (0.1.4) 137 | jekyll (~> 3.0) 138 | jekyll-feed (0.15.1) 139 | jekyll (>= 3.7, < 5.0) 140 | jekyll-gist (1.5.0) 141 | octokit (~> 4.2) 142 | jekyll-github-metadata (2.13.0) 143 | jekyll (>= 3.4, < 5.0) 144 | octokit (~> 4.0, != 4.4.0) 145 | jekyll-include-cache (0.2.1) 146 | jekyll (>= 3.7, < 5.0) 147 | jekyll-mentions (1.6.0) 148 | html-pipeline (~> 2.3) 149 | jekyll (>= 3.7, < 5.0) 150 | jekyll-optional-front-matter (0.3.2) 151 | jekyll (>= 3.0, < 5.0) 152 | jekyll-paginate (1.1.0) 153 | jekyll-postfiles (3.1.0) 154 | jekyll (>= 3.8.6, < 5) 155 | jekyll-readme-index (0.3.0) 156 | jekyll (>= 3.0, < 5.0) 157 | jekyll-redirect-from (0.16.0) 158 | jekyll (>= 3.3, < 5.0) 159 | jekyll-relative-links (0.6.1) 160 | jekyll (>= 3.3, < 5.0) 161 | jekyll-remote-theme (0.4.3) 162 | addressable (~> 2.0) 163 | jekyll (>= 3.5, < 5.0) 164 | jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0) 165 | rubyzip (>= 1.3.0, < 3.0) 166 | jekyll-sass-converter (1.5.2) 167 | sass (~> 3.4) 168 | jekyll-seo-tag (2.8.0) 169 | jekyll (>= 3.8, < 5.0) 170 | jekyll-sitemap (1.4.0) 171 | jekyll (>= 3.7, < 5.0) 172 | jekyll-swiss (1.0.0) 173 | jekyll-theme-architect (0.2.0) 174 | jekyll (> 3.5, < 5.0) 175 | jekyll-seo-tag (~> 2.0) 176 | jekyll-theme-cayman (0.2.0) 177 | jekyll (> 3.5, < 5.0) 178 | jekyll-seo-tag (~> 2.0) 179 | jekyll-theme-dinky (0.2.0) 180 | jekyll (> 3.5, < 5.0) 181 | jekyll-seo-tag (~> 2.0) 182 | jekyll-theme-hacker (0.2.0) 183 | jekyll (> 3.5, < 5.0) 184 | jekyll-seo-tag (~> 2.0) 185 | jekyll-theme-leap-day (0.2.0) 186 | jekyll (> 3.5, < 5.0) 187 | jekyll-seo-tag (~> 2.0) 188 | jekyll-theme-merlot (0.2.0) 189 | jekyll (> 3.5, < 5.0) 190 | jekyll-seo-tag (~> 2.0) 191 | jekyll-theme-midnight (0.2.0) 192 | jekyll (> 3.5, < 5.0) 193 | jekyll-seo-tag (~> 2.0) 194 | jekyll-theme-minimal (0.2.0) 195 | jekyll (> 3.5, < 5.0) 196 | jekyll-seo-tag (~> 2.0) 197 | jekyll-theme-modernist (0.2.0) 198 | jekyll (> 3.5, < 5.0) 199 | jekyll-seo-tag (~> 2.0) 200 | jekyll-theme-primer (0.6.0) 201 | jekyll (> 3.5, < 5.0) 202 | jekyll-github-metadata (~> 2.9) 203 | jekyll-seo-tag (~> 2.0) 204 | jekyll-theme-slate (0.2.0) 205 | jekyll (> 3.5, < 5.0) 206 | jekyll-seo-tag (~> 2.0) 207 | jekyll-theme-tactile (0.2.0) 208 | jekyll (> 3.5, < 5.0) 209 | jekyll-seo-tag (~> 2.0) 210 | jekyll-theme-time-machine (0.2.0) 211 | jekyll (> 3.5, < 5.0) 212 | jekyll-seo-tag (~> 2.0) 213 | jekyll-titles-from-headings (0.5.3) 214 | jekyll (>= 3.3, < 5.0) 215 | jekyll-watch (2.2.1) 216 | listen (~> 3.0) 217 | jemoji (0.12.0) 218 | gemoji (~> 3.0) 219 | html-pipeline (~> 2.2) 220 | jekyll (>= 3.0, < 5.0) 221 | kramdown (2.3.2) 222 | rexml 223 | kramdown-parser-gfm (1.1.0) 224 | kramdown (~> 2.0) 225 | liquid (4.0.3) 226 | listen (3.7.1) 227 | rb-fsevent (~> 0.10, >= 0.10.3) 228 | rb-inotify (~> 0.9, >= 0.9.10) 229 | mercenary (0.3.6) 230 | mini_portile2 (2.8.1) 231 | minima (2.5.1) 232 | jekyll (>= 3.5, < 5.0) 233 | jekyll-feed (~> 0.9) 234 | jekyll-seo-tag (~> 2.1) 235 | minitest (5.17.0) 236 | multipart-post (2.1.1) 237 | nokogiri (1.14.0) 238 | mini_portile2 (~> 2.8.0) 239 | racc (~> 1.4) 240 | octokit (4.22.0) 241 | faraday (>= 0.9) 242 | sawyer (~> 0.8.0, >= 0.5.3) 243 | pathutil (0.16.2) 244 | forwardable-extended (~> 2.6) 245 | public_suffix (4.0.7) 246 | racc (1.6.2) 247 | rb-fsevent (0.11.1) 248 | rb-inotify (0.10.1) 249 | ffi (~> 1.0) 250 | rexml (3.2.5) 251 | rouge (3.26.0) 252 | ruby2_keywords (0.0.5) 253 | rubyzip (2.3.2) 254 | safe_yaml (1.0.5) 255 | sass (3.7.4) 256 | sass-listen (~> 4.0.0) 257 | sass-listen (4.0.0) 258 | rb-fsevent (~> 0.9, >= 0.9.4) 259 | rb-inotify (~> 0.9, >= 0.9.7) 260 | sawyer (0.8.2) 261 | addressable (>= 2.3.5) 262 | faraday (> 0.8, < 2.0) 263 | simpleidn (0.2.1) 264 | unf (~> 0.1.4) 265 | terminal-table (1.8.0) 266 | unicode-display_width (~> 1.1, >= 1.1.1) 267 | thread_safe (0.3.6) 268 | typhoeus (1.4.0) 269 | ethon (>= 0.9.0) 270 | tzinfo (1.2.11) 271 | thread_safe (~> 0.1) 272 | unf (0.1.4) 273 | unf_ext 274 | unf_ext (0.0.8.1) 275 | unicode-display_width (1.8.0) 276 | webrick (1.7.0) 277 | zeitwerk (2.6.6) 278 | 279 | PLATFORMS 280 | ruby 281 | 282 | DEPENDENCIES 283 | github-pages 284 | jekyll-feed (~> 0.6) 285 | jekyll-postfiles 286 | tzinfo-data 287 | webrick (~> 1.7) 288 | 289 | BUNDLED WITH 290 | 2.3.5 291 | -------------------------------------------------------------------------------- /_posts/2019-07-16-dune-2-coming-soon.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: Dune 2.0.0 coming soon! 4 | author: jeremiedimino 5 | tags: [ocaml, dune, jbuilder] 6 | picture: /assets/imgs/dune-2.0.0.png 7 | discuss: https://discuss.ocaml.org/t/dune-2-0-0-coming-soon/4102 8 | --- 9 | 10 | The dune team is currently preparing the 2.0.0 release of Dune and we 11 | wanted to share our plans for it. As always, we put a lot of effort 12 | into backward compatibility and we expect that the 2.0.0 release will 13 | be barely noticeable for existing users, just as the renaming from 14 | Jbuilder to Dune was. 15 | 16 | However, in order to keep the project modern and prepare it for the 17 | future, we are planning to do a couple of changes in Dune 2.0.0. We 18 | invite all Dune users to read this post in order to understand the 19 | changes and how these changes could affect them. If you do not have 20 | time to read this whole post, you can read [this quick 21 | summary](#quick-summary) instead. 22 | 23 | At a high level, we encourage everyone to switch to the Dune 2 binary 24 | for local development as soon as it is released. However, if you care 25 | a lot about compatibility with older OCaml compilers, we advise that 26 | you keep the `(lang dune ...)` line in your `dune-project` as `(lang 27 | dune 1.x)`. This will forbid you access to the Dune 2 features and 28 | will ensure that your project is compatible with Dune >= 1.x. 29 | 30 | On our side, we will make sure to add a lot of cool new features in 31 | Dune 2 to convince all users to switch to the new dune language :) 32 | 33 | ## Quick summary 34 | 35 | - Dune 2.0.0 will be 99% backward compatible with Dune 1.x 36 | - Jbuilder and Dune 2.0.0 will be co-installable 37 | - a `dune-project` file will now be necessary to mark the root of a 38 | project 39 | - a few default behaviors will change in dune 2.0.0 but will not 40 | affect existing released packages 41 | - Dune 1.x will receive an additional year of support, possibly 2 42 | - Dune 2.0.0 will require OCaml 4.08 to build, however it still be 43 | able to build projects using older compilers and will be installable 44 | in older opam switches 45 | - `dune.configurator` will now be released as `dune-configurator`, 46 | however the name `dune.configurator` will still exist for backward 47 | compatibility reasons 48 | - lots of new exciting features in preparation! 49 | 50 | ## Dropping official support for Jbuilder 51 | 52 | The main motivation for bumping the major version number is dropping 53 | the support for jbuilder, as announced in previous posts: 54 | 55 | - [Dune 1.0.0 is comming soon, what about jbuilder projects?](https://discuss.ocaml.org/t/dune-1-0-0-is-coming-soon-what-about-jbuilder-projects/2237) 56 | - [Second stage of Jbuilder deprecation](https://dune.build/blog/second-step-deprecation/) 57 | 58 | This means that starting from Dune 2.0.0, the `dune` binary will no 59 | longer be guaranteed to understand projects with `jbuild` files. We 60 | originally planned that Dune 2 would fail hard when seeing a `jbuild` 61 | file and would install a dummy `jbuilder` binary that would 62 | systematically fail on startup. 63 | 64 | However, given the large number of projects that haven't switched from 65 | Jbuilder to Dune yet, we decided to adapt our plans. First of all, 66 | Dune 2 will not install a dummy `jbuilder` binary. In fact, it will 67 | not install a `jbuilder` binary at all. This means that Dune 2 and 68 | Jbuilder will be co-installable. In particular, in opam it will be 69 | possible to install packages using Jbuilder alongside packages using 70 | Dune 2. 71 | 72 | Additionally, Dune will still be able to understand Jbuilder projects 73 | to some extent. More precisely, it will use the same code as `dune 74 | upgrade` to convert `jbuild` files to `dune` files on the fly. This is 75 | not enough to provide full compatibility with `jbuilder` when 76 | installed via opam. However, it will be enough when a Jbuilder project 77 | is vendored inside a Dune project. 78 | 79 | With this plan, the Dune 2 release should be smooth for users and 80 | Jbuilder backward compatibility will not get in the way of new Dune 81 | developments. 82 | 83 | But really, we strongly encourage everybody maintaining a package 84 | still using `jbuilder` to release a new version using `dune` :) 85 | Upgrading from `jbuilder` to `dune` is extremely easy and automated as 86 | desribed in [this post](https://dune.build/blog/second-step-deprecation/). 87 | 88 | ### Note for Mono-repository users 89 | 90 | While Dune will still be able to understand Jbuilder projects vendored 91 | inside Dune ones, it will now require the presence of a `dune-project` 92 | to mark the root of such projects. Up to now, the presence of at least 93 | one `.opam` file was enough to mark the root of a project 94 | inside a bigger workspace. Marking the root of projects inside 95 | workspaces is important as otherwise private libraries of vendored 96 | projects will be globally visible and might clash with private 97 | libraries of other vendored projects. 98 | 99 | Since older projects are likely to not have such a file, you should 100 | add one when vendoring a Jbuilder project inside a Dune one. This 101 | `dune-project` file can simply contain `(lang dune 1.0)`. 102 | 103 | ## 99% backward compatibility with Dune 1.x 104 | 105 | When Dune sees `(lang dune x.y)` in a `dune-project` file, it adapts 106 | itself to behave as the `x.y` versoin of Dune. This is how we provide 107 | full backward compatibility in Dune. 108 | 109 | As a result we are expecting that Dune 2.0 will not break 110 | anything. Projects currently using `(lang dune 1.x)` will continue to 111 | build with all versions of Dune from 1.x onwards. 112 | 113 | There is one exception I can think of to this rule: currently if you 114 | have a file that is both generated by a rule and present in the source 115 | tree, this is a warning with older versions of the dune 116 | language. However, making this a warning rather than a hard error is 117 | costly and requires maintainig a complex piece of code in Dune. What 118 | is more, this scenario often led to confusion in the past so it is not 119 | a good thing to accept it. As a result, for this particular case we 120 | will make it a systematic error starting from Dune 2.0.0. This means 121 | that projects relying on the current behaviour will not build with 122 | Dune 2.x. For the record, the way to fix this is to add a `(mode 123 | promote)` or `(mode fallback)` field to the relevant stanza so that 124 | Dune knows what the intent is. 125 | 126 | Appart from that, things that currently trigger a warning will 127 | continue to trigger a warning when using `(lang dune 1.x)`, however 128 | they will become hard errors when using `(lang dune 2.x)`. We will 129 | also change a few defaults behaviors in Dune 2.x. We don't have the 130 | full list yet, but a few things that one has to enable explicitly now 131 | in the `dune-project` file will become the default, such as 132 | auto-formatting. Another example is what dune builds by default. It 133 | will be `@all` rather than `@install` since `@all` is now a more 134 | appropriate default. 135 | 136 | You can think of the `(lang dune x.y)` line in `dune-project` files as 137 | follow: while working on a project, you should use the latest version 138 | you can use. This will give you access to the most recent features of 139 | Dune and will give you the most up-to-date commonly accepted defaults. 140 | 141 | When not working on a project, this line is a way to remember which 142 | version of Dune was used the last time you worked on the project so 143 | that future versions of Dune can continue to understand it without you 144 | taking action. 145 | 146 | ## dune.configurator will now be dune-configurator 147 | 148 | We want to make Dune a pure binary so that it is easier to distribute. 149 | For this reason, the `dune.configurator` library will now be 150 | distributed as a separate `dune-configurator` package in opam. The 151 | name `dune.configurator` will still exist for backward compatibility 152 | reasons, however you should use the name `dune-configurator` rather 153 | than `dune.configurator` in new code. 154 | 155 | At least the first release of the `dune-configurator` package will be 156 | compatible with Dune 1, so projects that want to keep compatibility 157 | with Dune 1 can depend on `dune-configurator`. 158 | 159 | ## Requiring a recent version of OCaml to build Dune 160 | 161 | At the moment, Dune can build projects using all versions of OCaml 162 | since 4.02.3 and can also built itself using all versions of OCaml 163 | since 4.02.3. More precisely, Dune is aware of all past OCaml bugs and 164 | can work around them when building projects using an old compiler, and 165 | the code of Dune itself is compatible with older compilers. 166 | 167 | This makes Dune an easy dependency for authors who want their project 168 | to be compatible with older OCaml versions. However, making Dune 169 | itself compatible with older compilers is a heavy constraint on the 170 | development of Dune. In particular, it means that we need to increase 171 | our testing with each new compiler release and also that we cannot use 172 | new features of the language. The second constraint is particularly 173 | annoying given that Dune is actively developed and its code base could 174 | definitely benefit from the new language features. 175 | 176 | For this reason, starting from version 2.0.0 Dune will require a 177 | recent version of OCaml to build itself. Dune 2.0.0 will require 178 | 4.08.0, when 4.09.0 is released we will immediately start using its 179 | new features and so on. But more importantly, we will immediatly start 180 | using multicore when it becomes available, since Dune is a project 181 | that is likely to benefit from it. 182 | 183 | However, we will continue to support building projects using Dune with 184 | older compilers. For instance, it will still be possible to build Dune 185 | using OCaml 4.08.0 and use the resulting binary to build projects with 186 | OCaml 4.02.3. In particular, we will rely on this fact to make Dune 187 | 2.0.0 available in older opam switches. The only downside is that 188 | installing Dune 2 in an older opam switch will be slightly slower as 189 | it will need to first install a secondary compiler. However, we hope 190 | that this secondary compiler could be used by other platform tools who 191 | would want to adopt the same approach as Dune. 192 | 193 | Since this is method is new and might require some time before it is 194 | properly rolling, we will provide an additional 1 year of support for 195 | Dune 1, and possibly 2 if the burden is not too high on the Dune 196 | team. By support we mean the following: we will continue doing bugfix 197 | releases and making sure that the latest release of Dune 1.x builds 198 | will new OCaml compilers released during that period of time. 199 | 200 | ## What's next? 201 | 202 | There are a lot of new exciting features planned or in development. An 203 | important one is the integration of a shared artefacts cache in 204 | Dune. In a nutshell, it will make `opam install` cached and fast, 205 | which will be particularly useful for users of local switches. It will 206 | also be useful for non-opam users. We will talk more about this 207 | subject in a dedicated post. 208 | 209 | We are also continuing the work on the Core of Dune, and in particular 210 | improving and getting more out of [the memorisation 211 | system](https://dune.build/blog/new-computation-model/) we introduced 212 | last year. We plan to rely on it to make Dune scale and preserve the 213 | user experience when using Dune in large workspaces. 214 | 215 | And as always, we will continue formalising and integrating new 216 | features that will improve the workflow of Dune users! 217 | -------------------------------------------------------------------------------- /_posts/2020-02-26-dune-retreat-2020.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: blog 3 | title: Summary of the Dune retreat 2020! 4 | author: jeremiedimino 5 | tags: [ocaml, dune] 6 | picture: /assets/imgs/dune-retreat-2020.jpeg 7 | discuss: https://discuss.ocaml.org/t/summary-of-the-dune-retreat-2020/5224 8 | --- 9 | 10 | The Dune retreat, organised by Jane Street, is an event where all the 11 | developers of Dune meet for a week to hack on Dune. The Dune 12 | team has grown quite a lot and is now a mix of developers from Jane Street, 13 | OCaml Labs, and Tarides; developers from companies that 14 | are using or are looking at using Dune, such as LexiFi or CEA; 15 | developers from friend communities such as Coq; and open source 16 | contributors. The event is a great way for all the developers 17 | to get together. 18 | 19 | Just like for the first retreat, we all came out of it recharged with 20 | our heads full of ideas to fuel the development of Dune for the year 21 | to come. Here is a summary of things we discussed, worked on, and 22 | continue working on. 23 | 24 | # More dynamism! 25 | 26 | The initial architecture of Dune was as follows: 27 | 28 | - Step 1: Gather all the configuration (command line, dune files, etc...) 29 | - Step 2: Generate all the build rules 30 | - Step 3: Execute the build, i.e., follow the build DAG and execute commands 31 | 32 | Since then, the design has evolved a bit but is still mostly like 33 | this. In particular, we currently cannot use the result of 34 | executing a build command to produce more rules. While this is fine 35 | for the majority of projects, it is often quite limiting. We have 36 | now reached a point where it is clear that we cannot continue like 37 | this. The extra flexibility is needed in a number of cases, such 38 | as the [linked build contexts](#linked-build-contexts) feature discussed later in this post. 39 | 40 | The main difficulty is a well-known one for OCaml developers: we need 41 | to spread the monad. Indeed, the parallelism at Step 3 is obtained by 42 | using a concurrency monad. We traverse the build DAG and 43 | systematically fork execution when descending into the children of a 44 | node, thus spawning many lightweight threads that all try to 45 | start many external commands. It is then the scheduler that throttles 46 | the execution of the commands according to the `-j` option passed to 47 | dune. 48 | 49 | So if we want to use the result of a command, then we have to be in 50 | the monad. However, this monad is currently only used at Step 3. So if 51 | we want to compute more things dynamically, then we have to spread the 52 | monad through the rest of the codebase. 53 | 54 | There is obviously a potential performance concern; indeed we are 55 | turning direct code into monadic code and so creating more closures, 56 | putting more pressure on the gc, etc. And while we will certainly 57 | benchmark this change, we are not too concerned about it slowing down 58 | Dune. Indeed, the monad we are using in Dune is called `Fiber` and is 59 | very light. Unlike `Async` or `Lwt`, a `Fiber.t` can only be used 60 | once and sharing or forking must be explicit. As a result, `Fiber.t` 61 | is pretty much just a CPS monad and should be a piece of cake for the 62 | compiler to optimise away. 63 | 64 | Rudi Grinberg from OCaml Labs has started working on this gargantuan 65 | refactoring. 66 | 67 | # Linked build contexts 68 | 69 | Dune allows building against multiple configurations 70 | simultaneously. For instance, in a single invocation of Dune you can 71 | build: 72 | 73 | - against a 4.08 compiler 74 | - against a 4.09 compiler 75 | - against a 4.09 compiler with flamda 76 | - against a windows cross-compiler 77 | - ... 78 | 79 | At the moment, the configuration of each build context is determined 80 | once and for all at the beginning of Dune's execution. This is, however, 81 | too limited for complex workflows such as the ones used by the mirage 82 | community. In that world, we need to build and run user code in order 83 | to determine the configuration of a subsequent build context in which 84 | the target "unikernels" are built. 85 | 86 | To support such workflows, we are planning to allow configuring a 87 | build context using programs built in another build context. For 88 | instance, Dune will support obtaining the default C flags for building 89 | esp32 unikernels by running a command built from a separate 90 | configuration context. 91 | 92 | More generally, the concept we want to introduce is that the public 93 | elements of a particular build context will be visible to another 94 | build context--a bit like if we ran a first build, installed the 95 | artefacts and then ran another build using a different 96 | configuration. Except that all these steps will be transparent and 97 | orchestrated by Dune itself. 98 | 99 | This work will also make it possible to encode a full bootstrap build 100 | of the OCaml compiler with Dune. 101 | 102 | We discussed this feature during the retreat but this work is 103 | currently pending on spreading the fiber monad. 104 | 105 | # Distributed shared cache 106 | 107 | When using a distributed shared artifact cache, Dune will prefetch 108 | artefacts into the local shared cache so that they are ready when it 109 | needs them. But if they are not present when it needs them, it will 110 | just run the build command locally, creating some sort of race between 111 | the CPU and the network. For the distributed cache to be efficient and 112 | speed up builds, the prefetch heuristics need to be good. 113 | 114 | One idea would be to get help from the VCS, or in the case of Jane 115 | Street, from Iron, so that when working on a PR/feature the system 116 | would tell us which artefacts from the distributed cache we are the 117 | most likely to need. This idea is, however, quite complex to put in 118 | motion as it would require coordination between various systems. It 119 | might also be computationally expensive. 120 | 121 | Another idea is the following: we can decompose the execution of a 122 | rule into the three following steps: 123 | - Step 1: dune knows an over-approximation of the dependencies and needs 124 | to build a bit more in order to know the exact set of dependencies 125 | (for instance, it needs to call `ocamldep`) 126 | - Step 2: dune knows the exact set of dependencies, the exact command 127 | to run, etc... 128 | - Step 3: once all the dependencies are built, dune fires the rule 129 | 130 | At Step 2, we know exactly what we need and can send an exact 131 | prefetch request to the shared cache. Unfortunately, it is not clear 132 | that Step 2 happens soon enough to be efficient in practice. Indeed, it 133 | might happen quite close to Step 3, meaning that the CPU would win the 134 | race over the network. 135 | 136 | However, in Dune, Step 1 happens very early and the over-approximation 137 | is actually not that big compared to reality. So the idea is to 138 | compute a key for the shared cache that takes into account: 139 | - the current configuration (such as flambda, version of OCaml, ...) 140 | - an over-approximation of all the source files a rule transitively depends on 141 | 142 | This key is overly specific. Indeed, changing one comment in a file 143 | would change many such keys even though it wouldn't change the key 144 | computed at Step 2. However, because we know it very early, we might as 145 | well try it, as a hit on this key will pretty certainly gives us what 146 | we need. We hope that using prefetch hints at Steps 1 and/or 2 will 147 | be enough to find good prefetch candidates. 148 | 149 | The distributed shared cache is being developed by Quentin Hocquet 150 | from Tarides. 151 | 152 | ## Splitting out the daemon 153 | 154 | Right now, when using the shared cache, Dune can either promote things 155 | itself to the shared cache or talk to a daemon. This daemon is part of 156 | Dune itself. However, to add support for the distributed shared cache, 157 | we need more library dependencies such as Irmin. Since we cannot add 158 | these dependencies to Dune, we plan to instead extract the daemon as a 159 | separate project and what will remain in the main repository is only 160 | the "direct" mode where Dune operates without a daemon. 161 | 162 | This means that after installing Dune, users will only be able to use 163 | the local shared cache and will need to install an additional package 164 | in order to use a distributed cache, which seems fine. 165 | 166 | # Runtime data files and relocation 167 | 168 | Whenever possible, we build and distribute self-contained binary 169 | applications. However, this is not always possible and an application 170 | might need to read additional data files at runtime or even load 171 | plugins. There is a lot of boilerplate when we need to do so, such as: 172 | 173 | - where to find the files once the application is installed 174 | - where to find the files during testing before the application has been installed 175 | - in the case of plugins, how to handle dependencies between plugins 176 | - once installed, can the application and its data be moved to another location, i.e., is it relocatable 177 | 178 | Francois Bobot from CEA is very much interested in this question for 179 | Frama-C, as they are currently porting their old build system to use 180 | Dune. We discussed a proposal and he has been working on implementing 181 | it. This feature will allow packages to declare an "installation site" 182 | where the package itself and other packages might install data files 183 | or plugins. Then Dune will take care of the low-level details and 184 | expose a high-level interface to the application to load data files or 185 | plugins. 186 | 187 | # Better vendoring workflows 188 | 189 | Historically, OCaml packages were black boxes with three buttons on them: 190 | 191 | - ./configure 192 | - make 193 | - make install 194 | 195 | So if you wanted to use a third-party library in your project, you had 196 | to install it first. This was generally done via a package manager 197 | such as opam. 198 | 199 | Dune opened the door to an alternative workflow: when you put two 200 | projects side by side, Dune is able to open them and understand them 201 | as a whole as if they were a single project. This means that 202 | vendoring--importing the source code of a dependency--is very easy 203 | with Dune. More and more users are starting to make use of this 204 | feature for development because it simplifies their workflow. 205 | 206 | Indeed, in the past, when you would submit a pull request to a project, 207 | you would have to wait for the project to make a new release before 208 | you could start using it in your project. Or you could use "pins" but 209 | that was tedious and didn't scale well. With dune, users can vendor 210 | temporarily or permanently their dependencies to get going until the 211 | release. 212 | 213 | This all works well but there are a few points of friction around releasing a 214 | package with vendored dependencies in opam. In some cases, it is 215 | preferable to use the embedded vendored code, for instance when 216 | publishing a binary, and sometimes it is preferable to ignore the 217 | embedded vendored code and use what's installed by opam instead. This is 218 | not very well supported by Dune at the moment. 219 | 220 | We discussed various solutions and have at least one concrete 221 | proposal. However, because it is possible to work around this 222 | limitation by hacking the opam build commands, we decided to wait 223 | until a standard workflow has emerged before committing to a 224 | particular one in Dune. 225 | 226 | # GraphQL introspection 227 | 228 | Etienne Millon from Tarides is adding support for querying various 229 | information about a Dune project via GraphQL, a standard query 230 | language developed by Facebook. With this feature, it will be easy for 231 | third-party applications to extract various information out of Dune. 232 | 233 | Using GraphQL would be much easier than say, reading the dune files 234 | and interpreting them. 235 | 236 | # Modern CI integration 237 | 238 | A few people from OCaml Labs and Tarides are working on ocaml-ci, a 239 | modern configuration-less CI for OCaml projects based on 240 | OCurrent. ocaml-ci will rely on standard dune invocations for various 241 | tasks such as building, testing, and checking formatting, so that users 242 | have nothing to configure. 243 | 244 | Up to now, users have relied on CI scripts using opam. These require 245 | configurations and are not always very flexible. For instance, testing 246 | multi-package projects always tends to be difficult. The new CI system 247 | will be simpler and will just work out of the box. 248 | 249 | Craig Ferguson from Tarides, who is working on this project and 250 | attended the retreat, gave us a demo of the system and showed various 251 | interesting aspects of its implementation, such as a nice DSL they have 252 | to generate docker files. 253 | 254 | # Mdx 255 | 256 | Nathan Rebours from Tarides has been working on adding mdx support to 257 | Dune. Mdx is a tool allowing users to have verified OCaml toplevel and shell 258 | fragments in markdown and mli files, making it a good tool to write 259 | and maintain up-to-date documentation examples. At the moment, using 260 | mdx in Dune requires writing and maintaining tedious custom rules. 261 | 262 | With this new feature, all the user will have to do is write an `mdx` 263 | stanza. A follow-up of this work will be to improve the way mdx and 264 | dune interacts to make the overall system simpler and faster. 265 | 266 | As part of this work, Nathan will look at unifying the native and byte 267 | code versions of the OCaml toplevel libraries, which should benefit a 268 | lot of people. 269 | 270 | # Cram tests 271 | 272 | Since the beginning, we have been writing a lot of integration tests 273 | for Dune using a small [cram test tool](https://bitheap.org/cram/) 274 | written in OCaml. Such tests are very simple to read, write, and work 275 | with. And because their syntax is so simple and close to shell 276 | commands that a lot of people know, it is easy for contributors to 277 | write tests. 278 | 279 | Other projects are interested in writing such tests, so I have been 280 | working on generalising the cram test tool so that we can eventually 281 | publish it. 282 | 283 | One aspect I am looking at before releasing the tool is reducing the 284 | boilerplate to a minumun. Right now, to run a cram test one has to 285 | write at least the following: 286 | 287 | ``` 288 | (rule 289 | (progn 290 | (run dune-cram run %{dep:run.t}) 291 | (diff? run.t run.t.corrected))) 292 | ``` 293 | 294 | I'd like to make the last line unnecessary by adding a general 295 | mechanism for commands to report additional information to dune--in this case, 296 | that they produced a correction. If the dune 297 | boilerplate required becomes so simple that it is not necessary to 298 | have a dedicated `cram` stanza, it means that it will be easy for 299 | users to develop their own build or test tool that will integrate 300 | nicely with Dune without requiring Dune to know about it. 301 | 302 | # Polling mode 303 | 304 | We didn't particularly discuss this topic during the reteat, but 305 | since this post mentions many of the big new things that will happen 306 | during the year I thought it would be good to give an update on this 307 | topic. 308 | 309 | For a while dune has supported a "watch" mode where it watches the 310 | file system and automatically rebuilds when something changes. While 311 | this works, the implementation is not very much more than: 312 | 313 | ``` 314 | while true do 315 | dune build 316 | wait_for_change 317 | done 318 | ``` 319 | 320 | Which means that it doesn't scale well to large repositories. At Jane 321 | Street we are particularly interested in making it scale since we want 322 | to use Dune on our massive internal repository. So we have been 323 | working for a while on a principled system to make polling builds fast 324 | and scalable. This work started about two years ago when we introduced 325 | a [new computation model][comp-model] with Rudi Horn who interned at 326 | Jane Street. It then continued last year with my colleague Arseniy 327 | Alekseyev, who pushed the system forward and integrated it deeper into 328 | the core of Dune. And it is now Andrey Mokhov who is putting the last 329 | stone on the edifice and has been working on literally "purifying" the 330 | code of Dune, i.e., removing all side effects that are not tracked by 331 | the memoisation system. 332 | 333 | (At the same time continuing to rewrite and 334 | clean up some of the terrible original code I wrote. Indeed, that's a story 335 | for another post, but for the original development of Jbuilder, 336 | there was a ton of work to do in limited time and I didn't always take 337 | the time to polish every bit of my code as I usually like to do, 338 | especially for open source projects where the code is put up in the 339 | open for everyone to see!) 340 | 341 | # Thank you 342 | 343 | I'll conclude this post on this note and would like to thank all the 344 | people who are making the Dune project and event such as the retreat 345 | possible. Thanks to the whole Dune team for their hard work (and also great LAN 346 | parties :)), Jane Street who is putting both financing and developer 347 | time into the project, OCaml Labs and Tarides for their support and 348 | developer time, CEA and LexiFi for their developer time, the Coq team 349 | for choosing Dune and helping make it better, all the contributors who 350 | are helping in their free time, all the people who are reporting 351 | issues, and of course all of the users of Dune, without whom the project 352 | wouldn't need to exist. 353 | 354 | [comp-model]: https://dune.build/blog/new-computation-model/ 355 | --------------------------------------------------------------------------------