├── .gitignore ├── .travis.yml ├── CHAPTERS.txt ├── Dockerfile ├── README.md ├── YEARS.txt ├── content ├── 2017-focus.md ├── ack.md ├── collaborations.md ├── community-and-outreach.md ├── compiler.md ├── intro.md ├── mirageos.md ├── platform.md └── publications.md ├── graphics ├── ocl13rev │ ├── c.png │ ├── camel.png │ ├── compiler-hacking-thumb.jpg │ ├── compiler-hacking.jpg │ ├── daniel-presentation-vg.jpg │ ├── fpdays2013-04-thumb.jpeg │ ├── fpdays2013-04.jpg │ ├── icfp1.jpeg │ ├── js.jpg │ ├── lab_main.jpg │ ├── makespace-apr-2013.jpg │ ├── marius-yaron-icfp-thumb.jpeg │ ├── marius-yaron-icfp.jpg │ ├── mirageos.png │ ├── nymote.png │ ├── ocaml-org-thumb.png │ ├── opam11-beta-contributors.png │ ├── opam11-beta-packages.png │ ├── opam11-beta-unique-packages.png │ ├── opam11-contributors-dec13.png │ ├── opam11-packages-dec13.png │ ├── opam11-unique-packages-dec13.png │ ├── pfff-thumb.jpeg │ ├── pfff.jpg │ ├── rackspace.png │ ├── rwo.png │ ├── thomas-nycoug-2013.jpg │ ├── tony-stark-hud.jpg │ └── xenserver.png ├── ocl14rev │ ├── c.png │ ├── christs-dinner-thumb.jpg │ ├── christs-dinner.jpg │ ├── compiler-hacking-dsyme-thumb.jpg │ ├── compiler-hacking-dsyme.jpg │ ├── jeremy-rwo-thumb.jpg │ ├── jeremy-rwo.jpg │ ├── l23-thumb.jpg │ ├── l23.jpg │ ├── lab_main.jpg │ ├── nsdi-deadline-thumb.jpg │ ├── nsdi-deadline.jpg │ ├── ocaml-org-thumb.png │ ├── ocl-pub-thumb.jpg │ ├── ocl-pub.jpg │ ├── oleg-talk.jpg │ ├── opam-in-nice-thumb.jpg │ ├── opam-in-nice.jpg │ ├── opam12-contributors-mar14.png │ ├── opam12-packages-mar14.png │ ├── opam12-unique-packages-mar14.png │ ├── qcon-unikernel-talk-thumb.jpg │ ├── qcon-unikernel-talk.jpg │ ├── scotty-thumb.jpg │ └── scotty.jpg ├── origami-camel.png └── travis-mascot-200px.png ├── monthly ├── 2016 │ ├── 11 │ │ └── maxime.md │ └── 10-12 │ │ └── let-def.md ├── 2017 │ └── 2 │ │ └── let-def.md └── 2016-11 ├── output ├── github-pandoc.css └── graphics ├── scripts ├── build.sh ├── deploy.sh └── furore_id_rsa.enc ├── weekly ├── 2016 │ ├── 16 │ │ └── dinosaure.md │ ├── 17 │ │ ├── dinosaure.md │ │ └── engil.md │ ├── 19 │ │ ├── dinosaure.md │ │ ├── engil.md │ │ └── oliviernicole.md │ ├── 20 │ │ ├── dinosaure.md │ │ ├── engil.md │ │ └── oliviernicole.md │ ├── 21 │ │ ├── dinosaure.md │ │ └── oliviernicole.md │ ├── 22 │ │ ├── dinosaure.md │ │ └── oliviernicole.md │ ├── 23 │ │ ├── dinosaure.md │ │ ├── engil.md │ │ ├── oliviernicole.md │ │ └── philipdexter.md │ ├── 24 │ │ ├── dinosaure.md │ │ ├── oliviernicole.md │ │ └── philipdexter.md │ ├── 25 │ │ ├── dinosaure.md │ │ └── philipdexter.md │ ├── 26 │ │ ├── dinosaure.md │ │ ├── oliviernicole.md │ │ └── philipdexter.md │ ├── 27 │ │ ├── ciaran16.md │ │ ├── dinosaure.md │ │ ├── jdjakub.md │ │ ├── oliviernicole.md │ │ └── philipdexter.md │ ├── 28 │ │ ├── ciaran16.md │ │ ├── dinosaure.md │ │ ├── jdjakub.md │ │ ├── oliviernicole.md │ │ └── philipdexter.md │ ├── 29 │ │ ├── ciaran16.md │ │ ├── dinosaure.md │ │ ├── oliviernicole.md │ │ └── philipdexter.md │ ├── 30 │ │ ├── ciaran16.md │ │ ├── dinosaure.md │ │ └── jdjakub.md │ ├── 31 │ │ ├── ciaran16.md │ │ ├── dinosaure.md │ │ └── engil.md │ ├── 32 │ │ ├── ciaran16.md │ │ └── dinosaure.md │ ├── 33 │ │ └── dinosaure.md │ ├── 39 │ │ └── oliviernicole.md │ ├── 40 │ │ └── oliviernicole.md │ ├── 41 │ │ └── oliviernicole.md │ ├── 42 │ │ ├── oliviernicole.md │ │ └── ryanrhymes.md │ ├── 43 │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 44 │ │ ├── ryanrhymes.md │ │ └── seveneng.md │ ├── 45 │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 46 │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 47 │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 48 │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 49 │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 50 │ │ ├── let-def.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ └── 51 │ │ ├── dra27.md │ │ ├── let-def.md │ │ └── timada.md ├── 2017 │ ├── 1 │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── kayceesrk.md │ │ ├── let-def.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 2 │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── ilsordo.md │ │ ├── kayceesrk.md │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 3 │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── ilsordo.md │ │ ├── kayceesrk.md │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 4 │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── ilsordo.md │ │ ├── kayceesrk.md │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ ├── stedolan.md │ │ └── timada.md │ ├── 5 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── ilsordo.md │ │ ├── oliviernicole.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 6 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 7 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 8 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 9 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 10 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── gemmag.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ ├── stedolan.md │ │ └── timada.md │ ├── 11 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 12 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── gemmag.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 13 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── g2p.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 14 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 15 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── gemmag.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 16 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 17 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ ├── stedolan.md │ │ └── timada.md │ ├── 18 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ ├── stedolan.md │ │ └── timada.md │ ├── 19 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 20 │ │ ├── dinosaure.md │ │ ├── dra27.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 21 │ │ ├── dinosaure.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 22 │ │ ├── dinosaure.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 23 │ │ ├── dinosaure.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 24 │ │ ├── dinosaure.md │ │ ├── ryanrhymes.md │ │ ├── sanderspies.md │ │ └── timada.md │ ├── 25 │ │ ├── dinosaure.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 26 │ │ ├── dinosaure.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 27 │ │ ├── dinosaure.md │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 28 │ │ ├── dinosaure.md │ │ ├── ryanrhymes.md │ │ ├── sanderspies.md │ │ └── timada.md │ ├── 29 │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 30 │ │ ├── kayceesrk.md │ │ ├── ryanrhymes.md │ │ ├── sanderspies.md │ │ ├── seveneng.md │ │ └── timada.md │ ├── 31 │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 32 │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 33 │ │ ├── ryanrhymes.md │ │ └── timada.md │ ├── 34 │ │ └── timada.md │ ├── 35 │ │ └── timada.md │ ├── 36 │ │ └── timada.md │ ├── 37 │ │ ├── dinosaure.md │ │ └── timada.md │ ├── 38 │ │ └── dinosaure.md │ ├── 39 │ │ └── dinosaure.md │ ├── 40 │ │ └── dinosaure.md │ ├── 41 │ │ └── dinosaure.md │ ├── 42 │ │ └── dinosaure.md │ ├── 43 │ │ └── dinosaure.md │ └── dinosaure.md ├── intro.md ├── weeklies.md └── weekly.ml └── yearly ├── 2013.md ├── 2014.md └── intro.md /.gitignore: -------------------------------------------------------------------------------- 1 | .*.sw* 2 | output/*.epub 3 | output/*.html 4 | output/*.pdf 5 | weekly/weekly.md 6 | *.exe 7 | *~ 8 | 9 | weekly/2017/1/gemmag.md~HEAD 10 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: c 2 | script: 3 | - bash -ex scripts/build.sh && bash -ex scripts/deploy.sh 4 | services: 5 | - docker 6 | sudo: false 7 | before_install: 8 | - openssl aes-256-cbc -K $encrypted_5899a0a28dc8_key -iv $encrypted_5899a0a28dc8_iv 9 | -in scripts/furore_id_rsa.enc -out furore_id_rsa -d 10 | notifications: 11 | slack: 12 | secure: Wj2wu1IvZlzSPlbHXZvzXKe/vAWBkZEkQQ5723GoNFgXns21RBolOwPpILpE2dSCryumQAUGHJ87ZJbgCTmXWXmfzWD3ez6o9tFDmxSkNt5MUAnIox/e8QCCzkCHM12SYcKtIp3DEvc0Ox9BJmKm59NkSmc0bhISGwMDHoAWJu6qvgopzMzWC7L/EmYgcRHDQjhn5T959c9juKgfnltb4W+H1FjBVqOTY+QjK/VaPZHJj6SkNLgzzPlPVmbyXB8oaMYy3gHdsygf3mncaeIo5ujGINxS17rf9KBKbnLbiQgbhnYE78tFXgjKY4sJFkASs6L5YXOrKGxd+ve83ehzHJnamFEEqKJdzQkd5j4qFzb4Oxs3B95H6nifU+pj9Mr9Y9jfI8zdB4HzUVLW+Gy3o76mM9As+qf2+Rfwq1p0EC1KBC7pefH4Em1CC8NkMxz23l5Pagw+jgpJhPppccc9I92W1OQhvhxp/NJN+lXgCazaMF6oA57DkCWaWP9gsJu5BzGFKBcVQETLKRt6A23pc/Kx1jQ/PWC3bubyA1bX6KcDtPP2ar40Q3cWDC9+i4XPLNJTnuJoKSP6bHHUaU1wVssnsuCpCdzYkqWX+TE06/Karz4v/GyV51+5G8dA8GDd2ChPlwaU3z+udcvFLH4WUk6v3AN2j/4n/fO7iqQy0BI= 13 | -------------------------------------------------------------------------------- /CHAPTERS.txt: -------------------------------------------------------------------------------- 1 | content/intro.md 2 | content/platform.md 3 | content/mirageos.md 4 | content/community-and-outreach.md 5 | content/collaborations.md 6 | content/publications.md 7 | content/2017-focus.md 8 | content/ack.md 9 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:xenial 2 | RUN apt-get update && apt-get -y install pandoc python-pygments opam 3 | RUN opam init -ay 4 | RUN opam depext -i -j 4 -vy bos fpath rresult fmt ocamlscript 5 | ENTRYPOINT ["opam","config","exec","--"] 6 | #RUN apt-get -y texlive texlive-xetex texlive-fonts-extra 7 | #COPY * /mnt/ 8 | #RUN pandoc -f markdown intro.md -o report.pdf 9 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Work in Progress reports 2 | ======================== 3 | 4 | These are works in progress, please contact Gemma Gordon with any queries. 5 | 6 | * Latest: 7 | * Annual: (ePub: ) 8 | * Weekly: (ePub: ) 9 | * 10 | -------------------------------------------------------------------------------- /YEARS.txt: -------------------------------------------------------------------------------- 1 | yearly/2014.md 2 | yearly/2013.md 3 | -------------------------------------------------------------------------------- /content/2017-focus.md: -------------------------------------------------------------------------------- 1 | # 2017 Project Goals 2 | 3 | ## Core Language 4 | 5 | * Multicore - where it's going + effects, link 6 | 7 | * Modular Implicits 8 | 9 | * Macros 10 | 11 | 12 | ## Platform 13 | 14 | * Windows Support 15 | * ocamlbuild on Windows \(or other options...\) 16 | * Windows CI: visual studio test matrix 17 | * OCaml features: 18 | -> Unix ioctl 19 | -> Unix library overhaul/refactor 20 | -> GADT optional arguments 21 | -> Polymorphic module signatures? 22 | -> Build system 23 | -> ARM/ARM64 Windows Ports 24 | -> SpaceTime viewer Win32 port? 25 | * Libraries: PiFaceCAD; ODBC 26 | 27 | 28 | * Merlin 29 | 30 | * Parsing and integration with Menhir. [Plans](https://github.com/the-lambda-church/merlin/projects/1) 31 | 32 | 33 | * Mac OSX Support 34 | 35 | * Performance profiling, Spacetime 36 | 37 | 38 | ## MirageOS 39 | 40 | * MirageOS perf \(taka\) 41 | 42 | * MirageOS\/Databox\/Docker - Bushel 43 | 44 | * MirageOS storage - Gabriel 45 | 46 | * Git\/etc Romain 47 | 48 | * nicolas internship 49 | 50 | * cloud infrastructure from Microsoft, hardware from ARM 51 | 52 | * matt harrison - personal data silos 53 | 54 | 55 | ## Community 56 | 57 | * repo maintainership\/management 58 | 59 | * rwo 60 | 61 | * teaching 62 | 63 | * internship\/outreachy programs 64 | 65 | * ocaml.org 66 | 67 | * more events: compiler hacking \(feb\), mirageos marrakech \(march\), databox open source \(march\) 68 | 69 | 70 | 71 | 72 | -------------------------------------------------------------------------------- /content/ack.md: -------------------------------------------------------------------------------- 1 | Mention funding - grants, epsrc, at end - acknowledgements 2 | 3 | List of contributors 4 | 5 | -------------------------------------------------------------------------------- /content/collaborations.md: -------------------------------------------------------------------------------- 1 | Docker 2 | 3 | Jane Street 4 | 5 | ARM 6 | 7 | Microsoft 8 | 9 | Facebook - Reason ML 10 | 11 | Sean Grove 12 | 13 | Aesthetic Integration 14 | 15 | Citrix 16 | 17 | Hitachi 18 | 19 | -------------------------------------------------------------------------------- /content/compiler.md: -------------------------------------------------------------------------------- 1 | # OCaml Language and Compiler 2 | 3 | This refers to the core compiler toolchain, OCaml language and runtime system. Our work includes daily maintenance such as bug fixes and long term improvements to the type system and runtime libraries. We are actively engaging with the wider OCaml community to ensure that improvements and modifications we propose are thoroughly discussed, well-formulated and maintainable. 2016 has featured significant work towards multicore support for parallelism and concurrency in OCaml together with greater facilitation for metaprogramming approaches. 4 | 5 | * Multicore 6 | * * Reagents 7 | * Effects 8 | * links - daniel hillerstrom 9 | * ryan newton 10 | 11 | * Metaprogramming 12 | * * Macros 13 | * * MetaOCaml 14 | 15 | * Modular Implicits 16 | 17 | * Hardcaml 18 | 19 | 20 | * Bigraphs - michele 21 | * Approximate Computing: Philip Dexter 22 | -------------------------------------------------------------------------------- /content/intro.md: -------------------------------------------------------------------------------- 1 | # OCaml Labs 2 | 3 | **Note: This is a work in progress draft document** 4 | 5 | ## An Introduction 6 | 7 | OCaml Labs is an initiative within the Cambridge Computer Laboratory started by Anil Madhavapeddy in 2011 to promote research, growth and collaboration within the wider OCaml community. We contribute to management of the day-to-day OCaml maintenance load, and align research agendas with real-world projects to help progress the language and make it available and applicable to a larger audience. Building on 40 years of language development, together with INRIA our goal is to freely release and integrate all work upstream, allowing all prospective users access to the efficient, expressive language of OCaml. 8 | 9 | The OCaml Labs team is a distributed group of 50+ people, comprised of researchers at the University of Cambridge Computer Laboratory, industrial partners, student interns and individual collaborators. From a combination of research grant and industrial funding we are able to fund a small full-time team at the University of Cambridge, 2-4 student interns each academic year and regular community events to encourage contribution and collaboration. 10 | 11 | This report details the work carried out by our group in 2016 and outlines prospective projects for 2017. You can see posts from previous years, in [.html](https://ocamllabs.github.io/furore/yearly.html) and [.md format](https://github.com/ocamllabs/furore/tree/master/yearly). 12 | 13 | ### [OCaml Language and Compiler](./compiler.md) 14 | 15 | ### [OCaml Platform](./platform.md) 16 | 17 | ### [MirageOS](./mirageos.md) 18 | 19 | ### [Community and Outreach](./community-and-outreach.md) 20 | 21 | ### [External Collaborators](./collaborations.md) 22 | 23 | ### [Publications](./publications.md) 24 | 25 | ### Releases 26 | 27 | ### [2017: Future Work](2017-focus.md) 28 | -------------------------------------------------------------------------------- /graphics/ocl13rev/c.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/c.png -------------------------------------------------------------------------------- /graphics/ocl13rev/camel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/camel.png -------------------------------------------------------------------------------- /graphics/ocl13rev/compiler-hacking-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/compiler-hacking-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/compiler-hacking.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/compiler-hacking.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/daniel-presentation-vg.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/daniel-presentation-vg.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/fpdays2013-04-thumb.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/fpdays2013-04-thumb.jpeg -------------------------------------------------------------------------------- /graphics/ocl13rev/fpdays2013-04.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/fpdays2013-04.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/icfp1.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/icfp1.jpeg -------------------------------------------------------------------------------- /graphics/ocl13rev/js.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/js.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/lab_main.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/lab_main.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/makespace-apr-2013.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/makespace-apr-2013.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/marius-yaron-icfp-thumb.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/marius-yaron-icfp-thumb.jpeg -------------------------------------------------------------------------------- /graphics/ocl13rev/marius-yaron-icfp.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/marius-yaron-icfp.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/mirageos.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/mirageos.png -------------------------------------------------------------------------------- /graphics/ocl13rev/nymote.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/nymote.png -------------------------------------------------------------------------------- /graphics/ocl13rev/ocaml-org-thumb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/ocaml-org-thumb.png -------------------------------------------------------------------------------- /graphics/ocl13rev/opam11-beta-contributors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/opam11-beta-contributors.png -------------------------------------------------------------------------------- /graphics/ocl13rev/opam11-beta-packages.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/opam11-beta-packages.png -------------------------------------------------------------------------------- /graphics/ocl13rev/opam11-beta-unique-packages.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/opam11-beta-unique-packages.png -------------------------------------------------------------------------------- /graphics/ocl13rev/opam11-contributors-dec13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/opam11-contributors-dec13.png -------------------------------------------------------------------------------- /graphics/ocl13rev/opam11-packages-dec13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/opam11-packages-dec13.png -------------------------------------------------------------------------------- /graphics/ocl13rev/opam11-unique-packages-dec13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/opam11-unique-packages-dec13.png -------------------------------------------------------------------------------- /graphics/ocl13rev/pfff-thumb.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/pfff-thumb.jpeg -------------------------------------------------------------------------------- /graphics/ocl13rev/pfff.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/pfff.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/rackspace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/rackspace.png -------------------------------------------------------------------------------- /graphics/ocl13rev/rwo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/rwo.png -------------------------------------------------------------------------------- /graphics/ocl13rev/thomas-nycoug-2013.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/thomas-nycoug-2013.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/tony-stark-hud.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/tony-stark-hud.jpg -------------------------------------------------------------------------------- /graphics/ocl13rev/xenserver.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl13rev/xenserver.png -------------------------------------------------------------------------------- /graphics/ocl14rev/c.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/c.png -------------------------------------------------------------------------------- /graphics/ocl14rev/christs-dinner-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/christs-dinner-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/christs-dinner.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/christs-dinner.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/compiler-hacking-dsyme-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/compiler-hacking-dsyme-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/compiler-hacking-dsyme.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/compiler-hacking-dsyme.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/jeremy-rwo-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/jeremy-rwo-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/jeremy-rwo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/jeremy-rwo.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/l23-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/l23-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/l23.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/l23.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/lab_main.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/lab_main.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/nsdi-deadline-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/nsdi-deadline-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/nsdi-deadline.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/nsdi-deadline.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/ocaml-org-thumb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/ocaml-org-thumb.png -------------------------------------------------------------------------------- /graphics/ocl14rev/ocl-pub-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/ocl-pub-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/ocl-pub.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/ocl-pub.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/oleg-talk.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/oleg-talk.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/opam-in-nice-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/opam-in-nice-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/opam-in-nice.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/opam-in-nice.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/opam12-contributors-mar14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/opam12-contributors-mar14.png -------------------------------------------------------------------------------- /graphics/ocl14rev/opam12-packages-mar14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/opam12-packages-mar14.png -------------------------------------------------------------------------------- /graphics/ocl14rev/opam12-unique-packages-mar14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/opam12-unique-packages-mar14.png -------------------------------------------------------------------------------- /graphics/ocl14rev/qcon-unikernel-talk-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/qcon-unikernel-talk-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/qcon-unikernel-talk.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/qcon-unikernel-talk.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/scotty-thumb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/scotty-thumb.jpg -------------------------------------------------------------------------------- /graphics/ocl14rev/scotty.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/ocl14rev/scotty.jpg -------------------------------------------------------------------------------- /graphics/origami-camel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/origami-camel.png -------------------------------------------------------------------------------- /graphics/travis-mascot-200px.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/graphics/travis-mascot-200px.png -------------------------------------------------------------------------------- /monthly/2017/2/let-def.md: -------------------------------------------------------------------------------- 1 | February 2017 - Fred Bour 2 | 3 | - Currently porting merlin to OCaml 4.05. It went smoothly for linux & osx 4 | - Finishing the windows work, with some help from @dra27 5 | - Implemented some features from the roadmap, it is not yet stable but should be available for tests in a week or two. 6 | - ocaml-migrate-parsetree work and porting guide 7 | - Versioned ppx_tools in a manner similar to ocaml-migrate-parsetree 8 | -------------------------------------------------------------------------------- /output/graphics: -------------------------------------------------------------------------------- 1 | ../graphics -------------------------------------------------------------------------------- /scripts/build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh -ex 2 | 3 | docker build -t furore-local . 4 | 5 | RUN="docker run -w /mnt -v `pwd`:/mnt furore-local" 6 | 7 | # docker run -w /mnt -v `pwd`:/mnt furore-local pandoc --toc -f markdown `cat CHAPTERS.txt` -o output/report.pdf 8 | ${RUN} sh -c "cd weekly && rm -f weekly.ml.exe && opam config exec -- ./weekly.ml > weekly.md" 9 | ${RUN} pandoc --toc -f markdown weekly/intro.md weekly/weekly.md -t html5 --css github-pandoc.css -o output/weekly.html 10 | ${RUN} pandoc --toc -f markdown `cat CHAPTERS.txt` -t html5 --css github-pandoc.css -o output/index.html 11 | ${RUN} pandoc --toc -f markdown -S yearly/intro.md `cat YEARS.txt` --css github-pandoc.css -o output/yearly.html 12 | ${RUN} pandoc --toc --epub-cover-image=graphics/origami-camel.png -f markdown yearly/intro.md `cat YEARS.txt` -t epub -o output/yearly.epub 13 | ${RUN} pandoc --toc --epub-cover-image=graphics/origami-camel.png -f markdown weekly/intro.md weekly/weekly.md -t epub -o output/weekly.epub 14 | -------------------------------------------------------------------------------- /scripts/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh -ex 2 | 3 | eval `ssh-agent -s` 4 | chmod 600 furore_id_rsa 5 | ssh-add furore_id_rsa 6 | cd output 7 | git init 8 | git checkout -b gh-pages 9 | rm graphics 10 | cp -r ../graphics . 11 | git add * 12 | git commit -m sync -a 13 | git remote add origin git@github.com:ocamllabs/furore 14 | git push -u origin +gh-pages 15 | -------------------------------------------------------------------------------- /scripts/furore_id_rsa.enc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ocamllabs/furore/e9262050bb19f8659591f2ce9d66403ae38ce06d/scripts/furore_id_rsa.enc -------------------------------------------------------------------------------- /weekly/2016/16/dinosaure.md: -------------------------------------------------------------------------------- 1 | * A robust implementation of email (I check with a complete suit test from another implementation in PHP and JavaScript) 2 | * A robust implementation of meta-data of email (the header of email), it's compliant with old and new standard - 822, 2822 and 5322) 3 | * A beginning implementation of multipart email (I implemented Base64 and QuotedPrintable compute for encoded data) 4 | * Implementation of inline encoded data (compliant with the standard 2047) 5 | 6 | You can see all at: 7 | -------------------------------------------------------------------------------- /weekly/2016/17/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I fixed some bug with parsing of email and header 2 | * I implemented the RFC 5321 (to have a literal domain in an email) and use Ipaddr library for this part 3 | * I implemented the validator website (http://oklm-wsh.github.io/Validator/validator.html) with some tests of email 4 | * I implemented the header fields from RFC 2045 5 | * I attacked the implementation of RFC 2046 (a multipart email) 6 | * I implemented a PPX to inject an IANA database in MrMime project (and I talked about that with Rudy for an integrate of that in Cohttp maybe) 7 | 8 | I talked with Richard Mortier and I think I will avoid the implementation of RFC 3798 (to permit a long parameter in a Content-Type field) and move fast to the implementation in a unikernel when I finish the multipart. So I think it is the last last RFC (to much RFC ...). 9 | -------------------------------------------------------------------------------- /weekly/2016/17/engil.md: -------------------------------------------------------------------------------- 1 | * ocaml-tls binding: Separating handling of IOs from the OCaml code: now the C program must handle all IOs (networking, file reading)… 2 | * As of friday this is mostly finished and the handshake from a simple C TLS client is working. 3 | * A working prototype of the binding is to expect by the end of next week. 4 | * Worked on Canopy: Improved CSS, various bugfixes and PR reviews 5 | -------------------------------------------------------------------------------- /weekly/2016/19/dinosaure.md: -------------------------------------------------------------------------------- 1 | * As you know, I released Syndic 1.5 with ptime to compile in MirageOS. Hannes merged this change in Canopy 2 | * I optimized Decompress and I won 4 MB/s in Inflate 3 | * Jeremy Yallop send me a snippet for Decompress to win 2 MB/s (I didn't try this snippet yet but it is my TODO) 4 | * I came back to MrMime and fixed little bug in multipart parsing 5 | * I reorganized MrMime to have a better interface 6 | * I tried to use ocaml-imap to communicate with my GMail and it's work fine 7 | * But I'm focussing on the test of parsing now so I think the next, I will do just a many test to prove the maturity of MrMime (I already started) 8 | * May be I cook some cookies tomorrow for Monday :) 9 | -------------------------------------------------------------------------------- /weekly/2016/19/engil.md: -------------------------------------------------------------------------------- 1 | * TLS: Implemented a few simple clients and servers using various C TLS libraries (libtls, mbedtls) to understand their API. Started to implement the binding trying to replicate libtls's interface. 2 | * Various things on Canopy (bugfixes) 3 | * Spent some time playing with Reason (reading the doc, tried to implement a web toplevel for Reason using js_of_ocaml), tried to debug an issue with ocaml-git 4 | -------------------------------------------------------------------------------- /weekly/2016/20/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I fixed some little bug with MrMime and the "real world" email 2 | * Firstly, I use a script with fetchmail program to download my email 3 | * But, I implemented a library to communicate with my GMail (with the POP3 protocol), you can see the project: https://github.com/oklm-wsh/Jackson (it's a little project, and it just a prototype to test MrMime) 4 | * So I continue to fix MrMime with the real world and automatize the test suite 5 | * At the same time, I look sessions type and GADT in OCaml to create a tool to describe a protocol (like POP3 or SMTP) with GADT (so a typed protocol), may be if I have a good result with GADT (because it's very complex), I use that for Jackson and for the implementation of SMTP 6 | * So not specifically productive but, step by step, I have a little prototype to communicate with GMail (or another service) and a resilient implementation of email 7 | -------------------------------------------------------------------------------- /weekly/2016/20/engil.md: -------------------------------------------------------------------------------- 1 | * TLS: Halfway through the binding implementation, the configuration parsing from libtls is now working for ocaml-tls, only things left is writing and reading on sockets and the binding will be complete. 2 | -------------------------------------------------------------------------------- /weekly/2016/20/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Module lifting was implemented, i.e. external modules may now be used in static code by simply adding a caret (`^`) to their names, e.g.: 2 | 3 | Lifting of module defined inside the compilation unit (via `module M = 4 | struct... end`) is not supported, as the definition of such modules may 5 | depend on values that are only known at runtime. For this reason, only external, compiled modules can be lifted. 6 | * Support for accessing static values of internal modules was added, i.e. the following compiles and prints what is expected: 7 | 8 | ``` 9 | module M = struct 10 | open ^Pervasives 11 | 12 | static rec take n = function 13 | | [] -> [] 14 | | x :: xs -> if n > 0 then x :: take (n-1) xs else [] 15 | 16 | module N = struct 17 | static rec l = "0" :: l' 18 | and l' = "1" :: l 19 | end 20 | end 21 | 22 | open ^Pervasives 23 | 24 | static () = 25 | print_endline @@ ^String.concat "" @@ M.take 10 M.N.l 26 | ``` 27 | 28 | Limitations: 29 | * Static value description in signatures (i.e. `static val x : 30 | some_type`) are supported by the parser, but not handled by the typechecker for now, as it implies modifying `Includemod`. Handling static components in signatures will be necessary for the next step, that is, saving and loading static code from/to external files. 31 | 32 | -------------------------------------------------------------------------------- /weekly/2016/21/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I updated MrMime with a new program maildir. You can execute this program with ./maildir -p /path/to/your/mails/ -n LFif the newline of your email is LF character (and it's probably your case with mac osx). I retrieve two difficulties about the parsing: 2 | - The difficulty to predict the end of the email - so may be you catch an infinite loop when you scan your emails 3 | - The latency with Base64 and QuotedPrintable decoder (because, for each character, we need to try the boundary parser to stop the decoder) 4 | * So you can use the project on two way. The first way is to use maildirbinary. But if you catch a infinite loop for example (or a weird exception), you can try with a specific email with utop -init test.ml(the second way). At this moment, you can compile the project with debug information (with ./configure --enable-trace and make install) and see where MrMime fails. The parsing is resilient with the malformed header. So now, I will implement the pretty-printing of email and prove if the parsing and pretty-printing is bijective (just to be on the safe side). 5 | 6 | (you need to configure the project with --enable-teststo get the maildirbinary) 7 | -------------------------------------------------------------------------------- /weekly/2016/21/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Support for loading static components of external modules was added. This implied compiling the static part of modules to `.cmm` files. 2 | * When using lifted versions of standard modules (e.g. `Printf`), the compiler would load the corresponding `.cmo` file. The problem is that standard modules are already present in the bytecode of `ocamlc` itself, so lifted standard modules were, in a way, loaded twice. In addition of being useless, this second loading would cause problems with side effects implying 3 | buffering (e.g. when using `Printf.printf`, the ordering of outputs was wrong). This 4 | was fixed by adding a function `Symtable.init_static ()` that simply calls `init_toplevel ()` and then adds the lifted version of every global identifier (pointing to the same position in the bytecode) directly into the 5 | symtable. This way, standard modules are not loaded twice, and their use no longer involves reading an external `.cmo` file. 6 | * At that point the produced lambda code was highly unsafe to use because it pointed to runtime and static fields of external modules as though they were in a same record, which could lead to wrong indexes and segfaults. This was fixed for now by adding 0 as a placeholder for non-static (resp.non-runtime) components in `.cmm` (resp. `.cmo`) files. A more sophisticated way might be necessary in the long term. 7 | 8 | Limitations: 9 | * Constrained internal modules (i.e. `module M : sig ... end = struct...`) is not supported by `Translstatic` yet. Nor are functors. These constructions are translated as the unit lambda. 10 | * `Includemod` is still phase-agnostic, so interface mismatches involving values with the right name and the wrong phase will not be detected. 11 | -------------------------------------------------------------------------------- /weekly/2016/22/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I optimized the Base64 decoder but MrMime is slow with QuotedPrintable (but I have an idea to fix that) 2 | * MrMime is more resilient with bad email (like an email without Fromand Dateinformation) 3 | * I fixed some little bug (infinite loop, order of meta-data, multipart inception, unstructured data, and raised exception) 4 | * I retrieved a regression with my test (so, I have a good tests suite) and I fixed this regression - it's a specific problem with a wrong example from the standard RFC 822 5 | * I started the implementation of pretty-printing, so I continue the prototype (I am inspired by the Format module from OCaml standard lib to respect the 80 characters rules) 6 | * I will use an archive from the caml-list (from daniel) to have a good tests suit about the parsing of email (it's a huge example about the old/bad emails) 7 | * So this week, it's just some fixes and a minimal prototype about the pretty-printing :s 8 | -------------------------------------------------------------------------------- /weekly/2016/22/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Phase mismatch is now detected in signatures (i.e. the compiler will complain if you try to match a `static val` and a `val`). 2 | * We are now focusing on quoting and splicing. The splicing part involved quite a lot of refactoring in `Translstatic`: `Translstatic` used to translate an implementation file into a lambda creating the module and adding a reference to it in the symtable. Now the lambda generated by `Translstatic` not only creates the module but also returns an array of untyped ASTs, corresponding to all the splicings of code into phase zero (i.e. splicings that are not inside a quotation). 3 | * In a new module `Runstatic`, this static lambda code is run and the splicings are actually reified. They are then used by `Translcore` to "fill" phase-zero splicings every time it encounters one. This approach is implemented and compiles, but not yet thoroughly tested, since to test splicing something has to be quoted, and the quoting library is not integrated into the compiler yet. 4 | * Finally, I started working on integrating the quoting library written by Leo White. The main function `Translquote.quote_expression`, takes a typed tree and returns the lambda code that will generate the isomorphic *untyped* tree at run-time (well, at static run-time). This work is still going on. 5 | 6 | Limitations: 7 | * As stated above splicing is untested for now. 8 | * Because I didn't know whether I should reimplement some parts of `Translmod` in `Translstatic`, module coercions are still not handled by `Translstatic`. So for now, coercing static components of a `.ml` using a `.mli` won't work — the fix is easy but I'm focusing on quoting for now. 9 | 10 | -------------------------------------------------------------------------------- /weekly/2016/23/dinosaure.md: -------------------------------------------------------------------------------- 1 | * So I focused this week on my abstract for the ICFP (so Richard corrected the paper and I have sent Friday) 2 | * I continue to implement the pretty-printer of MrMime (I implemented without regresssion with my unit test) 3 | * I start to look S/MIME (secure mail) - so may be I will implement with ocaml-tls the secure mail one time 4 | -------------------------------------------------------------------------------- /weekly/2016/23/engil.md: -------------------------------------------------------------------------------- 1 | * My first step is to take @yallop script to generate ctypes bindings and make it generate some OCaml parse tree instead of generating OCaml sources so I try to find some ways of integrating it easily in a Reason/OCaml flow. One of the goal would be trying to avoid as much as possible fighting with a build system 2 | * I also took some time to finish a Reason js_of_ocaml REPL I started some months ago. We will probably communicate about it soon publicly, but for now it’s available there: https://engil.github.io/reason-web-toplevel/ 3 | 4 | -------------------------------------------------------------------------------- /weekly/2016/23/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * The integration of Leo White's quoting library, as well as the implementation of top-level splicing, were completed. As a consequence, it is now possible to compile programs containing macros as long as CSP is not needed (except CSP for global identifiers which comes for free), for example: 2 | 3 | ``` 4 | open ^Pervasives 5 | static rec power' n x = 6 | if n = 0 then 7 | << 1 >> 8 | else if n mod 2 = 0 then 9 | let y = power' (n/2) x in 10 | << let z = $y in Pervasives.( * ) z z >> 11 | else 12 | << Pervasives.( * ) $x $(power' (pred n) x) >> 13 | 14 | (* val power' : int -> int expr -> int expr = *) 15 | static power n = 16 | << fun x -> $(power' n <>) >> 17 | (* val power : int -> (int -> int) expr = *) 18 | let power_five = $(power 5) 19 | (* val power_five : int -> int = *) 20 | ``` 21 | 22 | * Execution of static bindings and splices was implemented in the REPL. Declaring modules with static components in the REPL is not supported for now. 23 | * The `Expr` module, for phase-invariant `'a -> 'a expr` conversions, was added. Using this module it is for instance possible to write the following function: 24 | 25 | ``` 26 | static rec range = function 27 | | 0 -> << [] >> 28 | | n -> << $(^Expr.of_int n) :: $(range (^Pervasives.pred n)) >> 29 | ``` 30 | 31 | * The tracking of stage (defined as the number of surrounding quotes, minus the number of non-top-level splices) was implemented. It makes it possible to tell whether a identifier in a quote will be usable at the splicing point or whether some form of CSP is needed. Since CSP is not implemented at the moment, when encountering a (non-global) identifier of the wrong stage a type error is thrown. (Type errors don't abruptly stop the program but show pretty-printed errors with the location of the problem). 32 | -------------------------------------------------------------------------------- /weekly/2016/23/philipdexter.md: -------------------------------------------------------------------------------- 1 | My work this summer is about providing approximate programming tools for OCaml programmers. There are several different styles of approximate computing; each requires of their users different technical knowledge of the subject. 2 | 3 | As one example, loop perforation is an approximate computing technique where iterations of a loop are skipped in the pursuit of reducing resource usage (be it time or energy). A loop perforation build system will generally have a self tuning step where, given test input and a fitness function from the user, the system can automatically deduce an approximation strategy. Further refinement by the user is indeed an option, but this is a good example of a technique which asks little of its user. 4 | 5 | Loop perforation's ease of use leads KC and I to believe that adding a loop perforation system in OCaml is a great first step in the broader plan of supplying a goodie bag of approximation tools. 6 | 7 | I have written a ppx extension for loop perforation. It is basic. In writing it I have both learned a lot about ppx extensions and expanded my compiler knowledge. The next steps are to decide whether to implement a self tuning system. If yes: the ppx extension and the self tuning system would make for a decent first release of an approximation system for OCaml. If no: there are other techniques for approximate computing that we can play with before we decide where to spend coding efforts. 8 | 9 | We have met with a group in the lab which is interested in prolonging the flight time of their drone operations. It is hard to say whether there is a collaboration opportunity; approximate computing is not always the right tool for the job. 10 | -------------------------------------------------------------------------------- /weekly/2016/24/dinosaure.md: -------------------------------------------------------------------------------- 1 | I did some technical updates in MrMime: 2 | * First, I added an abstraction about the operating system (Windows and Unix) 3 | * I reorganized the project to produce a good low-level API to manipulate an email 4 | * I finished, as you know, the encoder (and I can produce an email now) 5 | * I optimized the Base64 and the QuotedPrintable decoder (with tailcall optimization) 6 | * I finished the pretty-printing of an email 7 | * I fixed some little bugs between the encoder and the decoder 8 | * And I added the support of UTF-8 with the uutfsoftware by daniel - now, I have only 3 errors with 2000 mails (so ~0.15% fails - so MrMime is more resilient) 9 | * I talked with Daniel about the API and the support of UTF-8 (because an email can have many encoding data) and the main thing is to normalize the email on one encoding, the UTF-8. But, for that, I need another software to normalize any encoding to UTF-8 (daniel told me who will do) 10 | * May be, the next week, I will start the SMTP protocol with the unikernel 11 | -------------------------------------------------------------------------------- /weekly/2016/24/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * I finally implemented support of static components in coercions. Now, module interfaces can be fully constrained (not only a subset consisting only of runtime values). 2 | * As a necessary condition for that change, the code duplication between `Translmod` and `Translstatic` was reduced to a minimum. Now, all `Translstatic` does is create the necessary lambda code to run splices. 3 | * When trying to use external static code in REPL, I realized that the it couldn't work because of the coexistence of ambiguous identifiers in the symbol table (e.g. `Expr` may be bound to the runtime part of the `Expr` module used in runtime code, or the static part of that module used in static code, a problem that doesn't arise in `ocamlc` since only the static parts are run). I am currently working on adding phase information to the symbol table to resolve these ambiguities. 4 | -------------------------------------------------------------------------------- /weekly/2016/24/philipdexter.md: -------------------------------------------------------------------------------- 1 | A user cannot zealously apply approximate computing strategies to every line of every program they write. This week I have looked through three OCaml programs which I believe can benefit from approximation: 1. A ray tracer 2. A H.261 video decoder 3. The black--scholes algorithm for option pricing 2 | 3 | Not all three programs have explicit for-loops---OCaml programmers tend to favor recursion---but all three have perforation opportunity. 4 | 5 | My next step is to implement an interactive system which will guide a user in approximating their programs. It will first identify all potential loops/recursive calls in a program which allow perforation. Given this set it will then attempt to find the /best/ combination of perforation. Testing all possible combinations of loop perforation is unfeasible so some heuristics must be developed. 6 | 7 | I also spent some time this week working on multicore OCaml. I wet my feet by writing a recursive lock implementation. The code can be found on [github] along with the [issue] requesting the implementation. 8 | 9 | [github](https://github.com/ocamllabs/reagents/compare/master...philipdexter:recursive) 10 | 11 | [issue](https://github.com/ocamllabs/reagents/issues/2) 12 | -------------------------------------------------------------------------------- /weekly/2016/25/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I continue to optimize MrMime and I will use the Bigstring (which is more faster than the String module) with my RingBuffer 2 | * I finished to implement the interface (recommended by david) of my parser combinator (it is the same interface as the angstorm library from seliopou but with a different backend - as you know I use a ring-buffer) 3 | * So I continue the large background work on MrMime and keep the stability (and avoid any regression) with my unit tests. So, nothing impressive but needed to concurrent any other implementation of email :) 4 | -------------------------------------------------------------------------------- /weekly/2016/25/philipdexter.md: -------------------------------------------------------------------------------- 1 | Our perforation system allows programmers to annotate looping constructs which they believe can benefit from perforation. Compiling an annotated program into an optimized program which uses loop perforation is not straghtforward. 2 | 3 | The most difficult task of this system is in finding the optimal perforation configuration. One solution is to profile every possible permutation of configurations. As this is entirely unfeasible the most truly difficult task is finding a heuristic which allows us to prioritize configurations. 4 | 5 | After running some subset of all configurations, there will be a smaller subset of optimal configurations which form a time-error tradeoff curve. Our end result would present this graph to the user, allowing them to choose which tradeoff works best for them. 6 | 7 | My time this week has been split between converting a black--scholes C implementation to OCaml and a first draft of our perforation program. The perforation program can successfully run different perforation configurations however there are no heuristics involved yet. 8 | -------------------------------------------------------------------------------- /weekly/2016/26/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I continue to improve the interface of MrMime 2 | * I succeed to use the interface of david (it's the same of angstorm project) - now, the user can produce its parser 3 | * I find some littles regressions with my big test (my 2000 emails) - I will fix that 4 | * We can use the bigstring instead the string module now (so, possibly, we can improve the performance - but I need some tests about that) 5 | * I continue to documente the project and after we can have a release of MrMime 6 | * For this week, I will work on Decompress (as Richard advise me) about the performance, I already a PoC from zlib so I will test it. 7 | * I think I continue to work the morning on MrMime (because is just some fixes) and after I work on Decompress. 8 | -------------------------------------------------------------------------------- /weekly/2016/26/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Vast changes in the way dependencies of static code are loaded: although `.cmm` files are still automatically loaded from the include path (`-I`), `.cmo` and `.cma` dependencies of static code must now be specified using the `-m` command-line flag. Since dependencies of dependencies must be specified as well, it can lead to rather large compilation commands. This is intended as a backward-compatibility feature, since hopefully dependency management in OCaml should soon evolve to loading `.cmo` files from path during linking, if Leo White's proposal for that gets accepted. 2 | 3 | The changes made to dependency loading involved a bootstrap of the compiler, and a modification of the stdlib makefile (and 3 days of very painful debugging because of a line that hadn't been removed). 4 | * A rebase on `trunk` was performed to avoind falling behind, but some work is necessary to get the native compiler running again (without trying to make it support macros yet) before the testsuite can be run. 5 | * Static and runtime declarations can now be interleaved in the REPL. To achieve this, the symbol table used for linking now carries phase information (i.e. the symtable is now mapping (identifier, phase) pairs to positions in the global data array). 6 | * Cross-stage identifiers may now be quoted inside toplevel splices. To do that, it was necessary to make the typechecker track the top-levelness of the current environment, but also to propagate upwards the set of cross-stage identifiers encountered in the environment. The only missing form of cross-stage persistence (CSP) left to implement is *path closures*. 7 | * Redaction of a report about macros to motivate a second, 6-month internship, starting in September. 8 | -------------------------------------------------------------------------------- /weekly/2016/26/philipdexter.md: -------------------------------------------------------------------------------- 1 | Our perforation system is ready for initial testing so this week I've mostly worked on running experiments. I translated a swaptions program into OCaml. A lot of time was spent fixing bugs in the translation but once everything was set up I ran it through our loop perforation system and have some initial results. 2 | 3 | We haven't quite reproduced the results from the C perforation system in the paper I'm referring to. They show results where a 5x speedup is achieved with only a 1% loss in accuracy. Our system can only produce a 1.2x speedup while losing 5% accuracy. We can get to 2x speedup but we have to sacrifice 40% accuracy. We do not perform the same state space searching as the paper version does so when that is all set up we may be able to reproduce the results. 4 | 5 | I'm confident that once we perform the same sort of searches as the C version we can gain similar speedups. This will be an immediate goal of next week. 6 | -------------------------------------------------------------------------------- /weekly/2016/27/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I defunctorized Decompress, so the API is more simply and I won 2 Mb/s for the inflate but I losted 2 Mb/s for the deflate 2 | * Now Decompress uses memcpy function instead memmove (standard function used by OCaml), it's a technical aspect but a big technical point about the portability on Decompress (on Xen architecture specifically). So, I talk about that with Jeremy and we think we have no problem and may be it's a good change to improve the performance of Decompress 3 | * I used a project from LexiFy, landmarks (Thomas advised me to use that), and Spacetime from Mark to find the bottleneck in Decompress and I found this Friday. I have an idea to optimize Decompress (again and again) but I think, we found the the big problem about the performance in Decompress (and it converges with Jeremy's idea) 4 | -------------------------------------------------------------------------------- /weekly/2016/27/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Testing of rebased branch and subsequent corrections finished. The remaining test failures are (all but one) due to lack of support for recursive modules, except for an unexpectedly long type error, probably due to the twofold compilation. 2 | * Alain Frisch's native toplevel (`ocamlnat`) now compiles again, but has no macro support. 3 | * Macro support was added to `ocamlopt` (careful, not `ocamlc.opt`). There was almost nothing to do since `ocamlopt` is a bytecode program. 4 | * Installing packages with `opam` is possible again but the fix is very ugly (`ln -s ocamlc ocamlc.opt`). `opam` doesn't seem to support bytecode-only installations. 5 | -------------------------------------------------------------------------------- /weekly/2016/27/philipdexter.md: -------------------------------------------------------------------------------- 1 | This week I have generalized the perforation system to the point where I was able to take an off-the-shelf ocaml implementation of kmeans clustering and run it through the perforation system. The program was able to produce nice, readable results for different configurations [attached image]. However when more than 2 loops are perforated, the data starts becoming very hard to visualize. The system is really smooth but it's still lacking: I had to turn one use of List.iter into a for loop. However, I have created a plan to perforate recursive functions which will allow the system to work on programs which don't use for loops. 2 | 3 | This week I also worked with the reagents library for ocaml-multicore. In the beginning of the week I parallelized an existing ray tracer written in OCaml. (The ray tracer also is a good candidate for perforation.) Further, KC and I have discussed working on a lock-free hash table implementation to use in the Hack type checker (written in OCaml). Currently hack use a C hash table implementation strapped on the to OCaml code. Writing a competitive implementation in OCaml would be a great way to show of ocaml-multicore. 4 | -------------------------------------------------------------------------------- /weekly/2016/28/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I push my big change to MrMime, so Richard launched a test with ~18 000 emails and all emails are ok (we catch no error). 2 | * We find a performance problem (with the multipart and the quoted-printable encoding) but it's a specific problem (so when I have a time, I will fix this problem) 3 | * But, overall, MrMime is more faster (it's the result of improvement of internal buffer) 4 | * After, I worked on Decompress and I defunctorized the project, I optimized the translation between op-code and character and I avoided some closures allocations, but, with all this point, with Jeremy, we can't notice any imrpovement :disappointed: 5 | * So, I will try a deforestation of Decompress and we will see if we have a good result 6 | * For the hackaton, I worked on Decompress, but, as I said, no big change and just some experimentals modifications. 7 | * For the next week, I will start to fix some bug in ocaml-git as Richard advised me. Voilà! 8 | -------------------------------------------------------------------------------- /weekly/2016/28/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Automatic loading implemented in the bytecode linker, both at compile-time and for linking of runtime code. Automatic here means that the needed .cmm, .cmo and .cma file are automatically searched for in the include path. It's mainly for convenience reasons now, if it is too big a change it can be suppressed later. 2 | * `ocamlc.opt` should work very soon, I just need to make phase-1 built-in exceptions work. 3 | * I experimented a bit, trying to use macros with the Markup.ml library (with HTML templates in mind). The very simple following code runs and prints "

lol

": 4 | 5 | ``` 6 | static identity str = 7 | let open ^Markup in 8 | let open ^Pervasives in 9 | let res = 10 | (string str |> parse_html |> signals |> write_html |> to_string) in 11 | Expr.of_string res 12 | 13 | let () = print_endline $(identity "

lol") 14 | ``` 15 | 16 | I will try writing more useful macros ASAP. 17 | -------------------------------------------------------------------------------- /weekly/2016/28/philipdexter.md: -------------------------------------------------------------------------------- 1 | This week I split my time under two projects: the continuing of my auto perforator and the datakit project. 2 | 3 | The auto perforator is coming along very nicely. There is now a hill climbing algorithm built in which will explore the perforation state space. I have the system working under 3 domains: swaptions, ray tracer, and kmeans. I will give a talk about the system next Tuesday. 4 | 5 | The datakit project is under KC's guidance. The idea is to use datakit to power a distributed key-value store. The implementation would be a domain for testing systems which adapt to byzantine errors by changing the strength of their reads into the database. 6 | -------------------------------------------------------------------------------- /weekly/2016/29/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I finished fixing some bugs with Decompress, so we have a stable version after many optimization. As I explained, we don't have a big improvement on the performance but Jeremy will take the relay 2 | * After Richard focused me on a performance problem in MrMime, so I fixed the bug. It's a technical point bug I wait Richard to launch an other test 3 | * I started a little library to see any differences between an email before and after decoding/encoding, I think, I will finish this mini-project this week-end - and may be it's a useful project for irmin 4 | -------------------------------------------------------------------------------- /weekly/2016/29/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Macro support has been implemented in `ocamlc.opt` (bytecode compiler running on the native back-end), after various bug corrections. The execution of macros is as follows: 2 | 1. static code is compiled to bytecode and written to a temporary `.cmm` file 3 | 2. that `.cmm` file is linked with its dependencies to make a temporary bytecode executable 4 | 3. using `Sys.command`, `ocamlrun` is called on that file. The executable runs and writes the top-level splices to the file passed as a command-line 5 | argument. 6 | 4. The untyped parse trees are unmarshalled and spliced into the run-time code. 7 | * I pushed further the idea of HTML templates using the Markup.ml library. An example of a code that can be run (and that includes run-time strings in the template) is: 8 | 9 | ``` 10 | open Markup 11 | open Htmltemplate 12 | let par = 13 | Printf.printf "Please enter a string: "; 14 | read_line () 15 | let () = 16 | $(template [Lit {| 17 | 18 | 19 | 20 | Some page 21 | 22 | 23 |

Some page

24 |

|}; Var <>; Lit {|

25 | 26 | 27 | |} ]) 28 | |> of_list |> pretty_print |> write_html |> to_string 29 | |> print_endline 30 | ``` 31 | 32 | The HTML tree is constructed statically, entailing no parsing costs at run-time. The next step is to convert the Markup.ml tree to a TyXML tree to have the typechecker enforce the validity of the HTML tree. 33 | -------------------------------------------------------------------------------- /weekly/2016/29/philipdexter.md: -------------------------------------------------------------------------------- 1 | Automatic perforation is coming to the point where it could be ready for more users. I have slowly started to focus on providing a good user interface. 2 | 3 | This week I ran my auto perforation tool on two sets of inputs for kmeans with the goal of showing that results from a run of the automatic perforation program can be used to approximate further inputs. That is, running kmeans with the automatic perforator gives you a result on how to best perforate your loops. Using this result with future data sets previously unseen gives you just as good results. For this experiment I trained the perforation using 10 inputs with 100,000 points. I then ran the resulting perforated program on 10 other inputs of 1,000,000 points and achieved very similar results (a 5x speedup with only a 10% loss in accuracy). 4 | 5 | This week I've written a blog post about the troubles with writing generic approximate computing frameworks (found at: http://phfilip.com/the-difficulty-in-a-general-approximate-computing-framework.html) 6 | 7 | This week I've also discussed with KC at lengths about a use-case of approximate computing. The general idea is to add approximate reasoning to programs which work over distributed key-value stores. Programs in this domain are usually already written to handle consistency anomalies. The addition of a dynamic system to track and handle the levels of approximation could be a good fit for my approximate computing work. 8 | -------------------------------------------------------------------------------- /weekly/2016/30/ciaran16.md: -------------------------------------------------------------------------------- 1 | **Monday** 2 | - Rewrote a lot of the code as I was using quite a bit of mutable state. For example, using an array to store the current input. Now storing it as two int lists (ints as characters are unicode), the first representing the input before the cursor (stored in reverse), and the second representing the input after the cursor. Lots of other changes as well. 3 | **Tuesday** 4 | - Some more rewriting. Code a lot terser and almost no mutability. 5 | - Managed to get Notty working with scrolling etc by disabling canonical mode myself then passing the input from stdin into the notty module to filter escape sequences. A bit of a hack but it means I can use notty and have proper scrolling etc, and don't have to redraw everything each time. 6 | - Still a few things that are only half working. For example tab completion is kind of there but not actually 'wired up'. 7 | **Wednesday** 8 | - Not much to log, just starting to get the shell working with ocaml-9p, rather than just the couple of small examples I've been using so far. 9 | - I've written the shell to be synchronous at the moment, which won't work well with things like ocaml-9p, so I'm going to change it to use Lwt throughout. 10 | -------------------------------------------------------------------------------- /weekly/2016/30/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I just improved (again!) the interface of MrMime: 2 | - So the design of the API is approved by Daniel and I wait a feedback of Richard to know if we will release a new version or not 3 | * After, I did go to docker to meet Thomas and prepare a TODO list for ocaml-git. Now, I have a clear idea to what I need to do for this project. 4 | 5 | Not a big week so, but a good week to clarify some points about MrMime and ocaml-git. 6 | -------------------------------------------------------------------------------- /weekly/2016/31/ciaran16.md: -------------------------------------------------------------------------------- 1 | **Wednesday** 2 | 3 | Current state of the shell: 4 | - Commands work, but I haven't actually added most of the commands. E.g. connecting doesn't actually work as it's asynchronous, so using Lwt throughout is top priority. 5 | - Tab completion doesn't really work. It kind of did at one point, but I've changed quite a lot of stuff since then. Should be fairly easy to get working again though. 6 | - Parsing the input works mostly fine, handles quotes / strings, allows escaping etc. 7 | - Notty interface now works, which means I can continue using Notty, but it isn't perfect. 8 | -------------------------------------------------------------------------------- /weekly/2016/31/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I talked with Christophe about the optimization, I have not yet advised Jeremy about the benchmark but I keep this thing when Jeremy will try to optimize Decompress 2 | * I move Decompress to topkg because Anil wanted a Dockerfile inside the project 3 | * Richard retested MrMime and I fixed some bugs and optimized some computations. Now, we compute 180 mails per second and MrMime fails on 500 emails with 815 000 emails! So it's not bad, we will fix the rest with Richard when he comes back 4 | * I fixed a little bug for Qi Li on the Xen librairies 5 | * I continue to document MrMime 6 | * And I looked inside ocaml-git to understand where I need to start but I does not do revelant something for the moment. 7 | -------------------------------------------------------------------------------- /weekly/2016/31/engil.md: -------------------------------------------------------------------------------- 1 | * Worked on the ocaml-tls binding: Fixed a few memory leaks problems. Tried to run spacetime against the binding, to no avail Tried to run landmarks against the binding: not much success too. 2 | -------------------------------------------------------------------------------- /weekly/2016/32/ciaran16.md: -------------------------------------------------------------------------------- 1 | **Monday + Tuesday** 2 | 3 | Rewrote the lexer and parser to be able to understand more about the input, i.e. whether it's a positional arg, flag or optional arg etc. This also requires that the parser have access to the commands and their arguments so that required some more rewriting. This information can then be used for both predictions and syntax highlighting. It's also easier to extend. 4 | Parsing optional arguments is tricky because it means the next positional value might actually be their value. Quite a few edge cases, and haven't added support for a few widely used conventions like using -- to denote that the next value is a positional value, or using = when specifying values. 5 | 6 | **Wednesday** 7 | 8 | Working on making predictions better using the additional information. Quite a lot of cases. Just doing the basic ones for now. Flags only need their name predicted. Non flags need their name predicted but can also predict their value. Positional arguments can be predicted based on their index. 9 | Not working yet for optional arguments. Also not working for some other edge cases. 10 | 11 | **Thursday** 12 | 13 | Added syntax highlighting by traversing the raw input and the syntax tree together. 14 | Some more work on prediction. Refactoring and separating out modules. 15 | -------------------------------------------------------------------------------- /weekly/2016/32/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I continue to document MrMime, you can see the documentation at this link: http://oklm-wsh.github.io/MrMime/ (not very exciting) 2 | * I started a new prototype for ocaml-git, the name is sirodepac, you can see the reopository at this link: https://github.com/dinosaure/sirodepac 3 | For the sirodepac, the goal is to created a non-blocking encoder/decoder. So, Thomas follows the project and it's works for the moment despite some little bugs. 4 | -------------------------------------------------------------------------------- /weekly/2016/33/dinosaure.md: -------------------------------------------------------------------------------- 1 | * I continue to work on a prototype of ocaml-git. I just finished the decoder for the IDX file, I will try the implementation next week. 2 | * I continue to document MrMime - Richard showed me some emails when MrMime fails. These emails are wrong and the thing is, if you use MrMime by the best effort way, you can extract all information from the email, so MrMime can parse these email by the hard way - not the simple way. 3 | -------------------------------------------------------------------------------- /weekly/2016/41/oliviernicole.md: -------------------------------------------------------------------------------- 1 | : I finished merging branch `4.04` into `macros`, but the repo is not functional 2 | yet: I get a segfault on `utils/clflags.ml` when bootstrapping. Strangely, 3 | printf-debugging pointed the segfault to happen on a call to `open_in_bin` 4 | inside `Bytelink.load_object`, although several successful calls have been 5 | made to `open_in_bin` during the execution. This suggests that the global data 6 | containing the Pervasives module has been somehow corrupt. 7 | * Tuesday 8 | : A lot of time spent fighting with the built system and trying to find the 9 | cause of my bug(s), without success. 10 | * Wednesday 11 | : Since `macros` was based on `trunk`, I shouldn't have used `merge` to rebase 12 | it on `4.04`, since changes from `trunk` have been introduced on `4.04`. 13 | In order to rebase macros on `4.04` without having to resolve conflicts on 14 | each commit (which would be necessary to keep the commit history), I had to 15 | squash all the macro-related commits into one commit, and cherry-pick that 16 | commit onto `4.04`. 17 | * Thursday 18 | : Conflicts solved and `.depend` files updated. `make coreall` works but the 19 | segfault on `open_in_bin` is back. What's more, when trying to compile some 20 | dummy file with a static printf in it, the compiler segfaults in 21 | `Symtable.update_global_data`, more specifically on the operation: 22 | `glob.(slot) <- transl_const cst` 23 | where `glob` is `Meta.global_data ()`. I checked, it's not an out-of-bounds 24 | segfault. 25 | Another segfault I'm currently tracking is on a call to 26 | `Pervasives.output_string` (or `output_char`, one of the two). 27 | * Friday 28 | : Today, meeting with Mort and Anil to check the progress of macros. I need to 29 | start writing down ideas of applications. 30 | The runtime segfaults on the following line while executing the instruction 31 | APPTERM2: `pc = Code_val(accu);` 32 | Where `accu` has been set by the previous instruction: `PUSHGETGLOBALFIELD Pervasives, 48` 33 | -------------------------------------------------------------------------------- /weekly/2016/42/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | This week I have been working on the Plot module in Owl library by adding more basic plotting functions. Currently the Plot module supports many widely used plots. There are still many minor things in the module can be improved, but I will do that slowly in the future. I also wrote a tutorial on how to use the plot module and sent it to the ocamllabs mailing list. 2 | 3 | Besides the Plot module, I am also focusing on the Stats module because GSL only provides very basic ones and there are many useful statistical functions are missing. I will implement most of them by myself in OCaml since I do not want to introduce too many dependency in the library. So far, I have implemented z-test, t-test, jb-test, chi2-test, and so on. Based on the feedback, it becomes clearer now that Owl will evolve into a numerical library for OCaml and Matrix, Maths, Stats, Regression, and Plot will be the core modules. 4 | -------------------------------------------------------------------------------- /weekly/2016/43/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | 1. I have implemented complex number support in Owl. The Dense module now have two submodules: Real and Complex which support dense matrices of real number and complex number respectively. 2 | 3 | 2. I also split the current Sparse module into two parts to support both real and complex sparse matrices. The real number is already supported but the complex number support in Sparse module is still missing. 4 | 5 | 3. I removed the dependency on Ctypes.Foreign module in Owl and change completely to stub generation as suggested by Yallop. So the interface to GSL is supposed to be faster and type safer. 6 | 7 | 4. I tested using functor to generalise some matrix operation, it worked fine but caused some performance penalty in certain operations that I am still trying to figure out the reason. 8 | 9 | 5. I gave a pitch of my kvasir project last night in Petershouse last night, since it entered into Cambridge Enterprise Business Plan competition. However, I didn’t win the first prize in the end but nice experience and received a lot of publicity. 10 | -------------------------------------------------------------------------------- /weekly/2016/43/timada.md: -------------------------------------------------------------------------------- 1 | My research topic 2 | - Started learning OCaml, and finished installation of OCaml on my laptop. 3 | - Completed the chapter 1 of Real World OCaml and now following the chapter 2 4 | - Felt I need to have a flexible mind to understand the OCaml language expression. 5 | 6 | Others 7 | - Arrived at Cambridge 16th Oct. 8 | - Found my long term flat and I will make a contract with a landload in this weekend. 9 | - Still setting up IT services (e-mail address, lab network, printer/scanner). 10 | -------------------------------------------------------------------------------- /weekly/2016/44/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | 1. I have optimised the performance of sparse complex matrix module. Many operations are much faster than before now. I think this module is mostly done. 2 | 3 | 2. I have implemented a module for Fast Fourier Transforms with basic operations. I still need some time to finalise this module next week. 4 | 5 | 3. I start writing up some documents to prepare for an opam release in the following weeks. 6 | 7 | 4. I start allocating more time on the distributed data processing framework again this week. The progress on this was slow due to the distraction from Owl. 8 | -------------------------------------------------------------------------------- /weekly/2016/44/seveneng.md: -------------------------------------------------------------------------------- 1 | 1. Measure the performance of PIH-gatekeeper(infrastructure within UCN), mainly the operation latency when serving clients' requests. As there are different branches based on the validity of client certificate and/or the right to access a data holding unikernel, accordingly there are different scenarios where the PIH-gatekeeper need to be measured against. 2 | 3 | 2. As mort successfully updated the dom0 OS from Debian-based to Alpine earlier last week, I started to port the system to the new platform. The out-of-box SDcard image has insufficient storage space assigned for dom0, I can't install required tools/libraries in it, then I started to rebuild one image which has larger persistent storage space. 4 | -------------------------------------------------------------------------------- /weekly/2016/45/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Monday (2016-11-07) 2 | : Laptop still in repair, spent all day trying to get a desktop PC as a 3 | replacement. Will use one of the lab shared computers from now on. 4 | * Tuesday 5 | : Added a new `value_kind` for macros and made the typechecker tag macros with 6 | it. 7 | * Wednesday 8 | : Stopped keeping track of cross-stage identifiers in the typing environment. 9 | Instead, iterate through the typed tree to find them in a expression when 10 | needed. 11 | During the Compiler hacking event at Pembroke, paired with Maxime to work on 12 | signatured opens (https://github.com/ocamllabs/compiler-hacking/wiki/Things-to-work-on#signatured-open-command); we're about halfway through it. 13 | * Thursday 14 | : Add support (parser and typechecker) for macros in signatures. 15 | * Friday 16 | : Add a closure argument to macros, yet unused. 17 | -------------------------------------------------------------------------------- /weekly/2016/45/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | 1. I realised the first version of Owl to OPAM with the help of others, also included it in the MirageOS documentation. 2 | 3 | 2. I started shifting more efforts back to the distributed data processing framework. I rewrote the part of the data parallel and model parallel engine and wrapped them into a separate core module. On top of that, I implemented the one stochastic gradient decedent algorithm using model parallel engine. 4 | 5 | 3. I have been investigating the Latent Dirichelet Analysis model and identified some potential research questions. Me and mort met the student (Ben Caterall) who will work with on his part III project. He will also contribute to Owl. 6 | 7 | 4. I also implemented a simple LDA model using Owl, I hope Ben and me together can make a topic module for Owl. 8 | -------------------------------------------------------------------------------- /weekly/2016/45/seveneng.md: -------------------------------------------------------------------------------- 1 | 1. Working with sys-admin to set up the PIH service to our external collaborators. 2 | 3 | 2. Bumping into some bug in the MirageOS TCP/IP stack, only until yesterday did I be able to locate it , will try to fix it and issue a PR on github later 4 | 5 | 3. Continuing porting some services to Alpine based system. For now, the service accessible to the partners are running on a Debian based system. The binaries for unikernels were easy to deal with, but there are some parts need some dynamically linked libraries, I tried different ways: statically link these libraries; porting these parts together with the libraries; altering the unikernel's implementation to get rid of these parts. For now, I'm settling with the last solution, but I think I would try the `porting with libraries` later, case the bug in "2" made me think that solution wouldn't work initially. Since I'm not very familiar with the building tools (especially with the linking phase), I think the first solution will cause me more time. 6 | -------------------------------------------------------------------------------- /weekly/2016/45/timada.md: -------------------------------------------------------------------------------- 1 | What I have done in the recent weeks is as follows; 2 | - Finish reading (and testing codes in) Real World OCaml which will be related to my research topic 3 | - Investigated how MirageOS boots up on Xen by reading the MirageOS and mini-os source codes, and understood how OCaml based modules and the mini-os part in MirageOS interact 4 | - Started reading papers, documents, and source codes of Netmap and DPDK networking frameworks to consider how I can taking advantages of them in implementing my network acceleration feature on MirageOS 5 | -------------------------------------------------------------------------------- /weekly/2016/46/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | For this week, I have been focusing on the development of the data processing framework. I have done the following things: 2 | 3 | 1. I implemented a very simple DHT for the framework. 4 | 5 | 2. I finished the first version of peer-to-peer model parallel engine, on top of which I also implemented a SGD algorithm. 6 | 7 | Besides these, not much, the P2P model turned out to be much more complicated than the map reduce and parameter server models. Currently, I am focusing on the barrier control implementation in the framework. 8 | -------------------------------------------------------------------------------- /weekly/2016/46/seveneng.md: -------------------------------------------------------------------------------- 1 | - Finished porting to the Alpine system, now our PIH system is running on Alpine based cubeietruck full time. This involved putting the data persistence service and web interface for the end user into a debian-based domU. 2 | 3 | - Figured out and fixed various bug (well, at least temporarily), mainly about PIH-bridge. This part works like a Network Address Translation device, which is doing traffic relays between clients and data holding unikernels. It will keep using up its own port numbers on its network interfaces if there is no proper "garbage collection" for the port numbers. This bit became a problem when our collaborators started testing there demos against our PIH, where lots of connections would be involved. Indeed, to make it more robust, we have to address this problem. For the time being, each translation rule has its own lifetime(5 min for now), and a queue is maintained in memory to hold these rules, once the number of rules reaches a threshold(30_000 for now), a function will be invoked to clear out all expired rules. 4 | -------------------------------------------------------------------------------- /weekly/2016/46/timada.md: -------------------------------------------------------------------------------- 1 | - Made a list of functionality in existing networking software and hardware for virtualised environments. 2 | - both Netmap and DPDK have a similar approach (software-based packet switching, userspace packet processing) 3 | - using SR-IOV VFs is difficult to apply to some situations (for example, combined with DPDK on ARM) 4 | So I will conduct performance evaluation of the current implementation to understand which part is a bottleneck 5 | 6 | - Preparing for the performance evaluation for the coming MirageOS v3 release 7 | - I will be engaged in network related topics. 8 | - I have started building a MirageOS/Solo5 testbed (in Japan) to easily move to the actual evaluation in a local environment. I will learn and test Mirage-related operations on the testbed in advance 9 | -------------------------------------------------------------------------------- /weekly/2016/47/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | 1. I kept working on the P2P engine, and I implemented a distributed LDA atop of P2P engine in my data processing framework. I spent about two days in debugging it and finally managed to make it work. 2 | 3 | 2. I had a meeting with Mort to refine our thinking on the data framework and identified a couple of interesting research questions: barrier control, and model consistency. 4 | 5 | 3. I had a couple of meetings with those who are interested in owl and identified some potential directions: more advanced support for multi-dimensional array, and auto gradient derivation functions. I will focus on these two and try to get them ready by the end of this year. 6 | 7 | 4. I wrote a separate post for using owl to manipulate matrices, in case you didn’t notice before, here is the link: https://github.com/ryanrhymes/owl/wiki/Manipulate-Matrices-in-Owl 8 | -------------------------------------------------------------------------------- /weekly/2016/47/seveneng.md: -------------------------------------------------------------------------------- 1 | *Magnus is interesting in spending a day a week contributing to the MirageOS and DataBox efforts, specifically around the work he has already done on Jitsu, ARM and networking. He will spend Wednesdays in the Lab around the standup time and work with Qi on the new Alpine xen-arm-builder distribution, to get it up to speed. He is also working on mirage-vnetif and testing of the network stack.* 2 | 3 | - before Wednesday, run the system prepared for UCN final review to make sure it works, worked out the serial output from cubie so that without a vga screen we could still have a prompt to do the work, made SD card copies 4 | 5 | - from Wednesday on, in Portugal, for the UCN final review, discovered that both the original card and the copy were corrupted because of some last second invalid writes crossing partition boundaries on the card, panicked, put up something mimicking the behaviour of PIH to make sure our partners' demos could still run, since in the schedule, we wouldn't demo anything, it's just our collaborators using our platforms, then the review seemed going well... 6 | -------------------------------------------------------------------------------- /weekly/2016/47/timada.md: -------------------------------------------------------------------------------- 1 | - Completed building my MirageOS/Solo5 environments in Japan 2 | - they are operating perfectly. 3 | 4 | - Investigating and trying to execute existing network programs on MirageOS 5 | - found arp and iperf implementation, but found out they cannot use with the latest MirageOS branch (mirage-dev) due to rapidly changing MirageOS APIs 6 | - modifying their source codes: 7 | arp : modification finished and worked correctly 8 | iperf: under modification, network connection and data transfer were OK but a result printing part was wrong 9 | - I will move to learning of performance profiling on MirageOS after finishing the iperf modification 10 | -------------------------------------------------------------------------------- /weekly/2016/48/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Remove "zero" placeholders for other-phase values in the lambda code. This 2 | took me most of the week, as I had to phase information to `Path.t`, but also 3 | to `module_coercion`s. 4 | This modification should avoid over-approximating the static dependency tree 5 | as is currently the case. Currently, every run-time dependency is a static 6 | dependency. Since static dependencies must be compiled before there parent in 7 | the tree can be compiled, this limitation would break most OCaml projects. 8 | I expect this work to be finished, I'll do the tests later tonight. 9 | * Talked with Leo about linking and side effects. Maybe, at some point, it will 10 | be necessary to ban the `static` keyword, unless the effect system is 11 | integrated into OCaml soon enough. 12 | * In addition to switching the quoting library to producing lambda code, it 13 | would be good to detect scope extrusion. This would be done by passing 14 | `CamlinternalQuote.Var.t` to macros, instead of the current `Longident.t loc`. 15 | * Note for later: rename `CamlinternalQuote.Ident` to 16 | `CamlinternalQuote.Global` and move `lfrommacro` to `Exp`. 17 | -------------------------------------------------------------------------------- /weekly/2016/48/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | Sorry for the late email this time, I was travelling on Friday. For the last week, I had been only focusing on the N-dimensional array in Owl. 2 | 3 | 1. I implemented the first version of N-dimensional array, atop of which, I also implemented a N-dimensional view module which can pipeline some operations on the array to speed up the performance. 4 | 5 | 2. I later find a better way to further optimise the performance of ndarray by using blas/lapack library. I am currently rewriting some code for better performance. 6 | -------------------------------------------------------------------------------- /weekly/2016/48/timada.md: -------------------------------------------------------------------------------- 1 | - Preparing for the network performance evaluation 2 | - finished the iperf modification, now iperf can be conducted between independent MirageOS VMs 3 | - finished investigation of performance profiling schemes for MirageOS/Solo5 4 | - tested mirage-trace-viewer and it worked fine 5 | - investigated Xen and QEMU/KVM tracing facilities, and confirmed they can provide what I want to know 6 | (I will try them later) 7 | https://blog.xenproject.org/2012/09/27/tracing-with-xentrace-and-xenalyze/ 8 | http://www.linux-kvm.org/page/Perf_events 9 | http://vmsplice.net/~stefan/stefanha-tracing-summit-2014.pdf 10 | - Started implementing an automation framework of the network performance evaluation 11 | - Completed rough implementation design and checking other software required 12 | - I will implement it next week 13 | -------------------------------------------------------------------------------- /weekly/2016/49/oliviernicole.md: -------------------------------------------------------------------------------- 1 | **Compiler** 2 | 3 | * The compiler "without placeholders" (and thus without useless static 4 | dependencies) now broadly works, i.e. I was able to bootstrap (necessary 5 | because adding an argument to one constructor of `Path.t` changed the cmi 6 | format) and compile `re` (a regexp library that previously didn't compile 7 | because of dependency issues). 8 | * Fixed some tests, including `warnings/w53.ml` and other easy things. My 9 | changes appear to have broken recursive modules again (assertion failed in 10 | `CamlinternalMod`). 11 | * Fixed some tests involving recursives modules, but not all of them, by fixing 12 | the `init_shape` function. 13 | * Changed the lifting symbol to `~` (which unlike `^` should be non-breaking) in 14 | order to break less Opam packages. 15 | * Fixed various issues and bugs with the new system of coercion. It seems quite 16 | robust, all tests are passed (except `no-alias-deps/aliases.cmo` but that's 17 | for a completely different reason, and was expected), and a lot of code from 18 | Opam compiles fine. 19 | * Fixed priority of "illegal quoting" errors over phase errors after drup's 20 | remarks 21 | 22 | **Examples using macros** 23 | 24 | * I try to make a fork of camlp4 compile again, but I encountered a bug that 25 | wasn't caught by the tests. 26 | * A segfault occurs when running static code for the main source file of camlp4. 27 | Investigating on the issue. 28 | * Started to look at the Flick network DSL, and its OCaml implementation Motto, 29 | and whether it can benefit from compile-time metaprogramming. 30 | * Menhir is a dependency of motto, and doesn't compile because of wrong 31 | dependency tree — can be fixed by not translating type extensions into static 32 | code 33 | * Talked with Mort about networking examples. It would be nice to be able to 34 | optimize a packet stream processor like POF or P4 using staging (maybe trying 35 | to use or get inspiration from strymonas). 36 | -------------------------------------------------------------------------------- /weekly/2016/49/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | 1. I finished the first version of n-dimensional array in Owl, including some optimisation and the full documentation. 2 | 3 | 2. I also wrote a tutorial on Ndarray which you can find here: https://github.com/ryanrhymes/owl/wiki/Tutorial:-N-Dimensional-Array 4 | 5 | 3. I did some initial comparison to numpy and julia, it looks promising at the moment, and I will do more thorough evaluation next week. 6 | 7 | 4. I also talked to kc and michel and discussed about further optimisations on owl, I will look into the directions they suggested in the following weeks. 8 | -------------------------------------------------------------------------------- /weekly/2016/49/seveneng.md: -------------------------------------------------------------------------------- 1 | - talked with mort about the next project/direction to work on, for now, there are two possibilities: about PIH-store, develop a model-based compression engine inside of it, so that the data could take up much less storage space, yet with tolerant divergence from their real values; another one is about developing some temporal logic(together with buffer management) to extend the expressibility of packet capturing of network traffic 2 | 3 | - talked with Magnus about work within MirageOS tcp/ip stack testing, at the moment there are testing about different parts of a protocol (say for TCP, we have tests for options and window management), Magnus is working on more different vnetif backends (to simulate different network traffic failure modes), I think there may be something missing in between, like tests when different parts of a protocol working together as a whole under various scenarios, or a test for layers in the stack working together with different packets traces, there are some parts in test_rfc5961.ml and test_iperf.ml, but I think there should be a easier way to express these and a higher level interface to create packets and traces instead of manually created cstruct(s), I'd like to develop something like `packetdrill', probably together with a test environment and I've already started to look into this. 4 | -------------------------------------------------------------------------------- /weekly/2016/49/timada.md: -------------------------------------------------------------------------------- 1 | - Preparing for the network performance evaluation 2 | - Implementing an automation framework of the network performance evaluation 3 | - This will be finished in several days 4 | - Took long time to solve how to get logs of the MirageOS console (A logging scheme provided by Libvirt could not work, so I investigated and tried other logging frameworks) 5 | 6 | - Making a slide deck to introduce my work at OCaml Labs to Hitachi Data systems guys 7 | - 90% finished 8 | - I will be able to submit the slide to you for checking next week 9 | - I will also check what kind of checking process in Hitachi for the slide is needed. 10 | -------------------------------------------------------------------------------- /weekly/2016/50/let-def.md: -------------------------------------------------------------------------------- 1 | * Distributed witnesses, https://github.com/let-def/distwit (with @samoht ) 2 | - Solve the problem of marshalling exceptions / extensible type by externalizing the equality. 3 | * HyperLogLog, https://github.com/let-def/grenier/blob/master/hll/hll.mli 4 | - Cardinality estimation in constant space. I made a p-o-c implementation a while ago, now there is one user. 5 | - I added serializability, improved memory representation and fixed a bug causing small biases in estimations 6 | * Macho support for Owee, https://github.com/let-def/owee, WIP 7 | - Loading Macho would be useful for supporting macOS binaries in spacetime. 8 | - This would also a cheap lwt instrumentation tool for monitoring and backtraces. I discussed with @antron, maybe we will look at providing the necessary hooks in the future. 9 | * OCaml meeting (monday last week, some notes about stuff I want to look at) 10 | - Namespace: Didier Remy worried about specification of build (clumsy, not part of the langage) 11 | - Interested by bytecode/native interoperability (mentioned by @dim, @stedolan) 12 | - About release process: too much GPR, not enough testing & code review, quality not satisfying 13 | * Eliom presentation (yesterday) & talk with @drup 14 | - Seems feasible to add eliom support to Merlin. 15 | 16 | -------------------------------------------------------------------------------- /weekly/2016/50/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | I have been working on the following things. 2 | 3 | 1. I received a lot of feedback of Owl via github and emails from different people. Based on the feedback, I decided to restructure the Owl a bit by providing a more general matrix module using GADT. I already finished the initial version of Dense.Matrix module. In the near future, it will replace the current Dense.Real and Dense.Complex to provide matrix support for more types and precisions. 4 | 5 | 2. I met Huawei people with Jon and presented Owl and Actor (the data processing system) I developed. They showed some interests which is a good sign (maybe:) 6 | 7 | 3. For Actor, I have been collecting papers and doing literature survey to formulate the idea. 8 | 9 | For next week, I will fly back to Helsinki so I cannot attend the meeting any more this year. I will see you guys next year in January :) 10 | -------------------------------------------------------------------------------- /weekly/2016/50/timada.md: -------------------------------------------------------------------------------- 1 | - Preparing for the network performance evaluation -> Finished automation framework implementation for the network performance evaluation. 2 | - The automation script I implemented worked fine. -> Started building a MirageOS environment on another physical server @ packet.net. 3 | - MirageOS/Solo5: mirage-dev 4 | - OCaml version: 4.03.0, 4.04.0 5 | - Hypervisor: Xen, KVM(virtio) -> But now having a problem in executing MirageOS with OCaml v4.03.0 on Xen, so investigating how to solve it. 6 | - This seems a bug in MirageOS. -> Also started and finished the iperf network performance evaluation on Linux Virtual Machines. 7 | - on the physical server above. 8 | - Observed throughput: 38MB/s(128 bytes buffer length), 290MB/s(1024 bytes buffer length) 9 | -------------------------------------------------------------------------------- /weekly/2016/51/dra27.md: -------------------------------------------------------------------------------- 1 | **OCaml** 2 | Various mildly time-consuming bits 'n bobs: 3 | - Testsuite hardening to improve CI (GPR#974, GPR#975) 4 | - CI tweaks (GPR#978, GPR#979) 5 | - Windows asmrun build system merge test+review (GPR#941) 6 | - GPR#980 test+review 7 | -------------------------------------------------------------------------------- /weekly/2016/51/let-def.md: -------------------------------------------------------------------------------- 1 | I haven't progressed much on merlin this week, all my work was reason-related 2 | -------------------------------------------------------------------------------- /weekly/2016/51/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - Finished it on MirageOS and Linux VMs. 3 | - Observed network throughput on MirageOS VMs is worse than on Linux VMs. We will need to investigate why this happens. 4 | - 128-byte buffer size: 5 | - 3MB/s on MirageOS-Xen, 24MB/s on MirageOS-virtio 6 | - 46MB/s on Linux-Xen, 38MB/s on Linux-virtio 7 | - 1024-byte buffer size: 8 | - 16MB/s on MirageOS-Xen, 38MB/s on MirageOS-virtio 9 | - 230MB/s on Linux-Xen, 295MB/s on Linux-virtio 10 | - Also preparing for releasing my source codes on the Github. 11 | - this includes only source code sophistication for releasing, and will be finished in several days. 12 | -------------------------------------------------------------------------------- /weekly/2017/1/dra27.md: -------------------------------------------------------------------------------- 1 | **General** 2 | - Settling into the lab 3 | 4 | **OPAM-on-Windows** 5 | - MSR meeting. Immediate things to note: 6 | - Once rebase complete, liaise with @protz on package requirements to eliminate MSR internal OPAM scripts 7 | - Andreas Hauptmann (fdopen) has stopped (is stopping) supporting his Windows opam-repository 8 | 9 | **OCaml 4.05** 10 | - Reviewing "my" [GM]PRs and devising workplan ready for 1 Feb freeze 11 | -------------------------------------------------------------------------------- /weekly/2017/1/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage Storage** 2 | 3 | - Start refactoring the way cache items are addressed. 4 | This is a precondition for flush support. 5 | - Document many design and implementation decisions. 6 | 7 | -------------------------------------------------------------------------------- /weekly/2017/1/gemmag.md: -------------------------------------------------------------------------------- 1 | **General** 2 | - Catching up post-Christmas MANY EMAILS 3 | - Exploring using furore for weekly worklogs with a view to everyone adding their weekly logs directly 4 | - Paperwork and administration for visiting researchers to the lab 5 | - Working on our 2016 catch up report: https://ocamllabs.github.io/furore/index.html 6 | - Prepping Oct-Dec short report on activities 7 | - Helping settle David Allsopp in the lab for his first day 8 | - Writing up a blog about the MirageOS feedback 9 | - Working on MirageOS logos 10 | 11 | **Projects** 12 | - Talked with Fred re Merlin progress: 13 | - stateless frontend: working on wrapper for Windows 14 | - logging: working on currently 15 | - parsing: late Jan, needs to talk with Francois first 16 | - Talked with Mark Shinwell (JS) about Merlin use at JS and issues they are facing 17 | - Planning Romain's next project with OCL 18 | -------------------------------------------------------------------------------- /weekly/2017/1/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Ezirmin 2 | - Add `Sync` module that roughly mirrors `Irmin.Sync`. Discovered that `push` 3 | is broken. This is tracked in the following issues: 4 | - https://github.com/mirage/ocaml-git/issues/139 5 | - https://github.com/mirage/irmin/issues/379 6 | - Added support for history 7 | - Adding features to write a blog post 8 | - Write and submit DaLi paper to [SNAPL 2017](https://github.com/gowthamk/snapl) 9 | -------------------------------------------------------------------------------- /weekly/2017/1/let-def.md: -------------------------------------------------------------------------------- 1 | 2 | * OCaml compiler work (that affects Reason and OCaml): migrating the parse tree - with Alain Frisch and Jeremie Diminio - different solution. Useful for JS via ppx and FB via Reason. 3 | 4 | * Merlin: 5 | - Stateless frontend: Happy with it as it stands, has not optimised performance, but what is currently there works. Planning to spend some time at JS working on integration as it is quite a deep change - after POPL. 6 | 7 | Merlin TODO: 8 | - Stateless wrapper for Windows: started, more difficult than expected. The current design is not portable, so needs to work on that and then implement it. Needs Windows box - arriving next week. 9 | - Logging: Logging is made possible by the stateless frontend. Has a prototype but doesn't match - will progress quickly once started - likely early January. 10 | - Parsing: Meeting Francois to discuss features/direction 11 | -------------------------------------------------------------------------------- /weekly/2017/1/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Owl 2 | - Implement the module of sparse matrices. 3 | - Implement the module of sparse n-dimensional array. 4 | - Add the unit tests for Dense.Matrix, Dense.Ndarray, Sparse.Matrix, and Sparse.Ndarray modules using Alcotest. 5 | - Add the performance tests for the aforementioned four modules. 6 | -------------------------------------------------------------------------------- /weekly/2017/1/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - started preparing for performance bottleneck investigation. 3 | - finished building a QEMU/KVM hypervisor with a tracing feature by ftrace, and confirmed it can work. 4 | - building my MirageOS environment with the profiling support. 5 | - Tools for network performance measurement 6 | - Finished making them available on the GitHub Website. 7 | - (throughput) https://github.com/TImada/mirage_iperf 8 | - (latency) https://github.com/TImada/mirage_pingpong 9 | -------------------------------------------------------------------------------- /weekly/2017/10/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Decompress 2 | 3 | Decompress is released in version 0.5. So, it seems work (no issue for this 4 | moment). I have a bug about the opam file and hannes just fixed that. I will 5 | report this change in opam-repository. 6 | 7 | Finally, I decided to let 6 months to get the new release (1.0) of Decompress. 8 | 9 | #### `sirodepac`/`ocaml-git` 10 | 11 | I started the implementation of the BLAKE2B hash function. I send a message to 12 | david to know if it's the best to integrate this hash function inside nocrypto 13 | or outside (and if the best is to create a new library, may be the best is to 14 | locate all common hash function inside a new library). 15 | 16 | So, fortunately, Eyyub (a friend) created a projet inside the `oklm-wsh` 17 | repository to aggregate the common hash function (like SHA1, SHA256, etc.). So, 18 | I will reuse this project to provide the BLAKE2B hash function. 19 | 20 | For this moment, I look the reference implementation of BLAKE2B and the SSE 21 | implementation (to compute the hash fastly). So, I think, I can do a release 22 | this next week. 23 | 24 | Nothing else about `ocaml-git`. I met in the hackthon a manager of a dev team 25 | for a git project in go. So I explained what is git and why is the best to 26 | store a data, a commercial job :p . 27 | 28 | #### Farfadet 29 | 30 | So, I created the new repository `Farfadet`. I spoke with Spiros about the API 31 | and we fixed together the API. 32 | 33 | Another good news is from @Drup. In fact, I improved the API after my release 34 | and it seems good. So, I need to read the code and write the documentation now 35 | (and provide a good example - asked by KC) but I'm focus on the BLAKE2B hash 36 | now. 37 | 38 | #### Mr. MIME 39 | 40 | Nothing to do. 41 | -------------------------------------------------------------------------------- /weekly/2017/10/dra27.md: -------------------------------------------------------------------------------- 1 | ** OCaml 4.05 ** 2 | - Fixed the FlexDLL bootstrap allowing ocamlmklib to work correctly (GPR#1023). The 3 | fix is targeted for 4.04.1 and required a separate version for 4.05/trunk owing to 4 | the (very welcome) build system alterations. Lots of test cases... 5 | - Spent arguably too much time working on a test-case for the Unix.stat bug in GPR#1057. 6 | Interesting foray into kernel trickery on Windows (polite way of referring to disgusting 7 | code injection and re-writing tricks...) 8 | -------------------------------------------------------------------------------- /weekly/2017/10/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | * Expanding tests 4 | * Bug fixing 5 | * Refactoring 6 | * Collecting and logging statistics 7 | * NeedsFlush signal 8 | * LRU pull request 9 | 10 | -------------------------------------------------------------------------------- /weekly/2017/10/gemmag.md: -------------------------------------------------------------------------------- 1 | * Finished the Maintainerati blog post: http://reynard.io/2017/03/07/MaintaineratiWontFix.html 2 | * Planning for Databox launch 3 | * Paperwork 4 | * Proofreading ppx updates from Fred: https://github.com/let-def/ocaml-migrate-parsetree/blob/master/MANUAL.md 5 | * Moving more content over to ocamllabs.io 6 | * Following MirageOS hack retreat updates on Twitter: http://ocamllabs.io/events/2017/03/06/MirageHackUpdates.html 7 | -------------------------------------------------------------------------------- /weekly/2017/10/kayceesrk.md: -------------------------------------------------------------------------------- 1 | I've been lazy and not writing the report every week. Here are the updates from 2 | the last 2 weeks: 3 | 4 | - Submitted the following PRs to Irmin merge functions 5 | * https://github.com/mirage/irmin/pull/422 6 | * https://github.com/mirage/irmin/pull/420 7 | - Released mergeable-vector library: https://github.com/kayceesrk/mergeable-vector 8 | - Compiled Multicore OCaml programs to wasm which runs in Firefox! 9 | * https://github.com/kayceesrk/ocamlrun-wasm/tree/multicore 10 | * https://pbs.twimg.com/media/C6ai77CXQAYJAhI.jpg:large 11 | - Wrote a blog post on topkg + carcass: kcsrk.info/ocaml/opam/topkg/carcass/2017/03/05/building-and-publishing-an-OCaml-package/ 12 | - Submitted a paper on mergeable types 13 | - Fixed my homepage for moving from redcarpet to kramdown. 14 | -------------------------------------------------------------------------------- /weekly/2017/10/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - The GSL dependency has been removed from the core components in Owl. 2 | - Current modules (partly) rely on GSL are Maths, Stats, FFT. I am currently wrapping up Cephes to replace GSL-based modules. 3 | - I spend some time in refining the AD interface. The [code here](https://gist.github.com/ryanrhymes/582e1d1a5f3cd47a6b96fe5bed4914e8) show how to build a trivial neural network with AD (from scratch). Comments are welcome. 4 | - However, I noticed the memory consumption grows really fast in the previous naive neural network experiment. The allocated memory did not seem to be correctly released after usage. I still try to identify the (possible) memory leak issue. Let me know if anyone has any idea about this. 5 | -------------------------------------------------------------------------------- /weekly/2017/10/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-export-service 2 | - implemented a long-polling version endpoint, temporarily with the route `/lp/export` 3 | - added a test case against this endpoint 4 | 5 | - databox-bridge: 6 | - decided to try to implement at first with docker bridge, equivalent to use `docker network create ...` and `docker network connect ...` to connect pairs that need communications 7 | - implemented this [link](https://github.com/me-box/databox/pull/50), waiting for more polishing and testing before merged 8 | - catched up with mirage3, cause later there should be a stand-alone bridge component rather that a patch in databox's container manager 9 | -------------------------------------------------------------------------------- /weekly/2017/10/stedolan.md: -------------------------------------------------------------------------------- 1 | Many multicore yaks now bald. 2 | 3 | - Atomics module exists! (or part thereof) 4 | - Got the multicore test suite running properly, and integrated with Travis. Fixed a bunch of issues with the testsuite and the runtime. 5 | - Added GC stats in multicore (functions in `Gc` now work, and account properly for every word of the shared major heap). Found a bug in trunk OCaml while doing this, now fixed in trunk and 4.05. 6 | - Fixed heap verification in multicore (in particular, debug mode now checks at runtime that the GC stats aren't lies) 7 | - Got some pull requests to trunk OCaml merged (#1088, #1069, #973) 8 | -------------------------------------------------------------------------------- /weekly/2017/10/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Found that the low performance under Solo5-ukvm was caused by a fault in my source code, and confirmed that the UDP iperf performance was 40MB/sec. 3 | - However, the receiver side can achieve 70MB/sec. This indicates that a current bottlenec is in the sender side. 4 | - Further investigation in the virtualization layer showed that the virtualization layer is a main overhead, and GC handling as shown in the previous experiments is not a performance impact. 5 | - I will continue to design new networking scheme wich consideration of the current Solo5-ukvm by the end of this month. 6 | -------------------------------------------------------------------------------- /weekly/2017/11/dra27.md: -------------------------------------------------------------------------------- 1 | ** OPAM ** 2 | - Completed rebase and update of windows-build branch on to master. 3 | OPAM compiles, but the good stuff still needs cherry-picking/updating! 4 | 5 | ** OCaml ** 6 | - Finally set-up a proper test-bed for parallel testing Windows ports 7 | - Started Visual Studio 2017 testing for OCaml 8 | -------------------------------------------------------------------------------- /weekly/2017/11/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I had been dealing with high memory consumption issue of AD module in Owl last week. In the end, the issue was identified: calling `Bigarray.Genarray.change_layout` function will cause OCaml unable to free the memory allocated for the variable. 2 | - `change_layout` functions have been called a lot in order to pass variables to Lacaml. Because of the aforementioned issue, I had to remove all these calls. In the end, I rewrote many maths function locally in `c` so Owl does not depend on Lacaml any longer. 3 | -------------------------------------------------------------------------------- /weekly/2017/11/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-export-service 2 | - implemented websocket version of the endpoint, kept in a seperate branch for now: [ws](https://github.com/me-box/databox-export-service/tree/ws) 3 | - refactored bits of the test part and the export logic part, to ease the integration of ws endpoint 4 | 5 | - databox-bridge: 6 | - finished the first implementation, and merged 7 | - fixed some bugs, opened PRs to opium and upstream js libraries 8 | -------------------------------------------------------------------------------- /weekly/2017/11/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Started investigation of the current impelementation of Intel DPDK and Netmap to understand what can be a point to be considered. 3 | - Findings: 4 | - DPDK does packet handling in the user space whereas Netmap does it in the kernel layer 5 | - DPDK uses busy loop polling for packet arrival detection whereas Netmap uses a NAPI-like scheme (=mix of busy loop polling and interrupts) 6 | - Both DPDK and Netmap require a virtualized PCI device and virtualized interrupts for a virtual machine when we try to use the currently available features for virtual machines. However, Solo5-ukvm currently does not have the required functions. So they will ba a challenging point I must tackle. 7 | - learned the current behavior of vhost-net in handling network packets, found that eventfd, irqfd and ioeventfd are used to realize in-kernel network packet processing in vhost-net. 8 | - Started creating a pros/cons table of DPDK and Netmap to check i) how many gaps they have and ii) how large the gaps are when I try to integrate them into MirageOS. 9 | -------------------------------------------------------------------------------- /weekly/2017/12/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Decompress 2 | 3 | Nothing to do. @engil found a bug in Canopy but I think, it's not about 4 | Decompress. Indeed, I use Decompress in `sirodepac` and all is ok. But I will 5 | inspect what is wrong - because, may be it's `ocaml-git`. 6 | 7 | #### BLAKE2B / Digestif 8 | 9 | I continue to polish this package. I try to provide an interface with the 10 | `Bytes.t` modules and break the *hard* dependency with `cstruct`. But, in fact, 11 | the end-user can use `cstruct` if he wants (`cstruct` is a `bigstring` and by 12 | default, the API provide a `bigstring` compute). Another point is to separate 13 | the pure implementation of the hash function in OCaml (in Digestif library) and 14 | a C implementation with an OCaml interface (in Rakia library). So, Digestif can 15 | be used in Mirage, JavaScript and OCaml - but I need to work a long time in this 16 | project. 17 | 18 | I just finish to create a generic test suite between `bigstring` and `bytes` and 19 | I need to fix the problem with the `bytes` in the Rakia library. Then, I will 20 | re-implement the hash function in pure OCaml. 21 | 22 | #### sirodepac / ocaml-git 23 | 24 | I finished the serialization of the IDX file. I talked with @samoth about that 25 | and all works fine. To test, I deserialize an IDX file, serialize the IDX file, 26 | and deserialize the output and compare the result. I use a radix tree to store 27 | the IDX file and I have a lazy implemenation (like `ocaml-git`). 28 | 29 | I try to implement the patience diff (without `core_kernel` package) to try to 30 | serialize the PACK file. Indeed, inside the PACK file, all git object was 31 | compressed by Decompress and by a diff function. So, I already have a PoC in 32 | `sirodepac`. I don't know if I need to create a new package for this thing but 33 | the good thing is that we have no dependencies. When I check my implementation, 34 | I will implementation the core of the serialization of the PACK file. 35 | 36 | #### Farfadet 37 | 38 | Nothing to do. 39 | 40 | #### Mr. MIME 41 | 42 | Nothing to do. 43 | -------------------------------------------------------------------------------- /weekly/2017/12/dra27.md: -------------------------------------------------------------------------------- 1 | ** OCaml ** 2 | - Windows testing of 4.04.1/4.05.0 revealed a few issues with the CI (GPR#1115, GPR#1116) 3 | - AppVeyor testing is sooo slow - GPR#1116 in particular took ages to prepare waiting for CI results! 4 | - Supporting build system changes across 4.04 and 4.05 is also proving slow work (GPR#1101 / GPR#1023) but at least that's only for this release cycle! 5 | -------------------------------------------------------------------------------- /weekly/2017/12/gemmag.md: -------------------------------------------------------------------------------- 1 | * Databox launch on the Friday 2 | * Paperwork 3 | * Preparing contracts 4 | * Monthly and quarterly updates 5 | * Moving more content over to ocamllabs.io 6 | * Collating MirageOS hack retreat trip reports 7 | * Starting proposals for interns this summer 8 | -------------------------------------------------------------------------------- /weekly/2017/12/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Developed a CPS translation for L := λ calculus + effect handlers. 2 | - Developed a CEK machine for the L. 3 | - Working on a type inference for L. 4 | -------------------------------------------------------------------------------- /weekly/2017/12/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - The memory issue of Owl was completely addressed. Now the MNIST example only consumes about 400MB memory and GC works fine. AD module can now be used in practice. 2 | - The `change_layout` function was completely removed from Owl since Owl does not depend on Lacaml any longer. All the vectorised math functions were reimplemented in c last week. 3 | - The module structure was also significantly changed, now different number types are wrapped into corresponding modules (S, D, C, Z, Generic). This makes the API even simpler in programming. 4 | - I published a new release Owl 0.2.2 this Friday. The tutorials are also updated to be consistent with the new changes in Owl, also added AD stuff in readme. 5 | -------------------------------------------------------------------------------- /weekly/2017/12/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-export-service 2 | - bug fixes and working arounds to make the demo work for the launch, mainly tied to macaroons-related stuff 3 | - disabled the tests on dockerhub autobuild, refer to [PR](https://github.com/me-box/databox-export-service/pull/14) 4 | - NB: impression was that the image from autobuild could be corrupted? *Output like: Illegal Instruction (core dumped), through gdb found that the binaries tried to disable address space randomization?* 5 | 6 | - databox-bridge: 7 | - such unstable that could make it for the launch event, will keep persuing this end later 8 | - should figure out the right timing and order of network creating and connection 9 | - should provide better failure control within inner libs 10 | -------------------------------------------------------------------------------- /weekly/2017/12/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Conducted some experiments to check how long time the MirageOS network stack spends for packet processing. Found that the IP packet frame allocation occupies 30-35% of the total processing time in MirageOS OCaml part, and that the VMExit/Entry cost cannot be negligible. 3 | - I will try to reduce the former bottleneck by employing an additional upper layer on Netmap or DPDK, and the latter bottleneck by reduction of context switching. 4 | - Still filling out a Pros/Cons table on network acceleration schemes. 5 | -------------------------------------------------------------------------------- /weekly/2017/13/dra27.md: -------------------------------------------------------------------------------- 1 | ## OCaml 2 | - Lost quite a bit of time chasing down a non-existent platform bug :o( 3 | Issue was in fact a different OCaml being (incorrectly) picked up by the testsuite 4 | - More test cases for 4.04.1/4.05.0 5 | -------------------------------------------------------------------------------- /weekly/2017/13/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | - Insert: Fix a spill/split interaction 4 | - New LRU peek API 5 | 6 | -------------------------------------------------------------------------------- /weekly/2017/13/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I made an experimental module called [Owl_ext](https://github.com/ryanrhymes/owl/tree/master/lib/ext) which allows interoperation of different number types. This should make analytical app easier to write and the code should be more concise. I finished the first version and here is an [example](https://gist.github.com/ryanrhymes/f9cce1afcd06a5f4683aae45be01bdbe). **Please do let me know if you have any comments on the design.** 2 | - I made several c functions in Owl to cast bigarray between different number types. These functions serve Owl_ext module for converting number types whenever necessary during interoperation. 3 | - I start building neural network module atop of current Owl's functionality and AD module. I think Owl's future development should be motivated by the realistic applications. I will start looking into various application scenarios and neural network seems a good starting point. 4 | - I start working on the actor system and barrier control again, pushing forward the experiment to get more results out. 5 | -------------------------------------------------------------------------------- /weekly/2017/13/seveneng.md: -------------------------------------------------------------------------------- 1 | - made some PRs to upstream macaroons libraries 2 | - merged in the websocket endpoint to the service 3 | -------------------------------------------------------------------------------- /weekly/2017/13/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - No advance this week (due to a private trip) 3 | -------------------------------------------------------------------------------- /weekly/2017/14/dra27.md: -------------------------------------------------------------------------------- 1 | ## OCaml 2 | - Wrote and used test harness for FlexDLL testing in 4.04.1/4.05.1 3 | - Sorted out Cygwin issues with CRLF following on from February update of grep/awk/sed (GPR#1140) 4 | -------------------------------------------------------------------------------- /weekly/2017/14/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I have been working on the neural network module in Owl. Currently, I am focusing on various gradient descendent and learning rate algorithms. AD turns out to be really powerful in simplifying the design of neural network module. 2 | - For barrier control in actor system, I am still working on the experiments. The results reported by Ben seem really promising. 3 | -------------------------------------------------------------------------------- /weekly/2017/14/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Finished completing the Pros/Cons table, then I decided to employ Netmap rather than DPDK as a network acceleration scheme as DPDK usually requires a busy loop-based packet handling. 3 | - Desingning basic implementation to integrate Netmap into Solo5-ukvm. I will introduce a vhost-net like scheme to reduce the number of VMExit/Entry. 4 | -------------------------------------------------------------------------------- /weekly/2017/15/dra27.md: -------------------------------------------------------------------------------- 1 | ## Misc 2 | - While working on something unrelated, found a possible exponential blow-up type-checking GADTs, but need 3 | to investigate further (using an older OCaml, so it might have been fixed) 4 | 5 | ## OCaml 6 | - Misc reviews and CI tweaks 7 | -------------------------------------------------------------------------------- /weekly/2017/15/gemmag.md: -------------------------------------------------------------------------------- 1 | * Infrastructure quotes and specs 2 | * Attended the PowerSwitch Symposium - notes/blog post to follow 3 | * Planning Q3 activities and events 4 | -------------------------------------------------------------------------------- /weekly/2017/15/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Submitted a paper with Oleg Kiselyov on embedding eff programming language in 2 | OCaml to ML workshop post-proceedings. The draft of the paper is here: 3 | http://kcsrk.info/papers/caml-eff17.pdf. 4 | * Working on CPS translation of effect handlers with Sam Lindley, Daniel 5 | Hillestrom and Bob Atkey. 6 | * Working on a Dagstuhl proposal with Matija Pretnar and Tom Schrijvers. 7 | * Developing the multicore aarch64 backend. 8 | -------------------------------------------------------------------------------- /weekly/2017/15/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I updated the slicing function in Owl. It is more flexible than the previous version but can still be improved. 2 | - I released Owl.0.2.3. The current focus of the development is on neural network and nlp modules. 3 | - I did some experiments for actor system, this will continue and will be my focus in the following weeks. 4 | -------------------------------------------------------------------------------- /weekly/2017/15/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Learned how to write a user program with Netmap, and investigated vhost-net implementation detail. 3 | - Finished designing integration of Netmap into Solo5-ukvm. Decided to employ Netmap in the host user layer to reduce the integration cost. I found that using Netmap in the host kernel layer costs a lot for lack of Netmap APIs. 4 | -------------------------------------------------------------------------------- /weekly/2017/16/dra27.md: -------------------------------------------------------------------------------- 1 | ## opam 2 | - Working through rebase of features in windows branch onto beta2 3 | - Upstreaming PR#2912 to fix a rollback problem with failed package installations 4 | 5 | ## OCaml 6 | - Git precommit hook and also GPR validation against our syntax rules (WIP - GPR#1148) 7 | - Various investigations for future work: 8 | * Tweaks to bring coloured output trivially to Windows 10 OCaml users 9 | * Windows 10 1703 Creators Update requires some tweaks for the Unix module - for 4.06 10 | -------------------------------------------------------------------------------- /weekly/2017/16/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I have been working on the nlp module in Owl. The nlp module aims to provide stronger supports to deal with large text corpus. With Owl's numerical functions, I hope people can build up topic model easily. 2 | - I have been studying latest ocaml-tensorflow binding, and other libraries for fast prototyping such as keras.io, to see what Owl can learn from them. 3 | -------------------------------------------------------------------------------- /weekly/2017/16/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-export-service 2 | - change the inner service model from multiple clients one queue to one client one queue 3 | - produce basic latency and queueing effect analysis plots 4 | 5 | - databox-irmin-store 6 | - implement basic kv/ts stores read/write functionalites 7 | -------------------------------------------------------------------------------- /weekly/2017/16/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - No updates in this week. (attended DockerCon17 @ Austin) 3 | -------------------------------------------------------------------------------- /weekly/2017/17/dra27.md: -------------------------------------------------------------------------------- 1 | ## opam 2 | - PR#2912 merged 3 | - Discussions with Louis on file format features required for Windows 4 | * Improved mechanism for detecting undefined variables. 5 | Presently, you can use "%{foo}%" which expands to "" if foo is undefined. 6 | New proposal adds operator ? so that ?foo returns false if foo is not defined. 7 | * Allow variables in ternary operator in string expansion. 8 | Presently, you can use "%{foo?bar:baz}%". 9 | New proposal requires strings to be single quoted and interpret the rest as variables. 10 | So far, so hacky, but unfortunately you can't use scoped variables 11 | e.g. what should "%{foo?bar:path:lib}%" mean? 12 | Shelved, as there are other workarounds for this. 13 | * Mechanism for initialising switch global variables by extending /etc/opamrc 14 | - Upstreaming PR#2915 to fix issues with debugging failed updates 15 | - Upstreaming PR#2923 to fix issues with renaming the test and doc variables in opam-repository 16 | - Upstreaming PR#2921 to implement the defined operator ? described above 17 | -------------------------------------------------------------------------------- /weekly/2017/17/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Merged ARM64 backend for multicore OCaml: https://github.com/ocamllabs/ocaml-multicore/pull/120 2 | * Working on generating tests that would exhibit weak memory behaviours on arm64. 3 | -------------------------------------------------------------------------------- /weekly/2017/17/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I have been working on the redesign of math operators in Owl. I have looked into multiple implementations in different languages such as numpy, julia, and R. 2 | - I have been improving the NLP module, include Corpus, TFIDF, and Vocabulary modules. 3 | -------------------------------------------------------------------------------- /weekly/2017/17/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-irmin-store 2 | - write tests for kv/ts stores about r/w and websocket sub/unsub functionlities 3 | - dockerize this component and shrink the image size 4 | 5 | - databox-bridge 6 | - browse the repo ocaml-openflow investigating the possibilities of integration 7 | -------------------------------------------------------------------------------- /weekly/2017/17/stedolan.md: -------------------------------------------------------------------------------- 1 | Lots of figuring out the interactions between asynchronous interrupts 2 | (e.g. signals, cancellation) and the rest of the system. Stealing a 3 | lot from Haskell's well-designed async exceptions. 4 | -------------------------------------------------------------------------------- /weekly/2017/17/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Started implementing a new networking scheme using Netmap on Solo5-ukvm. 3 | - The first phase would be replacing a network tap device by a Netmap port, and the second phase would be integration of ioeventfd to reduce the number of VMExits. 4 | - Had discussion with Martin(@Docker) about the scheme above including how I can conduct the implementation. 5 | -------------------------------------------------------------------------------- /weekly/2017/18/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Decompress 2 | 3 | Prepare next 0.6 release. 4 | 5 | #### BLAKE2B / Digestif 6 | 7 | Prepare release 0.1. 8 | 9 | #### Farfadet 10 | 11 | Nothing to do. 12 | 13 | #### TypeBeat 14 | 15 | Nothing to do. 16 | 17 | #### sirodepac / ocaml-git 18 | 19 | It's DONE! I just finished to implement the git heuristic in `sirodepac` and the 20 | result is: 21 | - we produce a PACK file with 1.3MB 22 | - git produces a PACK file with 1.2MB 23 | 24 | So, yes ... now all is done. `sirodepac` is finished and, we can start the 25 | integration in `ocaml-git`. 26 | 27 | So I need to explain by step what I did: 28 | 29 | * Firstly, I functorize the *packer* by a zlib implementation module. Now, we 30 | can use `decompress` or `camlzip`. I did that because `decompress` is not the 31 | best to inflate (the diff is about some bytes) and to follow exactly what git 32 | does, I decide to functorize the implementation. 33 | 34 | The diff between a *packer* with `decompess` and `camlzip` is about 0.1MB. 35 | So, it's ok. 36 | 37 | * I reproduce exactly the same sort as git. This is the core of the PACK 38 | algorithm to find the best diff between 2 git objects. 39 | 40 | * I finish to implement the Rabin's fingerprint and the diff with that. I 41 | optimized the compute to avoid any allocation of `Cstruct.t`. The result is 42 | exactly the same as git. 43 | 44 | * I switch the window implementation of the *packer* to the `lru` project from 45 | David. So, I add a dependency but it's ok. It's to follow, again, what git 46 | does and when git find a good delta, it promotes this delta in the window. 47 | 48 | * I seperate the serialization from the compute of the delta-ification. A good 49 | point is to let a new optimization and thread the compute of the 50 | delta-ification. This is what git does but need lot of work. So, for the 51 | moment, the algorithm is sequential but we can improve independantly than the 52 | serializer. 53 | 54 | * Implement topological sort to ensure we don't miss any diff for all git 55 | object. 56 | 57 | * Handle a diff with an object outside the PACK file. 58 | 59 | * Clean all 60 | -------------------------------------------------------------------------------- /weekly/2017/18/dra27.md: -------------------------------------------------------------------------------- 1 | ## opam 2 | - Teleconference discussing directions for opam 2.0 3 | * Roadmap of beta releases to follow (proposed release candidate to be beta3) 4 | * Windows hopefully included in beta4 5 | * Improvements to the upgrade story required (aim to allow safe side-by-side installation of 1 and 2) 6 | - Upstreaming PR#2927 to add "%<...>%" syntax for translating paths from Unix 7 | forward-slash style to Windows back-slash style. Lots of discussions ongoing for 8 | this... 9 | - Upstreamed PR#2928 to ease using Merlin when developing opam (merged) 10 | - PR#2915 merged 11 | - PR#2921 merged 12 | - PR#2923 abandoned (won't be necessary after beta3 is released) 13 | 14 | ## OCaml 15 | - Fixing problems with AppVeyor caused by GPR#1127 16 | -------------------------------------------------------------------------------- /weekly/2017/18/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Submitted a paper on concurrent programming with effect handlers to TFP. Draft 2 | is [here](kcsrk.info/papers/system_effects_may_17.pdf) 3 | - As a part of the submission, benchmarked http/af with async and effects and 4 | compared it against a Go webserver. The results are included in the paper. 5 | - We discovered that effects version was leaking memory, which was down to the 6 | program leaking bigstrings due to finalizers not being implemented on 7 | Multicore. Finalizers for custom objects should be easy to fix. 8 | -------------------------------------------------------------------------------- /weekly/2017/18/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - The operators are re-designed last week with some newly added ones. The Ext module is also finished. I then released the version 0.2.4 on OPAM. 2 | - The work on neural network module continues, at the same time I add more functions into Owl's core. 3 | - I wrote two tutorials: one is on [operators and ext module](https://github.com/ryanrhymes/owl/wiki/Tutorial:-Operators-and-Ext-Module); one is on [algorithmic differentiation](https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic-Differentiation). 4 | -------------------------------------------------------------------------------- /weekly/2017/18/seveneng.md: -------------------------------------------------------------------------------- 1 | #### databox-bridge 2 | 3 | Thes first step is to intercept l2/l3 packets from multiple interfaces and forward them. 4 | 5 | All the existed MirageOS network io solutions may not seem to fit: 6 | 7 | - this compoment will be used inside a docker container, so `xen` is not good 8 | - `unix` and `ukvm`, `virtio` targets all use tap devices to do the network io, so for each container interface, there will be bridging and NATing, which will filter out the arp traffic and the ip traffic not targeting the same network 9 | 10 | So for now, looking into the possibility of a new network device for unikernels, which is on unix, and instead of openning tun/tap devices, open raw sockets and output the l2 packets 11 | 12 | Maybe could also modify the [mirage-net-unix](https://github.com/mirage/mirage-net-unix/blob/master/src/netif.ml#L58), to use `opentun`*(mixed opentun and opentap here, so this approach doesn't work)* inside the `connect` function call *(tried earlier, but resulted in failures, could look further in the [c code](https://github.com/mirage/ocaml-tuntap/blob/master/lib/tuntap_stubs.c#L73))* 13 | -------------------------------------------------------------------------------- /weekly/2017/18/stedolan.md: -------------------------------------------------------------------------------- 1 | - Worked out how to do asynchronous syscalls by dynamically 2 | adjusting concurrency level with SCHED_IDLE threads. Should get us 3 | something like Haskell's safe foreign calls, but with much lower 4 | overhead. 5 | 6 | - Helped out with a pretty panicked TFP submission on the design of 7 | our effectful I/O (blocking syscalls, asynchronous effects, and 8 | the likes) 9 | -------------------------------------------------------------------------------- /weekly/2017/18/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Confirmed that device initialization including queue allocation and finalization parts operates correctly. 3 | - Finished coding sender and receiver functions. I will check if they can operate correctly the next week. 4 | -------------------------------------------------------------------------------- /weekly/2017/19/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Digestif 2 | 3 | I did the release but I spoke with daniel about the interface and the 4 | organization of the library. So I just change the API. I reimplemented the SHA1 5 | and the SHA256 in OCaml. I will push all the next monday. 6 | 7 | We moved this repository in mirage organization. 8 | 9 | #### Decompress 10 | 11 | I did the release of 0.6. This release fixed a bug about the decompression but 12 | nothing change about the API. 13 | 14 | We moved this repository in mirage organization. 15 | 16 | #### TypeBeat 17 | 18 | Nothing to do. 19 | 20 | #### Farfadet 21 | 22 | Nothing to do. 23 | 24 | #### ocaml-git / sirodepac 25 | 26 | With Thomas, we push some PRs like the integration of decompress by default and 27 | the integration of digestif by default - but we keep all functors. Then, I fixed 28 | some bugs in sirodepac about the memory and integrate the optionnal usability of 29 | a cache (like a LRU cache) and an access of a hash by the offset. 30 | 31 | Thomas sended me a huge PACK file (~ 4 000 000 commits, 10 Go), so I will try to 32 | generate a new PACK file from this source. However, the compute is very low and 33 | may be crash because I launched the process in my server (and it's not my best 34 | machine). 35 | 36 | I wrote a documentation about `sirodepac` but only in french. When I finish, I 37 | will ask to fix some errors (in french), then I will try to translate to 38 | english - and publish an article :) ! 39 | 40 | Finally, with Thomas, we think about an abstraction and apply the generation of 41 | the PACK for something different than `git` object and I think it's possible, I 42 | have an idea about that. But I'm focus to integrate my work in ocaml-git for the 43 | moment. 44 | -------------------------------------------------------------------------------- /weekly/2017/19/dra27.md: -------------------------------------------------------------------------------- 1 | ## Platform 2 | - Discussion with Gemma & Anil on roadmap from here to ICFP (and a bit beyond...) 3 | 4 | ## OCaml 5 | - Started discussions on changes being made in GPR#1127 and GPR#1168 given 6 | nuisance it creates for Windows installations. 7 | 8 | ## opam 9 | - Upstreaming PR#2930 implementing the switch-defaults section for opamrc 10 | - Work on a --set option for opam init and opam switch allowing switch global 11 | variables to be specified at switch creation time (this builds on PR#2930) 12 | - Testing and updating lib-pkg build mechanism ready to transfer to Azure (I keep 13 | running out of disk space on my ageing opam development VM...) 14 | - Various Slack discussions about OPAM leading to docs change in PR#2941 15 | - Upstreaming PR#2935 & PR#2936 containing various minor tweaks and fixes from windows-build 16 | - Upstreaming PR#2937 containing developer build options for opam 17 | - Upstreaming PR#2938 panellising lib-ext compilation 18 | - Upstreaming PR#2939 fixing the configuration and build system for native Windows 19 | - Upstreaming PR#2940 fixing the build process itself for native Windows 20 | -------------------------------------------------------------------------------- /weekly/2017/19/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Working on fixing call-frame information (CFI) in native code. This is 2 | important not only for debuggers like gdb, but also for any code that needs to 3 | unwind the stack (such as profilers). Finished PR#127 4 | https://github.com/ocamllabs/ocaml-multicore/pull/127 which fixes the 5 | backtrace in the current branch. 6 | - TFP'17 submission on current system programming with effect handlers is 7 | accepted. 8 | - Working on extending the CFI information to support backtraces across 9 | handlers. 10 | -------------------------------------------------------------------------------- /weekly/2017/19/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I spent the whole week in studying convolution neural network stuff. I have looked into code in Tensorflow and implemented a convolution function in OCaml. I will do some comparison next week and focus on the implementation of convolution layer. 2 | -------------------------------------------------------------------------------- /weekly/2017/19/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-bridge 2 | - trying out a unix-based network device for mirage unikernel 3 | - following the idioms from [tuntap_stabs.c](https://github.com/mirage/ocaml-tuntap/blob/master/lib/tuntap_stubs.c), instead of `open("/dev/net/tun")`, get a packet socket by `socket(PF_PACKET, SOCK_RAW, ...)`, then add housekeeping and io functions around this socket 4 | -------------------------------------------------------------------------------- /weekly/2017/19/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Confirmed the first implementation operated correctly for the 1460 bytes MTU size. 3 | - But found that it could not operate when I tried larger MTU size such as 9000 bytes due to strange behavior in the Netmap layer(data frame was split into some blocks with the maximum size 2048 bytes). I will not investigate more on this issue. 4 | - I will move to performance evaluation in terms of 1) different MTU size smaller than 1460 bytes, and 2) scalability. 5 | -------------------------------------------------------------------------------- /weekly/2017/2/dra27.md: -------------------------------------------------------------------------------- 1 | (reduced week) 2 | 3 | **OCaml 4.05** 4 | - Change Log Processor 5 | -------------------------------------------------------------------------------- /weekly/2017/2/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage Storage** 2 | 3 | - build a free space map from a filesystem scan 4 | - flush support 5 | - upgrade to mirage 3 interfaces 6 | -------------------------------------------------------------------------------- /weekly/2017/2/gemmag.md: -------------------------------------------------------------------------------- 1 | * Completed first draft of MirageOS Feedback blog 2 | * Ongoing OCL Oct-Dec report 3 | * Admin/management for new visitors 4 | * Multicore planning with KC and Stephen - specifically repo organisation 5 | * Working on MirageOS 3.0 posts/articles and press details 6 | * Submit Slack for Education application 7 | * Plan POPL liveblog 8 | * Start work on OCL site with updated information 9 | * Start planning for Maintainerati 10 | -------------------------------------------------------------------------------- /weekly/2017/2/ilsordo.md: -------------------------------------------------------------------------------- 1 | - Set up a blog hosted on github pages. I plan to do a few posts about my attempts 2 | at typing and/or verifying the CPS translation 3 | - Made some progress on the Agda formalization of a lambda calculus with simple effects 4 | - The syntax and types are done 5 | - The small step semantics using substitution is almost done 6 | - The abstract machine needs some work 7 | -------------------------------------------------------------------------------- /weekly/2017/2/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Investigated the issue of slow compile times on 4.04.0. Reported the bug to mantis: https://caml.inria.fr/mantis/view.php?id=7456 2 | - Planning for DaLi research paper. tl;dr is to add language level support for 3 | mergeable datatypes. This builds on several pieces of work that has already 4 | been done: 5 | - [ezirmin](https://github.com/kayceesrk/ezirmin) 6 | - [Cuekeeper](roscidus.com/blog/blog/2015/04/28/cuekeeper-gitting-things-done-in-the-browser/) 7 | - [diff-datatypes](https://github.com/gprano/diff-datatypes) 8 | - Updated CV and website. 9 | - Updated the timeline :-) http://kcsrk.info/timeline.html 10 | - Submitted OCaml bug report for failing thread initialization: 11 | https://caml.inria.fr/mantis/view.php?id=7452. 12 | - xleroy confirmed the bug and has proposed the fix: 13 | https://github.com/ocaml/ocaml/pull/1009 14 | - Multicore planning. Have come up with a laundry list of features. 15 | -------------------------------------------------------------------------------- /weekly/2017/2/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Split out Owl's interface to Eigen3 as a standalone library: [Github Repo](https://github.com/ryanrhymes/eigen) 2 | - Manage to learn how to interface between OCaml and C++ template library using Ctypes when interfacing to Eigen3. 3 | - Submit Eigen OCaml library to OPAM on Friday and wait for approval now. 4 | -------------------------------------------------------------------------------- /weekly/2017/2/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - finished preparing for performance bottleneck investigation. 3 | - finished learning and testing how to conduct tracing on Xen as well as on QEMU/KVM. 4 | - finished building my MirageOS environment with the profiling support. 5 | - TCP-based iperf tracing 6 | - Found the vCPU utilization on the receiver side reached 100% though the sender side did not (~30%) 7 | - also found it difficult to capture their behavior due to complicated ACK/SYN mechanisms in the TCP stack, so decided to use UDP. 8 | - I will resume this after finishing the UDP-based iperf investigation 9 | - UDP-based iperf tracing 10 | - Finished coding UDP-based iperf based on TCP-based iperf 11 | - Tracing in the MirageOS layer and the Xen layer by using the mirage-trace-viewer, xentop and xentrace commandes 12 | -------------------------------------------------------------------------------- /weekly/2017/20/dra27.md: -------------------------------------------------------------------------------- 1 | ## OCaml 2 | - Various exchanges regarding the GPR#1127/GPR#1168/GPR#1172 build system changes 3 | - 4.05 testing leading to a few tweaks (GPR#1177) 4 | - Fixes pushed to trunk and 4.05 fixing Inria CI (MacOS, mingw32/64, ppc32/64) 5 | 6 | ## opam 7 | - Oozing towards being able to run the make win-zips test on Azure. 8 | * Various build system tweaks required, especially dealing with Warning 58 9 | (-opaque not used) in third party libs 10 | * msvs-tools needs updating for VS2017 and integrating - turns out that's 11 | not just a simple meta-data change (there's a new tool called vswhere 12 | which needs to be integrated with msvs-detect) 13 | - PR#2935 merged; various work rebasing and keeping up with the others 14 | - Complete sweep of open opam issues (now subscribed to all changes) 15 | -------------------------------------------------------------------------------- /weekly/2017/20/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Lots of fixes to DWARF unwinding information: 2 | https://github.com/ocamllabs/ocaml-multicore/commit/65b329deb4296ea8fcfba788b9fcad921dbe1a9a. 3 | -------------------------------------------------------------------------------- /weekly/2017/20/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - More types of layers have been added in neural module: convolution 2D, convolution 3D, maxpool, fully_connected, lambda layer, and etc. 2 | - A large part of the code in neural module was also refactored. Now, Owl is able to validate every layer in a network having consistent input and output shape when constructing a Feedforward network. 3 | - Algodiff module is able to support (partially though) N-dimensional array now. 4 | -------------------------------------------------------------------------------- /weekly/2017/20/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-bridge 2 | - developped a mirage network device that could be attached to the host interface directly 3 | - used packet socket in c layer, so not a compatible solution for all targets 4 | -------------------------------------------------------------------------------- /weekly/2017/20/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Finished writing some script files to conduct performance evaluation 3 | - Started the performance evaluation as follows: 4 | 1. a pair of receiver and sender unikernels on a host physical server 5 | 2. multiple pairs of the unikernels on a host physical server 6 | 3. multiple pairs of the unikernels on two different physical servers(multiple sender unikernels on a physical server, and multiple receiver unikernels on the other physical server) 7 | -------------------------------------------------------------------------------- /weekly/2017/21/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### ocaml-git / sirodepac 2 | 3 | So I most done my last task about `sirodepac` and control the memory 4 | consumption. But I found a bug and it's about the serialization of a hunk. I 5 | created a mini hunks decoder to help to find the bug. And now, I know precisely 6 | where is it. So I will fix this and continue to test some others PACK files with 7 | an implementation of `zlib` and `decompress`. 8 | 9 | I hope, I finish this week this bug, it's a very deep bug but it's ok, in same 10 | time, I put some useful comments to help me to understand my code and check my 11 | implementation. 12 | 13 | #### Conferences 14 | 15 | So, I created a new talk for the Functional Conference in India and my talk was 16 | accepted! Then, Mark Li asked me to do a CUFP tutorial about Git in OCaml. I 17 | think about this and OCaml labs was agreed to do this. So, I prepare in my mind 18 | what I will do and ship an abstract before the deadline. 19 | -------------------------------------------------------------------------------- /weekly/2017/21/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Adding support for [afl-fuzz for multicore](https://github.com/ocamllabs/ocaml-multicore/tree/afl). The aim is to fuzz the thread schedules. Progress is slow. 2 | * Admin stuff: trip planning, visa applications, paper reviews. 3 | -------------------------------------------------------------------------------- /weekly/2017/21/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I publish a new release for both Eigen.0.0.4 and Owl.0.2.5 on OPAM. 2 | - I am still focusing on the neural network module in Owl. I finished `Dropout`, `Reshape`, and `Flatten` layers this week. With these new layers, I implemented a convolutional version of the previous MNIST example which used simple Feedforward network. 3 | - I added a new `concatenate` function to Ndarray and Matrix module, this function is able to concatenate a list of ndarrays/matrices based on the specified axis. 4 | -------------------------------------------------------------------------------- /weekly/2017/21/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-bridge 2 | - investigated the DNS service within docker environment 3 | - each container has a local dns server sitting at 127.0.0.11 on not standard *(53)* port 4 | - the default setting for container could be changed to assign a cutomized endpoint for DNS queries 5 | - looked for ways to change network gateway for containers 6 | -------------------------------------------------------------------------------- /weekly/2017/21/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Evaluated the network throughput performance with the first Implementation. But the first implementation does not achieve good performance compared with the original Solo5-ukvm(up to 10% performance degradation). 3 | - The main reason of this performance is high overhead in data queue syncing with DMA triggering on a NIC on the sender side. The first implementation does packet sending one by one depending on data write function calls from the GuestOS layer, so highly frequent write function calls can easily lead to performance degradation. This is not anticipated because data queue syncing is managed by Netmap. 4 | - Additionally, I also found that softirq RX processing is always triggered when packet sending is done, and the softirq processing is heavy load. This is the specification of the current Netmap implementation. I need to reduce the number of packet sending calls as possible. 5 | - However, I will move to the second implementation phase because reduction of the number of packet sending calls is planned in the second phase. 6 | -------------------------------------------------------------------------------- /weekly/2017/22/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### ocaml-git / sirodepac 2 | 3 | Previously, I found a bug in my implementation. Now all is ok we can serialize 4 | all git object with a fixed size memory consumption. It's done by a 5 | deserialization with a fixed size memory consumption. So I played with some 6 | buffer between the serialization and the deserialization and all is fine. The 7 | produced PACK file work and we keep the integrity of the data. 8 | 9 | I asked to Gemma to have an access to a machine to run some huge tests about the 10 | compression. 11 | 12 | Then, I finish to write the documentation for ALL modules. All modules are 13 | explained and some tricks and implementation specified are described inside the 14 | ML file. I hope all is clear and someone can read and understand the code. 15 | 16 | I started to integrate all in ocaml-git. So, I did some big change. 17 | 18 | - I replaced the LRU module from Simon Cruanes by the implementation provided by 19 | David as the `lru` package in OPAM. A good point about this library is to 20 | control precisely the memory consumption of your object (and no how many 21 | object you can store). This point is good to keep the precise control about 22 | memory consumption because when we store some git object inside the cache, may 23 | be one took 1 Go and one other took 100 Ko. So, instead to keep these 2 24 | objects, we keep one of them if we limit the memory consumption by 5 Go for 25 | example. 26 | 27 | - I cleaned the interface of the CRC-32 checksum and provide a new type `t` 28 | which one is a `private int32`. So we keep the abstraction about the CRC-32 29 | (and don't do a mistake with the `int32`) and optimize the the computation 30 | when we want to serialize to an `int32` (by the sub-typing: `v :> int32`). 31 | 32 | I go to take my car to Sieam Reap sorry ... 33 | -------------------------------------------------------------------------------- /weekly/2017/22/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Submitted 3 papers to ICFP workshops 2 | - Mergeable Types: kcsrk.info/papers/mergeable_types_draft.pdf 3 | - Effectively Tackling the Awkward Squad: http://kcsrk.info/papers/awkward_effects_ml17.pdf 4 | - A Memory Model for Multicore OCaml: http://kcsrk.info/papers/memory_model_ocaml17.pdf 5 | -------------------------------------------------------------------------------- /weekly/2017/22/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I spent a lot of time in refactoring the API in neural module, and added several merge layers (add/mul/dot) into the module. 2 | - I finished the first version of `Owl_neural_graph` module. Owl can make a neural network of DAG topology. The new api is also easier to use. For example, making a convolutional neural network as below 3 | ```ocaml 4 | open Owl_neural;; 5 | open Owl_neural_feedforward;; 6 | 7 | let nn = input [|28;28;1|] 8 | |> conv2d [|5;5;1;32|] [|1;1|] ~act_typ:Activation.Relu 9 | |> max_pool2d [|2;2|] [|2;2|] 10 | |> conv2d [|5;5;32;64|] [|1;1|] ~act_typ:Activation.Relu 11 | |> max_pool2d [|2;2|] [|2;2|] 12 | |> dropout 0.1 13 | |> fully_connected 1024 ~act_typ:Activation.Relu 14 | |> linear 10 ~act_typ:Activation.Softmax 15 | in print nn;; 16 | ``` 17 | -------------------------------------------------------------------------------- /weekly/2017/22/seveneng.md: -------------------------------------------------------------------------------- 1 | - databox-bridge 2 | - started implementing a first version by plumbing through DNS for containers not on the same network 3 | - adopted the idea of a forward DNS server, if couldn't resolve, forward to `127.0.0.11:` 4 | -------------------------------------------------------------------------------- /weekly/2017/22/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Started implementation of the second phase. 3 | - Finished iothread porting from kvmtool[1], and confirmed ioeventfd-based trapping worked as expected. 4 | - Investigating the current design of the upper layers(mirage-tcpip and mirage-net-solo5) to have a new path between mirage-tcpip in a guest OS and Netmap on its host OS. 5 | 6 | [1] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git 7 | -------------------------------------------------------------------------------- /weekly/2017/23/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### ocaml-git / sirodepac 2 | 3 | I continue to integrate my work in ocaml-git. I decided to do a change 4 | *bottom-to-top* replacement: that means, I keep the interface of the Git module 5 | but I create a new implementation. Why ? ocaml-git is based on a stop the world 6 | serialization and we can see this constraint for the File System interface 7 | (which implements `read` and `write` but not on a non-blocking interface). 8 | 9 | I believe is that why ocaml-git consumes a lot memory because it does not work 10 | on a fixed size buffer. So I decide to add a non-blocking interface to the File 11 | System signature required. 12 | 13 | Then, I provide an Angstrom parser and a top layer to de-serialize a 14 | non-blocking input from an Angstrom parser. We have a limitation but I explain 15 | precisely why this limitation exists. 16 | 17 | So, I re-implement the loose file firstly and I will add my work then. It's a 18 | deep work, so when I have a result, I will notice. Another point is about the 19 | Hash. At this moment, we store the hash inside a bigarray (located directly in 20 | the major heap). May be we need to change this to a bytes to optimize the memory 21 | consumption (because the client should use a lot of hashes). 22 | 23 | Voilà, not so much this week to show but a deep work :) ! 24 | -------------------------------------------------------------------------------- /weekly/2017/23/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Planning for migrating Lwt and Async to direct-style effect-based I/O. 2 | * Experimenting with reify reflect for transforming monadic IO to direct-style. 3 | Experiments are here: https://github.com/kayceesrk/reify_reflect_concurrency 4 | -------------------------------------------------------------------------------- /weekly/2017/23/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I studied how the primitives of distributed computing are designed in Julia. 2 | - I implemented the experimental feature of distributed computing in Owl: for neural module and ndarray module. 3 | - I implemented ndarray.any module, it is an n-dimensional array module that supports any types besides numerical types. 4 | -------------------------------------------------------------------------------- /weekly/2017/23/seveneng.md: -------------------------------------------------------------------------------- 1 | #### databox-bridge 2 | - supported dns, containers from different networks that are connected by the bridge could resolve the domain names of each other 3 | - looked into [vpnkit](https://github.com/moby/vpnkit), code about de/multiplexing packets from/to multiple tcp/ip stack endpoints could probably reused by the bridge 4 | - when started, each stack registerd itself to some central dispatching unit 5 | - for input, push packets to the same stream 6 | - the central unit drain from the stream, parse packets, distribute packet to corresponding stack 7 | -------------------------------------------------------------------------------- /weekly/2017/23/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Finished implementation of Netmap buffer mapping test functions, and confirmed Netmap buffers on the host OS can be directly accessed from the guest Solo5-ukvm kernel. 3 | - The current memory mapping location is designed like MMIO, but tentative. So I will need to consider the best mapping address for Solo5-ukvm. 4 | - I will move to the next step, implementation of queue-based packet handling in the guest Solo5-ukvm kernel. 5 | -------------------------------------------------------------------------------- /weekly/2017/24/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### ocaml-git / sirodepac 2 | 3 | I continue to integrate my work in ocaml-git. So, the integration of the 4 | decoding is close to be done. Indeed, I can read any object from any PACK file 5 | with the same API. I decided to provide an *easy-to-use* API (same as the 6 | previous API of ocaml-git) and a more complex API to control precisely the 7 | allocation. With this API, we can compute in a parallel way the git object if we 8 | have some specific buffers available, otherwise, we can compute step by step 9 | each objects with a *global* buffer. 10 | 11 | So, now, I need to integrate the serialization inside ocaml-git and think about 12 | a good API. 13 | 14 | I continue to check my work step by step, try with `decompress` and `camlzip` to 15 | keep the compatibility. I put a documentation for all and describe some complex 16 | process inside the ML file to keep an understable code for other user. 17 | 18 | So, for this moment, all seems to be good and, I think, we will have a good API. 19 | 20 | #### CUFP 21 | 22 | I send my proposal after a review with Thomas, Gemma, Anil and KC, at the next 23 | CUFP, I'm waiting now the response :) ! 24 | 25 | #### OCaml network 26 | 27 | I'm currently in Singapore and met all people from Ahrefs. Obviously, I met 28 | Enguerrand (and he is good) but I met other people from this corporation and 29 | speak about ICFP, OCaml and other stuff. 30 | 31 | A good news, some people from this corporation keep an eye in the MirageOS 32 | project - and think that the unikernel is the future! 33 | -------------------------------------------------------------------------------- /weekly/2017/24/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I had been working on interfacing to CBLAS and LAPACKE last week. I wrote a parser to generate the interface automatically from cblas.h and lapacked.h. I also used Ctypes but the giant "foreign" function makes compilation extremely slow. I tried a couple of solutions then ended up using the c_stub code generated by ctypes but generate ocaml_stub myself in the parser. The parser also generates mli file to make sure the type signature is correct. This solution achieves both the strictness on type and efficiency on compiling. 2 | - Currently, Owl implements a full interface to all CBLAS and LAPACKE functions (over a thousand). These code are generated automatically, but I will write high level interface myself later this month. 3 | -------------------------------------------------------------------------------- /weekly/2017/24/sanderspies.md: -------------------------------------------------------------------------------- 1 | - started moving RWO examples from corebuild to jbuilder 2 | -------------------------------------------------------------------------------- /weekly/2017/24/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Designed implemented memory mapping of Netmap ring buffers so that they can be easily handled by the Guest OS layer. 3 | - I found the current Netmap implementation does not allow us to reduce the maximum number of slots on a ring buffer in order to reduce the amount of memory size, though I was expecting it allows us to do so. The maximum number depends on a network device I use. Therefore, the prototype will consume larger memory space than expected. 4 | -------------------------------------------------------------------------------- /weekly/2017/25/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I spent a lot of time in writing the interface between Owl and low-level interface to CBLAS and LAPACKE. 2 | - I started rebuilding the documentation of Owl, studied some new tools and also tried to find a new place to host the docs. 3 | - I have improved several functions in matrix module. 4 | -------------------------------------------------------------------------------- /weekly/2017/25/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Finished implementing the ring buffer manipulation functionality in the guest OS layer. I will start performance evaluation with it. 3 | - Tried to reduce the memory size mapped from the host OS layer. It worked fine, but requires a larger number of memory regions in Linux KVM. Memory regions are a concept of physical memory slots in KVM, so we cannot use a limited number of memory regions for a VM. 4 | -------------------------------------------------------------------------------- /weekly/2017/26/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### ocaml-git / sirodepac 2 | 3 | ##### Encoder 4 | 5 | I finish to integrate the core part of sirodepac inside ocaml-git. In the 6 | previous week, I explained that the couple Angstrom/Faraday is good about the 7 | serialization/deserialization. However, I have some bit problem with Faraday. 8 | The context of the serialization in ocaml-git is not the same than httpaf. The 9 | first difference is about where we serialize a Git object. 10 | 11 | Faraday was thinked to write directly to a file descriptor/socket. It was 12 | thinked to be share between 2 process, one to write in the internal buffer, the 13 | next to write to the file descriptor. But, in ocaml-git, we use the 14 | serialization of a Git object in a other case. One of this case is about the 15 | serialization to feed a context and get, at the end, an hash. So, the target of 16 | the serialization is not a file descriptor but an internal buffer. 17 | 18 | So, I decided to create a mini non blocking encoder with a fixed size memory 19 | fingerprint. And I finished in few day. It's used to produce the same as Faraday 20 | but with some news constraints. This is work perfectly and we keep a control 21 | about the memory consumption. 22 | 23 | ##### Write a PACK file 24 | 25 | Then, we can write a PACK file now but I need to polish deeply the API. This is 26 | the end thing to do. Before, I want to try my PACK file in the real world. 27 | 28 | ##### Smart Git Protocol 29 | 30 | To test my deserialization/serialization of the PACK file, I decided to interact 31 | with a Git server with the Smart Git Protocol. I saw the Sync.ml module, which 32 | implements all things about the communication. However, this module was thinked 33 | in the same way as the previous implementation of ocaml-git. 34 | 35 | So, I fund the same problem before and I decided to take down all. I 36 | re-implement the Smart Git Protocol in same way than ocaml-imap. I described why 37 | in a long comment (and why I don't choose Angstrom/Faraday). So, I can clone now 38 | and for this week I'm focus about the negociation with a Git server. 39 | -------------------------------------------------------------------------------- /weekly/2017/26/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Most of the time was spent in redesigning the linear algebra module. The new module is built atop of Owl's native interface to CBLAS and LAPACKE. 2 | - I was enhancing Owl's core functions: most of the vectorised math functions now support complex numbers. 3 | -------------------------------------------------------------------------------- /weekly/2017/26/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Finished performance evaluation with Netmap buffer manipulation in the guest OS side. I confirmed it can achieve 2.0 - 3.3x throughput under 1 sender/receiver pair on a host physical server. 3 | - However, its throughput under 6 sender/receiver pairs on two physical servers was similar to that with the original tap device. I found that this was mainly affected by non-Netmap processing. So I will try to identify what does affect the obtained performance. 4 | -------------------------------------------------------------------------------- /weekly/2017/27/kayceesrk.md: -------------------------------------------------------------------------------- 1 | * Published a new blog post: 2 | kcsrk.info/multicore/gc/2017/07/06/multicore-ocaml-gc/ which was discussed on 3 | HN: https://news.ycombinator.com/item?id=14780159 4 | * Submitted a grant proposal with Martin Kleppman et al. on set based CRDTs. 5 | Working on a Irmin based backend for this. 6 | * Multicore OCaml debugging. 7 | * Preparing for an SRG seminar on Multicore OCaml GC. 8 | * Preparations for ICFP talks / workshops / events / tutorials. 9 | -------------------------------------------------------------------------------- /weekly/2017/27/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I have been working on Owl's dependency last week. Due to the recently added interface to CBLAS and LAPACKE, Owl relies on OpenBLAS. However, CentOS does not provide the corresponding binary format of OpenBLAS. Building from source code using conf-openblas seems tricky. Anil suggested submitting an RPM. 2 | - I enhanced core functions in Owl, e.g., by add a new function to generate magic square matrix. 3 | - Owl was posted on Hacker News. This brought in a lot of publicity so I improved the documentation a bit. 4 | -------------------------------------------------------------------------------- /weekly/2017/27/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Trying to identify what factor affects the performance degradation. I found that GC related statistics under the solo5 with Netmap environment under single/concurrent send-recv pair(s) has not changed. 3 | - Started implementing an OCaml module for rdtsc() which is used to measure the execution time of a OCaml function on Solo5/ukvm. 4 | -------------------------------------------------------------------------------- /weekly/2017/28/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### ocaml-git / sirodepac 2 | 3 | ##### About fetch a repository 4 | 5 | So, first, I re-implement a negotiation engine about fetch. It follows the 6 | current non-optimal implementation of Git. I saw the paper about the bloom 7 | filter to optimize the negotiation but I don't have a time to implement this. 8 | 9 | The good point is the modularization of the fetch compute in ocaml-git. Indeed, 10 | we can change the negotiation engine by another engine easily. 11 | 12 | About the PACK file received, with Thomas Gazagnaire, we catch a weird compute 13 | from Git. In fact, when we receive a _thin-pack_ (which contains some externals 14 | references), Git took it and make a new canonical PACK file (which does not 15 | contain any external reference). Then, Git saves it in the store. 16 | 17 | It's weird because if we store directly the _thin-pack_ and avoid the next 18 | compute, all works. So, Thomas and me decide to let the choice to avoid the next 19 | compute and store directly the _thin-pack_ or generate a new canonical PACK 20 | file. This is prove my implementation of the encoder of the PACK file. 21 | 22 | ##### About push to a repository 23 | 24 | I continue to focus my work on the push command. When I try to understand what 25 | Git does, I saw a little DSL about set of commits (we can see this DSL with `git 26 | parse-rev`). This DSL is a key of which commit we need to send to the server. 27 | So, I implement this and some other stuff about reference. 28 | 29 | Then, I implement the smart protocol about the push command but I did not yet 30 | test the command. I think, it's done today or tomorrow :) ! 31 | -------------------------------------------------------------------------------- /weekly/2017/28/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I have been experimenting how to share small code snippets to extend Owl's numerical functionality. I implemented a "zoo" directive to let user share code snippets through gist. 2 | -------------------------------------------------------------------------------- /weekly/2017/28/sanderspies.md: -------------------------------------------------------------------------------- 1 | - Worked on the OCaml-webworker 2 | -------------------------------------------------------------------------------- /weekly/2017/28/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Conducted serval experiments to understand which part of the code is delayed, and found that udp.write and netif.writev functions on the sender side were delayed. 3 | - I will have further investigation on why this happens. 4 | -------------------------------------------------------------------------------- /weekly/2017/29/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I spent most of my time last week in enhancing the neural network module. The code in neural_graph module has been refactored and feedforward module will become obsoleted in the future version. 2 | -------------------------------------------------------------------------------- /weekly/2017/29/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (Netmap on Solo5-ukvm) 2 | - Found i) that the performance degradation occurs at the Ip.allocates_frame and Cstruct.concat and ii) LLC miss mainly affected it. 3 | - This LLC miss is highly related to repeated memory allocation for new packets to be sent. It can be reduced by using pre-allocated memory region provided by Netmap 4 | - But ... using the memory region does not have the backward compatibility in the IP layer, and I found it difficult to complete discussion on the compatibility and implement it by the end of my stay in Cambridge. 5 | - So I will not look into further performance improvement, and will move to implementation of a MQTT broker application. 6 | -------------------------------------------------------------------------------- /weekly/2017/3/dra27.md: -------------------------------------------------------------------------------- 1 | (reduced week) 2 | 3 | **OCaml 4.05** 4 | - Change Log Processor 5 | - Various GPR reviews 6 | -------------------------------------------------------------------------------- /weekly/2017/3/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage Storage** 2 | 3 | - reloading support, testing and bug fixing 4 | - indexing child and log structures from disk 5 | - freeing replaced nodes from the space map 6 | 7 | -------------------------------------------------------------------------------- /weekly/2017/3/gemmag.md: -------------------------------------------------------------------------------- 1 | * Finished MirageOS feedback blog: http://reynard.io/2017/01/18/MirageIRCFeedback.html 2 | * Ongoing report 3 | * MirageOS press details 4 | * New OCL site started: https://ocamllabs.github.io/ 5 | * Funding planning and allocation for next quarter 6 | -------------------------------------------------------------------------------- /weekly/2017/3/ilsordo.md: -------------------------------------------------------------------------------- 1 | - Started writing on the work I've done so far 2 | - Finished setting up the blog after some troubles with mathjax over https 3 | - Little progress on the Agda developments. Since we still don't have a typed cps translation I'm going to spend less time on this. 4 | - The plan for next week is to have a summary our attempts at "verifying" the cps translation either through a type system or a proof that it preserves semantics. 5 | Then we can start asking for outside feedback and see what we can do. 6 | -------------------------------------------------------------------------------- /weekly/2017/3/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Moving more issues to under Multicore issues. Organizing the issues under 2 | projects. 3 | - Chat with Daniel Hillestrom about progress on default handlers & ICFP papers. 4 | - Chat + planning with Gowtham for DaLi paper to ICFP/OOPSLA. 5 | -------------------------------------------------------------------------------- /weekly/2017/3/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Work done on lambda quoting: combinators are in place, now Translquote has to 2 | use them properly, i.e. the path arguments given to macros must become 3 | `lambda` instances (and not `Longident.t` instances). 4 | 5 | * After the last fixes on module type discovery, it seems that the compiler is 6 | now fully (except for syntax) backward-compatible, in the sense that projects 7 | that don't use macros should have the same dependency tree in 4.04 and in 8 | 4.04+macros. 9 | 10 | * My fork of Camlp4 now compiles on the macro switch. 11 | 12 | * Some reflexion on Flick and looking at the code of motto. 13 | 14 | * Flick: added a few elements in the DSL for channel declarations. For now, 15 | channels are implemented as a `'i option ref * 'o option ref` in the 16 | interpreter (i.e. a 1-capacity channel). Trying to replicate the program 17 | (https://github.com/NaaS/motto/blob/master/tests/runtime/factorial.ml)[factorial.ml]. 18 | 19 | * Flick: add operations on channels, temporarily implement channels in terms of 20 | two references on lists (for incoming and outgoing data) and write my first 21 | "embedded" Flick program using the unstaged interpreter. This program doesn't 22 | use processes. 23 | 24 | * Macros: Introduced a new constructor `Lescape` to denote when a piece of 25 | lambda code should not be lifted when constructing a lambda quote. My current 26 | problem is: how to construct Parsetree quotes and lambda quotes in the same 27 | recursion without doing a lot of pairing/unpairing? 28 | -------------------------------------------------------------------------------- /weekly/2017/3/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Release Owl 0.2.0 to OPAM. The new release contains a lot of significant changes in the fundamental data structures in Owl. The number types and precision can be passed in as parameters in creation functions. Meanwhile, the interface remains compatible with the old `Dense.Real` and `Dense.Complex` modules, the latter two have default 64-bit precision. 2 | - Write a tutorial on [Basic Data Types in Owl](https://github.com/ryanrhymes/owl/wiki/Tutorial:-Basic-Data-Types) 3 | - Write a tutorial on [Metric Systems in Owl](https://github.com/ryanrhymes/owl/wiki/Tutorial:-Metric-Systems) 4 | - Ben Caterall implemented a topic modelling algorithm (SparseLDA) using Owl, the pull request has been merged. This actual application of Owl will help in refining the interface of Owl. The code can be found [here on github](https://github.com/ryanrhymes/owl/blob/master/lib/topic/owl_topic_lda.ml). 5 | - LDA code revealed a problem of Sparse module in Owl. The new implementation of Sparse module actually slows down the LDA algorithm a lot. The reason is current matrix storage format (CRS) is not ideal for random access of matrix elements (even tough it speed up many linear algebra operations), whereas my previous implementation uses a combination of hash table and CRS. This needs to be solved in future. 6 | -------------------------------------------------------------------------------- /weekly/2017/3/seveneng.md: -------------------------------------------------------------------------------- 1 | - Read literatures about model-based data compression. Found two algorights that might be apt to integrate into pih-store, Adaptive Piecewise Constant Approximation and Slider Filter. They are both online processing algorights. The idea is to cut the stream of data into appropriate segments, and fit the data in each segment to a specific linear function. Next week, may try to implement the first version of this. 2 | 3 | - Continued learning about the design and structure of `packetdrill`. Still not clear about what shoud the statck test framwork should look like. 4 | 5 | - Browsed some repos that use `topkg` from [doc page](http://erratique.ch/software/topkg/doc/Topkg.html#menagerie), finished learning how to use it. 6 | -------------------------------------------------------------------------------- /weekly/2017/3/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - iperf tests for tracing done, so I am summarizing findings from the tests 3 | - Findings 4 | - the CPU utilization on the sender side reached 100% 5 | - there were waiting (or blocking) periods on the sender side, they are not anticipated and occupy 50% of the total network processing time. (Due to thread scheduling?) 6 | - the waiting periods always occurred during a "ring.write" phase 7 | - The receiver side had sleeping periods which would correspond to the waiting periods above, so its CPU utilization was not so high. 8 | - I will need to specify what triggers the waiting periods. 9 | -------------------------------------------------------------------------------- /weekly/2017/30/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - working on the multicore barriers to optimise the stop-the-world phase. Aiming to stay aligned with OCaml's current barrier model to ease upstreaming. 2 | - submitted end-of-year report for 1851 fellowship with Alan Mycroft's approval 3 | - submitted paper with Oleg about features of Eff using delimcc, and KC has one with multicore 4 | - discussion with Daniel Hillerstrom about ICFP tutorial for algebraic effects 5 | - this week: working on amd64 %r14 register usage for exception handling in multicore ocaml. Hard to measure these microbenchmarks on a macro scale, so need an opam-bench-repo for realistic use. 6 | -------------------------------------------------------------------------------- /weekly/2017/30/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I migrated the examples and performance tests in Owl to the newly introduced zoo system. 2 | - The indexing and slicing functions in Owl still needs enhancement. I also started studying the extension points in OCaml. 3 | -------------------------------------------------------------------------------- /weekly/2017/30/sanderspies.md: -------------------------------------------------------------------------------- 1 | gist tool: 2 | - autocomplete is now accessible via ctrl+space in the gist tool 3 | - moved to promise based communication with the webworker 4 | -------------------------------------------------------------------------------- /weekly/2017/30/seveneng.md: -------------------------------------------------------------------------------- 1 | - developping the bridge 2 | - adopted the idea of vpnkit: core part listening on a unix socket, unikernels, each responsible for a network interface, forwarding packets back and forth to the core part 3 | - provided two preliminary endpoints to configure the bridge: `/connect` and `/disconnect`, both accept a pair of container names, the bridge will resolve these names using a system resolver and translate them into ipv4 packets filter based on the resolved ip addresses 4 | - the configuration service is using a vnetif-based tcpip stack, pitfall got stuck in and out later after hours of debugging: `use_async_readers:true`, should've read the `README.md` : ( 5 | - control rules also include the limited name resolving rights, only `connected` components could resolve the name of each other 6 | 7 | -------------------------------------------------------------------------------- /weekly/2017/30/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Investigated which MQTT control packet I must implement, and found that I must 14 MQTT control packets to support the lowest QoS level. 3 | - Started implementing packet handlers for the 14 MQTT control packets. 4 | -------------------------------------------------------------------------------- /weekly/2017/31/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Enhance neural network modules, try to recreate Google's Inception for image classification. 2 | -------------------------------------------------------------------------------- /weekly/2017/31/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Finished implementing a packets parser for 9 MQTT control packets. 3 | - but need to implement packet forwarding database and a handler for it. I will do that next week. 4 | -------------------------------------------------------------------------------- /weekly/2017/32/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Refactor neural network module using functor, the new neural network module supports both single and double precision. 2 | - Replace Owl's old optimisation engine with the one in neural module. 3 | -------------------------------------------------------------------------------- /weekly/2017/32/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Finished writing code for the packet forwarding database. 3 | - I'll start functionality check for the whole application I implemented. 4 | -------------------------------------------------------------------------------- /weekly/2017/33/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Refactor the CBLAS interface, the number of functions is significantly reduced by providing a generic interface for S/D/C/Z types. 2 | - Optimise the core functions in Ndarray. I did some experiments trying to unifying matrix and ndarray types. It is doable but the current concern is then performance. Some operations like broadcast is faster in Matrix module than in Ndarray module. 3 | - Tune the interface between Actor and Owl, did some distributed learning experiments. 4 | -------------------------------------------------------------------------------- /weekly/2017/33/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Found a bug of higher CPU utilization (~90%) in hadling a incoming packet from MQTT publishers, and fixed it. 3 | - Confirmed the MQTT broker can operate as expected. Good result. 4 | - Started investigation of what kind of performance harness for MQTT brokers I can use to check the performance of the MQTT broker I implemented 5 | - Decided to use Gatling + its MQTT-plugin because I can easily modify them to change a workload definition depending on a scenario. I'm now learning and checking how to use it. 6 | -------------------------------------------------------------------------------- /weekly/2017/34/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Finished learning how to use Gatling + its MQTT-plugin for performance harness and confirmed they can issue a bunch of MQTT client requests as I expected (10,000 publishers and 1 subscriber). 3 | - Found a bug of my MQTT broker through the performance harness test. I also found out the bug is related to TCP connection close in handling MQTT disconnect requests, so I will fix it next week. 4 | -------------------------------------------------------------------------------- /weekly/2017/35/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Have fixed a bug on TCP connection close which I found in the last week. 3 | - Resumed performance evaluation of my MQTT broker. 4 | -------------------------------------------------------------------------------- /weekly/2017/36/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Finished performance evaluation of my MQTT broker, and found that my MQTT broker employing Netmap can handle 10,000 clients publishing 10,000 different topics with just 25% CPU utilization whereas a MQTT broker written in C (Mosquitto) indicated over 80% CPU utilization to handle the same clients. 3 | - Could not saturate the maximum MQTT broker performance due to the lack of resource shortage in the client side. But I found that my MQTT broker with Netmap can handle 15,000 clients with 43% CPU utilization. 4 | - I will write a report document to summarise my research work in OCaml Labs next week. 5 | -------------------------------------------------------------------------------- /weekly/2017/37/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Decompress 2 | 3 | I just started to test decompress with afl, we found 41 tests cases 4 | where decompress fails. So we can start to fix bugs. See the PR: 5 | 6 | https://github.com/mirage/decompress/pull/35 7 | 8 | #### Angstrom 9 | 10 | When I tried to produce a dot file for a git repository, I found a bug 11 | in Angstrom available in this issue: 12 | 13 | https://github.com/inhabitedtype/angstrom/pull/104 14 | 15 | #### Digestif 16 | 17 | I just finished the implementation of the RIPEMD160 hash algorithm 18 | (asked by @vbmithr) in C and in OCaml. The implementation is available 19 | in this PR: 20 | 21 | https://github.com/mirage/digestif/pull/12 22 | 23 | @hannesm points a problem about the license, so I wait a response from 24 | Antoon Bosselaers to use freely the C implementation. 25 | 26 | #### OCaml Git 27 | 28 | The PR#227 continues. I move the core library to `jbuilder`, implement 29 | a top-level and a program which produce a dot file of a Git 30 | repository - to test the current implementation. 31 | 32 | With these, I found some bugs and fixed all - bugs from the PR when I 33 | polished warnings. 34 | 35 | I found a segfault (which show me the bug in Angstrom) in 36 | ocaml+4.03.0. It's about a non-exhaustive pattern-matching on 37 | exceptions. Then, Angstrom raises an `Index out of bounds` and, 38 | because the pattern-matching is not exhaustive, we have an undefined 39 | behaviour (instead an exception from the OCaml runtime) and we finish 40 | to a segfault with a block with the tag 1002 (which should not appear 41 | after 3.11). 42 | 43 | I met Pierre Chambart and we discussed about that. When I handle the 44 | exhaustiveness of the pattern-matching, all is ok in ocaml+4.03.0 and 45 | in 4.04.0, OCaml raises a runtime exception about the exhaustiveness 46 | of the pattern-matching. 47 | 48 | Finally, I put an new issue in OCaml about warnings and the PPX, see 49 | #MPR7624. 50 | 51 | #### type-beat 52 | 53 | I updated type-beat to use the last version of Angstrom. 54 | -------------------------------------------------------------------------------- /weekly/2017/37/timada.md: -------------------------------------------------------------------------------- 1 | - Implementation (MQTT broker on MirageOS) 2 | - Finished writing a report document to summarise my research work in OCaml Labs. 3 | - I will do something needed to close my research at the lab such as termination of using servers. 4 | -------------------------------------------------------------------------------- /weekly/2017/4/dra27.md: -------------------------------------------------------------------------------- 1 | (reduced week) 2 | 3 | **OCaml 4.05** 4 | - Change Log Processor merge - failed to reach consensus. Shelved for now - will hopefully get turned into a Git merge driver instead. 5 | - Various GPR reviews 6 | - PR#7373, fixing a bug (of my own) in the FlexDLL bootstrap part of the build system. Tacking on some improvements to that which will feed into OPAM. 7 | -------------------------------------------------------------------------------- /weekly/2017/4/g2p.md: -------------------------------------------------------------------------------- 1 | 2 | **Mirage Storage** 3 | 4 | - Exercising the tree with synthetic data 5 | - Node splitting (work in progress) 6 | 7 | -------------------------------------------------------------------------------- /weekly/2017/4/gemmag.md: -------------------------------------------------------------------------------- 1 | * Continuing with OCL site: https://ocamllabs.github.io/ 2 | * Planning for new interns and visitors 3 | * Catching up with Platform activity: odoc, odig, topkg 4 | * Planning Compiler Hacking: https://ocamllabs.github.io/compiler-hacking/2017/01/24/february-compiler-hacking.html 5 | -------------------------------------------------------------------------------- /weekly/2017/4/ilsordo.md: -------------------------------------------------------------------------------- 1 | - Spent most of my time figuring out the application process for my future PhD... 2 | - Made some progress on the report, the introduction is almost done. 3 | -------------------------------------------------------------------------------- /weekly/2017/4/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Discussion with @stedolan about mergeable datatypes 2 | - *if cons is the basis of functional data structures, what is the basis of mergeable funcational data structures?* 3 | - Multicore OCaml 4 | - Submitted PR for OSX compilation: https://github.com/ocamllabs/ocaml-multicore/pull/103 5 | - Working on fixing backtraces: https://github.com/ocamllabs/ocaml-multicore/issues/93 6 | - Rubber ducking for memory model discussion with @stedolan. 7 | -------------------------------------------------------------------------------- /weekly/2017/4/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Study automatic algorithmic derivation, prepare for the implementation. 2 | - Work on distributed data processing system (Actor System), document what has been done so far. 3 | - Refine and formulate the idea of a new barrier control technique in distributed machine learning, write the initial draft, plan the evaluation. 4 | -------------------------------------------------------------------------------- /weekly/2017/4/seveneng.md: -------------------------------------------------------------------------------- 1 | - work done for Databox 2 | - meeting with mort and Liang to clarify the functionalities/patterns of interactions for the new system components to be developped: storage and bridge. 3 | - for storage, it will provide tamper-proof, append-only logging of each store operations, and it will support different data types: Json, Timeseries, Blob, etc. Later, we could be more high-level features like model based compression on this 4 | - for bridge, still needs more discussion and thinking, for the first step, we could build it with "redirect point" style, which allows third parties to register/update their services, look up the others' services, and also allows internal components to insert/delete their own plug-in functions. 5 | 6 | - supervision 7 | - IB course CompNet 8 | -------------------------------------------------------------------------------- /weekly/2017/4/stedolan.md: -------------------------------------------------------------------------------- 1 | - off sick monday / tues 2 | - progress on memory models! (more soon..) 3 | - mpi/multicore debugging with @dhil 4 | -------------------------------------------------------------------------------- /weekly/2017/4/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - Conducted a test on the unix configuration, and found that the sender and receiver sides behavior was quite simple without any blocking periods. This indicates Xen or the network bridging on the hostOS can affect the blocking periods. 3 | - Having additional tests where the sender(receiver) program on a MV and the receiver(sender) program on Dom0 to reduce noise from the virtualization side. 4 | - Had a meeting with Docker members to explain the findings. Comments from them are as follows; 5 | - To try to put one of the sender or receiver sides on Linux. 6 | - To try to have one unikernel VM having both the sender and receiver side using the vnetif module to completely remove noise from network bridging on the hostOS. 7 | - Checksum offloading and IP fragmentation offload will help us to improve the network performance. 8 | - ukvm rather than Xen would be better and easier for implementing a new network interface layer. 9 | -------------------------------------------------------------------------------- /weekly/2017/41/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Decompress 2 | 3 | I released the new 0.7 version of decompress. I currently work on a 4 | good API about javascript. I found some API in the npm repository but 5 | these API seems to be imperative - and it's not the case decompress. 6 | 7 | NOTE: I think, it's imperative because it's just a FFI with zlib 8 | directly (without convenience computation). 9 | 10 | #### Digestif 11 | 12 | I just finish to implement the BLAKE2S function and test it and merge 13 | it. I currently look about benchmark with Louis and provide something 14 | to test and compare with ocp-sha. 15 | 16 | About the test, I just look what is it exactly and we have lot of 17 | litterature to test SHA-*, BLAKE2{b,s} and RIPEMD160 implementation. 18 | So, I will integrate all. 19 | 20 | #### Callipyge 21 | 22 | cfcs catched a bug about Callipyge, it does not work. So, I think, I 23 | will re-implement it from the reference implementation. 24 | 25 | #### OCaml Git 26 | 27 | I just implemented the fetch command on the HTTP protocol. Now, I can 28 | start to implement the push command on the HTTP protocol. Louis asks 29 | me about the current implementation of my PR and he tried to test it. 30 | 31 | Obviously, I did not work yet on the polishement of opam file and we 32 | need to pin digestif and a specific version of angstrom (which has my 33 | fix). Then, he compiled ocaml-git without error but it's not easy, I 34 | started to figure about opam file. 35 | 36 | Again, I fixed some bugs specifically about reference (between 37 | absolute path of the reference and relative path) and the PACK decoder 38 | which handles an empty PACK file now. 39 | 40 | I cleaned the smart decoder and delete the redundant code and 41 | understand what is specifically the diff between the Smart protocol 42 | and the Smart HTTP protocol - because, in the Git documentation, we 43 | don't have any explanation about that. 44 | -------------------------------------------------------------------------------- /weekly/2017/42/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Decompress 2 | 3 | Find a way to provide a JS API of Decompress with a mutable state. 4 | 5 | ```ocaml 6 | type inflator = { mutable contents : (B.st, B.st) Inflate.t } 7 | ``` 8 | 9 | So, the code is specialized to manipulate only a bytes (but still ok 10 | where the JS world does not provide an equivalent of Bigarray - in 11 | performance). 12 | 13 | #### Digestif 14 | 15 | Find a new way to test the C implementation and the OCaml 16 | implementation with travis. 17 | 18 | Fixed a bug about the RIPEMD160 implementation in OCaml. 19 | 20 | Put some other tests (from the BLAKE2{b,s} reference implementation, 21 | and the RIPEMD160). 22 | 23 | Benchmark on the C implementation and the OCaml implementation with 24 | `core_bench`. However, I did not compare with `ocp-sha`, firstly 25 | because I lost my time to make a package for ocp-sha (Louis provides 26 | only a Makefile). And because ocp-sha was thinked to hash a file (not 27 | a stream or a buffer) and tricks on to be fast - however, I can not 28 | say ocp-sha is faster than digestif when ocp-sha uses some syscall 29 | like lseek. 30 | 31 | So, if we want to compare digestif and ocp-sha, it's about program 32 | which take a file but it's not very relevant when ocp-sha map the file 33 | and digestif could compute a stream of this file. 34 | 35 | Finally, make a release. 36 | 37 | #### OCaml Git 38 | 39 | Try to move the test implementation on the new API. Nothing change a 40 | lot, I missed some conveniences accessors and use infix operator of 41 | Lwt_result instead Lwt. 42 | -------------------------------------------------------------------------------- /weekly/2017/43/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### OCaml Git 2 | 3 | Focus on the test of OCaml Git (the common part). So, I put some 4 | constraints to prove the equality of the type Value.t between 2 5 | implementation of the Git repository with the same Hash module. 6 | 7 | I added some conveniences functions again (like Git.Ref.exists or 8 | Reference.head_contents). 9 | 10 | I fixed 2 bugs: 11 | - one about the semantic of `Mem.read_inflated` which differ from the 12 | `FS.read_inflated`, the first one put the header and the last one 13 | not. But we expect the last semantic. 14 | - A deep bug about the abstract serialization of a value 15 | 16 | Then, I test the Git.Pack.make function and add a returned value which 17 | is a protected variable to get the IDX tree - only available when we 18 | consume all of the stream. 19 | 20 | I implemented the push function for the Sync_http module (so the Smart 21 | HTTP protocol) but not test it yet. And finally put some /easy-to-use/ 22 | functions on the provided CoHTTP implementation of the Smart HTTP 23 | protocol (for Camelus). 24 | 25 | Louis reported to me a bug about fetch, I will fix it. 26 | -------------------------------------------------------------------------------- /weekly/2017/5/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Work on the implementation in Decompress 2 | 3 | Move the implementation of the compression (deflate) in the same way of the 4 | decompression. With the work of Yallop, the implementation of _blosclz_ and the 5 | advises of Thomas, I try to use a pure state for the Lz77 compression (first 6 | layer of Decompress) and the Zlib compression (second layer of Decompress) to use 7 | this implementation in `ocaml-git` to pack a git state. 8 | 9 | For the moment, I have a non-blocking implementation of the Lz77 compression 10 | with theses changes: 11 | * pure state 12 | * non-blocking computation 13 | * remove a compute of a window - that means when the algorithm search a 14 | pattern, we limit to the input of the user. So, if the user compute the 15 | compression with a chunk of 4096 bytes (for example), we search a pattern only 16 | on this chunk (but theorically, we can keep ~ 28671 bytes) and the ratio 17 | between the literal and the distance for this compression will be bad. But, in 18 | a practical world, we compute the compression with a chunk of 32K bytes - and 19 | in this way, we have an optimal compression. 20 | 21 | Finally, for a file (like an executable), we have a ratio about 1/3 for 22 | literals and 2/3 for matches - so 2/3 of compression without Huffman 23 | compression. 24 | 25 | Now, I attack the Zlib (Huffman) compression in the same way (pure state and 26 | non-blocking) to get the new release of Decompress. 27 | 28 | #### ocaml.org 29 | 30 | I try to find a bug inside ocaml.org about the feed generator (OCaml Planet). 31 | In the same time, I found lot of website with a bad or unaccesible feed. So, I 32 | sent an e-mail to advise some persons about that - like the website of Mirage 33 | or ocaml.io. 34 | 35 | Chris00 fixed the bug (server side bug). 36 | 37 | #### Encoding and Mr. MIME 38 | 39 | It's about the UTF-7. Some e-mails don't respect the encoding described by the 40 | standard (RFC 6532). The point is to create a library (but I already discuss 41 | about that with Daniel) to convert any encoding to UTF-8. 42 | 43 | Mr. MIME has this problem, we respect the standard but not the practical 44 | e-mail, we need to be more resilient about that. 45 | -------------------------------------------------------------------------------- /weekly/2017/5/dra27.md: -------------------------------------------------------------------------------- 1 | _Away for most of the week_ 2 | 3 | **OCaml 4.05** 4 | - Work on GPR#1010 (Prefixing the OCaml Standard Library 5 | -------------------------------------------------------------------------------- /weekly/2017/5/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | - Node splitting 4 | - Publish the work done in January to 5 | 6 | -------------------------------------------------------------------------------- /weekly/2017/5/gemmag.md: -------------------------------------------------------------------------------- 1 | * Adding more information to new OCL site - getting there! 2 | - Need to add projects next, most of the authors now added 3 | * Checking into Platform progress: odig, odoc, topkg 4 | * Preparing for Maintainerati in SF (15th Feb) 5 | * Planning Romain's next project with us 6 | * Preparing for Compiler Hacking next Tuesday 7 | -------------------------------------------------------------------------------- /weekly/2017/5/ilsordo.md: -------------------------------------------------------------------------------- 1 | I'm trying to figure out all the places in TypedTree which end up as function definitions or applications for the effect analysis. 2 | 3 | I also spent some time improving error messages in the fomega tool used for the advanced functional programming course. 4 | -------------------------------------------------------------------------------- /weekly/2017/5/oliviernicole.md: -------------------------------------------------------------------------------- 1 | * Fixed small bugs with lambda quoting, bootstrapped the compiler. 2 | * Added some (hopefully temporary) code to print lambda quotes in the REPL. 3 | * Fixed a bug on `macros_unstable` that caused the compiler to crash when 4 | encountering a structure item with more than one splice in it. 5 | * On the way of fixing the bug that makes codes like: 6 | ``` 7 | module M = struct 8 | $ << () >> 9 | end 10 | ``` 11 | segfault. 12 | -------------------------------------------------------------------------------- /weekly/2017/5/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - Implement a simple prototype of forward AD, but it does not support higher order derivatives. I need to read a bit more on algorithmic differentiation. 2 | - Try to test some barrier control logic in distributed learning. 3 | - Refactor the topic modeling module in Owl. 4 | -------------------------------------------------------------------------------- /weekly/2017/5/seveneng.md: -------------------------------------------------------------------------------- 1 | - storage component of Databox 2 | - implemented tamper-proof logging feature of the store 3 | - implemented store engine, providing support for json and blob(by Cstruct.t) data types 4 | - browsed repo [ocaml-macaroon][1] and [opium][2], for the purpose of integration into the store 5 | 6 | [1]: https://github.com/nojb/ocaml-macaroons 7 | [2]: https://github.com/rgrinberg/opium 8 | -------------------------------------------------------------------------------- /weekly/2017/5/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - Implemented my iperf program using Mirage vnetif as an interconnect, and found there were still unexpected waiting(blocking) periods. So I concluded that somthing in MirageOS triggered this behavior. I will investigate a root cause of it by starting with further tracing schemes in MirageOS. 3 | - Conducted further analysis based on the previously obtained results, and found that the host OS side processing(i.e. network bridging) can occupies higher than 65% of the total network processing time without the unexpected waiting periods. This is an expected result, not suprising. The host OS side processing time can be reduced by taking advantage of existing schemes such as Netmap or DPDK. 4 | - Prioritized topics to be conducted 5 | 1. investigation of the waiting periods not anticipated 6 | 2. implementation design to reduce overhead in the host OS side 7 | 3. implementation design to reduce overhead in the MirageOS network protocol layer 8 | -------------------------------------------------------------------------------- /weekly/2017/6/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Work on the implementation in Decompress 2 | 3 | I continue to implement the compression. Good news, the dynamic compression 4 | works with the dbuenzli's interface and the static compression should work too. 5 | The first layer (Lz77 compression) is done - the question is, it's good to 6 | functorize the implementation of the Lz77 or not ? I need to fix a bug in the 7 | flat compression - but it's very easy, may be one or two hours to fix that. 8 | Then, I will polish the interface. So, the second layer (Zlib compression) is 9 | most done. 10 | 11 | Now, we have a complete interface for Decompress (with differents flush methods, 12 | like `partial` for SSH, `full` and `sync`). It's possible to change the 13 | frequencies before than Decompress computes a canonic Huffman tree. That means, 14 | it's possible to specialize the canonic Huffman tree with an 15 | external database of frequencies. 16 | 17 | After polishing, I will do some benchmark with `landmarks` and try to compare 18 | with `zlib`. And, I will make the new release of Decompress (tagged 1.0). Then, 19 | I will attack the serialization to a PACK file. 20 | 21 | UPDATE: I finish to fix the flat compression bug, I will make a good test 22 | suites. 23 | 24 | UPDATE: I just add a test suites and `decompress.ml` works! 25 | 26 | #### Encoding and Mr. MIME 27 | 28 | I don't have internet for now (for two weeks) so I can't see the database about 29 | the translation between an encoding and utf-8. But when I can get this database, 30 | I will start the new library (the name will be `uuuu`) and normalize all 31 | output of Mr. MIME to utf-8. 32 | 33 | ---- 34 | 35 | May be, I will push (I can't for the moment because I don't have internet) my 36 | work in `dinosaure/sirodepac` repository in few days. 37 | -------------------------------------------------------------------------------- /weekly/2017/6/dra27.md: -------------------------------------------------------------------------------- 1 | _On "holiday" this week_ 2 | -------------------------------------------------------------------------------- /weekly/2017/6/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | * Node splitting 4 | * Prep for log spilling 5 | * Some refactoring of childlinks 6 | -------------------------------------------------------------------------------- /weekly/2017/6/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I re-implemented the forward mode of algorithmic differentiation [[code](https://github.com/ryanrhymes/owl/blob/master/lib/owl_algodiff_forward.ml)]. Now the Algodiff supports higher-order derivatives now, check [here](https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic-Differentiation) please. 2 | - To support higher-order derivatives, Algodiff uses many recursive functions and also recursive modules. However, I was told recursive module is not a good option in OCaml due to losing the capability of compiler optimisation. I do not have a solution at the moment. 3 | - I overloaded some common math operators in Algodiff module. However, some matrix operations still need to be implemented. 4 | - I did some barrier control experiments in distributed learning. The simulator and result analysis are all done on top of Owl. The current results look good, Actor system seems more scalable than other barrier control. But I need to investigate a more into the accuracy. 5 | -------------------------------------------------------------------------------- /weekly/2017/6/seveneng.md: -------------------------------------------------------------------------------- 1 | - implementation of [databox-storage][1] 2 | - design and implementation of [databox-bridge][2] 3 | 4 | 5 | [1]: https://github.com/sevenEng/databox-storage 6 | [2]: https://github.com/sevenEng/databox-bridge 7 | -------------------------------------------------------------------------------- /weekly/2017/6/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance evaluation 2 | - Found out that the long waiting periods is a Xen-specific issue, it did not occur under my QEMU/KVM configuration. 3 | - Observed waiting periods even under the QEMU/KVM configurations, but they were shorter than under my Xen configuration and their fequency was low. 4 | - I will not have furuther investigation on this issue if not required, so I will move to designing new software archtecture to achieve higher network perforamnce on MirageOS. 5 | 6 | - FOSDEM2017 7 | - Attended the event to get topics realted to network, micro-kernel, micro-service, virtualization. 8 | - Several presentations were intersting for me (For example, TCP acceleration by using Intel Transration Layer Development Kit which is based on DPDK). 9 | - I will have a short presentation to report this topic on 28th Feb. 10 | -------------------------------------------------------------------------------- /weekly/2017/7/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### Work on the implementation in Decompress 2 | 3 | I finish to polish Decompress (test and documentation). So all is ok and the 4 | new version of Decompress (0.5) is ready. I choose to create the 0.5 version 5 | (and not the 1.0) because I insert an experimental thing in API and I would 6 | like to create a new test and prove that is more reliable to get the 1.0 7 | version. 8 | 9 | Indeed, I inserted the flush methods (`partial`, `full` and `sync`) and you use 10 | Decompress without these flush methods. After, all code is in OCaml (`alder32` 11 | compute and `memcpy` implementation). 12 | 13 | Apparently, Decompress works with solo5, so it's a good news! 14 | 15 | May be, I will push the new version in the hackathon because I don't have a 16 | good internet for the moment. 17 | 18 | #### `ocaml-git` and `sirodepac` 19 | 20 | I continue to work on the `sirodepac` at the same time. I just create a mini 21 | non-blocking decoder. May be I will implement a little fun interface with 22 | `ocaml-d3`. So I have a full deserialization of PACK and IDX git file and we 23 | can make easily the glue between `sirodepac` and `ocaml-git`. 24 | 25 | So, I will create a mini non-blocking encoder now and after I will attack the 26 | main purpose of my mission, the create of the PACK file - to fix the `git push`. 27 | Thomas point me some bug, so I will focus now on this :) ! 28 | 29 | #### Mr. MIME and the buffer 30 | 31 | When I create the mini non-blocking decoder, I find a new way to handle the 32 | buffer and let the user to grow the buffer. It's the best choice (about the 33 | security) to let the user to grow the buffer. For the moment, it's Mr. MIME to 34 | grow the buffer and you can observe that but not precisely. 35 | 36 | So, I push this big thing in my TODO. It's not complex but this idea changes 37 | the API a lot. 38 | 39 | Voilà! 40 | -------------------------------------------------------------------------------- /weekly/2017/7/dra27.md: -------------------------------------------------------------------------------- 1 | ** OPAM ** 2 | - First attempt at rebasing onto Beta 2 (stuck at lib-pkg merge) 3 | 4 | ** OCaml 4.05** 5 | - Reviewing fix for GPR#861 relating to a DST-bug in Unix.stat on Windows. 6 | Far too much time spent reading MSDN and grep'ing source code of old CRTs... 7 | - Cygwin-32 fork crash. Moving mmap functions from Bigarray to Unix has broken 8 | Cygwin fork (reliable test case). Far too much time spent reading Cygwin 9 | source code - it's not yet clear whether this is a Cygwin bug, a FlexDLL bug 10 | or an OCaml bug... 11 | -------------------------------------------------------------------------------- /weekly/2017/7/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | - Anticipate out of space situations and raise an error if necessary 4 | - Log spilling: find which child can receive the most data 5 | 6 | -------------------------------------------------------------------------------- /weekly/2017/7/kayceesrk.md: -------------------------------------------------------------------------------- 1 | - Wrote blog post on Ezirmin: 2 | kcsrk.info/ocaml/irmin/crdt/2017/02/15/an-easy-interface-to-irmin-library/ 3 | - Added ropes + operation transformation to Ezirmin. 4 | - Implemented prefetch instruction for OCaml 5 | - Working on the semnatics of DaLi progrmaming model 6 | -------------------------------------------------------------------------------- /weekly/2017/7/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - I added several more functions into Owl module such as tile, repeat, and etc. 2 | - I added matrix type into current AD forward mode to support simple linear algebra. 3 | - I started to re-design another AD module in order to have better supports for both forward and backward AD (to be implemented). Moreover, in new AD implementation, I change from recursive modules to a chain of recursive functions. 4 | -------------------------------------------------------------------------------- /weekly/2017/7/seveneng.md: -------------------------------------------------------------------------------- 1 | - implementation of [databox-bridge][1] 2 | - using ocaml-macaroons to do the access right control of API 3 | - allow local driver/app to submit export request to a queue 4 | - worker thread (single one for now) extracts a request from the queue and processes it 5 | - driver/app polling the same API to get update on its request (may support websocket notification) 6 | 7 | - next week may spend some time to test the basic working flow, add logging operations, ws support maybe 8 | 9 | [1]: https://github.com/sevenEng/databox-storage 10 | -------------------------------------------------------------------------------- /weekly/2017/7/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Conducted investigation to understand which part (MirageOS or the host OS) affects the waiting periods, and found out only MirageOS probably does affect them. I confirmed that network processing on the host OS does not take such long time. 3 | - Started reading the source code of Solo5-ukvm to understand the current implementation of it. I found that (i) modular-based implementation in Solo5-ukvm helps me to easily add my new networking feature and (ii) the current Solo5-ukvm does not use vhost-net (= large room for performance improvement) 4 | - I will continue to understand the current Solo5-ukvm architecture in more detail next week. I will also start designing my networking feature after that. 5 | -------------------------------------------------------------------------------- /weekly/2017/8/dra27.md: -------------------------------------------------------------------------------- 1 | ** OCaml 4.05 ** 2 | - Cygwin-32 fork issue ongoing - identified that the problem is DLL load order, but 3 | not yet clear why this only affects Cygwin-32 or what the fix should be. 4 | - OCaml Developers' Meeting - a number of old GPRs discussed and triaged. Plea from 5 | Xavier that each dev triage, and preferably resolve, 5 Mantis PRs pcm. 6 | -------------------------------------------------------------------------------- /weekly/2017/8/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | * Log spilling 4 | * Move testing to ramdisks 5 | -------------------------------------------------------------------------------- /weekly/2017/8/gemmag.md: -------------------------------------------------------------------------------- 1 | * Attended [Maintainerati](https://maintainerati.org/) in SF last week (15th Feb) 2 | * Working on blog post about conference 3 | * Helped schedule OCL Platform blog posts go coincide with Mirage 3.0 release 4 | * General blog posts for OCL site 5 | * Porting the rest of our older content onto new site 6 | * Paperwork for new starter - Nicolas Assouad starting his internship next Monday (27th) 7 | * Wrote short blog post on Irmin release: http://ocamllabs.io/releases/2017/02/24/irmin1release.html 8 | * Designed and ordered MirageOS Hack t-shirts 9 | * Broke Atom on my laptop :( 10 | -------------------------------------------------------------------------------- /weekly/2017/8/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - The first version of backward AD was implemented in Algodiff module. I overloaded most of the commonly used math operators. 2 | - I added another sub-module in Algodiff which is able to provide numerical differentiation. This can be used to cross-validate the results from Algodiff.AD module. 3 | - I added more functions (softplus, softsign, sigmoid, softmax, etc.) to Owl to provide better support for machine learning algorithms. 4 | - I have been doing experiment for Paar paper, and at the same time we have been trying to make ARM-based docker image for Owl so that we can run it on Raspberry Pi. 5 | -------------------------------------------------------------------------------- /weekly/2017/8/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Found that the longer waiting periods under Solo5-virtio were caused due to GC, and that they can be managed by changing the minor heap size. I will not have further investitation on the above because they are not a dominant factor of the current performance bottleneck. 3 | - Completed reading the source code of Solo5-ukvm, learned how Solo5-ukvm operates in receiving and sending network packets. 4 | - Also found that Solo5-ukvm can have longer waiting periods not anticipated expecially in packet sending (maybe) due to its polling mechanism. 5 | - I will need to determine if I should tackle the new issue under Solo5-ukvm, and to start designing new networking archtecture for Solo5. 6 | -------------------------------------------------------------------------------- /weekly/2017/9/dra27.md: -------------------------------------------------------------------------------- 1 | _Ill for most of the week :( _ 2 | 3 | ** OCaml Labs ** 4 | - Preliminary discussions with @avsm, @gemmag on moving the CI into the lab. 5 | 6 | ** OCaml 4.05 ** 7 | - Cygwin fork issue fixed! Patch accepted upstream to the Cygwin DLL. Issue was 8 | that when DLLs are dlopen'd using FlexDLL, the dependencies between them are not 9 | known to Cygwin/Windows. Cygwin contained a dependency-based topological sort which 10 | was unstable for DLLs with no dependencies: the effect was that even calls to fork 11 | would work (as the list is reversed each time). DLL rebasing meant that the problem 12 | wasn't apparent on Cygwin64, but with correctly based DLLs, the same problem occurred. 13 | OCaml docs will be updated once the new Cygwin DLL has been released to note that the 14 | minimum version. 15 | -------------------------------------------------------------------------------- /weekly/2017/9/g2p.md: -------------------------------------------------------------------------------- 1 | **Mirage storage** 2 | 3 | * Refactoring 4 | * Child node splitting 5 | * Bug fixing 6 | * Functoria pull request to switch the block implementation 7 | -------------------------------------------------------------------------------- /weekly/2017/9/gemmag.md: -------------------------------------------------------------------------------- 1 | * Moving a lot of information over from ocaml.io to ocamllabs.io - taking a long time 2 | * Still working on maintainerati blog draft... 3 | -------------------------------------------------------------------------------- /weekly/2017/9/ryanrhymes.md: -------------------------------------------------------------------------------- 1 | - We finalised the paper on privacy-preserving model training paper and submitted to PoPETs. 2 | - I worked with Jianxin to installed Owl on Raspberry PI and did some topic modelling experiment. There is an ARM-based docker image for Owl now. 3 | - I spent a couple of days in studying the software licensing stuff and made a new plan for Owl development. I will gradually remove the dependency on GSL to make sure Owl will remain MIT licence. 4 | - I start building an extension atop of Owl so that different types of math objects (scalar, matrix, ndarray) can interoperate. 5 | -------------------------------------------------------------------------------- /weekly/2017/9/seveneng.md: -------------------------------------------------------------------------------- 1 | - [databox-export-service][1] (*we renamed the repo from 'databox-bridge' to this*) 2 | - Dockerised this repo, added docker hub autobuild, added opam file to use opam to install the service 3 | - Supported https by wrapping `cert` and `key` environment varialbles into local files, and passing them to the `opium` interface 4 | - Refactored [test.ml](../blob/master/test/test.ml) to make it easier to write more tests from client side rather the monolithic single test before this 5 | - Added some new tests to test against scenarios where we would have invalide id or macaroons etc. 6 | - Tried out some new libraries to make days easier: `depyt`, `rresult`, `bos` etc. 7 | 8 | - databox-bridge: 9 | - Discussed the functionalities and interactions with other components, should be able to see the very first version next week 10 | - Step one: assuming bridge has connected to the right network and so do other components, it should recognize DNS requests, give right responses and then forwarding ethernet packets to the right interfaces 11 | 12 | *Only realized forgot to add the weekly 8 while in the middle of this week, so this log actually covered work done for the past two weeks* 13 | 14 | [1]: https://github.com/me-box/databox-export-service.git 15 | -------------------------------------------------------------------------------- /weekly/2017/9/timada.md: -------------------------------------------------------------------------------- 1 | - Network performance 2 | - Attended MirageOS Hack Retreat on 1st-2nd March, and mainly had discussions on Solo5 with Martin. 3 | - Learned the current implementation philosophy of Solo5 and found that it focus on only simplicity and its functionality, not on performance. 4 | - Conducted iperf experiments under Solo5-ukvm, and confirmed that longer waiting periods in the sender side actually affets the iperf throughput performance. 5 | (Only 7MB/sec throughput due to the waiting periods though I confirmed the receiver side can achieve 70MB/sec throughput) 6 | - I will start designing new networking scheme wich consideration of the current Solo5-ukvm. 7 | -------------------------------------------------------------------------------- /weekly/2017/dinosaure.md: -------------------------------------------------------------------------------- 1 | #### OCaml Git 2 | 3 | Continue to integrate the test suit of ocaml-git with the new API. 4 | Again, it's about move `git_object option Lwt.t` to `(git_object, 5 | error) result Lwt.t`. However, I put lof of constraints now to prove 6 | the equivalence of Value.t type like: 7 | 8 | ```ocaml 9 | module V1 = Git.Value.Make(SHA1)(C_inflate)(C_deflate) 10 | module V2 = Git.Value.Make(SHA1)(OCaml_inflate)(OCaml_deflate) 11 | 12 | V1.t = V2.t 13 | ``` 14 | 15 | So, it's very cool for the end user because, in this way, he can use a 16 | `Mem.Value.t` (memory back-end) in a Store function (file-system 17 | back-end) which expects a `Store.Value.t` only if we prove the 18 | constraints and use the same Hash module implementation.a 19 | 20 | Then, I implemented, as Thomas asks, the index decoder whichs works. I 21 | tested ot and the main decoder and the extension works fine - however, 22 | it's not well-tested. 23 | 24 | Finally, I helped Louis to integrate OCaml Git in Camelus. Then, 25 | Thomas asks me to implement and easy library to clone/fetch a git 26 | repository. The fetch function has lot of argument, so I implemented 27 | git-cohttp-lwt-unix which is a specialized implementation of git-http 28 | with cohttp-lwt-unix. 29 | 30 | In this library, I provided `fetch_all` which is more easy to use than 31 | `fetch` and apply all functor needed to make the HTTP Smart protocol 32 | implementation. 33 | -------------------------------------------------------------------------------- /weekly/intro.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: OCaml Labs Weekly Reports 3 | author: OCaml Labs Group 4 | rights: CC BY-SA 4.0 5 | language: en-US 6 | --- 7 | 8 | These weekly notes are rough scribings of activities that happen on a weekly 9 | basis. They are intended to be an internal journal for Lab members rather than 10 | something consumed by the outside world. If you have any queries about the 11 | contents, please contact Gemma Gordon, KC Sivaramakrishnan or Anil Madhavapeddy. 12 | 13 | -------------------------------------------------------------------------------- /yearly/intro.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: OCaml Labs Annual Reviews (2011-2014) 3 | author: Anil Madhavapeddy, Gemma Gordon, KC Sivaramakrishnan 4 | rights: CC BY-SA 4.0 5 | language: en-US 6 | --- 7 | 8 | These are annual reviews of the activities of OCaml Labs -- an initiative within the Cambridge Computer Laboratory started by Anil Madhavapeddy in 2011 to promote research, growth and collaboration within the wider OCaml community. We contribute to management of the day-to-day OCaml maintenance load, and align research agendas with real-world projects to help progress the language and make it available and applicable to a larger audience. Together with INRIA and our other partners, our goal is to freely release and integrate all work upstream, allowing all prospective users access to the efficient, expressive language of OCaml. 9 | 10 | --------------------------------------------------------------------------------