├── .envrc ├── .gitignore ├── CHANGELOG.md ├── HACKING.md ├── LICENSE ├── README.asciidoc ├── Setup.hs ├── antora-playbook.yml ├── arion-compose.cabal ├── bors.toml ├── build ├── cabal.project ├── default.nix ├── dev ├── flake-module.nix ├── flake.lock └── flake.nix ├── docs ├── README.md ├── antora.yml ├── flake-module.nix ├── modules │ └── ROOT │ │ ├── examples │ │ ├── full-nixos │ │ │ └── arion-compose.nix │ │ ├── minimal │ │ │ └── arion-compose.nix │ │ └── nixos-unit │ │ │ └── arion-compose.nix │ │ ├── nav.adoc │ │ └── pages │ │ ├── deployment.adoc │ │ ├── index.adoc │ │ └── options.adoc ├── options.nix └── ui-bundle.zip ├── examples ├── flake │ ├── arion-compose.nix │ ├── arion-pkgs.nix │ ├── flake.lock │ └── flake.nix ├── full-nixos │ ├── arion-compose.nix │ └── arion-pkgs.nix ├── minimal │ ├── arion-compose.nix │ └── arion-pkgs.nix ├── nixos-unit │ ├── arion-compose.nix │ └── arion-pkgs.nix ├── todomvc │ └── README.md └── traefik │ ├── arion-compose.nix │ └── arion-pkgs.nix ├── flake.lock ├── flake.nix ├── live-unit-tests ├── nix ├── arion.nix ├── compat.nix ├── haskell-arion-compose.nix └── upstreamable │ └── default.nix ├── nixos-module.nix ├── repl ├── run-arion ├── run-arion-quick ├── run-arion-via-nix ├── shell.nix ├── src ├── arion-image │ └── Dockerfile ├── haskell │ ├── exe │ │ └── Main.hs │ ├── lib │ │ └── Arion │ │ │ ├── Aeson.hs │ │ │ ├── DockerCompose.hs │ │ │ ├── ExtendedInfo.hs │ │ │ ├── Images.hs │ │ │ ├── Nix.hs │ │ │ └── Services.hs │ ├── test │ │ ├── Arion │ │ │ └── NixSpec.hs │ │ ├── Spec.hs │ │ └── TestMain.hs │ └── testdata │ │ ├── Arion │ │ └── NixSpec │ │ │ ├── arion-compose.json │ │ │ ├── arion-compose.nix │ │ │ ├── arion-context-compose.json │ │ │ ├── arion-context-compose.nix │ │ │ └── build-context │ │ │ └── Dockerfile │ │ └── docker-compose-example.json └── nix │ ├── eval-composition.nix │ ├── lib.nix │ ├── modules.nix │ └── modules │ ├── composition │ ├── composition.nix │ ├── docker-compose.nix │ ├── host-environment.nix │ ├── images.nix │ ├── networks.nix │ └── service-info.nix │ ├── lib │ ├── README.md │ ├── assert.nix │ └── assertions.nix │ ├── networks │ └── network.nix │ ├── nixos │ ├── container-systemd.nix │ └── default-shell.nix │ └── service │ ├── all-modules.nix │ ├── check-sys_admin.nix │ ├── context.nix │ ├── default-exec.nix │ ├── docker-compose-service.nix │ ├── extended-info.nix │ ├── host-store.nix │ ├── image-recommended.nix │ ├── image.nix │ ├── nixos-init.nix │ └── nixos.nix └── tests ├── arion-test └── default.nix ├── flake-module.nix └── nixos-virtualization-arion-test ├── README.md ├── arion-compose.nix ├── arion-pkgs.nix ├── kafka └── server.properties ├── test.nix └── zookeeper ├── log4j.properties └── zoo.cfg /.envrc: -------------------------------------------------------------------------------- 1 | use flake 2 | 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | result 2 | result-* 3 | 4 | dist/ 5 | dist-newstyle/ 6 | cabal.project.local 7 | 8 | *.swp 9 | 10 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Revision history for Arion 2 | 3 | ## 0.2.2.0 -- 2024-12-11 4 | 5 | ### Added 6 | 7 | * [`serivces..service.blkio_config`](https://docs.hercules-ci.com/arion/options#_services_name_service_blkio_config) 8 | * [`services..service.build.dockerfile`](https://docs.hercules-ci.com/arion/options#_services_name_service_build_dockerfile) 9 | * [`services..service.stop_grace_period`](https://docs.hercules-ci.com/arion/options#_services_name_service_stop_grace_period) 10 | * In the [NixOS deployment module], `virtualisation.arion.projects..serviceName` to override `` used in systemd unit 11 | 12 | ### Fixed 13 | 14 | * `nix repl` did not pass `--file` to Nix for file-based repls 15 | * `services..service.build.context` was ignored 16 | * `boot.tmpOnTmpfs` -> `boot.tmp.useTmpfs` 17 | * Remove lorri from local development environment, as it would fail silently 18 | 19 | ### Removed 20 | 21 | * `defaultPackage` flake output. Use `packages..default` instead. 22 | 23 | ## 0.2.1.0 -- 2023-07-26 24 | 25 | ### Added 26 | 27 | * `service.networks` now supports attribute set values with various options, thanks to @pedorich-n. 28 | * `docker-compose.volumes` can now be specified in multiple modules, thanks to @qaifshaikh. 29 | * `image.fakeRootCommands` for making modifications to the image that aren't "add a link farm". 30 | 31 | ### Fixed 32 | 33 | * Regular maintenance fixes, including one by olebedev 34 | 35 | 36 | ## 0.2.0.0 -- 2022-12-02 37 | 38 | ### BREAKING 39 | 40 | * The `project.name` option is now mandatory for projects that aren't deployed with the NixOS module. 41 | 42 | * The NixOS module now sets the default network name to the project name (commonly referred to as `` in the option path). 43 | If this is not desired, for instance if you need the projects to be on the same network, set `networks.default.name` in each of them. 44 | 45 | * The NixOS module now sets the default project name. You can still set your own value with the `project.name` option. 46 | If you did not set one, docker compose heuristically determined the name to be `store`, so you may want to set `project.name = "store"` or prepare to rename the network manually. 47 | 48 | ### Removed 49 | 50 | - NixOS 20.09 support. Its docker-compose does not support the 51 | `networks..name` option, which is important in later versions. 52 | A newer, bundled docker compose may work there, but for now the decision 53 | is to drop this legacy version. 54 | 55 | ### Changed 56 | 57 | * Healthcheck-based dependencies in `service.depends_on`. 58 | 59 | ### Added 60 | 61 | * Support `service.healthcheck` for defining custom healthchecks. 62 | * Arion now declares a `networks.default` by default, with `name` set to 63 | `project.name`. This improves compatibility with container runtimes by 64 | copying pre-existing behavior. Most users will want to keep using this 65 | behavior, but it can be disabled with `enableDefaultNetwork`. 66 | 67 | ## 0.1.3.0 -- 2020-05-03 68 | 69 | ### Changed 70 | 71 | * `useHostStore` now uses an image derived from the `image.*` options. You may 72 | need to enable `enableRecommendedContents` because with this change, files 73 | like `/bin/sh` aren't added by default anymore. 74 | 75 | * Drop obsolete NixOS 19.03, 19.09 and 20.03 from CI. 76 | 77 | ### Added 78 | 79 | * NixOS-based containers can now run on Podman when it is configured to provide a docker socket. See the [installation docs](https://docs.hercules-ci.com/arion/#_nixos). 80 | 81 | * Support `service.dns`, for overriding the DNS servers used by containers. 82 | 83 | * Support `service.labels`, which is useful for autodiscovery among other things. 84 | 85 | * Add a tested example for Traefik with label-based routing. 86 | 87 | * Add a `flake.nix` and an experimental flake example 88 | 89 | * Add a warning when systemd `DynamicUser` is used but not available to the 90 | container. 91 | 92 | * CI with NixOS 21.05 93 | 94 | ## 0.1.2.0 -- 2020-03-05 95 | 96 | * Support use of prebuilt `docker-compose.yaml`. 97 | Separates build and execution without duplicating evaluation. 98 | 99 | * Avoid storing tarballs (wasting store space) by using 100 | `dockerTools.streamLayeredImage` if available. 101 | 102 | * Project name is now configurable via the `project.name` option 103 | 104 | * Support --no-ansi, --compatibility, --log-level options 105 | 106 | ## 0.1.1.1 -- 2020-03-20 107 | 108 | * Fix ambiguous import of `lines` 109 | * Improve base version constraint 110 | * Fix warnings 111 | 112 | ## 0.1.1.0 -- 2020-03-19 113 | 114 | * Support Nixpkgs 20.03 115 | * Fixes for macOS 116 | 117 | ## 0.1.0.0 -- 2019-10-04 118 | 119 | * First released version. Released on an unsuspecting world. 120 | 121 | 122 | 123 | [NixOS deployment module]: https://docs.hercules-ci.com/arion/deployment#_nixos_module -------------------------------------------------------------------------------- /HACKING.md: -------------------------------------------------------------------------------- 1 | 2 | # Hacking on the modules 3 | 4 | ## Easiest option 5 | 6 | The module system does not distinguish between modules and configurations. 7 | This mean you can prototype any feature by factoring out functionality in a real-world project. 8 | 9 | ## Changing the built-in modules 10 | 11 | If your change is not just an addition or if it's better implemented by refactoring, you'll want to fork and edit arion sources directly. 12 | 13 | For a fast iteration cycle (but possibly outdated arion command logic): 14 | 15 | ~/src/arion/run-arion-quick up -d 16 | 17 | To update the arion command logic on the next run 18 | 19 | rm ~/src/arion/result-run-arion-quick 20 | 21 | 22 | # Hacking on the arion command 23 | 24 | The arion command is written in Haskell. Anyone can make small changes to the code. 25 | Experience with Haskell tooling is not required. You can use the nixified scripts in the root of the repo for common tasks. 26 | - `build` or `live-check` for typechecking 27 | - `live-unit-tests` (only the test suite is "live" though) 28 | - `repl` for a Haskell REPL 29 | - `run-arion` to run an incrementally built arion 30 | - `run-arion-via-nix` to run a nix-built arion 31 | - ~~`run-arion-quick`~~ *not for command hacking;* use stale command. See previous section. 32 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright 2019 Hercules Labs OÜ 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | -------------------------------------------------------------------------------- /README.asciidoc: -------------------------------------------------------------------------------- 1 | Arion is a tool for building and running applications that 2 | consist of multiple docker containers using NixOS modules. 3 | It has special support for docker images that are built with Nix, 4 | for a smooth development experience and improved performance. 5 | 6 | 7 | # https://docs.hercules-ci.com/arion/[Intro and Documentation] 8 | -------------------------------------------------------------------------------- /Setup.hs: -------------------------------------------------------------------------------- 1 | import Distribution.Simple 2 | 3 | main = defaultMain 4 | -------------------------------------------------------------------------------- /antora-playbook.yml: -------------------------------------------------------------------------------- 1 | # This is not used when the docs are imported into the docs site. 2 | # It is only here for development preview. 3 | content: 4 | sources: 5 | - url: . 6 | start_path: docs 7 | branches: HEAD 8 | ui: 9 | bundle: 10 | url: ./docs/ui-bundle.zip 11 | output: 12 | dir: ./public 13 | 14 | -------------------------------------------------------------------------------- /arion-compose.cabal: -------------------------------------------------------------------------------- 1 | cabal-version: 2.4 2 | 3 | name: arion-compose 4 | version: 0.2.2.0 5 | synopsis: Run docker-compose with help from Nix/NixOS 6 | description: Arion is a tool for building and running applications that consist of multiple docker containers using NixOS modules. It has special support for docker images that are built with Nix, for a smooth development experience and improved performance. 7 | homepage: https://github.com/hercules-ci/arion#readme 8 | -- bug-reports: 9 | license: Apache-2.0 10 | license-file: LICENSE 11 | author: Robert Hensing 12 | maintainer: robert@hercules-ci.com 13 | -- copyright: 14 | category: Distribution, Network, Cloud, Distributed Computing 15 | extra-source-files: CHANGELOG.md, README.asciidoc, 16 | src/haskell/testdata/**/*.nix 17 | src/haskell/testdata/**/*.json 18 | data-files: nix/*.nix 19 | , nix/modules/composition/*.nix 20 | , nix/modules/networks/*.nix 21 | , nix/modules/nixos/*.nix 22 | , nix/modules/service/*.nix 23 | , nix/modules/lib/*.nix 24 | 25 | -- all data is verbatim from some sources 26 | data-dir: src 27 | 28 | source-repository head 29 | type: git 30 | location: https://github.com/hercules-ci/arion 31 | 32 | common common 33 | build-depends: base >=4.12.0.0 && <4.99 34 | , aeson >=2 35 | , aeson-pretty 36 | , async 37 | , bytestring 38 | , directory 39 | , lens 40 | , lens-aeson 41 | , process 42 | , temporary 43 | , text 44 | , protolude >= 0.2 45 | , unix 46 | ghc-options: -Wall 47 | default-extensions: 48 | BlockArguments 49 | DeriveAnyClass 50 | DeriveGeneric 51 | DerivingStrategies 52 | LambdaCase 53 | NoFieldSelectors 54 | OverloadedRecordDot 55 | OverloadedStrings 56 | 57 | flag ghci 58 | default: False 59 | manual: True 60 | 61 | library 62 | import: common 63 | exposed-modules: Arion.Nix 64 | Arion.Aeson 65 | Arion.DockerCompose 66 | Arion.ExtendedInfo 67 | Arion.Images 68 | Arion.Services 69 | other-modules: Paths_arion_compose 70 | autogen-modules: Paths_arion_compose 71 | -- other-extensions: 72 | hs-source-dirs: src/haskell/lib 73 | default-language: Haskell2010 74 | 75 | executable arion 76 | import: common 77 | main-is: Main.hs 78 | -- other-modules: 79 | -- other-extensions: 80 | build-depends: optparse-applicative 81 | , arion-compose 82 | hs-source-dirs: src/haskell/exe 83 | default-language: Haskell2010 84 | 85 | test-suite arion-unit-tests 86 | import: common 87 | if flag(ghci) 88 | hs-source-dirs: src/haskell/lib 89 | ghc-options: -Wno-missing-home-modules 90 | type: exitcode-stdio-1.0 91 | main-is: TestMain.hs 92 | other-modules: Spec 93 | , Arion.NixSpec 94 | -- other-extensions: 95 | build-depends: arion-compose 96 | , hspec 97 | , QuickCheck 98 | hs-source-dirs: src/haskell/test 99 | default-language: Haskell2010 100 | -------------------------------------------------------------------------------- /bors.toml: -------------------------------------------------------------------------------- 1 | status = [ 2 | "ci/hercules/onPush/default", 3 | "ci/hercules/evaluation", 4 | ] 5 | delete_merged_branches = true 6 | -------------------------------------------------------------------------------- /build: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env nix-shell 2 | #!nix-shell -i bash 3 | 4 | # Build the Haskell package via cabal, outside Nix 5 | 6 | cabal new-build --write-ghc-environment-files=never 7 | -------------------------------------------------------------------------------- /cabal.project: -------------------------------------------------------------------------------- 1 | packages: . -------------------------------------------------------------------------------- /default.nix: -------------------------------------------------------------------------------- 1 | let flake = import ./nix/compat.nix; 2 | in 3 | { pkgs ? import flake.inputs.nixpkgs { } 4 | , haskellPackages ? pkgs.haskellPackages 5 | }: 6 | let 7 | pkgsWithArion = pkgs.extend flake.overlays.default; 8 | in 9 | { 10 | inherit (pkgsWithArion) arion; 11 | } 12 | -------------------------------------------------------------------------------- /dev/flake-module.nix: -------------------------------------------------------------------------------- 1 | { 2 | perSystem = { config, self', inputs', pkgs, system, final, ... }: { 3 | pre-commit.settings.hooks.ormolu.enable = true; 4 | devShells.default = config.devShells.haskell-package.overrideAttrs (o: { 5 | nativeBuildInputs = o.nativeBuildInputs or [ ] ++ [ 6 | pkgs.docker-compose 7 | pkgs.nixpkgs-fmt 8 | config.haskellProjects.haskell-package.haskellPackages.releaser 9 | ]; 10 | shellHook = '' 11 | ${config.pre-commit.installationScript} 12 | echo 1>&2 "Welcome to the arion dev shell" 13 | ''; 14 | }); 15 | }; 16 | 17 | hercules-ci.flake-update = { 18 | enable = true; 19 | autoMergeMethod = "merge"; 20 | flakes = { 21 | "." = { }; 22 | "dev" = { }; 23 | }; 24 | when = { 25 | hour = [ 2 ]; 26 | dayOfMonth = [ 5 ]; 27 | }; 28 | }; 29 | 30 | herculesCI.ciSystems = [ 31 | "aarch64-darwin" 32 | # "aarch64-linux" 33 | # "x86_64-darwin" 34 | "x86_64-linux" 35 | ]; 36 | 37 | } 38 | -------------------------------------------------------------------------------- /dev/flake.lock: -------------------------------------------------------------------------------- 1 | { 2 | "nodes": { 3 | "flake-compat": { 4 | "flake": false, 5 | "locked": { 6 | "lastModified": 1696426674, 7 | "narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=", 8 | "owner": "edolstra", 9 | "repo": "flake-compat", 10 | "rev": "0f9255e01c2351cc7d116c072cb317785dd33b33", 11 | "type": "github" 12 | }, 13 | "original": { 14 | "owner": "edolstra", 15 | "repo": "flake-compat", 16 | "type": "github" 17 | } 18 | }, 19 | "flake-parts": { 20 | "inputs": { 21 | "nixpkgs-lib": [ 22 | "hercules-ci-effects", 23 | "nixpkgs" 24 | ] 25 | }, 26 | "locked": { 27 | "lastModified": 1733312601, 28 | "narHash": "sha256-4pDvzqnegAfRkPwO3wmwBhVi/Sye1mzps0zHWYnP88c=", 29 | "owner": "hercules-ci", 30 | "repo": "flake-parts", 31 | "rev": "205b12d8b7cd4802fbcb8e8ef6a0f1408781a4f9", 32 | "type": "github" 33 | }, 34 | "original": { 35 | "id": "flake-parts", 36 | "type": "indirect" 37 | } 38 | }, 39 | "git-hooks-nix": { 40 | "inputs": { 41 | "flake-compat": "flake-compat", 42 | "gitignore": "gitignore", 43 | "nixpkgs": "nixpkgs", 44 | "nixpkgs-stable": "nixpkgs-stable" 45 | }, 46 | "locked": { 47 | "lastModified": 1733665616, 48 | "narHash": "sha256-+XTFXYlFJBxohhMGLDpYdEnhUNdxN8dyTA8WAd+lh2A=", 49 | "owner": "cachix", 50 | "repo": "git-hooks.nix", 51 | "rev": "d8c02f0ffef0ef39f6063731fc539d8c71eb463a", 52 | "type": "github" 53 | }, 54 | "original": { 55 | "owner": "cachix", 56 | "repo": "git-hooks.nix", 57 | "type": "github" 58 | } 59 | }, 60 | "gitignore": { 61 | "inputs": { 62 | "nixpkgs": [ 63 | "git-hooks-nix", 64 | "nixpkgs" 65 | ] 66 | }, 67 | "locked": { 68 | "lastModified": 1709087332, 69 | "narHash": "sha256-HG2cCnktfHsKV0s4XW83gU3F57gaTljL9KNSuG6bnQs=", 70 | "owner": "hercules-ci", 71 | "repo": "gitignore.nix", 72 | "rev": "637db329424fd7e46cf4185293b9cc8c88c95394", 73 | "type": "github" 74 | }, 75 | "original": { 76 | "owner": "hercules-ci", 77 | "repo": "gitignore.nix", 78 | "type": "github" 79 | } 80 | }, 81 | "hercules-ci-effects": { 82 | "inputs": { 83 | "flake-parts": "flake-parts", 84 | "nixpkgs": "nixpkgs_2" 85 | }, 86 | "locked": { 87 | "lastModified": 1733333617, 88 | "narHash": "sha256-nMMQXREGvLOLvUa0ByhYFdaL0Jov0t1wzLbKjr05P2w=", 89 | "owner": "hercules-ci", 90 | "repo": "hercules-ci-effects", 91 | "rev": "56f8ea8d502c87cf62444bec4ee04512e8ea24ea", 92 | "type": "github" 93 | }, 94 | "original": { 95 | "owner": "hercules-ci", 96 | "repo": "hercules-ci-effects", 97 | "type": "github" 98 | } 99 | }, 100 | "nixpkgs": { 101 | "locked": { 102 | "lastModified": 1730768919, 103 | "narHash": "sha256-8AKquNnnSaJRXZxc5YmF/WfmxiHX6MMZZasRP6RRQkE=", 104 | "owner": "NixOS", 105 | "repo": "nixpkgs", 106 | "rev": "a04d33c0c3f1a59a2c1cb0c6e34cd24500e5a1dc", 107 | "type": "github" 108 | }, 109 | "original": { 110 | "owner": "NixOS", 111 | "ref": "nixpkgs-unstable", 112 | "repo": "nixpkgs", 113 | "type": "github" 114 | } 115 | }, 116 | "nixpkgs-stable": { 117 | "locked": { 118 | "lastModified": 1730741070, 119 | "narHash": "sha256-edm8WG19kWozJ/GqyYx2VjW99EdhjKwbY3ZwdlPAAlo=", 120 | "owner": "NixOS", 121 | "repo": "nixpkgs", 122 | "rev": "d063c1dd113c91ab27959ba540c0d9753409edf3", 123 | "type": "github" 124 | }, 125 | "original": { 126 | "owner": "NixOS", 127 | "ref": "nixos-24.05", 128 | "repo": "nixpkgs", 129 | "type": "github" 130 | } 131 | }, 132 | "nixpkgs_2": { 133 | "locked": { 134 | "lastModified": 1733212471, 135 | "narHash": "sha256-M1+uCoV5igihRfcUKrr1riygbe73/dzNnzPsmaLCmpo=", 136 | "owner": "NixOS", 137 | "repo": "nixpkgs", 138 | "rev": "55d15ad12a74eb7d4646254e13638ad0c4128776", 139 | "type": "github" 140 | }, 141 | "original": { 142 | "owner": "NixOS", 143 | "ref": "nixos-unstable", 144 | "repo": "nixpkgs", 145 | "type": "github" 146 | } 147 | }, 148 | "root": { 149 | "inputs": { 150 | "git-hooks-nix": "git-hooks-nix", 151 | "hercules-ci-effects": "hercules-ci-effects" 152 | } 153 | } 154 | }, 155 | "root": "root", 156 | "version": 7 157 | } 158 | -------------------------------------------------------------------------------- /dev/flake.nix: -------------------------------------------------------------------------------- 1 | { 2 | # TODO move back into main lock after https://github.com/NixOS/nix/issues/7730, remove this subflake 3 | description = "Development dependencies in a separate subflake so that they don't end up in your lock"; 4 | inputs = { 5 | hercules-ci-effects.url = "github:hercules-ci/hercules-ci-effects"; 6 | git-hooks-nix.url = "github:cachix/git-hooks.nix"; 7 | }; 8 | outputs = { ... }: { }; 9 | } 10 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | 2 | # Documentation 3 | 4 | Please refer to the [**rendered documentation**](https://docs.hercules-ci.com/arion), which includes the [**options.**](https://docs.hercules-ci.com/arion/options/) 5 | -------------------------------------------------------------------------------- /docs/antora.yml: -------------------------------------------------------------------------------- 1 | name: arion 2 | title: Arion Documentation 3 | version: 'master' 4 | nav: 5 | - modules/ROOT/nav.adoc 6 | - modules/reference/nav.adoc 7 | nix: true 8 | -------------------------------------------------------------------------------- /docs/flake-module.nix: -------------------------------------------------------------------------------- 1 | { 2 | perSystem = { config, pkgs, lib, ... }: { 3 | packages.generated-option-doc-arion = 4 | # TODO: use the render pipeline in flake-parts, 5 | # which has support for things like {options}`foo`. 6 | let 7 | eval = lib.evalModules { 8 | modules = import ../src/nix/modules.nix; 9 | }; 10 | in 11 | (pkgs.nixosOptionsDoc 12 | { 13 | options = eval.options; 14 | }).optionsCommonMark; 15 | 16 | packages.generated-antora-files = 17 | pkgs.runCommand "generated-antora-files" 18 | { 19 | nativeBuildInputs = [ pkgs.pandoc ]; 20 | doc_arion = config.packages.generated-option-doc-arion; 21 | } 22 | # TODO: use the render pipeline in flake-parts, 23 | # which has support for things like {options}`foo`. 24 | '' 25 | mkdir -p $out/modules/ROOT/partials 26 | pandoc --from=markdown --to=asciidoc \ 27 | < $doc_arion \ 28 | > $out/modules/ROOT/partials/arion-options.adoc 29 | ''; 30 | }; 31 | } 32 | -------------------------------------------------------------------------------- /docs/modules/ROOT/examples/full-nixos/arion-compose.nix: -------------------------------------------------------------------------------- 1 | ../../../../../examples/full-nixos/arion-compose.nix -------------------------------------------------------------------------------- /docs/modules/ROOT/examples/minimal/arion-compose.nix: -------------------------------------------------------------------------------- 1 | ../../../../../examples/minimal/arion-compose.nix -------------------------------------------------------------------------------- /docs/modules/ROOT/examples/nixos-unit/arion-compose.nix: -------------------------------------------------------------------------------- 1 | ../../../../../examples/nixos-unit/arion-compose.nix -------------------------------------------------------------------------------- /docs/modules/ROOT/nav.adoc: -------------------------------------------------------------------------------- 1 | * xref:index.adoc[Getting Started] 2 | * xref:options.adoc[Arion Options] 3 | * xref:deployment.adoc[Deployment] 4 | -------------------------------------------------------------------------------- /docs/modules/ROOT/pages/deployment.adoc: -------------------------------------------------------------------------------- 1 | = Deployment with Arion 2 | 3 | Arion projects can be deployed in Nix-like or Docker-like ways. 4 | 5 | == Docker images 6 | 7 | When you disable `useHostStore`, arion will build images, which can be deployed 8 | to any Docker host, including non-NixOS hosts. 9 | 10 | === Remote Docker socket 11 | 12 | NOTE: Access to a Docker socket is equivalent to root access on the host. 13 | 14 | Docker supports authentication via TLS client certificates. 15 | 16 | The xref:hercules-ci-effects:ROOT:reference/nix-functions/runArion.adoc[runArion Effect] uses this technique. 17 | 18 | Because this technique works with a single Docker host, it does not need a registry. 19 | 20 | === Upload to registry 21 | 22 | You can either use `arion push` or write custom push logic using the `arion cat` 23 | command, the `eval` function on the `arion` package, or the `lib.eval` function 24 | on the flake to retrieve the images defined in a project. 25 | 26 | == NixOS module 27 | 28 | Arion projects can be deployed as part of a NixOS configuration. This ties the 29 | project revision to the system configuration revision, which can be good or bad 30 | thing, depending on your deployment strategy. At a low level, a benefit is that 31 | no store paths need to be copied locally and remote NixOS deployments can use 32 | Nix's copy-closure algorithm for efficient transfers, and transparent binary 33 | caches rather than an inherently stateful Docker registry solution. 34 | 35 | Extend your NixOS configuration by adding the configuration elements to an 36 | existing configuration. You could create a new module file for it, if your 37 | choice of `imports` allows it. 38 | 39 | NOTE: This deployment method does NOT use an `arion-pkgs.nix` file, but reuses 40 | the host `pkgs`. 41 | 42 | ```nix 43 | { 44 | imports = [ 45 | # Pick one of: 46 | # - niv 47 | ((import ./nix/sources.nix).arion + "/nixos-module.nix") 48 | # - or flakes (where arion is a flake input) 49 | arion.nixosModules.arion 50 | # - or other: copy commit hash of arion and replace HASH in: 51 | (builtins.fetchTarball "https://github.com/hercules-ci/arion/archive/HASH.tar.gz" + "/nixos-module.nix") 52 | ]; 53 | 54 | virtualisation.arion = { 55 | backend = "podman-socket"; # or "docker" 56 | projects.example = { 57 | serviceName = "example"; # optional systemd service name, defaults to arion-example in this case 58 | settings = { 59 | # Specify you project here, or import it from a file. 60 | # NOTE: This does NOT use ./arion-pkgs.nix, but defaults to NixOS' pkgs. 61 | imports = [ ./arion-compose.nix ]; 62 | }; 63 | }; 64 | }; 65 | } 66 | ``` 67 | 68 | See also: 69 | 70 | - xref:hercules-ci-effects:ROOT:reference/nix-functions/runNixOS.adoc[runNixOS Effect] 71 | - xref:hercules-ci-effects:ROOT:reference/nix-functions/runNixOps2.adoc[runNixOps2 Effect] 72 | -------------------------------------------------------------------------------- /docs/modules/ROOT/pages/index.adoc: -------------------------------------------------------------------------------- 1 | = Welcome to Arion documentation 2 | 3 | == Introduction 4 | 5 | Arion is a tool for building and running applications that 6 | consist of multiple docker containers using NixOS modules. 7 | It has special support for docker images that are built with Nix, 8 | for a smooth development experience and improved performance. 9 | 10 | It is built on top of https://docs.docker.com/compose/overview/[Docker 11 | Compose], which implements the container orchestration functionality. 12 | 13 | Instead of configuring the compositions in YAML files like 14 | `docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix] 15 | language to declare the compositions. Because of this, Arion gives you 16 | the ability to declare your deployments, configuration and packaging 17 | in the same language. By replacing multiple tools with a single 18 | language, you decrease your mental load and you can more easily 19 | refactor and maintain your configurations. 20 | 21 | Although Arion can be used as a Docker Compose with an improved 22 | configuration front end, there is more to be gained from integrating 23 | with Nix. In particular, the more structured approach of Nix compared 24 | to Dockerfiles allows the following: 25 | 26 | * Build components of your image in *parallel, automatically* 27 | * *Share packages between images*, regardless of the order they were 28 | added 29 | * Improve performance by *skipping container 30 | image creation* 31 | * Work with *structured data instead of strings*, 32 | templates and a multitude of expression languages 33 | * Refactor across deployments, configuration and packaging 34 | 35 | Arion allows to compose containers with different granularity: 36 | 37 | * <> 38 | * <> 39 | * <> 40 | * <> 41 | 42 | Full NixOS is supported on 43 | 44 | * docker-compose + podman with docker socket (NixOS >= 21.05) 45 | * docker-compose + docker, before cgroupsv2 (NixOS < 21.05) 46 | 47 | `podman-compose` support is currently WIP on a separate branch. 48 | 49 | == Installation 50 | 51 | === Nix 52 | 53 | ```bash 54 | $ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master 55 | ``` 56 | 57 | === NixOS 58 | 59 | Add this module to your NixOS configuration: 60 | 61 | ```nix 62 | { pkgs, ... }: { 63 | environment.systemPackages = [ 64 | pkgs.arion 65 | 66 | # Do install the docker CLI to talk to podman. 67 | # Not needed when virtualisation.docker.enable = true; 68 | pkgs.docker-client 69 | ]; 70 | 71 | # Arion works with Docker, but for NixOS-based containers, you need Podman 72 | # since NixOS 21.05. 73 | virtualisation.docker.enable = false; 74 | virtualisation.podman.enable = true; 75 | virtualisation.podman.dockerSocket.enable = true; 76 | virtualisation.podman.defaultNetwork.dnsname.enable = true; 77 | 78 | # Use your username instead of `myuser` 79 | users.extraUsers.myuser.extraGroups = ["podman"]; 80 | } 81 | ``` 82 | 83 | //// 84 | 85 | == Not installing: use it in a project 86 | 87 | TODO: describe: using nix-shell or in a script, building images as 88 | part of nix-build, pinning, see also todomvc-nix. 89 | 90 | TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD) 91 | 92 | 93 | //// 94 | 95 | 96 | == Usage 97 | 98 | Arion is configured declaratively with two files: 99 | 100 | === arion-pkgs.nix 101 | 102 | Arion needs `arion-pkgs.nix` to import nixpkgs, for example: 103 | 104 | ```nix 105 | import { system = "x86_64-linux"; } 106 | ``` 107 | 108 | or more sophisticated (recommended) setup with https://github.com/nmattia/niv[Niv]. 109 | 110 | === arion-compose.nix 111 | 112 | Describe containers using NixOS-style modules. There are a few options: 113 | 114 | ==== Minimal: Plain command using nixpkgs 115 | 116 | `examples/minimal/arion-compose.nix` 117 | [,nix] 118 | ---- 119 | { pkgs, ... }: 120 | { 121 | project.name = "webapp"; 122 | services = { 123 | 124 | webserver = { 125 | image.enableRecommendedContents = true; 126 | service.useHostStore = true; 127 | service.command = [ "sh" "-c" '' 128 | cd "$$WEB_ROOT" 129 | ${pkgs.python3}/bin/python -m http.server 130 | '' ]; 131 | service.ports = [ 132 | "8000:8000" # host:container 133 | ]; 134 | service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual"; 135 | service.stop_signal = "SIGINT"; 136 | }; 137 | }; 138 | } 139 | ---- 140 | 141 | ==== NixOS: run full OS 142 | 143 | `examples/full-nixos/arion-compose.nix`: 144 | 145 | [,nix] 146 | ---- 147 | { 148 | project.name = "full-nixos"; 149 | services.webserver = { pkgs, lib, ... }: { 150 | nixos.useSystemd = true; 151 | nixos.configuration.boot.tmp.useTmpfs = true; 152 | nixos.configuration.services.nginx.enable = true; 153 | nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual"; 154 | nixos.configuration.services.nscd.enable = false; 155 | nixos.configuration.system.nssModules = lib.mkForce []; 156 | nixos.configuration.systemd.services.nginx.serviceConfig.AmbientCapabilities = 157 | lib.mkForce [ "CAP_NET_BIND_SERVICE" ]; 158 | service.useHostStore = true; 159 | service.ports = [ 160 | "8000:80" # host:container 161 | ]; 162 | }; 163 | } 164 | ---- 165 | 166 | ==== Docker image from DockerHub 167 | 168 | ```nix 169 | { 170 | services.postgres = { 171 | service.image = "postgres:10"; 172 | service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ]; 173 | service.environment.POSTGRES_PASSWORD = "mydefaultpass"; 174 | }; 175 | } 176 | ``` 177 | 178 | ==== NixOS: run only one systemd service 179 | 180 | Running individual units from NixOS is possible using an experimental script. 181 | See `examples/nixos-unit/arion-compose.nix`. 182 | 183 | === Run 184 | 185 | Start containers and watch their logs: 186 | 187 | ```bash 188 | $ arion up -d 189 | $ arion logs -f 190 | ``` 191 | 192 | You can go to `examples/*/` and run these commands to give it a quick try. 193 | 194 | === Inspect the config 195 | 196 | While developing an arion project, you can make use of `arion repl`, which launches 197 | a `nix repl` on the project configuration. 198 | 199 | ``` 200 | $ arion repl 201 | Launching a repl for you. To get started: 202 | 203 | To see deployment-wide configuration 204 | type config. and use tab completion 205 | To bring the top-level Nixpkgs attributes into scope 206 | type :a (config._module.args.pkgs) // { inherit config; } 207 | 208 | Welcome to Nix. Type :? for help. 209 | 210 | Loading '../../src/nix/eval-composition.nix'... 211 | Added 5 variables. 212 | 213 | nix-repl> config.services.webserver.service.command 214 | [ "sh" "-c" "cd \"$$WEB_ROOT\"\n/nix/store/66fbv9mmx1j4hrn9y06kcp73c3yb196r-python3-3.8.9/bin/python -m http.server\n" ] 215 | 216 | nix-repl> 217 | 218 | ``` 219 | 220 | == Build with Nix 221 | 222 | You can build a project with `nix-build` using an expression like 223 | 224 | ```nix 225 | arion.build { modules = [ ./arion-compose.nix ]; pkgs = import ./arion-pkgs.nix; } 226 | ``` 227 | 228 | If you deploy with xref:hercules-ci-effects:ROOT:reference/nix-functions/runArion.adoc[runArion], 229 | and your `pkgs` variable is equivalent to `import ./arion-pkgs.nix`, you can use: 230 | 231 | ```nix 232 | let 233 | deployment = pkgs.effects.runArion { /* ... */ }); 234 | in deployment.prebuilt 235 | ``` 236 | 237 | == Project Status 238 | 239 | This project was born out of a process supervision need for local 240 | development environments while working on 241 | https://www.hercules-ci.com[Hercules CI]. (It was also born out of 242 | ancient Greek deities disguised as horses. More on that later.) 243 | 244 | Arion can be used for simple single host deployments, using Docker's TLS 245 | client verification, or https://search.nixos.org/options?channel=unstable&show=virtualisation.podman.networkSocket.enable&query=virtualisation.podman[`virtualisation.podman.networkSocket` options]. 246 | Remote deployments do not support `useHostStore`, although an SSH-based deployment method could support this. 247 | Docker Swarm is not currently supported. 248 | 249 | Arion has run successfully on Linux distributions other than NixOS, but we only perform CI for Arion on NixOS. 250 | 251 | 252 | == How it works 253 | 254 | Arion is essentially a thin wrapper around Nix and docker-compose. When 255 | it runs, it does the following: 256 | 257 | * Evaluate the configuration using Nix, producing a 258 | `docker-compose.yaml` and a garbage collection root 259 | * Invoke `docker-compose` 260 | * Clean up the garbage collection root 261 | 262 | Most of the interesting stuff happens in Arion’s Nix expressions, where 263 | it runs the module system (known from NixOS) and provides the 264 | configuration that makes the Docker Compose file do the things it needs 265 | to do. 266 | 267 | One of the more interesting built-in modules is the 268 | https://github.com/hercules-ci/arion/blob/master/src/nix/modules/service/host-store.nix[host-store.nix module] which 269 | performs the bind mounts to make the host Nix store available in the 270 | container. 271 | 272 | == FAQ 273 | 274 | === Do I need to use Hercules CI? 275 | 276 | Nope, it’s just Nix and Docker Compose under the hood. 277 | 278 | It does xref:hercules-ci-effects:ROOT:reference/nix-functions/runArion.adoc[integrate] nicely though. 279 | 280 | === What about garbage collection? 281 | 282 | Arion removes the need for garbage collecting docker images, delegating 283 | this task to Nix when using `service.useHostStore`. 284 | 285 | Arion creates a garbage collection root that it cleans up after completing 286 | the command. This means that `arion up -d` should not be used with `useHostStore` 287 | in production. Instead, disable `useHostStore`, which will use `dockerTools` to 288 | generate images that can be used in production. 289 | 290 | === Why is my container not running latest code? 291 | 292 | Rebuild the image using `arion up -d --always-recreate-deps ` or simply `arion up -d`. 293 | 294 | Like `docker-compose restart`, `arion restart` does not update the image before starting. 295 | 296 | === What is messing with my environment variables? 297 | 298 | Docker Compose performs its own environment variable substitution. This 299 | can be a little annoying in `services.command` for example. Either 300 | reference a script from `pkgs.writeScript` or escape the dollar sign as 301 | `$$`. 302 | 303 | === Why name it ``Arion``? 304 | 305 | Arion comes from Greek mythology. Poseidon, the god of Docker -- I mean the seas -- 306 | had his eye on Demeter. Demeter tried to trick him by disguising as a 307 | horse, but Poseidon saw through the deception and they had Arion. 308 | 309 | So Arion is a super fast divine horse; the result of some weird mixing. 310 | Also it talks. 311 | 312 | (And we felt morally obliged to name our stuff after Greek mythology) 313 | -------------------------------------------------------------------------------- /docs/modules/ROOT/pages/options.adoc: -------------------------------------------------------------------------------- 1 | # Arion Options 2 | 3 | include::partial$arion-options.adoc[] 4 | -------------------------------------------------------------------------------- /docs/options.nix: -------------------------------------------------------------------------------- 1 | { pkgs ? import ../nix {} }: 2 | 3 | let 4 | eval = pkgs.lib.evalModules { 5 | modules = import ../src/nix/modules.nix; 6 | }; 7 | options = pkgs.nixosOptionsDoc { 8 | options = eval.options; 9 | }; 10 | 11 | in (pkgs.runCommand "agent-options.adoc" { } '' 12 | cat >$out <>$out 17 | '').overrideAttrs (o: { 18 | # Work around https://github.com/hercules-ci/hercules-ci-agent/issues/168 19 | allowSubstitutes = true; 20 | }) 21 | -------------------------------------------------------------------------------- /docs/ui-bundle.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hercules-ci/arion/4f59059633b14364b994503b179a701f5e6cfb90/docs/ui-bundle.zip -------------------------------------------------------------------------------- /examples/flake/arion-compose.nix: -------------------------------------------------------------------------------- 1 | { pkgs, ... }: 2 | let 3 | sh = pkgs.stdenv.mkDerivation { 4 | name = "sh"; 5 | phases = [ "installPhase" ]; 6 | 7 | installPhase = '' 8 | mkdir -p "$out"/bin 9 | ln -s ${pkgs.bash}/bin/sh "$out"/bin/sh 10 | ''; 11 | }; 12 | in{ 13 | config.project.name = "webapp"; 14 | config.services = { 15 | 16 | webserver = { 17 | image.contents = [ sh ]; 18 | service.useHostStore = true; 19 | service.command = [ "sh" "-c" '' 20 | cd "$$WEB_ROOT" 21 | ${pkgs.python3}/bin/python -m http.server 22 | '' ]; 23 | service.ports = [ 24 | "8000:8000" # host:container 25 | ]; 26 | service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual"; 27 | service.stop_signal = "SIGINT"; 28 | }; 29 | }; 30 | } 31 | -------------------------------------------------------------------------------- /examples/flake/arion-pkgs.nix: -------------------------------------------------------------------------------- 1 | let 2 | flake = if builtins ? getFlake 3 | then (builtins.getFlake (toString ./.)).pkgs 4 | else (import flake-compat { src = ./.; }).defaultNix; 5 | # NB: this is lazy 6 | lock = builtins.fromJSON (builtins.readFile ./flake.lock); 7 | inherit (lock.nodes.flake-compat.locked) owner repo rev narHash; 8 | flake-compat = builtins.fetchTarball { 9 | url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz"; 10 | sha256 = narHash; 11 | }; 12 | in 13 | flake.pkgs 14 | -------------------------------------------------------------------------------- /examples/flake/flake.lock: -------------------------------------------------------------------------------- 1 | { 2 | "nodes": { 3 | "flake-compat": { 4 | "flake": false, 5 | "locked": { 6 | "lastModified": 1606424373, 7 | "narHash": "sha256-oq8d4//CJOrVj+EcOaSXvMebvuTkmBJuT5tzlfewUnQ=", 8 | "owner": "edolstra", 9 | "repo": "flake-compat", 10 | "rev": "99f1c2157fba4bfe6211a321fd0ee43199025dbf", 11 | "type": "github" 12 | }, 13 | "original": { 14 | "owner": "edolstra", 15 | "repo": "flake-compat", 16 | "type": "github" 17 | } 18 | }, 19 | "nixpkgs": { 20 | "locked": { 21 | "lastModified": 1618853290, 22 | "narHash": "sha256-K4fddnrGOcKL+6CEchRrVmepiwvwvHxB87goqBTI5Bs=", 23 | "owner": "NixOS", 24 | "repo": "nixpkgs", 25 | "rev": "9a1672105db0eebe8ef59f310397435f2d0298d0", 26 | "type": "github" 27 | }, 28 | "original": { 29 | "owner": "NixOS", 30 | "ref": "nixos-20.09", 31 | "repo": "nixpkgs", 32 | "type": "github" 33 | } 34 | }, 35 | "root": { 36 | "inputs": { 37 | "flake-compat": "flake-compat", 38 | "nixpkgs": "nixpkgs" 39 | } 40 | } 41 | }, 42 | "root": "root", 43 | "version": 7 44 | } 45 | -------------------------------------------------------------------------------- /examples/flake/flake.nix: -------------------------------------------------------------------------------- 1 | { 2 | description = "A very basic flake"; 3 | 4 | inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-20.09"; 5 | inputs.flake-compat.url = "github:edolstra/flake-compat"; 6 | inputs.flake-compat.flake = false; 7 | 8 | outputs = { self, nixpkgs, ... }: { 9 | 10 | pkgs = nixpkgs.legacyPackages.x86_64-linux; 11 | # # alternative: 12 | # pkgs = import nixpkgs { config = { }; overlays = [ ]; system = "x86_64-linux"; }; 13 | 14 | packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello; 15 | 16 | defaultPackage.x86_64-linux = self.packages.x86_64-linux.hello; 17 | 18 | }; 19 | } 20 | -------------------------------------------------------------------------------- /examples/full-nixos/arion-compose.nix: -------------------------------------------------------------------------------- 1 | { 2 | project.name = "full-nixos"; 3 | services.webserver = { pkgs, lib, ... }: { 4 | nixos.useSystemd = true; 5 | nixos.configuration.boot.tmp.useTmpfs = true; 6 | nixos.configuration.networking.useDHCP = false; 7 | nixos.configuration.services.nginx.enable = true; 8 | nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual"; 9 | nixos.configuration.services.nscd.enable = false; 10 | nixos.configuration.system.nssModules = lib.mkForce []; 11 | nixos.configuration.systemd.services.nginx.serviceConfig.AmbientCapabilities = 12 | lib.mkForce [ "CAP_NET_BIND_SERVICE" ]; 13 | service.useHostStore = true; 14 | service.ports = [ 15 | "8000:80" # host:container 16 | ]; 17 | }; 18 | } 19 | -------------------------------------------------------------------------------- /examples/full-nixos/arion-pkgs.nix: -------------------------------------------------------------------------------- 1 | # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH 2 | import { 3 | # We specify the architecture explicitly. Use a Linux remote builder when 4 | # calling arion from other platforms. 5 | system = "x86_64-linux"; 6 | } 7 | -------------------------------------------------------------------------------- /examples/minimal/arion-compose.nix: -------------------------------------------------------------------------------- 1 | { pkgs, ... }: 2 | { 3 | project.name = "webapp"; 4 | services = { 5 | 6 | webserver = { 7 | image.enableRecommendedContents = true; 8 | service.useHostStore = true; 9 | service.command = [ "sh" "-c" '' 10 | cd "$$WEB_ROOT" 11 | ${pkgs.python3}/bin/python -m http.server 12 | '' ]; 13 | service.ports = [ 14 | "8000:8000" # host:container 15 | ]; 16 | service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual"; 17 | service.stop_signal = "SIGINT"; 18 | }; 19 | }; 20 | } 21 | -------------------------------------------------------------------------------- /examples/minimal/arion-pkgs.nix: -------------------------------------------------------------------------------- 1 | # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH 2 | import { 3 | # We specify the architecture explicitly. Use a Linux remote builder when 4 | # calling arion from other platforms. 5 | system = "x86_64-linux"; 6 | } 7 | -------------------------------------------------------------------------------- /examples/nixos-unit/arion-compose.nix: -------------------------------------------------------------------------------- 1 | 2 | /* 3 | 4 | DISCLAIMER 5 | 6 | This uses a somewhat hidden feature in NixOS, which is the 7 | "runner". It's a script that's available on systemd services 8 | that lets you run the service independently from systemd. 9 | However, it was clearly not intended for public consumption 10 | so please use it with care. 11 | It does not support all features of systemd so you are on 12 | your own if you use it in production. 13 | 14 | One known issue is that the script does not respond to docker's 15 | SIGTERM shutdown signal. 16 | 17 | */ 18 | 19 | { 20 | project.name = "nixos-unit"; 21 | services.webserver = { config, pkgs, ... }: { 22 | 23 | nixos.configuration = {config, lib, options, pkgs, ...}: { 24 | boot.isContainer = true; 25 | services.nginx = { 26 | enable = true; 27 | virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual"; 28 | } // lib.optionalAttrs (options?services.nginx.stateDir) { 29 | # Work around a problem in NixOS 20.03 30 | stateDir = "/var/lib/nginx"; 31 | }; 32 | system.build.run-nginx = pkgs.writeScript "run-nginx" '' 33 | #!${pkgs.bash}/bin/bash 34 | PATH='${config.systemd.services.nginx.environment.PATH}' 35 | echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd 36 | echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group 37 | echo 'nobody:x:65534:65534:Unprivileged account do not use:/var/empty:/run/current-system/sw/bin/nologin' >>/etc/passwd 38 | echo 'nogroup:x:65534:' >>/etc/group 39 | mkdir -p /var/log/nginx /run/nginx/ /var/cache/nginx /var/lib/nginx/{,logs,proxy_temp,client_body_temp,fastcgi_temp,scgi_temp,uwsgi_temp} /tmp/nginx_client_body 40 | chown nginx /var/log/nginx /run/nginx/ /var/cache/nginx /var/lib/nginx/{,logs,proxy_temp,client_body_temp,fastcgi_temp,scgi_temp,uwsgi_temp} /tmp/nginx_client_body 41 | ${config.systemd.services.nginx.runner} 42 | ''; 43 | }; 44 | service.command = [ config.nixos.build.run-nginx ]; 45 | service.useHostStore = true; 46 | service.ports = [ 47 | "8000:80" # host:container 48 | ]; 49 | }; 50 | } 51 | -------------------------------------------------------------------------------- /examples/nixos-unit/arion-pkgs.nix: -------------------------------------------------------------------------------- 1 | # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH 2 | import { 3 | # We specify the architecture explicitly. Use a Linux remote builder when 4 | # calling arion from other platforms. 5 | system = "x86_64-linux"; 6 | } 7 | -------------------------------------------------------------------------------- /examples/todomvc/README.md: -------------------------------------------------------------------------------- 1 | 2 | # todomvc 3 | 4 | This is the most full-featured example of a project. It's a [demo/template repo in its own right.](https://github.com/nix-community/todomvc-nix/tree/5ac9ecbdb556f5652d9cfc3d411e4e46854bcc44/deploy/arion) 5 | -------------------------------------------------------------------------------- /examples/traefik/arion-compose.nix: -------------------------------------------------------------------------------- 1 | /* 2 | 3 | An example of 4 | - traefik HTTP reverse proxy 5 | - minimal images 6 | - routing via docker labels 7 | 8 | Run `arion up -d` and open http://nix-docs.localhost/ 9 | 10 | */ 11 | { lib, pkgs, ... }: { 12 | config.project.name = "traefik"; 13 | config.networks = { 14 | traefik-custom = { 15 | name = "traefik-custom"; 16 | ipam = { 17 | config = [{ 18 | subnet = "172.32.0.0/16"; 19 | gateway = "172.32.0.1"; 20 | }]; 21 | }; 22 | }; 23 | }; 24 | config.services = { 25 | traefik = { 26 | image.command = [ 27 | "${pkgs.traefik}/bin/traefik" 28 | "--api.insecure=true" 29 | "--providers.docker=true" 30 | "--providers.docker.exposedbydefault=false" 31 | "--entrypoints.web.address=:80" 32 | ]; 33 | service = { 34 | container_name = "traefik"; 35 | stop_signal = "SIGINT"; 36 | ports = [ "80:80" "8080:8080" ]; 37 | volumes = [ "/var/run/docker.sock:/var/run/docker.sock:ro" ]; 38 | networks = [ "traefik-custom" ]; 39 | }; 40 | }; 41 | 42 | nix-docs = { 43 | image.command = ["${pkgs.writeScript "entrypoint" '' 44 | #!${pkgs.bash}/bin/bash 45 | cd ${pkgs.nix.doc}/share/doc/nix/manual 46 | ${pkgs.python3}/bin/python -m http.server 47 | ''}"]; 48 | service.container_name = "simple-service"; 49 | service.stop_signal = "SIGINT"; 50 | service.labels = { 51 | "traefik.enable" = "true"; 52 | "traefik.http.routers.nix-docs.rule" = "Host(`nix-docs.localhost`)"; 53 | "traefik.http.routers.nix-docs.entrypoints" = "web"; 54 | "traefik.http.services.nix-docs.loadBalancer.server.port" = "8000"; 55 | }; 56 | service.networks = { 57 | traefik-custom = { 58 | ipv4_address = "172.32.0.5"; 59 | }; 60 | }; 61 | }; 62 | }; 63 | } 64 | 65 | -------------------------------------------------------------------------------- /examples/traefik/arion-pkgs.nix: -------------------------------------------------------------------------------- 1 | # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH 2 | import { 3 | # We specify the architecture explicitly. Use a Linux remote builder when 4 | # calling arion from other platforms. 5 | system = "x86_64-linux"; 6 | } 7 | -------------------------------------------------------------------------------- /flake.lock: -------------------------------------------------------------------------------- 1 | { 2 | "nodes": { 3 | "flake-parts": { 4 | "inputs": { 5 | "nixpkgs-lib": [ 6 | "nixpkgs" 7 | ] 8 | }, 9 | "locked": { 10 | "lastModified": 1733312601, 11 | "narHash": "sha256-4pDvzqnegAfRkPwO3wmwBhVi/Sye1mzps0zHWYnP88c=", 12 | "owner": "hercules-ci", 13 | "repo": "flake-parts", 14 | "rev": "205b12d8b7cd4802fbcb8e8ef6a0f1408781a4f9", 15 | "type": "github" 16 | }, 17 | "original": { 18 | "owner": "hercules-ci", 19 | "repo": "flake-parts", 20 | "type": "github" 21 | } 22 | }, 23 | "haskell-flake": { 24 | "locked": { 25 | "lastModified": 1675296942, 26 | "narHash": "sha256-u1X1sblozi5qYEcLp1hxcyo8FfDHnRUVX3dJ/tW19jY=", 27 | "owner": "srid", 28 | "repo": "haskell-flake", 29 | "rev": "c2cafce9d57bfca41794dc3b99c593155006c71e", 30 | "type": "github" 31 | }, 32 | "original": { 33 | "owner": "srid", 34 | "ref": "0.1.0", 35 | "repo": "haskell-flake", 36 | "type": "github" 37 | } 38 | }, 39 | "nixpkgs": { 40 | "locked": { 41 | "lastModified": 1733212471, 42 | "narHash": "sha256-M1+uCoV5igihRfcUKrr1riygbe73/dzNnzPsmaLCmpo=", 43 | "owner": "NixOS", 44 | "repo": "nixpkgs", 45 | "rev": "55d15ad12a74eb7d4646254e13638ad0c4128776", 46 | "type": "github" 47 | }, 48 | "original": { 49 | "owner": "NixOS", 50 | "ref": "nixos-unstable", 51 | "repo": "nixpkgs", 52 | "type": "github" 53 | } 54 | }, 55 | "root": { 56 | "inputs": { 57 | "flake-parts": "flake-parts", 58 | "haskell-flake": "haskell-flake", 59 | "nixpkgs": "nixpkgs" 60 | } 61 | } 62 | }, 63 | "root": "root", 64 | "version": 7 65 | } 66 | -------------------------------------------------------------------------------- /flake.nix: -------------------------------------------------------------------------------- 1 | { 2 | description = "Arion - use Docker Compose via Nix"; 3 | 4 | inputs = { 5 | nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; 6 | haskell-flake.url = "github:srid/haskell-flake/0.1.0"; 7 | flake-parts.url = "github:hercules-ci/flake-parts"; 8 | flake-parts.inputs.nixpkgs-lib.follows = "nixpkgs"; 9 | }; 10 | 11 | outputs = inputs@{ self, flake-parts, ... }: 12 | flake-parts.lib.mkFlake { inherit inputs; } ({ config, lib, extendModules, ... }: { 13 | imports = [ 14 | inputs.haskell-flake.flakeModule 15 | inputs.flake-parts.flakeModules.easyOverlay 16 | inputs.flake-parts.flakeModules.partitions 17 | ./docs/flake-module.nix 18 | ]; 19 | 20 | partitions.dev = { 21 | extraInputsFlake = ./dev; 22 | module = { inputs, ... }: { 23 | imports = [ 24 | inputs.hercules-ci-effects.flakeModule 25 | inputs.git-hooks-nix.flakeModule 26 | ./tests/flake-module.nix 27 | ./dev/flake-module.nix 28 | ]; 29 | }; 30 | }; 31 | partitionedAttrs.devShells = "dev"; 32 | partitionedAttrs.checks = "dev"; 33 | partitionedAttrs.herculesCI = "dev"; 34 | 35 | systems = inputs.nixpkgs.lib.systems.flakeExposed; 36 | perSystem = { config, self', inputs', pkgs, system, final, ... }: 37 | let h = pkgs.haskell.lib.compose; in 38 | { 39 | overlayAttrs = { 40 | inherit (config.packages) arion; 41 | arionTestingFlags = { 42 | dockerSupportsSystemd = false; 43 | }; 44 | }; 45 | packages.default = config.packages.arion; 46 | packages.overlay-test = final.arion; 47 | packages.arion = import ./nix/arion.nix { inherit pkgs; }; 48 | haskellProjects.haskell-package = { 49 | # not autodetected: https://github.com/srid/haskell-flake/issues/49 50 | packages.arion-compose.root = ./.; 51 | 52 | overrides = 53 | self: super: { 54 | arion-compose = 55 | lib.pipe super.arion-compose [ 56 | (h.addBuildTools [ pkgs.nix ]) 57 | (h.overrideCabal (o: { 58 | src = pkgs.lib.sourceByRegex ./. [ 59 | ".*[.]cabal" 60 | "LICENSE" 61 | "src/?.*" 62 | "README.asciidoc" 63 | "CHANGELOG.md" 64 | ]; 65 | preCheck = '' 66 | export NIX_LOG_DIR=$TMPDIR 67 | export NIX_STATE_DIR=$TMPDIR 68 | export NIX_PATH=nixpkgs=${pkgs.path} 69 | ''; 70 | })) 71 | ]; 72 | }; 73 | }; 74 | }; 75 | 76 | flake = { 77 | debug = { inherit inputs config lib; }; 78 | 79 | lib = { 80 | eval = import ./src/nix/eval-composition.nix; 81 | build = args@{ ... }: 82 | let composition = self.lib.eval args; 83 | in composition.config.out.dockerComposeYaml; 84 | }; 85 | nixosModules.arion = ./nixos-module.nix; 86 | }; 87 | }); 88 | } 89 | -------------------------------------------------------------------------------- /live-unit-tests: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env nix-shell 2 | #!nix-shell ./shell.nix 3 | #!nix-shell -i bash 4 | set -eux -o pipefail 5 | 6 | cd "$(dirname "${BASH_SOURCE[0]}")" 7 | 8 | ghcid \ 9 | --command 'cabal v2-repl arion-compose:arion-unit-tests --flags ghci --write-ghc-environment-files=never' \ 10 | --test=Main.main \ 11 | --reload=src/haskell \ 12 | --restart=arion-compose.cabal \ 13 | ; 14 | -------------------------------------------------------------------------------- /nix/arion.nix: -------------------------------------------------------------------------------- 1 | # Like the upstreamable expression but wired up for the local arion. 2 | { pkgs ? import ./. {} 3 | , lib ? pkgs.lib 4 | , haskell ? pkgs.haskell 5 | , haskellPackages ? pkgs.haskellPackages 6 | , arion-compose ? import ./haskell-arion-compose.nix { inherit pkgs haskellPackages; } 7 | , runCommand ? pkgs.runCommand 8 | }: 9 | import ./upstreamable/default.nix { 10 | inherit pkgs lib haskell runCommand; 11 | haskellPackages = haskellPackages // { inherit arion-compose; }; 12 | evalSrc = ./..; 13 | } 14 | -------------------------------------------------------------------------------- /nix/compat.nix: -------------------------------------------------------------------------------- 1 | (import 2 | ( 3 | let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in 4 | fetchTarball { 5 | url = "https://github.com/edolstra/flake-compat/archive/009399224d5e398d03b22badca40a37ac85412a1.tar.gz"; 6 | sha256 = "sha256:0xcr9fibnapa12ywzcnlf54wrmbqqb96fmmv8043zhsycws7bpqy"; 7 | } 8 | ) 9 | { src = ../.; } 10 | ).defaultNix 11 | -------------------------------------------------------------------------------- /nix/haskell-arion-compose.nix: -------------------------------------------------------------------------------- 1 | 2 | # NOTE: This file produces a haskell library, not the arion package! 3 | 4 | { pkgs ? import ./default.nix {}, haskellPackages ? pkgs.haskellPackages }: 5 | let 6 | inherit (pkgs.haskell.lib) overrideCabal addBuildTools; 7 | in 8 | overrideCabal (addBuildTools (haskellPackages.callCabal2nix "arion-compose" ./.. {}) [pkgs.nix]) (o: o // { 9 | src = pkgs.lib.sourceByRegex ../. [ 10 | ".*[.]cabal" 11 | "LICENSE" 12 | "src/?.*" 13 | "README.asciidoc" 14 | ]; 15 | preCheck = '' 16 | export NIX_LOG_DIR=$TMPDIR 17 | export NIX_STATE_DIR=$TMPDIR 18 | export NIX_PATH=nixpkgs=${pkgs.path} 19 | ''; 20 | }) -------------------------------------------------------------------------------- /nix/upstreamable/default.nix: -------------------------------------------------------------------------------- 1 | args@ 2 | { pkgs 3 | , lib 4 | , haskellPackages 5 | , haskell 6 | , runCommand 7 | 8 | # Allow this expression file to be used more efficiently in situations where 9 | # the sources are more readily available. Unpacking haskellPackages.arion-compose.src 10 | # is not always the best choice for arion.eval. 11 | , evalSrc ? null 12 | }: 13 | 14 | let 15 | 16 | /* This derivation builds the arion tool. 17 | 18 | It is based on the arion-compose Haskell package, but adapted and extended to 19 | - have the correct name 20 | - have a smaller closure size 21 | - have functions to use Arion from inside Nix: arion.eval and arion.build 22 | - make it self-contained by including docker-compose 23 | */ 24 | arion = 25 | justStaticExecutables ( 26 | overrideCabal 27 | arion-compose 28 | cabalOverrides 29 | ); 30 | 31 | inherit (haskell.lib) justStaticExecutables overrideCabal; 32 | 33 | inherit (haskellPackages) arion-compose; 34 | 35 | cabalOverrides = o: { 36 | buildTools = (o.buildTools or []) ++ [pkgs.makeWrapper]; 37 | passthru = (o.passthru or {}) // { 38 | inherit eval build; 39 | }; 40 | # Patch away the arion-compose name. Unlike the Haskell library, the program 41 | # is called arion (arion was already taken on hackage). 42 | pname = "arion"; 43 | src = arion-compose.src; 44 | 45 | # PYTHONPATH 46 | # 47 | # We close off the python module search path! 48 | # 49 | # Accepting directories from the environment into the search path 50 | # tends to break things. Docker Compose does not have a plugin 51 | # system as far as I can tell, so I don't expect this to break a 52 | # feature, but rather to make the program more robustly self- 53 | # contained. 54 | 55 | postInstall = ''${o.postInstall or ""} 56 | mkdir -p $out/libexec 57 | mv $out/bin/arion $out/libexec 58 | makeWrapper $out/libexec/arion $out/bin/arion \ 59 | --unset PYTHONPATH \ 60 | --prefix PATH : ${lib.makeBinPath [ pkgs.docker-compose ]} \ 61 | ; 62 | ''; 63 | }; 64 | 65 | # Unpacked sources for evaluation by `eval` 66 | evalSrc' = args.evalSrc or (runCommand "arion-src" {} 67 | "mkdir $out; tar -C $out --strip-components=1 -xf ${arion-compose.src}"); 68 | 69 | /* Function for evaluating a composition 70 | 71 | Re-uses this Nixpkgs evaluation instead of `arion-pkgs.nix`. 72 | 73 | Returns the module system's `config` and `options` variables. 74 | */ 75 | eval = args@{...}: 76 | import (evalSrc' + "/src/nix/eval-composition.nix") 77 | ({ inherit pkgs; } // args); 78 | 79 | /* Function to derivation of the docker compose yaml file 80 | NOTE: The output will change: https://github.com/hercules-ci/arion/issues/82 81 | 82 | This function is particularly useful on CI. 83 | */ 84 | build = args@{...}: 85 | let composition = eval args; 86 | in composition.config.out.dockerComposeYaml; 87 | 88 | in arion 89 | -------------------------------------------------------------------------------- /nixos-module.nix: -------------------------------------------------------------------------------- 1 | { config, lib, options, pkgs, ... }: 2 | let 3 | inherit (lib) 4 | attrValues 5 | mkIf 6 | mkOption 7 | mkMerge 8 | types 9 | ; 10 | 11 | cfg = config.virtualisation.arion; 12 | 13 | projectType = types.submoduleWith { 14 | modules = [ projectModule ]; 15 | }; 16 | 17 | projectModule = { config, name, ... }: { 18 | options = { 19 | settings = mkOption { 20 | description = '' 21 | Arion project definition, otherwise known as arion-compose.nix contents. 22 | 23 | See https://docs.hercules-ci.com/arion/options/. 24 | ''; 25 | type = arionSettingsType name; 26 | visible = "shallow"; 27 | }; 28 | _systemd = mkOption { internal = true; }; 29 | serviceName = mkOption { 30 | description = "The name of the Arion project's systemd service"; 31 | type = types.str; 32 | default = "arion-${name}"; 33 | }; 34 | }; 35 | config = { 36 | _systemd.services.${config.serviceName} = { 37 | wantedBy = [ "multi-user.target" ]; 38 | after = [ "sockets.target" ]; 39 | 40 | path = [ 41 | cfg.package 42 | cfg.docker.client.package 43 | ]; 44 | environment.ARION_PREBUILT = config.settings.out.dockerComposeYaml; 45 | script = '' 46 | echo 1>&2 "docker compose file: $ARION_PREBUILT" 47 | arion --prebuilt-file "$ARION_PREBUILT" up 48 | ''; 49 | }; 50 | }; 51 | }; 52 | 53 | arionSettingsType = name: 54 | (cfg.package.eval { modules = [{ project.name = lib.mkDefault name; }]; }).type or ( 55 | throw "lib.evalModules did not produce a type. Please upgrade Nixpkgs to nixos-unstable or >=nixos-21.11" 56 | ); 57 | 58 | in 59 | { 60 | disabledModules = [ "virtualisation/arion.nix" ]; 61 | 62 | options = { 63 | virtualisation.arion = { 64 | backend = mkOption { 65 | type = types.enum [ "podman-socket" "docker" ]; 66 | description = '' 67 | Which container implementation to use. 68 | ''; 69 | }; 70 | package = mkOption { 71 | type = types.package; 72 | 73 | default = (import ./. { inherit pkgs; }).arion; 74 | description = '' 75 | Arion package to use. This will provide arion 76 | executable that starts the project. 77 | 78 | It also must provide the arion eval function as 79 | an attribute. 80 | ''; 81 | }; 82 | docker.client.package = mkOption { 83 | type = types.package; 84 | internal = true; 85 | }; 86 | projects = mkOption { 87 | type = types.attrsOf projectType; 88 | default = { }; 89 | description = '' 90 | Arion projects to be run as a service. 91 | ''; 92 | }; 93 | }; 94 | }; 95 | 96 | config = mkIf (cfg.projects != { }) ( 97 | mkMerge [ 98 | { 99 | systemd = mkMerge (map (p: p._systemd) (attrValues cfg.projects)); 100 | } 101 | (mkIf (cfg.backend == "podman-socket") { 102 | virtualisation.docker.enable = false; 103 | virtualisation.podman.enable = true; 104 | virtualisation.podman.dockerSocket.enable = true; 105 | virtualisation.podman.defaultNetwork = 106 | if options?virtualisation.podman.defaultNetwork.settings 107 | then { settings.dns_enabled = true; } # since 2023-01 https://github.com/NixOS/nixpkgs/pull/199965 108 | else { dnsname.enable = true; }; # compat <2023 109 | 110 | virtualisation.arion.docker.client.package = pkgs.docker-client; 111 | }) 112 | (mkIf (cfg.backend == "docker") { 113 | virtualisation.docker.enable = true; 114 | virtualisation.arion.docker.client.package = pkgs.docker; 115 | }) 116 | ] 117 | ); 118 | } 119 | -------------------------------------------------------------------------------- /repl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env nix-shell 2 | #!nix-shell -i bash 3 | 4 | # A Haskell REPL for hacking on the Haskell code 5 | 6 | cabal new-repl --write-ghc-environment-files=never 7 | -------------------------------------------------------------------------------- /run-arion: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env nix-shell 2 | #!nix-shell -i bash 3 | #!nix-shell ./shell.nix 4 | 5 | # For quick manual testing of a hacked arion 6 | 7 | # NB: Only works inside the project directory 8 | 9 | cabal \ 10 | new-run \ 11 | --write-ghc-environment-files=never \ 12 | :pkg:arion-compose:exe:arion \ 13 | -- \ 14 | "$@" \ 15 | ; 16 | -------------------------------------------------------------------------------- /run-arion-quick: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | projectRoot="$(dirname ${BASH_SOURCE[0]})" 4 | resultLink="$projectRoot/result-run-arion-quick" 5 | 6 | [[ -e "$resultLink" ]] || { 7 | echo 1>&2 "You don't have a prebuilt arion yet; building it." 8 | nix-build "$projectRoot" -A arion --out-link "$resultLink" 9 | } 10 | 11 | echo 1>&2 "Note that you will need to rm '$resultLink' to rebuild the arion executable when needed." 12 | 13 | export arion_compose_datadir="$projectRoot/src" 14 | 15 | exec "$resultLink/bin/arion" "$@" 16 | -------------------------------------------------------------------------------- /run-arion-via-nix: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # For manual testing of a hacked arion built via Nix. 4 | # Works when called from outside the project directory. 5 | 6 | exec nix run -f "$(dirname ${BASH_SOURCE[0]})" arion -- "$@" 7 | -------------------------------------------------------------------------------- /shell.nix: -------------------------------------------------------------------------------- 1 | (builtins.getFlake ("git+file://" + toString ./.)).devShells.${builtins.currentSystem}.default 2 | -------------------------------------------------------------------------------- /src/arion-image/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM scratch 2 | # scratch itself can't be run. 3 | 4 | # This seems like a no-op: 5 | CMD [] 6 | -------------------------------------------------------------------------------- /src/haskell/exe/Main.hs: -------------------------------------------------------------------------------- 1 | {-# LANGUAGE ApplicativeDo #-} 2 | {-# LANGUAGE RecordWildCards #-} 3 | {-# LANGUAGE NoImplicitPrelude #-} 4 | 5 | import Arion.Aeson 6 | import qualified Arion.DockerCompose as DockerCompose 7 | import Arion.ExtendedInfo (ExtendedInfo (images, projectName), loadExtendedInfoFromPath) 8 | import Arion.Images (loadImages) 9 | import Arion.Nix 10 | import Arion.Services (getDefaultExec) 11 | import Control.Monad.Fail 12 | import Data.Aeson (Value) 13 | import qualified Data.Text as T 14 | import qualified Data.Text.IO as T 15 | import Options.Applicative 16 | import Protolude hiding (Down, option) 17 | import System.Posix.User (getRealUserID) 18 | 19 | data CommonOptions = CommonOptions 20 | { files :: NonEmpty FilePath, 21 | pkgs :: Text, 22 | nixArgs :: [Text], 23 | prebuiltComposeFile :: Maybe FilePath, 24 | noAnsi :: Bool, 25 | compatibility :: Bool, 26 | logLevel :: Maybe Text 27 | } 28 | deriving stock (Show) 29 | 30 | newtype DockerComposeArgs = DockerComposeArgs {unDockerComposeArgs :: [Text]} 31 | 32 | ensureConfigFile :: [FilePath] -> NonEmpty FilePath 33 | ensureConfigFile [] = "./arion-compose.nix" :| [] 34 | ensureConfigFile (x : xs) = x :| xs 35 | 36 | parseOptions :: Parser CommonOptions 37 | parseOptions = do 38 | files <- 39 | ensureConfigFile 40 | <$> many 41 | ( strOption 42 | ( short 'f' 43 | <> long "file" 44 | <> metavar "FILE" 45 | <> help 46 | "Use FILE instead of the default ./arion-compose.nix. \ 47 | \Can be specified multiple times for a merged configuration" 48 | ) 49 | ) 50 | pkgs <- 51 | T.pack 52 | <$> strOption 53 | ( short 'p' 54 | <> long "pkgs" 55 | <> metavar "EXPR" 56 | <> showDefault 57 | <> value "./arion-pkgs.nix" 58 | <> help 59 | "Use Nix expression EXPR to get the Nixpkgs attrset used for bootstrapping \ 60 | \and evaluating the configuration." 61 | ) 62 | showTrace <- 63 | flag 64 | False 65 | True 66 | ( long "show-trace" 67 | <> help "Causes Nix to print out a stack trace in case of Nix expression evaluation errors. Specify before command." 68 | ) 69 | -- TODO --option support (https://github.com/pcapriotti/optparse-applicative/issues/284) 70 | userNixArgs <- many (T.pack <$> strOption (long "nix-arg" <> metavar "ARG" <> help "Pass an extra argument to nix. Example: --nix-arg --option --nix-arg substitute --nix-arg false")) 71 | prebuiltComposeFile <- 72 | optional $ 73 | strOption 74 | ( long "prebuilt-file" 75 | <> metavar "JSONFILE" 76 | <> help "Do not evaluate and use the prebuilt JSONFILE instead. Causes other evaluation-related options to be ignored." 77 | ) 78 | noAnsi <- 79 | flag 80 | False 81 | True 82 | ( long "no-ansi" 83 | <> help "Avoid ANSI control sequences" 84 | ) 85 | compatibility <- 86 | flag 87 | False 88 | True 89 | ( long "no-ansi" 90 | <> help "If set, Docker Compose will attempt to convert deploy keys in v3 files to their non-Swarm equivalent" 91 | ) 92 | logLevel <- optional $ fmap T.pack $ strOption (long "log-level" <> metavar "LEVEL" <> help "Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)") 93 | pure $ 94 | let nixArgs = userNixArgs <|> "--show-trace" <$ guard showTrace 95 | in CommonOptions {..} 96 | 97 | textArgument :: Mod ArgumentFields [Char] -> Parser Text 98 | textArgument = fmap T.pack . strArgument 99 | 100 | parseCommand :: Parser (CommonOptions -> IO ()) 101 | parseCommand = 102 | hsubparser 103 | ( command "cat" (info (pure runCat) (progDesc "Spit out the docker compose file as JSON" <> fullDesc)) 104 | <> command "repl" (info (pure runRepl) (progDesc "Start a nix repl for the whole composition" <> fullDesc)) 105 | <> command "exec" (info (parseExecCommand) (progDesc "Execute a command in a running container" <> fullDesc)) 106 | ) 107 | <|> hsubparser 108 | ( commandDC runBuildAndDC "build" "Build or rebuild services" 109 | <> commandDC runBuildAndDC "bundle" "Generate a Docker bundle from the Compose file" 110 | <> commandDC runEvalAndDC "config" "Validate and view the Compose file" 111 | <> commandDC runBuildAndDC "create" "Create services" 112 | <> commandDC runEvalAndDC "down" "Stop and remove containers, networks, images, and volumes" 113 | <> commandDC runEvalAndDC "events" "Receive real time events from containers" 114 | <> commandDC runDC "help" "Get help on a command" 115 | <> commandDC runEvalAndDC "images" "List images" 116 | <> commandDC runEvalAndDC "kill" "Kill containers" 117 | <> commandDC runEvalAndDC "logs" "View output from containers" 118 | <> commandDC runEvalAndDC "pause" "Pause services" 119 | <> commandDC runEvalAndDC "port" "Print the public port for a port binding" 120 | <> commandDC runEvalAndDC "ps" "List containers" 121 | <> commandDC runBuildAndDC "pull" "Pull service images" 122 | <> commandDC runBuildAndDC "push" "Push service images" 123 | <> commandDC runBuildAndDC "restart" "Restart services" 124 | <> commandDC runEvalAndDC "rm" "Remove stopped containers" 125 | <> commandDC runBuildAndDC "run" "Run a one-off command" 126 | <> commandDC runBuildAndDC "scale" "Set number of containers for a service" 127 | <> commandDC runBuildAndDC "start" "Start services" 128 | <> commandDC runEvalAndDC "stop" "Stop services" 129 | <> commandDC runEvalAndDC "top" "Display the running processes" 130 | <> commandDC runEvalAndDC "unpause" "Unpause services" 131 | <> commandDC runBuildAndDC "up" "Create and start containers" 132 | <> commandDC runDC "version" "Show the Docker-Compose version information" 133 | <> metavar "DOCKER-COMPOSE-COMMAND" 134 | <> commandGroup "Docker Compose Commands:" 135 | ) 136 | 137 | parseAll :: Parser (IO ()) 138 | parseAll = 139 | flip ($) <$> parseOptions <*> parseCommand 140 | 141 | parseDockerComposeArgs :: Parser DockerComposeArgs 142 | parseDockerComposeArgs = 143 | DockerComposeArgs 144 | <$> many (argument (T.pack <$> str) (metavar "DOCKER-COMPOSE ARGS...")) 145 | 146 | commandDC :: 147 | (Text -> DockerComposeArgs -> CommonOptions -> IO ()) -> 148 | Text -> 149 | Text -> 150 | Mod CommandFields (CommonOptions -> IO ()) 151 | commandDC run cmdStr helpText = 152 | command 153 | (T.unpack cmdStr) 154 | ( info 155 | (run cmdStr <$> parseDockerComposeArgs) 156 | (progDesc (T.unpack helpText) <> fullDesc <> forwardOptions) 157 | ) 158 | 159 | -------------------------------------------------------------------------------- 160 | 161 | runDC :: Text -> DockerComposeArgs -> CommonOptions -> IO () 162 | runDC cmd (DockerComposeArgs args) _opts = do 163 | DockerCompose.run 164 | DockerCompose.Args 165 | { files = [], 166 | otherArgs = [cmd] ++ args 167 | } 168 | 169 | runBuildAndDC :: Text -> DockerComposeArgs -> CommonOptions -> IO () 170 | runBuildAndDC cmd dopts opts = do 171 | withBuiltComposeFile opts $ callDC cmd dopts opts True 172 | 173 | runEvalAndDC :: Text -> DockerComposeArgs -> CommonOptions -> IO () 174 | runEvalAndDC cmd dopts opts = do 175 | withComposeFile opts $ callDC cmd dopts opts False 176 | 177 | callDC :: Text -> DockerComposeArgs -> CommonOptions -> Bool -> FilePath -> IO () 178 | callDC cmd dopts opts shouldLoadImages path = do 179 | extendedInfo <- loadExtendedInfoFromPath path 180 | when shouldLoadImages $ loadImages (extendedInfo.images) 181 | let firstOpts = projectArgs extendedInfo <> commonArgs opts 182 | DockerCompose.run 183 | DockerCompose.Args 184 | { files = [path], 185 | otherArgs = firstOpts ++ [cmd] ++ dopts.unDockerComposeArgs 186 | } 187 | 188 | projectArgs :: ExtendedInfo -> [Text] 189 | projectArgs extendedInfo = 190 | do 191 | n <- toList (extendedInfo.projectName) 192 | ["--project-name", n] 193 | 194 | commonArgs :: CommonOptions -> [Text] 195 | commonArgs opts = 196 | do 197 | guard opts.noAnsi 198 | ["--no-ansi"] 199 | <> do 200 | guard opts.compatibility 201 | ["--compatibility"] 202 | <> do 203 | l <- toList opts.logLevel 204 | ["--log-level", l] 205 | 206 | withBuiltComposeFile :: CommonOptions -> (FilePath -> IO r) -> IO r 207 | withBuiltComposeFile opts cont = case opts.prebuiltComposeFile of 208 | Just prebuilt -> do 209 | cont prebuilt 210 | Nothing -> do 211 | args <- defaultEvaluationArgs opts 212 | Arion.Nix.withBuiltComposition args cont 213 | 214 | withComposeFile :: CommonOptions -> (FilePath -> IO r) -> IO r 215 | withComposeFile opts cont = case opts.prebuiltComposeFile of 216 | Just prebuilt -> do 217 | cont prebuilt 218 | Nothing -> do 219 | args <- defaultEvaluationArgs opts 220 | Arion.Nix.withEvaluatedComposition args cont 221 | 222 | getComposeValue :: CommonOptions -> IO Value 223 | getComposeValue opts = case opts.prebuiltComposeFile of 224 | Just prebuilt -> do 225 | decodeFile prebuilt 226 | Nothing -> do 227 | args <- defaultEvaluationArgs opts 228 | Arion.Nix.evaluateComposition args 229 | 230 | defaultEvaluationArgs :: CommonOptions -> IO EvaluationArgs 231 | defaultEvaluationArgs co = do 232 | uid <- getRealUserID 233 | pure 234 | EvaluationArgs 235 | { posixUID = fromIntegral uid, 236 | evalModulesFile = co.files, 237 | pkgsExpr = co.pkgs, 238 | workDir = Nothing, 239 | mode = ReadWrite, 240 | extraNixArgs = co.nixArgs 241 | } 242 | 243 | runCat :: CommonOptions -> IO () 244 | runCat co = do 245 | v <- getComposeValue co 246 | T.hPutStrLn stdout (pretty v) 247 | 248 | runRepl :: CommonOptions -> IO () 249 | runRepl co = do 250 | putErrText 251 | "Launching a repl for you. To get started:\n\ 252 | \\n\ 253 | \To see deployment-wide configuration\n\ 254 | \ type config. and use tab completion\n\ 255 | \To bring the top-level Nixpkgs attributes into scope\n\ 256 | \ type :a (config._module.args.pkgs) // { inherit config; }\n\ 257 | \" 258 | Arion.Nix.replForComposition =<< defaultEvaluationArgs co 259 | 260 | detachFlag :: Parser Bool 261 | detachFlag = flag False True (long "detach" <> short 'd' <> help "Detached mode: Run command in the background.") 262 | 263 | privilegedFlag :: Parser Bool 264 | privilegedFlag = flag False True (long "privileged" <> help "Give extended privileges to the process.") 265 | 266 | userOption :: Parser Text 267 | userOption = strOption (long "user" <> short 'u' <> help "Run the command as this user.") 268 | 269 | noTTYFlag :: Parser Bool 270 | noTTYFlag = flag False True (short 'T' <> help "Disable pseudo-tty allocation. By default `exec` allocates a TTY.") 271 | 272 | indexOption :: Parser Int 273 | indexOption = 274 | option 275 | (auto >>= \i -> i <$ unless (i >= 1) (fail "container index must be >= 1")) 276 | (long "index" <> value 1 <> help "Index of the container if there are multiple instances of a service.") 277 | 278 | envOption :: Parser (Text, Text) 279 | envOption = option (auto >>= spl) (long "env" <> short 'e' <> help "Set environment variables (can be used multiple times, not supported in Docker API < 1.25)") 280 | where 281 | spl s = case T.break (== '=') s of 282 | (_, "") -> fail "--env parameter needs to combine key and value with = sign" 283 | (k, ev) -> pure (k, T.drop 1 ev) 284 | 285 | workdirOption :: Parser Text 286 | workdirOption = strOption (long "workdir" <> short 'w' <> metavar "DIR" <> help "Working directory in which to start the command in the container.") 287 | 288 | parseExecCommand :: Parser (CommonOptions -> IO ()) 289 | parseExecCommand = 290 | runExec 291 | <$> detachFlag 292 | <*> privilegedFlag 293 | <*> optional userOption 294 | <*> noTTYFlag 295 | <*> indexOption 296 | <*> many envOption 297 | <*> optional workdirOption 298 | <*> textArgument (metavar "SERVICE") 299 | <*> orEmpty' 300 | ( (:) 301 | <$> argument (T.pack <$> str) (metavar "COMMAND") 302 | <*> many (argument (T.pack <$> str) (metavar "ARG")) 303 | ) 304 | 305 | orEmpty' :: (Alternative f, Monoid a) => f a -> f a 306 | orEmpty' m = fromMaybe mempty <$> optional m 307 | 308 | runExec :: Bool -> Bool -> Maybe Text -> Bool -> Int -> [(Text, Text)] -> Maybe Text -> Text -> [Text] -> CommonOptions -> IO () 309 | runExec detach privileged user noTTY index envs workDir service commandAndArgs opts = 310 | withComposeFile opts $ \path -> do 311 | extendedInfo <- loadExtendedInfoFromPath path 312 | commandAndArgs'' <- case commandAndArgs of 313 | [] -> do 314 | cmd <- getDefaultExec path service 315 | case cmd of 316 | [] -> do 317 | putErrText "You must provide a command via service.defaultExec or on the command line." 318 | exitFailure 319 | _ -> 320 | pure cmd 321 | x -> pure x 322 | let commandAndArgs' = case commandAndArgs'' of 323 | [] -> ["/bin/sh"] 324 | x -> x 325 | 326 | let args = 327 | concat 328 | [ ["exec"], 329 | ("--detach" <$ guard detach :: [Text]), 330 | "--privileged" <$ guard privileged, 331 | "-T" <$ guard noTTY, 332 | (\(k, v) -> ["--env", k <> "=" <> v]) =<< envs, 333 | join $ toList (user <&> \u -> ["--user", u]), 334 | ["--index", show index], 335 | join $ toList (workDir <&> \w -> ["--workdir", w]), 336 | [service], 337 | commandAndArgs' 338 | ] 339 | DockerCompose.run 340 | DockerCompose.Args 341 | { files = [path], 342 | otherArgs = projectArgs extendedInfo <> commonArgs opts <> args 343 | } 344 | 345 | main :: IO () 346 | main = 347 | (join . arionExecParser) (info (parseAll <**> helper) fullDesc) 348 | where 349 | arionExecParser = customExecParser (prefs showHelpOnEmpty) 350 | -------------------------------------------------------------------------------- /src/haskell/lib/Arion/Aeson.hs: -------------------------------------------------------------------------------- 1 | module Arion.Aeson where 2 | 3 | import Data.Aeson 4 | import Data.Aeson.Encode.Pretty 5 | ( confCompare, 6 | confTrailingNewline, 7 | defConfig, 8 | ) 9 | import qualified Data.Aeson.Encode.Pretty 10 | import qualified Data.ByteString.Lazy as BL 11 | import qualified Data.Text.Lazy as TL 12 | import qualified Data.Text.Lazy.Builder as TB 13 | import Protolude 14 | import Prelude () 15 | 16 | pretty :: (ToJSON a) => a -> Text 17 | pretty = 18 | TL.toStrict 19 | . TB.toLazyText 20 | . Data.Aeson.Encode.Pretty.encodePrettyToTextBuilder' config 21 | where 22 | config = defConfig {confCompare = compare, confTrailingNewline = True} 23 | 24 | decodeFile :: (FromJSON a) => FilePath -> IO a 25 | decodeFile fp = do 26 | b <- BL.readFile fp 27 | case eitherDecode b of 28 | Left e -> panic (toS e) 29 | Right v -> pure v 30 | -------------------------------------------------------------------------------- /src/haskell/lib/Arion/DockerCompose.hs: -------------------------------------------------------------------------------- 1 | module Arion.DockerCompose where 2 | 3 | import Protolude 4 | import System.Process 5 | import Prelude () 6 | 7 | data Args = Args 8 | { files :: [FilePath], 9 | otherArgs :: [Text] 10 | } 11 | 12 | run :: Args -> IO () 13 | run args = do 14 | let fileArgs = args.files >>= \f -> ["--file", f] 15 | allArgs = fileArgs ++ map toS args.otherArgs 16 | 17 | procSpec = proc "docker-compose" allArgs 18 | 19 | -- hPutStrLn stderr ("Running docker-compose with " <> show allArgs :: Text) 20 | 21 | withCreateProcess procSpec $ \_in _out _err procHandle -> do 22 | exitCode <- waitForProcess procHandle 23 | 24 | case exitCode of 25 | ExitSuccess -> pass 26 | ExitFailure 1 -> exitFailure 27 | ExitFailure {} -> do 28 | throwIO $ FatalError $ "docker-compose failed with " <> show exitCode 29 | -------------------------------------------------------------------------------- /src/haskell/lib/Arion/ExtendedInfo.hs: -------------------------------------------------------------------------------- 1 | {- 2 | 3 | Parses the x-arion field in the generated compose file. 4 | 5 | -} 6 | module Arion.ExtendedInfo where 7 | 8 | import Arion.Aeson 9 | import Control.Lens 10 | import Data.Aeson as Aeson 11 | import Data.Aeson.Lens 12 | import Protolude 13 | import Prelude () 14 | 15 | data Image = Image 16 | { -- | image tar.gz file path 17 | image :: Maybe Text, 18 | -- | path to exe producing image tar 19 | imageExe :: Maybe Text, 20 | imageName :: Text, 21 | imageTag :: Text 22 | } 23 | deriving stock (Eq, Show, Generic) 24 | deriving anyclass (Aeson.ToJSON, Aeson.FromJSON) 25 | 26 | data ExtendedInfo = ExtendedInfo 27 | { projectName :: Maybe Text, 28 | images :: [Image] 29 | } 30 | deriving stock (Eq, Show) 31 | 32 | loadExtendedInfoFromPath :: FilePath -> IO ExtendedInfo 33 | loadExtendedInfoFromPath fp = do 34 | v <- decodeFile fp 35 | pure 36 | ExtendedInfo 37 | { -- TODO: use aeson derived instance? 38 | projectName = v ^? key "x-arion" . key "project" . key "name" . _String, 39 | images = (v :: Aeson.Value) ^.. key "x-arion" . key "images" . _Array . traverse . _JSON 40 | } 41 | -------------------------------------------------------------------------------- /src/haskell/lib/Arion/Images.hs: -------------------------------------------------------------------------------- 1 | module Arion.Images 2 | ( loadImages, 3 | ) 4 | where 5 | 6 | import Arion.ExtendedInfo (Image (..)) 7 | import qualified Data.Text as T 8 | import Protolude hiding (to) 9 | import qualified System.Process as Process 10 | import Prelude () 11 | 12 | type TaggedImage = Text 13 | 14 | -- | Subject to change 15 | loadImages :: [Image] -> IO () 16 | loadImages requestedImages = do 17 | loaded <- getDockerImages 18 | 19 | let isNew :: Image -> Bool 20 | isNew i = 21 | -- On docker, the image name is unmodified 22 | (i.imageName <> ":" <> i.imageTag) `notElem` loaded 23 | -- On podman, you used to automatically get a localhost prefix 24 | -- however, since NixOS 22.05, this expected to be part of the name instead 25 | && ("localhost/" <> i.imageName <> ":" <> i.imageTag) `notElem` loaded 26 | 27 | traverse_ loadImage . filter isNew $ requestedImages 28 | 29 | loadImage :: Image -> IO () 30 | loadImage Image {image = Just imgPath, imageName = name} = 31 | withFile (toS imgPath) ReadMode $ \fileHandle -> do 32 | let procSpec = 33 | (Process.proc "docker" ["load"]) 34 | { Process.std_in = Process.UseHandle fileHandle 35 | } 36 | Process.withCreateProcess procSpec $ \_in _out _err procHandle -> do 37 | e <- Process.waitForProcess procHandle 38 | case e of 39 | ExitSuccess -> pass 40 | ExitFailure code -> 41 | panic $ "docker load failed with exit code " <> show code <> " for image " <> name <> " from path " <> imgPath 42 | loadImage Image {imageExe = Just imgExe, imageName = name} = do 43 | let loadSpec = (Process.proc "docker" ["load"]) {Process.std_in = Process.CreatePipe} 44 | Process.withCreateProcess loadSpec $ \(Just inHandle) _out _err loadProcHandle -> do 45 | let streamSpec = Process.proc (toS imgExe) [] 46 | Process.withCreateProcess streamSpec {Process.std_out = Process.UseHandle inHandle} $ \_ _ _ streamProcHandle -> 47 | withAsync (Process.waitForProcess loadProcHandle) $ \loadExitAsync -> 48 | withAsync (Process.waitForProcess streamProcHandle) $ \streamExitAsync -> do 49 | r <- waitEither loadExitAsync streamExitAsync 50 | case r of 51 | Right (ExitFailure code) -> panic $ "image producer for image " <> name <> " failed with exit code " <> show code <> " from executable " <> imgExe 52 | Right ExitSuccess -> pass 53 | Left _ -> pass 54 | loadExit <- wait loadExitAsync 55 | case loadExit of 56 | ExitFailure code -> panic $ "docker load failed with exit code " <> show code <> " for image " <> name <> " produced by executable " <> imgExe 57 | _ -> pass 58 | pass 59 | loadImage Image {imageName = name} = do 60 | panic $ "image " <> name <> " doesn't specify an image file or imageExe executable" 61 | 62 | getDockerImages :: IO [TaggedImage] 63 | getDockerImages = do 64 | let procSpec = Process.proc "docker" ["images", "--filter", "dangling=false", "--format", "{{.Repository}}:{{.Tag}}"] 65 | map toS . T.lines . toS <$> Process.readCreateProcess procSpec "" 66 | -------------------------------------------------------------------------------- /src/haskell/lib/Arion/Nix.hs: -------------------------------------------------------------------------------- 1 | module Arion.Nix 2 | ( evaluateComposition, 3 | withEvaluatedComposition, 4 | buildComposition, 5 | withBuiltComposition, 6 | replForComposition, 7 | EvaluationArgs (..), 8 | EvaluationMode (..), 9 | ) 10 | where 11 | 12 | import Arion.Aeson (pretty) 13 | import Control.Arrow ((>>>)) 14 | import Data.Aeson 15 | import qualified Data.ByteString.Lazy as BL 16 | import qualified Data.List.NonEmpty as NE 17 | import qualified Data.String 18 | import qualified Data.Text.IO as T 19 | import Paths_arion_compose 20 | import Protolude 21 | import qualified System.Directory as Directory 22 | import System.IO (hClose) 23 | import System.IO.Temp (withTempFile) 24 | import System.Process 25 | import Prelude () 26 | 27 | data EvaluationMode 28 | = ReadWrite 29 | | ReadOnly 30 | 31 | data EvaluationArgs = EvaluationArgs 32 | { posixUID :: Int, 33 | evalModulesFile :: NonEmpty FilePath, 34 | pkgsExpr :: Text, 35 | workDir :: Maybe FilePath, 36 | mode :: EvaluationMode, 37 | extraNixArgs :: [Text] 38 | } 39 | 40 | evaluateComposition :: EvaluationArgs -> IO Value 41 | evaluateComposition ea = do 42 | evalComposition <- getEvalCompositionFile 43 | let commandArgs = 44 | [ "--eval", 45 | "--strict", 46 | "--json", 47 | "--attr", 48 | "config.out.dockerComposeYamlAttrs" 49 | ] 50 | args = 51 | [evalComposition] 52 | ++ commandArgs 53 | ++ modeArguments ea.mode 54 | ++ argArgs ea 55 | ++ map toS ea.extraNixArgs 56 | procSpec = 57 | (proc "nix-instantiate" args) 58 | { cwd = ea.workDir, 59 | std_out = CreatePipe 60 | } 61 | 62 | withCreateProcess procSpec $ \_in outHM _err procHandle -> do 63 | let outHandle = fromMaybe (panic "stdout missing") outHM 64 | 65 | out <- BL.hGetContents outHandle 66 | 67 | v <- Protolude.evaluate (eitherDecode out) 68 | 69 | exitCode <- waitForProcess procHandle 70 | 71 | case exitCode of 72 | ExitSuccess -> pass 73 | ExitFailure 1 -> exitFailure 74 | ExitFailure {} -> do 75 | throwIO $ FatalError $ "evaluation failed with " <> show exitCode 76 | 77 | case v of 78 | Right r -> pure r 79 | Left e -> throwIO $ FatalError ("Couldn't parse nix-instantiate output" <> show e) 80 | 81 | -- | Run with docker-compose.yaml tmpfile 82 | withEvaluatedComposition :: EvaluationArgs -> (FilePath -> IO r) -> IO r 83 | withEvaluatedComposition ea f = do 84 | v <- evaluateComposition ea 85 | withTempFile "." ".tmp-arion-docker-compose.yaml" $ \path yamlHandle -> do 86 | T.hPutStrLn yamlHandle (pretty v) 87 | hClose yamlHandle 88 | f path 89 | 90 | buildComposition :: FilePath -> EvaluationArgs -> IO () 91 | buildComposition outLink ea = do 92 | evalComposition <- getEvalCompositionFile 93 | let commandArgs = 94 | [ "--attr", 95 | "config.out.dockerComposeYaml", 96 | "--out-link", 97 | outLink 98 | ] 99 | args = 100 | [evalComposition] 101 | ++ commandArgs 102 | ++ argArgs ea 103 | ++ map toS ea.extraNixArgs 104 | procSpec = (proc "nix-build" args) {cwd = ea.workDir} 105 | 106 | withCreateProcess procSpec $ \_in _out _err procHandle -> do 107 | exitCode <- waitForProcess procHandle 108 | 109 | case exitCode of 110 | ExitSuccess -> pass 111 | ExitFailure 1 -> exitFailure 112 | ExitFailure {} -> do 113 | throwIO $ FatalError $ "nix-build failed with " <> show exitCode 114 | 115 | -- | Do something with a docker-compose.yaml. 116 | withBuiltComposition :: EvaluationArgs -> (FilePath -> IO r) -> IO r 117 | withBuiltComposition ea f = do 118 | withTempFile "." ".tmp-arion-docker-compose.yaml" $ \path emptyYamlHandle -> do 119 | hClose emptyYamlHandle 120 | -- Known problem: kills atomicity of withTempFile; won't fix because we should manage gc roots, 121 | -- impl of which will probably avoid this "problem". It seems unlikely to cause issues. 122 | Directory.removeFile path 123 | buildComposition path ea 124 | f path 125 | 126 | replForComposition :: EvaluationArgs -> IO () 127 | replForComposition ea = do 128 | evalComposition <- getEvalCompositionFile 129 | let args = 130 | ["repl", "--file", evalComposition] 131 | ++ argArgs ea 132 | ++ map toS ea.extraNixArgs 133 | procSpec = (proc "nix" args) {cwd = ea.workDir} 134 | 135 | withCreateProcess procSpec $ \_in _out _err procHandle -> do 136 | exitCode <- waitForProcess procHandle 137 | 138 | case exitCode of 139 | ExitSuccess -> pass 140 | ExitFailure 1 -> exitFailure 141 | ExitFailure {} -> do 142 | throwIO $ FatalError $ "nix repl failed with " <> show exitCode 143 | 144 | argArgs :: EvaluationArgs -> [[Char]] 145 | argArgs ea = 146 | [ "--argstr", 147 | "uid", 148 | show ea.posixUID, 149 | "--arg", 150 | "modules", 151 | modulesNixExpr ea.evalModulesFile, 152 | "--arg", 153 | "pkgs", 154 | toS ea.pkgsExpr 155 | ] 156 | 157 | getEvalCompositionFile :: IO FilePath 158 | getEvalCompositionFile = getDataFileName "nix/eval-composition.nix" 159 | 160 | modeArguments :: EvaluationMode -> [[Char]] 161 | modeArguments ReadWrite = ["--read-write-mode"] 162 | modeArguments ReadOnly = ["--readonly-mode"] 163 | 164 | modulesNixExpr :: NonEmpty FilePath -> [Char] 165 | modulesNixExpr = 166 | NE.toList >>> fmap pathExpr >>> Data.String.unwords >>> wrapList 167 | where 168 | pathExpr :: FilePath -> [Char] 169 | pathExpr path 170 | | isAbsolute path = "(/. + \"/${" <> toNixStringLiteral path <> "}\")" 171 | | otherwise = "(./. + \"/${" <> toNixStringLiteral path <> "}\")" 172 | 173 | isAbsolute ('/' : _) = True 174 | isAbsolute _ = False 175 | 176 | wrapList s = "[ " <> s <> " ]" 177 | 178 | toNixStringLiteral :: [Char] -> [Char] 179 | toNixStringLiteral = show -- FIXME: custom escaping including '$' 180 | -------------------------------------------------------------------------------- /src/haskell/lib/Arion/Services.hs: -------------------------------------------------------------------------------- 1 | {-# LANGUAGE CPP #-} 2 | 3 | module Arion.Services 4 | ( getDefaultExec, 5 | ) 6 | where 7 | 8 | import qualified Data.Aeson as Aeson 9 | import Protolude hiding (to) 10 | import Prelude () 11 | #if MIN_VERSION_lens_aeson(1,2,0) 12 | import qualified Data.Aeson.Key as AK 13 | #endif 14 | import Arion.Aeson (decodeFile) 15 | import Control.Lens 16 | import Data.Aeson.Lens 17 | 18 | #if MIN_VERSION_lens_aeson(1,2,0) 19 | type Key = AK.Key 20 | mkKey :: Text -> Key 21 | mkKey = AK.fromText 22 | #else 23 | type Key = Text 24 | mkKey :: Text -> Key 25 | mkKey = identity 26 | #endif 27 | 28 | -- | Subject to change 29 | getDefaultExec :: FilePath -> Text -> IO [Text] 30 | getDefaultExec fp service = do 31 | v <- decodeFile fp 32 | 33 | pure ((v :: Aeson.Value) ^.. key "x-arion" . key "serviceInfo" . key (mkKey service) . key "defaultExec" . _Array . traverse . _String) 34 | -------------------------------------------------------------------------------- /src/haskell/test/Arion/NixSpec.hs: -------------------------------------------------------------------------------- 1 | module Arion.NixSpec 2 | ( spec, 3 | ) 4 | where 5 | 6 | import Arion.Aeson 7 | import Arion.Nix 8 | import qualified Data.List.NonEmpty as NEL 9 | import qualified Data.Text as T 10 | import qualified Data.Text.IO as T 11 | import Protolude 12 | import Test.Hspec 13 | 14 | spec :: Spec 15 | spec = describe "evaluateComposition" $ do 16 | it "matches an example" $ do 17 | x <- 18 | Arion.Nix.evaluateComposition 19 | EvaluationArgs 20 | { posixUID = 123, 21 | evalModulesFile = 22 | NEL.fromList 23 | ["src/haskell/testdata/Arion/NixSpec/arion-compose.nix"], 24 | pkgsExpr = "import { system = \"x86_64-linux\"; }", 25 | workDir = Nothing, 26 | mode = ReadOnly, 27 | extraNixArgs = ["--show-trace"] 28 | } 29 | let actual = pretty x 30 | expected <- T.readFile "src/haskell/testdata/Arion/NixSpec/arion-compose.json" 31 | censorPaths actual `shouldBe` censorPaths expected 32 | 33 | it "matches an build.context example" $ do 34 | x <- 35 | Arion.Nix.evaluateComposition 36 | EvaluationArgs 37 | { posixUID = 1234, 38 | evalModulesFile = 39 | NEL.fromList 40 | ["src/haskell/testdata/Arion/NixSpec/arion-context-compose.nix"], 41 | pkgsExpr = "import { system = \"x86_64-linux\"; }", 42 | workDir = Nothing, 43 | mode = ReadOnly, 44 | extraNixArgs = ["--show-trace"] 45 | } 46 | let actual = pretty x 47 | expected <- T.readFile "src/haskell/testdata/Arion/NixSpec/arion-context-compose.json" 48 | censorPaths actual `shouldBe` censorPaths expected 49 | 50 | censorPaths :: Text -> Text 51 | censorPaths = censorImages . censorStorePaths 52 | 53 | censorStorePaths :: Text -> Text 54 | censorStorePaths x = case T.breakOn "/nix/store/" x of 55 | (prefix, tl) | (tl :: Text) == "" -> prefix 56 | (prefix, tl) -> 57 | prefix 58 | <> "" 59 | <> censorPaths 60 | (T.dropWhile isNixNameChar $ T.drop (T.length "/nix/store/") tl) 61 | 62 | -- Probably slow, due to not O(1) <> 63 | censorImages :: Text -> Text 64 | censorImages x = case T.break (\c -> c == ':' || c == '"') x of 65 | (prefix, tl) | tl == "" -> prefix 66 | (prefix, tl) 67 | | let imageId = T.take 33 (T.drop 1 tl), 68 | T.last imageId == '\"', 69 | -- Approximation of nix hash validation 70 | T.all (\c -> (c >= '0' && c <= '9') || (c >= 'a' && c <= 'z')) (T.take 32 imageId) -> 71 | prefix <> T.take 1 tl <> "" <> censorImages (T.drop 33 tl) 72 | (prefix, tl) -> prefix <> T.take 1 tl <> censorImages (T.drop 1 tl) 73 | 74 | -- | WARNING: THIS IS LIKELY WRONG: DON'T REUSE 75 | isNixNameChar :: Char -> Bool 76 | isNixNameChar c | c >= '0' && c <= '9' = True 77 | isNixNameChar c | c >= 'a' && c <= 'z' = True 78 | isNixNameChar c | c >= 'A' && c <= 'Z' = True 79 | isNixNameChar c | c == '-' = True 80 | isNixNameChar c | c == '.' = True 81 | isNixNameChar c | c == '_' = True -- WRONG? 82 | isNixNameChar _ = False -- WRONG? 83 | -------------------------------------------------------------------------------- /src/haskell/test/Spec.hs: -------------------------------------------------------------------------------- 1 | module Spec 2 | ( spec, 3 | ) 4 | where 5 | 6 | import qualified Arion.NixSpec 7 | import Test.Hspec 8 | 9 | spec :: Spec 10 | spec = do 11 | describe "Arion.Nix" Arion.NixSpec.spec 12 | -------------------------------------------------------------------------------- /src/haskell/test/TestMain.hs: -------------------------------------------------------------------------------- 1 | module Main where 2 | 3 | import Protolude 4 | import qualified Spec 5 | import Test.Hspec.Runner 6 | import Prelude () 7 | 8 | main :: IO () 9 | main = hspecWith config Spec.spec 10 | where 11 | config = defaultConfig {configColorMode = ColorAlways} 12 | -------------------------------------------------------------------------------- /src/haskell/testdata/Arion/NixSpec/arion-compose.json: -------------------------------------------------------------------------------- 1 | { 2 | "networks": { 3 | "default": { 4 | "name": "unit-test-data" 5 | } 6 | }, 7 | "services": { 8 | "webserver": { 9 | "command": [ 10 | "/usr/sbin/init" 11 | ], 12 | "environment": { 13 | "NIX_REMOTE": "", 14 | "PATH": "/usr/bin:/run/current-system/sw/bin/", 15 | "container": "docker" 16 | }, 17 | "image": "localhost/webserver:", 18 | "ports": [ 19 | "8000:80" 20 | ], 21 | "stop_signal": "SIGRTMIN+3", 22 | "sysctls": {}, 23 | "tmpfs": [ 24 | "/run", 25 | "/run/wrappers", 26 | "/tmp:exec,mode=777" 27 | ], 28 | "tty": true, 29 | "volumes": [ 30 | "/sys/fs/cgroup:/sys/fs/cgroup:ro", 31 | "/nix/store:/nix/store:ro" 32 | ] 33 | } 34 | }, 35 | "version": "3.4", 36 | "volumes": {}, 37 | "x-arion": { 38 | "images": [ 39 | { 40 | "imageExe": "", 41 | "imageName": "localhost/webserver", 42 | "imageTag": "" 43 | } 44 | ], 45 | "project": { 46 | "name": "unit-test-data" 47 | }, 48 | "serviceInfo": { 49 | "webserver": { 50 | "defaultExec": [ 51 | "/run/current-system/sw/bin/bash", 52 | "-l" 53 | ] 54 | } 55 | } 56 | } 57 | } 58 | -------------------------------------------------------------------------------- /src/haskell/testdata/Arion/NixSpec/arion-compose.nix: -------------------------------------------------------------------------------- 1 | { 2 | project.name = "unit-test-data"; 3 | services.webserver = { pkgs, ... }: { 4 | nixos.useSystemd = true; 5 | nixos.configuration.boot.tmp.useTmpfs = true; 6 | nixos.configuration.services.nginx.enable = true; 7 | nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual"; 8 | service.useHostStore = true; 9 | service.ports = [ 10 | "8000:80" # host:container 11 | ]; 12 | }; 13 | } 14 | -------------------------------------------------------------------------------- /src/haskell/testdata/Arion/NixSpec/arion-context-compose.json: -------------------------------------------------------------------------------- 1 | { 2 | "networks": { 3 | "default": { 4 | "name": "unit-test-data" 5 | } 6 | }, 7 | "services": { 8 | "webserver": { 9 | "build": { 10 | "context": "" 11 | }, 12 | "environment": {}, 13 | "ports": [ 14 | "8080:80" 15 | ], 16 | "sysctls": {}, 17 | "volumes": [] 18 | } 19 | }, 20 | "version": "3.4", 21 | "volumes": {}, 22 | "x-arion": { 23 | "images": [ 24 | { 25 | "imageExe": "", 26 | "imageName": "localhost/webserver", 27 | "imageTag": "" 28 | } 29 | ], 30 | "project": { 31 | "name": "unit-test-data" 32 | }, 33 | "serviceInfo": { 34 | "webserver": { 35 | "defaultExec": [ 36 | "/bin/sh" 37 | ] 38 | } 39 | } 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /src/haskell/testdata/Arion/NixSpec/arion-context-compose.nix: -------------------------------------------------------------------------------- 1 | { 2 | project.name = "unit-test-data"; 3 | services.webserver.service = { 4 | build.context = "${./build-context}"; 5 | ports = [ 6 | "8080:80" 7 | ]; 8 | }; 9 | } 10 | -------------------------------------------------------------------------------- /src/haskell/testdata/Arion/NixSpec/build-context/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM nginx 2 | 3 | RUN echo this is a dockerfile to be built 4 | 5 | -------------------------------------------------------------------------------- /src/haskell/testdata/docker-compose-example.json: -------------------------------------------------------------------------------- 1 | { 2 | "services": { 3 | "webserver": { 4 | "environment": { 5 | "container": "docker" 6 | }, 7 | "image": "webserver:xr4ljmz3qfcwlq9rl4mr4qdrzw93rl70", 8 | "ports": [ 9 | "8000:80" 10 | ], 11 | "stop_signal": "SIGRTMIN+3", 12 | "sysctls": {}, 13 | "tmpfs": [ 14 | "/run", 15 | "/run/wrappers", 16 | "/tmp:exec,mode=777" 17 | ], 18 | "tty": true, 19 | "volumes": [ 20 | "/sys/fs/cgroup:/sys/fs/cgroup:ro" 21 | ] 22 | } 23 | }, 24 | "version": "3.4", 25 | "x-arion": { 26 | "images": [ 27 | { 28 | "image": "/nix/store/xr4ljmz3qfcwlq9rl4mr4qdrzw93rl70-docker-image-webserver.tar.gz", 29 | "imageName": "webserver", 30 | "imageTag": "xr4ljmz3qfcwlq9rl4mr4qdrzw93rl70" 31 | } 32 | ], 33 | "project": { 34 | "name": null 35 | }, 36 | "serviceInfo": { 37 | "webserver": { 38 | "defaultExec": [ 39 | "/run/current-system/sw/bin/bash", 40 | "-l" 41 | ] 42 | } 43 | } 44 | } 45 | } 46 | -------------------------------------------------------------------------------- /src/nix/eval-composition.nix: -------------------------------------------------------------------------------- 1 | { modules ? [], uid ? "0", pkgs, hostNixStorePrefix ? "", }: 2 | 3 | let _pkgs = pkgs; 4 | in 5 | let 6 | pkgs = if builtins.typeOf _pkgs == "path" 7 | then import _pkgs 8 | else if builtins.typeOf _pkgs == "set" 9 | then _pkgs 10 | else builtins.abort "The pkgs argument must be an attribute set or a path to an attribute set."; 11 | 12 | inherit (pkgs) lib; 13 | 14 | composition = lib.evalModules { 15 | modules = builtinModules ++ modules; 16 | }; 17 | 18 | builtinModules = [ 19 | argsModule 20 | ] ++ import ./modules.nix; 21 | 22 | argsModule = { 23 | _file = ./eval-composition.nix; 24 | key = ./eval-composition.nix; 25 | config._module.args.pkgs = lib.mkIf (pkgs != null) (lib.mkForce pkgs); 26 | config._module.args.check = true; 27 | config.host.nixStorePrefix = hostNixStorePrefix; 28 | config.host.uid = lib.toInt uid; 29 | }; 30 | 31 | in 32 | # Typically you need composition.config.out.dockerComposeYaml 33 | composition // { 34 | # throw in lib and pkgs for repl convenience 35 | inherit lib; 36 | inherit (composition._module.args) pkgs; 37 | } 38 | -------------------------------------------------------------------------------- /src/nix/lib.nix: -------------------------------------------------------------------------------- 1 | { lib }: 2 | let 3 | 4 | link = url: text: ''[${text}](${url})''; 5 | 6 | composeSpecRev = "55b450aee50799a2f33cc99e1d714518babe305e"; 7 | 8 | serviceRef = fragment: 9 | ''See ${link "https://github.com/compose-spec/compose-spec/blob/${composeSpecRev}/05-services.md#${fragment}" "Compose Spec Services #${fragment}"}''; 10 | 11 | networkRef = fragment: 12 | ''See ${link "https://github.com/compose-spec/compose-spec/blob/${composeSpecRev}/06-networks.md#${fragment}" "Compose Spec Networks #${fragment}"}''; 13 | 14 | in 15 | { 16 | inherit 17 | link 18 | networkRef 19 | serviceRef 20 | ; 21 | } 22 | -------------------------------------------------------------------------------- /src/nix/modules.nix: -------------------------------------------------------------------------------- 1 | [ 2 | ./modules/composition/docker-compose.nix 3 | ./modules/composition/host-environment.nix 4 | ./modules/composition/images.nix 5 | ./modules/composition/networks.nix 6 | ./modules/composition/service-info.nix 7 | ./modules/composition/composition.nix 8 | ] -------------------------------------------------------------------------------- /src/nix/modules/composition/composition.nix: -------------------------------------------------------------------------------- 1 | { config, lib, ... }: 2 | let 3 | inherit (lib) types mkOption; 4 | 5 | link = url: text: 6 | ''[${text}](${url})''; 7 | 8 | in 9 | { 10 | options = { 11 | _module.args = mkOption { 12 | internal = true; 13 | }; 14 | project.name = mkOption { 15 | description = '' 16 | Name of the project. 17 | 18 | See ${link "https://docs.docker.com/compose/reference/envvars/#compose_project_name" "COMPOSE_PROJECT_NAME"} 19 | 20 | This is not optional, because getting the project name from a directory name tends to produce different results for different repo checkout location names. 21 | ''; 22 | type = types.str; 23 | }; 24 | }; 25 | config = { 26 | docker-compose.extended.project.name = config.project.name; 27 | }; 28 | } 29 | -------------------------------------------------------------------------------- /src/nix/modules/composition/docker-compose.nix: -------------------------------------------------------------------------------- 1 | /* 2 | 3 | This is a composition-level module. 4 | 5 | It defines the low-level options that are read by arion, like 6 | - out.dockerComposeYaml 7 | 8 | It declares options like 9 | - services 10 | 11 | */ 12 | compositionArgs@{ lib, config, options, pkgs, ... }: 13 | let 14 | inherit (lib) types; 15 | 16 | service = { 17 | imports = [ argsModule ] ++ import ../service/all-modules.nix; 18 | }; 19 | argsModule = 20 | { name, # injected by types.submodule 21 | ... 22 | }: { 23 | _file = ./docker-compose.nix; 24 | key = ./docker-compose.nix; 25 | 26 | config._module.args.pkgs = lib.mkDefault compositionArgs.pkgs; 27 | config.host = compositionArgs.config.host; 28 | config.composition = compositionArgs.config; 29 | config.service.name = name; 30 | }; 31 | 32 | in 33 | { 34 | imports = [ 35 | ../lib/assert.nix 36 | (lib.mkRenamedOptionModule ["docker-compose" "services"] ["services"]) 37 | ]; 38 | options = { 39 | out.dockerComposeYaml = lib.mkOption { 40 | type = lib.types.package; 41 | description = "A derivation that produces a docker-compose.yaml file for this composition."; 42 | readOnly = true; 43 | }; 44 | out.dockerComposeYamlText = lib.mkOption { 45 | type = lib.types.str; 46 | description = "The text of out.dockerComposeYaml."; 47 | readOnly = true; 48 | }; 49 | out.dockerComposeYamlAttrs = lib.mkOption { 50 | type = lib.types.attrsOf lib.types.unspecified; 51 | description = "The text of out.dockerComposeYaml."; 52 | readOnly = true; 53 | }; 54 | docker-compose.raw = lib.mkOption { 55 | type = lib.types.attrs; 56 | description = "Attribute set that will be turned into the docker-compose.yaml file, using Nix's toJSON builtin."; 57 | }; 58 | docker-compose.extended = lib.mkOption { 59 | type = lib.types.attrs; 60 | description = "Attribute set that will be turned into the x-arion section of the docker-compose.yaml file."; 61 | }; 62 | services = lib.mkOption { 63 | type = lib.types.attrsOf (lib.types.submodule service); 64 | description = "An attribute set of service configurations. A service specifies how to run an image as a container."; 65 | }; 66 | docker-compose.volumes = lib.mkOption { 67 | type = lib.types.attrsOf lib.types.unspecified; 68 | description = "A attribute set of volume configurations."; 69 | default = {}; 70 | }; 71 | }; 72 | config = { 73 | out.dockerComposeYaml = pkgs.writeText "docker-compose.yaml" config.out.dockerComposeYamlText; 74 | out.dockerComposeYamlText = builtins.toJSON (config.out.dockerComposeYamlAttrs); 75 | out.dockerComposeYamlAttrs = config.assertWarn config.docker-compose.raw; 76 | 77 | docker-compose.raw = { 78 | version = "3.4"; 79 | services = lib.mapAttrs (k: c: c.out.service) config.services; 80 | x-arion = config.docker-compose.extended; 81 | volumes = config.docker-compose.volumes; 82 | }; 83 | }; 84 | } 85 | -------------------------------------------------------------------------------- /src/nix/modules/composition/host-environment.nix: -------------------------------------------------------------------------------- 1 | { lib, ... }: 2 | 3 | { 4 | options = { 5 | 6 | host.uid = lib.mkOption { 7 | type = lib.types.int; 8 | description = '' 9 | The numeric user id (UID) of the user running arion. 10 | 11 | This lets you to write modules that interact with the host 12 | user's files, which is helpful for local development, but not 13 | intended for production-like deployment scenarios. 14 | ''; 15 | }; 16 | 17 | host.nixStorePrefix = lib.mkOption { 18 | type = lib.types.str; 19 | default = ""; 20 | example = "/mnt/foo"; 21 | description = '' 22 | Prefixes store paths on the host, allowing the Nix store to be 23 | stored at an alternate location without altering the format of 24 | store paths. 25 | 26 | For example: instead of mounting the host's `/nix/store` as the 27 | container's `/nix/store`, this will mount `/mnt/foo/nix/store` 28 | as the container's `/nix/store`. 29 | ''; 30 | }; 31 | 32 | }; 33 | } 34 | -------------------------------------------------------------------------------- /src/nix/modules/composition/images.nix: -------------------------------------------------------------------------------- 1 | { pkgs, lib, config, ... }: 2 | let 3 | inherit (lib.types) listOf package unspecified; 4 | 5 | serviceImages = 6 | lib.mapAttrs addDetails ( 7 | lib.filterAttrs filterFunction config.services 8 | ); 9 | 10 | filterFunction = serviceName: service: 11 | builtins.addErrorContext "while evaluating whether the service ${serviceName} defines an image" 12 | service.image.nixBuild; 13 | 14 | addDetails = serviceName: service: 15 | builtins.addErrorContext "while evaluating the image for service ${serviceName}" 16 | (let 17 | inherit (service) build; 18 | in { 19 | imageName = build.imageName or service.image.name; 20 | imageTag = 21 | if build.image.imageTag != "" 22 | then build.image.imageTag 23 | else lib.head (lib.strings.splitString "-" (baseNameOf build.image.outPath)); 24 | } // (if build.image.isExe or false 25 | then { 26 | imageExe = build.image.outPath; 27 | } 28 | else { 29 | image = build.image.outPath; 30 | } 31 | ) 32 | ); 33 | in 34 | { 35 | options = { 36 | build.imagesToLoad = lib.mkOption { 37 | type = listOf unspecified; 38 | internal = true; 39 | description = "List of `dockerTools` image derivations."; 40 | }; 41 | }; 42 | config = { 43 | build.imagesToLoad = lib.attrValues serviceImages; 44 | docker-compose.extended.images = config.build.imagesToLoad; 45 | }; 46 | } 47 | -------------------------------------------------------------------------------- /src/nix/modules/composition/networks.nix: -------------------------------------------------------------------------------- 1 | { config, lib, ... }: 2 | 3 | let 4 | inherit (lib) 5 | mkOption 6 | optionalAttrs 7 | types 8 | ; 9 | inherit (import ../../lib.nix { inherit lib; }) 10 | link 11 | ; 12 | in 13 | { 14 | options = { 15 | networks = mkOption { 16 | type = types.lazyAttrsOf (types.submoduleWith { 17 | modules = [ 18 | ../networks/network.nix 19 | ]; 20 | }); 21 | description = '' 22 | See ${link "https://docs.docker.com/compose/compose-file/06-networks/" "Docker Compose Networks"} 23 | ''; 24 | }; 25 | enableDefaultNetwork = mkOption { 26 | type = types.bool; 27 | description = '' 28 | Whether to define the default network: 29 | 30 | ```nix 31 | networks.default = { 32 | name = config.project.name; 33 | }; 34 | ``` 35 | ''; 36 | default = true; 37 | }; 38 | }; 39 | 40 | 41 | config = { 42 | 43 | networks = optionalAttrs config.enableDefaultNetwork { 44 | default = { 45 | name = config.project.name; 46 | }; 47 | }; 48 | 49 | docker-compose.raw.networks = 50 | lib.mapAttrs (k: v: v.out) config.networks; 51 | 52 | }; 53 | } 54 | -------------------------------------------------------------------------------- /src/nix/modules/composition/service-info.nix: -------------------------------------------------------------------------------- 1 | /* 2 | Adds extendedInfo from services to the Docker Compose file. 3 | 4 | This contains fields that are not in Docker Compose schema. 5 | */ 6 | { config, lib, ... }: 7 | let 8 | inherit (lib) mapAttrs filterAttrs; 9 | 10 | serviceInfo = 11 | filterAttrs (_k: v: v != {}) 12 | (mapAttrs (_serviceName: service: service.out.extendedInfo) 13 | config.services 14 | ); 15 | 16 | in 17 | { 18 | config = { 19 | docker-compose.extended.serviceInfo = serviceInfo; 20 | }; 21 | } 22 | -------------------------------------------------------------------------------- /src/nix/modules/lib/README.md: -------------------------------------------------------------------------------- 1 | `assertions.nix`: Nixpkgs 2 | `assert.nix`: extracted from Nixpkgs, adapted 3 | -------------------------------------------------------------------------------- /src/nix/modules/lib/assert.nix: -------------------------------------------------------------------------------- 1 | { config, lib, pkgs, ... }: 2 | 3 | # based on nixpkgs/nixos/modules/system/activation/top-level.nix 4 | 5 | let 6 | inherit (lib) 7 | concatStringsSep 8 | filter 9 | mkOption 10 | showWarnings 11 | types 12 | ; 13 | 14 | # Handle assertions and warnings 15 | failedAssertions = map (x: x.message) (filter (x: !x.assertion) config.assertions); 16 | 17 | assertWarn = if failedAssertions != [] 18 | then throw "\nFailed assertions:\n${concatStringsSep "\n" (map (x: "- ${x}") failedAssertions)}" 19 | else showWarnings config.warnings; 20 | 21 | in 22 | 23 | { 24 | imports = [ ./assertions.nix ]; 25 | options.assertWarn = mkOption { 26 | type = types.unspecified; # a function 27 | # It's for the wrapping program to know about this. User need not care. 28 | internal = true; 29 | readOnly = true; 30 | }; 31 | config = { inherit assertWarn; }; 32 | } 33 | 34 | -------------------------------------------------------------------------------- /src/nix/modules/lib/assertions.nix: -------------------------------------------------------------------------------- 1 | { lib, ... }: 2 | 3 | with lib; 4 | 5 | { 6 | 7 | options = { 8 | 9 | assertions = mkOption { 10 | type = types.listOf types.unspecified; 11 | internal = true; 12 | default = []; 13 | example = [ { assertion = false; message = "you can't enable this for that reason"; } ]; 14 | description = '' 15 | This option allows modules to express conditions that must 16 | hold for the evaluation of the system configuration to 17 | succeed, along with associated error messages for the user. 18 | ''; 19 | }; 20 | 21 | warnings = mkOption { 22 | internal = true; 23 | default = []; 24 | type = types.listOf types.str; 25 | example = [ "The `foo' service is deprecated and will go away soon!" ]; 26 | description = '' 27 | This option allows modules to show warnings to users during 28 | the evaluation of the system configuration. 29 | ''; 30 | }; 31 | 32 | }; 33 | # impl of assertions is in 34 | } 35 | -------------------------------------------------------------------------------- /src/nix/modules/networks/network.nix: -------------------------------------------------------------------------------- 1 | { config, lib, options, ... }: 2 | 3 | let 4 | inherit (lib) 5 | mkOption 6 | optionalAttrs 7 | types 8 | ; 9 | inherit (import ../../lib.nix { inherit lib; }) 10 | networkRef 11 | ; 12 | in 13 | { 14 | options = { 15 | driver = mkOption { 16 | description = '' 17 | `"none"`, `"host"`, or a platform-specific value. 18 | ${networkRef "driver"} 19 | ''; 20 | type = types.str; 21 | }; 22 | 23 | driver_opts = mkOption { 24 | description = '' 25 | ${networkRef "driver_opts"} 26 | ''; 27 | type = types.lazyAttrsOf types.raw or types.unspecified; 28 | }; 29 | 30 | attachable = mkOption { 31 | description = '' 32 | ${networkRef "attachable"} 33 | ''; 34 | type = types.bool; 35 | example = true; 36 | }; 37 | 38 | enable_ipv6 = mkOption { 39 | description = '' 40 | Whether we've entered the 21st century yet. 41 | 42 | ${networkRef "enable_ipv6"} 43 | ''; 44 | type = types.bool; 45 | }; 46 | 47 | ipam = mkOption { 48 | # TODO model sub-options 49 | description = '' 50 | Manage IP addresses. 51 | 52 | ${networkRef "ipam"} 53 | ''; 54 | type = types.raw or types.unspecified; 55 | }; 56 | 57 | internal = mkOption { 58 | description = '' 59 | Achieves "external isolation". 60 | 61 | ${networkRef "internal"} 62 | ''; 63 | defaultText = false; 64 | type = types.bool; 65 | }; 66 | 67 | labels = mkOption { 68 | description = '' 69 | Metadata. 70 | 71 | ${networkRef "labels"} 72 | ''; 73 | # no list support, because less expressive wrt overriding 74 | type = types.attrsOf types.str; 75 | }; 76 | 77 | external = mkOption { 78 | description = '' 79 | When `true`, don't create or destroy the network, but assume that it 80 | exists. 81 | 82 | ${networkRef "external"} 83 | ''; 84 | type = types.bool; 85 | }; 86 | 87 | name = mkOption { 88 | description = '' 89 | Set a custom name for the network. 90 | 91 | It shares a namespace with other projects' networks. `name` is used as-is. 92 | 93 | Note the `default` network's default `name` is set to `project.name` by Arion. 94 | 95 | ${networkRef "name"} 96 | ''; 97 | type = types.str; 98 | }; 99 | 100 | out = mkOption { 101 | internal = true; 102 | description = '' 103 | This network's contribution to the docker compose yaml file 104 | under the `networks.''${name}` key. 105 | ''; 106 | type = lib.types.attrsOf lib.types.raw or lib.types.unspecified; 107 | }; 108 | }; 109 | 110 | config = { 111 | out = 112 | lib.mapAttrs 113 | (k: opt: opt.value) 114 | (lib.filterAttrs 115 | (k: opt: opt.isDefined) 116 | { 117 | inherit (options) 118 | driver 119 | driver_opts 120 | attachable 121 | enable_ipv6 122 | ipam 123 | internal 124 | labels 125 | external 126 | name 127 | ; 128 | } 129 | ); 130 | }; 131 | } 132 | -------------------------------------------------------------------------------- /src/nix/modules/nixos/container-systemd.nix: -------------------------------------------------------------------------------- 1 | /* 2 | NixOS configuration to for running a mostly normal systemd-based 3 | NixOS in Docker. 4 | */ 5 | { pkgs, lib, ...}: { 6 | 7 | # imports = [ 8 | # # This profile doesn't seem to work well. 9 | # (pkgs.path + "/nixos/modules/profiles/docker-container.nix") 10 | # This one works, but can not be imported here due because imports can not depend on pkgs. 11 | # (pkgs.path + "/nixos/modules/profiles/minimal.nix") 12 | # ]; 13 | boot.isContainer = true; 14 | boot.specialFileSystems = lib.mkForce {}; 15 | networking.hostName = ""; 16 | 17 | services.journald.console = "/dev/console"; 18 | 19 | systemd.services.systemd-logind.enable = false; 20 | systemd.services.console-getty.enable = false; 21 | 22 | systemd.sockets.nix-daemon.enable = lib.mkDefault false; 23 | systemd.services.nix-daemon.enable = lib.mkDefault false; 24 | } 25 | -------------------------------------------------------------------------------- /src/nix/modules/nixos/default-shell.nix: -------------------------------------------------------------------------------- 1 | { config, utils, ... }: 2 | { 3 | config.system.build.x-arion-defaultShell = utils.toShellPath config.users.defaultUserShell; 4 | } 5 | -------------------------------------------------------------------------------- /src/nix/modules/service/all-modules.nix: -------------------------------------------------------------------------------- 1 | [ 2 | ./default-exec.nix 3 | ./docker-compose-service.nix 4 | ./extended-info.nix 5 | ./host-store.nix 6 | ./context.nix 7 | ./image.nix 8 | ./image-recommended.nix 9 | ./nixos.nix 10 | ./nixos-init.nix 11 | ../lib/assert.nix 12 | ./check-sys_admin.nix 13 | ] 14 | -------------------------------------------------------------------------------- /src/nix/modules/service/check-sys_admin.nix: -------------------------------------------------------------------------------- 1 | { config, lib, name, ... }: 2 | let 3 | inherit (lib) 4 | concatStringsSep 5 | optional 6 | ; 7 | 8 | dynamicUserServices = lib.attrNames ( 9 | lib.filterAttrs 10 | (k: v: 11 | v.enable && 12 | v.serviceConfig.DynamicUser or false) 13 | config.nixos.evaluatedConfig.systemd.services 14 | ); 15 | 16 | 17 | in 18 | { 19 | config = { 20 | warnings = 21 | optional (config.nixos.useSystemd && !(config.service.capabilities.SYS_ADMIN or false) && dynamicUserServices != []) ( 22 | ''In service ${name}, the following units require `SYS_ADMIN` capability 23 | because of DynamicUser. 24 | ${concatStringsSep "\n" (map (srv: " - services.${name}.nixos.configuration.systemd.services.${srv}") dynamicUserServices)} 25 | You can avoid DynamicUser or use 26 | services.${name}.service.capabilities.SYS_ADMIN = true; 27 | '' 28 | ); 29 | }; 30 | } -------------------------------------------------------------------------------- /src/nix/modules/service/context.nix: -------------------------------------------------------------------------------- 1 | { lib, ... }: 2 | { 3 | options = { 4 | host = lib.mkOption { 5 | type = lib.types.attrs; 6 | readOnly = true; 7 | description = '' 8 | The composition-level host option values. 9 | ''; 10 | }; 11 | composition = lib.mkOption { 12 | type = lib.types.attrs; 13 | readOnly = true; 14 | description = '' 15 | The composition configuration. 16 | ''; 17 | }; 18 | }; 19 | } 20 | -------------------------------------------------------------------------------- /src/nix/modules/service/default-exec.nix: -------------------------------------------------------------------------------- 1 | { config, lib, ... }: 2 | let 3 | inherit (lib) types mkOption; 4 | in 5 | { 6 | options = { 7 | service.defaultExec = mkOption { 8 | type = types.listOf types.str; 9 | default = ["/bin/sh"]; 10 | description = '' 11 | Container program and arguments to invoke when calling 12 | `arion exec ` without further arguments. 13 | ''; 14 | }; 15 | }; 16 | config = { 17 | out.extendedInfo.defaultExec = config.service.defaultExec; 18 | }; 19 | } -------------------------------------------------------------------------------- /src/nix/modules/service/docker-compose-service.nix: -------------------------------------------------------------------------------- 1 | /* 2 | 3 | This service-level module defines the out.service option, using 4 | the user-facing options service.image, service.volumes, etc. 5 | 6 | */ 7 | { pkgs, lib, config, options, ... }: 8 | 9 | let 10 | inherit (lib) mkOption types; 11 | inherit (types) listOf nullOr attrsOf str either int bool submodule enum; 12 | 13 | inherit (import ../../lib.nix { inherit lib; }) 14 | link 15 | serviceRef 16 | ; 17 | 18 | cap_add = lib.attrNames (lib.filterAttrs (name: value: value == true) config.service.capabilities); 19 | cap_drop = lib.attrNames (lib.filterAttrs (name: value: value == false) config.service.capabilities); 20 | 21 | in 22 | { 23 | imports = [ 24 | (lib.mkRenamedOptionModule ["build" "service"] ["out" "service"]) 25 | ]; 26 | 27 | options = { 28 | out.service = mkOption { 29 | type = attrsOf types.unspecified; 30 | description = '' 31 | Raw input for the service in `docker-compose.yaml`. 32 | 33 | You should not need to use this option. If anything is 34 | missing, please contribute the missing option. 35 | 36 | This option is user accessible because it may serve as an 37 | escape hatch for some. 38 | ''; 39 | apply = config.assertWarn; 40 | }; 41 | 42 | service.name = mkOption { 43 | type = str; 44 | description = '' 45 | The name of the service - `` in the composition-level `services.` 46 | ''; 47 | readOnly = true; 48 | }; 49 | 50 | service.volumes = mkOption { 51 | type = listOf types.unspecified; 52 | default = []; 53 | description = serviceRef "volumes"; 54 | }; 55 | service.tmpfs = mkOption { 56 | type = listOf types.str; 57 | default = []; 58 | description = serviceRef "tmpfs"; 59 | }; 60 | service.build.context = mkOption { 61 | type = nullOr str; 62 | default = null; 63 | description = '' 64 | Locates a Dockerfile to use for creating an image to use in this service. 65 | 66 | https://docs.docker.com/compose/compose-file/build/#context 67 | ''; 68 | }; 69 | service.build.dockerfile = mkOption { 70 | type = nullOr str; 71 | default = null; 72 | description = '' 73 | Sets an alternate Dockerfile. A relative path is resolved from the build context. 74 | https://docs.docker.com/compose/compose-file/build/#dockerfile 75 | ''; 76 | }; 77 | service.build.target = mkOption { 78 | type = nullOr str; 79 | default = null; 80 | description = '' 81 | Defines the stage to build as defined inside a multi-stage Dockerfile. 82 | https://docs.docker.com/compose/compose-file/build/#target 83 | ''; 84 | }; 85 | service.hostname = mkOption { 86 | type = nullOr str; 87 | default = null; 88 | description = '' 89 | ${serviceRef "hostname"} 90 | ''; 91 | }; 92 | service.tty = mkOption { 93 | type = nullOr bool; 94 | default = null; 95 | description = '' 96 | ${serviceRef "tty"} 97 | ''; 98 | }; 99 | service.environment = mkOption { 100 | type = attrsOf (either str int); 101 | default = {}; 102 | description = serviceRef "environment"; 103 | }; 104 | service.image = mkOption { 105 | type = nullOr str; 106 | default = null; 107 | description = serviceRef "image"; 108 | }; 109 | service.command = mkOption { 110 | type = nullOr types.unspecified; 111 | default = null; 112 | description = serviceRef "command"; 113 | }; 114 | service.container_name = mkOption { 115 | type = nullOr types.str; 116 | default = null; 117 | description = serviceRef "container_name"; 118 | }; 119 | service.depends_on = 120 | let conditionsModule = { 121 | options = { 122 | condition = mkOption { 123 | type = enum ["service_started" "service_healthy" "service_completed_successfully"]; 124 | description = serviceRef "depends_on"; 125 | default = "service_started"; 126 | }; 127 | }; 128 | }; 129 | in mkOption { 130 | type = either (listOf str) (attrsOf (submodule conditionsModule)); 131 | default = []; 132 | description = serviceRef "depends_on"; 133 | }; 134 | service.healthcheck = mkOption { 135 | description = serviceRef "healthcheck"; 136 | type = submodule ({ config, options, ...}: { 137 | options = { 138 | _out = mkOption { 139 | internal = true; 140 | default = lib.optionalAttrs (options.test.highestPrio < 1500) { 141 | inherit (config) test interval timeout start_period retries; 142 | }; 143 | }; 144 | test = mkOption { 145 | type = nullOr (listOf str); 146 | default = null; 147 | example = [ "CMD" "pg_isready" ]; 148 | description = serviceRef "healthcheck"; 149 | }; 150 | interval = mkOption { 151 | type = str; 152 | default = "30s"; 153 | example = "1m"; 154 | description = serviceRef "healthcheck"; 155 | }; 156 | timeout = mkOption { 157 | type = str; 158 | default = "30s"; 159 | example = "10s"; 160 | description = serviceRef "healthcheck"; 161 | }; 162 | start_period = mkOption { 163 | type = str; 164 | default = "0s"; 165 | example = "30s"; 166 | description = serviceRef "healthcheck"; 167 | }; 168 | retries = mkOption { 169 | type = int; 170 | default = 3; 171 | description = serviceRef "healthcheck"; 172 | }; 173 | }; 174 | }); 175 | }; 176 | service.devices = mkOption { 177 | type = listOf str; 178 | default = []; 179 | description = '' 180 | See ${link "https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities" 181 | "`docker run --device` documentation"} 182 | 183 | ${serviceRef "devices"} 184 | ''; 185 | }; 186 | service.dns = mkOption { 187 | type = listOf str; 188 | default = []; 189 | example = [ "8.8.8.8" "8.8.4.4" ]; 190 | description = serviceRef "dns"; 191 | }; 192 | service.labels = mkOption { 193 | type = attrsOf str; 194 | default = {}; 195 | example = { 196 | "com.example.foo" = "bar"; 197 | "traefik.enable" = "true"; 198 | "traefik.http.routers.my-service.rule" = "Host(`my-service.localhost`)"; 199 | "traefik.http.routers.my-service.entrypoints" = "web"; 200 | }; 201 | description = serviceRef "labels"; 202 | }; 203 | service.links = mkOption { 204 | type = listOf str; 205 | default = []; 206 | description = serviceRef "links"; 207 | }; 208 | service.external_links = mkOption { 209 | type = listOf str; 210 | default = []; 211 | description = serviceRef "external_links"; 212 | }; 213 | service.extra_hosts = mkOption { 214 | type = listOf str; 215 | default = []; 216 | description = serviceRef "extra_hosts"; 217 | }; 218 | service.working_dir = mkOption { 219 | type = nullOr str; 220 | default = null; 221 | description = '' 222 | ${serviceRef "working_dir"} 223 | ''; 224 | }; 225 | service.privileged = mkOption { 226 | type = nullOr bool; 227 | default = null; 228 | description = '' 229 | ${serviceRef "privileged"} 230 | ''; 231 | }; 232 | service.entrypoint = mkOption { 233 | type = nullOr str; 234 | default = null; 235 | description = serviceRef "entrypoint"; 236 | }; 237 | service.restart = mkOption { 238 | type = nullOr str; 239 | default = null; 240 | description = serviceRef "restart"; 241 | }; 242 | service.user = mkOption { 243 | type = nullOr str; 244 | default = null; 245 | description = '' 246 | ${serviceRef "user"} 247 | ''; 248 | }; 249 | service.ports = mkOption { 250 | type = listOf types.unspecified; 251 | default = []; 252 | description = '' 253 | Expose ports on host. "host:container" or structured. 254 | 255 | ${serviceRef "ports"} 256 | ''; 257 | }; 258 | service.expose = mkOption { 259 | type = listOf str; 260 | default = []; 261 | description = serviceRef "expose"; 262 | }; 263 | service.env_file = mkOption { 264 | type = listOf str; 265 | default = []; 266 | description = serviceRef "env_file"; 267 | }; 268 | service.network_mode = mkOption { 269 | type = nullOr str; 270 | default = null; 271 | description = serviceRef "network_mode"; 272 | }; 273 | service.networks = 274 | let 275 | networksModule = submodule ({ config, options, ...}: { 276 | options = { 277 | _out = mkOption { 278 | internal = true; 279 | readOnly = true; 280 | default = lib.mapAttrs (k: opt: opt.value) (lib.filterAttrs (_: opt: opt.isDefined) { inherit (options) aliases ipv4_address ipv6_address link_local_ips priority; }); 281 | }; 282 | aliases = mkOption { 283 | type = listOf str; 284 | description = serviceRef "aliases"; 285 | default = [ ]; 286 | }; 287 | ipv4_address = mkOption { 288 | type = str; 289 | description = serviceRef "ipv4_address-ipv6_address"; 290 | }; 291 | ipv6_address = mkOption { 292 | type = str; 293 | description = serviceRef "ipv4_address-ipv6_address"; 294 | }; 295 | link_local_ips = mkOption { 296 | type = listOf str; 297 | description = serviceRef "link_local_ips"; 298 | }; 299 | priority = mkOption { 300 | type = int; 301 | description = serviceRef "priority"; 302 | }; 303 | }; 304 | }); 305 | in 306 | mkOption { 307 | type = either (listOf str) (attrsOf networksModule); 308 | default = []; 309 | description = serviceRef "networks"; 310 | }; 311 | service.stop_signal = mkOption { 312 | type = nullOr str; 313 | default = null; 314 | description = serviceRef "stop_signal"; 315 | }; 316 | service.stop_grace_period = mkOption { 317 | type = nullOr str; 318 | default = null; 319 | description = serviceRef "stop_grace_period"; 320 | }; 321 | service.sysctls = mkOption { 322 | type = attrsOf (either str int); 323 | default = {}; 324 | description = serviceRef "sysctls"; 325 | }; 326 | service.capabilities = mkOption { 327 | type = attrsOf (nullOr bool); 328 | default = {}; 329 | example = { ALL = true; SYS_ADMIN = false; NET_ADMIN = false; }; 330 | description = '' 331 | Enable/disable linux capabilities, or pick Docker's default. 332 | 333 | Setting a capability to `true` means that it will be 334 | "added". Setting it to `false` means that it will be "dropped". 335 | 336 | Omitted and `null` capabilities will therefore be set 337 | according to Docker's ${ 338 | link "https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities" 339 | "default list of capabilities." 340 | } 341 | 342 | ${serviceRef "cap_add"} 343 | ${serviceRef "cap_drop"} 344 | ''; 345 | }; 346 | service.blkio_config = mkOption { 347 | default = null; 348 | example = { weight = 300; }; 349 | description = '' 350 | Block IO limits for the service. 351 | 352 | ${serviceRef "blkio_config"} 353 | ''; 354 | type = nullOr ( 355 | submodule ( 356 | { config, options, ... }: 357 | let 358 | deviceBpsModule = submodule ( 359 | { config, options, ... }: 360 | { 361 | options = { 362 | path = mkOption { 363 | type = str; 364 | example = "/dev/sda"; 365 | description = serviceRef "blkio_config"; 366 | }; 367 | rate = mkOption { 368 | type = str; 369 | example = "12mb"; 370 | description = serviceRef "blkio_config"; 371 | }; 372 | }; 373 | } 374 | ); 375 | deviceIopsModule = submodule ( 376 | { config, options, ... }: 377 | { 378 | options = { 379 | path = mkOption { 380 | type = str; 381 | example = "/dev/sda"; 382 | description = serviceRef "blkio_config"; 383 | }; 384 | rate = mkOption { 385 | type = int; 386 | example = 120; 387 | description = serviceRef "blkio_config"; 388 | }; 389 | }; 390 | } 391 | ); 392 | in 393 | { 394 | options = { 395 | _out = mkOption { 396 | internal = true; 397 | readOnly = true; 398 | default = lib.mapAttrs (k: opt: opt.value) (lib.filterAttrs (_: opt: opt.isDefined) { inherit (options) weight weight_device device_read_bps device_write_bps device_read_iops device_write_iops; }); 399 | }; 400 | weight = mkOption { 401 | type = int; 402 | example = 300; 403 | description = serviceRef "blkio_config"; 404 | }; 405 | weight_device = mkOption { 406 | description = serviceRef "blkio_config"; 407 | type = listOf (submodule ({ config, options, ... }: { 408 | options = { 409 | path = mkOption { 410 | type = str; 411 | example = "/dev/sda"; 412 | description = serviceRef "blkio_config"; 413 | }; 414 | weight = mkOption { 415 | type = int; 416 | example = 400; 417 | description = serviceRef "blkio_config"; 418 | }; 419 | }; 420 | })); 421 | }; 422 | device_read_bps = mkOption { 423 | type = listOf (deviceBpsModule); 424 | description = serviceRef "blkio_config"; 425 | }; 426 | device_write_bps = mkOption { 427 | type = listOf (deviceBpsModule); 428 | description = serviceRef "blkio_config"; 429 | }; 430 | device_read_iops = mkOption { 431 | type = listOf (deviceIopsModule); 432 | description = serviceRef "blkio_config"; 433 | }; 434 | device_write_iops = mkOption { 435 | type = listOf (deviceIopsModule); 436 | description = serviceRef "blkio_config"; 437 | }; 438 | }; 439 | } 440 | ) 441 | ); 442 | }; 443 | }; 444 | 445 | config.out.service = { 446 | inherit (config.service) 447 | volumes 448 | environment 449 | sysctls 450 | ; 451 | } // lib.optionalAttrs (config.service.image != null) { 452 | inherit (config.service) image; 453 | } // lib.optionalAttrs (config.service.build.context != null ) { 454 | build = lib.filterAttrs (n: v: v != null) config.service.build; 455 | } // lib.optionalAttrs (cap_add != []) { 456 | inherit cap_add; 457 | } // lib.optionalAttrs (cap_drop != []) { 458 | inherit cap_drop; 459 | } // lib.optionalAttrs (config.service.command != null) { 460 | inherit (config.service) command; 461 | } // lib.optionalAttrs (config.service.container_name != null) { 462 | inherit (config.service) container_name; 463 | } // lib.optionalAttrs (config.service.depends_on != []) { 464 | inherit (config.service) depends_on; 465 | } // lib.optionalAttrs (options.service.healthcheck.highestPrio < 1500) { 466 | healthcheck = config.service.healthcheck._out; 467 | } // lib.optionalAttrs (config.service.devices != []) { 468 | inherit (config.service) devices; 469 | } // lib.optionalAttrs (config.service.entrypoint != null) { 470 | inherit (config.service) entrypoint; 471 | } // lib.optionalAttrs (config.service.env_file != []) { 472 | inherit (config.service) env_file; 473 | } // lib.optionalAttrs (config.service.expose != []) { 474 | inherit (config.service) expose; 475 | } // lib.optionalAttrs (config.service.external_links != []) { 476 | inherit (config.service) external_links; 477 | } // lib.optionalAttrs (config.service.extra_hosts != []) { 478 | inherit (config.service) extra_hosts; 479 | } // lib.optionalAttrs (config.service.hostname != null) { 480 | inherit (config.service) hostname; 481 | } // lib.optionalAttrs (config.service.dns != []) { 482 | inherit (config.service) dns; 483 | } // lib.optionalAttrs (config.service.labels != {}) { 484 | inherit (config.service) labels; 485 | } // lib.optionalAttrs (config.service.links != []) { 486 | inherit (config.service) links; 487 | } // lib.optionalAttrs (config.service.ports != []) { 488 | inherit (config.service) ports; 489 | } // lib.optionalAttrs (config.service.privileged != null) { 490 | inherit (config.service) privileged; 491 | } // lib.optionalAttrs (config.service.network_mode != null) { 492 | inherit (config.service) network_mode; 493 | } // lib.optionalAttrs (config.service.networks != [] && config.service.networks != {}) { 494 | networks = 495 | if (builtins.isAttrs config.service.networks) then builtins.mapAttrs (_: v: v._out) config.service.networks 496 | else config.service.networks; 497 | } // lib.optionalAttrs (config.service.restart != null) { 498 | inherit (config.service) restart; 499 | } // lib.optionalAttrs (config.service.stop_signal != null) { 500 | inherit (config.service) stop_signal; 501 | } // lib.optionalAttrs (config.service.stop_grace_period != null) { 502 | inherit (config.service) stop_grace_period; 503 | } // lib.optionalAttrs (config.service.tmpfs != []) { 504 | inherit (config.service) tmpfs; 505 | } // lib.optionalAttrs (config.service.tty != null) { 506 | inherit (config.service) tty; 507 | } // lib.optionalAttrs (config.service.working_dir != null) { 508 | inherit (config.service) working_dir; 509 | } // lib.optionalAttrs (config.service.user != null) { 510 | inherit (config.service) user; 511 | } // lib.optionalAttrs (config.service.blkio_config != null) { 512 | blkio_config = config.service.blkio_config._out; 513 | }; 514 | } 515 | -------------------------------------------------------------------------------- /src/nix/modules/service/extended-info.nix: -------------------------------------------------------------------------------- 1 | { config, lib, ... }: 2 | let 3 | inherit (lib) mkOption; 4 | inherit (lib.types) attrsOf unspecified; 5 | in 6 | { 7 | imports = [ 8 | (lib.mkRenamedOptionModule ["build" "extendedInfo"] ["out" "extendedInfo"]) 9 | ]; 10 | options = { 11 | out.extendedInfo = mkOption { 12 | type = attrsOf unspecified; 13 | description = '' 14 | Information about a service to include in the Docker Compose file, 15 | but that will not be used by the `docker-compose` command 16 | itself. 17 | 18 | It will be inserted in `x-arion.serviceInfo.`. 19 | ''; 20 | default = {}; 21 | }; 22 | }; 23 | } -------------------------------------------------------------------------------- /src/nix/modules/service/host-store.nix: -------------------------------------------------------------------------------- 1 | /* 2 | 3 | This service-level module bind mounts the host store into the container 4 | when the service.useHostStore option is set to true. 5 | 6 | */ 7 | { lib, config, pkgs, ... }: 8 | 9 | let 10 | inherit (lib) mkOption types mkIf; 11 | escape = s: lib.replaceStrings ["$"] ["$$"] s; 12 | in 13 | { 14 | options = { 15 | service.useHostStore = mkOption { 16 | type = types.bool; 17 | default = false; 18 | description = "Bind mounts the host store if enabled, avoiding copying."; 19 | }; 20 | service.hostStoreAsReadOnly = mkOption { 21 | type = types.bool; 22 | default = true; 23 | description = "Adds a `:ro` (read-only) access mode to the host nix store bind mount."; 24 | }; 25 | service.useHostNixDaemon = mkOption { 26 | type = types.bool; 27 | default = false; 28 | description = "Make the host Nix daemon available."; 29 | }; 30 | }; 31 | config = mkIf config.service.useHostStore { 32 | image.includeStorePaths = false; 33 | service.environment.NIX_REMOTE = lib.optionalString config.service.useHostNixDaemon "daemon"; 34 | service.volumes = [ 35 | "${config.host.nixStorePrefix}/nix/store:/nix/store${lib.optionalString config.service.hostStoreAsReadOnly ":ro"}" 36 | ] ++ lib.optional config.service.useHostNixDaemon "/nix/var/nix/daemon-socket:/nix/var/nix/daemon-socket"; 37 | service.command = lib.mkDefault (map escape (config.image.rawConfig.Cmd or [])); 38 | }; 39 | } 40 | -------------------------------------------------------------------------------- /src/nix/modules/service/image-recommended.nix: -------------------------------------------------------------------------------- 1 | { config, lib, pkgs, ... }: 2 | let 3 | inherit (lib) 4 | mkIf 5 | mkOption 6 | types 7 | ; 8 | inherit (types) 9 | bool 10 | ; 11 | 12 | recommendedContents = { runCommand, bash, coreutils }: 13 | runCommand "recommended-contents" {} '' 14 | mkdir -p $out/bin $out/usr/bin $out/var/empty 15 | ln -s ${bash}/bin/sh $out/bin/sh 16 | ln -s ${coreutils}/bin/env $out/usr/bin/env 17 | ''; 18 | in 19 | { 20 | options = { 21 | image.enableRecommendedContents = mkOption { 22 | type = bool; 23 | default = false; 24 | description = '' 25 | Add the `/bin/sh` and `/usr/bin/env` symlinks and some lightweight 26 | files. 27 | ''; 28 | }; 29 | }; 30 | 31 | config = { 32 | image.contents = mkIf config.image.enableRecommendedContents [ 33 | (pkgs.callPackage recommendedContents {}) 34 | ]; 35 | }; 36 | } 37 | -------------------------------------------------------------------------------- /src/nix/modules/service/image.nix: -------------------------------------------------------------------------------- 1 | { pkgs, lib, config, options, ... }: 2 | let 3 | inherit (lib) 4 | functionArgs 5 | mkOption 6 | optionalAttrs 7 | types 8 | warn 9 | ; 10 | inherit (pkgs) 11 | dockerTools 12 | ; 13 | inherit (types) attrsOf listOf nullOr package str unspecified bool; 14 | 15 | # TODO: dummy-config is a useless layer. Nix 2.3 will let us inspect 16 | # the string context instead, so we can avoid this. 17 | contentsList = config.image.contents ++ [ 18 | (pkgs.writeText "dummy-config.json" (builtins.toJSON config.image.rawConfig)) 19 | ]; 20 | 21 | includeStorePathsWarningAndDefault = lib.warn '' 22 | You're using a version of Nixpkgs that doesn't support the includeStorePaths 23 | parameter in dockerTools.streamLayeredImage. Without this, Arion's 24 | useHostStore does not achieve the intended speedup. 25 | '' {}; 26 | 27 | buildOrStreamLayeredImage = args: 28 | let 29 | args_base = builtins.intersectAttrs 30 | { 31 | name = null; tag = null; contents = null; config = null; 32 | created = null; extraCommands = null; maxLayers = null; 33 | fakeRootCommands = null; 34 | } 35 | args; 36 | acceptedArgs = functionArgs dockerTools.streamLayeredImage; 37 | args_no_store = lib.optionalAttrs (!(args.includeStorePaths or true)) ( 38 | if acceptedArgs ? includeStorePaths 39 | then { inherit (args) includeStorePaths; } 40 | else includeStorePathsWarningAndDefault 41 | ); 42 | args_streamLayered = args_base // args_no_store; 43 | in 44 | if dockerTools?streamLayeredImage 45 | then dockerTools.streamLayeredImage args_streamLayered // { isExe = true; } 46 | else dockerTools.buildLayeredImage args_base; 47 | 48 | builtImage = buildOrStreamLayeredImage { 49 | inherit (config.image) 50 | name 51 | contents 52 | includeStorePaths 53 | ; 54 | config = config.image.rawConfig; 55 | maxLayers = 100; 56 | 57 | # TODO: allow use of image's Nix package instead 58 | # TODO: option to disable db generation 59 | extraCommands = '' 60 | echo "Generating the nix database..." 61 | echo "Warning: only the database of the deepest Nix layer is loaded." 62 | echo " If you want to use nix commands in the container, it would" 63 | echo " be better to only have one layer that contains a nix store." 64 | export NIX_REMOTE=local?root=$PWD 65 | ${pkgs.nix}/bin/nix-store --load-db < ${pkgs.closureInfo {rootPaths = contentsList;}}/registration 66 | mkdir -p nix/var/nix/gcroots/docker/ 67 | for i in ${lib.concatStringsSep " " contentsList}; do 68 | ln -s $i nix/var/nix/gcroots/docker/$(basename $i) 69 | done; 70 | ''; 71 | 72 | fakeRootCommands = config.image.fakeRootCommands; 73 | }; 74 | 75 | priorityIsDefault = option: option.highestPrio >= (lib.mkDefault true).priority; 76 | in 77 | { 78 | options = { 79 | build.image = mkOption { 80 | type = nullOr package; 81 | description = '' 82 | Docker image derivation to be `docker load`-ed. 83 | ''; 84 | internal = true; 85 | }; 86 | build.imageName = mkOption { 87 | type = str; 88 | description = "Derived from `build.image`"; 89 | internal = true; 90 | }; 91 | build.imageTag = mkOption { 92 | type = str; 93 | description = "Derived from `build.image`"; 94 | internal = true; 95 | }; 96 | image.nixBuild = mkOption { 97 | type = bool; 98 | description = '' 99 | Whether to build this image with Nixpkgs' 100 | `dockerTools.buildLayeredImage` 101 | and then load it with `docker load`. 102 | 103 | By default, an image will be built with Nix unless `service.image` 104 | is set. See also `image.name`, which defaults to 105 | the service name. 106 | ''; 107 | }; 108 | image.name = mkOption { 109 | type = str; 110 | default = "localhost/" + config.service.name; 111 | defaultText = lib.literalExpression or lib.literalExample ''"localhost/" + config.service.name''; 112 | description = '' 113 | A human readable name for the docker image. 114 | 115 | Shows up in the `docker ps` output in the 116 | `IMAGE` column, among other places. 117 | ''; 118 | }; 119 | image.contents = mkOption { 120 | type = listOf package; 121 | default = []; 122 | description = '' 123 | Top level paths in the container. 124 | ''; 125 | }; 126 | image.fakeRootCommands = mkOption { 127 | type = types.lines; 128 | default = ""; 129 | description = '' 130 | Commands that build the root of the container in the current working directory. 131 | 132 | See [`dockerTools.buildLayeredImage`](https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerTools-buildLayeredImage). 133 | ''; 134 | }; 135 | image.includeStorePaths = mkOption { 136 | type = bool; 137 | default = true; 138 | internal = true; 139 | description = '' 140 | Include all referenced store paths. You generally want this in your 141 | image, unless you load store paths via some other means, like `useHostStore = true`; 142 | ''; 143 | }; 144 | image.rawConfig = mkOption { 145 | type = attrsOf unspecified; 146 | default = {}; 147 | description = '' 148 | This is a low-level fallback for when a container option has not 149 | been modeled in the Arion module system. 150 | 151 | This attribute set does not have an appropriate merge function. 152 | Please use the specific `image` options instead. 153 | 154 | Run-time configuration of the container. A full list of the 155 | options is available in the [Docker Image Specification 156 | v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions). 157 | ''; 158 | }; 159 | image.command = mkOption { 160 | type = listOf str; 161 | default = []; 162 | description = '' 163 | ''; 164 | }; 165 | }; 166 | config = lib.mkMerge [{ 167 | build.image = builtImage; 168 | build.imageName = config.build.image.imageName; 169 | build.imageTag = 170 | if config.build.image.imageTag != "" 171 | then config.build.image.imageTag 172 | else lib.head (lib.strings.splitString "-" (baseNameOf config.build.image.outPath)); 173 | image.rawConfig.Cmd = config.image.command; 174 | image.nixBuild = lib.mkDefault (priorityIsDefault options.service.image); 175 | } 176 | ( lib.mkIf (config.service.build.context == null) 177 | { 178 | service.image = lib.mkDefault "${config.build.imageName}:${config.build.imageTag}"; 179 | }) 180 | ]; 181 | } 182 | -------------------------------------------------------------------------------- /src/nix/modules/service/nixos-init.nix: -------------------------------------------------------------------------------- 1 | /* 2 | Invokes the NixOS init system in the container. 3 | */ 4 | { config, lib, pkgs, ... }: 5 | let 6 | inherit (lib) types; 7 | in 8 | { 9 | options = { 10 | nixos.useSystemd = lib.mkOption { 11 | type = types.bool; 12 | default = false; 13 | description = '' 14 | When enabled, call the NixOS systemd-based init system. 15 | 16 | Configure NixOS with the `nixos.configuration` option. 17 | ''; 18 | }; 19 | }; 20 | 21 | config = lib.mkIf (config.nixos.useSystemd) { 22 | nixos.configuration.imports = [ 23 | ../nixos/container-systemd.nix 24 | ../nixos/default-shell.nix 25 | (pkgs.path + "/nixos/modules/profiles/minimal.nix") 26 | ]; 27 | image.command = [ "/usr/sbin/init" ]; 28 | image.contents = [ 29 | (pkgs.runCommand "root-init" {} '' 30 | mkdir -p $out/usr/sbin 31 | ln -s ${config.nixos.build.toplevel}/init $out/usr/sbin/init 32 | '') 33 | ]; 34 | service.environment.container = "docker"; 35 | service.environment.PATH = "/usr/bin:/run/current-system/sw/bin/"; 36 | service.volumes = [ 37 | "/sys/fs/cgroup:/sys/fs/cgroup:ro" 38 | ]; 39 | service.tmpfs = [ 40 | "/run" # noexec is fine because exes should be symlinked from elsewhere anyway 41 | "/run/wrappers" # noexec breaks this intentionally 42 | ] ++ lib.optional (config.nixos.evaluatedConfig.boot.tmp.useTmpfs) "/tmp:exec,mode=777"; 43 | 44 | service.stop_signal = "SIGRTMIN+3"; 45 | service.tty = true; 46 | service.defaultExec = [config.nixos.build.x-arion-defaultShell "-l"]; 47 | }; 48 | } 49 | -------------------------------------------------------------------------------- /src/nix/modules/service/nixos.nix: -------------------------------------------------------------------------------- 1 | /* 2 | Generic options for adding NixOS modules and evaluating the NixOS 3 | configuration. The methods for installing NixOS in the image are 4 | defined elsewhere. 5 | */ 6 | { pkgs, lib, config, ... }: 7 | let 8 | inherit (lib) types; 9 | inherit (types) attrs; 10 | in 11 | { 12 | options = { 13 | nixos.configuration = lib.mkOption { 14 | type = with types; coercedTo unspecified (a: [a]) (listOf unspecified); 15 | default = {}; 16 | description = '' 17 | Modules to add to the NixOS configuration. 18 | 19 | This option is unused by default, because not all images use NixOS. 20 | 21 | One way to use this is to enable `nixos.useSystemd`, but the 22 | NixOS configuration can be used in other ways. 23 | ''; 24 | }; 25 | 26 | nixos.build = lib.mkOption { 27 | type = attrs; 28 | readOnly = true; 29 | description = '' 30 | NixOS build products from `config.system.build`, such as `toplevel` and `etc`. 31 | 32 | This option is unused by default, because not all images use NixOS. 33 | 34 | One way to use this is to enable `nixos.useSystemd`, but the 35 | NixOS configuration can be used in other ways. 36 | ''; 37 | }; 38 | 39 | nixos.evaluatedConfig = lib.mkOption { 40 | type = attrs; 41 | readOnly = true; 42 | description = '' 43 | Evaluated NixOS configuration, to be read by service-level modules. 44 | 45 | This option is unused by default, because not all images use NixOS. 46 | 47 | One way to use this is to enable `nixos.useSystemd`, but the 48 | NixOS configuration can be used in other ways. 49 | ''; 50 | }; 51 | }; 52 | 53 | config = { 54 | nixos.build = builtins.addErrorContext "while evaluating the service nixos configuration" ( 55 | pkgs.nixos config.nixos.configuration 56 | ); 57 | nixos.configuration = { config, ... }: { system.build.theConfig = config; }; 58 | nixos.evaluatedConfig = config.nixos.build.theConfig; 59 | }; 60 | } 61 | -------------------------------------------------------------------------------- /tests/arion-test/default.nix: -------------------------------------------------------------------------------- 1 | { usePodman ? false, pkgs, lib ? pkgs.lib, ... }: 2 | 3 | let 4 | # To make some prebuilt derivations available in the vm 5 | preEval = modules: import ../../src/nix/eval-composition.nix { 6 | inherit modules; 7 | inherit pkgs; 8 | }; 9 | 10 | inherit (lib) 11 | concatMapStringsSep 12 | optionalAttrs 13 | optionalString 14 | ; 15 | 16 | haveSystemd = usePodman || pkgs.arionTestingFlags.dockerSupportsSystemd; 17 | 18 | concatPathLines = paths: concatMapStringsSep "\n" (x: "${x}") paths; 19 | 20 | in 21 | { 22 | name = "arion-test"; 23 | nodes.machine = { pkgs, lib, ... }: { 24 | environment.systemPackages = [ 25 | pkgs.arion 26 | ] ++ lib.optional usePodman pkgs.docker; 27 | virtualisation.docker.enable = !usePodman; 28 | virtualisation.podman = optionalAttrs usePodman { 29 | enable = true; 30 | dockerSocket.enable = true; 31 | }; 32 | 33 | # no caches, because no internet 34 | nix.settings.substituters = lib.mkForce []; 35 | 36 | virtualisation.writableStore = true; 37 | # Switch to virtualisation.additionalPaths when dropping all NixOS <= 21.05. 38 | environment.etc."extra-paths-for-test".text = concatPathLines [ 39 | # Pre-build the image because we don't want to build the world 40 | # in the vm. 41 | (preEval [ ../../examples/minimal/arion-compose.nix ]).config.out.dockerComposeYaml 42 | (preEval [ ../../examples/full-nixos/arion-compose.nix ]).config.out.dockerComposeYaml 43 | (preEval [ ../../examples/nixos-unit/arion-compose.nix ]).config.out.dockerComposeYaml 44 | (preEval [ ../../examples/traefik/arion-compose.nix ]).config.out.dockerComposeYaml 45 | pkgs.stdenv 46 | ]; 47 | 48 | virtualisation.memorySize = 2048; 49 | virtualisation.diskSize = 8000; 50 | }; 51 | testScript = '' 52 | machine.fail("curl --fail localhost:8000") 53 | machine.succeed("docker --version") 54 | 55 | # Tests 56 | # - arion up 57 | # - arion down 58 | # - examples/minimal 59 | with subtest("minimal"): 60 | machine.succeed( 61 | "rm -rf work && cp -frT ${../../examples/minimal} work && cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion up -d" 62 | ) 63 | machine.wait_until_succeeds("curl --fail localhost:8000") 64 | machine.succeed( 65 | "cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion down" 66 | ) 67 | machine.wait_until_fails("curl --fail localhost:8000") 68 | 69 | # Tests 70 | # - running same image again doesn't require a `docker load` 71 | with subtest("docker load only once"): 72 | # We assume image loading relies on the `docker images` and `docker load` commands, so this should fail 73 | machine.fail( 74 | "export REAL_DOCKER=$(which docker); rm -rf work && cp -frT ${../../examples/minimal} work && cd work && NIX_PATH=nixpkgs='${pkgs.path}' PATH=\"${pkgs.writeScriptBin "docker" '' 75 | #!${pkgs.runtimeShell} -eu 76 | echo 1>&2 "This failure is expected. Args were" "$@" 77 | echo "$@" >/tmp/docker-args 78 | exit 1 79 | ''}/bin:$PATH\" arion up -d" 80 | ) 81 | machine.succeed( 82 | "export REAL_DOCKER=$(which docker); rm -rf work && cp -frT ${../../examples/minimal} work && cd work && NIX_PATH=nixpkgs='${pkgs.path}' PATH=\"${pkgs.writeScriptBin "docker" '' 83 | #!${pkgs.runtimeShell} -eu 84 | case $1 in 85 | load) 86 | echo 1>&2 "arion must not docker load when upping the same deployment for the second time" 87 | exit 1 88 | ;; 89 | images) 90 | echo 1>&2 "execing docker to list images" 91 | exec $REAL_DOCKER "$@" 92 | ;; 93 | *) 94 | echo 1>&2 "Unknown docker invocation. This may be a shortcoming of this docker mock." 95 | echo 1>&2 "Invocation: docker" "$@" 96 | ;; 97 | esac 98 | ''}/bin:$PATH\" arion up -d" 99 | ) 100 | machine.wait_until_succeeds("curl --fail localhost:8000") 101 | machine.succeed( 102 | "cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion down" 103 | ) 104 | machine.wait_until_fails("curl --fail localhost:8000") 105 | 106 | 107 | # Tests 108 | # - examples/flake 109 | # This _test_ doesn't work because flake-compat fetches the github 110 | # tarballs without sha256 and/or Nix doesn't consult the store before 111 | # downloading. 112 | # See https://github.com/edolstra/flake-compat/pull/12 113 | # with subtest("flake"): 114 | # machine.succeed( 115 | # "rm -rf work && cp -frT ''${../../examples/flake} work && cd work && NIX_PATH= arion up -d" 116 | # ) 117 | # machine.wait_until_succeeds("curl --fail localhost:8000") 118 | # machine.succeed("cd work && NIX_PATH= arion down") 119 | # machine.wait_until_fails("curl --fail localhost:8000") 120 | 121 | ${optionalString haveSystemd '' 122 | # Tests 123 | # - arion exec 124 | # - examples/full-nixos 125 | with subtest("full-nixos"): 126 | machine.succeed( 127 | "rm -rf work && cp -frT ${../../examples/full-nixos} work && cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion up -d" 128 | ) 129 | machine.wait_until_succeeds("curl --fail localhost:8000") 130 | 131 | machine.succeed( 132 | """ 133 | set -eux -o pipefail 134 | cd work 135 | export NIX_PATH=nixpkgs='${pkgs.path}' 136 | echo 'target=world; echo Hello $target; exit' \ 137 | | script 'arion exec webserver' \ 138 | | grep 'Hello world' 139 | """ 140 | ), 141 | 142 | machine.succeed( 143 | "cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion down" 144 | ) 145 | machine.wait_until_fails("curl --fail localhost:8000") 146 | ''} 147 | 148 | # Tests 149 | # - examples/nixos-unit 150 | with subtest("nixos-unit"): 151 | machine.succeed( 152 | "rm -rf work && cp -frT ${../../examples/nixos-unit} work && cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion up -d" 153 | ) 154 | machine.wait_until_succeeds("curl --fail localhost:8000") 155 | machine.succeed( 156 | "cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion down" 157 | ) 158 | machine.wait_until_fails("curl --fail localhost:8000") 159 | 160 | # Tests 161 | # - examples/traefik 162 | # - labels 163 | with subtest("traefik"): 164 | machine.succeed( 165 | "rm -rf work && cp -frT ${../../examples/traefik} work && cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion up -d" 166 | ) 167 | machine.wait_until_succeeds("curl --fail nix-docs.localhost") 168 | machine.succeed( 169 | "cd work && NIX_PATH=nixpkgs='${pkgs.path}' arion down" 170 | ) 171 | machine.wait_until_fails("curl --fail nix-docs.localhost") 172 | ''; 173 | } 174 | -------------------------------------------------------------------------------- /tests/flake-module.nix: -------------------------------------------------------------------------------- 1 | { 2 | perSystem = { pkgs, final, ... }: 3 | let 4 | inherit (final) nixosTest arion lib; 5 | in 6 | { 7 | checks = lib.optionalAttrs pkgs.stdenv.isLinux { 8 | test = nixosTest ./arion-test; 9 | 10 | nixosModuleWithDocker = 11 | import ./nixos-virtualization-arion-test/test.nix final { 12 | virtualisation.arion.backend = "docker"; 13 | }; 14 | 15 | # Currently broken; kafka can't reach zookeeper 16 | # nixosModuleWithPodman = 17 | # import ./nixos-virtualization-arion-test/test.nix final { 18 | # virtualisation.arion.backend = "podman-socket"; 19 | # }; 20 | 21 | testWithPodman = 22 | nixosTest (import ./arion-test { usePodman = true; pkgs = final; }); 23 | 24 | testBuild = arion.build { 25 | 26 | # To be more accurate, we could do 27 | # pkgs = import ../examples/minimal/arion-pkgs.nix; 28 | # But let's avoid re-evaluating Nixpkgs 29 | pkgs = final; 30 | 31 | modules = [ ../examples/minimal/arion-compose.nix ]; 32 | }; 33 | 34 | }; 35 | }; 36 | } 37 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/README.md: -------------------------------------------------------------------------------- 1 | 2 | # NixOS module test 3 | 4 | This tests the NixOS module. 5 | 6 | The images used here are experimental and not meant for production. 7 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/arion-compose.nix: -------------------------------------------------------------------------------- 1 | { pkgs, ... }: { 2 | project.name = "whale"; 3 | 4 | docker-compose.raw = { 5 | volumes.zookeeper = { }; 6 | volumes.kafka = { }; 7 | }; 8 | 9 | services.kafka = { 10 | service.useHostStore = true; 11 | # service.volumes = [ 12 | # { 13 | # type = "volume"; 14 | # source = "kafka"; 15 | # target = "/data"; 16 | # # volume.nocopy = true; 17 | # } 18 | # ]; 19 | service.ports = [ "9092:9092" ]; 20 | service.depends_on = [ "zookeeper" ]; 21 | image.name = "localhost/kafka"; 22 | image.contents = [ 23 | (pkgs.runCommand "root" { } '' 24 | mkdir -p $out/bin 25 | ln -s ${pkgs.runtimeShell} $out/bin/sh 26 | '') 27 | ]; 28 | image.command = [ 29 | "${pkgs.apacheKafka}/bin/kafka-server-start.sh" 30 | "${./kafka/server.properties}" 31 | ]; 32 | }; 33 | 34 | services.zookeeper = { 35 | service.useHostStore = true; 36 | service.ports = [ "2181:2181" ]; 37 | # service.volumes = [ 38 | # { 39 | # type = "volume"; 40 | # source = "zookeeper"; 41 | # target = "/data"; 42 | # # volume.nocopy = true; 43 | # } 44 | # ]; 45 | image.name = "localhost/zookeeper"; 46 | image.contents = [ 47 | (pkgs.buildEnv { 48 | name = "root"; 49 | paths = [ 50 | # pkgs.sed 51 | pkgs.busybox 52 | ]; 53 | }) 54 | ]; 55 | image.command = [ 56 | "${pkgs.zookeeper}/bin/zkServer.sh" 57 | "--config" 58 | "${./zookeeper}" 59 | "start-foreground" 60 | ]; 61 | }; 62 | } 63 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/arion-pkgs.nix: -------------------------------------------------------------------------------- 1 | # NOTE: This isn't used in the module! 2 | import { 3 | # We specify the architecture explicitly. Use a Linux remote builder when 4 | # calling arion from other platforms. 5 | system = "x86_64-linux"; 6 | } 7 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/kafka/server.properties: -------------------------------------------------------------------------------- 1 | # Licensed to the Apache Software Foundation (ASF) under one or more 2 | # contributor license agreements. See the NOTICE file distributed with 3 | # this work for additional information regarding copyright ownership. 4 | # The ASF licenses this file to You under the Apache License, Version 2.0 5 | # (the "License"); you may not use this file except in compliance with 6 | # the License. You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | 16 | # see kafka.server.KafkaConfig for additional details and defaults 17 | 18 | ############################# Server Basics ############################# 19 | 20 | # The id of the broker. This must be set to a unique integer for each broker. 21 | broker.id=0 22 | 23 | ############################# Socket Server Settings ############################# 24 | 25 | # The address the socket server listens on. It will get the value returned from 26 | # java.net.InetAddress.getCanonicalHostName() if not configured. 27 | # FORMAT: 28 | # listeners = listener_name://host_name:port 29 | # EXAMPLE: 30 | # listeners = PLAINTEXT://your.host.name:9092 31 | listeners=LOCALHOST://0.0.0.0:9092,SERVICE://kafka:9093 32 | 33 | # Hostname and port the broker will advertise to producers and consumers. If not set, 34 | # it uses the value for "listeners" if configured. Otherwise, it will use the value 35 | # returned from java.net.InetAddress.getCanonicalHostName(). 36 | # advertised.listeners=PLAINTEXT://whale_kafka_1:9092 37 | advertised.listeners=LOCALHOST://localhost:9092,SERVICE://kafka:9093 38 | 39 | # ??? 40 | inter.broker.listener.name=LOCALHOST 41 | 42 | # Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details 43 | #listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL 44 | listener.security.protocol.map=LOCALHOST:PLAINTEXT,SERVICE:PLAINTEXT 45 | 46 | # The number of threads that the server uses for receiving requests from the network and sending responses to the network 47 | num.network.threads=3 48 | 49 | # The number of threads that the server uses for processing requests, which may include disk I/O 50 | num.io.threads=8 51 | 52 | # The send buffer (SO_SNDBUF) used by the socket server 53 | socket.send.buffer.bytes=102400 54 | 55 | # The receive buffer (SO_RCVBUF) used by the socket server 56 | socket.receive.buffer.bytes=102400 57 | 58 | # The maximum size of a request that the socket server will accept (protection against OOM) 59 | socket.request.max.bytes=104857600 60 | 61 | 62 | ############################# Log Basics ############################# 63 | 64 | # A comma separated list of directories under which to store log files 65 | log.dirs=/data/kafka 66 | 67 | # The default number of log partitions per topic. More partitions allow greater 68 | # parallelism for consumption, but this will also result in more files across 69 | # the brokers. 70 | num.partitions=1 71 | 72 | # The number of threads per data directory to be used for log recovery at startup and flushing at shutdown. 73 | # This value is recommended to be increased for installations with data dirs located in RAID array. 74 | num.recovery.threads.per.data.dir=1 75 | 76 | ############################# Internal Topic Settings ############################# 77 | # The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state" 78 | # For anything other than development testing, a value greater than 1 is recommended to ensure availability such as 3. 79 | offsets.topic.replication.factor=1 80 | transaction.state.log.replication.factor=1 81 | transaction.state.log.min.isr=1 82 | 83 | ############################# Log Flush Policy ############################# 84 | 85 | # Messages are immediately written to the filesystem but by default we only fsync() to sync 86 | # the OS cache lazily. The following configurations control the flush of data to disk. 87 | # There are a few important trade-offs here: 88 | # 1. Durability: Unflushed data may be lost if you are not using replication. 89 | # 2. Latency: Very large flush intervals may lead to latency spikes when the flush does occur as there will be a lot of data to flush. 90 | # 3. Throughput: The flush is generally the most expensive operation, and a small flush interval may lead to excessive seeks. 91 | # The settings below allow one to configure the flush policy to flush data after a period of time or 92 | # every N messages (or both). This can be done globally and overridden on a per-topic basis. 93 | 94 | # The number of messages to accept before forcing a flush of data to disk 95 | #log.flush.interval.messages=10000 96 | 97 | # The maximum amount of time a message can sit in a log before we force a flush 98 | #log.flush.interval.ms=1000 99 | 100 | ############################# Log Retention Policy ############################# 101 | 102 | # The following configurations control the disposal of log segments. The policy can 103 | # be set to delete segments after a period of time, or after a given size has accumulated. 104 | # A segment will be deleted whenever *either* of these criteria are met. Deletion always happens 105 | # from the end of the log. 106 | 107 | # The minimum age of a log file to be eligible for deletion due to age 108 | log.retention.hours=168 109 | 110 | # A size-based retention policy for logs. Segments are pruned from the log unless the remaining 111 | # segments drop below log.retention.bytes. Functions independently of log.retention.hours. 112 | #log.retention.bytes=1073741824 113 | 114 | # The maximum size of a log segment file. When this size is reached a new log segment will be created. 115 | log.segment.bytes=1073741824 116 | 117 | # The interval at which log segments are checked to see if they can be deleted according 118 | # to the retention policies 119 | log.retention.check.interval.ms=300000 120 | 121 | ############################# Zookeeper ############################# 122 | 123 | # Zookeeper connection string (see zookeeper docs for details). 124 | # This is a comma separated host:port pairs, each corresponding to a zk 125 | # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002". 126 | # You can also append an optional chroot string to the urls to specify the 127 | # root directory for all kafka znodes. 128 | zookeeper.connect=zookeeper:2181 129 | 130 | # Timeout in ms for connecting to zookeeper 131 | zookeeper.connection.timeout.ms=18000 132 | 133 | 134 | ############################# Group Coordinator Settings ############################# 135 | 136 | # The following configuration specifies the time, in milliseconds, that the GroupCoordinator will delay the initial consumer rebalance. 137 | # The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of max.poll.interval.ms. 138 | # The default value for this is 3 seconds. 139 | # We override this to 0 here as it makes for a better out-of-the-box experience for development and testing. 140 | # However, in production environments the default value of 3 seconds is more suitable as this will help to avoid unnecessary, and potentially expensive, rebalances during application startup. 141 | group.initial.rebalance.delay.ms=0 142 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/test.nix: -------------------------------------------------------------------------------- 1 | pkgs: module: 2 | 3 | pkgs.nixosTest { 4 | name = "test-basic-arion-kafka"; 5 | nodes = { 6 | machine = { ... }: { 7 | virtualisation.memorySize = 4096; 8 | virtualisation.diskSize = 10000; 9 | imports = [ 10 | ../../nixos-module.nix 11 | module 12 | ]; 13 | 14 | virtualisation.arion.projects.whale.settings = { 15 | imports = [ ./arion-compose.nix ]; 16 | }; 17 | }; 18 | }; 19 | testScript = '' 20 | machine.wait_for_unit("sockets.target") 21 | machine.wait_for_unit("arion-whale.service") 22 | 23 | machine.succeed(""" 24 | (echo "hello"; echo "world") \ 25 | | ${pkgs.apacheKafka}/bin/kafka-console-producer.sh \ 26 | --topic thetopic --bootstrap-server localhost:9092 27 | """) 28 | 29 | machine.succeed(""" 30 | ( 31 | set +o pipefail # we only care for head's exit code 32 | ( ${pkgs.apacheKafka}/bin/kafka-console-consumer.sh \ 33 | --topic thetopic --from-beginning --bootstrap-server localhost:9092 & \ 34 | echo $! >pid 35 | ) | grep --line-buffered hello | { read; kill $(/dev/console 37 | """) 38 | 39 | ''; 40 | } 41 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/zookeeper/log4j.properties: -------------------------------------------------------------------------------- 1 | # Copyright 2012 The Apache Software Foundation 2 | # 3 | # Licensed to the Apache Software Foundation (ASF) under one 4 | # or more contributor license agreements. See the NOTICE file 5 | # distributed with this work for additional information 6 | # regarding copyright ownership. The ASF licenses this file 7 | # to you under the Apache License, Version 2.0 (the 8 | # "License"); you may not use this file except in compliance 9 | # with the License. You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | 19 | # Define some default values that can be overridden by system properties 20 | zookeeper.root.logger=INFO, CONSOLE 21 | 22 | zookeeper.console.threshold=INFO 23 | 24 | zookeeper.log.dir=. 25 | zookeeper.log.file=zookeeper.log 26 | zookeeper.log.threshold=INFO 27 | zookeeper.log.maxfilesize=256MB 28 | zookeeper.log.maxbackupindex=20 29 | 30 | # zookeeper.tracelog.dir=${zookeeper.log.dir} 31 | # zookeeper.tracelog.file=zookeeper_trace.log 32 | 33 | log4j.rootLogger=${zookeeper.root.logger} 34 | 35 | # 36 | # console 37 | # Add "console" to rootlogger above if you want to use this 38 | # 39 | log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender 40 | log4j.appender.CONSOLE.Threshold=${zookeeper.console.threshold} 41 | log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout 42 | log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n 43 | 44 | # # 45 | # # Add ROLLINGFILE to rootLogger to get log file output 46 | # # 47 | # log4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender 48 | # log4j.appender.ROLLINGFILE.Threshold=${zookeeper.log.threshold} 49 | # log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/${zookeeper.log.file} 50 | # log4j.appender.ROLLINGFILE.MaxFileSize=${zookeeper.log.maxfilesize} 51 | # log4j.appender.ROLLINGFILE.MaxBackupIndex=${zookeeper.log.maxbackupindex} 52 | # log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout 53 | # log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n 54 | 55 | # # 56 | # # Add TRACEFILE to rootLogger to get log file output 57 | # # Log TRACE level and above messages to a log file 58 | # # 59 | # log4j.appender.TRACEFILE=org.apache.log4j.FileAppender 60 | # log4j.appender.TRACEFILE.Threshold=TRACE 61 | # log4j.appender.TRACEFILE.File=${zookeeper.tracelog.dir}/${zookeeper.tracelog.file} 62 | 63 | # log4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout 64 | # ### Notice we are including log4j's NDC here (%x) 65 | # log4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L][%x] - %m%n 66 | # # 67 | # # zk audit logging 68 | # # 69 | # zookeeper.auditlog.file=zookeeper_audit.log 70 | # zookeeper.auditlog.threshold=INFO 71 | # audit.logger=INFO, CONSOLE 72 | # log4j.logger.org.apache.zookeeper.audit.Log4jAuditLogger=${audit.logger} 73 | # log4j.additivity.org.apache.zookeeper.audit.Log4jAuditLogger=false 74 | # log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender 75 | # log4j.appender.RFAAUDIT.File=${zookeeper.log.dir}/${zookeeper.auditlog.file} 76 | # log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout 77 | # log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n 78 | # log4j.appender.RFAAUDIT.Threshold=${zookeeper.auditlog.threshold} 79 | 80 | # # Max log file size of 10MB 81 | # log4j.appender.RFAAUDIT.MaxFileSize=10MB 82 | # log4j.appender.RFAAUDIT.MaxBackupIndex=10 83 | -------------------------------------------------------------------------------- /tests/nixos-virtualization-arion-test/zookeeper/zoo.cfg: -------------------------------------------------------------------------------- 1 | tickTime=2000 2 | dataDir=/data 3 | clientPort=2181 4 | --------------------------------------------------------------------------------