├── LICENSE.txt └── README.md /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Attribution-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-ShareAlike 4.0 International Public 58 | License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-ShareAlike 4.0 International Public License ("Public 63 | License"). To the extent this Public License may be interpreted as a 64 | contract, You are granted the Licensed Rights in consideration of Your 65 | acceptance of these terms and conditions, and the Licensor grants You 66 | such rights in consideration of benefits the Licensor receives from 67 | making the Licensed Material available under these terms and 68 | conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. Share means to provide material to the public by any means or 126 | process that requires permission under the Licensed Rights, such 127 | as reproduction, public display, public performance, distribution, 128 | dissemination, communication, or importation, and to make material 129 | available to the public including in ways that members of the 130 | public may access the material from a place and at a time 131 | individually chosen by them. 132 | 133 | l. Sui Generis Database Rights means rights other than copyright 134 | resulting from Directive 96/9/EC of the European Parliament and of 135 | the Council of 11 March 1996 on the legal protection of databases, 136 | as amended and/or succeeded, as well as other essentially 137 | equivalent rights anywhere in the world. 138 | 139 | m. You means the individual or entity exercising the Licensed Rights 140 | under this Public License. Your has a corresponding meaning. 141 | 142 | 143 | Section 2 -- Scope. 144 | 145 | a. License grant. 146 | 147 | 1. Subject to the terms and conditions of this Public License, 148 | the Licensor hereby grants You a worldwide, royalty-free, 149 | non-sublicensable, non-exclusive, irrevocable license to 150 | exercise the Licensed Rights in the Licensed Material to: 151 | 152 | a. reproduce and Share the Licensed Material, in whole or 153 | in part; and 154 | 155 | b. produce, reproduce, and Share Adapted Material. 156 | 157 | 2. Exceptions and Limitations. For the avoidance of doubt, where 158 | Exceptions and Limitations apply to Your use, this Public 159 | License does not apply, and You do not need to comply with 160 | its terms and conditions. 161 | 162 | 3. Term. The term of this Public License is specified in Section 163 | 6(a). 164 | 165 | 4. Media and formats; technical modifications allowed. The 166 | Licensor authorizes You to exercise the Licensed Rights in 167 | all media and formats whether now known or hereafter created, 168 | and to make technical modifications necessary to do so. The 169 | Licensor waives and/or agrees not to assert any right or 170 | authority to forbid You from making technical modifications 171 | necessary to exercise the Licensed Rights, including 172 | technical modifications necessary to circumvent Effective 173 | Technological Measures. For purposes of this Public License, 174 | simply making modifications authorized by this Section 2(a) 175 | (4) never produces Adapted Material. 176 | 177 | 5. Downstream recipients. 178 | 179 | a. Offer from the Licensor -- Licensed Material. Every 180 | recipient of the Licensed Material automatically 181 | receives an offer from the Licensor to exercise the 182 | Licensed Rights under the terms and conditions of this 183 | Public License. 184 | 185 | b. Additional offer from the Licensor -- Adapted Material. 186 | Every recipient of Adapted Material from You 187 | automatically receives an offer from the Licensor to 188 | exercise the Licensed Rights in the Adapted Material 189 | under the conditions of the Adapter's License You apply. 190 | 191 | c. No downstream restrictions. You may not offer or impose 192 | any additional or different terms or conditions on, or 193 | apply any Effective Technological Measures to, the 194 | Licensed Material if doing so restricts exercise of the 195 | Licensed Rights by any recipient of the Licensed 196 | Material. 197 | 198 | 6. No endorsement. Nothing in this Public License constitutes or 199 | may be construed as permission to assert or imply that You 200 | are, or that Your use of the Licensed Material is, connected 201 | with, or sponsored, endorsed, or granted official status by, 202 | the Licensor or others designated to receive attribution as 203 | provided in Section 3(a)(1)(A)(i). 204 | 205 | b. Other rights. 206 | 207 | 1. Moral rights, such as the right of integrity, are not 208 | licensed under this Public License, nor are publicity, 209 | privacy, and/or other similar personality rights; however, to 210 | the extent possible, the Licensor waives and/or agrees not to 211 | assert any such rights held by the Licensor to the limited 212 | extent necessary to allow You to exercise the Licensed 213 | Rights, but not otherwise. 214 | 215 | 2. Patent and trademark rights are not licensed under this 216 | Public License. 217 | 218 | 3. To the extent possible, the Licensor waives any right to 219 | collect royalties from You for the exercise of the Licensed 220 | Rights, whether directly or through a collecting society 221 | under any voluntary or waivable statutory or compulsory 222 | licensing scheme. In all other cases the Licensor expressly 223 | reserves any right to collect such royalties. 224 | 225 | 226 | Section 3 -- License Conditions. 227 | 228 | Your exercise of the Licensed Rights is expressly made subject to the 229 | following conditions. 230 | 231 | a. Attribution. 232 | 233 | 1. If You Share the Licensed Material (including in modified 234 | form), You must: 235 | 236 | a. retain the following if it is supplied by the Licensor 237 | with the Licensed Material: 238 | 239 | i. identification of the creator(s) of the Licensed 240 | Material and any others designated to receive 241 | attribution, in any reasonable manner requested by 242 | the Licensor (including by pseudonym if 243 | designated); 244 | 245 | ii. a copyright notice; 246 | 247 | iii. a notice that refers to this Public License; 248 | 249 | iv. a notice that refers to the disclaimer of 250 | warranties; 251 | 252 | v. a URI or hyperlink to the Licensed Material to the 253 | extent reasonably practicable; 254 | 255 | b. indicate if You modified the Licensed Material and 256 | retain an indication of any previous modifications; and 257 | 258 | c. indicate the Licensed Material is licensed under this 259 | Public License, and include the text of, or the URI or 260 | hyperlink to, this Public License. 261 | 262 | 2. You may satisfy the conditions in Section 3(a)(1) in any 263 | reasonable manner based on the medium, means, and context in 264 | which You Share the Licensed Material. For example, it may be 265 | reasonable to satisfy the conditions by providing a URI or 266 | hyperlink to a resource that includes the required 267 | information. 268 | 269 | 3. If requested by the Licensor, You must remove any of the 270 | information required by Section 3(a)(1)(A) to the extent 271 | reasonably practicable. 272 | 273 | b. ShareAlike. 274 | 275 | In addition to the conditions in Section 3(a), if You Share 276 | Adapted Material You produce, the following conditions also apply. 277 | 278 | 1. The Adapter's License You apply must be a Creative Commons 279 | license with the same License Elements, this version or 280 | later, or a BY-SA Compatible License. 281 | 282 | 2. You must include the text of, or the URI or hyperlink to, the 283 | Adapter's License You apply. You may satisfy this condition 284 | in any reasonable manner based on the medium, means, and 285 | context in which You Share Adapted Material. 286 | 287 | 3. You may not offer or impose any additional or different terms 288 | or conditions on, or apply any Effective Technological 289 | Measures to, Adapted Material that restrict exercise of the 290 | rights granted under the Adapter's License You apply. 291 | 292 | 293 | Section 4 -- Sui Generis Database Rights. 294 | 295 | Where the Licensed Rights include Sui Generis Database Rights that 296 | apply to Your use of the Licensed Material: 297 | 298 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 299 | to extract, reuse, reproduce, and Share all or a substantial 300 | portion of the contents of the database; 301 | 302 | b. if You include all or a substantial portion of the database 303 | contents in a database in which You have Sui Generis Database 304 | Rights, then the database in which You have Sui Generis Database 305 | Rights (but not its individual contents) is Adapted Material, 306 | 307 | including for purposes of Section 3(b); and 308 | c. You must comply with the conditions in Section 3(a) if You Share 309 | all or a substantial portion of the contents of the database. 310 | 311 | For the avoidance of doubt, this Section 4 supplements and does not 312 | replace Your obligations under this Public License where the Licensed 313 | Rights include other Copyright and Similar Rights. 314 | 315 | 316 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 317 | 318 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 319 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 320 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 321 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 322 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 323 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 324 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 325 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 326 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 327 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 328 | 329 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 330 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 331 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 332 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 333 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 334 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 335 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 336 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 337 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 338 | 339 | c. The disclaimer of warranties and limitation of liability provided 340 | above shall be interpreted in a manner that, to the extent 341 | possible, most closely approximates an absolute disclaimer and 342 | waiver of all liability. 343 | 344 | 345 | Section 6 -- Term and Termination. 346 | 347 | a. This Public License applies for the term of the Copyright and 348 | Similar Rights licensed here. However, if You fail to comply with 349 | this Public License, then Your rights under this Public License 350 | terminate automatically. 351 | 352 | b. Where Your right to use the Licensed Material has terminated under 353 | Section 6(a), it reinstates: 354 | 355 | 1. automatically as of the date the violation is cured, provided 356 | it is cured within 30 days of Your discovery of the 357 | violation; or 358 | 359 | 2. upon express reinstatement by the Licensor. 360 | 361 | For the avoidance of doubt, this Section 6(b) does not affect any 362 | right the Licensor may have to seek remedies for Your violations 363 | of this Public License. 364 | 365 | c. For the avoidance of doubt, the Licensor may also offer the 366 | Licensed Material under separate terms or conditions or stop 367 | distributing the Licensed Material at any time; however, doing so 368 | will not terminate this Public License. 369 | 370 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 371 | License. 372 | 373 | 374 | Section 7 -- Other Terms and Conditions. 375 | 376 | a. The Licensor shall not be bound by any additional or different 377 | terms or conditions communicated by You unless expressly agreed. 378 | 379 | b. Any arrangements, understandings, or agreements regarding the 380 | Licensed Material not stated herein are separate from and 381 | independent of the terms and conditions of this Public License. 382 | 383 | 384 | Section 8 -- Interpretation. 385 | 386 | a. For the avoidance of doubt, this Public License does not, and 387 | shall not be interpreted to, reduce, limit, restrict, or impose 388 | conditions on any use of the Licensed Material that could lawfully 389 | be made without permission under this Public License. 390 | 391 | b. To the extent possible, if any provision of this Public License is 392 | deemed unenforceable, it shall be automatically reformed to the 393 | minimum extent necessary to make it enforceable. If the provision 394 | cannot be reformed, it shall be severed from this Public License 395 | without affecting the enforceability of the remaining terms and 396 | conditions. 397 | 398 | c. No term or condition of this Public License will be waived and no 399 | failure to comply consented to unless expressly agreed to by the 400 | Licensor. 401 | 402 | d. Nothing in this Public License constitutes or may be interpreted 403 | as a limitation upon, or waiver of, any privileges and immunities 404 | that apply to the Licensor or You, including from the legal 405 | processes of any jurisdiction or authority. 406 | 407 | 408 | ======================================================================= 409 | 410 | Creative Commons is not a party to its public 411 | licenses. Notwithstanding, Creative Commons may elect to apply one of 412 | its public licenses to material it publishes and in those instances 413 | will be considered the “Licensor.” The text of the Creative Commons 414 | public licenses is dedicated to the public domain under the CC0 Public 415 | Domain Dedication. Except for the limited purpose of indicating that 416 | material is shared under a Creative Commons public license or as 417 | otherwise permitted by the Creative Commons policies published at 418 | creativecommons.org/policies, Creative Commons does not authorize the 419 | use of the trademark "Creative Commons" or any other trademark or logo 420 | of Creative Commons without its prior written consent including, 421 | without limitation, in connection with any unauthorized modifications 422 | to any of its public licenses or any other arrangements, 423 | understandings, or agreements concerning use of licensed material. For 424 | the avoidance of doubt, this paragraph does not form part of the 425 | public licenses. 426 | 427 | Creative Commons may be contacted at creativecommons.org. 428 | 429 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | --- 2 | feature: Evaluation purity and caching builtins 3 | start-date: 2023-05-10 4 | author: Silvan Mosberger 5 | co-authors: (find a buddy later to help out with the RFC) 6 | shepherd-team: (names, to be nominated and accepted by RFC steering committee) 7 | shepherd-leader: (name to be appointed by RFC steering committee) 8 | related-issues: (will contain links to implementation PRs) 9 | --- 10 | 11 | # Summary 12 | [summary]: #summary 13 | 14 | Add new Nix language builtins, to allow explicit use of pure evaluation and evaluation caching. 15 | 16 | # Motivation 17 | [motivation]: #motivation 18 | 19 | Pure evaluation and evaluation caching are currently intertwined with Flakes, which is still labeled as experimental and suffers from a number of problems. 20 | This RFC proposes a way to make these two features independent from Flakes, allowing them to be used in stable Nix via new builtins. 21 | 22 | Doing this is a big step towards Flakes stabilisation, while also enabling many use cases independently of it. 23 | 24 | # Detailed design 25 | [design]: #detailed-design 26 | 27 | ## Evaluation purity builtin 28 | 29 | ### New Nix language value type `RelativeNixPath` 30 | 31 | Contains: 32 | - `prefix`: A String representing the Nix path prefix the path is under 33 | - `components`, A list of Strings representing the relative path under the nix path prefix 34 | 35 | This type is only available for pure evaluation mode, while the traditional path value type is only available for impure evaluation mode. 36 | 37 | Very similar to a path: 38 | - Construction: 39 | - `` turns into `("foo", ["bar"])` 40 | - `./.` turns into the prefix of the Nix path entry the current Nix file is contained in and the relative path under it 41 | - Not allowed for paths not included in the Nix path prefixes 42 | - `/foo` is not allowed 43 | - Operations: 44 | - `dirOf` is only valid if the components aren't empty, returns a new path with the last component removed 45 | - `baseNameOf` is only valid if the components aren't empty, returns the last component 46 | - `toString` returns `` 47 | - `pathExists` only valid if the `filter` doesn't filter out this path 48 | - `readDir` filters out entries in the `filter` 49 | - `readFile` and `import` only works on files not `filter`ed out 50 | - `builtins.path` and `builtins.filterSource` work, but the filter function gets a RelativeNixPath as the first argument, and only paths not filtered out by the nix path `filter` are available 51 | - `typeOf` is `path` 52 | 53 | What happens if a pure part of evaluation returns such a path to an impure part of the code? 54 | 55 | FIXME: Okay this is not doable, we need to keep the same path value type 56 | We should just change the behavior of builtin functions when pure mode is enabled 57 | 58 | 59 | ### Introduce `builtins.relativeToNixPath` 60 | 61 | Type: `Path -> { prefix :: Path, components :: [ String ] }` 62 | 63 | Only available for pure evaluation mode paths 64 | For a given path, returns the nix path prefix its in and the relative components 65 | 66 | ### Introduce `builtins.pureImport` 67 | 68 | Type: `{ pureNixPath :: AttrsOf { path :: Path, filter :: Path -> String -> Bool } } -> Path -> Any` 69 | 70 | ``` 71 | pureImport { 72 | pureNixPath.project = { 73 | path = ./.; 74 | filter = path: type: true; 75 | }; 76 | } ./foo.nix 77 | ``` 78 | 79 | Imports and evaluates a Nix file using pure evaluation mode and setting `builtins.nixPath` to `[ { prefix = "project"; path = ./.; filter = ...; hash = "..."; } ]`. 80 | The hash is computed lazily when needed, it hashes the contents of all included files. 81 | 82 | What are the restrictions related to path in pure eval mode? 83 | - The imported file, and any files it transitively imports for evaluation: 84 | - Must not contain absolute path expressions 85 | - All relative path expressions must refer to paths that escape all `builtins.nixPath` prefixes. 86 | E.g. `../../foo/bar` is only allowed if `../..` is in a nix path prefix 87 | - Parsed paths should be represented as a (prefix, components) pair, see `builtins.relativeToNixPath` below. 88 | - Overall it must be ensured that all paths during evaluation are in some nix path prefix 89 | - `toString` on such paths is not allowed 90 | - `dirOf` and `baseNameOf` is only valid if the parent path is a path under a nix path prefix 91 | - `readDir`, `readFile`, `readFileType`, `hashFile`, `import` and `scopedImport` is only valid on paths to files that aren't filtered out, it doesn't work on strings 92 | - `builtins.path` needs to be replaced with something better 93 | 94 | In the future, this builtins can be extended with more attribute arguments such as: 95 | - `currentSystem`: Allow pure code to access `builtins.currentSystem` 96 | - `currentTime` 97 | - `args`: With a potential new `builtins.args` that exposes the `--arg`/`--argstr`, allowing nested code to access it 98 | - And any other builtins that would normally be restricted to pure evaluation mode 99 | 100 | ### Introduce `builtins.inPureEvalMode` 101 | 102 | Type: `Bool` 103 | Default: `false` 104 | 105 | Whether this expression is evaluated in pure evaluation mode. This is almost an implementation detail, needed for thunks to know whether they need to be evaluated purely or not. 106 | 107 | The evaluator needs to ensure that this builtin cannot be shadowed, probably would be good for all builtins. 108 | 109 | ## Evaluation caching 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | - Maybe enforce that builtins cannot be overridden. Could be done only in pure evaluation, though it probably doesn't hurt to do it always. Or maybe only if `builtins.pureEval` is `true`, it cannot be set to `false`, should be enough 119 | - Add `builtins.allowedPaths`, some structure that can be queried for which paths are allowed 120 | By default set to all paths, impure, or maybe not set by default, along with `builtins.pure = false` 121 | - Add: 122 | ```nix 123 | builtins.pureImport { 124 | allowedPaths = builtins.gitTracked ./.; 125 | allowedPaths = ...; # optional, if not passed, no paths are allowed to be accessed 126 | currentSystem = ...; # optional, if not passed the builtin is disabled 127 | nixPath = ...; # optional, if not passed no nix paths can be resolved 128 | } file 129 | ``` 130 | 131 | Imports and evaluates `file` using the given pure environment configuration, exporting all of these variables under `builtins`, also setting `builtins.pure = true` or so 132 | - Evaluation makes sure that `builtins.allowedPaths` is checked before accessing a path 133 | - Add `builtins.cachedImport file args`, as described below, ensures that `builtins.pure = true` and uses all of the variable `builtins` values as a hash key, in addition to the `args` 134 | - `builtins.lazyUpdate` 135 | 136 | ## New Nix Builtins 137 | 138 | 139 | 140 | `builtins.allowedPaths` 141 | 142 | 143 | 144 | ### `builtins.lazyUpdate` 145 | 146 | Type: `Attrs -> Attrs -> Attrs` 147 | 148 | And maybe its cousin: `builtins.lazyListConcat` 149 | 150 | `builtins.lazyUpdate a b` is semantically similar to `a // b`. 151 | The difference is that when accessing an attribute of the result, if the attribute contained in `b`, `a` is not evaluated. 152 | This works: 153 | 154 | ```nix 155 | (builtins.lazyUpdate (throw "error") { x = null; }).x 156 | ``` 157 | 158 | This is effectively https://github.com/NixOS/nix/issues/4090 and has many applications. 159 | 160 | ### `builtins.cachedImport` 161 | 162 | Type: `Path -> Any -> Any` 163 | 164 | `builtins.cachedImport file args` is semantically similar to `import file args`. 165 | This builtin can only be used when pure evaluation mode is enabled. 166 | `args` and the result needs to be serializable. 167 | The result of the evaluation is cached under a combined key of: 168 | - The hash of all paths that can be accessed by pure evaluation mode, which will have to include `file` itself. 169 | - The hash of the serialization of `args` 170 | 171 | ### `builtins.pureImport` 172 | 173 | Type: `{ (optional) nixPath :: AttrsOf Path, allowedPaths :: [ Path ] } -> Path -> Any` 174 | 175 | ```nix 176 | builtins.pureImport { 177 | nixPath.nixpkgs = ...; 178 | root = ./.; 179 | allowedPaths = [ 180 | ./foo.nix 181 | ]; 182 | } file 183 | ``` 184 | is semantically similar to `import file`. 185 | However the file is evaluated with pure evaluation restricted to the paths specified in `allowedPaths`. 186 | This means that if `file` were to access `./bar.nix` it would fail, but `./foo.nix` would work. 187 | Also, `./foo.nix` would evaluate to just `/foo.nix`. 188 | 189 | ### `builtins.gitTracked :: Path -> [ Path ]` 190 | 191 | Like https://github.com/NixOS/nix/issues/2944 192 | 193 | ## Nix CLI changes 194 | 195 | When pure evaluation mode is enabled, `nix-build` evaluates to something nice using both `pureEval` and `cacheEval`. 196 | 197 | Look for `pure.nix` or so 198 | 199 | 200 | 201 | ``` 202 | let 203 | rev = "..."; 204 | sha256 = "..."; 205 | pkgs = import {}; 206 | x = builtins.cache "x" [ pkgs ] pkgs.hello; 207 | # No point, we need to always evaluate the entire `pkgs` to figure out whether we hit the cache 208 | y = builtins.cache ( 209 | (import (fetch { inherit rev sha256; }) {}).hello.outPath 210 | ); 211 | # Cache key is composed of `rev`, `sha256` and the hash of the expression 212 | # Evaluation is performed such that the expression can't access any variables outside 213 | 214 | # The free variables become the cache key! 215 | 216 | z = builtins.cache (builtins.foldl' (acc: el: acc + el) 0 (range 0 1000000)); 217 | # This would also work!! 218 | in 219 | ``` 220 | 221 | `builtins.cache :: FileSet -> a -> a` 222 | 223 | Evaluates `a`, caches it using the following as keys: 224 | - The hash of all files in the file set 225 | - All free variables in the second argument 226 | 227 | All keys and the return value itself must be serializable 228 | 229 | Serializable values are: 230 | - `null`, booleans, integers, floats 231 | - Strings (also with context), paths 232 | - Attribute sets and lists of serializable values 233 | 234 | Not serializable are: 235 | - Functions 236 | 237 | Problem: We can't really sell filesets to Nix 238 | 239 | How about this instead: 240 | 241 | ``` 242 | builtins.withFilesystem { 243 | "flake.nix" = ./flake.nix; 244 | src = {}; 245 | } foo 246 | ``` 247 | 248 | 249 | `builtins.pureEval :: { nixPath :: AttrsOf Path, allowedPaths :: NestedAttrsOf Path, allowedURIs :: [ String ] } -> a -> a` 250 | Evaluates the second argument with a filesystem according to the first argument 251 | 252 | Could also allow ETAG header fetching to use as a key, probably also what flakes does 253 | 254 | Expression must not have any free variables 255 | 256 | The first argument must also be evaluated with pure eval mode. 257 | 258 | How about having a file `pureEval.nix`. If it exists it will become the arguments to `builtins.pureEval`, and evaluate `default.nix` 259 | 260 | It's a good thing to only be able to access local files 261 | 262 | How about a `root.nix`. It indicates the root of the Nix flake. Only nested files can be accessed. 263 | 264 | `builtins.cacheExpr :: Scope -> Expression -> Value` 265 | 266 | ```nix 267 | builtins.cacheExpr { x = 10; } x 268 | ``` 269 | 270 | Can only refer to variables introduced in the scope 271 | 272 | This requires being nested in a `builtins.withFilesystem`, because otherwise it would cache the entire filesystem, which is too much! 273 | 274 | Aka, requires restricted eval 275 | 276 | Also requires pure evaluation 277 | 278 | ```nix 279 | builtins.pureEval { 280 | nixPath.nixpkgs = ; 281 | } (import ./pure.nix) 282 | ``` 283 | 284 | `builtins.pureEval :: a -> a` 285 | 286 | Requires restricted eval 287 | I guess this should require the expression to not have any free variables? 288 | 289 | Should maybe work the same as `builtins.cache`: Free variables become part of the input 290 | But how to evaluate those? In pure eval or not? 291 | 292 | ``` 293 | builtins.withFilesystem (builtins.gitTracked ./.) ( 294 | let 295 | 296 | in 297 | ) 298 | ``` 299 | 300 | 301 | # Examples and Interactions 302 | [examples-and-interactions]: #examples-and-interactions 303 | 304 | This section illustrates the detailed design. This section should clarify all 305 | confusion the reader has from the previous sections. It is especially important 306 | to counterbalance the desired terseness of the detailed design; if you feel 307 | your detailed design is rudely short, consider making this section longer 308 | instead. 309 | 310 | Example: Caching top-level derivations for a fetched nixpkgs: 311 | ```nix 312 | # nixpkgs.nix 313 | { nixpkgsRev, nixpkgsSha256, system }: 314 | let 315 | nixpkgs = fetchTarball { 316 | url = "https://github.com/NixOS/nixpkgs/tarball/${nixpkgsRev}"; 317 | sha256 = nixpkgsSha256; 318 | }; 319 | in import nixpkgs { inherit system; } 320 | ``` 321 | 322 | ```nix 323 | # eval.nix 324 | { nixpkgsSpec, attribute }: 325 | let 326 | pkgs = import ./nixpkgs.nix nixpkgsSpec; 327 | in { 328 | # We don't want all attributes because that includes things like `buildInputs` 329 | inherit (pkgs.${attribute}) 330 | type 331 | drvPath 332 | outPath 333 | outputs 334 | outputName; 335 | } 336 | ``` 337 | 338 | ```nix 339 | # pure.nix 340 | let 341 | nixpkgsSpec = { 342 | nixpkgsRev = "..."; 343 | nixpkgsSha256 = "..."; 344 | system = "..."; 345 | }; 346 | 347 | pkgs = import ./nixpkgs.nix nixpkgsSpec; 348 | 349 | cachedPkgs = mapAttrs (name: value: 350 | let 351 | cached = builtins.cachedImport ./eval.nix { 352 | inherit nixpkgsRev nixpkgsSha256 system; 353 | attribute = name; 354 | }; 355 | in 356 | builtins.lazyUpdate 357 | value 358 | cached 359 | ) pkgs; 360 | in cachedPkgs 361 | ``` 362 | 363 | ```nix 364 | # default.nix 365 | # `builtins.cachedImport` is only allowed in pure evaluation mode 366 | builtins.pureImport { 367 | # This is not strictly necessary, it limits access of ./pure.nix to just two files 368 | # But it does mean that changes to other files won't end up influencing whether it's a cache hit 369 | allowedPaths = { 370 | "eval.nix" = ./eval.nix; 371 | "nixpkgs.nix" = ./nixpkgs.nix; 372 | }; 373 | } ./pure.nix 374 | ``` 375 | 376 | Then calling `nix-build -A hello` will be very fast since it won't have to compute the `outPath`. However if `hello.meta` is used, it will be slow since `eval.nix` doesn't output `meta` for it to be cached. This could be added though. 377 | 378 | The cache keys will be: 379 | - The hash of `./eval.nix`, since they're the only files available to pure evaluation mode 380 | - nixpkgsRev, nixpkgsSha256, system and attribute 381 | 382 | It's not great how this requires multiple files. And using `builtins.writeFile "*.nix"` is weird, though it would work with a "single" file then. 383 | 384 | 385 | 386 | Further ideas that could be implemented: 387 | - Caching of all Nixpkgs packages within nixpkgs itself 388 | 389 | - RFC 140 390 | 391 | The `builtins.pureImport` primitive could provide an interface to local [lazy paths](https://github.com/NixOS/nix/pull/6530): 392 | ```nix 393 | path: builtins.pureImport { root = path; allowedPaths = [ path ]; } (builtins.toFile "root" "/.") 394 | ``` 395 | 396 | # Drawbacks 397 | [drawbacks]: #drawbacks 398 | 399 | Why should we *not* do this? 400 | 401 | # Alternatives 402 | [alternatives]: #alternatives 403 | 404 | What other designs have been considered? What is the impact of not doing this? 405 | 406 | # Unresolved questions 407 | [unresolved]: #unresolved-questions 408 | 409 | What parts of the design are still TBD or unknowns? 410 | 411 | # Future work 412 | [future]: #future-work 413 | 414 | What future work, if any, would be implied or impacted by this feature 415 | without being directly part of the work? 416 | --------------------------------------------------------------------------------