├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── db ├── default.nix ├── lwnvulns ├── .gitignore ├── Cargo.lock ├── Cargo.toml └── src │ ├── bin │ ├── instructions.md │ ├── new.rs │ ├── reformat.rs │ └── updatedb.rs │ ├── lib.rs │ ├── parse.rs │ ├── tokenize.rs │ ├── transform.rs │ └── writer.rs ├── notate.sh ├── ported-notes.sh ├── shell.nix └── state ├── notate_state.sh └── port_state.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .bash_hist 2 | result 3 | .#* 4 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: nix 2 | script: 3 | - nix-build . -A ci 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Graham Christensen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # lwn-vulns automation 2 | 3 | ## How It Works 4 | 5 | In the root of this repository is a file titled `db`. It is a new-line 6 | separated list of vulnerability IDs from the LWN database. 7 | 8 | The [`new`][new] tool screen-scrapes the database until it finds two 9 | full pages with no new vulnerabilities. 10 | 11 | The [`reformat`][reformat] tool updates an issue in progress to 12 | highlight remaining items to do. 13 | 14 | When a roundup issue is closed, the contents of it are sent to the 15 | [`updatedb`][updatedb] command which outputs a list of the 16 | vulnerability IDs marked as done in the roundup. This should then 17 | appended to the `db` file. 18 | 19 | The shell script `./notate.sh` peruses all the recent commits and 20 | prompts for security notes to be added. 21 | 22 | Finally, `./ported-notes.sh` takes the security notes and makes a 23 | security announcement email's text. 24 | 25 | ### Tool Interface 26 | 27 | These tools are a bit like plumbing right now, and I would like some 28 | simpler user interfaces to be developed on top. Right now, I think 29 | they work okay. 30 | 31 | ## Lifecycle of an Issue 32 | 33 | Here is a typical workflow. I'll be using `pbcopy` and `pbpaste` to 34 | copy and paste to/from my system clipboard. On Linux, it may be 35 | `xclip -sel clip -i` and `xclip -sel clip -o`. 36 | 37 | ### Build the tools 38 | 39 | Start with `nix-build ./default.nix -A lwnvulns.pkg`. This will create 40 | the `result` symlink referenced here. Note, if you're doing 41 | development, you can enter a `nix-shell` (just run `nix-shell`) and 42 | replace `./result/bin/` with `cargo run --bin `: 43 | 44 | security$ nix-shell 45 | 46 | [nix-shell:~/projects/security/lwnvulns]$ pbpaste | cargo run --bin reformat | pbcopy 47 | Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs 48 | Running `target/debug/reformat` 49 | 50 | 51 | ### Making a new issue 52 | 53 | $ ./result/bin/new | pbcopy 54 | Page 0 yielded 30 issues, after 0 pages with nothing 55 | Page 1 yielded 0 issues, after 0 pages with nothing 56 | Page 2 yielded 0 issues, after 1 pages with nothing 57 | 58 | My clipboard now contains a full report to open as an issue. It has a 59 | few sections of things in here for you to do. Starting with: 60 | 61 | # POSTING TODO (DELETE PRIOR TO POSTING) 62 | 63 | - [ ] Title it "Vulnerability Roundup " 64 | - [ ] Update the last roundup link 65 | - [ ] CC everyone who participated in the previous roundup 66 | - [ ] Label with `security` 67 | 68 | 69 | Where, obviously, `` is the last one +1. For example, if the last 70 | one was Roundup 9, this one would be Roundup 10. A bit later there 71 | will be a place to put a link to the previous roundup: 72 | 73 | 74 | Here are all the vulnerabilities from https://lwn.net/Vulnerabilities 75 | since our [last roundup]() 76 | 77 | between those two `()`. Make sure to correctly find the latest roundup 78 | and update this link accordingly. 79 | 80 | Then: 81 | 82 | cc: . 83 | 84 | Visit the last roundup and review all the contributors in the sidebar. 85 | It will say something like "7 participants". Make sure each one of 86 | those people are CC'd in the new issue. This way it is easy for people 87 | to join and drop out of roundups. If they participate in one, they'll 88 | be tagged in the next one. If they don't participate in that one, they 89 | won't be tagged on the one after that. 90 | 91 | ### Updating an issue 92 | 93 | 1. Refresh the issue's page. This is important to make sure we don't 94 | accidentally delete progress not loaded on your page. 95 | 2. Click edit on the issue, and copy the full markdown contents. 96 | 3. Run it through the [`reformat`][reformat] tool like this: `pbpaste 97 | | ./result/bin/reformat | pbcopy`. 98 | 4. Paste the newly altered contents in to the issue, and click Save. 99 | 100 | ### Closing an issue 101 | 102 | After the roundup is done and all the issues are solved, make sure to 103 | finish out the list at the top of each issue. 104 | 105 | ### Updating the database 106 | 107 | 1. Refresh the issue's page. This is important to make sure we don't 108 | accidentally delete progress not loaded on your page. 109 | 2. Click edit on the issue, and copy the full markdown contents. 110 | 3. `pbpaste | ./result/bin/updatedb >> db` 111 | 4. Commit these changes, and open a PR with the new changes. 112 | 113 | ### Review and backport commits from master to stable (`release-16.09`) 114 | 115 | Run `./notate.sh` from within the nixpkgs checkout to go through each 116 | commit since the previous review. It will ask if the commit should 117 | have security notes attached to it. Saying yes will open an editor to 118 | add notes. Try looking at release notes or the pull request to 119 | determine if there are security implications. If there are, add CVE 120 | information and perhaps some notes about what the issue is. Make sure 121 | any security fixes to master are applied to the stable branch as well. 122 | If not, cherry-pick them yourself. If you're not sure, open a PR with 123 | the backported commits. 124 | 125 | In the `state/notate_state.sh` state file, we track the last commit to 126 | be reviewed. 127 | 128 | ### Creating an Announcement 129 | 130 | 1. `cd` in to a nixpkgs clone and run `ported-notes.sh`. It 131 | will output a rough template of all the announcements to make, but 132 | make sure to audit it and review, by following the remainder of 133 | these steps. 134 | 2. Update the link at the end of the output to point to the latest 135 | security vulnerability roundup. 136 | 3. Commit and push and open a PR with the updated ported state file. 137 | 4. Send the generated email 138 | 139 | ## Developing 140 | 141 | Run `nix-shell` and within that, `cargo run --bin 142 | new|updatedb|reformat` etc. 143 | 144 | ### Architecture 145 | This tooling includes a tokenizer for tokenizing the issues, a parser 146 | to build a "syntax tree", and then various AST transformations to 147 | update the document later on. There is a tool for writing out a syntax 148 | tree as text. The [`new`][new] tool which generates new reports emits 149 | tokens to the parser, and then uses the writer to output the report. 150 | 151 | It is important to me that code be well formatted and have tests. One 152 | place in particular that I have failed to adhere to these standards in 153 | particular is the [`new`][new] code... this isn't an excuse to get 154 | sloppy. Sorry. :( 155 | 156 | ## Dataformat 157 | 158 | ### Terms 159 | 160 | #### Document 161 | 162 | A document is the entire markdown contents of a Github Issue. A 163 | document begins with a **preamble**, and ends with a **report**. 164 | 165 | #### Preamble 166 | 167 | The preamble is arbitrary text and has no specific rules about its 168 | content. When being parsed and generated, the preamble is left 169 | completely alone and is to be preserved bit-for-bit when outputted. 170 | 171 | The preamble is begins at the start of the document, and ends at the 172 | first occurance of a **Header**. 173 | 174 | #### Report 175 | 176 | The Report is a collection of **Headers** and **Issues**, where a 177 | Header is typically followed by zero or more Issues. 178 | 179 | **Any lines of data inside the Report which is not a Header or an 180 | Issue is considered garbage, and should be discarded.** 181 | 182 | #### Header 183 | 184 | A Header is defined by the following regular expression: 185 | 186 | ^### (.*) \((\d+) issues?\)( .*)?\n$ 187 | (1) (2) (3) 188 | 189 | (1) Package Name 190 | (2) Issue Count 191 | (3) Additional Notes (optional) 192 | 193 | The header is designed to start with arbitrary text describing the 194 | affected packages, the number of issues affecting the package, and 195 | then optional notes. Note that the number of issues in the header 196 | _does not_ necessarily reflect the true number of contained issues, 197 | however a well behaved parser will correctly update the counter. Note 198 | that the plural `s` in `issues` is optional. 199 | 200 | Here are some example headers: 201 | 202 | ### jasper (2 issues) 203 | ### jasper (0 issues) 204 | ### jasper (1 issue) 205 | ### foo bar baz tux!!! (1 issues) extra notes go here 206 | 207 | #### Issue 208 | 209 | An Issue is defined by the following regular expression: 210 | 211 | ^ ?- \[(x| )\] \[`#(.*)`\]\((.*)\) (.*)$ 212 | (1) (2) (3) (4) 213 | 214 | (1) Completion Indicator (x == complete) 215 | (2) Vulnerability ID 216 | (3) URL for the Vulnerability 217 | (4) Arbitraty text describing the vulnerability 218 | 219 | 220 | The issue is designed to be in a markdown formatted list with a 221 | beginning checkbox and a link. This checkbox may either be filled or 222 | unfilled, following a link indicating the primary URL for this issue. 223 | The link's text, currently, must start with a #, followed by an 224 | identifier to identify this issue. It is assumed this identifier is 225 | unique to this issue, and that this issue will never need to be 226 | addressed again. Following the link may be arbitrary content. The 227 | markdown list line may or may not be prefixed by a blank space. 228 | 229 | Here are some example Issues: 230 | 231 | - [x] [`#705362`](https://lwn.net/Vulnerabilities/705362/) ([search](http://search.nix.gsc.io/?q=bind&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=bind+in%3Apath&type=Code)) bind: denial of service 232 | - [ ] [`#705917`](https://lwn.net/Vulnerabilities/705917/) ([search](http://search.nix.gsc.io/?q=java-1.8.0-openjdk-aarch32&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=java-1.8.0-openjdk-aarch32+in%3Apath&type=Code)) java-1.8.0-openjdk-aarch32: multiple vulnerabilities 233 | - [x] [`#705362`](https://lwn.net/Vulnerabilities/705362/) ([search](http://search.nix.gsc.io/?q=bind&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=bind+in%3Apath&type=Code)) bind: denial of service 234 | - [ ] [`#705917`](https://lwn.net/Vulnerabilities/705917/) ([search](http://search.nix.gsc.io/?q=java-1.8.0-openjdk-aarch32&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=java-1.8.0-openjdk-aarch32+in%3Apath&type=Code)) java-1.8.0-openjdk-aarch32: multiple vulnerabilities 235 | 236 | ### Example Document 237 | 238 | Here are all the vulnerabilities from https://lwn.net/Vulnerabilities 239 | ## Notes on the list 240 | 1. The reports have been roughly grouped by the package name. This 241 | isn't perfect, but is intended to help identify if a whole group 242 | ### This is valid too, because it doesn't have an issue count! 243 | - [ ] even this isn't counted! 244 | 245 | 246 | ### Assorted (31 issues) 247 | - [ ] [`#705568`](https://lwn.net/Vulnerabilities/705568/) ([search](http://search.nix.gsc.io/?q=libvirt&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=libvirt+in%3Apath&type=Code)) libvirt: privilege escalation 248 | - [ ] [`#705361`](https://lwn.net/Vulnerabilities/705361/) ([search](http://search.nix.gsc.io/?q=java&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=java+in%3Apath&type=Code)) java: unspecified vulnerability 249 | - [ ] [`#705578`](https://lwn.net/Vulnerabilities/705578/) ([search](http://search.nix.gsc.io/?q=qemu&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=qemu+in%3Apath&type=Code)) qemu: multiple vulnerabilities 250 | This stuff is garbage and will be deleted when the parser is run again. 251 | ### tiff (2 issues) 252 | - [x] [`#705364`](https://lwn.net/Vulnerabilities/705364/) ([search](http://search.nix.gsc.io/?q=tiff&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=tiff+in%3Apath&type=Code)) tiff: multiple vulnerabilities 253 | - [x] [`#635993`](https://lwn.net/Vulnerabilities/635993/) ([search](http://search.nix.gsc.io/?q=tiff&i=fosho&repos=nixos-nixpkgs), [files](https://github.com/NixOS/nixpkgs/search?utf8=%E2%9C%93&q=tiff+in%3Apath&type=Code)) tiff: multiple vulnerabilities 254 | 255 | # Addendum 256 | 257 | ## Why LWN? Why not NVD? 258 | 259 | LWN nicely aggregates reports from distributions, who also aggregate 260 | CVE IDs they are responding to. This means instead of checking several 261 | CVE IDs individually, we know we just need to update a package. 262 | 263 | NVD and other CVE databases are frequently dreadfully out of date, and 264 | are won't have data for a CVE data for a long time, where as LWN will 265 | already have information about the report. 266 | 267 | ## Has LWN approved this? 268 | 269 | Yes. 270 | 271 | ## `new` emits `Problem with the SSL CA cert (path? access rights?)`? 272 | 273 | I was missing a `/etc/ssl/certs/ca-certificates.crt` and copied my 274 | `/etc/ssl/certs/ca-bundle.crt` to be there... _shrug_. 275 | 276 | 277 | ## Why rust? 278 | 279 | I originally wrote this tooling in python, but wanted to have strong 280 | typing to provide structure to the parser and tokenizer. I don't any 281 | functional languages that are vogue in the Nix ecosystem. 282 | 283 | [new]: ./lwnvulns/src/bin/new.rs 284 | [reformat]: ./lwnvulns/src/bin/reformat.rs 285 | [updatedb]: ./lwnvulns/src/bin/updatedb.rs 286 | -------------------------------------------------------------------------------- /db: -------------------------------------------------------------------------------- 1 | 318382 2 | 475668 3 | 480386 4 | 489072 5 | 500144 6 | 501450 7 | 514067 8 | 529991 9 | 536068 10 | 537753 11 | 559200 12 | 561443 13 | 568668 14 | 569768 15 | 570330 16 | 570812 17 | 575639 18 | 576777 19 | 578785 20 | 580396 21 | 580994 22 | 586336 23 | 588035 24 | 591371 25 | 592272 26 | 592675 27 | 594740 28 | 595046 29 | 595998 30 | 597671 31 | 598449 32 | 598856 33 | 601056 34 | 604034 35 | 604237 36 | 607575 37 | 610398 38 | 610419 39 | 615071 40 | 616041 41 | 616163 42 | 616446 43 | 619213 44 | 619813 45 | 620056 46 | 623865 47 | 624311 48 | 624315 49 | 624610 50 | 625494 51 | 626653 52 | 627589 53 | 628611 54 | 629676 55 | 629686 56 | 629994 57 | 632256 58 | 632571 59 | 633086 60 | 633536 61 | 633546 62 | 634468 63 | 635283 64 | 635765 65 | 635993 66 | 636684 67 | 638448 68 | 638544 69 | 639393 70 | 639578 71 | 639784 72 | 640613 73 | 641431 74 | 642646 75 | 642649 76 | 643372 77 | 644037 78 | 644511 79 | 645632 80 | 645925 81 | 646558 82 | 646898 83 | 646994 84 | 647618 85 | 647621 86 | 649716 87 | 650307 88 | 650896 89 | 651766 90 | 651768 91 | 651775 92 | 652174 93 | 652551 94 | 652799 95 | 652803 96 | 654275 97 | 654279 98 | 654283 99 | 654887 100 | 655117 101 | 655402 102 | 655645 103 | 656894 104 | 656982 105 | 657320 106 | 657322 107 | 657406 108 | 657992 109 | 658202 110 | 658311 111 | 658600 112 | 658823 113 | 658939 114 | 659043 115 | 659284 116 | 660668 117 | 660894 118 | 660897 119 | 661762 120 | 661765 121 | 661900 122 | 662052 123 | 662057 124 | 662269 125 | 662905 126 | 663069 127 | 663078 128 | 663516 129 | 663793 130 | 664215 131 | 664646 132 | 664752 133 | 665238 134 | 665241 135 | 665242 136 | 665248 137 | 665254 138 | 665808 139 | 665921 140 | 666133 141 | 666755 142 | 666889 143 | 666890 144 | 667153 145 | 667314 146 | 667315 147 | 668127 148 | 668130 149 | 668331 150 | 668545 151 | 668547 152 | 668949 153 | 669041 154 | 669403 155 | 669405 156 | 669406 157 | 669408 158 | 669524 159 | 669529 160 | 669659 161 | 669754 162 | 669862 163 | 670061 164 | 671098 165 | 671100 166 | 671444 167 | 671445 168 | 671465 169 | 671736 170 | 671739 171 | 672076 172 | 672312 173 | 672314 174 | 672317 175 | 672436 176 | 672559 177 | 672561 178 | 672564 179 | 673018 180 | 673451 181 | 673455 182 | 673458 183 | 673463 184 | 673469 185 | 673582 186 | 673772 187 | 673777 188 | 673782 189 | 674068 190 | 674069 191 | 674257 192 | 674258 193 | 674260 194 | 674262 195 | 674266 196 | 674380 197 | 674390 198 | 674491 199 | 674493 200 | 674494 201 | 674495 202 | 674496 203 | 674608 204 | 674609 205 | 674704 206 | 674706 207 | 674707 208 | 674833 209 | 674834 210 | 674835 211 | 674836 212 | 674837 213 | 674840 214 | 674841 215 | 674842 216 | 674843 217 | 674928 218 | 674929 219 | 675045 220 | 675049 221 | 675050 222 | 675051 223 | 675225 224 | 675227 225 | 675228 226 | 675229 227 | 675368 228 | 675369 229 | 675371 230 | 675372 231 | 675696 232 | 675698 233 | 675700 234 | 675701 235 | 675702 236 | 675820 237 | 675823 238 | 675830 239 | 675832 240 | 675834 241 | 676077 242 | 676082 243 | 676089 244 | 676094 245 | 676097 246 | 676106 247 | 676108 248 | 676272 249 | 676274 250 | 676784 251 | 676786 252 | 676787 253 | 676791 254 | 676794 255 | 676795 256 | 676796 257 | 676797 258 | 676927 259 | 676929 260 | 676932 261 | 677103 262 | 677107 263 | 677115 264 | 677332 265 | 677608 266 | 677956 267 | 677958 268 | 677959 269 | 677960 270 | 677961 271 | 677963 272 | 677964 273 | 677970 274 | 677975 275 | 677979 276 | 677981 277 | 677982 278 | 677984 279 | 678143 280 | 678148 281 | 678151 282 | 678156 283 | 678159 284 | 678161 285 | 678163 286 | 678387 287 | 678388 288 | 678389 289 | 678395 290 | 678626 291 | 678628 292 | 678631 293 | 678633 294 | 678634 295 | 678807 296 | 678809 297 | 678811 298 | 678818 299 | 678822 300 | 679124 301 | 679128 302 | 679130 303 | 679131 304 | 679255 305 | 679264 306 | 679400 307 | 679401 308 | 679404 309 | 679413 310 | 679612 311 | 679613 312 | 679614 313 | 679615 314 | 679616 315 | 679617 316 | 679618 317 | 679619 318 | 679620 319 | 679627 320 | 679629 321 | 679630 322 | 679760 323 | 679761 324 | 679764 325 | 679765 326 | 679766 327 | 680031 328 | 680036 329 | 680038 330 | 680040 331 | 680041 332 | 680044 333 | 680178 334 | 680184 335 | 680318 336 | 680320 337 | 680462 338 | 680601 339 | 680602 340 | 680793 341 | 680794 342 | 680795 343 | 680797 344 | 680800 345 | 681096 346 | 681098 347 | 681100 348 | 681103 349 | 681270 350 | 681272 351 | 681279 352 | 681387 353 | 681390 354 | 681391 355 | 681393 356 | 681395 357 | 681398 358 | 681399 359 | 681568 360 | 681647 361 | 681648 362 | 681753 363 | 681754 364 | 681755 365 | 681990 366 | 681991 367 | 682155 368 | 682157 369 | 682159 370 | 682384 371 | 682385 372 | 682387 373 | 682388 374 | 682389 375 | 682390 376 | 682567 377 | 682570 378 | 682571 379 | 682576 380 | 682756 381 | 682759 382 | 682760 383 | 682763 384 | 682974 385 | 682977 386 | 683314 387 | 683316 388 | 683317 389 | 683451 390 | 683452 391 | 683456 392 | 683716 393 | 683727 394 | 683728 395 | 683732 396 | 683843 397 | 683844 398 | 683853 399 | 683859 400 | 683985 401 | 683992 402 | 683994 403 | 683995 404 | 683998 405 | 684235 406 | 684236 407 | 684237 408 | 684319 409 | 684455 410 | 684456 411 | 684595 412 | 684596 413 | 684597 414 | 684745 415 | 684746 416 | 684747 417 | 684748 418 | 684749 419 | 684751 420 | 684752 421 | 684754 422 | 684755 423 | 685000 424 | 685002 425 | 685003 426 | 685004 427 | 685005 428 | 685007 429 | 685008 430 | 685009 431 | 685010 432 | 685012 433 | 685013 434 | 685137 435 | 685138 436 | 685139 437 | 685285 438 | 685287 439 | 685290 440 | 685291 441 | 685293 442 | 685294 443 | 685409 444 | 685412 445 | 685491 446 | 685492 447 | 685493 448 | 685867 449 | 685870 450 | 685873 451 | 685875 452 | 685877 453 | 685879 454 | 685881 455 | 685883 456 | 685885 457 | 685886 458 | 685887 459 | 685892 460 | 686084 461 | 686085 462 | 686288 463 | 686289 464 | 686291 465 | 686293 466 | 686446 467 | 686448 468 | 686450 469 | 686454 470 | 686574 471 | 686575 472 | 686577 473 | 686578 474 | 686579 475 | 686580 476 | 686747 477 | 686749 478 | 686751 479 | 686754 480 | 686756 481 | 686761 482 | 686769 483 | 686857 484 | 686861 485 | 686863 486 | 686864 487 | 686867 488 | 687039 489 | 687040 490 | 687042 491 | 687043 492 | 687044 493 | 687047 494 | 687049 495 | 687224 496 | 687227 497 | 687229 498 | 687231 499 | 687234 500 | 687235 501 | 687394 502 | 687395 503 | 687396 504 | 687397 505 | 687398 506 | 687400 507 | 687590 508 | 687591 509 | 687592 510 | 687596 511 | 687597 512 | 687599 513 | 687711 514 | 687713 515 | 687714 516 | 687715 517 | 687716 518 | 687718 519 | 687860 520 | 687861 521 | 687862 522 | 687864 523 | 687867 524 | 688051 525 | 688054 526 | 688055 527 | 688205 528 | 688206 529 | 688207 530 | 688208 531 | 688209 532 | 688210 533 | 688211 534 | 688446 535 | 688447 536 | 688448 537 | 688452 538 | 688454 539 | 688456 540 | 688457 541 | 688458 542 | 688596 543 | 688597 544 | 688728 545 | 688729 546 | 688730 547 | 688826 548 | 688827 549 | 688829 550 | 689242 551 | 689247 552 | 689249 553 | 689251 554 | 689252 555 | 689254 556 | 689256 557 | 689258 558 | 689260 559 | 689261 560 | 689263 561 | 689265 562 | 689266 563 | 689268 564 | 689273 565 | 689274 566 | 689277 567 | 689279 568 | 689280 569 | 689381 570 | 689384 571 | 689390 572 | 689391 573 | 689576 574 | 689578 575 | 689579 576 | 689580 577 | 689583 578 | 689714 579 | 689717 580 | 689718 581 | 690012 582 | 690015 583 | 690017 584 | 690019 585 | 690023 586 | 690024 587 | 690141 588 | 690142 589 | 690146 590 | 690400 591 | 690402 592 | 690403 593 | 690409 594 | 690411 595 | 690416 596 | 690420 597 | 690656 598 | 690791 599 | 690793 600 | 691093 601 | 691094 602 | 691095 603 | 691097 604 | 691098 605 | 691100 606 | 691101 607 | 691103 608 | 691104 609 | 691269 610 | 691497 611 | 691829 612 | 691830 613 | 691831 614 | 691832 615 | 691833 616 | 692024 617 | 692027 618 | 692028 619 | 692029 620 | 692030 621 | 692033 622 | 692035 623 | 692176 624 | 692179 625 | 692183 626 | 692185 627 | 692375 628 | 692378 629 | 692379 630 | 692380 631 | 692381 632 | 692518 633 | 692522 634 | 692523 635 | 692635 636 | 692849 637 | 692851 638 | 692853 639 | 692855 640 | 692856 641 | 692857 642 | 692859 643 | 692861 644 | 692862 645 | 692863 646 | 692864 647 | 692867 648 | 692923 649 | 692924 650 | 692925 651 | 692926 652 | 692930 653 | 692931 654 | 693003 655 | 693101 656 | 693102 657 | 693176 658 | 693177 659 | 693179 660 | 693180 661 | 693472 662 | 693473 663 | 693476 664 | 693478 665 | 693480 666 | 693574 667 | 693575 668 | 693576 669 | 693577 670 | 693578 671 | 693727 672 | 693865 673 | 693866 674 | 693867 675 | 694101 676 | 694102 677 | 694103 678 | 694238 679 | 694239 680 | 694240 681 | 694243 682 | 694514 683 | 694515 684 | 694623 685 | 694625 686 | 694626 687 | 694627 688 | 694629 689 | 694782 690 | 694783 691 | 694784 692 | 694785 693 | 694789 694 | 694861 695 | 694868 696 | 694871 697 | 694957 698 | 694959 699 | 694960 700 | 695089 701 | 695091 702 | 695092 703 | 695093 704 | 695097 705 | 695098 706 | 695167 707 | 695168 708 | 695169 709 | 695170 710 | 695319 711 | 695320 712 | 695321 713 | 695323 714 | 695325 715 | 695326 716 | 695458 717 | 695459 718 | 695556 719 | 695557 720 | 695558 721 | 695560 722 | 695561 723 | 695684 724 | 695689 725 | 695690 726 | 695692 727 | 695807 728 | 695808 729 | 695809 730 | 695953 731 | 695954 732 | 695956 733 | 695958 734 | 695959 735 | 695964 736 | 695968 737 | 696074 738 | 696077 739 | 696206 740 | 696207 741 | 696214 742 | 696215 743 | 696217 744 | 696219 745 | 696413 746 | 696418 747 | 696419 748 | 696549 749 | 696550 750 | 696551 751 | 696552 752 | 696553 753 | 696694 754 | 696696 755 | 696697 756 | 696698 757 | 696699 758 | 696700 759 | 696701 760 | 696702 761 | 696805 762 | 696807 763 | 696808 764 | 696810 765 | 696812 766 | 696813 767 | 696815 768 | 696829 769 | 696931 770 | 697020 771 | 697137 772 | 697140 773 | 697141 774 | 697262 775 | 697263 776 | 697264 777 | 697338 778 | 697339 779 | 697341 780 | 697438 781 | 697440 782 | 697441 783 | 697568 784 | 697571 785 | 697945 786 | 697946 787 | 697947 788 | 697948 789 | 697949 790 | 698054 791 | 698055 792 | 698136 793 | 698137 794 | 698139 795 | 698335 796 | 698337 797 | 698338 798 | 698339 799 | 698490 800 | 698492 801 | 698499 802 | 698651 803 | 698653 804 | 698658 805 | 698794 806 | 698795 807 | 698797 808 | 698984 809 | 698986 810 | 699161 811 | 699163 812 | 699175 813 | 699177 814 | 699682 815 | 699683 816 | 699684 817 | 699685 818 | 699803 819 | 699804 820 | 699805 821 | 699807 822 | 699946 823 | 699952 824 | 699955 825 | 699957 826 | 700105 827 | 700109 828 | 700110 829 | 700112 830 | 700113 831 | 700115 832 | 700116 833 | 700384 834 | 700386 835 | 700387 836 | 700388 837 | 700389 838 | 700391 839 | 700395 840 | 700519 841 | 700521 842 | 700649 843 | 700651 844 | 700652 845 | 700653 846 | 700654 847 | 700833 848 | 700835 849 | 700836 850 | 700837 851 | 700838 852 | 700840 853 | 700965 854 | 701138 855 | 701139 856 | 701140 857 | 701141 858 | 701142 859 | 701147 860 | 701151 861 | 701254 862 | 701347 863 | 701348 864 | 701352 865 | 701625 866 | 701626 867 | 701627 868 | 701629 869 | 701630 870 | 701631 871 | 701632 872 | 701633 873 | 701634 874 | 701727 875 | 701729 876 | 701734 877 | 701735 878 | 701739 879 | 701917 880 | 701918 881 | 701920 882 | 701921 883 | 701923 884 | 701926 885 | 701931 886 | 701997 887 | 701999 888 | 702000 889 | 702118 890 | 702121 891 | 702222 892 | 702224 893 | 702225 894 | 702312 895 | 702314 896 | 702315 897 | 702316 898 | 702317 899 | 702474 900 | 702475 901 | 702476 902 | 702477 903 | 702551 904 | 702552 905 | 702553 906 | 702621 907 | 702622 908 | 702623 909 | 702786 910 | 702787 911 | 702789 912 | 702790 913 | 702889 914 | 702896 915 | 702897 916 | 702898 917 | 703103 918 | 703104 919 | 703105 920 | 703106 921 | 703109 922 | 703111 923 | 703113 924 | 703114 925 | 703117 926 | 703119 927 | 703120 928 | 703121 929 | 703122 930 | 703123 931 | 703125 932 | 703241 933 | 703243 934 | 703244 935 | 703245 936 | 703246 937 | 703324 938 | 703325 939 | 703326 940 | 703328 941 | 703329 942 | 703330 943 | 703461 944 | 703466 945 | 703467 946 | 703606 947 | 703607 948 | 703609 949 | 703767 950 | 703769 951 | 703771 952 | 703862 953 | 703866 954 | 703868 955 | 703869 956 | 703975 957 | 703977 958 | 703978 959 | 703979 960 | 703983 961 | 703984 962 | 703985 963 | 703987 964 | 704120 965 | 704248 966 | 704249 967 | 704466 968 | 704467 969 | 704468 970 | 704469 971 | 704470 972 | 704471 973 | 704586 974 | 704589 975 | 704697 976 | 704698 977 | 704699 978 | 704700 979 | 704701 980 | 704702 981 | 704703 982 | 704704 983 | 704711 984 | 704712 985 | 704714 986 | 704737 987 | 704834 988 | 704922 989 | 704924 990 | 705119 991 | 705120 992 | 705124 993 | 705125 994 | 705211 995 | 705212 996 | 705213 997 | 705214 998 | 705216 999 | 705361 1000 | 705362 1001 | 705363 1002 | 705364 1003 | 705366 1004 | 705367 1005 | 705370 1006 | 705372 1007 | 705373 1008 | 705560 1009 | 705565 1010 | 705566 1011 | 705568 1012 | 705570 1013 | 705571 1014 | 705572 1015 | 705574 1016 | 705575 1017 | 705577 1018 | 705578 1019 | 705579 1020 | 705580 1021 | 705581 1022 | 705671 1023 | 705672 1024 | 705673 1025 | 705815 1026 | 705822 1027 | 705823 1028 | 705824 1029 | 705913 1030 | 705915 1031 | 705917 1032 | 705918 1033 | 706021 1034 | 706114 1035 | 706116 1036 | 706117 1037 | 706397 1038 | 706398 1039 | 706399 1040 | 706400 1041 | 706401 1042 | 706402 1043 | 706473 1044 | 706475 1045 | 706476 1046 | 706478 1047 | 706479 1048 | 706577 1049 | 706580 1050 | 706581 1051 | 706582 1052 | 706586 1053 | 706588 1054 | 706731 1055 | 706734 1056 | 706735 1057 | 706841 1058 | 706842 1059 | 706844 1060 | 706846 1061 | 706848 1062 | 706851 1063 | 706852 1064 | 707037 1065 | 707038 1066 | 707039 1067 | 707040 1068 | 707041 1069 | 707043 1070 | 707044 1071 | 707045 1072 | 707046 1073 | 707047 1074 | 707211 1075 | 707212 1076 | 707214 1077 | 707215 1078 | 707217 1079 | 707218 1080 | 707356 1081 | 707357 1082 | 707359 1083 | 707360 1084 | 707361 1085 | 707362 1086 | 707363 1087 | 707364 1088 | 707488 1089 | 707489 1090 | 707491 1091 | 707493 1092 | 707494 1093 | 707496 1094 | 707497 1095 | 707499 1096 | 707696 1097 | 707698 1098 | 707700 1099 | 707703 1100 | 707705 1101 | 707838 1102 | 707854 1103 | 707857 1104 | 707859 1105 | 707997 1106 | 708000 1107 | 708137 1108 | 708138 1109 | 708140 1110 | 708148 1111 | 708149 1112 | 708152 1113 | 708154 1114 | 708239 1115 | 708240 1116 | 708241 1117 | 708242 1118 | 708243 1119 | 708245 1120 | 708246 1121 | 708359 1122 | 708363 1123 | 708365 1124 | 708367 1125 | 708522 1126 | 708523 1127 | 708524 1128 | 708525 1129 | 708527 1130 | 708528 1131 | 708529 1132 | 708530 1133 | 708531 1134 | 708532 1135 | 708533 1136 | 708652 1137 | 708654 1138 | 708655 1139 | 708656 1140 | 708657 1141 | 708659 1142 | 708869 1143 | 708870 1144 | 708871 1145 | 708873 1146 | 708874 1147 | 708875 1148 | 708876 1149 | 708883 1150 | 708884 1151 | 708886 1152 | 708993 1153 | 708995 1154 | 708996 1155 | 708997 1156 | 709006 1157 | 709007 1158 | 709009 1159 | 709010 1160 | 709140 1161 | 709141 1162 | 709146 1163 | 709149 1164 | 709162 1165 | 709341 1166 | 709342 1167 | 709363 1168 | 709466 1169 | 709468 1170 | 709661 1171 | 709662 1172 | 709663 1173 | 709664 1174 | 709666 1175 | 709669 1176 | 709670 1177 | 709742 1178 | 709743 1179 | 709745 1180 | 709746 1181 | 709839 1182 | 709841 1183 | 709842 1184 | 709843 1185 | 709844 1186 | 709847 1187 | 709851 1188 | 709853 1189 | 709984 1190 | 709985 1191 | 709986 1192 | 709987 1193 | 709988 1194 | 709993 1195 | 710082 1196 | 710084 1197 | 710085 1198 | 710086 1199 | 710087 1200 | 710209 1201 | 710210 1202 | 710212 1203 | 710213 1204 | 710214 1205 | 710278 1206 | 710279 1207 | 710281 1208 | 710282 1209 | 710283 1210 | 710284 1211 | 710285 1212 | 710286 1213 | 710356 1214 | 710358 1215 | 710362 1216 | 710363 1217 | 710474 1218 | 710475 1219 | 710476 1220 | 710477 1221 | 710478 1222 | 710480 1223 | 710481 1224 | 710482 1225 | 710483 1226 | 710484 1227 | 710485 1228 | 710486 1229 | 710487 1230 | 710489 1231 | 710490 1232 | 710491 1233 | 710626 1234 | 710627 1235 | 710628 1236 | 710895 1237 | 710896 1238 | 710898 1239 | 710899 1240 | 710900 1241 | 711047 1242 | 711048 1243 | 711049 1244 | 711050 1245 | 711051 1246 | 711055 1247 | 711059 1248 | 711186 1249 | 711187 1250 | 711189 1251 | 711323 1252 | 711324 1253 | 711325 1254 | 711329 1255 | 711330 1256 | 711457 1257 | 711458 1258 | 711459 1259 | 711461 1260 | 711462 1261 | 711463 1262 | 711464 1263 | 711581 1264 | 711582 1265 | 711583 1266 | 711586 1267 | 711587 1268 | 711774 1269 | 711775 1270 | 711776 1271 | 711777 1272 | 711778 1273 | 711779 1274 | 711780 1275 | 711781 1276 | 711782 1277 | 711856 1278 | 711858 1279 | 711945 1280 | 711946 1281 | 712057 1282 | 712058 1283 | 712059 1284 | 712060 1285 | 712065 1286 | 712067 1287 | 712068 1288 | 712155 1289 | 712297 1290 | 712298 1291 | 712299 1292 | 712300 1293 | 712301 1294 | 712302 1295 | 712304 1296 | 712359 1297 | 712363 1298 | 712364 1299 | 712369 1300 | 712370 1301 | 712371 1302 | 712491 1303 | 712493 1304 | 712494 1305 | 712496 1306 | 712498 1307 | 712499 1308 | 712501 1309 | 712658 1310 | 712664 1311 | 712665 1312 | 712666 1313 | 712801 1314 | 712802 1315 | 712803 1316 | 713036 1317 | 713043 1318 | 713046 1319 | 713047 1320 | 713049 1321 | 713050 1322 | 713051 1323 | 713052 1324 | 713053 1325 | 713054 1326 | 713055 1327 | 713059 1328 | 713061 1329 | 713062 1330 | 713145 1331 | 713146 1332 | 713148 1333 | 713150 1334 | 713153 1335 | 713154 1336 | 713268 1337 | 713269 1338 | 713270 1339 | 713271 1340 | 713272 1341 | 713274 1342 | 713409 1343 | 713420 1344 | 713423 1345 | 713424 1346 | 713425 1347 | 713426 1348 | 713428 1349 | 713563 1350 | 713564 1351 | 713565 1352 | 713569 1353 | 713570 1354 | 713771 1355 | 713772 1356 | 713773 1357 | 713774 1358 | 713775 1359 | 713776 1360 | 713779 1361 | 713782 1362 | 713784 1363 | 713785 1364 | 713786 1365 | 713787 1366 | 713880 1367 | 713881 1368 | 713883 1369 | 713884 1370 | 713991 1371 | 713992 1372 | 713996 1373 | 714124 1374 | 714126 1375 | 714127 1376 | 714256 1377 | 714257 1378 | 714260 1379 | 714262 1380 | 714263 1381 | 714423 1382 | 714424 1383 | 714427 1384 | 714430 1385 | 714431 1386 | 714500 1387 | 714502 1388 | 714504 1389 | 714505 1390 | 714581 1391 | 714582 1392 | 714585 1393 | 714586 1394 | 714850 1395 | 714852 1396 | 714853 1397 | 715035 1398 | 715036 1399 | 715037 1400 | 715038 1401 | 715039 1402 | 715041 1403 | 715042 1404 | 715043 1405 | 715044 1406 | 715045 1407 | 715162 1408 | 715164 1409 | 715165 1410 | 715166 1411 | 715167 1412 | 715168 1413 | 715169 1414 | 715170 1415 | 715171 1416 | 715263 1417 | 715264 1418 | 715266 1419 | 715268 1420 | -------------------------------------------------------------------------------- /default.nix: -------------------------------------------------------------------------------- 1 | { pkgs ? import (builtins.fetchTarball "https://github.com/NixOS/nixpkgs-channels/tarball/nixos-unstable") {}, 2 | }: 3 | let 4 | inherit (pkgs) writeTextFile; 5 | inherit (pkgs.stdenv) mkDerivation; 6 | inherit (pkgs.rustPlatform) buildRustPackage; 7 | 8 | shellcheckCmd = '' 9 | echo "==========================================================" 10 | echo "----> Shellchecking:" 11 | for file in `find . -name '*.sh'`; do 12 | echo "----> $file" 13 | ${pkgs.shellcheck}/bin/shellcheck -e SC1090 -e SC1091 -x $file 14 | done 15 | ''; 16 | in rec { 17 | ci = writeTextFile { 18 | name = "tests"; 19 | text = '' 20 | ${lwnvulns.pkg} 21 | ${notate.pkgSrc} 22 | ${ported-notes.pkgSrc} 23 | ''; 24 | }; 25 | 26 | notate = rec { 27 | pkgSrc = mkDerivation { 28 | name = "notate-src"; 29 | src = ./notate.sh; 30 | 31 | unpackPhase = '' 32 | cp $src . 33 | ''; 34 | buildPhase = shellcheckCmd; 35 | 36 | installPhase = '' 37 | mkdir $out 38 | cp -r $src $out/notate.sh 39 | ''; 40 | }; 41 | }; 42 | 43 | ported-notes = rec { 44 | pkgSrc = mkDerivation { 45 | name = "ported-notes-src"; 46 | src = ./ported-notes.sh; 47 | 48 | unpackPhase = '' 49 | cp $src . 50 | ''; 51 | buildPhase = shellcheckCmd; 52 | 53 | installPhase = '' 54 | mkdir $out 55 | cp -r $src $out/ported-note.sh 56 | ''; 57 | }; 58 | }; 59 | 60 | 61 | lwnvulns = rec { 62 | dependencies = with pkgs; [ 63 | perl 64 | pkgconfig 65 | openssl.dev 66 | zlib.dev 67 | curl.dev 68 | ]; 69 | 70 | formatcheckCmd = '' 71 | echo "==========================================================" 72 | echo "----> Formatting:" 73 | for file in `find . -name '*.rs' -not -path '*/target/*'`; do # */ Hi emacs ... 74 | echo "----> $file" 75 | ${pkgs.rustfmt}/bin/rustfmt --write-mode=diff "$file" 76 | done 77 | ''; 78 | 79 | shell = mkDerivation { 80 | name = "nixos-security-tools-shell"; 81 | src = ./.; 82 | 83 | formatcheck = formatcheckCmd; 84 | 85 | buildInputs = dependencies ++ (with pkgs; [ 86 | rustfmt 87 | rustc 88 | cargo 89 | ]); 90 | 91 | shellHook = '' 92 | cd lwnvulns; 93 | ''; 94 | }; 95 | 96 | pkg = buildRustPackage { 97 | name = "lwn-vuln"; 98 | src = pkgSrc; 99 | 100 | depsSha256 = "19l6p7g8xy5hvy562saxaia1jxcbyrq657nnk7i055lla9i199p2"; 101 | 102 | preBuild = formatcheckCmd; 103 | 104 | buildInputs = dependencies; 105 | }; 106 | 107 | pkgSrc = mkDerivation { 108 | name = "nixos-security-tools-src"; 109 | src = builtins.filterSource ( 110 | path: type: 111 | let 112 | bpath = baseNameOf path; 113 | in !( 114 | ((builtins.substring 0 1 bpath) == ".") 115 | || (type == "symlink" && bpath == "result") 116 | || (type == "directory" && bpath == ".git") 117 | || (type == "directory" && bpath == "target") 118 | || (type == "file" && bpath == "db") 119 | || (type == "file" && bpath == "shell.nix") 120 | || (type == "file" && bpath == "default.nix") 121 | ) 122 | ) ./lwnvulns; 123 | 124 | buildPhase = formatcheckCmd; 125 | 126 | installPhase = '' 127 | cp -r $src $out 128 | ''; 129 | }; 130 | }; 131 | } 132 | -------------------------------------------------------------------------------- /lwnvulns/.gitignore: -------------------------------------------------------------------------------- 1 | target 2 | -------------------------------------------------------------------------------- /lwnvulns/Cargo.lock: -------------------------------------------------------------------------------- 1 | [root] 2 | name = "lwnvulns" 3 | version = "0.1.0" 4 | dependencies = [ 5 | "curl 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)", 6 | "lazy_static 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", 7 | "log 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", 8 | "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", 9 | "scraper 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", 10 | "serde 0.8.17 (registry+https://github.com/rust-lang/crates.io-index)", 11 | ] 12 | 13 | [[package]] 14 | name = "aho-corasick" 15 | version = "0.5.3" 16 | source = "registry+https://github.com/rust-lang/crates.io-index" 17 | dependencies = [ 18 | "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", 19 | ] 20 | 21 | [[package]] 22 | name = "bitflags" 23 | version = "0.7.0" 24 | source = "registry+https://github.com/rust-lang/crates.io-index" 25 | 26 | [[package]] 27 | name = "cssparser" 28 | version = "0.5.8" 29 | source = "registry+https://github.com/rust-lang/crates.io-index" 30 | dependencies = [ 31 | "encoding 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)", 32 | "matches 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 33 | ] 34 | 35 | [[package]] 36 | name = "curl" 37 | version = "0.4.1" 38 | source = "registry+https://github.com/rust-lang/crates.io-index" 39 | dependencies = [ 40 | "curl-sys 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", 41 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 42 | "openssl-probe 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", 43 | "openssl-sys 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", 44 | "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", 45 | ] 46 | 47 | [[package]] 48 | name = "curl-sys" 49 | version = "0.3.4" 50 | source = "registry+https://github.com/rust-lang/crates.io-index" 51 | dependencies = [ 52 | "gcc 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)", 53 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 54 | "libz-sys 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)", 55 | "openssl-sys 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)", 56 | "pkg-config 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", 57 | "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", 58 | ] 59 | 60 | [[package]] 61 | name = "debug_unreachable" 62 | version = "0.1.1" 63 | source = "registry+https://github.com/rust-lang/crates.io-index" 64 | dependencies = [ 65 | "unreachable 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 66 | ] 67 | 68 | [[package]] 69 | name = "ego-tree" 70 | version = "0.1.0" 71 | source = "registry+https://github.com/rust-lang/crates.io-index" 72 | 73 | [[package]] 74 | name = "encoding" 75 | version = "0.2.33" 76 | source = "registry+https://github.com/rust-lang/crates.io-index" 77 | dependencies = [ 78 | "encoding-index-japanese 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)", 79 | "encoding-index-korean 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)", 80 | "encoding-index-simpchinese 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)", 81 | "encoding-index-singlebyte 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)", 82 | "encoding-index-tradchinese 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)", 83 | ] 84 | 85 | [[package]] 86 | name = "encoding-index-japanese" 87 | version = "1.20141219.5" 88 | source = "registry+https://github.com/rust-lang/crates.io-index" 89 | dependencies = [ 90 | "encoding_index_tests 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 91 | ] 92 | 93 | [[package]] 94 | name = "encoding-index-korean" 95 | version = "1.20141219.5" 96 | source = "registry+https://github.com/rust-lang/crates.io-index" 97 | dependencies = [ 98 | "encoding_index_tests 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 99 | ] 100 | 101 | [[package]] 102 | name = "encoding-index-simpchinese" 103 | version = "1.20141219.5" 104 | source = "registry+https://github.com/rust-lang/crates.io-index" 105 | dependencies = [ 106 | "encoding_index_tests 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 107 | ] 108 | 109 | [[package]] 110 | name = "encoding-index-singlebyte" 111 | version = "1.20141219.5" 112 | source = "registry+https://github.com/rust-lang/crates.io-index" 113 | dependencies = [ 114 | "encoding_index_tests 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 115 | ] 116 | 117 | [[package]] 118 | name = "encoding-index-tradchinese" 119 | version = "1.20141219.5" 120 | source = "registry+https://github.com/rust-lang/crates.io-index" 121 | dependencies = [ 122 | "encoding_index_tests 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 123 | ] 124 | 125 | [[package]] 126 | name = "encoding_index_tests" 127 | version = "0.1.4" 128 | source = "registry+https://github.com/rust-lang/crates.io-index" 129 | 130 | [[package]] 131 | name = "fnv" 132 | version = "1.0.5" 133 | source = "registry+https://github.com/rust-lang/crates.io-index" 134 | 135 | [[package]] 136 | name = "futf" 137 | version = "0.1.2" 138 | source = "registry+https://github.com/rust-lang/crates.io-index" 139 | dependencies = [ 140 | "debug_unreachable 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 141 | "mac 0.0.2 (registry+https://github.com/rust-lang/crates.io-index)", 142 | ] 143 | 144 | [[package]] 145 | name = "gcc" 146 | version = "0.3.38" 147 | source = "registry+https://github.com/rust-lang/crates.io-index" 148 | 149 | [[package]] 150 | name = "gdi32-sys" 151 | version = "0.2.0" 152 | source = "registry+https://github.com/rust-lang/crates.io-index" 153 | dependencies = [ 154 | "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", 155 | "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 156 | ] 157 | 158 | [[package]] 159 | name = "html5ever" 160 | version = "0.5.4" 161 | source = "registry+https://github.com/rust-lang/crates.io-index" 162 | dependencies = [ 163 | "log 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", 164 | "mac 0.0.2 (registry+https://github.com/rust-lang/crates.io-index)", 165 | "phf 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 166 | "phf_codegen 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 167 | "rustc-serialize 0.3.21 (registry+https://github.com/rust-lang/crates.io-index)", 168 | "string_cache 0.2.29 (registry+https://github.com/rust-lang/crates.io-index)", 169 | "tendril 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", 170 | "time 0.1.35 (registry+https://github.com/rust-lang/crates.io-index)", 171 | ] 172 | 173 | [[package]] 174 | name = "kernel32-sys" 175 | version = "0.2.2" 176 | source = "registry+https://github.com/rust-lang/crates.io-index" 177 | dependencies = [ 178 | "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", 179 | "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 180 | ] 181 | 182 | [[package]] 183 | name = "lazy_static" 184 | version = "0.2.2" 185 | source = "registry+https://github.com/rust-lang/crates.io-index" 186 | 187 | [[package]] 188 | name = "libc" 189 | version = "0.2.17" 190 | source = "registry+https://github.com/rust-lang/crates.io-index" 191 | 192 | [[package]] 193 | name = "libz-sys" 194 | version = "1.0.7" 195 | source = "registry+https://github.com/rust-lang/crates.io-index" 196 | dependencies = [ 197 | "gcc 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)", 198 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 199 | "pkg-config 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", 200 | ] 201 | 202 | [[package]] 203 | name = "log" 204 | version = "0.3.6" 205 | source = "registry+https://github.com/rust-lang/crates.io-index" 206 | 207 | [[package]] 208 | name = "mac" 209 | version = "0.0.2" 210 | source = "registry+https://github.com/rust-lang/crates.io-index" 211 | 212 | [[package]] 213 | name = "matches" 214 | version = "0.1.4" 215 | source = "registry+https://github.com/rust-lang/crates.io-index" 216 | 217 | [[package]] 218 | name = "memchr" 219 | version = "0.1.11" 220 | source = "registry+https://github.com/rust-lang/crates.io-index" 221 | dependencies = [ 222 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 223 | ] 224 | 225 | [[package]] 226 | name = "num-traits" 227 | version = "0.1.36" 228 | source = "registry+https://github.com/rust-lang/crates.io-index" 229 | 230 | [[package]] 231 | name = "openssl-probe" 232 | version = "0.1.0" 233 | source = "registry+https://github.com/rust-lang/crates.io-index" 234 | 235 | [[package]] 236 | name = "openssl-sys" 237 | version = "0.9.1" 238 | source = "registry+https://github.com/rust-lang/crates.io-index" 239 | dependencies = [ 240 | "gdi32-sys 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", 241 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 242 | "pkg-config 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", 243 | "user32-sys 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)", 244 | ] 245 | 246 | [[package]] 247 | name = "phf" 248 | version = "0.7.19" 249 | source = "registry+https://github.com/rust-lang/crates.io-index" 250 | dependencies = [ 251 | "phf_shared 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 252 | ] 253 | 254 | [[package]] 255 | name = "phf_codegen" 256 | version = "0.7.19" 257 | source = "registry+https://github.com/rust-lang/crates.io-index" 258 | dependencies = [ 259 | "phf_generator 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 260 | "phf_shared 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 261 | ] 262 | 263 | [[package]] 264 | name = "phf_generator" 265 | version = "0.7.19" 266 | source = "registry+https://github.com/rust-lang/crates.io-index" 267 | dependencies = [ 268 | "phf_shared 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 269 | "rand 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)", 270 | ] 271 | 272 | [[package]] 273 | name = "phf_shared" 274 | version = "0.7.19" 275 | source = "registry+https://github.com/rust-lang/crates.io-index" 276 | 277 | [[package]] 278 | name = "pkg-config" 279 | version = "0.3.8" 280 | source = "registry+https://github.com/rust-lang/crates.io-index" 281 | 282 | [[package]] 283 | name = "quickersort" 284 | version = "2.1.1" 285 | source = "registry+https://github.com/rust-lang/crates.io-index" 286 | dependencies = [ 287 | "num-traits 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)", 288 | "unreachable 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 289 | ] 290 | 291 | [[package]] 292 | name = "rand" 293 | version = "0.3.14" 294 | source = "registry+https://github.com/rust-lang/crates.io-index" 295 | dependencies = [ 296 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 297 | ] 298 | 299 | [[package]] 300 | name = "regex" 301 | version = "0.1.80" 302 | source = "registry+https://github.com/rust-lang/crates.io-index" 303 | dependencies = [ 304 | "aho-corasick 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", 305 | "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", 306 | "regex-syntax 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", 307 | "thread_local 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", 308 | "utf8-ranges 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", 309 | ] 310 | 311 | [[package]] 312 | name = "regex-syntax" 313 | version = "0.3.9" 314 | source = "registry+https://github.com/rust-lang/crates.io-index" 315 | 316 | [[package]] 317 | name = "rustc-serialize" 318 | version = "0.3.21" 319 | source = "registry+https://github.com/rust-lang/crates.io-index" 320 | 321 | [[package]] 322 | name = "scraper" 323 | version = "0.4.0" 324 | source = "registry+https://github.com/rust-lang/crates.io-index" 325 | dependencies = [ 326 | "cssparser 0.5.8 (registry+https://github.com/rust-lang/crates.io-index)", 327 | "ego-tree 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", 328 | "html5ever 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)", 329 | "selectors 0.5.9 (registry+https://github.com/rust-lang/crates.io-index)", 330 | "string_cache 0.2.29 (registry+https://github.com/rust-lang/crates.io-index)", 331 | "tendril 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", 332 | ] 333 | 334 | [[package]] 335 | name = "selectors" 336 | version = "0.5.9" 337 | source = "registry+https://github.com/rust-lang/crates.io-index" 338 | dependencies = [ 339 | "bitflags 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", 340 | "cssparser 0.5.8 (registry+https://github.com/rust-lang/crates.io-index)", 341 | "fnv 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)", 342 | "matches 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 343 | "quickersort 2.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 344 | "smallvec 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)", 345 | "string_cache 0.2.29 (registry+https://github.com/rust-lang/crates.io-index)", 346 | ] 347 | 348 | [[package]] 349 | name = "serde" 350 | version = "0.8.17" 351 | source = "registry+https://github.com/rust-lang/crates.io-index" 352 | 353 | [[package]] 354 | name = "smallvec" 355 | version = "0.1.8" 356 | source = "registry+https://github.com/rust-lang/crates.io-index" 357 | 358 | [[package]] 359 | name = "string_cache" 360 | version = "0.2.29" 361 | source = "registry+https://github.com/rust-lang/crates.io-index" 362 | dependencies = [ 363 | "debug_unreachable 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 364 | "lazy_static 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", 365 | "phf_generator 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 366 | "phf_shared 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)", 367 | "serde 0.8.17 (registry+https://github.com/rust-lang/crates.io-index)", 368 | ] 369 | 370 | [[package]] 371 | name = "tendril" 372 | version = "0.2.3" 373 | source = "registry+https://github.com/rust-lang/crates.io-index" 374 | dependencies = [ 375 | "encoding 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)", 376 | "futf 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)", 377 | "mac 0.0.2 (registry+https://github.com/rust-lang/crates.io-index)", 378 | "utf-8 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)", 379 | ] 380 | 381 | [[package]] 382 | name = "thread-id" 383 | version = "2.0.0" 384 | source = "registry+https://github.com/rust-lang/crates.io-index" 385 | dependencies = [ 386 | "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", 387 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 388 | ] 389 | 390 | [[package]] 391 | name = "thread_local" 392 | version = "0.2.7" 393 | source = "registry+https://github.com/rust-lang/crates.io-index" 394 | dependencies = [ 395 | "thread-id 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)", 396 | ] 397 | 398 | [[package]] 399 | name = "time" 400 | version = "0.1.35" 401 | source = "registry+https://github.com/rust-lang/crates.io-index" 402 | dependencies = [ 403 | "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", 404 | "libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)", 405 | "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", 406 | ] 407 | 408 | [[package]] 409 | name = "unreachable" 410 | version = "0.1.1" 411 | source = "registry+https://github.com/rust-lang/crates.io-index" 412 | dependencies = [ 413 | "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", 414 | ] 415 | 416 | [[package]] 417 | name = "user32-sys" 418 | version = "0.2.0" 419 | source = "registry+https://github.com/rust-lang/crates.io-index" 420 | dependencies = [ 421 | "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", 422 | "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 423 | ] 424 | 425 | [[package]] 426 | name = "utf-8" 427 | version = "0.6.0" 428 | source = "registry+https://github.com/rust-lang/crates.io-index" 429 | dependencies = [ 430 | "matches 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)", 431 | ] 432 | 433 | [[package]] 434 | name = "utf8-ranges" 435 | version = "0.1.3" 436 | source = "registry+https://github.com/rust-lang/crates.io-index" 437 | 438 | [[package]] 439 | name = "void" 440 | version = "1.0.2" 441 | source = "registry+https://github.com/rust-lang/crates.io-index" 442 | 443 | [[package]] 444 | name = "winapi" 445 | version = "0.2.8" 446 | source = "registry+https://github.com/rust-lang/crates.io-index" 447 | 448 | [[package]] 449 | name = "winapi-build" 450 | version = "0.1.1" 451 | source = "registry+https://github.com/rust-lang/crates.io-index" 452 | 453 | [metadata] 454 | "checksum aho-corasick 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "ca972c2ea5f742bfce5687b9aef75506a764f61d37f8f649047846a9686ddb66" 455 | "checksum bitflags 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "aad18937a628ec6abcd26d1489012cc0e18c21798210f491af69ded9b881106d" 456 | "checksum cssparser 0.5.8 (registry+https://github.com/rust-lang/crates.io-index)" = "9447212c775f9e07f598fc5263fac48c7cea7a1c236646e4c9eb6070f13a85fd" 457 | "checksum curl 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "8fd5a1fdcebdb1a59578c5583e66ffed2d13850eac4f51ff730edf6dd6111eac" 458 | "checksum curl-sys 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "8e0016cc7b33b00fb7e7f6a314d8ee40748b13f377832ed9ff9e59dbb7f7ad27" 459 | "checksum debug_unreachable 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "9a032eac705ca39214d169f83e3d3da290af06d8d1d344d1baad2fd002dca4b3" 460 | "checksum ego-tree 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "40dab84b1ccdb937baa448b8a154abc691f72887de0f0abc5f4d7a2158b7978d" 461 | "checksum encoding 0.2.33 (registry+https://github.com/rust-lang/crates.io-index)" = "6b0d943856b990d12d3b55b359144ff341533e516d94098b1d3fc1ac666d36ec" 462 | "checksum encoding-index-japanese 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)" = "04e8b2ff42e9a05335dbf8b5c6f7567e5591d0d916ccef4e0b1710d32a0d0c91" 463 | "checksum encoding-index-korean 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)" = "4dc33fb8e6bcba213fe2f14275f0963fd16f0a02c878e3095ecfdf5bee529d81" 464 | "checksum encoding-index-simpchinese 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)" = "d87a7194909b9118fc707194baa434a4e3b0fb6a5a757c73c3adb07aa25031f7" 465 | "checksum encoding-index-singlebyte 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)" = "3351d5acffb224af9ca265f435b859c7c01537c0849754d3db3fdf2bfe2ae84a" 466 | "checksum encoding-index-tradchinese 1.20141219.5 (registry+https://github.com/rust-lang/crates.io-index)" = "fd0e20d5688ce3cab59eb3ef3a2083a5c77bf496cb798dc6fcdb75f323890c18" 467 | "checksum encoding_index_tests 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "a246d82be1c9d791c5dfde9a2bd045fc3cbba3fa2b11ad558f27d01712f00569" 468 | "checksum fnv 1.0.5 (registry+https://github.com/rust-lang/crates.io-index)" = "6cc484842f1e2884faf56f529f960cc12ad8c71ce96cc7abba0a067c98fee344" 469 | "checksum futf 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "e7a9689380a2553b51c564b3d9178075c68ebd0b397972c783acfd28b46c28ad" 470 | "checksum gcc 0.3.38 (registry+https://github.com/rust-lang/crates.io-index)" = "553f11439bdefe755bf366b264820f1da70f3aaf3924e594b886beb9c831bcf5" 471 | "checksum gdi32-sys 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "0912515a8ff24ba900422ecda800b52f4016a56251922d397c576bf92c690518" 472 | "checksum html5ever 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)" = "fd04d31858b8fd8ac9d55570da8c822bc6defd2c1ac18a47cb70fc280f42b432" 473 | "checksum kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7507624b29483431c0ba2d82aece8ca6cdba9382bff4ddd0f7490560c056098d" 474 | "checksum lazy_static 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "6abe0ee2e758cd6bc8a2cd56726359007748fbf4128da998b65d0b70f881e19b" 475 | "checksum libc 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)" = "044d1360593a78f5c8e5e710beccdc24ab71d1f01bc19a29bcacdba22e8475d8" 476 | "checksum libz-sys 1.0.7 (registry+https://github.com/rust-lang/crates.io-index)" = "283c2d162f78c5090522e13fc809820c33181570ae40de1bea84f3864c8759f9" 477 | "checksum log 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)" = "ab83497bf8bf4ed2a74259c1c802351fcd67a65baa86394b6ba73c36f4838054" 478 | "checksum mac 0.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1b1db08c0d0ddbb591e65f1da58d1cefccc94a2faa0c55bf979ce215a3e04d5e" 479 | "checksum matches 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "efd7622e3022e1a6eaa602c4cea8912254e5582c9c692e9167714182244801b1" 480 | "checksum memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "d8b629fb514376c675b98c1421e80b151d3817ac42d7c667717d282761418d20" 481 | "checksum num-traits 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)" = "a16a42856a256b39c6d3484f097f6713e14feacd9bfb02290917904fae46c81c" 482 | "checksum openssl-probe 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "756d49c8424483a3df3b5d735112b4da22109ced9a8294f1f5cdf80fb3810919" 483 | "checksum openssl-sys 0.9.1 (registry+https://github.com/rust-lang/crates.io-index)" = "95e9fb08acc32509fac299d6e5f4932e1e055bb70d764282c3ed8beaa87ab0e9" 484 | "checksum phf 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)" = "95cb41511b13e592110b5c8323c1d489513b6db919148f909b8b804be73a74b5" 485 | "checksum phf_codegen 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)" = "8b74506ea0ea5f6adbef815c1e964daa2d395e7c29b7196d390a67a31fa2a020" 486 | "checksum phf_generator 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)" = "d5e1d4b224dfc609b025ea389e6eb9b850ae5814272880d7d75d71acc3d57c88" 487 | "checksum phf_shared 0.7.19 (registry+https://github.com/rust-lang/crates.io-index)" = "9f3d84458c4040eb4b9e626cb551a2dc46d92ea96b1c30331aa9fce9abd2c438" 488 | "checksum pkg-config 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)" = "8cee804ecc7eaf201a4a207241472cc870e825206f6c031e3ee2a72fa425f2fa" 489 | "checksum quickersort 2.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e98eb85f2ef54d586967144db3af5990558d8f9e04f41cf2662fe258d755f349" 490 | "checksum rand 0.3.14 (registry+https://github.com/rust-lang/crates.io-index)" = "2791d88c6defac799c3f20d74f094ca33b9332612d9aef9078519c82e4fe04a5" 491 | "checksum regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)" = "4fd4ace6a8cf7860714a2c2280d6c1f7e6a413486c13298bbc86fd3da019402f" 492 | "checksum regex-syntax 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)" = "f9ec002c35e86791825ed294b50008eea9ddfc8def4420124fbc6b08db834957" 493 | "checksum rustc-serialize 0.3.21 (registry+https://github.com/rust-lang/crates.io-index)" = "bff9fc1c79f2dec76b253273d07682e94a978bd8f132ded071188122b2af9818" 494 | "checksum scraper 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4e6e16469ead09e3fd2da8aae95e5c3acc729789b59794f1f96079f3a03c2937" 495 | "checksum selectors 0.5.9 (registry+https://github.com/rust-lang/crates.io-index)" = "2d92facb7ec45ccd96ee9ceaf3cc9692d2c9094f147f373d162a5e1b4742a480" 496 | "checksum serde 0.8.17 (registry+https://github.com/rust-lang/crates.io-index)" = "784e249221c84265caeb1e2fe48aeada86f67f5acb151bd3903c4585969e43f6" 497 | "checksum smallvec 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "fcc8d19212aacecf95e4a7a2179b26f7aeb9732a915cf01f05b0d3e044865410" 498 | "checksum string_cache 0.2.29 (registry+https://github.com/rust-lang/crates.io-index)" = "f585562982abf1301fa97bd2226a3c4c5712b8beb9bcd16ed72b5e96810f8657" 499 | "checksum tendril 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "cebf864c2d90394a1b66d6fe45963f9a177f2af81a0edea5060f77627f9c4587" 500 | "checksum thread-id 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a9539db560102d1cef46b8b78ce737ff0bb64e7e18d35b2a5688f7d097d0ff03" 501 | "checksum thread_local 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)" = "8576dbbfcaef9641452d5cf0df9b0e7eeab7694956dd33bb61515fb8f18cfdd5" 502 | "checksum time 0.1.35 (registry+https://github.com/rust-lang/crates.io-index)" = "3c7ec6d62a20df54e07ab3b78b9a3932972f4b7981de295563686849eb3989af" 503 | "checksum unreachable 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "1f2ae5ddb18e1c92664717616dd9549dde73f539f01bd7b77c2edb2446bdff91" 504 | "checksum user32-sys 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "4ef4711d107b21b410a3a974b1204d9accc8b10dad75d8324b5d755de1617d47" 505 | "checksum utf-8 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a9aee9ba280438b56d1ebc5329f2094f0ff457f811eeeff0b278d75aa99db400" 506 | "checksum utf8-ranges 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "a1ca13c08c41c9c3e04224ed9ff80461d97e121589ff27c753a16cb10830ae0f" 507 | "checksum void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "6a02e4885ed3bc0f2de90ea6dd45ebcbb66dacffe03547fadbb0eeae2770887d" 508 | "checksum winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)" = "167dc9d6949a9b857f3451275e911c3f44255842c1f7a76f33c55103a909087a" 509 | "checksum winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2d315eee3b34aca4797b2da6b13ed88266e6d612562a0c46390af8299fc699bc" 510 | -------------------------------------------------------------------------------- /lwnvulns/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "lwnvulns" 3 | version = "0.1.0" 4 | authors = ["Graham Christensen "] 5 | 6 | [dependencies] 7 | regex = "0.1" 8 | lazy_static = "0.2.1" 9 | log = "0.3.6" 10 | scraper = "0.4.0" 11 | curl = "0.4.1" 12 | 13 | # Following are to make rustpackages work ... yuck! 14 | # Please try deleting these around, say, 2016-01-01 15 | serde = "=0.8.17" -------------------------------------------------------------------------------- /lwnvulns/src/bin/instructions.md: -------------------------------------------------------------------------------- 1 | # POSTING TODO (DELETE PRIOR TO POSTING) 2 | 3 | - [ ] Title it "Vulnerability Roundup " psst: find the old one via: 4 | https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+%22Vulnerability+Roundup%22 5 | - [ ] Update the last roundup link 6 | - [ ] CC everyone who participated in the previous roundup 7 | - [ ] Tag with `security` 8 | - [ ] Delete this todo text 9 | 10 | --- 11 | 12 | Here are all the vulnerabilities from https://lwn.net/Vulnerabilities 13 | since our [last roundup](). 14 | 15 | cc: . 16 | 17 | _Note:_ The list of people CC'd on this issue participated in the last 18 | roundup. If you participate on this roundup, I'll cc you on the next 19 | one. If you don't participate in the next one, you won't be CC'd on 20 | the one after that. If you would like to be CC'd on the next roundup, 21 | add a comment to the most recent vulnerability roundup. 22 | 23 | Permanent CC's: @NixOS/security-notifications, @joepie91, 24 | @phanimahesh, @the-kenny, @7c6f434c, @k0001, @peterhoeg, @nh2, @LnL7 25 | 26 | If you would like to be CC'd on _all_ roundups (or removed from the 27 | list), open a PR editing 28 | https://github.com/NixOS/security/blob/master/lwnvulns/src/bin/instructions.md. 29 | 30 | ## Notes on the list 31 | 1. The reports have been roughly grouped by the package name. This 32 | isn't perfect, but is intended to help identify if a whole group 33 | of reports is resolved already. 34 | 2. Some issues will be duplicated, because it affects multiple 35 | packages. For example, there are sometimes problems that impact 36 | thunderbird, and firefox. LWN might report in one vulnerability 37 | "thunderbird firefox". These names have been split to make sure 38 | both packages get addressed. 39 | 3. By each issue is a link to code search for the package name, and 40 | a Github search by filename. These are to help, but may not return 41 | results when we do in fact package the software. If a search 42 | doesn't turn up, please try altering the search criteria or 43 | looking in nixpkgs manually before asserting we don't have it. 44 | 4. This issue is created by https://github.com/NixOS/security 45 | 46 | # Instructions: 47 | 48 | 1. Triage a report: If we don't have the software or our version isn't 49 | vulnerable, tick the box or add a comment with the report number, 50 | stating it isn't vulnerable. 51 | 2. Fix the issue: If we do have the software and it is vulnerable, 52 | either leave a comment on this issue saying so, even open a pull 53 | request with the fix. If you open a PR, make sure to tag this 54 | issue so we can coordinate. 55 | 3. When an entire section is completed, move the section to the 56 | "Triaged and Resolved Issues" `details` block below. 57 | 58 | 59 | 60 | ## Upon Completion ... 61 | 62 | - [ ] Run the issue through `reformat` one last time 63 | - [ ] Review commits since last roundup for backport candidates 64 | - [ ] Send an update e-mail to nix-security-announce@googlegroups.com 65 | - [ ] Update the database at https://github.com/NixOS/security 66 | 67 | Without further ado... 68 | -------------------------------------------------------------------------------- /lwnvulns/src/bin/new.rs: -------------------------------------------------------------------------------- 1 | extern crate lwnvulns; 2 | extern crate scraper; 3 | extern crate curl; 4 | 5 | use lwnvulns::tokenize::{tokenize, Token, Header, Issue, SourceLink}; 6 | use lwnvulns::parse::parse; 7 | use lwnvulns::writer::write; 8 | use lwnvulns::transform::collapse_anemic; 9 | use scraper::{Html, Selector}; 10 | use curl::easy::Easy; 11 | 12 | use std::io::{Read, Write}; 13 | use std::fs::File; 14 | 15 | struct LwnRow { 16 | url: String, 17 | id: String, 18 | packages: String, 19 | description: String, 20 | } 21 | 22 | fn load_db(src: &str) -> Vec { 23 | let mut db: Vec = vec![]; 24 | { 25 | let mut f = File::open(src).unwrap(); 26 | let mut s = String::new(); 27 | f.read_to_string(&mut s).unwrap(); 28 | for entry in s.split_whitespace() { 29 | db.push(entry.to_string()); 30 | } 31 | } 32 | 33 | return db; 34 | } 35 | 36 | fn main() { 37 | let db = load_db("./db"); 38 | 39 | 40 | let mut tokens: Vec = tokenize(include_str!("./instructions.md").to_string()); 41 | 42 | 43 | let mut page = 0; 44 | let mut pages_with_nothing = 0; 45 | 46 | loop { 47 | let mut found = 0; 48 | for token in tokens_from_html(fetch_page(page)) { 49 | if let Token::Issue(ref issue) = token { 50 | if db.contains(&issue.source.id) { 51 | continue; 52 | } else { 53 | found += 1; 54 | } 55 | } 56 | tokens.push(token); 57 | } 58 | 59 | writeln!(&mut std::io::stderr(), 60 | "Page {page} yielded {found} issues, after {none} pages with nothing", 61 | page = page, 62 | found = found, 63 | none = pages_with_nothing) 64 | .unwrap(); 65 | 66 | page += 1; 67 | 68 | if found == 0 { 69 | pages_with_nothing += 1; 70 | } else { 71 | pages_with_nothing = 0; 72 | } 73 | 74 | if pages_with_nothing >= 2 { 75 | break; 76 | } 77 | 78 | if page > 10 { 79 | panic!("Not sure we should be hitting 10 pages!"); 80 | } 81 | } 82 | 83 | print!("{}", write(&collapse_anemic(parse(tokens).unwrap()))); 84 | 85 | } 86 | 87 | fn fetch_page(page: usize) -> scraper::Html { 88 | let url = format!("https://lwn.net/Vulnerabilities/?n={}&offset={}", 89 | 100, 90 | page * 100); 91 | let mut data = Vec::new(); 92 | 93 | let mut easy = Easy::new(); 94 | easy.url(&url).unwrap(); 95 | 96 | { 97 | let mut transfer = easy.transfer(); 98 | transfer.write_function(|chunk| { 99 | data.extend_from_slice(chunk); 100 | Ok(chunk.len()) 101 | }) 102 | .unwrap(); 103 | transfer.perform().unwrap(); 104 | } 105 | 106 | return Html::parse_document(&String::from_utf8(data).unwrap()); 107 | } 108 | 109 | fn tokens_from_html(html: scraper::Html) -> Vec { 110 | let mut tokens: Vec = vec![]; 111 | 112 | let rows = Selector::parse("tr").unwrap(); 113 | let links = Selector::parse("td:nth-child(1) a").unwrap(); 114 | let packages = Selector::parse("td:nth-child(3)").unwrap(); 115 | let descriptions = Selector::parse("td:nth-child(4)").unwrap(); 116 | for row in html.select(&rows) { 117 | if let Some(link) = row.select(&links).next() { 118 | let package = extract_cell_text(row.select(&packages)) 119 | .unwrap_or("unknown package".to_string()); 120 | let description = extract_cell_text(row.select(&descriptions)) 121 | .unwrap_or(package.clone()); // sometimes the desc is missing 122 | 123 | let lwnrow = LwnRow { 124 | url: link.value().attr("href").unwrap().to_string(), 125 | id: link.inner_html(), 126 | packages: package, 127 | description: description, 128 | }; 129 | for token in row_to_tokens(lwnrow) { 130 | tokens.push(token); 131 | } 132 | } 133 | } 134 | 135 | return tokens; 136 | } 137 | 138 | fn extract_cell_text(mut cell: scraper::element_ref::Select) -> Option { 139 | if let Some(td_elem) = cell.next() { 140 | let mut text_iterator = td_elem.text(); 141 | if let Some(description) = text_iterator.next() { 142 | return Some(description.to_string()); 143 | } 144 | } 145 | 146 | return None; 147 | } 148 | 149 | fn row_to_tokens(row: LwnRow) -> Vec { 150 | let mut tokens = vec![]; 151 | 152 | let url = format!("https://lwn.net{}", row.url); 153 | for pkg in split_packages(row.packages) { 154 | tokens.push(Token::Header(Header { 155 | issue_count: 0, 156 | notes: None, 157 | package: pkg.clone(), 158 | })); 159 | 160 | tokens.push(Token::Issue(Issue { 161 | complete: false, 162 | source: SourceLink { 163 | id: row.id.clone(), 164 | url: url.clone(), 165 | }, 166 | content: format!( 167 | "([search]({search_url}), [files]({files_url})) {description}", 168 | search_url = format!( 169 | "http://search.nix.gsc.io/?q={package}&i=fosho&repos=nixos-nixpkgs", 170 | package=pkg 171 | ), 172 | files_url = format!( 173 | "{root}?utf8=%E2%9C%93&q={package}+in%3Apath&type=Code", 174 | root="https://github.com/NixOS/nixpkgs/search", 175 | package=pkg 176 | ), 177 | description=row.description 178 | ), 179 | })); 180 | } 181 | 182 | return tokens; 183 | } 184 | 185 | fn split_packages(packages: String) -> Vec { 186 | let pkgs = packages.split_whitespace() 187 | .map(|s| s.replace(",", "").to_string()) 188 | .map(|s| if s == "tiff" { 189 | "libtiff".to_string() 190 | } else if s == "libgc" { 191 | "boehm-gc".to_string() 192 | } else if s == "perl-DBD-MySQL" { 193 | "DBD-mysql".to_string() 194 | } else if s == "httpd" { 195 | "apache-httpd".to_string() 196 | } else { 197 | s 198 | }) 199 | .collect(); 200 | 201 | return pkgs; 202 | } 203 | -------------------------------------------------------------------------------- /lwnvulns/src/bin/reformat.rs: -------------------------------------------------------------------------------- 1 | extern crate lwnvulns; 2 | 3 | use std::io::stdin; 4 | use std::io::Read; 5 | 6 | fn main() { 7 | let mut buf = String::new(); 8 | let stdsrc = stdin(); 9 | let mut handle = stdsrc.lock(); 10 | handle.read_to_string(&mut buf).unwrap(); 11 | 12 | let tokens = lwnvulns::tokenize::tokenize(buf); 13 | let ast = lwnvulns::parse::parse(tokens); 14 | 15 | let (todo, done) = lwnvulns::transform::partition_completed(ast.unwrap()); 16 | 17 | if todo.count_issues() == 0 { 18 | println!(" 19 | {todo_doc} 20 | {done_doc} 21 | ", 22 | todo_doc = lwnvulns::writer::write(&todo), 23 | done_doc = lwnvulns::writer::write(&done)); 24 | } else if done.count_issues() == 0 { 25 | println!(" 26 | {todo_doc} 27 | {done_doc} 28 | ", 29 | todo_doc = lwnvulns::writer::write(&todo), 30 | done_doc = lwnvulns::writer::write(&done)); 31 | } else { 32 | println!(" 33 | {todo_doc} 34 | 35 | --- 36 | ## {todo_count} remaining, {done_count} completed 37 | 38 |
39 | CLICK TO EXPAND Show all {done_count} triaged and resolved issues 40 | 41 | 42 | {done_doc} 43 | 44 |
45 | 46 | ", 47 | todo_doc=lwnvulns::writer::write(&todo), 48 | done_doc=lwnvulns::writer::write(&done), 49 | todo_count=todo.count_issues(), 50 | done_count=done.count_issues(), 51 | ); 52 | } 53 | } 54 | -------------------------------------------------------------------------------- /lwnvulns/src/bin/updatedb.rs: -------------------------------------------------------------------------------- 1 | extern crate lwnvulns; 2 | 3 | use std::io::stdin; 4 | use std::io::Read; 5 | 6 | fn main() { 7 | let mut buf = String::new(); 8 | let stdsrc = stdin(); 9 | let mut handle = stdsrc.lock(); 10 | handle.read_to_string(&mut buf).unwrap(); 11 | 12 | let tokens = lwnvulns::tokenize::tokenize(buf); 13 | let ast = lwnvulns::parse::parse(tokens); 14 | 15 | let (_, done) = lwnvulns::transform::partition_completed(ast.unwrap()); 16 | 17 | for (_, section) in done.report { 18 | for issue in section.children { 19 | print!("{}\n", issue.source.id); 20 | } 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /lwnvulns/src/lib.rs: -------------------------------------------------------------------------------- 1 | #[macro_use] 2 | extern crate lazy_static; 3 | #[macro_use] 4 | extern crate log; 5 | 6 | extern crate regex; 7 | 8 | pub mod tokenize; 9 | pub mod parse; 10 | pub mod transform; 11 | pub mod writer; 12 | 13 | pub mod lwnvulns { 14 | pub use tokenize; 15 | pub use parse; 16 | pub use transform; 17 | pub use writer; 18 | } 19 | -------------------------------------------------------------------------------- /lwnvulns/src/parse.rs: -------------------------------------------------------------------------------- 1 | use tokenize::{Token, Header, Issue}; 2 | use std::collections::HashMap; 3 | use std::hash::{Hash, Hasher}; 4 | 5 | 6 | #[derive(Debug,PartialEq)] 7 | pub struct Document { 8 | pub preamble: Vec, 9 | pub report: HashMap, 10 | } 11 | 12 | #[derive(Debug,PartialEq)] 13 | pub struct Section { 14 | pub header: Header, 15 | pub children: Vec, 16 | } 17 | 18 | impl Hash for Section { 19 | fn hash(&self, state: &mut H) { 20 | self.header.hash(state); 21 | } 22 | } 23 | 24 | impl Document { 25 | pub fn count_issues(&self) -> usize { 26 | let mut sum: usize = 0; 27 | 28 | for value in self.report.values() { 29 | sum += value.children.len(); 30 | } 31 | 32 | return sum; 33 | } 34 | } 35 | 36 | pub fn parse(mut input: Vec) -> Result { 37 | let mut doc = Document { 38 | preamble: vec![], 39 | report: HashMap::new(), 40 | }; 41 | 42 | input.reverse(); 43 | 44 | let mut token = input.pop(); 45 | while let Some(Token::Preamble(_)) = token { 46 | doc.preamble.push(token.unwrap()); 47 | token = input.pop(); 48 | } 49 | 50 | while let Some(Token::Header(mut header)) = token { 51 | header.issue_count = 0; 52 | let mut section = doc.report.entry(header.package.to_string()).or_insert(Section { 53 | header: header, 54 | children: vec![], 55 | }); 56 | 57 | token = input.pop(); 58 | 59 | loop { 60 | match token { 61 | Some(Token::Issue(issue)) => { 62 | section.children.push(issue); 63 | section.header.issue_count += 1; 64 | token = input.pop(); 65 | } 66 | _ => { 67 | break; 68 | } 69 | } 70 | } 71 | } 72 | 73 | return Ok(doc); 74 | } 75 | 76 | #[cfg(test)] 77 | mod tests { 78 | use super::*; 79 | use tokenize::{Token, Header, Issue, SourceLink}; 80 | use std::collections::HashMap; 81 | 82 | #[test] 83 | fn test_parse() { 84 | let input = vec![Token::Preamble("this".to_string()), 85 | Token::Preamble("is".to_string()), 86 | Token::Preamble("garbage".to_string()), 87 | Token::Preamble("### Here too! ( :) )".to_string()), 88 | Token::Header(Header { 89 | package: "This is a header".to_string(), 90 | issue_count: 0, 91 | notes: None, 92 | }), 93 | Token::Header(Header { 94 | package: "This is a header too".to_string(), 95 | issue_count: 1, 96 | notes: Some(" lol".to_string()), 97 | }), 98 | Token::Issue(Issue { 99 | complete: false, 100 | content: "not done".to_string(), 101 | source: SourceLink { 102 | id: "1".to_string(), 103 | url: "http://1".to_string(), 104 | }, 105 | }), 106 | Token::Issue(Issue { 107 | complete: true, 108 | content: "done".to_string(), 109 | source: SourceLink { 110 | id: "2".to_string(), 111 | url: "http://2".to_string(), 112 | }, 113 | })]; 114 | 115 | let mut expect = Document { 116 | preamble: vec![Token::Preamble("this".to_string()), 117 | Token::Preamble("is".to_string()), 118 | Token::Preamble("garbage".to_string()), 119 | Token::Preamble("### Here too! ( :) )".to_string())], 120 | report: HashMap::new(), 121 | }; 122 | 123 | expect.report.insert("This is a header".to_string(), 124 | Section { 125 | header: Header { 126 | package: "This is a header".to_string(), 127 | issue_count: 0, 128 | notes: None, 129 | }, 130 | children: vec![], 131 | }); 132 | 133 | expect.report.insert("This is a header too".to_string(), 134 | Section { 135 | header: Header { 136 | package: "This is a header too".to_string(), 137 | issue_count: 2, 138 | notes: Some(" lol".to_string()), 139 | }, 140 | children: vec![Issue { 141 | complete: false, 142 | content: "not done".to_string(), 143 | source: SourceLink { 144 | id: "1".to_string(), 145 | url: "http://1".to_string(), 146 | }, 147 | }, 148 | Issue { 149 | complete: true, 150 | content: "done".to_string(), 151 | source: SourceLink { 152 | id: "2".to_string(), 153 | url: "http://2".to_string(), 154 | }, 155 | }], 156 | }); 157 | 158 | assert_eq!(parse(input), Ok(expect)); 159 | 160 | } 161 | 162 | #[test] 163 | fn test_parse_duplicated_headers() { 164 | let input = vec![Token::Header(Header { 165 | package: "PackageA".to_string(), 166 | issue_count: 1, 167 | notes: Some("lol".to_string()), 168 | }), 169 | Token::Issue(Issue { 170 | complete: false, 171 | content: "not done".to_string(), 172 | source: SourceLink { 173 | id: "3".to_string(), 174 | url: "http://3".to_string(), 175 | }, 176 | }), 177 | Token::Header(Header { 178 | package: "PackageA".to_string(), 179 | issue_count: 1, 180 | notes: None, 181 | }), 182 | Token::Issue(Issue { 183 | complete: true, 184 | content: "done".to_string(), 185 | source: SourceLink { 186 | id: "4".to_string(), 187 | url: "http://4".to_string(), 188 | }, 189 | })]; 190 | 191 | let mut expect = Document { 192 | preamble: vec![], 193 | report: HashMap::new(), 194 | }; 195 | expect.report.insert("PackageA".to_string(), 196 | Section { 197 | header: Header { 198 | package: "PackageA".to_string(), 199 | issue_count: 2, 200 | notes: Some("lol".to_string()), 201 | }, 202 | children: vec![Issue { 203 | complete: false, 204 | content: "not done".to_string(), 205 | source: SourceLink { 206 | id: "3".to_string(), 207 | url: "http://3".to_string(), 208 | }, 209 | }, 210 | Issue { 211 | complete: true, 212 | content: "done".to_string(), 213 | source: SourceLink { 214 | id: "4".to_string(), 215 | url: "http://4".to_string(), 216 | }, 217 | }], 218 | }); 219 | 220 | assert_eq!(parse(input), Ok(expect)); 221 | 222 | } 223 | 224 | 225 | #[test] 226 | fn test_count_issues() { 227 | let mut doc = Document { 228 | preamble: vec![], 229 | report: HashMap::new(), 230 | }; 231 | doc.report.insert("PackageA".to_string(), 232 | Section { 233 | header: Header { 234 | package: "PackageA".to_string(), 235 | issue_count: 2, 236 | notes: Some("lol".to_string()), 237 | }, 238 | children: vec![Issue { 239 | complete: false, 240 | content: "not done".to_string(), 241 | source: SourceLink { 242 | id: "5".to_string(), 243 | url: "http://5".to_string(), 244 | }, 245 | }, 246 | Issue { 247 | complete: true, 248 | content: "done".to_string(), 249 | source: SourceLink { 250 | id: "6".to_string(), 251 | url: "http://6".to_string(), 252 | }, 253 | }], 254 | }); 255 | 256 | doc.report.insert("PackageB".to_string(), 257 | Section { 258 | header: Header { 259 | package: "PackageB".to_string(), 260 | issue_count: 2, 261 | notes: Some("lol".to_string()), 262 | }, 263 | children: vec![Issue { 264 | complete: false, 265 | content: "not done".to_string(), 266 | source: SourceLink { 267 | id: "7".to_string(), 268 | url: "http://7".to_string(), 269 | }, 270 | }, 271 | Issue { 272 | complete: true, 273 | content: "done".to_string(), 274 | source: SourceLink { 275 | id: "8".to_string(), 276 | url: "http://8".to_string(), 277 | }, 278 | }], 279 | }); 280 | 281 | 282 | assert_eq!(doc.count_issues(), 4) 283 | } 284 | } 285 | -------------------------------------------------------------------------------- /lwnvulns/src/tokenize.rs: -------------------------------------------------------------------------------- 1 | use regex::Regex; 2 | use std::hash::{Hash, Hasher}; 3 | 4 | #[derive(Debug,PartialEq)] 5 | pub enum Token { 6 | Preamble(String), 7 | Header(Header), 8 | Issue(Issue), 9 | } 10 | 11 | #[derive(Debug,PartialEq,Clone)] 12 | pub struct Header { 13 | pub package: String, 14 | pub issue_count: i32, 15 | pub notes: Option, 16 | } 17 | 18 | #[derive(Debug,PartialEq)] 19 | pub struct Issue { 20 | pub complete: bool, 21 | pub source: SourceLink, 22 | pub content: String, 23 | } 24 | 25 | #[derive(Debug,PartialEq)] 26 | pub struct SourceLink { 27 | pub url: String, 28 | pub id: String, 29 | } 30 | 31 | impl Hash for Header { 32 | fn hash(&self, state: &mut H) { 33 | self.package.hash(state); 34 | } 35 | } 36 | 37 | impl Header { 38 | pub fn to_string(&self) -> String { 39 | if let Some(_) = self.notes { 40 | if self.issue_count == 1 { 41 | return format!("### {} ({} issue) {}", 42 | self.package, 43 | self.issue_count, 44 | self.notes.clone().unwrap().clone()) 45 | .to_string(); 46 | } else { 47 | return format!("### {} ({} issues) {}", 48 | self.package, 49 | self.issue_count, 50 | self.notes.clone().unwrap().clone()) 51 | .to_string(); 52 | } 53 | } else { 54 | if self.issue_count == 1 { 55 | return format!("### {} ({} issue)", self.package, self.issue_count).to_string(); 56 | } else { 57 | return format!("### {} ({} issues)", self.package, self.issue_count).to_string(); 58 | } 59 | } 60 | } 61 | } 62 | 63 | impl Issue { 64 | pub fn to_string(&self) -> String { 65 | if self.complete { 66 | format!(" - [x] [`#{}`]({}) {}", 67 | self.source.id, 68 | self.source.url, 69 | self.content) 70 | .to_string() 71 | } else { 72 | format!(" - [ ] [`#{}`]({}) {}", 73 | self.source.id, 74 | self.source.url, 75 | self.content) 76 | .to_string() 77 | } 78 | } 79 | } 80 | 81 | impl Token { 82 | pub fn to_string(&self) -> String { 83 | match *self { 84 | Token::Preamble(ref res) => res.to_string(), 85 | Token::Header(ref header) => header.to_string(), 86 | Token::Issue(ref issue) => issue.to_string(), 87 | } 88 | } 89 | } 90 | 91 | fn parse_header(line: String) -> Option { 92 | lazy_static! { 93 | static ref RE: Regex = Regex::new( 94 | r"^### (.*) \((\d+) issues?\) ?(.+)?$" 95 | ).unwrap(); 96 | } 97 | if let Some(capture) = RE.captures(&line) { 98 | if let Some(package) = capture.at(1) { 99 | if let Some(count) = capture.at(2) { 100 | if let Ok(count) = count.parse::() { 101 | let mut notes: Option = None; 102 | 103 | if let Some(found_notes) = capture.at(3) { 104 | notes = Some(found_notes.to_string()) 105 | } 106 | 107 | return Some(Token::Header(Header { 108 | package: package.to_string(), 109 | issue_count: count, 110 | notes: notes, 111 | })); 112 | } 113 | } 114 | } 115 | } 116 | 117 | return None; 118 | } 119 | 120 | fn parse_issue(line: String) -> Option { 121 | lazy_static! { 122 | static ref RE: Regex = Regex::new( 123 | r"^ ?- \[(x| )\] \[`#(.*)`\]\((.*)\) (.*)$" 124 | ).unwrap(); 125 | } 126 | if let Some(capture) = RE.captures(&line) { 127 | if let Some(complete) = capture.at(1) { 128 | let complete = complete == "x"; 129 | return Some(Token::Issue(Issue { 130 | complete: complete, 131 | source: SourceLink { 132 | id: capture.at(2).unwrap().to_string(), 133 | url: capture.at(3).unwrap().to_string(), 134 | }, 135 | content: capture.at(4).unwrap().to_string(), 136 | })); 137 | } 138 | } 139 | 140 | return None; 141 | } 142 | 143 | pub fn tokenize(inputs: String) -> Vec { 144 | let mut ret: Vec = vec![]; 145 | let mut in_preamble: bool = true; 146 | 147 | for line in inputs.lines() { 148 | if let Some(header) = parse_header(line.to_string()) { 149 | in_preamble = false; 150 | ret.push(header) 151 | } else if in_preamble { 152 | ret.push(Token::Preamble(line.to_string())); 153 | } else if let Some(issue) = parse_issue(line.to_string()) { 154 | ret.push(issue) 155 | } else { 156 | 157 | } 158 | 159 | } 160 | return ret; 161 | } 162 | 163 | #[test] 164 | fn test_tokenize() { 165 | let input = "this 166 | is 167 | garbage 168 | ### Here too! ( :) ) 169 | ### This is a header (0 issues) 170 | ### This is a header too (1 issue) lol 171 | garbage1 172 | - [ ] [`#123456`](http://foo...) not done 173 | garbage2 174 | - [x] [`#7890`](http://bar...) done 175 | garbage3 176 | "; 177 | let expect = vec![Token::Preamble("this".to_string()), 178 | Token::Preamble("is".to_string()), 179 | Token::Preamble("garbage".to_string()), 180 | Token::Preamble("### Here too! ( :) )".to_string()), 181 | Token::Header(Header { 182 | package: "This is a header".to_string(), 183 | issue_count: 0, 184 | notes: None, 185 | }), 186 | Token::Header(Header { 187 | package: "This is a header too".to_string(), 188 | issue_count: 1, 189 | notes: Some("lol".to_string()), 190 | }), 191 | Token::Issue(Issue { 192 | complete: false, 193 | source: SourceLink { 194 | url: "http://foo...".to_string(), 195 | id: "123456".to_string(), 196 | }, 197 | content: "not done".to_string(), 198 | }), 199 | Token::Issue(Issue { 200 | complete: true, 201 | source: SourceLink { 202 | url: "http://bar...".to_string(), 203 | id: "7890".to_string(), 204 | }, 205 | content: "done".to_string(), 206 | })]; 207 | 208 | assert_eq!(tokenize(input.to_string()), expect); 209 | } 210 | -------------------------------------------------------------------------------- /lwnvulns/src/transform.rs: -------------------------------------------------------------------------------- 1 | use std::collections::HashMap; 2 | use parse::{Document, Section}; 3 | use tokenize::Header; 4 | 5 | pub fn partition_completed(src: Document) -> (Document, Document) { 6 | let mut todo = Document { 7 | preamble: src.preamble, 8 | report: HashMap::new(), 9 | }; 10 | let mut done = Document { 11 | preamble: vec![], 12 | report: HashMap::new(), 13 | }; 14 | 15 | for (package, section) in src.report { 16 | let mut header = section.header.clone(); 17 | header.issue_count = 0; 18 | 19 | for issue in section.children { 20 | let ref mut doc = if issue.complete { &mut done } else { &mut todo }; 21 | let mut section = doc.report.entry(package.clone()).or_insert(Section { 22 | header: header.clone(), 23 | children: vec![], 24 | }); 25 | 26 | section.children.push(issue); 27 | section.header.issue_count += 1; 28 | } 29 | } 30 | 31 | return (todo, done); 32 | } 33 | 34 | 35 | pub fn collapse_anemic(mut src: Document) -> Document { 36 | let mut collapsed = Document { 37 | preamble: src.preamble, 38 | report: HashMap::new(), 39 | }; 40 | 41 | let mut reports: Vec = src.report.keys().map(|key| key.clone()).collect(); 42 | reports.sort(); 43 | for key in reports { 44 | let section = src.report.remove(&key).unwrap(); 45 | 46 | if section.children.len() > 1 { 47 | collapsed.report.insert(section.header.package.to_string().clone(), section); 48 | } else { 49 | let mut destination = 50 | collapsed.report.entry("Assorted".to_string()).or_insert(Section { 51 | header: Header { 52 | package: "Assorted".to_string(), 53 | issue_count: 0, 54 | notes: None, 55 | }, 56 | children: vec![], 57 | }); 58 | 59 | for issue in section.children { 60 | destination.children.push(issue); 61 | destination.header.issue_count += 1; 62 | } 63 | } 64 | 65 | 66 | } 67 | 68 | return collapsed; 69 | } 70 | 71 | 72 | #[cfg(test)] 73 | mod tests { 74 | use std::collections::HashMap; 75 | use parse::{Document, Section}; 76 | use tokenize::{Token, Header, Issue, SourceLink}; 77 | use super::*; 78 | 79 | #[test] 80 | fn test_lwnvulns_partition() { 81 | let mut input = Document { 82 | preamble: vec![Token::Preamble("this".to_string()), 83 | Token::Preamble("is".to_string()), 84 | Token::Preamble("garbage".to_string()), 85 | Token::Preamble("### Here too! ( :) )".to_string())], 86 | report: HashMap::new(), 87 | }; 88 | 89 | input.report.insert("This is a header too".to_string(), 90 | Section { 91 | header: Header { 92 | package: "This is a header too".to_string(), 93 | issue_count: 2, 94 | notes: Some(" lol".to_string()), 95 | }, 96 | children: vec![Issue { 97 | complete: false, 98 | content: "not done".to_string(), 99 | source: SourceLink { 100 | id: "not done".to_string(), 101 | url: "http://not done".to_string(), 102 | }, 103 | }, 104 | Issue { 105 | complete: true, 106 | content: "done".to_string(), 107 | source: SourceLink { 108 | id: "done".to_string(), 109 | url: "http://done".to_string(), 110 | }, 111 | }], 112 | }); 113 | 114 | let mut expect_todo = Document { 115 | preamble: vec![Token::Preamble("this".to_string()), 116 | Token::Preamble("is".to_string()), 117 | Token::Preamble("garbage".to_string()), 118 | Token::Preamble("### Here too! ( :) )".to_string())], 119 | report: HashMap::new(), 120 | }; 121 | 122 | expect_todo.report.insert("This is a header too".to_string(), 123 | Section { 124 | header: Header { 125 | package: "This is a header too".to_string(), 126 | issue_count: 1, 127 | notes: Some(" lol".to_string()), 128 | }, 129 | children: vec![Issue { 130 | complete: false, 131 | content: "not done".to_string(), 132 | source: SourceLink { 133 | id: "not done".to_string(), 134 | url: "http://not done".to_string(), 135 | }, 136 | }], 137 | }); 138 | 139 | let mut expect_done = Document { 140 | preamble: vec![], 141 | report: HashMap::new(), 142 | }; 143 | 144 | expect_done.report.insert("This is a header too".to_string(), 145 | Section { 146 | header: Header { 147 | package: "This is a header too".to_string(), 148 | issue_count: 1, 149 | notes: Some(" lol".to_string()), 150 | }, 151 | children: vec![Issue { 152 | complete: true, 153 | content: "done".to_string(), 154 | source: SourceLink { 155 | id: "done".to_string(), 156 | url: "http://done".to_string(), 157 | }, 158 | }], 159 | }); 160 | 161 | 162 | 163 | let (todo, done) = partition_completed(input); 164 | assert_eq!(todo, expect_todo); 165 | assert_eq!(done, expect_done); 166 | 167 | } 168 | 169 | #[test] 170 | fn test_collapse_anemic() { 171 | let mut input = Document { 172 | preamble: vec![Token::Preamble("this".to_string()), 173 | Token::Preamble("is".to_string()), 174 | Token::Preamble("garbage".to_string()), 175 | Token::Preamble("### Here too! ( :) )".to_string())], 176 | report: HashMap::new(), 177 | }; 178 | 179 | input.report.insert("This is a header".to_string(), 180 | Section { 181 | header: Header { 182 | package: "This is a header".to_string(), 183 | issue_count: 1, 184 | notes: Some(" lol".to_string()), 185 | }, 186 | children: vec![Issue { 187 | complete: false, 188 | content: "not done".to_string(), 189 | source: SourceLink { 190 | id: "not done".to_string(), 191 | url: "http://not done".to_string(), 192 | }, 193 | }], 194 | }); 195 | 196 | input.report.insert("This is a header too".to_string(), 197 | Section { 198 | header: Header { 199 | package: "This is a header too".to_string(), 200 | issue_count: 1, 201 | notes: Some(" lol".to_string()), 202 | }, 203 | children: vec![Issue { 204 | complete: true, 205 | content: "done".to_string(), 206 | source: SourceLink { 207 | id: "done".to_string(), 208 | url: "http://done".to_string(), 209 | }, 210 | }], 211 | }); 212 | 213 | 214 | let mut expect = Document { 215 | preamble: vec![Token::Preamble("this".to_string()), 216 | Token::Preamble("is".to_string()), 217 | Token::Preamble("garbage".to_string()), 218 | Token::Preamble("### Here too! ( :) )".to_string())], 219 | report: HashMap::new(), 220 | }; 221 | 222 | expect.report.insert("Assorted".to_string(), 223 | Section { 224 | header: Header { 225 | package: "Assorted".to_string(), 226 | issue_count: 2, 227 | notes: None, 228 | }, 229 | children: vec![Issue { 230 | complete: false, 231 | content: "not done".to_string(), 232 | source: SourceLink { 233 | id: "not done".to_string(), 234 | url: "http://not done".to_string(), 235 | }, 236 | }, 237 | Issue { 238 | complete: true, 239 | content: "done".to_string(), 240 | source: SourceLink { 241 | id: "done".to_string(), 242 | url: "http://done".to_string(), 243 | }, 244 | }], 245 | }); 246 | 247 | 248 | let collapsed = collapse_anemic(input); 249 | assert_eq!(expect, collapsed); 250 | 251 | } 252 | 253 | } 254 | -------------------------------------------------------------------------------- /lwnvulns/src/writer.rs: -------------------------------------------------------------------------------- 1 | 2 | 3 | use parse::Document; 4 | use tokenize::Token; 5 | 6 | 7 | pub fn write(input: &Document) -> String { 8 | let mut ret = "".to_string(); 9 | for token in &input.preamble { 10 | if let &Token::Preamble(ref line) = token { 11 | ret = ret + &line + "\n"; 12 | } else { 13 | panic!("lol no other tokens here?"); 14 | } 15 | } 16 | 17 | let mut reports: Vec = input.report.keys().map(|key| key.clone()).collect(); 18 | reports.sort(); 19 | for key in reports { 20 | let section = input.report.get(&key).unwrap(); 21 | let mut header = section.header.clone(); 22 | header.issue_count = section.children.len() as i32; 23 | 24 | ret = ret + &header.to_string() + "\n"; 25 | 26 | for issue in §ion.children { 27 | ret = ret + &issue.to_string() + "\n"; 28 | } 29 | 30 | ret = ret + "\n"; 31 | } 32 | 33 | 34 | return ret; 35 | } 36 | 37 | #[cfg(test)] 38 | mod tests { 39 | use parse::{Document, Section}; 40 | use tokenize::{Token, Header, Issue, SourceLink}; 41 | use std::collections::HashMap; 42 | 43 | use super::*; 44 | 45 | #[test] 46 | fn test_exports_back() { 47 | let mut input = Document { 48 | preamble: vec![Token::Preamble("this".to_string()), 49 | Token::Preamble("is".to_string()), 50 | Token::Preamble("preamble".to_string()), 51 | Token::Preamble("### Here too! ( :) )".to_string())], 52 | report: HashMap::new(), 53 | }; 54 | input.report.insert("B This is a header too".to_string(), 55 | Section { 56 | header: Header { 57 | package: "B This is a header too".to_string(), 58 | issue_count: 2, 59 | notes: Some("lol".to_string()), 60 | }, 61 | children: vec![Issue { 62 | complete: false, 63 | content: "not done".to_string(), 64 | source: SourceLink { 65 | id: "not done".to_string(), 66 | url: "http://not done".to_string(), 67 | }, 68 | }, 69 | Issue { 70 | complete: true, 71 | content: "done".to_string(), 72 | source: SourceLink { 73 | id: "done".to_string(), 74 | url: "http://done".to_string(), 75 | }, 76 | }], 77 | }); 78 | 79 | input.report.insert("A This is a header".to_string(), 80 | Section { 81 | header: Header { 82 | package: "A This is a header".to_string(), 83 | issue_count: 0, 84 | notes: None, 85 | }, 86 | children: vec![], 87 | }); 88 | 89 | let expect = "this 90 | is 91 | preamble 92 | ### Here too! ( :) ) 93 | ### A This is a header (0 issues) 94 | 95 | ### B This is a header too (2 issues) lol 96 | - [ ] [`#not done`](http://not done) not done 97 | - [x] [`#done`](http://done) done 98 | 99 | "; 100 | 101 | assert_eq!(write(&input), expect); 102 | 103 | } 104 | } 105 | -------------------------------------------------------------------------------- /notate.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | #!/usr/bin/env nix-shell 3 | #!nix-shell -i bash -p git 4 | 5 | set -eu 6 | set -o pipefail 7 | 8 | # Disable Ctrl-C since it can be very frustrating to lose progress 9 | trap '' 2 10 | 11 | readonly height=$(stty -a | grep rows | cut -d";" -f2 | cut -d' ' -f3) 12 | readonly DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 13 | 14 | commit_to_file() { 15 | prefix=$(echo "$1" | head -c2); 16 | suffix=$(echo "$1" | tail -c+3); 17 | mkdir -p "$prefix"; 18 | echo "$prefix/$suffix"; 19 | } 20 | 21 | extract_cherrypick() { 22 | git show "$1" -- \ 23 | | grep "(cherry picked from " \ 24 | | sed -e "s/.*commit //" -e "s/)//" 25 | } 26 | 27 | mark_commit_ui() { 28 | sha="$1" 29 | sha_file=$(commit_to_file "$sha") 30 | clear 31 | set +e 32 | git show --notes=security --color=always "$sha" -- | head -n $((height - 8)); 33 | echo "..." 34 | 35 | local picked_from; 36 | picked_from=$(extract_cherrypick "$sha") 37 | if [ "x$picked_from" != "x" ]; then 38 | picked_from_file=$(commit_to_file "$picked_from") 39 | if test -f "./$picked_from_file"; then 40 | echo "Found these notes from $picked_from:" 41 | cat "$picked_from_file" 42 | echo "" 43 | echo "Want to use them? [Y/n]: " 44 | 45 | read -r x; 46 | if [ "x$x" != "xn" ]; then 47 | cp "./$picked_from_file" "./$sha_file" 48 | git add "./$sha_file" 49 | fi 50 | fi 51 | fi 52 | 53 | echo -n "Does this need security notes or editing? [quit, y/N, kernel, browser]: "; 54 | read -r x; 55 | EDITED=0 56 | if [ "x$x" = "xy" ]; then 57 | $EDITOR "$sha_file" 58 | EDITED=1 59 | elif [ "x$x" = "xbrowser" ]; then 60 | echo "All browser patches are considered security-sensitive." >> "$sha_file" 61 | EDITED=1 62 | elif [ "x$x" = "xkernel" ]; then 63 | echo "All kernel patches are considered security-sensitive." >> "$sha_file" 64 | EDITED=1 65 | elif [ "x$x" = "xquit" ]; then 66 | exit 1 67 | fi 68 | 69 | if [ $EDITED -eq 1 ]; then 70 | git add "./$sha_file" 71 | 72 | if [ "x$picked_from" != "x" ]; then 73 | picked_from_file=$(commit_to_file "$picked_from") 74 | if ! test -f "./$picked_from_file"; then 75 | echo "This commit was cherry-picked from $picked_from." 76 | echo "Backport your notes? [Y/n]:" 77 | 78 | read -r x; 79 | if [ "x$x" != "xn" ]; then 80 | cp "./$sha_file" "./$picked_from_file" 81 | git add "./$picked_from_file" 82 | fi 83 | fi 84 | fi 85 | fi 86 | set -e 87 | } 88 | 89 | git fetch origin 90 | git fetch origin refs/notes/security:refs/notes/security 91 | git checkout refs/notes/security 92 | cleanup_basic() { 93 | git checkout - 94 | } 95 | trap cleanup_basic EXIT 96 | 97 | prompt_next() { 98 | echo "Processing" "$@" 99 | echo "Press enter to continue, or quit to exit." 100 | read -r x 101 | if [ "x$x" = "xquit" ]; then 102 | exit 1 103 | fi 104 | } 105 | 106 | if [ "x${1:-}" != "x" ]; then 107 | mark_commit_ui "$1" 108 | else 109 | . "$DIR/state/notate_state.sh" 110 | 111 | next_MASTER_SEEN=$MASTER_SEEN 112 | next_RELEASE_16_09_SEEN=$RELEASE_16_09_SEEN 113 | next_RELEASE_17_03_SEEN=$RELEASE_17_03_SEEN 114 | cleanup() { 115 | echo "MASTER_SEEN=$next_MASTER_SEEN" > "$DIR/state/notate_state.sh" 116 | echo "RELEASE_16_09_SEEN=$next_RELEASE_16_09_SEEN" >> "$DIR/state/notate_state.sh" 117 | echo "RELEASE_17_03_SEEN=$next_RELEASE_17_03_SEEN" >> "$DIR/state/notate_state.sh" 118 | 119 | echo "Going to commit these changes now" 120 | git commit 121 | git update-ref refs/notes/security "$(git rev-parse HEAD)" 122 | git push origin refs/notes/security:refs/notes/security 123 | 124 | cleanup_basic 125 | } 126 | trap cleanup EXIT 127 | 128 | prompt_next "$MASTER_SEEN...origin/master" 129 | for sha in $(git rev-list --reverse --no-merges "$MASTER_SEEN...origin/master"); do 130 | mark_commit_ui "$sha" 131 | next_MASTER_SEEN="$sha" 132 | done 133 | 134 | prompt_next "$RELEASE_17_03_SEEN...origin/release-17.03" 135 | for sha in $(git rev-list --reverse --no-merges "$RELEASE_17_03_SEEN...origin/release-17.03"); do 136 | mark_commit_ui "$sha" 137 | next_RELEASE_17_03_SEEN="$sha" 138 | done 139 | 140 | prompt_next "$RELEASE_16_09_SEEN...origin/release-16.09" 141 | for sha in $(git rev-list --reverse --no-merges "$RELEASE_16_09_SEEN...origin/release-16.09"); do 142 | mark_commit_ui "$sha" 143 | next_RELEASE_16_09_SEEN="$sha" 144 | done 145 | 146 | fi 147 | -------------------------------------------------------------------------------- /ported-notes.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | #!/usr/bin/env nix-shell 3 | #!nix-shell --pure -i bash -p git 4 | 5 | set -eu 6 | set -o pipefail 7 | 8 | TMPDIR="/tmp" 9 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 10 | 11 | source "$DIR/state/port_state.sh" 12 | 13 | readonly TMPTO=$(mktemp -d ported.XXXXXXXXXX -p "$TMPDIR") 14 | cleanup() { 15 | rm -rf "$TMPTO" 16 | } 17 | trap cleanup EXIT 18 | 19 | debug() { 20 | set +u 21 | if ! [ "x$DEBUG" = "x" ]; then 22 | echo "$@" >&2 23 | fi 24 | set -u 25 | } 26 | 27 | commits() { 28 | local cache 29 | local branchname 30 | local range 31 | branchname="$1" 32 | range="$2" 33 | cache="$TMPTO/${branchname}_commits" 34 | if [ ! -f "$cache" ]; then 35 | git rev-list "${range}" | tee "$cache" 36 | else 37 | cat "$cache" 38 | fi 39 | } 40 | 41 | commit_has_note() { 42 | local commit 43 | commit="$1" 44 | 45 | if git notes --ref=security show "$commit" > /dev/null 2>&1; then 46 | return 0 47 | else 48 | return 1 49 | fi 50 | } 51 | 52 | commits_with_notes() { 53 | local cache 54 | local branchname 55 | local range 56 | branchname="$1" 57 | range="$2" 58 | 59 | cache="$TMPTO/${branchname}_commits_with_notes" 60 | if [ ! -f "$cache" ]; then 61 | local commit 62 | (for commit in $(commits "$branchname" "$range"); do 63 | if commit_has_note "$commit"; then 64 | echo "$commit" 65 | fi 66 | done) | tee "$cache" 67 | else 68 | cat "$cache" 69 | fi 70 | } 71 | 72 | log_commit() { 73 | local commit 74 | 75 | commit="$1" 76 | 77 | author=$(git show --no-patch --notes=security --pretty="format:%an" "${commit}") 78 | committer=$(git show --no-patch --notes=security --pretty="format:%cn" "${commit}") 79 | 80 | dontthank="Graham Christensen" 81 | if [ "x$author" = "x$committer" ]; then 82 | if [ "x$author" = "x$dontthank" ]; then 83 | thanks=""; 84 | else 85 | thanks="(Thank you, $author)" 86 | fi 87 | elif [ "x$author" = "x$dontthank" ]; then 88 | thanks="(Thank you, ${committer} (committer))" 89 | elif [ "x$committer" = "x$dontthank" ]; then 90 | thanks="(Thank you, ${author} (author))" 91 | else 92 | thanks="(Thank you: ${author} (author), ${committer} (committer))" 93 | fi 94 | 95 | git show --no-patch --notes=security --pretty="format: 96 | %h %<(60,trunc)%s 97 | " "${commit}" 98 | if [ "x$thanks" != "x" ]; then 99 | echo "$thanks" 100 | fi 101 | git show --no-patch --notes=security \ 102 | --pretty="format:%N" "${commit}" \ 103 | | sed -e 's/^/> /' 104 | 105 | } 106 | 107 | cat < 109 | To: nix-security-announce@googlegroups.com 110 | Subject: Security fixes from $(date -u "+%F %R %Z") 111 | --text follows this line-- 112 | <#secure method=pgp mode=sign> 113 | 114 | The following issues have been resolved in NixOS in release-16.09, 115 | release-17.03, and unstable. They remain potentially vulnerable on 116 | older major releases. 117 | 118 | These patches will be released to the release-16.09, release-17.03, 119 | and unstable channels when Hydra finishes building the "tested" job 120 | for each channel: 121 | 122 | - https://hydra.nixos.org/job/nixos/release-16.09/tested 123 | - https://hydra.nixos.org/job/nixos/release-17.03/tested 124 | - https://hydra.nixos.org/job/nixos/trunk-combined/tested 125 | 126 | Currently, 17.03 is considered beta. It will be released around the 127 | end of March. NixOS typically only supports one release at a time. 128 | This means when 17.03 is released you should upgrade as soon as 129 | possible. To ease this transition, I've decided to extend 16.09 130 | security patches for one month after 17.03 is released. 131 | 132 | You can switch from 16.09 to 17.03-beta via: 133 | 134 | $ sudo nix-channel --add https://nixos.org/channels/nixos-17.03 nixos 135 | $ sudo nix-channel --update 136 | $ sudo nixos-rebuild boot 137 | $ reboot 138 | 139 | Note: Don't use nixos-rebuild switch. The path to setuid wrappers has 140 | changed, and using switch will break setuid binaries (like sudo, ping, 141 | etc.) until you reboot. 142 | 143 | Please consider helping with the next security roundup by commenting on 144 | LATEST_ROUNDUP_URL. 145 | 146 | EOF 147 | 148 | 149 | changes_for() { 150 | local branch 151 | local range 152 | branch="$1" 153 | range="$2" 154 | 155 | echo "The following changes were applied to ${branch}" 156 | 157 | (for commit in $(commits_with_notes "$branch" "$range"); do 158 | log_commit "$commit" 159 | done) | cat 160 | } 161 | 162 | separator() { 163 | echo "======================================================================" 164 | echo "" 165 | echo "" 166 | echo "" 167 | } 168 | 169 | changes_for "release-16.09" "$RELEASE_16_09_SENT..origin/release-16.09" 170 | 171 | separator 172 | 173 | changes_for "release-17.03" "$RELEASE_17_03_SENT..origin/release-17.03" 174 | 175 | separator 176 | 177 | changes_for "unstable" "$MASTER_SENT..origin/master" 178 | 179 | cat < "$DIR/state/port_state.sh" 194 | -------------------------------------------------------------------------------- /shell.nix: -------------------------------------------------------------------------------- 1 | let 2 | sec = import ./default.nix {}; 3 | in sec.lwnvulns.shell 4 | -------------------------------------------------------------------------------- /state/notate_state.sh: -------------------------------------------------------------------------------- 1 | MASTER_SEEN=d12df18235ce8e7cd2f4cd066e89d53d00490080 2 | RELEASE_16_09_SEEN=1f9bd9246b1ea5af599a94a5104f3f207322be3e 3 | RELEASE_17_03_SEEN=8e72857c5535247c2853a774382f1eb2bd353986 4 | -------------------------------------------------------------------------------- /state/port_state.sh: -------------------------------------------------------------------------------- 1 | MASTER_SENT=84671200b6e029cba3f413d59b27f3944b3651c3 2 | RELEASE_16_09_SENT=1f9bd9246b1ea5af599a94a5104f3f207322be3e 3 | RELEASE_17.03_SENT=8e72857c5535247c2853a774382f1eb2bd353986 4 | --------------------------------------------------------------------------------