"]
24 |
25 | [dependencies]
26 | getopts = "~0.2.14"
27 | num = "~0.1.36"
28 | permutohedron = "~0.2.2"
29 | primal = "~0.2.3"
30 | ```
31 |
32 | Cargo can also run a test suite, generate documentation or upload your crate to the repository, but that's the topic for later.
33 |
34 | Crates and dependencies
35 | -----------------------
36 |
37 | If you don't know, Rust calls its compilation unit (be it a library or an executable) a **crate**. Your Rust program usually lives in one crate, but it can use other crates as dependencies. See the [Rust Guide on crates](http://doc.rust-lang.org/guide.html#crates-and-modules) for details on the difference between crates and modules.
38 |
39 | Executing `cargo build` compiles your crate and the resulting binary can be found inside the `target/` directory. For executables there's a nice shortcut - `cargo run` - that automatically runs your program after it's compiled.
40 |
41 | You probably noticed a `[dependencies]` section in the manifest file above. You guessed it - this is where you declare which external libraries you'll use. But `Cargo.toml` is not enough - it just tells Cargo to link your project with these crates. In order to use their APIs in your code, put `extern crate` imports in your project's crate root (that is usually the `main.rs` or `lib.rs` file), so that the compiler can resolve function names, types etc.
42 |
43 | ```rust
44 | // main.rs
45 | extern crate num;
46 |
47 | fn main () {
48 | // ... use stuff from num library
49 | }
50 | ```
51 |
52 | **Note**: At the moment Cargo supports only source dependencies; it downloads the source code for every crate your project depends on, compiles them locally on your machine and finally links with your main crate.
53 |
54 | What I personally like about Cargo is that it's very simple to build both a library and an executable using that library. It's just a matter of having two sections in the manifest. For example, this is what [rust-cpuid](https://github.com/zsiciarz/rust-cpuid) does:
55 |
56 | ```ini
57 | [lib]
58 | name = "cpuid"
59 |
60 | [[bin]]
61 | name = "cpuid"
62 | ```
63 |
64 | Then I just need to put `extern crate cpuid` in my `main.rs` file (this is the crate root for the executable) and Cargo will link it with the library it built alongside.
65 |
66 | **Update**: As [Steve Klabnik](http://www.reddit.com/r/rust/comments/2nybtm/24_days_of_rust_cargo_and_cratesio/cmip7xw) noted, these two sections are redundant as they're the defaults. If you have both `lib.rs` and `main.rs` in the source directory, Cargo will build the library and executable just the same.
67 |
68 | crates.io
69 | ---------
70 |
71 | [crates.io](https://crates.io/) is the central package repository for Cargo. (Python folks - think of PyPI.) Cargo pulls dependencies from there, although it can also install them from Git or local filesystem. Once you've built an awesome library and want to share it with the rest of the world, crates.io is the place to go. You'll need an account on the site (currently GitHub-based social login only, but this may change). The relevant command is `cargo publish`, documented in the [section on publishing crates](http://doc.crates.io/crates-io.html#publishing-crates). Once you've done it, congratulations, you are now a contributor to the Rust ecosystem!
72 |
--------------------------------------------------------------------------------
/book/vol2/day23.md:
--------------------------------------------------------------------------------
1 | # Day 23 - built with Rust
2 |
3 | Today's article is another throwback to the first edition of *24 days of Rust*.
4 | In one of the final posts I
5 | [wrote about interesting projects](http://siciarz.net/24-days-of-rust-built-with-rust/)
6 | built with Rust. That was even before the language hit version 1.0.
7 | Two years later and we're at 1.14. [Servo](https://github.com/servo/servo/),
8 | [iota](https://github.com/gchp/iota) and
9 | [coreutils](https://github.com/uutils/coreutils) are still going strong.
10 | It's surely worth checking them out again, but I'm going to introduce a few
11 | more Rust projects that emerged during the last two years.
12 |
13 | Redox
14 | -----
15 |
16 | How about starting with an **entire operating system** written in Rust? Because
17 | this is what [Redox](https://www.redox-os.org/) is about. It consists of:
18 |
19 | - a Unix-like microkernel
20 | - file system (RedoxFS)
21 | - userspace drivers
22 | - networking
23 | - GUI (Orbital)
24 | - and more...
25 |
26 | All built with Rust, free and open source! An impressive achievement for sure.
27 | While I wouldn't replace my Linux installs with Redox just yet, there are
28 | [.iso releases](https://github.com/redox-os/redox/releases) already. It should
29 | work in QEMU or VirtualBox.
30 |
31 | Project website: [https://www.redox-os.org/](https://www.redox-os.org/)
32 |
33 | ripgrep
34 | -------
35 |
36 | [Andrew Gallant](https://github.com/BurntSushi), aka `burntsushi`, is a
37 | *prolific* contributor to the Rust library ecosystem. Regexes, CSV,
38 | byte ordering, docopt, property-based testing - Andrew's your man.
39 | This time he took a stab at filesystem search, with an impressive outcome.
40 | [`ripgrep`](https://github.com/BurntSushi/ripgrep) is a CLI search tool
41 | that aims to surpass `grep`, `ack`, `ag` and others in both usability and
42 | performance. In the aptly titled introductory blog post -
43 | [ripgrep is faster than {grep, ag, git grep, ucg, pt, sift}](http://blog.burntsushi.net/ripgrep/) -
44 | Andrew explains his motivation and presents performance benchmarks.
45 |
46 | TL;DR - `ripgrep` is **fast**. Try it out!
47 |
48 | Project website: [https://github.com/BurntSushi/ripgrep](https://github.com/BurntSushi/ripgrep)
49 |
50 | panopticon
51 | ----------
52 |
53 | I learned about [panopticon](https://panopticon.re/) at
54 | [RustFest EU 2016](http://www.rustfest.eu/talks/#panopticon-a-libre-cross-platform-disassembler-in-rust).
55 | It's a GUI disassembler (for x86, AMD64, AVR and 6502) that aims to be a free,
56 | easy to use and powerful alternative to proprietary tools such as IDA.
57 | Panopticon was rewritten from C++ to Rust and the author expresses his joy
58 | [in a reddit thread](https://www.reddit.com/r/rust/comments/4ihtfa/panopticon_a_libre_crossplatform_disassembler/):
59 |
60 | > Programming in Rust is not only more fun, it's definitely easier too.
61 | > Panopticon used alot sum types that were implemented using boost::variant.
62 | > Like everything with Boost it kind of worked but was incredible ugly and
63 | > complex. Replacing them with enums was probably the biggest reason I switched
64 | > to Rust. Also I found iterator invalidation bugs simply by translating C++
65 | > to Rust, thanks Borrow Checker!
66 |
67 | Project website: [https://panopticon.re/](https://panopticon.re/)
68 |
69 | Mozilla Firefox
70 | ---------------
71 |
72 | Well, this may be a little stretch ;-) But just a little. Firefox itself wasn't
73 | suddenly rewritten in its entirety from C++ to Rust. However, there are already
74 | [some parts of the browser](https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/)
75 | built with Rust. [Stylo](https://wiki.mozilla.org/Stylo) is an upcoming
76 | bridge between Firefox and Servo's CSS engine. And with
77 | [Project Quantum](https://medium.com/mozilla-tech/a-quantum-leap-for-the-web-a3b7174b3c12#.kcpq0q16m)
78 | even more good stuff is coming!
79 |
80 | Now it's your turn!
81 | -------------------
82 |
83 | I'd love to see **you** building awesome Rust projects! Pick a problem that
84 | you'd like to solve and try doing that with Rust. It doesn't have to be an
85 | entire new operating system (that was a hell of an itch to scratch,
86 | Redox folks, right?).
87 |
88 | Here are some inspirations:
89 |
90 | - [Awesome Rust](https://github.com/kud1ing/awesome-rust) - a collection of
91 | programs, libraries and resources
92 | - [Contributing to Rust — libraries](https://www.rust-lang.org/en-US/contribute-libs.html)
93 | - [Are we Web yet?](http://www.arewewebyet.org/) - some areas aren't as green
94 | as I'd like them to be
95 | - [A list of practical projects that anyone can solve in any programming language](https://github.com/karan/Projects)
96 | - [Where do you get ideas to build side-projects?](https://news.ycombinator.com/item?id=5234692) -
97 | a Hacker News discussion
98 |
99 | To summarize this article:
100 |
101 | > Getting a project featured in *24 days of Rust* is left as an exercise for
102 | > the reader.
103 |
104 | Further reading
105 | ---------------
106 |
107 | - [Awesome Rust](https://github.com/kud1ing/awesome-rust) - includes applications written in Rust
108 | - [Friends of Rust](https://www.rust-lang.org/en-US/friends.html) - Organizations running Rust in production
109 | - [ripgrep code review](http://blog.mbrt.it/2016-12-01-ripgrep-code-review/)
110 | - [Quantum](https://wiki.mozilla.org/Quantum) at Mozilla Wiki
111 |
--------------------------------------------------------------------------------
/book/vol2/day7.md:
--------------------------------------------------------------------------------
1 | # Day 7 - static initialization
2 |
3 | [Static variables](https://en.wikipedia.org/wiki/Static_variable) are available
4 | throughout the entire life of a program. They are allocated in a block of
5 | memory known at compile time. Due to that, they tend to represent global
6 | state that the program can access.
7 | It's getting especially tricky if one static variable depends on another.
8 | Some language communities even talk about
9 | [static initialization order fiasco](https://isocpp.org/wiki/faq/ctors#static-init-order)
10 | (looking at you, C++). Other, like C, allow static intialization only with
11 | constant literals/expressions. Rust
12 | [belongs to this group](https://doc.rust-lang.org/beta/book/const-and-static.html)
13 | as well. But there are alternatives...
14 |
15 | Looking up colors by name
16 | -------------------------
17 |
18 | Suppose we're building a web browser engine. Among thousands of things to take
19 | care of, we should be able to render colorful text. ``
20 | should look like a paragraph set in a blue font. But `blue` is a human-readable
21 | name for a color; computers like numbers. Let's translate it with the help
22 | of a `Color` struct:
23 |
24 | ```rust
25 | #[derive(Clone, Debug)]
26 | pub struct Color {
27 | r: u8,
28 | g: u8,
29 | b: u8,
30 | }
31 | ```
32 |
33 | We cannot create a static `HashMap` and initialize it with a mapping
34 | from color names to `Color`s. So let's use pattern matching to find
35 | color by name:
36 |
37 | ```rust
38 | pub fn find_color(name: &str) -> Option {
39 | match name.to_lowercase().as_str() {
40 | "amber" => Some(Color { r: 255, g: 191, b: 0 }),
41 | // hundreds of other names...
42 | "zinnwaldite brown" => Some(Color { r: 44, g: 22, b: 8 }),
43 | _ => None,
44 | }
45 | }
46 | ```
47 |
48 | The downside is that matching string slices will probably generate a linear
49 | search. So the more colors we have, the slower `find_color` will be.
50 |
51 | Did I just say we cannot have a static `HashMap`?
52 |
53 | > *puts on Morpheus sunglasses*
54 |
55 | What if I told you...
56 |
57 | `lazy_static!`
58 | --------------
59 |
60 | [`lazy_static`](https://crates.io/crates/lazy_static/) is a crate that
61 | enables `static` variables that are initialized in a non-trivial way.
62 | For example a precomputed regular epression such as
63 | [the ones used in `docopt`](https://github.com/docopt/docopt.rs/blob/717c26c1924d2c95fac48814c96ac6979fe2f20d/src/parse.rs#L182),
64 | or a static `HashMap`!
65 |
66 | ```rust
67 | #[macro_use]
68 | extern crate lazy_static;
69 |
70 | use std::collections::HashMap;
71 |
72 | lazy_static! {
73 | static ref COLORS_MAP: HashMap<&'static str, Color> = {
74 | let mut map = HashMap::new();
75 | map.insert("amber", Color { r: 255, g: 191, b: 0 });
76 | // ...
77 | map.insert("zinnwaldite brown", Color { r: 44, g: 22, b: 8 });
78 | map
79 | };
80 | }
81 |
82 | pub fn find_color_lazy_static(name: &str) -> Option {
83 | COLORS_MAP.get(name.to_lowercase().as_str()).cloned()
84 | }
85 | ```
86 |
87 | `COLORS_MAP` will be evaluated on first access. We can now safely treat
88 | it as if it was a regular static variable.
89 |
90 | `phf`
91 | -----
92 |
93 | `HashMap` uses a *somewhat slow hash algorithm* (quoting the documentation)
94 | to avoid DoS attacks. With large enough maps there's also a possibility of
95 | collisions.
96 |
97 | On the other hand, [`phf`](https://crates.io/crates/phf) uses perfect hashing
98 | (hashing that guarantees no collisions) to build compile-time maps.
99 | This way we can have efficient constant-time lookups at runtime.
100 |
101 | ```rust
102 | #![feature(plugin)]
103 | #![plugin(phf_macros)]
104 |
105 | #[macro_use]
106 | extern crate phf;
107 |
108 | static COLORS: phf::Map<&'static str, Color> = phf_map! {
109 | "amber" => Color { r: 255, g: 191, b: 0 },
110 | // ...
111 | "zinnwaldite brown" => Color { r: 44, g: 22, b: 8 },
112 | };
113 |
114 | pub fn find_color_phf(name: &str) -> Option {
115 | COLORS.get(name.to_lowercase().as_str()).cloned()
116 | }
117 | ```
118 |
119 | Benchmarks!
120 | -----------
121 |
122 | So what's the difference in speed between these approaches?
123 |
124 | ```rust
125 | #[cfg(test)]
126 | mod tests {
127 | use super::*;
128 | use test::Bencher;
129 |
130 | #[bench]
131 | fn bench_match_lookup(b: &mut Bencher) {
132 | b.iter(|| find_color("White"))
133 | }
134 |
135 | #[bench]
136 | fn bench_lazy_static_map(b: &mut Bencher) {
137 | b.iter(|| find_color_lazy_static("White"))
138 | }
139 |
140 | #[bench]
141 | fn bench_phf_map(b: &mut Bencher) {
142 | b.iter(|| find_color_phf("White"))
143 | }
144 | }
145 | ```
146 |
147 | ```text
148 | $ cargo bench
149 | running 3 tests
150 | test tests::bench_lazy_static_map ... bench: 367 ns/iter (+/- 20)
151 | test tests::bench_match_lookup ... bench: 3,948 ns/iter (+/- 460)
152 | test tests::bench_phf_map ... bench: 350 ns/iter (+/- 33)
153 | ```
154 |
155 | Linear search at runtime in the `match` statement is the slowest, as expected.
156 | Both static `HashMap` and `phf::Map` are about an order of magnitude faster,
157 | with the latter leading by a small amount. I would personally prefer `phf`
158 | for lookups like this, as it is the intended use for static compile-time
159 | maps. `lazy_static` is more general in its intent and initializing maps
160 | is just one of its many potential uses.
161 |
162 | Further reading
163 | ---------------
164 |
165 | - [Static initializers will murder your family](https://meowni.ca/posts/static-initializers/) (in C++, fortunately not Rust)
166 | - [Perfect hashing functions](https://en.wikipedia.org/wiki/Perfect_hash_function)
167 | - [Match statement efficiency?](https://users.rust-lang.org/t/match-statement-efficiency/4488/1)
168 |
--------------------------------------------------------------------------------
/book/vol1/day21.md:
--------------------------------------------------------------------------------
1 | # Day 21 - rust-crypto
2 |
3 | > Relevancy: 1.9 stable
4 |
5 | The [rust-crypto](https://crates.io/crates/rust-crypto) crate is a collection of a lot of cryptography primitives and algorithms. There are tools for calculating hashes, verifying data integrity, encryption etc. One disclaimer - it hasn't had a proper security audit yet and although the algorithms are well known and researched, the library itself *might* have security bugs. But [which](http://heartbleed.com/) [one](https://www.mozilla.org/en-US/security/advisories/mfsa2014-73/) [doesn't](http://www.gnutls.org/security.html)?
6 |
7 | Cryptographic hashes
8 | --------------------
9 |
10 | Let's start with a simple task of computing a [cryptographic hash](http://en.wikipedia.org/wiki/Cryptographic_hash_function) of some value. We'll use the SHA-256 algorithm to demonstrate.
11 |
12 | [include:1-1](../../vol1/src/bin/day21.rs)
13 | [include:6-6](../../vol1/src/bin/day21.rs)
14 | [include:9-9](../../vol1/src/bin/day21.rs)
15 | [include:20-23](../../vol1/src/bin/day21.rs)
16 |
17 | All hash algorithms in rust-crypto implement the `Digest` trait, which defines the low level methods such as `input()` and `result()` operating on bytes, but also convenience string-based methods as shown above. We can use those low level methods for example to represent our hash as a base64-encoded string:
18 |
19 | [include:3-3](../../vol1/src/bin/day21.rs)
20 | [include:10-10](../../vol1/src/bin/day21.rs)
21 | [include:15-15](../../vol1/src/bin/day21.rs)
22 | [include:24-26](../../vol1/src/bin/day21.rs)
23 |
24 | Here's the output:
25 |
26 | ```sh
27 | $ cargo run
28 | c0535e4be2b79ffd93291305436bf889314e4a3faec05ecffcbb7df31ad9e51a
29 | wFNeS+K3n/2TKRMFQ2v4iTFOSj+uwF7P/Lt98xrZ5Ro=
30 | ```
31 |
32 | Ciphers
33 | -------
34 |
35 | To actually encrypt some data in a way that we can decrypt it back later, we need a **cipher**. A few of these are provided by the `rust-crypto` crate, here's an example of [AES](http://en.wikipedia.org/wiki/Advanced_Encryption_Standard) encryption in CTR mode:
36 |
37 | [include:5-5](../../vol1/src/bin/day21.rs)
38 | [include:10-10](../../vol1/src/bin/day21.rs)
39 | [include:16-16](../../vol1/src/bin/day21.rs)
40 | [include:28-39](../../vol1/src/bin/day21.rs)
41 |
42 | We generate the secret key and a nonce value with a secure random generator - `OsRng`. Then we need to call the `ctr()` function which returns a *best possible* implementation of AES-CTR cipher (taking into consideration CPU architecture etc.). What we get back is a trait object - a `SynchronousStreamCipher` value in a `Box`. Calling `process()` should encrypt our secret message and store the result bytes in the `output` vector.
43 |
44 | Here's the output:
45 |
46 | ```sh
47 | $ cargo run
48 | Key: NvDy+u51EfMC+amJzoJO+w==
49 | Nonce: d5+SLyfPGUSeug50nK1WGA==
50 | Ciphertext: vTVxjyUms4Z4jex/OcMcQlY=
51 | ```
52 |
53 | We can decrypt this with Python (who said it would be Rust all the way?):
54 |
55 | ```python
56 | import base64
57 | from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
58 | from cryptography.hazmat.backends import default_backend
59 |
60 | key = base64.decodebytes(b'NvDy+u51EfMC+amJzoJO+w==')
61 | nonce = base64.decodebytes(b'd5+SLyfPGUSeug50nK1WGA==')
62 | ct = base64.decodebytes(b'vTVxjyUms4Z4jex/OcMcQlY=')
63 | backend = default_backend()
64 | cipher = Cipher(algorithms.AES(key), modes.CTR(nonce), backend=backend)
65 | decryptor = cipher.decryptor()
66 | print(decryptor.update(ct) + decryptor.finalize())
67 | ```
68 |
69 | ```sh
70 | $ python3 decrypt.py
71 | b'I like Nickelback'
72 | ```
73 |
74 | HMAC
75 | ----
76 |
77 | Message authentication algorithms such as [HMAC](http://en.wikipedia.org/wiki/Hash-based_message_authentication_code) verify **both** data integrity and authenticity. Let's see how to generate a MAC for a given message:
78 |
79 | [include:7-8](../../vol1/src/bin/day21.rs)
80 | [include:13-13](../../vol1/src/bin/day21.rs)
81 | [include:41-48](../../vol1/src/bin/day21.rs)
82 |
83 | As with the AES cipher example, we generate a random secret key. Then we create the `Hmac` value, passing a cryptographic hash function object (anything that implements `Digest`). Since `Hmac` doesn't have these convenience methods for working with strings, we manually feed the bytes and encode the digest to hexadecimal string. The program outputs a key (we should give it to the recipient of the message through some other secure channel) and an HMAC digest.
84 |
85 | ```sh
86 | $ cargo run
87 | Message: Ceterum censeo Carthaginem esse delendam
88 | HMAC key: esU5jdGCbM7E/ME5WBECJ+BdX3kt7bcQ3HkeEK+W6ZQ=
89 | HMAC digest: b3240371a17e1e9755b89b23449f0d85c4c361e94e081c7adbe5a89c2d901aaa
90 | ```
91 |
92 | And here's a simple Python program to verify validity of the message using the [hmac module](https://docs.python.org/3.4/library/hmac.html):
93 |
94 | ```python
95 | import base64
96 | import hashlib
97 | import hmac
98 |
99 | key = base64.decodebytes(b'esU5jdGCbM7E/ME5WBECJ+BdX3kt7bcQ3HkeEK+W6ZQ=')
100 | message = b'Ceterum censeo Carthaginem esse delendam'
101 | expected = 'b3240371a17e1e9755b89b23449f0d85c4c361e94e081c7adbe5a89c2d901aaa'
102 | h = hmac.new(key, message, hashlib.sha256)
103 | print(hmac.compare_digest(h.hexdigest(), expected))
104 | ```
105 |
106 | So... is the message valid and authentic?
107 |
108 | ```sh
109 | $ python3 verify.py
110 | True
111 | ```
112 |
113 | See also
114 | --------
115 |
116 | * [Thoughts on Rust cryptography](https://speakerdeck.com/tarcieri/thoughts-on-rust-cryptography)
117 | * [Cryptography 1](https://www.coursera.org/course/crypto) - an introductory cryptography course - recommended!
118 | * [extensive comment](https://github.com/DaGenix/rust-crypto/blob/340cc5f142601077d6838eb6aa0c3b29b7f67358/src/rust-crypto/aessafe.rs#L9) in rust-crypto about AES timing issues
119 | * [Should we MAC-then-encrypt or encrypt-then-MAC?](http://crypto.stackexchange.com/questions/202/should-we-mac-then-encrypt-or-encrypt-then-mac) (short answer: **encrypt-then-MAC**)
120 |
--------------------------------------------------------------------------------
/vol1/src/bin/day15.rs:
--------------------------------------------------------------------------------
1 | #[cfg(target_family = "unix")]
2 | extern crate fuse;
3 | extern crate time;
4 | extern crate libc;
5 | extern crate rustc_serialize;
6 |
7 | #[cfg(target_family = "unix")]
8 | use std::collections::BTreeMap;
9 | #[cfg(target_family = "unix")]
10 | use std::env;
11 | #[cfg(target_family = "unix")]
12 | use std::ffi::OsStr;
13 | #[cfg(target_family = "unix")]
14 | use libc::ENOENT;
15 | #[cfg(target_family = "unix")]
16 | use time::Timespec;
17 | #[cfg(target_family = "unix")]
18 | use fuse::{FileAttr, FileType, Filesystem, Request, ReplyAttr, ReplyData, ReplyEntry,
19 | ReplyDirectory};
20 | #[cfg(target_family = "unix")]
21 | use rustc_serialize::json;
22 |
23 | #[cfg(target_family = "unix")]
24 | struct JsonFilesystem {
25 | tree: json::Object,
26 | attrs: BTreeMap,
27 | inodes: BTreeMap,
28 | }
29 |
30 | #[cfg(target_family = "unix")]
31 | impl JsonFilesystem {
32 | fn new(tree: &json::Object) -> JsonFilesystem {
33 | let mut attrs = BTreeMap::new();
34 | let mut inodes = BTreeMap::new();
35 | let ts = time::now().to_timespec();
36 | let attr = FileAttr {
37 | ino: 1,
38 | size: 0,
39 | blocks: 0,
40 | atime: ts,
41 | mtime: ts,
42 | ctime: ts,
43 | crtime: ts,
44 | kind: FileType::Directory,
45 | perm: 0o755,
46 | nlink: 0,
47 | uid: 0,
48 | gid: 0,
49 | rdev: 0,
50 | flags: 0,
51 | };
52 | attrs.insert(1, attr);
53 | inodes.insert("/".to_string(), 1);
54 | for (i, (key, value)) in tree.iter().enumerate() {
55 | let attr = FileAttr {
56 | ino: i as u64 + 2,
57 | size: value.pretty().to_string().len() as u64,
58 | blocks: 0,
59 | atime: ts,
60 | mtime: ts,
61 | ctime: ts,
62 | crtime: ts,
63 | kind: FileType::RegularFile,
64 | perm: 0o644,
65 | nlink: 0,
66 | uid: 0,
67 | gid: 0,
68 | rdev: 0,
69 | flags: 0,
70 | };
71 | attrs.insert(attr.ino, attr);
72 | inodes.insert(key.clone(), attr.ino);
73 | }
74 | JsonFilesystem {
75 | tree: tree.clone(),
76 | attrs: attrs,
77 | inodes: inodes,
78 | }
79 | }
80 | }
81 |
82 | #[cfg(target_family = "unix")]
83 | impl Filesystem for JsonFilesystem {
84 | fn getattr(&mut self, _req: &Request, ino: u64, reply: ReplyAttr) {
85 | println!("getattr(ino={})", ino);
86 | match self.attrs.get(&ino) {
87 | Some(attr) => {
88 | let ttl = Timespec::new(1, 0);
89 | reply.attr(&ttl, attr);
90 | }
91 | None => reply.error(ENOENT),
92 | };
93 | }
94 |
95 | fn lookup(&mut self, _req: &Request, parent: u64, name: &OsStr, reply: ReplyEntry) {
96 | println!("lookup(parent={}, name={})", parent, name.to_str().unwrap());
97 | let inode = match self.inodes.get(name.to_str().unwrap()) {
98 | Some(inode) => inode,
99 | None => {
100 | reply.error(ENOENT);
101 | return;
102 | }
103 | };
104 | match self.attrs.get(inode) {
105 | Some(attr) => {
106 | let ttl = Timespec::new(1, 0);
107 | reply.entry(&ttl, attr, 0);
108 | }
109 | None => reply.error(ENOENT),
110 | };
111 | }
112 |
113 | fn read(
114 | &mut self,
115 | _req: &Request,
116 | ino: u64,
117 | fh: u64,
118 | offset: i64,
119 | size: u32,
120 | reply: ReplyData,
121 | ) {
122 | println!(
123 | "read(ino={}, fh={}, offset={}, size={})",
124 | ino,
125 | fh,
126 | offset,
127 | size
128 | );
129 | for (key, &inode) in &self.inodes {
130 | if inode == ino {
131 | let value = &self.tree[key];
132 | reply.data(value.pretty().to_string().as_bytes());
133 | return;
134 | }
135 | }
136 | reply.error(ENOENT);
137 | }
138 |
139 | fn readdir(
140 | &mut self,
141 | _req: &Request,
142 | ino: u64,
143 | fh: u64,
144 | offset: i64,
145 | mut reply: ReplyDirectory,
146 | ) {
147 | println!("readdir(ino={}, fh={}, offset={})", ino, fh, offset);
148 | if ino == 1 {
149 | if offset == 0 {
150 | reply.add(1, 0, FileType::Directory, ".");
151 | reply.add(1, 1, FileType::Directory, "..");
152 | for (key, &inode) in &self.inodes {
153 | if inode == 1 {
154 | continue;
155 | }
156 | let offset = inode as i64; // hack
157 | println!("\tkey={}, inode={}, offset={}", key, inode, offset);
158 | reply.add(inode, offset, FileType::RegularFile, key);
159 | }
160 | }
161 | reply.ok();
162 | } else {
163 | reply.error(ENOENT);
164 | }
165 | }
166 | }
167 |
168 | #[cfg(target_family = "windows")]
169 | fn main() {
170 | println!("24 days of Rust - fuse (days 15 & 16)");
171 | println!("This example does not work on Windows :(");
172 | }
173 |
174 | #[cfg(target_family = "unix")]
175 | fn main() {
176 | println!("24 days of Rust - fuse (days 15 & 16)");
177 | let data = json::Json::from_str("{\"foo\": \"bar\", \"answer\": 42}").unwrap();
178 | let tree = data.as_object().unwrap();
179 | let fs = JsonFilesystem::new(tree);
180 | let mountpoint = match env::args().nth(1) {
181 | Some(path) => path,
182 | None => {
183 | println!("Usage: {} ", env::args().nth(0).unwrap());
184 | return;
185 | }
186 | };
187 | fuse::mount(fs, &mountpoint, &[]).expect("Couldn't mount filesystem");
188 | }
189 |
--------------------------------------------------------------------------------
/book/vol2/day13.md:
--------------------------------------------------------------------------------
1 | # Day 13 - zip and lzma compression
2 |
3 | The [`zip`](https://crates.io/crates/zip) crate is the most commonly used
4 | Rust library for manipulating ZIP archives. It supports reading and writing
5 | .zip files with different compression methods (store, deflate, bzip2).
6 |
7 | There are at least three crates for LZMA (de)compression on crates.io.
8 | [`lzma`](https://crates.io/crates/lzma) is pure Rust, but currently allows
9 | only reading from archives.
10 | [`rust-lzma`](https://crates.io/crates/rust-lzma) supports both reading
11 | and writing compressed data, but it's a wrapper for `liblzma`. Similarly
12 | the [`xz2`](https://crates.io/crates/xz2) crate is a binding for `liblzma`.
13 |
14 | Creating archives
15 | -----------------
16 |
17 | We're going to compress a text file, namely the `Cargo.lock` file which
18 | exists in every `cargo` project and keeps track of precise versions
19 | of dependencies. This is for illustration only, the lock file doesn't grow
20 | so much it would require compression.
21 |
22 | ```rust
23 | static FILE_CONTENTS: &'static [u8] = include_bytes!("../../Cargo.lock");
24 | ```
25 |
26 | The `include_bytes!` macro comes from the standard library and allows
27 | for embedding literal arrays of bytes in the source code.
28 |
29 | ```rust
30 | extern crate zip;
31 |
32 | use std::io::{Seek, Write};
33 | use zip::result::ZipResult;
34 | use zip::write::{FileOptions, ZipWriter};
35 |
36 | fn create_zip_archive(buf: &mut T) -> ZipResult<()> {
37 | let mut writer = ZipWriter::new(buf);
38 | writer.start_file("example.txt", FileOptions::default())?;
39 | writer.write(FILE_CONTENTS)?;
40 | writer.finish()?;
41 | Ok(())
42 | }
43 | ```
44 |
45 | The `zip` crate exposes a `ZipWriter` struct which wraps anything that's
46 | `Seek + Write` (a file, stdout, an in-memory buffer etc).
47 |
48 | ```rust
49 | fn main() {
50 | let mut file = File::create("example.zip").expect("Couldn't create file");
51 | create_zip_archive(&mut file).expect("Couldn't create archive");
52 | }
53 | ```
54 |
55 | After running this, we should now have an `example.zip` file in the current
56 | directory. You can verify with `unzip` or a GUI archive reader like 7-Zip
57 | that it contains correct data.
58 |
59 | Reading ZIP archives
60 | --------------------
61 |
62 | In the same vein as `ZipWriter` wraps a writable object, the `ZipArchive`
63 | is a wrapper around `Read + Seek`. We can use it to read archive contents
64 | like in the example below:
65 |
66 | ```rust
67 | fn browse_zip_archive(buf: &mut T, browse_func: F) -> ZipResult>
68 | where T: Read + Seek,
69 | F: Fn(&ZipFile) -> ZipResult
70 | {
71 | let mut archive = ZipArchive::new(buf)?;
72 | (0..archive.len())
73 | .map(|i| archive.by_index(i).and_then(|file| browse_func(&file)))
74 | .collect()
75 | }
76 | ```
77 |
78 | The `browse_zip_archive` function goes through all files in the archive and
79 | applies a callback function to each one. This flexibility allows the caller
80 | to decide what to do with each file in turn. The values returned by the
81 | callback are collected into a `Vec` and returned if all goes well. We're
82 | using a clever trick here: `Result`
83 | [implements `FromIterator`](https://doc.rust-lang.org/std/result/enum.Result.html#method.from_iter).
84 | This means we can turn an iterator of `Result`s into a `Result` wrapping a
85 | container (`Vec` here) with a single call to `collect()`. And if any element
86 | is an `Err`, the `Err` is returned from the entire function.
87 |
88 | ```rust
89 | let mut file = File::open("example.zip").expect("Couldn't open file");
90 | let files = browse_zip_archive(&mut file, |f| {
91 | Ok(format!("{}: {} -> {}", f.name(), f.size(), f.compressed_size()))
92 | });
93 | println!("{:?}", files);
94 | ```
95 |
96 | ```text
97 | $ cargo run
98 | Ok(["example.txt: 66386 -> 10570"])
99 | ```
100 |
101 | Other archive formats
102 | ---------------------
103 |
104 | ```rust
105 | fn create_bz2_archive(buf: &mut T) -> ZipResult<()> {
106 | let mut writer = ZipWriter::new(buf);
107 | writer.start_file("example.txt",
108 | FileOptions::default().compression_method(zip::CompressionMethod::Bzip2))?;
109 | writer.write(FILE_CONTENTS)?;
110 | writer.finish()?;
111 | Ok(())
112 | }
113 | ```
114 |
115 | We can use `zip` to create a BZIP2 archive. The only change is in the
116 | compression method used by `ZipWriter`.
117 |
118 | And now let's use the `rust-lzma` crate to compress our file to an `.xz`
119 | archive.
120 |
121 | ```rust
122 | use lzma::{LzmaWriter, LzmaError};
123 |
124 | fn create_xz_archive(buf: &mut T) -> Result<(), LzmaError> {
125 | let mut writer = LzmaWriter::new_compressor(buf, 6)?;
126 | writer.write(FILE_CONTENTS)?;
127 | writer.finish()?;
128 | Ok(())
129 | }
130 | ```
131 |
132 | LZMA compression doesn't require the buffer to be seekable, it just emits a
133 | stream of compressed bytes as it goes over the input. The other difference
134 | is that `LzmaWriter` supports different compression levels (6 is typically
135 | the default).
136 |
137 | Comparison
138 | ----------
139 |
140 | We may be interested in space efficiency of various compression methods.
141 | Let's use the [`metadata`](https://doc.rust-lang.org/std/fs/fn.metadata.html)
142 | function to retrieve size of each file:
143 |
144 | ```rust
145 | if let Ok(meta) = metadata("example.zip") {
146 | println!("ZIP file size: {} bytes", meta.len());
147 | }
148 | if let Ok(meta) = metadata("example.bz2") {
149 | println!("BZ2 file size: {} bytes", meta.len());
150 | }
151 | if let Ok(meta) = metadata("example.xz") {
152 | println!("XZ file size: {} bytes", meta.len());
153 | }
154 | ```
155 |
156 | ```text
157 | $ cargo run
158 | ZIP file size: 10696 bytes
159 | BZ2 file size: 8524 bytes
160 | XZ file size: 9154 bytes
161 | ```
162 |
163 | Further reading
164 | ---------------
165 |
166 | - [tar-rs](https://github.com/alexcrichton/tar-rs)
167 | - [brotli](https://crates.io/crates/brotli)
168 | - [Comparison of archive formats](https://en.wikipedia.org/wiki/Comparison_of_archive_formats)
169 | - [Gzip vs Bzip2 vs XZ Performance Comparison](https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/)
170 |
--------------------------------------------------------------------------------
/book/vol2/day6.md:
--------------------------------------------------------------------------------
1 | # Day 6 - derive_builder
2 |
3 | Today our focus is the [derive_builder](https://crates.io/crates/derive_builder)
4 | crate. It provides a macro to automatically generate setter methods
5 | for struct fields. And since all setters return the struct itself,
6 | we can chain them in a so called *builder pattern*.
7 |
8 | Builder and fluent interfaces
9 | -----------------------------
10 |
11 | [Builder pattern](https://en.wikipedia.org/wiki/Builder_pattern) (in the sense
12 | of the original Design Patterns book) is an object that knows how to construct
13 | other objects step by step. The builder approach aims to simplify object
14 | creation. Instead of a long argument list in the constructor (or a method
15 | like `new()` in Rust), we initialize the object by calling subsequent
16 | builder methods.
17 |
18 | Each builder method returns the builder itself. This allows for chaining
19 | the calls right after each other `just().like().this()`. For example
20 | [diesel](http://diesel.rs/) uses such method chaining to build database queries:
21 |
22 | ```rust
23 | let versions = Version::belonging_to(krate)
24 | .select(id)
25 | .order(num.desc())
26 | .limit(5);
27 | ```
28 |
29 | Fluent interfaces are useful in some contexts; they are not a silver bullet
30 | (nothing is). Marco Pivetta wrote a good critique in his blog post
31 | [Fluent interfaces are evil](https://ocramius.github.io/blog/fluent-interfaces-are-evil/).
32 |
33 | Game config example
34 | -------------------
35 |
36 | Imagine we're writing a computer game. We need to store game
37 | configuration such as screen resolution, save directory location etc. A lot
38 | of modern games start via a separate launcher which tries to autodetect
39 | these values and presents initial config to the actual game. Let's see
40 | how we could use `derive_builder` in such a launcher.
41 |
42 | ```rust
43 | #[macro_use]
44 | extern crate custom_derive;
45 | #[macro_use]
46 | extern crate derive_builder;
47 | ```
48 |
49 | [custom_derive](https://crates.io/crates/custom_derive) is a helper crate that
50 | allows deriving user-defined attributes. We'll see that later, but here's a
51 | regular derive:
52 |
53 | ```rust
54 | #[derive(Debug)]
55 | struct Resolution {
56 | width: u32,
57 | height: u32,
58 | }
59 |
60 | impl Default for Resolution {
61 | fn default() -> Resolution {
62 | Resolution {
63 | width: 1920,
64 | height: 1080,
65 | }
66 | }
67 | }
68 | ```
69 |
70 | `Resolution` is just a pair of integers with a debugging representation
71 | derived for us by the Rust compiler. We could also derive `Default`, but this
72 | would use default values for both dimensions, giving us a 0px x 0px screen. Not
73 | really useful...
74 |
75 | ```rust
76 | custom_derive! {
77 | #[derive(Debug, Default, Builder)]
78 | struct GameConfig {
79 | resolution: Resolution,
80 | save_dir: Option,
81 | autosave: bool,
82 | fov: f32,
83 | render_distance: u32,
84 | }
85 | }
86 | ```
87 |
88 | Here's the meat of the matter. `Debug` and `Default` are standard Rust traits
89 | that the compiler knows how to derive. `Builder` on the other hand is actually
90 | a `Builder!` macro, that `custom_derive!` knows how to call on our struct.
91 |
92 | So this is a plain Rust struct inside some macro. What does it buy us?
93 |
94 | ```rust
95 | let mut conf = GameConfig::default();
96 | conf.save_dir("saves".to_string()).fov(70.0).render_distance(1000u32);
97 | println!("{:?}", conf);
98 | ```
99 |
100 | ```text
101 | $ cargo run
102 | GameConfig { resolution: Resolution { width: 1920, height: 1080 }, save_dir: Some("saves"), autosave: false, fov: 70, render_distance: 1000 }
103 | ```
104 |
105 | Note how the resolution and autosave flag are still correctly set from the
106 | defaults, while we use generated setters for the rest of settings.
107 | Another interesting thing happens when we set `save_dir`. The struct contains
108 | an `Option`, but we're passing a `String` into the setter. How does
109 | that even compile?
110 |
111 | Since [Rust 1.12](https://blog.rust-lang.org/2016/09/29/Rust-1.12.html),
112 | `Option` implements `From`. And the generated setter signature
113 | would look like:
114 |
115 | ```rust
116 | pub fn save_dir>(&mut self, value: VALUE) -> &mut Self
117 | ```
118 |
119 | `From` and `Into` traits complement each other, hence everything typechecks.
120 |
121 | We can of course add `impl GameConfig` and implement our own methods - this is
122 | still a regular Rust struct. The only thing `derive_builder` does is adding
123 | public setter methods.
124 |
125 | (Non)-consuming builders
126 | ------------------------
127 |
128 | There's a small gotcha with `derive_builder`. We can't initialize config like
129 | this:
130 |
131 | ```rust
132 | let mut conf = GameConfig::default().fov(0.0);
133 | ```
134 |
135 | This will fail to compile with an error like below:
136 |
137 | ```text
138 | error: borrowed value does not live long enough
139 | --> src\day6.rs:34:50
140 | |
141 | 34 | let mut conf = GameConfig::default().fov(0.0);
142 | | --------------------- ^ temporary value dropped here while still borrowed
143 | | |
144 | | temporary value created here
145 | ...
146 | 37 | }
147 | | - temporary value needs to live until here
148 | |
149 | = note: consider using a `let` binding to increase its lifetime
150 | ```
151 |
152 | `derive_builder` prefers a so called *non-consuming* builder approach, as
153 | described in [The builder pattern](https://aturon.github.io/ownership/builders.html).
154 | See [this issue](https://github.com/colin-kiegel/rust-derive-builder/issues/2)
155 | for the rationale behind this choice.
156 |
157 | Further reading
158 | ---------------
159 |
160 | - [The builder pattern](https://aturon.github.io/ownership/builders.html)
161 | - [FluentInterface by Martin Fowler](http://martinfowler.com/bliki/FluentInterface.html)
162 | - [Fluent interfaces](http://www.erikschierboom.com/2014/10/08/fluent-interfaces/) (in C#)
163 | - [Fluent interfaces are evil](https://ocramius.github.io/blog/fluent-interfaces-are-evil/)
164 | - [smelter](https://crates.io/crates/smelter) - a similar crate for deriving builders
165 |
--------------------------------------------------------------------------------
/book/vol1/day4.md:
--------------------------------------------------------------------------------
1 | # Day 4 - docopt
2 |
3 | > Relevancy: 1.9 stable (macros only on nightly)
4 |
5 | One of the few chores when building a commandline program is argument parsing, so that `myprogram --config=myfile.conf --verbose -o output.txt` makes sense. Some arguments come in short and long variants, some are optional and some are positional only. There are a lot of libraries for argument parsing, some are even included in the respective languages' distributions. In Rust's case there's the [getopts crate](https://crates.io/crates/getopts).
6 |
7 | The first thing a moderately savvy user will do is... no, not read the documentation, but run `myprogram -h` (or `--help`) to discover available options. `getopts` and other libraries can derive such help summary for you, saving your time and reducing duplication. But what if it was the other way round? You'd write the usage message, listing possible options, and the tool would build an argument parser from that. Enter [docopt](http://docopt.org/).
8 |
9 | What is docopt?
10 | ---------------
11 |
12 | The **docopt** initiative [originated in the Python community](https://www.youtube.com/watch?v=pXhcPJK5cMc) in 2012 as an attempt to standardize common conventions for commandline arguments. The main idea is that the help message describes the interface of your program. Once you follow a few rules when writing that message, docopt can understand it and build an argument parser.
13 |
14 | docopt for Rust
15 | ---------------
16 |
17 | We'll start by declaring a dependency in `Cargo.toml`:
18 |
19 | ```ini
20 | [dependencies]
21 | docopt = "~0.6.78"
22 | docopt_macros = "~0.6.80"
23 | ```
24 |
25 | Cargo will now pull the [docopt crate](https://crates.io/crates/docopt) along with the macros which are distributed separately.
26 |
27 | **Note:** The macros use a language feature (syntax extensions) that didn't stabilize in time before 1.0 release. Therefore at present the `docopt!` macro can be used only with the nightly versions of the compiler.
28 |
29 | In our example application we'll try to mimic the `wc` tool for counting lines, words or characters in a file. Let's start with importing required libraries and writing the usage message:
30 |
31 | ```rust
32 | extern crate rustc_serialize;
33 | extern crate docopt;
34 |
35 | use docopt::Docopt;
36 |
37 | static USAGE: &'static str = "
38 | Usage: wc [options] []
39 |
40 | Options:
41 | -c, --bytes print the byte counts
42 | -m, --chars print the character counts
43 | -l, --lines print the newline counts
44 | -w, --words print the word counts
45 | -L, --max-line-length print the length of the longest line
46 | -h, --help display this help and exit
47 | -v, --version output version information and exit
48 | ";
49 | ```
50 |
51 | Nothing fancy here, just a long string. See the [docopt specification](http://docopt.org/) for the details of the format. We want to be able to decode the options into a struct, so it makes sense for the struct to implement the `Decodable` trait.
52 |
53 | ```rust
54 | #[derive(RustcDecodable)]
55 | struct Args {
56 | arg_file: Option,
57 | flag_bytes: bool,
58 | flag_chars: bool,
59 | flag_lines: bool,
60 | flag_words: bool,
61 | flag_max_line_length: bool,
62 | }
63 | ```
64 |
65 | The arguments must map to field names. Flags (toggle switches such as `-c` above) map to `flag_`-prefixed boolean fields, option arguments or meta-variables map to `arg_`-prefixed fields. If the argument is optional it can be represented by an `Option` value. But how do we actually turn the commandline args into a struct?
66 |
67 | ```rust
68 | let args: Args = Docopt::new(USAGE).and_then(|d| d.decode()).unwrap_or_else(|e| e.exit());
69 | ```
70 |
71 | Now this is a dense line of code. Let's walk through it one step at a time.
72 |
73 | ```rust
74 | let docopt = match Docopt::new(USAGE) {
75 | Ok(d) => d,
76 | Err(e) => e.exit(),
77 | };
78 | println!("{:?}", docopt);
79 | let args: Args = match docopt.decode() {
80 | Ok(args) => args,
81 | Err(e) => e.exit(),
82 | };
83 | ```
84 |
85 | `Docopt::new()` returns a `Result` value. The errors from docopt have a handy `exit()` method that prints the error message and quits the program. Printing a `Docopt` value gives us a lot of debugging information. The `decode()` method is responsible for creating our arguments object and we extract it from the `Ok` variant. We can now use `args` as any other struct in our program.
86 |
87 | [include:25-31](../../vol1/src/bin/day4.rs)
88 |
89 | docopt_macros
90 | -------------
91 |
92 | But hey, didn't I tell you earlier about reducing duplication? In the example above there are two sources of information, namely the usage string and the `Args` struct. It's quite clear that these two represent the same concept, but you'll need to mantain two separate code pieces instead of one and that is prone to errors.
93 |
94 | `docopt!` to the rescue! This is a funny (for some value of fun, of course) macro that will generate the struct for us.
95 |
96 | [include:1-18](../../vol1/src/bin/day4.rs)
97 |
98 | The macro takes the name of the type to generate, usage string and (optionally) types for the generated fields. It also validates that the usage message conforms to the docopt spec. The validation happens at compile time when the `Args` struct is generated so there's no runtime overhead. But most importantly we now have a **single** piece of information to maintain instead of two.
99 |
100 | There's one more advantage of the macro - our code inside `main()` can be simplified a bit. As the struct is generated from the usage message, we can get rid of one intermediate `Result` unwrapping; the struct has a static `docopt()` method which returns a `Docopt` value.
101 |
102 | [include:22-24](../../vol1/src/bin/day4.rs)
103 |
104 | Docopt for Rust recently gained an ability to generate tab completion files for the shell (only bash at the moment). See the [readme](https://github.com/docopt/docopt.rs#tab-completion-support) for more on that.
105 |
106 | See also
107 | --------
108 |
109 | * [using docopt in C](http://kblomqvist.github.io/2013/03/21/creating-beatiful-command-line-interfaces-for-embedded-systems-part1/) for embedded systems
110 | * docopt implementations in various languages under the [docopt organization on GitHub](https://github.com/docopt)
111 | * [wc in Rust](https://github.com/uutils/coreutils/blob/master/src/wc/wc.rs) from the coreutils rewrite project
112 |
--------------------------------------------------------------------------------