└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Algorithms for Decision Making 2 | *Mykel Kochenderfer, Tim Wheeler, and Kyle Wray* 3 | 4 | This book provides a broad introduction to algorithms for decision making under uncertainty. We cover a wide variety of topics related to decision making, introducing the underlying mathematical problem formulations and the algorithms for solving them. 5 | 6 | Please file issues for suggestions and comments (or email the address listed at the bottom of the pages of the PDF). 7 | 8 | We are interested in all forms of feedback including, but not limited to: 9 | 1. Errors 10 | 2. Improvements to code (especially improvements for clarity over speed) 11 | 3. Typos 12 | 4. Areas that are confusing 13 | 5. Critical topics that are missing 14 | 6. Ideas for examples or exercises 15 | 16 | Book webpage: https://algorithmsbook.com 17 | 18 | ## Errata 19 | 20 | ### First printing 21 | 22 | * [p. 108] Ex 5.4: Make edge E->D in the PDAG directed (thanks to Nikhil Raghuraman) 23 | * [p. 121] Alg 6.2: Caption updated to reflect that algorithm returns the value of information (thanks to Griffin Holt) 24 | * [p. 157] Ex 7.9: Change solution to "then both policies are optimal" (thanks to Liam Kruse) 25 | * [p. 174] Fig 8.13: Changed "polynomial" to "quadratic" (thanks to Balduin) 26 | * [p. 192] Ex 9.5: Change N(s0) to 1, as count is not incremented until after exploration (thanks to Paul Diederichs) 27 | * [p. 196] Ex 9.8: Change line plot colors to improve readability wrt color blindness (thanks to Zhen Wu) 28 | * [p. 199] Alg 9.9: P.S should be prepended with "π." (thanks to Griffin Holt) 29 | * [p. 226] Fix intermediate derivation in equation in solution of Ex 10.1 (thanks to Paul Diederichs) 30 | * [p. 207] Ex 9.11, for consistency, change the Gaussian to have N(s' | ...) 31 | * [p. 232] Alg 11.1: Added a note to indicate that the final state is not included in this data structure, as it is not needed in the ensuing algorithms (thanks to Joshua Ott) 32 | * [p. 246] Ex 11.2 solution: "theh" should be "the" (thanks to Marc Schlicting) 33 | * [p. 265] Ex 12.3 Solution, first equation: change 0.3/0.3 to 0.3/0.2 (thanks to Asakura publishing) 34 | * [p. 269] Alg 13.1 caption, change ∇θ to ∇Uθ and ∇θ to ∇ℓϕ (thanks to Asakura publishing) 35 | * [p. 274] Alg 13.3 caption, change ∇θ to ∇Uθ and ∇θ to ∇ℓϕ (thanks to Asakura publishing) 36 | * [p. 276] Note 6, 14th line: Change "Alpha Zero" to "AlphaGo Zero" (thanks to Asakura publishing) 37 | * [p. 278] Ex 13.2 (c)-(h): All instances of `l` should be `\ell` (thanks to Asakura publishing) 38 | * [p. 304] Ex 15.2 pinned the RNG to produce the output expected in the example description (thanks to Appoorva Dixit) 39 | * [p. 304] Ex 15.2 changed the model creation code to `model = BanditModel(fill(Beta(), 2))` 40 | * [p. 307] Ex 15.3 curve was not plotted with the correct demoninator, causing the curve to be too low (thanks to Hugo Buurmeijer) 41 | * [p. 314] Ex 15.7 values for arm 1 were fixed (thanks to Mykel) 42 | * [p. 317] Mention that when N(s,a) = 0, we can use a uniform distribution or set T(s,a) to zero (thanks to Mykel and Dylan Asmar) 43 | * [p. 329] Eq 16.8: b in second P(s'|s,b,a) should be theta (thanks to Griffin Holt) 44 | * [p. 330] Change the reference to Chapter 16 to be to Part 4 instead (thanks to Michael Sheehan) 45 | * [p. 382] Ex 19.2: change from "perfect observation" to "a more accurate observation" (thanks to Kathryn Lesh) 46 | * [p. 405] Ex 19.17 Solution, last equation: All instances of `\sigma _{p}` should be bold (thanks to Asakura publishing) 47 | * [p. 414] Ex 20.2: 3rd line: Bolding the 'b' in `b = [0.5,0.5]` (thanks to Asakura publishing) 48 | * [p. 436] Update Sec. 21.6 and implementation of utility() in Alg. 21.9 (thanks to Shengtong Zhang) 49 | * [p. 461] Ex 22.5: change O(o2 | a3, s2) to O(o2 | a3, s1) (thanks to Dylan Asmar) 50 | * [p. 467] Ex 22.1 Solution, first equation: `P (o | b, a)` should have a bold `b` (thanks to Asakura publishing) 51 | * [p. 468] Prob 22.3: update to reflect use of successor state in observation func (thanks to Dylan Asmar) 52 | * [p. 468] Ex 22.2 Solution: In the last column of the table's header row, "o" should be "o_a" (thanks to Asakura publishing) 53 | * [p. 479] Alg 23.4: change comment "prune identical from previous nodes" to "prune identical from new nodes" (thanks to Emma Passmore) 54 | * [p. 585] Eq. D.6: change - to + (thanks to Jonathan Larkin) 55 | * [p. 644] Sec. G.3.2: "x" in text should be "X" to be consistent with code (thanks to Ziyu Wang) 56 | --------------------------------------------------------------------------------