├── .gitignore ├── notes ├── 1_webppl_basics.md ├── 2_building_generative_models.md └── 3_doing_inference.md ├── older_years ├── 2017 │ ├── notes │ │ ├── 1_webppl_basics.md │ │ ├── 2_building_generative_models.md │ │ └── 3_doing_inference.md │ └── slides │ │ ├── slides.key │ │ └── slides.pdf └── 2018 │ ├── notes │ ├── 1_webppl_basics.md │ ├── 2_building_generative_models.md │ └── 3_doing_inference.md │ └── slides │ ├── slides.pdf │ └── slides.pptx ├── readme.html ├── readme.md └── slides ├── slides.pdf ├── slides.pptx └── ~$slides.pptx /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store -------------------------------------------------------------------------------- /notes/1_webppl_basics.md: -------------------------------------------------------------------------------- 1 | # WebPPL basics 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - You can open a new file and save any code you've written. 5 | - Check out [appendix-js-basics](http://probmods.org/chapters/13-appendix-js-basics.html) for other Javascript basics, including mathematical and logical operations. 6 | 7 | ## Declaring variables 8 | 9 | - Use `var` to name objects and assign a value to them. 10 | - Use display() to display results. 11 | 12 | ```javascript 13 | var strength = 2.5 // here we bind the symbol `strength` to the value 2.5 14 | display(strength) 15 | ``` 16 | 17 | - Objects are a useful data-structure in JS, similar to dictionaries in Python. 18 | - Here is how we create an object, and then refer to an object's attribute. 19 | 20 | ```javascript 21 | var kevin = { 22 | firstName: "Kevin", 23 | lastName: "Smith", 24 | strength: 2.5, 25 | lazy: false, //note the logical variable here 26 | } 27 | // Two ways of accessing the same attribute: 28 | display(kevin.lazy) 29 | display(kevin['lazy']) 30 | ``` 31 | 32 | - We can also create arrays, like so. 33 | - Note that javascript/webppl use 0 indexing. The first element of an array is indexed with 0 (not a 1). 34 | 35 | ```javascript 36 | var instructors = ["Kevin","Josh"] //an array of instructors 37 | display(instructors) //displaying the whole team 38 | display(instructors[0]) //displaying the first element of the instructor team 39 | ``` 40 | 41 | ## if-then statements 42 | 43 | - The general form is: if (condition) {...} else if (condition) {...} else {...} 44 | - Let's see whether Tobi is strong: 45 | 46 | ```javascript 47 | var kevin = { 48 | firstName: "Kevin", 49 | lastName: "Smith", 50 | strength: 2.5, 51 | lazy: false, //note the logical variable here 52 | } 53 | 54 | if (kevin.strength > 5){ 55 | display("Kevin is strong!") 56 | } else if (kevin.strength < 5) { 57 | display("Kevin is weak!") 58 | } else { 59 | display("Kevin is neither weak nor strong.") 60 | } 61 | ``` 62 | 63 | - There is also an abbreviated form of if-then for lazy coders. 64 | - Here, the general form is: `condition ? consequence : alternative` 65 | 66 | ```javascript 67 | kevin.lazy ? display("Kevin is lazy!") : display("Kevin is not lazy!") 68 | ``` 69 | 70 | ## Defining functions 71 | 72 | - We use the `function()` primitive to define functions. 73 | - The general form of a function expression is: `function(arguments){body; return ... }` 74 | - Here is an example: 75 | 76 | ```javascript 77 | var returnWinner = function(personA, personB){ 78 | if (personA.strength > personB.strength){ 79 | return personA.firstName 80 | } else { 81 | return personB.firstName 82 | } 83 | } 84 | ``` 85 | 86 | - Let's see who would win in a tug of war between Josh and Kevin. 87 | - Make sure that you have all the bits and pieces you need in the same code window. 88 | 89 | ```javascript 90 | var josh = { 91 | firstName: "Josh", 92 | lastName: "Tenenbaum", 93 | strength: 10, 94 | lazy: true, 95 | } 96 | 97 | var kevin = { 98 | firstName: "Kevin", 99 | lastName: "Smith", 100 | strength: 2.5, 101 | lazy: false, 102 | } 103 | 104 | var returnWinner = function(personA, personB){ 105 | if (personA.strength > personB.strength){ 106 | return personA.firstName 107 | } else { 108 | return personB.firstName 109 | } 110 | } 111 | 112 | display(returnWinner(kevin,josh) + " would win a tug of a war") 113 | ``` 114 | 115 | - Note that you can create functions that take no arguments. 116 | - This will be important later when our functions become stochastic. 117 | 118 | ```javascript 119 | var sayHi = function() { 120 | display("Hello!") 121 | } 122 | sayHi() 123 | ``` 124 | 125 | ## Higher order functions 126 | 127 | - Higher order functions are functions that take other functions as their input. 128 | - For example, `map` is such a higher order function. It takes another function as an argument, and applies that function to a list of arguments in turn. 129 | 130 | ```javascript 131 | var mySquare = function(x){return x*x} 132 | var someNumbers = [1,2,3,4] 133 | display(map(mySquare, someNumbers)) 134 | ``` 135 | 136 | ## Some useful keyboard shortcuts for use in [webppl.org](http://webppl.org) 137 | 138 | - `cmd+/`: comment/uncomment code 139 | - select code and press `tab`to fix indentation 140 | - `cmd+Enter`: run code 141 | 142 | ## Practice 143 | 144 | 1. Create several more objects like the `kevin` and `josh` objects from above for some of your fellow students. Assign them several properties, including strength. 145 | 146 | 163 | 164 | 2. Create a function `showStrength()` for displaying the strength of a student. 165 | 166 | 175 | 176 | 3. Map the function `showStrength()` to an array of students and instructors. 177 | 178 | 187 | 188 | 4. Amend the `returnWinner()` function to handle ties. 189 | 190 | 208 | -------------------------------------------------------------------------------- /notes/2_building_generative_models.md: -------------------------------------------------------------------------------- 1 | # Building generative models 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - Check out this chapter of the probmods book for more information: [02-generative-models](http://probmods.org/chapters/02-generative-models.html) 5 | 6 | ## Forward sampling and random primitives 7 | 8 | - `flip()` is a basic random primitive (akin to a coin-flip). 9 | - Run `flip()` a few times. 10 | - Tip: An evaluation on the last line is displayed by default (so you don't really need `display()` to show it.) 11 | 12 | ```javascript 13 | flip() 14 | ``` 15 | 16 | - When you get bored of running `flip()`, you can make use of the higher order function `repeat()`. 17 | - `repeat()` takes a number and a function as inputs, and runs the function that many times. 18 | 19 | ```javascript 20 | repeat(1000,flip) 21 | ``` 22 | 23 | - WebPPL comes with some simple visualization tools that we fire up using `viz()`. 24 | - Let's visualize flipping a biased coin with `p(true) = 0.7` 1000 times. 25 | 26 | ```javascript 27 | var biasedFlipping = function(){ 28 | return flip(0.7) 29 | } 30 | var repeatedFlipping = repeat(1000,biasedFlipping) 31 | viz(repeatedFlipping) 32 | ``` 33 | 34 | - Comprehension check: Why wouldn't `repeat(1000,flip(0.7))` work? 35 | 36 | ## Building simple models 37 | 38 | - Here is a model where we flip 3 biased coins, sum up the results, and visualize them. 39 | - Note that javascript turns `true` into `1`, and `false` into `0` for us, when we apply mathematical operations to logical variables. 40 | 41 | ```javascript 42 | var flippingAway = function(){ 43 | var a = flip(0.3) 44 | var b = flip(0.3) 45 | var c = flip(0.3) 46 | var d = a + b + c 47 | return d 48 | } 49 | viz(repeat(1000,flippingAway)) 50 | ``` 51 | 52 | ## Sample from probability distributions 53 | 54 | - WebPPL has a number of probability distributions that we can simply sample from. Below are some examples, the full list can be found here: [http://docs.webppl.org/en/master/distributions.html](http://docs.webppl.org/en/master/distributions.html) 55 | 56 | ```javascript 57 | var strengthGaussian = gaussian(50,10) //Gaussian strength 58 | var strengthExponential = exponential(1) //Exponential strength 59 | 60 | // we can easily combine distributions to make complex new ones 61 | var whatIsThisEven = function(){ 62 | Math.pow(Math.abs(gaussian(0,1)), uniform(0,1)*exponential(1)) 63 | } 64 | viz(repeat(1000,whatIsThisEven)) 65 | ``` 66 | 67 | ## Memoization 68 | 69 | - Sometimes we want to assume a probability distribution over a property or object, but we want to assure that we get the same answer whenever that property or object is queried. 70 | - For example, we might not know the color of Kevin's eyes. We can model this knowledge/uncertainty as a random distribution over different eye colors. 71 | - Run the following code and notice what's odd: 72 | 73 | ```javascript 74 | var eyeColor = function (person) { 75 | return uniformDraw(['blue', 'green', 'brown']) 76 | } 77 | display([eyeColor('Kevin'), eyeColor('Kevin'), eyeColor('Kevin')]) 78 | ``` 79 | 80 | - Each time we call the `eyeColor()` function on `'Kevin'` it draws again from a uniform distribution. 81 | - However, we want to ensure that whenever we ask for Kevin's eye color within our model, it will return the same eye color. 82 | - To do this we use the built-in procedure `mem`. A memoized procedure will sample its random values the first time it is run, but will re-use those same values on subsequent runs. 83 | - A memoized procedure will sample new values if it is passed new arguments. 84 | - Try playing with the following eye-color example: 85 | 86 | ```javascript 87 | var eyeColor = mem(function (person) { 88 | return uniformDraw(['blue', 'green', 'brown']) 89 | }) 90 | display([eyeColor('Kevin'), eyeColor('Kevin'), 91 | eyeColor('Josh'), eyeColor('Josh')]) 92 | ``` 93 | 94 | - Note that memoization requires **all** function arguments to be the same to recall its previous draw. 95 | - For instance, the eyeColor function below allows for different "Kevins" to have different eye colors: 96 | 97 | ```javascript 98 | var eyeColor = mem(function (firstName, lastName) { 99 | return uniformDraw(['blue', 'green', 'brown']) 100 | }) 101 | display([eyeColor('Kevin', 'Smith'), eyeColor('Kevin', 'Smith'), 102 | eyeColor('Kevin', 'McCallister'), eyeColor('Kevin', 'McCallister')]) 103 | ``` 104 | 105 | ## Recursion 106 | 107 | - Recursion is a very powerful technique. 108 | - For example, recursive functions are important for capturing theory-of-mind inferences such as: "Lisa thinks that Tom thinks that she thinks that he is interested in her." 109 | - A recursive function calls itself within the function call. It needs a stopping condition (which can be probabilistic). 110 | - Here is an example for how to implement a `while` loop using a recursive function. This function implements a "counting down to zero" procedure. 111 | 112 | ```javascript 113 | var countingDown = function(number){ 114 | display(number) 115 | return (number == 0) ? display('done with counting') : countingDown(number-1) 116 | } 117 | countingDown(4) 118 | ``` 119 | 120 | ## Practice 121 | 122 | 1. Create a function for returning the strength of a person. What distribution makes sense to you? Should you use memoization? 123 | 124 | 138 | 139 | 2. Suppose someone in a tug of war sometimes decides to be lazy (one third of the time) and not pull that hard. Create a function for returning the laziness of a person. Should this be memoized? 140 | 141 | 154 | 155 | 3. Create a function tugWinner. The function takes in two people, and checks their strength and laziness (using the previous functions). If one is lazy and the other is not, the non-lazy one should be returned. If both or neither are lazy, return the one with the greater strength. Note: The symbol for a logical AND is `&`, and the symbol for a logical NOT is `!`. 156 | 157 | 189 | 190 | 4. Recursion (BONUS). Try to build the following procedure: You take a coin, and flip it. If the coin returns `true` ("heads"), stop. If the coin returns `false` ("tails"), then flip the coin again. Keep going in this manner, counting up the total number of tails you see before you hit tails. 191 | 192 | - Below is the start of some code to construct this procedure, fill in the rest: 193 | 194 | ```javascript 195 | var flipTails = function(){ 196 | if (...){ 197 | return ... 198 | } else { 199 | return ... 200 | } 201 | } 202 | //viz(repeat(1000,flipTails)) //uncomment this bit to visualize your results once you've updated the function 203 | ``` 204 | 205 | 220 | -------------------------------------------------------------------------------- /notes/3_doing_inference.md: -------------------------------------------------------------------------------- 1 | # Doing inference 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - You can find a complete function reference here: [http://webppl.readthedocs.io/en/master/](http://webppl.readthedocs.io/en/master/) 5 | - Check out this chapter of the probmods book for more information: [03-conditioning](http://probmods.org/chapters/03-conditioning.html) 6 | 7 | ## Conditioning on variables 8 | 9 | ### Rejection sampling 10 | 11 | - We can use recursion to implement a rejection query. 12 | - Here we are interested in the value of `a`, conditioning on the fact that the sum of `a`, `b`, and `c` is `>= 2`. 13 | 14 | ```javascript 15 | var flippingAway = function () { 16 | var a = flip(0.3) 17 | var b = flip(0.3) 18 | var c = flip(0.3) 19 | var d = a + b + c 20 | return d >= 2 ? a : flippingAway() 21 | } 22 | viz(repeat(100, flippingAway)) 23 | ``` 24 | 25 | - Note how the `flippingAway()` function is called inside the `flippingAway()` function. The return statement is an if-else statement. If the condition is met `d >= 2`, the value of `a` is returned, otherwise the `flippingAway()` function is called again. 26 | - Comprehension check: Why would you not want to do this if you were flipping 20 coins and conditioning on at least 19 heads? 27 | 28 | ### Using WebPPL's inference procedures 29 | 30 | - To do inference, we simply add a `condition` statement to our model. 31 | - The general template is: 32 | 33 | ```javascript 34 | var model = function(){ 35 | var ... //our model description 36 | condition (...) // what we condition on 37 | return ... // what we are interested in 38 | } 39 | ``` 40 | 41 | - To run inference, we use the webppl procedure `Infer`. 42 | - `Infer` takes **options** (that define what kind of inference procedure to run), and a **function** as input. 43 | 44 | ```javascript 45 | var flippingAway = function(){ 46 | var a = flip(0.3) 47 | var b = flip(0.3) 48 | var c = flip(0.3) 49 | var d = a + b + c 50 | condition(d >= 2) //condition 51 | return a 52 | } 53 | var options = {method: 'rejection', samples: 1000} 54 | var dist = Infer(options, flippingAway) 55 | viz(dist) 56 | ``` 57 | 58 | - You can compare this posterior distribution to the prior distribution by simply changing the `condition(d >= 2)` to `condition(true)` (or by commenting out the `condition(...)` statement). 59 | 60 | ## Conditioning on arbitrary expressions 61 | 62 | - One of the powerful features of the webppl programming language is that it allows conditioning on arbitrary expressions composed of the variables in the model. 63 | - You can also return arbitrary expressions. 64 | - Here is an example: 65 | 66 | ```javascript 67 | var flippingAway = function(){ 68 | var a = flip(0.3) 69 | var b = flip(0.3) 70 | var c = flip(0.3) 71 | condition(a + b + c >= 2) //arbitrary expression 72 | return a + b //arbitrary expression 73 | } 74 | var options = {method: 'rejection', samples: 1000} 75 | var dist = Infer(options, flippingAway) 76 | viz(dist) 77 | ``` 78 | 79 | - This is one key strength since it allows us to cleanly separate out the description of the generative model, and the inference procedure. 80 | 81 | ## Other inference procedures 82 | 83 | - In the previous examples, we used rejection sampling (`method: 'rejection'`) to do inference. 84 | - WebPPL implements a number of inference procedures. You can find out more about these here: [http://webppl.readthedocs.io/en/master/inference/methods.html](http://webppl.readthedocs.io/en/master/inference/methods.html) 85 | - In general, you will **not** want to use rejection sampling. For simple models, the `MCMC` method is usually good enough to get by without the drawbacks of rejection sampling. 86 | - Here is how you can use MCMC instead of rejection sampling (note you also need the `kernel` argument for this method): 87 | 88 | ```javascript 89 | var flippingAway = function(){ 90 | var a = flip(0.3) 91 | var b = flip(0.3) 92 | var c = flip(0.3) 93 | condition(a + b + c >= 2) //arbitrary expression 94 | return a + b //arbitrary expression 95 | } 96 | var options = {method: 'MCMC', kernel: 'MH', samples: 1000} 97 | var dist = Infer(options, flippingAway) 98 | viz(dist) 99 | ``` 100 | 101 | ## Practice 102 | 103 | ### Inference in the tug of war model 104 | 105 | - You now have learned about all the bits and pieces that make up the tug of war model. 106 | - Here is the model: 107 | 108 | ```javascript 109 | var model = function(){ 110 | var strength = mem(function (person) {return gaussian(50, 10)}) 111 | var lazy = function(person) {return flip(1/3) } 112 | var pulling = function(person) { 113 | return lazy(person) ? strength(person) / 2 : strength(person) 114 | } 115 | var totalPulling = function (team) {return sum(map(pulling, team))} 116 | var winner = function (team1, team2) { 117 | totalPulling(team1) > totalPulling(team2) ? team1 : team2 118 | } 119 | var beat = function(team1,team2){winner(team1,team2) == team1} 120 | // Condition and Return statements go here ... 121 | } 122 | // Infer and visualize here ... 123 | ``` 124 | 125 | - First, make sure that you understand all the bits and pieces. 126 | - Condition on the fact that `"Tom"` beat both `"Steve"` and `"Bill"` at once, and return the strength of `"Tom"`. (Note: The `beat` function takes teams as input (i.e. two arrays of players). So even if the team only has one player, you still need to put that player into an array.) 127 | - For the inference options, please use the following: `var options = {method: 'MCMC', kernel: 'MH', samples: 10000}`. This implements a Markov Chain Monte Carlo inference using the Metropolis-Hastings algorithm. 128 | - If all goes well, the `viz` function will output a density function. You can print out the mean of the distribution by using the `expectation()` function: `display('Expected strength: ' + expectation(dist))` 129 | 130 | 156 | 157 | ### Extending the tug of war model 158 | 159 | - Extend the tug of war model so that you can ask whether a player was lazy in a particular match. 160 | - To do so, you need to amend the functions to not only feature persons (or teams) but also matches. 161 | - You need to make laziness a persistent property (using `mem`) that applies to a person in a match (but not always to the same person). 162 | - Once you've rewritten the code then try the following: 163 | + Condition on the fact that Tom beat Tim in match 1 (hint: `condition(beat(['Tom'],['Tim'],1))`), and ask for whether Tom was lazy in match 1 (and whether Tim was lazy in match 1). 164 | + How does the inference whether Tim was lazy in match1 change for the following series of matches?: 165 | * Match 1: Tim loses against Tom 166 | * Match 2: Tim wins against Steve 167 | * Match 3: Tim wins against Bill 168 | * Match 4: Tim wins against Mark 169 | * Match 5: Tim wins against Kevin 170 | * Match 6: Tim wins against Tobi 171 | + (Note: Use `&` to combine multiple pieces of evidence in the condition statement `condition()`). 172 | - You can also ask who would win in a game between Tim and Steve, based on prior information about each player. 173 | 174 | 226 | -------------------------------------------------------------------------------- /older_years/2017/notes/1_webppl_basics.md: -------------------------------------------------------------------------------- 1 | # WebPPL basics 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - You can open a new file and save any code you've written. 5 | - Check out [appendix-js-basics](http://probmods.org/chapters/13-appendix-js-basics.html) for other Javascript basics, including mathematical and logical operations. 6 | 7 | ## Declaring variables 8 | 9 | - Use `var` to name objects and assign a value to them. 10 | - Use display() to display results. 11 | 12 | ```javascript 13 | var strength = 2.5 // here we bind the symbol `strength` to the value 2.5 14 | display(strength) 15 | ``` 16 | 17 | - Objects are a useful data-structure in JS, similar to dictionaries in Python. 18 | - Here is how we create an object, and then refer to an object's attribute. 19 | 20 | ```javascript 21 | var tobi = { 22 | firstName: "Tobi", 23 | lastName: "Gerstenberg", 24 | strength: 2.5, 25 | lazy: false, //note the logical variable here 26 | } 27 | display(tobi.lazy) 28 | ``` 29 | 30 | - We can also create arrays, like so. 31 | - Note that javascript/webppl use 0 indexing. The first element of an array is indexed with 0 (not a 1). 32 | 33 | ```javascript 34 | var instructors = ["Tobi","Kevin"] //an array of instructors 35 | display(instructors) //displaying the whole team 36 | display(instructors[0]) //displaying the first element of the instructor team 37 | ``` 38 | 39 | ## if-then statements 40 | 41 | - The general form is: if (condition) {...} else if (condition) {...} else {...} 42 | - Let's see whether Tobi is strong: 43 | 44 | ```javascript 45 | var tobi = { 46 | firstName: "Tobi", 47 | lastName: "Gerstenberg", 48 | strength: 2.5, 49 | lazy: false, //note the logical variable here 50 | } 51 | 52 | if (tobi.strength > 5){ 53 | display("Tobi is strong!") 54 | } else if (tobi.strength < 5) { 55 | display("Tobi is weak!") 56 | } else { 57 | display("Tobi is neither weak nor strong.") 58 | } 59 | ``` 60 | 61 | - There is also an abbreviated form of if-then for lazy coders. 62 | - Here, the general form is: `condition ? consequence : alternative` 63 | 64 | ```javascript 65 | tobi.lazy ? display("Tobi is lazy!") : display("Tobi is not lazy!") 66 | ``` 67 | 68 | ## Defining functions 69 | 70 | - We use the `function()` primitive to define functions. 71 | - The general form of a function expression is: `function(arguments){body; return ... }` 72 | - Here is an example: 73 | 74 | ```javascript 75 | var returnWinner = function(personA, personB){ 76 | if (personA.strength > personB.strength){ 77 | return personA.firstName 78 | } else { 79 | return personB.firstName 80 | } 81 | } 82 | ``` 83 | 84 | - Let's see who would win in a tug of war between Kevin and Tobi. 85 | - Make sure that you have all the bits and pieces you need in the same code window. 86 | 87 | ```javascript 88 | var tobi = { 89 | firstName: "Tobi", 90 | lastName: "Gerstenberg", 91 | strength: 2.5, 92 | lazy: false, //note the logical variable here 93 | } 94 | 95 | var kevin = { 96 | firstName: "Kevin", 97 | lastName: "Smith", 98 | strength: 10, 99 | lazy: true, 100 | } 101 | 102 | var returnWinner = function(personA, personB){ 103 | if (personA.strength > personB.strength){ 104 | return personA.firstName 105 | } else { 106 | return personB.firstName 107 | } 108 | } 109 | 110 | display(returnWinner(kevin,tobi) + " would win a tug of a war") 111 | ``` 112 | 113 | ## Higher order functions 114 | 115 | - Higher order functions are functions that take other functions as their input. 116 | - For example, `map` is such a higher order function. It takes another function as an argument, and applies that function to a list of arguments in turn. 117 | 118 | ```javascript 119 | var mySquare = function(x){return x*x} 120 | var someNumbers = [1,2,3,4] 121 | display(map(mySquare, someNumbers)) 122 | ``` 123 | 124 | ## Some useful keyboard shortcuts for use in [webppl.org](http://webppl.org) 125 | 126 | - `cmd+/`: comment/uncomment code 127 | - select code and press `tab`to fix indentation 128 | - `cmd+Enter`: run code 129 | 130 | ## Practice 131 | 132 | 1. Create several student objects (like the instructor objects from before). Assign them several properties, including strength. 133 | 134 | 150 | 151 | 2. Create a function `showStrength()` for displaying the strength of a student. 152 | 153 | 162 | 163 | 3. Map the function `showStrength()` to an array of students and instructors. 164 | 165 | 174 | 175 | 4. Amend the `returnWinner()` function to handle ties. 176 | 177 | 195 | -------------------------------------------------------------------------------- /older_years/2017/notes/2_building_generative_models.md: -------------------------------------------------------------------------------- 1 | # Building generative models 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - Check out this chapter of the probmods book for more information: [02-generative-models](http://probmods.org/chapters/02-generative-models.html) 5 | 6 | ## Forward sampling and random primitives 7 | 8 | - `flip()` is a basic random primitive (akin to a coin-flip). 9 | - Run `flip()` a few times. 10 | - Tip: An evaluation on the last line is displayed by default (so you don't really need `display()` to show it.) 11 | 12 | ```javascript 13 | flip() 14 | ``` 15 | 16 | - When you get bored of running `flip()`, you can make use of the higher order function `repeat()`. 17 | - `repeat()` takes a number and a function as inputs, and runs the function that many times. 18 | 19 | ```javascript 20 | repeat(1000,flip) 21 | ``` 22 | 23 | - WebPPL comes with some simple visualization tools that we fire up using `viz()`. 24 | - Let's visualize flipping a biased coin with `p(true) = 0.7` 1000 times. 25 | 26 | ```javascript 27 | var biasedFlipping = function(){ 28 | flip(0.7) 29 | } 30 | var repeatedFlipping = repeat(1000,biasedFlipping) 31 | viz(repeatedFlipping) 32 | ``` 33 | 34 | - Comprehension check: Why wouldn't `repeat(1000,flip(0.7))` work? 35 | 36 | ## Building simple models 37 | 38 | - Here is a model where we flip 3 biased coins, sum up the results, and visualize them. 39 | - Note that javascript turns `true` into `1`, and `false` into `0` for us, when we apply mathematical operations to logical variables. 40 | 41 | ```javascript 42 | var flippingAway = function(){ 43 | var a = flip(0.3) 44 | var b = flip(0.3) 45 | var c = flip(0.3) 46 | var d = a + b + c 47 | return d 48 | } 49 | viz(repeat(1000,flippingAway)) 50 | ``` 51 | 52 | ## Sample from probability distributions 53 | 54 | - WebPPL has a number of probability distributions that we can simply sample from. Below are some examples, the full list can be found here: [http://docs.webppl.org/en/master/distributions.html](http://docs.webppl.org/en/master/distributions.html) 55 | 56 | ```javascript 57 | var strengthGaussian = gaussian(50,10) //Gaussian strength 58 | var strengthExponential = exponential(1) //Exponential strength 59 | 60 | // we can easily combine distributions to make complex new ones 61 | var whatIsThisEven = function(){ 62 | Math.pow(Math.abs(gaussian(0,1)), uniform(0,1)*exponential(1)) 63 | } 64 | viz(repeat(1000,whatIsThisEven)) 65 | ``` 66 | 67 | ## Memoization 68 | 69 | - Sometimes we want to assume a probability distribution over a property or object, but we want to assure that we get the same answer whenever that property or object is queried. 70 | - For example, we might not know the color of Kevin's eyes. We can model this knowledge/uncertainty as a random distribution over different eye colors. 71 | - Run the following code and notice what's odd: 72 | 73 | ```javascript 74 | var eyeColor = function (person) { 75 | return uniformDraw(['blue', 'green', 'brown']) 76 | } 77 | display([eyeColor('Kevin'), eyeColor('Kevin'), eyeColor('Kevin')]) 78 | ``` 79 | 80 | - Each time we call the `eyeColor()` function on `'Kevin'` it draws again from a uniform distribution. 81 | - However, we want to ensure that whenever we ask for Kevin's eye color within our model, it will return the same eye color. 82 | - To do this we use the built-in procedure `mem`. A memoized procedure will sample its random values the first time it is run, but will re-use those same values on subsequent runs. 83 | - A memoized procedure will sample new values if it is passed new arguments. 84 | - Try playing with the following eye-color example: 85 | 86 | ```javascript 87 | var eyeColor = mem(function (person) { 88 | return uniformDraw(['blue', 'green', 'brown']) 89 | }) 90 | display([eyeColor('Kevin'), eyeColor('Kevin'), eyeColor('Tobi'), eyeColor('Tobi')]) 91 | ``` 92 | 93 | ## Recursion 94 | 95 | - Recursion is a very powerful technique. 96 | - For example, recursive functions are important for capturing theory-of-mind inferences such as: "Lisa thinks that Tom thinks that she thinks that he is interested in her." 97 | - A recursive function calls itself within the function call. It needs a stopping condition (which can be probabilistic). 98 | - Here is an example for how to implement a `while` loop using a recursive function. This function implements a "counting down to zero" procedure. 99 | 100 | ```javascript 101 | var countingDown = function(number){ 102 | display(number) 103 | return (number == 0) ? display('done with counting') : countingDown(number-1) 104 | } 105 | countingDown(4) 106 | ``` 107 | 108 | ## Practice 109 | 110 | 1. Create a function for returning the strength of a person. What distribution makes sense to you? Should you use memoization? 111 | 112 | 126 | 127 | 2. Suppose someone in a tug of war sometimes decides to be lazy (one third of the time) and not pull that hard. Create a function for returning the laziness of a person. Should this be memoized? 128 | 129 | 142 | 143 | 3. Create a function tugWinner. The function takes in two people, and checks their strength and laziness (using the previous functions). If one is lazy and the other is not, the non-lazy one should be returned. If both or neither are lazy, return the one with the greater strength. Note: The symbol for a logical AND is `&`, and the symbol for a logical NOT is `!`. 144 | 145 | 177 | 178 | 4. Recursion (BONUS). Try to build the following procedure: You take a coin, and flip it. If the coin returns `true` ("heads"), stop. If the coin returns `false` ("tails"), then flip the coin again. Keep going in this manner, counting up the total number of tails you see before you hit tails. 179 | 180 | - Below is the start of some code to construct this procedure, fill in the rest: 181 | 182 | ```javascript 183 | var flipTails = function(){ 184 | if (...){ 185 | return ... 186 | } else { 187 | return ... 188 | } 189 | } 190 | //viz(repeat(1000,flipTails)) //uncomment this bit to visualize your results once you've updated the function 191 | ``` 192 | 193 | 208 | -------------------------------------------------------------------------------- /older_years/2017/notes/3_doing_inference.md: -------------------------------------------------------------------------------- 1 | # Doing inference 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - You can find a complete function reference here: [http://webppl.readthedocs.io/en/master/](http://webppl.readthedocs.io/en/master/) 5 | - Check out this chapter of the probmods book for more information: [03-conditioning](http://probmods.org/chapters/03-conditioning.html) 6 | 7 | ## Conditioning on variables 8 | 9 | ### Rejection sampling 10 | 11 | - We can use recursion to implement rejection query. 12 | - Here we are interested in the value of `a`, conditioning on the fact that the sum of `a`, `b`, and `c` is `>= 2`. 13 | 14 | ```javascript 15 | var flippingAway = function () { 16 | var a = flip(0.3) 17 | var b = flip(0.3) 18 | var c = flip(0.3) 19 | var d = a + b + c 20 | return d >= 2 ? a : flippingAway() 21 | } 22 | viz(repeat(100, flippingAway)) 23 | ``` 24 | 25 | - Note how the `flippingAway()` function is called inside the `flippingAway()` function. The return statement is an if-else statement. If the condition is met `d >= 2`, the value of `a` is returned, otherwise the `flippingAway()` function is called again. 26 | 27 | ### Using WebPPL's inference procedures 28 | 29 | - To do inference, we simply add a `condition` statement to our model. 30 | - The general template is: 31 | 32 | ```javascript 33 | var model = function(){ 34 | var ... //our model description 35 | condition (...) // what we condition on 36 | return ... // what we are interested in 37 | } 38 | ``` 39 | 40 | - To run inference, we use the webppl procedure `Infer`. 41 | - `Infer` takes **options** (that define what kind of inference procedure to run), and a **function** as input. 42 | 43 | ```javascript 44 | var flippingAway = function(){ 45 | var a = flip(0.3) 46 | var b = flip(0.3) 47 | var c = flip(0.3) 48 | var d = a + b + c 49 | condition(d >= 2) //condition 50 | return a 51 | } 52 | var options = {method: 'rejection', samples: 1000} 53 | var dist = Infer(options, flippingAway) 54 | viz(dist) 55 | ``` 56 | 57 | - You can compare this posterior distribution to the prior distribution by simply changing the `condition(d >= 2)` to `condition(true)` (or by commenting out the `condition(...)` statement). 58 | 59 | ## Conditioning on arbitrary expressions 60 | 61 | - One of the powerful features of the webppl programming language is that it allows conditioning on arbitrary expressions composed of the variables in the model. 62 | - You can also return arbitrary expressions. 63 | - Here is an example: 64 | 65 | ```javascript 66 | var flippingAway = function(){ 67 | var a = flip(0.3) 68 | var b = flip(0.3) 69 | var c = flip(0.3) 70 | condition(a + b + c >= 2) //arbitray expression 71 | return a + b //arbitrary expression 72 | } 73 | var options = {method: 'rejection', samples: 1000} 74 | var dist = Infer(options, flippingAway) 75 | viz(dist) 76 | ``` 77 | 78 | - This is one key strength since it allows us to cleanly separate out the description of the generative model, and the inference procedure. 79 | 80 | ## Other inference procedures 81 | 82 | - In the previous examples, we used rejection sampling (`model: 'rejection'`) to do inference. 83 | - WebPPL implements a number of inference procedures. You can find out more about these here: [http://webppl.readthedocs.io/en/master/inference/methods.html](http://webppl.readthedocs.io/en/master/inference/methods.html) 84 | 85 | ## Practice 86 | 87 | ### Inference in the tug of war model 88 | 89 | - You now have learned about all the bits and pieces that make up the tug of war model. 90 | - Here is the model: 91 | 92 | ```javascript 93 | var model = function(){ 94 | var strength = mem(function (person) {return gaussian(50, 10)}) 95 | var lazy = function(person) {return flip(1/3) } 96 | var pulling = function(person) { 97 | return lazy(person) ? strength(person) / 2 : strength(person) 98 | } 99 | var totalPulling = function (team) {return sum(map(pulling, team))} 100 | var winner = function (team1, team2) { 101 | totalPulling(team1) > totalPulling(team2) ? team1 : team2 102 | } 103 | var beat = function(team1,team2){winner(team1,team2) == team1} 104 | // Condition and Return statements go here ... 105 | } 106 | // Infer and visualize here ... 107 | ``` 108 | 109 | - First, make sure that you understand all the bits and pieces. 110 | - Condition on the fact that `"Tom"` beat `"Bill"`, and return the strength of `"Tom"`. (Note: The `beat` function takes teams as input (i.e. two arrays of players). So even if the team only has one player, you still need to put that player into an array.) 111 | - For the inference options, please use the following: `var options = {method: 'MCMC', kernel: 'MH', samples: 10000}`. This implements a Markov Chain Monte Carlo inference using the Metropolis-Hastings algorithm. 112 | - If all goes well, the `viz` function will output a density function. You can print out the mean of the distribution by using the `expectation()` function: `display('Expected strength: ' + expectation(dist))` 113 | 114 | 140 | 141 | ### Extending the tug of war model 142 | 143 | - Extend the tug of war model so that you can ask whether a player was lazy in a particular match. 144 | - To do so, you need to amend the functions to not only feature persons (or teams) but also matches. 145 | - You need to make laziness a persistent property (using `mem`) that applies to a person in a match. 146 | - Once you've rewritten the code then try the following: 147 | + Condition on the fact that Tom beat Tim in match 1 (hint: `condition(beat(['Tom'],['Tim'],1))`), and ask for whether Tom was lazy in match 1 (and whether Tim was lazy in match 1). 148 | + How does the inference whether Tim was lazy in match1 change for the following series of matches?: 149 | * Match 1: Tim loses against Tom 150 | * Match 2: Tim wins against Steve 151 | * Match 3: Tim wins against Bill 152 | * Match 4: Tim wins against Mark 153 | * Match 5: Tim wins against Kevin 154 | * Match 6: Tim wins against Tobi 155 | + (Note: Use `&` to combine multiple pieces of evidence in the condition statement `condition()`). 156 | - You can also ask who would win in a game between Tim and Steve, based on prior information about each player. 157 | 158 | 210 | -------------------------------------------------------------------------------- /older_years/2017/slides/slides.key: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tobiasgerstenberg/webppl_tutorial/15f5ed08cfb85418d3ae27105e482d6e61318149/older_years/2017/slides/slides.key -------------------------------------------------------------------------------- /older_years/2017/slides/slides.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tobiasgerstenberg/webppl_tutorial/15f5ed08cfb85418d3ae27105e482d6e61318149/older_years/2017/slides/slides.pdf -------------------------------------------------------------------------------- /older_years/2018/notes/1_webppl_basics.md: -------------------------------------------------------------------------------- 1 | # WebPPL basics 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - You can open a new file and save any code you've written. 5 | - Check out [appendix-js-basics](http://probmods.org/chapters/13-appendix-js-basics.html) for other Javascript basics, including mathematical and logical operations. 6 | 7 | ## Declaring variables 8 | 9 | - Use `var` to name objects and assign a value to them. 10 | - Use display() to display results. 11 | 12 | ```javascript 13 | var strength = 2.5 // here we bind the symbol `strength` to the value 2.5 14 | display(strength) 15 | ``` 16 | 17 | - Objects are a useful data-structure in JS, similar to dictionaries in Python. 18 | - Here is how we create an object, and then refer to an object's attribute. 19 | 20 | ```javascript 21 | var kevin = { 22 | firstName: "Kevin", 23 | lastName: "Smith", 24 | strength: 2.5, 25 | lazy: false, //note the logical variable here 26 | } 27 | // Two ways of accessing the same attribute: 28 | display(kevin.lazy) 29 | display(kevin['lazy']) 30 | ``` 31 | 32 | - We can also create arrays, like so. 33 | - Note that javascript/webppl use 0 indexing. The first element of an array is indexed with 0 (not a 1). 34 | 35 | ```javascript 36 | var instructors = ["Kelsey","Kevin"] //an array of instructors 37 | display(instructors) //displaying the whole team 38 | display(instructors[0]) //displaying the first element of the instructor team 39 | ``` 40 | 41 | ## if-then statements 42 | 43 | - The general form is: if (condition) {...} else if (condition) {...} else {...} 44 | - Let's see whether Tobi is strong: 45 | 46 | ```javascript 47 | var kevin = { 48 | firstName: "Kevin", 49 | lastName: "Smith", 50 | strength: 2.5, 51 | lazy: false, //note the logical variable here 52 | } 53 | 54 | if (kevin.strength > 5){ 55 | display("Kevin is strong!") 56 | } else if (kevin.strength < 5) { 57 | display("Kevin is weak!") 58 | } else { 59 | display("Kevin is neither weak nor strong.") 60 | } 61 | ``` 62 | 63 | - There is also an abbreviated form of if-then for lazy coders. 64 | - Here, the general form is: `condition ? consequence : alternative` 65 | 66 | ```javascript 67 | kevin.lazy ? display("Kevin is lazy!") : display("Kevin is not lazy!") 68 | ``` 69 | 70 | ## Defining functions 71 | 72 | - We use the `function()` primitive to define functions. 73 | - The general form of a function expression is: `function(arguments){body; return ... }` 74 | - Here is an example: 75 | 76 | ```javascript 77 | var returnWinner = function(personA, personB){ 78 | if (personA.strength > personB.strength){ 79 | return personA.firstName 80 | } else { 81 | return personB.firstName 82 | } 83 | } 84 | ``` 85 | 86 | - Let's see who would win in a tug of war between Kelsey and Kevin. 87 | - Make sure that you have all the bits and pieces you need in the same code window. 88 | 89 | ```javascript 90 | var kelsey = { 91 | firstName: "Kelsey", 92 | lastName: "Allen", 93 | strength: 10, 94 | lazy: true, //note the logical variable here 95 | } 96 | 97 | var kevin = { 98 | firstName: "Kevin", 99 | lastName: "Smith", 100 | strength: 2.5, 101 | lazy: false, 102 | } 103 | 104 | var returnWinner = function(personA, personB){ 105 | if (personA.strength > personB.strength){ 106 | return personA.firstName 107 | } else { 108 | return personB.firstName 109 | } 110 | } 111 | 112 | display(returnWinner(kevin,kelsey) + " would win a tug of a war") 113 | ``` 114 | 115 | - Note that you can create functions that take no arguments. 116 | - This will be important later when our functions become stochastic. 117 | 118 | ```javascript 119 | var sayHi = function() { 120 | display("Hello!") 121 | } 122 | sayHi() 123 | ``` 124 | 125 | ## Higher order functions 126 | 127 | - Higher order functions are functions that take other functions as their input. 128 | - For example, `map` is such a higher order function. It takes another function as an argument, and applies that function to a list of arguments in turn. 129 | 130 | ```javascript 131 | var mySquare = function(x){return x*x} 132 | var someNumbers = [1,2,3,4] 133 | display(map(mySquare, someNumbers)) 134 | ``` 135 | 136 | ## Some useful keyboard shortcuts for use in [webppl.org](http://webppl.org) 137 | 138 | - `cmd+/`: comment/uncomment code 139 | - select code and press `tab`to fix indentation 140 | - `cmd+Enter`: run code 141 | 142 | ## Practice 143 | 144 | 1. Create several more objects like the `kevin` and `kelsey` objects from above for some of your fellow students. Assign them several properties, including strength. 145 | 146 | 166 | 167 | 2. Create a function `showStrength()` for displaying the strength of a student. 168 | 169 | 178 | 179 | 3. Map the function `showStrength()` to an array of students and instructors. 180 | 181 | 190 | 191 | 4. Amend the `returnWinner()` function to handle ties. 192 | 193 | 211 | -------------------------------------------------------------------------------- /older_years/2018/notes/2_building_generative_models.md: -------------------------------------------------------------------------------- 1 | # Building generative models 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - Check out this chapter of the probmods book for more information: [02-generative-models](http://probmods.org/chapters/02-generative-models.html) 5 | 6 | ## Forward sampling and random primitives 7 | 8 | - `flip()` is a basic random primitive (akin to a coin-flip). 9 | - Run `flip()` a few times. 10 | - Tip: An evaluation on the last line is displayed by default (so you don't really need `display()` to show it.) 11 | 12 | ```javascript 13 | flip() 14 | ``` 15 | 16 | - When you get bored of running `flip()`, you can make use of the higher order function `repeat()`. 17 | - `repeat()` takes a number and a function as inputs, and runs the function that many times. 18 | 19 | ```javascript 20 | repeat(1000,flip) 21 | ``` 22 | 23 | - WebPPL comes with some simple visualization tools that we fire up using `viz()`. 24 | - Let's visualize flipping a biased coin with `p(true) = 0.7` 1000 times. 25 | 26 | ```javascript 27 | var biasedFlipping = function(){ 28 | return flip(0.7) 29 | } 30 | var repeatedFlipping = repeat(1000,biasedFlipping) 31 | viz(repeatedFlipping) 32 | ``` 33 | 34 | - Comprehension check: Why wouldn't `repeat(1000,flip(0.7))` work? 35 | 36 | ## Building simple models 37 | 38 | - Here is a model where we flip 3 biased coins, sum up the results, and visualize them. 39 | - Note that javascript turns `true` into `1`, and `false` into `0` for us, when we apply mathematical operations to logical variables. 40 | 41 | ```javascript 42 | var flippingAway = function(){ 43 | var a = flip(0.3) 44 | var b = flip(0.3) 45 | var c = flip(0.3) 46 | var d = a + b + c 47 | return d 48 | } 49 | viz(repeat(1000,flippingAway)) 50 | ``` 51 | 52 | ## Sample from probability distributions 53 | 54 | - WebPPL has a number of probability distributions that we can simply sample from. Below are some examples, the full list can be found here: [http://docs.webppl.org/en/master/distributions.html](http://docs.webppl.org/en/master/distributions.html) 55 | 56 | ```javascript 57 | var strengthGaussian = gaussian(50,10) //Gaussian strength 58 | var strengthExponential = exponential(1) //Exponential strength 59 | 60 | // we can easily combine distributions to make complex new ones 61 | var whatIsThisEven = function(){ 62 | Math.pow(Math.abs(gaussian(0,1)), uniform(0,1)*exponential(1)) 63 | } 64 | viz(repeat(1000,whatIsThisEven)) 65 | ``` 66 | 67 | ## Memoization 68 | 69 | - Sometimes we want to assume a probability distribution over a property or object, but we want to assure that we get the same answer whenever that property or object is queried. 70 | - For example, we might not know the color of Kevin's eyes. We can model this knowledge/uncertainty as a random distribution over different eye colors. 71 | - Run the following code and notice what's odd: 72 | 73 | ```javascript 74 | var eyeColor = function (person) { 75 | return uniformDraw(['blue', 'green', 'brown']) 76 | } 77 | display([eyeColor('Kevin'), eyeColor('Kevin'), eyeColor('Kevin')]) 78 | ``` 79 | 80 | - Each time we call the `eyeColor()` function on `'Kevin'` it draws again from a uniform distribution. 81 | - However, we want to ensure that whenever we ask for Kevin's eye color within our model, it will return the same eye color. 82 | - To do this we use the built-in procedure `mem`. A memoized procedure will sample its random values the first time it is run, but will re-use those same values on subsequent runs. 83 | - A memoized procedure will sample new values if it is passed new arguments. 84 | - Try playing with the following eye-color example: 85 | 86 | ```javascript 87 | var eyeColor = mem(function (person) { 88 | return uniformDraw(['blue', 'green', 'brown']) 89 | }) 90 | display([eyeColor('Kevin'), eyeColor('Kevin'), eyeColor('Kelsey'), eyeColor('Kelsey')]) 91 | ``` 92 | 93 | ## Recursion 94 | 95 | - Recursion is a very powerful technique. 96 | - For example, recursive functions are important for capturing theory-of-mind inferences such as: "Lisa thinks that Tom thinks that she thinks that he is interested in her." 97 | - A recursive function calls itself within the function call. It needs a stopping condition (which can be probabilistic). 98 | - Here is an example for how to implement a `while` loop using a recursive function. This function implements a "counting down to zero" procedure. 99 | 100 | ```javascript 101 | var countingDown = function(number){ 102 | display(number) 103 | return (number == 0) ? display('done with counting') : countingDown(number-1) 104 | } 105 | countingDown(4) 106 | ``` 107 | 108 | ## Practice 109 | 110 | 1. Create a function for returning the strength of a person. What distribution makes sense to you? Should you use memoization? 111 | 112 | 126 | 127 | 2. Suppose someone in a tug of war sometimes decides to be lazy (one third of the time) and not pull that hard. Create a function for returning the laziness of a person. Should this be memoized? 128 | 129 | 142 | 143 | 3. Create a function tugWinner. The function takes in two people, and checks their strength and laziness (using the previous functions). If one is lazy and the other is not, the non-lazy one should be returned. If both or neither are lazy, return the one with the greater strength. Note: The symbol for a logical AND is `&`, and the symbol for a logical NOT is `!`. 144 | 145 | 177 | 178 | 4. Recursion (BONUS). Try to build the following procedure: You take a coin, and flip it. If the coin returns `true` ("heads"), stop. If the coin returns `false` ("tails"), then flip the coin again. Keep going in this manner, counting up the total number of tails you see before you hit tails. 179 | 180 | - Below is the start of some code to construct this procedure, fill in the rest: 181 | 182 | ```javascript 183 | var flipTails = function(){ 184 | if (...){ 185 | return ... 186 | } else { 187 | return ... 188 | } 189 | } 190 | //viz(repeat(1000,flipTails)) //uncomment this bit to visualize your results once you've updated the function 191 | ``` 192 | 193 | 208 | -------------------------------------------------------------------------------- /older_years/2018/notes/3_doing_inference.md: -------------------------------------------------------------------------------- 1 | # Doing inference 2 | 3 | - Test the different code snippets by navigating here: [webppl.org](http://webppl.org) 4 | - You can find a complete function reference here: [http://webppl.readthedocs.io/en/master/](http://webppl.readthedocs.io/en/master/) 5 | - Check out this chapter of the probmods book for more information: [03-conditioning](http://probmods.org/chapters/03-conditioning.html) 6 | 7 | ## Conditioning on variables 8 | 9 | ### Rejection sampling 10 | 11 | - We can use recursion to implement rejection query. 12 | - Here we are interested in the value of `a`, conditioning on the fact that the sum of `a`, `b`, and `c` is `>= 2`. 13 | 14 | ```javascript 15 | var flippingAway = function () { 16 | var a = flip(0.3) 17 | var b = flip(0.3) 18 | var c = flip(0.3) 19 | var d = a + b + c 20 | return d >= 2 ? a : flippingAway() 21 | } 22 | viz(repeat(100, flippingAway)) 23 | ``` 24 | 25 | - Note how the `flippingAway()` function is called inside the `flippingAway()` function. The return statement is an if-else statement. If the condition is met `d >= 2`, the value of `a` is returned, otherwise the `flippingAway()` function is called again. 26 | - Comprehension check: Why would you not want to do this if you were flipping 20 coins and conditioning on at least 19 heads? 27 | 28 | ### Using WebPPL's inference procedures 29 | 30 | - To do inference, we simply add a `condition` statement to our model. 31 | - The general template is: 32 | 33 | ```javascript 34 | var model = function(){ 35 | var ... //our model description 36 | condition (...) // what we condition on 37 | return ... // what we are interested in 38 | } 39 | ``` 40 | 41 | - To run inference, we use the webppl procedure `Infer`. 42 | - `Infer` takes **options** (that define what kind of inference procedure to run), and a **function** as input. 43 | 44 | ```javascript 45 | var flippingAway = function(){ 46 | var a = flip(0.3) 47 | var b = flip(0.3) 48 | var c = flip(0.3) 49 | var d = a + b + c 50 | condition(d >= 2) //condition 51 | return a 52 | } 53 | var options = {method: 'rejection', samples: 1000} 54 | var dist = Infer(options, flippingAway) 55 | viz(dist) 56 | ``` 57 | 58 | - You can compare this posterior distribution to the prior distribution by simply changing the `condition(d >= 2)` to `condition(true)` (or by commenting out the `condition(...)` statement). 59 | 60 | ## Conditioning on arbitrary expressions 61 | 62 | - One of the powerful features of the webppl programming language is that it allows conditioning on arbitrary expressions composed of the variables in the model. 63 | - You can also return arbitrary expressions. 64 | - Here is an example: 65 | 66 | ```javascript 67 | var flippingAway = function(){ 68 | var a = flip(0.3) 69 | var b = flip(0.3) 70 | var c = flip(0.3) 71 | condition(a + b + c >= 2) //arbitrary expression 72 | return a + b //arbitrary expression 73 | } 74 | var options = {method: 'rejection', samples: 1000} 75 | var dist = Infer(options, flippingAway) 76 | viz(dist) 77 | ``` 78 | 79 | - This is one key strength since it allows us to cleanly separate out the description of the generative model, and the inference procedure. 80 | 81 | ## Other inference procedures 82 | 83 | - In the previous examples, we used rejection sampling (`model: 'rejection'`) to do inference. 84 | - WebPPL implements a number of inference procedures. You can find out more about these here: [http://webppl.readthedocs.io/en/master/inference/methods.html](http://webppl.readthedocs.io/en/master/inference/methods.html) 85 | - In general, you will **not** want to use rejection sampling. For simple models, the `MCMC` method is usually good enough to get by without the drawbacks of rejection sampling. 86 | 87 | ## Practice 88 | 89 | ### Inference in the tug of war model 90 | 91 | - You now have learned about all the bits and pieces that make up the tug of war model. 92 | - Here is the model: 93 | 94 | ```javascript 95 | var model = function(){ 96 | var strength = mem(function (person) {return gaussian(50, 10)}) 97 | var lazy = function(person) {return flip(1/3) } 98 | var pulling = function(person) { 99 | return lazy(person) ? strength(person) / 2 : strength(person) 100 | } 101 | var totalPulling = function (team) {return sum(map(pulling, team))} 102 | var winner = function (team1, team2) { 103 | totalPulling(team1) > totalPulling(team2) ? team1 : team2 104 | } 105 | var beat = function(team1,team2){winner(team1,team2) == team1} 106 | // Condition and Return statements go here ... 107 | } 108 | // Infer and visualize here ... 109 | ``` 110 | 111 | - First, make sure that you understand all the bits and pieces. 112 | - Condition on the fact that `"Tom"` beat `"Bill"`, and return the strength of `"Tom"`. (Note: The `beat` function takes teams as input (i.e. two arrays of players). So even if the team only has one player, you still need to put that player into an array.) 113 | - For the inference options, please use the following: `var options = {method: 'MCMC', kernel: 'MH', samples: 10000}`. This implements a Markov Chain Monte Carlo inference using the Metropolis-Hastings algorithm. 114 | - If all goes well, the `viz` function will output a density function. You can print out the mean of the distribution by using the `expectation()` function: `display('Expected strength: ' + expectation(dist))` 115 | 116 | 142 | 143 | ### Extending the tug of war model 144 | 145 | - Extend the tug of war model so that you can ask whether a player was lazy in a particular match. 146 | - To do so, you need to amend the functions to not only feature persons (or teams) but also matches. 147 | - You need to make laziness a persistent property (using `mem`) that applies to a person in a match. 148 | - Once you've rewritten the code then try the following: 149 | + Condition on the fact that Tom beat Tim in match 1 (hint: `condition(beat(['Tom'],['Tim'],1))`), and ask for whether Tom was lazy in match 1 (and whether Tim was lazy in match 1). 150 | + How does the inference whether Tim was lazy in match1 change for the following series of matches?: 151 | * Match 1: Tim loses against Tom 152 | * Match 2: Tim wins against Steve 153 | * Match 3: Tim wins against Bill 154 | * Match 4: Tim wins against Mark 155 | * Match 5: Tim wins against Kevin 156 | * Match 6: Tim wins against Tobi 157 | + (Note: Use `&` to combine multiple pieces of evidence in the condition statement `condition()`). 158 | - You can also ask who would win in a game between Tim and Steve, based on prior information about each player. 159 | 160 | 212 | -------------------------------------------------------------------------------- /older_years/2018/slides/slides.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tobiasgerstenberg/webppl_tutorial/15f5ed08cfb85418d3ae27105e482d6e61318149/older_years/2018/slides/slides.pdf -------------------------------------------------------------------------------- /older_years/2018/slides/slides.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tobiasgerstenberg/webppl_tutorial/15f5ed08cfb85418d3ae27105e482d6e61318149/older_years/2018/slides/slides.pptx -------------------------------------------------------------------------------- /readme.html: -------------------------------------------------------------------------------- 1 | 1018 | 1043 | readme

Readme

1044 |

This repository contains materials for the tutorial on WebPPL – a probabilistic programming language embedded in Javascript (see webppl.org for more information).

1045 |

A web book on Probabilistic Models of Cognition written by Noah D. Goodman and Joshua B. Tenenbaum is available here probmods.org

1046 |

Another great online resource is agentmodels.org which illustrates how WebPPL can be used to model decision-making, planning, inference, etc.

1047 |

Please contact Tobias Gerstenberg (tger@mit.edu) in case you have any questions.

1048 |

Slides

1049 | 1052 |

Notes

1053 |

1_webppl_basics

1054 | 1057 |

2_building_generative_models

1058 | 1061 |

3_doing_inference

1062 | 1065 |

Additional recources

1066 |
-------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # Readme 2 | 3 | This repository contains materials for the tutorial on WebPPL -- a probabilistic programming language embedded in Javascript (see [webppl.org](http://webppl.org) for more information) 4 | 5 | A web book on Probabilistic Models of Cognition written by Noah D. Goodman and Joshua B. Tenenbaum is available here [probmods.org](http://probmods.org) 6 | 7 | Another great online resource is [agentmodels.org](http://www.agentmodels.org) which illustrates how WebPPL can be used to model decision-making, planning, inference, etc. 8 | 9 | Please contact Tobias Gerstenberg ([gerstenberg@stanford.edu](mailto:gerstenberg@stanford.edu)) or Kevin Smith ([k2smith@mit.edu](mailto:k2smith@mit.edu)) in case you have any questions. 10 | 11 | ## Slides 12 | 13 | - Presentation slides in .key and .pdf format 14 | 15 | ## Notes 16 | 17 | ### 1_webppl_basics 18 | 19 | - declaring variables; basic operations; if-then statements; writing functions 20 | 21 | ### 2_building_generative_models 22 | 23 | - building simple models; random primitives; recursion; memoization 24 | 25 | ### 3_doing_inference 26 | 27 | - conditioning on variables; conditioning on arbitrary expression; rejection query, enumeration query and other kinds of inference 28 | 29 | ## Additional recources 30 | 31 | - [http://webppl.readthedocs.io/en/master/](http://webppl.readthedocs.io/en/master/) Function reference for WebPPL. 32 | - [http://forestdb.org/](http://forestdb.org/) Repository for generative models. 33 | - [http://dippl.org/](http://dippl.org/) Additional information about WebPPl. 34 | - [http://probabilistic-programming.org/wiki/Home](http://probabilistic-programming.org/wiki/Home) Information about different probabilistic programming languages. 35 | - [https://lodash.com/docs/4.17.4](https://lodash.com/docs/4.17.4) Function reference. 36 | -------------------------------------------------------------------------------- /slides/slides.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tobiasgerstenberg/webppl_tutorial/15f5ed08cfb85418d3ae27105e482d6e61318149/slides/slides.pdf -------------------------------------------------------------------------------- /slides/slides.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tobiasgerstenberg/webppl_tutorial/15f5ed08cfb85418d3ae27105e482d6e61318149/slides/slides.pptx -------------------------------------------------------------------------------- /slides/~$slides.pptx: -------------------------------------------------------------------------------- 1 | Microsoft Office User Microsoft Office User --------------------------------------------------------------------------------