├── 2015_03_11_GAM_in_JAGS ├── .Rhistory ├── jagam.bug ├── spatial_gam_poisson.Rmd ├── spatial_gam_poisson.html ├── spatial_gam_poisson.md └── spatial_gam_poisson_files │ └── figure-html │ ├── unnamed-chunk-10-1.png │ ├── unnamed-chunk-11-1.png │ ├── unnamed-chunk-2-1.png │ ├── unnamed-chunk-4-1.png │ └── unnamed-chunk-6-1.png ├── 2015_03_12_R_ms_template ├── .Rhistory ├── intro.md ├── manuscript_template.Rmd ├── manuscript_template.html ├── manuscript_template.pdf ├── manuscript_template.tex ├── manuscript_template_files │ └── figure-latex │ │ └── carDataPlot-1.pdf ├── mee.csl └── references.bib ├── 2015_04_15_The_age_of_postdoc ├── the_age_of_postdoc.html ├── the_age_of_postdoc.md └── the_age_of_postdoc.md~ ├── 2015_05_13_Survival_analysis ├── .Rhistory ├── ext_model.txt ├── survival_analysis.Rmd ├── survival_analysis.html ├── survival_analysis.pdf ├── survival_cancer.txt ├── survival_exp.txt └── survival_roaches.txt ├── 2015_05_18_Formats_in_ecological_journals ├── formats_in_ecological_jounrals.html └── formats_in_ecological_jounrals.md ├── 2015_06_01_Math_language_extinctions_climate ├── math_and_climate.html ├── math_and_climate.md ├── math_and_climate.pdf └── math_and_climate.tex ├── 2015_06_10_Coding_conference_posters ├── .keil_poster.tex.swp ├── Figure_1.pdf ├── Figure_2.pdf ├── Figure_3.pdf ├── Figure_5.pdf ├── coding_conference_posters.md ├── eu.jpg ├── fp7.jpg ├── keil_poster.aux ├── keil_poster.dvi ├── keil_poster.log ├── keil_poster.out ├── keil_poster.pdf ├── keil_poster.synctex.gz ├── keil_poster.tex ├── merged.jpg ├── pdf_merging_command.txt ├── poster1.pdf ├── poster2.pdf ├── poster3.pdf └── reef.jpg ├── 2015_06_26_The_Sixth_extinction_review ├── The_sixth_extinction_review.html ├── The_sixth_extinction_review.md └── The_sixth_extinction_review.pdf ├── 2015_08_16_Djokovic ├── .Rhistory ├── cannabis.jpg ├── djokovic (xps's conflicted copy 2015-08-17).Rmd ├── djokovic.Rmd ├── djokovic.html ├── skunk.jpg ├── skunkmap.jpg └── venue.jpg ├── 2015_10_02_Light_diversity ├── Light_diversity.html └── light_diversity ├── 2015_11_21_Academia_and_Media ├── academia_and_media.html ├── academia_and_media.md └── media.jpg ├── 2016_01_07_Kery_book_review ├── .Rhistory ├── kery_book_cover.jpeg └── kery_modelling_review.odt ├── 2016_03_02_Natural_history_vs_statistics ├── .Rhistory ├── nat.html ├── natural_history_vs_statistics.aux ├── natural_history_vs_statistics.log ├── natural_history_vs_statistics.md~ ├── natural_history_vs_statistics.out ├── natural_history_vs_statistics.pdf ├── natural_history_vs_statistics.synctex.gz └── natural_history_vs_statistics.tex ├── 2016_07_05_Log_scales ├── .Rhistory ├── log_scale.Rmd ├── log_scale.html ├── log_scale.r └── loglogplot.r ├── 2016_07_26_R_art_with_spatstat ├── .RData ├── .Rhistory ├── Figure_1A.png ├── Figure_1A_thumb.png ├── Figure_1B.png ├── Figure_1B_thumb.png ├── Figure_2A.png ├── Figure_2A_thumb.png ├── Figure_2B.png ├── Figure_2B_thumb.png ├── R_art.Rmd └── R_art.html ├── 2016_08_23_Long-term_horizon_for_science_readability ├── ape_planet_PK.png └── long_term_horizon.odt ├── 2017_01_14_Post-IBS_impressions └── post-ibs_impressions.odt ├── 2017_01_30_facts └── facts.odt ├── 2017_03_03_Single_observation_and_significance ├── .RData ├── .Rhistory ├── anova_and_a_single_point_N.Rmd ├── anova_and_a_single_point_N.html ├── anova_and_a_single_point_N.md ├── anova_and_a_single_point_N_files │ └── figure-html │ │ ├── unnamed-chunk-11-1.png │ │ ├── unnamed-chunk-12-1.png │ │ ├── unnamed-chunk-2-1.png │ │ ├── unnamed-chunk-7-1.png │ │ └── unnamed-chunk-8-1.png ├── figure │ ├── snake.png │ ├── unnamed-chunk-1.png │ ├── unnamed-chunk-10.png │ ├── unnamed-chunk-2.png │ ├── unnamed-chunk-61.png │ ├── unnamed-chunk-62.png │ ├── unnamed-chunk-91.png │ └── unnamed-chunk-92.png ├── fixed_anova.txt └── fixed_anova_relaxed.txt ├── 2017_05_11_Rhino_wars └── 2017_03_27_Rhinos.odt └── 2017_08_15_Web_scraping ├── .Rhistory ├── big_data_bw.jpg ├── firefox_arrow.png ├── web_scraping.Rmd └── web_scraping.html /2015_03_11_GAM_in_JAGS/.Rhistory: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_11_GAM_in_JAGS/.Rhistory -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/jagam.bug: -------------------------------------------------------------------------------- 1 | model { 2 | eta <- X %*% b ## linear predictor 3 | for (i in 1:n) { mu[i] <- exp(eta[i]) } ## expected response 4 | for (i in 1:n) { y[i] ~ dpois(mu[i]) } ## response 5 | ## Parameteric effect priors CHECK tau is appropriate! 6 | for (i in 1:1) { b[i] ~ dnorm(0,0.001) } 7 | ## prior for s(x,y)... 8 | K1 <- S1[1:29,1:29] * lambda[1] + S1[1:29,30:58] * lambda[2] 9 | b[2:30] ~ dmnorm(zero[2:30],K1) 10 | ## smoothing parameter priors CHECK... 11 | for (i in 1:2) { 12 | lambda[i] ~ dgamma(.05,.005) 13 | rho[i] <- log(lambda[i]) 14 | } 15 | } -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "GAM splines now easy in JAGS. An example on 2D spatial data" 3 | author: "Petr Keil" 4 | date: "03/02/2015" 5 | output: 6 | html_document: 7 | fig_caption: yes 8 | highlight: pygments 9 | keep_md: yes 10 | --- 11 | 12 | Last week I met Simon Wood, creator of `mgcv` package, which is THE tool for 13 | fitting Generalized Additive Models (GAM) in R. 14 | 15 | Simon brought my attention to function `jagam` which he has just 16 | added to `mgcv`. The function allows to transform the 'spline' or 'smooth' 17 | component of GAM model formula into BUGS code, meaning that the flexibility of 18 | GAMs is now available for routine MCMC model fitting. 19 | 20 | I often deal with geographically structured (spatially explicit) data, and 21 | so I am excited by the prospect of using `jagam` to build spatially explicit 22 | hierarchical models. 23 | 24 | # Why should spatial statisticians care about `jagam`? 25 | 26 | Because fitting of spatial splines in JAGS and OpenBUGS has so far been a pain. 27 | Yet from my experience and from personal communication with others 28 | (e.g. C. Dormann, B. O'Hara, C. Beale, S. Wood), **splines are a well-behaved 29 | way to model spatial autocorrelation** that can be easily examined and 30 | visualized separatedly from the rest of the model. 31 | 32 | Splines can also be a handy alternative to the popular Conditional Autoregressive 33 | Models (CAR) that are available in OpenBUGS, but unavailable in JAGS. Hence, 34 | the clarity and portability of JAGS is now available to spatial modellers. 35 | 36 | # The aim of this post 37 | 38 | Here I will demonstrate the `jagam` function in action. I will fit a simple Poisson GAM (with X and Y coordinates as predictors) to spatially explicit count data. 39 | I will also check if I get the same expected values 40 | from `mgcv` and `JAGS`, given the same model. 41 | 42 | These are the packages that I will need: 43 | 44 | ```{r, message=FALSE, warning=FALSE} 45 | library(mgcv) # fits GAMs 46 | library(spatstat) # the source of the example data 47 | library(raster) # for operations with rasters 48 | library(R2jags) # interface between R and JAGS 49 | ``` 50 | 51 | # The data 52 | 53 | I will use example dataset `bei` from `spatstat` package. The data 54 | are positions of 3605 individual trees of *Beilschmiedia pendula* (Lauraceae) in 55 | a 1000 by 500 metre rectangular sampling region in the tropical rainforest of 56 | Barro Colorado Island. The data are stored in a point process pattern `ppp` 57 | object. 58 | 59 | Let's plot the data: 60 | 61 | ```{r, fig.width=8.3, fig.height=5, message=FALSE, warning=FALSE} 62 | par(mai=c(0.5,0.3,0.3,0)) 63 | plot(bei, cex=0.1, main=NULL) 64 | ``` 65 | 66 | I will fit the data into a raster of 25 x 50 grid cells; each grid cell gives 67 | the count of individual trees that fall within the cell: 68 | 69 | ```{r} 70 | # cropping the data so that they have exactly 500 x 1000 cells 71 | ext <- extent(0, 1000, 0, 500) # spatial extent of the raster 72 | empty <- raster(ext, nrow=25, ncol=50) # empty raster 73 | 74 | # aggregating the point data into the raster 75 | xy <- data.frame(x = bei$x, y = bei$y) 76 | rst <- rasterize(xy, empty, fun = "count") 77 | 78 | # replacing the NA values by 0 79 | rst[is.na(rst)] <- 0 80 | 81 | # extracting the cell values and their coordinates to a data.frame 82 | coord <- xyFromCell(rst,1:ncell(rst)) 83 | count <- extract(rst, 1:ncell(rst)) 84 | all.data <- data.frame(coord, count=count) 85 | ``` 86 | 87 | This is the resulting rasterized dataset, with point locations of the trees 88 | plotted on top. The color gradient shows counts of trees in each grid cell. 89 | 90 | ```{r, fig.width=8.3, fig.height=5} 91 | plot(rst, axes=FALSE) 92 | points(xy, cex=0.1) 93 | ``` 94 | 95 | # Standard GAM in `mgcv` 96 | 97 | This is the standard way to fit X- and Y- splines in `mgcv`: 98 | 99 | ```{r} 100 | # the gam model with s() indicating that I fit splines 101 | space.only <- gam(count~s(x, y), data=all.data, family = "poisson") 102 | # extraction of the predictions 103 | preds.mgcv <- as.vector(predict(space.only, type = "response")) 104 | 105 | # putting the predictions into a raster 106 | rst.mgcv <- rst 107 | rst.mgcv[] <- preds.mgcv 108 | ``` 109 | 110 | This is the predicted surface on a map: 111 | 112 | ```{r, fig.width=8.3, fig.height=5} 113 | plot(rst.mgcv, axes=FALSE) 114 | points(xy, cex=0.1) 115 | ``` 116 | 117 | # The `jagam` function in action 118 | 119 | The main point here is that **the new `jagam` function takes the GAM 120 | formula and converts it into a piece of BUGS code**. The resulting code can 121 | then be run in JAGS, or even in OpenBUGS. Here it is in action: 122 | 123 | ```{r, eval=TRUE} 124 | jags.ready <- jagam(count~s(x, y), 125 | data=all.data, 126 | family="poisson", 127 | file="jagam.bug") 128 | ``` 129 | 130 | The `jagam` function does two things: **(1)** it creates an object that contains the data in the list format that can be readily used in the `jags` function 131 | (package `R2jags`), and **(2)** it writes the BUGS model definition into a file. That makes it really easy to fit GAM splines in JAGS. The idea is that 132 | more complex hierarchical structures can then be added directly into the BUGS 133 | code. 134 | 135 | Let's have a look into the `jagam.bug` file: 136 | 137 | ```{r, warning=FALSE} 138 | readLines("jagam.bug") 139 | ``` 140 | 141 | # Fitting the model in JAGS 142 | 143 | Here I fit the Bayesian model by calling `jags` function from package `R2jags`. 144 | I will monitor the expected values (`mu`) in each grid cell of the raster. 145 | 146 | ```{r} 147 | model.fit <- jags(data=jags.ready$jags.data, 148 | model.file="jagam.bug", 149 | parameters.to.save=c("mu"), 150 | n.chains=3, 151 | n.iter=1000, 152 | n.burnin=500) 153 | 154 | # extracting the fitted means 155 | preds.mu <- as.vector(model.fit$BUGSoutput$mean$mu) 156 | 157 | # inserting the fitted means to the raster 158 | rst.jags <- rst 159 | rst.jags[] <- preds.mu 160 | ``` 161 | 162 | Let's plot the JAGS results: 163 | 164 | ```{r, fig.width=8.3, fig.height=5} 165 | plot(rst.jags, axes=FALSE) 166 | points(xy, cex=0.1) # adding the positions of individual trees 167 | ``` 168 | 169 | It looks almost identical to the `mgcv` output. Let's have a closer look. 170 | 171 | # `mgcv` vs JAGS 172 | 173 | Here I compare the modelled counts from the two models. 174 | 175 | ```{r, fig.width=7, fig.height=7} 176 | plot(preds.mgcv, preds.mu, 177 | xlab="Counts from mgcv", 178 | ylab="Counts from JAGS") 179 | abline(a=0, b=1, col="red", lwd=2) 180 | ``` 181 | 182 | The predicted counts are not identical, JAGS predicts relatively higher 183 | counts than `mgcv`. I am not sure if this is important -- maybe it has something 184 | to do with my priors, I don't know. But I guess that I don't 185 | care that much, as long as the predictions are roughly similar. 186 | 187 | 188 | 189 | -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson.md: -------------------------------------------------------------------------------- 1 | # GAM splines now easy in JAGS. An example on 2D spatial data 2 | Petr Keil 3 | 03/02/2015 4 | 5 | Last week I met Simon Wood, creator of `mgcv` package, which is THE tool for 6 | fitting Generalized Additive Models (GAM) in R. 7 | 8 | Simon brought my attention to function `jagam` which he has just 9 | added to `mgcv`. The function allows to transform the 'spline' or 'smooth' 10 | component of GAM model formula into BUGS code, meaning that the flexibility of 11 | GAMs is now available for routine MCMC model fitting. 12 | 13 | I often deal with geographically structured (spatially explicit) data, and 14 | so I am excited by the prospect of using `jagam` to build spatially explicit 15 | hierarchical models. 16 | 17 | # Why should spatial statisticians care about `jagam`? 18 | 19 | Because fitting of spatial splines in JAGS and OpenBUGS has so far been a pain. 20 | Yet from my experience and from personal communication with others 21 | (e.g. C. Dormann, B. O'Hara, C. Beale, S. Wood), **splines are a well-behaved 22 | way to model spatial autocorrelation** that can be easily examined and 23 | visualized separatedly from the rest of the model. 24 | 25 | Splines can also be a handy alternative to the popular Conditional Autoregressive 26 | Models (CAR) that are available in OpenBUGS, but unavailable in JAGS. Hence, 27 | the clarity and portability of JAGS is now available to spatial modellers. 28 | 29 | # The aim of this post 30 | 31 | Here I will demonstrate the `jagam` function in action. I will fit a simple Poisson GAM (with X and Y coordinates as predictors) to spatially explicit count data. 32 | I will also check if I get the same expected values 33 | from `mgcv` and `JAGS`, given the same model. 34 | 35 | These are the packages that I will need: 36 | 37 | 38 | ```r 39 | library(mgcv) # fits GAMs 40 | library(spatstat) # the source of the example data 41 | library(raster) # for operations with rasters 42 | library(R2jags) # interface between R and JAGS 43 | ``` 44 | 45 | # The data 46 | 47 | I will use example dataset `bei` from `spatstat` package. The data 48 | are positions of 3605 individual trees of *Beilschmiedia pendula* (Lauraceae) in 49 | a 1000 by 500 metre rectangular sampling region in the tropical rainforest of 50 | Barro Colorado Island. The data are stored in a point process pattern `ppp` 51 | object. 52 | 53 | Let's plot the data: 54 | 55 | 56 | ```r 57 | par(mai=c(0.5,0.3,0.3,0)) 58 | plot(bei, cex=0.1, main=NULL) 59 | ``` 60 | 61 | ![](spatial_gam_poisson_files/figure-html/unnamed-chunk-2-1.png) 62 | 63 | I will fit the data into a raster of 25 x 50 grid cells; each grid cell gives 64 | the count of individual trees that fall within the cell: 65 | 66 | 67 | ```r 68 | # cropping the data so that they have exactly 500 x 1000 cells 69 | ext <- extent(0, 1000, 0, 500) # spatial extent of the raster 70 | empty <- raster(ext, nrow=25, ncol=50) # empty raster 71 | 72 | # aggregating the point data into the raster 73 | xy <- data.frame(x = bei$x, y = bei$y) 74 | rst <- rasterize(xy, empty, fun = "count") 75 | 76 | # replacing the NA values by 0 77 | rst[is.na(rst)] <- 0 78 | 79 | # extracting the cell values and their coordinates to a data.frame 80 | coord <- xyFromCell(rst,1:ncell(rst)) 81 | count <- extract(rst, 1:ncell(rst)) 82 | all.data <- data.frame(coord, count=count) 83 | ``` 84 | 85 | This is the resulting rasterized dataset, with point locations of the trees 86 | plotted on top. The color gradient shows counts of trees in each grid cell. 87 | 88 | 89 | ```r 90 | plot(rst, axes=FALSE) 91 | points(xy, cex=0.1) 92 | ``` 93 | 94 | ![](spatial_gam_poisson_files/figure-html/unnamed-chunk-4-1.png) 95 | 96 | # Standard GAM in `mgcv` 97 | 98 | This is the standard way to fit X- and Y- splines in `mgcv`: 99 | 100 | 101 | ```r 102 | # the gam model with s() indicating that I fit splines 103 | space.only <- gam(count~s(x, y), data=all.data, family = "poisson") 104 | # extraction of the predictions 105 | preds.mgcv <- as.vector(predict(space.only, type = "response")) 106 | 107 | # putting the predictions into a raster 108 | rst.mgcv <- rst 109 | rst.mgcv[] <- preds.mgcv 110 | ``` 111 | 112 | This is the predicted surface on a map: 113 | 114 | 115 | ```r 116 | plot(rst.mgcv, axes=FALSE) 117 | points(xy, cex=0.1) 118 | ``` 119 | 120 | ![](spatial_gam_poisson_files/figure-html/unnamed-chunk-6-1.png) 121 | 122 | # The `jagam` function in action 123 | 124 | The main point here is that **the new `jagam` function takes the GAM 125 | formula and converts it into a piece of BUGS code**. The resulting code can 126 | then be run in JAGS, or even in OpenBUGS. Here it is in action: 127 | 128 | 129 | ```r 130 | jags.ready <- jagam(count~s(x, y), 131 | data=all.data, 132 | family="poisson", 133 | file="jagam.bug") 134 | ``` 135 | 136 | The `jagam` function does two things: **(1)** it creates an object that contains the data in the list format that can be readily used in the `jags` function 137 | (package `R2jags`), and **(2)** it writes the BUGS model definition into a file. That makes it really easy to fit GAM splines in JAGS. The idea is that 138 | more complex hierarchical structures can then be added directly into the BUGS 139 | code. 140 | 141 | Let's have a look into the `jagam.bug` file: 142 | 143 | 144 | ```r 145 | readLines("jagam.bug") 146 | ``` 147 | 148 | ``` 149 | ## [1] "model {" 150 | ## [2] " eta <- X %*% b ## linear predictor" 151 | ## [3] " for (i in 1:n) { mu[i] <- exp(eta[i]) } ## expected response" 152 | ## [4] " for (i in 1:n) { y[i] ~ dpois(mu[i]) } ## response " 153 | ## [5] " ## Parameteric effect priors CHECK tau is appropriate!" 154 | ## [6] " for (i in 1:1) { b[i] ~ dnorm(0,0.001) }" 155 | ## [7] " ## prior for s(x,y)... " 156 | ## [8] " K1 <- S1[1:29,1:29] * lambda[1] + S1[1:29,30:58] * lambda[2]" 157 | ## [9] " b[2:30] ~ dmnorm(zero[2:30],K1) " 158 | ## [10] " ## smoothing parameter priors CHECK..." 159 | ## [11] " for (i in 1:2) {" 160 | ## [12] " lambda[i] ~ dgamma(.05,.005)" 161 | ## [13] " rho[i] <- log(lambda[i])" 162 | ## [14] " }" 163 | ## [15] "}" 164 | ``` 165 | 166 | # Fitting the model in JAGS 167 | 168 | Here I fit the Bayesian model by calling `jags` function from package `R2jags`. 169 | I will monitor the expected values (`mu`) in each grid cell of the raster. 170 | 171 | 172 | ```r 173 | model.fit <- jags(data=jags.ready$jags.data, 174 | model.file="jagam.bug", 175 | parameters.to.save=c("mu"), 176 | n.chains=3, 177 | n.iter=1000, 178 | n.burnin=500) 179 | ``` 180 | 181 | ``` 182 | ## module glm loaded 183 | ``` 184 | 185 | ``` 186 | ## Compiling model graph 187 | ## Resolving undeclared variables 188 | ## Allocating nodes 189 | ## Graph Size: 42982 190 | ## 191 | ## Initializing model 192 | ``` 193 | 194 | ```r 195 | # extracting the fitted means 196 | preds.mu <- as.vector(model.fit$BUGSoutput$mean$mu) 197 | 198 | # inserting the fitted means to the raster 199 | rst.jags <- rst 200 | rst.jags[] <- preds.mu 201 | ``` 202 | 203 | Let's plot the JAGS results: 204 | 205 | 206 | ```r 207 | plot(rst.jags, axes=FALSE) 208 | points(xy, cex=0.1) # adding the positions of individual trees 209 | ``` 210 | 211 | ![](spatial_gam_poisson_files/figure-html/unnamed-chunk-10-1.png) 212 | 213 | It looks almost identical to the `mgcv` output. Let's have a closer look. 214 | 215 | # `mgcv` vs JAGS 216 | 217 | Here I compare the modelled counts from the two models. 218 | 219 | 220 | ```r 221 | plot(preds.mgcv, preds.mu, 222 | xlab="Counts from mgcv", 223 | ylab="Counts from JAGS") 224 | abline(a=0, b=1, col="red", lwd=2) 225 | ``` 226 | 227 | ![](spatial_gam_poisson_files/figure-html/unnamed-chunk-11-1.png) 228 | 229 | The predicted counts are not identical, JAGS predicts relatively higher 230 | counts than `mgcv`. I am not sure if this is important -- maybe it has something 231 | to do with my priors, I don't know. But I guess that I don't 232 | care that much, as long as the predictions are roughly similar. 233 | 234 | 235 | 236 | -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-10-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-10-1.png -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-11-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-11-1.png -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-2-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-2-1.png -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-4-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-4-1.png -------------------------------------------------------------------------------- /2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-6-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_11_GAM_in_JAGS/spatial_gam_poisson_files/figure-html/unnamed-chunk-6-1.png -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/.Rhistory: -------------------------------------------------------------------------------- 1 | citation("knitr") 2 | citation("R") 3 | citation(R) 4 | citation() 5 | -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/intro.md: -------------------------------------------------------------------------------- 1 | # Simple template for scientific manuscripts in R markdown 2 | 3 | The good reasons to write scientific reports and manuscripts in LaTeX or Markdown are: improved document integrity (always), simplicity (not always) and reproducibility (always). I prefer the lightweight and simple Markdown over rich but more complex LaTeX -- I think that light and simple is good for reproducibility. I am also in love with `knitr`. 4 | 5 | Hence, I made a really simple template for the classical manuscript format for R markdown and knitr. Check out the the resulting [`.pdf`](https://github.com/petrkeil/Blog/raw/master/2015_03_12_R_ms_template/manuscript_template.pdf) and [`.html`](https://rawgit.com/petrkeil/Blog/master/2015_03_12_R_ms_template/manuscript_template.html). 6 | 7 | The template contains four important components of any scientific manuscript: 8 | 9 | * equations (using LaTeX syntax) 10 | * table with caption (done by `kable` package, but you can also use `xtable`) 11 | * figure with caption 12 | * citations and references (done by `knitcitations` package) 13 | 14 | The template uses *Methods in Ecology and Evolution* reference style, which is stored in the [`mee.csl`](https://raw.githubusercontent.com/petrkeil/Blog/master/2015_03_12_R_ms_template/mee.csl) file. 15 | 16 | The template does not have* line numbers* nor *wide line spacing*. To add these you will have to edit the `.tex` file, i.e. you will need to learn a little bit of LaTeX. 17 | 18 | ## How to use the template? 19 | 20 | 1. Go to the [GitHub repository](https://github.com/petrkeil/Blog/tree/master/2015_03_12_R_ms_template). 21 | 2. Save the `.Rmd` and `.csl` files to your local folder. 22 | 3. Open the `.Rmd` file with **R studio**. 23 | 4. Edit freely. 24 | 5. Hit the **Knit html** or **Knit PDF** button. 25 | 26 | 27 | 28 | -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/manuscript_template.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Template for writing scientific papers in R markdown" 3 | author: "Petr Keil, pkeil@seznam.cz" 4 | date: "11/1/2015" 5 | output: 6 | pdf_document: 7 | fig_caption: yes 8 | keep_tex: yes 9 | number_sections: yes 10 | html_document: 11 | fig_caption: yes 12 | force_captions: yes 13 | highlight: pygments 14 | number_sections: yes 15 | theme: cerulean 16 | csl: mee.csl 17 | bibliography: references.bib 18 | --- 19 | 20 | ```{r, echo=FALSE} 21 | # devtools::install_github("cboettig/knitcitations@v1") 22 | library(knitcitations); cleanbib() 23 | cite_options(citation_format = "pandoc", check.entries=FALSE) 24 | library(bibtex) 25 | ``` 26 | 27 | # Abstract 28 | 29 | *Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei.* 30 | 31 | 32 | 33 | # Introduction 34 | 35 | Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei. 36 | 37 | Putant numquam tacimates at eum. Aliquip torquatos ex vis, mei et quando debitis appareat, impetus accumsan corrumpit in usu. Nam mucius facilis singulis id, duo ei autem imperdiet instructior. Cu ceteros alienum mel, id vix putant impedit, ex idque eruditi forensibus eum. Posse dicunt id usu. Ei iracundia constituto sed, duo ne exerci ignota, an eum unum conceptam. 38 | 39 | Has audire salutandi no, ut eam dicat libris dicunt. Pri hendrerit quaerendum adversarium ea, dicat atqui munere et sea. Illum insolens eos ne, eu enim graece rationibus mea. At postea utamur mel, eius nonumes percipitur at vis. Numquam similique in per, te quo saepe utroque pericula. 40 | 41 | Ea nonumy volumus usu, no mel inermis dissentias. Dico partiendo vituperatoribus eum et. Mea accusam convenire te, usu populo qualisque gloriatur ut. Eu eum oratio altera option, ad mea ignota scriptorem. Ne suas latine vix, eos oblique sanctus pertinax cu. 42 | 43 | 44 | 45 | # Methods 46 | 47 | Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei. 48 | 49 | Putant numquam tacimates at eum. Aliquip torquatos ex vis, mei et quando debitis appareat, impetus accumsan corrumpit in usu. Nam mucius facilis singulis id, duo ei autem imperdiet instructior. Cu ceteros alienum mel, id vix putant impedit, ex idque eruditi forensibus eum. Posse dicunt id usu. Ei iracundia constituto sed, duo ne exerci ignota, an eum unum conceptam. 50 | 51 | ## Equations 52 | 53 | The deterministic part of the model is defined by this **in-line equation** as 54 | $\mu_i = \beta_0 + \beta_1x$, and the stochastic part by the **centered equation**: 55 | 56 | $$ \frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu_i)^2/(2\sigma^2)} $$ 57 | 58 | ## Tables 59 | 60 | ```{r kable, echo=FALSE} 61 | library(knitr) 62 | 63 | x <- rnorm(100) 64 | y <- 2*x + rnorm(100) 65 | out <- lm(y ~ x) 66 | kable(summary(out)$coef, digits=2, caption="This is a GLM summary table.") 67 | ``` 68 | 69 | ```{r xtable, results="asis", echo=FALSE, eval=FALSE, message=FALSE} 70 | library(xtable) 71 | tab <- xtable(summary(out)$coef, digits=c(0, 2, 2, 1, 2), 72 | caption="This is a GLM summary table.") 73 | print(tab, type="latex") 74 | ``` 75 | 76 | ## Plots 77 | 78 | ```{r carDataPlot, echo=FALSE, fig.cap="Relationship between x and y. The solid line is least-squares linear regression.", fig.width=4, fig.height=4} 79 | plot(x, y) 80 | abline(out) 81 | ``` 82 | 83 | 84 | 85 | ## Citations 86 | 87 | The relationship was first described by 88 | `r citet("10.1111/j.1461-0248.2005.00827.x")`. However, there are also opinions 89 | that the relationship is spurious `r citep("10.1111/j.2041-210x.2012.00264.x")`. 90 | We used R for our calculations `r citep(citation())`, and we used package `knitcitations` `r citep(citation("knitcitations"))` to make the bibliography. 91 | 92 | 93 | # Results and discussion 94 | 95 | Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei. 96 | 97 | Putant numquam tacimates at eum. Aliquip torquatos ex vis, mei et quando debitis appareat, impetus accumsan corrumpit in usu. Nam mucius facilis singulis id, duo ei autem imperdiet instructior. Cu ceteros alienum mel, id vix putant impedit, ex idque eruditi forensibus eum. Posse dicunt id usu. Ei iracundia constituto sed, duo ne exerci ignota, an eum unum conceptam. 98 | 99 | Has audire salutandi no, ut eam dicat libris dicunt. Pri hendrerit quaerendum adversarium ea, dicat atqui munere et sea. Illum insolens eos ne, eu enim graece rationibus mea. At postea utamur mel, eius nonumes percipitur at vis. Numquam similique in per, te quo saepe utroque pericula. 100 | 101 | Ea nonumy volumus usu, no mel inermis dissentias. Dico partiendo vituperatoribus eum et. Mea accusam convenire te, usu populo qualisque gloriatur ut. Eu eum oratio altera option, ad mea ignota scriptorem. Ne suas latine vix, eos oblique sanctus pertinax cu. 102 | 103 | # References 104 | 105 | ```{r, warning=FALSE, message=FALSE, echo=FALSE} 106 | write.bibtex(file="references.bib") 107 | ``` 108 | 109 | 110 | 111 | 112 | -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/manuscript_template.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_12_R_ms_template/manuscript_template.pdf -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/manuscript_template.tex: -------------------------------------------------------------------------------- 1 | \documentclass[]{article} 2 | \usepackage{lmodern} 3 | \usepackage{amssymb,amsmath} 4 | \usepackage{ifxetex,ifluatex} 5 | \usepackage{fixltx2e} % provides \textsubscript 6 | \ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex 7 | \usepackage[T1]{fontenc} 8 | \usepackage[utf8]{inputenc} 9 | \else % if luatex or xelatex 10 | \ifxetex 11 | \usepackage{mathspec} 12 | \else 13 | \usepackage{fontspec} 14 | \fi 15 | \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase} 16 | \fi 17 | % use upquote if available, for straight quotes in verbatim environments 18 | \IfFileExists{upquote.sty}{\usepackage{upquote}}{} 19 | % use microtype if available 20 | \IfFileExists{microtype.sty}{% 21 | \usepackage{microtype} 22 | \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts 23 | }{} 24 | \usepackage[margin=1in]{geometry} 25 | \usepackage{hyperref} 26 | \hypersetup{unicode=true, 27 | pdftitle={Template for writing scientific papers in R markdown}, 28 | pdfauthor={Petr Keil, pkeil@seznam.cz}, 29 | pdfborder={0 0 0}, 30 | breaklinks=true} 31 | \urlstyle{same} % don't use monospace font for urls 32 | \usepackage{longtable,booktabs} 33 | \usepackage{graphicx,grffile} 34 | \makeatletter 35 | \def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi} 36 | \def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi} 37 | \makeatother 38 | % Scale images if necessary, so that they will not overflow the page 39 | % margins by default, and it is still possible to overwrite the defaults 40 | % using explicit options in \includegraphics[width, height, ...]{} 41 | \setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio} 42 | \IfFileExists{parskip.sty}{% 43 | \usepackage{parskip} 44 | }{% else 45 | \setlength{\parindent}{0pt} 46 | \setlength{\parskip}{6pt plus 2pt minus 1pt} 47 | } 48 | \setlength{\emergencystretch}{3em} % prevent overfull lines 49 | \providecommand{\tightlist}{% 50 | \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}} 51 | \setcounter{secnumdepth}{5} 52 | % Redefines (sub)paragraphs to behave more like sections 53 | \ifx\paragraph\undefined\else 54 | \let\oldparagraph\paragraph 55 | \renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}} 56 | \fi 57 | \ifx\subparagraph\undefined\else 58 | \let\oldsubparagraph\subparagraph 59 | \renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}} 60 | \fi 61 | 62 | %%% Use protect on footnotes to avoid problems with footnotes in titles 63 | \let\rmarkdownfootnote\footnote% 64 | \def\footnote{\protect\rmarkdownfootnote} 65 | 66 | %%% Change title format to be more compact 67 | \usepackage{titling} 68 | 69 | % Create subtitle command for use in maketitle 70 | \newcommand{\subtitle}[1]{ 71 | \posttitle{ 72 | \begin{center}\large#1\end{center} 73 | } 74 | } 75 | 76 | \setlength{\droptitle}{-2em} 77 | \title{Template for writing scientific papers in R markdown} 78 | \pretitle{\vspace{\droptitle}\centering\huge} 79 | \posttitle{\par} 80 | \author{Petr Keil, \href{mailto:pkeil@seznam.cz}{\nolinkurl{pkeil@seznam.cz}}} 81 | \preauthor{\centering\large\emph} 82 | \postauthor{\par} 83 | \predate{\centering\large\emph} 84 | \postdate{\par} 85 | \date{11/1/2015} 86 | 87 | 88 | \begin{document} 89 | \maketitle 90 | 91 | \section{Abstract}\label{abstract} 92 | 93 | \emph{Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel 94 | erat falli ut. Feugiat legendos adipisci vix at, usu at laoreet 95 | argumentum suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor 96 | liberavisse sea. Ponderum vivendum te nec, id agam brute disputando 97 | mei.} 98 | 99 | \section{Introduction}\label{introduction} 100 | 101 | Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat 102 | falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum 103 | suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor 104 | liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei. 105 | 106 | Putant numquam tacimates at eum. Aliquip torquatos ex vis, mei et quando 107 | debitis appareat, impetus accumsan corrumpit in usu. Nam mucius facilis 108 | singulis id, duo ei autem imperdiet instructior. Cu ceteros alienum mel, 109 | id vix putant impedit, ex idque eruditi forensibus eum. Posse dicunt id 110 | usu. Ei iracundia constituto sed, duo ne exerci ignota, an eum unum 111 | conceptam. 112 | 113 | Has audire salutandi no, ut eam dicat libris dicunt. Pri hendrerit 114 | quaerendum adversarium ea, dicat atqui munere et sea. Illum insolens eos 115 | ne, eu enim graece rationibus mea. At postea utamur mel, eius nonumes 116 | percipitur at vis. Numquam similique in per, te quo saepe utroque 117 | pericula. 118 | 119 | Ea nonumy volumus usu, no mel inermis dissentias. Dico partiendo 120 | vituperatoribus eum et. Mea accusam convenire te, usu populo qualisque 121 | gloriatur ut. Eu eum oratio altera option, ad mea ignota scriptorem. Ne 122 | suas latine vix, eos oblique sanctus pertinax cu. 123 | 124 | \section{Methods}\label{methods} 125 | 126 | Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat 127 | falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum 128 | suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor 129 | liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei. 130 | 131 | Putant numquam tacimates at eum. Aliquip torquatos ex vis, mei et quando 132 | debitis appareat, impetus accumsan corrumpit in usu. Nam mucius facilis 133 | singulis id, duo ei autem imperdiet instructior. Cu ceteros alienum mel, 134 | id vix putant impedit, ex idque eruditi forensibus eum. Posse dicunt id 135 | usu. Ei iracundia constituto sed, duo ne exerci ignota, an eum unum 136 | conceptam. 137 | 138 | \subsection{Equations}\label{equations} 139 | 140 | The deterministic part of the model is defined by this \textbf{in-line 141 | equation} as \(\mu_i = \beta_0 + \beta_1x\), and the stochastic part by 142 | the \textbf{centered equation}: 143 | 144 | \[ \frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu_i)^2/(2\sigma^2)} \] 145 | 146 | \subsection{Tables}\label{tables} 147 | 148 | \begin{longtable}[]{@{}lrrrr@{}} 149 | \caption{This is a GLM summary table.}\tabularnewline 150 | \toprule 151 | & Estimate & Std. Error & t value & 152 | Pr(\textgreater{}\textbar{}t\textbar{})\tabularnewline 153 | \midrule 154 | \endfirsthead 155 | \toprule 156 | & Estimate & Std. Error & t value & 157 | Pr(\textgreater{}\textbar{}t\textbar{})\tabularnewline 158 | \midrule 159 | \endhead 160 | (Intercept) & 0.05 & 0.11 & 0.46 & 0.65\tabularnewline 161 | x & 2.00 & 0.12 & 16.91 & 0.00\tabularnewline 162 | \bottomrule 163 | \end{longtable} 164 | 165 | \subsection{Plots}\label{plots} 166 | 167 | \begin{figure}[htbp] 168 | \centering 169 | \includegraphics{manuscript_template_files/figure-latex/carDataPlot-1.pdf} 170 | \caption{Relationship between x and y. The solid line is least-squares 171 | linear regression.} 172 | \end{figure} 173 | 174 | \subsection{Citations}\label{citations} 175 | 176 | The relationship was first described by Halpern \emph{et al.} (2006). 177 | However, there are also opinions that the relationship is spurious (Keil 178 | \emph{et al.} 2012). We used R for our calculations (R Core Team 2016), 179 | and we used package \texttt{knitcitations} (Boettiger 2015) to make the 180 | bibliography. 181 | 182 | \section{Results and discussion}\label{results-and-discussion} 183 | 184 | Lorem ipsum dolor sit amet, est ad doctus eligendi scriptorem. Mel erat 185 | falli ut. Feugiat legendos adipisci vix at, usu at laoreet argumentum 186 | suscipiantur. An eos adhuc aliquip scriptorem, te adhuc dolor 187 | liberavisse sea. Ponderum vivendum te nec, id agam brute disputando mei. 188 | 189 | Putant numquam tacimates at eum. Aliquip torquatos ex vis, mei et quando 190 | debitis appareat, impetus accumsan corrumpit in usu. Nam mucius facilis 191 | singulis id, duo ei autem imperdiet instructior. Cu ceteros alienum mel, 192 | id vix putant impedit, ex idque eruditi forensibus eum. Posse dicunt id 193 | usu. Ei iracundia constituto sed, duo ne exerci ignota, an eum unum 194 | conceptam. 195 | 196 | Has audire salutandi no, ut eam dicat libris dicunt. Pri hendrerit 197 | quaerendum adversarium ea, dicat atqui munere et sea. Illum insolens eos 198 | ne, eu enim graece rationibus mea. At postea utamur mel, eius nonumes 199 | percipitur at vis. Numquam similique in per, te quo saepe utroque 200 | pericula. 201 | 202 | Ea nonumy volumus usu, no mel inermis dissentias. Dico partiendo 203 | vituperatoribus eum et. Mea accusam convenire te, usu populo qualisque 204 | gloriatur ut. Eu eum oratio altera option, ad mea ignota scriptorem. Ne 205 | suas latine vix, eos oblique sanctus pertinax cu. 206 | 207 | \section*{References}\label{references} 208 | \addcontentsline{toc}{section}{References} 209 | 210 | \hypertarget{refs}{} 211 | \hypertarget{ref-Boettiger_2015}{} 212 | Boettiger, C. (2015). \emph{Knitcitations: Citations for 'knitr' 213 | markdown files}. Retrieved from 214 | \url{https://CRAN.R-project.org/package=knitcitations} 215 | 216 | \hypertarget{ref-Halpern_2006}{} 217 | Halpern, B.S., Regan, H.M., Possingham, H.P. \& McCarthy, M.A. (2006). 218 | Accounting for uncertainty in marine reserve design. \emph{Ecology 219 | Letters}, \textbf{9}, 2--11. Retrieved from 220 | \url{https://doi.org/10.1111\%2Fj.1461-0248.2005.00827.x} 221 | 222 | \hypertarget{ref-Keil_2012}{} 223 | Keil, P., Belmaker, J., Wilson, A.M., Unitt, P. \& Jetz, W. (2012). 224 | Downscaling of species distribution models: A hierarchical approach (R. 225 | Freckleton, Ed.). \emph{Methods in Ecology and Evolution}, \textbf{4}, 226 | 82--94. Retrieved from 227 | \url{https://doi.org/10.1111\%2Fj.2041-210x.2012.00264.x} 228 | 229 | \hypertarget{ref-R_Core_Team_2016}{} 230 | R Core Team. (2016). \emph{R: A language and environment for statistical 231 | computing}. R Foundation for Statistical Computing, Vienna, Austria. 232 | Retrieved from \url{https://www.R-project.org/} 233 | 234 | 235 | \end{document} 236 | -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/manuscript_template_files/figure-latex/carDataPlot-1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_03_12_R_ms_template/manuscript_template_files/figure-latex/carDataPlot-1.pdf -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/mee.csl: -------------------------------------------------------------------------------- 1 | 2 | 182 | 183 | -------------------------------------------------------------------------------- /2015_03_12_R_ms_template/references.bib: -------------------------------------------------------------------------------- 1 | @Article{Halpern_2006, 2 | doi = {10.1111/j.1461-0248.2005.00827.x}, 3 | url = {https://doi.org/10.1111%2Fj.1461-0248.2005.00827.x}, 4 | year = {2006}, 5 | month = {jan}, 6 | publisher = {Wiley-Blackwell}, 7 | volume = {9}, 8 | number = {1}, 9 | pages = {2--11}, 10 | author = {Benjamin S. Halpern and Helen M. Regan and Hugh P. Possingham and Michael A. McCarthy}, 11 | title = {Accounting for uncertainty in marine reserve design}, 12 | journal = {Ecology Letters}, 13 | } 14 | 15 | @Manual{R_Core_Team_2016, 16 | title = {R: A Language and Environment for Statistical Computing}, 17 | author = {{R Core Team}}, 18 | organization = {R Foundation for Statistical Computing}, 19 | address = {Vienna, Austria}, 20 | year = {2016}, 21 | url = {https://www.R-project.org/}, 22 | } 23 | 24 | @Manual{Boettiger_2015, 25 | title = {knitcitations: Citations for 'Knitr' Markdown Files}, 26 | author = {Carl Boettiger}, 27 | year = {2015}, 28 | note = {R package version 1.0.7}, 29 | url = {https://CRAN.R-project.org/package=knitcitations}, 30 | } 31 | 32 | @Article{Keil_2012, 33 | doi = {10.1111/j.2041-210x.2012.00264.x}, 34 | url = {https://doi.org/10.1111%2Fj.2041-210x.2012.00264.x}, 35 | year = {2012}, 36 | month = {nov}, 37 | publisher = {Wiley-Blackwell}, 38 | volume = {4}, 39 | number = {1}, 40 | pages = {82--94}, 41 | author = {Petr Keil and Jonathan Belmaker and Adam M. Wilson and Philip Unitt and Walter Jetz}, 42 | editor = {Robert Freckleton}, 43 | title = {Downscaling of species distribution models: a hierarchical approach}, 44 | journal = {Methods in Ecology and Evolution}, 45 | } 46 | -------------------------------------------------------------------------------- /2015_04_15_The_age_of_postdoc/the_age_of_postdoc.html: -------------------------------------------------------------------------------- 1 | 2 | 3 |

4 | The age of Postdoc: towards liberation of academic middle class 5 |

6 |

7 | Nature has an article about the daunting prospects for current postdocs in some countries ( 8 | 9 | Nature 520, 144-147 10 | 11 | ). The article is stimulating but reports only a few relatively ad-hoc ways to fix the situation. In the discussion below the article Peter Jurica mentions 12 | 13 | postdocs as middle class 14 | 15 | (of academia). I think that this simple analogy can inspire more fundamental actions and paradigm shifts, potentially improving the system in favor of postdocs. I’ve tried to outline them below, building on my limited experience from the US, UK, France and the Czech Republic – hence, my (random) ideas will apply differently in different countries and types of universities. The specific countries which I have in mind are US, UK and Germany, where postdocs have especially hard time seeing the future for all the bottlenecks. 16 |

17 |

18 | These are my suggestions: 19 |

20 |
    21 |
  1. 22 |

    23 | 24 | Acknowledge academic careers that do not involve managing people. 25 | 26 | Some people excel in doing science, they can even be excellent collaborators, or they have great analytical skills, but are bad in managing people and leading a lab. Many of them would be happy with receiving lower salary relatively to a full lab leader, but they should still be given independence and treated better than today’s postdocs. The system potentially misallocates human resources when it encourages leadership as the only logical step after postdoc; an excellent scientist who is a bad manager can be, in some circumstances, more valuable than an average scientist who is a good manager. 27 |

    28 |
    2. 29 |
  2. 30 |

    31 | 32 | Legitimize serial postdocs. 33 | 34 | Many disciplines (journalism, art, photography, data analysis, consultancy, law, …) have the concept of a 35 | 36 | freelancer 37 | 38 | – someone who does intellectually demanding or creative work without being part of a big institution and without the stability of permanent position. Freelancing is a legitimate life-long career. Similarly, a highly qualified postdoc who moves from position to position (and place to place) on 1-4 year-ish basis, or who accepts smaller jobs from different institutions, could be considered a research freelancer – a legitimate academic career for those who are happy with the constant change. I see this as an option for research qualifications that involve quantitative and writing skills, as these are transferable between sub-disciplines. I also believe that a senior serial postdoc (an unthinkable and almost pejorative status these days) could have a unique perspective which a senior professor would hardly reach. 39 |

    40 |
    3. 41 |
  3. 42 |

    43 | 44 | Allocate resources from senior faculty positions to the academic middle class. 45 | 46 | Well, if there is not enough cash flowing into the system, one solution (left wing, communist, egalitarian, … start throwing the eggs) to brighten the future of early-stage academics could be to redistribute the cash more evenly. For example, in the US system, the raise of prestige, income, job security and power along the progressing career in academia could be made more gradual, without the abrupt step (bottleneck) between a postdoc and an assistant professor. This would increase opportunities for the early-stage researchers, and it could be sustained without pouring more money into the system. Where it would hurt, however, are the higher positions of today’s academic hierarchy. Yet as a postdoc and father myself, I am more comfortable with more opportunities and slower raise in future pay, than with a prospect of hitting a high-paid high-security jackpot, but with low probability. 47 |

    48 |
    4. 49 |
  4. 50 |

    51 | 52 | Clarify and possibly reduce the power of advisors over postdocs. 53 | 54 | It is crucial for the future of a postdoc to build his/her own research direction, independent on any advisor. Many advisors recognize that, but not all – the postdoc stories from Ivy League medical schools are full of expressions such as ‘slave’. I suggest that, by rule, anyone hired as a postdoc should be entitled to some days of research, per week basis, with zero involvement of the advisor – this entitlement could be a default and explicit part of postdoctoral contracts. I also think that some advisors don’t realize that they are not the bosses, and postdocs are not their subordinates – an 55 | 56 | advisor 57 | 58 | ’s role is to give 59 | 60 | advice 61 | 62 | (which is, by definition, not bonding) to someone who is on his/her own path. Maybe this could also be stated in every postdoctoral contract. 63 |

    64 |
    5. 65 |
  5. 66 |

    67 | 68 | Allow postdocs to compete for opportunities to teach as principal lecturers. 69 | 70 | Postdocs are often experts in their field, so why not giving them the opportunity to influence undergrads as principal lecturers? Being responsible for a whole course is a formative experience, completely different from being a mere teaching assistant. It is also a skill that requires some talent and tons of practice – hence the talent needs to be identified early. Also, many undergrad courses taught by full professors are just plain boring. For these reasons postdocs should have the opportunity to be considered as principal lecturers, if there is an indication that they can do it better than a faculty member. 71 |

    72 |
    6. 73 |
  6. 74 |

    75 | 76 | Allow postdocs to compete for the big grants. 77 | 78 | A postdoc is someone who has already proven that he/she can do science, defend it and publish it; that is why he/she holds the PhD degree, which I see as the formal invitation to the grown-ups’ game. So dear NSF and others, how about allowing postdocs to play the big game? They may not always succeed in getting the grant, but at least they will have the chance. And who else than young minds are better qualified to propose high risk – high gain projects (the golden calf of basic science)? 79 |

    80 |
    7. 81 |
  7. 82 |

    83 | 84 | A postdoc who secures funding should be allowed to tutor grad students. 85 | 86 | This is self-explanatory and obvious. Such postdoc could also be given a 87 | 88 | temporary 89 | 90 | status of assistant professor (for the duration of the funding). 91 |

    92 |
    8. 93 |
  8. 94 |

    95 | 96 | Make permanent positions less permanent and subject to periodic reevaluation. 97 | 98 | In other words, create vacant niches by penalizing below-average performance of professors; such penalty can be a decrease to an appropriate professional level, e.g. from professor back to postdoc, which is not the same as loss of the job. A modification of this idea would allow postdocs to challenge the faculty positions directly – if the challenger wins, he/she would swap the position with the professor. No jobs lost, no extra cash needed. 99 |

    100 |
    9. 101 |
  9. 102 |

    103 | 104 | Put lab assistants and technicians who contributed work as co-authors on the resulting publications. 105 | 106 | They do care! It costs almost nothing, and it makes people feel involved, appreciated and interested in the subject. It also makes sense because they are real part of the research effort. Finally, it means brighter prospects for a postdoc who voluntarily or involuntarily goes for a technician position but still loves science, or hopes to get back to an academic position one day. 107 |

    108 |
    10. 109 |
  10. 110 |

    111 | 112 | Change the label from ‘postdoc’ to ‘researcher’. 113 | 114 | This is just to clarify what postdocs really are. 115 |

    116 |
    11. 117 |
118 | 119 | -------------------------------------------------------------------------------- /2015_04_15_The_age_of_postdoc/the_age_of_postdoc.md: -------------------------------------------------------------------------------- 1 | # The age of Postdoc: towards liberation of academic middle class 2 | 3 | Nature has an article about the daunting prospects for current postdocs in some countries ([Nature 520, 144-147](http://www.nature.com/news/the-future-of-the-postdoc-1.17253)). The article is stimulating but reports only a few relatively ad-hoc ways to fix the situation. In the discussion below the article Peter Jurica mentions *postdocs as middle class* (of academia). I think that this simple analogy can inspire more fundamental actions and paradigm shifts, potentially improving the system in favor of postdocs. I've tried to outline them below, building on my limited experience from the US, UK, France and the Czech Republic – hence, my (random) ideas will apply differently, or can even be irrelevant, in different countries and types of universities. The specific countries which I have in mind are US, UK and Germany, where postdocs have especially hard time seeing the future for all the bottlenecks. 4 | 5 | These are my suggestions: 6 | 7 | 1. **Acknowledge academic careers that do not involve managing people.** Some people excel in doing science, they can even be excellent collaborators, or they have great analytical skills, but are bad in managing people and leading a lab. Many of them would be happy with receiving lower salary relatively to a full lab leader, but they should still be given independence and treated better than today's postdocs. The system potentially misallocates human resources when it encourages leadership as the only logical step after postdoc; an excellent scientist who is a bad manager can be, in some circumstances, more valuable than an average scientist who is a good manager. 8 | 9 | 2. **Legitimize serial postdocs.** Many disciplines (journalism, art, photography, data analysis, consultancy, law, …) have the concept of a *freelancer* – someone who does intellectually demanding or creative work without being part of a big institution and without the stability of permanent position. Freelancing is a legitimate life-long career. Similarly, a highly qualified postdoc who moves from position to position (and place to place) on 1-4 year-ish basis, or who accepts smaller jobs from different institutions, could be considered a research freelancer – a legitimate academic career for those who are happy with the constant change. I see this as an option for research qualifications that involve quantitative and writing skills, as these are transferable between sub-disciplines. I also believe that a senior serial postdoc (an unthinkable and almost pejorative status these days) could have a unique perspective which a senior professor would hardly reach. 10 | 11 | 3. **Allocate resources from senior faculty positions to the academic middle class.** Well, if there is not enough cash flowing into the system, one solution (left wing, communist, egalitarian, … start throwing the eggs) to brighten the future of early-stage academics could be to redistribute the cash more evenly. For example, in the US system, the raise of prestige, income, job security and power along the progressing career in academia could be made more gradual, without the abrupt step (bottleneck) between a postdoc and an assistant professor. This would increase opportunities for the early-stage researchers, and it could be sustained without pouring more money into the system. Where it would hurt, however, are the higher positions of today's academic hierarchy. Yet as a postdoc and father myself, I am more comfortable with more opportunities and slower raise in future pay, than with a prospect of hitting a high-paid high-security jackpot, but with low probability. 12 | 13 | 4. **Clarify and possibly reduce the power of advisors over postdocs.** It is crucial for the future of a postdoc to build his/her own research direction, independent on any advisor. Many advisors recognize that, but not all – the postdoc stories from Ivy League medical schools are full of expressions such as 'slave'. I suggest that, by rule, anyone hired as a postdoc should be entitled to some days of research, per week basis, with zero involvement of the advisor – this entitlement could be a default and explicit part of postdoctoral contracts. I also think that some advisors don't realize that they are not the bosses, and postdocs are not their subordinates – an *advisor*'s role is to give *advice* (which is, by definition, not bonding) to someone who is on his/her own path. Maybe this could also be stated in every postdoctoral contract. 14 | 15 | 5. **Allow postdocs to compete for opportunities to teach as principal lecturers.** Postdocs are often experts in their field, so why not giving them the opportunity to influence undergrads as principal lecturers? Being responsible for a whole course is a formative experience, completely different from being a mere teaching assistant. It is also a skill that requires some talent and tons of practice – hence the talent needs to be identified early. Also, many undergrad courses taught by full professors are just plain boring. For these reasons postdocs should have the opportunity to be considered as principal lecturers, if there is an indication that they can do it better than a faculty member. 16 | 17 | 6. **Allow postdocs to compete for the big grants.** A postdoc is someone who has already proven that he/she can do science, defend it and publish it; that is why he/she holds the PhD degree, which I see as the formal invitation to the grown-ups' game. So dear NSF and others, how about allowing postdocs to play the big game? They may not always succeed in getting the grant, but at least they will have the chance. And who else than young minds are better qualified to propose high risk – high gain projects (the golden calf of basic science)? 18 | 19 | 7. **A postdoc who secures funding should be allowed to tutor grad students.** This is self-explanatory and obvious. Such postdoc could also be given a *temporary* status of assistant professor (for the duration of the funding). 20 | 21 | 8. **Make permanent positions less permanent and subject to periodic reevaluation.** In other words, create vacant niches by penalizing below-average performance of professors; such penalty can be a decrease to an appropriate professional level, e.g. from professor back to postdoc, which is not the same as loss of the job. A modification of this idea would allow postdocs to challenge the faculty positions directly – if the challenger wins, he/she would swap the position with the professor. No jobs lost, no extra cash needed. 22 | 23 | 9. **Put lab assistants and technicians who contributed work as co-authors on the resulting publications.** They do care! It costs almost nothing, and it makes people feel involved, appreciated and interested in the subject. It also makes sense because they are real part of the research effort. Finally, it means brighter prospects for a postdoc who voluntarily or involuntarily goes for a technician position but still loves science, or hopes to get back to an academic position one day. 24 | 25 | 10. **Change the label from postdoc to researcher.** This is just to clarify what postdocs really are. 26 | -------------------------------------------------------------------------------- /2015_04_15_The_age_of_postdoc/the_age_of_postdoc.md~: -------------------------------------------------------------------------------- 1 | # The age of Postdoc: towards liberation of academic middle class 2 | 3 | Nature has an article about the daunting prospects for current postdocs in some countries ([Nature 520, 144-147](http://www.nature.com/news/the-future-of-the-postdoc-1.17253)). The article is stimulating but reports only a few relatively ad-hoc ways to fix the situation. In the discussion below the article Peter Jurica mentions *postdocs as middle class *(of academia). I think that this simple analogy can inspire more fundamental actions and paradigm shifts, potentially improving the system in favor of postdocs. I've tried to outline them below, building on my limited experience from the US, UK, France and the Czech Republic – hence, my (random) ideas will apply differently, or can even be irrelevant, in different countries and types of universities. The specific countries which I have in mind are US, UK and Germany, where postdocs have especially hard time seeing the future for all the bottlenecks. 4 | 5 | These are my suggestions: 6 | 7 | 1. **Acknowledge academic careers that do not involve managing people.** Some people excel in doing science, they can even be excellent collaborators, or they have great analytical skills, but are bad in managing people and leading a lab. Many of them would be happy with receiving lower salary relatively to a full lab leader, but they should still be given independence and treated better than today's postdocs. The system potentially misallocates human resources when it encourages leadership as the only logical step after postdoc; an excellent scientist who is a bad manager can be, in some circumstances, more valuable than an average scientist who is a good manager. 8 | 9 | 2. **Legitimize serial postdocs.** Many disciplines (journalism, art, photography, data analysis, consultancy, law, …) have the concept of a *freelancer* – someone who does intellectually demanding or creative work without being part of a big institution and without the stability of permanent position. Freelancing is a legitimate life-long career. Similarly, a highly qualified postdoc who moves from position to position (and place to place) on 1-4 year-ish basis, or who accepts smaller jobs from different institutions, could be considered a research freelancer – a legitimate academic career for those who are happy with the constant change. I see this as an option for research qualifications that involve quantitative and writing skills, as these are transferable between sub-disciplines. I also believe that a senior serial postdoc (an unthinkable and almost pejorative status these days) could have a unique perspective which a senior professor would hardly reach. 10 | 11 | 3. **Allocate resources from senior faculty positions to the academic middle class.** Well, if there is not enough cash flowing into the system, one solution (left wing, communist, egalitarian, … start throwing the eggs) to brighten the future of early-stage academics could be to redistribute the cash more evenly. For example, in the US system, the raise of prestige, income, job security and power along the progressing career in academia could be made more gradual, without the abrupt step (bottleneck) between a postdoc and an assistant professor. This would increase opportunities for the early-stage researchers, and it could be sustained without pouring more money into the system. Where it would hurt, however, are the higher positions of today's academic hierarchy. Yet as a postdoc and father myself, I am more comfortable with more opportunities and slower raise in future pay, than with a prospect of hitting a high-paid high-security jackpot, but with low probability. 12 | 13 | 4. **Clarify and possibly reduce the power of advisors over postdocs.** It is crucial for the future of a postdoc to build his/her own research direction, independent on any advisor. Many advisors recognize that, but not all – the postdoc stories from Ivy League medical schools are full of expressions such as 'slave'. I suggest that, by rule, anyone hired as a postdoc should be entitled to some days of research, per week basis, with zero involvement of the advisor – this entitlement could be a default and explicit part of postdoctoral contracts. I also think that some advisors don't realize that they are not the bosses, and postdocs are not their subordinates – an *advisor*'s role is to give *advice* (which is, by definition, not bonding) to someone who is on his/her own path. Maybe this could also be stated in every postdoctoral contract. 14 | 15 | 5. **Allow postdocs to compete for opportunities to teach as principal lecturers.** Postdocs are often experts in their field, so why not giving them the opportunity to influence undergrads as principal lecturers? Being responsible for a whole course is a formative experience, completely different from being a mere teaching assistant. It is also a skill that requires some talent and tons of practice – hence the talent needs to be identified early. Also, many undergrad courses taught by full professors are just plain boring. For these reasons postdocs should have the opportunity to be considered as principal lecturers, if there is an indication that they can do it better than a faculty member. 16 | 17 | 6. **Allow postdocs to compete for the big grants.** A postdoc is someone who has already proven that he/she can do science, defend it and publish it; that is why he/she holds the PhD degree, which I see as the formal invitation to the grown-ups' game. So dear NSF and others, how about allowing postdocs to play the big game? They may not always succeed in getting the grant, but at least they will have the chance. And who else than young minds are better qualified to propose high risk – high gain projects (the golden calf of basic science)? 18 | 19 | 7. **A postdoc who secures funding should be allowed to tutor grad students.** This is self-explanatory and obvious. Such postdoc could also be given a *temporary* status of assistant professor (for the duration of the funding). 20 | 21 | 8. **Make permanent positions less permanent and subject to periodic reevaluation.** In other words, create vacant niches by penalizing below-average performance of professors; such penalty can be a decrease to an appropriate professional level, e.g. from professor back to postdoc, which is not the same as loss of the job. A modification of this idea would allow postdocs to challenge the faculty positions directly – if the challenger wins, he/she would swap the position with the professor. No jobs lost, no extra cash needed. 22 | 23 | 9. **Put lab assistants and technicians who contributed work as co-authors on the resulting publications.** They do care! It costs almost nothing, and it makes people feel involved, appreciated and interested in the subject. It also makes sense because they are real part of the research effort. Finally, it means brighter prospects for a postdoc who voluntarily or involuntarily goes for a technician position but still loves science, or hopes to get back to an academic position one day. 24 | 25 | 10. **Change the label from 'postdoc' to 'researcher'.** This is just to clarify what postdocs really are. 26 | -------------------------------------------------------------------------------- /2015_05_13_Survival_analysis/.Rhistory: -------------------------------------------------------------------------------- 1 | install.packages("runjags") 2 | install.packages("rjags") 3 | library(survival) 4 | ?survreg 5 | cancer <- read.table("http://goo.gl/3cnoam", header=TRUE) 6 | cancer 7 | censored <- cancer$status==0 8 | is.censored <- censored*1 9 | t.to.death <- cancer$death 10 | t.to.death[censored] <- NA 11 | t.to.death 12 | t.cen <- rep(0, times=length(censored)) 13 | t.cen[censored] <- cancer$death[censored] 14 | t.cen 15 | # put the data together for JAGS 16 | cancer.data <- list(t.to.death = t.to.death, 17 | t.cen = t.cen, 18 | N = nrow(cancer), 19 | group = rep(1:4, each=30)) 20 | cancer.data 21 | head(cancer) 22 | cancer <- read.table("http://goo.gl/3cnoam", header=TRUE) 23 | head(cancer) 24 | roaches <- read.table("http://goo.gl/NvtpNb", header=TRUE) 25 | summary(roaches) 26 | m1 <- survreg(Surv(death, status) ~ group, dist="exponential") 27 | m1 <- survreg(Surv(death, status) ~ group, dist="exponential", data=roaches) 28 | m1 29 | m1 <- survreg(Surv(death, status) ~ weight, dist="exponential", data=roaches) 30 | m1 31 | m1 <- survreg(Surv(death, status) ~ weight, data=roaches) 32 | m1 33 | m1 <- survreg(Surv(death, status) ~ weight, dist="exponential", data=roaches) 34 | m1 35 | summary(m1) 36 | head(roaches) 37 | censored <- roaches$status==0 38 | is.censored <- censored*1 39 | t.to.death <- roaches$death 40 | t.to.death[censored] <- NA 41 | t.to.death 42 | t.cen <- rep(0, times=length(censored)) 43 | t.cen[censored] <- roaches$death[censored] 44 | t.cen 45 | # put the data together for JAGS 46 | roaches.data <- list(t.to.death = t.to.death, 47 | t.cen = t.cen, 48 | N = nrow(roaches)) 49 | ?survreg 50 | head(roaches) 51 | censored <- roaches$status==0 52 | is.censored <- censored*1 53 | t.to.death <- roaches$death 54 | t.to.death[censored] <- NA 55 | t.to.death 56 | t.cen <- rep(0, times=length(censored)) 57 | t.cen[censored] <- roaches$death[censored] 58 | t.cen 59 | # put the data together for JAGS 60 | roaches.data <- list(t.to.death = t.to.death, 61 | t.cen = t.cen, 62 | N = nrow(roaches), 63 | weight = roaches$weight) 64 | cat(" 65 | model 66 | { 67 | # priors 68 | for(i in 1:N) 69 | { 70 | # prior lambda for each group 71 | # lambda[j] ~ dgamma(0.001, 0.001) 72 | beta0 ~ dnorm(0, 0.001) 73 | beta1 ~ dnorm(0, 0.001) 74 | } 75 | # likelihood 76 | for(i in 1:N) 77 | { 78 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 79 | t.to.death[i] ~ dexp(lambda[i]) 80 | log(lambda[i]) <- beta0 + beta1*weight[i] 81 | mu[i] <- 1/lambda[i] # mean time to death 82 | } 83 | } 84 | ", file="survival_roaches.txt") 85 | library(runjags) 86 | library(coda) 87 | roaches.fit <- run.jags(data = cancer.data, 88 | model = "roaches_cancer.txt", 89 | monitor = c("beta1"), 90 | sample = 1000, burnin = 1000, n.chains = 3) 91 | library(runjags) 92 | library(coda) 93 | roaches.fit <- run.jags(data = cancer.data, 94 | model = "survival_roaches.txt", 95 | monitor = c("beta1"), 96 | sample = 1000, burnin = 1000, n.chains = 3) 97 | censored <- roaches$status==0 98 | is.censored <- censored*1 99 | t.to.death <- roaches$death 100 | t.to.death[censored] <- NA 101 | t.to.death 102 | t.cen <- rep(0, times=length(censored)) 103 | t.cen[censored] <- roaches$death[censored] 104 | t.cen 105 | # put the data together for JAGS 106 | roaches.data <- list(t.to.death = t.to.death, 107 | t.cen = t.cen, 108 | N = nrow(roaches), 109 | weight = roaches$weight) 110 | library(runjags) 111 | library(coda) 112 | roaches.fit <- run.jags(data = roaches.data, 113 | model = "survival_roaches.txt", 114 | monitor = c("beta1"), 115 | sample = 1000, burnin = 1000, n.chains = 3) 116 | censored <- roaches$status==0 117 | is.censored <- censored*1 118 | t.to.death <- roaches$death 119 | t.to.death[censored] <- NA 120 | t.to.death 121 | t.cen <- rep(0, times=length(censored)) 122 | t.cen[censored] <- roaches$death[censored] 123 | t.cen 124 | # put the data together for JAGS 125 | roaches.data <- list(t.to.death = t.to.death, 126 | t.cen = t.cen, 127 | N = nrow(roaches), 128 | weight = roaches$weight) 129 | cat(" 130 | model 131 | { 132 | # priors 133 | for(i in 1:N) 134 | { 135 | # prior lambda for each group 136 | # lambda[j] ~ dgamma(0.001, 0.001) 137 | beta0 ~ dnorm(0, 0.001) 138 | beta1 ~ dnorm(0, 0.001) 139 | } 140 | # likelihood 141 | for(i in 1:N) 142 | { 143 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 144 | t.to.death[i] ~ dexp(lambda[i]) 145 | log(lambda[i]) <- beta0 + beta1*weight[i] 146 | mu[i] <- 1/lambda[i] # mean time to death 147 | } 148 | } 149 | ", file="survival_roaches.txt") 150 | library(runjags) 151 | library(coda) 152 | roaches.fit <- run.jags(data = roaches.data, 153 | model = "survival_roaches.txt", 154 | monitor = c("beta1"), 155 | sample = 1000, burnin = 1000, n.chains = 3) 156 | cat(" 157 | model 158 | { 159 | # priors 160 | # prior lambda for each group 161 | # lambda[j] ~ dgamma(0.001, 0.001) 162 | beta0 ~ dnorm(0, 0.001) 163 | beta1 ~ dnorm(0, 0.001) 164 | # likelihood 165 | for(i in 1:N) 166 | { 167 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 168 | t.to.death[i] ~ dexp(lambda[i]) 169 | log(lambda[i]) <- beta0 + beta1*weight[i] 170 | mu[i] <- 1/lambda[i] # mean time to death 171 | } 172 | } 173 | ", file="survival_roaches.txt") 174 | library(runjags) 175 | library(coda) 176 | roaches.fit <- run.jags(data = roaches.data, 177 | model = "survival_roaches.txt", 178 | monitor = c("beta1"), 179 | sample = 1000, burnin = 1000, n.chains = 3) 180 | summary(roaches.fit) 181 | summary(m1) 182 | library(runjags) 183 | library(coda) 184 | roaches.fit <- run.jags(data = roaches.data, 185 | model = "survival_roaches.txt", 186 | monitor = c("beta1"), 187 | sample = 1000, burnin = 1000, n.chains = 3) 188 | summary(roaches.fit) 189 | library(survival) 190 | roaches <- read.table("http://goo.gl/NvtpNb", header=TRUE) 191 | summary(roaches) 192 | m1 <- survreg(Surv(death, status) ~ weight, dist="exponential", data=roaches) 193 | summary(m1) 194 | censored <- roaches$status==0 195 | is.censored <- censored*1 196 | t.to.death <- roaches$death 197 | t.to.death[censored] <- NA 198 | t.to.death 199 | t.cen <- rep(0, times=length(censored)) 200 | t.cen[censored] <- roaches$death[censored] 201 | t.cen 202 | # put the data together for JAGS 203 | roaches.data <- list(t.to.death = t.to.death, 204 | t.cen = t.cen, 205 | N = nrow(roaches), 206 | weight = roaches$weight) 207 | cat(" 208 | model 209 | { 210 | # priors 211 | # prior lambda for each group 212 | # lambda[j] ~ dgamma(0.001, 0.001) 213 | beta0 ~ dnorm(0, 0.001) 214 | beta1 ~ dnorm(0, 0.001) 215 | # likelihood 216 | for(i in 1:N) 217 | { 218 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 219 | t.to.death[i] ~ dexp(lambda[i]) 220 | log(lambda[i]) <- beta0 + beta1*weight[i] 221 | mu[i] <- 1/lambda[i] # mean time to death 222 | } 223 | } 224 | ", file="survival_roaches.txt") 225 | library(runjags) 226 | library(coda) 227 | roaches.fit <- run.jags(data = roaches.data, 228 | model = "survival_roaches.txt", 229 | monitor = c("beta0", "beta1"), 230 | sample = 1000, burnin = 1000, n.chains = 3) 231 | summary(roaches.fit) 232 | library(runjags) 233 | library(coda) 234 | roaches.fit <- run.jags(data = roaches.data, 235 | model = "survival_roaches.txt", 236 | monitor = c("beta0", "beta1"), 237 | sample = 10000, burnin = 5000, n.chains = 3) 238 | summary(roaches.fit) 239 | m1 <- survreg(Surv(death, status) ~ weight, dist="exponential", data=roaches) 240 | summary(m1) 241 | summary(roaches.fit) 242 | summary(roaches.fit) 243 | summary(roaches.fit) 244 | m1 <- survreg(Surv(death, status) ~ weight, dist="exponential", data=roaches) 245 | summary(m1) 246 | exp(-2.4) 247 | cat(" 248 | model 249 | { 250 | # priors 251 | beta0 ~ dnorm(0, 0.001) 252 | beta1 ~ dnorm(0, 0.001) 253 | # likelihood 254 | for(i in 1:N) 255 | { 256 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 257 | t.to.death[i] ~ dexp(lambda[i]) 258 | #log(lambda[i]) <- beta0 + beta1*weight[i] 259 | #mu[i] <- 1/lambda[i] # mean time to death 260 | lambda[i] <- 1/mu[i] 261 | log(mu[i]) <- beta0 + beta1*weight[i] 262 | } 263 | } 264 | ", file="survival_roaches.txt") 265 | library(runjags) 266 | library(coda) 267 | roaches.fit <- run.jags(data = roaches.data, 268 | model = "survival_roaches.txt", 269 | monitor = c("beta0", "beta1"), 270 | sample = 10000, burnin = 5000, n.chains = 3) 271 | summary(roaches.fit) 272 | m1 <- survreg(Surv(death, status) ~ weight, dist="exponential", data=roaches) 273 | summary(m1) 274 | hist(roaches.fit) 275 | plot(roaches.fit) 276 | par(mfrow=c(1,2)) 277 | densplot(as.mcmc(cancer.fit), xlim=c(2,20)) 278 | densplot(as.mcmc(roaches.fit), xlim=c(2,20)) 279 | densplot(as.mcmc(roaches.fit)) 280 | as.mcmc(roaches.fit) 281 | a <- as.mcmc(roaches.fit) 282 | str(a) 283 | ?densplot 284 | densplot 285 | class(a) 286 | a[[1]] 287 | a[[2]] 288 | a$beta0 289 | a@beta0 290 | str(a) 291 | head(a) 292 | densplot(as.mcmc(roaches.fit)[,1]) 293 | densplot(as.mcmc(roaches.fit)$beta0) 294 | densplot(as.mcmc(roaches.fit)[,"beta0"]) 295 | densplot(as.mcmc(roaches.fit)[,"beta0"]) 296 | densplot(as.mcmc(roaches.fit)[,"beta1"]) 297 | m1 298 | m1$coef 299 | abline(v=m1$coef[1]) 300 | summary(roaches.fit) 301 | par(mfrow=c(1,2)) 302 | densplot(as.mcmc(roaches.fit)[,"beta0"]) 303 | abline(v=m1$coef[1]) 304 | densplot(as.mcmc(roaches.fit)[,"beta1"]) 305 | abline(v=m1$coef[2]) 306 | summary(roaches.fit) 307 | par(mfrow=c(1,2)) 308 | densplot(as.mcmc(roaches.fit)[,"beta0"], xlim=c(1,4)) 309 | abline(v=m1$coef[1]) 310 | densplot(as.mcmc(roaches.fit)[,"beta1"]) 311 | abline(v=m1$coef[2]) 312 | install.packages("runjags") 313 | -------------------------------------------------------------------------------- /2015_05_13_Survival_analysis/ext_model.txt: -------------------------------------------------------------------------------- 1 | 2 | model 3 | { 4 | # prior 5 | P ~ dbeta(1,1) 6 | n.succ ~ dbin(P, n.trials) 7 | } 8 | -------------------------------------------------------------------------------- /2015_05_13_Survival_analysis/survival_analysis.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_05_13_Survival_analysis/survival_analysis.pdf -------------------------------------------------------------------------------- /2015_05_13_Survival_analysis/survival_cancer.txt: -------------------------------------------------------------------------------- 1 | 2 | model 3 | { 4 | # priors 5 | for(j in 1:4) 6 | { 7 | # prior lambda for each group 8 | lambda[j] ~ dgamma(0.001, 0.001) 9 | mu[j] <- 1/lambda[j] # mean time to death 10 | } 11 | # likelihood 12 | for(i in 1:N) 13 | { 14 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 15 | t.to.death[i] ~ dexp(lambda[group[i]]) 16 | } 17 | } 18 | -------------------------------------------------------------------------------- /2015_05_13_Survival_analysis/survival_exp.txt: -------------------------------------------------------------------------------- 1 | 2 | model 3 | { 4 | # prior 5 | lambda ~ dgamma(0.01, 0.01) 6 | 7 | # likelihood 8 | for(t in 1:N) 9 | { 10 | t.to.death[t] ~ dexp(lambda) 11 | } 12 | # mean time to death 13 | mu <- 1/lambda 14 | 15 | # predictions 16 | for(i in 1:new.N) 17 | { 18 | S.t[i] <- exp(-new.t[i]/mu) 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /2015_05_13_Survival_analysis/survival_roaches.txt: -------------------------------------------------------------------------------- 1 | 2 | model 3 | { 4 | # priors 5 | beta0 ~ dnorm(0, 0.001) 6 | beta1 ~ dnorm(0, 0.001) 7 | 8 | # likelihood 9 | for(i in 1:N) 10 | { 11 | is.censored[i] ~ dinterval(t.to.death[i], t.cen[i]) 12 | t.to.death[i] ~ dexp(lambda[i]) 13 | # NOTE: I am linking mu, not lambda to the predictor (unlike above) 14 | # log(lambda[i]) <- beta0 + beta1*weight[i] 15 | # mu[i] <- 1/lambda[i] # mean time to death 16 | 17 | lambda[i] <- 1/mu[i] 18 | log(mu[i]) <- beta0 + beta1*weight[i] 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /2015_05_18_Formats_in_ecological_journals/formats_in_ecological_jounrals.md: -------------------------------------------------------------------------------- 1 | --- 2 | output: html_document 3 | --- 4 | # Do ecological journals support LaTeX and open document formats? 5 | 6 | Last week I started to draft a manuscript that is heavy on the computation side and it uses large data.This requires everything to be well documented and organized, otherwise I get lost in my own code. The real challenge is then to confront my analyses with the co-authors and, ultimately, journal referees -- a little request ("oh, it would be nice to add this or that" or "you've forgot to log-transform this") can easily force me to re-analyze everything. 7 | 8 | One way to minimize the hassle and to prevent errors is to make everything transparent, [open and reproducible](https://ropensci.org/blog/2014/06/09/reproducibility/), and to link the the manuscript with the figures and tables produced directly by the code. An open framework that enables all of this is [LaTeX](https://en.wikipedia.org/wiki/LaTeX) with its `.tex` extension. 9 | 10 | Another 100% open format is [OpenDocument](https://en.wikipedia.org/wiki/OpenDocument) (with the `.odt` extension). It lacks the LaTeX's ability to link stuff and to deal with math, but it is the default format of [the most popular free office suite](https://www.libreoffice.org/), and so it is accessible to anyone with limited budget. Plus, it has all of the functionality of `.doc`, `.docx` and `.rtf`, including support for comments, revision control, equations, or Zotero references. 11 | 12 | In contrast and [in spite of some theoretical claims](https://en.wikipedia.org/wiki/Office_Open_XML), Microsoft's `.doc` and `.docx` are not open in practice -- a painful experience of anyone who tried to use LibreOffice to open an MS Word document containing equations, comments or reference fields. 13 | 14 | But back to my manuscript; because of its computational nature I'd prefer to write it in LaTeX; now I need to choose the suitable journal, which I've found to be *Global Change Biology*. However, when I check the author guidelines I realize that it doesn't take LaTeX. So I turn to *Global Ecology and Biogeography*, then to *Ecography*, only to realize that none of them takes LaTeX. 15 | 16 | So I've made a quick research, checking the acceptable manuscript formats in major ecological journals, plus Nature, Science, PLOS and PNAS. Here is what I've found: 17 | 18 | | Journal | Accepted formats | Accepts LaTeX? | 19 | |-------------------------------------------|-----------------------------------|----------------| 20 | | Diversity and Distributions | ? | ? | 21 | | Journal of Ecology | Word processor | ? | 22 | | Trends in Ecology and Evolution | Word | no | 23 | | Frontiers in Ecology and the Environment | .doc, .docx | no | 24 | | Ecography | .doc, .docx, .pdf | no | 25 | | Oecologia | Word, .rtf, .pdf | ? | 26 | | Conservation Biology | Word | no | 27 | | Functional ecology | Word, .rtf | no | 28 | | Global Change Biology | Word, .wpd, .rtf, .ps | no | 29 | | Journal of Biogeography | .doc, .docx, .rtf | no | 30 | | Oikos | Word, .rtf, .ps | no | 31 | | American Naturalist | Word, .rtf, .tex | yes | 32 | | Ann. Rev. of Ecol., Evol. and Systematics | .doc, .rtf, .tex | yes | 33 | | Biological Conservation | Word, .pdf, .tex | yes | 34 | | Global Ecology and Biogeography | Word, .tex | yes | 35 | | Journal of Animal Ecology | Word, .tex | yes | 36 | | Journal of Applied Ecology | Word, .tex | yes | 37 | | Journal of Evolutionary Biology | .doc, .docx, .rtf, .tex | yes | 38 | | Molecular Ecology | Word, .pdf, .tex | yes | 39 | | Science | .docx, .pdf | yes | 40 | | Evolution | Word, .rtf, .tex | yes | 41 | | Methods in Ecology and Evolution | Word, .tex | yes | 42 | | PNAS | Word, .rtf, .tex | yes | 43 | | Proceedings B | Word, .pdf, .tex | yes | 44 | | PLOS BIOLOGY | .doc, .docx, .rtf, .pdf, .tex | yes | 45 | | PLOS ONE | .doc, .docx, .rtf, .pdf, .tex | yes | 46 | | Ecological Monographs | .doc, .docx, .wpd, rtf., .tex | yes | 47 | | Ecological Applications | .doc, .docx, .wpd, rtf., .tex | yes | 48 | | Ecology | .doc, .docx, .wpd, rtf., .tex | yes | 49 | | Ecology Letters | .doc, .rtf., .pdf, .tex | yes | 50 | | Nature | Word, .wpd, .eps, .ps, .rtf, .tex | yes | 51 | 52 | ## Some highlights 53 | 54 | * *Diversity and Distributions* gives almost no information on what it accepts. So I assume that it accepts everything, but I'd still prefer the journal to state it explicitly. 55 | 56 | * *Ecography* is the strictest supporter of Microsoft and its proprietary formats by not even allowing `.rtf`, let alone `.tex`. 57 | 58 | * *Journal of Ecology*, *TREE*, *Frontiers* , *Conservation Biology*, *Oecologia*, *Functional Ecology*, *Global Change Biology*, *Journal of Biogeography* and *Oikos* seem not to take LaTeX (if they do, I missed it). I was especially surprised by *Oikos* which used to be known for publishing papers in theoretical population ecology, something that looks especially crisp in LaTeX. 59 | 60 | * There is an explicit statement in *Methods in Ecology and Evolution* guidelines that they cannot take OpenDocument. I wonder why is that. 61 | 62 | * Surprisingly, `.odt` is almost never accepted. I guess that it is because it is assumed that `.rtf` is open enough, but maybe there is another reason; I'd be curious to hear that. 63 | 64 | * **There are journals that seem to make an effort to accommodate LaTeX -- and they are the majority, which is great!** It is the group at the lower part of the table starting with *American Naturalist*. Especially welcoming is the attitude of all ESA journals (*Ecology*, *Ecological Monographs*, *Ecological Applications*), *Royal Society* journals, *Ecology Letters*, *PLOS*, *PNAS* and *Nature*. If these guys can take it, why can't the others? 65 | -------------------------------------------------------------------------------- /2015_06_01_Math_language_extinctions_climate/math_and_climate.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Math vs. language, extinctions vs. climate change" 3 | author: "Petr Keil" 4 | date: "30/05/2015" 5 | output: 6 | html_document: default 7 | pdf_document: 8 | keep_tex: yes 9 | --- 10 | 11 | Two unrelated insights that I've recently had: 12 | 13 | ## Math isn't more abstract than language, it's just exact 14 | 15 | The reason is that mathematical notation is part of the same language that we speak, and any mathematical formula is, in fact, a written (or spoken) sentence. Example: 16 | 17 | $$F(x)=\int_a^b \! f(x) \, \mathrm{d}x$$ 18 | 19 | Which is equivalent to: *The value of function F at x equals to the area of the region in the xy-plane that is bounded by the graph of f, the x-axis, and the vertical lines at a and b.* Here, math is English. But it can also be written as Chinese or Hungarian, or any other language. 20 | 21 | I realized that what makes math special in comparison with broader language isn't its abstractness, but rather the opposite: **mathematics emerges where things are exactly defined and follow strict deductive logic**. The rest of language is a minefield of metaphors, logical loopholes, and ambiguities. 22 | 23 | Note: Of course language can be pretty abstract, and hence math can also be very abstract. My insight concerns the relationship between the two. 24 | 25 | ## Extinctions and climate change are inverse scientific problems 26 | 27 | **Climate change**: We know relatively well what has been happenning with climate during the last 100 years -- we have all the measurements. However, the link between the observed change and human activity has been questioned. 28 | 29 | **Species extinctions**: We know what happens when habitats shrink: species go extinct and we lose biodiversity. This stems from the relationship between area and the number of unique species in that area, the *Endemics-Area Relationship (EAR)*. We also know that habitable area of the most diverse areas has been shrinking due to human activity. However, we lack direct measurements of the actual loss of species -- not because it doesn't occur, but because an extinction is, when it happens, difficult to register. 30 | 31 | Putting them next to each other: We know that humans have been causing extinctions, but we can't observe them. In contrast, we are not sure if humans are causing climate change, but we can directly measure it. It's inversed, in a way. 32 | 33 | 34 | 35 | 36 | 37 | 38 | -------------------------------------------------------------------------------- /2015_06_01_Math_language_extinctions_climate/math_and_climate.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_01_Math_language_extinctions_climate/math_and_climate.pdf -------------------------------------------------------------------------------- /2015_06_01_Math_language_extinctions_climate/math_and_climate.tex: -------------------------------------------------------------------------------- 1 | \documentclass[]{article} 2 | \usepackage{lmodern} 3 | \usepackage{amssymb,amsmath} 4 | \usepackage{ifxetex,ifluatex} 5 | \usepackage{fixltx2e} % provides \textsubscript 6 | \ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex 7 | \usepackage[T1]{fontenc} 8 | \usepackage[utf8]{inputenc} 9 | \else % if luatex or xelatex 10 | \ifxetex 11 | \usepackage{mathspec} 12 | \usepackage{xltxtra,xunicode} 13 | \else 14 | \usepackage{fontspec} 15 | \fi 16 | \defaultfontfeatures{Mapping=tex-text,Scale=MatchLowercase} 17 | \newcommand{\euro}{€} 18 | \fi 19 | % use upquote if available, for straight quotes in verbatim environments 20 | \IfFileExists{upquote.sty}{\usepackage{upquote}}{} 21 | % use microtype if available 22 | \IfFileExists{microtype.sty}{% 23 | \usepackage{microtype} 24 | \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts 25 | }{} 26 | \usepackage[margin=1in]{geometry} 27 | \ifxetex 28 | \usepackage[setpagesize=false, % page size defined by xetex 29 | unicode=false, % unicode breaks when used with xetex 30 | xetex]{hyperref} 31 | \else 32 | \usepackage[unicode=true]{hyperref} 33 | \fi 34 | \hypersetup{breaklinks=true, 35 | bookmarks=true, 36 | pdfauthor={Petr Keil}, 37 | pdftitle={Math vs.~language, extinctions vs.~climate change}, 38 | colorlinks=true, 39 | citecolor=blue, 40 | urlcolor=blue, 41 | linkcolor=magenta, 42 | pdfborder={0 0 0}} 43 | \urlstyle{same} % don't use monospace font for urls 44 | \setlength{\parindent}{0pt} 45 | \setlength{\parskip}{6pt plus 2pt minus 1pt} 46 | \setlength{\emergencystretch}{3em} % prevent overfull lines 47 | \setcounter{secnumdepth}{0} 48 | 49 | %%% Use protect on footnotes to avoid problems with footnotes in titles 50 | \let\rmarkdownfootnote\footnote% 51 | \def\footnote{\protect\rmarkdownfootnote} 52 | 53 | %%% Change title format to be more compact 54 | \usepackage{titling} 55 | \setlength{\droptitle}{-2em} 56 | \title{Math vs.~language, extinctions vs.~climate change} 57 | \pretitle{\vspace{\droptitle}\centering\huge} 58 | \posttitle{\par} 59 | \author{Petr Keil} 60 | \preauthor{\centering\large\emph} 61 | \postauthor{\par} 62 | \predate{\centering\large\emph} 63 | \postdate{\par} 64 | \date{30/05/2015} 65 | 66 | 67 | 68 | 69 | \begin{document} 70 | 71 | \maketitle 72 | 73 | 74 | Two unrelated insights that I've recently had: 75 | 76 | \subsection{Math isn't more abstract than language, it's just 77 | exact}\label{math-isnt-more-abstract-than-language-its-just-exact} 78 | 79 | The reason is that mathematical notation is part of the same language 80 | that we speak, and any mathematical formula is, in fact, a written (or 81 | spoken) sentence. Example: 82 | 83 | \[F(x)=\int_a^b \! f(x) \, \mathrm{d}x\] 84 | 85 | Which is equivalent to: \emph{The value of function F at x equals to the 86 | area of the region in the xy-plane that is bounded by the graph of f, 87 | the x-axis, and the vertical lines at a and b.} Here, math is English. 88 | But it can also be written as Chinese or Hungarian, or any other 89 | language. 90 | 91 | I realized that what makes math special in comparison with broader 92 | language isn't its abstractness, but rather the opposite: 93 | \textbf{mathematics emerges where things are exactly defined and follow 94 | strict deductive logic}. The rest of language is a minefield of 95 | metaphors, logical loopholes, and ambiguities. 96 | 97 | Note: Of course language can be pretty abstract, and hence math can also 98 | be very abstract. My insight concerns the relationship between the two. 99 | 100 | \subsection{Extinctions and climate change are inverse scientific 101 | problems}\label{extinctions-and-climate-change-are-inverse-scientific-problems} 102 | 103 | \textbf{Climate change}: We know relatively well what has been 104 | happenning with climate during the last 100 years -- we have all the 105 | measurements. However, the link between the observed change and human 106 | activity has been questioned. 107 | 108 | \textbf{Species extinctions}: We know what happens when habitats shrink: 109 | species go extinct and we lose biodiversity. This stems from the 110 | relationship between area and the number of unique species in that area, 111 | the \emph{Endemics-Area Relationship (EAR)}. We also know that habitable 112 | area of the most diverse areas has been shrinking due to human activity. 113 | However, we lack direct measurements of the actual loss of species -- 114 | not because it doesn't occur, but because an extinction is, when it 115 | happens, difficult to register. 116 | 117 | Putting them next to each other: We know that humans have been causing 118 | extinctions, but we can't observe them. In contrast, we are not sure if 119 | humans are causing climate change, but we can directly measure it. It's 120 | inversed, in a way. 121 | 122 | \end{document} 123 | -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/.keil_poster.tex.swp: -------------------------------------------------------------------------------- 1 | \documentclass[25pt, a0paper, portrait, margin=0mm, innermargin=15mm, 2 | blockverticalspace=15mm, colspace=15mm, subcolspace=8mm]{tikzposter} 3 | 4 | % LATEX PACKAGES 5 | % -------------- 6 | 7 | \usepackage{graphicx} % package for inserting images, including .pdf 8 | \usepackage{adjustbox} % package for cropping images 9 | \usepackage[colorlinks=true, urlcolor=blue]{hyperref} % package for url and hyperlinks 10 | \usepackage{wrapfig} 11 | 12 | % TITLE, AUTHORS, INSTITUTE 13 | % ------------------------- 14 | 15 | \title{\textbf{Scaling of biodiversity loss in global coral reef fish}} 16 | \author{Petr Keil*, Nicholas J. Gotelli, Or Givan, Sebastien Villeger, 17 | Valeriano Parravicini, Michel Kulbicki, \\ Jonathan Belmaker \& the GASPAR team} 18 | 19 | \institute{*Center for Theoretical Study, Charles University in Prague, Czech Republic \\ 20 | Website: \texttt{\href{www.petrkeil.com}{www.petrkeil.com}}, 21 | E-mail: \texttt{\href{mailto:pkeil@seznam.cz}{pkeil@seznam.cz}}} 22 | 23 | % THEME SETTING 24 | % ------------- 25 | 26 | \usetheme{Rays} 27 | %\usecolorstyle[colorPalette=BlueGrayOrange]{Default} 28 | %\useblockstyle{Default} 29 | %\usebackgroundstyle{Default} 30 | \usetitlestyle{Wave} 31 | 32 | 33 | % HEAD 34 | % ---- 35 | 36 | \begin{document} 37 | \maketitle 38 | \begin{columns} 39 | 40 | % ------------------------ 41 | % COLUMN 1 --------------- 42 | 43 | \column{0.5} 44 | 45 | % AIMS 46 | % ---- 47 | 48 | \block{Aims} 49 | { 50 | \begin{enumerate} 51 | \item To predict loss of coral reef fish diversity 52 | based on \textbf{simulated scenarios of loss of coral 53 | reef area}. 54 | \item To improve methodology of extinction estimation by moving beyond 55 | simplistic \textbf{species-area (SAR)} and 56 | \textbf{endemics-area (EAR)} relationships that 57 | ignore spatial configuration of habitat loss. 58 | \item To base the predictions on coral reef 59 | \textbf{conservation status, area, remoteness, 60 | human pressure and climate}, and also species-specific 61 | vulnerabilities. 62 | \end{enumerate} 63 | 64 | \begin{center} 65 | \includegraphics[scale=1.5]{reef.jpg} 66 | 67 | \small{"Ancient coral reefs" by Heinrich Harder (1858-1935) - Wikimedia Commons} 68 | \end{center} 69 | 70 | 71 | } 72 | 73 | % DATA 74 | % ---- 75 | 76 | \block{Data} 77 | { 78 | \begin{itemize} 79 | \item Species distributional data from the 80 | \textbf{GASPAR database} (Kulbicki \textit{et al.} 2013, 81 | Pravicini \textit{et al.} 2013), with 5331 species in 82 | 249 grid 5-degree cells that cover 99.76\% of 83 | global coral reef area. 84 | \item Data on species-specific \textbf{vulnerability 85 | to fishing and coral bleaching} 86 | (Cheung \textit{et al.} 2005, Graham \textit{et al.} 2011). 87 | \item Data on five cell-specific 88 | characteristics (Paravicini \textit{et al.} 2014): 89 | \end{itemize} 90 | \begin{center} 91 | \adjustbox{trim={.0\width} {.0\height} {0.0\width} {.35\height},clip}% 92 | {\includegraphics[scale=1.4]{Figure_1.pdf}} 93 | \end{center} 94 | } 95 | 96 | % METHODS 97 | % ------- 98 | 99 | \block{Methods} 100 | { 101 | We run two distinct algorithms to simulate species loss: 102 | \begin{enumerate} 103 | \item \textbf{Site removal algorithm}. Large segments 104 | of coral reefs are completely destroyed stepwise; in each step 105 | all species living in the reef are wiped out. 106 | \item \textbf{Removing presences in site-species 107 | occurrence (SSO) matrix}. We subsequently remove the 108 | site-specific species occurrences (SSO), which are the 109 | presences in the binary site by species 110 | matrix (panel b). The exact sequence of SSO loss 111 | depends on the properties both sites and species. 112 | \end{enumerate} 113 | 114 | Species loss is summarized by an \textbf{extinction curve} 115 | describing relationship between area (or SSO) and number of lost 116 | species. Area under the curve (\textbf{AUC}) is our measure 117 | extinction magnitude. 118 | 119 | \includegraphics[scale=0.9]{Figure_2.pdf} 120 | } 121 | 122 | % ------------------------ 123 | % COLUMN 2 --------------- 124 | 125 | \column{0.5} 126 | 127 | % RESULTS 128 | % ------- 129 | 130 | \block{Results 1: Site removal algorithm} 131 | { 132 | \begin{center} 133 | \adjustbox{trim={.0\width} {.52\height} {0.0\width} {.0\height},clip}% 134 | {\includegraphics[scale=1.4]{Figure_3.pdf}} 135 | \end{center} 136 | } 137 | 138 | \block{Results 2: Removing presences in site-species occurrence (SSO) matrix} 139 | { 140 | \begin{center} 141 | \includegraphics[scale=1.5]{Figure_5.pdf} 142 | \end{center} 143 | 144 | } 145 | 146 | % CONCLUSIONS 147 | % ----------- 148 | 149 | \block{Conclusions} 150 | { 151 | \begin{itemize} 152 | \item Global and regional fish diversity is \textbf{surprisingly difficult to reduce} 153 | by removal of compact blocks of coral reefs. 154 | \item Area loss starting at the most \textbf{isolated} sites leads to most severe species loss. 155 | \item Area loss that initially avoids \textbf{protected areas} leads to the lowest species loss. This indicates that a non-random 156 | and unique part of reef fish diversity is covered by protected areas. 157 | \item Simulated species loss due to \textbf{fishing and coral bleaching} is 158 | higher than random loss. 159 | \item Including real-world variables into scenarios of habitat 160 | loss give different predictions than simple \textbf{SAR} models. 161 | \end{itemize} 162 | 163 | } 164 | 165 | % REFERENCES 166 | % ---------- 167 | 168 | \block{References} 169 | { 170 | \begin{small} 171 | 172 | \hangindent=2cm Cheung \textit{et al.} (2005) A fuzzy logic expert system to 173 | estimate intrinsic extinction vulnerabilities of marine fishes 174 | to fishing. \textit{Biological Conservation}, \textbf{124}: 97-111. 175 | 176 | \hangindent=2cm Graham \textit{et al.} (2011) Extinction vulnerability of coral 177 | reef fishes. \textit{Ecology Letters}, \textbf{14}: 341-348. 178 | 179 | \hangindent=2cm Kulbicki \textit{et al.} (2013) Global biogeography of reef 180 | fishes: a hierarchical quantitative delineation of regions. 181 | \textit{PLoS ONE}, \textbf{8}: e81847. 182 | 183 | \hangindent=2cm Paravicini \textit{et al.} (2013) Global patterns and predictors of 184 | tropical reef fish species richness. \textit{Ecography}, \textbf{36}: 1-9. 185 | 186 | \hangindent=2cm Paravicini \textit{et al.} (2014) Global mismatch between 187 | species richness and vulnerability of reef fish assemblages. 188 | \textit{Ecology Letters}, \textbf{17}: 1101-1110. 189 | 190 | \end{small} 191 | } 192 | 193 | % ACKNOWLEDGEMENTS 194 | % ---------------- 195 | \note[targetoffsetx=-8cm, targetoffsety=-9cm, width=25cm, innersep=1cm] 196 | { 197 | \begin{wrapfigure}{r}{3cm} 198 | \vspace{-23pt} 199 | \includegraphics[scale=3]{eu.jpg} 200 | 201 | \includegraphics[scale=3]{fp7.jpg} 202 | \end{wrapfigure} 203 | \textbf{Funding:} This research has received funding 204 | from the People Programme (Marie Curie Actions) of the EU 7th Framework 205 | Programme (FP7/2007-2013), REA grant agreement no. 302868. 206 | 207 | } 208 | 209 | 210 | \end{columns} 211 | 212 | % ---------------- 213 | \end{document} 214 | \endinput 215 | %% 216 | %% End of file 217 | -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/Figure_1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/Figure_1.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/Figure_2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/Figure_2.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/Figure_3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/Figure_3.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/Figure_5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/Figure_5.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/coding_conference_posters.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/coding_conference_posters.md -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/eu.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/eu.jpg -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/fp7.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/fp7.jpg -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/keil_poster.aux: -------------------------------------------------------------------------------- 1 | \relax 2 | \providecommand\hyper@newdestlabel[2]{} 3 | \providecommand\HyperFirstAtBeginDocument{\AtBeginDocument} 4 | \HyperFirstAtBeginDocument{\ifx\hyper@anchor\@undefined 5 | \global\let\oldcontentsline\contentsline 6 | \gdef\contentsline#1#2#3#4{\oldcontentsline{#1}{#2}{#3}} 7 | \global\let\oldnewlabel\newlabel 8 | \gdef\newlabel#1#2{\newlabelxx{#1}#2} 9 | \gdef\newlabelxx#1#2#3#4#5#6{\oldnewlabel{#1}{{#2}{#3}}} 10 | \AtEndDocument{\ifx\hyper@anchor\@undefined 11 | \let\contentsline\oldcontentsline 12 | \let\newlabel\oldnewlabel 13 | \fi} 14 | \fi} 15 | \global\let\hyper@last\relax 16 | \gdef\HyperFirstAtBeginDocument#1{#1} 17 | \providecommand\HyField@AuxAddToFields[1]{} 18 | \providecommand\HyField@AuxAddToCoFields[2]{} 19 | \providecommand \oddpage@label [2]{} 20 | -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/keil_poster.dvi: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/keil_poster.dvi -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/keil_poster.out: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/keil_poster.out -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/keil_poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/keil_poster.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/keil_poster.synctex.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/keil_poster.synctex.gz -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/keil_poster.tex: -------------------------------------------------------------------------------- 1 | \documentclass[25pt, a0paper, portrait, margin=0mm, innermargin=15mm, 2 | blockverticalspace=15mm, colspace=15mm, subcolspace=8mm]{tikzposter} 3 | 4 | % LATEX PACKAGES 5 | % -------------- 6 | 7 | \usepackage{graphicx} % package for inserting images, including .pdf 8 | \usepackage{adjustbox} % package for cropping images 9 | \usepackage[colorlinks=true, urlcolor=blue]{hyperref} % package for url and hyperlinks 10 | \usepackage{wrapfig} 11 | 12 | % TITLE, AUTHORS, INSTITUTE 13 | % ------------------------- 14 | 15 | \title{\textbf{Scaling of biodiversity loss in global coral reef fish}} 16 | \author{Petr Keil*, Nicholas J. Gotelli, Or Givan, Sebastien Villeger, 17 | Valeriano Parravicini, Michel Kulbicki, \\ Jonathan Belmaker \& the GASPAR team} 18 | 19 | \institute{*Center for Theoretical Study, Charles University in Prague, Czech Republic \\ 20 | Website: \texttt{\href{www.petrkeil.com}{www.petrkeil.com}}, 21 | E-mail: \texttt{\href{mailto:pkeil@seznam.cz}{pkeil@seznam.cz}}} 22 | 23 | % THEME SETTING 24 | % ------------- 25 | 26 | \usetheme{Rays} 27 | %\usecolorstyle[colorPalette=BlueGrayOrange]{Default} 28 | %\useblockstyle{Default} 29 | %\usebackgroundstyle{Default} 30 | \usetitlestyle{Wave} 31 | 32 | 33 | % HEAD 34 | % ---- 35 | 36 | \begin{document} 37 | \maketitle 38 | \begin{columns} 39 | 40 | % ------------------------ 41 | % COLUMN 1 --------------- 42 | 43 | \column{0.5} 44 | 45 | % AIMS 46 | % ---- 47 | 48 | \block{Aims} 49 | { 50 | \begin{enumerate} 51 | \item To predict loss of coral reef fish diversity 52 | based on \textbf{simulated scenarios of loss of coral 53 | reef area}. 54 | \item To improve methodology of extinction estimation by moving beyond 55 | simplistic \textbf{species-area (SAR)} and 56 | \textbf{endemics-area (EAR)} relationships that 57 | ignore spatial configuration of habitat loss. 58 | \item To base the predictions on coral reef 59 | \textbf{conservation status, area, remoteness, 60 | human pressure and climate}, and also species-specific 61 | vulnerabilities. 62 | \end{enumerate} 63 | 64 | \begin{center} 65 | \includegraphics[scale=1.5]{reef.jpg} 66 | 67 | \small{"Ancient coral reefs" by Heinrich Harder (1858-1935) - Wikimedia Commons} 68 | \end{center} 69 | 70 | 71 | } 72 | 73 | % DATA 74 | % ---- 75 | 76 | \block{Data} 77 | { 78 | \begin{itemize} 79 | \item Species distributional data from the 80 | \textbf{GASPAR database} (Kulbicki \textit{et al.} 2013, 81 | Pravicini \textit{et al.} 2013), with 5331 species in 82 | 249 grid 5-degree cells that cover 99.76\% of 83 | global coral reef area. 84 | \item Data on species-specific \textbf{vulnerability 85 | to fishing and coral bleaching} 86 | (Cheung \textit{et al.} 2005, Graham \textit{et al.} 2011). 87 | \item Data on five cell-specific 88 | characteristics (Paravicini \textit{et al.} 2014): 89 | \end{itemize} 90 | \begin{center} 91 | \adjustbox{trim={.0\width} {.0\height} {0.0\width} {.35\height},clip}% 92 | {\includegraphics[scale=1.4]{Figure_1.pdf}} 93 | \end{center} 94 | } 95 | 96 | % METHODS 97 | % ------- 98 | 99 | \block{Methods} 100 | { 101 | We run two distinct algorithms to simulate species loss: 102 | \begin{enumerate} 103 | \item \textbf{Site removal algorithm}. Large segments 104 | of coral reefs are completely destroyed stepwise; in each step 105 | all species living in the reef are wiped out. 106 | \item \textbf{Removing presences in site-species 107 | occurrence (SSO) matrix}. We subsequently remove the 108 | site-specific species occurrences (SSO), which are the 109 | presences in the binary site by species 110 | matrix (panel b). The exact sequence of SSO loss 111 | depends on the properties both sites and species. 112 | \end{enumerate} 113 | 114 | Species loss is summarized by an \textbf{extinction curve} 115 | describing relationship between area (or SSO) and number of lost 116 | species. Area under the curve (\textbf{AUC}) is our measure 117 | extinction magnitude. 118 | 119 | \includegraphics[scale=0.9]{Figure_2.pdf} 120 | } 121 | 122 | % ------------------------ 123 | % COLUMN 2 --------------- 124 | 125 | \column{0.5} 126 | 127 | % RESULTS 128 | % ------- 129 | 130 | \block{Results 1: Site removal algorithm} 131 | { 132 | \begin{center} 133 | \adjustbox{trim={.0\width} {.52\height} {0.0\width} {.0\height},clip}% 134 | {\includegraphics[scale=1.4]{Figure_3.pdf}} 135 | \end{center} 136 | } 137 | 138 | \block{Results 2: Removing presences in site-species occurrence (SSO) matrix} 139 | { 140 | \begin{center} 141 | \includegraphics[scale=1.5]{Figure_5.pdf} 142 | \end{center} 143 | 144 | } 145 | 146 | % CONCLUSIONS 147 | % ----------- 148 | 149 | \block{Conclusions} 150 | { 151 | \begin{itemize} 152 | \item Global and regional fish diversity is \textbf{surprisingly difficult to reduce} 153 | by removal of compact blocks of coral reefs. 154 | \item Area loss starting at the most \textbf{isolated} sites leads to most severe species loss. 155 | \item Area loss that initially avoids \textbf{protected areas} leads to the lowest species loss. This indicates that a non-random 156 | and unique part of reef fish diversity is covered by protected areas. 157 | \item Simulated species loss due to \textbf{fishing and coral bleaching} is 158 | higher than random loss. 159 | \item Including real-world variables into scenarios of habitat 160 | loss give different predictions than simple \textbf{SAR} models. 161 | \end{itemize} 162 | 163 | } 164 | 165 | % REFERENCES 166 | % ---------- 167 | 168 | \block{References} 169 | { 170 | \begin{small} 171 | 172 | \hangindent=2cm Cheung \textit{et al.} (2005) A fuzzy logic expert system to 173 | estimate intrinsic extinction vulnerabilities of marine fishes 174 | to fishing. \textit{Biological Conservation}, \textbf{124}: 97-111. 175 | 176 | \hangindent=2cm Graham \textit{et al.} (2011) Extinction vulnerability of coral 177 | reef fishes. \textit{Ecology Letters}, \textbf{14}: 341-348. 178 | 179 | \hangindent=2cm Kulbicki \textit{et al.} (2013) Global biogeography of reef 180 | fishes: a hierarchical quantitative delineation of regions. 181 | \textit{PLoS ONE}, \textbf{8}: e81847. 182 | 183 | \hangindent=2cm Paravicini \textit{et al.} (2013) Global patterns and predictors of 184 | tropical reef fish species richness. \textit{Ecography}, \textbf{36}: 1-9. 185 | 186 | \hangindent=2cm Paravicini \textit{et al.} (2014) Global mismatch between 187 | species richness and vulnerability of reef fish assemblages. 188 | \textit{Ecology Letters}, \textbf{17}: 1101-1110. 189 | 190 | \end{small} 191 | } 192 | 193 | % ACKNOWLEDGEMENTS 194 | % ---------------- 195 | \note[targetoffsetx=-8cm, targetoffsety=-9cm, width=25cm, innersep=1cm] 196 | { 197 | \begin{wrapfigure}{r}{3cm} 198 | \vspace{-23pt} 199 | \includegraphics[scale=3]{eu.jpg} 200 | 201 | \includegraphics[scale=3]{fp7.jpg} 202 | \end{wrapfigure} 203 | \textbf{Funding:} This research has received funding 204 | from the People Programme (Marie Curie Actions) of the EU 7th Framework 205 | Programme (FP7/2007-2013), REA grant agreement no. 302868. 206 | 207 | } 208 | 209 | 210 | \end{columns} 211 | 212 | % ---------------- 213 | \end{document} 214 | \endinput 215 | %% 216 | %% End of file 217 | -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/merged.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/merged.jpg -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/pdf_merging_command.txt: -------------------------------------------------------------------------------- 1 | Here is the bash command to tile multiple .pdfs into one .jpg. 2 | It is using Imagemagick: 3 | 4 | montage -geometry +5+5 -border 100 -background '#000000' -fill '#000000' -bordercolor '#000000' poster1.pdf poster2.pdf poster3.pdf merged.jpg 5 | -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/poster1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/poster1.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/poster2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/poster2.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/poster3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/poster3.pdf -------------------------------------------------------------------------------- /2015_06_10_Coding_conference_posters/reef.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_10_Coding_conference_posters/reef.jpg -------------------------------------------------------------------------------- /2015_06_26_The_Sixth_extinction_review/The_sixth_extinction_review.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: '2015 Pulitzer prize awarded to book on conservation biogeography: The Sixth 3 | Extinction by Elisabeth Kolbert reviewed' 4 | author: "Petr Keil" 5 | output: html_document 6 | --- 7 | 8 | *Elisabeth Kolbert: The Sixth Extinction. Published on Feb 2014 by Henry Holt & Co., New York.* 9 | 10 | The [2015 Pulitzer prize](http://www.pulitzer.org/awards/2015) in the General Nonfiction category (awarded on May 28) went to Elisabeth Kolbert for her book on the science of extinction, which also happens to be an intro to conservation biogeography. 11 | 12 | The book has a wide scope stuffed to 220 pages of swift prose, covering the deepest paleontological past as well as the very present, dealing, among many other things, with mainland forests, coral reefs, and processes such as species invasions, ocean acidification, or habitat loss and fragmentation. 13 | 14 | Each chapter builds on conversations with experts, often set directly in their working environment -- Kolbert interviews researchers while hunting for 15 | extinct frogs in tropical forests, exploring caves of upstate New York, or fighting her way through rising tide on an atol. The conversations are intertwined with historical accounts and explanations of theoretical concepts, often zooming 16 | in and out across spatial and temporal scales, from individual animals to entire 17 | biomes, from instant events to eras. 18 | 19 | Readers hungry for new facts and familiar with [Lomolino et al.'s 20 | Biogeography](http://www.sinauer.com/biogeography-4e.html) will possibly yawn over 21 | Kolbert's book. However, for me the most inspiring were not the facts, but the 22 | author's effort to directly experience and even physically touch her subjects, 23 | while still keeping the large-scale, synthetic and didactic perspective, combined 24 | with an almost novel-like feeling to the style. 25 | 26 | As such The Sixth Extinction is a great introductory book for students that are new to the field of conservation biogeography and extinction science, as well as it is 27 | an educative summer read for the general public. 28 | 29 | ## Quotes I find inspiring 30 | 31 | **On the game-changing abruptness of mass extinction events:** 32 | 33 | > Whatever the explanation [of the extinction of dinosaurs and survival of mammals], the contrasting fate of the two groups raises a key point. Everything (and everyone) alive today is descended from an organism that somehow survived the impact. But it does not follow from this that they (or we) are any better adapter. In times of extreme stress, the whole concept of fitness, at least in the Darwinian sense, loses its meaning: how could a creature be adapted, either well or ill, for conditions it has never before encountered in its entire evolutionary history? ... At such moments, what Paul Taylor, a paleontologist at London's Natural History Museum, calls "the rules of the survival game" abruptly change. (p. 77) 34 | 35 | This is inaccurate as even extreme stress is a form of 36 | natural selection -- the game has not changed, it just gets too fast. It is nevertheless daunting (and almost biblical) that extreme events exterminate even those life forms that appear successful and dominant. 37 | 38 | **On the deadliness of low but sustained pressure on populations over long time periods:** 39 | 40 | > All that would have been required to do them in [i.e. for *Homo sapiens sapiens* to drive the Neanderthals and Denisovans to extinction] would have been a *sustained* downward pressure on the number of breeding adults. (p. 208) 41 | 42 | This motive appears throughout the whole book, and is mentioned even in 43 | the context of mammoths, hairy rhinos, and other big stuff. It also applies well to the pressure that we, humans, put on the current populations. 44 | 45 | **On reshuffling the deck chairs on the Titanic:** 46 | 47 | > To argue that the current extinction event could be 48 | averted if people just cared more and were willing to 49 | make more sacrifices is not wrong, exactly; still, it 50 | misses the point. It doesn't much matter whether people 51 | care or don't care. What matters is that people change 52 | the world (p. 218). 53 | 54 | 55 | ## Issues covered by the book 56 | - Recent **extinctions of amphibians** and the spread of the Chytrid fungus *Batrachochytridium dendrobatidis*. 57 | - The **ascent of paleontology** and the science of extinctions, with focus on *Georges Cuvier* and the discovery of mastodon skeletons. 58 | - *Charles Lyell*, the discovery of **plate tectonics and fluctuation of oceans**, and how this influenced Darwin. 59 | - The extinction of *Great auk*. 60 | - *Walter Alvarez* and the **impact event** at the end of Cretaceous. 61 | - Ammonite extinction, the **scarcity of fossil record** and **fossilization potential**. 62 | - The sudden **extinction of once dominant groups**, e.g. Graptolites in the end-Ordovician extinction. 63 | - History of the term **anthropocene** (coined by *Paul Crutzen*). 64 | - **Acidification of the oceans**, its role in at least two mass extinctions, plus a very long list of reasons why it's a major issue now (e.g. it increases the cost of calcification in corals by changing the saturation state of aragonite). 65 | - *Chris Langdon*'s experiments with corals in Biosphere 2. 66 | - **Macroecological patterns** such as latitudinal and altitudinal gradients, species-area relationships, and the ability of species to track changing climate. 67 | - The message from artificial **forest fragments** in Biological Dynamics of Forest Fragments Project (BDFFP) in Brazil. 68 | - **Species invasions**, homogenization of biota, and the bat white nose syndrome. 69 | - Sumatran rhinos, **ex-situ conservation**, **risks of being large** and the lethality of low but steady pressure over prolonged time frame. 70 | - Extinction of **Neanderthals** and Denisovans. 71 | 72 | ## Experts interviewed 73 | - Edgardo Griffith from [EVACC](http://amphibianrescue.org/tag/evacc/) on amphibians 74 | and the chytrid fungus. 75 | - [Neil Landmann](http://www.amnh.org/our-research/staff-directory/neil-landman) from American Museum of Natural History on ammonite extinction. 76 | - [Jan Zalasiewicz](http://www2.le.ac.uk/departments/geology/people/zalasiewicz-ja) from University of Leicester on stratigraphy and [Graptolites](https://en.wikipedia.org/wiki/Graptolithina). 77 | - [Jason Hall-Spencer](https://www.plymouth.ac.uk/staff/jason-hall-spencer) from Plymouth University on ocean acidification. 78 | - Stanford's [Ken Kaldeira](http://globalecology.stanford.edu/labs/caldeiralab/) on coral reefs. 79 | - [Miles Silman](http://users.wfu.edu/silmanmr/labpage/index.html) from Wake Forest University on elevation gradients and species tracking climate. 80 | - [Thomas E. Lovejoy](https://en.wikipedia.org/wiki/Thomas_Lovejoy) on [BDFFP](http://stri.si.edu/english/research/facilities/affiliated_stations/bdffp/index.php) 81 | plots and fragmentation of tropical forests. 82 | - [Mario Cohn-Haft](http://www.researchgate.net/profile/Mario_Cohn-Haft/publications) on BDFFP birds. 83 | - [Al Hicks](http://www.dec.ny.gov/pubs/46905.html) from NY Dpt. of Environmental Conservation on bat extinctions and the white nose syndrome. 84 | - [Terri Roth](http://cincinnatizoo.org/wp-content/uploads/2011/03/Roth-Bio-2012.pdf) from Cincinnati Zoo 85 | - [Svante Pääbo](https://en.wikipedia.org/wiki/Svante_P%C3%A4%C3%A4bo) from Max Planck Institute on Neanderthals. 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | -------------------------------------------------------------------------------- /2015_06_26_The_Sixth_extinction_review/The_sixth_extinction_review.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_06_26_The_Sixth_extinction_review/The_sixth_extinction_review.pdf -------------------------------------------------------------------------------- /2015_08_16_Djokovic/.Rhistory: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_08_16_Djokovic/.Rhistory -------------------------------------------------------------------------------- /2015_08_16_Djokovic/cannabis.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_08_16_Djokovic/cannabis.jpg -------------------------------------------------------------------------------- /2015_08_16_Djokovic/djokovic (xps's conflicted copy 2015-08-17).Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "What did Novak Djokovic really smell?" 3 | output: html_document 4 | --- 5 | 6 | As [reported by BBC](http://www.bbc.com/sport/0/tennis/33951245), World number one Novak Djokovic complained to the umpire that he could smell cannabis on court during his Rogers Cup win over Jeremy Chardy. After winning the first set, the Serb approached the chair, saying: 7 | 8 | > "Someone is smoking weed, I can smell it, I'm getting dizzy. ... You can't believe how bad it was. ... Whoever it is, I hope he doesn't come back tomorrow. He's probably on the seventh sky somewhere." 9 | 10 | When I moved to the US in 2011, I also smelled cannabis everywhere, sometimes the foul smell crawled into the bedroom through the open window and woke us up. 11 | 12 | Then I noticed that people must be smoking weed on highways -- on empty highways in the middle of nowhere in the middle of the night. Everything is quiet, you drive with the high beams through the hot summer night, no signs of a human being for miles, and all of a sudden the AC starts pumping in the smell of [Dutch Passion Skunk #1](http://www.dutch-passion.nl/en/cannabis-seeds/product/skunk-1/). 13 | 14 | Then I learned what it really was: 15 | 16 | !["Mephitis mephitis Striped skunk alerted Las Trampas sm" by Dcrjsr - Own work. Licensed under CC BY 4.0 via Commons - https://commons.wikimedia.org/wiki/File:Mephitis_mephitis_Striped_skunk_alerted_Las_Trampas_sm.jpg#/media/File:Mephitis_mephitis_Striped_skunk_alerted_Las_Trampas_sm.jpg](skunk.jpg) 17 | 18 | Now could Novak Djokovic in fact smell Striped skunk (*Mephitis mephitis*) instead of cannabis? 19 | 20 | Let's check if the skunk's [geographic range on Map of Life](http://species.mol.org/species/map/Mephitis_mephitis) covers Montreal, Canada (that's where semifinals of Rogers Cup are located): 21 | 22 | ![](skunkmap.png) 23 | 24 | Let's check if there are parks and gardens around the main tennis court: 25 | 26 | ![](venue.png) 27 | 28 | With this I conclude that Novak Djokovic could very well just smell a skunk, and as a European he could have mistaken if for cannabis. 29 | 30 | But then I cannot really reject the possibility that someone was smoking cannabis at the stadium either. In fact, I wonder how many of 31 | 32 | 33 | 34 | 35 | 36 | -------------------------------------------------------------------------------- /2015_08_16_Djokovic/djokovic.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "What did Novak Djokovic really smell?" 3 | output: 4 | html_document: 5 | fig_width: 5 6 | --- 7 | 8 | As [reported by BBC](http://www.bbc.com/sport/0/tennis/33951245), men's tennis World number one Novak Djokovic complained to the umpire that he could smell cannabis on court during his Rogers Cup win over Jeremy Chardy. After winning the first set, the Serb approached the chair, saying: 9 | 10 | > "Someone is smoking weed, I can smell it, I'm getting dizzy." 11 | 12 | And later: 13 | 14 | > "You can't believe how bad it was. ... Whoever it is, I hope he doesn't come back tomorrow. He's probably on the seventh sky somewhere." 15 | 16 | When I moved to the US in 2011, I also smelled cannabis everywhere, sometimes the foul smell crawled into the bedroom through the open window and woke me up. 17 | 18 | Then I noticed that Americans smoke cannabis on highways -- actually, on empty highways in the middle of nowhere. Imagine that everything is quiet, you drive all alone through the hot summer night, no signs of a human being for miles, and all of a sudden the ventilation starts pumping in the smell of [Dutch Passion Skunk #1](http://www.dutch-passion.nl/en/cannabis-seeds/product/skunk-1/). 19 | 20 | Then I learned what it really was: 21 | 22 | !["Mephitis mephitis Striped skunk alerted Las Trampas sm" by Dcrjsr - Own work. Licensed under CC BY 4.0 via Commons - https://commons.wikimedia.org/wiki/File:Mephitis_mephitis_Striped_skunk_alerted_Las_Trampas_sm.jpg#/media/File:Mephitis_mephitis_Striped_skunk_alerted_Las_Trampas_sm.jpg](skunk.jpg) 23 | 24 | Could Novak Djokovic in fact smell Striped skunk (*Mephitis mephitis*) instead of cannabis? 25 | 26 | Let's check [Map of Life](http://species.mol.org/species/map/Mephitis_mephitis) if Striped skunk's geographic range covers Montreal (that's where semifinals of Rogers Cup take place): 27 | 28 | ![](skunkmap.jpg) 29 | 30 | And let's check if there are parks and gardens around the main tennis court: 31 | 32 | ![](venue.jpg) 33 | 34 | Green suburbs everywere, ideal habitat for Striped skunk. So Novak Djokovic could have actually smell the animal instead of the plant. 35 | 36 | But then I cannot really reject the possibility that someone really was smoking cannabis in the stadium. Actually, I wonder how many of the Striped skunk detection points in GBIF and Map of Life are, in fact, cannabis. 37 | 38 | 39 | 40 | 41 | 42 | -------------------------------------------------------------------------------- /2015_08_16_Djokovic/skunk.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_08_16_Djokovic/skunk.jpg -------------------------------------------------------------------------------- /2015_08_16_Djokovic/skunkmap.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_08_16_Djokovic/skunkmap.jpg -------------------------------------------------------------------------------- /2015_08_16_Djokovic/venue.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_08_16_Djokovic/venue.jpg -------------------------------------------------------------------------------- /2015_10_02_Light_diversity/Light_diversity.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Made with Remarkable! 7 | 8 | 9 | 12 | 13 | 14 |

15 | Light diversity: obscuring the observed species 16 |

17 |

18 | Yesterday during an 19 | 20 | iDiv 21 | 22 | seminar we stumbled across 23 | 24 | dark diversity 25 | 26 | . For those unfamiliar with the concept, it has been defined by 27 | 28 | Pärtel 29 | 30 | et al. 31 | 32 | (2011) 33 | 34 | : 35 |

36 |
37 |

38 | It is possible to specify species that belong to a particular species pool but that are not locally present. We call this unaccounted set of species ‘dark diversity’. 39 |

40 |
41 |

42 | So it is a set of species that we, for whatever reason, expect to be at a locality, but are not observed there. 43 |

44 |

45 | I raised a concern that, for the sake of completeness, we should also consider 46 | 47 | light diversity 48 | 49 | , which would be the 50 | 51 | set of species that are not in the species pool, but we observe them anyway 52 | 53 | . 54 |

55 |

56 | Examples: 57 |

58 |
    59 |
  1. 60 | Observations of yet undescribed species. 61 |
    2. 62 |
  2. 63 | Observations of vagrant and migrant species (species that do not reproduce at the locality). 64 |
    3. 65 |
  3. 66 | Invasive species (thanks Marten). 67 |
    4. 68 |
  4. 69 | Species from taxonomic groups that are not part of our research, such as a cow in the middle of a botanical plot. 70 |
    5. 71 |
72 |

73 | Depending on the context, light diversity could be also called inconvenient diversity, nuisance diversity, unexpected diversity, surprising diversity, undesirable diversity, avoided diversity, or denial diversity. 74 |

75 |

76 | In order to make sense, dark diversity requires species pool to be philosophically 77 | 78 | realistic 79 | 80 | , i. e. we need to believe in the existence of a real species pool out there in the real world. In contrast, light diversity requires species pool to be 81 | 82 | nominalistic 83 | 84 | or 85 | 86 | conceptual 87 | 88 | , i.e. it is the species pool that we have in mind, or that we agree on in a broader community. 89 |

90 |

91 | The idea of light diversity was received with a lukewarm amusement, and I am not sure how useful it could be, but then I am also unsure why we should have dark diversity. Perhaps I should just get back to work, to the usual and dull grey diversity. 92 |

93 |

94 | 95 | Reference 96 | 97 |

98 |

99 | Pärtel M., Szava-Kovats R. & Zobel M. (2011) Dark diversity: shedding light on absent species. 100 | 101 | Trends in Ecology and Evolution 102 | 103 | , 104 | 105 | 26 106 | 107 | : 124-128. 108 |

109 | 111 | 114 | 116 | 119 | 120 | -------------------------------------------------------------------------------- /2015_10_02_Light_diversity/light_diversity: -------------------------------------------------------------------------------- 1 | # Light diversity: obscuring the observed species 2 | 3 | Yesterday during an [iDiv](https://www.idiv.de/) seminar we stumbled across **dark diversity**. For those unfamiliar with the concept, it has been defined by [ Pärtel *et al.* (2011)](http://www.sciencedirect.com/science/article/pii/S0169534710002922): 4 | 5 | > It is possible to specify species that belong to a particular species pool but that are not locally present. We call this unaccounted set of species ‘dark diversity’. 6 | 7 | So it is a set of species that we, for whatever reason, expect to be at a locality, but are not observed there. 8 | 9 | I raised a concern that, for the sake of completeness, we should also consider **light diversity**, which would be the *set of species that are not in the species pool, but we observe them anyway*. 10 | 11 | Examples: 12 | 13 | 1. Observations of yet undescribed species. 14 | 2. Observations of vagrant and migrant species (species that do not reproduce at the locality). 15 | 3. Invasive species (thanks Marten). 16 | 4. Species from taxonomic groups that are not part of our research, such as a cow in the middle of a botanical plot. 17 | 18 | Depending on the context, light diversity could be also called inconvenient diversity, nuisance diversity, unexpected diversity, surprising diversity, undesirable diversity, avoided diversity, or denial diversity. 19 | 20 | In order to make sense, dark diversity requires species pool to be philosophically [realistic](https://en.wikipedia.org/wiki/Philosophical_realism), i. e. we need to believe in the existence of a real species pool out there in the real world. In contrast, light diversity requires species pool to be [nominalistic](https://en.wikipedia.org/wiki/Nominalism) or [conceptual](https://en.wikipedia.org/wiki/Conceptualism), i.e. it is the species pool that we have in mind, or that we agree on in a broader community. 21 | 22 | The idea of light diversity was received with a lukewarm amusement, and I am not sure how useful it could be, but then I am also unsure why we should have dark diversity. Perhaps I should just get back to work, to the usual and dull grey diversity. 23 | 24 | **Reference** 25 | 26 | Pärtel M., Szava-Kovats R. & Zobel M. (2011) Dark diversity: shedding light on absent species. *Trends in Ecology and Evolution*, **26**: 124-128. 27 | 28 | -------------------------------------------------------------------------------- /2015_11_21_Academia_and_Media/academia_and_media.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Made with Remarkable! 7 | 8 | 9 | 12 | 13 | 14 |

15 | 16 | On soil larvae, Beverly Hills, passion, macroecology, and the problem of describing what I do 17 | 18 |

19 |

20 | I have always found it difficult to explain my profession to people. 21 |

22 |

23 | For my bachelor degree I studied larvae of family of inconspicuous flies (Insecta: Diptera: Therevidae) in soils of remote forest fragments in central Europe. There were about five people around the world that have ever heard of these flies, but I did not care as I was passionately in love with insects. Somehow I secured a small grant from Beverly Hills entrepreneur Rudolf H. Mattoni, and boy that was difficult to explain. A rich guy from Beverly Hills gave you money to do 24 | 25 | that 26 | 27 | ? Is it going to be your job? 28 |

29 |

30 | The money had run out fast. I had realized that if I followed my passion too far, I’d be living in a closet with a stereo microscope, observing ornamental bristle configurations on hind legs of obscure insects. I’d be really good at it, but unable to pay my rent, and with sex life limited to dissections of H-shaped 0.2 mm penises from the abdomens of Therevid males. Following the “follow your passion” cliché turned out to be impractical. 31 |

32 |

33 | So I decided that I need to push myself out of my comfort zone, to a scientific field that is far from insect penises as possible. 34 |

35 |

36 | I went to the library and scanned printed issues (it was 2003) of scientific journals, trying to figure out what is the most densely populated field of science that could accommodate a former entomologist. Macroecology seemed like the right choice (the 37 | 38 | macro 39 | 40 | prefix sounded like 41 | 42 | great 43 | 44 | ), Nature and Science seemed like the right venues to aim for, and Robert M. May, Edward O. Wilson and James H. Brown like the right role models. Off I went to pursue my master degree, and later on my PhD. 45 |

46 |

47 | First, I was surprised how fast a new kind of passion kicked in: Programming. To do macroecology one needs programming, and programming is similar to entomology. One spends most of the time in a contemplative search for bugs. Only programming is a bit less field-specific, and hence more social – all of a sudden I could discuss programming tricks with people outside of academia. 48 |

49 |

50 | But I also held a misconception that the more people there are working on a problem (and the higher the IF of the journals they publish in), the more important the problem is, and the more is its importance obvious to non-academics. Examples: What generates biodiversity patterns? What regulates populations? Are communities neutral? Massive ecological questions, but I have never managed to explain to my grandmother that answering them is my job. My wife got it after roughly a year, my parents are still working on it. My kids are too young still, but I can already see that my 2-year old son is more excited by seeing a fireman than by me reviewing a paper on biodiversity gradients. 51 |

52 |

53 | This is what I have been thinking: I could perhaps use my passion for chasing bugs to work on something that people get. And I noticed that there is one macroecological problem that everybody understands, although it is also the problem that is hardest to observe: Extinctions. The idea that something beautiful disappears because people take its space for living is crystal clear to my grandmother, to my wife, and I can imagine explaining it to my kids. 54 |

55 |

56 | It already seems to be the right way to go. Two weeks ago my first paper on extinctions came out (check it out 57 | 58 | here 59 | 60 | ) and I was asked to do a live BBC interview. I had no idea how to prepare for it, so I didn’t, and of course it was bad – I failed to explain why are our results important, I lacked confidence and all the usual crap (advice: never speak to media wearing pyjamas). But these are just technical problems that I can work on; the important and uplifting lesson is that I have maybe, finally, found an audience. 61 |

62 | 64 | 67 | 69 | 72 | 73 | -------------------------------------------------------------------------------- /2015_11_21_Academia_and_Media/academia_and_media.md: -------------------------------------------------------------------------------- 1 | **On soil larvae, Beverly Hills, passion, macroecology, and the problem of describing what I do** 2 | 3 | I have always found it difficult to explain my profession to people. 4 | 5 | For my bachelor degree I studied larvae of family of inconspicuous flies (Insecta: Diptera: Therevidae) in soils of remote forest fragments in central Europe. There were about five people around the world that have ever heard of these flies, but I did not care as I was passionately in love with insects. Somehow I secured a small grant from Beverly Hills entrepreneur Rudolf H. Mattoni, and boy that was difficult to explain. A rich guy from Beverly Hills gave you money to do *that*? Is it going to be your job? 6 | 7 | The money had run out fast. I had realized that if I followed my passion too far, I'd be living in a closet with a stereo microscope, observing ornamental bristle configurations on hind legs of obscure insects. I'd be really good at it, but unable to pay my rent, and with sex life limited to dissections of H-shaped 0.2 mm penises from the abdomens of Therevid males. Following the “follow your passion” cliché turned out to be impractical. 8 | 9 | So I decided that I need to push myself out of my comfort zone, to a scientific field that is far from insect penises as possible. 10 | 11 | I went to the library and scanned printed issues (it was 2003) of scientific journals, trying to figure out what is the most densely populated field of science that could accommodate a former entomologist. Macroecology seemed like the right choice (the *macro* prefix sounded like *great*), Nature and Science seemed like the right venues to aim for, and Robert M. May, Edward O. Wilson and James H. Brown like the right role models. Off I went to pursue my master degree, and later on my PhD. 12 | 13 | First, I was surprised how fast a new kind of passion kicked in: Programming. To do macroecology one needs programming, and programming is similar to entomology. One spends most of the time in a contemplative search for bugs. Only programming is a bit less field-specific, and hence more social – all of a sudden I could discuss programming tricks with people outside of academia. 14 | 15 | But I also held a misconception that the more people there are working on a problem (and the higher the IF of the journals they publish in), the more important the problem is, and the more is its importance obvious to non-academics. Examples: What generates biodiversity patterns? What regulates populations? Are communities neutral? Massive ecological questions, but I have never managed to explain to my grandmother that answering them is my job. My wife got it after roughly a year, my parents are still working on it. My kids are too young still, but I can already see that my 2-year old son is more excited by seeing a fireman than by me reviewing a paper on biodiversity gradients. 16 | 17 | This is what I have been thinking: I could perhaps use my passion for chasing bugs to work on something that people get. And I noticed that there is one macroecological problem that everybody understands, although it is also the problem that is hardest to observe: Extinctions. The idea that something beautiful disappears because people take its space for living is crystal clear to my grandmother, to my wife, and I can imagine explaining it to my kids. 18 | 19 | It already seems to be the right way to go. Two weeks ago my first paper on extinctions came out (check it out [here](http://www.nature.com/ncomms/2015/151117/ncomms9837/pdf/ncomms9837.pdf)) and I was asked to do a live BBC interview. I had no idea how to prepare for it, so I didn't, and of course it was bad – I failed to explain why are our results important, I lacked confidence and all the usual crap (advice: never speak to media wearing pyjamas). But these are just technical problems that I can work on; the important and uplifting lesson is that I have maybe, finally, found an audience. -------------------------------------------------------------------------------- /2015_11_21_Academia_and_Media/media.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2015_11_21_Academia_and_Media/media.jpg -------------------------------------------------------------------------------- /2016_01_07_Kery_book_review/.Rhistory: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_01_07_Kery_book_review/.Rhistory -------------------------------------------------------------------------------- /2016_01_07_Kery_book_review/kery_book_cover.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_01_07_Kery_book_review/kery_book_cover.jpeg -------------------------------------------------------------------------------- /2016_01_07_Kery_book_review/kery_modelling_review.odt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_01_07_Kery_book_review/kery_modelling_review.odt -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/.Rhistory: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_03_02_Natural_history_vs_statistics/.Rhistory -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/nat.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Is natural history more fundamental than statistics? 9 | 10 | 11 | 12 | 16 |

A couple of weeks ago at iDiv I had an exchange with Jonathan Chase about the importance of natural history, and whether it is more fundamental than statistics. Jon was arguing for fundamental importance of natural history, I disagreed. To quote Jon quoting Evelyn Hutchinson:

17 |
18 |

A quote Hutchinson wrote in 1975 about the importance of Natural history may shed some light on why I think that more fundamental than statistics, any good quantitative ecologist needs to be a good natural historian first…The quote is as relevant today (or more so) than when he wrote it 40 years ago:

19 |
20 |

Modern biological education, however, may let us down as ecologists if it does not insist…that a wide and quite deep understanding of organisms, past and present, is as basic a requirement as anything else in ecological education. It may be best self-taught, but how often is this difficult process made harder by a misplaced emphasis on a quite specious modernity. Robert MacArthur really knew his warblers.

21 |
22 |
23 |

I replied that dismissing statistics as specious modernity is unjust. Yes, there are bad statistical applications out there, and statistic can be used to conceal, divert attention, exaggerate, or to just make things too complex, but that is a problem of bad statisticians, not an inherent flaw of statistics.

24 |

However, neither of us (including Hutchinson) provided good arguments for why natural history or statistics should really be that fundamental. We haven’t continued the conversation, but the issue kept my brain busy nevertheless. Now I have this to say:

25 |

Any understanding of organisms begins with observation of nature or experimentation with it, or both. The observations and experiments then reveal rules (patterns, trends, laws) and exceptions to the rules, we are surprised or bored, we learn, we do more observations and experiments, we are surprised or bored, we learn, and so on. Children do it. People who do this professionally are scientists.

26 |

Now, in this process, where exactly is natural history and where is statistics? I’d say that Natural history, the deep understanding of nature, is the pile of rules and exceptions that one accumulates during the cycle of observation, experimentation and learning. I’d also say that Statistics is the way to identify the rules and exceptions among all the observations, and it does not really matter if this happens in children’s brain, or in R.

27 |

From this perspective, it seems that both me and Jon were kind of right, because both statistics and natural history are really important; however, their relative importance is perhaps a matter of taste. But are they also fundamental (= foundational)? Or is there something even more fundamental that keeps them both alive? Some infinite and renewable source of energy driving the cycle of observation, experimentation and learning?

28 |

I propose that such fundamental thing that keeps both natural history and statistics alive is curiosity. Simple curiosity. Maybe curiosity is the most fundamental basic skill that we, as scientists, should really profess and master (with a pinch of imagination and critical thinking). Deep knowledge and the ability to separate rules from exceptions will then come as a byproduct.

PS: In a recent post, Brian McGill argues that we are scientists because we count. I think that it mixes nicely with my proposition that we are scientists because we are curious.

PPS: Yesterday, Jon added that he did not want to question the relevance of statistics – rather, he has been worried about the lack of hypotheses and actual thinking in many of the modern statistical analyses. I partly second that worry. However, I’d also be cautious here: hypothesis-free exploratory analysis does have its merits, if done right.

29 | 30 | 31 | -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.aux: -------------------------------------------------------------------------------- 1 | \relax 2 | \providecommand\hyper@newdestlabel[2]{} 3 | \providecommand\HyperFirstAtBeginDocument{\AtBeginDocument} 4 | \HyperFirstAtBeginDocument{\ifx\hyper@anchor\@undefined 5 | \global\let\oldcontentsline\contentsline 6 | \gdef\contentsline#1#2#3#4{\oldcontentsline{#1}{#2}{#3}} 7 | \global\let\oldnewlabel\newlabel 8 | \gdef\newlabel#1#2{\newlabelxx{#1}#2} 9 | \gdef\newlabelxx#1#2#3#4#5#6{\oldnewlabel{#1}{{#2}{#3}}} 10 | \AtEndDocument{\ifx\hyper@anchor\@undefined 11 | \let\contentsline\oldcontentsline 12 | \let\newlabel\oldnewlabel 13 | \fi} 14 | \fi} 15 | \global\let\hyper@last\relax 16 | \gdef\HyperFirstAtBeginDocument#1{#1} 17 | \providecommand\HyField@AuxAddToFields[1]{} 18 | \providecommand\HyField@AuxAddToCoFields[2]{} 19 | -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.md~: -------------------------------------------------------------------------------- 1 | # Is natural history more fundamental than statistics? 2 | 3 | A couple of weeks ago at iDiv I had an exchange with Jonathan Chase about the importance of natural history, and whether it is more fundamental than statistics. I was highlighting the merits of statistics, Jon was arguing for fundamental importance of natural history. To quote Jon quoting Evelyn Hutchinson: 4 | 5 | > A quote Hutchinson wrote in 1975 about the importance of Natural history may shed some light on why I think that more fundamental than any statistical mumbo-jumbo, any good quantitative ecologist needs to be a good natural historian first…The quote is as relevant today (or more so) than when he wrote it 40 years ago: 6 | 7 | >> “*Modern biological education, however, may let us down as ecologists if it does not insist…that a wide and quite deep understanding of organisms, past and present, is as basic a requirement as anything else in ecological education. It may be best self-taught, but how often is this difficult process made harder by a misplaced emphasis on a quite specious modernity. Robert MacArthur really knew his warblers.*” 8 | 9 | I replied that dismissing statistics as mumbo-jumbo or specious modernity is unjust. Yes, there are bad statistical applications out there, and stats can be used to conceal, exaggerate, or to just make things too complex, but that is a problem of bad statisticians, it is not an inherent flaw of statistics. 10 | 11 | Today I see that neither of us (including Hutchinson) actually provided any substantial arguments for why natural history or statistics should really be that fundamental. We haven't continued the conversation, but the issue kept my brain busy nevertheless. Today I have this to say: 12 | 13 | Any understanding of organisms begins with observation of nature and/or experimentation with it. The observations and experiments then reveal rules (patterns, trends, laws) and exceptions to the rules, we are surprised or bored, we learn, we do more observations and experiments, we are surprised or bored, we learn, and so on. Children do it. People who do this professionally are scientists. 14 | 15 | Now, in this process, where exactly is natural history and where is statistics? I'd say that Natural history, the deep understanding of nature, is *the pile of rules and exceptions* that one accumulates during the cycle of observation, experimentation and learning. I'd also say that Statistics is *the way to identify the rules and exceptions* among all the observations, and it does not really matter if this happens in children's brain, or in R. 16 | 17 | From this perspective, it seems that both statistics and natural history are really important, and their relative importance is perhaps a matter of taste. But are they also fundamental (= foundational)? Or is there something even more fundamental that keeps them both alive? Some infinite and renewable source of energy driving the cycle of observation, experimentation and learning. 18 | 19 | I propose that such fundamental thing that keeps both natural history and statistics alive is curiosity. Simple curiosity. Maybe curiosity is the most fundamental basic skill that we, as scientists, should really profess and master. Deep knowledge and the ability to separate rules from exceptions will then come as a byproduct. 20 | 21 | PS: I have just . 22 | 23 | 24 | 25 | 26 | 27 | -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.out: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.out -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.pdf -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.synctex.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.synctex.gz -------------------------------------------------------------------------------- /2016_03_02_Natural_history_vs_statistics/natural_history_vs_statistics.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11pt,a4paper]{article} 2 | \usepackage[utf8]{inputenc} 3 | \usepackage{amsmath} 4 | \usepackage{amsfonts} 5 | \usepackage{amssymb} 6 | \usepackage{hyperref} 7 | 8 | \title{Is natural history more fundamental than statistics?} 9 | \author{Petr Keil} 10 | 11 | \begin{document} 12 | 13 | \maketitle 14 | 15 | A couple of weeks ago at iDiv I had an exchange with Jonathan Chase about the importance of natural history, and whether it is more fundamental than statistics. Jon was arguing for fundamental importance of natural history, I disagreed. To quote Jon quoting Evelyn Hutchinson: 16 | 17 | \begin{quote} 18 | \textit{A quote Hutchinson wrote in 1975 about the importance of Natural history may shed some light on why I think that more fundamental than statistics, any good quantitative ecologist needs to be a good natural historian first…The quote is as relevant today (or more so) than when he wrote it 40 years ago:} 19 | \begin{quote} 20 | \textit{Modern biological education, however, may let us down as ecologists if it does not insist…that a wide and quite deep understanding of organisms, past and present, is as basic a requirement as anything else in ecological education. It may be best self-taught, but how often is this difficult process made harder by a misplaced emphasis on a quite specious modernity. Robert MacArthur really knew his warblers.} 21 | \end{quote} 22 | 23 | \end{quote} 24 | 25 | I replied that dismissing statistics as specious modernity is unjust. Yes, there are bad statistical applications out there, and statistic can be used to conceal, divert attention, exaggerate, or to just make things too complex, but that is a problem of bad statisticians, not an inherent flaw of statistics. 26 | 27 | However, neither of us (including Hutchinson) provided good arguments for why natural history or statistics should really be that fundamental. We haven't continued the conversation, but the issue kept my brain busy nevertheless. Now I have this to say: 28 | 29 | Any understanding of organisms begins with observation of nature or experimentation with it, or both. The observations and experiments then reveal rules (patterns, trends, laws) and exceptions to the rules, we are surprised or bored, we learn, we do more observations and experiments, we are surprised or bored, we learn, and so on. Children do it. People who do this professionally are scientists. 30 | 31 | Now, in this process, where exactly is natural history and where is statistics? I'd say that \textbf{Natural history}, the deep understanding of nature, is the \textit{pile of rules and exceptions} that one accumulates during the cycle of observation, experimentation and learning. I'd also say that \textbf{Statistics} is \textit{the way to identify the rules and exceptions} among all the observations, and it does not really matter if this happens in children's brain, or in R. 32 | 33 | From this perspective, it seems that both me and Jon were kind of right, because both statistics and natural history are really important; however, their relative importance is perhaps a matter of taste. But are they also fundamental (= foundational)? Or is there something even more fundamental that keeps them both alive? Some infinite and renewable source of energy driving the cycle of observation, experimentation and learning? 34 | 35 | I propose that such fundamental thing that keeps both natural history and statistics alive is curiosity. Simple curiosity. Maybe curiosity is the most fundamental basic skill that we, as scientists, should really profess and master (with a pinch of imagination and critical thinking). Deep knowledge and the ability to separate rules from exceptions will then come as a byproduct. 36 | \\\\ 37 | PS: In a recent post, Brian McGill argues that \href{https://dynamicecology.wordpress.com/2016/02/29/we-arent-scientists-because-of-our-method-were-scientists-because-we-count/}{we are scientists because we count}. I think that it mixes nicely with my proposition that we are scientists because we are curious. 38 | \\\\ 39 | PPS: Yesterday, Jon added that he did not want to question the relevance of statistics -- rather, he has been worried about the lack of hypotheses and actual thinking in many of the modern statistical analyses. I partly second that worry. However, I'd also be cautious here: hypothesis-free exploratory analysis does have its merits, if done right. 40 | 41 | \end{document} -------------------------------------------------------------------------------- /2016_07_05_Log_scales/log_scale.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Logarithmic axes with linear gridlines in basic R plots" 3 | author: "Petr Keil" 4 | date: "July 6, 2016" 5 | output: html_document 6 | --- 7 | 8 | ```{r setup, include=FALSE} 9 | knitr::opts_chunk$set(echo = TRUE) 10 | ``` 11 | 12 | I like [Mathematica's](http://media.virtual-maxim.de/uploads/2010/12/ErrorBarLogPlots_Mathmatica.png) and [Matlab's log-log plots](http://www.mathworks.com/matlabcentral/mlc-downloads/downloads/submissions/35265/versions/2/previews/html/Loglog_Plot_01.png) with logarithmic axes and linear 13 | gridlines. In a way, they enable to imagine both multiplication and addition in a single figure. They also enable to more exactly visually connect data points with values. 14 | 15 | I haven't found a simple 'one-liner' that'd do such plots in R. In fact, I have always found R's treatment of logarithmic axes a bit dull - I want the fancy gridlines! 16 | 17 | ## Function `loglog.plot` 18 | 19 | To provide the log-linear gridlines, I have wrtitten function `loglog.plot`. 20 | 21 | **To load the function** from my GitHub repository: 22 | 23 | ```{r} 24 | source("https://raw.githubusercontent.com/petrkeil/Blog/master/2016_07_05_Log_scales/loglogplot.r") 25 | ``` 26 | 27 | **Arguments:** 28 | 29 | `xlim, ylim` - Numeric vectors of length 2, giving the x and y coordinates ranges on linear scale. 30 | 31 | `xlog, ylog` - Logical value indicating if x and y axes should be logarithmic (TRUE) or linear (FALSE). In case the linear scale is chosen, no gridlines are drawn. 32 | 33 | `xbase, ybase` - Base of the logarithm of the respective axes. Ignored if linear axis is specified. 34 | 35 | `...` - Further arguments to the generic R function `plot`. 36 | 37 | **Value:** 38 | 39 | Empty R base graphics plot, ready to be populated using `lines`, `points` and alike. 40 | 41 | ## Example 1 - both axes logarithmic 42 | 43 | Here I plot three power functions: one sub-linear (exponent = 0.8), one linear (exponent = 1) and one supra-linear (exponent = 1.2). 44 | 45 | 46 | ```{r, fig.width=10, fig.height=6} 47 | par(mfrow=c(1,2)) 48 | x <- seq(1, 1000, by=10) 49 | 50 | # left panel - both axes linear 51 | plot(x, x, ylim=c(0,4000)) 52 | points(x, x^0.8, col="blue") 53 | points(x, x^1.2, col="red") 54 | 55 | # right panel - loglog plot 56 | loglog.plot(xlab="x", ylab="y", ylim=c(1, 10000)) 57 | points(log10(x), log10(x)) 58 | points(log10(x), 0.8*log10(x), col="blue") 59 | points(log10(x), 1.2*log10(x), col="red") 60 | ``` 61 | 62 | ## Example 2 - x logarithmic, y linear 63 | 64 | In this example I plot a lognormal probability density function, and I only plot the gridlines along the x-axis. The y-axis is linear. 65 | 66 | ```{r, fig.width=10, fig.height=6} 67 | par(mfrow=c(1,2)) 68 | x <- 1:1000 69 | 70 | # left panel - linear 71 | plot(x, dlnorm(x, meanlog=4), ylim=c(0, 0.012), 72 | col="red", ylab="probability density") 73 | 74 | # right panel - loglog plot 75 | loglog.plot(ylog=FALSE, ylim=c(0,0.012), xlim=c(0.1, 1000), 76 | xlab="x", ylab="") 77 | points(log10(x), dlnorm(x, meanlog=4), col="red") 78 | ``` 79 | 80 | 81 | 82 | -------------------------------------------------------------------------------- /2016_07_05_Log_scales/log_scale.r: -------------------------------------------------------------------------------- 1 |  2 | 3 | x <- rlnorm(1000) 4 | y <- rlnorm(1000) 5 | xy <- data.frame(x, y) 6 | library(scales) 7 | 8 | 9 | loglog.plot <- function(xlim=c(0.01,1000), ylim=c(0.01,1000)) 10 | { 11 | xbreaks <- 10^(log10(xlim[1]):log10(xlim[2])) 12 | ybreaks <- 10^(log10(ylim[1]):log10(ylim[2])) 13 | 14 | plot(log10(xlim), log10(ylim), type="n", axes=FALSE, frame=TRUE, 15 | xlab="x", ylab="y") 16 | 17 | for(x in xbreaks) 18 | { 19 | subx <- log10(seq(from=x, to=x*10, length=10)) 20 | abline(v=subx, col="grey") 21 | } 22 | 23 | for(y in ybreaks) 24 | { 25 | suby <- log10(seq(from=y, to=y*10, length=10)) 26 | abline(h=suby, col="grey") 27 | } 28 | 29 | axis(side=1, at=log10(xbreaks), labels=xbreaks, tck=0.02) 30 | axis(side=2, at=log10(ybreaks), labels=ybreaks, las=2, tck=0.02) 31 | 32 | } 33 | loglog.plot() 34 | 35 | 36 | 37 | 38 | -------------------------------------------------------------------------------- /2016_07_05_Log_scales/loglogplot.r: -------------------------------------------------------------------------------- 1 | loglog.plot <- function(xlim=c(1,1000), xlog=TRUE, xbase=10, 2 | ylim=c(1,1000), ylog=TRUE, ybase=10, ...) 3 | { 4 | if(xlog) # rounding the X-axis limits on the log scale 5 | { 6 | xlim[1] <- floor(log(xlim[1], base=xbase)) 7 | xlim[2] <- round(log(xlim[2], base=xbase)) 8 | xbreaks <- xlim[1]:xlim[2] 9 | } 10 | if(ylog) # rounding the Y-axis limits on the log scale 11 | { 12 | ylim[1] <- floor(log(ylim[1], base=ybase)) 13 | ylim[2] <- round(log(ylim[2], base=ybase)) 14 | ybreaks <- ylim[1]:ylim[2] 15 | } 16 | 17 | ### the empty plot into which the axes will be drawn ### 18 | plot(xlim, ylim, type="n", axes=FALSE, frame=TRUE, ...) 19 | 20 | if(xlog) # plotting the X-axis tickmarks and grids 21 | { 22 | for(x in xbase^xbreaks) 23 | { 24 | subx <- log(seq(from=x, to=x*xbase, length=xbase) , base=xbase ) 25 | abline(v=subx, col="grey") 26 | } 27 | axis(side=1, at=xbreaks, labels=xbase^xbreaks, tck=0.02) 28 | } 29 | else axis(side=1, tck=0.02) 30 | 31 | if(ylog) # plotting the Y-axis tickmarks and grids 32 | { 33 | for(y in ybase^ybreaks) 34 | { 35 | suby <- log(seq(from=y, to=y*ybase, length=ybase), base=ybase) 36 | abline(h=suby, col="grey") 37 | } 38 | axis(side=2, at=ybreaks, labels=ybase^ybreaks, las=2, tck=0.02) 39 | } 40 | else axis(side=2, las=2, tck=0.02) 41 | } -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/.RData: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/.RData -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_1A.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_1A.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_1A_thumb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_1A_thumb.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_1B.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_1B.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_1B_thumb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_1B_thumb.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_2A.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_2A.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_2A_thumb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_2A_thumb.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_2B.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_2B.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/Figure_2B_thumb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_07_26_R_art_with_spatstat/Figure_2B_thumb.png -------------------------------------------------------------------------------- /2016_07_26_R_art_with_spatstat/R_art.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Reproducible psychedelic art with R: a tribute to spatstat" 3 | author: "Petr Keil" 4 | output: 5 | html_document: 6 | highlight: tango 7 | theme: cerulean 8 | --- 9 | 10 | LICENSE: This is a public domain work. Feel free to do absolutely whatever you want with the code or the images, there are no restrictions on the use. 11 | 12 | ---------------------------------------- 13 | 14 | Loading the `spatstat` library 15 | 16 | ```{r} 17 | library(spatstat) 18 | ``` 19 | 20 | Defining the 2:3 window 21 | 22 | ```{r} 23 | W23 <- as.owin(list(xrange=c(0,3), yrange=c(0,2))) 24 | ``` 25 | 26 | # Figure 1 27 | 28 | Generate the patterns: 29 | 30 | ```{r} 31 | set.seed(1612) 32 | 33 | # The exponent 34 | Lambda <- 5 35 | 36 | # Lay down the longest lines 37 | main.lines <- psp(runif(3),runif(3),runif(3),runif(3), window=W23) 38 | 39 | # Sample points proportionally to the distance to the lines 40 | dist.from.line <- distmap(main.lines) 41 | # exponential transformation of the distance 42 | dist.from.line.exp <- Lambda*exp(-Lambda*dist.from.line) 43 | 44 | samp.pois <- rpoispp(dist.from.line.exp*15) 45 | 46 | # Project to the lines using project2segment(b, a) 47 | Xproj <- project2segment(samp.pois, main.lines)$Xproj 48 | sub.lines <- psp(samp.pois$x, samp.pois$y, Xproj$x, Xproj$y, window=W23) 49 | 50 | # Connect with the lines 51 | all.lines <- append.psp(main.lines, sub.lines) 52 | dist.from.all.lines <- distmap(all.lines, dimyx=c(1200, 800)) 53 | ``` 54 | 55 | ## 1A 56 | 57 | ```{r} 58 | png(filename = "Figure_1A.png", width=6000, height=4000, res=400) 59 | par(mai=c(0,0,0,0)) 60 | dist.from.all.lines.exp <- exp(-Lambda*dist.from.all.lines) 61 | plot(dist.from.all.lines.exp, 62 | legend=FALSE, main="", box=FALSE, ribbon=FALSE) 63 | contour(dist.from.all.lines.exp, add=TRUE, col="white") 64 | contour(dist.from.all.lines, col="black", add=TRUE, lwd=4, levels=0.2) 65 | contour(dist.from.all.lines, col="black", add=TRUE, lwd=2, levels=0.3) 66 | contour(dist.from.all.lines, col="black", add=TRUE, lwd=2) 67 | plot(sub.lines, add=T, lty=2, col="black", lwd=1) 68 | plot(main.lines, add=T, col="black", lwd=4) 69 | dev.off() 70 | ``` 71 | 72 | ## 1B 73 | 74 | ```{r} 75 | png(filename = "Figure_1B.png", width=6000, height=4000, res=400) 76 | par(mai=c(0,0,0,0)) 77 | dist.from.all.lines.exp <- exp(-Lambda*dist.from.all.lines) 78 | plot(dist.from.all.lines.exp, col=terrain.colors(100), 79 | legend=FALSE, main="", box=FALSE, ribbon=FALSE) 80 | contour(dist.from.all.lines.exp, add=TRUE, lwd=2) 81 | plot(sub.lines, add=T, lty=2, col="black", lwd=1) 82 | plot(main.lines, add=T, col="black", lwd=4) 83 | dev.off() 84 | ``` 85 | 86 | # Figure 2 87 | 88 | Generate the pattern: 89 | 90 | ```{r} 91 | set.seed(16211) 92 | rc <- rpoispp(function(x,y){50 * exp(-3*(max(y)-y))}, 100, win=W23) 93 | rcdist <- distmap(rc, dimyx=c(1200, 800)) 94 | rc2 <- rpoispp(1/rcdist*50) 95 | rcd <- dirichlet(rc2) 96 | ``` 97 | 98 | ## 2A 99 | 100 | ```{r} 101 | png(filename = "Figure_2A.png", width=6000, height=4000, res=400) 102 | par(mai=c(0,0,0,0)) 103 | plot(rcdist, legend=FALSE, main="", frame=FALSE, box=FALSE, ribbon=FALSE) 104 | plot(rcd, add=T) 105 | plot(rc, add=T, col="black", pch=19, cex=2.5) 106 | plot(rjitter(rc, 0.01), add=T, col="white", pch=19, cex=0.4) 107 | contour(rcdist, add=T, col="white") 108 | dev.off() 109 | ``` 110 | 111 | ## 2B 112 | 113 | ```{r} 114 | png(filename = "Figure_2B.png", width=6000, height=4000, res=400) 115 | par(mai=c(0,0,0,0)) 116 | plot(rcdist, legend=FALSE, main="", frame=FALSE, box=FALSE, ribbon=FALSE, 117 | col=terrain.colors(100)) 118 | plot(rcd, add=T, col=terrain.colors(100)[30], lwd=2) 119 | plot(rc, add=T, col=terrain.colors(100)[30], pch=19, cex=3) 120 | plot(rc, add=T, col="white", pch=19, cex=2) 121 | plot(rjitter(rc, 0.01), add=T, col="black", pch=19, cex=0.4) 122 | contour(rcdist, add=T, nlevels=5, lwd=2) 123 | dev.off() 124 | # dd 125 | ``` 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | -------------------------------------------------------------------------------- /2016_08_23_Long-term_horizon_for_science_readability/ape_planet_PK.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_08_23_Long-term_horizon_for_science_readability/ape_planet_PK.png -------------------------------------------------------------------------------- /2016_08_23_Long-term_horizon_for_science_readability/long_term_horizon.odt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2016_08_23_Long-term_horizon_for_science_readability/long_term_horizon.odt -------------------------------------------------------------------------------- /2017_01_14_Post-IBS_impressions/post-ibs_impressions.odt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_01_14_Post-IBS_impressions/post-ibs_impressions.odt -------------------------------------------------------------------------------- /2017_01_30_facts/facts.odt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_01_30_facts/facts.odt -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/.RData: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/.RData -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/.Rhistory: -------------------------------------------------------------------------------- 1 | Bayesian resources 2 | ======================================================== 3 | author: Petr Keil 4 | date: February 2014 5 | Software 6 | ======================================================== 7 | **OpenBUGS** [www.openbugs.net](http://openbugs.net/w/FrontPage) 8 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2014/02/snakes.csv") 9 | # we will artificially delete 9 data points in the first population 10 | snakes <- snakes[-(1:9),] 11 | summary(snakes) 12 | par(mfrow=c(1,2)) 13 | plot(snout.vent ~ population, data=snakes, 14 | ylab="Snout-vent length [cm]") 15 | boxplot(snout.vent ~ population, data=snakes, 16 | ylab="Snout-vent length [cm]", 17 | xlab="population", 18 | col="grey") 19 | snake.data <- list(y=snakes$snout.vent, 20 | x=snakes$population, 21 | N=nrow(snakes), 22 | N.pop=5) 23 | library(R2jags) 24 | cat(" 25 | model 26 | { 27 | # priors 28 | sigma ~ dunif(0,100) 29 | tau <- 1/(sigma*sigma) 30 | for(j in 1:N.pop) 31 | { 32 | alpha[j] ~ dnorm(0, 0.001) 33 | } 34 | # likelihood 35 | for(i in 1:N) 36 | { 37 | y[i] ~ dnorm(alpha[x[i]], tau) 38 | } 39 | } 40 | ", file="fixed_anova.txt") 41 | model.fit.fix <- jags(data=snake.data, 42 | model.file="fixed_anova.txt", 43 | parameters.to.save=c("alpha"), 44 | n.chains=3, 45 | n.iter=2000, 46 | n.burnin=1000, 47 | DIC=FALSE) 48 | plot(as.mcmc(model.fit.fix)) 49 | model.fit.fix 50 | autocorr.plot(as.mcmc(model.fit.fix)) 51 | autocorr.plot(model.fit.fix) 52 | ?autocorr.plot 53 | autocorr.plot(as.mcmc(model.fit.fix)) 54 | autocorr.plot(as.mcmc(model.fit.fix)) 55 | gelman.plot(as.mcmc(model.fit.fix)) 56 | cat(" 57 | model 58 | { 59 | # priors 60 | grand.mean ~ dnorm(0, 0.001) 61 | grand.sigma ~ dunif(0,100) 62 | grand.tau <- 1/(grand.sigma*grand.sigma) 63 | group.sigma ~ dunif(0, 100) 64 | group.tau <- 1/(group.sigma*group.sigma) 65 | for(j in 1:N.pop) 66 | { 67 | alpha[j] ~ dnorm(grand.mean, grand.tau) 68 | } 69 | # likelihood 70 | for(i in 1:N) 71 | { 72 | y[i] ~ dnorm(alpha[x[i]], group.tau) 73 | } 74 | } 75 | ", file="random_anova.txt") 76 | model.fit.rnd <- jags(data=snake.data, 77 | model.file="random_anova.txt", 78 | parameters.to.save=c("alpha"), 79 | n.chains=3, 80 | n.iter=2000, 81 | n.burnin=1000, 82 | DIC=FALSE) 83 | plot(as.mcmc(model.fit.rnd)) 84 | model.fit.rnd 85 | cat(" 86 | model 87 | { 88 | # priors 89 | grand.mean ~ dnorm(0, 0.001) 90 | grand.sigma ~ dunif(0,100) 91 | grand.tau <- 1/(grand.sigma*grand.sigma) 92 | group.sigma ~ dunif(0, 100) 93 | group.tau <- 1/(group.sigma*group.sigma) 94 | for(j in 1:N.pop) 95 | { 96 | alpha[j] ~ dnorm(grand.mean, grand.tau) 97 | } 98 | # likelihood 99 | for(i in 1:N) 100 | { 101 | y[i] ~ dnorm(alpha[x[i]], group.tau) 102 | } 103 | between.vs.within <- grand.sigma - group.sigma 104 | } 105 | ", file="random_anova.txt") 106 | model.fit.rnd <- jags(data=snake.data, 107 | model.file="random_anova.txt", 108 | parameters.to.save=c("alpha", "between.vs.within"), 109 | n.chains=3, 110 | n.iter=2000, 111 | n.burnin=1000, 112 | DIC=FALSE) 113 | plot(as.mcmc(model.fit.rnd)) 114 | model.fit.rnd 115 | model.fit.rnd 116 | library(mcmcplots) 117 | caterplot(model.fit.fix) 118 | ?caterplot 119 | library(mcmcplots) 120 | caterplot(model.fit.fix, horizontal=FALSE) 121 | library(mcmcplots) 122 | caterplot(model.fit.fix, horizontal=FALSE, reorder=FALSE) 123 | ?caterplot 124 | caterplot(model.fit.rnd, params="alpha", horizontal=FALSE, reorder=FALSE) 125 | caterplot(model.fit.rnd, parms="alpha", horizontal=FALSE, reorder=FALSE) 126 | mcmcplot(model.fit.rnd, parms="between.minus.within") 127 | denplot(model.fit.rnd, parms="between.minus.within") 128 | model.fit.rnd <- jags(data=snake.data, 129 | model.file="random_anova.txt", 130 | parameters.to.save=c("alpha", "between.minus.within"), 131 | n.chains=3, 132 | n.iter=2000, 133 | n.burnin=1000, 134 | DIC=FALSE) 135 | cat(" 136 | model 137 | { 138 | # priors 139 | grand.mean ~ dnorm(0, 0.001) 140 | grand.sigma ~ dunif(0,100) 141 | grand.tau <- 1/(grand.sigma*grand.sigma) 142 | group.sigma ~ dunif(0, 100) 143 | group.tau <- 1/(group.sigma*group.sigma) 144 | for(j in 1:N.pop) 145 | { 146 | alpha[j] ~ dnorm(grand.mean, grand.tau) 147 | } 148 | # likelihood 149 | for(i in 1:N) 150 | { 151 | y[i] ~ dnorm(alpha[x[i]], group.tau) 152 | } 153 | between.minus.within <- grand.sigma - group.sigma 154 | } 155 | ", file="random_anova.txt") 156 | model.fit.rnd <- jags(data=snake.data, 157 | model.file="random_anova.txt", 158 | parameters.to.save=c("alpha", "between.minus.within"), 159 | n.chains=3, 160 | n.iter=2000, 161 | n.burnin=1000, 162 | DIC=FALSE) 163 | library(mcmcplots) 164 | caterplot(model.fit.rnd, parms="alpha", horizontal=FALSE, reorder=FALSE) 165 | denplot(model.fit.rnd, parms="between.vs.within") 166 | caterplot(model.fit.rnd, parms="between.vs.within") 167 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2017/02/snakes_lengths.csv) 168 | # we will artificially delete 9 data points in the first population 169 | snakes <- snakes[-(1:9),] 170 | summary(snakes) 171 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2017/02/snakes_lengths.csv") 172 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2017/02/snakes_lengths.csv") 173 | summary(snakes) 174 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2017/02/snakes_lengths.csv") 175 | summary(snakes) 176 | par(mfrow=c(1,2)) 177 | plot(snout.vent ~ population, data=snakes, 178 | ylab="Snout-vent length [cm]") 179 | boxplot(snout.vent ~ population, data=snakes, 180 | ylab="Snout-vent length [cm]", 181 | xlab="population", 182 | col="grey") 183 | snake.data <- list(y=snakes$snout.vent, 184 | x=snakes$population, 185 | N=nrow(snakes), 186 | N.pop=5) 187 | library(R2jags) 188 | cat(" 189 | model 190 | { 191 | # priors 192 | sigma ~ dunif(0,100) 193 | tau <- 1/(sigma*sigma) 194 | for(j in 1:N.pop) 195 | { 196 | alpha[j] ~ dnorm(0, 0.001) 197 | } 198 | # likelihood 199 | for(i in 1:N) 200 | { 201 | y[i] ~ dnorm(alpha[x[i]], tau) 202 | } 203 | # derived quantity 204 | delta12 <- alpha[1] - alpha[2] 205 | } 206 | ", file="fixed_anova.txt") 207 | model.fit.fix <- jags(data=snake.data, 208 | model.file="fixed_anova.txt", 209 | parameters.to.save=c("alpha", "delta12"), 210 | n.chains=3, 211 | n.iter=2000, 212 | n.burnin=1000, 213 | DIC=FALSE) 214 | model.fit.fix 215 | library(mcmcplots) 216 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 217 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, ylab="Posterior density") 218 | ?caterplot 219 | caterplot(model.fit.fix, parms="alpha", horizontal=TRUE, reorder=FALSE, ylab="Posterior density") 220 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, ylab="Posterior density") 221 | library(ggmcmc) 222 | install.packages("ggmcmc") 223 | library(ggmcmc) 224 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, ylab="Posterior density") 225 | axis() 226 | axis(side=2) 227 | axis(side=2, labels = "ahoj") 228 | ?axis 229 | mtext("") 230 | mtext("ajhpj") 231 | mtext("ajhpj", side=2) 232 | mtext("ajhpj", side=2, hadj=-0.5) 233 | mtext("ajhpj", side=2, adj=-0.5) 234 | ?mtext 235 | mtext("ajhpj", side=2, padj=-0.5) 236 | mtext("ajhpj", side=2, padj=-1) 237 | mtext("ajhpj", side=2, padj=-2) 238 | mtext("ajhpj", side=2, padj=-4) 239 | library(mcmcplots) 240 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 241 | mtext("Posterior density", side=2, padj=-4) 242 | library(mcmcplots) 243 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 244 | mtext("Posterior density", side=2, padj=-3) 245 | library(mcmcplots) 246 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 247 | mtext("Posterior density", side=2, padj=-3.5) 248 | library(mcmcplots) 249 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 250 | mtext("Body length", side=2, padj=-3.5) 251 | caterplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE) 252 | mtext("Body length", side=2, padj=-3.5) 253 | caterplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE) 254 | mtext("Difference between pop. 1 and 2", side=2, padj=-3.5) 255 | caterplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE) 256 | mtext("Mean difference", side=2, padj=-3.5) 257 | caterplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE) 258 | mtext("Difference of mean lengths", side=2, padj=-3.5) 259 | library(mcmcplots) 260 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 261 | mtext("Body length", side=2, padj=-3.5) 262 | points(snakes$snout.vent ~ snakes$population) 263 | library(mcmcplots) 264 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE) 265 | mtext("Body length", side=2, padj=-3.5) 266 | points(snakes$snout.vent ~ snakes$population) 267 | ?caterplot 268 | plot(snakes$snout.vent ~ snakes$population) 269 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, add=TRUE) 270 | plot(snakes$snout.vent ~ snakes$population) 271 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, add=TRUE) 272 | library(mcmcplots) 273 | plot(snakes$snout.vent ~ snakes$population, xlab="Population", ylab="Body length", 274 | ylim=c(35, 65)) 275 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, add=TRUE) 276 | library(mcmcplots) 277 | plot(snakes$snout.vent ~ snakes$population, xlab="Population", ylab="Body length", 278 | ylim=c(35, 65)) 279 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, reorder=FALSE, add=TRUE, 280 | labels=FALSE) 281 | library(mcmcplots) 282 | plot(snakes$snout.vent ~ snakes$population, xlab="Population", ylab="Body length", 283 | ylim=c(35, 65)) 284 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, 285 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2) 286 | library(mcmcplots) 287 | plot(snakes$snout.vent ~ snakes$population, xlab="Population", ylab="Body length", 288 | ylim=c(35, 65), col="grey") 289 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, 290 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 291 | model.fit.fix <- jags(data = snake.data, 292 | model.file = "fixed_anova.txt", 293 | parameters.to.save = c("alpha", "delta12"), 294 | n.chains = 3, 295 | n.iter = 10000, 296 | n.burnin = 5000, 297 | DIC = FALSE) 298 | library(mcmcplots) 299 | plot(snakes$snout.vent ~ snakes$population, xlab="Population", ylab="Body length", 300 | ylim=c(35, 65), col="grey") 301 | caterplot(model.fit.fix, parms="alpha", horizontal=FALSE, 302 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 303 | denplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE) 304 | denplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE, style="") 305 | denplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE, style="plain") 306 | denplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE, style="plain") 307 | mtext("Difference of mean lengths", side=2, padj=-3.5) 308 | denplot(model.fit.fix, parms="delta12", horizontal=FALSE, reorder=FALSE, style="plain") 309 | denplot(model.fit.fix, parms="delta12", style="plain") 310 | par(mai=c(0.5, 0.5, 0.1, 0.1)) 311 | denplot(model.fit.fix, parms="delta12", style="plain") 312 | par(mai=c(0.5, 0.5, 0.1, 0.1)) 313 | denplot(model.fit.fix, parms="delta12", style="plain") 314 | par(mai=c(0.5, 0.5, 0.1, 0.1)) 315 | denplot(model.fit.fix, parms="delta12", style="plain") 316 | ?denplot 317 | par(mai=c(0.5, 0.5, 0.1, 0.1)) 318 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means") 319 | par(mai=c(0.5, 0.5, 0.1, 0.1)) 320 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means") 321 | par(mai=c(1, 1, 0.1, 0.1)) 322 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means") 323 | par(mai=c(1, 1, 0.1, 0.1)) 324 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means", mar=c(1,1,0.1,0.1)) 325 | par(mai=c(1, 1, 0.1, 0.1)) 326 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means", mar=c(5,1,0.1,0.1)) 327 | par(mai=c(1, 1, 0.1, 0.1)) 328 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means", mar=c(5,5,1,1)) 329 | par(mai=c(1, 1, 0.1, 0.1)) 330 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means", ylab="Posterior density", mar=c(5,5,2,1)) 331 | par(mai=c(1, 1, 0.1, 0.1)) 332 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means of populations 1 and 2", ylab="Posterior density", mar=c(5,5,2,1), main="") 333 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means of populations 1 and 2", ylab="Posterior density", mar=c(8,8,2,1), main="") 334 | library(mcmcplots) 335 | plot(snakes$snout.vent ~ snakes$population, ylim=c(0, 100), col="grey", 336 | xlab="Population", ylab="Body length") 337 | caterplot(model.fit.1, parms="mu", horizontal=FALSE, 338 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 339 | cat(" 340 | model 341 | { 342 | # priors 343 | for(j in 1:N.pop) 344 | { 345 | mu[j] ~ dnorm(0, 0.001)T(0,) 346 | sigma[j] ~ dunif(0,100) 347 | tau[j] <- 1/(sigma[j]*sigma[j]) 348 | } 349 | # likelihood 350 | for(i in 1:N) 351 | { 352 | y[i] ~ dnorm(mu[x[i]], tau[x[i]]) 353 | } 354 | # the difference between populations 1 and 2: 355 | delta12 <- mu[1] - mu[2] 356 | } 357 | ", file="fixed_anova_relaxed.txt") 358 | model.fit.1 <- jags(data = snake.data, 359 | model.file = "fixed_anova_relaxed.txt", 360 | parameters.to.save = c("mu", "delta12"), 361 | n.chains = 3, 362 | n.iter = 20000, 363 | n.burnin = 10000, 364 | DIC = FALSE) 365 | library(mcmcplots) 366 | plot(snakes$snout.vent ~ snakes$population, ylim=c(0, 100), col="grey", 367 | xlab="Population", ylab="Body length") 368 | caterplot(model.fit.1, parms="mu", horizontal=FALSE, 369 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 370 | cat(" 371 | model 372 | { 373 | # priors 374 | for(j in 1:N.pop) 375 | { 376 | mu[j] ~ dnorm(0, 0.0001)T(0,) 377 | sigma[j] ~ dunif(0,100) 378 | tau[j] <- 1/(sigma[j]*sigma[j]) 379 | } 380 | # likelihood 381 | for(i in 1:N) 382 | { 383 | y[i] ~ dnorm(mu[x[i]], tau[x[i]]) 384 | } 385 | # the difference between populations 1 and 2: 386 | delta12 <- mu[1] - mu[2] 387 | } 388 | ", file="fixed_anova_relaxed.txt") 389 | model.fit.1 <- jags(data = snake.data, 390 | model.file = "fixed_anova_relaxed.txt", 391 | parameters.to.save = c("mu", "delta12"), 392 | n.chains = 3, 393 | n.iter = 20000, 394 | n.burnin = 10000, 395 | DIC = FALSE) 396 | denplot(model.fit.fix, parms="delta12", style="plain", xlab="Difference between means") 397 | model.fit.2 <- jags(data = snake.data, 398 | model.file = "fixed_anova_relaxed.txt", 399 | parameters.to.save = c("mu", "delta12"), 400 | n.chains = 3, 401 | n.iter = 200000, 402 | n.burnin = 100000, 403 | DIC = FALSE) 404 | library(mcmcplots) 405 | plot(snakes$snout.vent ~ snakes$population, ylim=c(0, 100), col="grey", 406 | xlab="Population", ylab="Body length") 407 | caterplot(model.fit.2, parms="mu", horizontal=FALSE, 408 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 409 | denplot(model.fit.2, parms="delta12", style="plain", mar=c(5,5,2,1), main="", 410 | xlab="Difference between means of populations 1 and 2", 411 | ylab="Posterior density") 412 | -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Bayesian ANOVA: Powerful inference with within-group sample size of 1" 3 | author: "Petr Keil" 4 | date: "March 2017" 5 | output: 6 | html_document: 7 | highlight: pygments 8 | keep_md: yes 9 | number_sections: yes 10 | theme: cerulean 11 | toc: yes 12 | pdf_document: default 13 | --- 14 | 15 | *** 16 | 17 | This post is inspired by a question by Dylan Craven that he raised during my Bayesian stats course. 18 | 19 | # Objective 20 | 21 | My aim here is to demonstrate that, in Bayesian setting, one can make powerful inference about within-group ANOVA means $\mu_i$ even for groups with sample size $N=1$, i.e. groups with only a single measurement. The main reason this is possible is the assumption of constant variance ($\sigma^2$) among groups: groups with sufficiently large $N$ increase credibility of the $\sigma^2$ estimate, which limits the credible interval of $\mu$ in the group with $N=1$. Conversely, when the assumption is relaxed and $\sigma^2$ is allowed to vary, groups with sample size of 1 are no longer that useful. 22 | 23 | # The data 24 | 25 | I will use modified artificially-generated data from the example from [Marc Kery's Introduction to WinBUGS for Ecologists](https://www.mbr-pwrc.usgs.gov/software/kerybook/), page 119 (Chapter 9 - ANOVA). The (artificial) data are supposed to describe snout-vent lengths in five populations of Smooth snake (*Coronella austriaca*); I modified the data so that **the first population is only represented by a single observation**, as indicated by the arrow in the figure below. 26 | 27 | 28 | *** 29 | 30 | Loading the data from the web: 31 | 32 | ```{r, tidy=FALSE} 33 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2017/02/snakes_lengths.csv") 34 | 35 | ``` 36 | 37 | Plotting the data: 38 | 39 | ```{r, fig.width=7, fig.height=3.6} 40 | par(mfrow=c(1,2), mai=c(0.8,0.8, 0.1, 0.1)) 41 | plot(snout.vent ~ population, data=snakes, ylab="Length [cm]") 42 | arrows(x1=1.2, y1=58, x0=2, y0=61, col="red", lwd=2, angle=25, length=0.2) 43 | boxplot(snout.vent ~ population, data=snakes, ylab="Length [cm]", xlab="population", col="grey") 44 | ``` 45 | 46 | *** 47 | 48 | # Fixed-effects ANOVA in JAGS 49 | 50 | First, I will model the data using a traditional fixed-effect ANOVA model. For a given snake $i$ in population $j$ **the model** can be written as: 51 | 52 | $$y_{ij} \sim Normal(\mu_j, \sigma)$$ 53 | 54 | Where $\mu_j$ is a mean of $j$-th population ($j \in \{1,2,3,4,5\}$), and $i$ identifies individual measurements. 55 | 56 | I am interested in a simple question: **Is there a statistically significant difference between population 1 and 2?** Will I be able to infere the difference even when population 1 has a sample size of 1? 57 | 58 | *** 59 | 60 | I will fit the model using MCMC in [JAGS](http://mcmc-jags.sourceforge.net/). Hence, I will prepare the data in a list format: 61 | 62 | ```{r, tidy=FALSE} 63 | snake.data <- list(y=snakes$snout.vent, 64 | x=snakes$population, 65 | N=nrow(snakes), 66 | N.pop=5) 67 | ``` 68 | 69 | Loading the library that enables R to communicate with JAGS: 70 | 71 | ```{r, message=FALSE, warning=FALSE} 72 | library(R2jags) 73 | ``` 74 | 75 | JAGS Model definition: 76 | 77 | ```{r, tidy=FALSE} 78 | cat(" 79 | model 80 | { 81 | # priors 82 | sigma ~ dunif(0,100) # (you may want to use a more proper gamma prior) 83 | tau <- 1/(sigma*sigma) 84 | for(j in 1:N.pop) 85 | { 86 | mu[j] ~ dnorm(0, 0.0001) 87 | } 88 | 89 | # likelihood 90 | for(i in 1:N) 91 | { 92 | y[i] ~ dnorm(mu[x[i]], tau) 93 | } 94 | 95 | # the difference between populations 1 and 2: 96 | delta12 <- mu[1] - mu[2] 97 | } 98 | ", file="fixed_anova.txt") 99 | 100 | ``` 101 | 102 | And here I fit the model: 103 | 104 | ```{r, message=FALSE, warnings=FALSE} 105 | model.fit.1 <- jags(data = snake.data, 106 | model.file = "fixed_anova.txt", 107 | parameters.to.save = c("mu", "delta12"), 108 | n.chains = 3, 109 | n.iter = 20000, 110 | n.burnin = 10000, 111 | DIC = FALSE) 112 | ``` 113 | 114 | Plotting parameter estimates with `mcmcplots`: 115 | 116 | ```{r} 117 | library(mcmcplots) 118 | 119 | plot(snakes$snout.vent ~ snakes$population, ylim=c(35, 65), col="grey", 120 | xlab="Population", ylab="Body length") 121 | 122 | caterplot(model.fit.1, parms="mu", horizontal=FALSE, 123 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 124 | ``` 125 | 126 | The red dots and bars show posterior densities of $\mu$. Grey dots are the data. 127 | 128 | *** 129 | 130 | **So what is the difference between population 1 and 2?** 131 | 132 | ```{r} 133 | denplot(model.fit.1, parms="delta12", style="plain", mar=c(5,5,2,1), main="", 134 | xlab="Difference between means of populations 1 and 2", 135 | ylab="Posterior density") 136 | ``` 137 | 138 | There is a clear non-zero difference $\mu_1 - \mu_2$ (the posterior density does not overlap zero). So we can conclude that there is a statistically significant difference between populations 1 and 2, and we can conclude this despite having only $N=1$ for population 1. 139 | 140 | *** 141 | 142 | # Relaxing the assumption of constant variance 143 | 144 | We can have a look what happens when the assumption of constant $\sigma$ is relaxed. 145 | In other words, now every population $j$ has its own $\sigma$, and the model is: 146 | 147 | $$y_{ij} \sim Normal(\mu_j, \sigma_j)$$ 148 | 149 | *** 150 | 151 | Here is JAGS definition of such model: 152 | 153 | ```{r, tidy=FALSE} 154 | cat(" 155 | model 156 | { 157 | # priors 158 | for(j in 1:N.pop) 159 | { 160 | mu[j] ~ dnorm(0, 0.0001)T(0,) # Note that I truncate the priors here 161 | sigma[j] ~ dunif(0,100) # again, you may want to use proper gamma prior here 162 | tau[j] <- 1/(sigma[j]*sigma[j]) 163 | } 164 | 165 | # likelihood 166 | for(i in 1:N) 167 | { 168 | y[i] ~ dnorm(mu[x[i]], tau[x[i]]) 169 | } 170 | 171 | # the difference between populations 1 and 2: 172 | delta12 <- mu[1] - mu[2] 173 | } 174 | ", file="fixed_anova_relaxed.txt") 175 | 176 | ``` 177 | 178 | Let's fit the model: 179 | 180 | ```{r, message=FALSE, warnings=FALSE} 181 | model.fit.2 <- jags(data = snake.data, 182 | model.file = "fixed_anova_relaxed.txt", 183 | parameters.to.save = c("mu", "delta12"), 184 | n.chains = 3, 185 | n.iter = 20000, 186 | n.burnin = 10000, 187 | DIC = FALSE) 188 | ``` 189 | 190 | Here are parameter estimates plotted with `mcmcplots` 191 | 192 | ```{r} 193 | library(mcmcplots) 194 | 195 | plot(snakes$snout.vent ~ snakes$population, ylim=c(0, 100), col="grey", 196 | xlab="Population", ylab="Body length") 197 | 198 | caterplot(model.fit.2, parms="mu", horizontal=FALSE, reorder=FALSE, add=TRUE, 199 | labels=FALSE, cex=2, col="red") 200 | ``` 201 | 202 | Clearly, the posterior density of $\mu_1$ is essentially the priror. In other words, 203 | the single observation has little power on its own to overdrive the prior. 204 | 205 | We can plot the posterior density of the difference $\mu_1 - \mu_2$ 206 | 207 | ```{r} 208 | denplot(model.fit.2, parms="delta12", style="plain", mar=c(5,5,2,1), main="", 209 | xlab="Difference between means of populations 1 and 2", 210 | ylab="Posterior density") 211 | ``` 212 | 213 | And we see that it is all over the place. Huge uncertainty, no clear inference is possible. 214 | 215 | *** 216 | 217 | # Conclusion 218 | 219 | I have shown that one can make powerful inference about population means even if the sample size for the population is super small (e.g. 1). This is possible when all of the following conditions are met: 220 | 221 | (1) Bayesian approach to ANOVA must be used. 222 | 223 | (2) We must have sufficient data from other populations. 224 | 225 | (3) We must be confident that the assumption of constant variance among the groups is justified. 226 | 227 | 228 | 229 | 230 | 231 | 232 | 233 | 234 | -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N.md: -------------------------------------------------------------------------------- 1 | # Bayesian ANOVA: Powerful inference with within-group sample size of 1 2 | Petr Keil 3 | March 2017 4 | 5 | *** 6 | 7 | This post is inspired by a question by Dylan Craven that he raised during my Bayesian stats course. 8 | 9 | # Objective 10 | 11 | My aim here is to demonstrate that, in Bayesian setting, one can make powerful inference about within-group ANOVA means $\mu_i$ even for groups with sample size $N=1$, i.e. groups with only a single measurement. The main reason this is possible is the assumption of constant variance ($\sigma^2$) among groups: groups with sufficiently large $N$ increase credibility of the $\sigma^2$ estimate, which limits the credible interval of $\mu$ in the group with $N=1$. Conversely, when the assumption is relaxed and $\sigma^2$ is allowed to vary, groups with sample size of 1 are no longer that useful. 12 | 13 | # The data 14 | 15 | I will use modified artificially-generated data from the example from [Marc Kery's Introduction to WinBUGS for Ecologists](https://www.mbr-pwrc.usgs.gov/software/kerybook/), page 119 (Chapter 9 - ANOVA). The (artificial) data are supposed to describe snout-vent lengths in five populations of Smooth snake (*Coronella austriaca*); I modified the data so that **the first population is only represented by a single observation**, as indicated by the arrow in the figure below. 16 | 17 | 18 | *** 19 | 20 | Loading the data from the web: 21 | 22 | 23 | ```r 24 | snakes <- read.csv("http://www.petrkeil.com/wp-content/uploads/2017/02/snakes_lengths.csv") 25 | ``` 26 | 27 | Plotting the data: 28 | 29 | 30 | ```r 31 | par(mfrow=c(1,2), mai=c(0.8,0.8, 0.1, 0.1)) 32 | plot(snout.vent ~ population, data=snakes, ylab="Length [cm]") 33 | arrows(x1=1.2, y1=58, x0=2, y0=61, col="red", lwd=2, angle=25, length=0.2) 34 | boxplot(snout.vent ~ population, data=snakes, ylab="Length [cm]", xlab="population", col="grey") 35 | ``` 36 | 37 | ![](anova_and_a_single_point_N_files/figure-html/unnamed-chunk-2-1.png) 38 | 39 | *** 40 | 41 | # Fixed-effects ANOVA in JAGS 42 | 43 | First, I will model the data using a traditional fixed-effect ANOVA model. For a given snake $i$ in population $j$ **the model** can be written as: 44 | 45 | $$y_{ij} \sim Normal(\mu_j, \sigma)$$ 46 | 47 | Where $\mu_j$ is a mean of $j$-th population ($j \in \{1,2,3,4,5\}$), and $i$ identifies individual measurements. 48 | 49 | I am interested in a simple question: **Is there a statistically significant difference between population 1 and 2?** Will I be able to infere the difference even when population 1 has a sample size of 1? 50 | 51 | *** 52 | 53 | I will fit the model using MCMC in [JAGS](http://mcmc-jags.sourceforge.net/). Hence, I will prepare the data in a list format: 54 | 55 | 56 | ```r 57 | snake.data <- list(y=snakes$snout.vent, 58 | x=snakes$population, 59 | N=nrow(snakes), 60 | N.pop=5) 61 | ``` 62 | 63 | Loading the library that enables R to communicate with JAGS: 64 | 65 | 66 | ```r 67 | library(R2jags) 68 | ``` 69 | 70 | JAGS Model definition: 71 | 72 | 73 | ```r 74 | cat(" 75 | model 76 | { 77 | # priors 78 | sigma ~ dunif(0,100) # (you may want to use a more proper gamma prior) 79 | tau <- 1/(sigma*sigma) 80 | for(j in 1:N.pop) 81 | { 82 | mu[j] ~ dnorm(0, 0.0001) 83 | } 84 | 85 | # likelihood 86 | for(i in 1:N) 87 | { 88 | y[i] ~ dnorm(mu[x[i]], tau) 89 | } 90 | 91 | # the difference between populations 1 and 2: 92 | delta12 <- mu[1] - mu[2] 93 | } 94 | ", file="fixed_anova.txt") 95 | ``` 96 | 97 | And here I fit the model: 98 | 99 | 100 | ```r 101 | model.fit.1 <- jags(data = snake.data, 102 | model.file = "fixed_anova.txt", 103 | parameters.to.save = c("mu", "delta12"), 104 | n.chains = 3, 105 | n.iter = 20000, 106 | n.burnin = 10000, 107 | DIC = FALSE) 108 | ``` 109 | 110 | ``` 111 | ## Compiling model graph 112 | ## Resolving undeclared variables 113 | ## Allocating nodes 114 | ## Graph information: 115 | ## Observed stochastic nodes: 41 116 | ## Unobserved stochastic nodes: 6 117 | ## Total graph size: 107 118 | ## 119 | ## Initializing model 120 | ``` 121 | 122 | Plotting parameter estimates with `mcmcplots`: 123 | 124 | 125 | ```r 126 | library(mcmcplots) 127 | 128 | plot(snakes$snout.vent ~ snakes$population, ylim=c(35, 65), col="grey", 129 | xlab="Population", ylab="Body length") 130 | 131 | caterplot(model.fit.1, parms="mu", horizontal=FALSE, 132 | reorder=FALSE, add=TRUE, labels=FALSE, cex=2, col="red") 133 | ``` 134 | 135 | ![](anova_and_a_single_point_N_files/figure-html/unnamed-chunk-7-1.png) 136 | 137 | The red dots and bars show posterior densities of $\mu$. Grey dots are the data. 138 | 139 | *** 140 | 141 | **So what is the difference between population 1 and 2?** 142 | 143 | 144 | ```r 145 | denplot(model.fit.1, parms="delta12", style="plain", mar=c(5,5,2,1), main="", 146 | xlab="Difference between means of populations 1 and 2", 147 | ylab="Posterior density") 148 | ``` 149 | 150 | ![](anova_and_a_single_point_N_files/figure-html/unnamed-chunk-8-1.png) 151 | 152 | There is a clear non-zero difference $\mu_1 - \mu_2$ (the posterior density does not overlap zero). So we can conclude that there is a statistically significant difference between populations 1 and 2, and we can conclude this despite having only $N=1$ for population 1. 153 | 154 | *** 155 | 156 | # Relaxing the assumption of constant variance 157 | 158 | We can have a look what happens when the assumption of constant $\sigma$ is relaxed. 159 | In other words, now every population $j$ has its own $\sigma$, and the model is: 160 | 161 | $$y_{ij} \sim Normal(\mu_j, \sigma_j)$$ 162 | 163 | *** 164 | 165 | Here is JAGS definition of such model: 166 | 167 | 168 | ```r 169 | cat(" 170 | model 171 | { 172 | # priors 173 | for(j in 1:N.pop) 174 | { 175 | mu[j] ~ dnorm(0, 0.0001)T(0,) # Note that I truncate the priors here 176 | sigma[j] ~ dunif(0,100) # again, you may want to use proper gamma prior here 177 | tau[j] <- 1/(sigma[j]*sigma[j]) 178 | } 179 | 180 | # likelihood 181 | for(i in 1:N) 182 | { 183 | y[i] ~ dnorm(mu[x[i]], tau[x[i]]) 184 | } 185 | 186 | # the difference between populations 1 and 2: 187 | delta12 <- mu[1] - mu[2] 188 | } 189 | ", file="fixed_anova_relaxed.txt") 190 | ``` 191 | 192 | Let's fit the model: 193 | 194 | 195 | ```r 196 | model.fit.2 <- jags(data = snake.data, 197 | model.file = "fixed_anova_relaxed.txt", 198 | parameters.to.save = c("mu", "delta12"), 199 | n.chains = 3, 200 | n.iter = 20000, 201 | n.burnin = 10000, 202 | DIC = FALSE) 203 | ``` 204 | 205 | ``` 206 | ## Compiling model graph 207 | ## Resolving undeclared variables 208 | ## Allocating nodes 209 | ## Graph information: 210 | ## Observed stochastic nodes: 41 211 | ## Unobserved stochastic nodes: 10 212 | ## Total graph size: 140 213 | ## 214 | ## Initializing model 215 | ``` 216 | 217 | Here are parameter estimates plotted with `mcmcplots` 218 | 219 | 220 | ```r 221 | library(mcmcplots) 222 | 223 | plot(snakes$snout.vent ~ snakes$population, ylim=c(0, 100), col="grey", 224 | xlab="Population", ylab="Body length") 225 | 226 | caterplot(model.fit.2, parms="mu", horizontal=FALSE, reorder=FALSE, add=TRUE, 227 | labels=FALSE, cex=2, col="red") 228 | ``` 229 | 230 | ![](anova_and_a_single_point_N_files/figure-html/unnamed-chunk-11-1.png) 231 | 232 | Clearly, the posterior density of $\mu_1$ is essentially the priror. In other words, 233 | the single observation has little power on its own to overdrive the prior. 234 | 235 | We can plot the posterior density of the difference $\mu_1 - \mu_2$ 236 | 237 | 238 | ```r 239 | denplot(model.fit.2, parms="delta12", style="plain", mar=c(5,5,2,1), main="", 240 | xlab="Difference between means of populations 1 and 2", 241 | ylab="Posterior density") 242 | ``` 243 | 244 | ![](anova_and_a_single_point_N_files/figure-html/unnamed-chunk-12-1.png) 245 | 246 | And we see that it is all over the place. Huge uncertainty, no clear inference is possible. 247 | 248 | *** 249 | 250 | # Conclusion 251 | 252 | I have shown that one can make powerful inference about population means even if the sample size for the population is super small (e.g. 1). This is possible when all of the following conditions are met: 253 | 254 | (1) Bayesian approach to ANOVA must be used. 255 | 256 | (2) We must have sufficient data from other populations. 257 | 258 | (3) We must be confident that the assumption of constant variance among the groups is justified. 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-11-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-11-1.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-12-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-12-1.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-2-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-2-1.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-7-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-7-1.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-8-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/anova_and_a_single_point_N_files/figure-html/unnamed-chunk-8-1.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/snake.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/snake.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-1.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-10.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-2.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-61.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-61.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-62.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-62.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-91.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-91.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-92.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_03_03_Single_observation_and_significance/figure/unnamed-chunk-92.png -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/fixed_anova.txt: -------------------------------------------------------------------------------- 1 | 2 | model 3 | { 4 | # priors 5 | sigma ~ dunif(0,100) # (you may want to use a more proper gamma prior) 6 | tau <- 1/(sigma*sigma) 7 | for(j in 1:N.pop) 8 | { 9 | mu[j] ~ dnorm(0, 0.0001) 10 | } 11 | 12 | # likelihood 13 | for(i in 1:N) 14 | { 15 | y[i] ~ dnorm(mu[x[i]], tau) 16 | } 17 | 18 | # the difference between populations 1 and 2: 19 | delta12 <- mu[1] - mu[2] 20 | } 21 | -------------------------------------------------------------------------------- /2017_03_03_Single_observation_and_significance/fixed_anova_relaxed.txt: -------------------------------------------------------------------------------- 1 | 2 | model 3 | { 4 | # priors 5 | for(j in 1:N.pop) 6 | { 7 | mu[j] ~ dnorm(0, 0.0001)T(0,) # Note that I truncate the priors here 8 | sigma[j] ~ dunif(0,100) # again, you may want to use proper gamma prior here 9 | tau[j] <- 1/(sigma[j]*sigma[j]) 10 | } 11 | 12 | # likelihood 13 | for(i in 1:N) 14 | { 15 | y[i] ~ dnorm(mu[x[i]], tau[x[i]]) 16 | } 17 | 18 | # the difference between populations 1 and 2: 19 | delta12 <- mu[1] - mu[2] 20 | } 21 | -------------------------------------------------------------------------------- /2017_05_11_Rhino_wars/2017_03_27_Rhinos.odt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_05_11_Rhino_wars/2017_03_27_Rhinos.odt -------------------------------------------------------------------------------- /2017_08_15_Web_scraping/big_data_bw.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_08_15_Web_scraping/big_data_bw.jpg -------------------------------------------------------------------------------- /2017_08_15_Web_scraping/firefox_arrow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/petrkeil/Blog/a32e8fd240fe0237ea9f64854aca0f7dfec0b7b6/2017_08_15_Web_scraping/firefox_arrow.png -------------------------------------------------------------------------------- /2017_08_15_Web_scraping/web_scraping.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Basic web scraping in R, with focus on `rvest` and `RSelenium`" 3 | author: "Petr Keil" 4 | date: "August 17, 2017" 5 | output: 6 | html_document: 7 | highlight: pygments 8 | number_sections: yes 9 | theme: cerulean 10 | toc: yes 11 | --- 12 | 13 | ```{r setup, include=FALSE} 14 | knitr::opts_chunk$set(echo = TRUE) 15 | ``` 16 | 17 | # Web scraping? 18 | 19 | Web *scraping* is the metaphor used for the practice of getting data that weren't designed 20 | to be programatically consumed off the web ([Dale 2016](http://kyrandale.com/blog/data-visualization-python-javascript/)). Almost any repetitive Web structure or pattern that you see in your browser can be 21 | scraped and turned to scientific data. 22 | 23 | # Inspecting web elements 24 | 25 | To do scraping, you need to be able to inspect *web elements*. In Firefox or Chrome you can hit `ctrl + shift + c` to invoke the element inspector. It looks like this: 26 | 27 | ![](firefox_arrow.png) 28 | 29 | \n \n 30 | 31 | The red arrow shows the button which allows to identify the 32 | elements on any web page using **locator schemes**. The most important locator schemes are: 33 | 34 | - [CSS selector](https://en.wikipedia.org/wiki/Cascading_Style_Sheets) 35 | - id 36 | - class 37 | - [xpath](https://en.wikipedia.org/wiki/XPath) 38 | 39 | You can right-click anywhere in the code, then `Copy` and select `CSS Selector`. Now it's in your clipboard, ready for R. In Google chrome you can also copy the xpaths. 40 | 41 | Alternatively, check the [Selectorgadget](http://selectorgadget.com/) extension 42 | for Chrome. 43 | 44 | -------------------------------------------------------------------------------- 45 | 46 | # R packages for scraping 47 | 48 | An overview of what R can do with the web is on the [web technologies CRAN Task View](https://cran.r-project.org/web/views/WebTechnologies.html). 49 | 50 | Two important web-scraping R packages are: 51 | 52 | - [rvest](https://github.com/hadley/rvest) by Hadley Wickham. This is part of Wickham's user-friendly *tidyverse*. It works great with *pipes* and packages such as `magrittr`, `lubridate` or `plyr`. The main element locator schemes are *CSS selectors* and *xpaths*. 53 | - [RSelenium](https://cran.r-project.org/web/packages/RSelenium/vignettes/RSelenium-basics.html) which is an R interface to [Selenium 2.0 WebDriver](https://seleniumhq.github.io/docs/wd.html). The Swiss army knife of R web scraping, it can work with all sorts of interactive web elements, and it works with many locator schemes, including *id*, *xpath*, *CSS selectors*, *classes* and so on. 54 | 55 | For even more heavyweight scraping look at Python's package `Scrapy`. 56 | 57 | -------------------------------------------------------------------------------- 58 | 59 | # Simple scraping with `rvest` 60 | 61 | ```{r, message=FALSE, warning=FALSE} 62 | library(rvest) 63 | library(magrittr) 64 | ``` 65 | 66 | Note: It pays off to understand the pipe operator `%>%` when working with `rvest`. 67 | 68 | ## Example: downloading a simple html table 69 | 70 | Here we will use `rvest` to download a [table with numbers of Nobel laureates](https://en.wikipedia.org/wiki/List_of_countries_by_Nobel_laureates_per_capita) in different countries from Wikipedia. 71 | 72 | First, I'll use the element locator in my browser (`ctrl + shift + c`) to get the *CSS selector* of the table. Then I'll download the entire web page to R, and parse it to XML using `read_html`, extract the table node using `html_node`, and then convert it to a `data_frame` using `html_table`: 73 | 74 | ```{r} 75 | nobel.table <- read_html("https://en.wikipedia.org/wiki/List_of_countries_by_Nobel_laureates_per_capita") %>% 76 | html_node(css = "#mw-content-text > div > table:nth-child(9)") %>% 77 | html_table() 78 | ``` 79 | 80 | Check the table: 81 | 82 | ```{r} 83 | head(nobel.table) 84 | ``` 85 | 86 | We may still need some string operations (`gsub`, `grep`, `strsplit`, ...) to clean the data. 87 | 88 | 89 | 90 | ## Example: `data.frame` from a web structure that isn't a table 91 | 92 | Here we will scrape a web structure that is not a table, but looks sufficiently regular ('pattern-ish') to be convertible to a table. It is a simple [list of Nobel laureates by country](https://en.wikipedia.org/wiki/List_of_Nobel_laureates_by_country). 93 | 94 | We'll need some more sophisticated work with the CSS selectors. Check out the element inspector in your browser first (`ctrl + shift + c`), and look for some common properties of the headers and the text. 95 | 96 | First, download and parse the page with `read_html`, extract the country names with `html_nodes` using the CSS selector, and then convert the XML to text using `html_text`. 97 | 98 | ```{r} 99 | countries <- read_html("https://en.wikipedia.org/wiki/List_of_Nobel_laureates_by_country") %>% 100 | html_nodes(css = "h2 > span:first-child") %>% 101 | html_text() 102 | 103 | countries <- countries[2:(length(countries)-2)] # delete some non-country headings 104 | countries <- append(countries, "Tibet", after=56) # we need to accomodate the 14th Dalai Lama 105 | countries 106 | ``` 107 | 108 | Second, get the lists of laureates using exactly the same approach, but a different CSS selector: 109 | 110 | ```{r} 111 | laureates <- read_html("https://en.wikipedia.org/wiki/List_of_Nobel_laureates_by_country") %>% 112 | html_nodes(css = "h2 ~ ol") %>% 113 | html_text() %>% 114 | strsplit(split="\n") 115 | ``` 116 | 117 | Finally, put the laureates and the countries together: 118 | 119 | ```{r} 120 | names(laureates) <- countries 121 | laureates <- stack(laureates) 122 | ``` 123 | 124 | Let's check the resulting `data.frame`: 125 | 126 | ```{r} 127 | head(laureates) 128 | ``` 129 | 130 | As in the previous example, we may need some string operations (`gsub`, `grep`, `strsplit`, ...) to further clean the data. 131 | 132 | -------------------------------------------------------------------------------- 133 | 134 | 135 | # Advanced scraping with `RSelenium` 136 | 137 | Interactive elements on the Web can be simple `html` forms, and these can be 138 | scarped with `rvest`. However, often there are forms, clickable buttons, interactive graphics 139 | and fill-in forms based on *JavaScript*, and for these we need some niftier tools. 140 | 141 | With `RSelenium` you can create an R object virutally from any element of a webpage, 142 | and you can emulate actions such as mouse clicks, you can fill-in forms, etc. -- you essentially 143 | give R your browser, your mouse and your keyboard, and you specify what R should do with these tools. 144 | 145 | There is a really nice and [comprehensive tutorial](https://cran.r-project.org/web/packages/RSelenium/vignettes/RSelenium-basics.html). You can also open it by typing: 146 | 147 | ```{r, eval=FALSE} 148 | vignette("RSelenium-basics", package = "RSelenium") 149 | ``` 150 | 151 | 152 | ## Installation of `RSelenium`: Docker and Selenium Server 153 | 154 | Installation of the R package is easy: 155 | 156 | ```{r, eval=FALSE} 157 | install.packages("RSelenium") 158 | ``` 159 | 160 | However, to make `RSelenium` work, you may need to fiddle a bit, since it won't run on its own -- 161 | you need to install some additional stuff on your computer. You will need to install and run a **Selenium Server**. The most reliable way to do this is to use something called a **Docker container**. For instructions on how to set Docker and Selenium Server on your operating system type: 162 | 163 | ```{r, eval=FALSE} 164 | vignette("RSelenium-docker", package = "RSelenium") 165 | ``` 166 | 167 | It took me about 2 hours to figure it out. 168 | 169 | Also, before using `RSelenium`, you need to start the Selenium Server. I am on Ubuntu Linux, so do it in the terminal: 170 | 171 | ``` 172 | sudo docker run -d -p 4445:4444 selenium/standalone-firefox:2.53.0 173 | sudo docker ps 174 | ``` 175 | 176 | ## Example: Scraping biodiversity data from a page with interactive web elements 177 | 178 | In this example we will interact with [GlobalTreeSearch form](http://www.bgci.org/global_tree_search.php?sec=globaltreesearch): we will fill in the genus and species fields, search the database, and retrieve the results, all from within R. 179 | 180 | First, explore the page with the element inspector in your browser first (`ctrl + shift + c`). 181 | 182 | To interact with the page, I've written a simple function `get.tree` which takes the `genus` and `species` arguments (character strings), and returns countries in which the species occurs -- the comments should make the idea quite obvious: 183 | 184 | ```{r} 185 | get.tree <- function(genus, species) 186 | { 187 | require(RSelenium) 188 | 189 | # open the remote driver 190 | remDr <- remoteDriver(port = 4445L) 191 | remDr$open(silent = TRUE) 192 | # 193 | # go to the webpage 194 | remDr$navigate("http://www.bgci.org/global_tree_search.php?sec=globaltreesearch") 195 | remDr$refresh() # refresh the page 196 | 197 | # create R objects from the website elements 198 | genusElem <- remDr$findElement(using = 'id', value = "genus-field") 199 | specElem <- remDr$findElement(using = 'id', value = "species-field") 200 | buttElem <- remDr$findElement(using = 'class', value = "btn_ohoDO") 201 | 202 | # fill in the forms with the genus and species names 203 | genusElem$sendKeysToElement(list(genus)) 204 | specElem$sendKeysToElement(list(species)) 205 | 206 | # click the search button 207 | buttElem$clickElement() 208 | 209 | # get the output 210 | out <- remDr$findElement(using = "css", value="td.cell_1O3UaG:nth-child(4)") 211 | out <- out$getElementText()[[1]] # extract the actual text string 212 | out <- strsplit(out, split="; ")[[1]] # split the text to a character vector 213 | 214 | # close the remote driver 215 | remDr$close() 216 | 217 | return(out) 218 | } 219 | ``` 220 | 221 | Let's try it out: 222 | 223 | ```{r} 224 | get.tree("Abies","alba") 225 | ``` 226 | 227 | # Other ideas, notes, useful stuff 228 | 229 | - **Chrome developer tools**: there is more to them then just the elements tab. Have a look at Network for live monitoring and console for direct interaction (js). Ressource http://discover-devtools.codeschool.com 230 | 231 | - **tcpdump/wireshark** can inspect the actual network traffic (also outside of the browser). 232 | 233 | - **wget** and curl for fetching directly accessible data. They have switches for authentication and user agent (string that client supplies to server to communicate capabilities). 234 | 235 | - Knowledge of **regular expressions** 236 | 237 | -------------------------------------------------------------------------------- 238 | 239 | # Session information 240 | 241 | ```{r} 242 | sessionInfo() 243 | ``` 244 | 245 | 246 | 247 | 248 | --------------------------------------------------------------------------------