├── .nojekyll ├── figures ├── marwick_workflow.png ├── screenshot_rrtools.png ├── screenshot_rstudio.png ├── marwick_T_Pi_researcher.png └── Data-Managerment-title-slide-16-9.jpg ├── rrtools.tutorial.mpi2020.Rproj ├── pkgname └── .Rprofile ├── .gitignore ├── LICENSE ├── styles.css └── index.Rmd /.nojekyll: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /figures/marwick_workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nevrome/rrtools.tutorial.mpi2020/master/figures/marwick_workflow.png -------------------------------------------------------------------------------- /figures/screenshot_rrtools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nevrome/rrtools.tutorial.mpi2020/master/figures/screenshot_rrtools.png -------------------------------------------------------------------------------- /figures/screenshot_rstudio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nevrome/rrtools.tutorial.mpi2020/master/figures/screenshot_rstudio.png -------------------------------------------------------------------------------- /figures/marwick_T_Pi_researcher.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nevrome/rrtools.tutorial.mpi2020/master/figures/marwick_T_Pi_researcher.png -------------------------------------------------------------------------------- /figures/Data-Managerment-title-slide-16-9.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nevrome/rrtools.tutorial.mpi2020/master/figures/Data-Managerment-title-slide-16-9.jpg -------------------------------------------------------------------------------- /rrtools.tutorial.mpi2020.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | BuildType: Website 16 | -------------------------------------------------------------------------------- /pkgname/.Rprofile: -------------------------------------------------------------------------------- 1 | cat(crayon::bold(' 2 | This project was set up by rrtools. 3 | ')) 4 | cat(' 5 | You can start working now or apply some more basic configuration. 6 | ') 7 | cat('Check out ') 8 | cat(crayon::underline('https://github.com/benmarwick/rrtools')) 9 | cat(' for an explanation of all the project configuration functions of rrtools. 10 | ') 11 | cat('Or run the rrtools configuration addin: ') 12 | cat(crayon::cyan('rrtools.addin::rrtools_assistant() ')) 13 | cat(crayon::underline('https://github.com/nevrome/rrtools.addin 14 | 15 | ')) 16 | invisible(file.remove('.Rprofile')) 17 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # History files 2 | .Rhistory 3 | .Rapp.history 4 | 5 | # Session Data files 6 | .RData 7 | 8 | # User-specific files 9 | .Ruserdata 10 | 11 | # Example code in package build process 12 | *-Ex.R 13 | 14 | # Output files from R CMD build 15 | /*.tar.gz 16 | 17 | # Output files from R CMD check 18 | /*.Rcheck/ 19 | 20 | # RStudio files 21 | .Rproj.user/ 22 | 23 | # produced vignettes 24 | vignettes/*.html 25 | vignettes/*.pdf 26 | 27 | # OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3 28 | .httr-oauth 29 | 30 | # knitr and R markdown default cache directories 31 | *_cache/ 32 | /cache/ 33 | 34 | # Temporary files created by R markdown 35 | *.utf8.md 36 | *.knit.md 37 | 38 | # R Environment Variables 39 | .Renviron 40 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Clemens Schmid 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /styles.css: -------------------------------------------------------------------------------- 1 | p { 2 | font-size: 20px !important; 3 | } 4 | 5 | li { 6 | font-size: 20px !important; 7 | } 8 | 9 | details { 10 | font-size: 20px !important; 11 | } 12 | 13 | #main .nav-pills > li.active > a, 14 | #main .nav-pills > li.active > a:hover, 15 | #main .nav-pills > li.active > a:focus { 16 | background-color: #22983B; 17 | } 18 | 19 | #main .nav-pills > li > a:hover { 20 | background-color: #22983B; 21 | } 22 | 23 | h1,h2,h3,h4,h5,h6,legend{ 24 | color: #22983B; 25 | } 26 | 27 | #nav-top span.glyphicon { 28 | color: #22983B; 29 | } 30 | 31 | #table-of-contents header{ 32 | color:#22983B; 33 | } 34 | 35 | #table-of-contents h2{ 36 | background-color:#22983B; 37 | } 38 | 39 | #main a { 40 | background-image: linear-gradient(180deg,#d64a70,#d64a70); 41 | color:#c7254e; 42 | } 43 | 44 | a:hover{ 45 | color:#3d1308 46 | } 47 | 48 | a:visited{ 49 | color:#3d1308 50 | } 51 | 52 | #table-of-contents { 53 | background:#3D3D3D !important; 54 | } 55 | 56 | #sidebar { 57 | background:#3D3D3D !important; 58 | } 59 | 60 | #sidebar h2{ 61 | background-color:#22983B !important; 62 | } 63 | 64 | #postamble { 65 | background:#3F88C5 !important; 66 | border-top:solid 10px #3D3D3D; 67 | } 68 | 69 | 70 | /* scroll gallery */ 71 | .gallery { 72 | display: flex; 73 | height: 90vh; 74 | overflow-x: scroll; 75 | scroll-snap-type: x mandatory; 76 | margin-top: 30px; 77 | margin-bottom: 30px; 78 | border-style: solid; 79 | border-width: 1px; 80 | border-color: lightgrey; 81 | } 82 | 83 | .gallery div { 84 | scroll-snap-align: start; 85 | min-width: 700px; 86 | justify-content: center; 87 | align-items: center; 88 | padding-left: 20px; 89 | padding-right: 20px; 90 | margin-right: 20px; 91 | } 92 | 93 | -------------------------------------------------------------------------------- /index.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | name: "rrtools.tutorial.mpi2020" 3 | output_dir: "." 4 | author: Clemens Schmid 5 | date: January 2020 6 | title: Research Compendia with R 7 | output: 8 | rmdformats::readthedown: 9 | code_folding: NULL 10 | self_contained: true 11 | thumbnails: false 12 | lightbox: false 13 | css: styles.css 14 | editor_options: 15 | chunk_output_type: console 16 | --- 17 | 18 | ```{r setup, include=FALSE} 19 | knitr::opts_chunk$set(eval = FALSE) 20 | 21 | astro <- function() { 22 | fontawesome::fa("user-astronaut", fill = "#c7254e") 23 | } 24 | rocket <- function() { 25 | fontawesome::fa("rocket", fill = "#c7254e") 26 | } 27 | ``` 28 | 29 | [**https://nevrome.github.io/rrtools.tutorial.mpi2020**](https://nevrome.github.io/rrtools.tutorial.mpi2020) 30 | Github: https://github.com/nevrome/rrtools.tutorial.mpi2020 31 | 32 | ![](figures/Data-Managerment-title-slide-16-9.jpg) 33 | 34 | This tutorial was compiled by [Clemens Schmid](https://nevrome.de/) for a [workshop on reproducible research and data management at MPI-SHH](https://rrdm-shh.github.io) in January 2020. It's based on and inspired by two workshops prepared by [Ben Marwick](http://faculty.washington.edu/bmarwick/) for the [SAA2019 conference](https://benmarwick.github.io/2019-04-10-saa/) and [Brown University Digital Archaeology Lab](https://benmarwick.github.io/2019-12-09-brown/). 35 | 36 | # The research compendium 37 | 38 |
39 |
40 | 41 | ## Idea 42 | 43 | > A research compendium is a manuscript accompanied by code and data files (or persistent links to reputable online repositories) that allows reviewers and readers to reproduce and extend the results without needing any further materials from the original authors [...]. 44 | > 45 | > -- Marwick 2017, 442. 46 | 47 | A research compendium contains **relevant information** to make the **scientific process** behind a book, book chapter or journal article **more transparent** and **reproducible**. 48 | 49 | Ideally it contains **all** **data**, **code** and **text** necessary to compile the published document. 50 | 51 | This might be **difficult sometimes** due to raw data size, publishing **restrictions** or parts of the workflow that are **not quantitative** or inherently impossible to reproduce. 52 | 53 | The term research compendium therefore covers a whole **range of imaginable degrees of completeness**. 54 | 55 |
56 |
57 | 58 | ## Data 59 | 60 | Research can only be **reproduced**, **checked** and **expanded** when underlying raw data is available. 61 | 62 | Data sharing is a **moral obligation** and **non-trivial**. 63 | 64 | Marwick & Birch 2018 have some recommendations: 65 | 66 | - Anticipate how your data will be used 67 | - Keep raw data **raw** 68 | - Store data in **open formats** 69 | - Data should be structured for analysis (**tidy data**) 70 | - Data should be uniquely identifiable (**persistent references**) 71 | - Provide relevant **metadata** 72 | - Adopt the proper **privacy** protocols 73 | - Use a trustworthy **repository** 74 | - Use an open **license** 75 | 76 |
77 |
78 | 79 | ## Code 80 | 81 | In most contexts of **quantitative research** data analysis can be expressed in the form of a **computer script**. Other researchers should be able to run our script to **obtain the same statistical results and data visualizations**. 82 | 83 | **Computational reproducibility** is an important foundation of scientific progress, that requires a **new type of researcher**: 84 | 85 | ![](figures/marwick_T_Pi_researcher.png) 86 | 87 |
88 |
89 | 90 | ## Text 91 | 92 | Text can be integrated with data and code to be more **interlinked**, **transparent** and **didactically powerful**: 93 | 94 | This can preciscly answer questions like **"106 samples? Where is this number coming from?"**. Hopefully not `round(runif(1) * 200)`, but solid algorithms like `number_of_samples_with_raikenburg_treatment()`. 95 | 96 | In the future **interactive documents** may become more popular: e.g. Shiny Documents (https://github.com/nevrome/neiman1995) 97 | 98 |
99 |
100 | 101 | ## What we will do today 102 | 103 | Research compendia can be compiled in many different ways. 104 | 105 | I will present **one particular workflow** based on our R package [rrtools](https://github.com/benmarwick/rrtools). 106 | 107 | ![](figures/screenshot_rrtools.png) 108 | 109 |
110 |
111 | 112 | The following figure by [Marwick 2017](https://www.practicereproducibleresearch.org/case-studies/benmarwick.html) illustrates one implementation of this workflow. 113 | 114 |
115 | 116 |
117 | 118 | # R for reproducible research 119 | 120 |
121 |
122 | 123 | ## R scripting language 124 | 125 | [R](https://www.r-project.org/) is a **scripting language** and a framework for statistical data analysis. The way it is used for data analysis fits well to the research compendium concept. 126 | 127 | ```{r, eval=FALSE} 128 | library(magrittr) 129 | eat_carrot <- function(x) {append(x, "eats a carrot")} 130 | `%dance%` <- function(x, y) {c(x, "dances with", y)} 131 | horse1 <- "Betsy"; horse2 <- "Trapper" 132 | horse1 %>% eat_carrot() 133 | horse1 %dance% horse2 134 | ``` 135 | 136 | 1. It is an easy to learn programming language with a huge community in science. 137 | 2. It is supported by an advanced working environment: **RStudio** 138 | 3. It allows for direct integration of code and text: **RMarkdown** 139 | 4. It offers an established data structure to manage code, data and text: the **R package** 140 | 141 | If you never worked with R, you can use [swirl](https://swirlstats.com/students.html) to learn the basics. 142 | 143 |
144 |
145 | 146 | ## RStudio 147 | 148 | [RStudio](https://rstudio.com/products/rstudio/) is an **integrated development environment** (IDE) for R. 149 | 150 | ![](figures/screenshot_rstudio.png) 151 | 152 |
153 |
154 | 155 | ## RStudio 156 | 157 | `r astro()``r rocket()` 158 | Open RStudio. 159 | 160 | `r astro()``r rocket()` 161 | Inspect the RStudio interface. 162 | 163 | `r astro()``r rocket()` 164 | Open an R code file, add some code (`1 + 1`), select it and run it in the R console with ctrl + enter. 165 | 166 |
167 |
168 | 169 | ## R Markdown 170 | 171 | [Markdown](https://daringfireball.net/projects/markdown/) is a lightweight and easy-to-use **markup language** for styling your writing. 172 | 173 | ``` 174 | # Header 1 175 | ## Header 2 176 | 177 | - Bulleted 178 | - List 179 | 180 | **Bold** and _Italic_ and `Code` text or [Link](url) and ![Image](src) 181 | ``` 182 | 183 | [Here](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet)'s a cheatsheet that documents all basic Markdown features. 184 | 185 | Text written in Markdown can **easily converted** to more advanced layout systems like **HTML** or **LaTex** or **MS Word**. This website is written in Markdown. 186 | 187 | [RMarkdown](https://rmarkdown.rstudio.com/) is an advanced implementation of Markdown that allows to **combine text and code**. It adds the possibility to define chunks of code that run when the document is rendered. 188 | 189 |
190 |
191 | 192 | ## R Markdown 193 | 194 | `r astro()``r rocket()` 195 | Open an Rmarkdown file based on the default RStudio template, inspect it and render it with the Knit button. 196 | 197 |
198 |
199 | 200 | ## R Packages 201 | 202 | R packages are a core feature of R. They contain **ready to use functions, documentation and example data** in a **standard structure**. 203 | 204 | There currently are **>15000 R packages** mostly written and maintained by **volunteers** for all sorts of research questions and applications. 205 | 206 | ``` 207 | mypackage/ 208 | | 209 | ├── DESCRIPTION # Package metadata 210 | ├── R/ # R code 211 | ├── man/ # Function documentation 212 | ├── NAMESPACE/ # Exported names 213 | ├── vignettes/ # Extended documentation 214 | ├── data/ # (Example) data 215 | ├── tests/ # Unit tests 216 | ├── src/ # Compiled language code 217 | └── inst/ # Arbitrary, additional files 218 | ``` 219 | 220 | An example: https://github.com/nevrome/bleiglas 221 | 222 | [Here](http://r-pkgs.had.co.nz/)'s an excellent introduction if you want to create an own package. 223 | 224 |
225 |
226 | 227 | # rrtools setup 228 | 229 |
230 |
231 | 232 | ## rrtools 233 | 234 | [rrtools](https://github.com/benmarwick/rrtools) is a **mighty wizard** package written by Ben Marwick and colleagues that facilitates some of the steps of research compendium creation and maintenance. It has a **lot of dependencies** and it is **strongly opinionated**. 235 | 236 | ![](figures/screenshot_rrtools.png) 237 | 238 |
239 |
240 | 241 | ## Create an R package 242 | 243 | If rrtools is installed on your system you can start immediately to use it in R. 244 | 245 | `r astro()``r rocket()` 246 | Run `rrtools::use_compendium("~/test/mycompendium")` to create a basic R package with the name `project`. 247 | 248 | `r astro()``r rocket()` 249 | Inspect both the command output in the old and the new RStudio session. 250 | 251 | `r astro()``r rocket()` 252 | Inspect the compendium file structure. 253 | 254 | `r astro()``r rocket()` 255 | Edit the `DESCRIPTION` file (located in your `project` directory) to include some better metadata. 256 | 257 | For the future keep in mind to periodically update the `Imports:` section of the `DESCRIPTION` file with the names of packages used in the code we write in `/R` and the Rmd document(s). 258 | 259 |
260 |
261 | 262 | ## Activate version control 263 | 264 | `r astro()``r rocket()` 265 | Run `usethis::use_git_config(user.name = "", user.email = "")` to configure git. 266 | 267 | `r astro()``r rocket()` 268 | Run `usethis::use_git()` to initiate git for this project (`git init`). Follow the command line instruction of this function. 269 | 270 | `r astro()``r rocket()` 271 | Inspect the new Git tab in the top right RStudio panel. 272 | 273 | `r astro()``r rocket()` 274 | Run `usethis::browse_github_pat()` to get to the right page to create a github access token. This access token is needed to control Github remotely. When you generate the token (click “Generate new token”), make sure the “repo” scope is included by checking the “repo” box. Don’t save this token in your project, keep it elsewhere. 275 | 276 | `r astro()``r rocket()` 277 | Run `usethis::use_github(protocol = "ssh", auth_token = "your token")` to create a repository for your local project on Github. 278 | 279 | `r astro()``r rocket()` 280 | Test your Git + Github setup by editing the DESCRIPTION file ones more and by pushing the result. 281 | 282 |
283 |
284 | 285 | ## Establish the compendium file structure 286 | 287 | `r astro()``r rocket()` 288 | Run `rrtools::use_analysis()` to create the basic files and directory structure and transform this R package into a research compendium. 289 | 290 | `r astro()``r rocket()` 291 | Inspect the command line output of this function and the resulting file structure. 292 | 293 | ``` 294 | analysis/ 295 | | 296 | ├── paper/ 297 | │ ├── paper.Rmd # this is the main document to edit 298 | │ └── references.bib # this contains the reference list information 299 | │ 300 | ├── figures/ # location of the figures produced by the Rmd 301 | | 302 | ├── data/ 303 | │ ├── raw_data/ # data obtained from elsewhere 304 | │ └── derived_data/ # data generated during the analysis 305 | | 306 | └── templates 307 | ├── journal-of-archaeological-science.csl 308 | | # this sets the style of citations & reference list 309 | ├── template.docx # used to style the output of the paper.Rmd 310 | └── template.Rmd 311 | ``` 312 | 313 | `r astro()``r rocket()` 314 | Commit and push the setup. 315 | 316 |
317 |
318 | 319 | ## Inspect the manuscript 320 | 321 | `r astro()``r rocket()` 322 | Inspect the `./analysis/paper/paper.Rmd` file. 323 | 324 | `r astro()``r rocket()` 325 | Render it with the Knit button and inspect the resulting `.docx` file. 326 | 327 |
328 |
329 | 330 | # rrtools workflow 331 | 332 |
333 |
334 | 335 | ## Collect some data 336 | 337 | `r astro()``r rocket()` 338 | Create a file `names.txt` in `./analysis/data/raw_data` that contains the first names of some of the people around you. 339 | 340 | ``` 341 | Paul 342 | Hannah 343 | Anne 344 | Maxime 345 | Clemens 346 | ``` 347 | 348 |
349 |
350 | 351 | ## Add an R function to analyse and plot this data 352 | 353 | `r astro()``r rocket()` 354 | Create a file `plot_name_dist_matrix.R` in `./R` to define an R function. 355 | 356 | ```{r} 357 | #' Calculate Levenshtein distance between strings in the input vector 358 | #' 359 | #' @param x A character vector 360 | #' 361 | #' @return Nothing. Only called for the plot 362 | #' @export 363 | plot_name_dist_matrix <- function(x) { 364 | # calculate distances 365 | dist_matrix <- as.matrix(stringdist::stringdistmatrix(x, method = "lv")) 366 | # plot 367 | image(dist_matrix, axes = FALSE) 368 | axis(1, at = seq(0, 1, length = length(x)), labels = x) 369 | axis(2, at = seq(0, 1, length = length(x)), labels = x) 370 | text( 371 | expand.grid(seq(0, 1, length = length(x)), seq(0, 1, length = length(x))), 372 | labels = dist_matrix 373 | ) 374 | } 375 | ``` 376 | 377 |
378 |
379 | 380 | ## Make this function available in the manuscript file 381 | 382 | `r astro()``r rocket()` 383 | 384 | ### The lazy way 385 | 386 | Add a call to a function at the beginning of the `paper.Rmd` file that simply loads every function in the package: 387 | 388 | ```{r} 389 | devtools::load_all() 390 | ``` 391 | 392 | ### The knigths of R way 393 | 394 | Build the package documentation (CTRL+Shift+D) and install the package (CTRL+Shift+B). 395 | 396 |
397 |
398 | 399 | ## Incorporate data and code into the manuscript 400 | 401 | `r astro()``r rocket()` 402 | Edit `./analysis/paper/paper.Rmd` to include our analysis in an R code chunk with the chunk options `{r, fig.width = 10, fig.height=10}`. 403 | 404 | ```{r} 405 | names_vector <- readLines(here::here("analysis/data/raw_data/names.txt")) 406 | 407 | mycompendium::plot_name_dist_matrix(names_vector) 408 | ``` 409 | 410 | `r astro()``r rocket()` 411 | Render `paper.Rmd` again to see the result. 412 | 413 |
414 |
415 | 416 | # rrtools advanced 417 | 418 |
419 |
420 | 421 | ## README, Code of Conduct and Contribution 422 | 423 | Beyond the bare code an established open source software project in the 21st century should have at least the following three things: 424 | 425 | - A `README.md` file that gives a minimal description what this project is about, how it can be used and who made it 426 | - A guide how to contribute to this project `CONTRIBUTING.md` 427 | - A code of conduct that defines which behaviour we expect from participants: `CONDUCT.md` 428 | 429 | These documentation files are valuable for your research compendium as well. 430 | 431 | `r astro()``r rocket()` 432 | Run `rrtools::use_readme_rmd()` to create these files and inspect them. 433 | 434 | `r astro()``r rocket()` 435 | Render the `README.Rmd` file to a `README.md` file. Why is this intermediate step necessary? 436 | 437 |
438 |
439 | 440 | ## Licensing 441 | 442 | Code and data (!) in a research repository should come with a license declaration that declares the copyright holder and what can and can not be done legally with your intellectual property. 443 | 444 | Sometimes it's not easy to decide which of the established licenses fits your purpose best. There are websites that give you an overview, e.g. https://choosealicense.com, but real legal advice is always recommended. 445 | 446 | `r astro()``r rocket()` 447 | Test one of the following functions and see what it does. 448 | 449 | `usethis::use_mit_license(name = "your name")` 450 | `usethis::use_gpl3_license(name = "your name")` 451 | `usethis::use_lgpl_license(name = "your name")` 452 | `usethis::use_apl2_license(name = "your name")` 453 | `usethis::use_cc0_license(name = "your name")` 454 | `usethis::use_ccby_license(name = "your name")` 455 | 456 |
457 |
458 | 459 | ## CI/CD 460 | 461 | To test your workflow on another, independent system or to outsource some document processing steps you can work with services that provide on-the-fly virtual machines. 462 | 463 | `rrtools::use_travis(docker = FALSE)` creates a configuration file (`.travis.yml`) for the TravisCI service which you can directly link to your Github repository. 464 | 465 | An example: https://github.com/ISAAKiel/recexcavAAR 466 | And another example: https://github.com/nevrome/neomod_textdev 467 | 468 |
469 |
470 | 471 | ## Virtualisation 472 | 473 | To go beyond CI and to make our computational workflow completely independent of version changes in the software we use, we have to encapsulate it in a virtual environment, that simulates a computer with exactly the right software. 474 | 475 | A good solution for this is the [Docker container system](https://docs.docker.com/get-started/). 476 | 477 | You can start to set this up with `rrtools::use_dockerfile()`, which creates default configuration file (`Dockerfile`), but it requires some further considerations. 478 | 479 | An example: https://github.com/nevrome/cultrans.bronzeageburials.article2019 480 | 481 |
482 |
483 | 484 | ## Unit tests 485 | 486 | If the code in your compendium becomes more and more complex and you maintain a significant number of functions in the `R/` directory, it might be useful to establish [unit tests](http://r-pkgs.had.co.nz/tests.html). 487 | 488 | Unit tests reduce the number of bugs, force you to structure your code in a better way and make it overall more robust. 489 | 490 | `usethis::use_testthat()` adds the main components to get you started. 491 | 492 | An example: https://github.com/ropensci/c14bazAAR 493 | 494 |
495 |
496 | 497 | # Further reading 498 | 499 |

Eglen, S. J., Marwick, B., Halchenko, Y. O., Hanke, M., Sufi, S., Gleeson, P., … & Wachtler, T. (2017). Toward standard practices for sharing computer code and programs in neuroscience. Nature Neuroscience 20(6), 770-773. [DOI] [preprint] [PDF]

500 | 501 |

Marwick, B. 2017 Computational reproducibility in archaeological research: Basic principles and a case study of their implementation. Journal of Archaeological Method and Theory 24(2), 424-450. [DOI] [preprint] [code & data]

502 | 503 |

Marwick 2017 Using R and Related Tools for Reproducible Research in Archaeology. In Kitzes, J., Turek, D., & Deniz, F. (Eds.) The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. Oakland, CA: University of California Press. [online]

504 | 505 |

Marwick, B., & Birch, S. 2018 A Standard for the Scholarly Citation of Archaeological Data as an Incentive to Data Sharing. Advances in Archaeological Practice 1-19. [DOI] [preprint] [PDF] [code & data]

506 | 507 |

Marwick, B., Boettiger, C., & Mullen, L. 2017 Packaging data analytical work reproducibly using R (and friends). The American Statistician [DOI] [preprint]

508 | 509 |

Marwick, B, d’Alpoim Guedes, J., Barton, C. M., Bates, L. A., Baxter, M., Bevan, A., Bollwerk, E. A., Bocinsky, R. K., Brughmans, T., Carter, A. K., Conrad, C., Contreras, D. A., Costa, S., Crema, E. R., Daggett, A., Davies, B., Drake, B. L., Dye, T. S., France, P., Fullagar, R., Giusti, D., Graham, S., Harris, M. D., Hawks, J., Health, S., Huffer, D., Kansa, E. C., Kansa, S. W., Madsen, M. E., Melcher, J., Negre, J., Neiman, F. D., Opitz, R., Orton, D. C., Przstupa, P., Raviele, M., Riel-Savatore, J., Riris, P., Romanowska, I., Smith, J., Strupler, N., Ullah, I. I., Van Vlack, H. G., VanValkenburgh, N., Watrall, E. C., Webster, C., Wells, J., Winters, J., and Wren, C. D. (2017) Open science in archaeology. SAA Archaeological Record, 17(4), pp. 8-14. [PDF] [preprint]

510 | 511 |

Ram, K. B. Marwick 2017 Building Towards a Future Where Reproducible, Open Science is the Norm. In Kitzes, J., Turek, D., & Deniz, F. (Eds.) The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. Oakland, CA: University of California Press. [online]

512 | 513 |

Rokem, A., B. Marwick, V. Staneva 2017 Assessing Reproducibility. In Kitzes, J., Turek, D., & Deniz, F. (Eds.) The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. Oakland, CA: University of California Press. University of California Press. [online]

514 | 515 | --------------------------------------------------------------------------------