├── .Rbuildignore ├── .gitignore ├── .travis.yml ├── DESCRIPTION ├── LICENSE ├── LICENSE.md ├── NAMESPACE ├── R ├── dfs_idx.R ├── hexSticker.R ├── replace_null.R ├── roomba-package.R ├── rrroomba.R ├── run_shiny.R └── utils.R ├── README.Rmd ├── README.html ├── README.md ├── _pkgdown.yml ├── data ├── reddit.rda ├── simple.rda ├── twitter_data.rda └── vimeo.rda ├── docs ├── LICENSE-text.html ├── LICENSE.html ├── articles │ ├── index.html │ ├── twitter.html │ └── twitter_files │ │ └── figure-html │ │ └── unnamed-chunk-3-1.png ├── authors.html ├── docsearch.css ├── images │ └── shinydemo.gif ├── img │ └── sticker.png ├── index.html ├── jquery.sticky-kit.min.js ├── link.svg ├── pkgdown.css ├── pkgdown.js ├── pkgdown.yml └── reference │ ├── dfs_idx.html │ ├── index.html │ ├── list_names.html │ ├── pipe.html │ ├── reddit.html │ ├── replace_null.html │ ├── roomba-package.html │ ├── roomba.html │ ├── simple.html │ ├── twitter_data.html │ └── vimeo.html ├── images └── shinydemo.gif ├── img └── sticker.png ├── inst ├── Shiny │ ├── server.R │ └── ui.R ├── extdata │ └── topknitting.json └── figures │ ├── hexSticker.png │ └── vacuum.png ├── man ├── dfs_idx.Rd ├── list_names.Rd ├── reddit.Rd ├── replace_null.Rd ├── roomba-package.Rd ├── roomba.Rd ├── shiny_roomba.Rd ├── simple.Rd ├── twitter_data.Rd └── vimeo.Rd ├── roomba.Rproj ├── tests ├── testthat.R └── testthat │ ├── test-indexing.R │ └── test-utils.R └── vignettes ├── 2018-06-26-roomba.Rmd ├── 2018-06-26-roomba.html ├── 2018-06-26-roomba.md ├── runconf18_roomba.html ├── sticker.png └── twitter.Rmd /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.*\\.Rproj$ 2 | ^\\.Rproj\\.user$ 3 | ^\\.httr-oauth$ 4 | ^\\.travis\\.yml$ 5 | ^README\\.Rmd$ 6 | ^LICENSE\\.md$ 7 | ^\\images$ 8 | ^\\docs$ 9 | docs 10 | img 11 | images 12 | 13 | ^_pkgdown\.yml$ 14 | ^docs$ 15 | ^pkgdown$ 16 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .Rproj.user 2 | .Rhistory 3 | .RData 4 | .Ruserdata 5 | .httr-oauth 6 | docs 7 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | # R for travis: see documentation at https://docs.travis-ci.com/user/languages/r 2 | 3 | language: R 4 | sudo: false 5 | cache: packages 6 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Type: Package 2 | Package: roomba 3 | Title: Tidy up nested list hairballs 4 | Version: 0.1.0 5 | Authors@R: c( 6 | person("Amanda", "Dobbyn", role = "aut"), 7 | person("Christine", "Stawitz", role = "aut"), 8 | person("Isabella", "Velasquez", role = "aut"), 9 | person("Jim", "Hester", email = "james.f.hester@gmail.com", role = c("aut", "cre")), 10 | person("Laura", "DeCicco", role = "aut") 11 | ) 12 | Description: This is a package to transform large, multi-nested lists into a more 13 | user-friendly format. The initial focus is on making 14 | processing of return values from `jsonlite::fromJSON()` queries more seamless, 15 | but ideally this package should be useful for deeply-nested lists from an array 16 | of sources. 17 | License: MIT + file LICENSE 18 | Depends: 19 | R (>= 3.0) 20 | Imports: 21 | assertthat, 22 | dplyr, 23 | magrittr, 24 | purrr 25 | Suggests: 26 | ggplot2, 27 | jsonlite, 28 | knitr, 29 | rmarkdown, 30 | testthat, 31 | shiny 32 | VignetteBuilder: knitr 33 | Encoding: UTF-8 34 | LazyData: true 35 | Roxygen: list(markdown = TRUE) 36 | RoxygenNote: 6.0.1.9000 37 | URL: https://github.com/ropenscilabs/roomba 38 | BugReports: https://github.com/ropenscilabs/roomba/issues 39 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | YEAR: 2018 2 | COPYRIGHT HOLDER: Jim Hester 3 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | # MIT License 2 | 3 | Copyright (c) 2018 Jim Hester 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2: do not edit by hand 2 | 3 | export("%>%") 4 | export(dfs_idx) 5 | export(list_names) 6 | export(replace_null) 7 | export(roomba) 8 | export(shiny_roomba) 9 | importFrom(magrittr,"%>%") 10 | -------------------------------------------------------------------------------- /R/dfs_idx.R: -------------------------------------------------------------------------------- 1 | #' Perform a recursive depth first search of a function 2 | #' @inheritParams purrr::map 3 | #' @export 4 | dfs_idx <- function(.x, .f) { 5 | .f <- purrr::as_mapper(.f) 6 | res <- list() 7 | num <- 0L 8 | walk <- function(x, idx) { 9 | for (i in seq_along(x)) { 10 | if (isTRUE(tryCatch(.f(x[[i]]), error = function(e) FALSE))) { 11 | res[[num <<- num + 1L]] <<- append(idx, i) 12 | } 13 | if (is.list(x[[i]])) { 14 | walk(x[[i]], append(idx, i)) 15 | } 16 | } 17 | } 18 | walk(.x, integer()) 19 | res 20 | } 21 | -------------------------------------------------------------------------------- /R/hexSticker.R: -------------------------------------------------------------------------------- 1 | # library(hexSticker) 2 | # 3 | # # Icon made by freepik.com from www.flaticon.com 4 | # 5 | # outline = "#EAEBEF" 6 | # background = "#64A709" # official Roomba green! 7 | # sticker(".\\inst\\figures\\vacuum.png", 8 | # package = "roomba", 9 | # h_fill = background, 10 | # h_color = outline, 11 | # p_size = 25, 12 | # s_width = 0.5, 13 | # s_height = 0.5, 14 | # s_x = 1, 15 | # filename = ".\\inst\\figures\\hexSticker.png") 16 | -------------------------------------------------------------------------------- /R/replace_null.R: -------------------------------------------------------------------------------- 1 | #' Replace NULLs 2 | #' 3 | #' @description Replace all the empty values in a list 4 | #' @param x list to use 5 | #' @param replacement Replacement value for missing values 6 | #' @examples 7 | #' list(a = NULL, b = 1, c = list(foo = NULL, bar = NULL)) %>% replace_null() 8 | #' 9 | #' @export 10 | 11 | replace_null <- function(x, replacement = NA) { 12 | empty_idx <- dfs_idx(x, ~ length(.x) == 0 || is.na(.x)) 13 | for (i in empty_idx) { 14 | x[[i]] <- replacement 15 | } 16 | x 17 | } 18 | -------------------------------------------------------------------------------- /R/roomba-package.R: -------------------------------------------------------------------------------- 1 | #' roomba package 2 | #' 3 | #' \tabular{ll}{ 4 | #' Package: \tab roomba\cr 5 | #' Type: \tab Package\cr 6 | #' } 7 | #' 8 | #' Collection of functions to do deal with deeply nested data 9 | #' 10 | #' @name roomba-package 11 | #' @docType package 12 | NULL 13 | 14 | #' Example Twitter data 15 | #' 16 | #' Example data from Twitter API 17 | #' 18 | #' twitter_data[[1]][["id"]] 19 | #' @source 20 | "twitter_data" 21 | 22 | #' Example data from reddit 23 | #' @source 24 | "reddit" 25 | 26 | #' Example data from vimeo 27 | #' @source 28 | "vimeo" 29 | 30 | #' Example data with simple nested list 31 | "simple" 32 | -------------------------------------------------------------------------------- /R/rrroomba.R: -------------------------------------------------------------------------------- 1 | #' Roomba 2 | #' 3 | #' @description Tidy your nested list 4 | #' @param inp List to tidy 5 | #' @param cols Columns to keep 6 | #' @param default Replacement for NULL values. Defaults to NA. 7 | #' @param keep Should all or any data be kept? 8 | #' 9 | #' @importFrom magrittr %>% 10 | #' @export 11 | #' 12 | #' @examples 13 | #' 14 | #' simple %>% roomba(cols = c("name", "goodstuff"), keep = any) 15 | #' simple %>% roomba(cols = c("name", "goodstuff"), keep = any) 16 | roomba <- function(inp, cols = NULL, default = NA, 17 | keep = all) { 18 | 19 | assertthat::assert_that(length(inp) > 0, 20 | msg = "Input is of length 0.") 21 | 22 | assertthat::assert_that(!is.null(cols), 23 | msg = "cols must be non-NULL.") 24 | 25 | keep <- match.fun(keep) 26 | 27 | inp_clean <- inp %>% 28 | replace_null(replacement = default) 29 | # -- Message that NULLs were replaced with NAs? 30 | 31 | has_good_stuff <- function(data, cols) { 32 | keep(purrr::map_lgl(cols, ~ length(data[[.x]]) > 0)) 33 | } 34 | 35 | indices <- 36 | dfs_idx(inp_clean, ~ has_good_stuff(data = .x, cols = cols)) 37 | 38 | out <- indices %>% purrr::map( 39 | function(.x) { 40 | res <- inp[[.x]] 41 | res[names(res) %in% cols] 42 | }) %>% replace_null() %>% 43 | dplyr::bind_rows() 44 | 45 | return(out) 46 | } 47 | 48 | -------------------------------------------------------------------------------- /R/run_shiny.R: -------------------------------------------------------------------------------- 1 | #' Run the roomba app 2 | #' 3 | #' Run the roomba app 4 | #' @export 5 | #' @param browse Logical. Use browser for running Shiny app. 6 | #' @examples 7 | #' \dontrun{ 8 | #' if(require(shiny)){ 9 | #' shiny_roomba() 10 | #' } 11 | #' } 12 | shiny_roomba <- function(browse=TRUE){ 13 | shiny::runApp(system.file('Shiny', package='roomba'), launch.browser = browse) 14 | } 15 | -------------------------------------------------------------------------------- /R/utils.R: -------------------------------------------------------------------------------- 1 | #' List all names in a list 2 | #' @param x list to use 3 | #' @export 4 | list_names <- function(x) { 5 | name_idx <- dfs_idx(x, ~ length(names(.x)) > 0) 6 | unique(unlist(purrr::map(name_idx, ~ names(x[[.x]])))) 7 | } 8 | 9 | replace_single_null <- function(e, replacement = NA_character_) { 10 | if (length(e)) { 11 | return(e) 12 | } else { 13 | e <- replacement 14 | return(e) 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /README.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | output: github_document 3 | --- 4 | 5 | ```{r setup, include = FALSE} 6 | knitr::opts_chunk$set( 7 | collapse = TRUE, 8 | comment = "#>", 9 | fig.path = "man/figures/README-", 10 | out.width = "100%" 11 | ) 12 | ``` 13 | 14 | # roomba roomba_gif 15 | 16 | [![Travis build status](https://travis-ci.com/ropenscilabs/roomba.svg?branch=master)](https://travis-ci.com/ropenscilabs/roomba) 17 | 18 | 19 | This is a package to transform large, multi-nested lists into a more user-friendly format (i.e. a `tibble`) in `R`. The initial focus is on making processing of return values from `jsonlite::fromJSON()` queries more seamless, but ideally this package should be useful for deeply-nested lists from an array of sources. 20 | 21 | 22 | 23 | 24 |

25 | roomba_gif 26 |

27 | 28 | 29 | *Key features:* 30 | 31 | 32 | * `roomba()` searches deeply-nested list for names specified in `cols` (a character vector) and returns a `tibble` with the associated column titles. Nothing further about nesting hierarchy or depth need be specified. 33 | 34 | * Handles empty values gracefully by substituting `NULL` values with `NA` or user-specified value in `default`, or truncates lists appropriately. 35 | 36 | * If you're only interested in sniffing out and replacing all `NULL`s, turn to the `replace_null()` function. 37 | 38 | * Option to `keep` `any` or `all` data from the columns supplied 39 | 40 | ## Installation 41 | 42 | You can install the development version from [GitHub](https://github.com/) with: 43 | 44 | ```{r, eval=FALSE} 45 | # install.packages("devtools") 46 | devtools::install_github("cstawitz/roomba") 47 | ``` 48 | 49 | ## Usage 50 | 51 | Say we have some JSON from a pesky API. 52 | 53 | ```{r example, message=FALSE} 54 | library(roomba) 55 | 56 | json <- ' 57 | { 58 | "stuff": { 59 | "buried": { 60 | "deep": [ 61 | { 62 | "location": "here", 63 | "name": "Laura DeCicco", 64 | "super_power": "fixing merge conflicts", 65 | "other_secret_power": [] 66 | }, 67 | { 68 | "location": "here", 69 | "name": "Amanda Dobbyn", 70 | "super_power": "flight", 71 | "more_nested_stuff": 4 72 | } 73 | ], 74 | "alsodeep": 2342423234, 75 | "stilldeep": { 76 | "even_deeper": [ 77 | { 78 | "location": "not here", 79 | "name": "Jim Hester", 80 | "super_power": [] 81 | }, 82 | { 83 | "location": "here", 84 | "name": "Christine Stawitz", 85 | "super_power": "invisibility", 86 | "more_nested_stuff": 5 87 | }, 88 | { 89 | "location": "here", 90 | "name": "Isabella Velasquez", 91 | "super_power": "teleportation" 92 | } 93 | ] 94 | } 95 | } 96 | } 97 | }' 98 | ``` 99 | 100 | The JSON becomes a nested R list, 101 | 102 | ```{r} 103 | super_data <- json %>% 104 | jsonlite::fromJSON(simplifyVector = FALSE) 105 | ``` 106 | 107 | which we can pull data into the columns we want with `roomba`. 108 | 109 | ```{r} 110 | super_data %>% 111 | roomba(cols = c("name", "super_power", "more_nested_stuff"), keep = any) 112 | ``` 113 | 114 | 115 |
116 | 117 | Let's try with a real-world Twitter example (see package data to use this data). 118 | 119 | 120 | ```{r} 121 | roomba(twitter_data, c("created_at", "name")) 122 | ``` 123 | 124 | # Shiny app included! 125 | 126 |

127 | roomba_gif 128 |

129 | 130 | Run the app like this: 131 | ```{r eval=FALSE} 132 | shiny_roomba() 133 | 134 | ``` 135 | 136 |
137 | 138 | # What did that original data look like??? 139 | 140 | Feast your eyes on the original `super_data` list! 141 | 142 | ```{r} 143 | super_data 144 | ``` 145 | 146 | And just the *first* element of the `twitter` dataset `r emo::ji("scream")` 147 | 148 | ```{r} 149 | twitter_data[[1]] 150 | ``` 151 | 152 | 153 |
154 | 155 | **Happy cleaning!** 156 | 157 |

158 | roomba_gif 159 |

160 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | roomba roomba_gif 3 | ==================================================================================== 4 | 5 | [![Travis build status](https://travis-ci.com/ropenscilabs/roomba.svg?branch=master)](https://travis-ci.com/ropenscilabs/roomba) 6 | 7 | This is a package to transform large, multi-nested lists into a more user-friendly format (i.e. a `tibble`) in `R`. The initial focus is on making processing of return values from `jsonlite::fromJSON()` queries more seamless, but ideally this package should be useful for deeply-nested lists from an array of sources. 8 | 9 | 10 |

11 | roomba_gif 12 |

13 | *Key features:* 14 | 15 | - `roomba()` searches deeply-nested list for names specified in `cols` (a character vector) and returns a `tibble` with the associated column titles. Nothing further about nesting hierarchy or depth need be specified. 16 | 17 | - Handles empty values gracefully by substituting `NULL` values with `NA` or user-specified value in `default`, or truncates lists appropriately. 18 | 19 | - If you're only interested in sniffing out and replacing all `NULL`s, turn to the `replace_null()` function. 20 | 21 | - Option to `keep` `any` or `all` data from the columns supplied 22 | 23 | Installation 24 | ------------ 25 | 26 | You can install the development version from [GitHub](https://github.com/) with: 27 | 28 | ``` r 29 | # install.packages("devtools") 30 | devtools::install_github("cstawitz/roomba") 31 | ``` 32 | 33 | Usage 34 | ----- 35 | 36 | Say we have some JSON from a pesky API. 37 | 38 | ``` r 39 | library(roomba) 40 | 41 | json <- ' 42 | { 43 | "stuff": { 44 | "buried": { 45 | "deep": [ 46 | { 47 | "location": "here", 48 | "name": "Laura DeCicco", 49 | "super_power": "fixing merge conflicts", 50 | "other_secret_power": [] 51 | }, 52 | { 53 | "location": "here", 54 | "name": "Amanda Dobbyn", 55 | "super_power": "flight", 56 | "more_nested_stuff": 4 57 | } 58 | ], 59 | "alsodeep": 2342423234, 60 | "stilldeep": { 61 | "even_deeper": [ 62 | { 63 | "location": "not here", 64 | "name": "Jim Hester", 65 | "super_power": [] 66 | }, 67 | { 68 | "location": "here", 69 | "name": "Christine Stawitz", 70 | "super_power": "invisibility", 71 | "more_nested_stuff": 5 72 | }, 73 | { 74 | "location": "here", 75 | "name": "Isabella Velasquez", 76 | "super_power": "teleportation" 77 | } 78 | ] 79 | } 80 | } 81 | } 82 | }' 83 | ``` 84 | 85 | The JSON becomes a nested R list, 86 | 87 | ``` r 88 | super_data <- json %>% 89 | jsonlite::fromJSON(simplifyVector = FALSE) 90 | ``` 91 | 92 | which we can pull data into the columns we want with `roomba`. 93 | 94 | ``` r 95 | super_data %>% 96 | roomba(cols = c("name", "super_power", "more_nested_stuff"), keep = any) 97 | #> # A tibble: 5 x 3 98 | #> name super_power more_nested_stuff 99 | #> 100 | #> 1 Laura DeCicco fixing merge conflicts NA 101 | #> 2 Amanda Dobbyn flight 4 102 | #> 3 Jim Hester NA 103 | #> 4 Christine Stawitz invisibility 5 104 | #> 5 Isabella Velasquez teleportation NA 105 | ``` 106 | 107 |
108 | 109 | Let's try with a real-world Twitter example (see package data to use this data). 110 | 111 | ``` r 112 | roomba(twitter_data, c("created_at", "name")) 113 | #> # A tibble: 24 x 2 114 | #> name created_at 115 | #> 116 | #> 1 Code for America Mon Aug 10 18:59:29 +0000 2009 117 | #> 2 Ben Lorica 罗瑞卡 Mon Dec 22 22:06:18 +0000 2008 118 | #> 3 Dan Sholler Thu Apr 03 20:09:24 +0000 2014 119 | #> 4 Code for America Mon Aug 10 18:59:29 +0000 2009 120 | #> 5 FiveThirtyEight Tue Jan 21 21:39:32 +0000 2014 121 | #> 6 Digital Impact Wed Oct 07 21:10:53 +0000 2009 122 | #> 7 Drew Williams Thu Aug 07 18:41:29 +0000 2014 123 | #> 8 joe Fri May 29 13:25:25 +0000 2009 124 | #> 9 Data Analysts 4 Good Wed May 07 16:55:33 +0000 2014 125 | #> 10 Ryan Frederick Sun Mar 01 19:06:53 +0000 2009 126 | #> # ... with 14 more rows 127 | ``` 128 | 129 | Shiny app included! 130 | =================== 131 | 132 |

133 | roomba_gif 134 |

135 | Run the app like this: 136 | 137 | ``` r 138 | shiny_roomba() 139 | ``` 140 | 141 |
142 | 143 | What did that original data look like??? 144 | ======================================== 145 | 146 | Feast your eyes on the original `super_data` list! 147 | 148 | ``` r 149 | super_data 150 | #> $stuff 151 | #> $stuff$buried 152 | #> $stuff$buried$deep 153 | #> $stuff$buried$deep[[1]] 154 | #> $stuff$buried$deep[[1]]$location 155 | #> [1] "here" 156 | #> 157 | #> $stuff$buried$deep[[1]]$name 158 | #> [1] "Laura DeCicco" 159 | #> 160 | #> $stuff$buried$deep[[1]]$super_power 161 | #> [1] "fixing merge conflicts" 162 | #> 163 | #> $stuff$buried$deep[[1]]$other_secret_power 164 | #> list() 165 | #> 166 | #> 167 | #> $stuff$buried$deep[[2]] 168 | #> $stuff$buried$deep[[2]]$location 169 | #> [1] "here" 170 | #> 171 | #> $stuff$buried$deep[[2]]$name 172 | #> [1] "Amanda Dobbyn" 173 | #> 174 | #> $stuff$buried$deep[[2]]$super_power 175 | #> [1] "flight" 176 | #> 177 | #> $stuff$buried$deep[[2]]$more_nested_stuff 178 | #> [1] 4 179 | #> 180 | #> 181 | #> 182 | #> $stuff$buried$alsodeep 183 | #> [1] 2342423234 184 | #> 185 | #> $stuff$buried$stilldeep 186 | #> $stuff$buried$stilldeep$even_deeper 187 | #> $stuff$buried$stilldeep$even_deeper[[1]] 188 | #> $stuff$buried$stilldeep$even_deeper[[1]]$location 189 | #> [1] "not here" 190 | #> 191 | #> $stuff$buried$stilldeep$even_deeper[[1]]$name 192 | #> [1] "Jim Hester" 193 | #> 194 | #> $stuff$buried$stilldeep$even_deeper[[1]]$super_power 195 | #> list() 196 | #> 197 | #> 198 | #> $stuff$buried$stilldeep$even_deeper[[2]] 199 | #> $stuff$buried$stilldeep$even_deeper[[2]]$location 200 | #> [1] "here" 201 | #> 202 | #> $stuff$buried$stilldeep$even_deeper[[2]]$name 203 | #> [1] "Christine Stawitz" 204 | #> 205 | #> $stuff$buried$stilldeep$even_deeper[[2]]$super_power 206 | #> [1] "invisibility" 207 | #> 208 | #> $stuff$buried$stilldeep$even_deeper[[2]]$more_nested_stuff 209 | #> [1] 5 210 | #> 211 | #> 212 | #> $stuff$buried$stilldeep$even_deeper[[3]] 213 | #> $stuff$buried$stilldeep$even_deeper[[3]]$location 214 | #> [1] "here" 215 | #> 216 | #> $stuff$buried$stilldeep$even_deeper[[3]]$name 217 | #> [1] "Isabella Velasquez" 218 | #> 219 | #> $stuff$buried$stilldeep$even_deeper[[3]]$super_power 220 | #> [1] "teleportation" 221 | ``` 222 | 223 | And just the *first* element of the `twitter` dataset 😱 224 | 225 | ``` r 226 | twitter_data[[1]] 227 | #> $created_at 228 | #> [1] "Mon May 21 17:58:09 +0000 2018" 229 | #> 230 | #> $id 231 | #> [1] 9.98624e+17 232 | #> 233 | #> $id_str 234 | #> [1] "998623997397876743" 235 | #> 236 | #> $text 237 | #> [1] "Could a program like food stamps have a Cambridge Analytica moment? How do we allow for the innovation that data pl… https://t.co/7tVf1qmNmq" 238 | #> 239 | #> $truncated 240 | #> [1] TRUE 241 | #> 242 | #> $entities 243 | #> $entities$hashtags 244 | #> list() 245 | #> 246 | #> $entities$symbols 247 | #> list() 248 | #> 249 | #> $entities$user_mentions 250 | #> list() 251 | #> 252 | #> $entities$urls 253 | #> $entities$urls[[1]] 254 | #> $entities$urls[[1]]$url 255 | #> [1] "https://t.co/7tVf1qmNmq" 256 | #> 257 | #> $entities$urls[[1]]$expanded_url 258 | #> [1] "https://twitter.com/i/web/status/998623997397876743" 259 | #> 260 | #> $entities$urls[[1]]$display_url 261 | #> [1] "twitter.com/i/web/status/9…" 262 | #> 263 | #> $entities$urls[[1]]$indices 264 | #> $entities$urls[[1]]$indices[[1]] 265 | #> [1] 117 266 | #> 267 | #> $entities$urls[[1]]$indices[[2]] 268 | #> [1] 140 269 | #> 270 | #> 271 | #> 272 | #> 273 | #> 274 | #> $source 275 | #> [1] "TweetDeck" 276 | #> 277 | #> $in_reply_to_status_id 278 | #> NULL 279 | #> 280 | #> $in_reply_to_status_id_str 281 | #> NULL 282 | #> 283 | #> $in_reply_to_user_id 284 | #> NULL 285 | #> 286 | #> $in_reply_to_user_id_str 287 | #> NULL 288 | #> 289 | #> $in_reply_to_screen_name 290 | #> NULL 291 | #> 292 | #> $user 293 | #> $user$id 294 | #> [1] 64482503 295 | #> 296 | #> $user$id_str 297 | #> [1] "64482503" 298 | #> 299 | #> $user$name 300 | #> [1] "Code for America" 301 | #> 302 | #> $user$screen_name 303 | #> [1] "codeforamerica" 304 | #> 305 | #> $user$location 306 | #> [1] "San Francisco, California" 307 | #> 308 | #> $user$description 309 | #> [1] "Government can work for the people, by the people, in the 21st century. Help us make it so." 310 | #> 311 | #> $user$url 312 | #> [1] "https://t.co/l9lokka0rJ" 313 | #> 314 | #> $user$entities 315 | #> $user$entities$url 316 | #> $user$entities$url$urls 317 | #> $user$entities$url$urls[[1]] 318 | #> $user$entities$url$urls[[1]]$url 319 | #> [1] "https://t.co/l9lokka0rJ" 320 | #> 321 | #> $user$entities$url$urls[[1]]$expanded_url 322 | #> [1] "http://codeforamerica.org" 323 | #> 324 | #> $user$entities$url$urls[[1]]$display_url 325 | #> [1] "codeforamerica.org" 326 | #> 327 | #> $user$entities$url$urls[[1]]$indices 328 | #> $user$entities$url$urls[[1]]$indices[[1]] 329 | #> [1] 0 330 | #> 331 | #> $user$entities$url$urls[[1]]$indices[[2]] 332 | #> [1] 23 333 | #> 334 | #> 335 | #> 336 | #> 337 | #> 338 | #> $user$entities$description 339 | #> $user$entities$description$urls 340 | #> list() 341 | #> 342 | #> 343 | #> 344 | #> $user$protected 345 | #> [1] FALSE 346 | #> 347 | #> $user$followers_count 348 | #> [1] 49202 349 | #> 350 | #> $user$friends_count 351 | #> [1] 1716 352 | #> 353 | #> $user$listed_count 354 | #> [1] 2659 355 | #> 356 | #> $user$created_at 357 | #> [1] "Mon Aug 10 18:59:29 +0000 2009" 358 | #> 359 | #> $user$favourites_count 360 | #> [1] 4490 361 | #> 362 | #> $user$utc_offset 363 | #> [1] -25200 364 | #> 365 | #> $user$time_zone 366 | #> [1] "Pacific Time (US & Canada)" 367 | #> 368 | #> $user$geo_enabled 369 | #> [1] TRUE 370 | #> 371 | #> $user$verified 372 | #> [1] TRUE 373 | #> 374 | #> $user$statuses_count 375 | #> [1] 15912 376 | #> 377 | #> $user$lang 378 | #> [1] "en" 379 | #> 380 | #> $user$contributors_enabled 381 | #> [1] FALSE 382 | #> 383 | #> $user$is_translator 384 | #> [1] FALSE 385 | #> 386 | #> $user$is_translation_enabled 387 | #> [1] FALSE 388 | #> 389 | #> $user$profile_background_color 390 | #> [1] "EBEBEB" 391 | #> 392 | #> $user$profile_background_image_url 393 | #> [1] "http://abs.twimg.com/images/themes/theme7/bg.gif" 394 | #> 395 | #> $user$profile_background_image_url_https 396 | #> [1] "https://abs.twimg.com/images/themes/theme7/bg.gif" 397 | #> 398 | #> $user$profile_background_tile 399 | #> [1] FALSE 400 | #> 401 | #> $user$profile_image_url 402 | #> [1] "http://pbs.twimg.com/profile_images/615534833645678592/iAO_Lytr_normal.jpg" 403 | #> 404 | #> $user$profile_image_url_https 405 | #> [1] "https://pbs.twimg.com/profile_images/615534833645678592/iAO_Lytr_normal.jpg" 406 | #> 407 | #> $user$profile_banner_url 408 | #> [1] "https://pbs.twimg.com/profile_banners/64482503/1497895952" 409 | #> 410 | #> $user$profile_link_color 411 | #> [1] "CF1B41" 412 | #> 413 | #> $user$profile_sidebar_border_color 414 | #> [1] "FFFFFF" 415 | #> 416 | #> $user$profile_sidebar_fill_color 417 | #> [1] "F3F3F3" 418 | #> 419 | #> $user$profile_text_color 420 | #> [1] "333333" 421 | #> 422 | #> $user$profile_use_background_image 423 | #> [1] FALSE 424 | #> 425 | #> $user$has_extended_profile 426 | #> [1] FALSE 427 | #> 428 | #> $user$default_profile 429 | #> [1] FALSE 430 | #> 431 | #> $user$default_profile_image 432 | #> [1] FALSE 433 | #> 434 | #> $user$following 435 | #> [1] TRUE 436 | #> 437 | #> $user$follow_request_sent 438 | #> [1] FALSE 439 | #> 440 | #> $user$notifications 441 | #> [1] FALSE 442 | #> 443 | #> $user$translator_type 444 | #> [1] "none" 445 | #> 446 | #> 447 | #> $geo 448 | #> NULL 449 | #> 450 | #> $coordinates 451 | #> NULL 452 | #> 453 | #> $place 454 | #> NULL 455 | #> 456 | #> $contributors 457 | #> NULL 458 | #> 459 | #> $is_quote_status 460 | #> [1] FALSE 461 | #> 462 | #> $retweet_count 463 | #> [1] 0 464 | #> 465 | #> $favorite_count 466 | #> [1] 0 467 | #> 468 | #> $favorited 469 | #> [1] FALSE 470 | #> 471 | #> $retweeted 472 | #> [1] FALSE 473 | #> 474 | #> $possibly_sensitive 475 | #> [1] FALSE 476 | #> 477 | #> $possibly_sensitive_appealable 478 | #> [1] FALSE 479 | #> 480 | #> $lang 481 | #> [1] "en" 482 | ``` 483 | 484 |
485 | 486 | **Happy cleaning!** 487 | 488 |

489 | roomba_gif 490 |

491 | -------------------------------------------------------------------------------- /_pkgdown.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/_pkgdown.yml -------------------------------------------------------------------------------- /data/reddit.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/data/reddit.rda -------------------------------------------------------------------------------- /data/simple.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/data/simple.rda -------------------------------------------------------------------------------- /data/twitter_data.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/data/twitter_data.rda -------------------------------------------------------------------------------- /data/vimeo.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/data/vimeo.rda -------------------------------------------------------------------------------- /docs/LICENSE-text.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | License • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 55 | 56 | 57 | 58 | 59 | 60 | 61 |
62 |
63 | 117 | 118 | 119 | 120 |
121 | 122 |
123 |
124 | 127 | 128 |
YEAR: 2018
129 | COPYRIGHT HOLDER: Jim Hester
130 | 
131 | 132 |
133 | 134 | 139 | 140 |
141 | 142 | 143 | 144 |
145 | 148 | 149 |
150 |

Site built with pkgdown 1.6.1.

151 |
152 | 153 |
154 |
155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | -------------------------------------------------------------------------------- /docs/LICENSE.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | MIT License • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 55 | 56 | 57 | 58 | 59 | 60 | 61 |
62 |
63 | 117 | 118 | 119 | 120 |
121 | 122 |
123 |
124 | 127 | 128 |
129 | 130 |

Copyright (c) 2018 Jim Hester

131 |

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

132 |

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

133 |

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

134 |
135 | 136 |
137 | 138 | 143 | 144 |
145 | 146 | 147 | 148 |
149 | 152 | 153 |
154 |

Site built with pkgdown 1.6.1.

155 |
156 | 157 |
158 |
159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | -------------------------------------------------------------------------------- /docs/articles/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Articles • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 55 | 56 | 57 | 58 | 59 | 60 | 61 |
62 |
63 | 117 | 118 | 119 | 120 |
121 | 122 |
123 |
124 | 127 | 128 |
129 |

All vignettes

130 |

131 | 132 |
133 |
A package for tidying nested lists
134 |
135 |
Twitter API Data
136 |
137 |
138 |
139 |
140 |
141 | 142 | 143 |
144 | 147 | 148 |
149 |

Site built with pkgdown 1.6.1.

150 |
151 | 152 |
153 |
154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | -------------------------------------------------------------------------------- /docs/articles/twitter.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Twitter API Data • roomba 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 20 | 21 | 22 |
23 |
78 | 79 | 80 | 81 | 82 |
83 |
84 | 92 | 93 | 94 | 95 |

This vignette demonstrates the functionality of the roomba package on Twitter API data.

96 |
97 |

98 | Downloading Data

99 |

For more information on downloading Twitter data, please check out the httr package.

100 |

The output req will be a nested list; you can save it using write_rda() from the tidyverse package.

101 |
102 | library(httr)
103 | library(jsonlite)
104 | 
105 | oauth_endpoints("twitter")
106 | 
107 | # edit the keys with own information
108 | myapp <- oauth_app("twitter",
109 |                    key = "EOy06ORJM56b8mk1yoUo6bnjG",
110 |                    secret = "8z4PMPIJrXKYE9JrALjI4TnzDJksB8xRphHj0L5JpWpSiEtbs6"
111 | )
112 | 
113 | twitter_token <- oauth1.0_token(oauth_endpoints("twitter"), myapp)
114 | 
115 | req <- GET("https://api.twitter.com/1.1/statuses/home_timeline.json",
116 |            config(token = twitter_token))
117 | 
118 | stop_for_status(req)
119 | 
120 | content(req)
121 |
122 |
123 |

124 | Example

125 |

We provide actual Twitter data as an example, which can be loaded using data(twitter_data).

126 | 128 |
## Warning: package 'ggplot2' was built under R version 4.0.5
129 | 131 |
## Warning: package 'magrittr' was built under R version 4.0.3
132 | 134 |

Using the roomba() function will gather information based on your variables of interest (in this case, followers_count and friends_count). From there, you can use other dplyr functions on your data.

135 |
136 | twitter_data <- twitter_data
137 | 
138 | twitter_data %>% roomba(cols = c("followers_count", "friends_count")) %>% 
139 |   ggplot(aes(x = followers_count, y = friends_count)) +
140 |   geom_point() + 
141 |   theme_minimal()
142 |

143 |
144 |
145 | 146 | 151 | 152 |
153 | 154 | 155 | 156 |
159 | 160 |
161 |

Site built with pkgdown 1.6.1.

162 |
163 | 164 |
165 |
166 | 167 | 168 | 169 | 170 | 171 | 172 | -------------------------------------------------------------------------------- /docs/articles/twitter_files/figure-html/unnamed-chunk-3-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/docs/articles/twitter_files/figure-html/unnamed-chunk-3-1.png -------------------------------------------------------------------------------- /docs/authors.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Authors • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 55 | 56 | 57 | 58 | 59 | 60 | 61 |
62 |
63 | 117 | 118 | 119 | 120 |
121 | 122 |
123 |
124 | 127 | 128 |
    129 |
  • 130 |

    Amanda Dobbyn. Author. 131 |

    132 |
  • 133 |
  • 134 |

    Christine Stawitz. Author. 135 |

    136 |
  • 137 |
  • 138 |

    Isabella Velasquez. Author. 139 |

    140 |
  • 141 |
  • 142 |

    Jim Hester. Author, maintainer. 143 |

    144 |
  • 145 |
  • 146 |

    Laura DeCicco. Author. 147 |

    148 |
  • 149 |
150 | 151 |
152 | 153 |
154 | 155 | 156 | 157 |
158 | 161 | 162 |
163 |

Site built with pkgdown 1.6.1.

164 |
165 | 166 |
167 |
168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | -------------------------------------------------------------------------------- /docs/docsearch.css: -------------------------------------------------------------------------------- 1 | /* Docsearch -------------------------------------------------------------- */ 2 | /* 3 | Source: https://github.com/algolia/docsearch/ 4 | License: MIT 5 | */ 6 | 7 | .algolia-autocomplete { 8 | display: block; 9 | -webkit-box-flex: 1; 10 | -ms-flex: 1; 11 | flex: 1 12 | } 13 | 14 | .algolia-autocomplete .ds-dropdown-menu { 15 | width: 100%; 16 | min-width: none; 17 | max-width: none; 18 | padding: .75rem 0; 19 | background-color: #fff; 20 | background-clip: padding-box; 21 | border: 1px solid rgba(0, 0, 0, .1); 22 | box-shadow: 0 .5rem 1rem rgba(0, 0, 0, .175); 23 | } 24 | 25 | @media (min-width:768px) { 26 | .algolia-autocomplete .ds-dropdown-menu { 27 | width: 175% 28 | } 29 | } 30 | 31 | .algolia-autocomplete .ds-dropdown-menu::before { 32 | display: none 33 | } 34 | 35 | .algolia-autocomplete .ds-dropdown-menu [class^=ds-dataset-] { 36 | padding: 0; 37 | background-color: rgb(255,255,255); 38 | border: 0; 39 | max-height: 80vh; 40 | } 41 | 42 | .algolia-autocomplete .ds-dropdown-menu .ds-suggestions { 43 | margin-top: 0 44 | } 45 | 46 | .algolia-autocomplete .algolia-docsearch-suggestion { 47 | padding: 0; 48 | overflow: visible 49 | } 50 | 51 | .algolia-autocomplete .algolia-docsearch-suggestion--category-header { 52 | padding: .125rem 1rem; 53 | margin-top: 0; 54 | font-size: 1.3em; 55 | font-weight: 500; 56 | color: #00008B; 57 | border-bottom: 0 58 | } 59 | 60 | .algolia-autocomplete .algolia-docsearch-suggestion--wrapper { 61 | float: none; 62 | padding-top: 0 63 | } 64 | 65 | .algolia-autocomplete .algolia-docsearch-suggestion--subcategory-column { 66 | float: none; 67 | width: auto; 68 | padding: 0; 69 | text-align: left 70 | } 71 | 72 | .algolia-autocomplete .algolia-docsearch-suggestion--content { 73 | float: none; 74 | width: auto; 75 | padding: 0 76 | } 77 | 78 | .algolia-autocomplete .algolia-docsearch-suggestion--content::before { 79 | display: none 80 | } 81 | 82 | .algolia-autocomplete .ds-suggestion:not(:first-child) .algolia-docsearch-suggestion--category-header { 83 | padding-top: .75rem; 84 | margin-top: .75rem; 85 | border-top: 1px solid rgba(0, 0, 0, .1) 86 | } 87 | 88 | .algolia-autocomplete .ds-suggestion .algolia-docsearch-suggestion--subcategory-column { 89 | display: block; 90 | padding: .1rem 1rem; 91 | margin-bottom: 0.1; 92 | font-size: 1.0em; 93 | font-weight: 400 94 | /* display: none */ 95 | } 96 | 97 | .algolia-autocomplete .algolia-docsearch-suggestion--title { 98 | display: block; 99 | padding: .25rem 1rem; 100 | margin-bottom: 0; 101 | font-size: 0.9em; 102 | font-weight: 400 103 | } 104 | 105 | .algolia-autocomplete .algolia-docsearch-suggestion--text { 106 | padding: 0 1rem .5rem; 107 | margin-top: -.25rem; 108 | font-size: 0.8em; 109 | font-weight: 400; 110 | line-height: 1.25 111 | } 112 | 113 | .algolia-autocomplete .algolia-docsearch-footer { 114 | width: 110px; 115 | height: 20px; 116 | z-index: 3; 117 | margin-top: 10.66667px; 118 | float: right; 119 | font-size: 0; 120 | line-height: 0; 121 | } 122 | 123 | .algolia-autocomplete .algolia-docsearch-footer--logo { 124 | background-image: url("data:image/svg+xml;utf8,"); 125 | background-repeat: no-repeat; 126 | background-position: 50%; 127 | background-size: 100%; 128 | overflow: hidden; 129 | text-indent: -9000px; 130 | width: 100%; 131 | height: 100%; 132 | display: block; 133 | transform: translate(-8px); 134 | } 135 | 136 | .algolia-autocomplete .algolia-docsearch-suggestion--highlight { 137 | color: #FF8C00; 138 | background: rgba(232, 189, 54, 0.1) 139 | } 140 | 141 | 142 | .algolia-autocomplete .algolia-docsearch-suggestion--text .algolia-docsearch-suggestion--highlight { 143 | box-shadow: inset 0 -2px 0 0 rgba(105, 105, 105, .5) 144 | } 145 | 146 | .algolia-autocomplete .ds-suggestion.ds-cursor .algolia-docsearch-suggestion--content { 147 | background-color: rgba(192, 192, 192, .15) 148 | } 149 | -------------------------------------------------------------------------------- /docs/images/shinydemo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/docs/images/shinydemo.gif -------------------------------------------------------------------------------- /docs/img/sticker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/docs/img/sticker.png -------------------------------------------------------------------------------- /docs/jquery.sticky-kit.min.js: -------------------------------------------------------------------------------- 1 | /* Sticky-kit v1.1.2 | WTFPL | Leaf Corcoran 2015 | */ 2 | /* 3 | Source: https://github.com/leafo/sticky-kit 4 | License: MIT 5 | */ 6 | (function(){var b,f;b=this.jQuery||window.jQuery;f=b(window);b.fn.stick_in_parent=function(d){var A,w,J,n,B,K,p,q,k,E,t;null==d&&(d={});t=d.sticky_class;B=d.inner_scrolling;E=d.recalc_every;k=d.parent;q=d.offset_top;p=d.spacer;w=d.bottoming;null==q&&(q=0);null==k&&(k=void 0);null==B&&(B=!0);null==t&&(t="is_stuck");A=b(document);null==w&&(w=!0);J=function(a,d,n,C,F,u,r,G){var v,H,m,D,I,c,g,x,y,z,h,l;if(!a.data("sticky_kit")){a.data("sticky_kit",!0);I=A.height();g=a.parent();null!=k&&(g=g.closest(k)); 7 | if(!g.length)throw"failed to find stick parent";v=m=!1;(h=null!=p?p&&a.closest(p):b("
"))&&h.css("position",a.css("position"));x=function(){var c,f,e;if(!G&&(I=A.height(),c=parseInt(g.css("border-top-width"),10),f=parseInt(g.css("padding-top"),10),d=parseInt(g.css("padding-bottom"),10),n=g.offset().top+c+f,C=g.height(),m&&(v=m=!1,null==p&&(a.insertAfter(h),h.detach()),a.css({position:"",top:"",width:"",bottom:""}).removeClass(t),e=!0),F=a.offset().top-(parseInt(a.css("margin-top"),10)||0)-q, 8 | u=a.outerHeight(!0),r=a.css("float"),h&&h.css({width:a.outerWidth(!0),height:u,display:a.css("display"),"vertical-align":a.css("vertical-align"),"float":r}),e))return l()};x();if(u!==C)return D=void 0,c=q,z=E,l=function(){var b,l,e,k;if(!G&&(e=!1,null!=z&&(--z,0>=z&&(z=E,x(),e=!0)),e||A.height()===I||x(),e=f.scrollTop(),null!=D&&(l=e-D),D=e,m?(w&&(k=e+u+c>C+n,v&&!k&&(v=!1,a.css({position:"fixed",bottom:"",top:c}).trigger("sticky_kit:unbottom"))),eb&&!v&&(c-=l,c=Math.max(b-u,c),c=Math.min(q,c),m&&a.css({top:c+"px"})))):e>F&&(m=!0,b={position:"fixed",top:c},b.width="border-box"===a.css("box-sizing")?a.outerWidth()+"px":a.width()+"px",a.css(b).addClass(t),null==p&&(a.after(h),"left"!==r&&"right"!==r||h.append(a)),a.trigger("sticky_kit:stick")),m&&w&&(null==k&&(k=e+u+c>C+n),!v&&k)))return v=!0,"static"===g.css("position")&&g.css({position:"relative"}), 10 | a.css({position:"absolute",bottom:d,top:"auto"}).trigger("sticky_kit:bottom")},y=function(){x();return l()},H=function(){G=!0;f.off("touchmove",l);f.off("scroll",l);f.off("resize",y);b(document.body).off("sticky_kit:recalc",y);a.off("sticky_kit:detach",H);a.removeData("sticky_kit");a.css({position:"",bottom:"",top:"",width:""});g.position("position","");if(m)return null==p&&("left"!==r&&"right"!==r||a.insertAfter(h),h.remove()),a.removeClass(t)},f.on("touchmove",l),f.on("scroll",l),f.on("resize", 11 | y),b(document.body).on("sticky_kit:recalc",y),a.on("sticky_kit:detach",H),setTimeout(l,0)}};n=0;for(K=this.length;n 2 | 3 | 5 | 8 | 12 | 13 | -------------------------------------------------------------------------------- /docs/pkgdown.css: -------------------------------------------------------------------------------- 1 | /* Sticky footer */ 2 | 3 | /** 4 | * Basic idea: https://philipwalton.github.io/solved-by-flexbox/demos/sticky-footer/ 5 | * Details: https://github.com/philipwalton/solved-by-flexbox/blob/master/assets/css/components/site.css 6 | * 7 | * .Site -> body > .container 8 | * .Site-content -> body > .container .row 9 | * .footer -> footer 10 | * 11 | * Key idea seems to be to ensure that .container and __all its parents__ 12 | * have height set to 100% 13 | * 14 | */ 15 | 16 | html, body { 17 | height: 100%; 18 | } 19 | 20 | body { 21 | position: relative; 22 | } 23 | 24 | body > .container { 25 | display: flex; 26 | height: 100%; 27 | flex-direction: column; 28 | } 29 | 30 | body > .container .row { 31 | flex: 1 0 auto; 32 | } 33 | 34 | footer { 35 | margin-top: 45px; 36 | padding: 35px 0 36px; 37 | border-top: 1px solid #e5e5e5; 38 | color: #666; 39 | display: flex; 40 | flex-shrink: 0; 41 | } 42 | footer p { 43 | margin-bottom: 0; 44 | } 45 | footer div { 46 | flex: 1; 47 | } 48 | footer .pkgdown { 49 | text-align: right; 50 | } 51 | footer p { 52 | margin-bottom: 0; 53 | } 54 | 55 | img.icon { 56 | float: right; 57 | } 58 | 59 | img { 60 | max-width: 100%; 61 | } 62 | 63 | /* Fix bug in bootstrap (only seen in firefox) */ 64 | summary { 65 | display: list-item; 66 | } 67 | 68 | /* Typographic tweaking ---------------------------------*/ 69 | 70 | .contents .page-header { 71 | margin-top: calc(-60px + 1em); 72 | } 73 | 74 | dd { 75 | margin-left: 3em; 76 | } 77 | 78 | /* Section anchors ---------------------------------*/ 79 | 80 | a.anchor { 81 | margin-left: -30px; 82 | display:inline-block; 83 | width: 30px; 84 | height: 30px; 85 | visibility: hidden; 86 | 87 | background-image: url(./link.svg); 88 | background-repeat: no-repeat; 89 | background-size: 20px 20px; 90 | background-position: center center; 91 | } 92 | 93 | .hasAnchor:hover a.anchor { 94 | visibility: visible; 95 | } 96 | 97 | @media (max-width: 767px) { 98 | .hasAnchor:hover a.anchor { 99 | visibility: hidden; 100 | } 101 | } 102 | 103 | 104 | /* Fixes for fixed navbar --------------------------*/ 105 | 106 | .contents h1, .contents h2, .contents h3, .contents h4 { 107 | padding-top: 60px; 108 | margin-top: -40px; 109 | } 110 | 111 | /* Navbar submenu --------------------------*/ 112 | 113 | .dropdown-submenu { 114 | position: relative; 115 | } 116 | 117 | .dropdown-submenu>.dropdown-menu { 118 | top: 0; 119 | left: 100%; 120 | margin-top: -6px; 121 | margin-left: -1px; 122 | border-radius: 0 6px 6px 6px; 123 | } 124 | 125 | .dropdown-submenu:hover>.dropdown-menu { 126 | display: block; 127 | } 128 | 129 | .dropdown-submenu>a:after { 130 | display: block; 131 | content: " "; 132 | float: right; 133 | width: 0; 134 | height: 0; 135 | border-color: transparent; 136 | border-style: solid; 137 | border-width: 5px 0 5px 5px; 138 | border-left-color: #cccccc; 139 | margin-top: 5px; 140 | margin-right: -10px; 141 | } 142 | 143 | .dropdown-submenu:hover>a:after { 144 | border-left-color: #ffffff; 145 | } 146 | 147 | .dropdown-submenu.pull-left { 148 | float: none; 149 | } 150 | 151 | .dropdown-submenu.pull-left>.dropdown-menu { 152 | left: -100%; 153 | margin-left: 10px; 154 | border-radius: 6px 0 6px 6px; 155 | } 156 | 157 | /* Sidebar --------------------------*/ 158 | 159 | #pkgdown-sidebar { 160 | margin-top: 30px; 161 | position: -webkit-sticky; 162 | position: sticky; 163 | top: 70px; 164 | } 165 | 166 | #pkgdown-sidebar h2 { 167 | font-size: 1.5em; 168 | margin-top: 1em; 169 | } 170 | 171 | #pkgdown-sidebar h2:first-child { 172 | margin-top: 0; 173 | } 174 | 175 | #pkgdown-sidebar .list-unstyled li { 176 | margin-bottom: 0.5em; 177 | } 178 | 179 | /* bootstrap-toc tweaks ------------------------------------------------------*/ 180 | 181 | /* All levels of nav */ 182 | 183 | nav[data-toggle='toc'] .nav > li > a { 184 | padding: 4px 20px 4px 6px; 185 | font-size: 1.5rem; 186 | font-weight: 400; 187 | color: inherit; 188 | } 189 | 190 | nav[data-toggle='toc'] .nav > li > a:hover, 191 | nav[data-toggle='toc'] .nav > li > a:focus { 192 | padding-left: 5px; 193 | color: inherit; 194 | border-left: 1px solid #878787; 195 | } 196 | 197 | nav[data-toggle='toc'] .nav > .active > a, 198 | nav[data-toggle='toc'] .nav > .active:hover > a, 199 | nav[data-toggle='toc'] .nav > .active:focus > a { 200 | padding-left: 5px; 201 | font-size: 1.5rem; 202 | font-weight: 400; 203 | color: inherit; 204 | border-left: 2px solid #878787; 205 | } 206 | 207 | /* Nav: second level (shown on .active) */ 208 | 209 | nav[data-toggle='toc'] .nav .nav { 210 | display: none; /* Hide by default, but at >768px, show it */ 211 | padding-bottom: 10px; 212 | } 213 | 214 | nav[data-toggle='toc'] .nav .nav > li > a { 215 | padding-left: 16px; 216 | font-size: 1.35rem; 217 | } 218 | 219 | nav[data-toggle='toc'] .nav .nav > li > a:hover, 220 | nav[data-toggle='toc'] .nav .nav > li > a:focus { 221 | padding-left: 15px; 222 | } 223 | 224 | nav[data-toggle='toc'] .nav .nav > .active > a, 225 | nav[data-toggle='toc'] .nav .nav > .active:hover > a, 226 | nav[data-toggle='toc'] .nav .nav > .active:focus > a { 227 | padding-left: 15px; 228 | font-weight: 500; 229 | font-size: 1.35rem; 230 | } 231 | 232 | /* orcid ------------------------------------------------------------------- */ 233 | 234 | .orcid { 235 | font-size: 16px; 236 | color: #A6CE39; 237 | /* margins are required by official ORCID trademark and display guidelines */ 238 | margin-left:4px; 239 | margin-right:4px; 240 | vertical-align: middle; 241 | } 242 | 243 | /* Reference index & topics ----------------------------------------------- */ 244 | 245 | .ref-index th {font-weight: normal;} 246 | 247 | .ref-index td {vertical-align: top; min-width: 100px} 248 | .ref-index .icon {width: 40px;} 249 | .ref-index .alias {width: 40%;} 250 | .ref-index-icons .alias {width: calc(40% - 40px);} 251 | .ref-index .title {width: 60%;} 252 | 253 | .ref-arguments th {text-align: right; padding-right: 10px;} 254 | .ref-arguments th, .ref-arguments td {vertical-align: top; min-width: 100px} 255 | .ref-arguments .name {width: 20%;} 256 | .ref-arguments .desc {width: 80%;} 257 | 258 | /* Nice scrolling for wide elements --------------------------------------- */ 259 | 260 | table { 261 | display: block; 262 | overflow: auto; 263 | } 264 | 265 | /* Syntax highlighting ---------------------------------------------------- */ 266 | 267 | pre { 268 | word-wrap: normal; 269 | word-break: normal; 270 | border: 1px solid #eee; 271 | } 272 | 273 | pre, code { 274 | background-color: #f8f8f8; 275 | color: #333; 276 | } 277 | 278 | pre code { 279 | overflow: auto; 280 | word-wrap: normal; 281 | white-space: pre; 282 | } 283 | 284 | pre .img { 285 | margin: 5px 0; 286 | } 287 | 288 | pre .img img { 289 | background-color: #fff; 290 | display: block; 291 | height: auto; 292 | } 293 | 294 | code a, pre a { 295 | color: #375f84; 296 | } 297 | 298 | a.sourceLine:hover { 299 | text-decoration: none; 300 | } 301 | 302 | .fl {color: #1514b5;} 303 | .fu {color: #000000;} /* function */ 304 | .ch,.st {color: #036a07;} /* string */ 305 | .kw {color: #264D66;} /* keyword */ 306 | .co {color: #888888;} /* comment */ 307 | 308 | .message { color: black; font-weight: bolder;} 309 | .error { color: orange; font-weight: bolder;} 310 | .warning { color: #6A0366; font-weight: bolder;} 311 | 312 | /* Clipboard --------------------------*/ 313 | 314 | .hasCopyButton { 315 | position: relative; 316 | } 317 | 318 | .btn-copy-ex { 319 | position: absolute; 320 | right: 0; 321 | top: 0; 322 | visibility: hidden; 323 | } 324 | 325 | .hasCopyButton:hover button.btn-copy-ex { 326 | visibility: visible; 327 | } 328 | 329 | /* headroom.js ------------------------ */ 330 | 331 | .headroom { 332 | will-change: transform; 333 | transition: transform 200ms linear; 334 | } 335 | .headroom--pinned { 336 | transform: translateY(0%); 337 | } 338 | .headroom--unpinned { 339 | transform: translateY(-100%); 340 | } 341 | 342 | /* mark.js ----------------------------*/ 343 | 344 | mark { 345 | background-color: rgba(255, 255, 51, 0.5); 346 | border-bottom: 2px solid rgba(255, 153, 51, 0.3); 347 | padding: 1px; 348 | } 349 | 350 | /* vertical spacing after htmlwidgets */ 351 | .html-widget { 352 | margin-bottom: 10px; 353 | } 354 | 355 | /* fontawesome ------------------------ */ 356 | 357 | .fab { 358 | font-family: "Font Awesome 5 Brands" !important; 359 | } 360 | 361 | /* don't display links in code chunks when printing */ 362 | /* source: https://stackoverflow.com/a/10781533 */ 363 | @media print { 364 | code a:link:after, code a:visited:after { 365 | content: ""; 366 | } 367 | } 368 | -------------------------------------------------------------------------------- /docs/pkgdown.js: -------------------------------------------------------------------------------- 1 | /* http://gregfranko.com/blog/jquery-best-practices/ */ 2 | (function($) { 3 | $(function() { 4 | 5 | $('.navbar-fixed-top').headroom(); 6 | 7 | $('body').css('padding-top', $('.navbar').height() + 10); 8 | $(window).resize(function(){ 9 | $('body').css('padding-top', $('.navbar').height() + 10); 10 | }); 11 | 12 | $('[data-toggle="tooltip"]').tooltip(); 13 | 14 | var cur_path = paths(location.pathname); 15 | var links = $("#navbar ul li a"); 16 | var max_length = -1; 17 | var pos = -1; 18 | for (var i = 0; i < links.length; i++) { 19 | if (links[i].getAttribute("href") === "#") 20 | continue; 21 | // Ignore external links 22 | if (links[i].host !== location.host) 23 | continue; 24 | 25 | var nav_path = paths(links[i].pathname); 26 | 27 | var length = prefix_length(nav_path, cur_path); 28 | if (length > max_length) { 29 | max_length = length; 30 | pos = i; 31 | } 32 | } 33 | 34 | // Add class to parent
  • , and enclosing
  • if in dropdown 35 | if (pos >= 0) { 36 | var menu_anchor = $(links[pos]); 37 | menu_anchor.parent().addClass("active"); 38 | menu_anchor.closest("li.dropdown").addClass("active"); 39 | } 40 | }); 41 | 42 | function paths(pathname) { 43 | var pieces = pathname.split("/"); 44 | pieces.shift(); // always starts with / 45 | 46 | var end = pieces[pieces.length - 1]; 47 | if (end === "index.html" || end === "") 48 | pieces.pop(); 49 | return(pieces); 50 | } 51 | 52 | // Returns -1 if not found 53 | function prefix_length(needle, haystack) { 54 | if (needle.length > haystack.length) 55 | return(-1); 56 | 57 | // Special case for length-0 haystack, since for loop won't run 58 | if (haystack.length === 0) { 59 | return(needle.length === 0 ? 0 : -1); 60 | } 61 | 62 | for (var i = 0; i < haystack.length; i++) { 63 | if (needle[i] != haystack[i]) 64 | return(i); 65 | } 66 | 67 | return(haystack.length); 68 | } 69 | 70 | /* Clipboard --------------------------*/ 71 | 72 | function changeTooltipMessage(element, msg) { 73 | var tooltipOriginalTitle=element.getAttribute('data-original-title'); 74 | element.setAttribute('data-original-title', msg); 75 | $(element).tooltip('show'); 76 | element.setAttribute('data-original-title', tooltipOriginalTitle); 77 | } 78 | 79 | if(ClipboardJS.isSupported()) { 80 | $(document).ready(function() { 81 | var copyButton = ""; 82 | 83 | $(".examples, div.sourceCode").addClass("hasCopyButton"); 84 | 85 | // Insert copy buttons: 86 | $(copyButton).prependTo(".hasCopyButton"); 87 | 88 | // Initialize tooltips: 89 | $('.btn-copy-ex').tooltip({container: 'body'}); 90 | 91 | // Initialize clipboard: 92 | var clipboardBtnCopies = new ClipboardJS('[data-clipboard-copy]', { 93 | text: function(trigger) { 94 | return trigger.parentNode.textContent; 95 | } 96 | }); 97 | 98 | clipboardBtnCopies.on('success', function(e) { 99 | changeTooltipMessage(e.trigger, 'Copied!'); 100 | e.clearSelection(); 101 | }); 102 | 103 | clipboardBtnCopies.on('error', function() { 104 | changeTooltipMessage(e.trigger,'Press Ctrl+C or Command+C to copy'); 105 | }); 106 | }); 107 | } 108 | })(window.jQuery || window.$) 109 | -------------------------------------------------------------------------------- /docs/pkgdown.yml: -------------------------------------------------------------------------------- 1 | pandoc: 2.9.1.1 2 | pkgdown: 1.6.1 3 | pkgdown_sha: ~ 4 | articles: 5 | 2018-06-26-roomba: 2018-06-26-roomba.html 6 | twitter: twitter.html 7 | last_built: 2021-07-21T19:08Z 8 | 9 | -------------------------------------------------------------------------------- /docs/reference/dfs_idx.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Perform a recursive depth first search of a function — dfs_idx • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Perform a recursive depth first search of a function

    133 |
    134 | 135 |
    dfs_idx(.x, .f)
    136 | 137 |

    Arguments

    138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 162 | 163 |
    .x

    A list or atomic vector.

    .f

    A function, formula, or atomic vector.

    147 |

    If a function, it is used as is.

    148 |

    If a formula, e.g. ~ .x + 2, it is converted to a function. There 149 | are three ways to refer to the arguments:

      150 |
    • For a single argument function, use .

    • 151 |
    • For a two argument function, use .x and .y

    • 152 |
    • For more arguments, use ..1, ..2, ..3 etc

    • 153 |
    154 | 155 |

    This syntax allows you to create very compact anonymous functions.

    156 |

    If character vector, numeric vector, or list, it 157 | is converted to an extractor function. Character vectors index by name 158 | and numeric vectors index by position; use a list to index by position 159 | and name at different levels. Within a list, wrap strings in get-attr() 160 | to extract named attributes. If a component is not present, the value of 161 | .default will be returned.

    164 | 165 | 166 |
    167 | 172 |
    173 | 174 | 175 |
    176 | 179 | 180 |
    181 |

    Site built with pkgdown 1.6.1.

    182 |
    183 | 184 |
    185 |
    186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | 194 | -------------------------------------------------------------------------------- /docs/reference/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Function reference • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 55 | 56 | 57 | 58 | 59 | 60 | 61 |
    62 |
    63 | 117 | 118 | 119 | 120 |
    121 | 122 |
    123 |
    124 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 153 | 154 | 155 | 156 | 159 | 160 | 161 | 162 | 165 | 166 | 167 | 168 | 171 | 172 | 173 | 174 | 177 | 178 | 179 | 180 | 183 | 184 | 185 | 186 | 189 | 190 | 191 | 192 | 195 | 196 | 197 | 198 | 201 | 202 | 203 | 204 | 207 | 208 | 209 | 210 |
    139 |

    All functions

    140 |

    141 |
    151 |

    dfs_idx()

    152 |

    Perform a recursive depth first search of a function

    157 |

    list_names()

    158 |

    List all names in a list

    163 |

    reddit

    164 |

    Example data from reddit

    169 |

    replace_null()

    170 |

    Replace NULLs

    175 |

    roomba-package

    176 |

    roomba package

    181 |

    roomba()

    182 |

    Roomba

    187 |

    shiny_roomba()

    188 |

    Run the roomba app

    193 |

    simple

    194 |

    Example data with simple nested list

    199 |

    twitter_data

    200 |

    Example Twitter data

    205 |

    vimeo

    206 |

    Example data from vimeo

    211 |
    212 | 213 | 218 |
    219 | 220 | 221 |
    222 | 225 | 226 |
    227 |

    Site built with pkgdown 1.6.1.

    228 |
    229 | 230 |
    231 |
    232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | -------------------------------------------------------------------------------- /docs/reference/list_names.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | List all names in a list — list_names • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    List all names in a list

    133 |
    134 | 135 |
    list_names(x)
    136 | 137 |

    Arguments

    138 | 139 | 140 | 141 | 142 | 143 | 144 |
    x

    list to use

    145 | 146 | 147 |
    148 | 153 |
    154 | 155 | 156 |
    157 | 160 | 161 |
    162 |

    Site built with pkgdown 1.6.1.

    163 |
    164 | 165 |
    166 |
    167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | -------------------------------------------------------------------------------- /docs/reference/pipe.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Pipe operator — %>% • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 44 | 45 | 46 | 47 | 48 | 49 |
    50 |
    51 | 102 | 103 | 104 |
    105 | 106 |
    107 |
    108 | 113 | 114 |
    115 | 116 |

    See magrittr::%>% for details.

    117 | 118 |
    119 | 120 |
    lhs %>% rhs
    121 | 122 | 123 |
    124 | 130 |
    131 | 132 |
    133 | 136 | 137 |
    138 |

    Site built with pkgdown.

    139 |
    140 | 141 |
    142 |
    143 | 144 | 145 | 146 | 147 | 148 | 149 | -------------------------------------------------------------------------------- /docs/reference/reddit.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Example data from reddit — reddit • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Example data from reddit

    133 |
    134 | 135 |
    reddit
    136 | 137 | 138 |

    Format

    139 | 140 |

    An object of class list of length 2.

    141 |

    Source

    142 | 143 |

    https://www.reddit.com/dev/api/

    144 | 145 |
    146 | 151 |
    152 | 153 | 154 |
    155 | 158 | 159 |
    160 |

    Site built with pkgdown 1.6.1.

    161 |
    162 | 163 |
    164 |
    165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | -------------------------------------------------------------------------------- /docs/reference/replace_null.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Replace NULLs — replace_null • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Replace all the empty values in a list

    133 |
    134 | 135 |
    replace_null(x, replacement = NA)
    136 | 137 |

    Arguments

    138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 |
    x

    list to use

    replacement

    Replacement value for missing values

    149 | 150 | 151 |

    Examples

    152 |
    list(a = NULL, b = 1, c = list(foo = NULL, bar = NULL)) %>% replace_null() 153 |
    #> $a 154 | #> [1] NA 155 | #> 156 | #> $b 157 | #> [1] 1 158 | #> 159 | #> $c 160 | #> $c$foo 161 | #> [1] NA 162 | #> 163 | #> $c$bar 164 | #> [1] NA 165 | #> 166 | #>
    167 |
    168 |
    169 | 174 |
    175 | 176 | 177 |
    178 | 181 | 182 |
    183 |

    Site built with pkgdown 1.6.1.

    184 |
    185 | 186 |
    187 |
    188 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 196 | -------------------------------------------------------------------------------- /docs/reference/roomba-package.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | roomba package — roomba-package • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 62 | 63 | 64 | 65 | 66 | 67 | 68 |
    69 |
    70 | 124 | 125 | 126 | 127 |
    128 | 129 |
    130 |
    131 | 136 | 137 |
    138 | 139 | 140 | 141 | 142 |
    Package:roomba
    Type:Package
    143 | 144 | 145 |
    146 | 147 | 148 | 149 |

    Details

    150 | 151 |

    Collection of functions to do deal with deeply nested data

    152 | 153 |
    154 | 159 |
    160 | 161 | 162 |
    163 | 166 | 167 |
    168 |

    Site built with pkgdown 1.6.1.

    169 |
    170 | 171 |
    172 |
    173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | -------------------------------------------------------------------------------- /docs/reference/roomba.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Roomba — roomba • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Tidy your nested list

    133 |
    134 | 135 |
    roomba(inp, cols = NULL, default = NA, keep = all)
    136 | 137 |

    Arguments

    138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 |
    inp

    List to tidy

    cols

    Columns to keep

    default

    Replacement for NULL values. Defaults to NA.

    keep

    Should all or any data be kept?

    157 | 158 | 159 |

    Examples

    160 |
    161 | simple %>% roomba(cols = c("name", "goodstuff"), keep = any) 162 |
    #> # A tibble: 0 x 0
    simple %>% roomba(cols = c("name", "goodstuff"), keep = any) 163 |
    #> # A tibble: 0 x 0
    164 |
    165 | 170 |
    171 | 172 | 173 |
    174 | 177 | 178 |
    179 |

    Site built with pkgdown 1.6.1.

    180 |
    181 | 182 |
    183 |
    184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | -------------------------------------------------------------------------------- /docs/reference/simple.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Example data with simple nested list — simple • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Example data with simple nested list

    133 |
    134 | 135 |
    simple
    136 | 137 | 138 |

    Format

    139 | 140 |

    An object of class character of length 1.

    141 | 142 |
    143 | 148 |
    149 | 150 | 151 |
    152 | 155 | 156 |
    157 |

    Site built with pkgdown 1.6.1.

    158 |
    159 | 160 |
    161 |
    162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | -------------------------------------------------------------------------------- /docs/reference/twitter_data.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Example Twitter data — twitter_data • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Example data from Twitter API

    133 |
    134 | 135 |
    twitter_data
    136 | 137 | 138 |

    Format

    139 | 140 |

    An object of class list of length 20.

    141 |

    Source

    142 | 143 |

    https://developer.twitter.com/en/docs

    144 |

    Details

    145 | 146 |

    twitter_data[[1]][["id"]]

    147 |

    [[1]: R:[1 148 | ["id"]: R:

    149 | 150 |
    151 | 156 |
    157 | 158 | 159 |
    160 | 163 | 164 |
    165 |

    Site built with pkgdown 1.6.1.

    166 |
    167 | 168 |
    169 |
    170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | -------------------------------------------------------------------------------- /docs/reference/vimeo.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Example data from vimeo — vimeo • roomba 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
    63 |
    64 | 118 | 119 | 120 | 121 |
    122 | 123 |
    124 |
    125 | 130 | 131 |
    132 |

    Example data from vimeo

    133 |
    134 | 135 |
    vimeo
    136 | 137 | 138 |

    Format

    139 | 140 |

    An object of class response of length 10.

    141 |

    Source

    142 | 143 |

    https://developer.vimeo.com/

    144 | 145 |
    146 | 151 |
    152 | 153 | 154 |
    155 | 158 | 159 |
    160 |

    Site built with pkgdown 1.6.1.

    161 |
    162 | 163 |
    164 |
    165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | -------------------------------------------------------------------------------- /images/shinydemo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/images/shinydemo.gif -------------------------------------------------------------------------------- /img/sticker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/img/sticker.png -------------------------------------------------------------------------------- /inst/Shiny/server.R: -------------------------------------------------------------------------------- 1 | # Define server logic required to draw a histogram ---- 2 | server <- function(input, output, session) { 3 | require(jsonlite) 4 | require(tools) 5 | library(roomba) 6 | require(ggplot2) 7 | 8 | # Allow data to change as input changes 9 | datasetInput <- reactive({ 10 | #Determine if input file is .json or .rda 11 | if(tools::file_ext(input$data$name)=="json") { 12 | x <- jsonlite::fromJSON(input$data$datapath) 13 | } else if(tools::file_ext(input$data$name)=="rda"){ 14 | load(input$data$datapath) 15 | assign("x", get(ls())) 16 | } 17 | return(x) 18 | }) 19 | 20 | #Print names of first level list items 21 | observe({ 22 | if(!is.null(input$data)){ 23 | dataset <- datasetInput() 24 | column.names <- roomba::list_names(dataset) 25 | updateCheckboxGroupInput(session, "Variables", 26 | choices=sort(column.names)) 27 | } 28 | }) 29 | 30 | output$plot <- eventReactive(input$makePlot, 31 | { 32 | dataset <- datasetInput() 33 | thedata <-roomba::roomba(dataset, input$Variables) 34 | thedata$created_at <- 35 | as.POSIXct(thedata$created_at, format="%a %b %d %H:%M:%S %z %Y") 36 | 37 | output$plot <- renderPlot( 38 | ggplot(thedata) + 39 | geom_point(aes(x=name, y=created_at)) + 40 | theme(axis.text.x = element_text(angle = 90, hjust = 1), 41 | text = element_text(size=16)) 42 | )}) 43 | } 44 | -------------------------------------------------------------------------------- /inst/Shiny/ui.R: -------------------------------------------------------------------------------- 1 | require(shiny) 2 | require(roomba) 3 | require(purrr) 4 | ui <- fluidPage( 5 | 6 | titlePanel("roomba"), 7 | sidebarLayout( 8 | sidebarPanel( 9 | p("An R package that allows for easy parsing of nested lists."), 10 | # Input: Slider to select JSON file ---- 11 | fileInput(inputId = "data", 12 | label = "Select a data file to import:", 13 | accept = c('.json', '.rda'), multiple=FALSE), 14 | 15 | # Output: Names of first level list objects---- 16 | actionButton("makePlot", "Plot"), 17 | checkboxGroupInput("Variables", "Select variables to plot:") 18 | 19 | ), 20 | mainPanel(plotOutput("plot")) 21 | ) 22 | ) 23 | -------------------------------------------------------------------------------- /inst/figures/hexSticker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/inst/figures/hexSticker.png -------------------------------------------------------------------------------- /inst/figures/vacuum.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/inst/figures/vacuum.png -------------------------------------------------------------------------------- /man/dfs_idx.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/dfs_idx.R 3 | \name{dfs_idx} 4 | \alias{dfs_idx} 5 | \title{Perform a recursive depth first search of a function} 6 | \usage{ 7 | dfs_idx(.x, .f) 8 | } 9 | \arguments{ 10 | \item{.x}{A list or atomic vector.} 11 | 12 | \item{.f}{A function, formula, or atomic vector. 13 | 14 | If a \strong{function}, it is used as is. 15 | 16 | If a \strong{formula}, e.g. \code{~ .x + 2}, it is converted to a function. There 17 | are three ways to refer to the arguments: 18 | \itemize{ 19 | \item For a single argument function, use \code{.} 20 | \item For a two argument function, use \code{.x} and \code{.y} 21 | \item For more arguments, use \code{..1}, \code{..2}, \code{..3} etc 22 | } 23 | 24 | This syntax allows you to create very compact anonymous functions. 25 | 26 | If \strong{character vector}, \strong{numeric vector}, or \strong{list}, it 27 | is converted to an extractor function. Character vectors index by name 28 | and numeric vectors index by position; use a list to index by position 29 | and name at different levels. Within a list, wrap strings in \code{\link[=get-attr]{get-attr()}} 30 | to extract named attributes. If a component is not present, the value of 31 | \code{.default} will be returned.} 32 | } 33 | \description{ 34 | Perform a recursive depth first search of a function 35 | } 36 | -------------------------------------------------------------------------------- /man/list_names.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/utils.R 3 | \name{list_names} 4 | \alias{list_names} 5 | \title{List all names in a list} 6 | \usage{ 7 | list_names(x) 8 | } 9 | \arguments{ 10 | \item{x}{list to use} 11 | } 12 | \description{ 13 | List all names in a list 14 | } 15 | -------------------------------------------------------------------------------- /man/reddit.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/roomba-package.R 3 | \docType{data} 4 | \name{reddit} 5 | \alias{reddit} 6 | \title{Example data from reddit} 7 | \format{An object of class \code{list} of length 2.} 8 | \source{ 9 | \url{https://www.reddit.com/dev/api/} 10 | } 11 | \usage{ 12 | reddit 13 | } 14 | \description{ 15 | Example data from reddit 16 | } 17 | \keyword{datasets} 18 | -------------------------------------------------------------------------------- /man/replace_null.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/replace_null.R 3 | \name{replace_null} 4 | \alias{replace_null} 5 | \title{Replace NULLs} 6 | \usage{ 7 | replace_null(x, replacement = NA) 8 | } 9 | \arguments{ 10 | \item{x}{list to use} 11 | 12 | \item{replacement}{Replacement value for missing values} 13 | } 14 | \description{ 15 | Replace all the empty values in a list 16 | } 17 | \examples{ 18 | list(a = NULL, b = 1, c = list(foo = NULL, bar = NULL)) \%>\% replace_null() 19 | 20 | } 21 | -------------------------------------------------------------------------------- /man/roomba-package.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/roomba-package.R 3 | \docType{package} 4 | \name{roomba-package} 5 | \alias{roomba-package} 6 | \title{roomba package} 7 | \description{ 8 | \tabular{ll}{ 9 | Package: \tab roomba\cr 10 | Type: \tab Package\cr 11 | } 12 | } 13 | \details{ 14 | Collection of functions to do deal with deeply nested data 15 | } 16 | -------------------------------------------------------------------------------- /man/roomba.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/rrroomba.R 3 | \name{roomba} 4 | \alias{roomba} 5 | \title{Roomba} 6 | \usage{ 7 | roomba(inp, cols = NULL, default = NA, keep = all) 8 | } 9 | \arguments{ 10 | \item{inp}{List to tidy} 11 | 12 | \item{cols}{Columns to keep} 13 | 14 | \item{default}{Replacement for NULL values. Defaults to NA.} 15 | 16 | \item{keep}{Should all or any data be kept?} 17 | } 18 | \description{ 19 | Tidy your nested list 20 | } 21 | \examples{ 22 | 23 | simple \%>\% roomba(cols = c("name", "goodstuff"), keep = any) 24 | simple \%>\% roomba(cols = c("name", "goodstuff"), keep = any) 25 | } 26 | -------------------------------------------------------------------------------- /man/shiny_roomba.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/run_shiny.R 3 | \name{shiny_roomba} 4 | \alias{shiny_roomba} 5 | \title{Run the roomba app} 6 | \usage{ 7 | shiny_roomba(browse = TRUE) 8 | } 9 | \arguments{ 10 | \item{browse}{Logical. Use browser for running Shiny app.} 11 | } 12 | \description{ 13 | Run the roomba app 14 | } 15 | \examples{ 16 | \dontrun{ 17 | if(require(shiny)){ 18 | shiny_roomba() 19 | } 20 | } 21 | } 22 | -------------------------------------------------------------------------------- /man/simple.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/roomba-package.R 3 | \docType{data} 4 | \name{simple} 5 | \alias{simple} 6 | \title{Example data with simple nested list} 7 | \format{An object of class \code{character} of length 1.} 8 | \usage{ 9 | simple 10 | } 11 | \description{ 12 | Example data with simple nested list 13 | } 14 | \keyword{datasets} 15 | -------------------------------------------------------------------------------- /man/twitter_data.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/roomba-package.R 3 | \docType{data} 4 | \name{twitter_data} 5 | \alias{twitter_data} 6 | \title{Example Twitter data} 7 | \format{An object of class \code{list} of length 20.} 8 | \source{ 9 | \url{https://developer.twitter.com/en/docs} 10 | } 11 | \usage{ 12 | twitter_data 13 | } 14 | \description{ 15 | Example data from Twitter API 16 | } 17 | \details{ 18 | twitter_data[[1]][["id"]] 19 | 20 | [[1]: R:[1 21 | ["id"]: R:%22id%22 22 | } 23 | \keyword{datasets} 24 | -------------------------------------------------------------------------------- /man/vimeo.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/roomba-package.R 3 | \docType{data} 4 | \name{vimeo} 5 | \alias{vimeo} 6 | \title{Example data from vimeo} 7 | \format{An object of class \code{response} of length 10.} 8 | \source{ 9 | \url{https://developer.vimeo.com/} 10 | } 11 | \usage{ 12 | vimeo 13 | } 14 | \description{ 15 | Example data from vimeo 16 | } 17 | \keyword{datasets} 18 | -------------------------------------------------------------------------------- /roomba.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | AutoAppendNewline: Yes 16 | StripTrailingWhitespace: Yes 17 | 18 | BuildType: Package 19 | PackageUseDevtools: Yes 20 | PackageInstallArgs: --no-multiarch --with-keep.source 21 | PackageRoxygenize: rd,collate,namespace 22 | -------------------------------------------------------------------------------- /tests/testthat.R: -------------------------------------------------------------------------------- 1 | library(testthat) 2 | library(roomba) 3 | test_check("roomba") 4 | -------------------------------------------------------------------------------- /tests/testthat/test-indexing.R: -------------------------------------------------------------------------------- 1 | context("roomba indexing tests") 2 | 3 | test_that("", { 4 | 5 | twitter_data <- twitter_data 6 | x <- dfs_idx(twitter_data, ~ .x$id_str == "998623997397876743") 7 | expect_length(x, 1) 8 | }) 9 | -------------------------------------------------------------------------------- /tests/testthat/test-utils.R: -------------------------------------------------------------------------------- 1 | context("roomba utility function tests") 2 | 3 | library(purrr) 4 | library(dplyr) 5 | 6 | test_that("Test that replace_null() works as expected", { 7 | # Test that replace_null() works as expected 8 | lst <- list(c("a", "b"), x = NULL) 9 | cleaned_lst <- roomba:::replace_null(lst, replacement = "foo") 10 | expect_equal(cleaned_lst, 11 | list(c("a", "b"), x = "foo")) 12 | }) 13 | 14 | test_that("Test that replace_null() works with numbers", { 15 | # Test that replace_null() works as expected 16 | 17 | toy_data <- jsonlite::fromJSON(' 18 | { 19 | "stuff": { 20 | "buried": { 21 | "deep": [ 22 | { 23 | "goodstuff": "here", 24 | "secret_power": 5, 25 | "other_secret_power": [] 26 | }, 27 | { 28 | "goodstuff": "here", 29 | "name": "Amanda Dobbyn", 30 | "more_nested_stuff": 4 31 | } 32 | ], 33 | "alsodeep": 2342423234, 34 | "deeper": { 35 | "foo": [ 36 | { 37 | "goodstuff": "not here", 38 | "name": "barb", 39 | "secret_power": [] 40 | }, 41 | { 42 | "goodstuff": "here", // More deeply nested "goodstuff" than "goodstuff"s above 43 | "name": "borris" 44 | } 45 | ] 46 | } 47 | } 48 | } 49 | }', simplifyVector = FALSE) 50 | 51 | 52 | cleaned_lst <- roomba:::replace_null(toy_data) 53 | 54 | secret_power <- cleaned_lst %>% dfs_idx(~ .x$goodstuff == "here") %>% 55 | purrr:::map_dfr(~ cleaned_lst[[.x]]) %>% 56 | dplyr::pull(secret_power) 57 | 58 | expect_type(secret_power, "integer") 59 | 60 | toy_names <- cleaned_lst %>% dfs_idx(~ .x$goodstuff == "here") %>% 61 | purrr:::map_dfr(~ cleaned_lst[[.x]]) %>% 62 | dplyr::pull(name) 63 | 64 | expect_type(toy_names, "character") 65 | 66 | }) 67 | -------------------------------------------------------------------------------- /vignettes/2018-06-26-roomba.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | slug: "roomba" 3 | title: A package for tidying nested lists 4 | package_version: 0.0.1 5 | authors: 6 | - name: Amanda Dobbyn 7 | url: https://github.com/aedobbyn 8 | - name: Jim Hester 9 | url: https://github.com/jimhester 10 | - name: Laura DeCicco 11 | url: https://github.com/ldecicco-USGS 12 | - name: Christine Stawitz 13 | url: https://github.com/cstawitz 14 | - name: Isabella Velasquez 15 | url: https://github.com/ivelasq 16 | date: 2018-06-26 17 | categories: blog 18 | topicid: 925 19 | output: md_document 20 | tags: 21 | - r 22 | - community 23 | - software 24 | - package 25 | - meetings 26 | - unconf 27 | - unconf18 28 | --- 29 | Data == knowledge! Much of the data we use, whether it be from government repositories, social media, GitHub, or e-commerce sites comes from public-facing APIs. The quantity of data available is truly staggering, but munging JSON output into a format that is easily analyzable in R is an equally staggering undertaking. When JSON is turned into an R object, it usually becomes a deeply nested list riddled with missing values that is difficult to untangle into a tidy format. Moreover, every API presents its own challenges; code you've written to clean up data from GitHub isn't necessarily going to work on Twitter data, as each API spews data out in its own unique, headache-inducing nested list structure. To ease and generalize this process, [Amanda Dobbyn proposed](https://github.com/ropensci/unconf18/issues/24) an #unconf18 project for a general API response tidier! Welcome `roomba`, our first stab at easing the process of tidying nested lists! 30 | 31 | 32 |
    Drawing
    33 | 34 | 35 | `roomba` will eventually be able to walk nested lists in a variety of different structures from JSON output, replace `NULL` or `.empty` values with `NA`s or a user-specified value, and return a `tibble` with names matching a user-specified list. Of course, in two days we haven't *fully* achieved this vision, but we're off to a promising start. 36 | 37 | 38 | ## The birth of roomba 39 | 40 | 41 | It was clear Amanda was on to something good by the lively discussion in the [#runconf18](https://github.com/ropensci/unconf18/issues/) issues repository leading up to the unconf. Thanks to input from Jenny Bryan, Jim Hester, Carl Boettinger, Scott Chamberlain, Bob Rudis, and Noam Ross, we had a lot of ideas to work with when the unconf began. Fortunately, Jim already had a function called `dfs_idx()` ([here](https://github.com/ropenscilabs/roomba/blob/master/R/dfs_idx.R)) written to perform depth-first searches of nested lists from the [GitNub GraphQL API](https://developer.github.com/v4/). With the core list-traversal code out of the way, we split our efforts between developing a usable interface, stockpiling `.JSON` files to test on, and developing a Shiny app. 42 | 43 | 44 | ## What's working 45 | We've got the basic structure of `roomba` sorted out, and you should install it from GitHub to try out! Here are a few of the examples we've put together. 46 | 47 | 48 | ```{r} 49 | library(roomba) 50 | #load twitter data example 51 | data(twitter_data) 52 | 53 | #roomba-fy! 54 | roomba(twitter_data, c("created_at", "name")) 55 | ``` 56 | 57 | And just the *first* element of the `twitter_data` list will show you that `roomba` has simplified this process quite a bit. 58 | 59 | ```{r} 60 | twitter_data[[1]] 61 | 62 | ``` 63 | 64 | We created a Shiny app too, which in its current state allows you to select a `.Rda` or `.JSON` file, pick two variables, and create a scatterplot of them. 65 | 66 | 67 | Run the app like this: 68 | ```{r eval=FALSE} 69 | shiny_roomba() 70 | ``` 71 | 72 | ## What's not 73 | Of course, in two days we weren't able to build a magical one-size-fits-all solution to every API response data headache. Right now, the main barrier to usability is that both the `roomba()` function and `shiny_roomba()` app only work on sub-list items of the same length and same data type stored at the same depth. To illustrate on the `twitter_data`: 74 | 75 | ```{r} 76 | #This doesn't work because "user" has data of different types and lengths 77 | roomba(twitter_data, c("user")) 78 | 79 | 80 | #This doesn't work because "name" and "retweet_count" are at different depths. 81 | roomba(twitter_data, c("name","retweet_count")) 82 | ``` 83 | 84 | 85 | In addition, we've got some features we want to add, such as handling a larger variety of column names (i.e. passing a string for a single column name, keeping all values even if they are all `NULL`). We would love your feedback on other things we can add! 86 | 87 | 88 | ## The team 89 | 90 | **Amanda Dobbyn**
    91 | Job: Data Scientist at Earlybird Software
    92 | Project contributions: initial GH issue, package name, wrapper for `dfs_idx()`
    93 | 94 | **Jim Hester**
    95 | Job: Software Engineer at RStudio
    96 | Project contributions: `dfs_idx()` and `remove_nulls()` functions, package building, README, and debugging
    97 | 98 | **Christine Stawitz**
    99 | Job: Fishery Biologist at NOAA Fisheries
    100 | Project contributions: Shiny app, README and blog post writing
    101 | 102 | **Laura DeCicco**
    103 | Job: Data Scientist at U.S. Geological Survey
    104 | Project contributions: Fixing merge conflicts :)
    105 | 106 | **Isabella Velasquez**
    107 | Job: Data Analyst at the Bill & Melinda Gates Foundation
    108 | Project contributions: hex sticker!
    109 | -------------------------------------------------------------------------------- /vignettes/2018-06-26-roomba.md: -------------------------------------------------------------------------------- 1 | Data == knowledge! Much of the data we use, whether it be from 2 | government repositories, social media, GitHub, or e-commerce sites comes 3 | from public-facing APIs. The quantity of data available is truly 4 | staggering, but munging JSON output into a format that is easily 5 | analyzable in R is an equally staggering undertaking. When JSON is 6 | turned into an R object, it usually becomes a deeply nested list riddled 7 | with missing values that is difficult to untangle into a tidy format. 8 | Moreover, every API presents its own challenges; code you've written to 9 | clean up data from GitHub isn't necessarily going to work on Twitter 10 | data, as each API spews data out in its own unique, headache-inducing 11 | nested list structure. To ease and generalize this process, [Amanda 12 | Dobbyn proposed](https://github.com/ropensci/unconf18/issues/24) an 13 | \#unconf18 project for a general API response tidier! Welcome `roomba`, 14 | our first stab at easing the process of tidying nested lists! 15 | 16 |
    17 | Drawing 18 |
    19 | `roomba` will eventually be able to walk nested lists in a variety of 20 | different structures from JSON output, replace `NULL` or `.empty` values 21 | with `NA`s or a user-specified value, and return a `tibble` with names 22 | matching a user-specified list. Of course, in two days we haven't 23 | *fully* achieved this vision, but we're off to a promising start. 24 | 25 | The birth of roomba 26 | ------------------- 27 | 28 | It was clear Amanda was on to something good by the lively discussion in 29 | the [\#runconf18](https://github.com/ropensci/unconf18/issues/) issues 30 | repository leading up to the unconf. Thanks to input from Jenny Bryan, 31 | Jim Hester, Carl Boettinger, Scott Chamberlain, Bob Rudis, and Noam 32 | Ross, we had a lot of ideas to work with when the unconf began. 33 | Fortunately, Jim already had a function called `dfs_idx()` 34 | ([here](https://github.com/ropenscilabs/roomba/blob/master/R/dfs_idx.R)) 35 | written to perform depth-first searches of nested lists from the [GitNub 36 | GraphQL API](https://developer.github.com/v4/). With the core 37 | list-traversal code out of the way, we split our efforts between 38 | developing a usable interface, stockpiling `.JSON` files to test on, and 39 | developing a Shiny app. 40 | 41 | What's working 42 | -------------- 43 | 44 | We've got the basic structure of `roomba` sorted out, and you should 45 | install it from GitHub to try out! Here are a few of the examples we've 46 | put together. 47 | 48 | library(roomba) 49 | #load twitter data example 50 | data(twitter_data) 51 | 52 | #roomba-fy! 53 | roomba(twitter_data, c("created_at", "name")) 54 | 55 | ## # A tibble: 24 x 2 56 | ## name created_at 57 | ## 58 | ## 1 Code for America Mon Aug 10 18:59:29 +0000 2009 59 | ## 2 Ben Lorica Mon Dec 22 22:06:18 +0000 2008 60 | ## 3 Dan Sholler Thu Apr 03 20:09:24 +0000 2014 61 | ## 4 Code for America Mon Aug 10 18:59:29 +0000 2009 62 | ## 5 FiveThirtyEight Tue Jan 21 21:39:32 +0000 2014 63 | ## 6 Digital Impact Wed Oct 07 21:10:53 +0000 2009 64 | ## 7 Drew Williams Thu Aug 07 18:41:29 +0000 2014 65 | ## 8 joe Fri May 29 13:25:25 +0000 2009 66 | ## 9 Data Analysts 4 Good Wed May 07 16:55:33 +0000 2014 67 | ## 10 Ryan Frederick Sun Mar 01 19:06:53 +0000 2009 68 | ## # ... with 14 more rows 69 | 70 | And just the *first* element of the `twitter_data` list will show you 71 | that `roomba` has simplified this process quite a bit. 72 | 73 | twitter_data[[1]] 74 | 75 | ## $created_at 76 | ## [1] "Mon May 21 17:58:09 +0000 2018" 77 | ## 78 | ## $id 79 | ## [1] 9.98624e+17 80 | ## 81 | ## $id_str 82 | ## [1] "998623997397876743" 83 | ## 84 | ## $text 85 | ## [1] "Could a program like food stamps have a Cambridge Analytica moment? How do we allow for the innovation that data pl… https://t.co/7tVf1qmNmq" 86 | ## 87 | ## $truncated 88 | ## [1] TRUE 89 | ## 90 | ## $entities 91 | ## $entities$hashtags 92 | ## list() 93 | ## 94 | ## $entities$symbols 95 | ## list() 96 | ## 97 | ## $entities$user_mentions 98 | ## list() 99 | ## 100 | ## $entities$urls 101 | ## $entities$urls[[1]] 102 | ## $entities$urls[[1]]$url 103 | ## [1] "https://t.co/7tVf1qmNmq" 104 | ## 105 | ## $entities$urls[[1]]$expanded_url 106 | ## [1] "https://twitter.com/i/web/status/998623997397876743" 107 | ## 108 | ## $entities$urls[[1]]$display_url 109 | ## [1] "twitter.com/i/web/status/9…" 110 | ## 111 | ## $entities$urls[[1]]$indices 112 | ## $entities$urls[[1]]$indices[[1]] 113 | ## [1] 117 114 | ## 115 | ## $entities$urls[[1]]$indices[[2]] 116 | ## [1] 140 117 | ## 118 | ## 119 | ## 120 | ## 121 | ## 122 | ## $source 123 | ## [1] "TweetDeck" 124 | ## 125 | ## $in_reply_to_status_id 126 | ## NULL 127 | ## 128 | ## $in_reply_to_status_id_str 129 | ## NULL 130 | ## 131 | ## $in_reply_to_user_id 132 | ## NULL 133 | ## 134 | ## $in_reply_to_user_id_str 135 | ## NULL 136 | ## 137 | ## $in_reply_to_screen_name 138 | ## NULL 139 | ## 140 | ## $user 141 | ## $user$id 142 | ## [1] 64482503 143 | ## 144 | ## $user$id_str 145 | ## [1] "64482503" 146 | ## 147 | ## $user$name 148 | ## [1] "Code for America" 149 | ## 150 | ## $user$screen_name 151 | ## [1] "codeforamerica" 152 | ## 153 | ## $user$location 154 | ## [1] "San Francisco, California" 155 | ## 156 | ## $user$description 157 | ## [1] "Government can work for the people, by the people, in the 21st century. Help us make it so." 158 | ## 159 | ## $user$url 160 | ## [1] "https://t.co/l9lokka0rJ" 161 | ## 162 | ## $user$entities 163 | ## $user$entities$url 164 | ## $user$entities$url$urls 165 | ## $user$entities$url$urls[[1]] 166 | ## $user$entities$url$urls[[1]]$url 167 | ## [1] "https://t.co/l9lokka0rJ" 168 | ## 169 | ## $user$entities$url$urls[[1]]$expanded_url 170 | ## [1] "http://codeforamerica.org" 171 | ## 172 | ## $user$entities$url$urls[[1]]$display_url 173 | ## [1] "codeforamerica.org" 174 | ## 175 | ## $user$entities$url$urls[[1]]$indices 176 | ## $user$entities$url$urls[[1]]$indices[[1]] 177 | ## [1] 0 178 | ## 179 | ## $user$entities$url$urls[[1]]$indices[[2]] 180 | ## [1] 23 181 | ## 182 | ## 183 | ## 184 | ## 185 | ## 186 | ## $user$entities$description 187 | ## $user$entities$description$urls 188 | ## list() 189 | ## 190 | ## 191 | ## 192 | ## $user$protected 193 | ## [1] FALSE 194 | ## 195 | ## $user$followers_count 196 | ## [1] 49202 197 | ## 198 | ## $user$friends_count 199 | ## [1] 1716 200 | ## 201 | ## $user$listed_count 202 | ## [1] 2659 203 | ## 204 | ## $user$created_at 205 | ## [1] "Mon Aug 10 18:59:29 +0000 2009" 206 | ## 207 | ## $user$favourites_count 208 | ## [1] 4490 209 | ## 210 | ## $user$utc_offset 211 | ## [1] -25200 212 | ## 213 | ## $user$time_zone 214 | ## [1] "Pacific Time (US & Canada)" 215 | ## 216 | ## $user$geo_enabled 217 | ## [1] TRUE 218 | ## 219 | ## $user$verified 220 | ## [1] TRUE 221 | ## 222 | ## $user$statuses_count 223 | ## [1] 15912 224 | ## 225 | ## $user$lang 226 | ## [1] "en" 227 | ## 228 | ## $user$contributors_enabled 229 | ## [1] FALSE 230 | ## 231 | ## $user$is_translator 232 | ## [1] FALSE 233 | ## 234 | ## $user$is_translation_enabled 235 | ## [1] FALSE 236 | ## 237 | ## $user$profile_background_color 238 | ## [1] "EBEBEB" 239 | ## 240 | ## $user$profile_background_image_url 241 | ## [1] "http://abs.twimg.com/images/themes/theme7/bg.gif" 242 | ## 243 | ## $user$profile_background_image_url_https 244 | ## [1] "https://abs.twimg.com/images/themes/theme7/bg.gif" 245 | ## 246 | ## $user$profile_background_tile 247 | ## [1] FALSE 248 | ## 249 | ## $user$profile_image_url 250 | ## [1] "http://pbs.twimg.com/profile_images/615534833645678592/iAO_Lytr_normal.jpg" 251 | ## 252 | ## $user$profile_image_url_https 253 | ## [1] "https://pbs.twimg.com/profile_images/615534833645678592/iAO_Lytr_normal.jpg" 254 | ## 255 | ## $user$profile_banner_url 256 | ## [1] "https://pbs.twimg.com/profile_banners/64482503/1497895952" 257 | ## 258 | ## $user$profile_link_color 259 | ## [1] "CF1B41" 260 | ## 261 | ## $user$profile_sidebar_border_color 262 | ## [1] "FFFFFF" 263 | ## 264 | ## $user$profile_sidebar_fill_color 265 | ## [1] "F3F3F3" 266 | ## 267 | ## $user$profile_text_color 268 | ## [1] "333333" 269 | ## 270 | ## $user$profile_use_background_image 271 | ## [1] FALSE 272 | ## 273 | ## $user$has_extended_profile 274 | ## [1] FALSE 275 | ## 276 | ## $user$default_profile 277 | ## [1] FALSE 278 | ## 279 | ## $user$default_profile_image 280 | ## [1] FALSE 281 | ## 282 | ## $user$following 283 | ## [1] TRUE 284 | ## 285 | ## $user$follow_request_sent 286 | ## [1] FALSE 287 | ## 288 | ## $user$notifications 289 | ## [1] FALSE 290 | ## 291 | ## $user$translator_type 292 | ## [1] "none" 293 | ## 294 | ## 295 | ## $geo 296 | ## NULL 297 | ## 298 | ## $coordinates 299 | ## NULL 300 | ## 301 | ## $place 302 | ## NULL 303 | ## 304 | ## $contributors 305 | ## NULL 306 | ## 307 | ## $is_quote_status 308 | ## [1] FALSE 309 | ## 310 | ## $retweet_count 311 | ## [1] 0 312 | ## 313 | ## $favorite_count 314 | ## [1] 0 315 | ## 316 | ## $favorited 317 | ## [1] FALSE 318 | ## 319 | ## $retweeted 320 | ## [1] FALSE 321 | ## 322 | ## $possibly_sensitive 323 | ## [1] FALSE 324 | ## 325 | ## $possibly_sensitive_appealable 326 | ## [1] FALSE 327 | ## 328 | ## $lang 329 | ## [1] "en" 330 | 331 | We created a Shiny app too, which in its current state allows you to 332 | select a `.Rda` or `.JSON` file, pick two variables, and create a 333 | scatterplot of them. 334 | 335 | Run the app like this: 336 | 337 | shiny_roomba() 338 | 339 | What's not 340 | ---------- 341 | 342 | Of course, in two days we weren't able to build a magical 343 | one-size-fits-all solution to every API response data headache. Right 344 | now, the main barrier to usability is that both the `roomba()` function 345 | and `shiny_roomba()` app only work on sub-list items of the same length 346 | and same data type stored at the same depth. To illustrate on the 347 | `twitter_data`: 348 | 349 | #This doesn't work because "user" has data of different types and lengths 350 | roomba(twitter_data, c("user")) 351 | 352 | ## # A tibble: 1,007 x 1 353 | ## user 354 | ## 355 | ## 1 356 | ## 2 357 | ## 3 358 | ## 4 359 | ## 5 360 | ## 6 361 | ## 7 362 | ## 8 363 | ## 9 364 | ## 10 365 | ## # ... with 997 more rows 366 | 367 | #This doesn't work because "name" and "retweet_count" are at different depths. 368 | roomba(twitter_data, c("name","retweet_count")) 369 | 370 | ## # A tibble: 0 x 0 371 | 372 | In addition, we've got some features we want to add, such as handling a 373 | larger variety of column names (i.e. passing a string for a single 374 | column name, keeping all values even if they are all `NULL`). We would 375 | love your feedback on other things we can add! 376 | 377 | The team 378 | -------- 379 | 380 | **Amanda Dobbyn**
    Job: Data Scientist at Earlybird Software
    381 | Project contributions: initial GH issue, package name, wrapper for 382 | `dfs_idx()`
    383 | 384 | **Jim Hester**
    Job: Software Engineer at RStudio
    Project 385 | contributions: `dfs_idx()` and `remove_nulls()` functions, package 386 | building, README, and debugging
    387 | 388 | **Christine Stawitz**
    Job: Postdoctoral researcher at UW's School 389 | of Aquatic and Fishery Sciences
    Project contributions: Shiny app, 390 | README and blog post writing
    391 | 392 | **Laura DeCicco**
    Job: Data Scientist at U.S. Geological Survey 393 |
    Project contributions: Fixing merge conflicts :)
    394 | 395 | **Isabella Velasquez**
    Job: Data Analyst at the Bill & Melinda 396 | Gates Foundation
    Project contributions: hex sticker!
    397 | -------------------------------------------------------------------------------- /vignettes/sticker.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cstawitz/roomba/95be18c10e18558467b22e0675bb98480a264219/vignettes/sticker.png -------------------------------------------------------------------------------- /vignettes/twitter.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Twitter API Data" 3 | output: rmarkdown::html_vignette 4 | vignette: > 5 | %\VignetteIndexEntry{Twitter API Data} 6 | %\VignetteEngine{knitr::rmarkdown} 7 | \usepackage[utf8]{inputenc} 8 | --- 9 | 10 | This vignette demonstrates the functionality of the `roomba` package on Twitter API data. 11 | 12 | ## Downloading Data 13 | 14 | For more information on downloading Twitter data, please check out the [`httr`](https://github.com/r-lib/httr/blob/master/demo/oauth1-twitter.r) package. 15 | 16 | The output `req` will be a nested list; you can save it using `write_rda()` from the `tidyverse` package. 17 | 18 | ```{r, eval = FALSE} 19 | library(httr) 20 | library(jsonlite) 21 | 22 | oauth_endpoints("twitter") 23 | 24 | # edit the keys with own information 25 | myapp <- oauth_app("twitter", 26 | key = "EOy06ORJM56b8mk1yoUo6bnjG", 27 | secret = "8z4PMPIJrXKYE9JrALjI4TnzDJksB8xRphHj0L5JpWpSiEtbs6" 28 | ) 29 | 30 | twitter_token <- oauth1.0_token(oauth_endpoints("twitter"), myapp) 31 | 32 | req <- GET("https://api.twitter.com/1.1/statuses/home_timeline.json", 33 | config(token = twitter_token)) 34 | 35 | stop_for_status(req) 36 | 37 | content(req) 38 | ``` 39 | 40 | ## Example 41 | 42 | We provide actual Twitter data as an example, which can be loaded using `data(twitter_data).` 43 | 44 | ```{r, message = FALSE} 45 | library(ggplot2) 46 | library(magrittr) 47 | library(roomba) 48 | ``` 49 | 50 | Using the `roomba()` function will gather information based on your variables of interest (in this case, followers_count and friends_count). From there, you can use other `dplyr` functions on your data. 51 | 52 | ```{r} 53 | twitter_data <- twitter_data 54 | 55 | twitter_data %>% roomba(cols = c("followers_count", "friends_count")) %>% 56 | ggplot(aes(x = followers_count, y = friends_count)) + 57 | geom_point() + 58 | theme_minimal() 59 | ``` 60 | --------------------------------------------------------------------------------