├── .github ├── .gitignore ├── FUNDING.yml └── workflows │ └── R-CMD-check.yaml ├── vignettes ├── .gitignore └── Introduction.Rmd ├── LICENSE ├── .gitignore ├── man ├── figures │ ├── hex.png │ ├── Thumbs.db │ └── gif_baris.gif ├── BARIS_home.Rd ├── BARIS_explain.Rd ├── BARIS_set_fromJSON.Rd ├── BARIS_resources.Rd ├── BARIS_search.Rd └── BARIS_extract.Rd ├── inst └── rstudio │ └── addins.dcf ├── .Rbuildignore ├── BARIS.Rproj ├── cran-comments.md ├── NEWS.md ├── NAMESPACE ├── DESCRIPTION ├── LICENSE.md ├── R ├── BARIS_explain.R ├── memoise.R ├── BARIS_resources.R ├── BARIS_home.R ├── BARIS_search.R ├── BARIS_extract.R └── BARIS_ui.R ├── CODE_OF_CONDUCT.md ├── README.Rmd └── README.md /.github/.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | -------------------------------------------------------------------------------- /vignettes/.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | *.R 3 | -------------------------------------------------------------------------------- /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | custom: ['https://www.buymeacoffee.com/Fodil'] 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | YEAR: 2020 2 | COPYRIGHT HOLDER: Mohamed El Fodil Ihaddaden 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .Rproj.user 2 | .Rhistory 3 | .RData 4 | .Ruserdata 5 | inst/doc 6 | -------------------------------------------------------------------------------- /man/figures/hex.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feddelegrand7/BARIS/HEAD/man/figures/hex.png -------------------------------------------------------------------------------- /man/figures/Thumbs.db: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feddelegrand7/BARIS/HEAD/man/figures/Thumbs.db -------------------------------------------------------------------------------- /man/figures/gif_baris.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/feddelegrand7/BARIS/HEAD/man/figures/gif_baris.gif -------------------------------------------------------------------------------- /inst/rstudio/addins.dcf: -------------------------------------------------------------------------------- 1 | Name: BARIS 2 | Description: Interacting with the data.gouv.fr api using BARIS. 3 | Binding: BARIS_ui 4 | Interactive: true 5 | 6 | -------------------------------------------------------------------------------- /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.*\.Rproj$ 2 | ^\.Rproj\.user$ 3 | ^README\.Rmd$ 4 | ^LICENSE\.md$ 5 | ^CRAN-RELEASE$ 6 | ^cran-comments\.md$ 7 | ^CODE_OF_CONDUCT\.md$ 8 | ^\.travis\.yml$ 9 | ^man/figures/gif_baris\.gif$ 10 | ^\.github$ 11 | -------------------------------------------------------------------------------- /BARIS.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | AutoAppendNewline: Yes 16 | StripTrailingWhitespace: Yes 17 | 18 | BuildType: Package 19 | PackageUseDevtools: Yes 20 | PackageInstallArgs: --no-multiarch --with-keep.source 21 | PackageCheckArgs: --as-cran 22 | -------------------------------------------------------------------------------- /man/BARIS_home.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/BARIS_home.R 3 | \name{BARIS_home} 4 | \alias{BARIS_home} 5 | \title{Home Page Datasets} 6 | \usage{ 7 | BARIS_home() 8 | } 9 | \value{ 10 | A dataframe with datasets characteristics 11 | } 12 | \description{ 13 | This function displays the most important information about the datasets that are currently displayed within the home page of the data.gouv portal. 14 | } 15 | \examples{ 16 | \donttest{ 17 | 18 | BARIS_home()} 19 | } 20 | -------------------------------------------------------------------------------- /cran-comments.md: -------------------------------------------------------------------------------- 1 | -- R CMD check results ------------ BARIS 1.1.2 ---- 2 | Duration: 1m 15.2s 3 | 4 | 0 errors √ | 0 warnings √ | 0 notes √ 5 | 6 | * Following the email of Professor Brian Ripley, now the functions that call resources on the internet fail gracefully, they return `NULL` in case of the `http` response is not successful. 7 | 8 | * I've modified the `BARIS_search` function by removing the `n_pages` parameter and adding two new parameters to query the results, namely: `page_number` and `page_size`. 9 | 10 | Thanks. 11 | -------------------------------------------------------------------------------- /man/BARIS_explain.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/BARIS_explain.R 3 | \name{BARIS_explain} 4 | \alias{BARIS_explain} 5 | \title{Data set description} 6 | \usage{ 7 | BARIS_explain(datasetId) 8 | } 9 | \arguments{ 10 | \item{datasetId}{the unique id of a data set} 11 | } 12 | \value{ 13 | a character string describing a data set 14 | } 15 | \description{ 16 | This function provides in French a detailed description of a data set. 17 | } 18 | \examples{ 19 | \donttest{ 20 | 21 | BARIS_explain("53699934a3a729239d2051a1")} 22 | } 23 | -------------------------------------------------------------------------------- /man/BARIS_set_fromJSON.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/memoise.R 3 | \name{BARIS_set_fromJSON} 4 | \alias{BARIS_set_fromJSON} 5 | \title{Sets memoise cache function} 6 | \usage{ 7 | BARIS_set_fromJSON(cache_folder = tempdir()) 8 | } 9 | \arguments{ 10 | \item{cache_folder}{The local folder where to save files} 11 | } 12 | \value{ 13 | A memoised function 14 | } 15 | \description{ 16 | Uses the memoise package to set a cache function for jsonlite::fromJSON 17 | The choice of the cache folder defines where to save the local files. If not set, will use tempdir() 18 | } 19 | \examples{ 20 | \dontrun{ 21 | mem_fromJSON <- BARIS_set_fromJSON() 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /NEWS.md: -------------------------------------------------------------------------------- 1 | # BARIS 1.1.3 2 | * Checking for the `http` responses using a `HEAD` method instead of a `GET` which takes too much time on large `CSV` files. 3 | 4 | # BARIS 1.1.2 5 | * The functions that query data now check for the API response, and if not successful, they return `NULL`. 6 | 7 | * In `BARIS_search`, removing the `n_pages` parameter and adding two new parameters to query the results, namely: `page_number` and `page_size` 8 | 9 | 10 | # BARIS 1.1.1 11 | 12 | * minor bugs fixing 13 | * added assertion features 14 | 15 | # BARIS 1.1.0 16 | 17 | * I've added an add-in that allows the user to use the package in an interactive manner. 18 | 19 | # BARIS 1.0.0 20 | 21 | * First release of BARIS. 22 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2: do not edit by hand 2 | 3 | export(BARIS_explain) 4 | export(BARIS_extract) 5 | export(BARIS_home) 6 | export(BARIS_resources) 7 | export(BARIS_search) 8 | export(BARIS_set_fromJSON) 9 | importFrom(XML,xmlParse) 10 | importFrom(XML,xmlToDataFrame) 11 | importFrom(checkmate,assert_character) 12 | importFrom(data.table,fread) 13 | importFrom(downloader,download) 14 | importFrom(dplyr,as_tibble) 15 | importFrom(httr,BROWSE) 16 | importFrom(janitor,clean_names) 17 | importFrom(jsonlite,fromJSON) 18 | importFrom(magrittr,"%>%") 19 | importFrom(memoise,cache_filesystem) 20 | importFrom(memoise,memoise) 21 | importFrom(rgdal,readOGR) 22 | importFrom(rio,import) 23 | importFrom(sf,read_sf) 24 | importFrom(stringi,stri_trans_general) 25 | importFrom(stringr,str_replace_all) 26 | importFrom(stringr,str_trim) 27 | importFrom(utils,unzip) 28 | -------------------------------------------------------------------------------- /man/BARIS_resources.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/BARIS_resources.R 3 | \name{BARIS_resources} 4 | \alias{BARIS_resources} 5 | \title{Displaying resources within a data set} 6 | \usage{ 7 | BARIS_resources(datasetId) 8 | } 9 | \arguments{ 10 | \item{datasetId}{the unique identifier of a data set} 11 | } 12 | \value{ 13 | a table of all the data frames (resources) included within a specific data set and their main characteristics 14 | } 15 | \description{ 16 | The data.gouv API provides access to several data sets in which one can find several individual data frames to exploit. The BARIS_resources lists all the data frames available within a specific data set along with the main information concerning the data frames. 17 | } 18 | \examples{ 19 | \donttest{ 20 | 21 | BARIS_resources("53699934a3a729239d2051a1")} 22 | } 23 | -------------------------------------------------------------------------------- /man/BARIS_search.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/BARIS_search.R 3 | \name{BARIS_search} 4 | \alias{BARIS_search} 5 | \title{Searching for a data set} 6 | \usage{ 7 | BARIS_search(query, page_number = 1, page_size = 20) 8 | } 9 | \arguments{ 10 | \item{query}{a character string defining the research.} 11 | 12 | \item{page_number}{numeric value denoting which page to search for, defaults to 1} 13 | 14 | \item{page_size}{numeric value, for a specific page (see \code{page_number}), how 15 | many results do you want to query. Defaults to 20} 16 | } 17 | \value{ 18 | a character string 19 | } 20 | \description{ 21 | This function searches specific data sets through the data.gouv API according to the pattern provided into the query argument. 22 | } 23 | \examples{ 24 | \donttest{ 25 | 26 | BARIS_search("Marseille")} 27 | } 28 | -------------------------------------------------------------------------------- /man/BARIS_extract.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/BARIS_extract.R 3 | \name{BARIS_extract} 4 | \alias{BARIS_extract} 5 | \title{Extracting A Data frame From The French Portal data.gouv API} 6 | \usage{ 7 | BARIS_extract(resourceId, format) 8 | } 9 | \arguments{ 10 | \item{resourceId}{the unique identifier of the individual data frame (resource) to read into R. Not to confound with the global data sets ID that includes many data frames} 11 | 12 | \item{format}{the format of the required file (json, csv, xls, xlsx, xml, geojson or shp)} 13 | } 14 | \value{ 15 | a data frame 16 | } 17 | \description{ 18 | This function read directly into R a specific data frame (resource) by the mean of its unique ID. Currently, BARIS_extract() can read directly the following types of file : json, csv, xls, xlsx, xml, geojson, and shp. 19 | } 20 | \examples{ 21 | \donttest{ 22 | BARIS_extract("59ea7bba-f38a-4d75-b85f-2d1955050e53", format = "csv")} 23 | } 24 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: BARIS 2 | Type: Package 3 | Title: Access and Import Data from the French Open Data Portal 4 | Version: 1.1.3 5 | Author: Mohamed El Fodil Ihaddaden 6 | Maintainer: Mohamed El Fodil Ihaddaden 7 | Description: Allows the user to access and import data from the rich French open data portal through the provided free API . 8 | The portal is free, and no credential is required for extracting datasets. 9 | License: MIT + file LICENSE 10 | Encoding: UTF-8 11 | LazyData: true 12 | URL: https://github.com/feddelegrand7/BARIS 13 | BugReports: https://github.com/feddelegrand7/BARIS/issues 14 | VignetteBuilder: knitr 15 | Imports: 16 | magrittr, 17 | dplyr, 18 | data.table, 19 | downloader, 20 | sf, 21 | XML, 22 | httr, 23 | rgdal, 24 | janitor, 25 | rio, 26 | jsonlite, 27 | stringr, 28 | utils, 29 | memoise, 30 | stringi, 31 | shiny, 32 | miniUI, 33 | rstudioapi, 34 | checkmate, 35 | glue 36 | Suggests: 37 | knitr, 38 | rmarkdown 39 | RoxygenNote: 7.1.1 40 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | # MIT License 2 | 3 | Copyright (c) 2020 Mohamed El Fodil Ihaddaden 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /R/BARIS_explain.R: -------------------------------------------------------------------------------- 1 | 2 | #' Data set description 3 | #' 4 | #' @description This function provides in French a detailed description of a data set. 5 | #' @param datasetId the unique id of a data set 6 | #' 7 | #' @return a character string describing a data set 8 | #' 9 | #' @examples \donttest{ 10 | #' 11 | #' BARIS_explain("53699934a3a729239d2051a1")} 12 | #' @export 13 | #' @importFrom magrittr %>% 14 | #' @importFrom dplyr as_tibble 15 | #' @importFrom stringr str_replace_all 16 | #' @importFrom stringr str_trim 17 | #' @importFrom checkmate assert_character 18 | 19 | 20 | BARIS_explain <- function(datasetId) { 21 | 22 | checkmate::assert_character(datasetId) 23 | 24 | basic_url <- "https://www.data.gouv.fr/api/1/datasets/" 25 | 26 | final_url <- paste("https://www.data.gouv.fr/api/1/datasets/", datasetId, "/", sep = "") 27 | 28 | response_status <- is_response_successfull(final_url) 29 | 30 | if (!response_status) { 31 | return(NULL) 32 | } 33 | 34 | # set memoise cache fct 35 | mem_fromJSON <- BARIS_set_fromJSON() 36 | 37 | df_explain <- mem_fromJSON(final_url) 38 | 39 | Encoding(df_explain$description) <- "UTF-8" 40 | 41 | 42 | return(df_explain$description) 43 | } 44 | 45 | 46 | 47 | 48 | 49 | -------------------------------------------------------------------------------- /R/memoise.R: -------------------------------------------------------------------------------- 1 | 2 | is_response_successfull <- function(url) { 3 | 4 | response <- httr::HEAD( 5 | url = url 6 | ) 7 | 8 | there_is_an_error <- httr::http_error(response) 9 | 10 | if (there_is_an_error) { 11 | 12 | message <- httr::http_status(response) 13 | message <- message$message 14 | 15 | message( 16 | glue::glue( 17 | "Something went wrong, here the response message: 18 | <<< {message} >>> 19 | 20 | returning NULL 21 | " 22 | ) 23 | ) 24 | return(FALSE) 25 | } 26 | 27 | return(TRUE) 28 | 29 | } 30 | 31 | 32 | #' Sets memoise cache function 33 | #' 34 | #' Uses the memoise package to set a cache function for jsonlite::fromJSON 35 | #' The choice of the cache folder defines where to save the local files. If not set, will use tempdir() 36 | #' 37 | #' @param cache_folder The local folder where to save files 38 | #' 39 | #' @return A memoised function 40 | #' 41 | #' @examples \dontrun{ 42 | #' mem_fromJSON <- BARIS_set_fromJSON() 43 | #' } 44 | #' @export 45 | #' @importFrom memoise memoise 46 | #' @importFrom memoise cache_filesystem 47 | #' @importFrom jsonlite fromJSON 48 | 49 | 50 | BARIS_set_fromJSON <- function(cache_folder = tempdir()) { 51 | 52 | mem_fromJSON <- memoise::memoise(jsonlite::fromJSON, 53 | cache = memoise::cache_filesystem(cache_folder)) 54 | 55 | return(mem_fromJSON) 56 | 57 | } 58 | -------------------------------------------------------------------------------- /R/BARIS_resources.R: -------------------------------------------------------------------------------- 1 | #' Displaying resources within a data set 2 | #' 3 | #' @description The data.gouv API provides access to several data sets in which one can find several individual data frames to exploit. The BARIS_resources lists all the data frames available within a specific data set along with the main information concerning the data frames. 4 | #' 5 | #' @param datasetId the unique identifier of a data set 6 | #' 7 | #' @return a table of all the data frames (resources) included within a specific data set and their main characteristics 8 | #' 9 | #' @examples \donttest{ 10 | #' 11 | #' BARIS_resources("53699934a3a729239d2051a1")} 12 | #' @export 13 | #' @importFrom dplyr as_tibble 14 | #' @importFrom checkmate assert_character 15 | 16 | 17 | BARIS_resources <- function(datasetId) { 18 | 19 | 20 | checkmate::assert_character(datasetId) 21 | 22 | basic_url <- "https://www.data.gouv.fr/api/1/datasets/" 23 | 24 | final_url <- paste("https://www.data.gouv.fr/api/1/datasets/", datasetId, "/", sep = "") 25 | 26 | response_status <- is_response_successfull(final_url) 27 | 28 | if (!response_status) { 29 | return(NULL) 30 | } 31 | 32 | # set memoise cache fct 33 | mem_fromJSON <- BARIS_set_fromJSON() 34 | 35 | df <- mem_fromJSON(final_url) 36 | 37 | df <- df$resources[, c("id", "title",'format', 'published', 'url', 'description')] 38 | 39 | df <- as_tibble(df) 40 | 41 | return(df) 42 | } 43 | 44 | 45 | 46 | -------------------------------------------------------------------------------- /R/BARIS_home.R: -------------------------------------------------------------------------------- 1 | 2 | #' Home Page Datasets 3 | #' 4 | #' @description This function displays the most important information about the datasets that are currently displayed within the home page of the data.gouv portal. 5 | #' 6 | #' @return A dataframe with datasets characteristics 7 | #' 8 | #' @examples \donttest{ 9 | #' 10 | #' BARIS_home()} 11 | #' @export 12 | #' @importFrom dplyr as_tibble 13 | #' 14 | BARIS_home <- function() { 15 | 16 | my_url <- "https://www.data.gouv.fr/api/1/site/home/datasets/" 17 | 18 | response_status <- is_response_successfull(my_url) 19 | 20 | if (!response_status) { 21 | return(NULL) 22 | } 23 | 24 | # set memoise cache fct 25 | mem_fromJSON <- BARIS_set_fromJSON() 26 | 27 | df_info <- mem_fromJSON(my_url) 28 | 29 | id <- df_info$id 30 | 31 | title <- df_info$title 32 | 33 | frequency <- df_info$frequency 34 | 35 | created_at <- df_info$created_at 36 | 37 | last_modified <- df_info$last_modified 38 | 39 | last_update <- df_info$last_update 40 | 41 | archived <- df_info$archived 42 | 43 | deleted <- df_info$deleted 44 | 45 | views <- df_info$metrics[[4]] 46 | 47 | organization <- df_info$organization[[6]] 48 | 49 | temporal_cov_start <- df_info$temporal_coverage[[1]] 50 | 51 | temporal_cov_end <- df_info$temporal_coverage[[2]] 52 | 53 | page <- df_info$page 54 | 55 | df_info2 <- cbind( 56 | id, 57 | title, 58 | organization, 59 | page, 60 | views, 61 | frequency, 62 | temporal_cov_start, 63 | temporal_cov_end, 64 | created_at, 65 | last_modified, 66 | last_update, 67 | archived, 68 | deleted 69 | 70 | ) 71 | 72 | df_info2 <- as_tibble(df_info2) 73 | 74 | return(df_info2) 75 | } 76 | 77 | 78 | 79 | -------------------------------------------------------------------------------- /R/BARIS_search.R: -------------------------------------------------------------------------------- 1 | #' Searching for a data set 2 | #' 3 | #' @description This function searches specific data sets through the data.gouv API according to the pattern provided into the query argument. 4 | #' 5 | #' 6 | #' @param query a character string defining the research. 7 | #' @param page_number numeric value denoting which page to search for, defaults to 1 8 | #' @param page_size numeric value, for a specific page (see \code{page_number}), how 9 | #' many results do you want to query. Defaults to 20 10 | #' @return a character string 11 | #' @examples \donttest{ 12 | #' 13 | #' BARIS_search("Marseille")} 14 | #' @export 15 | #' @importFrom dplyr as_tibble 16 | #' @importFrom stringr str_replace_all 17 | #' @importFrom stringi stri_trans_general 18 | 19 | 20 | BARIS_search <- function(query, page_number = 1, page_size = 20) { 21 | 22 | base_url <- "https://www.data.gouv.fr/api/1/datasets/?q=" 23 | 24 | search <- query 25 | 26 | search <- str_replace_all(search, c(" " = "%20", " " = "%20")) 27 | 28 | search <- stri_trans_general(search, id = "Latin-ASCII") 29 | 30 | complement <- glue::glue("&page={page_number}&page_size={page_size}") 31 | 32 | final_url <- paste0(base_url, search, complement) 33 | 34 | response_status <- is_response_successfull(final_url) 35 | 36 | if (!response_status) { 37 | return(NULL) 38 | } 39 | 40 | # set memoise cache fct 41 | mem_fromJSON <- BARIS_set_fromJSON() 42 | 43 | df_info <- mem_fromJSON(final_url) 44 | 45 | df_info <- df_info$data 46 | 47 | if (is.null(df_info) || length(df_info) == 0) { 48 | message("No data found for the requested search, 49 | returning NULL") 50 | return(NULL) 51 | } 52 | 53 | id <- df_info$id 54 | 55 | title <- df_info$title 56 | 57 | frequency <- df_info$frequency 58 | 59 | created_at <- df_info$created_at 60 | 61 | last_modified <- df_info$last_modified 62 | 63 | last_update <- df_info$last_update 64 | 65 | archived <- df_info$archived 66 | 67 | deleted <- df_info$deleted 68 | 69 | views <- df_info$metrics[[4]] 70 | 71 | organization <- df_info$organization[[6]] 72 | 73 | page <- df_info$page 74 | 75 | df_info2 <- cbind( 76 | id, 77 | title, 78 | organization, 79 | page, 80 | views, 81 | frequency, 82 | created_at, 83 | last_modified, 84 | last_update, 85 | archived, 86 | deleted 87 | 88 | ) 89 | 90 | df_info2 <- as_tibble(df_info2) 91 | 92 | return(df_info2) 93 | 94 | } 95 | 96 | -------------------------------------------------------------------------------- /.github/workflows/R-CMD-check.yaml: -------------------------------------------------------------------------------- 1 | # For help debugging build failures open an issue on the RStudio community with the 'github-actions' tag. 2 | # https://community.rstudio.com/new-topic?category=Package%20development&tags=github-actions 3 | on: 4 | push: 5 | branches: 6 | - main 7 | - master 8 | pull_request: 9 | branches: 10 | - main 11 | - master 12 | 13 | name: R-CMD-check 14 | 15 | jobs: 16 | R-CMD-check: 17 | runs-on: ${{ matrix.config.os }} 18 | 19 | name: ${{ matrix.config.os }} (${{ matrix.config.r }}) 20 | 21 | strategy: 22 | fail-fast: false 23 | matrix: 24 | config: 25 | - {os: windows-latest, r: 'release'} 26 | - {os: macOS-latest, r: 'release'} 27 | - {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"} 28 | - {os: ubuntu-20.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"} 29 | 30 | env: 31 | R_REMOTES_NO_ERRORS_FROM_WARNINGS: true 32 | RSPM: ${{ matrix.config.rspm }} 33 | GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} 34 | 35 | steps: 36 | - uses: actions/checkout@v2 37 | 38 | - uses: r-lib/actions/setup-r@v1 39 | with: 40 | r-version: ${{ matrix.config.r }} 41 | 42 | - uses: r-lib/actions/setup-pandoc@v1 43 | 44 | - name: Query dependencies 45 | run: | 46 | install.packages('remotes') 47 | saveRDS(remotes::dev_package_deps(dependencies = TRUE), ".github/depends.Rds", version = 2) 48 | writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), ".github/R-version") 49 | shell: Rscript {0} 50 | 51 | - name: Cache R packages 52 | if: runner.os != 'Windows' 53 | uses: actions/cache@v2 54 | with: 55 | path: ${{ env.R_LIBS_USER }} 56 | key: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-${{ hashFiles('.github/depends.Rds') }} 57 | restore-keys: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1- 58 | 59 | - name: Install system dependencies 60 | if: runner.os == 'Linux' 61 | run: | 62 | while read -r cmd 63 | do 64 | eval sudo $cmd 65 | done < <(Rscript -e 'writeLines(remotes::system_requirements("ubuntu", "20.04"))') 66 | 67 | - name: Install dependencies 68 | run: | 69 | remotes::install_deps(dependencies = TRUE) 70 | remotes::install_cran("rcmdcheck") 71 | shell: Rscript {0} 72 | 73 | - name: Check 74 | env: 75 | _R_CHECK_CRAN_INCOMING_REMOTE_: false 76 | run: rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check") 77 | shell: Rscript {0} 78 | 79 | - name: Upload check results 80 | if: failure() 81 | uses: actions/upload-artifact@main 82 | with: 83 | name: ${{ runner.os }}-r${{ matrix.config.r }}-results 84 | path: check 85 | -------------------------------------------------------------------------------- /vignettes/Introduction.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Introduction" 3 | output: rmarkdown::html_vignette 4 | vignette: > 5 | %\VignetteIndexEntry{Introduction} 6 | %\VignetteEngine{knitr::rmarkdown} 7 | %\VignetteEncoding{UTF-8} 8 | --- 9 | 10 | ```{r, include = FALSE} 11 | knitr::opts_chunk$set( 12 | collapse = TRUE, 13 | comment = "#>" 14 | ) 15 | ``` 16 | The French official open data portal offers a huge quantity of information. They also provide a well structured [API](https://doc.data.gouv.fr/api/reference/). The [BARIS](https://github.com/feddelegrand7/BARIS) package allows you to exploit this API in order to get the required data from the portal. 17 | 18 | > Within the portal there is the concept of a data set which contains one or several data frames or resources. So, if I use the resource term, you need to apprehend it as the data frame inside a data set. 19 | 20 | 21 | The package is available on CRAN, you can also install the development version from Github: 22 | 23 | 24 | ```{r, eval=FALSE} 25 | 26 | install.packages("BARIS") 27 | 28 | ``` 29 | 30 | 31 | Too much talking, let's dive into a reproducible example. 32 | 33 | 34 | # BARIS_search() 35 | 36 | The `BARIS_search()` function allows you to search for a specified data set. **A quick tip: within your query, use plain Nouns and avoid prepositions and determinants: le, la, de, des, en, à ... and so on** : 37 | 38 | ```{r message=FALSE, warning=FALSE} 39 | library(BARIS) 40 | 41 | BARIS_search(query = "Monuments Historiques Marseille") 42 | 43 | ``` 44 | 45 | Cool we have our data set ... but wait it would be better to get some explanation about it. 46 | 47 | # BARIS_explain() 48 | 49 | The `BARIS_explain()` function provides a description of a data set. The function takes one argument which is the ID of the data set: 50 | 51 | 52 | ```{r} 53 | 54 | BARIS_explain(datasetId = "5cebfa8306e3e77ffdb31ef5") 55 | 56 | 57 | ``` 58 | 59 | 60 | Don't panic if you're not a french speaker. You can always use the great [googleLanguageR](https://CRAN.R-project.org/package=googleLanguageR). 61 | 62 | 63 | Now, it's time to list the resources contained within this data set !!! 64 | 65 | 66 | 67 | # BARIS_resources() 68 | 69 | 70 | The `BARIS_resources` function displays the available resources or data frames within a data set. The function takes as argument the ID of the data set: 71 | 72 | ```{r} 73 | BARIS_resources(datasetId = "5cebfa8306e3e77ffdb31ef5") 74 | 75 | ``` 76 | 77 | 78 | You can see from above that the data set has two resources, a *csv* and a *pdf*. Now, we've reached the interesting part: extracting the data frame that you'll work on ! 79 | 80 | # BARIS_extract() 81 | 82 | Using `BARIS_extract()` you can extract directly into your R session the needed data set. Currently, “only” theses formats are supported: json, csv, xls, xlsx, xml, geojson and shp, nevertheless you can always rely on the url of the resource to download it manually. 83 | 84 | In order to use the function you'll have to specify two arguments: The ID of the resource and its format. 85 | 86 | > You can visually catch the structure difference between the ID of a data set and the ID of a resource. 87 | 88 | 89 | ```{r} 90 | 91 | data <- BARIS_extract(resourceId = "59ea7bba-f38a-4d75-b85f-2d1955050e53", format = "csv") 92 | 93 | head(data) 94 | 95 | ``` 96 | 97 | 98 | End of the vignette. 99 | -------------------------------------------------------------------------------- /R/BARIS_extract.R: -------------------------------------------------------------------------------- 1 | 2 | #' Extracting A Data frame From The French Portal data.gouv API 3 | #' 4 | #' @description This function read directly into R a specific data frame (resource) by the mean of its unique ID. Currently, BARIS_extract() can read directly the following types of file : json, csv, xls, xlsx, xml, geojson, and shp. 5 | #' 6 | #' @param resourceId the unique identifier of the individual data frame (resource) to read into R. Not to confound with the global data sets ID that includes many data frames 7 | #' @param format the format of the required file (json, csv, xls, xlsx, xml, geojson or shp) 8 | #' 9 | #' @return a data frame 10 | #' 11 | #' @examples \donttest{ 12 | #' BARIS_extract("59ea7bba-f38a-4d75-b85f-2d1955050e53", format = "csv")} 13 | #' @export 14 | #' @importFrom magrittr %>% 15 | #' @importFrom dplyr as_tibble 16 | #' @importFrom data.table fread 17 | #' @importFrom downloader download 18 | #' @importFrom sf read_sf 19 | #' @importFrom XML xmlParse 20 | #' @importFrom XML xmlToDataFrame 21 | #' @importFrom httr BROWSE 22 | #' @importFrom rgdal readOGR 23 | #' @importFrom janitor clean_names 24 | #' @importFrom rio import 25 | #' @importFrom jsonlite fromJSON 26 | #' @importFrom utils unzip 27 | #' @importFrom checkmate assert_character 28 | 29 | 30 | 31 | BARIS_extract <- function(resourceId, format) { 32 | 33 | checkmate::assert_character(resourceId) 34 | 35 | checkmate::assert_character(format) 36 | 37 | 38 | basic_url <- "https://www.data.gouv.fr/fr/datasets/r/" 39 | 40 | final_url <- paste(basic_url, resourceId, sep = "") 41 | 42 | response_status <- is_response_successfull(final_url) 43 | 44 | if (!response_status) { 45 | return(NULL) 46 | } 47 | 48 | #### Reading CSV files ############################################################ 49 | 50 | if(format == "csv" || format == "text" || format == "text/csv" | format == "txt"){ 51 | 52 | df_csv <- fread(final_url, stringsAsFactors = F, sep = "auto", encoding = "UTF-8", showProgress = F) 53 | df_csv <- as_tibble(df_csv) 54 | 55 | df_csv <- clean_names(df_csv) 56 | return(df_csv) 57 | 58 | #### Reading Excel files ############################################################ 59 | 60 | 61 | } else if (format == "xls" || format == "xlsx"){ 62 | 63 | 64 | df_xlsx <- import(final_url) 65 | 66 | df_xlsx <- clean_names(df_xlsx) 67 | 68 | return(df_xlsx) 69 | 70 | #### Reading XML files ############################################################ 71 | 72 | 73 | } else if (format == "xml" ){ 74 | 75 | 76 | xml_info <- xmlParse(httr::GET(final_url)) 77 | 78 | xmldf <- xmlToDataFrame(xml_info) 79 | 80 | 81 | return(xmldf) 82 | 83 | #### Reading SHP files ############################################################ 84 | 85 | 86 | } else if (format == "shp" || format == "shapefile"){ 87 | 88 | temp <- tempfile() 89 | temp2 <- tempfile() 90 | 91 | download(final_url, dest=temp, mode="wb") 92 | 93 | unzip(zipfile = temp, exdir = temp2) 94 | 95 | SHP_file <- list.files(temp2, pattern = ".shp$",full.names=TRUE) 96 | 97 | return(read_sf(SHP_file)) 98 | 99 | unlink(c(temp, temp2)) 100 | 101 | 102 | 103 | #### Downloading ZIP file ############################################################ 104 | 105 | 106 | } else if (format == "zip" || format == "html.zip" ){ 107 | 108 | BROWSE(final_url) 109 | 110 | #### Downloading GEOJSON file ######################################################### 111 | 112 | 113 | } else if (format == "geojson"){ 114 | 115 | temp <- tempfile() 116 | temp2 <- tempfile() 117 | 118 | download(final_url, dest=temp, mode="wb") 119 | 120 | unzip(zipfile = temp, exdir = temp2) 121 | 122 | SHP_file <- list.files(temp2, pattern = "geojson$",full.names=TRUE) 123 | 124 | return(readOGR(SHP_file, layer = "OGRGeoJSON")) 125 | 126 | unlink(c(temp, temp2)) 127 | 128 | } else if(format == "json") { 129 | 130 | df_json <- fromJSON(final_url) 131 | 132 | 133 | return(df_json) 134 | 135 | } else { 136 | 137 | stop("The available extraction capabilities are limited to 138 | json, csv, text, xls, xlsx, xml, geojson, and shp files. And for downloading, only PDF and ZIP") 139 | 140 | 141 | } 142 | 143 | 144 | 145 | } 146 | 147 | 148 | 149 | 150 | 151 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | We as members, contributors, and leaders pledge to make participation in our 6 | community a harassment-free experience for everyone, regardless of age, body 7 | size, visible or invisible disability, ethnicity, sex characteristics, gender 8 | identity and expression, level of experience, education, socio-economic status, 9 | nationality, personal appearance, race, religion, or sexual identity and 10 | orientation. 11 | 12 | We pledge to act and interact in ways that contribute to an open, welcoming, 13 | diverse, inclusive, and healthy community. 14 | 15 | ## Our Standards 16 | 17 | Examples of behavior that contributes to a positive environment for our 18 | community include: 19 | 20 | * Demonstrating empathy and kindness toward other people 21 | * Being respectful of differing opinions, viewpoints, and experiences 22 | * Giving and gracefully accepting constructive feedback 23 | * Accepting responsibility and apologizing to those affected by our mistakes, 24 | and learning from the experience 25 | * Focusing on what is best not just for us as individuals, but for the overall 26 | community 27 | 28 | Examples of unacceptable behavior include: 29 | 30 | * The use of sexualized language or imagery, and sexual attention or 31 | advances of any kind 32 | * Trolling, insulting or derogatory comments, and personal or political attacks 33 | * Public or private harassment 34 | * Publishing others' private information, such as a physical or email 35 | address, without their explicit permission 36 | * Other conduct which could reasonably be considered inappropriate in a 37 | professional setting 38 | 39 | ## Enforcement Responsibilities 40 | 41 | Community leaders are responsible for clarifying and enforcing our standards 42 | of acceptable behavior and will take appropriate and fair corrective action in 43 | response to any behavior that they deem inappropriate, threatening, offensive, 44 | or harmful. 45 | 46 | Community leaders have the right and responsibility to remove, edit, or reject 47 | comments, commits, code, wiki edits, issues, and other contributions that are 48 | not aligned to this Code of Conduct, and will communicate reasons for moderation 49 | decisions when appropriate. 50 | 51 | ## Scope 52 | 53 | This Code of Conduct applies within all community spaces, and also applies 54 | when an individual is officially representing the community in public spaces. 55 | Examples of representing our community include using an official e-mail 56 | address, posting via an official social media account, or acting as an appointed 57 | representative at an online or offline event. 58 | 59 | ## Enforcement 60 | 61 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 62 | reported to the community leaders responsible for enforcement at [INSERT CONTACT 63 | METHOD]. All complaints will be reviewed and investigated promptly and fairly. 64 | 65 | All community leaders are obligated to respect the privacy and security of the 66 | reporter of any incident. 67 | 68 | ## Enforcement Guidelines 69 | 70 | Community leaders will follow these Community Impact Guidelines in determining 71 | the consequences for any action they deem in violation of this Code of Conduct: 72 | 73 | ### 1. Correction 74 | 75 | **Community Impact**: Use of inappropriate language or other behavior deemed 76 | unprofessional or unwelcome in the community. 77 | 78 | **Consequence**: A private, written warning from community leaders, providing 79 | clarity around the nature of the violation and an explanation of why the 80 | behavior was inappropriate. A public apology may be requested. 81 | 82 | ### 2. Warning 83 | 84 | **Community Impact**: A violation through a single incident or series of 85 | actions. 86 | 87 | **Consequence**: A warning with consequences for continued behavior. No 88 | interaction with the people involved, including unsolicited interaction with 89 | those enforcing the Code of Conduct, for a specified period of time. This 90 | includes avoiding interactions in community spaces as well as external channels 91 | like social media. Violating these terms may lead to a temporary or permanent 92 | ban. 93 | 94 | ### 3. Temporary Ban 95 | 96 | **Community Impact**: A serious violation of community standards, including 97 | sustained inappropriate behavior. 98 | 99 | **Consequence**: A temporary ban from any sort of interaction or public 100 | communication with the community for a specified period of time. No public or 101 | private interaction with the people involved, including unsolicited interaction 102 | with those enforcing the Code of Conduct, is allowed during this period. 103 | Violating these terms may lead to a permanent ban. 104 | 105 | ### 4. Permanent Ban 106 | 107 | **Community Impact**: Demonstrating a pattern of violation of community 108 | standards, including sustained inappropriate behavior, harassment of an 109 | individual, or aggression toward or disparagement of classes of individuals. 110 | 111 | **Consequence**: A permanent ban from any sort of public interaction within the 112 | community. 113 | 114 | ## Attribution 115 | 116 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 117 | version 2.0, 118 | available at https://www.contributor-covenant.org/version/2/0/ 119 | code_of_conduct.html. 120 | 121 | Community Impact Guidelines were inspired by [Mozilla's code of conduct 122 | enforcement ladder](https://github.com/mozilla/diversity). 123 | 124 | [homepage]: https://www.contributor-covenant.org 125 | 126 | For answers to common questions about this code of conduct, see the FAQ at 127 | https://www.contributor-covenant.org/faq. Translations are available at https:// 128 | www.contributor-covenant.org/translations. 129 | -------------------------------------------------------------------------------- /R/BARIS_ui.R: -------------------------------------------------------------------------------- 1 | BARIS_ui <- function() { 2 | 3 | 4 | ui <- miniUI::miniPage(shiny::tags$style(".fa-lemon {color:gold}"), 5 | shiny::tags$style("body{background-color:#111111 ; color:#F4F4F6"), 6 | 7 | 8 | 9 | 10 | miniUI::miniTabstripPanel( 11 | # home tab ---------------------------------------------------------------- 12 | 13 | miniUI::miniTabPanel( 14 | "Home", 15 | icon = shiny::icon("lemon"), 16 | miniUI::gadgetTitleBar( 17 | "Home data.gouv", 18 | right = miniUI::miniTitleBarButton("done", "Get", primary = T), 19 | left = miniUI::miniTitleBarButton("cancel", "Cancel", primary = T) 20 | ), 21 | 22 | miniUI::miniContentPanel( 23 | 24 | shiny::p("Click on Get to display data.gouv.fr home page data sets"), 25 | 26 | shiny::tableOutput(outputId = "home") 27 | 28 | ) 29 | 30 | ), 31 | 32 | 33 | # searching tab ----------------------------------------------------------- 34 | 35 | 36 | 37 | miniUI::miniTabPanel( 38 | "Searching", 39 | miniUI::gadgetTitleBar( 40 | "Searching", 41 | right = miniUI::miniTitleBarButton("done1", "Get", primary = T), 42 | left = miniUI::miniTitleBarButton("cancel1", "Cancel", primary = T) 43 | ), 44 | miniUI::miniContentPanel( 45 | shiny::p("Queries must be written in French, click on Get the first time, after that just retype another query"), 46 | shiny::textInput(inputId = "search_input", label = "What are you looking for ? "), 47 | shiny::tableOutput(outputId = "search_output") 48 | 49 | ) 50 | ), 51 | 52 | 53 | # description tab --------------------------------------------------------- 54 | 55 | miniUI::miniTabPanel( 56 | "Description", 57 | miniUI::gadgetTitleBar( 58 | "Description", 59 | right = miniUI::miniTitleBarButton("done2", "Get", primary = T), 60 | left = miniUI::miniTitleBarButton("cancel2", "Cancel", primary = T) 61 | ), 62 | miniUI::miniContentPanel( 63 | shiny::p("Paste the ID of the data set to get a description, click on Get"), 64 | shiny::textInput(inputId = "description_input", label = "The ID of a Data Set"), 65 | shiny::textOutput(outputId = "description_output") 66 | 67 | ) 68 | ), 69 | 70 | 71 | 72 | 73 | # resource tab ------------------------------------------------------------ 74 | 75 | miniUI::miniTabPanel( 76 | "Resources", 77 | miniUI::gadgetTitleBar( 78 | "Resources", 79 | right = miniUI::miniTitleBarButton("done3", "Get", primary = T), 80 | left = miniUI::miniTitleBarButton("cancel3", "Cancel", primary = T) 81 | ), 82 | miniUI::miniContentPanel( 83 | shiny::p("Paste the ID of the data set to list the resources contained in it"), 84 | shiny::textInput(inputId = "resource_input", label = "The ID of a Data Set"), 85 | shiny::tableOutput(outputId = "resource_output") 86 | 87 | ) 88 | ), 89 | 90 | 91 | 92 | 93 | # extract tab ------------------------------------------------------------- 94 | 95 | 96 | miniUI::miniTabPanel( 97 | "Extract", 98 | miniUI::gadgetTitleBar( 99 | "Extracting", 100 | right = miniUI::miniTitleBarButton("done4", "Get", primary = T), 101 | left = miniUI::miniTitleBarButton("cancel4", "Cancel", primary = T) 102 | ), 103 | miniUI::miniContentPanel( 104 | shiny::textInput(inputId = "resource_id", label = "The ID the resource"), 105 | shiny::textInput(inputId = "resource_format", label = "The format of the resource (in lower case)"), 106 | shiny::textInput(inputId = "resource_name", label = "Give it a name"), 107 | 108 | shiny::p("Run the following code in your R session"), 109 | 110 | shiny::verbatimTextOutput(outputId = "extract_output", placeholder = T) 111 | 112 | ) 113 | ) 114 | 115 | )) 116 | 117 | 118 | # server ------------------------------------------------------------------ 119 | 120 | 121 | 122 | server <- function(input, output) { 123 | # home output ------------------------------------------------------------- 124 | 125 | shiny::observeEvent(input$done, { 126 | output$home <- shiny::renderTable(BARIS::BARIS_home()) 127 | 128 | 129 | 130 | }) 131 | 132 | 133 | 134 | # search output ----------------------------------------------------------- 135 | 136 | 137 | toListen <- shiny::reactive({ 138 | input$done1 139 | }) 140 | 141 | shiny::observeEvent(toListen(), { 142 | output$search_output <- shiny::renderTable(BARIS::BARIS_search(query = input$search_input)) 143 | 144 | 145 | 146 | }) 147 | 148 | 149 | 150 | # description output ------------------------------------------------------ 151 | 152 | description <- shiny::reactive({ 153 | input$done2 154 | }) 155 | 156 | shiny::observeEvent(description(), { 157 | output$description_output <- shiny::renderPrint(BARIS::BARIS_explain(datasetId = input$description_input)) 158 | 159 | 160 | 161 | }) 162 | 163 | 164 | 165 | 166 | # resource output --------------------------------------------------------- 167 | 168 | resource <- shiny::reactive({ 169 | input$done3 170 | }) 171 | 172 | 173 | shiny::observeEvent(resource(), { 174 | output$resource_output <- shiny::renderTable(BARIS::BARIS_resources(datasetId = input$resource_input)) 175 | 176 | 177 | 178 | }) 179 | 180 | 181 | 182 | # extract output ---------------------------------------------------------- 183 | 184 | 185 | extract <- shiny::reactive({ 186 | input$done4 187 | }) 188 | 189 | 190 | shiny::observeEvent(extract(), { 191 | 192 | output$extract_output <- shiny::renderText( 193 | 194 | paste(input$resource_name, " <- ", "BARIS::BARIS_extract(", "'",input$resource_id,"'" , " , ","'",input$resource_format,"'", " )", sep = "") 195 | 196 | ) 197 | 198 | 199 | 200 | }) 201 | 202 | 203 | 204 | 205 | 206 | # cancel ------------------------------------------------------------------ 207 | 208 | 209 | 210 | shiny::observeEvent(input$cancel, { 211 | shiny::stopApp(stop(""), call = FALSE) 212 | 213 | 214 | }) 215 | 216 | 217 | shiny::observeEvent(input$cancel1, { 218 | shiny::stopApp(stop(""), call = FALSE) 219 | 220 | 221 | }) 222 | 223 | 224 | shiny::observeEvent(input$cancel2, { 225 | shiny::stopApp(stop(""), call = FALSE) 226 | 227 | 228 | }) 229 | 230 | 231 | shiny::observeEvent(input$cancel3, { 232 | shiny::stopApp(stop(""), call = FALSE) 233 | 234 | 235 | }) 236 | 237 | shiny::observeEvent(input$cancel4, { 238 | shiny::stopApp(stop(""), call = FALSE) 239 | 240 | 241 | }) 242 | 243 | } 244 | 245 | shiny::runGadget(ui, server, viewer = shiny::paneViewer()) 246 | 247 | 248 | } 249 | 250 | 251 | 252 | -------------------------------------------------------------------------------- /README.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | output: github_document 3 | --- 4 | 5 | 6 | 7 | ```{r, include = FALSE} 8 | knitr::opts_chunk$set( 9 | collapse = TRUE, 10 | comment = "#>", 11 | fig.path = "man/figures/README-", 12 | out.width = "100%" 13 | ) 14 | ``` 15 | 16 | # BARIS 17 | 18 | 19 | [![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/BARIS)](https://cran.r-project.org/package=BARIS) 20 | [![CRAN_time_from_release](https://www.r-pkg.org/badges/ago/BARIS)](https://cran.r-project.org/package=BARIS) 21 | [![CRAN_latest_release_date](https://www.r-pkg.org/badges/last-release/BARIS)](https://cran.r-project.org/package=BARIS) 22 | [![metacran downloads](https://cranlogs.r-pkg.org/badges/BARIS)](https://cran.r-project.org/package=BARIS) 23 | [![metacran downloads](https://cranlogs.r-pkg.org/badges/grand-total/BARIS)](https://cran.r-project.org/package=BARIS) 24 | [![license](https://img.shields.io/github/license/mashape/apistatus.svg)](https://choosealicense.com/licenses/mit/) 25 | [![R badge](https://img.shields.io/badge/Build%20with-♥%20and%20R-blue)](https://github.com/feddelegrand7/BARIS) 26 | [![R build status](https://github.com/feddelegrand7/BARIS/workflows/R-CMD-check/badge.svg)](https://github.com/feddelegrand7/BARIS/actions) 27 | 28 | 29 | 30 | 31 | With **BARIS** you can interact with the [French Official Open Data Portal](https://www.data.gouv.fr/fr/) API directly from R. The package offers several capabilities, from listing the available data sets to extracting the needed resources. Nevertheless, there are many features offered by the API (e.g. uploading a data set, removing a resource ... among others) that are not covered within the **BARIS** package which instead focus on the data extraction aspect of the API. The good news is that the user doesn't need an API key or any credential to run the available functions provided by **BARIS**. Finally, in order to fully apprehend the package, a distinction has to be made. The data.gouv API provides several **data sets which contain one or many data frames.** The unique identifier (ID) of a data set has this form : **53699934a3a729239d2051a1** while the ID of an individual data frame or resource has this form: **59ea7bba-f38a-4d75-b85f-2d1955050e53**. 32 | 33 | ## Installation 34 | 35 | You can install the BARIS package from 36 | [CRAN](https://cran.r-project.org/) with: 37 | 38 | ```{r eval=FALSE, message=FALSE, warning=FALSE} 39 | 40 | install.packages("BARIS") 41 | 42 | ``` 43 | 44 | 45 | You can also install the development version from [GitHub](https://github.com/) with: 46 | 47 | 48 | ```{r eval=FALSE, message=FALSE, warning=FALSE} 49 | 50 | devtools::install_github("feddelegrand7/BARIS") 51 | 52 | ``` 53 | 54 | ## BARIS_home() 55 | 56 | Using the function `BARIS_home()` you can list the displayed datasets within the [home page of the data.gouv website](https://www.data.gouv.fr/fr/). The function doesn't take any argument. It will return a data frame with many useful information about the data set. 57 | 58 | ```{r message=FALSE, warning=FALSE} 59 | library(BARIS) 60 | 61 | BARIS_home() 62 | 63 | ``` 64 | The data is quite condensed so you should use the `View()` or `DT::datatable()` functions. 65 | 66 | 67 | ## BARIS_search() 68 | 69 | The `BARIS_search()` function allows you to search for a specific data set. Suppose we're curious about the city of Marseille. 70 | 71 | 72 | ```{r message=FALSE, warning=FALSE} 73 | 74 | Marseille_data <- BARIS_search(query = "Marseille", page_number = 1, page_size = 20) 75 | 76 | Marseille_data[, c("id", "title")] 77 | 78 | ``` 79 | 80 | 81 | Suppose we're interested in the dataset entitled **Marseille - Monuments historiques** with its corresponding ID: 82 | **5cebfa8306e3e77ffdb31ef5** and we want to know more about this data. In this case, the `BARIS_explain()` function can be useful. 83 | 84 | 85 | ## BARIS_explain() 86 | 87 | `BARIS_explain()` returns a description of a dataset. It has one argument which is the ID of the dataset of interest. 88 | 89 | ```{r message=FALSE, warning=FALSE} 90 | 91 | BARIS_explain("5cebfa8306e3e77ffdb31ef5") 92 | 93 | ``` 94 | 95 | The description is in French but even non-French speakers can use this function in conjunction with a translation tool, for the example the [googleLanguageR package]( https://CRAN.R-project.org/package=googleLanguageR). 96 | 97 | 98 | ## BARIS_resources() 99 | 100 | As mentioned previously, each data set contains one or several data frames or as the API call them **resources**. The `BARIS_resources()` function allows you to list all the available resources within a determined data set. 101 | 102 | 103 | ```{r message=FALSE, warning=FALSE} 104 | 105 | BARIS_resources("5cebfa8306e3e77ffdb31ef5") # The "Marseille - Monuments historiques" ID 106 | 107 | 108 | ``` 109 | 110 | 111 | Many useful information related to the resource are provided: The id, the title, the format, the date of publication, the url of the resource, and a description. 112 | 113 | 114 | ## BARIS_extract() 115 | 116 | The `BARIS_extract()` function allows you to extract the needed resource into your R session. You have to specify the id of the resource and its format. Currently, "only" theses formats are supported: json, csv, xls, xlsx, xml, geojson and shp, nevertheless you can always rely on the url of the resource to download whatever you need. 117 | 118 | As an example, let us extract the above listed csv file: *MARSEILLE_MONUMENTS_HISTORIQUES_2018.csv*: 119 | 120 | 121 | ```{r message=FALSE, warning=FALSE} 122 | 123 | BARIS_extract(resourceId = "59ea7bba-f38a-4d75-b85f-2d1955050e53", format = "csv") 124 | 125 | ``` 126 | 127 | 128 | # BARIS Addin 129 | 130 | BARIS comes with an integrated Addin that generates a Shiny widget allowing the user to interact with the package in an interactive manner. You can trigger the addin from the `Addins` menu in `RStudio` or you can run: 131 | 132 | 133 | ```{r,eval=FALSE} 134 | 135 | BARIS:::BARIS_ui() 136 | 137 | ``` 138 | 139 | 140 | ![](man/figures/gif_baris.gif) 141 | 142 | 143 | 144 | # Citation 145 | 146 | If you use the BARIS package for your work, research or teaching, I'd appreciate if you could cite it as follows: 147 | 148 | Mohamed El Fodil Ihaddaden (2020). BARIS: Access and 149 | Import Data from the French Open Data Portal. R package 150 | version 1.1.1. https://CRAN.R-project.org/package=BARIS 151 | 152 | A BibTeX entry for LaTeX users is 153 | 154 | @Manual{, 155 | title = {BARIS: Access and Import Data from the French Open Data Portal}, 156 | author = {Mohamed El Fodil Ihaddaden}, 157 | year = {2020}, 158 | note = {R package version 1.1.1}, 159 | url = {https://CRAN.R-project.org/package=BARIS}, 160 | } 161 | 162 | 163 | ## Code of Conduct 164 | 165 | Please note that the BARIS project is released with a [Contributor Code of Conduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms. Finally, I appreciate any feedback, feel free to reach out at [moh_fodil](https://twitter.com/moh_fodil) or open an issue on [Github](https://github.com/feddelegrand7/BARIS/issues). 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | # BARIS 5 | 6 | 7 | 8 | [![CRAN\_Status\_Badge](https://www.r-pkg.org/badges/version/BARIS)](https://cran.r-project.org/package=BARIS) 9 | [![CRAN\_time\_from\_release](https://www.r-pkg.org/badges/ago/BARIS)](https://cran.r-project.org/package=BARIS) 10 | [![CRAN\_latest\_release\_date](https://www.r-pkg.org/badges/last-release/BARIS)](https://cran.r-project.org/package=BARIS) 11 | [![metacran 12 | downloads](https://cranlogs.r-pkg.org/badges/BARIS)](https://cran.r-project.org/package=BARIS) 13 | [![metacran 14 | downloads](https://cranlogs.r-pkg.org/badges/grand-total/BARIS)](https://cran.r-project.org/package=BARIS) 15 | [![license](https://img.shields.io/github/license/mashape/apistatus.svg)](https://choosealicense.com/licenses/mit/) 16 | [![R 17 | badge](https://img.shields.io/badge/Build%20with-♥%20and%20R-blue)](https://github.com/feddelegrand7/BARIS) 18 | [![R build 19 | status](https://github.com/feddelegrand7/BARIS/workflows/R-CMD-check/badge.svg)](https://github.com/feddelegrand7/BARIS/actions) 20 | 21 | 22 | With **BARIS** you can interact with the [French Official Open Data 23 | Portal](https://www.data.gouv.fr/fr/) API directly from R. The package 24 | offers several capabilities, from listing the available data sets to 25 | extracting the needed resources. Nevertheless, there are many features 26 | offered by the API (e.g. uploading a data set, removing a resource … 27 | among others) that are not covered within the **BARIS** package which 28 | instead focus on the data extraction aspect of the API. The good news is 29 | that the user doesn’t need an API key or any credential to run the 30 | available functions provided by **BARIS**. Finally, in order to fully 31 | apprehend the package, a distinction has to be made. The data.gouv API 32 | provides several **data sets which contain one or many data frames.** 33 | The unique identifier (ID) of a data set has this form : 34 | **53699934a3a729239d2051a1** while the ID of an individual data frame or 35 | resource has this form: **59ea7bba-f38a-4d75-b85f-2d1955050e53**. 36 | 37 | ## Installation 38 | 39 | - The package has been archived from CRAN. I'm in 40 | the process of writing a new lightweight wrapper with very few 41 | dependencies. Indeed, BARIS provided functions to read the data directly into 42 | R however I think, the user should have the flexibility to decide how the data 43 | can be read. 44 | 45 | You can install the development version from 46 | [GitHub](https://github.com/) with: 47 | 48 | ``` r 49 | devtools::install_github("feddelegrand7/BARIS") 50 | ``` 51 | 52 | ## BARIS\_home() 53 | 54 | Using the function `BARIS_home()` you can list the displayed datasets 55 | within the [home page of the data.gouv 56 | website](https://www.data.gouv.fr/fr/). The function doesn’t take any 57 | argument. It will return a data frame with many useful information about 58 | the data set. 59 | 60 | ``` r 61 | library(BARIS) 62 | 63 | BARIS_home() 64 | #> # A tibble: 9 x 13 65 | #> id title organization page views frequency temporal_cov_st~ 66 | #> 67 | #> 1 6246ac~ Temps de ~ https://static.~ https://~ 47 irregular 2022-03-07 68 | #> 2 623dd1~ Déclarati~ https://static.~ https://~ 30 unknown 69 | #> 3 541014~ Prix des ~ https://static.~ https://~ 16053 unknown 70 | #> 4 61f3ba~ Transitio~ https://static.~ https://~ 11 punctual 71 | #> 5 620bc1~ Liste des~ https://static.~ https://~ 1 punctual 2023-12-31 72 | #> 6 6151bd~ Feuilles ~ https://static.~ https://~ 1481 annual 73 | #> 7 5ff5e6~ Baromètre~ https://static.~ https://~ 3445 punctual 74 | #> 8 60190d~ Synthèse ~ https://static.~ https://~ 42012 daily 75 | #> 9 61a73d~ Index Ega~ https://static.~ https://~ 132 daily 76 | #> # ... with 6 more variables: temporal_cov_end , created_at , 77 | #> # last_modified , last_update , archived , deleted 78 | ``` 79 | 80 | The data is quite condensed so you should use the `View()` or 81 | `DT::datatable()` functions. 82 | 83 | ## BARIS\_search() 84 | 85 | The `BARIS_search()` function allows you to search for a specific data 86 | set. Suppose we’re curious about the city of Marseille. 87 | 88 | ``` r 89 | Marseille_data <- BARIS_search(query = "Marseille", page_number = 1, page_size = 20) 90 | 91 | Marseille_data[, c("id", "title")] 92 | #> # A tibble: 20 x 2 93 | #> id title 94 | #> 95 | #> 1 5cebfa839ce2e76116c3af0d Marseille - Élus 96 | #> 2 5cebfa8506e3e77ffdb31ef8 Marseille - Cimetières 97 | #> 3 5cebfa869ce2e76116c3af11 Marseille - Délibérations 98 | #> 4 5cebfa869ce2e764aac3af2b Marseille - Subventions 99 | #> 5 5cebfa8706e3e77c78b31f0a Marseille - Crèches 100 | #> 6 5f654cd6e39b90e17b8021e3 Marseille - Photographies 101 | #> 7 604ff6a2b0a782adba71fa22 Marseille - Biodiversité 102 | #> 8 604ff6ddbc5a324a0c7ce5a7 Marseille - Eclairage 103 | #> 9 53698f14a3a729239d203643 Arrondissements de Marseille 104 | #> 10 5878ee29a3a7291485cac7b2 Arrondissements de Marseille 105 | #> 11 5878ee75a3a7291484cac81f Quartiers de Marseille 106 | #> 12 5cebfa8206e3e77ffdb31ef4 Marseille - Élections municipales 107 | #> 13 5cebfa829ce2e76116c3af0c Marseille - Élections européennes 108 | #> 14 5cebfa8306e3e77c78b31f05 Marseille - Écoles élémentaires 109 | #> 15 5cebfa8306e3e77ffdb31ef5 Marseille - Monuments historiques 110 | #> 16 5cebfa8306e3e77ffdb31ef6 Marseille - Compte administratif 111 | #> 17 5cebfa839ce2e76116c3af0e Marseille - Wifi public 112 | #> 18 5cebfa839ce2e764aac3af27 Marseille - Marchés forains 113 | #> 19 5cebfa8406e3e77ffdb31ef7 Marseille - Équipements sportifs 114 | #> 20 5cebfa849ce2e764aac3af28 Marseille - Écoles maternelles 115 | ``` 116 | 117 | Suppose we’re interested in the dataset entitled **Marseille - Monuments 118 | historiques** with its corresponding ID: **5cebfa8306e3e77ffdb31ef5** 119 | and we want to know more about this data. In this case, the 120 | `BARIS_explain()` function can be useful. 121 | 122 | ## BARIS\_explain() 123 | 124 | `BARIS_explain()` returns a description of a dataset. It has one 125 | argument which is the ID of the dataset of interest. 126 | 127 | ``` r 128 | BARIS_explain("5cebfa8306e3e77ffdb31ef5") 129 | #> [1] "Monuments historiques situés sur le territoire de Marseille, avec adresse, numéro de base Mérimée (base de données du Ministère de la Culture recensant les monuments historiques de toute la France) et points de géolocalisation" 130 | ``` 131 | 132 | The description is in French but even non-French speakers can use this 133 | function in conjunction with a translation tool, for the example the 134 | [googleLanguageR 135 | package](https://CRAN.R-project.org/package=googleLanguageR). 136 | 137 | ## BARIS\_resources() 138 | 139 | As mentioned previously, each data set contains one or several data 140 | frames or as the API call them **resources**. The `BARIS_resources()` 141 | function allows you to list all the available resources within a 142 | determined data set. 143 | 144 | ``` r 145 | BARIS_resources("5cebfa8306e3e77ffdb31ef5") # The "Marseille - Monuments historiques" ID 146 | #> # A tibble: 2 x 6 147 | #> id title format published url description 148 | #> 149 | #> 1 59ea7bba-~ MARSEILLE_~ csv 2019-05-27~ https://trouve~ Monuments historiqu~ 150 | #> 2 6328f8b3-~ Plan des M~ pdf 2019-05-27~ https://trouve~ Edition Janvier 2013 151 | ``` 152 | 153 | Many useful information related to the resource are provided: The id, 154 | the title, the format, the date of publication, the url of the resource, 155 | and a description. 156 | 157 | ## BARIS\_extract() 158 | 159 | The `BARIS_extract()` function allows you to extract the needed resource 160 | into your R session. You have to specify the id of the resource and its 161 | format. Currently, “only” theses formats are supported: json, csv, xls, 162 | xlsx, xml, geojson and shp, nevertheless you can always rely on the url 163 | of the resource to download whatever you need. 164 | 165 | As an example, let us extract the above listed csv file: 166 | *MARSEILLE\_MONUMENTS\_HISTORIQUES\_2018.csv*: 167 | 168 | ``` r 169 | BARIS_extract(resourceId = "59ea7bba-f38a-4d75-b85f-2d1955050e53", format = "csv") 170 | #> # A tibble: 80 x 10 171 | #> n_base_merimee date_de_protection_~ denomination adresse code_postal 172 | #> 173 | #> 1 PA00081336 Classement : liste ~ Ancienne église de~ "/" 13002 174 | #> 2 PA00081340 Classement: 13/09/1~ Eglise Saint-Laure~ "Esplana~ 13002 175 | #> 3 PA00081331 Classement: 29/01/1~ Chapelle et Hospic~ "2, Rue ~ 13002 176 | #> 4 PA00081344 Classement: 16/06/1~ Fort Saint-Jean "" 13002 177 | #> 5 PA00081325 Inscription : 23/11~ Les deux bâtiments~ "Quai du~ 13002 178 | #> 6 PA00081334 Inscription : 07/07~ Clocher des Accoul~ "Montée ~ 13002 179 | #> 7 PA00081348 Classement: 12/01/1~ Hôtel Daviel (anci~ "Place D~ 13002 180 | #> 8 PA00081363 Classement: 02/05/1~ Maison dite de l’É~ "27, Gra~ 13002 181 | #> 9 PA00081349 Inscription : 14/10~ Hôtel-Dieu- Façade~ "6, Plac~ 13002 182 | #> 10 PA00081354 Classement: 30/04/1~ Hôtel de Ville "Quai du~ 13002 183 | #> # ... with 70 more rows, and 5 more variables: proprietaire_du_monument , 184 | #> # epoque_de_construction , date_de_construction , longitude , 185 | #> # latitude 186 | ``` 187 | 188 | # BARIS Addin 189 | 190 | BARIS comes with an integrated Addin that generates a Shiny widget 191 | allowing the user to interact with the package in an interactive manner. 192 | You can trigger the addin from the `Addins` menu in `RStudio` or you can 193 | run: 194 | 195 | ``` r 196 | BARIS:::BARIS_ui() 197 | ``` 198 | 199 | ![](man/figures/gif_baris.gif) 200 | 201 | # Citation 202 | 203 | If you use the BARIS package for your work, research or teaching, I’d 204 | appreciate if you could cite it as follows: 205 | 206 | Mohamed El Fodil Ihaddaden (2020). BARIS: Access and Import Data from 207 | the French Open Data Portal. R package version 1.1.1. 208 | 209 | 210 | A BibTeX entry for LaTeX users is 211 | 212 | @Manual{, title = {BARIS: Access and Import Data from the French Open 213 | Data Portal}, author = {Mohamed El Fodil Ihaddaden}, year = {2020}, note 214 | = {R package version 1.1.1}, url = 215 | {}, } 216 | 217 | ## Code of Conduct 218 | 219 | Please note that the BARIS project is released with a [Contributor Code 220 | of 221 | Conduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html). 222 | By contributing to this project, you agree to abide by its terms. 223 | Finally, I appreciate any feedback, feel free to reach out at 224 | [moh\_fodil](https://twitter.com/moh_fodil) or open an issue on 225 | [Github](https://github.com/feddelegrand7/BARIS/issues). 226 | --------------------------------------------------------------------------------