├── .Rbuildignore ├── .gitignore ├── .travis.yml ├── DESCRIPTION ├── NAMESPACE ├── R └── data.R ├── README.Rmd ├── README.md ├── README_files └── figure-markdown_github │ └── unnamed-chunk-2-1.png ├── cran-comments.md ├── data-raw └── get_data.R ├── data └── csp.rda ├── man └── csp.Rd ├── statepolicycorrelates.Rproj ├── tests ├── testthat.R └── testthat │ └── test-version.R └── vignettes └── csp.Rmd /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.*\.Rproj$ 2 | ^\.Rproj\.user$ 3 | ^data-raw$ 4 | ^README\.Rmd$ 5 | ^README\.md$ 6 | ^README-.*\.png$ 7 | ^README_files$ 8 | ^cran-comments\.md$ 9 | ^\.travis\.yml$ 10 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .Rproj.user 2 | .Rhistory 3 | .RData 4 | .Ruserdata 5 | inst/doc 6 | /data-raw/*.dta 7 | /data-raw/*.docx -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | # R for travis: see documentation at https://docs.travis-ci.com/user/languages/r 2 | 3 | language: R 4 | sudo: false 5 | cache: packages 6 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: csp 2 | Type: Package 3 | Title: Correlates of State Policy Data Set 4 | Version: 0.4.0 5 | Date: 2017-09-19 6 | Authors@R: person("Eric", "Persson", email = "expersso5@gmail.com", 7 | role = c("aut", "cre")) 8 | Description: Provides the Correlates of State Policy data set, which includes 9 | more than seven-hundred variables, with observations across the U.S. 50 10 | states and time (1900 - 2016). These variables represent policy outputs or 11 | political, social, or economic factors that may influence policy differences 12 | across the states. 13 | License: CC0 14 | LazyData: FALSE 15 | URL: http://ippsr.msu.edu/public-policy/correlates-state-policy 16 | https://github.com/expersso/csp 17 | Suggests: 18 | knitr, 19 | rmarkdown, 20 | ggplot2, 21 | testthat, 22 | rvest 23 | VignetteBuilder: knitr 24 | Depends: R (>= 2.10) 25 | RoxygenNote: 6.0.1 26 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2: do not edit by hand 2 | 3 | -------------------------------------------------------------------------------- /R/data.R: -------------------------------------------------------------------------------- 1 | #' Correlates of State Policy Dataset 2 | #' 3 | #' This package contains the Correlates of State Policy dataset, which includes 4 | #' more than seven-hundred variables, with observations across the U.S. 50 5 | #' states and time (1900 - 2016). 6 | #' 7 | #' For more information, go to the 8 | #' \href{http://ippsr.msu.edu/public-policy/correlates-state-policy}{official website}. 9 | #' @examples 10 | #' \dontrun{ 11 | #' library(csp) 12 | #' data(csp, package = "csp") 13 | #' dim(csp) 14 | #' names(csp) 15 | #' csp[1:2, ]} 16 | "csp" -------------------------------------------------------------------------------- /README.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | output: 3 | md_document: 4 | variant: markdown_github 5 | --- 6 | 7 | ```{r setup, echo = FALSE} 8 | knitr::opts_chunk$set(eval = FALSE) 9 | ``` 10 | 11 | ## Introduction 12 | 13 | Since the Correlates of State Policy project now provides their data in an 14 | easy-to-use CSV file, this package is deprecated. To read the data into R, just 15 | use: 16 | 17 | ```{r} 18 | uri <- "http://ippsr.msu.edu/sites/default/files/correlatesofstatepolicyprojectv2_1.csv" 19 | csp <- read.csv(uri) 20 | str(csp) 21 | ``` 22 | 23 | See [their website](http://ippsr.msu.edu/public-policy/correlates-state-policy) 24 | for up-to-date information. 25 | 26 | This package contains the [Correlates of State 27 | Policy](http://ippsr.msu.edu/public-policy/correlates-state-policy) 28 | dataset version 1.11, which 29 | 30 | > ... includes more than seven-hundred variables, with observations across the 31 | > U.S. 50 > states and time (1900 – 2016). These variables represent policy 32 | > outputs or > political, social, or economic factors that may influence policy 33 | > differences > across the states. The codebook includes the variable name, a 34 | > short description > of the variable, the variable time frame, a longer 35 | > description of the variable, > and the variable source(s) and notes. 36 | 37 | Suggested citation: 38 | 39 | > Jordan, Marty P. and Matt Grossmann. 2016. The Correlates of State Policy 40 | > Project v.1.0. East Lansing, MI: Institute for Public Policy and Social Research 41 | > (IPPSR). 42 | 43 | The package allows the user to load and work with the dataset using the R 44 | programming language. Crucially, it incorporates the entire codebook into the 45 | dataset, which allows for easier filtering, finding units and sources, etc. 46 | 47 | ## Example use 48 | 49 | ```{r, warning=FALSE, message=FALSE} 50 | library(csp) 51 | library(ggplot2) 52 | 53 | data(csp, package = "csp") 54 | dim(csp) 55 | names(csp) 56 | csp[1:2, ] 57 | ``` 58 | 59 | ```{r, fig.width = 6, fig.height = 4, warning=FALSE, message=FALSE} 60 | df <- subset(csp, variable == "real2_pc_inc_quar") 61 | df$value <- as.numeric(df$value) 62 | 63 | ggplot(df, aes(x = year, y = value, color = state)) + 64 | geom_line(show.legend = FALSE) + 65 | scale_color_grey() + 66 | theme_light() + 67 | labs(x = NULL, y = NULL, 68 | title = df$var_desc[1], 69 | subtitle = df$var_long_desc[1]) 70 | ``` 71 | 72 | ## Disclaimer 73 | 74 | This package is not affiliated with, nor endorsed by, the Correlates of State 75 | Policy Project. All credit go to the original authors, and questions should be 76 | directed to them. Please check the [official website](http://ippsr.msu.edu/public-policy/correlates-state-policy) 77 | for further details on citations, etc. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Introduction 2 | ------------ 3 | 4 | Since the Correlates of State Policy project now provides their data in 5 | an easy-to-use CSV file, this package is deprecated. To read the data 6 | into R, just use: 7 | 8 | ``` r 9 | uri <- "http://ippsr.msu.edu/sites/default/files/correlatesofstatepolicyprojectv2_1.csv" 10 | csp <- read.csv(uri) 11 | str(csp) 12 | ``` 13 | 14 | See [their 15 | website](http://ippsr.msu.edu/public-policy/correlates-state-policy) for 16 | up-to-date information. 17 | 18 | This package contains the [Correlates of State 19 | Policy](http://ippsr.msu.edu/public-policy/correlates-state-policy) 20 | dataset version 1.11, which 21 | 22 | > … includes more than seven-hundred variables, with observations across 23 | > the U.S. 50 > states and time (1900 – 2016). These variables 24 | > represent policy outputs or > political, social, or economic 25 | > factors that may influence policy differences > across the states. 26 | > The codebook includes the variable name, a short description > of 27 | > the variable, the variable time frame, a longer description of the 28 | > variable, > and the variable source(s) and notes. 29 | 30 | Suggested citation: 31 | 32 | > Jordan, Marty P. and Matt Grossmann. 2016. The Correlates of State 33 | > Policy Project v.1.0. East Lansing, MI: Institute for Public Policy 34 | > and Social Research (IPPSR). 35 | 36 | The package allows the user to load and work with the dataset using the 37 | R programming language. Crucially, it incorporates the entire codebook 38 | into the dataset, which allows for easier filtering, finding units and 39 | sources, etc. 40 | 41 | Example use 42 | ----------- 43 | 44 | ``` r 45 | library(csp) 46 | library(ggplot2) 47 | 48 | data(csp, package = "csp") 49 | dim(csp) 50 | names(csp) 51 | csp[1:2, ] 52 | ``` 53 | 54 | ``` r 55 | df <- subset(csp, variable == "real2_pc_inc_quar") 56 | df$value <- as.numeric(df$value) 57 | 58 | ggplot(df, aes(x = year, y = value, color = state)) + 59 | geom_line(show.legend = FALSE) + 60 | scale_color_grey() + 61 | theme_light() + 62 | labs(x = NULL, y = NULL, 63 | title = df$var_desc[1], 64 | subtitle = df$var_long_desc[1]) 65 | ``` 66 | 67 | Disclaimer 68 | ---------- 69 | 70 | This package is not affiliated with, nor endorsed by, the Correlates of 71 | State Policy Project. All credit go to the original authors, and 72 | questions should be directed to them. Please check the [official 73 | website](http://ippsr.msu.edu/public-policy/correlates-state-policy) for 74 | further details on citations, etc. 75 | -------------------------------------------------------------------------------- /README_files/figure-markdown_github/unnamed-chunk-2-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/expersso/csp/558d549f3dc85bf1986e8adbeb856d211a1d1bc8/README_files/figure-markdown_github/unnamed-chunk-2-1.png -------------------------------------------------------------------------------- /cran-comments.md: -------------------------------------------------------------------------------- 1 | ## Test environments 2 | * ubuntu 14.04, R 3.2.2 3 | * Windows 10 4 | * win-builder (devel and release) 5 | 6 | ## R CMD check results 7 | 8 | 1 NOTE: 9 | 10 | installed size is 11.0Mb 11 | sub-directories of 1Mb or more: 12 | data 10.9Mb 13 | 14 | * The package is a pure data package, and the data has been maximally compressed. 15 | 16 | ## Changes in version 0.2 following comments from Profs Hornik and Ripley 17 | 18 | * Wrapped example that was running slowly on CRAN in \dontrun{} 19 | (the package includes a comprehensive vignette explaining usage) 20 | 21 | * Expanded description of what the package does 22 | 23 | * Removed LazyData = TRUE 24 | 25 | * Tested using both 32- and 64-bit R -------------------------------------------------------------------------------- /data-raw/get_data.R: -------------------------------------------------------------------------------- 1 | library(rvest) 2 | library(haven) 3 | library(tidyverse) 4 | library(devtools) 5 | 6 | get_file_url <- function(filetype) { 7 | "http://ippsr.msu.edu/public-policy/correlates-state-policy" %>% 8 | read_html() %>% 9 | html_nodes(xpath = sprintf("//a[contains(@href, '%s')]", filetype)) %>% 10 | html_attr("href") 11 | } 12 | 13 | get_file <- function(url, output_dir = ".") { 14 | file_name <- file.path(output_dir, basename(url)) 15 | download.file(url, file_name, mode = "wb") 16 | invisible(TRUE) 17 | } 18 | 19 | add_names_as_row <- function(df, new_names = NULL, ...) { 20 | new_row <- names(df) %>% set_names() 21 | out <- lift(partial(add_row, .data = df))(new_row, ...) 22 | if(!is.null(new_names)) { 23 | out <- set_names(out, new_names) 24 | } 25 | out 26 | } 27 | 28 | get_title_text <- function(x) { 29 | sprintf("//table[%s]//preceding-sibling::h1[1]", x) 30 | } 31 | 32 | get_codebook_df <- function(codebook_file) { 33 | 34 | tmp <- tempfile(fileext = ".html") 35 | on.exit(unlink(tmp)) 36 | shell(sprintf("pandoc -f docx -t html -o %s %s", tmp, codebook_file)) 37 | 38 | page <- read_html(tmp, "utf-8") 39 | tbls <- html_table(page, TRUE, TRUE, TRUE) 40 | 41 | garbled <- map_lgl(tbls, ~names(.)[1] != "Variable Name") 42 | 43 | tbls[garbled] <- tbls[garbled] %>% 44 | map(add_names_as_row, 45 | new_names = c("Variable Name", "Variable-Short Description", "Dates", 46 | "Variable-Longer Description", "Sources and Notes"), 47 | .before = 1 48 | ) 49 | 50 | tbl_topics <- seq_along(tbls) %>% 51 | map_chr(~page %>% 52 | html_nodes(xpath = get_title_text(.x)) %>% 53 | html_text(TRUE)) 54 | 55 | tbls %>% 56 | map(~mutate_all(., as.character)) %>% 57 | set_names(tbl_topics) %>% 58 | bind_rows(.id = "topic") %>% 59 | set_names(c("topic", "variable", "var_desc", "dates", 60 | "var_long_desc", "sources_notes")) %>% 61 | mutate_all(funs(iconv(., "utf-8", "latin1"))) %>% 62 | filter(!is.na(var_desc), variable != "") 63 | } 64 | 65 | ## IO 66 | data_url <- get_file_url(".dta") 67 | codebook_url <- get_file_url(".docx") 68 | data_file <- file.path("data-raw", basename(data_url)) 69 | codebook_file <- file.path("data-raw", basename(codebook_url)) 70 | 71 | get_file(data_url, "data-raw") 72 | get_file(codebook_url, "data-raw") 73 | 74 | codebook <- get_codebook_df(codebook_file) 75 | csp <- read_stata(data_file) %>% 76 | gather(variable, value, -(year:state_icpsr)) %>% 77 | left_join(codebook) %>% 78 | zap_formats() %>% 79 | mutate(value = as.numeric(value)) 80 | 81 | use_data(csp, overwrite = TRUE) 82 | -------------------------------------------------------------------------------- /data/csp.rda: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/expersso/csp/558d549f3dc85bf1986e8adbeb856d211a1d1bc8/data/csp.rda -------------------------------------------------------------------------------- /man/csp.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/data.R 3 | \docType{data} 4 | \name{csp} 5 | \alias{csp} 6 | \title{Correlates of State Policy Dataset} 7 | \format{An object of class \code{tbl_df} (inherits from \code{tbl}, \code{data.frame}) with 5507541 rows and 13 columns.} 8 | \usage{ 9 | csp 10 | } 11 | \description{ 12 | This package contains the Correlates of State Policy dataset, which includes 13 | more than seven-hundred variables, with observations across the U.S. 50 14 | states and time (1900 - 2016). 15 | } 16 | \details{ 17 | For more information, go to the 18 | \href{http://ippsr.msu.edu/public-policy/correlates-state-policy}{official website}. 19 | } 20 | \examples{ 21 | \dontrun{ 22 | library(csp) 23 | data(csp, package = "csp") 24 | dim(csp) 25 | names(csp) 26 | csp[1:2, ]} 27 | } 28 | \keyword{datasets} 29 | -------------------------------------------------------------------------------- /statepolicycorrelates.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: knitr 13 | LaTeX: pdfLaTeX 14 | 15 | StripTrailingWhitespace: Yes 16 | 17 | BuildType: Package 18 | PackageUseDevtools: Yes 19 | PackageInstallArgs: --with-keep.source 20 | PackageCheckArgs: --as-cran 21 | PackageRoxygenize: rd,collate,namespace 22 | -------------------------------------------------------------------------------- /tests/testthat.R: -------------------------------------------------------------------------------- 1 | library(testthat) 2 | library(csp) 3 | 4 | test_check("csp") 5 | -------------------------------------------------------------------------------- /tests/testthat/test-version.R: -------------------------------------------------------------------------------- 1 | context("version.R") 2 | 3 | # test_that("Current package version is still latest versions", { 4 | # library(rvest) 5 | # 6 | # get_file_url <- function(filetype) { 7 | # "http://ippsr.msu.edu/public-policy/correlates-state-policy" %>% 8 | # read_html() %>% 9 | # html_nodes(xpath = sprintf("//a[contains(@href, '%s')]", filetype)) %>% 10 | # html_attr("href") 11 | # } 12 | # 13 | # expected_filename <- "correlatesofstatepolicyprojectv1_11.dta" 14 | # current_filename <- basename(get_file_url(".dta")) 15 | # expect_equal(expected_filename, current_filename) 16 | # }) 17 | -------------------------------------------------------------------------------- /vignettes/csp.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "csp: Correlates of State Policy Dataset in R" 3 | author: "Eric Persson" 4 | date: "`r Sys.Date()`" 5 | output: rmarkdown::html_vignette 6 | vignette: > 7 | %\VignetteIndexEntry{csp} 8 | %\VignetteEngine{knitr::rmarkdown} 9 | %\VignetteEncoding{UTF-8} 10 | --- 11 | 12 | ## Introduction 13 | 14 | This package contains the [Correlates of State 15 | Policy](http://ippsr.msu.edu/public-policy/correlates-state-policy) 16 | dataset, which 17 | 18 | > ... includes more than seven-hundred variables, with observations across the 19 | > U.S. 50 > states and time (1900 – 2016). These variables represent policy 20 | > outputs or > political, social, or economic factors that may influence policy 21 | > differences > across the states. The codebook includes the variable name, a 22 | > short description > of the variable, the variable time frame, a longer 23 | > description of the variable, > and the variable source(s) and notes. 24 | 25 | Suggested citation: 26 | 27 | > Jordan, Marty P. and Matt Grossmann. 2016. The Correlates of State Policy 28 | > Project v.1.0. East Lansing, MI: Institute for Public Policy and Social Research 29 | > (IPPSR). 30 | 31 | The package allows the user to load and work with the dataset using the R 32 | programming language. Crucially, it incorporates the entire codebook into the 33 | dataset, which allows for easier filtering, finding units and sources, etc. 34 | 35 | ## Example use 36 | 37 | ```{r, warning=FALSE, message=FALSE} 38 | library(csp) 39 | library(ggplot2) 40 | 41 | data(csp, package = "csp") 42 | dim(csp) 43 | names(csp) 44 | csp[1:2, ] 45 | ``` 46 | 47 | ```{r, fig.width = 6, fig.height = 4, warning=FALSE, message=FALSE} 48 | df <- subset(csp, variable == "real2_pc_inc_quar") 49 | df$value <- as.numeric(df$value) 50 | 51 | ggplot(df, aes(x = year, y = value, color = state)) + 52 | geom_line(show.legend = FALSE) + 53 | scale_color_grey() + 54 | theme_light() + 55 | labs(x = NULL, y = NULL, 56 | title = df$var_desc[1], 57 | subtitle = df$var_long_desc[1]) 58 | ``` 59 | 60 | ## Disclaimer 61 | 62 | This package is not affiliated with, nor endorsed by, the Correlates of State 63 | Policy Project. All credit go to the original authors, and questions should be 64 | directed to them. Please check the [official website](http://ippsr.msu.edu/public-policy/correlates-state-policy) 65 | for further details on citations, etc. --------------------------------------------------------------------------------