├── .gitattributes ├── .gitignore ├── 00-Getting-started.Rmd ├── 01-tidyhydat-intro.Rmd ├── 02-Transform.Rmd ├── 03-Dates-Joins.Rmd ├── 04-Visualize.Rmd ├── CODE_OF_CONDUCT.md ├── COMPLIANCE.yaml ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── cheatsheets ├── dplyr-data-transformation.pdf ├── ggplot2-data-visualization.pdf ├── lubridate-dates-times.pdf ├── rstudio-ide.pdf └── tidyr-readr-data-import-tidy.pdf ├── email-to-participants.md ├── intro-to-tidyhydat-and-tidyverse.Rproj └── slides ├── .gitignore ├── 00-Getting-started.pdf ├── 01-tidyhydat-intro.pdf ├── 02-Transform.pdf ├── 03-Dates-Joins.pdf ├── 04-Visualize.pdf └── 05-wrapping-up.pdf /.gitattributes: -------------------------------------------------------------------------------- 1 | * linguist-vendored 2 | *.R linguist-vendored=false 3 | *.R linguist-language=R 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .Rproj.user 2 | .Rhistory 3 | .RData 4 | .Ruserdata 5 | *.html 6 | /tmp 7 | Thumbs.db 8 | -------------------------------------------------------------------------------- /00-Getting-started.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Getting started" 3 | --- 4 | 5 | 9 | 10 | ```{r setup} 11 | library(tidyverse) 12 | library(tidyhydat) 13 | ``` 14 | 15 | ## R notebooks 16 | 17 | This is an [R Markdown](http://rmarkdown.rstudio.com) Notebook. When you execute code within the notebook, the results appear beneath the code. 18 | 19 | R code goes in **code chunks**, denoted by three backticks. Try executing this chunk by clicking the *Run* button within the chunk or by placing your cursor inside it and pressing *Crtl+Shift+Enter* (Windows) or *Cmd+Shift+Enter* (Mac). 20 | 21 | __Your turn__: Run the code in this block. 22 | 23 | ```{r} 24 | fraser_flows <- hy_daily_flows(station_number = "08MF005") 25 | 26 | ggplot(data = fraser_flows) + 27 | geom_point(mapping = aes(x = Date, y = Value)) 28 | ``` 29 | 30 | Add a new chunk by clicking the *Insert* button on the toolbar, then selecting *R* or by pressing *Ctrl+Alt+I* (Windows) or *Cmd+Option+I* (Mac). 31 | 32 | You can __Knit__ a document ((click the *knitr* button or press *Ctrl+Shift+K* (Windows) or *Cmd+Shift+K* (Mac)) to produce a final report that you can share with others. This will re-run all the code and create a self-contained html file can you share with others. 33 | 34 | __Your turn__: Knit this document and find the html file that it generates. 35 | 36 | __Your turn__: You're all done, so please put up a blue post-it! -------------------------------------------------------------------------------- /01-tidyhydat-intro.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "tidyhydat intro" 3 | --- 4 | 5 | 9 | 10 | ```{r setup} 11 | library(tidyverse) 12 | library(tidyhydat) 13 | ``` 14 | 15 | ```{r, eval = FALSE} 16 | help(package = "tidyhydat") 17 | ?hy_daily_levels 18 | ``` 19 | 20 | ## Steps for tidyhydat 21 | ### Your turn 22 | - Try grabbing daily flow data for the following stations: 07EF001, 02HA013 23 | - Give each resulting object a unique name 24 | ```{r} 25 | peace <- hy_daily_flows(station_number = "07EF001") 26 | 27 | niagara <- hy_daily_flows(station_number = "02HA013") 28 | ``` 29 | 30 | - Inspect the results 31 | - In the Environment pane, click `niagara` and `peace` to view each object. `hy_daily_flows()` creates a ***data.frame object*** 32 | 33 | ## Creating vectors 34 | In R, you can use `c()` to *combine* other objects or values. 35 | 36 | ```{r, create_vector} 37 | sams_vector <- c("this","is","my","vector") 38 | 39 | sams_vector2 <- c(1, 4, 5, 6) 40 | ``` 41 | 42 | ### Your turn 43 | Station numbers can be supplied directly to the function like this: 44 | ```{r} 45 | hy_daily_flows(station_number = c("07EF001","02HA013")) 46 | ``` 47 | 48 | Create a vector using `c()` called `my_vector`: 49 | ```{r} 50 | ## Your turn 51 | ``` 52 | 53 | Construct a data.frame object with `hy_daily_flows` that includes any two stations and inspect that dataframe 54 | ```{r} 55 | 56 | ``` 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | -------------------------------------------------------------------------------- /02-Transform.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Transform data" 3 | --- 4 | 5 | 9 | 10 | ```{r packages} 11 | library(tidyverse) 12 | library(tidyhydat) 13 | ``` 14 | 15 | ## hy_stn_data_range() 16 | - When `STATION_NUMBER` and `PROV_TERR_STATE_LOC` is missing all stations in Canada are returned. 17 | 18 | ```{r, eval = FALSE} 19 | ?hy_stn_data_range 20 | ``` 21 | 22 | Create an object from hy_stn_data_range() called data_range which omit all arguments 23 | ```{r} 24 | data_range <- hy_stn_data_range() 25 | ``` 26 | 27 | ## filter() 28 | 29 | ```{r} 30 | filter(data_range, DATA_TYPE == "Q") 31 | ``` 32 | 33 | ### Your turn 34 | 35 | Use the logical operators to show: 36 | 37 | - Stations that begun operation before and including the year 1900 38 | ```{r} 39 | filter(data_range) 40 | ``` 41 | 42 | - Stations that are not discharge stations (i.e. `"Q"`) 43 | ```{r} 44 | filter(data_range) 45 | ``` 46 | 47 | - Stations that have more than 100 years of data 48 | ```{r} 49 | filter(data_range) 50 | ``` 51 | 52 | ### Your turn 53 | 54 | Use Boolean operators to return only the rows that contain: 55 | 56 | - Find stations that are level (`"H"`) and have more than 80 years record 57 | 58 | ```{r} 59 | filter(data_range) 60 | ``` 61 | 62 | - Find stations that start before 1890 or after 2016 63 | 64 | ```{r} 65 | filter(data_range) 66 | ``` 67 | 68 | - These stations in one data.frame 08KA009, 05JA005, 06AC006 (hint look at `%in%`) 69 | 70 | ```{r} 71 | filter(data_range) 72 | ``` 73 | 74 | ### Two common mistakes 75 | 1. Use `=` instead of `==` 76 | ```{r error=TRUE} 77 | filter(data_range, DATA_TYPE = "Q") 78 | filter(data_range, DATA_TYPE == "Q") 79 | ``` 80 | 81 | 2. Forgetting quotes 82 | ```{r error=TRUE} 83 | filter(data_range, DATA_TYPE == Q) 84 | filter(data_range, DATA_TYPE == "Q") 85 | ``` 86 | 87 | 88 | ## pull() 89 | `pull()` is a very simple function but as we will see it will help us feed the station numbers we want to the hydrometric data that we need. 90 | 91 | ### Your turn 92 | ```{r} 93 | ##YEAR_FROM 94 | pull(data_range) 95 | 96 | ##STATION_NUMBER 97 | pull(data_range) 98 | ``` 99 | 100 | ## Pipe `%>%` 101 | 102 | These do the same thing. Try it: 103 | ```{r} 104 | filter(data_range, DATA_TYPE == "Q") 105 | data_range %>% filter(DATA_TYPE == "Q") 106 | ``` 107 | 108 | ## mutate() 109 | ```{r} 110 | hy_monthly_flows("08MF005") %>% mutate(Value_cfs = Value * 0.0167) 111 | ``` 112 | 113 | ```{r} 114 | x <- 1:10 115 | ifelse(x > 5, "large", "small") 116 | ``` 117 | 118 | ### Your turn 119 | ```{r} 120 | hy_monthly_flows("08MF005") %>% 121 | mutate(category = ifelse(Value > 500, "Above 500", "Below 500")) 122 | ``` 123 | 124 | ## summarise() 125 | First we need some data to work with: 126 | ```{r} 127 | annual_gt_100 <- hy_stn_data_range() %>% 128 | filter(RECORD_LENGTH > 100, DATA_TYPE == "Q") %>% 129 | pull(STATION_NUMBER) %>% 130 | hy_annual_stats() 131 | ``` 132 | 133 | First we can see `mean()` with `na.rm = FALSE`, the default 134 | 135 | ```{r} 136 | annual_gt_100 %>% 137 | summarise(mean_value = mean(Value)) 138 | ``` 139 | 140 | Note the difference with when we set `na.rm = TRUE` 141 | ```{r} 142 | annual_gt_100 %>% 143 | summarise(mean_value = mean(Value, na.rm = TRUE)) 144 | ``` 145 | 146 | We can also compute more summaries: 147 | 148 | ```{r} 149 | annual_gt_100 %>% 150 | summarise(mean_value = mean(Value, na.rm = TRUE), 151 | min_value = min(Value, na.rm = TRUE)) 152 | ``` 153 | 154 | ### Your turn 155 | 156 | Use `summarise()` and the `annual_gt_100 data to compute three statistics about the data: 157 | 158 | - The first (minimum) year in the dataset 159 | - The median mean annual flow (Value) in the dataset 160 | - The number of stations represented in the data (Hint: use cheatsheet) 161 | 162 | ```{r} 163 | annual_gt_100 %>% 164 | summarise() 165 | ``` 166 | 167 | 168 | ### Your Turn 169 | 170 | Compute the summaries as before but only the mean annual flow (i.e. where Sum_stat is MEAN) 171 | 172 | ```{r} 173 | 174 | 175 | ``` 176 | 177 | ## group_by() 178 | 179 | ```{r} 180 | annual_gt_100 %>% 181 | group_by(Year) 182 | ``` 183 | 184 | Summarise by group - the number of stations that have a record longer than 100 years grouped by year 185 | 186 | ```{r} 187 | annual_gt_100 %>% 188 | group_by(Year) %>% 189 | summarise(num_stations = n_distinct(STATION_NUMBER)) 190 | ``` 191 | 192 | ### Your Turn 193 | 194 | ```{r} 195 | annual_gt_100 196 | ``` 197 | 198 | ## select 199 | ```{r} 200 | annual_gt_100 %>% 201 | select(STATION_NUMBER, Value) 202 | ``` 203 | 204 | ## arrange 205 | ```{r} 206 | annual_gt_100 %>% 207 | arrange(Value) 208 | ``` 209 | 210 | 211 | ## Challenge! 212 | Extract daily flow information from all active stations on Prince Edward Island. 213 | ```{r} 214 | 215 | ``` 216 | -------------------------------------------------------------------------------- /03-Dates-Joins.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Working with Dates and Joins" 3 | --- 4 | 5 | 9 | 10 | 11 | ```{r setup} 12 | library(tidyverse) 13 | library(lubridate) 14 | library(tidyhydat) 15 | ``` 16 | 17 | # Review 18 | Remember some of the date types we have seen previously: 19 | 20 | ```{r} 21 | fraser_flow <- hy_daily_levels(station_number = "08MF005") 22 | ``` 23 | 24 | # Times and Dates 25 | ## Your Turn 26 | 27 | For each of the following formats (of the same date), pick the right `ymd()` function to parse them: 28 | 29 | ```{r} 30 | "2018 Feb 01" 31 | "2-1-18" 32 | "01/02/2018" 33 | ``` 34 | 35 | ### Your Turn 36 | 37 | Fill in the blanks to for `fraser_flows` data: 38 | 39 | - Extract the month from date. 40 | - Extract the year from date. 41 | - Calculate the mean flow with a measurement for each year/month. 42 | - Filter the results for only the month of June 43 | 44 | 45 | ```{r, error = TRUE} 46 | fraser_flow %>% 47 | mutate(year = _____, 48 | month = _____) %>% 49 | group_by(_____, ____) %>% 50 | summarise(mean_monthly_flow = _____) %>% 51 | filter(____) 52 | ``` 53 | 54 | ### realtime data 55 | ```{r} 56 | mackenzie_realtime <- realtime_dd(station_number = "10LC014") 57 | 58 | mackenzie_realtime %>% 59 | mutate(seconds = second(Date), 60 | minute = minute(Date), 61 | hour = hour(Date)) %>% 62 | filter(hour == 7) 63 | ``` 64 | 65 | or this way 66 | 67 | ```{r} 68 | mackenzie_realtime %>% 69 | filter(hour(Date) == 7) 70 | ``` 71 | 72 | ## filter() and dates 73 | ```{r} 74 | "01-01-1950" 75 | 76 | dmy("01-01-1950") 77 | ``` 78 | 79 | 80 | ```{r} 81 | fraser_flow %>% 82 | filter(Date >= "01-01-1950") %>% 83 | summarise(min_date = min(Date)) 84 | ``` 85 | 86 | ```{r} 87 | fraser_flow %>% 88 | filter(Date >= dmy("01-01-1950"))%>% 89 | summarise(min_date = min(Date)) 90 | ``` 91 | 92 | 93 | ## Challenge 94 | 95 | Get the province/territory station codes like this: 96 | ```{r} 97 | unique(allstations$PROV_TERR_STATE_LOC) 98 | ``` 99 | 100 | Try using the `hy_stations` function for station metadata: 101 | ```{r} 102 | hy_stations(prov_terr_state_loc = "BC") 103 | ``` 104 | 105 | 106 | # Joining data 107 | ```{r} 108 | bc_stations <- hy_stations(prov_terr_state_loc = "BC") %>% 109 | select(STATION_NUMBER, STATION_NAME) 110 | 111 | bc_mad <- hy_annual_stats(prov_terr_state_loc = "BC") %>% 112 | filter(Sum_stat == "MEAN") 113 | 114 | bc_stations %>% 115 | left_join(bc_mad, by = c("STATION_NUMBER")) 116 | ``` 117 | 118 | 119 | 120 | ## Your turn 121 | 122 | Find the Unit flow (Flow per watershed area) for all BC stations for all years 123 | 124 | ```{r} 125 | 126 | ``` 127 | 128 | -------------------------------------------------------------------------------- /04-Visualize.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Visualize data" 3 | --- 4 | 5 | 9 | 10 | ```{r setup} 11 | library(tidyverse) 12 | library(tidyhydat) 13 | ``` 14 | 15 | # ggplot2 and tidyhydat 16 | 17 | Create a dataframe of the number of stations and average length of record by real time and activity status. 18 | ```{r} 19 | data_range <- hy_stn_data_range() %>% 20 | group_by(STATION_NUMBER) %>% 21 | summarise(total_record_length = sum(RECORD_LENGTH, na.rm = TRUE)) 22 | 23 | cdn_prov <- c("NB", "PE", "NS", "ON", "QC", "NL", "MB", "AB", "SK", "NU", "NT", "BC", "YT") 24 | 25 | station_meta <- hy_stations() %>% 26 | filter(PROV_TERR_STATE_LOC %in% cdn_prov) %>% 27 | group_by(PROV_TERR_STATE_LOC, HYD_STATUS, REAL_TIME) %>% 28 | left_join(data_range, by = c("STATION_NUMBER")) %>% 29 | summarise(number_stns = n(), mean_record_length = mean(total_record_length, na.rm = TRUE)) 30 | ``` 31 | 32 | ## Your turn 33 | 34 | What relationships might interest you with this data? 35 | 36 | Type in the code from the slide, and run it to make a graph. Pay strict attention to spelling, capitalization, and parentheses. 37 | 38 | I strongly recommend typing it, even though copying and pasting is easier! Re-typing it will help your fingers learn the patterns you'll need to apply to your own data analyses. 39 | 40 | ```{r} 41 | ggplot(data = station_meta, mapping = aes(x = number_stns, y = mean_record_length)) + 42 | geom_point() 43 | ``` 44 | 45 | Remember to raise a post-it if you get stuck, and then once you're done! 46 | 47 | # Mappings 48 | 49 | ## Your Turn 50 | 51 | In the next chunk, modify which data is mapped to x, y, color, size, alpha, and shape aesthetics to your graph. Experiment. 52 | - Generate two different examples. 53 | 54 | 55 | ```{r} 56 | ggplot(data = station_meta, mapping = aes(x = number_stns, y = mean_record_length)) + 57 | geom_point() 58 | ``` 59 | 60 | # Geoms 61 | 62 | ## Your Turn 63 | Use the mean annual flow from all PEI stations 64 | 65 | ```{r data} 66 | pei_mean <- hy_annual_stats(prov_terr_state_loc = "PE") %>% 67 | filter(Sum_stat == "MEAN", Parameter == "Flow") 68 | ``` 69 | 70 | Replace this scatterplot with one that draws boxplots. Use the cheatsheet. Guess! 71 | - `theme` code is to rotate axes labels. 72 | ```{r} 73 | ggplot(data = pei_mean) + 74 | geom_point(mapping = aes(x = STATION_NUMBER, y = Value)) + 75 | theme(axis.text.x = element_text(angle=90, vjust=0.5)) 76 | ``` 77 | 78 | ```{r boxplot} 79 | ggplot(data = pei_mean) 80 | ``` 81 | 82 | 83 | ## Your Turn 84 | 85 | Make a hydrograph of the 08GB014 HORSESHOE RIVER ABOVE LOIS LAKE BC station using the Year and yearday columns created. For this we again need to make use of the lubridate package. 86 | 87 | ```{r} 88 | library(lubridate) 89 | horseshoe <- hy_daily_flows(station_number = "08GB014") %>% 90 | mutate(Year = year(Date), yearday = yday(Date)) 91 | ``` 92 | 93 | ### Grouping 94 | ```{r group} 95 | ggplot(data = horseshoe) 96 | ``` 97 | 98 | ### Facet 99 | ```{r facet} 100 | ggplot(data = horseshoe) 101 | ``` 102 | 103 | ## Your Turn 104 | 105 | Make a histogram plot of `horseshoe`. 106 | 107 | ```{r} 108 | ggplot(data = horseshoe, mapping = aes(x = Value)) + 109 | geom_histogram() 110 | ``` 111 | 112 | And then if you would like to modify the bin size, simply supply the bidwidth argument. Experiment with different values: 113 | ```{r histo} 114 | ggplot(data = horseshoe, mapping = aes(x = Value)) + 115 | geom_histogram(binwidth = 1) 116 | ``` 117 | 118 | And to add your distribution visualization, we can add *rug* marks. Note that the colour aesthetic is only mapped to the `geom_rug` function: 119 | 120 | ```{r rug} 121 | ggplot(data = horseshoe, mapping = aes(x = Value)) + 122 | geom_histogram(binwidth = 1) + 123 | geom_rug(mapping = aes(colour = Value)) 124 | ``` 125 | 126 | # Saving Graphs 127 | 128 | ## Your Turn 129 | 130 | What does `getwd()` return? 131 | 132 | ```{r} 133 | getwd() 134 | ``` 135 | 136 | ## Your Turn 9 137 | 138 | Save the last plot and then locate it in the files pane. Next find it on your desktop. 139 | 140 | ```{r} 141 | 142 | ``` 143 | 144 | *** 145 | 146 | # Take aways 147 | 148 | You can use this code template to make thousands of graphs with **ggplot2**. 149 | 150 | ```{r, eval = FALSE} 151 | ggplot(data = ) + 152 | (mapping = aes()) 153 | ``` 154 | 155 | # What else? 156 | 157 | ## Basic plot 158 | ```{r} 159 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 160 | geom_bar(stat = "identity", aes(fill = HYD_STATUS)) 161 | ``` 162 | 163 | ## Titles and captions + labs() 164 | ```{r} 165 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 166 | geom_bar(stat = "identity", aes(fill = HYD_STATUS)) + 167 | labs(title = "Hydrometric Stations in Canada", 168 | subtitle = "All stations are part of the Water Survey of Canada network", 169 | caption = "Figures generated with tidyhydat and ggplot2", 170 | y = "Number of stations", 171 | x = "Province /Territory") 172 | ``` 173 | 174 | ## Modifying the legend title 175 | ```{r} 176 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 177 | geom_bar(stat = "identity", aes(fill = HYD_STATUS)) + 178 | labs(title = "Hydrometric Stations in Canada", 179 | subtitle = "All stations are part of the Water Survey of Canada network", 180 | caption = "Figures generated with tidyhydat and ggplot2", 181 | y = "Number of stations", 182 | x = "Province /Territory") + 183 | scale_fill_discrete(name = "Status") 184 | ``` 185 | 186 | ## Position Adjustments 187 | 188 | ```{r} 189 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 190 | geom_bar(stat = "identity", aes(fill = HYD_STATUS), position = position_dodge()) + 191 | labs(title = "Hydrometric Stations in Canada", 192 | subtitle = "All stations are part of the Water Survey of Canada network", 193 | caption = "Figures generated with tidyhydat and ggplot2", 194 | y = "Number of stations", 195 | x = "Province /Territory") + 196 | scale_fill_discrete(name = "Status") 197 | ``` 198 | 199 | ## Theme - Visual appearance of non-data elements 200 | 201 | ```{r} 202 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 203 | geom_bar(stat = "identity", aes(fill = HYD_STATUS), position = position_dodge()) + 204 | labs(title = "Hydrometric Stations in Canada", 205 | subtitle = "All stations are part of the Water Survey of Canada network", 206 | caption = "Figures generated with tidyhydat and ggplot2", 207 | y = "Number of stations", 208 | x = "Province /Territory") + 209 | scale_fill_discrete(name = "Status") + 210 | theme_minimal() 211 | ``` 212 | 213 | ## Scales - Customize color scales, other mappings 214 | 215 | ```{r} 216 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 217 | geom_bar(stat = "identity", aes(fill = HYD_STATUS), position = position_dodge()) + 218 | labs(title = "Hydrometric Stations in Canada", 219 | subtitle = "All stations are part of the Water Survey of Canada network", 220 | caption = "Figures generated with tidyhydat and ggplot2", 221 | y = "Number of stations", 222 | x = "Province /Territory") + 223 | scale_fill_brewer(name = "Status", palette = "Accent") + 224 | theme_minimal() 225 | ``` 226 | 227 | ## Facets - Subplots that display subsets of the data. 228 | 229 | ```{r} 230 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 231 | geom_bar(stat = "identity", aes(fill = HYD_STATUS), position = position_dodge()) + 232 | labs(title = "Hydrometric Stations in Canada", 233 | subtitle = "All stations are part of the Water Survey of Canada network", 234 | caption = "Figures generated with tidyhydat and ggplot2", 235 | y = "Number of stations", 236 | x = "Province /Territory") + 237 | scale_fill_brewer(name = "Status", palette = "Accent") + 238 | theme_minimal() + 239 | facet_wrap(~REAL_TIME, ncol = 1) 240 | ``` 241 | 242 | ## Coordinate systems 243 | 244 | ```{r} 245 | ggplot(data = station_meta, mapping = aes(x = PROV_TERR_STATE_LOC, y = number_stns)) + 246 | geom_bar(stat = "identity", aes(fill = HYD_STATUS), position = position_dodge()) + 247 | labs(title = "Hydrometric Stations in Canada", 248 | subtitle = "All stations are part of the Water Survey of Canada network", 249 | caption = "Figures generated with tidyhydat and ggplot2", 250 | y = "Number of stations", 251 | x = "Province /Territory") + 252 | scale_fill_brewer(name = "Status", palette = "Accent") + 253 | theme_minimal() + 254 | facet_wrap(~REAL_TIME) + 255 | coord_polar() 256 | ``` 257 | 258 | ## Just cool stuff 259 | 260 | ```{r, error=TRUE} 261 | ##install.packages("plotly") 262 | library(plotly) 263 | 264 | library(lubridate) 265 | horseshoe <- hy_daily_flows(station_number = "08GB014") %>% 266 | mutate(Year = year(Date), yearday = yday(Date)) 267 | 268 | hydrograph <- ggplot(data = horseshoe, mapping = aes(x = yearday, y = Value, colour = Year)) + 269 | geom_line(mapping = aes(group = Year)) 270 | 271 | ggplotly(hydrograph) 272 | 273 | 274 | ``` 275 | 276 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Code of Conduct 2 | 3 | As contributors and maintainers of this project, and in the interest of 4 | fostering an open and welcoming community, we pledge to respect all people who 5 | contribute through reporting issues, posting feature requests, updating 6 | documentation, submitting pull requests or patches, and other activities. 7 | 8 | We are committed to making participation in this project a harassment-free 9 | experience for everyone, regardless of level of experience, gender, gender 10 | identity and expression, sexual orientation, disability, personal appearance, 11 | body size, race, ethnicity, age, religion, or nationality. 12 | 13 | Examples of unacceptable behavior by participants include: 14 | 15 | * The use of sexualized language or imagery 16 | * Personal attacks 17 | * Trolling or insulting/derogatory comments 18 | * Public or private harassment 19 | * Publishing other's private information, such as physical or electronic 20 | addresses, without explicit permission 21 | * Other unethical or unprofessional conduct 22 | 23 | Project maintainers have the right and responsibility to remove, edit, or 24 | reject comments, commits, code, wiki edits, issues, and other contributions 25 | that are not aligned to this Code of Conduct, or to ban temporarily or 26 | permanently any contributor for other behaviors that they deem inappropriate, 27 | threatening, offensive, or harmful. 28 | 29 | By adopting this Code of Conduct, project maintainers commit themselves to 30 | fairly and consistently applying these principles to every aspect of managing 31 | this project. Project maintainers who do not follow or enforce the Code of 32 | Conduct may be permanently removed from the project team. 33 | 34 | This Code of Conduct applies both within project spaces and in public spaces 35 | when an individual is representing the project or its community. 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 38 | reported by contacting a project maintainer at sam.albers@gov.bc.ca. All complaints will be reviewed and investigated 39 | and will result in a response that is deemed necessary and appropriate to the 40 | circumstances. Maintainers are obligated to maintain confidentiality with regard 41 | to the reporter of an incident. 42 | 43 | 44 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 45 | version 1.3.0, available at 46 | [http://contributor-covenant.org/version/1/3/0/][version] 47 | 48 | [homepage]: http://contributor-covenant.org 49 | [version]: http://contributor-covenant.org/version/1/3/0/ 50 | 51 | --- 52 | *This project was created using the [bcgovr](https://github.com/bcgov/bcgovr) package.* 53 | -------------------------------------------------------------------------------- /COMPLIANCE.yaml: -------------------------------------------------------------------------------- 1 | name: compliance 2 | description: | 3 | This document is used to track a projects PIA and STRA 4 | compliance. 5 | spec: 6 | - name: PIA 7 | status: not-required 8 | last-updated: '2020-04-15T00:14:53.587Z' 9 | - name: STRA 10 | status: not-required 11 | last-updated: '2020-04-15T00:14:53.587Z' 12 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | ## How to contribute 2 | Government employees, public and members of the private sector are encouraged to contribute to the repository by **forking and submitting a pull request**. 3 | 4 | (If you are new to GitHub, you might start with a [basic tutorial](https://help.github.com/articles/set-up-git) and check out a more detailed guide to [pull requests](https://help.github.com/articles/using-pull-requests/).) 5 | 6 | Pull requests will be evaluated by the repository guardians on a schedule and if deemed beneficial will be committed to the master. 7 | 8 | All contributors retain the original copyright to their stuff, but by contributing to this project, you grant a world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license to all users **under the terms of the license under which this project is distributed.** 9 | 10 | --- 11 | *This project was created using the [bcgovr](https://github.com/bcgov/bcgovr) package.* 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Attribution 4.0 International Public License 2 | By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions. 3 | 4 | Section 1 – Definitions. 5 | 6 | Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image. 7 | Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License. 8 | Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights. 9 | Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements. 10 | Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material. 11 | Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License. 12 | Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license. 13 | Licensor means the individual(s) or entity(ies) granting rights under this Public License. 14 | Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them. 15 | Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world. 16 | You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning. 17 | Section 2 – Scope. 18 | 19 | License grant. 20 | Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to: 21 | reproduce and Share the Licensed Material, in whole or in part; and 22 | produce, reproduce, and Share Adapted Material. 23 | Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions. 24 | Term. The term of this Public License is specified in Section 6(a). 25 | Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material. 26 | Downstream recipients. 27 | Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License. 28 | No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material. 29 | No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i). 30 | Other rights. 31 | 32 | Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise. 33 | Patent and trademark rights are not licensed under this Public License. 34 | To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties. 35 | Section 3 – License Conditions. 36 | 37 | Your exercise of the Licensed Rights is expressly made subject to the following conditions. 38 | 39 | Attribution. 40 | 41 | If You Share the Licensed Material (including in modified form), You must: 42 | 43 | retain the following if it is supplied by the Licensor with the Licensed Material: 44 | identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated); 45 | a copyright notice; 46 | a notice that refers to this Public License; 47 | a notice that refers to the disclaimer of warranties; 48 | a URI or hyperlink to the Licensed Material to the extent reasonably practicable; 49 | indicate if You modified the Licensed Material and retain an indication of any previous modifications; and 50 | indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License. 51 | You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information. 52 | If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable. 53 | If You Share Adapted Material You produce, the Adapter's License You apply must not prevent recipients of the Adapted Material from complying with this Public License. 54 | Section 4 – Sui Generis Database Rights. 55 | 56 | Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material: 57 | 58 | for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database; 59 | if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and 60 | You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database. 61 | For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights. 62 | Section 5 – Disclaimer of Warranties and Limitation of Liability. 63 | 64 | Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You. 65 | To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You. 66 | The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability. 67 | Section 6 – Term and Termination. 68 | 69 | This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically. 70 | Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates: 71 | 72 | automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or 73 | upon express reinstatement by the Licensor. 74 | For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License. 75 | For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License. 76 | Sections 1, 5, 6, 7, and 8 survive termination of this Public License. 77 | Section 7 – Other Terms and Conditions. 78 | 79 | The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed. 80 | Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License. 81 | Section 8 – Interpretation. 82 | 83 | For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License. 84 | To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions. 85 | No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor. 86 | Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority. 87 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![img](https://img.shields.io/badge/Lifecycle-Retired-d45500)](https://github.com/bcgov/repomountie/blob/master/doc/lifecycle-badges.md) 2 | 3 | # Introduction to R and the tidyverse in Hydrology 4 | 5 | This is the repo for *Introduction to R and the tidyverse in Hydrology* to be delivered at the [Canadian Water Resources Association](https://conference.cwra.org/) 2018 conference in Victoria, British Columbia on May 28, 2018. The workshop repo is currently under development and all materials should be considered a work in progress. 6 | 7 | This is a 4 hour hands-on workshop that leans heavily on the book [R for Data Science](http://r4ds.had.co.nz/) and on the [tidyhydat](https://CRAN.R-project.org/package=tidyhydat) package. This workshop is intended as an introduction for people interested in hydrological analysis and who would like to try using R. In this workshop you will learn how to: 8 | 9 | - Recognize basic elements of the R programming language 10 | - Use the R package tidyhydat to access realtime and historical Water Survey of Canada data directly from R 11 | - Conduct a variety basic data tidying steps 12 | - Generate high quality plots in R 13 | - Understand where and how to seek help for R 14 | - Take away your R recipes for future analysis 15 | 16 | You will learn how to visualize and transform untidy data formats. Along the way, you will learn and use several packages from the tidyverse including ggplot2, dplyr, tidyr, and purrr. In addition, we will make use of the tidyhydat package, which imports and tidies [Water Survey of Canada](https://wateroffice.ec.gc.ca/index_e.html) hydrometric data into R. 17 | 18 | 19 | 20 | ## Software requirements 21 | You will need a working installation of R and RStudio available from here: 22 | 23 | Install R from here: 24 | https://cloud.r-project.org/ 25 | 26 | Install RStudio from here: 27 | https://www.rstudio.com/products/rstudio/download/#download 28 | 29 | You'll need the following packages: 30 | 31 | ```R 32 | install.packages(c("tidyverse", "tidyhydat", "usethis", "rmarkdown")) 33 | ``` 34 | 35 | Then you can grab a local copy of all the slides, code and data: 36 | 37 | ```R 38 | usethis::use_course("https://bit.ly/2L95hCb") 39 | ``` 40 | 41 | To get back to this project later, double-click on "intro-to-tidyhydat-and-tidyverse.Rproj". 42 | 43 | ## License 44 | 45 | Creative Commons License 46 | 47 | Original repo *Data Science in the tidyverse* by Charlotte Wickham is licensed under a Creative Commons Attribution 4.0 International License. Based on a work at https://github.com/rstudio/master-the-tidyverse. 48 | 49 | ## Acknowledgements 50 | 51 | I have forked this repo from [Hadley Wickham](https://github.com/hadley/data-science-in-tidyverse) who in turn forked it from [Charlotte Wickham](https://github.com/cwickham/data-science-in-tidyverse), who forked it from [RStudio](https://github.com/rstudio/master-the-tidyverse). The feel, look and flow of the material in this workshop was originally created by these authors. My main contribution was adapt the examples to hydrologically based data. Huge thanks to [Hadley](http://hadley.nz/), [Charlotte](http://cwick.co.nz) and [Garrett](https://github.com/garrettgman) for creating the slides and accompanying materials! 52 | -------------------------------------------------------------------------------- /cheatsheets/dplyr-data-transformation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/cheatsheets/dplyr-data-transformation.pdf -------------------------------------------------------------------------------- /cheatsheets/ggplot2-data-visualization.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/cheatsheets/ggplot2-data-visualization.pdf -------------------------------------------------------------------------------- /cheatsheets/lubridate-dates-times.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/cheatsheets/lubridate-dates-times.pdf -------------------------------------------------------------------------------- /cheatsheets/rstudio-ide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/cheatsheets/rstudio-ide.pdf -------------------------------------------------------------------------------- /cheatsheets/tidyr-readr-data-import-tidy.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/cheatsheets/tidyr-readr-data-import-tidy.pdf -------------------------------------------------------------------------------- /email-to-participants.md: -------------------------------------------------------------------------------- 1 | Text available at: https://github.com/bcgov/intro-to-tidyhydat-and-tidyverse/blob/master/email-to-participants.md 2 | 3 | **Is this workshop for me?** 4 | 5 | Do you have any experience with R? If the answer is no, don't worry. This workshop is designed for people with little to no experience with R but are interested in bringing reproducible analysis to their hydrological data analysis needs. 6 | 7 | **What should you bring?** 8 | - Laptop installed with R and RStudio that is capable of connecting to Wifi 9 | 10 | **Considerations** 11 | - You have administrative access to your computer 12 | - All software and packages should be installed on your computer rather than a network drive 13 | 14 | 15 | **Is there anything you should do before arriving?** 16 | 17 | Yes! Four things. 18 | 19 | Install R from here: 20 | https://cloud.r-project.org/ 21 | 22 | Install RStudio from here: 23 | https://www.rstudio.com/products/rstudio/download/#download 24 | 25 | Open R and paste the following code into the blinking console to install required R packages and download the HYDAT database: 26 | 27 | 28 | .libPaths() 29 | 30 | pkgs <- c("tidyverse","tidyhydat","rmarkdown","usethis") 31 | install.packages(pkgs) 32 | install_success <- pkgs %in% rownames(installed.packages()) 33 | 34 | if(any(!install_success)){ 35 | miss_pkgs <- pkgs[which(!install_success)] 36 | cat("The following packages were not successfully installed:", paste0(miss_pkgs, collapse = ", ")) 37 | } 38 | 39 | 40 | did_it_work <- tidyhydat::download_hydat() 41 | if(isTRUE(!did_it_work)) cat("HYDAT did not download successfully") 42 | 43 | 44 | This code will take about 10 to fifteen minutes to run. 45 | 46 | **What should I do if I have problem?** 47 | Email me! I am happy to sort of R installation issues, HYDAT downloading issues, package library locations or whatever other weird installation problems you may have ***before the workshop***. If we can solve these problems beforehand, we will be able to squeeze in much more actual R work during our time together. 48 | 49 | If you have any other questions, feel free to email me. 50 | 51 | 52 | 53 | -------------------------------------------------------------------------------- /intro-to-tidyhydat-and-tidyverse.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | -------------------------------------------------------------------------------- /slides/.gitignore: -------------------------------------------------------------------------------- 1 | *.key 2 | *.pptx 3 | -------------------------------------------------------------------------------- /slides/00-Getting-started.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/slides/00-Getting-started.pdf -------------------------------------------------------------------------------- /slides/01-tidyhydat-intro.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/slides/01-tidyhydat-intro.pdf -------------------------------------------------------------------------------- /slides/02-Transform.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/slides/02-Transform.pdf -------------------------------------------------------------------------------- /slides/03-Dates-Joins.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/slides/03-Dates-Joins.pdf -------------------------------------------------------------------------------- /slides/04-Visualize.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/slides/04-Visualize.pdf -------------------------------------------------------------------------------- /slides/05-wrapping-up.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bcgov/intro-to-tidyhydat-and-tidyverse/caf2642dbf269b251d53fe39d6688057f5d69412/slides/05-wrapping-up.pdf --------------------------------------------------------------------------------