├── README.md
└── krypsiv_3.ipynb
/README.md:
--------------------------------------------------------------------------------
1 | # Krypsiv (bulbous rushes)
2 |
3 | Repo containing notebooks for TFM's Krypsiv project.
4 |
--------------------------------------------------------------------------------
/krypsiv_3.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 1,
6 | "metadata": {
7 | "collapsed": true
8 | },
9 | "outputs": [],
10 | "source": [
11 | "%matplotlib inline\n",
12 | "import matplotlib.pyplot as plt, mpld3, glob, os\n",
13 | "import pandas as pd, seaborn as sn, numpy as np\n",
14 | "sn.set_context('talk')"
15 | ]
16 | },
17 | {
18 | "cell_type": "markdown",
19 | "metadata": {},
20 | "source": [
21 | "# Krypsiv (part 3)\n",
22 | "\n",
23 | "A [previous notebook](http://nbviewer.jupyter.org/github/JamesSample/krypsiv/blob/master/krypsiv.ipynb) explored various plotting options for the krypsiv data, and Therese has since chosen the plots she'd like to use in the final report (see e-mail received 02/11/2016 at 09:54). In a second notebook (`krypsiv_2.ipynb`), I've modified my code to produce tidier versions of the required plots and saved them all as PNGs.\n",
24 | "\n",
25 | "Rather than linking to the visualisations in the second notebook, I think it's probably better to view the PNG plots themselves, as the appearance online is sometimes slightly different to the finished image files. Ultimately, it's the PNGs that will be used in the report, so these versions need the most careful checking.\n",
26 | "\n",
27 | "My aims with this notebook are:\n",
28 | "\n",
29 | " 1. Provide an overview of the plots generated so far and highlight a few things that need checking, and
\n",
30 | " \n",
31 | " 2. To take a look at some of the other data processing tasks (e.g. catchment area calculations, periods when the water level sensors are above the water etc.). \n",
32 | " \n",
33 | "**NB:** We're still waiting for some data from the lab, so some of these plots will change in the future.\n",
34 | "\n",
35 | "**NB2:** I still haven't produced plots showing the variance of the replicate pH, redox and conductivity measurements (point 6 in Therese's e-mail). **Come back to this**.\n",
36 | " \n",
37 | "## 1. PNG plots\n",
38 | "\n",
39 | "I don't have permission to upload files to the project folder on the network, so I've created a temporary folder for the plots here:\n",
40 | "\n",
41 | "K:\\Prosjekter\\JES\\TFM_Krypsiv\\Plots\n",
42 | "\n",
43 | "This folder currently contains 8 PNG files and a subfolder called `chem_ts_by_site`, which contains a further 19 PNGs.\n",
44 | "\n",
45 | "**Things to check:**\n",
46 | "\n",
47 | " 1. For all files, please check the axis labels, units (if appropriate) and my Norwegian (!).
\n",
48 | " \n",
49 | " 2. For *algal_proportions.png*, all the proportions add to 100%, except for station HIL during 2014. Is this correct?
\n",
50 | " \n",
51 | " 3. For *substrate_proportions.png*, I've adjusted the \"Organisk\" class to make the totals sum to 100% (as suggested by Therese in her e-mail). For each plot, asterisks (\\*) on the years on the x-axis highlight where adjustments have been made. The following results still look a bit strange and need further checking:\n",
52 | " \n",
53 | " * VAL in 2014 has 200% boulders *and* 100% organic. The overall total is currently 380% \n",
54 | " * Even removing the organic column, the total for VAL in 2016 is 110%, so one of the other columns also needs reducing \n",
55 | " * SØR in 2016 currently sums to 157%, but with only 15% organic. Some of the other classes will therefore need reducing too\n",
56 | " \n",
57 | "## 2. Estimate discharge for MID and FYG\n",
58 | "\n",
59 | "Based on the preliminary analysis in section 3 of the [previous notebook](http://nbviewer.jupyter.org/github/JamesSample/krypsiv/blob/master/krypsiv.ipynb), it looks reasonable to estimate discharge at stations MID and FYG by area-scaling the flow data from the NVE site at Kjølemo.\n",
60 | "\n",
61 | "I've used the map in *Fig. 7* of the 2015 krypsiv report to identify the approximate locations of FIG and MID on the [NEVINA website](http://nevina.nve.no/), which gives the following catchment areas:\n",
62 | "\n",
63 | "| Site | Area ($km^2$) |\n",
64 | "|:----:|:-------------:|\n",
65 | "| MID | 1787.3 |\n",
66 | "| FYG | 1534.1 |\n",
67 | "\n",
68 | "According to NVE, the catchment area for Kjølemo is $1757.7 \\; km^2$. Based on this, we can estimate flow at FYG and MID using the code below (the pan and zoom tools can be used to explore the plot, as before).\n",
69 | "\n",
70 | "The NEVINA website also produces PDFs of catchment properties, which may conatin useful information. For MID and FYG, these can be found here:\n",
71 | "\n",
72 | "K:\\Prosjekter\\JES\\TFM_Krypsiv\\Catchment_Props"
73 | ]
74 | },
75 | {
76 | "cell_type": "code",
77 | "execution_count": 2,
78 | "metadata": {
79 | "collapsed": false
80 | },
81 | "outputs": [
82 | {
83 | "data": {
84 | "text/html": [
85 | "\n",
86 | "\n",
87 | "\n",
90 | "\n",
91 | "\n",
92 | ""
128 | ],
129 | "text/plain": [
130 | ""
131 | ]
132 | },
133 | "execution_count": 2,
134 | "metadata": {},
135 | "output_type": "execute_result"
136 | }
137 | ],
138 | "source": [
139 | "# Catchment areas (km2)\n",
140 | "kjo_area = 1757.7\n",
141 | "mid_area = 1787.3\n",
142 | "fyg_area = 1534.1\n",
143 | "\n",
144 | "# Read Kjølemo flow data from NVE\n",
145 | "kjo_xls = r'C:\\Data\\James_Work\\Staff\\Therese_FM\\Bulbous_Rushes\\Data\\Tidied\\nve_kjolemo_22-4-0.xlsx'\n",
146 | "\n",
147 | "df = pd.read_excel(kjo_xls, sheetname='Sheet1',\n",
148 | " index_col=0)\n",
149 | "\n",
150 | "df = df.resample('D').mean()\n",
151 | "\n",
152 | "df.dropna(how='any', inplace=True)\n",
153 | "\n",
154 | "# Area scaling\n",
155 | "df['MID'] = df['kjo_q_m3ps'] * mid_area / kjo_area\n",
156 | "df['FYG'] = df['kjo_q_m3ps'] * fyg_area / kjo_area\n",
157 | "\n",
158 | "del df['kjo_q_m3ps']\n",
159 | "\n",
160 | "# Plot\n",
161 | "df.plot(figsize=(12,6))\n",
162 | "plt.xlabel('')\n",
163 | "plt.ylabel('Discharge (m3/s)')\n",
164 | "plt.title('Estimated daily discharge')\n",
165 | "mpld3.display()"
166 | ]
167 | },
168 | {
169 | "cell_type": "markdown",
170 | "metadata": {},
171 | "source": [
172 | "## 3. Periods when the pressure sensors are dry\n",
173 | "\n",
174 | "In an e-mail received 02/11/2016 at 12:50, Therese asked for a way of summarising periods when the water level pressure sensors are out of the water. The easiest way to do this is probably to look at the proportion of zero and negative values in each series, to get an idea of which stations dry out most often.\n",
175 | "\n",
176 | "Kate has uploaded all the logger data to a folder here:\n",
177 | "\n",
178 | "K:\\Avdeling\\318 Ferskvannsøkologi\\Prosjekter\\2016 JBovervåk16-18 (TFM)\\04 Vannføring\\HOBO\\Calibrated_hobologgers\n",
179 | "\n",
180 | "I've made a local copy of this data and then editied it so that I can read it in easily below. The tidied version is here:\n",
181 | "\n",
182 | "C:\\Data\\James_Work\\Staff\\Therese_FM\\Bulbous_Rushes\\Data\\Tidied\\stage\n",
183 | "\n",
184 | "In the code below, the raw data for each series is read in first, then any \"no data\" values are removed and the percentage of each record where values are less than or equal to zero is calculated and displayed. "
185 | ]
186 | },
187 | {
188 | "cell_type": "code",
189 | "execution_count": 3,
190 | "metadata": {
191 | "collapsed": false,
192 | "scrolled": false
193 | },
194 | "outputs": [
195 | {
196 | "name": "stdout",
197 | "output_type": "stream",
198 | "text": [
199 | "Percentages of zero and negative values in each water level series:\n"
200 | ]
201 | },
202 | {
203 | "data": {
204 | "text/html": [
205 | "\n",
206 | "
\n",
207 | " \n",
208 | " \n",
209 | " | \n",
210 | " Pct_Dry | \n",
211 | "
\n",
212 | " \n",
213 | " \n",
214 | " \n",
215 | " | ASE | \n",
216 | " 9.665386 | \n",
217 | "
\n",
218 | " \n",
219 | " | LOG | \n",
220 | " 5.546796 | \n",
221 | "
\n",
222 | " \n",
223 | " | SME | \n",
224 | " 1.159107 | \n",
225 | "
\n",
226 | " \n",
227 | " | FYG | \n",
228 | " 1.070513 | \n",
229 | "
\n",
230 | " \n",
231 | " | MID | \n",
232 | " 1.067853 | \n",
233 | "
\n",
234 | " \n",
235 | " | SVE | \n",
236 | " 0.863317 | \n",
237 | "
\n",
238 | " \n",
239 | " | BRO | \n",
240 | " 0.636090 | \n",
241 | "
\n",
242 | " \n",
243 | " | VEN | \n",
244 | " 0.491685 | \n",
245 | "
\n",
246 | " \n",
247 | " | BJO | \n",
248 | " 0.324657 | \n",
249 | "
\n",
250 | " \n",
251 | " | OSE | \n",
252 | " 0.251214 | \n",
253 | "
\n",
254 | " \n",
255 | " | KIL | \n",
256 | " 0.000000 | \n",
257 | "
\n",
258 | " \n",
259 | " | HIL | \n",
260 | " 0.000000 | \n",
261 | "
\n",
262 | " \n",
263 | " | HER | \n",
264 | " 0.000000 | \n",
265 | "
\n",
266 | " \n",
267 | " | DRA | \n",
268 | " 0.000000 | \n",
269 | "
\n",
270 | " \n",
271 | " | SOR | \n",
272 | " 0.000000 | \n",
273 | "
\n",
274 | " \n",
275 | " | STR | \n",
276 | " 0.000000 | \n",
277 | "
\n",
278 | " \n",
279 | " | VAL | \n",
280 | " 0.000000 | \n",
281 | "
\n",
282 | " \n",
283 | " | APA | \n",
284 | " 0.000000 | \n",
285 | "
\n",
286 | " \n",
287 | "
\n",
288 | "