├── .gitignore
├── HTML
├── PythonBootcamp.html
└── Visualization.html
├── LICENSE
├── PythonBootcamp.ipynb
├── README.md
├── Skyfit.ipynb
├── Visualization.ipynb
├── Visualizing Graphs with NetworkX.pdf
├── data
├── SN2011iv_B_SWO_DC_2011_12_11SN.fits
├── SN2011iv_B_template.fits
├── SNIa_DM.dat
├── galsample1.dat
├── galsample2.dat
└── im3433.fits
├── scripts
├── plot_Hubble.py
└── plot_diff.py
└── skyfit_spyder.py
/.gitignore:
--------------------------------------------------------------------------------
1 | .ipynb_checkpoints
2 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | The MIT License (MIT)
2 |
3 | Copyright (c) 2016 Observatories Code Repository
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/PythonBootcamp.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Carnegie Python Bootcamp\n",
8 | "\n",
9 | "Welcome to the python bootcamp. This thing you're reading is called an _ipython notebook_ and will be your first introduction to the Python programming language. Notebooks are a combination of text markup and code that you can run in real time.\n",
10 | "\n",
11 | "## Importing what you need\n",
12 | "\n",
13 | "It is often said that python comes with the _batteries included_, which means it comes with almost everything you need, bundled up in seperate _modules_. But not everything is loaded into memory automatically. You need to import the modules you need. Try running the following to import the antigravity module. Just hit `CTRL-ENTER` in the cell to execute the code"
14 | ]
15 | },
16 | {
17 | "cell_type": "code",
18 | "execution_count": null,
19 | "metadata": {
20 | "collapsed": true
21 | },
22 | "outputs": [],
23 | "source": [
24 | "import antigravity"
25 | ]
26 | },
27 | {
28 | "cell_type": "markdown",
29 | "metadata": {},
30 | "source": [
31 | "You will find that python is full of nerdy humour like this. By the way, python is named after Monty Python's Flying Circus, *not* the reptile. Let's go ahead and import some useful modules you **will** need. Use this next command cell to import the the modules named ``os``, ``sys``, and ``numpy``."
32 | ]
33 | },
34 | {
35 | "cell_type": "code",
36 | "execution_count": null,
37 | "metadata": {
38 | "collapsed": true
39 | },
40 | "outputs": [],
41 | "source": [
42 | "import os,sys,numpy"
43 | ]
44 | },
45 | {
46 | "cell_type": "markdown",
47 | "metadata": {},
48 | "source": [
49 | "The `os` module gives you functions relating to the operating system, `sys` gives you information on the python interpreter and your script, and `numpy` is the mathematical powerhorse of python, allowing you to manipulate arrays of numbers.\n",
50 | "\n",
51 | "Once you import a module, all the functions and data for that module live in its _namespace_. So if I wanted to use the `getcwd` function from the `os` module, I would have to refer to it as ``os.getcwd``. Try running the next command:"
52 | ]
53 | },
54 | {
55 | "cell_type": "code",
56 | "execution_count": null,
57 | "metadata": {
58 | "collapsed": false,
59 | "scrolled": true
60 | },
61 | "outputs": [],
62 | "source": [
63 | "os.getcwd()"
64 | ]
65 | },
66 | {
67 | "cell_type": "markdown",
68 | "metadata": {},
69 | "source": [
70 | "As you can probably guess, `os.getcwd` gets the current working directory. Try some commands for yourself. The ``numpy`` module has many mathematical functions. Try computing the square root (``sqrt``) of 2. You can also try computing $\\sin(\\pi)$ (numpy has a built-in value ``numpy.pi``) and even $e^{i\\pi}$ (python uses the engineering convention of ``1j`` as $\\sqrt{-1}$)."
71 | ]
72 | },
73 | {
74 | "cell_type": "code",
75 | "execution_count": null,
76 | "metadata": {
77 | "collapsed": false
78 | },
79 | "outputs": [],
80 | "source": [
81 | "numpy.sqrt(2)\n",
82 | "numpy.sin(numpy.pi)\n",
83 | "numpy.exp(numpy.pi*1j)"
84 | ]
85 | },
86 | {
87 | "cell_type": "markdown",
88 | "metadata": {},
89 | "source": [
90 | "There can also be namespaces inside namespaces. For example, to test if a file exists, you would do:"
91 | ]
92 | },
93 | {
94 | "cell_type": "code",
95 | "execution_count": null,
96 | "metadata": {
97 | "collapsed": false
98 | },
99 | "outputs": [],
100 | "source": [
101 | "print (os.path.isfile('.bash_profile'))"
102 | ]
103 | },
104 | {
105 | "cell_type": "markdown",
106 | "metadata": {},
107 | "source": [
108 | "Here is a non-comprehensive list of modules you may find useful. Documentation for all of them can be found with a quick google search.\n",
109 | "\n",
110 | "- `os`, `sys`: As mentioned above, these give you access to the operating system, files, and environment.\n",
111 | "- `numpy`: Gives you arrays (vectors, matrices) and the ability to do math on them.\n",
112 | "- `scipy`: Think of this as \"Numerical Recipes for Python\". Root-finding, fitting functions, integration, special mathematical functions, etc.\n",
113 | "- `pandas`: Primarily used for reading/writing data. Useful for [data wrangling](https://en.wikipedia.org/wiki/Data_wrangling).\n",
114 | "- `astropy`: Astronomy-related functions, including FITS image reading, astrometry, and cosmological calculations.\n",
115 | "\n",
116 | "## Getting Help\n",
117 | "\n",
118 | "Almost everything in python has documentation. Just use the `help()` function on anything in python to get information. Here is an example help output on `numpy.sqrt`. Try others."
119 | ]
120 | },
121 | {
122 | "cell_type": "code",
123 | "execution_count": null,
124 | "metadata": {
125 | "collapsed": false,
126 | "scrolled": true
127 | },
128 | "outputs": [],
129 | "source": [
130 | "help(numpy.sqrt)"
131 | ]
132 | },
133 | {
134 | "cell_type": "markdown",
135 | "metadata": {
136 | "collapsed": true
137 | },
138 | "source": [
139 | "## Variables and Types\n",
140 | "\n",
141 | "A variable is like a box that you put stuff (values) in for later use. In Python, there are lots of different ``types`` of variables corresponding to what you put in. Unlike other languages, you don't have to tell ``python`` what type each variable is: ``python`` figures it out on its own. To put the value 1 into a box called x, just use the equals sign, like you would when solving a math problem."
142 | ]
143 | },
144 | {
145 | "cell_type": "code",
146 | "execution_count": null,
147 | "metadata": {
148 | "collapsed": true
149 | },
150 | "outputs": [],
151 | "source": [
152 | "x=1"
153 | ]
154 | },
155 | {
156 | "cell_type": "markdown",
157 | "metadata": {},
158 | "source": [
159 | "Now you can later use this value. Try printing it out using the ``print()`` function. You can also modify the variable. Try adding or subtracting from x and print it out again."
160 | ]
161 | },
162 | {
163 | "cell_type": "code",
164 | "execution_count": null,
165 | "metadata": {
166 | "collapsed": false,
167 | "scrolled": false
168 | },
169 | "outputs": [],
170 | "source": [
171 | "print(x)\n",
172 | "x=x+1\n",
173 | "print(x)"
174 | ]
175 | },
176 | {
177 | "cell_type": "markdown",
178 | "metadata": {},
179 | "source": [
180 | "But be careful about the type. Try assigning the value of 1 to ``y``. Now divide it by two and print out the result. "
181 | ]
182 | },
183 | {
184 | "cell_type": "code",
185 | "execution_count": null,
186 | "metadata": {
187 | "collapsed": false
188 | },
189 | "outputs": [],
190 | "source": [
191 | "y = 1\n",
192 | "y = y/2\n",
193 | "print(y)"
194 | ]
195 | },
196 | {
197 | "cell_type": "markdown",
198 | "metadata": {},
199 | "source": [
200 | "Repeat, but this time assign the value of 1.0 to y."
201 | ]
202 | },
203 | {
204 | "cell_type": "code",
205 | "execution_count": null,
206 | "metadata": {
207 | "collapsed": false
208 | },
209 | "outputs": [],
210 | "source": [
211 | "# make sure you include a decimal in the number one! (fyi, this is a comment)\n",
212 | "y = 1.0\n",
213 | "y = y/2\n",
214 | "print (y)"
215 | ]
216 | },
217 | {
218 | "cell_type": "markdown",
219 | "metadata": {},
220 | "source": [
221 | "You get different results because in the first instance, ``y`` was interpreted as an integer, so the result of dividing by 2 must also be an integer. In the second case, ``y`` is interpreted as a real number (aka floating point or float), so the result of dividing by zero is what you expect. There are other types of variables. The most commonly used are strings, lists, and arrays. But literally **anything** can be assigned to an array. You can even assign the ``numpy`` module to the variable ``np``."
222 | ]
223 | },
224 | {
225 | "cell_type": "code",
226 | "execution_count": null,
227 | "metadata": {
228 | "collapsed": false
229 | },
230 | "outputs": [],
231 | "source": [
232 | "np = numpy"
233 | ]
234 | },
235 | {
236 | "cell_type": "markdown",
237 | "metadata": {},
238 | "source": [
239 | "Now you can use ``np.sqrt()`` rather than ``numpy.sqrt()``. Most python programmers do this with commonly-used modules. In fact, we usually just use the special form: ``import numpy as np``."
240 | ]
241 | },
242 | {
243 | "cell_type": "markdown",
244 | "metadata": {},
245 | "source": [
246 | "# Strings\n",
247 | "Strings are collections of alpha-numeric characters and are the primary way data are represented outside your code. So you will find yourself manipulating strings in Python ALL THE TIME. You might need to convert strings from a text file into python data ojects to work with or you might need to do the opposite and generate an output file with final results that can be read by humans. Below are the most common things we need.\n",
248 | "\n",
249 | "Strings are enclosed in matching singe (`'`) or double (`\"`) quotes. Python doesn't distinguish as long as you match them consistent. "
250 | ]
251 | },
252 | {
253 | "cell_type": "code",
254 | "execution_count": null,
255 | "metadata": {
256 | "collapsed": false
257 | },
258 | "outputs": [],
259 | "source": [
260 | "# Strings are enclosed by single or double quotes\n",
261 | "s='this is a string'\n",
262 | "print (s)"
263 | ]
264 | },
265 | {
266 | "cell_type": "markdown",
267 | "metadata": {},
268 | "source": [
269 | "We can use the `len()` function to determine the length of the string. Try this out below."
270 | ]
271 | },
272 | {
273 | "cell_type": "code",
274 | "execution_count": null,
275 | "metadata": {
276 | "collapsed": true
277 | },
278 | "outputs": [],
279 | "source": []
280 | },
281 | {
282 | "cell_type": "markdown",
283 | "metadata": {},
284 | "source": [
285 | "Strings are very similar to lists in `python` (we'll see that below). Each alpha-numeric character is an element in the string, and you can refer to individual elements (characters) using an index enclosed in square brackets (`[]`). You can also specify a range of indices by separating them with a colon (`:`). `Python` indexes from 0 (not 1), so the first element is index `[0]`, the second `[1]` and so on."
286 | ]
287 | },
288 | {
289 | "cell_type": "code",
290 | "execution_count": null,
291 | "metadata": {
292 | "collapsed": false
293 | },
294 | "outputs": [],
295 | "source": [
296 | "print(s[2])\n",
297 | "print(s[2:6])"
298 | ]
299 | },
300 | {
301 | "cell_type": "markdown",
302 | "metadata": {},
303 | "source": [
304 | "Specifying a range of indices (as well as more complicated indexing we'll see later) is called *slicing*.\n",
305 | "\n",
306 | "## Formatting strings\n",
307 | "Sometimes you'll need your integers and floats to be converted into strings and written out into a table with specific formats (e.g., number of significant figures). This involves a syntax that's almost a separate language itself (though if you've used `C` or `C++` it will be very familiar). Here is a good reference: https://pyformat.info/\n",
308 | "\n",
309 | "We'll cover the most important. First, if you just print out a regular floating point number, you get some arbitrary number of significant figures. The same is true if you just try to convert the float to a string using `str()`:"
310 | ]
311 | },
312 | {
313 | "cell_type": "code",
314 | "execution_count": null,
315 | "metadata": {
316 | "collapsed": false
317 | },
318 | "outputs": [],
319 | "source": [
320 | "x=3.141592653589\n",
321 | "print(x)\n",
322 | "\n",
323 | "s=str(x)\n",
324 | "print(s)"
325 | ]
326 | },
327 | {
328 | "cell_type": "markdown",
329 | "metadata": {},
330 | "source": [
331 | "If you only want two significant figures, or you want the whole number of span 15 spaces (to make nicely lined-up colums for a table), you need to use a format string. A format string is just like a regular string, but has special place-holders for your numbers. These start with a percent ('%'), followed by an optional width, an optional precision after a period, and the type (`f` for float, `d` for integer, `s` for string). For example, a float with two decimals:"
332 | ]
333 | },
334 | {
335 | "cell_type": "code",
336 | "execution_count": null,
337 | "metadata": {
338 | "collapsed": false
339 | },
340 | "outputs": [],
341 | "source": [
342 | "fmt = \"This is a float with two decimals: %.2f\"\n",
343 | "print (fmt)"
344 | ]
345 | },
346 | {
347 | "cell_type": "markdown",
348 | "metadata": {},
349 | "source": [
350 | "By itself, the format string didn't do much. In order to inject the number, we use another `%`, but as an operator:"
351 | ]
352 | },
353 | {
354 | "cell_type": "code",
355 | "execution_count": null,
356 | "metadata": {
357 | "collapsed": false
358 | },
359 | "outputs": [],
360 | "source": [
361 | "# two decimal places\n",
362 | "print(fmt % x)"
363 | ]
364 | },
365 | {
366 | "cell_type": "markdown",
367 | "metadata": {},
368 | "source": [
369 | "You may want to make a format string with more than one number or string. You can do this by grouping the arguments into a list with round brackets (called a *tuple*):"
370 | ]
371 | },
372 | {
373 | "cell_type": "code",
374 | "execution_count": null,
375 | "metadata": {
376 | "collapsed": false
377 | },
378 | "outputs": [],
379 | "source": [
380 | "fmt = \"Here is a float: '%.2f', and another '%8.4f', an integer %d, and a string %s\"\n",
381 | "print (fmt % (x, np.pi, 1000, 'look ma, no quotes!'))"
382 | ]
383 | },
384 | {
385 | "cell_type": "markdown",
386 | "metadata": {},
387 | "source": [
388 | "We will not cover the *new* string formatting now available in `python 2.7` and `python 3.x`. It has some advantages over the old style, but both are fully supported by both versions, so we've stuck with the old as it is by far more common. The reference above describes both methods. You can mix and match as you see fit."
389 | ]
390 | },
391 | {
392 | "cell_type": "markdown",
393 | "metadata": {},
394 | "source": [
395 | "# Lists and numpy arrays\n",
396 | "Lists contain a sequence of objects, typically, a sequence of numbers. The are enclosed by square brackets, (e.g., ``[]``), and each item separated by a comma. Here are some examples."
397 | ]
398 | },
399 | {
400 | "cell_type": "code",
401 | "execution_count": null,
402 | "metadata": {
403 | "collapsed": false
404 | },
405 | "outputs": [],
406 | "source": [
407 | "# A list of floats\n",
408 | "x1=[1.,2.,7.,2500.]\n",
409 | "print(x1)\n",
410 | "\n",
411 | "# A list of strings\n",
412 | "x2=['a','b','c','x','y','z']\n",
413 | "print(x2)\n",
414 | "\n",
415 | "# Lists can also contain a mix of object types\n",
416 | "x3=[1.,'a',6,'z']\n",
417 | "print(x3)"
418 | ]
419 | },
420 | {
421 | "cell_type": "markdown",
422 | "metadata": {},
423 | "source": [
424 | "Numpy arrays allow for more functionality than lists. While they may also contain a mix of object types, you will primarily be working with numpy arrays that are comprised of numbers: either integers or floats. For example, you will at some point read in a table of data into a numpy array and do things to it, like add, multiply, etc.\n",
425 | "\n",
426 | "Above, we imported the `numpy` module as `np`. We will use the latter to create arrays."
427 | ]
428 | },
429 | {
430 | "cell_type": "code",
431 | "execution_count": null,
432 | "metadata": {
433 | "collapsed": false
434 | },
435 | "outputs": [],
436 | "source": [
437 | "x=np.array([1.,2.,3.,4.])\n",
438 | "print(x)"
439 | ]
440 | },
441 | {
442 | "cell_type": "code",
443 | "execution_count": null,
444 | "metadata": {
445 | "collapsed": false
446 | },
447 | "outputs": [],
448 | "source": [
449 | "# We can add one number to every element:\n",
450 | "y1=x+2\n",
451 | "print(y1)\n",
452 | "\n",
453 | "# Or raise every element to some power:\n",
454 | "y2=x**2\n",
455 | "print(y2)"
456 | ]
457 | },
458 | {
459 | "cell_type": "markdown",
460 | "metadata": {},
461 | "source": [
462 | "Here is where the real power of numpy arrays comes into play. We can use `np` to carry out all kinds of mathematical tasks that in other programming languages (like ``C``, ``FORTRAN``, etc) would require some kind of loop. Here are some of the most common tasks we'll use.\n",
463 | "\n",
464 | "### Array creation\n",
465 | "\n",
466 | "There are many functions in ``numpy`` that allow you to make arrays from scratch."
467 | ]
468 | },
469 | {
470 | "cell_type": "markdown",
471 | "metadata": {},
472 | "source": [
473 | "We can create an array of zeros:"
474 | ]
475 | },
476 | {
477 | "cell_type": "code",
478 | "execution_count": null,
479 | "metadata": {
480 | "collapsed": false
481 | },
482 | "outputs": [],
483 | "source": [
484 | "x1=np.zeros(5)\n",
485 | "print(x1)"
486 | ]
487 | },
488 | {
489 | "cell_type": "markdown",
490 | "metadata": {},
491 | "source": [
492 | "Or an array of ones:"
493 | ]
494 | },
495 | {
496 | "cell_type": "code",
497 | "execution_count": null,
498 | "metadata": {
499 | "collapsed": false
500 | },
501 | "outputs": [],
502 | "source": [
503 | "x2=np.ones(5)\n",
504 | "print(x2)"
505 | ]
506 | },
507 | {
508 | "cell_type": "markdown",
509 | "metadata": {},
510 | "source": [
511 | "Or an array of pi's:"
512 | ]
513 | },
514 | {
515 | "cell_type": "code",
516 | "execution_count": null,
517 | "metadata": {
518 | "collapsed": false
519 | },
520 | "outputs": [],
521 | "source": [
522 | "x3=np.ones(5)*np.pi\n",
523 | "print(x3)"
524 | ]
525 | },
526 | {
527 | "cell_type": "markdown",
528 | "metadata": {},
529 | "source": [
530 | "We can create a sequence of numbers using np.arange:"
531 | ]
532 | },
533 | {
534 | "cell_type": "code",
535 | "execution_count": null,
536 | "metadata": {
537 | "collapsed": false
538 | },
539 | "outputs": [],
540 | "source": [
541 | "x1=np.arange(5.)\n",
542 | "print(x1)"
543 | ]
544 | },
545 | {
546 | "cell_type": "markdown",
547 | "metadata": {},
548 | "source": [
549 | "Or spaced by a specified amount between two numbers:"
550 | ]
551 | },
552 | {
553 | "cell_type": "code",
554 | "execution_count": null,
555 | "metadata": {
556 | "collapsed": false
557 | },
558 | "outputs": [],
559 | "source": [
560 | "x2=np.arange(0,10.,2)\n",
561 | "print(x2)"
562 | ]
563 | },
564 | {
565 | "cell_type": "markdown",
566 | "metadata": {},
567 | "source": [
568 | "Or _N_ elements equally spaced between two specified numbers:"
569 | ]
570 | },
571 | {
572 | "cell_type": "code",
573 | "execution_count": null,
574 | "metadata": {
575 | "collapsed": false
576 | },
577 | "outputs": [],
578 | "source": [
579 | "x3=np.linspace(0,8.,5)\n",
580 | "print(x3)"
581 | ]
582 | },
583 | {
584 | "cell_type": "markdown",
585 | "metadata": {},
586 | "source": [
587 | "One can also create N-dimensional numpy arrays. For example, images that you read into Python will typically be stored into 2D numpy arrays."
588 | ]
589 | },
590 | {
591 | "cell_type": "code",
592 | "execution_count": null,
593 | "metadata": {
594 | "collapsed": false
595 | },
596 | "outputs": [],
597 | "source": [
598 | "x=np.ones((4,2))\n",
599 | "print(x)\n",
600 | "print(x.shape)"
601 | ]
602 | },
603 | {
604 | "cell_type": "markdown",
605 | "metadata": {},
606 | "source": [
607 | "### Array Math\n",
608 | "\n",
609 | "Just as with spreadsheets, you can do math on entire arrays using operators (``+``,``-``,``*``, and ``/``) and ``numpy`` functions.\n",
610 | "\n",
611 | "We can add, multiply, etc., numpy arrays:"
612 | ]
613 | },
614 | {
615 | "cell_type": "code",
616 | "execution_count": null,
617 | "metadata": {
618 | "collapsed": false
619 | },
620 | "outputs": [],
621 | "source": [
622 | "x4=0.5+x1+x2*x3/2.\n",
623 | "print(x4)"
624 | ]
625 | },
626 | {
627 | "cell_type": "markdown",
628 | "metadata": {},
629 | "source": [
630 | "We also have access to most mathematical functions you'll need:"
631 | ]
632 | },
633 | {
634 | "cell_type": "code",
635 | "execution_count": null,
636 | "metadata": {
637 | "collapsed": false
638 | },
639 | "outputs": [],
640 | "source": [
641 | "x5 = np.power(x4,2)\n",
642 | "print(x5)\n",
643 | "x6 = np.log10(x5)\n",
644 | "print (x6)"
645 | ]
646 | },
647 | {
648 | "cell_type": "markdown",
649 | "metadata": {},
650 | "source": [
651 | "### Matrix Math\n",
652 | "\n",
653 | "``numpy`` can treat 2D arrays as matrices and leverage special compiled libraries so that you can do linear algebra routines easily (and very fast). Here, we construct a rotation matrix and use ``dot`` do do the dot-product. Try changing theta to different angles (in radians!)."
654 | ]
655 | },
656 | {
657 | "cell_type": "code",
658 | "execution_count": null,
659 | "metadata": {
660 | "collapsed": false
661 | },
662 | "outputs": [],
663 | "source": [
664 | "# CCW rotation by 180 degrees\n",
665 | "theta=np.pi/4\n",
666 | "# Rotation matrix\n",
667 | "x=np.array([[np.cos(theta),-np.sin(theta)],\n",
668 | " [np.sin(theta),np.cos(theta)]])\n",
669 | "print(x)\n",
670 | "\n",
671 | "# Lets rotate (1,1) about the origin\n",
672 | "y=np.array([1.,1.])\n",
673 | "z=np.dot(x,y)\n",
674 | "\n",
675 | "print(y)\n",
676 | "print(z)"
677 | ]
678 | },
679 | {
680 | "cell_type": "markdown",
681 | "metadata": {},
682 | "source": [
683 | "The other common tasks matrix tasks can also be done with either ``numpy`` functions, ``numpy.linalg``, or member functions of the object itself"
684 | ]
685 | },
686 | {
687 | "cell_type": "code",
688 | "execution_count": null,
689 | "metadata": {
690 | "collapsed": false
691 | },
692 | "outputs": [],
693 | "source": [
694 | "# Taking the transpose\n",
695 | "x_t = x.T\n",
696 | "print(x_t)\n",
697 | "\n",
698 | "# Computing the inverse\n",
699 | "x_i = np.linalg.inv(x)\n",
700 | "\n",
701 | "# Matrix Multiplication\n",
702 | "I = np.dot(x_i,x)\n",
703 | "print (I)"
704 | ]
705 | },
706 | {
707 | "cell_type": "markdown",
708 | "metadata": {},
709 | "source": [
710 | "### Array Slicing\n",
711 | "\n",
712 | "Often, you need to access elements or sub-arrays within your array. This is referred to as *slicing*. We can select individual elements in an array using indices (note that 0 is the first element and negative indices count backwards from the end):"
713 | ]
714 | },
715 | {
716 | "cell_type": "code",
717 | "execution_count": null,
718 | "metadata": {
719 | "collapsed": false
720 | },
721 | "outputs": [],
722 | "source": [
723 | "x=np.arange(5)\n",
724 | "print(x)\n",
725 | "print(x[0])\n",
726 | "print(x[4])\n",
727 | "print(x[2:]) # 3rd element onward\n",
728 | "print(x[:-1]) # all elements up until the last one\n",
729 | "print(x[-1:]) # last element\n",
730 | "print(x[::-1]) # reverse array"
731 | ]
732 | },
733 | {
734 | "cell_type": "markdown",
735 | "metadata": {},
736 | "source": [
737 | "We can also \"slice\" N-dimensional arrays:"
738 | ]
739 | },
740 | {
741 | "cell_type": "code",
742 | "execution_count": null,
743 | "metadata": {
744 | "collapsed": false
745 | },
746 | "outputs": [],
747 | "source": [
748 | "x=np.arange(8)\n",
749 | "print(x)\n",
750 | "x=x.reshape((4,2))\n",
751 | "print(x)\n",
752 | "\n",
753 | "print(x[0,:])\n",
754 | "print(x[:,0])"
755 | ]
756 | },
757 | {
758 | "cell_type": "markdown",
759 | "metadata": {},
760 | "source": [
761 | "# Control blocks"
762 | ]
763 | },
764 | {
765 | "cell_type": "markdown",
766 | "metadata": {},
767 | "source": [
768 | "So far, we've been running individual sets of commands and looking at their results immediately. Later, we will write a complete *program*, which is really just bundling up instructions into a recipe for solving a given task. But as the tasks we want to perform become more complicated, we need *control blocks*. These allow us to:\n",
769 | "\n",
770 | "* Repeat tasks again and again (loops)\n",
771 | "* Perform tasks only if certain conditions are met (if-else blocks)\n",
772 | "* Group instructions into a single logical task (user-defined functions)\n",
773 | "\n",
774 | "`python` is rather unique in that it uses indenting to indicate the beginning and end of a logical block. This actually forces you to write readable code, which is a really good thing!\n",
775 | "\n",
776 | "### For loops\n",
777 | "\n",
778 | "`for` loops are useful for repeating a series of operations a given number of times. In `python`, you loop over elements of an list or array. So if you want to loop over a sequence of integers (say, they indices of an array), then you would use the `range()` function to generate the list of integers. You might also use the `len()` function if you need the length of the array."
779 | ]
780 | },
781 | {
782 | "cell_type": "code",
783 | "execution_count": null,
784 | "metadata": {
785 | "collapsed": false
786 | },
787 | "outputs": [],
788 | "source": [
789 | "# range(n) creates a list of n elements\n",
790 | "print(range(5))\n",
791 | "\n",
792 | "# We can use it to iterate over a for loop\n",
793 | "for ii in range(5):\n",
794 | " print(ii)"
795 | ]
796 | },
797 | {
798 | "cell_type": "markdown",
799 | "metadata": {},
800 | "source": [
801 | "### Notice that after the line containing the for statement, the subsequent lines are indented to indicate that they are all part of the same block"
802 | ]
803 | },
804 | {
805 | "cell_type": "markdown",
806 | "metadata": {},
807 | "source": [
808 | "You can use a `for` loop to build up a list of elements by appending to an existing list (using its `append()` member function. For example, to create the first N elements of the Fibonacci sequence:"
809 | ]
810 | },
811 | {
812 | "cell_type": "code",
813 | "execution_count": null,
814 | "metadata": {
815 | "collapsed": false
816 | },
817 | "outputs": [],
818 | "source": [
819 | "fib = [1,1]\n",
820 | "N = 100\n",
821 | "for i in range(N-2):\n",
822 | " fib.append(fib[-2]+fib[-1])\n",
823 | "print (fib)"
824 | ]
825 | },
826 | {
827 | "cell_type": "markdown",
828 | "metadata": {},
829 | "source": [
830 | "You may notice (if N is large enough) that you get numbers that have an `L` at the end. `python` has a special type called a *long integer* which allows for abitrarily large numbers.\n",
831 | "\n",
832 | "Here is an example of what **not** to do with for loops (if you can help it). `For` loops are more computationally expensive in python than using the `numpy` functions do do the math. *Always* try to cast the problem in terms of `numpy` math and functions. It will make your code faster. Try making N in the following example larger and larger and you'll see the difference."
833 | ]
834 | },
835 | {
836 | "cell_type": "code",
837 | "execution_count": null,
838 | "metadata": {
839 | "collapsed": false
840 | },
841 | "outputs": [],
842 | "source": [
843 | "# A slightly more complex example of a for loop:\n",
844 | "import time\n",
845 | "N = 100\n",
846 | "x=np.arange(N)\n",
847 | "y=np.zeros(N)\n",
848 | "\n",
849 | "t1 = time.time() # start time\n",
850 | "for ii in range(x.size):\n",
851 | " y[ii]=x[ii]**2\n",
852 | "t2 = time.time() # end time\n",
853 | " \n",
854 | "print (\"for loop took \"+str(t2-t1)+\" seconds\")\n",
855 | "\n",
856 | "# Note that the above could have been acheived more easily with straight array operations:\n",
857 | "t1 = time.time()\n",
858 | "y2=x**2\n",
859 | "t2 = time.time()\n",
860 | "print (\"numpy square took \"+str(t2-t1)+\" seconds\")"
861 | ]
862 | },
863 | {
864 | "cell_type": "markdown",
865 | "metadata": {},
866 | "source": [
867 | "### If blocks\n",
868 | "\n",
869 | "`if` statements act as a way to carry out a task only if a certain _condition_ is met. As with the `for` loops above, everything that is indented after the `if` statement is part of the block of code that is executed if the condition is met. Likewise for the `else` block."
870 | ]
871 | },
872 | {
873 | "cell_type": "code",
874 | "execution_count": null,
875 | "metadata": {
876 | "collapsed": false
877 | },
878 | "outputs": [],
879 | "source": [
880 | "x=5\n",
881 | "if x==5:\n",
882 | " print('Yes! x is 5')\n",
883 | " \n",
884 | "# The two equal signs evaluate whether x is equal to 5. One can also use >, >=, <, <=, != (not equal to)"
885 | ]
886 | },
887 | {
888 | "cell_type": "code",
889 | "execution_count": null,
890 | "metadata": {
891 | "collapsed": true
892 | },
893 | "outputs": [],
894 | "source": [
895 | "x=5\n",
896 | "if x==3:\n",
897 | " print('Yes! x is 3')"
898 | ]
899 | },
900 | {
901 | "cell_type": "markdown",
902 | "metadata": {},
903 | "source": [
904 | "`if-else` statements execute the code in the `if` block if the condition is `true`, otherwise it executes the code in the `else` block:"
905 | ]
906 | },
907 | {
908 | "cell_type": "code",
909 | "execution_count": null,
910 | "metadata": {
911 | "collapsed": false
912 | },
913 | "outputs": [],
914 | "source": [
915 | "x=5\n",
916 | "if x==3:\n",
917 | " print('Yes! x is 3')\n",
918 | "else:\n",
919 | " print('x is not 3')\n",
920 | " print('x is '+str(x))"
921 | ]
922 | },
923 | {
924 | "cell_type": "markdown",
925 | "metadata": {},
926 | "source": [
927 | "One can also have a series of conditions in the form of an `elif` block:"
928 | ]
929 | },
930 | {
931 | "cell_type": "code",
932 | "execution_count": null,
933 | "metadata": {
934 | "collapsed": false
935 | },
936 | "outputs": [],
937 | "source": [
938 | "x=5\n",
939 | "\n",
940 | "if x==2:\n",
941 | " print('Yes! x is 2')\n",
942 | "elif x==3:\n",
943 | " print('Yes! x is 3')\n",
944 | "elif x==4:\n",
945 | " print('Yes! x is 4')\n",
946 | "else:\n",
947 | " print('x is '+str(x))"
948 | ]
949 | },
950 | {
951 | "cell_type": "markdown",
952 | "metadata": {},
953 | "source": [
954 | "You can also have multiple conditions that are evaluated:"
955 | ]
956 | },
957 | {
958 | "cell_type": "code",
959 | "execution_count": null,
960 | "metadata": {
961 | "collapsed": false
962 | },
963 | "outputs": [],
964 | "source": [
965 | "x=5\n",
966 | "\n",
967 | "# '&' below stands for AND: both conditions have to be met to execute the if block\n",
968 | "if (x>2) & (x*2==10):\n",
969 | " print('x is 5')\n",
970 | " \n",
971 | " \n",
972 | "# '|' below stands for OR: either condition can be met to execute the if block\n",
973 | "if (x>7) | (x*2==10):\n",
974 | " print('x is 5')\n"
975 | ]
976 | },
977 | {
978 | "cell_type": "markdown",
979 | "metadata": {},
980 | "source": [
981 | "### Functions\n",
982 | "\n",
983 | "Functions allow you to make a bundle of python statements that are executed whenever the function is called. Each function has arguments you pass in and value(s) that are returned. For example, you've been using the `print` function. There are also some functions above that you have been using. Now we will make our own."
984 | ]
985 | },
986 | {
987 | "cell_type": "code",
988 | "execution_count": null,
989 | "metadata": {
990 | "collapsed": false
991 | },
992 | "outputs": [],
993 | "source": [
994 | "# the function 'myfunc' takes two numbers, x and y, adds them together and returns the results\n",
995 | "def myfunc(x,y):\n",
996 | " z=x+y\n",
997 | " return z\n",
998 | "\n",
999 | "# to call the function, we simply invoke the name and feed it the requisite inputs:\n",
1000 | "g=myfunc(2,3.)\n",
1001 | "print(g)"
1002 | ]
1003 | },
1004 | {
1005 | "cell_type": "code",
1006 | "execution_count": null,
1007 | "metadata": {
1008 | "collapsed": false
1009 | },
1010 | "outputs": [],
1011 | "source": [
1012 | "# you can set input parameters to have a default values\n",
1013 | "def myfunc2(x,y=5.):\n",
1014 | " z=x+y\n",
1015 | " return z\n",
1016 | "\n",
1017 | "g=myfunc2(2.)\n",
1018 | "print(g)\n",
1019 | "\n",
1020 | "g=myfunc2(2.,4.)\n",
1021 | "print(g)"
1022 | ]
1023 | },
1024 | {
1025 | "cell_type": "markdown",
1026 | "metadata": {},
1027 | "source": [
1028 | "Let's put the `if`, `for` and `def` together into one example. Here we will make a Fibonacci list generator. It takes a single argument: the length of the list to make."
1029 | ]
1030 | },
1031 | {
1032 | "cell_type": "code",
1033 | "execution_count": null,
1034 | "metadata": {
1035 | "collapsed": true
1036 | },
1037 | "outputs": [],
1038 | "source": [
1039 | "def Fibonacci(N):\n",
1040 | " # check the input value\n",
1041 | " if N < 2:\n",
1042 | " print(\"Sorry, N must be at least 2\")\n",
1043 | " return None\n",
1044 | " elif N > 1000:\n",
1045 | " print (\"Are you nuts?! That will take too long!\")\n",
1046 | " return None\n",
1047 | " fib = [1,1]\n",
1048 | " for i in range(N-2):\n",
1049 | " fib.append(fib[-1]+fib[-2])\n",
1050 | " return fib"
1051 | ]
1052 | },
1053 | {
1054 | "cell_type": "markdown",
1055 | "metadata": {},
1056 | "source": [
1057 | "Note that in cases where the function fails, we return a special python object called `None`. It's a good thing to use when you want to indicate that somethign didn't go right. Now go ahead and try running the Fibonacci function with different input values."
1058 | ]
1059 | },
1060 | {
1061 | "cell_type": "code",
1062 | "execution_count": null,
1063 | "metadata": {
1064 | "collapsed": false
1065 | },
1066 | "outputs": [],
1067 | "source": [
1068 | "Fibonacci(1001)"
1069 | ]
1070 | },
1071 | {
1072 | "cell_type": "markdown",
1073 | "metadata": {},
1074 | "source": [
1075 | "# Importing Python packages: examples\n",
1076 | "\n",
1077 | "One of the major advantages of `python` is a wealth of specialized packages for doing common scientific tasks. Sure, you could write your own least-squares fitter using what we've shown you so far, but before you attempt anything like that, take a little time to \"google that\" and see if a solution exists already.\n",
1078 | "\n",
1079 | "You will have to `import` Python modules/packages to carry out many of the tasks you will need for your research. As already discussed, `numpy` is probably the most useful. `scipy` and `astropy` are other popular packages. Lets play around with a few of these to give you an idea of how useful they can be.\n",
1080 | "\n",
1081 | "### Random Numbers"
1082 | ]
1083 | },
1084 | {
1085 | "cell_type": "code",
1086 | "execution_count": null,
1087 | "metadata": {
1088 | "collapsed": true
1089 | },
1090 | "outputs": [],
1091 | "source": [
1092 | "# I like to declare all of my imported packages at the top of my script so that I know what is available.\n",
1093 | "# Also note that there are many ways to import packages.\n",
1094 | "\n",
1095 | "import numpy.random as npr\n",
1096 | "from scipy import stats\n",
1097 | "import scipy.interpolate as si\n",
1098 | "from astropy.cosmology import FlatLambdaCDM"
1099 | ]
1100 | },
1101 | {
1102 | "cell_type": "code",
1103 | "execution_count": null,
1104 | "metadata": {
1105 | "collapsed": false,
1106 | "scrolled": true
1107 | },
1108 | "outputs": [],
1109 | "source": [
1110 | "# Random numbers are useful for many tasks.\n",
1111 | "\n",
1112 | "# draw random numbers from a uniform distribution:\n",
1113 | "x1=npr.rand(5)\n",
1114 | "print(x1)\n",
1115 | "\n",
1116 | "# draw random numbers from a normal distribution:\n",
1117 | "x2=npr.randn(5)\n",
1118 | "print(x2)\n",
1119 | "\n",
1120 | "# draw 10 random integers between 0 and 5(exclusive)\n",
1121 | "x3=npr.randint(0,5,10)\n",
1122 | "print(x3)"
1123 | ]
1124 | },
1125 | {
1126 | "cell_type": "markdown",
1127 | "metadata": {},
1128 | "source": [
1129 | "Here is a practical example of using random numbers. Often in statistics, we have to compute the mean of some population based on a limited sample. For example, a survey may ask car drivers their age and make/model of car. A marketing team may want to know the average age of drivers of Ford Mustangs so they can target their audience. Calculating a mean is easy, but what about the uncertainty in that mean? You could compute the population standard deviation, but that pre-supposes the underlying distribution is Gaussian. Another method, that does not make any assumptions about the distribution is *bootstrapping*. Randomly remove values from the data and replace them with copies of other values. Compute a new mean. Do that N times and compute the standard deviation of these *bootstrapped* mean values.\n",
1130 | "\n",
1131 | "Below is an example of bootstrapping a sample to determine the uncertainty on a measurement. In this case, we will compute the mean of a sample of ages, and the uncertainty on the mean."
1132 | ]
1133 | },
1134 | {
1135 | "cell_type": "code",
1136 | "execution_count": null,
1137 | "metadata": {
1138 | "collapsed": false
1139 | },
1140 | "outputs": [],
1141 | "source": [
1142 | "# x below represents a measurement of the ages of N people, where N=x.size\n",
1143 | "x=np.array([19.,20.,22.,19.,21.,24.,35.,22.,21.])\n",
1144 | "# This is the mean age:\n",
1145 | "print(np.mean(x))\n",
1146 | "\n",
1147 | "# Now we \"bootstrap\" to determine the error on this measurement:\n",
1148 | "ntrials=10000 # number of times we will draw a random sample of N ages\n",
1149 | "x_arr=np.zeros(ntrials) # store the mean of each random sample in this array\n",
1150 | "\n",
1151 | "for ii in range(ntrials):\n",
1152 | " # draw N random integers, where N equals the number of samples in x\n",
1153 | " ix=npr.randint(0,x.size,x.size)\n",
1154 | " # subscript the original array with these random indices to get a new sample and compute the mean\n",
1155 | " x_arr[ii]=np.mean(x[ix])\n",
1156 | "\n",
1157 | "# Finally, compute the standard deviation of the array of mean values to get the uncertainty on the *mean* age\n",
1158 | "print(np.std(x_arr))\n"
1159 | ]
1160 | },
1161 | {
1162 | "cell_type": "markdown",
1163 | "metadata": {},
1164 | "source": [
1165 | "### Binning Data\n",
1166 | "\n",
1167 | "Another common statistical task is *binning* data. This is often done on noisy spectra to make \"real\" features more visible. In the following example, we are going to bin galaxies based on their redshift and compute the mean stellar mass for each bin."
1168 | ]
1169 | },
1170 | {
1171 | "cell_type": "code",
1172 | "execution_count": null,
1173 | "metadata": {
1174 | "collapsed": false
1175 | },
1176 | "outputs": [],
1177 | "source": [
1178 | "# This is an example of binning data and computing a particular value for the data in each bin.\n",
1179 | "# The scipy package is used to carry out this task.\n",
1180 | "\n",
1181 | "# Lets make some fake data of galaxies spanning random redshifts between 0`). And the explanation is pretty clear (when you're used to `numpy` arrays). You can't add two arrays unless they have the same shape.\n",
1443 | "\n",
1444 | "These kinds of compile-time bugs are the easiest to fix. You've written something wrong (syntax error), or you've done some illegal operation (like above) and your code grinds to a halt. You fix the problem and then find another and so on until your code runs.\n",
1445 | "\n",
1446 | "But then you might not get the \"right\" answer. Or your code does something unexpected. Or you get a \"divide by zero\" error and it's not completely obvious where things went wrong. That's when you need to do real debugging. There are several approaches to dealing with this:\n",
1447 | "\n",
1448 | "- Print the values of variables throughout the program by simply injecting \"print statements\" in your code. This is easy to do and works \"anywhere\". Usually, with a few of these you can see the dumb mistake you made (but python didn't catch).\n",
1449 | "- Use a debugger. `Spyder` has several debugging tools. You can set checkpoints (where you want your code to stop) and examine the values of variables. But you need to load your program into `Spyder` and run it from there.\n",
1450 | "- If you want to run your script in the Python interpreter and have it leave all the variables available after a run: \n",
1451 | " - `>>> exec(open(\"myscript.py\").read(), globals())`"
1452 | ]
1453 | },
1454 | {
1455 | "cell_type": "markdown",
1456 | "metadata": {},
1457 | "source": [
1458 | "# Final exercise\n",
1459 | "### (if time permits)\n",
1460 | "To wrap this all up, we're going to leave the notebook and get you to write a stand-alone script that can be run from the command-line. The script should:\n",
1461 | "- get the name of a file from command-line argument\n",
1462 | "- open the file and read its contents (columns of numbers) \n",
1463 | "- convert columns in the file into `numpy` arrays,\n",
1464 | "- compute and report the mean of each column.\n",
1465 | "Try to use the concepts we've covered in this tutorial (e.g. make a function that reads columns and converts to arrays). Run the script on the file we created earlier (`a_test_file`).\n",
1466 | "\n",
1467 | "# References\n",
1468 | "\n",
1469 | "- Carnegie python links:\n",
1470 | " - http://obs.carnegiescience.edu/content/python-resources\n",
1471 | " \n",
1472 | "- Python \"experts\" at Carnegie\n",
1473 | " - Shannon Patel (#215)\n",
1474 | " - Chris Burns (#234)\n",
1475 | " - Eduardo Banados (#205)\n",
1476 | " \n",
1477 | "- Google. Chances are someone else had the same question you do, asked it, and had it answered on `stackoverflow`."
1478 | ]
1479 | }
1480 | ],
1481 | "metadata": {
1482 | "kernelspec": {
1483 | "display_name": "Python 2",
1484 | "language": "python",
1485 | "name": "python2"
1486 | },
1487 | "language_info": {
1488 | "codemirror_mode": {
1489 | "name": "ipython",
1490 | "version": 2
1491 | },
1492 | "file_extension": ".py",
1493 | "mimetype": "text/x-python",
1494 | "name": "python",
1495 | "nbconvert_exporter": "python",
1496 | "pygments_lexer": "ipython2",
1497 | "version": "2.7.11"
1498 | }
1499 | },
1500 | "nbformat": 4,
1501 | "nbformat_minor": 0
1502 | }
1503 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | Authors: Chris Burns, Shannon Patel and Amber (Carnegie Observatories)
2 |
3 | Just a safe place to put our ipython notebook tutorials for the summer bootcamp held at the Carnegie Observatories.
4 |
5 | The notebooks make reference to the data files that are also located in this repository. There are also some sample scripts that can be useful seeing how to do some of the exercises.
6 |
--------------------------------------------------------------------------------
/Skyfit.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Putting it All Together\n",
8 | "\n",
9 | "This notebook is a case study in working with python and several 3rd-party modules. There are *many* ways to attack a problem such as this; this is simply one way. The point is to illustrate how you can get existing modules to do the heavy-lifting for you and that visualization is a powerful diagnostic tool. Try not to get caught up in the details of the model (read the citation if you're really interested).\n",
10 | "\n",
11 | "This notebook requires the following modules:\n",
12 | "* `numpy`: dealing with arrays of numbers and mathematics\n",
13 | "* `scipy`: collection of scientific algorithms\n",
14 | "* `matplotlib`: de-facto plotting module\n",
15 | "* `pandas`: module for organizing arrays of number into tables\n",
16 | "* `ephem`: A module that computes positions of astronomical objects on the sky.\n",
17 | "* `sqlalchemy`: An abstraction layer module for dealing with many flavours of SQL databases.\n",
18 | "* `pymysql`: Module for making connections to mySQL databases.\n",
19 | "* `bokeh`: another module for plotting, with emphasis on interactive visualization\n",
20 | "\n",
21 | "The problem I needed to solve: predict the background sky brightness caused by the moon at a given location in the sky on a given date. This is to help plan observations at the telescope. As with all problems of this type, we need to do several things:\n",
22 | "\n",
23 | "* Download/import/munge training data\n",
24 | "* Model the training data\n",
25 | "* Extract model parameters\n",
26 | "* Graph the result(s) to see how well we do, maybe modify the model\n",
27 | "* Use final model and parameters to make future predictions\n",
28 | "\n",
29 | "### 1) The Data\n",
30 | "\n",
31 | "In this case, the data to model is roughly 10 years of photometry from the Carnegie Supernova Project (CSP). Each and every measurement of the flux from a standard star has an associated estimate of the sky background (which must be subtracted from the raw counts of the star). This data is located in a MySQL database. MySQL has its own internal language for querying data, joining tables together, filtering, etc. We won't cover that here, but simply give you the SQL query needed to extract the data we need: coordinates (RA,DEC) on the sky, date of observation, and observed sky brightness. Let's start by getting the data. We'll use `pandas` and its build-in SQL engine. First, we need to create a connection engine to the MySQL database using its `URL`:"
32 | ]
33 | },
34 | {
35 | "cell_type": "code",
36 | "execution_count": null,
37 | "metadata": {
38 | "collapsed": true
39 | },
40 | "outputs": [],
41 | "source": [
42 | "from sqlalchemy import create_engine\n",
43 | "engine = create_engine('mysql+pymysql://bootcamp:pmactoob@kepler.obs.carnegiescience.edu/Phot')"
44 | ]
45 | },
46 | {
47 | "cell_type": "markdown",
48 | "metadata": {},
49 | "source": [
50 | "Next, we create an SQL query and have `pandas` retrieve the results into a data frame. Again, there's no time to explain the intricacies of SQL syntax, but you can find many tutorials online if you want to learn about it."
51 | ]
52 | },
53 | {
54 | "cell_type": "code",
55 | "execution_count": null,
56 | "metadata": {
57 | "collapsed": false
58 | },
59 | "outputs": [],
60 | "source": [
61 | "import pandas as pd\n",
62 | "query = '''\n",
63 | "select MAGINS.jd,MAGINS.alpha as RA, MAGINS.delta as Decl,\n",
64 | " -2.5*log10(MAGINS.sky / 0.435/0.435) + 2.5*log10(MAGINS.expt) + MAGFIT1.zp + 25 as magsky\n",
65 | "from (MAGINS,MAGFIT1)\n",
66 | "where MAGINS.night=MAGFIT1.night and MAGINS.filt=MAGFIT1.filt\n",
67 | " and MAGINS.filt='B' and MAGINS.field like 'SA%%' and MAGINS.sky > 0\n",
68 | "'''\n",
69 | "data = pd.read_sql_query(query, engine)"
70 | ]
71 | },
72 | {
73 | "cell_type": "markdown",
74 | "metadata": {},
75 | "source": [
76 | "We can take a quick look at what's in this `DataFrame` by printing out the first few rows."
77 | ]
78 | },
79 | {
80 | "cell_type": "code",
81 | "execution_count": null,
82 | "metadata": {
83 | "collapsed": false
84 | },
85 | "outputs": [],
86 | "source": [
87 | "print(data[0:10])"
88 | ]
89 | },
90 | {
91 | "cell_type": "markdown",
92 | "metadata": {},
93 | "source": [
94 | "Let's have a look at the distribution of sky brightnesses to make sure they \"make sense\". The units should be magnitudes per square-arc-second and be on order of 22 or so, but should be smaller for bright time. Since we're just doing a quick-look, we can use `pandas`' built-in histogram plotter."
95 | ]
96 | },
97 | {
98 | "cell_type": "code",
99 | "execution_count": null,
100 | "metadata": {
101 | "collapsed": false
102 | },
103 | "outputs": [],
104 | "source": [
105 | "% matplotlib inline\n",
106 | "data.hist('magsky', bins=50)"
107 | ]
108 | },
109 | {
110 | "cell_type": "markdown",
111 | "metadata": {},
112 | "source": [
113 | "As you can see, there is peak near 22 mag/square-arc-sec, as expected, but a broader peak at brighter backgrounds. We expect this is due to moonlight. Something to think about: why would this be bi-modal?\n",
114 | "\n",
115 | "Whatever model we use is going to require knowledge of the moon's position and phase. There are mathematical formulae for this, but we'll use a 3rd-party module called `pyephem` which gives us a powerful ephemeris tool to compute locations of objects in the night sky. To begin, we load `pyephem` and create an observer object corresponding to Las Campanas Observatory."
116 | ]
117 | },
118 | {
119 | "cell_type": "code",
120 | "execution_count": null,
121 | "metadata": {
122 | "collapsed": false
123 | },
124 | "outputs": [],
125 | "source": [
126 | "import ephem\n",
127 | "OBS = ephem.Observer()\n",
128 | "OBS.long = \"-70.6926\" # longitude (negative --> West)\n",
129 | "OBS.lat = \"-20.0146\" # latitude (negative --> South)\n",
130 | "OBS.elev = 2380 # elevation in meters"
131 | ]
132 | },
133 | {
134 | "cell_type": "markdown",
135 | "metadata": {},
136 | "source": [
137 | "Next, we create a moon object, which is built-in to `ephem`. We'll also need a 'sky' object to represent the locations of our standard star fields."
138 | ]
139 | },
140 | {
141 | "cell_type": "code",
142 | "execution_count": null,
143 | "metadata": {
144 | "collapsed": true
145 | },
146 | "outputs": [],
147 | "source": [
148 | "moon = ephem.Moon()\n",
149 | "sky = ephem.FixedBody()"
150 | ]
151 | },
152 | {
153 | "cell_type": "markdown",
154 | "metadata": {},
155 | "source": [
156 | "We can now compute the phase of the moon for each observation that was made. Unfortunately, `ephem` does not allow one to do this to an array of dates all at once, so we use a for-loop. Also, ephem uses Dublin Julian Day rather than Julian Day as date parameter. The difference between the two is 2415020 days."
157 | ]
158 | },
159 | {
160 | "cell_type": "code",
161 | "execution_count": null,
162 | "metadata": {
163 | "collapsed": false
164 | },
165 | "outputs": [],
166 | "source": [
167 | "import numpy as np\n",
168 | "JDoff = 2415020\n",
169 | "phases = [] # place to hold results\n",
170 | "for JD in data['jd']:\n",
171 | " OBS.date = JD - JDoff\n",
172 | " moon.compute(OBS)\n",
173 | " phases.append(moon.moon_phase)\n",
174 | "phases = np.array(phases)\n",
175 | "data['phase'] = pd.Series(phases, index=data.index)"
176 | ]
177 | },
178 | {
179 | "cell_type": "markdown",
180 | "metadata": {},
181 | "source": [
182 | "Now that we have the phase information, let's see if our earlier hypothesis about the moon being a source of background light is valid. We'll plot one versus the other."
183 | ]
184 | },
185 | {
186 | "cell_type": "code",
187 | "execution_count": null,
188 | "metadata": {
189 | "collapsed": false
190 | },
191 | "outputs": [],
192 | "source": [
193 | "data.plot.scatter('phase','magsky')"
194 | ]
195 | },
196 | {
197 | "cell_type": "markdown",
198 | "metadata": {},
199 | "source": [
200 | "Great! There's a definite trend there. We can also split up the data based on the phase and plot the resulting histograms together. You can run this next snippet of code with different `phasecut` values to see how they separate out."
201 | ]
202 | },
203 | {
204 | "cell_type": "code",
205 | "execution_count": null,
206 | "metadata": {
207 | "collapsed": false
208 | },
209 | "outputs": [],
210 | "source": [
211 | "from matplotlib.pyplot import gca,legend\n",
212 | "phasecut = 0.8\n",
213 | "res = data[data.phase>phasecut].hist('magsky', bins=50, label='> %.2f illum.' % phasecut, alpha=0.7)\n",
214 | "ax = gca()\n",
215 | "res = data[data.phase \\pi/2$. To see this, we can simply plot `X(Z)` as a function of 'Z':"
437 | ]
438 | },
439 | {
440 | "cell_type": "code",
441 | "execution_count": null,
442 | "metadata": {
443 | "collapsed": false
444 | },
445 | "outputs": [],
446 | "source": [
447 | "from numpy import linspace\n",
448 | "from matplotlib.pyplot import plot, xlabel, ylabel\n",
449 | "Z = linspace(0, 3*pi/4, 100) # make a range of Zenith angles\n",
450 | "plot(Z*180/pi, X(Z), '-')\n",
451 | "xlabel('Zenith angle (degrees)')\n",
452 | "ylabel('Airmass')"
453 | ]
454 | },
455 | {
456 | "cell_type": "markdown",
457 | "metadata": {},
458 | "source": [
459 | "So the airmass (amount of air the light travels through) increases as you get to the horizon ($Z=90^\\circ$), but then decreases. That's not the right behaviour. What we need is a better formula for the airmass as a function of zenith angle. Wikipedia to the rescue! There is an article on this here: [https://en.wikipedia.org/wiki/Air_mass_(astronomy)](https://en.wikipedia.org/wiki/Air_mass_(astronomy%29).\n",
460 | "\n",
461 | "### 5) Exercise\n",
462 | "\n",
463 | "As an exercise, go to the above Wikipedia article and try out different airmass functions. Most of them are relatively simple to code. All you need to do is go to the cell above, where we define `X(Z)`, and change it. Then select `Kernel->Restart & Run all` from the Jupyter notebook menu. The graphs will all update and you can see how things change.\n",
464 | "\n",
465 | "### 6) Final Remarks\n",
466 | "\n",
467 | "At this point you might be feeling overwhelmed. How did I know which modules to use? How did I know how to use them? The answer: Google, ADS, and 20+ years (eek!) of experience coding in Python. I also neglected to show all the dead-ends and mistakes I made on the way to getting the final solution, all the emails I sent to Kevin asking about the details of his paper, and advice I got from Shannon about using Bokeh.\n",
468 | "\n",
469 | "Before you start tackling a particular problem it's well worth your time to research whether there is already a solution \"out there\" that you can use or modify for your use. It has never been so easy to do this, thanks to search engines (Google, et al.), data/software catalogs (PyPI, et al.), discussion groups (stackoverflow, et al.) and even social media (python facebook group, etc). And your friendly neighborhood python experts are there to make helpful suggestions.\n",
470 | "\n",
471 | "Don't re-invent the wheel, but improve it by all means."
472 | ]
473 | }
474 | ],
475 | "metadata": {
476 | "kernelspec": {
477 | "display_name": "Python 2",
478 | "language": "python",
479 | "name": "python2"
480 | },
481 | "language_info": {
482 | "codemirror_mode": {
483 | "name": "ipython",
484 | "version": 2
485 | },
486 | "file_extension": ".py",
487 | "mimetype": "text/x-python",
488 | "name": "python",
489 | "nbconvert_exporter": "python",
490 | "pygments_lexer": "ipython2",
491 | "version": "2.7.13"
492 | }
493 | },
494 | "nbformat": 4,
495 | "nbformat_minor": 2
496 | }
497 |
--------------------------------------------------------------------------------
/Visualization.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Visualization\n",
8 | "\n",
9 | "In this boot camp we're going to learn how to:\n",
10 | "- Read in data tables (the easy way!)\n",
11 | "- Select different samples from the data set\n",
12 | "- Make figures to visualize the data"
13 | ]
14 | },
15 | {
16 | "cell_type": "markdown",
17 | "metadata": {},
18 | "source": [
19 | "# Reading in data\n",
20 | "Often, you will be given data in the form of an ASCII table. This table will have multiple rows that indicate a star, or a galaxy, or a particle in a simulation, and the columns indicate different properties for the object such as its ID, brightness, mass, or distance. Before we can select different samples of objects to study, we need to read in this data. There are many ways to do this in Python: numpy, pandas, and astropy are commonly used packages.\n",
21 | "\n",
22 | "- Pandas: http://pandas.pydata.org/pandas-docs/stable/10min.html\n",
23 | "- Astropy: http://docs.astropy.org/en/stable/io/ascii/\n",
24 | "\n",
25 | "Let's first import these packages. Again, use CTRL-ENTER to execute each cell."
26 | ]
27 | },
28 | {
29 | "cell_type": "code",
30 | "execution_count": null,
31 | "metadata": {
32 | "collapsed": false
33 | },
34 | "outputs": [],
35 | "source": [
36 | "import numpy as np\n",
37 | "import pandas as pd\n",
38 | "from astropy.io import ascii"
39 | ]
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "metadata": {},
44 | "source": [
45 | "Let's start with an example of reading in data into a Pandas `DataFrame`. Just as a `numpy` array is like a python list with extra super-powers, you can think of a Pandas `DataFrame` as an augmented `numpy` array with extra member functions to help you work with the data.\n",
46 | "\n",
47 | "The data we will be using in this notebook should have been downloaded with the rest of the package into a \"data\" folder, so we use this for the file's location (e.g., your `Downloads` folder) in the next command. If the fine isn't there, you'll get an error."
48 | ]
49 | },
50 | {
51 | "cell_type": "code",
52 | "execution_count": null,
53 | "metadata": {
54 | "collapsed": false
55 | },
56 | "outputs": [],
57 | "source": [
58 | "data1=pd.read_csv('data/SNIa_DM.dat',delim_whitespace=True, skiprows=4)"
59 | ]
60 | },
61 | {
62 | "cell_type": "markdown",
63 | "metadata": {},
64 | "source": [
65 | "Note that we had to tell Pandas to skip the first 4 rows (comments) and use white-space (spaces and tabs) to delimit the columns.\n",
66 | "\n",
67 | "We can now look at the properties of data1. Such as its dimensions:"
68 | ]
69 | },
70 | {
71 | "cell_type": "code",
72 | "execution_count": null,
73 | "metadata": {
74 | "collapsed": false
75 | },
76 | "outputs": [],
77 | "source": [
78 | "print(data1.shape)"
79 | ]
80 | },
81 | {
82 | "cell_type": "markdown",
83 | "metadata": {},
84 | "source": [
85 | "There are 281 rows and 5 columns. What are the names of the columns?"
86 | ]
87 | },
88 | {
89 | "cell_type": "code",
90 | "execution_count": null,
91 | "metadata": {
92 | "collapsed": false
93 | },
94 | "outputs": [],
95 | "source": [
96 | "print(data1.columns.values)"
97 | ]
98 | },
99 | {
100 | "cell_type": "markdown",
101 | "metadata": {},
102 | "source": [
103 | "And what if we want to peak at all of the properties of the first 5 SN? Pandas `DataFrames` act just like arrays in this respect. Use the `numpy` array slicing you learned in our last tutorial to print out only the first 5 rows of `data1`."
104 | ]
105 | },
106 | {
107 | "cell_type": "code",
108 | "execution_count": null,
109 | "metadata": {
110 | "collapsed": false
111 | },
112 | "outputs": [],
113 | "source": [
114 | "print(data1[0:5])"
115 | ]
116 | },
117 | {
118 | "cell_type": "markdown",
119 | "metadata": {},
120 | "source": [
121 | "We can also just select specific columns by using the names you found above."
122 | ]
123 | },
124 | {
125 | "cell_type": "code",
126 | "execution_count": null,
127 | "metadata": {
128 | "collapsed": false
129 | },
130 | "outputs": [],
131 | "source": [
132 | "print(data1['DM'][0:5])"
133 | ]
134 | },
135 | {
136 | "cell_type": "markdown",
137 | "metadata": {},
138 | "source": [
139 | "Now see if you can print out just the first 5 rows of the `'DM'` column. Try other combinations of column names and row slices.\n",
140 | "\n",
141 | "Note that the index is always shown here. If we just want the raw data in the form of a numpy array, we just do the following:"
142 | ]
143 | },
144 | {
145 | "cell_type": "code",
146 | "execution_count": null,
147 | "metadata": {
148 | "collapsed": false
149 | },
150 | "outputs": [],
151 | "source": [
152 | "print(data1['DM'].values)"
153 | ]
154 | },
155 | {
156 | "cell_type": "markdown",
157 | "metadata": {},
158 | "source": [
159 | "Next, we do a similar exercise reading in the data into an astropy Table:"
160 | ]
161 | },
162 | {
163 | "cell_type": "code",
164 | "execution_count": null,
165 | "metadata": {
166 | "collapsed": false
167 | },
168 | "outputs": [],
169 | "source": [
170 | "data2=ascii.read('data/SNIa_DM.dat')\n",
171 | "\n",
172 | "print(data2.colnames)\n",
173 | "print(data2[:5])\n",
174 | "print(data2['DM'][:5])"
175 | ]
176 | },
177 | {
178 | "cell_type": "markdown",
179 | "metadata": {},
180 | "source": [
181 | "Note that the `astropy.ascii` package requires fewer arguments (it's a little smarter about comments and delimiters), but `astropy` tables have a bit less functionality. Luckily, the `astropy` developers recognise that Pandas is pretty awesome, so you can convert an Astropy `Tables` object to a Pandas `DataFrame` object! (You can also convert Pandas DFs to Astropy Tables)"
182 | ]
183 | },
184 | {
185 | "cell_type": "code",
186 | "execution_count": null,
187 | "metadata": {
188 | "collapsed": false
189 | },
190 | "outputs": [],
191 | "source": [
192 | "data3=data2.to_pandas()\n",
193 | "print(data3[:5])"
194 | ]
195 | },
196 | {
197 | "cell_type": "markdown",
198 | "metadata": {},
199 | "source": [
200 | "## Writing data\n",
201 | "Writing data in Pandas is just as simple as reading it in:"
202 | ]
203 | },
204 | {
205 | "cell_type": "code",
206 | "execution_count": null,
207 | "metadata": {
208 | "collapsed": false
209 | },
210 | "outputs": [],
211 | "source": [
212 | "data3.to_csv('data/output.dat',index=False,sep=' ')\n",
213 | "# the 'index' keyword gives you the option of also printing the Pandas index\n",
214 | "# the 'sep' keyword specifies the delimiter between columns"
215 | ]
216 | },
217 | {
218 | "cell_type": "markdown",
219 | "metadata": {},
220 | "source": [
221 | "# Selecting Data\n",
222 | "Most of the time you will be dealing with subsets of your data set, e.g., galaxies at a certain redshift, stars at a particular distance or brightness, you want to get rid of outliers, etc. We therefore need to select the desired sample using conditionals. We'll use a Pandas DataFrame as an example. Here, we create a `pandas` column called `ix` that contains True/False values for each row in `data1` based on conditions: True if redshift is less than 0.02 and the survey is from CSP (equals 1), False otherwise. This new array, if used as an index, will pick out only those rows that are True. This is often referred to as creating a 'mask'."
223 | ]
224 | },
225 | {
226 | "cell_type": "code",
227 | "execution_count": null,
228 | "metadata": {
229 | "collapsed": false
230 | },
231 | "outputs": [],
232 | "source": []
233 | },
234 | {
235 | "cell_type": "markdown",
236 | "metadata": {},
237 | "source": [
238 | "We could also have accomplished the above using numpy's `where` function:"
239 | ]
240 | },
241 | {
242 | "cell_type": "code",
243 | "execution_count": null,
244 | "metadata": {
245 | "collapsed": false
246 | },
247 | "outputs": [],
248 | "source": [
249 | "ix=np.where((data1['zcmb']<0.02) & (data1['survey']==1))[0]\n",
250 | "\n",
251 | "# Note the different way of subscripting the Pandas DataFrame here\n",
252 | "print(data1.loc[ix])"
253 | ]
254 | },
255 | {
256 | "cell_type": "markdown",
257 | "metadata": {},
258 | "source": [
259 | "# Plotting\n",
260 | "The most popular plotting package in Python is `matplotlib` but there are several others one might want to explore. Here we will show you how to make basic plots in matplotlib. The references below are useful:\n",
261 | "\n",
262 | "- http://matplotlib.org/gallery.html\n",
263 | "- http://matplotlib.org/api/pyplot_api.html\n",
264 | "\n",
265 | "First, let's invoke an iPython magic command (i.e., %) so that the plots that are made will show up in this notebook."
266 | ]
267 | },
268 | {
269 | "cell_type": "code",
270 | "execution_count": null,
271 | "metadata": {
272 | "collapsed": false
273 | },
274 | "outputs": [],
275 | "source": [
276 | "%matplotlib inline"
277 | ]
278 | },
279 | {
280 | "cell_type": "markdown",
281 | "metadata": {},
282 | "source": [
283 | "`pyplot` is the main plotting function in `matplotlib`, it is commonly imported on its own as `plt`."
284 | ]
285 | },
286 | {
287 | "cell_type": "code",
288 | "execution_count": null,
289 | "metadata": {
290 | "collapsed": true
291 | },
292 | "outputs": [],
293 | "source": [
294 | "import matplotlib.pyplot as plt"
295 | ]
296 | },
297 | {
298 | "cell_type": "markdown",
299 | "metadata": {},
300 | "source": [
301 | "Now let's make some fake data to play with and plot. This is just a suggestion, try your own functions, maybe even some random numbers!"
302 | ]
303 | },
304 | {
305 | "cell_type": "code",
306 | "execution_count": null,
307 | "metadata": {
308 | "collapsed": false
309 | },
310 | "outputs": [],
311 | "source": [
312 | "x=np.arange(10.)\n",
313 | "y=x**2"
314 | ]
315 | },
316 | {
317 | "cell_type": "markdown",
318 | "metadata": {},
319 | "source": [
320 | "Armed with this data, this is now probably the quickest way to make a figure:"
321 | ]
322 | },
323 | {
324 | "cell_type": "code",
325 | "execution_count": null,
326 | "metadata": {
327 | "collapsed": false
328 | },
329 | "outputs": [],
330 | "source": [
331 | "plt.plot(x,y,'bo')\n",
332 | "\n",
333 | "# The string 'bo' above indicates that the points will be blue(b) circles(o)"
334 | ]
335 | },
336 | {
337 | "cell_type": "markdown",
338 | "metadata": {},
339 | "source": [
340 | "Note that `matplotlib` does a lot of things automatically for you, like setting the limits on the axes. As well as the interval between major tick marks. These are all things that you can adjust manually with more code.\n",
341 | "\n",
342 | "Below is the long form to getting the same symbols as above. By including more code, you can start to tinker around with different aspects of the plot. Many of matplotlib's optional arguments have a long and short form (e.g., you can specify `linestyle='-'` or the shorter `ls='-'`. Try different linestyles: '--', '-.'. Try different symbols: 's', 'd', '*'."
343 | ]
344 | },
345 | {
346 | "cell_type": "code",
347 | "execution_count": null,
348 | "metadata": {
349 | "collapsed": false
350 | },
351 | "outputs": [],
352 | "source": [
353 | "plt.plot(x,y,color='blue',marker='o',ms=6,ls='None')"
354 | ]
355 | },
356 | {
357 | "cell_type": "markdown",
358 | "metadata": {},
359 | "source": [
360 | "Yet another way to make the same plot, using the `scatter` function:"
361 | ]
362 | },
363 | {
364 | "cell_type": "code",
365 | "execution_count": null,
366 | "metadata": {
367 | "collapsed": false
368 | },
369 | "outputs": [],
370 | "source": [
371 | "plt.scatter(x,y,s=10*np.sqrt(y),c=np.log10(x),edgecolors='black')"
372 | ]
373 | },
374 | {
375 | "cell_type": "markdown",
376 | "metadata": {},
377 | "source": [
378 | "The above examples are the quickest, easiest ways to produce a figure in `matplotlib`. However, you'll find that including more lines of code will make things easier down the road as it will provide for more functionality. Below, we invoke the `subplots` function, which returns `figure` and `axis` objects. We can also play around with the colors, symbols, and line styles. This way of using `matplotlib` is more *pythonic*. The previous way was more akin to using `MATLAB` and it was the developer's intention to make it more intuitive. Most examples use the *pythonic* methods."
379 | ]
380 | },
381 | {
382 | "cell_type": "code",
383 | "execution_count": null,
384 | "metadata": {
385 | "collapsed": false
386 | },
387 | "outputs": [],
388 | "source": [
389 | "fig, ax = plt.subplots()\n",
390 | "\n",
391 | "# We then use ax to do our plotting\n",
392 | "ax.plot(x,y,color='royalblue',marker='*',ms=15,ls='None')\n",
393 | "ax.plot(x,y,color='red',ls='-',lw=2,alpha=0.5)\n",
394 | "plt.invert_yaxis()"
395 | ]
396 | },
397 | {
398 | "cell_type": "markdown",
399 | "metadata": {},
400 | "source": [
401 | "So this produces the same kind of plot as above, but now we can use `ax` to play with the axes:"
402 | ]
403 | },
404 | {
405 | "cell_type": "code",
406 | "execution_count": null,
407 | "metadata": {
408 | "collapsed": false
409 | },
410 | "outputs": [],
411 | "source": [
412 | "fig, ax = plt.subplots()\n",
413 | "ax.set_ylim((0,200)) # set limits on y-axis\n",
414 | "ax.set_xlabel('x',fontsize=15) # set label for x-axis\n",
415 | "ax.set_ylabel('y',fontsize=15) # set label for y-axis\n",
416 | "ax.plot(x,y,color='mediumseagreen',marker='s',ms=15,ls='None')\n",
417 | "ax.plot(x,y,color='darkorange',ls='--',lw=2)"
418 | ]
419 | },
420 | {
421 | "cell_type": "markdown",
422 | "metadata": {},
423 | "source": [
424 | "Often, you'll want an axis to be on a logarithmic scale:"
425 | ]
426 | },
427 | {
428 | "cell_type": "code",
429 | "execution_count": null,
430 | "metadata": {
431 | "collapsed": false
432 | },
433 | "outputs": [],
434 | "source": [
435 | "fig, ax = plt.subplots()\n",
436 | "\n",
437 | "ax.set_yscale('log') # set y-axis to be in log\n",
438 | "\n",
439 | "ax.set_xlabel('x',fontsize=15)\n",
440 | "ax.set_ylabel('y',fontsize=15)\n",
441 | "ax.plot(x,y,color='gold',mec='mediumvioletred',mew=2,marker='p',ms=25,ls='None')\n",
442 | "ax.plot(x,y,color='mediumvioletred',ls='-.',lw=2)"
443 | ]
444 | },
445 | {
446 | "cell_type": "markdown",
447 | "metadata": {},
448 | "source": [
449 | "## Multiple figures\n",
450 | "\n",
451 | "Sometimes you'll want two figures side-by-side, or a grid of 3x3, etc, etc..."
452 | ]
453 | },
454 | {
455 | "cell_type": "code",
456 | "execution_count": null,
457 | "metadata": {
458 | "collapsed": false
459 | },
460 | "outputs": [],
461 | "source": [
462 | "# two figures, side by side:\n",
463 | "fig, axarr = plt.subplots(1,2,figsize=(10,5))\n",
464 | "\n",
465 | "# axarr is an array of axis objects, with each element representing one subplot\n",
466 | "\n",
467 | "# first subplot\n",
468 | "ax=axarr[0]\n",
469 | "ax.plot(x,y,color='blue',marker='s',ms=15,ls='None')\n",
470 | "\n",
471 | "# second subplot\n",
472 | "ax=axarr[1]\n",
473 | "ax.plot(x,y,color='red',marker='*',ms=15,ls='None')\n",
474 | "\n",
475 | "# change title on first subplot\n",
476 | "ax=axarr[0]\n",
477 | "ax.set_xlabel('x',fontsize=16)"
478 | ]
479 | },
480 | {
481 | "cell_type": "markdown",
482 | "metadata": {},
483 | "source": [
484 | "Using subplots, which returns an axis array, is useful for going back and forth between different figures. Note how we can go back to `axarr[0]` at the end and change the label on the x axis.\n",
485 | "\n",
486 | "If you have a bunch of images or the same type of figure for multiple objects, it helps to make a giant grid of subplots. And rather than manually declaring a new subplot each time, it helps to automate the process with `for` loop(s)."
487 | ]
488 | },
489 | {
490 | "cell_type": "code",
491 | "execution_count": null,
492 | "metadata": {
493 | "collapsed": false
494 | },
495 | "outputs": [],
496 | "source": [
497 | "nrow=2\n",
498 | "ncol=3\n",
499 | "fig, axarr = plt.subplots(nrow,ncol,figsize=(9,6))\n",
500 | "\n",
501 | "for ii in range(nrow):\n",
502 | " for jj in range(ncol):\n",
503 | " ax=axarr[ii,jj]\n",
504 | " if ii==nrow-1: ax.set_yscale('log')\n",
505 | " ax.set_xlabel('x')\n",
506 | " ax.set_ylabel('y')\n",
507 | " ax.plot(x,y,color='blue',marker='s',ms=5,ls='None')\n",
508 | " ax.plot(x,y,color='red',ls='--',lw=2)\n",
509 | "fig.tight_layout()"
510 | ]
511 | },
512 | {
513 | "cell_type": "markdown",
514 | "metadata": {},
515 | "source": [
516 | "`subplots` has many useful features, like the the `sharex` and `sharey` keywords, which allow you to declare that the axes for each subplot have the same scale. Invoking `subplots_adjust` can then optionally allow you to squish the subplots together. "
517 | ]
518 | },
519 | {
520 | "cell_type": "code",
521 | "execution_count": null,
522 | "metadata": {
523 | "collapsed": false
524 | },
525 | "outputs": [],
526 | "source": [
527 | "nrow=2\n",
528 | "ncol=3\n",
529 | "fig, axarr = plt.subplots(nrow,ncol,figsize=(9,6),sharex=True,sharey=True)\n",
530 | "plt.subplots_adjust(hspace=0,wspace=0)\n",
531 | "\n",
532 | "for ii in range(nrow):\n",
533 | " for jj in range(ncol):\n",
534 | " ax=axarr[ii,jj]\n",
535 | " if ii==nrow-1:\n",
536 | " ax.set_xlabel('x',fontsize=16)\n",
537 | " if jj==0:\n",
538 | " ax.set_ylabel('y',fontsize=16)\n",
539 | " ax.plot(x,y,color='blue',marker='s',ms=5,ls='None')\n",
540 | " ax.plot(x,y,color='red',ls='--',lw=2)\n",
541 | " ax.tick_params(axis='both',direction='inout', top=True, right=True)"
542 | ]
543 | },
544 | {
545 | "cell_type": "markdown",
546 | "metadata": {},
547 | "source": [
548 | "# Labeling\n",
549 | "Legends, text labels, Latex"
550 | ]
551 | },
552 | {
553 | "cell_type": "code",
554 | "execution_count": null,
555 | "metadata": {
556 | "collapsed": false
557 | },
558 | "outputs": [],
559 | "source": [
560 | "x=np.arange(10)+1.\n",
561 | "y1=x**2\n",
562 | "y2=np.sqrt(x)\n",
563 | "\n",
564 | "fig, ax = plt.subplots()\n",
565 | "ax.set_yscale('log')\n",
566 | "# The `label` keyword below is used to specify the label for the particular data set\n",
567 | "ax.plot(x,y1,color='blue',mec='orange',mew=2,marker='*',ms=20,ls='None',label='$y=x^2$')\n",
568 | "ax.plot(x,y2,color='red',marker='s',ms=15,ls='None',label='$y=\\sqrt{x}$')\n",
569 | "ax.text(6,10,'$y=x^{n}$',fontsize=24)\n",
570 | "ax.legend(loc='upper left',fontsize=16,numpoints=1)"
571 | ]
572 | },
573 | {
574 | "cell_type": "markdown",
575 | "metadata": {},
576 | "source": [
577 | "## Other useful types of figures: \n",
578 | "- Error bars: We're scientists! Almost every data point you put on a figure should have an error bar.\n",
579 | "- Histograms: useful for looking at distribution of values for a particular variable\n",
580 | "- Images: e.g., cutouts of stars, galaxies, nebulae, etc.\n",
581 | "- Many others that we don't have time to cover here!\n",
582 | "\n",
583 | "Error bars:"
584 | ]
585 | },
586 | {
587 | "cell_type": "code",
588 | "execution_count": null,
589 | "metadata": {
590 | "collapsed": false
591 | },
592 | "outputs": [],
593 | "source": [
594 | "# first, generate some fake data\n",
595 | "x=np.arange(10)+1.\n",
596 | "y=x**2\n",
597 | "\n",
598 | "xerr=np.zeros(10)+0.5 # fixed error in x\n",
599 | "yerr=np.sqrt(y) # Poisson error in y\n",
600 | "\n",
601 | "plt.errorbar(x,y,xerr=xerr,yerr=yerr,marker='o',ls='None')"
602 | ]
603 | },
604 | {
605 | "cell_type": "markdown",
606 | "metadata": {},
607 | "source": [
608 | "Histogram of Gaussians:"
609 | ]
610 | },
611 | {
612 | "cell_type": "code",
613 | "execution_count": null,
614 | "metadata": {
615 | "collapsed": false
616 | },
617 | "outputs": [],
618 | "source": [
619 | "import numpy.random as npr\n",
620 | "\n",
621 | "x1=npr.randn(10000) # mean=0, std=1.0\n",
622 | "x2=npr.randn(10000)*0.5+1. # mean=1, std=0.5\n",
623 | "\n",
624 | "bins=np.linspace(-5.,5.,21)\n",
625 | "\n",
626 | "info1 = plt.hist(x1,bins=bins, color='red',alpha=0.5,lw=0)\n",
627 | "info2 = plt.hist(x2,bins=bins, color='blue',alpha=0.5,lw=0)"
628 | ]
629 | },
630 | {
631 | "cell_type": "markdown",
632 | "metadata": {},
633 | "source": [
634 | " Display an image from a FITS file. First download this file: [im3433.fits](https://dl.dropboxusercontent.com/u/28917100/carnegie/summer2016/im3433.fits)"
635 | ]
636 | },
637 | {
638 | "cell_type": "code",
639 | "execution_count": null,
640 | "metadata": {
641 | "collapsed": false
642 | },
643 | "outputs": [],
644 | "source": [
645 | "# Display an image from a FITS file\n",
646 | "\n",
647 | "from astropy.io import fits\n",
648 | "\n",
649 | "im3433=fits.open('data/im3433.fits')\n",
650 | "im=im3433[0].data # the first extension (i.e., index 0), contains the image data\n",
651 | "\n",
652 | "fig, ax = plt.subplots()\n",
653 | "ax.imshow(im,interpolation='none',origin='lower',cmap='gray')\n",
654 | "ax.get_xaxis().set_visible(False) # comment these two lines to see what happens\n",
655 | "ax.get_yaxis().set_visible(False)"
656 | ]
657 | },
658 | {
659 | "cell_type": "markdown",
660 | "metadata": {},
661 | "source": [
662 | "# Doing some math on images\n",
663 | "\n",
664 | "Here's another example using some supernova data, which will have been downloaded as part of our github pakcage.\n",
665 | "\n",
666 | "We will plot these two images side-by-side. We specify `vmin=-40` and `vmax=40` in the `imshow()` function, which will set appropriate\n",
667 | "limits on the color map (there are saturated pixels that will cause the image to be washed out otherwise). We also choose the\n",
668 | "reverse color map (`gray_r`). Question: where's the supernova? "
669 | ]
670 | },
671 | {
672 | "cell_type": "code",
673 | "execution_count": null,
674 | "metadata": {
675 | "collapsed": false
676 | },
677 | "outputs": [],
678 | "source": [
679 | "im1 = fits.open('data/SN2011iv_B_SWO_DC_2011_12_11SN.fits')\n",
680 | "im2 = fits.open('data/SN2011iv_B_template.fits')\n",
681 | "fig,ax = plt.subplots(1,2, figsize=(15,8))\n",
682 | "ax[0].imshow(im1[0].data, vmin=-40,vmax=40, cmap='gray_r')\n",
683 | "ax[1].imshow(im2[0].data, vmin=-40, vmax=40, cmap='gray_r')"
684 | ]
685 | },
686 | {
687 | "cell_type": "markdown",
688 | "metadata": {},
689 | "source": [
690 | "These two images are from different epochs. So if you subtract one from the other and plot out the results, the supernova (and anything else that changed) should stand out. Try this. There's a surprise."
691 | ]
692 | },
693 | {
694 | "cell_type": "code",
695 | "execution_count": null,
696 | "metadata": {
697 | "collapsed": false
698 | },
699 | "outputs": [],
700 | "source": []
701 | },
702 | {
703 | "cell_type": "markdown",
704 | "metadata": {},
705 | "source": [
706 | "## Saving a figure to a directory\n",
707 | "Inline plotting like what we've been doing above is useful for quick viewing, but most of the time you'll want to save your figure to a folder somewhere and in a specific format. Let's save one of the figures above to disk:"
708 | ]
709 | },
710 | {
711 | "cell_type": "code",
712 | "execution_count": null,
713 | "metadata": {
714 | "collapsed": false
715 | },
716 | "outputs": [],
717 | "source": [
718 | "x=np.arange(10)+1.\n",
719 | "y1=x**2\n",
720 | "y2=np.sqrt(x)\n",
721 | "\n",
722 | "fig, ax = plt.subplots()\n",
723 | "ax.set_yscale('log')\n",
724 | "ax.plot(x,y1,color='blue',marker='*',ms=15,ls='None',label='$y=x^2$')\n",
725 | "ax.plot(x,y2,color='red',marker='s',ms=15,ls='None',label='$y=\\sqrt{x}$')\n",
726 | "ax.text(6,10,'Hi there!')\n",
727 | "ax.legend(loc='upper left',fontsize=16,numpoints=1)\n",
728 | "\n",
729 | "# Note these two lines\n",
730 | "fig.savefig('exampe.pdf',format='pdf')\n",
731 | "plt.close()"
732 | ]
733 | },
734 | {
735 | "cell_type": "markdown",
736 | "metadata": {
737 | "collapsed": true
738 | },
739 | "source": [
740 | "# Analyze one of these data sets by making figures...\n",
741 | "Now that you've learned the basics of Python and its plotting package, `matplotlib`, download one of these data sets, make some figures, and tell us what you see:\n",
742 | "- **Galaxies**: \n",
743 | " - Data: [galsample1.dat](https://dl.dropboxusercontent.com/u/28917100/carnegie/summer2016/galsample1.dat), [galsample2.dat](https://dl.dropboxusercontent.com/u/28917100/carnegie/summer2016/galsample2.dat)\n",
744 | " - The two files represent two different galaxy samples. The columns represent, (1) the galaxy ID, (2) right ascension (RA) in units of decimal degrees, (3) declination (Dec) in units of decimal degrees (i.e., latter two columns are similar to longitude and latitude, but project on the sky), (4) the redshift (i.e., a measure of the recession velocity due to the Doppler shift of light from the galaxy. If one assumes a cosmology, the redshift translates into a distance), (5), the total stellar mass of the galaxy in units of $\\log(M/M_{\\odot})$, and (6) the star formation rate of the galaxy in units of $M_{\\odot}~{\\rm yr}^{-1}$.\n",
745 | " - What is the difference between the two samples? Qualitatively, and quantitatively?\n",
746 | "- **Supernovae**:\n",
747 | " - Data: [SNIa_DM.dat](https://dl.dropboxusercontent.com/u/28917100/carnegie/summer2016/SNIa_DM.dat)\n",
748 | " - There are five columns: SN name, redshift (`zcmb`), Distance modulus (`DM`), error `eDM`, and a survey number.\n",
749 | " - Try plotting a Hubble diagram (`DM` vs. `zcmb`) with errorbars.\n",
750 | " - Use logarithmic x-axis to make the Hubble diagram straight.\n",
751 | " - Plot the two different surveys in different colors/symbols.\n",
752 | " - Try plotting different *theoretical* Hubble laws (with different Hubble constants) with lines:\n",
753 | " $$ DM = 5\\log_{10}\\left(\\frac{cz}{H_0}\\right) + 25 $$\n",
754 | " - Do chi-by-eye to meausre $H_0$. Note that if $H_0$ is in $km/s/Mpc$, then $c = 3\\times 10^{5} km/s$."
755 | ]
756 | },
757 | {
758 | "cell_type": "code",
759 | "execution_count": null,
760 | "metadata": {
761 | "collapsed": true
762 | },
763 | "outputs": [],
764 | "source": []
765 | }
766 | ],
767 | "metadata": {
768 | "kernelspec": {
769 | "display_name": "Python 2",
770 | "language": "python",
771 | "name": "python2"
772 | },
773 | "language_info": {
774 | "codemirror_mode": {
775 | "name": "ipython",
776 | "version": 2
777 | },
778 | "file_extension": ".py",
779 | "mimetype": "text/x-python",
780 | "name": "python",
781 | "nbconvert_exporter": "python",
782 | "pygments_lexer": "ipython2",
783 | "version": "2.7.13"
784 | }
785 | },
786 | "nbformat": 4,
787 | "nbformat_minor": 0
788 | }
789 |
--------------------------------------------------------------------------------
/Visualizing Graphs with NetworkX.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/AstronomerAmber/Python-Tutorials/baaec88f3ff1d94d3a304a9fd589621aaa62cb43/Visualizing Graphs with NetworkX.pdf
--------------------------------------------------------------------------------
/data/SN2011iv_B_SWO_DC_2011_12_11SN.fits:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/AstronomerAmber/Python-Tutorials/baaec88f3ff1d94d3a304a9fd589621aaa62cb43/data/SN2011iv_B_SWO_DC_2011_12_11SN.fits
--------------------------------------------------------------------------------
/data/SN2011iv_B_template.fits:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/AstronomerAmber/Python-Tutorials/baaec88f3ff1d94d3a304a9fd589621aaa62cb43/data/SN2011iv_B_template.fits
--------------------------------------------------------------------------------
/data/SNIa_DM.dat:
--------------------------------------------------------------------------------
1 | # Distance modulii for Carnegie Supernova Project (CSP) and
2 | # Harvard Center for Astrophysics (CfA) type Ia supernovae
3 | # survey = 1 --> CfA
4 | # survey = 2 --> CSP
5 | SN zcmb DM eDM survey
6 | SN2001C 0.01109 33.295 0.086 1
7 | SN2001G 0.01733 34.204 0.041 1
8 | SN2001N 0.02207 34.807 0.213 1
9 | SN2001V 0.01604 33.699 0.035 1
10 | SN2001ah 0.05833 36.929 0.117 1
11 | SN2001ay 0.03091 35.686 0.147 1
12 | SN2001az 0.04060 36.180 0.132 1
13 | SN2001bf 0.01520 33.936 0.135 1
14 | SN2001da 0.01598 34.261 0.142 1
15 | SN2001eh 0.03620 35.905 0.121 1
16 | SN2001en 0.01494 33.883 0.137 1
17 | SN2001ep 0.01306 33.561 0.036 1
18 | SN2001fe 0.01449 33.794 0.035 1
19 | SN2001fh 0.01211 33.503 0.046 1
20 | SN2001gb 0.02679 35.241 0.200 1
21 | SN2001gc 0.01931 34.420 0.202 1
22 | SN2001ic 0.04300 36.293 0.159 1
23 | SN2001ie 0.03120 35.527 0.140 1
24 | SN2002bo 0.00529 31.775 0.070 1
25 | SN2002bz 0.03782 36.013 0.156 1
26 | SN2002cd 0.00974 33.297 0.075 1
27 | SN2002ck 0.03031 35.454 0.136 1
28 | SN2002cr 0.01038 33.175 0.048 1
29 | SN2002de 0.02827 35.325 0.040 1
30 | SN2002dj 0.01044 32.614 0.031 1
31 | SN2002do 0.01517 34.128 0.167 1
32 | SN2002dp 0.01045 33.203 0.040 1
33 | SN2002es 0.02791 35.194 0.169 1
34 | SN2002fb 0.01474 34.113 0.208 1
35 | SN2002fk 0.00662 32.406 0.035 1
36 | SN2002ha 0.01305 33.774 0.034 1
37 | SN2002hd 0.03151 35.527 0.145 1
38 | SN2002he 0.02484 35.103 0.093 1
39 | SN2002hu 0.02924 35.773 0.038 1
40 | SN2002hw 0.01633 34.364 0.066 1
41 | SN2002jy 0.01871 34.810 0.136 1
42 | SN2002kf 0.01947 34.627 0.127 1
43 | SN2003D 0.02324 34.897 0.207 1
44 | SN2003K 0.02187 34.706 0.199 1
45 | SN2003U 0.02818 35.326 0.049 1
46 | SN2003W 0.02113 34.467 0.060 1
47 | SN2003ae 0.03377 35.745 0.161 1
48 | SN2003ai 0.03692 35.933 0.097 1
49 | SN2003cg 0.00531 31.816 0.092 1
50 | SN2003ch 0.02934 35.576 0.056 1
51 | SN2003du 0.00665 32.710 0.036 1
52 | SN2003ic 0.05458 36.809 0.132 1
53 | SN2003it 0.02396 35.047 0.037 1
54 | SN2003iv 0.03358 35.897 0.039 1
55 | SN2003kc 0.03408 35.742 0.152 1
56 | SN2003kf 0.00769 32.386 0.036 1
57 | SN2004as 0.03209 35.908 0.062 1
58 | SN2004bg 0.02212 34.782 0.169 1
59 | SN2004ef 0.02977 35.425 0.048 1
60 | SN2004ef 0.02977 35.497 0.022 2
61 | SN2004eo 0.01474 33.959 0.032 2
62 | SN2004ey 0.01463 34.008 0.021 2
63 | SN2004fu 0.00867 33.025 0.111 1
64 | SN2004gc 0.03071 35.417 0.123 2
65 | SN2004gs 0.02828 35.525 0.027 2
66 | SN2004gu 0.04690 36.345 0.044 2
67 | SN2005A 0.01834 34.305 0.096 2
68 | SN2005M 0.02558 34.818 0.157 1
69 | SN2005M 0.02558 35.107 0.028 2
70 | SN2005W 0.00795 32.810 0.027 2
71 | SN2005ag 0.08008 37.594 0.026 2
72 | SN2005al 0.01329 34.042 0.028 2
73 | SN2005am 0.00897 32.374 0.127 1
74 | SN2005am 0.00897 32.415 0.026 2
75 | SN2005be 0.03558 35.832 0.111 2
76 | SN2005bg 0.02416 34.909 0.110 2
77 | SN2005bl 0.02511 35.174 0.084 2
78 | SN2005bo 0.01501 34.004 0.027 2
79 | SN2005cf 0.00705 32.198 0.030 1
80 | SN2005dv 0.01064 33.500 0.141 1
81 | SN2005el 0.01488 33.824 0.031 1
82 | SN2005el 0.01488 34.034 0.033 2
83 | SN2005eq 0.02835 35.331 0.028 2
84 | SN2005eq 0.02835 35.381 0.040 1
85 | SN2005eu 0.03412 35.629 0.082 1
86 | SN2005hc 0.04450 36.522 0.029 2
87 | SN2005hf 0.04210 36.250 0.142 1
88 | SN2005hj 0.05695 36.850 0.044 2
89 | SN2005hk 0.01183 33.944 0.088 1
90 | SN2005hk 0.01183 33.959 0.078 2
91 | SN2005iq 0.03293 35.872 0.027 2
92 | SN2005iq 0.03293 35.880 0.037 1
93 | SN2005ir 0.07523 37.444 0.039 2
94 | SN2005kc 0.01389 33.846 0.068 1
95 | SN2005kc 0.01389 34.019 0.031 2
96 | SN2005ke 0.00448 31.584 0.066 1
97 | SN2005ke 0.00448 31.646 0.057 2
98 | SN2005ki 0.02037 34.592 0.035 1
99 | SN2005ki 0.02037 34.627 0.026 2
100 | SN2005ku 0.04418 36.348 0.117 2
101 | SN2005ls 0.02051 34.568 0.122 1
102 | SN2005lu 0.03129 35.684 0.153 1
103 | SN2005lu 0.03129 35.734 0.121 2
104 | SN2005mc 0.02601 35.216 0.120 2
105 | SN2005mc 0.02601 35.351 0.043 1
106 | SN2005ms 0.02592 35.243 0.037 1
107 | SN2005mz 0.01703 34.134 0.062 1
108 | SN2005na 0.02681 35.089 0.027 2
109 | SN2005na 0.02681 35.138 0.118 1
110 | SN2006D 0.00964 32.859 0.034 1
111 | SN2006D 0.00964 32.895 0.024 2
112 | SN2006H 0.01348 33.896 0.160 1
113 | SN2006N 0.01427 33.945 0.123 1
114 | SN2006S 0.03294 35.795 0.040 1
115 | SN2006X 0.00632 30.854 0.101 1
116 | SN2006X 0.00632 30.986 0.097 2
117 | SN2006ac 0.02395 34.765 0.029 1
118 | SN2006ak 0.03911 36.024 0.126 1
119 | SN2006ar 0.02292 34.940 0.056 1
120 | SN2006ax 0.01796 34.182 0.033 1
121 | SN2006ax 0.01796 34.280 0.025 2
122 | SN2006az 0.03147 35.487 0.036 1
123 | SN2006bb 0.02584 35.093 0.121 1
124 | SN2006bd 0.02680 35.133 0.171 1
125 | SN2006bd 0.02680 35.284 0.102 2
126 | SN2006bh 0.01049 33.366 0.025 2
127 | SN2006bk 0.04997 36.007 0.100 1
128 | SN2006bq 0.02146 34.768 0.138 1
129 | SN2006br 0.02552 35.324 0.144 2
130 | SN2006br 0.02552 35.392 0.149 1
131 | SN2006bt 0.03248 35.690 0.050 1
132 | SN2006bw 0.03077 35.465 0.131 1
133 | SN2006bz 0.02880 35.120 0.084 1
134 | SN2006cc 0.03267 35.827 0.044 1
135 | SN2006cf 0.04229 36.286 0.145 1
136 | SN2006cg 0.02894 35.192 0.150 1
137 | SN2006cm 0.01526 34.442 0.089 1
138 | SN2006cp 0.02332 34.941 0.065 1
139 | SN2006cq 0.04914 36.622 0.048 1
140 | SN2006cs 0.02437 34.847 0.185 1
141 | SN2006cz 0.04253 36.084 0.139 1
142 | SN2006ef 0.01703 34.322 0.164 1
143 | SN2006ef 0.01703 34.378 0.151 2
144 | SN2006ej 0.01934 34.518 0.147 1
145 | SN2006ej 0.01934 34.645 0.124 2
146 | SN2006em 0.01853 34.225 0.185 1
147 | SN2006en 0.03080 35.518 0.122 1
148 | SN2006eq 0.04837 36.613 0.112 2
149 | SN2006et 0.02165 34.614 0.046 1
150 | SN2006et 0.02165 34.731 0.027 2
151 | SN2006eu 0.02293 34.784 0.179 1
152 | SN2006ev 0.02760 35.433 0.114 1
153 | SN2006ev 0.02760 35.493 0.116 2
154 | SN2006fw 0.08199 37.808 0.109 2
155 | SN2006gj 0.02697 35.477 0.060 1
156 | SN2006gj 0.02697 35.710 0.035 2
157 | SN2006gr 0.03348 35.796 0.040 1
158 | SN2006gt 0.04364 36.311 0.127 1
159 | SN2006gt 0.04364 36.367 0.043 2
160 | SN2006ha 0.02943 35.500 0.165 1
161 | SN2006hb 0.01534 33.936 0.134 1
162 | SN2006hb 0.01534 33.985 0.122 2
163 | SN2006hx 0.04440 36.502 0.143 2
164 | SN2006is 0.03142 35.432 0.116 2
165 | SN2006je 0.03712 35.967 0.137 1
166 | SN2006ke 0.01713 34.610 0.207 1
167 | SN2006kf 0.02080 34.738 0.049 1
168 | SN2006kf 0.02080 34.817 0.029 2
169 | SN2006le 0.01727 34.034 0.026 1
170 | SN2006lf 0.01297 33.150 0.040 1
171 | SN2006lu 0.05440 36.756 0.102 2
172 | SN2006mo 0.03601 35.960 0.113 1
173 | SN2006mq 0.00405 30.126 0.056 1
174 | SN2006mr 0.00551 31.924 0.106 2
175 | SN2006nz 0.03715 36.040 0.110 1
176 | SN2006oa 0.05891 37.089 0.047 1
177 | SN2006ob 0.05826 36.909 0.045 2
178 | SN2006ob 0.05826 37.002 0.053 1
179 | SN2006or 0.02198 34.985 0.136 1
180 | SN2006os 0.03208 35.651 0.148 1
181 | SN2006os 0.03208 35.688 0.116 2
182 | SN2006ot 0.05232 36.763 0.142 1
183 | SN2006py 0.05664 36.838 0.061 2
184 | SN2006qo 0.03078 35.372 0.031 1
185 | SN2006sr 0.02298 34.990 0.035 1
186 | SN2006td 0.01505 34.249 0.030 1
187 | SN2006te 0.03214 35.670 0.116 1
188 | SN2007A 0.01646 34.327 0.031 2
189 | SN2007F 0.02416 35.005 0.023 1
190 | SN2007N 0.01400 33.764 0.170 1
191 | SN2007N 0.01400 34.063 0.087 2
192 | SN2007O 0.03659 35.861 0.122 1
193 | SN2007R 0.03132 35.646 0.124 1
194 | SN2007S 0.01502 33.915 0.042 2
195 | SN2007S 0.01502 33.949 0.041 1
196 | SN2007ae 0.06425 37.105 0.122 1
197 | SN2007af 0.00629 31.908 0.023 1
198 | SN2007af 0.00629 32.011 0.025 2
199 | SN2007ai 0.03199 35.684 0.122 1
200 | SN2007ai 0.03199 35.788 0.040 2
201 | SN2007al 0.01326 33.946 0.149 2
202 | SN2007al 0.01332 33.816 0.170 1
203 | SN2007ap 0.01616 34.221 0.124 1
204 | SN2007ar 0.05335 36.785 0.128 1
205 | SN2007as 0.01790 34.262 0.025 2
206 | SN2007au 0.02086 34.692 0.049 1
207 | SN2007ax 0.00764 32.682 0.213 1
208 | SN2007ax 0.00764 32.945 0.080 2
209 | SN2007ba 0.03905 35.650 0.043 2
210 | SN2007ba 0.03905 35.898 0.142 1
211 | SN2007bc 0.02185 34.729 0.040 1
212 | SN2007bc 0.02185 34.764 0.028 2
213 | SN2007bd 0.03183 35.470 0.037 1
214 | SN2007bd 0.03183 35.650 0.035 2
215 | SN2007bm 0.00744 32.152 0.049 1
216 | SN2007bm 0.00744 32.300 0.043 2
217 | SN2007bz 0.02274 35.270 0.078 1
218 | SN2007ca 0.01507 34.258 0.041 1
219 | SN2007ca 0.01507 34.391 0.029 2
220 | SN2007cg 0.03417 35.696 0.131 2
221 | SN2007cg 0.03419 35.668 0.146 1
222 | SN2007ci 0.01922 34.442 0.063 1
223 | SN2007co 0.02656 35.150 0.029 1
224 | SN2007cp 0.03772 36.033 0.138 1
225 | SN2007hj 0.01288 33.868 0.032 2
226 | SN2007hx 0.07887 37.727 0.107 2
227 | SN2007jd 0.07191 37.534 0.111 2
228 | SN2007jg 0.03658 36.184 0.030 2
229 | SN2007jh 0.04027 36.213 0.065 2
230 | SN2007le 0.00552 32.175 0.031 2
231 | SN2007mm 0.06887 36.963 0.089 2
232 | SN2007nq 0.04390 36.227 0.032 2
233 | SN2007ol 0.05488 37.023 0.075 2
234 | SN2007on 0.00618 31.538 0.040 2
235 | SN2007qe 0.01886 34.988 0.039 1
236 | SN2007so 0.02893 35.378 0.092 2
237 | SN2007sr 0.00666 31.406 0.131 1
238 | SN2007sr 0.00666 31.501 0.123 2
239 | SN2007st 0.02065 34.526 0.136 2
240 | SN2007ux 0.03200 35.653 0.034 2
241 | SN2008C 0.01707 34.249 0.136 2
242 | SN2008L 0.01886 34.206 0.132 1
243 | SN2008O 0.03932 36.142 0.044 2
244 | SN2008R 0.01289 33.733 0.035 2
245 | SN2008af 0.03411 35.709 0.131 1
246 | SN2008ar 0.02725 35.259 0.030 2
247 | SN2008bc 0.01571 33.966 0.023 2
248 | SN2008bd 0.03128 35.306 0.113 2
249 | SN2008bf 0.02508 34.982 0.024 2
250 | SN2008bi 0.01439 34.070 0.194 2
251 | SN2008bq 0.03444 35.822 0.028 2
252 | SN2008bt 0.01658 34.079 0.049 2
253 | SN2008bz 0.06133 37.119 0.099 2
254 | SN2008cc 0.01012 32.968 0.130 2
255 | SN2008cd 0.00846 33.211 0.259 2
256 | SN2008cf 0.04686 36.364 0.102 2
257 | SN2008ff 0.01867 34.207 0.040 2
258 | SN2008fl 0.01942 34.524 0.119 2
259 | SN2008fp 0.00629 31.839 0.039 2
260 | SN2008fr 0.03791 35.949 0.110 2
261 | SN2008fu 0.05183 36.757 0.117 2
262 | SN2008fw 0.00951 33.008 0.127 2
263 | SN2008gg 0.03107 35.529 0.111 2
264 | SN2008gl 0.03295 35.725 0.024 2
265 | SN2008go 0.06131 37.020 0.030 2
266 | SN2008gp 0.03282 35.544 0.028 2
267 | SN2008hj 0.03672 35.904 0.025 2
268 | SN2008hu 0.05056 36.742 0.046 2
269 | SN2008hv 0.01358 33.822 0.028 2
270 | SN2008ia 0.02254 34.960 0.029 2
271 | SN2009D 0.02465 35.047 0.030 2
272 | SN2009F 0.01291 34.027 0.076 2
273 | SN2009I 0.02543 35.574 0.062 2
274 | SN2009P 0.02632 35.202 0.058 2
275 | SN2009Y 0.01010 32.842 0.035 2
276 | SN2009aa 0.02848 35.314 0.029 2
277 | SN2009ab 0.01086 33.595 0.027 2
278 | SN2009ad 0.02832 35.234 0.028 2
279 | SN2009ag 0.00925 33.226 0.036 2
280 | SN2009al 0.02328 35.058 0.064 2
281 | SN2009cz 0.02198 34.891 0.035 2
282 | SN2009ds 0.02047 34.454 0.034 2
283 | SN2009le 0.01703 34.232 0.030 2
284 |
--------------------------------------------------------------------------------
/data/im3433.fits:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/AstronomerAmber/Python-Tutorials/baaec88f3ff1d94d3a304a9fd589621aaa62cb43/data/im3433.fits
--------------------------------------------------------------------------------
/scripts/plot_Hubble.py:
--------------------------------------------------------------------------------
1 | '''
2 | Plot a Hubble diagram
3 | (python considers tripple-quotes as a super-string that can span several lines
4 | so it's very handy for long comments).
5 | '''
6 |
7 | from matplotlib import pyplot as plt
8 | from matplotlib import rcParams # lets you change defaults
9 | import numpy as np
10 | from astropy.io import ascii
11 |
12 | rcParams['font.size'] = 18 # Nice big fonts!
13 | rcParams['font.family'] = 'serif' # My preference
14 |
15 | data = ascii.read('SNIa_DM.dat')
16 |
17 | fig,ax = plt.subplots()
18 |
19 | # Let's differentiate between CfA (survey==1) and CSP (survey==2) data
20 | labels = ['CfA','CSP']
21 | colors = ['blue','red']
22 | symbols = ['o', 's']
23 |
24 | # group the data by survey. I just looked this up in the astropy docs
25 | gdata = data.group_by('survey')
26 |
27 | # we can now do a loop to plot the (in this case) 2 surveys
28 | for i in [0,1]:
29 | ax.errorbar(gdata.groups[i]['zcmb'], gdata.groups[i]['DM'], fmt=symbols[i],
30 | yerr=gdata.groups[i]['eDM'], capsize=0, mec='black', mfc=colors[i],
31 | label=labels[i])
32 | ax.legend(loc='upper left')
33 |
34 | # Now let's plot some Hubble laws, one line for each different value of the
35 | # Hubble constant.
36 | zz = np.linspace(data['zcmb'].min(),data['zcmb'].max(), 1000)
37 | for Ho in [50,60,70,80,90,100]:
38 | # For Ho in km/s/Mpc, the Hubble distance modulus is:
39 | HDM = 5.0*np.log10(zz*3e5/Ho) + 25
40 | ax.plot(zz, HDM, '-', color='k')
41 | # Place a little label at the end of each
42 | ax.text(zz[-1], HDM[-1], str(Ho), va='center', ha='left', fontsize=10)
43 | ax.set_xlabel('$z_{cmb}$', fontsize=18)
44 | ax.set_ylabel('Distance Modulus', fontsize=16)
45 |
46 | # Make space for everything
47 | plt.tight_layout()
48 | fig.savefig('Hubble.pdf')
49 |
50 | ax.set_xscale('log')
51 | fig.savefig('Hubble_log.pdf')
52 |
53 |
--------------------------------------------------------------------------------
/scripts/plot_diff.py:
--------------------------------------------------------------------------------
1 | '''
2 | Plot the difference image between two epochs of NGC 1404.
3 | '''
4 |
5 | from astropy.io import fits # package for loading FITS files
6 | from matplotlib import pyplot as plt # plotting
7 | from matplotlib import cm # color maps
8 | import numpy as np # math, stats, etc
9 |
10 | f1 = fits.open('SN2011iv_B_SWO_DC_2011_12_11SN.fits')
11 | f2 = fits.open('SN2011iv_B_template.fits')
12 |
13 | image1 = f1[0].data # Data from the primary FITS HDU
14 | image2 = f2[0].data # (see http://fits.gsfc.nasa.gov/fits_primer.html)
15 |
16 | # image data from the telescope often has saturated/bad pixels that make
17 | # the real objects we're interested hard to see. Use percentile to figure
18 | # out where most of the pixel values lie (1% -> 99%)
19 | vmin1,vmax1 = np.percentile(np.ravel(image1), (1,99))
20 | vmin2,vmax2 = np.percentile(np.ravel(image2), (1,99))
21 |
22 | # Make a nice wide figure
23 | fig = plt.figure(figsize=(18,6))
24 | # 3x1 panels
25 | ax1 = fig.add_subplot(131)
26 | # Use the vmin,vmax calculated above. Also use a reverse greyscale color
27 | # map and put the origin of the coordinates at lower left.
28 | ax1.imshow(image1, cmap=cm.gray_r, vmin=vmin1, vmax=vmax1, origin='lower')
29 | ax1.set_xlabel('X (pix)')
30 | ax1.set_ylabel('Y (pix)')
31 | ax1.set_title('NGC 1404 Dec. 11 2011')
32 |
33 | ax2 = fig.add_subplot(132)
34 | ax2.imshow(image2, cmap=cm.gray_r, vmin=vmin2, vmax=vmax2, origin='lower')
35 | ax2.set_xlabel('X (pix)')
36 | ax2.set_ylabel('Y (pix)')
37 | ax2.set_title('NGC 1404 Stack 2007')
38 |
39 | ax3 = fig.add_subplot(133)
40 | ax3.imshow(image1-image2, cmap=cm.gray_r, vmin=vmin1, vmax=vmax1,
41 | origin='lower')
42 | ax3.set_xlabel('X (pix)')
43 | ax3.set_ylabel('Y (pix)')
44 | ax3.set_title('Difference')
45 |
46 | fig.savefig('SN2011iv_diff.pdf')
47 |
--------------------------------------------------------------------------------
/skyfit_spyder.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python2
2 | # -*- coding: utf-8 -*-
3 | """
4 | Created on Thu Jun 15 09:07:08 2017
5 |
6 | @author: cburns
7 |
8 | Fit the sky background at LCO using CSP I and II data
9 | """
10 |
11 | from sqlalchemy import create_engine
12 | import pandas as pd
13 | import ephem
14 | from numpy import power,absolute,cos,sin,pi,exp,log10,array
15 | from scipy.optimize import leastsq
16 | import sys,os,string
17 |
18 | # I like putting all my functions at the top of the script and save the main
19 | # script for below
20 | def X(Z):
21 | # return the airmass as a function of zenith angle in radians
22 | return power(1-0.96*power(sin(Z),2),-0.5)
23 |
24 | def modelsky(alpha, rho, kx, Z, Zm, mdark):
25 | # Given the variables, compute the sky brightness in mag/square-arc-sec
26 | # due to the moon
27 | Istar = power(10, -0.4*(3.84+0.026*absolute(alpha)+4e-9*power(alpha,4)))
28 | frho = power(10, 5.36)*(1.06 + power(cos(rho),2))+power(10, 6.15-rho*180./pi/40)
29 | Bmoon = frho*Istar*power(10,-0.4*kx*X(Zm))*(1-power(10,-0.4*kx*X(Z)))
30 | Bdark = 34.08*exp(20.723 - 0.92104*mdark)*power(10,-0.4*kx*(X(Z)-1))*X(Z)
31 | return mdark - 2.5*log10((Bmoon+Bdark)/Bdark)
32 |
33 | def fitfunc(p, alpha, rho, Z, Zm, magsky):
34 | # function needed by scipiy.optimize.leastsq
35 | mdark,kx = p
36 | return magsky - modelsky(alpha, rho, kx, Z, Zm, mdark)
37 |
38 | # __MAIN__
39 |
40 | # Which filter do we want to fit?
41 | filt = sys.argv[1]
42 | engine = create_engine('mysql+pymysql://bootcamp:pmactoob@kepler.obs.carnegiescience.edu/Phot')
43 |
44 | query = '''
45 | select MAGINS.jd,MAGINS.alpha as RA, MAGINS.delta as Decl, MAGINS.airm,
46 | -2.5*log10(MAGINS.sky / 0.435/0.435) + 2.5*log10(MAGINS.expt) + MAGFIT1.zp + 25 as magsky
47 | from (MAGINS,MAGFIT1)
48 | where MAGINS.night=MAGFIT1.night and MAGINS.filt=MAGFIT1.filt
49 | and MAGINS.filt='{}' and MAGINS.field like 'SA%%' and MAGINS.sky > 0'''
50 |
51 | # do the query and get the data as a Pandas dataFrame
52 | data = pd.read_sql_query(query.format(filt), engine)
53 |
54 | # Now we need a loop to compute all the angles using ephem
55 | OBS = ephem.Observer()
56 | OBS.long = "-70.6926" # longitude (negative --> West)
57 | OBS.lat = "-20.0146" # latitude (negative --> South)
58 | OBS.elev = 2380 # elevation in meters
59 | JDoff = 2415020
60 |
61 | sky = ephem.FixedBody()
62 | moon = ephem.Moon()
63 | sun = ephem.Sun()
64 | alpha = []
65 | rho = []
66 | Z = []
67 | Zm = []
68 | sun = ephem.Sun()
69 | for JD,RA,DEC in zip(data['jd'],data['RA'],data['Decl']):
70 | sky._ra = RA*pi/180 # in radians
71 | sky._dec = DEC*pi/180
72 | sky._epoch = ephem.J2000
73 | OBS.date = JD-JDoff
74 | sky.compute(OBS)
75 | moon.compute(OBS)
76 | sun.compute(OBS)
77 | alpha.append((pi - ephem.separation(moon,sun))*180./pi) # in degrees
78 | rho.append(ephem.separation(moon,sky)) # in radians
79 | Z.append(pi/2 - sky.alt) # in radians
80 | Zm.append(pi/2 - moon.alt) # in radians
81 |
82 | data['alpha'] = pd.Series(array(alpha), index=data.index)
83 | data['rho'] = pd.Series(array(rho), index=data.index)
84 | data['Z'] = pd.Series(array(Z), index=data.index) # radians
85 | data['Zm'] = pd.Series(array(Zm), index=data.index)
86 |
87 | # Do the least-squares fit.
88 | pars,stat = leastsq(fitfunc, [22,0.2], args=(data['alpha'],data['rho'],data['Z'],
89 | data['Zm'],data['magsky']))
90 | print pars
91 | data['modelsky']=pd.Series(modelsky(data['alpha'],data['rho'],pars[1],
92 | data['Z'],data['Zm'],pars[0]), index=data.index)
93 |
94 | # Visualize!
95 | from bokeh.plotting import figure
96 | from bokeh.layouts import gridplot
97 | from bokeh.io import show,output_file
98 | from bokeh.models import ColumnDataSource
99 |
100 | output_file('fitsky_%s.html' % filt)
101 | source = ColumnDataSource(data)
102 | TOOLS = ['box_select','lasso_select','reset','box_zoom','help']
103 | vars = [('modelsky','magsky'),('alpha','rho'),('alpha','Zm'),
104 | ('jd','alpha'),('Z','Zm'),('RA','Decl')]
105 | plots = []
106 | for var in vars:
107 | s = figure(tools=TOOLS, plot_width=300, plot_height=300)
108 | s.circle(*var, source=source, selection_color='red', alpha=0.1)
109 | s.xaxis.axis_label = var[0]
110 | s.yaxis.axis_label = var[1]
111 | plots.append(s)
112 | plots[0].line([17.8,22.3],[17.8,22.3], line_color='orangered')
113 |
114 | p = gridplot([plots[0:3],plots[3:]])
115 | show(p)
116 |
--------------------------------------------------------------------------------