├── .gitignore ├── README.md ├── api ├── api.py └── requirements.txt ├── bin ├── .check_perf.swp ├── check_drq_scheme ├── check_perf ├── parse_data ├── parse_data.batch ├── qn_export ├── qn_parse_truth ├── qn_train └── refine_redshift ├── data └── .gitignore ├── etc ├── .gitignore ├── architecture.png ├── qn_QA.ipynb └── qn_desi.ipynb ├── py └── quasarnet │ ├── __init__.py │ ├── __init__.pyc │ ├── io.py │ ├── io.pyc │ ├── models.py │ └── utils.py ├── requirements.txt ├── setup.py ├── web ├── css │ └── qnet_styles.css ├── index.html └── pics │ └── architecture.png └── weights └── .gitignore /.gitignore: -------------------------------------------------------------------------------- 1 | **.pyc 2 | data/* 3 | weights/* 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # QuasarNET 2 | ### A convolutional neural network for redshifting and classification of astrophysical spectra 3 | 4 | ![architecture](etc/architecture.png) 5 | 6 | #### Installation instructions (requires python3): 7 | 8 | * on a standard system: 9 | ```bash 10 | git clone https://github.com/ngbusca/QuasarNET.git 11 | cd QuasarNET 12 | pip install -r requirements.txt --user 13 | python setup.py install --user 14 | ``` 15 | * at NERSC (specially if you wish to run this notebook at jupyter.nersc.gov) 16 | ```bash 17 | conda create -n qnet python=3 qnet scipy numpy fitsio h5py ipykernel 18 | source activate qnet 19 | python -m ipykernel install --user --name qnet --display-name qnet 20 | pip install tensorflow 21 | pip install keras>=2.2.4 22 | git clone https://github.com/ngbusca/QuasarNET.git 23 | cd QuasarNET 24 | python setup.py install 25 | ``` 26 | #### Download the data 27 | These data are a reprocessing of data release 12 (DR12) of the Sloan Digital Sky Survey (https://www.sdss.org/dr12/) 28 | 29 | They are available on Kaggle: https://www.kaggle.com/ngbusca/qnet_data 30 | 31 | A practical way to download the data is to use the kaggle-api, which will allow you to do it from the command line. Otherwise you can simply click the download link on the website. 32 | 33 | Download the data to the QuasarNET/data/ directory, unzip the file and set read/write permissions (skip the `kaggle datasets...` line if you've downloaded the data through the website). 34 | 35 | ```bash 36 | cd data 37 | kaggle datasets download ngbusca/qnet_data 38 | unzip qnet_data.zip 39 | chmod 600 * 40 | ``` 41 | #### Download the pre-trained weights 42 | The pre-trained weights are available at: https://www.kaggle.com/ngbusca/qnet_trained_models 43 | 44 | Download the weights to the QuasarNET/weights/ directory, unzip the file and set read/write permissions (skip the `kaggle datasets...` line if you've downloaded the data through the website). 45 | 46 | ```bash 47 | cd weights 48 | kaggle datasets download ngbusca/qnet_trained_models 49 | unzip qnet_trained_models.zip 50 | chmod 600 * 51 | ``` 52 | -------------------------------------------------------------------------------- /api/api.py: -------------------------------------------------------------------------------- 1 | import flask 2 | from flask import request, render_template 3 | import sys 4 | import numpy as np 5 | 6 | from quasarnet import io 7 | from quasarnet.utils import process_preds 8 | from tensorflow.keras.models import load_model 9 | from quasarnet.models import custom_loss 10 | 11 | app = flask.Flask(__name__) 12 | app.config["DEBUG"] = True 13 | model = load_model(sys.argv[1], custom_objects = {'custom_loss':custom_loss}) 14 | lines = ['LYA','CIV(1548)','CIII(1909)','MgII(2796)','Hbeta','Halpha'] 15 | lines_bal = ['CIV(1548)'] 16 | 17 | @app.route('/predict', methods=['POST']) 18 | def predict(): 19 | cl = request.content_length 20 | if cl is not None and cl > 20 * 1024 * 1024: 21 | return "input file too big (limit 20MB)" 22 | 23 | req_data = request.get_json() 24 | fl = np.array(req_data['flux']).astype(float) 25 | we = np.array(req_data['ivar']).astype(float) 26 | 27 | if 'wave' in req_data: 28 | ll = np.log10(np.array(req_data['wave']).astype(float)) 29 | ## binning matrix 30 | B = (np.log10(io.wave[:,None])-ll[None,:])/io.dll 31 | w = (B>0) & (B<1) 32 | B[w] = 1. 33 | B[~w] = 0. 34 | 35 | fl_qn = [] 36 | we_qn = [] 37 | 38 | for i in range(len(fl)): 39 | fl_aux = B.dot(fl[i]*we[i]) 40 | we_aux = B.dot(we[i]) 41 | w = we_aux > 0 42 | fl_aux[w]/=we_aux[w] 43 | fl_qn.append(fl_aux) 44 | we_qn.append(we_aux) 45 | 46 | fl = np.array(fl_qn) 47 | we = np.array(we_qn) 48 | 49 | if we is None: 50 | we = fl*0 + 1. 51 | X = fl-np.average(fl, weights=we, axis=1)[:,None] 52 | 53 | norm = X.std(axis=1) 54 | w = norm>0 55 | X[w]/= norm[w,None] 56 | 57 | p = model.predict(X[:,:,None]) 58 | p_line,z_line,zbest,p_bal,z_bal = process_preds(p, lines, lines_bal) 59 | ret = {"zbest": [z for z in zbest], "lines":lines, "lines_BAL":lines_bal} 60 | print(len(p_line)) 61 | for i,l in enumerate(lines): 62 | ret["p_{}".format(l)] = [str(p) for p in p_line[i]] 63 | ret["z_{}".format(l)] = [str(z) for z in z_line[i]] 64 | for i, l in enumerate(lines_bal): 65 | ret["pBAL_{}".format(l)] = [str(p) for p in p_bal[i]] 66 | ret["zBAL_{}".format(l)] = [str(z) for z in z_bal[i]] 67 | 68 | return ret 69 | 70 | if __name__ == '__main__': 71 | app.run(host='0.0.0.0') 72 | -------------------------------------------------------------------------------- /api/requirements.txt: -------------------------------------------------------------------------------- 1 | flask 2 | -------------------------------------------------------------------------------- /bin/.check_perf.swp: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /bin/check_drq_scheme: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import fitsio 4 | import argparse 5 | import numpy as np 6 | 7 | 8 | parser = argparse.ArgumentParser() 9 | parser.add_argument('--drq', type=str, required=True) 10 | parser.add_argument('--prefix', type=str, required=True) 11 | parser.add_argument('--spall', type=str, required=True) 12 | 13 | args=parser.parse_args() 14 | 15 | drq=fitsio.FITS(args.drq) 16 | zconf = drq[1]['Z_CONF_PERSON'][:] 17 | zclass = drq[1]['CLASS_PERSON'][:] 18 | w = ((zconf!=0) | (zclass!=0) ) 19 | plate=drq[1]['PLATE'][:][w] 20 | mjd=drq[1]['MJD'][:][w] 21 | fid = drq[1]['FIBERID'][:][w] 22 | zdrq=drq[1]['Z_VI'][:][w] 23 | zconf=zconf[w] 24 | zclass=zclass[w] 25 | 26 | Ydrq=np.zeros((len(plate),5)) 27 | for i in range(Ydrq.shape[0]): 28 | if zconf[i]==3: 29 | if zclass[i]==3 or zclass[i]==30: 30 | if zdrq[i]>2.1: 31 | Ydrq[i,3]=1 32 | else: 33 | Ydrq[i,2]=1 34 | 35 | elif zclass[i]==4: 36 | Ydrq[i,1]=1 37 | 38 | elif zclass[i]==1: 39 | Ydrq[i,0]=1 40 | 41 | else: 42 | Ydrq[i,4]=1 43 | 44 | spall = fitsio.FITS(args.spall) 45 | zw_dict = {(p,m,f) : zw for p,m,f,zw in zip(spall[1]['PLATE'][:], spall[1]['MJD'][:], spall[1]['FIBERID'][:], spall[1]['ZWARNING'][:])} 46 | 47 | spzall_dict={} 48 | 49 | class toto(object): 50 | pass 51 | 52 | pm=np.zeros(len(plate), dtype=toto) 53 | pm[:]=list(zip(plate, mjd)) 54 | pm=np.unique(pm) 55 | for p,m in pm: 56 | print(p,m) 57 | h=fitsio.FITS(args.prefix+'/{0}/v5_10_0/spZall-{0}-{1}.fits'.format(p,m)) 58 | wpm=(drq[1]['PLATE'][:]==p) & (drq[1]['MJD'][:]==m) 59 | fids = drq[1]['FIBERID'][:][wpm] 60 | ntemplates = len(h[1]['Z'][:])//1000 61 | zs = h[1]['Z'][:].reshape(-1,ntemplates) 62 | chi2 = (h[1]['RCHI2'][:]*h[1]['DOF'][:]).reshape(-1,ntemplates) 63 | clas = h[1]['CLASS'][:].reshape(-1,ntemplates) 64 | 65 | wfib = np.in1d(1+np.arange(1000), fids) 66 | zs = zs[wfib,:] 67 | chi2 = chi2[wfib,:] 68 | clas = clas[wfib,:] 69 | 70 | for i,f in enumerate(np.sort(fids)): 71 | spzall_dict[(p,m,f,'z')] = zs[i] 72 | spzall_dict[(p,m,f,'chi2')]=chi2[i] 73 | spzall_dict[(p,m,f,'class')]=clas[i] 74 | 75 | Yauto = np.zeros((len(plate), 5)) 76 | zauto = np.zeros(len(plate)) 77 | 78 | for i,(p,m,f) in enumerate(zip(plate, mjd, fid)): 79 | z = spzall_dict[(p,m,f,'z')][0] 80 | clas = spzall_dict[(p,m,f,'class')][:5] 81 | chi2 = spzall_dict[(p,m,f,'chi2')] 82 | if clas[0]==b'STAR ': 83 | Yauto[i,0]=1 84 | zauto[i]=0 85 | elif (clas[0]==b'GALAXY') and (z<1.): 86 | Yauto[i,1]=1 87 | zauto[i]=z 88 | elif (clas[0] == b'GALAXY') and ((clas==b'GALAXY').sum()>2): 89 | Yauto[i,1]=1 90 | zauto[i]=z 91 | elif (zw_dict[(p,m,f)]==0) and (clas[0]==b'QSO ') and ((clas==b'STAR ').sum()<2): 92 | zauto[i]=z 93 | if z>2.1: 94 | Yauto[i,3]=1 95 | else: 96 | Yauto[i,2]=1 97 | elif (zw_dict[(p,m,f)]>0) and ((clas==b'STAR ').sum()>1): 98 | Yauto[i,0]=1 99 | zauto[i]=0 100 | if Yauto[i].sum()==0: 101 | Yauto[i]=Ydrq[i] 102 | zauto[i]=zdrq[i] 103 | -------------------------------------------------------------------------------- /bin/check_perf: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import fitsio 4 | import argparse 5 | import numpy as np 6 | 7 | from tensorflow.keras.models import load_model, Model 8 | from quasarnet.io import read_data, read_sdrq, wave, read_desi_truth 9 | from scipy.interpolate import interp1d 10 | from quasarnet.models import custom_loss 11 | from picca import constants 12 | 13 | parser=argparse.ArgumentParser() 14 | 15 | parser.add_argument("--model",type=None,required=True) 16 | parser.add_argument("--super-drq",type=str,required=False, default = None) 17 | parser.add_argument("--truth",type=str,required=False, default = None) 18 | parser.add_argument("--spall",type=str,required=False, default = None) 19 | parser.add_argument("--data",type=str,required=True, default = None) 20 | args=parser.parse_args() 21 | 22 | if args.super_drq is not None: 23 | truth, tids2pmf = read_sdrq(args.super_drq) 24 | elif args.truth is not None: 25 | truth = read_desi_truth(args.truth) 26 | elif args.spall is not None: 27 | h=fitsio.FITS(args.spall) 28 | tids_spall = h[1]['THING_ID'][:] 29 | z_spall = h[1]['Z'][:] 30 | zw_spall = h[1]['ZWARNING'][:] 31 | class_spall = h[1]['CLASS'][:] 32 | specprim = h[1]['SPECPRIMARY'][:] 33 | 34 | class_truth = np.zeros(len(z_spall)) 35 | w=(class_spall=='STAR ') & (specprim==1) 36 | class_truth[w]=1 37 | 38 | w=(class_spall=='QSO ') & (specprim==1) 39 | class_truth[w]=3 40 | 41 | w=(class_spall=='GALAXY') & (specprim==1) 42 | class_truth[w]=4 43 | 44 | w=(zw_spall!=0) & (specprim==1) 45 | class_truth[w]=1 46 | 47 | zc=3*((zw_spall==0) & (specprim==1)) 48 | w=zc==0 49 | zc[w]=1 50 | truth = {t:(a,b,c) for t,a,b,c in zip(tids_spall, class_truth, zc, z_spall)} 51 | 52 | tids,X,Y,z,bal = read_data(args.data, truth) 53 | 54 | if not isinstance(args.model, Model): 55 | model=load_model(args.model, custom_objects = {'custom_loss': custom_loss}) 56 | else: 57 | model=args.model 58 | 59 | aux = model.predict(X[:,:,None]) 60 | p, bal_pred, box_pred_lya, box_pred_civ, box_pred_mgii, box_pred_hbeta, z_pred = aux 61 | z_pred=z_pred[:,0] 62 | bal_pred = bal_pred[:,0] 63 | nboxes = box_pred_lya.shape[1]/2 64 | box_lya = box_pred_lya[:,:nboxes].argmax(axis=1) 65 | box_civ = box_pred_civ[:,:nboxes].argmax(axis=1) 66 | box_mgii = box_pred_mgii[:,:nboxes].argmax(axis=1) 67 | box_hbeta = box_pred_hbeta[:,:nboxes].argmax(axis=1) 68 | 69 | nspec = box_lya.shape[0] 70 | ispec = np.arange(nspec) 71 | offset_lya = box_pred_lya[ispec, nboxes+box_lya] 72 | offset_civ = box_pred_civ[ispec, nboxes+box_civ] 73 | offset_mgii = box_pred_mgii[ispec, nboxes+box_mgii] 74 | offset_hbeta = box_pred_hbeta[ispec, nboxes+box_hbeta] 75 | 76 | i_to_wave = interp1d(np.arange(len(wave)), wave, bounds_error=False, fill_value='extrapolate') 77 | z_pred_lya = i_to_wave(X.shape[1]*(box_lya+offset_lya)/nboxes) 78 | z_pred_lya = z_pred_lya/constants.absorber_IGM["LYA"]-1 79 | z_pred_civ = i_to_wave(X.shape[1]*(box_civ+offset_civ)/nboxes) 80 | z_pred_civ = z_pred_civ/constants.absorber_IGM["CIV(1548)"]-1 81 | z_pred_mgii = i_to_wave(X.shape[1]*(box_mgii+offset_mgii)/nboxes) 82 | z_pred_mgii = z_pred_mgii/constants.absorber_IGM["MgII(2796)"]-1 83 | z_pred_hbeta = i_to_wave(X.shape[1]*(box_hbeta+offset_hbeta)/nboxes) 84 | z_pred_hbeta = z_pred_hbeta/constants.absorber_IGM["Hbeta"]-1 85 | perf = np.zeros((5,5)) 86 | for i in range(5): 87 | for j in range(5): 88 | w=(p.argmax(axis=1)==i) & (Y.argmax(axis=1)==j) 89 | perf[i,j]=w.sum() 90 | 91 | norm = perf.sum(axis=0) 92 | perf/=norm 93 | -------------------------------------------------------------------------------- /bin/parse_data: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import numpy as np 4 | from quasarnet import io 5 | import fitsio 6 | import sys 7 | 8 | import argparse 9 | 10 | parser=argparse.ArgumentParser() 11 | parser.add_argument('--spplates',type=str, nargs="+", required=True) 12 | parser.add_argument('--spall',type=str, required=True) 13 | parser.add_argument('--sdrq',type=str, required=False, default=None) 14 | parser.add_argument('--out',type=str, required=True) 15 | 16 | args=parser.parse_args() 17 | 18 | 19 | ## this is a dictionary (plate, mjd, fiberid) => thing_id 20 | print('INFO: reading spall') 21 | spall=fitsio.FITS(args.spall) 22 | print('INFO: done') 23 | 24 | tids_spall = spall[1]['THING_ID'][:] 25 | plate_quality = spall[1]['PLATEQUALITY'][:] 26 | pmf_spall = list(zip(spall[1]['PLATE'][:],spall[1]['MJD'][:],spall[1]['FIBERID'][:])) 27 | thid_db = {pmf:t for pmf,t in zip(pmf_spall,tids_spall)} 28 | 29 | if args.sdrq is not None: 30 | print('INFO: reading sdrq') 31 | sdrq = fitsio.FITS(args.sdrq) 32 | tids_sdrq = sdrq[1]['THING_ID'][:] 33 | print('INFO: done') 34 | w = np.in1d(tids_spall, tids_sdrq) & (tids_spall>0) 35 | else: 36 | print('getting quasars from target bits') 37 | tb = {'BOSS_TARGET1':[10,11,12,13,14,15,16,17,18,19,40,41,42,43,44], 38 | 'EBOSS_TARGET0':[10,11,12,13,14,15,16,17,18,20,22,30,31,33,34,35,40], 39 | 'EBOSS_TARGET1':[9,10,11,12,13,14,15,16,17,18,30,31], 40 | 'EBOSS_TARGET2':[0,2,4,20,21,23,24,25,26,27,31,32,33,34,50,51, 41 | 52,53,54,55,56,57,58,59,60,61,62], 42 | 'ANCILLARY_TARGET1':[6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22, 43 | 23,24,25,26,27,28,29,30,31,50,51,52,53,54,55,58,59], 44 | 'ANCILLARY_TARGET2':[0,1,2,3,4,5,7,8,9,10,13,14,15,24, 45 | 25,26,27,31,32,33,53,54,55,56] 46 | } 47 | 48 | wqso = np.zeros(len(tids_spall), dtype=bool) 49 | for kw,val in tb.items(): 50 | mask = sum([2**b for b in val]) 51 | wqso_kw = (mask & spall[1][kw][:])>0 52 | print('found {} quasar targets with target bits {}'.format(wqso_kw.sum(),kw)) 53 | wqso |= wqso_kw 54 | 55 | print('found {} quasar targets'.format(wqso.sum())) 56 | 57 | wqso &= plate_quality == b'good' 58 | print('after rejecting BAD plates: {}'.format(wqso.sum())) 59 | 60 | plate_mjd = {} 61 | print('populating dict') 62 | i=0 63 | for p,m,f in np.array(pmf_spall)[wqso]: 64 | sys.stderr.write('\rINFO: populating dict: {}'.format(i*100/len(pmf_spall))) 65 | k = (p,m) 66 | if not k in plate_mjd: 67 | plate_mjd[k] = [] 68 | plate_mjd[k].append(f) 69 | 70 | print('\ndone') 71 | fl = [] 72 | tids = [] 73 | plate = [] 74 | mjd = [] 75 | fibs = [] 76 | 77 | nread=0 78 | for p,m in plate_mjd: 79 | fname = 'spPlate-{}-{}.fits'.format(p,m) 80 | f = [toto for toto in args.spplates if fname in toto] 81 | if len(f)==0: 82 | print('WARNING: file {} not found'.format(fname)) 83 | print('WARNING: args is {} '.format(args.spplates[0])) 84 | sys.exit(1) 85 | if len(f)>1: 86 | print('WARNING: multiple files found', f) 87 | sys.exit(1) 88 | 89 | f=f[0] 90 | print('Reading file {}, nread={}, ntot={}'.format(f,nread,len(plate_mjd))) 91 | nread+=1 92 | fibers = plate_mjd[(p,m)] 93 | aux = io.read_spplate(f, fibers) 94 | if aux is not None: 95 | tids_plate = [thid_db[(p,m,fib)] for fib in aux[0]] 96 | tids.append(tids_plate) 97 | fl.append(aux[1]) 98 | plate.append([p]*len(aux[0])) 99 | mjd.append([m]*len(aux[0])) 100 | fibs.append(aux[0]) 101 | 102 | tids = np.concatenate(tids) 103 | fl = np.concatenate(fl) 104 | plate=np.concatenate(plate) 105 | mjd = np.concatenate(mjd) 106 | fibs = np.concatenate(fibs) 107 | 108 | h=fitsio.FITS(args.out,'rw',clobber=True) 109 | h.write(fl) 110 | h.write([tids, plate, mjd, fibs], names=['TARGETID','PLATE','MJD','FIBERID']) 111 | h.close() 112 | -------------------------------------------------------------------------------- /bin/parse_data.batch: -------------------------------------------------------------------------------- 1 | #!/bin/bash -l 2 | #SBATCH -N 1 #Use 2 nodes 3 | #SBATCH -t 01:30:00 #Set 30 minute time limit 4 | #SBATCH -p regular #Submit to the regular 'partition' 5 | #SBATCH -L SCRATCH #Job requires $SCRATCH file system 6 | #SBATCH -C haswell #Use Haswell nodes 7 | 8 | srun -n 1 -c 64 parse_data --spplates ~/dr14/eboss/spectro/redux/v5_10_0/*/spPlate* --spall /global/projecta/projectdirs/sdss/data/sdss/dr12/boss/spectro/redux/spAll-DR12.fits --out data/data_dr12_v5_10_0.fits 9 | -------------------------------------------------------------------------------- /bin/qn_export: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import glob 4 | from tensorflow.keras.models import load_model 5 | from quasarnet.models import custom_loss 6 | from quasarnet.io import read_data, read_desi_truth 7 | from scipy.interpolate import interp1d 8 | from numpy import in1d, array 9 | from quasarnet.io import read_data 10 | from quasarnet.utils import absorber_IGM, process_preds 11 | import fitsio 12 | import argparse 13 | import numpy as np 14 | 15 | parser=argparse.ArgumentParser() 16 | parser.add_argument('--model',type=str,required=True, 17 | help='the model, output of qn_train') 18 | 19 | parser.add_argument('--data',type=str,required=True,nargs="*", 20 | help='the data to be inspected, output of parse_data') 21 | 22 | parser.add_argument('--data-training',type=str,required=True, 23 | help='the data used for training') 24 | 25 | parser.add_argument('--spall',type=str,required=True, 26 | help='the spall file (same pipeline version as the --data') 27 | 28 | parser.add_argument('--lines',type=str,nargs="*", 29 | required=False,default=['LYA', 'CIV(1548)', 30 | 'CIII(1909)', 'MgII(2796)' ,'Hbeta', 'Halpha'], 31 | help='the list of emission lines used to train the model' 32 | ) 33 | 34 | parser.add_argument('--lines-bal',type=str,nargs="*", 35 | required=False,default=['CIV(1548)'], 36 | help='the list of BAL lines' 37 | ) 38 | 39 | parser.add_argument('--c-th',type=float,required=False,default=0.4, 40 | help='threshold confidence value for a line-detection') 41 | 42 | parser.add_argument('--n-th',type=int,required=False,default=1, 43 | help='the threshold number of confident lines to define a quasar') 44 | 45 | parser.add_argument('--out-suffix',type=str,required=True, 46 | help='the output prefix') 47 | 48 | args=parser.parse_args() 49 | 50 | tids,X,plate,mjd,fid = read_data(args.data, return_pmf=True) 51 | 52 | m=args.model 53 | print('loading model {}'.format(m)) 54 | model=load_model(m, custom_objects={'custom_loss':custom_loss}) 55 | 56 | print('predicting...') 57 | aux=model.predict(X[:,:,None]) 58 | print('prediction done') 59 | 60 | c_line,z_line,zbest,c_line_bal,z_line_bal =\ 61 | process_preds(aux, args.lines, args.lines_bal) 62 | 63 | isqso = (c_line>args.c_th).sum(axis=0)>=args.n_th 64 | 65 | spall = fitsio.FITS(args.spall) 66 | tids_spall = spall[1]['THING_ID'][:] 67 | plate_spall = spall[1]['PLATE'][:] 68 | mjd_spall = spall[1]['MJD'][:] 69 | fid_spall = spall[1]['FIBERID'][:] 70 | ra_spall = spall[1]['RA'][:] 71 | dec_spall = spall[1]['DEC'][:] 72 | sprim_spall = spall[1]['SPECPRIMARY'][:] 73 | spall_dict = {(p,m,f):t for t,p,m,f in zip(tids_spall, 74 | plate_spall, mjd_spall, fid_spall)} 75 | 76 | dic_spall = {t:(r,d,s) for t,r,d,s in zip(tids_spall,ra_spall, 77 | dec_spall,sprim_spall)} 78 | 79 | spall.close() 80 | 81 | h_train=fitsio.FITS(args.data_training) 82 | plate_train = h_train[1]['PLATE'][:] 83 | mjd_train = h_train[1]['MJD'][:] 84 | fid_train = h_train[1]['FIBERID'][:] 85 | 86 | tids_train = [spall_dict[(p,m,f)] for p,m,f in zip(plate_train, 87 | mjd_train, fid_train) if (p,m,f) in spall_dict] 88 | 89 | in_train = in1d(tids, tids_train) 90 | 91 | print('INFO: found {} spectra in the training sample'.format(in_train.sum())) 92 | 93 | hout=fitsio.FITS('qnAll-'+args.out_suffix+'.fits','rw',clobber=True) 94 | 95 | cols = [tids] 96 | names = ['THING_ID'] 97 | cols += [plate,mjd,fid] 98 | names += ['PLATE','MJD','FIBERID'] 99 | 100 | cols += [zbest, isqso.astype(int), in_train.astype(int)] 101 | names += ['ZBEST','IS_QSO','IN_TRAIN'] 102 | cols.append(c_line.T) 103 | names.append('C_LINES') 104 | cols.append(z_line.T) 105 | names.append('Z_LINES') 106 | header=[{'name':'LINE_{}'.format(il),'value':absorber_IGM[l],'comment':l} for il,l in enumerate(args.lines)] 107 | header=[{'name':'LINE_BAL_{}'.format(il),'value':absorber_IGM[l],'comment':l} for il,l in enumerate(args.lines_bal)] 108 | 109 | cols.append(c_line_bal.T) 110 | names.append('C_LINES_BAL') 111 | cols.append(z_line_bal.T) 112 | names.append('Z_LINES_BAL') 113 | hout.write(cols,names=names,header=header) 114 | 115 | hout.close() 116 | 117 | hout = fitsio.FITS('DRQ_QN-'+args.out_suffix+'.fits', 118 | 'rw',clobber=True) 119 | ra = [dic_spall[t][0] for t in tids] 120 | ra = array(ra) 121 | 122 | dec = [dic_spall[t][1] for t in tids] 123 | dec = array(dec) 124 | 125 | sprim = [dic_spall[t][2] for t in tids] 126 | sprim = array(sprim) 127 | 128 | isqso = isqso & (sprim==1) 129 | 130 | print('INFO: found {} unique quasars'.format(isqso.sum())) 131 | 132 | cols = [tids[isqso]] 133 | names = ['THING_ID'] 134 | cols += [plate[isqso], mjd[isqso], fid[isqso]] 135 | names += ['PLATE','MJD','FIBERID'] 136 | 137 | isbal = (c_line_bal>args.c_th).sum(axis=0)>=1 138 | cols += [zbest[isqso], ra[isqso], dec[isqso], isbal[isqso]] 139 | names += ['Z','RA','DEC','BI_CIV'] 140 | 141 | hout.write(cols, names=names) 142 | 143 | -------------------------------------------------------------------------------- /bin/qn_parse_truth: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import numpy as np 4 | import fitsio 5 | import argparse 6 | 7 | parser = argparse.ArgumentParser() 8 | 9 | parser.add_argument('--sdrq', type=str, help='Superset_DRQ file', 10 | required=True) 11 | 12 | parser.add_argument('--drq', type=str, help='DRQ file', 13 | required=True) 14 | 15 | parser.add_argument('--out', type=str, help='output file', 16 | required=True) 17 | 18 | args = parser.parse_args() 19 | 20 | h = fitsio.FITS(args.sdrq) 21 | 22 | colnames = ['THING_ID','Z_VI','PLATE','MJD','FIBERID',\ 23 | 'CLASS_PERSON','Z_CONF_PERSON'] 24 | 25 | cols = [] 26 | for c in colnames: 27 | cols.append(h[1][c][:]) 28 | 29 | h.close() 30 | 31 | h = fitsio.FITS(args.drq) 32 | tids = h[1]['THING_ID'][:] 33 | bal_flag_vi_drq = h[1]['BAL_FLAG_VI'][:] 34 | bi_civ_drq = h[1]['BI_CIV'][:] 35 | h.close() 36 | 37 | drq = {t:(bfv,bi) for t,bfv,bi in zip(tids,bal_flag_vi_drq,bi_civ_drq)} 38 | 39 | w = np.in1d(cols[0], tids) 40 | 41 | bal_flag_vi = np.zeros(len(cols[0])) 42 | bal_flag_vi[w] = np.array([drq[t][0] for t in cols[0][w]]) 43 | 44 | bi_civ = np.zeros(len(cols[0])) 45 | bi_civ[w] = np.array([drq[t][1] for t in cols[0][w]]) 46 | 47 | cols += [bal_flag_vi, bi_civ] 48 | colnames += ['BAL_FLAG_VI', 'BI_CIV'] 49 | 50 | h = fitsio.FITS(args.out, 'rw', clobber=True) 51 | h.write(cols, names=colnames) 52 | h.close() 53 | -------------------------------------------------------------------------------- /bin/qn_train: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import argparse 3 | import numpy as np 4 | import fitsio 5 | 6 | from quasarnet.models import QuasarNET, custom_loss 7 | from tensorflow.keras.optimizers import Adam 8 | from quasarnet import io 9 | 10 | parser = argparse.ArgumentParser() 11 | 12 | parser.add_argument("-t","--truth", type = str, required=False,nargs="*") 13 | parser.add_argument("-d","--data", type = str, required=True,nargs="*") 14 | parser.add_argument("-e","--epochs", type = int, 15 | required=False, default = 5) 16 | parser.add_argument("-o","--out-prefix", type = str, required=True) 17 | parser.add_argument("-l","--lines", type = str, 18 | required=False, default = ["LYA"], nargs="*") 19 | parser.add_argument("-b","--lines-bal", type = str, 20 | required=False, default = ["CIV(1548)"], nargs="*") 21 | parser.add_argument("--decay", type = float, required=False, 22 | default = 0.) 23 | 24 | args = parser.parse_args() 25 | 26 | truth = io.read_truth(args.truth) 27 | tids,X,Y,z,bal = io.read_data(args.data, truth) 28 | 29 | model = QuasarNET( 30 | X[0,:,None].shape, 31 | nlines=len(args.lines)+len(args.lines_bal) 32 | ) 33 | 34 | loss = [] 35 | for i in args.lines: 36 | loss.append(custom_loss) 37 | 38 | for i in args.lines_bal: 39 | loss.append(custom_loss) 40 | 41 | adam = Adam(decay=args.decay) 42 | model.compile(optimizer=adam, loss=loss, metrics=[]) 43 | 44 | box, sample_weight = io.objective(z,Y,bal,lines=args.lines, 45 | lines_bal=args.lines_bal) 46 | 47 | print( "starting fit") 48 | history = model.fit(X[:,:,None], box, 49 | epochs = args.epochs, 50 | batch_size = 256, 51 | sample_weight = sample_weight) 52 | 53 | model.save(args.out_prefix+".h5") 54 | model.summary() 55 | 56 | cols = [] 57 | fout = fitsio.FITS(args.out_prefix+"_hist.fits",'rw',clobber=True) 58 | for v in history.history.values(): 59 | cols.append(np.array(v)) 60 | 61 | fout.write(cols, names=list(history.history.keys())) 62 | fout.close() 63 | -------------------------------------------------------------------------------- /bin/refine_redshift: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import h5py 4 | import argparse 5 | import fitsio 6 | import numpy as np 7 | 8 | parser=argparse.ArgumentParser() 9 | parser.add_argument("--sdrq", type=str, required=True) 10 | parser.add_argument("--prefix", type=str, required=True) 11 | parser.add_argument("--pm", type=int, required=False, nargs="+", default=None) 12 | 13 | 14 | args = parser.parse_args() 15 | 16 | drq=fitsio.FITS(args.sdrq) 17 | plate=drq[1]['PLATE'][:] 18 | mjd=drq[1]['MJD'][:] 19 | fid=drq[1]['FIBERID'][:] 20 | 21 | class toto(object): 22 | pass 23 | 24 | pm=np.zeros(len(plate), dtype=toto) 25 | pm[:]=zip(plate, mjd) 26 | if args.pm is not None: 27 | pm[:] = zip(args.pm[:2:], args.pm[1:2:]) 28 | pm=np.unique(pm) 29 | 30 | rr_dict={} 31 | for p,m in pm: 32 | print p,m 33 | rr=h5py.File(args.prefix+"/{0}/rrdetail-{0}-{1}.h5".format(p,m)) 34 | w=(plate==p) & (mjd==m) 35 | f = fid[w] 36 | z = rr['zscan/QSO/redshifts'][...] 37 | chi2=rr['zscan/QSO/zchi2'][...] 38 | rr_f = rr['targetids'][...]%10000 39 | wfib=np.in1d(rr_f,f) 40 | rr_f = rr_f[wfib] 41 | chi2=chi2[wfib] 42 | for i,j in enumerate(rr_f): 43 | rr_dict[(p,m,j,'chi2')] = chi2[i] 44 | rr_dict[(p,m,j,'z')]=z 45 | -------------------------------------------------------------------------------- /data/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | !.gitignore 3 | -------------------------------------------------------------------------------- /etc/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | !*.ipynb 3 | !.gitignore 4 | !architecture.png 5 | -------------------------------------------------------------------------------- /etc/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ngbusca/QuasarNET/7d3e641027a202f1bd940168f74949d0d0be4f30/etc/architecture.png -------------------------------------------------------------------------------- /etc/qn_QA.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### QuasarNET: the Quality Assesment notebook\n", 8 | "\n", 9 | "This notebook will walk you through how to run [QuasarNET](https://arxiv.org/pdf/1808.09955.pdf]):\n", 10 | " * load training and validation data\n", 11 | " * train the network (on a small training sample, optional)\n", 12 | " * load pre-trained weights\n", 13 | " * plot example spectra\n", 14 | " * produce QA plots and confusion matrix\n", 15 | "\n", 16 | "#### Installation instructions (requires python3):\n", 17 | "##### - on a standard system\n", 18 | "\n", 19 | "```bash\n", 20 | "git clone https://github.com/ngbusca/QuasarNET.git\n", 21 | "cd QuasarNET\n", 22 | "pip install -r requirements.txt --user\n", 23 | "python setup.py install --user\n", 24 | "```\n", 25 | "\n", 26 | "##### - at NERSC (e.g. if you wish to run this notebook at jupyter.nersc.gov)\n", 27 | "\n", 28 | "```bash\n", 29 | "conda create -n qnet python=3 qnet scipy numpy fitsio h5py ipykernel\n", 30 | "source activate qnet\n", 31 | "python -m ipykernel install --user --name qnet --display-name qnet\n", 32 | "pip install tensorflow\n", 33 | "pip install keras>=2.2.4\n", 34 | "git clone https://github.com/ngbusca/QuasarNET.git\n", 35 | "cd QuasarNET\n", 36 | "python setup.py install\n", 37 | "```\n", 38 | "\n", 39 | "#### - Download the data \n", 40 | "These data are a reprocessing of data release 12 (DR12) of the Sloan Digital Sky Survey (https://www.sdss.org/dr12/)\n", 41 | "\n", 42 | "They are available on Kaggle: https://www.kaggle.com/ngbusca/qnet_data\n", 43 | "\n", 44 | "A practical way to download the data is to use the [kaggle-api](https://github.com/Kaggle/kaggle-api), which will allow you to do it from the command line. Otherwise you can simply click the `download` link on the website.\n", 45 | "\n", 46 | "Download the data to the `QuasarNET/data/` directory, unzip the file and set read/write permissions\n", 47 | "(skip the `kaggle datasets...` line if you've downloaded the data through the website).\n", 48 | "\n", 49 | "```bash\n", 50 | "cd data\n", 51 | "kaggle datasets download ngbusca/qnet_data\n", 52 | "unzip qnet_data.zip\n", 53 | "chmod 600 *\n", 54 | "```\n", 55 | "#### - Download the pre-trained weights\n", 56 | "The pre-trained weights are available at: https://www.kaggle.com/ngbusca/qnet_trained_models\n", 57 | "\n", 58 | "Download the weights to the `QuasarNET/weights/` directory, unzip the file and set read/write permissions\n", 59 | "(skip the `kaggle datasets...` line if you've downloaded the data through the website).\n", 60 | "\n", 61 | "```bash\n", 62 | "cd weights\n", 63 | "kaggle datasets download ngbusca/qnet_trained_models\n", 64 | "unzip qnet_trained_models.zip\n", 65 | "chmod 600 *\n", 66 | "```" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": 1, 72 | "metadata": {}, 73 | "outputs": [ 74 | { 75 | "name": "stderr", 76 | "output_type": "stream", 77 | "text": [ 78 | "Using TensorFlow backend.\n" 79 | ] 80 | } 81 | ], 82 | "source": [ 83 | "import numpy as np\n", 84 | "from matplotlib import pyplot as plt\n", 85 | "import fitsio\n", 86 | "from keras.optimizers import Adam\n", 87 | "from keras.models import load_model\n", 88 | "from quasarnet.models import QuasarNET, custom_loss\n", 89 | "from quasarnet.io import read_truth, read_data, wave, objective\n", 90 | "from quasarnet.utils import process_preds, absorber_IGM\n", 91 | "%matplotlib inline" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "#### Load the validation and training data\n", 99 | "\n", 100 | "The full data sample was divided into 8 80/20 training/validation splits and pre-trained weights are available for each split. The random split to load is controled by the `isplit` variable.\n", 101 | "\n", 102 | "The next cell loads first the truth table, the full data sample and the chosen training sample.\n", 103 | "It finally excludes the training data from the full sample to form the validation sample.\n", 104 | "\n", 105 | "The full data sample (~560,000 spectra) is abour 4GB and is fully loaded in memory. If your computer has less memory than that, set the nspec variable below to load a smaller number of spectra." 106 | ] 107 | }, 108 | { 109 | "cell_type": "code", 110 | "execution_count": 2, 111 | "metadata": {}, 112 | "outputs": [ 113 | { 114 | "name": "stdout", 115 | "output_type": "stream", 116 | "text": [ 117 | "INFO: reading data from ../data/data_dr12.fits\n", 118 | "INFO: removing 20766 spectra missing in truth\n", 119 | "INFO: found (617289,) spectra in file ../data/data_dr12.fits\n", 120 | "INFO: removing 2567 spectra with zero weights\n", 121 | "INFO: removing 7 spectra with zero flux\n" 122 | ] 123 | } 124 | ], 125 | "source": [ 126 | "## set nspec to the number of spectra to load or to None for the full sample\n", 127 | "nspec = None\n", 128 | "isplit=0\n", 129 | "truth_file=(['../data/truth_DR12Q.fits'])\n", 130 | "truth = read_truth(truth_file)\n", 131 | "tids_full,X_full,Y_full,z_full,bal_full = read_data(['../data/data_dr12.fits'], truth, nspec=nspec)\n", 132 | "\n", 133 | "data_file = '../data/data_train_{}.fits'.format(isplit)\n", 134 | "h = fitsio.FITS(data_file)\n", 135 | "tids_train = h[1]['TARGETID'][:]\n", 136 | "w = np.in1d(tids_full, tids_train)\n", 137 | "X_train = X_full[w]\n", 138 | "Y_train = Y_full[w]\n", 139 | "z_train = z_full[w]\n", 140 | "bal_train = bal_full[w]\n", 141 | "\n", 142 | "## to get the validation data, remove the spectra in the training sample from the full sample\n", 143 | "w = ~np.in1d(tids_full, tids_train)\n", 144 | "tids_val = tids_full[w]\n", 145 | "X_val = X_full[w]\n", 146 | "Y_val = Y_full[w]\n", 147 | "z_val = z_full[w]\n", 148 | "bal_val = bal_full[w]" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": {}, 154 | "source": [ 155 | "#### Define the \"features\" that the network is trained to recognize\n", 156 | "\n", 157 | "The features are defined by their rest wavelength. \n", 158 | "A dictionary `{feature_name:feature_wavelength}` is defined in `quasarnet.util.absorber_IGM`, which currently contains typical quasar broad emission lines. It could be easily extended to include other features by extending the dictionary." 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 3, 164 | "metadata": {}, 165 | "outputs": [], 166 | "source": [ 167 | "lines=['LYA','CIV(1548)','CIII(1909)', 'MgII(2796)','Hbeta','Halpha']\n", 168 | "lines_bal=['CIV(1548)']" 169 | ] 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "metadata": {}, 174 | "source": [ 175 | "#### Train the network (optional).\n", 176 | "\n", 177 | "The best way to train the network is to use a multicore computer. Training over 100 epochs on a full training sample takes ~24h on a standard 24 core unit, and ~7 hours on a Cori/NERSC node (64 cores).\n", 178 | "\n", 179 | "As an example, the following two cells will instantiate the model (first cell) and run the training (second cell) on a single epoch and fewer training spectra. You can run the second cell many times to train over many epochs." 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": {}, 186 | "outputs": [], 187 | "source": [ 188 | "ntrain = 10000\n", 189 | "nbins = X_train.shape[1]\n", 190 | "model = QuasarNET((nbins,1), nlines=len(lines)+len(lines_bal))\n", 191 | "optimizer = Adam()\n", 192 | "loss = [custom_loss]*(len(lines)+len(lines_bal))\n", 193 | "model.compile(optimizer=optimizer, loss=loss, metrics=[])\n", 194 | "objective, sample_weight = objective(z_train[:ntrain],Y_train[:ntrain],bal_train[:ntrain],lines=lines,lines_bal=lines_bal)" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": null, 200 | "metadata": {}, 201 | "outputs": [], 202 | "source": [ 203 | "loss_history = model.fit(X_train[:ntrain,:,None], objective, epochs=1, batch_size=32, sample_weight=sample_weight)" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "metadata": {}, 209 | "source": [ 210 | "#### Load a pre-trained model\n", 211 | "\n", 212 | "The following cell loads pre-trained weights for the network, corresponding to the split defined earlier. The pre-training was done over the full training data sample and 200 epochs." 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 4, 218 | "metadata": {}, 219 | "outputs": [], 220 | "source": [ 221 | "#model = load_model('../weights/qn_train_{}.h5'.format(isplit),custom_objects={'custom_loss':custom_loss})\n", 222 | "model = load_model('../runs/v19.0/qn_train_{}.h5'.format(isplit),custom_objects={'custom_loss':custom_loss})" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": {}, 228 | "source": [ 229 | "#### Example spectra\n", 230 | "\n", 231 | "Let's now take a look at the network output by examining a few examples. If you skipped loading the pre-trained weights you will be looking at the model you trained (it's actually not that bad!).\n", 232 | "\n", 233 | "The network outputs confidences and positions of the features defined earlier. The following plot shows a spectrum from the validation sample and the detected features. You can change the index `ival` to change the spectrum to be shown." 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": 19, 239 | "metadata": {}, 240 | "outputs": [ 241 | { 242 | "name": "stdout", 243 | "output_type": "stream", 244 | "text": [ 245 | "INFO: nspec = 1, nboxes=13\n" 246 | ] 247 | }, 248 | { 249 | "data": { 250 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEJCAYAAACAKgxxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXeYW9WZ/z9HfSRNn/G4YWwDrtgG\nbEwHmx4gIYTsLxDSCSQkgc1u2ARSN22TbLLJkkZCyoaEHkgCoTebjsHGNrji3u3pRaMund8f90qj\nOlUejeT38zx6Rrr36NxXV5rvfe973vMepbVGEARBKB8sxTZAEARBKCwi7IIgCGWGCLsgCEKZIcIu\nCIJQZoiwC4IglBki7IIgCGWGCLsgCEKZIcIuCIJQZoxpYVdK+VIecaVUIOX1NUqpnUqp8zPe8wml\n1Mspr5NtzOeHlFKelP2fVkotz+jjKqXUCqVUr1Kq2Xz+OaWUMvdvUUodN8jPkHn8gFKqRynVqZR6\nVSn1WaXUYfselFLHKaWCSqm7MrYvN7cnzufmQr1XKXWXUuqAUqpbKfWuUurTKft8GY+YUuoX5j6n\nUuoPSqld5jlarZR6T0bfX1BKrVRKhZRSfxrqZypl+jt3OdrWKaX+bv6GdymlPpyxv7/zmHdfuTHQ\necrRvt/vQCk1Wyn1vFKqSym1VSl1Rcq+Uft9jmlh11p7Ew9gN/DelG13D7NbG/Cv+XYqpb4E3Ab8\nGBgPNAGfBc4AHGazx4FLhnn892qtK4GjgR8CXwH+MMy+BsOvgDfz7PtCyvmcWcD3/gCYqrWuAt4H\nfE8ptRCyvtMmIAD81XyfDdgDnANUA98AHlBKTU3pez/wPeCPw/xMJcsA5y6TXwFhs901wO1Kqbkp\n+/s7jwOd43JioPOURn/fgVLKBjwMPArUAdcDdymlZqR0MSq/zzEt7IeJHwM3K6VqMncopaqB7wCf\n01o/qLXu0QartdbXaK1DZtORCDsAWusurfUjwIeAjyuljh9Jf7lQSl0FdALPjeZ7tdbrU86VNh/H\n5Gj6QaAZeMl8X6/W+j+11ju11nGt9aPADmBhSt9/01r/A2gbql39oZT6UIYnFsq8kxtjpJ27VMw7\n0iuBb2itfVrrl4FHgI8m2vR3HkdyjkvpPA7mPA1A5ncwC5gI/ExrHdNaPw+8MoT+CsaRKOwrgeXA\nzTn2nQY4Ma66/bEcWKRSQjrDRWv9BrAXOCvXfqXUo2bYJtfj0Xz9KqWqMC5SX+rn8D9QSrUqpV5R\nSi0pxHtT+vi1UsoPbAIOYFwMM/k48Gedp2CRUqoJmAGs78eOIdmVD631/Sme2ERgO3DvEI47aIb7\nnWbQ37mbAcS01u+mbFsL5PVEC8VonsdMhnFeR3qeMr8DlcssINVpG9bvc6jYDlfHo8g/lFLRlNcO\n4K0B3vNN4BWl1G0Z2xuAVq11sj+l1KvAHAzBv0hr/aLWOqSUegU4D+MKP1L2Y9y6ZaG1vmyYfX4X\n+IPWeo9SuX5vfAXYgHEbehXwT6XUCVrrbSN8b8LuzymlbsS4WC4BQqkdKKWmYIRcrs11AKWUHbgb\nuFNrvWmQn3lAuwZCGeMd9wDLtda/Hez7hsIIvlNg4HMHeIGujG1dQOVIjjsUDtd5VEp9FXhDa/1s\n5r5hnNdhn6c838EmDA/+P5RSPwOWmm2WmftH/PscLOXgsb9fa12TeACfG+gNWut1GHGwWzJ2tQEN\nZqws0fZ0s9820s/XiMMxKUwC2gvUF0qpE4DzgZ/la6O1XmGGmkJa6zsxbhkvGcl7c7SLmbe3k4Eb\nMnZ/DHhZa70jh/0W4C8Y/wBfGODjDtmuAfg+xj/2TUN832iS99yZ+ICqjG1VQM9htSqdIZ9HNbgk\ngrnAO8M1KoORnKes70BrHQHeD1wKHMS4430A4468UL/PQVEOwj5cvgVchyGqCV7D8CwvH8T7HwPe\nM2CrAVBKnWza8HKe/U/kGIlPPJ7I0+0SYCqwWyl1ECPsdKVSqr87GY1x2ziS9+bDRnaM/WPAnZkN\nlXGL8AeMgakrzX+W4TKQXZnHvgq4Gvhg6nGVUt81Mxo2KaXONLe9oZT6qVJqjVLqpnzb8hxnON9p\nKjnPXQrvAjaVnrm1gKGFtIZNP+dxnVLqEWVkO305ZftbSqnbgd8rpT6qjKySlUqppeb+G5SRQfYH\nYILW+lCe4w71vI7kPOX8DrTWb2utz9Fa12utLwKmA2/k6WNIv88hobUuiQewEzh/ENs+gXElzWqT\n2R74HYYnvjxl25eBQxgDI16Mi98JQAewJONY7wDHD9bujOdVwGXANow4XSHPlRsjoyfx+AnwINBo\n7q8BLgJcGKJ7DdALzBzJe8394zBuM72A1WzbC1yeYt/p5rbKHLb/Bngd8Ob5bDbz2D/A8OoTdvRr\nl/nePwF/ytPviUALcEKOfR7z78kYdzIN5vdWCdQCT+Xadpj+D/Keu4x292HEtj0YGV1dwNyBzuNA\n+4Z7Hs3vZ595nhzAGnN7A0YmVB1GLPoeDLGrwnCejjc/iwIWA88V+Hz2e56G+h0A883z5cZwinZg\nhHEH/H0W9HMdjk4P0w96J4UX9qOAICnCbm6/BuMq6zd/pCswUpccGe1+BHxlsHabzwMYt3pdGHcI\nnwesh/nc/SdwV8rrRow0xh6MzJfXgQsK8V5z/wvmvm6Mi991GX3+FvhLjmMdjeHFBDFukxOPazLs\n0RmP/xzMZ8LI8Lmun88ZzTjuExgXqr9gxEnXAP8OnAt833zfDOD2XNsO03eZ79w9AXw15XUd8A8M\n8dgNfDjH5806jwPtG8F5PAv4SWof5t9zgR+Zz79hnuPl5uMec9u55v5FwP8W+HwOdJ7Szmt/34G5\n78cYTmDicx871P+5QjyUeVBhGCilzgG+o7U+p9i2CP2jlHJgZDzM10MI7yhj8sljWusnlVJ/xPDu\n5gJ7tNYPKaU+hCEOzsxtWuvbC/9JissIzuPngNla6xuVMQlomtb6+0qpLwJ7tdYPKqX+B/i7NsZl\nEnnhPwGe0Fo/pYyJcsu01odz3kdZcCTH2AvBKxi34cIYR2sd1lrPHooYmbwC/EQp9X8YYYG3gXkY\nniUYoYd828qOEZzHeYBWSj2HMbj4k5TtifP2W+CHSqllSqmnMcI0fwH+Vyn1FIUdOC1rxGMfIWba\n04Y8u+dorXePpj2CMBZRSj2DkS4cL7YtRwLlkMdeVEzh9hbbDkEY4zhE1EcP8dgFQRDKjKJ47A0N\nDXrq1KnFOHQavb29eDwjrgpQNMT+4iL2F5dStn+4tq9atapVa904ULuiCPvUqVNZuXJlMQ6dxvLl\ny1myZEmxzRg2Yn9xEfuLSynbP1zblVK7BtNOsmIEQRDKjIIIu1Lq35RS680pw/cqpVyF6FcQBEEY\nOiMWdqXUJIxCP4u01sdjTCO/aqT9CoIgCMOjUKEYG1BhzhRzY5ShFQRBEIrAiIVda70PYxbZbowF\nFbq01k+PtF9BEARheIw4j10pVQs8hLHEWyfG+n8Paq0zF0C+HqOQFk1NTQvvu+++ER23EPh8Prze\n0p1bJPYXF7G/uJSy/cO1fenSpau01osGbFiA6mj/grHaTuL1x4Bf9/eehQsX6rHAsmXLim3CiBD7\ni4vYX1xK2f7h2g6s1IPQ5ULE2HcDpyql3OYiCecBGwvQryAIgjAMChFjX4GxEMNbGJXXLMAdI+1X\nEARBGB4FmXmqtf4WxlJzgiAIQpGRmaeCIAhlhgi7IAhCmSHCLgiCUGaIsAuCIJQZIuyCIAhlhgi7\nIAhCmSHCLgiCUGaIsAuCIJQZIuyCIAhlhgi7IAhCmSHCLgiCUGaIsAuCIJQZIuyCIAhlhgi7IAhC\nmSHCLgiCUGaIsAuCIJQZIuyCIAhlhgi7IAhCmSHCLgiCUGaIsAuCIIwCO1t7+cxfVrJuX9dhP5YI\nuyAIwijQ3BPiqfWH6ApEDvuxRNgFQRBGAX84CkCFw3rYjyXCLgiCMAr4wzEA3KUi7EqpGqXUg0qp\nTUqpjUqp0wrRryAIQrmQFHa77bAfq1BHuA14Umv9QaWUA3AXqF9BEISyIDCKoZgRC7tSqgo4G/gE\ngNY6DIRH2q8gCEI5MZqhGKW1HlkHSp0A3AFsABYAq4B/1Vr3ZrS7HrgeoKmpaeF99903ouMWAp/P\nh9frLbYZw0bsLy5if3EpNfv/viXMw9si/PEiN/7e3mHZvnTp0lVa60UDNtRaj+gBLAKiwCnm69uA\n7/b3noULF+qxwLJly4ptwogQ+4uL2F9cSs3+7z+2Qc/8+uNa6+HbDqzUg9DlQgye7gX2aq1XmK8f\nBE4qQL+CIAhlgz8cxeM4/AOnUICsGK31QWCPUmqmuek8jLCMIAiCYOIPx0Zl4BQKlxVzI3C3mRGz\nHfhkgfoVBEEoCwLh2KgMnEKBhF1rvQYj1i4IgiDkoDcco2KUQjGjcxShpAiFQjQ3N9Pd3Y3b7aax\nsTFrBH8wbQRB6CMQjuK2j47HLiUFhCw6OzuxWq3MnDkTj8dDT08P0Wh0yG0EQejDX2qhmCMRv99P\nb28vfr+fcDhMTU0NNpuN2traYps2IsLhMPF4HKfTic1mw+124/P5CAaDSY98MG0EQUgnUIKDp0cU\n0WgUn89HTU0NWmsmTZqEzWYjHo8X27S8tLa20tramph7kKSyspLGxkacTicAsZgxO85qtab9TWwf\nbBtBEAxW7+7gx09tpjMQEY99LBMKhXC5XEQiEVwuFzabcRpbWlqorq5Ga017ezvhcJj6+nqqqqpo\nb2/H5XJhsVhwuVy0t7cDxgQxm82W/NvW1kY0GmXy5MkAyddOpxOr1Up1dTU7d+6kqqqKUChELBaj\nq6sLn8+H1WrF6/WitcZut+PxeJI2NzQ00NDQMPonSxCOcF7b3sar29oAcMvg6dhGKZX2NxaL4XA4\n6O3txWKx4PF4qKysxOv10t7eTm9vL6FQCI/Hg91uJxgMUl1dnQxrWCwWvF4vXq+X1tZWAoEAbrcb\np9OJ2+3GZrOhlKK1tZX6+nrcbjehUIhoNIrWmqqqKrq6unC5XMnwUKqwD9Zjz/S+M73zwbYRBMGg\n3ddXOktCMWMYp9NJe3s7NTU1dHZ2UlFRQTQaxeFw0NraSmVlJS6XCzBi8Tabjfr6egKBALFYjFAo\nhNY6TQjtdnvyuc/nY9KkSdjtdurr64nH4+zatYsJEyZQVVVFd3d3UtDtdjs+nw+lFI2NjckLSzwe\nT14wYPAeu8PhwGKxJC8afr8frXXy8wy2jSAIBu29fcLuGSVhl6yYYWCz2fB6vXR2dhIIBNixYwcH\nDhwgEAgQj8dxOBxEo1HsdnsyU8RmsyVDNh0dHdTV1RGJRLBarWitk55/d3c3TqczTegTsftwOJy8\ngCQ8eL/fT01NDY2NjQSDQZRSSTEfbsy/urqaaDTKpk2b6Onpwev1JsNEzc3NxGKxvG0EQUintTfV\nY5dQzJjG7XbjdveVne/o6KC2tpbq6mp8Ph/d3d1UV1djt9s5dOgQ9fX12O12QqEQkUiEiooK2tvb\nqaysxOfzEYvF6OnpoaWlhXg8zqFDh6iuriYYDNLd3U1DQwOVlZXEYjEOHTqEUoqKigpcLhddXV10\ndnZisVioq6sjHDZ+SMMNjbhcLqZMmZK1vb6+PvncarXmbCMIQjrtvaHkcxk8LTESaY52u53a2trk\na7/fnwyBVFRU4Ha7k/vq6uoAsFgsSS+4srIyrV+Xy0VNTU3ytdVqZeLEiQBs3rw5GeZJiK7Wmkgk\ngsViSd4FCIJQPFJj7CLsZUKqV5+PioqKZLnNkYpxLBZLy9QRBKF4aK3TQzGjNPNU/vv7IRQK0dXV\nBRgCnWvyzWDaDITD4RiZoSmkxvIFQSguveEY4Wgch9VCOBYftXRHGTzth87OTurr62lsbCQQCOSc\nMj+YNoIgHJkkwjAzxxsh1tFKdxzzwu73+2lpaaGlpYWOjo5RO244HMZms2G1WpMDlcFgcMhtBEE4\ncmk1B06XzGzEYbUwqaZiVI47pu/ZE1P3GxoakrnZIyV1kk5nZyctLS0AVFVVJSfpgBGrzpyUk8g2\nGUobQRCOXBIe+/mzm7jpvOOwW0fHlx7Twp6Yup/Iy078bW9vT2aUBINBYrEYHo8Hn8+HzWbD5XLR\n0dGB3W7PinmnTtJJ5H8LgiAcDhKTk+o8jlETdRjjwg5kZYlkesmJuitKKZRSuFwugsEgLpeLUCiU\n2d2gPXar1ZpV+CozL3wwbQ4n9q4uaG3t2yC1YARhTNFmCnu9t3AJEoNhTAt7Yuq+x+NJhmIikUja\nrEwgWZArUYArEAhQW1tLIBDI6nOwHnti9mgsFsNisST7HGqbw0nM5YKUejCCIIwtDnYFqHTaRi0b\nJsGYFvbE1P3W1laUUtjtdiwWCxUV6QMQ8Xg8uc3n86G1pqurK1lPZbi54dXV1bS1taG1ThbiAqPi\nYk1NTbLaYq42o0Hc6YSK0RmMEQRh6OzrDDCpdvT/R8e0sEP21P2EiIIxscfhcBCJRPB4PMRiMWKx\nWDL+3tPTQyQSGXaeuMvlylnYKnVqfb42o0IsZjwSSHVFQRhT7O0IjFomTCpjXtgzSRXVBInwh9Vq\nTZt+nzk9v9yw+3yQmgIqMXZBGBP89OnN7Gzzs68zwOJpdaN+/JITdkEQhLHOU+sPsbXFRyyumVyE\nUMyYn6AkCIJQSgQjsaSoA0yqGbheVKEpmLArpaxKqdVKqUcL1acgCEKpsbW5T9SBogyeFtJj/1dg\nYwH7EwRBKDk2HOgGwGFOSCrG4GlBhF0pNRm4FPh9IfoTBkesogK83r6HIAhFZ8P+birsVs48rgGn\nzULDKE9OAlCZixsPqxOlHgR+AFQCN2utL8vR5nrgeoCmpqaF991334iPO1ws5sSl7lhsWGV2xwo+\nn0/sLyJif3EZq/b/6I0AoRhcPcvB7p44502xZ7UZru1Lly5dpbVeNGDDxAIPw30AlwG/Np8vAR4d\n6D0LFy7UY4Fly5YV24QRsfzZZ7WORPoeJUapn3+xv7iMVfsv+Oly/Zk/r+y3zXBtB1bqQehyIUIx\nZwDvU0rtBO4DzlVK3VWAfg8fWhuPEsfm80FnZ99DEISi094bodaT7aWPJiMWdq31rVrryVrrqcBV\nwPNa64+M2LLDSVub8RAEQSggWms6/WFq3aMfV09F8tgFQRAKRE8oSjSuqfOUkbBrrZfrHAOnQuH4\n4RObOOfHy4pthiAIOegwy/QW22OXkgIlxm9e2FZsEwRByENiYY1ix9hF2EuUeFwTraiAMi90Jgil\nRKc/AojHXhzco1+7odD0hKJohwNSVn0SBKG4pC6FV0xE2EuUnmAEFY1CJNK30V7c2z9BONLp8CdC\nMSLso088XmwLRkx3IIqttxe6uvo2Sj12QSgq7b1hbBZFpbO40npkCnt7e7EtGDE9wcjAjQRBGFU6\n/BFq3I5hL8dZKCSPvcRI/F66g9HiGiIIQhYdvWHqipwRAyLsJYfTZnxl4rELwtijfQzMOgUR9pLD\nZTcWrO4OiLALwlhge4uPSMwYt+voFWEvOg++G+bp9QeLbcaQsJixmGA0TtTthqqqvocgCKNKlz/C\nxf/7Eve+sRuAg11Bxle7imzVkSrsHg94PDy6PcL1f1lVbGuGRTgaR9vt4HD0PQRBGFX2dvoJx+Ks\n2d1JdzBCTyjKhDEg7EdmVkzF6C9VVSgSaymGo3FUPALhcN9OEXdBGFUOdgUB2HiwhwOdxvOJRVgK\nL5MjU9hjsWRMrNQIRw27I7E4tqAfurv7dkoeuyCMKvtNYd/a3MOutl4AJtaIx14cOjoIlOjgY9i8\nIIWipXlhEoRy4mCXscxmJKZ5eWsrABOqi++xH5kxdiAQieXd1+YLMfWWx3h1W+soWjQwsbjuC8WU\n6B2HIJQTB7qCWC1GQsNzG5uxWhTjKotfv+mIFXZ/OP8En7V7jWXm7nhx+2iZMyjCKV56WDx2QSg6\nB7uCHD+pGo/Dyr7OAE2VTmzW4stq8S0oEoFwfmFMpBQmvOOxQqqYl+oYgSCUEwe6gkyureC9CyYC\nUOkq/qxTKGFhX7Onkw37uwdumIf+PPbErVV8BAte+0JR1uwp7ALToVhf+CgcjRP1eKC6uu8hDItQ\nKERzczPNzc34fL4RtxOODLTWHOgKMLHaxdWLpwCwo7W3yFYZlOzg6ft/9QoAO3946dDf7PXSa/cD\nYMlRq6cQHvsNd63ipS2tbPzOxVQ4rMPuJ5XMUIy22aRUbwHo7OykoaEBi8VCa2srLpcLmy37XyOz\nXSyWf5xGKH+6AhGCkTjjqyuYP7maqxcfxVnHNRbbLKCEhX1EuFz0KOOj54qHJTz1kVT3fWmLMfDa\n0hNiSv3I6r+v29dFjdtOJNZ3oQnH4qhwGEKhvoZjbNENv99Pb6/hwdhsNmpra4tsUTbhcBibzYbV\nalx8KyoqCAaDeL3eAduFU+cQCEccB8xUxwnVLpRS/OAD84tsUR9HprBHo0RDxj9lruKaCc84NsxQ\njE55X4svyJR6N1/7+ztcOm8Cpx+bnWve0hPCYbVQ7c7tfV/2i5cBeOymM5PbQtE4tkAAenr6Go4h\nYY9Go/h8vqSHGy9CDfzW1ta07yJBVVUVTvNcxWKxpFgDWK3WnIKdq5147Ec2B8xUx7FQQiCTkhT2\nXP+sQ6KzM32BihQ+feebeMwi+cMNxZz6g+eSz1t6wvSGoty9Yjd3r9idFjr60gNrOXFKDV//xzqm\nN3h4/uYlWX2lflafWarXalFjfvA0FArhcrmwWIw7osTf9vZ26urqAOhJvShl0NnZSU1NzaC356JB\nJmwJh5FUj32sMWJhV0odBfwZGA/EgTu01reNtN/+8IVGXos8l2hrrXl2Y3Py9XAGT32hKIe6+8Ij\nLb5Q8gfgsKWHfR56ay8PvbUXgO15Bl0Si+Mm+gZjPcVSSHfMXGwg0+tNXLR6enqIx+NYLBYqKyvR\nWqOUIh6P09bWhtPpJBqNUltbSzQapbu7m2g0Sl1dXdZ7UxmMx57peWfamGCw7YQjh4NmDvu4yrEn\n7IXIiokCX9JazwZOBT6vlJpTgH7zUohFJmI5PN7UGDYMz2M/0Gncnv3sQwtQygizJG7Zqlx919HB\n3nU09/RdJJLC7h77wu50OgkEAskQTDweJxKJEIlE6OrqorOzE4vFkhRLi8WSDIFEIhHsdjvhcJiK\nigqqqqqwWq1EIhFcLhdVVVUopXK+N5WGhgYaGxuzHs6UkJXD4SAajRKLxdBaEwgEcLmy/1FztXNI\nbZ4jmv2dQcZVOpNZdGOJEQu71vqA1vot83kPsBGYNNJ++6MQi0wkRFwDvaEoXf4IoWh6zHQgYe8J\nRvjVsq0EU2ax7jOFfUqdm3qPwxR2w2P3pKyDmGvma0Lsd7X1Jp+3pAh7j3lBq/XYx/zMU5vNhtfr\npbW1lZaWFrq7u4lEIlRXV1NdXZ3MPOnp6cHr9eJ2u5MecDgcxm63JwUejAtDOBxOiq5SKud7h0N1\ndTVtbW00NzdnZcS0tbUlLyCZ7cRjP7I52B0Yk/F1KHCMXSk1FTgRWFHIfhNsbe6huSeUXGwCSN62\nD5VoymDeWf+9jPbeMG9+7fy0NgOFYh5Zu58fP7WZnmCUW94zCzCu4mDUi2jwOnlrVwfVFYY4uR02\nbl++jdOOqWd8VfYP4tG3D/DdRzfQ3BPii+cfxxfPn0FzTzC5v9u8oNV7nOxq8xP1emGQ8eZi4Ha7\ncbv7MoLa29uT2SaRSASbzYbNZsPn8xGPx5MiHo1G8Xq9ye3BYBC3200gEMBmsyVDLxaLJeu9w8Hl\ncuX00gHq6+sH1U448jjQFWTW+MqBGxaBggm7UsoLPAR8UWudNXNIKXU9cD1AU1MTy5cvH/Ix/rw+\nxJsHo9xwQt8/13PLlmMb4q2QCofZsD8MGHHc9l7jNv6Fl19Ja9fj62X58uXEtWZPT5x6l4U73g7x\nwRl2plRZ2b7XENrH39rBqRXGgh2vvRvGomDjW6+jQ0E2d8TZfMgYJNx4oJuNB7qZ6FV8fkG2QNx4\n7+rk89uXbeEE235W7Oi7O1m5YRsK6O1ooTcQpSegufuJF/naKwHOnGRjZ1eM755RUfSFdAdLMBhk\n5cqVeff39PRkxc3HEj6fb1i/47GC2D98tNbsa/dznDs0LBsOt+0FEXallB1D1O/WWv8tVxut9R3A\nHQCLFi3SS5YsGfJxnutcx+q2/cycczy8aQjCyaedmfSIh8KaZ7fAznfRKIyADJy4aDG88EKyjc3p\nYo9rGt/4xzoATptez9utflRFJQ9//gx2v7YT1q3H6/WyZMlZADzSvIYJ1e2cd+5SfrT2BSB7hmLc\n4mTW/BPgldfy2heKwbR5J1Mb2AObtwGgK2qpdHUwdcpkVrXupcqhaLFMwBndxJu7woRsDuYuPI1x\nOe4GxiLLly9nOL+DsYLYX1yKaX+XP0Loqac5+fjjWHLW9CG//3DbPuIYuzLcwz8AG7XWPx25Sf0d\ny5Dg1Ph0sJ8qjXmJRIiHjdh1ahw9FEmPW/eGYvziuS3J1wnPO2rGt/3hxOAdbD7Yw033rmZ3mz+Z\n/pSa0ZJgWoOHg91BVu/OX27gG5cZY8/PbDhEe29fjP3dQz4qXXacNgvhaBxrIECkuxt3OIA7bMT2\ntzTnnuo+rPOUh2gsPvKUU0EoMfzhKMs3G1lz+82EiKYx6kQVIivmDOCjwLlKqTXm45IC9JuFRSni\ncU0gnF4zZch0dWHpzs6hDmYMnnYFImlZKYmQTeJi4E9Ju/z3B9bwyNr9rNzVkVxB5RdXn5h1jI+d\ndjQA33tsY3LbpIwVV644cRKzJ1Tx9PpDtPeGqTEnLu3rDOBxWnHaLISiceJaJ1dtSbDlUPbn6ugN\nM+sbT3L78m1Z+4aK1ppjv/YE33h43Yj7EoRS4oE39/CJ/3uTrc09bDpoRJuPa/IO8K7iUIismJe1\n1kprPV9rfYL5eLwQxuU8Hune53Byzd/Z18UDK3dnbU+9YED+rJhoXPOVB9/m589vBQxPPzXlKSHs\np0yv56EbTmPJzEb++IlF3PmpxZx+TPakmcwVV6pcNs48tp7Vezpo6w0zd2IVFeaAcb3HSZUZegpE\njTUXU8mVD9/iMy5OP3pyU84yDmZxAAAgAElEQVTPkyAQjnHtn95M/mhzkUg1vev17PMnlD5SEC0/\nmw8Zn3PFjnbe3ttFhd3KsY1jU9hLauapMsPhgTRhH3o/P3pykxFGyfj0XYNcVSkai3P/yj3J14FI\njMqUHPVJKUK98Og6/vTJxcnXuapKNmYU5rdZLTR4nURimj3tAU4/pp7pjR7W7+9mfLWLWreRP+2P\naFp60vO397SnCz2kX7Dicc3+rgCTarIHWV/e2spzm5qJxjV3fqrPZl8oSigSo97rTK7xKJQnwy2I\nlq9dObHNDHOu3NnBnnY/cydWjYna67kYm1blQaGMGHtKLfU3drQx4+tPJMMkg6EnzwSnn6fE0/sj\ncyJTMBJL+4L7WxrL7bBR6Uz/B7hk3oSsdonwS6svRJ3HwbQGDwDjqpzJfd9+LcDejnQh39sR4G9v\n7eW+NwyPeldbL5f/qi/bZ/pXH+fMHy3j1W1tWcdM9FWTUbPmgp++wMLvPQv0xRYB/vzazryf80jB\n7/fT0tJCS0sLHR0dxTZnRKQWOlNKJQuiDbddubGtxRD217e3sW5/F/Mmj91S2SUl7BZlxHhTPfZf\nLttKOBrnrV3GP9XWZh9Tb3mMFdvThWvNns5kfRV7nk+96WBffPr82U157cgM0QTCMawqOxSTjzXf\nujD5/I6PLuTSeROy8tqrK/pmNdZ5HNR5jNe1bkdSeLtcXtodHtxNDXS5jFvCLc0+/v2Btdzyt3eM\n/vOsAnXbs1v46B9W8NeVe7jhrlW8e6iHjQeMEEzmXdCBFC891WP/5sPr+/2co0prq/EYRWKxGD6f\nj/r6ehobG6kewzXxExPFUh+dnZ2EUqqDDrbQ2ZFYEK2jN0xbb5ij6io40BUkGIkzfwwLe0ndOyll\niE5qjD1mes9mjSlW7mwH4K+r9tJmfhmnTKvj/b96hevPns5XL5lNxFOJr7v/GM43LptNS0+QtXuz\ni4VlhlMCkVha/ZrJdf0Lu9WiOGlKDefNbuLCueMBeOWWc3lq/cHkZ0v1mus8juT2QDiWFP24xfjn\nWji9kUfW7s86TpsvxN0rsmPhE6pdvGGep0R54WMavbxrxhCbu3N7X8FIjFe2potnfxPE2nwh3vfL\nV/j9xxcxe0JVzjYFowiimpgJm1noLBAIJEscKKWoqqpKFj/L/NvV1UVVVRXxeDxZ96auro5oNEpP\nTw8WiwWHw0FFRQVaa7rM4nVOpxO73Z7VJhKJEA6H8Xg8abbmKohWU1OTVl5ByM9W01u/5eLZrNjR\nxpZDvjFTez0XJSbsCo1Om04fNd1LZRbgTSxN1ROM8Lm73wLgrmtPAeCdhEg7nUSs/ee+11Q4uO2q\nE7n2zjfZ1mIMSE6td7OzzZ9VqyauDa92wVE1/OKqE6kaxPJYf/vcGWmvrRaVFpJJzc2v9zi4+Pjx\nbDjQzTWnTkkuBOKMGN7WOUd5eWpliGOPbmR9yqpSH/5d7gnAi6bW8c+MC8HB7iDbzR9vaiZQ6rjD\nq9taefTtA2nv6w5E85YbfuHdFvZ1BvjNC9u47arsDKGCUqQFRzIvauFwOFk6AYwLX8LDzfybqKOj\nlMJqtVJTU0N7u3HBDQaDeDweHA4H7e3tyXBHYvZrR0cHsVgsq43dbsfn82UJe66CaAmPXQqiDcwW\n0+mZP7maS+dnh07HGiUVilGA1sYAYIJEWCTx/5VITnlq/aFkm7D5I0y0ceso9pghWC99eSnXn509\nwaDSZWNqg4f7P3NactsjN57JFSf2lcG5+cIZfOL0qYARC7/q5KNGvKhGgkyPvcHr5E+fXMy4Sldf\niYJIEHckyMIGO6/eeArfvCy99trmHKmPADeeeywAl86fwNP/djaLp9bxwrstyQvWwa5gUgRSB2Nf\n326IzglH9ZUxONgdJBKLs2pXB0t+vCyttk0iUyiWkaJ6WAiF0hcdGQUcDkdWoTO/358mqkqpZM2b\nfH9zkSih0N3dnew/U0BztUkcM7P+fa6CaJke+0gKopV7qYVH397PpJqKrNTksUpJCTvmBKXUGHfS\nYzdVO1dxrcREIYtSaK2pCPbiDRmCdVSdmwvnpMfT//iJRVhMUUqtS1PlsjMzpTbEhXPHp+Wx/r9F\nR43k06VRkxJjr/emVxG0Wy2cMq0u+froeg/1XuegCxLNaKpk43cu5pdXn8iMpkqq3fakIC+Z2Ugg\nEuPq373OT57azLJNfWWM15pruKZm8RzoCvCpP73Jlbe/ys42P8s2Z5c9Xru3k9nffDKtr4LT05O+\n6MgoYLVaswqd5SKfoMfj8bxhLIvFQnV1NVVVVckQT6annKtNYvtwFzbJVxAttRhaf+3Kka3NPby6\nrY0PnzIlqQtjnZL6Nizm1NPUlY2SHrv5OpewJzJmLBbFtFsfp9afHjfPLElw7qw+oXdm1FD3mOuX\nNngdzGiqTA7annVcQ0HLd6auk1rnyY6D3v+Z0/jhr/Zx5sK5SXFo8A4+Xpra/2XzJ/DMhkNUV9i5\n8qTJLN/cwuvb25MeulLGndLqPZ1U2K1pF7sNB7qTcfoE+zsD3PzXtcnzvqfdyKT5+fNbeH5TM7de\nMgu3o6R+ennJLHSWqBdvsVjQWlNVVUUkEqGysjJZ+ya1Bk5i8DIRY49EIvh8PioqKujp6UFrnSyc\n5nK56OrqSi5iEovFstrAyEIj+QqdpRZD669dOfLImv1YLYqrTi6c43a4Kan/LoXhBcbTPHbDM7ln\nxW4WTK7JmcaX8NjjeZLe88WIwfCOU0kIUuJicM7MRj5y6hT+48JZg/8gg+Sua0/h3jd2U5OnFs5Z\nk+2ccWzfAI7Hmf11fvOyOXzn0Q0AfH7pMSydOS6rzeUnTOLyE4wQU6svO5zxgRMn8/LWFg51hzim\n0cOVJ01KxugfWrU3re03/rEOl92ac07A6t2drN7dyfRGD588Y1q+j13S5FrbNbFiVOZfu92O328u\nqm5636mZNZkrRSmlsrZlvtZaY7FYSqYQ3EgIh8M0Nxt3gW63O2udWjAunIkB53xtBmJ7ay+Tayuo\nH4LjVGxKKhSTqBWTmkaeuON8cv1BFnznaR7LGNyDvvrt3XnquKeGPQbCbXq6CY93QnUF33v/vH4v\nDsPlzOMa+NU1J43o9i91QPY/LprFoql1/bQ2BmozWTKzkasXTzH2e50smTmOnT+8lIVH1yYHlhOE\nonG6AhFOmpK/nHDirioc09z/5u68F9wjgUS2S6FIDKgeCaSmmgYCAaLR7PkpnZ2dA7YZiD3tfqbU\nFWbsbLQoMY/diJHn8tj7I2gW98pXCCtzybpMpjd4eL85aJoQ9Ap78bMAIpWVUJcu1I/ddCZVLjs3\n3L2Kdfu6aaoampehlGLtNy/E5bDwq2Xb+PWyrZx5bAMXzR1Pg9fJnIl9aYtzJlSxalcHVotKG/c4\nflIV/3bBDD76hzdyHuPh1ft57/yJPLw1wmM73qG6wsj6ORIp9CpMhY51+/1+ent7k31n3o0Ui3A4\njNVqTYacEllDqR556kSqfG0Gw+52f85JhGOZkhJ2S47B08E4e4niXiGzYFiPc2geTeoi04nDucaA\nsGOx9CXwm8ydaNzK33PdqbT5wsO6JU/cfVx/9nQunNNErenFf+TUo9PaLZpay19e30Usrnn8prO4\n5OcvAfDw58/kUJ5ceDCyda7780rqLMbZ3Nvh5943drNiexvfu2Ie3hwhpQEZwwuOlCrRaBSfz5cs\nHTDcAdkEiQVS+mMw69RC7klSmcsjDqbNQHQHI3T4I+KxH1aUQmcMng6GRKpdwmOPWm3Mm1TNR48Z\n2pcMfcvXDWWg8nBhCQbBn1JSIGUQr8plT+bT17jtOUsID4TXaeP4Sfkn/lyQkk00Z2IVv//YIg52\nJxb47f/8dAUiVDiNc7n5YA8vbWnlYHeQeq8zWbZ4SJRxVkaxSAzSZk7ASkyuAujo6KC2tpbOzs6s\neH8m3d3dA7bJNZGqmCTSfUXYDyMJ33OoMdmg6aknPHZHNMKsOgfj3H3xtj9/ajEf+2Pu0EEq58wY\nx03nHsunziz+4J81FMor7Km8esu5wyqWNhBuh43ffWwRDWY65vkpQj9QcaRxVS4OtBsDtev3dyfH\nQf6+eh9fuXjWgOGxLBK1So6QTI3RIvOOL9ML1lqjtU7OlI1Go9TW1qK1TmbsgJFFk7gD8Hq9yVm2\nFoslbZWswXrsg5kkVYiJVLvb+tKiS4nSEnbzNxbtR6UeuuE05kyoZvY3n0xu85mikVhIwxP24wml\nF886e0YjVovKymnPxGpR/PuFM4djftE4nKmFF/Rzvn5+9Yl4HFauvbNv+bvzZo1jT4effR1+2v3G\n97jBrFFz/uxxPLuxmde3t3H2jL5sn+aeIG6Hrf8QTaJ0rAh7wXA6nbS3t+PxeJKhmEgkQiQSoaur\nC601VquVSCRCRUUFHo+Hzs5O4vE4gUAgmaETjUaxWCzJNgmxtVgsWaGRwXrsDoeDWCxGLBbDYrEQ\nCASy4v+pE6nytRmI3QmPvUATD0eL0hJ2+mYyZg7YJXA7bGk52gBvmasVpS6kYbVYgPSY4bb/Oizr\ngxyxvG/BRAC+fulsZk+o4vRj6lFK8YMnNvLbF4ziZAuOqklOfPrkGdN4cUsrL77bkhR2fzjK4u8/\nxxnH1nP3p08tzgcZyyRm2x6Gmi82my05AUsphd1ux2q1Ul1djd1uJxgMJsU+MWibEPNEWYWEx+/3\n+5OzbHt6eqiuriYej4+oeJjX66WtrQ2tNW63O20yVU1NTdLWXG0Gy+aDPdR7HIMqEzKWKClhT2T9\nRWJx7Nbcwp6Zd55K6h3eUBfAFobPpzPWhGyq7POqLzl+fFLY506sYvHUOp7b1MzXLp2N1nDfG0bd\n+1e2thGNxbFZLWw80E0kFmf+5BrW7etiQrWL9OkzRxCJ2baHqZhX5gSs9vb2ZFZJJBLB5XLR29tL\nLBYjGAwm27pcLjo7O7FarTidTiwWC36/H4vFgs1mw+fzEY/H85ZUGAwOh4Nx47LnZaROphrJRKp4\nXPPCuy2cedzYivsPhpIS9kQoJhbX2K2WZBpjKo5BFr4v5CxRYWhMqjXqbUz0qLSSDDVuB+8/cRI3\n/3UtS36ynH0dAU47pu+fdM2eThZNreM9txnZNzt/eCmX/eJlGiudvPmZE0b3Qxyh1KWk1yZi47kG\nRHMJauL1cCYJjTb3v7mbp9cfoq03zLmzsi8eY50Sm6BkiHHUFPZc2KyDE2xrGczMi1RVQX1936NE\nOG/WOG6/5iS+dVoF4yrT//nfu2AClS4bu9r8ROOal7a0cqmZQ/zGzva0YmKJ2a0tPSF8oShX/PoV\nXt06ujXZhfLkB09s4rlNzVgUnDNj7JbnzUdJCXsCw2PPLcz9hWISdLu8RKoPc33w0UCp9EeJYLNa\neM+8CThtinEZE6icNitzMmq3nz9nHNMbPTy57mDaoPgbO9qTz+ff9gZrfYqv/UMW2RaGz0+e2swP\nnzCWzqx12/nShTOpcRd2EtloUFKhmEQd8kgsji3PRIfBhGJiFis2e+l9WZlYAgHoTZnSX4JTyevN\nAmcXze3Lrplc62aFKdrnzGjkornjeXVrG3/NqEuTuuhHYtGRwa5bKwi5ePTt/exu9xPX8N9Xzk/O\nOC81SspjT42x58tzzheKSY2TOaNhLOHSX6PRGg5DIND3KEGsFsVrt57Lz6/uW4gjsTD4t983lzs/\ntRi3w5Yzznn/m8bAqsdhxRkJ4YyE6ApE2HKoh+/8cwO9oSi9oaHXBikpampk1m2BiMU1+zoDyTkf\ncyeW7l19aQm7+XeooZirFx/FiSmLQ7jDAaJdo1u7W8jPhOoKnLa+FNUvnHssV5w4iStO6vOWEpOf\nHDYLv//YIq47a1qymNifrz0luehILK654Gcv8sdXdvDVv7/D3G89xZPrsgvDlQ02m8y6LRDGojHm\nhCq7hemNY3+QNx8FEXal1MVKqc1Kqa1KqVsK0Wfu4xh/o3GdNxSTS/BddmvWTEhfsMw9uRKmwevk\nZx86IS132G618MJ/LOHF/1jK+XOaeN+CPtFPXfzkpvOOS04ye3iNUVp480HfgMd89O39rN7dUaiP\nMHoEg32zboURkZhlOq7SyclT60o6c27El3qllBX4FXABsBd4Uyn1iNZ6w0j7zjpWygQle55QTK6i\nVxV2a1beem84BoyBQl7CoDm6vm8MIbXKpMdh5eK545lYU8GNF8wA4JivPp6c59CTUa65uSfIL5/f\nypcvnoXXacMfjvKFe1YDRgplSVEOM25bzbGSIteJ2dNhCPvdnz6l5EoIZFIIj30xsFVrvV1rHQbu\nAy4vQL9ZqNQJSkO4mk6t92Rdfcs+9lrmWC2KSrPEgFKKH1w5nxvPOy65P3XyWntvmO5ghC/c8xb7\nOwN8+58b+PNru/j7W8Zg7MtbJEVSMAp+WRRMbfCMjeqtI6AQwblJwJ6U13uBUzIbKaWuB64HaGpq\nYvny5UM+0LadhucVCIXo7cmd/ZDZ70dmO2j0beWdPelCfrCtA5/POyw7xgo+q5Xl60o3vc/n843o\n/P/gDAeBqJ3ly5djN1fJiZgrENU4FZ0hzQSP4t09B/ndw608+naIFzYeIGTOa/vxExtQrdt5frfx\nW6p3qSHZM1L7C0Hm5x4KY8F+GP5nKKT9G9ti/GlNkFqn4pWXXixIn/1xuM99IYQ9l+ucNddfa30H\ncAfAokWL9JIlS4Z8oO0v74BNG7BY7TQ21LCxvSWrTbLfJx8D4MYrzqKpysWBFbthwzsAdLm8fP5D\ni7C3vctw7BgrLF++XOxPkKgVbo69/P34Xtbv7+LBVXtZvrmFda3GQGtPxFgl6r0LJvKnV3fy3RWh\nZG2hkLYOyZ4xcf5HEMYYE/YDJFY1GuIgcCHt/9StjxHXsKCpiiVLzixIn/1xuM99IUIxe4HUVV4n\nA/sL0G8WaSUFBhmKSYRgUmPsf7/xbC6YN7Hg9glFJGPRkWkNHi6bP5G6jKX+vrD0WB664XT+831z\nee3Wc4nE4sla9b5QNO8qW8JhpMiZPc09QeLaWPnr25cfXzQ7CkkhhP1N4Dil1DSllAO4CnikAP1m\nkTpBaTAzTAHs5j97an67LRQs2bxvIQ9+f3ptepPMJQxvvmgmUxuMQdgJ1RXJQbIqM3e+pSd7Me+x\nTKiyiqC3dPOtgaJn9mw8YKQ+f/3SOZxwVHnMCRixsGuto8AXgKeAjcADWuv1I+03F2ke+yAXYkgI\neurgqSOUMWNTKH3yCHvzAEKdmC6+wPyHfmNHO69uG3uDqa9ubeXOV3dmbf/SQ+u47m4joyccHdnS\ndUXD5+vL7hlllm1u5ol3jHkOs8eX+AUyhYLksWutH9daz9BaH6O1/n4h+sxFQpqjQwjFJIQ9Ne+9\nlPNThaFx9gAlV12mgzB/sjFw96W/ruXDv1vBK1tb82ZOjTSj6mfPvMtPn96cc18kFmfdvq7k6y5/\nBK01t7+wjf96fGNato/Wmjc27GPN5gPsafcz4+tP8JfXd2X1ua8zkLYqkT9c+JBTc3ewJMJYK3e2\nJ9Nfd7T2cu2f3uS+N/dQ53Ek1/otB0pq5mlqoauhhmJSxVyE/cjhI6cezZpvXoDXaePGc4/N2p+Y\nXXj8xPSMjGt+v4Irb3+VWFzz2ra2pGit2N7Ggm8/nSa+gyEQjvGVB99m1a4OfrVsK79YtpV/rt3P\nZ/+yijZf313Fd/65gct+8TJ/eW0nn7/7LRZ852keXLWXtXs6CUXj7Gzru9Pc3tpLpLsHW9CfrKPz\njX+sSxPY/Z0Bzvjh8/z4KeNCorXm6t+t4MKfvUhPeGjrJR7oCnDbs1uylqaMxzUX3/YSv3x+65D6\nG226AhE+dMfr/P6lHVz8vy9y+S9fTjp8iVBcuVBSnyZVjnPVhPnyxdlL1llMEU+dkSqLbBw5KKWo\ncTtY9+2Lcu7/+qWzmT+5mouPH8+Dnz2N37+0gyfXHwRg08EevvXIOu56fTcfPmUK/3XFPP759n6i\ncc0T6w5wcp61LbTWxHW6A/H0hoPcv3IP96/cg0UZi43feK8RQnly/UE+c/Z0DnYHk7Nlv/vYxmRo\n5VuPrMdvlivedKCHWFxT47bzn4/0RTwfXrMv+Xz55hYuPn48q3Z18OZOo5jar5dv4+YLZ/L8pubk\nwia/e8fKZRfonJP6cnHvG3v4+XNbeM+88cxo6pvtu68zQHtvmBU72gbVT7HY2uwjZn537x4yQj+f\nW3IMcyZWMbW+9Aro9UdJCbtlAI/9mlOOTj4/dXodr2/vK+ua+l7x2IUEHqeNqxdPAWDR1Drj8b1n\nWXh0Df5wjLte3w3APSt2c6AzkBxoe25jMyefYIRObnt2C62+EN+/Yh5Wi+Lz97zFy1ta+dGV85lQ\nU8EbO9qS6+0CzJ5QxYdOPopvPtwnzL990VgqcN6kaj5w0iS+/U9j4vYHTprE397qE+07X9uZVq74\nGI8DUGxr8xsTthQs29TMlDo3V9/xOuFY33Ff3trK31fvo6nKyXVnTed7j23kobf28cGFk/Oen/2d\nATYe6Gb17k5e224I9/YWHz3BCAe7Qlw6fwLbW427iHf2dQ0psWG02dZsiHlC1B/+/BnMn1w96Atb\nKVFSwp56/nPVhEkV7P/7xGLa/X0L5dampL1VjB8HFeUTTxMo6EIjr916LhaleG1bGy+lzEpdttmY\nNzFvUjXv7OtiTbOTr/14Ofs6jQyr17a38aULZ/L4O4bH//3HN2K3WtjRmj5QP3diFR9ePIVwNM4V\nJ05iZ1svV97+GrVuOw/dcDqd/jDf/ucG7FbFdy8/nmMaveztCPDGjrakqB83zsvNF83kRHeUXz63\nlW1be5nW6OGoWjePv3OAZZubk6JeXWFHa81Db+1l48FuFkyu4VNnTOOelzfz06c3894FE3DarPhC\nUboDESbWGCtcrdzZztW/ez1ZGCvBtpZe7nhxO+v2d3PGsfVsbzGEMhiJ8+6hHuamhLWae4L87a19\nfPiUKfnXDR3iAtPDZVtL3wCt3aqYNaGyLEUdSizGnvoVWHJ43akhlgqHlUnmDxRgweRqXvryUl67\n9VyqPc60nGehDCjgYiN2qwWrRXH6MdkXC4uCn/6/BVgtiv99K0R3IMKfPnky33rvHJq7Q9xkhle+\nedkc9nYEskQdYM6EKmxWC58+azr1XicnTall4dG1XHPK0ThsFsZVuZje4GHOhCo8ThufX3osP/jA\nPL5/xTw+tOgo7v70KTzz7+dw0dzxjKt0sWSWscJPS0+I/3fyUfSEojT3hLj1PbPMz6N43wkTeWr9\nQXa29jJrfCUWi+KDMxzs7wpy072reXr9QRZ97xnO/+kLNPcE0Vrzbw+sYWJNBfdcdwr1KY7R6t0d\nrNnTSTga58Z7V/PK1rakU7V6d2daDP4PL+3gh09s4r2/eJneUDR3vXyr1XgcZrY29wn7ceMq0yqK\nlhsl67EnlrZbOrMx6UlZ+vnHVkr1FfbJkRYnlDiJ9NUCLjZisSje+c8LsSjF23u7+PJDa5lQVcFx\nTZV85JQpPL5mN9+9cj5LZo5jyUw4ptHLTfet5quXzObKkybzwMo9bDrYw13XnsJLW1p4ZO1+DnQF\n04qZgfHbfOiG09O23XbViVnjSKdOr+fU6RkXm7o6TltUDY/u5Nozp3HOjEYev+ksDnYHWDpzHPs7\nA1wybwIOmyUZVpplrlA1p97K1y+dzX89vpGn1h/CblUEY3EWf/85zpnRyJ72AN967xxOP6aBm847\njm89sp7GSifPbmwGoMHrSN7RTK6tQCm489Wd/O+zW7jurGlsbfbx4hbjf3NXm58Tv/MMMa3Z+v33\npHvKiTklFX2OWCoHu4I0VjqHHEINR+P85OnNXDS3iYVH17G1xceCo2pYu6ezpGutD4YSE/b0OPkb\nXzuP6go7M7/+ZHLboBBhLz8S4lDgVaQqzfDBacfU83+fWIzLbtzpffvy41la3cqS4yck2549o5HV\n37gg+Tv92+dOZ83uTk4/toEzj2vg46dP5TcvbOOMYwee/j9v8iDrplgsVLgsbP+vS5J3sXMmViWr\nXyZmUmqtmd7oYXtLb1qZ40+fNR2HzcIPn9jEnz+1mGc2HOK3L27nhXcNQU60/fjpUzl/ThO/Wb6N\nv7y+ixq3naf/7RzW7u3kk//3JseN8zJ7QhW/Xr4NMNYMTXDzhTO46/XdHOw2JiHtbvcnL2572v24\nuttp9LpyCntzd5Czf7yMb713TtoYWiZdgQhf/ds7HNfk5dNnTWf9vi46/BHueHE7d7y4nW9eNoc9\n7X4uXzCR6Q0erijRlZEGS2kJe8pzj9OWtRCyjIkKh5Njxw288EKq8+F22Dg9RcQn1lTwnUJPWTed\nFIu7/zKzSimuPXMad7y4PSsD5GOnTeWqk6fgsFlYNLWO45oqufmvawGYlTJpZ1JNBR9cOJlAJMYn\nz5hKncfB0pnjePKLZ9HgddLmC/Pr5ds4f/Y4Nh/qweu0s/FAN0tnjUNr+J9n3gVgzZ5Ojqp1E47F\nufL2V6kPdHPvdaei3BG8LhtWi2JXWy/jq1288G4L4Wic17a1pQl7PK5582CUBb1haj0OfvvCNh57\n5wBqHTzw5h72dwWTs47nT67mO49uoMZt58qFk7PumMqR0hL2lH8ajzPb9HIdCBGEvCTuPgcQdjCy\nxvJ5valLTZ44xZiF21jpzKq1s+ComuQs3QQJ8W/wOvnH589gzoQqHDYLkVict/d2MXdiNXMmVHHt\nWdNY+N1nWbOnk+WbW3h2wyF6QlFigSDffGQ9r3YZA9ORmOblra186YIZbD5kZCGt3t2Zdsz7V+7h\nV2tC3PPucoKROMFojPefMJFTp9dzy9/eoc7joL03bGQYvW8uP3lqMx846cgQdSg1YU957nX2DXzM\nGl/JpoOy1J0gFILpDR5q3HZmpYRsBktqrRW71cLCo42MF6UUboeNeZOqeXlLK1vMgcxTp9dxbkMD\nty/fRoe7mmWbW5Ih1U/8dq0AAAtoSURBVBfebWFriw+HzcK+zgAtPSEaK520+UL895ObmFplYfrE\nWibWuLAoxeeWHMv4ahenTK/H7bBy3Z9Xcs0pU6h02cumuNdgKS1hT1F2r7Mvdeqvnz2NA12yPJgg\nFAKlFP/zLwto8OaZgTUCLpzbxPce2wjAA585zSjl0NrKsk3NzDx+KgCLp9WxalcHf3h5BwDXnTWN\n3720g2Wbm/mXhZO55W/v0BuK8aUTnXzkvSdnHWOaWeTtkS8c/vK7Y5WSEnZLWiimz2OvdNmTg1yD\nooA5z8IYocjLqpUb581uOiz9fvz0qfx15V56w1FOnlprhE+bGrn7y5dgSUk/tCjFH17eQa3bzr+e\nP4OXtrRyy0Nv8+S6gzy/qZmvXzqbybHdh8XGcqCkkrlTPfZK5wgmGBUw51kQhMFjt1q4+7pTuO/6\nU/vGxCyWNFEHWDS1FouCy0+YhNdp48EbTuc98ybw/KZmLp03gWvPnFYE60uHkvLYU0n12IeMlOwV\nyoUSvPvMCvHkGABu8Dp54DOnJXPuvU4bv7z6RD5x+tSyLQNQSEpK2FO/zMRyZsNCFtkQyoVyELg8\nmT2LptalvVZKcXLGNiE3JRWKSc1Tt0lJAEEw7j7lDlTIoKTUUaUkPErpXUHAuPuUO1Ahg9IS9hQt\nz1UETBAEQSg1YU95Lh67IAhCbkps8LTv+YgWy5CcZ0EYO5RgZs9Yp8SEXWLsglB2lENmzxijtIQ9\n5bksbycIlMfd52GopX+kU1ox9pQru0xQEIQyQTJ7Cs6IhF0p9WOl1Cal1NtKqb8rpWoGftcIjnc4\nOxcEQSgTRuqxPwMcr7WeD7wL3Dpyk/Ijc5IEQRAGZkRSqbV+WmsdNV++DkweuUn5UeKzC4IgDIjS\nWg/cajAdKfVP4H6t9V159l8PXA/Q1NS08L777hvyMd5uifLTVSEA/nTxyAdafD4fXu/Ay52NVcT+\n4iL2FwZ7VxcAkepBrvNqMlbsHw7DtX3p0qWrtNaLBmyote73ATwLrMvxuDylzdeAv2NeKAZ6LFy4\nUA+HZZsO6aO/8qg++iuPDuv9Wf0tW1aQfoqF2F9cxP7iUsr2D9d2YKUehMYOmO6otT6/v/1KqY8D\nlwHnmQc+bEgmjCAIwsCMKI9dKXUx8BXgHK21vzAm5UdS1wVBEAZmpHkmvwQqgWeUUmuUUr8pgE15\nkcFTQRCEgRmRx661PrZQhgwGicQIgiAMTEllhouuC4IgDExpCbu47IIgCANSYsJebAsEQRDGPqUl\n7MU2QBAEoQQoLWEXl10QBGFASkzYi22BIAjC2KekhF0mKAmCIAxMSQm7RNkFQRAGpqSEPRGKkfVO\nBUEQ8lNawm7+rXSV1FKtgiAIo0pJCbvFdNk9ThF2QRCEfJSUsCdCMV4RdkEQhLyUlLAHI3FAhF0Q\nBKE/SkrYfaEIAF6JsQuCIOSlxIQ9BojHLgiC0B8lJexLZzayeGodN184s9imCIIgjFlKyvWtdNl5\n4LOnFdsMQRCEMU1JeeyCIAjCwIiwC4IglBki7IIgCGWGCLsgCEKZIcIuCIJQZoiwC4IglBkFEXal\n1M1KKa2UaihEf4IgCMLwGbGwK6WOAi4Ado/cHEEQBGGkFMJj/xnwZUAXoC9BEARhhCith6/HSqn3\nAedprf9VKbUTWKS1bs3T9nrgeoCmpqaF991337CPWyh8Ph9er7fYZgwbsb+4iP3FpZTtH67tS5cu\nXaW1XjRgQ611vw/gWWBdjsflwAqg2my3E2gYqD+tNQsXLtRjgWXLlhXbhBEh9hcXsb+4lLL9w7Ud\nWKkHobED1orRWp+fa7tSah4wDVirjBUwJgNvKaUWa60PDnhFEQRBEA4Lwy4CprV+BxiXeD1QKEYQ\nBEEYHSSPXRAEocwoWNlerfXUQvUlCIIgDB/x2AVBEMoMEXZBEIQyQ4RdEAShzBBhFwRBKDNE2AVB\nEMoMEXZBEIQyQ4RdEAShzBBhFwRBKDNE2AVBEMoMEXZBEIQyQ4RdEAShzBBhFwRBKDNE2AVBEMoM\nEXZBEIQyQ4RdEAShzBjRYtbDPqhSLcCuUT9wNg1AKa/4JPYXF7G/uJSy/cO1/WitdeNAjYoi7GMF\npdRKPZgVv8coYn9xEfuLSynbf7htl1CMIAhCmSHCLgiCUGYc6cJ+R7ENGCFif3ER+4tLKdt/WG0/\nomPsgiAI5ciR7rELgiCUHSLsgiAIZUZZCrtSyqqUWq2UetR8PU0ptUIptUUpdb9SymFud5qvt5r7\np6b0cau5fbNS6qJRtH2nUuodpdQapdRKc1udUuoZ0/5nlFK15nallPq5aefbSqmTUvr5uNl+i1Lq\n46Nof41S6kGl1Cal1Eal1GmlYr9SaqZ53hOPbqXUF0vFfvO4/6aUWq+UWqeUulcp5Sqx3/+/mrav\nV0p90dw2Zs+/UuqPSqlmpdS6lG0Fs1cptdDUg63me9WgDNNal90D+HfgHuBR8/UDwFXm898AN5jP\nPwf8xnx+FXC/+XwOsBZwAtOAbYB1lGzfCTRkbPtv4Bbz+S3Aj8znlwBPAAo4FVhhbq8Dtpt/a83n\ntaNk/53Ap83nDqCmlOxP+RxW4CBwdKnYD0wCdgAVKb/7T5TK7x84HlgHuAEb8Cxw3Fg+/8DZwEnA\nupRtBbMXeAM4zXzPE8B7BmXXaP6zjMYDmAw8B5wLPGqekFbAZu4/DXjKfP4UcJr53Ga2U8CtwK0p\nfSbbjYL9O8kW9s3ABPP5BGCz+fy3wNWZ7YCrgd+mbE9rdxhtrzKFRZWi/Rk2Xwi8Ukr2Ywj7HlMg\nbObv/6JS+f0D/wL8PuX1N/j/7ZxPSNRBFMc/D4RCCSvBsCxsL11LJCRBAkMQwiA6GIFRQZfuQXjq\nGHTwUHQpOnQoKPrjqT9gh05FBpaVoVHY5qZSJNFJ6XWYt/lzcXctdtv9/Xgf+PGbeTOz+935zb6d\neTMsnK72/gdaWO7YS6LXysYj9mX1Cl1JDMUMEgbDL8s3AN9VddHyacIXAJa+CFj5vNX/Y1+hTblR\n4KGIjIjISbNtUtWM6cwAjWbPp7NS+lPAHHBVQijssojUER/9UfqA65aOhX5V/QycB6aADGE8jxCf\n8T8GdIpIg4jUEma4W4lJ/0cold4tls61FyVRjl1E9gOzqjoSNa9QVYuUFWpTbjpUtRXoAU6JSGeB\nutWmv4awLL2kqruAn4SlaD6qTT8AFoPuBW4Wq7qCrWL6LZZ7gBA+2QzUEcZRPi1VpV9V3wLngEfA\nfUI4aLFAk6rSvwr+Vu8/f45EOXagA+gVkY/ADUI4ZhBYLyI1VqcZmLZ0mjAjwMrrgW9R+wptyoqq\nTtt9FrgD7AZmRKTJdDYBs7n6c3RWSn8aSKvqU8vfIjj6uOjP0gO8UNUZy8dF/z7gg6rOqeoCcBvY\nQ7zG/xVVbVXVTtMyQXz6P0up9KYtnWsvSqIcu6qeUdVmVW0hLKWHVfUI8Bg4ZNWOAvcsPWR5rHxY\nQzBrCOizUwPbCRs4z8qtX0TqRGRdNk2I847l6MzV32+77e3AvC39HgDdIrLBZnHdZisrqvoF+CQi\nO8zUBbyJi/4Ih1kKw2R1xkH/FNAuIrV2eiLb/7EY/wAi0mj3bcBBwnOIS/9nKYleK/shIu32PPsj\nr1WYcm+IVOoC9rJ0KiZFGJiThOX1GrOvtfyklaci7QcIpwHescqd6BJoThGWn6PAa2DA7A2EDeEJ\nu280uwAXTecroC3yWsftc00Cx/5jv+8EngMvgbuEXf446a8FvgL1EVuc9J8FxgkTgmuEky2xGP/2\nvk8IP0ajQFe19z/hhycDLBBm2CdKqRdos2f5HrhAzsGEfJf/pYDjOE7CSFQoxnEcx3HH7jiOkzjc\nsTuO4yQMd+yO4zgJwx274zhOwnDH7jiOkzDcsTuO4ySM3zJVHUIcodcCAAAAAElFTkSuQmCC\n", 251 | "text/plain": [ 252 | "
" 253 | ] 254 | }, 255 | "metadata": {}, 256 | "output_type": "display_data" 257 | } 258 | ], 259 | "source": [ 260 | "ival = abs(z_val-7.).argmin()\n", 261 | "p=model.predict(X_val[ival:ival+1,:,None])\n", 262 | "c_line, z_line, zbest, c_line_bal, z_line_bal = process_preds(p, lines, lines_bal)\n", 263 | "plt.plot(wave, X_val[ival])\n", 264 | "plt.title(r'THING\\_ID = {}, z$_{{ann}}$ = {}, z$_{{pred}}$ = {}'.format(tids_val[ival],z_val[ival],round(zbest[0],3)))\n", 265 | "m = X_val[ival].min()\n", 266 | "M = X_val[ival].max()\n", 267 | "plt.grid()\n", 268 | "plt.ylim(m-2,M+2)\n", 269 | "for il,l in enumerate(lines):\n", 270 | " lam = absorber_IGM[l]*(1+z_line[il])\n", 271 | " w = abs(wave-lam)<100\n", 272 | " m = X_val[ival,w].min()-1\n", 273 | " M = X_val[ival,w].max()+1\n", 274 | " plt.plot([lam,lam], [m,M],'r--', alpha=0.1+0.9*c_line[il])\n", 275 | " plt.text(lam,M+0.5,'c$_{{{}}}={}$'.format(l,round(c_line[il,0],3)),\n", 276 | " horizontalalignment='center',alpha=0.1+0.9*c_line[il])\n", 277 | " " 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "#### Quality Assesment\n", 285 | "##### Purity and completeness (all redshift confused)\n", 286 | "\n", 287 | "QuasarNET assumes a spectrum corresponds to a quasar if there are at least `ndetect` features detected with a confidence higher than a threshold confidence (`c_th`). The redshift is defined as that of the most confident line.\n", 288 | "\n", 289 | "The following cells calculate the predictions of the network on the validation sample and plot the purity and completeness as a function of threshold confidence and minimum number of detected lines." 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": 31, 295 | "metadata": {}, 296 | "outputs": [ 297 | { 298 | "name": "stdout", 299 | "output_type": "stream", 300 | "text": [ 301 | "calculating predictions\n", 302 | "done\n", 303 | "INFO: nspec = 62395, nboxes=13\n" 304 | ] 305 | } 306 | ], 307 | "source": [ 308 | "## predict the validation sample\n", 309 | "print('calculating predictions')\n", 310 | "p_val = model.predict(X_val[:,:,None])\n", 311 | "print('done')\n", 312 | "c_line, z_line, zbest, c_line_bal, z_line_bal = process_preds(p_val, lines, lines_bal)" 313 | ] 314 | }, 315 | { 316 | "cell_type": "code", 317 | "execution_count": 34, 318 | "metadata": {}, 319 | "outputs": [ 320 | { 321 | "data": { 322 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD8CAYAAABpcuN4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl4VdXV+PHvykhCQoAEAiTM8yDz\n7MBgUcABZ7FqxaqoFdva6ltt+1NfWl/U2kGFaqmiYKto0SoqijYSoArILDMEVAgEwhgIkECS9ftj\nn8g1hOQSktzc3PV5nvvk3nP2OWcvErJy9nREVTHGGGPOJCzQFTDGGFOzWaIwxhhTJksUxhhjymSJ\nwhhjTJksURhjjCmTJQpjjDFl8itRiMg0EckWkbVn2C8i8pyIZIjIVyLS22ffbSKyxXvd5rO9j4is\n8Y55TkTE295QRD71yn8qIg3ONUhjjDEV5+8dxavAyDL2jwLae6/xwAvgfukDjwEDgP7AYz6/+F/w\nyhYfV3z+h4E0VW0PpHmfjTHGBIhfiUJVFwAHyigyBpihzmKgvog0BS4FPlXVA6p6EPgUGOntq6eq\ni9TN+JsBXOVzrune++k+240xxgRARCWdJwXY4fM509tW1vbMUrYDJKtqFoCqZolI49IuKCLjcXck\nxMTE9GnevHmFKl5UVERYWOh11YRi3KEYM4Rm3KEYM5x93Js3b96nqo3KK1dZiUJK2aYV2O43VZ0K\nTAXo27evLlu27GwO/056ejpDhw6t0LHBLBTjDsWYITTjDsWY4ezjFpFv/SlXWSk3E/D9kz4V2FXO\n9tRStgPs8Zqm8L5mV1IdjTHGVEBlJYrZwI+80U8DgRyv+WgucImINPA6sS8B5nr7jojIQG+004+A\n93zOVTw66jaf7cYYYwLAr6YnEXkDGAokiUgmbiRTJICqvgjMAUYDGcAx4HZv3wER+R2w1DvVRFUt\n7hS/FzeaKgb4yHsBPAm8JSJ3ANuB6ysenjHGmHPlV6JQ1ZvK2a/AfWfYNw2YVsr2ZUC3UrbvBy72\np17GGGOqXugNCzDGGHNWLFEYY4wpkyUKY4wxZbJEYYwxpkyWKIwxxpTJEoUxxpgyWaIwxhhTJksU\nxhhjymSJwhhjTJksURhjjCmTJQpjjDFlskRhjDGmTJYojDHGlMkShTHGmDJZojDGGFMmSxTGGGPK\nZInCGGNMmSxRGGOMKZNfiUJERorIJhHJEJGHS9nfUkTSROQrEUkXkVSffU+JyFrvdaPP9oUissp7\n7RKRd73tQ0Ukx2ffo5URqDHGmIop95nZIhIOTAFGAJnAUhGZrarrfYo9A8xQ1ekiMhyYBNwqIpcB\nvYGeQDQwX0Q+UtXDqnqhzzXeBt7zOd9CVb38XIMzxhhz7vy5o+gPZKjqNlU9AcwExpQo0wVI897P\n89nfBZivqgWqehRYDYz0PVBE4oHhwLsVC8EYY0xV8idRpAA7fD5nett8rQau9d5fDcSLSKK3fZSI\nxIpIEjAMaF7i2KuBNFU97LNtkIisFpGPRKSrn7EYY4ypAuU2PQFSyjYt8flBYLKIjAMWADuBAlX9\nRET6AV8Ae4FFQEGJY28CXvL5vAJoqaq5IjIad6fR/rRKiYwHxgMkJyeTnp7uRyiny83NrfCxwSwU\n4w7FmCE04w7FmKEK41bVMl/AIGCuz+dHgEfKKB8HZJ5h3+vAaJ/PicB+oE4Z5/sGSCqrjn369NGK\nmjdvXoWPDWahGHcoxqwamnGHYsyqZx83sEzLyQGq6lfT01KgvYi0FpEoYCww27eAiCSJSPG5HgGm\nedvDvSYoRKQ70B34xOfQ64EPVDXP51xNRES89/1xzWP7/ainMcaYKlBu05OqFojIBGAuEA5MU9V1\nIjIRl41mA0OBSSKiuKan+7zDI4GF3u/9w8Atqurb9DQWeLLEJa8D7hWRAuA4MNbLfMYYYwLAnz4K\nVHUOMKfEtkd93s8CZpVyXB5u5NOZzju0lG2Tgcn+1MsYY0zVs5nZxhhjymSJwhhjTJksURhjjClT\naCeKwpMkHFoL1ldujDFnFNqJYvVMeq36DexZG+iaGGNMjRXaiaLDpSgCmz4KdE2MMabGCu1EEdeY\nw/U6wKY55Zc1xpgQFdqJAtif2B92rYTDuwJdFWOMqZFCPlHsS+rv3mz+OLAVMcaYGirkE8Wx2ObQ\noJX1UxhjzBmEfKJABDpeBtvmQ35uoGtjjDE1jiUKgI6joDAfts0LdE2MMabGsUQB0GIg1KlvzU/G\nGFMKSxQA4ZHQ/hLXoV1UGOjaGGNMjWKJolin0XBsP7wyCr56CwryA10jY4ypESxRFOs8Bi6dBLnZ\n8M5d8KcusPQlu8MwxoQ8SxTFwsJg0E/g/hVwyzvQuDN8+Et4eQRkrQ507YwxJmD8esJdSAkLg3YX\nQ9vhsOZfMPfXMHUoNDkPmvVyrzZD3dwLY4wJAZYozkQEut/gOrmXvAjbF8G6f8PyV93+xl1dv8Z5\n10OjjgGtqjHGVCW/mp5EZKSIbBKRDBF5uJT9LUUkTUS+EpF0EUn12feUiKz1Xjf6bH9VRL4WkVXe\nq6e3XUTkOe9aX4lI78oItMJi6sPQh+FH78GvvoX7lsIlT7jtC/8IU/rDq5fD2retA9wYUyuVe0ch\nIuHAFGAEkAksFZHZqrrep9gzwAxVnS4iw4FJwK0ichnQG+gJRAPzReQjVT3sHfeQqs4qcclRQHvv\nNQB4wfsaeCLQqIN7DZ7gOr5X/gOWvwKzfgxR8dD+B26md/N+EN8UIqIDXWtjjDkn/jQ99QcyVHUb\ngIjMBMYAvomiC/CA934e8K7P9vmqWgAUiMhqYCTwVhnXG4NLOgosFpH6ItJUVbP8DaraxDWGC38B\n5/8Mts6DDe/Bpo9dE1WxmIYQHe+SDEBcE+g40iWTpPanthtjTA0lWs5jQEXkOmCkqt7pfb4VGKCq\nE3zKvA4sUdVnReQa4G0gCegDPIa7G4kFvgSmqOofReRVYBCQD6QBD6tqvoh8ADypqv/1zp0G/EpV\nl5Wo13hgPEBycnKfmTNnVugfIDc3l7i4uAodWyotot7hLcQe20F0/gGiTuwnvDAPAFEl9lgm8blb\nAciPakB+dBL50Q05EdWQ/OhETkQ15HhMEw7X64iGVV0XUqXHHQRCMWYIzbhDMWY4+7iHDRu2XFX7\nllfOn99Epf3JWzK7PAhMFpFxwAJgJ1Cgqp+ISD/gC2AvsAgo8I55BNgNRAFTgV8BE/28Hqo61TuO\nvn376tChQ/0I5XTp6elU9NgzG1727pydsGkO0btWEn0kC47shgObIO/QqTLRCdB+hHs1aAXxTdzd\nSGSdSqlh1cRds4VizBCacYdizFB1cfuTKDKB5j6fU4HvPeVHVXcB1wCISBxwrarmePueAJ7w9r0O\nbPG2Fzcl5YvIK7hk49f1gl5CCvS/6/TtJ4+7pLFnHWz+yDVjrfXpwgmLhM6XQ98fQ6sLrdnKGFMt\n/EkUS4H2ItIad6cwFvihbwERSQIOqGoR7k5hmrc9HKivqvtFpDvQHfjE29dUVbNERICrgLXe6WYD\nE7y+kAFATo3sn6gKkTHQsLV7db7czQrft9k9fe/Ibti9Bla/4fpAkjpAl6vcEN2mPS1pGGOqTLmJ\nQlULRGQCMBcIB6ap6joRmQgsU9XZwFBgkogorunpPu/wSGChywUcBm7xOrYB/ikijXBNTauAe7zt\nc4DRQAZwDLj9nKMMVmHhboZ4486ntv3gMZcoVrwGC5+BBU9DfDO3Am5Kb2jaA+oknCof08A1WUVE\nVX/9jTG1gl+9pao6B/cL3Hfboz7vZwElh7miqnm4kU+lnbPUhnxvtNN9pe0zuLuOnj90r6P7YPNc\n2PIJZC6Fde+c+bjYJDdcN74J1GtKq/35ELfNJZmGbSCxnZuVbowxJdjM7GBWNwl63exeALl7YfdX\nUOBGWaEKxw/A4Sw4sguO7HFfs1bT8uhe+PbNU+eKiodmPaFRJ6jX1CWQ+CankkudBGveMiZEWaKo\nTeIauXWq/LDgszSG9OkER7Jg70bYtRJ2rnBLrOfnnH6AhPHdgLSoutCku0ssyV1dJ3tJYeHQuIub\nKxIWXvGYjDEBZ4kiRGlYuBt9lZACqX2h1y2ndp446jrPi4fuHtkNxw+e2p93CHatgi//7h4hW5bI\nutCkm0supYlpeOrOpV7TU3cwCc3dA6WMMQFnicKcLqouJLZ1r7IUnICcHa6J67R9x2H3Wnensmct\n5B85vYwWwYGvXUIqbi4rFlHHW7G3t1sypTiBJLaHOvUqHpsx5qxZojAVFxFVdjJpch70vKn886i6\nO5bcPS5pHM6C7PWuKWzlP+Dk0VNlwyKg1QVuCZTUPl6TWBnCIiEuGWIT/YvJGHMaSxQm8EQgtqF7\n+Q4FBjeX5Oheby5JFuxYAhvnwEcPnd01wiIYFBEPK2K9a4a7Pp34pm7NruJ+Fglz/SrNekFyNxtW\nbAyWKExNFxbu9WE0cZ87XQYjJsK+LbA/o/zjC/LdKr9HsjiQ8RVNm3rnKSyAo9lwYJt71kjxI28L\nT566gwmPgpbnQ8fR0HEU1G9e+jWMqeUsUZjglNTevc7Cpoh0mpa3Do4qHNoOu1ZA5jI3T+Wjh9yr\nQSvXZ9KsF7S+0GbEm5BhicIYXyLQoKV7db0aLn3C3b1s/hh2fOmSR/HExnop7k4jtd+pkVtRcaeS\nR0wDN0HSmCBnicKY8pS8e8nNhoz/wMYPYdXrsPSl0o+LqOOer95xFLS+yE1irKTVf42pTpYojDlb\ncY1PLaNSkA85madGaxX3b6jC3k2w6UN3N1IspuH3llIhoYVbn6tZL4hPDkw8xpTDEoUx5yIiuuw5\nJyMnuWXjd63wllDJOvXKXu8mMxY/biWhhXv+SKfRbhl5e4yuKUdBYRER4VW/RpslCmOqkoibmd6k\nW+n783Pd8vG7VsK3n7tl5Je97Ga0N+vp7jRSvA70Bq2t89x8p6hIGfbHdG7o05z7Lz67gR1nyxKF\nMYEUHQctB7nXoJ+4h1dtmw9b09yEQ99lUurUh6bdXYc5AN78k/imrtkqwqfjvL7XpGWz2GutVZmH\n2HHgOKkNq37AhCUKY2qSyBjoONK9wC2Tkr3e3XHsWumasYrX3VKFncvdfBAtKv18ie3pRkPIne0S\nSpPzoM0QG41VC8xdu5uIMGF4p6rv27JEYUxNFhHlNUH15IzP8CoscLPXC0+4z1oE+7d+l1zqZK6F\ndVvckvMAkbHQdri74yheETim/qnZ6DYyq8ZTVT5et5vB7ZJIiKn6xTMtURgT7MIj3AgqXw1bQ/sf\nALAsPZ2hQ4e6Zq3ti9wSKJs+go0fnH6usAi3PHxx30jyeVCvGdRt5K5jaoSNu4/w7f5j3DOknIU7\nK4l9540JFZEx7k6i7XAY/YdTy5YA5O4+9UySXStg/XuwYvqp/RLm5oE07QEpvaBJD0hIdcN8YxpY\nJ3s1+3jtbkRgRJfqGVLtV6IQkZHAs7hnZr+kqk+W2N8SmAY0Ag7gno2d6e17CrjMK/o7VX3T2/5P\noC9wEvgSuFtVT4rIUOA94GvvmHdUdWKFIzTGnE7k+3cICanu1fkK91kVDn4N2RtPPZfk4NcumWz6\n8Pvniox1TVYpvSGlD7QZ5hZcNFXm47W76deqIUlx1TOEutxEISLhwBRgBJAJLBWR2aq63qfYM8AM\nVZ0uIsOBScCtInIZ0BvoCUQD80XkI1U9DPwTKH5azuvAncAL3ueFqnr5uYdnjKkQEfcs9YZtTt+X\nlwN71p9KIIe2Q9YqWDEDlrwICDTv72aktxjsOtCjYqs9hNpq295cNu05wqOXd6m2a/pzR9EfyFDV\nbQAiMhMYA/gmii7AA977ecC7Ptvnq2oBUCAiq4GRwFuqOqf4YBH5Ekg9l0CMMdWkToIbzltSUaGb\nE7J5rrvr+M/jbruEu2exN2x9alZ6ySca1qlvzVd+mrtuDwAjuzWptmuKlvZ0Mt8CItcBI1X1Tu/z\nrcAAVZ3gU+Z1YImqPisi1wBvA0lAH+Ax3N1ILK6JaYqq/tHn2EhgCfAzVV3oNT29jbt72QU8qKrr\nSqnXeGA8QHJycp+ZM2dW6B8gNzeXuLi48gvWMqEYdyjGDIGLOyr/APFHthB/JIP4I1upk7eXqBMH\niCzIPa1sYVgUJ6IaciS+DfsT+7M/sS8FkfEVvnZt/l5PXHQcBR4bdPoQ57ONe9iwYctVtW955fy5\noygtzZfMLg8Ck0VkHLAA2AkUqOonItIP+ALYCywCCkoc+1dggaou9D6vAFqqaq6IjMbdnZw27VBV\npwJTAfr27atDy1s++gzSi0eEhJhQjDsUY4YaGPeJY67z/Mhu90Cq3D2EH8ki5vAuYr79gsYbv3B3\nIS0GueVMOo52dyNnocbFXEmWfnOAbR8v4uFRnRhayoinqorbn0SRCfg+sSUV95f+d1R1F3ANgIjE\nAdeqao637wngCW/f68CW4uNE5DFcB/jdPuc67PN+joj8VUSSVHXf2YVmjKmRomLP3P9RVARZK70h\nvHNg7q/dq/lAN1Kraffqr28NcaKgiEfeWUNK/Rh+NKhltV7bn9WklgLtRaS1iEQBY4HZvgVEJEnk\nu4cXP4IbAYWIhItIove+O9Ad+MT7fCdwKXCT6qlppSLSRMQ1VopIf6+O+yseojEmaISFuZFTF/8/\n+Mki+OkquOT37mmGU4fAx49A/pFA1zIg/jZ/KxnZufz+qm7ERlXvzIZyr6aqBSIyAZiLGx47TVXX\nichEYJmqzgaGApNERHFNT/d5h0cCC73f+4dxw2aLm55eBL4FFnn7i4fBXgfcKyIFwHFgrJbXkWKM\nqZ0atobB90OvWyBtIix+AVb+w62y23G0+1onIdC1rHLb9uby/LwMLjuvKcM6Na726/uVlrwRSnNK\nbHvU5/0sYFYpx+XhRj6Vds5Sr62qk4HJ/tTLGBMiYhrA5X92CWPZNNj0Max9G8IiodUFtfq55qrK\nb99dS3R4GI9dUX1DYn3ZzGxjTPBI6eNeRYWQudT1Y2ycc+q55k26Q8fRxB9JgsLzIbzq10GqalMX\nbOOLrfv5/VXdaFwvMOtwWaIwxgSfsHBoMdC9Rkx0zzXf+KFbw2r+U/RB4avfusTh+0yPxPauHyRI\nLNq6n6c+3sjo85pw84AWAauHJQpjTPBLag8X/Ny9cvey/sMX6JKQ55YcWfkafPk3V65OAvzgcehz\ne42f4LfncB73v7GSVkl1efq6HkgA62uJwhhTu8Q1Ijv5IroUzycoKoR9m13SWD0TPngAMtLgyufd\ng59qoBMFRdz3zxUcO1HAG3cNIC46sL+qg+cezBhjKiIsHBp3hp4/hFvfhUuecMuMvHA+bF8S6Nqd\npqhIefBfq1n27UGevLY77ZMrPkO9sliiMMaEjrAwGDwB7kpzD2h69TJY8Vqga/UdVWXiB+uZvXoX\nvxrZiSt7NAt0lQBLFMaYUNS0B9z1mRtaO3sCfPSwe1JggP01fSuvfvENd1zQmnuGlDJzPUAsURhj\nQlNMA7h5Fgz8CSx5AV4dDQe/DVh1Zn65nT/M3cTVvVL4zejOAe28LskShTEmdIVHwMhJcO3LkL0B\nXrwA1pw2d7jKfbx2N7/+9xqGdGjE09d1Jyys5iQJsERhjDFw3nVwz0L33Iy374BXLnMzvwtOVPml\nF2/bz09nrqR7an1euKU3keE179dyzauRMcYEQoNWcPtHcOn/Qc4OmPVj+HNXt8bUoe1Vcsll3xzg\nrunLaNEwllfG9av2xf78ZYnCGGOKhUfAoPvcqrU3z3LLhfz3z/BsD3j9RjcXoxKoKq8t+oaxUxfT\nMC6KGT/uT4O6UZVy7qpQM9OXMcYEUliYW5m2/Qg4tAOWvworpsO0UXDja257BeWdLOS3765l1vJM\nhnVsxF/G9iIhpmavSWV3FMYYU5b6zd3zMe79wi0V8sbYCnd45xw7yY9e/pJZyzP52cXtefm2fjU+\nSYAlCmOM8U9cYxj3ATQfAG/fCV88D2fxqJysnOPc8LdFrNxxkOdv6sUDIzrUuNFNZ2KJwhhj/FUn\nAW55GzpfDp/8Fl6/AXL3lnvYpt1HuPavX7Dz0HGm396fK2rIjGt/WaIwxpizERkDN7wGo5+BbfPh\nhcFuifMz3F18vDaLq//6OSeLlJnjBzK4XVI1V/jcWaIwxpizJQL974Lx6VA3CWb+EF4eAVvnfZcw\nioqUZ+Zu4p5/rKBDcjzvT7iAbinB+dhWvxKFiIwUkU0ikiEiD5eyv6WIpInIVyKSLiKpPvueEpG1\n3utGn+2tRWSJiGwRkTdFJMrbHu19zvD2tzr3MI0xpgokd4G7F8Dlf4HDWfDaVfD6jeQczuHOGcuY\nPC+DG/s25827B9IkITBPp6sM5SYKEQkHpgCjcM+/vklESj649Rlghqp2ByYCk7xjLwN6Az2BAcBD\nIlLPO+Yp4M+q2h44CNzhbb8DOKiq7YA/e+WMMaZmCo+EvrfDT1fAiN+hWz5h87NXsnjzTn53VTee\nvPY8oiPCA13Lc+LPHUV/IENVt6nqCWAmMKZEmS5Amvd+ns/+LsB8VS1Q1aPAamCkuNWuhgPFY8ym\nA1d578d4n/H2Xyw1aXUsY4wpTUQ0aQ1v5LdFd9OvcBVftHmVW/s2qVGL+1WUPxPuUoAdPp8zcXcH\nvlYD1wLPAlcD8SKS6G1/TET+BMQCw4D1QCJwSFULfM6ZUvJ6qlogIjle+X2+FxSR8cB4gOTkZNLT\n0/0I5XS5ubkVPjaYhWLcoRgzhGbcgYh5xZ4CpqzKp3n8UFamCL2+foF9L1zO+i4PURRePbOuqypu\nfxJFaemwZPf+g8BkERkHLAB2AgWq+omI9AO+APYCi4CCcs7pz/VQ1anAVIC+ffvq0OLHHp6l9PR0\nKnpsMAvFuEMxZgjNuKs75k/X7+GFT5dzXmp9ZtzRn3p1RsGXbUma8yAXZT4PY1+H6Kp/Ul1Vxe1P\n01Mm0Nzncyqwy7eAqu5S1WtUtRfwG29bjvf1CVXtqaojcElgC+7uoL6IRJRyzu+u5+1PAA5UIDZj\njKlyn67fw0/+uZwuzRK8JOHNtO5/F1z9N/jmc5h+BRzdV/aJajB/EsVSoL03SikKGAvM9i0gIkki\nUnyuR4Bp3vZwrwkKEekOdAc+UVXF9WVc5x1zG/Ce93629xlv/2deeWOMqVHeWraDe/6xnC5N6zHj\nxz5JoliPse5uInsDTBvp1o0KQuUmCq8fYQIwF9gAvKWq60Rkoohc6RUbCmwSkc1AMvCEtz0SWCgi\n63HNRLf49Ev8CviFiGTg+iBe9ra/DCR6238BnDYc1xhjAklVmTIvg/+Z9RWD2ybyz7sGnnnNpo4j\n4dZ/Q+4emHYp7N1cvZWtBH6tHquqc4A5JbY96vN+FqdGMPmWycONfCrtnNtwI6pKO+Z6f+pljDHV\nLe9kIRM/WM/rS7Yzpmcz/nBdD6Iiyvmbu+VgGPch/OMalyxueRtSeldPhSuBzcw2xhg/rd2Zw2XP\nLeT1Jdu5Z0hb/nxDz/KTRLGm3eHHcyE6DqZfCUd2V21lK5ElCmOMKcfunDz+MHcjV035nNz8Al67\noz8Pj+p09qu/JraFW9+Fk8fc6rNBwh5cZIwxZ/BFxj5e+eIbPtuYTWGRcmWPZkwc05X6secwLyKx\nLZx3PSybBhc84NaKquEsURhjTAmFRcqfPt3ElHlbSYqLZvxFbRjbrzktE+tWzgUu/CV89SYsmgw/\neLxyzlmFLFEYY4yPnOMn+fnMlczbtJex/Zrzv2O6Vv5aTY06QNer4cu/w+CfQmzDyj1/JbNEYYwJ\neScKivjy6wN8tjGbj9ZmsfdIPr+/qhs3D2hRdWs1XfQgrHsHlrwIw35dNdeoJJYojDEhKftIHvM2\nZvPZxmz+u2UfR08UEhURxuC2iTx/Uy/6tqriv/KTu0Kny2HxizDoPvf0vBrKEoUxJqQUFBbx1/St\nPJe2hYIipWlCHcb0SmF4x8YMbpdIbFQ1/loc8j+w8QNY9FcY9kj1XfcsWaIwxoSMr/cd5YE3V7Fq\nxyHG9GzGPUPa0qlJfOCWAm/aAzpfAYv/CgPurrF9FZYojDG1Vt7JQuau283qHTms3ZXD6h2HiI4I\n4/mbenFFj2aBrp4z9New4QM3r+IHjwW6NqWyRGGMqZXSNuzhf99fz/YDx6gTGUbnpvW4qX8L7h7S\nhqYJMYGu3inJXaDbNa5Te+BPIK5RoGt0GksUxphaZeeh4/xleR6r9i6jXeM4Zvy4P4PbJhIRXoMX\nohj6CKz7N3z+F7j0ifLLVzNLFMaYWuO9VTv57btrOXGykEdGdeL281v7vxZTICW1h+5jYelLMGgC\n1Gsa6Bp9TxD8CxpjTNkOHTvBT99Yyc9mrqJ94zh+d34Mdw9pGxxJotiQ/4GiAlj4x0DX5DRB9K9o\njDHfV1SkvLl0O8P/OJ8P12TxixEdeOvuQTSODcJfbQ1bQ69bYPmrcGh7oGvzPUH4r2mMCXW5+QV8\nvHY317zwBb96ew1tG9Xl/QkX8NOL29fsvojyXPQQiMCCPwS6Jt9jfRTGmKCQm1/AB6t38eGaLJZs\nO8CJwiKS4qL50w09uLpXSuDmQlSmhFToc7vrqzj/526l2RrAEoUxpsbal5vPmp05zF27m/dX7+Lo\niULaJNVl3PmtGNaxMX1bNSAymO8gSnPhL2DFdJj/NFzzt0DXBvAzUYjISOBZIBx4SVWfLLG/JTAN\naAQcwD0bO9Pb9zRwGa6Z61PgZ0AcsNDnFKnAP1T15yIyDvgDsNPbN1lVX6pQdMaYoHLg6AnSN2WT\ntjGb5d8cZPfhPABiIsO5vHtTxvZvQe8W9WvH3cOZxDeB/nfBoimuKSqpXaBrVH6iEJFwYAowAsgE\nlorIbFVd71PsGWCGqk4XkeHAJOBWERkMnA9098r9FxiiqulAT59rLAfe8Tnfm6o6oeJhGWOCSd7J\nQn75r9XMWZOFKjSKj2Zw20TOS0mga7MEzktNIC46hBpABtzrZmpv/hiSAv+r0J9/+f5AhqpuAxCR\nmcAYwDdRdAEe8N7PA9713itQB4gCBIgE9vieXETaA435/h2GMSZE5J0sZPxry1m4ZS/jL2zDZd2b\n0q1Zwtk/ZrQ2SUiB+i1hx2IM7La9AAAY3UlEQVQgOBJFCrDD53MmMKBEmdXAtbjmqauBeBFJVNVF\nIjIPyMIlismquqHEsTfh7iDUZ9u1InIRsBl4QFV3lDgGERkPjAdITk4mPT3dj1BOl5ubW+Fjg1ko\nxh2KMUPNjvtEofL8ynzW7ivk9m5RDIrdw4GMPSzIOLfz1uSY/dUpujUNMxbwxbx5biSUH6oqbn8S\nRWk11BKfHwQme/0LC3D9CwUi0g7ojOuDAPhURC5S1QU+x44FbvX5/D7whqrmi8g9wHRg+GkVUJ0K\nTAXo27evDh061I9QTpeenk5Fjw1moRh3KMYMNSPuzIPHWLn9EGt25rB+12GOnigA4ODRE3yzv5Cn\nr+3ODf2aV9r1akLM5yxuG3yQztDuLfwe/VRVcfuTKDIB3+9gKrDLt4Cq7gKuARCROOBaVc3x/upf\nrKq53r6PgIG4ZIKI9AAiVHW5z7n2+5z678BTZxuUMSbw8gsKmbtuD68v+ZbF2w4AEBUeRscm8dSP\njQQgLjqCB0Z0YEzPlEBWtWZqMch93b444MNk/UkUS4H2ItIad6cwFvihbwERSQIOqGoR8AhuBBTA\nduAuEZmEuzMZAvzF59CbgDdKnKupqmZ5H68ESjZVGWNquI/WZPH/3lvLvtwTNG8Yw0OXdmRIh0Z0\nSI4PrmU1Aimpo3vq3Y7F0OvmgFal3EShqgUiMgGYixseO01V14nIRGCZqs4GhgKTRERxdwv3eYfP\nwjUbrcE1V32squ/7nP4GYHSJS/5URK4ECnBDbcdVMDZjTDU7nHeSx2ev450VO+memsCfb+zJ+W2T\nQrtjuqLCwqD5QHdHEWB+jTdT1TnAnBLbHvV5PwuXFEoeVwjcXcZ525Sy7RHcXYkxJggUFimrdhwi\nbcMe/r1yJ9lH8vnpxe25f3i72jcZrrq1GABb5sLR/VA3MWDVCKGBycaYypSRfYQ3vtzBuyt3sv/o\nCcLDhP6tGjLl5t70btEg0NWrHYr7KXYsgU4lG1+qjyUKY4zfVJW0DdlMXbCNL785QGS4MKJLMqO6\nNeWiDo1IiIkMdBVrl2a9ICzS9VNYojDG1DSFRcquQ8cpnuG0Yfdhnv9sC2t3Hia1QQwPj+rEdX1S\nSYqLDmxFa7PIGJcsAtxPYYnCGPM9BYVFfPBVFs9/toWte49+b1/LxFj+cF13ruqVYv0P1aXFAFjy\nNziZB5F1AlIFSxTGGACO5J3kvVW7ePm/X/P1vqN0ahLP78Z0JTbK/ZqoFxPJsI6Ngvt5D8GoxSC3\n7tOuldByUECqYInCmBB2srCIpV8f4N8rd/LBV1kcP1lIt5R6vHhLHy7pkmzDWmuC5t6KSZlfWqIw\nxlS9/IJCNu/OZc3OHL7Yuo/5m/dyJK+A2KhwrurVjLH9WtA9NaF2L+MdbOomuQUCdy4vv2wVsURh\nTC13olB5Z0UmM5fuYMW3Bykocr3TSXHRjOrWhIs7J3Nh+6TvmphMDZTSBzKXBuzy9pNhTC11OO8k\nU+Zl8NrnxzhWsJpWibHcdVEbujVLoFtKPZo3iLWmpWCR2hfWvQNH9kB8crVf3hKFMbWMqvLvlTv5\nvzkb2X80n37J4TxwRT8GtmloTUrBKqWP+7pzeUDmU1iiMKYWUFUysnP5z4Zs5qzJYs3OHHo0r8+0\ncX05kLGKQW0Dt/yDqQRNuoOEW6Iwxpy9vUfymbU8k7eW7eDrfW7OQ9dm9Xjq2vO4vk9zwsKE9HN8\nCJCpAaJiIblLwDq0LVEYE2QOHj3BvE3ZzF23m7QN2RQUKQNaN+TOC1szvFNjmibEBLqKpiqk9IW1\n70BRkVtZthpZojAmCKgq/9mQzUsLt7H0mwMUKTSKj+b281sxtn8L2jaKC3QVTVVL6QPLX4EDWyGp\nfbVe2hKFMTXYiYIi0jbs4fnPMlifdZgWDWOZMKwdF3dO5ryUBBu1FEp8O7QtURgT2o6fKOTDNVmk\nbdjDwi37yM0voFViLM9c34OrejazJTRCVaOOEFnXJYoeY6v10pYojKkhjp0o4B+Lv2Xqgm3syz1B\ncr1orujRjIs7NWaorbFkwsLdSrKZy6r90pYojAmgvJOFLNq2n3kbs/ngqywOHD3Bhe2TuG9YOwa0\ntnkPpoTUPrDor1CQDxHVt7y7X4lCREYCz+Kemf2Sqj5ZYn9LYBrQCPec61tUNdPb9zRwGRAGfAr8\nTFVVRNKBpsBx7zSXqGq2iEQDM4A+wH7gRlX95lyCNKYm2XM4j882ZpO2IZvPM/Zx/GQhMZHhDOnQ\niLsuakOflvZ0OHMGKX2g6CTsXuuSRjUpN1GISDgwBRgBZAJLRWS2qq73KfYMMENVp4vIcGAScKuI\nDAbOB7p75f4LDAHSvc83q2rJ+6g7gIOq2k5ExgJPATdWKDpjapDMg8f4/Qcb+HjdbgBS6sdwXZ9U\nLu7cmIFtEqkTGR7gGpoar7hDe9eKmpUogP5AhqpuAxCRmcAYwDdRdAEe8N7PA9713itQB4gCBIgE\n9pRzvTHA4977WcBkERHV4udsGRNcjuYXMO2/XzMlPQNBuH94Oy7v3owOyXHWtGTOTr0UiIiBg99U\n62X9SRQpwA6fz5nAgBJlVgPX4pqnrgbiRSRRVReJyDwgC5coJqvqBp/jXhGRQuBt4PdeMvjueqpa\nICI5QCKwz/eCIjIeGA+QnJxMenq6H6GcLjc3t8LHBrNQjLu6Yi5S5cvdhazKLuDbw0XsPqoo0Dc5\nnJs6RZEYlUXWxiyyNlZ5VQD7Xtc2/SMbkJuxgvXR6aftq6q4/UkUpf3JU/Kv+wdxf/mPAxYAO4EC\nEWkHdAZSvXKfishFqroA1+y0U0TicYniVlzfhD/XQ1WnAlMB+vbtq0OHDvUjlNOlp6dT0WODWSjG\nXdUxFxYp76/exXOfbWHb3nwax0fTvUV9bkypx/ntkujXqmGVXbss9r2uZbZ3IDY/l8alxFdVcfuT\nKDKB5j6fU4FdvgVUdRdwDYCIxAHXqmqO91f/YlXN9fZ9BAwEFqjqTu/YIyLyOq6Ja4bP9TJFJAJI\nwHWQG1Mj7Tmcx7+W7WDm0h1kHjxOpybxvHBzby7t2sQmxJnKl5AKW/5TrZf0J1EsBdqLSGvcncJY\n4Ie+BUQkCTigqkXAI7gRUADbgbtEZBLuTmEI8BcvAdRX1X0iEglcDhRHPhu4DVgEXAd8Zv0TpiZa\ntyuHKfMymLtuD4VFyuC2ifz2ss5c0sUShKlCCc0hd3e1DpEtN1F4/QQTgLm44bHTVHWdiEwElqnq\nbGAoMElEFNf0dJ93+CxgOLAG13z0saq+LyJ1gblekgjHJYm/e8e8DLwmIhm4O4nqnYJoTDnWZObw\nbNoW/rNhD/F1Irjzwtbc1K8FrZLqBrpqJhTUS3FfD++Chq2r5ZJ+zaNQ1TnAnBLbHvV5PwuXFEoe\nVwjcXcr2o7h5EqVdKw+43p96GVOdVu04xLP/2cy8TXtJiInkFyM6cNvgViTERAa6aiaUJHhdvod3\n1qxEYUyoOllYRNqGbP655FsWbtlH/dhIHrzEJYj4OpYgTAAkeF3GOZnVdklLFMaU4tv9R5m5dAez\nlmey90g+yfWieejSjtw2uBVx0fbfxgRQgtf0lLOj7HKVyH7ijfEUFBYxd90eXv/yWz7P2E+YwPBO\njRnbr4UtymdqjsgYiE2EnJ3VdklLFCbkFRQW8d6qXUyel8HX+46SUj+GX47owPV9m9MkoU6gq2fM\n6RJSrenJmOpwsrCIf6/cyZR5GXy7/xhdmtbjxVt62/BWU/MlNIf9W6vtcpYoTMjJO1nIe6t2Mnle\nBjsOHKdbSj2m3tqHEV2Sbe0lExwSUmHb/Gq7nCUKExJUlcwjRTw+ex3/XrmTnOMn6Z6awONXdGV4\np8aWIExwqZcCJ45AXg7USajyy1miMLVWRnYu763ayZqdOazdeZh9uflEhW/nkq7J3NS/BYPbJlqC\nMMGpeC5FTqYlCmMqIiP7CM+lZfD+V7sIE6F94ziGdGhE3bxsfn7tEBrWjQp0FY05N75zKZK7Vvnl\nLFGYoKeqbNt3lM82ZJO2cQ9Lvj5ATGQ44y9qw10XtiEpzq2Hk56ebknC1A7f3VFUz1wKSxQmqG3a\nfYTHZ69j0bb9AHRqEs/9w9tz26CWJMZV3zOFjalWcY0hLKLa5lJYojBB6XDeSf7y6RamL/qG+DoR\n/GZ0Z0ad14TUBrGBrpoxVS8sHOo1q7a5FJYoTNDZceAYt73yJV/vO8pN/Vvw0CUdaWBNSibUJDS3\nRGFMadbuzGHcK0s5WVjEG3cNZGCbxEBXyZjASEiF7Yuq5VKWKEyNV1BYRMbeXJZ+c5An52ygfmwU\nM8cPoF3j+EBXzZjAqZfinklRVOiaoqqQJQpTY23bm8v/zdnIwi17yS8oAqBrs3pMG9eP5Hq2BpMJ\ncQmpUFQAuXtcf0UVskRhapzc/AKe/2wL0/77NXUiwrl5QEt6NE+ga7ME2iTVtXWYjIHvz6WwRGFC\nyecZ+3jwX6vJysnjuj6p/GpkJxrF2zBXY07jOzu7ef8qvZRfC+yLyEgR2SQiGSLycCn7W4pImoh8\nJSLpIpLqs+9pEVknIhtE5DlxYkXkQxHZ6O170qf8OBHZKyKrvNedlROqqcnyThby+Ox13PzSEmKi\nwnn73sE8c30PSxLGnMl3DzCq+pFP5d5RiEg4MAUYAWQCS0Vktqqu9yn2DDBDVaeLyHBgEnCriAwG\nzge6e+X+CwwBvgSeUdV5IhIFpInIKFX9yCv3pqpOqIwATc23dW8u97y2nC3ZuYwb3IpfjexETFTV\nds4ZE/TqJEBMA8g/XOWX8qfpqT+QoarbAERkJjAG8E0UXYAHvPfzgHe99wrUAaIAASKBPap6zCuH\nqp4QkRVAKibkfLJuN794azXREWHM+HF/LurQKNBVMiZ4PLS1ykc8gX9NTymA74Iimd42X6uBa733\nVwPxIpKoqotwCSHLe81V1Q2+B4pIfeAKIM1n87VeM9YsEWnudzQmaBQWKX/8ZBPjX1tOm0Z1ef/+\nCyxJGHO2qiFJAIiqll1A5HrgUlW90/t8K9BfVe/3KdMMmAy0BhbgkkZXoBHwLHCjV/RT4FequsA7\nLgJ4H5dA/uJtSwRyVTVfRO4BblDV4aXUazwwHiA5ObnPzJkzK/QPkJubS1xcXIWODWaBjPvoSeXF\n1fms2VfIhSkR3Noliqjwqh/JZN/r0BGKMcPZxz1s2LDlqtq33IKqWuYLGIT7RV78+RHgkTLKxwGZ\n3vuHgP/ns+9R4H98Pk8DnivjXOFATnl17NOnj1bUvHnzKnxsMAtU3Ot35eiFT32m7X79of5j8Tda\nVFRUbde273XoCMWYVc8+bmCZlvP7VVX9anpaCrQXkdZex/NYYLZvARFJEpHicz3iJQCA7cAQEYkQ\nkUhcR/YG75jfAwnAz0ucq6nPxyuLy5vglneykOfStnDVlM/JLyjkzbsHcfOAlvbgIGOCQLmd2apa\nICITgLm4v/Cnqeo6EZmIy0azgaHAJBFRXNPTfd7hs4DhwBpcx/bHqvq+N3z2N8BGYIX3y2Kyqr4E\n/FRErgQKgAPAuMoK1lQ/VeU/G7L53Qfr2X7gGJd1b8pjV3ShcbzNrDYmWPg14U5V5wBzSmx71Of9\nLFxSKHlcIXB3KdszcaOgSrvWI7i7EhPE9h7JZ9byTN5atoOv9x2lXeM4/nnnAM5vlxToqhljzpLN\nzDaVJiM7l8827iFtQzbLvj1IYZHSv1VDJgxrxxU9mhEV4df8TmNMDWOJwpyzw3kn+d/Z63l7hZsh\n2rlpPe4d0pare6fQtlHojTwxpraxRGHOyaKt+3nwX6vZfTiP+4a15eYBLWlWPybQ1TLGVCJLFKZC\nCouUZ/+zmefnZdCyYSz/umcQvVs0CHS1jDFVwBKFOWs5x07yszdXkr5pL9f1SWXimK7ERtmPkjG1\nlf3vNmdl7c4cfvLPFWTlHOeJq7vxw/4tbC6EMbWcJQrjl4LCIl5I38qzaVtIiovmzbutqcmYUGGJ\nwpQrIzuXh2atZuX2Q1zRoxm/G9OV+rFRga6WMaaaWKIwZ5SbX8Bzae6RpLFR4Tx3Uy+u7FG1j1w0\nxtQ8lihMqdI27OGRd9aQfSSf6/uk8j/2SFJjQpYlCnOa1xZ9w2Oz19GxST3+dmsfellfhDEhzRKF\n+U5RkfL03E28OH8rP+jcmOdv6m2PJDXGWKIwzqodh/jTp5tZsHkvPxzQgolXdiUi3NZmMsZYoghJ\n+QWF7D5axOJt+9lzOI9ZyzNZuGUf9WMjeeyKLowb3MrmRhhjvmOJIoQUFSmzVmTy9Mcb2Zd7AhYu\nBiCxbhQPj+rELQNbEhdtPxLGmO+z3wohYu3OHH777lpW7ThEn5YNuLo1DOnfi8b1omnRMJY6kdYX\nYYwpnSWKEJB58Bg3/G0RsVER/OmGHlzdK4X58+dzQXt7iJAxpnyWKGo5VeXR99YB8O59g0ltEBvg\nGhljgo0Na6nlPvgqi882ZvPLSzpakjDGVIhfiUJERorIJhHJEJGHS9nfUkTSROQrEUkXkVSffU+L\nyDoR2SAiz4k3nEZE+ojIGu+cvtsbisinIrLF+2qzvSoo59hJ/vf9dXRPTWDc4FaBro4xJkiVmyhE\nJByYAowCugA3iUiXEsWeAWaoandgIjDJO3YwcD7QHegG9AOGeMe8AIwH2nuvkd72h4E0VW0PpHmf\nzVlSVZ6Ys56Dx04y6ZrzCA+z4a7GmIrx546iP5ChqttU9QQwExhTokwX3C91gHk++xWoA0QB0UAk\nsEdEmgL1VHWRqiowA7jKO2YMMN17P91nu/FTYZEy8YP1vLUsk/EXtaFrs4RAV8kYE8T86cxOAXb4\nfM4EBpQosxq4FngWuBqIF5FEVV0kIvOALECAyaq6QUT6eufxPWeK9z5ZVbMAVDVLRBqXVikRGY+7\nIwHIFZFNfsRSmiRgXwWPrfEefuqMt2S1Ou4zCMWYITTjDsWY4ezjbulPIX8SRWltFlri84PAZBEZ\nBywAdgIFItIO6AwU91l8KiIXAcf9OGeZVHUqMPVsjimNiCxT1b7nep5gE4pxh2LMEJpxh2LMUHVx\n+9P0lAk09/mcCuzyLaCqu1T1GlXtBfzG25aDu7tYrKq5qpoLfAQM9M6ZeoZzFjdN4X3NPuuojDHG\nVBp/EsVSoL2ItBaRKGAsMNu3gIgkiUjxuR4BpnnvtwNDRCRCRCJxHdkbvKalIyIy0Bvt9CPgPe+Y\n2cBt3vvbfLYbY4wJgHIThaoWABOAucAG4C1VXSciE0XkSq/YUGCTiGwGkoEnvO2zgK3AGlw/xmpV\nfd/bdy/wEpDhlfnI2/4kMEJEtgAjvM9V6Zybr4JUKMYdijFDaMYdijFDFcUtbtCRMcYYUzqbmW2M\nMaZMliiMMcaUKWQShR/LkESLyJve/iUi0qr6a1n5/Ij7FyKy3lt+JU1E/BpXXZOVF7NPuetERL15\nPUHPn7hF5Abv+71ORF6v7jpWNj9+vluIyDwRWen9jI8ORD0rk4hME5FsEVl7hv3iLYuU4cXc+5wv\nqqq1/gWE4zrM2+Bmia8GupQo8xPgRe/9WODNQNe7muIeBsR67+8N9rj9idkrF4+b87MY6BvoelfT\n97o9sBJo4H1uHOh6V0PMU4F7vfddgG8CXe9KiPsioDew9gz7R+MGBwluOsKSc71mqNxR+LMMie/S\nIbOAi4sXKgxi5catqvNU9Zj3cTHfn98SjPz5XgP8DngayKvOylUhf+K+C5iiqgcBVDXY5yj5E7MC\n9bz3CZSYAxaMVHUBcKCMImNwa++pqi4G6hfPTauoUEkUpS1DknKmMuqGBOcAidVSu6rjT9y+7uDU\nMOVgVW7MItILaK6qH1RnxaqYP9/rDkAHEflcRBaLyEiCmz8xPw7cIiKZwBzg/uqpWkCd7f/7coXK\ng4v8WYbEnzLBxu+YROQWoC+nVvcNVmXG7E0M/TMwrroqVE38+V5H4JqfhuLuHBeKSDdVPVTFdasq\n/sR8E/Cqqv5RRAYBr3kxF1V99QKm0n+XhcodRbnLkPiWEZEI3G1qWbd3wcCfuBGRH+CWXrlSVfOr\nqW5VpbyY43FL3qeLyDe4NtzZtaBD29+f8fdU9aSqfg1swiWOYOVPzHcAbwGo6iLcata1/RnAfv2/\nPxuhkijKXYaE7y8dch3wmXo9Q0HMn+VXegF/wyWJYG+zhnJiVtUcVU1S1Vaq2grXL3Olqi4LTHUr\njT8/4+/iBi8gIkm4pqht1VrLyuVPzNuBiwFEpDMuUeyt1lpWv9nAj7zRTwOBHPVW5K6okGh6UtUC\nESlehiQcmKbeMiTAMlWdDbyMuy3NwN1JjA1cjSuHn3H/AYgD/uX13W9X1SvPeNIazs+Yax0/454L\nXCIi64FC4CFV3R+4Wp8bP2P+JfB3EXkA1/wyLtj/ABSRN3DNh0le38tjuGf9oKov4vpiRuOWRzoG\n3H7O1wzyfzNjjDFVLFSanowxxlSQJQpjjDFlskRhjDGmTJYojDHGlMkShTHGmDJZojDGGFMmSxTG\nGGPK9P8BndwThLtZia4AAAAASUVORK5CYII=\n", 323 | "text/plain": [ 324 | "
" 325 | ] 326 | }, 327 | "metadata": {}, 328 | "output_type": "display_data" 329 | } 330 | ], 331 | "source": [ 332 | "c_th=np.arange(0,1,0.01)\n", 333 | "pur = c_th*0\n", 334 | "com = c_th*0\n", 335 | "isqso_truth = (Y_val.argmax(axis=1)==2) | (Y_val.argmax(axis=1)==3)\n", 336 | "dv_max = 6000./300000.\n", 337 | "is_bad = Y_val.argmax(axis=1)==4\n", 338 | "zgood = (z_val>0) & (abs(zbest-z_val) < dv_max*(1+z_val))\n", 339 | "ndetect = 2\n", 340 | "for i,cth in enumerate(c_th):\n", 341 | " isqso_qn = (c_line>cth).sum(axis=0)>=ndetect\n", 342 | " ntrue_positives = (isqso_qn & zgood & ~is_bad).sum()\n", 343 | " pur[i] = ntrue_positives/(isqso_qn & (~is_bad)).sum()\n", 344 | " com[i] = (isqso_qn & zgood & isqso_truth).sum()/isqso_truth.sum()\n", 345 | " \n", 346 | "plt.plot(c_th, pur)\n", 347 | "plt.plot(c_th, com)\n", 348 | "plt.ylim(0.98,1.0)\n", 349 | "plt.grid()" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "#### Quality Assesment\n", 357 | "##### Purity and completenss vs. redshift\n", 358 | "\n", 359 | "The following plot show the purity and completeness vs. redshift. \n", 360 | "\n", 361 | "For a given redshift interval, defined using the network predictions, the purity is the fraction of spectra in the sample that have a predicted redshift better than 6000 km/s from the true redshift ( classification).\n", 362 | "\n", 363 | "For a given redshift interval, defined using the true redshift, the completeness is the fraction of quasar spectra in the sample that the network detects with a redshift better than 6000 km/s from the true redshift." 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": 35, 369 | "metadata": {}, 370 | "outputs": [ 371 | { 372 | "name": "stderr", 373 | "output_type": "stream", 374 | "text": [ 375 | "/global/homes/n/nbusca/.conda/envs/qnet/lib/python3.6/site-packages/ipykernel_launcher.py:14: RuntimeWarning: invalid value encountered in long_scalars\n", 376 | " \n", 377 | "/global/homes/n/nbusca/.conda/envs/qnet/lib/python3.6/site-packages/ipykernel_launcher.py:18: RuntimeWarning: invalid value encountered in long_scalars\n" 378 | ] 379 | }, 380 | { 381 | "data": { 382 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAE7tJREFUeJzt3X+QXXV5x/H3Q0gCJMRQwiyRAEFL\nHReTxnYbVKa62tYSUFOFqUBbhek0zlRm7DhpB2ZanKajdCrtOB0ca6rBoVbBRq1YY5FJuaUKKokC\nMYnBGBBCAqh0AytI3OXpH3vpXNcNe7N799y95/t+zexwft7v87Dw2e+ee/bcyEwkSWU4ptsFSJKq\nY+hLUkEMfUkqiKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCnJstwsYb8mSJbl8+fIpn/+Tn/yE\nBQsWdK6gWcTeeled+7O32WH79u0/ysxTJjtu1oX+8uXL2bZt25TPbzQaDA4Odq6gWcTeeled+7O3\n2SEiftDOcV7ekaSCGPqSVJBJQz8iNkXE4xHxnSPsj4j4x4jYGxH3RcSvtex7Z0R8r/n1zk4WLkk6\neu3M9D8BnP8C+9cAZze/1gEfAYiIXwLeB5wLrAbeFxEnTadYSdL0TBr6mXkH8MQLHLIWuDHHfB1Y\nHBFLgd8FbsvMJzLzf4HbeOEfHpKkGdaJa/qnAQ+3rO9vbjvSdklSl3Tils2YYFu+wPZffIGIdYxd\nGqKvr49GozHlYoaHh6d1/mxmb72rzv3ZW2/pROjvB05vWV8GHGhuHxy3vTHRC2TmRmAjwMDAQE7n\nvtheuq/2aNlbh2y7AXZsrmaspj3zV/KyN11b6ZhV8b/L3tKJyzu3AO9o3sXzKuBQZh4EbgXeGBEn\nNd/AfWNzm9RdOzbDozuqG+/RHfQ9dkd140kvYNKZfkR8mrEZ+5KI2M/YHTlzATLzn4AtwAXAXuBp\n4Irmvici4m+Au5svtSEzX+gNYZVq2w2s+vbH4IHF1Yz36A44dQVc8aVqxrvhQhgaqmYsaRKThn5m\nXjrJ/gTefYR9m4BNUytNxdixmYXDD8DiV1Yz3qkrYMXF1YwlzTKz7tk7KtPwwrNYXNHM+1PfeIgv\nbH8Ett9VyXjX/PgQC455jop+j5FekI9hUHG+cM8j7Dr4ZGXjPX14lEOHJ7xxTaqcM3113WNP/ZRH\nnxrlXR+tZua96+CT9C9dxM3venUl4+38wBxGRkYqGUuajKGvrvvR8LM8O1rdeP1LF7F2VbV/J/jS\n534w9oZuFVZcDANXVDOWeo6hr1lh/hwqm3lX7WvHv56fPDvCgoOHZnys5T/bx/BTP6XP0NcRGPrS\nDFv4mj/hrxqvYfHimX8rd/3B93LC8LP0zfhI6lWGvjTDLjv3DF78zD4GB2f+N5mdH5gz42Oot3n3\njiQVxNCXpIIY+pJUEK/pSzWz/Gf7qrs9FFg6fyU//0BdzWaGvlQjXzv+9QCcU9WAj+6g7zgfJtdL\nDH2pRraecAFbT7iAm6+o6G8efIJozzH0pZrZdfBJ3l7RIy18mFzvMfSlGqn68RJPHx5l5BgfJtdL\nDH2pRi479wwuO/eMysbzYXK9x9Cfqi58zmqVd0l86hsP8YV7HqlkrPWHR5nnzcNSJQz9KXrszk+y\n8H938+Dcl1Qy3vKf7WP+/Oo+bfL5Z873L10042OdMG8OC46p8DGbUsEM/Sn60fCzPJRnct3JH6xk\nvPUH38u8ij+Io7Jnzt/wIoa8A0SqhKE/DSfMm1PbD+L4rae3cN4zt8MNL5r5wR7dAcedPvPjaEY8\nO0pldwvB2JvVVb5vUTdeSdWEznvm9rG/7KzCqSt4rO+11YyljlqycD7zK3yw566DT1b2XlNdOdPX\nET049yWcU9GHlR9sNHhZJSOpk/pOPI75oz+t7DfeKn+jqCtn+pJUEENfkgpi6EtSQQx9SSqIoS9J\nBfHunR5S5f3Q6w+PcsI8P2Rbqhtn+j2i6vuhT5g3hyUL51c3oKRKONPvEVXfD13JX+JKqpyh30MW\nDj9Q3WefProDTl1RzViSKuPlnV6x4mKGF55V3XinroAVF1c3nqRKONPvFQNXcM/wWQwODna7Ekk9\nzJm+JBXE0JekgrQV+hFxfkTsiYi9EXHVBPvPjIitEXFfRDQiYlnLvr+LiJ0RsTsi/jEiopMNSJLa\nN2noR8Qc4MPAGqAfuDQi+scddh1wY2auBDYA1zbPfQ1wHrASeAXwG8DrOla9JOmotDPTXw3szcx9\nmXkYuAlYO+6YfmBrc/n2lv0JHAfMA+YDc4HHplu0JGlq2gn904CHW9b3N7e1uhe4qLn8VuDEiDg5\nM+9i7IfAwebXrZm5e3olS5Kmqp1bNie6Bj/+E7rXA9dHxOXAHcAjwEhE/DLwcuD5a/y3RcRrM/OO\nnxsgYh2wDqCvr49Go9F2A+MNDw9P6/x2ndT8vNoqxnpeVb11Q517g/r2t2poiNHR0cp6Gxp6Bqju\n/7s6ft/aCf39QOunVi8DDrQekJkHgLcBRMRC4KLMPNQM869n5nBz35eBVzH2g6H1/I3ARoCBgYGc\nzr3ojUajknvZd9459q+uyvvmq+qtG+rcG9S4vwcWMzQ0VFlvH9kz9sDBwcFqHkdSx+9bO5d37gbO\njoizImIecAlwS+sBEbEkIp5/rauBTc3lh4DXRcSxETGXsTdxvbwjSV0yaehn5ghwJXArY4H9mczc\nGREbIuItzcMGgT0RcT/QB7y/uX0z8H1gB2PX/e/NzC92tgVJUrvaegxDZm4Btozbdk3L8mbGAn78\neaPAu6ZZo6RZrMoHAV7z40NjCxU9BXbp/JWMzWnrw2fvSJq6FRczPDTE4m7XMRMe3UHfcUPdrqLj\nDH1JU1fxgwA3ND857uYrKngj94YLYah+oe+zdySpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JB6nXL\n5pevYtV3/wcemPm7hpf/bB8Pzn3JjI8jSZ3kTH+KHpz7Er52/Ou7XYYkHZV6zfTX/C33HF/NU/Ge\n/yORdTM+kiR1jjN9SSqIoS9JBanV5Z2//uJO7tz1zP9/0MJM2nXwSfqXLprxcSSpk5zpT1H/0kWs\nXTX+o4IlaXar1Uz/fW8+h8aJP6zso9Qkqdc405ekghj6klQQQ1+SCmLoS1JBDH1JKoihL0kFMfQl\nqSCGviQVxNCXpIIY+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqSVBBDX5IK\n0lboR8T5EbEnIvZGxFUT7D8zIrZGxH0R0YiIZS37zoiIr0TE7ojYFRHLO1e+JOloTBr6ETEH+DCw\nBugHLo2I/nGHXQfcmJkrgQ3AtS37bgQ+mJkvB1YDj3eicEnS0Wtnpr8a2JuZ+zLzMHATsHbcMf3A\n1uby7c/vb/5wODYzbwPIzOHMfLojlUuSjlo7oX8a8HDL+v7mtlb3Ahc1l98KnBgRJwO/AgxFxOci\n4tsR8cHmbw6SpC44to1jYoJtOW59PXB9RFwO3AE8Aow0X/83gVcCDwE3A5cDH/+5ASLWAesA+vr6\naDQa7db/C4aHh6d1/mxmb72rzv1V2dvQ0DMAlYy3amiI0dHR2n3f2gn9/cDpLevLgAOtB2TmAeBt\nABGxELgoMw9FxH7g25m5r7nv34FXMS70M3MjsBFgYGAgBwcHp9QMjP3HMJ3zZzN761117q/K3j6y\n5y52HXySj+yZP+NjXfNMsOCYqN33rZ3Qvxs4OyLOYmwGfwlwWesBEbEEeCIznwOuBja1nHtSRJyS\nmT8E3gBs61TxksqydtX4K8sz5+nDo4wcM/6iRu+bNPQzcyQirgRuBeYAmzJzZ0RsALZl5i3AIHBt\nRCRjl3fe3Tx3NCLWA1sjIoDtwD/PTCuS6u6yc8/gsnPPqGSsnR+Yw8jISCVjVamdmT6ZuQXYMm7b\nNS3Lm4HNRzj3NmDlNGqUJHWIf5ErSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JKoih\nL0kFMfQlqSCGviQVxNCXpIIY+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqS\nVBBDX5IKYuhLUkEMfUkqiKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JKoihL0kF\nMfQlqSCGviQVpK3Qj4jzI2JPROyNiKsm2H9mRGyNiPsiohERy8btXxQRj0TE9Z0qXJJ09CYN/YiY\nA3wYWAP0A5dGRP+4w64DbszMlcAG4Npx+/8G+O/plytJmo52Zvqrgb2ZuS8zDwM3AWvHHdMPbG0u\n3966PyJ+HegDvjL9ciVJ09FO6J8GPNyyvr+5rdW9wEXN5bcCJ0bEyRFxDPD3wJ9Pt1BJ0vQd28Yx\nMcG2HLe+Hrg+Ii4H7gAeAUaAPwW2ZObDERO9THOAiHXAOoC+vj4ajUYbZU1seHh4WufPZvbWu+rc\nX117O2lkBJLa9dZO6O8HTm9ZXwYcaD0gMw8AbwOIiIXARZl5KCJeDfxmRPwpsBCYFxHDmXnVuPM3\nAhsBBgYGcnBwcIrtjH2DpnP+bGZvvavO/dW1t513HsvIyEjtemsn9O8Gzo6IsxibwV8CXNZ6QEQs\nAZ7IzOeAq4FNAJn5By3HXA4MjA98SVJ1Jr2mn5kjwJXArcBu4DOZuTMiNkTEW5qHDQJ7IuJ+xt60\nff8M1StJmoZ2Zvpk5hZgy7ht17QsbwY2T/IanwA+cdQVSpI6xr/IlaSCGPqSVBBDX5IKYuhLUkEM\nfUkqiKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JKoihL0kFMfQlqSCGviQVxNCX\npIIY+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqSVBBDX5IKYuhLUkEMfUkq\niKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCtJW6EfE+RGxJyL2RsRVE+w/MyK2RsR9EdGIiGXN\n7asi4q6I2Nnc9/ZONyBJat+koR8Rc4APA2uAfuDSiOgfd9h1wI2ZuRLYAFzb3P408I7MPAc4H/hQ\nRCzuVPGSpKPTzkx/NbA3M/dl5mHgJmDtuGP6ga3N5duf35+Z92fm95rLB4DHgVM6Ubgk6ei1E/qn\nAQ+3rO9vbmt1L3BRc/mtwIkRcXLrARGxGpgHfH9qpUqSpuvYNo6JCbbluPX1wPURcTlwB/AIMPL/\nLxCxFPgX4J2Z+dwvDBCxDlgH0NfXR6PRaKf2CQ0PD0/r/NnM3npXnfura28njYxAUrve2gn9/cDp\nLevLgAOtBzQv3bwNICIWAhdl5qHm+iLgS8BfZubXJxogMzcCGwEGBgZycHDw6Lpo0Wg0mM75s5m9\n9a4691fX3nbeeSwjIyO1662dyzt3A2dHxFkRMQ+4BLil9YCIWBIRz7/W1cCm5vZ5wOcZe5P33zpX\ntiRpKiYN/cwcAa4EbgV2A5/JzJ0RsSEi3tI8bBDYExH3A33A+5vbfx94LXB5RNzT/FrV6SYkSe1p\n5/IOmbkF2DJu2zUty5uBzROc90ngk9OsUZLUIf5FriQVpK2ZviSV6KXP/QBuuLC6AU9dAWv+dkaH\nMPQlaQJfO/71jIyM8KvdLqTDDH1JmsDWEy7gs4dfw61XrOl2KR3lNX1JKoihL0kFMfQlqSCGviQV\nxNCXpIIY+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqSVBAfrSxJR/DQU8/x\n9o/eVdl4/S9exPvefM6MjuFMX5ImsHbVaZxxYv0i0pm+JE3gsnPP4MXP7GNw8NXdLqWj6vdjTJJ0\nRIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JKoihL0kFiczsdg0/JyJ+CPxgGi+xBPhRh8qZbeyt\nd9W5P3ubHc7MzFMmO2jWhf50RcS2zBzodh0zwd56V537s7fe4uUdSSqIoS9JBalj6G/sdgEzyN56\nV537s7ceUrtr+pKkI6vjTF+SdAS1Cf2IOD8i9kTE3oi4qtv1dFJEbIqIxyPiO92updMi4vSIuD0i\ndkfEzoh4T7dr6pSIOC4ivhkR9zZ7++tu19RpETEnIr4dEf/R7Vo6LSIejIgdEXFPRGzrdj2dUovL\nOxExB7gf+B1gP3A3cGlm7upqYR0SEa8FhoEbM/MV3a6nkyJiKbA0M78VEScC24Hfq8P3LiICWJCZ\nwxExF/gq8J7M/HqXS+uYiHgvMAAsysw3dbueToqIB4GBzOyV+/TbUpeZ/mpgb2buy8zDwE3A2i7X\n1DGZeQfwRLfrmAmZeTAzv9VcfgrYDZzW3ao6I8cMN1fnNr96f5bVFBHLgAuBj3W7FrWvLqF/GvBw\ny/p+ahIcJYmI5cArgW90t5LOaV7+uAd4HLgtM2vTG/Ah4C+A57pdyAxJ4CsRsT0i1nW7mE6pS+jH\nBNtqM6MqQUQsBD4L/FlmPtntejolM0czcxWwDFgdEbW4PBcRbwIez8zt3a5lBp2Xmb8GrAHe3bzM\n2vPqEvr7gdNb1pcBB7pUi45S83r3Z4F/zczPdbuemZCZQ0ADOL/LpXTKecBbmte9bwLeEBGf7G5J\nnZWZB5r/fBz4PGOXkXteXUL/buDsiDgrIuYBlwC3dLkmtaH5ZufHgd2Z+Q/drqeTIuKUiFjcXD4e\n+G3gu92tqjMy8+rMXJaZyxn7/+2/MvMPu1xWx0TEguaNBUTEAuCNQC3unqtF6GfmCHAlcCtjbwR+\nJjN3dreqzomITwN3AS+LiP0R8cfdrqmDzgP+iLGZ4j3Nrwu6XVSHLAVuj4j7GJuY3JaZtbu1sab6\ngK9GxL3AN4EvZeZ/drmmjqjFLZuSpPbUYqYvSWqPoS9JBTH0Jakghr4kFcTQl6SCGPqSVBBDX5IK\nYuhLUkH+D8FS4gKWiUstAAAAAElFTkSuQmCC\n", 383 | "text/plain": [ 384 | "
" 385 | ] 386 | }, 387 | "metadata": {}, 388 | "output_type": "display_data" 389 | } 390 | ], 391 | "source": [ 392 | "c_th = 0.7\n", 393 | "ndetect = 1\n", 394 | "isqso_qn = (c_line>c_th).sum(axis=0)>=ndetect\n", 395 | "\n", 396 | "dz_int = 0.5\n", 397 | "zmin = 0\n", 398 | "zmax = 7\n", 399 | "z_int = np.arange(zmin+dz_int/2,zmax+dz_int/2,dz_int)\n", 400 | "pur_z = z_int*0\n", 401 | "com_z = z_int*0\n", 402 | "for i, zint in enumerate(z_int):\n", 403 | " w0 = isqso_qn & zgood & (abs(zbest-zint)=2.1)\n", 457 | "print('{:^6} | '.format(\"\"),end=\"\")\n", 458 | "print('{:^6} | '.format('NSPEC'),end=\"\")\n", 459 | "for c in classes_qn:\n", 460 | " print('{:^6} | '.format(c),end=\"\")\n", 461 | "print(\"\")\n", 462 | "for i in range(nclass):\n", 463 | " w = Y_val.argmax(axis=1)==i\n", 464 | " print('{:^6} | '.format(classes[i]),end=\"\")\n", 465 | " print('{:^6} | '.format(w.sum()),end=\"\")\n", 466 | " for j in range(nclass_qn):\n", 467 | " conf_mat[i,j] = (w & class_qn[j]).sum()/w.sum()\n", 468 | " print(\"{:^6} | \".format(round(conf_mat[i,j]*100,2)), end=\"\")\n", 469 | " print(\"\")" 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": 10, 475 | "metadata": {}, 476 | "outputs": [ 477 | { 478 | "name": "stdout", 479 | "output_type": "stream", 480 | "text": [ 481 | "35121\n", 482 | "767.889477105195\n" 483 | ] 484 | }, 485 | { 486 | "name": "stderr", 487 | "output_type": "stream", 488 | "text": [ 489 | "/global/homes/n/nbusca/.conda/envs/qnet/lib/python3.6/site-packages/ipykernel_launcher.py:3: RuntimeWarning: divide by zero encountered in true_divide\n", 490 | " This is separate from the ipykernel package so we can avoid doing imports until\n" 491 | ] 492 | }, 493 | { 494 | "data": { 495 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAFj1JREFUeJzt3X+M5Hd93/Hnqz5sAgR8xmtq7o7e\nkVxoDaLF3RqnaSMXB2MbxFEploxQuYClU4MNpGkKdi3FaiiSCVUJVijoiq/YFbFxDCnX4tRcHFxa\nCf9Y88PYGPBiqG9jh1t0xkmKAjl494/5HB7vzd7uzuzNzu73+ZBG8/2+v5+Z+Xx2Z77v+Xw+3/l+\nU1VIkrrnb611BSRJa8MEIEkdZQKQpI4yAUhSR5kAJKmjTACS1FEmAEnqKBOAJHWUCUCSOmrTWlfg\neE4//fTavn37WldDktaV++6773tVNbVUuSUTQJJ9wOuAQ1X1sr7424ErgCPAZ6rqXS1+FXAZ8GPg\nHVV1e4tfCHwQOAn4aFVdu9Rrb9++nZmZmaWKSZL6JPm/yym3nB7Ax4DfB27se/J/BuwCXl5VP0xy\nRoufBVwKvBR4IfAnSX6hPexDwKuBOeDeJPur6mvLa44kabUtmQCq6vNJti8I/zpwbVX9sJU51OK7\ngJtb/NtJZoFz2rbZqnoEIMnNrawJQJLWyLCTwL8A/NMkdyf5X0n+UYtvAQ72lZtrscXix0iyJ8lM\nkpn5+fkhqydJWsqwCWATsBk4F/g3wC1JAmRA2TpO/Nhg1d6qmq6q6ampJecwJElDGvYooDngU9W7\nmMA9SX4CnN7i2/rKbQUea8uLxSVJa2DYHsB/A14F0CZ5Twa+B+wHLk1ySpIdwE7gHuBeYGeSHUlO\npjdRvH/UykuShrecw0BvAs4DTk8yB1wD7AP2JXkA+BGwu/UGHkxyC73J3SPA5VX14/Y8VwC30zsM\ndF9VPXgC2iNJWqZM8iUhp6eny98BSNLKJLmvqqaXKuepICSpoyb6VBDSpNl+5Wd+uvyda1+7hjWR\nRmcPQJI6ygQgSR1lApCkjjIBSFJHmQAkqaNMAJLUUSYASeoofwcgrQJ/H6D1yAQgLaF/5y5tJA4B\nSVJH2QOQhmTPQOudPQBJ6igTgCR1lAlAkjrKOQBpgFHG9z0kVOvFkj2AJPuSHGqXf1y47beSVJLT\n23qSXJdkNsn9Sc7uK7s7ycPttnt1myFJWqnlDAF9DLhwYTDJNuDVwKN94YvoXQh+J7AH+HArexq9\nawm/EjgHuCbJ5lEqLkkazZIJoKo+DxwesOkDwLuA/osK7wJurJ67gFOTnAm8BjhQVYer6gngAAOS\niiRpfIaaBE7yeuDPquorCzZtAQ72rc+12GJxSdIaWfEkcJJnAVcDFwzaPCBWx4kPev499IaPeNGL\nXrTS6kmSlmmYHsDPATuAryT5DrAV+GKSv03vm/22vrJbgceOEz9GVe2tqumqmp6amhqiepKk5Vhx\nAqiqr1bVGVW1vaq209u5n11Vfw7sB97cjgY6F3iyqh4HbgcuSLK5Tf5e0GKSpDWynMNAbwK+ALwk\nyVySy45T/DbgEWAW+M/A2wCq6jDwHuDedvudFpMkrZEl5wCq6o1LbN/et1zA5YuU2wfsW2H9JEkn\niKeCkKSO8lQQ0gnkaSE0yewBSFJHmQAkqaNMAJLUUSYASeooE4AkdZRHAUmNF3lX19gDkKSOMgFI\nUkeZACSpo0wAktRRJgBJ6igTgCR1lIeBqtPGeeinJ4bTpLEHIEkdZQKQpI4yAUhSRy3nmsD7khxK\n8kBf7P1Jvp7k/iR/lOTUvm1XJZlN8o0kr+mLX9his0muXP2mSJJWYjk9gI8BFy6IHQBeVlUvB74J\nXAWQ5CzgUuCl7TH/KclJSU4CPgRcBJwFvLGVlSStkSUTQFV9Hji8IPbZqjrSVu8CtrblXcDNVfXD\nqvo2MAuc026zVfVIVf0IuLmVlSStkdU4DPStwCfa8hZ6CeGouRYDOLgg/spVeG1pxTzrp9Qz0iRw\nkquBI8DHj4YGFKvjxAc9554kM0lm5ufnR6meJOk4hk4ASXYDrwPeVFVHd+ZzwLa+YluBx44TP0ZV\n7a2q6aqanpqaGrZ6kqQlDJUAklwIvBt4fVX9oG/TfuDSJKck2QHsBO4B7gV2JtmR5GR6E8X7R6u6\nJGkUS84BJLkJOA84PckccA29o35OAQ4kAbirqv5lVT2Y5Bbga/SGhi6vqh+357kCuB04CdhXVQ+e\ngPZI64KnhdAkWDIBVNUbB4SvP0759wLvHRC/DbhtRbWTJJ0w/hJYkjrKBCBJHWUCkKSOMgFIUkeZ\nACSpo0wAktRRXhJSneD5f6Rj2QOQpI4yAUhSR5kAJKmjTACS1FEmAEnqKBOAJHWUCUCSOsoEIEkd\nZQKQpI4yAUhSR3kqCGmNeXlIrZUlewBJ9iU5lOSBvthpSQ4kebjdb27xJLkuyWyS+5Oc3feY3a38\nw0l2n5jmSJKWazlDQB8DLlwQuxK4o6p2Ane0dYCLgJ3ttgf4MPQSBr2Lyb8SOAe45mjSkCStjSUT\nQFV9Hji8ILwLuKEt3wC8oS9+Y/XcBZya5EzgNcCBqjpcVU8ABzg2qUiSxmjYSeAXVNXjAO3+jBbf\nAhzsKzfXYovFj5FkT5KZJDPz8/NDVk+StJTVngTOgFgdJ35ssGovsBdgenp6YBlpObwGgHR8w/YA\nvtuGdmj3h1p8DtjWV24r8Nhx4pKkNTJsAtgPHD2SZzfw6b74m9vRQOcCT7YhotuBC5JsbpO/F7SY\nJGmNLDkElOQm4Dzg9CRz9I7muRa4JcllwKPAJa34bcDFwCzwA+AtAFV1OMl7gHtbud+pqoUTy5Kk\nMVoyAVTVGxfZdP6AsgVcvsjz7AP2rah2kqQTxlNBSFJHmQAkqaNMAJLUUZ4MTpognhhO42QPQJI6\nygQgSR1lApCkjjIBSFJHOQksTSgnhHWimQC0YXj2T2llHAKSpI4yAUhSR5kAJKmjTACS1FEmAEnq\nKBOAJHWUCUCSOmqkBJDkXyV5MMkDSW5K8swkO5LcneThJJ9IcnIre0pbn23bt69GAyRJwxk6ASTZ\nArwDmK6qlwEnAZcC7wM+UFU7gSeAy9pDLgOeqKqfBz7QykmS1sioQ0CbgJ9Jsgl4FvA48Crg1rb9\nBuANbXlXW6dtPz9JRnx9SdKQhk4AVfVnwH8AHqW3438SuA/4flUdacXmgC1teQtwsD32SCv//GFf\nX5I0mlGGgDbT+1a/A3gh8GzgogFF6+hDjrOt/3n3JJlJMjM/Pz9s9SRJSxhlCOhXgG9X1XxV/Q3w\nKeAfA6e2ISGArcBjbXkO2AbQtj8POLzwSatqb1VNV9X01NTUCNWTJB3PKAngUeDcJM9qY/nnA18D\nPgf8aiuzG/h0W97f1mnb/7SqjukBSJLGY5Q5gLvpTeZ+Efhqe669wLuB30wyS2+M//r2kOuB57f4\nbwJXjlBvSdKIRroeQFVdA1yzIPwIcM6Asn8NXDLK60kLeQ0AaXj+EliSOsoEIEkdZQKQpI7ymsBa\ndxz3l1aHCUBaB/qT3neufe0a1kQbiUNAktRRJgBJ6igTgCR1lAlAkjrKSWBpnXFCWKvFHoAkdZQJ\nQJI6ygQgSR1lApCkjnISWOuCp3+QVp89AEnqKBOAJHWUCUCSOmqkBJDk1CS3Jvl6koeS/GKS05Ic\nSPJwu9/cyibJdUlmk9yf5OzVaYIkaRij9gA+CPzPqvq7wN8HHqJ3sfc7qmoncAdPXfz9ImBnu+0B\nPjzia0uSRjD0UUBJngv8MvBrAFX1I+BHSXYB57ViNwB3Au8GdgE3VlUBd7Xew5lV9fjQtZc6ztNC\naBSj9ABeDMwD/yXJl5J8NMmzgRcc3am3+zNa+S3Awb7Hz7WYJGkNjJIANgFnAx+uqlcA/4+nhnsG\nyYBYHVMo2ZNkJsnM/Pz8CNWTJB3PKAlgDpirqrvb+q30EsJ3k5wJ0O4P9ZXf1vf4rcBjC5+0qvZW\n1XRVTU9NTY1QPUnS8QydAKrqz4GDSV7SQucDXwP2A7tbbDfw6ba8H3hzOxroXOBJx/8lae2MeiqI\ntwMfT3Iy8AjwFnpJ5ZYklwGPApe0srcBFwOzwA9aWUnSGhkpAVTVl4HpAZvOH1C2gMtHeT1J0urx\nl8CS1FEmAEnqKBOAJHWUCUCSOsoLwmhieREY6cQyAUgbhOcF0ko5BCRJHWUCkKSOMgFIUkeZACSp\no0wAktRRHgWkieKhn9L42AOQpI6yB6A157d+aW3YA5CkjjIBSFJHmQAkqaNMAJLUUSMngCQnJflS\nkv/R1nckuTvJw0k+0a4XTJJT2vps27591NeWJA1vNXoA7wQe6lt/H/CBqtoJPAFc1uKXAU9U1c8D\nH2jlJElrZKQEkGQr8Frgo209wKuAW1uRG4A3tOVdbZ22/fxWXtIq237lZ356kxYzag/g94B3AT9p\n688Hvl9VR9r6HLClLW8BDgK07U+28k+TZE+SmSQz8/PzI1ZPkrSYoRNAktcBh6rqvv7wgKK1jG1P\nBar2VtV0VU1PTU0NWz1J0hJG+SXwLwGvT3Ix8EzgufR6BKcm2dS+5W8FHmvl54BtwFySTcDzgMMj\nvL4kaQRDJ4Cqugq4CiDJecBvVdWbkvwh8KvAzcBu4NPtIfvb+hfa9j+tqmN6AJJW18J5AC8XqaNO\nxLmA3g3cnOTfA18Crm/x64H/mmSW3jf/S0/Aa2udcHJSWnurkgCq6k7gzrb8CHDOgDJ/DVyyGq8n\nSRqdvwSWpI4yAUhSR5kAJKmjvCCM1DH9E/AeEdRt9gAkqaNMAJLUUSYASeoo5wA0Nv74S5os9gAk\nqaNMAJLUUQ4B6YRy2GeyeUhot9kDkKSOMgFIUkeZACSpo0wAktRRTgJLApwQ7iJ7AJLUUUMngCTb\nknwuyUNJHkzyzhY/LcmBJA+3+80tniTXJZlNcn+Ss1erEZKklRtlCOgI8K+r6otJfha4L8kB4NeA\nO6rq2iRXAlfSu07wRcDOdnsl8OF2rw3GY/+l9WHoBFBVjwOPt+W/TPIQsAXYBZzXit1A71rB727x\nG6uqgLuSnJrkzPY8kiaI8wHdsCpzAEm2A68A7gZecHSn3u7PaMW2AAf7HjbXYpKkNTByAkjyHOCT\nwG9U1V8cr+iAWA14vj1JZpLMzM/Pj1o9SdIiRkoASZ5Bb+f/8ar6VAt/N8mZbfuZwKEWnwO29T18\nK/DYwuesqr1VNV1V01NTU6NUT5J0HEPPASQJcD3wUFX9x75N+4HdwLXt/tN98SuS3Exv8vdJx//X\nN8eJu8H/88Y1ylFAvwT8C+CrSb7cYv+W3o7/liSXAY8Cl7RttwEXA7PAD4C3jPDamjAe+dMNJoON\nZZSjgP4Pg8f1Ac4fUL6Ay4d9PU0Gd/TSxuEvgSWpo0wAktRRJgBJ6igTgCR1lKeDljSUxQ4I8Oig\n9cMegCR1lD0ALclDP6WNyQSggdzpa1j+WGz9MAHop9zpS91iApB0wtgbmGwmgA7yQ6m14Ptu8pgA\nJE0Mk8R4eRioJHWUPYB1buHEbf+3puV8m3LiV2vBb/qTwQSwwSy2Q3dHL2khE4CkibRYL8Hew+pJ\n7zotk2l6erpmZmbWuhoTzW/2Uo/J4ClJ7quq6aXK2QOYACv9RuNOXzqWPYOVG3sCSHIh8EHgJOCj\nVXXtuOswTu7cpfEbJhl0MYGMdQgoyUnAN4FXA3PAvcAbq+prg8qvpyEgd9zSxrdeEsOkDgGdA8xW\n1SMASW4GdgEDE8A4uOOWNIxR9h2TkkjGnQC2AAf71ueAV56oF3PnLmk1rdY+ZTnPM44kMe4EkAGx\np41BJdkD7Gmrf5XkGyO83unA90Z4/KTYKO0A2zKpNkpbNko7yPtGasvfWU6hcSeAOWBb3/pW4LH+\nAlW1F9i7Gi+WZGY542CTbqO0A2zLpNoobdko7YDxtGXc5wK6F9iZZEeSk4FLgf1jroMkiTH3AKrq\nSJIrgNvpHQa6r6oeHGcdJEk9Y/8dQFXdBtw2ppdblaGkCbBR2gG2ZVJtlLZslHbAGNoy0aeCkCSd\nOF4PQJI6at0mgCTvSXJ/ki8n+WySF7Z4klyXZLZtP7vvMbuTPNxuu/vi/zDJV9tjrksy6HDVE9mW\n9yf5eqvvHyU5tW/bVa1e30jymr74hS02m+TKvviOJHe3Nn6iTbaPqx2XJHkwyU+STC/Ytm7asZTF\n6jxJkuxLcijJA32x05IcaH/TA0k2t/iKPzNjbMe2JJ9L8lB7b71zHbflmUnuSfKV1pZ/1+ID3+tJ\nTmnrs2379r7nGvh5WrGqWpc34Ll9y+8APtKWLwb+mN5vDs4F7m7x04BH2v3mtry5bbsH+MX2mD8G\nLhpzWy4ANrXl9wHva8tnAV8BTgF2AN+iN3l+Ult+MXByK3NWe8wtwKVt+SPAr4+xHX8PeAlwJzDd\nF19X7ViijYvWeZJuwC8DZwMP9MV+F7iyLV/Z9z5b8WdmjO04Ezi7Lf8svVPJnLVO2xLgOW35GcDd\nrY4D3+vA23hqv3Yp8Im2PPDzNEyd1m0PoKr+om/12Tz1g7JdwI3VcxdwapIzgdcAB6rqcFU9ARwA\nLmzbnltVX6jeX/dG4A3jawlU1Wer6khbvYve7yOOtuXmqvphVX0bmKV3Oo2fnlKjqn4E3Azsaj2X\nVwG3tsffwBjbUlUPVdWgH+6tq3YsYWCd17hOx6iqzwOHF4R30ftbwtP/piv6zJz42j+lqh6vqi+2\n5b8EHqJ3RoH12Jaqqr9qq89ot2Lx93p/G28Fzm+fjcU+Tyu2bhMAQJL3JjkIvAn47RYedLqJLUvE\n5wbE18pb6X2DgZW35fnA9/uSyVq35aiN0g5YvM7rwQuq6nHo7ViBM1p8pf+fNdGGQF5B75vzumxL\nkpOSfBk4RC8JfYvF3+s/rXPb/iS9z8aqtWWiE0CSP0nywIDbLoCqurqqtgEfB644+rABT1VDxFfV\nUm1pZa4GjtBrD0PU+YS3ZTntGPSwReq1pv+TIU1y3YY18f+HJM8BPgn8xoLe/zFFB8Qmpi1V9eOq\n+gf0evnn0Bs2PaZYuz/hbZnoC8JU1a8ss+gfAJ8BrmHx003MAectiN/Z4lsHlF9VS7WlTUq9Dji/\nDUXB8U+dMSj+PXpd3k3tG8Oqt2UF/5N+E9eOESx5OpMJ9t0kZ1bV421Y5FCLr/QzM1ZJnkFv5//x\nqvpUC6/LthxVVd9Pcie9OYDF3utH2zKXZBPwPHrDeqv3HhznJMhq3oCdfctvB25ty6/l6ZNA99RT\nk0DfpjcBtLktn9a23dvKHp0EvnjMbbmQ3imxpxbEX8rTJ3seoTcJuakt7+CpiciXtsf8IU+fUHrb\nGvxv7uTpk8Drsh2LtG3ROk/aDdjO0yeB38/TJ05/ty2v+DMzxjaE3rzc7y2Ir8e2TAGntuWfAf43\nvS99A9/rwOU8fRL4lrY88PM0VJ3W+k06wh/zk8ADwP3Afwe29L1hPkRvbO2rC3ZEb6U3YTILvKUv\nPt2e61vA79N+IDfGtszSG9P7crt9pG/b1a1e36Dv6CR6Rzt8s227ui/+YnpHNc22N9YpY2zHP6f3\n7eSHwHeB29djO5bRzoF1nqQbcBPwOPA37X9yGb3x4zuAh9v90S9AK/7MjLEd/4Te8Mb9fZ+Pi9dp\nW14OfKm15QHgt1t84HsdeGZbn23bX9z3XAM/Tyu9+UtgSeqoiZ4EliSdOCYASeooE4AkdZQJQJI6\nygQgSR1lApCkjjIBSFJHmQAkqaP+P1kJqAfUJ17uAAAAAElFTkSuQmCC\n", 496 | "text/plain": [ 497 | "
" 498 | ] 499 | }, 500 | "metadata": {}, 501 | "output_type": "display_data" 502 | } 503 | ], 504 | "source": [ 505 | "wqso = (c_line>0.5).sum(axis=0)>=1\n", 506 | "print(wqso.sum())\n", 507 | "dv = 300000*(zbest-z_val)/(1+z_val)\n", 508 | "w=abs(dv)<10000\n", 509 | "plt.hist(dv[w],range=(-3000,3000),bins=100)\n", 510 | "print(np.std(dv[wqso&w]))" 511 | ] 512 | }, 513 | { 514 | "cell_type": "code", 515 | "execution_count": null, 516 | "metadata": {}, 517 | "outputs": [], 518 | "source": [] 519 | } 520 | ], 521 | "metadata": { 522 | "kernelspec": { 523 | "display_name": "qnet", 524 | "language": "python", 525 | "name": "qnet" 526 | }, 527 | "language_info": { 528 | "codemirror_mode": { 529 | "name": "ipython", 530 | "version": 3 531 | }, 532 | "file_extension": ".py", 533 | "mimetype": "text/x-python", 534 | "name": "python", 535 | "nbconvert_exporter": "python", 536 | "pygments_lexer": "ipython3", 537 | "version": "3.6.5" 538 | } 539 | }, 540 | "nbformat": 4, 541 | "nbformat_minor": 2 542 | } 543 | -------------------------------------------------------------------------------- /etc/qn_desi.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### QuasarNET: Inspect DESI sims\n", 8 | "\n", 9 | "This notebook will walk you through how to run [QuasarNET](https://arxiv.org/pdf/1808.09955.pdf]):\n", 10 | " * load training data\n", 11 | " * load DESI sims validation data\n", 12 | " * train the network (on a small training sample, optional)\n", 13 | " * load pre-trained weights\n", 14 | " * plot example spectra\n", 15 | " * produce QA plots and confusion matrix\n", 16 | "\n", 17 | "#### Installation instructions (requires python3):\n", 18 | "##### - on a standard system\n", 19 | "\n", 20 | "```bash\n", 21 | "git clone https://github.com/ngbusca/QuasarNET.git\n", 22 | "cd QuasarNET\n", 23 | "pip install -r requirements.txt --user\n", 24 | "python setup.py install --user\n", 25 | "```\n", 26 | "\n", 27 | "##### - at NERSC (e.g. if you wish to run this notebook at jupyter.nersc.gov)\n", 28 | "\n", 29 | "```bash\n", 30 | "conda create -n qnet python=3 qnet scipy numpy fitsio h5py ipykernel\n", 31 | "source activate qnet\n", 32 | "python -m ipykernel install --user --name qnet --display-name qnet\n", 33 | "pip install tensorflow\n", 34 | "pip install keras>=2.2.4\n", 35 | "git clone https://github.com/ngbusca/QuasarNET.git\n", 36 | "cd QuasarNET\n", 37 | "python setup.py install\n", 38 | "```\n", 39 | "\n", 40 | "#### - Download the pre-trained weights\n", 41 | "The pre-trained weights are available at: https://www.kaggle.com/ngbusca/qnet_trained_models\n", 42 | "\n", 43 | "Download the weights to the `QuasarNET/weights/` directory, unzip the file and set read/write permissions\n", 44 | "(skip the `kaggle datasets...` line if you've downloaded the data through the website).\n", 45 | "\n", 46 | "```bash\n", 47 | "cd weights\n", 48 | "kaggle datasets download ngbusca/qnet_trained_models\n", 49 | "unzip qnet_trained_models.zip\n", 50 | "chmod 600 *\n", 51 | "```" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 1, 57 | "metadata": {}, 58 | "outputs": [ 59 | { 60 | "name": "stderr", 61 | "output_type": "stream", 62 | "text": [ 63 | "Using TensorFlow backend.\n" 64 | ] 65 | } 66 | ], 67 | "source": [ 68 | "import numpy as np\n", 69 | "from matplotlib import pyplot as plt\n", 70 | "import fitsio\n", 71 | "from keras.models import load_model\n", 72 | "from quasarnet.models import custom_loss\n", 73 | "from quasarnet.io import read_desi_spectra, wave\n", 74 | "from quasarnet.utils import process_preds, absorber_IGM\n", 75 | "%matplotlib inline" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | "#### Load the DESI sims\n", 83 | "\n", 84 | "The next cell loads all spectra from a spectra file. It firsts gets the resampled fluxes on the QNet wavelength grid, then removes the mean and sets the rms of the flux to 1." 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 2, 90 | "metadata": {}, 91 | "outputs": [ 92 | { 93 | "name": "stdout", 94 | "output_type": "stream", 95 | "text": [ 96 | "WARN: can't load desi_mask, ignoring mask!\n", 97 | "INFO: found 1021 quasar targets\n", 98 | "INFO: founds 1021 good spectra\n" 99 | ] 100 | } 101 | ], 102 | "source": [ 103 | "file = '/global/projecta/projectdirs/desi/mocks/lya_forest/london/v4.0/quick-0.0/spectra-16/0/0/spectra-16-0.fits'\n", 104 | "tids_val, flux = read_desi_spectra(file, ignore_quasar_mask=True)\n", 105 | "mflux = np.average(flux[:,:443], weights=flux[:,443:],axis=1)\n", 106 | "sflux = np.average((flux[:,:443]-mflux[:,None])**2, weights=flux[:,443:], axis=1)\n", 107 | "sflux = np.sqrt(sflux)\n", 108 | "X_val = (flux[:,:443]-mflux[:,None])/sflux[:,None]\n", 109 | "nspec = X_val.shape[0]" 110 | ] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "metadata": {}, 115 | "source": [ 116 | "#### Define the \"features\" that the network was trained to recognize\n", 117 | "\n", 118 | "The features are defined by their rest wavelength. \n", 119 | "A dictionary `{feature_name:feature_wavelength}` is defined in `quasarnet.util.absorber_IGM`, which currently contains typical quasar broad emission lines. It could be easily extended to include other features by extending the dictionary." 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 3, 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "lines=['LYA','CIV(1548)','CIII(1909)', 'MgII(2796)','Hbeta','Halpha']\n", 129 | "lines_bal=['CIV(1548)']" 130 | ] 131 | }, 132 | { 133 | "cell_type": "markdown", 134 | "metadata": {}, 135 | "source": [ 136 | "#### Load a pre-trained model\n", 137 | "\n", 138 | "The following cell loads pre-trained weights for the network, corresponding to the split defined earlier. The pre-training was done over the full training data sample and 200 epochs." 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": 4, 144 | "metadata": {}, 145 | "outputs": [], 146 | "source": [ 147 | "imodel=0\n", 148 | "#model = load_model('../weights/qn_train_{}.h5'.format(imodel),custom_objects={'custom_loss':custom_loss})\n", 149 | "model = load_model('../runs/v18.0/qn_train_{}.h5'.format(imodel),custom_objects={'custom_loss':custom_loss})" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "#### Example spectra\n", 157 | "\n", 158 | "Let's now take a look at the network output by examining a few examples. If you skipped loading the pre-trained weights you will be looking at the model you trained (it's actually not that bad!).\n", 159 | "\n", 160 | "The network outputs confidences and positions of the features defined earlier. The following plot shows a spectrum from the validation sample and the detected features. You can change the index `ival` to change the spectrum to be shown." 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": 5, 166 | "metadata": {}, 167 | "outputs": [ 168 | { 169 | "name": "stdout", 170 | "output_type": "stream", 171 | "text": [ 172 | "INFO: nspec = 1, nboxes=13\n" 173 | ] 174 | }, 175 | { 176 | "data": { 177 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEJCAYAAACE39xMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XecVNX5+PHPmZntjWWXXcrSe+/W\nLEJQUZQohihqENTE3muU5CcmXywJidFYohFFjBSDjRhBRdgEUUAQkCa9LH3Z3nfK+f1x7wyzu7Ns\nmy0z87xfr3nt3HvPvfeZ2dlnz5x77jlKa40QQojAZ2npAIQQQviHJHQhhAgSktCFECJISEIXQogg\nIQldCCGChCR0IYQIEpLQhRAiSEhCF0KIICEJvZkopYq8Hi6lVKnX8o1e5TKUUrlKqYgq+x/02ueE\nUmqeUiq2SpmpSql1SqlipdQp8/ldSilVw3Hcj5frEp+578VVjlOolMpTSn2jlLpDKeX3z5RS6h6l\n1AalVLlSal6VbRFKqblKqUNmLJuUUpfXZV9ze1GVh1Mp9Tev7f9USh1XShUopXYrpX5VZf+pSqmd\n5nu+TymV7u/X39Jqe4/rW978jJd5vee7quzfXym1UimVr5Taq5Sa3JSvL6horeXRzA/gIHCxj/Xd\nACeQA/yipn2A9sAWYLbX9oeBk8AUIA5QwHDgPSCitnPXMT7vGLyfJwA/Aw4AbzfB+3UNcDXwGjCv\nyrYYYJb53lmAK4FCoFtt+/o4TwxQBIzxWjfQ/f4B/YATwEhz+RLgEHCeee5OQKeW/nw1wft/1ve4\nvuWBDOBXNexrA3YDDwFW4KdAMdCnpd+HQHhIDb11uQlYC8wDptdUSGt9AvgcGAaglEoAfg/cpbVe\norUu1IZNWusbtdblTRm01jpfa70UuA6YrpQa5Ofjf6i1/hjI9rGtWGs9S2t9UGvt0lp/ivGPZWRt\n+/owBTgFrPY6/nav90+bj57m8tPA77XWa81zH9VaH63tJEqp66p8KyhXSmXUIb4WUdt73NjyVfQD\nOgIvaK2dWuuVwBpgmp9eTlCThN663IRRo34PmKCUSvVVSCmVBlwO7DVXnQ9EAJ80R5A10VqvB44A\nPpsdlFKfms0zvh6f+iMG8z3rA2xvwO7TgfnarCp6HfNVpVQJ8CNwHPhMKWUFRgHtzGaBI2bTVVRt\nJ9FaL9Zax2qtYzGS135gYQPirTd//A7q+x7XUP5ZpdRppdQapdRY7+K+DgH4tZIQrCShtxJKqZ8A\nXYH3tdYbgX3ADVWKfayUKgQyMWqST5nrk4HTWmuH1/G+Mf9IS5VSY3wcx/sP+dd+fCnHgLa+Nmit\nr9Rat6nhcWVjT6yUCsP4Z/iO1vrHeu7bBbgIeMdH3HdhNGOlAx8C5UAqEIZRq0/H+LY0HPhtPc5p\nARYAGVrr1+sTbx2O/aT7eoe3xv4O6vse11D+caAHRhPVG8C/lVLubz0/Yny2H1VKhSmlLsX4vUTX\n/qqFJPTWYzrwhdb6tLm8gOrNLldrreOAsRhfTZPN9dlAslLK5i6otb5Aa93G3Fb193x1lT/kf/jx\ndXTCuAbQrMzk+C5QAdzTgEPcBHyttT7ga6P59f9rIA24Eyg1N/1Na33c/L39BZhYj3POxvhHcV9d\nd6jHReeBwNZ6xFLXc9f5Pa6pvNZ6ndksWK61fgejSWWiuc2Occ3jCozrFQ8D72N88xO1kITeCphf\n068FLlJGD5YTwIPAUKXU0Krltdb/xWhnn2Ou+haj1nhV80Tsm1JqNEZC/7qG7ct89CpxP5Y14rwK\nmItRa/65mRTq6yZ81M59sAE9tda5GEmmQeNPK6WmAtcDU6rGq5TappRaavYOecxc971S6jXgTXN5\nmtkTZINSapy57k7zm9lcoIPW+qSP8zbod1Df97ie5TVeTS1a6x+01hdprZO01hMwavPrz3Y+YbDV\nXkQ0g6sxercMxqjNuL2PkWge9rHPX4GDSqlhWuvNSqmngVfNP6TlQAkwBKPHQZNSSsUDY4AXgX9q\nrX3WDLXWNXZ1q+X4NozPqhWwKqUiAYdXE9NrQH+MXjel9dwXpdQFGP+I/lVl3xSMXhafYtTIL8ZI\nwu6msLeBe5VSywE78IBZ1r3/PPN1z6hy3OHA34BLtNZZVba1ARKBW4ACYL1S6i2gHTBTa51jXnS+\nHBiPUcNfqJTKwmiauBAYDTzr671s6O+As7zH9Slvvr5zgf8CDowL6WMw3jt3mSEYPV0swF1AB4wK\njKiNv7rLyKPuD6p0C8RIwH/2Ue5ajK+dtqr7mNtfAz7wWr4RoyZTAmQB64DbgPAq5y7F6J7nfnx0\ntvh8rfc6TiGQj/Et4W7A2gTv1yzO9DBxP2aZ27qay2VVXtONte3rdfzXgXd9nLcdRuLJw0iuW4Ff\ne20PA141t58AXgIivbZ/5V2+yutxVIl3mbktHZhT5Rg/BZ73Wvc7YDNG978MjOa53wE/NbePAv7q\nx/f/rO+xWWYZ8GRt5c339Dvzc5OH0avrkirn+xOQ635fgF4t/TcbKA9lvoFCCD9SSoVj3CswRNej\nCUgpdRfQX2t9r1LqBqA7Rj/sI1rrJWaZP2P8E/7aXLZhNL8t01p/rpT6J7BKaz3Xv69KtHaS0IVo\nRcx2cjvGRc0TGE0vrwLPaq33mmX6AG+Z5ewYzXKdgH8Ch4EU4HZtdCMVIUQSehVme+WVwCmttc++\nr0qpyzDai63Am1rr55oxRBHElFJfAhO01q6WjkUEHunlUt084LKaNpo3lLyCcVFqAHC9UmpA84Qm\nQkC4JHPRUK0yoSulblJK/aCU2qKUerc5z621/h9n70d9DrBXa71fa10BLKKFuwuK4KG1vqilYxCB\nq9YmF6VUZ2A+xoBQLuANrfWLSqlZwK8xelOAcYX7s7MdKzk5WXfr1u2s5ystLWXfvn3069cPm82G\nw+HAZmtc78pdu3bhdDo9yy6XC4vFQlpaGvHx8dXKl5eXs3fvXgYOHFhtW25uLvn5+bhfR3Z2NsXF\nxXTp0qVRMdZHcXExMTFN3huxyUj8LUvibzkNjX3jxo2ntdbtai1Yhy5LHYAR5vM4jP6hAzC6Xj1S\nny41I0eO1LV56aWX9JNPPllt/ZVXXul5/sEHH+hXXnlFa631M888o5cuXaq11vraa6/Vc+bMqfUc\nq1atOuv2AwcO6IEDB/rc9v777+tbb73Vszx//nx9zz331HpOf6ot/tZO4m9ZEn/LaWjswAZdhxxb\na9VXa30cY0AitNaFSqmdGFfUm4TWGqUqj89z8OBBvGv2kydP5uqrryYqKoq2bdsyadIkPvnkE668\n8kpWrFhR7Zjp6ekUFhZ6louKioiNjWXOnDlcfHG14S7OKi0tjczMTM/ykSNH6NixY72OIYQQTaFe\nbehKqW4YAxCtM1fdY7Z1v6WUSvRHQOPHj+f9998nO9sY7TQnJ4fvv/+eESNGeMfBpEmT2Lx5M7ff\nfjtlZWX861//Ytq0aeTn51c75urVq9m8ebPn8eabb7J58+Z6J3OA0aNHs2fPHg4cOEBFRQWLFi3i\nZz/7WcNfsBBC+Emduy0qY3ac/2JMqvChOSTmaYw7wv6AMXbELT72uw3jbkVSU1NHLlq0qNZzLV++\nnMWLF2OxWOjduzfJycmMGzeOnj17esosXryYvn37MmzYMObPn8/u3btp27Yt33//PXPnziUiIqLG\n47tr6L784Q9/YPPmzeTn55OYmMiMGTO44oor+M1vfsMjjzxCcnIya9eu5ZVXXsHlcnH55Zfzy1/+\nstbX5E9niz8QSPwtS+JvOQ2Nfdy4cRu11qNqLViXdhmMW5w/Bx6qYXs3YFttx6lLG7ovl112mb79\n9tv13XffrTMyMrTWWt9www06Pz9fHzp0SN9yyy2esrNmzdJr16496/ECuQ1Oa4m/pUn8LSuQ42/x\nNnSvUdN2aq3/4rW+gzba1wEmA9vq9z+n7pYtqz4I3HvvvQdAfHw8c+eeucP5qaeeqlZWCCFCQV36\nA16IMf3TVqXUZnPdkxg31AzDaHI5CNzeJBEKIYSok7r0cvka39NCnbXPeaszdqzxMyOjJaMQQogm\n0yrvFBVCCFF/ktCFECJISEIXQoggIQldCCGCROjMKSoXQ4UQQS5kaugul+b1/+7jZEFZS4cihBBN\nImQSeubM33N61jP8/t87WjoUIYRoEiGT0CM+X8b4feuJjQidViYhRGgJmYReZjdm9eqaHN3CkQgh\nRNMImYSuzVElLcrXTa9CCBH4Qiahu/O401W34YKFECLQhEyDsiM8kjKbxuHUIfSqhRChJGRS28Kn\nX2fu1we4z+Vq6VCEEKJJhEyTi3tiJmcdZ2gSQohAEzI19J8seo2Yw7lUXDSzpUMRQogmETIJvccP\na4nOL2OFU2roQojgFHJNLg7p5SKECFIhk9DdpNuiECJYhVxClxq6ECJYhUwbenFcArn2cJzSbVEI\nEaRCJqG/9+gLLFx/mGukhi6ECFIh0+TiHstF2tCFEMEqZGroE/75Il1OFLB9yBMtHYoQQjSJkEno\nXXZvIaqwnB+kH7oQIkiFTJOLm/RyEUIEq5BJ6O407pKxXIQQQSpkEjpyp6gQIsiFTBt6XtsUjqtS\n6YcuhAhatdbQlVKdlVKrlFI7lVLblVL3m+vbKqW+VErtMX8mNn24DffuPbN5cNIjxgQXQggRhOrS\n5OIAHtZa9wfOA+5WSg0AfgN8pbXuDXxlLrda7pYW6YcuhAhWtSZ0rfVxrfX35vNCYCfQCbgKeMcs\n9g5wdVMF6Q/XzP8T/2/FG9KGLoQIWvW6KKqU6gYMB9YBqVrr42AkfSDF38H5U6eDuxlwar/U0IUQ\nQavOF0WVUrHAB8ADWusCpVRd97sNuA0gNTWVjIyMBoTZeO3sFQDkFRRSVORssTj8oaioSOJvQRJ/\nywrk+Js69joldKVUGEYyf09r/aG5+qRSqoPW+rhSqgNwyte+Wus3gDcARo0apceOHdv4qBtgd1gY\nVFQQFR1NbKympeLwh4yMDIm/BUn8LSuQ42/q2OvSy0UBc4GdWuu/eG1aCkw3n08HPvF/eP7jbmiR\nNnQhRLCqSw39QmAasFUptdlc9yTwHPC+UupW4DDwi6YJ0T9OpnbhSF6p2YZet+YiIYQIJLUmdK31\n19ScAcf7N5ymM//mmazYeZJOTknoQojgFDK3/st46EKIYBcyCf2mec/wzPK/4ZTBuYQQQSpkxnJJ\nPXGYiJIKqaELIYJWyNTQPb1cnDI4lxAiOIVMQkfa0IUQQS50ErpJ+qELIYJVyLShH+zcm+N5ZVJD\nF0IErZBJ6O9e9yDf7MsGl/Z0YRRCiGASMk0u3nOJSjoXQgSjkEnod8+dxQv/ngOATFokhAhGIdPk\n0jY3i7AyOwAyragQIhiFTA3dm9TQhRDBKGQSuncOl44uQohgFDIJ3TulS0IXQgSjkGhD/9/uLLYl\n9fIsywBdQohgFBIJ/U+f72LrRTM8y1JDF0IEo5Bocqk6n7VcFBVCBKOQSOgAr330DK999AwgNXQh\nRHAKiSYXpRSJpQWeZamhCyGCUUjU0KvOICo1dCFEMAqNhF61DV0yuhAiCIVGQq+yLPlcCBGMQqYN\nfU3XoZ5laUMXQgSj0EjowN8uvN6zLDV0IUQwCo0mF+mHLoQIAaGR0FHMe/8p5r3/FCA1dCFEcAqN\nJhcFkY5yz7JMFC2ECEahUUOXJhchRAioNaErpd5SSp1SSm3zWjdLKXVUKbXZfExs2jAbR1XpuGiX\nGYuEEEGoLjX0ecBlPta/oLUeZj4+829Y/lW1hi5NLkKIYFRrG7rW+n9KqW5NH0rTUQq+6nmOZ9kh\nNXQhRBBqzEXRe5RSNwEbgIe11rl+isnvFIp/nHuNZ1maXIQQwUjpOszeY9bQP9VaDzKXU4HTGPO6\n/QHooLW+pYZ9bwNuA0hNTR25aNEivwReH3/6rpTt2Wey+NXdNFf3i232OPylqKiI2FiJv6VI/C0r\nkONvaOzjxo3bqLUeVWtBrXWtD6AbsK2+26o+Ro4cqVvCL99cq7/tPEh/22Ww7vr4p/rBNz9vkTj8\nZdWqVS0dQqNI/C1L4m85DY0d2KDrkGMb1G1RKdXBa3EysK2msq2BMq+Kuq+NShu6ECIY1dqGrpRa\nCIwFkpVSR4CngLFKqWEYTS4HgdubMMZGs3j1cgm3WSShCyGCUl16uVzvY/XcJoilybjzudaaCKsF\nu3RbFEIEoRC5U/RMFT1MauhCiCAVGmO5AJ/2Swcg3GrB4ZKMLoQIPqGR0BX8c8QVAHSxSZOLECI4\nhURCB0WkvQyAcFusNLkIIYJSiLShw7x/zWLev2aZTS4tHZEQQvhfaCR0r+fSbVEIEaxCI6F790O3\nWnDWYbgDIYQINKGR0L3q6Darkhq6ECIohURCd3nVyG1Wi8xYJIQISiHRy8WlYcngiwEIsyhJ6EKI\noBQyNfQlgy/mV3OfJsxqkRmLhBBBKWQS+k/aaPrZKrBZFU5pQxdCBKGQSOhOl+Y3c38HU6YQJm3o\nQoggFRIJXWs8ndHDpJeLECJIBX1Czy+18/Xe07jH47JJP3QhRJAK+oT+5ur9AJRUOADp5SKECF5B\nn9C9x0IHjDZ0aXIRQgShoO+HHm41Evo/h0/kvBtGYJPBuYQQQSroa+g2q/ESP+0/Bq67jjCrNLkI\nIYJT8Cd0c4boDgVZkJmJzWJBY3RlFEKIYBL0CT3MrKG/8OmfYdo0wmxGgrdLQ7oQIsgEfUKvKsxi\nvGRJ6EKIYBP0Cb1q4raZF0kd0pAuhAgyIZDQKydudxOM1NCFEMEm6BN6RZU+imFmDd0uF0WFEEEm\n6Puhu2viyU89ASlx2Mw2dIfU0IUQQSYkEnqEzUKvW28AIGzLMc96IYQIJkGf0CucLsKtFti1C4Do\nsDYAlFQ4WzIsIYTwu1rb0JVSbymlTimltnmta6uU+lIptcf8mdi0YTac3eki3GaB22+H228nJsL4\nH1ZU7mjhyIQQwr/qclF0HnBZlXW/Ab7SWvcGvjKXWyW7Q3t6tgDERZoJvUwSuhAiuNSa0LXW/wNy\nqqy+CnjHfP4OcLWf4/Ibu9PluTsUkBq6ECJoNbQNPVVrfRxAa31cKZVSU0Gl1G3AbQCpqalkZGQ0\n8JQNc+R4GfZyF3l5eQD8sHE9AJu27aRtwd5mjcVfioqKmv199CeJv2VJ/C2nqWNv8ouiWus3gDcA\nRo0apceOHdvUp6xkYeYGCiihTRvjYuiEn46BVcvp0KU7Y8f2atZY/CUjI4Pmfh/9SeJvWRJ/y2nq\n2Bua0E8qpTqYtfMOwCl/BuVPdqfZhv7b3wIQYbNgUVAsTS5CiCDT0IS+FJgOPGf+/MRvEfmZ3eky\n7g69+GLAmCs6yiYXRYUQwacu3RYXAt8CfZVSR5RSt2Ik8kuUUnuAS8zlVqnc4TJq6Js3Gw8gwqoo\nKpd+6EKI4FJrDV1rfX0Nm8b7OZYmYXe6iI2wwQMPGCsyMoiwQpldEroQIrgE/eBcRpNL5ZcZblWU\nSkIXQgSZ4E/oDu0ZYdEtwgolFdKGXtUtt9xCSkoKgwYNqrHM8uXL6du3L7169eK551ptS5sQISn4\nE7rTRbjNWmmdUUOXwbmqmjFjBsuXL69xu9Pp5O6772bZsmXs2LGDhQsXsmPHjmaMUAhxNkGf0Cvc\nvVy8RFihtJE19Pnz5zNkyBCGDh3KtGnTGnWs1mLMmDG0bdu2xu3r16+nV69e9OjRg/DwcKZOncon\nn7TaDk5ChJygH23R7h5t8ZlnPOvCrVBa1vA29O3btzN79mzWrFlDcnIyOTlVR0ZoPdLT0yksLKy2\nfs6cOVxsduWsq6NHj9K5c2fPclpaGuvWrWt0jEII/wiBhG7eWHTBBZ51EVZFaSOGz125ciVTpkwh\nOTkZwFOrXbx4MWvWrMHlchETE8Pzzz/P5MmTmTx5MomJiUyaNAmHw8FVV13FBx98wOOPP86zzz7L\niRMnmD17Nvn5+SxZsoSMjAx+97vfMXDgQKZOncrYsWMpLi5mzJgxPP3008TGxnL48GHuuecekpOT\n6dOnD5dccgkbNmzg9ttvrxTr6tWrG/w6q9K6+ixPSikfJYUQLSHom1zs7n7o33xjPIAIC41K6Frr\naons22+/Zd26dbz00ku8/PLL/OEPfyAzM5NOnToxZMgQtm7dCsBrr73Gr3/9a0pKSlBKER0dTY8e\nPZg7d67nWEopYmNjKSsrIy0tDYDnn3+ea6+91lNm9+7dXHHFFbz11lvs2LGDkSNH+kze6enpDBs2\nrNpjxYoV9X7daWlpZGZmepaPHDlCx44d630cIUTTCPoaeoV7tMUnnzRWZGQQblOU2B0+E3NdjB8/\nnsmTJ/Pggw+SlJRETk4O8+bNY+bMmZ4y4eHhbNy4kZEjRzJgwAD+8pe/kJOTwzfffMPChQtZuXIl\nAwYM8Hn89PR0LrroIk6ePMlDDz3EzTffzIABAygrK/OUGT58OLNnz2bx4sWeNvzIyEhOnjxJamqq\np5w/a+ijR49mz549HDhwgE6dOrFo0SIWLFjgt+MLIRon+Gvo7jZ0LxEW0Nq4i7QhBg4cyMyZM7no\noosYOnQoDz30EGVlZdhsZ/4/Op1ONm7cyIgRIwgPD6e8vJzZs2fz1FNPAZCTk+MZMKwqiznvaWJi\nIuXl5axatYq1a9eyYMEC/vGPf+ByuXj77bd5+umnWblyJf/5z3885QsKChr0mgCuv/56zj//fHbt\n2kVaWprnW8PEiRM5duwYNpuNl19+mQkTJtC/f3+uvfZaBg4c2ODzCSH8K6hr6A6nC5em2o1FEWav\nl9IKJ5FhVl+71mr69OlMnz7ds7x9+3Yefvhh2rVrR2FhIS+88ALbt2/3JPCIiAjCw8Pp168fAH36\n9PF0EczOzmbmzJls2rSJZ599lr59+/L555+Tl5fHPffc4xmdbd68eSQnJ2OxWLjsssuYNWsWCxYs\noFu3boBx0bJLly4Nej0ACxcu9Ln+s88+8zyfOHEiEydObPA5hBBNJ6gTut1pXMSrfqeo8bPU7sRf\nc+cNHDiwWkL88MMPPc/nz59fadvgwYN59dVXAUhKSuLvf/97pe3XXHNNtXPMmDEDMIbgHDRoEEuW\nLPFsKyoqIj4+noiIiEa9DiFE4ArqJpcKp9GkUr0furHckhNFK6W48cYbKSkp8cvxjh07xqOPPuqX\nYwkhAlOQ19CNhB5us8Bf/+pZ766ht/QAXenp6Q3f2T1Ivjn7SZ8+fRodjxAisIVGQrdaYNgwz/rW\nUEMXQgh/C+oml3K7u8nFAitWGA8qt6ELIUSwCOoaujthR4db4f/+z1h58cVEuBO6jLgohAgiQZ3Q\n3U0qUeHVR1uEAK+he41NI4QQEOQJ3X17f3R45ZfprqEHdBu619g0QggBQd6G7p7EIiqshhp6ACf0\n0oz/ecamEUIICPYaut13k0ug19C3ZjmIfPhBBnZMIG7t1y0djhCilQjuhF7hdVH09dc9620WRWSY\nhcIye0uF1igHC1yMBrKKyolr6WCEEK1GUCf0Eu+E3rdvpW1tosLJKwnMhO4elTyQm4yEEP4X1G3o\nlZpc/v1v42FKiAojvzQwE3pBuZHSHa7qE04IIUJXkNfQHViUeafon/9srJw0CYCE6DDyAjWhV5gJ\n3SkTXQshzgjqhF5a4SIqzOpzEouEqDAyc/wzMFZzK6jQ/H78bVgtsLSBk3QIIYJPUCd0u9NlDMzl\nQ5uoMLYGaBt6QbnmWGoPAIrKHcRFhrVwREKI1iCoE7rD5cJm9Z3QA7kNvcgOYzO3YHdqcovHSUIX\nQgBBntArHLra9HNubaLDKLU7KXc4ibA1bNaillLi0Dy47n1KK5zklNxNl6Tolg5JCNEKNCqhK6UO\nAoWAE3BorUf5Iyh/MWroZvvyu+9W2pYQHQ5AfqmdlLjASehldicOF0TarJRWOMktqWjpkIQQrYQ/\naujjtNan/XAcv3M49Znp5zp3rrQtIcpopigotZMSF9ncoTVYgXkzlHsu1NxiSehCCENQ90OvcLqw\nWcwa+uLFxsPUxkzogXZzUUGpMT5NRJjxq8uRhC6EMDW2hq6BL5RSGnhda/1G1QJKqduA2wBSU1PJ\nMKdMaw4ns8ooL9dkZGQw7NlnAdicmkpRURGndv4AwNfrv6foYOBcStibZ9ws5SgvBWDLzr1kOA+3\nZEj1VlRU1KyfA3+T+FtWIMff1LE3NpNdqLU+ppRKAb5USv2otf6fdwEzyb8BMGrUKD3WPRdmM3hz\n7zpsFQ7Gjr0Q2rQBYOzYsWRkZDB48Gh+/20GaT37MXZkWrPF1Gi7TsHa78j72z/40yfb6NuuA2PH\nDm7pqOolIyOD5vwc+JvE37ICOf6mjr1RTS5a62Pmz1PAR8A5/gjKX+xO15k29CpS4yNRCo7mljZz\nVI1TUGY0uUQOGkB+157Shi6E8GhwQldKxSil4tzPgUuBbf4KzB/OltAjw6x0iI/kUHZxM0fVOO6+\n88mrPueSveull4sQwqMxNfRU4Gul1BZgPfAfrfVy/4TlHw6XJsxa823xXZKi+XDTUU4WlDVjVI2T\nZ9bI41/9G1evWkRucWBd1BVCNJ0Gt6FrrfcDQ/0Yi99VOLzuFF2ypNr283oksXZ/Dlsy87h0YPtm\njq5hckoqiLSCRUGYxUKO1NCFEKag7rbocHndKZqcbDy8/HyEcTE0kIYAyC2uIC7c+NZhsyrySirQ\nummG0S0vL+fUqVOcOnWKoqKiRpcTQjStoE3oFQ4Xe08VnblTdN484+El3hwDJaASeomdWE9Ct2B3\naorKHU1yrry8PJKSkmjXrh2lpaU4HL7PU9dyQoimFbQJ/aH3NwNeydpHQo+LtKHUmZ4jzaWkpISs\nrCyysrLIzc2t1765JRXEhZkJ3bxpqina0SsqKrDZbFitxvDDUVFRlJVVv9ZQ13JCiKYXtAn90x+O\nA2e/E9RiUWgNL321h/+Y5Zuaw+GgqKjIU6NNSEio1/45xRXEhAPvvsvuP70CUO+eLqdPn/b8Q/F+\nlJeXe8o4nU6s1jNj3FitVpzO6lPe1bVcS7vllltISUlh0KBBZy23fPly+vbtS69evXjuueeaKToh\n/CNoE7pbQR0ngr57wfccz2+s13OlAAAgAElEQVT6Punl5eVERkZisRhvvftnTk6Op0xNtXatNVnZ\nOUYNvXNnont1N/YtqSAvL6/OMSQnJ9OuXbtqj4iIiIa+rFZvxowZLF9+9k5YTqeTu+++m2XLlrFj\nxw4WLlzIjh07milCIRovYBL6+gM5uBowh6Z77JO6OF3YPD1Gqs4wVLWW677IWVhYSH5+PoWFhQAc\nzy+l1O4iJQqy/vEPEj5ejLO0kJyichwOBwUFBZ5/DFX39VaXGvrChQu54IILGDp0KNOmTasWo1vV\nGnlN5VramDFjaNu27VnLrF+/nl69etGjRw/Cw8OZOnUqn3zySTNFKETjBcQgJt8dzOHa17/lkUv7\ncM9Pe9da3rvXR2019L9eN4wHFhvt7ScLyhhM/ZpA6isiIoKcnBxiYmKwWCy4XC7sdjt2u538/Hy0\n1pWSpMVioaLC+Eez7XAOWKx0iHIRNXc+bZUFNfJuMk8XENk3kdjYWHJzc33u6y25Sm+fqrZv387z\nzz/PRx99RN++fcnLy6O0tJTExMRqZcPDw3E4HDidTiwWS43lmkp6errPf1pz5szh4osvrtexjh49\nSmevUTnT0tJYt25do2MUorkEREI/XWjUHL/dn809P+3N4u8Os/N4IU9NGuBzPs3iijM1xvH9Uown\nn33m89hXD+/kSeinCst9lvEnm81GbGwsp0+fRilFWFgYVquVhIQEwsLCKCsrw+VyUVhYSEJCAi6X\ny5Ogtx3JRllspEa6CFMKq0WR1iaSbZk5REb2BYzav69962PlypVMmTKFHj16kJ2djdaa6OhoPvjg\nA9asWYPL5cJisfDCCy8wZcoUFi5cyKRJk3j77be59dZb+fTTT7n//vt59tlnOXHiBLNnzyY/P58l\nS5awY8cOZs2aRf/+/Rk/fjxTpkyhuLiYu+66i/DwcMaOHcvw4cOZNWsWSUlJnjIbN25kw4YN3H77\n7ZViXb16deN/KSZf3T9lvlYRSAIiobtvnjlZUM7GQ7k8/sFWAC4dmMoFPc/UNvNL7Hyy5SiXDEgF\n4IGLe3PHRT2NjdE1z+rz4tRh3L9oMyeaoQ3dCCWaaK94cnJyiI2NBcButxMZGYnL5aKoqAiXy0VY\nmNG98sdj+aQlxRFhKabM5aLM5WJI1xR+OJSFzWbzJFqLxVJt3/rQ5sTTkZGRREYaY8V/++23rFu3\njpdeegkwerccO3aMTp06kZWVRY8ePaioqKBbt27k5OSglCI6OpoePXowd+5cpkyZAsCyZcu45ppr\nuO+++/jZz37GlClT+PDDD5kyZQqTJk3iuuuu48SJE9x7772kp6d7yowcOZIXXnihWkL3Zw09LS2N\nzMxMz/KRI0fo2LFjvY4hREsKiIR+qsCoOe/LKuLnr33jWX/Ea2Atp0sz9PdfABBhTgzds12sZyII\nXn3V+HnXXdWOf9WwTry4Yg87jldPDM3Bu203Li4OwGciPlCg6d8hDigmwWb86gZ3TWb5j9nkl9pJ\niAojPj6+0fGMHz+eyZMn8+CDD5KUlEROTg7z5s1j5syZnjLh4eFs3LiRkSNHVvu5efNmBgwY4PPY\n06ZN47bbbiMzM5Ps7GzASJyDBxsjRlqtVqZNm8bTTz/N0qVLPWUAIiMjOXnyJKmpqZ51/qyhjx49\nmj179nDgwAE6derEokWLWLBggd+OL0RTC4iLoqcKjX7NVb8RP7bkB47lGUn9g++PeNZnmU0ncZFe\n/6/ef9941GBY5zas2HmSVbtO+Slq/yqtcLL/dDH92sd7kj7AoE5Gm/+OYwV+O9fAgQOZOXMmF110\nEUOHDuWhhx6irKwMm+3M++l0Otm4cSMjRoyo9jMnJ4c25nDFVaWkpPDAAw/w3HPPedry09LSOHLE\n+P25XC5SUlJ45ZVXKpUBSExMpKCgYa/z+uuv5/zzz2fXrl2kpaUxd+5cACZOnMixY8cAozns5Zdf\nZsKECfTv359rr72WgQMHNuh8IrDVpZtra+ziGhAJ/bdXDOCLB8dg8dGcueukUavO8mr/zswxkny7\nuLp3w5txYTcAVu/2z2x6+SV2MnNK/HIsgDV7T+N0ac7tYdbmlyyBJUsY2NGokW87mu97x9OnjUc9\nTZ8+nW3btrFlyxbmzZvHY489xsMPP8x9993HzTffTGFhIdu3b2fgwIHVfvbp04eDBw8CkJ2dzR13\n3MGmTZt49tlnOXjwIHPmzOGmm27i0UcfBeCaa67hgw8+4M4772TSpEkcPHiQ2267rVIZMC5adunS\npd6vBYxeO8ePH8dut3PkyBFuvfVWAD777LNKzSoTJ05k9+7d7Nu3r9I3EhFaauvm2lq7uAZEk0tM\nhI0+qXGkxEVyosrIiO6xWiocLs+6NftOE26z0DsljroaktaGrknRvLXmAL8e050OCVENjnfvKaNp\nKL/UjlLw2X3p9O/QuKaQL3ecJC7Cxrndk/jmKJ5xaZKB3imx/PuHY/wqvXuTXcQbOHAgCxcurLTu\nww8/9Plz8ODBvGo2cSUlJfH3v/+90n6PPPJIpUH+Y2JiePvttyuVeeONypNfFRUVER8fH9R95QPJ\n/PnzmTNnDkophgwZwrtVJmEPdGPGjPFUSnzx7uIKeLq41tTU2FwCoobulhJf/Y/Z7nTx/oZMXvxq\nj2fdkdxS+qTGEm6r38tzmv3cv9h+EoDVe7I4nF2/WnZxuYNrX//WM+SA1vDPtYfqdQxfcX3140nG\n9ks585q8hjK46fyu/HAkn02Zdb+5qCkppbjxxhspKfHfN5Rjx45Vqq2LlrN9+3Zmz57NypUr2bJl\nCy+++GJLh1Qn6enpDBs2rNpjxYoV9T6Wry6uR48e9We4DRJQCd09mJa3zNxSHlvyQ7X1ndrUv4b9\n+rSRAJTZnRSW2Zk2dz1j/rSKg6frPgnGsm0nyCmuYP4t57DvmYmM7JrI3lONG4Fw5/ECThdVnOmC\nCZUS+jUj0oiLsLF4fabP/VtCenp6pZ489ZaZaTxMffr0oWfPnn6ITDSWu1ur+/qG+6J+ly5dWLp0\nKQDPPPMM48aNq/EYkydPrvTz/vvvZ8GCBfz617/mqquu4osvjA4Oq1ev5o477uBXv/oVF1xwgafb\n65133skSc0hsl8vFzJkzuffee3nnnXfYuHEjr7/+erVzrl69ms2bN1d71Lc3FLTeLq4B0eTiluKj\nTfx3H/ueJKlak0kdJmYd0CGeMKsit8TO4FlfeNaPnZNB/w7xvDh1GH1Sa27GySmu4Pf/3k5ybDjp\nvZNRStGrXSwrdp6s9dxn4655j+zq+4admAgbAzrGc8DXP55a7o5stbxqP6J1cXdr9ZaZmckFF1zA\n1q1bGTJkCPv27WP48OGUl5fzwAMPkJiYyJo1a/j4448pKiqiU6dOZGZm0qlTJ0831xtuuIEbbriB\n3NxcHnnkES699FLS09NJT0/n448/ZvTo0T67vX7yySccPXqUtm3bkpaWFtJdXAMqoQ/vmsiHm47S\nr30c+7KKsDtrHgrA1YAxwpVSJESFV0rAS+44ny92nOSN/+3n5ZV7eXRCX178ag8JUWGc270tHdtE\neXqa/OHTHRSUOeiWFO35wA/pnMDiDZl8dzCH0d0allw3HcolKSactMSav3V0SIhk42EfY8BYAupL\n2BmLFxs/r7uuZeMQ1fjq1rpx40YmTZrEunXrmDNnDmPHjsVisfDqq69y8803c8455zB58mQSExP5\n73//e9Zurv/3f//H3XffXemcCxYs4M0336SsrKxSt9fy8nI2btzIwIEDufPOO5kxYwbjx4+v1MW1\nvLyc/Px8PvjgA6Kjoz33fDRGa+3iGlB/7deP7sxvr+jPB3dewKb/d2m17cmx4fz4h8u48dwu3Hxh\n9wadIzE6jL2niogJt7L2ifGM6taWJyf2Z3y/FJZuOUb6H1exZOMR5n59gNve3ciVf/saMGotm8yE\n+uiEfp7jTR7eiXZxEdz13vee7pf1kVdSwbJtJxjXL+WsX+naJ0RxIr+s+ng3JSXGA9icmcfcrw9Q\nZm99oyFW89prxkO0Or66tbqT89GjRzn//PPZvXs3I0eOZMuWLQwZMoSioiLatzdmBaupm6vWmscf\nf5zLL7+cESNGeM53+PBhEhISiI+Pr9btNS8vjz59+tClSxdKS0s9fyPeXVwbMl5/bd1cW2sX14Cq\nodusFn6VblxVrmmWnsgwK7MnD27wOfaY7d3Xje5C+4RIz/rfXTmATZl55JhzevZOifWUnfLaN1zU\npx0Hs0v448+HcMWQDp79osNtvHvrOfzs5TW8lrGPpybV75f+9d7TlNqd3HDu2bvrdWwTid2pyS6u\nqNxd031hMjqa3/97O98fziO/pIKHLu1brzhE6CopKaG42GjOs9lsJCYmMn36dKZPn+4pc91119Gn\nTx8WLVpEWFiYZ3nChAnceuutxMfHM3z4cMC4qPrUU0/x9NNP89RTT6GUYvny5fztb39jxYoV5Ofn\ns3fvXu644w4A5s6dy8033wzg6fb6j3/8gwceeACbzcaUKVO49957WblyJeeffz5wpour93j9gGe8\n/tpq6VV7dLl95jWEyMSJE5k4cWJD3tImE1AJ3Zv7P3G3pGh+c3l/7vjnRr8c9xcj0/ho01EenVA5\n4XVLjuH7313Csq3HCbdZGNWtLUOfNtrZNxzKZYPZLPKLUWnVjtmvfTyjuyXy311Z6CuN9keXS7P5\nSB6DOyUQZq3+RenjTUcZ0DGe7w/lEWGzMLhTlUHDqoxN0z7e+Ofz5y928dzPh1Q7XlG5gx+OGH3V\nP9p8lAcv6dMqLuKI1s09fn9ycrJnMDlfFptNZO7hoN3Ldrudjh07orXmpptuwuVy1djN9e9//zv3\n3Xcfp0+fNoaKzsoC4J577gGMoae7devm6fZaWlpKeXk50dHRzJ07l9LSUioqKip1cS0tLa02Xr+v\nAeuCRcAmdIAVD42hXVwk3x+q36w/Z/P8z4fwf5MHEWHzPQTs5YPP1L5v/Ul35n59gPH9Uth/uphp\n53WtMUme2z2Jv3y5m+5PfEbf1Dh+0juZuV8f4JrhnfjLdcMoszuZ/Z+dXNgrmehwKw8s3kxSTDhx\nkTaGprWpnvSr9CC5sFcy6b2TWbwhk1+MSqOgzMG4vmd6xSzZkInDpZk6ujOLvsvk+8O5jOzaei+Y\nakD+3bS8s43f7+7dkpubS2JiInl5edXuEL7pppsqLfsq493NNTo6utbRQGsTyl1cAzqh9zJvHDrT\n37zxKcBiUURY6jae98OX9uGc7m2ZMLB9rWVnXNiNl77ag8Ol2XWy0HOH66dbj/OLUZ0pczh5d+0h\n3vXqs55dXEFOSQVP/cxHM02VsWliImz87frhpD+/ip+/9i0Ad4/ryV2D2uDSmhe/2sMFPZP47ZUD\n+HjzUT7edMxnQl+9J4vSCieX1uE1NZVdJwopPJRLl7bRpNReXDSxuozfr7XG4XBQWFiIw+EgMTER\nrTWFhYWe5tHIyEhPjT82NpbCwkLPgHLp6eme47lr6FVVvbHMarXivOIKCAuDjAxPXH369KlcJgDG\n6/eXgE7obmcSev17tjRGdLitTskcjD70n92fzu3vbvR0L7xqWEc+2XyM6/+x1lNu1qQBHMop4dfp\nPVh/IIfBaQn0bOejvc89Lo3XYGNtosO5a1wvnl/+IwCvrNrHgv8YzSy50Qk8ObE/sRE2Lu6fyrtr\nD5FXaqdPSiw/7Z/C0s3H2HmikP/tNr7mfnrvTzy9d5rL/qwipr+9nsycUhInPQ7AeqfLZ5OUaB51\nHb/fbrcTFRVFTEwMeXl5uFwuSktL0VpjsVhwOBxYLBZPmbON2V/XGnp4eDgOrXFqjUVrn2Pxt/R4\n/c0tKBJ6RD3vCG0pfVLjWPXIWMC4+9Oi4PJBHfjtx9s4XVTOkLQEZnj1zrl6eKd6n+OOi3qQHBvO\n+T2T+GTzMf5kJvdrRnTyJOipo7vw6Q/H+fcWY1CqP3+5u9Ixwm0WHn5/Cx/cdQGxEZU/Ir76IPtD\nhcPF4u8yPePw5EYbse46Udjs/1jEGXUdv99ut3sGb3MncbvdTkJCgufzUlJS4hlFtLFj9rsl2Gxk\n2+3oU6eIjo72xJCdnU2bNm08sXqP6+89yFywCYpX5s8ml+ZiNUcau2xQewZ0iOeXc9fx/65s/DgQ\nSil+Mcq4Kefucb24a2xP9mUVkZZ4ps39J72TWf3YOJ7+93aUUhzOLuG60Z35bOtxLhvUnj6pcdw8\n7zt+/c4G5t0ymtW7T5MYE0ZhmYPffLCVv1w7lBFdE7FalN9qz48t2cLHm4/Rs10M6b3b8dO1n7F0\nyzE2Zw6ShN7C6jJ+f3FxMU6nk7KyMk/ZyMhI8vLysFqtREREYLFYKCkpwWKxYLPZGjVmv1vkhx8S\nCZ6xjdySkpLOlPEa1z/YBUdCD/Cv5F2SovnfYzXfJt0YqqSEXjEWCKvcbti5bTRvTh9dad0tPznz\n7eCPPx/Cw//aQt/fnhlxrk10GHkldm56az2RYVaGd2nDOzefg8XXMJi1WLDuMAezi3lsQl/2ny7m\n483Gt4W7xvbi5yPT0H+5m5hDufxxy7Vcf04Xzz9A0fJ8jd/va7hkX4nUveyPm3uAaok81AVHQg+Q\nJpcWUWpOAhITU6/drhnRiYXrD7PhUC5xETYiwqwkx4bzpylDeW/dITJ2ZbF6z2kefH8zCVFh9Gsf\nz/xvDzL/1nNIiTP+aLOLylm8IZMbz+1KQpRRC9Nak13q4snlxqxTn245xrH8MuIibHz+4Bg6mmPw\nKIx/OusO5PCXL3dVullLCA9zPCNmzGjJKFqNoEjoIXnRrA5j0zSGUor5t55Dmd1F25jwStsuGZBq\n9Ct+az2fmDVrt4kvfk1ybDhD09rw/sZMtIZNh/N4bEJfnl++i+8O5nhGogQ4ll9Geu9k7h/f25PM\n3VLiIpg6ujOvrNpHmd3FoxP6npmBSgiQhF5FoxK6Uuoy4EXACryptW6RaTvcX8Zt8rXcr6LDbUSH\n+96mlGLu9NFsPZpPbnEFa/dnk1dqZ+WPp/jxRCE/niikbUw4P+mVzNItx/hyx5nxcS7saOPRyedS\nbneSHBfhuxeP6Q9XD0IpmPv1AX48UcAtF3ZnfP/UGsuLAOa+aake4w/lllRwOKeEzd8c5Kbza74P\nJFQ0OKErpazAK8AlwBHgO6XUUq11s0/b0S4ugtvH9ODnI6vfpSmaTrjN4hkB8uIBZ5LsjycKuG/h\nJp69ZghD0hLILi4nt9jOb6/sz7aj+XS1H2ZYZ99T1FUVZrXw7DVD2HGsgDV7s1mzN5s/ThnCtaM6\n43JpHC4tTW7Bop4DyS3fdoI2JwqxWhRPLd1OdlF5yA9p0Zga+jnAXq31fgCl1CLgKqDZE7pSiicm\n9m/u04oa9GsfzxcPXuRZfu9X53meX9AzmYyMOozbXmVogycm9udPn++ioNTOzI+2smTjEXYeL6DM\n7sRmsRAZZqFNdDjj+qbQr0McvxiZVq22tjkzj14psYRZFcu2nsClNRE2KxMHtyevxE5spC00m+9a\nC69xh2rznx+O8+iSLSw2h46+dlQaL63cS6/UOH42tOWHsW0pqqZBrmrdUakpwGVa61+Zy9OAc7XW\n91QpdxtwG0BqaurIRYsWNS5iP3DfqRaoQjn+3DIX7+6oILdM0zXBgsJocsst12w6daY/c/cEC/nl\nmpgwRYQVyp2QWeh7HJJ2UYqsUo0CerWxcFn3MBIjFCnRFmLCqt8p2Zrf/+9POqhwwaAkK6dLXXSJ\nt2AJkPjD8o2b4OwJZ++muu5wEa/tUPRIsLB44ROEWWDjX15g9royskpczLogiqSo1vmPuaHv/bhx\n4zZqrUfVVq4xCf0XwIQqCf0crfW9Ne0zatQovWHDhgadz58yMjIqzWkZaEIi/ipDG9RFZk4Jz3y2\nE6XghyP5DOgQT4XTxc7jBZwqLOcXI9PYcCiX/VnF9O8Qzy/P68Kek0VsO5pP/w7xlYZdcGsXF8F1\nozrTKTGKxOgwsgrL+faH3cy4dBQdEiKJibBRXO4gLTEKpRS5xRXERtrILalg76kiyh0uImwWRndr\ni82iWH8gh54pxvSI4VaLMefs1uOc2z2JnOIK3lpzgB3HCujfIZ79WUXccVFPMnZlUVzhYNp5XYkO\nt9G/QxxlDhd5JRUczy+jR3IMGw/lsv1YAS+v2gsYXXlL7U4u7p9K+4QI7h/fh3ZxEThdmkff/pJx\nowaRU1zB7pOFRNisnNM9kXH9UrAqhc1qobTCSbjN0rzdRd2Tmfvoiuh0aRasP8za/dn8b+dxOraN\nY+m9FxJRYU4OHx3N7pOFXPXyGqwWxWu/HEF673bNF3sdNfRvVylVp4TemCaXI4D3tDJpwLEaygpR\nPz6GNqhN57bRvPbLkdXWlzucnMwvp0uS8VXe4XShlKqWrKad35XcYmP8+d6psZTZXSzbetyTJL19\n9vq3lZZ7tIshv8ROQZmdhKgwThdVvp19dLdEImxWvt57mrgIG4XlDqwW5Um8sRE2ImwW7E4XfVLj\n+GjTUcJtFu5873tsFkV0uJVPfzgOQJhVVZrcRSlj7lowZrU6VVhGfGQYwzq34b11hwFjntwLeiZx\nOKeE7w/b+XDPJsC4t6Dc7uKtNQdQChKjwxnZNZFVP56iR7sYRndrS9uYcFLjIxnZNZG+qXFYLMbN\naDERVpJijbFV9p4qpHPb6BoHtWsIu9PF8m0niLBZWLHzJO9vOELntlF0irPwwtRhxrlsZ5pn+qTG\nsfwBY3iNu/75Pb+Z2I9LB7SvPJx0kGtMQv8O6K2U6g4cBaYCN/glKiH8KMJm9SRzMMbV98U9veC5\nPc7cZXjLhd0oqXCSW1LB6aIK2sdHsnL1N3xbmEj35Bhiwq1kFZbz3rrDDElLoNTuZPuxAtrHRzIk\nLYFzeySx+LvDfHcwl/hIG/eP782KnSf5Sa9kwm0WisudjOjahvfWHub7w7l8cOcFDOqUwIHTxSRG\nh7HuQA69U2JJjY/kyx0ncWnNzuMFJMVGkBgdRnxkGOsP5pDeO5lzuicRG2Hz1K4VcNWwTtisij8t\n38Xa/TnER9mY3CuMCecNoVdKLL1SYnG6NP/ZepwfjxewP6uYHccLmDi4Awezi1m+7QS5JRW4501p\nEx1G+/hIfjxRSKc2UZ6L4ku3HKN7cgwPXtKH/JIKcortnNujLcmxEfRsF0NxhZMKR/UusFVVOFzs\nO17Apz8cY9nWE+z3mlbx9jE9eGJifzIyjCkhgWrf5LomxTB3xmhmvLWemR9t47cfb6NjQhQ3X9iN\nWy7s3qCb4AJJg5tcAJRSE4G/YnRbfEtrPfts5aXJxT9CIn739ibub98QvuJ3ubQnWZTZnZX6yzuc\nLjZl5jGwYzzR4b7rUFprisodxPmYCN3f6vv5cbo0x/JKWX8gh7X7s8kprqB3ahzvrT2Exhhr//we\nSZwsKKuUgN0So8Mos7uwWRTpfZLZn1VMj3YxJESFkV9qx+nSOF2arANHOZJbQnZUAlaLYmSXRG75\nSTfaJ0SREBVG9+SY6vHX8DnRWvPjiUJW/niKb/adZs3ebLq0jSa/1E6/9nEUlDn4ab92PHJp32rX\nSNw5UWvYl1VESYWTIWkJfukS2ZqbXNBafwZ8VmtBIYKcd82v6s1PNqul1vlklVLNkswbwmpRdG4b\nTee20ZW6Bv8qvTtRYVZyS4xvLk6t+XZfNr1SYomNsLEpM4+T+WVsOZJPmFWxdn82/92Vxbk9kth0\nOI9yh4ukmHAsSmGxKLr2SOOi9nF0TYohvXcyKfENH39FKUX/DvH07xDPXWN78ukPx3l/QyaJ0eFs\nNOdPeGXVPnadKKR9QiTxkcb1kf2nizl4uhiNMaXl7pPGrGTdkqJpGxNOXomdtLbRpPdKprDcwWUD\n2zOgY3yj3l9/Coo7RYUQzS/ZbD+PMUfktAFjvSZVcU+wMvUcY7nc4cTu1NVG8GxqSikmDe3IJK/u\njC6X5k9f7GLp5mNsPJRLYZmDqHArgzomcFGfdhSU2TlRUMYzkwdjUbBi50lKKpz0bR/J7pOFzP5s\nJwB/z9jHGPNbx/AuiTx7zeAWvS9CErponVphU4tonAiblbPmcnPe0vqOO9QQFovi8cv68fhlxhhB\n5Q4nFlXz6KFTzzkzp6/WmqN5pYRZLfz5i11szswjOS6CD74/QmSYpVFzGjeWJHQhROvQwIHk/KE+\nvXOUUp7hqP84Zahn/W3zN/DNvmy/x1YfktCFEIGrFX2T65Max1c/nqLC4WqxZpfWeTuVEEIEmO7J\nMThdmszckhaLQRK6EEL4QY92RlPR/qzqXTebiyR0IYTwgx7JxhgtB04XtVgM0oYuhGgdAnw6uYTo\nMJJiwlu0hi4JXQgh/KRHuxi+2HGSo3nrqm17pBnGapcmFyGE8JPrRneha1I0ReWOag9nI4ZZqSup\noQshhJ9MGZnGlLPMnJaxv2nPLzV0IYQIEpLQhRAiSEhCF0KIICEJXQghgoQkdCGECBKS0IUQIkhI\nQhdCiCAhCV0IIYKEJHQhhAgSktCFECJISEIXQoggIQldCCGChCR0IYQIEko3w5COnpMplQUcarYT\n1iwZON3SQTSCxN+yJP6WFcjxNzT2rlrrdrUVataE3loopTZorUe1dBwNJfG3LIm/ZQVy/E0duzS5\nCCFEkJCELoQQQSJUE/obLR1AI0n8LUvib1mBHH+Txh6SbehCCBGMQrWGLoQQQUcSuhBCBImgSuhK\nKatSapNS6lNzubtSap1Sao9SarFSKtxcH2Eu7zW3d/M6xhPm+l1KqQnNGPtBpdRWpdRmpdQGc11b\npdSXZvxfKqUSzfVKKfWSGecPSqkRXseZbpbfo5Sa3ozxt1FKLVFK/aiU2qmUOj9Q4ldK9TXfd/ej\nQCn1QKDEb573QaXUdqXUNqXUQqVUZIB9/u83Y9+ulHrAXNdq33+l1FtKqVNKqW1e6/wWr1JqpJkP\n9pr7qjoFprUOmgfwEBx3rf4AAAQWSURBVLAA+NRcfh+Yaj7/O3Cn+fwu4O/m86nAYvP5AGALEAF0\nB/YB1maK/SCQXGXdH4HfmM9/AzxvPp8ILAMUcB6wzlzfFthv/kw0nyc2U/zvAL8yn4cDbQIpfq/X\nYQVOAF0DJX6gE3AAiPL63M8IlM8/MAjYBkQDNmAF0Ls1v//AGGAEsM1rnd/iBdYD55v7LAMur1Nc\nzfnH0sQfijTgK+CnwKfmG3EasJnbzwc+N59/DpxvPreZ5RTwBPCE1zE95Zoh/oNUT+i7gA7m8w7A\nLvP568D1VcsB1wOve62vVK4JY483E4oKxPirxHwpsCaQ4sdI6JlmYrCZn/8JgfL5B34BvOm1/Dvg\nsdb+/gPdqJzQ/RKvue1Hr/WVyp3tEUxNLn/F+BC4zOUkIE9r7TCXj2B88OHMHwDm9nyzvGe9j32a\nmga+UEptVErdZq5L1VofN+M8DqSY62uKs6Xi7wFkAW8ro8nrTaVUDIETv7epwELzeUDEr7U+CswB\nDgPHMT7PGwmcz/82YIxSKkkpFY1Ro+1MgLz/XvwVbyfzedX1tQqKhK6UuhI4pbXe6L3aR1Fdy7az\n7dPULtRajwAuB+5WSo05S9nWFr8N4+vna1rr4UAxxlfOmrS2+AEw25h/BvyrtqI+1rVY/GZb7VUY\nzSQdgRiMz1FNsbSq+LXWO4HngS+B5RjNPo6z7NKq4q+D+sbb4NcRFAkduBD4mVLqILAIo9nlr0Ab\npZTNLJMGHDOfH8GoAWBuTwByvNf72KdJaa2PmT9PAR8B5wAnlVIdzDg7AKeqxl8lzpaK/whwRGu9\nzlxegpHgAyV+t8uB77XWJ83lQIn/YuCA1jpLa20HPgQuILA+/3O11iO01mPMWPYQOO+/m7/iPWI+\nr7q+VkGR0LXWT2it07TW3TC+Mq/UWt8IrAKmmMWmA5+Yz5eay5jbV2qjsWopMNXsBdAd48LM+qaO\nXykVo5SKcz/HaMfdViXOqvHfZF49Pw/IN7/ifQ5cqpRKNGttl5rrmpTW+gSQqZTqa64aD+wIlPi9\nXM+Z5hZ3nIEQ/2HgPKVUtNkbwv3+B8TnH0AplWL+7AJcg/F7CJT3380v8ZrbCpVS55m/z5u8jnV2\nTX3Bo7kfwFjO9HLpgfGB3IvxNTrCXB9pLu81t/fw2n8mxtX9XdTxyrIfYu6B8TVzC7AdmGmuT8K4\n0LvH/NnWXK+AV8w4twKjvI51i/m69gI3N+P7PgzYAPwAfIxx1T6Q4o8GsoEEr3WBFP/TwI8YFYF3\nMXqqBMTn3zzvaox/QluA8a39/cf4h3McsGPUqG/1Z7zAKPN3uQ94mSodDmp6yK3/QggRJIKiyUUI\nIYQkdCGECBqS0IUQIkhIQhdCiCAhCV0IIYKEJHQhhAgSktCFECJI/H9oSLye+1SJZQAAAABJRU5E\nrkJggg==\n", 178 | "text/plain": [ 179 | "
" 180 | ] 181 | }, 182 | "metadata": {}, 183 | "output_type": "display_data" 184 | } 185 | ], 186 | "source": [ 187 | "ival = 900\n", 188 | "p=model.predict(X_val[ival:ival+1,:,None])\n", 189 | "c_line, z_line, zbest, c_line_bal, z_line_bal = process_preds(p, lines, lines_bal)\n", 190 | "plt.plot(wave, X_val[ival])\n", 191 | "plt.title(r'TARGETID = {}, z$_{{pred}}$ = {}'.format(tids_val[ival],round(zbest[0],3)))\n", 192 | "m = X_val[ival].min()\n", 193 | "M = X_val[ival].max()\n", 194 | "plt.grid()\n", 195 | "plt.ylim(m-2,M+2)\n", 196 | "for il,l in enumerate(lines):\n", 197 | " lam = absorber_IGM[l]*(1+z_line[il])\n", 198 | " w = abs(wave-lam)<100\n", 199 | " m = X_val[ival,w].min()-1\n", 200 | " M = X_val[ival,w].max()+1\n", 201 | " plt.plot([lam,lam], [m,M],'r--', alpha=0.1+0.9*c_line[il])\n", 202 | " plt.text(lam,M+0.5,'c$_{{{}}}={}$'.format(l,round(c_line[il,0],3)),\n", 203 | " horizontalalignment='center',alpha=0.1+0.9*c_line[il])\n", 204 | " " 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": {}, 210 | "source": [ 211 | "#### Quality assesment\n", 212 | "\n", 213 | "The next cells assume that simulated spectra are all quasars and there are no BALs in the sample. \n", 214 | "\n", 215 | "QuasarNET calls a spectrum a quasar if there are more than `nlines` detected with a confidence greater than `c_th`, and similarly for BAL quasars." 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": 6, 221 | "metadata": {}, 222 | "outputs": [], 223 | "source": [ 224 | "## minimum number of lines required for quasar\n", 225 | "nlines = 1\n", 226 | "## line detection threshold\n", 227 | "c_th = 0.8\n", 228 | "\n", 229 | "## same thing for BAL\n", 230 | "nlines_bal = 1\n", 231 | "c_th_bal = 0.5" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": 7, 237 | "metadata": {}, 238 | "outputs": [ 239 | { 240 | "name": "stdout", 241 | "output_type": "stream", 242 | "text": [ 243 | "INFO: nspec = 1021, nboxes=13\n", 244 | "INFO: found 1021 QSOs out of 1021 spectra\n", 245 | "INFO: found 0 BAL out of 1021 spectra\n" 246 | ] 247 | } 248 | ], 249 | "source": [ 250 | "p=model.predict(X_val[:,:,None])\n", 251 | "c_line, z_line, zbest, c_line_bal, z_line_bal = process_preds(p, lines, lines_bal)\n", 252 | "isqso = (c_line>c_th).sum(axis=0)>=nlines\n", 253 | "isbal = isqso & ((c_line_bal>c_th_bal).sum(axis=0)>=nlines_bal)\n", 254 | "nqso = isqso.sum()\n", 255 | "nbal = isbal.sum()\n", 256 | "\n", 257 | "print('INFO: found {} QSOs out of {} spectra'.format(nqso, nspec))\n", 258 | "print('INFO: found {} BAL out of {} spectra'.format(nbal, nspec))" 259 | ] 260 | }, 261 | { 262 | "cell_type": "code", 263 | "execution_count": 8, 264 | "metadata": {}, 265 | "outputs": [ 266 | { 267 | "data": { 268 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAEABJREFUeJzt3X+sZGV9x/H3xwVExRZwr3YDrheT\njRGbKnZDqDStokYKCphqgrFmbTCbWms1Nm3RJjba/oFpUk3TNpaK6dpYfhS1UNRWwo/Y1rJ2UZQf\nW11EagnEXUVUakOz+O0fcy4M1707c++dMzP73Pcrmdxzzpxz58vDuZ995nnmnElVIUk68j1p1gVI\nkibDQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ14qhpvtjmzZtrcXFxmi8pSUe8\nW2+99TtVtTBqv6kG+uLiInv27JnmS0rSES/Jf42zn0MuktQIA12SGmGgS1IjDHRJaoSBLkmNMNAl\nqREGuiQ1wkCXpEYY6JLUiKleKbpRLF786ceW773k3BlWImkjsYcuSY0w0CWpEQa6JDXCQJekRhjo\nktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqRFjB3qSTUm+nOS6bv2UJLuT7EtyZZJj+itTkjTKanro\n7wD2Dq1/APhgVW0DvgdcNMnCJEmrM1agJzkZOBf4SLce4Czg6m6XXcAFfRQoSRrPuD30DwG/B/y4\nW38G8FBVHezW7wNOOtSBSXYm2ZNkz4EDB9ZVrCRpZSMDPcmrgf1Vdevw5kPsWoc6vqourartVbV9\nYWFhjWVKkkYZ537oZwLnJTkHOBb4KQY99uOTHNX10k8G7u+vTEnSKCN76FX17qo6uaoWgQuBG6vq\njcBNwOu63XYA1/RWpSRppPV8Dv33gXcluZvBmPplkylJkrQWq/oKuqq6Gbi5W74HOH3yJUmS1sIr\nRSWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANd\nkhqxqrstzrvFiz/92PK9l5w7w0okafrsoUtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgD\nXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNOOK/\nU3T4e0Q3Ir9HVdISe+iS1AgDXZIaYaBLUiMMdElqhIEuSY0YGehJjk3yxSRfSXJnkvd1209JsjvJ\nviRXJjmm/3IlSSsZp4f+CHBWVb0QeBFwdpIzgA8AH6yqbcD3gIv6K1OSNMrIQK+Bh7vVo7tHAWcB\nV3fbdwEX9FKhJGksY42hJ9mU5DZgP3A98A3goao62O1yH3DSCsfuTLInyZ4DBw5MomZJ0iGMFehV\n9WhVvQg4GTgdeP6hdlvh2EurantVbV9YWFh7pZKkw1rVp1yq6iHgZuAM4PgkS7cOOBm4f7KlSZJW\nY5xPuSwkOb5bfgrwCmAvcBPwum63HcA1fRUpSRptnJtzbQF2JdnE4B+Aq6rquiR3AVck+WPgy8Bl\nPdYpSRphZKBX1VeB0w6x/R4G4+mSpDnglaKS1AgDXZIaYaBLUiMMdElqxBH/FXR6nF9HJ21s9tAl\nqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRXim6Sl6NKWle2UOXpEYY6JLU\nCANdkhqx4cbQHQOX1Cp76JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1IgN\nd6XoLM3DVarzUIOkfthDl6RGGOiS1AgDXZIaYaBLUiOOyEnR4Ym9Fqz03+OkpaTVsIcuSY0w0CWp\nEQa6JDXCQJekRhyRk6JHktYmcCXNr5E99CTPTnJTkr1J7kzyjm77iUmuT7Kv+3lC/+VKklYyzpDL\nQeB3qur5wBnA25KcClwM3FBV24AbunVJ0oyMDPSqeqCqvtQt/xDYC5wEnA/s6nbbBVzQV5GSpNFW\nNYaeZBE4DdgNPKuqHoBB6Cd55grH7AR2AmzdunXNha5nLHpWx7Zsebt4EZQ0e2N/yiXJccAngHdW\n1Q/GPa6qLq2q7VW1fWFhYS01SpLGMFagJzmaQZh/vKo+2W3+dpIt3fNbgP39lChJGsc4n3IJcBmw\nt6r+dOipa4Ed3fIO4JrJlydJGtc4Y+hnAm8Cbk9yW7ftPcAlwFVJLgK+Bby+nxIlSeMYGehV9a9A\nVnj65ZMtZ3LmfTJzmhO1K+3v19FJbfHSf0lqhIEuSY0w0CWpEQa6JDXCuy2qV068StNjD12SGmGg\nS1IjDHRJaoSBLkmNcFK0s5bJu4024TfvV99KG509dElqhIEuSY0w0CWpEQa6JDXCSdFGrecWuxth\ngldqkT10SWqEgS5JjTDQJakRBrokNWJDT4pO8srHlq6inOVVs07OSmtnD12SGmGgS1IjDHRJasSG\nHkOfd/MwnjwvcwPz0BbSvLOHLkmNMNAlqREGuiQ1wkCXpEY4KaqpWWlic14mXqUjnT10SWqEgS5J\njTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhFeKHsI8Xrk4jzX1baX/5lndSneleryd\nr+bFyB56ko8m2Z/kjqFtJya5Psm+7ucJ/ZYpSRplnCGXvwHOXrbtYuCGqtoG3NCtS5JmaGSgV9Xn\ngQeXbT4f2NUt7wIumHBdkqRVWuuk6LOq6gGA7uczJ1eSJGktep8UTbIT2AmwdevWvl9Oc2CcCdxJ\nTfKOM0Hq95Fqo1hrD/3bSbYAdD/3r7RjVV1aVduravvCwsIaX06SNMpaA/1aYEe3vAO4ZjLlSJLW\napyPLV4O/DvwvCT3JbkIuAR4ZZJ9wCu7dUnSDI0cQ6+qN6zw1MsnXIvUO8fT1TIv/ZekRhjoktQI\nA12SGmGgS1IjvNuixMqTpeu9AMpJWE2TPXRJaoSBLkmNMNAlqREGuiQ1wklRaZmN+HV/aoM9dElq\nhIEuSY0w0CWpEQa6JDXCSVGpEV6VKnvoktQIA12SGmGgS1IjDHRJaoSTomrGPEwKruUq0/XUvZ6r\nWuehvTRZ9tAlqREGuiQ1wkCXpEYY6JLUCCdFtWFN6ja5650IHWefeZjknfeJ0yOp1r7YQ5ekRhjo\nktQIA12SGuEYuprU2tfITXN8eBpt53h3P+yhS1IjDHRJaoSBLkmNMNAlqRFOikpTstqLiSb1WsOT\njn1PeK70+w838TnOBOlafu+o37/a7as1i4lfe+iS1AgDXZIaYaBLUiMMdElqhJOi0hGmj4nN9UzY\nzvJKz9W2xXr2X+0E7Cysq4ee5OwkX0tyd5KLJ1WUJGn11hzoSTYBfwH8CnAq8IYkp06qMEnS6qyn\nh346cHdV3VNV/wdcAZw/mbIkSau1nkA/CfjvofX7um2SpBlIVa3twOT1wKuq6i3d+puA06vq7cv2\n2wns7FafB3xthV+5GfjOmorp37zWZl2rN6+1zWtdML+1zWtdMPnanlNVC6N2Ws+nXO4Dnj20fjJw\n//KdqupS4NJRvyzJnqravo56ejOvtVnX6s1rbfNaF8xvbfNaF8yutvUMufwHsC3JKUmOAS4Erp1M\nWZKk1VpzD72qDib5LeCfgU3AR6vqzolVJklalXVdWFRVnwE+M6FaRg7LzNC81mZdqzevtc1rXTC/\ntc1rXTCj2tY8KSpJmi/ey0WSGtF7oCf5aJL9Se5Y4fnfTXJb97gjyaNJTuyeuzfJ7d1zeyZc17OT\n3JRkb5I7k7zjEPskyZ91tzb4apIXDz23I8m+7rFjBrW9savpq0m+kOSFQ8/10m5j1vXSJN8f+n/6\n3qHnerlVxJh1zeo8OzbJF5N8pavtfYfY58lJruzaZXeSxaHn3t1t/1qSV025rncluas7x25I8pyh\n5x4das+JfhhizNrenOTAUA1vGXqul7/NMev64FBNX0/y0NBzvbXZY6qq1wfwS8CLgTvG2Pc1wI1D\n6/cCm3uqawvw4m756cDXgVOX7XMO8FkgwBnA7m77icA93c8TuuUTplzbS5Zek8HtF3b33W5j1vVS\n4LpDHLsJ+AbwXOAY4CvLj+2zrhmeZwGO65aPBnYDZyzb5zeBD3fLFwJXdsundu30ZOCUrv02TbGu\nlwFP7ZbfulRXt/5wH+21itreDPz5IY7t7W9znLqW7f92Bh8W6b3Nlh6999Cr6vPAg2Pu/gbg8h7L\neUxVPVBVX+qWfwjs5SevdD0f+FgN3AIcn2QL8Crg+qp6sKq+B1wPnD3N2qrqC91rA9zC4DqAXo3Z\nZivp7VYRa6hrmudZVdXD3erR3WP5xNX5wK5u+Wrg5UnSbb+iqh6pqm8CdzNox6nUVVU3VdWPutWp\nnGPj1nYYvf1trqGuqZ1nS+ZmDD3JUxk0/CeGNhfwuSS3ZnDFaV+vvQicxuBf3GEr3d5garc9OExt\nwy5i8E5iSe/tNqKuX+jeln42yQu6bVNps1HtNYvzLMmmJLcB+xmEzYrnWVUdBL4PPIOe22yMuoYt\nP8eOTbInyS1JLphUTaus7Ve74aCrkyxd5DgXbdYNT50C3Di0udc2g/m6H/prgH+rquHe/JlVdX+S\nZwLXJ/nPrsc/MUmOY/DH/c6q+sHypw9xSB1m+0SNqG1pn5cx+GP7xaHNvbbbiLq+xOAy5YeTnAP8\nA7CNKbTZOO3FDM6zqnoUeFGS44FPJfnZqhqeU5rJeTZGXYPikl8DtgO/PLR5a9dmzwVuTHJ7VX1j\nirX9I3B5VT2S5DcYvMM5izlpMwZDZ1d3+y/ptc1gjnroDBrgCW9Pqur+7ud+4FNM6O3mkiRHMwiA\nj1fVJw+xy0q3Nxjrtgc910aSnwM+ApxfVd9d2t5nu42qq6p+sPS2tAbXKRydZDM9t9k47dWZ+nk2\n9DoPATfzk0MAj7VNkqOAn2YwTNn7eTaiLpK8AvgD4LyqemTomKU2u6c79rRJ13W42qrqu0P1/DXw\n893yzNusc7jzrL82m+SA/EoPYJHDTIry+An8tKFtTwOePrT8BeDsCdYU4GPAhw6zz7k8cVL0i/X4\nxMs3GUy6nNAtnzjl2rYyGFN9ybLtvbXbmHX9DI9f33A68K3uuKMYTFCdwuOToi+YVl0zPM8WgOO7\n5acA/wK8etk+b+OJk6JXdcsv4ImTovcwuUnRceo6jcFE7LZl208Antwtbwb2MaEJ7lXUtmVo+bXA\nLd1yb3+b49TVPfc8BhPtmVabLT16H3JJcjmDTz5sTnIf8IcMJhOoqg93u70W+FxV/c/Qoc9i8JYG\nBmHwd1X1TxMs7UzgTcDt3ZgYwHsYBOVSbZ9h8EmXu4EfAb/ePfdgkj9icD8bgPfXE9/CT6O29zIY\nZ/3Lro0O1uBmQH222zh1vQ54a5KDwP8CF9bgLO7zVhHj1AWzOc+2ALsy+EKYJzEI6+uSvB/YU1XX\nApcBf5vkbgb/4FzY1X1nkquAu4CDwNvqiW/h+67rT4DjgL/v2udbVXUe8Hzgr5L8uDv2kqq6a0J1\njVvbbyc5j0G7PMjgUy99/22OUxcMJkOv6M77JX23GeCVopLUjHkaQ5ckrYOBLkmNMNAlqREGuiQ1\nwkCXpEYY6JLUCANdkhphoEtSI/4fZGHJi3e2/c0AAAAASUVORK5CYII=\n", 269 | "text/plain": [ 270 | "
" 271 | ] 272 | }, 273 | "metadata": {}, 274 | "output_type": "display_data" 275 | } 276 | ], 277 | "source": [ 278 | "_=plt.hist(zbest, bins=100)" 279 | ] 280 | }, 281 | { 282 | "cell_type": "code", 283 | "execution_count": null, 284 | "metadata": {}, 285 | "outputs": [], 286 | "source": [] 287 | } 288 | ], 289 | "metadata": { 290 | "kernelspec": { 291 | "display_name": "qnet", 292 | "language": "python", 293 | "name": "qnet" 294 | }, 295 | "language_info": { 296 | "codemirror_mode": { 297 | "name": "ipython", 298 | "version": 3 299 | }, 300 | "file_extension": ".py", 301 | "mimetype": "text/x-python", 302 | "name": "python", 303 | "nbconvert_exporter": "python", 304 | "pygments_lexer": "ipython3", 305 | "version": "3.6.5" 306 | } 307 | }, 308 | "nbformat": 4, 309 | "nbformat_minor": 2 310 | } 311 | -------------------------------------------------------------------------------- /py/quasarnet/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ngbusca/QuasarNET/7d3e641027a202f1bd940168f74949d0d0be4f30/py/quasarnet/__init__.py -------------------------------------------------------------------------------- /py/quasarnet/__init__.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ngbusca/QuasarNET/7d3e641027a202f1bd940168f74949d0d0be4f30/py/quasarnet/__init__.pyc -------------------------------------------------------------------------------- /py/quasarnet/io.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | 3 | from os.path import dirname 4 | 5 | import numpy as np 6 | from numpy import random 7 | import fitsio 8 | from random import randint 9 | import glob 10 | from os.path import dirname 11 | 12 | 13 | def read_sdrq(sdrq): 14 | ''' 15 | Constructs two dictionaries thing_id: class, z_conf, z, 16 | and (plate, mjd, fiberid): thing_id 17 | input: (str) full path to Superset_DRQ.fits 18 | output: (list of dictionaries) 19 | ''' 20 | 21 | sdrq = fitsio.FITS(sdrq) 22 | thid = sdrq[1]["THING_ID"][:] 23 | class_person = sdrq[1]["CLASS_PERSON"][:] 24 | z_conf = sdrq[1]["Z_CONF_PERSON"][:] 25 | z_vi = sdrq[1]["Z_VI"][:] 26 | 27 | plate = sdrq[1]['PLATE'][:] 28 | mjd = sdrq[1]['MJD'][:] 29 | fiberid = sdrq[1]['FIBERID'][:] 30 | 31 | sdrq = {t:(c,zc,z) for t,c,zc,z in zip(thid, class_person, z_conf, z_vi)} 32 | pmf2tid = {(p,m,f):t for p,m,f,t in zip(plate, mjd, fiberid, thid)} 33 | 34 | return sdrq, pmf2tid 35 | 36 | def read_truth(fi): 37 | ''' 38 | reads a list of truth files and returns a truth dictionary 39 | 40 | Arguments: 41 | fi -- list of truth files (list of string) 42 | 43 | Returns: 44 | truth -- dictionary of THING_ID: metadata instance 45 | 46 | ''' 47 | 48 | class metadata: 49 | pass 50 | 51 | cols = ['Z_VI','PLATE', 52 | 'MJD','FIBERID','CLASS_PERSON', 53 | 'Z_CONF_PERSON','BAL_FLAG_VI','BI_CIV'] 54 | 55 | truth = {} 56 | 57 | for f in fi: 58 | h = fitsio.FITS(f) 59 | tids = h[1]['THING_ID'][:] 60 | cols_dict = {c.lower():h[1][c][:] for c in cols} 61 | h.close() 62 | for i,t in enumerate(tids): 63 | m = metadata() 64 | for c in cols_dict: 65 | setattr(m,c,cols_dict[c][i]) 66 | truth[t] = m 67 | 68 | return truth 69 | 70 | def read_data(fi, truth=None, z_lim=2.1, 71 | return_pmf=False, nspec=None): 72 | ''' 73 | reads data from input file 74 | 75 | Arguments: 76 | fi -- list of data files (string iterable) 77 | truth -- dictionary thind_id => metadata 78 | z_lim -- hiz/loz cut (float) 79 | return_pmf -- if True also return plate,mjd,fiberid 80 | nspec -- read this many spectra 81 | 82 | Returns: 83 | tids -- list of thing_ids 84 | X -- spectra reformatted to be fed to the network (numpy array) 85 | Y -- truth vector (nqso, 5): 86 | STAR = (1,0,0,0,0), GAL = (0,1,0,0,0) 87 | QSO_LZ = (0,0,1,0,0), QSO_HZ = (0,0,0,1,0) 88 | BAD = (0,0,0,0,1) 89 | z -- redshift (numpy array) 90 | bal -- 1 if bal, 0 if not (numpy array) 91 | ''' 92 | 93 | tids = [] 94 | X = [] 95 | Y = [] 96 | z = [] 97 | bal = [] 98 | 99 | if return_pmf: 100 | plate = [] 101 | mjd = [] 102 | fid = [] 103 | 104 | for f in fi: 105 | print('INFO: reading data from {}'.format(f)) 106 | h=fitsio.FITS(f) 107 | if nspec is None: 108 | nspec = h[1].get_nrows() 109 | aux_tids = h[1]['TARGETID'][:nspec].astype(int) 110 | ## remove thing_id == -1 or not in sdrq 111 | w = (aux_tids != -1) 112 | if truth is not None: 113 | w_in_truth = np.in1d(aux_tids, list(truth.keys())) 114 | print("INFO: removing {} spectra missing in truth".format((~w_in_truth).sum()),flush=True) 115 | w &= w_in_truth 116 | aux_tids = aux_tids[w] 117 | aux_X = h[0][:nspec,:] 118 | 119 | aux_X = aux_X[w] 120 | if return_pmf: 121 | aux_plate = h[1]['PLATE'][:][w] 122 | aux_mjd = h[1]['MJD'][:][w] 123 | aux_fid = h[1]['FIBERID'][:][w] 124 | plate += list(aux_plate) 125 | mjd += list(aux_mjd) 126 | fid += list(aux_fid) 127 | 128 | X.append(aux_X) 129 | tids.append(aux_tids) 130 | 131 | print("INFO: found {} spectra in file {}".format(aux_tids.shape, f)) 132 | 133 | tids = np.concatenate(tids) 134 | X = np.concatenate(X) 135 | 136 | if return_pmf: 137 | plate = np.array(plate) 138 | mjd = np.array(mjd) 139 | fid = np.array(fid) 140 | 141 | we = X[:,443:] 142 | w = we.sum(axis=1)==0 143 | print("INFO: removing {} spectra with zero weights".format(w.sum())) 144 | X = X[~w] 145 | tids = tids[~w] 146 | 147 | if return_pmf: 148 | plate = plate[~w] 149 | mjd = mjd[~w] 150 | fid = fid[~w] 151 | 152 | mdata = np.average(X[:,:443], weights = X[:,443:], axis=1) 153 | sdata = np.average((X[:,:443]-mdata[:,None])**2, 154 | weights = X[:,443:], axis=1) 155 | sdata=np.sqrt(sdata) 156 | 157 | w = sdata == 0 158 | print("INFO: removing {} spectra with zero flux".format(w.sum())) 159 | X = X[~w] 160 | tids = tids[~w] 161 | mdata = mdata[~w] 162 | sdata = sdata[~w] 163 | 164 | if return_pmf: 165 | plate = plate[~w] 166 | mjd = mjd[~w] 167 | fid = fid[~w] 168 | 169 | X = X[:,:443]-mdata[:,None] 170 | X /= sdata[:,None] 171 | 172 | if truth==None: 173 | if return_pmf: 174 | return tids,X,plate,mjd,fid 175 | else: 176 | return tids,X 177 | 178 | ## remove zconf == 0 (not inspected) 179 | observed = [(truth[t].class_person>0) or (truth[t].z_conf_person>0) for t in tids] 180 | observed = np.array(observed, dtype=bool) 181 | tids = tids[observed] 182 | X = X[observed] 183 | 184 | if return_pmf: 185 | plate = plate[observed] 186 | mjd = mjd[observed] 187 | fid = fid[observed] 188 | 189 | ## fill redshifts 190 | z = np.zeros(X.shape[0]) 191 | z[:] = [truth[t].z_vi for t in tids] 192 | 193 | ## fill bal 194 | bal = np.zeros(X.shape[0]) 195 | bal[:] = [(truth[t].bal_flag_vi*(truth[t].bi_civ>0))-\ 196 | (not truth[t].bal_flag_vi)*(truth[t].bi_civ==0) for t in tids] 197 | 198 | ## fill classes 199 | ## classes: 0 = STAR, 1=GALAXY, 2=QSO_LZ, 3=QSO_HZ, 4=BAD (zconf != 3) 200 | nclasses = 5 201 | sdrq_class = np.array([truth[t].class_person for t in tids]) 202 | z_conf = np.array([truth[t].z_conf_person for t in tids]) 203 | 204 | Y = np.zeros((X.shape[0],nclasses)) 205 | ## STAR 206 | w = (sdrq_class==1) & (z_conf==3) 207 | Y[w,0] = 1 208 | 209 | ## GALAXY 210 | w = (sdrq_class==4) & (z_conf==3) 211 | Y[w,1] = 1 212 | 213 | ## QSO_LZ 214 | w = ((sdrq_class==3) | (sdrq_class==30)) & (z=z_lim) & (z_conf==3) 219 | Y[w,3] = 1 220 | 221 | ## BAD 222 | w = z_conf != 3 223 | Y[w,4] = 1 224 | 225 | ## check that all spectra have exactly one classification 226 | assert (Y.sum(axis=1).min()==1) and (Y.sum(axis=1).max()==1) 227 | 228 | if return_pmf: 229 | return tids,X,Y,z,bal,plate,mjd,fid 230 | 231 | return tids,X,Y,z,bal 232 | 233 | def read_desi_truth(fin): 234 | h=fitsio.FITS(fin) 235 | truth = {} 236 | for t,c,z in zip(h[1]["TARGETID"][:], h[1]["TRUESPECTYPE"][:], h[1]["TRUEZ"][:]): 237 | c = c.strip() 238 | if c==b"QSO": 239 | c=3 240 | elif c==b"GALAXY": 241 | c=4 242 | elif c==b"STAR": 243 | c=1 244 | assert isinstance(c,int) 245 | truth[t] = (c,3,z) 246 | return truth 247 | 248 | 249 | llmin = np.log10(3600) 250 | llmax = np.log10(10000) 251 | dll = 1e-3 252 | 253 | nbins = int((llmax-llmin)/dll) 254 | wave = 10**(llmin + np.arange(nbins)*dll) 255 | nmasked_max = len(wave)+1 256 | 257 | def read_spcframe(b_spcframe,r_spcframe): 258 | data = [] 259 | fids = [] 260 | 261 | hb = fitsio.FITS(b_spcframe) 262 | hr = fitsio.FITS(r_spcframe) 263 | target_bits = hb[5]["BOSS_TARGET1"][:] 264 | wqso = np.zeros(len(target_bits),dtype=bool) 265 | mask = [10,11,12,13,14,15,16,17,18,19,40,41,42,43,44] 266 | for i in mask: 267 | wqso = wqso | (target_bits & 2**i) 268 | ## SEQUELS 269 | try: 270 | mask = [10, 11 ,12 ,13, 14, 15, 16, 17, 18] 271 | target_bits = h[5]["EBOSS_TARGET0"][:] 272 | for i in mask: 273 | wqso = wqso | (target_bits & 2**i) 274 | except: 275 | pass 276 | 277 | ## EBOSS 278 | try: 279 | mask = [10, 11 ,12 ,13, 14, 15, 16, 17, 18] 280 | target_bits = h[5]["EBOSS_TARGET1"][:] 281 | for i in mask: 282 | wqso = wqso | (target_bits & 2**i) 283 | except: 284 | pass 285 | wqso = wqso>0 286 | print("INFO: found {} quasars in file {}".format(wqso.sum(),b_spcframe)) 287 | 288 | plate = hb[0].read_header()["PLATEID"] 289 | fid = hb[5]["FIBERID"][:] 290 | fl = np.hstack((hb[0].read(),hr[0].read())) 291 | iv = np.hstack((hb[1].read()*(hb[2].read()==0),hr[1].read()*(hr[2].read()==0))) 292 | ll = np.hstack((hb[3].read(),hr[3].read())) 293 | 294 | fid = fid[wqso] 295 | fl = fl[wqso,:] 296 | iv = iv[wqso,:] 297 | ll = ll[wqso,:] 298 | 299 | for i in range(fl.shape[0]): 300 | fl_aux = np.zeros(nbins) 301 | iv_aux = np.zeros(nbins) 302 | bins = ((ll[i]-llmin)/dll).astype(int) 303 | wbin = (bins>=0) & (bins0) 304 | bins=bins[wbin] 305 | c = np.bincount(bins,weights=fl[i,wbin]*iv[i,wbin]) 306 | fl_aux[:len(c)]=+c 307 | c = np.bincount(bins,weights=iv[i,wbin]) 308 | iv_aux[:len(c)]=+c 309 | nmasked = (iv_aux==0).sum() 310 | if nmasked >= nmasked_max : 311 | print("INFO: skipping specrum {} with too many masked pixels {}".format(fid[i],nmasked)) 312 | continue 313 | data.append(np.hstack((fl_aux,iv_aux))) 314 | fids.append(fid[i]) 315 | 316 | assert ~np.isnan(fl_aux,iv_aux).any() 317 | 318 | if len(data)==0: 319 | return 320 | 321 | data = np.vstack(data) 322 | assert ~np.isnan(data).any() 323 | ## now normalize coadded fluxes 324 | norm = data[:,nbins:]*1. 325 | w = norm==0 326 | norm[w] = 1. 327 | data[:,:nbins]/=norm 328 | 329 | assert ~np.isnan(data).any() 330 | 331 | return fids, data 332 | 333 | def read_spall(spall): 334 | spall = fitsio.FITS(spall) 335 | plate=spall[1]["PLATE"][:] 336 | mjd = spall[1]["MJD"][:] 337 | fid = spall[1]["FIBERID"][:] 338 | tid = spall[1]["THING_ID"][:].astype(int) 339 | specprim=spall[1]["SPECPRIMARY"][:] 340 | 341 | pmf2tid = {(p,m,f):t for p,m,f,t,s in zip(plate,mjd,fid,tid,specprim)} 342 | spall.close() 343 | return pmf2tid 344 | 345 | def read_exposures(plates,pmf2tid,nplates=None, random_exp=False): 346 | 347 | ''' 348 | Given a list of plates, returns the thing_id list and the 349 | rebinned fluxes for all the exposures in the plates 350 | 351 | input: 352 | -- plates: list of str. List of paths to the spPlate files 353 | -- pmf2tid: dictionary containing (plate, mjd, fiber): thing_id 354 | -- nplates: use only the first nplates in the list 355 | -- random_exp: read only one random exposure from all the available 356 | exposures 357 | output: thid, data 358 | -- thid: list of thing ids of length equal the the number of exposures 359 | -- data: numpy array of float of shape (nexps, nbins) 360 | ''' 361 | 362 | data = [] 363 | read_plates = 0 364 | tids = [] 365 | 366 | plate_mjd_in_pmf2tid = np.empty(len(pmf2tid), dtype=object) 367 | print('calculating plates-mjd combos') 368 | plate_mjd_in_pmf2tid[:] =[(k[0], k[1]) for k in pmf2tid.keys()] 369 | print('uniq-ing') 370 | plate_mjd_in_pmf2tid = list(np.unique(plate_mjd_in_pmf2tid)) 371 | print('done') 372 | 373 | if nplates is not None: 374 | plates = plates[:nplates] 375 | for p in plates: 376 | h=fitsio.FITS(p) 377 | head = h[0].read_header() 378 | plateid = head['PLATEID'] 379 | m = head['MJD'] 380 | if (plateid,m) not in plate_mjd_in_pmf2tid: 381 | print('{} {} not in list'.format(plateid,m)) 382 | continue 383 | 384 | 385 | exps = [] 386 | ## read b,r exposures 387 | try: 388 | nexp_b = head["NEXP_B1"]+head["NEXP_B2"] 389 | except: 390 | continue 391 | if nexp_b>99: 392 | nexp_b=99 393 | for exp in range(nexp_b): 394 | str_exp = str(exp+1) 395 | if exp<9: 396 | str_exp = '0'+str_exp 397 | exp_b = head["EXPID{}".format(str_exp)][:11] 398 | exp_r = exp_b.replace("b", "r") 399 | exps.append((exp_b, exp_r)) 400 | 401 | exps_spectro_1 = [e for e in exps if 'b1' in e[0]] 402 | exps_spectro_2 = [e for e in exps if 'b2' in e[0]] 403 | if random_exp: 404 | irand1 = randint(0,len(exps_spectro_1)-1) 405 | irand2 = randint(0,len(exps_spectro_2)-1) 406 | exps = [exps_spectro_1[irand1], exps_spectro_2[irand2]] 407 | 408 | for exp_b, exp_r in exps: 409 | spcframe_b = dirname(p)+"/spCFrame-{}.fits".format(exp_b) 410 | spcframe_r = dirname(p)+"/spCFrame-{}.fits".format(exp_r) 411 | res = read_spcframe(spcframe_b, spcframe_r) 412 | if res is not None: 413 | plate_fid, plate_data = res 414 | data.append(plate_data) 415 | tids = tids + [pmf2tid[(plateid,m,f)] for f in plate_fid] 416 | 417 | if nplates is not None: 418 | if len(data)//2==nplates: 419 | break 420 | 421 | data = np.vstack(data) 422 | 423 | return tids, data 424 | 425 | def export_data(fout,tids,data): 426 | h = fitsio.FITS(fout,"rw",clobber=True) 427 | h.write(data,extname="DATA") 428 | tids = np.array(tids) 429 | h.write([tids],names=["TARGETID"],extname="METADATA") 430 | h.close() 431 | 432 | 433 | def read_desi_spectra(fin, ignore_quasar_mask=False): 434 | try: 435 | from desitarget import desi_mask 436 | quasar_mask = desi_mask.mask('QSO') 437 | except: 438 | print("WARN: can't load desi_mask, ignoring mask!") 439 | quasar_mask = 1 440 | 441 | h=fitsio.FITS(fin) 442 | nbins = int((llmax-llmin)/dll) 443 | wqso = h[1]['DESI_TARGET'][:] & quasar_mask 444 | if ignore_quasar_mask: 445 | wqso |= 1 446 | wqso = wqso>0 447 | print("INFO: found {} quasar targets".format(wqso.sum())) 448 | tids = h[1]["TARGETID"][:][wqso] 449 | utids = np.unique(tids) 450 | 451 | nspec = len(utids) 452 | fl = np.zeros((nspec, nbins)) 453 | iv = np.zeros((nspec, nbins)) 454 | if nspec == 0: return None 455 | for band in ["B", "R", "Z"]: 456 | wave = h["{}_WAVELENGTH".format(band)].read() 457 | w = (np.log10(wave)>llmin) & (np.log10(wave)0 473 | fl[w]/=iv[w] 474 | fl = np.hstack((fl,iv)) 475 | 476 | print("INFO: founds {} good spectra".format(wqso.sum())) 477 | return utids, fl 478 | 479 | 480 | def read_spplate(fin, fibers): 481 | 482 | ''' 483 | reads data from spplates 484 | ''' 485 | 486 | h=fitsio.FITS(fin) 487 | head = h[0].read_header() 488 | c0 = head["COEFF0"] 489 | c1 = head["COEFF1"] 490 | p = head["PLATEID"] 491 | m = head["MJD"] 492 | 493 | fids = h[5]["FIBERID"][:] 494 | wqso = np.in1d(fids, fibers) 495 | fids=fids[wqso] 496 | 497 | nspec = len(fibers) 498 | nbins = int((llmax-llmin)/dll) 499 | fl = np.zeros((nspec, nbins)) 500 | iv = np.zeros((nspec, nbins)) 501 | nbins = fl.shape[1] 502 | 503 | fl_aux = h[0].read()[wqso,:] 504 | iv_aux = h[1].read()[wqso,:]*((h[2].read()[wqso]&2**25)==0) 505 | wave = 10**(c0 + c1*np.arange(fl_aux.shape[1])) 506 | bins = np.floor((np.log10(wave)-llmin)/dll).astype(int) 507 | w = (bins>=0) & (bins0 519 | fl[w]/=iv[w] 520 | fl = np.hstack((fl,iv)) 521 | print(fl.shape) 522 | wbad = iv==0 523 | w=wbad.sum(axis=1)>nmasked_max 524 | print('INFO: rejecting {} spectra with too many bad pixels'.format(w.sum())) 525 | if (~w).sum()==0: 526 | return None 527 | fl=fl[~w,:] 528 | return fids[~w],fl 529 | 530 | from .utils import absorber_IGM 531 | from scipy.interpolate import interp1d 532 | def box_offset(z, line='LYA', nboxes = 13): 533 | wave_to_i = interp1d(wave, np.arange(len(wave)), 534 | bounds_error=False, fill_value=-1) 535 | wave_line = (1+z)*absorber_IGM[line] 536 | pos = wave_to_i(wave_line)/len(wave)*nboxes 537 | ipos = np.floor(pos).astype(int) 538 | 539 | box = np.zeros((len(z), nboxes)) 540 | offset = np.zeros((len(z), nboxes)) 541 | 542 | w = ipos>=0 543 | box[w, ipos[w]] = 1 544 | offset[w, ipos[w]] = (pos-ipos)[w] 545 | weights = np.ones(len(z)) 546 | weights[~w]=0 547 | 548 | return box, offset, weights 549 | 550 | def objective(z, Y, bal, lines=['LYA'], 551 | lines_bal=['CIV(1548)'], nboxes=13): 552 | box=[] 553 | sample_weight = [] 554 | for l in lines: 555 | box_line, offset_line, weight_line = box_offset(z, 556 | line = l, nboxes=nboxes) 557 | 558 | w = (Y.argmax(axis=1)==2) | (Y.argmax(axis=1)==3) 559 | ## set to zero where object is not a QSO 560 | ## (the line confidence should be zero) 561 | box_line[~w]=0 562 | box.append(np.concatenate([box_line, offset_line], axis=-1)) 563 | sample_weight.append(np.ones(Y.shape[0])) 564 | 565 | for l in lines_bal: 566 | box_line, offset_line, weight_line = box_offset(z, 567 | line = l, nboxes=nboxes) 568 | 569 | ## set to zero for non-quasars 570 | wqso = (Y.argmax(axis=1)==2) | (Y.argmax(axis=1)==3) 571 | box_line[~wqso] = 0 572 | 573 | ## set to zero for confident non-bals: 574 | wnobal = (bal==-1) 575 | box_line[wnobal] = 0 576 | 577 | ## use only spectra where visual flag and bi_civ do agree 578 | bal_weight = bal != 0 579 | box.append(np.concatenate([box_line, offset_line], axis=-1)) 580 | sample_weight.append(bal_weight) 581 | 582 | return box, sample_weight 583 | -------------------------------------------------------------------------------- /py/quasarnet/io.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ngbusca/QuasarNET/7d3e641027a202f1bd940168f74949d0d0be4f30/py/quasarnet/io.pyc -------------------------------------------------------------------------------- /py/quasarnet/models.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import numpy as np 3 | 4 | import tensorflow as tf 5 | from tensorflow.keras.layers import Input, Dense, Activation, BatchNormalization, Flatten, Conv1D, concatenate, Lambda 6 | from tensorflow.keras.models import Model 7 | import tensorflow.keras.backend as K 8 | from tensorflow.keras.utils import plot_model 9 | from tensorflow.keras.initializers import glorot_uniform, glorot_uniform 10 | from tensorflow.keras import regularizers 11 | from tensorflow.keras.activations import softmax, relu 12 | 13 | def QuasarNET(input_shape = None, boxes = 13, nlines = 1, reg_conv = 0., reg_fc=0): 14 | 15 | X_input = Input(input_shape) 16 | X = X_input 17 | 18 | nlayers=4 19 | nfilters_max = 100 20 | filter_size=10 21 | strides = 2 22 | for stage in range(nlayers): 23 | nfilters = 100 24 | X = Conv1D(nfilters, filter_size, strides = strides, 25 | name = 'conv_{}'.format(stage+1), 26 | kernel_initializer=glorot_uniform(), 27 | kernel_regularizer=regularizers.l2(reg_conv))(X) 28 | X = BatchNormalization(axis=-1)(X) 29 | X = Activation('relu')(X) 30 | 31 | X = Flatten()(X) 32 | X = Dense(nfilters_max, activation='linear', name='fc_common')(X) 33 | X = BatchNormalization()(X) 34 | X = Activation('relu', name='fc_activation')(X) 35 | 36 | outputs = [] 37 | X_box = [] 38 | for i in range(nlines): 39 | X_box_aux = Dense(boxes, activation='sigmoid', 40 | name='fc_box_{}'.format(i), 41 | kernel_initializer=glorot_uniform())(X) 42 | X_offset_aux = Dense(boxes, activation='sigmoid', 43 | #X_offset_aux = Dense(boxes, activation='linear', 44 | name='fc_offset_{}'.format(i), 45 | kernel_initializer=glorot_uniform())(X) 46 | ## rescale the offsets to output between -0.1 and 1.1 47 | X_offset_aux = Lambda(lambda x:-0.1+1.2*x)(X_offset_aux) 48 | X_box_aux = concatenate([X_box_aux, X_offset_aux], 49 | name="conc_box_{}".format(i)) 50 | X_box.append(X_box_aux) 51 | 52 | for b in X_box: 53 | outputs.append(b) 54 | 55 | model = Model(inputs=X_input, outputs=outputs, name='QuasarNET') 56 | 57 | return model 58 | 59 | def custom_loss(y_true, y_pred): 60 | 61 | assert y_pred.shape[1]%2 == 0 62 | 63 | nboxes = y_pred.get_shape().as_list()[1]//2 64 | 65 | N1 = tf.math.reduce_sum(y_true[...,0:nboxes], axis=1) + K.epsilon() 66 | N2 = tf.math.reduce_sum((1-y_true[...,0:nboxes]), axis=1) + K.epsilon() 67 | loss_class = -tf.math.reduce_sum(y_true[...,0:nboxes]*tf.math.log(K.clip(y_pred[...,0:nboxes], K.epsilon(), 1-K.epsilon())), axis=1)/N1 68 | loss_class -= tf.math.reduce_sum((1-y_true[...,0:nboxes])*tf.math.log(K.clip(1-y_pred[...,0:nboxes], K.epsilon(), 1-K.epsilon())), axis=1)/N2 69 | 70 | offset_true = y_true[...,nboxes:] 71 | 72 | offset_pred = y_pred[...,nboxes:] 73 | doffset = tf.math.subtract(offset_true, offset_pred) 74 | loss_offset = tf.math.reduce_sum(y_true[...,0:nboxes]*tf.math.square(doffset), axis=1)/N1 75 | 76 | return tf.math.add(loss_class, loss_offset) 77 | -------------------------------------------------------------------------------- /py/quasarnet/utils.py: -------------------------------------------------------------------------------- 1 | from scipy.interpolate import interp1d 2 | from numpy import zeros, arange, array 3 | from .io import wave 4 | 5 | def process_preds(preds, lines, lines_bal): 6 | ''' 7 | Convert network predictions to c_lines, z_lines and z_best 8 | 9 | Arguments: 10 | preds: float, array 11 | model predictions, output of model.predict 12 | 13 | lines: string, array 14 | list of line names 15 | 16 | lines_bal: string, array 17 | list of BAL line names 18 | 19 | Returns: 20 | c_line: float, array 21 | line confidences, shape: (nlines, nspec) 22 | z_line: float, array 23 | line redshifts, shape: (nlines, nspec) 24 | zbest: float, array 25 | redshift of highest confidence line, shape: (nspec) 26 | c_line_bal: float, array 27 | line confidences of BAL lines, shape: (nlines_bal, nspec) 28 | z_line_bal: float, array 29 | line redshfits of BAL lines, shape: (nlines_bal, nspec) 30 | ''' 31 | assert len(lines)+len(lines_bal)==len(preds) 32 | 33 | nspec, nboxes = preds[0].shape 34 | nboxes //=2 35 | print('INFO: nspec = {}, nboxes={}'.format(nspec, nboxes)) 36 | nlines = len(lines) 37 | c_line=zeros((nlines, nspec)) 38 | z_line=zeros((nlines, nspec)) 39 | i_to_wave = interp1d(arange(len(wave)), wave, 40 | bounds_error=False, fill_value='extrapolate') 41 | 42 | for il in range(len(lines)): 43 | l=absorber_IGM[lines[il]] 44 | j = preds[il][:,:nboxes].argmax(axis=1) 45 | offset = preds[il][arange(nspec, dtype=int), nboxes+j] 46 | c_line[il]=preds[il][:,:nboxes].max(axis=1) 47 | z_line[il]=i_to_wave((j+offset)*len(wave)/nboxes)/l-1 48 | 49 | zbest = z_line[c_line.argmax(axis=0),arange(nspec)] 50 | zbest = array(zbest) 51 | 52 | nlines_bal = len(lines_bal) 53 | c_line_bal=zeros((nlines_bal, nspec)) 54 | z_line_bal=zeros((nlines_bal, nspec)) 55 | 56 | for il in range(len(lines_bal)): 57 | l = absorber_IGM[lines_bal[il]] 58 | j = preds[nlines+il][:,:nboxes].argmax(axis=1) 59 | offset = preds[il+nlines][arange(nspec, dtype=int), nboxes+j] 60 | c_line_bal[il]=preds[il+nlines][:,:nboxes].max(axis=1) 61 | z_line_bal[il]=i_to_wave((j+offset)*len(wave)/nboxes)/l-1 62 | 63 | return c_line, z_line, zbest, c_line_bal, z_line_bal 64 | 65 | absorber_IGM = { 66 | 'Halpha' : 6562.8, 67 | 'OIII(5008)' : 5008.24, 68 | 'OIII(4933)' : 4932.68, 69 | 'Hbeta' : 4862.68, 70 | 'MgI(2853)' : 2852.96, 71 | 'MgII(2804)' : 2803.5324, 72 | 'MgII(2796)' : 2796.3511, 73 | 'FeII(2600)' : 2600.1724835, 74 | 'FeII(2587)' : 2586.6495659, 75 | 'MnII(2577)' : 2576.877, 76 | 'FeII(2383)' : 2382.7641781, 77 | 'FeII(2374)' : 2374.4603294, 78 | 'FeII(2344)' : 2344.2129601, 79 | 'CIII(1909)' : 1908.734, 80 | 'AlIII(1863)' : 1862.79113, 81 | 'AlIII(1855)' : 1854.71829, 82 | 'AlII(1671)' : 1670.7886, 83 | 'FeII(1608)' : 1608.4511, 84 | 'CIV(1551)' : 1550.77845, 85 | 'CIV(eff)' : 1549.06, 86 | 'CIV(1548)' : 1548.2049, 87 | 'SiII(1527)' : 1526.70698, 88 | 'SiIV(1403)' : 1402.77291, 89 | 'SiIV(1394)' : 1393.76018, 90 | 'CII(1335)' : 1334.5323, 91 | 'SiII(1304)' : 1304.3702, 92 | 'OI(1302)' : 1302.1685, 93 | 'SiII(1260)' : 1260.4221, 94 | 'NV(1243)' : 1242.804, 95 | 'NV(1239)' : 1238.821, 96 | 'LYA' : 1215.67, 97 | 'SiIII(1207)' : 1206.500, 98 | 'NI(1200)' : 1200., 99 | 'SiII(1193)' : 1193.2897, 100 | 'SiII(1190)' : 1190.4158, 101 | 'OI(1039)' : 1039.230, 102 | 'OVI(1038)' : 1037.613, 103 | 'OVI(1032)' : 1031.912, 104 | 'LYB' : 1025.72, 105 | } 106 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | scipy 3 | h5py 4 | fitsio 5 | tensorflow >= 2 6 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import glob 4 | 5 | from setuptools import setup, find_packages 6 | 7 | scripts = glob.glob('bin/*') 8 | 9 | description = "CNN for quasar classification and redshifting" 10 | 11 | version="2.0" 12 | setup(name="quasarnet", 13 | version=version, 14 | description=description, 15 | url="https://github.com/ngbusca/QuasarNET", 16 | author="Nicolas Busca et al", 17 | author_email="nbusca@gmail.com", 18 | packages=['quasarnet'], 19 | package_dir = {'': 'py'}, 20 | install_requires=['scipy','numpy', 21 | 'fitsio','h5py','tensorflow'], 22 | #test_suite='picca.test.test_cor', 23 | scripts = scripts 24 | ) 25 | 26 | -------------------------------------------------------------------------------- /web/css/qnet_styles.css: -------------------------------------------------------------------------------- 1 | /* class panel: thin border and shadow; */ 2 | 3 | .panel { 4 | margin-top: 10px; 5 | width: 600px; 6 | margin-left: 240px; 7 | padding: 10px; 8 | background: #FFF; 9 | border-width: 0px; 10 | border-style: solid; 11 | box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); 12 | overflow-x: scroll; 13 | overflow-y: scroll; 14 | } 15 | 16 | /* Fixed sidenav, full height */ 17 | .sidenav { 18 | width: 240px; 19 | position: fixed; 20 | z-index: 1; 21 | top: 0; 22 | left: 0; 23 | overflow-x: hidden; 24 | padding-top: 20px; 25 | } 26 | 27 | /* Style the sidenav links and the dropdown button */ 28 | .sidenav a, .dropdown-btn { 29 | padding: 6px 8px 6px 16px; 30 | font-family: sans-serif; 31 | font-size: 20px; 32 | text-decoration: none; 33 | color: #111; 34 | display: block; 35 | border: none; 36 | background: none; 37 | width:100%; 38 | text-align: left; 39 | cursor: pointer; 40 | outline: none; 41 | } 42 | 43 | /* On mouse-over */ 44 | .sidenav a:hover, .dropdown-btn:hover { 45 | color: #999; 46 | } 47 | 48 | /* Style page content */ 49 | .main { 50 | margin-left: 260px; /* Same as the width of the sidebar */ 51 | padding: 10px; 52 | width: 600px; 53 | font-family: sans-serif; 54 | font-size: 14px; 55 | } 56 | 57 | /* Style for code blocks*/ 58 | pre { 59 | font-family: monospace; 60 | background: #DDD; 61 | overflow:auto; 62 | margin-left: 20px; 63 | padding: 5px; 64 | } 65 | 66 | /* Add an active class to the active dropdown button */ 67 | .active { 68 | background-color: green; 69 | color: white; 70 | } 71 | 72 | /* Dropdown container (hidden by default). Optional: add a lighter background color and some left padding to change the design of the dropdown content */ 73 | .dropdown-container { 74 | /*display: none;*/ 75 | padding-left: 8px; 76 | } 77 | 78 | /* Optional: Style the caret down icon */ 79 | .fa-caret-down { 80 | float: right; 81 | padding-right: 8px; 82 | } 83 | 84 | /* On smaller screens, where height is less than 450px, change the style of the sidebar (less padding and a smaller font size) */ 85 | @media screen and (max-height: 450px) { 86 | .sidenav {padding-top: 15px;} 87 | .sidenav a {font-size: 18px;} 88 | } -------------------------------------------------------------------------------- /web/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | QuasarNET 6 | 7 | 8 | 9 | 10 | 11 | 12 |
13 | home 14 | about 15 | 16 | 19 | 20 |
21 | try it online! 22 | or install on your system 23 |
24 | 25 | contact 26 | 27 |
28 | 29 | 30 |
31 |

QuasarNET

32 | architecture 33 |
34 | 35 |
36 |

Computer vision meets astrophysics! QuasarNET uses computer vision techniques to identify and locate emission lines in astrophysical spectra. Find out more: arXiv:1808.09955

37 |
38 | 39 |
40 |

QuasarNET online server

41 |

No installation required! Get your predictions directly from the QuasarNET server.

42 |

From the command line:

43 |

Send your spectra via a POST request to the QuasarNET server and save the response to predictions.json.

44 | 
curl -o predictions.json -X POST -H "Content-Type: application/json" --data @spectra.json https://thequasar.net:5000/predict
 45 |
46 |

The file spectra.json that contains your spectra should look like this:

47 | 
{
 48 |   "wave":["l1","l2","l3",...],
 49 |   "flux":[["f1_1","f2_1","f3_1",...], ["f1_2","f2_2","f3_2", ...], ...] ,
 50 |   "ivar":[["iv1_1","iv2_1","iv3_1",...], ["iv1_2","iv2_2","iv3_2",...], ... ]
 51 | }
52 |

where the "wave" array contains the wavelenghts in Angstrom, "flux" is an array of arrays, each array containing the flux values (any units), and ivar is an array of arrays containing the inverse variances. The double quotes around the values are required. Note that all spectra are assumed to have a common wavelength grid.

53 | 54 |

The returned json contains the following fields:

55 | 
"lines": list of line-names searched for
 56 | "lines_BAL": list of lines where BAL were searched for
 57 | "p_<line name>": array of probabilities for the line given by <line name> (one probability per spectrum per line-name)
 58 | "pBAL_<line name>": same but for BAL
 59 | "z_<line name>": array of redshifts for the given line (one redshift per spectrum per line)
 60 | "zBAL_<line name>": same but for BAL
 61 | "zbest": array of redshift, chosen as that of the line with highest probability
62 | It is up to you to decide on a threshold probability and a threshold number of lines to determine when a spectrum is classified as a quasar. Please refer to the paper for recommendations on these values. 63 | 64 |

From a python session:

65 | 66 | 
## Organize your spectra in a dictionary:
 67 | spectra = {}
 68 | 
 69 | ## wave is an array containing the wavelength values
 70 | ## (doesn't need to be sorted). E.g. for three wavelength bands:
 71 | wave = numpy.hstack(wave_b, wave_r, wave_z)
 72 | spectra['wave'] = [str(lam) for lam in wave]
 73 | 
 74 | ## fluxes is a list of arrays, each containing the flux of a spectrum
 75 | ## e.g. for three bands (r, b, z):
 76 | fluxes = [numpy.hstack(flux1_b, flux1_r, flux1_z), numpy.hstack(flux2_b, flux2_r, flux2_z), ...]
 77 | spectra['flux'] = [[str(fl) for fl in flux] for flux in fluxes]
 78 | 
 79 | ## ivar is a list of arrays, each containing the inverse variance of a spectrum
 80 | ivars = [numpy.hstack(ivar0_b, ivar0_r, ivar0_z), numpy.hstack(ivar1_b, ivar1_r, ivar1_z), ...]
 81 | spectra['ivar'] = [[str(iv) for iv in ivar] for ivar in ivars]
 82 | 
 83 | ## now send the post request
 84 | ## you might need to
 85 | ## pip install requests
 86 | ## if you don't have the requests library installed
 87 | 
 88 | import requests
 89 | url = "http://thequasar.net:5000/predict"
 90 | r = requests.post(url = url, json = spectra)
 91 | 
 92 | ## store the results on the 'out' dictionary:
 93 | out = r.json()
 94 |
95 |
96 | 97 |
98 |

Installation instructions

99 | 100 |

QuasarNET runs on python3

101 | 102 | 
## clone the project 
103 | git clone https://github.com/ngbusca/QuasarNET.git
104 | cd QuasarNET
105 | ## install external packages
106 | pip install -r requirements.txt --user
107 | ## install QuasarNET
108 | python setup.py install --user
109 |
110 | 111 |

Download the pre-trained weights

112 | 113 |

Download the weights to the QuasarNET/weights/ directory, unzip the file and fix read/write permissions.

114 | 115 |

The pre-trained weights are available for direct download at https://www.kaggle.com/ngbusca/qnet_trained_models

116 | 117 |

or use the kaggel API:

118 | 119 | 
cd weights
120 | kaggle datasets download ngbusca/qnet_trained_models
121 | unzip qnet_trained_models.zip
122 | chmod 600 *
123 |
124 | 125 |

Download the data

126 | 127 |

These data are a reprocessing of data release 12 (DR12) of the Sloan Digital Sky Survey (https://www.sdss.org/dr12/)

128 | 129 |

They are available on Kaggle: https://www.kaggle.com/ngbusca/qnet_data

130 | 131 |

A practical way to download the data is to use the kaggle-api, which will allow you to do it from the command line. Otherwise you can simply click the download link on the website.

132 | 133 |

Download the data to the QuasarNET/data/ directory, unzip the file and set read/write permissions (skip the kaggle datasets... line if you've downloaded the data through the website).

134 | 135 | 
cd data
136 | kaggle datasets download ngbusca/qnet_data
137 | unzip qnet_data.zip
138 | chmod 600 *
139 |
140 |
141 | 142 |

Contact

143 |

Questions? Suggestions? contact@thequasar.net

144 | 145 | 146 | 163 | 164 | 165 | 166 | -------------------------------------------------------------------------------- /web/pics/architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ngbusca/QuasarNET/7d3e641027a202f1bd940168f74949d0d0be4f30/web/pics/architecture.png -------------------------------------------------------------------------------- /weights/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | !.gitignore 3 | --------------------------------------------------------------------------------