├── README.md ├── Urban Air Pollution Challenge by #ZindiWeekendz ├── Instructions.md ├── README.md ├── zindi-weekendz-pollution-blend.ipynb └── zindi-weekendz-pollution-v4.ipynb └── The Zimnat Insurance Assurance Challenge by #ZindiWeekendz └── zindi_weekend_insurance_final_solution.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # zindi-winning-solutions 2 | My solutions to various zindi competitions. Zindi is an African data science platform, which can be found here https://zindi.africa/ 3 | -------------------------------------------------------------------------------- /Urban Air Pollution Challenge by #ZindiWeekendz/Instructions.md: -------------------------------------------------------------------------------- 1 | ## Instructions for running. 2 | 3 | 1. Run the notebooks from zindi-weekendz-pollution-v1.ipynb to zindi-weekendz-pollution-v4.ipynb to generate the output submission files for each of them. 4 | 2. Change the data path from '/kaggle/input' to where the files are stored in your local machine. 5 | 3. Finally run the notebook zindi-weekendz-pollution-blend.ipynb, to generate the final output file, **submission.csv** 6 | 7 | ## Approach 8 | 9 | Only the first notebook **zindi-weekendz-pollution-v1.ipynb** has comments added, because all the notebooks are almost same. 10 | 11 | The idea was simply to use future and past values to predict the future, along with the current readings. If you see the feature importances you will see that future and past values are indeed important. Some kinds of features that were added were: 12 | 13 | 1. Past and previous values of target for arbitary number of days, which you can experiment. 14 | 2. Past and previous values of sensor readings for arbitary number of days. 15 | 3. Extracting information from the date column about weekday, month, is_month_start or is_month_end. 16 | 4. Adding cyclic features because of the cyclic behaviour of time, consider this Monday repeats after every 7 days. Sunday is just before Monday, but while doing encoding we would encode Sunday as 6, and Monday as 0. Adding cyclic features help eliminate some of this problem. 17 | 5. Adding frequency encoding for **place_ID** since some place_IDs had different number of occurences in the data compared to others, I added a feature to capture it. 18 | 19 | The final model was a blended version of 4 different lightGBM models I created, weights were adjusted to ensure optimal performance on the leaderboard. 20 | 21 | 22 | -------------------------------------------------------------------------------- /Urban Air Pollution Challenge by #ZindiWeekendz/README.md: -------------------------------------------------------------------------------- 1 | # Urban Air Pollution Challenge by #ZindiWeekendz 2 | 3 | ## Can you predict air quality in cities around the world using satellite data? 4 | 5 | The objective of this challenge is to predict PM2.5 particulate matter concentration in the air every day for each city. PM2.5 refers to atmospheric particulate matter that have a diameter of less than 2.5 micrometers and is one of the most harmful air pollutants. PM2.5 is a common measure of air quality that normally requires ground-based sensors to measure. The data covers the last three months, spanning hundreds of cities across the globe. 6 | 7 | The data comes from three main sources: 8 | 9 | * Ground-based air quality sensors. These measure the target variable (PM2.5 particle concentration). In addition to the target column (which is the daily mean concentration) there are also columns for minimum and maximum readings on that day, the variance of the readings and the total number (count) of sensor readings used to compute the target value. This data is only provided for the train set - you must predict the target variable for the test set. 10 | * The Global Forecast System (GFS) for weather data. Humidity, temperature and wind speed, which can be used as inputs for your model. 11 | * The Sentinel 5P satellite. This satellite monitors various pollutants in the atmosphere. For each pollutant, we queried the offline Level 3 (L3) datasets available in Google Earth Engine (you can read more about the individual products here: https://developers.google.com/earth-engine/datasets/catalog/sentinel-5p). For a given pollutant, for example NO2, we provide all data from the Sentinel 5P dataset for that pollutant. This includes the key measurements like NO2_column_number_density (a measure of NO2 concentration) as well as metadata like the satellite altitude. We recommend that you focus on the key measurements, either the column_number_density or the tropospheric_X_column_number_density (which measures density closer to Earth’s surface). 12 | Unfortunately, this data is not 100% complete. Some locations have no sensor readings for a particular day, and so those rows have been excluded. There are also gaps in the input data, particularly the satellite data for CH4. 13 | 14 | ### Variable Definitions: Read about the datasets at the following pages: 15 | 16 | * Weather Data: https://developers.google.com/earth-engine/datasets/catalog/NOAA_GFS0P25 17 | * Sentinel 5P data: https://developers.google.com/earth-engine/datasets/catalog/sentinel-5p - all columns begin with the dataset name (eg L3_NO2 corresponds to https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S5P_OFFL_L3_NO2) - look at the corresponding dataset on GEE for detailed descriptions of the image bands - band names should match the second half of the column titles. 18 | 19 | ## Leaderboard 20 | 21 | * **[Private LB](https://zindi.africa/hackathons/urban-air-pollution-challenge/leaderboard)** : **1st/115 Rank** 22 | -------------------------------------------------------------------------------- /Urban Air Pollution Challenge by #ZindiWeekendz/zindi-weekendz-pollution-blend.ipynb: -------------------------------------------------------------------------------- 1 | {"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os","execution_count":33,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ID_COL = 'Place_ID X Date'","execution_count":34,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"df = pd.read_csv('preds_lgbm_v1.csv')[[ID_COL]]\ndf['v1'] = pd.read_csv('preds_lgbm_v1.csv')['target']\ndf['v2'] = pd.read_csv('preds_lgbm_v2.csv')['target']\ndf['v3'] = pd.read_csv('preds_lgbm_v3.csv')['target']\ndf['v4'] = pd.read_csv('preds_lgbm_v4.csv')['target']","execution_count":35,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Blending the different versions"},{"metadata":{},"cell_type":"markdown","source":"## The weights were decided based on the performance of each version on the leaderboard"},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_df = df[[ID_COL]]\nsub_df['target'] = ((df['v1']*0.75 + df['v2']*0.25)*0.6 + df['v4']*0.4)*0.9 + df['v3']*0.1","execution_count":36,"outputs":[{"output_type":"stream","text":"/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:2: SettingWithCopyWarning: \nA value is trying to be set on a copy of a slice from a DataFrame.\nTry using .loc[row_indexer,col_indexer] = value instead\n\nSee the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n \n","name":"stderr"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_df.to_csv('submission.csv', index=False)","execution_count":37,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_df.head(10)","execution_count":38,"outputs":[{"output_type":"execute_result","execution_count":38,"data":{"text/plain":" Place_ID X Date target\n0 0OS9LVX X 2020-01-02 32.680554\n1 0OS9LVX X 2020-01-03 31.289176\n2 0OS9LVX X 2020-01-04 28.412420\n3 0OS9LVX X 2020-01-05 34.441060\n4 0OS9LVX X 2020-01-06 56.556865\n5 0OS9LVX X 2020-01-07 57.094444\n6 0OS9LVX X 2020-01-08 27.458843\n7 0OS9LVX X 2020-01-09 29.744958\n8 0OS9LVX X 2020-01-10 26.061362\n9 0OS9LVX X 2020-01-11 34.992617","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Place_ID X Datetarget
00OS9LVX X 2020-01-0232.680554
10OS9LVX X 2020-01-0331.289176
20OS9LVX X 2020-01-0428.412420
30OS9LVX X 2020-01-0534.441060
40OS9LVX X 2020-01-0656.556865
50OS9LVX X 2020-01-0757.094444
60OS9LVX X 2020-01-0827.458843
70OS9LVX X 2020-01-0929.744958
80OS9LVX X 2020-01-1026.061362
90OS9LVX X 2020-01-1134.992617
\n
"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_df['target'].describe()","execution_count":39,"outputs":[{"output_type":"execute_result","execution_count":39,"data":{"text/plain":"count 16136.000000\nmean 58.007864\nstd 35.462370\nmin 2.591207\n25% 31.461230\n50% 48.513680\n75% 75.257445\nmax 280.533302\nName: target, dtype: float64"},"metadata":{}}]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4} -------------------------------------------------------------------------------- /The Zimnat Insurance Assurance Challenge by #ZindiWeekendz/zindi_weekend_insurance_final_solution.ipynb: -------------------------------------------------------------------------------- 1 | {"cells":[{"metadata":{},"cell_type":"markdown","source":"# The Zimnat Insurance Assurance Challenge by #ZindiWeekendz - Farzi Data Scientists (Rank 3)\n\n*The data describes 51,685 life assurance policies, each identified by a unique Policy ID. Each year, some policies lapse as clients change jobs, move countries etc. The information on these policies are in multiple files.*\n\n*Numerical quantities have been transformed, and many categories have been assigned unique IDs in place of the original text.*\n\n*The objective of this hackathon is to develop a predictive model that determines the likelihood for a customer to churn - to seek an alternative insurer or simply stop paying for insurance altogether.*\n\n*sample_submission.csv contains rows for the remaining Policy IDs (with ‘?’s in TRAIN). You must predict which of these policies marked with '?' will lapse in 2020*\n\n* client_data.csv - Contains some personal information on the principal member, such as location, branch and agent code, age etc.\n* payment_history.csv - Contains payment history up to the end of 2018 tied to Policy ID. Payments made in 2019 are not provided.\n* policy_data.csv - Describes the policies themselves. There may be multiple rows for each Policy ID since policies can cover more than one person.\n* train.csv - contains a list of all the policies. Policies that lapsed in 2017, 2018 or 2019 are identified with a 1 in the ‘Lapse’ column, and the year is provided. The policies with a '?' in the 'Lapse' and 'Lapse Year' column are the policies that remained and had not lapsed as of the end of 2019. You must estimate the likelihood that these policies lapsed or not in 2020.\n* sample_submission.csv - is an example of what your submission should look like. The order of the rows does not matter but the name of the ID must be correct.\n* variable_defintions.txt - definitions of the variables"},{"metadata":{"trusted":true},"cell_type":"code","source":"from lightgbm import LGBMClassifier\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nfrom tqdm import *\nfrom sklearn.metrics import *\nimport warnings \nwarnings.simplefilter('ignore')\nfrom IPython.core.interactiveshell import InteractiveShell\nInteractiveShell.ast_node_interactive = \"all\"","execution_count":99,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\npayment_history = pd.read_csv('payment_history.csv')\nclient_data = pd.read_csv('client_data.csv')\npolicy_data = pd.read_csv('policy_data.csv')\ntrain = pd.read_csv('train.csv')\nsample_sub = pd.read_csv('sample_sub.csv')","execution_count":101,"outputs":[{"output_type":"stream","text":"CPU times: user 1.85 s, sys: 36 ms, total: 1.89 s\nWall time: 1.88 s\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"Our final solution used only **policy_data**.\n**client_data** cannot be used since it will not vary with time. (Or maybe it did, and we did not find out)\n**payment_history** was given only till 2018, and caused overfitting for 2020 test data, hence we ignored it in our final solution."},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data.shape, payment_history.shape","execution_count":102,"outputs":[{"output_type":"execute_result","execution_count":102,"data":{"text/plain":"((282815, 14), (495503, 5))"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data = policy_data.drop_duplicates()\npayment_history = payment_history.drop_duplicates()","execution_count":103,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data.shape, payment_history.shape","execution_count":104,"outputs":[{"output_type":"execute_result","execution_count":104,"data":{"text/plain":"((278988, 14), (482179, 5))"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data.nunique()","execution_count":105,"outputs":[{"output_type":"execute_result","execution_count":105,"data":{"text/plain":"Policy ID 51685\nNP2_EFFECTDATE 43\nPPR_PRODCD 17\nNPR_PREMIUM 2234\nNPH_LASTNAME 25275\nCLF_LIFECD 6\nNSP_SUBPROPOSAL 171\nNPR_SUMASSURED 1200\nNLO_TYPE 6\nNLO_AMOUNT 974\nAAG_AGCODE 591\nPCL_LOCATCODE 15\nOCCUPATION 240\nCATEGORY 6\ndtype: int64"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data.info()","execution_count":106,"outputs":[{"output_type":"stream","text":"\nInt64Index: 278988 entries, 0 to 282814\nData columns (total 14 columns):\n # Column Non-Null Count Dtype \n--- ------ -------------- ----- \n 0 Policy ID 278988 non-null object \n 1 NP2_EFFECTDATE 278988 non-null object \n 2 PPR_PRODCD 278988 non-null object \n 3 NPR_PREMIUM 278934 non-null float64\n 4 NPH_LASTNAME 278988 non-null object \n 5 CLF_LIFECD 278988 non-null int64 \n 6 NSP_SUBPROPOSAL 278988 non-null int64 \n 7 NPR_SUMASSURED 185357 non-null float64\n 8 NLO_TYPE 278988 non-null object \n 9 NLO_AMOUNT 88654 non-null float64\n 10 AAG_AGCODE 278988 non-null object \n 11 PCL_LOCATCODE 278988 non-null object \n 12 OCCUPATION 278988 non-null object \n 13 CATEGORY 278988 non-null object \ndtypes: float64(3), int64(2), object(9)\nmemory usage: 31.9+ MB\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data.head()","execution_count":107,"outputs":[{"output_type":"execute_result","execution_count":107,"data":{"text/plain":" Policy ID NP2_EFFECTDATE PPR_PRODCD NPR_PREMIUM \\\n0 PID_EPZDSP8 1/9/2019 PPR_PRODCD_B2KVCE7 265.724174 \n1 PID_6M6G9IB 1/8/2018 PPR_PRODCD_64QNIHM 2795.069380 \n2 PID_UL0F7LH 1/8/2017 PPR_PRODCD_KOFUYNN 2492.759107 \n3 PID_TRGUBTU 1/4/2018 PPR_PRODCD_KOFUYNN 3982.538095 \n4 PID_TODLPIB 1/12/2019 PPR_PRODCD_KOFUYNN 1143.953733 \n\n NPH_LASTNAME CLF_LIFECD NSP_SUBPROPOSAL NPR_SUMASSURED \\\n0 NPH_LASTNAME_BPN2LEB 2 222 NaN \n1 NPH_LASTNAME_U2H3GC6 1 111 213380.713197 \n2 NPH_LASTNAME_B68RERV 1 111 238857.872515 \n3 NPH_LASTNAME_NPN3VGI 1 111 74968.903115 \n4 NPH_LASTNAME_9VSNH0E 3 555 238857.872515 \n\n NLO_TYPE NLO_AMOUNT AAG_AGCODE PCL_LOCATCODE \\\n0 NLO_TYPE_DPBHSAH NaN AAG_AGCODE_APWOOPE PCL_LOCATCODE_7SHK7I9 \n1 NLO_TYPE_XTHV3A3 609.054794 AAG_AGCODE_9Z3FBGA PCL_LOCATCODE_7VFS3EQ \n2 NLO_TYPE_XAJI0Y6 1339.461987 AAG_AGCODE_Y0LKFF0 PCL_LOCATCODE_SKPRCR4 \n3 NLO_TYPE_XAJI0Y6 7870.961557 AAG_AGCODE_1OCF2N0 PCL_LOCATCODE_SPQHMX5 \n4 NLO_TYPE_DPBHSAH NaN AAG_AGCODE_E31VV8B PCL_LOCATCODE_0T6GYGX \n\n OCCUPATION CATEGORY \n0 OCCUPATION_NNHJ7XV CATEGORY_GWW4FYB \n1 OCCUPATION_IKCIDKW CATEGORY_R821UZV \n2 OCCUPATION_NUJZA7T CATEGORY_8DALFYO \n3 OCCUPATION_W9XA3KX CATEGORY_LXSLG6M \n4 OCCUPATION_NNHJ7XV CATEGORY_GWW4FYB ","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Policy IDNP2_EFFECTDATEPPR_PRODCDNPR_PREMIUMNPH_LASTNAMECLF_LIFECDNSP_SUBPROPOSALNPR_SUMASSUREDNLO_TYPENLO_AMOUNTAAG_AGCODEPCL_LOCATCODEOCCUPATIONCATEGORY
0PID_EPZDSP81/9/2019PPR_PRODCD_B2KVCE7265.724174NPH_LASTNAME_BPN2LEB2222NaNNLO_TYPE_DPBHSAHNaNAAG_AGCODE_APWOOPEPCL_LOCATCODE_7SHK7I9OCCUPATION_NNHJ7XVCATEGORY_GWW4FYB
1PID_6M6G9IB1/8/2018PPR_PRODCD_64QNIHM2795.069380NPH_LASTNAME_U2H3GC61111213380.713197NLO_TYPE_XTHV3A3609.054794AAG_AGCODE_9Z3FBGAPCL_LOCATCODE_7VFS3EQOCCUPATION_IKCIDKWCATEGORY_R821UZV
2PID_UL0F7LH1/8/2017PPR_PRODCD_KOFUYNN2492.759107NPH_LASTNAME_B68RERV1111238857.872515NLO_TYPE_XAJI0Y61339.461987AAG_AGCODE_Y0LKFF0PCL_LOCATCODE_SKPRCR4OCCUPATION_NUJZA7TCATEGORY_8DALFYO
3PID_TRGUBTU1/4/2018PPR_PRODCD_KOFUYNN3982.538095NPH_LASTNAME_NPN3VGI111174968.903115NLO_TYPE_XAJI0Y67870.961557AAG_AGCODE_1OCF2N0PCL_LOCATCODE_SPQHMX5OCCUPATION_W9XA3KXCATEGORY_LXSLG6M
4PID_TODLPIB1/12/2019PPR_PRODCD_KOFUYNN1143.953733NPH_LASTNAME_9VSNH0E3555238857.872515NLO_TYPE_DPBHSAHNaNAAG_AGCODE_E31VV8BPCL_LOCATCODE_0T6GYGXOCCUPATION_NNHJ7XVCATEGORY_GWW4FYB
\n
"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data['NP2_EFFECTDATE'] = pd.to_datetime(policy_data['NP2_EFFECTDATE'], format = '%d/%m/%Y')\nobj_cols = [c for c in policy_data.select_dtypes('object').columns.tolist() if c != 'Policy ID']\npolicy_data[obj_cols] = policy_data[obj_cols].apply(lambda x: pd.factorize(x)[0])\n\npolicy_data['NPR_PREMIUM - NLO_AMOUNT'] = policy_data['NPR_PREMIUM'] - policy_data['NLO_AMOUNT']\npolicy_data['NPR_PREMIUM / NPR_SUMASSURED'] = policy_data['NPR_PREMIUM'] / policy_data['NPR_SUMASSURED']\n\ndef get_last_payment_diff(x):\n try:\n return (x['NP2_EFFECTDATE'].values[-1] - x['NP2_EFFECTDATE'].values[-2])/ np.timedelta64(1, 'D')\n except:\n return np.nan\n\ndef get_pd_agg(pd):\n pd['policy_count'] = pd['Policy ID'].map(pd['Policy ID'].value_counts())\n aggs = {'NP2_EFFECTDATE': ['min', 'max', 'nunique', 'size'],\n 'NPR_PREMIUM': ['mean', 'min', 'max', 'sum', 'std', 'nunique'],\n 'NPR_SUMASSURED': ['mean','min', 'max', 'sum'],\n 'NLO_AMOUNT': ['mean', 'min', 'max', 'sum', 'std', 'nunique'],\n 'policy_count': ['sum', 'mean', 'std',],\n 'NPR_PREMIUM - NLO_AMOUNT': ['sum', 'mean', 'std'],\n 'NPR_PREMIUM / NPR_SUMASSURED': ['mean', 'std',]}\n pd_agg = pd.groupby('Policy ID').agg(aggs)\n pd_agg.columns = ['_'.join(c).strip('_') for c in pd_agg.columns]\n return pd_agg","execution_count":108,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"policy_data.columns","execution_count":109,"outputs":[{"output_type":"execute_result","execution_count":109,"data":{"text/plain":"Index(['Policy ID', 'NP2_EFFECTDATE', 'PPR_PRODCD', 'NPR_PREMIUM',\n 'NPH_LASTNAME', 'CLF_LIFECD', 'NSP_SUBPROPOSAL', 'NPR_SUMASSURED',\n 'NLO_TYPE', 'NLO_AMOUNT', 'AAG_AGCODE', 'PCL_LOCATCODE', 'OCCUPATION',\n 'CATEGORY', 'NPR_PREMIUM - NLO_AMOUNT', 'NPR_PREMIUM / NPR_SUMASSURED'],\n dtype='object')"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_features(df, year = 2020):\n '''\n This function calculates the stats and generates new features only upto the year specified.\n e.g: year = 2018 implies years 2019 and 2020 are ignored while creating new features\n '''\n \n ### Policy data features\n pdata = policy_data[policy_data['NP2_EFFECTDATE'].dt.year <= year]\n \n ### Filter the policy data so it contains data only upto the previous year\n prev_pdata = policy_data[policy_data['NP2_EFFECTDATE'].dt.year <= year-1]\n \n ### Get aggregate features upto current year\n pd_agg = get_pd_agg(pdata)\n df = pd.merge(left=df, right=pd_agg, on = 'Policy ID', how = 'left')\n \n ### Change in sum of numerical features between current year and previous year\n for c in ['NPR_SUMASSURED', 'NPR_PREMIUM', 'NLO_AMOUNT']:\n t1 = pdata.groupby('Policy ID')[c].sum()\n t2 = prev_pdata.groupby('Policy ID')[c].sum()\n t_diff = (t1 - t2).reset_index().rename({c: f'{c}_sum_change'}, axis=1).fillna(0)\n df = pd.merge(df, t_diff, on = 'Policy ID', how = 'left')\n \n \n ### Change and ratio of policy counts between current year and Previous year\n policy_cnt_curr_yr = pdata['Policy ID'].value_counts()\n policy_cnt_prev_yr = prev_pdata['Policy ID'].value_counts()\n \n policy_cnt_diff = (policy_cnt_curr_yr - policy_cnt_prev_yr).reset_index().rename({'index': 'Policy ID', 'Policy ID': 'policy_count_change'}, axis=1).fillna(0)\n df = pd.merge(df, policy_cnt_diff, on = 'Policy ID', how = 'left')\n \n policy_cnt_change_ratio = (policy_cnt_curr_yr / policy_cnt_prev_yr).reset_index().rename({'index': 'Policy ID', 'Policy ID': 'policy_count_change_ratio'}, axis=1).fillna(0)\n df = pd.merge(df, policy_cnt_change_ratio, on = 'Policy ID', how = 'left')\n \n ### Change in mean of numerical features between current year and previous year\n for c in ['NPR_PREMIUM', 'NPR_SUMASSURED','NLO_AMOUNT']:\n curr = pdata.groupby('Policy ID')[c].mean()\n prev = prev_pdata.groupby('Policy ID')[c].mean()\n diff = (curr - prev).reset_index().rename({'index': 'Policy ID', c: f'{c}_change'}, axis=1).fillna(0)\n df = pd.merge(df, diff, on = 'Policy ID', how = 'left')\n \n return df","execution_count":110,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Train using only the data upto 2018. Data upto 2018 implies any policies that have lapsed in 2019 and 2020, haven't lapsed yet (We are still in 2018 :P), so are 0.\n\ntrn_2k18 = train.copy()\ntrn_2k18['target'] = 1\ntrn_2k18.loc[~trn_2k18['Lapse Year'].isin(['2017', '2018']), 'target'] = 0\ntrn_2k18 = get_features(trn_2k18, year = 2018)","execution_count":111,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Validate using 2019's data\nval_2k19 = train[~train['Lapse Year'].isin(['2017', '2018'])]\nval_2k19['target'] = 1\nval_2k19.loc[val_2k19['Lapse Year'] == '?', 'target'] = 0\nval_2k19 = get_features(val_2k19, year = 2019)\n\n#### Train again using complete data, and predict on test data\ntrn_all = train.copy()\ntrn_all['target'] = 1\ntrn_all.loc[trn_all['Lapse Year'] == '?', 'target'] = 0\ntrn_all = get_features(trn_all, year = 2020)\n\ntest = train[train['Lapse Year'] == '?']\ntest = pd.merge(sample_sub[['Policy ID']], test, on = 'Policy ID', how = 'left')\ntest = get_features(test, year = 2020)\n\ntest.shape, sample_sub.shape","execution_count":112,"outputs":[{"output_type":"execute_result","execution_count":112,"data":{"text/plain":"((43707, 39), (43707, 2))"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Highly imbalanced dataset.\ntrn_2k18['target'].value_counts(normalize=True)","execution_count":113,"outputs":[{"output_type":"execute_result","execution_count":113,"data":{"text/plain":"0 0.942517\n1 0.057483\nName: target, dtype: float64"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Highly imbalanced dataset.\ntrn_all['target'].value_counts(normalize=True)","execution_count":114,"outputs":[{"output_type":"execute_result","execution_count":114,"data":{"text/plain":"0 0.845642\n1 0.154358\nName: target, dtype: float64"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Try setting all the submission values, to this constant value, you will get 0.30xxx as score, without doing anything. That was a baseline :D\n### 0.057 -> 1 count for year upto 2018\n### 0.154 -> 1 count for year upto 2019\n\n(0.057 + 0.154)/2","execution_count":115,"outputs":[{"output_type":"execute_result","execution_count":115,"data":{"text/plain":"0.1055"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Some features were overfitting hence ignored\n\nfeatures = [c for c in trn_all.columns if c not in ['Policy ID', 'Lapse', 'Lapse Year', 'NP2_EFFECTDATE_min', 'NP2_EFFECTDATE_max', 'target']]\nfeatures","execution_count":116,"outputs":[{"output_type":"execute_result","execution_count":116,"data":{"text/plain":"['NP2_EFFECTDATE_nunique',\n 'NP2_EFFECTDATE_size',\n 'NPR_PREMIUM_mean',\n 'NPR_PREMIUM_min',\n 'NPR_PREMIUM_max',\n 'NPR_PREMIUM_sum',\n 'NPR_PREMIUM_std',\n 'NPR_PREMIUM_nunique',\n 'NPR_SUMASSURED_mean',\n 'NPR_SUMASSURED_min',\n 'NPR_SUMASSURED_max',\n 'NPR_SUMASSURED_sum',\n 'NLO_AMOUNT_mean',\n 'NLO_AMOUNT_min',\n 'NLO_AMOUNT_max',\n 'NLO_AMOUNT_sum',\n 'NLO_AMOUNT_std',\n 'NLO_AMOUNT_nunique',\n 'policy_count_sum',\n 'policy_count_mean',\n 'policy_count_std',\n 'NPR_PREMIUM - NLO_AMOUNT_sum',\n 'NPR_PREMIUM - NLO_AMOUNT_mean',\n 'NPR_PREMIUM - NLO_AMOUNT_std',\n 'NPR_PREMIUM / NPR_SUMASSURED_mean',\n 'NPR_PREMIUM / NPR_SUMASSURED_std',\n 'NPR_SUMASSURED_sum_change',\n 'NPR_PREMIUM_sum_change',\n 'NLO_AMOUNT_sum_change',\n 'policy_count_change',\n 'policy_count_change_ratio',\n 'NPR_PREMIUM_change',\n 'NPR_SUMASSURED_change',\n 'NLO_AMOUNT_change']"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"trn_all.head()","execution_count":117,"outputs":[{"output_type":"execute_result","execution_count":117,"data":{"text/plain":" Policy ID Lapse Lapse Year target NP2_EFFECTDATE_min NP2_EFFECTDATE_max \\\n0 PID_4928TWH ? ? 0 2017-08-01 2017-08-01 \n1 PID_KBLLEGK ? ? 0 2018-05-01 2018-05-01 \n2 PID_90F0QA3 ? ? 0 2019-09-01 2019-09-01 \n3 PID_18F3NHF ? ? 0 2019-12-01 2019-12-01 \n4 PID_SX4QUVO ? ? 0 2019-03-01 2019-03-01 \n\n NP2_EFFECTDATE_nunique NP2_EFFECTDATE_size NPR_PREMIUM_mean \\\n0 1 1 42911.077278 \n1 1 2 3561.268991 \n2 1 2 6164.812836 \n3 1 1 2278.189789 \n4 1 1 1619.046308 \n\n NPR_PREMIUM_min ... NPR_PREMIUM / NPR_SUMASSURED_mean \\\n0 42911.077278 ... 0.125962 \n1 3561.268991 ... 0.021576 \n2 6164.812836 ... 0.024418 \n3 2278.189789 ... 0.009538 \n4 1619.046308 ... 0.011165 \n\n NPR_PREMIUM / NPR_SUMASSURED_std NPR_SUMASSURED_sum_change \\\n0 NaN 0.0 \n1 0.0 0.0 \n2 0.0 0.0 \n3 NaN 0.0 \n4 NaN 0.0 \n\n NPR_PREMIUM_sum_change NLO_AMOUNT_sum_change policy_count_change \\\n0 0.0 0.0 0.0 \n1 0.0 0.0 0.0 \n2 0.0 0.0 0.0 \n3 0.0 0.0 0.0 \n4 0.0 0.0 0.0 \n\n policy_count_change_ratio NPR_PREMIUM_change NPR_SUMASSURED_change \\\n0 1.0 0.0 0.0 \n1 1.0 0.0 0.0 \n2 1.0 0.0 0.0 \n3 1.0 0.0 0.0 \n4 1.0 0.0 0.0 \n\n NLO_AMOUNT_change \n0 0.0 \n1 0.0 \n2 0.0 \n3 0.0 \n4 0.0 \n\n[5 rows x 40 columns]","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Policy IDLapseLapse YeartargetNP2_EFFECTDATE_minNP2_EFFECTDATE_maxNP2_EFFECTDATE_nuniqueNP2_EFFECTDATE_sizeNPR_PREMIUM_meanNPR_PREMIUM_min...NPR_PREMIUM / NPR_SUMASSURED_meanNPR_PREMIUM / NPR_SUMASSURED_stdNPR_SUMASSURED_sum_changeNPR_PREMIUM_sum_changeNLO_AMOUNT_sum_changepolicy_count_changepolicy_count_change_ratioNPR_PREMIUM_changeNPR_SUMASSURED_changeNLO_AMOUNT_change
0PID_4928TWH??02017-08-012017-08-011142911.07727842911.077278...0.125962NaN0.00.00.00.01.00.00.00.0
1PID_KBLLEGK??02018-05-012018-05-01123561.2689913561.268991...0.0215760.00.00.00.00.01.00.00.00.0
2PID_90F0QA3??02019-09-012019-09-01126164.8128366164.812836...0.0244180.00.00.00.00.01.00.00.00.0
3PID_18F3NHF??02019-12-012019-12-01112278.1897892278.189789...0.009538NaN0.00.00.00.01.00.00.00.0
4PID_SX4QUVO??02019-03-012019-03-01111619.0463081619.046308...0.011165NaN0.00.00.00.01.00.00.00.0
\n

5 rows × 40 columns

\n
"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Some features can be removed still\n\ndrop_features = [c for c in features if trn_all[c].sum() == 0]\nfeatures = [c for c in features if c not in drop_features]\nprint(drop_features)","execution_count":118,"outputs":[{"output_type":"stream","text":"['policy_count_std']\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"### Some features were overfitting hence ignored\n\nfeatures = [c for c in features if c not in ['AMOUNTPAID', 'NP2_EFFECTDATE_days_diff', 'policy_count_std']]","execution_count":119,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training on 2018, validation on 2019"},{"metadata":{"trusted":true},"cell_type":"code","source":"clf = LGBMClassifier(num_leaves=32,max_depth=18, learning_rate=0.01, reg_alpha=1,reg_lambda=1, n_estimators=600, subsample=1, subsample_freq=1, colsample_bytree=0.55)\nclf.fit(trn_2k18[features], trn_2k18['target'], eval_set = [(val_2k19[features], val_2k19['target'])], verbose = 20)","execution_count":120,"outputs":[{"output_type":"stream","text":"[20]\tvalid_0's binary_logloss: 0.317732\n[40]\tvalid_0's binary_logloss: 0.300291\n[60]\tvalid_0's binary_logloss: 0.291181\n[80]\tvalid_0's binary_logloss: 0.283651\n[100]\tvalid_0's binary_logloss: 0.278971\n[120]\tvalid_0's binary_logloss: 0.275966\n[140]\tvalid_0's binary_logloss: 0.273309\n[160]\tvalid_0's binary_logloss: 0.271762\n[180]\tvalid_0's binary_logloss: 0.270549\n[200]\tvalid_0's binary_logloss: 0.269712\n[220]\tvalid_0's binary_logloss: 0.269263\n[240]\tvalid_0's binary_logloss: 0.268986\n[260]\tvalid_0's binary_logloss: 0.268713\n[280]\tvalid_0's binary_logloss: 0.268712\n[300]\tvalid_0's binary_logloss: 0.268737\n[320]\tvalid_0's binary_logloss: 0.268734\n[340]\tvalid_0's binary_logloss: 0.268875\n[360]\tvalid_0's binary_logloss: 0.269038\n[380]\tvalid_0's binary_logloss: 0.269172\n[400]\tvalid_0's binary_logloss: 0.269256\n[420]\tvalid_0's binary_logloss: 0.269424\n[440]\tvalid_0's binary_logloss: 0.269646\n[460]\tvalid_0's binary_logloss: 0.269832\n[480]\tvalid_0's binary_logloss: 0.270026\n[500]\tvalid_0's binary_logloss: 0.270278\n[520]\tvalid_0's binary_logloss: 0.270428\n[540]\tvalid_0's binary_logloss: 0.270593\n[560]\tvalid_0's binary_logloss: 0.270737\n[580]\tvalid_0's binary_logloss: 0.270912\n[600]\tvalid_0's binary_logloss: 0.271107\n","name":"stdout"},{"output_type":"execute_result","execution_count":120,"data":{"text/plain":"LGBMClassifier(boosting_type='gbdt', class_weight=None, colsample_bytree=0.55,\n importance_type='split', learning_rate=0.01, max_depth=18,\n min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0,\n n_estimators=600, n_jobs=-1, num_leaves=32, objective=None,\n random_state=None, reg_alpha=1, reg_lambda=1, silent=True,\n subsample=1, subsample_for_bin=200000, subsample_freq=1)"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"## Training on complete data.\n\n* The number of iterations have been increased a little more compared to the best iteration for validation"},{"metadata":{"trusted":true},"cell_type":"code","source":"tp = pd.DataFrame()\nfor i in tqdm_notebook(range(7)):\n clf = LGBMClassifier(num_leaves=32, learning_rate=0.01, reg_alpha=2, n_estimators=380, subsample=1, subsample_freq=1, colsample_bytree=0.5, random_state=2**i)\n clf.fit(trn_all[features], trn_all['target'], eval_set = [(val_2k19[features], val_2k19['target'])], verbose = 380)\n tp[i] = clf.predict_proba(test[features])[:, 1]\ntest_preds = tp.mean(axis=1)","execution_count":121,"outputs":[{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=7.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"bd2a68089e004589bd43bf907aa05045"}},"metadata":{}},{"output_type":"stream","text":"[380]\tvalid_0's binary_logloss: 0.252244\n[380]\tvalid_0's binary_logloss: 0.251058\n[380]\tvalid_0's binary_logloss: 0.251556\n[380]\tvalid_0's binary_logloss: 0.251686\n[380]\tvalid_0's binary_logloss: 0.251647\n[380]\tvalid_0's binary_logloss: 0.251662\n[380]\tvalid_0's binary_logloss: 0.251376\n\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"Lets see some stats about our target"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_preds = tp.mean(axis=1)\nprint(pd.Series(test_preds).describe())","execution_count":122,"outputs":[{"output_type":"stream","text":"count 43707.000000\nmean 0.128244\nstd 0.130293\nmin 0.003687\n25% 0.003703\n50% 0.104520\n75% 0.234366\nmax 0.653318\ndtype: float64\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"This thresholding greatly improved our score, from 0.260x to 0.24xx :D. The intuition is log loss heavily penalizes confident wrong predictions. Also most of the values were 0, so using a threshold to rectify it, seemed ok."},{"metadata":{"trusted":true},"cell_type":"code","source":"test_preds[test_preds < 0.03] = 0\ntest_preds[test_preds > 0.18] = 0.18\npd.Series(test_preds).describe()","execution_count":123,"outputs":[{"output_type":"execute_result","execution_count":123,"data":{"text/plain":"count 43707.000000\nmean 0.090678\nstd 0.086338\nmin 0.000000\n25% 0.000000\n50% 0.104520\n75% 0.180000\nmax 0.180000\ndtype: float64"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"SUB_FILE_NAME = 'all_preds.csv'\nsample_sub['Lapse'] = test_preds\nsample_sub.to_csv(SUB_FILE_NAME, index=False)\n\nfrom IPython.display import HTML\ndef create_download_link(title = \"Download CSV file\", filename = \"data.csv\"): \n html = '{title}'\n html = html.format(title=title,filename=filename)\n return HTML(html)\ncreate_download_link(filename = SUB_FILE_NAME)","execution_count":124,"outputs":[{"output_type":"execute_result","execution_count":124,"data":{"text/plain":"","text/html":"Download CSV file"},"metadata":{}}]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4} -------------------------------------------------------------------------------- /Urban Air Pollution Challenge by #ZindiWeekendz/zindi-weekendz-pollution-v4.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", 8 | "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5" 9 | }, 10 | "outputs": [], 11 | "source": [ 12 | "import numpy as np\n", 13 | "import pandas as pd\n", 14 | "import lightgbm as lgb\n", 15 | "import gc\n", 16 | "from fastai.tabular import *\n", 17 | "from sklearn.metrics import mean_squared_error as mse\n", 18 | "from sklearn.model_selection import KFold, StratifiedKFold\n", 19 | "from IPython.core.interactiveshell import InteractiveShell\n", 20 | "InteractiveShell.ast_node_interactivity = \"all\"\n", 21 | "import datetime\n", 22 | "from tqdm import tqdm_notebook" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 2, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "def rmse(y_true, y_pred):\n", 32 | " return np.sqrt(mse(y_true, y_pred))" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 3, 38 | "metadata": { 39 | "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0", 40 | "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a" 41 | }, 42 | "outputs": [], 43 | "source": [ 44 | "train = pd.read_csv('/kaggle/input/zindi-weekendz-pollution/Train.csv')\n", 45 | "test = pd.read_csv('/kaggle/input/zindi-weekendz-pollution/Test.csv')\n", 46 | "sample_sub = pd.read_csv('/kaggle/input/zindi-weekendz-pollution/SampleSubmission.csv')\n", 47 | "\n", 48 | "train['Date'] = pd.to_datetime(train['Date'], format='%Y-%m-%d')\n", 49 | "test['Date'] = pd.to_datetime(test['Date'], format='%Y-%m-%d')\n", 50 | "\n", 51 | "ID_COL, TARGET_COL = 'Place_ID X Date', 'target'" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 4, 57 | "metadata": {}, 58 | "outputs": [ 59 | { 60 | "name": "stderr", 61 | "output_type": "stream", 62 | "text": [ 63 | "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:1: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n", 64 | "of pandas will change to not sort by default.\n", 65 | "\n", 66 | "To accept the future behavior, pass 'sort=False'.\n", 67 | "\n", 68 | "To retain the current behavior and silence the warning, pass 'sort=True'.\n", 69 | "\n", 70 | " \"\"\"Entry point for launching an IPython kernel.\n" 71 | ] 72 | }, 73 | { 74 | "data": { 75 | "text/plain": [ 76 | "38" 77 | ] 78 | }, 79 | "execution_count": 4, 80 | "metadata": {}, 81 | "output_type": "execute_result" 82 | } 83 | ], 84 | "source": [ 85 | "df = pd.concat([train, test]).reset_index(drop=True)\n", 86 | "features = [c for c in df.columns if c not in ['Date', 'target_count', 'target_min', 'Place_ID X Date', 'target_variance', 'Place_ID', 'target_max', 'target']]\n", 87 | "simple_feats = [c for c in features if ('angle' not in c) & ('height' not in c) & ('altittude' not in c)]\n", 88 | "len(simple_feats)" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": 5, 94 | "metadata": {}, 95 | "outputs": [ 96 | { 97 | "name": "stderr", 98 | "output_type": "stream", 99 | "text": [ 100 | "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:6: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0\n", 101 | "Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`\n", 102 | " \n" 103 | ] 104 | }, 105 | { 106 | "data": { 107 | "application/vnd.jupyter.widget-view+json": { 108 | "model_id": "cbea4e3d24a645eeb3733f9508cf69ef", 109 | "version_major": 2, 110 | "version_minor": 0 111 | }, 112 | "text/plain": [ 113 | "HBox(children=(FloatProgress(value=0.0, max=24.0), HTML(value='')))" 114 | ] 115 | }, 116 | "metadata": {}, 117 | "output_type": "display_data" 118 | }, 119 | { 120 | "name": "stdout", 121 | "output_type": "stream", 122 | "text": [ 123 | "\n" 124 | ] 125 | }, 126 | { 127 | "name": "stderr", 128 | "output_type": "stream", 129 | "text": [ 130 | "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:10: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0\n", 131 | "Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`\n", 132 | " # Remove the CWD from sys.path while we load stuff.\n" 133 | ] 134 | }, 135 | { 136 | "data": { 137 | "application/vnd.jupyter.widget-view+json": { 138 | "model_id": "a6234022dc9d4ea983e62bc863344757", 139 | "version_major": 2, 140 | "version_minor": 0 141 | }, 142 | "text/plain": [ 143 | "HBox(children=(FloatProgress(value=0.0, max=44.0), HTML(value='')))" 144 | ] 145 | }, 146 | "metadata": {}, 147 | "output_type": "display_data" 148 | }, 149 | { 150 | "name": "stdout", 151 | "output_type": "stream", 152 | "text": [ 153 | "\n" 154 | ] 155 | }, 156 | { 157 | "name": "stderr", 158 | "output_type": "stream", 159 | "text": [ 160 | "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:15: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0\n", 161 | "Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`\n", 162 | " from ipykernel import kernelapp as app\n" 163 | ] 164 | }, 165 | { 166 | "data": { 167 | "application/vnd.jupyter.widget-view+json": { 168 | "model_id": "41d874433a334d6abda65442beba3550", 169 | "version_major": 2, 170 | "version_minor": 0 171 | }, 172 | "text/plain": [ 173 | "HBox(children=(FloatProgress(value=0.0, max=21.0), HTML(value='')))" 174 | ] 175 | }, 176 | "metadata": {}, 177 | "output_type": "display_data" 178 | }, 179 | { 180 | "name": "stdout", 181 | "output_type": "stream", 182 | "text": [ 183 | "\n" 184 | ] 185 | }, 186 | { 187 | "data": { 188 | "text/html": [ 189 | "
\n", 190 | "\n", 203 | "\n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | " \n", 213 | " \n", 214 | " \n", 215 | " \n", 216 | " \n", 217 | " \n", 218 | " \n", 219 | " \n", 220 | " \n", 221 | " \n", 222 | " \n", 223 | " \n", 224 | " \n", 225 | " \n", 226 | " \n", 227 | " \n", 228 | " \n", 229 | " \n", 230 | " \n", 231 | " \n", 232 | " \n", 233 | " \n", 234 | " \n", 235 | " \n", 236 | " \n", 237 | " \n", 238 | " \n", 239 | " \n", 240 | " \n", 241 | " \n", 242 | " \n", 243 | " \n", 244 | " \n", 245 | " \n", 246 | " \n", 247 | " \n", 248 | " \n", 249 | " \n", 250 | " \n", 251 | " \n", 252 | " \n", 253 | " \n", 254 | " \n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | " \n", 269 | " \n", 270 | " \n", 271 | " \n", 272 | " \n", 273 | " \n", 274 | " \n", 275 | " \n", 276 | " \n", 277 | " \n", 278 | " \n", 279 | " \n", 280 | " \n", 281 | " \n", 282 | " \n", 283 | " \n", 284 | " \n", 285 | " \n", 286 | " \n", 287 | " \n", 288 | " \n", 289 | " \n", 290 | " \n", 291 | " \n", 292 | " \n", 293 | " \n", 294 | " \n", 295 | " \n", 296 | " \n", 297 | " \n", 298 | " \n", 299 | " \n", 300 | " \n", 301 | " \n", 302 | " \n", 303 | " \n", 304 | " \n", 305 | " \n", 306 | " \n", 307 | " \n", 308 | " \n", 309 | " \n", 310 | " \n", 311 | " \n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | " \n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | " \n", 411 | " \n", 412 | " \n", 413 | " \n", 414 | " \n", 415 | " \n", 416 | " \n", 417 | " \n", 418 | " \n", 419 | " \n", 420 | " \n", 421 | " \n", 422 | " \n", 423 | " \n", 424 | " \n", 425 | " \n", 426 | " \n", 427 | " \n", 428 | " \n", 429 | " \n", 430 | " \n", 431 | " \n", 432 | " \n", 433 | " \n", 434 | " \n", 435 | " \n", 436 | " \n", 437 | " \n", 438 | " \n", 439 | " \n", 440 | " \n", 441 | " \n", 442 | " \n", 443 | " \n", 444 | " \n", 445 | " \n", 446 | " \n", 447 | " \n", 448 | " \n", 449 | " \n", 450 | " \n", 451 | " \n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | "
L3_AER_AI_absorbing_aerosol_indexL3_AER_AI_sensor_altitudeL3_AER_AI_sensor_azimuth_angleL3_AER_AI_sensor_zenith_angleL3_AER_AI_solar_azimuth_angleL3_AER_AI_solar_zenith_angleL3_CH4_CH4_column_volume_mixing_ratio_dry_airL3_CH4_aerosol_heightL3_CH4_aerosol_optical_depthL3_CH4_sensor_azimuth_angle...fortnightwhich_fortnightCurrent_Date_weekday_cosCurrent_Date_weekday_sinCurrent_Date_day_month_cosCurrent_Date_day_month_sinCurrent_Date_month_year_cosCurrent_Date_month_year_sinCurrent_Date_day_year_cosCurrent_Date_day_year_sin
0-1.231330840209.87461976.53751238.634284-61.73671922.3581671793.7935793227.8554690.01057974.481049...20-0.9009690.4338840.9795300.2012991.000000e+000.0000000.9998530.017166
1-1.082553840772.941995-14.70803659.624912-67.69350928.6148041789.9604493384.2265620.01510475.630043...30-0.900969-0.4338840.9189580.3943561.000000e+000.0000000.9994110.034328
2-1.001241841410.713456-105.20133849.839714-78.34270134.296977NaNNaNNaNNaN...40-0.222521-0.9749280.8207630.5712681.000000e+000.0000000.9986740.051479
3-0.777023841103.242368-104.33405629.180977-73.89657230.545393NaNNaNNaNNaN...500.623490-0.7818310.6889670.7247931.000000e+000.0000000.9976430.068615
40.366324840763.05549958.8501790.797294-68.61248026.899694NaNNaNNaNNaN...601.0000000.0000000.5289640.8486441.000000e+000.0000000.9963180.085731
..................................................................
46688-1.011116833312.35369163.24109051.611371-176.52172038.857262NaNNaNNaNNaN...120.6234900.7818310.979530-0.2012995.000000e-010.8660250.0257480.999668
46689-1.363941833125.012462-17.86984755.06552115.07937940.886310NaNNaNNaNNaN...10-0.2225210.9749281.0000000.0000006.123234e-171.0000000.0085830.999963
46690-1.776128832573.386814-99.64771427.132738-152.25415641.190326NaNNaNNaNNaN...20-0.9009690.4338840.9781480.2079126.123234e-171.000000-0.0085830.999963
46691-2.971064832687.22302438.7392180.564620-158.88335239.4311930.0000000.0000000.0000000.000000...30-0.900969-0.4338840.9135450.4067376.123234e-171.000000-0.0257480.999668
46692-1.582332832878.41826669.81206227.292029-165.90042938.047473NaNNaNNaNNaN...40-0.222521-0.9749280.8090170.5877856.123234e-171.000000-0.0429050.999079
\n", 497 | "

46693 rows × 3732 columns

\n", 498 | "
" 499 | ], 500 | "text/plain": [ 501 | " L3_AER_AI_absorbing_aerosol_index L3_AER_AI_sensor_altitude \\\n", 502 | "0 -1.231330 840209.874619 \n", 503 | "1 -1.082553 840772.941995 \n", 504 | "2 -1.001241 841410.713456 \n", 505 | "3 -0.777023 841103.242368 \n", 506 | "4 0.366324 840763.055499 \n", 507 | "... ... ... \n", 508 | "46688 -1.011116 833312.353691 \n", 509 | "46689 -1.363941 833125.012462 \n", 510 | "46690 -1.776128 832573.386814 \n", 511 | "46691 -2.971064 832687.223024 \n", 512 | "46692 -1.582332 832878.418266 \n", 513 | "\n", 514 | " L3_AER_AI_sensor_azimuth_angle L3_AER_AI_sensor_zenith_angle \\\n", 515 | "0 76.537512 38.634284 \n", 516 | "1 -14.708036 59.624912 \n", 517 | "2 -105.201338 49.839714 \n", 518 | "3 -104.334056 29.180977 \n", 519 | "4 58.850179 0.797294 \n", 520 | "... ... ... \n", 521 | "46688 63.241090 51.611371 \n", 522 | "46689 -17.869847 55.065521 \n", 523 | "46690 -99.647714 27.132738 \n", 524 | "46691 38.739218 0.564620 \n", 525 | "46692 69.812062 27.292029 \n", 526 | "\n", 527 | " L3_AER_AI_solar_azimuth_angle L3_AER_AI_solar_zenith_angle \\\n", 528 | "0 -61.736719 22.358167 \n", 529 | "1 -67.693509 28.614804 \n", 530 | "2 -78.342701 34.296977 \n", 531 | "3 -73.896572 30.545393 \n", 532 | "4 -68.612480 26.899694 \n", 533 | "... ... ... \n", 534 | "46688 -176.521720 38.857262 \n", 535 | "46689 15.079379 40.886310 \n", 536 | "46690 -152.254156 41.190326 \n", 537 | "46691 -158.883352 39.431193 \n", 538 | "46692 -165.900429 38.047473 \n", 539 | "\n", 540 | " L3_CH4_CH4_column_volume_mixing_ratio_dry_air L3_CH4_aerosol_height \\\n", 541 | "0 1793.793579 3227.855469 \n", 542 | "1 1789.960449 3384.226562 \n", 543 | "2 NaN NaN \n", 544 | "3 NaN NaN \n", 545 | "4 NaN NaN \n", 546 | "... ... ... \n", 547 | "46688 NaN NaN \n", 548 | "46689 NaN NaN \n", 549 | "46690 NaN NaN \n", 550 | "46691 0.000000 0.000000 \n", 551 | "46692 NaN NaN \n", 552 | "\n", 553 | " L3_CH4_aerosol_optical_depth L3_CH4_sensor_azimuth_angle ... \\\n", 554 | "0 0.010579 74.481049 ... \n", 555 | "1 0.015104 75.630043 ... \n", 556 | "2 NaN NaN ... \n", 557 | "3 NaN NaN ... \n", 558 | "4 NaN NaN ... \n", 559 | "... ... ... ... \n", 560 | "46688 NaN NaN ... \n", 561 | "46689 NaN NaN ... \n", 562 | "46690 NaN NaN ... \n", 563 | "46691 0.000000 0.000000 ... \n", 564 | "46692 NaN NaN ... \n", 565 | "\n", 566 | " fortnight which_fortnight Current_Date_weekday_cos \\\n", 567 | "0 2 0 -0.900969 \n", 568 | "1 3 0 -0.900969 \n", 569 | "2 4 0 -0.222521 \n", 570 | "3 5 0 0.623490 \n", 571 | "4 6 0 1.000000 \n", 572 | "... ... ... ... \n", 573 | "46688 1 2 0.623490 \n", 574 | "46689 1 0 -0.222521 \n", 575 | "46690 2 0 -0.900969 \n", 576 | "46691 3 0 -0.900969 \n", 577 | "46692 4 0 -0.222521 \n", 578 | "\n", 579 | " Current_Date_weekday_sin Current_Date_day_month_cos \\\n", 580 | "0 0.433884 0.979530 \n", 581 | "1 -0.433884 0.918958 \n", 582 | "2 -0.974928 0.820763 \n", 583 | "3 -0.781831 0.688967 \n", 584 | "4 0.000000 0.528964 \n", 585 | "... ... ... \n", 586 | "46688 0.781831 0.979530 \n", 587 | "46689 0.974928 1.000000 \n", 588 | "46690 0.433884 0.978148 \n", 589 | "46691 -0.433884 0.913545 \n", 590 | "46692 -0.974928 0.809017 \n", 591 | "\n", 592 | " Current_Date_day_month_sin Current_Date_month_year_cos \\\n", 593 | "0 0.201299 1.000000e+00 \n", 594 | "1 0.394356 1.000000e+00 \n", 595 | "2 0.571268 1.000000e+00 \n", 596 | "3 0.724793 1.000000e+00 \n", 597 | "4 0.848644 1.000000e+00 \n", 598 | "... ... ... \n", 599 | "46688 -0.201299 5.000000e-01 \n", 600 | "46689 0.000000 6.123234e-17 \n", 601 | "46690 0.207912 6.123234e-17 \n", 602 | "46691 0.406737 6.123234e-17 \n", 603 | "46692 0.587785 6.123234e-17 \n", 604 | "\n", 605 | " Current_Date_month_year_sin Current_Date_day_year_cos \\\n", 606 | "0 0.000000 0.999853 \n", 607 | "1 0.000000 0.999411 \n", 608 | "2 0.000000 0.998674 \n", 609 | "3 0.000000 0.997643 \n", 610 | "4 0.000000 0.996318 \n", 611 | "... ... ... \n", 612 | "46688 0.866025 0.025748 \n", 613 | "46689 1.000000 0.008583 \n", 614 | "46690 1.000000 -0.008583 \n", 615 | "46691 1.000000 -0.025748 \n", 616 | "46692 1.000000 -0.042905 \n", 617 | "\n", 618 | " Current_Date_day_year_sin \n", 619 | "0 0.017166 \n", 620 | "1 0.034328 \n", 621 | "2 0.051479 \n", 622 | "3 0.068615 \n", 623 | "4 0.085731 \n", 624 | "... ... \n", 625 | "46688 0.999668 \n", 626 | "46689 0.999963 \n", 627 | "46690 0.999963 \n", 628 | "46691 0.999668 \n", 629 | "46692 0.999079 \n", 630 | "\n", 631 | "[46693 rows x 3732 columns]" 632 | ] 633 | }, 634 | "execution_count": 5, 635 | "metadata": {}, 636 | "output_type": "execute_result" 637 | } 638 | ], 639 | "source": [ 640 | "df['placeID_freq'] = df['Place_ID'].map(df['Place_ID'].value_counts())\n", 641 | "\n", 642 | "for i in range(1, 20):\n", 643 | " df[f'prev_target_{i}'] = df.sort_values(by='Date')[TARGET_COL].fillna(method='ffill').shift(i).sort_index()\n", 644 | " df[f'next_target_{i}'] = df.sort_values(by='Date')[TARGET_COL].fillna(method='bfill').shift(-i).sort_index()\n", 645 | "for i in tqdm_notebook(range(1, 25)):\n", 646 | " df[f'magic_{i}'] = df.sort_values(by='Date')[TARGET_COL].shift(i).expanding().mean().fillna(method='ffill').sort_index()\n", 647 | " df[f'magic2_{i}'] = df.sort_values(by='Date')[TARGET_COL].shift(-i).expanding().mean().fillna(method='bfill').sort_index()\n", 648 | " \n", 649 | "for i in tqdm_notebook(range(1, 45)):\n", 650 | " c = 'Date' + str(i+1)\n", 651 | " df[c] = df['Date'] + datetime.timedelta(days=i)\n", 652 | " _ = add_cyclic_datepart(df, c, prefix = c)\n", 653 | " \n", 654 | "for i in tqdm_notebook(range(1, 22)):\n", 655 | " tmp = df.sort_values(by='Date').groupby('Place_ID')[simple_feats].shift(i).sort_index()\n", 656 | " tmp_diff_prev = df[simple_feats] - tmp\n", 657 | " tmp.columns = [c + f'_prev_{i}' for c in tmp.columns]\n", 658 | " tmp_diff_prev.columns = [c + f'_prev_diff_{i}' for c in tmp_diff_prev.columns]\n", 659 | " df = pd.concat([df, tmp, tmp_diff_prev], axis=1)\n", 660 | " \n", 661 | " tmp = df.sort_values(by='Date').groupby('Place_ID')[simple_feats].shift(-i).sort_index()\n", 662 | " tmp_diff_next = df[simple_feats] - tmp\n", 663 | " tmp.columns = [c + f'_next_{i}' for c in tmp.columns]\n", 664 | " tmp_diff_next.columns = [c + f'_next_diff_{i}' for c in tmp_diff_next.columns]\n", 665 | " df = pd.concat([df, tmp, tmp_diff_next], axis=1)\n", 666 | " \n", 667 | "for attr in ['day', 'month', 'week', 'dayofweek', 'weekofyear', 'days_in_month', 'is_month_start', 'is_month_end', 'dayofyear']:\n", 668 | " df[attr] = getattr(df['Date'].dt, attr)\n", 669 | "df['is_weekend'] = (df['dayofweek'] >= 5)*1\n", 670 | "df['fortnight'] = df['day']%15\n", 671 | "df['which_fortnight'] = df['day']//15\n", 672 | "\n", 673 | "add_cyclic_datepart(df, \"Date\", prefix = \"Current_Date_\")" 674 | ] 675 | }, 676 | { 677 | "cell_type": "code", 678 | "execution_count": 6, 679 | "metadata": {}, 680 | "outputs": [ 681 | { 682 | "data": { 683 | "text/plain": [ 684 | "3725" 685 | ] 686 | }, 687 | "execution_count": 6, 688 | "metadata": {}, 689 | "output_type": "execute_result" 690 | } 691 | ], 692 | "source": [ 693 | "features = [c for c in df.columns if c not in ['Date', 'target_count', 'target_min', 'Place_ID X Date', 'target_variance', 'Place_ID',\n", 694 | " 'target_max', 'target', 'month_year_cos','month_year_sin','day_year_cos','day_year_sin']]\n", 695 | "train = df[:train.shape[0]].reset_index(drop=True)\n", 696 | "test = df[train.shape[0]:].reset_index(drop=True)\n", 697 | "target = train[TARGET_COL]\n", 698 | "len(features)" 699 | ] 700 | }, 701 | { 702 | "cell_type": "code", 703 | "execution_count": 7, 704 | "metadata": {}, 705 | "outputs": [ 706 | { 707 | "data": { 708 | "text/plain": [ 709 | "0" 710 | ] 711 | }, 712 | "execution_count": 7, 713 | "metadata": {}, 714 | "output_type": "execute_result" 715 | } 716 | ], 717 | "source": [ 718 | "del df\n", 719 | "gc.collect()" 720 | ] 721 | }, 722 | { 723 | "cell_type": "code", 724 | "execution_count": 8, 725 | "metadata": {}, 726 | "outputs": [], 727 | "source": [ 728 | "param = {'num_leaves': 100,\n", 729 | " 'min_data_in_leaf': 40, \n", 730 | " 'objective':'regression',\n", 731 | " 'max_depth': -1,\n", 732 | " 'learning_rate': 0.05,\n", 733 | " \"boosting\": \"gbdt\",\n", 734 | " \"feature_fraction\": 0.35,\n", 735 | " \"metric\": 'auc',\n", 736 | " \"lambda_l1\": 1,\n", 737 | " \"lambda_l2\": 1,\n", 738 | " \"random_state\": 6,\n", 739 | " \"verbosity\": -1,\n", 740 | " 'metric' : 'rmse',\n", 741 | " 'num_iterations': 2200}" 742 | ] 743 | }, 744 | { 745 | "cell_type": "code", 746 | "execution_count": 9, 747 | "metadata": {}, 748 | "outputs": [], 749 | "source": [ 750 | "target_cols = ['target']\n", 751 | "oofs_df = pd.DataFrame()\n", 752 | "preds_df = pd.DataFrame()\n", 753 | "for t_col in target_cols:\n", 754 | " oofs_df[t_col] = np.zeros(len(train))\n", 755 | " preds_df[t_col] = np.zeros(len(test))" 756 | ] 757 | }, 758 | { 759 | "cell_type": "code", 760 | "execution_count": 10, 761 | "metadata": {}, 762 | "outputs": [ 763 | { 764 | "name": "stderr", 765 | "output_type": "stream", 766 | "text": [ 767 | "/opt/conda/lib/python3.6/site-packages/sklearn/model_selection/_split.py:296: FutureWarning: Setting a random_state has no effect since shuffle is False. This will raise an error in 0.24. You should leave random_state to its default (None), or set shuffle=True.\n", 768 | " FutureWarning\n" 769 | ] 770 | }, 771 | { 772 | "name": "stdout", 773 | "output_type": "stream", 774 | "text": [ 775 | "\n", 776 | "fold n°0\n", 777 | "\n", 778 | "\n", 779 | "**** target ****\n", 780 | "\n" 781 | ] 782 | }, 783 | { 784 | "name": "stderr", 785 | "output_type": "stream", 786 | "text": [ 787 | "/opt/conda/lib/python3.6/site-packages/lightgbm/engine.py:148: UserWarning: Found `num_iterations` in params. Will use it instead of argument\n", 788 | " warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n" 789 | ] 790 | }, 791 | { 792 | "name": "stdout", 793 | "output_type": "stream", 794 | "text": [ 795 | "Training until validation scores don't improve for 200 rounds\n", 796 | "[50]\ttraining's rmse: 23.5108\tvalid_1's rmse: 27.2943\n", 797 | "[100]\ttraining's rmse: 17.6672\tvalid_1's rmse: 24.9727\n", 798 | "[150]\ttraining's rmse: 14.3511\tvalid_1's rmse: 24.2854\n", 799 | "[200]\ttraining's rmse: 11.9442\tvalid_1's rmse: 24.0457\n", 800 | "[250]\ttraining's rmse: 10.1414\tvalid_1's rmse: 23.8479\n", 801 | "[300]\ttraining's rmse: 8.71988\tvalid_1's rmse: 23.7446\n", 802 | "[350]\ttraining's rmse: 7.58742\tvalid_1's rmse: 23.6734\n", 803 | "[400]\ttraining's rmse: 6.65648\tvalid_1's rmse: 23.6176\n", 804 | "[450]\ttraining's rmse: 5.85834\tvalid_1's rmse: 23.5647\n", 805 | "[500]\ttraining's rmse: 5.19224\tvalid_1's rmse: 23.5328\n", 806 | "[550]\ttraining's rmse: 4.60524\tvalid_1's rmse: 23.5228\n", 807 | "[600]\ttraining's rmse: 4.09876\tvalid_1's rmse: 23.5144\n", 808 | "[650]\ttraining's rmse: 3.65281\tvalid_1's rmse: 23.502\n", 809 | "[700]\ttraining's rmse: 3.27241\tvalid_1's rmse: 23.5033\n", 810 | "[750]\ttraining's rmse: 2.94167\tvalid_1's rmse: 23.4796\n", 811 | "[800]\ttraining's rmse: 2.64577\tvalid_1's rmse: 23.4679\n", 812 | "[850]\ttraining's rmse: 2.38538\tvalid_1's rmse: 23.4593\n", 813 | "[900]\ttraining's rmse: 2.15481\tvalid_1's rmse: 23.4656\n", 814 | "[950]\ttraining's rmse: 1.9501\tvalid_1's rmse: 23.4672\n", 815 | "[1000]\ttraining's rmse: 1.7749\tvalid_1's rmse: 23.4626\n", 816 | "[1050]\ttraining's rmse: 1.62105\tvalid_1's rmse: 23.4556\n", 817 | "[1100]\ttraining's rmse: 1.48257\tvalid_1's rmse: 23.4534\n", 818 | "[1150]\ttraining's rmse: 1.3632\tvalid_1's rmse: 23.4495\n", 819 | "[1200]\ttraining's rmse: 1.25276\tvalid_1's rmse: 23.4504\n", 820 | "[1250]\ttraining's rmse: 1.15613\tvalid_1's rmse: 23.4461\n", 821 | "[1300]\ttraining's rmse: 1.06911\tvalid_1's rmse: 23.4485\n", 822 | "[1350]\ttraining's rmse: 0.98995\tvalid_1's rmse: 23.4459\n", 823 | "[1400]\ttraining's rmse: 0.919237\tvalid_1's rmse: 23.4462\n", 824 | "[1450]\ttraining's rmse: 0.85598\tvalid_1's rmse: 23.4428\n", 825 | "[1500]\ttraining's rmse: 0.799376\tvalid_1's rmse: 23.4423\n", 826 | "[1550]\ttraining's rmse: 0.747357\tvalid_1's rmse: 23.4405\n", 827 | "[1600]\ttraining's rmse: 0.700451\tvalid_1's rmse: 23.4407\n", 828 | "[1650]\ttraining's rmse: 0.65895\tvalid_1's rmse: 23.4396\n", 829 | "[1700]\ttraining's rmse: 0.621099\tvalid_1's rmse: 23.4395\n", 830 | "[1750]\ttraining's rmse: 0.586378\tvalid_1's rmse: 23.4387\n", 831 | "[1800]\ttraining's rmse: 0.555159\tvalid_1's rmse: 23.4387\n", 832 | "[1850]\ttraining's rmse: 0.526326\tvalid_1's rmse: 23.4391\n", 833 | "[1900]\ttraining's rmse: 0.499345\tvalid_1's rmse: 23.4379\n", 834 | "[1950]\ttraining's rmse: 0.474703\tvalid_1's rmse: 23.438\n", 835 | "[2000]\ttraining's rmse: 0.452027\tvalid_1's rmse: 23.4393\n", 836 | "[2050]\ttraining's rmse: 0.431394\tvalid_1's rmse: 23.4379\n", 837 | "[2100]\ttraining's rmse: 0.412163\tvalid_1's rmse: 23.4391\n", 838 | "Early stopping, best iteration is:\n", 839 | "[1923]\ttraining's rmse: 0.487809\tvalid_1's rmse: 23.4368\n" 840 | ] 841 | } 842 | ], 843 | "source": [ 844 | "max_iter = 10\n", 845 | "folds = StratifiedKFold(n_splits=max_iter, random_state=1901)\n", 846 | "\n", 847 | "for fold_, (trn_idx, val_idx) in enumerate(folds.split(train.values, pd.qcut(target, 10, labels=False, duplicates='drop'))):\n", 848 | " print(\"\\nfold n°{}\".format(fold_))\n", 849 | " X_trn, X_val, X_test = train.iloc[trn_idx][features], train.iloc[val_idx][features], test[features]\n", 850 | " for t_col in target_cols:\n", 851 | " target = train[t_col]\n", 852 | " print(f\"\\n\\n**** {t_col} ****\\n\")\n", 853 | " y_trn, y_val = target.iloc[trn_idx], target.iloc[val_idx]\n", 854 | " trn_data = lgb.Dataset(X_trn, y_trn)\n", 855 | " val_data = lgb.Dataset(X_val, y_val)\n", 856 | " \n", 857 | " clf = lgb.train(param, trn_data, valid_sets = [trn_data, val_data], verbose_eval=50, early_stopping_rounds = 200)\n", 858 | "\n", 859 | " oofs_df[t_col][val_idx] = clf.predict(X_val, num_iteration=clf.best_iteration)\n", 860 | " current_test_pred = clf.predict(X_test, num_iteration=clf.best_iteration)\n", 861 | " current_test_pred[current_test_pred < 0] = 0\n", 862 | " preds_df[t_col] += current_test_pred / folds.n_splits\n", 863 | "\n", 864 | " break\n", 865 | " \n", 866 | " break" 867 | ] 868 | }, 869 | { 870 | "cell_type": "code", 871 | "execution_count": 11, 872 | "metadata": {}, 873 | "outputs": [ 874 | { 875 | "data": { 876 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA30AAAI/CAYAAADDbbBqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeZRcVbn+8e/DbIiAAnqDCkFAmYKBNGgEFJALahxAgQioBLxwUbzgAJdcGQQRBVEmQbiAgCAgs0xeQcMQQKbMCbNAUIEfgmIkTDI8vz/OLigqVdWdztDdleezVq9U7bPP3u85dTqr3373OS3bRERERERERGdarK8DiIiIiIiIiAUnSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHW6KvA4iI6M9WWmklDx06tK/DiIiIiOjWxIkTn7a9cmN7kr6IiDaGDh3KhAkT+jqMiIiIiG5JerRZe5Z3RkREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLA8vTMioo3pj81i6Nhr+jqMiIiIGKBmHjWqr0NIpS8iIiIiIqKTJemLiIiIiIjoYEn6IiIiIiIiOliSvn5O0uxutq8g6Wt171eRdMl8nH+MpJPm13gNY/9G0gpN2g+TtH95/T1JW5fX35A0qBfzvEfSDZLulXS3pP3mPfq287X9zDqRpKGSZiyEed50vbfoM1zSbeWzniZpdN22r0v6oyRLWmlBxxsRERHRHyTp6wdU6e1nsQLw+g/Bth+3vcP8iWzBsv1J2//ops+htn9f3n4DmOukD3gF+LbtdYAPAftIWrcX4wxokjrhwU1vut5beB74su31gI8Dx9f9cuFWYGvg0QUXYkRERET/kqSvj5TKyL2SfgZMAg6RdFepTBzepP9gSeMkTZI0XdJny6ajgDUkTZF0TH3FRdIdktarG+NGSSMkLSvpzDLf5LqxWllF0m8lPSjpR3Xjza57vYOks8vrsyWdUqprD0v6aJnv3lqf0m9mrdoi6SBJ90v6PfD+uj5nl7H3BVYBbijjfkXScXX99pR0bLPgbT9he1J5/SxwL/CuunNynKTxJb6NJV1WjvX77U6KpF9LmlgqSns1bPtJ+azGSVq5tA2XdHv5jC+X9DZJ60i6s26/oZKmldcjJN1U5rhW0pA2sWxcxr2tXAe1a2CMpIslXQVcJ+ntJe5pJZYNSr/Xq6vl/YwSS+06Pb0c53WS3lIX31RJtwH7dHOuxpTz2uw62qbEPanEOljSaqXfSpIWk3SzpG1ouN6bzWX7AdsPltePA38FVi7vJ9ue2S7WiIiIiE6TpK9vvR84BziQKgnZBBgOjJD0kYa+LwLb294I2BL4iSQBY4GHbA+3fUDDPr8CdgIoCcMqticCBwHX2964jHWMpGXbxDkcGA0MA0ZLek8Pju1twFbAN4GrgOOA9YBhkobXd5Q0AvgCsCHwOWDjxsFsnwg8Dmxpe8tybJ+RtGTpsjtwVndBSRpa5rmjrvlftj8CnApcQZXArA+MkbRim+H2sD0C6AL2reu7LDCpfFY3Ad8t7ecAB9reAJgOfNf2vcBSkt5b+owGLirH9VNghzLHmcCRbWI5C9jb9kjg1YZtI4HdbG8FHA5MLjF8p8TUnbWAk0vl7B/A5+vm3LfM2RNzXEcl6T8Y2LqcrwnAt2w/ChxN9Zl8G7jH9nW0v97nIGkTYCngoR7GWNtvL0kTJE149flZc7NrRERERL+TpK9vPWr7dmCb8jWZquq3NtUP2vUE/KBUgX5PlSS+s5vxLwJ2LK93Ai4ur7cBxkqaAtwILAOs2maccbZn2X4RuAdYrftD4yrbpkpunrQ93fZrwN3A0Ia+mwOX237e9j+BK7sb3PZzwPXApyStDSxpe3q7fSQNBi4FvlHmqanNNx24u1QGXwIeBtoluPtKmgrcXvrVPrPXgAvL618Cm0laHljB9k2l/RdALbG/iJKcUyVFF1L9QmB94HflczoYeHeL41oBeKvtP5Sm8xu6/M7238vrzYBzAWxfD6xYYmvnEdtTyuuJwNAmx3NuN2NA8+voQ8C6wK3lOHcr7dg+A3grsDewf/MhWyu/6DgX2L1cez1m+zTbXba7Fh/U3emJiIiI6N864R6fgey58q+AH9r+3zZ9d6VaojbC9suSZlIlay3ZfkzS38oSvtHAf9bN93nb9/cwzpfqXr/KG9eN69obY6nt81rD/q/R/Lpzk7bunEFVrbqPbqp8pXJ2KXCe7cvmMVYkbUF1b9hI289LupHWn0d3x3YhcLGkywDbflDSMKoEtCdVNHWz/bm61836muq+x/pfAtUfS+Pn/5Yyztx+Zs2uI1ElpTs3dlb10J5aojsYeLanE0laDrgGOLj8YiUiIiJikZVKX/9wLbBHqUQh6V2S3tHQZ3ngryXh25I3qm3PUlVDWvkV8N/A8nWVsGuB/yrLQ5G0YS/jfrLck7YYsH0vxwAYD2wv6S2S3gp8ukW/Nx2r7TuoKmy7ABe0Grwc58+Be203ve+vF5YHnikJ39pUFauaxYDaw3R2AW6xPQt4RtLmpf1LVEs/sf0QVRJ0CG9UCO8HVpY0shzDkqq7P7Oe7WeAZyXVYvhCm7jHU/0CoZa4Pl2qnjOBjUr7RsDq7Q6+PIBnlqTNStOu7fq3cTuwqaQ1y9yDJL2vbDsaOA84FDi9tHV3vSNpKeBy4BzbF7frGxEREbEoSNLXD5R7lc4HbpM0HbiEOX+wPQ/okjSB6gfs+8q+f6NaGjejxYMtLqFKAi6qazsCWBKYVh74cUQvQx8LXE21zPKJXo5BecjKhcAUqmrczS26ngb8n6Qb6touAm4tiU8rm1IlWVuVB4BMkfTJ3sZb/BZYoiy3PYIqeal5DlhP0kSq+xq/V9p3o7p/chrV/W3fq9vnQuCL5Xiw/S+qxPHosoR0CvDhNvF8BTitPFRFQKsb0Q6juo6mUT0UZbfSfinw9rLE8qvAA22PvrI7cHKZ84Ue9J+D7aeAMcAFJabbgbUlfZTq3s6jbZ8H/EvS7j243qFaKvsRqnsya5/3cABJ+0r6C1UFcZqkM3oTd0RERMRAouq2q4iBSdLVwHG2x/V1LH1J0mDbs8vrscAQ2wv07xEuKpYespaH7HZ8X4cRERERA9TMo0YttLkkTbTd1dieSl8MSKr+SPcDwAuLesJXjCoVrRlUD8Zp++cmIiIiImLRkUpfACBpW6p7qOo9Ynte7tVbqMqfTGiWAH6sLAvsF2POC0knUy1XrXeC7W7/XMXCsDCvo/Kwm8anhr5k+4Pzc56uri5PmDBhfg4ZERERsUC0qvQl6YuIaCNJX0RERAwUWd4ZERERERGxCErSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB0vSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB0vSFxERERER0cGS9EVERERERHSwJfo6gIiI/mz6Y7MYOvaavg4jIiKiV2YeNaqvQ4h+IJW+iIiIiIiIDpakLyIiIiIiooMl6YuIiIiIiOhgSfoGAEmzm7TtLWm6pCmSbpG0bjdjbCJpvKT7Jd0n6QxJgySNkXRSQ98bJXU1tF0pacb8OaIFR9IWkq6ej+MNlbRL3fsuSSf2cqw5zmubvnN8LnMxT2PMvR5rYar/7CR9RtLY8nplSXdImixpc0k7SrpX0g0txllR0g2SZje5to+U9Odm31MRERERnSpJ38B1vu1htocDPwKObdVR0juBi4EDbb8fWAf4LfDWnkwk6XPAAv0hWVJ/fajQUOD1BMr2BNv79l04PTKUupgXhAX9edm+0vZR5e3HgPtsb2j7ZuArwNdsb9li9xeBQ4D9m2y7CthkvgccERER0Y8l6RugbP+z7u2ygNt03wf4he3byr62fYntJ7ubR9Jg4FvA93vQd09Jd0maKulSSYNK+8rl/V3la9PSfpik0yRdB5wjaTVJ4yRNK/+uWvrtKGlGGXd8aVtG0lml2jlZUqsEoDHGt0v6dZnjdkkb1MVyrqTrJT0oac+yy1HA5qWi+s2GatTguhimSfp8aT9F0gRJd0s6vCdxlf12l/SApJuATeva252/bmMubatI+m3p96Nu4pgt6SeSJpXPYeXSfqOkH5T49pM0QtJNkiZKulbSEEnrSLqzbqyhkqa1mevjqirPtwCfq2sfI+kkSbVfanyyHM93gc2AUyUd02xM28/ZvoUq+WvcdrvtJ9odf0RERESn6a/VlegBSftQJWRLAVu16bo+8Is220dL2qzu/Zp1r48AfgI834OQLrN9eont+1QVmZ8CJwDH2b6lJHLXUlUbAUYAm9l+QdJVwDm2fyFpD+BEYDvgUGBb249JWqHstw+A7WGS1gauk/S+HsR4ODDZ9naStgLOAYaXbRsAH6JKoidLugYYC+xv+1PluLaoG+sQYJbtYWXb20r7Qbb/LmlxYJykDWy3THzKvkNKbCOAWcANwOSyud3560nMY8oxbgi8BNwv6ae2/9winGWBSba/LelQ4LvA18u2FWx/VNKSwE3AZ20/JWk0cKTtPSQtJem9th8GRgMXtTjmZYDTqa7dPwIXNvaxPaXE0GX762W/LcvxTWh9RueNpL2AvQAWX27lBTVNRERExEKRSt8AZvtk22sABwIHz8NQF9oeXvsCJgCUKsuati/v4TjrS7pZ0nRgV2C90r41cJKkKcCVwHKSaktLr7T9Qnk9Eji/vD6XqqIDcCtwdqlkLV7aNit9sH0f8CjQk6Svfr/rgRUlLV+2XWH7BdtPUyVd3S0D3Bo4ufbG9jPl5U6SJlElbesBbe+3LD4I3Gj7Kdv/4s0JULvz19OYx9meZftF4B5gtTaxvFY3/y9543Ogrv39VL9M+F2J62Dg3WXbRcBO5fVomiRzxdrAI7YftO0yV79g+zTbXba7Fh+0fPc7RERERPRjqfR1hl8Bp7TZfjdVBemKuRx3JDBC0kyqa+Udkm60vUWL/mcD29meWqpLtX6LASPrkjsAJAE812Z+A9jeW9IHgVHAlJKMai6P5fVpW83DnEtk2y2ZrY31pj6SVqe6l2xj289IOhtYpoextZqv3fnracwv1b1+lbn73q8fs/Z5Cbjb9sgm/S8ELpZ0GdVq4gd7OHZERERELACp9A1QktaqezsKaPeD9UnAbiVxqu3/RUn/1m4O26fYXsX2UKpqzwNtEj6oHgzzRFn6t2td+3W8sTywVkFs5g/AF8rrXYFbSv81bN9h+1DgaeA9wPjaHGVZ56rA/e2Op6jfbwvg6br7Iz+r6l7BFakS1ruAZ2n9wJvG43obsBxVYjRL1QN0PtGDmADuALZQ9eTJJYEd28xTf/7mNuaeWAzYobzehfI5NLgfWFnSyBLTkpLWA7D9EFVieQitq3wA9wGrS1qjvN95HmKOiIiIiBZS6RsYBkn6S937Y4HVJG0NvAw8A+zWamfbT0r6AvBjSe+gWr43HrhsPsd5CFXy8igwnTcSj32Bk8sDPZYoc+/dZP99gTMlHQA8Bexe2o8pSa6AccBUqoTh1LKU9BVgjO2XSvWrncOAs0osz/Pm83YncA1VAnmE7cclPQW8ImkqVSVzcl3/75fjmkGV5Bxu+zJJk6mqqw9TLU3tlu0nJB0G3AY8AUzijaWs7c5fT2KuLTvtqeeA9SRNpLq/cHSTeP8laQfgxLI8dgng+HLcUCV7xwCrtznmF8u9c9dIepoquVx/LmNtqlSnlwOWkrQdsI3te8pDbHbhje+pM2wfNj/mjIiIiOivVN1KE7FoKwnXbNs/7utYempBxSxptu3B83PMgWzpIWt5yG7H93UYERERvTLzqFF9HUIsRJIm2p7j70JneWdEREREREQHS6Wvg0jaFji6ofkR29vP53lOpu7vyBUn2D5rfs4zLyTtDuzX0Hyr7X36Ip56ku4Alm5o/pLt6Z0ah6TLmXOp54G2r52HMRfK9d7V1eUJExbYX4eIiIiImG9aVfqS9EVEtJGkLyIiIgaKLO+MiIiIiIhYBCXpi4iIiIiI6GBJ+iIiIiIiIjpYkr6IiIiIiIgOlqQvIiIiIiKigyXpi4iIiIiI6GBJ+iIiIiIiIjpYkr6IiIiIiIgOlqQvIiIiIiKigyXpi4iIiIiI6GBJ+iIiIiIiIjrYEn0dQEREfzb9sVkMHXtNX4cRERGLiJlHjerrEKIDpdIXERERERHRwZL0RUREREREdLAkfRERERERER0sSd8AIGl2k7a9JU2XNEXSLZLW7WaMTSSNl3S/pPsknSFpkKQxkk5q6HujpK6GtislzZg/R7TgSNpC0tXzcbyhknape98l6cRejjXHeW3Td47PZS7maYy512PNizLvKnPR//XPTtJnJI0tr1eWdIekyZI2l7SjpHsl3dBinBUl3SBpdpNre+fyfTNN0m8lrTQvxxgRERExECTpG7jOtz3M9nDgR8CxrTpKeidwMXCg7fcD6wC/Bd7ak4kkfQ6YI/GcnyT114cKDQVeT6BsT7C9b9+F0yNDqYu5D40Bepz01bN9pe2jytuPAffZ3tD2zcBXgK/Z3rLF7i8ChwD71zeWa+wEYEvbGwDTgK/3Jr6IiIiIgSRJ3wBl+591b5cF3Kb7PsAvbN9W9rXtS2w/2d08kgYD3wK+34O+e0q6S9JUSZdKGlTaVy7v7ypfm5b2wySdJuk64BxJq0kaV6ow4yStWvrtKGlGGXd8aVtG0lmlajNZUqsEoDHGt0v6dZnjdkkb1MVyrqTrJT0oac+yy1HA5qWi+s2GatTguhimSfp8aT9F0gRJd0s6vCdxlf12l/SApJuATeva252/bmMubauUytaDkn7UTRyzJR1Zzvft5ZcG7eK4QtKXy+v/lHSepB2ALuC8EsdbWsz1cVWV51uAz9W1j5F0kqTaLzU+Wcb5LrAZcKqkY5qNafs527dQJX9vmq58LStJwHLA4+3ORUREREQn6K/VlegBSftQJWRLAVu16bo+8Is220dL2qzu/Zp1r48AfgI834OQLrN9eont+1QVmZ9SVVeOs31LSeSupao2AowANrP9gqSrgHNs/0LSHsCJwHbAocC2th+TtELZbx8A28MkrQ1cJ+l9PYjxcGCy7e0kbQWcAwwv2zYAPkSVRE+WdA0wFtjf9qfKcW1RN9YhwCzbw8q2t5X2g2z/XdLiwDhJG9ie1i4oSUNKbCOAWcANwOSyud3560nMY8oxbgi8BNwv6ae2/9winGWB220fVBLEPamS/lZx7AXcKukR4NvAh8rxf73EMaHFMS8DnE517f4RuLCxj+0pkg4Fumx/vey3ZbtxW7H9sqSvAtOB54AHKddRk9j2KsfF4sutPDfTRERERPQ7qfQNYLZPtr0GcCBw8DwMdaHt4bUvYAJAqbKsafvyHo6zvqSbJU0HdgXWK+1bAydJmgJcCSwnqba09ErbL5TXI4Hzy+tzqSo6ALcCZ5dK1uKlbbPSB9v3AY8CPUn66ve7HlhR0vJl2xW2X7D9NFXStUk3Y20NnFx7Y/uZ8nInSZOokrb1gLb3WxYfBG60/ZTtf/HmBKjd+etpzONsz7L9InAPsFqbWP4F1O6LnEi1XLRlHKVifGiZ/9u2/96D4wVYG3jE9oO2Dfyyh/v1iqQlga9SJb+rUC3v/J9mfW2fZrvLdtfig5Zv1iUiIiJiwEilrzP8Cjilzfa7qSpIV8zluCOBEZJmUl0r75B0o+0tWvQ/G9jO9tRSXar1WwwYWZfcAVCtsOO5NvMbwPbekj4IjAKmlGRUc3ksr0/bah7mXCLbbslsbaw39ZG0OtW9ZBvbfkbS2cAyPYyt1Xztzl9PY36p7vWrtP/ef7kkYY19m8ZRDAP+xtzfw9fdOZ6fhgPYfghA0kVUVdGIiIiIjpZK3wAlaa26t6Oolqq1chKwW0mcavt/UdK/tZvD9im2V7E9lKpC9kCbhA+qB8M8USoqu9a1X0fdAzNK0tbMH4AvlNe7AreU/mvYvsP2ocDTwHuA8bU5yrLOVYH72x1PUb/fFsDTdfdHflbVvYIrUiWsdwHP0vqBN43H9Taq+8SeA2aVe+E+0YOYAO4AtlD15MklgR3bzFN//uY25nnRNA5Jm1Ad54bA/iXxpQdx3AesLmmN8n7n+R7xmz0GrCuptl7z34F7F/CcEREREX0ulb6BYZCkv9S9PxZYTdLWwMvAM8BurXa2/aSkLwA/lvQO4DWq5Oey+RznIVTJy6NU903VfuDfFzhZ0jSqa248sHeT/fcFzpR0APAUsHtpP6YkuQLGAVOpEoZTy1LSV4Axtl8q1a92DgPOKrE8z5vP253ANVQJ5BG2H5f0FPCKpKlUlczJdf2/X45rBlVF7HDbl0maTFVdfZhqaWq3bD8h6TDgNuAJYBJvLGVtd/56EnNt2em8miMOSftR3Ze3e5n721Sf4VZl7lMlvUCTCqHtF8u9c9dIepoqyV9/fgRaqtPLAUtJ2g7YxvY9qh6sM17Sy1TX6Zj5MV9EREREf6Y3VnFFLLpKwjXb9o/7OpaeGogxD0RLD1nLQ3Y7vq/DiIiIRcTMo0b1dQgxgEmaaHuOvwud5Z0REREREREdLJW+DiJpW+DohuZHbG8/n+c5mbq/I1ecYPus+TnPvJC0O7BfQ/Ottps+on9hknQHsHRD85dsT+/UOCRdDqze0Hyg7WvnYcyFcr13dXV5woS5+usQEREREX2iVaUvSV9ERBtJ+iIiImKgyPLOiIiIiIiIRVCSvoiIiIiIiA6WpC8iIiIiIqKDJemLiIiIiIjoYEn6IiIiIiIiOliSvoiIiIiIiA6WpC8iIiIiIqKDJemLiIiIiIjoYEn6IiIiIiIiOliSvoiIiIiIiA6WpC8iIiIiIqKDJemLiIiIiIjoYEv0dQAREf3Z9MdmMXTsNX0dRkRE9NLMo0b1dQgRfS6VvoiIiIiIiA6WpC8iIiIiIqKDJekboCTN7mb7CpK+Vvd+FUmXzMf5x0g6aX6N1zD2bySt0KT9MEn7l9ffk7R1ef0NSYN6Mc97JN0g6V5Jd0vab96jbztf28+sE0kaKmnGQpjnTdd7m35HS5pRvkYv6LgiIiIi+oMkff2YKr39jFYAXv8h2PbjtneYP5EtWLY/afsf3fQ51Pbvy9tvAHOd9AGvAN+2vQ7wIWAfSev2YpwBTVIn3Nv7puu9GUmjgI2A4cAHgQMkLbcQYouIiIjoU0n6+plSGblX0s+AScAhku6SNE3S4U36D5Y0TtIkSdMlfbZsOgpYQ9IUScfUV1wk3SFpvboxbpQ0QtKyks4s802uG6uVVST9VtKDkn5UN97sutc7SDq7vD5b0imluvawpI+W+e6t9Sn9Zkpaqbw+SNL9kn4PvL+uz9ll7H2BVYAbyrhfkXRcXb89JR3bLHjbT9ieVF4/C9wLvKvunBwnaXyJb2NJl5Vj/X67kyLp15ImlurhXg3bflI+q3GSVi5twyXdXj7jyyW9TdI6ku6s22+opGnl9QhJN5U5rpU0pE0sG5dxbyvXQe0aGCPpYklXAddJenuJe1qJZYPS7/Xqank/o8RSu05PL8d5naS31MU3VdJtwD7dnKsx5bw2u462KXFPKrEOlrRa6beSpMUk3SxpGxqu9xbTrQvcZPsV288BU4GPt4svIiIiohMk6euf3g+cAxxIlYRsQlWdGCHpIw19XwS2t70RsCXwE0kCxgIP2R5u+4CGfX4F7ARQEoZVbE8EDgKut71xGesYScu2iXM4MBoYBoyW9J4eHNvbgK2AbwJXAccB6wHDJA2v7yhpBPAFYEPgc8DGjYPZPhF4HNjS9pbl2D4jacnSZXfgrO6CkjS0zHNHXfO/bH8EOBW4giqBWR8YI2nFNsPtYXsE0AXsW9d3WWBS+axuAr5b2s8BDrS9ATAd+K7te4GlJL239BkNXFSO66fADmWOM4Ej28RyFrC37ZHAqw3bRgK72d4KOByYXGL4TompO2sBJ9teD/gH8Pm6Ofctc/bEHNdRSfoPBrYu52sC8C3bjwJHU30m3wbusX0d7a/3mqnAJyQNKuNvCTS9ZiXtJWmCpAmvPj+rh4cRERER0T8l6eufHrV9O7BN+ZpMVfVbm+oH7XoCflCqQL+nShLf2c34FwE7ltc7AReX19sAYyVNAW4ElgFWbTPOONuzbL8I3AOs1v2hcZVtUyU3T9qebvs14G5gaEPfzYHLbT9v+5/Ald0NXio41wOfkrQ2sKTt6e32kTQYuBT4RpmnpjbfdODuUhl8CXiYFslCsa+kqcDtpV/tM3sNuLC8/iWwmaTlgRVs31TafwHUEvuLKMk5VVJ0IdUvBNYHflc+p4OBd7c4rhWAt9r+Q2k6v6HL72z/vbzeDDgXwPb1wIoltnYesT2lvJ4IDG1yPOd2MwY0v44+RFWZu7Uc526lHdtnAG8F9gb2bz7knEpy+BvgD8AFwG1US3yb9T3NdpftrsUHdXcaIiIiIvq3TriXpxM9V/4V8EPb/9um767AysAI2y9LmkmVrLVk+zFJfytL+EYD/1k33+dt39/DOF+qe/0qb1xPrmtvjKW2z2sN+79G8+vRTdq6cwZVteo+uqnylcrZpcB5ti+bx1iRtAWwNTDS9vOSbqT159HdsV0IXCzpMsC2H5Q0jCoB7UkVTd1sf67udbO+pkqK6n85VH8sjZ//W8o4c/uZNbuORJWU7tzYWdVDe2qJ7mDg2Z5OZPtISmVU0vnAg3MZa0RERMSAk0pf/3YtsEepRCHpXZLe0dBneeCvJeHbkjeqbc9SVUNa+RXw38DydZWwa4H/KstDkbRhL+N+styTthiwfS/HABgPbC/pLZLeCny6Rb83HavtO6gqbLtQVXSaKsf5c+Be203v++uF5YFnSsK3NlXFqmYxoPYwnV2AW2zPAp6RtHlp/xLV0k9sP0SVBB3CGxXC+4GVJY0sx7Ck6u7PrGf7GeBZSbUYvtAm7vFUv0CoJa5Pl6rnTKqHnyBpI2D1dgdfHsAzS9JmpWnXdv3buB3YVNKaZe5Bkt5Xth0NnAccCpxe2rq73pG0eG2pbfmFxwbAdb2MLyIiImLASKWvHyKD4C0AACAASURBVLN9naR1gNtKHjYb+CLw17pu5wFXSZoATKGqbmH7b5JuLQ/u+D/g5IbhLwFOAI6oazsCOB6YVhKimcCnehH6WOBq4M/ADKpqzFyzPUnShVTH9Shwc4uupwH/J+mJcl8fVEsjh5fEp5VNqZKs6WUJIcB3bP+mN/EWvwX2Lstt76dKXmqeA9aTNBGYRVVlhWrp4qmlgvUw1X2INRcCx1CSLdv/krQDcGJZSrkE1Wd2d4t4vgKcLuk5qiW7rW5QOww4q8T9fIkJqirol8v5uQt4oLsTUOI/U9LzVL9ImGu2n5I0BrhA0tKl+eByD+rGwKa2X5X0eUm72z6r/npvcV/fksDN5Xvpn8AXbTdd3hkRERHRSVTdXhXRWSRdDRxne1xfx9KXJA22Pbu8HgsMsb1A/x5hp1l6yFoestvxfR1GRET00syjRvV1CBELjaSJtrsa27O8MzqKqj/S/QDwwqKe8BWjyp8xmEH1YJy2f24iIiIiIjpPKn3RlqRtqe6hqveI7Xm5V2+hKvdxNUsAP2b7b/1lzHkh6WSq5ar1TrDd7Z+rWBgW5nVUHnbT+NTQl2x/sDfjdXV1ecKECfMeWERERMQC1qrSl6QvIqKNJH0RERExUGR5Z0RERERExCIoSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAl+jqAiIj+bPpjsxg69pq+DiMiYsCbedSovg4hYpGVSl9EREREREQHS9IXERERERHRwQZs0idpdpO2vSVNlzRF0i2S1m2z/xaSrm5oO1vSDuX1kpKOkvSgpBmS7pT0ibJtpqSVWo0laTtJ0yTdV+LZrptjeX3eZscn6X2SfiPpj5LulXSRpHf24BiWknS8pIfKcVwh6d3tYmkR342SuuZ2v7LvFpI+3Jt924w5x2c/kEk6TNL+C3iOMZJOms9jnlH7HpP0nfk5di/j6TYGSfuV7+e7JX1jYcQVERER0dcGbNLXwvm2h9keDvwIOHYexjoCGAKsb3t94NPAW7vbSdIHgB8Dn7W9NvAZ4MeSNuhNEJKWAa4BTrG9pu11gFOAlXuw+w9KzO+zvRbwa+AySepNLL20BTBfk754gyp98n1s+z9s31Pe9nnSRzcxSFof2BPYBPgA8ClJay2MwCIiIiL6Ukclfbb/Wfd2WcC9GUfSIKofDv/L9ktl7CdtX9SD3fcHfmD7kbLfI8APgQN6EwuwC3Cb7atqDbZvsD2jB8ewO/BN26+W/c4CXgK2arHPspKukTS1VENGN+lziqQJpVJyeF37TEmHS5pUqptrSxoK7A18s1RfN28x7zslXV7mnVqrDEr6VoljRrOqTJMK60mSxtTF8wNJt5V4N5J0bal67l23/42SLilV2fPaJcTNjrG0v6lSV+IdWr7uKxWxGWX8rSXdWiqvm9QN/wFJ15f2PevGOkDSXaVyfHhpG1oqvj8DJgHvaRHv7pIekHQTsGld+8qSLi3j3iVp07rjOLOck4cl7Vvam14XtQqwpKOAt5TP+DxJR0jar26+I2tjtfgMm34GkkZIuknSxPLZDZG0vKT7Jb2/9LlA0p6NMbT4CNcBbrf9vO1XgJuA7Vv0jYiIiOgYHff0Tkn7AN8ClqJFclNnc0lT6t6vClwNrAn8qSGJbHSDpFfL68HAfeX1elSVvnoTgH26ieUYSQc3aV8fmNhmv7k9hgklxnFNxvo48LjtUQCSlm/S5yDbf5e0ODBO0ga2p5VtT9veSNLXgP1t/4ekU4HZthvPSb0TgZtsb1/GHSxpBFXS+kFAwB2SbrI9uc04jf5se6Sk44CzqRKfZYC7gVNLnw2pzsfjwK2lzy1txnzTMQL/0U0MawI7AnsBd1El8ZtRVYC/A9SW/m4AfIjqlxWTJV1D9dmvRVWZEnClpI8AfwLeD+xu+2vNJpU0BDgcGAHMAm4AaufuBOA427dIWhW4liohAlgb2JKqQny/pFPo5rqwPVbS10uFHVXJ/mXACaqqkF8ox9DKHJ+BpDuAn1JVzJ8qieaRtveQ9HXgbEknAG+zfXqZ9/UYWpgBHClpReAF4JNU3w8RERERHa3jkj7bJwMnS9oFOBjYrU33m21/qvZG0tlzMdWWtp8u+21BlQBA9cN5Y4WxWVujA2xfUhdLT+9ba3UMreZsF8t0qqWoRwNX2765SZ+dJO1Fde0MAdYFaknfZeXficDnehg/VMn5lwFKVXKWpM2Ay20/V47rMmBz3khceuLK8u90YLDtZ4FnJb0oaYWy7U7bfylzTAGG0j7pm9tjfMT29DL+3cA425Y0vcxVc4XtF4AXJN1AlSRtBmzDG8c8mCoJ/BPwqO3b28z7QeBG20+VuS8E3le2bQ2sW1fUXE5SbenyNaW6/ZKkvwLvpGfXxetsz5T0N0kblv0n2/5bm12afQb/oEp6f1fiXBx4ooz/O0k7AidTLdPsEdv3lmP4HTAbmAq80qxvucb3Alh8uZ6spI6IiIjovzpqeWeDX/FGFWVu/RFYte4H4blxN9D40JONgHua9O3peCN6sd8fgdWaHEPLWGw/UOaaDvxQ0qH12yWtTpXcfsz2BlT3Gi5T1+Wl8u+rzPsvFHpy3+ErvPkaXqZhey2e1+pe194v0dAHehZ3s2NsF0fjvPUx1c/VmIib6hz80Pbw8rWm7Z+X7c91E2ezMWsWA0bWjfuukhA3xvsqsER310ULZwBjqKq1Z3bTt9lnIODuuhiH2d4GoFQP16Gq1r29B7G8zvbPbW9k+yPA34EHW/Q7zXaX7a7FBzUreEdEREQMHB2V9OnND2UYRYsf6Lpj+3ng58CJkpYqYw+R9MUe7P5j4H/KErfaUrfvAD/pTSzA+cCHJb3+F00lfVzSsHY7lQrZL4Bjy5JJJH0ZGARc32wfSasAz9v+ZTmOjRq6LEeVbMyS9E7gEz2I/1m6fwDOOOCrJYbFJS0HjAe2kzRI0rJU9141VpgepapYLV2WHH6sB/EsCDMp50rSRsDqvRjjs5KWKUsPt6BaCnotsIekwWXsd0l6Rw/HuwPYQtKKkpakWmJacx3w9dobSe2WRPbkugB4ucxTcznVstCNy3HMrfuBlSWNLDEsKWm9su2bwL3AzsCZdfM2xtDsWN5R/l2VqlJ7QS9ii4iIiBhQBvLyzkGS/lL3/liqytbWwMvAM7Rf2tmdg4HvA/dIepEq2em2wmF7iqQDgavKD6AvA/9te0o3u7Ya7wVJnwKOl3R8GW8asB+wYje7/w/VD+kPSHqN6r7D7W23qgANo7q38LUyz1cbYpkqaTJV9fFhqvuvunMVcImkz1I9GKfZ0sD9gNMkfYWq0vNV27eVpap3lj5nNN7PZ/vPki6iOh8PMndLP+enS4Evl6WJdwEP9GKMO6kqp6sCR9h+HHhc0jrAbWWJ42zgi1TnqC3bT0g6DLiNalnkJKolkgD7Ui2Bnkb1f8B4qgfutNL2uihOA6ZJmmR7V9v/KstU/1F7kNDcKPvvQPWLl+VLnMdLepnqPspNbD8raTzV9+p3G2NoMfSlJbF+GdjH9jNzG1tERETEQKPWP/9HRPROWYI5CdjRdq8q7v3F0kPW8pDdju/rMCIiBryZR43qvlNEzBNJE23P8fe1O2p5Z0T0PVV/sP2PVA+tGdAJX0REREQnGMjLO3tE0rbA0Q3Nj9he6H+fS9LJ1P29tOKE8vfzFmYcK9L8TzZ8rJunLM7rvAfx5nvLAC62feSCmrM3JF3OnPflHWi7N/emLXDlzxss3dD8pdpTQxe28gfb31vfVu5BPbeh60u2Pzi/55/f1/ewdy3PhPx2OiIiIgawLO+MiGijq6vLEybkz/lFRERE/5flnREREREREYugJH0REREREREdLElfREREREREB0vSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB0vSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB0vSFxERERER0cGW6OsAIiL6s+mPzWLo2Gv6OoyIiH5r5lGj+jqEiOhGKn0REREREREdLElfREREREREB0vSFxERERER0cGS9HUwSbObtO0tabqkKZJukbRum/0HSTqv9J9R+g8u294t6QpJD0p6SNIJkpYq2/5d0sSy30RJW/Uy/pmSVurNvm3G3ELS1S22/UbSCvNzvv6gu/Mo6Q9zOV7Lc9jfSfqspGnl+p8gabO+jikiIiJiQUvSt+g53/Yw28OBHwHHtum7H/Bk6b8+8BXgZUkCLgN+bXst4H3AYODIst/TwKdtDwN2A85dQMcyVyS1fXCR7U/a/sfCiqcdVRbK96ftDy+MeeZFd5/dXBgHfKBc/3sAZ8yncSMiIiL6rSR9ixjb/6x7uyzgNt2HAI/V7Xu/7ZeArYAXbZ9V2l8FvgnsIWmQ7cm2Hy+73Q0sI2npVpNIOqVUXe6WdHjD5gMk3Vm+1iz9dyyVx6mSxpe2ZSSdVaqLkyVtWdrHSLpY0lXAdWXM5SRdLukeSafWkqtaRUzSUEn3Sjq9xHSdpLeUPhuXStFtko6RNKPNcQ2VdLOkSeXrw3XbDpB0Vxnr8Lr+90r6GTAJeE9tjnJco0u/IZLGl2rVDEmbl/ad66qyR7eKq0mcs8u/W0i6UdIlku4rVV6VbR8vbbcAn6vbd1lJZ5ZjmSzps6X9REmHltfblnib/n8j6ezyOdws6QFJn2r22bWZ6w5J69WNd6OkEc3msj3bdu2a7+76j4iIiOgISfoWQZL2kfQQVaVv3zZdzwQOLAnO9yWtVdrXAybWdyzJ5J+ANRvG+DwwuSSLrRxkuwvYAPiopA3qtv3T9ibAScDxpe1QYFvbHwA+U9r2KXEMA3YGfiFpmbJtJLCb7doy002AbwPDgDWoS2LqrAWcbHs94B/lOADOAva2PRJ4tc0xAfwV+HfbGwGjgRMBJG1Txt8EGA6MkPSRss/7gXNsbwh0le0fALYGjpE0BNgFuLZUqz4ATJG0CnA0VUI+HNhY0nbdxNfMhsA3gHWB9wKblvN4OvBpYHPg3+r6HwRcb3tjYMsS47LAWGB0Sb5PBHa3/VqbeYcCHwVGAae2+OxazfUrYCeoEmJgFdsTaUHS9pLuA66hqvY167NX+UXEhFefn9Um7IiIiIj+L0nfIsj2ybbXAA4EDm7TbwrVD/7HAG8H7pK0DiCaV0je1F6qL0cD/9lNSDtJmgRMpkoo6+8zvKDu35Hl9a3A2ZL2BBYvbZtRlpHavg94lGrZKcDvbP+9bsw7bT9cKpQXlH0bPVKOH6oEd6iq+/3eart2D9z53RzXksDpkqYDF9cd1zblazJVRW9tqiQQ4FHbt9cd0wW2X7X9JHATsDFwF7C7pMOAYbafLe032n7K9ivAeUAtkZwbd9r+S0nQplAlY2tTnY8HS5Xsl3X9twHGSpoC3AgsA6xq+3lgT+B3wEm2H+pm3otsv2b7QeDhMie8+bNrOhdwEbBj6bMT1bluyfblttcGtgOOaNHnNNtdtrsWH7R8N6FHRERE9G/54+yLtl8Bp7TrYHs21f17l0l6DfgkMJU3Kl8ASFoOeA/wUHn/buBy4MvtfuCXtDqwP7Cx7WcknU31w/zrITS+tr23pA9SVYWmSBpOlXC28lzjYXXzHqC+Mvkq8JZu5mjmm8CTVNW4xYAXS7uAH9r+3/rOkoY2xNp0PtvjS2VwFHCupGOAfzbr2wuNx137P6LVMkgBn7d9f5Ntw4C/Aav0YN5Wn0nj+Wg6l6S/lQrxaLr/JUM1QXUe15C0ku2ne7JPRERExECUSt8ipm6JJlRJw4Nt+m4q6W3l9VJUlapHqR6GMUjSl8u2xYGfAGfbfr5UxK4B/sf2rd2EtBzVD/azJL0T+ETD9tF1/95W5lvD9h22D6V6aMx7gPHArmX7+6gqQM0SEYBNJK1e7jEbDdzSTYwA2H4GeFbSh0rTF7rZZXngiVI1+xJvVCWvpbr/sfYk1HdJekeT/cdTLZFcXNLKVJW7OyWtBvzV9unAz4GNgDuolsauVD6Pnakqg/PDfcDqktYo73eu23Yt8F919/5tWP5djWoJ7YbAJ0qS3s6OkhYrc7yX5p9d07mKXwH/DSxve3qrSSStWbf/RsBSVIlpRERERMdKpa+zDZL0l7r3xwKrSdoaeBl4hurpmq2sAZxSfkhejCqRu9S2JW0P/EzSIWXbb4DvlP2+TnVv3yFlO8A2tv/aOIHtqZImUz3w5WGqpZv1lpZ0R5mjlmwcU5JXUSWgU6kSk1PLUspXgDG2Xyo/3ze6DTiKqhI1nqoi2VNfoVqy+RzVEsN2N3z9DLhU0o7ADZSqle3ryjLZ20p8s4EvMuc9gpdTLWmdSlX5+m/b/0/SblQPuHm57Ptl209I+p8yj4Df2L5iLo6rJdsvStoLuEbS01RJ8vpl8xFU91pOK9fJTEmfpkpG97f9uKSvUC3H3dj2i83moErybgLeSXXP5ItNPrs55gI+VbZdApxAi+WadT4PfLmcuxeA0XUPdomIiIjoSMrPOxE9J2lwWfKKpLHAENv79XFYA1pZ0nu17Uv6OpZmlh6ylofsdnz3HSMiFlEzjxrV1yFERCFpYnlA4puk0hcxd0aVitoSVEtdx/RtOLGgDXvX8kzIDzQRERExgCXpCyRtS/WUzXqP2N5+Ps9zB9D49/q+1O4erP7G9oXAhfVtC+v8zQtJK1IthW30MdsL5Z42SQfxxlM2ay62PWYBzLU70FiBvdX2PvN7roiIiIj+Lss7IyLa6Orq8oQJE/o6jIiIiIhutVremad3RkREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHW6KvA4iI6M+mPzaLoWOv6eswIiLmi5lHjerrECKiD6TSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfB5M0u0nb3pKmS5oi6RZJ67bZf5Ck80r/GaX/4LLt3ZKukPSgpIcknSBpqbLt3yVNLPtNlLRVL+OfKWml3uzbZswtJF3dYttvJK0wP+frD7o7j5L+MJfjtTyHfUnScEmf7KbP2yRdLmmapDslrb+w4ouIiIjoK0n6Fj3n2x5mezjwI+DYNn33A54s/dcHvgK8LEnAZcCvba8FvA8YDBxZ9nsa+LTtYcBuwLkL6FjmiqS2Dy6y/Unb/1hY8bSjykL5/rT94YUxz0IwHGib9AHfAabY3gD4MnDCAo8qIiIioo8l6VvE2P5n3dtlAbfpPgR4rG7f+22/BGwFvGj7rNL+KvBNYA9Jg2xPtv142e1uYBlJS7eaRNIpkiZIulvS4Q2bDygVmTslrVn671gqj1MljS9ty0g6q1QXJ0vasrSPkXSxpKuA68qYy5Vqzz2STq0lV7WKmKShku6VdHqJ6TpJbyl9Ni5VotskHSNpRpvjGirpZkmTyteH67YdIOmuMtbhdf3vlfQzYBLwntoc5bhGl35DJI0v1doZkjYv7TvXVWWPbhVXkzhnl3+3kHSjpEsk3VeqvCrbPl7abgE+V7fvspLOLMcyWdJnS/uJkg4tr7ct8Tb9/0bS2aX/HyQ9LGmHbs7T9pJ+XxLjIZIekLQq8D1gdDkvo1sc7rrAOADb9wFDJb2zp+cqIiIiYiBK0rcIkrSPpIeoKn37tul6JnBgSXC+L2mt0r4eMLG+Y0km/wSs2TDG54HJJVls5SDbXcAGwEclbVC37Z+2NwFOAo4vbYcC29r+APCZ0rZPiWMYsDPwC0nLlG0jgd1s15aZbgJ8GxgGrEFdElNnLeBk2+sB/yjHAXAWsLftkcCrbY4J4K/Av9veCBgNnAggaZsy/iZU1akRkj5S9nk/cI7tDYGusv0DwNbAMZKGALsA15Zq7QeAKZJWAY6mSsiHAxtL2q6b+JrZEPgGVXL0XmDTch5PBz4NbA78W13/g4DrbW8MbFliXBYYS5WAbVmOe3fbr7WZdwiwGfAp4Kh258n25cD/o/rMTwe+a/tPVNfFhbaH276wxTxTKZ+3pE2A1YB3N3aStFf5RcSEV5+f1SbsiIiIiP4vSd8iyPbJttcADgQObtNvCtUP/scAbwfukrQOIJpXCN/ULmk9qkTkP7sJaSdJk4DJVAll/X2GF9T9O7K8vhU4W9KewOKlbTPKMtJSwXmUatkpwO9s/71uzDttP1wqlBeUfRs9Uo4fqgR3qKr7/d5qu3YP3PndHNeSwOmSpgMX1x3XNuVrMlVFb22q5AbgUdu31x3TBbZftf0kcBOwMXAXsLukw4Bhtp8t7Tfafsr2K8B5QC2RnBt32v5LSdD+P3v3Hm/pXPd//PU2Ts04VJJ7JEYaSQ2DGRLKKWJyupFKOSRSQnXrRkrULVMk3A6RmA6SW4hMv6gJg8aYYWaMsxujQk63hjGI8f79cX13c1mz1tqzx57Zs5f38/GYx17re32v7/dzXWvteazP/nyva00FhpT4HrJ9v20Dv6j13w44StJU4DpgWWB127OBA4E/AGfYfqCbeX9j+1XbdwFdlbd25+lQ4GjgJdsXNQ7WxmjgLSXeQ8vYrzR2sn2u7RG2RwwYuGIPho+IiIhY/OTL2d/YfgWc3a6D7VlU1+9dJulVqmumpjG38gWApBWAdwIPlOerAZcD+7T7wC9pTeAIYKTtZySNoUoc/hVC42PbB0vaBBhFVeUaTpVwtvJ842F18xygXpmcA7ypmzma+QrwOFU1bgngxdIu4ETb59Q7SxrSEGvT+WyPL5XBUcDPJZ0EPNus7wJoPO6u/yNaLQMWsLvte5tsGwY8Dazaw3lV+znPeSreAbwKrCJpiW6qiP9SKtL7Q3XdJPBQ+RcRERHRsVLpe4OpLdGEKmm4v03fzSS9pTxemqpS9TDVNVEDJe1Ttg0AfgCMsT27VMTGAkfbvqmbkFagSnRmlmurdmjYvlft54Qy31q2J9o+luqmMe8ExgN7l+1rA6sDzRIRgI0lrVmuMdsLuLGbGAGw/QzwnKQPlKZPdLPLisBjJSH5DHOrkldTXf/YdSfUd0h6e5P9x1MtkRwgaWWqyt0tktYAnrD9Y+AnwIbARKqlsW8rr8cnqSqDveEeYE1Ja5Xnn6xtuxo4tHbt3wbl5xpUS2g3AHYoSXpPNT1Pqm7IcwHVMte7ga+W/s8By7cbUNKby3sZ4HPA+IbrXCMiIiI6Tip9nW2gpL/Vnp8CrCFpW+Bl4Bmqu2u2shZwdvlAvwRVInepbUvaDThL0jfLtt9R3RkR4EtU1/Z9s2wH2M72E40T2J4maQrVDV8epFq6WbeMpIlljq5k46SSvIoqAZ1GlZj8qCylfAXYz/ZLJRdpNIFqmd8wqsTq8jbnoNEBVEs2n6daztjugq+zgEsl7QlcS6ni2b6mLJOdUOKbBXyaea8RvJxqSes0qkrbf9r+u6R9qW5w83LZdx/bj0k6uswj4He2r+jBcbVk+0VJBwFjJT1FlSR3fdXBd6iutby9vE9mSNqJKhk9wvajkg6gWo470vaLzeZoMW+r83QwcIPtG8oyzUmSxpZj71pqemKL6/reC/xM0hzgLqrXMyIiIqKjqbpEJyLmh6TlypJXJB0FDLZ9eB+HFQvRMoOHevC+p3bfMSKiH5gxelRfhxARC5GkW8sNEl8jlb6InhlVKmpLUi113a9vw4mIiIiIaC+VvkDS9lR32ax7yPZuvTzPRKDx+/o+Y3t6b86zqC2q8/d6SFqJ8v10Dbax/fQiiuEYYM+G5ktsn7AQ5tofaKzA3mT7kJ6ONWLECE+ePLl3AouIiIhYiFpV+pL0RUS0kaQvIiIi+otWSV/u3hkREREREdHBkvRFRERERER0sCR9ERERERERHSxJX0RERERERAdL0hcREREREdHBkvRFRERERER0sCR9ERERERERHSxJX0RERERERAdL0hcREREREdHBkvRFRERERER0sCR9ERERERERHWzJvg4gImJxNv2RmQw5amxfhxERHWzG6FF9HUJEdLhU+iIiIiIiIjpYkr6IiIiIiIgOlqQvIiIiIiKigyXp6yckzWrSdrCk6ZKmSrpR0rrdjLGxpPGS7pV0j6TzJA0s23aVdHtpny5p1/mI6YjS/w5J0yTtU9qXlnSqpAck3S/pCkmrLeixN5n3OElH9NZ4i4Nmr+9CmGOGpLf14ngjJJ1eHm8p6YO9NfYCxtNtDJLWkDSuvNev6833ZURERMTiKklf//ZL28NsDwe+D5zSqqOkVYBLgCNtvwd4L/B7YHlJ6wMnA7vYXgfYGThZ0nptxjsY+Aiwse33Ax8CVDZ/F1geWNv2UOA3wGWS1HSweF0k9ckNmWxPtn1Yebol0KdJ33zGcDLwM9vrAd8GTlzYQUVERET0tSR9/ZjtZ2tPBwFu0/0Q4Ke2J5R9bfvXth8HjgC+a/uhsu0hqg/DX2sz3teBL3bFYHum7Z+WyuH+wFdszynbLgBeArZuNZikfUr1ZZqkn5e2elVmnKTVm+x3naQR5fHbJM0oj/eT9BtJv5X0kKQvSfqqpCmSbpb01tr+35N0i6T7JG3RJsb9JF0m6felgvn92rZZtcd7SBpTHo+RdLakayU9KOnDks6XdHdXn9p+P5B0WznWlUvbWmW+WyXdIGmd2rinSLoW+F6LeFeSdE055nOYm5Qj6dPlmKdKOkfSgK7jkHRCeR1uLn8sQNKetYru+NK2paSrJA0BDga+UsbbopzzpUq/FUqVcakWcTZ9DSQNkHSSpEnlPfD50v5VSeeXx8NKXOs2xtDiZVwXGFceXwvs0qJfREREVP6kZwAAIABJREFURMdI0tfPSTpE0gNUlb7D2nR9P3Bri23va7JtcmlvNufywPK2H2iy+d3AXxoS0u7Gex9wDLC17fWBw8umM5hblbkQOL1F/K28H/gUsDFwAjDb9gbABGCfWr8lbW8MfBn4VjdjDgf2AoYBe0l653zE8RaqhPcrwG+BH1Kdi2GShpc+g4DbbG8IXF+L41zgUNsbUSXnZ9XGXRvY1vZ/tJj3W8CN5ZivBFYHkPTecgyblSrxHGDvWhw3l9dhPHBgaT8W2L6071yfxPYM4EfAD20Pt30DcB3QdQ/yTwCX2n65zTlq9hocAMy0PRIYCRwoaU3gVODdknYDLgA+b/uuJjE0Mw3YvTzejarSvVJjJ0kHSZosafKc2TPbhB0RERGx+EvS18/ZPtP2WsCRwDcWcBgxb5WwWdvr2dZun62BX9t+CsD2/5X2TYFflsc/BzZvsX8r19p+zvaTwEyqhAtgOjCk1u+y8vPWhvZmxpWq5ovAXcAa8xHHb227zPu47em2XwXurM33KnBxefwLYHNJy1EtV7xE0lTgHGBwbdxLuqqpLXyojIXtscAzpX0bYCNgUhl3G+BdZds/gavK4/r5uAkYI+lAYMB8HPN5VBVfys8Luunf7DXYDtinxDgRWAkYWs7dflTviett3zQf8XQ5AviwpCnAh4FHgFcaO9k+1/YI2yMGDFyxB8NHRERELH7y5eyd41fA2W2230n1Qf+KFttGALfX2jakSmrmYftZSc9LepftBxs2/y+whqTlbT/XMN5vaa5dQviaqZu0vcLcP14s27DtpdrjV2vPX+W17/2u9jl0/ztRH7Pevx5bqzjqMTSLo85Ux/WPUo1r5vluYm2Mq4uolvoe3WTbyyVBhdrx2T5Y0iZU1buptQpl80ntmyQNkfRhYIDtO7qJs9lrIKoq59VN+g8FZgGrdjNuY1yPAv8OUJLq3W2nlBcREREdLZW+fkzS0NrTUcD9bbqfAexbPrh37f9pSf9GdXOLo8u1WZSfXwd+0Ga8E4EzJa1Q9llB0kG2nwd+CpxSu05sH2Ag8KcWY40DPt61zK7rejvgz1RLA6Fafnhjk31nUCWzAHu0iXdhe1zSeyUtQbVssKeWYG78n6Jalvks8JCkPQFUWb8HY46nLNuUtAPVMlOozvcekt5etr1VUtuKpaS1bE+0fSzwFNC4rPU5qpv31P0MuIjuq3ytXA18oXZt4NqSBklaETiNqpK5kqSu89YshsbjeFt5jQCOBs5fwNgiIiIi+o0kff3HQEl/q/37KvAlSXeW5W9fBfZttXO5YcsnqO7Kea+ku4EtgGdtT6VaHvpbSfdQVeT+s7S3cjbVjTAmSbqD6jq02WXb0cCLwH2S7gf2BHarVZAaY7uT6pq76yVNY+5dSA8D9pd0O/AZ5l7rV3cyVWLwZ6DXvo5gARxFtSzyT8BjC7D/88D7JN1Ktdz126V9b+CAcl7upGc3Hjke+JCk26iWSv4FoFz/9g3gmnJu/8Brl402c5Kqr/K4gyqZnNaw/bfAbg03UbmQKtG8qAcx151HVW2+rcx7DlUV8IfAWbbvo7rub3RJYJvF0GhL4F5J9wGrUL3vIiIiIjqaWnwOj4h4XUoFbhfbn+nrWF6PZQYP9eB9T+3rMCKig80YPar7ThER80HSrbZHNLbnmr6I6HWS/hvYAdixr2OJiIiIeKNLpa/DSNqeeb+37SHbC3KdGZLOBDZraD6tfPdeT8daibnfkVa3je2nFyS+haG3z+HCJml/5l36epPtQ/oinlZ68700H3MdQ7WsuO4S2z1ezjlixAhPnjy5dwKLiIiIWIhaVfqS9EVEtJGkLyIiIvqLVklfbuQSERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER0sSV9EREREREQHS9IXERERERHRwZL0RUREREREdLAkfRERERERER1syb4OICJicTb9kZkMOWpsX4cREf3UjNGj+jqEiIhU+iIiIiIiIjpZkr6IiIiIiIgOlqQvIiIiIiKigyXp61CS3izpi30dR3ckfVnSwIU8x1cl3SXpdknjJK2xEOc6TtIRC2v8xZWkMZL2WATz7Cpp3W76nCTpnvJ6Xy7pzaV9JUnXSpol6YyFHWtERETE4iJJX+d6M9DnSZ8q7d5nXwZ6lPRJ6ukNiKYAI2yvB/wa+H4P9+8Ikgb0dQy9YFegbdIH/AF4f3m97wOOLu0vAt8E3nBJeURERLyxJenrXKOBtSRNLZWPr0maVKofxwNIGlIqIudJukPShZK2lXSTpPslbVz6HSfp55L+VNoP7Jqkzbh3SzoLuA14p6SzJU2WdGet32HAqsC1kq4tbbNqY+8haUx5PEbSKaXf9yStJen3km6VdIOkdVqdCNvX2p5dnt4MrFbG3FLS9ZL+R9J9kkZL2lvSLZKmS1qr1ZiSdpI0UdIUSX+UtEpt8/qN56okvyeV8zxd0l6l/WJJO9bGHSNpd0kDSv+uc/v5NrEsIemscm6vkvS7rqqbpBmSjpV0I7CnpE+W+e+Q9L3aGO3O++mS/izpwdq4knSGqgrqWODtreKrxXG8pNvK/OuU9kGSzi/HOUXSLqX9dEnHlsfbSxov6YPAzsBJ5X3d9PWxfY3tV8rTf73etp+3fSNV8hcRERHxhpGvbOhcR1FVO4ZL2g7YA9gYEHClpA8BfwHeDewJHARMAj4FbE714frrVJUVgPWADwCDgCnlg/77gaEtxn0PsL/tLwJIOsb2/6mqNo2TtJ7t0yV9FdjK9lPzcUxrA9vaniNpHHCw7fslbQKcBWw9H2McAPy/2vP1gfcC/wc8CJxne2NJhwOHUlUim7kR+IBtS/oc8J/Af5Rtzc7VpsDwMt/bgEmSxgO/AvYCfidpaWAb4Aslzpm2R0paBrhJ0jW2H2oSy78DQ4BhVMnX3cD5te0v2t5c0qpUSdBGwDPANZJ2tf2bbs7ZYKr3xDrAlVTV0t2oXuNhwCrAXQ1zNvOU7Q1VLTs+AvgccAzwJ9ufVbUM8xZJf6R6/06SdANwOrCj7QckXQlcZfvX3czV5bPAxfPZ918kHUT1O8GAFVbu6e4RERERi5UkfW8M25V/U8rz5aiStb8AD9meDiDpTmBcSWSmUyUSXa6w/QLwQqm2bUyVCLQa92HbN9f2/3j5IL0kVRKxLnB7D4/jkpLwLQd8ELhEUte2ZbrbWdKngRHAh2vNk2w/VrY/AFxT2qcDW7UZbjXgYkmDgaWBejLW6lxdZHsO8Lik64GRVAno6SWx+ygw3vYLJVFfT3Ovk1uR6tw2S/o2pzo3rwJ/L3PWdSU9I4HrbD9ZjvdC4ENAd0nfb8rYd9Uqmh+qHc+jkv7UzRgAl5Wft1IlqlC9f3bW3OsglwVWt313qZKOB75i+4H5GP81JB0DvAJc2NN9bZ8LnAuwzOCh7un+EREREYuTJH1vDAJOtH3OaxqlIcBLtaZXa89f5bXvj8YPvu5m3Odrz9ekquyMtP1MWTq4bItY6/M09ukacwngH7aHtxhjHpK2paoqfdh2/Zjn9/gb/Tdwiu0rJW0JHFfb1upczcP2i5KuA7anqvhd1BUycKjtq9vE0KXp2DVd561dv3bnvX6O6mP0NBnqGmcOc8+tgN1t39uk/zDgaaolwD0iaV/gY8A2tpO0RURExBtarunrXM8By5fHVwOfLRUyJL1DUttrsJrYRdKyklYCtqRaCjq/465AlXjMLJWiHVrECVUV7L2qbv6yW7NAbD8LPCRpzzKvJK3fKnBJGwDnADvbfmL+DrdbKwKPlMf7Nmxrdq7GA3uVa/VWpqqU3VL6/wrYH9iC6pxSfn5B0lLlGNaWNKhFLDcCu5dr+1YpczYzEfiwpLeVZbafBK4v27o97w3GA58oxzOY9lXRdq4GDlUp2ZbXClV3WP0PYANgh7KEF+Z9v8xD0keBI6le79nt+kZERES8EaTS16FsP63qhix3UC0h/CUwoXy2ngV8mqriMr9uAcYCqwPfsf0o1bK+93Y3ru1pkqYAd1JdN3dTbfO5wP+T9Jjtraiu5boK+CtwB9WS0Wb2Bs6W9A1gKarEaVqLvieVcbqWg/7F9s49OPZmjivjPUJ1ndyatW3znCtJl1Nd1zeNqkL2n7b/XvpfA/wMuNL2P0vbeVTLa28rCdGTzL2+stGlVNcC3kF1t8qJwMzGTrYfk3Q0cC1Vhe13tq8om+f3vHe5nOoayullzuvbd2/pO8CpwO3lOGdI2gn4CXBEOXcHAGMkjaR6nX+s6iZAe7RY9nkG1XLfP5TX+2bbB0N1QxmqP0IsLWlXYDvbdy1g7BERERH9grLyKboj6Thglu2T+zqWaE7ScrZnleriLcBmtaQyXodlBg/14H1P7eswIqKfmjF6VF+HEBFvIJJutT2isT2VvojOcFW5++XSVNXFJHwRERERAaTSFx2k3K1xz4bmS2yfsDiN+TpiGQb8vKH5JdubNOvfF8oy1jUbmo+czxvS9HSuM4HNGppPs31Bb84zYsQIT548uTeHjIiIiFgoWlX6kvRFRLSRpC8iIiL6i1ZJX+7eGRERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB0vSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB0vSFxERERER0cGS9EVERERERHSwJH0REREREREdLElfREREREREB1uyrwOIiFicTX9kJkOOGtvXYUREL5oxelRfhxARsUil0hcREREREdHBkvRFRERERER0sH6T9Ema1aTtYEnTJU2VdKOkddvsP1DShaX/HaX/cmXbapKukHS/pAcknSZp6bLtI5JuLfvdKmnrbuL8bOl7e5lnl9IuSd8oc9wn6VpJ76vFNlbSPZLulDT69ZyrJjHNc+76M0nHSTpiIc+xn6QzennM87reo5K+3ptj9yCGZST9sfzO7LWQ59q13e9kk/5DJN1RHo+QdHp5/JqYJW1Rfk+mSnpTi7F+L+kfkq5qaN9a0m3ld/OnkrLEPSIiIjpev0n6Wvil7WG2hwPfB05p0/dw4PHS//3AAcDLkgRcBvzG9lBgbWA54ISy31PATraHAfsCP281gaTVgGOAzW2vB3wAuL1sPgT4ILC+7bWBE4ErJS1btp9sex1gA2AzSTv06EzEfCnJd5+8721/zvZd5WmfJH1U76+lbA+3ffFCnmtXYL6Tvjrbk20fVp42xrw31e/LcNsvtBjiJOAz9Ybyuv8U+ET5P+Bhqt/piIiIiI7Wr5M+28/Wng4C3Kb7YOCR2r732n4J2Bp40fYFpX0O8BXgs5IG2p5i+9Gy253AspKWaTHH24HngFllrFm2HyrbjgQOtT27bLsG+DOwt+3Ztq8t7f8EbgNWa3UgklaRdLmkaeXfB0v7V0sF4w5JX26y35b1yoekMyTtVx7PkPRdSRMkTZa0oaSrS+Xz4Nr+10n6dalKXliS5lZxzpB0fKmsTJe0Tml/TaWuxDuk/LunVMTuKONvK+mmUiHduDb8+pL+VNoPrI31NUmTSqX1+NI2RNLdks4q5/adLeLdv1Rhrwc2q7WvLOnSMu4kSZvVjuP8ck4elHRYaR9UKrfTynHsVdqvKxWs0cCbSqXqQknfkXR4bb4TusZqEuNyksbVzmlXJbnbcyfp7cAvgOFl7rV6+LoNKsc7SdKU2tynSzq2PN5e0vjyntwZOKmbuTYq52kC1R9Gutq3lHRVk5g/D3wcOFbShc3GBLA9jup3sW4l4CXb95XnfwB2bzVGRERERKfo10kfgKRDJD1AVelr+kG5OB84siQ1/yVpaGl/H3BrvWNJJv8CvLthjN2BKSVZbGYa8DjwkKQLJO1UYlwBGGT7gYb+k8v89eN5M7ATMK7NsZwOXG97fWBD4E5JGwH7A5tQVRgPlLRBmzGa+avtTYEbgDHAHmWsb9f6bAB8maqC8y5qyVELT9neEDgbmJ8lme8GTgPWA9YBPgVsXvatV8fWA0YBm1IlAKtK2g4YCmwMDAc2kvSh0v89wM9sb2D74cZJJQ0Gji/H8xFeW6E6Dfih7ZFU74HzatvWAbYvc35L0lLAR4FHba9fKkq/r89l+yjghVKp2hv4CaXipKoa9QmgVULzIrBbOadbAT+oJd5tz53tJ4DPATeUuRvfj3XNXrdjgD+V87AVVUI3CDgK2EvSVlTvzf1t/xm4EvhaN3NdABxW3nfzaBLzObVx924Tf9NjApaSNKI834PWfwA4SNUfPybPmT2zh9NERERELF76fdJn+0zba1FV0r7Rpt9UqiTlJOCtwCRJ7wVE8wrha9pVXX/3PeDzbeaYQ/WBfw/gPuCHko5rE37jHEsCFwGn236wzX5bU30Yx/Yc2zOpPtxfbvt527Oolqxu0WaMZq4sP6cDE20/Z/tJ4MWSjALcYvtvtl8FpgJDuhnzsvLz1vnoC/CQ7ell/DuBcbZdYqrvf4XtF2w/BVxLlXRtV/5NoarorUOVBAI8bPvmNvNuAlxn+8lSba0vfdwWOEPSVKpztIKk5cu2sbZfKnE8AaxSYt1W0vckbVFen5ZszwCeLkn6dlR/WHi6RXcB35V0O/BH4B1lTpj/czc/mr1u2wFHlfNwHbAssHqpXh9IVTk7o5tkcu6BSCsCb7Z9fWlquXS6N5Rz8Qmq38tbqCqBr7Toe67tEbZHDBi44sIMKyIiImKh66SbGPyKkgi1UkuGLpP0KrAjVXXuNUu8SmXuncAD5flqwOXAPt19oC0fLG8BbpH0B+AC28dJel7SuxqSuQ2B62vPzwXut31qt0c7r5bLLGte4bWJ/rIN27sqmK/WHnc9X7KhD8Acun8PdfWv920XR+O89ZjqczUm6qY6ByeWatC/SBoCPN9NnM3G7LIEsGnj9WOlwDbP+bB9X6m87gicKOka2/VqaTPnAfsB/0ZVlW5lb2BlYCPbL0uawdzzN7/nbn40e90E7G773ib9hwFPA6v2YI5Wf3BZaGxPoPwxpFSG116U80dERET0hX5d6ast0YRqqd/9bfpuJukt5fHSVMv3HqZaRjlQ0j5l2wDgB8AY27NLhWsscLTtm7qJZ1VJG9aahpc5oKownq5yt0FJ21JV535Znv8XsCLV0snujAO+0BVvSVLHA7uquhPoIGA3qmWadQ8D66q6G+KKwDbzMdfCMIMq4aWcrzUXYIxdJC0raSVgS2AScDXVtZhdd2V9R7kmbH5MBLaUtFJZorlnbds1wJe6nkga3m4gSasCs23/AjiZcqwNXi7zdLmcqko8shxHKysCT5SEbytgjXax9LKrgUO7lpN2LR+WtAbwH1RLf3eQtEnp/xywfLOBAGz/A5gpafPS1NPlmj3W9X5QdV3ukcCPFvacEREREX2tP1X6Bkr6W+35KcAaJXl6GXiG9nfiWws4u3xgXYIqkbvUtiXtBpwl6Ztl2++Ye/3Yl6iulfpm2Q6wXbnWqNFSwMnlQ/+LwJPAwWXbfwNvAaZLmgP8HdjF9guae9fPe4DbymfqM2yf1zhBcThwrqQDqCoxX7A9QdIYqiojwHm2p9R3sv1XSf9DdUfR+6mWQfaFS4F9yjLBSVRLYXvqFqrXcHXgO+VmO4+WJbsTyjmcBXya6hy1ZfuxshR3AvAY1fLQAWXzYcCZZUnlklQJ9sHNximGUV3v9irVe/MLTfqcC9wu6Tbbe9v+p6RrgX+UZcKtXAj8VtJkquW193R3bL3oO8CpVHELmFGuW/0JcITtR8t7coykkVTV9x+ruinNHi2q5PsD50uaTftkt0ck3UC1vHe58v/GAbavBr4m6WNUv+dn2/5Tb80ZERERsbhStRoxIvpSuYHLbcCetltWrGPRW2bwUA/ed0FWXEfE4mrG6FF9HUJExEIh6VbbIxrb+/XyzohOoOoLzP+X6sYrSfgiIiIiolf1p+Wd80XS9lR32ax7yPZuvTzPRKDx+/o+Y3t6L85xDK+9tgzgEtsnNOvfVyRdzrzX5R1ZltMtdhbFa9cTrr6w/V31NknDmPduli/Z3oResihfN0lnMu/Xe5zm8v2YCzjmQj9HAMPesSKTUxWIiIiIfizLOyMi2hgxYoQnT57c12FEREREdCvLOyMiIiIiIt6AkvRFRERERER0sCR9ERERERERHSxJX0RERERERAdL0hcREREREdHBkvRFRERERER0sCR9ERERERERHSxJX0RERERERAdL0hcREREREdHBkvRFRERERER0sCR9ERERERERHSxJX0RERERERAdbsq8DiIhYnE1/ZCZDjhrb12FERA/MGD2qr0OIiFispNIXERERERHRwZL0RUREREREdLB+k/RJmtWk7WBJ0yVNlXSjpHXb7D9Q0oWl/x2l/3Jl22qSrpB0v6QHJJ0maemy7SOSbi373Spp627i/Gzpe3uZZ5fSLknfKHPcJ+laSe+rxTZW0j2S7pQ0+vWcqyYxzXPu+jNJx0k6YiHPsZ+kM3p5zPO63qOSvt6bY/cghmUk/bH8zuy1kOfatd3vZJP+QyTdUR6PkHR6efyamCVtUX5Ppkp6U4uxfi/pH5KuamjfRtJttf8z3v16jjEiIiKiP+g3SV8Lv7Q9zPZw4PvAKW36Hg48Xvq/HzgAeFmSgMuA39geCqwNLAecUPZ7CtjJ9jBgX+DnrSaQtBpwDLC57fWADwC3l82HAB8E1re9NnAicKWkZcv2k22vA2wAbCZphx6diZgvJfnuk/e97c/Zvqs87ZOkj+r9tZTt4bYvXshz7QrMd9JXZ3uy7cPK08aY96b6fRlu+4UWQ5wEfKZJ+9nA3uX/jF8C31iQ+CIiIiL6k36d9Nl+tvZ0EOA23QcDj9T2vdf2S8DWwIu2Lyjtc4CvAJ+VNND2FNuPlt3uBJaVtEyLOd4OPAfMKmPNsv1Q2XYkcKjt2WXbNcCfqT6AzrZ9bWn/J3AbsFqrA5G0iqTLJU0r/z5Y2r9aqot3SPpyk/22rFc+JJ0hab/yeIak70qaIGmypA0lXV0qnwfX9r9O0q9LVfLCkjS3inOGpONLZWW6pHVK+2sqdSXeIeXfPaUidkcZf1tJN5UK6ca14deX9KfSfmBtrK9JmlQqrceXtiGS7pZ0Vjm372wR7/6lCns9sFmtfWVJl5ZxJ0narHYc55dz8qCkw0r7oFK5nVaOY6/Sfl2pYI0G3lSqTRdK+o6kw2vzndA1VpMYl5M0rnZOuyrJ3Z47SW8HfgEML3Ov1cPXbVA53kmSptTmPl3SseXx9pLGl/fkzsBJ3cy1UTlPE6j+MNLVvqWkq5rE/Hng48Cxki5sNiaA7XFUv4vzbAJWKI9XBB5t0iciIiKio/T7u3dKOgT4KrA0VQLXyvnANZL2AMYBP7V9P/A+4NZ6R9vPSvoL8G7mVuoAdgemlGSxmWnA48BDksYBl9n+raQVgEG2H2joP7nMXz+eNwM7Aae1OZbTgett7yZpALCcpI2A/YFNAAETJV1ve0qbcRr91famkn4IjKFKfJalSnZ/VPpsUGJ+FLip9LmxzZhP2d5Q0heBI4DPdRPDu4E9gYOAScCngM2pEoivU1WPALoqqYOAKZLGAu8HhgIbU52DKyV9CPgL8B5gf9tfbDappMHA8cBGwEzgWqDr3J0G/ND2jZJWB64G3lu2rQNsBSwP3CvpbOCjwKO2R5WxV6zPZfsoSV8q1SYkDaGqNp+mqgr5iXIMzbwI7Fbeo28DbpZ05fycO9u7SvoccITtj7UYv0uz1+0Y4E+2P1vep7dI+iNwFDBJ0g1U780dbT9Q4rrK9q/bzHMB1R9Drpd0UuNG2080xixp0/kYt5XPAb+T9ALwLNV7KCIiIqKj9etKH4DtM22vRVVJa7lUy/ZU4F1Uy77eSvUh9b1UyUGzCuFr2lVdf/c94PNt5phD9YF/D+A+4IeSjmsTfuMcSwIXAafbfrDNfltTLVPD9hzbM6k+3F9u+3nbs6iSiC3ajNFMV/IwHZho+znbTwIvlg/5ALfY/pvtV4GpwJBuxrys/Lx1PvoCPGR7ehn/TmCcbZeY6vtfYfsF209RJWgbA9uVf1OoKnrrUCWBAA/bvrnNvJsA19l+slRb60sftwXOkDSV6hytIGn5sm2s7ZdKHE8Aq5RYt5X0PUlblNenJdszgKclbdAVv+2nW3QX8F1JtwN/BN5R5oT5P3fzo9nrth1wVDkP11H9QWD1Ur0+EPgDcEaTP240P5AqGX6z7etLU8ul073oK1RJ6WpUCWfTJeGSDlJV8Z48Z3bbly8iIiJisdfvK301v6IkQq3UkqHLJL0K7EhVndu93q9U5t4JPFCerwZcDuzT3Qfa8iH7FqoqyB+AC2wfJ+l5Se9qSOY2BK6vPT8XuN/2qd0e7bxaLrOseYXXJvrLNmzvqmC+Wnvc9XzJhj4Ac+j+PdTVv963XRyN89Zjqs/VmKib6hycaPuc+oZSSXu+mzibjdllCWDTxuvHVK1sned82L6vVF53BE6UdI3tb3cz93nAfsC/UVWlW9kbWBnYyPbLkmYw9/zN77mbH81eNwG72763Sf9hwNPAqj2Yo9UfXBYKSStTXVM7sTRdDPy+WV/b51L9PrLM4KGLLMaIiIiIhaFfV/okDa09HQXc36bvZpLeUh4vTXWDiYeplnoOlLRP2TYA+AEwxvbsUuEaCxxt+6Zu4llV0oa1puFlDqgqjKer3G1Q0rZU1blfluf/RXWN0TzX4jUxDvhCV7wlSR0P7KrqTqCDgN2AGxr2exhYV9XdEFcEtpmPuRaGGVQJL+V8rbkAY+wiaVlJKwFbUi1nvJrqWsyuu7K+o1wTNj8mAltKWknSUlTLJLtcA3yp64mk4e0GkrQqMNv2L4CTKcfa4OUyT5fLqarEI8txtLIi8ERJ+LYC1mgXSy+7GjhUJdstlUkkrQH8B9Uz7CJhAAAgAElEQVTS3x0kbVL6P0e17LUp2/8AZkravDTtvbACL54BVpS0dnn+EeDuhTxnRERERJ/rT5W+gZL+Vnt+CrBGSZ5epvpAt2+b/dcCzi4fWJegSuQutW1JuwFnSfpm2fY75t5d8UtU10p9s2wH2M72E03mWAo4uXzofxF4Eji4bPtv4C3AdElzgL8Du9h+QXPv+nkPcFv5TH2G7fNaHMvhwLmSDqCqxHzB9gRJY6iqjADnNV7PZ/uvkv6H6jrF+5l7zdqidimwT1kmOIlqKWxP3UL1Gq4OfKfcbOfRsmR3QjmHs4BPU52jtmw/VpbiTgAeo1oeOqBsPgw4syypXJIqwT642TjFMKobmLxK9d78QpM+5wK3S7rN9t62/ynpWuAfZZlwKxcCv5U0mWp57T3dHVsv+g5wKlXcAmZI2gn4CdU1d4+W9+QYSSOpqu8/VnVTmj1aVMn3B86XNJv2yW6PlOsL16G63vVvwAG2r1Z1059Ly2vzDPDZ3pozIiIiYnGlajViRPSlcgOX24A9yw2GYjGxzOChHrzvgqy4joi+MmP0qL4OISKiT0i61faIxvZ+vbwzohOo+gLz/6W68UoSvoiIiIjoVf1peed8kbQ91V026x6yvVsvzzMRaPy+vs/Ynt6LcxzDa68tA7jE9gnN+vcVSZcz73V5R9ruteV6vWlRvHY94eoL299Vb5M0jHnvZvmS7U3oJYvydZN0JrXvPixO6/p+zAUcc6GfI4Bh71iRyakaRERERD+W5Z0REW2MGDHCkydP7uswIiIiIrqV5Z0RERERERFvQEn6IiIiIiIiOliSvoiIiIiIiA6WpC8iIiIiIqKDJemLiIiIiIjoYEn6IiIiIiIiOliSvoiIiIiIiA6WpC8iIiIiIqKDJemLiIiIiIjoYEn6IiIiIiIiOliSvoiIiIiIiA6WpC8iIiIiIqKDLdnXAURELM6mPzKTIUeN7eswIqIHZowe1dchREQsVlLpi4iIiIiI6GBJ+iIiIiIiIjpYkr6IiIiIiIgOlqQvIrol6TpJI3pxvCGS7iiPt5Q0U9IUSfdKGi/pY93sv7KkiWWfLXorroiIiIhOlBu5RMTi4AbbHwOQNBz4jaQXbI9r0X8b4B7b+zZukDTA9pyFGGtEREREv5JKX0T8S6nA3SPpp5Jul/RrSQMb+pwtabKkOyUdX2sfKenPkqZJukXS8pIGSDpJ0qQy3ue7i8H2VODbwJdaxDgc+D6wo6Spkt4kaZakb0uaCGwq6dMlhqmSzpE0oOy7v6T7JF0v6ceSzngdpysiIiKiX0jSFxGN3gOca3s94Fngiw3bj7E9AlgP+LCk9SQtDVwMHG57fWBb4AXgAGCm7ZHASOBASWvORwy3Aes021CSwmOBi20Pt/0CMAi4w/YmwNPAXsBmtocDc4C9JQ0Gjgc2Az4CrNtqckkHlcR28pzZM+cj3IiIiIjFV5Z3RkSjv9q+qTz+BXBYw/aPSzqI6v+PwVTJk4HHbE8CsP0sgKTtgPUk7VH2XREYCtzXTQzqYcxzgEvL422AjYBJkgDeBDwBbAJcZ/vJEtvFwNrNBrN9LnAuwDKDh7qHsUREREQsVpL0RUSjxiTnX89Lle4IYKTtZySNAZalStKaJUcCDrV99WsapSHdxLABcHcPYn6xdh2fgJ/aPrphzl1bxBgRERHR0bK8MyIarS5p0/L4k8CNtW0rAM8DMyWtAuxQ2u8BVpU0EqBcz7ckcDXwBUlLlfa1JQ1qN7mk9YBvAmcuYPzjgD0kvb2M91ZJawATgS0lrVTi2XMBx4+IiIjoV1Lpi4hGdwP7SjoHuB84G9gJwPY0SVOAO4EHgZtK+z8l7QX8t6Q3UV3Pty1wHjAEuE3VWssngV2bzLlFGXcg1VLMw9rcubMt23dJ+gZwjaQlgJeBQ2zfLOk4YALwGNV1gwMWZI6IiIiI/kR2VjtFRKUsu7zK9vv7OJSFTtJ+wAjbTe8S2mWZwUM9eN9TF01QEdErZowe1dchRET0CUm3lhvuvUaWd0ZERERERHSwVPoiYrEl6RjmvfbuEtsnLKoYRowY4cmTJy+q6SIiIiIWWKtKX67pi4jFVknuFlmCFxEREdGJsrwzIiIiIiKigyXpi4iIiIiI6GBJ+iIiIiIiIjpYkr6IiIiIiIgOlqQvIiIiIiKigyXpi4iIiIiI6GBJ+iIiIiIiIjpYkr6IiIiIiIgOlqQvIiIiIiKigyXpi4iIiIiI6GBJ+iIiIiIiIjrYkn0dQETE4mz6IzMZctTYvg4jIlqYMXpUX4cQEbHYS6UvIiIiIiKigyXpi4iIiIiI6GBJ+iIiIiIiIjpYkr5FSNKsJm0HS5ouaaqkGyWt22b/gZIuLP3vKP2XK9tWk3SFpPslPSDpNElLl20fkXRr2e9WSVsvYPwzJL1tQfZtM+aWkq5qse13kt7cm/MtDro7j5L+3MPxWp7DviRpuKQde7jPv85N/TxIOknSneXnypImSpoiaYsmYwyUNFbSPWWf0bVtH5J0m6RXJO3xeo4vIiIior9I0tf3fml7mO3hwPeBU9r0PRx4vPR/P3AA8LIkAZcBv7E9FFgbWA44oez3FLCT7WHAvsDPF9Kx9IiktjcSsr2j7X8sqnjaUWWR/L7Y/uCimGcRGA70KOmrazgPnwc2tP01YBvgHtsb2L6hxe4n214H2ADYTNIOpf0vwH7ALxc0roiIiIj+JklfH7P9bO3pIMBtug8GHqnte6/tl4CtgRdtX1Da5wBfAT4raaDtKbYfLbvdCSwraZlWk0g6W9LkUiU5vmHz1yTdUv69u/Tfs1Qep0kaX9qWlXRBqS5OkbRVad9P0iWSfgtcU8ZcQdLlku6S9KOu5Kqr6iNpiKS7Jf24xHSNpDeVPiMl3S5pQqkC3dHmuIZIuqFUem6T9MHatq9JmlTGOr7W/25JZwG3Ae/smqMc116l32BJ40u19o6u6pOkT9aqst9rFVeTOGeVn1tKuk7Sr0vV6sKS4CPpo6XtRuDfa/sOknR+OZYpknYp7adLOrY83r7E2/T3X9KY0v/Pkh6sV8RanKfdJP2xJMaDJd0naXXg28Be5bzs1WKulcrrOUXSOYCanIcrqX43Jko6kuqPIzuWcd/UOKbt2bavLY//SfXarVaez7B9O/DqfLwUERERER0hSd9iQNIhkh6g+jB7WJuu5wNHlgTnvyQNLe3vA26tdyzJ5F+AdzeMsTswpSSLrRxjewSwHvBhSevVtj1re2PgDODU0nYssL3t9YGdS9shJY5hwCeBn0patmzbFNjXdtcy042B/wCGAWtRS2JqhgJn2n4f8I9yHAAXAAfb3hSY0+aYAJ4APmJ7Q2Av4HQASduV8Temqk5tJOlDZZ/3AD+zvQEwomxfH9gWOEnSYOBTwNWlWrs+MFXSqsD3qBLy4cBISbt2E18zGwBfBtYF3kVVtVoW+DGwE7AF8G+1/scAf7I9EtiqxDgIOIoqAduqHPf+ttslPoOBzYGPAaPbnSfblwN/p3rNfwx8y/ZfqN4XF9sebvviFvN8C7ixnN8rgdUbO9jeGXihjPO9hnFfaHMMqFoevBMwrl2/JvsdpOoPH5PnzJ7Zk10jIiIiFjtJ+hYDts+0vRZwJPCNNv2mUn3wPwl4KzBJ0nupqiPNKoSvaZf0PqpE5PPdhPRxSbcBU6gSyvp1hhfVfm5aHt8EjJF0IDCgtG1OWUZq+x7gYaplpwB/sP1/tTFvsf1gqVBeVPZt9FA5fqgS3CHlA/3ytruu/epuyd5SwI8lTQcuqR3XduXfFKqq0DpUyQ3Aw7Zvrh3TRbbn2H4cuB4YCUwC9pd0HDDM9nOl/TrbT9p+BbgQ6Eoke+IW238rCdpUYEiJ7yHb99s28Ita/+2AoyRNBa4DlgVWtz0bOBD4A3CG7Qe6mfc3tl+1fRewSm3sVufpUOBo4CXbFzUO1saHuuK3PRZ4pgf7tqVq+fBFwOm2H+zJvrbPtT3C9ogBA1fsrZAiIiIi+kS+nH3x8ivg7HYdbM+iun7vMkmvUl0zNY25lS8AJK0AvBN4oDxfDbgc2KfdB35JawJHACNtPyNpDFXi8K8QGh/bPljSJsAoqirXcGrL9Jp4vvGwunkOUK9MzgHe1M0czXwFeJyqGrcE8GJpF3Ci7XPqnSUNaYi16Xy2x5fK4Cjg55JOAp5t1ncBNB531+9sq2XAAna3fW+TbcOAp4FVezivaj/nOU/FO6iWTK4iaYluqoiN2i1pfj3OBe63fWq3PSMiIiI6WCp9fay2RBOqpOH+Nn03k/SW8nhpqkrVw1RL1wZK2qdsGwD8ABhje3apiI0FjrZ9UzchrUCV6MyUtAqwQ8P2vWo/J5T51rI90faxVDeNeScwHti7bF+batles0QEYGNJa5ZrzPYCbuwmRgBsPwM8J+kDpekT3eyyIvBYSUg+w9yq5NVU1z923Qn1HZLe3mT/8VRLJAdIWpmqSnWLpDWAJ2z/GPgJsCEwkWpp7NvK6/FJqspgb7gHWFPSWuX5J2vbrgYOrV37t0H5uQbVEtoNgB1Kkt5TTc9TqahdQLXM9W7gq6X/c8Dy3YxZf5/sALxlAeKah6T/onq9v9wb40VERET0Z6n0LVoDJf2t9vwUYA1J2wIvUy1t27fN/msBZ5cP9EtQJXKX2rak3YCzJH2zbPsd8PWy35eoru37ZtkOsJ3tJxonsD1N0hSqG748SLV0s24ZSRPLHF3JxkkleRVVAjqNKjH5UVlK+Qqwn+2XSi7SaALVdWPDqJKAy9ucg0YHUC3ZfJ5qOWO7C7DOAi6VtCdwLaWKZ/uaskx2QolvFvBp5r1G8HKqJa3TqKpT/2n775L2pbrBzctl331sPybp6DKPgN/ZvqIHx9WS7RclHQSMlfQUVZL8/rL5O1TXWt5e3iczJO1ElYweYftRSQdQLccdafvFZnO0mLfVeToYuMH2DWVZ6SRJY8uxdy01PbHFdX3HAxeV5cTXU12H+rqUqvYxVO/B20qsZ9g+T9JIqtfxLcBOko4v14lGREREdCxVlwRF9E+SlitLXpF0FDDY9uF9HFZ0kGUGD/XgfbNCNGJxNWP0qL4OISJisSHp1nJDxtdIpS/6u1GlorYk1VLX/fo2nIiIiIiIxUsqfYshSdtT3WWz7iHbu/XyPBOBxu/r+4zt6b05z6K2qM7f6yFpJZp/jcA2tp9eRDEcA+zZ0HyJ7RMWwlz7A40V2JtsH/I6x13o7+ERI0Z48uTJvTVcRERExELTqtKXpC8ioo0kfREREdFftEr6cvfOiIiIiIiIDpakLyIiIiIiooMl6YuIiIiIiOhgSfoiIiIiIiI6WJK+iIiIiIiIDpakLyIiIiIiooMl6YuIiIiIiOhgSfoiIiIiIiI6WJK+iIiIiIiIDpakLyIiIiIiooMl6YuIiIiIiOhgS/Z1ABERi7Ppj8xkyFFj+zqMiChmjB7V1yFERPQ7qfRFRERERER0sCR9ERERERERHSxJX0RERERERAdL0hcLhaRZTdoOljRd0lRJN0pat5sxNpY0XtK9ku6RdJ6kgWXbrpJuL+3TJe26sI6lL0jaT9IZ5fGu9XMl6duSti2Pv9x1Tno4/jyvz+Ks4XwcLGmf8nid8n6aImktSYdJulvShS3G2aW8b6ZKmixp80V5HBERERF9ITdyiUXpl7Z/BCBpZ+AU4KPNOkpaBbgE+ITtCZIE7A4sL2kocDLwEdsPSVoT+IOkB23fvkiOpIckLWn7lQXcfVfgKuAuANvH1rZ9GfgFMPv1Rdh7JA2wPWdhjd/1Hip2Ba6w/a0y9xeBHWw/1GL3ccCVti1pPeB/gHUWVqwRERERi4NU+mKRsf1s7ekgwG26HwL81PaEsq9t/9r248ARwHe7PtiXnycCX2s1WKkA3VWqPL8qbYMknS9pUqkU7VLa95N0maTfS7pf0vdL+wBJYyTdUaqLXyntwyXdXMa+XNJbSvt1kr4r6Xrg8BZx7SRpYpn/jyXZrW//ILAzcFKpTq1VYthD0mHAqsC1kq4t/WfV9t1D0pjyeE1JE8qxfqdhjq+V9tslHd/mHA4pldWflr6/rlVeZ0g6VtKNwJ76/+zde7xmY/3/8dfb2RiHkmocp1DCMBinUEhUk+IbqZQZFSlFB31NiagUKSIiFNIkCRH9QnKOyZzMjJzCKPENxTiL8f79sa47yz33fe+9Z/bMnn17Px+Peex1X+ta13Wtw57H+uzPtdYt7Vj6myzpPElDJb1b0q9q7W0r6bcd+ttb0p3l+G1VKz9c0kGS3kMV9H5S0lWSTgHeCFzcODfNbD9pu3Hd9XQNRkRERHSFBH2xQEnaX9LdwHeBAzpUXR+Y1Gbdei3WTSzl7YwDNrK9AbBfKTsE+KPtTYHtqAKrZcq6kcAewAhgD0mrlbJVbK9vewRwRqn7M+Dg0vZ04Ou1flew/Xbb328zruuBLWxvBPwS+N/6Stt/Ai4Gvmx7pO27a+tOAB4AtrO9XYd9BzgeOLns6/81CiXtCKwNbFb2bxNJb+vQzpuBU8u+Pg58prbuWdtbA38AvgbsYHtjqnPzReAKYIvaMd4DOLdVJ5KGAUdQBXvvBOaYCmz7d8ApwHG2t7O9Hy8dj+Pa7YCkXSXdDlwKfLxNnX3L9M+Js5+e1a6piIiIiEEhQV8sULZPsr0mcDBVYDA3xJwZmlZlddOA8ZI+CjSmWe4IjJM0FbgaWApYvay70vYs289STatcA7gHeKOkH0p6F/C4pOWpArtrynZnAfWgqWVQU7MqcJmk6VSZyk6B67zYCjinLJ9dK9+x/JsCTKaa6rh2h3b+bvuGsvxzoP5MXGNft6AK0m4ox3YMsEaZ3vp7YGdJiwGjgYva9LM5cLXth23/h56PY6/ZvtD2OlRTQ7/Zps6ptkfZHrXokOX7q+uIiIiIAZGgLwbKL6luutu5Fdikw7pRTWUbU555a2M0cFJpc1IJOgR8oGTQRtpe3fZtpf5ztW1nA4vZfhTYkCpA3B84vUN/DU/1sP6HwIklc/gpqsBzXtQD3+a2WgXFAr5TOwZr2f5JL9tv/tzYVwFX1Npc1/YnyrpzgQ8C2wM3236iD331K9vXAmtKes387CciIiJioCXoiwWmvIClYTRwV4fqJwJjJG1e2/6jkl5P9RKXr0gaXsqHA18FWk6hlLQIsJrtq6imT64ADAUuAz5XXhKDpI16GP9rgEVsnw8cCmxsexbwqKRtSrWPAde0a6OF5YF/lOUxbeo8ASzby3X/lPSWss+71spvAD5UlveslV8GfFzSUABJq0h6bYfxri5py7L8Yarpqc1uAraStFZpc4ikN5V1V1MF6PvQOXs3AdhW0oqSFgd271C31yStVTvfGwNLAP/qj7YjIiIiFlZ5e2fML0Mk3V/7fCywhqqvGngeeJT2QQ62/ynpQ8D3ShDyInAtcIHt/5N0MPDbEhA8D/yv7altmlsU+HmZiimqZ8AeKy80+QEwrQQCM4H3dtinVYAzSkAF8JXycwxwSnmpyT3A3h3aaHY4cJ6kf1AFS29oUeeXwGnlxS27Na07Ffh/kh4sz/WNo3rT59+BGVTBLVQvkvmFpAOB8xsb275c0luAG0ss9CTwUeChNuO9jSoY/zFV0H5ycwXbD0saC5wjaclS/DXgTtuzJV0CjKXz+X9Q0uHAjcCDVFNPF21Xvw8+AOwl6XngGWCP2otdIiIiIrqScr8TEb1RMqqX2F5/gIeyQC05bG0PG/ODgR5GRBQzjxo90EOIiFhoSZpku/kxqEzvjIiIiIiI6GaZ3hkDStJOwNFNxffa3rVV/V60dxK173Qrjrd9Rqv6C4qkQ5jzubTzbB85EOPpRNKKVF9i3uwd8yPLJ2kCsGRT8cdsT5+HNvdmzu9GvMH2/n1ta8QqyzMxmYWIiIgYxDK9MyKig1GjRnnixIkDPYyIiIiIHmV6Z0RERERExCtQgr6IiIiIiIgulqAvIiIiIiKiiyXoi4iIiIiI6GIJ+iIiIiIiIrpYgr6IiIiIiIgulqAvIiIiIiKiiyXoi4iIiIiI6GIJ+iIiIiIiIrpYgr6IiIiIiIgulqAvIiIiIiKiiyXoi4iIiIiI6GKLDfQAIiIWZtP/MYvh4y4d6GFEvOLMPGr0QA8hIqJrJNMXERERERHRxRL0RUREREREdLEEfdHvJD3Zomw/SdMlTZV0vaR1O2y/rSRL2rlWdomkbcvyEpJ+IOluSXdJukjSqmXdapKuknSbpFslHTgX4x8uaUZft2vT1lhJJ3ZYv5KkCZKmSNpmHvsaLukjtc+jJJ0wL20uLOrHsVxLe5Xldco1NUXSmpIOKOd+fIe2ti3b3CrpmgW1DxEREREDJc/0xYLyC9unAEh6H3As8K4O9e8HDgF+22Ldt4FlgTfZni1pb+ACSZsDLwBfsj1Z0rLAJElX2P5Lf+5MP3oHcLvtMc0rJC1qe3Yf2hoOfAT4BYDticDE/hhkX8zFuPukcR0VuwAX2f566fszwLtt39tmbCsAPwLeZftvkl47v8YZERERsbBIpi8WCNuP1z4uA7iHTW4BZkl6Z71Q0hBgb+ALjcDC9hnAc8D2th+0PbmUPwHcBqzSrhNJa0n6g6RbJE2WtGbT+qUknVGylFMkbVfKX5bBa8pE7i3pzpJF2qpD3yOB7wLvKZmnpSU9KekbkiYAW0o6TNLNkmZIOlWSOoz7KGCb0tYXSkbrklL/1ZJ+I2mapJskbVDKD5f0U0lXS7pH0gEdxjtc0u2Szirt/LqcDyTNLGO9Hthd0o6SbixjO0/SUEnvlvSrWnvbSmoV1DfWtzyOZcwHSXoP8HngkyW7ewrwRuBiSV9o0+xHgAts/w3A9kPt+o+IiIjoFgn6YoGRtL+ku6kCnbbBRc23gK81la0F/K0piIQqo7VeU3/DgY2ACR36GA+cZHtD4K3Ag03r9wewPQL4MHCWpKXaNSZpGHAEVZDyTqDtNFbbU4HDgHNtj7T9DFVAPMP25ravB060vant9YGlgfd2GPc44LrS1nFN3R0BTLG9AfBV4Ge1desAOwGbAV+XtHi7MQNvBk4t7TwOfKa27lnbWwN/oDpvO9jemOrcfBG4AthC0jKl/h7Aua066c1xtP074BTgONvb2d4PeADYrsX+N7wJeFUJcic1pom26H9fSRMlTZz99Ky2ByMiIiJiMEjQFwuM7ZNsrwkczJzBXKv61wE0PesmWmcJX1YuaShwPvD5FgFio86ywCq2Lyz9PWv76aZqWwNnl/W3A/dRBQ7tbA5cbfth2/+hTVDTwewy7obtVD3zNx3YHlivl+NuVt+PPwIrSlq+rLvU9nO2HwEeAl7XoZ2/276hLP+8tNvQ2NctqIK0GyRNBcYAa9h+Afg9sLOkxYDRwEVt+pnX49jOYsAmpe+dgEMlzXE+bZ9qe5TtUYsOWb55dURERMSgkmf6YiD8Eji5l3WPpHq274Xy+a/AGpKWLdM3GzamPP9XMlXnA+NtX9ChbfWi/3Z1XuDlfzSpZ/96mrraybONaaslo/gjYJTtv0s6vPTTm3E3a7VNY5zP1cpm0/n/heZ9q39+qtbXFbY/3GL7c6myp/8Gbm46hz311R/uBx6x/RTwlKRrgQ2BO+dDXxERERELhWT6YoGQtHbt42jgrt5sZ/ty4FVUN+aUm/WzgGMlLVra3gsYAvyxPPP2E+A228f20PbjwP2SdintLNl4Rq3mWmDPsv5NwOrAHcBMYKSkRSStRjU1EqqppNtKWrEEn7v3Zj/baASSj5TM5W49jPsJqhfctFLfj22pAp+WGdAerC5py7L8YeD6FnVuAraStFbpb0gtm3Y1VYC+D52zd/15HOsuonrucbFyzDaneu4zIiIiomsl0xfzwxBJ99c+H0uVndsBeB54lGrKX28dycunAX4F+B5wp6QXgduBXW1b0tbAx4DpZWohwFfL81+tfAz4saRvlLHtDrxYW/8j4JQyvfIFYKzt5yTdANwLTAdmAI2XxzxYMnI3Uj1nNxlYtA/7+l+2H5N0WuljJnBzD+OeBrwg6RbgTGBKrf7hwBmSpgFP07fjX3cbMEbSj6kC9zkytrYfljQWOEfSkqX4a8Cd5W2rlwBjO42hP49jU7u3Sfo91bF6ETjddr98PUdERETEwkr2/JhBFRHdprwY55LyUplXjCWHre1hY34w0MOIeMWZedTogR5CRMSgI2mS7VHN5ZneGRERERER0cUyvTMGjKSdgKObiu+1vet86Osk5vzOvOPLd/zNd5IOYc7n0s6zfeSC6L8vJK0IXNli1TvmR5ZP1XcSLtlU/DHb0+ehzb2BA5uKb7C9f1/bGrHK8kxMxiEiIiIGsUzvjIjoYNSoUZ44ceJADyMiIiKiR5neGRERERER8QqUoC8iIiIiIqKLJeiLiIiIiIjoYgn6IiIiIiIiuliCvoiIiIiIiC6WoC8iIiIiIqKLJeiLiIiIiIjoYgn6IiIiIiIiuliCvoiIiIiIiC6WoC8iIiIiIqKLJeiLiIiIiIjoYgn6IiIiIiIiuthiAz2AiIiF2fR/zGL4uEsHehgRrxgzjxo90EOIiOg6yfRFRERERER0sQR9ERERERERXWzQBH2SnmxRtp+k6ZKmSrpe0rodth8iaXypP6PUH1rWrSrpIkl3Sbpb0vGSlijr3ilpUtlukqTtexjnx0vdaaWf95dySfpa6eNOSVdJWq82tksl3S7pVklHzcuxajGmOY7dYCbpcEkHzec+xko6sZ/bPL1xjUr6an+23YcxLCnpD+V3Zo/53NcunX4nW9QfLmlGWR4l6YSy/LIxS9qm/J5MlbR0i3ZGSrqx1JlW309Jn5X0V0mW9Jr+2M+IiIiIhd2gCfra+IXtEbZHAt8Fju1Q90Dgn6X++sAngOclCbgA+I3ttYE3AUOBI8t2jwA72x4BjAHObteBpFWBQ4CtbW8AbAFMK6v3B94KbGj7TcB3gAVOBbQAACAASURBVIslLVXWf8/2OsBGwFaS3t2nIxG9UoLvAbnubX/S9l/KxwEJ+qiur8Vtj7R97nzuaxeg10Ffne2Jtg8oH5vHvCfV78tI28+02PxpYC/b6wHvAn4gaYWy7gZgB+C+uRlXRERExGA0qIM+24/XPi4DuEP1YcA/atveYfs5YHvgWdtnlPLZwBeAj0saYnuK7QfKZrcCS0lask0frwWeAJ4sbT1p+96y7mDgc7afLusuB/4E7Gn7adtXlfL/AJOBVdvtiKTXSbpQ0i3l31tL+RdLdnGGpM+32G5bSZfUPp8oaWxZninp2yVDMlHSxpIuK5nP/WrbXy3p1yUrOb4Eze3GOVPSEZIml+znOqX8ZZm6Mt7h5d/tJSM2o7S/g6QbSoZ0s1rzG0r6Yynfp9bWlyXdXDI8R5Sy4ZJuk/SjcmxXazPevUsW9hpgq1r5SpLOL+3eLGmr2n78tByTeyQdUMqXKZnbW8p+7FHKry4ZrKOApUumarykb0o6sNbfkY22WoxxqKQra8e0kUnu8dhJei3wc2Bk6XvNPp63Zcr+3ixpSq3vEyQdVpZ3knRtuSbfBxzTQ1+blON0I9UfRhrl20q6pMWYPwV8EDhM0vhWbdq+0/ZdZfkB4CFgpfJ5iu2ZrbaLiIiI6FaDOugDkLS/pLupMn0tb5SLnwIHl6DmW5LWLuXrAZPqFUsw+TdgraY2PgBMKcFiK7cA/wTulXSGpJ3LGJcDlrF9d1P9iaX/+v6sAOwMXNlhX04ArrG9IbAxcKukTYC9gc2pMoz7SNqoQxut/N32lsB1wJnAbqWtb9TqbAR8niqD80ZqwVEbj9jeGDgZ6M2UzLWA44ENgHWAjwBbl23r2bENgNHAllQBwMqSdgTWBjYDRgKbSHpbqf9m4Ge2N7I9R5ZH0jDgiLI/7+TlGarjgeNsb0p1DZxeW7cOsFPp8+uSFqfKLj1ge8OSVf59vS/b44BnSqZqT+AnVFlkVGUhPwS0DGiAZ4FdyzHdDvh+LfDueOxsPwR8Eriu9N18Pda1Om+HAH8sx2E7qoBuGWAcsIek7aiuzb1t/wm4GPhyD32dARxQrrs5tBjzj2vt7tlh/ACUPxQsAXTa11bb7avqjx8TZz89qy+bRkRERCx0Bn3QZ/sk22tSZdK+1qHeVKog5Rjg1cDNkt4CiNYZwpeVq3r+7mjgUx36mE11w78bcCdwnKTDOwy/uY/FgHOAE2zf02G77aluxrE92/Ysqpv7C20/ZftJqimr23Roo5WLy8/pwATbT9h+GHhWL02P+7Pt+22/CEwFhvfQ5gXl56Re1AW41/b00v6twJW2XcZU3/4i28/YfgS4iiro2rH8m0KV0VuHKggEuM/2TR363Ry42vbDJdtan/q4A3CipKlUx2g5ScuWdZfafq6M4yHgdWWsO0g6WtI25fy0VTJP/ypB+o5Uf1j4V5vqAr4taRrwB2CV0if0/tj1RqvztiMwrhyHq4GlgNVL9nof4ArgxB6CyZd2RFoeWMH2NaWo7dTpuVEC+bOpgtAX+7Kt7VNtj7I9atEhy/fnsCIiIiIWuG76nr5fUgKhdmrB0AWSXgTeQ5Wd+0C9XsnMrUbJDqh6Vu9CqueEOt7QlpvsPwN/lnQFcIbtwyU9JemNTcHcxsA1tc+nAnfZ/kGPezunttMsa17g5YH+Uk3rGxnMF2vLjc+LNdUBmE3P11Cjfr1up3E091sfU72v5kDdVMfgOyUb9F+ShgNP9TDOVm02LAJs2fz8WEmwzXE8bN9ZMq/vAb4j6XLb9WxpK6cDY4HXU2Wl29mTaqriJraflzSTl45fb49db7Q6bwI+YPuOFvVHAP8CVu5DH+3+4DLPyu/wpcDXegj2IyIiIrreoM701aZoQjXV764OdbeS9KqyvATV9L37qKZRDpG0V1m3KPB94EzbT5cM16XAV2zf0MN4Vpa0ca1oJC+9MOIY4ASVtw1K2oEqO/eL8vlbwPJUUyd7ciXw6cZ4yw3utcAuqt4EugywK9U0zbr7gHVVvQ1xeeAdvehrfphJFfBSjtcb5qKN90taStKKwLbAzcBlVM9iNt7Kukp5Jqw3JgDbSlqxTNHcvbbucuCzjQ+SRnZqSNLKwNO2fw58j7KvTZ4v/TRcSJUl3rTsRzvLAw+VgG87YI1OY+lnlwGfa0wnbUwflrQG8CWqqb/vlrR5qf8EsGyrhgBsPwbMkrR1KepxumZvlN/vC6mm857XH21GREREDGaDKdM3RNL9tc/HAmuU4Ol54FHKc1FtrAmcXG5YF6EK5M63bUm7Aj+SdGhZ9zteen7ss1TPSh1a1gPsWJ41arY48L1y0/8s8DCwX1n3Q+BVwHRJs4H/A95v+xm99NbP24HJ5Z76RNunN3dQHAicKukTVJmYT9u+UdKZVFlGgNNtT6lvZPvvkn5F9UbRu6imQQ6E84G9yjTBm6mmwvbVn6nO4erAN8sLOx4oU3ZvLMfwSeCjVMeoI9sPlqm4NwIPUk0PXbSsPgA4qUypXIwqwN6vVTvFCKrn3V6kujY/3aLOqcA0SZNt72n7P5KuAh4r04TbGQ/8VtJEqum1t/e0b/3om8APqMYtYGZ5bvUnwEG2HyjX5JmSNqXKvp+m6qU0u7XJku8N/FTS03QOdvvig8DbgBVVXlQEjLU9tYzlf6kyqtMk/c72J/up34iIiIiFkqrZiBExkMoLXCYDuzfePBkLhyWHre1hY+ZmxnVEzI2ZR40e6CFERAxakibZHtVcPqind0Z0A1VfYP5XqhevJOCLiIiIiH41mKZ39oqknajesll3r+1d+7mfCUDz9/V9zPb0fuzjEF7+bBnAebaPbFV/oEi6kDmfyzvYdn9N1+tXC+Lc9YWrL2x/Y71M0gjmfJvlc7Y3p58syPMm6STm/HqP412+H3Mu25zvxwhgxCrLMzGZh4iIiBjEMr0zIqKDUaNGeeLEiQM9jIiIiIgeZXpnRERERETEK1CCvoiIiIiIiC6WoC8iIiIiIqKLJeiLiIiIiIjoYgn6IiIiIiIiuliCvoiIiIiIiC6WoC8iIiIiIqKLJeiLiIiIiIjoYgn6IiIiIiIiuliCvoiIiIiIiC6WoC8iIiIiIqKLJeiLiIiIiIjoYosN9AAiIhZm0/8xi+HjLh3oYUQMejOPGj3QQ4iIeMVKpi8iIiIiIqKLJeiLiIiIiIjoYgn6IiIiIiIiuliCvkFA0pMtyvaTNF3SVEnXS1q3hzY2k3StpDsk3S7pdElDJI2VdGJT3asljWoqu1jSjP7Zo/lH0raSLunH9oZL+kjt8yhJJ8xlW3Mc1w515zgvfeinecxz3da8KP2u3If6/z13kt4naVxZXknSBElTJG0jaXdJt0m6qk0775Q0qfx+TJK0fW3dJqX8r5JOkKR53c+IiIiIhV2CvsHrF7ZH2B4JfBc4tl1FSa8DzgMOtv1m4C3A74Fle9ORpP8B5gg8+5OkhfWlQsOB/wZQtifaPmDghtMrw6mNeQCNBXod9NXZvtj2UeXjO4DbbW9k+zrgE8BnbG/XZvNHgJ1tjwDGAGfX1p0M7AusXf69a27GFxERETGYJOgbpGw/Xvu4DOAO1fcHzrJ9Y9nWtn9t+5899SNpKPBF4Fu9qLuPpJsl3SLpfElDSvlK5fPN5d9WpfxwSadKuhz4maQ1JF0paVr5uXqpt7ukGaXda0vZUpLOKFmbKZLaBQDNY3y1pN+UPm6StEFtLGdL+qOkuyTtUzY5CtimZFS/0JSNGlobwzRJHyjlJ0uaKOlWSUf0Zlxlu70l3SnpGmCrWnmn49fjmEvZypJ+X+p9t4dxPCnpyHK8byp/NOg0josk7VWWPyVpvKTdgFHA+DKOpdv09S5Vmefrgf+plY+VdKKkxh813lPa+TqwNXCKpGNatWl7iu0HysdbgaUkLSlpGLCc7RttG/gZsEunYxERERHRDRbW7Er0gqT9qQKyJYDtO1RdHzirw/o9JG1d+7xWbfmbwPeBp3sxpAtsn1bG9i2qjMwPgeOB42xfXwK5y6iyjQCbAFvbfkbSb4Gf2T5L0seBE6huyg8DdrL9D0krlO32B7A9QtI6wOWS3tSLMR4BTLG9S5n29zNgZFm3AbAFVRA9RdKlwDjgINvvLfu1ba2tQ4FZJaOEpFeV8kNs/1vSosCVkjawPa3ToEpAckQ5HrOAq4ApZXWn49ebMY8t+7gR8Bxwh6Qf2v57m+EsA9xk+5ASIO5DFfS3G8e+wA2S7gW+BGxR9v+zZRwT2+zzUsBpVNfuX4Fzm+vYnirpMGCU7c+W7bbr1G6TD1Cd7+ckrQLcX1t3P7BKm7HtW/aLRZdbqRfdRERERCy8EvQNYrZPAk5S9fzW16imss2Ncxs31FA9e1Z+jgTWsv0FScN70c76JdhbARhKFRQA7ACsq5cen1pOUmNq6cW2nynLW/JStudsqgwPwA3AmZJ+BVxQyramCiixfbuk+4DeBH1bUwUC2P6jpBUlLV/WXVTG8oyq58U2Ax7r0NYOwIcaH2w/WhY/WIKGxYBhwLpAx6AP2By42vbDAJLOre1Pp+PX2zFfaXtWafsvwBpAu6DvP0DjuchJwDs7jcP2P0tgdhWwq+1/97CvDesA99q+q4zr55RAqz9IWg84GtixUdSiWssMue1TgVMBlhy2dqcsekRERMRCL0Ffd/gl1bNK7dxKlUG6qI/tbglsImkm1bXyWklX2962Tf0zgV1s31KyS416iwBb1oI7AErw8FSH/g1gez9JmwOjgaklGJ3bF3B0uvFvvrnv6WZfzXUkvQE4CNjU9qOSzgSW6uXY2vXX6fj1dszP1ZZn0/l3//ky/bG5bstxFCOAf9H3Z/jmS0AlaVXgQmAv23eX4vuBVWvVVgUeaN42IiIiotvkmb5BStLatY+jgbs6VD8RGFMCp8b2H5X0+k592D7Z9sq2h1NlyO7sEPBB9WKYByUtDuxZK78cqGcSRzZvWPyJlzJnewLXl/pr2p5g+zCql3SsBlzb6KNM61wduKPT/hT17bYFHqk9H/l+Vc8KrkgVsN4MPEH7F94079ergOWoAtlZ5Vm4d/diTAATgG1L5nFxYPcO/dSPX1/HPC9ajkPSZlT7uRFwUAl86cU4bgfeIGnN8vnD/THIMgX4UuArtm9olNt+EHhC0haqIua96PsfQiIiIiIGnWT6BochkurPIh0LrCFpB+B54FE6TO0s0+8+BHxP0muBF6mCnwvabTOXDqUKXu4DpvPSDf8BVNNQp1Fdc9cC+7XY/gDgp5K+DDwM7F3KjylBroArgVuoAoZTJE0HXgDGlue2ehrj4cAZZSxP8/Lj9meqYGF14Ju2H5D0MPCCpFuoMplTavW/VfZrBlVG7AjbF0iaQpVdvYdqamqPbD8o6XDgRuBBYDKwaO24tDt+vRlzY9rpvJpjHJIOpHoub+/S95eozuH2pe9TJD1Diwyh7WfLNNhLJT1CFeSv3w/j/CzVc6mHSjq0lO1o+yHg02VcSwP/r/yLiIiI6Gp6aRZXxCtXCbietP29gR5Lbw3GMQ9GSw5b28PG/GCghxEx6M08avRADyEioutJmmR7ju+FTqYvIqKDEassz8TcrEZERMQglqCvi0jaiepthXX32t61n/s5idr3yBXH2z6jP/uZF5L2Bg5sKr7B9v6t6ts+fL4PqpA0AViyqfhjtqf3pZ15HXN/jaOXfV0IvKGp+GDbl7Wq38s2F8j1HhERETHYZXpnREQHo0aN8sSJvflKwIiIiIiB1W56Z97eGRERERER0cUS9EVERERERHSxBH0RERERERFdLEFfREREREREF0vQFxERERER0cUS9EVERERERHSxBH0RERERERFdLEFfREREREREF0vQFxERERER0cUS9EVERERERHSxBH0RERERERFdbLGBHkBExMJs+j9mMXzcpQM9jIhBY+ZRowd6CBER0SSZvoiIiIiIiC6WoC8iIiIiIqKLJeiLiIiIiIjoYgn6FnKSnuxh/QqSPlP7vLKkX/dj/2Mlndhf7TW1/TtJK7QoP1zSQWX5G5J2KMuflzRkLvpZTdJVkm6TdKukA+d99B3763jOupGk4ZJmLIB+Xna9d6j3e0mPSbqkqfwNkiZIukvSuZKWmH+jjYiIiFg4JOhbCKgyt+diBeC/N8G2H7C9W/+MbP6y/R7bj/VQ5zDbfygfPw/0OegDXgC+ZPstwBbA/pLWnYt2BjVJ3fDippdd7x0cA3ysRfnRwHG21wYeBT7Rj2OLiIiIWCgl6BsgJTNym6QfAZOBQyXdLGmapCNa1B8q6UpJkyVNl/T+suooYE1JUyUdU8+4lIzGerU2rpa0iaRlJP209Del1lY7K5fMyV2Svltr78na8m6SzizLZ0o6uWTX7pH09tLfbY06pd5MSa8py4dIukPSH4A31+qcWdo+AFgZuKq0+wlJx9Xq7SPp2FaDt/2g7cll+QngNmCV2jE5TtK1ZXybSrqg7Ou3Oh0USb+RNKlkD/dtWvf9cq6ulLRSKRsp6aZyji+U9CpJb5H059p2wyVNK8ubSLqm9HGZpGEdxrJpaffGch00roGxks6T9FvgckmvLuOeVsayQan33+xq+TyjjKVxnZ5W9vNySUvXxneLpBuB/Xs4VmPLcW11He1Yxj25jHWopDVKvddIWkTSdZJ2pOl6b9ef7SuBJ5rGIGB7oJEJPwvYpdO4IyIiIrpBgr6B9WbgZ8DBVEHIZsBIYBNJb2uq+yywq+2Nge2A75eb2HHA3bZH2v5y0za/BD4IUAKGlW1PAg4B/mh709LWMZKW6TDOkcAewAhgD0mr9WLfXkV1g/0F4LfAccB6wAhJI+sVJW0CfAjYCPgfYNPmxmyfADwAbGd7u7Jv75O0eKmyN3BGT4OSNLz0M6FW/B/bbwNOAS6iCmDWB8ZKWrFDcx+3vQkwCjigVncZYHI5V9cAXy/lPwMOtr0BMB34uu3bgCUkvbHU2QP4VdmvHwK7lT5+ChzZYSxnAPvZ3hKY3bRuS2CM7e2BI4ApZQxfLWPqydrASbbXAx4DPlDr84DSZ2/McR2VoP9rwA7leE0Evmj7Pqqs3CnAl4C/2L6cztd7T1YEHrP9Qvl8PyX4byZpX0kTJU2c/fSsPnYTERERsXBJ0Dew7rN9E7Bj+TeFKuu3DtWNdp2Ab5cs0B+oblZf10P7vwJ2L8sfBM4ryzsC4yRNBa4GlgJW79DOlbZn2X4W+AuwRs+7xm9tmyq4+aft6bZfBG4FhjfV3Qa40PbTth8HLu6pcdtPAX8E3itpHWBx29M7bSNpKHA+8PnST0Ojv+nArSUz+BxwD9ApwD1A0i3ATaVe45y9CJxbln8ObC1peWAF29eU8rOARmD/K0pwThUUnUv1B4H1gSvKefoasGqb/VoBWNb2n0rRL5qqXGH732V5a+BsANt/BFYsY+vkXttTy/IkYHiL/Tm7hzag9XW0BbAucEPZzzGlHNunA8sC+wEHtW6yT9SizK0q2j7V9ijboxYd0tPhiYiIiFi4dcMzPoPZU+WngO/Y/nGHunsCKwGb2H5e0kyqYK0t2/+Q9K8yhW8P4FO1/j5g+45ejvO52vJsXrpu6jfMzWNpbPNi0/Yv0vq6a3nz3YPTqbJVt9NDlq9kzs4Hxtu+YB7HiqRtgR2ALW0/Lelq2p+PnvbtXOA8SRcAtn2XpBFUAWhvsmitgpm6p2rL7QKfF3j5H4Hq+9J8/pcu7fT1nLW6jkQVlH64ubKql/Y0At2hNE3XnAuPACtIWqxk+1alyh5HREREdLVk+hYOlwEfL5koJK0i6bVNdZYHHioB33a8lG17giob0s4vgf8Flq9lwi4DPlemhyJpo7kc9z/LM2mLALvOZRsA1wK7Slpa0rLAzm3qvWxfbU+gyrB9BDinXeNlP38C3Ga75XN/c2F54NES8K1DlbFqWARovEznI8D1tmcBj0rappR/jGrqJ7bvpgqCDuWlDOEdwEqStiz7sLhqz2fW2X4UeEJSYwwf6jDua6n+gNAIXB8pWc+ZwMalfGPgDZ12vryAZ5akrUvRnp3qd3ATsJWktUrfQyS9qaw7GhgPHAacVsp6ut47jdnAVbx0bsZQTeeNiIiI6GoJ+hYC5VmlXwA3SppO9aKJ5hvb8cAoSROpbrBvL9v+i2pq3Iw2L7b4NVUQ8Kta2TeBxYFp5YUf35zLoY8DLqGaZvngXLZBecnKucBUqmzcdW2qngr8P0lX1cp+BdxQAp92tqIKsrYvLwCZKuk9czve4vfAYmW67TepgpeGp4D1JE2ieq7xG6V8DNXzk9Oonm/7Rm2bc4GPlv3B9n+ogpOjyxTSqcBbO4znE8Cp5aUqAto9iHY41XU0jeqlKGNK+fnAq8sUy08Dd3bc+8rewEmlz2d6UX8Oth8GxgLnlDHdBKwj6e1Uz3YebXs88B9Je/fiegdA0nVU05nfIel+STuVVQcDX5T0V6pn/H4yN+OOiIiIGExU/fE7YnBS9T1sx5W3Nb5iSRpq+8myPA4YZnu+fh/hK8WSw9b2sDE/GOhhRAwaM48aPdBDiIh4xZI0yfao5vJk+mJQUvUl3XcCz7zSA75idMlgzqB6MU7Hr5uIiIiIiFeOZPoCgDL97eim4nttz8uzegtU+cqEVgHgO8q0wIWizXkh6SSq6ap1x9vu8esqFoQFeR2Vl900vzX0Odub92c/o0aN8sSJE/uzyYiIiIj5ol2mL0FfREQHCfoiIiJisMj0zoiIiIiIiFegBH0RERERERFdLEFfREREREREF0vQFxERERER0cUS9EVERERERHSxBH0RERERERFdLEFfREREREREF0vQFxERERER0cUS9EVERERERHSxBH0RERERERFdLEFfREREREREF1tsoAcQEbEwm/6PWQwfd+lADyNiwM08avRADyEiIuZSMn0RERERERFdLEFfREREREREF0vQFxERERER0cUS9C1Akp5sUbafpOmSpkq6XtK6HbYfIml8qT+j1B9a1q0q6SJJd0m6W9LxkpYo694paVLZbpKk7edy/DMlvWZutu3Q5raSLmmz7neSVujP/hYGPR1HSX/qY3ttj+FAkjRS0nv6uM1/j039OEg6RtKt5edKkiZImiJpmxZtDJF0qaTbyzZH1dYtKelcSX8tbQyf+z2MiIiIGBwS9A28X9geYXsk8F3g2A51DwT+WeqvD3wCeF6SgAuA39heG3gTMBQ4smz3CLCz7RHAGODs+bQvfSKp44uEbL/H9mMLajydqLJAfl9sv3VB9LMAjAT6FPTVNR2HTwEb2/4y8A7gdtsb2b6uzebfs70OsBGwlaR3l/JPAI/aXgs4Djh6bscXERERMVgk6Btgth+vfVwGcIfqw4B/1La9w/ZzwPbAs7bPKOWzgS8AH5c0xPYU2w+UzW4FlpK0ZLtOJJ0saWLJkhzRtPrLkv5c/q1V6u9eMo+3SLq2lC0l6YySXZwiabtSPlbSeZJ+C1xe2lxO0oWS/iLplEZw1cj6SBou6TZJp5UxXS5p6VJnU0nTJN1YskAzOuzXcEnXSZpc/r21tu7Lkm4ubR1Rq3+bpB8Bk4HVGn2U/dqj1Bsm6dqSrZ3RyD5J+nAtK9vr4KKRES4ZvKsl/bpkrcaXAB9J7ypl1wP/U9t2GUk/LfsyRdL7S/kJkg4ryzuV8bb8/Zd0Zqn/J0n3SNqth+O0q6Q/lMB4mKQ7Ja0OfAPYoxyXPdr0tWI5n1Mk/RhQi+NwMdXvxgRJB1P9ceQ9pd2lm9u0/bTtq8ryf6jO3apl9fuBs8ryr4F3NI5pRERERLdK0LcQkLS/pLupbmYP6FD1p8DBJcD5lqS1S/l6wKR6xRJM/g1Yq6mNDwBTSrDYziG2RwEbAG+XtEFt3eO2NwNOBH5Qyg4DdrK9IfC+UrZ/GccI4MPAWZKWKuu2BMbYbkwz3Qz4EjACWJNaEFOzNnCS7fWAx8p+AJwB7Gd7S2B2h30CeAh4p+2NgT2AEwAk7Vja34wqO7WJpLeVbd4M/Mz2RsCosn5DYAfgGEnDgI8Al5Vs7YbAVEkrU2WRti/bbCpplx7G18pGwOeBdYE3UmWtlgJOA3YGtgFeX6t/CPBH25sC25UxLgOMowrAtiv7vbftFzv0OwzYGngvcFSn42T7QuD/qM75acDXbf+N6ro41/ZI2+e26efrwPXl+F4MrN5cwfb7gGdKO0c3tftMh31A1fTgnYErS9EqwN9Luy8As4AVW2y3r6o/fEyc/fSsTl1ERERELPQS9C0EbJ9ke03gYOBrHepNpbrxPwZ4NXCzpLdQZUdaZQhfVi5pPapA5FM9DOmDkiYDU6gCyvpzhufUfm5Zlm8AzpS0D7BoKduaMo3U9u3AfVTTTgGusP3vWpt/tn1PyVCeU7Ztdm/Zf6gC3OHlhn5Z241nv37Rw34tDpwmaTpwXm2/diz/plBlhdahCm4A7rN9U22fzrE92/Y/gWuATYGbgb0lHQ6MsP1EKb/a9sMluBgPNALJvviz7ftLgDYVGF7Gd6/tu2wb+Hmt/o7AOElTgauBpYDVbT8N7ANcAZxo++4e+v2N7Rdt/wV4Xa3tdsfpc8BXgOdsn9PcWAdva4zf9qXAo33YtiNV04fPAU6wfU+juEXVOX53bJ9qe5TtUYsOWb6/hhQRERExIPLl7AuXXwInd6pg+0mq5/cukPQi1TNTt/BS5gsAScsBqwF3l8+rAhcCe3W64Zf0BuAgYFPbj0o6kypw+O8Qmpdt7ydpc2A0VZZrJK1vrhueat6tHj4D1DOTs4Gle+ijlS8A/6TKxi0CPFvKBXzH9o/rlVW95KM+1pb92b62ZAZHA2dLOgZ4vFXdudC8343f2XbTgAV8wPYdLdaNAP4FrNzHflX7OcdxKlYBXgReJ2mRHrKIzTpNaZ4XpwJ32f5Brex+qt+L+0tQuDzw71YbR0RERHSLZPoGWG2KJlRBw10deTUbvQAAE5JJREFU6m4l6VVleQmqTNV9VFPXhkjaq6xbFPg+cKbtp0tG7FLgK7Zv6GFIy1EFOrMkvQ54d9P6PWo/byz9rWl7gu3DqF4asxpwLbBnWf8mqml7rQIRgM0kvaE8Y7YHcH0PYwTA9qPAE5K2KEUf6mGT5YEHS0DyMV7KSl5G9fxj402oq0h6bYvtr6WaIrmopJWoslR/lrQG8JDt04CfABsDE6imxr6mnI8PU2UG+8PtwBskrVk+f7i27jLgc7Vn/zYqP9egmkK7EfDuEqT3VcvjVIKnM6imud4GfLHUfwJYtoc269fJu4FXzcW45iDpW1Tn+/NNqy6mepkRwG5UU2HnV9AZERERsVBIpm/BGiLp/trnY4E1JO0APE81tW1Myy0rawInlxv6RagCufNtW9KuwI8kHVrW/Q74atnus1TP9h1a1gPsaPuh5g5s3yJpCtULX+6hmrpZt6SkCaWPRrBxTAleRRWA3kIVmJxSplK+AIy1/Vybd2bcSPXc2AiqIODCDseg2Seopmw+RTWdsdMDWD8Czpe0O3AVJYtn+/IyTfbGMr4ngY8y5zOCF1JNab2FKjv1v7b/T9IYqhfcPF+23cv2g5K+UvoR8DvbF/Vhv9qy/aykfYFLJT1CFSSvX1Z/k+pZy2nlOpkpaWeqYPQg2w9I+gTVdNxNbT/bqo82/bY7TvsB19m+rkwrvVnSpWXfG1NNv9Pmub4jgHPKdOJrqJ5DnSclq30I1TU4uYz1RNunUx2HsyX9lSrD19MfCiIiIiIGPeWP3DGYSRpaprwiaRwwzPaBAzys6CJLDlvbw8b8oOeKEV1u5lGjB3oIERHRA0mTygsZXyaZvhjsRpeM2mJUU13HDuxwIiIiIiIWLsn0LYQk7cScXxp9r+1d+7mfCUDz9/V9zPb0/uxnQVtQx29eSFqRl75GoO4dtv+1gMZwCLB7U/F5to+cD33tDTRnYG+wvf88tjvfr+FRo0Z54sSJ/dVcRERExHzTLtOXoC8iooMEfRERETFYtAv68vbOiIiIiIiILpagLyIiIiIioosl6IuIiIiIiOhiCfoiIiIiIiK6WIK+iIiIiIiILpagLyIiIiIioosl6IuIiIiIiOhiCfoiIiIiIiK6WIK+iIiIiIiILpagLyIiIiIioosl6IuIiIiIiOhiiw30ACIiFmbT/zGL4eMuHehhRAyYmUeNHughRETEPEqmLyIiIiIioosl6IuIiIiIiOhiCfoiIiIiIiK6WI9Bn6QnW5TtJ2m6pKmSrpe0boftt5V0SVPZmZJ2K8uLSzpK0l2SZkj6s6R3l3UzJb2mXVuSdpE0TdLtZTy79LAvYyWt3NM+L0itjs9ctLGfpL36uM1wSTOayg6XdFDt80Hl2M6QdEujD0lXSxrVri1JW5fzeHv5t+/c713Lsc9xTQ5mzcd9PvUxVtKJ/dzm6Y3ffUlf7c+253I8PY5B0gqSfl2uy9skbbkgxhYRERExkOY20/cL2yNsjwS+Cxw7D2P4JjAMWN/2+sDOwLI9bSRpQ+B7wPttrwO8D/iepA06bDYWaBn0SVq0j+NeKEhazPYptn/Wz+3uB7wT2Kycl7cB6sV2rwd+AexXzsvWwKck5U0A84EqA5Kxt/1J238pHwc86KN3Yzge+H25NjcEbpu/Q4qIiIgYeHN1s2j78drHZQDPTTuShgD7AJ+z/Vxp+5+2f9WLzQ8Cvm373rLdvcB3gC+36Ws3YBQwvmQoly6ZxMMkXQ/sLunDJWM4Q9LRtW2flPR9SZMlXSlppVI+UtJNJdt4oaRXlfIDJP2llP+ylB0u6WxJfyxZzX1qwxtayz6Ml6SyzSaSrpE0SdJlkoaV8qslfVvSNcCB9UyRpLUk/aFk5iZLWrOXp6PZV4HPNM617Vm2z+rFdvsDZ9qeXLZ7BPhfYFy7DSS9rhy/W8q/t5byL5ZzMUPS51ts15z5PVHS2LI8sxyjGyVNlLRxOYZ3l4C2sf3VrY59m3HOlHREOa7TJa1TypszpDNK9nN4aff0UjZe0g6SbijXwGa15jdsdW1I+rKkm8u1dEQpG16yVD8CJgOrtRnv3pLuLNfJVrXylSSdX9q9WdJWtf34aTkm90g6oJQvI+nScm5mSNqjlF8taZSko4Cly+/VeEnflHRgrb8jG221OYctz0Gr61/S8pLukPTmUuccSfs0j6FNX8tR/fHiJwC2/2P7sVZ1IyIiIrrJXH9lg6T9gS8CSwDb91B9G0lTa59XBy4B1gL+1hRENrtK0uyyPBS4vSyvR5Xpq5tIFXTMwfavJX0WOMj2xLIPAM/a3lrVtM+bgE2AR4HLJe1i+zdUge1k21+SdBjwdeCzwM+oAtZrJH2jlH+eKsB5g+3nJK1QG8YGwBalvSmSGu+B36jszwPADcBWkiYAP6TKZD5cbrSPBD5etlnB9tvLfhxe62M8cJTtCyUtRefAfs2m8/J6qmzpssCytu/usO14Sc+U5SWAF8vyekBzcDixlLdzAnCN7V1VZVyHStoE2BvYnCrDOEHSNbandGin2d9tbynpOOBMqsBnKeBW4JRSZ45jD1zfoc1HbG8s6TNUf3j4ZA9jWAvYHdgXuBn4CFX2831UgXVjSnKra2N9YG1gM6pjcLGktwF/A94M7G37M606VfUHgiOorudZwFVA49gdDxxn+3pJqwOXAW8p69YBtqPKtt8h6WTgXcADtkeXtpev92V7nKTPlsw/koYDFwDHq8pCfqjsQzu9vv5tf7z8Hp8p6XjgVbZPK/3+dwxtvBF4GDhD1UyBScCBtp9qcfz2pTpnLLrcSh2ajIiIiFj4zXXQZ/sk4CRJHwG+BozpUP062+9tfJB0Zh+62q5ki5C0LdWNNlQ3wc0ZxlZlPTm3/NwUuNr2w6Wv8VRZgd9QBTSNej8HLig3vivYvqaUnwWcV5anUQVFvynbN1xk+xngGUlXUd0IPwb82fb9pd+pwPBSvj5wRQlOFwUebDHul3a+CtZWsX0hgO1ne9j3u+s3ybXgsTfHcc9a8DycKojvtG2n9rYH9ipjng3MkrQ1cGHjhlzSBcA2vBS49MbF5ed0YKjtJ4AnJD1bC8ZbHftOQd8F5eck4H96MYZ7bU8v7d8KXGnbkqaXvhpaXRtbAzvy0j4PpQoC/wbcZ/umDv1uzsuv53OBN5V1OwDr6qWk5nLl2gG4tGTdn5P0EPA6quP3PVXZ70tsX9dph23PlPQvSRuV7afY/leHTfp0/du+QtLuwElUUzR7azFgY6o/1EwoQeM44NAW+3AqcCrAksPWnquZDBERERELi/74cvZfAifP5bZ/BVaXtGy5Ie+LW6mma06rlW0M/KV19bYaf+Xv8Xm1mp5uAkdTBYzvAw6V1MhyNW/X+PxcrWw21XkRcKvtdi+amCM7Qd/2oS3bj0t6StIbbd/Tx80b5+XiWtkm9P289GZfXuDlmcylmtY3juuLvPwYv8hL136rY99Jo369bqdxNPdbH1O9r1bXhoDv2P5xfUUJslud/2btrtNFgC1LkFlvt3m8s4HFbN9ZMq/vAb4j6XLb3+ih79OpnqF9PfDTHur26fov2cO3AM8Arwbu76H9hvuB+21PKJ9/TYdpxxERERHdYq6e6ZO0du3jaOCuuWnH9tNUz9ecIGmJ0vYwSR/txebfA75SboAbN8JfBb7fYZsnaP+SmAnA2yW9pkwx/DDQyOItAuxWlj8CXG97FvCopG1K+ceAa8oN6Wq2r6J6lm0FqgwNwPslLSVpRWBbqul+7dwBrKTydkFVbzntNEWy8azl/SpvMZW0pKrnJufGd6gyucuVtpZT797CeRIwVlJjqt+KwNFUL/xp50rg06X+oqXPa4FdJA2RtAywK9CcYbqPKmO1ZMm8vqP3u9evZlL9wQFJGwNvmIs2Wl0blwEflzS0tL2KpNf2sr0JwLaS/n979x4jV1XAcfz7S0tBQK0UNIQiSGwUDFKaqhgMQSCKSEQTiBAeDcEQCVE0Gh5qBB8kGhMoRIIiT8EXFiugxkCgpv5RC22pgCBSK7FNC9VAsYUIRX7+cc7S6ewwZXdmdzqX3yfZzNwzd++e/e29kz1zHneGpJ0oQ0xH3EUZnkw9brchkdShz8/bvoVy3c3psNuW+nNGLKQMC31f/T3Gqtv5/0XKAiynANe3/Nz2OmzD9pPAGtX5gJTzZawfRkREREQMndfS07erpNZP0i8D9pN0DLCFMv+t29DO7fka8G3gEUn/pfRgfH1732R7paQLgDvrP3pbgPNtr+zybTcCP6hz0bbpQbC9XtJFlLlPAn5n+/b68nPAeyQtp8yP+nQtn1ePtyuwmjIHbQpwS22EiDJ3amPtRbkP+C1lTuO3bK+TNDLkrv33e1Fl8Zkr67GmAvMpPWndnA78UGWO4RbKP/tj7a2D0nu7O3C/pC31WN0a1CP1Xl8b7T+qQwYFzLd9Z5dvOw+4RtJZlJ6ec2wvqcOA76v7XNs+n8/2Gkm3Unp7H2dsQz/76TbgjDo08X7gb+M4xqhzA1gn6UBgST1/NgOnUTLqqv4dLgGWUIZFrqCcmwCfpzToH6ScV4uBz3Y53MHA9yS9TDkPzumwzzXAg5JW2D61nr+LgI11yO6YvNr5X8/Fz1BWld0kaTHlPeTi9jq8yqE/Rxl6PY2t12xEREREo8nOdJXtkbTZ9u7b37PrMS4BNttuX3wmonFqj/cK4CTb4xoJsKPYee9Z3nve/EFXI2JgnvhO7rgTETEsJC23Pbe9fCD394qI5lK5YfsqyqI1Q93gi4iIiGiCvvX0SfooZe5Wq3/Y/lRffsDY6nIVLfclq66wfcNk12XQJB0M3NxW/ILtDwygLl9l27llAL+0felk16UbSQsZPS/vAtvjmZs24VRub7BzW/HpI6uG7ggm8zys8yLv6fDS0dtZRbSjuXPnetmyZb1XLCIiImKCvVpPX4Z3RkR0kUZfREREDIsM74yIiIiIiHgdSqMvIiIiIiKiwdLoi4iIiIiIaLA0+iIiIiIiIhosjb6IiIiIiIgGy+qdERFdSNoEPDboejTMnsC/B12JhkmmEyO59l8ynRjJtf+GNdP9bO/VXjh1EDWJiBgij3Va+jjGT9KyZNpfyXRiJNf+S6YTI7n2X9MyzfDOiIiIiIiIBkujLyIiIiIiosHS6IuI6O6aQVeggZJp/yXTiZFc+y+ZTozk2n+NyjQLuURERERERDRYevoiIiIiIiIaLI2+iIgOJB0r6TFJqyRdOOj6DBNJ10vaIOnhlrI9JN0t6fH6+JZaLklX1pwflDRncDXfcUnaV9IiSY9K+ouk82p5ch0nSbtIuk/Sn2um36jl75C0tGb6C0nTavnOdXtVfX3/QdZ/RyZpiqQHJP2mbifTHkl6QtJDklZKWlbLcv33SNJ0SQsk/bW+v36wqbmm0RcR0UbSFOAq4GPAQcApkg4abK2Gyo3AsW1lFwL32J4F3FO3oWQ8q36dDVw9SXUcNi8BX7J9IHAYcG49J5Pr+L0AHGX7EGA2cKykw4DvApfXTJ8Bzqr7nwU8Y/udwOV1v+jsPODRlu1k2h8ftj275TYCuf57dwXwe9vvBg6hnLeNzDWNvoiI0d4PrLK92vaLwM+BEwZcp6FhezHwdFvxCcBN9flNwCdbyn/s4k/AdEl7T05Nh4ft9bZX1OebKP+Y7ENyHbeazea6uVP9MnAUsKCWt2c6kvUC4GhJmqTqDg1JM4GPA9fWbZFMJ0qu/x5IehNwBHAdgO0XbW+kobmm0RcRMdo+wJqW7bW1LMbvbbbXQ2nAAG+t5cl6jOoQuEOBpSTXntRhiCuBDcDdwN+BjbZfqru05vZKpvX1Z4EZk1vjoTAfOB94uW7PIJn2g4G7JC2XdHYty/XfmwOAfwE31OHI10rajYbmmkZfRMRonT5pzlLHEyNZj4Gk3YHbgC/Y/k+3XTuUJdc2tv9nezYwk9LDf2Cn3epjMt0OSccDG2wvby3usGsyHbvDbc+hDDE8V9IRXfZNrq/NVGAOcLXtQ4Hn2DqUs5OhzjWNvoiI0dYC+7ZszwTWDaguTfHUyDCY+rihlifr10jSTpQG309s/6oWJ9c+qEO6/kCZLzld0tT6Umtur2RaX38zo4cxv94dDnxC0hOUYfFHUXr+kmmPbK+rjxuAhZQPKXL992YtsNb20rq9gNIIbGSuafRFRIx2PzCrrjg3DTgZuGPAdRp2dwDz6vN5wO0t5WfUVdEOA54dGVYTW9V5TtcBj9q+rOWl5DpOkvaSNL0+fwNwDGWu5CLgxLpbe6YjWZ8I3Ovc7Hgbti+yPdP2/pT3zXttn0oy7Ymk3SS9ceQ58BHgYXL998T2k8AaSe+qRUcDj9DQXHNz9oiIDiQdR/mEegpwve1LB1yloSHpZ8CRwJ7AU8DFwK+BW4G3A/8ETrL9dG3MfJ+y2ufzwJm2lw2i3jsySR8C/gg8xNa5Ul+hzOtLruMg6b2URRqmUD4Ev9X2NyUdQOml2gN4ADjN9guSdgFupsynfBo42fbqwdR+xyfpSODLto9Ppr2p+S2sm1OBn9q+VNIMcv33RNJsyqJD04DVwJnU9wMalmsafREREREREQ2W4Z0RERERERENlkZfREREREREg6XRFxERERER0WBp9EVERERERDRYGn0RERERERENlkZfREREREREg6XRFxERERER0WBp9EVERERERDTY/wEAvMXdBCdZtgAAAABJRU5ErkJggg==\n", 877 | "text/plain": [ 878 | "
" 879 | ] 880 | }, 881 | "metadata": { 882 | "needs_background": "light" 883 | }, 884 | "output_type": "display_data" 885 | } 886 | ], 887 | "source": [ 888 | "_ = plt.figure(figsize=(10, 10))\n", 889 | "fi = pd.Series(index=features, data=clf.feature_importance())\n", 890 | "_ = fi.sort_values()[-20:].plot(kind='barh')" 891 | ] 892 | }, 893 | { 894 | "cell_type": "code", 895 | "execution_count": 12, 896 | "metadata": {}, 897 | "outputs": [ 898 | { 899 | "data": { 900 | "text/plain": [ 901 | "73.71866974918078" 902 | ] 903 | }, 904 | "execution_count": 12, 905 | "metadata": {}, 906 | "output_type": "execute_result" 907 | } 908 | ], 909 | "source": [ 910 | "rmse(target.values, oofs_df['target'].values)" 911 | ] 912 | }, 913 | { 914 | "cell_type": "code", 915 | "execution_count": 13, 916 | "metadata": {}, 917 | "outputs": [], 918 | "source": [ 919 | "predictions_test = current_test_pred\n", 920 | "predictions_test[predictions_test < 0] = 0" 921 | ] 922 | }, 923 | { 924 | "cell_type": "code", 925 | "execution_count": 14, 926 | "metadata": {}, 927 | "outputs": [ 928 | { 929 | "data": { 930 | "text/html": [ 931 | "
\n", 932 | "\n", 945 | "\n", 946 | " \n", 947 | " \n", 948 | " \n", 949 | " \n", 950 | " \n", 951 | " \n", 952 | " \n", 953 | " \n", 954 | " \n", 955 | " \n", 956 | " \n", 957 | " \n", 958 | " \n", 959 | " \n", 960 | " \n", 961 | " \n", 962 | " \n", 963 | " \n", 964 | " \n", 965 | " \n", 966 | " \n", 967 | " \n", 968 | " \n", 969 | " \n", 970 | " \n", 971 | " \n", 972 | " \n", 973 | " \n", 974 | " \n", 975 | " \n", 976 | " \n", 977 | " \n", 978 | " \n", 979 | " \n", 980 | " \n", 981 | " \n", 982 | " \n", 983 | " \n", 984 | " \n", 985 | " \n", 986 | " \n", 987 | " \n", 988 | " \n", 989 | " \n", 990 | " \n", 991 | " \n", 992 | " \n", 993 | " \n", 994 | " \n", 995 | " \n", 996 | " \n", 997 | " \n", 998 | " \n", 999 | " \n", 1000 | " \n", 1001 | " \n", 1002 | " \n", 1003 | " \n", 1004 | " \n", 1005 | "
Place_ID X Datetarget
00OS9LVX X 2020-01-0232.214232
10OS9LVX X 2020-01-0331.719516
20OS9LVX X 2020-01-0426.271360
30OS9LVX X 2020-01-0535.206412
40OS9LVX X 2020-01-0640.994261
50OS9LVX X 2020-01-0744.029983
60OS9LVX X 2020-01-0824.994910
70OS9LVX X 2020-01-0931.687251
80OS9LVX X 2020-01-1021.737808
90OS9LVX X 2020-01-1134.294713
\n", 1006 | "
" 1007 | ], 1008 | "text/plain": [ 1009 | " Place_ID X Date target\n", 1010 | "0 0OS9LVX X 2020-01-02 32.214232\n", 1011 | "1 0OS9LVX X 2020-01-03 31.719516\n", 1012 | "2 0OS9LVX X 2020-01-04 26.271360\n", 1013 | "3 0OS9LVX X 2020-01-05 35.206412\n", 1014 | "4 0OS9LVX X 2020-01-06 40.994261\n", 1015 | "5 0OS9LVX X 2020-01-07 44.029983\n", 1016 | "6 0OS9LVX X 2020-01-08 24.994910\n", 1017 | "7 0OS9LVX X 2020-01-09 31.687251\n", 1018 | "8 0OS9LVX X 2020-01-10 21.737808\n", 1019 | "9 0OS9LVX X 2020-01-11 34.294713" 1020 | ] 1021 | }, 1022 | "execution_count": 14, 1023 | "metadata": {}, 1024 | "output_type": "execute_result" 1025 | } 1026 | ], 1027 | "source": [ 1028 | "SUB_FILE_NAME = 'preds_lgbm_v4.csv'\n", 1029 | "sub_df = pd.DataFrame()\n", 1030 | "sub_df[ID_COL] = test[ID_COL]\n", 1031 | "sub_df[TARGET_COL] = predictions_test\n", 1032 | "sub_df.to_csv(SUB_FILE_NAME, index=False)\n", 1033 | "sub_df.head(10)" 1034 | ] 1035 | }, 1036 | { 1037 | "cell_type": "code", 1038 | "execution_count": 15, 1039 | "metadata": {}, 1040 | "outputs": [ 1041 | { 1042 | "data": { 1043 | "text/plain": [ 1044 | "count 16136.000000\n", 1045 | "mean 58.149073\n", 1046 | "std 36.062610\n", 1047 | "min 0.000000\n", 1048 | "25% 31.245136\n", 1049 | "50% 48.706268\n", 1050 | "75% 75.947502\n", 1051 | "max 295.974090\n", 1052 | "Name: target, dtype: float64" 1053 | ] 1054 | }, 1055 | "execution_count": 15, 1056 | "metadata": {}, 1057 | "output_type": "execute_result" 1058 | } 1059 | ], 1060 | "source": [ 1061 | "sub_df[TARGET_COL].describe()" 1062 | ] 1063 | }, 1064 | { 1065 | "cell_type": "code", 1066 | "execution_count": null, 1067 | "metadata": {}, 1068 | "outputs": [], 1069 | "source": [] 1070 | } 1071 | ], 1072 | "metadata": { 1073 | "kernelspec": { 1074 | "display_name": "Python 3", 1075 | "language": "python", 1076 | "name": "python3" 1077 | }, 1078 | "language_info": { 1079 | "codemirror_mode": { 1080 | "name": "ipython", 1081 | "version": 3 1082 | }, 1083 | "file_extension": ".py", 1084 | "mimetype": "text/x-python", 1085 | "name": "python", 1086 | "nbconvert_exporter": "python", 1087 | "pygments_lexer": "ipython3", 1088 | "version": "3.6.6" 1089 | }, 1090 | "widgets": { 1091 | "application/vnd.jupyter.widget-state+json": { 1092 | "state": { 1093 | "030d7185573247abaa627f1a7e2115d4": { 1094 | "model_module": "@jupyter-widgets/controls", 1095 | "model_module_version": "1.5.0", 1096 | "model_name": "FloatProgressModel", 1097 | "state": { 1098 | "_dom_classes": [], 1099 | "_model_module": "@jupyter-widgets/controls", 1100 | "_model_module_version": "1.5.0", 1101 | "_model_name": "FloatProgressModel", 1102 | "_view_count": null, 1103 | "_view_module": "@jupyter-widgets/controls", 1104 | "_view_module_version": "1.5.0", 1105 | "_view_name": "ProgressView", 1106 | "bar_style": "success", 1107 | "description": "100%", 1108 | "description_tooltip": null, 1109 | "layout": "IPY_MODEL_191f40184aae453bb2a45ddbba2ffc19", 1110 | "max": 44.0, 1111 | "min": 0.0, 1112 | "orientation": "horizontal", 1113 | "style": "IPY_MODEL_2965529bdce54910bf5f40f20810abb1", 1114 | "value": 44.0 1115 | } 1116 | }, 1117 | "0e37b01edfcf4863b898652696d295ca": { 1118 | "model_module": "@jupyter-widgets/controls", 1119 | "model_module_version": "1.5.0", 1120 | "model_name": "FloatProgressModel", 1121 | "state": { 1122 | "_dom_classes": [], 1123 | "_model_module": "@jupyter-widgets/controls", 1124 | "_model_module_version": "1.5.0", 1125 | "_model_name": "FloatProgressModel", 1126 | "_view_count": null, 1127 | "_view_module": "@jupyter-widgets/controls", 1128 | "_view_module_version": "1.5.0", 1129 | "_view_name": "ProgressView", 1130 | "bar_style": "success", 1131 | "description": "100%", 1132 | "description_tooltip": null, 1133 | "layout": "IPY_MODEL_fd80052dcc3e407ea28c1b330428537c", 1134 | "max": 24.0, 1135 | "min": 0.0, 1136 | "orientation": "horizontal", 1137 | "style": "IPY_MODEL_1001fe6544a3468ebaeabd97dc22c023", 1138 | "value": 24.0 1139 | } 1140 | }, 1141 | "1001fe6544a3468ebaeabd97dc22c023": { 1142 | "model_module": "@jupyter-widgets/controls", 1143 | "model_module_version": "1.5.0", 1144 | "model_name": "ProgressStyleModel", 1145 | "state": { 1146 | "_model_module": "@jupyter-widgets/controls", 1147 | "_model_module_version": "1.5.0", 1148 | "_model_name": "ProgressStyleModel", 1149 | "_view_count": null, 1150 | "_view_module": "@jupyter-widgets/base", 1151 | "_view_module_version": "1.2.0", 1152 | "_view_name": "StyleView", 1153 | "bar_color": null, 1154 | "description_width": "initial" 1155 | } 1156 | }, 1157 | "191f40184aae453bb2a45ddbba2ffc19": { 1158 | "model_module": "@jupyter-widgets/base", 1159 | "model_module_version": "1.2.0", 1160 | "model_name": "LayoutModel", 1161 | "state": { 1162 | "_model_module": "@jupyter-widgets/base", 1163 | "_model_module_version": "1.2.0", 1164 | "_model_name": "LayoutModel", 1165 | "_view_count": null, 1166 | "_view_module": "@jupyter-widgets/base", 1167 | "_view_module_version": "1.2.0", 1168 | "_view_name": "LayoutView", 1169 | "align_content": null, 1170 | "align_items": null, 1171 | "align_self": null, 1172 | "border": null, 1173 | "bottom": null, 1174 | "display": null, 1175 | "flex": null, 1176 | "flex_flow": null, 1177 | "grid_area": null, 1178 | "grid_auto_columns": null, 1179 | "grid_auto_flow": null, 1180 | "grid_auto_rows": null, 1181 | "grid_column": null, 1182 | "grid_gap": null, 1183 | "grid_row": null, 1184 | "grid_template_areas": null, 1185 | "grid_template_columns": null, 1186 | "grid_template_rows": null, 1187 | "height": null, 1188 | "justify_content": null, 1189 | "justify_items": null, 1190 | "left": null, 1191 | "margin": null, 1192 | "max_height": null, 1193 | "max_width": null, 1194 | "min_height": null, 1195 | "min_width": null, 1196 | "object_fit": null, 1197 | "object_position": null, 1198 | "order": null, 1199 | "overflow": null, 1200 | "overflow_x": null, 1201 | "overflow_y": null, 1202 | "padding": null, 1203 | "right": null, 1204 | "top": null, 1205 | "visibility": null, 1206 | "width": null 1207 | } 1208 | }, 1209 | "2965529bdce54910bf5f40f20810abb1": { 1210 | "model_module": "@jupyter-widgets/controls", 1211 | "model_module_version": "1.5.0", 1212 | "model_name": "ProgressStyleModel", 1213 | "state": { 1214 | "_model_module": "@jupyter-widgets/controls", 1215 | "_model_module_version": "1.5.0", 1216 | "_model_name": "ProgressStyleModel", 1217 | "_view_count": null, 1218 | "_view_module": "@jupyter-widgets/base", 1219 | "_view_module_version": "1.2.0", 1220 | "_view_name": "StyleView", 1221 | "bar_color": null, 1222 | "description_width": "initial" 1223 | } 1224 | }, 1225 | "2dadc47f8f3e414bb12bd960f319d145": { 1226 | "model_module": "@jupyter-widgets/controls", 1227 | "model_module_version": "1.5.0", 1228 | "model_name": "HTMLModel", 1229 | "state": { 1230 | "_dom_classes": [], 1231 | "_model_module": "@jupyter-widgets/controls", 1232 | "_model_module_version": "1.5.0", 1233 | "_model_name": "HTMLModel", 1234 | "_view_count": null, 1235 | "_view_module": "@jupyter-widgets/controls", 1236 | "_view_module_version": "1.5.0", 1237 | "_view_name": "HTMLView", 1238 | "description": "", 1239 | "description_tooltip": null, 1240 | "layout": "IPY_MODEL_a6d5a9bf4f62451e909f1e4d2bced84c", 1241 | "placeholder": "​", 1242 | "style": "IPY_MODEL_32ad1df76afc4c00b17a9911b46fef69", 1243 | "value": " 44/44 [02:33<00:00, 3.50s/it]" 1244 | } 1245 | }, 1246 | "32ad1df76afc4c00b17a9911b46fef69": { 1247 | "model_module": "@jupyter-widgets/controls", 1248 | "model_module_version": "1.5.0", 1249 | "model_name": "DescriptionStyleModel", 1250 | "state": { 1251 | "_model_module": "@jupyter-widgets/controls", 1252 | "_model_module_version": "1.5.0", 1253 | "_model_name": "DescriptionStyleModel", 1254 | "_view_count": null, 1255 | "_view_module": "@jupyter-widgets/base", 1256 | "_view_module_version": "1.2.0", 1257 | "_view_name": "StyleView", 1258 | "description_width": "" 1259 | } 1260 | }, 1261 | "38372d2000b5486fa5b5a9cdc6bf0c0f": { 1262 | "model_module": "@jupyter-widgets/controls", 1263 | "model_module_version": "1.5.0", 1264 | "model_name": "ProgressStyleModel", 1265 | "state": { 1266 | "_model_module": "@jupyter-widgets/controls", 1267 | "_model_module_version": "1.5.0", 1268 | "_model_name": "ProgressStyleModel", 1269 | "_view_count": null, 1270 | "_view_module": "@jupyter-widgets/base", 1271 | "_view_module_version": "1.2.0", 1272 | "_view_name": "StyleView", 1273 | "bar_color": null, 1274 | "description_width": "initial" 1275 | } 1276 | }, 1277 | "39f0e497edbb4f8bbc1497bd49594e35": { 1278 | "model_module": "@jupyter-widgets/controls", 1279 | "model_module_version": "1.5.0", 1280 | "model_name": "DescriptionStyleModel", 1281 | "state": { 1282 | "_model_module": "@jupyter-widgets/controls", 1283 | "_model_module_version": "1.5.0", 1284 | "_model_name": "DescriptionStyleModel", 1285 | "_view_count": null, 1286 | "_view_module": "@jupyter-widgets/base", 1287 | "_view_module_version": "1.2.0", 1288 | "_view_name": "StyleView", 1289 | "description_width": "" 1290 | } 1291 | }, 1292 | "41d874433a334d6abda65442beba3550": { 1293 | "model_module": "@jupyter-widgets/controls", 1294 | "model_module_version": "1.5.0", 1295 | "model_name": "HBoxModel", 1296 | "state": { 1297 | "_dom_classes": [], 1298 | "_model_module": "@jupyter-widgets/controls", 1299 | "_model_module_version": "1.5.0", 1300 | "_model_name": "HBoxModel", 1301 | "_view_count": null, 1302 | "_view_module": "@jupyter-widgets/controls", 1303 | "_view_module_version": "1.5.0", 1304 | "_view_name": "HBoxView", 1305 | "box_style": "", 1306 | "children": [ 1307 | "IPY_MODEL_c117e29c19b14726ab0b174dd40629a9", 1308 | "IPY_MODEL_87a795c4b14746abbbfe17dba8c2666a" 1309 | ], 1310 | "layout": "IPY_MODEL_c7b1a45472324ba4a1e156e028454101" 1311 | } 1312 | }, 1313 | "6c61dfe88f2b45e2bc75d29094de32f7": { 1314 | "model_module": "@jupyter-widgets/base", 1315 | "model_module_version": "1.2.0", 1316 | "model_name": "LayoutModel", 1317 | "state": { 1318 | "_model_module": "@jupyter-widgets/base", 1319 | "_model_module_version": "1.2.0", 1320 | "_model_name": "LayoutModel", 1321 | "_view_count": null, 1322 | "_view_module": "@jupyter-widgets/base", 1323 | "_view_module_version": "1.2.0", 1324 | "_view_name": "LayoutView", 1325 | "align_content": null, 1326 | "align_items": null, 1327 | "align_self": null, 1328 | "border": null, 1329 | "bottom": null, 1330 | "display": null, 1331 | "flex": null, 1332 | "flex_flow": null, 1333 | "grid_area": null, 1334 | "grid_auto_columns": null, 1335 | "grid_auto_flow": null, 1336 | "grid_auto_rows": null, 1337 | "grid_column": null, 1338 | "grid_gap": null, 1339 | "grid_row": null, 1340 | "grid_template_areas": null, 1341 | "grid_template_columns": null, 1342 | "grid_template_rows": null, 1343 | "height": null, 1344 | "justify_content": null, 1345 | "justify_items": null, 1346 | "left": null, 1347 | "margin": null, 1348 | "max_height": null, 1349 | "max_width": null, 1350 | "min_height": null, 1351 | "min_width": null, 1352 | "object_fit": null, 1353 | "object_position": null, 1354 | "order": null, 1355 | "overflow": null, 1356 | "overflow_x": null, 1357 | "overflow_y": null, 1358 | "padding": null, 1359 | "right": null, 1360 | "top": null, 1361 | "visibility": null, 1362 | "width": null 1363 | } 1364 | }, 1365 | "716984afacf64c8096c82f790f6073c7": { 1366 | "model_module": "@jupyter-widgets/base", 1367 | "model_module_version": "1.2.0", 1368 | "model_name": "LayoutModel", 1369 | "state": { 1370 | "_model_module": "@jupyter-widgets/base", 1371 | "_model_module_version": "1.2.0", 1372 | "_model_name": "LayoutModel", 1373 | "_view_count": null, 1374 | "_view_module": "@jupyter-widgets/base", 1375 | "_view_module_version": "1.2.0", 1376 | "_view_name": "LayoutView", 1377 | "align_content": null, 1378 | "align_items": null, 1379 | "align_self": null, 1380 | "border": null, 1381 | "bottom": null, 1382 | "display": null, 1383 | "flex": null, 1384 | "flex_flow": null, 1385 | "grid_area": null, 1386 | "grid_auto_columns": null, 1387 | "grid_auto_flow": null, 1388 | "grid_auto_rows": null, 1389 | "grid_column": null, 1390 | "grid_gap": null, 1391 | "grid_row": null, 1392 | "grid_template_areas": null, 1393 | "grid_template_columns": null, 1394 | "grid_template_rows": null, 1395 | "height": null, 1396 | "justify_content": null, 1397 | "justify_items": null, 1398 | "left": null, 1399 | "margin": null, 1400 | "max_height": null, 1401 | "max_width": null, 1402 | "min_height": null, 1403 | "min_width": null, 1404 | "object_fit": null, 1405 | "object_position": null, 1406 | "order": null, 1407 | "overflow": null, 1408 | "overflow_x": null, 1409 | "overflow_y": null, 1410 | "padding": null, 1411 | "right": null, 1412 | "top": null, 1413 | "visibility": null, 1414 | "width": null 1415 | } 1416 | }, 1417 | "87a795c4b14746abbbfe17dba8c2666a": { 1418 | "model_module": "@jupyter-widgets/controls", 1419 | "model_module_version": "1.5.0", 1420 | "model_name": "HTMLModel", 1421 | "state": { 1422 | "_dom_classes": [], 1423 | "_model_module": "@jupyter-widgets/controls", 1424 | "_model_module_version": "1.5.0", 1425 | "_model_name": "HTMLModel", 1426 | "_view_count": null, 1427 | "_view_module": "@jupyter-widgets/controls", 1428 | "_view_module_version": "1.5.0", 1429 | "_view_name": "HTMLView", 1430 | "description": "", 1431 | "description_tooltip": null, 1432 | "layout": "IPY_MODEL_6c61dfe88f2b45e2bc75d29094de32f7", 1433 | "placeholder": "​", 1434 | "style": "IPY_MODEL_39f0e497edbb4f8bbc1497bd49594e35", 1435 | "value": " 21/21 [01:10<00:00, 3.33s/it]" 1436 | } 1437 | }, 1438 | "95ef2ac500564f7faeff03ffff594a14": { 1439 | "model_module": "@jupyter-widgets/base", 1440 | "model_module_version": "1.2.0", 1441 | "model_name": "LayoutModel", 1442 | "state": { 1443 | "_model_module": "@jupyter-widgets/base", 1444 | "_model_module_version": "1.2.0", 1445 | "_model_name": "LayoutModel", 1446 | "_view_count": null, 1447 | "_view_module": "@jupyter-widgets/base", 1448 | "_view_module_version": "1.2.0", 1449 | "_view_name": "LayoutView", 1450 | "align_content": null, 1451 | "align_items": null, 1452 | "align_self": null, 1453 | "border": null, 1454 | "bottom": null, 1455 | "display": null, 1456 | "flex": null, 1457 | "flex_flow": null, 1458 | "grid_area": null, 1459 | "grid_auto_columns": null, 1460 | "grid_auto_flow": null, 1461 | "grid_auto_rows": null, 1462 | "grid_column": null, 1463 | "grid_gap": null, 1464 | "grid_row": null, 1465 | "grid_template_areas": null, 1466 | "grid_template_columns": null, 1467 | "grid_template_rows": null, 1468 | "height": null, 1469 | "justify_content": null, 1470 | "justify_items": null, 1471 | "left": null, 1472 | "margin": null, 1473 | "max_height": null, 1474 | "max_width": null, 1475 | "min_height": null, 1476 | "min_width": null, 1477 | "object_fit": null, 1478 | "object_position": null, 1479 | "order": null, 1480 | "overflow": null, 1481 | "overflow_x": null, 1482 | "overflow_y": null, 1483 | "padding": null, 1484 | "right": null, 1485 | "top": null, 1486 | "visibility": null, 1487 | "width": null 1488 | } 1489 | }, 1490 | "a6234022dc9d4ea983e62bc863344757": { 1491 | "model_module": "@jupyter-widgets/controls", 1492 | "model_module_version": "1.5.0", 1493 | "model_name": "HBoxModel", 1494 | "state": { 1495 | "_dom_classes": [], 1496 | "_model_module": "@jupyter-widgets/controls", 1497 | "_model_module_version": "1.5.0", 1498 | "_model_name": "HBoxModel", 1499 | "_view_count": null, 1500 | "_view_module": "@jupyter-widgets/controls", 1501 | "_view_module_version": "1.5.0", 1502 | "_view_name": "HBoxView", 1503 | "box_style": "", 1504 | "children": [ 1505 | "IPY_MODEL_030d7185573247abaa627f1a7e2115d4", 1506 | "IPY_MODEL_2dadc47f8f3e414bb12bd960f319d145" 1507 | ], 1508 | "layout": "IPY_MODEL_716984afacf64c8096c82f790f6073c7" 1509 | } 1510 | }, 1511 | "a6d5a9bf4f62451e909f1e4d2bced84c": { 1512 | "model_module": "@jupyter-widgets/base", 1513 | "model_module_version": "1.2.0", 1514 | "model_name": "LayoutModel", 1515 | "state": { 1516 | "_model_module": "@jupyter-widgets/base", 1517 | "_model_module_version": "1.2.0", 1518 | "_model_name": "LayoutModel", 1519 | "_view_count": null, 1520 | "_view_module": "@jupyter-widgets/base", 1521 | "_view_module_version": "1.2.0", 1522 | "_view_name": "LayoutView", 1523 | "align_content": null, 1524 | "align_items": null, 1525 | "align_self": null, 1526 | "border": null, 1527 | "bottom": null, 1528 | "display": null, 1529 | "flex": null, 1530 | "flex_flow": null, 1531 | "grid_area": null, 1532 | "grid_auto_columns": null, 1533 | "grid_auto_flow": null, 1534 | "grid_auto_rows": null, 1535 | "grid_column": null, 1536 | "grid_gap": null, 1537 | "grid_row": null, 1538 | "grid_template_areas": null, 1539 | "grid_template_columns": null, 1540 | "grid_template_rows": null, 1541 | "height": null, 1542 | "justify_content": null, 1543 | "justify_items": null, 1544 | "left": null, 1545 | "margin": null, 1546 | "max_height": null, 1547 | "max_width": null, 1548 | "min_height": null, 1549 | "min_width": null, 1550 | "object_fit": null, 1551 | "object_position": null, 1552 | "order": null, 1553 | "overflow": null, 1554 | "overflow_x": null, 1555 | "overflow_y": null, 1556 | "padding": null, 1557 | "right": null, 1558 | "top": null, 1559 | "visibility": null, 1560 | "width": null 1561 | } 1562 | }, 1563 | "bd260a119b74443298c94e883b1daf5b": { 1564 | "model_module": "@jupyter-widgets/controls", 1565 | "model_module_version": "1.5.0", 1566 | "model_name": "DescriptionStyleModel", 1567 | "state": { 1568 | "_model_module": "@jupyter-widgets/controls", 1569 | "_model_module_version": "1.5.0", 1570 | "_model_name": "DescriptionStyleModel", 1571 | "_view_count": null, 1572 | "_view_module": "@jupyter-widgets/base", 1573 | "_view_module_version": "1.2.0", 1574 | "_view_name": "StyleView", 1575 | "description_width": "" 1576 | } 1577 | }, 1578 | "c117e29c19b14726ab0b174dd40629a9": { 1579 | "model_module": "@jupyter-widgets/controls", 1580 | "model_module_version": "1.5.0", 1581 | "model_name": "FloatProgressModel", 1582 | "state": { 1583 | "_dom_classes": [], 1584 | "_model_module": "@jupyter-widgets/controls", 1585 | "_model_module_version": "1.5.0", 1586 | "_model_name": "FloatProgressModel", 1587 | "_view_count": null, 1588 | "_view_module": "@jupyter-widgets/controls", 1589 | "_view_module_version": "1.5.0", 1590 | "_view_name": "ProgressView", 1591 | "bar_style": "success", 1592 | "description": "100%", 1593 | "description_tooltip": null, 1594 | "layout": "IPY_MODEL_e20facfd07f048b785e0a89eb2496eac", 1595 | "max": 21.0, 1596 | "min": 0.0, 1597 | "orientation": "horizontal", 1598 | "style": "IPY_MODEL_38372d2000b5486fa5b5a9cdc6bf0c0f", 1599 | "value": 21.0 1600 | } 1601 | }, 1602 | "c7b1a45472324ba4a1e156e028454101": { 1603 | "model_module": "@jupyter-widgets/base", 1604 | "model_module_version": "1.2.0", 1605 | "model_name": "LayoutModel", 1606 | "state": { 1607 | "_model_module": "@jupyter-widgets/base", 1608 | "_model_module_version": "1.2.0", 1609 | "_model_name": "LayoutModel", 1610 | "_view_count": null, 1611 | "_view_module": "@jupyter-widgets/base", 1612 | "_view_module_version": "1.2.0", 1613 | "_view_name": "LayoutView", 1614 | "align_content": null, 1615 | "align_items": null, 1616 | "align_self": null, 1617 | "border": null, 1618 | "bottom": null, 1619 | "display": null, 1620 | "flex": null, 1621 | "flex_flow": null, 1622 | "grid_area": null, 1623 | "grid_auto_columns": null, 1624 | "grid_auto_flow": null, 1625 | "grid_auto_rows": null, 1626 | "grid_column": null, 1627 | "grid_gap": null, 1628 | "grid_row": null, 1629 | "grid_template_areas": null, 1630 | "grid_template_columns": null, 1631 | "grid_template_rows": null, 1632 | "height": null, 1633 | "justify_content": null, 1634 | "justify_items": null, 1635 | "left": null, 1636 | "margin": null, 1637 | "max_height": null, 1638 | "max_width": null, 1639 | "min_height": null, 1640 | "min_width": null, 1641 | "object_fit": null, 1642 | "object_position": null, 1643 | "order": null, 1644 | "overflow": null, 1645 | "overflow_x": null, 1646 | "overflow_y": null, 1647 | "padding": null, 1648 | "right": null, 1649 | "top": null, 1650 | "visibility": null, 1651 | "width": null 1652 | } 1653 | }, 1654 | "cbea4e3d24a645eeb3733f9508cf69ef": { 1655 | "model_module": "@jupyter-widgets/controls", 1656 | "model_module_version": "1.5.0", 1657 | "model_name": "HBoxModel", 1658 | "state": { 1659 | "_dom_classes": [], 1660 | "_model_module": "@jupyter-widgets/controls", 1661 | "_model_module_version": "1.5.0", 1662 | "_model_name": "HBoxModel", 1663 | "_view_count": null, 1664 | "_view_module": "@jupyter-widgets/controls", 1665 | "_view_module_version": "1.5.0", 1666 | "_view_name": "HBoxView", 1667 | "box_style": "", 1668 | "children": [ 1669 | "IPY_MODEL_0e37b01edfcf4863b898652696d295ca", 1670 | "IPY_MODEL_cfc1024d941a48989e795452092d5590" 1671 | ], 1672 | "layout": "IPY_MODEL_fe05c561beb942cb9577b0d269229673" 1673 | } 1674 | }, 1675 | "cfc1024d941a48989e795452092d5590": { 1676 | "model_module": "@jupyter-widgets/controls", 1677 | "model_module_version": "1.5.0", 1678 | "model_name": "HTMLModel", 1679 | "state": { 1680 | "_dom_classes": [], 1681 | "_model_module": "@jupyter-widgets/controls", 1682 | "_model_module_version": "1.5.0", 1683 | "_model_name": "HTMLModel", 1684 | "_view_count": null, 1685 | "_view_module": "@jupyter-widgets/controls", 1686 | "_view_module_version": "1.5.0", 1687 | "_view_name": "HTMLView", 1688 | "description": "", 1689 | "description_tooltip": null, 1690 | "layout": "IPY_MODEL_95ef2ac500564f7faeff03ffff594a14", 1691 | "placeholder": "​", 1692 | "style": "IPY_MODEL_bd260a119b74443298c94e883b1daf5b", 1693 | "value": " 24/24 [00:04<00:00, 5.24it/s]" 1694 | } 1695 | }, 1696 | "e20facfd07f048b785e0a89eb2496eac": { 1697 | "model_module": "@jupyter-widgets/base", 1698 | "model_module_version": "1.2.0", 1699 | "model_name": "LayoutModel", 1700 | "state": { 1701 | "_model_module": "@jupyter-widgets/base", 1702 | "_model_module_version": "1.2.0", 1703 | "_model_name": "LayoutModel", 1704 | "_view_count": null, 1705 | "_view_module": "@jupyter-widgets/base", 1706 | "_view_module_version": "1.2.0", 1707 | "_view_name": "LayoutView", 1708 | "align_content": null, 1709 | "align_items": null, 1710 | "align_self": null, 1711 | "border": null, 1712 | "bottom": null, 1713 | "display": null, 1714 | "flex": null, 1715 | "flex_flow": null, 1716 | "grid_area": null, 1717 | "grid_auto_columns": null, 1718 | "grid_auto_flow": null, 1719 | "grid_auto_rows": null, 1720 | "grid_column": null, 1721 | "grid_gap": null, 1722 | "grid_row": null, 1723 | "grid_template_areas": null, 1724 | "grid_template_columns": null, 1725 | "grid_template_rows": null, 1726 | "height": null, 1727 | "justify_content": null, 1728 | "justify_items": null, 1729 | "left": null, 1730 | "margin": null, 1731 | "max_height": null, 1732 | "max_width": null, 1733 | "min_height": null, 1734 | "min_width": null, 1735 | "object_fit": null, 1736 | "object_position": null, 1737 | "order": null, 1738 | "overflow": null, 1739 | "overflow_x": null, 1740 | "overflow_y": null, 1741 | "padding": null, 1742 | "right": null, 1743 | "top": null, 1744 | "visibility": null, 1745 | "width": null 1746 | } 1747 | }, 1748 | "fd80052dcc3e407ea28c1b330428537c": { 1749 | "model_module": "@jupyter-widgets/base", 1750 | "model_module_version": "1.2.0", 1751 | "model_name": "LayoutModel", 1752 | "state": { 1753 | "_model_module": "@jupyter-widgets/base", 1754 | "_model_module_version": "1.2.0", 1755 | "_model_name": "LayoutModel", 1756 | "_view_count": null, 1757 | "_view_module": "@jupyter-widgets/base", 1758 | "_view_module_version": "1.2.0", 1759 | "_view_name": "LayoutView", 1760 | "align_content": null, 1761 | "align_items": null, 1762 | "align_self": null, 1763 | "border": null, 1764 | "bottom": null, 1765 | "display": null, 1766 | "flex": null, 1767 | "flex_flow": null, 1768 | "grid_area": null, 1769 | "grid_auto_columns": null, 1770 | "grid_auto_flow": null, 1771 | "grid_auto_rows": null, 1772 | "grid_column": null, 1773 | "grid_gap": null, 1774 | "grid_row": null, 1775 | "grid_template_areas": null, 1776 | "grid_template_columns": null, 1777 | "grid_template_rows": null, 1778 | "height": null, 1779 | "justify_content": null, 1780 | "justify_items": null, 1781 | "left": null, 1782 | "margin": null, 1783 | "max_height": null, 1784 | "max_width": null, 1785 | "min_height": null, 1786 | "min_width": null, 1787 | "object_fit": null, 1788 | "object_position": null, 1789 | "order": null, 1790 | "overflow": null, 1791 | "overflow_x": null, 1792 | "overflow_y": null, 1793 | "padding": null, 1794 | "right": null, 1795 | "top": null, 1796 | "visibility": null, 1797 | "width": null 1798 | } 1799 | }, 1800 | "fe05c561beb942cb9577b0d269229673": { 1801 | "model_module": "@jupyter-widgets/base", 1802 | "model_module_version": "1.2.0", 1803 | "model_name": "LayoutModel", 1804 | "state": { 1805 | "_model_module": "@jupyter-widgets/base", 1806 | "_model_module_version": "1.2.0", 1807 | "_model_name": "LayoutModel", 1808 | "_view_count": null, 1809 | "_view_module": "@jupyter-widgets/base", 1810 | "_view_module_version": "1.2.0", 1811 | "_view_name": "LayoutView", 1812 | "align_content": null, 1813 | "align_items": null, 1814 | "align_self": null, 1815 | "border": null, 1816 | "bottom": null, 1817 | "display": null, 1818 | "flex": null, 1819 | "flex_flow": null, 1820 | "grid_area": null, 1821 | "grid_auto_columns": null, 1822 | "grid_auto_flow": null, 1823 | "grid_auto_rows": null, 1824 | "grid_column": null, 1825 | "grid_gap": null, 1826 | "grid_row": null, 1827 | "grid_template_areas": null, 1828 | "grid_template_columns": null, 1829 | "grid_template_rows": null, 1830 | "height": null, 1831 | "justify_content": null, 1832 | "justify_items": null, 1833 | "left": null, 1834 | "margin": null, 1835 | "max_height": null, 1836 | "max_width": null, 1837 | "min_height": null, 1838 | "min_width": null, 1839 | "object_fit": null, 1840 | "object_position": null, 1841 | "order": null, 1842 | "overflow": null, 1843 | "overflow_x": null, 1844 | "overflow_y": null, 1845 | "padding": null, 1846 | "right": null, 1847 | "top": null, 1848 | "visibility": null, 1849 | "width": null 1850 | } 1851 | } 1852 | }, 1853 | "version_major": 2, 1854 | "version_minor": 0 1855 | } 1856 | } 1857 | }, 1858 | "nbformat": 4, 1859 | "nbformat_minor": 4 1860 | } 1861 | --------------------------------------------------------------------------------