├── images ├── ds_masters_lb.png ├── current_status.png ├── film_rating_prediction_lb.png └── patient_survival_prediction_lb.png ├── README.md └── code └── lightautoml-baseline-film-ratings.ipynb /images/ds_masters_lb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sb-ai-lab/automl-in-practice/main/images/ds_masters_lb.png -------------------------------------------------------------------------------- /images/current_status.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sb-ai-lab/automl-in-practice/main/images/current_status.png -------------------------------------------------------------------------------- /images/film_rating_prediction_lb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sb-ai-lab/automl-in-practice/main/images/film_rating_prediction_lb.png -------------------------------------------------------------------------------- /images/patient_survival_prediction_lb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sb-ai-lab/automl-in-practice/main/images/patient_survival_prediction_lb.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AutoML на практике: сделать AI за 60 секунд 2 | 3 | ### QR код на этот репозиторий: 4 |

5 | 6 |

7 | 8 | ### Спикер - Рыжков Александр 9 |

10 | 11 |

12 | 13 | - Окончил МГУ им. М.В.Ломоносова в 2015 году 14 | - 3х Kaggle GrandMaster – в такой комбинации звание есть только у **5 человек в мире** 15 | - Руководитель команды LightAutoML 16 | 17 | E-mail: alexmryzhkov@gmail.com 18 | 19 | ### Описание репозитория 20 | 21 | В этом репозитории собраны примеры **быстрого решения Kaggle соревнований на основе библиотеки LightAutoML**: 22 | 23 | - **Выживет ли пациент?** - [соревнование](https://www.kaggle.com/competitions/patient-survival-prediction), [решение](./code/lightautoml-patient-survival.ipynb), метрика ROC-AUC 24 |

25 | 26 |

27 | 28 | - **Вернут ли кредит?** - [соревнование](https://www.kaggle.com/competitions/ds-masters-math-retake/overview), [решение](./code/lightautoml-dsmasters.ipynb), метрика Accuracy 29 |

30 | 31 |

32 | 33 | - **Предсказание рейтинга фильма** - [соревнование](https://www.kaggle.com/competitions/tadmo-faru-pm/overview), [решение](./code/lightautoml-baseline-film-ratings.ipynb), метрика MAE 34 |

35 | 36 |

37 | 38 | ### Дополнительные материалы: 39 | 1) [LightAutoML Github](https://github.com/sb-ai-lab/LightAutoML) 40 | 2) [Телеграм канал с новостями LightAutoML](https://t.me/lightautoml) 41 | 3) [Чат в телеграм с практическими вопросами по LightAutoML](https://t.me/joinchat/sp8P7sdAqaU0YmRi) 42 | 4) [Обучающий курс по LightAutoML](https://developers.sber.ru/help/lightautoml) 43 | 5) [Документация](https://lightautoml.readthedocs.io/en/latest/) 44 | 6) [PyPI](https://pypi.org/project/lightautoml) 45 | 7) [Официальный сайт LightAutoML](https://developers.sber.ru/portal/products/lightautoml) 46 | 47 | ------------ 48 | # AutoML in practice: how to make AI in 60 seconds 49 | 50 | ### Speaker - Ryzhkov Alexander 51 |

52 | 53 |

54 | 55 | - Graduated from Lomonosov Moscow State University in 2015 56 | - 3х Kaggle GrandMaster – only 5 people in the world have such combination 57 | - Head of LightAutoML group 58 | 59 | E-mail: alexmryzhkov@gmail.com 60 | 61 | ### Repository contents 62 | 63 | This repository contains **fast LightAutoML solutions for Kaggle competitions** : 64 | 65 | - **Patient survival** - [competition](https://www.kaggle.com/competitions/patient-survival-prediction), [solution](./code/lightautoml-patient-survival.ipynb), ROC-AUC metric 66 |

67 | 68 |

69 | 70 | - **Credit default** - [competition](https://www.kaggle.com/competitions/ds-masters-math-retake/overview), [solution](./code/lightautoml-dsmasters.ipynb), Accuracy metric 71 |

72 | 73 |

74 | 75 | - **Film rating prediction** - [competition](https://www.kaggle.com/competitions/tadmo-faru-pm/overview), [solution](./code/lightautoml-baseline-film-ratings.ipynb), MAE metric 76 |

77 | 78 |

79 | 80 | ### Additional materials: 81 | 1) [LightAutoML Github](https://github.com/sb-ai-lab/LightAutoML) 82 | 2) [Telegram channel with LightAutoML news](https://t.me/lightautoml) 83 | 3) [Telegram chat with LightAutoML practical questions](https://t.me/joinchat/sp8P7sdAqaU0YmRi) 84 | 4) [LightAutoML 101 course](https://developers.sber.ru/help/lightautoml) 85 | 5) [Documentation](https://lightautoml.readthedocs.io/en/latest/) 86 | 6) [PyPI](https://pypi.org/project/lightautoml) 87 | 7) [LightAutoML official site](https://developers.sber.ru/portal/products/lightautoml) -------------------------------------------------------------------------------- /code/lightautoml-baseline-film-ratings.ipynb: -------------------------------------------------------------------------------- 1 | {"cells":[{"cell_type":"markdown","metadata":{"papermill":{"duration":0.055031,"end_time":"2022-05-10T22:31:33.681191","exception":false,"start_time":"2022-05-10T22:31:33.62616","status":"completed"},"tags":[]},"source":["\"LightAutoML"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.053669,"end_time":"2022-05-10T22:31:33.789035","exception":false,"start_time":"2022-05-10T22:31:33.735366","status":"completed"},"tags":[]},"source":["# LightAutoML baseline\n","\n","Official LightAutoML github repository is [here](https://github.com/AILab-MLTools/LightAutoML). "]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.055266,"end_time":"2022-05-10T22:31:34.021507","exception":false,"start_time":"2022-05-10T22:31:33.966241","status":"completed"},"tags":[]},"source":["## This notebook is the updated copy of our [Tutorial_1 from the GIT repository](https://github.com/AILab-MLTools/LightAutoML/blob/master/examples/tutorials/Tutorial_1_basics.ipynb). Please check our [tutorials folder](https://github.com/AILab-MLTools/LightAutoML/blob/master/examples/tutorials) if you are interested in other examples of LightAutoML functionality."]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.055424,"end_time":"2022-05-10T22:31:34.133584","exception":false,"start_time":"2022-05-10T22:31:34.07816","status":"completed"},"tags":[]},"source":["## 0. Prerequisites"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.053949,"end_time":"2022-05-10T22:31:34.241795","exception":false,"start_time":"2022-05-10T22:31:34.187846","status":"completed"},"tags":[]},"source":["### 0.0. install LightAutoML"]},{"cell_type":"code","execution_count":3,"metadata":{"_kg_hide-output":true,"execution":{"iopub.execute_input":"2022-06-02T11:28:22.879478Z","iopub.status.busy":"2022-06-02T11:28:22.879038Z","iopub.status.idle":"2022-06-02T11:31:01.697851Z","shell.execute_reply":"2022-06-02T11:31:01.695562Z","shell.execute_reply.started":"2022-06-02T11:28:22.879452Z"},"papermill":{"duration":132.884228,"end_time":"2022-05-10T22:33:47.180161","exception":false,"start_time":"2022-05-10T22:31:34.295933","status":"completed"},"scrolled":true,"tags":[],"trusted":true},"outputs":[],"source":["%%capture\n","!pip install -U lightautoml"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.054045,"end_time":"2022-05-10T22:33:47.293341","exception":false,"start_time":"2022-05-10T22:33:47.239296","status":"completed"},"tags":[]},"source":["### 0.1. Import libraries\n","\n","Here we will import the libraries we use in this kernel:\n","- Standard python libraries for timing, working with OS etc.\n","- Essential python DS libraries like numpy, pandas, scikit-learn and torch (the last we will use in the next cell)\n","- LightAutoML modules: `TabularAutoML` preset for AutoML model creation and Task class to setup what kind of ML problem we solve (binary/multiclass classification or regression)"]},{"cell_type":"code","execution_count":4,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:32:32.808554Z","iopub.status.busy":"2022-06-02T11:32:32.808156Z","iopub.status.idle":"2022-06-02T11:32:35.963833Z","shell.execute_reply":"2022-06-02T11:32:35.962645Z","shell.execute_reply.started":"2022-06-02T11:32:32.808515Z"},"papermill":{"duration":8.870692,"end_time":"2022-05-10T22:33:56.218407","exception":false,"start_time":"2022-05-10T22:33:47.347715","status":"completed"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["\n"],"text/plain":[""]},"metadata":{},"output_type":"display_data"}],"source":["# Standard python libraries\n","import os\n","import time\n","\n","# Essential DS libraries\n","import numpy as np\n","import pandas as pd\n","from sklearn.metrics import mean_absolute_error\n","import torch\n","\n","# LightAutoML presets, task and report generation\n","from lightautoml.automl.presets.tabular_presets import TabularAutoML\n","from lightautoml.tasks import Task"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.055772,"end_time":"2022-05-10T22:33:56.330791","exception":false,"start_time":"2022-05-10T22:33:56.275019","status":"completed"},"tags":[]},"source":["### 0.2. Constants\n","\n","Here we setup the constants to use in the kernel:\n","- `N_THREADS` - number of vCPUs for LightAutoML model creation\n","- `N_FOLDS` - number of folds in LightAutoML inner CV\n","- `RANDOM_STATE` - random seed for better reproducibility\n","- `TEST_SIZE` - houldout data part size \n","- `TIMEOUT` - limit in seconds for model to train\n","- `TARGET_NAME` - target column name in dataset"]},{"cell_type":"code","execution_count":13,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:38:41.811666Z","iopub.status.busy":"2022-06-02T11:38:41.811224Z","iopub.status.idle":"2022-06-02T11:38:41.818020Z","shell.execute_reply":"2022-06-02T11:38:41.816840Z","shell.execute_reply.started":"2022-06-02T11:38:41.811633Z"},"papermill":{"duration":0.063297,"end_time":"2022-05-10T22:33:56.449182","exception":false,"start_time":"2022-05-10T22:33:56.385885","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["N_THREADS = 4\n","N_FOLDS = 5\n","RANDOM_STATE = 42\n","TIMEOUT = 8 * 3600 # equal to 8 hours\n","TARGET_NAME = 'Средний рейтинг'"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.054426,"end_time":"2022-05-10T22:33:56.558664","exception":false,"start_time":"2022-05-10T22:33:56.504238","status":"completed"},"tags":[]},"source":["### 0.3. Imported models setup\n","\n","For better reproducibility fix numpy random seed with max number of threads for Torch (which usually try to use all the threads on server):"]},{"cell_type":"code","execution_count":6,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:32:44.678728Z","iopub.status.busy":"2022-06-02T11:32:44.677554Z","iopub.status.idle":"2022-06-02T11:32:44.718031Z","shell.execute_reply":"2022-06-02T11:32:44.717293Z","shell.execute_reply.started":"2022-06-02T11:32:44.678667Z"},"papermill":{"duration":0.102128,"end_time":"2022-05-10T22:33:56.715858","exception":false,"start_time":"2022-05-10T22:33:56.61373","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["np.random.seed(RANDOM_STATE)\n","torch.set_num_threads(N_THREADS)"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.055184,"end_time":"2022-05-10T22:33:56.833656","exception":false,"start_time":"2022-05-10T22:33:56.778472","status":"completed"},"tags":[]},"source":["### 0.4. Data loading\n","Let's check the data we have:"]},{"cell_type":"code","execution_count":7,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:32:57.493283Z","iopub.status.busy":"2022-06-02T11:32:57.492669Z","iopub.status.idle":"2022-06-02T11:32:57.497692Z","shell.execute_reply":"2022-06-02T11:32:57.496560Z","shell.execute_reply.started":"2022-06-02T11:32:57.493249Z"},"papermill":{"duration":0.061516,"end_time":"2022-05-10T22:33:56.9501","exception":false,"start_time":"2022-05-10T22:33:56.888584","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["INPUT_DIR = '../input/tadmo-faru-pm/'"]},{"cell_type":"code","execution_count":8,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:33:03.921975Z","iopub.status.busy":"2022-06-02T11:33:03.921147Z","iopub.status.idle":"2022-06-02T11:33:04.482613Z","shell.execute_reply":"2022-06-02T11:33:04.481703Z","shell.execute_reply.started":"2022-06-02T11:33:03.921910Z"},"papermill":{"duration":9.16211,"end_time":"2022-05-10T22:34:06.168324","exception":false,"start_time":"2022-05-10T22:33:57.006214","status":"completed"},"scrolled":true,"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["(52304, 13)\n"]},{"data":{"text/html":["
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
IDНазваниеЖанрыЯзыкОписаниеПопулярностьПроизводителиДата релизаБюджетПрибыльДлительностьСредний рейтингКол-во отзывов
0471925I Scream When I Knew What You Did in Friday th...Comedy-HorrorptSeven years after the massacre of the first fi...9.218Necrófilos Produções Artísticas2011-10-200.00.082.05.54.0
1416371La notte non fa più pauraDramaitA group of workers try overcoming their differ...0.600NaN2016-05-030.00.065.05.83.0
2669656MOTOAKI FURUKAWA with VOYAGER LIVE 2008 TOKYO ...MusicjaRecorded live in Tokyo in September 2008 and K...0.600Studio AS2009-08-250.00.070.00.00.0
3145218Brorsan Såsett - En kärlekshistoriaNaNsvNaN0.600NaNNaN0.00.00.07.01.0
4746669Transform!DocumentaryjaNaN0.600NaN2021-06-190.00.093.00.00.0
\n","
"],"text/plain":[" ID Название Жанры \\\n","0 471925 I Scream When I Knew What You Did in Friday th... Comedy-Horror \n","1 416371 La notte non fa più paura Drama \n","2 669656 MOTOAKI FURUKAWA with VOYAGER LIVE 2008 TOKYO ... Music \n","3 145218 Brorsan Såsett - En kärlekshistoria NaN \n","4 746669 Transform! Documentary \n","\n"," Язык Описание Популярность \\\n","0 pt Seven years after the massacre of the first fi... 9.218 \n","1 it A group of workers try overcoming their differ... 0.600 \n","2 ja Recorded live in Tokyo in September 2008 and K... 0.600 \n","3 sv NaN 0.600 \n","4 ja NaN 0.600 \n","\n"," Производители Дата релиза Бюджет Прибыль Длительность \\\n","0 Necrófilos Produções Artísticas 2011-10-20 0.0 0.0 82.0 \n","1 NaN 2016-05-03 0.0 0.0 65.0 \n","2 Studio AS 2009-08-25 0.0 0.0 70.0 \n","3 NaN NaN 0.0 0.0 0.0 \n","4 NaN 2021-06-19 0.0 0.0 93.0 \n","\n"," Средний рейтинг Кол-во отзывов \n","0 5.5 4.0 \n","1 5.8 3.0 \n","2 0.0 0.0 \n","3 7.0 1.0 \n","4 0.0 0.0 "]},"execution_count":8,"metadata":{},"output_type":"execute_result"}],"source":["train_data = pd.read_csv(INPUT_DIR + 'train.csv')\n","print(train_data.shape)\n","train_data.head()"]},{"cell_type":"code","execution_count":9,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:33:46.011406Z","iopub.status.busy":"2022-06-02T11:33:46.010963Z","iopub.status.idle":"2022-06-02T11:33:46.197648Z","shell.execute_reply":"2022-06-02T11:33:46.196448Z","shell.execute_reply.started":"2022-06-02T11:33:46.011366Z"},"papermill":{"duration":7.096881,"end_time":"2022-05-10T22:34:13.322112","exception":false,"start_time":"2022-05-10T22:34:06.225231","status":"completed"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["(13076, 12)\n"]},{"data":{"text/html":["
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
IDНазваниеЖанрыЯзыкОписаниеПопулярностьПроизводителиДата релизаБюджетПрибыльДлительностьКол-во отзывов
0881783MiaDramalvMia an obedient daughter of a piano teacher ch...0.600Latvijas Kultūras akadēmija-Neonorma2022-02-280.00.029.00.0
1507769GoatsAnimationruThe group of travelling goats is trying to get...0.600Soyuzmultfilm2016-02-260.00.02.01.0
2160064Nature Unleashed: EarthquakeAction-Adventure-ThrillerfrAn earthquake destroys a Russian Nuclear Power...3.659Lithuanian Film Studio-Nu Image-GFT Entertainm...2005-01-112600000.00.092.020.0
3802763Zhaba.byFantasyruA young man thrown into prison \"for no reason\"...1.125NaN2020-11-250.00.017.00.0
4339029A Sweet LifeNaNjaA girl goes out destroying things0.600NaN2010-01-010.00.025.00.0
\n","
"],"text/plain":[" ID Название Жанры Язык \\\n","0 881783 Mia Drama lv \n","1 507769 Goats Animation ru \n","2 160064 Nature Unleashed: Earthquake Action-Adventure-Thriller fr \n","3 802763 Zhaba.by Fantasy ru \n","4 339029 A Sweet Life NaN ja \n","\n"," Описание Популярность \\\n","0 Mia an obedient daughter of a piano teacher ch... 0.600 \n","1 The group of travelling goats is trying to get... 0.600 \n","2 An earthquake destroys a Russian Nuclear Power... 3.659 \n","3 A young man thrown into prison \"for no reason\"... 1.125 \n","4 A girl goes out destroying things 0.600 \n","\n"," Производители Дата релиза Бюджет \\\n","0 Latvijas Kultūras akadēmija-Neonorma 2022-02-28 0.0 \n","1 Soyuzmultfilm 2016-02-26 0.0 \n","2 Lithuanian Film Studio-Nu Image-GFT Entertainm... 2005-01-11 2600000.0 \n","3 NaN 2020-11-25 0.0 \n","4 NaN 2010-01-01 0.0 \n","\n"," Прибыль Длительность Кол-во отзывов \n","0 0.0 29.0 0.0 \n","1 0.0 2.0 1.0 \n","2 0.0 92.0 20.0 \n","3 0.0 17.0 0.0 \n","4 0.0 25.0 0.0 "]},"execution_count":9,"metadata":{},"output_type":"execute_result"}],"source":["test_data = pd.read_csv(INPUT_DIR + 'test.csv')\n","print(test_data.shape)\n","test_data.head()"]},{"cell_type":"code","execution_count":10,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:35:30.670710Z","iopub.status.busy":"2022-06-02T11:35:30.669681Z","iopub.status.idle":"2022-06-02T11:35:30.699007Z","shell.execute_reply":"2022-06-02T11:35:30.698331Z","shell.execute_reply.started":"2022-06-02T11:35:30.670663Z"},"papermill":{"duration":0.293707,"end_time":"2022-05-10T22:34:13.674017","exception":false,"start_time":"2022-05-10T22:34:13.38031","status":"completed"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["(13076, 2)\n"]},{"data":{"text/html":["
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
IDСредний рейтинг
08817830.0
15077690.0
21600640.0
38027630.0
43390290.0
\n","
"],"text/plain":[" ID Средний рейтинг\n","0 881783 0.0\n","1 507769 0.0\n","2 160064 0.0\n","3 802763 0.0\n","4 339029 0.0"]},"execution_count":10,"metadata":{},"output_type":"execute_result"}],"source":["submission = pd.read_csv(INPUT_DIR + 'sample.csv')\n","print(submission.shape)\n","submission.head()"]},{"cell_type":"code","execution_count":11,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:36:22.319421Z","iopub.status.busy":"2022-06-02T11:36:22.319078Z","iopub.status.idle":"2022-06-02T11:36:22.331716Z","shell.execute_reply":"2022-06-02T11:36:22.330416Z","shell.execute_reply.started":"2022-06-02T11:36:22.319395Z"},"trusted":true},"outputs":[{"data":{"text/plain":["1.0"]},"execution_count":11,"metadata":{},"output_type":"execute_result"}],"source":["np.mean(test_data['ID'].values == submission['ID'].values)"]},{"cell_type":"markdown","metadata":{},"source":["### 0.5. Feature engineering\n","Let's convert genre feature into columns set:"]},{"cell_type":"code","execution_count":19,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:48:05.955980Z","iopub.status.busy":"2022-06-02T11:48:05.955632Z","iopub.status.idle":"2022-06-02T11:48:06.008643Z","shell.execute_reply":"2022-06-02T11:48:06.007912Z","shell.execute_reply.started":"2022-06-02T11:48:05.955954Z"},"trusted":true},"outputs":[{"data":{"text/plain":["nan 21169\n","Drama 15073\n","Documentary 10025\n","Comedy 9497\n","Animation 3813\n","Romance 3551\n","Music 3121\n","Horror 3094\n","Thriller 3074\n","Action 2995\n","Crime 2261\n","Family 1975\n","Adventure 1622\n","TV Movie 1492\n","Fantasy 1484\n","Science Fiction 1396\n","Mystery 1309\n","History 1118\n","War 820\n","Western 759\n","dtype: int64"]},"execution_count":19,"metadata":{},"output_type":"execute_result"}],"source":["genres = list(train_data['Жанры'].values) + list(test_data['Жанры'].values)\n","all_genres = []\n","for g in genres:\n"," all_genres += str(g).split('-')\n","unique_genres = pd.Series(all_genres).value_counts()\n","unique_genres"]},{"cell_type":"code","execution_count":22,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:48:56.630833Z","iopub.status.busy":"2022-06-02T11:48:56.630459Z","iopub.status.idle":"2022-06-02T11:48:57.975745Z","shell.execute_reply":"2022-06-02T11:48:57.974836Z","shell.execute_reply.started":"2022-06-02T11:48:56.630807Z"},"trusted":true},"outputs":[],"source":["splitted_genres_train = train_data['Жанры'].astype(str).str.split('-')\n","splitted_genres_test = test_data['Жанры'].astype(str).str.split('-')\n","for g in unique_genres.index.values:\n"," train_data['genre_'+g] = splitted_genres_train.map(lambda x: g in x).astype(int)\n"," test_data['genre_'+g] = splitted_genres_test.map(lambda x: g in x).astype(int)"]},{"cell_type":"code","execution_count":23,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:49:15.491466Z","iopub.status.busy":"2022-06-02T11:49:15.491139Z","iopub.status.idle":"2022-06-02T11:49:15.517942Z","shell.execute_reply":"2022-06-02T11:49:15.516653Z","shell.execute_reply.started":"2022-06-02T11:49:15.491440Z"},"trusted":true},"outputs":[{"data":{"text/html":["
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
IDНазваниеЖанрыЯзыкОписаниеПопулярностьПроизводителиДата релизаБюджетПрибыльДлительностьСредний рейтингКол-во отзывовgenre_nangenre_Dramagenre_Documentarygenre_Comedygenre_Animationgenre_Romancegenre_Musicgenre_Horrorgenre_Thrillergenre_Actiongenre_Crimegenre_Familygenre_Adventuregenre_TV Moviegenre_Fantasygenre_Science Fictiongenre_Mysterygenre_Historygenre_Wargenre_Western
0471925I Scream When I Knew What You Did in Friday th...Comedy-HorrorptSeven years after the massacre of the first fi...9.218Necrófilos Produções Artísticas2011-10-200.00.082.05.54.000010001000000000000
1416371La notte non fa più pauraDramaitA group of workers try overcoming their differ...0.600NaN2016-05-030.00.065.05.83.001000000000000000000
2669656MOTOAKI FURUKAWA with VOYAGER LIVE 2008 TOKYO ...MusicjaRecorded live in Tokyo in September 2008 and K...0.600Studio AS2009-08-250.00.070.00.00.000000010000000000000
3145218Brorsan Såsett - En kärlekshistoriaNaNsvNaN0.600NaNNaN0.00.00.07.01.010000000000000000000
4746669Transform!DocumentaryjaNaN0.600NaN2021-06-190.00.093.00.00.000100000000000000000
\n","
"],"text/plain":[" ID Название Жанры \\\n","0 471925 I Scream When I Knew What You Did in Friday th... Comedy-Horror \n","1 416371 La notte non fa più paura Drama \n","2 669656 MOTOAKI FURUKAWA with VOYAGER LIVE 2008 TOKYO ... Music \n","3 145218 Brorsan Såsett - En kärlekshistoria NaN \n","4 746669 Transform! Documentary \n","\n"," Язык Описание Популярность \\\n","0 pt Seven years after the massacre of the first fi... 9.218 \n","1 it A group of workers try overcoming their differ... 0.600 \n","2 ja Recorded live in Tokyo in September 2008 and K... 0.600 \n","3 sv NaN 0.600 \n","4 ja NaN 0.600 \n","\n"," Производители Дата релиза Бюджет Прибыль Длительность \\\n","0 Necrófilos Produções Artísticas 2011-10-20 0.0 0.0 82.0 \n","1 NaN 2016-05-03 0.0 0.0 65.0 \n","2 Studio AS 2009-08-25 0.0 0.0 70.0 \n","3 NaN NaN 0.0 0.0 0.0 \n","4 NaN 2021-06-19 0.0 0.0 93.0 \n","\n"," Средний рейтинг Кол-во отзывов genre_nan genre_Drama genre_Documentary \\\n","0 5.5 4.0 0 0 0 \n","1 5.8 3.0 0 1 0 \n","2 0.0 0.0 0 0 0 \n","3 7.0 1.0 1 0 0 \n","4 0.0 0.0 0 0 1 \n","\n"," genre_Comedy genre_Animation genre_Romance genre_Music genre_Horror \\\n","0 1 0 0 0 1 \n","1 0 0 0 0 0 \n","2 0 0 0 1 0 \n","3 0 0 0 0 0 \n","4 0 0 0 0 0 \n","\n"," genre_Thriller genre_Action genre_Crime genre_Family genre_Adventure \\\n","0 0 0 0 0 0 \n","1 0 0 0 0 0 \n","2 0 0 0 0 0 \n","3 0 0 0 0 0 \n","4 0 0 0 0 0 \n","\n"," genre_TV Movie genre_Fantasy genre_Science Fiction genre_Mystery \\\n","0 0 0 0 0 \n","1 0 0 0 0 \n","2 0 0 0 0 \n","3 0 0 0 0 \n","4 0 0 0 0 \n","\n"," genre_History genre_War genre_Western \n","0 0 0 0 \n","1 0 0 0 \n","2 0 0 0 \n","3 0 0 0 \n","4 0 0 0 "]},"execution_count":23,"metadata":{},"output_type":"execute_result"}],"source":["pd.set_option('display.max_columns', 100)\n","train_data.head()"]},{"cell_type":"code","execution_count":24,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:53:50.383153Z","iopub.status.busy":"2022-06-02T11:53:50.382101Z","iopub.status.idle":"2022-06-02T11:53:50.393041Z","shell.execute_reply":"2022-06-02T11:53:50.391772Z","shell.execute_reply.started":"2022-06-02T11:53:50.383108Z"},"trusted":true},"outputs":[{"data":{"text/plain":["Index(['ID', 'Название', 'Жанры', 'Язык', 'Описание', 'Популярность',\n"," 'Производители', 'Дата релиза', 'Бюджет', 'Прибыль', 'Длительность',\n"," 'Средний рейтинг', 'Кол-во отзывов', 'genre_nan', 'genre_Drama',\n"," 'genre_Documentary', 'genre_Comedy', 'genre_Animation', 'genre_Romance',\n"," 'genre_Music', 'genre_Horror', 'genre_Thriller', 'genre_Action',\n"," 'genre_Crime', 'genre_Family', 'genre_Adventure', 'genre_TV Movie',\n"," 'genre_Fantasy', 'genre_Science Fiction', 'genre_Mystery',\n"," 'genre_History', 'genre_War', 'genre_Western'],\n"," dtype='object')"]},"execution_count":24,"metadata":{},"output_type":"execute_result"}],"source":["train_data.columns"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.13147,"end_time":"2022-05-11T03:46:52.221279","exception":false,"start_time":"2022-05-11T03:46:52.089809","status":"completed"},"tags":[]},"source":["# 1. Task definition"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.13262,"end_time":"2022-05-11T03:46:52.514656","exception":false,"start_time":"2022-05-11T03:46:52.382036","status":"completed"},"tags":[]},"source":["### 1.1. Task type\n","\n","On the cell below we create Task object - the class to setup what task LightAutoML model should solve with specific loss and metric if necessary (more info can be found [here](https://lightautoml.readthedocs.io/en/latest/pages/modules/generated/lightautoml.tasks.base.Task.html#lightautoml.tasks.base.Task) in our documentation):"]},{"cell_type":"code","execution_count":12,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:36:51.861761Z","iopub.status.busy":"2022-06-02T11:36:51.861401Z","iopub.status.idle":"2022-06-02T11:36:51.868462Z","shell.execute_reply":"2022-06-02T11:36:51.867179Z","shell.execute_reply.started":"2022-06-02T11:36:51.861733Z"},"papermill":{"duration":0.142447,"end_time":"2022-05-11T03:46:52.788728","exception":false,"start_time":"2022-05-11T03:46:52.646281","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["task = Task('reg', loss = 'mae', metric = 'mae')"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.135627,"end_time":"2022-05-11T03:46:53.056301","exception":false,"start_time":"2022-05-11T03:46:52.920674","status":"completed"},"tags":[]},"source":["### 1.2. Feature roles setup"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.131703,"end_time":"2022-05-11T03:46:53.320032","exception":false,"start_time":"2022-05-11T03:46:53.188329","status":"completed"},"tags":[]},"source":["To solve the task, we need to setup columns roles. The **only role you must setup is target role**, everything else (drop, numeric, categorical, group, weights etc.) is up to user - LightAutoML models have automatic columns typization inside:"]},{"cell_type":"code","execution_count":25,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:54:24.586645Z","iopub.status.busy":"2022-06-02T11:54:24.586289Z","iopub.status.idle":"2022-06-02T11:54:24.601166Z","shell.execute_reply":"2022-06-02T11:54:24.595887Z","shell.execute_reply.started":"2022-06-02T11:54:24.586620Z"},"papermill":{"duration":0.139165,"end_time":"2022-05-11T03:46:53.5898","exception":false,"start_time":"2022-05-11T03:46:53.450635","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["roles = {\n"," 'target': TARGET_NAME,\n"," 'drop': ['ID', 'Название', 'Описание']\n","}"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.130994,"end_time":"2022-05-11T03:46:53.853469","exception":false,"start_time":"2022-05-11T03:46:53.722475","status":"completed"},"tags":[]},"source":["### 1.3. LightAutoML model creation - TabularAutoML preset"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.132328,"end_time":"2022-05-11T03:46:54.115757","exception":false,"start_time":"2022-05-11T03:46:53.983429","status":"completed"},"tags":[]},"source":["In next the cell we are going to create LightAutoML model with `TabularAutoML` class - preset with default model structure like in the image below:\n","\n","\"TabularAutoML\n","\n","in just several lines. Let's discuss the params we can setup:\n","- `task` - the type of the ML task (the only **must have** parameter)\n","- `timeout` - time limit in seconds for model to train\n","- `cpu_limit` - vCPU count for model to use\n","- `reader_params` - parameter change for Reader object inside preset, which works on the first step of data preparation: automatic feature typization, preliminary almost-constant features, correct CV setup etc. For example, we setup `n_jobs` threads for typization algo, `cv` folds and `random_state` as inside CV seed.\n","\n","**Important note**: `reader_params` key is one of the YAML config keys, which is used inside `TabularAutoML` preset. [More details](https://github.com/AILab-MLTools/LightAutoML/blob/master/lightautoml/automl/presets/tabular_config.yml) on its structure with explanation comments can be found on the link attached. Each key from this config can be modified with user settings during preset object initialization. To get more info about different parameters setting (for example, ML algos which can be used in `general_params->use_algos`) please take a look at our [article on TowardsDataScience](https://towardsdatascience.com/lightautoml-preset-usage-tutorial-2cce7da6f936).\n","\n","Moreover, to receive the automatic report for our model we can use `ReportDeco` decorator and work with the decorated version in the same way as we do with usual one (more details in [this tutorial](https://github.com/AILab-MLTools/LightAutoML/blob/master/examples/tutorials/Tutorial_1_basics.ipynb))"]},{"cell_type":"code","execution_count":26,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T11:54:32.360452Z","iopub.status.busy":"2022-06-02T11:54:32.360051Z","iopub.status.idle":"2022-06-02T11:54:32.426863Z","shell.execute_reply":"2022-06-02T11:54:32.422975Z","shell.execute_reply.started":"2022-06-02T11:54:32.360422Z"},"papermill":{"duration":0.179009,"end_time":"2022-05-11T03:46:54.42478","exception":false,"start_time":"2022-05-11T03:46:54.245771","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["automl = TabularAutoML(\n"," task = task, \n"," timeout = TIMEOUT,\n"," cpu_limit = N_THREADS,\n"," reader_params = {'n_jobs': N_THREADS, 'cv': N_FOLDS, 'random_state': RANDOM_STATE},\n",")"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.133697,"end_time":"2022-05-11T03:46:54.690292","exception":false,"start_time":"2022-05-11T03:46:54.556595","status":"completed"},"tags":[]},"source":["# 2. AutoML training"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.131342,"end_time":"2022-05-11T03:46:54.952589","exception":false,"start_time":"2022-05-11T03:46:54.821247","status":"completed"},"tags":[]},"source":["To run autoML training use fit_predict method:\n","- `train_data` - Dataset to train.\n","- `roles` - Roles dict.\n","- `verbose` - Controls the verbosity: the higher, the more messages.\n"," <1 : messages are not displayed;\n"," >=1 : the computation process for layers is displayed;\n"," >=2 : the information about folds processing is also displayed;\n"," >=3 : the hyperparameters optimization process is also displayed;\n"," >=4 : the training process for every algorithm is displayed;\n","\n","Note: out-of-fold prediction is calculated during training and returned from the fit_predict method"]},{"cell_type":"code","execution_count":28,"metadata":{"_kg_hide-output":true,"execution":{"iopub.execute_input":"2022-06-02T11:54:41.968720Z","iopub.status.busy":"2022-06-02T11:54:41.968239Z","iopub.status.idle":"2022-06-02T12:11:41.309528Z","shell.execute_reply":"2022-06-02T12:11:41.308519Z","shell.execute_reply.started":"2022-06-02T11:54:41.968686Z"},"papermill":{"duration":1100.343738,"end_time":"2022-05-11T04:05:15.432529","exception":false,"start_time":"2022-05-11T03:46:55.088791","status":"completed"},"scrolled":true,"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["[11:54:41] Stdout logging level is INFO3.\n","[11:54:41] Task: reg\n","\n","[11:54:41] Start automl preset with listed constraints:\n","[11:54:42] - time: 28800.00 seconds\n","[11:54:42] - CPU: 4 cores\n","[11:54:42] - memory: 16 GB\n","\n","[11:54:42] \u001b[1mTrain data shape: (52304, 33)\u001b[0m\n","\n","[11:54:52] Feats was rejected during automatic roles guess: []\n","[11:54:52] Layer \u001b[1m1\u001b[0m train process start. Time left 28789.14 secs\n","[11:54:56] Start fitting \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m ...\n","[11:54:56] ===== Start working with \u001b[1mfold 0\u001b[0m for \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m =====\n","[11:54:57] Linear model: C = 1e-05 score = -2.268310868941784\n","[11:54:57] Linear model: C = 5e-05 score = -2.268310868941784\n","[11:54:57] Linear model: C = 0.0001 score = -2.268310868941784\n","[11:54:57] ===== Start working with \u001b[1mfold 1\u001b[0m for \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m =====\n","[11:54:57] Linear model: C = 1e-05 score = -2.277373137084969\n","[11:54:57] Linear model: C = 5e-05 score = -2.277373137084969\n","[11:54:57] Linear model: C = 0.0001 score = -2.277373137084969\n","[11:54:57] ===== Start working with \u001b[1mfold 2\u001b[0m for \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m =====\n","[11:54:58] Linear model: C = 1e-05 score = -2.288060414874295\n","[11:54:58] Linear model: C = 5e-05 score = -2.288060414874295\n","[11:54:58] Linear model: C = 0.0001 score = -2.288060414874295\n","[11:54:58] ===== Start working with \u001b[1mfold 3\u001b[0m for \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m =====\n","[11:54:58] Linear model: C = 1e-05 score = -2.2644202275117102\n","[11:54:58] Linear model: C = 5e-05 score = -2.2644202275117102\n","[11:54:59] Linear model: C = 0.0001 score = -2.2644202275117102\n","[11:54:59] ===== Start working with \u001b[1mfold 4\u001b[0m for \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m =====\n","[11:54:59] Linear model: C = 1e-05 score = -2.3039005736137668\n","[11:54:59] Linear model: C = 5e-05 score = -2.3039005736137668\n","[11:54:59] Linear model: C = 0.0001 score = -2.3039005736137668\n","[11:54:59] Fitting \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m finished. score = \u001b[1m-2.2804125953473133\u001b[0m\n","[11:54:59] \u001b[1mLvl_0_Pipe_0_Mod_0_LinearL2\u001b[0m fitting and predicting completed\n","[11:54:59] Time left 28782.44 secs\n","\n","[11:55:00] [1]\tvalid's l1: 2.19941\n","[11:55:00] Training until validation scores don't improve for 200 rounds\n","[11:55:03] [100]\tvalid's l1: 0.523585\n","[11:55:05] [200]\tvalid's l1: 0.484525\n","[11:55:07] [300]\tvalid's l1: 0.480544\n","[11:55:09] [400]\tvalid's l1: 0.478641\n","[11:55:11] [500]\tvalid's l1: 0.47769\n","[11:55:13] [600]\tvalid's l1: 0.476938\n","[11:55:15] [700]\tvalid's l1: 0.476507\n","[11:55:17] [800]\tvalid's l1: 0.475847\n","[11:55:19] [900]\tvalid's l1: 0.475512\n","[11:55:21] [1000]\tvalid's l1: 0.475074\n","[11:55:23] [1100]\tvalid's l1: 0.474953\n","[11:55:25] [1200]\tvalid's l1: 0.474808\n","[11:55:26] \u001b[1mSelector_LightGBM\u001b[0m fitting and predicting completed\n","[11:55:30] Start fitting \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m ...\n","[11:55:30] ===== Start working with \u001b[1mfold 0\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m =====\n","[11:55:30] [1]\tvalid's l1: 2.19941\n","[11:55:30] Training until validation scores don't improve for 200 rounds\n","[11:55:33] [100]\tvalid's l1: 0.519828\n","[11:55:37] [200]\tvalid's l1: 0.490675\n","[11:55:39] [300]\tvalid's l1: 0.486851\n","[11:55:41] [400]\tvalid's l1: 0.483735\n","[11:55:43] [500]\tvalid's l1: 0.482953\n","[11:55:45] [600]\tvalid's l1: 0.482753\n","[11:55:47] [700]\tvalid's l1: 0.482724\n","[11:55:49] [800]\tvalid's l1: 0.482594\n","[11:55:51] [900]\tvalid's l1: 0.482418\n","[11:55:52] [1000]\tvalid's l1: 0.48237\n","[11:55:54] [1100]\tvalid's l1: 0.482276\n","[11:55:56] [1200]\tvalid's l1: 0.482057\n","[11:55:57] ===== Start working with \u001b[1mfold 1\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m =====\n","[11:55:58] [1]\tvalid's l1: 2.20858\n","[11:55:58] Training until validation scores don't improve for 200 rounds\n","[11:56:00] [100]\tvalid's l1: 0.521391\n","[11:56:02] [200]\tvalid's l1: 0.493482\n","[11:56:04] [300]\tvalid's l1: 0.489446\n","[11:56:06] [400]\tvalid's l1: 0.488538\n","[11:56:10] [500]\tvalid's l1: 0.488143\n","[11:56:12] [600]\tvalid's l1: 0.488013\n","[11:56:14] [700]\tvalid's l1: 0.487632\n","[11:56:16] [800]\tvalid's l1: 0.487497\n","[11:56:18] [900]\tvalid's l1: 0.487367\n","[11:56:19] [1000]\tvalid's l1: 0.487201\n","[11:56:21] [1100]\tvalid's l1: 0.487157\n","[11:56:23] [1200]\tvalid's l1: 0.487064\n","[11:56:24] ===== Start working with \u001b[1mfold 2\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m =====\n","[11:56:25] [1]\tvalid's l1: 2.21896\n","[11:56:25] Training until validation scores don't improve for 200 rounds\n","[11:56:27] [100]\tvalid's l1: 0.54507\n","[11:56:29] [200]\tvalid's l1: 0.514458\n","[11:56:31] [300]\tvalid's l1: 0.50764\n","[11:56:33] [400]\tvalid's l1: 0.507155\n","[11:56:36] [500]\tvalid's l1: 0.50703\n","[11:56:38] [600]\tvalid's l1: 0.506755\n","[11:56:40] [700]\tvalid's l1: 0.506811\n","[11:56:44] [800]\tvalid's l1: 0.506609\n","[11:56:46] [900]\tvalid's l1: 0.506557\n","[11:56:49] [1000]\tvalid's l1: 0.505981\n","[11:56:50] [1100]\tvalid's l1: 0.505949\n","[11:56:52] [1200]\tvalid's l1: 0.505856\n","[11:56:53] ===== Start working with \u001b[1mfold 3\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m =====\n","[11:56:54] [1]\tvalid's l1: 2.19606\n","[11:56:54] Training until validation scores don't improve for 200 rounds\n","[11:56:56] [100]\tvalid's l1: 0.529748\n","[11:56:58] [200]\tvalid's l1: 0.499831\n","[11:57:01] [300]\tvalid's l1: 0.496226\n","[11:57:03] [400]\tvalid's l1: 0.495904\n","[11:57:05] [500]\tvalid's l1: 0.495555\n","[11:57:07] [600]\tvalid's l1: 0.494663\n","[11:57:09] [700]\tvalid's l1: 0.494224\n","[11:57:12] [800]\tvalid's l1: 0.494009\n","[11:57:14] [900]\tvalid's l1: 0.493508\n","[11:57:18] [1000]\tvalid's l1: 0.493317\n","[11:57:20] [1100]\tvalid's l1: 0.493362\n","[11:57:22] [1200]\tvalid's l1: 0.493498\n","[11:57:23] ===== Start working with \u001b[1mfold 4\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m =====\n","[11:57:23] [1]\tvalid's l1: 2.23359\n","[11:57:23] Training until validation scores don't improve for 200 rounds\n","[11:57:26] [100]\tvalid's l1: 0.538571\n","[11:57:28] [200]\tvalid's l1: 0.508702\n","[11:57:30] [300]\tvalid's l1: 0.50467\n","[11:57:33] [400]\tvalid's l1: 0.502978\n","[11:57:35] [500]\tvalid's l1: 0.501108\n","[11:57:37] [600]\tvalid's l1: 0.500734\n","[11:57:39] [700]\tvalid's l1: 0.500567\n","[11:57:41] [800]\tvalid's l1: 0.50031\n","[11:57:43] [900]\tvalid's l1: 0.499824\n","[11:57:45] [1000]\tvalid's l1: 0.499842\n","[11:57:46] [1100]\tvalid's l1: 0.499829\n","[11:57:48] Fitting \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m finished. score = \u001b[1m-0.49360894743501255\u001b[0m\n","[11:57:48] \u001b[1mLvl_0_Pipe_1_Mod_0_LightGBM\u001b[0m fitting and predicting completed\n","[11:57:48] Start hyperparameters optimization for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m ... Time budget is 300.00 secs\n","[11:57:48] [1]\tvalid's l1: 2.24072\n","[11:57:48] Training until validation scores don't improve for 200 rounds\n","[11:57:55] [100]\tvalid's l1: 0.627424\n","[11:58:00] [200]\tvalid's l1: 0.528698\n","[11:58:06] [300]\tvalid's l1: 0.517847\n","[11:58:11] [400]\tvalid's l1: 0.514595\n","[11:58:16] [500]\tvalid's l1: 0.512884\n","[11:58:21] [600]\tvalid's l1: 0.512049\n","[11:58:28] [700]\tvalid's l1: 0.511604\n","[11:58:33] [800]\tvalid's l1: 0.510824\n","[11:58:37] [900]\tvalid's l1: 0.510318\n","[11:58:42] [1000]\tvalid's l1: 0.509927\n","[11:58:48] [1100]\tvalid's l1: 0.509223\n","[11:58:52] [1200]\tvalid's l1: 0.508852\n","[11:58:58] \u001b[1mTrial 1\u001b[0m with hyperparameters {'feature_fraction': 0.6872700594236812, 'num_leaves': 244, 'bagging_fraction': 0.8659969709057025, 'min_sum_hessian_in_leaf': 0.24810409748678125, 'reg_alpha': 2.5361081166471375e-07, 'reg_lambda': 2.5348407664333426e-07} scored -0.5088111627675244 in 0:01:10.088565\n","[11:58:59] [1]\tvalid's l1: 2.24905\n","[11:58:59] Training until validation scores don't improve for 200 rounds\n","[11:59:03] [100]\tvalid's l1: 0.699877\n","[11:59:07] [200]\tvalid's l1: 0.547292\n","[11:59:10] [300]\tvalid's l1: 0.52402\n","[11:59:12] [400]\tvalid's l1: 0.516918\n","[11:59:16] [500]\tvalid's l1: 0.514136\n","[11:59:20] [600]\tvalid's l1: 0.512601\n","[11:59:25] [700]\tvalid's l1: 0.511614\n","[11:59:30] [800]\tvalid's l1: 0.510794\n","[11:59:38] [900]\tvalid's l1: 0.510534\n","[11:59:42] [1000]\tvalid's l1: 0.510147\n","[11:59:49] [1100]\tvalid's l1: 0.509543\n","[12:00:00] [1200]\tvalid's l1: 0.509077\n","[12:00:05] \u001b[1mTrial 2\u001b[0m with hyperparameters {'feature_fraction': 0.5290418060840998, 'num_leaves': 223, 'bagging_fraction': 0.8005575058716043, 'min_sum_hessian_in_leaf': 0.679657809075816, 'reg_alpha': 1.5320059381854043e-08, 'reg_lambda': 5.360294728728285} scored -0.5090630181898693 in 0:01:06.938687\n","[12:00:05] [1]\tvalid's l1: 2.19943\n","[12:00:05] Training until validation scores don't improve for 200 rounds\n","[12:00:09] [100]\tvalid's l1: 0.510191\n","[12:00:12] [200]\tvalid's l1: 0.480505\n","[12:00:14] [300]\tvalid's l1: 0.478592\n","[12:00:15] [400]\tvalid's l1: 0.478308\n","[12:00:16] [500]\tvalid's l1: 0.477821\n","[12:00:17] [600]\tvalid's l1: 0.477427\n","[12:00:18] [700]\tvalid's l1: 0.477188\n","[12:00:19] [800]\tvalid's l1: 0.476976\n","[12:00:20] [900]\tvalid's l1: 0.476673\n","[12:00:21] [1000]\tvalid's l1: 0.476646\n","[12:00:22] [1100]\tvalid's l1: 0.476343\n","[12:00:23] [1200]\tvalid's l1: 0.476348\n","[12:00:23] \u001b[1mTrial 3\u001b[0m with hyperparameters {'feature_fraction': 0.9162213204002109, 'num_leaves': 66, 'bagging_fraction': 0.5909124836035503, 'min_sum_hessian_in_leaf': 0.00541524411940254, 'reg_alpha': 5.472429642032198e-06, 'reg_lambda': 0.00052821153945323} scored -0.4762184556322967 in 0:00:18.533547\n","[12:00:23] [1]\tvalid's l1: 2.23415\n","[12:00:23] Training until validation scores don't improve for 200 rounds\n","[12:00:25] [100]\tvalid's l1: 0.609824\n","[12:00:26] [200]\tvalid's l1: 0.51981\n","[12:00:28] [300]\tvalid's l1: 0.507742\n","[12:00:29] [400]\tvalid's l1: 0.50444\n","[12:00:30] [500]\tvalid's l1: 0.501341\n","[12:00:31] [600]\tvalid's l1: 0.499625\n","[12:00:34] [700]\tvalid's l1: 0.498473\n","[12:00:36] [800]\tvalid's l1: 0.498033\n","[12:00:37] [900]\tvalid's l1: 0.497338\n","[12:00:40] [1000]\tvalid's l1: 0.496858\n","[12:00:41] [1100]\tvalid's l1: 0.496249\n","[12:00:43] [1200]\tvalid's l1: 0.495695\n","[12:00:44] \u001b[1mTrial 4\u001b[0m with hyperparameters {'feature_fraction': 0.7159725093210578, 'num_leaves': 85, 'bagging_fraction': 0.8059264473611898, 'min_sum_hessian_in_leaf': 0.003613894271216527, 'reg_alpha': 4.258943089524393e-06, 'reg_lambda': 1.9826980964985924e-05} scored -0.49569355311324725 in 0:00:20.233426\n","[12:00:44] [1]\tvalid's l1: 2.23408\n","[12:00:44] Training until validation scores don't improve for 200 rounds\n","[12:00:45] [100]\tvalid's l1: 0.609029\n","[12:00:47] [200]\tvalid's l1: 0.520523\n","[12:00:49] [300]\tvalid's l1: 0.512312\n","[12:00:50] [400]\tvalid's l1: 0.509523\n","[12:00:52] [500]\tvalid's l1: 0.507929\n","[12:00:54] [600]\tvalid's l1: 0.506915\n","[12:00:55] [700]\tvalid's l1: 0.506088\n","[12:00:57] [800]\tvalid's l1: 0.505932\n","[12:00:59] [900]\tvalid's l1: 0.505533\n","[12:01:00] [1000]\tvalid's l1: 0.505634\n","[12:01:04] [1100]\tvalid's l1: 0.505469\n","[12:01:08] [1200]\tvalid's l1: 0.505266\n","[12:01:14] \u001b[1mTrial 5\u001b[0m with hyperparameters {'feature_fraction': 0.728034992108518, 'num_leaves': 204, 'bagging_fraction': 0.5998368910791798, 'min_sum_hessian_in_leaf': 0.11400863701127326, 'reg_alpha': 0.0021465011216654484, 'reg_lambda': 2.6185068507773707e-08} scored -0.5052662436160744 in 0:00:30.108885\n","[12:01:14] [1]\tvalid's l1: 2.19941\n","[12:01:14] Training until validation scores don't improve for 200 rounds\n","[12:01:16] [100]\tvalid's l1: 0.566533\n","[12:01:19] [200]\tvalid's l1: 0.511179\n","[12:01:21] [300]\tvalid's l1: 0.506064\n","[12:01:24] [400]\tvalid's l1: 0.502865\n","[12:01:26] [500]\tvalid's l1: 0.500364\n","[12:01:28] [600]\tvalid's l1: 0.499025\n","[12:01:30] [700]\tvalid's l1: 0.49777\n","[12:01:33] [800]\tvalid's l1: 0.496771\n","[12:01:35] [900]\tvalid's l1: 0.49549\n","[12:01:37] [1000]\tvalid's l1: 0.49495\n","[12:01:39] [1100]\tvalid's l1: 0.494477\n","[12:01:42] [1200]\tvalid's l1: 0.49389\n","[12:01:43] \u001b[1mTrial 6\u001b[0m with hyperparameters {'feature_fraction': 0.8037724259507192, 'num_leaves': 56, 'bagging_fraction': 0.5325257964926398, 'min_sum_hessian_in_leaf': 6.245139574743075, 'reg_alpha': 4.905556676028774, 'reg_lambda': 0.18861495878553936} scored -0.49386180661976975 in 0:00:29.495181\n","[12:01:43] [1]\tvalid's l1: 2.23636\n","[12:01:43] Training until validation scores don't improve for 200 rounds\n","[12:01:48] [100]\tvalid's l1: 0.64133\n","[12:01:50] [200]\tvalid's l1: 0.532105\n","[12:01:52] [300]\tvalid's l1: 0.520545\n","[12:01:55] [400]\tvalid's l1: 0.51556\n","[12:01:56] [500]\tvalid's l1: 0.510258\n","[12:01:57] [600]\tvalid's l1: 0.506972\n","[12:01:58] [700]\tvalid's l1: 0.504127\n","[12:01:58] [800]\tvalid's l1: 0.502181\n","[12:01:59] [900]\tvalid's l1: 0.5005\n","[12:02:00] [1000]\tvalid's l1: 0.499567\n","[12:02:01] [1100]\tvalid's l1: 0.498588\n","[12:02:02] [1200]\tvalid's l1: 0.497636\n","[12:02:02] \u001b[1mTrial 7\u001b[0m with hyperparameters {'feature_fraction': 0.6523068845866853, 'num_leaves': 39, 'bagging_fraction': 0.8421165132560784, 'min_sum_hessian_in_leaf': 0.057624872164786026, 'reg_alpha': 1.254134495897175e-07, 'reg_lambda': 0.00028614897264046574} scored -0.4976362964936784 in 0:00:18.874790\n","[12:02:02] [1]\tvalid's l1: 2.24935\n","[12:02:02] Training until validation scores don't improve for 200 rounds\n","[12:02:04] [100]\tvalid's l1: 0.730958\n","[12:02:06] [200]\tvalid's l1: 0.560976\n","[12:02:07] [300]\tvalid's l1: 0.530566\n","[12:02:09] [400]\tvalid's l1: 0.521362\n","[12:02:11] [500]\tvalid's l1: 0.51878\n","[12:02:13] [600]\tvalid's l1: 0.517296\n","[12:02:14] [700]\tvalid's l1: 0.516265\n","[12:02:16] [800]\tvalid's l1: 0.515196\n","[12:02:18] [900]\tvalid's l1: 0.514839\n","[12:02:20] [1000]\tvalid's l1: 0.514169\n","[12:02:24] [1100]\tvalid's l1: 0.513659\n","[12:02:28] [1200]\tvalid's l1: 0.513234\n","[12:02:32] \u001b[1mTrial 8\u001b[0m with hyperparameters {'feature_fraction': 0.5171942605576092, 'num_leaves': 234, 'bagging_fraction': 0.6293899908000085, 'min_sum_hessian_in_leaf': 0.4467752817973907, 'reg_alpha': 6.388511557344611e-06, 'reg_lambda': 0.0004793052550782129} scored -0.5132004535522512 in 0:00:29.607361\n","[12:02:32] [1]\tvalid's l1: 2.23583\n","[12:02:32] Training until validation scores don't improve for 200 rounds\n","[12:02:33] [100]\tvalid's l1: 0.58766\n","[12:02:34] [200]\tvalid's l1: 0.511651\n","[12:02:35] [300]\tvalid's l1: 0.503353\n","[12:02:36] [400]\tvalid's l1: 0.50174\n","[12:02:37] [500]\tvalid's l1: 0.500224\n","[12:02:38] [600]\tvalid's l1: 0.498424\n","[12:02:39] [700]\tvalid's l1: 0.49743\n","[12:02:40] [800]\tvalid's l1: 0.496633\n","[12:02:40] [900]\tvalid's l1: 0.495617\n","[12:02:41] [1000]\tvalid's l1: 0.495054\n","[12:02:42] [1100]\tvalid's l1: 0.49477\n","[12:02:43] [1200]\tvalid's l1: 0.494451\n","[12:02:44] \u001b[1mTrial 9\u001b[0m with hyperparameters {'feature_fraction': 0.7733551396716398, 'num_leaves': 60, 'bagging_fraction': 0.9847923138822793, 'min_sum_hessian_in_leaf': 1.2604664585649468, 'reg_alpha': 2.854239907497756, 'reg_lambda': 1.1309571585271483} scored -0.49443682189681776 in 0:00:11.839444\n","[12:02:44] [1]\tvalid's l1: 2.20038\n","[12:02:44] Training until validation scores don't improve for 200 rounds\n","[12:02:45] [100]\tvalid's l1: 0.563013\n","[12:02:47] [200]\tvalid's l1: 0.508776\n","[12:02:49] [300]\tvalid's l1: 0.505088\n","[12:02:53] [400]\tvalid's l1: 0.503152\n","[12:02:57] [500]\tvalid's l1: 0.502727\n","[12:02:59] [600]\tvalid's l1: 0.502707\n","[12:03:01] [700]\tvalid's l1: 0.502114\n","[12:03:05] [800]\tvalid's l1: 0.501506\n","[12:03:10] [900]\tvalid's l1: 0.501033\n","[12:03:15] [1000]\tvalid's l1: 0.500684\n","[12:03:17] [1100]\tvalid's l1: 0.500747\n","[12:03:19] [1200]\tvalid's l1: 0.500375\n","[12:03:21] \u001b[1mTrial 10\u001b[0m with hyperparameters {'feature_fraction': 0.7989499894055425, 'num_leaves': 237, 'bagging_fraction': 0.5442462510259598, 'min_sum_hessian_in_leaf': 0.006080390190296602, 'reg_alpha': 2.5529693461039728e-08, 'reg_lambda': 8.471746987003668e-06} scored -0.5003269122797968 in 0:00:37.633061\n","[12:03:21] Hyperparameters optimization for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m completed\n","[12:03:21] The set of hyperparameters \u001b[1m{'feature_fraction': 0.9162213204002109, 'num_leaves': 66, 'bagging_fraction': 0.5909124836035503, 'min_sum_hessian_in_leaf': 0.00541524411940254, 'reg_alpha': 5.472429642032198e-06, 'reg_lambda': 0.00052821153945323}\u001b[0m\n"," achieve -0.4762 mae\n","[12:03:21] Start fitting \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m ...\n","[12:03:21] ===== Start working with \u001b[1mfold 0\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m =====\n","[12:03:21] [1]\tvalid's l1: 2.1535\n","[12:03:21] Training until validation scores don't improve for 100 rounds\n","[12:03:23] [100]\tvalid's l1: 0.481238\n","[12:03:24] [200]\tvalid's l1: 0.477139\n","[12:03:25] [300]\tvalid's l1: 0.476745\n","[12:03:26] [400]\tvalid's l1: 0.476705\n","[12:03:27] [500]\tvalid's l1: 0.476438\n","[12:03:27] [600]\tvalid's l1: 0.47625\n","[12:03:28] [700]\tvalid's l1: 0.476191\n","[12:03:29] [800]\tvalid's l1: 0.476185\n","[12:03:30] ===== Start working with \u001b[1mfold 1\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m =====\n","[12:03:30] [1]\tvalid's l1: 2.16301\n","[12:03:30] Training until validation scores don't improve for 100 rounds\n","[12:03:31] [100]\tvalid's l1: 0.488522\n","[12:03:32] [200]\tvalid's l1: 0.485734\n","[12:03:33] [300]\tvalid's l1: 0.485557\n","[12:03:34] [400]\tvalid's l1: 0.485355\n","[12:03:35] [500]\tvalid's l1: 0.485142\n","[12:03:36] [600]\tvalid's l1: 0.485302\n","[12:03:36] ===== Start working with \u001b[1mfold 2\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m =====\n","[12:03:36] [1]\tvalid's l1: 2.17302\n","[12:03:36] Training until validation scores don't improve for 100 rounds\n","[12:03:37] [100]\tvalid's l1: 0.503444\n","[12:03:38] [200]\tvalid's l1: 0.499864\n","[12:03:39] [300]\tvalid's l1: 0.499578\n","[12:03:40] [400]\tvalid's l1: 0.499406\n","[12:03:41] [500]\tvalid's l1: 0.499288\n","[12:03:42] ===== Start working with \u001b[1mfold 3\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m =====\n","[12:03:43] [1]\tvalid's l1: 2.14905\n","[12:03:43] Training until validation scores don't improve for 100 rounds\n","[12:03:44] [100]\tvalid's l1: 0.491251\n","[12:03:45] [200]\tvalid's l1: 0.487495\n","[12:03:46] [300]\tvalid's l1: 0.487197\n","[12:03:47] [400]\tvalid's l1: 0.487025\n","[12:03:48] [500]\tvalid's l1: 0.486888\n","[12:03:48] [600]\tvalid's l1: 0.486767\n","[12:03:50] ===== Start working with \u001b[1mfold 4\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m =====\n","[12:03:50] [1]\tvalid's l1: 2.18616\n","[12:03:50] Training until validation scores don't improve for 100 rounds\n","[12:03:51] [100]\tvalid's l1: 0.501921\n","[12:03:52] [200]\tvalid's l1: 0.498995\n","[12:03:52] [300]\tvalid's l1: 0.498809\n","[12:03:53] [400]\tvalid's l1: 0.498854\n","[12:03:54] Fitting \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m finished. score = \u001b[1m-0.48913397757219135\u001b[0m\n","[12:03:54] \u001b[1mLvl_0_Pipe_1_Mod_1_Tuned_LightGBM\u001b[0m fitting and predicting completed\n","[12:03:54] Start fitting \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m ...\n","[12:03:54] ===== Start working with \u001b[1mfold 0\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m =====\n","[12:03:54] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 66.2ms\tremaining: 2m 12s\n","[12:03:55] 99:\tlearn: 0.5152896\ttest: 0.5054664\tbest: 0.5054664 (99)\ttotal: 828ms\tremaining: 15.7s\n","[12:03:56] 199:\tlearn: 0.5007902\ttest: 0.4938313\tbest: 0.4938139 (196)\ttotal: 1.58s\tremaining: 14.3s\n","[12:03:57] 299:\tlearn: 0.4945476\ttest: 0.4902180\tbest: 0.4901886 (298)\ttotal: 2.34s\tremaining: 13.3s\n","[12:03:57] 399:\tlearn: 0.4900887\ttest: 0.4873613\tbest: 0.4873613 (399)\ttotal: 3.1s\tremaining: 12.4s\n","[12:03:58] 499:\tlearn: 0.4863068\ttest: 0.4852389\tbest: 0.4852389 (499)\ttotal: 3.88s\tremaining: 11.7s\n","[12:03:59] 599:\tlearn: 0.4842125\ttest: 0.4848601\tbest: 0.4848496 (597)\ttotal: 4.68s\tremaining: 10.9s\n","[12:04:00] 699:\tlearn: 0.4819852\ttest: 0.4841600\tbest: 0.4841288 (683)\ttotal: 5.48s\tremaining: 10.2s\n","[12:04:01] 799:\tlearn: 0.4798067\ttest: 0.4836468\tbest: 0.4836468 (799)\ttotal: 6.36s\tremaining: 9.54s\n","[12:04:02] 899:\tlearn: 0.4780710\ttest: 0.4832166\tbest: 0.4832083 (896)\ttotal: 7.16s\tremaining: 8.75s\n","[12:04:02] 999:\tlearn: 0.4759972\ttest: 0.4824444\tbest: 0.4824117 (978)\ttotal: 7.94s\tremaining: 7.94s\n","[12:04:03] 1099:\tlearn: 0.4740877\ttest: 0.4827733\tbest: 0.4820273 (1039)\ttotal: 8.73s\tremaining: 7.15s\n","[12:04:04] 1199:\tlearn: 0.4726979\ttest: 0.4829599\tbest: 0.4820273 (1039)\ttotal: 9.51s\tremaining: 6.34s\n","[12:04:05] 1299:\tlearn: 0.4710549\ttest: 0.4826065\tbest: 0.4820273 (1039)\ttotal: 10.3s\tremaining: 5.54s\n","[12:04:05] Stopped by overfitting detector (300 iterations wait)\n","[12:04:05] bestTest = 0.4820272824\n","[12:04:05] bestIteration = 1039\n","[12:04:05] Shrink model to first 1040 iterations.\n","[12:04:05] ===== Start working with \u001b[1mfold 1\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m =====\n","[12:04:05] 0:\tlearn: 2.1670880\ttest: 2.1638858\tbest: 2.1638858 (0)\ttotal: 10.2ms\tremaining: 20.3s\n","[12:04:06] 99:\tlearn: 0.5139979\ttest: 0.5082569\tbest: 0.5082569 (99)\ttotal: 956ms\tremaining: 18.2s\n","[12:04:07] 199:\tlearn: 0.5035560\ttest: 0.5000076\tbest: 0.4999695 (193)\ttotal: 1.82s\tremaining: 16.4s\n","[12:04:08] 299:\tlearn: 0.4958859\ttest: 0.4937361\tbest: 0.4937361 (299)\ttotal: 2.57s\tremaining: 14.6s\n","[12:04:09] 399:\tlearn: 0.4907764\ttest: 0.4909833\tbest: 0.4909815 (398)\ttotal: 3.34s\tremaining: 13.4s\n","[12:04:10] 499:\tlearn: 0.4869673\ttest: 0.4892179\tbest: 0.4892005 (497)\ttotal: 4.08s\tremaining: 12.2s\n","[12:04:10] 599:\tlearn: 0.4842935\ttest: 0.4882945\tbest: 0.4882801 (590)\ttotal: 4.85s\tremaining: 11.3s\n","[12:04:11] 699:\tlearn: 0.4815103\ttest: 0.4872588\tbest: 0.4872478 (656)\ttotal: 5.64s\tremaining: 10.5s\n","[12:04:12] 799:\tlearn: 0.4792640\ttest: 0.4866574\tbest: 0.4866574 (799)\ttotal: 6.43s\tremaining: 9.65s\n","[12:04:13] 899:\tlearn: 0.4767958\ttest: 0.4860929\tbest: 0.4860929 (899)\ttotal: 7.21s\tremaining: 8.81s\n","[12:04:14] 999:\tlearn: 0.4745571\ttest: 0.4854359\tbest: 0.4854266 (994)\ttotal: 7.98s\tremaining: 7.98s\n","[12:04:14] 1099:\tlearn: 0.4728123\ttest: 0.4850264\tbest: 0.4850256 (1098)\ttotal: 8.79s\tremaining: 7.19s\n","[12:04:15] 1199:\tlearn: 0.4708895\ttest: 0.4846830\tbest: 0.4846682 (1196)\ttotal: 9.54s\tremaining: 6.36s\n","[12:04:16] 1299:\tlearn: 0.4696686\ttest: 0.4847139\tbest: 0.4846647 (1204)\ttotal: 10.3s\tremaining: 5.56s\n","[12:04:17] 1399:\tlearn: 0.4683280\ttest: 0.4846950\tbest: 0.4846647 (1204)\ttotal: 11.1s\tremaining: 4.76s\n","[12:04:18] 1499:\tlearn: 0.4667174\ttest: 0.4843062\tbest: 0.4842434 (1489)\ttotal: 11.9s\tremaining: 3.98s\n","[12:04:19] 1599:\tlearn: 0.4653726\ttest: 0.4844362\tbest: 0.4842434 (1489)\ttotal: 12.7s\tremaining: 3.18s\n","[12:04:19] 1699:\tlearn: 0.4637605\ttest: 0.4842007\tbest: 0.4841944 (1698)\ttotal: 13.5s\tremaining: 2.39s\n","[12:04:20] 1799:\tlearn: 0.4627119\ttest: 0.4840865\tbest: 0.4840798 (1796)\ttotal: 14.3s\tremaining: 1.59s\n","[12:04:21] 1899:\tlearn: 0.4614862\ttest: 0.4840688\tbest: 0.4840269 (1873)\ttotal: 15.1s\tremaining: 795ms\n","[12:04:22] 1999:\tlearn: 0.4600839\ttest: 0.4837244\tbest: 0.4837078 (1987)\ttotal: 15.9s\tremaining: 0us\n","[12:04:22] bestTest = 0.4837077763\n","[12:04:22] bestIteration = 1987\n","[12:04:22] Shrink model to first 1988 iterations.\n","[12:04:22] ===== Start working with \u001b[1mfold 2\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m =====\n","[12:04:22] 0:\tlearn: 2.1644848\ttest: 2.1739781\tbest: 2.1739781 (0)\ttotal: 9.26ms\tremaining: 18.5s\n","[12:04:23] 99:\tlearn: 0.5088204\ttest: 0.5242099\tbest: 0.5242099 (99)\ttotal: 785ms\tremaining: 14.9s\n","[12:04:24] 199:\tlearn: 0.4955685\ttest: 0.5138324\tbest: 0.5137465 (197)\ttotal: 1.59s\tremaining: 14.3s\n","[12:04:25] 299:\tlearn: 0.4888387\ttest: 0.5109640\tbest: 0.5108275 (298)\ttotal: 2.4s\tremaining: 13.6s\n","[12:04:25] 399:\tlearn: 0.4841286\ttest: 0.5089924\tbest: 0.5089924 (399)\ttotal: 3.23s\tremaining: 12.9s\n","[12:04:26] 499:\tlearn: 0.4806125\ttest: 0.5071888\tbest: 0.5071731 (495)\ttotal: 4.05s\tremaining: 12.1s\n","[12:04:27] 599:\tlearn: 0.4778329\ttest: 0.5062924\tbest: 0.5062924 (599)\ttotal: 4.84s\tremaining: 11.3s\n","[12:04:28] 699:\tlearn: 0.4748010\ttest: 0.5054902\tbest: 0.5054316 (686)\ttotal: 5.63s\tremaining: 10.5s\n","[12:04:29] 799:\tlearn: 0.4730213\ttest: 0.5052629\tbest: 0.5052629 (799)\ttotal: 6.38s\tremaining: 9.57s\n","[12:04:29] 899:\tlearn: 0.4710909\ttest: 0.5050392\tbest: 0.5050097 (854)\ttotal: 7.15s\tremaining: 8.74s\n","[12:04:30] 999:\tlearn: 0.4687453\ttest: 0.5049687\tbest: 0.5049555 (955)\ttotal: 7.95s\tremaining: 7.95s\n","[12:04:31] 1099:\tlearn: 0.4668951\ttest: 0.5054240\tbest: 0.5048653 (1032)\ttotal: 8.73s\tremaining: 7.14s\n","[12:04:32] 1199:\tlearn: 0.4656486\ttest: 0.5053304\tbest: 0.5048653 (1032)\ttotal: 9.49s\tremaining: 6.33s\n","[12:04:33] 1299:\tlearn: 0.4639458\ttest: 0.5052172\tbest: 0.5048653 (1032)\ttotal: 10.4s\tremaining: 5.58s\n","[12:04:33] Stopped by overfitting detector (300 iterations wait)\n","[12:04:33] bestTest = 0.5048653344\n","[12:04:33] bestIteration = 1032\n","[12:04:33] Shrink model to first 1033 iterations.\n","[12:04:33] ===== Start working with \u001b[1mfold 3\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m =====\n","[12:04:33] 0:\tlearn: 2.1694725\ttest: 2.1509784\tbest: 2.1509784 (0)\ttotal: 10ms\tremaining: 20.1s\n","[12:04:34] 99:\tlearn: 0.5129994\ttest: 0.5119367\tbest: 0.5119367 (99)\ttotal: 796ms\tremaining: 15.1s\n","[12:04:35] 199:\tlearn: 0.4993698\ttest: 0.4997137\tbest: 0.4997137 (199)\ttotal: 1.59s\tremaining: 14.3s\n","[12:04:36] 299:\tlearn: 0.4916979\ttest: 0.4942534\tbest: 0.4942534 (299)\ttotal: 2.35s\tremaining: 13.3s\n","[12:04:36] 399:\tlearn: 0.4873668\ttest: 0.4920321\tbest: 0.4920300 (398)\ttotal: 3.1s\tremaining: 12.4s\n","[12:04:37] 499:\tlearn: 0.4837618\ttest: 0.4904301\tbest: 0.4904048 (496)\ttotal: 3.87s\tremaining: 11.6s\n","[12:04:38] 599:\tlearn: 0.4810165\ttest: 0.4894180\tbest: 0.4894022 (596)\ttotal: 4.63s\tremaining: 10.8s\n","[12:04:39] 699:\tlearn: 0.4787709\ttest: 0.4885659\tbest: 0.4885659 (699)\ttotal: 5.39s\tremaining: 10s\n","[12:04:40] 799:\tlearn: 0.4767320\ttest: 0.4880660\tbest: 0.4880660 (799)\ttotal: 6.14s\tremaining: 9.21s\n","[12:04:40] 899:\tlearn: 0.4748104\ttest: 0.4876275\tbest: 0.4876275 (899)\ttotal: 6.9s\tremaining: 8.44s\n","[12:04:41] 999:\tlearn: 0.4728498\ttest: 0.4871867\tbest: 0.4871780 (998)\ttotal: 7.67s\tremaining: 7.67s\n","[12:04:42] 1099:\tlearn: 0.4709716\ttest: 0.4865983\tbest: 0.4865983 (1099)\ttotal: 8.48s\tremaining: 6.93s\n","[12:04:43] 1199:\tlearn: 0.4696479\ttest: 0.4863812\tbest: 0.4863812 (1199)\ttotal: 9.25s\tremaining: 6.17s\n","[12:04:44] 1299:\tlearn: 0.4682911\ttest: 0.4862960\tbest: 0.4862905 (1294)\ttotal: 10s\tremaining: 5.4s\n","[12:04:44] 1399:\tlearn: 0.4672746\ttest: 0.4859769\tbest: 0.4859711 (1398)\ttotal: 10.8s\tremaining: 4.63s\n","[12:04:45] 1499:\tlearn: 0.4661130\ttest: 0.4857374\tbest: 0.4857333 (1477)\ttotal: 11.6s\tremaining: 3.87s\n","[12:04:46] 1599:\tlearn: 0.4649082\ttest: 0.4855749\tbest: 0.4855734 (1594)\ttotal: 12.4s\tremaining: 3.1s\n","[12:04:47] 1699:\tlearn: 0.4633869\ttest: 0.4855726\tbest: 0.4853207 (1630)\ttotal: 13.2s\tremaining: 2.33s\n","[12:04:48] 1799:\tlearn: 0.4623886\ttest: 0.4854351\tbest: 0.4853207 (1630)\ttotal: 14s\tremaining: 1.55s\n","[12:04:49] 1899:\tlearn: 0.4612864\ttest: 0.4855129\tbest: 0.4853207 (1630)\ttotal: 14.8s\tremaining: 777ms\n","[12:04:49] Stopped by overfitting detector (300 iterations wait)\n","[12:04:49] bestTest = 0.4853207203\n","[12:04:49] bestIteration = 1630\n","[12:04:49] Shrink model to first 1631 iterations.\n","[12:04:49] ===== Start working with \u001b[1mfold 4\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m =====\n","[12:04:49] 0:\tlearn: 2.1600403\ttest: 2.1868379\tbest: 2.1868379 (0)\ttotal: 9.96ms\tremaining: 19.9s\n","[12:04:50] 99:\tlearn: 0.5098505\ttest: 0.5251172\tbest: 0.5251172 (99)\ttotal: 802ms\tremaining: 15.2s\n","[12:04:51] 199:\tlearn: 0.4958078\ttest: 0.5134559\tbest: 0.5134559 (199)\ttotal: 1.56s\tremaining: 14.1s\n","[12:04:51] 299:\tlearn: 0.4882550\ttest: 0.5085830\tbest: 0.5085830 (299)\ttotal: 2.33s\tremaining: 13.2s\n","[12:04:52] 399:\tlearn: 0.4840235\ttest: 0.5063497\tbest: 0.5063497 (399)\ttotal: 3.1s\tremaining: 12.4s\n","[12:04:53] 499:\tlearn: 0.4805835\ttest: 0.5048696\tbest: 0.5048696 (499)\ttotal: 3.84s\tremaining: 11.5s\n","[12:04:54] 599:\tlearn: 0.4772243\ttest: 0.5037921\tbest: 0.5037921 (599)\ttotal: 4.59s\tremaining: 10.7s\n","[12:04:55] 699:\tlearn: 0.4751076\ttest: 0.5031082\tbest: 0.5031082 (699)\ttotal: 5.35s\tremaining: 9.94s\n","[12:04:55] 799:\tlearn: 0.4729673\ttest: 0.5026883\tbest: 0.5026883 (799)\ttotal: 6.14s\tremaining: 9.22s\n","[12:04:56] 899:\tlearn: 0.4711268\ttest: 0.5021351\tbest: 0.5021351 (899)\ttotal: 6.93s\tremaining: 8.47s\n","[12:04:57] 999:\tlearn: 0.4695440\ttest: 0.5017525\tbest: 0.5017525 (999)\ttotal: 7.89s\tremaining: 7.89s\n","[12:04:58] 1099:\tlearn: 0.4680125\ttest: 0.5016699\tbest: 0.5016692 (1098)\ttotal: 8.68s\tremaining: 7.1s\n","[12:04:59] 1199:\tlearn: 0.4664938\ttest: 0.5015175\tbest: 0.5015094 (1197)\ttotal: 9.46s\tremaining: 6.31s\n","[12:05:00] 1299:\tlearn: 0.4648853\ttest: 0.5013084\tbest: 0.5012893 (1297)\ttotal: 10.2s\tremaining: 5.51s\n","[12:05:01] 1399:\tlearn: 0.4634305\ttest: 0.5013473\tbest: 0.5012893 (1297)\ttotal: 11s\tremaining: 4.73s\n","[12:05:01] 1499:\tlearn: 0.4621185\ttest: 0.5011114\tbest: 0.5011104 (1496)\ttotal: 11.9s\tremaining: 3.96s\n","[12:05:02] 1599:\tlearn: 0.4608891\ttest: 0.5011236\tbest: 0.5010993 (1554)\ttotal: 12.8s\tremaining: 3.19s\n","[12:05:03] 1699:\tlearn: 0.4598383\ttest: 0.5011389\tbest: 0.5010584 (1656)\ttotal: 13.6s\tremaining: 2.39s\n","[12:05:04] 1799:\tlearn: 0.4584777\ttest: 0.5011016\tbest: 0.5010584 (1656)\ttotal: 14.4s\tremaining: 1.6s\n","[12:05:05] 1899:\tlearn: 0.4575965\ttest: 0.5009550\tbest: 0.5009550 (1899)\ttotal: 15.3s\tremaining: 807ms\n","[12:05:06] 1999:\tlearn: 0.4565481\ttest: 0.5009208\tbest: 0.5009019 (1985)\ttotal: 16.1s\tremaining: 0us\n","[12:05:06] bestTest = 0.5009019218\n","[12:05:06] bestIteration = 1985\n","[12:05:06] Shrink model to first 1986 iterations.\n","[12:05:06] Fitting \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m finished. score = \u001b[1m-0.49136518112908373\u001b[0m\n","[12:05:06] \u001b[1mLvl_0_Pipe_1_Mod_2_CatBoost\u001b[0m fitting and predicting completed\n","[12:05:06] Start hyperparameters optimization for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m ... Time budget is 300.00 secs\n","[12:05:06] 0:\tlearn: 2.1677265\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.49ms\tremaining: 19s\n","[12:05:07] 99:\tlearn: 0.5233802\ttest: 0.5123769\tbest: 0.5123769 (99)\ttotal: 915ms\tremaining: 17.4s\n","[12:05:08] 199:\tlearn: 0.5103500\ttest: 0.5006084\tbest: 0.5006084 (199)\ttotal: 1.73s\tremaining: 15.6s\n","[12:05:09] 299:\tlearn: 0.5032400\ttest: 0.4947266\tbest: 0.4947266 (299)\ttotal: 2.43s\tremaining: 13.8s\n","[12:05:09] 399:\tlearn: 0.4984281\ttest: 0.4915502\tbest: 0.4915502 (399)\ttotal: 3.11s\tremaining: 12.4s\n","[12:05:10] 499:\tlearn: 0.4955406\ttest: 0.4894158\tbest: 0.4894158 (499)\ttotal: 3.78s\tremaining: 11.3s\n","[12:05:11] 599:\tlearn: 0.4928883\ttest: 0.4874990\tbest: 0.4874990 (599)\ttotal: 4.48s\tremaining: 10.4s\n","[12:05:11] 699:\tlearn: 0.4908529\ttest: 0.4860911\tbest: 0.4860873 (697)\ttotal: 5.19s\tremaining: 9.64s\n","[12:05:12] 799:\tlearn: 0.4890768\ttest: 0.4851408\tbest: 0.4851394 (789)\ttotal: 5.9s\tremaining: 8.85s\n","[12:05:13] 899:\tlearn: 0.4874309\ttest: 0.4843147\tbest: 0.4843084 (896)\ttotal: 6.62s\tremaining: 8.09s\n","[12:05:14] 999:\tlearn: 0.4865388\ttest: 0.4839954\tbest: 0.4839601 (979)\ttotal: 7.35s\tremaining: 7.35s\n","[12:05:15] 1099:\tlearn: 0.4854802\ttest: 0.4837207\tbest: 0.4837100 (1097)\ttotal: 8.08s\tremaining: 6.61s\n","[12:05:15] 1199:\tlearn: 0.4846195\ttest: 0.4834759\tbest: 0.4833264 (1193)\ttotal: 8.8s\tremaining: 5.87s\n","[12:05:16] 1299:\tlearn: 0.4835831\ttest: 0.4828609\tbest: 0.4828609 (1299)\ttotal: 9.49s\tremaining: 5.11s\n","[12:05:17] 1399:\tlearn: 0.4827808\ttest: 0.4828878\tbest: 0.4826588 (1346)\ttotal: 10.2s\tremaining: 4.36s\n","[12:05:17] 1499:\tlearn: 0.4817256\ttest: 0.4838736\tbest: 0.4826588 (1346)\ttotal: 10.9s\tremaining: 3.64s\n","[12:05:18] 1599:\tlearn: 0.4809835\ttest: 0.4843621\tbest: 0.4826588 (1346)\ttotal: 11.6s\tremaining: 2.91s\n","[12:05:19] Stopped by overfitting detector (300 iterations wait)\n","[12:05:19] bestTest = 0.4826588178\n","[12:05:19] bestIteration = 1346\n","[12:05:19] Shrink model to first 1347 iterations.\n","[12:05:19] \u001b[1mTrial 1\u001b[0m with hyperparameters {'max_depth': 4, 'nan_mode': 'Max', 'l2_leaf_reg': 0.0024430162614261413, 'min_data_in_leaf': 4} scored -0.4826595653170336 in 0:00:12.726158\n","[12:05:19] 0:\tlearn: 2.1677264\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.01ms\tremaining: 18s\n","[12:05:20] 99:\tlearn: 0.5303042\ttest: 0.5187633\tbest: 0.5187633 (99)\ttotal: 858ms\tremaining: 16.3s\n","[12:05:20] 199:\tlearn: 0.5192455\ttest: 0.5087111\tbest: 0.5087111 (199)\ttotal: 1.64s\tremaining: 14.7s\n","[12:05:21] 299:\tlearn: 0.5123311\ttest: 0.5034476\tbest: 0.5034476 (299)\ttotal: 2.34s\tremaining: 13.3s\n","[12:05:22] 399:\tlearn: 0.5085609\ttest: 0.5006934\tbest: 0.5006934 (399)\ttotal: 2.97s\tremaining: 11.9s\n","[12:05:23] 499:\tlearn: 0.5048914\ttest: 0.4979796\tbest: 0.4979796 (499)\ttotal: 3.6s\tremaining: 10.8s\n","[12:05:23] 599:\tlearn: 0.5022753\ttest: 0.4954736\tbest: 0.4954736 (599)\ttotal: 4.24s\tremaining: 9.88s\n","[12:05:24] 699:\tlearn: 0.5005963\ttest: 0.4942074\tbest: 0.4942074 (699)\ttotal: 4.84s\tremaining: 8.98s\n","[12:05:24] 799:\tlearn: 0.4992868\ttest: 0.4932967\tbest: 0.4932967 (799)\ttotal: 5.45s\tremaining: 8.18s\n","[12:05:25] 899:\tlearn: 0.4981002\ttest: 0.4926074\tbest: 0.4925830 (856)\ttotal: 6.05s\tremaining: 7.39s\n","[12:05:26] 999:\tlearn: 0.4965622\ttest: 0.4914207\tbest: 0.4914207 (999)\ttotal: 6.66s\tremaining: 6.66s\n","[12:05:26] 1099:\tlearn: 0.4954636\ttest: 0.4906769\tbest: 0.4906709 (1096)\ttotal: 7.32s\tremaining: 5.99s\n","[12:05:27] 1199:\tlearn: 0.4945001\ttest: 0.4901696\tbest: 0.4901681 (1198)\ttotal: 7.97s\tremaining: 5.32s\n","[12:05:28] 1299:\tlearn: 0.4934221\ttest: 0.4894774\tbest: 0.4894578 (1275)\ttotal: 8.61s\tremaining: 4.63s\n","[12:05:28] 1399:\tlearn: 0.4925096\ttest: 0.4886546\tbest: 0.4886546 (1399)\ttotal: 9.23s\tremaining: 3.96s\n","[12:05:29] 1499:\tlearn: 0.4918531\ttest: 0.4883635\tbest: 0.4883635 (1499)\ttotal: 9.86s\tremaining: 3.29s\n","[12:05:30] 1599:\tlearn: 0.4912760\ttest: 0.4884506\tbest: 0.4879111 (1582)\ttotal: 10.5s\tremaining: 2.63s\n","[12:05:30] 1699:\tlearn: 0.4905157\ttest: 0.4880944\tbest: 0.4879111 (1582)\ttotal: 11.2s\tremaining: 1.98s\n","[12:05:31] 1799:\tlearn: 0.4899348\ttest: 0.4877214\tbest: 0.4877171 (1797)\ttotal: 11.8s\tremaining: 1.31s\n","[12:05:32] 1899:\tlearn: 0.4892516\ttest: 0.4871649\tbest: 0.4871649 (1899)\ttotal: 12.5s\tremaining: 657ms\n","[12:05:33] 1999:\tlearn: 0.4888119\ttest: 0.4871315\tbest: 0.4871202 (1994)\ttotal: 13.1s\tremaining: 0us\n","[12:05:33] bestTest = 0.4871202265\n","[12:05:33] bestIteration = 1994\n","[12:05:33] Shrink model to first 1995 iterations.\n","[12:05:33] \u001b[1mTrial 2\u001b[0m with hyperparameters {'max_depth': 3, 'nan_mode': 'Min', 'l2_leaf_reg': 0.002570603566117598, 'min_data_in_leaf': 15} scored -0.48712097949683775 in 0:00:13.916779\n","[12:05:33] 0:\tlearn: 2.1677264\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 8.5ms\tremaining: 17s\n","[12:05:34] 99:\tlearn: 0.5307436\ttest: 0.5196176\tbest: 0.5196176 (99)\ttotal: 860ms\tremaining: 16.3s\n","[12:05:34] 199:\tlearn: 0.5187504\ttest: 0.5083459\tbest: 0.5083459 (199)\ttotal: 1.63s\tremaining: 14.7s\n","[12:05:35] 299:\tlearn: 0.5163787\ttest: 0.5057561\tbest: 0.5057561 (299)\ttotal: 2.37s\tremaining: 13.4s\n","[12:05:36] 399:\tlearn: 0.5160479\ttest: 0.5054490\tbest: 0.5054480 (396)\ttotal: 3.1s\tremaining: 12.4s\n","[12:05:37] 499:\tlearn: 0.5160473\ttest: 0.5054557\tbest: 0.5054480 (396)\ttotal: 3.84s\tremaining: 11.5s\n","[12:05:38] 599:\tlearn: 0.5160473\ttest: 0.5054557\tbest: 0.5054480 (396)\ttotal: 4.68s\tremaining: 10.9s\n","[12:05:38] Stopped by overfitting detector (300 iterations wait)\n","[12:05:38] bestTest = 0.5054479958\n","[12:05:38] bestIteration = 396\n","[12:05:38] Shrink model to first 397 iterations.\n","[12:05:38] \u001b[1mTrial 3\u001b[0m with hyperparameters {'max_depth': 3, 'nan_mode': 'Max', 'l2_leaf_reg': 8.148018307012941e-07, 'min_data_in_leaf': 4} scored -0.5054488961949658 in 0:00:05.768270\n","[12:05:38] 0:\tlearn: 2.1677264\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.14ms\tremaining: 18.3s\n","[12:05:39] 99:\tlearn: 0.5303042\ttest: 0.5187633\tbest: 0.5187633 (99)\ttotal: 855ms\tremaining: 16.2s\n","[12:05:40] 199:\tlearn: 0.5192455\ttest: 0.5087111\tbest: 0.5087111 (199)\ttotal: 1.64s\tremaining: 14.8s\n","[12:05:41] 299:\tlearn: 0.5123311\ttest: 0.5034476\tbest: 0.5034476 (299)\ttotal: 2.36s\tremaining: 13.4s\n","[12:05:42] 399:\tlearn: 0.5085609\ttest: 0.5006934\tbest: 0.5006934 (399)\ttotal: 3.03s\tremaining: 12.1s\n","[12:05:42] 499:\tlearn: 0.5048914\ttest: 0.4979796\tbest: 0.4979796 (499)\ttotal: 3.7s\tremaining: 11.1s\n","[12:05:43] 599:\tlearn: 0.5022753\ttest: 0.4954736\tbest: 0.4954736 (599)\ttotal: 4.34s\tremaining: 10.1s\n","[12:05:44] 699:\tlearn: 0.5005963\ttest: 0.4942074\tbest: 0.4942074 (699)\ttotal: 4.98s\tremaining: 9.26s\n","[12:05:44] 799:\tlearn: 0.4992868\ttest: 0.4932967\tbest: 0.4932967 (799)\ttotal: 5.63s\tremaining: 8.44s\n","[12:05:45] 899:\tlearn: 0.4981002\ttest: 0.4926074\tbest: 0.4925830 (856)\ttotal: 6.27s\tremaining: 7.66s\n","[12:05:46] 999:\tlearn: 0.4965622\ttest: 0.4914207\tbest: 0.4914207 (999)\ttotal: 6.93s\tremaining: 6.93s\n","[12:05:46] 1099:\tlearn: 0.4954636\ttest: 0.4906769\tbest: 0.4906709 (1096)\ttotal: 7.59s\tremaining: 6.21s\n","[12:05:47] 1199:\tlearn: 0.4945001\ttest: 0.4901696\tbest: 0.4901681 (1198)\ttotal: 8.24s\tremaining: 5.49s\n","[12:05:48] 1299:\tlearn: 0.4934221\ttest: 0.4894774\tbest: 0.4894578 (1275)\ttotal: 8.92s\tremaining: 4.8s\n","[12:05:48] 1399:\tlearn: 0.4925096\ttest: 0.4886546\tbest: 0.4886546 (1399)\ttotal: 9.55s\tremaining: 4.09s\n","[12:05:49] 1499:\tlearn: 0.4918531\ttest: 0.4883635\tbest: 0.4883635 (1499)\ttotal: 10.2s\tremaining: 3.39s\n","[12:05:50] 1599:\tlearn: 0.4912760\ttest: 0.4884506\tbest: 0.4879111 (1582)\ttotal: 10.8s\tremaining: 2.71s\n","[12:05:51] 1699:\tlearn: 0.4905157\ttest: 0.4880944\tbest: 0.4879111 (1582)\ttotal: 11.5s\tremaining: 2.03s\n","[12:05:51] 1799:\tlearn: 0.4899348\ttest: 0.4877214\tbest: 0.4877171 (1797)\ttotal: 12.1s\tremaining: 1.35s\n","[12:05:52] 1899:\tlearn: 0.4892516\ttest: 0.4871649\tbest: 0.4871649 (1899)\ttotal: 12.8s\tremaining: 675ms\n","[12:05:53] 1999:\tlearn: 0.4888119\ttest: 0.4871315\tbest: 0.4871202 (1994)\ttotal: 13.5s\tremaining: 0us\n","[12:05:53] bestTest = 0.4871202265\n","[12:05:53] bestIteration = 1994\n","[12:05:53] Shrink model to first 1995 iterations.\n","[12:05:53] \u001b[1mTrial 4\u001b[0m with hyperparameters {'max_depth': 3, 'nan_mode': 'Min', 'l2_leaf_reg': 7.71800699380605e-05, 'min_data_in_leaf': 6} scored -0.48712097949683775 in 0:00:14.342502\n","[12:05:53] 0:\tlearn: 2.1688168\ttest: 2.1544035\tbest: 2.1544035 (0)\ttotal: 12ms\tremaining: 23.9s\n","[12:05:54] 99:\tlearn: 0.5096320\ttest: 0.5012202\tbest: 0.5011960 (98)\ttotal: 888ms\tremaining: 16.9s\n","[12:05:55] 199:\tlearn: 0.4952301\ttest: 0.4913061\tbest: 0.4913061 (199)\ttotal: 1.73s\tremaining: 15.6s\n","[12:05:55] 299:\tlearn: 0.4880124\ttest: 0.4873842\tbest: 0.4873842 (299)\ttotal: 2.59s\tremaining: 14.7s\n","[12:05:56] 399:\tlearn: 0.4827827\ttest: 0.4863148\tbest: 0.4856108 (365)\ttotal: 3.46s\tremaining: 13.9s\n","[12:05:57] 499:\tlearn: 0.4780216\ttest: 0.4848450\tbest: 0.4848450 (499)\ttotal: 4.35s\tremaining: 13.1s\n","[12:05:58] 599:\tlearn: 0.4740684\ttest: 0.4836834\tbest: 0.4836834 (599)\ttotal: 5.23s\tremaining: 12.2s\n","[12:05:59] 699:\tlearn: 0.4712689\ttest: 0.4838708\tbest: 0.4834260 (639)\ttotal: 6.11s\tremaining: 11.3s\n","[12:06:00] 799:\tlearn: 0.4690978\ttest: 0.4843655\tbest: 0.4834260 (639)\ttotal: 6.97s\tremaining: 10.5s\n","[12:06:01] 899:\tlearn: 0.4667832\ttest: 0.4844902\tbest: 0.4834260 (639)\ttotal: 7.84s\tremaining: 9.58s\n","[12:06:01] Stopped by overfitting detector (300 iterations wait)\n","[12:06:01] bestTest = 0.4834260275\n","[12:06:01] bestIteration = 639\n","[12:06:01] Shrink model to first 640 iterations.\n","[12:06:01] \u001b[1mTrial 5\u001b[0m with hyperparameters {'max_depth': 6, 'nan_mode': 'Min', 'l2_leaf_reg': 1.9826980964985924e-05, 'min_data_in_leaf': 10} scored -0.4834267582082168 in 0:00:08.690170\n","[12:06:02] 0:\tlearn: 2.1688168\ttest: 2.1544035\tbest: 2.1544035 (0)\ttotal: 10.3ms\tremaining: 20.6s\n","[12:06:02] 99:\tlearn: 0.5096320\ttest: 0.5012202\tbest: 0.5011960 (98)\ttotal: 933ms\tremaining: 17.7s\n","[12:06:03] 199:\tlearn: 0.4952301\ttest: 0.4913061\tbest: 0.4913061 (199)\ttotal: 1.78s\tremaining: 16s\n","[12:06:04] 299:\tlearn: 0.4880124\ttest: 0.4873842\tbest: 0.4873842 (299)\ttotal: 2.66s\tremaining: 15.1s\n","[12:06:05] 399:\tlearn: 0.4827827\ttest: 0.4863148\tbest: 0.4856108 (365)\ttotal: 3.52s\tremaining: 14.1s\n","[12:06:06] 499:\tlearn: 0.4780216\ttest: 0.4848450\tbest: 0.4848450 (499)\ttotal: 4.38s\tremaining: 13.1s\n","[12:06:07] 599:\tlearn: 0.4740684\ttest: 0.4836834\tbest: 0.4836834 (599)\ttotal: 5.28s\tremaining: 12.3s\n","[12:06:08] 699:\tlearn: 0.4712689\ttest: 0.4838708\tbest: 0.4834260 (639)\ttotal: 6.19s\tremaining: 11.5s\n","[12:06:09] 799:\tlearn: 0.4690978\ttest: 0.4843655\tbest: 0.4834260 (639)\ttotal: 7.23s\tremaining: 10.8s\n","[12:06:10] 899:\tlearn: 0.4667832\ttest: 0.4844902\tbest: 0.4834260 (639)\ttotal: 8.2s\tremaining: 10s\n","[12:06:10] Stopped by overfitting detector (300 iterations wait)\n","[12:06:10] bestTest = 0.4834260275\n","[12:06:10] bestIteration = 639\n","[12:06:10] Shrink model to first 640 iterations.\n","[12:06:10] \u001b[1mTrial 6\u001b[0m with hyperparameters {'max_depth': 6, 'nan_mode': 'Min', 'l2_leaf_reg': 0.0021465011216654484, 'min_data_in_leaf': 1} scored -0.4834267582082168 in 0:00:09.073666\n","[12:06:11] 0:\tlearn: 2.1688168\ttest: 2.1544035\tbest: 2.1544035 (0)\ttotal: 11.2ms\tremaining: 22.4s\n","[12:06:12] 99:\tlearn: 0.5096307\ttest: 0.5007818\tbest: 0.5005704 (97)\ttotal: 975ms\tremaining: 18.5s\n","[12:06:13] 199:\tlearn: 0.4951970\ttest: 0.4919968\tbest: 0.4919968 (199)\ttotal: 1.89s\tremaining: 17s\n","[12:06:13] 299:\tlearn: 0.4871518\ttest: 0.4872868\tbest: 0.4872859 (298)\ttotal: 2.76s\tremaining: 15.6s\n","[12:06:14] 399:\tlearn: 0.4819787\ttest: 0.4861972\tbest: 0.4861937 (398)\ttotal: 3.65s\tremaining: 14.6s\n","[12:06:15] 499:\tlearn: 0.4773539\ttest: 0.4846302\tbest: 0.4843498 (482)\ttotal: 4.54s\tremaining: 13.6s\n","[12:06:16] 599:\tlearn: 0.4739498\ttest: 0.4839954\tbest: 0.4839932 (598)\ttotal: 5.44s\tremaining: 12.7s\n","[12:06:17] 699:\tlearn: 0.4708376\ttest: 0.4834758\tbest: 0.4833803 (692)\ttotal: 6.35s\tremaining: 11.8s\n","[12:06:18] 799:\tlearn: 0.4680119\ttest: 0.4835408\tbest: 0.4832742 (761)\ttotal: 7.26s\tremaining: 10.9s\n","[12:06:19] 899:\tlearn: 0.4650910\ttest: 0.4840307\tbest: 0.4832742 (761)\ttotal: 8.16s\tremaining: 9.97s\n","[12:06:20] 999:\tlearn: 0.4621586\ttest: 0.4836454\tbest: 0.4832742 (761)\ttotal: 9.06s\tremaining: 9.06s\n","[12:06:21] Stopped by overfitting detector (300 iterations wait)\n","[12:06:21] bestTest = 0.483274167\n","[12:06:21] bestIteration = 761\n","[12:06:21] Shrink model to first 762 iterations.\n","[12:06:21] \u001b[1mTrial 7\u001b[0m with hyperparameters {'max_depth': 6, 'nan_mode': 'Max', 'l2_leaf_reg': 3.4671276804481113, 'min_data_in_leaf': 20} scored -0.4832749435587272 in 0:00:10.140958\n","[12:06:21] 0:\tlearn: 2.1698661\ttest: 2.1554650\tbest: 2.1554650 (0)\ttotal: 12.5ms\tremaining: 25s\n","[12:06:22] 99:\tlearn: 0.5041294\ttest: 0.4979055\tbest: 0.4979055 (99)\ttotal: 1.03s\tremaining: 19.7s\n","[12:06:23] 199:\tlearn: 0.4896569\ttest: 0.4893669\tbest: 0.4893669 (199)\ttotal: 2.06s\tremaining: 18.5s\n","[12:06:24] 299:\tlearn: 0.4795697\ttest: 0.4855189\tbest: 0.4855189 (299)\ttotal: 3.03s\tremaining: 17.2s\n","[12:06:25] 399:\tlearn: 0.4719033\ttest: 0.4831956\tbest: 0.4831878 (396)\ttotal: 4.09s\tremaining: 16.4s\n","[12:06:26] 499:\tlearn: 0.4662442\ttest: 0.4835189\tbest: 0.4831875 (404)\ttotal: 5.14s\tremaining: 15.4s\n","[12:06:27] 599:\tlearn: 0.4616382\ttest: 0.4859876\tbest: 0.4831875 (404)\ttotal: 6.2s\tremaining: 14.5s\n","[12:06:28] 699:\tlearn: 0.4575575\ttest: 0.4862313\tbest: 0.4831875 (404)\ttotal: 7.25s\tremaining: 13.5s\n","[12:06:28] Stopped by overfitting detector (300 iterations wait)\n","[12:06:28] bestTest = 0.483187469\n","[12:06:28] bestIteration = 404\n","[12:06:28] Shrink model to first 405 iterations.\n","[12:06:28] \u001b[1mTrial 8\u001b[0m with hyperparameters {'max_depth': 7, 'nan_mode': 'Max', 'l2_leaf_reg': 0.014391207615728067, 'min_data_in_leaf': 9} scored -0.4831882169891197 in 0:00:07.705083\n","[12:06:28] 0:\tlearn: 2.1677264\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 8.85ms\tremaining: 17.7s\n","[12:06:29] 99:\tlearn: 0.5307436\ttest: 0.5196176\tbest: 0.5196176 (99)\ttotal: 857ms\tremaining: 16.3s\n","[12:06:30] 199:\tlearn: 0.5181426\ttest: 0.5078462\tbest: 0.5078462 (199)\ttotal: 1.65s\tremaining: 14.8s\n","[12:06:31] 299:\tlearn: 0.5144165\ttest: 0.5047509\tbest: 0.5047509 (299)\ttotal: 2.39s\tremaining: 13.5s\n","[12:06:32] 399:\tlearn: 0.5117660\ttest: 0.5025890\tbest: 0.5025890 (399)\ttotal: 3.07s\tremaining: 12.3s\n","[12:06:32] 499:\tlearn: 0.5082327\ttest: 0.4996091\tbest: 0.4996077 (496)\ttotal: 3.72s\tremaining: 11.2s\n","[12:06:33] 599:\tlearn: 0.5079167\ttest: 0.4995494\tbest: 0.4995254 (549)\ttotal: 4.39s\tremaining: 10.3s\n","[12:06:34] 699:\tlearn: 0.5060342\ttest: 0.4963527\tbest: 0.4963527 (699)\ttotal: 5.05s\tremaining: 9.38s\n","[12:06:34] 799:\tlearn: 0.5060224\ttest: 0.4963632\tbest: 0.4963103 (729)\ttotal: 5.74s\tremaining: 8.61s\n","[12:06:35] 899:\tlearn: 0.5060225\ttest: 0.4963661\tbest: 0.4963103 (729)\ttotal: 6.43s\tremaining: 7.85s\n","[12:06:36] 999:\tlearn: 0.5060225\ttest: 0.4963662\tbest: 0.4963103 (729)\ttotal: 7.15s\tremaining: 7.15s\n","[12:06:36] Stopped by overfitting detector (300 iterations wait)\n","[12:06:36] bestTest = 0.4963103231\n","[12:06:36] bestIteration = 729\n","[12:06:36] Shrink model to first 730 iterations.\n","[12:06:36] \u001b[1mTrial 9\u001b[0m with hyperparameters {'max_depth': 3, 'nan_mode': 'Max', 'l2_leaf_reg': 1.527156759251193, 'min_data_in_leaf': 6} scored -0.4963110505121668 in 0:00:07.821092\n","[12:06:36] 0:\tlearn: 2.1688168\ttest: 2.1544035\tbest: 2.1544035 (0)\ttotal: 11.5ms\tremaining: 23s\n","[12:06:37] 99:\tlearn: 0.5096320\ttest: 0.5012202\tbest: 0.5011960 (98)\ttotal: 913ms\tremaining: 17.3s\n","[12:06:38] 199:\tlearn: 0.4952301\ttest: 0.4913061\tbest: 0.4913061 (199)\ttotal: 1.77s\tremaining: 15.9s\n","[12:06:39] 299:\tlearn: 0.4880124\ttest: 0.4873842\tbest: 0.4873842 (299)\ttotal: 2.61s\tremaining: 14.8s\n","[12:06:40] 399:\tlearn: 0.4827827\ttest: 0.4863148\tbest: 0.4856108 (365)\ttotal: 3.48s\tremaining: 13.9s\n","[12:06:41] 499:\tlearn: 0.4780216\ttest: 0.4848450\tbest: 0.4848450 (499)\ttotal: 4.41s\tremaining: 13.2s\n","[12:06:42] 599:\tlearn: 0.4740684\ttest: 0.4836834\tbest: 0.4836834 (599)\ttotal: 5.38s\tremaining: 12.6s\n","[12:06:43] 699:\tlearn: 0.4712689\ttest: 0.4838708\tbest: 0.4834260 (639)\ttotal: 6.31s\tremaining: 11.7s\n","[12:06:44] 799:\tlearn: 0.4690978\ttest: 0.4843655\tbest: 0.4834260 (639)\ttotal: 7.23s\tremaining: 10.8s\n","[12:06:45] 899:\tlearn: 0.4667832\ttest: 0.4844902\tbest: 0.4834260 (639)\ttotal: 8.11s\tremaining: 9.91s\n","[12:06:45] Stopped by overfitting detector (300 iterations wait)\n","[12:06:45] bestTest = 0.4834260275\n","[12:06:45] bestIteration = 639\n","[12:06:45] Shrink model to first 640 iterations.\n","[12:06:45] \u001b[1mTrial 10\u001b[0m with hyperparameters {'max_depth': 6, 'nan_mode': 'Min', 'l2_leaf_reg': 0.0008325158565947976, 'min_data_in_leaf': 4} scored -0.4834267582082168 in 0:00:08.953142\n","[12:06:45] 0:\tlearn: 2.1677265\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.27ms\tremaining: 18.5s\n","[12:06:46] 99:\tlearn: 0.5233802\ttest: 0.5123769\tbest: 0.5123769 (99)\ttotal: 892ms\tremaining: 16.9s\n","[12:06:47] 199:\tlearn: 0.5103500\ttest: 0.5006084\tbest: 0.5006084 (199)\ttotal: 1.71s\tremaining: 15.4s\n","[12:06:48] 299:\tlearn: 0.5032400\ttest: 0.4947266\tbest: 0.4947266 (299)\ttotal: 2.43s\tremaining: 13.8s\n","[12:06:49] 399:\tlearn: 0.4984281\ttest: 0.4915502\tbest: 0.4915502 (399)\ttotal: 3.13s\tremaining: 12.5s\n","[12:06:49] 499:\tlearn: 0.4955406\ttest: 0.4894158\tbest: 0.4894158 (499)\ttotal: 3.82s\tremaining: 11.5s\n","[12:06:50] 599:\tlearn: 0.4928883\ttest: 0.4874990\tbest: 0.4874990 (599)\ttotal: 4.54s\tremaining: 10.6s\n","[12:06:51] 699:\tlearn: 0.4908529\ttest: 0.4860911\tbest: 0.4860873 (697)\ttotal: 5.26s\tremaining: 9.77s\n","[12:06:51] 799:\tlearn: 0.4890768\ttest: 0.4851408\tbest: 0.4851394 (789)\ttotal: 5.94s\tremaining: 8.91s\n","[12:06:52] 899:\tlearn: 0.4874309\ttest: 0.4843147\tbest: 0.4843084 (896)\ttotal: 6.67s\tremaining: 8.16s\n","[12:06:53] 999:\tlearn: 0.4865388\ttest: 0.4839954\tbest: 0.4839601 (979)\ttotal: 7.38s\tremaining: 7.38s\n","[12:06:54] 1099:\tlearn: 0.4854802\ttest: 0.4837207\tbest: 0.4837100 (1097)\ttotal: 8.07s\tremaining: 6.6s\n","[12:06:54] 1199:\tlearn: 0.4846195\ttest: 0.4834759\tbest: 0.4833264 (1193)\ttotal: 8.76s\tremaining: 5.84s\n","[12:06:55] 1299:\tlearn: 0.4835831\ttest: 0.4828609\tbest: 0.4828609 (1299)\ttotal: 9.46s\tremaining: 5.09s\n","[12:06:56] 1399:\tlearn: 0.4827808\ttest: 0.4828878\tbest: 0.4826588 (1346)\ttotal: 10.2s\tremaining: 4.35s\n","[12:06:57] 1499:\tlearn: 0.4817256\ttest: 0.4838736\tbest: 0.4826588 (1346)\ttotal: 10.9s\tremaining: 3.62s\n","[12:06:57] 1599:\tlearn: 0.4809835\ttest: 0.4843621\tbest: 0.4826588 (1346)\ttotal: 11.6s\tremaining: 2.9s\n","[12:06:58] Stopped by overfitting detector (300 iterations wait)\n","[12:06:58] bestTest = 0.4826588178\n","[12:06:58] bestIteration = 1346\n","[12:06:58] Shrink model to first 1347 iterations.\n","[12:06:58] \u001b[1mTrial 11\u001b[0m with hyperparameters {'max_depth': 4, 'nan_mode': 'Max', 'l2_leaf_reg': 1.1692997958212103e-08, 'min_data_in_leaf': 14} scored -0.4826595653170336 in 0:00:12.645166\n","[12:06:58] 0:\tlearn: 2.1677265\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.34ms\tremaining: 18.7s\n","[12:06:59] 99:\tlearn: 0.5233802\ttest: 0.5123769\tbest: 0.5123769 (99)\ttotal: 920ms\tremaining: 17.5s\n","[12:07:00] 199:\tlearn: 0.5103500\ttest: 0.5006084\tbest: 0.5006084 (199)\ttotal: 1.72s\tremaining: 15.5s\n","[12:07:00] 299:\tlearn: 0.5032400\ttest: 0.4947266\tbest: 0.4947266 (299)\ttotal: 2.42s\tremaining: 13.7s\n","[12:07:01] 399:\tlearn: 0.4984281\ttest: 0.4915502\tbest: 0.4915502 (399)\ttotal: 3.13s\tremaining: 12.5s\n","[12:07:02] 499:\tlearn: 0.4955406\ttest: 0.4894158\tbest: 0.4894158 (499)\ttotal: 3.82s\tremaining: 11.5s\n","[12:07:03] 599:\tlearn: 0.4928883\ttest: 0.4874990\tbest: 0.4874990 (599)\ttotal: 4.56s\tremaining: 10.6s\n","[12:07:03] 699:\tlearn: 0.4908529\ttest: 0.4860911\tbest: 0.4860873 (697)\ttotal: 5.27s\tremaining: 9.79s\n","[12:07:04] 799:\tlearn: 0.4890768\ttest: 0.4851408\tbest: 0.4851394 (789)\ttotal: 5.96s\tremaining: 8.94s\n","[12:07:05] 899:\tlearn: 0.4874309\ttest: 0.4843147\tbest: 0.4843084 (896)\ttotal: 6.68s\tremaining: 8.16s\n","[12:07:06] 999:\tlearn: 0.4865388\ttest: 0.4839954\tbest: 0.4839601 (979)\ttotal: 7.38s\tremaining: 7.38s\n","[12:07:06] 1099:\tlearn: 0.4854802\ttest: 0.4837207\tbest: 0.4837100 (1097)\ttotal: 8.08s\tremaining: 6.61s\n","[12:07:07] 1199:\tlearn: 0.4846195\ttest: 0.4834759\tbest: 0.4833264 (1193)\ttotal: 8.82s\tremaining: 5.88s\n","[12:07:08] 1299:\tlearn: 0.4835831\ttest: 0.4828609\tbest: 0.4828609 (1299)\ttotal: 9.52s\tremaining: 5.13s\n","[12:07:09] 1399:\tlearn: 0.4827808\ttest: 0.4828878\tbest: 0.4826588 (1346)\ttotal: 10.3s\tremaining: 4.39s\n","[12:07:09] 1499:\tlearn: 0.4817256\ttest: 0.4838736\tbest: 0.4826588 (1346)\ttotal: 11s\tremaining: 3.65s\n","[12:07:10] 1599:\tlearn: 0.4809835\ttest: 0.4843621\tbest: 0.4826588 (1346)\ttotal: 11.7s\tremaining: 2.92s\n","[12:07:10] Stopped by overfitting detector (300 iterations wait)\n","[12:07:10] bestTest = 0.4826588178\n","[12:07:10] bestIteration = 1346\n","[12:07:10] Shrink model to first 1347 iterations.\n","[12:07:10] \u001b[1mTrial 12\u001b[0m with hyperparameters {'max_depth': 4, 'nan_mode': 'Max', 'l2_leaf_reg': 1.3815073407783126e-08, 'min_data_in_leaf': 16} scored -0.4826595653170336 in 0:00:12.729174\n","[12:07:11] 0:\tlearn: 2.1677265\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.09ms\tremaining: 18.2s\n","[12:07:12] 99:\tlearn: 0.5233802\ttest: 0.5123769\tbest: 0.5123769 (99)\ttotal: 922ms\tremaining: 17.5s\n","[12:07:12] 199:\tlearn: 0.5103500\ttest: 0.5006084\tbest: 0.5006084 (199)\ttotal: 1.7s\tremaining: 15.3s\n","[12:07:13] 299:\tlearn: 0.5032400\ttest: 0.4947266\tbest: 0.4947266 (299)\ttotal: 2.43s\tremaining: 13.7s\n","[12:07:14] 399:\tlearn: 0.4984281\ttest: 0.4915502\tbest: 0.4915502 (399)\ttotal: 3.22s\tremaining: 12.9s\n","[12:07:15] 499:\tlearn: 0.4955406\ttest: 0.4894158\tbest: 0.4894158 (499)\ttotal: 3.96s\tremaining: 11.9s\n","[12:07:16] 599:\tlearn: 0.4928883\ttest: 0.4874990\tbest: 0.4874990 (599)\ttotal: 4.71s\tremaining: 11s\n","[12:07:16] 699:\tlearn: 0.4908529\ttest: 0.4860911\tbest: 0.4860873 (697)\ttotal: 5.39s\tremaining: 10s\n","[12:07:17] 799:\tlearn: 0.4890768\ttest: 0.4851408\tbest: 0.4851394 (789)\ttotal: 6.05s\tremaining: 9.08s\n","[12:07:18] 899:\tlearn: 0.4874309\ttest: 0.4843147\tbest: 0.4843084 (896)\ttotal: 6.75s\tremaining: 8.25s\n","[12:07:18] 999:\tlearn: 0.4865388\ttest: 0.4839954\tbest: 0.4839601 (979)\ttotal: 7.45s\tremaining: 7.45s\n","[12:07:19] 1099:\tlearn: 0.4854802\ttest: 0.4837207\tbest: 0.4837100 (1097)\ttotal: 8.14s\tremaining: 6.66s\n","[12:07:20] 1199:\tlearn: 0.4846195\ttest: 0.4834759\tbest: 0.4833264 (1193)\ttotal: 8.88s\tremaining: 5.92s\n","[12:07:21] 1299:\tlearn: 0.4835831\ttest: 0.4828609\tbest: 0.4828609 (1299)\ttotal: 9.59s\tremaining: 5.16s\n","[12:07:21] 1399:\tlearn: 0.4827808\ttest: 0.4828878\tbest: 0.4826588 (1346)\ttotal: 10.3s\tremaining: 4.4s\n","[12:07:22] 1499:\tlearn: 0.4817256\ttest: 0.4838736\tbest: 0.4826588 (1346)\ttotal: 11s\tremaining: 3.66s\n","[12:07:23] 1599:\tlearn: 0.4809835\ttest: 0.4843621\tbest: 0.4826588 (1346)\ttotal: 11.7s\tremaining: 2.93s\n","[12:07:23] Stopped by overfitting detector (300 iterations wait)\n","[12:07:23] bestTest = 0.4826588178\n","[12:07:23] bestIteration = 1346\n","[12:07:23] Shrink model to first 1347 iterations.\n","[12:07:23] \u001b[1mTrial 13\u001b[0m with hyperparameters {'max_depth': 4, 'nan_mode': 'Max', 'l2_leaf_reg': 1.3862271211272805e-08, 'min_data_in_leaf': 14} scored -0.4826595653170336 in 0:00:12.723237\n","[12:07:23] 0:\tlearn: 2.1677265\ttest: 2.1530705\tbest: 2.1530705 (0)\ttotal: 9.11ms\tremaining: 18.2s\n","[12:07:24] 99:\tlearn: 0.5233802\ttest: 0.5123769\tbest: 0.5123769 (99)\ttotal: 898ms\tremaining: 17.1s\n","[12:07:25] 199:\tlearn: 0.5103500\ttest: 0.5006084\tbest: 0.5006084 (199)\ttotal: 1.71s\tremaining: 15.4s\n","[12:07:26] 299:\tlearn: 0.5032400\ttest: 0.4947266\tbest: 0.4947266 (299)\ttotal: 2.41s\tremaining: 13.7s\n","[12:07:27] 399:\tlearn: 0.4984281\ttest: 0.4915502\tbest: 0.4915502 (399)\ttotal: 3.11s\tremaining: 12.5s\n","[12:07:27] 499:\tlearn: 0.4955406\ttest: 0.4894158\tbest: 0.4894158 (499)\ttotal: 3.84s\tremaining: 11.5s\n","[12:07:28] 599:\tlearn: 0.4928883\ttest: 0.4874990\tbest: 0.4874990 (599)\ttotal: 4.51s\tremaining: 10.5s\n","[12:07:29] 699:\tlearn: 0.4908404\ttest: 0.4862078\tbest: 0.4862078 (699)\ttotal: 5.22s\tremaining: 9.69s\n","[12:07:30] 799:\tlearn: 0.4891706\ttest: 0.4851415\tbest: 0.4851415 (799)\ttotal: 5.94s\tremaining: 8.91s\n","[12:07:30] 899:\tlearn: 0.4882911\ttest: 0.4851977\tbest: 0.4849759 (867)\ttotal: 6.64s\tremaining: 8.12s\n","[12:07:31] 999:\tlearn: 0.4867637\ttest: 0.4845286\tbest: 0.4845286 (999)\ttotal: 7.35s\tremaining: 7.35s\n","[12:07:32] 1099:\tlearn: 0.4857946\ttest: 0.4840344\tbest: 0.4840344 (1099)\ttotal: 8.06s\tremaining: 6.59s\n","[12:07:33] 1199:\tlearn: 0.4848311\ttest: 0.4838475\tbest: 0.4837825 (1141)\ttotal: 8.79s\tremaining: 5.86s\n","[12:07:33] 1299:\tlearn: 0.4839343\ttest: 0.4844097\tbest: 0.4837825 (1141)\ttotal: 9.5s\tremaining: 5.12s\n","[12:07:34] 1399:\tlearn: 0.4831983\ttest: 0.4841172\tbest: 0.4837825 (1141)\ttotal: 10.2s\tremaining: 4.39s\n","[12:07:34] Stopped by overfitting detector (300 iterations wait)\n","[12:07:34] bestTest = 0.4837825454\n","[12:07:34] bestIteration = 1141\n","[12:07:34] Shrink model to first 1142 iterations.\n","[12:07:34] \u001b[1mTrial 14\u001b[0m with hyperparameters {'max_depth': 4, 'nan_mode': 'Max', 'l2_leaf_reg': 0.12867346028747534, 'min_data_in_leaf': 13} scored -0.4837832680716448 in 0:00:11.211871\n","[12:07:35] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 10.4ms\tremaining: 20.8s\n","[12:07:35] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 804ms\tremaining: 15.3s\n","[12:07:36] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.56s\tremaining: 14s\n","[12:07:37] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.32s\tremaining: 13.2s\n","[12:07:38] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.08s\tremaining: 12.3s\n","[12:07:39] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.87s\tremaining: 11.6s\n","[12:07:39] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.63s\tremaining: 10.8s\n","[12:07:40] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.43s\tremaining: 10.1s\n","[12:07:41] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.24s\tremaining: 9.36s\n","[12:07:42] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 7.06s\tremaining: 8.63s\n","[12:07:43] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.83s\tremaining: 7.83s\n","[12:07:44] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.63s\tremaining: 7.06s\n","[12:07:44] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.42s\tremaining: 6.28s\n","[12:07:45] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.2s\tremaining: 5.51s\n","[12:07:46] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11s\tremaining: 4.73s\n","[12:07:47] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 12s\tremaining: 3.99s\n","[12:07:48] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.8s\tremaining: 3.21s\n","[12:07:49] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.7s\tremaining: 2.41s\n","[12:07:50] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.5s\tremaining: 1.61s\n","[12:07:51] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15.3s\tremaining: 806ms\n","[12:07:51] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 16.1s\tremaining: 0us\n","[12:07:51] bestTest = 0.4824646526\n","[12:07:51] bestIteration = 1811\n","[12:07:51] Shrink model to first 1812 iterations.\n","[12:07:51] \u001b[1mTrial 15\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 2.238936632044194e-06, 'min_data_in_leaf': 19} scored -0.48246547286748365 in 0:00:17.032765\n","[12:07:52] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 9.4ms\tremaining: 18.8s\n","[12:07:52] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 795ms\tremaining: 15.1s\n","[12:07:53] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.54s\tremaining: 13.9s\n","[12:07:54] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.29s\tremaining: 13s\n","[12:07:55] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.05s\tremaining: 12.2s\n","[12:07:56] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.83s\tremaining: 11.5s\n","[12:07:56] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.6s\tremaining: 10.7s\n","[12:07:57] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.4s\tremaining: 10s\n","[12:07:58] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.18s\tremaining: 9.27s\n","[12:07:59] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 6.98s\tremaining: 8.54s\n","[12:08:00] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.73s\tremaining: 7.73s\n","[12:08:00] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.51s\tremaining: 6.96s\n","[12:08:01] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.29s\tremaining: 6.19s\n","[12:08:02] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.1s\tremaining: 5.44s\n","[12:08:03] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 10.9s\tremaining: 4.67s\n","[12:08:04] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.7s\tremaining: 3.91s\n","[12:08:05] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.5s\tremaining: 3.12s\n","[12:08:05] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.3s\tremaining: 2.34s\n","[12:08:06] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.1s\tremaining: 1.57s\n","[12:08:07] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 14.9s\tremaining: 784ms\n","[12:08:08] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 15.7s\tremaining: 0us\n","[12:08:08] bestTest = 0.4824646526\n","[12:08:08] bestIteration = 1811\n","[12:08:08] Shrink model to first 1812 iterations.\n","[12:08:08] \u001b[1mTrial 16\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 2.3341255208652357e-06, 'min_data_in_leaf': 19} scored -0.48246547286748365 in 0:00:16.555955\n","[12:08:08] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 8.89ms\tremaining: 17.8s\n","[12:08:09] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 799ms\tremaining: 15.2s\n","[12:08:10] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.56s\tremaining: 14s\n","[12:08:11] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.33s\tremaining: 13.2s\n","[12:08:11] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.1s\tremaining: 12.4s\n","[12:08:12] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.88s\tremaining: 11.6s\n","[12:08:13] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.64s\tremaining: 10.8s\n","[12:08:14] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.4s\tremaining: 10s\n","[12:08:15] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.21s\tremaining: 9.32s\n","[12:08:15] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 7.01s\tremaining: 8.56s\n","[12:08:16] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.8s\tremaining: 7.8s\n","[12:08:17] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.56s\tremaining: 7.01s\n","[12:08:18] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.35s\tremaining: 6.23s\n","[12:08:19] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.1s\tremaining: 5.46s\n","[12:08:20] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11s\tremaining: 4.71s\n","[12:08:20] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.8s\tremaining: 3.94s\n","[12:08:21] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.6s\tremaining: 3.15s\n","[12:08:22] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.4s\tremaining: 2.37s\n","[12:08:23] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.2s\tremaining: 1.58s\n","[12:08:24] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15s\tremaining: 790ms\n","[12:08:25] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 15.8s\tremaining: 0us\n","[12:08:25] bestTest = 0.4824646526\n","[12:08:25] bestIteration = 1811\n","[12:08:25] Shrink model to first 1812 iterations.\n","[12:08:25] \u001b[1mTrial 17\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 1.9086775629618743e-06, 'min_data_in_leaf': 20} scored -0.48246547286748365 in 0:00:16.598813\n","[12:08:25] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 9.55ms\tremaining: 19.1s\n","[12:08:26] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 822ms\tremaining: 15.6s\n","[12:08:26] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.63s\tremaining: 14.7s\n","[12:08:27] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.42s\tremaining: 13.7s\n","[12:08:28] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.19s\tremaining: 12.7s\n","[12:08:29] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.94s\tremaining: 11.8s\n","[12:08:30] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.69s\tremaining: 10.9s\n","[12:08:31] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.47s\tremaining: 10.2s\n","[12:08:31] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.26s\tremaining: 9.39s\n","[12:08:32] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 7.04s\tremaining: 8.6s\n","[12:08:33] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.8s\tremaining: 7.8s\n","[12:08:34] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.58s\tremaining: 7.02s\n","[12:08:35] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.39s\tremaining: 6.26s\n","[12:08:35] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.2s\tremaining: 5.5s\n","[12:08:36] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11s\tremaining: 4.72s\n","[12:08:37] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.8s\tremaining: 3.94s\n","[12:08:38] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.6s\tremaining: 3.15s\n","[12:08:39] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.4s\tremaining: 2.37s\n","[12:08:40] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.3s\tremaining: 1.59s\n","[12:08:41] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15.1s\tremaining: 795ms\n","[12:08:41] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 15.9s\tremaining: 0us\n","[12:08:41] bestTest = 0.4824646526\n","[12:08:41] bestIteration = 1811\n","[12:08:41] Shrink model to first 1812 iterations.\n","[12:08:41] \u001b[1mTrial 18\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 7.56056282296409e-07, 'min_data_in_leaf': 18} scored -0.48246547286748365 in 0:00:16.798384\n","[12:08:42] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 10.2ms\tremaining: 20.4s\n","[12:08:42] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 794ms\tremaining: 15.1s\n","[12:08:43] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.56s\tremaining: 14.1s\n","[12:08:44] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.36s\tremaining: 13.4s\n","[12:08:45] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.1s\tremaining: 12.4s\n","[12:08:46] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.85s\tremaining: 11.6s\n","[12:08:46] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.61s\tremaining: 10.8s\n","[12:08:47] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.37s\tremaining: 9.97s\n","[12:08:48] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.2s\tremaining: 9.29s\n","[12:08:49] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 6.98s\tremaining: 8.53s\n","[12:08:50] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.77s\tremaining: 7.77s\n","[12:08:51] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.58s\tremaining: 7.02s\n","[12:08:51] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.35s\tremaining: 6.23s\n","[12:08:52] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.2s\tremaining: 5.51s\n","[12:08:53] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11.1s\tremaining: 4.74s\n","[12:08:54] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.8s\tremaining: 3.95s\n","[12:08:55] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.6s\tremaining: 3.16s\n","[12:08:56] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.4s\tremaining: 2.37s\n","[12:08:56] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.2s\tremaining: 1.58s\n","[12:08:57] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15s\tremaining: 789ms\n","[12:08:58] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 15.8s\tremaining: 0us\n","[12:08:58] bestTest = 0.4824646526\n","[12:08:58] bestIteration = 1811\n","[12:08:58] Shrink model to first 1812 iterations.\n","[12:08:58] \u001b[1mTrial 19\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 1.41312777974931e-07, 'min_data_in_leaf': 17} scored -0.48246547286748365 in 0:00:16.637526\n","[12:08:58] 0:\tlearn: 2.1698661\ttest: 2.1554650\tbest: 2.1554650 (0)\ttotal: 12.8ms\tremaining: 25.5s\n","[12:08:59] 99:\tlearn: 0.5041294\ttest: 0.4979055\tbest: 0.4979055 (99)\ttotal: 1.04s\tremaining: 19.8s\n","[12:09:00] 199:\tlearn: 0.4896569\ttest: 0.4893669\tbest: 0.4893669 (199)\ttotal: 2.05s\tremaining: 18.4s\n","[12:09:01] 299:\tlearn: 0.4795697\ttest: 0.4855189\tbest: 0.4855189 (299)\ttotal: 3.07s\tremaining: 17.4s\n","[12:09:03] 399:\tlearn: 0.4719033\ttest: 0.4831956\tbest: 0.4831878 (396)\ttotal: 4.16s\tremaining: 16.6s\n","[12:09:04] 499:\tlearn: 0.4662442\ttest: 0.4835189\tbest: 0.4831875 (404)\ttotal: 5.22s\tremaining: 15.7s\n","[12:09:05] 599:\tlearn: 0.4616382\ttest: 0.4859876\tbest: 0.4831875 (404)\ttotal: 6.27s\tremaining: 14.6s\n","[12:09:06] 699:\tlearn: 0.4575575\ttest: 0.4862313\tbest: 0.4831875 (404)\ttotal: 7.31s\tremaining: 13.6s\n","[12:09:06] Stopped by overfitting detector (300 iterations wait)\n","[12:09:06] bestTest = 0.483187469\n","[12:09:06] bestIteration = 404\n","[12:09:06] Shrink model to first 405 iterations.\n","[12:09:06] \u001b[1mTrial 20\u001b[0m with hyperparameters {'max_depth': 7, 'nan_mode': 'Max', 'l2_leaf_reg': 1.7754789920320172e-07, 'min_data_in_leaf': 17} scored -0.4831882169891197 in 0:00:07.769865\n","[12:09:06] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 10.7ms\tremaining: 21.4s\n","[12:09:07] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 820ms\tremaining: 15.6s\n","[12:09:08] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.59s\tremaining: 14.4s\n","[12:09:08] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.35s\tremaining: 13.3s\n","[12:09:09] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.09s\tremaining: 12.4s\n","[12:09:10] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.86s\tremaining: 11.6s\n","[12:09:11] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.62s\tremaining: 10.8s\n","[12:09:12] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.42s\tremaining: 10.1s\n","[12:09:12] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.19s\tremaining: 9.28s\n","[12:09:13] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 6.96s\tremaining: 8.51s\n","[12:09:14] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.75s\tremaining: 7.75s\n","[12:09:15] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.53s\tremaining: 6.98s\n","[12:09:16] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.29s\tremaining: 6.19s\n","[12:09:17] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.1s\tremaining: 5.42s\n","[12:09:17] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 10.9s\tremaining: 4.66s\n","[12:09:18] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.6s\tremaining: 3.88s\n","[12:09:19] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.4s\tremaining: 3.11s\n","[12:09:20] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.2s\tremaining: 2.33s\n","[12:09:21] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14s\tremaining: 1.56s\n","[12:09:22] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 14.8s\tremaining: 780ms\n","[12:09:22] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 15.7s\tremaining: 0us\n","[12:09:22] bestTest = 0.4824646526\n","[12:09:22] bestIteration = 1811\n","[12:09:22] Shrink model to first 1812 iterations.\n","[12:09:22] \u001b[1mTrial 21\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 1.9669097330232616e-05, 'min_data_in_leaf': 18} scored -0.48246547286748365 in 0:00:16.535382\n","[12:09:23] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 10ms\tremaining: 20.1s\n","[12:09:23] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 819ms\tremaining: 15.6s\n","[12:09:24] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.56s\tremaining: 14.1s\n","[12:09:25] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.4s\tremaining: 13.6s\n","[12:09:26] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.2s\tremaining: 12.8s\n","[12:09:27] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.99s\tremaining: 12s\n","[12:09:28] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.79s\tremaining: 11.2s\n","[12:09:28] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.58s\tremaining: 10.4s\n","[12:09:29] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.41s\tremaining: 9.62s\n","[12:09:30] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 7.21s\tremaining: 8.81s\n","[12:09:31] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 8s\tremaining: 8s\n","[12:09:32] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.85s\tremaining: 7.24s\n","[12:09:33] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.63s\tremaining: 6.42s\n","[12:09:33] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.4s\tremaining: 5.62s\n","[12:09:34] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11.3s\tremaining: 4.84s\n","[12:09:35] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 12.1s\tremaining: 4.03s\n","[12:09:36] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.8s\tremaining: 3.21s\n","[12:09:37] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.6s\tremaining: 2.41s\n","[12:09:38] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.4s\tremaining: 1.6s\n","[12:09:38] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15.2s\tremaining: 801ms\n","[12:09:39] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 16s\tremaining: 0us\n","[12:09:39] bestTest = 0.4824646526\n","[12:09:39] bestIteration = 1811\n","[12:09:39] Shrink model to first 1812 iterations.\n","[12:09:39] \u001b[1mTrial 22\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 2.8948014169502267e-05, 'min_data_in_leaf': 17} scored -0.48246547286748365 in 0:00:16.843511\n","[12:09:39] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 10.6ms\tremaining: 21.2s\n","[12:09:40] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 814ms\tremaining: 15.5s\n","[12:09:41] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.6s\tremaining: 14.4s\n","[12:09:42] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.39s\tremaining: 13.5s\n","[12:09:43] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.17s\tremaining: 12.7s\n","[12:09:44] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.94s\tremaining: 11.8s\n","[12:09:44] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.73s\tremaining: 11s\n","[12:09:45] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.52s\tremaining: 10.3s\n","[12:09:46] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.32s\tremaining: 9.47s\n","[12:09:47] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 7.1s\tremaining: 8.67s\n","[12:09:48] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.87s\tremaining: 7.87s\n","[12:09:48] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.65s\tremaining: 7.07s\n","[12:09:49] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.43s\tremaining: 6.29s\n","[12:09:50] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.3s\tremaining: 5.53s\n","[12:09:51] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11.1s\tremaining: 4.75s\n","[12:09:52] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.9s\tremaining: 3.97s\n","[12:09:53] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.7s\tremaining: 3.17s\n","[12:09:53] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 13.5s\tremaining: 2.38s\n","[12:09:54] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.3s\tremaining: 1.59s\n","[12:09:55] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15.1s\tremaining: 794ms\n","[12:09:56] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 15.8s\tremaining: 0us\n","[12:09:56] bestTest = 0.4824646526\n","[12:09:56] bestIteration = 1811\n","[12:09:56] Shrink model to first 1812 iterations.\n","[12:09:56] \u001b[1mTrial 23\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 2.2848312840552222e-07, 'min_data_in_leaf': 12} scored -0.48246547286748365 in 0:00:16.697173\n","[12:09:56] 0:\tlearn: 2.1688168\ttest: 2.1544035\tbest: 2.1544035 (0)\ttotal: 12ms\tremaining: 24s\n","[12:09:57] 99:\tlearn: 0.5101596\ttest: 0.5008584\tbest: 0.5007533 (97)\ttotal: 907ms\tremaining: 17.2s\n","[12:09:58] 199:\tlearn: 0.4957684\ttest: 0.4902583\tbest: 0.4902398 (197)\ttotal: 1.81s\tremaining: 16.3s\n","[12:09:59] 299:\tlearn: 0.4875843\ttest: 0.4873339\tbest: 0.4873299 (298)\ttotal: 2.73s\tremaining: 15.5s\n","[12:10:00] 399:\tlearn: 0.4824203\ttest: 0.4854949\tbest: 0.4854949 (399)\ttotal: 3.63s\tremaining: 14.5s\n","[12:10:01] 499:\tlearn: 0.4783011\ttest: 0.4850727\tbest: 0.4849208 (461)\ttotal: 4.54s\tremaining: 13.6s\n","[12:10:02] 599:\tlearn: 0.4742069\ttest: 0.4843100\tbest: 0.4843076 (592)\ttotal: 5.45s\tremaining: 12.7s\n","[12:10:03] 699:\tlearn: 0.4713216\ttest: 0.4845019\tbest: 0.4841443 (608)\ttotal: 6.42s\tremaining: 11.9s\n","[12:10:04] 799:\tlearn: 0.4687111\ttest: 0.4847689\tbest: 0.4841443 (608)\ttotal: 7.3s\tremaining: 10.9s\n","[12:10:05] 899:\tlearn: 0.4655522\ttest: 0.4846853\tbest: 0.4841443 (608)\ttotal: 8.24s\tremaining: 10.1s\n","[12:10:05] Stopped by overfitting detector (300 iterations wait)\n","[12:10:05] bestTest = 0.4841442973\n","[12:10:05] bestIteration = 608\n","[12:10:05] Shrink model to first 609 iterations.\n","[12:10:05] \u001b[1mTrial 24\u001b[0m with hyperparameters {'max_depth': 6, 'nan_mode': 'Max', 'l2_leaf_reg': 1.2534688755926362e-07, 'min_data_in_leaf': 11} scored -0.4841450411623014 in 0:00:08.800237\n","[12:10:05] 0:\tlearn: 2.1687491\ttest: 2.1541678\tbest: 2.1541678 (0)\ttotal: 11.3ms\tremaining: 22.7s\n","[12:10:06] 99:\tlearn: 0.5143391\ttest: 0.5049223\tbest: 0.5049223 (99)\ttotal: 792ms\tremaining: 15s\n","[12:10:07] 199:\tlearn: 0.5015720\ttest: 0.4954717\tbest: 0.4949265 (198)\ttotal: 1.57s\tremaining: 14.1s\n","[12:10:07] 299:\tlearn: 0.4946520\ttest: 0.4913143\tbest: 0.4913143 (299)\ttotal: 2.32s\tremaining: 13.2s\n","[12:10:08] 399:\tlearn: 0.4901233\ttest: 0.4885204\tbest: 0.4885204 (399)\ttotal: 3.06s\tremaining: 12.3s\n","[12:10:09] 499:\tlearn: 0.4868040\ttest: 0.4866724\tbest: 0.4866395 (491)\ttotal: 3.84s\tremaining: 11.5s\n","[12:10:10] 599:\tlearn: 0.4841547\ttest: 0.4856474\tbest: 0.4856468 (597)\ttotal: 4.61s\tremaining: 10.8s\n","[12:10:11] 699:\tlearn: 0.4818755\ttest: 0.4848884\tbest: 0.4848884 (699)\ttotal: 5.38s\tremaining: 9.99s\n","[12:10:11] 799:\tlearn: 0.4792616\ttest: 0.4846707\tbest: 0.4846707 (799)\ttotal: 6.18s\tremaining: 9.26s\n","[12:10:12] 899:\tlearn: 0.4771271\ttest: 0.4847623\tbest: 0.4844642 (810)\ttotal: 6.96s\tremaining: 8.51s\n","[12:10:13] 999:\tlearn: 0.4755356\ttest: 0.4845767\tbest: 0.4844642 (810)\ttotal: 7.74s\tremaining: 7.74s\n","[12:10:14] 1099:\tlearn: 0.4742169\ttest: 0.4848995\tbest: 0.4844491 (1014)\ttotal: 8.54s\tremaining: 6.99s\n","[12:10:15] 1199:\tlearn: 0.4723144\ttest: 0.4844164\tbest: 0.4843018 (1185)\ttotal: 9.3s\tremaining: 6.2s\n","[12:10:16] 1299:\tlearn: 0.4707318\ttest: 0.4839955\tbest: 0.4839954 (1298)\ttotal: 10.2s\tremaining: 5.48s\n","[12:10:16] 1399:\tlearn: 0.4694091\ttest: 0.4836554\tbest: 0.4835461 (1369)\ttotal: 11s\tremaining: 4.73s\n","[12:10:17] 1499:\tlearn: 0.4683123\ttest: 0.4833897\tbest: 0.4833890 (1493)\ttotal: 11.9s\tremaining: 3.96s\n","[12:10:18] 1599:\tlearn: 0.4669262\ttest: 0.4831299\tbest: 0.4831108 (1594)\ttotal: 12.8s\tremaining: 3.19s\n","[12:10:20] 1699:\tlearn: 0.4657653\ttest: 0.4829451\tbest: 0.4829165 (1671)\ttotal: 14s\tremaining: 2.47s\n","[12:10:20] 1799:\tlearn: 0.4644975\ttest: 0.4825362\tbest: 0.4825291 (1797)\ttotal: 14.9s\tremaining: 1.65s\n","[12:10:21] 1899:\tlearn: 0.4632325\ttest: 0.4828039\tbest: 0.4824647 (1811)\ttotal: 15.7s\tremaining: 825ms\n","[12:10:22] 1999:\tlearn: 0.4620700\ttest: 0.4825494\tbest: 0.4824647 (1811)\ttotal: 16.5s\tremaining: 0us\n","[12:10:22] bestTest = 0.4824646526\n","[12:10:22] bestIteration = 1811\n","[12:10:22] Shrink model to first 1812 iterations.\n","[12:10:22] \u001b[1mTrial 25\u001b[0m with hyperparameters {'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 3.2799031509032256e-05, 'min_data_in_leaf': 17} scored -0.48246547286748365 in 0:00:17.379548\n","[12:10:22] Hyperparameters optimization for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m completed\n","[12:10:22] The set of hyperparameters \u001b[1m{'max_depth': 5, 'nan_mode': 'Max', 'l2_leaf_reg': 2.238936632044194e-06, 'min_data_in_leaf': 19}\u001b[0m\n"," achieve -0.4825 mae\n","[12:10:22] Start fitting \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m ...\n","[12:10:22] ===== Start working with \u001b[1mfold 0\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m =====\n","[12:10:22] 0:\tlearn: 2.2146247\ttest: 2.1998251\tbest: 2.1998251 (0)\ttotal: 10.5ms\tremaining: 31.4s\n","[12:10:23] 99:\tlearn: 0.5362655\ttest: 0.5242917\tbest: 0.5242917 (99)\ttotal: 832ms\tremaining: 24.1s\n","[12:10:24] 199:\tlearn: 0.5107199\ttest: 0.5014256\tbest: 0.5014256 (199)\ttotal: 1.61s\tremaining: 22.6s\n","[12:10:25] 299:\tlearn: 0.5035674\ttest: 0.4959155\tbest: 0.4959155 (299)\ttotal: 2.38s\tremaining: 21.4s\n","[12:10:26] 399:\tlearn: 0.4990067\ttest: 0.4928726\tbest: 0.4928726 (399)\ttotal: 3.12s\tremaining: 20.3s\n","[12:10:26] 499:\tlearn: 0.4950729\ttest: 0.4906013\tbest: 0.4905393 (486)\ttotal: 3.89s\tremaining: 19.4s\n","[12:10:27] 599:\tlearn: 0.4923459\ttest: 0.4891017\tbest: 0.4891017 (599)\ttotal: 4.71s\tremaining: 18.8s\n","[12:10:28] 699:\tlearn: 0.4897807\ttest: 0.4878686\tbest: 0.4878686 (699)\ttotal: 5.48s\tremaining: 18s\n","[12:10:29] 799:\tlearn: 0.4874452\ttest: 0.4863417\tbest: 0.4863086 (796)\ttotal: 6.25s\tremaining: 17.2s\n","[12:10:30] 899:\tlearn: 0.4858905\ttest: 0.4861057\tbest: 0.4858861 (880)\ttotal: 7.02s\tremaining: 16.4s\n","[12:10:30] Stopped by overfitting detector (100 iterations wait)\n","[12:10:30] bestTest = 0.4858860979\n","[12:10:30] bestIteration = 880\n","[12:10:30] Shrink model to first 881 iterations.\n","[12:10:30] ===== Start working with \u001b[1mfold 1\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m =====\n","[12:10:30] 0:\tlearn: 2.2127218\ttest: 2.2092807\tbest: 2.2092807 (0)\ttotal: 10.7ms\tremaining: 32.1s\n","[12:10:32] 99:\tlearn: 0.5357413\ttest: 0.5290981\tbest: 0.5290981 (99)\ttotal: 1.11s\tremaining: 32.3s\n","[12:10:32] 199:\tlearn: 0.5104249\ttest: 0.5059904\tbest: 0.5059904 (199)\ttotal: 2.03s\tremaining: 28.4s\n","[12:10:33] 299:\tlearn: 0.5036275\ttest: 0.5005432\tbest: 0.5005432 (299)\ttotal: 2.77s\tremaining: 24.9s\n","[12:10:34] 399:\tlearn: 0.4977911\ttest: 0.4957297\tbest: 0.4957297 (399)\ttotal: 3.51s\tremaining: 22.8s\n","[12:10:35] 499:\tlearn: 0.4936034\ttest: 0.4931898\tbest: 0.4931898 (499)\ttotal: 4.27s\tremaining: 21.3s\n","[12:10:36] 599:\tlearn: 0.4906737\ttest: 0.4915225\tbest: 0.4915225 (599)\ttotal: 5.02s\tremaining: 20.1s\n","[12:10:36] 699:\tlearn: 0.4888049\ttest: 0.4906498\tbest: 0.4906482 (696)\ttotal: 5.76s\tremaining: 18.9s\n","[12:10:37] 799:\tlearn: 0.4866248\ttest: 0.4895151\tbest: 0.4895151 (799)\ttotal: 6.53s\tremaining: 18s\n","[12:10:38] 899:\tlearn: 0.4848977\ttest: 0.4888968\tbest: 0.4888968 (899)\ttotal: 7.3s\tremaining: 17s\n","[12:10:39] 999:\tlearn: 0.4834442\ttest: 0.4884257\tbest: 0.4884257 (999)\ttotal: 8.07s\tremaining: 16.1s\n","[12:10:40] 1099:\tlearn: 0.4818500\ttest: 0.4878494\tbest: 0.4878494 (1099)\ttotal: 8.83s\tremaining: 15.3s\n","[12:10:40] 1199:\tlearn: 0.4806257\ttest: 0.4874498\tbest: 0.4874498 (1199)\ttotal: 9.61s\tremaining: 14.4s\n","[12:10:41] 1299:\tlearn: 0.4793709\ttest: 0.4870439\tbest: 0.4870439 (1299)\ttotal: 10.4s\tremaining: 13.6s\n","[12:10:42] 1399:\tlearn: 0.4782466\ttest: 0.4867017\tbest: 0.4866954 (1398)\ttotal: 11.2s\tremaining: 12.8s\n","[12:10:43] 1499:\tlearn: 0.4771049\ttest: 0.4864397\tbest: 0.4864324 (1496)\ttotal: 12s\tremaining: 12s\n","[12:10:44] 1599:\tlearn: 0.4762142\ttest: 0.4861944\tbest: 0.4861935 (1598)\ttotal: 12.8s\tremaining: 11.2s\n","[12:10:45] 1699:\tlearn: 0.4752021\ttest: 0.4859181\tbest: 0.4859181 (1699)\ttotal: 13.6s\tremaining: 10.4s\n","[12:10:45] 1799:\tlearn: 0.4744174\ttest: 0.4858708\tbest: 0.4858596 (1795)\ttotal: 14.3s\tremaining: 9.55s\n","[12:10:46] 1899:\tlearn: 0.4736156\ttest: 0.4857982\tbest: 0.4857550 (1877)\ttotal: 15.1s\tremaining: 8.72s\n","[12:10:47] 1999:\tlearn: 0.4725219\ttest: 0.4855576\tbest: 0.4855555 (1997)\ttotal: 15.8s\tremaining: 7.92s\n","[12:10:48] 2099:\tlearn: 0.4716579\ttest: 0.4854488\tbest: 0.4854389 (2036)\ttotal: 16.6s\tremaining: 7.12s\n","[12:10:49] 2199:\tlearn: 0.4708838\ttest: 0.4853897\tbest: 0.4853856 (2154)\ttotal: 17.4s\tremaining: 6.33s\n","[12:10:49] 2299:\tlearn: 0.4701825\ttest: 0.4853383\tbest: 0.4853354 (2298)\ttotal: 18.2s\tremaining: 5.53s\n","[12:10:50] 2399:\tlearn: 0.4694050\ttest: 0.4853166\tbest: 0.4852971 (2321)\ttotal: 18.9s\tremaining: 4.74s\n","[12:10:50] Stopped by overfitting detector (100 iterations wait)\n","[12:10:50] bestTest = 0.485297132\n","[12:10:50] bestIteration = 2321\n","[12:10:50] Shrink model to first 2322 iterations.\n","[12:10:50] ===== Start working with \u001b[1mfold 2\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m =====\n","[12:10:50] 0:\tlearn: 2.2100911\ttest: 2.2196110\tbest: 2.2196110 (0)\ttotal: 9.58ms\tremaining: 28.7s\n","[12:10:51] 99:\tlearn: 0.5316663\ttest: 0.5460206\tbest: 0.5460206 (99)\ttotal: 818ms\tremaining: 23.7s\n","[12:10:52] 199:\tlearn: 0.5052554\ttest: 0.5215941\tbest: 0.5215941 (199)\ttotal: 1.6s\tremaining: 22.4s\n","[12:10:53] 299:\tlearn: 0.4980044\ttest: 0.5166051\tbest: 0.5166051 (299)\ttotal: 2.38s\tremaining: 21.5s\n","[12:10:54] 399:\tlearn: 0.4927191\ttest: 0.5132271\tbest: 0.5132271 (399)\ttotal: 3.1s\tremaining: 20.2s\n","[12:10:54] 499:\tlearn: 0.4884843\ttest: 0.5109684\tbest: 0.5108756 (484)\ttotal: 3.84s\tremaining: 19.2s\n","[12:10:55] 599:\tlearn: 0.4852426\ttest: 0.5097315\tbest: 0.5097315 (599)\ttotal: 4.61s\tremaining: 18.4s\n","[12:10:56] 699:\tlearn: 0.4829473\ttest: 0.5087535\tbest: 0.5087535 (699)\ttotal: 5.36s\tremaining: 17.6s\n","[12:10:57] 799:\tlearn: 0.4805440\ttest: 0.5074639\tbest: 0.5074639 (799)\ttotal: 6.16s\tremaining: 16.9s\n","[12:10:58] 899:\tlearn: 0.4786451\ttest: 0.5068835\tbest: 0.5068087 (887)\ttotal: 6.95s\tremaining: 16.2s\n","[12:10:59] 999:\tlearn: 0.4771994\ttest: 0.5066887\tbest: 0.5066502 (990)\ttotal: 7.71s\tremaining: 15.4s\n","[12:10:59] Stopped by overfitting detector (100 iterations wait)\n","[12:10:59] bestTest = 0.5066501971\n","[12:10:59] bestIteration = 990\n","[12:10:59] Shrink model to first 991 iterations.\n","[12:10:59] ===== Start working with \u001b[1mfold 3\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m =====\n","[12:10:59] 0:\tlearn: 2.2154478\ttest: 2.1963551\tbest: 2.1963551 (0)\ttotal: 12ms\tremaining: 36.1s\n","[12:11:00] 99:\tlearn: 0.5342412\ttest: 0.5326381\tbest: 0.5326381 (99)\ttotal: 864ms\tremaining: 25.1s\n","[12:11:01] 199:\tlearn: 0.5092506\ttest: 0.5083060\tbest: 0.5083060 (199)\ttotal: 1.64s\tremaining: 23s\n","[12:11:02] 299:\tlearn: 0.5018570\ttest: 0.5021473\tbest: 0.5021473 (299)\ttotal: 2.4s\tremaining: 21.6s\n","[12:11:03] 399:\tlearn: 0.4960248\ttest: 0.4972912\tbest: 0.4972912 (399)\ttotal: 3.19s\tremaining: 20.7s\n","[12:11:04] 499:\tlearn: 0.4914359\ttest: 0.4939901\tbest: 0.4939887 (498)\ttotal: 4.07s\tremaining: 20.4s\n","[12:11:04] 599:\tlearn: 0.4889140\ttest: 0.4926567\tbest: 0.4926567 (599)\ttotal: 4.85s\tremaining: 19.4s\n","[12:11:05] 699:\tlearn: 0.4867101\ttest: 0.4914365\tbest: 0.4914323 (694)\ttotal: 5.58s\tremaining: 18.3s\n","[12:11:06] 799:\tlearn: 0.4849528\ttest: 0.4905853\tbest: 0.4905853 (799)\ttotal: 6.36s\tremaining: 17.5s\n","[12:11:07] 899:\tlearn: 0.4835859\ttest: 0.4900374\tbest: 0.4900374 (899)\ttotal: 7.11s\tremaining: 16.6s\n","[12:11:08] 999:\tlearn: 0.4823184\ttest: 0.4895847\tbest: 0.4895804 (995)\ttotal: 7.88s\tremaining: 15.8s\n","[12:11:08] 1099:\tlearn: 0.4811752\ttest: 0.4892105\tbest: 0.4892105 (1099)\ttotal: 8.63s\tremaining: 14.9s\n","[12:11:09] 1199:\tlearn: 0.4798084\ttest: 0.4886570\tbest: 0.4886512 (1196)\ttotal: 9.41s\tremaining: 14.1s\n","[12:11:10] 1299:\tlearn: 0.4786531\ttest: 0.4883486\tbest: 0.4883485 (1293)\ttotal: 10.2s\tremaining: 13.3s\n","[12:11:11] 1399:\tlearn: 0.4776484\ttest: 0.4880140\tbest: 0.4880118 (1398)\ttotal: 11s\tremaining: 12.6s\n","[12:11:12] 1499:\tlearn: 0.4767691\ttest: 0.4877730\tbest: 0.4877730 (1499)\ttotal: 11.8s\tremaining: 11.8s\n","[12:11:12] 1599:\tlearn: 0.4758844\ttest: 0.4874397\tbest: 0.4874397 (1599)\ttotal: 12.5s\tremaining: 11s\n","[12:11:13] 1699:\tlearn: 0.4748706\ttest: 0.4870833\tbest: 0.4870817 (1697)\ttotal: 13.3s\tremaining: 10.2s\n","[12:11:14] 1799:\tlearn: 0.4741012\ttest: 0.4869707\tbest: 0.4869635 (1770)\ttotal: 14.1s\tremaining: 9.37s\n","[12:11:15] 1899:\tlearn: 0.4734530\ttest: 0.4867956\tbest: 0.4867956 (1899)\ttotal: 14.8s\tremaining: 8.58s\n","[12:11:16] 1999:\tlearn: 0.4726258\ttest: 0.4865902\tbest: 0.4865902 (1999)\ttotal: 15.6s\tremaining: 7.79s\n","[12:11:16] 2099:\tlearn: 0.4717941\ttest: 0.4863450\tbest: 0.4863450 (2099)\ttotal: 16.3s\tremaining: 7.01s\n","[12:11:17] 2199:\tlearn: 0.4711143\ttest: 0.4863203\tbest: 0.4863146 (2194)\ttotal: 17.1s\tremaining: 6.23s\n","[12:11:18] 2299:\tlearn: 0.4703270\ttest: 0.4861688\tbest: 0.4861665 (2296)\ttotal: 17.9s\tremaining: 5.45s\n","[12:11:19] 2399:\tlearn: 0.4697892\ttest: 0.4860418\tbest: 0.4860367 (2398)\ttotal: 18.7s\tremaining: 4.67s\n","[12:11:20] 2499:\tlearn: 0.4692571\ttest: 0.4859861\tbest: 0.4859728 (2495)\ttotal: 19.4s\tremaining: 3.88s\n","[12:11:20] 2599:\tlearn: 0.4686865\ttest: 0.4859234\tbest: 0.4859065 (2593)\ttotal: 20.2s\tremaining: 3.11s\n","[12:11:21] 2699:\tlearn: 0.4679597\ttest: 0.4858605\tbest: 0.4858601 (2697)\ttotal: 21s\tremaining: 2.33s\n","[12:11:22] 2799:\tlearn: 0.4672786\ttest: 0.4858027\tbest: 0.4857940 (2784)\ttotal: 21.9s\tremaining: 1.56s\n","[12:11:23] 2899:\tlearn: 0.4666012\ttest: 0.4857162\tbest: 0.4857052 (2883)\ttotal: 22.6s\tremaining: 781ms\n","[12:11:24] Stopped by overfitting detector (100 iterations wait)\n","[12:11:24] bestTest = 0.4857051619\n","[12:11:24] bestIteration = 2883\n","[12:11:24] Shrink model to first 2884 iterations.\n","[12:11:24] ===== Start working with \u001b[1mfold 4\u001b[0m for \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m =====\n","[12:11:24] 0:\tlearn: 2.2058406\ttest: 2.2336630\tbest: 2.2336630 (0)\ttotal: 12.4ms\tremaining: 37.3s\n","[12:11:25] 99:\tlearn: 0.5323612\ttest: 0.5457618\tbest: 0.5457618 (99)\ttotal: 819ms\tremaining: 23.8s\n","[12:11:25] 199:\tlearn: 0.5068679\ttest: 0.5216544\tbest: 0.5216544 (199)\ttotal: 1.6s\tremaining: 22.4s\n","[12:11:26] 299:\tlearn: 0.4985958\ttest: 0.5153452\tbest: 0.5153452 (299)\ttotal: 2.36s\tremaining: 21.2s\n","[12:11:27] 399:\tlearn: 0.4923422\ttest: 0.5111849\tbest: 0.5111849 (399)\ttotal: 3.11s\tremaining: 20.2s\n","[12:11:28] 499:\tlearn: 0.4876773\ttest: 0.5082559\tbest: 0.5082559 (499)\ttotal: 3.9s\tremaining: 19.5s\n","[12:11:29] 599:\tlearn: 0.4849986\ttest: 0.5068808\tbest: 0.5068749 (598)\ttotal: 4.64s\tremaining: 18.6s\n","[12:11:29] 699:\tlearn: 0.4829545\ttest: 0.5058469\tbest: 0.5058469 (699)\ttotal: 5.4s\tremaining: 17.7s\n","[12:11:30] 799:\tlearn: 0.4813130\ttest: 0.5053415\tbest: 0.5053263 (792)\ttotal: 6.16s\tremaining: 16.9s\n","[12:11:31] 899:\tlearn: 0.4798456\ttest: 0.5048138\tbest: 0.5048121 (898)\ttotal: 6.9s\tremaining: 16.1s\n","[12:11:32] 999:\tlearn: 0.4787060\ttest: 0.5045563\tbest: 0.5045551 (997)\ttotal: 7.67s\tremaining: 15.3s\n","[12:11:33] 1099:\tlearn: 0.4772272\ttest: 0.5039379\tbest: 0.5039361 (1098)\ttotal: 8.49s\tremaining: 14.7s\n","[12:11:34] 1199:\tlearn: 0.4760100\ttest: 0.5035561\tbest: 0.5035458 (1187)\ttotal: 9.28s\tremaining: 13.9s\n","[12:11:34] 1299:\tlearn: 0.4745580\ttest: 0.5029502\tbest: 0.5029502 (1299)\ttotal: 10.1s\tremaining: 13.2s\n","[12:11:35] 1399:\tlearn: 0.4734831\ttest: 0.5026959\tbest: 0.5026934 (1395)\ttotal: 10.9s\tremaining: 12.4s\n","[12:11:36] 1499:\tlearn: 0.4725175\ttest: 0.5025981\tbest: 0.5025961 (1497)\ttotal: 11.7s\tremaining: 11.7s\n","[12:11:37] 1599:\tlearn: 0.4717332\ttest: 0.5024031\tbest: 0.5024031 (1599)\ttotal: 12.4s\tremaining: 10.9s\n","[12:11:38] 1699:\tlearn: 0.4705071\ttest: 0.5022977\tbest: 0.5022960 (1697)\ttotal: 13.2s\tremaining: 10.1s\n","[12:11:38] 1799:\tlearn: 0.4695235\ttest: 0.5021761\tbest: 0.5021255 (1778)\ttotal: 14s\tremaining: 9.31s\n","[12:11:39] 1899:\tlearn: 0.4679901\ttest: 0.5018926\tbest: 0.5018786 (1892)\ttotal: 14.7s\tremaining: 8.53s\n","[12:11:40] 1999:\tlearn: 0.4672833\ttest: 0.5018924\tbest: 0.5018586 (1967)\ttotal: 15.5s\tremaining: 7.74s\n","[12:11:41] Stopped by overfitting detector (100 iterations wait)\n","[12:11:41] bestTest = 0.5018586183\n","[12:11:41] bestIteration = 1967\n","[12:11:41] Shrink model to first 1968 iterations.\n","[12:11:41] Fitting \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m finished. score = \u001b[1m-0.4930799691359825\u001b[0m\n","[12:11:41] \u001b[1mLvl_0_Pipe_1_Mod_3_Tuned_CatBoost\u001b[0m fitting and predicting completed\n","[12:11:41] Time left 27780.93 secs\n","\n","[12:11:41] \u001b[1mLayer 1 training completed.\u001b[0m\n","\n","[12:11:41] Blending: optimization starts with equal weights and score \u001b[1m-0.6576107362405303\u001b[0m\n","[12:11:41] Blending: iteration \u001b[1m0\u001b[0m: score = \u001b[1m-0.4874515838307017\u001b[0m, weights = \u001b[1m[0. 0. 0.66945016 0.3305498 0. ]\u001b[0m\n","[12:11:41] Blending: iteration \u001b[1m1\u001b[0m: score = \u001b[1m-0.48740093277893093\u001b[0m, weights = \u001b[1m[0. 0. 0.60242707 0.39757293 0. ]\u001b[0m\n","[12:11:41] Blending: iteration \u001b[1m2\u001b[0m: score = \u001b[1m-0.48740093277893093\u001b[0m, weights = \u001b[1m[0. 0. 0.60242707 0.39757293 0. ]\u001b[0m\n","[12:11:41] Blending: no score update. Terminated\n","\n","[12:11:41] \u001b[1mAutoml preset training completed in 1019.29 seconds\u001b[0m\n","\n","[12:11:41] Model description:\n","Final prediction for new objects (level 0) = \n","\t 0.60243 * (5 averaged models Lvl_0_Pipe_1_Mod_1_Tuned_LightGBM) +\n","\t 0.39757 * (5 averaged models Lvl_0_Pipe_1_Mod_2_CatBoost) \n","\n","CPU times: user 42min 16s, sys: 1min 48s, total: 44min 4s\n","Wall time: 16min 59s\n"]}],"source":["%%time \n","oof_pred = automl.fit_predict(train_data, roles = roles, verbose = 3)"]},{"cell_type":"code","execution_count":29,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T12:11:50.488734Z","iopub.status.busy":"2022-06-02T12:11:50.488220Z","iopub.status.idle":"2022-06-02T12:11:50.495401Z","shell.execute_reply":"2022-06-02T12:11:50.494474Z","shell.execute_reply.started":"2022-06-02T12:11:50.488693Z"},"papermill":{"duration":0.183914,"end_time":"2022-05-11T04:05:15.786503","exception":false,"start_time":"2022-05-11T04:05:15.602589","status":"completed"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Final prediction for new objects (level 0) = \n","\t 0.60243 * (5 averaged models Lvl_0_Pipe_1_Mod_1_Tuned_LightGBM) +\n","\t 0.39757 * (5 averaged models Lvl_0_Pipe_1_Mod_2_CatBoost) \n"]}],"source":["print(automl.create_model_str_desc())"]},{"cell_type":"code","execution_count":30,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T12:11:53.754576Z","iopub.status.busy":"2022-06-02T12:11:53.754177Z","iopub.status.idle":"2022-06-02T12:11:53.761373Z","shell.execute_reply":"2022-06-02T12:11:53.760348Z","shell.execute_reply.started":"2022-06-02T12:11:53.754548Z"},"papermill":{"duration":0.570189,"end_time":"2022-05-11T04:05:16.525738","exception":false,"start_time":"2022-05-11T04:05:15.955549","status":"completed"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["TRAIN out-of-fold score: 0.487400932155109\n"]}],"source":["print(f'TRAIN out-of-fold score: {mean_absolute_error(train_data[TARGET_NAME].values, oof_pred.data[:, 0])}')"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.168756,"end_time":"2022-05-11T04:05:16.862165","exception":false,"start_time":"2022-05-11T04:05:16.693409","status":"completed"},"tags":[]},"source":["# 3. Feature importances calculation \n","\n","For feature importances calculation we have 2 different methods in LightAutoML:\n","- Fast (`fast`) - this method uses feature importances from feature selector LGBM model inside LightAutoML. It works extremely fast and almost always (almost because of situations, when feature selection is turned off or selector was removed from the final models with all GBM models). no need to use new labelled data.\n","- Accurate (`accurate`) - this method calculate *features permutation importances* for the whole LightAutoML model based on the **new labelled data**. It always works but can take a lot of time to finish (depending on the model structure, new labelled dataset size etc.)."]},{"cell_type":"code","execution_count":31,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T12:11:56.941347Z","iopub.status.busy":"2022-06-02T12:11:56.940377Z","iopub.status.idle":"2022-06-02T12:11:57.407041Z","shell.execute_reply":"2022-06-02T12:11:57.405167Z","shell.execute_reply.started":"2022-06-02T12:11:56.941301Z"},"papermill":{"duration":1.474733,"end_time":"2022-05-11T04:05:18.509011","exception":false,"start_time":"2022-05-11T04:05:17.034278","status":"completed"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["CPU times: user 117 ms, sys: 148 ms, total: 265 ms\n","Wall time: 119 ms\n"]},{"data":{"text/plain":[""]},"execution_count":31,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAABsUAAAKvCAYAAAAyfcGmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAACAhklEQVR4nOzdd5htZ1k34N9DAhKIoQhGpIUmiARUIl0NVRAEVESQLooFKYJIEPkQbIjSFRSkI0ZAwdAEBALSIbRQRCKggBSld0h4vj/WmmRycloOs2edefd9X9e5zuy19z77ec/M7L3W+3tLdXcAAAAAAABgZOdYugAAAAAAAABYNaEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDO3TpArbahS50oT7qqKO29TW/8pWv5LznPe+2vuZ2W4c2Jto5mnVo5zq0MdHOkaxDGxPtHM06tHMd2pho52jWoZ3r0MZEO0eyDm1MtHM069DOdWhjop2jWYd2rkMbk2XaedJJJ/1fd194d/cNF4odddRRedvb3ratr3niiSfm2GOP3dbX3G7r0MZEO0ezDu1chzYm2jmSdWhjop2jWYd2rkMbE+0czTq0cx3amGjnSNahjYl2jmYd2rkObUy0czTr0M51aGOyTDur6r/2dJ/lEwEAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEdunQBB4ujjnvxAT/3vkefmjsfwPM/8rCbHvBrAgAAAAAAsP/MFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHj7HYpV1SFV9Y6qetF8+1JV9eaqOqWq/qGqzjUf/6759inz/Udt+jceMB//QFX91KbjN56PnVJVx206vtvXAAAAAAAAgLPj7MwUu1eS92+6/WdJHtXdl03yuSR3nY/fNcnn5uOPmh+Xqrpiktsk+aEkN07y+DloOyTJXyW5SZIrJrnt/Ni9vQYAAAAAAADst/0KxarqYklumuRv59uV5HpJnjc/5OlJbjl/fYv5dub7rz8//hZJju/ub3T3h5OckuRq859TuvtD3f3NJMcnucU+XgMAAAAAAAD22/7OFHt0kt9N8u359vck+Xx3nzrf/liSi85fXzTJR5Nkvv8L8+NPP77Lc/Z0fG+vAQAAAAAAAPutunvvD6i6WZKf7u7frKpjk/xOkjsnedO8rGGq6uJJXtrdV6qq9yS5cXd/bL7vP5NcPckfzM951nz8yUleOr/Mjbv7V+bjd9jl8Wd5jd3UeLckd0uSI4888qrHH3/82f6POPnjXzjbz9lw5GHJp7529p939EXPd8Cvud2+/OUv5/DDD1+6jJXTzrGsQzvXoY2Jdo5kHdqYaOdo1qGd69DGRDtHsw7tXIc2Jto5knVoY6Kdo1mHdq5DGxPtHM06tHMd2pgs087rXve6J3X3Mbu779D9eP61k9y8qn46ybmTHJHkMUnOX1WHzjO5Lpbk4/PjP57k4kk+VlWHJjlfks9sOr5h83N2d/wze3mNM+nuJyZ5YpIcc8wxfeyxx+5Hs87szse9+Gw/Z8N9jz41jzh5f/4rz+wjtzv2gF9zu5144ok5kP/XnUY7x7IO7VyHNibaOZJ1aGOinaNZh3auQxsT7RzNOrRzHdqYaOdI1qGNiXaOZh3auQ5tTLRzNOvQznVoY3LwtXOfyyd29wO6+2LdfVSS2yR5VXffLsmrk9xqftidkvzz/PUJ8+3M97+qp+loJyS5TVV9V1VdKsnlkrwlyVuTXK6qLlVV55pf44T5OXt6DQAAAAAAANhv+7un2O7cP8l9quqUTPt/PXk+/uQk3zMfv0+S45Kku9+b5DlJ3pfkX5LcvbtPm2eB/VaSlyV5f5LnzI/d22sAAAAAAADAfjtba/5194lJTpy//lCSq+3mMV9P8gt7eP4fJ/nj3Rx/SZKX7Ob4bl8DAAAAAAAAzo7vZKYYAAAAAAAA7AhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGt89QrKrOXVVvqap3VdV7q+oh8/FLVdWbq+qUqvqHqjrXfPy75tunzPcftenfesB8/ANV9VObjt94PnZKVR236fhuXwMAAAAAAADOjv2ZKfaNJNfr7qsk+eEkN66qayT5sySP6u7LJvlckrvOj79rks/Nxx81Py5VdcUkt0nyQ0lunOTxVXVIVR2S5K+S3CTJFZPcdn5s9vIaAAAAAAAAsN/2GYr15MvzzXPOfzrJ9ZI8bz7+9CS3nL++xXw78/3Xr6qajx/f3d/o7g8nOSXJ1eY/p3T3h7r7m0mOT3KL+Tl7eg0AAAAAAADYb/u1p9g8o+udST6d5BVJ/jPJ57v71PkhH0ty0fnriyb5aJLM938hyfdsPr7Lc/Z0/Hv28hoAAAAAAACw36q79//BVedP8vwkD0rytHlZw1TVxZO8tLuvVFXvSXLj7v7YfN9/Jrl6kj9I8qbuftZ8/MlJXjr/0zfu7l+Zj99hl8ef5TV2U9fdktwtSY488sirHn/88Wfjv2By8se/cLafs+HIw5JPfe3sP+/oi57vgF9zu335y1/O4YcfvnQZK6edY1mHdq5DGxPtHMk6tDHRztGsQzvXoY2Jdo5mHdq5Dm1MtHMk69DGRDtHsw7tXIc2Jto5mnVo5zq0MVmmnde97nVP6u5jdnffoWfnH+ruz1fVq5NcM8n5q+rQeSbXxZJ8fH7Yx5NcPMnHqurQJOdL8plNxzdsfs7ujn9mL6+xa11PTPLEJDnmmGP62GOPPTvNSpLc+bgXn+3nbLjv0afmESefrf/KJMlHbnfsAb/mdjvxxBNzIP+vO412jmUd2rkObUy0cyTr0MZEO0ezDu1chzYm2jmadWjnOrQx0c6RrEMbE+0czTq0cx3amGjnaNahnevQxuTga+c+l0+sqgvPM8RSVYcluWGS9yd5dZJbzQ+7U5J/nr8+Yb6d+f5X9TQd7YQkt6mq76qqSyW5XJK3JHlrkstV1aWq6lxJbpPkhPk5e3oNAAAAAAAA2G/7M73pIkmeXlWHZArRntPdL6qq9yU5vqr+KMk7kjx5fvyTkzyzqk5J8tlMIVe6+71V9Zwk70tyapK7d/dpSVJVv5XkZUkOSfKU7n7v/G/dfw+vAQAAAAAAAPttn6FYd787yY/s5viHklxtN8e/nuQX9vBv/XGSP97N8Zckecn+vgYAAAAAAACcHftcPhEAAAAAAAB2OqEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMPbZyhWVRevqldX1fuq6r1Vda/5+AWr6hVV9cH57wvMx6uqHltVp1TVu6vqRzf9W3eaH//BqrrTpuNXraqT5+c8tqpqb68BAAAAAAAAZ8f+zBQ7Ncl9u/uKSa6R5O5VdcUkxyV5ZXdfLskr59tJcpMkl5v/3C3JE5Ip4Ery4CRXT3K1JA/eFHI9Icmvbnrejefje3oNAAAAAAAA2G/7DMW6+xPd/fb56y8leX+Siya5RZKnzw97epJbzl/fIskzevKmJOevqosk+akkr+juz3b355K8IsmN5/uO6O43dXcnecYu/9buXgMAAAAAAAD229naU6yqjkryI0nenOTI7v7EfNcnkxw5f33RJB/d9LSPzcf2dvxjuzmevbwGAAAAAAAA7LeaJmftxwOrDk/ymiR/3N3/VFWf7+7zb7r/c919gap6UZKHdffr5uOvTHL/JMcmOXd3/9F8/EFJvpbkxPnxN5iP/3iS+3f3zfb0Grup7W6ZlmrMkUceedXjjz/+7P0vJDn5418428/ZcORhyae+dvafd/RFz3fAr7ndvvzlL+fwww9fuoyV086xrEM716GNiXaOZB3amGjnaNahnevQxkQ7R7MO7VyHNibaOZJ1aGOinaNZh3auQxsT7RzNOrRzHdqYLNPO6173uid19zG7u+/Q/fkHquqcSf4xyd919z/Nhz9VVRfp7k/MSyB+ej7+8SQX3/T0i83HPp4pGNt8/MT5+MV28/i9vcaZdPcTkzwxSY455pg+9thjd/ewvbrzcS8+28/ZcN+jT80jTt6v/8oz+cjtjj3g19xuJ554Yg7k/3Wn0c6xrEM716GNiXaOZB3amGjnaNahnevQxkQ7R7MO7VyHNibaOZJ1aGOinaNZh3auQxsT7RzNOrRzHdqYHHzt3OfyiVVVSZ6c5P3d/chNd52Q5E7z13dK8s+bjt+xJtdI8oV5CcSXJblRVV2gqi6Q5EZJXjbf98Wqusb8Wnfc5d/a3WsAAAAAAADAftuf6U3XTnKHJCdX1TvnY7+X5GFJnlNVd03yX0luPd/3kiQ/neSUJF9Ncpck6e7PVtUfJnnr/LiHdvdn569/M8nTkhyW5KXzn+zlNQAAAAAAAGC/7TMUm/cGqz3cff3dPL6T3H0P/9ZTkjxlN8ffluRKuzn+md29BgAAAAAAAJwd+1w+EQAAAAAAAHY6oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADC8Q5cugO111HEvPqDn3ffoU3PnA3zuRx520wN6HgAAAAAAwFYxUwwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEdunQBsApHHffiA3refY8+NXc+wOd+5GE3PaDnAQAAAAAAq2emGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMM7dOkCgANz1HEvPuDn3vfoU3PnA3j+Rx520wN+TQAAAAAAWJKZYgAAAAAAAAxPKAYAAAAAAMDwhGIAAAAAAAAMTygGAAAAAADA8IRiAAAAAAAADE8oBgAAAAAAwPD2GYpV1VOq6tNV9Z5Nxy5YVa+oqg/Of19gPl5V9diqOqWq3l1VP7rpOXeaH//BqrrTpuNXraqT5+c8tqpqb68BAAAAAAAAZ9f+zBR7WpIb73LsuCSv7O7LJXnlfDtJbpLkcvOfuyV5QjIFXEkenOTqSa6W5MGbQq4nJPnVTc+78T5eAwAAAAAAAM6WfYZi3f3aJJ/d5fAtkjx9/vrpSW656fgzevKmJOevqosk+akkr+juz3b355K8IsmN5/uO6O43dXcnecYu/9buXgMAAAAAAADOlgPdU+zI7v7E/PUnkxw5f33RJB/d9LiPzcf2dvxjuzm+t9cAAAAAAACAs6WmCVr7eFDVUUle1N1Xmm9/vrvPv+n+z3X3BarqRUke1t2vm4+/Msn9kxyb5Nzd/Ufz8Qcl+VqSE+fH32A+/uNJ7t/dN9vTa+yhvrtlWq4xRx555FWPP/74s/N/kCQ5+eNfONvP2XDkYcmnvnb2n3f0Rc93wK95oA60nQfaxkQ7V2Vdfma/E1/+8pdz+OGHL13GSq1DGxPtHMk6tDHRztGsQzvXoY2Jdo5mHdq5Dm1MtHMk69DGRDtHsw7tXIc2Jto5mnVo5zq0MVmmnde97nVP6u5jdnffoQf4b36qqi7S3Z+Yl0D89Hz840kuvulxF5uPfTxTMLb5+Inz8Yvt5vF7e42z6O4nJnlikhxzzDF97LHH7umhe3Tn4158tp+z4b5Hn5pHnHz2/ys/crtjD/g1D9SBtvNA25ho56qsy8/sd+LEE0/Mgbwf7CTr0MZEO0eyDm1MtHM069DOdWhjop2jWYd2rkMbE+0cyTq0MdHO0axDO9ehjYl2jmYd2rkObUwOvnYe6PKJJyS50/z1nZL886bjd6zJNZJ8YV4C8WVJblRVF6iqCyS5UZKXzfd9saquUVWV5I67/Fu7ew0AAAAAAAA4W/Y5VaSq/j7TLK8LVdXHkjw4ycOSPKeq7prkv5Lcen74S5L8dJJTknw1yV2SpLs/W1V/mOSt8+Me2t2fnb/+zSRPS3JYkpfOf7KX1wAAAAAAAICzZZ+hWHffdg93XX83j+0kd9/Dv/OUJE/ZzfG3JbnSbo5/ZnevAQAAAAAAAGfXgS6fCAAAAAAAADuGUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEJxQAAAAAAABieUAwAAAAAAIDhCcUAAAAAAAAYnlAMAAAAAACA4QnFAAAAAAAAGJ5QDAAAAAAAgOEdunQBAHtz1HEvPuDn3vfoU3PnA3j+Rx520wN+TQAAAAAADk5migEAAAAAADA8oRgAAAAAAADDE4oBAAAAAAAwPKEYAAAAAAAAwxOKAQAAAAAAMDyhGAAAAAAAAMMTigEAAAAAADA8oRgAAAAAAADDO3TpAgBIjjruxQf0vPsefWrufIDP/cjDbnpAzwMAAAAA2InMFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4hy5dAADr46jjXnxAz7vv0afmzgfw3I887KYH9HoAAAAAwHjMFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4QjEAAAAAAACGJxQDAAAAAABgeEIxAAAAAAAAhicUAwAAAAAAYHhCMQAAAAAAAIYnFAMAAAAAAGB4hy5dAACM5KjjXnzAz73v0afmzgfw/I887KYH/JoAAAAAsC6EYgDA2Xag4d+BBn+J8A8AAACA74xQDABgD4R/AAAAAOM46EOxqrpxksckOSTJ33b3wxYuCQBgGOuy5Oe6tBMAAADYs4M6FKuqQ5L8VZIbJvlYkrdW1Qnd/b5lKwMAgIPPusxuXId2CnIBAAC23kEdiiW5WpJTuvtDSVJVxye5RRKhGAAAwA63LuHfOgS5yfa3U5ALAMDZdbCHYhdN8tFNtz+W5OoL1QIAAACsMUHu3gly983M49XQzr3bST+zyXoMsvAzu2/r0E6/m3u3qjZWd6/kH94KVXWrJDfu7l+Zb98hydW7+7d2edzdktxtvnn5JB/Y1kKTCyX5v21+ze22Dm1MtHM069DOdWhjop0jWYc2Jto5mnVo5zq0MdHO0axDO9ehjYl2jmQd2pho52jWoZ3r0MZEO0ezDu1chzYmy7Tzkt194d3dcbDPFPt4kotvun2x+diZdPcTkzxxu4raVVW9rbuPWer1t8M6tDHRztGsQzvXoY2Jdo5kHdqYaOdo1qGd69DGRDtHsw7tXIc2Jto5knVoY6Kdo1mHdq5DGxPtHM06tHMd2pgcfO08x9IF7MNbk1yuqi5VVedKcpskJyxcEwAAAAAAADvMQT1TrLtPrarfSvKyJIckeUp3v3fhsgAAAAAAANhhDupQLEm6+yVJXrJ0Hfuw2NKN22gd2pho52jWoZ3r0MZEO0eyDm1MtHM069DOdWhjop2jWYd2rkMbE+0cyTq0MdHO0axDO9ehjYl2jmYd2rkObUwOsnZWdy9dAwAAAAAAAKzUwb6nGAAAAAAAAHzHhGIAAAAAAAAMTyh2gKrqfPPf16iqm1XVQb8/GwAHn6o6eukagDOrqnNU1a2XrgM4s/l381pL18HWqKprVdUvVdUdN/4sXdNWq6ojq+rJVfXS+fYVq+quS9fFd6aqzrN0DcD6WYe+g6o6ZOkatsM6tPNgv6YWih2Aqnp2kvdX1ROT/EmSX0/yrGWrWp2qulBV/VFVPbyqLr50PVtpDjXfWlVfrqpvVtVpVfXFpetapao6d1Wdd+k6VmHXE4Sq+q6qethS9axCVT21qp6y65+l6+I78viqektV/ebGgIvRVNWFq+ovquolVfWqjT9L17UKVfX2pWvgO9fd307yu0vXwdaqqqtV1evn99wbLF3PVquqp1fV+TfdvsBo5wjz7+ZfLV3Hdhg9MKqqZyb5iyTXSfJj859jFi1qNZ6W5GVJvn++/R9J7r1UMatSVT9TVcP3L82/l+9L8u/z7atU1eMXLmslquo6VXWX+esLV9Wllq6Js6+q/qmqbjr672dV/UBVvbKq3jPfvnJV/f7Sda3A8H0HST5YVX9eVVdcupAVG76dB/s19dBviit0TJJLJ/nZJDfo7psludKyJa3U3ybpJP+T5O8WrmWr/WWS2yb5YJLDkvxKBr7Qnk9qP5rpzfd3lq5nBZ5eVT+RJFV13SRvS/L5RSvaei9K8uL5z4s23R5OVZ1cVe/e9Ofkqnr30nVtte7+8SS3S3LxJCdV1bOr6oYLl7XV/i7J+5NcKslDknwkyVuXLGiFaukCVqWqrl1Vr6iq/6iqD1XVh6vqQ0vXtUL/WlW/U1UXr6oLbvxZuqitVFU/V1UfrKovVNUXq+pLgw8OekSS/5fk15IMNWhmduXu/vzGje7+XJIfWa6clXllVf18VY38frsOgdExSa7d3b/Z3feY/9xz6aJW4ELd/Zwk306S7j41yWnLlrQSv5jpGvPhVXWFpYtZoUcl+akkn0mS7n5Xkp9YtKIVqKoHJ7l/kgfMh86ZAQeCr8l50OOT/FKm38+HVdXlly5oRZ6U6ef1W0nS3e9OcptFK1qBNek7uEqmASR/W1Vvqqq7VdURSxe1AuvSzoP2mrq6e+kadpyqent3/+jG35uPLV3bKuzSzn+b34SHUFVv6+5jqurd3X3l+dg7unvEDoRU1UlJbpTky0ne0N1XXbikLVVV35fk+Uk+meSIJL/e3R9ctqrVqKqrJ3lkpguUB3b3KxYuactV1bOS/FCmDszTw7Du/q/FilqhmqbP3zLJY5N8MVO48nvd/U9L1rUVquqk7r7qLu+1b+3uH1u6tq1WVV9NcsrmQ0l6o907WVX9e5LfTnJSNnXodfdnFitqharqw7s53N196W0vZkWq6pQkP9Pd71+6lu2wyznta7t7qI7MqnpXkmPnMCzzBedrunuopXaq6ktJzpvpfehrOeN9dpiOhKp6f5Ir9sAX61X13CT37O5PLF3LKlXViUl+Pskr5j6EayT5s+7+yWUr23pzZ95tk9wl06Dapyb5++7+0qKFbaGqenN3X31zn0FVvau7r7J0bVupqt6ZaVDF2ze1890jnM9utk7nQfOsotsmeWCmgdJPSvKs7v7WooVtkY1ry11+N9/Z3T+8cGkrMXLfwWZV9ZNJnp3k/Emel+QPu/uUvT5pBxq5nQfzNbV9sA7MVebRI+eZ/64k5164pi1XVRsh32FV9SOZ2jnasntfrapzJXlnVT08yScy9gzK2ui8rKqvLF3MVuvuT1bVjTIFY/88aiA2+4tMYdFnM53QDheKdfftq+pKSf4oyZeS/L/u3t0H6o5WVVfO1Hlw00zfx5/p7rdX1fcneWOSEU5sNy62PlFVN8008/igGB20Ah9O8jNLF7EiX+july5dxHbp7nVYJuhTa9IRdJ/5y++dv64kF12wpFV5RJI3zmFDJblVkj9etqSt193fvXQN2+A9Sb4v07XJqC6U5H1V9ZYk39g42N03X66klbhPkhOSXKaqXp/kwpl+N4fT3V+squdlWoHl3plW1rlfVT22ux+3aHFb56M17WvYVXXOJPfKtBrCaL7Z3V1VnSQ16PYLWZ/zoO9Jcvskd0jyjkyreFwnyZ2SHLtcZVvq/6rqMpkC+VTVrTLgZ+g69B3Mgd9NM7XzqEznt3+X5MeTvCTJDyxW3BZal3YezNfUZoqxR1X16t0d7+7rbnctq1JVl0zyqSTnyjT6/XxJHj9KIr+hql6Y6eTgJ5K8NlNHyTW7+0KLFrbF5pHDneSQTBdjX8lgI4c3jD7aPTl9hPuGayd5cJI3dfdvLVTSSlTVazItU/u87v7aLvfdobufuUxlW6eqbpbk3zIt8/C4TDM5H9LdJyxa2AoMPtv4YZneX/8pZ+7AHHIftbmz6zdyxrJIJyb5m1FG1CZJVT0mU8f7C3Lm7+mOv6DebF4G6iy6+yHbXcuq1bQvwfXmm6/q7vctWc8qzMsm3i7Jpbr7D2va8/gi3f2WhUvbMvN12A8nGTYwmkdFn0V3v2a7a1m1qjo0yeUzXYN9YKTPkQ1VdfNMHXuXTfKMJE/v7k9X1XmSvK+7j1qyvq1SVRdK8pgkN8j0/Xx5knuNNmu+pq0WLpfkhkn+NMkvJ3n2QOFmkvU4D6qq52d6/3lmkqdtnp27sXLSYsVtoaq6dJInJrlWks9lGqh4++7+yJJ1bbU16Tv4UJJXJ3lyd79hl/seO8pSy+vSziSZB7tfMZsmFHX3M5araCIUO0DzSd/pnSTd/aIl61mFqrrI6MtZJElVnau7v1lVhyf5nhGXZluni87RbRrtfp9MyydWkt/s7sssV9VqzNOsNz6kNvYNOSimWW+VeXTQM7v7l5auha1RVdfp7tctXccq7GGwTHf39XZzfMerqr/NtETt0+dDd0hyWnf/ynJVba2qeupuDnd3//K2F8MBq6oj5hkau52B292f3e6aVqmqnpBpf6brdfcPVtUFkrx8pCV51+XcvaqOzLRfWpK8pbs/vWQ9W6mqrtfdr6qqn9vd/SN1uidJVT09U8fea3dz3/W7+5ULlLWl5vP2Z3T37ZauZTvUtE/RjTJdh71s0OX6hz8PqqrrdvduB7yPaJ7VeI6Rlm3dsA59B3MbH9jdD126llWrqsO7+8tL17Fq8+DEYzOFYi9JcpMkr+vuxWfNC8UOwDxS+scyTWtMpnV539bdD9jzs3aeGniftA1V9YdJ7pnkCUmun+R7k/xtd//hooVtsar6g+7+g6XrWLWq2u1sqd1dnO1U6zTafV1U1b8luX53f3PpWlalqi6V5B6ZlgU4fenmkUa8b5g7+P4kyfd3903mWRvX7O4nL1waZ9Pu9ggZcd+QdVBV797d8VH2RqmqF3X3zXYZTJKcsdfWMINJkjOuUdZgT59hA6MkqapbJ/nzTLNwK9NyQffr7uctWddWqaqHdPeD16HTfZ1U1esyBfLDnrcnp5+7f6K7vz7fPizJkSPNupk73/+su39n6VpWYU+B/IYBg/nzJ7ljznq9Ocxsm2Rt+g7e0t1XW7qOVZu38PmjTPvj/kuSKyf57e5+1qKFbbGqOjnJVZK8o7uvMp/fPqu7b7hwafYUO0A/neSHu/vbyemjot6RZKhQbE38fJJLJvlYpmW9Tkvy1iRDhWJJbp7kD5YuYhvcb9PXnbkzKNOSkUPYCL9q2si6RxwBtWHTrLgz6e5HbnctK/bhJK+vqhMyLfmZZLh2viDJk5O8MNPo/pE9LdPG8g+cb/9Hkn/I1P4db94T7ody5qUPRh3Jd1pVXaa7/zM5fVmW0xauaUtV1cUyLWl67fnQv2VaBupjy1W1EodkOn8fUnffbP77oF2zf4t9a+7M3Ng35MIZ7LNlN4HR46pqmMBo9sAkP7YR9s3fx3/NtMH8jtfdGwPZfqW7h/rs2GzT8vW7NeAy9h/K+OftSfLcTMvQbThtPjbMjNzuPq2qrr3vR+5Ye9vjuDPA3lO7eEmSNyU5OYOdE+xiHfoOXl9Vf5npGnpzG0dbsv9G3f27VfWzST6S5Ocy9V0OFYol+Vp3f7uqTp37MT+dqf99cUKxA3f+JBtLkZxvwTpW6cpV9cVNtzdGm450YvvV7v58Vb2quz+XJFX1tX09aQf63t0FDIN9cKa7fyZJqurcmZa5OmemtbOHUVXHZOpw/+759heS/HJ3n7RoYatxvyR/vXQR2+A/5z/nyPx9HdDXu/uxSxexTS7U3c+pqgckSXefWlVDdIZV1V8nOU+S62Zay/5Wmfa7GdXvJHn1vN57ZRpEc5dlS9pyT03y7CS/MN++/Xxs8ZF7W+zUJJ9P8o2NUe8jmjv33tndX6mq2yf50SSP7u7/Xri0rfbYJM/PdH77x5nei35/2ZK23NCB0ewcu8x++0ymc6HRfLiq/iVT596rerClerp745rkD5N8ItO118a+fxdZsLRVWYfz9iQ5dPNMlHm7iXMtWdCKvHMOF56bM3e+7/jAqLtHO2fdl3N3924H1Q5mHd6Dfnj+e/PAy84Ze+aO4pzz3zdN8tzu/sK0be5w3jbP5HxSkpOSfDnJGxetaCYUOzB/muQd894alWlvseOWLWklTt5YkmRgb0jOWMKrqs6XKbUezSFJDs8Z+zKN7omZPjQ/l+kE98bLlrOlnpJpD7F/S6b9izJ1YA6xBNQuPrEOy0KuQxuTPGZe+vPlOfMm1qON9kqSr1TV9+SMGQzXSPKFZUvaMtfq7itX1bu7+yFV9YgkL126qFWYZ6FcJdMm85efD3+gu7+x52ftSBfu7s1Lez2tqu69VDErdL4k705ynpquNt+Y5N4bswAH8oQkV6mqqyS5b6bw+plJdrs/1U7V3X9XVSdlWvq8ktyyu9+/cFlbbR0Co3+pqpcl+fv59i9mGuk/miskuVmSuyd5clW9KMnxA+4/evNdljB9QlW9K8n/W6qgVViT8/Yk+d+qunl3n5AkVXWLJP+3cE2rcO5M76+bO9uHmEVVVbfv7met0eorz6yqX03yopz5enOofVU3rRx0nu7+6tL1rEJ3X3fpGrbJC6vq3zMtn/gb8wCoYQbvVdUtk7yhu39zPvTX8yChI7p7t0vbbzeh2AHo7r+vqhNzxtTx+3f3JxcsiQO06/rC3f2FTJvJjuaTAy9vtTtX7u4fTpKqev3CtWy10zYCsSTp7tdV1alLFrRCl66qF2Q6MfifJK/v7n9ctqStN5/8/G7OuiTdSCOhjs40e/N6OWM5ixFHeyXJfZKckOQy8/vPhXPGLJydbmMm9Ver6vszdSKMOAp8Y0md23b3ozKFKaP6zDyjaKNT+raZvq9D6e6jNr6uqu/K9Dv5tEx7GI3k1O7uufPyL7v7yVV116WL2ipVdcFNNz+dM35uU1UXHKzja/jAqLvvV1U/nzOWb31idz9/yZpWYe60fE6S51TVBZI8JslrMg1aHMlXqup2SY7PdI5322yaeTOKeWD0WWb7DXbeniS/nuTv5iXMKslHM+3XNJTBZ1Odd/571NlEu/pmpmWHH5gzfkc7yWj7ql4z07L8hye5xDwQ6tc2BQ873jxZ4cGZJqAk02fmQ+f+2mF093HzvmJfmK89v5LkFkvXtYVun+SvquqrmSakvD5TSHbQXFvXYLP3t8W8yehluvs9VXWbJBdK8ozu/uI+nrqjVNWlu/tDS9exSusyaqaqHt7dv7t0Hau2qbPk1UmOzXQC/6LuvtYen7TDVNWjkxyWqZOkM3WSfD3zusMjzbypqp/M1GFwWJLvz7TG8n90970WLWyLVdXLMy2p8zuZLkDvlOR/u/v+ixa2harqlCRXHHlD4M2q6tBMs4sq0+yiby1c0paoqgdl2n/q+kn+KtN70N9294MWLWxFqupRmZa1GHY9+6q6ZKbv6TUzfT/fkOSeAy63dxZVdcvufsHSdWylqnpNpo2675KpI+HTSd7V3UcvWtgWqaoP54w9Yzecvodsd4/W8bU5MPq3EQOjdTGf0/5iptUr3pbkH0Yb6FVVR2UK/K6d6ffy9Zlm5H5kwbK2XFVdddPNc2fao/zUUa+1q+rwJOnuLy9dyypU1Q9kmmV9ZHdfqaqunGnW4x8tXBpn07zc+dW6e8QZjaerqjdnWjb6hI2VvarqPd19pWUr2zpV9Y9J3pPk6fOhOyS5Snf/3HJVrUZVXSnJFXPmwdHPWK6irVdVl8p0rXmt+e9LJHlrdy++37NQ7ADMo/aOTPLJTBebX0py2e7+qUUL22JV9fRMm61/fr59gSSP6O5fXrSwLVRVn0vyX5n2JTjdaMsizMt3vbe7vzTfPiLJD3b3m5etbGvtqbNkpE6SeXTinvSAoxRPNy9n9ozuvt3StWylqjqpu686L0l35fnYW7t7mI2s5xl/d9tlKaghVdU1u/uNm25fIMnDu/tXFyxry82zbc492oi9zfbwfjv0++yoqmq3o9sHvOj8viS/lOlC89+q6hJJjh2tnexsVfW67r5OVX0pZ55xM+L+1amqjyR5R6bZYid093Czp9ZdVb2lu6+2dB1baT7P+/kkR2XTClOjrT4zDya5X5K/GThguFSSe+Ss38ubL1XTKswDTW856pKCG6rqzd199ap6x6af2XftsnztjlZV79xY+Wlvx3a6eXuJYzOFYi9JcpMkr+vuWy1Z1ypU1RUyDZq5VpJrJPn0wbBMpuUTD8zFk1wpyUe7+6LJ9Ca0bEkrceWNQCxJuvtzVTXaHmOXSfKATKPeH9rd/7pwPavyhEybrW/48m6O7Xjdfamla1i1g+GDYzvNsxguN/9univTTKrRbMwi+kRV3TTTUpEX3Mvjd6LzJ/n3qnprzrzG+1AXY7M/q6rHd/fxVfUrSe6V5I+XLmqrVNW1sumiuqqGCxaS00P4E+blE4dTVb/b3Q+vqsdl98tA3XM3T9vJNg8yOH1mUZKhfnbn5dwfuen2f2egNlbVXs9bR5jFuQ6BUXdfZ/57XZb0uvJoK8rszrrMuNllGddzJLlqpn0rR/PPmfbEPSmbzt0HdJ7ufsu03ejpRtua4AWZltt7Yc5Yxn5EX0nyznlQ2+brzdHOaT86X491VZ0z07XmaPuqfq2qrtPz3ptVde2csYz/SG6VaQ/rd3T3XarqyMwrQI2gqn4v08ywCyf5QJI3JfnLTIOlT1uytg1CsQPzrUwdfJ+ZR4DX3h++Y52jqi7Q3Z9LTj8BHOpnZt574H7z3igPrqrfSfKg7n7rwqVttepN00K7+9vz8l5DmS/G7pRpn5CbZwqv/6S7P7hkXVupqna7WfVoo/aSpKaNcu+WKSC6TJKLJfnrTCH2SP5oXjf7vpmWMTsiyb0XrWjrPXjpArbRjTJt9PzgJC9Pcq2NWbo7XVU9M9Pv4juTbJzIDhcsJGfsKZZkyFAsZ1w8v23RKrZJd98jOX0Px3tnWhbzcUvWtAq7BCnnytTOL3f3KB22j9jLfUPsU7lOgVFVPbO777CvYwM4Yl6B5fSlMDOtxvKxBWtahSdlnnGTJN397qp6dpKhQrFMIdHG4IpTk3w4yTB7N25yse6+8dJFbIP/q6rLZP7srKpbJfnEsiVtua9392OXLmIbvGD+M7pfz7RU7UWTfDzT9eYw+4nNfj3JM+Y+kiT5XKZ+vtF8be6bPXVezevTmSbhjOKOmcLqF2Zaov/NB9sqM8N1im+T82XqRKgkGyMSR1yH8hFJ3lhVz83U1ltloNHuSVJVL8wZ37vKtLbpmzLexscfqqp7Zhq9l0wfmiPuF/fMJP+a5BWZ2vquTAHZtffynJ1mY8mVeyd59HJlbIu7J7lakjcnSXd/sKq+d9mSVuJz88nBF5JcNzl9NNQwuvs1S9ewja6Y5M+S/GGmUVGXm2dT7fgZDEmOybQ33IjnPLvz+po2mB9uT7HufuH85Ve7+7mb76uqX1igpO3ypCQnZ5qR+3c5YwPvIWwOUmoa9n6LTEuUDGGdZsuvSWD0Q5tvzAP2rrqHx+5kT03y7CQb7623n4/dcLGKVmMdZtwk0xYEX998YF5qcDRvqKqju/vkpQtZsbsneWKSK1TVxzOFnEMt1Z/kMZsG622eQbXjz2c36+6n7/tRQ7j8rttJzH0Hr1+onlX4YndfZQ6K0t1fnJcBHc3bqur8ma5PTsq0otcb9/qMHaS7rzBPrrlWpmUij5v3qXxXkjd091OXrC+xpxj7UFVXzDTqspO8urvft3BJW2re9PgsRuvAnYOEx+aMEbT/mmnj46H299m0N9OHNvYRG2195Q2b15Ae1a7rZc+dJW/f2HdrFFX19u7+0X0d28nmfQ0fl+QHM81eOCTJV0ZYBmpX85IdnTkQy3RiO8Q+VPMgmXt292gjaHdrHfYUW4f3n8122X/h37r7x5euadVGOl+oqut196uqarcbrXf3P213Tauy6+/hfA707u6+4oJlbYmqekCS30tyWJKNvV8qyTeTPLG7H7BUbauwRnujvDTJbyV5bnf/6Dzj5q7dfZOFS9tS6/K5WVXvS3LZTCHRN3LGEq6jXYddqrs/XFXnTXKO7v7SxrGla9sqVfWnSe6Q5D9zxvKJw5zPVtVzuvvWVXVydr8k+Gg/s8O/B+2hjSd194gDZ5IkVXVUkiO6+91L17IKmwY+/USSX0tyqe5efDKKmWIHoNZks+7ZOXPG8pDnXLKQFRl95FOSZA6/brN0Hdtg4yTv7puOjbq86TqMaHjNvA7xYVV1w0wzHF+4j+fsGFV1zUyjZi5cVffZdNcRGW+26l9meg96bqbZRndM8gOLVrQi3X3dqjo6yYuS3La7n790TVvoQkneV1Vvyfh7ww09K6WqbpLkp5NctKo2L6lzRAYc2b9pL6pz17Q/biU574IlrcQuYdE5Mr3ffn0PD9+JfjLJq5L8zG7u6yQ7PhTbHBhV1cY+VKcHRosVtoW6+0+T/GlV/eloAdgefKaqbp/k7+fbt03ymQXrWZXdzbi5/bIlbZ2q+r5My5UdtulzJJk+N8+zWGGrM1SYuRf/mORHu/srm449L2PNWv2FJJfu7m8uXciK3Gv++2aLVrFi69B3UFVXyDSL/Hy7nNMekeTcy1S1OlX1yu6+fpJ090d2PbbTVdXNM/3MXjvT9/W9mWY03jfTcoqLE4odmLXYrLuq7pXkVzOdKFSSZ1XVE7t7pD0YPpFpHd7NwUknufQy5axGVV0s0yyN0deyv0uSdPdLk6SqvjvJgxataIttWvLz0lV1wsbxQTulj8u0Rv/JmUaTvCTJ3y5a0dY6V5LDM30Wb9475IuZlqsdSnefUlWHzJuqPrWq3pFkuM6wOWz4g0zLDT+kqi7c3UN0ZGZq19qY17F/cM5YYu81SR56sK2FfoD+J9NS4DfPtFzHhi8l+e1FKlqtjb2oPpnkkfPXI3wfd7U5LDo1yUcyLaE4hO5+cFWdI8lLu/s5S9ezCmsWGL2lqs638Z46LyF0bHe/YNGqtt4vZ7oOe1Smc/g3ZL5mGUl3fyjJDTbPuFm6pi32U0nunGmP40duOv6lTEH2ULr7v5LTV5wZsTN6nTrf35Pk/Jn2KxpOd3+iqg5J8rSRB7RlPfoOLp8p3Dx/znxO+6VMfdNDqKpzZxpMcaGqukDOPMjioosVtvXunCkE+90kJx2MwbzlE78Dtctm3d390WUr2lpV9e4k19wYNTOf4L5xpOnHIy0pszdV9YpMa9k/cz50+yS36+4h1rKvqg9n9zOnNpZ5GCbkXJclP5Okqu42UJiwR1V1yY0Lz1FV1WuT3CBTqPnJTAMS7jzo0qYvT3Lr7v78vGb2Q5Jco7uH2idud6rqjd19zaXr2CpV9Y+ZOhI29ii4Q5KrdPdul27biarqnN39raXrgLOjqt7W3ccsXccqVdXPJnnVyIHRHpYVXItrsxHNP6N3THJUNg2+7u57LlTSSlTVz3f3Py5dx6rNI/wfkeT7M4Upl0zy/u7+ob0+cYeoqlskuWWmwUEnbLrrS0mO7+6DYhbDVqiqE5NcOclbM/BKD1X1yiQ/N8jgtT3a3HcwDxQ6vLu/uI+n7ShVdc3uHmZvrV3NE1Dunen9dfMkjS8meVJ3/+VCpS1iyT4Eodh3oKpekGkGw2cyvfkOtVn3vCbvj/W8keycZr+1u49etrKtU1UfyjQi+huZRk2/r7tHXDZo6LXs5wB3Y1RQZVpa5/RRQt094hIlwxttbew9qaofSPI7OWsnwhDrvCfTyXuST2Ua4fbbSc6X5PHdfcqiha1AVR266+dIVV2pu9+zVE3bZbTOzNE/O5Okqi6X5E+TXDGbRkaPMpikqm7c3f8y+Ky/7LIE5lkM2Cn9sCT/l+Qfkpy+5FV3f3axorbYOgRGVfXuXQdbVtXJo1xrruHv5RuSvClT/8jGkvbp7qfv8Uk7UFV9V5Kfz1nP2x+6VE2rUFXvyrQf+b/2tLfzdZPcvrvvunBpW2r0zvdkfQbVVtU/J/mRJK/Imc8NRnuvfXaSX09yWqag84gkj+nuP1+0sC1UVQ9P8kdJvpbkXzKFur/d3c9atLAtVlX3GGwltgOy5Pmt5RO/M5fs7lsm04ihhWtZhacmeXNVbeyHcsskT16unJV4TaaT2sMypfSXrKpf3Vh+byCjr2V/2ubgq6qOSHKl0U70kqSqrpFpCZYfzBQwHJLkK919xKKFrcb5d1nOIknS3Tt+z5BdPDfJX2eaRXXawrVsuXk5iz/p7ttl2tvmIQuXtGpHVJ1lK8PfnpdzfdTgF96jjbT6WlVdp7tflyRVde1MF2cjeWqmsOhRmQaT3CXTXlSj+JNMF9NPzdRRe+v5+B0yndOOstzMr2ea1ficTIO8Rt1PdcMvzn9v3kN2tOXPd/d7ONq1+9uq6pFJ/mq+ffeceTnXnW7dfi/P3d332ffDdrx/zrT87knZNOtmQN/q7s9U1Tmq6hzd/eqqevTSRa3Az1bVezNw5/uIfSJ78E8ZYG/R/XDF7v5iVd0uyUszbTlxUpJhQrEkN+ru351nzX8kyc8leW2SYX4vk6S7H1dV18pZB1kMtTXTflisD2G0E+ttUWuyWXd3P3Kean2d+dBduvsdC5a05br7TOu5V9Vlk7wg04fLSEZfy/6QqvqJJG/MtCnwp5L8WVX9R5L7dPf/LVrd1vrLJLfJFKQck2mZkh9YtKLVOV+mNaV33fNvtJPdU7v7CUsXsSrdfVpVXbKqznUwriO9Apv3qtzYd/Qi3T3a/gTr4DeSPH2eZVRJPptpbfSRHNbdr6yqmpdi+YOqOinJ/1u6sC3yjTmYv+wuy14+pKreuVBNq3CRJL+QKSw6NdMMqud19+eXLGpVuvtSS9ewDXYNjH4rYwVGSXKPTHv//sN8+xU5c9C5063V72WSZ1bVryZ5Uc68RNswMzhnF+vuGy9dxDbYWAb8tUn+rqo+nU2zbwYyfOf76INq521tLrzrrNSq+qGMuY/aOavqnJkmLfxld3+rqkYbmHjO+e+bJnlud39hN4NOd7yqemaSyyR5Z84YHN1J1i0UW4xQ7MCsxWbdVXWJTEuTvGDzse7+78WKWrHuPqWqhthna7O5o2uoNaN38YAkz8+0IefHMwVj78t0sf22TCMvhjH/nB7S3acleWpVvSPT/8Fo/ru7f3npIrbBC6vqNzP9DI/aifChJK+vqhNy5uUsHrnnp+xY79t1+v/8O7oOhrpa6e53JrnKPPs4o63XP/vGvB/BB6vqtzJ9hh6+cE1b6dWZBgF9eTez/oZZLnueLf/XSf66qi6WafDM+6rq/t39zL0/e2dag5G1uwZGL0/ywOXK2Xo97Vt93NJ1rMoa/l5+M9NMhQfmjFHfo83gTJI3VNXR3X3y0oWs2C0yrfDw20lul2mw4lBLRM7WofN99EG1j0vy+N0cv2Cm96Nf2t5yVu5vMgW470ry2nmbgtGuUV5YVf+eaQbnb8zB59cXrmkVjsk082+0UPPsWuxN155i7NG8p1gyncj+Z+ZR77uu/b6TzXv5PCHJkd19paq6cpKbd/cfLVzalqqqp2Y3U1JHChxqOnu9UHf/7y7HL9rdH1+orC1XVa9NcoNMS+19MtOslDt391UWLWwFqupZ3X37petYtar68G4O9yh7+iRJVT14d8e7e7ilFKvqg5lmE30uycfni+th9sebL7wu193/WlWHJTm0u7803zfE3mlVtdfln0YKc6vqx5K8P9Ogkj/M1On18O5+05J1bZWa9sP9myTHJrl4kv/OdD779SR37O43L1fd1ptXs7htkhtmmlX0iO5+37JVbb09jawdbd+QzebBir84wp4hVfXo7r53Vb0wu78+GWog3xr9Xn4oydUGW6HjLKrqfUkum+TDmQazDddHstk8MGjz4IORBu1t7FF5y0yd71fLdD70ou6++oJlbamqelt3H7N5H8eR9qjcaN8e7ntPd19pu2tapU2DozduV5JDepc9rXe6qrpgki/Mq86cJ8kR3f3JpevaSlX13CT37O5PLF3Lqh2sfQhCsQMwjy79zUwjLm6X5IeS/N6o+4SM9IG5q6p6TZL7JfmbjTYO+sG5sefdw5P87sbx7v7HZSri7Kqq83f35+cPk09lWvrgtzNtrPqC7v63RQsEkiRzJ98hmWbbXCLJRzOdAH7fooVtgXlZpLsluWB3X6aqLpfkr7v7+guXtqV2CXF/LVOocroRw9zRVdV5k3xPps7LL3b35xYuaUtV1UMzjXJ/f5Ljk/zLaJ0jm1XV+7MGI2vnkdG/kClQ+f4kz+/u31m2qu9cVf1od7+9qn5yd/ePsv/NGv5evjzJLbv7q0vXskrztdhZzCuzDKOqfi3TPsBfT/LtnBH+DTNob8Mune/nTfLdI3W+jz6otqo+0N2XP7v37VTzAITnJXlqd79/6Xq2UlVdr7tfVbvZVz4Zb2/5qnp1kh9O8pacecWg0QYHHbR9CEKxAzAvg/T4JH+aad3zbyZ50CijwHc10gj3XVXVW7v7xzYHf1X1zu7+4YVLW4mRA87RVdV7ktyiu/9z07GLJPmzJJcfaTTbhqr6Us4YQbwxpbpHWf98wzzy6T5JLtHdd5tPEi7f3S9auLQtUVV3SnKvJBsXJO9P8tjBlrnao6q6ZpKXZFoe8wnd/daFSzpg8x5MV0vy5k2fmSd399GLFrZCo39uVtUxmZaWuWTOPBJ8qBHvo19cV9W3M81a2OiM3vzZOdwMhpFH1lbVd2fa0+aXMi1v9U+ZZohdbNHCtlBVvby7bzR//YDu/tOla1qFNfy9fH6mwcKvzpk794abwVlV18k04Ompc3h9eHfvbuWHHWte+eCaazDzb9jrsKo6tLtP3c2g2vMleXx3n7JogVukql6c5K+6+yW7HL9JpnOFmyxT2WrM5wm3ybQ8+DmSPCXJ8T3AMu9V9ZDufvC80tWueqSVrpJk9MFBGw7mPgR7ih2Yb3f3k6rqd7r7H5KkqkbZkPx0mzoQzr+5M2GUDoTZ/1XVZTJfpFTVrTKNnBmVFHznum+Sl1fVLyd5Q6aT91/JFM7facnCVugxSa6X5I+7+8VLF7NCT820nM615tsfz7Tm+wgXY3dKcu9MP69vz9QR9KNJ/ryqusfcU+NMuvuNVXXFTBei/7uvxx/kvtHd36x5r4WqOjTjf66M3r6/yzRj/uRMI8FH9Q+Z9ho9KZsGWWQKHEZwqaUL2A6bltv77kx7M404svbTmUYM/36S13V3V9XPLlzTVrvwpq9/IdO57IjW4vdykxdk017ko5pnkx+TabDXUzPtSfWsJNdesq4V+M+cEeiObNjrsEyfJT/a3f9VVY/r7ntkmv03mnsneXFV3TrT9zKZfkevmeRmSxW1KvOSc09K8qQ5VHl2kkdV1fOS/OFODju7+8Hz33dZupbtMFr4tRcHbR+CUOzAbKzfeuskqWmD8nMsV87K/Mz892s2fT1SB0IyzfR7YpIrVNXHM43mu92yJW29mvaH6ySXrap3Z9ARiiPr7pdV1c9k+v07JMkJSa46woigPenu359HXz5o3uPn/3X365euawUu092/WFW3TZLu/mrVMDs8/0aSn+3uj2w69qp5SdfjkwwXis3LrhyX5MfnQydm2qNphI6F11TV7yU5rKpumGkp6RcuXBPfmf/t7hOWLmIbXCnTnmmHZ1rd4QML17Ol9nfZrqp6Y3dfc9X1rNBfLF3ANnhAphHgj0/y91X1DwvXswoHRUfIqo22nN6+dPfTq+pcmWY4JskHuvtbS9a0Ij+b5EcyDfZKd//PPHNjNA9I8oaqenPGnvk38nXY5naMFtqerrs/WFVHZ5phvbENymuS/Fp3f325ylajqg7JtDTvXZIcleQRmQa5/Xim1Ul+YI9PPsjVGu3rnCRVdY0kj0vyg5kG0B6S5CujrYyUg7gPQSh2YH46Sbr7XfPt82RaH3Mo65DOd/eHktxg7sQ8xzzqYkTDjZBZR939vnlPw+dPN8cNxJLTNyZPkqdlGm37+Kr6aHeP9vP8zZo2G92YsXqZbLr43OGO2CUQS5J090dq2rh7RE/KtF7/RqfBHTMNvrj9YhVtnftnmqF6cqa9tl6SaX+CoWwaSJKcMZgkGXNAyYOr6m+TvDJn7vQaaQBU5hDs1lV11SSPrKr/SfIH3f3xhUvbbudeuoDv0AM3lt0bVXc/Osmjq+rSmcKxFyT5/qq6f6Y9xf5jwfK2yqWr6oRM76kbX59ukBl/a6eqjk3y9CQfyfS9vXhV3am7X7tgWavwzXkG58Z5+3mXLmhF/ibJqzL+TPKRr8PWYgBCknT3NzLN+tujAQYGbfhgpmVq/7y737Dp+POq6icWqmmrbB5gcJZ9nQf0l5nO9Z6baXbjHbODQ829OGj7EOwpdgCq6ondPVwItqv5YuwxSa6R6QP1jUnuPdJ62VX1PUkenOQ6mdr4uiQP7e7PLFrYClXVDTKNQnhZd5+2r8dzcNilk/a7klw2yX8l+UrG66RNcvrGo2fR3dfd7lpWaR4t8/tJrpjk5ZlG8t25u09csq6tUFUndfdVz+59O1lVvat32bh6hL0q51GJ7+3uKyxdy6rNey/s0Uij/6vqWUmukOS9OaPTa8Q1+x+XM+/n85NJLtvd51muqu1XO3yf4J1e/4GqqisluW2mvcUuu3Q936k97aGxYY2WExpKVZ2U5Jc2ZuJW1Q8k+fvRzvWq6neSXC7JDTMt/fnLSZ7d3Y9btLAtNvqeqhuq6kaZ9lYd8Trsq0lOyXTec5n562TMQV77NMrPdFUd3t1fXrqOVRvl+7U3VfW27j6mqt698fs4WrsP9j4EM8UOzDFLF7BNnp3krzItEZBMCfbxSa6+WEVb7/gkr03y8/Pt22Xad+IGi1W0QlX16CRXSfKFJHfIdIHNzrB5dtSDMq3xfqdM38shjRZ+7Ul3v6Kq3p5pAEIluddAm1r/4KZZNptVkktvdzHb5JCqunJ3vztJ5uU8Tl24pu9Yd59WVR+oqkt0938vXc8qrdFSdEnyY919+aWL2AZv28dtdoYz7XO8q9FmOG7o7vdk6rR94Maxnfz+sy6hV1XdL1Mo9LGla9km59y8NG13/0dVnXPJglahu/9iHtD2xUz7iv2/7n7FwmWtwkur6m6ZlrjaPJP8s8uVtHWq6pZJ3tDdL58D3SGvw5Yu4CCzo2eEVNVjN319lvsHXNp0R3+/9tNX52WH31lVD0/yiQy2NdPB3ocgFDswF9v8hrRhwDeh83T35v1enjWf3I/kIt39h5tu/1FV/eJi1azeT2bah+rbVfWmpYth//W0Qe45My0L8ItJbr7R6T6qTcsnbvbnST6T5FHd/cZtLmmVLpppDelDk/xEVY3SubeOF2P3z7Ru9ocyXVyfJ1OAPYILJHlvVb0l0yzVJGu91NVOX4oumfYLuWJ3v2/pQlZp0143V8h0kf2B7v7mwmUtYafvk3K+TIOEdteO0fY93pcR3n9G9/1J3lhVH0ny90me293/u2xJK/W2eTneZ823b58BByDMe978w6BB2GYbg2cfsOlYZ5xBbbdP8lfzbKo3JHl9ppBslEBs3QZ5rYNfT/KeJM9J8j/Z+ed0a6uqDu3uUzNNVDhHkt9K8ttJLp4zJmyM5KDtQxCKHZivJTlp6SK2wUur6rhMs6k6U0f8S6rqgskwo4ReXlW3yfTBkiS3SvKyBetZte7ujeWR1rEzaMeqqgsl+eck/5ZpJNujq+qOSe7b3f+zaHGr85okb82ZT/iuOtrGo1X1lCRXzi7Ll2WAzr11vBjr7hdX1UUzdQTdJMl/9zhrVT9o6QIOMiN8X6+RaXTihzONBB9ySZ2q+ulM+xL8Z6Y2Xqqqfq27X7psZVtvXv7zct39r/M+KYdu2jP3DguWthX+a7SlPb8DI7z/DK27f3sOUH4i04orD6qqd2UKyP5pwL2sfyPJ3XPGnqqvTfKE5cpZme/O1Ifw2UwrzDy3uz+1cE1brrsvtXQNq9Tdt0qSqjoqybXmP79WVZdI8tbu/ukFy9tu6zLIYqeHSBdJ8guZ+mVPzfT+87zu/vySRW2lNdrX+S1JfnQe+P647r5HkocsXdQKHbR9CPYUOwCjrfG5J3MHyZ50d+/4UUJV9aUk502ysbfWITkjue5ROt/ndnamGQtfzfShcu7uHm5Ji1FV1X8k+dPufuqmY3fI9AHz9O7+48WKW5HdvdeO+P5bVe/r7isuXceSRvq+bjqZv2zOWLs/A53EMxthf6M97Z820r5pSVJV/57kZt19ynz7MklefLCub3+gqupXk9wtyQW7+zJVdbkkf93d11+4tC0x0mfFd2qE958NVXWe7v7q0nWs2ryvxg2SPCzJ5UfZ07CqLpzkwrvOOK6qH0ry6VFnx1XVlTN1Tv98ko9191DbL8wrlPxGplA3SU5M8jfd/a3FilqRqrpCpr3ErpVpsNCn12UZ/2S4z5M9DgyqqivNyxHveFV1sUyDLe6T5P67rPC1Y63Lvs6bz2dH+v3bicwUOzBPX7qA7TD66KAk6e7vXrqG7bAu7Rzc3XqXDX+7+5lV9fwkv79MSSt3waq6XZLPJflod28eOTSSN67D8mX7MNL39Wb7fsjONO/l82dJvjfT4IqNUXtDDCA5ADt2xGlVHdHdX0wy2kyFPfnSRiA2+1DGbPvdk1wtyZuTpLs/WFXfu2xJW2q/ZrqNNPt4L3bs+8+GqrpWkr9NcniSS1TVVZL8Wnf/5rKVbb15f9HbZApR/i9nXpJup3tcksfv5vgFM+2D90vbW862+XSST2Za1n2k99kNT0hyzpzxvb3DfOxXFqtoC1XV7yW5ZpILJ/lAkjcl+ctM19yn7e25HJw2DwxKcpkkF0vy10mun5y+P+eON28xcdskN0zy0gy0itkarTIzUt/HPh3MfQhmih2gqrp5zhg185rufuGS9azCvDTbWXT3M7a7Fr4ze9qUfJA9ixhUVT0u0+zNw5NcKtOSARfs7gsuWtgWq6qfTHJCpgvrYZcv25tRR0hV1fd092eWrmOrVNUpSX6mu9+/dC3bZdQRp1X1ou6+2bwqQOfMHexDrAawWVU9IcklMy2X3ZmWn/nvJP+ajHM+VFVv7u6rb4xArapDk7x9nT5PknFmlI36/rOhqt6caen6EzaNmH5Pd19p2cq2xjxT8zbzn9MybUlwfHd/aNHCtlhVva27j9nDfcN8PzdU1W8muXWmMOW5SZ4z4sC2qnpXd19lX8d2qnkG+VeSvDDTnmJv7u4vLFvVMgb6zHxn5oFBmz5TTu7uoxctbItU1UOT3DTJ+zN9nvxLT/tSrZ2d/jM772V4Sqbrr8vkjNVlhuwHOpj7EMwUOwBV9aeZ3mz/bj50z6q6Znf/3oJlrcJfZHqzrUwnfhv7bgnFdp4nZep032yIPYsY17y28unmzqG3VtWrkjx015lzO9iTM42+PDln7Cm2bkYY8f6H3f2g+eurJ3leknPOyyXdubtfvGiBW+NTB+PJ7KqMPOK0u282/z38qgCzcyf5VJKfnG//b5LDkvxMxjofes08+v2wqrphkt/M1OG3bnb8qM+R33826+6PVp3pFGCkGRr/kmn/sF8c5fu1B3tbkWTEpfovnuTe3f3OpQtZsdOq6jLd/Z9JUlWXzkC/n919haq6YKYlE49NclxVHZ7kXUnesHnLghEMvt/ohm909zc3PlPmgUE7/nxgk99P8uEkV5n//Mnc1iGDlH3Y6d/XH1y6gG120PYhmCl2AObN/n64u7893z4kyTtGexPaZZ3T9ye56jqs9z6inT6SAkY2wPT//bIGI95Pn+1WVa9M8sDuftO8T8E/jDCytqoek+T7krwg06zGJOPMstnV6CNON1TVRTPNojp9sFx3v3a5ijhQNfWO/EqSG2XqJHlZkr/tNbvgG2H28Tq8/1TV85I8MtOSZVdPcq8kx3T3bRYtbItU1Su6+4ZL17FqVfXiJH/V3S/Z5fhNktyzu2+yTGVbaw5Q9qi7P7tdtWyHqrp+kqdmWm64Mp0n3KW7X71oYSswhydXzbQS1K8luVR3H7JsVVtn9P1GN1TVw5N8Pskdk9wj08Cg93X3A5esa6usy35b+2OE87z9MUo/0cHch2Cm2IE7f5KNE5/zLVjHKp2zqn4kyRGZRti+oqru2t3/vnBdW27eb+HcG7e7+78XLGcVLlpVj07y9ST/k+T13T3M2sOMad5A9nFJrpNpNNC/JblXd39s0cK23juq6tmZRvMfVCcJW2VNRrxvHup+we5+U5J0979X1SgzAI9I8tVMHe4bRppls6vRR5ymqv4s0x4378sZI8A7yRChWFU9dm/3d/c9t6uWVZsH6b23u6+QaYWAdbbjZx9nDd5/kvx6ksckuWiSjyd5eaZ98UbxPUsXsE3uneTFVXXrnLG3zTGZ9msaaZ/Vk3LGcsOXyLTncWXqF/rvTEu9D6O7XzmHJ5efD30g0/d1CPN2KNdKcu0kP5TkvUlen+S+mZZTHMno+41uuH+mgUEnZwo3X5Jp38ohrNF+W/tjhPO8/XHufT9kRzho+xCEYgfmTzN1Yr460y/jTyQ5btmSVuL+mS6qT800pfp/kjwtZ+yltuPNJ0OPSPL9mTbLvWSmNXp/aMm6VuB+mfZmOixTW59SVX/f3Q9btizYq6cmeXamvV+S5PbzsdFG3R6WKQw76E4SttA6XIz1Hr7e3e0dqbvvsnQN22wdlqK7ZZLLd/c39vXAHeoWSf7f0kVsh+4+rao+UFWXGHBw11mswVJQw7//dPf/Jbnd0nWs0Pn3tK9zMs7gp/mc7ugkv5RkY/+w1yT5te7++nKVba2N5Yar6klJnr8xM26eEXfLBUvbUvMAi1tnCqtf2t3vrqqbZRrMdliSUVafuXOmEOx3k5zU3d9ctpyVGn6QhYFBZzJEkLIG53n7Y4jf04O5D8HyiQeoqi6S5Mfmm2/p7k8uWc92qapzjXTCUFXvSnK9JP/a04bk101y++6+68KlrdT8ofKS7r7u0rXAnlTVO7v7h/d1jINfVb25u6++sZTrfDH29pGWHa6q0zJt2F2ZOg02lhuuJOfu7h2/r0ZV/UCSJyQ5sruvVFVXTnLz7v6jhUtbiXVYiq6qXprkF7r7y0vXsgrrtnx0Vb02U4flWzK9HyVJuvvmixW1AuuwFNSavP88PdMKAJ+fb18gySO6+5cXLWyLVNVnkvxzdj+ivUdp5/4aZfbC7pYxHWlp06p6WqZ9096SaVnT/8m0tOADuvsFy1W2jBF+bkdfVnBDVf1zknusw8CgvRlhacF1OM/bHyN8L5ODuw/BTLED1N2fSHLC0nWsUlXdZw93PXJbC1mtb3X3Z6rqHFV1ju5+9bzM4HDmIOwS3f2B7v5aEoEYB7vPVNXtM21SniS3TfKZBetZiap6anYzCmiwzpJ1GPE+zN4De/GkTDOP/yZJ5tHDz06y+AntVlujEadfTfLOeR+8zcu3jrKs4DABwn560NIFbJOhZx+v0fvPlTcCsSTp7s/NS/eP4r8GO5f7Tg0xeyHJ/1TV7yd51nz7dpmCo1Eck+l389tVde4kn0xyme4e7hpsP43wczv0soKbXCDJe6tq6IFBa2Lo87yzYZRlIg/aPgShGHtzv0zT5Ef2+ao6PNPeGX9XVZ/Opg/QUczLRP55knMluVRV/XCShzpB4CD3y5n2FHvUfPv1SQ7aqdffgRdt+vrcSX42Y11cJ+tzMTa683T3WzaWX5mdulQxq7RGS9GdkLMO8lq3IGkY3f2apWvYJkMvBbVG7z/nqKoLdPfnkqSqLpix+idG6czaKqP8jt42yYOTPH++/Zokt1munC33ze7+dpJ099er6kNrHIglO/zndo0GWSTrMzBoX0b47Bn6PG+zNVkm8qDtQxjppJOt94nufsjSRazYLZJ8LclvZxrldb4kD120otV4cKaRFicmSXe/s6ouvWhFsA/zZrLDB7fd/Y+bb1fV3yd53ULlbLk1uxgb3f9V1WUyX5RU1a2SfGLZklZq+BGn3f30zber6uIZq3PvKlX1xd0cr0zLlx2x3QWt0rx/0Z8l+d5MbRyynVmD2cdZg/efTPs6v7GqnpvpZ/VWSf542ZK21H51Zo2wPNs66e7PJrnXxu2qukSmgV9/vlhRW+sKVfXu+etKcpn59sbnyTBLn6+DNRpksU4Dg9YhSFmH87wzLROZ5DJJLpZpYsr1k6S737NcdVvqoO1DEIodoKq6SpIfn2/+W3e/a8l6VuTSVfWCJF/PNGvh9bt23g7gft39B0m+neTp+3jsTvat7v7CLsn8t5cqBvZHVR2b5GZJnppp2dbvybSe/SsWLGs7XC5Th+YQ1ulibA3cPckTM3WYfDzJhzMNKBnVWow4raoLJ/mFTKPfvz9njH7f8dZkWdPNHp7kZ7r7/UsXsmLrMPt4+Pef7n5GVZ2UM5Z0/7nuft+SNW2ls9GZNcLybPtjhNkLScb+3Ezyg0sXcJAZ4ed2HQZZrM3AoDUJUtbhPC9Zn2UiD9o+BKHYAaiqeyX51ST/NB96VlU9sbsft2BZq3CLJIckOSzTyd6vVNVPdPe99v60HeXmSf5g6SK2wXur6peSHDJvUnnPJG9YuCbYl8cneUqSV2e66PxSppOhoUYoVtWXcublAD6Z6URwJGtxMbYGbpnpouTVSc6R6Xt5g6o6qbvfuWBdKzHyiNOq+u4kP5fkl5L8QKZz2kt198UWLYzv1KdGD8TWZfbxyO8/u/j3JJ/L3C+xpgNohlkSauTZC+vyuTmv1LFPI81wHPnndjb8IIvZugwMGjpIWZfzvNm6LBN5yxykfQhCsQNz1yRX7+6vJElV/VmSN2ba+2YYu16MVdVTkjxjoXJW5Xur6j67HuzuRy5RzArdI8kDk3wjybOTvCwHwaaGsA/f7O6/qKq7dPcrk6SqDoq1h7dSd3/30jVsg3W5GBvdMfOfEzKNvrx9kncn+fWqem53P3zJ4rba4CNOP53kLUl+P8nrurur6mcXronv3Nuq6h+SvCDTOV+SpLv/aY/P2GHWZfbx4O8/SZKqukemJd4/leS0zG3MYIOf1sUazF7wuXlmQ8xwXIOf23UaZDH8wKDZ0EHKupznzdZimcgcxH0IQrEDU5lO3DdsnMQPZb5QedbG5sfdfVoOkimOW+iQJIdnwO/fLq7X3Q/MFIzBTnGhObQ+3/x3JbnwwjVtufmC+lXd/YX59vmTHNvdL1iyrq20Rhdjo7tYkh/t7i8nSVU9OMmLk/xEkpMyjdAcycgjTh+Qae+wxyf5+zlIYec7IslXk9xo07HOGatbjGIdZh+P/P6z4V5JLt/dn1m6kIWNch069OyF+Nzc1Sid8KP/3K7FIIvZ8AODZusQpKzDeV6yPstEHrR9CEKxA/PUJG+uqudn+kC5RZInL1vSShyZ5K1V9fZMS5i9rLtHOfnZ8MnufujSRWyDhyZ50dJFwNn0pCTfvenvZMyThAd39+l7EXT35+cThRcsV9LWWqOLsdF9bzZdZCb5VpIju/trVfWNPTxnJxt2xGl3PzrJo6vq0pk6+V6Q5Pur6v5Jnt/d/7FgeRyg7r7L0jVsk3WYfTzs+88mH03yhaWL2A5rsDxbMv7shUfH5+aIhv65na3DIItkfQYGrUOQMvx53potE3nQ9iEIxQ5Adz+yqk5Mcp350F26+x0LlrQS3f37VfWgTB8qd0nyl1X1nCRP7u7/XLa6LfOKpQvYJuepqh/JLiMRu/vtC9UD+9TdD9l8u6oO7e7hlk/MtK7yrkb7fF6Xi7HR/V2mQUH/PN/+mSTPrqrzJnnfcmWtzPAjTrv7Q0n+JMmfVNWVMu3f+JIkl120MA5IVf1AkidkutC8UlVdOcnNu3uoJbPXZPbx8O8/ST6U5MSqenHO3MahlrFfh+XZZuswe8Hn5hlGmeG4Dj+36zDIYi0GBq1LkLIO53lrtkzkQduHUONN/NkeVXW+7v5CVV0jyYWS/MugnbWpqqtkCsVunGljvGskeUV3/+6ihW2heYr86etij/amVFVfSvLWnPnktbv7eguVBPtUVb+Rad3+P0lypySXS/K73T3UCeC8X+Pnk/zVfOjuSS7Y3XdeqqatVlWv7+5rL10H37mqOibJxvfy9d39tiXrWaWqeupuDnd3//K2F7OQqnpjd19z6TrYP1X1miT3S/I33f0j87H3dPeVlq1sa63D7ON1eP+ZZ8Wfxa6Dona6qnpn5uXZNv1entzdRy9a2BaraarNr2QaUFuZ9rD+2wFXmtmrkT439zbDsaquNEKguw4/t1X1mCTfl7EHWazNwKA5WLjHaH2Wm63DeV6S/P/27j3as7K+7/j7w8VkAI24ItaUKEIxiKCAo42QoNZLKAKixZgWGl3GC9F4KSumrQkag9alKcZoV6JoNMZCvKxYxRJLRQ2kxIAgA3IRQQQNUiuKOCJ3vv1j7+P8OA4Dc/ids895nvdrLZZn79+cmc9x9uzb93m+T5KzgP0Z1qtsuU3kqn2HYFFsCZKcAjyVoR3dv2CYovujqvqNKXPNW5LXAL8JXM8wHfeTVXV7km2AK6pqj0kDzkGSw4F3AL/AsHjuI4HLquqxkwabsyQXLDyESWtFkkuAI4ENwN7AHcAZVfWYCWPN3ThC5njgGeOuzwJvrqqb7vm71pZeHsak1nj/sLYk+VJVPXH27y3Jhqrab+Joc5XkSpx93IwkO1TVj6fOsVySnFNV/3Lh3+XYnu3LVfW4qbPNy6LZC11r5bo5O8OxqvZIsifwnqp6+sTR5qaX47aHQRbQ1cCg5gspvdznJXnK5vb3MFNutWitPdNKWQ/sztAH/WFVdVeSNT9KZjMeAjyvqq6Z3Tn+vIdNlGne3sww8+2M8SHlacAxE2daDjdMHUBagltqWOz48oXzUJJbpg41b2Px6z8leeCwOSxA2pheeryrIb2MOL0Xjp5bW65Psgfj31uSo4Drpo20LJpvBdXD+SfJkxnW5d4JeMTYneTlVfWKaZPNXfPt2TprA3VvWrluvpJxhiPA+Ey2y7SR5quX47aHtoKjHarq3IX14UYtdvNqfr0tOrjPA4tfq4FFsaX5UVXdkuRbVXXXuO+2SRMtj78ASPKI2Z1V9c2GTlC3V9X3kmyTZJuq+kKSd04dat5sk6g16hsAVXUAwFg0umuL37EGJdkX+CuGgQgkuR54YQstSRZ09DCmtryPccQpQFVdNHYLaOaltJrzSuAkYK8k1zJcR4+eNtKy6GG9rR7OP+8Efg04FaCqLkxy8KSJlsd/ZGjP9hXg5QzrT71/0kTLY2fgkiTNzl7ozK1VddtCgWGc4dhKwW9W88dtD4MsRl0MDOqkkNLDfV43bSJXM4tiS/P4JD8Edhj/N8ysR9WQy4Cvs2gdKqCZVg/AD5LsBJwFnJzk/zFzM9SKJBct3sVwsm3p71KNqaqjFm1vTHLgVHmW0XuB46rqCwBJnsrwUrOZn7WjhzG1pZcRp1uSe/8lWkWOZHjh/gVgG4Z72mckOb+qNkyYa956mH3cxfmnqr616Ge8c6osy2FRe7am1sTdjB5mL9wXrVw3m5/hOOrhuO1hkAV0MjCok0JKD/d5AG+ngzaRq5lFsSWoqm2nzrBCvtZCP+x78RzgFuA/MFwwfw74o0kTLY+LgMcCbxi/lla98YZvc1q7GdpxoSAGUFV/N64z1pJeHsbUli5GnCZ5JLBnVZ2RZB2wXVVtHD/+9xNG09ZbP/53KsNLkmMY7vuOTfLxqnr7lOHmpZPZxz2cf741DnaqJNsDr2EYlNmMXtqzQTezF4BurptdzHDs5LjtYpAF/QwMar6Q0sl9HnTSJnI1syh2PyX5w6r6w6lzLJMWp8ffzbiWz4IPASQ5LMlDxn0frqo1//9DVR2TZB+GF9AbgTdU1TcmjiXdm48ClwLns2nUZYsjhK5Kcjzw4XH7GOCqCfMsh14extSW5kecJnkp8DKG9q17ALsC7wGeDtBSG9dO7AocsLA2ZZI3AqcBBzNcS5soinUy+7j58w9wLPCnwD8HrgX+N8PP3Zrm27NBN7MXurhu9jTDsZPjtodBFtDJwCA6KKR0cp8HnbSJXM3SwPv+SSX58sJ6N61JchXwu4v3t/APNMkbtvDxsYyzGYATZtaNW7NminwABwFvBP6xqn5nokjSvUryS8AJ4+bxVXX5lHmWS5KdgTcBvzLuOgt4U1XdMF2q+UryGeB3gI9X1QHjw9hvVdW/njiadI+SHDd+uY5NI05vBJoZcZpkA/Ak4JyF7gBJvlJV+04aTEuS5KvAvlV1+7j9M8CFVbVXkgta6QCR5EzG2cczx+3FVbXPtMnmp4fzTy+SPGVz+1uboZLkShqfvQD9XDeTfAp4VeszHHs4bpPszqbW/DcwDrKoqmsmDTZnSc4CDp0ZGLQTw8CgQxiunXtPmW9ekvwp8M9ouJDSw30eQJIPbmZ3VdWLVzxMp5wpdv+10jN6c84EDl+0r5VZGi8D/uQePruzqt60kmFWwPlsmvm3cMweOlEW6T4Zi2C/nuQJwDuSfBv4w6q6duJoczUWv149dY5l1sOId7WnhxGnt1bVbQuzOJNsRwedAhp2MnDO+DIThvv4U8aWvJdOF2vueph93Oz5J8nvVdXbk7ybzZxvqqqpe6LWil9b0PzshVEv180uZjjSx3F7JH20FdyFmSIRcDvDTKObk9x6D9+zFvWw3lYP93k9tYlctSyK3X9PmDrAcmn8H+h3q+rEzX2Q5JiVDrPcqupRU2eQttailyVXAU8BrgB2mCzUnCV5IcMaGr807roMeFdV/dV0qZbFkfTxMKa29NCK7swkrwfWJXkm8Arg0xNn0hJV1QnjzNyDxl3HVtV549ctDUTooRVUy+efhRfQ523xVzWik/Zs0E8bqF6um8dPHWCF9HDcNjvIYpEuBgY1/p52QQ/3eT21iVy1bJ+4BElO3dz+1kbNJNkVeDebHqz/HnhNVf3TdKnmI8nFDNOobwM2VtXNM5811xJzpgXL3VTVO1Y6i3RfjQWjn1JVH1rpLMth/PleCxwHfJnhIeUA4I+Bd1bVh+/5u9eWJKdw94exwxgexnZjaKnYysOYGtJDK7oMQzBfwjDaNMDpwPtbWE9V7eqhFVQP559e9NCeDfppA+V1sy09HLe9tBUESLKeTe8vz54ZGNSMHgopPdznQT9tIlczi2JLkOQKhhuhu2mtNUKSzwKnAAsvZo9hOBE9c7pU8zEWxe4CHgA8ENgJ+BrwReCwqtp9wnhzl+Q6hgWA76bBNpFqTJIHAI8eNy9feDnUgiT/CPxGVV29aP9uwEeq6penyLUcenoYUzuSHA88F5gdcXoqcCJwUlWt6Zk3SbYFLqmqvabOIm2NHtbbav38Az951nx+Vf1g3N6Z4f7n1yYNNmdJzq6qg+79V2q16+m62dEMx+Y5yKItPRRSerjPA0jypap64uy/wyQbqmq/iaN1w/aJS7OxtQLYPXhoVc2OnPnLJK+dKsw8Lb5gJNkG2B14AbBbkt8cP/pwI6O+rrMAprUmyVOBDwFXMzyI/WKSF1bVWRPGmqcHLS6IAVTV1Ulae+Dspce7GtJ6K7qqujPJ5UkeUVXfnDqPtBWabwXV+vln9NCFghgMa6wm2WXCPMulh/ZsXcxe6Oy6+Xb6mOHY/HFLJ20FO9LDelvN3+eNumgTuZpZFFuaxyf5AXAL8G3gbOBNVXX9pKnm73vj+lp/PW7/W+B7E+ZZNlV1F3Al8JYk3wMexXBiCm0snLt7kk8yc8xW1d9MG0m6VycCz6qqy+EnDy1/TTtrOd68xM/WIh/GtCaNL6Gba70yY2fgkiTnMozCBNprCa7mtLze1k90cP65c7a4kOSRtPHctdiDgB8ztNtbUEBTRTHgfYyzFwCq6qKxfXZLxQXo57r5ndYLYqPmj9tOBln0pIdCShf3ecArGdpE7pXkWsY2kdNG6otFsSWoqm3HmUXrgF8Afp1hNsOzJw02fy9mWFPsTxhOuP8ANL+oY1X9VJvBBjwH2JZNx+xLkvxqVb120lTSlm2/UBADqKqvJdl+ykBz9pgkF21mfxhmrjbDhzFp1Tp+6gDSEjj7uA2/D/yfsRVUgF8FXjZtpPmrquafn0c9zF6Afq6bXcxwpJPjtoNBFj3poZDSy33ekcDfAl9gU5vIZyRpqk3kamZRbInGmUU3AVcwzC561cSR5m5cxLC1EU9dWtzuM8lfAN9N8uVxVyttItWW85K8H/jv4/bRtHUz/5ipA6wkH8ak1aeTduBqj7OPG1BV/yvJAcAvMwzAfG2DnVd6ac8Gfcxe6Om62csMxy6OWzXlSNovpPRyn9dLm8hVK74HX5okRzBM3QQ4s6o+PWWeeUryri19XlWvXqksun+SvGELH/82sDAr7oSx0CutGuMiwK8EfmXc9ffAn1VVS6OD7lWSL1bVk6fOIak9SZ4HvI1hRGbG/6qqWlvXUI1Jsp5Ns4/Pnpl9rFVubJP4g6q6cdx+GsNLvmuA/1ZVt00Yb+7GmXCvA95bVfuP+y5evMb1Wpdkd4bZCwcCNzDOXhgH2jbD62Zbejlu1Y6xvedsIeUwhkLKbkAzhZQe7vOSnAUcOtMmcieGNpGHAOdX1d5T5uuBRbElSPJW4EkM1WsY1tr6UlW9frpU85PkGuAeiylV9aEVjKP7Ick/MbS/3JzXVtUvrmQeaWsleQDDjKq7gMtbe1FyXyS5YOEliiTNU5IrgcM7WTdE0iqQ5BzguVX17ST7AWcAbwUeB9xeVS+ZMt+8JflSVT1x9n4uyYaq2m/iaHOV5Ljxy3Vsmr1wI8OLvQ1T5Zq3Xq6bvcxw7OW4VTsspLQjyVeBfavq9nH7Z4ALq2ov3wGtDNsnLs2zgf0WZtYk+RBwAdBEUQz4voWvZny3qk7c3AdJjlnpMNLWSPJshtmMX2cYBfWoJC+vqs9Mm2zFOXpF0nL5Tusv9iStOuuq6tvj18cAH6iqE8c1uzdMF2vZ9NKerZc2UL1cN9/HOMMRoKouGmeoNFUUo5/jVu3oZb2tHvTSJnLVsii2dA8Gvj9+/XMT5lgOvoBtx/ZJdgVuAzZW1c0zn/n3rNXuROBpVXUlwPhC4TSgt6KYJC2X85J8FPgkMw/YVdXamiGSVo/MfP2vgP8Mw5rdSTb/HWvbKxnas+2V5FqG9mwtDk7cFThgZvbCGxnu2w8GzgdaKS70ct3coarOXfRv8o6pwiyjXo5btcNCSiOq6oQkn2FTm8hjZ9pEHj1RrK5YFFuatwIXJPkCw039wYw389Iq9LfAA4AHjlOrvwZ8kaGwK61mGxcKYqOrgI1ThZlQk2+IJK0KDwJ+DDxrZl8Brb3ck7R6fD7JxxhmS+0MfB4gycMZBvI1paquAp4xvrDcpqpavZftZfZCL9fNXmY49nLcqhEWUtoy/t01t17aWuGaYks03rQ/cdw8t6r+75R55inJnQy9lH/qI1xEdk0b25LsDrwAOAF40fjRh8uTgVaZJH8OPBL4GMMD2fOBbzKsPdHUiMxx0fk9q+qMJOuA7RZemiTZp6ounjahJEnS/Zdh6skLgIcDH6uqa8f9+wO7VNXpU+abt5k1i2Y1t2ZRkuOB5wKzsxdOZej8cFJV+bJ2DUmyO8MMxwOBGxhmOB5dVddMGmzOPG4lqV8WxbZCkmdX1Wmb2f9A4L9U1asmiCUtSZJjgYcxFBvevLBGnrRaJPngFj6uqnrxioVZRkleCrwMeEhV7ZFkT+A9VfX0iaNJalySRwN/zjAqep8kjwOOqKrW1gyRtMYk+WJVPXnqHPfXuA7TeuDT467DGNYs2g1oas2iJOvZNHvh7JnZC83o5bo5U8xdB2zDMGi6uWIu9HHcSpJ+mkWxrZDkCuCtVfWBmX3/DngLwwLBJ0wWTpK0JiXZADwJOKeq9h/3faWq9p00mKTmJTkTeB3w3pnzz8VVtc+0yST1LskFC+eltSzJWcChM2sW7cSwZtEhDAWGvafMp63Ty3Vzpph7KkPHoGaLuZKkPrmm2NY5GDgtya7AR4A/Y+g5/Iyq+vqkySSpEUnetaXPq+rVK5VlhdxaVbctLGSdZDvG/v2StMx2qKpzF84/ozumCiNJM1q5F3LNorb0ct3cFThgppj7RoZi7sHA+YBFMUnSmmZRbCtU1XVJnsKwiOrrgRdV1UcmjiVJrXkO8IapQ6ygM5O8HliX5JnAK9jUYkeSltP1SfZgfPmc5CjgumkjSVJTTgbOSTK7ZtEpSXYELp0ulpaol+umxVxJUtNsn7gE4yj+DwA7A8+vqlsmjiRJzWilXc59NS44/xLgWQztSU4H3l9eoCUtsyS7AycBBwI3AN8Ajq6qayYNJql7Ld0PumZRO3q5biY5HnguMFvMPRU4ETipqo6eKpskSfNgUWwrJNnIpjYOAXYEbgHuBKqqHjRVNklqRZIvV9UBU+dYCUm2BS6pqr2mziKpP0mOG79cB2wD3ATcyLDOzYapcknqQ5JHAntW1RlJ1gHbVdXG8bN9quriaRNKd9fTddNiriSpZRbFJEmrSk9FMYCxnc6rquqbU2eR1JckpwDrGUZ/BzgMuAjYDfh4VblmiKRlkeSlwMuAh1TVHkn2BN5TVU+fOJp0j7xuSpLUBotikqRVJcmdDKMuf+ojGpyVm+QsYH/gXGZ+7qo6YrJQkrownn8Oraofjds7AacBhzCMet97ynyS2pVkA/Ak4JyFNolJvlJV+04aTNoCr5uSJLVhu6kDSJI0q6q2nTrDCjt+6gCSurULcOvM9u3Aw6rq5iS33sP3SNI83FpVtw1Lq/5k3W5H7Gq187opSVIDLIpJkjShqjpz6gySunUycM7YxhXgcOCUJDsCl04XS1IHzkzyemBdkmcCrwA+PXEm6d543ZQkqQG2T5QkaUJJnge8jWHkaWi0TaSk1SnJeuCgcfPsqjpvyjyS+pBhithLgGcx3PucDry/fEGhVc7rpiRJa59FMUmSJpTkSuDwqrps6iySJEnLLcm2wCVVtdfUWSRJktSfbaYOIElS575jQUySJPWiqu4ELk/yiKmzSJIkqT+uKSZJ0rTOS/JR4JPMLNxdVZ+YLJEkSdLy2hm4JMm5wE0LO6vqiOkiSZIkqQcWxSRJmtaDgB8zrKmxoACLYpIkqVXHTx1AkiRJfXJNMUmSJEmSJEmSJDXPNcUkSZpQkkcn+VySi8ftxyX5g6lzSZIkLZckz0tyRZIbk/wwycYkP5w6lyRJktrnTDFJkiaU5EzgdcB7q2r/cd/FVbXPtMkkSZKWR5IrgcOr6rKps0iSJKkvzhSTJGlaO1TVuYv23TFJEkmSpJXxHQtikiRJmsJ2UweQJKlz1yfZAyiAJEcB100bSZIkaVmdl+SjwCeBWxd2VtUnJkskSZKkLtg+UZKkCSXZHTgJOBC4AfgGcHRVXTNpMEmSpGWS5IOb2V1V9eIVDyNJkqSuWBSTJGlCSY4bv1zH0Nb4JuBG4Pyq2jBVLkmSJEmSJKk1rikmSdK01gPHAjsDDwZeDhwCvC/J702YS5IkaVkkeXSSzyW5eNx+XJI/mDqXJEmS2udMMUmSJpTkLODQqvrRuL0TcBpDYez8qtp7ynySJEnzluRM4HXAe6tq/3HfxVW1z7TJJEmS1DpnikmSNK1dmFlgHrgdeFhV3bxovyRJUit2qKpzF+27Y5IkkiRJ6sp2UweQJKlzJwPnJPnUuH04cEqSHYFLp4slSZK0bK5PsgdQAEmOAq6bNpIkSZJ6YPtESZImlmQ9cNC4eXZVnTdlHkmSpOWUZHfgJOBA4AbgG8AxVXX1lLkkSZLUPotikiRJkiRpxY0z47epqo1TZ5EkSVIfLIpJkiRJkqQVk+S4zey+ETi/qjascBxJkiR1xKKYJEmSJElaMUlOAdYDnx53HQZcBOwGfLyq3j5RNEmSJDXOopgkSZIkSVoxSc4CDq2qH43bOwGnAYcwzBbbe8p8kiRJatc2UweQJEmSJEld2QW4dWb7duBhVXXzov2SJEnSXG03dQBJkiRJktSVk4Fzknxq3D4cOCXJjsCl08WSJElS62yfKEmSJEmSVlSS9cBB4+bZVXXelHkkSZLUB4tikiRJkiRJkiRJap5rikmSJEmSJEmSJKl5FsUkSZIkSZIkSZLUPItikiRJkrQKJLkzyYaZ/3Zbwu9xZJK9lyGeJEmSJK15200dQJIkSZIEwM1Vtd/9/D2OBP4ncOl9/YYk21XVHffzz5UkSZKkVc+ZYpIkSZK0SiV5QpIzk5yf5PQkDx/3vzTJl5JcmORvkuyQ5EDgCOCPx5lmeyT5uyTrx+/5+SRXj1+/KMmpST4PfC7Jjkk+kOTcJBckec5UP7MkSZIkLReLYpIkSZK0OqybaZ34P5JsD7wbOKqqngB8AHjL+Gs/UVVPrKrHA5cBv1VV/wCcCryuqvarqq/fy593wPh7PwX4feDzVfUk4GkMhbUdl+FnlCRJkqTJ2D5RkiRJklaHu7VPTLIPsA/w2SQA2wLXjR/vk+TNwIOBnYDTl/Dnfbaqvj9+/SzgiCS/O27/LPAIhoKbJEmSJDXBopgkSZIkrU4BLqmqJ2/ms78EjqyqC5O8CHjqPfwed7CpQ8jPLvrspkV/1r+pqsuXnFaSJEmSVjnbJ0qSJEnS6nQ58NAkTwZIsn2Sx46fPRC4bmyxePTM92wcP1twNfCE8eujtvBnnQ68KuOUtCT73//4kiRJkrS6WBSTJEmSpFWoqm5jKGS9LcmFwAbgwPHj44FzgLOBr85820eA1yW5IMkewH8FfjvJBcDPb+GPOwHYHrgoySXjtiRJkiQ1JVU1dQZJkiRJkiRJkiRpWTlTTJIkSZIkSZIkSc2zKCZJkiRJkiRJkqTmWRSTJEmSJEmSJElS8yyKSZIkSZIkSZIkqXkWxSRJkiRJkiRJktQ8i2KSJEmSJEmSJElqnkUxSZIkSZIkSZIkNc+imCRJkiRJkiRJkpr3/wG0pTw+h9XulQAAAABJRU5ErkJggg==","text/plain":["
"]},"metadata":{"needs_background":"light"},"output_type":"display_data"}],"source":["%%time\n","\n","# Fast feature importances calculation\n","fast_fi = automl.get_feature_scores('fast')\n","top_3_features = fast_fi['Feature'].values[:3]\n","fast_fi.set_index('Feature')['Importance'].plot.bar(figsize = (30, 10), grid = True)"]},{"cell_type":"code","execution_count":32,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T12:12:13.935913Z","iopub.status.busy":"2022-06-02T12:12:13.934831Z","iopub.status.idle":"2022-06-02T12:12:13.945336Z","shell.execute_reply":"2022-06-02T12:12:13.944522Z","shell.execute_reply.started":"2022-06-02T12:12:13.935849Z"},"papermill":{"duration":0.185287,"end_time":"2022-05-11T04:05:18.864914","exception":false,"start_time":"2022-05-11T04:05:18.679627","status":"completed"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
FeatureImportance
0Кол-во отзывов475387.880208
1Популярность155872.650142
2Дата релиза127909.501239
3Длительность110062.479156
4Язык72720.727179
\n","
"],"text/plain":[" Feature Importance\n","0 Кол-во отзывов 475387.880208\n","1 Популярность 155872.650142\n","2 Дата релиза 127909.501239\n","3 Длительность 110062.479156\n","4 Язык 72720.727179"]},"execution_count":32,"metadata":{},"output_type":"execute_result"}],"source":["fast_fi.head()"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2022-05-12T14:38:55.906446Z","iopub.status.busy":"2022-05-12T14:38:55.906181Z","iopub.status.idle":"2022-05-12T14:38:55.910045Z","shell.execute_reply":"2022-05-12T14:38:55.909341Z","shell.execute_reply.started":"2022-05-12T14:38:55.906414Z"},"papermill":{"duration":0.227872,"end_time":"2022-05-11T04:07:19.390919","exception":false,"start_time":"2022-05-11T04:07:19.163047","status":"completed"},"tags":[],"trusted":true},"outputs":[],"source":["#%%time\n","\n","# Accurate feature importances calculation (Permutation importances) - can take long time to calculate on bigger datasets\n","# accurate_fi = automl.get_feature_scores('accurate', te_data, silent = False)\n","# accurate_fi.set_index('Feature')['Importance'].plot.bar(figsize = (30, 10), grid = True)"]},{"cell_type":"markdown","metadata":{"papermill":{"duration":0.223583,"end_time":"2022-05-11T04:07:19.850624","exception":false,"start_time":"2022-05-11T04:07:19.627041","status":"completed"},"tags":[]},"source":["# 4. Predict for test dataset\n","\n","We are also ready to predict for our test competition dataset and submission file creation:"]},{"cell_type":"code","execution_count":34,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T12:16:05.923215Z","iopub.status.busy":"2022-06-02T12:16:05.921032Z","iopub.status.idle":"2022-06-02T12:16:09.695151Z","shell.execute_reply":"2022-06-02T12:16:09.688718Z","shell.execute_reply.started":"2022-06-02T12:16:05.923103Z"},"papermill":{"duration":67.131549,"end_time":"2022-05-11T04:08:27.205094","exception":false,"start_time":"2022-05-11T04:07:20.073545","status":"completed"},"tags":[],"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Prediction for te_data:\n","array([[ 1.1119626e-04],\n"," [ 6.9836912e+00],\n"," [ 5.0787916e+00],\n"," [-6.3805520e-03],\n"," [-1.4231394e-07],\n"," [ 8.0792660e-08],\n"," [-1.4294294e-04],\n"," [ 5.1703835e+00],\n"," [ 6.0861902e+00],\n"," [ 6.1515681e-03]], dtype=float32)\n","Shape = (13076, 1)\n"]}],"source":["test_pred = automl.predict(test_data)\n","print(f'Prediction for te_data:\\n{test_pred[:10]}\\nShape = {test_pred.shape}')"]},{"cell_type":"code","execution_count":35,"metadata":{"execution":{"iopub.execute_input":"2022-06-02T12:17:06.986122Z","iopub.status.busy":"2022-06-02T12:17:06.985377Z","iopub.status.idle":"2022-06-02T12:17:07.090270Z","shell.execute_reply":"2022-06-02T12:17:07.086183Z","shell.execute_reply.started":"2022-06-02T12:17:06.986097Z"},"papermill":{"duration":2.224661,"end_time":"2022-05-11T04:08:29.643539","exception":false,"start_time":"2022-05-11T04:08:27.418878","status":"completed"},"tags":[],"trusted":true},"outputs":[{"data":{"text/html":["
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
IDСредний рейтинг
08817831.111963e-04
15077696.983691e+00
21600645.078792e+00
38027630.000000e+00
43390290.000000e+00
.........
130716893026.327456e-05
130725655367.016068e+00
130734935345.611053e+00
130743888756.473909e+00
130751453628.079266e-08
\n","

13076 rows × 2 columns

\n","
"],"text/plain":[" ID Средний рейтинг\n","0 881783 1.111963e-04\n","1 507769 6.983691e+00\n","2 160064 5.078792e+00\n","3 802763 0.000000e+00\n","4 339029 0.000000e+00\n","... ... ...\n","13071 689302 6.327456e-05\n","13072 565536 7.016068e+00\n","13073 493534 5.611053e+00\n","13074 388875 6.473909e+00\n","13075 145362 8.079266e-08\n","\n","[13076 rows x 2 columns]"]},"execution_count":35,"metadata":{},"output_type":"execute_result"}],"source":["submission[TARGET_NAME] = np.clip(test_pred.data[:, 0], 0, None)\n","submission.to_csv('lightautoml_tabularautoml.csv', index = False)\n","submission"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.12"}},"nbformat":4,"nbformat_minor":4} 2 | --------------------------------------------------------------------------------