├── .gitignore
├── README.md
├── part1-get-data-and-non-graph-modeling-prep.ipynb
├── part2-simple-non-graph-model-and-pca.ipynb
├── part3-prepare-papers-for-import.ipynb
├── part4-prepare-authors-and-inst-for-import.ipynb
├── part5-admin-import.md
├── part6-analysis-in-neo4j-gds.ipynb
├── part7-graph-feature-engineering-in-gds.ipynb
└── part8-graph-feature-model.ipynb
/.gitignore:
--------------------------------------------------------------------------------
1 | data/
2 | scratch/
3 | .idea/
4 | .ipynb_checkpoints/
5 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # GDS Webinar Demo: Graph Data Science for Really Big Data
2 |
3 | This repo contains demo code from the 2022 GDS February Webinar - "Graph Data Science for Really Big Data". The exact pattern here may vary slightly from what you have seen in the webinar, most of the commands have been placed in notebooks for example, but the overall steps should be the same.
4 |
5 | The purpose of this demo is to explore engineering graph features using Neo4j and the [Graph Data Science (GDS) Library](https://neo4j.com/docs/graph-data-science/current/) on a larger dataset to see if we can improve accuracy for a classification problem.
6 |
7 | The graph used here is the [MAG240M OGB Large-Scale-Challenge Graph](https://ogb.stanford.edu/docs/lsc/mag240m/). It is a heterogeneous academic paper graph that contains around 240 Million Nodes and 1.7 Billion Relationships.
8 |
9 | ## Demo Outline and Notebooks Parts
10 |
11 | This demo walks through multiple steps including running a reference model before using graph, formatting and importing data into Neo4j, analyzing the graph and engineering graph features with GDS, and exporting data to re-run a model with those graph features.
12 |
13 | The demo here is ultimately split up into 8 parts, 7 of which are ipython notebooks. Hopefully the file names are descriptive as to what they cover
14 |
15 | - Parts 1 and 2 focus on understanding the data and running a classification model with available features before leveraging Neo4j/GDS/graph
16 | - `part1-get-data-and-non-graph-modeling-prep.ipynb`
17 | - `part2-simple-non-graph-model-and-pca.ipynb`
18 |
19 | - Parts 3-5 are focused on pre-formatting the data and importing into graph
20 | - `part3-prepare-papers-for-import.ipynb`
21 | - `part4-prepare-authors-and-inst-for-import.ipynb`
22 | - `part5-admin-import.md`
23 |
24 | - Part 6 and 7 focus on work in Neo4j and GDS. Part 6 is mostly inspecting the graph and demoing native projections and the WCC algorithm. Part 7 is focused on actually generating and exporting graph features (FastRP Node Embeddings)
25 | - `part6-analysis-in-neo4j-gds.ipynb`
26 | - `part7-graph-feature-engineering-in-gds.ipynb`
27 |
28 | - Finally Part 8 re-runs the classification model with the graph features (FastRP Node Embeddings). In this very rough exploratory first pass we get an ~9% point increase in classification accuracy.
29 | - `part8-graph-feature-model.ipynb`
30 |
31 |
32 |
33 | ## Prerequisites & Environment for Running the Demo
34 |
35 | ### Software Versions
36 | - Neo4j = Enterprise Edition 4.4.3
37 | - GDS = Enterprise Edition 1.8.3
38 | - APOC = 4.4.0.3
39 | - Python = 3.9.7
40 |
41 | Important Note: Enterprise (as opposed to Community) Editions were used for both the Neo4j Database and GDS library in this demo. The use of GDS Enterprise, in particular, provides high-concurrency and optimized in-memory compression which are not available in Community Edition and key to performance at these scales.
42 |
43 | ### Instance
44 | This demo was run on a single AWS ec2 x1.16xlarge instance (64 vCPUs, 976 GB Memory).
45 |
46 | ### Neo4j Configuration
47 | I tweaked a few things but the below are the most critical which you can update in the neo4j settings/configuration (a.k.a `neo4j.conf`)
48 |
49 | - `dbms.memory.heap.max_size=760G`
50 | - `gds.export.location=/data/neo-export` # or set to whatever directory you would like data exports from Neo4j to go
51 |
52 | Depending on your environment and specific needs you may need to tune this and other configuration like min heap size, pagecache, etc. For more details on optimizing Neo4j configuration for data science and analytics at scale I recommend looking into the [Graph Data Science Configuration Guide](https://neo4j.com/whitepapers/graph-data-science-configuration-guide/).
53 |
54 |
55 | ## Future Experimentation & Improvements
56 |
57 | This demo was just a rough first pass to explore what is possible. There are many ways to improve upon this analysis! Here are just a few areas to experiment:
58 |
59 | 1. Improved tuning of FastRP node Embeddings
60 | 2. Inclusion of more graph features
61 | 3. Streamlined data formatting and ETL
62 | 4. Better-tuned and/or more sophisticated classification models and frameworks
63 | 5. Exploration of Semi-supervised transductive approaches to label the rest of the papers, such as Label Propagation Algorithm (LPA) or K-Nearest Neighbor (KNN)
--------------------------------------------------------------------------------
/part1-get-data-and-non-graph-modeling-prep.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "12fe1bee",
6 | "metadata": {},
7 | "source": [
8 | "# Part 1: Prepare Data For Non-Graph (\"Flat\") Modeling"
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": 1,
14 | "id": "50a66a9b",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "from ogb.lsc import MAG240MDataset\n",
19 | "import numpy as np\n",
20 | "import os\n",
21 | "import pandas as pd"
22 | ]
23 | },
24 | {
25 | "cell_type": "markdown",
26 | "id": "ec956515",
27 | "metadata": {},
28 | "source": [
29 | "## Notebook Setup"
30 | ]
31 | },
32 | {
33 | "cell_type": "markdown",
34 | "id": "70aa26f6",
35 | "metadata": {},
36 | "source": [
37 | "Root Directory for data storage. Will be used in following parts as well."
38 | ]
39 | },
40 | {
41 | "cell_type": "code",
42 | "execution_count": 2,
43 | "id": "266ad13e",
44 | "metadata": {},
45 | "outputs": [],
46 | "source": [
47 | "ROOT_DATA_DIR = \"/data\""
48 | ]
49 | },
50 | {
51 | "cell_type": "code",
52 | "execution_count": 3,
53 | "id": "cc481218",
54 | "metadata": {},
55 | "outputs": [
56 | {
57 | "name": "stdout",
58 | "output_type": "stream",
59 | "text": [
60 | "Directory /data already exists\n"
61 | ]
62 | }
63 | ],
64 | "source": [
65 | "if not os.path.exists(ROOT_DATA_DIR):\n",
66 | " os.mkdir(ROOT_DATA_DIR)\n",
67 | " print(f'Created new directory: {ROOT_DATA_DIR}')\n",
68 | "else:\n",
69 | " print(f'Directory {ROOT_DATA_DIR} already exists')"
70 | ]
71 | },
72 | {
73 | "cell_type": "markdown",
74 | "id": "74bdc7db",
75 | "metadata": {},
76 | "source": [
77 | "### Get the Dataset Object\n",
78 | "The dataset object handles downloading and easy access to the data and its features. The dataset object leverages [numpy memmap](https://numpy.org/doc/stable/reference/generated/numpy.memmap.html) functionality to reference large pieces of the dataset on disk so it does not need to load all the features into memory at a time. For more information, please see the [OGB MAG240M Page](https://ogb.stanford.edu/kddcup2021/mag240m/).\n",
79 | "\n",
80 | "__Note: This command takes a while in the *first* run (several hours to a day)__ as the source data needs to be download from OGB. Sequential runs should be near instantaneous though.\n"
81 | ]
82 | },
83 | {
84 | "cell_type": "code",
85 | "execution_count": 4,
86 | "id": "81db468a",
87 | "metadata": {},
88 | "outputs": [],
89 | "source": [
90 | "dataset = MAG240MDataset(root = ROOT_DATA_DIR)"
91 | ]
92 | },
93 | {
94 | "cell_type": "markdown",
95 | "id": "75a27300",
96 | "metadata": {},
97 | "source": [
98 | "## Examine Data Splitting and Labels\n",
99 | "\n",
100 | "Only a fraction of the papers (the arXiv papers) are labeled. An `idx_split` object is provided with indexes mapping the labeled papers to training, validate, and test sets. As we will see below, the training sets have their labels hidden for purposes of previous competition. More information on the data and labeling process can be found at the [OGB MAG240M Page](https://ogb.stanford.edu/kddcup2021/mag240m/)"
101 | ]
102 | },
103 | {
104 | "cell_type": "code",
105 | "execution_count": 5,
106 | "id": "100a55a5",
107 | "metadata": {},
108 | "outputs": [],
109 | "source": [
110 | "#get the indexes for arXiv paper data splits\n",
111 | "split_dict = dataset.get_idx_split()"
112 | ]
113 | },
114 | {
115 | "cell_type": "code",
116 | "execution_count": 6,
117 | "id": "4080b496",
118 | "metadata": {},
119 | "outputs": [
120 | {
121 | "name": "stdout",
122 | "output_type": "stream",
123 | "text": [
124 | "------------------\n",
125 | "train index size = 1112392\n",
126 | "------------------\n",
127 | "valid index size = 138949\n",
128 | "------------------\n",
129 | "test-whole index size = 146818\n",
130 | "------------------\n",
131 | "test-dev index size = 88092\n",
132 | "------------------\n",
133 | "test-challenge index size = 58726\n"
134 | ]
135 | }
136 | ],
137 | "source": [
138 | "#get the relative sizes of each set\n",
139 | "for i in split_dict.keys():\n",
140 | " print('------------------')\n",
141 | " print(f'{i} index size = {len(split_dict[i])}')"
142 | ]
143 | },
144 | {
145 | "cell_type": "code",
146 | "execution_count": 7,
147 | "id": "7466446c",
148 | "metadata": {},
149 | "outputs": [
150 | {
151 | "name": "stdout",
152 | "output_type": "stream",
153 | "text": [
154 | "Paper labels for the \"train\" set:\n",
155 | "--------\n",
156 | "Sample values = [17. 29. 38. 5. 1.]\n",
157 | "Number non-missing = 1112392\n",
158 | "============================\n",
159 | "\n",
160 | "Paper labels for the \"valid\" set:\n",
161 | "--------\n",
162 | "Sample values = [140. 129. 33. 59. 24.]\n",
163 | "Number non-missing = 138949\n",
164 | "============================\n",
165 | "\n",
166 | "Paper labels for the \"test-whole\" set:\n",
167 | "--------\n",
168 | "Sample values = [-1. -1. -1. -1. -1.]\n",
169 | "Number non-missing = 0\n",
170 | "============================\n",
171 | "\n",
172 | "Paper labels for the \"test-dev\" set:\n",
173 | "--------\n",
174 | "Sample values = [-1. -1. -1. -1. -1.]\n",
175 | "Number non-missing = 0\n",
176 | "============================\n",
177 | "\n",
178 | "Paper labels for the \"test-challenge\" set:\n",
179 | "--------\n",
180 | "Sample values = [-1. -1. -1. -1. -1.]\n",
181 | "Number non-missing = 0\n",
182 | "============================\n",
183 | "\n"
184 | ]
185 | }
186 | ],
187 | "source": [
188 | "# Note that we only have known labels in the train and validate sets. \n",
189 | "# A value of -1 implies a hidden label\n",
190 | "for i in split_dict.keys():\n",
191 | " paper_labels = dataset.paper_label[split_dict[i]]\n",
192 | " print(f'Paper labels for the \"{i}\" set:')\n",
193 | " print('--------')\n",
194 | " print(f'Sample values = {paper_labels[:5]}')\n",
195 | " print(f'Number non-missing = {sum(dataset.paper_label[split_dict[i]] > -1)}')\n",
196 | " print('============================\\n')"
197 | ]
198 | },
199 | {
200 | "cell_type": "markdown",
201 | "id": "439a60ae",
202 | "metadata": {},
203 | "source": [
204 | "## Building a DataFrame for Supervised Model Testing\n",
205 | "\n",
206 | "We will use the 'train' and 'valid' set for pre-graph supervised model analysis\n",
207 | "since they are the only ones with labels"
208 | ]
209 | },
210 | {
211 | "cell_type": "code",
212 | "execution_count": 8,
213 | "id": "30a40151",
214 | "metadata": {},
215 | "outputs": [],
216 | "source": [
217 | "#get the training set\n",
218 | "feat_cols = [f'paper_encoding_{i}' for i in range(768)]\n",
219 | "paper_df_train = pd.DataFrame(dataset.paper_feat[split_dict['train']], columns = feat_cols)\n",
220 | "paper_df_train['split_segment'] = 'TRAIN'\n",
221 | "paper_df_train['paper_subject'] = dataset.paper_label[split_dict['train']]\n",
222 | "paper_df_train['paper_year'] = dataset.paper_year[split_dict['train']]"
223 | ]
224 | },
225 | {
226 | "cell_type": "code",
227 | "execution_count": 9,
228 | "id": "9c0f485e",
229 | "metadata": {},
230 | "outputs": [],
231 | "source": [
232 | "#get the validation set\n",
233 | "paper_df_validate = pd.DataFrame(dataset.paper_feat[split_dict['valid']], columns = feat_cols)\n",
234 | "paper_df_validate['split_segment'] = 'VALIDATE'\n",
235 | "paper_df_validate['paper_subject'] = dataset.paper_label[split_dict['valid']]\n",
236 | "paper_df_validate['paper_year'] = dataset.paper_year[split_dict['valid']]"
237 | ]
238 | },
239 | {
240 | "cell_type": "code",
241 | "execution_count": 10,
242 | "id": "49a7873b",
243 | "metadata": {},
244 | "outputs": [
245 | {
246 | "data": {
247 | "text/html": [
248 | "
"
642 | ],
643 | "text/plain": [
644 | " paper_encoding_0\n",
645 | "paper_subject \n",
646 | "0.0 28041\n",
647 | "1.0 2856\n",
648 | "2.0 3907\n",
649 | "3.0 1530\n",
650 | "4.0 1910\n",
651 | "... ...\n",
652 | "148.0 865\n",
653 | "149.0 815\n",
654 | "150.0 837\n",
655 | "151.0 22696\n",
656 | "152.0 1139\n",
657 | "\n",
658 | "[153 rows x 1 columns]"
659 | ]
660 | },
661 | "execution_count": 5,
662 | "metadata": {},
663 | "output_type": "execute_result"
664 | }
665 | ],
666 | "source": [
667 | "papers_df[['paper_subject', 'paper_encoding_0']].groupby('paper_subject').count()"
668 | ]
669 | },
670 | {
671 | "cell_type": "markdown",
672 | "id": "88beb827",
673 | "metadata": {},
674 | "source": [
675 | "## Logistic Regression Using Entire 768 Dimensional Encoding\n",
676 | "\n",
677 | "\n",
678 | "As a first pass we will try to fit this model with simple logistic regression using just the 768 dimensional RoBERTa encoding vectors as features. \n",
679 | "\n",
680 | "__Note: this model fitting step can take a while (several hours) to complete__\n",
681 | "\n",
682 | "We will get convergence warnings when running the below model model. I tried various different parameters to try and avoid this in sklearn but could not seem to do so. In a more rigorous setting I would recommend looking deeper into tuning parameters, different model types, different machine learning libraries/frameworks, etc. But for purposes of this demo we are just trying to get an initial rough benchmark. In the following sections we will apply a very simple solution of dimensionality reduction with Principal Components Analysis (PCA) to see the effect on results. "
683 | ]
684 | },
685 | {
686 | "cell_type": "code",
687 | "execution_count": 6,
688 | "id": "aa8a82dc",
689 | "metadata": {},
690 | "outputs": [],
691 | "source": [
692 | "papers_df = papers_df.astype({'paper_subject':'int32'})"
693 | ]
694 | },
695 | {
696 | "cell_type": "code",
697 | "execution_count": 7,
698 | "id": "4863519d",
699 | "metadata": {},
700 | "outputs": [],
701 | "source": [
702 | "X = papers_df[['paper_encoding_' + str(x) for x in range(768)]]\n",
703 | "y = papers_df.paper_subject"
704 | ]
705 | },
706 | {
707 | "cell_type": "code",
708 | "execution_count": 8,
709 | "id": "486dfa78",
710 | "metadata": {},
711 | "outputs": [],
712 | "source": [
713 | "X_train = X[papers_df.split_segment == \"TRAIN\"]\n",
714 | "X_validate = X[papers_df.split_segment == \"VALIDATE\"]\n",
715 | "y_train = y[papers_df.split_segment == \"TRAIN\"]\n",
716 | "y_validate = y[papers_df.split_segment == \"VALIDATE\"]"
717 | ]
718 | },
719 | {
720 | "cell_type": "code",
721 | "execution_count": 9,
722 | "id": "e3d95251",
723 | "metadata": {},
724 | "outputs": [],
725 | "source": [
726 | "model = LogisticRegression(multi_class='ovr', solver='saga', n_jobs=60, max_iter=200)"
727 | ]
728 | },
729 | {
730 | "cell_type": "code",
731 | "execution_count": 10,
732 | "id": "91cba657",
733 | "metadata": {},
734 | "outputs": [
735 | {
736 | "name": "stderr",
737 | "output_type": "stream",
738 | "text": [
739 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
740 | " warnings.warn(\n",
741 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
742 | " warnings.warn(\n",
743 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
744 | " warnings.warn(\n",
745 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
746 | " warnings.warn(\n",
747 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
748 | " warnings.warn(\n",
749 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
750 | " warnings.warn(\n",
751 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
752 | " warnings.warn(\n",
753 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
754 | " warnings.warn(\n",
755 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
756 | " warnings.warn(\n",
757 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
758 | " warnings.warn(\n",
759 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
760 | " warnings.warn(\n",
761 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
762 | " warnings.warn(\n",
763 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
764 | " warnings.warn(\n",
765 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
766 | " warnings.warn(\n",
767 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
768 | " warnings.warn(\n",
769 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
770 | " warnings.warn(\n",
771 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
772 | " warnings.warn(\n",
773 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
774 | " warnings.warn(\n",
775 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
776 | " warnings.warn(\n",
777 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
778 | " warnings.warn(\n",
779 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
780 | " warnings.warn(\n",
781 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
782 | " warnings.warn(\n",
783 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
784 | " warnings.warn(\n",
785 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
786 | " warnings.warn(\n",
787 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
788 | " warnings.warn(\n",
789 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
790 | " warnings.warn(\n",
791 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
792 | " warnings.warn(\n",
793 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
794 | " warnings.warn(\n",
795 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
796 | " warnings.warn(\n",
797 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
798 | " warnings.warn(\n",
799 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
800 | " warnings.warn(\n",
801 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
802 | " warnings.warn(\n",
803 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
804 | " warnings.warn(\n",
805 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
806 | " warnings.warn(\n",
807 | "/home/ubuntu/.conda/envs/graph2/lib/python3.9/site-packages/sklearn/linear_model/_sag.py:352: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n",
808 | " warnings.warn(\n"
809 | ]
810 | },
811 | {
812 | "data": {
813 | "text/plain": [
814 | "LogisticRegression(max_iter=200, multi_class='ovr', n_jobs=60, solver='saga')"
815 | ]
816 | },
817 | "execution_count": 10,
818 | "metadata": {},
819 | "output_type": "execute_result"
820 | }
821 | ],
822 | "source": [
823 | "#Note: This can take a while (several hours)\n",
824 | "model.fit(X_train, y_train)"
825 | ]
826 | },
827 | {
828 | "cell_type": "code",
829 | "execution_count": 11,
830 | "id": "a826e1b8",
831 | "metadata": {},
832 | "outputs": [
833 | {
834 | "name": "stdout",
835 | "output_type": "stream",
836 | "text": [
837 | "Accuracy of logistic regression classifier on VALIDATE set: 0.49\n"
838 | ]
839 | }
840 | ],
841 | "source": [
842 | "print('Accuracy of logistic regression classifier on VALIDATE set: {:.2f}'\\\n",
843 | " .format(model.score(X_validate, y_validate)))"
844 | ]
845 | },
846 | {
847 | "cell_type": "markdown",
848 | "id": "67621ceb",
849 | "metadata": {},
850 | "source": [
851 | "## Reducing Dimensionality with Principal Components Analysis (PCA)"
852 | ]
853 | },
854 | {
855 | "cell_type": "code",
856 | "execution_count": 12,
857 | "id": "1cea5185",
858 | "metadata": {},
859 | "outputs": [
860 | {
861 | "data": {
862 | "text/plain": [
863 | "PCA()"
864 | ]
865 | },
866 | "execution_count": 12,
867 | "metadata": {},
868 | "output_type": "execute_result"
869 | }
870 | ],
871 | "source": [
872 | "from sklearn.decomposition import PCA\n",
873 | "pca = PCA()\n",
874 | "pca.fit(X_train)"
875 | ]
876 | },
877 | {
878 | "cell_type": "code",
879 | "execution_count": 13,
880 | "id": "0bcbf2b2",
881 | "metadata": {},
882 | "outputs": [
883 | {
884 | "data": {
885 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl20lEQVR4nO3de5wcVZ338c83CUlIuCdZRIEMV9moqDAgNxVFEVDJwy4I7LCAopFVvCzeQFweYM3u4u6CogTMgoAwchFveRBECIqIChmQOwQCBAKyEgIkIQFy+z1/1Omk05npqZpMdffMfN+vV7266lRV16/n9ptT59Q5igjMzMyKGNbsAMzMbOBx8jAzs8KcPMzMrDAnDzMzK8zJw8zMCnPyMDOzwpw8zFqcpOMl/b7ZcZhVc/KwIUfSfpL+IGmhpBcl3S5pjybHdIak5ZJekfRyim/vPrzPbyV9sowYzao5ediQImkT4Drgu8AWwJuAM4HXC77PiP6PjqsjYiNgAvB74KeSVMJ1zNabk4cNNTsDRMSVEbEyIl6NiF9HxH2VAyR9StLDkhZLekjSbql8rqSvSboPWCJphKS9Ui3hZUn3Stq/6n02lXSxpOckPSvpm5KG9xZgRCwHLgPeAIyr3S9pH0mzUs1plqR9UvlU4N3A91IN5nvr84Uyq8fJw4aaR4GVki6TdLCkzat3SjoCOAM4FtgEOBRYUHXI0cCHgc2ALYFfAt8kq8V8GfiJpAnp2EuBFcCOwDuBA4FebylJGgUcD8yLiBdq9m2RrnkeWWI5B/ilpHERcRpwG3BSRGwUESf1/uUw6xsnDxtSImIRsB8QwP8A8yXNkLRlOuSTwLciYlZk5kTEU1VvcV5EzIuIV4FjgOsj4vqIWBURNwFdwCHp/Q4BvhgRSyLieeBc4Kg64X1M0svAPGB34LBujvkw8FhEXB4RKyLiSuAR4KN9+4qY9U0Z923NWlpEPEz2nz2SdgGuAL5NVqvYBni8zunzqtYnAkdIqv7DvQHwm7RvA+C5qmaLYTXn17omIo7pJfw3Ak/VlD1F1nZj1jBOHjakRcQjki4FPp2K5gE71Dulan0ecHlEfKr2IElbkTXCj4+IFf0ULsBfyBJTtW2BX3UTn1lpfNvKhhRJu0j6kqSt0/Y2ZDWOP6VDLgK+LGl3ZXaUVPvHuuIK4KOSPiRpuKTRkvaXtHVEPAf8GvhvSZtIGiZpB0nvXc+PcD2ws6R/SA32RwKTyHqQAfwV2H49r2HWKycPG2oWA+8C7pC0hCxpPAB8CSAifgxMBX6Ujv05WWP4OiJiHjAZ+Down6wm8hXW/F4dC4wEHgJeAq4Ftlqf4CNiAfCRFO8C4KvAR6oa1r8DHC7pJUnnrc+1zOqRJ4MyM7OiXPMwM7PCnDzMzKwwJw8zMyvMycPMzAobNM95jB8/Ptra2podhpnZgHLXXXe9EBETej9ybYMmebS1tdHV1dXsMMzMBhRJtSMW5OLbVmZmVpiTh5mZFebkYWZmhTl5mJlZYU4eZmZWmJNHZye0tcGwYdlrZ2ezIzIza3mDpqtun3R2wpQpsHRptv3UU9k2QEdH8+IyM2txQ7vmcdppaxJHxdKlWbmZmfVoaCePp58uVm5mZsBQTx7bblus3MzMgKGePKZOhQ02WLtszJis3MzMejS0k0dHBxx55JrtiRNh+nQ3lpuZ9WJoJw+APffMXk86CebOdeIwM8vByWNY+hKsWtXcOMzMBhAnDycPM7PCnDyk7NXJw8wsNycP1zzMzApz8qgkj4jmxmFmNoCUmjwkHSRptqQ5kk7pZv8oSVen/XdIakvlG0i6TNL9kh6WdGppQbrmYWZWWGnJQ9Jw4HzgYGAScLSkSTWHnQC8FBE7AucCZ6fyI4BREfE2YHfg05XE0u+cPMzMCiuz5rEnMCcinoiIZcBVwOSaYyYDl6X1a4EDJAkIYKykEcCGwDJgUSlROnmYmRVWZvJ4EzCvavuZVNbtMRGxAlgIjCNLJEuA54Cngf+KiBdrLyBpiqQuSV3z58/vW5ROHmZmhbVqg/mewErgjcB2wJckbV97UERMj4j2iGifMGFC367k5GFmVliZyeNZYJuq7a1TWbfHpFtUmwILgH8AfhURyyPieeB2oL2UKJ08zMwKKzN5zAJ2krSdpJHAUcCMmmNmAMel9cOBWyIiyG5VvR9A0lhgL+CRUqJ08jAzK6y05JHaME4CbgQeBq6JiAclnSXp0HTYxcA4SXOAk4FKd97zgY0kPUiWhC6JiPtKCdTJw8yssFLnMI+I64Hra8pOr1p/jaxbbu15r3RXXgonDzOzwlq1wbxxnDzMzArrseYh6X6y5y26FRG7lhJRozl5mJkVVu+21UfS62fT6+XpdXDNluTkYWZWWI/JIyKeApD0wYh4Z9WuUyTdzZrG7YHNycPMrLA8bR6StG/Vxj45zxsYnDzMzArL09vqBOAHkjZN2y8DnygtokZz8jAzK6zX5BERdwFvrySPiFhYelSN5JkEzcwK6/X2k6QtJV0MXBURCyVNknRCA2JrDNc8zMwKy9N2cSnZU+JvTNuPAl8sKZ7G80yCZmaF5Uke4yPiGmAVrB52ZGWpUTWSax5mZoXlSR5LJI0jPTAoaS+yeTcGBycPM7PC8vS2Opls9NsdJN0OTCAbAXdwcPIwMyssT2+ruyW9F3gzIGB2RCwvPbJGcfIwMyss76i6ewJt6fjdJBERPywtqkZy8jAzK6zX5CHpcmAH4B7WNJQH4ORhZjZE5al5tAOT0gx/g4+Th5lZYXl6Wz0AvKHsQJrGycPMrLA8NY/xwEOS7gRerxRGxKE9nzKAOHmYmRWWJ3mcUXYQTeXkYWZWWJ6uurc2IpCmcfIwMyus3jS0v4+I/SQtZu3paAVERGxSenSN4ORhZlZYvZkE90uvGzcunCZw8jAzKyzvQ4JI+htgdGU7Ip4uJaJGc/IwMyssz3weh0p6DHgSuBWYC9xQclyN48mgzMwKy/Ocx78CewGPRsR2wAHAn0qNqpFc8zAzKyxP8lgeEQuAYZKGRcRvyJ46Hxw8GZSZWWF52jxelrQR8DugU9LzwJJyw2og1zzMzArLU/OYDLwK/DPwK+Bx4KNlBtVQTh5mZoXleUiwupZxWYmxNIeTh5lZYfUeEuz24UD8kKCZ2ZBX7yHBwf1wYIWTh5lZYbkeEpS0G7AfWc3j9xHx51KjaiQnDzOzwvI8JHg6WVvHOLLh2S+V9I2yA2sYJw8zs8Ly1Dw6gLdHxGsAkv6DbErab5YYV+M4eZiZFZanq+5fqBrTChgFPFtOOE1w7bXZ68KF0NYGnZ1NDcfMbCDIU/NYCDwo6SayNo8PAndKOg8gIj5fYnzl6uyEz1eF/9RTMGVKtt7R0ZyYzMwGAEUvw3JIOq7e/ohoiWc/2tvbo6urq9hJbW1Zwqg1cSLMndsfYZmZtTRJd0VE4SGn8tQ8boiI52su9uaImJ0jqIOA7wDDgYsi4j9q9o8CfgjsDiwAjoyIuWnfrsD3gU2AVcAelXaXfvN0D6PK91RuZmZAvjaP2yR9rLIh6UvAz3o7SdJw4HzgYGAScLSkSTWHnQC8FBE7AucCZ6dzRwBXACdGxFuA/YHlOWItZttti5WbmRmQL3nsD/yjpB9L+h2wM7BnjvP2BOZExBMRsQy4imycrGqTWTPkybXAAZIEHAjcFxH3AkTEgohYmeOaxUydCmPGrF02ZkxWbmZmPeo1eUTEc2QDIu4NtAGXRcQrOd77TcC8qu1nUlm3x0TECrLG+XFkCSok3SjpbklfzXG94jo64MIL12xPnAjTp7ux3MysF722eUi6may77luBbYCLJf0uIr5cclz7AXsAS4GZqVFnZk1sU4ApANv29VbTMcfAscdm608+uWZmQTMz61Ge21bfi4hjI+LliLgf2IeshtCbZ8mSTcXWrPt8yOpjUjvHpmQN588Av4uIFyJiKXA9sFvtBSJiekS0R0T7hAkTcoTUjepk4QmhzMxy6TF5SNoFICJ+nnpFkbZXADfleO9ZwE6StpM0EjgKmFFzzAyg0hX4cOCWyPoO3wi8TdKYlFTeCzyU8zMV59kEzcwKqVfz+FHV+h9r9k3r7Y1TkjmJLBE8DFwTEQ9KOkvSoemwi4FxkuYAJwOnpHNfAs4hS0D3AHdHxC97/zh95CFKzMwKqdfmoR7Wu9vuVkRcT3bLqbrs9Kr114Ajejj3CrLuuuVz8jAzK6RezSN6WO9ue2Bz8jAzK6RezWPrNH6VqtZJ27Vdbgc2Jw8zs0LqJY+vVK3XDhpVcBCpFufkYWZWSL1paFtiwMOGcPIwMyskz3Meg5+Th5lZIU4e4ORhZlaQkwc4eZiZFdRr8pC0s6SZkh5I27tK+kb5oTWQk4eZWSF5ah7/A5xKmk8jIu4jG2pk8HDyMDMrJE/yGBMRd9aUrSgjmKZx8jAzKyRP8nhB0g6kp8olHQ48V2pUjebkYWZWSJ45zD8LTAd2kfQs8CRwTKlRNVplWHYnDzOzXHpNHhHxBPABSWOBYRGxuPywGsw1DzOzQvL0tvo3SZtFxJKIWCxpc0nfbERwDePkYWZWSJ42j4Mj4uXKRppr45DSImoGTwZlZlZInuQxvHomQUkbAqPqHD/wuOZhZlZIngbzTmCmpEvS9seBwTVoopOHmVkheRrMz5Z0H3BAKvrXiLix3LAazMnDzKyQPDUPIuIG4IaSY2keJw8zs0Ly9Lb6O0mPSVooaZGkxZIWNSK4hnHyMDMrJE/N41vARyPi4bKDaRonDzOzQvL0tvrroE4c4ORhZlZQnppHl6SrgZ8Dr1cKI+KnZQXVcE4eZmaF5EkemwBLgQOrygJw8jAzG6LydNX9eCMCaSonDzOzQnpNHpJGAycAbwFGV8oj4hMlxtVYTh5mZoXkaTC/HHgD8CHgVmBrYHCNrOvkYWZWSJ7ksWNE/AuwJCIuAz4MvKvcsBrMycPMrJA8yWN5en1Z0luBTYG/KS+kJvBkUGZmheTpbTVd0ubAvwAzgI2A00uNqtFc8zAzKyRPb6uL0uqtwPblhtMkTh5mZoX0mDwkHRMRV0g6ubv9EXFOeWE1mJOHmVkh9WoeY9Prxo0IpKk8k6CZWSE9Jo+I+L6k4cCiiDi3gTE1nmseZmaF1O1tFRErgaMbFEvzOHmYmRWSp7fV7ZK+B1wNLKkURsTdpUXVaE4eZmaF5Eke70ivZ1WVBfD+fo+mWZw8zMwKydNV932NCKRpOjvhppuy9U99Cl55BTo6mhuTmVmLy/OEOZI+LOmrkk6vLDnPO0jSbElzJJ3Szf5Rkq5O+++Q1Fazf1tJr0j6cq5PU1RnJ0yZAq++mm3Pn59td3aWcjkzs8EizxzmFwJHAp8DBBwBTMxx3nDgfOBgYBJwtKRJNYedALwUETsC5wJn1+w/B7iht2v12WmnwdKla5ctXZqVm5lZj/LUPPaJiGPJ/sifCewN7JzjvD2BORHxREQsA64CJtccMxm4LK1fCxwgZQNNSfo/wJPAgzmu1TdPP12s3MzMgHzJI93TYamkN5INlLhVjvPeBMyr2n4mlXV7TESsABYC4yRtBHwNOLPeBSRNkdQlqWv+/Pk5Qqqx7bbFys3MDMiXPK6TtBnwn8DdwFzgRyXGBHAGcG5EvFLvoIiYHhHtEdE+YcKE4leZOhXGjFm7bMyYrNzMzHpUb2yr68mSROWP+E8kXQeMjoiFOd77WWCbqu2tU1l3xzwjaQTZcO8LyOYLOVzSt4DNgFWSXouI7+X7WDlVelVNmbKm7WPDDfv1EmZmg1G9msf3ySZ+ekLSNZIOAyJn4gCYBewkaTtJI4GjyIZ0rzYDOC6tHw7cEpl3R0RbRLQB3wb+rd8TR7UVK9asL1jgHldmZr3oMXlExC8i4migDfgJcCzwtKRLJH2wtzdObRgnATcCDwPXRMSDks6SdGg67GKyNo45wMnAOt15S3faabBs2dpl7nFlZlaXosBIspJ2JesdtWtEDC8tqj5ob2+Prq6u4icOG9b9aLqSnzg3s0FP0l0R0V70vDzPeWwp6XOSbgd+TlaT2K14iC3KPa7MzArrMXlI+pSkW8h6WO0EfCUito+IUyLi3oZFWLapU2H06LXL3OPKzKyuemNb7Q38OzAzIgbv/ZuOjuyhwK9/PdueODFLHB7fysysR/UazD8RETcN6sRRcdhh2eub3wxz5zpxmJn1ItfAiIPeBhtkr7W9rszMrFv12jy2a2QgTTVyZPbq5GFmlku9mse1AJJmNiiW5nHyMDMrpF6D+TBJXwd2lnRy7c6IOKe8sBqscttq+fLmxmFmNkDUq3kcBawkSzAbd7MMHq55mJkV0mPNIyJmA2dLui8iypuQqRU4eZiZFZKnt9UfJJ1TmTdD0n9L2rT0yBqpcttqxYruhyoxM7O15EkePwAWAx9LyyLgkjKDajgJRqRKmNs9zMx6Va/BvGKHiPj7qu0zJd1TUjzNM3JkVvNYtmzNbSwzM+tWrmloJe1X2ZC0L2umph0cOjvh1fSR/vZvPZeHmVkv8tQ8TgR+WNXO8RJrJnAa+Do7s8mfKm0dzzyTbYOHKTEz60Hu+TwkbQIQEYtKjaiP+jyfR1sbPPXUuuUTJ2bjXJmZDWJ9nc8jT80DaN2ksd6efrpYuZmZeWDEHid92mKLxsZhZjaAOHlMnbrmOY9qixe74dzMrAe52jwk7QO0UXWbKyJ+WF5YxfW5zQNg/HhYsGDdcrd7mNkgV1qbh6TLgR2Ae8jGugIIoKWSx3p58cXuy93uYWbWrTwN5u3ApMjbLWsg2nbb7ntc9dQeYmY2xOVp83gAeEPZgTTV1KkwatTaZWPGZOVmZraOPMljPPCQpBslzagsZQfWUB0d8OEPr9kePhyOO84PCZqZ9SDPbaszyg6i6To74frr12yvXAmXXQb77usEYmbWjV5rHhFxK/AIayaBejiVDR6nnQavvbZ22dKlWbmZma2j1+Qh6WPAncARZEOy3yHp8LIDayg/ZW5mVkie21anAXtExPMAkiYANwPXlhlYQ7m3lZlZIXkazIdVEkeyIOd5A8fUqVnvqmrubWVm1qM8SeBXqafV8ZKOB34JXN/LOQNLR0fWu6pi2DD3tjIzqyNPg/lXgOnArmmZHhFfKzuwhurszHpXVaxalW17bCszs27lns+j1a3X2FY9zekxbhy88MJ6xWVm1sr6OrZVjzUPSb9Pr4slLapaFksaXHN79NSrasEC1z7MzLrRY/KIiP3S68YRsUnVsnFEbNK4EBugXq8qP+thZraOPM95XJ6nbECr16vKz3qYma0jT2+rt1RvSBoB7F5OOE3S0QFjx3a/r7YLr5mZ1W3zOFXSYmDX6vYO4K/ALxoWYaOMHt19+ZIlbvcwM6tRr83j34FNgR/WtHeMi4hT87y5pIMkzZY0R9Ip3ewfJenqtP8OSW2p/IOS7pJ0f3p9fx8/X349TQgFbvcwM6tR97ZVRKwC9ujLG0saDpwPHAxMAo6WNKnmsBOAlyJiR+Bc4OxU/gLw0Yh4G3AcUH4bS71Gc7d7mJmtJU+bx92S+pJA9gTmRMQTEbEMuAqYXHPMZKDydN61wAGSFBF/joi/pPIHgQ0l1czW1M+mTgWp+30e48rMbC15kse7gD9KelzSfelW0n05znsTMK9q+5lU1u0xEbECWAiMqznm74G7I+L12gtImiKpS1LX/Pnzc4RUR0cHvL+Hu2OHHLJ+721mNsjkGVX3Q6VH0QNJbyG7lXVgd/sjYjrZ0Cm0t7ev/6Pyc+Z0X3794BrKy8xsfeUZ2+opYDPgo2nZLJX15llgm6rtrVNZt8ekLsCbko3ai6StgZ8Bx0bE4zmut/66G6KkXrmZ2RCV5yHBLwCdwN+k5QpJn8vx3rOAnSRtJ2kkcBRQO/f5DLIGcYDDgVsiIiRtRjZ67ykRcXuuT9Ifhg/veZ+765qZrdbrwIipfWPviFiStscCf4yIXXt9c+kQ4NvAcOAHETFV0llAV0TMkDSarCfVO4EXgaMi4glJ3wBOBR6rersDa+YVWct6DYy4JuCe902cCHPnrt/7m5m1mL4OjJgnedxPNpPga2l7NDArdaNtGf2SPHoaXbdikIxAbGZW0e+j6la5hGze8jMknQn8Cbi46IUGhHpjXNW7pWVmNsTkaTA/B/g42W2lF4CPR8S3S46rOerNHLhyZePiMDNrcUXmIlfN6+DUUw3DNQ8zs9Xy9LY6newp8M2B8cAlqUF7cOqphuGah5nZanlqHh1kDeZnRMT/BfYC/rHcsJpo4sSe933mM42Lw8ysheVJHn8BqscrH8W6D/sNHvUazS+8sHFxmJm1sDzJYyHwoKRLJV0CPAC8LOk8SeeVG14T1Gs0j/DDgmZm5Bvb6mdpqfhtOaG0kOHDe27j+PSn6ycYM7MhoNfkERGXpeFFdk5FsyNieblhNdmUKXDBBd3vW7KksbGYmbWgPL2t9icbJuR8YBrwqKT3lBtWk02bVn+/b12Z2RCXp83jv8nGlXpvRLyHbIj2c8sNqwUMq/Ol+fSnGxeHmVkLypM8NoiI2ZWNiHgU2KC8kFpEvQSxZIlrH2Y2pOVJHndJukjS/mn5H2A9RyAcAHq7dXXaaY2Jw8ysBeVJHicCDwGfT8tDwD+VGVTLGFc7I24VTxBlZkNY3d5WkoYD90bELsA5jQmphXznO3DMMc2Owsys5dSteUTESmC2pG0bFE9r6e15Dg9XYmZDVJ6HBDcne8L8TmD1Qw4RcWhpUQ0UF1wA++7rhwbNbMjJkzz+pfQoWtm4cbBgQc/7v/AFJw8zG3J6TB5putkTgR2B+4GLI2JFowJrGb21e9RLLGZmg1S9No/LgHayxHEw2cOCQ09HB2y0Uf1j3PZhZkNMveQxKSKOiYjvA4cD725QTK2nt6HYL7jADw2a2ZBSL3msHvxwSN6uqpanTeMLXyg/DjOzFlEvebxd0qK0LAZ2raxLWtSoAFtGvRkGIWv7cO3DzIaIHpNHRAyPiE3SsnFEjKha36SRQbaEejMMVnziE+XHYWbWAvIMT2KQ3br6p15GZVm2DDbYwDUQMxv0nDyKmDat/lDtACtWZF173QPLzAYxJ4+i8s7lccEF8IEPlBuLmVmTOHkUNW0aHHBAvmNnznQCMbNBycmjL26+GUaPznfszJluBzGzQcfJo68uuij/sZV2ECcRMxsknDz6qqMDrrgChg/Pf04liUhuUDezAc3JY310dGQJIW8bSLULLnASMbMBy8mjP9x8c+/PgPSkkkQk2Hhj39YyswHByaO/TJvW9wRS8cora25rVZYNN3RCMbOW4+TRn6ZNK94O0pvXXls3oQwb5ttdZtZUTh79rdIO0t9JpFrE2re76i2uuZhZCZw8ylKdREaObF4c3dVcWn1xzcqs5ZWaPCQdJGm2pDmSTulm/yhJV6f9d0hqq9p3aiqfLelDZcZZqo4OeP31rLbQl15ZQ1GRmpUXL16ypcH/dJWWPCQNB84nm8J2EnC0pEk1h50AvBQROwLnAmencycBRwFvAQ4CpqX3G9huvtlJxMzKUfmnq0EJpMyax57AnIh4IiKWAVcBk2uOmUw2VzrAtcABkpTKr4qI1yPiSWBOer/BoZJEIta/h5aZWbXp0xtymTKTx5uAeVXbz6Sybo9JU90uBMblPBdJUyR1SeqaP39+P4beQNOmrUkklaXZ7SRmNnCtXNmQywzoBvOImB4R7RHRPmHChGaH03+q20mqE8rYsc2OzMxaXVm9PGuUmTyeBbap2t46lXV7jKQRwKbAgpznDi0dHdlDhLW1lO4W11zMhq4pUxpymTKTxyxgJ0nbSRpJ1gA+o+aYGcBxaf1w4JaIiFR+VOqNtR2wE3BnibEOLt3VXFp9cc3KbP1IWRvqtGkNudyIst44IlZIOgm4ERgO/CAiHpR0FtAVETOAi4HLJc0BXiRLMKTjrgEeAlYAn42IxtzIs+bo6MgWMxsQlP2jP/C1t7dHV1dXs8MwMxtQJN0VEe1FzxvQDeZmZtYcTh5mZlaYk4eZmRXm5GFmZoUNmgZzSfOBp/p4+njghX4Mp785vr5r5djA8a2PVo4NBk58EyOi8FPWgyZ5rA9JXX3pbdAojq/vWjk2cHzro5Vjg8Efn29bmZlZYU4eZmZWmJNHpjFjGPed4+u7Vo4NHN/6aOXYYJDH5zYPMzMrzDUPMzMrzMnDzMwKG/LJQ9JBkmZLmiPplCbF8ANJz0t6oKpsC0k3SXosvW6eyiXpvBTvfZJ2Kzm2bST9RtJDkh6U9IUWi2+0pDsl3ZviOzOVbyfpjhTH1WlaANIw/1en8jsktZUZX7rmcEl/lnRdC8Y2V9L9ku6R1JXKWuJ7m665maRrJT0i6WFJe7dCfJLenL5mlWWRpC+2QmxVMf5z+p14QNKV6Xel/372ImLILmRDxT8ObA+MBO4FJjUhjvcAuwEPVJV9CzglrZ8CnJ3WDwFuAATsBdxRcmxbAbul9Y2BR4FJLRSfgI3S+gbAHem61wBHpfILgX9K658BLkzrRwFXN+D7ezLwI+C6tN1Ksc0FxteUtcT3Nl3zMuCTaX0ksFkrxZeuOxz4X2Biq8RGNm33k8CGVT9zx/fnz17pX9hWXoC9gRurtk8FTm1SLG2snTxmA1ul9a2A2Wn9+8DR3R3XoDh/AXywFeMDxgB3A+8ie3J2RO33mWx+mb3T+oh0nEqMaWtgJvB+4Lr0x6MlYkvXmcu6yaMlvrdkM4s+Wfs1aJX4qq5zIHB7K8VGljzmAVukn6XrgA/158/eUL9tVfkCVzyTylrBlhHxXFr/X2DLtN60mFNV9p1k/923THzpttA9wPPATWS1yZcjYkU3MayOL+1fCIwrMbxvA18FVqXtcS0UG0AAv5Z0l6TK/KWt8r3dDpgPXJJu+10kaWwLxVdxFHBlWm+J2CLiWeC/gKeB58h+lu6iH3/2hnryGBAi+3egqX2qJW0E/AT4YkQsqt7X7PgiYmVEvIPsv/w9gV2aFUs1SR8Bno+Iu5odSx37RcRuwMHAZyW9p3pnk7+3I8hu514QEe8ElpDdClqt2T97qc3gUODHtfuaGVtqa5lMloDfCIwFDurPawz15PEssE3V9taprBX8VdJWAOn1+VTe8JglbUCWODoj4qetFl9FRLwM/IasOr6ZpMo0y9UxrI4v7d8UWFBSSPsCh0qaC1xFduvqOy0SG7D6P1Qi4nngZ2TJt1W+t88Az0TEHWn7WrJk0irxQZZ0746Iv6btVontA8CTETE/IpYDPyX7eey3n72hnjxmATulHggjyaqfM5ocU8UM4Li0fhxZW0Ol/NjUe2MvYGFVNbnfSRLZXPMPR8Q5LRjfBEmbpfUNydpjHiZLIof3EF8l7sOBW9J/iP0uIk6NiK0joo3sZ+uWiOhohdgAJI2VtHFlneze/QO0yPc2Iv4XmCfpzanoAOChVokvOZo1t6wqMbRCbE8De0kak36HK1+7/vvZK7sxqdUXsl4Qj5LdJz+tSTFcSXZfcjnZf1snkN1vnAk8BtwMbJGOFXB+ivd+oL3k2PYjq3rfB9yTlkNaKL5dgT+n+B4ATk/l2wN3AnPIbimMSuWj0/actH/7Bn2P92dNb6uWiC3FcW9aHqz8/LfK9zZd8x1AV/r+/hzYvFXiI7sVtADYtKqsJWJL1zwTeCT9XlwOjOrPnz0PT2JmZoUN9dtWZmbWB04eZmZWmJOHmZkV5uRhZmaFOXmYmVlhTh7WsiStTCOWPiDpx5LG9HDcH/r4/u2SzluP+F7pofwNkq6S9Hga9uN6STv39TqtQNL+kvZpdhzWOpw8rJW9GhHviIi3AsuAE6t3Vp6UjYg+/VGLiK6I+Pz6h7lWTCJ7Uvu3EbFDROxONuDmlvXPbHn7A04etpqThw0UtwE7pv+Ab5M0g+yJ2dU1gLTvt1oz/0Nn+mOOpD0k/UHZvB93Sto4HV+ZY+MMSZdL+qOyuRg+lco3kjRT0t3K5r2Y3Euc7wOWR8SFlYKIuDcibktPF/9nqkndL+nIqrhvlfQLSU9I+g9JHSnO+yXtkI67VNKFkrokPaps7KzKnCaXpGP/LOl9qfx4ST+V9Kv0mb5ViUnSgemz3p1qdRul8rmSzqz6vLsoGxDzROCfU03w3ev5vbRBYETvh5g1V6phHAz8KhXtBrw1Ip7s5vB3Am8B/gLcDuwr6U7gauDIiJglaRPg1W7O3ZVsroWxwJ8l/ZJsbKLDImKRpPHAnyTNiJ6frn0r2eil3fk7siem3w6MB2ZJ+l3a93bgb4EXgSeAiyJiT2WTb30O+GI6ro1s/KkdgN9I2hH4LNk4fG+TtAvZKLmV22TvSF+T14HZkr6bPvs3gA9ExBJJXyObc+SsdM4LEbGbpM8AX46IT0q6EHglIv6rh89mQ4yTh7WyDZUNtQ5ZzeNislsnd/aQOEj7ngFI57aRDS/9XETMAog0KnCqlFT7RUS8Crwq6Tdkf6R/CfybstFmV5ENXb0l2XDbRe0HXBkRK8kG0LsV2ANYBMyKNNaRpMeBX6dz7ierzVRcExGrgMckPUE2gvB+wHfTZ3tE0lNAJXnMjIiF6X0fIpuwaDOyCb1uT1+DkcAfq65RGfzyLrKEZ7YOJw9rZa9GNtT6aumP3ZI657xetb6SYj/jtbWJADqACcDuEbFc2Qi5o+u8x4OsGXiuiOq4V1Vtr2Ltz9BdjHnft/L1EHBTRBzdyzlFv342hLjNw4aC2cBWkvYASO0d3f1RnJzaD8aRNRDPIhua+vmUON5H9p97PbcAo7RmYiUk7ZraCW4DjlQ2edUEsumH7yz4WY6QNCy1g2yfPtttZEmOdLtq21Tekz+R3c7bMZ0zNkdvsMVk0xCbAU4eNgRExDLgSOC7ku4lm22wu9rDfWRDVv8J+NeI+AvQCbRLuh84lmyU0nrXCuAw4APKuuo+CPw72W2un6Vr3EuWZL4a2bDjRTxNlnBuAE6MiNeAacCwFOPVwPER8XpPbxAR88nms75S0n1kt6x6m0Dr/wGHucHcKjyqrhlZbytavEFY0qVkw7pf2+xYzFzzMDOzwlzzMDOzwlzzMDOzwpw8zMysMCcPMzMrzMnDzMwKc/IwM7PC/j/3O5XOVIi+agAAAABJRU5ErkJggg==\n",
886 | "text/plain": [
887 | "