├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── data ├── movies.csv ├── ratings.csv └── user_item_matrix.npz ├── docs ├── Makefile ├── conf.py ├── index.rst └── intro.rst ├── images ├── cb-filtering-1.png ├── cb-filtering.png ├── cf-filtering.png ├── knn.png ├── matrix-factorization.png ├── movie-features-matrix.png ├── user-movie-matrix.png └── utility-matrix.png ├── part-1-item-item-recommender.ipynb ├── part-2-cold-start-problem.ipynb ├── part-3-implicit-feedback-recommender.ipynb ├── presentation ├── images │ ├── amazon-ecommerce.png │ ├── amazon-example.png │ ├── bookstore.png │ ├── collaborative-filtering.png │ ├── content-based-filtering.png │ ├── cosine-sim.png │ ├── gypsy-musical.png │ ├── knn.png │ ├── lamerica.png │ ├── long-tail-book.png │ ├── medium-example.png │ ├── netflix-example.png │ ├── recommender-examples.png │ ├── recommender-ml-1.png │ ├── recommender-ml-2.png │ ├── recommender-ml-3.png │ ├── recommender-ml-4.png │ ├── spotify-example.png │ ├── tasting-booth.png │ └── utility-matrix.png ├── intro_to_recommenders_slides.ipynb └── slides.html ├── recommender-basics.md └── utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints 2 | *.DS_Store 3 | *.npz -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | install: 2 | - pip install jupyter 3 | - cd presentation 4 | - wget https://github.com/hakimel/reveal.js/archive/master.zip 5 | - unzip master.zip 6 | - mv reveal.js-master reveal.js 7 | 8 | script: 9 | - jupyter nbconvert intro_to_recommenders_slides.ipynb --to slides 10 | 11 | after_success: | 12 | if [ -n "$GITHUB_API_KEY" ]; then 13 | git checkout --orphan gh-pages 14 | git rm -rf --cached . 15 | mv intro_to_recommenders_slides.slides.html index.html 16 | git add -f --ignore-errors index.html images reveal.js 17 | git -c user.name='travis' -c user.email='travis' commit -m init 18 | git push -f -q https://$GITHUB_USER:$GITHUB_API_KEY@github.com/$TRAVIS_REPO_SLUG gh-pages 19 | fi -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2018, Jill Cates 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Recommendation Systems 101 2 | 3 | This series of tutorials explores different types of recommendation systems and their implementations. Topics include: 4 | 5 | - collaborative vs. content-based filtering 6 | - implicit vs. explicit feedback 7 | - handling the cold start problem 8 | - recommendation model evaluation 9 | 10 | We will build various recommendation systems using data from the [MovieLens](https://movielens.org/) database. You will need Jupyter Lab to run the notebooks for each part of this series. Alternatively, you can also use Google’s new [colab platform](https://colab.research.google.com) which allows you to run a Jupyter notebook environment in the cloud. You won't need to install any local dependencies; however, you will need a gmail account. 11 | 12 | The series is divided into 3 parts: 13 | 14 | 1. [Building an Item-Item Recommender with Collaborative Filtering](#part-1-building-an-item-item-recommender-with-collaborative-filtering) 15 | 2. [Handling the Cold Start Problem with Content-based Filtering](#part-2-handling-the-cold-start-problem-with-content-based-filtering) 16 | 3. [Building an Implicit Feedback Recommender System](#part-3-building-an-implicit-feedback-recommender-system) 17 | 18 | 19 | More information on each part can be found in the descriptions below. 20 | 21 | ### Part 1: Building an Item-Item Recommender with Collaborative Filtering 22 | 23 | | |Description | 24 | |:-----------|:----------| 25 | |Objective|Want to know how Spotify, Amazon, and Netflix generate "similar item" recommendations for users? In this tutorial, we will build an item-item recommendation system by computing similarity using nearest neighbor techniques.| 26 | |Key concepts|collaborative filtering, content-based filtering, k-Nearest neighbors, cosine similarity| 27 | |Requirements|Python 3.6+, Jupyter Lab, numpy, pandas, matplotlib, seaborn, scikit-learn| 28 | |Tutorial link|[Jupyter Notebook](part-1-item-item-recommender.ipynb)| 29 | |Resources|[Item-item collaborative filtering](https://www.wikiwand.com/en/Item-item_collaborative_filtering), [Amazon.com Recommendations](https://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf), [Various Implementations of Collaborative Filtering](https://towardsdatascience.com/various-implementations-of-collaborative-filtering-100385c6dfe0) | 30 | 31 | 32 | ### Part 2: Handling the Cold Start Problem with Content-based Filtering 33 | 34 | | |Description | 35 | |:-----------|:----------| 36 | |Objective|Collaborative filtering fails to incorporate new users who haven't rated yet and new items that don't have any ratings or reviews. This is called the cold start problem. In this tutorial, we will learn about clustering techniques that are used to tackle the cold start problem of collaborative filtering.| 37 | |Requirements|Python 3.6+, Jupyter Lab, numpy, pandas, matplotlib, seaborn, scikit-learn| 38 | |Tutorial link|[Jupyter Notebook](part-2-cold-start-problem.ipynb)| 39 | 40 | 41 | ### Part 3: Building an Implicit Feedback Recommender System 42 | 43 | | |Description | 44 | |:-----------|:----------| 45 | |Objective|Unlike explicit feedback (e.g., user ratings), implicit feedback infers a user's degree of preference toward an item by looking at their indirect interactions with that item. In this tutorial, we will investigate a recommender model that specifically handles implicit feedback datasets.| 46 | |Requirements|Python 3.6+, Jupyter Lab, numpy, pandas, implicit| 47 | |Tutorial link|[Jupyter Notebook](part-3-implicit-feedback-recommender.ipynb)| -------------------------------------------------------------------------------- /data/user_item_matrix.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/data/user_item_matrix.npz -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SPHINXPROJ = recommender-tutorial 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Configuration file for the Sphinx documentation builder. 4 | # 5 | # This file does only contain a selection of the most common options. For a 6 | # full list see the documentation: 7 | # http://www.sphinx-doc.org/en/master/config 8 | 9 | # -- Path setup -------------------------------------------------------------- 10 | 11 | # If extensions (or modules to document with autodoc) are in another directory, 12 | # add these directories to sys.path here. If the directory is relative to the 13 | # documentation root, use os.path.abspath to make it absolute, like shown here. 14 | # 15 | # import os 16 | # import sys 17 | # sys.path.insert(0, os.path.abspath('.')) 18 | import nbsphinx 19 | 20 | # -- Project information ----------------------------------------------------- 21 | 22 | project = 'recommender-tutorial' 23 | copyright = '2018, Jill Cates' 24 | author = 'Jill Cates' 25 | 26 | # The short X.Y version 27 | version = '' 28 | # The full version, including alpha/beta/rc tags 29 | release = '0.1' 30 | 31 | 32 | # -- General configuration --------------------------------------------------- 33 | 34 | # If your documentation needs a minimal Sphinx version, state it here. 35 | # 36 | # needs_sphinx = '1.0' 37 | 38 | # Add any Sphinx extension module names here, as strings. They can be 39 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 40 | # ones. 41 | extensions = [ 42 | 'sphinx.ext.napoleon', 43 | 'sphinx.ext.mathjax', 44 | 'sphinx.ext.githubpages', 45 | 'nbsphinx', 46 | 'sphinx.ext.viewcode' 47 | ] 48 | 49 | # Add any paths that contain templates here, relative to this directory. 50 | templates_path = ['_templates'] 51 | 52 | # The suffix(es) of source filenames. 53 | # You can specify multiple suffix as a list of string: 54 | # 55 | # source_suffix = ['.rst', '.md'] 56 | source_suffix = '.rst' 57 | 58 | # The master toctree document. 59 | master_doc = 'index' 60 | 61 | # The language for content autogenerated by Sphinx. Refer to documentation 62 | # for a list of supported languages. 63 | # 64 | # This is also used if you do content translation via gettext catalogs. 65 | # Usually you set "language" from the command line for these cases. 66 | language = None 67 | 68 | # List of patterns, relative to source directory, that match files and 69 | # directories to ignore when looking for source files. 70 | # This pattern also affects html_static_path and html_extra_path . 71 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] 72 | 73 | # The name of the Pygments (syntax highlighting) style to use. 74 | pygments_style = 'sphinx' 75 | 76 | 77 | # -- Options for HTML output ------------------------------------------------- 78 | 79 | # The theme to use for HTML and HTML Help pages. See the documentation for 80 | # a list of builtin themes. 81 | # 82 | html_theme = 'sphinx_rtd_theme' 83 | 84 | # Theme options are theme-specific and customize the look and feel of a theme 85 | # further. For a list of options available for each theme, see the 86 | # documentation. 87 | # 88 | # html_theme_options = {} 89 | 90 | # Add any paths that contain custom static files (such as style sheets) here, 91 | # relative to this directory. They are copied after the builtin static files, 92 | # so a file named "default.css" will overwrite the builtin "default.css". 93 | html_static_path = ['_static'] 94 | 95 | # Custom sidebar templates, must be a dictionary that maps document names 96 | # to template names. 97 | # 98 | # The default sidebars (for documents that don't match any pattern) are 99 | # defined by theme itself. Builtin themes are using these templates by 100 | # default: ``['localtoc.html', 'relations.html', 'sourcelink.html', 101 | # 'searchbox.html']``. 102 | # 103 | # html_sidebars = {} 104 | 105 | 106 | # -- Options for HTMLHelp output --------------------------------------------- 107 | 108 | # Output file base name for HTML help builder. 109 | htmlhelp_basename = 'recommender-tutorialdoc' 110 | 111 | 112 | # -- Options for LaTeX output ------------------------------------------------ 113 | 114 | latex_elements = { 115 | # The paper size ('letterpaper' or 'a4paper'). 116 | # 117 | # 'papersize': 'letterpaper', 118 | 119 | # The font size ('10pt', '11pt' or '12pt'). 120 | # 121 | # 'pointsize': '10pt', 122 | 123 | # Additional stuff for the LaTeX preamble. 124 | # 125 | # 'preamble': '', 126 | 127 | # Latex figure (float) alignment 128 | # 129 | # 'figure_align': 'htbp', 130 | } 131 | 132 | # Grouping the document tree into LaTeX files. List of tuples 133 | # (source start file, target name, title, 134 | # author, documentclass [howto, manual, or own class]). 135 | latex_documents = [ 136 | (master_doc, 'recommender-tutorial.tex', 'recommender-tutorial Documentation', 137 | 'Jill Cates', 'manual'), 138 | ] 139 | 140 | 141 | # -- Options for manual page output ------------------------------------------ 142 | 143 | # One entry per manual page. List of tuples 144 | # (source start file, name, description, authors, manual section). 145 | man_pages = [ 146 | (master_doc, 'recommender-tutorial', 'recommender-tutorial Documentation', 147 | [author], 1) 148 | ] 149 | 150 | 151 | # -- Options for Texinfo output ---------------------------------------------- 152 | 153 | # Grouping the document tree into Texinfo files. List of tuples 154 | # (source start file, target name, title, author, 155 | # dir menu entry, description, category) 156 | texinfo_documents = [ 157 | (master_doc, 'recommender-tutorial', 'recommender-tutorial Documentation', 158 | author, 'recommender-tutorial', 'One line description of project.', 159 | 'Miscellaneous'), 160 | ] 161 | 162 | 163 | # -- Extension configuration ------------------------------------------------- -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. recommender-tutorial documentation master file, created by 2 | sphinx-quickstart on Fri Aug 10 16:53:05 2018. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | An Introduction to Recommendation Systems in Python 7 | =================================================== 8 | 9 | Skill level: Intermediate 10 | 11 | Tutorial requirements 12 | --------------------- 13 | - Python 3.6+ 14 | - Jupyter Lab 15 | - numpy 16 | - scikit-learn 17 | 18 | Alternatively, you can also use Google’s new `colab platform `_ which allows you to run a Jupyter notebook environment in the cloud. You won't need to locally install any of the above; however, you will need a gmail account. 19 | 20 | 21 | Description 22 | ----------- 23 | In this tutorial, we will explore the different types of recommendation systems and their implementations. We will also build our own recommendation system using data from the `MovieLens `_ database. 24 | 25 | .. toctree:: 26 | :maxdepth: 1 27 | :caption: Tutorials: 28 | 29 | part-1-building-from-scratch.ipynb 30 | 31 | .. toctree:: 32 | :maxdepth: 2 33 | :caption: Contents: 34 | 35 | intro.rst 36 | -------------------------------------------------------------------------------- /docs/intro.rst: -------------------------------------------------------------------------------- 1 | What is a recommendation system? 2 | ================================ 3 | 4 | A recommendation system is an algorithm that matches items to users. Its goal is to predict a user's preference toward an item. 5 | 6 | Examples 7 | +++++++++ 8 | - recommending products based on past purchases or product searches (Amazon) 9 | - suggesting TV shows or movies based on prediction of a user's interests (Netflix) 10 | - creating well-curated playlists based on song history (Spotify) 11 | - personalized ads based on "liked" posts or previous websites visited (Facebook) 12 | 13 | The two most commonly used methods for recommendation systems are **collaborative filtering** and **content-based filtering**. 14 | 15 | Collaborative Filtering 16 | ++++++++++++++++++++++++ 17 | 18 | Collaborative filering (CF) is based on the concept of "homophily" - similar users like similar things. It uses item preferences from other users to predict which item a particular user will like best. Collaborative filtering uses a user-item matrix to generate recommendations. This matrix is populated with values that indicate a given user's preference towards a given item. It's very unlikely that a user will have interacted with every item, so in most real-life cases, the user-item matrix is very sparse. 19 | 20 | 21 | .. image:: images/utility-matrix.png 22 | :width: 280px 23 | 24 | Collaborative filtering can be further divided into two categories: memory-based and model-based. 25 | 26 | - **Memory-based** algorithms look at item-item, user-item, or user-user similarity using different similarity metrics such as Pearson correlation coefficient, cosine similarity, etc. This approach is easy to apply to your user-item matrix and very interpretable. However, its performance decreases as the dataset becomes more sparse. 27 | - **Model-based** algorithms use matrix factorization techniques such as Single Vector Decomposition (SVD_) and Non-negative Matrix Factorization (NMF_) to extract latent/hidden, meaningful factors from the data. 28 | 29 | A major disadvantage of collaborative filtering is the **cold start problem**. You can only get recommendations for users and items that already have "interactions" in the user-item matrix. Collaborative filtering fails to provide personalized recommendations for brand new users or newly released items. 30 | 31 | .. _NMF: https://www.wikiwand.com/en/Non-negative_matrix_factorization 32 | .. _SVD: https://www.wikiwand.com/en/Singular-value_decomposition 33 | 34 | Content-based Filtering 35 | ++++++++++++++++++++++++ 36 | 37 | Content-based filtering is a type of supervised learning that generates recommendations based on user and item features. Given a set of item features (movie genre, release date, country, language, etc.), it predicts how a user will rate an item based on their ratings of previous movies. 38 | 39 | Content-based filtering handles the "cold start" problem because it is able to provide personalized recommendations for brand new users and features. 40 | 41 | 42 | .. image:: images/cb-filtering.png 43 | :width: 550px 44 | 45 | 46 | How do we define a user's "preference" towards an item? 47 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 48 | 49 | There are two types of feedback data: 50 | 51 | 1. Explicit feedback, which considers a user's direct response to an item (e.g., rating, like/dislike) 52 | 53 | 2. Implicit feedback, which looks at a user's indirect behaviour towards an item (e.g., number of times a user has watched a movie) 54 | 55 | Before using this data in your recommendation system, it is important to perform some data pre-processing. For example, you should normalize ratings of different users to the same scale. More information on how to normalize data in recommendation systems is described `here `_. -------------------------------------------------------------------------------- /images/cb-filtering-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/cb-filtering-1.png -------------------------------------------------------------------------------- /images/cb-filtering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/cb-filtering.png -------------------------------------------------------------------------------- /images/cf-filtering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/cf-filtering.png -------------------------------------------------------------------------------- /images/knn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/knn.png -------------------------------------------------------------------------------- /images/matrix-factorization.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/matrix-factorization.png -------------------------------------------------------------------------------- /images/movie-features-matrix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/movie-features-matrix.png -------------------------------------------------------------------------------- /images/user-movie-matrix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/user-movie-matrix.png -------------------------------------------------------------------------------- /images/utility-matrix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/images/utility-matrix.png -------------------------------------------------------------------------------- /part-1-item-item-recommender.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Part 1: Building an Item-Item Recommender\n", 8 | "\n", 9 | "If you use Netflix, you will notice that there is a section titled \"Because you watched Movie X\", which provides recommendations for movies based on a recent movie that you've watched. This is a classic example of an item-item recommendation. \n", 10 | "\n", 11 | "In this tutorial, we will generate item-item recommendations using a technique called [collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering). Let's get started! " 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "## Step 1: Import the Dependencies\n", 19 | "\n", 20 | "We will be representing our data as a pandas [DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html). \n", 21 | "\n", 22 | "**What is a DataFrame?**\n", 23 | "\n", 24 | "- a two-dimensional Pandas data structure\n", 25 | "- columns represent features, rows represent items\n", 26 | "- analogous to an Excel spreadsheet or SQL table\n", 27 | "- documentation can be found here\n", 28 | "\n", 29 | "We will also be using two plotting packages: [matplotlib](https://matplotlib.org/) and [seaborn](https://seaborn.pydata.org/) (which is a wrapper of matplotlib) to visualize our data." 30 | ] 31 | }, 32 | { 33 | "cell_type": "code", 34 | "execution_count": 2, 35 | "metadata": {}, 36 | "outputs": [], 37 | "source": [ 38 | "import numpy as np\n", 39 | "import pandas as pd\n", 40 | "import sklearn\n", 41 | "import matplotlib.pyplot as plt\n", 42 | "import seaborn as sns\n", 43 | "\n", 44 | "import warnings\n", 45 | "warnings.simplefilter(action='ignore', category=FutureWarning)" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "## Step 2: Load the Data\n", 53 | "\n", 54 | "Let's download a small version of the [MovieLens](https://www.wikiwand.com/en/MovieLens) dataset. You can access it via the zip file url [here](https://grouplens.org/datasets/movielens/), or directly download [here](http://files.grouplens.org/datasets/movielens/ml-latest-small.zip). We're working with data in `ml-latest-small.zip` and will need to add the following files to our local directory: \n", 55 | "- ratings.csv\n", 56 | "- movies.csv\n", 57 | "\n", 58 | "These are also located in the data folder inside this GitHub repository. \n", 59 | "\n", 60 | "Alternatively, you can access the data here: \n", 61 | "- https://s3-us-west-2.amazonaws.com/recommender-tutorial/movies.csv\n", 62 | "- https://s3-us-west-2.amazonaws.com/recommender-tutorial/ratings.csv\n", 63 | "\n", 64 | "Let's load in our data and take a peek at the structure." 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": 2, 70 | "metadata": {}, 71 | "outputs": [ 72 | { 73 | "data": { 74 | "text/html": [ 75 | "
\n", 76 | "\n", 89 | "\n", 90 | " \n", 91 | " \n", 92 | " \n", 93 | " \n", 94 | " \n", 95 | " \n", 96 | " \n", 97 | " \n", 98 | " \n", 99 | " \n", 100 | " \n", 101 | " \n", 102 | " \n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | "
userIdmovieIdratingtimestamp
0114.0964982703
1134.0964981247
2164.0964982224
31475.0964983815
41505.0964982931
\n", 137 | "
" 138 | ], 139 | "text/plain": [ 140 | " userId movieId rating timestamp\n", 141 | "0 1 1 4.0 964982703\n", 142 | "1 1 3 4.0 964981247\n", 143 | "2 1 6 4.0 964982224\n", 144 | "3 1 47 5.0 964983815\n", 145 | "4 1 50 5.0 964982931" 146 | ] 147 | }, 148 | "execution_count": 2, 149 | "metadata": {}, 150 | "output_type": "execute_result" 151 | } 152 | ], 153 | "source": [ 154 | "ratings = pd.read_csv(\"https://s3-us-west-2.amazonaws.com/recommender-tutorial/ratings.csv\")\n", 155 | "ratings.head()" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": 3, 161 | "metadata": {}, 162 | "outputs": [ 163 | { 164 | "data": { 165 | "text/html": [ 166 | "
\n", 167 | "\n", 180 | "\n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | " \n", 202 | " \n", 203 | " \n", 204 | " \n", 205 | " \n", 206 | " \n", 207 | " \n", 208 | " \n", 209 | " \n", 210 | " \n", 211 | " \n", 212 | " \n", 213 | " \n", 214 | " \n", 215 | " \n", 216 | " \n", 217 | " \n", 218 | " \n", 219 | " \n", 220 | " \n", 221 | "
movieIdtitlegenres
01Toy Story (1995)Adventure|Animation|Children|Comedy|Fantasy
12Jumanji (1995)Adventure|Children|Fantasy
23Grumpier Old Men (1995)Comedy|Romance
34Waiting to Exhale (1995)Comedy|Drama|Romance
45Father of the Bride Part II (1995)Comedy
\n", 222 | "
" 223 | ], 224 | "text/plain": [ 225 | " movieId title \\\n", 226 | "0 1 Toy Story (1995) \n", 227 | "1 2 Jumanji (1995) \n", 228 | "2 3 Grumpier Old Men (1995) \n", 229 | "3 4 Waiting to Exhale (1995) \n", 230 | "4 5 Father of the Bride Part II (1995) \n", 231 | "\n", 232 | " genres \n", 233 | "0 Adventure|Animation|Children|Comedy|Fantasy \n", 234 | "1 Adventure|Children|Fantasy \n", 235 | "2 Comedy|Romance \n", 236 | "3 Comedy|Drama|Romance \n", 237 | "4 Comedy " 238 | ] 239 | }, 240 | "execution_count": 3, 241 | "metadata": {}, 242 | "output_type": "execute_result" 243 | } 244 | ], 245 | "source": [ 246 | "movies = pd.read_csv(\"https://s3-us-west-2.amazonaws.com/recommender-tutorial/movies.csv\")\n", 247 | "movies.head()" 248 | ] 249 | }, 250 | { 251 | "cell_type": "markdown", 252 | "metadata": {}, 253 | "source": [ 254 | "## Step 3: Exploratory Data Analysis\n", 255 | "\n", 256 | "In Part 1 of this tutorial series, we will focus on the `ratings` dataset. We'll need `movies` for subsequent sections. `Ratings` contains users' ratings for a given movie. Let's see how many ratings, unique movies, and unique users are in our dataset. " 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 4, 262 | "metadata": {}, 263 | "outputs": [ 264 | { 265 | "name": "stdout", 266 | "output_type": "stream", 267 | "text": [ 268 | "Number of ratings: 100836\n", 269 | "Number of unique movieId's: 9724\n", 270 | "Number of unique users: 610\n", 271 | "Average number of ratings per user: 165.3\n", 272 | "Average number of ratings per movie: 10.37\n" 273 | ] 274 | } 275 | ], 276 | "source": [ 277 | "n_ratings = len(ratings)\n", 278 | "n_movies = ratings['movieId'].nunique()\n", 279 | "n_users = ratings['userId'].nunique()\n", 280 | "\n", 281 | "print(f\"Number of ratings: {n_ratings}\")\n", 282 | "print(f\"Number of unique movieId's: {n_movies}\")\n", 283 | "print(f\"Number of unique users: {n_users}\")\n", 284 | "print(f\"Average number of ratings per user: {round(n_ratings/n_users, 2)}\")\n", 285 | "print(f\"Average number of ratings per movie: {round(n_ratings/n_movies, 2)}\")" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "Now, let's take a look at users' rating counts. We can do this using pandas' `groupby()` and `count()` which groups the data by `userId`'s and counts the number of ratings for each userId. " 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": 5, 298 | "metadata": {}, 299 | "outputs": [ 300 | { 301 | "data": { 302 | "text/html": [ 303 | "
\n", 304 | "\n", 317 | "\n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | "
userIdn_ratings
01232
1229
2339
34216
4544
\n", 353 | "
" 354 | ], 355 | "text/plain": [ 356 | " userId n_ratings\n", 357 | "0 1 232\n", 358 | "1 2 29\n", 359 | "2 3 39\n", 360 | "3 4 216\n", 361 | "4 5 44" 362 | ] 363 | }, 364 | "execution_count": 5, 365 | "metadata": {}, 366 | "output_type": "execute_result" 367 | } 368 | ], 369 | "source": [ 370 | "user_freq = ratings[['userId', 'movieId']].groupby('userId').count().reset_index()\n", 371 | "user_freq.columns = ['userId', 'n_ratings']\n", 372 | "user_freq.head()" 373 | ] 374 | }, 375 | { 376 | "cell_type": "code", 377 | "execution_count": 6, 378 | "metadata": {}, 379 | "outputs": [ 380 | { 381 | "name": "stdout", 382 | "output_type": "stream", 383 | "text": [ 384 | "Mean number of ratings for a given user: 165.30.\n" 385 | ] 386 | } 387 | ], 388 | "source": [ 389 | "print(f\"Mean number of ratings for a given user: {user_freq['n_ratings'].mean():.2f}.\")" 390 | ] 391 | }, 392 | { 393 | "cell_type": "markdown", 394 | "metadata": {}, 395 | "source": [ 396 | "On average, a user will have rated ~165 movies. Looks like we have some avid movie watchers in our dataset." 397 | ] 398 | }, 399 | { 400 | "cell_type": "code", 401 | "execution_count": 7, 402 | "metadata": {}, 403 | "outputs": [ 404 | { 405 | "data": { 406 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAFNCAYAAAAHJy4nAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3XtclGX+//HXMMNwxjNo6vpdVyzz3GpqHkgMURDxxLq1WVJ9Kw+ZWfbFSis7fsvy1Faau67bOf15KHHzlKnlVtvBpVq0te+aKDioICdhhhnu3x/EJAIKMiMI7+fj0aPhPn6uG7mv+zPXdV+XyTAMAxEREREREak1n/oOQERERERE5HKlhEpEREREROQiKaESERERERG5SEqoRERERERELpISKhERERERkYukhEpEREREROQiKaGSerdgwQL++Mc/euRYGRkZ9O3bF5fLBcCUKVNYu3atR44NcMcdd7BhwwaPHa+mFi9ezIABAxg8ePAlP3e5c6/tpfbll18SExNTL+cWETlXcnIyixcvrpdzG4bBvHnz6N+/P5MmTaqXGKDx3JfXr1/PjTfeWN9hyGXMUt8BSOMWFRXFyZMnMZvNmM1munTpQkJCApMnT8bHpyyfX7hwYY2P9eSTT3LddddVu80VV1zBN99845HYly9fzk8//cSiRYvcy1atWuWRY9dGRkYGq1evZteuXbRq1eqSn7+cJ69tTVx55ZVs27aNTp06AdCvXz+2bt16yc4vIpeXqKgoioqK2LlzJ4GBgQCsXbuW999/n9dff72eo/Osr776ik8//ZTdu3e7y1ofGsp9ecqUKYwdO5bExMT6DkWaKLVQide9+uqrfPPNN+zatYv//u//5rXXXuPhhx/2+HmcTqfHj9kQZGRk0Lx583pNpjytsf6uRKR+lZaW8te//rW+w6i12rb8Hzt2jPbt29drMnWpNKb6or7K0piuYUOlhEoumZCQEEaMGMGSJUvYsGEDP/zwA1Cx20R2djZ33XUX/fr149prr+Wmm26itLSUuXPnkpGRwd13303fvn157bXXOHr0KFdeeSVr167l+uuv59Zbb3UvO/vmceTIESZNmsQ111zDtGnTOH36NACff/45w4YNqxBjVFQU+/btY8+ePaxYsYK//e1v9O3bl7FjxwIVuxCWlpby8ssvM3z4cAYNGsSDDz5Ifn4+gDuODRs2cP311zNgwABeeeWVaq9Nfn4+Dz74IAMHDmT48OG8/PLLlJaWsm/fPm677TaysrLo27cvycnJlfYtL8drr73GoEGDGDJkCDt27GD37t3ExMRw7bXX8uqrr7q3dzgcPPXUUwwZMoQhQ4bw1FNP4XA4ABg9ejS7du1yb+t0Ohk4cCDff/99pWubn5/PQw89xJAhQxg6dCiLFy+u9qFg+fLlzJo1iwceeIBrrrmGDRs2kJqayuTJk+nXrx9Dhgxh4cKF7jj+8Ic/AJCQkEDfvn3ZsmVLpd9XVFQUf/rTn4iPj+e3v/0ts2fPxm63u9e/9tpr7jKuXbuWK6+8kp9++gmA3bt3ExsbS9++fRk6dCh/+tOfqv3diMjl4/bbb+fPf/4zeXl5ldZVVT+cfU9fv349v//973n66afp168fI0aM4Ouvv2b9+vVERkYyaNCgSl2+c3JySEpKom/fvtx8880cO3bMve7HH38kKSmJa6+9lpiYGLZs2eJel5yczKOPPsp///d/06dPHz7//PNK8dpsNu6++26uvfZaoqOjee+994CyVrdHHnmE/fv307dvX5YtW1Zp39qWpbo6yOFw0K9fP3d9DWX1dK9evTh16lSl+7LNZuOee+5h4MCBREVFVUhuU1NTmTBhAtdccw3XXXcdzzzzTBW/wV/qtJUrVzJ48GDmzZtHbm4ud911FwMHDqR///7cddddHD9+HCjrEv/ll1+ycOFC+vbt6+71cr7rn5OTw913380111zDpEmTOHLkSJWxwC//bt599113nXJ2nVFaWsrKlSu54YYbGDBgAPfee6/7OaOq55SqflfndjesaX21a9cuEhIS6NevH7///e85cOCAe11UVBQrV64kPj6ePn36KKnyNkPEi4YPH258+umnlZZHRkYab775pmEYhvE///M/xosvvmgYhmEsWrTImD9/vuFwOAyHw2H84x//MEpLS6s8Vnp6utG1a1dj7ty5RmFhoVFUVOReVlJSYhiGYdx8883GkCFDjIMHDxqFhYXGzJkzjfvvv98wDMP47LPPjKFDh1Yb77Jly9zblrv55puN9957zzAMw1i7dq1xww03GEeOHDEKCgqMGTNmGA888ECF2B5++GGjqKjISEtLM7p3724cOnSoyus0d+5c4+677zby8/ON9PR0Y+TIke7zVBXn2T777DOjW7duxvLlyw2Hw2G8++67xoABA4w5c+YY+fn5xg8//GD07NnTOHLkiGEYhrFkyRIjMTHROHnypHHq1Clj8uTJxuLFiw3DMIzly5cbc+bMcR97165dxqhRoyqUqfzaTp8+3Zg/f75RWFhonDx50pg4caLx9ttvVxnjsmXLjKuvvtrYvn274XK5jKKiIuPbb781vvnmG6OkpMRIT083Ro0aZaxevdq9T9euXY3Dhw9XKOfZ12H48OHGxIkTjePHjxs5OTnGqFGjjLfeesswDMPYvXu3cd111xk//PCDcebMGeP++++vcLzBgwcb//jHPwzDMIzTp08b3333XbXXV0QuD+X37xkzZrjrlPfee8+4+eabDcOofA8zjIr39P/3//6f0a1bN2PdunWG0+k0XnzxRSMyMtJ47LHHDLvdbuzdu9fo06ePUVBQYBhGWd3Vp08f44svvjDsdrvxxBNPGL///e8NwzCMwsJCY9iwYca6deuMkpIS4/vvvzeuvfZa49///rd732uuucb48ssvDZfLZRQXF1cqz0033WQ8+uijRnFxsfGvf/3LGDBggLFv3z53rOXnqkpty3K+Oig5Odl9PQ3DMN544w3jtttuMwyj4n3Z5XIZ48ePN5YvX27Y7XbjyJEjRlRUlLFnzx7DMAzjd7/7nbFhwwbDMAyjoKDA+Oabb6qMvbxOe+655wy73W4UFRUZ2dnZxocffmicOXPGyM/PN+655x5j2rRpVf4ea3L9Z8+ebcyaNcsoLCw0Dh48aAwZMqTa61n+7+a+++4zCgsLjQMHDhgDBgxwPyv85S9/MRITE43MzEzDbrcb8+fPN+67774K+579nFLV7+rcc9ekvvr++++NgQMHGvv37zecTqexfv16Y/jw4YbdbjcMo+zvYezYsUZGRkaV5xXPUguV1IuwsDByc3MrLbdYLJw4cYKMjAx8fX3p168fJpPpvMe65557CAwMxN/fv8r1CQkJdO3alcDAQO69914+/PBDjwys8MEHHzB16lQ6duxIUFAQc+bMYcuWLRW+BZo5cyb+/v5cddVVXHXVVRW+PSrncrnYsmUL999/P8HBwXTo0IGkpCTef//9GsdisViYNm0avr6+xMbGkpOTwy233EJwcDARERF06dKFgwcPuuOeMWMGrVq1omXLlsyYMcN9rvj4eD766COKiorc28bFxVU638mTJ9m9ezcPPfQQgYGBtGrViqlTp5KSklJtjH369OGGG27Ax8cHf39/evToQZ8+fbBYLHTo0IHJkyfzj3/8o8ZlhrJvl8PDw2nevDnDhw8nLS0NgL/97W9MmDCBiIgIAgICuOeeeypdr0OHDlFQUECzZs3o3r17rc4rIg3XrFmzeOONN8jOzq71vh06dGDixImYzWZiY2PJzMxkxowZWK1WhgwZgtVqrdCacf3119O/f3+sViv33Xcf+/fvJzMzk48//pj27dszceJELBYLV199NTExMXz44YfufUeMGMFvf/tbfHx88PPzqxBHZmYmX3/9NQ888AB+fn5069aNxMRENm3a5PGyXKgOio+Pr3Bv/+CDD4iPj690vm+//Zbs7GxmzpyJ1WqlY8eO/O53v3O3DFksFo4cOUJ2djZBQUH06dOn2th9fHyYNWsWVqsVf39/WrRoQUxMDAEBAQQHBzNt2rTz1hfnu/4ul4tt27Yxa9YsAgMD6dq1K+PHj7/g9ZwxYwaBgYFceeWVTJgwgc2bNwPwzjvvcN9999G2bVusViszZ85k69atFZ4FLvSccj7V1VfvvvsukydPpnfv3pjNZsaPH4+vry/79+937ztlyhTatWt3UeeV2tGgFFIvbDYbzZo1q7T89ttv56WXXuK2224DYPLkydx5553nPVbbtm3Pu75du3buz1dccQUlJSXk5ORcRNQVZWVl0b59e/fP7du3x+l0curUKfey1q1buz8HBARw5syZSsfJycmhpKSEK664okKcNputxrE0b94cs9kM4L5xnv3OlZ+fH4WFhe64zz1XVlYWAJ06deI3v/kNu3btYvjw4Xz00Uds3Lix0vkyMjJwOp0MGTLEvay0tLTCtT7Xub+n//znPzz77LN89913FBUV4XK5ap3YtGnTxv05ICDAXY6srCx69OjhXnduXMuWLeOVV17hhRde4Morr+T++++nb9++tTq3iDRMXbt25frrr2flypX85je/qdW+Z983y++lZ9/Hz76XQsX7WlBQEM2aNSMrK4tjx46RmppKv3793OtdLpe7+zhUvi+dLSsri2bNmhEcHOxedsUVV/Ddd995vCwXqoMGDBhAcXEx//znP2nVqhUHDhzghhtuqHS+Y8eOkZWVVanM5T8/9dRTLFu2jNGjR9OhQwdmzpzJ8OHDq4y9RYsWFZLMoqIinnnmGfbu3ev+MrawsBCXy+Wu+86Npbrrn52djdPprPRscCFnb9++fXt3N8iMjAxmzJjhHmgLyhLCs58FLvSccj7V1VcZGRls3LiRN954w71tSUmJux48N2bxLiVUcsmlpqZis9n47W9/W2ldcHAwycnJJCcn88MPP3DrrbfSs2dPBg0aVO3xLtSClZmZWeGzr68vLVq0ICAggOLiYvc6l8tV4RvNCx03LCysQn/5jIwMLBYLrVq1cvftrokWLVrg6+tLRkYGXbp0cccZHh5e42PURlhYGBkZGURERLjPFRYW5l4/ZswYNm/eTGlpKV26dHGPsne28m/iPvvsMyyWmt1Gzr2ejz32GFdffTUvvPACwcHB/OUvf/HYaFFhYWEVEtKz/w0A9OrVi1deeYWSkhLefPNNZs+eze7duz1ybhGpf7NmzWL8+PHuL+cA9wAOxcXF7kTlxIkTdTrP2ff6wsJCcnNzCQsLo127dvTv35/Vq1df1HHLe3EUFBS4Y/VWvXChOshsNjNq1Cg2b95M69atuf766yskeuXatWtHhw4d2LZtW5Xn+a//+i9efPFFSktL3S1En3/+eZUDa5xbX/z5z3/mP//5D++99x5t2rQhLS2NcePGYRhGlec63/V3uVxYLBYyMzPdCfe5dURVzt4+IyPDXW+2bduWp59+uspnmqNHj1ZZnrOd+yxy7r/J6uqrdu3acffddzNt2rRqj32h5xjxHHX5k0umoKCAXbt2MWfOHMaOHcuVV15ZaZtdu3bx008/YRgGISEhmM1m9w2hdevWpKen1/q877//PocOHaKoqIilS5cSExOD2Wzm17/+NXa7nY8//piSkhJeeeUV96AIUPbt3rFjxygtLa3yuGPGjGHNmjWkp6dTWFjI4sWLGT16dI0TjHLlldXixYspKCjg2LFjrF69usI3mZ4UFxfHK6+8QnZ2NtnZ2fzxj3+s0H0jNjaWTz/9lLfffpsxY8ZUeYywsDAGDx7Ms88+S0FBAaWlpRw5coQvvviixnEUFhYSFBREUFAQP/74I2+//XaF9Rf7+wYYNWoU69ev58cff6SoqIiXX37Zvc7hcPD++++Tn5+Pr68vQUFBFb5ZFJHLX6dOnYiNja0wXHrLli0JDw9n06ZNuFwu1q1bd9H3mHK7d+/myy+/xOFwsHTpUnr37k27du24/vrrOXz4MBs3bqSkpISSkhJSU1P58ccfa3Tcdu3a0bdvX1588UXsdjsHDhxg3bp1XqkXalIHxcfH87e//Y0PPvig2nqhV69eBAUFsXLlSoqLi3G5XPzwww+kpqYCsGnTJrKzs/Hx8SE0NBSgxvfewsJC/Pz8CA0N5fTp07z00ksV1p9bX5zv+pvNZqKjo3nppZcoKiri0KFDNZpf8uWXX6aoqIh///vfrF+/ntjYWABuvPFGlixZ4v6CNTs7mx07dtSoXABXXXUV//73v0lLS8Nut7N8+XL3uvPVV4mJibzzzjv885//xDAMzpw5w8cff0xBQUGNzy2eo6cI8brykfkiIyN59dVXSUpKqnZ0n59++sk9YtLkyZO58cYbGThwIAB33nknr7zyCv369avVqGwJCQkkJyczePBgHA6He8j2kJAQHn30UR555BGGDRtGQEBAhWb5UaNGAWXdHarqXz1x4kTGjh3LzTffzIgRI7BarcyfP7/GcZ1t/vz5BAQEcMMNN3DTTTcxZswYJk6ceFHHupDp06fTo0cPxo4dy9ixY+nevTvTp093rw8LC6NPnz5888037gqjKs899xwlJSXExsbSv39/Zs2aVatve//nf/6HzZs3c8011zB//vxK55o5cybJycn069evwuhMNREZGcmUKVO45ZZbiI6Opnfv3gBYrVagrGKPiorimmuu4Z133uH555+v1fFFpOGbMWNGpW7WTzzxBH/6058YMGAAhw4dqnNX3zFjxvDHP/6RAQMG8P3337vvJcHBwfzpT39iy5YtDB06lCFDhrBo0aIKX9pdyIsvvsixY8cYOnQoM2fO5J577jnvPIx1caE6qHfv3u5u1eeOjlvObDbz6quvcuDAAUaMGMHAgQN55JFH3A/4e/fuJS4ujr59+/LUU0+xePHiGr/bc+utt2K32xk4cCCTJ09m6NChFdbfcsstbN26lf79+/Pkk09e8PovWLCAM2fOMHjwYJKTk5kwYcIFYygfbXHq1Kncdttt7i7vt9xyC1FRUdx222307duX3/3ud+4ksiZ+/etfM2PGDKZOncrIkSMrtXRVV1/17NmTJ554goULF9K/f39GjhzJ+vXra3xe8SyTUV17qYhII/Hjjz8yZswYvv3221q3IIqISNN19OhRRowYwffff6/6Q6qlFioRaZS2b9+Ow+EgNzeX559/nuHDh6syFBEREY9TQiUijdI777zDoEGDiI6Oxmw289hjj9V3SCIiItIIqcufiIiIiIjIRVILlYiIiIiIyEVSQiUiIiIiInKRmtwb2vv3768w+7aIiFx6drudPn361HcYDVJDrKfsdnudYjp48CBAlfMP1oe6lqchaUxlAZWnIWtMZYELl6c29VSTS6j8/Pzo1q1bfYchItKkpaWl1XcIDVZDrKfS0tLqFNNf//pXAMaNG+epkOqkruVpSBpTWUDlacgaU1ngwuWpTT3V5BIqERERubSqm8xdRKQx0DtUIiIiIiIiF0kJlYiIiHjVxIkTmThxYn2HISLiFeryJyIiIl516tSp+g5BRMRr1EIlIiIiIiJykZRQiYiIiIiIXCQlVCIiIiIiIhdJ71CJiIiIV40YMaK+QxAR8RolVCIiIuJV8+fPr+8QRES8Rl3+RERERERELpISKhEREamVv3z6H+Zv/A6701Wj7UePHs3o0aO9HJWISP1QQiUijVaJq6RRn08uzp49e4iJiSE6OpqVK1dWWu9wOJg9ezbR0dEkJiZy9OhR97oVK1YQHR1NTEwMe/fuBcButzNp0iTGjh1LXFwcy5Ytc2+fnJxMVFQUCQkJJCQkkJaW5v0CXgIb9mfw+mc/cfOqzzl9xnHB7YuKiigqKroEkYmIXHp6h0pEGi1fsy9Jf5tzyc63evSLl+xccnFcLhcLFy5k9erVhIeHM2nSJKKioujSpYt7m7Vr1xIaGsr27dtJSUlh0aJFLFmyhEOHDpGSkkJKSgo2m42kpCS2bt2K1WplzZo1BAUFUVJSwk033cSwYcPo06cPAA8++CCjRo2qryJ7xfHcIv6rVSD7008z4eV9/CXpWn7VKrC+wxIRqRdqoRIRkSYjNTWVTp060bFjR6xWK3FxcezcubPCNh999BHjx48HICYmhr///e8YhsHOnTuJi4vDarXSsWNHOnXqRGpqKiaTiaCgIACcTidOpxOTyXTJy3aplLhKycq3MyyiNQ+N7saJfDuJK/bhcJbWd2giIvVCCZWIiDQZNpuNtm3bun8ODw/HZrNV2qZdu3YAWCwWQkJCyMnJOe++LpeLhIQErrvuOq677jp69+7t3m7x4sXEx8fz9NNP43BcuHtcQ5eVb8cwoEWQlavahTL52o7Y8uzY8orrOzQRkXqhLn8iIiJ1ZDab2bRpE3l5ecyYMYMffviBrl27MmfOHNq0aUNJSQnz589n5cqVzJw587zHstvtDe5dq+LiYndM/8oqS5xcRfkcOVKIccYOwD++/zcFra1V7t+/f3+ABlOus8tzuWtMZQGVpyFrTGUBz5ZHCZWIiDQZ4eHhHD9+3P2zzWYjPDy80jaZmZm0bdsWp9NJfn4+LVq0qNG+oaGhDBgwgL1799K1a1fCwsIAsFqtTJgwgT//+c8XjNHPz49u3brVpZgel5aW5o7px5IMIIPOHdrSrlkAPiFn4KtsfIJb061b+yr3f/755y9htBd2dnkud42pLKDyNGSNqSxw4fLUJtlSlz8REWkyevbsyeHDh0lPT8fhcJCSkkJUVFSFbaKiotiwYQMAW7duZeDAgZhMJqKiokhJScHhcJCens7hw4fp1asX2dnZ5OXlAWXfeO7bt4/OnTsDkJWVBYBhGOzYsYOIiIhLWFrvOJ5b1kLVMsha4f8ZpzWKn4g0TWqhEhGRJsNisbBgwQLuuOMOXC4XEydOJCIigqVLl9KjRw9GjBjBpEmTmDt3LtHR0TRr1ozFixcDEBERwejRo4mNjcVsNrNgwQLMZjNZWVkkJyfjcrkwDINRo0YxfPhwAB544AFycnIwDIOrrrqKxx9/vD6L7xEZp4sJ8DUT4Gum1IAAXzP+vj7nTaiuv/56AD7++ONLE6SIyCWkhEpERJqUyMhIIiMjKyy799573Z/9/PwqzCV1tmnTpjFt2rQKy6666io2btxY5fZ//etf6xhtw3M8r4jwUD/3zyaTiZZBVjJzNSiFiDRN6vInIiIiNZaZW0ybEH9KjV+WtQzy0yh/ItJkKaESERGRGss8XUybkIqj+bUKspKVb6+niERE6pcSKhEREakRp6uUrPxiWgX5VVjeItDKyQI7Tpcm9xWRpkfvUImIiEiNnCiwU2pAi0DfCstbBlkpNeBkgYO2zfwr7fe73/3uUoUoInLJKaESERGRGikfeOLchKrVz0OnZ+YWVZlQTZ8+3fvBiYjUE691+cvMzGTKlCnExsYSFxfHmjVrAFi+fDlDhw4lISGBhIQEdu/e7d5nxYoVREdHExMTw969e93L9+zZQ0xMDNHR0axcudK9PD09ncTERKKjo5k9ezYOh8NbxREREWnyMk+XJVShgRXfoWoZ/EtCVZUzZ85w5swZ7wYnIlJPvNZCZTabSU5Opnv37hQUFDBx4kQGDx4MwNSpU7n99tsrbH/o0CFSUlJISUnBZrORlJTE1q1bAVi4cCGrV68mPDycSZMmERUVRZcuXVi0aBFTp04lLi6OBQsWsG7dOm666SZvFUlERKRJK0+YWp7b5e/nBOtYTtUJVWxsLKB5qESkcfJaC1VYWBjdu3cHIDg4mM6dO2Oz2ardfufOncTFxWG1WunYsSOdOnUiNTWV1NRUOnXqRMeOHbFarcTFxbFz504Mw+Czzz4jJiYGgPHjx7Nz505vFUdERKTJO55bjL/Fh0Brxe9jQ/wtWHxMZGguKhFpgi7JO1RHjx4lLS2N3r178/XXX/Pmm2+yceNGevToQXJyMs2aNcNms9G7d2/3PuHh4e4ErG3bthWWp6amkpOTQ2hoKBaLxb3N+RK2cna7nbS0NA+XUEQaom7dul3yc+r+Io1ZZm4x4aGV35EymUy00uS+ItJEeT2hKiwsZNasWTz00EMEBwdz4403Mn36dEwmE0uXLuXZZ5/lmWee8XYYbn5+fvXykCUiTYPuLzWjxPPylJlbRJtQvwqT+pZrEWTFpoRKRJogr85DVVJSwqxZs4iPj2fkyJEAtG7dGrPZjI+PD4mJiXz77bdAWcvT8ePH3fvabDbCw8OrXd6iRQvy8vJwOp0AHD9+nPDwcG8WR0REpEk7nltMm2C/Kte1DLKSla+ESkSaHq8lVIZh8PDDD9O5c2eSkpLcy7Oystyfd+zYQUREBABRUVGkpKTgcDhIT0/n8OHD9OrVi549e3L48GHS09NxOBykpKQQFRWFyWRiwIAB7oErNmzYQFRUlLeKIyIi0qS5Sg1s+Xb3EOnnKkuo7BhG5earqVOnMnXqVC9HKCJSP7zW5e+rr75i06ZNdO3alYSEBADmzJnD5s2bOXDgAADt27dn4cKFAERERDB69GhiY2Mxm80sWLAAs9kMwIIFC7jjjjtwuVxMnDjRnYTNnTuX++67jyVLltCtWzcSExO9VRwREZEm7US+HVepQctqEqpWQVZKXAbZhQ5andOKpWRKRBozryVU/fr14+DBg5WWR0ZGVrvPtGnTmDZtWpX7VLVfx44dWbduXd0CFRERkQsqHzK9+TlDppdrGeT383bFlRKqkydPAmXd/kVEGhuvvkMlIiIijcPxnwecCA2ovssfVD2576RJk5g0aZL3ghMRqUdKqEREROSCyueYahVUXQtVWUKVcVoDU4hI06KESkRERC7oeG4RflVM6luueYAvPibIOF25hUpEpDFTQiUiIiIXlJlbTFho1UOmA/j4mGgRqMl9RaTpUUIlIiIiF3Q8t5jwEP8qJ/Ut1zLI6n7XSkSkqfDaKH8iIiLSeGTmFtOzfbPzbtMiyEpWXuWEqqoRfEVEGgslVCIiInJehmGQlV9My6DzD3veMsjKt0dPYxgGJpPJvXzy5MneDlFEpN6oy5+IiIicl91pUOIyCPI7//ewrYKsFJWUkm93Vlienp5Oenq6N0MUEak3aqESERGR8ypwlAIQ4GsvhHfwAAAgAElEQVQ+73blQ6cfzy0m1P+X4dWnTJkCwMcff+ydAEVE6pFaqEREROS8CkvKEir/CyVUgeWT+2pgChFpOpRQiYiIyHkVlrdQWc+fUDULKGuVOpGvhEpEmg4lVCIiInJe5V3+/C3nT6iC/cveJMgudHg9JhGRhkIJlYiIiJzXmZ8TqsALtFAFWS2YUEIlIk2LBqUQERGR8yqoYULl42MiyM9SKaG6//77vRabiEh9UwuViIg0KXv27CEmJobo6GhWrlxZab3D4WD27NlER0eTmJjI0aNH3etWrFhBdHQ0MTEx7N27FwC73c6kSZMYO3YscXFxLFu2zL19eno6iYmJREdHM3v2bByOy7Pl5kxJzd6hAgjxt5BzpqTCsvj4eOLj470Sm4hIfVNCJSIiTYbL5WLhwoWsWrWKlJQUNm/ezKFDhypss3btWkJDQ9m+fTtTp05l0aJFABw6dIiUlBRSUlJYtWoVjz/+OC6XC6vVypo1a3j//ffZuHEje/fuZf/+/QAsWrSIqVOnsn37dkJDQ1m3bt0lL7MnFDhc+JpN+Jov/NgQ7Gch95yE6uDBgxw8eNBb4YmI1CslVCIi0mSkpqbSqVMnOnbsiNVqJS4ujp07d1bY5qOPPmL8+PEAxMTE8Pe//x3DMNi5cydxcXFYrVY6duxIp06dSE1NxWQyERQUBIDT6cTpdGIymTAMg88++4yYmBgAxo8fX+lcl4tCRynBF5jUt1yIv4XTRRVb4u666y7uuusub4QmIlLv9A6ViIg0GTabjbZt27p/Dg8PJzU1tdI27dq1A8BisRASEkJOTg42m43evXtX2NdmswFlLV8TJkzgyJEj3HTTTfTu3Zvs7GxCQ0OxWMqq2rZt27q3Px+73U5aWlqdy+pJecVOAiyQnn4EV6lx3m1NTjun8ksqlOHMmTMADaZcxcXFDSaWumpMZQGVpyFrTGUBz5ZHCZWIiEgdmc1mNm3aRF5eHjNmzOCHH36gdevWF3UsPz8/unXr5uEI66Z4RybNgwLo0OFXnD+dgvAM+P6ErUIZAgMDARpMudLS0hpMLHXVmMoCKk9D1pjKAhcuT22SLXX5ExGRJiM8PJzjx4+7f7bZbISHh1faJjMzEyjrwpefn0+LFi1qtG9oaCgDBgxg7969tGjRgry8PJxOJwDHjx+vtP3lorzL34WSKYAQPwvFJaUUl7i8HpeISEOghEpERJqMnj17cvjwYdLT03E4HKSkpBAVFVVhm6ioKDZs2ADA1q1bGThwICaTiaioKFJSUnA4HKSnp3P48GF69epFdnY2eXl5QFkXkn379tG5c2dMJhMDBgxg69atAGzYsKHSuS4XBY5SgvwuPMIf/DK5b86Zy3NEQxGR2lKXPxERaTIsFgsLFizgjjvuwOVyMXHiRCIiIli6dCk9evRgxIgRTJo0iblz5xIdHU2zZs1YvHgxABEREYwePZrY2FjMZjMLFizAbDaTlZVFcnIyLpcLwzAYNWoUw4cPB2Du3Lncd999LFmyhG7dupGYmFifxb9oZ0pKCarFoBQAOYUltGsWAMAjjzzitdhEROqbEioREWlSIiMjiYyMrLDs3nvvdX/28/OrMJfU2aZNm8a0adMqLLvqqqvYuHFjldt37Njxsh0q/WwFjlKCajAHFZR1+QM4fVYL1Q033OCVuEREGgJ1+RMREZFqlbhKsTsNAq01+w422N8XgFOFvyRU+/fvd8/NJSLS2KiFSkRERKqVX1w2qEZADVuoyueryi60u5fNnj0bgI8//tizwYmINABqoRIREZFq5RWVABDgW8Mufz+/Q3WqQINSiEjToIRKREREqvVLC1XNHhl8zT74WXzILlRCJSJNgxIqERERqVZecVkLlX8NW6gAQv0tnP65ZUtEpLFTQiUiIiLV+qXLX81fuw7299U8VCLSZGhQChEREalWeZe/wBoOSgFlA1Pknvmlherpp5/2eFwiIg2FEioRERGpVnmXv5qO8gcQ7G8hPfuM++frrrvO43GJiDQU6vInIiIi1cordmKi5qP8QdnkvrlnvUO1b98+9u3b54XoRETqn1qoREREpFp5RSUEWn0wYQKMGu0T7G8hv9iJ01WKxezDQw89BGgeKhFpnNRCJSIiItXKKy4h0NcHo4bJFECIny9AhVYqEZHGSgmViIiIVCu/2EmQ1YxR83zKPblvzhklVCLS+CmhEhERkWrlFZUQZPWpRftU2Sh/AKc1dLqINAFKqERERKRa+cXOWg1IAWXvUAFkFyqhEpHGT4NSiIiISLXyikto3bx2jwshP7dQnSooS6iWLFni8bhERBoKJVQiIiJSrbyiEgLC/Gq1j7uF6ucuf3369PF4XCIiDYXXuvxlZmYyZcoUYmNjiYuLY82aNQCcPn2apKQkRo4cSVJSErm5uQAYhsGTTz5JdHQ08fHxfP/99+5jbdiwgZEjRzJy5Eg2bNjgXv7dd98RHx9PdHQ0Tz75JEZt3pgVERGR8yotNSiwO/E3m2q1X4CvGbOPiexCOwA7duxgx44d3ghRRKTeeS2hMpvNJCcns2XLFt59913eeustDh06xMqVKxk0aBDbtm1j0KBBrFy5EoA9e/Zw+PBhtm3bxhNPPMFjjz0GlCVgL730Eu+99x5r167lpZdecidhjz32GE888QTbtm3j8OHD7Nmzx1vFERERaXIKHU5KDQjwrd3jgslkIsTPQk5h2Sh/Tz75JE8++aQ3QhQRqXdeS6jCwsLo3r07AMHBwXTu3BmbzcbOnTsZN24cAOPGjXN/Y1W+3GQy0adPH/Ly8sjKyuKTTz5h8ODBNG/enGbNmjF48GD27t1LVlYWBQUF9OnTB5PJxLhx49i5c6e3iiMiItLk5Bc7AfCz1K6FCsq6/eVolD8RaQIuySh/R48eJS0tjd69e3Pq1CnCwsIAaNOmDadOnQLAZrPRtm1b9z5t27bFZrNVWh4eHl7l8vLtRZoau/PSzvNyqc8nIvUnr7js793fUvvHhWA/C6c1sa+INAFeH5SisLCQWbNm8dBDDxEcHFxhnclkwmSq/bdedWG320lLS7uk5xTxpm7dutF70aOX7Hz/fODxy+ZvqFu3bpf8nJfLtRGpCXcLVe1GTQfKJvctH+VPRKQx82pCVVJSwqxZs4iPj2fkyJEAtGrViqysLMLCwsjKyqJly5ZAWcvT8ePH3fseP36c8PBwwsPD+eKLL9zLbTYb1157bbXbX4ifn1+9PGSJNCb6G6qerk3NKPG8POT93MJkNV9MC5Uv/3ei0NMhiYg0OF7r8mcYBg8//DCdO3cmKSnJvTwqKoqNGzcCsHHjRkaMGFFhuWEY7N+/n5CQEMLCwhgyZAiffPIJubm55Obm8sknnzBkyBDCwsIIDg5m//79GIZR4VgiIiJSd790+at9b5IQ/7Iuf4ZhsGLFClasWOHp8EREGgSvtVB99dVXbNq0ia5du5KQkADAnDlzuPPOO5k9ezbr1q3jiiuucE/2FxkZye7du4mOjiYgIICnn34agObNmzN9+nQmTZoEwIwZM2jevDkAjz76KPPmzaO4uJhhw4YxbNgwbxVHRESkyanLoBQh/hZcPw+7fuWVV3o6NBGRBsNrCVW/fv04ePBglevK56Q6m8lk4tFHq34PZNKkSe6E6mw9e/Zk8+bNdQtUREREqlTe5e9iWqiC/coeMU6fKeHj7R8CEB8f77ngREQaCK8PSiEiIiKXp/xiJ/4WH8w+FzdsOkDOGQcvvPACoIRKRBqnSzJsuoiIiFx+8opLCPa3YBhGrfcN8fMFIOeMhk4XkcZNCZWIiDQpe/bsISYmhujoaFauXFlpvcPhYPbs2URHR5OYmMjRo0fd61asWEF0dDQxMTHs3bsXgMzMTKZMmUJsbCxxcXEVurUvX76coUOHkpCQQEJCArt37/Z+AT0or8hJsJ+F0otIqMpbqE4V2D0dlohIg6IufyIi0mS4XC4WLlzI6tWrCQ8PZ9KkSURFRdGlSxf3NmvXriU0NJTt27eTkpLCokWLWLJkCYcOHSIlJYWUlBRsNhtJSUls3boVs9lMcnIy3bt3p6CggIkTJzJ48GD3MadOncrtt99eX0Wukzq1UCmhEpEmQi1UIiLSZKSmptKpUyc6duyI1WolLi6OnTt3Vtjmo48+Yvz48QDExMTw97//HcMw2LlzJ3FxcVitVjp27EinTp1ITU0lLCyM7t27AxAcHEznzp2x2WyXvGzekFdc1kJ1EfkUwVYLJiBbXf5EpJFTQiUiIk2GzWajbdu27p/Dw8MrJT82m4127doBYLFYCAkJIScnp0b7Hj16lLS0NHr37u1e9uabbxIfH8+8efPIzc31RrG8Jr+4xD1aX235+JgI9DOTU+jg9ddf5/XXX/dwdCIiDYO6/ImIiHhAYWEhs2bN4qGHHiI4OBiAG2+8kenTp2MymVi6dCnPPvsszzzzzHmPY7fbSUtLuxQhX1BOfjFGMwsOh4UjR36q9f4BZjh28jQFBWUDVDSUchUXFzeYWOqqMZUFVJ6GrDGVBTxbHiVUIiLSZISHh3P8+HH3zzabjfDw8ErbZGZm0rZtW5xOJ/n5+bRo0eK8+5aUlDBr1izi4+MZOXKke5vWrVu7PycmJnL33XdfMEY/Pz+6det20WX0pELnYcJahGK1OvnVrzrVev/mwfk48CU1NRWAyZMnezrEi5KWltZgrnFdNaaygMrTkDWmssCFy1ObZEtd/kREpMno2bMnhw8fJj09HYfDQUpKClFRURW2iYqKYsOGDQBs3bqVgQMHYjKZiIqKIiUlBYfDQXp6OocPH6ZXr14YhsHDDz9M586dSUpKqnCsrKws9+cdO3YQERHh/UJ6SHGJC4ezlACr+aKPEexnIbeohFdeeYVXXnnFg9GJiDQcaqESEZEmw2KxsGDBAu644w5cLhcTJ04kIiKCpUuX0qNHD0aMGMGkSZOYO3cu0dHRNGvWjMWLFwMQERHB6NGjiY2NxWw2s2DBAsxmM19++SWbNm2ia9euJCQkADBnzhwiIyN5/vnnOXDgAADt27dn4cKF9Vb22sovdgIQWIeEKsTPwvHcYvw9FZSISAOkhEpERJqUyMhIIiMjKyy799573Z/9/PxYtmxZlftOmzaNadOmVVjWr18/Dh48WOX2zz//fB2jrT/5xWWj89WlhSrE30JukUMJlYg0auryJyIiIpXk/dxCFeBbhy5//r4UlZRe1MTAIiKXCyVUIiIiUknBzwmVn6Vu71ABOEuVUIlI46UufyIiIlJJgb2sy5+/rxln0cUdI8S/7DHjhVfXEBEe4qnQREQaFLVQiYiISCX57i5/F/+o4J4U2D+0whDyIiKNiRIqERERqaTAXpZQ+ddxUAqA9976K3/5y188EZaISIOjhEpEREQqKX+HKsAD71B9uOFdJVQi0mgpoRIREZFKCuxO/Cw+mM0X/6gQ4u8LQIlLg1KISOOlhEpEREQqybc7CfKzYNRhyHOrxQc/iw9OV6kHIxMRaViUUImIiEglhXYnQX4X392vXLCfRcOmi0ijpoRKREREKikodhJktVDXOXlD/C2UqIVKRBoxJVQiIiJSSf7PLVR1bVsK9rPw2zufY8uWLR6JS0SkoVFCJSIiIpUUFDsJtFrqfJxgfwsFLjOBgYEeiEpEpOFRQiUiIiKVFNidBNZhDqpywX6+/Lh7PS+//LIHohIRaXiUUImIiEglBXYnAb51T6hC/S1kf/sx7733ngeiEhFpeJRQiYiISCUFxU4CPNFC5W/BAI30JyKNlhIqERERqcDudOFwlXqkhSrYr+w9LM1FJSKNlRIqERERqaDQ7gLwSEIV4l+WUJW41EIlIo2TEioRERGpoKDYCYC/b90fE4L9fAFwlqqFSkQap7qPhyoiIiKNSr69BAB/D7VQtb3pWR6a2LPOxxIRaYjUQiUiIiIVlLdQ+Vk89w5VdoGjzscSEWmIlFCJiIhIBQX2nxMqD3T5C7Sayf9iPe+/saLOxxIRaYjU5U9EREQqKE+oAnzr/phgMpmw/9+XfGfzrfOxREQaIrVQiYiISAX5xeUJlWceE8w+Jpwa5U9EGiklVCIiIlJBob18lL+6v0MFPydUmthXRBopJVQiInJZ+uijjyjVUNxeUWB34mMCq8WTLVT6XYlI46SESkRELktbtmxh5MiRPPfcc/z444/1HU6jkl/sJMjPggmTR47n5+dPqVnvUIlI46RBKURE5LK0aNEiCgoK2Lx5M/PmzcNkMjFhwgTi4uIIDg6u7/AuawV2J8F+Fgw8001v3LyX2Pb9cQzDwGTyTJImItJQqIVKREQuW8HBwcTExBAbG8uJEyfYvn07EyZM4PXXX692nz179hATE0N0dDQrV66stN7hcDB79myio6NJTEzk6NGj7nUrVqwgOjqamJgY9u7dC0BmZiZTpkwhNjaWuLg41qxZ497+9OnTJCUlMXLkSJKSksjNzfVg6b2noNhJoNWM4aHXnkL8fXG4DIpKXJ45oIhIA+K1hGrevHkMGjSIMWPGuJctX76coUOHkpCQQEJCArt373avq6qSguorvvT0dBITE4mOjmb27Nk4HJowUESkKdmxYwczZszglltuwel0snbtWlatWsWmTZtYvXp1lfu4XC4WLlzIqlWrSElJYfPmzRw6dKjCNmvXriU0NJTt27czdepUFi1aBMChQ4dISUkhJSWFVatW8fjjj+NyuTCbzSQnJ7Nlyxbeffdd3nrrLfcxV65cyaBBg9i2bRuDBg2qMoFriArsToKsFg+1T8E/1r/G6U/fJudMiYeOKCLScNQoobr11ltrtOxsEyZMYNWqVZWWT506lU2bNrFp0yYiIyOB6iup81V8ixYtYurUqWzfvp3Q0FDWrVtXk6KIiEgjUZ7wfPDBB9xxxx20atUKgICAAJ566qkq90lNTaVTp0507NgRq9VKXFwcO3furLDNRx99xPjx4wGIiYnh73//O4ZhsHPnTuLi4rBarXTs2JFOnTqRmppKWFgY3bt3B8pazDp37ozNZgNg586djBs3DoBx48axY8cOr1wLT8u3Own088wIfwA/ffs5xT/9k5xCffkpIo3PeRMqu93O6dOnycnJITc3l9OnT3P69GmOHj3qriyq079/f5o1a1ajIKqrpKqr+AzD4LPPPiMmJgaA8ePHV6oQRUSkcWvdujX9+/evsOz5558HYNCgQVXuY7PZaNu2rfvn8PDwSvWZzWajXbt2AFgsFkJCQsjJyanRvkePHiUtLY3evXsDcOrUKcLCwgBo06YNp06dupiiXnKFdieBVs+9Zm32KXtv6rRaqESkETrv3fKdd95hzZo1ZGVlMWHCBIyfO1MHBwdz8803X9QJ33zzTTZu3EiPHj1ITk6mWbNm2Gw2d+UDFSupcyuv1NRUcnJyCA0NxWKxuLe5UIInIiKNy759+yot27NnD3Pnzq2HaKCwsJBZs2bx0EMPVTkohslkqtGADHa7nbS0NG+EWGOnC4r4VYgPR478BJS9V1b++WK4nGUtU9//+BOtnCc8EmNdFBcX1/s19pTGVBZQeRqyxlQW8Gx5zptQ3Xrrrdx66628/vrrTJkypc4nu/HGG5k+fTomk4mlS5fy7LPP8swzz9T5uLXRECoqEU/q1q3bJT/n5fI3pGvTOL311lu8/fbbHDlyhPj4ePfywsJCrrnmmvPuGx4ezvHjx90/22w2wsPDK22TmZlJ27ZtcTqd5Ofn06JFi/PuW1JSwqxZs4iPj2fkyJHubVq1akVWVhZhYWFkZWXRsmXLC5bPz8+vXv7tnq3YdYRWzUP51a86AHDkyE/86ledLvp4Af7+kFuCb3ALunXr7KkwL1paWlq9X2NPaUxlAZWnIWtMZYELl6c29XmN2vOnTJnC119/zbFjx3C5fhmhp7xfeE21bt3a/TkxMZG7774bOH8FV9XyFi1akJeXh9PpxGKxcPz48UoVYnUaQkUlcrnT31D1dG1qpi6JZ3x8PMOGDePFF1/k/vvvdy8PCgqiefPm5923Z8+eHD58mPT0dMLDw0lJSeGFF16osE1UVBQbNmygb9++bN26lYEDB2IymYiKiuL+++8nKSkJm83G4cOH6dWrF4Zh8PDDD9O5c2eSkpIqHWvjxo3ceeedbNy4kREjRlx0uS+V0lKDAnvZKH+e0qJlS8x5JrL1DpWINEI1GpRi7ty5PPfcc3z11Vd8++23fPvtt3z33Xe1PllWVpb7844dO4iIiADKKpyUlBQcDgfp6enuSursis/hcJCSkkJUVBQmk4kBAwawdetWADZs2EBUVFSt4xERkcuPyWSiQ4cOLFiwgKCgIPd/UDZM+flYLBYWLFjAHXfcQWxsLKNHjyYiIoKlS5e638WdNGkSp0+fJjo6mtWrV/PAAw8AEBERwejRo4mNjeWOO+5gwYIFmM1mvvrqKzZt2sRnn31WaRTbO++8k08//ZSRI0eyb98+7rzzTi9eGc8odDgB8Pf1XEL17Mtr6DR5PjmFeodKRBqfGrVQfffdd2zZsqVWk/HNmTOHL774gpycHIYNG8Y999zDF198wYEDBwBo3749CxcuBCpWUmaz2V1JAe6Kz+VyMXHiRHcSNnfuXO677z6WLFlCt27dSExMrFXBRUTk8nT//fezYsUKJkyYgMlkcr/fC2XJ1oUGKYqMjHSPMlvu3nvvdX/28/Nj2bJlVe47bdo0pk2bVmFZv379OHjwYJXbt2jRosK8VJeDAnt5QuXZmVVC/CzkFKmFSkQanxolVBEREZw4ccI9UlFNvPjii5WWnS/pqaqSgqorPoCOHTtqqHQRkSZoxYoVQNnw5uJ5BcVlCZWfxXMtVK88/wS2f5/g9O/v8dgxRUQaiholVDk5OcTFxdGrVy98fX3dy1999VWvBSYiInI+X331Fd26dSMwMJBNmzbxr3/9i1tvvZUrrriivkO7rJW3UAV48B2q7775BwXZZzRsuog0SjVKqO65R98oiYhIw/LYY4/x/vvvc+DAAVavXk1iYiIPPvggb7zxRn2HdlkrT6j8LJ7t8mf20aAUItI41Sihuvbaa70dh4iISK1YLBZMJhM7duzgD3/4A4mJieoK7gHlXf78PdjlD8Bi9iH7jAPDMGr1TraISENXo4Sqb9++7ptfSUkJTqeTgIAAvv76a68GJyIiUp2goCBWrFjBBx98wBtvvEFpaSlOp7O+w7rs5Xuhyx+AxceEw1lKgd1JiL/vhXcQEblM1Cih+uabb9yfDcNg586d7N+/32tBiYiIXMjixYvZvHkzTz31FG3atCEjI4Pbb7+9vsO67LlbqDyYULVpewXk28kGThY4lFCJSKNS6w7SJpOJG264gU8++cQb8YiIiNRImzZtSEpKol+/fgBcccUVtZ5wXipzD0rhwS5/j734Knc9uhiAE/l2jx1XRKQhqFEL1bZt29yfS0tL+e677/Dz8/NaUCIiIheybds2Fi1axKlTpzAMw/1ujrqj102B3Ym/xQcfHxOuUuPCO9RQs4CyVqmsvGKPHVNEpCGoUUK1a9cu92ez2Uz79u15+eWXvRaUiIjIhTz//PO8+uqr/OY3v6nvUBqVAruTID9LhQmT62rJkw9jL3FBh3Fk5SuhEpHGpUYJ1TPPPOPtOERERGqlVatWSqa8oKD454TKg8f897++xQBMHcepy5+INDo1SqiOHz/OE0884e5G0a9fPx5++GHatm3r1eBERESq06NHD2bPns0NN9yA1Wp1Lx85cmQ9RnX5K2uhMuPRjAowAaH+vpzI11xUItK41CihmjdvHmPGjGHp0qUAvP/++8ybN4/Vq1d7NTgREZHqFBYWEhAQwKefflphuRKquikodhJo9WwLVblmAb6cKFCXPxFpXGqUUGVnZzNx4kT3zxMmTGDNmjVeC0pERORC1B3dO/LtTtoEWy+84UVoFuDLqQK1UIlI41KjYdObN2/Opk2bcLlcuFwuNm3aRPPmzb0dm4iIeImr9NK+x+KN8/3nP//h1ltvZcyYMQAcOHBAAyZ5QIG9xOOT+nb89W/o+OvflCVUhUqoRKRxqVEL1dNPP80TTzzBM888g8lkom/fvjz77LPejk1ELjN2Zwl+lks7YWd9nLMxMPv48cHnQy7Z+eIHeH7uwvnz5/Pggw+yYMECAK666ioeeOABpk+f7vFzNSVlXf48m1AlP1U2B9Wbn/9EdqHDPcS9iEhjUKOEatmyZfzv//4vzZo1A+D06dP87//+r7pbiEgFfhZfhvzl4Ut6zk+mPnVJzycNR1FREb169aqwzGz2bCLQFBXYnQT4euc6Ngvwxe4spcDuJMRfX4SISONQo4Tq4MGD7mQKyroApqWleS0oERGRC2nRogVHjhxxt3R8+OGHtGnTpp6jurzZnS5KXIbHu/w9+/B9AAye+hAAJwscSqhEpNGoUUJVWlpKbm5uhRYql8vl1cBERETO59FHH2X+/Pn83//9H0OHDqVDhw4sWrSovsO6rBUUOwE83kKV/p8fgbIWKoCTBXZ+3TrIo+cQEakvNUqobrvtNiZPnsyoUaOAsm8B7777bq8GJiIiUpWzp+yIjIxkwIABlJaWEhgYyLZt20hKSqrH6C5vBfayhMrP4r0ufwBZeRo6XUQajxolVOPGjaNHjx589tlnALz00kt06dLFq4GJiIhUpbCwECgb5e/bb79lxIgRGIbB+++/T8+ePes5ustbfnkLlbVGgwDXWnlCZVNCJSKNSI0SKoAuXbooiRIRkXo3c+ZMAP7whz+wfv16goOD3cvvuuuu+gztslfeQmX10uAeof6+mExwIv/SDtsvIuJNNU6oREREGpKTJ09itf4yAa3VauXkyZP1GNHlr/wdKj8Pv0MVcXVZy6GPj4lQf19O5GsuKhFpPJRQiYjIZWncuHFMmjSJ6OhoAHbs2MGECRPqOarLW3kLVaCHE6rZj/wyvQmcW50AACAASURBVEGzAF9OFqiFSkQaDyVUIiJyWZo2bRrDhg3jyy+/BOCZZ57h6quvrueoLm/lCZW/l96hAghVQiUijYwSKhERuWx1796d7t2713cYjUZ5QhXg4VH+HptTNjLwYy++SvMAX348UeDR44uI1CclVCIiIgJAfnEJZh8TvhYfSg3PHffE8Qz352YBvmQXOjAMwz0ps4jI5cx7bfoiIiJyWcktKiHU37vftTYL8MXuLKXQ4fLqeURELhUlVCIiIgJAXpGT4EuQUAGc1NDpItJIKKESEZEmZc+ePcTExBAdHc3KlSsrrXc4HMyePZvo6GgSExM5evSoe92KFSuIjo4mJiaGvXv3upfPmzePQYMGMWbMmArHWr58OUOHDiUhIYGEhAR2797tvYJ5QF5xCSF+vhge7O53rvKE6oQGphCRRkIJlYiINBkul4uFCxeyatUqUlJS2Lx5M4cOHaqwzdq1awkNDWX79u1MnTqVRYsWAXDo0CFSUlJISUlh1apVPP7447hcZd3WJkyYwKpVq6o859SpU9m0aRObNm0iMjLSuwWso9yiEkL8LXg6n+rRtz89+vYHoHlgWUKVlVfs4bOIiNQPJVTy/9u78/ioyrv//69ZMpN9g2QCGFE0KiJItRZQSmpoEiAie7+1v9aHVIoFLVoqFTdUXGqVgrjcvaHYFpebVlGgN7EFCS2LinhbaUQDohgISiYQshGSmczM+f0RGI1JSAiZJcn7+XjwIHPOdc71uc6cmTOfua65johIj1FYWEj//v1JT0/HZrORl5dHQUFBkzKbN29m0qRJAOTm5vLOO+9gGAYFBQXk5eVhs9lIT0+nf//+FBYWAnDVVVeRkJAQ9PZ0tuq6BmLsnT/kb9a8+5k1737gqx6qMg35E5FuQgmViIj0GE6nk7S0NP9jh8OB0+lsVqZPnz4AWK1W4uLiqKioaNe2LXn55ZcZP348d999N1VVVZ3UksCoqvMQG4CE6uviIiMwoYRKRLoPTZsuIiISIDfccAOzZ8/GZDKxdOlSHn/8cX7zm9+cdhuXy0VRUVGQImyq6oQbw13HwYMHmix3u93Nlp2JxQ/MA2DuQ08CEGMzs//LoyFrZ319fcjq7mzdqS2g9oSz7tQW6Nz2KKESEZEew+FwUFpa6n/sdDpxOBzNyhw+fJi0tDQ8Hg81NTUkJSW1a9tv6t27t//vadOm8fOf/7zNGO12OwMHDmxvkzpNfYOXBt9+HL0SOPfctCbrDh48wLnn9u/wvhtcjb+XOrWPpJhKXNhC0k6AoqKikNXd2bpTW0DtCWfdqS3QdnvOJNnSkD8REekxBg8eTHFxMSUlJbjdbvLz88nKympSJisrizVr1gCwYcMGhg8fjslkIisri/z8fNxuNyUlJRQXFzNkyJDT1ldWVub/e9OmTWRkZHR+ozpJdV0DAFG2wH/XmhBt42ithvyJSPegHioREekxrFYrCxYsYMaMGXi9XqZMmUJGRgZLly7lsssuY/To0UydOpV58+aRnZ1NQkICS5YsASAjI4OxY8cybtw4LBYLCxYswGKxADB37lx27txJRUUFo0aN4he/+AXTpk3jySefZM+ePQD069ePhQsXhqztbamuP5VQWQJeV0JUBJ8fOR7wekREgkEJlYiI9CiZmZnNpi+//fbb/X/b7XaefvrpFredNWsWs2bNarZ88eLFLZZ/8sknzyLS4Ko62UMVHRGchKq81o1hGJhMpoDXJyISSEqoREREhOo6DwCRAUiovn31qCaPE6IicHl81Lq9AZ9VUEQk0PQuJiIiIv4hfzEB+A3V9NvubPI4OcYGQGlVHRemxnV6fSIiwaRJKUREROSrIX/2wA/5S4m1A3Cooi7gdYmIBFrAEqq7776bESNGcN111/mXVVZWMn36dHJycpg+fbr/BoeGYfDII4+QnZ3N+PHj+eijj/zbrFmzhpycHHJycvyzLgHs3r2b8ePHk52dzSOPPIJhGIFqioiISLfnn+UvAEP+5v70/zH3p//P/7h3bGMP1YHyE51el4hIsAUsoZo8eTIrVqxosmz58uWMGDGCjRs3MmLECJYvXw7A1q1bKS4uZuPGjTz88MM8+OCDQGMC9uyzz/LKK6/w6quv8uyzz/qTsAcffJCHH36YjRs3UlxczNatWwPVFBERkW6vqq6BqAgzVkvnfzRw1dfhqv+qNyop2obFbKKkQgmViHR9AUuorrrqKhISEposKygoYOLEiQBMnDiRTZs2NVluMpkYOnQo1dXVlJWVsX37dq655hoSExNJSEjgmmuuYdu2bZSVlXH8+HGGDh2KyWRi4sSJFBQUBKopIiIi3V51nYfYyIigjPgwm030jrFx6JgSKhHp+oL6G6ry8nJSU1MBSElJoby8HGi823xa2ld3ZU9LS8PpdDZb7nA4Wlx+qryIiIh0TFVdA/GRVoI1gr53nJ0vK+uDU5mISACFbJY/k8kUkntPuFwuioqKgl6vSKAMHDgw6HW29hoKRSwQXvF0lfcXHRv5pur6BmLtVoL1i+TesXY++rIqSLWJiAROUBOqXr16UVZWRmpqKmVlZSQnJwONPU+lpaX+cqWlpTgcDhwOBzt37vQvdzqdfOc732m1fHvY7faQfegT6S7C7TUUTvGEUyzh5uvHRslV+KmubyAxyhaQfV+TldNsWUqcnaPH3bg8XuzWwM8sKCISKEEd8peVlcXatWsBWLt2LaNHj26y3DAMdu3aRVxcHKmpqYwcOZLt27dTVVVFVVUV27dvZ+TIkaSmphIbG8uuXbswDKPJvkREROTMVdU1EBOgKdN/NOM2fjTjtibLep+cOv2whv2JSBcXsB6quXPnsnPnTioqKhg1ahS/+MUvmDlzJnfccQerV6+mb9++PPXUUwBkZmayZcsWsrOziYqK4rHHHgMgMTGR2bNnM3XqVABuvfVWEhMTAXjggQe4++67qa+vZ9SoUYwaNarlQERERKRN1XUeYu3BG7iSEvfVvajO6x0TtHpFRDpbwN45Fy9e3OLylStXNltmMpl44IEHWiw/depUf0L1dYMHD2b9+vVnF6SIiIjg8xlU1zcQbQtMD9WtP7oegOf+52/+ZSkn70V18Fgt0Dsg9YqIBENQh/yJiIhI+Dnu9mAYEGMLXg9VcowdswkOaup0EenilFCJiIj0cNV1DQAB66FqicVsIjnGxqGKurYLi4iEMSVUIiIiPVzVyYQqMogJFTROTPGFEioR6eKUUImIiPRw1XUeAKIigptQpcTaOVylhEpEuraQ3dhXREREwsOpHqqoAP2GKmvcxBaX946z8/ZnR/F4fVgt+o5XRLomJVQiIiI9XHX9yd9QRQQmqZny45+2uDwl1o7XgMNV9aQnRwekbhGRQNPXQSIiIj2cf1KKAN2Hqr7uBPV1zWfz633yXlRfVGrYn4h0XUqoREREerjqugZMpsD9hupXN/+QX938w2bLU2IbEypNnS4iXZkSKhERkR6uut5DnN2KCVNQ6+3V5Oa+IiJdkxIqEZEg8PjcPaJO6Zqq6hqIi4zAwAhqvREWM0nRERw6piF/ItJ1aVIKEZEgsJpt/Hb7j4Na510jXwpqfdJ1Vdc1EBdpxQhuPgWcvBeVfkMlIl2YeqhERER6uKq6BmIjrUHun2qUEmfncGV9CGoWEekc6qESERHp4arrG+ibGBWw/Y+bckOr63rH2tn5+TF8PgOzObi/4RIR6QxKqERERHq46joPFzkC95Eg7zQJVUqcHY/PoKzGRVpCZMBiEBEJFA35ExER6eGq6hqIsQVmynSAymPlVB4rb3Fd75NTpx+q0NTpItI1KaESEZEeZevWreTm5pKdnc3y5cubrXe73dxxxx1kZ2czbdo0Dh065F+3bNkysrOzyc3NZdu2bf7ld999NyNGjOC6665rsq/KykqmT59OTk4O06dPp6qqKnAN6yC3x0ddg5eYAN3UF+De26Zz723TW1yXevLmvgfKlVCJSNekhEpERHoMr9fLwoULWbFiBfn5+axfv55PP/20SZlXX32V+Ph43nzzTW666SYWLVoEwKeffkp+fj75+fmsWLGChx56CK/XC8DkyZNZsWJFs/qWL1/OiBEj2LhxIyNGjGgxgQu16voGAKJtofkVQGq8HavZxMeHwy/ZFBFpDyVUIiLSYxQWFtK/f3/S09Ox2Wzk5eVRUFDQpMzmzZuZNGkSALm5ubzzzjsYhkFBQQF5eXnYbDbS09Pp378/hYWFAFx11VUkJCQ0q6+goICJEycCMHHiRDZt2hTgFp656rrGhCrKFpqPBFazmX5JUewprQlJ/SIiZ0sJlYiI9BhOp5O0tDT/Y4fDgdPpbFamT58+AFitVuLi4qioqGjXtt9UXl5OamoqACkpKZSXt/w7olCqrvcAEBURuN9QtSU9KZp9zuMhq19E5Gxolj/pElwNHuwRwTtdg12fiHR/JpMJk6ntacFdLhdFRUVBiKjR7i8af7t0vOoYBxsqWyzjdrs5ePBAh+uod7kAWt1HnNlFWY2L/yv8mJiIwE+dXl9fH9RjHEjdqS2g9oSz7tQW6Nz26BOjdAn2CCtX3/Zw0Op7+9n7g1aXiASPw+GgtLTU/9jpdOJwOJqVOXz4MGlpaXg8HmpqakhKSmrXtt/Uq1cvysrKSE1NpaysjOTk5DZjtNvtDBw48Axb1nGfur8ESunftw+p8S1PW37w4AHOPbd/h+u44ac/B2h1H0NMFfz9k7344tIYeH7bx+hsFRUVBfUYB1J3aguoPeGsO7UF2m7PmSRbGvInIiI9xuDBgykuLqakpAS3201+fj5ZWVlNymRlZbFmzRoANmzYwPDhwzGZTGRlZZGfn4/b7aakpITi4mKGDBly2vqysrJYu3YtAGvXrmX06NGBadhZqKo7NSlF4Ib8fT9vEt/Pm9Tq+vSkaAA+/lITU4hI16OESkREegyr1cqCBQuYMWMG48aNY+zYsWRkZLB06VL/5BRTp06lsrKS7Oxs/vSnP3HnnXcCkJGRwdixYxk3bhwzZsxgwYIFWCyNScjcuXP54Q9/yOeff86oUaN49dVXAZg5cyZvvfUWOTk5vP3228ycOTM0DT+NYMzy5/zyC5xfftHq+uQYGzE2Cx8frg5YDCIigaIhfyIi0qNkZmaSmZnZZNntt9/u/9tut/P000+3uO2sWbOYNWtWs+WLFy9usXxSUhIrV648i2gDr6quAZvFTITVjNdnBKSOhXc2HrPn/udvLa43mUykJ0fziVMz/YlI16MeKhERkR6sus5DXKQVwwhMMtVe6cnRfFpWG/I4RETOlBIqERGRHqy6vqExoQpxHOlJ0Rx3efiyqj7EkYiInBklVCIiIj1YdV0DcZERhLpj6Nzkxokp9pbqd1Qi0rUooRIREenBjtW6iY8K/U+q05OjAPjoCyVUItK1hP4dVEREREKmrMbFgN4xAa3jhptnt1km2malV4yNIvVQiUgXo4RKRESkh/L6DMqPu0iKjghoPSNHj2lXucaZ/o4HNBYRkc6mIX8iIiI9VHmtC58BidG2gNZzYP8+Duzf12a5c5OjKT5aS4PXF9B4REQ6k3qoREREeqiyahcAsZGB/TjwxH2/Alq/D9Up6cnReHwG+4/UcnFaXEBjEhHpLOqhEhER6aGOHG9MqOIjAzvkr73Skxonpig6XBXiSERE2k8JlYiISA915GQPVUKYJFT9EqOwmE189KUmphCRrkMJlYiISA9VVtN4E934qPBIqKwWM+cmR/Pvg5WhDkVEpN2UUImIiPRQR2pcxEVaibCGz8eBQX3jKTxUSZ3bG+pQRETaRZNSiIiI9FBlNS56x9oxDCOg9dx066/aXXZQ33jWFx7m/QMVjMzoHcCoREQ6hxIqERGRHupIjYteMTZ8gc2nuOqazHaXvdgRj8VsYtu+I0qoRKRLCJ8+fpEuwtXg6RF1ikj3V1bjIjkmsPegAvjk4w/55OMP21U2ymbhgpQY3vrsaICjEhHpHOqhEjlD9ggrV967MKh1vv/ogqDWJyLdn2EYHKlxMWJAcsDrWvrIvUDb96E6ZVDfBNbt+oLq+oawmdJdRKQ1IemhysrKYvz48UyYMIHJkycDUFlZyfTp08nJyWH69OlUVTXeg8IwDB555BGys7MZP348H330kX8/a9asIScnh5ycHNasWROKpoiIiHRJx10e6hq8JEaHX8IyqG88PgPe+/xYqEMREWlTyIb8rVy5knXr1vH6668DsHz5ckaMGMHGjRsZMWIEy5cvB2Dr1q0UFxezceNGHn74YR588EGgMQF79tlneeWVV3j11Vd59tln/UmYiIiInF5Zzcmb+kYFfsjfmcpIjSPCYmLrviOhDkVEpE1h8xuqgoICJk6cCMDEiRPZtGlTk+Umk4mhQ4dSXV1NWVkZ27dv55prriExMZGEhASuueYatm3bFsomiIiIdBlHTiZUcfbwG/1vs5q5yBHHO5+VhzoUEZE2hSyhuvnmm5k8eTJ//etfASgvLyc1NRWAlJQUyssb30SdTidpaWn+7dLS0nA6nc2WOxwOnE5nEFsgIiLSdX3VQxV+Q/6g8XdUnziPc6zWHepQREROKyRfS61atQqHw0F5eTnTp09nwIABTdabTCZMJlNA6na5XBQVFQVk3xI4AwcODHqdrZ0noYgFwiuecIoFwiuecIoFwisevfeGl1M9VAlRgf8o8PM77zvjbQb1jQdgx/5yxg3u09khiYh0mpAkVA6HA4BevXqRnZ1NYWEhvXr1oqysjNTUVMrKykhOTvaXLS0t9W9bWlqKw+HA4XCwc+dO/3Kn08l3vvOdNuu22+0h+2AjXUu4nSfhFE84xQLhFU84xQLhFc/XY1FyFXplNfVEWExE26wBvw/V4Cvavj5/04CUGCIjzGz95IgSKhEJa0Ef8nfixAmOHz/u//utt94iIyODrKws1q5dC8DatWsZPXo0gH+5YRjs2rWLuLg4UlNTGTlyJNu3b6eqqoqqqiq2b9/OyJEjg90cERGRLulIjYvesfag1PXhv3fy4b93tl3wa6xmMwPT4tm27yiGEeCMT0TkLAS9h6q8vJxbb70VAK/Xy3XXXceoUaMYPHgwd9xxB6tXr6Zv37489dRTAGRmZrJlyxays7OJioriscceAyAxMZHZs2czdepUAG699VYSExOD3RwREZEu6UiNi16xNoKRq/z3okeA9t+H6pRhA5L57y37+b8DFVx1XuDvlyUi0hFBT6jS09P529+av6EmJSWxcuXKZstNJhMPPPBAi/uaOnWqP6ESERGR9iurdpGWEEk49/0MO78XK98+wKp3DyqhEpGwFTbTpouIiEjwHDnuIikMb+r7dZERFkZc0Is3dh+mpr4h1OGIiLRICZWIiEgP4/b4OFbrJjHMEyqA712UQn2Dj//9z5ehDkVEpEVKqERERHqY8tpTU6aHf0J1YWos5yRF8Zf3SkIdiohIi5RQiYhIj7J161Zyc3PJzs5m+fLlzda73W7uuOMOsrOzmTZtGocOHfKvW7ZsGdnZ2eTm5rJt27Y29zl//nyysrKYMGECEyZMCJvp4suqg3tT39vve5Tb73u0Q9uaTCa+d1EqhYeq2Oes6eTIRETOnhIqERHpMbxeLwsXLmTFihXk5+ezfv16Pv300yZlXn31VeLj43nzzTe56aabWLRoEQCffvop+fn55Ofns2LFCh566CG8Xm+b+/z1r3/NunXrWLduXdjcF6zs5E194+zBmZvqoksHc9Glgzu8/XczemM1m1i182AnRiUi0jmUUImISI9RWFhI//79SU9Px2azkZeXR0FBQZMymzdvZtKkSQDk5ubyzjvvYBgGBQUF5OXlYbPZSE9Pp3///hQWFrZrn+HmSE1wh/y999YW3ntrS4e3j4+K4Ir+Sbz+wRe4PN5OjExE5OwFfdp0ERGRUHE6naSlpfkfOxwOCgsLm5Xp06cPAFarlbi4OCoqKnA6nVx++eVNtnU6nQCn3eeSJUt47rnnGDFiBHfeeSc2m+20MbpcroAPDfx4fwUA1cecVHp9bZZ3u90cPHigw/X99+LGe0g60s/r8D4u7wU7P2/gyXX/x5RL4zu8H4D6+vqwGX55trpTW0DtCWfdqS3Que1RQiUiIhIgc+fOJSUlhYaGBu6//36WL1/Obbfddtpt7HZ7wIcGGns+JCn6OP3PORdvO+7se/DgAc49t3+H64u02wHOah/nngsfHNnLy7sqmPH9y0lLiOzwvoqKisJm+OXZ6k5tAbUnnHWntkDb7TmTZEtD/kREpMdwOByUlpb6HzudThwOR7Myhw8fBsDj8VBTU0NSUlKr255un6mpqZhMJmw2G5MnT+bDDz8MZPParazGRa9YO76wvq1vcz8e3h+Pz8djb3wc6lBERPyUUImISI8xePBgiouLKSkpwe12k5+fT1ZWVpMyWVlZrFmzBoANGzYwfPhwTCYTWVlZ5Ofn43a7KSkpobi4mCFDhpx2n2VlZQAYhsGmTZvIyMgIboNbcaTGRa8YG+3onAorjvhIxg/py9/+c5h395eHOhwREUBD/kREpAexWq0sWLCAGTNm4PV6mTJlChkZGSxdupTLLruM0aNHM3XqVObNm0d2djYJCQksWbIEgIyMDMaOHcu4ceOwWCwsWLAAi8UC0OI+Ae68804qKiowDINLLrmEhx56KGRt/7ojNS6GnJMQ6jA65Pqhfdm67wj3r9vNG3O+i9Wi74ZFJLSUUImISI+SmZlJZmZmk2W33367/2+73c7TTz/d4razZs1i1qxZ7donwAsvvHCW0XY+wzA4UuMiMTp4N/X99SO/67R92a0Wfjy8P09t2seSTZ8wL/eSTtu3iEhHKKESERHpQb6sqsft9ZEaaw9anf0HdO5Qx++cl0zWxak898/P6JMQxY+Hd3yyCxGRs6WESkREpAfZc7gagD5JUUGrc3vBPwAYOXpMp+zPZDLx05HnU1nnZsG63aTE2ckdlNb2hiIiAaCBxyIiIj3IntIaAPrEd3za8TO16vn/YtXz/9Wp+7SYTfwiK4MLUmKZs+oD3is+1qn7FxFpLyVUIiIiPcje0hr6JkQSFdH1B6lERli4M+diesfa+Mnz7/L3Dw+HOiQR6YGUUIURt7uhW9cnIiKht6e0mgEpse26oW9XEB8Vwf3XDeK8XjHMevnfLNvyGUY3aZuIdA1d/+upbsRmiyBvzP1Bqy//Hw8HrS4REQk9l8fL/iO1fOe85FCH0qkSoiK4e+xAlm/7jN/8fQ+fH61l4YTLsFn1vbGIBJ4SKhERkR7is7JaPD6D9OTgTUgRLDarmdnfu5DUuEj+8l4Jnx05zu9/fCW9gziboYj0TEqoREREeoi9zsYZ/tISgptQLVj0+6DUYzaZ+MG300lPiuK/t+7n+me384cbv82gvl3zJsYi0jWoL1xERKSH2FNaQ4TFFNR7UAE4+vbD0bdf0OobcUFvHrzuUtweH1N//w5vfuwMWt0i0vMooRIREekh9hyu4fzeMZjNpqDWuyl/DZvy1wS1zvNTYnl4wmWkJ0dxy4v/x8q3i4Nav4j0HEqoREREeoi9pTUM6B2DL8iT4K15+U+seflPwa0USIy2cc+4gXy7fzIP/O0jHl7/Mb5gN15Euj0lVCIiIj1A5Qk3pdX19O8VE+pQgsputXD76AzGXpbG89s/57ZV/8bt9YU6LBHpRjQphYiISA+wp7QGgL6J3W+Gv7aYzSZuHHEeKbF2XthxgJIjUbw04CISoiNCHZqIdAPqoRIREekB9p5MqPrER4Y4ktAZO7gPc7Iu5OOyOqb+99t8WVkX6pBEpBtQQiUtcrs9PaJOEZGeYk9pDQlRET2+V2bEBb2ZcWUvvqisY+Jzb/HBwYpQhyQiXZyG/EmLbDYr3///Hg5qnZtevj+o9YmI9CR7Squ5MCUGIwRzMjz6bPAnpDidAck2Hhw/iMVv7uX/Ld/B45MHM/mKc0Idloh0UeqhEhER6eZ8PoNPSms4r3cMoZjjLjG5F4nJvUJQc+vSk6N5aMJlXOSIZe4r/+GR9R/j8nhDHZaIdEFKqERERLq5QxV11Lq9nJscHZL6819bRf5rq0JS9+nER0Zw15hLyB3kYMX2z7n+me3s/qIq1GGJSBfToxMqt6uhR9QpIiI9247PywHolxSaGf7eeG0Vb4RhQgVgNZu56erzuWvMxRw97mbCc2+xeONe6tzqrRKR9unRv6Gy2SMYN3hWUOt848PfB7U+ERGRv75Xwnm9ojk3KRqv7mvboqHpSTw+JY6X3z3A05s/ZdV7Jcz+3gXc8J1ziYywhDo8EQljPbqHSkREpLvbW1rD+wcqyBvSR8lUG2LtVm4ZdQEPjr+UPgmRPPS/H5P55D957p+f4qyuD3V4IhKmenQPlYiISHe3audBIiwmvnNecqhD6TIuTovn7rHxFB2uZu2uL3hyw15+t3Ev116cyqQr+vG9i1OJtesjlIg00ruBiIhIN1Xf4OX1fx8i65JU7FZLSGb468oG9olnYJ94nNV1bNt3lH99coSCPWXYLGauubAX37/UwaiMFNJDNNmHiIQHJVQiIiLd1BsfHqa63kPWJSkhTaZ+9/xfQlj72XPERzH1ynQmf+scPjtynA8OVvBu8TH+ufcIAOf3jmHkhb0ZNiCZq85LxhEfGeKIRSSYlFCJiIh0U6t2HqR/cjTn94oN6e+nIqO6Rw+O2WwiwxFHhiOOad9Op7S6no+/rGb3l1Wsfv8QL+44AMC5ydEM7pfAxWlxXJwWx/m9Y0iJtZMYHYHJZGp1/4Zh4PL4cDX4aPD58BkGGGAymYiyWYiKsGAxt769iISGEioREZFuaJ+zhveKK5j9vQEhn4zitZf+CMCUH/80tIF0IpPJRJ+EKPokRDF6oAOvz0fJsTo+PVLDJ87j7CqpJP/Dw022ibCYSIiyYTWbsJhNmM3g9vj8SVS9FwOiKAAAGIxJREFUx4vRxnNlt5rpHWsnJc5OapydaKOOK6sOMKB3DBemxpIaZz9t0iYinU8JlYiISDdTU9/AfWt3N05GcX6vUIfD5jfWAt0rofomi9nMeb1jOK93DN8f2LjM7fHyRWUdx467qXY1UF3nodblwWcYeA0wfAZWixmb1USExYzdaibCYsZmMWM2m2jsjDI19lx5fbgbfNQ3eKmqa+DYCTefHjnOFxUnWFu02x9HUnQEA/vEc2mfeC7rl8Bl/eI5v3eserZEAqjLJ1Rbt27l0UcfxefzMW3aNGbOnBnqkEREJIy1dd1wu938+te/5qOPPiIxMZElS5ZwzjnnALBs2TJWr16N2Wzmvvvu47vf/e5p91lSUsLcuXOprKxk0KBBPPHEE9hstoC27+hxFzf9aSd7Dtcwf+wlRGoyipCxWS2c3zuW83sHro5DJQeI692H0ioXh6vqKDl2gs+P1vLCOwdwe30AREVYyHDEkpEax0WOWPr3isYRH0laQiTJMTZsFnOzXi3DMHB7fdQ3+HA1eGnwGRiGgWGAxWwixmYl2m4hwqI78Ih06YTK6/WycOFC/vSnP+FwOJg6dSpZWVlceOGFoQ5NRETCUHuuG6+++irx8fG8+eab5Ofns2jRIp566ik+/fRT8vPzyc/Px+l0Mn36dDZs2ADQ6j4XLVrETTfdRF5eHgsWLGD16tX86Ec/Clj7DlWc4Mbnd/JlVR0LJwxiQO9YJVPdnM+AhCgbCVE2Lk6L8y83DIPDVfUcLK/lYEVjorX1kyO89u9DzfZhMkGk1UKExYTHZ+DxGv5krC12q5mUODuO+Egc8XbOSYomPTmac0/+65cYhc2qpEu6ty6dUBUWFtK/f3/S09MByMvLo6CgQAmViIi0qD3Xjc2bN3PbbbcBkJuby8KFCzEMg4KCAvLy8rDZbKSnp9O/f38KCwsBWtznBRdcwI4dO/jd734HwKRJk3j22WcDmlD95o09HD3u4jeTBpOWEKVkqgczmUz0TYyib2IUw7+2vK7BS/lxFxUn3FTXNVDr8uD2Gni8Pjw+A4vJhNViwmIyEdFkCCI0jkA04fM1Tp5R3+Clzu2loq6BY7VuPvqymk0flzVJxkwm6JsQRb+kKPomRJKWEEVavJ2kGBuJ0TaSoiOIsVuJPjnpRn2DD7fHR4TFdNa/BTMMA58BDSfb5mrwfhX3ydhr3V5qXY1DMU+4vdS6PY29ch4vbo8Pr++rV5HZZCLCYsJmNWO3WoiMMBMVYcEeYfEP14ywmDGb4OR8Ihw4WMsnri9we3zUexp7+07V3+A1GmPzGhgYmE0mfx2RERYiIxqPSYzdQozd2tgraLMQbbMSZWus32Y1Y7dYGp8zc+P2FrPJ33afYeDxGbg9Phq8Tdte5/Zy3OWh1u2h1nVy2cn1p9re4PXhM8BqNlFVWUHKZx+fbH9j3baTbbZaGus+5dS2bs/Jf96v/vf5GmPzGgZmE/7jZrOYibZbiI6wEH2qvXYLsXYrUREW/8QsdquZiK/VbTYR8t8NdumEyul0kpaW5n/scDj8FzcREZFvas91w+l00qdPHwCsVitxcXFUVFTgdDq5/PLLm2zrdDoBWtxnRUUF8fHxWK1Wf5lT5QNl/thLuCVzAPUNPoy2Zjc4A9aTH1o67OS24fIznrNuTxg507bE2CzEnOw96myNH2oNquo8lFXXc/S4i6PHXZTVuCitque94gqc1Yfx+No6N4sBTiYIYMKEydSYnMFXj09pTF4MfxJzKpnwtllP62yWxoTB+rWD6zUak4QGj4H3jF5fzV/3ZlNjHVZLYx0mU2Nvo89n0OBrnKQkVF+IRFhM2CxmLCcnTzGdTKIbvF6M/bW4Pb52PIfNnUo6/c+rqTHx85xMLF2ejrfZbGo8X06dGxEWM8/c8C2uvSS1g3s8M106oeoIl8tFUVGR//HvXpkT1Pq/XndLFi0J3DeX39RWLM/cNzVIkTRqK57nbw1ePG3F8tKPpwUpkkZtxfOXvB8GKZK2Y/nDsB8HKZJGbcXz6/N+FqRI2o7l+l73BimSRm3Fc2H8H4IUSfNYXC5X0Oruar55nTpTFsMgupM/iV3cywZ1HU8GX3j+5Ll2FvvoTGfbnnASjm2JBvrEAXEAlpNLWk/gvkqUmq1pVqY9muc6RouPOvE7h6D7+vE4k+8GunLbWz9Pvlra5DwxyikqKj/tPk/3Xnsm16kunVA5HA5KS0v9j51OJw6H47TbDB06NNBhiYhImGrPdcPhcHD48GHS0tLweDzU1NSQlJR02m1bWp6UlER1dTUejwer1UppaWmb1yjQdUpEpKvp0r8SHDx4MMXFxZSUlOB2u8nPzycrKyvUYYmISJhqz3UjKyuLNWvWALBhwwaGDx+OyWQiKyuL/Px83G43JSUlFBcXM2TIkFb3aTKZGDZsmH/iijVr1ugaJSLSDZmMzhxkHQJbtmzhsccew+v1MmXKFGbNmhXqkEREJIy1dN1YunQpl112GaNHj8blcjFv3jyKiopISEhgyZIl/gknfv/73/Paa69hsVi45557yMzMbHWf0Dht+i9/+UuqqqoYOHAgixYtCvi06SIiElxdPqESEREREREJlS495E9ERERERCSUlFCJiIiIiIh0kBKq09i6dSu5ublkZ2ezfPnyZutff/11hg8fzoQJE5gwYQKvvvpqwGK5++67GTFiBNddd12L6w3D4JFHHiE7O5vx48fz0UcfBSyW9sTz7rvvcuWVV/qPzbPPPhuwWA4fPsxPfvITxo0bR15eHitXrmxWJljHpz2xBPPYuFwupk6dyvXXX09eXh5PP/10szJut5s77riD7Oxspk2bxqFDh0IWSzBfU6d4vV4mTpzILbfc0mxdsI5Ne2IJ9rHJyspi/PjxTJgwgcmTJzdbH+z3HAmdtq6F4ail87eyspLp06eTk5PD9OnTqaqqAsLzXG7pGtuR+NesWUNOTg45OTn+iVaCraW2PPPMM3z3u9/1v59t2bLFv27ZsmVkZ2eTm5vLtm3b/MvD5Txs7TrfFZ+f1trSVZ+f1j5nlJSUMG3aNLKzs7njjjtwu93A6a/xrbWzVYa0yOPxGKNHjzYOHjxouFwuY/z48ca+ffualHnttdeMhx56KCjx7Ny509i9e7eRl5fX4vp//etfxs0332z4fD7jgw8+MKZOnRrSeHbs2GHMnDkzoDGc4nQ6jd27dxuGYRg1NTVGTk5Os+cqWMenPbEE89j4fD7j+PHjhmEYhtvtNqZOnWp88MEHTcq89NJLxv33328YhmGsX7/euP3220MWSzBfU6f88Y9/NObOndvicxKsY9OeWIJ9bK699lqjvLy81fXBfs+R0GjPtTActXT+/va3vzWWLVtmGIZhLFu2zHjiiScMwwjPc7mla+yZxl9RUWFkZWUZFRUVRmVlpZGVlWVUVlaGRVuefvppY8WKFc3K7tu3zxg/frzhcrmMgwcPGqNHjzY8Hk9YnYetXee74vPTWlu66vPT2ueMOXPmGOvXrzcMwzDuv/9+4+WXXzYMo/VrfGvtPB31ULWisLCQ/v37k56ejs1mIy8vj4KCgpDFc9VVV5GQkNDq+oKCAiZOnIjJZGLo0KFUV1dTVlYWsniCKTU1lUGDBgEQGxvLgAEDcDqb3uQwWMenPbEEk8lkIiYmBgCPx4PH4zl5J/uvbN68mUmTJgGQm5vLO++8gxGAuWraE0uwlZaW8q9//YupU1u+aXSwjk17Ygk3wX7PkdAIt2vh2Th1zgJMnDiRTZs2NVkeTudyS9fYM41/+/btXHPNNSQmJpKQkMA111zTvm/ag9CW1hQUFJCXl4fNZiM9PZ3+/ftTWFgYVudha9f5rvj8nOlnlnB/flr7nLFjxw5yc3MBmDRpkj+21q7xrbXzdJRQtcLpdJKWluZ/7HA4WjzJNm7cyPjx45kzZw6HDx8OZohNfDPetLS0kH6QB9i1axfXX389M2bMYN++fUGp89ChQxQVFXH55Zc3WR6K49NaLBDcY+P1epkwYQJXX301V199dYvHpk+fPgBYrVbi4uKoqKgISSwQ3NfUY489xrx58zCbW34rDOaxaSsWCP77zc0338zkyZP561//2mxdOL7nSOdr77UwHH3z/C0vLyc1NRWAlJQUysvLga5zLp9p/OH+3L388suMHz+eu+++2z88rrWYw7UtX7/Od/Xn55ufWbrq8/PNzxnp6enEx8djtVqBpq/v1q7xHWmPEqqzcO2117J582b+93//l6uvvpq77ror1CGFjUGDBrF582b+9re/8ZOf/IRbb7014HXW1tYyZ84c7rnnHmJjYwNeX0djCfaxsVgsrFu3ji1btlBYWMgnn3wS0PrOJpZgvqb++c9/kpyczGWXXRawOjozlmC/36xatYo1a9bwhz/8gZdffpn33nsvoPWJdKa2zl+TyRTyHvKz0dXjv+GGG3jzzTdZt24dqampPP7446EO6Yyd7jrf1Z6fb7alKz8/3/ycsX///qDUq4SqFQ6Hg9LSUv9jp9OJw+FoUiYpKcl/g8Zp06aF9Ies34y3tLS0WbzBFBsb6+92zczMxOPxcOzYsYDV19DQwJw5cxg/fjw5OTnN1gfz+LQVS7CPzSnx8fEMGzas2ZACh8Ph7+3weDzU1NSQlJQUkliC+Zr697//zebNm8nKymLu3Lns2LGDO++8s0mZYB2b9sQS7PebU6+PXr16kZ2d3Wy4Q7i950hgtOdaGI5aOn979erlH8pXVlZGcnKyv2xXOJfPNP5wfu569+6NxWLBbDYzbdo0PvzwQ6D18y3c2tLSdb6rPj8ttaWrPz/w1eeMXbt2UV1djcfjAZq+vlu7xnekPUqoWjF48GCKi4spKSnB7XaTn59PVlZWkzJfH2O9efNmLrjggmCH6ZeVlcXatWsxDINdu3YRFxfn73oOhSNHjvh/a1JYWIjP5wvYh3TDMLj33nsZMGAA06dPb7FMsI5Pe2IJ5rE5duwY1dXVANTX1/P2228zYMCAJmWysrL8swtt2LCB4cOHB+SbtfbEEszX1K9+9Su2bt3K5s2bWbx4McOHD2fRokVNygTr2LQnlmAemxMnTnD8+HH/32+99RYZGRlNyoTbe44ERnuuheGmtfP31DkLsHbtWkaPHg10nXP5TOMfOXIk27dvp6qqiqqqKrZv387IkSND2QS/r7+fbdq0yf/+kpWVRX5+Pm63m5KSEoqLixkyZEhYnYetXee74vPTWlu66vPT0ueMCy64gGHDhrFhwwagcWbFU7G1do1vrZ2nYw1gu7o0q9XKggULmDFjBl6vlylTppCRkcHSpUu57LLLGD16NC+++CKbN2/GYrGQkJDAb37zm4DFM3fuXHbu3ElFRQWjRo3iF7/4hT/bvuGGG8jMzGTLli1kZ2cTFRXFY489FrBY2hPPhg0bWLVqFRaLhcjISBYvXhyw7u/333+fdevWcdFFFzFhwgR/fF9++aU/nmAdn/bEEsxjU1ZWxvz58/F6vRiGwZgxY7j22mubnMdTp05l3rx5ZGdnk5CQwJIlS0IWSzBfU60JxbFpTyzBPDbl5eX+oaher5frrruOUaNGsWrVKiA07zkSGq1dC8NZa+fv4MGDueOOO1i9ejV9+/blqaeeAgjLc7mla+zMmTPPKP7ExERmz57tn+jm1ltvJTExMSzasnPnTvbs2QNAv379WLhwIQAZGRmMHTuWcePGYbFYWLBgARaLBSBszsPWrvNd8flprS3r16/vks9Pa58zLrzwQn75y1/y1FNPMXDgQKZNmwbQ6jX+dO1sjckI1JRVIiIiIiIi3ZyG/ImIiIiIiHSQEioREREREZEOUkIlIiIiIiLSQUqoREREREREOkgJlYiIiIiISAcpoRIJc3/+85+pq6vzP/7Zz37mv8+CiIj0HL/73e/YsWMHmzZtYtmyZWe1r6KiIrZs2eJ/XFBQwPLly882RJEeSQmVSBgwDAOfz9fiuhdeeKFJQvWHP/yB+Pj4YIUmIiJh4j//+Q9Dhw5l586dfPvb326z/Kn7Q7bkmwnV6NGjmTlzZqfE2dlOd43sSnVI96X7UImEyKFDh7j55pu5/PLL+eijjxgyZAh79+7F5XKRm5vLnDlzeOGFF3jiiSc4//zzSUxM5MUXXyQrK4vVq1dz4sQJfvazn3HllVfywQcf4HA4+K//+i8iIyMpLCzk3nvvxWw2c/XVV7Nt2zbWr18f6iaLiEgH/Pa3v2X79u0cOnSIc889l4MHD3LOOeeQm5vLbbfd1qTs/PnzsdlsFBUVccUVV5CXl8ejjz6Ky+UiMjKSxx57jHPOOYecnBzq6+txOBzccsst1NfXs3v3bhYsWMD8+fOJjY1l9+7dHDlyhHnz5jFmzBh8Ph8LFy5kx44d9OnTB6vVypQpUxgzZgyLFi3y33x85MiR3HXXXU3ieuaZZzh48CAHDx6koqKCGTNm8IMf/ACAFStW8Pe//x232012djZz5sxpdo1cvnw5/fr18+/v1LUwOTmZDz/8kCeeeIIXX3yRnTt38uijjwJgMpl46aWXiI2N7VAdIu1lDXUAIj3ZgQMH+O1vf8vQoUOprKwkMTERr9fLTTfdxJ49e7jxxhv585//zMqVK0lOTm5x+8WLF/PII49w++23s2HDBiZMmMA999zDww8/zLe+9S0WLVoUgpaJiEhnueuuuxg7dizr1q1j/vz5/OQnP+Evf/lLq+WdTid/+ctfsFgsHD9+nJdffhmr1crbb7/NkiVLeOaZZ5gzZ44/gQJ4/fXXm+yjrKyM//mf/2H//v3MmjWLMWPGsHHjRr744gveeOMNysvLGTduHFOmTKGiooI333yTf/zjH5hMplaHpe/du5dXXnmFEydOMGnSJDIzM9m3bx8HDhxg9erVGIbBrFmzeO+99+jTp0+Ta2R7/fGPf2TBggVceeWV1NbWYrfb2b59e6fWIfJNSqhEQqhv377+N/G///3vvPLKK3g8Ho4cOcJnn33GJZdcctrtzznnHAYOHAjAoEGD+OKLL6iurqa2tpZvfetbAFx33XX861//Cmg7REQksD7++GMuueQS9u/fzwUXXHDasmPGjMFisQBQU1PDXXfdxYEDBzCZTDQ0NLSrvu9///uYzWYuvPBCjh49CsD777/PmDFjMJvNpKSkMGzYMADi4uKw2+3cc889XHvttXzve99rcZ+jR48mMjKSyMhIhg0bxocffsj777/PW2+9xcSJEwE4ceIExcXF9OnTp8k1sr2uuOIKHn/8ccaPH09OTg4xMTG89dZbnVqHyDcpoRIJoejoaABKSkr44x//yOrVq0lISGD+/Pm4XK42t7fZbP6/LRZLu7YREZGuo6ioiPnz51NaWkpSUhL19fUYhsGECRP461//SmRkZLNtoqKi/H8vXbqUYcOG8dxzz3Ho0CFuvPHGdtX79etLW6xWK6tXr+add97hH//4By+99BIvvPBCs3Imk6nZMsMwmDlzJj/84Q+bLD906JD/GtkSi8XCqV+tfP3aN3PmTDIzM9myZQs33HADK1as6HAdIu2lSSlEwkBtbS1RUVHExcVx9OhRtm7d6l8XExNDbW1tu/cVHx9PTEwM//nPfwB44403Oj1eEREJjoEDB7Ju3TrOP/983njjDYYPH87zzz/PunXrWkymvqmmpgaHwwHAmjVr/MvP9NoCjb0/GzduxOfzcfToUXbu3Ak0XsNqamrIzMzknnvuYe/evS1uX1BQgMvloqKigp07dzJ48GBGjhzJa6+95o/F6XRSXl7eZiz9+vVj9+7dAGzcuNG//ODBg1x88cXMnDmTwYMH8/nnn3e4DpH2Ug+VSBi45JJLuPTSSxk7dixpaWlcccUV/nU/+MEPmDFjBqmpqbz44ovt2t+jjz7Kfffdh9ls5qqrriI2NjZQoYuISIAdO3aM+Ph4zGYz+/fv58ILL2z3tjNmzGD+/Pn8/ve/JzMz07982LBhLF++nAkTJnDLLbe0a1+5ubm88847jBs3jj59+nDppZcSFxdHbW0ts2fP9vcUzZ8/v8XtL774Ym688UYqKiqYPXs2DocDh8PBZ5995u89io6O5sknn8RsPv13/rfddhv33nuvvwfulJUrV/Luu+9iMpnIyMhg1KhR2Gy2DtUh0l6a5U+kG6qtrSUmJgaA5cuXU1ZWxn333RfiqEREpKs7dX2pqKhg2rRprFq1ipSUlDa3e+aZZ4iOjubmm28OQpQiwaUeKpFuaMuWLSxbtgyv10vfvn15/PHHQx2SiIh0Az//+c+prq6moaGB2bNntyuZEunu1EMlIiIiIiLSQRo8KiIiIiIi0kFKqERERERERDpICZWIiIiIiEgHKaESERERERHpICVUIiIiIiIiHaSESkREREREpIP+fztAGy1lpRsAAAAAAElFTkSuQmCC\n", 407 | "text/plain": [ 408 | "
" 409 | ] 410 | }, 411 | "metadata": {}, 412 | "output_type": "display_data" 413 | } 414 | ], 415 | "source": [ 416 | "sns.set_style(\"whitegrid\")\n", 417 | "plt.figure(figsize=(14,5))\n", 418 | "plt.subplot(1,2,1)\n", 419 | "ax = sns.countplot(x=\"rating\", data=ratings, palette=\"viridis\")\n", 420 | "plt.title(\"Distribution of movie ratings\")\n", 421 | "\n", 422 | "plt.subplot(1,2,2)\n", 423 | "ax = sns.kdeplot(user_freq['n_ratings'], shade=True, legend=False)\n", 424 | "plt.axvline(user_freq['n_ratings'].mean(), color=\"k\", linestyle=\"--\")\n", 425 | "plt.xlabel(\"# ratings per user\")\n", 426 | "plt.ylabel(\"density\")\n", 427 | "plt.title(\"Number of movies rated per user\")\n", 428 | "plt.show()" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "The most common rating is 4.0, while lower ratings such as 0.5 or 1.0 are much more rare. " 436 | ] 437 | }, 438 | { 439 | "cell_type": "markdown", 440 | "metadata": {}, 441 | "source": [ 442 | "### Which movie has the lowest and highest average rating?" 443 | ] 444 | }, 445 | { 446 | "cell_type": "code", 447 | "execution_count": 8, 448 | "metadata": {}, 449 | "outputs": [ 450 | { 451 | "data": { 452 | "text/html": [ 453 | "
\n", 454 | "\n", 467 | "\n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | "
movieIdtitlegenres
26893604Gypsy (1962)Musical
\n", 485 | "
" 486 | ], 487 | "text/plain": [ 488 | " movieId title genres\n", 489 | "2689 3604 Gypsy (1962) Musical" 490 | ] 491 | }, 492 | "execution_count": 8, 493 | "metadata": {}, 494 | "output_type": "execute_result" 495 | } 496 | ], 497 | "source": [ 498 | "mean_rating = ratings.groupby('movieId')[['rating']].mean()\n", 499 | "\n", 500 | "lowest_rated = mean_rating['rating'].idxmin()\n", 501 | "movies.loc[movies['movieId'] == lowest_rated]" 502 | ] 503 | }, 504 | { 505 | "cell_type": "markdown", 506 | "metadata": {}, 507 | "source": [ 508 | "Santa with Muscles is the worst rated movie!" 509 | ] 510 | }, 511 | { 512 | "cell_type": "code", 513 | "execution_count": 9, 514 | "metadata": {}, 515 | "outputs": [ 516 | { 517 | "data": { 518 | "text/html": [ 519 | "
\n", 520 | "\n", 533 | "\n", 534 | " \n", 535 | " \n", 536 | " \n", 537 | " \n", 538 | " \n", 539 | " \n", 540 | " \n", 541 | " \n", 542 | " \n", 543 | " \n", 544 | " \n", 545 | " \n", 546 | " \n", 547 | " \n", 548 | " \n", 549 | " \n", 550 | "
movieIdtitlegenres
4853Lamerica (1994)Adventure|Drama
\n", 551 | "
" 552 | ], 553 | "text/plain": [ 554 | " movieId title genres\n", 555 | "48 53 Lamerica (1994) Adventure|Drama" 556 | ] 557 | }, 558 | "execution_count": 9, 559 | "metadata": {}, 560 | "output_type": "execute_result" 561 | } 562 | ], 563 | "source": [ 564 | "highest_rated = mean_rating['rating'].idxmax()\n", 565 | "movies.loc[movies['movieId'] == highest_rated]" 566 | ] 567 | }, 568 | { 569 | "cell_type": "markdown", 570 | "metadata": {}, 571 | "source": [ 572 | "Lamerica may be the \"highest\" rated movie, but how many ratings does it have?" 573 | ] 574 | }, 575 | { 576 | "cell_type": "code", 577 | "execution_count": 10, 578 | "metadata": {}, 579 | "outputs": [ 580 | { 581 | "data": { 582 | "text/html": [ 583 | "
\n", 584 | "\n", 597 | "\n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | " \n", 609 | " \n", 610 | " \n", 611 | " \n", 612 | " \n", 613 | " \n", 614 | " \n", 615 | " \n", 616 | " \n", 617 | " \n", 618 | " \n", 619 | " \n", 620 | " \n", 621 | " \n", 622 | " \n", 623 | "
userIdmovieIdratingtimestamp
1336885535.0889468268
96115603535.0963180003
\n", 624 | "
" 625 | ], 626 | "text/plain": [ 627 | " userId movieId rating timestamp\n", 628 | "13368 85 53 5.0 889468268\n", 629 | "96115 603 53 5.0 963180003" 630 | ] 631 | }, 632 | "execution_count": 10, 633 | "metadata": {}, 634 | "output_type": "execute_result" 635 | } 636 | ], 637 | "source": [ 638 | "ratings[ratings['movieId']==highest_rated]" 639 | ] 640 | }, 641 | { 642 | "cell_type": "markdown", 643 | "metadata": {}, 644 | "source": [ 645 | "Lamerica has only 2 ratings. A better approach for evaluating movie popularity is to look at the [Bayesian average](https://en.wikipedia.org/wiki/Bayesian_average)." 646 | ] 647 | }, 648 | { 649 | "cell_type": "markdown", 650 | "metadata": {}, 651 | "source": [ 652 | "#### Bayesian Average\n", 653 | "\n", 654 | "Bayesian Average is defined as:\n", 655 | "\n", 656 | "$r_{i} = \\frac{C \\times m + \\Sigma{\\text{reviews}}}{C+N}$\n", 657 | "\n", 658 | "where $C$ represents our confidence, $m$ represents our prior, and $N$ is the total number of reviews for movie $i$. In this case, our prior will be the average rating across all movies. By defintion, C represents \"the typical dataset size\". Let's make $C$ be the average number of ratings for a given movie." 659 | ] 660 | }, 661 | { 662 | "cell_type": "code", 663 | "execution_count": 11, 664 | "metadata": {}, 665 | "outputs": [], 666 | "source": [ 667 | "movie_stats = ratings.groupby('movieId')[['rating']].agg(['count', 'mean'])\n", 668 | "movie_stats.columns = movie_stats.columns.droplevel()" 669 | ] 670 | }, 671 | { 672 | "cell_type": "code", 673 | "execution_count": 12, 674 | "metadata": {}, 675 | "outputs": [], 676 | "source": [ 677 | "C = movie_stats['count'].mean()\n", 678 | "m = movie_stats['mean'].mean()\n", 679 | "\n", 680 | "def bayesian_avg(ratings):\n", 681 | " bayesian_avg = (C*m+ratings.sum())/(C+ratings.count())\n", 682 | " return bayesian_avg\n", 683 | "\n", 684 | "bayesian_avg_ratings = ratings.groupby('movieId')['rating'].agg(bayesian_avg).reset_index()\n", 685 | "bayesian_avg_ratings.columns = ['movieId', 'bayesian_avg']\n", 686 | "movie_stats = movie_stats.merge(bayesian_avg_ratings, on='movieId')" 687 | ] 688 | }, 689 | { 690 | "cell_type": "code", 691 | "execution_count": 13, 692 | "metadata": {}, 693 | "outputs": [ 694 | { 695 | "data": { 696 | "text/html": [ 697 | "
\n", 698 | "\n", 711 | "\n", 712 | " \n", 713 | " \n", 714 | " \n", 715 | " \n", 716 | " \n", 717 | " \n", 718 | " \n", 719 | " \n", 720 | " \n", 721 | " \n", 722 | " \n", 723 | " \n", 724 | " \n", 725 | " \n", 726 | " \n", 727 | " \n", 728 | " \n", 729 | " \n", 730 | " \n", 731 | " \n", 732 | " \n", 733 | " \n", 734 | " \n", 735 | " \n", 736 | " \n", 737 | " \n", 738 | " \n", 739 | " \n", 740 | " \n", 741 | " \n", 742 | " \n", 743 | " \n", 744 | " \n", 745 | " \n", 746 | " \n", 747 | " \n", 748 | " \n", 749 | " \n", 750 | " \n", 751 | " \n", 752 | " \n", 753 | " \n", 754 | " \n", 755 | " \n", 756 | " \n", 757 | " \n", 758 | " \n", 759 | " \n", 760 | " \n", 761 | " \n", 762 | " \n", 763 | " \n", 764 | "
movieIdcountmeanbayesian_avgtitle
2773183174.4290224.392070Shawshank Redemption, The (1994)
6598581924.2890624.236457Godfather, The (1972)
222429592184.2729364.227052Fight Club (1999)
2242602514.2310764.192646Star Wars: Episode IV - A New Hope (1977)
46502044.2377454.190567Usual Suspects, The (1995)
\n", 765 | "
" 766 | ], 767 | "text/plain": [ 768 | " movieId count mean bayesian_avg \\\n", 769 | "277 318 317 4.429022 4.392070 \n", 770 | "659 858 192 4.289062 4.236457 \n", 771 | "2224 2959 218 4.272936 4.227052 \n", 772 | "224 260 251 4.231076 4.192646 \n", 773 | "46 50 204 4.237745 4.190567 \n", 774 | "\n", 775 | " title \n", 776 | "277 Shawshank Redemption, The (1994) \n", 777 | "659 Godfather, The (1972) \n", 778 | "2224 Fight Club (1999) \n", 779 | "224 Star Wars: Episode IV - A New Hope (1977) \n", 780 | "46 Usual Suspects, The (1995) " 781 | ] 782 | }, 783 | "execution_count": 13, 784 | "metadata": {}, 785 | "output_type": "execute_result" 786 | } 787 | ], 788 | "source": [ 789 | "movie_stats = movie_stats.merge(movies[['movieId', 'title']])\n", 790 | "movie_stats.sort_values('bayesian_avg', ascending=False).head()" 791 | ] 792 | }, 793 | { 794 | "cell_type": "markdown", 795 | "metadata": {}, 796 | "source": [ 797 | "Using the Bayesian average, we see that `Shawshank Redemption`, `The Godfather`, and `Fight Club` are the most highly rated movies. This result makes much more sense since these movies are critically acclaimed films.\n", 798 | "\n", 799 | "Now which movies are the worst rated, according to the Bayesian average?" 800 | ] 801 | }, 802 | { 803 | "cell_type": "code", 804 | "execution_count": 14, 805 | "metadata": {}, 806 | "outputs": [ 807 | { 808 | "data": { 809 | "text/html": [ 810 | "
\n", 811 | "\n", 824 | "\n", 825 | " \n", 826 | " \n", 827 | " \n", 828 | " \n", 829 | " \n", 830 | " \n", 831 | " \n", 832 | " \n", 833 | " \n", 834 | " \n", 835 | " \n", 836 | " \n", 837 | " \n", 838 | " \n", 839 | " \n", 840 | " \n", 841 | " \n", 842 | " \n", 843 | " \n", 844 | " \n", 845 | " \n", 846 | " \n", 847 | " \n", 848 | " \n", 849 | " \n", 850 | " \n", 851 | " \n", 852 | " \n", 853 | " \n", 854 | " \n", 855 | " \n", 856 | " \n", 857 | " \n", 858 | " \n", 859 | " \n", 860 | " \n", 861 | " \n", 862 | " \n", 863 | " \n", 864 | " \n", 865 | " \n", 866 | " \n", 867 | " \n", 868 | " \n", 869 | " \n", 870 | " \n", 871 | " \n", 872 | " \n", 873 | " \n", 874 | " \n", 875 | " \n", 876 | " \n", 877 | "
movieIdcountmeanbayesian_avgtitle
11721556191.6052632.190377Speed 2: Cruise Control (1997)
26793593191.6578952.224426Battlefield Earth (2000)
13721882331.9545452.267268Godzilla (1998)
11441499271.9259262.296800Anaconda (1997)
19882643161.6875002.306841Superman IV: The Quest for Peace (1987)
\n", 878 | "
" 879 | ], 880 | "text/plain": [ 881 | " movieId count mean bayesian_avg \\\n", 882 | "1172 1556 19 1.605263 2.190377 \n", 883 | "2679 3593 19 1.657895 2.224426 \n", 884 | "1372 1882 33 1.954545 2.267268 \n", 885 | "1144 1499 27 1.925926 2.296800 \n", 886 | "1988 2643 16 1.687500 2.306841 \n", 887 | "\n", 888 | " title \n", 889 | "1172 Speed 2: Cruise Control (1997) \n", 890 | "2679 Battlefield Earth (2000) \n", 891 | "1372 Godzilla (1998) \n", 892 | "1144 Anaconda (1997) \n", 893 | "1988 Superman IV: The Quest for Peace (1987) " 894 | ] 895 | }, 896 | "execution_count": 14, 897 | "metadata": {}, 898 | "output_type": "execute_result" 899 | } 900 | ], 901 | "source": [ 902 | "movie_stats.sort_values('bayesian_avg', ascending=True).head()" 903 | ] 904 | }, 905 | { 906 | "cell_type": "markdown", 907 | "metadata": {}, 908 | "source": [ 909 | "With Bayesian averaging, it looks like `Speed 2: Cruise Control`, `Battlefield Earth`, and `Godzilla` are the worst rated movies. `Gypsy` isn't so bad after all!" 910 | ] 911 | }, 912 | { 913 | "cell_type": "markdown", 914 | "metadata": {}, 915 | "source": [ 916 | "## Step 4: Transforming the data\n", 917 | "\n", 918 | "We will be using a technique called [collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering) to generate user recommendations. This technique is based on the assumption of \"homophily\" - similar users like similar things. Collaborative filtering is a type of unsupervised learning that makes predictions about the interests of a user by learning from the interests of a larger population.\n", 919 | "\n", 920 | "The first step of collaborative filtering is to transform our data into a `user-item matrix` - also known as a \"utility\" matrix. In this matrix, rows represent users and columns represent items. The beauty of collaborative filtering is that it doesn't require any information about the users or items to generate recommendations. \n", 921 | "\n", 922 | "\n", 923 | "" 924 | ] 925 | }, 926 | { 927 | "cell_type": "markdown", 928 | "metadata": {}, 929 | "source": [ 930 | "The `create_X()` function outputs a sparse matrix X with four mapper dictionaries:\n", 931 | "- **user_mapper:** maps user id to user index\n", 932 | "- **movie_mapper:** maps movie id to movie index\n", 933 | "- **user_inv_mapper:** maps user index to user id\n", 934 | "- **movie_inv_mapper:** maps movie index to movie id\n", 935 | "\n", 936 | "We need these dictionaries because they map which row and column of the utility matrix corresponds to which user ID and movie ID, respectively.\n", 937 | "\n", 938 | "The **X** (user-item) matrix is a [scipy.sparse.csr_matrix](scipylinkhere) which stores the data sparsely." 939 | ] 940 | }, 941 | { 942 | "cell_type": "code", 943 | "execution_count": 15, 944 | "metadata": {}, 945 | "outputs": [], 946 | "source": [ 947 | "from scipy.sparse import csr_matrix\n", 948 | "\n", 949 | "def create_X(df):\n", 950 | " \"\"\"\n", 951 | " Generates a sparse matrix from ratings dataframe.\n", 952 | " \n", 953 | " Args:\n", 954 | " df: pandas dataframe\n", 955 | " \n", 956 | " Returns:\n", 957 | " X: sparse matrix\n", 958 | " user_mapper: dict that maps user id's to user indices\n", 959 | " user_inv_mapper: dict that maps user indices to user id's\n", 960 | " movie_mapper: dict that maps movie id's to movie indices\n", 961 | " movie_inv_mapper: dict that maps movie indices to movie id's\n", 962 | " \"\"\"\n", 963 | " N = df['userId'].nunique()\n", 964 | " M = df['movieId'].nunique()\n", 965 | "\n", 966 | " user_mapper = dict(zip(np.unique(df[\"userId\"]), list(range(N))))\n", 967 | " movie_mapper = dict(zip(np.unique(df[\"movieId\"]), list(range(M))))\n", 968 | " \n", 969 | " user_inv_mapper = dict(zip(list(range(N)), np.unique(df[\"userId\"])))\n", 970 | " movie_inv_mapper = dict(zip(list(range(M)), np.unique(df[\"movieId\"])))\n", 971 | " \n", 972 | " user_index = [user_mapper[i] for i in df['userId']]\n", 973 | " movie_index = [movie_mapper[i] for i in df['movieId']]\n", 974 | "\n", 975 | " X = csr_matrix((df[\"rating\"], (movie_index, user_index)), shape=(M, N))\n", 976 | " \n", 977 | " return X, user_mapper, movie_mapper, user_inv_mapper, movie_inv_mapper" 978 | ] 979 | }, 980 | { 981 | "cell_type": "code", 982 | "execution_count": 16, 983 | "metadata": {}, 984 | "outputs": [], 985 | "source": [ 986 | "X, user_mapper, movie_mapper, user_inv_mapper, movie_inv_mapper = create_X(ratings)" 987 | ] 988 | }, 989 | { 990 | "cell_type": "markdown", 991 | "metadata": {}, 992 | "source": [ 993 | "Let's check out the sparsity of our X matrix.\n", 994 | "\n", 995 | "Here, we calculate sparsity by dividing the number of non-zero elements by total number of elements as described in the equation below: \n", 996 | "\n", 997 | "$$S=\\frac{\\text{# non-zero elements}}{\\text{total elements}}$$" 998 | ] 999 | }, 1000 | { 1001 | "cell_type": "code", 1002 | "execution_count": 17, 1003 | "metadata": {}, 1004 | "outputs": [ 1005 | { 1006 | "name": "stdout", 1007 | "output_type": "stream", 1008 | "text": [ 1009 | "Matrix sparsity: 1.7%\n" 1010 | ] 1011 | } 1012 | ], 1013 | "source": [ 1014 | "sparsity = X.count_nonzero()/(X.shape[0]*X.shape[1])\n", 1015 | "\n", 1016 | "print(f\"Matrix sparsity: {round(sparsity*100,2)}%\")" 1017 | ] 1018 | }, 1019 | { 1020 | "cell_type": "markdown", 1021 | "metadata": {}, 1022 | "source": [ 1023 | "Only 1.7% of cells in our user-item matrix are populated with ratings. But don't be discouraged by this sparsity! User-item matrices are typically very sparse. A general rule of thumb is that your matrix sparsity should be no lower than 0.5% to generate decent results." 1024 | ] 1025 | }, 1026 | { 1027 | "cell_type": "markdown", 1028 | "metadata": {}, 1029 | "source": [ 1030 | "### Writing your matrix to a file\n", 1031 | "\n", 1032 | "We're going to save our user-item matrix for the next part of this tutorial series. Since our matrix is represented as a scipy sparse matrix, we can use the [scipy.sparse.save_npz](https://docs.scipy.org/doc/scipy-1.1.0/reference/generated/scipy.sparse.load_npz.html) method to write the matrix to a file. " 1033 | ] 1034 | }, 1035 | { 1036 | "cell_type": "code", 1037 | "execution_count": 18, 1038 | "metadata": {}, 1039 | "outputs": [], 1040 | "source": [ 1041 | "from scipy.sparse import save_npz\n", 1042 | "\n", 1043 | "save_npz('data/user_item_matrix.npz', X)" 1044 | ] 1045 | }, 1046 | { 1047 | "cell_type": "markdown", 1048 | "metadata": {}, 1049 | "source": [ 1050 | "\n", 1051 | "\n", 1052 | "## Step 5: Finding similar movies using k-Nearest Neighbours\n", 1053 | "\n", 1054 | "This approach looks for the $k$ nearest neighbours of a given movie by identifying $k$ points in the dataset that are closest to movie $m$. kNN makes use of distance metrics such as:\n", 1055 | "\n", 1056 | "1. Cosine similarity\n", 1057 | "2. Euclidean distance\n", 1058 | "3. Manhattan distance\n", 1059 | "4. Pearson correlation \n", 1060 | "\n", 1061 | "Although difficult to visualize, we are working in a M-dimensional space where M represents the number of movies in our X matrix. " 1062 | ] 1063 | }, 1064 | { 1065 | "cell_type": "code", 1066 | "execution_count": 19, 1067 | "metadata": {}, 1068 | "outputs": [], 1069 | "source": [ 1070 | "from sklearn.neighbors import NearestNeighbors\n", 1071 | "\n", 1072 | "def find_similar_movies(movie_id, X, k, metric='cosine', show_distance=False):\n", 1073 | " \"\"\"\n", 1074 | " Finds k-nearest neighbours for a given movie id.\n", 1075 | " \n", 1076 | " Args:\n", 1077 | " movie_id: id of the movie of interest\n", 1078 | " X: user-item utility matrix\n", 1079 | " k: number of similar movies to retrieve\n", 1080 | " metric: distance metric for kNN calculations\n", 1081 | " \n", 1082 | " Returns:\n", 1083 | " list of k similar movie ID's\n", 1084 | " \"\"\"\n", 1085 | " neighbour_ids = []\n", 1086 | " \n", 1087 | " movie_ind = movie_mapper[movie_id]\n", 1088 | " movie_vec = X[movie_ind]\n", 1089 | " k+=1\n", 1090 | " kNN = NearestNeighbors(n_neighbors=k, algorithm=\"brute\", metric=metric)\n", 1091 | " kNN.fit(X)\n", 1092 | " if isinstance(movie_vec, (np.ndarray)):\n", 1093 | " movie_vec = movie_vec.reshape(1,-1)\n", 1094 | " neighbour = kNN.kneighbors(movie_vec, return_distance=show_distance)\n", 1095 | " for i in range(0,k):\n", 1096 | " n = neighbour.item(i)\n", 1097 | " neighbour_ids.append(movie_inv_mapper[n])\n", 1098 | " neighbour_ids.pop(0)\n", 1099 | " return neighbour_ids" 1100 | ] 1101 | }, 1102 | { 1103 | "cell_type": "markdown", 1104 | "metadata": {}, 1105 | "source": [ 1106 | "`find_similar_movies()` takes in a movieId and user-item X matrix, and outputs a list of $k$ movies that are similar to the movieId of interest. \n", 1107 | "\n", 1108 | "Let's see how it works in action. We will first create another mapper that maps `movieId` to `title` so that our results are interpretable. " 1109 | ] 1110 | }, 1111 | { 1112 | "cell_type": "code", 1113 | "execution_count": 20, 1114 | "metadata": {}, 1115 | "outputs": [ 1116 | { 1117 | "name": "stdout", 1118 | "output_type": "stream", 1119 | "text": [ 1120 | "Because you watched Toy Story (1995)\n", 1121 | "Toy Story 2 (1999)\n", 1122 | "Jurassic Park (1993)\n", 1123 | "Independence Day (a.k.a. ID4) (1996)\n", 1124 | "Star Wars: Episode IV - A New Hope (1977)\n", 1125 | "Forrest Gump (1994)\n", 1126 | "Lion King, The (1994)\n", 1127 | "Star Wars: Episode VI - Return of the Jedi (1983)\n", 1128 | "Mission: Impossible (1996)\n", 1129 | "Groundhog Day (1993)\n", 1130 | "Back to the Future (1985)\n" 1131 | ] 1132 | } 1133 | ], 1134 | "source": [ 1135 | "movie_titles = dict(zip(movies['movieId'], movies['title']))\n", 1136 | "\n", 1137 | "movie_id = 1\n", 1138 | "\n", 1139 | "similar_ids = find_similar_movies(movie_id, X, k=10)\n", 1140 | "movie_title = movie_titles[movie_id]\n", 1141 | "\n", 1142 | "print(f\"Because you watched {movie_title}\")\n", 1143 | "for i in similar_ids:\n", 1144 | " print(movie_titles[i])" 1145 | ] 1146 | }, 1147 | { 1148 | "cell_type": "markdown", 1149 | "metadata": {}, 1150 | "source": [ 1151 | "The results above show the 10 most similar movies to Toy Story. Most movies in this list are family movies from the 1990s, which seems pretty reasonable. Note that these recommendations are based solely on user-item ratings. Movie features such as genres are not taken into consideration in this approach. \n", 1152 | "\n", 1153 | "You can also play around with the kNN distance metric and see what results you would get if you use \"manhattan\" or \"euclidean\" instead of \"cosine\"." 1154 | ] 1155 | }, 1156 | { 1157 | "cell_type": "code", 1158 | "execution_count": 21, 1159 | "metadata": {}, 1160 | "outputs": [ 1161 | { 1162 | "name": "stdout", 1163 | "output_type": "stream", 1164 | "text": [ 1165 | "Because you watched Toy Story (1995):\n", 1166 | "Toy Story 2 (1999)\n", 1167 | "Mission: Impossible (1996)\n", 1168 | "Independence Day (a.k.a. ID4) (1996)\n", 1169 | "Bug's Life, A (1998)\n", 1170 | "Nutty Professor, The (1996)\n", 1171 | "Willy Wonka & the Chocolate Factory (1971)\n", 1172 | "Babe (1995)\n", 1173 | "Groundhog Day (1993)\n", 1174 | "Mask, The (1994)\n", 1175 | "Honey, I Shrunk the Kids (1989)\n" 1176 | ] 1177 | } 1178 | ], 1179 | "source": [ 1180 | "movie_titles = dict(zip(movies['movieId'], movies['title']))\n", 1181 | "\n", 1182 | "movie_id = 1\n", 1183 | "similar_ids = find_similar_movies(movie_id, X, k=10, metric=\"euclidean\")\n", 1184 | "\n", 1185 | "movie_title = movie_titles[movie_id]\n", 1186 | "print(f\"Because you watched {movie_title}:\")\n", 1187 | "for i in similar_ids:\n", 1188 | " print(movie_titles[i])" 1189 | ] 1190 | } 1191 | ], 1192 | "metadata": { 1193 | "kernelspec": { 1194 | "display_name": "Python 3", 1195 | "language": "python", 1196 | "name": "python3" 1197 | }, 1198 | "language_info": { 1199 | "codemirror_mode": { 1200 | "name": "ipython", 1201 | "version": 3 1202 | }, 1203 | "file_extension": ".py", 1204 | "mimetype": "text/x-python", 1205 | "name": "python", 1206 | "nbconvert_exporter": "python", 1207 | "pygments_lexer": "ipython3", 1208 | "version": "3.7.6" 1209 | } 1210 | }, 1211 | "nbformat": 4, 1212 | "nbformat_minor": 2 1213 | } 1214 | -------------------------------------------------------------------------------- /part-3-implicit-feedback-recommender.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Part 3: Building a Recommender System with Implicit Feedback\n", 8 | "\n", 9 | "In this tutorial, we will build an implicit feedback recommender system using the [implicit](https://github.com/benfred/implicit) package.\n", 10 | "\n", 11 | "What is implicit feedback, exactly? Let's revisit collaborative filtering. In [Part 1](https://github.com/topspinj/recommender-tutorial/blob/master/part-1-item-item-recommender.ipynb), we learned that [collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering) is based on the assumption that `similar users like similar things`. The user-item matrix, or \"utility matrix\", is the foundation of collaborative filtering. In the utility matrix, rows represent users and columns represent items. \n", 12 | "\n", 13 | "\n", 14 | "\n", 15 | "The cells of the matrix are populated by a given user's degree of preference towards an item, which can come in the form of:\n", 16 | "\n", 17 | "1. **explicit feedback:** direct feedback towards an item (e.g., movie ratings which we explored in [Part 1](https://github.com/topspinj/recommender-tutorial/blob/master/part-1-item-item-recommender.ipynb))\n", 18 | "2. **implicit feedback:** indirect behaviour towards an item (e.g., purchase history, browsing history, search behaviour)\n", 19 | "\n", 20 | "Implicit feedback makes assumptions about a user's preference based on their actions towards items. Let's take Netflix for example. If you binge-watch a show and blaze through all seasons in a week, there's a high chance that you like that show. However, if you start watching a series and stop halfway through the first episode, there's suspicion to believe that you probably don't like that show. \n", 21 | "\n", 22 | "" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "### Step 1: Import Dependencies\n", 30 | "\n", 31 | "We'll be using the following packages to build our implicit feedback recommender system:\n", 32 | "\n", 33 | "- [numpy](https://numpy.org/)\n", 34 | "- [pandas](https://pandas.pydata.org/)\n", 35 | "- [implicit](https://github.com/benfred/implicit)\n", 36 | "- scipy (specifically, the [csr_matrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html) class)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": 1, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "import numpy as np\n", 46 | "import pandas as pd\n", 47 | "from scipy.sparse import csr_matrix\n", 48 | "\n", 49 | "import implicit\n", 50 | "\n", 51 | "import warnings\n", 52 | "warnings.simplefilter(action='ignore', category=FutureWarning)" 53 | ] 54 | }, 55 | { 56 | "cell_type": "markdown", 57 | "metadata": {}, 58 | "source": [ 59 | "### Step 2: Load the Data\n", 60 | "\n", 61 | "Since we're already familiar with MovieLens from Part 1 and 2 of this tutorial series, we'll continue using this dataset. You can access the MovieLens dataset that we'll be working with via this zip file url [here](https://grouplens.org/datasets/movielens/), or directly download [here](http://files.grouplens.org/datasets/movielens/ml-latest-small.zip). We're working with data in `ml-latest-small.zip` and will need to add the following files to our local directory: \n", 62 | "- ratings.csv\n", 63 | "- movies.csv\n", 64 | "\n", 65 | "Alternatively, you can access the data here: \n", 66 | "- https://s3-us-west-2.amazonaws.com/recommender-tutorial/movies.csv\n", 67 | "- https://s3-us-west-2.amazonaws.com/recommender-tutorial/ratings.csv" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": 2, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "ratings = pd.read_csv(\"https://s3-us-west-2.amazonaws.com/recommender-tutorial/ratings.csv\")\n", 77 | "movies = pd.read_csv(\"https://s3-us-west-2.amazonaws.com/recommender-tutorial/movies.csv\")" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 3, 83 | "metadata": {}, 84 | "outputs": [ 85 | { 86 | "data": { 87 | "text/html": [ 88 | "
\n", 89 | "\n", 102 | "\n", 103 | " \n", 104 | " \n", 105 | " \n", 106 | " \n", 107 | " \n", 108 | " \n", 109 | " \n", 110 | " \n", 111 | " \n", 112 | " \n", 113 | " \n", 114 | " \n", 115 | " \n", 116 | " \n", 117 | " \n", 118 | " \n", 119 | " \n", 120 | " \n", 121 | " \n", 122 | " \n", 123 | " \n", 124 | " \n", 125 | " \n", 126 | " \n", 127 | " \n", 128 | " \n", 129 | " \n", 130 | " \n", 131 | " \n", 132 | " \n", 133 | " \n", 134 | " \n", 135 | " \n", 136 | " \n", 137 | " \n", 138 | " \n", 139 | " \n", 140 | " \n", 141 | " \n", 142 | " \n", 143 | " \n", 144 | " \n", 145 | " \n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | "
userIdmovieIdratingtimestamp
0114.0964982703
1134.0964981247
2164.0964982224
31475.0964983815
41505.0964982931
\n", 150 | "
" 151 | ], 152 | "text/plain": [ 153 | " userId movieId rating timestamp\n", 154 | "0 1 1 4.0 964982703\n", 155 | "1 1 3 4.0 964981247\n", 156 | "2 1 6 4.0 964982224\n", 157 | "3 1 47 5.0 964983815\n", 158 | "4 1 50 5.0 964982931" 159 | ] 160 | }, 161 | "execution_count": 3, 162 | "metadata": {}, 163 | "output_type": "execute_result" 164 | } 165 | ], 166 | "source": [ 167 | "ratings.head()" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": {}, 173 | "source": [ 174 | "For this implicit feedback tutorial, we'll treat movie ratings as the number of times that a user watched a movie. For example, if Jane (a user in our database) gave `Batman` a rating of 1 and `Legally Blonde` a rating of 5, we'll assume that Jane watched Batman one time and Legally Blonde five times. " 175 | ] 176 | }, 177 | { 178 | "cell_type": "markdown", 179 | "metadata": {}, 180 | "source": [ 181 | "### Step 3: Transforming the Data\n", 182 | "\n", 183 | "Similar to [Part 1](https://github.com/topspinj/recommender-tutorial/blob/master/part-1-item-item-recommender.ipynb), we need to transform the `ratings` dataframe into a user-item matrix where rows represent users and columns represent movies. The cells of this matrix will be populated with implicit feedback: in this case, the number of times a user watched a movie. \n", 184 | "\n", 185 | "The `create_X()` function outputs a sparse matrix **X** with four mapper dictionaries:\n", 186 | "- **user_mapper:** maps user id to user index\n", 187 | "- **movie_mapper:** maps movie id to movie index\n", 188 | "- **user_inv_mapper:** maps user index to user id\n", 189 | "- **movie_inv_mapper:** maps movie index to movie id\n", 190 | "\n", 191 | "We need these dictionaries because they map which row and column of the utility matrix corresponds to which user ID and movie ID, respectively.\n", 192 | "\n", 193 | "The **X** (user-item) matrix is a [scipy.sparse.csr_matrix](scipylinkhere) which stores the data sparsely.\n", 194 | "\n", 195 | "\n", 196 | "\n", 197 | "" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": 4, 203 | "metadata": {}, 204 | "outputs": [], 205 | "source": [ 206 | "def create_X(df):\n", 207 | " \"\"\"\n", 208 | " Generates a sparse matrix from ratings dataframe.\n", 209 | " \n", 210 | " Args:\n", 211 | " df: pandas dataframe\n", 212 | " \n", 213 | " Returns:\n", 214 | " X: sparse matrix\n", 215 | " user_mapper: dict that maps user id's to user indices\n", 216 | " user_inv_mapper: dict that maps user indices to user id's\n", 217 | " movie_mapper: dict that maps movie id's to movie indices\n", 218 | " movie_inv_mapper: dict that maps movie indices to movie id's\n", 219 | " \"\"\"\n", 220 | " N = df['userId'].nunique()\n", 221 | " M = df['movieId'].nunique()\n", 222 | "\n", 223 | " user_mapper = dict(zip(np.unique(df[\"userId\"]), list(range(N))))\n", 224 | " movie_mapper = dict(zip(np.unique(df[\"movieId\"]), list(range(M))))\n", 225 | " \n", 226 | " user_inv_mapper = dict(zip(list(range(N)), np.unique(df[\"userId\"])))\n", 227 | " movie_inv_mapper = dict(zip(list(range(M)), np.unique(df[\"movieId\"])))\n", 228 | " \n", 229 | " user_index = [user_mapper[i] for i in df['userId']]\n", 230 | " movie_index = [movie_mapper[i] for i in df['movieId']]\n", 231 | "\n", 232 | " X = csr_matrix((df[\"rating\"], (movie_index, user_index)), shape=(M, N))\n", 233 | " \n", 234 | " return X, user_mapper, movie_mapper, user_inv_mapper, movie_inv_mapper" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": 5, 240 | "metadata": {}, 241 | "outputs": [], 242 | "source": [ 243 | "X, user_mapper, movie_mapper, user_inv_mapper, movie_inv_mapper = create_X(ratings)" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "### Creating Movie Title Mappers\n", 251 | "\n", 252 | "We need to interpret a movie title from its index in the user-item matrix and vice versa. Let's create 2 helper functions that make this interpretation easy:\n", 253 | "\n", 254 | "- `get_movie_index()` - converts a movie title to movie index\n", 255 | " - Note that this function uses [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy)'s string matching to get the approximate movie title match based on the string that gets passed in. This means that you don't need to know the exact spelling and formatting of the title to get the corresponding movie index.\n", 256 | "- `get_movie_title()` - converts a movie index to movie title" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 6, 262 | "metadata": {}, 263 | "outputs": [], 264 | "source": [ 265 | "from fuzzywuzzy import process\n", 266 | "\n", 267 | "def movie_finder(title):\n", 268 | " all_titles = movies['title'].tolist()\n", 269 | " closest_match = process.extractOne(title,all_titles)\n", 270 | " return closest_match[0]\n", 271 | "\n", 272 | "movie_title_mapper = dict(zip(movies['title'], movies['movieId']))\n", 273 | "movie_title_inv_mapper = dict(zip(movies['movieId'], movies['title']))\n", 274 | "\n", 275 | "def get_movie_index(title):\n", 276 | " fuzzy_title = movie_finder(title)\n", 277 | " movie_id = movie_title_mapper[fuzzy_title]\n", 278 | " movie_idx = movie_mapper[movie_id]\n", 279 | " return movie_idx\n", 280 | "\n", 281 | "def get_movie_title(movie_idx): \n", 282 | " movie_id = movie_inv_mapper[movie_idx]\n", 283 | " title = movie_title_inv_mapper[movie_id]\n", 284 | " return title " 285 | ] 286 | }, 287 | { 288 | "cell_type": "markdown", 289 | "metadata": {}, 290 | "source": [ 291 | "It's time to test it out! Let's get the movie index of `Legally Blonde`. " 292 | ] 293 | }, 294 | { 295 | "cell_type": "code", 296 | "execution_count": 7, 297 | "metadata": {}, 298 | "outputs": [ 299 | { 300 | "data": { 301 | "text/plain": [ 302 | "3282" 303 | ] 304 | }, 305 | "execution_count": 7, 306 | "metadata": {}, 307 | "output_type": "execute_result" 308 | } 309 | ], 310 | "source": [ 311 | "get_movie_index('Legally Blonde')" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "Let's pass this index value into `get_movie_title()`. We're expecting Legally Blonde to get returned." 319 | ] 320 | }, 321 | { 322 | "cell_type": "code", 323 | "execution_count": 8, 324 | "metadata": {}, 325 | "outputs": [ 326 | { 327 | "data": { 328 | "text/plain": [ 329 | "'Legally Blonde (2001)'" 330 | ] 331 | }, 332 | "execution_count": 8, 333 | "metadata": {}, 334 | "output_type": "execute_result" 335 | } 336 | ], 337 | "source": [ 338 | "get_movie_title(3282)" 339 | ] 340 | }, 341 | { 342 | "cell_type": "markdown", 343 | "metadata": {}, 344 | "source": [ 345 | "Great! These helper functions will be useful when we want to interpret our recommender results." 346 | ] 347 | }, 348 | { 349 | "cell_type": "markdown", 350 | "metadata": {}, 351 | "source": [ 352 | "### Step 4: Building Our Implicit Feedback Recommender Model\n", 353 | "\n", 354 | "\n", 355 | "We've transformed and prepared our data so that we can start creating our recommender model.\n", 356 | "\n", 357 | "The [implicit](https://github.com/benfred/implicit) package is built around a linear algebra technique called [matrix factorization](https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems)), which can help us discover latent features underlying the interactions between users and movies. These latent features give a more compact representation of user tastes and item descriptions. Matrix factorization is particularly useful for very sparse data and can enhance the quality of recommendations. The algorithm works by factorizing the original user-item matrix into two factor matrices:\n", 358 | "\n", 359 | "- user-factor matrix (n_users, k)\n", 360 | "- item-factor matrix (k, n_items)\n", 361 | "\n", 362 | "We are reducing the dimensions of our original matrix into \"taste\" dimensions. We cannot interpret what each latent feature $k$ represents. However, we could imagine that one latent feature may represent users who like romantic comedies from the 1990s, while another latent feature may represent movies which are independent foreign language films.\n", 363 | "\n", 364 | "$$X_{mn} \\approx P_{mk} \\times Q_{nk}^T = \\hat{X}$$\n", 365 | "\n", 366 | "\n", 367 | "\n", 368 | "In traditional matrix factorization, such as SVD, we would attempt to solve the factorization at once which can be very computationally expensive. As a more practical alternative, we can use a technique called `Alternating Least Squares (ALS)` instead. With ALS, we solve for one factor matrix at a time:\n", 369 | "\n", 370 | "- Step 1: hold user-factor matrix fixed and solve for the item-factor matrix\n", 371 | "- Step 2: hold item-factor matrix fixed and solve for the user-item matrix\n", 372 | "\n", 373 | "We alternate between Step 1 and 2 above, until the dot product of the item-factor matrix and user-item matrix is approximately equal to the original X (user-item) matrix. This approach is less computationally expensive and can be run in parallel.\n", 374 | "\n", 375 | "The [implicit](https://github.com/benfred/implicit) package implements matrix factorization using Alternating Least Squares (see docs [here](https://implicit.readthedocs.io/en/latest/als.html)). Let's initiate the model using the `AlternatingLeastSquares` class." 376 | ] 377 | }, 378 | { 379 | "cell_type": "code", 380 | "execution_count": 9, 381 | "metadata": {}, 382 | "outputs": [ 383 | { 384 | "name": "stderr", 385 | "output_type": "stream", 386 | "text": [ 387 | "WARNING:root:OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading\n" 388 | ] 389 | } 390 | ], 391 | "source": [ 392 | "model = implicit.als.AlternatingLeastSquares(factors=50)" 393 | ] 394 | }, 395 | { 396 | "cell_type": "markdown", 397 | "metadata": {}, 398 | "source": [ 399 | "This model comes with a couple of hyperparameters that can be tuned to generate optimal results:\n", 400 | "\n", 401 | "- factors ($k$): number of latent factors,\n", 402 | "- regularization ($\\lambda$): prevents the model from overfitting during training\n", 403 | "\n", 404 | "In this tutorial, we'll set $k = 50$ and $\\lambda = 0.01$ (the default). In a real-world scenario, I highly recommend tuning these hyperparameters before generating recommendations to generate optimal results.\n", 405 | "\n", 406 | "The next step is to fit our model with our user-item matrix. " 407 | ] 408 | }, 409 | { 410 | "cell_type": "code", 411 | "execution_count": 10, 412 | "metadata": {}, 413 | "outputs": [ 414 | { 415 | "name": "stderr", 416 | "output_type": "stream", 417 | "text": [ 418 | "100%|██████████| 15.0/15 [00:06<00:00, 2.30it/s]\n" 419 | ] 420 | } 421 | ], 422 | "source": [ 423 | "model.fit(X)" 424 | ] 425 | }, 426 | { 427 | "cell_type": "markdown", 428 | "metadata": {}, 429 | "source": [ 430 | "Now, let's test out the model's recommendations. We can use the model's `similar_items()` method which returns the most relevant movies of a given movie. We can use our helpful `get_movie_index()` function to get the movie index of the movie that we're interested in." 431 | ] 432 | }, 433 | { 434 | "cell_type": "code", 435 | "execution_count": 11, 436 | "metadata": {}, 437 | "outputs": [ 438 | { 439 | "data": { 440 | "text/plain": [ 441 | "[(314, 0.9999999),\n", 442 | " (277, 0.87162775),\n", 443 | " (510, 0.8275397),\n", 444 | " (257, 0.82705915),\n", 445 | " (97, 0.7511661),\n", 446 | " (461, 0.7223397),\n", 447 | " (418, 0.6889433),\n", 448 | " (1938, 0.6659612),\n", 449 | " (123, 0.6441393),\n", 450 | " (43, 0.61916924)]" 451 | ] 452 | }, 453 | "execution_count": 11, 454 | "metadata": {}, 455 | "output_type": "execute_result" 456 | } 457 | ], 458 | "source": [ 459 | "movie_of_interest = 'forrest gump'\n", 460 | "\n", 461 | "movie_index = get_movie_index(movie_of_interest)\n", 462 | "related = model.similar_items(movie_index)\n", 463 | "related" 464 | ] 465 | }, 466 | { 467 | "cell_type": "markdown", 468 | "metadata": {}, 469 | "source": [ 470 | "The output of `similar_items()` is not user-friendly. We'll need to use our `get_movie_title()` function to interpret what our results are. " 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 12, 476 | "metadata": {}, 477 | "outputs": [ 478 | { 479 | "name": "stdout", 480 | "output_type": "stream", 481 | "text": [ 482 | "Because you watched Forrest Gump (1994)...\n", 483 | "Shawshank Redemption, The (1994)\n", 484 | "Silence of the Lambs, The (1991)\n", 485 | "Pulp Fiction (1994)\n", 486 | "Braveheart (1995)\n", 487 | "Schindler's List (1993)\n", 488 | "Jurassic Park (1993)\n", 489 | "Matrix, The (1999)\n", 490 | "Apollo 13 (1995)\n", 491 | "Seven (a.k.a. Se7en) (1995)\n" 492 | ] 493 | } 494 | ], 495 | "source": [ 496 | "print(f\"Because you watched {movie_finder(movie_of_interest)}...\")\n", 497 | "for r in related:\n", 498 | " recommended_title = get_movie_title(r[0])\n", 499 | " if recommended_title != movie_finder(movie_of_interest):\n", 500 | " print(recommended_title)" 501 | ] 502 | }, 503 | { 504 | "cell_type": "markdown", 505 | "metadata": {}, 506 | "source": [ 507 | "When we treat user ratings as implicit feedback, the results look pretty good! You can test out other movies by changing the `movie_of_interest` variable." 508 | ] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": {}, 513 | "source": [ 514 | "### Step 5: Generating User-Item Recommendations\n", 515 | "\n", 516 | "A cool feature of [implicit](https://github.com/benfred/implicit) is that you can pull personalized recommendations for a given user. Let's test it out on a user in our dataset." 517 | ] 518 | }, 519 | { 520 | "cell_type": "code", 521 | "execution_count": 13, 522 | "metadata": {}, 523 | "outputs": [], 524 | "source": [ 525 | "user_id = 95" 526 | ] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "execution_count": 14, 531 | "metadata": {}, 532 | "outputs": [ 533 | { 534 | "name": "stdout", 535 | "output_type": "stream", 536 | "text": [ 537 | "Number of movies rated by user 95: 168\n" 538 | ] 539 | } 540 | ], 541 | "source": [ 542 | "user_ratings = ratings[ratings['userId']==user_id].merge(movies[['movieId', 'title']])\n", 543 | "user_ratings = user_ratings.sort_values('rating', ascending=False)\n", 544 | "print(f\"Number of movies rated by user {user_id}: {user_ratings['movieId'].nunique()}\")" 545 | ] 546 | }, 547 | { 548 | "cell_type": "markdown", 549 | "metadata": {}, 550 | "source": [ 551 | "User 95 watched 168 movies. Their highest rated movies are below:" 552 | ] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "execution_count": 15, 557 | "metadata": {}, 558 | "outputs": [ 559 | { 560 | "data": { 561 | "text/html": [ 562 | "
\n", 563 | "\n", 576 | "\n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | " \n", 601 | " \n", 602 | " \n", 603 | " \n", 604 | " \n", 605 | " \n", 606 | " \n", 607 | " \n", 608 | " \n", 609 | " \n", 610 | " \n", 611 | " \n", 612 | " \n", 613 | " \n", 614 | " \n", 615 | " \n", 616 | " \n", 617 | " \n", 618 | " \n", 619 | " \n", 620 | " \n", 621 | " \n", 622 | " \n", 623 | " \n", 624 | " \n", 625 | " \n", 626 | " \n", 627 | " \n", 628 | " \n", 629 | "
userIdmovieIdratingtimestamptitle
249510895.01048382826Reservoir Dogs (1992)
349512215.01043340018Godfather: Part II, The (1974)
839530195.01043340112Drugstore Cowboy (1989)
269511755.01105400882Delicatessen (1991)
279511965.01043340018Star Wars: Episode V - The Empire Strikes Back...
\n", 630 | "
" 631 | ], 632 | "text/plain": [ 633 | " userId movieId rating timestamp \\\n", 634 | "24 95 1089 5.0 1048382826 \n", 635 | "34 95 1221 5.0 1043340018 \n", 636 | "83 95 3019 5.0 1043340112 \n", 637 | "26 95 1175 5.0 1105400882 \n", 638 | "27 95 1196 5.0 1043340018 \n", 639 | "\n", 640 | " title \n", 641 | "24 Reservoir Dogs (1992) \n", 642 | "34 Godfather: Part II, The (1974) \n", 643 | "83 Drugstore Cowboy (1989) \n", 644 | "26 Delicatessen (1991) \n", 645 | "27 Star Wars: Episode V - The Empire Strikes Back... " 646 | ] 647 | }, 648 | "execution_count": 15, 649 | "metadata": {}, 650 | "output_type": "execute_result" 651 | } 652 | ], 653 | "source": [ 654 | "user_ratings = ratings[ratings['userId']==user_id].merge(movies[['movieId', 'title']])\n", 655 | "user_ratings = user_ratings.sort_values('rating', ascending=False)\n", 656 | "top_5 = user_ratings.head()\n", 657 | "top_5" 658 | ] 659 | }, 660 | { 661 | "cell_type": "markdown", 662 | "metadata": {}, 663 | "source": [ 664 | "Their lowest rated movies:" 665 | ] 666 | }, 667 | { 668 | "cell_type": "code", 669 | "execution_count": 16, 670 | "metadata": {}, 671 | "outputs": [ 672 | { 673 | "data": { 674 | "text/html": [ 675 | "
\n", 676 | "\n", 689 | "\n", 690 | " \n", 691 | " \n", 692 | " \n", 693 | " \n", 694 | " \n", 695 | " \n", 696 | " \n", 697 | " \n", 698 | " \n", 699 | " \n", 700 | " \n", 701 | " \n", 702 | " \n", 703 | " \n", 704 | " \n", 705 | " \n", 706 | " \n", 707 | " \n", 708 | " \n", 709 | " \n", 710 | " \n", 711 | " \n", 712 | " \n", 713 | " \n", 714 | " \n", 715 | " \n", 716 | " \n", 717 | " \n", 718 | " \n", 719 | " \n", 720 | " \n", 721 | " \n", 722 | " \n", 723 | " \n", 724 | " \n", 725 | " \n", 726 | " \n", 727 | " \n", 728 | " \n", 729 | " \n", 730 | " \n", 731 | " \n", 732 | " \n", 733 | " \n", 734 | " \n", 735 | " \n", 736 | " \n", 737 | " \n", 738 | " \n", 739 | " \n", 740 | " \n", 741 | " \n", 742 | "
userIdmovieIdratingtimestamptitle
939536902.01043339908Porky's Revenge (1985)
1229552832.01043339957National Lampoon's Van Wilder (2002)
1009540152.01043339957Dude, Where's My Car? (2000)
1649573731.01105401093Hellboy (2004)
1099547321.01043339283Bubble Boy (2001)
\n", 743 | "
" 744 | ], 745 | "text/plain": [ 746 | " userId movieId rating timestamp title\n", 747 | "93 95 3690 2.0 1043339908 Porky's Revenge (1985)\n", 748 | "122 95 5283 2.0 1043339957 National Lampoon's Van Wilder (2002)\n", 749 | "100 95 4015 2.0 1043339957 Dude, Where's My Car? (2000)\n", 750 | "164 95 7373 1.0 1105401093 Hellboy (2004)\n", 751 | "109 95 4732 1.0 1043339283 Bubble Boy (2001)" 752 | ] 753 | }, 754 | "execution_count": 16, 755 | "metadata": {}, 756 | "output_type": "execute_result" 757 | } 758 | ], 759 | "source": [ 760 | "bottom_5 = user_ratings[user_ratings['rating']<3].tail()\n", 761 | "bottom_5" 762 | ] 763 | }, 764 | { 765 | "cell_type": "markdown", 766 | "metadata": {}, 767 | "source": [ 768 | "Based on their preferences above, we can get a sense that user 95 likes action and crime movies from the early 1990's over light-hearted American comedies from the early 2000's. Let's see what recommendations our model will generate for user 95.\n", 769 | "\n", 770 | "We'll use the `recommend()` method, which takes in the user index of interest and transposed user-item matrix. " 771 | ] 772 | }, 773 | { 774 | "cell_type": "code", 775 | "execution_count": 17, 776 | "metadata": {}, 777 | "outputs": [ 778 | { 779 | "data": { 780 | "text/plain": [ 781 | "[(855, 1.127779),\n", 782 | " (1043, 0.98673713),\n", 783 | " (1210, 0.9256185),\n", 784 | " (3633, 0.90900886),\n", 785 | " (1978, 0.8929481),\n", 786 | " (4155, 0.84075284),\n", 787 | " (2979, 0.82858247),\n", 788 | " (3609, 0.78015),\n", 789 | " (4791, 0.7672245),\n", 790 | " (4010, 0.7530525)]" 791 | ] 792 | }, 793 | "execution_count": 17, 794 | "metadata": {}, 795 | "output_type": "execute_result" 796 | } 797 | ], 798 | "source": [ 799 | "X_t = X.T.tocsr()\n", 800 | "\n", 801 | "user_idx = user_mapper[user_id]\n", 802 | "recommendations = model.recommend(user_idx, X_t)\n", 803 | "recommendations" 804 | ] 805 | }, 806 | { 807 | "cell_type": "markdown", 808 | "metadata": {}, 809 | "source": [ 810 | "We can't interpret the results as is since movies are represented by their index. We'll have to loop over the list of recommendations and get the movie title for each movie index. " 811 | ] 812 | }, 813 | { 814 | "cell_type": "code", 815 | "execution_count": 18, 816 | "metadata": {}, 817 | "outputs": [ 818 | { 819 | "name": "stdout", 820 | "output_type": "stream", 821 | "text": [ 822 | "Abyss, The (1989)\n", 823 | "Star Trek: First Contact (1996)\n", 824 | "Hunt for Red October, The (1990)\n", 825 | "Lord of the Rings: The Fellowship of the Ring, The (2001)\n", 826 | "Star Wars: Episode I - The Phantom Menace (1999)\n", 827 | "Chicago (2002)\n", 828 | "Crouching Tiger, Hidden Dragon (Wo hu cang long) (2000)\n", 829 | "Ocean's Eleven (2001)\n", 830 | "Lord of the Rings: The Return of the King, The (2003)\n", 831 | "Punch-Drunk Love (2002)\n" 832 | ] 833 | } 834 | ], 835 | "source": [ 836 | "for r in recommendations:\n", 837 | " recommended_title = get_movie_title(r[0])\n", 838 | " print(recommended_title)" 839 | ] 840 | }, 841 | { 842 | "cell_type": "markdown", 843 | "metadata": {}, 844 | "source": [ 845 | "User 95's recommendations consist of action, crime, and thrillers. None of their recommendations are comedies. " 846 | ] 847 | } 848 | ], 849 | "metadata": { 850 | "kernelspec": { 851 | "display_name": "Python 3", 852 | "language": "python", 853 | "name": "python3" 854 | }, 855 | "language_info": { 856 | "codemirror_mode": { 857 | "name": "ipython", 858 | "version": 3 859 | }, 860 | "file_extension": ".py", 861 | "mimetype": "text/x-python", 862 | "name": "python", 863 | "nbconvert_exporter": "python", 864 | "pygments_lexer": "ipython3", 865 | "version": "3.7.6" 866 | } 867 | }, 868 | "nbformat": 4, 869 | "nbformat_minor": 4 870 | } 871 | -------------------------------------------------------------------------------- /presentation/images/amazon-ecommerce.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/amazon-ecommerce.png -------------------------------------------------------------------------------- /presentation/images/amazon-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/amazon-example.png -------------------------------------------------------------------------------- /presentation/images/bookstore.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/bookstore.png -------------------------------------------------------------------------------- /presentation/images/collaborative-filtering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/collaborative-filtering.png -------------------------------------------------------------------------------- /presentation/images/content-based-filtering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/content-based-filtering.png -------------------------------------------------------------------------------- /presentation/images/cosine-sim.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/cosine-sim.png -------------------------------------------------------------------------------- /presentation/images/gypsy-musical.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/gypsy-musical.png -------------------------------------------------------------------------------- /presentation/images/knn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/knn.png -------------------------------------------------------------------------------- /presentation/images/lamerica.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/lamerica.png -------------------------------------------------------------------------------- /presentation/images/long-tail-book.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/long-tail-book.png -------------------------------------------------------------------------------- /presentation/images/medium-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/medium-example.png -------------------------------------------------------------------------------- /presentation/images/netflix-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/netflix-example.png -------------------------------------------------------------------------------- /presentation/images/recommender-examples.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/recommender-examples.png -------------------------------------------------------------------------------- /presentation/images/recommender-ml-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/recommender-ml-1.png -------------------------------------------------------------------------------- /presentation/images/recommender-ml-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/recommender-ml-2.png -------------------------------------------------------------------------------- /presentation/images/recommender-ml-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/recommender-ml-3.png -------------------------------------------------------------------------------- /presentation/images/recommender-ml-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/recommender-ml-4.png -------------------------------------------------------------------------------- /presentation/images/spotify-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/spotify-example.png -------------------------------------------------------------------------------- /presentation/images/tasting-booth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/tasting-booth.png -------------------------------------------------------------------------------- /presentation/images/utility-matrix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/topspinj/recommender-tutorial/d643f8b131fae6e2aaf91e5d389c358a1e823426/presentation/images/utility-matrix.png -------------------------------------------------------------------------------- /recommender-basics.md: -------------------------------------------------------------------------------- 1 | ### What is a recommendation system? 2 | 3 | A recommendation system is an algorithm that predicts a user's preference toward an item. In most cases, its goal is to **drive user engagement**. 4 | 5 | **Examples:** 6 | 7 | - recommending products based on past purchases or product searches (Amazon) 8 | - suggesting TV shows or movies based on prediction of a user's interests (Netflix) 9 | - creating well-curated playlists based on song history (Spotify) 10 | - personalized ads based on "liked" posts or previous websites visited (Facebook) 11 | 12 | The two most common recommendation system techniques are: 1) collaborative filtering, and 2) content-based filtering. 13 | 14 | ### Collaborative Filtering 15 | 16 | Collaborative filering (CF) is based on the concept of "homophily" - similar users like similar things. It uses item preferences from other users to predict which item a particular user will like best. Collaborative filtering uses a user-item matrix to generate recommendations. This matrix is populated with values that indicate a given user's preference towards a given item. It's very unlikely that a user will have interacted with every item, so in most real-life cases, the user-item matrix is very sparse. 17 | 18 | 19 | 20 | 21 | 22 | Collaborative filtering can be further divided into two categories: memory-based and model-based. 23 | 24 | - **Memory-based** algorithms look at item-item, user-item, or user-user similarity using different similarity metrics such as Pearson correlation coefficient, cosine similarity, etc. This approach is easy to apply to your user-item matrix and very interpretable. However, its performance decreases as the dataset becomes more sparse. 25 | - **Model-based** algorithms use matrix factorization techniques such as Single Vector Decomposition ([SVD](https://www.wikiwand.com/en/Singular-value_decomposition)) and Non-negative Matrix Factorization ([NMF](https://www.wikiwand.com/en/Non-negative_matrix_factorization)) to extract latent/hidden, meaningful factors from the data. 26 | 27 | A major disadvantage of collaborative filtering is the **cold start problem**. You can only get recommendations for users and items that already have "interactions" in the user-item matrix. Collaborative filtering fails to provide personalized recommendations for brand new users or newly released items. 28 | 29 | 30 | ### Content-based Filtering 31 | 32 | Content-based filtering generates recommendations based on user and item features. Given a set of item features (movie genre, release date, country, language, etc.), it predicts how a user will rate an item based on their ratings of previous movies. 33 | 34 | Content-based filtering handles the "cold start" problem because it is able to provide personalized recommendations for brand new users and features. 35 | 36 | 37 | 38 | 39 | ### How do we define a user's "preference" towards an item? 40 | 41 | There are two types of feedback data: 42 | 43 | 1. Explicit feedback, which considers a user's direct response to an item (e.g., rating, like/dislike) 44 | 45 | 2. Implicit feedback, which looks at a user's indirect behaviour towards an item (e.g., number of times a user has watched a movie) 46 | 47 | Before using this data in your recommendation system, it is important to perform some data pre-processing. For example, you should normalize ratings of different users to the same scale. More information on how to normalize data in recommendation systems is described [here](https://www.cs.purdue.edu/homes/lsi/sigir04-cf-norm.pdf). -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Utils 3 | ===== 4 | """ 5 | 6 | import numpy as np 7 | import pandas as pd 8 | from scipy.sparse import csr_matrix 9 | from sklearn.neighbors import NearestNeighbors 10 | 11 | def create_X(df): 12 | """ 13 | Generates a sparse matrix from ratings dataframe. 14 | 15 | Args: 16 | df: pandas dataframe 17 | 18 | Returns: 19 | X: sparse matrix 20 | user_mapper: dict that maps user id's to user indices 21 | user_inv_mapper: dict that maps user indices to user id's 22 | movie_mapper: dict that maps movie id's to movie indices 23 | movie_inv_mapper: dict that maps movie indices to movie id's 24 | """ 25 | N = df['userId'].nunique() 26 | M = df['movieId'].nunique() 27 | 28 | user_mapper = dict(zip(np.unique(df["userId"]), list(range(N)))) 29 | movie_mapper = dict(zip(np.unique(df["movieId"]), list(range(M)))) 30 | 31 | user_inv_mapper = dict(zip(list(range(N)), np.unique(df["userId"]))) 32 | movie_inv_mapper = dict(zip(list(range(M)), np.unique(df["movieId"]))) 33 | 34 | user_index = [user_mapper[i] for i in df['userId']] 35 | item_index = [movie_mapper[i] for i in df['movieId']] 36 | 37 | X = csr_matrix((df["rating"], (item_index, user_index)), shape=(M, N)) 38 | 39 | return X, user_mapper, movie_mapper, user_inv_mapper, movie_inv_mapper 40 | 41 | def find_similar_movies(movie_id, X, k, movie_mapper, movie_inv_mapper, metric='cosine', show_distance=False): 42 | """ 43 | Finds k-nearest neighbours for a given movie id. 44 | 45 | Args: 46 | movie_id: id of the movie of interest 47 | X: user-item utility matrix 48 | k: number of similar movies to retrieve 49 | metric: distance metric for kNN calculations 50 | 51 | Returns: 52 | list of k similar movie ID's 53 | """ 54 | neighbour_ids = [] 55 | 56 | movie_ind = movie_mapper[movie_id] 57 | movie_vec = X[movie_ind] 58 | k+=1 59 | kNN = NearestNeighbors(n_neighbors=k, algorithm="brute", metric=metric) 60 | kNN.fit(X) 61 | if isinstance(movie_vec, (np.ndarray)): 62 | movie_vec = movie_vec.reshape(1,-1) 63 | neighbour = kNN.kneighbors(movie_vec, return_distance=show_distance) 64 | for i in range(0,k): 65 | n = neighbour.item(i) 66 | neighbour_ids.append(movie_inv_mapper[n]) 67 | neighbour_ids.pop(0) 68 | return neighbour_ids --------------------------------------------------------------------------------