├── README.md ├── sampledocs ├── stopwords ├── article4 ├── article1 ├── article2 ├── article3 └── article5 └── LocalitySensitiveHashing.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Locality Sensitive Hashing Tutorial 2 | 3 | As the name suggests, this is a tutorial on locality sensitive hashing. All of the information is contained in the notebook. 4 | 5 | The sampledocs folder contains some artificial data for performing the document similarity task. It consists of news articles pulled from cnn, with one document consisting of partial concatenations of the others. This is to create artificilly similar documents, which our algorithms are trying to find. 6 | 7 | The similarity task for vectors can easily generate synthetic data by just creating random matrices, so we do that in the notebook. 8 | -------------------------------------------------------------------------------- /sampledocs/stopwords: -------------------------------------------------------------------------------- 1 | i 2 | me 3 | my 4 | myself 5 | we 6 | our 7 | ours 8 | ourselves 9 | you 10 | your 11 | yours 12 | yourself 13 | yourselves 14 | he 15 | him 16 | his 17 | himself 18 | she 19 | her 20 | hers 21 | herself 22 | it 23 | its 24 | itself 25 | they 26 | them 27 | their 28 | theirs 29 | themselves 30 | what 31 | which 32 | who 33 | whom 34 | this 35 | that 36 | these 37 | those 38 | am 39 | is 40 | are 41 | was 42 | were 43 | be 44 | been 45 | being 46 | have 47 | has 48 | had 49 | having 50 | do 51 | does 52 | did 53 | doing 54 | a 55 | an 56 | the 57 | and 58 | but 59 | if 60 | or 61 | because 62 | as 63 | until 64 | while 65 | of 66 | at 67 | by 68 | for 69 | with 70 | about 71 | against 72 | between 73 | into 74 | through 75 | during 76 | before 77 | after 78 | above 79 | below 80 | to 81 | from 82 | up 83 | down 84 | in 85 | out 86 | on 87 | off 88 | over 89 | under 90 | again 91 | further 92 | then 93 | once 94 | here 95 | there 96 | when 97 | where 98 | why 99 | how 100 | all 101 | any 102 | both 103 | each 104 | few 105 | more 106 | most 107 | other 108 | some 109 | such 110 | no 111 | nor 112 | not 113 | only 114 | own 115 | same 116 | so 117 | than 118 | too 119 | very 120 | s 121 | t 122 | can 123 | will 124 | just 125 | don 126 | should 127 | now 128 | -------------------------------------------------------------------------------- /sampledocs/article4: -------------------------------------------------------------------------------- 1 | ew quarantine hobbies have unearthed new passions, some bringing with them a literal silver lining. 2 | This year, backyard archaeologists in the United Kingdom have recorded discoveries of more than 47,000 objects, the British Museum announced this week. 3 | Regular people found the vast majority of the historical artifacts by traversing the countryside with metal detectors, before adding or updating records through the museum's Portable Antiquities Scheme. 4 | The British Museum said the program also saw an uptick in people updating digital records of antiquities while the country was under a full lockdown between March 22 and May 13. 5 | That database contains records of more than 1.5 million objects discovered since 1998 by the general public rather than by professional archaeologists. 6 | "It is brilliant to see the scheme growing from strength to strength during lockdown thanks to garden discoveries and digital reporting," said UK Culture Minister Caroline Dinenage, in a news release. 7 | This list of garden treasures dug up this year includes a 13th century medieval seal, bearing a Latin inscription reading "David, God's messenger, bishop of St. Andrews." 8 | And shining out among the new discoveries are two hoards of coins. 9 | One of those troves, which contained 50 South African solid gold coins, was unearthed in Milton Keynes, a town about 50 miles northwest of London. 10 | It's a mystery how those coins, minted in the 1970s during South Africa's apartheid era, wound up buried in a British backyard after a half-century. 11 | The other major coin hoard, which held 63 gold coins and one silver coin featuring monarchs Edward IV and Henry VIII, was likely buried in the 16th century. It included coins bearing the initials of several of Henry VIII's wives, including Catherine of Aragon, Anne Boleyn and Jane Seymour. 12 | Nearly 500 years later during the Covid-19 pandemic, residents rediscovered them while weeding in their garden. 13 | Another amateur find during the pandemic was an ancient Roman furniture fitting made of a copper alloy, and clearly featuring the face of the god Oceanus. 14 | That artifact, found in Old Basing about 50 miles southeast of London, dates as far back as the 1st century. 15 | A new report from PAS shows some 81,602 objects added to the scheme in 2019, before the recent spate of lockdown treasure hunting, each of those items now coming under public ownership. 16 | The UK's Treasure Act of 1996 requires that finders report each discovery more than 300 years old to the local coroner in the area in which they found it. 17 | If the local authority defines the object as treasure, then it takes the find to the British Museum to be valued. The government then pays a fair market price to the discoverer. 18 | The law is intended to allow national or local museums to ultimately acquire the historic treasures so that the overall public can benefit. 19 | Even during the pandemic the Portable Antiquities Scheme's liaison officers have been able to reach out to finders and obtain relics of significance, Michael Lewis, who heads the program, said in the news release. 20 | The mission continues to "ensure finds, important for understanding Britain's past, are not lost but instead recorded for posterity," he said. 21 | -------------------------------------------------------------------------------- /sampledocs/article1: -------------------------------------------------------------------------------- 1 | Washington (CNN)White House chief of staff Mark Meadows told Food and Drug Administration Commissioner Dr. Stephen Hahn he needed to grant an emergency use authorization for Pfizer/BioNTech's coronavirus vaccine by the end of Friday, and if not, he needs to resign, an administration official and a source familiar with the situation tell CNN. 2 | 3 | Another person familiar with matter, who also confirmed the demand that the vaccine be authorized by the end of Friday, said President Donald Trump has been venting about the FDA chief since the vaccine was rolled out in the UK earlier this week. 4 | The two men had a call Friday morning. A White House official said they do not comment on private conversations but the chief "regularly requests updates on the progress toward a vaccine." 5 | Hahn quickly disputed the description of the conversation, which was first reported by The Washington Post, but the news is likely to raise additional questions about the extent to which Trump administration political interests are involved with the vaccine authorization process, and could undermine public confidence in the effort.Vaccine advisers to the FDA voted Thursday to recommend the agency grant emergency use authorization to the vaccine, and it's expected to be authorized imminently."This is an untrue representation of the phone call with the Chief of Staff. The FDA was encouraged to continue working expeditiously on Pfizer-BioNTech's (emergency use authorization) request," Hahn said in a statement Friday afternoon. "FDA is committed to issuing this authorization quickly, as we noted in our statement this morning." 6 | A White House official familiar with the conversation between Meadows and Hahn said it's doubtful the FDA commissioner will actually be fired. The blunt warning from the chief of staff that he might as well resign if Pfizer's emergency use authorization isn't granted by Friday was a larger sign of the President's frustration, this person said. 7 | Public health experts have been fearful all along that White House officials would put undue pressure on the authorization process and, in turn, compromise public confidence in the vaccine, a source close to the White House coronavirus task force told CNN. 8 | This source said it's unclear why Meadows would make such a threat this late in the process and that authorization of the vaccine is expected at any moment. This person added that authorization could come by the end of Friday. 9 | Dr. Moncef Slaoui, the head of the US government's effort to develop a vaccine against Covid-19, told CNN's Jake Tapper on "The Lead" Friday he was concerned about the reports' potential effect on undermining confidence in the vaccine's safety. 10 | "Yes, I think there is an opportunity there for people to see undue pressure if the story is right," he said while also defending the FDA's authorization process, which he described as "an effective, transparent, thorough, in-depth review." 11 | Trump has grown impatient with the authorization process in recent weeks, with one person familiar with the President's thinking telling CNN that Trump wants to rush out as many vaccines as possible before he leaves office. 12 | On Friday, he called the FDA "a big, old, slow turtle." 13 | "Get the dam vaccines out NOW, Dr. Hahn @SteveFDA," Trump tweeted. "Stop playing games and start saving lives!!!" 14 | President-elect Joe Biden, in remarks Friday afternoon at an event announcing additional top administration picks, did not address the news of the Meadows demand to Hahn but urged the public to have faith in the vaccine and expressed gratitude "to the scientists and the public experts who evaluated its safety and efficiency, free from political influence." 15 | "I want to make it clear to the public, you should have confidence in this -- there is no political influence," Biden said. "These are first-rate scientists taking their time looking at all of the elements that need to be looked at. Scientific integrity led us to this point." 16 | -------------------------------------------------------------------------------- /sampledocs/article2: -------------------------------------------------------------------------------- 1 | Investigators with the Manhattan district attorney's office have interviewed several employees at President Donald Trump's lender and insurer in recent weeks as part of a wide-ranging investigation into the Trump Organization, according to multiple people familiar with the investigation. 2 | 3 | Two employees of Deutsche Bank, which has loaned more than $300 million to the Trump Organization, were interviewed by prosecutors, according to sources familiar with the matter. 4 | The interviews took place after the November presidential election, the people said, and focused on general questions about how bankers assess loans and underwriting criteria. 5 | The questioning was not specific to the bank's dealing with the Trump Organization or the President, the people said, with one person adding that it was the beginning of the process. Additional interviews are expected in the near future, they said. 6 | Prosecutors also interviewed at least one employee at Aon, an insurance broker who has done work with the President's company, according to one source familiar with the matter. 7 | A spokeswoman for Aon confirmed the company received a subpoena and said it is cooperating with the investigation. The spokeswoman declined to comment on any employee interviews. Representatives for Deutsche Bank and the district attorney's office, led by Cyrus Vance, also declined to comment. Deutsche Bank was subpoenaed as part of the investigation last year and has said it cooperates with authorized investigations. 8 | The New York Times first reported on the interviews with Deutsche Bank and Aon employees. 9 | The interviews with Trump counterparties comes as prosecutors wait for a decision by the US Supreme Court over a grand jury subpoena for the President's tax returns. The President has lost several legal challenges in an attempt to block the subpoena to Mazars USA, his long-time accounting firm, for eight years of his personal and business records and tax returns. 10 | Last month, the 2nd US Circuit Court of Appeals denied Trump's latest effort to block the subpoena paving the way for it to be enforced. The President's lawyers have asked the Supreme Court to stay, or halt, the ruling and a decision is expected any day. 11 | The Mazars records are critical to the investigation, prosecutors have said. The Manhattan district attorney investigation is the only criminal inquiry facing Trump, his business and his family and will continue after he leaves office. Trump has had discussions about issuing pardons to his family members and possibly himself, CNN has reported, but those pardons would not insulate him from a state criminal indictment. 12 | In court filings the district attorney's office has suggested the inquiry could involve tax fraud, insurance fraud and schemes to defraud its lenders. They also recently subpoenaed the Trump Organization for records relating to fees it has paid to consultants, including a payment made to a company controlled by the President's daughter, Ivanka Trump, according to people familiar with the matter. 13 | The Trump Organization has denied any wrongdoing and said applicable taxes were paid. 14 | Last year prosecutors with the district attorney's office interviewed Michael Cohen, the President's former personal attorney, at least three times about his knowledge of the Trump Organization's business dealings. 15 | Cohen testified before Congress in February 2019 that the Trump Organization allegedly manipulated its financial statements to suit its desired outcomes. Cohen said Trump "deflated his assets to reduce his real estate taxes." And he alleged company officials would play with the financial numbers when dealing with insurance companies and Deutsche Bank. 16 | Specifically, Cohen alleged the President inflated the value of his assets at times, including in 2014 when Trump submitted documents to Deutsche Bank as part of an attempt to bid for the Buffalo Bills football team. Trump never did the loan. 17 | Cohen pleaded guilty to federal crimes, including campaign finance charges for facilitating hush-money payments to silence two woman's allegations of affairs with Trump. Trump has denied the affairs. Cohen is serving a three year prison sentence and was released to home confinement earlier this year due to the coronavirus pandemic. 18 | 19 | -------------------------------------------------------------------------------- /sampledocs/article3: -------------------------------------------------------------------------------- 1 | In the final days of his desperate, dishonest campaign to upend last month's election, President Donald Trump tossed off a particularly audacious and offensive challenge aimed at those he somehow thinks can change the outcome. 2 | 3 | "Let's see if they have the courage to do what everybody in this country knows is right," he said. 4 | With time running out, his blatant hope was to intimidate and bait state legislatures or the Supreme Court into overturning a vote of the people, by legislative or judicial fiat. 5 | Trump's perverse definition of "courage" and "right," of course, amounts to a willingness to bend truth to his will and prize his continuation in office over American democracy. 6 | Yet against this madness, we have witnessed many acts of genuine courage. Of people of both parties bravely doing right.Secretaries of state and election authorities in the contested states, Republicans and Democrats, have weathered death threats and vows of political retribution simply for doing their jobs and counting, recounting and certifying the vote. They have shown enormous courage and deserve our gratitude and respect. 7 | Thousands of everyday Americans worked around the clock -- in the midst of a pandemic and sometimes with the din of howling mobs in the background -- to count and recount the votes. These patriotic Americans showed inspiring courage. 8 | So, too, did the governors who loyally campaigned for Trump but refused to bow to his threats and intimidation after the election. 9 | Yes, they were simply doing what the law required and democracy demands by certifying the vote in their states. But these officials acted knowing that in the hothouse of Trump's Republican Party, doing their duty now could cost them their jobs in primaries later. 10 | Dozens of judges -- some appointed by Trump -- have summarily dismissed his ferocious, groundless assault on the election results. They have shown gratifying fidelity to the law. 11 | Trump has made clear from the very start of his presidency that he believed that every branch, every person in government, should be beholden to him before the law and their oaths of office.He told us before the election, in rushing through the Supreme Court confirmation of Amy Coney Barrett, that he wanted nine justices to rule on election disputes. His implication was clear: My justices. My way. 12 | But this very conservative Supreme Court would not (so far, at least) be enlisted in the profane and unconstitutional mission of overturning an election on his behalf. 13 | As they and other federal judges have lifetime appointments, perhaps it took less courage than it did for those elected officials who have risked their careers -- and even their lives -- to stand by the rule of law. But the justices deserve credit, too, for firmly and unanimously rejecting the absurd lawsuits Trump and his loyalists have pushed their way. 14 | Courageous, too, have been those handful of elected Republicans who have acknowledged the results, congratulated President-elect Joe Biden and urged a peaceful, orderly transition of power. 15 | But if we have seen acts of courage, we also have seen cowardice. 16 | Rudy Giuliani and the Trump legal team have debased themselves, our legal system and democracy by filing one frivolous lawsuit after another, crying fraud on TV -- but not in the courtroom, where evidence is required.And too many Republican officials, fearful of getting sideways with Trump's base, have dutifully echoed his dishonest charges of vote fraud. 17 | A group of Republican state attorneys general lined up in support of a preposterous eleventh-hour filing from the attorney general of Texas, asking the Supreme Court to overturn the results in four states Trump lost. 18 | The suit was filed without standing, supporting evidence or a colorable argument. Nonetheless, Rep. Mike Johnson of Louisiana circulated an email among his Republican House colleagues urging them to sign an amicus brief in support of this outrageous folly. Trump is "anxiously awaiting the final list" to see who signs on to the amicus brief, Johnson wrote, implying dire consequences for anyone who failed join. 19 | Within 24 hours, more than 100 House Republicans complied. 20 | Supporters of Trump have gone so far as to argue that Republicans may stage a battle on the House floor to reject the Biden electors from contested states, a symbolic gesture that would fail but set a chilling precedent.Almost as egregious have been the many Republican members of the House and Senate who have refused to acknowledge Biden's victory and countenanced weeks of delay in the transition. 21 | Through their silence, they have given credence to Trump's blatant lies, which have now gained traction among a large majority of Republicans nationally. 22 | It's crazy, authoritarian stuff. If it were occurring anywhere else, Americans would condemn it as an appalling attempt to undermine democracy. 23 | History will scorn the cowards who meekly complied with Trump's scheme to tarnish and overturn the election -- and honor the many who showed courage and fidelity to the rule of law during this time of trial. 24 | -------------------------------------------------------------------------------- /sampledocs/article5: -------------------------------------------------------------------------------- 1 | ew quarantine hobbies have unearthed new passions, some bringing with them a literal silver lining. 2 | This year, backyard archaeologists in the United Kingdom have recorded discoveries of more than 47,000 objects, the British Museum announced this week. 3 | Regular people found the vast majority of the historical artifacts by traversing the countryside with metal detectors, before adding or updating records through the museum's Portable Antiquities Scheme. 4 | The British Museum said the program also saw an uptick in people updating digital records of antiquities while the country was under a full lockdown between March 22 and May 13. 5 | That database contains records of more than 1.5 million objects discovered since 1998 by the general public rather than by professional archaeologists. 6 | "It is brilliant to see the scheme growing from strength to strength during lockdown thanks to garden discoveries and digital reporting," said UK Culture Minister Caroline Dinenage, in a news release. 7 | This list of garden treasures dug up this year includes a 13th century medieval seal, bearing a Latin inscription reading "David, God's messenger, bishop of St. Andrews." 8 | The other major coin hoard, which held 63 gold coins and one silver coin featuring monarchs Edward IV and Henry VIII, was likely buried in the 16th century. It included coins bearing the initials of several of Henry VIII's wives, including Catherine of Aragon, Anne Boleyn and Jane Seymour. 9 | Nearly 500 years later during the Covid-19 pandemic, residents rediscovered them while weeding in their garden. 10 | Another amateur find during the pandemic was an ancient Roman furniture fitting made of a copper alloy, and clearly featuring the face of the god Oceanus. 11 | That artifact, found in Old Basing about 50 miles southeast of London, dates as far back as the 1st century. 12 | A new report from PAS shows some 81,602 objects added to the scheme in 2019, before the recent spate of lockdown treasure hunting, each of those items now coming under public ownership. 13 | If the local authority defines the object as treasure, then it takes the find to the British Museum to be valued. The government then pays a fair market price to the discoverer. 14 | The law is intended to allow national or local museums to ultimately acquire the historic treasures so that the overall public can benefit. 15 | Even during the pandemic the Portable Antiquities Scheme's liaison officers have been able to reach out to finders and obtain relics of significance, Michael Lewis, who heads the program, said in the news release. 16 | The mission continues to "ensure finds, important for understanding Britain's past, are not lost but instead recorded for posterity," he said. 17 | 18 | Another person familiar with matter, who also confirmed the demand that the vaccine be authorized by the end of Friday, said President Donald Trump has been venting about the FDA chief since the vaccine was rolled out in the UK earlier this week. 19 | The two men had a call Friday morning. A White House official said they do not comment on private conversations but the chief "regularly requests updates on the progress toward a vaccine." 20 | Hahn quickly disputed the description of the conversation, which was first reported by The Washington Post, but the news is likely to raise additional questions about the extent to which Trump administration political interests are involved with the vaccine authorization process, and could undermine public confidence in the effort.Vaccine advisers to the FDA voted Thursday to recommend the agency grant emergency use authorization to the vaccine, and it's expected to be authorized imminently."This is an untrue representation of the phone call with the Chief of Staff. The FDA was encouraged to continue working expeditiously on Pfizer-BioNTech's (emergency use authorization) request," Hahn said in a statement Friday afternoon. "FDA is committed to issuing this authorization quickly, as we noted in our statement this morning." 21 | Public health experts have been fearful all along that White House officials would put undue pressure on the authorization process and, in turn, compromise public confidence in the vaccine, a source close to the White House coronavirus task force told CNN. 22 | This source said it's unclear why Meadows would make such a threat this late in the process and that authorization of the vaccine is expected at any moment. This person added that authorization could come by the end of Friday. 23 | Dr. Moncef Slaoui, the head of the US government's effort to develop a vaccine against Covid-19, told CNN's Jake Tapper on "The Lead" Friday he was concerned about the reports' potential effect on undermining confidence in the vaccine's safety. 24 | President-elect Joe Biden, in remarks Friday afternoon at an event announcing additional top administration picks, did not address the news of the Meadows demand to Hahn but urged the public to have faith in the vaccine and expressed gratitude "to the scientists and the public experts who evaluated its safety and efficiency, free from political influence." 25 | "I want to make it clear to the public, you should have confidence in this -- there is no political influence," Biden said. "These are first-rate scientists taking their time looking at all of the elements that need to be looked at. Scientific integrity led us to this point." 26 | Investigators with the Manhattan district attorney's office have interviewed several employees at President Donald Trump's lender and insurer in recent weeks as part of a wide-ranging investigation into the Trump Organization, according to multiple people familiar with the investigation. 27 | Two employees of Deutsche Bank, which has loaned more than $300 million to the Trump Organization, were interviewed by prosecutors, according to sources familiar with the matter. 28 | The questioning was not specific to the bank's dealing with the Trump Organization or the President, the people said, with one person adding that it was the beginning of the process. Additional interviews are expected in the near future, they said. 29 | Prosecutors also interviewed at least one employee at Aon, an insurance broker who has done work with the President's company, according to one source familiar with the matter. 30 | A spokeswoman for Aon confirmed the company received a subpoena and said it is cooperating with the investigation. The spokeswoman declined to comment on any employee interviews. Representatives for Deutsche Bank and the district attorney's office, led by Cyrus Vance, also declined to comment. Deutsche Bank was subpoenaed as part of the investigation last year and has said it cooperates with authorized investigations. 31 | The New York Times first reported on the interviews with Deutsche Bank and Aon employees. 32 | In court filings the district attorney's office has suggested the inquiry could involve tax fraud, insurance fraud and schemes to defraud its lenders. They also recently subpoenaed the Trump Organization for records relating to fees it has paid to consultants, including a payment made to a company controlled by the President's daughter, Ivanka Trump, according to people familiar with the matter. 33 | Specifically, Cohen alleged the President inflated the value of his assets at times, including in 2014 when Trump submitted documents to Deutsche Bank as part of an attempt to bid for the Buffalo Bills football team. Trump never did the loan. 34 | Cohen pleaded guilty to federal crimes, including campaign finance charges for facilitating hush-money payments to silence two woman's allegations of affairs with Trump. Trump has denied the affairs. Cohen is serving a three year prison sentence and was released to home confinement earlier this year due to the coronavirus pandemic. 35 | 36 | -------------------------------------------------------------------------------- /LocalitySensitiveHashing.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Locality Sensitive Hashing\n", 8 | "\n", 9 | "This notebook is a tutorial and python implementation of locality sensitive hashing.\n", 10 | "\n", 11 | "We demonstrate the tools using a very very small dataset, consisting of 4 articles pulled from CNN, and a 5th that is a partial concatenation of 3 of those, to artificialy produce some high similarity scores.\n", 12 | "\n", 13 | "Since this is a small dataset, we can easily compare our approximations to the true similarity scores amongst text files. We do so throughout. The layout of this notebook is as follows:\n", 14 | "\n", 15 | "Part I: computing similarity amongst text documents\n", 16 | "1. Introduce a shingle function. Clean and split each text file into a set of K-shingles\n", 17 | "2. Compute the exact Jaccard similarity (intersection over union) between all pairs\n", 18 | "3. Create and apply a MinHashing class:\n", 19 | " 1. Initialize with a dictionary of key-value pairs for the shingles\n", 20 | " 2. Apply \"universal hashing\" to perform minhashing on a shingle set\n", 21 | " 3. can be called like a function to compute a **signature matrix**\n", 22 | "4. Evaluate MinHashing effectiveness by computing scores of all pairs\n", 23 | "5. Introduce LSH for finding **candidate pairs**, i.e. use a banded signature matrix to find all pairs whose similarity is likely above a threshold\n", 24 | "6. Make this efficient, by using hash table for band, column ids, allowing O(n) comparison\n", 25 | "\n", 26 | "Part II: computing similarity amongst vectors\n", 27 | "\n", 28 | "* Afterwards, we provide an additional LSH family for Euclidean spaces, namely cosine similarity. \n", 29 | "* This is used to ascertain the similarity of vectors in a D-dimensional space.\n", 30 | "* Can be implemented using the *Random Hyperplanes* hashing method." 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 26, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "import os\n", 40 | "import time\n", 41 | "import itertools\n", 42 | "import collections\n", 43 | "import numpy as np\n", 44 | "import matplotlib.pyplot as plt" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "# Part I: document similarity" 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "## MinHashing without Locality Sensitive Hashing" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "### Shingling\n", 66 | "\n", 67 | "We illustrate how to convert a word document into a list of shingles.\n", 68 | "We will use three similar strings, and see how this shows similarity.\n", 69 | "I made a directory and copied 4 files from CNN, then the 5th is a concatenation of 3 of them, randomly deleting some lines" 70 | ] 71 | }, 72 | { 73 | "cell_type": "code", 74 | "execution_count": 4, 75 | "metadata": {}, 76 | "outputs": [ 77 | { 78 | "name": "stdout", 79 | "output_type": "stream", 80 | "text": [ 81 | "Average char-length: 3651.6\n", 82 | "Min char-length: 2412\n", 83 | "Max char-length: 5873\n" 84 | ] 85 | } 86 | ], 87 | "source": [ 88 | "import os\n", 89 | "HOME = os.getcwd()\n", 90 | "TARGET = os.path.join(HOME, 'sampledocs/')\n", 91 | "\n", 92 | "documents = []\n", 93 | "for article in os.listdir(TARGET):\n", 94 | " if article == 'stopwords':\n", 95 | " continue\n", 96 | " path = os.path.join(TARGET, article)\n", 97 | " with open(path, 'r') as file:\n", 98 | " documents.append(file.read())\n", 99 | " \n", 100 | "stopwords = []\n", 101 | "with open(os.path.join(TARGET, 'stopwords'), 'r') as file:\n", 102 | " for line in file:\n", 103 | " stopwords.append(line.strip())\n", 104 | " \n", 105 | "for i, doc in enumerate(documents):\n", 106 | " doc = doc.strip().replace('\\n', ' ').lower()\n", 107 | " for word in stopwords:\n", 108 | " doc = doc.replace(' '+word+' ', ' ')\n", 109 | " documents[i] = doc\n", 110 | "\n", 111 | "print(f\"Average char-length: \\\n", 112 | "{np.mean(np.array([len(x) for x in documents]))}\")\n", 113 | "print(f\"Min char-length: {min(len(x) for x in documents)}\")\n", 114 | "print(f\"Max char-length: {max(len(x) for x in documents)}\")" 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": 5, 120 | "metadata": {}, 121 | "outputs": [ 122 | { 123 | "name": "stdout", 124 | "output_type": "stream", 125 | "text": [ 126 | "Found 2060 unique shingles, out of 3009 possible.\n", 127 | "Found 1918 unique shingles, out of 2412 possible.\n", 128 | "Found 2782 unique shingles, out of 3673 possible.\n", 129 | "Found 2091 unique shingles, out of 3291 possible.\n", 130 | "Found 3953 unique shingles, out of 5873 possible.\n" 131 | ] 132 | } 133 | ], 134 | "source": [ 135 | "# create K-shingles by sliding window approach\n", 136 | "def getShingles(str1, K=5):\n", 137 | " d1 = set()\n", 138 | " for i in range(len(str1)-K):\n", 139 | " d1.add(str1[i:i+K])\n", 140 | " print(f\"Found {len(d1)} unique shingles, out of {len(str1)} possible.\")\n", 141 | " return d1\n", 142 | "doc_shingles = [getShingles(s, 5) for s in documents]" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "### Define the Jaccard similarity (intersection over union)" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": 7, 155 | "metadata": {}, 156 | "outputs": [ 157 | { 158 | "name": "stdout", 159 | "output_type": "stream", 160 | "text": [ 161 | "**~~~~~~ True similarity scores ~~~~~~**\n", 162 | "Pair\tScore\n", 163 | "--------------\n", 164 | "(0, 1)\t0.050\n", 165 | "(0, 2)\t0.069\n", 166 | "(0, 3)\t0.093\n", 167 | "(0, 4)\t0.336\n", 168 | "(1, 2)\t0.052\n", 169 | "(1, 3)\t0.051\n", 170 | "(1, 4)\t0.400\n", 171 | "(2, 3)\t0.081\n", 172 | "(2, 4)\t0.083\n", 173 | "(3, 4)\t0.294\n" 174 | ] 175 | } 176 | ], 177 | "source": [ 178 | "def jaccardSim(d1,d2):\n", 179 | " return len(d1.intersection(d2))/len(d1.union(d2))\n", 180 | "\n", 181 | "# itertools.combinations finds all (,n) n-pairs\n", 182 | "# then we use a map op on the tuples with jaccardSim\n", 183 | "pairs = itertools.combinations(documents, 2)\n", 184 | "pair_labels = []\n", 185 | "pair_sims = []\n", 186 | "for x1, x2 in itertools.combinations(zip(range(len(doc_shingles)),doc_shingles), 2):\n", 187 | " pair_labels.append((x1[0],x2[0]))\n", 188 | " pair_sims.append(jaccardSim(x1[1],x2[1]))\n", 189 | " \n", 190 | "print(f\"**~~~~~~ True similarity scores ~~~~~~**\")\n", 191 | "print(\"Pair\\tScore\")\n", 192 | "print(\"-\"*14)\n", 193 | "for pair, score in zip(pair_labels, pair_sims):\n", 194 | " print(f\"{pair}\\t{score:.3f}\")" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": 8, 200 | "metadata": {}, 201 | "outputs": [ 202 | { 203 | "name": "stdout", 204 | "output_type": "stream", 205 | "text": [ 206 | "There are 7534 shingles\n" 207 | ] 208 | } 209 | ], 210 | "source": [ 211 | "# Take union of all sets. Convert to an array and assign\n", 212 | "# each element an integer based on position in array\n", 213 | "fullset = set.union(*doc_shingles)\n", 214 | "shingle_dict = dict(zip(list(fullset),range(len(fullset))))\n", 215 | "print(f\"There are {len(shingle_dict)} shingles\")" 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": 10, 221 | "metadata": {}, 222 | "outputs": [], 223 | "source": [ 224 | "# shingle_dict" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "### 3. Define the MinHash class, capable of creating a signature matrix\n", 232 | "\n", 233 | "Note: this only takes sets as input (not matrices) allowing us to efficiently deal with sparse matrices" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": 11, 239 | "metadata": {}, 240 | "outputs": [ 241 | { 242 | "name": "stdout", 243 | "output_type": "stream", 244 | "text": [ 245 | "Initialization test: passed\n", 246 | "Set parameters to right size: passed\n", 247 | "Permuting a row integer returns array: passed\n", 248 | "Compute minhashed signature matrix: passed\n" 249 | ] 250 | } 251 | ], 252 | "source": [ 253 | "# Create a hash function\n", 254 | "# define as a callable class, so that we only\n", 255 | "# intialize random functions once\n", 256 | "class HashManager():\n", 257 | " def __init__(self, shingle_dict):\n", 258 | " self.shingle_dict = shingle_dict\n", 259 | " self.N = len(shingle_dict)\n", 260 | " self.params = None\n", 261 | " \n", 262 | " def _initParams(self, n_sig):\n", 263 | " self.params = np.random.randint(self.N, size=[n_sig,2])\n", 264 | " \n", 265 | " def _permuteRow(self, row):\n", 266 | " return (self.params@np.array([1,row]))%self.N\n", 267 | " \n", 268 | " def __call__(self, docs, n_sig, init=True):\n", 269 | " # Initialize if we change signature matrix length\n", 270 | " # or if we request to re-initialize\n", 271 | " if self.params is None or len(self.params) != n_sig or init:\n", 272 | " self._initParams(n_sig)\n", 273 | " \n", 274 | " #initialize signature matrix\n", 275 | " sig = np.full((n_sig, len(docs)), np.inf)\n", 276 | " \n", 277 | " # each doc in docs is assumed to be an iterable object\n", 278 | " for j, doc in enumerate(docs):\n", 279 | " for shingle in doc:\n", 280 | " orig_row = shingle_dict[shingle]\n", 281 | " curr_col = self._permuteRow(orig_row)\n", 282 | " sig[:,j] = np.minimum(sig[:,j],curr_col)\n", 283 | " return sig.astype(int)\n", 284 | " \n", 285 | "# run some tests:\n", 286 | "try:\n", 287 | " print(\"Initialization test: \", end=\"\")\n", 288 | " hm = HashManager(shingle_dict)\n", 289 | " print(\"passed\")\n", 290 | "\n", 291 | " print(\"Set parameters to right size: \", end=\"\")\n", 292 | " hm._initParams(n_sig=4)\n", 293 | " assert(hm.params.shape == (4,2))\n", 294 | " print(\"passed\")\n", 295 | "\n", 296 | " print(\"Permuting a row integer returns array: \", end=\"\")\n", 297 | " curr_col = hm._permuteRow(3)\n", 298 | " assert(curr_col.shape == (4,))\n", 299 | " print(\"passed\")\n", 300 | "\n", 301 | " print(\"Compute minhashed signature matrix: \", end=\"\")\n", 302 | " hm(doc_shingles, 4)\n", 303 | " print(\"passed\")\n", 304 | "except Exception as e:\n", 305 | " print(\"failure\")\n", 306 | " print(e.args)" 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": 12, 312 | "metadata": {}, 313 | "outputs": [], 314 | "source": [ 315 | "hm = HashManager(shingle_dict)" 316 | ] 317 | }, 318 | { 319 | "cell_type": "markdown", 320 | "metadata": {}, 321 | "source": [ 322 | "### 4. Use MinHashing to compute similarity scores, and see how well it does" 323 | ] 324 | }, 325 | { 326 | "cell_type": "code", 327 | "execution_count": 17, 328 | "metadata": {}, 329 | "outputs": [ 330 | { 331 | "name": "stdout", 332 | "output_type": "stream", 333 | "text": [ 334 | "**~~~~~~ Similarity score comparison ~~~~~~**\n", 335 | "Pair\t\tApprox\t\tTrue\t\t%Error\n", 336 | "(0, 1)\t\t0.000\t\t0.050\t\t100.00\n", 337 | "(0, 2)\t\t0.100\t\t0.069\t\t45.19\n", 338 | "(0, 3)\t\t0.300\t\t0.093\t\t221.78\n", 339 | "(0, 4)\t\t0.300\t\t0.336\t\t10.77\n", 340 | "(1, 2)\t\t0.100\t\t0.052\t\t93.46\n", 341 | "(1, 3)\t\t0.000\t\t0.051\t\t100.00\n", 342 | "(1, 4)\t\t0.200\t\t0.400\t\t50.02\n", 343 | "(2, 3)\t\t0.200\t\t0.081\t\t147.01\n", 344 | "(2, 4)\t\t0.200\t\t0.083\t\t141.05\n", 345 | "(3, 4)\t\t0.700\t\t0.294\t\t137.69\n", 346 | "True pairs: {(0, 4), (3, 4), (1, 4)}\n", 347 | "Candidate pairs: {(0, 3), (0, 4), (3, 4)}\n", 348 | "False negatives: 1\n", 349 | "Potential false positives: 1\n" 350 | ] 351 | } 352 | ], 353 | "source": [ 354 | "def trueSimScores(doc_shingles):\n", 355 | " pair_labels = []\n", 356 | " pair_sims = []\n", 357 | " idxs = range(len(doc_shingles))\n", 358 | " for x1, x2 in itertools.combinations(zip(idxs,doc_shingles), 2):\n", 359 | " pair_labels.append((x1[0], x2[0]))\n", 360 | " pair_sims.append(jaccardSim(x1[1], x2[1]))\n", 361 | " return dict(zip(pair_labels, pair_sims))\n", 362 | " \n", 363 | "def sigSimScores(sig_mat):\n", 364 | "# cols = [sig_mat[:,i] for i in range(sig_mat.shape[1])]\n", 365 | " cols = sig_mat.T\n", 366 | " idxs = range(sig_mat.shape[1])\n", 367 | " \n", 368 | " pair_labels = []\n", 369 | " pair_sims = []\n", 370 | " for (i,col1), (j,col2) in itertools.combinations(zip(idxs, cols),2):\n", 371 | " pair_labels.append((i,j))\n", 372 | " pair_sims.append(np.mean(col1==col2))\n", 373 | " \n", 374 | " return dict(zip(pair_labels, pair_sims))\n", 375 | "\n", 376 | "def printScoreComparison(true_dict, approx_dict):\n", 377 | " print(f\"**~~~~~~ Similarity score comparison ~~~~~~**\")\n", 378 | " print(\"Pair\\t\\tApprox\\t\\tTrue\\t\\t%Error\")\n", 379 | " for pair, true_value in true_dict.items():\n", 380 | " approx_value = approx_dict[pair]\n", 381 | " err = 100*abs(true_value-approx_value)/true_value\n", 382 | " print(f\"{pair}\\t\\t{approx_value:.3f}\\t\\t{true_value:.3f}\\t\\t{err:.2f}\")\n", 383 | "\n", 384 | "def candidatePairs(score_dict, threshold):\n", 385 | " return set(pair for pair, scr in score_dict.items() if scr>=threshold)\n", 386 | "\n", 387 | "def accMatrix(true_dict, approx_dict, threshold):\n", 388 | " true_pairs = candidatePairs(true_dict, threshold)\n", 389 | " approx_pairs = candidatePairs(approx_dict, threshold)\n", 390 | " false_negatives = len(true_pairs - approx_pairs)\n", 391 | " false_positives = len(approx_pairs - true_pairs)\n", 392 | " print(f\"False negatives: {false_negatives}\")\n", 393 | " print(f\"Potential false positives: {false_positives}\")\n", 394 | "\n", 395 | "sig_mat = hm(doc_shingles, 10)\n", 396 | "true_score_dict = trueSimScores(doc_shingles)\n", 397 | "approx_score_dict = sigSimScores(sig_mat)\n", 398 | "printScoreComparison(true_score_dict, approx_score_dict)\n", 399 | "\n", 400 | "print(\"True pairs:\",candidatePairs(true_score_dict, 0.25))\n", 401 | "print(\"Candidate pairs:\",candidatePairs(approx_score_dict, 0.25))\n", 402 | "accMatrix(true_score_dict, approx_score_dict, 0.4)\n", 403 | "\n", 404 | "# print(f\"**~~~~~~ Approximate similarity scores ~~~~~~**\")\n", 405 | "# print(\"Pair\\t\\tApproximate Score\\t\\tTrue Score\")\n", 406 | "# print(\"-\"*14)\n", 407 | "# for pair, score in sigSimScores(sig_mat):\n", 408 | "# print(f\"{pair}\\t{score:.3f}\")\n", 409 | " \n", 410 | "# print(f\"**~~~~~~ True similarity scores ~~~~~~**\")\n", 411 | "# print(\"Pair\\tScore\")\n", 412 | "# print(\"-\"*14)\n", 413 | "# for pair, score in zip(pair_labels, pair_sims):\n", 414 | "# print(f\"{pair}\\t{score:.3f}\")" 415 | ] 416 | }, 417 | { 418 | "cell_type": "markdown", 419 | "metadata": {}, 420 | "source": [ 421 | "## Adding Locality Sensitive Hashing: preliminary band-structure theory, how to choose band size" 422 | ] 423 | }, 424 | { 425 | "cell_type": "markdown", 426 | "metadata": {}, 427 | "source": [ 428 | "### Effects of changing b,r at fixed n" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "Now implement Locality-sensitive hashing. We use a band structure on the signature matrix. If the matrix has $n$ rows, then we divide it into $b$ bands each of width $r$, such that\n", 436 | "$$n = b*r$$\n", 437 | "\n", 438 | "Let $p$ be the true similarity score (match percent) between a pair. The probability of matching every integer in a band is\n", 439 | "\n", 440 | "$$\\text{prob. one band doesn't match } = 1-p^r$$\n", 441 | "\n", 442 | "Now the probability that NONE of the $b$ bands match is given by\n", 443 | "\n", 444 | "$$\\text{prob. no bands match } = (1-p^r)^b$$\n", 445 | "\n", 446 | "Therefore, the probability that at least one band matches, is\n", 447 | "\n", 448 | "$$P(\\geq 1\\text{ match}) = 1-(1-p^r)^b$$\n", 449 | "\n", 450 | "Now, we want this to be our criteria for a candidate pair. That is, two signatures columns are a candidate pair if $P \\geq 1/2$. We will see in the plot below, that as the signature matrix increases, we can tune $r$ and $b$ to approximately be a step function around the true value of $p$. The goal is to find as few candidate pairs as possible, while also making sure we find them all.\n", 451 | "\n", 452 | "Draw a vertical line at the true p value\n", 453 | "The area UNDER the curve $P(x" 470 | ] 471 | }, 472 | "metadata": { 473 | "needs_background": "light" 474 | }, 475 | "output_type": "display_data" 476 | } 477 | ], 478 | "source": [ 479 | "import matplotlib.pyplot as plt\n", 480 | "n = 200\n", 481 | "ops = [(2,100),(4,50),(10,20),(20,10),(50,4),(100,2)]\n", 482 | "yval = lambda p,r,b: 1-(1-p**r)**b\n", 483 | "pts = np.linspace(0,1,100)\n", 484 | "yval(pts,.2,.2)\n", 485 | "for op in ops:\n", 486 | " plt.plot(pts, yval(pts,op[0],op[1]), label=op)\n", 487 | "plt.plot(pts,0*pts+0.5,'k--', label=\"P=1/2\")\n", 488 | "plt.plot([0.4,0.4],[0,1], 'k-.')\n", 489 | "plt.legend()\n", 490 | "plt.xlabel('p')\n", 491 | "plt.ylabel('Probability')\n", 492 | "plt.title(\"legend: (r,b). p_true=0.4 (vertical line)\")\n", 493 | "plt.show()" 494 | ] 495 | }, 496 | { 497 | "cell_type": "markdown", 498 | "metadata": {}, 499 | "source": [ 500 | "### Effects of changing n while adjusting b,r to keep P=1/2 argument constant" 501 | ] 502 | }, 503 | { 504 | "cell_type": "markdown", 505 | "metadata": {}, 506 | "source": [ 507 | "Now let's solve for the optimal values r,b. \n", 508 | "\n", 509 | "We suppose that we *fix* $n,r,b$. Given these, we determine at which approximate point $p$ is the crossover. We should find $p=p(b,r)$. Afterwards, we can then approximately solve for $r,b$ to be a desired $p$\n", 510 | "\n", 511 | "We begin with for $P=1/2$.\n", 512 | "$$1/2 = 1-(1-p^r)^b$$\n", 513 | "$$ 1-p^r = 2^{-1/b}$$\n", 514 | "$$p = (1-2^{-1/b})^{1/r} = (1-e^{-(1/b)\\ln2})^{1/r} \\approx (1/b)^{1/r}*\\text{const}$$\n", 515 | "\n", 516 | "Finally, if we fix $r$ and $p$, we can find the required bands to be about\n", 517 | "$$ b \\approx 1/p^r\n", 518 | "\n", 519 | "Takeaways:\n", 520 | "\n", 521 | "increasing r -> \n", 522 | "* moves the curve right.\n", 523 | "* more the false negatives\n", 524 | "* means lower chance to match\n", 525 | " \n", 526 | "increasing b -> \n", 527 | "* moves the curve left.\n", 528 | "* more false positives\n", 529 | "* means higher chance to match\n", 530 | " \n", 531 | "increasing n -> \n", 532 | "* the curve approaches a step function.\n", 533 | "* fewer false anythings\n", 534 | "* always good!\n", 535 | "\n", 536 | "Now watch what happens as we pick the optimal r,b and then increase n.\n", 537 | "\n", 538 | "We will try to keep p centered around $p=0.5$" 539 | ] 540 | }, 541 | { 542 | "cell_type": "code", 543 | "execution_count": 19, 544 | "metadata": {}, 545 | "outputs": [ 546 | { 547 | "data": { 548 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVyUxR/A8c/DpeAtqCmoqOAdkrdZnnkfeUuZZ6lpZlpaWlpWmvXzyNI07cIrb80T8D5BBe8LvFDBk8MTRGB3fn+MIscCCyws6Lxfr33JPjP7PLO4fp2dZ+Y7mhACRVEUJe+zMHcDFEVRFNNQAV1RFOUFoQK6oijKC0IFdEVRlBeECuiKoigvCCtzXdjBwUE4Ozub6/KKoih50pEjR8KFECUMlZktoDs7OxMQEGCuyyuKouRJmqZdTa1MDbkoiqK8IFRAVxRFeUGogK4oivKCMNsYuiFxcXGEhoYSExNj7qYoeVz+/PlxcnLC2tra3E1RlByTqwJ6aGgohQoVwtnZGU3TzN0cJY8SQhAREUFoaCgVKlQwd3MUJcekO+Siadrfmqbd0TTtdCrlmqZpv2qadlHTtJOaptXObGNiYmKwt7dXwVzJEk3TsLe3V9/0lJeOMWPonkDbNMrbAa5PH0OAeVlpkArmiimoz5HyMkp3yEUIsVfTNOc0qrwNLBIyD+9BTdOKappWWghx00RtVJSXXrw+nuC7wVx/eJ27j+9yL+Yej2IfEauLJU4fR7w+Hr3QI4RAL/QJr4vTxxEdF01UbBRRcVGyvi6OOH0cOr0OvdDL1yEQQiT5E0AIiI+HuDj5p04nH3q9fAjx/M9nmbgT/kz8BkSy5+aQ0Fg96MXzRj/7mWfH4HmDk72phHOl+CFD3rJvwt8T52TqtWkxxRi6IxCS6Hno02MpArqmaUOQvXjKlStngkub1pUrV+jYsSOnTxscXUrT7t27efvttxPGbLt168bXX3+d5TatX7+eiRMnYmFhgZWVFbNmzeKNN97I8nlTM2jQIDZt2kTJkiWT/B4mTZrEH3/8QYkScoHaDz/8QPv27bOtHS+7WF0s2y9vZ9XZVfhf9+dC5AVidbHmbZTZI3IWaYCluRsh+UTosuW8pgjohr7bGvyrF0IsABYA1K1bN69/PFJ488032bRpk9H1dTodlpZpf8JatmxJ586d0TSNkydP0qtXLwIDA7Pa1FQNGDCAESNG0K9fvxRlo0ePZsyYMdl27dxi+/btALz11ls5fu27j+/yze5vWHxyMfdi7lEkXxGalG9Cx8odqepQlfJFylPMthhF8xeloE1BhBCsPbeW5WeWs//qfuJFPPks81HVoSo1Stagin0VShcsTckCJSlRoASFbAphZ12AC2ftOLDXhr27rTnoa0VsjCUIC2xsNCo4W1DBWaN8eXjlFXBwAHt7KFoUChaUDzs7yJdPPmxswMoKLC3BwuL5n5qW8pElcXFw+jScOAGnTsnH+fNw7VrSHrSlpWx4qVLyUaIEFC8uH0WKQOHCUKgQFCgg34idHeTP//wNWVvLh6Vl0oeFheE3louYIqCHAmUTPXcCbpjgvGYRHx9P//79OXbsGJUrV2bRokXY2dmZ7PwFCxbk008/xcfHhxkzZqTb2y5YsGDCz1FRUUaNDTs7O9O/f382btxIXFwcq1atomrVqka1r0mTJly5csWoui+qyZMnAzkb0IUQrDizglHeowiPDuedV9/Bo4YHb1V8i3xW+VLUf/jkIb8H/M6sQ7O48fAGVeyrMLrRaNq5tKNxucbYWNqkeM2VK7BoPixaBJcuyWNubvDREKhXD9zdwdVVBudc4e5d2LtXPg4ehKNH4dmN7vz5oXp1aNwYBgyQDa9QAcqVg9KlZQB+CZnir24DMELTtOVAA+C+ScbPR42C48ezfJok3N1h1qw0qwQFBfHXX3/RuHFjBg0axNy5cxkzZgyjR49m165dKep7eHgwbtw4APz8/KhVqxZlypRh+vTp1KhRI0X9qKgoatasyXfffQdg1HnXrVvH+PHjuXPnDps3bzbqrTo4OHD06FHmzp3L9OnT+fPPP9m1axejR49OUdfOzg5fX990zzlnzhwWLVpE3bp1mTFjBsWKFTOqLUraYuJj8Fjtwfqg9dQtUxfv97xxf8U91fo7g3fS/7/+hD4IpWWFlvzz9j+0qtgq1f/sT56E776DNWvk8+bN4auvoH172YHNNXQ6OHQINm0Cb2/5718IGbzr1IFhw6BBA/nv2MXlpQ3aaRJCpPkAliHHw+OQvfH3gQ+BD5+Wa8BvwCXgFFA3vXMKIahTp45I7uzZs8+ffPKJEE2bmvbxyScprplYcHCwKFu2bMLzHTt2iLfffjvN1zxz//598fDhQyGEEJs3bxYuLi4G61laWor4+Hijzpncnj17RMuWLdOtV758eREaGiqEEOLgwYNGvSax4OBgUaNGjSTHbt26JeLj44VOpxNffvmlGDhwYIbOaQ5JPk8Z0LRpU9G0aVPTNiYVsfGxouO/HYU2SRPTDkwT8brUPxvRsdFilNcowSRE5dmVxYFrB9I89/nzQnTvLu/8FS4sxFdfCXHliqnfQRbFxQmxdasQ778vhL29bKylpRBNmgjx3XdC7N0rREyMuVuZqwABIpW4aswsl3fSKRfAR5n/LyUV6fSks0vyXs6z5+n1pAsXLpxwrH379gwfPpzw8HAcHByS1M+fP3+ScXNjeujPNGnShEuXLhk8b3L58smv6ZaWlsTHxwNkqYdeKlFXbvDgwXTs2DHN+kr6dHodfdf1ZdP5TczrMI8P636Yat17Mfdos6QNh68f5qN6H/G/Vv/DztrwUGB8PPz8M3z9tRwK/vpr+YU3V32hOnkS/voLli2DsDA5pt25s3y0bi0H7JUMyy2jZbnGtWvX8PPzo1GjRixbtixhjPvnn39O83W3bt2iVKlSaJrG4cOH0ev12NvbA/LG5qJFi3B0dEzxuvTOe/HiRSpVqoSmaRw9epTY2FijzmtI8+bNOZ7JYaybN29SunRpQA4B1axZM1PnUSQhBMM2D2PFmRX8763/pRnMIx9H0npxa07ePsmaXmvoVq1bqnXPnYO+feHIEejSBX77DcqUyY53kAkxMTKAz5sH/v7yburbb8M770DbtmBra+4W5nkqoCdTrVo1Fi5cyNChQ3F1dWXYsGFGvW716tXMmzcPKysrbG1tWb58OZqmodfruXjxIsWLF89Ue9asWcOiRYuwtrbG1taWFStWmOS8qXnnnXfYvXs34eHhODk58e233/L+++/z+eefc/z4cTRNw9nZmfnz55v0urlJTry3tefW8sfRPxjXeBxjG49NtV5EdAStFrfiTNgZ1vVeR4fKHVKtu3mzjI3588PKldCjRy6ZhHH7NsydKwN5WBjUqCG/gb/3npw+o5hOamMx2f1Idwz9BXHq1CkxevToPHPeF0lu/Tzdj7kvyswoI9x/dxdxurhU68XExYh6C+qJfN/nE14XvFKtp9cL8dNPQmiaELVrC3HtWna0OhNCQ+V9q/z5ZeM6dRJixw7ZYCXTSGMMXQV05YWV2c/Thg0bxIYNG0zcmudGbhkptEmaOBR6KM16H2/5WDAJsebsmlTr6HRCDB0q/yX37i1EVJSpW5sJt28L8fHHQtjYyBucAwYIERRk7la9MNIK6GrIRVGSmTFjBgCdOnUy+bkDbgQw+/BshtcbTn3H+qnWW312NbMPz2Z0w9GpjpkLAR99BPPnw7hx8MMPZh5iiYqSd2P/9z+IjoaBA+HLL+X8cCVHqICuKDlEp9cxdNNQXin4ClNaTEm13qXIS7y/4X3qO9bnx7d+NFhHCBg5En7/Hb74wszBXAhYtQo++wxCQ6FrV5g6FapUMVODXl4qoCtKDtl4fiNHbx5lSdclFMlfxGAdnV7Hu2vfxUKzYEWPFQZXfAKMHw9z5sgYOnWqGYP5uXPw8cewYwe89hosXy5XbypmobagU5QcMtNvJs5Fnelds3eqdf45/g+Hrx9mTrs5OBd1Nlhn0SL46ScYOhSmTTNTMI+Lg8mT5arNI0fk/Eh/fxXMzUz10BUlB/hf92fftX383OZnrCwM/7O7F3OP8TvG80a5N3j31XcN1jl4EAYPlsv3Z882UzA/ehQGDZJJsnr3hl9/hZIlzdAQJTnVQ0/kypUrWV4w4+/vj6WlJatXr0445u3tTZUqVXBxceHHHw2PiWZVbGwsQ4YMoXLlylStWpU1zxJ3ZIOZM2dSvXp13NzcaNmyJVevXk1S/uDBAxwdHRkxYkTCseDgYBo0aICrqyu9e/cmNtbMqWDTsHjxYhYvXmzSc/588GcK5yvMoNcGpVpn0u5JRD6OZHa72Qbzsly/LoenHR3lkHWOb5eq08GPP8p8Krdvw7p1cohFBfNcQwV0E9LpdHzxxRe0adMmybGPPvoILy8vzp49y7Jlyzh79qzR53y2bD89U6ZMoWTJkpw/f56zZ8/StGnTDLffWK+99hoBAQGcPHmSHj168PnnnycpnzhxYorrf/HFF4wePZoLFy5QrFgx/vrrr2xrX1aVLVuWsmXLpl/RSCH3Q1h5ZiWDaw+mcL7CBuucDTvLnMNzGFJ7iMHEXPHx0KsXPHoEGzaYYT1OSAi0bCkH77t2hTNn5FJUJVdRAT2ZZ+lz3dzc6NGjB9HR0Ua/dvbs2XTv3p2SiXoshw8fxsXFhYoVK2JjY4OHhwfr169P8zyTJk1iyJAhtG7d2mBeckP+/vtvxo8fD4CFhUW6uV48PT3p1q0bbdu2xdXVNUVQTkvz5s0TUgo3bNiQ0NDQhLIjR45w+/ZtWrdunXBMCMHOnTvp0aMHAP379+e///4z+no5bcWKFaxYscJk55t9eDYAIxuMTLXOKO9RFM5XmO9bfG+w/McfwddXTlHM8awLmzZBrVpyrNzTE1askLnFlVwn146hmyl7bqbT516/fp1169axc+dO/P39E8qvX7+epLfn5OTEoUOH0m3rkSNH2L9/P7a2tgQFBdG7t+Ebabt37074eeLEiezevZtKlSoxZ86cJAm1DDl+/DjHjh0jX758VKlShY8//piyZcvSu3dvgoKCUtT/9NNPU/wH89dff9GuXTsA9Ho9n332GYsXL2bHjh0JdSIiIihatChWTxNtOzk5cf369XR/B+Yyb57cFje133lGPIp9xIIjC+hZoyflihjepWv/tf1su7yNma1n4mCX8j/igAD49lvw8IB3DQ+tZ4/4eJnZa+pUOYNl5UqZtlbJtXJtQDeXsmXL0vjpnfr33nuPX3/9lTFjxqSbRGvUqFH89NNPKXYgEsn3IsS4DYw7d+6M7dNkRVWqVEkzqVZ4eDihoaE0btyYmTNnMnPmTMaMGZPuOHDLli0pUkROn6tevTpXr16lbNmyRvdOlyxZQkBAAHv27AFg7ty5tG/fPsVwRWZ/By+C/wL/4/6T+4yoNyLVOlP3T8XBzoGhdYemKIuOlsm2SpWS6VByTHi4vOG5cycMGQK//CKTxCi5Wq4N6GbKnpvp9LkBAQF4eHgAMsBu2bIFKysrnJycCAl5vuVqaGgoZYxIf1egQIGEn9Prodvb22NnZ0fXrl0B6Nmzp1Fj1M9S7ELSNLvG9NC3b9/OlClT2LNnT8J5/Pz82LdvH3PnzuXRo0fExsZSsGBBpk6dyr1794iPj8fKysro38GLYPnp5ZQvUp7Xy75usPz4reNsubCFKS2mGEyHO348BAbC9u05mP721CmZxvbWLTnE0r9/Dl1YyapcG9DNJbPpc4ODgxN+HjBgAB07dqRLly7Ex8dz4cIFgoODcXR0ZPny5fz777+A3AEISDIbxJD0euggl6nv3r2bFi1asGPHDqpXrw7IVLeHDx9m6tSpab/xRNLroR87doyhQ4fi7e2d5H7B0qVLE3729PQkICAgYVZP8+bNWb16NR4eHixcuJC3337b6PbkVRHREfhc8uHThp+m+o3kx/0/UsimEMPrDU9R5u8vpyaOGCHvR+aI9etlFsTCheXWb/Xq5dCFFVNQN0WTeZY+183NjcjISKPT56bGysqKOXPm0KZNG6pVq0avXr0StqYLDAxMyG2eVT/99BOTJk3Czc2NxYsXJ+QjuXTpUpLNN0xh7NixPHr0iJ49e+Lu7k7nzp2Nat/MmTNxcXEhIiKC999/36Rtyo3WnltLvD4ej5oeBsvPR5xn5ZmVfFTvI4rmT7qhg04Hw4fLoZanW5xmLyFg5kw5c6V6dfm/iQrmeU9qWbuy+6GyLQrRoUMH8eTJk2y9Rp8+fcSdO3ey9Rq5VWY/T2FhYSIsLCzL12+xsIWoPLuy0KeSLvb99e+L/JPzi1sPb6UomzdPZlBcujTLzUhffLwQI0fKC/boIUR0dA5cVMksVLbF3GnTpk3Zfo0lS5Zk+zVeNOlN+TTGzYc32RW8i4lNJhocbrn96DaLTixicO3BlCqYdDZSWJhMUti8udywIls9fiyHWNauhdGjYfp0sFBf3PMqFdAVJRlPT09A3gvJrNVnVyMQqeZt8TzuSZw+jo8bfJyi7Isv5AKi337L5qX9Dx7Im59798pZCJ98ko0XU3KCCuiKkowpAvryM8txK+VG9RLVU5TphZ4FRxfQtHxTqjpUTVJ24oScWDJmDFSrlunLpy88XO7jeeIELF2aA18FlJygvlspioldu38N3xBfPGoYvhm64/IOLt+9zJA6Q1KUjR8vN7x/uug3e1y/Dm++KZfv//efCuYvENVDVxQT2xi0EYDu1bsbLJ9/ZD72tvZ0r5a0fPdu8PKSKXGzbc55SIgcnL99G7y9IRtz/ig5T/XQFcXEvC95U7FYRVyLu6You/XoFuuD1jPAfQD5rJ4v7BJCjp2XLSvnnWeLa9egWTN513XrVhXMX0AqoCeSlfS5gYGBNGrUiHz58jF9+vQkZc7Ozrz66qu4u7tTt27dhOORkZG0atUKV1dXWrVqxd27d7PUfkOuXbtG8+bNee2113Bzc2PLli0mv0Zibdu2pWjRonTs2DHJ8T59+lClShVq1qzJoEGDiIuLA+S02ZEjR+Li4oKbmxtHjx5NeE1OpB02tZj4GHYG76SdSzuDs1v+OfYP8fr4FMMta9fC4cMyZ0u2rLC/elUG8IgI2LYNGjXKhosoZpfafMbsfuTGeejBwcGiRo0amXrt7du3xeHDh8WXX34ppk2blqSsfPnyBuc1jx07VkydOlUIIcTUqVPF559/bvT19Hq90Ol06dYbPHiwmDt3rhBCiDNnzojy5csbfY3M2L59u9iwYYPo0KFDkuObN28Wer1e6PV64eHhkdCmzZs3i7Zt2wq9Xi/8/PxE/fr1hRBCxMfHi4oVK4pLly6JJ0+eCDc3N3HmzJkMtSWzn6eoqCgRFRWVqdduu7RNMAmxMWhjijKdXicqzKogmnk2S3I8Pl6IKlWEqFFD/mxyoaFCVKwoRNGiQvj7Z8MFlJxEGvPQVQ89mcymzy1ZsiT16tXDOgO7Dqxfv57+T/NkGJNS9sqVK1SrVo3hw4dTu3btJDliUqNpGg8ePADg/v37RuVQKViwIF999RW1atWiYcOG3L5924h3I7Vs2ZJChQqlON6+fXs0TUPTNOrXr5+Qcnf9+vX069cPTdNo2LAh9+7d4+bNm5lKO2wqdnZ2CemBM8rrghc2ljY0d26eomzf1X0E3wtmcO3BSY6vXAlBQfDdd5Ast1vW3b4t8waEhYGPDyT6hqi8eHLtTdFR3qM4fsu0+XPdX3FnVtu0s35lNn1uWjRNo3Xr1miaxtChQxkyRH7dvn37NqVLlwagdOnS3LlzJ933EBQUxD///MPcp6n30kukNWnSJFq3bs3s2bOJiopi+/bt6V4jKiqKhg0bMmXKFD7//HP++OMPJkyYwNKlS5k2bVqK+i4uLkl2aEpLXFwcixcv5pdffgEMpxe+fv16ptMOm8Kz3+3w4Snzq6TH66IXTcs3pYBNgRRl/576lwLWBXi7yvM8Nno9TJkic5ybfL+IiAho1UreCPX2hvr1TXwBJbfJtQHdXDKbPjctBw4coEyZMty5c4dWrVpRtWpVmjRpkqlzlS9fnoYNGyY8Ty+R1rJlyxgwYACfffYZfn5+9O3bl9OnT2ORxmpAGxubhDHwOnXqsG3bNkCOg/fp0ydT7X5m+PDhNGnShDfffBNIPbVuasdzwsqVK4GMB/Sr965yLvxcih44QKwullVnV9Glapckwf6//+TswWXLTLxA89EjaN8ezp+XG1Q8/X0rL7ZcG9DT60lnl8ymz03Ls2GOkiVL0rVrVw4fPkyTJk0oVaoUN2/epHTp0ty8eTNJ5sLUJE6rC+n30P/66y+8vb0BaNSoETExMYSHh6d5LWtr64T3nTitblZ76N9++y1hYWHMnz8/4Vhq6YVjY2MzlXbYnLwvyt9zW5e2Kcp8LvpwN+Zuks2fhZCJt1xdoWdPEzbkyRPo1k3ujLF2Lbz1lglPruRmuTagm0tm0+emJioqCr1eT6FChYiKimLr1q18/fXXgNzEYuHChYwbNy5JStnr16/Tr1+/JLv+pCa9Hnq5cuXYsWMHAwYM4Ny5c8TExFCiRAkAqlatSmBgoNHvJSs99D///BMfHx927NiR5NtB586dmTNnDh4eHhw6dIgiRYpQunRpSpQokWra4dzK66IX5YuUT7H6E+Df0/9ib2tPq4qtEo5t2QLHjsE//5hw7FynkztibNsGf/8NL0GaYiWR1O6WJn4AbYEg4CIwzkB5EWAjcAI4AwxM75y5dZZLtWrVxNChQ8Wrr74qunXrZvRsh5s3bwpHR0dRqFAhUaRIEeHo6Cju378vLl26JNzc3ISbm5uoXr26mDx5csJrwsPDRYsWLYSLi4to0aKFiIiIEEII4e/vL1q3bm2wfRmdhXPmzBnx+uuvCzc3N1GrVi3h4+MjhJAZBStXrmzwNQUKFEj4edWqVaJ///5GX++NN94QDg4OIn/+/MLR0VF4e3sLIYSwtLQUFStWFLVq1RK1atUS3377rRBCztYZPny4qFixoqhZs6bwTzQLY/PmzcLV1VVUrFgxye/NWJn9PDVt2lQ0bdo0Q695Ev9EFPyhoPhw44cpyh4+eShsJ9uKYZuGJRzT64Vo0EAIZ2chYmMz1cyU9HohPvpIZk2cPt1EJ1VyG9KY5WJMMLcELgEVAZunQbt6sjpfAj89/bkEEAnYpHXe3BjQc4vZs2eL9evXZ+s1Nm7cKH755ZdsvYa55WRA3xW8SzAJ8d+5/1KULTmxRDAJse/qvoRje/bIf31PZ2+axk8/yZOOGWPCkyq5TVoB3Zghl/rARSHEZQBN05YDbwNnE3f0gUKaHHgt+DSgx2fmG4OS/g5GppB84Y/yXOKNt421M3gnFpoFzSuknK649NTSFNvQzZwJ9vYm3N3t33/lUlMPD/jpJxOdVMlrjLmv7ggknvAc+vRYYnOAasAN4BTwiRBCn/xEmqYN0TQtQNO0gLCwsEw2WVFynz1X91C7dG0K50u6O1R4dDhbL23lnZrvYKHJf24XLsCGDXJHokxOd09q1y4YMECuBPX0VPnMX2LG/M0bmiuWfE5ZG+A4UAZwB+ZompZi3zMhxAIhRF0hRN1nN+YUJbeZPn16ivQNaXkc95iDoQdpVr5ZirL1gevRCR29avRKODZrFlhby4CeZUFBckaLq6ucA5lo42/l5WNMQA8FyiZ67oTsiSc2EFj7dIjnIhAMpLzVryh5wKZNmzK0m9Sh64eI1cXS1Dllsqt1getwLuqM+yvuAERGylktffrAK69ksaHh4dChA9jYwObNMu+u8lIzJqD7A66aplXQNM0G8AA2JKtzDWgJoGlaKaAKcNmUDVWU3Gr3ld1oaLxR7o0kxx88ecC2y9voWrVrwrz+33+Xu759+mkWL/rkCXTtCqGhsmfu7JzFEyovgnRvigoh4jVNGwH4IGe8/C2EOKNp2odPy38Hvgc8NU07hRyi+UIIEZ6N7VaUXGPP1T28Vvo1iuZP2kP2uuBFrC6WbtW6ARAbC3PmQOvWcql/pgkBQ4bA/v2wfLnKnKgkMOruiRBiixCishCikhBiytNjvz8N5gghbgghWgshXhVC1BRC5MmdibOSPnfatGm4u7vj7u5OzZo1sbS0JDIyEkg9DWxOpM/du3cvtWvXxsrKKsVqzoULF+Lq6oqrqysLFy40+bUTe/PNNxN+P2XKlKHL08QlIhPpc3Pi92asmPgY/EL8aFre8HBLyQIlaeQkA+6aNXDzJowalcWLzpwJixbBpEnQ2/CepcpLKrX5jNn9yI3z0LOSPjexDRs2iObNmwsh0k4DmxPpc4ODg8WJEydE3759xapVqxKOR0REiAoVKoiIiAgRGRkpKlSoICIjIzPyNjOtW7duYuHChUKIzKXPNfb3ltnPU9u2bUXbtm2Nqrvnyh7BJMT6wKTrBh7HPRYFfygoBm8YnHCscWMhXFyEMOKvLXVbtghhYSFEjx5ZPJGSV6HS5xovs+lzE1u2bBnvPN2nMa00sDmRPtfZ2Rk3N7cUybh8fHxo1aoVxYsXp1ixYrRq1Soh50tqBgwYwMiRI3n99depWLGi0RkWE3v48CE7d+5M6KFnJn1uRn9vGeXl5YWXl5dRdZ+Nn79ZLmnyqx2Xd/Ao9lHCcMuJE3DgAAwbloVZhYGBcp65m5uanqgYlGtzueTV9LnR0dF4e3szZ84cwHB62GdpYHMifW5qUktbm56bN2+yf/9+AgMD6dy5Mz169ODhw4cJ2ROT+/fff6levXrC83Xr1tGyZUsKFy6cZjtM/XvLLnuu7qHWK7UoZpt0E9B1gesonK8wLSq0AOC338DWFgYOzOSF7t+XeVny5YP166FAyvS8ipJrA7q5ZDV97saNG2ncuDHFixcHUk8Pm1kZTZ+bmsy2q0uXLlhYWFC9evWEjS8KFSrE8ePG/ee7bNkyPvjgg3TbYerfW0Z8//33AEycODHNek/in+Ab4svQOkOTHI/Xx7M+aD0dXDtgY2nDvXuwdCm8+24mN3/W62XCrcuXYccOKFcuEydRXga5NqDn1fS5y5cvTxhugdTTwwI5kj43NU5OTkmWuIeGhtKsWbN0r58vX+KNjWm1dqMAACAASURBVGXQNbaHHhERweHDh1m3bl2SdmQ0fW5mfm8Z8SzLZXoB3f+GPzHxMSluiPqG+BIeHU7Xql0BOToSHQ0ffZTJBk2eDBs3wq+/Qibz6CsvidQG17P7kVtvigLC19dXCCHEBx98IKZnIGvdvXv3RLFixcSjR48SjsXFxYkKFSqIy5cvJ9zcO336tBBCiDFjxiS5uTd27FghhBChoaGiRYsWBtuX2Zu2/fv3T3FT1NnZWURGRorIyEjh7OyckO1x3LhxYu3atemeI3FWRmPMmzdP9OvXL8mxTZs2JbkpWq9ePSFE5n5vyWV3cq4f9v4gmIQIi0q6X+zYrWOF9XfW4kHMA6HTCeHqKkSjRplqihAbN8qEW/36yWyKyksPdVPUeNWqVWPhwoW4ubkRGRnJsGHDjH7tunXraN26dZJetJWVFXPmzKFNmzZUq1aNXr16UaNGDQDGjRvHtm3bcHV1Zdu2bQk9/Zs3b2JlZZovT/7+/jg5ObFq1SqGDh2acO3ixYszceJE6tWrR7169fj6668TholOnTrFK1lexphS8m8vIPcarVixIi4uLgwePDjh3kBmfm85zTfUl6oOVXGwc0hyfNP5TTR1bkqhfIXYtUvmbsnUMv+LF+G996B2bbkiKYeGnJQ8LLVIn92P3NhDzy1yIn1uWgzlYs+LsrOHrtfrRfGfiotB/w1KcvxixEXBJMQsv1lCCCF69RKieHEhHj/OYCOiooRwcxOiWDEhgoMz+GLlRUYW0+cqOSwn0uemxcfHx6zXNzd7e/t06wRFBBH5ODJJSlyAzRc2A9CxckfCwmDdOjl2nj9/BhoghJzfeOqUzNGilvUrRlIBXVGSWbNmTbp1fEN8AQwG9KoOValUvBLTp0NcHAxOuWd02v74Q64E/eYbaNcugy9WXma5bgxdGJiupigZld2fI98QX4rbFqeKQ5WEYw+fPGT3ld10dO2IEDIuN24Miabhp+/oUfj4Y2jTBp7uPasoxspVAT1//vxERESooK5kiRCCiIgI8mdonOO58ePHM378+DTr+Ib40sipUcKmFQDbL28nVhdLh8od2LsXzp+XObSMdv8+9OwJJUvCkiVqJaiSYblqyMXJyYnQ0FDUbkZKVuXPnx8nJ6dMvdbPzy/N8sjHkZwLP8d7bu8lOb7p/CaK5CtC47KNGfAVFCkCPXoYeVEhYNAguHYN9uwBB4f0X6MoyeSqgG5tbU2FChXM3QxFSdPB0INA0vFzvdCz+cJm2rq05cE9a9askWPnRm8x9+uvsHYtTJ8Or7+efn1FMUB9p1OUDPIN8cVSs6RemXoJx47cOMLtqNt0rNyRpUvl/hOJMhykzd8fxo6Fzp1NsPOF8jLLVT10RckLDoQc4LXSr1HA5vkCMq+LXmhotKnUhtb9oU4dqFXLiJPdvy9zmpcuLfemU4uHlCxQAV1Rkklr7D1OF8fh64f54LWk3W+vi17Uc6xH6PkSHD8usyum69nOQ9euwb598HSlrqJklgroipLMkiWpb7h18vZJouOik4yfR0RHcPj6YSa8OYF//pEZbpNlODBswQJYuRJ+/FFtI6eYhAroipIBzxYUNSr7PABvu7wNvdDTolw7ui6Rezenmyb39Gm5F12bNnL8XFFMQN0UVZRkRo0axahUNv48eP0gjoUcKVfkeU5y74veFLctzs2Aety9K2cfpik6Wo6bFykiV4Sq+eaKiageuqIkk9ZmHQdDD9LQ6fkGI3qhx/uiN60rtWbhn5aULQstWqRzgU8/hbNnYetWuYhIUUxEdQ0UxUhhUWFcvns5SUA/cesEt6NuU794W3x8YMAAsLRM4yRr1sD8+fDFF9CqVba3WXm5qICuKEY6dF3uadrAsUHCMa+LcjPpML+2CCEDeqquXZOT0+vXh6fb3CmKKamArihGOhh6EEvNkjpl6iQc877oTe3StVmzsBRNmkDFiqm8WKeTm1XodPDvv2BtnTONVl4qKqArSjKVK1emcuXKKY4fDD1IrVdqYWct1/Pfi7mHb4gvr9q25fx56N8/jZNOnSrnmv/2G1SqlE0tV1526qaooiSzYMGCFMd0eh2Hrx+mr1vfhGM7Lu9AJ3TcPdwOOzuZKNEgPz+YNAnefVf20hUlm6geuqIY4Vz4OR7GPkxyQ9T7ojeFbQqz598GdOsGhQoZeOGDB9CnD5QtC3PnqqX9SrZSPXRFSWbI0yTmiXvqh0Kf3hB1kjdEhRD4XPKhSr6W+Edapz7cMmIEXL0Ke/fKeeeKko1UQFeUZM6fP5/i2MHQgxTLXwzX4q4ABIYHEvIghMLBX1G2LDRvbuBEy5fD4sVyK7nGjbO51YqihlwUxSgHr8sFRdrTIROfS3Ij7bMb2tC3r4G559euwYcfQsOGMGFCDrdWeVkZFdA1TWuraVqQpmkXNU0bl0qdZpqmHdc07YymaXtM20xFMZ8HTx5w5s6ZJOPnPpd8KKFVRtx1pl+/ZC/Q6aBvX/nnkiVgpb4IKzkj3U+apmmWwG9AKyAU8Nc0bYMQ4myiOkWBuUBbIcQ1TdPUemblheF/3R+BSAjoMfEx7LmyB9vzH9CwIVSpkuwF06bJMfN//lFTFJUcZUzXoT5wUQhxGUDTtOXA28DZRHXeBdYKIa4BCCHumLqhipJT3N3dkzx/tkK0vmN9APZd3cfj+Mc8PtyGfqOTvfjoUfj6a7mZaJoT0xXF9IwJ6I5ASKLnoUCDZHUqA9aapu0GCgG/CCEWJT+RpmlDgCEA5cqVS16sKLnCrFmzkjw/GHqQKvZVKJq/KCCHWyyFDRY3mtG7d6KK0dFyimKJEvD772qKopLjjBlDN/SpFMmeWwF1gA5AG2CipmkpltoJIRYIIeoKIeqWKFEiw41VlJwmhEiRYdH7og8W19+gc9sCSTcZ+uILCAwET0+wt8/xtiqKMQE9FCib6LkTcMNAHW8hRJQQIhzYCxizo6Ki5Drvvfce7z1d0Xnl3hXCosMSAvr1B9c5E3aauHNtkt4M9faGOXPkphUqi6JiJsYEdH/AVdO0Cpqm2QAewIZkddYDb2qaZqVpmh1ySOacaZuqKDkjNDSU0NBQQA63wPMMi1svbQWgaHgb2rZ9+oLwcBg4EGrUkDlbFMVM0h1DF0LEa5o2AvABLIG/hRBnNE378Gn570KIc5qmeQMnAT3wpxDidHY2XFFywqHrh7C1suXVUq8CsPGcDzx6hfdau2Fjg9zoeehQiIiQvfT8+c3bYOWlZtQEWSHEFmBLsmO/J3s+DZhmuqYpivkdDD1IPcd6WFlYodPr2HpxG1zsyIAJT28tLVwIa9fCTz9BLTXKqJiXWimqKKl4Ev+EY7eOJQy3HLl5hCgRSZnHbahdGwgOhpEjoUkT+Owz8zZWUVC5XBQlhUaNGgFw/NZxYnWxCTdElx32AaExqFkrNL2OhLuiixals++couQMFdAVJZmpT29s/nroV+D5DdE1J3zgRh2Gfl0Cpv8E+/fLIZfy5c3WVkVJTA25KEoqDoYexKmwE46FHbn3+D4h4iDOujY4hR+HiRPlatC+fdM/kaLkENVDV5RkunfvDsDxpscTeue/ee0ACx196raQuw45OKjVoEquo3roipJMREQEt+7c4vLdywnj50v8fOBJIcac9YIzZ+Dvv9VqUCXXUT10RTHgwZMHgBw/j44WBOl8KPukMUXnTIfhw3m+qkhRcg/VQ1cUAx7EPsBSs6ROmTrMX30eUeQq3U/egcqVZXpcRcmFVA9dUQx48OQBbqXcsLO2449dPuAMI06dAG8/sLMzd/MUxSDVQ1eUZJq3aE60UzSvl32dmzfhXJwXxSPsqfTJ11CvnrmbpyipUgFdUZLpMqQLsW/E0sipEZ5zb4HzbjrcLQZffmnupilKmlRAV5Rk/EL9AGhYpgHz1/mAdQzvfPCF2htUyfXUJ1RRkvl+yPfYRNtw99FhrpY9hrXehmav9zF3sxQlXXkyoDdr1izFsV69ejF8+HCio6Np3759ivIBAwYwYMAAwsPD6dGjR4ryYcOG0bt3b0JCQuhrYPXfZ599RqdOnQgKCmLo0KEpyidMmMBbb73F8ePHGTVqVIryH374gddffx1fX1++NPDVfdasWbi7u7N9+3YmT56conz+/PlUqVKFjRs3MmPGjBTlixcvpmzZsqxYsYJ58+alKF+9ejUODg54enri6emZonzLli3Y2dkxd+5cVq5cmaJ89+7dAEyfPp1NmzYlKbO1tcXLywuA77//nh07diQpt7e3Z82aNQCMHz8ePz+/JOVOTk4sWbIEgFGjRnH8+PEk5ZUrV2bBggUADBkyhPPnzycpd3d3T9g27r333kvIZf5Mo0aNEpbzd+/enYiIiCTlLVu2ZOLEiQC0a9eOm+duYqlZ0GX6JCgTSvnCztha2wLqs6c+e6b57D17T6aWJwO6omSXWF0sQgis4vXcsioM1o+pXrKquZulKEbRhEi+PWjOqFu3rggICDDLtRUlNRuDNtK5TWcq3CpM8Ks/QsfhBH4USBWHKuZumqIAoGnaESFEXUNlqoeuKIn4+a4A4L5WCZuaXjgWrUBl+xT7nStKrqRmuSjKM/fv43doNWUcrbkb1xNRfidtXdqiqQRcSh6hArqiPBX/8Ucctn9CxSbvIso1IE6Lop1LO3M3S1GMpgK6ogCsWMHJ7UuJtoErJ9rwypte2Fja0LxCc3O3TFGMpsbQFSUkBD78EL9W5YGrhG79Beuyp2j6VhMK2hQ0d+sUxWiqh6683PR66N8f4uLw61ALO11pNAtBnIimg2sHc7dOUTJEBXTl5TZzJuzaBb/8wr7IE8RdakyBEpEAdKzc0cyNU5SMUQFdeXkdOyYTbnXtSkj3Vlx7cJW4S2+AbQS21ra4FHcxdwsVJUNUQFdeTtHR8O67UKIE/PEH+0MOAFBSV4co3T3sbdX2ckreowK68nIaMwYCA2HhQrC3Z8uZ/fCkEM063kFUF3Tr0c3cLVSUDFOzXJSXz8aNMG+eDOpvvQXA1sB9ENIIi/ZeFLIpxLTP1TZzSt6jeujKy+XmTRg0CNzd4WlmwfBHd7nDaVzzvcHem1toWbYl8U/izdxQRck4FdCVl4deD/36QVQULFsG+fIBMGeDL2iClg1e4cbDG5ybdc5gGlxFye1UQFdeHjNnwvbt8MsvUPV5Stxl+/eDzpriztfQ0ChuW9yMjVSUzFMBXXk5HDkipyh27w4ffJBw+NYtOB+7j9LUYdsVH+o51sPG0saMDVWUzDMqoGua1lbTtCBN0y5qmjYujXr1NE3TaZqWclsWRTGXhw/hnXegVClYsAASZU/80zMGSvvTtLI7/jf86VKlixkbqihZk25A1zTNEvgNaAdUB97RNK16KvV+AnxM3UhFyZIRI+DSJVi6FIo/H07R6+H3Df5gFUvRovLY21XfNlMjFSXrjJm2WB+4KIS4DKBp2nLgbeBssnofA2uAeiZtoaJkxeLFsGgRTJoETZokKdqxA65b7gcgMCIQ1+KuVHOoxoABA3K+nYpiAsYEdEcgJNHzUKBB4gqapjkCXYEWpBHQNU0bAgwBKFeuXEbbqigZc+ECDBsmA/mECSmKf/8drF32UbF4VQ5cO8AnDT5B0zQV0JU8y5gxdEPbtSTfiHQW8IUQQpfWiYQQC4QQdYUQdUuUKGFsGxUl4548AQ8POTVx6VKwtExSfOMG/LcxDsrvo1zRssTp4+hSVY6fh4eHEx4ebo5WK0qWGNNDDwXKJnruBNxIVqcusPzpVl0OQHtN0+KFEP+ZpJWKklFjx8LRo7B+PTg5pSj+6y/QlzqCXntEVGwUJQuUpKFTQwB69JD39Hfv3p2TLVaULDOmh+4PuGqaVkHTNBvAA9iQuIIQooIQwlkI4QysBoarYK6Yzbp1MHs2jBoFnTunKI6Pl5NdXFrtBODE7RN0qtwJSwvLFHUVJS9JN6ALIeKBEcjZK+eAlUKIM5qmfahp2ofZ3UBFyZArV+TS/rp14aefDFbZsgVCQ8Guxk4qFqtIVFxUwnCLouRlRiXnEkJsAbYkO/Z7KnUHZL1ZipIJsbFy3FyvhxUrwMbwAqF586C00xPOxxzA1c6VAtYFaFmhZQ43VlFMT60UVV4c48bBoUNygLxiRYNVLlwAb29o88FBYuJjCHkQQluXttha2+ZwYxXF9FT6XOXFsG4d/PwzjBwJPVJfqPzbb2BtDcVr78TimAX3Yu7Rs3rPJHWGDRuW3a1VlGyhCZF8BmLOqFu3rggICDDLtZUXzKVLUKcOVKkC+/alOtTy8CE4Osr7pFdbvsn5iPM8ePKAsLFhFLQpmMONVpTM0TTtiBCirqEyNeSi5G0xMdCrF1hYpDluDnJzoocPYfDwKA6FHiIqNooOrh1SBPOQkBBCQkJSOYui5F5qyEXJ20aOlPPNN2wAZ+dUq+n1MGcO1K8PT0odIE4fR5w+LsVwC0Dfvn0BNQ9dyXtUQFfyrn/+gT/+gPHjoVOnNKtu2wZBQTK1y87gnVhoFthY2NChcoccaqyiZD815KLkTcePw/Dh0KIFfPddutVnz5bZc3v2hB3BO7DULOlQOeVwi6LkZSqgK3nPvXtyowp7e7mVnFXaXzQDA2HzZvjwQ3gQH8aRG0dSHW5RlLxMDbkoeYteD336QEgI7N4NJUum+5KZMyF/ftmh33ppKwJBPst8arhFeeGogK7kLZMmybX7c+fC66+nW/32bZkOfcAAGfs379uMhmZwdsszn332mWnbrCg5RAV0Je9Yvx6+/x4GDpTjJ0b47TeZSXf0aNALPZsvbEYg6Furb6qv6ZTODVZFya3UGLqSNwQGQt++MunW3LlJ9gVNTXS0rNq5s1xzFHAjgAdPHlDAugDtXdun+rqgoCCCgoJM2XpFyRGqh67kfvfuwdtvy4HwNWvkn0ZYuBAiImDMGPn8v0CZ0blH9R7YWKa+AGno0KGAmoeu5D0qoCu5m04H77wDwcFyE1Ajty7U6eTN0Pr14Y035LEVZ1YAMKTOkOxqraKYlQroSu42frxMjzh/Prz5ptEvW7UKLl6E1avl6Ex4dDiX716mWP5iNHJqlI0NVhTzUWPoSu61ZAlMmyY3eh5ifK9ar4cpU6BaNejaVR5bdnoZAN2rdUczYvxdUfIi1UNXcidfX3j/fWjWDH75JUMv3bgRTp+Wy/wtnnZZPI95AvBZIzUlUXlxqYCu5D5XrkCXLnK8fPVqmcDcSELA5MlyfwsPD3ksXhfPidsnsLe1p2qJqumeY8KECZlsuKKYlwroSu7y8KFMtBUbK7va9vYZevnWrRAQIDeBfpYRYMHRBeiEjm7Vuhl1jrfeeiujrVaUXEEFdCX3iI+Xuc3PnQMvL6iafm86uSlTwMkJ+vV7fuy3w78B8G2zb406x/HjxwFwd3fP8PUVxZxUQFdyByFgxIjnM1patcrwKXbskBsW/for5Msnj919fJdz4ecoX6Q8pQuVNuo8o0aNAtQ8dCXvUbNclNxh+nQZyL/4IkMzWp4RAr76SvbOBw9+fnzq/qkIBAPdB5qwsYqSO6keumJ+K1fC55/L4ZYffsjUKTZtgkOH5Nj5s4WkQgg8j3uioTGywUgTNlhRcifVQ1fMa88emaOlcWPw9Hw+zzAD9HqYOBEqVZJZFZ/xv+5PWHQY1UpUo5htMZM1WVFyK9VDV8zn1CmZo6VSJbknqK1tpk6zejWcOCHXISWe4fjj/h8BGFp7qClaqyi5ngroinmEhEC7dlCggLwRWrx4pk4THw9ffw01ajyfdw5wL+Yemy5sQkPD41WP1E9gwA+ZHPZRFHNTAV3JeeHh0Lq1nHO+b5/RCbcM+esvufnz2rVgafn8+B9H/iBOH0ed0nUoWSD9XY0Se92IjTMUJTdSAV3JWQ8eyJ75lSvg4wNublk61cSJMmdXly7Pj8fp4pjhNwMgU7NbfH19ARXYlbxHBXQl58TEyDHz48fhv/+gSZMsnW7qVAgLkzvSJc63tebcGm5H3cbKwgqPmhkbbgH48ssvATUPXcl7VEBXckZcnJyWuHu3vHvZIWsbNF+5Aj///HwTo2eEEEz3nY6lZkmnyp2wt8tY6gBFycuMmiOmaVpbTdOCNE27qGnaOAPlfTRNO/n04atpWi3TN1XJs+LjoU8fmZvlt9/kz1k0fryc4ThlStLjviG+HLl5BJ3QMcB9QJavoyh5SboBXdM0S+A3oB1QHXhH07TqyaoFA02FEG7A98ACUzdUyaP0ehg0SO44MWMGDB+e5VMeOADLl8ut5cqWTVo28+BMrC2scbBzoJ1LuyxfS1HyEmN66PWBi0KIy0KIWGA58HbiCkIIXyHE3adPDwJOpm2mkifp9fDhhzIx+eTJ8OmnWT5lXJzc76JsWbm4NLHzEedZd24deqGnz6t9sLY0Pu2uorwIjBlDdwRCEj0PBRqkUf99wMtQgaZpQ4AhAOWyMFVNyQP0ehg6FP78EyZMkIlWTODXX+V6pHXroGDBpGWT907GysKKOH0c/Wv1z/Q1Zs2alcVWKop5GBPQDe3XJQxW1LTmyID+hqFyIcQCng7H1K1b1+A5lBeAXi8TbP31l5xX+K1xaWvTExIC33wDHTvKyTKJnY84z9JTSylVoBQOdg64v5L51Lcqba6SVxkz5BIKJB6pdAJuJK+kaZob8CfwthAiwjTNU/IcnQ4++EAG82++ge++SzqnMAtGjZL/V8yenfKUk/dOxtrCmpuPbjLAfUCW9g3dvn0727dvz2JrFSXnGdND9wdcNU2rAFwHPIB3E1fQNK0csBboK4Q4b/JWKnlDXJycR7hiBUyaJAO6iWzcKFeD/vADODsnLXvWO6/uUJ3L9y5neXbL5MmTAbVzkZL3pBvQhRDxmqaNAHwAS+BvIcQZTdM+fFr+O/A1YA/MfdozihdC1E3tnMoLKCZGzjPfuBGmTZNTUEwkMlKO4Li5wWcG9niesm8KNhY2XIi8wED3gRS3zVxeGEXJ64xaWCSE2AJsSXbs90Q/fwB8YNqmKXnGw4fQtavcMmjuXDkNxYRGjpTpX7y8wMYmadm5sHMsObmERk6NOBByQOU9V15qaqWokjV37kD79nI5/6JFcsjFhNatg6VL5X1VQ/cqx2wbQwHrAlyMvEibSm2oVqKaSa+vKHmJ2uBCybzgYLkxxdmzsH69yYN5eLicxv7aa3JlaHJbL21ly4UtdKrcidtRt/mkwScmvb6i5DWqh65kzpEjcv7gkydyqKVRI5OeXq+Xuw/duwfbtiXduAIgXh/Ppz6fUqlYJS5EXqCyfWXauLQxybXnz59vkvMoSk5TPXQl4zZulJkS8+WD/ftNHsxBJt7avFlmCzCUYfePI39wJuwM/Wr1w/+GPyPrj8RCM83HuUqVKlSpUsUk51KUnKQJYZ71PXXr1hUBAQFmubaSBXPmwCefQO3aMrC/8orJL3HoELzxBnTqBGvWpJxzfi/mHq6zXanuUB290BN8L5iLIy+S3yq/Sa6/ceNGADp16mSS8ymKKWmadiS1WYSqh64YJzZWzl75+GMZaXfvzpZgfvcu9O4NTk5ybZKh9UFjt44l8nEkvWr0Yn/Ifr568yuTBXOAGTNmMGPGDJOdT1FyihpDV9IXFgY9e8KePfDFFzJnbeL93kxEp4N334Xr1+VITrFiKevsuLyDP4/9ydhGY/E84Un5IuV5v/b7Jm+LouRFqoeupO3oUahfHw4elBtT/PhjtgRzkNkTvb1lyvQGBtK/RcVGMXjjYFyLu1LPsR4BNwL4puk32FjapKysKC8h1UNXUvf33zJ/eYkSsHevDOzZxNMTZs6EESPkqlBDJuycQPC9YHb138Un3p/gWtyVvrVMO1VSUfIyFdCVlB4/lssz//wTWraEZctkUM8mBw7ITLtvvSVntxiy/9p+fjn0C8PrDif4bjAnb59kabelWFmoj7CiPKP+NShJnTsn70qeOiVX83z/fbYNsYC8TMeOMuHWihVgZeATGR4djsdqDyoWq8iY18dQ7496vFHujUxtAG2MxYsXZ8t5FSW7qYCuSELIIZaPP5Y7R3h5Qdu22XrJy5ehdWsoUAC2boXiBnJq6YWevuv6Eh4djt/7fny/93vuP7nP7x1+N9m88+TKJt/XTlHyCBXQFTmLZehQmTilZUu5ZVzp0tl6yVu3oFUrORty3z4oX95wvf8d+B/eF72Z12EeD2Mf8s/xfxjXeBw1StbItratWLECgN69e2fbNRQlO6iA/rLbsAEGD5Zr7P/3P5mf1iJ7Jz9dvy7Hy2/fllkDqiffcvypXcG7mLBzAr1r9Gag+0Bem/8azkWdmdh0Yra2b968eYAK6EreowL6yyoiAkaPlr3xWrVg+3Z49dVsv+zVq9CihUzSuGWL4emJAGfunKHriq5UcajCgk4LmLhrIufCz7HpnU3YWdtlezsVJS9S89BfNkLIu4/VqsnZKxMmyLX2ORDML16UKWAiI+X/H02aGK534+EN2i1th521HV59vNh7dS/TfKcxvO5wOlTukO3tVJS8SvXQXyYXLsjpiN7eUK+ejKqGMl9lA19fubGzELBzp0yJa8jDJw/p8G8H7sbcZe+AvQD0/68/r73yGjPaqOX4ipIW1UN/GURHw8SJULOmnPQ9axb4+eVYMF+2TA6zFCsmA3tqwfx+zH3aLm3LqdunWNVzFTVL1sRjtQdxujhW9lxp0nwtivIiUj30F5leL7f7+fJLCA2FPn3kfp/ZPIPlGZ1O7jT0/fdyeGXtWrC3N1z37uO7tFnShmO3jrGixwraVGrD+xvexy/UjxU9VuBS3CVH2gywevXqHLuWopiSCugvqt27YexYCAiAOnVkYE9t0Dob3Lwp///YtQsGDoTff0+5H+gz4dHhtFrcirNhZ1nbay0dK3dk7Lax/HP8H75p+g29avTKsXYDODg45Oj1FMVU1JDLi+bQITknsHlzOdl78WI4oXU17gAAC8RJREFUfDhHg/nWrXL/z4MH4Z9/5Hql1IL56Tunqf9HfQLDA1nvsZ5OVTrxvwP/Y4bfDEbUG8E3Tb/JsXY/4+npiaenZ45fV1GySgX0F4Wfn1xD37AhnDwpM11duADvvZft88qfuX9fTmlv0wYcHMDfX24jl5r1getp9FcjHsc/Zlf/XbR1acvPfj8zbsc43qn5Dr+0+wXNUEL0bKYCupJXqYCelwkhu8MtWsDrr8su8ZQpck396NGQP2duIgoh94iuXl32xj//XI701EhlMWesLpYJOyfQZUUXqjlUI2BwAPXK1GOU9yg+3fop3at1x7OLZ7Yt7VeUF5UaQ8+LHj+WuclnzYKzZ+VNzpkzZd7ZAgVytCmnTsGnnz5fl7R+PdQ1uDmWdPrOafqt68exW8cY6D6Q39r/BkCv1b1Ye24toxqMYnrr6VhaZF9CMEV5UamAnpcEBsKCBTJ5+N27coWnpyd4eMgNm3NQcDD88IPskRctCr/+Ch9+CNbWhuvHxMcww3cG3+39jiL5irCu9zq6VO3CiVsn6LO2D2fDzvJzm58Z1XBUjr4PRXmRqICe292/D6tXw8KFMouVlRV07So3nmja1PCmm9nowgUZyBcvlll1P/4Yvv7acKZEACEEq8+u5vPtn3Pl3hW6VevGvA7zcLBzYLrvdL7a+RXFbYvj1ceLNi5tcvS9KMqLRgX03Cg6WqavXbVKjmHExEDlyjB1qpwDWKpUjjZHrwcfH5gzRzYrXz746CM5K9LJKZXXCD2bzm/ih30/cOj6IV4t+Srb+26nZcWW7L26lzFbx+B/w59u1boxv+N8HOxyz1TBLVu2mLsJipIpKqDnFuHhMlpu3AibN8ugXqIEDBoE/frJ7d9yuDd+/rycvr5kibzP+sorcsHpsGHyZ0OiYqNYeWYlM/xmcCbsDM5FnVnQcQGDXhtEYHggXZZ3YX3QehwLObKk6xLeffVds8xkSYudnUr+peRNKqCbS1ycnDO+fTts2yZnqOj1MlL26wc9e8q544a28MkmQsibnBs3wn//yZkqmiYn0UyZAt26GZ5Prhd6/EL88DzuyfIzy3kU+4iaJWuypOsSetboybZL22i3tB3bLm+joE1BprSYwqiGo3Jt1sS5c+cCMHz4cDO3RFEyRhNCmOXCdevWFQEBAWa5tlncvy8nZu/fLx8HD0JUlJwjXqcOtG8v55HXrp1j88ZBZgTYvVuu6NyxQ6a3BZm7q3dveb/V0THl6x4+ecjeq3vZELSB9UHruR11GztrO3rV6MUg90Hkt8rPqrOrWHlmJVfvX6VMoTJ8VO8jhtQZkquGVwxp1qwZALt37zZrOxTFEE3TjgghDM4lUz10UxNCrtA8dUou8Dl+XAby8+dluabJ2SkDB8qub7NmMmtVDjTr5k3ZrGdNOnRIBnSQTWjaVGbT7dAhaboXIQTX7l8j4EYA/jf82XN1D/7X/dEJHQWsC9DOpR1NyjfB1toW3xBf+qztQ8iDEKwsrGhdqTVTW06le/Xu2FimslxUURSTMCqga5rWFvgFsAT+FEL8mKxce1reHogGBgghjpq4rblHfLyMjteuyfl7ly/LZN9BQfJx//7zuo6OcmJ2v36y29ugARQpki3Nio2FGzdks65dgytX5KyUCxdksyIjn9etVAnefFMOzTdrJhMvPop7wJV7V/C/d4XLVy8TGB7I2bCznA07S8TjCACsNCuqlahGe5f2FMxXkLuP77L32l5Wn5MJrYrbFqf5/9u7n9i4riqO49/f/E/iiR2wG5zYJk1akrBIo4bSivKnBSHqbqpIXaAgKqpKFSogxKqIBVFhQzYIIf5EUalQJUQXUEEQBYTUQqlKoCC1cUtFFQotUUnscR3c2sat48PiTtznsad+dt78e3M+0tXMm3fHPsczOvN85753d93M0Y8c5fD+w7xjU53pL865xK1Z0CVlge8CHwfOAk9JOmlmf4t0GwWurrbrge9Xb9vXwkKYPTI7G4Y+Zmbg9ddhejq0qalQAScnw5qb58+Hdu5caIuLy3/ejh2wbx8cORJuDxwIZ9rUu7wg4aj54sUwnD4/Hwry/Hw4b+hSuxTWpdAuXAifF1NTIbTKpDFeeZNzE28wNT0PuXnIzUHuf5CfpX/HDIMjMxy87jX6hy5QvuJVin2TvLY4wcTsBD+arfDNx85T+WWFuYW5ZfEVs0W2FrdSzBXZvmU70/PTzC3MMTY+xtj4GFll2T+wn9GrRjk0eIgbR27k4LsO+hmezrVInCP09wNnzOxFAEkPAbcB0YJ+G/CghQH5U5L6JA2a2X+SDvgzX7uHB9/8wQaeucHvCorAkGDZ9Lzqn03Rn1oBngitAjxabWuGYqC6O2v2V/eVDAaBwdrnrlSpNiD87/Svt+8flc/mKRfLDGweYGjrEMNbhxnuHWbPtj3s7d/L7m27fRjFuTYSp6DvBP4d2T7LyqPv1frsBJYVdEl3A3cDjIyMrDdWALaXByi+8jZDFqsWOC27q+hjij5Yc1t3Ol318ZraK63osbSl6K/LvNVfemu/JDLR24zISmQyoeWyIpvNkM2IrLJklFlquUxuqeWzefKZPIVsgVKuxKbcJkr5EuVCmXKxTE++h835zZRyJUq5Ej2FHrYUttBT6KG32EtfqY/eUm/XLijhX4a6ThWnoK9W1WoPI+P0wcxOACcgzHKJ8btXOPal+zjGfRt5qnPOpVqcwc6zwHBkewh4ZQN9nHPONVCcgv4UcLWkKyUVgE8CJ2v6nATuUHAD8N9GjJ8755yrb80hFzNbkPR54DeEaYsPmNlzkj5b3X8ceIQwZfEM4au3OxsXsnPOudXEmoduZo8Qinb0seOR+wZ8LtnQnHPOrYdPGHbOuZTwgu6ccynhBd0551LCC7pzzqVEyy6fK2kCeGmDT+8nckZ7l/Ccu4Pn3B0uJ+d3m9nAajtaVtAvh6S/1LsecFp5zt3Bc+4OjcrZh1yccy4lvKA751xKdGpBP9HqAFrAc+4OnnN3aEjOHTmG7pxzbqVOPUJ3zjlXwwu6c86lRFsXdEm3SPq7pDOSvrzKfkn6dnX/aUnXtiLOJMXI+VPVXE9LelLSNa2IM0lr5Rzpd52ki5Jub2Z8jRAnZ0k3SXpa0nOSft/sGJMW473dK+kXkp6p5tzRV22V9ICkcUnP1tmffP0ys7ZshEv1/gPYDRSAZ4D31vS5FfgVYcWkG4A/tTruJuT8AWBb9f5oN+Qc6fco4aqft7c67ia8zn2EdXtHqttXtDruJuT8FeBY9f4A8CpQaHXsl5Hzh4FrgWfr7E+8frXzEfrS4tRm9gZwaXHqqKXFqc3sFNAnabDZgSZozZzN7Ekzm6punqJm+eoOFOd1BvgC8FNgvJnBNUicnI8AD5vZywBm1ul5x8nZgLIkAT2Egr7Q3DCTY2aPE3KoJ/H61c4Fvd7C0+vt00nWm89dhE/4TrZmzpJ2AoeB46RDnNf5PcA2Sb+T9FdJdzQtusaIk/N3gP2E5SvHgC+a2WJzwmuJxOtXrAUuWiSxxak7SOx8JN1MKOgfbGhEjRcn528B95rZxXDw1vHi5JwDDgEfAzYBf5R0ysxeaHRwDRIn508ATwMfBfYAv5X0BzObbnRwLZJ4/Wrngt6Ni1PHykfSAeB+YNTMJpsUW6PEyfl9wEPVYt4P3Cppwcx+1pwQExf3vV0xsxlgRtLjwDVApxb0ODnfCXzDwgDzGUn/BPYBf25OiE2XeP1q5yGXblyces2cJY0ADwOf7uCjtag1czazK81sl5ntAn4C3NPBxRzivbd/DnxIUk7SZuB64Pkmx5mkODm/TPiPBEnbgb3Ai02NsrkSr19te4RuXbg4dcycvwq8E/he9Yh1wTr4SnUxc06VODmb2fOSfg2cBhaB+81s1elvnSDm6/x14IeSxgjDEfeaWcdeVlfSj4GbgH5JZ4GjQB4aV7/81H/nnEuJdh5ycc45tw5e0J1zLiW8oDvnXEp4QXfOuZTwgu6ccynhBd0551LCC7pzzqXE/wGVuEh+5bmWfwAAAABJRU5ErkJggg==\n", 549 | "text/plain": [ 550 | "
" 551 | ] 552 | }, 553 | "metadata": { 554 | "needs_background": "light" 555 | }, 556 | "output_type": "display_data" 557 | } 558 | ], 559 | "source": [ 560 | "x = np.linspace(0,1,100)\n", 561 | "\n", 562 | "b,r = 5, 3\n", 563 | "y = 1-(1-x**r)**b\n", 564 | "plt.plot(x,y,'r-',label=f\"b={b},r={r}, n={b*r}\")\n", 565 | "\n", 566 | "b,r = 40, 6\n", 567 | "y = 1-(1-x**r)**b\n", 568 | "plt.plot(x,y,'b-',label=f\"b={b}, r={r}, n={b*r}\")\n", 569 | "\n", 570 | "b,r = 150, 8\n", 571 | "y = 1-(1-x**r)**b\n", 572 | "plt.plot(x,y,'g-',label=f\"b={b}, r={r}, n={b*r}\")\n", 573 | "\n", 574 | "b,r = 700, 10\n", 575 | "y = 1-(1-x**r)**b\n", 576 | "plt.plot(x,y,'g-',label=f\"b={b}, r={r}, n={b*r}\")\n", 577 | "\n", 578 | "\n", 579 | "plt.plot(x,0*x+0.5,'k--')\n", 580 | "plt.plot([0.5,0.5],[0,1],'k--')\n", 581 | "plt.legend()\n", 582 | "plt.show()" 583 | ] 584 | }, 585 | { 586 | "cell_type": "markdown", 587 | "metadata": {}, 588 | "source": [ 589 | "Now, we use all of the above to try to optimize our Localitly Sensitive Hash function from before.\n", 590 | "\n", 591 | "Let's say we want a threshold of $p=0.3$. We can calculate $b\\approx p^{-r}$ for the number of bands. \n", 592 | "\n", 593 | "For the first 6 values of band size we determine we should keep\n", 594 | "\n", 595 | "$r=1 \\to b=3,\\,n=3$\n", 596 | "\n", 597 | "$r=2 \\to b=11,\\,n=22$\n", 598 | "\n", 599 | "$r=3 \\to b=37,\\,n=111$\n", 600 | "\n", 601 | "$r=4 \\to b=123,\\,n=492$\n", 602 | "\n", 603 | "$r=5 \\to b=412,\\,n=2060$\n", 604 | "\n", 605 | "Let's try the intermediate choice, and use round numbers $n=120$, $b=40$, $r=3$\n", 606 | "\n", 607 | "Let's see how good our results are using the previous defined functions. We first need to construct the banded similarity measure to be used on the signature matrix. Put this into a function" 608 | ] 609 | }, 610 | { 611 | "cell_type": "markdown", 612 | "metadata": {}, 613 | "source": [ 614 | "### Brute force banded candidate pair function, for checking hash method later" 615 | ] 616 | }, 617 | { 618 | "cell_type": "code", 619 | "execution_count": 21, 620 | "metadata": {}, 621 | "outputs": [ 622 | { 623 | "name": "stdout", 624 | "output_type": "stream", 625 | "text": [ 626 | "True pairs: {(0, 4), (1, 4)}\n", 627 | "LSH pairs: {(0, 4), (1, 4)}\n", 628 | "Vanilla MinHash pairs: {(0, 4), (3, 4), (1, 4)}\n", 629 | "**~~~~~~ Similarity score comparison ~~~~~~**\n", 630 | "Pair\t\tApprox\t\tTrue\t\t%Error\n", 631 | "(0, 1)\t\t0.117\t\t0.050\t\t133.89\n", 632 | "(0, 2)\t\t0.142\t\t0.069\t\t105.69\n", 633 | "(0, 3)\t\t0.150\t\t0.093\t\t60.89\n", 634 | "(0, 4)\t\t0.475\t\t0.336\t\t41.28\n", 635 | "(1, 2)\t\t0.183\t\t0.052\t\t254.68\n", 636 | "(1, 3)\t\t0.175\t\t0.051\t\t240.45\n", 637 | "(1, 4)\t\t0.408\t\t0.400\t\t2.03\n", 638 | "(2, 3)\t\t0.233\t\t0.081\t\t188.18\n", 639 | "(2, 4)\t\t0.225\t\t0.083\t\t171.18\n", 640 | "(3, 4)\t\t0.325\t\t0.294\t\t10.36\n" 641 | ] 642 | } 643 | ], 644 | "source": [ 645 | "def bandedCandidatePair(col1, col2, b, r):\n", 646 | " \"\"\"Returns a boolean if the two columns are a candidate pair\n", 647 | " inputs must obey n=len(col1)=len(col2)=b*r\"\"\"\n", 648 | " n = len(col1)\n", 649 | " assert(n==b*r)\n", 650 | " assert(n==len(col2))\n", 651 | " truth_array = (col1==col2)\n", 652 | " return any(all(band) for band in np.array_split(truth_array,b))\n", 653 | "\n", 654 | "def bandedCandidatePairs(sig_mat, b, r):\n", 655 | " d = sig_mat.shape[1]\n", 656 | " idxs = range(d)\n", 657 | " cols = [sig_mat[:,i] for i in range(d)]\n", 658 | " pairs = set()\n", 659 | " for (i,col1), (j,col2) in itertools.combinations(zip(idxs,cols),2):\n", 660 | " if bandedCandidatePair(col1,col2,b,r):\n", 661 | " pairs.add((i,j))\n", 662 | " return pairs\n", 663 | "\n", 664 | "# set p = 0.3 arbitrarily\n", 665 | "p = 0.3\n", 666 | "n = 120\n", 667 | "b = 30\n", 668 | "r = 4\n", 669 | "\n", 670 | "# see how many candidate pairs we got right!\n", 671 | "sig_mat = hm(doc_shingles, n)\n", 672 | "true_score_dict = trueSimScores(doc_shingles)\n", 673 | "approx_score_dict = sigSimScores(sig_mat)\n", 674 | "print(\"True pairs:\",candidatePairs(true_score_dict, p))\n", 675 | "print(\"LSH pairs:\",bandedCandidatePairs(sig_mat, b, r))\n", 676 | "print(\"Vanilla MinHash pairs:\",candidatePairs(approx_score_dict, p))\n", 677 | "# accMatrix(true_score_dict, approx_score_dict, 0.4)\n", 678 | "\n", 679 | "# sig_mat = hm(doc_shingles, n)\n", 680 | "# true_score_dict = trueSimScores(doc_shingles)\n", 681 | "printScoreComparison(true_score_dict, approx_score_dict)" 682 | ] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "metadata": {}, 687 | "source": [ 688 | "### Finally, we perform the fast candidate pair search, using a hash table of band and column id's" 689 | ] 690 | }, 691 | { 692 | "cell_type": "markdown", 693 | "metadata": {}, 694 | "source": [ 695 | "Lastly, we will improve upon the whole efficiency by performing the comparisons of banded pairs in $\\mathcal{O}(n)$ time as opposed to $\\mathcal{O}(n^2)$\n", 696 | "\n", 697 | "For each band of a column, we place a copy of the column index into a hash table, with key the row values.\n", 698 | "\n", 699 | "For example, let col=[4,2,5,6]. This might band to say [(4,2),(5,6)], and we ould enter into a dictionary d={(4,2):[col,...], (5,6):[col,...], ...}" 700 | ] 701 | }, 702 | { 703 | "cell_type": "code", 704 | "execution_count": 25, 705 | "metadata": {}, 706 | "outputs": [ 707 | { 708 | "name": "stdout", 709 | "output_type": "stream", 710 | "text": [ 711 | "True pairs:\t {(0, 4), (1, 4)}\n", 712 | "True LSH pairs:\t {(2, 3), (3, 4), (1, 4)}\n", 713 | "Fast LSH pairs:\t {(2, 3), (3, 4), (1, 4)}\n", 714 | "MinHash pairs:\t {(0, 4), (3, 4), (1, 4)}\n" 715 | ] 716 | } 717 | ], 718 | "source": [ 719 | "def fastCandidatePairs(sig_mat, b, r):\n", 720 | " n, d = sig_mat.shape\n", 721 | " assert(n==b*r)\n", 722 | " hashbuckets = collections.defaultdict(set)\n", 723 | " bands = np.array_split(sig_mat, b, axis=0)\n", 724 | " for i,band in enumerate(bands):\n", 725 | " for j in range(d):\n", 726 | " # The last value must be made a string, to prevent accidental\n", 727 | " # key collisions of r+1 integers when we really only want\n", 728 | " # keys of r integers plus a band index\n", 729 | " band_id = tuple(list(band[:,j])+[str(i)])\n", 730 | " hashbuckets[band_id].add(j)\n", 731 | " candidate_pairs = set()\n", 732 | " for bucket in hashbuckets.values():\n", 733 | " if len(bucket) > 1:\n", 734 | " for pair in itertools.combinations(bucket, 2):\n", 735 | " candidate_pairs.add(pair)\n", 736 | " return candidate_pairs\n", 737 | "\n", 738 | "\n", 739 | "# to make sure it works, \n", 740 | "# compare with the brute force method on a few trials\n", 741 | "\n", 742 | "# set p = 0.3 arbitrarily\n", 743 | "p = 0.3\n", 744 | "n = 120\n", 745 | "b = 30\n", 746 | "r = 4\n", 747 | "\n", 748 | "# see how many candidate pairs we got right!\n", 749 | "sig_mat = hm(doc_shingles, n)\n", 750 | "true_score_dict = trueSimScores(doc_shingles)\n", 751 | "approx_score_dict = sigSimScores(sig_mat)\n", 752 | "print('True pairs:\\t', candidatePairs(true_score_dict, p))\n", 753 | "print(\"True LSH pairs:\\t\", bandedCandidatePairs(sig_mat, b, r))\n", 754 | "print(\"Fast LSH pairs:\\t\", fastCandidatePairs(sig_mat, b, r))\n", 755 | "print(\"MinHash pairs:\\t\", candidatePairs(approx_score_dict, p))" 756 | ] 757 | }, 758 | { 759 | "cell_type": "markdown", 760 | "metadata": {}, 761 | "source": [ 762 | "# Part II: vector similarity" 763 | ] 764 | }, 765 | { 766 | "cell_type": "code", 767 | "execution_count": 30, 768 | "metadata": {}, 769 | "outputs": [], 770 | "source": [ 771 | "def cossim(u,v):\n", 772 | " norm = np.linalg.norm(u)*np.linalg.norm(v)\n", 773 | " cosine = u@v/norm\n", 774 | " ang = np.arccos(cosine)\n", 775 | " return 1-ang/np.pi" 776 | ] 777 | }, 778 | { 779 | "cell_type": "markdown", 780 | "metadata": {}, 781 | "source": [ 782 | "## Generate data\n", 783 | "\n", 784 | "We create N random vectors of dimension D. We then reassign 5 of these to be artifically close to another set of 5. This is done by setting them equal and adding a small amount of noise.\n", 785 | "\n", 786 | "By artificially making some vectors closer to others, we can easily tune the target similarity threshold, allowing us to experiment better." 787 | ] 788 | }, 789 | { 790 | "cell_type": "code", 791 | "execution_count": 31, 792 | "metadata": {}, 793 | "outputs": [], 794 | "source": [ 795 | "# get some random data points\n", 796 | "N = 1000\n", 797 | "D= 50\n", 798 | "A = np.random.randn(N,D)\n", 799 | "# artificially make some similar to others\n", 800 | "A[5] = A[99] +np.random.randn(D)*0.05\n", 801 | "A[20] = A[85] +np.random.randn(D)*0.15\n", 802 | "A[13] = A[19] +np.random.randn(D)*0.25\n", 803 | "A[56] = A[71] +np.random.randn(D)*0.5\n", 804 | "A[45] = A[49] +np.random.randn(D)*0.66" 805 | ] 806 | }, 807 | { 808 | "cell_type": "markdown", 809 | "metadata": {}, 810 | "source": [ 811 | "## Brute force calculation baseline\n", 812 | "\n", 813 | "This is straightforward. We compute the cosine similarity of every possible pair, and keep only those that are at or above the chosen threshold\n", 814 | "\n", 815 | "We know beforehand that the smallest \"artificially most similar\" pair is going to be around ~0.82, so we set the threshold to be 0.8.\n", 816 | "\n", 817 | "In practice, setting the threshold is an art, and depends on how similar we want things to really be." 818 | ] 819 | }, 820 | { 821 | "cell_type": "code", 822 | "execution_count": 108, 823 | "metadata": {}, 824 | "outputs": [ 825 | { 826 | "name": "stdout", 827 | "output_type": "stream", 828 | "text": [ 829 | "Brute force calculation time: 6.184\n", 830 | "Discovered pairs:\n", 831 | "Pair: (5, 99),\tSimilarity: 0.98.\n", 832 | "Pair: (13, 19),\tSimilarity: 0.92.\n", 833 | "Pair: (20, 85),\tSimilarity: 0.95.\n", 834 | "Pair: (45, 49),\tSimilarity: 0.82.\n", 835 | "Pair: (56, 71),\tSimilarity: 0.86.\n" 836 | ] 837 | } 838 | ], 839 | "source": [ 840 | "true_pairs_dict = {}\n", 841 | "\n", 842 | "thresh = 0.8\n", 843 | "\n", 844 | "start = time.time()\n", 845 | "for (i,u),(j,v) in itertools.combinations([(i,x) for i,x in enumerate(A)],2):\n", 846 | " val = cossim(u,v)\n", 847 | " if val > thresh:\n", 848 | " true_pairs_dict[(i,j)] = val\n", 849 | "t_brute = time.time()-start\n", 850 | "\n", 851 | "# save just the keys without the values. Easier to compare later to LSH\n", 852 | "true_pairs = set(true_pairs_dict.keys())\n", 853 | "\n", 854 | "print(f\"Brute force calculation time: {t_brute:.3f}\")\n", 855 | "print(f\"Discovered pairs:\")\n", 856 | "for k, v in true_pairs_dict.items():\n", 857 | " print(f\"Pair: {k},\\tSimilarity: {v:.2f}.\")" 858 | ] 859 | }, 860 | { 861 | "cell_type": "markdown", 862 | "metadata": {}, 863 | "source": [ 864 | "## Locality Sensitive Hashing approach" 865 | ] 866 | }, 867 | { 868 | "cell_type": "markdown", 869 | "metadata": {}, 870 | "source": [ 871 | "### Find r, b parameters given threshold\n", 872 | "\n", 873 | "We must have that D%b==0, that is the number of bands divides the dimension. That's our only constraint. Remember, higher n is always better!\n", 874 | "\n", 875 | "There's no magic here, we just trial and error some combinations until we get near the desired threshold, in this case 0.75 (chosen in the previous cell)" 876 | ] 877 | }, 878 | { 879 | "cell_type": "code", 880 | "execution_count": 105, 881 | "metadata": {}, 882 | "outputs": [ 883 | { 884 | "name": "stdout", 885 | "output_type": "stream", 886 | "text": [ 887 | "Transition probability: 0.80\n" 888 | ] 889 | } 890 | ], 891 | "source": [ 892 | "b, r = 50, 18\n", 893 | "\n", 894 | "n = b*r\n", 895 | "print(f\"Transition probability: {(1/b)**(1/r):.2f}\")" 896 | ] 897 | }, 898 | { 899 | "cell_type": "markdown", 900 | "metadata": {}, 901 | "source": [ 902 | "### Compute LSH pairs\n", 903 | "\n", 904 | "Let $S$ be the signature matrix and $R$ be the matrix of random hyperplanes.\n", 905 | "\n", 906 | "In the code implementation, we work with the transpose of our equations. This means that $S = (N,D)$, and so we have to split into bands across axis=1" 907 | ] 908 | }, 909 | { 910 | "cell_type": "code", 911 | "execution_count": 106, 912 | "metadata": {}, 913 | "outputs": [ 914 | { 915 | "name": "stdout", 916 | "output_type": "stream", 917 | "text": [ 918 | "LSH calculation time: 0.249\n" 919 | ] 920 | } 921 | ], 922 | "source": [ 923 | "start = time.time()\n", 924 | "\n", 925 | "# Compute signature matrix\n", 926 | "R = A@np.random.randn(D,n)\n", 927 | "S = np.where(R>0,1,0)\n", 928 | "\n", 929 | "# Break into bands\n", 930 | "S = np.split(S, b, axis=1)\n", 931 | "\n", 932 | "# column vector to convert binary vector to integer e.g. (1,0,1)->5\n", 933 | "binary_column = 2**np.arange(r).reshape(-1,1)\n", 934 | "\n", 935 | "# convert each band into a single integer, \n", 936 | "# i.e. convert band matrices to band columns\n", 937 | "S = np.hstack([M@binary_column for M in S])\n", 938 | "\n", 939 | "# Every value in the matrix represents a hash bucket assignment\n", 940 | "# For every bucket in row i, add index i to that bucket\n", 941 | "d = collections.defaultdict(set)\n", 942 | "with np.nditer(S,flags=['multi_index']) as it:\n", 943 | " for x in it:\n", 944 | " d[int(x)].add(it.multi_index[0])\n", 945 | " \n", 946 | "# For every bucket, find all pairs. These are the LSH pairs.\n", 947 | "candidate_pairs = set()\n", 948 | "for k,v in d.items():\n", 949 | " if len(v) > 1:\n", 950 | " for pair in itertools.combinations(v,2):\n", 951 | " candidate_pairs.add(tuple(sorted(pair)))\n", 952 | "\n", 953 | "# Finally, perform the actually similarity computation\n", 954 | "# to weed out false positive\n", 955 | "lsh_pairs = set()\n", 956 | "for (i,j) in candidate_pairs:\n", 957 | " if cossim(A[i],A[j]) > thresh:\n", 958 | " lsh_pairs.add((i,j))\n", 959 | " \n", 960 | "t_lsh = time.time()-start\n", 961 | "\n", 962 | "print(f\"LSH calculation time: {t_lsh:.3f}\")" 963 | ] 964 | }, 965 | { 966 | "cell_type": "markdown", 967 | "metadata": {}, 968 | "source": [ 969 | "## Compare the two methods\n", 970 | "\n", 971 | "The above should show that we get a factor of about 25x speedup, while simultaneously finding all pairs, with no false negatives!\n", 972 | "\n", 973 | "Very useful!" 974 | ] 975 | }, 976 | { 977 | "cell_type": "code", 978 | "execution_count": 107, 979 | "metadata": {}, 980 | "outputs": [ 981 | { 982 | "name": "stdout", 983 | "output_type": "stream", 984 | "text": [ 985 | "t_brute: 6.142\t t_lsh: 0.249. Speed-up: 25x\n", 986 | "True pairs: {(45, 49), (20, 85), (56, 71), (13, 19), (5, 99)}\n", 987 | "LSH pairs: {(45, 49), (20, 85), (56, 71), (13, 19), (5, 99)}\n", 988 | "Candidate pairs: 4876.\n", 989 | "False negatives: 0\n" 990 | ] 991 | } 992 | ], 993 | "source": [ 994 | "print(f\"t_brute: {t_brute:.3f}\\t t_lsh: {t_lsh:.3f}. Speed-up: {t_brute/t_lsh:.0f}x\")\n", 995 | "print(\"True pairs: \", pairs_true)\n", 996 | "print(\"LSH pairs: \", lsh_pairs)\n", 997 | "print(f\"Candidate pairs: {len(candidate_pairs)}.\\n\\\n", 998 | "False negatives: {len(pairs_true-lsh_pairs)}\")" 999 | ] 1000 | } 1001 | ], 1002 | "metadata": { 1003 | "kernelspec": { 1004 | "display_name": "Python 3", 1005 | "language": "python", 1006 | "name": "python3" 1007 | }, 1008 | "language_info": { 1009 | "codemirror_mode": { 1010 | "name": "ipython", 1011 | "version": 3 1012 | }, 1013 | "file_extension": ".py", 1014 | "mimetype": "text/x-python", 1015 | "name": "python", 1016 | "nbconvert_exporter": "python", 1017 | "pygments_lexer": "ipython3", 1018 | "version": "3.8.3" 1019 | } 1020 | }, 1021 | "nbformat": 4, 1022 | "nbformat_minor": 4 1023 | } 1024 | --------------------------------------------------------------------------------