├── LICENSE ├── README.md ├── dataset.json ├── flow.png └── legal cases search.ipynb /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # legal-cases-search-using-RAG-self-query 2 | 3 | ---------------------------------------------------------------------------------------------------- 4 | 5 | Building a Legal case search engine using Qdrant, llama3, langchain and exploring different filtering techniques. 6 | 7 | Read step by step approach here: https://medium.com/ai-advances/building-a-legal-case-search-engine-using-qdrant-llama-3-langchain-and-exploring-different-655ed5b25f30 8 | 9 | ![](https://github.com/sachink1729/legal-cases-search-using-self-query-qdrant-llama3-langchain/blob/main/flow.png) 10 | -------------------------------------------------------------------------------- /dataset.json: -------------------------------------------------------------------------------- 1 | { 2 | "cases_list": [ 3 | { 4 | "page_content": "The Supreme Court upheld the Centre's 2016 demonetisation scheme in a 4:1 majority, ruling that demonetisation was proportionate to the Union’s objectives and implemented reasonably.", 5 | "metadata": { 6 | "year": 2023, 7 | "court": "Supreme Court of India", 8 | "judges": [ 9 | "S. Abdul Nazeer", 10 | "B.R. Gavai", 11 | "A.S. Bopanna", 12 | "V. Ramasubramanian", 13 | "B.V. Nagarathna" 14 | ], 15 | "legal_topics": [ 16 | "Constitutional Law", 17 | "Economic Policy" 18 | ], 19 | "relevant_laws": [ 20 | "Article 370 of the Indian Constitution" 21 | ] 22 | } 23 | }, 24 | { 25 | "page_content": "The United States Supreme Court ruled that the Chief Election Commissioner and Election Commissioners must be appointed by a three-member committee comprising the Prime Minister, leader of the Opposition, and the Chief Justice.", 26 | "metadata": { 27 | "year": 2023, 28 | "court": "Supreme Court of the United States", 29 | "judges": [ 30 | "K.M. Joseph", 31 | "Ajay Rastogi", 32 | "Aniruddha Bose", 33 | "Hrishikesh Roy", 34 | "C.T. Ravikumar" 35 | ], 36 | "legal_topics": [ 37 | "Election Law", 38 | "Constitutional Law" 39 | ], 40 | "relevant_laws": [ 41 | "Election Commission Act" 42 | ] 43 | } 44 | }, 45 | { 46 | "page_content": "In a case concerning the rights of students with disabilities, the United States Department of Justice concluded that Alabama’s foster care system discriminates against students with emotional and behavioral disabilities.", 47 | "metadata": { 48 | "year": 2022, 49 | "court": "United States Department of Justice", 50 | "judges": [ 51 | "Investigation by the Civil Rights Division" 52 | ], 53 | "legal_topics": [ 54 | "Disability Law", 55 | "Education Law" 56 | ], 57 | "relevant_laws": [ 58 | "Title II of the Americans with Disabilities Act" 59 | ] 60 | } 61 | }, 62 | { 63 | "page_content": "The U.S. Supreme Court ruled that a state cannot require out-of-state sellers to collect sales tax unless they have a significant presence in the state.", 64 | "metadata": { 65 | "year": 2018, 66 | "court": "Supreme Court of the United States", 67 | "judges": [ 68 | "Majority opinion by Justice Kennedy" 69 | ], 70 | "legal_topics": [ 71 | "Tax Law", 72 | "Commerce Clause" 73 | ], 74 | "relevant_laws": [ 75 | "Commerce Clause of the U.S. Constitution" 76 | ] 77 | } 78 | }, 79 | { 80 | "page_content": "The Supreme Court of India modified guidelines for passive euthanasia, allowing a two-tiered process for authorizing the withdrawal of life support for terminally ill patients.", 81 | "metadata": { 82 | "year": 2023, 83 | "court": "Supreme Court of India", 84 | "judges": [ 85 | "K.M. Joseph", 86 | "Ajay Rastogi", 87 | "Aniruddha Bose", 88 | "Hrishikesh Roy", 89 | "C.T. Ravikumar" 90 | ], 91 | "legal_topics": [ 92 | "Health Law", 93 | "Right to Die" 94 | ], 95 | "relevant_laws": [ 96 | "Guidelines for Passive Euthanasia" 97 | ] 98 | } 99 | }, 100 | { 101 | "page_content": "In a landmark decision, the Supreme Court of Canada upheld a lower court's ruling that police must obtain a warrant before accessing subscriber information from internet service providers.", 102 | "metadata": { 103 | "year": 2014, 104 | "court": "Supreme Court of Canada", 105 | "judges": [ 106 | "Justice Thomas Cromwell" 107 | ], 108 | "legal_topics": [ 109 | "Privacy Law", 110 | "Criminal Procedure" 111 | ], 112 | "relevant_laws": [ 113 | "Canadian Charter of Rights and Freedoms" 114 | ] 115 | } 116 | }, 117 | { 118 | "page_content": "The Ninth Circuit Court of Appeals held that the Second Amendment protects an individual's right to carry a firearm in public for self-defense.", 119 | "metadata": { 120 | "year": 2021, 121 | "court": "Ninth Circuit Court of Appeals", 122 | "judges": [ 123 | "Majority opinion by Judge O’Scannlain" 124 | ], 125 | "legal_topics": [ 126 | "Second Amendment", 127 | "Gun Rights" 128 | ], 129 | "relevant_laws": [ 130 | "Second Amendment of the U.S. Constitution" 131 | ] 132 | } 133 | }, 134 | { 135 | "page_content": "The European Court of Human Rights ruled that the UK's bulk interception regime violated the right to privacy and lacked sufficient safeguards.", 136 | "metadata": { 137 | "year": 2018, 138 | "court": "European Court of Human Rights", 139 | "judges": [ 140 | "Grand Chamber" 141 | ], 142 | "legal_topics": [ 143 | "Privacy Law", 144 | "Surveillance" 145 | ], 146 | "relevant_laws": [ 147 | "European Convention on Human Rights" 148 | ] 149 | } 150 | }, 151 | { 152 | "page_content": "The Supreme Court of Texas ruled that landowners cannot sue for trespass against oil and gas companies that use hydraulic fracturing on adjacent properties.", 153 | "metadata": { 154 | "year": 2008, 155 | "court": "Supreme Court of Texas", 156 | "judges": [ 157 | "Justice Nathan Hecht" 158 | ], 159 | "legal_topics": [ 160 | "Property Law", 161 | "Environmental Law" 162 | ], 163 | "relevant_laws": [ 164 | "Texas Property Code" 165 | ] 166 | } 167 | }, 168 | { 169 | "page_content": "The High Court of Australia found that the Commonwealth's immigration detention regime did not violate the Australian Constitution.", 170 | "metadata": { 171 | "year": 2004, 172 | "court": "High Court of Australia", 173 | "judges": [ 174 | "Chief Justice Murray Gleeson" 175 | ], 176 | "legal_topics": [ 177 | "Immigration Law", 178 | "Constitutional Law" 179 | ], 180 | "relevant_laws": [ 181 | "Migration Act 1958" 182 | ] 183 | } 184 | }, 185 | { 186 | "page_content": "The European Court of Justice ruled that Google must remove links to personal data when requested by individuals under the 'Right to be Forgotten' principle.", 187 | "metadata": { 188 | "year": 2014, 189 | "court": "European Court of Justice", 190 | "judges": [ 191 | "Judge Antonio Tizzano" 192 | ], 193 | "legal_topics": [ 194 | "Data Privacy", 195 | "Internet Law" 196 | ], 197 | "relevant_laws": [ 198 | "General Data Protection Regulation (GDPR)" 199 | ] 200 | } 201 | }, 202 | { 203 | "page_content": "The U.S. Supreme Court decided that same-sex marriage is a constitutional right under the Fourteenth Amendment.", 204 | "metadata": { 205 | "year": 2015, 206 | "court": "Supreme Court of the United States", 207 | "judges": [ 208 | "Majority opinion by Justice Anthony Kennedy" 209 | ], 210 | "legal_topics": [ 211 | "Constitutional Law", 212 | "Civil Rights" 213 | ], 214 | "relevant_laws": [ 215 | "Fourteenth Amendment of the U.S. Constitution" 216 | ] 217 | } 218 | }, 219 | { 220 | "page_content": "The High Court of Australia held that the plain packaging laws for tobacco products were not unconstitutional.", 221 | "metadata": { 222 | "year": 2012, 223 | "court": "High Court of Australia", 224 | "judges": [ 225 | "Chief Justice Robert French" 226 | ], 227 | "legal_topics": [ 228 | "Health Law", 229 | "Intellectual Property" 230 | ], 231 | "relevant_laws": [ 232 | "Tobacco Plain Packaging Act 2011" 233 | ] 234 | } 235 | }, 236 | { 237 | "page_content": "The Supreme Court of the United Kingdom ruled that Uber drivers are workers and entitled to employment rights such as minimum wage and holiday pay.", 238 | "metadata": { 239 | "year": 2021, 240 | "court": "Supreme Court of the United Kingdom", 241 | "judges": [ 242 | "Lord Reed", 243 | "Lord Hodge", 244 | "Lady Arden" 245 | ], 246 | "legal_topics": [ 247 | "Employment Law", 248 | "Gig Economy" 249 | ], 250 | "relevant_laws": [ 251 | "Employment Rights Act 1996" 252 | ] 253 | } 254 | }, 255 | { 256 | "page_content": "The Constitutional Court of South Africa ruled that the use of corporal punishment in the home is unconstitutional.", 257 | "metadata": { 258 | "year": 2019, 259 | "court": "Constitutional Court of South Africa", 260 | "judges": [ 261 | "Chief Justice Mogoeng Mogoeng" 262 | ], 263 | "legal_topics": [ 264 | "Family Law", 265 | "Human Rights" 266 | ], 267 | "relevant_laws": [ 268 | "South African Constitution" 269 | ] 270 | } 271 | }, 272 | { 273 | "page_content": "The U.S. Supreme Court invalidated a federal law prohibiting sports betting outside Nevada, holding it violated the anti-commandeering rule.", 274 | "metadata": { 275 | "year": 2018, 276 | "court": "Supreme Court of the United States", 277 | "judges": [ 278 | "Majority opinion by Justice Samuel Alito" 279 | ], 280 | "legal_topics": [ 281 | "Sports Law", 282 | "Constitutional Law" 283 | ], 284 | "relevant_laws": [ 285 | "Professional and Amateur Sports Protection Act (PASPA)" 286 | ] 287 | } 288 | }, 289 | { 290 | "page_content": "The Supreme Court of Canada ruled that a drug-sniffing dog’s alert to the presence of drugs provided the police with the reasonable grounds necessary for a search.", 291 | "metadata": { 292 | "year": 2008, 293 | "court": "Supreme Court of Canada", 294 | "judges": [ 295 | "Justice Louise Charron" 296 | ], 297 | "legal_topics": [ 298 | "Criminal Law", 299 | "Search and Seizure" 300 | ], 301 | "relevant_laws": [ 302 | "Canadian Charter of Rights and Freedoms" 303 | ] 304 | } 305 | }, 306 | { 307 | "page_content": "The International Court of Justice ruled that Myanmar must take measures to prevent the genocide of the Rohingya people.", 308 | "metadata": { 309 | "year": 2020, 310 | "court": "International Court of Justice", 311 | "judges": [ 312 | "President Abdulqawi Yusuf" 313 | ], 314 | "legal_topics": [ 315 | "International Law", 316 | "Human Rights" 317 | ], 318 | "relevant_laws": [ 319 | "Convention on the Prevention and Punishment of the Crime of Genocide" 320 | ] 321 | } 322 | }, 323 | { 324 | "page_content": "The Court of Justice of the European Union ruled that data retention laws requiring telecommunications companies to retain users’ data indiscriminately are invalid.", 325 | "metadata": { 326 | "year": 2016, 327 | "court": "Court of Justice of the European Union", 328 | "judges": [ 329 | "Judge Koen Lenaerts" 330 | ], 331 | "legal_topics": [ 332 | "Data Privacy", 333 | "Telecommunications Law" 334 | ], 335 | "relevant_laws": [ 336 | "Directive 2006/24/EC" 337 | ] 338 | } 339 | }, 340 | { 341 | "page_content": "The Supreme Court of India declared the practice of instant triple talaq (talaq-e-biddat) as unconstitutional.", 342 | "metadata": { 343 | "year": 2017, 344 | "court": "Supreme Court of India", 345 | "judges": [ 346 | "Chief Justice J.S. Khehar", 347 | "Justice Kurian Joseph" 348 | ], 349 | "legal_topics": [ 350 | "Family Law", 351 | "Religious Law" 352 | ], 353 | "relevant_laws": [ 354 | "Article 14 of the Indian Constitution" 355 | ] 356 | } 357 | }, 358 | { 359 | "page_content": "The U.S. Supreme Court ruled that state laws banning abortion are unconstitutional, recognizing a woman's right to privacy in making medical decisions.", 360 | "metadata": { 361 | "year": 1973, 362 | "court": "Supreme Court of the United States", 363 | "judges": [ 364 | "Majority opinion by Justice Harry Blackmun" 365 | ], 366 | "legal_topics": [ 367 | "Constitutional Law", 368 | "Reproductive Rights" 369 | ], 370 | "relevant_laws": [ 371 | "Fourteenth Amendment of the U.S. Constitution" 372 | ] 373 | } 374 | }, 375 | { 376 | "page_content": "The Supreme Court of Canada upheld the right of individuals to physician-assisted death under certain conditions, ruling that prohibiting it violated the Canadian Charter of Rights and Freedoms.", 377 | "metadata": { 378 | "year": 2015, 379 | "court": "Supreme Court of Canada", 380 | "judges": [ 381 | "Chief Justice Beverley McLachlin" 382 | ], 383 | "legal_topics": [ 384 | "Health Law", 385 | "Human Rights" 386 | ], 387 | "relevant_laws": [ 388 | "Canadian Charter of Rights and Freedoms" 389 | ] 390 | } 391 | }, 392 | { 393 | "page_content": "The European Court of Human Rights ruled that the French ban on wearing face veils in public did not violate the European Convention on Human Rights.", 394 | "metadata": { 395 | "year": 2014, 396 | "court": "European Court of Human Rights", 397 | "judges": [ 398 | "Judge Dean Spielmann" 399 | ], 400 | "legal_topics": [ 401 | "Human Rights", 402 | "Religious Freedom" 403 | ], 404 | "relevant_laws": [ 405 | "European Convention on Human Rights" 406 | ] 407 | } 408 | }, 409 | { 410 | "page_content": "The Constitutional Court of South Africa ruled that domestic workers are entitled to the same compensation benefits as other workers.", 411 | "metadata": { 412 | "year": 2020, 413 | "court": "Constitutional Court of South Africa", 414 | "judges": [ 415 | "Chief Justice Mogoeng Mogoeng" 416 | ], 417 | "legal_topics": [ 418 | "Labor Law", 419 | "Equality" 420 | ], 421 | "relevant_laws": [ 422 | "Compensation for Occupational Injuries and Diseases Act" 423 | ] 424 | } 425 | }, 426 | { 427 | "page_content": "The Supreme Court of India decriminalized consensual homosexual acts between adults, overturning a colonial-era law under Section 377 of the Indian Penal Code.", 428 | "metadata": { 429 | "year": 2018, 430 | "court": "Supreme Court of India", 431 | "judges": [ 432 | "Chief Justice Dipak Misra" 433 | ], 434 | "legal_topics": [ 435 | "Human Rights", 436 | "LGBTQ+ Rights" 437 | ], 438 | "relevant_laws": [ 439 | "Section 377 of the Indian Penal Code" 440 | ] 441 | } 442 | }, 443 | { 444 | "page_content": "The U.S. Supreme Court held that the Defense of Marriage Act (DOMA) was unconstitutional as it violated the Fifth Amendment by denying federal recognition of same-sex marriages.", 445 | "metadata": { 446 | "year": 2013, 447 | "court": "Supreme Court of the United States", 448 | "judges": [ 449 | "Majority opinion by Justice Anthony Kennedy" 450 | ], 451 | "legal_topics": [ 452 | "Constitutional Law", 453 | "LGBTQ+ Rights" 454 | ], 455 | "relevant_laws": [ 456 | "Fifth Amendment of the U.S. Constitution" 457 | ] 458 | } 459 | }, 460 | { 461 | "page_content": "The International Criminal Court found Jean-Pierre Bemba guilty of war crimes and crimes against humanity for his role in the Central African Republic conflict.", 462 | "metadata": { 463 | "year": 2016, 464 | "court": "International Criminal Court", 465 | "judges": [ 466 | "Judge Sylvia Steiner" 467 | ], 468 | "legal_topics": [ 469 | "International Law", 470 | "Human Rights" 471 | ], 472 | "relevant_laws": [ 473 | "Rome Statute of the International Criminal Court" 474 | ] 475 | } 476 | }, 477 | { 478 | "page_content": "The Supreme Court of the United Kingdom ruled that Parliament must vote on triggering Article 50 to begin the Brexit process.", 479 | "metadata": { 480 | "year": 2017, 481 | "court": "Supreme Court of the United Kingdom", 482 | "judges": [ 483 | "President Lord Neuberger" 484 | ], 485 | "legal_topics": [ 486 | "Constitutional Law", 487 | "European Union Law" 488 | ], 489 | "relevant_laws": [ 490 | "Article 50 of the Treaty on European Union" 491 | ] 492 | } 493 | }, 494 | { 495 | "page_content": "The Supreme Court of Japan upheld a law requiring married couples to share a surname, ruling it did not violate constitutional guarantees of equality.", 496 | "metadata": { 497 | "year": 2015, 498 | "court": "Supreme Court of Japan", 499 | "judges": [ 500 | "Chief Justice Itsuro Terada" 501 | ], 502 | "legal_topics": [ 503 | "Family Law", 504 | "Equality" 505 | ], 506 | "relevant_laws": [ 507 | "Japanese Civil Code" 508 | ] 509 | } 510 | }, 511 | { 512 | "page_content": "The Constitutional Court of South Africa ruled that the President’s appointment of the National Director of Public Prosecutions was invalid due to a conflict of interest.", 513 | "metadata": { 514 | "year": 2012, 515 | "court": "Constitutional Court of South Africa", 516 | "judges": [ 517 | "Deputy Chief Justice Dikgang Moseneke" 518 | ], 519 | "legal_topics": [ 520 | "Constitutional Law", 521 | "Public Administration" 522 | ], 523 | "relevant_laws": [ 524 | "South African Constitution" 525 | ] 526 | } 527 | } 528 | ] 529 | } -------------------------------------------------------------------------------- /flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sachink1729/legal-cases-search-using-self-query-qdrant-llama3-langchain/5291db08d1491a061811408caec80afc9e8352b1/flow.png -------------------------------------------------------------------------------- /legal cases search.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "# !pip install langchain-huggingface\n", 10 | "# !pip install qdrant-client\n", 11 | "# !pip install langchain-qdrant\n", 12 | "# !pip install langchain-community\n", 13 | "# !pip install lark" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 59, 19 | "metadata": {}, 20 | "outputs": [], 21 | "source": [ 22 | "import json\n", 23 | "dataset = json.load(open(\"dataset.json\"))['cases_list']" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 88, 29 | "metadata": {}, 30 | "outputs": [ 31 | { 32 | "data": { 33 | "text/plain": [ 34 | "{'page_content': \"The Supreme Court upheld the Centre's 2016 demonetisation scheme in a 4:1 majority, ruling that demonetisation was proportionate to the Union’s objectives and implemented reasonably.\",\n", 35 | " 'metadata': {'year': 2023,\n", 36 | " 'court': 'Supreme Court of India',\n", 37 | " 'judges': ['S. Abdul Nazeer',\n", 38 | " 'B.R. Gavai',\n", 39 | " 'A.S. Bopanna',\n", 40 | " 'V. Ramasubramanian',\n", 41 | " 'B.V. Nagarathna'],\n", 42 | " 'legal_topics': ['Constitutional Law', 'Economic Policy'],\n", 43 | " 'relevant_laws': ['Article 370 of the Indian Constitution']}}" 44 | ] 45 | }, 46 | "execution_count": 88, 47 | "metadata": {}, 48 | "output_type": "execute_result" 49 | } 50 | ], 51 | "source": [ 52 | "dataset[0]" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 89, 58 | "metadata": {}, 59 | "outputs": [ 60 | { 61 | "name": "stderr", 62 | "output_type": "stream", 63 | "text": [ 64 | "c:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\huggingface_hub\\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n", 65 | " warnings.warn(\n" 66 | ] 67 | } 68 | ], 69 | "source": [ 70 | "from langchain_huggingface import HuggingFaceEmbeddings\n", 71 | "\n", 72 | "model_name = \"BAAI/bge-small-en-v1.5\"\n", 73 | "model_kwargs = {'device': 'cpu'}\n", 74 | "encode_kwargs = {'normalize_embeddings': True}\n", 75 | "hf = HuggingFaceEmbeddings(\n", 76 | " model_name=model_name,\n", 77 | " model_kwargs=model_kwargs,\n", 78 | " encode_kwargs=encode_kwargs\n", 79 | ")\n" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": 93, 85 | "metadata": {}, 86 | "outputs": [ 87 | { 88 | "data": { 89 | "text/plain": [ 90 | "[-0.08326546847820282,\n", 91 | " 0.022219663485884666,\n", 92 | " -0.017243655398488045,\n", 93 | " -0.053167618811130524,\n", 94 | " 0.05313488468527794,\n", 95 | " -0.016739359125494957,\n", 96 | " 0.045978739857673645,\n", 97 | " -0.013614344410598278,\n", 98 | " 0.014157246798276901,\n", 99 | " -0.002596378792077303,\n", 100 | " -0.007901007309556007,\n", 101 | " -0.038068678230047226,\n", 102 | " -0.018489299342036247,\n", 103 | " 0.014195187948644161,\n", 104 | " 0.04925695061683655,\n", 105 | " 0.030631814152002335,\n", 106 | " -0.0024389438331127167,\n", 107 | " 0.03351893648505211,\n", 108 | " -0.027505427598953247,\n", 109 | " 0.013234520331025124,\n", 110 | " 0.05578543618321419,\n", 111 | " -0.09169687330722809,\n", 112 | " 0.008015230298042297,\n", 113 | " 0.002464858815073967,\n", 114 | " 0.0008859741501510143,\n", 115 | " -0.0645192414522171,\n", 116 | " 0.04931394010782242,\n", 117 | " 0.02818586304783821,\n", 118 | " -0.02374214492738247,\n", 119 | " -0.15073446929454803,\n", 120 | " 0.04553466662764549,\n", 121 | " -0.04948839917778969,\n", 122 | " -0.010095835663378239,\n", 123 | " -0.017972297966480255,\n", 124 | " 0.04311444237828255,\n", 125 | " -0.0001938668719958514,\n", 126 | " 0.009973517619073391,\n", 127 | " 0.05950810760259628,\n", 128 | " -0.045189421623945236,\n", 129 | " 0.00553807383403182,\n", 130 | " -0.0160306878387928,\n", 131 | " 0.05328197032213211,\n", 132 | " -0.01825469173491001,\n", 133 | " -0.04073579981923103,\n", 134 | " -0.02955484576523304,\n", 135 | " -0.04916971176862717,\n", 136 | " 0.015369133092463017,\n", 137 | " 0.012933961115777493,\n", 138 | " -0.019825533032417297,\n", 139 | " -0.07850152999162674,\n", 140 | " -0.002757840557023883,\n", 141 | " -0.0289602093398571,\n", 142 | " 0.050903405994176865,\n", 143 | " -0.001060040551237762,\n", 144 | " -0.031101901084184647,\n", 145 | " -0.013894295319914818,\n", 146 | " 0.06615923345088959,\n", 147 | " 0.007093074265867472,\n", 148 | " 0.024887623265385628,\n", 149 | " 0.011715903878211975,\n", 150 | " 0.04374003782868385,\n", 151 | " 0.02510894276201725,\n", 152 | " -0.14118444919586182,\n", 153 | " 0.03555767610669136,\n", 154 | " 0.021065473556518555,\n", 155 | " -0.023624680936336517,\n", 156 | " 0.042660534381866455,\n", 157 | " -0.08280530571937561,\n", 158 | " 0.00317375804297626,\n", 159 | " 0.04066447168588638,\n", 160 | " -0.047864943742752075,\n", 161 | " -0.006743596866726875,\n", 162 | " 0.005174955353140831,\n", 163 | " 0.04344658553600311,\n", 164 | " -0.013889029622077942,\n", 165 | " 0.02071125991642475,\n", 166 | " 0.051248595118522644,\n", 167 | " 0.026923194527626038,\n", 168 | " -0.07655970752239227,\n", 169 | " 0.019301310181617737,\n", 170 | " -0.013958411291241646,\n", 171 | " 0.023905495181679726,\n", 172 | " -0.025996236130595207,\n", 173 | " -0.0021570452954620123,\n", 174 | " -0.016144637018442154,\n", 175 | " -0.00587574765086174,\n", 176 | " 0.007637702394276857,\n", 177 | " 0.003612929955124855,\n", 178 | " 0.03726716339588165,\n", 179 | " 0.03199515491724014,\n", 180 | " 0.01635560393333435,\n", 181 | " 0.022592121735215187,\n", 182 | " 0.006319664418697357,\n", 183 | " 0.005361567717045546,\n", 184 | " -0.005053266417235136,\n", 185 | " -0.02008374221622944,\n", 186 | " 0.005936732050031424,\n", 187 | " 0.03616277500987053,\n", 188 | " -0.0473904050886631,\n", 189 | " 0.31652840971946716,\n", 190 | " 0.0005468543968163431,\n", 191 | " -0.003618766088038683,\n", 192 | " -0.00820055603981018,\n", 193 | " -0.06510823965072632,\n", 194 | " 0.05025618150830269,\n", 195 | " 0.04850873351097107,\n", 196 | " 0.03280629217624664,\n", 197 | " 0.019894365221261978,\n", 198 | " -0.028520112857222557,\n", 199 | " -0.014702113345265388,\n", 200 | " -0.010978595353662968,\n", 201 | " 0.029922645539045334,\n", 202 | " 0.06474003940820694,\n", 203 | " -0.025950830429792404,\n", 204 | " 0.034861210733652115,\n", 205 | " 0.08858642727136612,\n", 206 | " 0.03913937881588936,\n", 207 | " 0.05586954578757286,\n", 208 | " -0.09601674228906631,\n", 209 | " 0.0011855325428768992,\n", 210 | " 0.02904840186238289,\n", 211 | " -0.0009743861737661064,\n", 212 | " -0.026259714737534523,\n", 213 | " 0.009826622903347015,\n", 214 | " 0.024854406714439392,\n", 215 | " -0.023906877264380455,\n", 216 | " 0.02476298250257969,\n", 217 | " 0.11404205113649368,\n", 218 | " -0.0041105193085968494,\n", 219 | " -0.03781480714678764,\n", 220 | " -0.09081188589334488,\n", 221 | " -0.05617869645357132,\n", 222 | " -0.012969838455319405,\n", 223 | " -0.025309640914201736,\n", 224 | " 0.02726690098643303,\n", 225 | " 0.020682984963059425,\n", 226 | " -0.03012801706790924,\n", 227 | " 0.06963024288415909,\n", 228 | " 0.02194656990468502,\n", 229 | " 0.049473538994789124,\n", 230 | " -0.022007141262292862,\n", 231 | " -0.03171854466199875,\n", 232 | " -0.017608439549803734,\n", 233 | " -0.07687560468912125,\n", 234 | " -0.007787334267050028,\n", 235 | " 0.036477699875831604,\n", 236 | " -0.051921624690294266,\n", 237 | " -0.043949954211711884,\n", 238 | " -0.04396820440888405,\n", 239 | " 0.012641996145248413,\n", 240 | " 0.05418572202324867,\n", 241 | " 0.06219325587153435,\n", 242 | " -0.03189379349350929,\n", 243 | " -0.046170178800821304,\n", 244 | " -0.016153328120708466,\n", 245 | " -0.01743088848888874,\n", 246 | " -0.008360626175999641,\n", 247 | " -0.018609214574098587,\n", 248 | " -0.05770939961075783,\n", 249 | " -0.008877158164978027,\n", 250 | " 0.0723780021071434,\n", 251 | " -0.07230621576309204,\n", 252 | " -0.005439219065010548,\n", 253 | " 0.17953066527843475,\n", 254 | " 0.06667235493659973,\n", 255 | " -0.008271001279354095,\n", 256 | " 0.0063162269070744514,\n", 257 | " -0.006056896410882473,\n", 258 | " 0.02210439369082451,\n", 259 | " 0.0022074549924582243,\n", 260 | " 0.03077583573758602,\n", 261 | " -0.002509413054212928,\n", 262 | " -0.003386080265045166,\n", 263 | " -0.02353527583181858,\n", 264 | " 0.011988396756350994,\n", 265 | " -0.04115185886621475,\n", 266 | " 0.03061852976679802,\n", 267 | " 0.028947005048394203,\n", 268 | " -0.06960324943065643,\n", 269 | " 0.033138457685709,\n", 270 | " 0.019192412495613098,\n", 271 | " -0.03088211826980114,\n", 272 | " -0.04505741968750954,\n", 273 | " 0.02444908209145069,\n", 274 | " 0.0967826172709465,\n", 275 | " -0.016514020040631294,\n", 276 | " -0.06110897660255432,\n", 277 | " -0.05842067673802376,\n", 278 | " -0.0025536008179187775,\n", 279 | " 0.08263703435659409,\n", 280 | " -0.050623632967472076,\n", 281 | " 0.03338361158967018,\n", 282 | " -0.046463146805763245,\n", 283 | " 0.038217730820178986,\n", 284 | " 0.035319019109010696,\n", 285 | " -0.02476787380874157,\n", 286 | " -0.005784853361546993,\n", 287 | " -0.032952334731817245,\n", 288 | " -0.04733457416296005,\n", 289 | " 0.04780178889632225,\n", 290 | " 0.07048702239990234,\n", 291 | " 0.013445441611111164,\n", 292 | " -0.06243104115128517,\n", 293 | " -0.03842722997069359,\n", 294 | " 0.04181297868490219,\n", 295 | " 0.08094648271799088,\n", 296 | " 1.732437522150576e-05,\n", 297 | " -0.04057943820953369,\n", 298 | " 0.012313471175730228,\n", 299 | " 0.02012881450355053,\n", 300 | " 0.011500362306833267,\n", 301 | " 0.027079368010163307,\n", 302 | " 0.0008867628639563918,\n", 303 | " -0.004841003101319075,\n", 304 | " 0.0165211521089077,\n", 305 | " 0.02827044390141964,\n", 306 | " -0.024699918925762177,\n", 307 | " 0.015545210801064968,\n", 308 | " 0.018956050276756287,\n", 309 | " -0.009883429855108261,\n", 310 | " 0.0610891617834568,\n", 311 | " 0.03722121939063072,\n", 312 | " -0.028240898624062538,\n", 313 | " -0.3214947283267975,\n", 314 | " -0.0026431416627019644,\n", 315 | " -0.05558514967560768,\n", 316 | " -0.06113176420331001,\n", 317 | " 0.013782983645796776,\n", 318 | " -0.024307632818818092,\n", 319 | " 0.02983257919549942,\n", 320 | " -0.08525410294532776,\n", 321 | " -0.034632883965969086,\n", 322 | " 0.0461745411157608,\n", 323 | " 0.00598470913246274,\n", 324 | " -0.012667699716985226,\n", 325 | " -0.054044630378484726,\n", 326 | " 0.10220249742269516,\n", 327 | " -0.014480584301054478,\n", 328 | " -0.061991237103939056,\n", 329 | " 0.021785330027341843,\n", 330 | " 0.0003505109343677759,\n", 331 | " -0.028844760730862617,\n", 332 | " -0.029320117086172104,\n", 333 | " -0.047158900648355484,\n", 334 | " 0.01175930816680193,\n", 335 | " -0.001192743773572147,\n", 336 | " -0.041614048182964325,\n", 337 | " 0.05469181016087532,\n", 338 | " 0.007678293157368898,\n", 339 | " 0.1428881734609604,\n", 340 | " 0.009676373563706875,\n", 341 | " -0.007680752780288458,\n", 342 | " -0.018874458968639374,\n", 343 | " 0.00503634475171566,\n", 344 | " -0.014481689780950546,\n", 345 | " -0.060289207845926285,\n", 346 | " -0.10472579300403595,\n", 347 | " 0.0665341466665268,\n", 348 | " -0.024205688387155533,\n", 349 | " -0.017171761021018028,\n", 350 | " -0.0329493023455143,\n", 351 | " -0.03850715607404709,\n", 352 | " 0.013526128605008125,\n", 353 | " -0.0006781163392588496,\n", 354 | " 0.06515263020992279,\n", 355 | " -0.02240077219903469,\n", 356 | " -0.030810130760073662,\n", 357 | " 0.02493714541196823,\n", 358 | " 0.028945690020918846,\n", 359 | " -0.026827389374375343,\n", 360 | " 0.016264386475086212,\n", 361 | " 0.008967147208750248,\n", 362 | " 0.0458054281771183,\n", 363 | " 0.032177768647670746,\n", 364 | " -0.023768538609147072,\n", 365 | " 0.04991832748055458,\n", 366 | " 0.07579486817121506,\n", 367 | " 0.09004918485879898,\n", 368 | " -0.059660255908966064,\n", 369 | " -0.053649112582206726,\n", 370 | " -0.0019177772337570786,\n", 371 | " -0.008793890476226807,\n", 372 | " 0.03974112868309021,\n", 373 | " 0.02185637503862381,\n", 374 | " 0.016239570453763008,\n", 375 | " -0.024728834629058838,\n", 376 | " -0.018217405304312706,\n", 377 | " -0.028360772877931595,\n", 378 | " -0.012765880674123764,\n", 379 | " -0.01621786504983902,\n", 380 | " -0.009319632314145565,\n", 381 | " 0.007931031286716461,\n", 382 | " 0.010698341764509678,\n", 383 | " 0.021840102970600128,\n", 384 | " 0.00938621535897255,\n", 385 | " 0.01828211359679699,\n", 386 | " 0.0010678662220016122,\n", 387 | " 0.0701085552573204,\n", 388 | " 0.08006846159696579,\n", 389 | " 0.03665046766400337,\n", 390 | " 0.021714597940444946,\n", 391 | " -0.06143973022699356,\n", 392 | " 0.024243537336587906,\n", 393 | " 0.002396652242168784,\n", 394 | " 0.04599449411034584,\n", 395 | " -0.015162691473960876,\n", 396 | " -0.03940795734524727,\n", 397 | " 0.02120872214436531,\n", 398 | " -0.026622343808412552,\n", 399 | " 0.026093004271388054,\n", 400 | " -0.031805261969566345,\n", 401 | " -0.006291212979704142,\n", 402 | " -0.07605791836977005,\n", 403 | " -0.04634767025709152,\n", 404 | " -5.402874739957042e-05,\n", 405 | " -0.03368206322193146,\n", 406 | " 0.04461998492479324,\n", 407 | " -0.0034680338576436043,\n", 408 | " 0.0022879522293806076,\n", 409 | " -0.29827582836151123,\n", 410 | " -0.041814174503088,\n", 411 | " -0.027332156896591187,\n", 412 | " 0.030990874394774437,\n", 413 | " -0.037341728806495667,\n", 414 | " 0.04143257811665535,\n", 415 | " -0.0005830226000398397,\n", 416 | " 0.07081376016139984,\n", 417 | " -0.039708539843559265,\n", 418 | " -0.006586560048162937,\n", 419 | " -0.017360789701342583,\n", 420 | " 0.059576865285634995,\n", 421 | " 0.027560370042920113,\n", 422 | " -0.010947047732770443,\n", 423 | " 0.04786258935928345,\n", 424 | " -0.07567749917507172,\n", 425 | " 0.015969382598996162,\n", 426 | " -0.03356342390179634,\n", 427 | " -0.0031506603118032217,\n", 428 | " -0.0314052477478981,\n", 429 | " 0.017654020339250565,\n", 430 | " 0.011964955367147923,\n", 431 | " 0.15740065276622772,\n", 432 | " -0.061965540051460266,\n", 433 | " 0.0032535248901695013,\n", 434 | " 0.016356276348233223,\n", 435 | " -0.01507490873336792,\n", 436 | " 0.050942130386829376,\n", 437 | " 0.03531096503138542,\n", 438 | " -0.010228526778519154,\n", 439 | " 0.035419780761003494,\n", 440 | " 0.018847981467843056,\n", 441 | " 0.12580406665802002,\n", 442 | " 0.0041145752184093,\n", 443 | " 0.015823187306523323,\n", 444 | " -0.01920614019036293,\n", 445 | " -0.03265044838190079,\n", 446 | " -0.045140475034713745,\n", 447 | " 0.018717152997851372,\n", 448 | " -0.056646253913640976,\n", 449 | " -0.044920023530721664,\n", 450 | " -0.031166192144155502,\n", 451 | " 0.05610523372888565,\n", 452 | " -0.00646101264283061,\n", 453 | " 0.07270033657550812,\n", 454 | " 0.04635011404752731,\n", 455 | " -0.06371420621871948,\n", 456 | " -0.03930968791246414,\n", 457 | " 0.033636074513196945,\n", 458 | " 0.06477994471788406,\n", 459 | " 0.020246583968400955,\n", 460 | " -0.05125589296221733,\n", 461 | " 0.04974193125963211,\n", 462 | " -0.002314390614628792,\n", 463 | " 0.06734129041433334,\n", 464 | " -0.05128942430019379,\n", 465 | " -0.035932883620262146,\n", 466 | " -0.005738924257457256,\n", 467 | " 0.016213826835155487,\n", 468 | " -0.06395038962364197,\n", 469 | " -0.043265871703624725,\n", 470 | " 0.06921685487031937,\n", 471 | " -0.032498423010110855,\n", 472 | " -0.01773829013109207,\n", 473 | " 0.0005499106482602656]" 474 | ] 475 | }, 476 | "execution_count": 93, 477 | "metadata": {}, 478 | "output_type": "execute_result" 479 | } 480 | ], 481 | "source": [ 482 | "hf.embed_query(\"demonitization case\")" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": 94, 488 | "metadata": {}, 489 | "outputs": [], 490 | "source": [ 491 | "langchain_dataset = json.load(open(\"dataset.json\"))['cases_list']" 492 | ] 493 | }, 494 | { 495 | "cell_type": "code", 496 | "execution_count": 95, 497 | "metadata": {}, 498 | "outputs": [], 499 | "source": [ 500 | "import re\n", 501 | "\n", 502 | "def remove_punctuation(text):\n", 503 | " no_punct_text = re.sub(r'[^\\w\\s,]', '', text)\n", 504 | " return no_punct_text.lower()" 505 | ] 506 | }, 507 | { 508 | "cell_type": "code", 509 | "execution_count": 96, 510 | "metadata": {}, 511 | "outputs": [], 512 | "source": [ 513 | "for x in langchain_dataset:\n", 514 | " for key, val in x['metadata'].items():\n", 515 | " x['metadata'][key] = remove_punctuation(str(val))\n", 516 | " for val in x['page_content']:\n", 517 | " x['page_content'] = remove_punctuation(str(x['page_content']))" 518 | ] 519 | }, 520 | { 521 | "cell_type": "code", 522 | "execution_count": 97, 523 | "metadata": {}, 524 | "outputs": [ 525 | { 526 | "data": { 527 | "text/plain": [ 528 | "{'page_content': 'the supreme court upheld the centres 2016 demonetisation scheme in a 41 majority, ruling that demonetisation was proportionate to the unions objectives and implemented reasonably',\n", 529 | " 'metadata': {'year': '2023',\n", 530 | " 'court': 'supreme court of india',\n", 531 | " 'judges': 's abdul nazeer, br gavai, as bopanna, v ramasubramanian, bv nagarathna',\n", 532 | " 'legal_topics': 'constitutional law, economic policy',\n", 533 | " 'relevant_laws': 'article 370 of the indian constitution'}}" 534 | ] 535 | }, 536 | "execution_count": 97, 537 | "metadata": {}, 538 | "output_type": "execute_result" 539 | } 540 | ], 541 | "source": [ 542 | "langchain_dataset[0]" 543 | ] 544 | }, 545 | { 546 | "cell_type": "code", 547 | "execution_count": 70, 548 | "metadata": {}, 549 | "outputs": [], 550 | "source": [ 551 | "from langchain_core.documents import Document\n", 552 | "from langchain_community.vectorstores.qdrant import Qdrant\n", 553 | "for i, x in enumerate(langchain_dataset):\n", 554 | " langchain_dataset[i] = Document(\n", 555 | " page_content=x['page_content'],\n", 556 | " metadata=x['metadata'],\n", 557 | " )\n", 558 | "\n", 559 | "\n", 560 | "vectorstore = Qdrant.from_documents(langchain_dataset, hf, \n", 561 | " location=\":memory:\", \n", 562 | " collection_name=\"langchain_legal\",)" 563 | ] 564 | }, 565 | { 566 | "cell_type": "markdown", 567 | "metadata": {}, 568 | "source": [ 569 | "## Langchain self query filter" 570 | ] 571 | }, 572 | { 573 | "cell_type": "code", 574 | "execution_count": null, 575 | "metadata": {}, 576 | "outputs": [], 577 | "source": [ 578 | "from langchain.chains.query_constructor.base import AttributeInfo\n", 579 | "\n", 580 | "metadata_field_info = [\n", 581 | " AttributeInfo(\n", 582 | " name=\"court\",\n", 583 | " description=\"The judiciary court's name.\",\n", 584 | " type=\"string\",\n", 585 | " ),\n", 586 | " AttributeInfo(\n", 587 | " name=\"year\",\n", 588 | " description=\"The year the court decision was given.\",\n", 589 | " type=\"string\",\n", 590 | " ),\n", 591 | " AttributeInfo(\n", 592 | " name=\"judges\",\n", 593 | " description=\"The list of names of the case judges.\",\n", 594 | " type=\"string\",\n", 595 | " ),\n", 596 | " AttributeInfo(\n", 597 | " name=\"legal_topics\",\n", 598 | " description=\"list of the topic names of the case\",\n", 599 | " type=\"string\",\n", 600 | " ),\n", 601 | " AttributeInfo(\n", 602 | " name=\"relevant_laws\",\n", 603 | " description=\"list of relevant laws that applied on the case decision\",\n", 604 | " type=\"string\",\n", 605 | " )\n", 606 | "]\n", 607 | "document_content_description = \"Brief summary of a court case\"" 608 | ] 609 | }, 610 | { 611 | "cell_type": "code", 612 | "execution_count": 105, 613 | "metadata": {}, 614 | "outputs": [], 615 | "source": [ 616 | "from langchain.retrievers.self_query.base import SelfQueryRetriever\n", 617 | "from langchain_groq import ChatGroq\n", 618 | "llm = ChatGroq(temperature=0,api_key=\"your-groq-key\")\n", 619 | "retriever = SelfQueryRetriever.from_llm(\n", 620 | " llm,\n", 621 | " vectorstore,\n", 622 | " document_content_description,\n", 623 | " metadata_field_info,\n", 624 | ")" 625 | ] 626 | }, 627 | { 628 | "cell_type": "code", 629 | "execution_count": 104, 630 | "metadata": {}, 631 | "outputs": [ 632 | { 633 | "data": { 634 | "text/plain": [ 635 | "[]" 636 | ] 637 | }, 638 | "execution_count": 104, 639 | "metadata": {}, 640 | "output_type": "execute_result" 641 | } 642 | ], 643 | "source": [ 644 | "retriever.invoke(\"demonetization case of Supreme court of India\")" 645 | ] 646 | }, 647 | { 648 | "cell_type": "code", 649 | "execution_count": 103, 650 | "metadata": {}, 651 | "outputs": [ 652 | { 653 | "ename": "OutputParserException", 654 | "evalue": "Parsing text\n```json\n{\n \"query\": \"\",\n \"filter\": \"eq(contains(judges, \\\"Sylvia Steiner\\\"), true)\"\n}\n```\nExplanation:\n- The query is an empty string because the user is not looking for any specific text in the content of the cases.\n- The filter uses the `eq` comparison statement to check if the `judges` attribute contains the name \"Sylvia Steiner\".\n- The `contains` function is used to check if the `judges` attribute is a string that includes the name \"Sylvia Steiner\".\n- The `true` value is used as the comparison value for the `eq` statement to check if the `contains` function returns `true`.\n raised following error:\nUnexpected token Token('COMMA', ',') at line 1, column 19.\nExpected one of: \n\t* LPAR\nPrevious tokens: [Token('CNAME', 'judges')]\n", 655 | "output_type": "error", 656 | "traceback": [ 657 | "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", 658 | "\u001b[1;31mUnexpectedCharacters\u001b[0m Traceback (most recent call last)", 659 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\lexer.py:665\u001b[0m, in \u001b[0;36mContextualLexer.lex\u001b[1;34m(self, lexer_state, parser_state)\u001b[0m\n\u001b[0;32m 664\u001b[0m lexer \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlexers[parser_state\u001b[38;5;241m.\u001b[39mposition]\n\u001b[1;32m--> 665\u001b[0m \u001b[38;5;28;01myield\u001b[39;00m \u001b[43mlexer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnext_token\u001b[49m\u001b[43m(\u001b[49m\u001b[43mlexer_state\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparser_state\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 666\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mEOFError\u001b[39;00m:\n", 660 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\lexer.py:598\u001b[0m, in \u001b[0;36mBasicLexer.next_token\u001b[1;34m(self, lex_state, parser_state)\u001b[0m\n\u001b[0;32m 597\u001b[0m allowed \u001b[38;5;241m=\u001b[39m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\u001b[39m\u001b[38;5;124m\"\u001b[39m}\n\u001b[1;32m--> 598\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m UnexpectedCharacters(lex_state\u001b[38;5;241m.\u001b[39mtext, line_ctr\u001b[38;5;241m.\u001b[39mchar_pos, line_ctr\u001b[38;5;241m.\u001b[39mline, line_ctr\u001b[38;5;241m.\u001b[39mcolumn,\n\u001b[0;32m 599\u001b[0m allowed\u001b[38;5;241m=\u001b[39mallowed, token_history\u001b[38;5;241m=\u001b[39mlex_state\u001b[38;5;241m.\u001b[39mlast_token \u001b[38;5;129;01mand\u001b[39;00m [lex_state\u001b[38;5;241m.\u001b[39mlast_token],\n\u001b[0;32m 600\u001b[0m state\u001b[38;5;241m=\u001b[39mparser_state, terminals_by_name\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mterminals_by_name)\n\u001b[0;32m 602\u001b[0m value, type_ \u001b[38;5;241m=\u001b[39m res\n", 661 | "\u001b[1;31mUnexpectedCharacters\u001b[0m: No terminal matches ',' in the current parser context, at line 1 col 19\n\neq(contains(judges, \"Sylvia Steiner\"), true)\n ^\nExpected one of: \n\t* LPAR\n\nPrevious tokens: Token('CNAME', 'judges')\n", 662 | "\nDuring handling of the above exception, another exception occurred:\n", 663 | "\u001b[1;31mUnexpectedToken\u001b[0m Traceback (most recent call last)", 664 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain\\chains\\query_constructor\\base.py:56\u001b[0m, in \u001b[0;36mStructuredQueryOutputParser.parse\u001b[1;34m(self, text)\u001b[0m\n\u001b[0;32m 55\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m---> 56\u001b[0m parsed[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfilter\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mast_parse\u001b[49m\u001b[43m(\u001b[49m\u001b[43mparsed\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfilter\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 57\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m parsed\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mlimit\u001b[39m\u001b[38;5;124m\"\u001b[39m):\n", 665 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\lark.py:658\u001b[0m, in \u001b[0;36mLark.parse\u001b[1;34m(self, text, start, on_error)\u001b[0m\n\u001b[0;32m 641\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Parse the given text, according to the options provided.\u001b[39;00m\n\u001b[0;32m 642\u001b[0m \n\u001b[0;32m 643\u001b[0m \u001b[38;5;124;03mParameters:\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 656\u001b[0m \n\u001b[0;32m 657\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m--> 658\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparser\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtext\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstart\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstart\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mon_error\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mon_error\u001b[49m\u001b[43m)\u001b[49m\n", 666 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\parser_frontends.py:104\u001b[0m, in \u001b[0;36mParsingFrontend.parse\u001b[1;34m(self, text, start, on_error)\u001b[0m\n\u001b[0;32m 103\u001b[0m stream \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_make_lexer_thread(text)\n\u001b[1;32m--> 104\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mparser\u001b[38;5;241m.\u001b[39mparse(stream, chosen_start, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkw)\n", 667 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\parsers\\lalr_parser.py:42\u001b[0m, in \u001b[0;36mLALR_Parser.parse\u001b[1;34m(self, lexer, start, on_error)\u001b[0m\n\u001b[0;32m 41\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m---> 42\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparser\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse\u001b[49m\u001b[43m(\u001b[49m\u001b[43mlexer\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstart\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 43\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m UnexpectedInput \u001b[38;5;28;01mas\u001b[39;00m e:\n", 668 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\parsers\\lalr_parser.py:88\u001b[0m, in \u001b[0;36m_Parser.parse\u001b[1;34m(self, lexer, start, value_stack, state_stack, start_interactive)\u001b[0m\n\u001b[0;32m 87\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m InteractiveParser(\u001b[38;5;28mself\u001b[39m, parser_state, parser_state\u001b[38;5;241m.\u001b[39mlexer)\n\u001b[1;32m---> 88\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse_from_state\u001b[49m\u001b[43m(\u001b[49m\u001b[43mparser_state\u001b[49m\u001b[43m)\u001b[49m\n", 669 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\parsers\\lalr_parser.py:111\u001b[0m, in \u001b[0;36m_Parser.parse_from_state\u001b[1;34m(self, state, last_token)\u001b[0m\n\u001b[0;32m 110\u001b[0m \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[1;32m--> 111\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[0;32m 112\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n", 670 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\parsers\\lalr_parser.py:100\u001b[0m, in \u001b[0;36m_Parser.parse_from_state\u001b[1;34m(self, state, last_token)\u001b[0m\n\u001b[0;32m 99\u001b[0m token \u001b[38;5;241m=\u001b[39m last_token\n\u001b[1;32m--> 100\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m token \u001b[38;5;129;01min\u001b[39;00m state\u001b[38;5;241m.\u001b[39mlexer\u001b[38;5;241m.\u001b[39mlex(state):\n\u001b[0;32m 101\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m token \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n", 671 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\lark\\lexer.py:674\u001b[0m, in \u001b[0;36mContextualLexer.lex\u001b[1;34m(self, lexer_state, parser_state)\u001b[0m\n\u001b[0;32m 673\u001b[0m token \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mroot_lexer\u001b[38;5;241m.\u001b[39mnext_token(lexer_state, parser_state)\n\u001b[1;32m--> 674\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m UnexpectedToken(token, e\u001b[38;5;241m.\u001b[39mallowed, state\u001b[38;5;241m=\u001b[39mparser_state, token_history\u001b[38;5;241m=\u001b[39m[last_token], terminals_by_name\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mroot_lexer\u001b[38;5;241m.\u001b[39mterminals_by_name)\n\u001b[0;32m 675\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m UnexpectedCharacters:\n", 672 | "\u001b[1;31mUnexpectedToken\u001b[0m: Unexpected token Token('COMMA', ',') at line 1, column 19.\nExpected one of: \n\t* LPAR\nPrevious tokens: [Token('CNAME', 'judges')]\n", 673 | "\nDuring handling of the above exception, another exception occurred:\n", 674 | "\u001b[1;31mOutputParserException\u001b[0m Traceback (most recent call last)", 675 | "Cell \u001b[1;32mIn[103], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mretriever\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minvoke\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcases where Judge Sylvia Steiner was the judge\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", 676 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\retrievers.py:194\u001b[0m, in \u001b[0;36mBaseRetriever.invoke\u001b[1;34m(self, input, config, **kwargs)\u001b[0m\n\u001b[0;32m 175\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Invoke the retriever to get relevant documents.\u001b[39;00m\n\u001b[0;32m 176\u001b[0m \n\u001b[0;32m 177\u001b[0m \u001b[38;5;124;03mMain entry point for synchronous retriever invocations.\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 191\u001b[0m \u001b[38;5;124;03m retriever.invoke(\"query\")\u001b[39;00m\n\u001b[0;32m 192\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m 193\u001b[0m config \u001b[38;5;241m=\u001b[39m ensure_config(config)\n\u001b[1;32m--> 194\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mget_relevant_documents(\n\u001b[0;32m 195\u001b[0m \u001b[38;5;28minput\u001b[39m,\n\u001b[0;32m 196\u001b[0m callbacks\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mcallbacks\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[0;32m 197\u001b[0m tags\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtags\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[0;32m 198\u001b[0m metadata\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmetadata\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[0;32m 199\u001b[0m run_name\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_name\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[0;32m 200\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[0;32m 201\u001b[0m )\n", 677 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\_api\\deprecation.py:148\u001b[0m, in \u001b[0;36mdeprecated..deprecate..warning_emitting_wrapper\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m 146\u001b[0m warned \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[0;32m 147\u001b[0m emit_warning()\n\u001b[1;32m--> 148\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m wrapped(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", 678 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\retrievers.py:323\u001b[0m, in \u001b[0;36mBaseRetriever.get_relevant_documents\u001b[1;34m(self, query, callbacks, tags, metadata, run_name, **kwargs)\u001b[0m\n\u001b[0;32m 321\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[0;32m 322\u001b[0m run_manager\u001b[38;5;241m.\u001b[39mon_retriever_error(e)\n\u001b[1;32m--> 323\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[0;32m 324\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m 325\u001b[0m run_manager\u001b[38;5;241m.\u001b[39mon_retriever_end(\n\u001b[0;32m 326\u001b[0m result,\n\u001b[0;32m 327\u001b[0m )\n", 679 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\retrievers.py:316\u001b[0m, in \u001b[0;36mBaseRetriever.get_relevant_documents\u001b[1;34m(self, query, callbacks, tags, metadata, run_name, **kwargs)\u001b[0m\n\u001b[0;32m 314\u001b[0m _kwargs \u001b[38;5;241m=\u001b[39m kwargs \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_expects_other_args \u001b[38;5;28;01melse\u001b[39;00m {}\n\u001b[0;32m 315\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_new_arg_supported:\n\u001b[1;32m--> 316\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_get_relevant_documents(\n\u001b[0;32m 317\u001b[0m query, run_manager\u001b[38;5;241m=\u001b[39mrun_manager, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m_kwargs\n\u001b[0;32m 318\u001b[0m )\n\u001b[0;32m 319\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m 320\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_get_relevant_documents(query, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m_kwargs)\n", 680 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain\\retrievers\\self_query\\base.py:202\u001b[0m, in \u001b[0;36mSelfQueryRetriever._get_relevant_documents\u001b[1;34m(self, query, run_manager)\u001b[0m\n\u001b[0;32m 191\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_get_relevant_documents\u001b[39m(\n\u001b[0;32m 192\u001b[0m \u001b[38;5;28mself\u001b[39m, query: \u001b[38;5;28mstr\u001b[39m, \u001b[38;5;241m*\u001b[39m, run_manager: CallbackManagerForRetrieverRun\n\u001b[0;32m 193\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m List[Document]:\n\u001b[0;32m 194\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"Get documents relevant for a query.\u001b[39;00m\n\u001b[0;32m 195\u001b[0m \n\u001b[0;32m 196\u001b[0m \u001b[38;5;124;03m Args:\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 200\u001b[0m \u001b[38;5;124;03m List of relevant documents\u001b[39;00m\n\u001b[0;32m 201\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[1;32m--> 202\u001b[0m structured_query \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mquery_constructor\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minvoke\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 203\u001b[0m \u001b[43m \u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mquery\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mquery\u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mconfig\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcallbacks\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mrun_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m}\u001b[49m\n\u001b[0;32m 204\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 205\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mverbose:\n\u001b[0;32m 206\u001b[0m logger\u001b[38;5;241m.\u001b[39minfo(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mGenerated Query: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mstructured_query\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n", 681 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\runnables\\base.py:4525\u001b[0m, in \u001b[0;36mRunnableBindingBase.invoke\u001b[1;34m(self, input, config, **kwargs)\u001b[0m\n\u001b[0;32m 4519\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21minvoke\u001b[39m(\n\u001b[0;32m 4520\u001b[0m \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m 4521\u001b[0m \u001b[38;5;28minput\u001b[39m: Input,\n\u001b[0;32m 4522\u001b[0m config: Optional[RunnableConfig] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m 4523\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs: Optional[Any],\n\u001b[0;32m 4524\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Output:\n\u001b[1;32m-> 4525\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbound\u001b[38;5;241m.\u001b[39minvoke(\n\u001b[0;32m 4526\u001b[0m \u001b[38;5;28minput\u001b[39m,\n\u001b[0;32m 4527\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_merge_configs(config),\n\u001b[0;32m 4528\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m{\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mkwargs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs},\n\u001b[0;32m 4529\u001b[0m )\n", 682 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\runnables\\base.py:2499\u001b[0m, in \u001b[0;36mRunnableSequence.invoke\u001b[1;34m(self, input, config)\u001b[0m\n\u001b[0;32m 2497\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 2498\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m i, step \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39msteps):\n\u001b[1;32m-> 2499\u001b[0m \u001b[38;5;28minput\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[43mstep\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minvoke\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 2500\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m 2501\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# mark each step as a child run\u001b[39;49;00m\n\u001b[0;32m 2502\u001b[0m \u001b[43m \u001b[49m\u001b[43mpatch_config\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 2503\u001b[0m \u001b[43m \u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcallbacks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrun_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_child\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43mf\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mseq:step:\u001b[39;49m\u001b[38;5;132;43;01m{\u001b[39;49;00m\u001b[43mi\u001b[49m\u001b[38;5;241;43m+\u001b[39;49m\u001b[38;5;241;43m1\u001b[39;49m\u001b[38;5;132;43;01m}\u001b[39;49;00m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 2504\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 2505\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 2506\u001b[0m \u001b[38;5;66;03m# finish the root run\u001b[39;00m\n\u001b[0;32m 2507\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n", 683 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\output_parsers\\base.py:169\u001b[0m, in \u001b[0;36mBaseOutputParser.invoke\u001b[1;34m(self, input, config)\u001b[0m\n\u001b[0;32m 165\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21minvoke\u001b[39m(\n\u001b[0;32m 166\u001b[0m \u001b[38;5;28mself\u001b[39m, \u001b[38;5;28minput\u001b[39m: Union[\u001b[38;5;28mstr\u001b[39m, BaseMessage], config: Optional[RunnableConfig] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 167\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m T:\n\u001b[0;32m 168\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, BaseMessage):\n\u001b[1;32m--> 169\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_with_config\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 170\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mlambda\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43minner_input\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse_result\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 171\u001b[0m \u001b[43m \u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatGeneration\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmessage\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minner_input\u001b[49m\u001b[43m)\u001b[49m\u001b[43m]\u001b[49m\n\u001b[0;32m 172\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 173\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m 174\u001b[0m \u001b[43m \u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 175\u001b[0m \u001b[43m \u001b[49m\u001b[43mrun_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mparser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m 176\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 177\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m 178\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_with_config(\n\u001b[0;32m 179\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m inner_input: \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mparse_result([Generation(text\u001b[38;5;241m=\u001b[39minner_input)]),\n\u001b[0;32m 180\u001b[0m \u001b[38;5;28minput\u001b[39m,\n\u001b[0;32m 181\u001b[0m config,\n\u001b[0;32m 182\u001b[0m run_type\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mparser\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 183\u001b[0m )\n", 684 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\runnables\\base.py:1626\u001b[0m, in \u001b[0;36mRunnable._call_with_config\u001b[1;34m(self, func, input, config, run_type, **kwargs)\u001b[0m\n\u001b[0;32m 1622\u001b[0m context \u001b[38;5;241m=\u001b[39m copy_context()\n\u001b[0;32m 1623\u001b[0m context\u001b[38;5;241m.\u001b[39mrun(var_child_runnable_config\u001b[38;5;241m.\u001b[39mset, child_config)\n\u001b[0;32m 1624\u001b[0m output \u001b[38;5;241m=\u001b[39m cast(\n\u001b[0;32m 1625\u001b[0m Output,\n\u001b[1;32m-> 1626\u001b[0m context\u001b[38;5;241m.\u001b[39mrun(\n\u001b[0;32m 1627\u001b[0m call_func_with_variable_args, \u001b[38;5;66;03m# type: ignore[arg-type]\u001b[39;00m\n\u001b[0;32m 1628\u001b[0m func, \u001b[38;5;66;03m# type: ignore[arg-type]\u001b[39;00m\n\u001b[0;32m 1629\u001b[0m \u001b[38;5;28minput\u001b[39m, \u001b[38;5;66;03m# type: ignore[arg-type]\u001b[39;00m\n\u001b[0;32m 1630\u001b[0m config,\n\u001b[0;32m 1631\u001b[0m run_manager,\n\u001b[0;32m 1632\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[0;32m 1633\u001b[0m ),\n\u001b[0;32m 1634\u001b[0m )\n\u001b[0;32m 1635\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[0;32m 1636\u001b[0m run_manager\u001b[38;5;241m.\u001b[39mon_chain_error(e)\n", 685 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\runnables\\config.py:347\u001b[0m, in \u001b[0;36mcall_func_with_variable_args\u001b[1;34m(func, input, config, run_manager, **kwargs)\u001b[0m\n\u001b[0;32m 345\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m run_manager \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m accepts_run_manager(func):\n\u001b[0;32m 346\u001b[0m kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_manager\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m run_manager\n\u001b[1;32m--> 347\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m func(\u001b[38;5;28minput\u001b[39m, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", 686 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\output_parsers\\base.py:170\u001b[0m, in \u001b[0;36mBaseOutputParser.invoke..\u001b[1;34m(inner_input)\u001b[0m\n\u001b[0;32m 165\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21minvoke\u001b[39m(\n\u001b[0;32m 166\u001b[0m \u001b[38;5;28mself\u001b[39m, \u001b[38;5;28minput\u001b[39m: Union[\u001b[38;5;28mstr\u001b[39m, BaseMessage], config: Optional[RunnableConfig] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 167\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m T:\n\u001b[0;32m 168\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, BaseMessage):\n\u001b[0;32m 169\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_with_config(\n\u001b[1;32m--> 170\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m inner_input: \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse_result\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 171\u001b[0m \u001b[43m \u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatGeneration\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmessage\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minner_input\u001b[49m\u001b[43m)\u001b[49m\u001b[43m]\u001b[49m\n\u001b[0;32m 172\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m,\n\u001b[0;32m 173\u001b[0m \u001b[38;5;28minput\u001b[39m,\n\u001b[0;32m 174\u001b[0m config,\n\u001b[0;32m 175\u001b[0m run_type\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mparser\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 176\u001b[0m )\n\u001b[0;32m 177\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m 178\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_with_config(\n\u001b[0;32m 179\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m inner_input: \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mparse_result([Generation(text\u001b[38;5;241m=\u001b[39minner_input)]),\n\u001b[0;32m 180\u001b[0m \u001b[38;5;28minput\u001b[39m,\n\u001b[0;32m 181\u001b[0m config,\n\u001b[0;32m 182\u001b[0m run_type\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mparser\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 183\u001b[0m )\n", 687 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain_core\\output_parsers\\base.py:221\u001b[0m, in \u001b[0;36mBaseOutputParser.parse_result\u001b[1;34m(self, result, partial)\u001b[0m\n\u001b[0;32m 208\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mparse_result\u001b[39m(\u001b[38;5;28mself\u001b[39m, result: List[Generation], \u001b[38;5;241m*\u001b[39m, partial: \u001b[38;5;28mbool\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m T:\n\u001b[0;32m 209\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"Parse a list of candidate model Generations into a specific format.\u001b[39;00m\n\u001b[0;32m 210\u001b[0m \n\u001b[0;32m 211\u001b[0m \u001b[38;5;124;03m The return value is parsed from only the first Generation in the result, which\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 219\u001b[0m \u001b[38;5;124;03m Structured output.\u001b[39;00m\n\u001b[0;32m 220\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[1;32m--> 221\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse\u001b[49m\u001b[43m(\u001b[49m\u001b[43mresult\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtext\u001b[49m\u001b[43m)\u001b[49m\n", 688 | "File \u001b[1;32mc:\\Users\\ASUS\\anaconda3\\envs\\new\\lib\\site-packages\\langchain\\chains\\query_constructor\\base.py:63\u001b[0m, in \u001b[0;36mStructuredQueryOutputParser.parse\u001b[1;34m(self, text)\u001b[0m\n\u001b[0;32m 59\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m StructuredQuery(\n\u001b[0;32m 60\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m{k: v \u001b[38;5;28;01mfor\u001b[39;00m k, v \u001b[38;5;129;01min\u001b[39;00m parsed\u001b[38;5;241m.\u001b[39mitems() \u001b[38;5;28;01mif\u001b[39;00m k \u001b[38;5;129;01min\u001b[39;00m allowed_keys}\n\u001b[0;32m 61\u001b[0m )\n\u001b[0;32m 62\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m---> 63\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m OutputParserException(\n\u001b[0;32m 64\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mParsing text\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;132;01m{\u001b[39;00mtext\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m raised following error:\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m 65\u001b[0m )\n", 689 | "\u001b[1;31mOutputParserException\u001b[0m: Parsing text\n```json\n{\n \"query\": \"\",\n \"filter\": \"eq(contains(judges, \\\"Sylvia Steiner\\\"), true)\"\n}\n```\nExplanation:\n- The query is an empty string because the user is not looking for any specific text in the content of the cases.\n- The filter uses the `eq` comparison statement to check if the `judges` attribute contains the name \"Sylvia Steiner\".\n- The `contains` function is used to check if the `judges` attribute is a string that includes the name \"Sylvia Steiner\".\n- The `true` value is used as the comparison value for the `eq` statement to check if the `contains` function returns `true`.\n raised following error:\nUnexpected token Token('COMMA', ',') at line 1, column 19.\nExpected one of: \n\t* LPAR\nPrevious tokens: [Token('CNAME', 'judges')]\n" 690 | ] 691 | } 692 | ], 693 | "source": [ 694 | "retriever.invoke(\"cases where Judge Sylvia Steiner was the judge\")" 695 | ] 696 | }, 697 | { 698 | "cell_type": "code", 699 | "execution_count": 101, 700 | "metadata": {}, 701 | "outputs": [ 702 | { 703 | "data": { 704 | "text/plain": [ 705 | "[Document(page_content=\"The Supreme Court upheld the Centre's 2016 demonetisation scheme in a 4:1 majority, ruling that demonetisation was proportionate to the Union’s objectives and implemented reasonably.\", metadata={'year': '2023', 'court': 'supreme court of india', 'judges': 's abdul nazeer, br gavai, as bopanna, v ramasubramanian, bv nagarathna', 'legal_topics': 'constitutional law, economic policy', 'relevant_laws': 'article 370 of the indian constitution', '_id': 'aaf5d793903e4a6aa2bcd68ee11ccfad', '_collection_name': 'langchain_legal'}),\n", 706 | " Document(page_content='The Supreme Court of the United Kingdom ruled that Uber drivers are workers and entitled to employment rights such as minimum wage and holiday pay.', metadata={'year': '2021', 'court': 'supreme court of the united kingdom', 'judges': 'lord reed, lord hodge, lady arden', 'legal_topics': 'employment law, gig economy', 'relevant_laws': 'employment rights act 1996', '_id': '07dc3ad5dbf243cfa6c67e3bea2589a0', '_collection_name': 'langchain_legal'}),\n", 707 | " Document(page_content='The U.S. Supreme Court invalidated a federal law prohibiting sports betting outside Nevada, holding it violated the anti-commandeering rule.', metadata={'year': '2018', 'court': 'supreme court of the united states', 'judges': 'majority opinion by justice samuel alito', 'legal_topics': 'sports law, constitutional law', 'relevant_laws': 'professional and amateur sports protection act paspa', '_id': 'cd0196513549463798f0805851bcad48', '_collection_name': 'langchain_legal'}),\n", 708 | " Document(page_content='The Supreme Court of India decriminalized consensual homosexual acts between adults, overturning a colonial-era law under Section 377 of the Indian Penal Code.', metadata={'year': '2018', 'court': 'supreme court of india', 'judges': 'chief justice dipak misra', 'legal_topics': 'human rights, lgbtq rights', 'relevant_laws': 'section 377 of the indian penal code', '_id': 'f6f893a7103f4ec1a355d80691e74cfd', '_collection_name': 'langchain_legal'})]" 709 | ] 710 | }, 711 | "execution_count": 101, 712 | "metadata": {}, 713 | "output_type": "execute_result" 714 | } 715 | ], 716 | "source": [ 717 | "retriever.invoke(\"demonetization case summary\")" 718 | ] 719 | }, 720 | { 721 | "cell_type": "code", 722 | "execution_count": 102, 723 | "metadata": {}, 724 | "outputs": [ 725 | { 726 | "data": { 727 | "text/plain": [ 728 | "[Document(page_content='The International Court of Justice ruled that Myanmar must take measures to prevent the genocide of the Rohingya people.', metadata={'year': '2020', 'court': 'international court of justice', 'judges': 'president abdulqawi yusuf', 'legal_topics': 'international law, human rights', 'relevant_laws': 'convention on the prevention and punishment of the crime of genocide', '_id': '7829c80e30eb4d31948623f6873ff46f', '_collection_name': 'langchain_legal'}),\n", 729 | " Document(page_content='The Constitutional Court of South Africa ruled that domestic workers are entitled to the same compensation benefits as other workers.', metadata={'year': '2020', 'court': 'constitutional court of south africa', 'judges': 'chief justice mogoeng mogoeng', 'legal_topics': 'labor law, equality', 'relevant_laws': 'compensation for occupational injuries and diseases act', '_id': 'bb34eeb9ce5a49f39da01e4ed3764367', '_collection_name': 'langchain_legal'})]" 730 | ] 731 | }, 732 | "execution_count": 102, 733 | "metadata": {}, 734 | "output_type": "execute_result" 735 | } 736 | ], 737 | "source": [ 738 | "# This example only specifies a filter\n", 739 | "retriever.invoke(\"cases in year 2020\")" 740 | ] 741 | }, 742 | { 743 | "cell_type": "code", 744 | "execution_count": 84, 745 | "metadata": {}, 746 | "outputs": [ 747 | { 748 | "data": { 749 | "text/plain": [ 750 | "[Document(page_content='The U.S. Supreme Court decided that same-sex marriage is a constitutional right under the Fourteenth Amendment.', metadata={'year': '2015', 'court': 'supreme court of the united states', 'judges': 'majority opinion by justice anthony kennedy', 'legal_topics': 'constitutional law, civil rights', 'relevant_laws': 'fourteenth amendment of the us constitution', '_id': '6480fdbd1a8143b383b8fa03b8748cab', '_collection_name': 'langchain_legal'}),\n", 751 | " Document(page_content=\"The U.S. Supreme Court ruled that state laws banning abortion are unconstitutional, recognizing a woman's right to privacy in making medical decisions.\", metadata={'year': '1973', 'court': 'supreme court of the united states', 'judges': 'majority opinion by justice harry blackmun', 'legal_topics': 'constitutional law, reproductive rights', 'relevant_laws': 'fourteenth amendment of the us constitution', '_id': '0b193c8b40794f6dac578dfd046fbbd1', '_collection_name': 'langchain_legal'}),\n", 752 | " Document(page_content='The U.S. Supreme Court held that the Defense of Marriage Act (DOMA) was unconstitutional as it violated the Fifth Amendment by denying federal recognition of same-sex marriages.', metadata={'year': '2013', 'court': 'supreme court of the united states', 'judges': 'majority opinion by justice anthony kennedy', 'legal_topics': 'constitutional law, lgbtq rights', 'relevant_laws': 'fifth amendment of the us constitution', '_id': '2b7a5bb46e1a4492b88d312fbc335014', '_collection_name': 'langchain_legal'}),\n", 753 | " Document(page_content='The Constitutional Court of South Africa ruled that the use of corporal punishment in the home is unconstitutional.', metadata={'year': '2019', 'court': 'constitutional court of south africa', 'judges': 'chief justice mogoeng mogoeng', 'legal_topics': 'family law, human rights', 'relevant_laws': 'south african constitution', '_id': 'b284224c8cbb4878b2b43906ef57073d', '_collection_name': 'langchain_legal'})]" 754 | ] 755 | }, 756 | "execution_count": 84, 757 | "metadata": {}, 758 | "output_type": "execute_result" 759 | } 760 | ], 761 | "source": [ 762 | "# This example only specifies a filter\n", 763 | "retriever.invoke(\"cases with topic Constitutional Law\")" 764 | ] 765 | }, 766 | { 767 | "cell_type": "code", 768 | "execution_count": null, 769 | "metadata": {}, 770 | "outputs": [], 771 | "source": [] 772 | }, 773 | { 774 | "cell_type": "markdown", 775 | "metadata": {}, 776 | "source": [ 777 | "## Qdrant Payload filter" 778 | ] 779 | }, 780 | { 781 | "cell_type": "code", 782 | "execution_count": null, 783 | "metadata": {}, 784 | "outputs": [], 785 | "source": [ 786 | "from qdrant_client.models import PointStruct\n", 787 | "from qdrant_client import QdrantClient\n", 788 | "from qdrant_client.models import Distance, VectorParams\n", 789 | "\n", 790 | "\n", 791 | "client = QdrantClient(\":memory:\")\n", 792 | "\n", 793 | "\n", 794 | "client.recreate_collection(\n", 795 | " collection_name=\"law_docs\",\n", 796 | " vectors_config=VectorParams(size=384, distance=Distance.COSINE),\n", 797 | ")\n", 798 | "\n", 799 | "# NOTE: consider splitting the data into chunks to avoid hitting the server's payload size limit\n", 800 | "# or use `upload_collection` or `upload_points` methods which handle this for you\n", 801 | "# WARNING: uploading points one-by-one is not recommended due to requests overhead\n", 802 | "client.upsert(\n", 803 | " collection_name=\"law_docs\",\n", 804 | " points=[\n", 805 | " PointStruct(\n", 806 | " id=idx,\n", 807 | " vector=hf.embed_query(element['page_content']),\n", 808 | " payload=element['metadata']\n", 809 | " )\n", 810 | " for idx, element in enumerate(dataset)\n", 811 | " ]\n", 812 | ")" 813 | ] 814 | }, 815 | { 816 | "cell_type": "code", 817 | "execution_count": 106, 818 | "metadata": {}, 819 | "outputs": [ 820 | { 821 | "data": { 822 | "text/plain": [ 823 | "([Record(id=0, payload={'year': 2023, 'court': 'Supreme Court of India', 'judges': ['S. Abdul Nazeer', 'B.R. Gavai', 'A.S. Bopanna', 'V. Ramasubramanian', 'B.V. Nagarathna'], 'legal_topics': ['Constitutional Law', 'Economic Policy'], 'relevant_laws': ['Article 370 of the Indian Constitution']}, vector=None, shard_key=None),\n", 824 | " Record(id=1, payload={'year': 2023, 'court': 'Supreme Court of the United States', 'judges': ['K.M. Joseph', 'Ajay Rastogi', 'Aniruddha Bose', 'Hrishikesh Roy', 'C.T. Ravikumar'], 'legal_topics': ['Election Law', 'Constitutional Law'], 'relevant_laws': ['Election Commission Act']}, vector=None, shard_key=None),\n", 825 | " Record(id=4, payload={'year': 2023, 'court': 'Supreme Court of India', 'judges': ['K.M. Joseph', 'Ajay Rastogi', 'Aniruddha Bose', 'Hrishikesh Roy', 'C.T. Ravikumar'], 'legal_topics': ['Health Law', 'Right to Die'], 'relevant_laws': ['Guidelines for Passive Euthanasia']}, vector=None, shard_key=None)],\n", 826 | " None)" 827 | ] 828 | }, 829 | "execution_count": 106, 830 | "metadata": {}, 831 | "output_type": "execute_result" 832 | } 833 | ], 834 | "source": [ 835 | "from qdrant_client import models\n", 836 | "client.scroll(\n", 837 | " collection_name=\"law_docs\",\n", 838 | " scroll_filter=models.Filter(\n", 839 | " must=[\n", 840 | " models.FieldCondition(\n", 841 | " key=\"year\",\n", 842 | " match=models.MatchValue(value=2023),\n", 843 | " ),\n", 844 | " # models.FieldCondition(\n", 845 | " # key=\"color\",\n", 846 | " # match=models.MatchValue(value=\"red\"),\n", 847 | " # ),\n", 848 | " ]\n", 849 | " ),\n", 850 | ")" 851 | ] 852 | }, 853 | { 854 | "cell_type": "markdown", 855 | "metadata": {}, 856 | "source": [ 857 | "## Semantic filtering" 858 | ] 859 | }, 860 | { 861 | "cell_type": "code", 862 | "execution_count": 113, 863 | "metadata": {}, 864 | "outputs": [ 865 | { 866 | "name": "stderr", 867 | "output_type": "stream", 868 | "text": [ 869 | "C:\\Users\\ASUS\\AppData\\Local\\Temp\\ipykernel_19604\\3696088665.py:12: DeprecationWarning: `recreate_collection` method is deprecated and will be removed in the future. Use `collection_exists` to check collection existence and `create_collection` instead.\n", 870 | " metadata_client.recreate_collection(\n" 871 | ] 872 | }, 873 | { 874 | "data": { 875 | "text/plain": [ 876 | "UpdateResult(operation_id=0, status=)" 877 | ] 878 | }, 879 | "execution_count": 113, 880 | "metadata": {}, 881 | "output_type": "execute_result" 882 | } 883 | ], 884 | "source": [ 885 | "metadata_fields = [x['metadata'] for x in dataset]\n", 886 | "metadata_list = []\n", 887 | "for elem in metadata_fields:\n", 888 | " s = ''\n", 889 | " for key in elem:\n", 890 | " s = s + f\"{key} : {elem[key]}\\n\"\n", 891 | " s = s.strip().lower().replace('.','').replace(\"'\",'').replace('[','').replace(']','')\n", 892 | " # s = remove_punctuation(s)\n", 893 | " metadata_list.append(s)\n", 894 | "\n", 895 | "metadata_client = QdrantClient(\":memory:\")\n", 896 | "metadata_client.recreate_collection(\n", 897 | " collection_name=\"metadata\",\n", 898 | " vectors_config=VectorParams(size=384, distance=Distance.COSINE),\n", 899 | ")\n", 900 | "\n", 901 | "metadata_client.upsert(\n", 902 | " collection_name=\"metadata\",\n", 903 | " points=[\n", 904 | " PointStruct(\n", 905 | " id=idx,\n", 906 | " vector=hf.embed_query(element),\n", 907 | " )\n", 908 | " for idx, element in enumerate(metadata_list)\n", 909 | " ]\n", 910 | ")" 911 | ] 912 | }, 913 | { 914 | "cell_type": "code", 915 | "execution_count": 118, 916 | "metadata": {}, 917 | "outputs": [], 918 | "source": [ 919 | "def find_hit_details(hit_list, hit):\n", 920 | " for i, x in enumerate(hit_list):\n", 921 | " if hit == x.id:\n", 922 | " return i\n", 923 | " return -1\n", 924 | "\n", 925 | "def semantic_filtering(text):\n", 926 | " first_level = set()\n", 927 | " second_level = set()\n", 928 | " matching_hits = {}\n", 929 | "\n", 930 | " query_vector = hf.embed_query(text)\n", 931 | " hits = client.search(\"law_docs\", query_vector, limit=5)\n", 932 | " for h in hits:\n", 933 | " first_level.add(h.id)\n", 934 | " \n", 935 | " filter_hits = metadata_client.search(\"metadata\", query_vector, limit=5)\n", 936 | " filter_hits_dict = {fh.id: fh for fh in filter_hits}\n", 937 | " for fh in filter_hits:\n", 938 | " second_level.add(fh.id)\n", 939 | " \n", 940 | " common_hits = first_level & second_level\n", 941 | " for hit in common_hits:\n", 942 | " filter_hit_detail = filter_hits_dict[hit]\n", 943 | " if filter_hit_detail.score > 0.65:\n", 944 | " matching_hits[filter_hit_detail.score] = hit\n", 945 | "\n", 946 | " sorted_matching_hits = sorted(matching_hits.items(), reverse=True)\n", 947 | " \n", 948 | " if sorted_matching_hits:\n", 949 | " print(\"semantic_filtering\")\n", 950 | " return [dataset[hit] for score, hit in sorted_matching_hits]\n", 951 | " else:\n", 952 | " print(\"No filter found\")\n", 953 | " return [dataset[hit] for hit in first_level]\n" 954 | ] 955 | }, 956 | { 957 | "cell_type": "code", 958 | "execution_count": 119, 959 | "metadata": {}, 960 | "outputs": [ 961 | { 962 | "name": "stdout", 963 | "output_type": "stream", 964 | "text": [ 965 | "semantic_filtering\n" 966 | ] 967 | }, 968 | { 969 | "data": { 970 | "text/plain": [ 971 | "[{'page_content': 'The U.S. Supreme Court held that the Defense of Marriage Act (DOMA) was unconstitutional as it violated the Fifth Amendment by denying federal recognition of same-sex marriages.',\n", 972 | " 'metadata': {'year': 2013,\n", 973 | " 'court': 'Supreme Court of the United States',\n", 974 | " 'judges': ['Majority opinion by Justice Anthony Kennedy'],\n", 975 | " 'legal_topics': ['Constitutional Law', 'LGBTQ+ Rights'],\n", 976 | " 'relevant_laws': ['Fifth Amendment of the U.S. Constitution']}},\n", 977 | " {'page_content': 'The Supreme Court of Japan upheld a law requiring married couples to share a surname, ruling it did not violate constitutional guarantees of equality.',\n", 978 | " 'metadata': {'year': 2015,\n", 979 | " 'court': 'Supreme Court of Japan',\n", 980 | " 'judges': ['Chief Justice Itsuro Terada'],\n", 981 | " 'legal_topics': ['Family Law', 'Equality'],\n", 982 | " 'relevant_laws': ['Japanese Civil Code']}}]" 983 | ] 984 | }, 985 | "execution_count": 119, 986 | "metadata": {}, 987 | "output_type": "execute_result" 988 | } 989 | ], 990 | "source": [ 991 | "# Example usage\n", 992 | "text = \"gay marriage cases\"\n", 993 | "results = semantic_filtering(text)\n", 994 | "results" 995 | ] 996 | }, 997 | { 998 | "cell_type": "code", 999 | "execution_count": 143, 1000 | "metadata": {}, 1001 | "outputs": [ 1002 | { 1003 | "name": "stdout", 1004 | "output_type": "stream", 1005 | "text": [ 1006 | "No filter found\n" 1007 | ] 1008 | }, 1009 | { 1010 | "data": { 1011 | "text/plain": [ 1012 | "[{'page_content': \"The Supreme Court upheld the Centre's 2016 demonetisation scheme in a 4:1 majority, ruling that demonetisation was proportionate to the Union’s objectives and implemented reasonably.\",\n", 1013 | " 'metadata': {'year': 2023,\n", 1014 | " 'court': 'Supreme Court of India',\n", 1015 | " 'judges': ['S. Abdul Nazeer',\n", 1016 | " 'B.R. Gavai',\n", 1017 | " 'A.S. Bopanna',\n", 1018 | " 'V. Ramasubramanian',\n", 1019 | " 'B.V. Nagarathna'],\n", 1020 | " 'legal_topics': ['Constitutional Law', 'Economic Policy'],\n", 1021 | " 'relevant_laws': ['Article 370 of the Indian Constitution']}},\n", 1022 | " {'page_content': 'In a case concerning the rights of students with disabilities, the United States Department of Justice concluded that Alabama’s foster care system discriminates against students with emotional and behavioral disabilities.',\n", 1023 | " 'metadata': {'year': 2022,\n", 1024 | " 'court': 'United States Department of Justice',\n", 1025 | " 'judges': ['Investigation by the Civil Rights Division'],\n", 1026 | " 'legal_topics': ['Disability Law', 'Education Law'],\n", 1027 | " 'relevant_laws': ['Title II of the Americans with Disabilities Act']}},\n", 1028 | " {'page_content': 'The U.S. Supreme Court ruled that a state cannot require out-of-state sellers to collect sales tax unless they have a significant presence in the state.',\n", 1029 | " 'metadata': {'year': 2018,\n", 1030 | " 'court': 'Supreme Court of the United States',\n", 1031 | " 'judges': ['Majority opinion by Justice Kennedy'],\n", 1032 | " 'legal_topics': ['Tax Law', 'Commerce Clause'],\n", 1033 | " 'relevant_laws': ['Commerce Clause of the U.S. Constitution']}},\n", 1034 | " {'page_content': 'The Supreme Court of the United Kingdom ruled that Uber drivers are workers and entitled to employment rights such as minimum wage and holiday pay.',\n", 1035 | " 'metadata': {'year': 2021,\n", 1036 | " 'court': 'Supreme Court of the United Kingdom',\n", 1037 | " 'judges': ['Lord Reed', 'Lord Hodge', 'Lady Arden'],\n", 1038 | " 'legal_topics': ['Employment Law', 'Gig Economy'],\n", 1039 | " 'relevant_laws': ['Employment Rights Act 1996']}},\n", 1040 | " {'page_content': 'The Court of Justice of the European Union ruled that data retention laws requiring telecommunications companies to retain users’ data indiscriminately are invalid.',\n", 1041 | " 'metadata': {'year': 2016,\n", 1042 | " 'court': 'Court of Justice of the European Union',\n", 1043 | " 'judges': ['Judge Koen Lenaerts'],\n", 1044 | " 'legal_topics': ['Data Privacy', 'Telecommunications Law'],\n", 1045 | " 'relevant_laws': ['Directive 2006/24/EC']}}]" 1046 | ] 1047 | }, 1048 | "execution_count": 143, 1049 | "metadata": {}, 1050 | "output_type": "execute_result" 1051 | } 1052 | ], 1053 | "source": [ 1054 | "semantic_filtering(\"cases about money\".lower())" 1055 | ] 1056 | }, 1057 | { 1058 | "cell_type": "code", 1059 | "execution_count": null, 1060 | "metadata": {}, 1061 | "outputs": [], 1062 | "source": [] 1063 | }, 1064 | { 1065 | "cell_type": "code", 1066 | "execution_count": null, 1067 | "metadata": {}, 1068 | "outputs": [], 1069 | "source": [] 1070 | } 1071 | ], 1072 | "metadata": { 1073 | "kernelspec": { 1074 | "display_name": "new", 1075 | "language": "python", 1076 | "name": "python3" 1077 | }, 1078 | "language_info": { 1079 | "codemirror_mode": { 1080 | "name": "ipython", 1081 | "version": 3 1082 | }, 1083 | "file_extension": ".py", 1084 | "mimetype": "text/x-python", 1085 | "name": "python", 1086 | "nbconvert_exporter": "python", 1087 | "pygments_lexer": "ipython3", 1088 | "version": "3.10.13" 1089 | } 1090 | }, 1091 | "nbformat": 4, 1092 | "nbformat_minor": 2 1093 | } 1094 | --------------------------------------------------------------------------------