├── .gitignore ├── README.md ├── data └── pre_merged_anime.csv ├── images └── oreilly.png ├── notebooks ├── anime_category_classification_model_freezing.ipynb ├── llama_quantization.ipynb ├── openai_fine_tuned_classification.ipynb ├── retrieval_augmented_generation.ipynb ├── semantic_search.ipynb └── simple_finetuning_bert_classification.ipynb ├── requirements.txt └── streamlit ├── .streamlit └── secrets.toml ├── openai_playground ├── .streamlit │ └── secrets.toml ├── README.md ├── app.py └── requirements.txt ├── retrieval_augmented_generation ├── .streamlit │ └── secrets.toml ├── README.md ├── app.py └── requirements.txt ├── single_prompt_example ├── .streamlit │ └── secrets.toml ├── README.md ├── app.py └── requirements.txt └── wine_prototype ├── .streamlit └── secrets.toml ├── README.md ├── app.py └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .ipynb_checkpoints/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Using Open- and Closed-Source LLMs in Real World Applications 2 | 3 | ![O'Reilly](images/oreilly.png) 4 | 5 | ## Description 6 | 7 | This repository contains Jupyter notebooks for the course ["Using Open- and Closed-Source LLMs in Real World Applications"](https://learning.oreilly.com/live-events/using-open-and-closed-source-llms-in-real-world-applications/0636920094342/) by Sinan Ozdemir. Published by Pearson, the course covers effective best practices and industry case studies in using Large Language Models (LLMs). 8 | 9 | In this course, you'll explore both open- and closed-source Large Language Models (LLMs) and learn best practices for working with them. Over the course of four interactive hours, we'll dive deep into the world of open-source LLMs like FLAN-T5 and GPT-J, as well as closed-source LLMs such as ChatGPT and Cohere. Additionally, you'll have the opportunity to discuss and analyze real-world LLM applications in various industries. 10 | 11 | This course is the second in a three-part series by Sinan Ozdemir designed for machine learning engineers and software developers who want to expand their skill set and learn how to work with LLMs like ChatGPT and FLAN-T5. The series provides practical instruction on prompt engineering, language modeling, moving LLM prototypes to production, and fine-tuning GPT models. The three live courses in the series are: 12 | 13 | 1. [LLMs, GPT, and Prompt Engineering for Developers](https://learning.oreilly.com/live-events/llms-gpt-and-prompt-engineering-for-developers/0636920094338/0636920094337/#liveEventSchedule) 14 | 15 | 2. **[Using Open- and Closed-Source LLMs in Real World Applications](https://learning.oreilly.com/live-events/using-open-and-closed-source-llms-in-real-world-applications/0636920094342/)** 16 | 17 | 3. [LLMs from Prototypes to Production](https://learning.oreilly.com/live-events/llms-from-prototypes-to-production/0636920095639/) 18 | 19 | The book [Quick Start Guide to LLMs](https://learning.oreilly.com/library/view/quick-start-guide/9780138199425) by Sinan Ozdemir is recommended as companion material for post-class reference. 20 | 21 | ### What You'll Learn 22 | 23 | - The differences between open- and closed-source LLMs 24 | - How to set up and configure development environments for both 25 | - Best practices for working with LLMs 26 | - Real-world applications across various industries 27 | 28 | This course is part of a three-course series designed for machine learning engineers and software developers. 29 | 30 | ## Table of Contents 31 | 32 | 1. [Course Set-Up](#course-set-up) 33 | 2. [Notebooks](#notebooks) 34 | 3. [Prerequisites](#prerequisites) 35 | 4. [Schedule](#schedule) 36 | 5. [Resources](#resources) 37 | 38 | ## Course Set-Up 39 | 40 | - Jupyter notebooks can be run alongside the instructor, but you can also follow along without coding by viewing pre-run notebooks here. 41 | 42 | ### Prerequisites 43 | 44 | - Experience with machine learning and proficiency in Python programming 45 | - Familiarity with NLP is helpful but not required 46 | 47 | ### Recommended Preparation 48 | 49 | - Attend the course "LLMs, GPT, and Prompt Engineering for Developers" 50 | - Read the book "Quick Start Guide to Large Language Models" 51 | 52 | 53 | ## Schedule 54 | 55 | - **Session 1**: Open-Source LLMs (80 minutes) 56 | - `simple_finetuning_bert_classification.ipynb`: [Fine-tuning BERT for classification tasks](notebooks/simple_finetuning_bert_classification.ipynb) 57 | - `anime_category_classification_model_freezing.ipynb`: [Working with anime category classification](notebooks/anime_category_classification_model_freezing.ipynb) 58 | - `distillation.ipynb`: [Notebook covering distillation techniques](https://colab.research.google.com/drive/1GO8w1gC2TRII9-aaRNaFN6mkCglm2pJa?usp=sharing) 59 | - `Llama 3 Instruction Fine-tuning` [Colab Notebook](https://colab.research.google.com/drive/1gN7jsUFQTPAj5uFrq06HcSLQSZzT7hZz?usp=sharing) 60 | - `llama_quantization.ipynb` [Testing Llama 3 Quantization](notebooks/llama_quantization.ipynb) 61 | - `rl_flan_t5_summaries.ipynb`: [Working with FLAN-T5 models using Reinforcement Learning](https://colab.research.google.com/drive/1wG8lv6drn872HNZHrT7V9kl6JIF1SXpr?usp=sharing) 62 | 63 | - **Session 2**: Closed-Source LLMs (70 minutes) 64 | - `openai_fine_tuned_classification.ipynb`: [Fine-tuning OpenAI models for classification](notebooks/openai_fine_tuned_classification.ipynb) 65 | - `semantic_search.ipynb`: [Implementing semantic search algorithms](notebooks/semantic_search.ipynb) 66 | - `retrieval_augmented_generation.ipynb`: [Building a retrieval augmented generation (RAG) system](notebooks/retrieval_augmented_generation.ipynb) 67 | - **Session 3**: Use Case Discussion (30 minutes) 68 | - Checkout out the [Streamlit](streamlit/) directory for more! 69 | 70 | For a detailed schedule, refer to the [Course Description](#description). 71 | 72 | ## Resources 73 | 74 | - [Course Website](https://learning.oreilly.com/live-events/using-open-and-closed-source-llms-in-real-world-applications/0636920094342/) 75 | - [Other Useful Links](https://learning.oreilly.com/playlists/2953f6c7-0e13-49ac-88e2-b951e11388de/) 76 | -------------------------------------------------------------------------------- /images/oreilly.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinanuozdemir/pearson-gpt-training-engineer/65fff0cb4d9311dfe764211f471734ce63ce8e14/images/oreilly.png -------------------------------------------------------------------------------- /notebooks/retrieval_augmented_generation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "id": "mqFr1pdYJUF4" 7 | }, 8 | "source": [ 9 | "# Putting the G(enerate) in RAG" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 37, 15 | "metadata": { 16 | "id": "l8aCLNE-HxjF" 17 | }, 18 | "outputs": [], 19 | "source": [ 20 | "from openai import OpenAI\n", 21 | "from datetime import datetime\n", 22 | "import hashlib\n", 23 | "import re\n", 24 | "import os\n", 25 | "from sentence_transformers import CrossEncoder\n", 26 | "\n", 27 | "\n", 28 | "from tqdm import tqdm\n", 29 | "import numpy as np\n", 30 | "from torch import nn\n", 31 | "\n", 32 | "import logging\n", 33 | "from pinecone import Pinecone, ServerlessSpec\n", 34 | "\n", 35 | "logger = logging.getLogger()\n", 36 | "logger.setLevel(logging.CRITICAL)\n" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": 44, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [ 45 | "from pydantic import BaseModel, Field\n", 46 | "from typing import List, Dict, Tuple" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": 45, 52 | "metadata": { 53 | "id": "PYDst0awyOBb" 54 | }, 55 | "outputs": [], 56 | "source": [ 57 | "# Initialize the OpenAI client with the API key from user data\n", 58 | "client = OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))\n" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": 46, 64 | "metadata": { 65 | "id": "Z-f5mP9o0ZRJ" 66 | }, 67 | "outputs": [], 68 | "source": [ 69 | "# Define a class for the Chat Language Model\n", 70 | "class ChatLLM(BaseModel):\n", 71 | " model: str = 'gpt-4o' # Default model to use\n", 72 | " temperature: float = 0.0 # Default temperature for generating responses\n", 73 | "\n", 74 | " # Method to generate a response from the model based on the provided prompt\n", 75 | " def generate(self, prompt: str, stop: List[str] = None):\n", 76 | " # Create a completion request to the OpenAI API with the given parameters\n", 77 | " response = client.chat.completions.create(\n", 78 | " model=self.model,\n", 79 | " messages=[{\"role\": \"user\", \"content\": prompt}],\n", 80 | " temperature=self.temperature,\n", 81 | " stop=stop\n", 82 | " )\n", 83 | "\n", 84 | " # Return the generated response content\n", 85 | " return response.choices[0].message.content\n" 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "execution_count": 47, 91 | "metadata": { 92 | "colab": { 93 | "base_uri": "https://localhost:8080/", 94 | "height": 35 95 | }, 96 | "id": "g4KCGB3tUYYP", 97 | "outputId": "90deb2d6-a99b-4579-ed77-0b29310f70b2" 98 | }, 99 | "outputs": [ 100 | { 101 | "data": { 102 | "text/plain": [ 103 | "'Hello! How can I assist you today?'" 104 | ] 105 | }, 106 | "execution_count": 47, 107 | "metadata": {}, 108 | "output_type": "execute_result" 109 | } 110 | ], 111 | "source": [ 112 | "c = ChatLLM()\n", 113 | "c.generate('hi')" 114 | ] 115 | }, 116 | { 117 | "cell_type": "code", 118 | "execution_count": 48, 119 | "metadata": { 120 | "colab": { 121 | "base_uri": "https://localhost:8080/" 122 | }, 123 | "id": "1zO716qOKDGv", 124 | "outputId": "9a75c204-8c24-4bb0-ec63-6caeb88865a7" 125 | }, 126 | "outputs": [ 127 | { 128 | "data": { 129 | "text/plain": [ 130 | "3072" 131 | ] 132 | }, 133 | "execution_count": 48, 134 | "metadata": {}, 135 | "output_type": "execute_result" 136 | } 137 | ], 138 | "source": [ 139 | "pinecone_key = os.environ.get('PINECONE_API_KEY')\n", 140 | "INDEX_NAME = 'semantic-search-test'\n", 141 | "ENGINE = 'text-embedding-3-large'\n", 142 | "NAMESPACE = 'default'\n", 143 | "\n", 144 | "pc = Pinecone(\n", 145 | " api_key=pinecone_key\n", 146 | ")\n", 147 | "\n", 148 | "# helper functions to get lists of embeddings from the OpenAI API\n", 149 | "def get_embedding(text, engine=ENGINE):\n", 150 | " response = client.embeddings.create(\n", 151 | " input=[text],\n", 152 | " model=engine\n", 153 | " )\n", 154 | " return response.data[0].embedding\n", 155 | "\n", 156 | "len(get_embedding('hi'))" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": { 163 | "id": "vR4aYMURKkgi" 164 | }, 165 | "outputs": [], 166 | "source": [] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": 49, 171 | "metadata": { 172 | "colab": { 173 | "base_uri": "https://localhost:8080/" 174 | }, 175 | "id": "1blAL-k0Kpcw", 176 | "outputId": "dd15a30a-7d4e-4f9e-cb6a-10ace6fb1d3c" 177 | }, 178 | "outputs": [ 179 | { 180 | "data": { 181 | "text/plain": [ 182 | "" 183 | ] 184 | }, 185 | "execution_count": 49, 186 | "metadata": {}, 187 | "output_type": "execute_result" 188 | } 189 | ], 190 | "source": [ 191 | "# Store the index as a variable\n", 192 | "index = pc.Index(name=INDEX_NAME)\n", 193 | "index" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": 50, 199 | "metadata": { 200 | "colab": { 201 | "base_uri": "https://localhost:8080/" 202 | }, 203 | "id": "kZVHwOpcKjEg", 204 | "outputId": "57b3bbe5-c71b-4394-a525-37082322e401" 205 | }, 206 | "outputs": [ 207 | { 208 | "data": { 209 | "text/plain": [ 210 | "{'date_uploaded': '2024-02-04T15:47:12.501860',\n", 211 | " 'text': 'The X-24A was a fat, short teardrop shape with vertical fins for control. It made its first, unpowered, glide flight on April 17, 1969 with Air Force Maj. Jerauld R. Gentry at the controls. Gentry also piloted its first powered flight on March 19, 1970. The craft was taken to around 45,000 feet (13.7 km) by a modified B-52 and then drop launched, then either glided down or used its rocket engine to ascend to higher altitudes before gliding down. The X-24A was flown 28 times at speeds up to 1,036 mph (1,667 km/h) and altitudes up to 71,400 feet (21.8 km).'}" 212 | ] 213 | }, 214 | "execution_count": 50, 215 | "metadata": {}, 216 | "output_type": "execute_result" 217 | } 218 | ], 219 | "source": [ 220 | "def query_from_pinecone(query, top_k=1, include_metadata=True):\n", 221 | " # get embedding from THE SAME embedder as the documents\n", 222 | " query_embedding = get_embedding(query, engine=ENGINE)\n", 223 | "\n", 224 | " return index.query(\n", 225 | " vector=query_embedding,\n", 226 | " top_k=top_k,\n", 227 | " namespace=NAMESPACE,\n", 228 | " include_metadata=include_metadata # gets the metadata (dates, text, etc)\n", 229 | " ).get('matches')\n", 230 | "\n", 231 | "query_from_pinecone('What is the X-24?')[0]['metadata']" 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": null, 237 | "metadata": { 238 | "id": "jxUMGOLDU3iB" 239 | }, 240 | "outputs": [], 241 | "source": [] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "metadata": {}, 247 | "outputs": [], 248 | "source": [] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": null, 253 | "metadata": {}, 254 | "outputs": [], 255 | "source": [] 256 | }, 257 | { 258 | "cell_type": "code", 259 | "execution_count": 60, 260 | "metadata": { 261 | "id": "P2Fg5dJ1PAOp" 262 | }, 263 | "outputs": [], 264 | "source": [ 265 | "FINAL_ANSWER_TOKEN = \"Assistant Response:\"\n", 266 | "STOP = '[END]'\n", 267 | "PROMPT_TEMPLATE = \"\"\"Today is {today} and you can retrieve information from a database. Respond the user's input as best as you can.\n", 268 | "\n", 269 | "Here is an example of the conversation format:\n", 270 | "\n", 271 | "[START]\n", 272 | "User Input: the input question you must answer\n", 273 | "Context: retrieved context from the database\n", 274 | "Context Score : a score from 0 - 1 of how strong the information is a match\n", 275 | "Assistant Thought: This context has sufficient information to answer the question.\n", 276 | "Assistant Response: your final answer to the original input question which could be I don't have sufficient information to answer the question.\n", 277 | "[END]\n", 278 | "[START]\n", 279 | "User Input: another input question you must answer\n", 280 | "Context: more retrieved context from the database\n", 281 | "Context Score : another score from 0 - 1 of how strong the information is a match\n", 282 | "Assistant Thought: This context does not have sufficient information to answer the question.\n", 283 | "Assistant Response: your final answer to the second input question which could be I don't have sufficient information to answer the question.\n", 284 | "[END]\n", 285 | "[START]\n", 286 | "User Input: another input question you must answer\n", 287 | "Context: NO CONTEXT FOUND\n", 288 | "Context Score : 0\n", 289 | "Assistant Thought: We either could not find something or we don't need to look something up\n", 290 | "Assistant Response: Reason through the best way to answer\n", 291 | "[END]\n", 292 | "\n", 293 | "Begin:\n", 294 | "\n", 295 | "{running_convo}\n", 296 | "\"\"\"\n", 297 | "\n", 298 | "class RagBot(BaseModel):\n", 299 | " llm: ChatLLM\n", 300 | " prompt_template: str = PROMPT_TEMPLATE\n", 301 | " stop_pattern: List[str] = [STOP]\n", 302 | " user_inputs: List[str] = []\n", 303 | " ai_responses: List[str] = []\n", 304 | " contexts: List[Tuple[str, float]] = []\n", 305 | " verbose: bool = False\n", 306 | " threshold: float = 0.6\n", 307 | "\n", 308 | " def query_from_pinecone(self, query, top_k=1, include_metadata=True):\n", 309 | " return query_from_pinecone(query, top_k, include_metadata)\n", 310 | "\n", 311 | " @property\n", 312 | " def running_convo(self):\n", 313 | " convo = ''\n", 314 | " for index in range(len(self.user_inputs)):\n", 315 | " convo += f'[START]\\nUser Input: {self.user_inputs[index]}\\n'\n", 316 | " convo += f'Context: {self.contexts[index][0]}\\nContext Score: {self.contexts[index][1]}\\n'\n", 317 | " if len(self.ai_responses) > index:\n", 318 | " convo += self.ai_responses[index]\n", 319 | " convo += '\\n[END]\\n'\n", 320 | " return convo.strip()\n", 321 | "\n", 322 | " def run(self, question: str):\n", 323 | " self.user_inputs.append(question)\n", 324 | " top_response = self.query_from_pinecone(question)[0]\n", 325 | " if top_response['score'] >= self.threshold:\n", 326 | " self.contexts.append(\n", 327 | " (top_response['metadata']['text'], top_response['score']))\n", 328 | " else:\n", 329 | " self.contexts.append(('NO CONTEXT FOUND', 'NONE', 0))\n", 330 | "\n", 331 | " prompt = self.prompt_template.format(\n", 332 | " today = datetime.today(),\n", 333 | " running_convo=self.running_convo\n", 334 | " )\n", 335 | " if self.verbose:\n", 336 | " print('--------')\n", 337 | " print('PROMPT')\n", 338 | " print('--------')\n", 339 | " print(prompt)\n", 340 | " print('--------')\n", 341 | " print('END PROMPT')\n", 342 | " print('--------')\n", 343 | " generated = self.llm.generate(prompt, stop=self.stop_pattern)\n", 344 | " if self.verbose:\n", 345 | " print('--------')\n", 346 | " print('GENERATED')\n", 347 | " print('--------')\n", 348 | " print(generated)\n", 349 | " print('--------')\n", 350 | " print('END GENERATED')\n", 351 | " print('--------')\n", 352 | " self.ai_responses.append(generated)\n", 353 | " if FINAL_ANSWER_TOKEN in generated:\n", 354 | " generated = generated.split(FINAL_ANSWER_TOKEN)[-1]\n", 355 | " return generated" 356 | ] 357 | }, 358 | { 359 | "cell_type": "code", 360 | "execution_count": null, 361 | "metadata": { 362 | "id": "gVl2uWv9c4ry" 363 | }, 364 | "outputs": [], 365 | "source": [] 366 | }, 367 | { 368 | "cell_type": "code", 369 | "execution_count": 84, 370 | "metadata": { 371 | "colab": { 372 | "base_uri": "https://localhost:8080/" 373 | }, 374 | "id": "fWwgHUTaXFBa", 375 | "outputId": "845ca235-7e99-4b9d-ba96-20e4c875111c" 376 | }, 377 | "outputs": [ 378 | { 379 | "name": "stdout", 380 | "output_type": "stream", 381 | "text": [ 382 | " The X-24A was an experimental aircraft with a fat, short teardrop shape and vertical fins for control. It made its first unpowered glide flight on April 17, 1969, piloted by Air Force Maj. Jerauld R. Gentry, who also piloted its first powered flight on March 19, 1970. The X-24A was drop launched from a modified B-52 at around 45,000 feet and could either glide down or use its rocket engine to ascend to higher altitudes before gliding down. It was flown 28 times, reaching speeds up to 1,036 mph and altitudes up to 71,400 feet.\n" 383 | ] 384 | } 385 | ], 386 | "source": [ 387 | "r = RagBot(llm=ChatLLM(temperature=0.0), stop_pattern=['[END]'])\n", 388 | "print(r.run('What is the X-24?'))" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": 85, 394 | "metadata": {}, 395 | "outputs": [ 396 | { 397 | "name": "stdout", 398 | "output_type": "stream", 399 | "text": [ 400 | " Barack Obama is a former President of the United States, having served two terms from January 20, 2009, to January 20, 2017. He was the first African American to hold the office. Before his presidency, he was a U.S. Senator from Illinois. He is a member of the Democratic Party.\n" 401 | ] 402 | } 403 | ], 404 | "source": [ 405 | "print(r.run('Who is Obama?'))" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": 86, 411 | "metadata": { 412 | "colab": { 413 | "base_uri": "https://localhost:8080/" 414 | }, 415 | "id": "ATrmOKUGYQCa", 416 | "outputId": "e34a9f67-0081-4751-a1ca-1389f32dc871" 417 | }, 418 | "outputs": [ 419 | { 420 | "name": "stdout", 421 | "output_type": "stream", 422 | "text": [ 423 | "[START]\n", 424 | "User Input: What is the X-24?\n", 425 | "Context: The X-24A was a fat, short teardrop shape with vertical fins for control. It made its first, unpowered, glide flight on April 17, 1969 with Air Force Maj. Jerauld R. Gentry at the controls. Gentry also piloted its first powered flight on March 19, 1970. The craft was taken to around 45,000 feet (13.7 km) by a modified B-52 and then drop launched, then either glided down or used its rocket engine to ascend to higher altitudes before gliding down. The X-24A was flown 28 times at speeds up to 1,036 mph (1,667 km/h) and altitudes up to 71,400 feet (21.8 km).\n", 426 | "Context Score: 0.686325669\n", 427 | "Assistant Thought: This context has sufficient information to answer the question.\n", 428 | "Assistant Response: The X-24A was an experimental aircraft with a fat, short teardrop shape and vertical fins for control. It made its first unpowered glide flight on April 17, 1969, piloted by Air Force Maj. Jerauld R. Gentry, who also piloted its first powered flight on March 19, 1970. The X-24A was drop launched from a modified B-52 at around 45,000 feet and could either glide down or use its rocket engine to ascend to higher altitudes before gliding down. It was flown 28 times, reaching speeds up to 1,036 mph and altitudes up to 71,400 feet.\n", 429 | "[END]\n", 430 | "[START]\n", 431 | "User Input: Who is Obama?\n", 432 | "Context: NO CONTEXT FOUND\n", 433 | "Context Score: NONE\n", 434 | "Assistant Thought: We either could not find something or we don't need to look something up.\n", 435 | "Assistant Response: Barack Obama is a former President of the United States, having served two terms from January 20, 2009, to January 20, 2017. He was the first African American to hold the office. Before his presidency, he was a U.S. Senator from Illinois. He is a member of the Democratic Party.\n", 436 | "[END]\n" 437 | ] 438 | } 439 | ], 440 | "source": [ 441 | "print(r.running_convo)" 442 | ] 443 | }, 444 | { 445 | "cell_type": "markdown", 446 | "metadata": { 447 | "id": "l8n6on6fJ_5H" 448 | }, 449 | "source": [ 450 | "# Attempting to use Llama-3 as our Generator" 451 | ] 452 | }, 453 | { 454 | "cell_type": "code", 455 | "execution_count": 64, 456 | "metadata": { 457 | "colab": { 458 | "base_uri": "https://localhost:8080/", 459 | "height": 145, 460 | "referenced_widgets": [ 461 | "307f1ab676974387b50b6d390fca6ed4", 462 | "b755ac8bc04349c39e0dc5fc6cb4ac88", 463 | "ad27a257e4f647b89d809e47d470f79f", 464 | "4999a10141364b8ab18839320bffd6ed", 465 | "862ae1541542421bbd5f359f9d0e717d", 466 | "3ff0e775916e4c8da887a117728c604c", 467 | "f710b0def2af4728aa9516a0e54eb705", 468 | "2178111bcc3240159c12e8d4b81d35df", 469 | "4b4cc248d49648539b78d6c82b389353", 470 | "a1f4feb0f44b47e1b3c6f72b8b8f8bcb", 471 | "6e8d9fbb287241ce9415bf4b88989226", 472 | "b8a19e7cda0d47ccbe8ce81a08ec6c42", 473 | "9b8aba5af6af42c0a3377c7676fc73cb", 474 | "a1bb86a49a5845c6a3bb936dfa257b38", 475 | "45a689a7024243d3a7c3d2265c272e8d", 476 | "8a468e6ee5e5483e9dfe6c5c8ff845b0", 477 | "f6e1698b96914a92abc612c511bd3ba9", 478 | "ba309009b20343e59c061c5fd7b8a150", 479 | "3d447500bd6e49b892b7a2215360b5a5", 480 | "ebfc7b152a224333bfc77090c2ef934c", 481 | "805204df415c4e2aada1366b7fec94ea", 482 | "05a4343887d947d7ba85c7e4966113ae", 483 | "327273aa89264bf3a081ce85bd9ab0f4", 484 | "3272e3e26dc14331b7cac6f0fea0bc36", 485 | "8ab1eb870c8a4942bc99db73d723f659", 486 | "050b5ef402924081847a46c0c6dc5985", 487 | "001e2bb0a89d41e9b6aa9280ad55f063", 488 | "f6844220a96c46caa1bdb3a673da534f", 489 | "051f32d2c1bf40bd9e269be3f73a74be", 490 | "9f960cba4b92495cb01a2586a622deaa", 491 | "dc84beff92194850b193fb4a5d0df46b", 492 | "111a2a7a9d0743619f22cf5311b6093a", 493 | "70766048465b44c0b3a343d3dda36482", 494 | "041d716a7a294bdfaa804935def5327b", 495 | "046182a56d2544e5a54aeefc4bc0ca9e", 496 | "755217e6353f4902bf23debc8625a734", 497 | "49584303ce4a40b0ad10e75703adcc6c", 498 | "54a02369c6b8491cb9d148401985ef96", 499 | "20d3f1a259814b6ea2d540b907eddfbd", 500 | "61b6e1279a1745fd94ba8c68c513c2c4", 501 | "5c2e730f8e694d81bd78b61364d19d28", 502 | "8dda5ca9aa9140adba5205dcb668c8a0", 503 | "e02c90ef979b40e289d0e0d5a373df49", 504 | "6a85a0cf4384496d8519ad68acef5816" 505 | ] 506 | }, 507 | "id": "2Udf6YLAPt_c", 508 | "outputId": "ea8bcf66-c4c9-4158-c843-21038f4a0eac" 509 | }, 510 | "outputs": [ 511 | { 512 | "name": "stderr", 513 | "output_type": "stream", 514 | "text": [ 515 | "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n" 516 | ] 517 | } 518 | ], 519 | "source": [ 520 | "from transformers import AutoTokenizer\n", 521 | "\n", 522 | "tokenizer = AutoTokenizer.from_pretrained(\"meta-llama/Meta-Llama-3-8B-Instruct\")" 523 | ] 524 | }, 525 | { 526 | "cell_type": "code", 527 | "execution_count": 73, 528 | "metadata": { 529 | "colab": { 530 | "base_uri": "https://localhost:8080/" 531 | }, 532 | "id": "3fHe9Od26kCe", 533 | "outputId": "924cf933-2e6b-411d-897b-9c151bbfc30f" 534 | }, 535 | "outputs": [ 536 | { 537 | "name": "stdout", 538 | "output_type": "stream", 539 | "text": [ 540 | "PROMPT:\n", 541 | "------\n", 542 | "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n", 543 | "\n", 544 | "1+1=?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n", 545 | "\n", 546 | "\n", 547 | "------\n", 548 | "RESPONSE\n", 549 | "------\n", 550 | "2\n" 551 | ] 552 | } 553 | ], 554 | "source": [ 555 | "import requests\n", 556 | "\n", 557 | "terminators = [\n", 558 | " tokenizer.eos_token_id,\n", 559 | " tokenizer.convert_tokens_to_ids(\"<|eot_id|>\"),\n", 560 | " tokenizer.convert_tokens_to_ids(\"assistant\"),\n", 561 | "\n", 562 | "]\n", 563 | "\n", 564 | "def test_prompt_llama_3_8b(prompt, suppress=False, **kwargs):\n", 565 | "\n", 566 | " API_URL = \"https://my03m9749ssz7t6h.us-east-1.aws.endpoints.huggingface.cloud\"\n", 567 | " headers = {\n", 568 | " \t\"Accept\" : \"application/json\",\n", 569 | " \t\"Authorization\": f\"Bearer {os.environ.get('HF_TOKEN')}\",\n", 570 | " \t\"Content-Type\": \"application/json\"\n", 571 | " }\n", 572 | "\n", 573 | " llama_prompt = f\"<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n\"\n", 574 | "\n", 575 | " def query(payload):\n", 576 | " \tresponse = requests.post(API_URL, headers=headers, json=payload)\n", 577 | " \treturn response.json()\n", 578 | "\n", 579 | " kwargs[\"return_text\"] = False\n", 580 | " kwargs[\"return_full_text\"] = False\n", 581 | " kwargs['max_new_tokens'] = 512\n", 582 | " kwargs['stop'] = [\"<|end_of_text|>\", \"<|eot_id|>\"]\n", 583 | "\n", 584 | " output = query({\n", 585 | " \t\"inputs\": llama_prompt,\n", 586 | " \t\"parameters\": kwargs\n", 587 | " })\n", 588 | " answer = output[0]['generated_text']\n", 589 | " if not suppress:\n", 590 | " print(f'PROMPT:\\n------\\n{llama_prompt}\\n------\\nRESPONSE\\n------\\n{answer}')\n", 591 | " else:\n", 592 | " return answer\n", 593 | "\n", 594 | "test_prompt_llama_3_8b('1+1=?')" 595 | ] 596 | }, 597 | { 598 | "cell_type": "code", 599 | "execution_count": 74, 600 | "metadata": { 601 | "colab": { 602 | "base_uri": "https://localhost:8080/", 603 | "height": 53 604 | }, 605 | "id": "EZ9CVB277JN5", 606 | "outputId": "3214a2f4-bcba-4ebc-8ae4-edad8b3aa604" 607 | }, 608 | "outputs": [ 609 | { 610 | "data": { 611 | "text/plain": [ 612 | "'2'" 613 | ] 614 | }, 615 | "execution_count": 74, 616 | "metadata": {}, 617 | "output_type": "execute_result" 618 | } 619 | ], 620 | "source": [ 621 | "test_prompt_llama_3_8b('1+1=?', suppress=True)" 622 | ] 623 | }, 624 | { 625 | "cell_type": "code", 626 | "execution_count": 75, 627 | "metadata": { 628 | "id": "0KuXAxt7Cmtt" 629 | }, 630 | "outputs": [], 631 | "source": [ 632 | "class LlamaChatLLM(ChatLLM):\n", 633 | " temperature: float = 0.3\n", 634 | " do_sample: bool = True\n", 635 | " max_new_tokens: int = 256\n", 636 | "\n", 637 | " def generate(self, prompt: str, stop: List[str] = None):\n", 638 | " response = test_prompt_llama_3_8b(prompt, suppress=True)\n", 639 | " return response" 640 | ] 641 | }, 642 | { 643 | "cell_type": "code", 644 | "execution_count": 87, 645 | "metadata": { 646 | "colab": { 647 | "base_uri": "https://localhost:8080/" 648 | }, 649 | "id": "8ERXTzqCOqiM", 650 | "outputId": "9c640e20-7aa4-4978-9963-d4507c21aaf6" 651 | }, 652 | "outputs": [ 653 | { 654 | "name": "stdout", 655 | "output_type": "stream", 656 | "text": [ 657 | " The X-24A was a experimental aircraft that made its first flight in 1969 and was flown 28 times, reaching speeds of up to 1,036 mph and altitudes of up to 71,400 feet.\n" 658 | ] 659 | } 660 | ], 661 | "source": [ 662 | "llama_rag = RagBot(llm=llama_llm, verbose=False, stop_pattern=['[END]'])\n", 663 | "print(llama_rag.run('What is the X-24?'))" 664 | ] 665 | }, 666 | { 667 | "cell_type": "markdown", 668 | "metadata": { 669 | "id": "hruk_d3dKFIV" 670 | }, 671 | "source": [ 672 | "# Attempting to use [Command-R](https://cohere.com/blog/command-r?ref=cohere-ai.ghost.io) as our Generator" 673 | ] 674 | }, 675 | { 676 | "cell_type": "code", 677 | "execution_count": null, 678 | "metadata": { 679 | "id": "BPAsW6Z09P9a" 680 | }, 681 | "outputs": [], 682 | "source": [ 683 | "# https://huggingface.co/CohereForAI/c4ai-command-r-v01-4bit" 684 | ] 685 | }, 686 | { 687 | "cell_type": "code", 688 | "execution_count": null, 689 | "metadata": { 690 | "colab": { 691 | "base_uri": "https://localhost:8080/" 692 | }, 693 | "collapsed": true, 694 | "id": "2haTwk1t-TKZ", 695 | "jupyter": { 696 | "outputs_hidden": true 697 | }, 698 | "outputId": "7e540f0e-6afa-4fde-d497-ad93b876072e" 699 | }, 700 | "outputs": [ 701 | { 702 | "name": "stdout", 703 | "output_type": "stream", 704 | "text": [ 705 | "Collecting bitsandbytes\n", 706 | " Downloading bitsandbytes-0.43.1-py3-none-manylinux_2_24_x86_64.whl (119.8 MB)\n", 707 | "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m119.8/119.8 MB\u001b[0m \u001b[31m13.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", 708 | "\u001b[?25hCollecting accelerate\n", 709 | " Downloading accelerate-0.30.1-py3-none-any.whl (302 kB)\n", 710 | "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.6/302.6 kB\u001b[0m \u001b[31m30.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", 711 | "\u001b[?25hRequirement already satisfied: torch[transformers] in /usr/local/lib/python3.10/dist-packages (2.3.0+cu121)\n", 712 | "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from bitsandbytes) (1.25.2)\n", 713 | "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (24.0)\n", 714 | "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n", 715 | "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0.1)\n", 716 | "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.23.1)\n", 717 | "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.4.3)\n", 718 | "\u001b[33mWARNING: torch 2.3.0+cu121 does not provide the extra 'transformers'\u001b[0m\u001b[33m\n", 719 | "\u001b[0mRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (3.14.0)\n", 720 | "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (4.11.0)\n", 721 | "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (1.12)\n", 722 | "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (3.3)\n", 723 | "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (3.1.4)\n", 724 | "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (2023.6.0)\n", 725 | "Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch[transformers])\n", 726 | " Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)\n", 727 | "Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch[transformers])\n", 728 | " Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)\n", 729 | "Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch[transformers])\n", 730 | " Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)\n", 731 | "Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch[transformers])\n", 732 | " Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)\n", 733 | "Collecting nvidia-cublas-cu12==12.1.3.1 (from torch[transformers])\n", 734 | " Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)\n", 735 | "Collecting nvidia-cufft-cu12==11.0.2.54 (from torch[transformers])\n", 736 | " Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)\n", 737 | "Collecting nvidia-curand-cu12==10.3.2.106 (from torch[transformers])\n", 738 | " Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)\n", 739 | "Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch[transformers])\n", 740 | " Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)\n", 741 | "Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch[transformers])\n", 742 | " Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)\n", 743 | "Collecting nvidia-nccl-cu12==2.20.5 (from torch[transformers])\n", 744 | " Using cached nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)\n", 745 | "Collecting nvidia-nvtx-cu12==12.1.105 (from torch[transformers])\n", 746 | " Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)\n", 747 | "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch[transformers]) (2.3.0)\n", 748 | "Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch[transformers])\n", 749 | " Downloading nvidia_nvjitlink_cu12-12.5.40-py3-none-manylinux2014_x86_64.whl (21.3 MB)\n", 750 | "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.3/21.3 MB\u001b[0m \u001b[31m74.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", 751 | "\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2.31.0)\n", 752 | "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (4.66.4)\n", 753 | "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch[transformers]) (2.1.5)\n", 754 | "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch[transformers]) (1.3.0)\n", 755 | "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.3.2)\n", 756 | "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.7)\n", 757 | "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2.0.7)\n", 758 | "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2024.2.2)\n", 759 | "Installing collected packages: nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusparse-cu12, nvidia-cudnn-cu12, nvidia-cusolver-cu12, bitsandbytes, accelerate\n", 760 | "Successfully installed accelerate-0.30.1 bitsandbytes-0.43.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.40 nvidia-nvtx-cu12-12.1.105\n" 761 | ] 762 | } 763 | ], 764 | "source": [ 765 | "!pip install bitsandbytes accelerate torch[transformers]" 766 | ] 767 | }, 768 | { 769 | "cell_type": "code", 770 | "execution_count": null, 771 | "metadata": { 772 | "colab": { 773 | "base_uri": "https://localhost:8080/" 774 | }, 775 | "id": "rEuX15yLcYR4", 776 | "outputId": "229f7d04-496e-4143-c9c0-20b3dd2efed5" 777 | }, 778 | "outputs": [ 779 | { 780 | "name": "stderr", 781 | "output_type": "stream", 782 | "text": [ 783 | "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n" 784 | ] 785 | }, 786 | { 787 | "name": "stdout", 788 | "output_type": "stream", 789 | "text": [ 790 | "<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|># Safety Preamble\n", 791 | "The instructions in this section override those in the task description and style guide sections. Don't answer questions that are harmful or immoral.\n", 792 | "\n", 793 | "# System Preamble\n", 794 | "## Basic Rules\n", 795 | "You are a powerful conversational AI trained by Cohere to help people. You are augmented by a number of tools, and your job is to use and consume the output of these tools to best help the user. You will see a conversation history between yourself and a user, ending with an utterance from the user. You will then see a specific instruction instructing you what kind of response to generate. When you answer the user's requests, you cite your sources in your answers, according to those instructions.\n", 796 | "\n", 797 | "# User Preamble\n", 798 | "## Task and Context\n", 799 | "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n", 800 | "\n", 801 | "## Style Guide\n", 802 | "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Whats the biggest penguin in the world?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>\n", 803 | "Document: 0\n", 804 | "title: Tall penguins\n", 805 | "text: Emperor penguins are the tallest growing up to 122 cm in height.\n", 806 | "\n", 807 | "Document: 1\n", 808 | "title: Penguin habitats\n", 809 | "text: Emperor penguins only live in Antarctica.\n", 810 | "<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>Carefully perform the following instructions, in order, starting each with a new line.\n", 811 | "Firstly, Decide which of the retrieved documents are relevant to the user's last input by writing 'Relevant Documents:' followed by comma-separated list of document numbers. If none are relevant, you should instead write 'None'.\n", 812 | "Secondly, Decide which of the retrieved documents contain facts that should be cited in a good answer to the user's last input by writing 'Cited Documents:' followed a comma-separated list of document numbers. If you dont want to cite any of them, you should instead write 'None'.\n", 813 | "Thirdly, Write 'Answer:' followed by a response to the user's last input in high quality natural english. Use the retrieved documents to help you. Do not insert any citations or grounding markup.\n", 814 | "Finally, Write 'Grounded answer:' followed by a response to the user's last input in high quality natural english. Use the symbols and to indicate when a fact comes from a document in the search result, e.g my fact for a fact from document 0.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>\n" 815 | ] 816 | } 817 | ], 818 | "source": [ 819 | "from transformers import AutoTokenizer\n", 820 | "\n", 821 | "model_id = \"CohereForAI/c4ai-command-r-v01\"\n", 822 | "tokenizer = AutoTokenizer.from_pretrained(model_id)\n", 823 | "\n", 824 | "# define conversation input:\n", 825 | "conversation = [\n", 826 | " {\"role\": \"user\", \"content\": \"Whats the biggest penguin in the world?\"}\n", 827 | "]\n", 828 | "# define documents to ground on:\n", 829 | "documents = [\n", 830 | " { \"title\": \"Tall penguins\", \"text\": \"Emperor penguins are the tallest growing up to 122 cm in height.\" },\n", 831 | " { \"title\": \"Penguin habitats\", \"text\": \"Emperor penguins only live in Antarctica.\"}\n", 832 | "]\n", 833 | "\n", 834 | "# render the tool use prompt as a string:\n", 835 | "grounded_generation_prompt = tokenizer.apply_grounded_generation_template(\n", 836 | " conversation,\n", 837 | " documents=documents,\n", 838 | " citation_mode=\"accurate\", # or \"fast\"\n", 839 | " tokenize=False,\n", 840 | " add_generation_prompt=True,\n", 841 | ")\n", 842 | "print(grounded_generation_prompt)\n" 843 | ] 844 | }, 845 | { 846 | "cell_type": "code", 847 | "execution_count": null, 848 | "metadata": { 849 | "colab": { 850 | "base_uri": "https://localhost:8080/" 851 | }, 852 | "id": "xx8IfIXR-JCA", 853 | "outputId": "ca8771cf-b7b7-4415-dd18-661b26715a1f" 854 | }, 855 | "outputs": [ 856 | { 857 | "name": "stdout", 858 | "output_type": "stream", 859 | "text": [ 860 | "torch.Size([1, 579])\n" 861 | ] 862 | } 863 | ], 864 | "source": [ 865 | "# render the tool use prompt as a string:\n", 866 | "grounded_generation_tokens = tokenizer.apply_grounded_generation_template(\n", 867 | " conversation,\n", 868 | " documents=documents,\n", 869 | " citation_mode=\"accurate\", # or \"fast\"\n", 870 | " tokenize=True,\n", 871 | " add_generation_prompt=True,\n", 872 | " return_tensors=\"pt\"\n", 873 | ")\n", 874 | "print(grounded_generation_tokens.shape)\n" 875 | ] 876 | }, 877 | { 878 | "cell_type": "code", 879 | "execution_count": null, 880 | "metadata": { 881 | "id": "NUXvujJv-N2N" 882 | }, 883 | "outputs": [], 884 | "source": [] 885 | }, 886 | { 887 | "cell_type": "code", 888 | "execution_count": null, 889 | "metadata": { 890 | "colab": { 891 | "base_uri": "https://localhost:8080/", 892 | "height": 123, 893 | "referenced_widgets": [ 894 | "af6ad6bf16e646d5a0567ddfa73c6ca5", 895 | "b61c0f80b9f94fd9bf57deabbd5495a9", 896 | "829757d3e5174cefbfae580ca6036e37", 897 | "97229a811297410db94d6aca18d1f392", 898 | "346b3544032a475fadc9c0fdc6049eab", 899 | "490d23257adb40cab16051b2982b7990", 900 | "4646ae17db21490dae6d024d1fa1d1f9", 901 | "5e9f19d874ba42d3b63b7f8b17ea73cb", 902 | "08d61ff53e88466da8c2112a0539eddf", 903 | "3b36a53161634d1ebc86bbb59b8e2a4f", 904 | "258526fc9c1d44c59100a35defc1b478" 905 | ] 906 | }, 907 | "id": "Jj92pRa8Dj1e", 908 | "outputId": "e9eb5f4e-4be4-4fa8-c2b8-8a641a4b9903" 909 | }, 910 | "outputs": [ 911 | { 912 | "name": "stderr", 913 | "output_type": "stream", 914 | "text": [ 915 | "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n", 916 | "Unused kwargs: ['_load_in_4bit', '_load_in_8bit', 'quant_method']. These kwargs are not used in .\n", 917 | "`low_cpu_mem_usage` was None, now set to True since model is quantized.\n" 918 | ] 919 | }, 920 | { 921 | "data": { 922 | "application/vnd.jupyter.widget-view+json": { 923 | "model_id": "af6ad6bf16e646d5a0567ddfa73c6ca5", 924 | "version_major": 2, 925 | "version_minor": 0 926 | }, 927 | "text/plain": [ 928 | "Loading checkpoint shards: 0%| | 0/5 [00:00=4.39.1' bitsandbytes accelerate\n", 937 | "from transformers import AutoTokenizer, AutoModelForCausalLM\n", 938 | "\n", 939 | "model_id = \"CohereForAI/c4ai-command-r-v01-4bit\"\n", 940 | "model = AutoModelForCausalLM.from_pretrained(model_id)" 941 | ] 942 | }, 943 | { 944 | "cell_type": "code", 945 | "execution_count": null, 946 | "metadata": { 947 | "colab": { 948 | "base_uri": "https://localhost:8080/" 949 | }, 950 | "id": "wJqfbO0a9Obz", 951 | "outputId": "69ed610b-7a2c-479b-b774-8ad45e6a4696" 952 | }, 953 | "outputs": [ 954 | { 955 | "name": "stderr", 956 | "output_type": "stream", 957 | "text": [ 958 | "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1659: UserWarning: You are calling .generate() with the `input_ids` being on a device type different than your model's device. `input_ids` is on cpu, whereas the model is on cuda. You may experience unexpected behaviors or slower generation. Please make sure that you have put `input_ids` to the correct device by calling for example input_ids = input_ids.to('cuda') before running `.generate()`.\n", 959 | " warnings.warn(\n" 960 | ] 961 | }, 962 | { 963 | "name": "stdout", 964 | "output_type": "stream", 965 | "text": [ 966 | "<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|># Safety Preamble\n", 967 | "The instructions in this section override those in the task description and style guide sections. Don't answer questions that are harmful or immoral.\n", 968 | "\n", 969 | "# System Preamble\n", 970 | "## Basic Rules\n", 971 | "You are a powerful conversational AI trained by Cohere to help people. You are augmented by a number of tools, and your job is to use and consume the output of these tools to best help the user. You will see a conversation history between yourself and a user, ending with an utterance from the user. You will then see a specific instruction instructing you what kind of response to generate. When you answer the user's requests, you cite your sources in your answers, according to those instructions.\n", 972 | "\n", 973 | "# User Preamble\n", 974 | "## Task and Context\n", 975 | "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n", 976 | "\n", 977 | "## Style Guide\n", 978 | "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Whats the biggest penguin in the world?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>\n", 979 | "Document: 0\n", 980 | "title: Tall penguins\n", 981 | "text: Emperor penguins are the tallest growing up to 122 cm in height.\n", 982 | "\n", 983 | "Document: 1\n", 984 | "title: Penguin habitats\n", 985 | "text: Emperor penguins only live in Antarctica.\n", 986 | "<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>Carefully perform the following instructions, in order, starting each with a new line.\n", 987 | "Firstly, Decide which of the retrieved documents are relevant to the user's last input by writing 'Relevant Documents:' followed by comma-separated list of document numbers. If none are relevant, you should instead write 'None'.\n", 988 | "Secondly, Decide which of the retrieved documents contain facts that should be cited in a good answer to the user's last input by writing 'Cited Documents:' followed a comma-separated list of document numbers. If you dont want to cite any of them, you should instead write 'None'.\n", 989 | "Thirdly, Write 'Answer:' followed by a response to the user's last input in high quality natural english. Use the retrieved documents to help you. Do not insert any citations or grounding markup.\n", 990 | "Finally, Write 'Grounded answer:' followed by a response to the user's last input in high quality natural english. Use the symbols and to indicate when a fact comes from a document in the search result, e.g my fact for a fact from document 0.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>Relevant Documents: 0,1\n", 991 | "Cited Documents: 0\n", 992 | "Answer: The tallest species of penguin in the world is the emperor penguin (Apteryx spp.). Emperor penguins can grow up to 122 cm tall.\n", 993 | "Grounded answer: The tallest species of penguin in the world is the emperor penguin (Apteryx spp.). Emperor penguins can grow up to 122 cm tall.<|END_OF_TURN_TOKEN|>\n" 994 | ] 995 | } 996 | ], 997 | "source": [ 998 | "gen_tokens = model.generate(\n", 999 | " grounded_generation_tokens,\n", 1000 | " max_new_tokens=512,\n", 1001 | " do_sample=True,\n", 1002 | " temperature=0.3,\n", 1003 | " )\n", 1004 | "\n", 1005 | "gen_text = tokenizer.decode(gen_tokens[0])\n", 1006 | "print(gen_text)\n" 1007 | ] 1008 | }, 1009 | { 1010 | "cell_type": "code", 1011 | "execution_count": null, 1012 | "metadata": { 1013 | "id": "rXfTSItZD_F_" 1014 | }, 1015 | "outputs": [], 1016 | "source": [] 1017 | }, 1018 | { 1019 | "cell_type": "code", 1020 | "execution_count": null, 1021 | "metadata": { 1022 | "colab": { 1023 | "base_uri": "https://localhost:8080/" 1024 | }, 1025 | "id": "9dwbu3ESERtF", 1026 | "outputId": "48d161cd-9d65-4404-f82b-bb05e86143bb" 1027 | }, 1028 | "outputs": [ 1029 | { 1030 | "data": { 1031 | "text/plain": [ 1032 | "3" 1033 | ] 1034 | }, 1035 | "execution_count": 17, 1036 | "metadata": {}, 1037 | "output_type": "execute_result" 1038 | } 1039 | ], 1040 | "source": [ 1041 | "def format_for_command_r(documents):\n", 1042 | " return [{'title': f'Document {index + 1}', 'text': document['metadata']['text']} for index, document in enumerate(documents)]\n", 1043 | "\n", 1044 | "command_r_docs = format_for_command_r(query_from_pinecone('I lost my card', top_k=3, include_metadata=True))\n", 1045 | "\n", 1046 | "len(command_r_docs)" 1047 | ] 1048 | }, 1049 | { 1050 | "cell_type": "code", 1051 | "execution_count": null, 1052 | "metadata": { 1053 | "colab": { 1054 | "base_uri": "https://localhost:8080/" 1055 | }, 1056 | "id": "0Ti3s4oK-I1M", 1057 | "outputId": "73cfe021-1d48-41ca-e599-3552ea4de0c4" 1058 | }, 1059 | "outputs": [ 1060 | { 1061 | "name": "stdout", 1062 | "output_type": "stream", 1063 | "text": [ 1064 | "<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|># Safety Preamble\n", 1065 | "The instructions in this section override those in the task description and style guide sections. Don't answer questions that are harmful or immoral.\n", 1066 | "\n", 1067 | "# System Preamble\n", 1068 | "## Basic Rules\n", 1069 | "You are a powerful conversational AI trained by Cohere to help people. You are augmented by a number of tools, and your job is to use and consume the output of these tools to best help the user. You will see a conversation history between yourself and a user, ending with an utterance from the user. You will then see a specific instruction instructing you what kind of response to generate. When you answer the user's requests, you cite your sources in your answers, according to those instructions.\n", 1070 | "\n", 1071 | "# User Preamble\n", 1072 | "## Task and Context\n", 1073 | "You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.\n", 1074 | "\n", 1075 | "## Style Guide\n", 1076 | "Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>I lost my card<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>\n", 1077 | "Document: 0\n", 1078 | "title: Document 1\n", 1079 | "text: \n", 1080 | "\n", 1081 | "\n", 1082 | "\n", 1083 | "You’re offline. This is a read only version of the page.\n", 1084 | "\n", 1085 | "\n", 1086 | "\n", 1087 | "\n", 1088 | "\n", 1089 | "\n", 1090 | "\n", 1091 | "\n", 1092 | "\n", 1093 | "\n", 1094 | "\n", 1095 | "\n", 1096 | "\n", 1097 | "\n", 1098 | "\n", 1099 | "\n", 1100 | "\n", 1101 | "\n", 1102 | "\n", 1103 | "\n", 1104 | "\n", 1105 | "\n", 1106 | "\n", 1107 | "\n", 1108 | "\n", 1109 | "\n", 1110 | "Skip to content\n", 1111 | "\n", 1112 | "\n", 1113 | " \n", 1114 | "\n", 1115 | "\n", 1116 | "\n", 1117 | "\n", 1118 | "\n", 1119 | "\n", 1120 | "Do you need to submit W-2s to SSA? Business Services Online registration has changed!\n", 1121 | "\n", 1122 | "\n", 1123 | "\n", 1124 | "\n", 1125 | "\n", 1126 | "\n", 1127 | "\n", 1128 | "\n", 1129 | "\n", 1130 | "\n", 1131 | "What should I do if I get a call claiming there's a problem with my Social Security number or account?\n", 1132 | "\n", 1133 | "\n", 1134 | "\n", 1135 | "Skip to main content Social Security Search Menu Español Sign in\n", 1136 | "\n", 1137 | "\n", 1138 | "\n", 1139 | "\n", 1140 | "Frequently Asked Questions\n", 1141 | "\n", 1142 | "\n", 1143 | "\n", 1144 | "\n", 1145 | "Last Modified: \n", 1146 | "\n", 1147 | "\n", 1148 | "\n", 1149 | "\n", 1150 | "\n", 1151 | "\n", 1152 | "\n", 1153 | "\n", 1154 | "\n", 1155 | "\n", 1156 | "\n", 1157 | "\n", 1158 | "\n", 1159 | "\n", 1160 | "\n", 1161 | "\n", 1162 | "\n", 1163 | "FAQ Home\n", 1164 | "\n", 1165 | "\n", 1166 | "Topics\n", 1167 | "\n", 1168 | "\r\n", 1169 | "\t\t\t\t\tKA-01735\r\n", 1170 | "\t\t\t\t\n", 1171 | "\n", 1172 | "\n", 1173 | "\n", 1174 | "\n", 1175 | "\n", 1176 | " Print\n", 1177 | "\n", 1178 | "\n", 1179 | "\n", 1180 | "How do I get a replacement Medicare card? \n", 1181 | "\n", 1182 | "\n", 1183 | "\n", 1184 | "\n", 1185 | "\n", 1186 | "\n", 1187 | "\n", 1188 | "\n", 1189 | "\n", 1190 | "\n", 1191 | "\n", 1192 | "\n", 1193 | "\n", 1194 | "\n", 1195 | "\n", 1196 | "\n", 1197 | "\n", 1198 | "\n", 1199 | "\n", 1200 | "\n", 1201 | "\n", 1202 | "\n", 1203 | "\n", 1204 | "\n", 1205 | "\n", 1206 | "Views: \n", 1207 | "\n", 1208 | "\n", 1209 | "\n", 1210 | "If your Medicare card was lost, stolen, or destroyed, you can request a replacement online at Medicare.gov.\n", 1211 | "You can print an official copy of your card from your online Medicare account \n", 1212 | "or call 1-800-MEDICARE (1-800-633-4227 TTY 1-877-486-2048) to order a replacement card to be sent in the mail.\n", 1213 | "\n", 1214 | "\n", 1215 | "\n", 1216 | "\n", 1217 | "\n", 1218 | "\n", 1219 | "\n", 1220 | "\n", 1221 | "\n", 1222 | "\n", 1223 | "\n", 1224 | "\n", 1225 | "\n", 1226 | "\n", 1227 | "\n", 1228 | "\n", 1229 | "\n", 1230 | "\n", 1231 | "Comments (0)\n", 1232 | "\n", 1233 | "\n", 1234 | "\n", 1235 | "\n", 1236 | "\n", 1237 | "\n", 1238 | "\n", 1239 | "\n", 1240 | "\n", 1241 | "\n", 1242 | "\n", 1243 | "\n", 1244 | "\n", 1245 | "\n", 1246 | "\n", 1247 | "Footer menu\n", 1248 | "\n", 1249 | "\n", 1250 | "\n", 1251 | "\n", 1252 | "\n", 1253 | "\n", 1254 | "\n", 1255 | "Give us Feedback.\n", 1256 | "\n", 1257 | "Did this answer your question?\n", 1258 | "\n", 1259 | "No\n", 1260 | "Yes\n", 1261 | "No\n", 1262 | "\n", 1263 | "Thanks for your feedback.\n", 1264 | "\n", 1265 | "\n", 1266 | "\n", 1267 | "\n", 1268 | "\n", 1269 | "\n", 1270 | "\n", 1271 | "\n", 1272 | "Document: 1\n", 1273 | "title: Document 2\n", 1274 | "text: \n", 1275 | "\n", 1276 | "\n", 1277 | "\n", 1278 | "You’re offline. This is a read only version of the page.\n", 1279 | "\n", 1280 | "\n", 1281 | "\n", 1282 | "\n", 1283 | "\n", 1284 | "\n", 1285 | "\n", 1286 | "\n", 1287 | "\n", 1288 | "\n", 1289 | "\n", 1290 | "\n", 1291 | "\n", 1292 | "\n", 1293 | "\n", 1294 | "\n", 1295 | "\n", 1296 | "\n", 1297 | "\n", 1298 | "\n", 1299 | "\n", 1300 | "\n", 1301 | "\n", 1302 | "\n", 1303 | "\n", 1304 | "\n", 1305 | "Skip to content\n", 1306 | "\n", 1307 | "\n", 1308 | " \n", 1309 | "\n", 1310 | "\n", 1311 | "\n", 1312 | "\n", 1313 | "\n", 1314 | "\n", 1315 | "Do you need to submit W-2s to SSA? Business Services Online registration has changed!\n", 1316 | "\n", 1317 | "\n", 1318 | "\n", 1319 | "\n", 1320 | "\n", 1321 | "\n", 1322 | "\n", 1323 | "\n", 1324 | "\n", 1325 | "\n", 1326 | "What should I do if I get a call claiming there's a problem with my Social Security number or account?\n", 1327 | "\n", 1328 | "\n", 1329 | "\n", 1330 | "Skip to main content Social Security Search Menu Español Sign in\n", 1331 | "\n", 1332 | "\n", 1333 | "\n", 1334 | "\n", 1335 | "Frequently Asked Questions\n", 1336 | "\n", 1337 | "\n", 1338 | "\n", 1339 | "\n", 1340 | "Last Modified: \n", 1341 | "\n", 1342 | "\n", 1343 | "\n", 1344 | "\n", 1345 | "\n", 1346 | "\n", 1347 | "\n", 1348 | "\n", 1349 | "\n", 1350 | "\n", 1351 | "\n", 1352 | "\n", 1353 | "\n", 1354 | "\n", 1355 | "\n", 1356 | "\n", 1357 | "\n", 1358 | "FAQ Home\n", 1359 | "\n", 1360 | "\n", 1361 | "Topics\n", 1362 | "\n", 1363 | "\r\n", 1364 | "\t\t\t\t\tKA-02148\r\n", 1365 | "\t\t\t\t\n", 1366 | "\n", 1367 | "\n", 1368 | "\n", 1369 | "\n", 1370 | "\n", 1371 | " Print\n", 1372 | "\n", 1373 | "\n", 1374 | "\n", 1375 | "What is TRICARE ?\n", 1376 | "\n", 1377 | "\n", 1378 | "\n", 1379 | "\n", 1380 | "\n", 1381 | "\n", 1382 | "\n", 1383 | "\n", 1384 | "\n", 1385 | "\n", 1386 | "\n", 1387 | "\n", 1388 | "\n", 1389 | "\n", 1390 | "\n", 1391 | "\n", 1392 | "\n", 1393 | "\n", 1394 | "\n", 1395 | "\n", 1396 | "\n", 1397 | "\n", 1398 | "\n", 1399 | "\n", 1400 | "\n", 1401 | "Views: \n", 1402 | "\n", 1403 | "\n", 1404 | "\n", 1405 | "TRICARE is the health care program serving military service members, retirees, their families and survivors worldwide.  \n", 1406 | "More Information\n", 1407 | "TRICARE and Recent Changes In The Law\n", 1408 | "TRICARE (Defense Health Agency), (Disclaimer).\n", 1409 | "\n", 1410 | "\n", 1411 | "\n", 1412 | "\n", 1413 | "\n", 1414 | "\n", 1415 | "\n", 1416 | "\n", 1417 | "\n", 1418 | "\n", 1419 | "\n", 1420 | "\n", 1421 | "\n", 1422 | "\n", 1423 | "\n", 1424 | "\n", 1425 | "\n", 1426 | "\n", 1427 | "Comments (0)\n", 1428 | "\n", 1429 | "\n", 1430 | "\n", 1431 | "\n", 1432 | "\n", 1433 | "\n", 1434 | "\n", 1435 | "\n", 1436 | "\n", 1437 | "\n", 1438 | "\n", 1439 | "\n", 1440 | "\n", 1441 | "\n", 1442 | "\n", 1443 | "Footer menu\n", 1444 | "\n", 1445 | "\n", 1446 | "\n", 1447 | "\n", 1448 | "\n", 1449 | "\n", 1450 | "\n", 1451 | "Give us Feedback.\n", 1452 | "\n", 1453 | "Did this answer your question?\n", 1454 | "\n", 1455 | "No\n", 1456 | "Yes\n", 1457 | "No\n", 1458 | "\n", 1459 | "Thanks for your feedback.\n", 1460 | "\n", 1461 | "\n", 1462 | "\n", 1463 | "\n", 1464 | "\n", 1465 | "\n", 1466 | "\n", 1467 | "\n", 1468 | "Document: 2\n", 1469 | "title: Document 3\n", 1470 | "text: \n", 1471 | "\n", 1472 | "\n", 1473 | "\n", 1474 | "You’re offline. This is a read only version of the page.\n", 1475 | "\n", 1476 | "\n", 1477 | "\n", 1478 | "\n", 1479 | "\n", 1480 | "\n", 1481 | "\n", 1482 | "\n", 1483 | "\n", 1484 | "\n", 1485 | "\n", 1486 | "\n", 1487 | "\n", 1488 | "\n", 1489 | "\n", 1490 | "\n", 1491 | "\n", 1492 | "\n", 1493 | "\n", 1494 | "\n", 1495 | "\n", 1496 | "\n", 1497 | "\n", 1498 | "\n", 1499 | "\n", 1500 | "\n", 1501 | "Skip to content\n", 1502 | "\n", 1503 | "\n", 1504 | " \n", 1505 | "\n", 1506 | "\n", 1507 | "\n", 1508 | "\n", 1509 | "\n", 1510 | "\n", 1511 | "Do you need to submit W-2s to SSA? Business Services Online registration has changed!\n", 1512 | "\n", 1513 | "\n", 1514 | "\n", 1515 | "\n", 1516 | "\n", 1517 | "\n", 1518 | "\n", 1519 | "\n", 1520 | "\n", 1521 | "\n", 1522 | "What should I do if I get a call claiming there's a problem with my Social Security number or account?\n", 1523 | "\n", 1524 | "\n", 1525 | "\n", 1526 | "Skip to main content Social Security Search Menu Español Sign in\n", 1527 | "\n", 1528 | "\n", 1529 | "\n", 1530 | "\n", 1531 | "Frequently Asked Questions\n", 1532 | "\n", 1533 | "\n", 1534 | "\n", 1535 | "\n", 1536 | "Last Modified: \n", 1537 | "\n", 1538 | "\n", 1539 | "\n", 1540 | "\n", 1541 | "\n", 1542 | "\n", 1543 | "\n", 1544 | "\n", 1545 | "\n", 1546 | "\n", 1547 | "\n", 1548 | "\n", 1549 | "\n", 1550 | "\n", 1551 | "\n", 1552 | "\n", 1553 | "\n", 1554 | "FAQ Home\n", 1555 | "\n", 1556 | "\n", 1557 | "Topics\n", 1558 | "\n", 1559 | "\r\n", 1560 | "\t\t\t\t\tKA-02713\r\n", 1561 | "\t\t\t\t\n", 1562 | "\n", 1563 | "\n", 1564 | "\n", 1565 | "\n", 1566 | "\n", 1567 | " Print\n", 1568 | "\n", 1569 | "\n", 1570 | "\n", 1571 | "How do I terminate my Medicare Part B (medical insurance)?\n", 1572 | "\n", 1573 | "\n", 1574 | "\n", 1575 | "\n", 1576 | "\n", 1577 | "\n", 1578 | "\n", 1579 | "\n", 1580 | "\n", 1581 | "\n", 1582 | "\n", 1583 | "\n", 1584 | "\n", 1585 | "\n", 1586 | "\n", 1587 | "\n", 1588 | "\n", 1589 | "\n", 1590 | "\n", 1591 | "\n", 1592 | "\n", 1593 | "\n", 1594 | "\n", 1595 | "\n", 1596 | "\n", 1597 | "Views: \n", 1598 | "\n", 1599 | "\n", 1600 | "\n", 1601 | "You can voluntarily terminate your Medicare Part B (Medical Insurance). However, you may need to have a personal interview with \n", 1602 | " us to review the risks of dropping coverage and for assistance with your request. To find out more about how to terminate Medicare Part B or to schedule a personal interview, contact us at 1-800-772-1213 (TTY: 1-800-325-0778) or visit your nearest Social Security office.\n", 1603 | "If you send us a request to terminate your Part B and then change your mind, you must send a request to cancel the request of termination before the date that Part B ends.\n", 1604 | "For additional information, go to the How to drop Part A & Part B page.\n", 1605 | "\n", 1606 | "\n", 1607 | "\n", 1608 | "\n", 1609 | "\n", 1610 | "\n", 1611 | "\n", 1612 | "\n", 1613 | "\n", 1614 | "\n", 1615 | "\n", 1616 | "\n", 1617 | "\n", 1618 | "\n", 1619 | "\n", 1620 | "\n", 1621 | "\n", 1622 | "\n", 1623 | "Comments (0)\n", 1624 | "\n", 1625 | "\n", 1626 | "\n", 1627 | "\n", 1628 | "\n", 1629 | "\n", 1630 | "\n", 1631 | "\n", 1632 | "\n", 1633 | "\n", 1634 | "\n", 1635 | "\n", 1636 | "\n", 1637 | "\n", 1638 | "\n", 1639 | "Footer menu\n", 1640 | "\n", 1641 | "\n", 1642 | "\n", 1643 | "\n", 1644 | "\n", 1645 | "\n", 1646 | "\n", 1647 | "Give us Feedback.\n", 1648 | "\n", 1649 | "Did this answer your question?\n", 1650 | "\n", 1651 | "No\n", 1652 | "Yes\n", 1653 | "No\n", 1654 | "\n", 1655 | "Thanks for your feedback.\n", 1656 | "\n", 1657 | "\n", 1658 | "\n", 1659 | "\n", 1660 | "\n", 1661 | "\n", 1662 | "\n", 1663 | "<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>Carefully perform the following instructions, in order, starting each with a new line.\n", 1664 | "Firstly, Decide which of the retrieved documents are relevant to the user's last input by writing 'Relevant Documents:' followed by comma-separated list of document numbers. If none are relevant, you should instead write 'None'.\n", 1665 | "Secondly, Decide which of the retrieved documents contain facts that should be cited in a good answer to the user's last input by writing 'Cited Documents:' followed a comma-separated list of document numbers. If you dont want to cite any of them, you should instead write 'None'.\n", 1666 | "Finally, Write 'Grounded answer:' followed by a response to the user's last input in high quality natural english. Use the symbols and to indicate when a fact comes from a document in the search result, e.g my fact for a fact from document 0.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>Relevant Documents: 0\n", 1667 | "Cited Documents: 0\n", 1668 | "Grounded answer: If you've lost your Medicare card, you can request a replacement online at Medicare.gov. You can access an official copy from your online Medicare account, or call 1-800-MEDICARE to have a replacement mailed to you.<|END_OF_TURN_TOKEN|>\n" 1669 | ] 1670 | } 1671 | ], 1672 | "source": [ 1673 | "# define conversation input:\n", 1674 | "conversation = [\n", 1675 | " {\"role\": \"user\", \"content\": \"I lost my card\"}\n", 1676 | "]\n", 1677 | "\n", 1678 | "# render the tool use prompt as a string:\n", 1679 | "grounded_generation_tokens = tokenizer.apply_grounded_generation_template(\n", 1680 | " conversation,\n", 1681 | " documents=command_r_docs,\n", 1682 | " citation_mode=\"citation\",\n", 1683 | " tokenize=True,\n", 1684 | " add_generation_prompt=True,\n", 1685 | " return_tensors=\"pt\"\n", 1686 | ")\n", 1687 | "\n", 1688 | "gen_tokens = model.generate(\n", 1689 | " grounded_generation_tokens,\n", 1690 | " max_new_tokens=512,\n", 1691 | " do_sample=True,\n", 1692 | " temperature=0.3,\n", 1693 | " )\n", 1694 | "\n", 1695 | "gen_text = tokenizer.decode(gen_tokens[0])\n", 1696 | "print(gen_text)\n" 1697 | ] 1698 | }, 1699 | { 1700 | "cell_type": "code", 1701 | "execution_count": null, 1702 | "metadata": { 1703 | "id": "xICDFTOOE1Ee" 1704 | }, 1705 | "outputs": [], 1706 | "source": [] 1707 | } 1708 | ], 1709 | "metadata": { 1710 | "accelerator": "GPU", 1711 | "colab": { 1712 | "gpuType": "A100", 1713 | "machine_shape": "hm", 1714 | "provenance": [] 1715 | }, 1716 | "kernelspec": { 1717 | "display_name": "Python 3 (ipykernel)", 1718 | "language": "python", 1719 | "name": "python3" 1720 | }, 1721 | "language_info": { 1722 | "codemirror_mode": { 1723 | "name": "ipython", 1724 | "version": 3 1725 | }, 1726 | "file_extension": ".py", 1727 | "mimetype": "text/x-python", 1728 | "name": "python", 1729 | "nbconvert_exporter": "python", 1730 | "pygments_lexer": "ipython3", 1731 | "version": "3.11.5" 1732 | }, 1733 | "widgets": { 1734 | "application/vnd.jupyter.widget-state+json": { 1735 | "001e2bb0a89d41e9b6aa9280ad55f063": { 1736 | "model_module": "@jupyter-widgets/base", 1737 | "model_module_version": "1.2.0", 1738 | "model_name": "LayoutModel", 1739 | "state": { 1740 | "_model_module": "@jupyter-widgets/base", 1741 | "_model_module_version": "1.2.0", 1742 | "_model_name": "LayoutModel", 1743 | "_view_count": null, 1744 | "_view_module": "@jupyter-widgets/base", 1745 | "_view_module_version": "1.2.0", 1746 | "_view_name": "LayoutView", 1747 | "align_content": null, 1748 | "align_items": null, 1749 | "align_self": null, 1750 | "border": null, 1751 | "bottom": null, 1752 | "display": null, 1753 | "flex": null, 1754 | "flex_flow": null, 1755 | "grid_area": null, 1756 | "grid_auto_columns": null, 1757 | "grid_auto_flow": null, 1758 | "grid_auto_rows": null, 1759 | "grid_column": null, 1760 | "grid_gap": null, 1761 | "grid_row": null, 1762 | "grid_template_areas": null, 1763 | "grid_template_columns": null, 1764 | "grid_template_rows": null, 1765 | "height": null, 1766 | "justify_content": null, 1767 | "justify_items": null, 1768 | "left": null, 1769 | "margin": null, 1770 | "max_height": null, 1771 | "max_width": null, 1772 | "min_height": null, 1773 | "min_width": null, 1774 | "object_fit": null, 1775 | "object_position": null, 1776 | "order": null, 1777 | "overflow": null, 1778 | "overflow_x": null, 1779 | "overflow_y": null, 1780 | "padding": null, 1781 | "right": null, 1782 | "top": null, 1783 | "visibility": null, 1784 | "width": null 1785 | } 1786 | }, 1787 | "041d716a7a294bdfaa804935def5327b": { 1788 | "model_module": "@jupyter-widgets/controls", 1789 | "model_module_version": "1.5.0", 1790 | "model_name": "HBoxModel", 1791 | "state": { 1792 | "_dom_classes": [], 1793 | "_model_module": "@jupyter-widgets/controls", 1794 | "_model_module_version": "1.5.0", 1795 | "_model_name": "HBoxModel", 1796 | "_view_count": null, 1797 | "_view_module": "@jupyter-widgets/controls", 1798 | "_view_module_version": "1.5.0", 1799 | "_view_name": "HBoxView", 1800 | "box_style": "", 1801 | "children": [ 1802 | "IPY_MODEL_046182a56d2544e5a54aeefc4bc0ca9e", 1803 | "IPY_MODEL_755217e6353f4902bf23debc8625a734", 1804 | "IPY_MODEL_49584303ce4a40b0ad10e75703adcc6c" 1805 | ], 1806 | "layout": "IPY_MODEL_54a02369c6b8491cb9d148401985ef96" 1807 | } 1808 | }, 1809 | "046182a56d2544e5a54aeefc4bc0ca9e": { 1810 | "model_module": "@jupyter-widgets/controls", 1811 | "model_module_version": "1.5.0", 1812 | "model_name": "HTMLModel", 1813 | "state": { 1814 | "_dom_classes": [], 1815 | "_model_module": "@jupyter-widgets/controls", 1816 | "_model_module_version": "1.5.0", 1817 | "_model_name": "HTMLModel", 1818 | "_view_count": null, 1819 | "_view_module": "@jupyter-widgets/controls", 1820 | "_view_module_version": "1.5.0", 1821 | "_view_name": "HTMLView", 1822 | "description": "", 1823 | "description_tooltip": null, 1824 | "layout": "IPY_MODEL_20d3f1a259814b6ea2d540b907eddfbd", 1825 | "placeholder": "​", 1826 | "style": "IPY_MODEL_61b6e1279a1745fd94ba8c68c513c2c4", 1827 | "value": "special_tokens_map.json: 100%" 1828 | } 1829 | }, 1830 | "050b5ef402924081847a46c0c6dc5985": { 1831 | "model_module": "@jupyter-widgets/controls", 1832 | "model_module_version": "1.5.0", 1833 | "model_name": "HTMLModel", 1834 | "state": { 1835 | "_dom_classes": [], 1836 | "_model_module": "@jupyter-widgets/controls", 1837 | "_model_module_version": "1.5.0", 1838 | "_model_name": "HTMLModel", 1839 | "_view_count": null, 1840 | "_view_module": "@jupyter-widgets/controls", 1841 | "_view_module_version": "1.5.0", 1842 | "_view_name": "HTMLView", 1843 | "description": "", 1844 | "description_tooltip": null, 1845 | "layout": "IPY_MODEL_111a2a7a9d0743619f22cf5311b6093a", 1846 | "placeholder": "​", 1847 | "style": "IPY_MODEL_70766048465b44c0b3a343d3dda36482", 1848 | "value": " 21.0/21.0 [00:00<00:00, 1.27kB/s]" 1849 | } 1850 | }, 1851 | "051f32d2c1bf40bd9e269be3f73a74be": { 1852 | "model_module": "@jupyter-widgets/controls", 1853 | "model_module_version": "1.5.0", 1854 | "model_name": "DescriptionStyleModel", 1855 | "state": { 1856 | "_model_module": "@jupyter-widgets/controls", 1857 | "_model_module_version": "1.5.0", 1858 | "_model_name": "DescriptionStyleModel", 1859 | "_view_count": null, 1860 | "_view_module": "@jupyter-widgets/base", 1861 | "_view_module_version": "1.2.0", 1862 | "_view_name": "StyleView", 1863 | "description_width": "" 1864 | } 1865 | }, 1866 | "05a4343887d947d7ba85c7e4966113ae": { 1867 | "model_module": "@jupyter-widgets/controls", 1868 | "model_module_version": "1.5.0", 1869 | "model_name": "DescriptionStyleModel", 1870 | "state": { 1871 | "_model_module": "@jupyter-widgets/controls", 1872 | "_model_module_version": "1.5.0", 1873 | "_model_name": "DescriptionStyleModel", 1874 | "_view_count": null, 1875 | "_view_module": "@jupyter-widgets/base", 1876 | "_view_module_version": "1.2.0", 1877 | "_view_name": "StyleView", 1878 | "description_width": "" 1879 | } 1880 | }, 1881 | "08d61ff53e88466da8c2112a0539eddf": { 1882 | "model_module": "@jupyter-widgets/controls", 1883 | "model_module_version": "1.5.0", 1884 | "model_name": "ProgressStyleModel", 1885 | "state": { 1886 | "_model_module": "@jupyter-widgets/controls", 1887 | "_model_module_version": "1.5.0", 1888 | "_model_name": "ProgressStyleModel", 1889 | "_view_count": null, 1890 | "_view_module": "@jupyter-widgets/base", 1891 | "_view_module_version": "1.2.0", 1892 | "_view_name": "StyleView", 1893 | "bar_color": null, 1894 | "description_width": "" 1895 | } 1896 | }, 1897 | "111a2a7a9d0743619f22cf5311b6093a": { 1898 | "model_module": "@jupyter-widgets/base", 1899 | "model_module_version": "1.2.0", 1900 | "model_name": "LayoutModel", 1901 | "state": { 1902 | "_model_module": "@jupyter-widgets/base", 1903 | "_model_module_version": "1.2.0", 1904 | "_model_name": "LayoutModel", 1905 | "_view_count": null, 1906 | "_view_module": "@jupyter-widgets/base", 1907 | "_view_module_version": "1.2.0", 1908 | "_view_name": "LayoutView", 1909 | "align_content": null, 1910 | "align_items": null, 1911 | "align_self": null, 1912 | "border": null, 1913 | "bottom": null, 1914 | "display": null, 1915 | "flex": null, 1916 | "flex_flow": null, 1917 | "grid_area": null, 1918 | "grid_auto_columns": null, 1919 | "grid_auto_flow": null, 1920 | "grid_auto_rows": null, 1921 | "grid_column": null, 1922 | "grid_gap": null, 1923 | "grid_row": null, 1924 | "grid_template_areas": null, 1925 | "grid_template_columns": null, 1926 | "grid_template_rows": null, 1927 | "height": null, 1928 | "justify_content": null, 1929 | "justify_items": null, 1930 | "left": null, 1931 | "margin": null, 1932 | "max_height": null, 1933 | "max_width": null, 1934 | "min_height": null, 1935 | "min_width": null, 1936 | "object_fit": null, 1937 | "object_position": null, 1938 | "order": null, 1939 | "overflow": null, 1940 | "overflow_x": null, 1941 | "overflow_y": null, 1942 | "padding": null, 1943 | "right": null, 1944 | "top": null, 1945 | "visibility": null, 1946 | "width": null 1947 | } 1948 | }, 1949 | "20d3f1a259814b6ea2d540b907eddfbd": { 1950 | "model_module": "@jupyter-widgets/base", 1951 | "model_module_version": "1.2.0", 1952 | "model_name": "LayoutModel", 1953 | "state": { 1954 | "_model_module": "@jupyter-widgets/base", 1955 | "_model_module_version": "1.2.0", 1956 | "_model_name": "LayoutModel", 1957 | "_view_count": null, 1958 | "_view_module": "@jupyter-widgets/base", 1959 | "_view_module_version": "1.2.0", 1960 | "_view_name": "LayoutView", 1961 | "align_content": null, 1962 | "align_items": null, 1963 | "align_self": null, 1964 | "border": null, 1965 | "bottom": null, 1966 | "display": null, 1967 | "flex": null, 1968 | "flex_flow": null, 1969 | "grid_area": null, 1970 | "grid_auto_columns": null, 1971 | "grid_auto_flow": null, 1972 | "grid_auto_rows": null, 1973 | "grid_column": null, 1974 | "grid_gap": null, 1975 | "grid_row": null, 1976 | "grid_template_areas": null, 1977 | "grid_template_columns": null, 1978 | "grid_template_rows": null, 1979 | "height": null, 1980 | "justify_content": null, 1981 | "justify_items": null, 1982 | "left": null, 1983 | "margin": null, 1984 | "max_height": null, 1985 | "max_width": null, 1986 | "min_height": null, 1987 | "min_width": null, 1988 | "object_fit": null, 1989 | "object_position": null, 1990 | "order": null, 1991 | "overflow": null, 1992 | "overflow_x": null, 1993 | "overflow_y": null, 1994 | "padding": null, 1995 | "right": null, 1996 | "top": null, 1997 | "visibility": null, 1998 | "width": null 1999 | } 2000 | }, 2001 | "2178111bcc3240159c12e8d4b81d35df": { 2002 | "model_module": "@jupyter-widgets/base", 2003 | "model_module_version": "1.2.0", 2004 | "model_name": "LayoutModel", 2005 | "state": { 2006 | "_model_module": "@jupyter-widgets/base", 2007 | "_model_module_version": "1.2.0", 2008 | "_model_name": "LayoutModel", 2009 | "_view_count": null, 2010 | "_view_module": "@jupyter-widgets/base", 2011 | "_view_module_version": "1.2.0", 2012 | "_view_name": "LayoutView", 2013 | "align_content": null, 2014 | "align_items": null, 2015 | "align_self": null, 2016 | "border": null, 2017 | "bottom": null, 2018 | "display": null, 2019 | "flex": null, 2020 | "flex_flow": null, 2021 | "grid_area": null, 2022 | "grid_auto_columns": null, 2023 | "grid_auto_flow": null, 2024 | "grid_auto_rows": null, 2025 | "grid_column": null, 2026 | "grid_gap": null, 2027 | "grid_row": null, 2028 | "grid_template_areas": null, 2029 | "grid_template_columns": null, 2030 | "grid_template_rows": null, 2031 | "height": null, 2032 | "justify_content": null, 2033 | "justify_items": null, 2034 | "left": null, 2035 | "margin": null, 2036 | "max_height": null, 2037 | "max_width": null, 2038 | "min_height": null, 2039 | "min_width": null, 2040 | "object_fit": null, 2041 | "object_position": null, 2042 | "order": null, 2043 | "overflow": null, 2044 | "overflow_x": null, 2045 | "overflow_y": null, 2046 | "padding": null, 2047 | "right": null, 2048 | "top": null, 2049 | "visibility": null, 2050 | "width": null 2051 | } 2052 | }, 2053 | "258526fc9c1d44c59100a35defc1b478": { 2054 | "model_module": "@jupyter-widgets/controls", 2055 | "model_module_version": "1.5.0", 2056 | "model_name": "DescriptionStyleModel", 2057 | "state": { 2058 | "_model_module": "@jupyter-widgets/controls", 2059 | "_model_module_version": "1.5.0", 2060 | "_model_name": "DescriptionStyleModel", 2061 | "_view_count": null, 2062 | "_view_module": "@jupyter-widgets/base", 2063 | "_view_module_version": "1.2.0", 2064 | "_view_name": "StyleView", 2065 | "description_width": "" 2066 | } 2067 | }, 2068 | "307f1ab676974387b50b6d390fca6ed4": { 2069 | "model_module": "@jupyter-widgets/controls", 2070 | "model_module_version": "1.5.0", 2071 | "model_name": "HBoxModel", 2072 | "state": { 2073 | "_dom_classes": [], 2074 | "_model_module": "@jupyter-widgets/controls", 2075 | "_model_module_version": "1.5.0", 2076 | "_model_name": "HBoxModel", 2077 | "_view_count": null, 2078 | "_view_module": "@jupyter-widgets/controls", 2079 | "_view_module_version": "1.5.0", 2080 | "_view_name": "HBoxView", 2081 | "box_style": "", 2082 | "children": [ 2083 | "IPY_MODEL_b755ac8bc04349c39e0dc5fc6cb4ac88", 2084 | "IPY_MODEL_ad27a257e4f647b89d809e47d470f79f", 2085 | "IPY_MODEL_4999a10141364b8ab18839320bffd6ed" 2086 | ], 2087 | "layout": "IPY_MODEL_862ae1541542421bbd5f359f9d0e717d" 2088 | } 2089 | }, 2090 | "327273aa89264bf3a081ce85bd9ab0f4": { 2091 | "model_module": "@jupyter-widgets/controls", 2092 | "model_module_version": "1.5.0", 2093 | "model_name": "HBoxModel", 2094 | "state": { 2095 | "_dom_classes": [], 2096 | "_model_module": "@jupyter-widgets/controls", 2097 | "_model_module_version": "1.5.0", 2098 | "_model_name": "HBoxModel", 2099 | "_view_count": null, 2100 | "_view_module": "@jupyter-widgets/controls", 2101 | "_view_module_version": "1.5.0", 2102 | "_view_name": "HBoxView", 2103 | "box_style": "", 2104 | "children": [ 2105 | "IPY_MODEL_3272e3e26dc14331b7cac6f0fea0bc36", 2106 | "IPY_MODEL_8ab1eb870c8a4942bc99db73d723f659", 2107 | "IPY_MODEL_050b5ef402924081847a46c0c6dc5985" 2108 | ], 2109 | "layout": "IPY_MODEL_001e2bb0a89d41e9b6aa9280ad55f063" 2110 | } 2111 | }, 2112 | "3272e3e26dc14331b7cac6f0fea0bc36": { 2113 | "model_module": "@jupyter-widgets/controls", 2114 | "model_module_version": "1.5.0", 2115 | "model_name": "HTMLModel", 2116 | "state": { 2117 | "_dom_classes": [], 2118 | "_model_module": "@jupyter-widgets/controls", 2119 | "_model_module_version": "1.5.0", 2120 | "_model_name": "HTMLModel", 2121 | "_view_count": null, 2122 | "_view_module": "@jupyter-widgets/controls", 2123 | "_view_module_version": "1.5.0", 2124 | "_view_name": "HTMLView", 2125 | "description": "", 2126 | "description_tooltip": null, 2127 | "layout": "IPY_MODEL_f6844220a96c46caa1bdb3a673da534f", 2128 | "placeholder": "​", 2129 | "style": "IPY_MODEL_051f32d2c1bf40bd9e269be3f73a74be", 2130 | "value": "added_tokens.json: 100%" 2131 | } 2132 | }, 2133 | "346b3544032a475fadc9c0fdc6049eab": { 2134 | "model_module": "@jupyter-widgets/base", 2135 | "model_module_version": "1.2.0", 2136 | "model_name": "LayoutModel", 2137 | "state": { 2138 | "_model_module": "@jupyter-widgets/base", 2139 | "_model_module_version": "1.2.0", 2140 | "_model_name": "LayoutModel", 2141 | "_view_count": null, 2142 | "_view_module": "@jupyter-widgets/base", 2143 | "_view_module_version": "1.2.0", 2144 | "_view_name": "LayoutView", 2145 | "align_content": null, 2146 | "align_items": null, 2147 | "align_self": null, 2148 | "border": null, 2149 | "bottom": null, 2150 | "display": null, 2151 | "flex": null, 2152 | "flex_flow": null, 2153 | "grid_area": null, 2154 | "grid_auto_columns": null, 2155 | "grid_auto_flow": null, 2156 | "grid_auto_rows": null, 2157 | "grid_column": null, 2158 | "grid_gap": null, 2159 | "grid_row": null, 2160 | "grid_template_areas": null, 2161 | "grid_template_columns": null, 2162 | "grid_template_rows": null, 2163 | "height": null, 2164 | "justify_content": null, 2165 | "justify_items": null, 2166 | "left": null, 2167 | "margin": null, 2168 | "max_height": null, 2169 | "max_width": null, 2170 | "min_height": null, 2171 | "min_width": null, 2172 | "object_fit": null, 2173 | "object_position": null, 2174 | "order": null, 2175 | "overflow": null, 2176 | "overflow_x": null, 2177 | "overflow_y": null, 2178 | "padding": null, 2179 | "right": null, 2180 | "top": null, 2181 | "visibility": null, 2182 | "width": null 2183 | } 2184 | }, 2185 | "3b36a53161634d1ebc86bbb59b8e2a4f": { 2186 | "model_module": "@jupyter-widgets/base", 2187 | "model_module_version": "1.2.0", 2188 | "model_name": "LayoutModel", 2189 | "state": { 2190 | "_model_module": "@jupyter-widgets/base", 2191 | "_model_module_version": "1.2.0", 2192 | "_model_name": "LayoutModel", 2193 | "_view_count": null, 2194 | "_view_module": "@jupyter-widgets/base", 2195 | "_view_module_version": "1.2.0", 2196 | "_view_name": "LayoutView", 2197 | "align_content": null, 2198 | "align_items": null, 2199 | "align_self": null, 2200 | "border": null, 2201 | "bottom": null, 2202 | "display": null, 2203 | "flex": null, 2204 | "flex_flow": null, 2205 | "grid_area": null, 2206 | "grid_auto_columns": null, 2207 | "grid_auto_flow": null, 2208 | "grid_auto_rows": null, 2209 | "grid_column": null, 2210 | "grid_gap": null, 2211 | "grid_row": null, 2212 | "grid_template_areas": null, 2213 | "grid_template_columns": null, 2214 | "grid_template_rows": null, 2215 | "height": null, 2216 | "justify_content": null, 2217 | "justify_items": null, 2218 | "left": null, 2219 | "margin": null, 2220 | "max_height": null, 2221 | "max_width": null, 2222 | "min_height": null, 2223 | "min_width": null, 2224 | "object_fit": null, 2225 | "object_position": null, 2226 | "order": null, 2227 | "overflow": null, 2228 | "overflow_x": null, 2229 | "overflow_y": null, 2230 | "padding": null, 2231 | "right": null, 2232 | "top": null, 2233 | "visibility": null, 2234 | "width": null 2235 | } 2236 | }, 2237 | "3d447500bd6e49b892b7a2215360b5a5": { 2238 | "model_module": "@jupyter-widgets/base", 2239 | "model_module_version": "1.2.0", 2240 | "model_name": "LayoutModel", 2241 | "state": { 2242 | "_model_module": "@jupyter-widgets/base", 2243 | "_model_module_version": "1.2.0", 2244 | "_model_name": "LayoutModel", 2245 | "_view_count": null, 2246 | "_view_module": "@jupyter-widgets/base", 2247 | "_view_module_version": "1.2.0", 2248 | "_view_name": "LayoutView", 2249 | "align_content": null, 2250 | "align_items": null, 2251 | "align_self": null, 2252 | "border": null, 2253 | "bottom": null, 2254 | "display": null, 2255 | "flex": null, 2256 | "flex_flow": null, 2257 | "grid_area": null, 2258 | "grid_auto_columns": null, 2259 | "grid_auto_flow": null, 2260 | "grid_auto_rows": null, 2261 | "grid_column": null, 2262 | "grid_gap": null, 2263 | "grid_row": null, 2264 | "grid_template_areas": null, 2265 | "grid_template_columns": null, 2266 | "grid_template_rows": null, 2267 | "height": null, 2268 | "justify_content": null, 2269 | "justify_items": null, 2270 | "left": null, 2271 | "margin": null, 2272 | "max_height": null, 2273 | "max_width": null, 2274 | "min_height": null, 2275 | "min_width": null, 2276 | "object_fit": null, 2277 | "object_position": null, 2278 | "order": null, 2279 | "overflow": null, 2280 | "overflow_x": null, 2281 | "overflow_y": null, 2282 | "padding": null, 2283 | "right": null, 2284 | "top": null, 2285 | "visibility": null, 2286 | "width": null 2287 | } 2288 | }, 2289 | "3ff0e775916e4c8da887a117728c604c": { 2290 | "model_module": "@jupyter-widgets/base", 2291 | "model_module_version": "1.2.0", 2292 | "model_name": "LayoutModel", 2293 | "state": { 2294 | "_model_module": "@jupyter-widgets/base", 2295 | "_model_module_version": "1.2.0", 2296 | "_model_name": "LayoutModel", 2297 | "_view_count": null, 2298 | "_view_module": "@jupyter-widgets/base", 2299 | "_view_module_version": "1.2.0", 2300 | "_view_name": "LayoutView", 2301 | "align_content": null, 2302 | "align_items": null, 2303 | "align_self": null, 2304 | "border": null, 2305 | "bottom": null, 2306 | "display": null, 2307 | "flex": null, 2308 | "flex_flow": null, 2309 | "grid_area": null, 2310 | "grid_auto_columns": null, 2311 | "grid_auto_flow": null, 2312 | "grid_auto_rows": null, 2313 | "grid_column": null, 2314 | "grid_gap": null, 2315 | "grid_row": null, 2316 | "grid_template_areas": null, 2317 | "grid_template_columns": null, 2318 | "grid_template_rows": null, 2319 | "height": null, 2320 | "justify_content": null, 2321 | "justify_items": null, 2322 | "left": null, 2323 | "margin": null, 2324 | "max_height": null, 2325 | "max_width": null, 2326 | "min_height": null, 2327 | "min_width": null, 2328 | "object_fit": null, 2329 | "object_position": null, 2330 | "order": null, 2331 | "overflow": null, 2332 | "overflow_x": null, 2333 | "overflow_y": null, 2334 | "padding": null, 2335 | "right": null, 2336 | "top": null, 2337 | "visibility": null, 2338 | "width": null 2339 | } 2340 | }, 2341 | "45a689a7024243d3a7c3d2265c272e8d": { 2342 | "model_module": "@jupyter-widgets/controls", 2343 | "model_module_version": "1.5.0", 2344 | "model_name": "HTMLModel", 2345 | "state": { 2346 | "_dom_classes": [], 2347 | "_model_module": "@jupyter-widgets/controls", 2348 | "_model_module_version": "1.5.0", 2349 | "_model_name": "HTMLModel", 2350 | "_view_count": null, 2351 | "_view_module": "@jupyter-widgets/controls", 2352 | "_view_module_version": "1.5.0", 2353 | "_view_name": "HTMLView", 2354 | "description": "", 2355 | "description_tooltip": null, 2356 | "layout": "IPY_MODEL_805204df415c4e2aada1366b7fec94ea", 2357 | "placeholder": "​", 2358 | "style": "IPY_MODEL_05a4343887d947d7ba85c7e4966113ae", 2359 | "value": " 500k/500k [00:00<00:00, 574kB/s]" 2360 | } 2361 | }, 2362 | "4646ae17db21490dae6d024d1fa1d1f9": { 2363 | "model_module": "@jupyter-widgets/controls", 2364 | "model_module_version": "1.5.0", 2365 | "model_name": "DescriptionStyleModel", 2366 | "state": { 2367 | "_model_module": "@jupyter-widgets/controls", 2368 | "_model_module_version": "1.5.0", 2369 | "_model_name": "DescriptionStyleModel", 2370 | "_view_count": null, 2371 | "_view_module": "@jupyter-widgets/base", 2372 | "_view_module_version": "1.2.0", 2373 | "_view_name": "StyleView", 2374 | "description_width": "" 2375 | } 2376 | }, 2377 | "490d23257adb40cab16051b2982b7990": { 2378 | "model_module": "@jupyter-widgets/base", 2379 | "model_module_version": "1.2.0", 2380 | "model_name": "LayoutModel", 2381 | "state": { 2382 | "_model_module": "@jupyter-widgets/base", 2383 | "_model_module_version": "1.2.0", 2384 | "_model_name": "LayoutModel", 2385 | "_view_count": null, 2386 | "_view_module": "@jupyter-widgets/base", 2387 | "_view_module_version": "1.2.0", 2388 | "_view_name": "LayoutView", 2389 | "align_content": null, 2390 | "align_items": null, 2391 | "align_self": null, 2392 | "border": null, 2393 | "bottom": null, 2394 | "display": null, 2395 | "flex": null, 2396 | "flex_flow": null, 2397 | "grid_area": null, 2398 | "grid_auto_columns": null, 2399 | "grid_auto_flow": null, 2400 | "grid_auto_rows": null, 2401 | "grid_column": null, 2402 | "grid_gap": null, 2403 | "grid_row": null, 2404 | "grid_template_areas": null, 2405 | "grid_template_columns": null, 2406 | "grid_template_rows": null, 2407 | "height": null, 2408 | "justify_content": null, 2409 | "justify_items": null, 2410 | "left": null, 2411 | "margin": null, 2412 | "max_height": null, 2413 | "max_width": null, 2414 | "min_height": null, 2415 | "min_width": null, 2416 | "object_fit": null, 2417 | "object_position": null, 2418 | "order": null, 2419 | "overflow": null, 2420 | "overflow_x": null, 2421 | "overflow_y": null, 2422 | "padding": null, 2423 | "right": null, 2424 | "top": null, 2425 | "visibility": null, 2426 | "width": null 2427 | } 2428 | }, 2429 | "49584303ce4a40b0ad10e75703adcc6c": { 2430 | "model_module": "@jupyter-widgets/controls", 2431 | "model_module_version": "1.5.0", 2432 | "model_name": "HTMLModel", 2433 | "state": { 2434 | "_dom_classes": [], 2435 | "_model_module": "@jupyter-widgets/controls", 2436 | "_model_module_version": "1.5.0", 2437 | "_model_name": "HTMLModel", 2438 | "_view_count": null, 2439 | "_view_module": "@jupyter-widgets/controls", 2440 | "_view_module_version": "1.5.0", 2441 | "_view_name": "HTMLView", 2442 | "description": "", 2443 | "description_tooltip": null, 2444 | "layout": "IPY_MODEL_e02c90ef979b40e289d0e0d5a373df49", 2445 | "placeholder": "​", 2446 | "style": "IPY_MODEL_6a85a0cf4384496d8519ad68acef5816", 2447 | "value": " 435/435 [00:00<00:00, 14.5kB/s]" 2448 | } 2449 | }, 2450 | "4999a10141364b8ab18839320bffd6ed": { 2451 | "model_module": "@jupyter-widgets/controls", 2452 | "model_module_version": "1.5.0", 2453 | "model_name": "HTMLModel", 2454 | "state": { 2455 | "_dom_classes": [], 2456 | "_model_module": "@jupyter-widgets/controls", 2457 | "_model_module_version": "1.5.0", 2458 | "_model_name": "HTMLModel", 2459 | "_view_count": null, 2460 | "_view_module": "@jupyter-widgets/controls", 2461 | "_view_module_version": "1.5.0", 2462 | "_view_name": "HTMLView", 2463 | "description": "", 2464 | "description_tooltip": null, 2465 | "layout": "IPY_MODEL_a1f4feb0f44b47e1b3c6f72b8b8f8bcb", 2466 | "placeholder": "​", 2467 | "style": "IPY_MODEL_6e8d9fbb287241ce9415bf4b88989226", 2468 | "value": " 746/746 [00:00<00:00, 16.9kB/s]" 2469 | } 2470 | }, 2471 | "4b4cc248d49648539b78d6c82b389353": { 2472 | "model_module": "@jupyter-widgets/controls", 2473 | "model_module_version": "1.5.0", 2474 | "model_name": "ProgressStyleModel", 2475 | "state": { 2476 | "_model_module": "@jupyter-widgets/controls", 2477 | "_model_module_version": "1.5.0", 2478 | "_model_name": "ProgressStyleModel", 2479 | "_view_count": null, 2480 | "_view_module": "@jupyter-widgets/base", 2481 | "_view_module_version": "1.2.0", 2482 | "_view_name": "StyleView", 2483 | "bar_color": null, 2484 | "description_width": "" 2485 | } 2486 | }, 2487 | "54a02369c6b8491cb9d148401985ef96": { 2488 | "model_module": "@jupyter-widgets/base", 2489 | "model_module_version": "1.2.0", 2490 | "model_name": "LayoutModel", 2491 | "state": { 2492 | "_model_module": "@jupyter-widgets/base", 2493 | "_model_module_version": "1.2.0", 2494 | "_model_name": "LayoutModel", 2495 | "_view_count": null, 2496 | "_view_module": "@jupyter-widgets/base", 2497 | "_view_module_version": "1.2.0", 2498 | "_view_name": "LayoutView", 2499 | "align_content": null, 2500 | "align_items": null, 2501 | "align_self": null, 2502 | "border": null, 2503 | "bottom": null, 2504 | "display": null, 2505 | "flex": null, 2506 | "flex_flow": null, 2507 | "grid_area": null, 2508 | "grid_auto_columns": null, 2509 | "grid_auto_flow": null, 2510 | "grid_auto_rows": null, 2511 | "grid_column": null, 2512 | "grid_gap": null, 2513 | "grid_row": null, 2514 | "grid_template_areas": null, 2515 | "grid_template_columns": null, 2516 | "grid_template_rows": null, 2517 | "height": null, 2518 | "justify_content": null, 2519 | "justify_items": null, 2520 | "left": null, 2521 | "margin": null, 2522 | "max_height": null, 2523 | "max_width": null, 2524 | "min_height": null, 2525 | "min_width": null, 2526 | "object_fit": null, 2527 | "object_position": null, 2528 | "order": null, 2529 | "overflow": null, 2530 | "overflow_x": null, 2531 | "overflow_y": null, 2532 | "padding": null, 2533 | "right": null, 2534 | "top": null, 2535 | "visibility": null, 2536 | "width": null 2537 | } 2538 | }, 2539 | "5c2e730f8e694d81bd78b61364d19d28": { 2540 | "model_module": "@jupyter-widgets/base", 2541 | "model_module_version": "1.2.0", 2542 | "model_name": "LayoutModel", 2543 | "state": { 2544 | "_model_module": "@jupyter-widgets/base", 2545 | "_model_module_version": "1.2.0", 2546 | "_model_name": "LayoutModel", 2547 | "_view_count": null, 2548 | "_view_module": "@jupyter-widgets/base", 2549 | "_view_module_version": "1.2.0", 2550 | "_view_name": "LayoutView", 2551 | "align_content": null, 2552 | "align_items": null, 2553 | "align_self": null, 2554 | "border": null, 2555 | "bottom": null, 2556 | "display": null, 2557 | "flex": null, 2558 | "flex_flow": null, 2559 | "grid_area": null, 2560 | "grid_auto_columns": null, 2561 | "grid_auto_flow": null, 2562 | "grid_auto_rows": null, 2563 | "grid_column": null, 2564 | "grid_gap": null, 2565 | "grid_row": null, 2566 | "grid_template_areas": null, 2567 | "grid_template_columns": null, 2568 | "grid_template_rows": null, 2569 | "height": null, 2570 | "justify_content": null, 2571 | "justify_items": null, 2572 | "left": null, 2573 | "margin": null, 2574 | "max_height": null, 2575 | "max_width": null, 2576 | "min_height": null, 2577 | "min_width": null, 2578 | "object_fit": null, 2579 | "object_position": null, 2580 | "order": null, 2581 | "overflow": null, 2582 | "overflow_x": null, 2583 | "overflow_y": null, 2584 | "padding": null, 2585 | "right": null, 2586 | "top": null, 2587 | "visibility": null, 2588 | "width": null 2589 | } 2590 | }, 2591 | "5e9f19d874ba42d3b63b7f8b17ea73cb": { 2592 | "model_module": "@jupyter-widgets/base", 2593 | "model_module_version": "1.2.0", 2594 | "model_name": "LayoutModel", 2595 | "state": { 2596 | "_model_module": "@jupyter-widgets/base", 2597 | "_model_module_version": "1.2.0", 2598 | "_model_name": "LayoutModel", 2599 | "_view_count": null, 2600 | "_view_module": "@jupyter-widgets/base", 2601 | "_view_module_version": "1.2.0", 2602 | "_view_name": "LayoutView", 2603 | "align_content": null, 2604 | "align_items": null, 2605 | "align_self": null, 2606 | "border": null, 2607 | "bottom": null, 2608 | "display": null, 2609 | "flex": null, 2610 | "flex_flow": null, 2611 | "grid_area": null, 2612 | "grid_auto_columns": null, 2613 | "grid_auto_flow": null, 2614 | "grid_auto_rows": null, 2615 | "grid_column": null, 2616 | "grid_gap": null, 2617 | "grid_row": null, 2618 | "grid_template_areas": null, 2619 | "grid_template_columns": null, 2620 | "grid_template_rows": null, 2621 | "height": null, 2622 | "justify_content": null, 2623 | "justify_items": null, 2624 | "left": null, 2625 | "margin": null, 2626 | "max_height": null, 2627 | "max_width": null, 2628 | "min_height": null, 2629 | "min_width": null, 2630 | "object_fit": null, 2631 | "object_position": null, 2632 | "order": null, 2633 | "overflow": null, 2634 | "overflow_x": null, 2635 | "overflow_y": null, 2636 | "padding": null, 2637 | "right": null, 2638 | "top": null, 2639 | "visibility": null, 2640 | "width": null 2641 | } 2642 | }, 2643 | "61b6e1279a1745fd94ba8c68c513c2c4": { 2644 | "model_module": "@jupyter-widgets/controls", 2645 | "model_module_version": "1.5.0", 2646 | "model_name": "DescriptionStyleModel", 2647 | "state": { 2648 | "_model_module": "@jupyter-widgets/controls", 2649 | "_model_module_version": "1.5.0", 2650 | "_model_name": "DescriptionStyleModel", 2651 | "_view_count": null, 2652 | "_view_module": "@jupyter-widgets/base", 2653 | "_view_module_version": "1.2.0", 2654 | "_view_name": "StyleView", 2655 | "description_width": "" 2656 | } 2657 | }, 2658 | "6a85a0cf4384496d8519ad68acef5816": { 2659 | "model_module": "@jupyter-widgets/controls", 2660 | "model_module_version": "1.5.0", 2661 | "model_name": "DescriptionStyleModel", 2662 | "state": { 2663 | "_model_module": "@jupyter-widgets/controls", 2664 | "_model_module_version": "1.5.0", 2665 | "_model_name": "DescriptionStyleModel", 2666 | "_view_count": null, 2667 | "_view_module": "@jupyter-widgets/base", 2668 | "_view_module_version": "1.2.0", 2669 | "_view_name": "StyleView", 2670 | "description_width": "" 2671 | } 2672 | }, 2673 | "6e8d9fbb287241ce9415bf4b88989226": { 2674 | "model_module": "@jupyter-widgets/controls", 2675 | "model_module_version": "1.5.0", 2676 | "model_name": "DescriptionStyleModel", 2677 | "state": { 2678 | "_model_module": "@jupyter-widgets/controls", 2679 | "_model_module_version": "1.5.0", 2680 | "_model_name": "DescriptionStyleModel", 2681 | "_view_count": null, 2682 | "_view_module": "@jupyter-widgets/base", 2683 | "_view_module_version": "1.2.0", 2684 | "_view_name": "StyleView", 2685 | "description_width": "" 2686 | } 2687 | }, 2688 | "70766048465b44c0b3a343d3dda36482": { 2689 | "model_module": "@jupyter-widgets/controls", 2690 | "model_module_version": "1.5.0", 2691 | "model_name": "DescriptionStyleModel", 2692 | "state": { 2693 | "_model_module": "@jupyter-widgets/controls", 2694 | "_model_module_version": "1.5.0", 2695 | "_model_name": "DescriptionStyleModel", 2696 | "_view_count": null, 2697 | "_view_module": "@jupyter-widgets/base", 2698 | "_view_module_version": "1.2.0", 2699 | "_view_name": "StyleView", 2700 | "description_width": "" 2701 | } 2702 | }, 2703 | "755217e6353f4902bf23debc8625a734": { 2704 | "model_module": "@jupyter-widgets/controls", 2705 | "model_module_version": "1.5.0", 2706 | "model_name": "FloatProgressModel", 2707 | "state": { 2708 | "_dom_classes": [], 2709 | "_model_module": "@jupyter-widgets/controls", 2710 | "_model_module_version": "1.5.0", 2711 | "_model_name": "FloatProgressModel", 2712 | "_view_count": null, 2713 | "_view_module": "@jupyter-widgets/controls", 2714 | "_view_module_version": "1.5.0", 2715 | "_view_name": "ProgressView", 2716 | "bar_style": "success", 2717 | "description": "", 2718 | "description_tooltip": null, 2719 | "layout": "IPY_MODEL_5c2e730f8e694d81bd78b61364d19d28", 2720 | "max": 435, 2721 | "min": 0, 2722 | "orientation": "horizontal", 2723 | "style": "IPY_MODEL_8dda5ca9aa9140adba5205dcb668c8a0", 2724 | "value": 435 2725 | } 2726 | }, 2727 | "805204df415c4e2aada1366b7fec94ea": { 2728 | "model_module": "@jupyter-widgets/base", 2729 | "model_module_version": "1.2.0", 2730 | "model_name": "LayoutModel", 2731 | "state": { 2732 | "_model_module": "@jupyter-widgets/base", 2733 | "_model_module_version": "1.2.0", 2734 | "_model_name": "LayoutModel", 2735 | "_view_count": null, 2736 | "_view_module": "@jupyter-widgets/base", 2737 | "_view_module_version": "1.2.0", 2738 | "_view_name": "LayoutView", 2739 | "align_content": null, 2740 | "align_items": null, 2741 | "align_self": null, 2742 | "border": null, 2743 | "bottom": null, 2744 | "display": null, 2745 | "flex": null, 2746 | "flex_flow": null, 2747 | "grid_area": null, 2748 | "grid_auto_columns": null, 2749 | "grid_auto_flow": null, 2750 | "grid_auto_rows": null, 2751 | "grid_column": null, 2752 | "grid_gap": null, 2753 | "grid_row": null, 2754 | "grid_template_areas": null, 2755 | "grid_template_columns": null, 2756 | "grid_template_rows": null, 2757 | "height": null, 2758 | "justify_content": null, 2759 | "justify_items": null, 2760 | "left": null, 2761 | "margin": null, 2762 | "max_height": null, 2763 | "max_width": null, 2764 | "min_height": null, 2765 | "min_width": null, 2766 | "object_fit": null, 2767 | "object_position": null, 2768 | "order": null, 2769 | "overflow": null, 2770 | "overflow_x": null, 2771 | "overflow_y": null, 2772 | "padding": null, 2773 | "right": null, 2774 | "top": null, 2775 | "visibility": null, 2776 | "width": null 2777 | } 2778 | }, 2779 | "829757d3e5174cefbfae580ca6036e37": { 2780 | "model_module": "@jupyter-widgets/controls", 2781 | "model_module_version": "1.5.0", 2782 | "model_name": "FloatProgressModel", 2783 | "state": { 2784 | "_dom_classes": [], 2785 | "_model_module": "@jupyter-widgets/controls", 2786 | "_model_module_version": "1.5.0", 2787 | "_model_name": "FloatProgressModel", 2788 | "_view_count": null, 2789 | "_view_module": "@jupyter-widgets/controls", 2790 | "_view_module_version": "1.5.0", 2791 | "_view_name": "ProgressView", 2792 | "bar_style": "success", 2793 | "description": "", 2794 | "description_tooltip": null, 2795 | "layout": "IPY_MODEL_5e9f19d874ba42d3b63b7f8b17ea73cb", 2796 | "max": 5, 2797 | "min": 0, 2798 | "orientation": "horizontal", 2799 | "style": "IPY_MODEL_08d61ff53e88466da8c2112a0539eddf", 2800 | "value": 5 2801 | } 2802 | }, 2803 | "862ae1541542421bbd5f359f9d0e717d": { 2804 | "model_module": "@jupyter-widgets/base", 2805 | "model_module_version": "1.2.0", 2806 | "model_name": "LayoutModel", 2807 | "state": { 2808 | "_model_module": "@jupyter-widgets/base", 2809 | "_model_module_version": "1.2.0", 2810 | "_model_name": "LayoutModel", 2811 | "_view_count": null, 2812 | "_view_module": "@jupyter-widgets/base", 2813 | "_view_module_version": "1.2.0", 2814 | "_view_name": "LayoutView", 2815 | "align_content": null, 2816 | "align_items": null, 2817 | "align_self": null, 2818 | "border": null, 2819 | "bottom": null, 2820 | "display": null, 2821 | "flex": null, 2822 | "flex_flow": null, 2823 | "grid_area": null, 2824 | "grid_auto_columns": null, 2825 | "grid_auto_flow": null, 2826 | "grid_auto_rows": null, 2827 | "grid_column": null, 2828 | "grid_gap": null, 2829 | "grid_row": null, 2830 | "grid_template_areas": null, 2831 | "grid_template_columns": null, 2832 | "grid_template_rows": null, 2833 | "height": null, 2834 | "justify_content": null, 2835 | "justify_items": null, 2836 | "left": null, 2837 | "margin": null, 2838 | "max_height": null, 2839 | "max_width": null, 2840 | "min_height": null, 2841 | "min_width": null, 2842 | "object_fit": null, 2843 | "object_position": null, 2844 | "order": null, 2845 | "overflow": null, 2846 | "overflow_x": null, 2847 | "overflow_y": null, 2848 | "padding": null, 2849 | "right": null, 2850 | "top": null, 2851 | "visibility": null, 2852 | "width": null 2853 | } 2854 | }, 2855 | "8a468e6ee5e5483e9dfe6c5c8ff845b0": { 2856 | "model_module": "@jupyter-widgets/base", 2857 | "model_module_version": "1.2.0", 2858 | "model_name": "LayoutModel", 2859 | "state": { 2860 | "_model_module": "@jupyter-widgets/base", 2861 | "_model_module_version": "1.2.0", 2862 | "_model_name": "LayoutModel", 2863 | "_view_count": null, 2864 | "_view_module": "@jupyter-widgets/base", 2865 | "_view_module_version": "1.2.0", 2866 | "_view_name": "LayoutView", 2867 | "align_content": null, 2868 | "align_items": null, 2869 | "align_self": null, 2870 | "border": null, 2871 | "bottom": null, 2872 | "display": null, 2873 | "flex": null, 2874 | "flex_flow": null, 2875 | "grid_area": null, 2876 | "grid_auto_columns": null, 2877 | "grid_auto_flow": null, 2878 | "grid_auto_rows": null, 2879 | "grid_column": null, 2880 | "grid_gap": null, 2881 | "grid_row": null, 2882 | "grid_template_areas": null, 2883 | "grid_template_columns": null, 2884 | "grid_template_rows": null, 2885 | "height": null, 2886 | "justify_content": null, 2887 | "justify_items": null, 2888 | "left": null, 2889 | "margin": null, 2890 | "max_height": null, 2891 | "max_width": null, 2892 | "min_height": null, 2893 | "min_width": null, 2894 | "object_fit": null, 2895 | "object_position": null, 2896 | "order": null, 2897 | "overflow": null, 2898 | "overflow_x": null, 2899 | "overflow_y": null, 2900 | "padding": null, 2901 | "right": null, 2902 | "top": null, 2903 | "visibility": null, 2904 | "width": null 2905 | } 2906 | }, 2907 | "8ab1eb870c8a4942bc99db73d723f659": { 2908 | "model_module": "@jupyter-widgets/controls", 2909 | "model_module_version": "1.5.0", 2910 | "model_name": "FloatProgressModel", 2911 | "state": { 2912 | "_dom_classes": [], 2913 | "_model_module": "@jupyter-widgets/controls", 2914 | "_model_module_version": "1.5.0", 2915 | "_model_name": "FloatProgressModel", 2916 | "_view_count": null, 2917 | "_view_module": "@jupyter-widgets/controls", 2918 | "_view_module_version": "1.5.0", 2919 | "_view_name": "ProgressView", 2920 | "bar_style": "success", 2921 | "description": "", 2922 | "description_tooltip": null, 2923 | "layout": "IPY_MODEL_9f960cba4b92495cb01a2586a622deaa", 2924 | "max": 21, 2925 | "min": 0, 2926 | "orientation": "horizontal", 2927 | "style": "IPY_MODEL_dc84beff92194850b193fb4a5d0df46b", 2928 | "value": 21 2929 | } 2930 | }, 2931 | "8dda5ca9aa9140adba5205dcb668c8a0": { 2932 | "model_module": "@jupyter-widgets/controls", 2933 | "model_module_version": "1.5.0", 2934 | "model_name": "ProgressStyleModel", 2935 | "state": { 2936 | "_model_module": "@jupyter-widgets/controls", 2937 | "_model_module_version": "1.5.0", 2938 | "_model_name": "ProgressStyleModel", 2939 | "_view_count": null, 2940 | "_view_module": "@jupyter-widgets/base", 2941 | "_view_module_version": "1.2.0", 2942 | "_view_name": "StyleView", 2943 | "bar_color": null, 2944 | "description_width": "" 2945 | } 2946 | }, 2947 | "97229a811297410db94d6aca18d1f392": { 2948 | "model_module": "@jupyter-widgets/controls", 2949 | "model_module_version": "1.5.0", 2950 | "model_name": "HTMLModel", 2951 | "state": { 2952 | "_dom_classes": [], 2953 | "_model_module": "@jupyter-widgets/controls", 2954 | "_model_module_version": "1.5.0", 2955 | "_model_name": "HTMLModel", 2956 | "_view_count": null, 2957 | "_view_module": "@jupyter-widgets/controls", 2958 | "_view_module_version": "1.5.0", 2959 | "_view_name": "HTMLView", 2960 | "description": "", 2961 | "description_tooltip": null, 2962 | "layout": "IPY_MODEL_3b36a53161634d1ebc86bbb59b8e2a4f", 2963 | "placeholder": "​", 2964 | "style": "IPY_MODEL_258526fc9c1d44c59100a35defc1b478", 2965 | "value": " 5/5 [00:09<00:00,  1.71s/it]" 2966 | } 2967 | }, 2968 | "9b8aba5af6af42c0a3377c7676fc73cb": { 2969 | "model_module": "@jupyter-widgets/controls", 2970 | "model_module_version": "1.5.0", 2971 | "model_name": "HTMLModel", 2972 | "state": { 2973 | "_dom_classes": [], 2974 | "_model_module": "@jupyter-widgets/controls", 2975 | "_model_module_version": "1.5.0", 2976 | "_model_name": "HTMLModel", 2977 | "_view_count": null, 2978 | "_view_module": "@jupyter-widgets/controls", 2979 | "_view_module_version": "1.5.0", 2980 | "_view_name": "HTMLView", 2981 | "description": "", 2982 | "description_tooltip": null, 2983 | "layout": "IPY_MODEL_f6e1698b96914a92abc612c511bd3ba9", 2984 | "placeholder": "​", 2985 | "style": "IPY_MODEL_ba309009b20343e59c061c5fd7b8a150", 2986 | "value": "tokenizer.model: 100%" 2987 | } 2988 | }, 2989 | "9f960cba4b92495cb01a2586a622deaa": { 2990 | "model_module": "@jupyter-widgets/base", 2991 | "model_module_version": "1.2.0", 2992 | "model_name": "LayoutModel", 2993 | "state": { 2994 | "_model_module": "@jupyter-widgets/base", 2995 | "_model_module_version": "1.2.0", 2996 | "_model_name": "LayoutModel", 2997 | "_view_count": null, 2998 | "_view_module": "@jupyter-widgets/base", 2999 | "_view_module_version": "1.2.0", 3000 | "_view_name": "LayoutView", 3001 | "align_content": null, 3002 | "align_items": null, 3003 | "align_self": null, 3004 | "border": null, 3005 | "bottom": null, 3006 | "display": null, 3007 | "flex": null, 3008 | "flex_flow": null, 3009 | "grid_area": null, 3010 | "grid_auto_columns": null, 3011 | "grid_auto_flow": null, 3012 | "grid_auto_rows": null, 3013 | "grid_column": null, 3014 | "grid_gap": null, 3015 | "grid_row": null, 3016 | "grid_template_areas": null, 3017 | "grid_template_columns": null, 3018 | "grid_template_rows": null, 3019 | "height": null, 3020 | "justify_content": null, 3021 | "justify_items": null, 3022 | "left": null, 3023 | "margin": null, 3024 | "max_height": null, 3025 | "max_width": null, 3026 | "min_height": null, 3027 | "min_width": null, 3028 | "object_fit": null, 3029 | "object_position": null, 3030 | "order": null, 3031 | "overflow": null, 3032 | "overflow_x": null, 3033 | "overflow_y": null, 3034 | "padding": null, 3035 | "right": null, 3036 | "top": null, 3037 | "visibility": null, 3038 | "width": null 3039 | } 3040 | }, 3041 | "a1bb86a49a5845c6a3bb936dfa257b38": { 3042 | "model_module": "@jupyter-widgets/controls", 3043 | "model_module_version": "1.5.0", 3044 | "model_name": "FloatProgressModel", 3045 | "state": { 3046 | "_dom_classes": [], 3047 | "_model_module": "@jupyter-widgets/controls", 3048 | "_model_module_version": "1.5.0", 3049 | "_model_name": "FloatProgressModel", 3050 | "_view_count": null, 3051 | "_view_module": "@jupyter-widgets/controls", 3052 | "_view_module_version": "1.5.0", 3053 | "_view_name": "ProgressView", 3054 | "bar_style": "success", 3055 | "description": "", 3056 | "description_tooltip": null, 3057 | "layout": "IPY_MODEL_3d447500bd6e49b892b7a2215360b5a5", 3058 | "max": 499723, 3059 | "min": 0, 3060 | "orientation": "horizontal", 3061 | "style": "IPY_MODEL_ebfc7b152a224333bfc77090c2ef934c", 3062 | "value": 499723 3063 | } 3064 | }, 3065 | "a1f4feb0f44b47e1b3c6f72b8b8f8bcb": { 3066 | "model_module": "@jupyter-widgets/base", 3067 | "model_module_version": "1.2.0", 3068 | "model_name": "LayoutModel", 3069 | "state": { 3070 | "_model_module": "@jupyter-widgets/base", 3071 | "_model_module_version": "1.2.0", 3072 | "_model_name": "LayoutModel", 3073 | "_view_count": null, 3074 | "_view_module": "@jupyter-widgets/base", 3075 | "_view_module_version": "1.2.0", 3076 | "_view_name": "LayoutView", 3077 | "align_content": null, 3078 | "align_items": null, 3079 | "align_self": null, 3080 | "border": null, 3081 | "bottom": null, 3082 | "display": null, 3083 | "flex": null, 3084 | "flex_flow": null, 3085 | "grid_area": null, 3086 | "grid_auto_columns": null, 3087 | "grid_auto_flow": null, 3088 | "grid_auto_rows": null, 3089 | "grid_column": null, 3090 | "grid_gap": null, 3091 | "grid_row": null, 3092 | "grid_template_areas": null, 3093 | "grid_template_columns": null, 3094 | "grid_template_rows": null, 3095 | "height": null, 3096 | "justify_content": null, 3097 | "justify_items": null, 3098 | "left": null, 3099 | "margin": null, 3100 | "max_height": null, 3101 | "max_width": null, 3102 | "min_height": null, 3103 | "min_width": null, 3104 | "object_fit": null, 3105 | "object_position": null, 3106 | "order": null, 3107 | "overflow": null, 3108 | "overflow_x": null, 3109 | "overflow_y": null, 3110 | "padding": null, 3111 | "right": null, 3112 | "top": null, 3113 | "visibility": null, 3114 | "width": null 3115 | } 3116 | }, 3117 | "ad27a257e4f647b89d809e47d470f79f": { 3118 | "model_module": "@jupyter-widgets/controls", 3119 | "model_module_version": "1.5.0", 3120 | "model_name": "FloatProgressModel", 3121 | "state": { 3122 | "_dom_classes": [], 3123 | "_model_module": "@jupyter-widgets/controls", 3124 | "_model_module_version": "1.5.0", 3125 | "_model_name": "FloatProgressModel", 3126 | "_view_count": null, 3127 | "_view_module": "@jupyter-widgets/controls", 3128 | "_view_module_version": "1.5.0", 3129 | "_view_name": "ProgressView", 3130 | "bar_style": "success", 3131 | "description": "", 3132 | "description_tooltip": null, 3133 | "layout": "IPY_MODEL_2178111bcc3240159c12e8d4b81d35df", 3134 | "max": 746, 3135 | "min": 0, 3136 | "orientation": "horizontal", 3137 | "style": "IPY_MODEL_4b4cc248d49648539b78d6c82b389353", 3138 | "value": 746 3139 | } 3140 | }, 3141 | "af6ad6bf16e646d5a0567ddfa73c6ca5": { 3142 | "model_module": "@jupyter-widgets/controls", 3143 | "model_module_version": "1.5.0", 3144 | "model_name": "HBoxModel", 3145 | "state": { 3146 | "_dom_classes": [], 3147 | "_model_module": "@jupyter-widgets/controls", 3148 | "_model_module_version": "1.5.0", 3149 | "_model_name": "HBoxModel", 3150 | "_view_count": null, 3151 | "_view_module": "@jupyter-widgets/controls", 3152 | "_view_module_version": "1.5.0", 3153 | "_view_name": "HBoxView", 3154 | "box_style": "", 3155 | "children": [ 3156 | "IPY_MODEL_b61c0f80b9f94fd9bf57deabbd5495a9", 3157 | "IPY_MODEL_829757d3e5174cefbfae580ca6036e37", 3158 | "IPY_MODEL_97229a811297410db94d6aca18d1f392" 3159 | ], 3160 | "layout": "IPY_MODEL_346b3544032a475fadc9c0fdc6049eab" 3161 | } 3162 | }, 3163 | "b61c0f80b9f94fd9bf57deabbd5495a9": { 3164 | "model_module": "@jupyter-widgets/controls", 3165 | "model_module_version": "1.5.0", 3166 | "model_name": "HTMLModel", 3167 | "state": { 3168 | "_dom_classes": [], 3169 | "_model_module": "@jupyter-widgets/controls", 3170 | "_model_module_version": "1.5.0", 3171 | "_model_name": "HTMLModel", 3172 | "_view_count": null, 3173 | "_view_module": "@jupyter-widgets/controls", 3174 | "_view_module_version": "1.5.0", 3175 | "_view_name": "HTMLView", 3176 | "description": "", 3177 | "description_tooltip": null, 3178 | "layout": "IPY_MODEL_490d23257adb40cab16051b2982b7990", 3179 | "placeholder": "​", 3180 | "style": "IPY_MODEL_4646ae17db21490dae6d024d1fa1d1f9", 3181 | "value": "Loading checkpoint shards: 100%" 3182 | } 3183 | }, 3184 | "b755ac8bc04349c39e0dc5fc6cb4ac88": { 3185 | "model_module": "@jupyter-widgets/controls", 3186 | "model_module_version": "1.5.0", 3187 | "model_name": "HTMLModel", 3188 | "state": { 3189 | "_dom_classes": [], 3190 | "_model_module": "@jupyter-widgets/controls", 3191 | "_model_module_version": "1.5.0", 3192 | "_model_name": "HTMLModel", 3193 | "_view_count": null, 3194 | "_view_module": "@jupyter-widgets/controls", 3195 | "_view_module_version": "1.5.0", 3196 | "_view_name": "HTMLView", 3197 | "description": "", 3198 | "description_tooltip": null, 3199 | "layout": "IPY_MODEL_3ff0e775916e4c8da887a117728c604c", 3200 | "placeholder": "​", 3201 | "style": "IPY_MODEL_f710b0def2af4728aa9516a0e54eb705", 3202 | "value": "tokenizer_config.json: 100%" 3203 | } 3204 | }, 3205 | "b8a19e7cda0d47ccbe8ce81a08ec6c42": { 3206 | "model_module": "@jupyter-widgets/controls", 3207 | "model_module_version": "1.5.0", 3208 | "model_name": "HBoxModel", 3209 | "state": { 3210 | "_dom_classes": [], 3211 | "_model_module": "@jupyter-widgets/controls", 3212 | "_model_module_version": "1.5.0", 3213 | "_model_name": "HBoxModel", 3214 | "_view_count": null, 3215 | "_view_module": "@jupyter-widgets/controls", 3216 | "_view_module_version": "1.5.0", 3217 | "_view_name": "HBoxView", 3218 | "box_style": "", 3219 | "children": [ 3220 | "IPY_MODEL_9b8aba5af6af42c0a3377c7676fc73cb", 3221 | "IPY_MODEL_a1bb86a49a5845c6a3bb936dfa257b38", 3222 | "IPY_MODEL_45a689a7024243d3a7c3d2265c272e8d" 3223 | ], 3224 | "layout": "IPY_MODEL_8a468e6ee5e5483e9dfe6c5c8ff845b0" 3225 | } 3226 | }, 3227 | "ba309009b20343e59c061c5fd7b8a150": { 3228 | "model_module": "@jupyter-widgets/controls", 3229 | "model_module_version": "1.5.0", 3230 | "model_name": "DescriptionStyleModel", 3231 | "state": { 3232 | "_model_module": "@jupyter-widgets/controls", 3233 | "_model_module_version": "1.5.0", 3234 | "_model_name": "DescriptionStyleModel", 3235 | "_view_count": null, 3236 | "_view_module": "@jupyter-widgets/base", 3237 | "_view_module_version": "1.2.0", 3238 | "_view_name": "StyleView", 3239 | "description_width": "" 3240 | } 3241 | }, 3242 | "dc84beff92194850b193fb4a5d0df46b": { 3243 | "model_module": "@jupyter-widgets/controls", 3244 | "model_module_version": "1.5.0", 3245 | "model_name": "ProgressStyleModel", 3246 | "state": { 3247 | "_model_module": "@jupyter-widgets/controls", 3248 | "_model_module_version": "1.5.0", 3249 | "_model_name": "ProgressStyleModel", 3250 | "_view_count": null, 3251 | "_view_module": "@jupyter-widgets/base", 3252 | "_view_module_version": "1.2.0", 3253 | "_view_name": "StyleView", 3254 | "bar_color": null, 3255 | "description_width": "" 3256 | } 3257 | }, 3258 | "e02c90ef979b40e289d0e0d5a373df49": { 3259 | "model_module": "@jupyter-widgets/base", 3260 | "model_module_version": "1.2.0", 3261 | "model_name": "LayoutModel", 3262 | "state": { 3263 | "_model_module": "@jupyter-widgets/base", 3264 | "_model_module_version": "1.2.0", 3265 | "_model_name": "LayoutModel", 3266 | "_view_count": null, 3267 | "_view_module": "@jupyter-widgets/base", 3268 | "_view_module_version": "1.2.0", 3269 | "_view_name": "LayoutView", 3270 | "align_content": null, 3271 | "align_items": null, 3272 | "align_self": null, 3273 | "border": null, 3274 | "bottom": null, 3275 | "display": null, 3276 | "flex": null, 3277 | "flex_flow": null, 3278 | "grid_area": null, 3279 | "grid_auto_columns": null, 3280 | "grid_auto_flow": null, 3281 | "grid_auto_rows": null, 3282 | "grid_column": null, 3283 | "grid_gap": null, 3284 | "grid_row": null, 3285 | "grid_template_areas": null, 3286 | "grid_template_columns": null, 3287 | "grid_template_rows": null, 3288 | "height": null, 3289 | "justify_content": null, 3290 | "justify_items": null, 3291 | "left": null, 3292 | "margin": null, 3293 | "max_height": null, 3294 | "max_width": null, 3295 | "min_height": null, 3296 | "min_width": null, 3297 | "object_fit": null, 3298 | "object_position": null, 3299 | "order": null, 3300 | "overflow": null, 3301 | "overflow_x": null, 3302 | "overflow_y": null, 3303 | "padding": null, 3304 | "right": null, 3305 | "top": null, 3306 | "visibility": null, 3307 | "width": null 3308 | } 3309 | }, 3310 | "ebfc7b152a224333bfc77090c2ef934c": { 3311 | "model_module": "@jupyter-widgets/controls", 3312 | "model_module_version": "1.5.0", 3313 | "model_name": "ProgressStyleModel", 3314 | "state": { 3315 | "_model_module": "@jupyter-widgets/controls", 3316 | "_model_module_version": "1.5.0", 3317 | "_model_name": "ProgressStyleModel", 3318 | "_view_count": null, 3319 | "_view_module": "@jupyter-widgets/base", 3320 | "_view_module_version": "1.2.0", 3321 | "_view_name": "StyleView", 3322 | "bar_color": null, 3323 | "description_width": "" 3324 | } 3325 | }, 3326 | "f6844220a96c46caa1bdb3a673da534f": { 3327 | "model_module": "@jupyter-widgets/base", 3328 | "model_module_version": "1.2.0", 3329 | "model_name": "LayoutModel", 3330 | "state": { 3331 | "_model_module": "@jupyter-widgets/base", 3332 | "_model_module_version": "1.2.0", 3333 | "_model_name": "LayoutModel", 3334 | "_view_count": null, 3335 | "_view_module": "@jupyter-widgets/base", 3336 | "_view_module_version": "1.2.0", 3337 | "_view_name": "LayoutView", 3338 | "align_content": null, 3339 | "align_items": null, 3340 | "align_self": null, 3341 | "border": null, 3342 | "bottom": null, 3343 | "display": null, 3344 | "flex": null, 3345 | "flex_flow": null, 3346 | "grid_area": null, 3347 | "grid_auto_columns": null, 3348 | "grid_auto_flow": null, 3349 | "grid_auto_rows": null, 3350 | "grid_column": null, 3351 | "grid_gap": null, 3352 | "grid_row": null, 3353 | "grid_template_areas": null, 3354 | "grid_template_columns": null, 3355 | "grid_template_rows": null, 3356 | "height": null, 3357 | "justify_content": null, 3358 | "justify_items": null, 3359 | "left": null, 3360 | "margin": null, 3361 | "max_height": null, 3362 | "max_width": null, 3363 | "min_height": null, 3364 | "min_width": null, 3365 | "object_fit": null, 3366 | "object_position": null, 3367 | "order": null, 3368 | "overflow": null, 3369 | "overflow_x": null, 3370 | "overflow_y": null, 3371 | "padding": null, 3372 | "right": null, 3373 | "top": null, 3374 | "visibility": null, 3375 | "width": null 3376 | } 3377 | }, 3378 | "f6e1698b96914a92abc612c511bd3ba9": { 3379 | "model_module": "@jupyter-widgets/base", 3380 | "model_module_version": "1.2.0", 3381 | "model_name": "LayoutModel", 3382 | "state": { 3383 | "_model_module": "@jupyter-widgets/base", 3384 | "_model_module_version": "1.2.0", 3385 | "_model_name": "LayoutModel", 3386 | "_view_count": null, 3387 | "_view_module": "@jupyter-widgets/base", 3388 | "_view_module_version": "1.2.0", 3389 | "_view_name": "LayoutView", 3390 | "align_content": null, 3391 | "align_items": null, 3392 | "align_self": null, 3393 | "border": null, 3394 | "bottom": null, 3395 | "display": null, 3396 | "flex": null, 3397 | "flex_flow": null, 3398 | "grid_area": null, 3399 | "grid_auto_columns": null, 3400 | "grid_auto_flow": null, 3401 | "grid_auto_rows": null, 3402 | "grid_column": null, 3403 | "grid_gap": null, 3404 | "grid_row": null, 3405 | "grid_template_areas": null, 3406 | "grid_template_columns": null, 3407 | "grid_template_rows": null, 3408 | "height": null, 3409 | "justify_content": null, 3410 | "justify_items": null, 3411 | "left": null, 3412 | "margin": null, 3413 | "max_height": null, 3414 | "max_width": null, 3415 | "min_height": null, 3416 | "min_width": null, 3417 | "object_fit": null, 3418 | "object_position": null, 3419 | "order": null, 3420 | "overflow": null, 3421 | "overflow_x": null, 3422 | "overflow_y": null, 3423 | "padding": null, 3424 | "right": null, 3425 | "top": null, 3426 | "visibility": null, 3427 | "width": null 3428 | } 3429 | }, 3430 | "f710b0def2af4728aa9516a0e54eb705": { 3431 | "model_module": "@jupyter-widgets/controls", 3432 | "model_module_version": "1.5.0", 3433 | "model_name": "DescriptionStyleModel", 3434 | "state": { 3435 | "_model_module": "@jupyter-widgets/controls", 3436 | "_model_module_version": "1.5.0", 3437 | "_model_name": "DescriptionStyleModel", 3438 | "_view_count": null, 3439 | "_view_module": "@jupyter-widgets/base", 3440 | "_view_module_version": "1.2.0", 3441 | "_view_name": "StyleView", 3442 | "description_width": "" 3443 | } 3444 | } 3445 | } 3446 | } 3447 | }, 3448 | "nbformat": 4, 3449 | "nbformat_minor": 4 3450 | } 3451 | -------------------------------------------------------------------------------- /notebooks/simple_finetuning_bert_classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "id": "17c1ceb7-3ed7-49c5-8d86-75f05d5b9e46", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from transformers import AutoTokenizer, AutoModelForSequenceClassification, DataCollatorWithPadding, Trainer, TrainingArguments\n", 11 | "import evaluate\n", 12 | "import numpy as np\n", 13 | "from datasets import load_metric, load_dataset\n", 14 | "import random\n" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 2, 20 | "id": "e589b8a4-c8b5-464a-aad0-8c8e13054e48", 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "# Defining a constant SEED for reproducibility in random operations\n", 25 | "SEED = 42\n", 26 | "\n", 27 | "# Setting the seed for the random library to ensure consistent results\n", 28 | "random.seed(SEED)" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 3, 34 | "id": "7d2dc0c4-8932-4889-975d-05160be113ce", 35 | "metadata": {}, 36 | "outputs": [ 37 | { 38 | "name": "stderr", 39 | "output_type": "stream", 40 | "text": [ 41 | "Found cached dataset app_reviews (/Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de)\n", 42 | "Loading cached processed dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-2ed8b4950d3481dc.arrow\n", 43 | "Loading cached processed dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-8340e0b17b85391f.arrow\n" 44 | ] 45 | }, 46 | { 47 | "data": { 48 | "text/plain": [ 49 | "Dataset({\n", 50 | " features: ['package_name', 'review', 'date', 'star'],\n", 51 | " num_rows: 288065\n", 52 | "})" 53 | ] 54 | }, 55 | "execution_count": 3, 56 | "metadata": {}, 57 | "output_type": "execute_result" 58 | } 59 | ], 60 | "source": [ 61 | "# 'star' is a column in our dataset and we want to convert it to a ClassLabel column\n", 62 | "# so we can stratify our samples.\n", 63 | "\n", 64 | "# Importing the ClassLabel module to represent categorical class labels\n", 65 | "from datasets import ClassLabel\n", 66 | "\n", 67 | "# Loading the 'app_reviews' dataset's training split into the 'dataset' variable\n", 68 | "dataset = load_dataset('app_reviews', split='train')\n", 69 | "\n", 70 | "# Converting the 'star' column in our dataset to a ClassLabel type\n", 71 | "# This allows for categorical representation and easier handling of classes\n", 72 | "dataset = dataset.class_encode_column('star')\n", 73 | "\n", 74 | "# Displaying the dataset to see the changes\n", 75 | "dataset" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 4, 81 | "id": "ffe447c4-fc0e-4727-8538-aee1d4d2fa53", 82 | "metadata": {}, 83 | "outputs": [ 84 | { 85 | "name": "stderr", 86 | "output_type": "stream", 87 | "text": [ 88 | "Loading cached split indices for dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-7b0bb7f9411c7832.arrow and /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-8e6b458ed2fd8d99.arrow\n", 89 | "Loading cached split indices for dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-aad9818ddedac634.arrow and /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-35c54ad93c1d9924.arrow\n" 90 | ] 91 | }, 92 | { 93 | "data": { 94 | "text/plain": [ 95 | "DatasetDict({\n", 96 | " train: Dataset({\n", 97 | " features: ['package_name', 'review', 'date', 'star'],\n", 98 | " num_rows: 172839\n", 99 | " })\n", 100 | " test: Dataset({\n", 101 | " features: ['package_name', 'review', 'date', 'star'],\n", 102 | " num_rows: 57613\n", 103 | " })\n", 104 | " val: Dataset({\n", 105 | " features: ['package_name', 'review', 'date', 'star'],\n", 106 | " num_rows: 57613\n", 107 | " })\n", 108 | "})" 109 | ] 110 | }, 111 | "execution_count": 4, 112 | "metadata": {}, 113 | "output_type": "execute_result" 114 | } 115 | ], 116 | "source": [ 117 | "# Splitting the dataset into a training set and a test set.\n", 118 | "# We reserve 20% of the data for testing and use stratification on the 'star' column\n", 119 | "# to ensure both sets have an equal distribution of each star category.\n", 120 | "dataset = dataset.train_test_split(test_size=0.2, seed=SEED, stratify_by_column='star')\n", 121 | "\n", 122 | "# Now, we further split our training dataset to reserve 25% of it for validation.\n", 123 | "# Again, we stratify by the 'star' column to keep the distribution consistent.\n", 124 | "df = dataset['train'].train_test_split(test_size=.25, seed=SEED, stratify_by_column='star')\n", 125 | "\n", 126 | "# Assigning the split datasets to their respective keys:\n", 127 | "# - The remaining 75% of our initial training data becomes the new training dataset.\n", 128 | "dataset['train'] = df['train']\n", 129 | "\n", 130 | "# - The 25% split from our initial training data becomes the validation dataset.\n", 131 | "dataset['val'] = df['test']\n", 132 | "\n", 133 | "# Displaying the dataset to see the distribution across train, test, and validation sets.\n", 134 | "dataset" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 5, 140 | "id": "f25c4761-bc78-408c-aa4d-21119b9779b1", 141 | "metadata": {}, 142 | "outputs": [], 143 | "source": [ 144 | "MODEL = 'distilbert-base-cased'\n", 145 | "\n", 146 | "tokenizer = AutoTokenizer.from_pretrained(MODEL)" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 6, 152 | "id": "1c17e580-5916-450b-98da-c7c9b7474646", 153 | "metadata": {}, 154 | "outputs": [], 155 | "source": [ 156 | "# simple function to batch tokenize utterances with truncation\n", 157 | "def preprocess_function(examples): # each example is an element from the Dataset\n", 158 | " return tokenizer(examples[\"review\"], truncation=True)" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": 7, 164 | "id": "8f17cad6-e4e9-4b10-ab03-7dd2e5e7d669", 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "# DataCollatorWithPadding creates batch of data. It also dynamically pads text to the \n", 169 | "# length of the longest element in the batch, making them all the same length. \n", 170 | "# It's possible to pad your text in the tokenizer function with padding=True, dynamic padding is more efficient.\n", 171 | "\n", 172 | "data_collator = DataCollatorWithPadding(tokenizer=tokenizer)" 173 | ] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "execution_count": null, 178 | "id": "626208d0-1e66-4559-a80b-f2888cc1a1cf", 179 | "metadata": {}, 180 | "outputs": [], 181 | "source": [] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": 8, 186 | "id": "2728591b-2795-4bb3-b1a6-d83a31a51c47", 187 | "metadata": {}, 188 | "outputs": [ 189 | { 190 | "name": "stderr", 191 | "output_type": "stream", 192 | "text": [ 193 | "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.weight', 'pre_classifier.bias', 'classifier.bias', 'pre_classifier.weight']\n", 194 | "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" 195 | ] 196 | } 197 | ], 198 | "source": [ 199 | "sequence_clf_model = AutoModelForSequenceClassification.from_pretrained(\n", 200 | " MODEL,\n", 201 | " num_labels=5,\n", 202 | ")" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 9, 208 | "id": "a44cd57d-d439-4432-b899-995dbc35d712", 209 | "metadata": {}, 210 | "outputs": [ 211 | { 212 | "data": { 213 | "text/plain": [ 214 | "DistilBertForSequenceClassification(\n", 215 | " (distilbert): DistilBertModel(\n", 216 | " (embeddings): Embeddings(\n", 217 | " (word_embeddings): Embedding(28996, 768, padding_idx=0)\n", 218 | " (position_embeddings): Embedding(512, 768)\n", 219 | " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", 220 | " (dropout): Dropout(p=0.1, inplace=False)\n", 221 | " )\n", 222 | " (transformer): Transformer(\n", 223 | " (layer): ModuleList(\n", 224 | " (0-5): 6 x TransformerBlock(\n", 225 | " (attention): MultiHeadSelfAttention(\n", 226 | " (dropout): Dropout(p=0.1, inplace=False)\n", 227 | " (q_lin): Linear(in_features=768, out_features=768, bias=True)\n", 228 | " (k_lin): Linear(in_features=768, out_features=768, bias=True)\n", 229 | " (v_lin): Linear(in_features=768, out_features=768, bias=True)\n", 230 | " (out_lin): Linear(in_features=768, out_features=768, bias=True)\n", 231 | " )\n", 232 | " (sa_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", 233 | " (ffn): FFN(\n", 234 | " (dropout): Dropout(p=0.1, inplace=False)\n", 235 | " (lin1): Linear(in_features=768, out_features=3072, bias=True)\n", 236 | " (lin2): Linear(in_features=3072, out_features=768, bias=True)\n", 237 | " (activation): GELUActivation()\n", 238 | " )\n", 239 | " (output_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", 240 | " )\n", 241 | " )\n", 242 | " )\n", 243 | " )\n", 244 | " (pre_classifier): Linear(in_features=768, out_features=768, bias=True)\n", 245 | " (classifier): Linear(in_features=768, out_features=5, bias=True)\n", 246 | " (dropout): Dropout(p=0.2, inplace=False)\n", 247 | ")" 248 | ] 249 | }, 250 | "execution_count": 9, 251 | "metadata": {}, 252 | "output_type": "execute_result" 253 | } 254 | ], 255 | "source": [ 256 | "sequence_clf_model" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 10, 262 | "id": "92a44187-d471-40f7-a47c-1df5e3083440", 263 | "metadata": {}, 264 | "outputs": [ 265 | { 266 | "name": "stderr", 267 | "output_type": "stream", 268 | "text": [ 269 | "Loading cached processed dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-1d507ec62ce5ddd6.arrow\n", 270 | "Loading cached processed dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-423ab1e7a2c17013.arrow\n", 271 | "Loading cached processed dataset at /Users/sinanozdemir/.cache/huggingface/datasets/app_reviews/default/0.0.0/20335b51b604b9bc04b7be253cd8445caa9ba93f15f39a4b0492b9e9102853de/cache-899b4eef64bd6fb3.arrow\n" 272 | ] 273 | } 274 | ], 275 | "source": [ 276 | "dataset = dataset.map(preprocess_function, batched=True)" 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 11, 282 | "id": "1b42374d-8e48-4b0e-88a7-87b801067855", 283 | "metadata": {}, 284 | "outputs": [ 285 | { 286 | "data": { 287 | "text/plain": [ 288 | "DatasetDict({\n", 289 | " train: Dataset({\n", 290 | " features: ['label', 'input_ids', 'attention_mask'],\n", 291 | " num_rows: 172839\n", 292 | " })\n", 293 | " test: Dataset({\n", 294 | " features: ['label', 'input_ids', 'attention_mask'],\n", 295 | " num_rows: 57613\n", 296 | " })\n", 297 | " val: Dataset({\n", 298 | " features: ['label', 'input_ids', 'attention_mask'],\n", 299 | " num_rows: 57613\n", 300 | " })\n", 301 | "})" 302 | ] 303 | }, 304 | "execution_count": 11, 305 | "metadata": {}, 306 | "output_type": "execute_result" 307 | } 308 | ], 309 | "source": [ 310 | "dataset = dataset.rename_column(\"star\", \"label\")\n", 311 | "dataset = dataset.remove_columns(['package_name', 'review', 'date'])\n", 312 | "dataset" 313 | ] 314 | }, 315 | { 316 | "cell_type": "code", 317 | "execution_count": null, 318 | "id": "c5824878-f25d-450b-8afd-81a3a0879abc", 319 | "metadata": {}, 320 | "outputs": [], 321 | "source": [ 322 | "import pandas as pd\n", 323 | "\n", 324 | "input_ids = dataset['train']['input_ids']\n", 325 | "pd.Series(input_ids).apply(len).hist()" 326 | ] 327 | }, 328 | { 329 | "cell_type": "code", 330 | "execution_count": 12, 331 | "id": "d278028c-193e-4cd6-b443-46eabfbe11d2", 332 | "metadata": {}, 333 | "outputs": [ 334 | { 335 | "data": { 336 | "text/plain": [ 337 | "DatasetDict({\n", 338 | " train: Dataset({\n", 339 | " features: ['label', 'input_ids', 'attention_mask'],\n", 340 | " num_rows: 172839\n", 341 | " })\n", 342 | " test: Dataset({\n", 343 | " features: ['label', 'input_ids', 'attention_mask'],\n", 344 | " num_rows: 57613\n", 345 | " })\n", 346 | " val: Dataset({\n", 347 | " features: ['label', 'input_ids', 'attention_mask'],\n", 348 | " num_rows: 57613\n", 349 | " })\n", 350 | "})" 351 | ] 352 | }, 353 | "execution_count": 12, 354 | "metadata": {}, 355 | "output_type": "execute_result" 356 | } 357 | ], 358 | "source": [ 359 | "dataset" 360 | ] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": 13, 365 | "id": "f4136b95-d3e1-4b1a-861c-e92b6f44a423", 366 | "metadata": {}, 367 | "outputs": [], 368 | "source": [ 369 | "def compute_metrics(p):\n", 370 | " preds = np.argmax(p.predictions, axis=1)\n", 371 | " return {\"accuracy\": (preds == p.label_ids).mean()}\n" 372 | ] 373 | }, 374 | { 375 | "cell_type": "code", 376 | "execution_count": 14, 377 | "id": "f59d37f9-d732-4543-90c0-315bedf9415b", 378 | "metadata": {}, 379 | "outputs": [], 380 | "source": [ 381 | "epochs = 2\n", 382 | "\n", 383 | "training_args = TrainingArguments(\n", 384 | " output_dir=\"./bert_clf_results\",\n", 385 | " num_train_epochs=epochs,\n", 386 | " per_device_train_batch_size=16,\n", 387 | " gradient_accumulation_steps=2,\n", 388 | " per_device_eval_batch_size=32,\n", 389 | " load_best_model_at_end=True,\n", 390 | " \n", 391 | " # some deep learning parameters that the Trainer is able to take in\n", 392 | " warmup_ratio=0.1,\n", 393 | " weight_decay = 0.05,\n", 394 | " \n", 395 | " logging_steps=1,\n", 396 | " log_level='info',\n", 397 | " evaluation_strategy='epoch',\n", 398 | " eval_steps=50,\n", 399 | " save_strategy='epoch'\n", 400 | ")\n", 401 | "\n", 402 | "# Define the trainer:\n", 403 | "\n", 404 | "trainer = Trainer(\n", 405 | " model=sequence_clf_model,\n", 406 | " args=training_args,\n", 407 | " train_dataset=dataset['train'],\n", 408 | " eval_dataset=dataset['val'],\n", 409 | " compute_metrics=compute_metrics, # optional\n", 410 | " data_collator=data_collator # technically optional\n", 411 | ")" 412 | ] 413 | }, 414 | { 415 | "cell_type": "code", 416 | "execution_count": 15, 417 | "id": "5bfdefac-a0d4-435f-be2f-1287547b4397", 418 | "metadata": {}, 419 | "outputs": [ 420 | { 421 | "name": "stderr", 422 | "output_type": "stream", 423 | "text": [ 424 | "***** Running Evaluation *****\n", 425 | " Num examples = 57613\n", 426 | " Batch size = 32\n", 427 | "You're using a DistilBertTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n" 428 | ] 429 | }, 430 | { 431 | "data": { 432 | "text/html": [ 433 | "\n", 434 | "
\n", 435 | " \n", 436 | " \n", 437 | " [1801/1801 1:10:47]\n", 438 | "
\n", 439 | " " 440 | ], 441 | "text/plain": [ 442 | "" 443 | ] 444 | }, 445 | "metadata": {}, 446 | "output_type": "display_data" 447 | }, 448 | { 449 | "name": "stderr", 450 | "output_type": "stream", 451 | "text": [ 452 | "Automatic Weights & Biases logging enabled, to disable set os.environ[\"WANDB_DISABLED\"] = \"true\"\n", 453 | "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mprofoz\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n" 454 | ] 455 | }, 456 | { 457 | "data": { 458 | "text/html": [ 459 | "wandb version 0.16.0 is available! To upgrade, please run:\n", 460 | " $ pip install wandb --upgrade" 461 | ], 462 | "text/plain": [ 463 | "" 464 | ] 465 | }, 466 | "metadata": {}, 467 | "output_type": "display_data" 468 | }, 469 | { 470 | "data": { 471 | "text/html": [ 472 | "Tracking run with wandb version 0.15.7" 473 | ], 474 | "text/plain": [ 475 | "" 476 | ] 477 | }, 478 | "metadata": {}, 479 | "output_type": "display_data" 480 | }, 481 | { 482 | "data": { 483 | "text/html": [ 484 | "Run data is saved locally in /Users/sinanozdemir/Teaching/Pearson/oreilly-hands-on-transformers/notebooks/wandb/run-20231119_082851-b6ujyjgx" 485 | ], 486 | "text/plain": [ 487 | "" 488 | ] 489 | }, 490 | "metadata": {}, 491 | "output_type": "display_data" 492 | }, 493 | { 494 | "data": { 495 | "text/html": [ 496 | "Syncing run dulcet-energy-198 to Weights & Biases (docs)
" 497 | ], 498 | "text/plain": [ 499 | "" 500 | ] 501 | }, 502 | "metadata": {}, 503 | "output_type": "display_data" 504 | }, 505 | { 506 | "data": { 507 | "text/html": [ 508 | " View project at https://wandb.ai/profoz/huggingface" 509 | ], 510 | "text/plain": [ 511 | "" 512 | ] 513 | }, 514 | "metadata": {}, 515 | "output_type": "display_data" 516 | }, 517 | { 518 | "data": { 519 | "text/html": [ 520 | " View run at https://wandb.ai/profoz/huggingface/runs/b6ujyjgx" 521 | ], 522 | "text/plain": [ 523 | "" 524 | ] 525 | }, 526 | "metadata": {}, 527 | "output_type": "display_data" 528 | }, 529 | { 530 | "data": { 531 | "text/plain": [ 532 | "{'eval_loss': 1.554797887802124,\n", 533 | " 'eval_accuracy': 0.5844340686997727,\n", 534 | " 'eval_runtime': 376.4524,\n", 535 | " 'eval_samples_per_second': 153.042,\n", 536 | " 'eval_steps_per_second': 4.784}" 537 | ] 538 | }, 539 | "execution_count": 15, 540 | "metadata": {}, 541 | "output_type": "execute_result" 542 | } 543 | ], 544 | "source": [ 545 | "trainer.evaluate()" 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": 17, 551 | "id": "603b86ef-3da5-4c58-a593-7c8386229d8a", 552 | "metadata": {}, 553 | "outputs": [ 554 | { 555 | "name": "stderr", 556 | "output_type": "stream", 557 | "text": [ 558 | "***** Running training *****\n", 559 | " Num examples = 172,839\n", 560 | " Num Epochs = 2\n", 561 | " Instantaneous batch size per device = 16\n", 562 | " Total train batch size (w. parallel, distributed & accumulation) = 32\n", 563 | " Gradient Accumulation steps = 2\n", 564 | " Total optimization steps = 10,802\n", 565 | " Number of trainable parameters = 65,785,349\n" 566 | ] 567 | }, 568 | { 569 | "data": { 570 | "text/html": [ 571 | "\n", 572 | "
\n", 573 | " \n", 574 | " \n", 575 | " [10802/10802 2:01:54, Epoch 1/2]\n", 576 | "
\n", 577 | " \n", 578 | " \n", 579 | " \n", 580 | " \n", 581 | " \n", 582 | " \n", 583 | " \n", 584 | " \n", 585 | " \n", 586 | " \n", 587 | " \n", 588 | " \n", 589 | " \n", 590 | " \n", 591 | " \n", 592 | " \n", 593 | " \n", 594 | " \n", 595 | " \n", 596 | " \n", 597 | " \n", 598 | " \n", 599 | " \n", 600 | "
EpochTraining LossValidation LossAccuracy
00.5654000.8276520.712738
10.8564000.8207360.717685

" 601 | ], 602 | "text/plain": [ 603 | "" 604 | ] 605 | }, 606 | "metadata": {}, 607 | "output_type": "display_data" 608 | }, 609 | { 610 | "name": "stderr", 611 | "output_type": "stream", 612 | "text": [ 613 | "***** Running Evaluation *****\n", 614 | " Num examples = 57613\n", 615 | " Batch size = 32\n", 616 | "Saving model checkpoint to ./bert_clf_results/checkpoint-10802\n", 617 | "Configuration saved in ./bert_clf_results/checkpoint-10802/config.json\n", 618 | "Model weights saved in ./bert_clf_results/checkpoint-10802/pytorch_model.bin\n", 619 | "\n", 620 | "\n", 621 | "Training completed. Do not forget to share your model on huggingface.co/models =)\n", 622 | "\n", 623 | "\n", 624 | "Loading best model from ./bert_clf_results/checkpoint-10802 (score: 0.8207359910011292).\n" 625 | ] 626 | }, 627 | { 628 | "data": { 629 | "text/plain": [ 630 | "TrainOutput(global_step=10802, training_loss=0.8283754970834909, metrics={'train_runtime': 7323.4982, 'train_samples_per_second': 47.201, 'train_steps_per_second': 1.475, 'total_flos': 7015951744063140.0, 'train_loss': 0.8283754970834909, 'epoch': 2.0})" 631 | ] 632 | }, 633 | "execution_count": 17, 634 | "metadata": {}, 635 | "output_type": "execute_result" 636 | } 637 | ], 638 | "source": [ 639 | "trainer.train()" 640 | ] 641 | }, 642 | { 643 | "cell_type": "code", 644 | "execution_count": 19, 645 | "id": "d513fba4-4640-430b-8c62-35e92fe39274", 646 | "metadata": {}, 647 | "outputs": [], 648 | "source": [ 649 | "# See colab link for comparison on there! https://colab.research.google.com/drive/1elfu-6gaj0KWtIQMyeHYWqqkNUgA6hFn?usp=sharing\n", 650 | "# compare to fine tuned gpt 3.5\n", 651 | " # accuracy\n", 652 | " # cost to train/host (on HF or make your own API)\n", 653 | " # latency/throughput\n", 654 | " " 655 | ] 656 | }, 657 | { 658 | "cell_type": "code", 659 | "execution_count": null, 660 | "id": "8e117977-822c-460a-8b91-462eb6d4a340", 661 | "metadata": {}, 662 | "outputs": [], 663 | "source": [] 664 | }, 665 | { 666 | "cell_type": "code", 667 | "execution_count": null, 668 | "id": "c55b7b6e-64f9-4da3-802f-f480c9045ac4", 669 | "metadata": {}, 670 | "outputs": [], 671 | "source": [] 672 | } 673 | ], 674 | "metadata": { 675 | "kernelspec": { 676 | "display_name": "Python 3 (ipykernel)", 677 | "language": "python", 678 | "name": "python3" 679 | }, 680 | "language_info": { 681 | "codemirror_mode": { 682 | "name": "ipython", 683 | "version": 3 684 | }, 685 | "file_extension": ".py", 686 | "mimetype": "text/x-python", 687 | "name": "python", 688 | "nbconvert_exporter": "python", 689 | "pygments_lexer": "ipython3", 690 | "version": "3.11.5" 691 | } 692 | }, 693 | "nbformat": 4, 694 | "nbformat_minor": 5 695 | } 696 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pandas==2.0.3 2 | jupyter==1.0.0 3 | openai==1.10.0 4 | streamlit==1.31.0 5 | datasets==2.13.1 6 | torch==2.0.1 7 | transformers==4.36.2 8 | cohere==4.31 9 | -------------------------------------------------------------------------------- /streamlit/.streamlit/secrets.toml: -------------------------------------------------------------------------------- 1 | openai_key = "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" -------------------------------------------------------------------------------- /streamlit/openai_playground/.streamlit/secrets.toml: -------------------------------------------------------------------------------- 1 | openai_key = "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" -------------------------------------------------------------------------------- /streamlit/openai_playground/README.md: -------------------------------------------------------------------------------- 1 | # OpenAI Playground Web App 2 | 3 | ## Overview 4 | 5 | This application is an interactive playground for OpenAI's powerful GPT models. It allows users to experiment with different versions of GPT models in a convenient and user-friendly interface. 6 | 7 | You can provide a system prompt to set a context for the model, a user prompt as the query for the model, and select which GPT model they wish to use for generating the response. This application uses OpenAI's Python client library to interact with the API and Streamlit to create the web interface. 8 | 9 | ## How to Run 10 | 11 | Here are the steps to run this application: 12 | 13 | 1. **Install Dependencies**: 14 | 15 | First, make sure Python is installed on your system. This application requires Python 3.6 or later. 16 | 17 | Next, install the necessary Python packages. You can do this by running the following command: 18 | 19 | ``` 20 | pip install streamlit openai 21 | ``` 22 | 23 | 2. **Get an OpenAI API Key**: 24 | 25 | In order to use OpenAI's models, you need to have an API key from OpenAI. You can obtain this by creating an account on [OpenAI's website](https://www.openai.com/) and subscribing to their API. 26 | 27 | 3. **Set up Streamlit Secrets**: 28 | 29 | You can securely set your OpenAI API key using Streamlit secrets. In your Streamlit application directory, run: 30 | 31 | ``` 32 | streamlit secrets new 33 | ``` 34 | 35 | In the text editor that opens up, add the following line: 36 | 37 | ``` 38 | [openai] 39 | openai_key = "your-openai-api-key" 40 | ``` 41 | 42 | Replace "your-openai-api-key" with your actual OpenAI API key. Save and close the editor. 43 | 44 | 4. **Run the Streamlit App**: 45 | 46 | Finally, in your terminal, navigate to the directory containing the Python script for the Streamlit app and run the following command: 47 | 48 | ``` 49 | streamlit run your_script.py 50 | ``` 51 | 52 | Replace "your_script.py" with the actual filename of your Streamlit script. 53 | -------------------------------------------------------------------------------- /streamlit/openai_playground/app.py: -------------------------------------------------------------------------------- 1 | # Import the necessary modules 2 | import random # Standard Python library for generating random numbers 3 | 4 | import openai # OpenAI's Python client library for calling the OpenAI API 5 | import streamlit as st # Streamlit library for creating web apps 6 | 7 | # Set the OpenAI API key from Streamlit secrets, enabling secure API calls to OpenAI 8 | openai.api_key = st.secrets["openai_key"] 9 | 10 | 11 | # Define function for running a conversation with OpenAI 12 | def run_prompt_through_openai(system_prompt, user_prompt, model): 13 | # Use the OpenAI API to create a ChatCompletion instance, simulating a conversation with the AI 14 | chat_completion = openai.ChatCompletion.create( 15 | model=model, # Specify the model to use for the conversation 16 | messages=[ # Define the messages for the conversation 17 | {'role': 'system', 'content': system_prompt}, # The initial system message 18 | {'role': 'user', 'content': user_prompt} # The user's message 19 | ] 20 | ) 21 | # Extract and return the AI's response from the conversation 22 | return chat_completion.choices[0].message.content 23 | 24 | 25 | # Set a title for the Streamlit app 26 | st.title("OpenAI Playground") 27 | 28 | # Create a text input field for the system prompt, with a default value 29 | system_prompt = st.text_input( 30 | "System Prompt", value="You are a language translator. You translate between English and Turkish." 31 | ) 32 | 33 | # Create a text input field for the user prompt, with a default value 34 | user_prompt = st.text_input("User Prompt", value="Hello, how are you?") 35 | 36 | # Create a select box for choosing the OpenAI model 37 | model = st.selectbox("Model", ['gpt-3.5-turbo', 'gpt-4', 'gpt-3.5-turbo-16k']) 38 | 39 | # Create a button for executing the AI conversation 40 | if st.button("Run"): 41 | # If the button is clicked, run the user and system prompts through the chosen AI model 42 | response = run_prompt_through_openai(system_prompt, user_prompt, model) 43 | 44 | # Write the AI's response in the app 45 | st.write(f"Response:\n---\n{response}") 46 | -------------------------------------------------------------------------------- /streamlit/openai_playground/requirements.txt: -------------------------------------------------------------------------------- 1 | streamlit 2 | openai 3 | altair<5 -------------------------------------------------------------------------------- /streamlit/retrieval_augmented_generation/.streamlit/secrets.toml: -------------------------------------------------------------------------------- 1 | openai_key = "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" -------------------------------------------------------------------------------- /streamlit/retrieval_augmented_generation/README.md: -------------------------------------------------------------------------------- 1 | # Document Question Answering System 2 | 3 | ## Overview 4 | 5 | This web application uses OpenAI's GPT model to answer questions based on a context scraped from a URL provided by the user. The application uses OpenAI's Python client library to interact with the GPT model, the BeautifulSoup library to parse HTML content, and Streamlit for the web interface. 6 | 7 | The user provides a URL, a question, and an optional system prompt. The application fetches and parses the HTML content from the URL, uses it as context to answer the question using the GPT model, and displays the answer on the web interface. 8 | 9 | ## How to Run 10 | 11 | Here are the steps to run this application: 12 | 13 | 1. **Install Dependencies**: 14 | 15 | Make sure Python is installed on your system. Python 3.6 or later is required. 16 | 17 | Install the necessary Python packages with the following command: 18 | 19 | ``` 20 | pip install streamlit openai beautifulsoup4 requests 21 | ``` 22 | 23 | 2. **Get an OpenAI API Key**: 24 | 25 | To use OpenAI's models, you need an API key from OpenAI. Obtain this by creating an account on [OpenAI's website](https://www.openai.com/) and subscribing to their API. 26 | 27 | 3. **Set up Streamlit Secrets**: 28 | 29 | Securely set your OpenAI API key using Streamlit secrets. In your Streamlit application directory, run: 30 | 31 | ``` 32 | streamlit secrets new 33 | ``` 34 | 35 | In the text editor that opens, add the following line: 36 | 37 | ``` 38 | [openai] 39 | openai_key = "your-openai-api-key" 40 | ``` 41 | 42 | Replace "your-openai-api-key" with your actual OpenAI API key. Save and close the editor. 43 | 44 | 4. **Run the Streamlit App**: 45 | 46 | Navigate to the directory containing the Python script for the Streamlit app in your terminal and run: 47 | 48 | ``` 49 | streamlit run app.py 50 | ``` 51 | 52 | Replace "your_script.py" with the actual filename of your Python script. 53 | -------------------------------------------------------------------------------- /streamlit/retrieval_augmented_generation/app.py: -------------------------------------------------------------------------------- 1 | # Import necessary libraries 2 | import openai # Library for OpenAI API 3 | import requests # Library for making HTTP requests 4 | import streamlit as st # Library for creating web apps 5 | from bs4 import BeautifulSoup # Library for parsing HTML and XML documents 6 | 7 | # Set the OpenAI API key from Streamlit secrets 8 | openai.api_key = st.secrets["openai_key"] 9 | 10 | 11 | # Define function to communicate with OpenAI's GPT model 12 | def qa_gpt(question, context, system_prompt=None): 13 | # Initialize conversation prompts with user's question and context 14 | prompts = [{'role': 'user', 'content': f"Answer given the following context: {context}\n\nQuestion: {question}"}] 15 | 16 | # If there's a system prompt, add it to the beginning of conversation 17 | if system_prompt is not None: 18 | prompts = [{'role': 'system', 'content': system_prompt}] + prompts 19 | 20 | # Send conversation prompts to GPT model and get response 21 | chat_completion = openai.ChatCompletion.create( 22 | model="gpt-3.5-turbo-16k", 23 | messages=prompts 24 | ) 25 | 26 | # Extract response content and return as a dictionary 27 | return {'response_text': chat_completion.choices[0].message.content} 28 | 29 | 30 | # Set title for the Streamlit app 31 | st.title('Document Question Answering System') 32 | 33 | # Create a text input field for users to enter a URL 34 | url = st.text_area( 35 | label="URL to scrape for context", 36 | value="https://www.sinanozdemir.ai" 37 | ) 38 | 39 | # Create a text input field for users to enter their question 40 | query = st.text_input("Query") 41 | 42 | # Create a text input field for users to enter system prompt 43 | system_prompt = st.text_input("System Prompt") 44 | 45 | # Process query if there is one 46 | if len(query) > 0: 47 | # Make a HTTP request to the provided URL and parse the HTML content 48 | soup = BeautifulSoup(requests.get(url).text, "html.parser") 49 | 50 | # Extract text from the parsed HTML document 51 | text = soup.get_text() 52 | 53 | # Call the GPT model function with the extracted text and user's question 54 | answer = qa_gpt(query, text, system_prompt=system_prompt) 55 | 56 | # Display the model's response on the web app 57 | st.write(answer["response_text"]) 58 | -------------------------------------------------------------------------------- /streamlit/retrieval_augmented_generation/requirements.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | torch 3 | transformers 4 | sentence-transformers 5 | streamlit 6 | openai 7 | bs4 8 | requests 9 | altair<5 -------------------------------------------------------------------------------- /streamlit/single_prompt_example/.streamlit/secrets.toml: -------------------------------------------------------------------------------- 1 | openai_key = "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" -------------------------------------------------------------------------------- /streamlit/single_prompt_example/README.md: -------------------------------------------------------------------------------- 1 | # OpenAI Playground Web App 2 | 3 | ## Overview 4 | 5 | This is a simple application for a set of prompts with a single input. 6 | 7 | You can provide a system prompt to set a context for the model, a user prompt as the query for the model, and select which GPT model they wish to use for generating the response. This application uses OpenAI's Python client library to interact with the API and Streamlit to create the web interface. 8 | 9 | ## How to Run 10 | 11 | Here are the steps to run this application: 12 | 13 | 1. **Install Dependencies**: 14 | 15 | First, make sure Python is installed on your system. This application requires Python 3.6 or later. 16 | 17 | Next, install the necessary Python packages. You can do this by running the following command: 18 | 19 | ``` 20 | pip install streamlit openai 21 | ``` 22 | 23 | 2. **Get an OpenAI API Key**: 24 | 25 | In order to use OpenAI's models, you need to have an API key from OpenAI. You can obtain this by creating an account on [OpenAI's website](https://www.openai.com/) and subscribing to their API. 26 | 27 | 3. **Set up Streamlit Secrets**: 28 | 29 | You can securely set your OpenAI API key using Streamlit secrets. In your Streamlit application directory, run: 30 | 31 | ``` 32 | streamlit secrets new 33 | ``` 34 | 35 | In the text editor that opens up, add the following line: 36 | 37 | ``` 38 | [openai] 39 | openai_key = "your-openai-api-key" 40 | ``` 41 | 42 | Replace "your-openai-api-key" with your actual OpenAI API key. Save and close the editor. 43 | 44 | 4. **Run the Streamlit App**: 45 | 46 | Finally, in your terminal, navigate to the directory containing the Python script for the Streamlit app and run the following command: 47 | 48 | ``` 49 | streamlit run your_script.py 50 | ``` 51 | 52 | Replace "your_script.py" with the actual filename of your Streamlit script. 53 | -------------------------------------------------------------------------------- /streamlit/single_prompt_example/app.py: -------------------------------------------------------------------------------- 1 | # Import the necessary modules 2 | import random # Standard Python library for generating random numbers 3 | 4 | import openai # OpenAI's Python client library for calling the OpenAI API 5 | import streamlit as st # Streamlit library for creating web apps 6 | 7 | # Set the OpenAI API key from Streamlit secrets, enabling secure API calls to OpenAI 8 | openai.api_key = st.secrets["openai_key"] 9 | 10 | 11 | # Define a function for running a conversation with OpenAI 12 | def run_prompt_through_openai(system_prompt, user_prompt, model='gpt-3.5-turbo'): 13 | # Use the OpenAI API to create a ChatCompletion instance, simulating a conversation with the AI 14 | chat_completion = openai.ChatCompletion.create( 15 | model=model, # Specify the model to use for the conversation 16 | messages=[ # Define the messages for the conversation 17 | {'role': 'system', 'content': system_prompt}, # The initial system message 18 | {'role': 'user', 'content': user_prompt} # The user's message 19 | ], 20 | temperature=0.9, 21 | top_p=1, 22 | ) # TODO add any custom parameters here 23 | # Extract and return the AI's response from the conversation 24 | return chat_completion.choices[0].message.content 25 | 26 | 27 | # Set a title for the Streamlit app 28 | st.title("Single Input Example") 29 | st.write("This is an example of a single input field for a conversation with the AI.") 30 | 31 | # Create a text input field for the system prompt, with a default value 32 | system_prompt = "You are a Twitter bot that helps people with their tweets" 33 | 34 | # Create a text input field for the user prompt, with a default value 35 | user_input = st.text_input("Description of a tweet you want", value="I need a tweet about GPT-4") 36 | 37 | user_prompt = ''' 38 | Input: I need a tweet about GPT-4 39 | Tweet: "Wow! I just read about GPT-4 and it's amazing! I can't wait to see what it can do! #gpt4 #ai #machinelearning" 40 | Input: Dogs in the summer and avoiding fleas and ticks 41 | Tweet: "I love my dog, but I hate the fleas and ticks that come with him. I'm going to try to avoid them this summer." 42 | Input: San Francisco's Golden Gate Bridge 43 | Tweet: "I love the Golden Gate Bridge. It's a beautiful sight to see. I can't wait to go back to San Francisco." 44 | Input: {user_input}''' # This is where the user's input will be added 45 | 46 | # Create a button for executing the AI conversation 47 | if st.button("Run"): 48 | user_prompt = user_prompt.format(user_input=user_input) 49 | # If the button is clicked, run the user and system prompts through the chosen AI model 50 | response = run_prompt_through_openai(system_prompt, user_prompt) 51 | 52 | # Write the AI's response in the app 53 | st.write(f"# System Prompt\n{system_prompt}\n# User Prompt\n{user_prompt}\n# AI Response\n{response}") 54 | -------------------------------------------------------------------------------- /streamlit/single_prompt_example/requirements.txt: -------------------------------------------------------------------------------- 1 | streamlit 2 | openai 3 | altair<5 -------------------------------------------------------------------------------- /streamlit/wine_prototype/.streamlit/secrets.toml: -------------------------------------------------------------------------------- 1 | openai_key = "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" -------------------------------------------------------------------------------- /streamlit/wine_prototype/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Wine Recommendation Bot 3 | emoji: 🍷 4 | colorFrom: "#FF0000" 5 | colorTo: "#8B0000" 6 | sdk: streamlit 7 | sdk_version: "1.0.0" 8 | app_file: app.py 9 | pinned: false 10 | --- 11 | 12 | # Wine Recommendation System 13 | 14 | ## Overview 15 | 16 | This web application leverages OpenAI's GPT model to generate wine recommendations based on a client's description. The application uses the Hugging Face Datasets library to load a dataset of wines, OpenAI's Python client library to interact with the GPT model, and Streamlit for the web interface. 17 | 18 | When a user provides a description of a client and the number of wines they'd like to choose from, the application randomly selects the specified number of wines from the dataset. It then uses the GPT model to generate recommendations based on the selected wines and the client's description, and displays these recommendations on the web interface. 19 | 20 | ## How to Run 21 | 22 | ### 1. Install Dependencies 23 | 24 | Ensure Python is installed on your system. Python 3.6 or later is required. 25 | 26 | Install necessary Python packages with the following command: 27 | 28 | ``` 29 | pip install streamlit openai datasets 30 | ``` 31 | 32 | 33 | ### 2. Get an OpenAI API Key 34 | 35 | You need an API key from OpenAI to use OpenAI's models. You can get this by creating an account on [OpenAI's website](https://www.openai.com/) and subscribing to their API. 36 | 37 | ### 3. Set up Streamlit Secrets 38 | 39 | Securely set your OpenAI API key using Streamlit secrets. In your Streamlit application directory, run: 40 | 41 | ``` 42 | streamlit secrets new 43 | ``` 44 | 45 | 46 | In the text editor that opens, add the following line: 47 | 48 | ``` 49 | [openai] 50 | openai_key = "your-openai-api-key" 51 | 52 | ``` 53 | 54 | Replace "your-openai-api-key" with your actual OpenAI API key. Save and close the editor. 55 | 56 | ### 4. Run the Streamlit App 57 | 58 | Navigate to the directory containing the Python script for the Streamlit app in your terminal and run: 59 | 60 | ``` 61 | streamlit run your_script.py 62 | ``` -------------------------------------------------------------------------------- /streamlit/wine_prototype/app.py: -------------------------------------------------------------------------------- 1 | # Import required modules 2 | import random 3 | 4 | import openai # OpenAI's Python client library 5 | import streamlit as st # The main library we're using to create a web interface 6 | from datasets import load_dataset 7 | 8 | # Set the OpenAI API key from Streamlit secrets 9 | openai.api_key = st.secrets["openai_key"] 10 | 11 | 12 | @st.cache_resource 13 | def load_wines(): 14 | wine_dataset = load_dataset("alfredodeza/wine-ratings") 15 | return list(wine_dataset['test']) + list(wine_dataset['train']) + list(wine_dataset['validation']) 16 | 17 | 18 | # make a table of wine_dataset to display and make clickable for the user 19 | # Show the table 20 | def convert_wine_to_string(wine): 21 | description = f'{wine["name"]} is from {wine["region"]} and is a {wine["variety"]}. {wine["notes"]}' 22 | 23 | return description 24 | 25 | 26 | def get_recommendations(n=3, user_description=''): 27 | wines = random.sample(load_wines(), n) 28 | st.table(wines) 29 | wines_formatted = "\n---\n".join([convert_wine_to_string(w) for w in wines]) 30 | print(f'User Description: {user_description}\nWines to select from:\n{wines_formatted}') 31 | 32 | chat_completion = openai.ChatCompletion.create( 33 | model='gpt-3.5-turbo', 34 | messages=[ 35 | {'role': 'system', 36 | 'content': 'You are a wine bot that helps clients understand what kind of wine they want. ' 37 | 'Given a list of wines and a description of the client, tell me what wines they ' 38 | 'want by giving me the names of the wines. Include a reason preceding each pick ' 39 | 'to explain to the user why they might like it. Give me the information as a numbered list of' 40 | ' wines with reasons why they might like it.'}, 41 | {'role': 'user', 42 | 'content': f'User Description: {user_description}\nWines to select from:\n{wines_formatted}'} 43 | ] 44 | ) 45 | st.write(chat_completion.choices[0].message.content) 46 | 47 | 48 | user_description = st.text_input( 49 | "Describe the client", "The client likes red wine and is looking for a wine to drink with dinner." 50 | ) 51 | n = st.number_input("How many wines to pull from the cellar?", min_value=1, max_value=10, value=3, step=1) 52 | st.button("Get recommendations", on_click=get_recommendations, kwargs={'n': n, 'user_description': user_description}) 53 | -------------------------------------------------------------------------------- /streamlit/wine_prototype/requirements.txt: -------------------------------------------------------------------------------- 1 | datasets 2 | streamlit 3 | openai --------------------------------------------------------------------------------