├── .gitignore ├── LICENSE ├── README.md ├── app ├── __init__.py ├── chat │ ├── __init__.py │ ├── chat.py │ └── plugins │ │ ├── __init__.py │ │ ├── plugin.py │ │ ├── pythoninterpreter.py │ │ ├── webscraper.py │ │ └── websearch.py ├── routes.py ├── static │ └── img │ │ └── throbber.gif └── templates │ └── chat.html ├── run.py └── web-search-plugin-demo.gif /.gitignore: -------------------------------------------------------------------------------- 1 | venv/* 2 | *.pyc 3 | .env 4 | __pycache__/* 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Abhinav Upadhyay 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Creating ChatGPT Plugins using the Function Call Feature 2 | 3 | The introduction of the function call features in the chat completions API of OpenAI opens up the possibilities of implmenting plugins similar to the plugins supported in ChatGPT. However, 4 | OpenAI documentation does not show how this can be done. I took the opportunity to try to do this myself by building a chat application powered by GPT and then using the function call 5 | feature to design and implment plugins. 6 | 7 | ## Write-Ups 8 | 9 | - This repo contains code for a [tutorial on building ChatGPT like plugins using the newly introduced function call feature.](https://codeconfessions.substack.com/p/creating-chatgpt-plugins-using-the) 10 | In this tutorial we build a Flask-based chat application using the ChatGPT APIs and then proceed to implement a web browsing and Python code interpreter plugin. 11 | 12 | - A follow-up [article on the topic of some gotchas you can run into when using function calls in your code and how to solve them gracefully without hacks in your code](https://codeconfessions.substack.com/p/navigating-sharp-edges-in-openais) 13 | 14 | 15 | ## Structure of a Plugin 16 | Creating a plugin in this system requires doing two things. 17 | - Extend the PluginInterface class, which defines the API that a plugin should follow 18 | - Implement the 4 API methods expected by a plugin. These are: 19 | - `get_name`: Returns the name of the plugin 20 | - `get_description`: Provides a description of what the plugin does 21 | - `get_parameters`: Gives a JSON specification of the parameters of the plugin. 22 | - `execute`: This is the meat of the plugin, where it receives the parameters as declared by it in the `get_parameters` method and it executes its function. 23 | 24 | ### Implemented Plugins 25 | - [Web search plugin](https://github.com/abhinav-upadhyay/chatgpt_plugins/blob/main/app/chat/plugins/websearch.py) 26 | - [Python code interpreter plugin](https://github.com/abhinav-upadhyay/chatgpt_plugins/blob/main/app/chat/plugins/pythoninterpreter.py) 27 | - [Web scraper plugin](https://github.com/abhinav-upadhyay/chatgpt_plugins/blob/main/app/chat/plugins/webscraper.py) 28 | 29 | 30 | ## Setup Requirements for Running This Locally 31 | Install following Python packages in a virtual environment: 32 | 33 | ```shell 34 | pip install openai --upgrade 35 | pip install flask requests python-dotenv 36 | ``` 37 | 38 | ### Create an OpenAI API Key 39 | - Create an OpenAI account and generate a key from their accounts page: [https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys) 40 | - Put the generated API key in the .env file in the root directory of your project as follows: 41 | ```shell 42 | OPEN_AI_KEY="" 43 | ``` 44 | 45 | ### Create Brave Search API Key 46 | This is required for the web search/browser plugin. For this, you need to create an account with Brave at: [https://brave.com/search/api/](https://brave.com/search/api/). Next, you need to create an API key from Brave. You can select the Free plan to start with. The free plan allows 2000 requests per month. Once you have generated the key, put this also in the `.env` file as shown below: 47 | ```shell 48 | BRAVE_API_KEY="" 49 | ``` 50 | ### Generating a key for Flask 51 | Flask also needs a secret key in order to create a user session. You can use something like uuid to generate the key. For example: 52 | ```python 53 | import uuid 54 | print(str(uuid.uuid4())) 55 | ``` 56 | Put the generated value from the above code in the .env, as shown below: 57 | ```shell 58 | CHAT_APP_SECRET_KEY="" 59 | ``` 60 | 61 | ## Running the Application 62 | Use the following command to run the application: 63 | ```shell 64 | flask --app run.py run 65 | ``` 66 | 67 | ## Demo 68 | Following is the web search plugin in action: 69 | ![Web search plugin in action](https://github.com/abhinav-upadhyay/chatgpt_plugins/blob/2388cb60ea93286127228a9145bef91482b5fbad/web-search-plugin-demo.gif) 70 | 71 | -------------------------------------------------------------------------------- /app/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhinav-upadhyay/chatgpt_plugins/f735958b10d1145cd55392b6cef62de1d0450d2d/app/__init__.py -------------------------------------------------------------------------------- /app/chat/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhinav-upadhyay/chatgpt_plugins/f735958b10d1145cd55392b6cef62de1d0450d2d/app/chat/__init__.py -------------------------------------------------------------------------------- /app/chat/chat.py: -------------------------------------------------------------------------------- 1 | import openai 2 | import requests 3 | import json 4 | from typing import List, Dict 5 | import uuid 6 | from .plugins.plugin import PluginInterface 7 | from .plugins.websearch import WebSearchPlugin 8 | from .plugins.webscraper import WebScraperPlugin 9 | from .plugins.pythoninterpreter import PythonInterpreterPlugin 10 | 11 | GPT_MODEL = "gpt-3.5-turbo-16k-0613" 12 | SYSTEM_PROMPT = """ 13 | You are a helpful AI assistant. You answer the user's queries. 14 | When you are not sure of an answer, you take the help of 15 | functions provided to you. 16 | NEVER make up an answer if you don't know, just respond 17 | with "I don't know" when you don't know. 18 | """ 19 | 20 | class Conversation: 21 | """ 22 | This class represents a conversation with the ChatGPT model. 23 | It stores the conversation history in the form of a list of messages. 24 | """ 25 | def __init__(self): 26 | self.conversation_history: List[Dict] = [] 27 | 28 | def add_message(self, role, content): 29 | message = {"role": role, "content": content} 30 | self.conversation_history.append(message) 31 | 32 | 33 | class ChatSession: 34 | """ 35 | Represents a chat session. 36 | Each session has a unique id to associate it with the user. 37 | It holds the conversation history 38 | and provides functionality to get new response from ChatGPT 39 | for user query. 40 | """ 41 | 42 | def __init__(self): 43 | self.session_id = str(uuid.uuid4()) 44 | self.conversation = Conversation() 45 | self.plugins: Dict[str, PluginInterface] = {} 46 | self.register_plugin(WebSearchPlugin()) 47 | self.register_plugin(WebScraperPlugin()) 48 | self.register_plugin(PythonInterpreterPlugin()) 49 | self.conversation.add_message("system", SYSTEM_PROMPT) 50 | 51 | def register_plugin(self, plugin: PluginInterface): 52 | """ 53 | Register a plugin for use in this session 54 | """ 55 | self.plugins[plugin.get_name()] = plugin 56 | 57 | def get_messages(self) -> List[Dict]: 58 | """ 59 | Return the list of messages from the current conversaion 60 | """ 61 | if len(self.conversation.conversation_history) == 1: 62 | return [] 63 | return self.conversation.conversation_history[1:] 64 | 65 | def _get_functions(self) -> List[Dict]: 66 | """ 67 | Generate the list of functions that can be passed to the chatgpt 68 | API call. 69 | """ 70 | return [self._plugin_to_function(p) for 71 | p in self.plugins.values()] 72 | 73 | def _plugin_to_function(self, plugin: PluginInterface) -> Dict: 74 | """ 75 | Convert a plugin to the function call specification as 76 | required by the ChatGPT API: 77 | https://platform.openai.com/docs/api-reference/chat/create#chat/create-functions 78 | """ 79 | function = {} 80 | function["name"] = plugin.get_name() 81 | function["description"] = plugin.get_description() 82 | function["parameters"] = plugin.get_parameters() 83 | return function 84 | 85 | def _execute_plugin(self, func_call) -> str: 86 | """ 87 | If a plugin exists for the given function call, execute it. 88 | """ 89 | func_name = func_call.get("name") 90 | print(f"Executing plugin {func_name}") 91 | if func_name in self.plugins: 92 | arguments = json.loads(func_call.get("arguments")) 93 | plugin = self.plugins[func_name] 94 | plugin_response = plugin.execute(**arguments) 95 | else: 96 | plugin_response = {"error": f"No plugin found with name {func_call}"} 97 | 98 | # We need to pass the plugin response back to ChatGPT 99 | # so that it can process it. In order to do this we 100 | # need to append the plugin response into the conversation 101 | # history. However, this is just temporary so we make a 102 | # copy of the messages and then append to that copy. 103 | print(f"Response from plugin {func_name}: {plugin_response}") 104 | messages = list(self.conversation.conversation_history) 105 | messages.append({"role": "function", 106 | "content": json.dumps(plugin_response), 107 | "name": func_name}) 108 | next_chatgpt_response = self._chat_completion_request(messages) 109 | 110 | # If ChatGPT is asking for another function call, then 111 | # we need to call _execute_plugin again. We will keep 112 | # doing this until ChatGPT keeps returning function_call 113 | # in its response. Although it might be a good idea to 114 | # cut it off at some point to avoid an infinite loop where 115 | # it gets stuck in a plugin loop. 116 | if next_chatgpt_response.get("function_call"): 117 | return self._execute_plugin(next_chatgpt_response.get("function_call")) 118 | return next_chatgpt_response.get("content") 119 | 120 | 121 | def get_chatgpt_response(self, user_message: str) -> str: 122 | """ 123 | For the given user_message, 124 | get the response from ChatGPT 125 | """ 126 | self.conversation.add_message("user", user_message) 127 | try: 128 | chatgpt_response = self._chat_completion_request( 129 | self.conversation.conversation_history) 130 | 131 | if chatgpt_response.get("function_call"): 132 | chatgpt_message = self._execute_plugin( 133 | chatgpt_response.get("function_call")) 134 | else: 135 | chatgpt_message = chatgpt_response.get("content") 136 | self.conversation.add_message("assistant", chatgpt_message) 137 | return chatgpt_message 138 | except Exception as e: 139 | print(e) 140 | return "something went wrong" 141 | 142 | 143 | def _chat_completion_request(self, messages: List[Dict]): 144 | headers = { 145 | "Content-Type": "application/json", 146 | "Authorization": "Bearer " + openai.api_key, 147 | } 148 | json_data = {"model": GPT_MODEL, "messages": messages, "temperature": 0.7} 149 | if self.plugins: 150 | json_data.update({"functions": self._get_functions()}) 151 | try: 152 | response = requests.post( 153 | "https://api.openai.com/v1/chat/completions", 154 | headers=headers, 155 | json=json_data, 156 | ) 157 | return response.json()["choices"][0]["message"] 158 | except Exception as e: 159 | print("Unable to generate ChatCompletion response") 160 | print(f"Exception: {e}") 161 | return e 162 | 163 | -------------------------------------------------------------------------------- /app/chat/plugins/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhinav-upadhyay/chatgpt_plugins/f735958b10d1145cd55392b6cef62de1d0450d2d/app/chat/plugins/__init__.py -------------------------------------------------------------------------------- /app/chat/plugins/plugin.py: -------------------------------------------------------------------------------- 1 | from abc import ABC, abstractmethod 2 | from typing import Dict 3 | 4 | class PluginInterface(ABC): 5 | 6 | @abstractmethod 7 | def get_name(self) -> str: 8 | """ 9 | return the name of the plugin (should be snake case) 10 | """ 11 | pass 12 | 13 | @abstractmethod 14 | def get_description(self) -> str: 15 | """ 16 | return a detailed description of what the plugin does 17 | """ 18 | pass 19 | 20 | @abstractmethod 21 | def get_parameters(self) -> Dict: 22 | """ 23 | Return the list of parameters to execute this plugin in the form of 24 | JSON schema as specified in the OpenAI documentation: 25 | https://platform.openai.com/docs/api-reference/chat/create#chat/create-parameters 26 | """ 27 | pass 28 | 29 | @abstractmethod 30 | def execute(self, **kwargs) -> Dict: 31 | """ 32 | Execute the plugin and return a JSON serializable dict. 33 | The parameters are passed in the form of kwargs 34 | """ 35 | pass 36 | -------------------------------------------------------------------------------- /app/chat/plugins/pythoninterpreter.py: -------------------------------------------------------------------------------- 1 | from .plugin import PluginInterface 2 | from typing import Dict 3 | from io import StringIO 4 | import sys 5 | import traceback 6 | 7 | 8 | 9 | class PythonInterpreterPlugin(PluginInterface): 10 | def get_name(self) -> str: 11 | """ 12 | return the name of the plugin (should be snake case) 13 | """ 14 | return "python_interpreter" 15 | 16 | def get_description(self) -> str: 17 | return """ 18 | Execute the given python code return the result from stdout. 19 | """ 20 | 21 | 22 | def get_parameters(self) -> Dict: 23 | """ 24 | Return the list of parameters to execute this plugin in 25 | the form of JSON schema as specified in the 26 | OpenAI documentation: 27 | https://platform.openai.com/docs/api-reference/chat/create#chat/create-parameters 28 | """ 29 | parameters = { 30 | "type": "object", 31 | "properties": { 32 | "code": { 33 | "type": "string", 34 | "description": "Python code which needs to be executed" 35 | } 36 | } 37 | } 38 | return parameters 39 | 40 | def execute(self, **kwargs) -> Dict: 41 | """ 42 | Execute the plugin and return a JSON response. 43 | The parameters are passed in the form of kwargs 44 | """ 45 | output = StringIO() 46 | 47 | try: 48 | global_namespace = {} 49 | local_namespace = {} 50 | sys.stdout = output 51 | exec(kwargs['code'], local_namespace, global_namespace) 52 | result = output.getvalue() 53 | if not result: 54 | return {'error': 'No result written to stdout. Please print result on stdout'} 55 | return {"result": result} 56 | except Exception: 57 | error = traceback.format_exc() 58 | return {"error": error} 59 | finally: 60 | sys.stdout = sys.__stdout__ 61 | 62 | 63 | 64 | -------------------------------------------------------------------------------- /app/chat/plugins/webscraper.py: -------------------------------------------------------------------------------- 1 | from .plugin import PluginInterface 2 | from typing import Dict 3 | import requests 4 | from bs4 import BeautifulSoup 5 | 6 | 7 | 8 | class WebScraperPlugin(PluginInterface): 9 | def get_name(self) -> str: 10 | """ 11 | return the name of the plugin (should be snake case) 12 | """ 13 | return "webscraper" 14 | 15 | def get_description(self) -> str: 16 | return "Extracts the content of the web page at the given URL" 17 | 18 | 19 | def get_parameters(self) -> Dict: 20 | """ 21 | Return the list of parameters to execute this plugin in the form of 22 | JSON schema as specified in the OpenAI documentation: 23 | https://platform.openai.com/docs/api-reference/chat/create#chat/create-parameters 24 | """ 25 | parameters = { 26 | "type": "object", 27 | "properties": { 28 | "url": { 29 | "type": "string", 30 | "description": "URL of the web page which needs to be scraped" 31 | } 32 | } 33 | } 34 | return parameters 35 | 36 | def execute(self, **kwargs) -> Dict: 37 | """ 38 | Execute the plugin and return a JSON response. 39 | The parameters are passed in the form of kwargs 40 | """ 41 | 42 | # Send a GET request to the URL 43 | response = requests.get(kwargs['url']) 44 | 45 | # Create a BeautifulSoup object to parse the HTML 46 | soup = BeautifulSoup(response.text, "html.parser") 47 | 48 | # Extract the text content from the parsed HTML 49 | text_content = soup.get_text() 50 | return {"content": text_content} 51 | 52 | 53 | -------------------------------------------------------------------------------- /app/chat/plugins/websearch.py: -------------------------------------------------------------------------------- 1 | from .plugin import PluginInterface 2 | from typing import Dict 3 | import requests 4 | import os 5 | 6 | BRAVE_API_KEY = os.getenv("BRAVE_API_KEY") 7 | BRAVE_API_URL = "https://api.search.brave.com/res/v1/web/search" 8 | 9 | 10 | class WebSearchPlugin(PluginInterface): 11 | def get_name(self) -> str: 12 | """ 13 | return the name of the plugin (should be snake case) 14 | """ 15 | return "websearch" 16 | 17 | def get_description(self) -> str: 18 | return """ 19 | Executes a web search for the given query 20 | and returns a list of snipptets of matching 21 | text from top 10 pages 22 | """ 23 | 24 | 25 | def get_parameters(self) -> Dict: 26 | """ 27 | Return the list of parameters to execute this plugin in the form of 28 | JSON schema as specified in the OpenAI documentation: 29 | https://platform.openai.com/docs/api-reference/chat/create#chat/create-parameters 30 | """ 31 | parameters = { 32 | "type": "object", 33 | "properties": { 34 | "q": { 35 | "type": "string", 36 | "description": "the user query" 37 | } 38 | } 39 | } 40 | return parameters 41 | 42 | def execute(self, **kwargs) -> Dict: 43 | """ 44 | Execute the plugin and return a JSON response. 45 | The parameters are passed in the form of kwargs 46 | """ 47 | 48 | headers = { 49 | "Accept": "application/json", 50 | "X-Subscription-Token": BRAVE_API_KEY 51 | } 52 | 53 | params = { 54 | "q": kwargs["q"] 55 | } 56 | 57 | response = requests.get(BRAVE_API_URL, 58 | headers=headers, 59 | params=params) 60 | 61 | if response.status_code == 200: 62 | results = response.json()['web']['results'] 63 | snippets = [r['description'] for r in results] 64 | return {"web_search_results": snippets} 65 | else: 66 | return {"error": 67 | f"Request failed with status code: {response.status_code}"} 68 | 69 | -------------------------------------------------------------------------------- /app/routes.py: -------------------------------------------------------------------------------- 1 | from flask import Flask, render_template, request, session, jsonify 2 | from typing import Dict 3 | 4 | from flask import Flask 5 | import os 6 | from dotenv import load_dotenv 7 | from .chat.chat import ChatSession 8 | 9 | load_dotenv() 10 | app = Flask(__name__) 11 | app.secret_key = os.getenv("CHAT_APP_SECRET_KEY") 12 | 13 | 14 | chat_sessions: Dict[str, ChatSession] = {} 15 | 16 | @app.route("/") 17 | def index(): 18 | chat_session = _get_user_session() 19 | return render_template("chat.html", conversation=chat_session.get_messages()) 20 | 21 | @app.route('/chat', methods=['POST']) 22 | def chat(): 23 | message: str = request.json['message'] 24 | chat_session = _get_user_session() 25 | chatgpt_message = chat_session.get_chatgpt_response(message) 26 | return jsonify({"message": chatgpt_message}) 27 | 28 | 29 | 30 | def _get_user_session() -> ChatSession: 31 | chat_session_id = session.get("chat_session_id") 32 | if chat_session_id: 33 | chat_session = chat_sessions.get(chat_session_id) 34 | if not chat_session: 35 | chat_session = ChatSession() 36 | chat_sessions[chat_session.session_id] = chat_session 37 | session["chat_session_id"] = chat_session.session_id 38 | else: 39 | chat_session = ChatSession() 40 | chat_sessions[chat_session.session_id] = chat_session 41 | session["chat_session_id"] = chat_session.session_id 42 | return chat_session -------------------------------------------------------------------------------- /app/static/img/throbber.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhinav-upadhyay/chatgpt_plugins/f735958b10d1145cd55392b6cef62de1d0450d2d/app/static/img/throbber.gif -------------------------------------------------------------------------------- /app/templates/chat.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Chat App 5 | 9 | 67 | 68 | 69 |
70 | 71 |
    72 |
    73 | 79 | 80 | 81 |
    82 |
    83 | 84 | 183 | 184 | 185 | -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- 1 | from app.routes import app 2 | 3 | if __name__ == '__main__': 4 | 5 | app.run(host='0.0.0.0:5000', debug=True) 6 | 7 | -------------------------------------------------------------------------------- /web-search-plugin-demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/abhinav-upadhyay/chatgpt_plugins/f735958b10d1145cd55392b6cef62de1d0450d2d/web-search-plugin-demo.gif --------------------------------------------------------------------------------