├── .gitignore ├── README.md ├── function_documentation.txt ├── gen_function_calling.gbnf ├── generate_example_functions_grammar_and_documentation.py └── llm_function_calling ├── __init__.py ├── function_call.py ├── function_calling_grammar_generator.py ├── gpt_functions.py ├── llm_function_caller.py ├── llm_function_calling ├── __init__.py ├── function_call.py ├── function_calling_grammar_generator.py ├── gpt_functions.py ├── llm_function_caller.py └── primitive.gbnf └── primitive.gbnf /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | llm_function_calling/__pycache__/__init__.cpython-311.pyc 3 | llm_function_calling/__pycache__/gpt_functions.cpython-311.pyc 4 | llm_function_calling/__pycache__/function_calling_grammar_generator.cpython-311.pyc 5 | llm_function_calling/__pycache__/function_call.cpython-311.pyc 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # OLD Project - New Project [GitHub link](https://github.com/Maximilian-Winter/llama-cpp-agent). 2 | 3 | 4 | 5 | ### llama.cpp Grammar Generator for LLM Function Calls 6 | 7 | #### Overview 8 | This package is designed for generating grammars and documentation for structured function calls in GGML BNF (GGML Backus-Naur Form), tailored for Large Language Models (LLMs) used with llama.cpp. The documentation is designed to explain the functions to the LLM. The `LLMFunctionCaller` is for dynamic function execution based on JSON inputs generated through the grammar generated. 9 | 10 | At the moment it supports the following types as function parameters: 11 | 12 | - `object` 13 | - `array` 14 | - `string` 15 | - `boolean` 16 | - `number` 17 | - `float` 18 | 19 | #### Components 20 | `function_call.py`: 21 | - `FunctionParameter`: Class to define a parameter of a function, including type, requirement status, description, and enumeration values if applicable. 22 | - `FunctionParameters`: Class to encapsulate multiple `FunctionParameter` instances. 23 | - `FunctionCall`: Class representing a function call, including its name and parameters. 24 | 25 | `function_calling_grammar_generator.py`: 26 | - `format_function_names`, `generate_gbnf_grammar`, `generate_gbnf_rule`, `capitalize_rule_name`: Functions to generate GGML BNF grammar based on the defined functions. 27 | - `generate_documentation`, `save_documentation_to_file`: Functions to generate and save documentation for the function calls. 28 | - `save_grammar_to_file`: Function to save the generated GGML BNF grammar to a file. 29 | 30 | `gpt_functions.py`: 31 | Example usage showing generating `MemGPT` like functions. 32 | 33 | `llm_function_caller.py`: 34 | - Dynamically executes functions based on JSON input generated through the grammar generated. 35 | - Supports adding new functions and parameter transformers at runtime. 36 | - Incorporates robust error handling and logging. 37 | 38 | #### Usage 39 | 1. **Defining Function Calls**: Create `FunctionCall` instances for each function you want the LLM to call, defining parameters using `FunctionParameter` and `FunctionParameters`. 40 | 2. **Generating GGML BNF Grammar**: Use `generate_gbnf_grammar` to create GGML BNF grammar rules for these function calls, for use with llama.cpp. 41 | 3. **Generating Documentation**: Use `generate_documentation` to produce human-readable documentation for these function calls. 42 | 4. **Generate Function Call with LLM**: Use llama.cpp to generate a JSON function call with the grammar. 43 | 5. **Executing Function Calls with LLMFunctionCaller**: Use the `LLMFunctionCaller` to execute these defined function calls dynamically. 44 | 45 | #### LLMFunctionCaller Methods 46 | - `add_function(name, function)`: Adds a function to the executor. 47 | - `add_param_transformer(function_name, transformer)`: Adds a parameter transformer for a specific function. 48 | - `execute_function(json_input)`: Executes a function based on JSON input generated through the grammar generated. 49 | 50 | #### Example for LLMFunctionCaller 51 | ```python 52 | function_caller = LLMFunctionCaller() 53 | function_caller.add_function("sample_function", sample_function) 54 | function_caller.add_param_transformer("sample_function", example_transformer) 55 | json_input = '{"function":"sample_function","params":{"arg1": "hello", "arg2": "world"}}' 56 | result = function_caller.execute_function(json_input) 57 | print(result) 58 | ``` 59 | 60 | #### File Saving 61 | - Use `save_grammar_to_file` to save the generated GGML BNF grammar. 62 | - Use `save_documentation_to_file` to save the documentation. 63 | 64 | ### Example output using OpenHermes and the example functions in `gpt_functions.py` converted to a grammar 65 | 66 | 67 | ````text 68 | >Can you write a long poem about the USA in the "HelloUSA.txt" file? 69 | <|im_start|>system 70 | Available Functions: 71 | 72 | send_message: 73 | Description:Sends a message to the User. 74 | Parameters: 75 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the message. 76 | message (string, required): Message you want to send to the user. 77 | 78 | read_file: 79 | Description:Returns content of a file. 80 | Parameters: 81 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the file path. 82 | File (string, required): The path of the file you want to open. 83 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 84 | 85 | write_file: 86 | Description:Writes to a file. 87 | Parameters: 88 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the file. 89 | File (string, required): The path of the file you want to write. 90 | Content (string, required): The content of the file you want to write. 91 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 92 | 93 | python_interpreter_command: 94 | Description:Execute Python command. 95 | Parameters: 96 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the command. 97 | command (string, required): The Python command to execute. 98 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together.<|im_end|> 99 | <|im_start|>user 100 | Maximilian: "Can you write a long poem about the USA in the "HelloUSA.txt" file?" <|im_end|> 101 | <|im_start|>assistant 102 | Llama.generate: prefix-match hit 103 | {"function":"write_file","params":{"inner_thoughts":"I'm thinking about composing a poem on the United States and saving it as 'HelloUSA.txt'.","File":"HelloUSA.txt","Content":"In search of liberty, they came from far and wide,\nA melting pot of cultures, a land so grand and vast.\nFrom coast to coast, through mountains high and tide,\nTheir dreams intertwined, their stories merged at last.\n\nBoldly they ventured, brave and free,\nSeeking opportunity, hope, and solace too,\nWith unity and strength, they forged a nation strong,\nAnd freedom's light would forever shine anew.\n\nThrough strife and struggle, battles fierce and fight,\nThey built a home where all could find their place,\nWhere justice reigns, and rights are right,\nAnd every soul can carve its own unique grace.\n\nUnited we stand, with colors bold,\nRed, white, and blue our flag unfurls,\nTogether now, our future told,\nOf unity, not division, evermore.\n\nThis land of dreams, where destiny unfolds,\nA beacon bright, a shining star that guides,\nA story written by hearts of gold,\nAcross these lands, across these tides.","require_heartbeat":true}} 104 | ```` 105 | -------------------------------------------------------------------------------- /function_documentation.txt: -------------------------------------------------------------------------------- 1 | Available Functions: 2 | 3 | send_message: 4 | Description: Sends a message to the User. 5 | Parameters: 6 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the message. 7 | message (string, required): Message you want to send to the user. 8 | 9 | core_memory_Append: 10 | Description: Append to Core Memory. 11 | Parameters: 12 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the value. 13 | key (string, required): The memory key, like 'persona', 'goals', 'situation' or 'human'. 14 | field (string, required): The field which should be appended. 15 | value (string, required): The information you would like to save. 16 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 17 | 18 | core_memory_replace: 19 | Description: Replace parts of Core Memory. 20 | Parameters: 21 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the value. 22 | key (string, required): The memory key, like 'persona', 'goals', 'situation' or 'human'. 23 | field (string, required): The field where the memory should be replaced like 'personality' or 'traits'. 24 | value (string, required): The information you would like to write into the field. 25 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 26 | 27 | archival_memory_insert: 28 | Description: Add memory to Archival Memory. 29 | Parameters: 30 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the value, 31 | value (string, required): The content you want to add. 32 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 33 | 34 | archival_memory_search: 35 | Description: Search in Archival Memory. 36 | Parameters: 37 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the query. 38 | query (string, required): Your search query. 39 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 40 | 41 | cmd_command: 42 | Description: Execute CMD command. 43 | Parameters: 44 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the command. 45 | command (string, required): The CMD command to execute. 46 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 47 | 48 | web_browsing: 49 | Description: Opens a website and returns content. 50 | Parameters: 51 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the url. 52 | URL (string, required): The URL you want to access. 53 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 54 | 55 | web_download: 56 | Description: Downloads a file. 57 | Parameters: 58 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the url. 59 | URL (string, required): The URL you want to download. 60 | Path (string, required): The Path you want to download the file to. 61 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 62 | 63 | read_file: 64 | Description: Returns content of a file. 65 | Parameters: 66 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the file path. 67 | File (string, required): The path of the file you want to open. 68 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 69 | 70 | write_file: 71 | Description: Writes to a file. 72 | Parameters: 73 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the file. 74 | File (string, required): The path of the file you want to write. 75 | Content (string, required): The content of the file you want to write. 76 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 77 | 78 | python_interpreter_command: 79 | Description: Execute Python command. 80 | Parameters: 81 | inner_thoughts (string, required): Your inner thoughts or inner monologue while writing the command. 82 | command (string, required): The Python command to execute. 83 | require_heartbeat (boolean, required): Set this to true to get control back after execution, to chain functions together. 84 | 85 | -------------------------------------------------------------------------------- /gen_function_calling.gbnf: -------------------------------------------------------------------------------- 1 | root ::= Function 2 | Function ::= SendMessage | CoreMemoryAppend | CoreMemoryReplace | ArchivalMemoryInsert | ArchivalMemorySearch | CmdCommand | WebBrowsing | WebDownload | ReadFile | WriteFile | PythonInterpreterCommand 3 | SendMessage ::= "{" ws "\"function\":" ws "\"send_message\"," ws "\"params\":" ws SendMessageParams "}" 4 | CoreMemoryAppend ::= "{" ws "\"function\":" ws "\"core_memory_Append\"," ws "\"params\":" ws CoreMemoryAppendParams "}" 5 | CoreMemoryReplace ::= "{" ws "\"function\":" ws "\"core_memory_replace\"," ws "\"params\":" ws CoreMemoryReplaceParams "}" 6 | ArchivalMemoryInsert ::= "{" ws "\"function\":" ws "\"archival_memory_insert\"," ws "\"params\":" ws ArchivalMemoryInsertParams "}" 7 | ArchivalMemorySearch ::= "{" ws "\"function\":" ws "\"archival_memory_search\"," ws "\"params\":" ws ArchivalMemorySearchParams "}" 8 | CmdCommand ::= "{" ws "\"function\":" ws "\"cmd_command\"," ws "\"params\":" ws CmdCommandParams "}" 9 | WebBrowsing ::= "{" ws "\"function\":" ws "\"web_browsing\"," ws "\"params\":" ws WebBrowsingParams "}" 10 | WebDownload ::= "{" ws "\"function\":" ws "\"web_download\"," ws "\"params\":" ws WebDownloadParams "}" 11 | ReadFile ::= "{" ws "\"function\":" ws "\"read_file\"," ws "\"params\":" ws ReadFileParams "}" 12 | WriteFile ::= "{" ws "\"function\":" ws "\"write_file\"," ws "\"params\":" ws WriteFileParams "}" 13 | PythonInterpreterCommand ::= "{" ws "\"function\":" ws "\"python_interpreter_command\"," ws "\"params\":" ws PythonInterpreterCommandParams "}" 14 | SendMessageParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"message\":" ws string ws "}" 15 | CoreMemoryAppendParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"key\":" ws string"," ws "\"field\":" ws string"," ws "\"value\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 16 | CoreMemoryReplaceParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"key\":" ws string"," ws "\"field\":" ws string"," ws "\"value\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 17 | ArchivalMemoryInsertParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"value\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 18 | ArchivalMemorySearchParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"query\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 19 | CmdCommandParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"command\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 20 | WebBrowsingParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"URL\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 21 | WebDownloadParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"URL\":" ws string"," ws "\"Path\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 22 | ReadFileParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"File\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 23 | WriteFileParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"File\":" ws string"," ws "\"Content\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 24 | PythonInterpreterCommandParams ::= "{" ws "\"inner_thoughts\":" ws string"," ws "\"command\":" ws string"," ws "\"require_heartbeat\":" ws boolean ws "}" 25 | string ::= "\"" ([^"\[\]{}]*) "\"" 26 | boolean ::= "true" | "false" 27 | ws ::= "" 28 | number ::= [0-9]+ 29 | float ::= number "." number [exponent] 30 | exponent ::= ("e" | "E") ["+" | "-"] number -------------------------------------------------------------------------------- /generate_example_functions_grammar_and_documentation.py: -------------------------------------------------------------------------------- 1 | from llm_function_calling import gpt_functions 2 | 3 | 4 | gpt_functions.generate_gpt_functions_grammar_and_documentation() 5 | 6 | -------------------------------------------------------------------------------- /llm_function_calling/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Maximilian-Winter/llama_cpp_function_calling/70a3c608328a4cae7077c47914f593cedfe234cd/llm_function_calling/__init__.py -------------------------------------------------------------------------------- /llm_function_calling/function_call.py: -------------------------------------------------------------------------------- 1 | from enum import Enum 2 | from typing import Optional, Dict, List 3 | 4 | 5 | class DataType(Enum): 6 | STRING = "string" 7 | BOOLEAN = "boolean" 8 | NUMBER = "number" 9 | FLOAT = "float" 10 | OBJECT = "object" # New type for objects 11 | ARRAY = "array" # New type for arrays 12 | 13 | 14 | class FunctionParameter: 15 | def __init__(self, parameter_type: DataType, required: bool, 16 | description: Optional[str] = None, 17 | enum: Optional[List[str]] = None, 18 | structure: Optional[Dict[str, 'FunctionParameter']] = None, # For objects 19 | element_type: Optional['FunctionParameter'] = None): # For arrays 20 | self.type = parameter_type 21 | self.required = required 22 | self.description = description 23 | self.enum = enum 24 | self.structure = structure 25 | self.element_type = element_type 26 | 27 | 28 | class FunctionParameters: 29 | def __init__(self, properties: Dict[str, FunctionParameter]): 30 | self.properties = properties 31 | 32 | 33 | class FunctionCall: 34 | def __init__(self, name: str, description: str, parameters: FunctionParameters): 35 | self.name = name 36 | self.parameters = parameters 37 | self.description = description 38 | -------------------------------------------------------------------------------- /llm_function_calling/function_calling_grammar_generator.py: -------------------------------------------------------------------------------- 1 | import os 2 | from typing import List, Dict 3 | 4 | from .function_call import FunctionCall, DataType, FunctionParameter 5 | 6 | 7 | def format_function_names(function_calls): 8 | # Extract the names from each FunctionCall instance 9 | function_names = [capitalize_rule_name(func.name) for func in function_calls] 10 | 11 | # Join the names with ' | ' as separator 12 | formatted_names = ' | '.join(function_names) 13 | 14 | return formatted_names 15 | 16 | 17 | def generate_object_rule(name: str, structure: Dict[str, FunctionParameter]) -> str: 18 | """ 19 | Generate GBNF rules for an object with given structure. 20 | """ 21 | rules = [f"{name} ::= \"{{\" ws "] 22 | for i, (field, param) in enumerate(structure.items()): 23 | if i > 0: 24 | rules.append("\",\" ws ") 25 | rules.append(f"\"\\\"{field}\\\":\" ws {param.type.value}") 26 | rules.append(" ws \"}\"") 27 | return ''.join(rules) 28 | 29 | 30 | def generate_array_rule(name: str, element_type: FunctionParameter) -> str: 31 | """ 32 | Generate GBNF rules for an array of a specific type. 33 | """ 34 | return f"{name} ::= \"[\" ws ({element_type.type.value} (\",\" ws {element_type.type.value})*)? ws \"]\"" 35 | 36 | 37 | def generate_gbnf_rule(function_call): 38 | """ 39 | Generate a GBNF grammar rule for a given FunctionCall with capitalized names and comma handling, 40 | including support for OBJECT and ARRAY data types. 41 | """ 42 | 43 | # Capitalize and format the function name 44 | function_name = capitalize_rule_name(function_call.name) 45 | 46 | # Start the main function rule 47 | function_rule = f"{function_name} ::= \"{{\" ws \"\\\"function\\\":\" ws \"\\\"{function_call.name}\\\",\" ws \"\\\"params\\\":\" ws {function_name}Params \"}}\"" 48 | 49 | # Initialize parameter rules 50 | param_rules = f"{function_name}Params ::= \"{{\" ws " 51 | 52 | # Iterate over parameters and generate rules 53 | for i, (param_name, param) in enumerate(function_call.parameters.properties.items()): 54 | if i > 0: 55 | param_rules += "\",\" ws " # Include a comma separator for multiple parameters 56 | 57 | # Handling different data types 58 | if param.type in [DataType.OBJECT, DataType.ARRAY]: 59 | # Generate a specific rule for OBJECT or ARRAY type 60 | nested_rule_name = f"{function_name}{param_name.capitalize()}" 61 | param_rules += f"\"\\\"{param_name}\\\":\" ws {nested_rule_name} " 62 | 63 | if param.type == DataType.OBJECT: 64 | object_rule = generate_object_rule(nested_rule_name, param.structure) 65 | param_rules += "\n" + object_rule 66 | elif param.type == DataType.ARRAY: 67 | array_rule = generate_array_rule(nested_rule_name, param.element_type) 68 | param_rules += "\n" + array_rule 69 | else: 70 | # Handle basic data types 71 | param_rules += f"\"\\\"{param_name}\\\":\" ws {param.type.value}" 72 | 73 | # Close the parameter rules 74 | param_rules += " ws \"}\"" 75 | 76 | return function_rule, param_rules 77 | 78 | 79 | def generate_gbnf_grammar(function_calls: List[FunctionCall]) -> str: 80 | """ 81 | Generate a complete GBNF grammar from a list of FunctionCall instances, 82 | placing all function rules first followed by all parameter rules. 83 | """ 84 | # Start with the root rule 85 | root_rule = "root ::= " + format_function_names(function_calls) 86 | 87 | # Initialize lists to store function rules and parameter rules separately 88 | function_rules = [root_rule] 89 | param_rules = [] 90 | 91 | # Iterate over each FunctionCall and generate corresponding GBNF rules 92 | for function_call in function_calls: 93 | function_rule, param_rule = generate_gbnf_rule(function_call) 94 | 95 | # Append the generated rules to their respective lists 96 | function_rules.append(function_rule) 97 | param_rules.append(param_rule) 98 | 99 | # Combine function rules and parameter rules into a single grammar string 100 | complete_grammar = "\n".join(function_rules + param_rules) 101 | 102 | return complete_grammar 103 | 104 | 105 | def capitalize_rule_name(name): 106 | """ 107 | Capitalize each part of the rule name. 108 | """ 109 | return ''.join(word.capitalize() for word in name.split('_')) 110 | 111 | 112 | def save_grammar_to_file(grammar_str, file_path): 113 | try: 114 | script_path = os.path.abspath(__file__) 115 | script_dir = os.path.dirname(script_path) 116 | with open(f"{script_dir}/primitive.gbnf", 'r', encoding='utf-8') as file: 117 | primary_grammar = file.read() 118 | with open(file_path, 'w', encoding='utf-8') as file: 119 | file.write(grammar_str + "\n" + primary_grammar) 120 | print(f"Grammar successfully saved to {file_path}") 121 | except IOError as e: 122 | print(f"An error occurred while writing to the file: {e}") 123 | 124 | 125 | def generate_documentation(function_calls): 126 | """ 127 | Generate documentation for a list of FunctionCall instances, including details of objects and arrays. 128 | """ 129 | 130 | def document_parameter(param_name, param, indent_level): 131 | """ 132 | Recursively document a parameter, handling objects and arrays. 133 | """ 134 | indent = " " * indent_level 135 | doc = f"{indent}{param_name} ({param.type.value}, {'required' if param.required else 'optional'}): {param.description}\n" 136 | 137 | if param.type == DataType.OBJECT and param.structure: 138 | doc += f"{indent} Structure:\n" 139 | for sub_param_name, sub_param in param.structure.items(): 140 | doc += document_parameter(sub_param_name, sub_param, indent_level + 2) 141 | 142 | elif param.type == DataType.ARRAY and param.element_type: 143 | doc += f"{indent} Element Type:\n" 144 | doc += document_parameter("item", param.element_type, indent_level + 2) 145 | 146 | return doc 147 | 148 | documentation = "Available Functions:\n\n" 149 | for func in function_calls: 150 | # Function name and description 151 | documentation += f"{func.name}:\n Description: {func.description}\n Parameters:\n" 152 | 153 | # Parameters and their descriptions 154 | for param_name, param in func.parameters.properties.items(): 155 | documentation += document_parameter(param_name, param, 2) 156 | 157 | documentation += "\n" # Add a blank line between function calls 158 | 159 | return documentation 160 | 161 | 162 | def save_documentation_to_file(documentation, file_path): 163 | try: 164 | with open(file_path, 'w') as file: 165 | file.write(documentation) 166 | print(f"Documentation successfully saved to {file_path}") 167 | except IOError as e: 168 | print(f"An error occurred while saving the file: {e}") 169 | -------------------------------------------------------------------------------- /llm_function_calling/gpt_functions.py: -------------------------------------------------------------------------------- 1 | from .function_call import DataType 2 | from .function_calling_grammar_generator import generate_gbnf_grammar, generate_documentation, \ 3 | save_documentation_to_file 4 | from .function_calling_grammar_generator import save_grammar_to_file 5 | from .function_call import FunctionCall, FunctionParameters, FunctionParameter 6 | 7 | # SendMessage FunctionCall 8 | send_message = FunctionCall( 9 | name='send_message', 10 | description="Sends a message to the User.", 11 | parameters=FunctionParameters({ 12 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 13 | description="Your inner thoughts or inner monologue while writing the message."), 14 | "message": FunctionParameter(parameter_type=DataType.STRING, required=True, 15 | description="Message you want to send to the user.") 16 | }) 17 | ) 18 | 19 | # CoreMemoryAppend FunctionCall 20 | core_memory_append = FunctionCall( 21 | name='core_memory_Append', 22 | description="Append to Core Memory.", 23 | parameters=FunctionParameters({ 24 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 25 | description="Your inner thoughts or inner monologue while writing the value."), 26 | "key": FunctionParameter(parameter_type=DataType.STRING, required=True, 27 | description="The memory key, like 'persona', 'goals', 'situation' or 'human'."), 28 | "field": FunctionParameter(parameter_type=DataType.STRING, required=True, 29 | description="The field which should be appended."), 30 | "value": FunctionParameter(parameter_type=DataType.STRING, required=True, 31 | description="The information you would like to save."), 32 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 33 | description="Set this to true to get control back after execution, to chain functions together.") 34 | }) 35 | ) 36 | 37 | # CoreMemoryReplace FunctionCall 38 | core_memory_replace = FunctionCall( 39 | name='core_memory_replace', 40 | description="Replace parts of Core Memory.", 41 | parameters=FunctionParameters({ 42 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 43 | description="Your inner thoughts or inner monologue while writing the value."), 44 | "key": FunctionParameter(parameter_type=DataType.STRING, required=True, 45 | description="The memory key, like 'persona', 'goals', 'situation' or 'human'."), 46 | "field": FunctionParameter(parameter_type=DataType.STRING, required=True, 47 | description="The field where the memory should be replaced like 'personality' or 'traits'."), 48 | "value": FunctionParameter(parameter_type=DataType.STRING, required=True, 49 | description="The information you would like to write into the field."), 50 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 51 | description="Set this to true to get control back after execution, to chain functions together.") 52 | }) 53 | ) 54 | 55 | # ArchivalMemoryInsert FunctionCall 56 | archival_memory_insert = FunctionCall( 57 | name='archival_memory_insert', 58 | description="Add memory to Archival Memory.", 59 | parameters=FunctionParameters({ 60 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 61 | description="Your inner thoughts or inner monologue while writing the value,"), 62 | "value": FunctionParameter(parameter_type=DataType.STRING, required=True, 63 | description="The content you want to add."), 64 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 65 | description="Set this to true to get control back after execution, to chain functions together.") 66 | }) 67 | ) 68 | 69 | # ArchivalMemorySearch FunctionCall 70 | archival_memory_search = FunctionCall( 71 | name='archival_memory_search', 72 | description="Search in Archival Memory.", 73 | parameters=FunctionParameters({ 74 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 75 | description="Your inner thoughts or inner monologue while writing the query."), 76 | "query": FunctionParameter(parameter_type=DataType.STRING, required=True, 77 | description="Your search query."), 78 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 79 | description="Set this to true to get control back after execution, to chain functions together.") 80 | }) 81 | ) 82 | 83 | # CmdCommand FunctionCall 84 | cmd_command = FunctionCall( 85 | name='cmd_command', 86 | description="Execute CMD command.", 87 | parameters=FunctionParameters({ 88 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 89 | description="Your inner thoughts or inner monologue while writing the command."), 90 | "command": FunctionParameter(parameter_type=DataType.STRING, required=True, 91 | description="The CMD command to execute."), 92 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 93 | description="Set this to true to get control back after execution, to chain functions together.") 94 | }) 95 | ) 96 | 97 | # WebBrowsing FunctionCall 98 | web_browsing = FunctionCall( 99 | name='web_browsing', 100 | description="Opens a website and returns content.", 101 | parameters=FunctionParameters({ 102 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 103 | description="Your inner thoughts or inner monologue while writing the url."), 104 | "URL": FunctionParameter(parameter_type=DataType.STRING, required=True, 105 | description="The URL you want to access."), 106 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 107 | description="Set this to true to get control back after execution, to chain functions together.") 108 | }) 109 | ) 110 | 111 | # WebDownload FunctionCall 112 | web_download = FunctionCall( 113 | name='web_download', 114 | description="Downloads a file.", 115 | parameters=FunctionParameters({ 116 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 117 | description="Your inner thoughts or inner monologue while writing the url."), 118 | "URL": FunctionParameter(parameter_type=DataType.STRING, required=True, 119 | description="The URL you want to download."), 120 | "Path": FunctionParameter(parameter_type=DataType.STRING, required=True, 121 | description="The Path you want to download the file to."), 122 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 123 | description="Set this to true to get control back after execution, to chain functions together.") 124 | }) 125 | ) 126 | 127 | # ReadFile FunctionCall 128 | read_file = FunctionCall( 129 | name='read_file', 130 | description="Returns content of a file.", 131 | parameters=FunctionParameters({ 132 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 133 | description="Your inner thoughts or inner monologue while writing the file path."), 134 | "File": FunctionParameter(parameter_type=DataType.STRING, required=True, 135 | description="The path of the file you want to open."), 136 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 137 | description="Set this to true to get control back after execution, to chain functions together.") 138 | }) 139 | ) 140 | 141 | # WriteFile FunctionCall 142 | write_file = FunctionCall( 143 | name='write_file', 144 | description="Writes to a file.", 145 | parameters=FunctionParameters({ 146 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 147 | description="Your inner thoughts or inner monologue while writing the file."), 148 | "File": FunctionParameter(parameter_type=DataType.STRING, required=True, 149 | description="The path of the file you want to write."), 150 | "Content": FunctionParameter(parameter_type=DataType.STRING, required=True, 151 | description="The content of the file you want to write."), 152 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 153 | description="Set this to true to get control back after execution, to chain functions together.") 154 | }) 155 | ) 156 | 157 | # PythonInterpreterCommand FunctionCall 158 | python_interpreter_command = FunctionCall( 159 | name='python_interpreter_command', 160 | description="Execute Python command.", 161 | parameters=FunctionParameters({ 162 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 163 | description="Your inner thoughts or inner monologue while writing the command."), 164 | "command": FunctionParameter(parameter_type=DataType.STRING, required=True, 165 | description="The Python command to execute."), 166 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 167 | description="Set this to true to get control back after execution, to chain functions together.") 168 | }) 169 | ) 170 | 171 | 172 | def generate_gpt_functions_grammar_and_documentation(): 173 | # List of FunctionCall instances 174 | function_calls = [send_message, core_memory_append, core_memory_replace, archival_memory_insert, 175 | archival_memory_search, cmd_command, web_browsing, web_download, read_file, 176 | write_file, python_interpreter_command] 177 | 178 | # function_calls = [create_user_profile, create_order] 179 | # Generate the documentation 180 | doc = generate_documentation(function_calls) 181 | print(doc) 182 | save_documentation_to_file(doc, 'function_documentation.txt') 183 | 184 | save_grammar_to_file(generate_gbnf_grammar(function_calls), "gen_function_calling.gbnf") 185 | 186 | 187 | class Address: 188 | def __init__(self, street, city, zip_code): 189 | self.street = street 190 | self.city = city 191 | self.zip_code = zip_code 192 | 193 | 194 | class UserProfile: 195 | def __init__(self, username, email, address: Address, is_active): 196 | self.username = username 197 | self.email = email 198 | self.address = address 199 | self.is_active = is_active 200 | 201 | 202 | class OrderItem: 203 | def __init__(self, item_id, quantity): 204 | self.item_id = item_id 205 | self.quantity = quantity 206 | 207 | 208 | class Order: 209 | def __init__(self, user_id, items: [OrderItem]): 210 | self.user_id = user_id 211 | self.items = items 212 | 213 | 214 | # Define the structure for an Address in GBNF 215 | address_structure = { 216 | "street": FunctionParameter(DataType.STRING, True, "Street name and number"), 217 | "city": FunctionParameter(DataType.STRING, True, "City name"), 218 | "zip_code": FunctionParameter(DataType.STRING, True, "Postal or ZIP code") 219 | } 220 | 221 | # Define the structure for a UserProfile in GBNF 222 | user_profile_params = FunctionParameters({ 223 | "username": FunctionParameter(DataType.STRING, True, "User's unique username"), 224 | "email": FunctionParameter(DataType.STRING, True, "User's email address"), 225 | "address": FunctionParameter(DataType.OBJECT, True, "User's physical address", structure=address_structure), 226 | "is_active": FunctionParameter(DataType.BOOLEAN, True, "Indicates if the user profile is active") 227 | }) 228 | 229 | # Create FunctionCall instance for creating a user profile 230 | create_user_profile = FunctionCall("create_user_profile", 231 | "Create a new user profile with username, email, address, and activity status", 232 | user_profile_params) 233 | 234 | # Define the structure for an OrderItem in GBNF 235 | order_item_structure = { 236 | "item_id": FunctionParameter(DataType.STRING, True, "Unique identifier for the item"), 237 | "quantity": FunctionParameter(DataType.NUMBER, True, "Number of items to order") 238 | } 239 | 240 | # Define the structure for an Order in GBNF 241 | order_params = FunctionParameters({ 242 | "user_id": FunctionParameter(DataType.STRING, True, "Identifier of the user placing the order"), 243 | "items": FunctionParameter(DataType.ARRAY, True, "List of items to be included in the order", 244 | element_type=FunctionParameter(DataType.OBJECT, True, structure=order_item_structure)) 245 | }) 246 | 247 | # Create FunctionCall instance for creating an order 248 | create_order = FunctionCall("create_order", 249 | "Create a new order for a user, including a list of items and quantities", 250 | order_params) 251 | -------------------------------------------------------------------------------- /llm_function_calling/llm_function_caller.py: -------------------------------------------------------------------------------- 1 | import json 2 | import logging 3 | from typing import Callable, Dict, Any 4 | 5 | 6 | class LLMFunctionCaller: 7 | def __init__(self): 8 | self.function_map: Dict[str, Callable] = {} 9 | self.param_transformers: Dict[str, Callable[[Dict[str, Any]], Dict[str, Any]]] = {} 10 | logging.basicConfig(level=logging.INFO) 11 | 12 | def add_function(self, name: str, function: Callable) -> None: 13 | self.function_map[name] = function 14 | 15 | def add_param_transformer(self, function_name: str, 16 | transformer: Callable[[Dict[str, Any]], Dict[str, Any]]) -> None: 17 | self.param_transformers[function_name] = transformer 18 | 19 | def execute_function(self, json_input: str) -> Any: 20 | try: 21 | data: Dict[str, Any] = json.loads(json_input) 22 | func_name: str = data["function"] 23 | params: Dict[str, Any] = data["params"] 24 | 25 | if func_name not in self.function_map: 26 | raise ValueError("Function not defined") 27 | 28 | # Apply parameter transformation if available 29 | if func_name in self.param_transformers: 30 | params = self.param_transformers[func_name](params) 31 | 32 | return self.function_map[func_name](**params) 33 | 34 | except Exception as e: 35 | logging.error(f"Error executing function: {e}") 36 | raise e 37 | -------------------------------------------------------------------------------- /llm_function_calling/llm_function_calling/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Maximilian-Winter/llama_cpp_function_calling/70a3c608328a4cae7077c47914f593cedfe234cd/llm_function_calling/llm_function_calling/__init__.py -------------------------------------------------------------------------------- /llm_function_calling/llm_function_calling/function_call.py: -------------------------------------------------------------------------------- 1 | from enum import Enum 2 | from typing import Optional, Dict, List 3 | 4 | 5 | class DataType(Enum): 6 | STRING = "string" 7 | BOOLEAN = "boolean" 8 | NUMBER = "number" 9 | FLOAT = "float" 10 | OBJECT = "object" # New type for objects 11 | ARRAY = "array" 12 | ENUM = "enum" 13 | 14 | 15 | class FunctionParameter: 16 | def __init__(self, parameter_type: DataType, required: bool, 17 | description: Optional[str] = None, 18 | enum: Optional[List[str]] = None, 19 | structure: Optional[Dict[str, 'FunctionParameter']] = None, # For objects 20 | element_type: Optional['FunctionParameter'] = None, # For arrays 21 | precision: Optional[int] = None): # Precision for floating point numbers 22 | self.type = parameter_type 23 | self.required = required 24 | self.description = description 25 | self.enum = enum 26 | self.structure = structure 27 | self.element_type = element_type 28 | self.precision = precision 29 | 30 | 31 | class FunctionParameters: 32 | def __init__(self, properties: Dict[str, FunctionParameter]): 33 | self.properties = properties 34 | 35 | 36 | class FunctionCall: 37 | def __init__(self, name: str, description: str, parameters: FunctionParameters): 38 | self.name = name 39 | self.parameters = parameters 40 | self.description = description 41 | -------------------------------------------------------------------------------- /llm_function_calling/llm_function_calling/function_calling_grammar_generator.py: -------------------------------------------------------------------------------- 1 | import os 2 | from typing import List, Dict 3 | 4 | from .function_call import FunctionCall, DataType, FunctionParameter 5 | 6 | 7 | def format_function_names(function_calls): 8 | # Extract the names from each FunctionCall instance 9 | function_names = [format_rule_name(func.name) for func in function_calls] 10 | 11 | # Join the names with ' | ' as separator 12 | formatted_names = ' | '.join(function_names) 13 | 14 | return formatted_names 15 | 16 | 17 | def generate_enum_rule(enum_rule_name, enum_values): 18 | # Start the enum rule with its name 19 | enum_rule = f"{enum_rule_name} ::= " 20 | # Add each enum value 21 | for i, value in enumerate(enum_values): 22 | if i > 0: 23 | enum_rule += " | " # Add a separator for multiple values 24 | enum_rule += f'\"\\"{value}\\\""' 25 | return enum_rule 26 | 27 | 28 | def generate_object_rule(nested_rule_name, structure): 29 | object_rule = f"{format_rule_name(nested_rule_name)} ::= \"{{\" ws " 30 | for j, (sub_param_name, sub_param) in enumerate(structure.items()): 31 | if j > 0: 32 | object_rule += "\",\" ws " # Include a comma separator for multiple sub-parameters 33 | 34 | if sub_param.type in [DataType.OBJECT, DataType.ARRAY]: 35 | # Recursive call for nested OBJECT or ARRAY 36 | sub_nested_rule_name = f"{format_rule_name(nested_rule_name)}-{format_rule_name(sub_param_name)}" 37 | object_rule += f"\"\\\"{sub_param_name}\\\":\" ws {sub_nested_rule_name} " 38 | if sub_param.type == DataType.OBJECT: 39 | sub_object_rule = generate_object_rule(sub_nested_rule_name, sub_param.structure) 40 | object_rule += "\n" + sub_object_rule 41 | elif sub_param.type == DataType.ARRAY: 42 | sub_array_rule = generate_array_rule(sub_nested_rule_name, sub_param.element_type) 43 | object_rule += "\n" + sub_array_rule 44 | elif sub_param.type == DataType.FLOAT and sub_param.precision is not None: 45 | # Handling FLOAT type with precision in nested object 46 | float_rule_name = f"float-{sub_param.precision}" 47 | object_rule += f"\"\\\"{sub_param_name}\\\":\" ws {float_rule_name} " 48 | else: 49 | # Basic data types or ENUM for nested parameters 50 | formatted_type = format_data_type(sub_param.type, 51 | nested_rule_name=f"{nested_rule_name}{sub_param_name.capitalize()}", 52 | precision=sub_param.precision) 53 | object_rule += f"\"\\\"{sub_param_name}\\\":\" ws {formatted_type}" 54 | 55 | object_rule += " ws \"}\"" 56 | return object_rule 57 | 58 | 59 | def generate_array_rule(nested_rule_name, element_type): 60 | array_rule = f"{nested_rule_name} ::= \"[\" ws " 61 | if element_type.type in [DataType.OBJECT, DataType.ARRAY]: 62 | # Recursive call for nested OBJECT or ARRAY in an array 63 | array_element_rule_name = f"{nested_rule_name}-element" 64 | array_rule += f"{array_element_rule_name} (\",\" ws {array_element_rule_name})* " 65 | if element_type.type == DataType.OBJECT: 66 | array_object_rule = generate_object_rule(array_element_rule_name, element_type.structure) 67 | array_rule += "\n" + array_object_rule 68 | elif element_type.type == DataType.ARRAY: 69 | array_array_rule = generate_array_rule(array_element_rule_name, element_type.element_type) 70 | array_rule += "\n" + array_array_rule 71 | else: 72 | # Basic data types for array elements 73 | array_rule += element_type.type.value + '("," ws ' + element_type.type.value + ')*' 74 | 75 | array_rule += " ws \"]\"" 76 | return array_rule 77 | 78 | 79 | def format_enum_values(enum_values): 80 | """Format enum values for GBNF grammar.""" 81 | return ' | '.join(f'"{value}"' for value in enum_values) 82 | 83 | 84 | def collect_float_precisions(function_calls): 85 | precisions = set() 86 | for func in function_calls: 87 | for param in func.parameters.properties.values(): 88 | if param.type in [DataType.OBJECT, DataType.ARRAY]: 89 | test = collect_nested_float_precisions(param) 90 | if test is not None: 91 | precisions.add(test) 92 | if param.type == DataType.FLOAT and param.precision is not None: 93 | precisions.add(param.precision) 94 | return precisions 95 | 96 | 97 | def collect_nested_float_precisions(nested): 98 | if nested.type == DataType.OBJECT: 99 | for param in nested.structure.values(): 100 | if param.type == DataType.OBJECT: 101 | return collect_nested_float_precisions(param) 102 | if param.type == DataType.ARRAY: 103 | if param.element_type.type == DataType.OBJECT: 104 | return collect_nested_float_precisions(param) 105 | if param.element_type.type == DataType.FLOAT and param.element_type.precision is not None: 106 | return param.element_type.precision 107 | if param.type == DataType.FLOAT and param.precision is not None: 108 | return param.precision 109 | else: 110 | if nested.element_type.type == DataType.OBJECT: 111 | return collect_nested_float_precisions(nested.element_type) 112 | if nested.element_type.type == DataType.FLOAT and nested.element_type.precision is not None: 113 | return nested.element_type.precision 114 | 115 | 116 | def format_data_type(data_type, nested_rule_name=None, precision=None): 117 | """Return the formatted data type or a nested rule name for complex types.""" 118 | if data_type in [DataType.OBJECT, DataType.ARRAY]: 119 | return nested_rule_name if nested_rule_name else 'object' 120 | elif data_type == DataType.FLOAT and precision is not None: 121 | return f"float-{precision}" # Custom float rule name based on precision 122 | else: 123 | return data_type.value 124 | 125 | 126 | def generate_gbnf_float_rules(precisions): 127 | gbnf_rules = "" 128 | gbnf_rules2 = "" 129 | 130 | for precision in precisions: 131 | # Define the floating point rules with specific precision 132 | gbnf_rules += f"\nfloat-{precision} ::= integer-part \".\" fractional-part-{precision}\n" 133 | rule_part = ''.join(["[0-9]" for _ in range(precision)]) 134 | gbnf_rules2 += f"fractional-part-{precision} ::= {rule_part}\n" 135 | # Define the integer_part rule only once 136 | gbnf_rules += "\ninteger-part ::= [0-9]+\n" 137 | return gbnf_rules + gbnf_rules2 + "\n" 138 | 139 | 140 | def generate_nested_rule(nested_rule_name, param): 141 | """Generate rules for nested structures like OBJECT and ARRAY.""" 142 | if param.type == DataType.OBJECT: 143 | return generate_object_rule(nested_rule_name, param.structure) 144 | elif param.type == DataType.ARRAY: 145 | return generate_array_rule(nested_rule_name, param.element_type) 146 | 147 | 148 | def generate_parameter_rules(function_name, parameters): 149 | """Generate GBNF rules for function parameters.""" 150 | param_rules = f"{function_name}-params ::= \"{{\" ws " 151 | 152 | for i, (param_name, param) in enumerate(parameters.items()): 153 | param_rules += "\",\" ws " if i > 0 else "" # Add comma separator for multiple parameters 154 | nested_rule_name = f"{function_name}-{format_rule_name(param_name)}" 155 | param_rules += f"\"\\\"{format_rule_name(param_name)}\\\":\" ws " 156 | if param.type in [DataType.OBJECT, DataType.ARRAY]: 157 | param_rules += f"{nested_rule_name} " 158 | param_rules += "\n" + generate_nested_rule(nested_rule_name, param) 159 | elif param.enum: 160 | param_rules += f"{nested_rule_name} " 161 | param_rules += "\n" + generate_enum_rule(nested_rule_name, param.enum) 162 | else: 163 | param_rules += format_data_type(param.type) 164 | param_rules += " ws \"}\"" 165 | return param_rules 166 | 167 | 168 | def generate_gbnf_rule(function_call): 169 | """Generate a GBNF grammar rule for a given FunctionCall.""" 170 | function_name = format_rule_name(function_call.name) 171 | function_rule = f"{function_name} ::= \"{{\" ws \"\\\"function\\\":\" ws \"\\\"{function_call.name}\\\",\" ws \"\\\"params\\\":\" ws {function_name}-params \"}}\"" 172 | param_rules = generate_parameter_rules(function_name, function_call.parameters.properties) 173 | return function_rule, param_rules 174 | 175 | 176 | def generate_gbnf_grammar(function_calls: List[FunctionCall]) -> str: 177 | """ 178 | Generate a complete GBNF grammar from a list of FunctionCall instances, 179 | placing all function rules first followed by all parameter rules. 180 | """ 181 | # Start with the root rule 182 | root_rule = "root ::= " + format_function_names(function_calls) 183 | 184 | # Initialize lists to store function rules and parameter rules separately 185 | function_rules = [root_rule] 186 | param_rules = [] 187 | 188 | # Iterate over each FunctionCall and generate corresponding GBNF rules 189 | for function_call in function_calls: 190 | function_rule, param_rule = generate_gbnf_rule(function_call) 191 | 192 | # Append the generated rules to their respective lists 193 | function_rules.append(function_rule) 194 | param_rules.append(param_rule) 195 | 196 | # Collect required float precisions 197 | float_precisions = collect_float_precisions(function_calls) 198 | 199 | # Generate precision-specific float rules 200 | float_rules = generate_gbnf_float_rules(float_precisions) 201 | function_rules[0] += """ ws "}" """ 202 | # Combine all rules 203 | complete_grammar = "\n".join(function_rules + param_rules + [float_rules]) 204 | 205 | return complete_grammar 206 | 207 | 208 | def format_rule_name(name): 209 | """ 210 | Convert rule names from snake_case to kebab-case and ensure 'Params' becomes 'params'. 211 | """ 212 | # Replace underscores with dashes and convert to lowercase 213 | return name.replace('_', '-').lower() 214 | 215 | 216 | def save_grammar_to_file(grammar_str, file_path): 217 | try: 218 | script_path = os.path.abspath(__file__) 219 | script_dir = os.path.dirname(script_path) 220 | with open(f"{script_dir}/primitive.gbnf", 'r', encoding='utf-8') as file: 221 | primary_grammar = file.read() 222 | with open(file_path, 'w', encoding='utf-8') as file: 223 | file.write(grammar_str + "\n" + primary_grammar) 224 | print(f"Grammar successfully saved to {file_path}") 225 | except IOError as e: 226 | print(f"An error occurred while writing to the file: {e}") 227 | 228 | 229 | def generate_documentation(function_calls): 230 | """ 231 | Generate documentation for a list of FunctionCall instances, including details of objects and arrays. 232 | """ 233 | 234 | def document_parameter(param_name, param, indent_level): 235 | """ 236 | Recursively document a parameter, handling objects, arrays, and enums. 237 | """ 238 | indent = " " * indent_level 239 | 240 | # Handling Enum type 241 | if param.type == DataType.ENUM and param.enum: 242 | enum_values = ", ".join(param.enum[:-1]) + " or " + param.enum[-1] 243 | param_description = f"{param.description} (Valid values: {enum_values})" 244 | else: 245 | param_description = param.description 246 | 247 | doc = f"{indent}{param_name} ({param.type.value}, {'required' if param.required else 'optional'}): {param_description}\n" 248 | 249 | if param.type == DataType.OBJECT and param.structure: 250 | doc += f"{indent} Structure:\n" 251 | for sub_param_name, sub_param in param.structure.items(): 252 | doc += document_parameter(sub_param_name, sub_param, indent_level + 2) 253 | 254 | elif param.type == DataType.ARRAY and param.element_type: 255 | doc += f"{indent} Element Type:\n" 256 | doc += document_parameter("item", param.element_type, indent_level + 2) 257 | 258 | return doc 259 | 260 | documentation = "Available Functions:\n\n" 261 | for func in function_calls: 262 | # Function name and description 263 | documentation += f"{func.name}:\n Description: {func.description}\n Parameters:\n" 264 | 265 | # Parameters and their descriptions 266 | for param_name, param in func.parameters.properties.items(): 267 | documentation += document_parameter(param_name, param, 2) 268 | 269 | documentation += "\n" # Add a blank line between function calls 270 | 271 | return documentation 272 | 273 | 274 | def save_documentation_to_file(documentation, file_path): 275 | try: 276 | with open(file_path, 'w') as file: 277 | file.write(documentation) 278 | print(f"Documentation successfully saved to {file_path}") 279 | except IOError as e: 280 | print(f"An error occurred while saving the file: {e}") 281 | -------------------------------------------------------------------------------- /llm_function_calling/llm_function_calling/gpt_functions.py: -------------------------------------------------------------------------------- 1 | from enum import Enum 2 | from typing import List 3 | 4 | from .function_call import DataType 5 | from .function_calling_grammar_generator import generate_documentation, \ 6 | save_documentation_to_file, format_rule_name, generate_enum_rule, generate_object_rule, generate_array_rule, \ 7 | format_function_names, generate_gbnf_grammar 8 | from .function_calling_grammar_generator import save_grammar_to_file 9 | from .function_call import FunctionCall, FunctionParameters, FunctionParameter 10 | 11 | 12 | 13 | 14 | # SendMessage FunctionCall 15 | send_message = FunctionCall( 16 | name='send_message', 17 | description="Sends a message to the User.", 18 | parameters=FunctionParameters({ 19 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 20 | description="Your inner thoughts or inner monologue while writing the message."), 21 | "message": FunctionParameter(parameter_type=DataType.STRING, required=True, 22 | description="Message you want to send to the user.") 23 | }) 24 | ) 25 | 26 | # CoreMemoryAppend FunctionCall 27 | core_memory_append = FunctionCall( 28 | name='core_memory_append', 29 | description="Append to Core Memory.", 30 | parameters=FunctionParameters({ 31 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 32 | description="Your inner thoughts or inner monologue while writing the value."), 33 | "key": FunctionParameter(parameter_type=DataType.STRING, required=True, 34 | description="The memory key, like 'persona', 'goals', 'situation' or 'human'."), 35 | "field": FunctionParameter(parameter_type=DataType.STRING, required=True, 36 | description="The field which should be appended."), 37 | "value": FunctionParameter(parameter_type=DataType.STRING, required=True, 38 | description="The information you would like to save."), 39 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 40 | description="Set this to true to get control back after execution, to chain functions together.") 41 | }) 42 | ) 43 | 44 | # CoreMemoryReplace FunctionCall 45 | core_memory_replace = FunctionCall( 46 | name='core_memory_replace', 47 | description="Replace parts of Core Memory.", 48 | parameters=FunctionParameters({ 49 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 50 | description="Your inner thoughts or inner monologue while writing the value."), 51 | "key": FunctionParameter(parameter_type=DataType.STRING, required=True, 52 | description="The memory key, like 'persona', 'goals', 'situation' or 'human'."), 53 | "field": FunctionParameter(parameter_type=DataType.STRING, required=True, 54 | description="The field where the memory should be replaced like 'personality' or 'traits'."), 55 | "value": FunctionParameter(parameter_type=DataType.STRING, required=True, 56 | description="The information you would like to write into the field."), 57 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 58 | description="Set this to true to get control back after execution, to chain functions together.") 59 | }) 60 | ) 61 | 62 | # ArchivalMemoryInsert FunctionCall 63 | archival_memory_insert = FunctionCall( 64 | name='archival_memory_insert', 65 | description="Add memory to Archival Memory.", 66 | parameters=FunctionParameters({ 67 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 68 | description="Your inner thoughts or inner monologue while writing the value,"), 69 | "value": FunctionParameter(parameter_type=DataType.STRING, required=True, 70 | description="The content you want to add."), 71 | "memory_importance": FunctionParameter(parameter_type=DataType.FLOAT, required=True, 72 | description="The importance of the current memory considering situation and persona."), 73 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 74 | description="Set this to true to get control back after execution, to chain functions together.") 75 | }) 76 | ) 77 | 78 | # ArchivalMemorySearch FunctionCall 79 | archival_memory_search = FunctionCall( 80 | name='archival_memory_search', 81 | description="Search in Archival Memory.", 82 | parameters=FunctionParameters({ 83 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 84 | description="Your inner thoughts or inner monologue while writing the query."), 85 | "query": FunctionParameter(parameter_type=DataType.STRING, required=True, 86 | description="Your search query."), 87 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 88 | description="Set this to true to get control back after execution, to chain functions together.") 89 | }) 90 | ) 91 | 92 | # RecallMemorySearch FunctionCall 93 | recall_memory_search = FunctionCall( 94 | name='recall_memory_search', 95 | description="Search in Recall Memory.", 96 | parameters=FunctionParameters({ 97 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 98 | description="Your inner thoughts or inner monologue while writing the query."), 99 | "query": FunctionParameter(parameter_type=DataType.STRING, required=True, 100 | description="Your search query."), 101 | "page": FunctionParameter(parameter_type=DataType.NUMBER, required=True, 102 | description="The page number you want to retrieve."), 103 | "start_date": FunctionParameter(parameter_type=DataType.STRING, required=True, 104 | description="The start date for your search in '%Y-%m-%d %H:%M:%S' format."), 105 | "end_date": FunctionParameter(parameter_type=DataType.STRING, required=True, 106 | description="The end date for your search in '%Y-%m-%d %H:%M:%S' format."), 107 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 108 | description="Set this to true to get control back after execution, to chain functions together.") 109 | }) 110 | ) 111 | 112 | # CmdCommand FunctionCall 113 | cmd_command = FunctionCall( 114 | name='cmd_command', 115 | description="Execute CMD command.", 116 | parameters=FunctionParameters({ 117 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 118 | description="Your inner thoughts or inner monologue while writing the command."), 119 | "command": FunctionParameter(parameter_type=DataType.STRING, required=True, 120 | description="The CMD command to execute."), 121 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 122 | description="Set this to true to get control back after execution, to chain functions together.") 123 | }) 124 | ) 125 | 126 | # WebBrowsing FunctionCall 127 | web_browsing = FunctionCall( 128 | name='web_browsing', 129 | description="Opens a website and returns content.", 130 | parameters=FunctionParameters({ 131 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 132 | description="Your inner thoughts or inner monologue while writing the url."), 133 | "URL": FunctionParameter(parameter_type=DataType.STRING, required=True, 134 | description="The URL you want to access."), 135 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 136 | description="Set this to true to get control back after execution, to chain functions together.") 137 | }) 138 | ) 139 | 140 | # WebDownload FunctionCall 141 | web_download = FunctionCall( 142 | name='web_download', 143 | description="Downloads a file.", 144 | parameters=FunctionParameters({ 145 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 146 | description="Your inner thoughts or inner monologue while writing the url."), 147 | "URL": FunctionParameter(parameter_type=DataType.STRING, required=True, 148 | description="The URL you want to download."), 149 | "Path": FunctionParameter(parameter_type=DataType.STRING, required=True, 150 | description="The Path you want to download the file to."), 151 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 152 | description="Set this to true to get control back after execution, to chain functions together.") 153 | }) 154 | ) 155 | 156 | # ReadFile FunctionCall 157 | read_file = FunctionCall( 158 | name='read_file', 159 | description="Returns content of a file.", 160 | parameters=FunctionParameters({ 161 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 162 | description="Your inner thoughts or inner monologue while writing the file path."), 163 | "File": FunctionParameter(parameter_type=DataType.STRING, required=True, 164 | description="The path of the file you want to open."), 165 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 166 | description="Set this to true to get control back after execution, to chain functions together.") 167 | }) 168 | ) 169 | 170 | # WriteFile FunctionCall 171 | write_file = FunctionCall( 172 | name='write_file', 173 | description="Writes to a file.", 174 | parameters=FunctionParameters({ 175 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 176 | description="Your inner thoughts or inner monologue while writing the file."), 177 | "File": FunctionParameter(parameter_type=DataType.STRING, required=True, 178 | description="The path of the file you want to write."), 179 | "Content": FunctionParameter(parameter_type=DataType.STRING, required=True, 180 | description="The content of the file you want to write."), 181 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 182 | description="Set this to true to get control back after execution, to chain functions together.") 183 | }) 184 | ) 185 | 186 | # PythonInterpreterCommand FunctionCall 187 | python_interpreter_command = FunctionCall( 188 | name='python_interpreter_command', 189 | description="Execute Python command.", 190 | parameters=FunctionParameters({ 191 | "inner_thoughts": FunctionParameter(parameter_type=DataType.STRING, required=True, 192 | description="Your inner thoughts or inner monologue while writing the command."), 193 | "command": FunctionParameter(parameter_type=DataType.STRING, required=True, 194 | description="The Python command to execute."), 195 | "require_heartbeat": FunctionParameter(parameter_type=DataType.BOOLEAN, required=True, 196 | description="Set this to true to get control back after execution, to chain functions together.") 197 | }) 198 | ) 199 | 200 | 201 | def generate_gpt_functions_grammar_and_documentation(): 202 | # List of FunctionCall instances 203 | # function_calls = [send_message, core_memory_append, core_memory_replace, archival_memory_insert, 204 | # archival_memory_search, cmd_command, web_browsing, web_download, read_file, 205 | # write_file, python_interpreter_command] 206 | 207 | # Enum values for a hypothetical "role" parameter 208 | roles_enum = ["Admin", "User", "Guest"] 209 | 210 | # Nested structure for address parameter 211 | address_structure = { 212 | "street": FunctionParameter(DataType.STRING, True), 213 | "city": FunctionParameter(DataType.STRING, True), 214 | "zip_code": FunctionParameter(DataType.STRING, True) 215 | } 216 | 217 | # Parameters for the CreateUserProfile function 218 | create_user_profile_params = FunctionParameters({ 219 | "username": FunctionParameter(DataType.STRING, True), 220 | "email": FunctionParameter(DataType.STRING, True), 221 | "address": FunctionParameter(DataType.OBJECT, True, structure=address_structure), 222 | "is_active": FunctionParameter(DataType.BOOLEAN, True), 223 | "role": FunctionParameter(DataType.ENUM, True, enum=roles_enum) 224 | }) 225 | 226 | # Enum for processing options 227 | processing_options_enum = ["Standard", "Extended", "Custom"] 228 | 229 | # Nested structure for experiment instrumentation 230 | instrumentation_structure = { 231 | "type": FunctionParameter(DataType.STRING, True), 232 | "model": FunctionParameter(DataType.STRING, True) 233 | } 234 | 235 | # Nested structure for experiment details 236 | experiment_details_structure = { 237 | "title": FunctionParameter(DataType.STRING, True), 238 | "researcher": FunctionParameter(DataType.STRING, True), 239 | "date": FunctionParameter(DataType.STRING, True), 240 | "instrumentation": FunctionParameter(DataType.OBJECT, True, structure=instrumentation_structure) 241 | } 242 | 243 | # Sample measurements structure (array of floats) 244 | sample_structure = { 245 | "sampleId": FunctionParameter(DataType.STRING, True), 246 | "measurements": FunctionParameter(DataType.ARRAY, True, element_type=FunctionParameter(DataType.FLOAT, precision=5, required=True)) 247 | } 248 | 249 | # Parameters for AnalyzeScientificData function 250 | analyze_scientific_data_params = FunctionParameters({ 251 | "experiment_details": FunctionParameter(DataType.OBJECT, True, structure=experiment_details_structure), 252 | "samples": FunctionParameter(DataType.ARRAY, True, 253 | element_type=FunctionParameter(DataType.OBJECT, True, structure=sample_structure)), 254 | "processing-options": FunctionParameter(DataType.ENUM, True, enum=processing_options_enum) 255 | }) 256 | 257 | # Create the FunctionCall instance 258 | analyze_scientific_data_function_call = FunctionCall("analyze_scientific_data", 259 | "Process scientific data from experiments", 260 | analyze_scientific_data_params) 261 | 262 | # Create the FunctionCall instance 263 | create_user_profile_function_call = FunctionCall("create_user_profile", "Create a user profile", 264 | create_user_profile_params) 265 | 266 | 267 | function_calls = [send_message, core_memory_append, core_memory_replace, archival_memory_insert, 268 | archival_memory_search, recall_memory_search] 269 | function_calls = [analyze_scientific_data_function_call] 270 | # Generate the documentation 271 | doc = generate_documentation(function_calls) 272 | print(doc) 273 | save_documentation_to_file(doc, 'function_documentation.txt') 274 | 275 | save_grammar_to_file(generate_gbnf_grammar(function_calls), "gen_function_calling.gbnf") 276 | 277 | 278 | class Address: 279 | def __init__(self, street, city, zip_code): 280 | self.street = street 281 | self.city = city 282 | self.zip_code = zip_code 283 | 284 | 285 | class UserProfile: 286 | def __init__(self, username, email, address: Address, is_active): 287 | self.username = username 288 | self.email = email 289 | self.address = address 290 | self.is_active = is_active 291 | 292 | 293 | # Define an Enum for item categories 294 | class ItemCategory(Enum): 295 | ELECTRONICS = "Electronics" 296 | CLOTHING = "Clothing" 297 | FOOD = "Food" 298 | BOOKS = "Books" 299 | 300 | 301 | # Updated OrderItem class with an additional category attribute 302 | class OrderItem: 303 | def __init__(self, item_id, quantity, category: ItemCategory): 304 | self.item_id = item_id 305 | self.quantity = quantity 306 | self.category = category 307 | 308 | 309 | # Define the structure for an OrderItem in GBNF with the category enum 310 | order_item_structure = { 311 | "item_id": FunctionParameter(DataType.STRING, True, "Unique identifier for the item"), 312 | "quantity": FunctionParameter(DataType.NUMBER, True, "Number of items to order"), 313 | # Adding the category as an Enum type 314 | "category": FunctionParameter(DataType.ENUM, True, "Category of the item", 315 | enum=[e.value for e in ItemCategory]) 316 | } 317 | 318 | 319 | class Order: 320 | def __init__(self, user_id, items: [OrderItem]): 321 | self.user_id = user_id 322 | self.items = items 323 | 324 | 325 | # Define the structure for an Address in GBNF 326 | address_structure = { 327 | "street": FunctionParameter(DataType.STRING, True, "Street name and number"), 328 | "city": FunctionParameter(DataType.STRING, True, "City name"), 329 | "zip_code": FunctionParameter(DataType.STRING, True, "Postal or ZIP code") 330 | } 331 | 332 | # Define the structure for a UserProfile in GBNF 333 | user_profile_params = FunctionParameters({ 334 | "username": FunctionParameter(DataType.STRING, True, "User's unique username"), 335 | "email": FunctionParameter(DataType.STRING, True, "User's email address"), 336 | "address": FunctionParameter(DataType.OBJECT, True, "User's physical address", structure=address_structure), 337 | "is_active": FunctionParameter(DataType.BOOLEAN, True, "Indicates if the user profile is active") 338 | }) 339 | 340 | # Create FunctionCall instance for creating a user profile 341 | create_user_profile = FunctionCall("create_user_profile", 342 | "Create a new user profile with username, email, address, and activity status", 343 | user_profile_params) 344 | 345 | # Define the structure for an Order in GBNF 346 | order_params = FunctionParameters({ 347 | "user_id": FunctionParameter(DataType.STRING, True, "Identifier of the user placing the order"), 348 | "items": FunctionParameter(DataType.ARRAY, True, "List of items to be included in the order", 349 | element_type=FunctionParameter(DataType.OBJECT, True, structure=order_item_structure)) 350 | }) 351 | 352 | # Create FunctionCall instance for creating an order 353 | create_order = FunctionCall("create_order", 354 | "Create a new order for a user, including a list of items and quantities", 355 | order_params) 356 | -------------------------------------------------------------------------------- /llm_function_calling/llm_function_calling/llm_function_caller.py: -------------------------------------------------------------------------------- 1 | import json 2 | import logging 3 | from typing import Callable, Dict, Any 4 | 5 | 6 | class LLMFunctionCaller: 7 | def __init__(self): 8 | self.function_map: Dict[str, Callable] = {} 9 | self.param_transformers: Dict[str, Callable[[Dict[str, Any]], Dict[str, Any]]] = {} 10 | logging.basicConfig(level=logging.INFO) 11 | 12 | def add_function(self, name: str, function: Callable) -> None: 13 | self.function_map[name] = function 14 | 15 | def add_param_transformer(self, function_name: str, 16 | transformer: Callable[[Dict[str, Any]], Dict[str, Any]]) -> None: 17 | self.param_transformers[function_name] = transformer 18 | 19 | def execute_function(self, json_input: str) -> Any: 20 | try: 21 | data: Dict[str, Any] = json.loads(json_input) 22 | func_name: str = data["function"] 23 | params: Dict[str, Any] = data["params"] 24 | 25 | if func_name not in self.function_map: 26 | raise ValueError("Function not defined") 27 | 28 | # Apply parameter transformation if available 29 | if func_name in self.param_transformers: 30 | params = self.param_transformers[func_name](params) 31 | 32 | return self.function_map[func_name](**params) 33 | 34 | except Exception as e: 35 | logging.error(f"Error executing function: {e}") 36 | raise e 37 | -------------------------------------------------------------------------------- /llm_function_calling/llm_function_calling/primitive.gbnf: -------------------------------------------------------------------------------- 1 | string ::= "\"" ([^"\[\]{}]*) "\"" 2 | boolean ::= "true" | "false" 3 | ws ::= "" 4 | number ::= [0-9]+ 5 | float ::= number "." number exponent? 6 | exponent ::= ("e" | "E") ("+" | "-")? number 7 | object ::= "{" ws (keyValuePair ("," ws keyValuePair)*)? ws "}" 8 | array ::= "[" ws (value ("," ws value)*)? ws "]" 9 | keyValuePair ::= string ":" ws value 10 | value ::= string | number | float | boolean | object | array -------------------------------------------------------------------------------- /llm_function_calling/primitive.gbnf: -------------------------------------------------------------------------------- 1 | string ::= "\"" ([^"\[\]{}]*) "\"" 2 | boolean ::= "true" | "false" 3 | ws ::= "" 4 | number ::= [0-9]+ 5 | float ::= number "." number exponent? 6 | exponent ::= ("e" | "E") ("+" | "-")? number 7 | object ::= "{" ws (keyValuePair ("," ws keyValuePair)*)? ws "}" 8 | array ::= "[" ws (value ("," ws value)*)? ws "]" 9 | keyValuePair ::= string ":" ws value 10 | value ::= string | number | float | boolean | object | array --------------------------------------------------------------------------------