├── .env.sample ├── .github └── workflows │ ├── publish.yml │ └── test.yml ├── .gitignore ├── README.md ├── autocoder ├── __init__.py ├── helpers │ ├── __init__.py │ ├── code.py │ ├── context.py │ └── files.py ├── main.py ├── steps │ ├── __init__.py │ ├── act.py │ └── reason.py └── tests │ ├── __init__.py │ ├── e2e │ ├── __init__.py │ ├── test_create_hello_world.py │ ├── test_hello_world_bad_test.py │ ├── test_hello_world_broken.py │ ├── test_hello_world_finished.py │ ├── test_hello_world_good_test.py │ └── test_hello_world_replace.py │ ├── helpers │ ├── __init__.py │ ├── code.py │ ├── context.py │ └── files.py │ └── steps │ ├── __init__.py │ ├── act.py │ └── reason.py ├── docker-compose.yml ├── dockerfile ├── requirements.txt ├── resources ├── image.jpg └── youcreatethefuture.jpg ├── setup.py ├── start.py └── test.py /.env.sample: -------------------------------------------------------------------------------- 1 | # Update and save as .env if using docker 2 | OPENAI_API_KEY=Your-API-Key 3 | 4 | -------------------------------------------------------------------------------- /.github/workflows/publish.yml: -------------------------------------------------------------------------------- 1 | # This workflow will upload a Python Package using Twine when a release is created 2 | # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries 3 | 4 | # This workflow uses actions that are not certified by GitHub. 5 | # They are provided by a third-party and are governed by 6 | # separate terms of service, privacy policy, and support 7 | # documentation. 8 | 9 | name: Upload Python Package 10 | 11 | on: 12 | release: 13 | types: [published] 14 | 15 | permissions: 16 | contents: read 17 | 18 | jobs: 19 | deploy: 20 | 21 | runs-on: ubuntu-latest 22 | 23 | steps: 24 | - uses: actions/checkout@v3 25 | - name: Set up Python 26 | uses: actions/setup-python@v3 27 | with: 28 | python-version: '3.x' 29 | - name: Install dependencies 30 | run: | 31 | python -m pip install --upgrade pip 32 | pip install build 33 | - name: Build package 34 | run: python -m build 35 | - name: Publish package 36 | uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29 37 | with: 38 | user: ${{ secrets.pypi_username }} 39 | password: ${{ secrets.pypi_password }} 40 | -------------------------------------------------------------------------------- /.github/workflows/test.yml: -------------------------------------------------------------------------------- 1 | name: Lint and Test 2 | 3 | on: [push] 4 | 5 | jobs: 6 | build: 7 | runs-on: ubuntu-latest 8 | strategy: 9 | matrix: 10 | python-version: ["3.10"] 11 | steps: 12 | - uses: actions/checkout@v3 13 | - name: Set up Python ${{ matrix.python-version }} 14 | uses: actions/setup-python@v3 15 | with: 16 | python-version: ${{ matrix.python-version }} 17 | - name: Install dependencies 18 | run: | 19 | python -m pip install --upgrade pip 20 | pip install pytest 21 | pip install -r requirements.txt 22 | - name: Running tests 23 | run: | 24 | pytest test.py 25 | env: 26 | OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} 27 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | .env 3 | .vscode 4 | *.wav 5 | *.mp3 6 | *.sf2 7 | backups/ 8 | logs/ 9 | project_data/ 10 | .DS_Store 11 | memory/ 12 | projects/ 13 | .project_cache/ 14 | .pytest_cache/ 15 | .preferences 16 | *.mid 17 | *.midi 18 | build/ 19 | dist/ 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # autocoder 2 | 3 | Code that basically writes itself. 4 | 5 | 6 | 7 | # Quickstart 8 | 9 | To run with a prompt 10 | 11 | ``` 12 | python start.py 13 | ``` 14 | 15 | To use autocoder inside other projects and agents 16 | 17 | ```python 18 | from autocoder import autocoder 19 | data = { 20 | "project_name": "random_midi_generator", # name of the project 21 | "goal": "Generate a 2 minute midi track with multiple instruments. The track must contain at least 500 notes, but can contain any number of notes. The track should be polyphonic and have multiple simultaneous instruments", # goal of the project 22 | "project_dir": "random_midi_generator", # name of the project directory 23 | "log_level": "normal", # normal, debug, or quiet 24 | "step": False, # whether to step through the loop manually, False by default 25 | "model": "gpt-3.5-turbo", # default 26 | "api_key": # can also be passed in via env var OPENAI_API_KEY 27 | } 28 | 29 | autocoder(project_data) 30 | ``` 31 | 32 | # Core Concepts 33 | 34 | Autocoder is a ReAct-style python coding agent. It is designed to be run standalone with a CLI or programmatically by other agents. 35 | 36 | More information on ReAct (Reasoning and Acting) agents can be found here. 37 | 38 | ## Loop 39 | 40 | Autocoder works by looping between a "reason"" and "act" step until the project is validated and tested. The loop runs forever but you can enable "step" mode in options to step through the loop manually. 41 | 42 | ## Actions 43 | 44 | Autocoder uses OpenAI function calling to select and call what we call _actions_. Actions are functions that take in a context object and return a context object. They are called during the "act" step of the loop. 45 | 46 | ## Context 47 | 48 | Over the course of the loop, a context object gets built up which contains all of the data from the previous steps. This can be injeced into prompt templates using the `compose_prompt` function. 49 | 50 | Here is the data that is available in the context object at each step: 51 | 52 | ```python 53 | # Initial context object 54 | context = { 55 | epoch, 56 | name, 57 | goal, 58 | project_dir, 59 | project_path 60 | } 61 | 62 | # Added by reasoning step 63 | context = { 64 | reasoning, 65 | file_count, 66 | filetree, 67 | filetree_formatted, 68 | python_files, 69 | main_success, 70 | main_error, 71 | backup, 72 | project_code: [{ 73 | rel_path, 74 | file_path, 75 | content, 76 | validation_success, 77 | validation_error, 78 | test_success, 79 | test_error, 80 | }], 81 | project_code_formatted 82 | } 83 | 84 | # Action step context 85 | context = { 86 | name, # project name 87 | goal, # project goal 88 | project_dir, 89 | project_path, 90 | file_count, 91 | filetree, 92 | filetree_formatted, # formatted for prompt template 93 | python_files, 94 | main_success, # included in project_code_formatted 95 | main_error, # included in project_code_formatted 96 | backup, 97 | available_actions, # list of available actions 98 | available_action_names, # list of just the action names 99 | project_code_formatted, # formatted for prompt template 100 | action_name, 101 | reasoning, # formatted for prompt template 102 | project_code: [{ 103 | rel_path, 104 | file_path, 105 | content, 106 | validation_success, 107 | validation_error, 108 | test_success, 109 | test_error, 110 | }] 111 | } 112 | ``` 113 | 114 | 115 | -------------------------------------------------------------------------------- /autocoder/__init__.py: -------------------------------------------------------------------------------- 1 | from .main import * -------------------------------------------------------------------------------- /autocoder/helpers/__init__.py: -------------------------------------------------------------------------------- 1 | from .code import * 2 | from .context import * 3 | from .files import * -------------------------------------------------------------------------------- /autocoder/helpers/code.py: -------------------------------------------------------------------------------- 1 | import os 2 | import subprocess 3 | import ast 4 | import astor 5 | import sys 6 | import black 7 | from importlib_metadata import distributions 8 | from agentlogger import log 9 | 10 | 11 | def is_runnable(filename): 12 | try: 13 | result = subprocess.run( 14 | ["python", "-m", "py_compile", filename], 15 | stdout=subprocess.PIPE, 16 | stderr=subprocess.PIPE, 17 | universal_newlines=True, 18 | ) 19 | 20 | if result.stderr and result.stderr != "": 21 | return False 22 | 23 | if result.returncode != 0: 24 | return False 25 | except Exception as e: 26 | log(f"An error occurred: {e}", title="is_runnable", type="error") 27 | return False 28 | 29 | return True 30 | 31 | 32 | def contains_function_definition(code): 33 | """Checks if a string of Python code contains any function definitions.""" 34 | 35 | def visit(node): 36 | """Recursively visit nodes in the AST.""" 37 | if isinstance(node, ast.FunctionDef): 38 | # If we're at a function definition node, return True 39 | return True 40 | else: 41 | # For all other nodes, recursively visit their children and return True if any of them return True 42 | for child in ast.iter_child_nodes(node): 43 | if visit(child): 44 | return True 45 | # If no function definition was found, return False 46 | return False 47 | 48 | try: 49 | tree = ast.parse(code) 50 | return visit(tree) 51 | except SyntaxError: 52 | return False 53 | 54 | 55 | def has_functions_called(code): 56 | """Checks if a string is valid Python code and if it performs a function call at root level.""" 57 | 58 | def visit(node, inside_function_def=False): 59 | """Recursively visit nodes in the AST.""" 60 | if isinstance(node, ast.FunctionDef): 61 | # If we're at a function definition node, recursively visit its children without updating the result 62 | for child in ast.iter_child_nodes(node): 63 | visit(child, inside_function_def=True) 64 | elif isinstance(node, ast.Call) and not inside_function_def: 65 | # If we're at a call node and it's not inside a function definition, return True 66 | return True 67 | else: 68 | # For all other nodes, recursively visit their children and return True if any of them return True 69 | for child in ast.iter_child_nodes(node): 70 | if visit(child, inside_function_def): 71 | return True 72 | # If no function call at the root level was found, return False 73 | return False 74 | 75 | try: 76 | tree = ast.parse(code) 77 | return visit(tree) 78 | except SyntaxError: 79 | return False 80 | 81 | 82 | def file_exists(filename): 83 | if subprocess.call(["test", "-f", filename]) == 0: 84 | return True 85 | else: 86 | return False 87 | 88 | 89 | def count_lines(code, exclude_comments=True, exclude_empty_lines=True): 90 | lines = code.split("\n") 91 | if exclude_comments: 92 | lines = [line for line in lines if not line.startswith("#")] 93 | if exclude_empty_lines: 94 | lines = [line for line in lines if line.strip() != ""] 95 | return len(lines) 96 | 97 | 98 | def validate_file(filename): 99 | if not is_runnable(filename): 100 | return { 101 | "success": False, 102 | "error": "The file is not runnable, or didn't compile.", 103 | } 104 | return validate_code(open(filename, "r").read()) 105 | 106 | 107 | def validate_code(code): 108 | if count_lines(code) == 0: 109 | return { 110 | "success": False, 111 | "error": "The file doesn't have any code in it.", 112 | } 113 | if has_functions_called(code) is False and contains_function_definition is False: 114 | return { 115 | "success": False, 116 | "error": "The file doesn't call any functions or have any functions. Please make sure the code meets the specifications and goals.", 117 | } 118 | if count_lines(code) == 1 and len(code) > 50: 119 | return { 120 | "success": False, 121 | "error": "The file has more than 50 characters but only one line, probably one massive comment or something.", 122 | } 123 | 124 | # if count_lines(code) < 4: 125 | # return { 126 | # "success": False, 127 | # "error": "The file is not long enough to do much.", 128 | # } 129 | 130 | # if "import" not in code: 131 | # return { 132 | # "success": False, 133 | # "error": "The file doesn't have any imports. Imports are needed to do anything useful. Please add some imports to the top of the file.", 134 | # } 135 | 136 | if "def" not in code: 137 | return { 138 | "success": False, 139 | "error": "The file doesn't have any functions. Please encapsulate all code inside functions.", 140 | } 141 | 142 | if "TODO" in code: 143 | return { 144 | "success": False, 145 | "error": "The file has a TODO in it. Please replace the TODO with real code or remove it.", 146 | } 147 | 148 | if "..." in code: 149 | return { 150 | "success": False, 151 | "error": "The file has a '...' in it. This indicates that it is not a complete file. Please respond with the complete script and do not omit any functions, code, tests or sections. Your response should include all code, including imports, and tests, not just changes to code.", 152 | } 153 | 154 | return {"success": True, "error": None} 155 | 156 | 157 | def save_code(code, filename): 158 | try: 159 | code = format_code(code) 160 | code = organize_imports(code) 161 | except Exception as e: 162 | log(f"Code could not be formatted: {e}", title="save_code", type="warning") 163 | with open(filename, "w") as f: 164 | f.write(code) 165 | 166 | 167 | def organize_imports(code): 168 | tree = ast.parse(code) 169 | 170 | imports = [] 171 | other_lines = [] 172 | 173 | for node in tree.body: 174 | if isinstance(node, (ast.Import, ast.ImportFrom)): 175 | imports.append(node) 176 | else: 177 | other_lines.append(node) 178 | 179 | # Deduplicate and sort 180 | seen_imports = set() 181 | unique_sorted_imports = [] 182 | for node in sorted(imports, key=lambda node: (node.__class__.__name__, astor.to_source(node).strip())): 183 | import_line = astor.to_source(node).strip() 184 | if import_line not in seen_imports: 185 | seen_imports.add(import_line) 186 | unique_sorted_imports.append(node) 187 | 188 | # Reconstruct tree 189 | tree.body = unique_sorted_imports + other_lines 190 | 191 | return astor.to_source(tree) 192 | 193 | def format_code(code: str, line_length: int = 110) -> str: 194 | """Format python code string with black. 195 | 196 | Parameters: 197 | code (str): The python code to be formatted. 198 | line_length (int): Maximum line length. Default is 88. 199 | 200 | Returns: 201 | str: The formatted python code. 202 | """ 203 | return black.format_str(code, mode=black.FileMode(line_length=line_length)) 204 | 205 | 206 | def run_code(filename): 207 | process = subprocess.Popen( 208 | ["python", filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE 209 | ) 210 | output, error = process.communicate() 211 | output = output.decode("utf-8") 212 | error = error.decode("utf-8") 213 | if error == "": 214 | error = None 215 | success = process.returncode == 0 and error == None 216 | return {"success": success, "error": error, "output": output} 217 | 218 | 219 | def run_code_tests(script_path): 220 | """Run pytest on a given Python file.""" 221 | 222 | # Run the command and get the output 223 | result = subprocess.run( 224 | ["python", "-m", "pytest", script_path], capture_output=True, text=True 225 | ) 226 | if result.stderr == "" or result.stderr is None: 227 | result.stderr = False 228 | # Return the exit code. The exit code is 0 if the tests pass. 229 | return { 230 | "success": result.returncode == 0, 231 | "output": result.stdout, 232 | "error": result.stderr, 233 | } 234 | 235 | def extract_imports(code, directory): 236 | try: 237 | tree = ast.parse(code) 238 | except SyntaxError: 239 | log("Syntax error", title="extract_imports", type="warning", ) 240 | return set() 241 | 242 | all_imports = [] 243 | 244 | # List of built-in modules to be ignored 245 | builtin_modules = list(sys.builtin_module_names) + common_std_libs 246 | 247 | for node in ast.walk(tree): 248 | if isinstance(node, ast.Import): 249 | for alias in node.names: 250 | package = alias.name.split(".")[0] # only keep top-level package 251 | if package not in builtin_modules and not os.path.exists( 252 | os.path.join(directory, f"{package}.py") 253 | ): 254 | all_imports.append(package) 255 | elif isinstance(node, ast.ImportFrom): 256 | if node.level > 0: # Skip relative imports 257 | continue 258 | if node.module is not None: 259 | package = node.module.split(".")[0] # only keep top-level package 260 | if package not in builtin_modules and not os.path.exists( 261 | os.path.join(directory, f"{package}.py") 262 | ): 263 | all_imports.append(package) 264 | return set(all_imports) 265 | 266 | common_std_libs = [ 267 | "os", 268 | "sys", 269 | "math", 270 | "datetime", 271 | "json", 272 | "random", 273 | "re", 274 | "collections", 275 | "itertools", 276 | "functools", 277 | "pickle", 278 | "subprocess", 279 | "multiprocessing", 280 | "threading", 281 | "log_level", 282 | "argparse", 283 | "shutil", 284 | "glob", 285 | "time", 286 | "pathlib", 287 | "socket", 288 | "asyncio", 289 | "csv", 290 | "urllib", 291 | "heapq", 292 | "hashlib", 293 | "base64", 294 | "timeit", 295 | "contextlib", 296 | "io" 297 | ] -------------------------------------------------------------------------------- /autocoder/helpers/context.py: -------------------------------------------------------------------------------- 1 | import subprocess 2 | from pathlib import Path 3 | import pkg_resources 4 | import sys 5 | import ast 6 | 7 | from autocoder.helpers.files import ( 8 | count_files, 9 | file_tree_to_dict, 10 | file_tree_to_string, 11 | get_python_files, 12 | zip_python_files, 13 | ) 14 | from autocoder.helpers.code import extract_imports, file_exists, run_code, run_code_tests, validate_file 15 | from agentlogger import log 16 | 17 | 18 | def get_file_count(context): 19 | project_dir = context["project_dir"] 20 | context["file_count"] = count_files(project_dir) 21 | return context 22 | 23 | 24 | def read_and_format_code(context): 25 | # Read the contents of all python files and create a list of dicts 26 | project_files_str = "Project Files:" 27 | 28 | main_success = context.get("main_success", None) 29 | main_error = context.get("main_error", None) 30 | 31 | for project_dict in context["project_code"]: 32 | rel_path = project_dict["relative_path"] 33 | absolute_path = project_dict["absolute_path"] 34 | content = project_dict["content"] 35 | validation_success = project_dict["validation_success"] 36 | validation_error = project_dict["validation_error"] 37 | test_success = project_dict.get("test_success", None) 38 | test_error = project_dict.get("test_error", None) 39 | 40 | # adding file content to the string with line numbers 41 | project_files_str += "\n================================================================================\n" 42 | project_files_str += "Path (Relative): {}\nPath (Absolute): {}\n".format( 43 | str(rel_path), absolute_path 44 | ) 45 | if "main.py" in str(rel_path): 46 | project_files_str += "(Project Entrypoint)\n" 47 | project_files_str += "Run Success: {}\n".format(main_success) 48 | if main_success is False: 49 | project_files_str += "Run Error: {}\n".format(main_error) 50 | project_files_str += "Validated: {}\n".format(validation_success) 51 | if validation_success is False: 52 | project_files_str += "Validation Error: {}\n".format(validation_error) 53 | if test_success is not None: 54 | project_files_str += "Tests Passed: {}\n".format(test_success) 55 | # if test_success is False: 56 | # project_files_str += "Pytest Error: {}\n".format(test_error) 57 | project_files_str += "\nLine # ------------------------------ CODE -------------------------------------\n" 58 | for i, line in enumerate(content.split('\n')): 59 | project_files_str += "[{}] {}\n".format(i + 1, line) 60 | project_files_str += "\n================================================================================\n" 61 | 62 | context["project_code_formatted"] = project_files_str 63 | return context 64 | 65 | 66 | def collect_files(context): 67 | project_dir = context["project_dir"] 68 | # create a file tree dict of all files 69 | context["filetree"] = file_tree_to_dict(project_dir) 70 | 71 | # format file tree to string 72 | context["filetree_formatted"] = file_tree_to_string(project_dir) 73 | 74 | # Create an array of paths to all python files 75 | context["python_files"] = get_python_files(project_dir) 76 | 77 | project_code = [] 78 | 79 | for file_path in context["python_files"]: 80 | with open(file_path, "r") as file: 81 | content = file.read() 82 | rel_path = Path(file_path).relative_to(project_dir) 83 | 84 | file_dict = { 85 | "relative_path": str(rel_path), 86 | "absolute_path": file_path, 87 | "content": content, 88 | } 89 | project_code.append(file_dict) 90 | context["project_code"] = project_code 91 | 92 | return context 93 | 94 | 95 | def validate_files(context): 96 | project_code = context["project_code"] 97 | project_validated = True 98 | for file_dict in project_code: 99 | file_path = file_dict["absolute_path"] 100 | validation = validate_file(file_path) 101 | file_dict["validation_success"] = validation["success"] 102 | file_dict["validation_error"] = validation["error"] 103 | if validation["success"] is False: 104 | project_validated = False 105 | context["project_code"] = project_code 106 | context["project_validated"] = project_validated 107 | return context 108 | 109 | 110 | def run_tests(context): 111 | print('***** RUNNING TESTS') 112 | # get python files which don't contain test in their name 113 | 114 | # if not, error 115 | # call pytest on each file 116 | # no tests? error 117 | # tests failed? error 118 | # tests passed? success 119 | 120 | project_code = context["project_code"] 121 | 122 | project_code_notests = [] 123 | project_code_tests = [] 124 | project_tested = True 125 | # get python_files which also have test in the name 126 | for file_dict in project_code: 127 | file_path = file_dict["absolute_path"] 128 | if "test" in file_path: 129 | project_code_tests.append(file_dict) 130 | else: 131 | project_code_notests.append(file_dict) 132 | 133 | print('****** TESTS') 134 | for file_dict in project_code_tests: 135 | file_path = file_dict["absolute_path"] 136 | test = run_code_tests(file_path) 137 | print(test) 138 | if test["success"] is False: 139 | project_tested = False 140 | file_dict["test_success"] = test["success"] 141 | file_dict["test_error"] = test["output"] 142 | context["project_tested"] = project_tested 143 | context["project_code"] = project_code_notests + project_code_tests 144 | return context 145 | 146 | 147 | def run_main(context): 148 | project_code = context["project_code"] 149 | # get entry from project code where the relative path includes main.py 150 | main_file = None 151 | for file_dict in project_code: 152 | if "main.py" in file_dict["relative_path"]: 153 | file_dict["test_success"] = None 154 | main_file = file_dict 155 | 156 | if main_file is None: 157 | return context 158 | 159 | result = run_code(main_file["absolute_path"]) 160 | 161 | context["main_success"] = result["success"] 162 | if result["success"] is False: 163 | context["main_error"] = result["error"] 164 | else: 165 | context["main_error"] = None 166 | context["main_output"] = result["output"] 167 | 168 | return context 169 | 170 | 171 | def collect_errors(context): 172 | # get all errors from project_code 173 | project_code = context["project_code"] 174 | project_errors = [] 175 | for file_dict in project_code: 176 | if file_dict.get("validation_success") is False: 177 | project_errors.append(str(file_dict["relative_path"]) + ": " + str(file_dict["validation_error"])) 178 | if file_dict.get("test_success") is False: 179 | project_errors.append(str(file_dict["relative_path"]) + ": " + str(file_dict["test_error"])) 180 | # add main_error 181 | if context.get("main_success") is False: 182 | project_errors.append(str(context["main_error"])) 183 | context["errors"] = project_errors 184 | 185 | # format errors 186 | error_str = "" 187 | for error in project_errors: 188 | error_str += f"{error}\n" 189 | if error_str != "": 190 | context["errors_formatted"] = error_str 191 | else: 192 | context["errors_formatted"] = "" 193 | return context 194 | 195 | 196 | def backup_project(context): 197 | project_dir = context["project_dir"] 198 | project_name = context["project_name"] 199 | context["backup"] = zip_python_files(project_dir, project_name) 200 | return context 201 | 202 | 203 | def handle_packages(context): 204 | debug = context.get("log_level", "normal") == "debug" 205 | should_log = context.get("log_level", "normal") != "quiet" 206 | 207 | # Get the set of standard library modules 208 | std_module_set = set(sys.builtin_module_names) 209 | 210 | packages = [] 211 | 212 | # get the content from every entry in project_code 213 | project_code = context["project_code"] 214 | project_dir = context["project_dir"] 215 | 216 | for file_dict in project_code: 217 | try: 218 | ast.parse(file_dict["content"]) 219 | except SyntaxError: 220 | log("Couldn't parse file to extract imports", title="extract_imports", type="warning", log=debug) 221 | imports = list(extract_imports(file_dict["content"], file_dict["absolute_path"])) 222 | if len(imports) > 0: 223 | packages = list(packages) + imports 224 | 225 | # Get a list of currently installed packages 226 | installed = {pkg.key for pkg in pkg_resources.working_set} 227 | installed_packages = [] 228 | 229 | # Loop through the packages 230 | for package in packages: 231 | # Check if package is installed and it is not a built-in package 232 | if package not in installed and package not in std_module_set: 233 | # Install missing package 234 | try: 235 | result = subprocess.run( 236 | ["python", "-m", "pip", "install", package], 237 | stdout=subprocess.PIPE, 238 | stderr=subprocess.PIPE, 239 | text=True, 240 | check=True, 241 | ) 242 | installed_packages.append(package) 243 | 244 | except subprocess.CalledProcessError as e: 245 | # Check if the error message contains the expected error 246 | if ( 247 | "Could not find a version that satisfies the requirement" 248 | in e.stderr 249 | ): 250 | # Extract the package name from the error message 251 | error_package = e.stderr.split(" ")[10] 252 | packages.remove(error_package) 253 | 254 | if len(installed_packages) > 0: 255 | log( 256 | f"Installing packages: {packages}", 257 | title="packages", 258 | type="system", 259 | log=should_log, 260 | ) 261 | 262 | # for each package in packages, add to project_dir/requirements.txt 263 | # with open(f"{project_dir}/requirements.txt", "w") as f: 264 | # # Only add to requirements.txt if it's not a built-in package 265 | # f.write("\n".join([p for p in packages if p not in std_module_set])) 266 | 267 | return context 268 | -------------------------------------------------------------------------------- /autocoder/helpers/files.py: -------------------------------------------------------------------------------- 1 | import os 2 | import time 3 | import zipfile 4 | from pathlib import Path 5 | import fnmatch 6 | from agentlogger import log 7 | 8 | 9 | def count_files(dir): 10 | # check if the dir exists 11 | if not os.path.exists(dir): 12 | log("Directory does not exist.", title="count_files", type="error") 13 | return 0 14 | count = len( 15 | [name for name in os.listdir(dir) if os.path.isfile(os.path.join(dir, name))] 16 | ) 17 | return count 18 | 19 | 20 | def get_full_path(filepath, project_dir): 21 | """ 22 | Takes a filepath and a project directory, and constructs a full path based on the inputs. 23 | Returns the absolute path to the file. 24 | 25 | :param filepath: The input filepath, which can include directories and a filename. 26 | :type filepath: str 27 | :param project_dir: The directory where the project files are located. 28 | :type project_dir: str 29 | :return: The absolute path to the file, given the filename. 30 | :rtype: str 31 | """ 32 | filename = os.path.basename(filepath) 33 | directory_path = os.path.dirname(filepath) 34 | 35 | if not directory_path: 36 | directory_path = "." 37 | 38 | # Calculate the common path between project_dir and directory_path 39 | common_path = os.path.commonpath([project_dir, directory_path]) 40 | 41 | # Remove the common_path from directory_path and join it with project_dir 42 | sub_path = directory_path.replace(common_path, "").lstrip(os.path.sep) 43 | full_path = os.path.join(project_dir, sub_path) 44 | 45 | if not os.path.exists(full_path): 46 | os.makedirs(full_path) 47 | 48 | full_path = os.path.join(full_path, filename) 49 | full_path = os.path.abspath(full_path) 50 | return full_path 51 | 52 | 53 | def file_tree_to_dict(startpath): 54 | file_tree = {} 55 | for root, dirs, files in os.walk(startpath): 56 | rel_path = os.path.relpath(root, startpath) 57 | parent_dict = file_tree 58 | 59 | # Skip the root directory 60 | if rel_path == ".": 61 | rel_dirs = [] 62 | else: 63 | rel_dirs = rel_path.split(os.sep) 64 | 65 | for d in rel_dirs[:-1]: # traverse down to the current directory 66 | parent_dict = parent_dict.setdefault(d, {}) 67 | 68 | if rel_dirs: # if not at the root directory 69 | parent_dict[rel_dirs[-1]] = dict.fromkeys(files, None) 70 | else: # at the root directory 71 | parent_dict.update(dict.fromkeys(files, None)) 72 | 73 | return file_tree 74 | 75 | 76 | def file_tree_to_string(startpath): 77 | tree_string = "" 78 | for root, dirs, files in os.walk(startpath): 79 | level = root.replace(startpath, "").count(os.sep) 80 | indent = " " * 4 * (level) 81 | tree_string += "{}{}/\n".format(indent, os.path.basename(root)) 82 | subindent = " " * 4 * (level + 1) 83 | for f in files: 84 | tree_string += "{}{}\n".format(subindent, f) 85 | return tree_string 86 | 87 | 88 | def get_python_files(startpath): 89 | py_files = [] 90 | for root, dirnames, filenames in os.walk(startpath): 91 | for filename in fnmatch.filter(filenames, "*.py"): 92 | py_files.append(os.path.join(root, filename)) 93 | return py_files 94 | 95 | 96 | def zip_python_files(project_dir, project_name): 97 | # Check if ./.project_cache exists, create if not 98 | cache_dir = Path("./.project_cache") 99 | cache_dir.mkdir(exist_ok=True) 100 | 101 | # Check if "./.project_cache/{context["project_name"]}" dir exists, and create if not 102 | project_cache_dir = cache_dir / project_name 103 | project_cache_dir.mkdir(exist_ok=True) 104 | 105 | # create a timestamp 106 | epoch = int(time.time()) 107 | 108 | # Create zip file path 109 | zip_file_path = f"{project_cache_dir}/{project_name}_{str(epoch)}.zip" 110 | 111 | # Create a zip file 112 | with zipfile.ZipFile(zip_file_path, "w", zipfile.ZIP_DEFLATED) as zipf: 113 | for root, dirs, files in os.walk(project_dir): 114 | for file in files: 115 | if file.endswith(".py"): 116 | # Get absolute path of the file 117 | abs_file_path = os.path.join(root, file) 118 | # Get relative path for storing in the zip 119 | rel_file_path = os.path.relpath(abs_file_path, project_dir) 120 | zipf.write(abs_file_path, arcname=rel_file_path) 121 | 122 | return zip_file_path 123 | -------------------------------------------------------------------------------- /autocoder/main.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | 4 | from dotenv import load_dotenv 5 | 6 | from agentloop import start, step_with_input_key 7 | 8 | from autocoder.steps import reason 9 | from autocoder.steps import act 10 | from agentlogger import log, print_header 11 | 12 | # Suppress warning 13 | os.environ["TOKENIZERS_PARALLELISM"] = "False" 14 | 15 | load_dotenv() # take environment variables from .env. 16 | 17 | 18 | def autocoder(project_data): 19 | """ 20 | Main entrypoint for autocoder. Usually called from the CLI. 21 | """ 22 | 23 | if project_data["log_level"] != "quiet": 24 | print_header(text="autocoder", color="yellow", font="slant") 25 | log("Initializing...", title="autocoder", type="system", panel=False) 26 | 27 | 28 | project_data["project_dir"] = f"./project_data/{project_data['project_name']}" 29 | 30 | # check if project_dir exists and create it if it doesn't 31 | if not os.path.exists(project_data["project_dir"]): 32 | os.makedirs(project_data["project_dir"]) 33 | 34 | def initialize(context): 35 | if context is None: 36 | # Should only run on the first run 37 | context = project_data 38 | context["running"] = True 39 | return context 40 | 41 | loop_dict = start([initialize, reason, act], paused=project_data["step"]) 42 | if project_data["step"]: 43 | step_with_input_key(loop_dict) 44 | 45 | return loop_dict 46 | -------------------------------------------------------------------------------- /autocoder/steps/__init__.py: -------------------------------------------------------------------------------- 1 | from .reason import step as reason 2 | from .act import step as act 3 | 4 | __all__ = ["reason", "act"] -------------------------------------------------------------------------------- /autocoder/steps/act.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | from easycompletion import compose_function, compose_prompt, openai_function_call 4 | from autocoder.helpers.code import save_code 5 | from autocoder.helpers.context import handle_packages 6 | 7 | from autocoder.helpers.files import get_full_path 8 | from agentlogger import log 9 | 10 | create_prompt = """Task: Create a Python module that meets the stated goals, along with a set of tests for that module. 11 | 12 | This is my project goal: 13 | {{goal}} 14 | 15 | Write the main.py with code and tests to complete this goal. You should include the following details: 16 | - Reasoning: Explain your approach 17 | - Code: The full code for main.py, including all imports and code 18 | - There should be a main function which is called if __name__ == '__main__' (which should be located at the bottom of the script) 19 | - Use a functional style, avoid classes unless necessary 20 | - Test: The code for main_test.py, including all imports and functions. Tests should use a functional style with the assert keyword and run with pytest 21 | - All tests should be in their own functions and have setup and teardown so that they are isolated from each other 22 | - There should be multiple tests for each function, including tests for edge cases, different argument cases and failure cases 23 | - Do not use fixtures or anything else that is not a standard part of pytest""" 24 | 25 | edit_prompt = """{{available_actions}} 26 | Notes: 27 | - Use a functional style for code and tests 28 | - Do not include line numbers [#] at the beginning of the lines in your response 29 | - Include the correct tabs at the beginning of lines in your response 30 | - main.py must contain a main function which is called if __name__ == '__main__' (at the bottom of the file) 31 | - Replace broken code. If my code is really broken, write the code again in its entirety 32 | - For something small like an import, just insert the import 33 | - When rewriting the code, include the complete script, including all imports and code 34 | - Do NOT shorten or abbreviate anything, DO NOT use "..." or "# Rest of the code" - ALWAYS put the complete code in your response 35 | 36 | 37 | 38 | This is my project goal: 39 | {{goal}} 40 | 41 | {{reasoning}} 42 | {{project_code_formatted}} 43 | {{errors_formatted}} 44 | {{available_action_names}} 45 | 46 | Task: 47 | - First reason out loud about what you are going to do 48 | - Based on your reasoning, choose a function by name 49 | - Then choose which file to edit. You can also create or remove a file if necessary 50 | - Respond with the function, code, reasoning and necessary inputs to call the function 51 | - 52 | 53 | """ 54 | 55 | create_function = compose_function( 56 | name="create", 57 | description="Create a new script called main.py, as well as a test for main.py named main_test.py.", 58 | properties={ 59 | "reasoning": { 60 | "type": "string", 61 | "description": "Explain your reasoning step-by-step.", 62 | }, 63 | "code": { 64 | "type": "string", 65 | "description": "The full code for main.py, including all imports and code, with no abbreviations.", 66 | }, 67 | "test": { 68 | "type": "string", 69 | "description": "The full code for main_test.py, including all imports and code, with no abbreviations and compatible with pytest.", 70 | }, 71 | }, 72 | required_properties=["reasoning", "code", "test"], 73 | ) 74 | 75 | 76 | def create_handler(arguments, context): 77 | should_log = context.get("log_level", "normal") != "quiet" 78 | 79 | reasoning = arguments["reasoning"] 80 | code = arguments["code"] 81 | test = arguments["test"] 82 | 83 | project_dir = context["project_dir"] 84 | 85 | log( 86 | f"Creating main.py in {project_dir}" 87 | + f"\n\nReasoning:\n{reasoning}" 88 | + f"\n\nCode:\n{code}" 89 | + f"\n\nTest:\n{test}", 90 | title="action", 91 | type="create", 92 | log=should_log, 93 | ) 94 | 95 | save_code(code, f"{project_dir}/main.py") 96 | save_code(test, f"{project_dir}/main_test.py") 97 | 98 | return context 99 | 100 | 101 | def remove_line_numbers(text): 102 | # Regular expression pattern to match '[n]' at the beginning of a line 103 | pattern = r"^\s*\[\d+\]\s*" 104 | 105 | # Split the text into lines 106 | lines = text.split("\n") 107 | 108 | # Remove the pattern from each line 109 | cleaned_lines = [re.sub(pattern, "", line) for line in lines] 110 | 111 | # Join the cleaned lines back into a single string 112 | cleaned_text = "\n".join(cleaned_lines) 113 | 114 | return cleaned_text 115 | 116 | 117 | def write_complete_script_handler(arguments, context): 118 | should_log = context.get("log_level", "normal") != "quiet" 119 | reasoning = arguments["reasoning"] 120 | code = remove_line_numbers(arguments["code"]) 121 | 122 | filepath = arguments["filepath"] 123 | write_path = get_full_path(filepath, context["project_dir"]) 124 | 125 | log( 126 | f"Writing complete script to {write_path}" 127 | + f"\n\nReasoning:\n{reasoning}" 128 | + f"\n\nCode:\n{code}", 129 | title="action", 130 | type="write", 131 | log=should_log, 132 | ) 133 | 134 | save_code(code, write_path) 135 | return context 136 | 137 | 138 | def insert_code_handler(arguments, context): 139 | should_log = context.get("log_level", "normal") != "quiet" 140 | reasoning = arguments["reasoning"] 141 | code = arguments["code"] 142 | start_line = arguments["start_line"] 143 | filepath = arguments["filepath"] 144 | write_path = get_full_path(filepath, context["project_dir"]) 145 | 146 | log( 147 | f"Inserting code into {write_path} at line {start_line}" 148 | + f"\n\nReasoning:\n{reasoning}" 149 | + f"\n\nCode:\n{code}", 150 | title="action", 151 | type="insert", 152 | log=should_log, 153 | ) 154 | 155 | with open(write_path, "r") as f: 156 | text = f.read() 157 | lines = text.split("\n") 158 | lines.insert(start_line, code) 159 | text = "\n".join(lines) 160 | 161 | log(f"New code:\n{text}", title="action", type="insert", log=should_log) 162 | 163 | save_code(text, write_path) 164 | return context 165 | 166 | 167 | def edit_code_handler(arguments, context): 168 | edit_type = arguments["edit_type"] 169 | if edit_type == "insert": 170 | return insert_code_handler(arguments, context) 171 | elif edit_type == "replace": 172 | return replace_code_handler(arguments, context) 173 | elif edit_type == "remove": 174 | return remove_code_handler(arguments, context) 175 | 176 | 177 | def replace_code_handler(arguments, context): 178 | should_log = context.get("log_level", "normal") != "quiet" 179 | reasoning = arguments["reasoning"] 180 | code = arguments["code"] 181 | start_line = int(arguments["start_line"]) 182 | end_line = int(arguments["end_line"]) 183 | filepath = arguments["filepath"] 184 | write_path = get_full_path(filepath, context["project_dir"]) 185 | 186 | log( 187 | f"Replacing code in {write_path} from line {start_line} to {end_line}" 188 | + f"\n\nReasoning:\n{reasoning}" 189 | + f"\n\nCode:\n{code}", 190 | title="action", 191 | type="replace", 192 | log=should_log, 193 | ) 194 | 195 | # floor start_line to 1 196 | if start_line < 1: 197 | start_line = 1 198 | 199 | # Replace the code between start_lines and end_lines with new code 200 | with open(write_path, "r") as f: 201 | text = f.read() 202 | lines = text.split("\n") 203 | lines[start_line - 1 : end_line] = [code] # python's list indices start at 0 204 | text = "\n".join(lines) 205 | 206 | log(f"New code:\n{lines}", title="action", type="replace", log=should_log) 207 | 208 | save_code(text, write_path) 209 | 210 | return context 211 | 212 | 213 | def remove_code_handler(arguments, context): 214 | should_log = context.get("log_level", "normal") != "quiet" 215 | reasoning = arguments["reasoning"] 216 | start_line = int(arguments["start_line"]) 217 | end_line = int(arguments["end_line"]) 218 | filepath = arguments["filepath"] 219 | write_path = get_full_path(filepath, context["project_dir"]) 220 | 221 | log( 222 | f"Removing code in {write_path} from line {start_line} to {end_line}" 223 | + f"\n\nReasoning:\n{reasoning}", 224 | title="action", 225 | type="remove", 226 | log=should_log, 227 | ) 228 | 229 | # Remove the code between start_lines and end_lines 230 | with open(write_path, "r") as f: 231 | text = f.read() 232 | lines = text.split("\n") 233 | del lines[start_line - 1 : end_line] # python's list indices start at 0 234 | 235 | lines = "\n".join(lines) 236 | 237 | log(f"New code:\n{lines}", title="action", type="remove", log=should_log) 238 | 239 | save_code(lines, write_path) 240 | 241 | return context 242 | 243 | 244 | def create_new_file_handler(arguments, context): 245 | should_log = context.get("log_level", "normal") != "quiet" 246 | reasoning = arguments["reasoning"] 247 | filepath = arguments["filepath"] 248 | code = arguments["code"] 249 | test = arguments["test"] 250 | 251 | log( 252 | f"Creating new file {filepath}" 253 | + f"\n\nReasoning:\n{reasoning}" 254 | + f"\n\nCode:\n{code}" 255 | + f"\n\nTest:\n{test}", 256 | title="action", 257 | type="create", 258 | log=should_log, 259 | ) 260 | 261 | # Create a new file at filepath with code 262 | write_path = get_full_path(filepath, context["project_dir"]) 263 | save_code(code, write_path) 264 | 265 | # Create a new file at filepath_test.py with test 266 | test_path = get_full_path( 267 | f"{os.path.splitext(filepath)[0]}_test.py", context["project_dir"] 268 | ) 269 | save_code(test, test_path) 270 | 271 | return context 272 | 273 | 274 | def delete_file_handler(arguments, context): 275 | should_log = context.get("log_level", "normal") != "quiet" 276 | reasoning = arguments["reasoning"] 277 | filepath = arguments["filepath"] 278 | 279 | if "main.py" in filepath or "main_test.py" in filepath: 280 | log( 281 | f"File {filepath} contains main.py or main_test.py, so it will not be deleted", 282 | title="action", 283 | type="warning", 284 | log=should_log, 285 | ) 286 | return context 287 | 288 | log( 289 | f"Deleting file {filepath}" + f"\n\nReasoning:\n{reasoning}", 290 | title="action", 291 | type="delete", 292 | log=should_log, 293 | ) 294 | 295 | # Delete the file at filepath 296 | file_path = get_full_path(filepath, context["project_dir"]) 297 | # check if the file exists 298 | if os.path.exists(file_path): 299 | os.remove(file_path) 300 | # if file_path didn't contain _test.py, then delete the _test.py file 301 | if "_test.py" not in file_path: 302 | test_path = get_full_path( 303 | f"{os.path.splitext(filepath)[0]}_test.py", context["project_dir"] 304 | ) 305 | if os.path.exists(test_path): 306 | os.remove(test_path) 307 | else: 308 | log( 309 | f"File {filepath} does not exist", 310 | title="action", 311 | type="warning", 312 | log=should_log, 313 | ) 314 | 315 | return context 316 | 317 | 318 | def get_actions(): 319 | return [ 320 | # { 321 | # "function": compose_function( 322 | # name="create_new_file", 323 | # description="Create a Python file", 324 | # properties={ 325 | # "reasoning": { 326 | # "type": "string", 327 | # "description": "Explain your reasoning step-by-step.", 328 | # }, 329 | # "filepath": { 330 | # "type": "string", 331 | # "description": "The path of the file to create.", 332 | # }, 333 | # "code": { 334 | # "type": "string", 335 | # "description": "The full code for the module, including all imports and code, with no abbreviations", 336 | # }, 337 | # "test": { 338 | # "type": "string", 339 | # "description": "A set of functional pytest-compatible tests for the module code.", 340 | # }, 341 | # }, 342 | # required_properties=[ 343 | # "reasoning", 344 | # "filepath", 345 | # "code", 346 | # "test", 347 | # ], 348 | # ), 349 | # "handler": create_new_file_handler, 350 | # }, 351 | # { 352 | # "function": compose_function( 353 | # name="delete_file", 354 | # description="Delete a file that is unnecessary or contains duplicate functionality", 355 | # properties={ 356 | # "reasoning": { 357 | # "type": "string", 358 | # "description": "Why are we deleting this file? Explain step-by-step.", 359 | # }, 360 | # "filepath": { 361 | # "type": "string", 362 | # "description": "The path of the file to delete.", 363 | # }, 364 | # }, 365 | # required_properties=["reasoning", "filepath"], 366 | # ), 367 | # "handler": delete_file_handler, 368 | # }, 369 | # { 370 | # "function": compose_function( 371 | # name="edit_code", 372 | # description="Insert, replace or remove code. Insert: adds a line or more of code at start_line. Replace: replace broken code with new code, from start_line through end_line. Remove: removes start_line through end_line.", 373 | # properties={ 374 | # "reasoning": { 375 | # "type": "string", 376 | # "description": "Explain your reasoning step-by-step.", 377 | # }, 378 | # "filepath": { 379 | # "type": "string", 380 | # "description": "The path of the file to edit.", 381 | # }, 382 | # "edit_type": { 383 | # "type": "string", 384 | # "enum": ["insert", "replace", "remove"], 385 | # "description": "The type of edit to perform.", 386 | # }, 387 | # "code": { 388 | # "type": "string", 389 | # "description": "The new code to insert or replace existing code with.", 390 | # }, 391 | # "start_line": { 392 | # "type": "number", 393 | # "description": "The start line number where code is being inserted, replaced or removed.", 394 | # }, 395 | # "end_line": { 396 | # "type": "number", 397 | # "description": "The end line number where code is being replaced. Required for replace and remove, ignored for insert (set to -1).", 398 | # }, 399 | # }, 400 | # required_properties=[ 401 | # "reasoning", 402 | # "filepath", 403 | # "edit_type", 404 | # "code", 405 | # "start_line", 406 | # "end_line", 407 | # ], 408 | # ), 409 | # "handler": edit_code_handler, 410 | # }, 411 | { 412 | "function": compose_function( 413 | name="replace_code", 414 | description="Replace some of the existing code with new code, from start_line through end_line. Replace with an emptry string to remove lines, or insert new lines by setting start_line and end_line to the same line number.", 415 | properties={ 416 | "reasoning": { 417 | "type": "string", 418 | "description": "Explain your reasoning step-by-step.", 419 | }, 420 | "filepath": { 421 | "type": "string", 422 | "description": "The path of the file to edit.", 423 | }, 424 | "code": { 425 | "type": "string", 426 | "description": "The new code to insert or replace existing code with.", 427 | }, 428 | "start_line": { 429 | "type": "number", 430 | "description": "The start line number where code is being inserted, replaced or removed.", 431 | }, 432 | "end_line": { 433 | "type": "number", 434 | "description": "The end line number where code is being replaced. Required for replace and remove, ignored for insert (set to -1).", 435 | }, 436 | }, 437 | required_properties=[ 438 | "reasoning", 439 | "filepath", 440 | "code", 441 | "start_line", 442 | "end_line", 443 | ], 444 | ), 445 | "handler": replace_code_handler, 446 | }, 447 | { 448 | "function": compose_function( 449 | name="write_code", 450 | description="Write all of the code for the module, including imports and functions. No snippets, abbreviations or single lines. Must be a complete code file with the structure 'imports', 'functions', and 'main entry' for main.py -- where main entry is called if __name__ == '__main__'.", 451 | properties={ 452 | "reasoning": { 453 | "type": "string", 454 | "description": "Explain your reasoning step-by-step.", 455 | }, 456 | "filepath": { 457 | "type": "string", 458 | "description": "Path to where the file will be written (usually just filename).", 459 | }, 460 | "code": { 461 | "type": "string", 462 | "description": "The full code for the module, including all imports and code, with no abbreviations.", 463 | }, 464 | }, 465 | required_properties=["reasoning", "filepath", "code"], 466 | ), 467 | "handler": write_complete_script_handler, 468 | }, 469 | ] 470 | 471 | 472 | def step(context): 473 | """ 474 | This function serves as the 'Act' stage in the OODA loop. It executes the selected action from the 'Decide' stage. 475 | 476 | Args: 477 | context (dict): The dictionary containing data about the current state of the system, including the selected action to be taken. 478 | 479 | Returns: 480 | dict: The updated context dictionary after the 'Act' stage, which will be used in the next iteration of the OODA loop. 481 | """ 482 | 483 | if context["running"] == False: 484 | return context 485 | 486 | should_log = context.get("log_level", "normal") != "quiet" 487 | debug = context.get("log_level", "normal") == "debug" 488 | 489 | log( 490 | "Act Step Started", 491 | title="step", 492 | type="info", 493 | log=should_log, 494 | ) 495 | 496 | prompt = edit_prompt 497 | actions = get_actions() 498 | 499 | # each entry in functions is a dict 500 | # # get the "fuction" value from each entry in the function list 501 | functions = [f["function"] for f in actions] 502 | 503 | if context["file_count"] == 0: 504 | prompt = create_prompt 505 | functions = [create_function] 506 | 507 | context["available_actions"] = "Available functions:\n" 508 | for fn in actions: 509 | context[ 510 | "available_actions" 511 | ] += f"{fn['function']['name']}: {fn['function']['description']}\n" 512 | # include properties and descriptions 513 | for prop, details in fn["function"]["parameters"]["properties"].items(): 514 | context["available_actions"] += f" {prop}: {details['description']}\n" 515 | context["available_actions"] += "\n" 516 | 517 | context["available_action_names"] = "Available functions: " + ", ".join( 518 | [fn["function"]["name"] for fn in actions] 519 | ) 520 | 521 | if context.get("reasoning") is None: 522 | # find {{reasoning}} in prompt and replace with empty string 523 | prompt = prompt.replace("{{reasoning}}", "") 524 | else: 525 | log( 526 | f"Reasoning:\n{context['reasoning']}", 527 | title="action", 528 | type="reasoning", 529 | log=should_log, 530 | ) 531 | 532 | text = compose_prompt(prompt, context) 533 | 534 | response = openai_function_call( 535 | text=text, 536 | functions=functions, 537 | debug=debug, 538 | model=context.get("model", "gpt-3.5-turbo-0613"), 539 | ) 540 | 541 | # find the function in functions with the name that matches response["function_name"] 542 | # then call the handler with the arguments and context 543 | function_name = response["function_name"] 544 | 545 | # if function_name is create, then we need to create a new file 546 | if function_name == "create": 547 | create_handler(response["arguments"], context) 548 | return context 549 | 550 | arguments = response["arguments"] 551 | action = None 552 | 553 | for f in actions: 554 | if f["function"]["name"] == function_name: 555 | action = f 556 | break 557 | 558 | args_str = "\n".join( 559 | f"*** {key} ***\n{str(value)}" 560 | if "\n" in str(value) 561 | else f"*** {key}: {str(value)}" 562 | for key, value in arguments.items() 563 | ) 564 | 565 | log( 566 | f"Running action {function_name} with arguments:\n{args_str}", 567 | title="action", 568 | type="handler", 569 | log=debug, 570 | ) 571 | 572 | # call the handler with the arguments and context 573 | context = action["handler"](arguments, context) 574 | 575 | # install any new imports if there are any 576 | context = handle_packages(context) 577 | 578 | return context 579 | -------------------------------------------------------------------------------- /autocoder/steps/reason.py: -------------------------------------------------------------------------------- 1 | from easycompletion import ( 2 | openai_function_call, 3 | compose_prompt, 4 | compose_function, 5 | ) 6 | from autocoder.helpers.code import validate_file 7 | 8 | from autocoder.helpers.context import ( 9 | backup_project, 10 | collect_errors, 11 | collect_files, 12 | get_file_count, 13 | read_and_format_code, 14 | run_main, 15 | run_tests, 16 | validate_files, 17 | ) 18 | 19 | from agentlogger import log 20 | 21 | from agentloop import stop 22 | 23 | reasoning_prompt = """This is my code: 24 | {{project_code_formatted}} 25 | This is my goal: 26 | {{goal}} 27 | {{errors_formatted}} 28 | Your task: Evaluate the code and determine if it meets the goal, or what could be improved about it. 29 | - Please provide reasoning for why the code is valid and complete (or not) and respond with is_valid_and_complete=True 30 | - If it does not meet the goals, please provide explain why it does not, and what could be improved. 31 | - If there is way to improve the code, respond with is_valid_and_complete=False. 32 | """ 33 | 34 | 35 | def compose_project_validation_function(): 36 | """ 37 | This function defines the structure and requirements of the 'assess' function to be called in the 'Decide' stage of the OODA loop. 38 | 39 | Returns: 40 | dict: A dictionary containing the details of the 'assess' function, such as its properties, description, and required properties. 41 | """ 42 | return compose_function( 43 | name="project_validation_action", 44 | description="Decide which action to take next.", 45 | properties={ 46 | "reasoning": { 47 | "type": "string", 48 | "description": "Does the code fill the specification and complete all of my goals? Provide reasoning for why it does or doesn't, and what could be improved.", 49 | }, 50 | "is_valid_and_complete": { 51 | "type": "boolean", 52 | "description": "Does the code fill the specification and complete all of my goals?. True if there is nothing else that needs to be improved, False otherwise.", 53 | }, 54 | }, 55 | required_properties=["reasoning", "is_valid_and_complete"], 56 | ) 57 | 58 | 59 | def step(context, loop_dict): 60 | """ 61 | This function serves as the 'Decide' stage in the OODA loop. It uses the current context data to assess which action should be taken next. 62 | 63 | Args: 64 | context (dict): The dictionary containing data about the current state of the system. 65 | 66 | Returns: 67 | dict: The updated context dictionary after the 'Decide' stage, including the selected action and reasoning behind the action. 68 | """ 69 | if context["running"] == False: 70 | return context 71 | should_log = context.get("log_level", "normal") != "quiet" 72 | 73 | log( 74 | "Reasoning Step Started", 75 | title="step", 76 | type="info", 77 | log=should_log, 78 | ) 79 | 80 | context = get_file_count(context) 81 | 82 | debug = context.get("log_level", "normal") == "debug" 83 | should_log = context.get("log_level", "normal") != "quiet" 84 | 85 | # If we have no files, go immediately to the create step 86 | if context["file_count"] == 0: 87 | log( 88 | "Creating main.py for new project", 89 | title="new project", 90 | type="start", 91 | log=should_log, 92 | ) 93 | return context 94 | 95 | context = backup_project(context) 96 | context = collect_files(context) 97 | context = validate_files(context) 98 | context = run_tests(context) 99 | context = run_main(context) 100 | context = collect_errors(context) 101 | context = read_and_format_code(context) 102 | 103 | # format context into a string of key:value 104 | context_str = "" 105 | for key, value in context.items(): 106 | context_str += f"{key}:\n\t{value}\n" 107 | 108 | log( 109 | "Context"+"\n"+context_str, 110 | title="context", 111 | type="context", 112 | color="yellow", 113 | log=debug, 114 | ) 115 | 116 | # If we have an error, go immediately to the edit step 117 | if context.get("main_success") is False and context.get("main_error") is not None: 118 | log( 119 | f"main.py failed to run\nError:\n{context.get('main_error', 'unknown')}", 120 | title="main.py", 121 | type="error", 122 | log=should_log, 123 | ) 124 | context[ 125 | "reasoning" 126 | ] = "main.py failed to run - I probably need to fix that before I can do anything else." 127 | return context 128 | 129 | # If any of the files failed to validate for any reason, go immediately to the edit step 130 | if context["project_validated"] is False: 131 | validation_errors = "" 132 | for file_dict in context["project_code"]: 133 | file_path = file_dict["absolute_path"] 134 | validation = validate_file(file_path) 135 | if validation["success"] is False: 136 | validation_errors += f"\n{file_path}:\n{validation['error']}\n" 137 | if validation_errors != "": 138 | log( 139 | "Project failed to validate. Errors:\n" + validation_errors, 140 | title="validation", 141 | type="error", 142 | log=should_log, 143 | ) 144 | context[ 145 | "reasoning" 146 | ] = "The project failed to validate. I need to fix the validation errors." 147 | return context 148 | 149 | if context["project_tested"] is False: 150 | test_errors = "" 151 | for file_dict in context["project_code"]: 152 | if ( 153 | file_dict.get("test_success") is False 154 | and file_dict.get("test_error") is not None 155 | ): 156 | test_errors += ( 157 | f"\n{file_dict['absolute_path']}:\n{file_dict['test_error']}\n" 158 | ) 159 | if test_errors != "": 160 | log( 161 | "Project failed in testing. Errors:\n" + test_errors, 162 | title="test", 163 | type="error", 164 | log=should_log, 165 | ) 166 | context[ 167 | "reasoning" 168 | ] = "The project failed in testing. I need to fix the test errors." 169 | return context 170 | 171 | text = compose_prompt(reasoning_prompt, context) 172 | functions = compose_project_validation_function() 173 | 174 | # Handle the auto case 175 | response = openai_function_call( 176 | text=text, functions=functions, debug=debug, model=context.get("model", "gpt-3.5-turbo-0613") 177 | ) 178 | 179 | # Add the action reasoning to the context object 180 | is_valid_and_complete = response["arguments"]["is_valid_and_complete"] 181 | 182 | if is_valid_and_complete is True: 183 | log( 184 | "Project is valid and complete. Good luck!", 185 | title="validation", 186 | type="success", 187 | log=should_log, 188 | ) 189 | stop(loop_dict) 190 | context["running"] = False 191 | 192 | else: 193 | log( 194 | response["arguments"]["reasoning"], 195 | title="validation", 196 | type="reasoning", 197 | log=should_log, 198 | ) 199 | 200 | context["reasoning"] = response["arguments"]["reasoning"] 201 | return context 202 | -------------------------------------------------------------------------------- /autocoder/tests/__init__.py: -------------------------------------------------------------------------------- 1 | from .helpers import * 2 | from .steps import * 3 | from .e2e import * -------------------------------------------------------------------------------- /autocoder/tests/e2e/__init__.py: -------------------------------------------------------------------------------- 1 | from .test_create_hello_world import * 2 | from .test_hello_world_bad_test import * 3 | from .test_hello_world_finished import * 4 | from .test_hello_world_replace import * 5 | from .test_hello_world_good_test import * 6 | from .test_hello_world_broken import * -------------------------------------------------------------------------------- /autocoder/tests/e2e/test_create_hello_world.py: -------------------------------------------------------------------------------- 1 | import os 2 | from time import sleep 3 | from dotenv import load_dotenv 4 | import subprocess 5 | import shutil 6 | 7 | load_dotenv() 8 | 9 | 10 | def test_create_hello_world_e2e(): 11 | api_key = os.getenv("OPENAI_API_KEY") 12 | 13 | project = { 14 | "project_name": "helloworld_e2e", 15 | "goal": "print hello world if the main file is run", 16 | "project_dir": "project_data/helloworld_e2e", 17 | "log_level": "debug", 18 | "step": False, 19 | "api_key": api_key, 20 | } 21 | 22 | from autocoder import autocoder 23 | 24 | loop_data = autocoder(project) 25 | while loop_data["thread"].is_alive(): 26 | sleep(1) 27 | 28 | output = subprocess.check_output( 29 | ["python", "project_data/helloworld_e2e/main.py"] 30 | ).decode("utf-8") 31 | assert "hello world" in output.lower() 32 | # remove the project_data/helloworld_e2e folder 33 | shutil.rmtree("project_data/helloworld_e2e") 34 | -------------------------------------------------------------------------------- /autocoder/tests/e2e/test_hello_world_bad_test.py: -------------------------------------------------------------------------------- 1 | import os 2 | from time import sleep 3 | from dotenv import load_dotenv 4 | import subprocess 5 | import shutil 6 | 7 | load_dotenv() 8 | 9 | main_py = """\ 10 | def main(): 11 | print("hello world") 12 | 13 | 14 | if __name__ == "__main__": 15 | main() 16 | """ 17 | 18 | main_test_py = """\ 19 | def test_main(capsys): 20 | captured = capsys.readouterr() 21 | 22 | from main import main 23 | main() 24 | 25 | captured = capsys.readouterr() 26 | assert captured.out == "hello world2\n" 27 | """ 28 | 29 | 30 | def test_create_hello_world_bad_test(): 31 | api_key = os.getenv("OPENAI_API_KEY") 32 | 33 | project = { 34 | "project_name": "helloworld_e2e_badtest", 35 | "goal": "print hello world if the main file is run", 36 | "project_dir": "project_data/helloworld_e2e_badtest", 37 | "log_level": "debug", 38 | "step": False, 39 | "api_key": api_key, 40 | } 41 | 42 | os.makedirs("project_data/helloworld_e2e_badtest") 43 | with open("project_data/helloworld_e2e_badtest/main.py", "w") as f: 44 | f.write(main_py) 45 | 46 | with open("project_data/helloworld_e2e_badtest/main_test.py", "w") as f: 47 | f.write(main_test_py) 48 | 49 | from autocoder import autocoder 50 | 51 | loop_data = autocoder(project) 52 | while loop_data["thread"].is_alive(): 53 | sleep(1) 54 | 55 | output = subprocess.check_output( 56 | ["python", "project_data/helloworld_e2e_badtest/main.py"] 57 | ).decode("utf-8") 58 | assert "hello world" in output.lower() 59 | # remove the project_data/helloworld_e2e folder 60 | shutil.rmtree("project_data/helloworld_e2e_badtest") 61 | -------------------------------------------------------------------------------- /autocoder/tests/e2e/test_hello_world_broken.py: -------------------------------------------------------------------------------- 1 | import os 2 | from time import sleep 3 | from dotenv import load_dotenv 4 | import subprocess 5 | import shutil 6 | 7 | load_dotenv() 8 | 9 | main_py = """\ 10 | def main(): 11 | print("this is clearly not right") 12 | 13 | 14 | if __name__ == "__main__": 15 | main() 16 | """ 17 | 18 | main_test_py = """\ 19 | def test_main(capsys): 20 | captured = capsys.readouterr() 21 | 22 | from main import main 23 | main() 24 | 25 | captured = capsys.readouterr() 26 | assert captured.out == "hello world\n" 27 | """ 28 | 29 | 30 | def test_create_hello_world_broken(): 31 | api_key = os.getenv("OPENAI_API_KEY") 32 | 33 | project = { 34 | "project_name": "helloworld_e2e_broken", 35 | "goal": "print hello world if the main file is run", 36 | "project_dir": "project_data/helloworld_e2e_broken", 37 | "log_level": "debug", 38 | "step": False, 39 | "api_key": api_key, 40 | } 41 | 42 | # write main_py to helloworld_e2e_broken/main.py 43 | os.makedirs("project_data/helloworld_e2e_broken") 44 | with open("project_data/helloworld_e2e_broken/main.py", "w") as f: 45 | f.write(main_py) 46 | 47 | # write main_test_py to helloworld_e2e_broken/main_test.py 48 | with open("project_data/helloworld_e2e_broken/main_test.py", "w") as f: 49 | f.write(main_test_py) 50 | 51 | from autocoder import autocoder 52 | 53 | loop_data = autocoder(project) 54 | while loop_data["thread"].is_alive(): 55 | sleep(1) 56 | 57 | output = subprocess.check_output( 58 | ["python", "project_data/helloworld_e2e_broken/main.py"] 59 | ).decode("utf-8") 60 | assert "hello world" in output.lower() 61 | # remove the project_data/helloworld_e2e folder 62 | shutil.rmtree("project_data/helloworld_e2e_broken") 63 | -------------------------------------------------------------------------------- /autocoder/tests/e2e/test_hello_world_finished.py: -------------------------------------------------------------------------------- 1 | import os 2 | from time import sleep 3 | from dotenv import load_dotenv 4 | import subprocess 5 | import shutil 6 | 7 | load_dotenv() 8 | 9 | main_py = """\ 10 | def main(): 11 | print("hello world") 12 | 13 | 14 | if __name__ == "__main__": 15 | main() 16 | """ 17 | 18 | main_test_py = """\ 19 | def test_main(capsys): 20 | captured = capsys.readouterr() 21 | 22 | from main import main 23 | main() 24 | 25 | captured = capsys.readouterr() 26 | assert captured.out == "hello world\n" 27 | """ 28 | 29 | 30 | def test_create_hello_world_finished(): 31 | api_key = os.getenv("OPENAI_API_KEY") 32 | 33 | project = { 34 | "project_name": "helloworld_e2e_finished", 35 | "goal": "print hello world if the main file is run", 36 | "project_dir": "project_data/helloworld_e2e_finished", 37 | "log_level": "debug", 38 | "step": False, 39 | "api_key": api_key, 40 | } 41 | 42 | # write main_py to helloworld_e2e_finished/main.py 43 | os.makedirs("project_data/helloworld_e2e_finished") 44 | with open("project_data/helloworld_e2e_finished/main.py", "w") as f: 45 | f.write(main_py) 46 | 47 | # write main_test_py to helloworld_e2e_finished/main_test.py 48 | with open("project_data/helloworld_e2e_finished/main_test.py", "w") as f: 49 | f.write(main_test_py) 50 | 51 | from autocoder import autocoder 52 | 53 | loop_data = autocoder(project) 54 | while loop_data["thread"].is_alive(): 55 | sleep(1) 56 | 57 | output = subprocess.check_output( 58 | ["python", "project_data/helloworld_e2e_finished/main.py"] 59 | ).decode("utf-8") 60 | assert "hello world" in output.lower() 61 | # remove the project_data/helloworld_e2e folder 62 | shutil.rmtree("project_data/helloworld_e2e_finished") 63 | -------------------------------------------------------------------------------- /autocoder/tests/e2e/test_hello_world_good_test.py: -------------------------------------------------------------------------------- 1 | import os 2 | from time import sleep 3 | from dotenv import load_dotenv 4 | import subprocess 5 | import shutil 6 | 7 | load_dotenv() 8 | 9 | main_py = """\ 10 | def main(): 11 | print("hello world") 12 | 13 | 14 | if __name__ == "__main__": 15 | main() 16 | """ 17 | 18 | main_test_py = """\ 19 | def test_main(capsys): 20 | captured = capsys.readouterr() 21 | 22 | from main import main 23 | main() 24 | 25 | captured = capsys.readouterr() 26 | assert captured.out == "hello world2\n" 27 | """ 28 | 29 | 30 | def test_create_hello_world_good_test(): 31 | api_key = os.getenv("OPENAI_API_KEY") 32 | 33 | project = { 34 | "project_name": "helloworld_e2e_goodtest", 35 | "goal": "print hello world if the main file is run", 36 | "project_dir": "project_data/helloworld_e2e_goodtest", 37 | "log_level": "debug", 38 | "step": False, 39 | "api_key": api_key, 40 | } 41 | 42 | os.makedirs("project_data/helloworld_e2e_goodtest") 43 | with open("project_data/helloworld_e2e_goodtest/main.py", "w") as f: 44 | f.write(main_py) 45 | 46 | with open("project_data/helloworld_e2e_goodtest/main_test.py", "w") as f: 47 | f.write(main_test_py) 48 | 49 | from autocoder import autocoder 50 | 51 | loop_data = autocoder(project) 52 | while loop_data["thread"].is_alive(): 53 | sleep(1) 54 | 55 | output = subprocess.check_output( 56 | ["python", "project_data/helloworld_e2e_goodtest/main.py"] 57 | ).decode("utf-8") 58 | assert "hello world" in output.lower() 59 | # remove the project_data/helloworld_e2e folder 60 | shutil.rmtree("project_data/helloworld_e2e_goodtest") 61 | -------------------------------------------------------------------------------- /autocoder/tests/e2e/test_hello_world_replace.py: -------------------------------------------------------------------------------- 1 | import os 2 | from time import sleep 3 | from dotenv import load_dotenv 4 | import subprocess 5 | import shutil 6 | 7 | load_dotenv() 8 | 9 | main_py = """\ 10 | def main(): 11 | print("this is clearly not right") 12 | 13 | 14 | if __name__ == "__main__": 15 | main() 16 | """ 17 | 18 | main_test_py = """\ 19 | def test_main(capsys): 20 | captured = capsys.readouterr() 21 | 22 | from main import main 23 | main() 24 | 25 | captured = capsys.readouterr() 26 | assert captured.out == "hello world\n" 27 | """ 28 | 29 | 30 | def test_create_hello_world_replace(): 31 | api_key = os.getenv("OPENAI_API_KEY") 32 | 33 | project = { 34 | "project_name": "helloworld_e2e_replace", 35 | "goal": "print hello world if the main file is run", 36 | "project_dir": "project_data/helloworld_e2e_replace", 37 | "log_level": "debug", 38 | "step": False, 39 | "api_key": api_key, 40 | } 41 | 42 | # write main_py to helloworld_e2e_replace/main.py 43 | os.makedirs("project_data/helloworld_e2e_replace") 44 | with open("project_data/helloworld_e2e_replace/main.py", "w") as f: 45 | f.write(main_py) 46 | 47 | # write main_test_py to helloworld_e2e_replace/main_test.py 48 | with open("project_data/helloworld_e2e_replace/main_test.py", "w") as f: 49 | f.write(main_test_py) 50 | 51 | from autocoder import autocoder 52 | 53 | loop_data = autocoder(project) 54 | while loop_data["thread"].is_alive(): 55 | sleep(1) 56 | 57 | output = subprocess.check_output( 58 | ["python", "project_data/helloworld_e2e_replace/main.py"] 59 | ).decode("utf-8") 60 | assert "hello world" in output.lower() 61 | # remove the project_data/helloworld_e2e folder 62 | shutil.rmtree("project_data/helloworld_e2e_replace") 63 | -------------------------------------------------------------------------------- /autocoder/tests/helpers/__init__.py: -------------------------------------------------------------------------------- 1 | from .code import * 2 | from .context import * 3 | from .files import * -------------------------------------------------------------------------------- /autocoder/tests/helpers/code.py: -------------------------------------------------------------------------------- 1 | import tempfile 2 | import os 3 | from autocoder.helpers.code import ( 4 | contains_function_definition, 5 | file_exists, 6 | has_functions_called, 7 | is_runnable, 8 | count_lines, 9 | organize_imports, 10 | validate_file, 11 | validate_code, 12 | save_code, 13 | run_code, 14 | run_code_tests, 15 | ) 16 | 17 | 18 | def test_is_runnable_success(): 19 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 20 | tmp.write(b"print('Hello, world!')") 21 | assert is_runnable(tmp.name) == True 22 | os.remove(tmp.name) 23 | 24 | 25 | def test_is_runnable_failure(): 26 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 27 | tmp.write(b"\n()prin('Hello, world!asdfs)asdfsafsa\ndfgs\n") 28 | tmp.flush() 29 | tmp.close() 30 | 31 | assert is_runnable(tmp.name) == False 32 | os.remove(tmp.name) 33 | 34 | 35 | def test_contains_function_definition_true(): 36 | code = """ 37 | def hello(): 38 | print('Hello, world!') 39 | hello() 40 | """ 41 | assert contains_function_definition(code) == True 42 | 43 | 44 | def test_has_functions_called_true(): 45 | code = """ 46 | print('Hello, world!') 47 | """ 48 | assert has_functions_called(code) == True 49 | 50 | 51 | def test_has_functions_called_false(): 52 | code = """ 53 | def hello(): 54 | print('Hello, world!') 55 | """ 56 | assert has_functions_called(code) == False 57 | 58 | 59 | def test_file_exists_true(): 60 | with tempfile.NamedTemporaryFile(delete=False) as tmp: 61 | tmp.write(b"print('Hello, world!')") 62 | assert file_exists(tmp.name) == True 63 | os.remove(tmp.name) 64 | 65 | 66 | def test_file_exists_false(): 67 | assert file_exists("/path/to/nonexistent/file") == False 68 | 69 | 70 | def test_count_lines_with_comments_and_empty_lines(): 71 | code = """ 72 | # This is a comment 73 | print('Hello, world!') # This is another comment 74 | 75 | # This is yet another comment 76 | """ 77 | assert count_lines(code) == 1 78 | 79 | 80 | def test_count_lines_without_comments_and_empty_lines(): 81 | code = """ 82 | # This is a comment 83 | print('Hello, world!') # This is another comment 84 | 85 | # This is yet another comment 86 | """ 87 | assert ( 88 | count_lines(code, exclude_comments=True, exclude_empty_lines=True) == 1 89 | ), "Should be 1 line but is {}".format( 90 | count_lines(code, exclude_comments=True, exclude_empty_lines=True) 91 | ) 92 | 93 | 94 | def test_validate_code_success(): 95 | code = """ 96 | import os 97 | 98 | def hello(): 99 | print('Hello, world!') 100 | 101 | hello() 102 | """ 103 | assert validate_code(code) == {"success": True, "error": None} 104 | 105 | 106 | def test_validate_code_failure(): 107 | code = """ 108 | import os 109 | TODO: Implement function here 110 | """ 111 | result = validate_code(code) 112 | assert result["success"] == False 113 | 114 | 115 | def test_save_code(): 116 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 117 | code = 'print("Hello, world!"")' 118 | save_code(code, tmp.name) 119 | with open(tmp.name, "r") as f: 120 | new_code = f.read() 121 | assert new_code == code 122 | os.remove(tmp.name) 123 | 124 | 125 | def test_run_code_success(): 126 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 127 | tmp.write(b"print('Hello, world!')") 128 | result = run_code(tmp.name) 129 | assert result["success"] == True 130 | assert result["error"] == None 131 | assert result["output"].strip() == "Hello, world!" 132 | os.remove(tmp.name) 133 | 134 | 135 | def test_run_code_failure(): 136 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 137 | tmp.write(b"aprint(foo)a") 138 | result = run_code(tmp.name) 139 | assert result["success"] == False 140 | os.remove(tmp.name) 141 | 142 | 143 | def test_contains_function_definition_false(): 144 | code = """ 145 | def hello(): 146 | print('Hello, world!') 147 | """ 148 | assert contains_function_definition(code) == True 149 | 150 | 151 | def test_validate_file_success(): 152 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 153 | tmp.write( 154 | b"import os\ndef hello():\n\tprint('Hello, world!')\nhello()\n\ndef goodbye():\n\tprint('Goodbye, world!')\ngoodbye()" 155 | ) 156 | output = validate_file(tmp.name) 157 | assert validate_file(tmp.name) == {"success": True, "error": None} 158 | os.remove(tmp.name) 159 | 160 | 161 | def test_validate_file_failure(): 162 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 163 | tmp.write(b"print('Hello, world!") 164 | assert validate_file(tmp.name) == { 165 | "success": False, 166 | "error": "The file is not runnable, or didn't compile.", 167 | } 168 | os.remove(tmp.name) 169 | 170 | 171 | def test_test_code_success(): 172 | with tempfile.NamedTemporaryFile(suffix=".py", delete=False) as tmp: 173 | tmp.write( 174 | b""" 175 | def test_hello(): 176 | assert 'Hello, world!' == 'Hello, world!' 177 | test_hello() 178 | """ 179 | ) 180 | result = run_code_tests(tmp.name) 181 | assert result["success"] == True 182 | # assert "1 passed" in result["output"] 183 | # assert result["error"] == "" 184 | os.remove(tmp.name) 185 | 186 | 187 | def test_organize_imports(): 188 | expected_output = """\ 189 | import os 190 | import sys 191 | from random import randint 192 | from random import randint, shuffle 193 | 194 | 195 | def foo(): 196 | return randint(1, 10) 197 | 198 | 199 | print(foo()) 200 | """ 201 | to_test = "" 202 | # get the first two lines of expected_output and add to to_test 203 | for line in expected_output.split("\n")[:2]: 204 | to_test += line + "\n" 205 | # now add all of expected_output to to_test to simulate doubled imports 206 | to_test += expected_output 207 | actual_output = organize_imports(to_test) 208 | assert expected_output.strip() == actual_output.strip() 209 | -------------------------------------------------------------------------------- /autocoder/tests/helpers/context.py: -------------------------------------------------------------------------------- 1 | import os 2 | import shutil 3 | from pathlib import Path 4 | from autocoder.helpers.code import extract_imports 5 | 6 | from autocoder.helpers.context import ( 7 | get_file_count, 8 | read_and_format_code, 9 | collect_files, 10 | validate_files, 11 | run_tests, 12 | run_main, 13 | backup_project, 14 | ) 15 | from autocoder.helpers.files import get_full_path 16 | 17 | # Assuming you have a directory for testing 18 | TEST_DIR = "test_dir" 19 | PROJECT_NAME = "test_project" 20 | 21 | 22 | def setup_function(): 23 | # Create a temporary directory for testing if TEST_DIR doesn't exist 24 | if not Path(TEST_DIR).exists(): 25 | os.mkdir(TEST_DIR) 26 | 27 | # Add some python files 28 | with open(os.path.join(TEST_DIR, "main.py"), "w") as f: 29 | f.write('print("Hello, world!")') 30 | 31 | with open(os.path.join(TEST_DIR, "test_main.py"), "w") as f: 32 | f.write("def test_main(): assert True") 33 | 34 | 35 | def teardown_function(): 36 | # Remove the directory after test 37 | shutil.rmtree(TEST_DIR) 38 | 39 | 40 | def test_get_file_count(): 41 | setup_function() 42 | 43 | context = {"project_dir": TEST_DIR} 44 | result = get_file_count(context) 45 | 46 | assert "file_count" in result 47 | assert result["file_count"] == 2 # We've created 2 files in setup 48 | 49 | teardown_function() 50 | 51 | 52 | def test_read_and_format_code(): 53 | setup_function() 54 | 55 | context = {"project_dir": TEST_DIR} 56 | context = collect_files(context) 57 | context = validate_files(context) 58 | context = run_tests(context) 59 | context = run_main(context) 60 | result = read_and_format_code(context) 61 | 62 | assert "project_code_formatted" in result 63 | assert "Hello, world!" in result["project_code_formatted"] 64 | 65 | teardown_function() 66 | 67 | 68 | def test_collect_files(): 69 | setup_function() 70 | 71 | context = {"project_dir": TEST_DIR} 72 | result = collect_files(context) 73 | 74 | assert "filetree" in result 75 | assert "filetree_formatted" in result 76 | assert "python_files" in result 77 | assert "project_code" in result 78 | 79 | teardown_function() 80 | 81 | 82 | def test_run_main(): 83 | setup_function() 84 | 85 | context = {"project_dir": TEST_DIR} 86 | context = collect_files(context) 87 | result = run_main(context) 88 | 89 | assert "main_success" in result 90 | assert ( 91 | result["main_success"] is True 92 | ) # main.py just prints a string, so it should succeed 93 | 94 | teardown_function() 95 | 96 | 97 | def test_backup_project(): 98 | setup_function() 99 | 100 | context = {"project_dir": TEST_DIR, "project_name": PROJECT_NAME} 101 | result = backup_project(context) 102 | 103 | assert "backup" in result 104 | assert Path(result["backup"]).exists() # The backup file should be created 105 | 106 | teardown_function() 107 | 108 | 109 | def test_validate_files(): 110 | setup_function() 111 | 112 | context = {"project_dir": TEST_DIR} 113 | context = collect_files(context) 114 | context = validate_files(context) 115 | 116 | assert "project_code" in context, "project_code should be in the context" 117 | assert "project_validated" in context, "project_validated should be in the context" 118 | assert context["project_validated"] is False, "project_validated should be False" 119 | 120 | teardown_function() 121 | 122 | 123 | def test_run_tests(): 124 | setup_function() 125 | 126 | context = {"project_dir": TEST_DIR} 127 | context = collect_files(context) 128 | context = run_tests(context) 129 | 130 | assert "project_tested" in context 131 | assert context["project_tested"] is False 132 | 133 | teardown_function() 134 | 135 | 136 | def test_get_full_path(): 137 | project_dir = "./example/project" 138 | filepath = "subdir/file.txt" 139 | expected_full_path = os.getcwd() + "/example/project/subdir/file.txt" 140 | 141 | assert get_full_path(filepath, project_dir) == expected_full_path 142 | # remove ./example and everything inside 143 | shutil.rmtree("./example") 144 | 145 | 146 | def test_extract_imports(): 147 | test_cases = [ 148 | # Here you can add more test cases 149 | ( 150 | "from langchain.llms import OpenAI\nimport langchain.chat_models\nimport numpy as np\nfrom pandas import DataFrame\nimport os\n", 151 | ["langchain", "numpy", "pandas"], 152 | ), 153 | ("import start\n", []), 154 | ] 155 | 156 | for code, expected_output in test_cases: 157 | print(extract_imports(code, ".")) 158 | print("expected") 159 | print(set(expected_output)) 160 | assert set(extract_imports(code, ".")) == set(expected_output) 161 | -------------------------------------------------------------------------------- /autocoder/tests/helpers/files.py: -------------------------------------------------------------------------------- 1 | import os 2 | import tempfile 3 | import shutil 4 | from pathlib import Path 5 | import zipfile 6 | from autocoder.helpers.files import ( 7 | count_files, 8 | file_tree_to_dict, 9 | file_tree_to_string, 10 | get_python_files, 11 | zip_python_files, 12 | ) 13 | 14 | 15 | def test_file_tree_to_dict(): 16 | with tempfile.TemporaryDirectory() as tmpdirname: 17 | os.makedirs(os.path.join(tmpdirname, "dir1/dir2")) 18 | open(os.path.join(tmpdirname, "file1.txt"), "w").close() 19 | open(os.path.join(tmpdirname, "dir1/file2.txt"), "w").close() 20 | open(os.path.join(tmpdirname, "dir1/dir2/file3.txt"), "w").close() 21 | 22 | file_tree = file_tree_to_dict(tmpdirname) 23 | expected_tree = { 24 | "file1.txt": None, 25 | "dir1": {"file2.txt": None, "dir2": {"file3.txt": None}}, 26 | } 27 | 28 | assert file_tree == expected_tree 29 | 30 | 31 | def test_count_files(): 32 | dirpath = tempfile.mkdtemp() 33 | open(os.path.join(dirpath, "file1.txt"), "w").close() 34 | open(os.path.join(dirpath, "file2.txt"), "w").close() 35 | open(os.path.join(dirpath, "file3.txt"), "w").close() 36 | 37 | assert count_files(dirpath) == 3 38 | 39 | # Clean up 40 | shutil.rmtree(dirpath) 41 | 42 | # Test with non-existing directory 43 | assert count_files("nonexistingdirectory") == 0 44 | 45 | 46 | def test_file_tree_to_string(): 47 | # create a temporary directory 48 | with tempfile.TemporaryDirectory() as tmpdirname: 49 | os.makedirs(os.path.join(tmpdirname, "dir1/dir2")) 50 | open(os.path.join(tmpdirname, "file1.txt"), "w").close() 51 | open(os.path.join(tmpdirname, "dir1/file2.txt"), "w").close() 52 | open(os.path.join(tmpdirname, "dir1/dir2/file3.txt"), "w").close() 53 | 54 | file_tree_str = file_tree_to_string(tmpdirname) 55 | assert "dir1/\n" in file_tree_str 56 | assert "dir2/\n" in file_tree_str 57 | assert "file1.txt\n" in file_tree_str 58 | assert "file2.txt\n" in file_tree_str 59 | assert "file3.txt\n" in file_tree_str 60 | 61 | 62 | def test_get_python_files(): 63 | dirpath = tempfile.mkdtemp() 64 | open(os.path.join(dirpath, "file1.py"), "w").close() 65 | open(os.path.join(dirpath, "file2.py"), "w").close() 66 | open(os.path.join(dirpath, "file3.txt"), "w").close() 67 | 68 | py_files = get_python_files(dirpath) 69 | assert len(py_files) == 2 70 | assert all([f.endswith(".py") for f in py_files]) 71 | 72 | # Clean up 73 | shutil.rmtree(dirpath) 74 | 75 | 76 | def test_zip_python_files(): 77 | project_dir = tempfile.mkdtemp() 78 | open(os.path.join(project_dir, "file1.py"), "w").close() 79 | open(os.path.join(project_dir, "file2.py"), "w").close() 80 | open(os.path.join(project_dir, "file3.txt"), "w").close() 81 | 82 | zip_file_path = zip_python_files(project_dir, "test_project") 83 | 84 | assert Path(zip_file_path).is_file() 85 | 86 | # Check the zip file contents 87 | with zipfile.ZipFile(zip_file_path, "r") as zipf: 88 | assert len(zipf.namelist()) == 2 89 | assert all([f.endswith(".py") for f in zipf.namelist()]) 90 | 91 | # Clean up 92 | shutil.rmtree(project_dir) 93 | os.remove(zip_file_path) 94 | -------------------------------------------------------------------------------- /autocoder/tests/steps/__init__.py: -------------------------------------------------------------------------------- 1 | from .act import * 2 | from .reason import * -------------------------------------------------------------------------------- /autocoder/tests/steps/act.py: -------------------------------------------------------------------------------- 1 | import os 2 | import shutil 3 | from autocoder.helpers.files import get_full_path 4 | 5 | from autocoder.steps.act import ( 6 | create_handler, 7 | create_new_file_handler, 8 | delete_file_handler, 9 | insert_code_handler, 10 | remove_code_handler, 11 | replace_code_handler, 12 | write_complete_script_handler, 13 | ) 14 | 15 | 16 | def setup_function(): 17 | if not os.path.exists("test_dir"): 18 | os.makedirs("test_dir") 19 | 20 | 21 | def teardown_function(): 22 | if os.path.exists("test_dir"): 23 | for root, dirs, files in os.walk("test_dir", topdown=False): 24 | for name in files: 25 | os.remove(os.path.join(root, name)) 26 | for name in dirs: 27 | os.rmdir(os.path.join(root, name)) 28 | # remove test_dir 29 | if os.path.exists("test_dir"): 30 | # use shutil to recursively remove the directory 31 | shutil.rmtree("test_dir") 32 | 33 | 34 | def test_create_handler(): 35 | setup_function() 36 | context = {"project_dir": "test_dir"} 37 | arguments = { 38 | "reasoning": "Testing create handler", 39 | "code": "print('Hello, World!')", 40 | "test": "def test_main(): assert True", 41 | } 42 | create_handler(arguments, context) 43 | 44 | assert os.path.isfile("test_dir/main.py") 45 | assert os.path.isfile("test_dir/main_test.py") 46 | teardown_function() 47 | 48 | 49 | def test_write_complete_script_handler(): 50 | setup_function() 51 | context = {"project_dir": "test_dir"} 52 | arguments = { 53 | "reasoning": "Testing write complete script handler", 54 | "code": "print('Hello, World!')", 55 | "filepath": "main.py", 56 | } 57 | write_complete_script_handler(arguments, context) 58 | 59 | assert os.path.isfile("test_dir/main.py") 60 | # read and print file contents 61 | with open("test_dir/main.py", "r") as f: 62 | text = f.read() 63 | lines = text.split("\n") 64 | assert "Hello, World!" in lines[0] 65 | print(text) 66 | teardown_function() 67 | 68 | 69 | def test_insert_code_handler(): 70 | setup_function() 71 | context = {"project_dir": "test_dir"} 72 | arguments = { 73 | "reasoning": "Testing insert code handler", 74 | "code": "print('Inserted line')", 75 | "start_line": 1, 76 | "filepath": "main.py", 77 | } 78 | 79 | write_arguments = { 80 | "reasoning": "Testing write complete script handler", 81 | "code": "print('Hello, World!')\nprint('An inserted line should come before this!')", 82 | "filepath": "main.py", 83 | } 84 | write_complete_script_handler( 85 | write_arguments, context 86 | ) # First, create a file to insert into 87 | 88 | insert_code_handler(arguments, context) 89 | 90 | with open("test_dir/main.py", "r") as f: 91 | text = f.read() 92 | lines = text.split("\n") 93 | print("Insert code:\n====================") 94 | print(text) 95 | print("====================") 96 | assert "Inserted line" in lines[1] 97 | teardown_function() 98 | 99 | 100 | def test_replace_code_handler(): 101 | setup_function() 102 | context = {"project_dir": "test_dir"} 103 | arguments = { 104 | "reasoning": "Testing replace code handler", 105 | "code": "print('New line')\nprint('Second new line')", 106 | "start_line": 1, 107 | "end_line": 2, 108 | "filepath": "main.py", 109 | } 110 | 111 | # write test_dir/main.py 112 | with open("test_dir/main.py", "w") as f: 113 | f.write("print('Old line')\nprint('Second old line')\nprint('Third old line')") 114 | 115 | replace_code_handler(arguments, context) 116 | 117 | with open("test_dir/main.py", "r") as f: 118 | text = f.read() 119 | lines = text.split("\n") 120 | print("Replace code:") 121 | print(text) 122 | assert "New line" in lines[0] 123 | teardown_function() 124 | 125 | 126 | def test_remove_code_handler(): 127 | setup_function() 128 | context = {"project_dir": "test_dir"} 129 | arguments = { 130 | "reasoning": "Testing remove code handler", 131 | "start_line": 1, 132 | "end_line": 1, 133 | "filepath": "main.py", 134 | } 135 | write_arguments = { 136 | "reasoning": "Testing write complete script handler", 137 | "code": "print('Hello, World!')", 138 | "filepath": "main.py", 139 | } 140 | write_complete_script_handler( 141 | write_arguments, context 142 | ) # First, create a file to remove from 143 | remove_code_handler(arguments, context) 144 | 145 | with open("test_dir/main.py", "r") as f: 146 | text = f.read() 147 | print(text) 148 | lines = text.split("\n") 149 | assert "New line" not in lines 150 | teardown_function() 151 | 152 | 153 | def test_create_new_file_handler(): 154 | setup_function() 155 | context = {"project_dir": "test_dir"} 156 | arguments = { 157 | "reasoning": "Testing create new file handler", 158 | "filepath": "new_file.py", 159 | "code": "print('Hello, New File!')", 160 | "test": "def test_new_file(): assert True", 161 | } 162 | create_new_file_handler(arguments, context) 163 | 164 | assert os.path.isfile("test_dir/new_file.py") 165 | assert os.path.isfile("test_dir/new_file_test.py") 166 | teardown_function() 167 | 168 | 169 | def test_delete_file_handler(): 170 | setup_function() 171 | 172 | # Add a file that will be deleted 173 | with open(os.path.join("test_dir", "file_to_delete.py"), "w") as f: 174 | f.write('print("This file will be deleted!")') 175 | 176 | # Add the corresponding test file 177 | with open(os.path.join("test_dir", "file_to_delete_test.py"), "w") as f: 178 | f.write("def test_file_to_delete(): assert True") 179 | 180 | context = {"project_dir": "test_dir", "log_level": "debug"} 181 | 182 | arguments = { 183 | "reasoning": "Testing delete function", 184 | "filepath": "file_to_delete.py", 185 | } 186 | 187 | context = delete_file_handler(arguments, context) 188 | 189 | file_path = get_full_path("file_to_delete.py", context["project_dir"]) 190 | test_file_path = get_full_path("file_to_delete_test.py", context["project_dir"]) 191 | 192 | # Ensure the files were deleted 193 | assert not os.path.exists(file_path), f"{file_path} was not deleted" 194 | assert not os.path.exists(test_file_path), f"{test_file_path} was not deleted" 195 | 196 | teardown_function() 197 | -------------------------------------------------------------------------------- /autocoder/tests/steps/reason.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JoinTheAlliance/autocoder/f4711f6125a3c806d67848773662f5d98a12ab6b/autocoder/tests/steps/reason.py -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | ### Autocoder docker-compose 2 | # 3 | ## Build with # docker-compose up -d --build 4 | ## Run with # docker exec -it autocoder python start.py 5 | ## 6 | # 7 | ## Settings commented for the following: 8 | # 9 | # Expose ports for your autocoder 10 | # You may wish to map it's project_data dir to a different or unique dir 11 | # Adjust the current hardcap of 90% CPU usage and 7GB ram usage 12 | # 13 | version: '3' 14 | services: 15 | autocoder: 16 | container_name: autocoder 17 | build: 18 | context: . 19 | ## Update ports if you need them exposed 20 | # ports: 21 | # - "7860:7860" 22 | volumes: 23 | - ./project_data:/app/project_data 24 | env_file: 25 | - .env 26 | deploy: 27 | resources: 28 | limits: 29 | ## Adjust limits of CPU % and max ram usage 30 | cpus: '0.9' 31 | memory: 7G 32 | -------------------------------------------------------------------------------- /dockerfile: -------------------------------------------------------------------------------- 1 | # Use an official Python runtime as a parent image 2 | FROM python:slim-buster 3 | 4 | # Set the working directory in the container to /app 5 | WORKDIR /app 6 | 7 | # Add the parent directory contents into the container at /app 8 | ADD . /app 9 | 10 | # Create .preferences file as blank json 11 | RUN echo "{}" > .preferences 12 | 13 | # Install build essentials for gcc, required for buidling some of the following python requirements 14 | RUN apt-get update && \ 15 | apt-get install -y --no-install-recommends build-essential 16 | 17 | # Install any needed packages specified in requirements.txt 18 | RUN pip install . 19 | 20 | 21 | # Run a shell upon starting up and keep it running 22 | CMD tail -f /dev/null 23 | 24 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | python-dotenv 2 | agentmemory 3 | agentbrowser 4 | easycompletion 5 | agentloop 6 | agentlogger 7 | pytest 8 | prompt_toolkit 9 | black 10 | astor -------------------------------------------------------------------------------- /resources/image.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JoinTheAlliance/autocoder/f4711f6125a3c806d67848773662f5d98a12ab6b/resources/image.jpg -------------------------------------------------------------------------------- /resources/youcreatethefuture.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JoinTheAlliance/autocoder/f4711f6125a3c806d67848773662f5d98a12ab6b/resources/youcreatethefuture.jpg -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | from setuptools import setup 3 | 4 | project_name = "autocoder" 5 | version="0.0.1" 6 | 7 | long_description = "" 8 | with open("README.md", "r") as fh: 9 | long_description = fh.read() 10 | # search for any lines that contain