.
675 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | ProgramEngineerGPT
2 |
3 |
4 |
5 | 
6 |
7 |
8 |
9 |
10 | [](https://github.com/hackedbyagirl/ProgramEngineerGPT)
11 | [](https://github.com/hackedbyagirl/ProgramEngineerGPT/issues)
12 | [](https://twitter.com/hackedbyagirl)
13 |
14 |
15 |
16 | ---
17 |
18 | ProgramEngineerGPT is an interactive command line tool that leverages the power of AI to assist developers with code comprehension, exploration, and generation. It serves as a virtual assistant that can analyze codebases, answer queries about code, and even help in setting up new coding projects.
19 |
20 |
21 |
22 | ## Table of Contents
23 | - [About](#about)
24 | - [Getting Started](#getting_started)
25 | - [Usage](#usage)
26 | - [Examples](#examples)
27 | - [Acknowledgments](#acknowledgement)
28 |
29 | ## About
30 | ProgramEngineerGPT is an AI-powered tool designed to assist developers with code comprehension, exploration, and generation. ProgramEngineerGPT can be used as a comprehensive developer's assistant that can understand code at a deep level and can provide valuable insights and assistance. Whether you're trying to understand a complex codebase or starting a new project, ProgramEngineerGPT can be run using two modes of operation
31 | - Analyze Mode
32 | - Develop Mode.
33 |
34 | ### Analyze Mode
35 |
36 | In the 'Analyze' mode, ProgramEngineerGPT will thoroughly examine a provided code repository. You will be engaged in an interactive chat session where you can pose queries about the codebase. This could include questions about its structure, dependencies, functions, or any other aspect. The AI will respond with insights, helping you gain a deeper understanding of the code repository and how it funtions.
37 |
38 | ### Develop Mode
39 |
40 | In the 'Develop' mode, ProgramEngineerGPT can assist you in setting up a new coding project. This includes planning the project structure, setting up the development environment, and other setup tasks. You will enter an interactive session where you will provide a project description of the program/project you want to create. After you provide a project description, the AI system will ask further questions to gather more information about your project. Your responses will guide the AI in providing the best assistance for your project.
41 |
42 | **Features:**
43 | - `Setup a Coding Project`: ProgramEngineerGPT can help you set up a new coding project, including planning the project structure and setting up the development environment.
44 | - `Gathers Program Requirements and Architecture`: It can ask you questions about your project to gather more information and understand your project requirements and desired architecture.
45 | - `Creates a Project Directory Structure`: Based on your project requirements, ProgramEngineerGPT can create a suitable directory structure for your project.
46 | - `Generates Initial Code`: It can generate initial code for your project based on the gathered requirements and architecture.
47 | - `Generates Unit Testing`: ProgramEngineerGPT can generate unit tests for your code to ensure its correctness and robustness.
48 | - `Generates Code Documentation`: It can generate documentation for your code base, making it easier for others to understand and contribute to your project.
49 |
50 | ## Getting Started
51 | These instructions will get you a copy of the project up and running for development and testing purposes.
52 |
53 | ### Prerequisites
54 | - [OpenAI Account](https://openai.com/)
55 |
56 | ### Known Issues
57 | ChromaDB Fails to Install on MacOS:
58 |
59 | ```
60 | # Failed Command
61 | pip install chromadb
62 | ```
63 |
64 | To address this issue, please run the following command:
65 |
66 | ```
67 | export HNSWLIB_NO_NATIVE=1
68 | ```
69 |
70 | ### Setup
71 | Instructions on how to get ProgramEngineerGPT configured locally.
72 |
73 | Before running, it is important that you have the correct environmental variables set.
74 | Setup required Environmental Variables. You can either change the `test.env` to `.env` and add the required environmental variables.
75 |
76 | If you would like to export them locally, please use the following keys.
77 |
78 | Linux or MacOS
79 | ```bash
80 | # OpenAI API
81 | export OPENAI_API_KEY=""
82 | ```
83 | Windows
84 |
85 | ```bash
86 | # OpenAI API
87 | setx OPENAI_API_KEY
88 | ```
89 |
90 | Clone the repository
91 | ```bash
92 | #Download Repo and Navigate to Directory
93 | git clone https://github.com/hackedbyagirl/program-engineer-gpt.git
94 | cd program-engineer-gpt
95 | ```
96 | Install all the required packages
97 | ```
98 | python3 -m pip install -r requirements.txt
99 | ```
100 |
101 | ## Usage
102 | You can start using ProgramEngineerGPT by running the main script and selecting the mode of operation. Depending on the mode, you will be asked to provide further details such as the code repository URL or the project description.
103 |
104 | However, this program does depend on API keys so make sure to set them!
105 |
106 | ```bash
107 | # Python program
108 | python3 programengineergpt.py
109 | ```
110 |
111 | ## Examples
112 | ### Program Launch
113 | Main Program Home Screen
114 |
115 | 
116 |
117 | ### Analyze Mode
118 | Loading Code of Current Working Directory
119 |
120 | 
121 |
122 | Engaging in conversation about the code
123 |
124 | 
125 |
126 | ### Develop Mode
127 | Providing Developer Mode with a Project Description
128 |
129 | 
130 |
131 | Engaging with Developer AI Assistant
132 |
133 | 
134 |
135 | ## Contributing
136 |
137 | Contributions are welcome! Please refer to the contributing guide provided in the repository.
138 |
139 | ## License
140 |
141 | Please refer to the license file provided in the repository.
142 |
143 | ## Acknowledgements
144 | Inspiration
145 | - [engineer-gpt](https://github.com/AntonOsika/gpt-engineer)
146 | - [PDF Chatbot](https://github.com/mayooear/gpt4-pdf-chatbot-langchain)
147 |
--------------------------------------------------------------------------------
/docs/index.md:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/docs/index.md
--------------------------------------------------------------------------------
/docs/installation.md:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/docs/installation.md
--------------------------------------------------------------------------------
/docs/usage.md:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/docs/usage.md
--------------------------------------------------------------------------------
/imgs/chat.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/imgs/chat.jpg
--------------------------------------------------------------------------------
/imgs/cwd.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/imgs/cwd.jpg
--------------------------------------------------------------------------------
/imgs/dev-gen.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/imgs/dev-gen.jpg
--------------------------------------------------------------------------------
/imgs/dev-launch.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/imgs/dev-launch.jpg
--------------------------------------------------------------------------------
/imgs/kat.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/imgs/kat.png
--------------------------------------------------------------------------------
/imgs/pgpt.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/imgs/pgpt.jpg
--------------------------------------------------------------------------------
/programengineergpt.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | from programengineergpt import __main__
4 |
5 | __main__.run()
6 |
--------------------------------------------------------------------------------
/programengineergpt/__init__.py:
--------------------------------------------------------------------------------
1 | """Top-level package for ProgramEngineerGPT."""
2 | __app_name__ = "programengineergpt"
3 | __author__ = """Brette Geary"""
4 | __email__ = "hackedbg@outlook.com"
5 | __version__ = "0.6.9"
6 |
--------------------------------------------------------------------------------
/programengineergpt/__main__.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # imports
4 | from programengineergpt.main import run
5 |
6 | ########################################################################
7 |
8 | if __name__ == "__main__":
9 | run()
10 |
--------------------------------------------------------------------------------
/programengineergpt/agents/__init__.py:
--------------------------------------------------------------------------------
1 | from programengineergpt.agents.actions import Actions
2 | from programengineergpt.agents.ai import AIAgent
3 |
4 | __all__ = ["AIAgent", "Actions"]
5 |
--------------------------------------------------------------------------------
/programengineergpt/agents/actions.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 | import questionary
5 |
6 | from programengineergpt.utils.colors import Color
7 |
8 |
9 | class Actions:
10 | """AI Actions"""
11 |
12 | def __init__(self, ai_agent):
13 | self.ai_agent = ai_agent
14 |
15 | def clarify(self, messages):
16 | """Answer clarification questions"""
17 | while True:
18 | # Ask the user if they want to clarify anything
19 | print("\n")
20 | user_clarification = questionary.rawselect(
21 | "How would you like to proceed:",
22 | choices=["Answer Question", "Skip Question", "End Question Answering"],
23 | ).ask()
24 |
25 | # Check if the user wants to move on
26 | if user_clarification == "End Question Answering":
27 | print("\n")
28 | break
29 |
30 | # Get the user's clarification
31 | if user_clarification == "Skip Question":
32 | clarification = "Skip"
33 | print("\n")
34 |
35 | # Get the user's clarification
36 | else:
37 | Color.print("\n{B}Please provide answer clarification:")
38 | clarification = input()
39 |
40 | # Pass the user's clarification to the AI for further processing
41 | messages = self.ai_agent.next_step(messages, clarification)
42 |
43 | return messages
44 |
45 | def gen_design(self, system_prompt, messages):
46 | """Launch Project Design Phase"""
47 | return self.ai_agent.next_step(messages, system_prompt)
48 |
49 | def gen_code_structure(self, system_prompt, messages):
50 | """Launch Project Code Development Phase"""
51 | return self.ai_agent.next_step(messages, system_prompt)
52 |
53 | def gen_init_code(self, system_prompt, messages):
54 | """Generate Initial Code"""
55 | return self.ai_agent.next_step(messages, system_prompt)
56 |
57 | def gen_unit_tests(self, system_prompt, messages):
58 | """Generate Unit Tests"""
59 | return self.ai_agent.next_step(messages, system_prompt)
60 |
61 | def gen_doumentaion(self, system_prompt, messages):
62 | """Generate Code Documentation"""
63 | return self.ai_agent.next_step(messages, system_prompt)
64 |
--------------------------------------------------------------------------------
/programengineergpt/agents/ai.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 |
5 | import openai
6 |
7 | from programengineergpt.config import Config
8 |
9 |
10 | class AIAgent:
11 | def __init__(self):
12 | self.model = Config.model
13 | self.temperature = 0.1
14 |
15 | try:
16 | openai.Model.retrieve(self.model)
17 | except openai.InvalidRequestError:
18 | print(
19 | f"Model {self.model} not available for provided API key. Please Update your configuration file"
20 | )
21 |
22 | def launch(self, system_prompt, user_prompt):
23 | # Create a list of two dictionaries representing a system role and a user role
24 | msg = [
25 | {"role": "system", "content": system_prompt},
26 | {"role": "user", "content": user_prompt},
27 | ]
28 |
29 | # Send the message to the next method
30 | return self.next_step(msg)
31 |
32 | def next_step(self, messages: list[dict[str, str]], user_prompt=None):
33 | # Add user prompt to messages if provided
34 | if user_prompt:
35 | messages.append({"role": "user", "content": user_prompt})
36 |
37 | response = openai.ChatCompletion.create(
38 | messages=messages,
39 | stream=True,
40 | model=self.model,
41 | temperature=self.temperature,
42 | )
43 |
44 | chat = []
45 | for chunk in response:
46 | delta = chunk["choices"][0]["delta"]
47 | msg = delta.get("content", "")
48 | print(msg, end="")
49 | chat.append(msg)
50 | print()
51 |
52 | messages.append({"role": "assistant", "content": "".join(chat)})
53 |
54 | return messages
55 |
56 | def gen_system_prompt(self, msg):
57 | return {"role": "system", "content": msg}
58 |
59 | def gen_user_prompt(self, msg):
60 | return {"role": "user", "content": msg}
61 |
62 | def gen_assistant_prompt(self, msg):
63 | return {"role": "assistant", "content": msg}
64 |
--------------------------------------------------------------------------------
/programengineergpt/agents/chatbot.py:
--------------------------------------------------------------------------------
1 | import chromadb
2 | from chromadb.config import Settings
3 | import datetime
4 | import json
5 | import openai
6 | import time
7 |
8 | from time import sleep
9 |
10 | from programengineergpt.utils.colors import Color
11 | from programengineergpt.utils.display import Display
12 | from programengineergpt.prompts.build_user_prompt import build_user_prompt
13 | from programengineergpt.prompts.chat import SYSTEM_CHAT_PROMPT
14 |
15 |
16 | class ChatBot:
17 | def __init__(self, code_collection):
18 | self.code_collection = code_collection
19 | self.openai_api = openai.ChatCompletion()
20 | self.chat_history = []
21 | self.model = "gpt-3.5-turbo-16k"
22 | self.temperature = 0.1
23 | self.chat_file = f"chat_log_{time.time()}.json"
24 |
25 | def launch(self):
26 | # Display banner
27 | Display.display_interactive_chat_banner()
28 |
29 |
30 | # Set initial system prompt
31 | self.chat_history.append({"role": "system", "content": SYSTEM_CHAT_PROMPT})
32 |
33 | # Get First Question and context
34 | Color.print("\n{G}Question: ")
35 | question = input()
36 | context = self.retrieve_context(question)
37 |
38 | # Build first prompt
39 | user_prompt = build_user_prompt(question, context)
40 | self.chat_history.append(user_prompt)
41 |
42 | # Launch first question
43 | self.next_step(self.chat_history)
44 |
45 | def interact(self):
46 | """
47 | Queries an index to allow conversational QA
48 | """
49 | # Launch initial ai interaction
50 | self.launch()
51 |
52 | # Engage in Interactive chat loop
53 | while True:
54 | # Get question
55 | Color.print("\n{G}Question: ")
56 | question = input()
57 | if question.lower() == "exit":
58 | self.save_chat_log()
59 | break
60 |
61 | # Build user prompt
62 | context = self.retrieve_context(question)
63 | user_prompt = build_user_prompt(question, context)
64 | self.chat_history.append(user_prompt)
65 |
66 | # continue chat
67 | self.next_step(self.chat_history)
68 |
69 | def retrieve_context(self, query):
70 | results = self.code_collection.query(
71 | query_texts=[query], include=["documents", "metadatas"], n_results=3
72 | )
73 | if results:
74 | return [
75 | (doc, meta["filename"])
76 | for doc, meta in zip(results["documents"][0], results["metadatas"][0])
77 | ]
78 | else:
79 | return None
80 |
81 | def next_step(self, messages):
82 | max_retry = 7
83 | retry = 0
84 | while True:
85 | try:
86 | response = self.openai_api.create(
87 | model=self.model,
88 | messages=messages,
89 | stream=True,
90 | temperature=self.temperature,
91 | )
92 |
93 | Color.print("\n{B}Answer: \n")
94 | chat = []
95 | for chunk in response:
96 | delta = chunk["choices"][0]["delta"]
97 | msg = delta.get("content", "")
98 | print(msg, end="")
99 | chat.append(msg)
100 | print()
101 |
102 | chat_response = {"role": "assistant", "content": "".join(chat)}
103 | self.chat_history.append(chat_response)
104 |
105 | break
106 |
107 | except Exception as oops:
108 | print(f'\n\nError communicating with OpenAI: "{oops}"')
109 | if "maximum context length" in str(oops):
110 | self.chat_history.pop(0)
111 | print("\n\n DEBUG: Trimming oldest message")
112 | continue
113 | retry += 1
114 | if retry >= max_retry:
115 | print(f"\n\nExiting due to excessive errors in API: {oops}")
116 | exit(1)
117 | print(f"\n\nRetrying in {2 ** (retry - 1) * 5} seconds...")
118 | sleep(2 ** (retry - 1) * 5)
119 |
120 | def save_chat_log(self):
121 | with open(self.chat_file, 'w') as f:
122 | json.dump(self.chat_history, f)
--------------------------------------------------------------------------------
/programengineergpt/config.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 | import os
5 |
6 | from dotenv import load_dotenv
7 |
8 | from programengineergpt.utils.args import Arguments
9 |
10 |
11 | class Config(object):
12 | """Stores configuration variables and functions for CodeAssistantGPT"""
13 |
14 | initialized = False
15 | verbose = 0
16 |
17 | @classmethod
18 | def init(cls):
19 | """
20 | Sets up default initial configuration values.
21 | Also sets config values based on command-line arguments.
22 |
23 | Args:
24 | cls : Argument
25 |
26 | """
27 |
28 | if cls.initialized:
29 | return
30 | cls.initialized = True
31 |
32 | cls.verbose = 0
33 |
34 | # Environmental Variable
35 | cls.openai_key = None
36 |
37 | # Arguments
38 | cls.model = "gpt-3.5-turbo-16k" # gpt-4 if you have the access
39 | cls.chunk_size = 1000
40 | cls.chunk_overlap = 0
41 | cls.embedded_size = 1536
42 |
43 | # Will overwrite provided variables above
44 | cls.load_args()
45 | cls.load_env()
46 |
47 | ############################################
48 |
49 | @classmethod
50 | def load_args(cls):
51 | """Sets configuration values based on Argument.args object"""
52 |
53 | args = Arguments(cls).args
54 |
55 | # get arguments
56 | cls.chunk_size = args.chunk_size if args.chunk_size else cls.chunk_size
57 | cls.embedded_size = (
58 | args.embedding_size if args.embedding_size else cls.embedded_size
59 | )
60 | cls.model = args.model if args.model else cls.model
61 |
62 | @classmethod
63 | def load_env(cls):
64 | """Gets Environmental Variables"""
65 |
66 | # Get Environmental Variable
67 | load_dotenv()
68 | cls.openai_key = os.getenv("OPENAI_API_KEY")
69 |
70 | @classmethod
71 | def exit(cls, code=0):
72 | """Exit Program"""
73 | print("Stopping program")
74 |
75 | exit(code)
76 |
77 |
78 | ###############################################################
79 |
80 | if __name__ == "__main__":
81 | Config.init()
82 |
--------------------------------------------------------------------------------
/programengineergpt/core/__init__.py:
--------------------------------------------------------------------------------
1 | from programengineergpt.core.developer import Developer
2 | from programengineergpt.core.loader import CodeLoader
3 | from programengineergpt.core.retriever import Retriever
4 |
5 | __all__ = ["Developer", "CodeLoader", "Retriever"]
6 |
--------------------------------------------------------------------------------
/programengineergpt/core/developer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 | import os
5 |
6 | from programengineergpt.agents.actions import Actions
7 | from programengineergpt.agents.ai import AIAgent
8 | from programengineergpt.utils.colors import Color
9 | from programengineergpt.utils.display import Display
10 |
11 |
12 | class Developer:
13 | def __init__(self, project_description, project_folder):
14 | self.project_description = project_description
15 | self.project_folder = project_folder
16 |
17 | # Create an empty folder for the project
18 | if not os.path.exists(project_folder):
19 | os.makedirs(project_folder)
20 |
21 | # Save project descrition
22 | self.write_output()
23 |
24 | # Initialize an AI agent
25 | self.ai_agent = AIAgent()
26 | self.actions = Actions(self.ai_agent)
27 |
28 | self.start()
29 |
30 | def start(self):
31 | # Display
32 | Display.display_interactive_developer_banner()
33 |
34 | # Initialize Actions class
35 | Color.print("\n{L}Analyzing your project...\n")
36 |
37 | reqs = self.process_project_requirements()
38 | design = self.process_project_design(reqs)
39 | code_structure = self.process_code_structure(design)
40 | initial_code = self.write_initial_code(code_structure)
41 | unit_tests = self.write_unit_tests(initial_code)
42 | self.write_code_documentation(unit_tests)
43 |
44 | def process_project_requirements(self):
45 | from programengineergpt.prompts.project_reqs import PROJECT_REQS
46 |
47 | system_prompt = PROJECT_REQS
48 | user_prompt = self.project_description
49 | messages = self.ai_agent.launch(system_prompt, user_prompt)
50 | return self.actions.clarify(messages)
51 |
52 | def process_project_design(self, requirements):
53 | from programengineergpt.prompts.project_design import PROJECT_DESIGN
54 |
55 | Display.clear_screen()
56 | system_prompt = PROJECT_DESIGN
57 | messages = self.actions.gen_design(system_prompt, requirements)
58 | return self.actions.clarify(messages)
59 |
60 | def process_code_structure(self, design):
61 | from programengineergpt.prompts.code_design import CODE_DESIGN
62 |
63 | Display.clear_screen()
64 | system_prompt = CODE_DESIGN
65 | return self.actions.gen_code_structure(system_prompt, design)
66 |
67 | def write_initial_code(self, structure):
68 | from programengineergpt.prompts.code_writer import CODE_WRITER
69 |
70 | system_prompt = CODE_WRITER
71 | return self.actions.gen_code_structure(system_prompt, structure)
72 |
73 | def write_unit_tests(self, code):
74 | from programengineergpt.prompts.code_tests import UNIT_TEST_GENERATOR
75 |
76 | system_prompt = UNIT_TEST_GENERATOR
77 | return self.actions.gen_unit_tests(system_prompt, code)
78 |
79 | def write_code_documentation(self, code):
80 | from programengineergpt.prompts.project_documentation import (
81 | DOCUMENTATION_WRITER,
82 | )
83 |
84 | system_prompt = DOCUMENTATION_WRITER
85 | return self.actions.gen_doumentaion(system_prompt, code)
86 |
87 | def write_output(self):
88 | # Save the user project input into the project folder directory
89 | with open(
90 | os.path.join(self.project_folder, "project_instructions.txt"), "w"
91 | ) as f:
92 | f.write(self.project_description)
93 |
--------------------------------------------------------------------------------
/programengineergpt/core/loader.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | import os
4 | import chromadb
5 | import fnmatch
6 |
7 | from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
8 |
9 | from programengineergpt.utils.colors import Color
10 | from programengineergpt.utils.file_processing import extensions_include, extensions_ignore, dirs_ignore
11 |
12 |
13 | class CodeLoader:
14 | def __init__(self):
15 | self.embedding_function = OpenAIEmbeddingFunction()
16 | self.chromadb_client = chromadb.Client()
17 |
18 | self.code_files = []
19 | self.files = []
20 | self.chunks = []
21 | self.extensions = extensions_include
22 | self.extensions_ignore = extensions_ignore
23 | self.directories_ignore = dirs_ignore
24 |
25 | Color.print("\n\n{G}Launching Code Loader...\n")
26 |
27 | def load_online_repo(self, url):
28 | """
29 | Load all code files from the specified repository.
30 | """
31 | if "github.com/" in url or "gitlab.com/" in url:
32 | # Clone repository
33 | try:
34 | Color.print("{G}Step 1: {W}Retrieving Code from Online Repository")
35 | os.system(f"git clone --quiet {url} temp_repo")
36 | vector_store = self.load("temp_repo")
37 |
38 | except Exception as e:
39 | Color.print(
40 | "{R}!!!: {W}Failed to clone GitHub repository from {Y}" + url
41 | )
42 | Color.p_error(e)
43 |
44 | return vector_store
45 |
46 | def load_directory(self, path):
47 | """
48 | Load all code files from a specified directory.
49 | """
50 | # Get code from provided directory
51 | Color.print("{G}Step 1: {W}Retrieving Code from Local Repository")
52 | if not os.path.isdir(path):
53 | raise Exception(f"Invalid local directory: {path}")
54 |
55 | vector_store = self.load(path)
56 | return vector_store
57 |
58 | def load_current_directory(self):
59 | """
60 | Load all code files from the current directory.
61 | """
62 | Color.print("{G}Step 1: {W}Retrieving Code from Current Directory")
63 | vector_store = self.load(os.getcwd())
64 | return vector_store
65 |
66 | def load(self, root_dir, col_name=None):
67 | """
68 | Load all code files from the root directory.
69 | """
70 | try:
71 | Color.print("{G}Step 2: {W}Loading Code for Indexing and Embedding")
72 | for dirpath, dirnames, filenames in os.walk(root_dir):
73 | # Modifying dirnames list will update os.walk behavior
74 | dirnames[:] = [d for d in dirnames if d not in self.directories_ignore]
75 | for file in filenames:
76 | # Check if file matches any ignore pattern and skip loading
77 | if any(fnmatch.fnmatch(file, pattern) for pattern in self.extensions_ignore):
78 | continue # Skip this file
79 |
80 | if file.split(".")[-1] in self.extensions:
81 | with open(os.path.join(dirpath, file), 'r') as f:
82 | self.files.append((file, f.read()))
83 |
84 | except Exception as e:
85 | Color.print("{R}!!!: {W}Failed to load code files from {Y}" + root_dir)
86 | Color.p_error(e)
87 |
88 | # print loaded files
89 | len_files = str(len(self.files))
90 | Color.print("{Y}Number of Loaded Files: {W}" + len_files)
91 |
92 | # Split code
93 | Color.print("{G}Step 3: {W}Splitting and Chunking Files")
94 | self.chunks = self.split_code()
95 |
96 | # embed code
97 | Color.print("{G}Step 4: {W}Embedding and Uploading to ChromaDB")
98 |
99 | if col_name is None:
100 | Color.print("\n{Y}Please enter a Collection Name for you Code Base: ")
101 | col_name = input("Collection Name: ")
102 |
103 | code_collection = self.chromadb_client.create_collection(
104 | name=col_name, embedding_function=self.embedding_function
105 | )
106 |
107 | counter = 0
108 | batch_size = 100
109 | for i in range(0, len(self.chunks), batch_size):
110 | batch = self.chunks[i : i + batch_size]
111 | documents = []
112 | ids = []
113 | metadatas = []
114 | for full_path, chunks in batch:
115 | for j, chunk in enumerate(chunks):
116 | documents.append(chunk)
117 | # Use the counter to create unique IDs
118 | ids.append(f"{full_path}_{counter}")
119 | metadatas.append({"filename": full_path})
120 | # Increment the counter
121 | counter += 1
122 | code_collection.add(ids=ids, documents=documents, metadatas=metadatas)
123 |
124 | if os.path.exists("temp_repo"):
125 | Color.print('{G}Step 5: {W}Removing "temp_repo" contents')
126 | os.system("rm -rf temp_repo")
127 |
128 | return code_collection
129 |
130 | def split_code(self):
131 | """Split the code of all files into chunks."""
132 | all_chunks = []
133 | for filename, code in self.files:
134 | chunks = [code[i : i + 1000] for i in range(0, len(code), 1000)]
135 | all_chunks.append((filename, chunks))
136 | return all_chunks
137 |
138 | def remove_temp_dir(self):
139 | """
140 | Remove 'temp_repo' if a repository was cloned
141 | """
142 | if os.path.exists("temp_repo"):
143 | Color.print('{G}Step 5: {W}Removing "temp_repo" contents')
144 | os.system("rm -rf temp_repo")
145 |
--------------------------------------------------------------------------------
/programengineergpt/core/retriever.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 | import chromadb
5 | from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
6 |
7 | from programengineergpt.utils.colors import Color
8 |
9 |
10 | class Retriever:
11 | def __init__(self, col_name=None):
12 | if col_name is None:
13 | Color.print("\n{Y}Please enter collection name for code base: ")
14 | self.collection_name = col_name = input("Collection Name: ")
15 |
16 | self.collection_name = col_name
17 | self.embedding_function = OpenAIEmbeddingFunction()
18 | self.chromadb_client = chromadb.Client()
19 | self.collection = self.chromadb_client.get_collection(
20 | name=self.collection_name, embedding_function=self.embedding_function
21 | )
22 |
23 | Color.print("\n\n{G}Retrieving Code Index...\n")
24 |
25 | def retrive_code_index(self):
26 | return self.collection
27 |
--------------------------------------------------------------------------------
/programengineergpt/main.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # imports
4 | from programengineergpt.program_engineer import ProgramEngineerGPT
5 |
6 |
7 | ########################################################################
8 | def run():
9 | """
10 | Launches App
11 | """
12 | pgpt = ProgramEngineerGPT()
13 | pgpt.launch()
14 |
--------------------------------------------------------------------------------
/programengineergpt/program_engineer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 |
3 | # imports
4 | from programengineergpt.config import Config
5 | from programengineergpt.utils.display import Display
6 |
7 |
8 | class ProgramEngineerGPT(object):
9 | """
10 | An AI-powered tool designed to assist with a variety of coding tasks.
11 | This includes:
12 | - Understanding the structure, dependencies, and other details of a codebase.
13 | - Getting assistance in setting up a new coding project, including planning and setup.
14 | - Having the AI generate code snippets based on your requirements.
15 | - Getting help in debugging your code and suggestions for improvements.
16 | """
17 |
18 | def __init__(self):
19 | """
20 | Main Entry point for the code
21 |
22 | Args:
23 | self : Argument
24 |
25 | """
26 | Display.display_banner()
27 | Display.display_main_description()
28 |
29 | # Initialize Config
30 | Config.init()
31 |
32 | def launch(self):
33 | """
34 | Launches the main interactive interface.
35 |
36 | Args:
37 | self : Argument
38 |
39 | """
40 | from programengineergpt.utils.cli import CLI
41 |
42 | app = CLI()
43 | app.launc_main_cli()
44 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/build_user_prompt.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 |
4 | def build_user_prompt(question, references):
5 | context = "\n".join([f"From file: {ref[0]} Data: {ref[1]}" for ref in references])
6 | return {
7 | "role": "user",
8 | "content": f"""Context:\n{context}\n\nAssess the following context to answer the users question.\n\nQuestion:\n{question}""",
9 | }
10 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/chat.py:
--------------------------------------------------------------------------------
1 | SYSTEM_CHAT_PROMPT = """
2 | As an AI Assistant, you are entering into a code discussion environment. Your task is to assist the user in understanding a provided codebase. This may involve explaining various components of the code, describing their functions, and outlining how they work together.
3 |
4 | The user might need help understanding programming concepts, the use of specific libraries, or interpreting certain parts of the codebase. It's your responsibility to make these aspects as clear as possible to help them understand the codebase better.
5 |
6 | You will be provided with the context and history of the code discussion, which will help you understand previous conversations and the specific details of the codebase. You'll use this information to deliver accurate and helpful responses.
7 |
8 | Your role requires active engagement in the conversation, answering the user's queries accurately and providing further information or clarification when necessary. Always be ready to dive deep into the codebase to extract the required information.
9 |
10 | Remember, your ultimate goal is to ensure that the user gains a clear understanding of the codebase and feels confident in working with it. Your assistance should empower the user to navigate and modify the codebase effectively.
11 |
12 | Let's begin by understanding the context and history of the codebase in question.
13 | """
14 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/code_design.py:
--------------------------------------------------------------------------------
1 | # Project Code Design
2 |
3 | CODE_DESIGN = """
4 | As an AI Assistant, you're now embarking on the code design phase of the user's new coding project. Having gathered essential information about the project description, system requirements, and architecture, your role now is to devise a rudimentary layout of the project.
5 |
6 | First, you will think step by step and reason yourself the right decisions of building this project. This includes taking into considerations the primary classes, functions, and methods required for the system to function effectively. Moreover, you must take into account the structure and hierarchy of these components within the codebase to ensure the project is setup efficiently. You will follow best practices for the requested languages and frameworks in terms of naming conventions and describing the code written.
7 |
8 | Your goal is to provide the user with a clear and concise roadmap for implementing their project.
9 | First you will provide the user will a directory structure layout aiding them in visualizing how their codebase will be structured.
10 | Then, you will provide the user with a comprehensive description of each file in the project, outlining what it represents and its role in the systems overall operation.
11 |
12 | The following tokens must be replaced as follows.
13 |
14 | DIR_STRUCTURE: A markdown codeblock representing the directory structure layout
15 | F_DESCRIPTIONS: A comprehensive description of each file in the project
16 |
17 | Expected Output:
18 | Based on the project description, system requirements, and architecture, the initial layout of your Python project should be setup as followed:
19 |
20 | Directory Structure:
21 | DIR_STRUCTURE
22 |
23 | File Descriptions:
24 | F_DESCRIPTIONS
25 | """
26 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/code_tests.py:
--------------------------------------------------------------------------------
1 | # Code Unit Testing
2 |
3 | UNIT_TEST_GENERATOR = """
4 | As an AI Assistant, you're now entering the unit test generation phase of the user's new coding project. Given the completed codebase, your task is to create unit tests that ensure each component of the system functions as expected.
5 |
6 | Begin by identifying the core functionalities, methods, and classes within the code that need to be tested. These are usually components that carry out critical operations and form the backbone of your application.
7 |
8 | Following that, you will design and generate unit tests for these identified components. Your unit tests should cover a wide range of scenarios, including edge cases, to ensure that your code is robust and can handle unexpected inputs or situations.
9 |
10 | Remember, the purpose of these tests is to validate that each individual unit of the software performs as designed.
11 |
12 | The following tokens must be replaced as follows:
13 |
14 | TEST_FILENAME: The filename for the test file, all lowercase, including the file extension.
15 | LANG: The markup code block language for the code's language.
16 | TEST_CODE: The actual unit test code written for each test file.
17 |
18 | Expected output:
19 |
20 | TEST_FILENAME
21 | ```LANG
22 | TEST_CODE
23 | ```
24 |
25 | Ensure each test is well-documented, fully functional, and follows the best practices for the language and framework being used. Also, ensure the tests are isolated, meaning the result of one test should not affect the result of another.
26 |
27 | After writing the tests, double-check that all key parts of the software are tested and that the tests accurately validate the functionality of the code under a variety of conditions.
28 | """
29 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/code_writer.py:
--------------------------------------------------------------------------------
1 | # Project Code Generation
2 |
3 | CODE_WRITER = """
4 | As an AI Assistant, you're now begining the vital phase of code generation for the user's new coding project. Given detailed information about the project description, system requirements, high-level architecture, and code design, your task now is to write the initial codebase.
5 |
6 | You will begin this process by mentally traversing through the logical steps of the project. You will make strategic coding decisions based on previously gathered information. This includes identifying and implimenting the core classes, functions, and methods necessary for the system's efficient operation. Keep the structure and hierarchy of these components within the codebase in mind to ensure maintainability and scalability.
7 |
8 | Based on the outlined project structure, begin by developing the initial scripts. You will start the main "entrypoint" file, and then move on to the files it imports, and so on, ensuring each file fulfills its designated role.
9 |
10 | While writing the project's code, remember the following rules:
11 | - Output the content of each file, including ALL code, using a strict markdown code block format.
12 | - Ensure all files are compatible with each other, including necessary imports, types, etc.
13 | - Adhere to best practices for the specified languages and frameworks. This includes writing clean, efficient, well-documented code, using proper naming conventions, and adding descriptive comments.
14 | - Ensure the code is fully functional, with no placeholders.
15 |
16 | Replace the following tokens as follows:
17 |
18 | FILENAME: The filename, all lowercase, including the file extension
19 | LANG: The markup code block language for the code's language
20 | CODE: The actual code to be written
21 |
22 | Expected output:
23 |
24 | FILENAME
25 | ```LANG
26 | CODE
27 | ```
28 |
29 | Before you finish, double check that all parts of the architecture are present in the files to ensure that the user has a solid foundation to start their new project.
30 | """
31 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/intro.py:
--------------------------------------------------------------------------------
1 | # Intro Prompt
2 |
3 | INTRO_SYSTEM_PROMPT = """
4 | You are an AI Assistant that will assist the user through the process of setting up a new coding project, from planning the inital phase of designing the project structure to the final stage of setting up the development environment. The user will provide you will provide you with instructions related to the project they wish to develop.
5 |
6 | You will read the instructions provided by the user and ask further questions to gather more information about the project. First, you will provide the user with a short, numbered list of areas that require further clarification. Then you will instruct the user of which question you would like them to answer first, and then wait for a response from the user.
7 |
8 | Example Input:
9 | A python program that takes user input and then prints it out
10 |
11 | Example Output:
12 | Summary: Great! I can help you set up a Python program that takes user input and prints it out. Before we begin, I have a few questions to clarify the details of your project. Please answer the following questions:
13 | Questions:
14 | 1. Do you have a specific name in mind for your Python program?
15 | 2. Should the program prompt the user for input, or should it accept input as command-line arguments?
16 | 3. Should the program print the user input as is, or do you want any specific formatting or modifications to be applied to the input before printing?
17 |
18 | Please answer these questions in order, starting with question 1.
19 | 1. Do you have a specific name in mind for your Python program?
20 | """
21 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/project_design.py:
--------------------------------------------------------------------------------
1 | # Project Setup Prompt
2 |
3 | PROJECT_DESIGN = """
4 | You will now begin guiding the user in defining the system architecture and creating a high-level design for their new coding project. Based on the description and requirements of the project, your responsibility is to ask insightful questions that help you understand the potential architecture of their system and its high-level design. Consider aspects like the software requirements, the scalability needs, and the interaction between different components of the system.
5 |
6 | Afterward, you'll provide a brief, numbered list identifying areas that require further clarification to accurately define the system's architecture and its high-level design.
7 | Subsequently, you will guide the user to address the first question from the list and await their response before moving on. If the user does not know the answer to a question, they will respond with skip and you will move on to the next question.
8 | Ensure to keep the conversation structured and well-organized, aiding the user through the complex process of system architecture and high-level design. If the user has answered all the questions you have provided, you will provide further questions. The user will tell you when they are done answering questions.
9 |
10 |
11 | Example Output:
12 | Summary: Based on the description and requirements your Python program, we'll need to determine the system architecture and create a high-level design. To do this, I'll need your input on a few crucial aspects. Consider aspects like software requirements, scalability needs, and the interactions between different components of the system. Here is a numbered list of questions which I'd like you to consider:
13 |
14 | Questions:
15 | 1. What is the anticipated scale of your Python program? Will it be a simple script, or are we looking at a larger system with several interacting components?
16 | 2. What are the main components or modules of the system and how do you envisage them interacting with each other?
17 | 3. How should your program handle data persistence? Are we considering the use of a database, and if so, what type (SQL, NoSQL, in-memory, etc.) would be most appropriate?
18 | 4. What kind of user interface, if any, does the system have and how does it interact with the other system components?
19 | 5. What are the performance requirements of your Python program? Understanding the expected load, response times, and scalability needs could significantly influence the architecture and design.
20 |
21 | Could you please provide answers to these questions in sequence, beginning with question 1? If you do not have an answer to a specific question, please respond with 'skip', and we'll proceed to the next question. This information will significantly aid us in defining the architecture and high-level design of your project.
22 |
23 | 1. What is the anticipated scale of your Python program? Will it be a simple script, or are we looking at a larger system with several interacting components?
24 | """
25 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/project_documentation.py:
--------------------------------------------------------------------------------
1 | DOCUMENTATION_WRITER = """
2 | As an AI Assistant, you are now in the project documentation phase of the user's new coding project. Leveraging all the information about the project, your task is to create a comprehensive README file that outlines all the necessary details of the project. Keep in mind that the README is often the first point of contact for other developers and users, so it should be clear, concise, and informative.
3 |
4 | You will provide the README file in a markdown code block.
5 |
6 | The following tokens must be replaced as follows:
7 | FILENAME: README.md
8 | PROJ_TITLE: The name of the Project
9 | ABOUT: A Brief description of the project, its purpose, and its functionality.
10 | REQS: Guide users on how to setup and install the project, including the necessary software, tools, and libraries required.
11 | USAGE: Explain how to run the project, providing examples where necessary.
12 | TESTING: Provide instructions on how to run the unit tests.
13 |
14 |
15 | Expected output:
16 |
17 | FILENAME
18 | ```
19 | # PROJ_TITLE
20 |
21 | # About
22 | ABOUT
23 |
24 | # Setup and Installation
25 | REQS
26 |
27 | # Usage
28 | USAGE
29 |
30 | # Testing
31 | TESTING
32 | ```
33 |
34 | Ensure the README is well-structured and follows markdown formatting for readability. Use bullet points, headers, and subheaders appropriately. Before finalizing, double-check that the instructions are clear, accurate, and comprehensive, making it easy for users and other developers to understand and use the project.
35 | """
36 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/project_reqs.py:
--------------------------------------------------------------------------------
1 | # Project Requirements
2 |
3 | PROJECT_REQS = """
4 | You are an AI Assistant that will assist the user through the process of setting up a new coding project, from planning the inital phase of designing the project structure to the final stage of setting up the development environment. To begin, the user will provide you will provide you with instructions related to the project they wish to develop.
5 |
6 | Your first task is to read and understand the instructions provided by the user and then ask furhter questions to gain deeper insights into the project's specifics. Initially, you'll provide a brief, numbered list identifying areas that require further clarification. Subsequently, you will guide the user to address the first question from the list and await their response before moving on. If the user does not know the answer to a question, they will respond with `Skip` and you will move on to the next question. If the user has answered all the questions you have provided, you will provide further questions. The user will tell you when they are done answering questions.
7 |
8 | Example Input:
9 | A python program that takes user input and then prints it out
10 |
11 | Example Output:
12 | Summary: Excellent! I am here to assist you in setting up a Python program that captures user input and subsequently displays it. To get started and ensure the project aligns with your expectations, I'll need to gather some detailed information. Please provide responses to the following questions:
13 |
14 | Questions:
15 | 1. Who are the intended users of this program?
16 | 2. Aside from Python, are there any specific technologies or tools you'd like to utilize in this program?
17 | 3. Are there any specific performance criteria your program needs to meet?
18 | 4. Are there any specific security requirements or standards this program needs to adhere to?
19 | 5. Do you have a specific name in mind for your Python program?
20 |
21 | Kindly provide your responses in sequence, starting with the first question. If you do not know the answer to a question, please respond with skip and move on to the next question.
22 |
23 | 1. Who are the intended users of this program?
24 | """
25 |
--------------------------------------------------------------------------------
/programengineergpt/prompts/setup_project.py:
--------------------------------------------------------------------------------
1 | # Project Setup Prompt
2 |
3 | PROJECT_SETUP = """
4 | You will now begin code generation for the project. Based on the description of the project, you will think step by step and reason yourself to the right decisions of building this program to get it right. You will follow best practices for the requested languages in terms of describing the code written as a defined package/project.
5 |
6 | You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose.
7 | Then you will output the content of each file including ALL code. Each file must strictly follow a markdown code block format.
8 | The following tokens must be replaced as follows.
9 |
10 | FILENAME will be the filename in all lowercasem including the file extension
11 | LANG will be the markup code block language for the code's language
12 | CODE will be the code
13 |
14 | Expected output:
15 |
16 | FILENAME
17 | ```LANG
18 | CODE
19 | ```
20 |
21 | The code should be fully functional and contain no placeholders.
22 |
23 | You will begin with the "entrypoint" file, then you will go to the ones that are imported by that file, and so on.
24 | You will ensure to use best practice for file naming conventions based on the language and framework.
25 | Ensure that all files are compatible with each other and contain all imports, types, etc. required.
26 | Before you finish, double check that all parts of the architecture are present in the files to ensure that the user has a solid foundation to start their new project.
27 | """
28 |
--------------------------------------------------------------------------------
/programengineergpt/tools/README.md:
--------------------------------------------------------------------------------
1 | # Agent Tools
2 | ## Current
3 | 1. `CodeLoaderTool`: This tool is responsible for loading code from a given repository. It takes a repository URL or local path as input and returns a list of loaded code files.
4 | 2. `CodeAnalyzerTool`: This tool takes a question and the code embeddings as input and uses a language model to generate an answers to the question based on the code.
5 | 3. `CodeWriterTool`: This tool takes a coding task or goal as input and use a language model to generate code that accomplishes the task or goal. It could also use the code embeddings as context to generate code that is consistent with the existing codebase.
6 |
7 | ## Future
8 | 1. `CodeTesterTool`: This tool would automatically generate unit tests for the generated code. This would help ensure that the code is functioning as expected.
9 | 2. `CodeDocumentationTool`: This tool would automatically generate documentation for the generated code. It could use comments in the code and the structure of the code to generate useful documentation.
10 | 3. `CodeDependencyTool`: This tool would analyze the code to determine its dependencies. It could provide information about which libraries and packages the code depends on, and could potentially even automatically install these dependencies.
--------------------------------------------------------------------------------
/programengineergpt/tools/code_analyzer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 |
5 | from typing import Optional, Type
6 |
7 | from langchain.callbacks.manager import (
8 | AsyncCallbackManagerForToolRun,
9 | CallbackManagerForToolRun,
10 | )
11 | from langchain.tools import BaseTool
12 | from pydantic import BaseModel, Field
13 |
14 | from ..core.analyzer import AnalyzeCode
15 |
16 |
17 | # Define the input schema for the tool
18 | class CodeAnalyzingInput(BaseModel):
19 | username: str = Field(description="The username of the DeepLake account.")
20 | dataset_name: str = Field(description="The name of the dataset in DeepLake.")
21 | model_name: str = Field(description="The name of the model to use for analysis.")
22 | question: str = Field(description="The question to ask about the code.")
23 |
24 |
25 | class CodeAnalyzerTool(BaseTool):
26 | name = "CodeAnalyzer"
27 | description = "Analyzes code from a given dataset in DeepLake"
28 | args_schema: Type[BaseModel] = CodeAnalyzingInput
29 |
30 | def _run(
31 | self,
32 | username: str,
33 | dataset_name: str,
34 | model_name: str,
35 | question: str,
36 | run_manager: Optional[CallbackManagerForToolRun] = None,
37 | ) -> str:
38 | """Use the tool."""
39 | # Initialize the AnalyzeCode with the provided parameters
40 | analyzer = AnalyzeCode(username, dataset_name, model_name)
41 |
42 | # Ask the question and get the answer
43 | answer = analyzer.ask_question(question)
44 |
45 | # Return the answer
46 | return answer
47 |
48 | async def _arun(
49 | self,
50 | username: str,
51 | dataset_name: str,
52 | model_name: str,
53 | question: str,
54 | run_manager: Optional[AsyncCallbackManagerForToolRun] = None,
55 | ) -> str:
56 | """Use the tool asynchronously."""
57 | # This would be similar to the _run method, but with async calls
58 | # For the sake of this example, we will just return a dummy string
59 | return (
60 | f"Async code analyzing for dataset '{dataset_name}' completed successfully."
61 | )
62 |
--------------------------------------------------------------------------------
/programengineergpt/tools/code_docs.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/programengineergpt/tools/code_docs.py
--------------------------------------------------------------------------------
/programengineergpt/tools/code_loader.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 | from typing import Optional, Type
5 |
6 | from langchain.callbacks.manager import (
7 | AsyncCallbackManagerForToolRun,
8 | CallbackManagerForToolRun,
9 | )
10 | from langchain.tools import BaseTool
11 | from pydantic import BaseModel, Field
12 |
13 | from ..core.embedder import CodeEmbedder
14 | from ..core.loader import CodeLoader
15 |
16 |
17 | # Define the input schema for the tool
18 | class CodeLoadingInput(BaseModel):
19 | repository: str = Field(
20 | description="The URL of the GitHub repository or the path of the local repository to analyze."
21 | )
22 | deeplake_username: str = Field(description="Username of deeplake account")
23 | dataset_name: str = Field(
24 | description="The name of the dataset that will be stored in DeepLake"
25 | )
26 |
27 |
28 | class CodeLoaderTool(BaseTool):
29 | name = "CodeLoader"
30 | description = "Loads code from a given repository to DeepLake Vectorstore"
31 | args_schema: Type[BaseModel] = CodeLoadingInput
32 |
33 | def _run(
34 | self,
35 | repository: str,
36 | deeplake_username: str,
37 | dataset_name: str,
38 | run_manager: Optional[CallbackManagerForToolRun] = None,
39 | ) -> str:
40 | """Use the tool."""
41 | # Initialize the CodeLoader with the provided repository
42 | loader = CodeLoader(repository)
43 |
44 | # Load the repository and split the code into chunks
45 | code_chunks = loader.load_repository()
46 |
47 | # Embed the Code and Upload to DataLake VectorStore
48 | embed = CodeEmbedder(dataset_name, deeplake_username)
49 | embed.embed_and_upload(code_chunks)
50 |
51 | # For the sake of this example, we will just return a dummy string
52 | return f"Code loading for repository '{repository}' completed successfully."
53 |
54 | async def _arun(
55 | self,
56 | repository: str,
57 | run_manager: Optional[AsyncCallbackManagerForToolRun] = None,
58 | ) -> str:
59 | """Use the tool asynchronously."""
60 | # This would be similar to the _run method, but with async calls
61 | # For the sake of this example, we will just return a dummy string
62 | return (
63 | f"Async code loading for repository '{repository}' completed successfully."
64 | )
65 |
--------------------------------------------------------------------------------
/programengineergpt/tools/code_writer.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # imports
4 | from typing import Optional, Type
5 |
6 | from langchain.callbacks.manager import (
7 | AsyncCallbackManagerForToolRun,
8 | CallbackManagerForToolRun,
9 | )
10 | from langchain.tools import BaseTool
11 | from langchain.vectorstores import DeepLake
12 | from openai import OpenAI
13 | from pydantic import BaseModel, Field
14 |
15 |
16 | # Define the input schema for the tool
17 | class CodeWritingInput(BaseModel):
18 | task: str = Field(description="The code task to generate code for")
19 | username: str = Field(description="The username of the DeepLake account.")
20 | dataset_path: str = Field(description="The path to the dataset in DeepLake")
21 |
22 |
23 | class CodeWriterTool(BaseTool):
24 | name = "CodeWriter"
25 | description = "Generates code based on a provided task"
26 | args_schema: Type[BaseModel] = CodeWritingInput
27 |
28 | def _run(
29 | self,
30 | task: str,
31 | username: str,
32 | dataset_path: str,
33 | run_manager: Optional[CallbackManagerForToolRun] = None,
34 | ) -> str:
35 | """Use the tool."""
36 | # Initialize the OpenAI API with your API key
37 | openai = OpenAI(api_key="your-api-key")
38 |
39 | # Retrieve the embeddings from DeepLake
40 | db = DeepLake(dataset_path)
41 | embeddings = db.get_all_embeddings()
42 |
43 | # Convert the embeddings to a string representation
44 | embeddings_str = str(embeddings)
45 |
46 | # Use the chat models to generate code, with the embeddings as context
47 | response = openai.ChatCompletion.create(
48 | model="gpt-3.5-turbo",
49 | messages=[
50 | {"role": "system", "content": "You are a helpful assistant."},
51 | {"role": "system", "content": embeddings_str},
52 | {"role": "user", "content": task},
53 | ],
54 | )
55 |
56 | # Return the generated code
57 | return response["choices"][0]["message"]["content"]
58 |
59 | async def _arun(
60 | self,
61 | task: str,
62 | dataset_path: str,
63 | run_manager: Optional[AsyncCallbackManagerForToolRun] = None,
64 | ) -> str:
65 | """Use the tool asynchronously."""
66 | # This would be similar to the _run method, but with async calls
67 | # For the sake of this example, we will just return a dummy string
68 | return f"Async code generation for task '{task}' completed successfully."
69 |
--------------------------------------------------------------------------------
/programengineergpt/utils/__init__.py:
--------------------------------------------------------------------------------
1 | from programengineergpt.utils.args import Arguments
2 | from programengineergpt.utils.cli import CLI
3 | from programengineergpt.utils.colors import Color
4 | from programengineergpt.utils.display import Display
5 | from programengineergpt.utils.input import get_project_description
6 |
7 | __all__ = ["Arguments", "CLI", "Color", "Display", "get_project_description"]
8 |
--------------------------------------------------------------------------------
/programengineergpt/utils/args.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | import argparse
4 | import sys
5 |
6 | from programengineergpt.utils.colors import Color
7 |
8 |
9 | class Arguments(object):
10 | """Holds arguments used by CodeAssistantGPT"""
11 |
12 | def __init__(self, configuration):
13 | self.verbose = "-v" in sys.argv or "-hv" in sys.argv or "-vh" in sys.argv
14 | self.config = configuration
15 | self.args = self.get_arguments()
16 |
17 | def _verbose(self, msg):
18 | if self.verbose:
19 | return Color.s(msg)
20 | else:
21 | return argparse.SUPPRESS
22 |
23 | def get_arguments(self):
24 | """Returns parser.args() containing all program arguments"""
25 | parser = argparse.ArgumentParser(
26 | description="CodeAssistantGPT: An AI tool designed to assist with a variety of coding tasks."
27 | )
28 |
29 | parser.add_argument(
30 | "-c",
31 | "--chunk_size",
32 | type=int,
33 | default=1000,
34 | help="The size of the chunks to split the code into for analysis. Default is 1000.",
35 | )
36 |
37 | parser.add_argument(
38 | "-e",
39 | "--embedding_size",
40 | type=int,
41 | default=1536,
42 | help="The size of the embeddings to use for semantic analysis. Default is 1536.",
43 | )
44 |
45 | parser.add_argument(
46 | "-m",
47 | "--model",
48 | type=str,
49 | default="gpt-3.5-turbo-16k",
50 | help="The name of the GPT model to use for answering questions. Default is 'gpt-3.5-turbo-16k'.",
51 | )
52 |
53 | return parser.parse_args()
54 |
55 |
56 | if __name__ == "__main__":
57 | from programengineergpt.config import Config
58 |
59 | Config.init()
60 | a = Arguments(Config)
61 | args = a.args
62 | for key, value in sorted(args.__dict__.items()):
63 | Color.pl("{C}%s: {G}%s{W}" % (key.ljust(21), value))
64 |
--------------------------------------------------------------------------------
/programengineergpt/utils/chat_log.py:
--------------------------------------------------------------------------------
1 | import datetime
2 |
3 | def create_chat_log(chat_file):
4 | # Get current time
5 | current_time = datetime.datetime.now().strftime("%H:%M:%S")
6 |
7 | # Create a file to store the chat log
8 | with open(chat_file, "w") as file:
9 | file.write(f"Chat Log: {current_time}")
10 |
11 | def append_user_chat_message(chat_file, message):
12 | # Get current time
13 | current_time = datetime.datetime.now().strftime("%H:%M:%S")
14 |
15 | with open(chat_file, 'a') as file:
16 | # Check if the role is 'user' before writing to the file
17 | file.write(f"USER <{current_time}>: {message}")
18 | #file.write(f"USER <{current_time}>: {message.get('content', '')}\n")
--------------------------------------------------------------------------------
/programengineergpt/utils/cli.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 |
3 | # Imports
4 | import re
5 | import questionary
6 |
7 | from questionary import Style, ValidationError, Validator
8 |
9 | from programengineergpt.utils.input import get_project_description
10 | from programengineergpt.utils.colors import Color
11 |
12 | ## CLI Styler
13 | custom_style = Style(
14 | [
15 | ("separator", "fg:#6C6C6C"),
16 | ("qmark", "fg:#FF9D00 bold"),
17 | ("question", "bold"),
18 | ("selected", "fg:#5F819D"),
19 | ("pointer", "fg:#FF9D00 bold"),
20 | ("choice", "fg:#FF9D00 bold"),
21 | ("answer", "fg:#5F819D bold"),
22 | ]
23 | )
24 |
25 |
26 | class CLI(object):
27 | """
28 | Initial CLI
29 | """
30 |
31 | def __init__(self):
32 | """Init Class"""
33 | self.mode = ""
34 |
35 | def launc_main_cli(self):
36 | """Launch main CLI"""
37 |
38 | while True:
39 | self.mode = questionary.rawselect(
40 | "Select Mode:",
41 | choices=["Analyze", "Develop", "Exit"],
42 | style=custom_style,
43 | ).ask()
44 |
45 | if self.mode == "Exit":
46 | break
47 |
48 | if self.mode == "Analyze":
49 | self.handle_analyze_mode()
50 |
51 | elif self.mode == "Develop":
52 | self.handle_develop_mode()
53 |
54 | def handle_analyze_mode(self):
55 | """
56 | Handles required parameters for this Analyze Mode
57 | """
58 | from programengineergpt.utils.display import Display
59 |
60 | Display.display_analyze_mode_description()
61 | method = questionary.rawselect(
62 | "Please select a method for how you would like to provide your code:",
63 | choices=["URL", "Directory Path", "Current directory", "Back"],
64 | style=custom_style,
65 | ).ask()
66 |
67 | if method == "Back":
68 | return
69 |
70 | # Handle the selected method
71 | if method == "URL":
72 | self.handle_url()
73 |
74 | elif method == "Directory Path":
75 | self.handle_path()
76 |
77 | elif method == "Current directory":
78 | self.handle_cwd()
79 |
80 | # elif method == "Local Index":
81 | # self.handle_existing()
82 |
83 | def handle_develop_mode(self):
84 | """
85 | Handles the Develop New Program mode
86 | """
87 | from programengineergpt.utils.display import Display
88 | from programengineergpt.core.developer import Developer
89 |
90 | Display.display_develop_mode_description() # Display the description
91 |
92 | # Get Project Name
93 | Color.print("\n{B}Step 1: {W}Please provide folder name for your project")
94 | project_name = input("Project Folder: ")
95 |
96 | # Get project Description
97 | Color.print(
98 | "\n{B}Step 2: {W}Please provide a description for your new project. Feel free to provide as much detail as possible about your project.You are able to enter multiple lines using the 'ENTER' button."
99 | )
100 | Color.print("{Y}NOTE: {W}Use Ctrl-D (or Ctrl-Z on Windows) when finished.")
101 | Color.print("\n\n{P}Project Description:\n")
102 | project_description = get_project_description()
103 |
104 | # Initialize Developer
105 | Developer(project_description, project_name)
106 |
107 | def handle_url(self):
108 | """
109 | Gets URL and sends to loader
110 |
111 | Args:
112 | self : Argument
113 |
114 | """
115 | from programengineergpt.agents.chatbot import ChatBot
116 | from programengineergpt.core.loader import CodeLoader
117 |
118 | repo_url = questionary.text(
119 | "Please provide a link to an online code repository",
120 | instruction="\n Please use the following format\n https://github.com/username/repo\n\n URL: ",
121 | validate=URLValidator,
122 | style=custom_style,
123 | ).ask()
124 |
125 | loader = CodeLoader()
126 | vector_store = loader.load_online_repo(repo_url)
127 | chatbot = ChatBot(vector_store)
128 | chatbot.interact()
129 |
130 | def handle_path(self):
131 | """
132 | Gets Code Directory Path and sends to loader
133 | """
134 |
135 | from programengineergpt.agents.chatbot import ChatBot
136 | from programengineergpt.core.loader import CodeLoader
137 |
138 | dir_path = questionary.path(
139 | "Please provide the path to the code directory",
140 | only_directories=True,
141 | style=custom_style,
142 | ).ask()
143 |
144 | loader = CodeLoader()
145 | vector_store = loader.load_directory(dir_path)
146 | chatbot = ChatBot(vector_store)
147 | chatbot.interact()
148 |
149 | def handle_cwd(self):
150 | """
151 | Launches loader that handles current directory
152 | """
153 | from programengineergpt.agents.chatbot import ChatBot
154 | from programengineergpt.core.loader import CodeLoader
155 |
156 | loader = CodeLoader()
157 | vector_store = loader.load_current_directory()
158 | chatbot = ChatBot(vector_store)
159 | chatbot.interact()
160 |
161 | def handle_existing(self):
162 | """
163 | Handles and loads existing code index from deeplake
164 | """
165 | from programengineergpt.agents.chatbot import ChatBot
166 | from programengineergpt.core.retriever import Retriever
167 |
168 | Color.print("\n{Y}Please enter collection name for code base: ")
169 | col_name = input("Collection Name: ")
170 |
171 | index_retriever = Retriever(col_name)
172 | vector_store = index_retriever.retrive_code_index()
173 | chatbot = ChatBot(vector_store)
174 | chatbot.interact()
175 |
176 |
177 | class URLValidator(Validator):
178 | """
179 | Validate URL for Input
180 | """
181 |
182 | def validate(self, document):
183 | """
184 | Args:
185 | self : Argument
186 | document : Argument
187 |
188 | """
189 | url_pattern = re.compile(
190 | r"http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\\(\\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+"
191 | )
192 | if not url_pattern.match(document.text):
193 | raise ValidationError(
194 | message="Please enter a valid URL", cursor_position=len(document.text)
195 | ) # Move cursor to end
196 |
--------------------------------------------------------------------------------
/programengineergpt/utils/colors.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | import re
4 |
5 | from colored import attr, fg, stylize
6 |
7 |
8 | class Color(object):
9 |
10 | """Helpers for printing to terminal"""
11 |
12 | colors = {
13 | "W": 15, # white (normal)
14 | "R": 1,
15 | "G": 2,
16 | "O": 214,
17 | "B": 4,
18 | "P": 141,
19 | "C": 81,
20 | "X": 245, # Gray
21 | "Y": 226,
22 | "L": 121, # Light Green
23 | }
24 |
25 | @staticmethod
26 | def print(text):
27 | """
28 | Prints text using colored format on same line.
29 | Example:
30 | Color.print('{R}This text is red. {W}This text is white')
31 |
32 | Args:
33 | text : Argument. g colored format on same line.
34 | Example:
35 | Color.print('{R}This text is red. {W}This text is white')
36 |
37 | """
38 | output = Color.s(text)
39 | print(output)
40 |
41 | @staticmethod
42 | def s(text):
43 | """
44 | Args:
45 | text : Argument
46 |
47 | """
48 | base = text
49 | output = ""
50 | start = 0
51 |
52 | match_pattern = r"\{([A-Z])\}"
53 | matches = re.findall(match_pattern, text)
54 |
55 | for match in matches:
56 | color_key = match
57 | color_code = Color.colors[color_key]
58 | r_code = "{" + color_key + "}"
59 |
60 | base = base.replace(r_code, "")
61 | regex = re.search(match_pattern, base)
62 |
63 | if regex:
64 | end = regex.start()
65 | substring = base[start:end]
66 | start = end
67 | else:
68 | substring = base[start:]
69 |
70 | c_string = Color.color(substring, color_code) + " "
71 | output = "".join([output, c_string])
72 |
73 | return output
74 |
75 | @staticmethod
76 | def color(text, code):
77 | """
78 | Args:
79 | text : Argument
80 | code : Argument
81 |
82 | """
83 | colored_text = stylize(text, fg(code))
84 | return colored_text
85 |
86 | @staticmethod
87 | def p_error(error):
88 | """Prints an exception. Includes stack trace if necessary.
89 | Args:
90 | error : Argument
91 |
92 | """
93 | error_style = fg(1) + attr(1)
94 | msg_style = fg(15) + attr(0)
95 | error_message = (
96 | f"{stylize('Error:', error_style)} {stylize(str(error), msg_style)}"
97 | )
98 | print(error_message)
99 |
--------------------------------------------------------------------------------
/programengineergpt/utils/display.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # Imports
4 | import os
5 |
6 | from programengineergpt.utils.colors import Color
7 |
8 |
9 | class Display(object):
10 | @staticmethod
11 | def display_banner():
12 | design = (
13 | "\n"
14 | " ____ ______ _ __________ ______ \n"
15 | " / __ \_________ ____ __________ _____ ___ / ________ ____ _(_____ ___ ___ _____/ ____/ __ /_ __/ \n"
16 | " / /_/ / ___/ __ \/ __ `/ ___/ __ `/ __ `__ \/ __/ / __ \/ __ `/ / __ \/ _ \/ _ \/ ___/ / __/ /_/ // / \n"
17 | " / ____/ / / /_/ / /_/ / / / /_/ / / / / / / /___/ / / / /_/ / / / / / __/ __/ / / /_/ / ____// / \n"
18 | " /_/ /_/ \____/\__, /_/ \__,_/_/ /_/ /_/_____/_/ /_/\__, /_/_/ /_/\___/\___/_/ \____/_/ /_/ \n"
19 | " /____/ /____/ \n"
20 | "\n"
21 | )
22 |
23 | Color.print("{B}" + design)
24 | header = "{R} @hackedbyagirl {W}| {C} Hack the World\n"
25 | Color.print(header)
26 |
27 | @staticmethod
28 | def display_main_description():
29 | """
30 | Display main screen at launch
31 | """
32 | # Welcome Message
33 | Color.print("\n\n\t\t\t\t{B} *** Welcome to ProgramEngineerGPT! ***")
34 | Color.print(
35 | "\n{W}ProgramEngineerGPT is an AI tool designed to assist with a variety of coding tasks. Here's how you can use the tool:"
36 | )
37 |
38 | # Modes of Operation
39 | Color.print("\n\n{B}Modes of Operation:")
40 |
41 | # Mode: Analyze
42 | Color.print("\n{G}Analyze Mode:")
43 | Color.print(
44 | "\n{W}In the 'Analyze' mode, ProgramEngineerGPT will thoroughly examine the provided code repository. You will be engaged in an interactive chat session where you can pose queries about the codebase. This could include questions about its structure, dependencies, functions, or any other aspect. The AI will respond with insights, helping you gain a deeper understanding of the code repository."
45 | )
46 | Color.print(
47 | "\n{L}Use Case: {W}You have a large codebase and you want to understand its structure, dependencies, and other details. You can use the 'Analyze' mode to get a comprehensive analysis of the codebase."
48 | )
49 |
50 | # Mode: Develop
51 | Color.print("\n{G}Develop Mode:")
52 | Color.print(
53 | "\n{W}In the 'Develop' mode, ProgramEngineerGPT can assist you in setting up a new coding project. This includes planning the project structure, setting up the development environment, and other setup tasks."
54 | )
55 | Color.print(
56 | "\n{L}Use Case: {W}You are starting a new project and you need assistance in planning the project structure and setting up the development environment. You can use the 'Develop' mode to get assistance with these tasks."
57 | )
58 |
59 | Color.print(
60 | "\n\n{Y}Please select a mode to start using the tool's functionality.\n"
61 | )
62 |
63 | @staticmethod
64 | def display_analyze_mode_description():
65 | Display.clear_screen()
66 | Color.print("\n{B}*** Welcome to Analyze Mode *** \n")
67 | Color.print(
68 | "{W}In this mode, ProgramEngineerGPT acts as your personal code assistant. You'll enter an interactive chat where you can ask about the code's structure, dependencies, and more. The AI will provide insightful responses to help you understand your codebase better."
69 | )
70 | Color.print(
71 | "\n{X}Alternatively, if you already have a codebase indexed using Deeplake, you can provide the required information to your code hub\n"
72 | )
73 | Color.print(
74 | "\n{Y}Please provide the necessary details to start the 'Analyze' mode."
75 | )
76 |
77 | @staticmethod
78 | def display_develop_mode_description():
79 | """
80 | Display Develop Mode description screen
81 | """
82 | Display.clear_screen()
83 | Color.print("\n{B}*** Welcome Develop Mode*** \n")
84 | Color.print(
85 | "{W}In this mode, ProgramEngineerGPT will assist you in creating a new project. You will enter an interactive session where you will provide a project description of the program/project you want to create.\n"
86 | )
87 | Color.print(
88 | "{W}After you provide a project description, the AI system will ask further questions to gather more information about your project. Your responses will guide the AI in providing the best assistance for your project.\n"
89 | )
90 |
91 | @staticmethod
92 | def display_interactive_chat_banner():
93 | """
94 | Display welcome banner for interactive chat in Analyze Mode
95 | """
96 | Display.clear_screen()
97 | Color.print("\n{B}Welcome to the Interactive Chat Session in Analyze Mode!\n")
98 | Color.print(
99 | "\n{W}You are now in an interactive chat session with ProgramEngineerGPT. Feel free to ask any questions about the code repository you provided. The AI will provide insightful responses based on its analysis of the codebase.\n"
100 | )
101 |
102 | Color.print(
103 | "\n{B}To Begin: {W}Please enter your question (or 'exit' to stop): "
104 | )
105 |
106 | @staticmethod
107 | def display_interactive_developer_banner():
108 | """
109 | Display welcome banner for interactive project developer
110 | """
111 | Display.clear_screen()
112 | Color.print("\n{B}Welcome to the Interactive Project Developer!\n")
113 | Color.print(
114 | "\n{W}You are now in an interactive session with ProgramEngineerGPT. This mode will assist you in setting up a new coding project, from planning the project structure to setting up the development environment. The AI will provide assistance based on the information you provide.\n"
115 | )
116 |
117 | @staticmethod
118 | def clear_screen():
119 | """Clear Screen"""
120 | os.system("cls" if os.name == "nt" else "clear")
121 |
--------------------------------------------------------------------------------
/programengineergpt/utils/file_processing.py:
--------------------------------------------------------------------------------
1 | # file_processing.py
2 |
3 | # File extensions to consider
4 | extensions_include = [
5 | "bash",
6 | "cfg",
7 | "conf",
8 | "cpp",
9 | "cs",
10 | "css",
11 | "dockerignore",
12 | "editorconfig",
13 | "gitignore",
14 | "go",
15 | "html",
16 | "htm",
17 | "ini",
18 | "java",
19 | "javascript",
20 | "json",
21 | "js",
22 | "markdown",
23 | "md",
24 | "php",
25 | "py",
26 | "rb",
27 | "scala",
28 | "scss",
29 | "sh",
30 | "sql",
31 | "toml",
32 | "txt",
33 | "xml",
34 | "yaml",
35 | "yml",
36 | ]
37 |
38 | # File extensions to ignore
39 | extensions_ignore = [
40 | "*.py[cod]", # Byte-compiled / optimized / DLL files
41 | "*$py.class", # Byte-compiled / optimized / DLL files
42 | "*.egg", # Distributions and packaging
43 | "*.egg-info", # Distributions and packaging
44 | "*.chroma", # Chroma
45 | "*.env", # ENV variables
46 | ]
47 |
48 | # Directories to ignore
49 | dirs_ignore = [
50 | "__pycache__", # Byte-compiled / optimized / DLL files
51 | ".pytest_cache", # Byte-compiled / optimized / DLL files
52 | "venv", # Virtual Environments
53 | ".Python", # Distribution / packaging
54 | "build", # Distribution / packaging
55 | "develop-eggs", # Distribution / packaging
56 | "dist", # Distribution / packaging
57 | "downloads", # Distribution / packaging
58 | "eggs", # Distribution / packaging
59 | ".eggs", # Distribution / packaging
60 | "lib", # Distribution / packaging
61 | "lib64", # Distribution / packaging
62 | "parts", # Distribution / packaging
63 | "sdist", # Distribution / packaging
64 | "var", # Distribution / packaging
65 | "wheels", # Distribution / packaging
66 | "share/python-wheels", # Distribution / packaging
67 | "*.egg-info", # Distribution / packaging
68 | ".installed.cfg", # Distribution / packaging
69 | "MANIFEST", # Distribution / packaging
70 | ".chroma", # Chroma
71 | ]
--------------------------------------------------------------------------------
/programengineergpt/utils/input.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 |
3 | # Imports
4 |
5 |
6 | def get_project_description():
7 | lines = []
8 | try:
9 | while True:
10 | line = input()
11 | lines.append(line)
12 | except EOFError:
13 | pass
14 | results = "\n".join(lines)
15 | return results
16 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | aioboto3==11.2.0
2 | aiobotocore==2.5.0
3 | aiohttp==3.8.4
4 | aioitertools==0.11.0
5 | aiosignal==1.3.1
6 | anyio==3.7.0
7 | argon2-cffi==21.3.0
8 | argon2-cffi-bindings==21.2.0
9 | arrow==1.2.3
10 | asttokens==2.2.1
11 | async-lru==2.0.2
12 | async-timeout==4.0.2
13 | attrs==23.1.0
14 | Babel==2.12.1
15 | backcall==0.2.0
16 | backoff==2.2.1
17 | beautifulsoup4==4.12.2
18 | black==23.3.0
19 | bleach==6.0.0
20 | boto3==1.26.76
21 | botocore==1.29.76
22 | bs4==0.0.1
23 | certifi==2023.5.7
24 | cffi==1.15.1
25 | charset-normalizer==3.1.0
26 | chromadb==0.3.26
27 | click==8.1.3
28 | clickhouse-connect==0.6.4
29 | cmake==3.26.4
30 | colorama==0.4.6
31 | colored==2.2.2
32 | coloredlogs==15.0.1
33 | comm==0.1.3
34 | contourpy==1.1.0
35 | cryptography==41.0.1
36 | cycler==0.11.0
37 | dataclasses-json==0.5.8
38 | debugpy==1.6.7
39 | decorator==5.1.1
40 | defusedxml==0.7.1
41 | dill==0.3.6
42 | docutils==0.20.1
43 | duckdb==0.8.1
44 | entrypoints==0.4
45 | exceptiongroup==1.1.1
46 | executing==1.2.0
47 | fastapi==0.98.0
48 | fastjsonschema==2.17.1
49 | filelock==3.12.2
50 | flatbuffers==23.5.26
51 | fonttools==4.40.0
52 | fqdn==1.5.1
53 | frozenlist==1.3.3
54 | ghp-import==2.1.0
55 | gitdb==4.0.10
56 | GitPython==3.1.31
57 | greenlet==2.0.2
58 | h11==0.14.0
59 | hnswlib==0.7.0
60 | httptools==0.5.0
61 | humanfriendly==10.0
62 | humbug==0.3.1
63 | idna==3.4
64 | importlib-metadata==6.7.0
65 | iniconfig==2.0.0
66 | ipykernel==6.23.2
67 | ipython==8.14.0
68 | isoduration==20.11.0
69 | isort==5.12.0
70 | jaraco.classes==3.2.3
71 | jedi==0.18.2
72 | jeepney==0.8.0
73 | Jinja2==3.1.2
74 | jmespath==1.0.1
75 | joblib==1.2.0
76 | json5==0.9.14
77 | jsonpointer==2.3
78 | jsonschema==4.17.3
79 | jupyter-events==0.6.3
80 | jupyter-lsp==2.2.0
81 | jupyter-server-mathjax==0.2.6
82 | jupyter_client==8.2.0
83 | jupyter_core==5.3.1
84 | jupyter_server==2.6.0
85 | jupyter_server_terminals==0.4.4
86 | jupyterlab==4.0.2
87 | jupyterlab-git==0.41.0
88 | jupyterlab-pygments==0.2.2
89 | jupyterlab_server==2.23.0
90 | keyring==24.1.0
91 | kiwisolver==1.4.4
92 | langchain==0.0.212
93 | langchainplus-sdk==0.0.17
94 | lit==16.0.6
95 | lz4==4.3.2
96 | Markdown==3.3.7
97 | markdown-it-py==3.0.0
98 | MarkupSafe==2.1.3
99 | marshmallow==3.19.0
100 | marshmallow-enum==1.5.1
101 | matplotlib==3.7.1
102 | matplotlib-inline==0.1.6
103 | mdurl==0.1.2
104 | mergedeep==1.3.4
105 | mistune==2.0.5
106 | mkdocs==1.4.3
107 | monotonic==1.6
108 | more-itertools==9.1.0
109 | mpmath==1.3.0
110 | multidict==6.0.4
111 | multiprocess==0.70.14
112 | mypy-extensions==1.0.0
113 | nbclient==0.8.0
114 | nbconvert==7.5.0
115 | nbdime==3.2.1
116 | nbformat==5.9.0
117 | nest-asyncio==1.5.6
118 | networkx==3.1
119 | notebook_shim==0.2.3
120 | numcodecs==0.11.0
121 | numexpr==2.8.4
122 | numpy==1.24.3
123 | onnxruntime==1.15.1
124 | openai==0.27.8
125 | openapi-schema-pydantic==1.2.4
126 | overrides==7.3.1
127 | packaging==23.1
128 | pandas==2.0.2
129 | pandocfilters==1.5.0
130 | parso==0.8.3
131 | pathos==0.3.0
132 | pathspec==0.11.1
133 | pexpect==4.8.0
134 | pickleshare==0.7.5
135 | Pillow==9.5.0
136 | pkginfo==1.9.6
137 | platformdirs==3.5.3
138 | plotly==5.15.0
139 | pluggy==1.2.0
140 | posthog==3.0.1
141 | pox==0.3.2
142 | ppft==1.7.6.6
143 | prometheus-client==0.17.0
144 | prompt-toolkit==3.0.38
145 | protobuf==4.23.3
146 | psutil==5.9.5
147 | ptyprocess==0.7.0
148 | pulsar-client==3.2.0
149 | pure-eval==0.2.2
150 | pycparser==2.21
151 | pydantic==1.10.9
152 | Pygments==2.15.1
153 | PyJWT==2.7.0
154 | pyparsing==3.0.9
155 | pyproject_hooks==1.0.0
156 | pyrsistent==0.19.3
157 | pytest==7.4.0
158 | pytest-ruff==0.1
159 | python-dateutil==2.8.2
160 | python-dotenv==1.0.0
161 | python-json-logger==2.0.7
162 | pytz==2023.3
163 | PyYAML==6.0
164 | pyyaml_env_tag==0.1
165 | pyzmq==25.1.0
166 | questionary==1.10.0
167 | readme-renderer==40.0
168 | regex==2023.6.3
169 | requests==2.31.0
170 | requests-toolbelt==1.0.0
171 | rfc3339-validator==0.1.4
172 | rfc3986==2.0.0
173 | rfc3986-validator==0.1.1
174 | rich==13.4.2
175 | ruff==0.0.275
176 | s3transfer==0.6.1
177 | scikit-learn==1.2.2
178 | scipy==1.10.1
179 | seaborn==0.12.2
180 | SecretStorage==3.3.3
181 | Send2Trash==1.8.2
182 | six==1.16.0
183 | smmap==5.0.0
184 | sniffio==1.3.0
185 | soupsieve==2.4.1
186 | SQLAlchemy==2.0.17
187 | stack-data==0.6.2
188 | starlette==0.27.0
189 | sympy==1.12
190 | tenacity==8.2.2
191 | terminado==0.17.1
192 | threadpoolctl==3.1.0
193 | tiktoken==0.4.0
194 | tinycss2==1.2.1
195 | tokenizers==0.13.3
196 | tomli==2.0.1
197 | torch==2.0.1
198 | tornado==6.3.2
199 | tqdm==4.65.0
200 | traitlets==5.9.0
201 | triton==2.0.0
202 | typing==3.7.4.3
203 | typing-inspect==0.9.0
204 | typing_extensions==4.6.3
205 | tzdata==2023.3
206 | uri-template==1.2.0
207 | urllib3==1.26.16
208 | uvicorn==0.22.0
209 | uvloop==0.17.0
210 | watchdog==3.0.0
211 | watchfiles==0.19.0
212 | wcwidth==0.2.6
213 | webcolors==1.13
214 | webencodings==0.5.1
215 | websocket-client==1.5.3
216 | websockets==11.0.3
217 | wrapt==1.15.0
218 | yarl==1.9.2
219 | zipp==3.15.0
220 | zstandard==0.21.0
221 |
--------------------------------------------------------------------------------
/tests/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/tests/__init__.py
--------------------------------------------------------------------------------
/tests/test_code_assistant.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/hackedbyagirl/program-engineer-gpt/c75e8c22e175a3bb248ca67a1895278e83ac418c/tests/test_code_assistant.py
--------------------------------------------------------------------------------
/tests/test_config.py:
--------------------------------------------------------------------------------
1 | from src.config import Config
2 |
3 |
4 | def test_init():
5 | # Call the init method
6 | Config.init()
7 |
8 | # Check that the initialized attribute is set to True
9 | assert Config.initialized is True
10 |
11 | # Check that the default values are set correctly
12 | assert Config.verbose == 0
13 | assert Config.openai_key is not None
14 | assert Config.activeloop_key is not None
15 | assert Config.activeloop_username is not None
16 | assert Config.repo is None
17 | assert Config.model == 'gpt-3.5-turbo'
18 | assert Config.question is None
19 | assert Config.chunk_size == 1000
20 | assert Config.chunk_overlap == 0
21 | assert Config.embedded_size == 1536
22 |
--------------------------------------------------------------------------------
/tests/test_embedder.py:
--------------------------------------------------------------------------------
1 | from unittest.mock import MagicMock, patch
2 |
3 | from src.core.embedder import CodeEmbedder
4 |
5 |
6 | def test_embed_code():
7 | # Mock the OpenAIEmbeddings and DeepLake classes
8 | with patch('src.core.embedder.OpenAIEmbeddings') as MockOpenAIEmbeddings, \
9 | patch('src.core.embedder.DeepLake') as MockDeepLake:
10 | # Create a CodeEmbedder object
11 | embedder = CodeEmbedder(["def test(): pass"])
12 |
13 | # Mock the DeepLake.from_documents method to return a DeepLake object
14 | mock_deeplake = MagicMock()
15 | MockDeepLake.from_documents.return_value = mock_deeplake
16 |
17 | # Call the embed_code method
18 | embedder.embed_code()
19 |
20 | # Check that the OpenAIEmbeddings class was instantiated
21 | MockOpenAIEmbeddings.assert_called_once()
22 |
23 | # Check that the DeepLake.from_documents method was called with the correct arguments
24 | MockDeepLake.from_documents.assert_called_once_with(
25 | dataset_path="hub://hackedbyagirl/langchain-code",
26 | embedding_function=MockOpenAIEmbeddings.return_value,
27 | public=False
28 | )
29 |
30 | # Check that the add_documents method of the DeepLake object was called with the correct arguments
31 | mock_deeplake.add_documents.assert_called_once_with(embedder.code_chunks)
32 |
--------------------------------------------------------------------------------
/tests/test_load_and_split.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | import os
4 |
5 | import pytest
6 |
7 | from src.core.loader import CodeLoader
8 |
9 |
10 | @pytest.fixture
11 | def cleanup():
12 | # Setup: nothing to do
13 | yield
14 | # Teardown: remove the temp_repo directory if it exists
15 | if os.path.exists("temp_repo"):
16 | os.system("rm -rf temp_repo")
17 |
18 | def test_load_repository_local(cleanup):
19 | # Create a temporary directory with a Python file
20 | os.mkdir("temp_dir")
21 | with open("temp_dir/test.py", "w") as f:
22 | f.write("print('Hello, world!')")
23 |
24 | # Load the directory with CodeLoader
25 | loader = CodeLoader("temp_dir")
26 | loader.load_repository()
27 |
28 | # Check that the Python file was loaded
29 | loaded_files = [doc.metadata['source'] for doc in loader.split_code()]
30 | assert "temp_dir/test.py" in loaded_files
31 |
32 | # Clean up the temporary directory
33 | os.remove("temp_dir/test.py")
34 | os.rmdir("temp_dir")
35 |
36 | def test_load_repository_invalid(cleanup):
37 | # Try to load an invalid directory
38 | with pytest.raises(Exception) as e:
39 | loader = CodeLoader("invalid_dir")
40 | loader.load_repository()
41 | assert str(e.value) == "Invalid local directory: invalid_dir"
42 |
43 | def test_load_repository_github(cleanup):
44 | # Try to load a GitHub repository
45 | # Note: This test requires internet access and the repository to be public
46 | loader = CodeLoader("https://github.com/githubtraining/hellogitworld")
47 | loader.load_repository()
48 |
49 | # Check that at least one file was loaded
50 | assert len(loader.split_code()) > 0
51 |
--------------------------------------------------------------------------------