├── .gitignore ├── Modelfile ├── README.md ├── app.py ├── custom_logger ├── __init__.py └── helper.py ├── models └── .gitkeep ├── requirements.txt ├── screenshots ├── screenshot_1.png └── screenshot_2.png └── utils ├── __init__.py ├── common_libraries.py ├── constants.py └── helper.py /.gitignore: -------------------------------------------------------------------------------- 1 | **/*.gguf 2 | **/*__pycache__ 3 | **/*logs 4 | -------------------------------------------------------------------------------- /Modelfile: -------------------------------------------------------------------------------- 1 | FROM models/codellama-7b-instruct.Q4_K_M.gguf 2 | 3 | TEMPLATE "[INST] <> {{ .System }} <> {{ .Prompt }} [/INST]" 4 | PARAMETER rope_frequency_base 1e+06 5 | PARAMETER stop "[INST]" 6 | PARAMETER stop "[/INST]" 7 | PARAMETER stop "<>" 8 | PARAMETER stop "<>" 9 | SYSTEM "You are an intelligent coding assistant - CodyBot. Answer all the code related to the questions asked." -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Personal Code Assistant - Ollama + Langchain + Streamlit 2 | 3 | This project demonstrates how to create a personal code assistant using a local open-source large language model (LLM). We will utilize Codellama, a fine-tuned version of Llama specifically developed for coding tasks, along with Ollama, Langchain and Streamlit to build a robust, interactive, and user-friendly interface. 4 | 5 | ## Table of Contents 6 | - [Prerequisites](#prerequisites) 7 | - [Setting up the Environment](#setting-up-the-environment) 8 | - [Customizing and Testing the Model](#customizing-and-testing-the-model) 9 | - [Building UI with Langchain and Streamlit](#building-ui-with-langchain-and-streamlit) 10 | - [References](#references) 11 | 12 | ## Prerequisites 13 | 14 | Before getting started, ensure that you have the following installed: 15 | - [Conda](https://docs.conda.io/en/latest/miniconda.html): Package and environment management system. 16 | - [Ollama](https://ollama.com/download): Software package that facilitates the use of LLMs easily. 17 | 18 | Also, for doing this project, it is good to have experience using OpenAI API with Langchain integration for getting it done faster. 19 | 20 | ## Setting up the Environment 21 | 22 | - Create new conda environment: 23 | ```bash 24 | conda create -n personal_code_assistant python=3.11 25 | ``` 26 | - Activate the environment: 27 | ```bash 28 | conda activate personal_code_assistant 29 | ``` 30 | - Install all the required packages: 31 | ```bash 32 | pip install -r requirements.txt 33 | ``` 34 | 35 | ## Customizing and testing the model 36 | 37 | We will be customizing the model for this project as we want the model to behave as closely as we want it to. `Modefile` enables us to achieve this. 38 | 39 | Steps as follows: 40 | 41 | 1. For this project, we will be downloading the quantized version (Q4_K_M) of **codellama** [model](https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF). 42 | 43 | 2. Go to models directory, download the model in it and then go back to the main directory of the project: 44 | ```bash 45 | cd models 46 | wget https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF/resolve/main/codellama-7b-instruct.Q4_K_M.gguf 47 | cd .. 48 | ``` 49 | 50 | 3. Include the following in `Modelfile`: 51 | ``` 52 | FROM models/codellama-7b-instruct.Q4_K_M.gguf 53 | 54 | TEMPLATE "[INST] <> {{ .System }} <> {{ .Prompt }} [/INST]" 55 | PARAMETER rope_frequency_base 1e+06 56 | PARAMETER stop "[INST]" 57 | PARAMETER stop "[/INST]" 58 | PARAMETER stop "<>" 59 | PARAMETER stop "<>" 60 | SYSTEM """You are an intelligent coding assistant - CodyBot. Answer all the code related to the questions asked. Your capabilities include understanding and generating code in multiple programming languages, troubleshooting coding issues, optimizing algorithms, and providing explanations of complex programming concepts in a clear and concise manner.""" 61 | ``` 62 | 63 | 4. Run the command: 64 | ```bash 65 | ollama create codybot -f Modelfile 66 | ``` 67 | Message at the end - success will indicate if the model is successfully customized. For further customization, checkout the [link](https://github.com/ollama/ollama/blob/main/docs/modelfile.md). 68 | 69 | 5. Before going ahead with running in Python followed by building UI using Streamlit library, lets try our code assistant in the terminal: 70 | ```bash 71 | ollama run codybot 72 | ``` 73 | 74 | Once we are satisfied with the behavior of how model is responding, we will proceed ahead. 75 | 76 | ## Building and Running the Application - Langchain + Streamlit 77 | 78 | - Langchain provides wrapper to access local models - `ChatOllama` which can be used in the same way as `ChatOpenAI` module. 79 | - `utils` folder includes all of the helper functions (`helper.py`), and commonly used libraries (`common_libraries.py`) organized. Prompt template(s) and other environment variable(s) (if used) are included `constants.py` folder. 80 | - `app.py` includes the Streamlit implementation - making use of the libraries and helper functions from `utils` folder. 81 | 82 | Once the code is ready, we can run the following command: 83 | ```bash 84 | streamlit run app.py 85 | ``` 86 | 87 | Example of demo as follows: 88 | 89 | ![Screenshot 1](https://github.com/di37/coding-assistant-codellama-streamlit/blob/main/screenshots/screenshot_1.png?raw=true) 90 | ![Screenshot 2](https://github.com/di37/coding-assistant-codellama-streamlit/blob/main/screenshots/screenshot_2.png?raw=true) 91 | 92 | 93 | ## References 94 | - End To End Multi Programming Code Assistant App Using CodeLlama LLAMA2 Large Language Model: https://www.youtube.com/watch?v=-28YtPZ5u4s&t=96s&ab_channel=KrishNaik 95 | - Unlock Ollama's Modelfile: https://www.youtube.com/watch?v=QTv3DQ1tY6I&ab_channel=PromptEngineer 96 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | from utils import * 2 | 3 | # app config 4 | st.set_page_config(page_title="🤖 Friendly Coding Assistant ✨", page_icon="🤖") 5 | st.title("🤖 CodyBot - Your Friendly, Coding Assistant! ✨") 6 | 7 | # Setting generation configuration - This will override the parameter that is set in the Modelfile 8 | get_config_gen = configure_generation() 9 | 10 | if "chat_history" not in st.session_state: 11 | st.session_state.chat_history = [ 12 | AIMessage(content="Hello, I am CodyBot - Your Friendly, Coding Assistant!. How can I help you?"), 13 | ] 14 | 15 | for message in st.session_state.chat_history: 16 | with st.chat_message("AI" if isinstance(message, AIMessage) else "Human"): 17 | st.write(message.content) 18 | 19 | # user input 20 | if user_query := st.chat_input("Type your message here..."): 21 | st.session_state.chat_history.append(HumanMessage(content=user_query)) 22 | with st.chat_message("Human"): 23 | st.write(user_query) 24 | 25 | with st.chat_message("AI"): 26 | response_placeholder = st.empty() 27 | full_response = "" 28 | for chunk in get_realtime_response(user_prompt=user_query, **get_config_gen): 29 | full_response += chunk 30 | response_placeholder.write(full_response) 31 | logger.info("Response successfully generated.") 32 | 33 | st.session_state.chat_history.append(AIMessage(content=full_response)) 34 | -------------------------------------------------------------------------------- /custom_logger/__init__.py: -------------------------------------------------------------------------------- 1 | from custom_logger.helper import * -------------------------------------------------------------------------------- /custom_logger/helper.py: -------------------------------------------------------------------------------- 1 | import os, sys 2 | from os.path import dirname as up 3 | 4 | sys.path.append(os.path.abspath(os.path.join(up(__file__), os.pardir))) 5 | 6 | import logging 7 | from logging.handlers import RotatingFileHandler 8 | 9 | # Setup basic parameters for logging 10 | logging_str = ( 11 | "[%(asctime)s: %(levelname)s: %(module)s: %(funcName)s: %(lineno)d] %(message)s" 12 | ) 13 | log_dir = "logs" 14 | log_filename = "running_logs.log" 15 | log_filepath = os.path.join(log_dir, log_filename) 16 | max_log_size = 10 * 1024 * 1024 # 10 MB 17 | backup_count = 3 # Number of backup logs to keep 18 | 19 | # Ensure the log directory exists 20 | os.makedirs(log_dir, exist_ok=True) 21 | 22 | # Create a logger with a specified name 23 | logger = logging.getLogger("") 24 | logger.setLevel(logging.DEBUG) # Set the logger to capture all levels of messages 25 | 26 | # Create handlers for both file and console with different log levels 27 | file_handler = RotatingFileHandler( 28 | log_filepath, maxBytes=max_log_size, backupCount=backup_count 29 | ) 30 | file_handler.setLevel(logging.DEBUG) # More verbose level for file 31 | file_handler.setFormatter(logging.Formatter(logging_str)) 32 | 33 | console_handler = logging.StreamHandler(sys.stdout) 34 | console_handler.setLevel( 35 | logging.INFO 36 | ) # Less verbose level for console, but will show warnings and errors 37 | console_handler.setFormatter(logging.Formatter(logging_str)) 38 | 39 | # Add handlers to the logger 40 | logger.addHandler(file_handler) 41 | logger.addHandler(console_handler) 42 | 43 | # Demonstrate logging at different levels 44 | # logger.debug("This is a debug message.") 45 | # logger.info("This is an info message.") 46 | # logger.warning("This is a warning message.") 47 | # logger.error("This is an error message.") 48 | -------------------------------------------------------------------------------- /models/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/di37/coding-assistant-codellama-streamlit/7dcaf8c86803d9df9f3d9a50cca8ec358ce8e062/models/.gitkeep -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | transformers 2 | langchain 3 | langchain-community 4 | langchain-core 5 | langchain-experimental 6 | streamlit -------------------------------------------------------------------------------- /screenshots/screenshot_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/di37/coding-assistant-codellama-streamlit/7dcaf8c86803d9df9f3d9a50cca8ec358ce8e062/screenshots/screenshot_1.png -------------------------------------------------------------------------------- /screenshots/screenshot_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/di37/coding-assistant-codellama-streamlit/7dcaf8c86803d9df9f3d9a50cca8ec358ce8e062/screenshots/screenshot_2.png -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- 1 | from utils.common_libraries import * 2 | from utils.helper import * 3 | from utils.constants import * 4 | -------------------------------------------------------------------------------- /utils/common_libraries.py: -------------------------------------------------------------------------------- 1 | import os, sys 2 | from os.path import dirname as up 3 | 4 | sys.path.append(os.path.abspath(os.path.join(up(__file__), os.pardir))) 5 | 6 | from langchain_community.chat_models import ChatOllama 7 | from langchain_core.output_parsers import StrOutputParser 8 | from langchain_core.prompts import ChatPromptTemplate 9 | from langchain_core.messages import AIMessage, HumanMessage 10 | 11 | import streamlit as st 12 | 13 | from custom_logger import * -------------------------------------------------------------------------------- /utils/constants.py: -------------------------------------------------------------------------------- 1 | import os, sys 2 | from os.path import dirname as up 3 | 4 | sys.path.append(os.path.abspath(os.path.join(up(__file__), os.pardir))) 5 | 6 | CODE_TEMPLATE = """Users come to you with a wide range of requests, from beginners asking for explanations of basic programming principles to seasoned developers seeking assistance with intricate coding problems. Your responses should be accurate, helpful, and tailored to the user's level of expertise. Whether it's writing a snippet of code to solve a specific problem, explaining the nuances of a programming language, or offering best practices for software development, your goal is to empower users to improve their coding skills and accomplish their programming tasks more efficiently. You maintain a friendly and professional tone, encouraging users to explore and learn from the coding process. 7 | 8 | Question: 9 | {user_prompt} 10 | 11 | Answer: 12 | """ -------------------------------------------------------------------------------- /utils/helper.py: -------------------------------------------------------------------------------- 1 | import os, sys 2 | from os.path import dirname as up 3 | 4 | sys.path.append(os.path.abspath(os.path.join(up(__file__), os.pardir))) 5 | 6 | from utils.common_libraries import * 7 | from utils.constants import * 8 | 9 | def get_realtime_response(user_prompt: str, model="codybot", **kwargs): 10 | """ 11 | This function takes a user prompt as a string, an optional model parameter, and arbitrary keyword arguments, 12 | and returns a realtime response. 13 | """ 14 | llm = ChatOllama(model=model, **kwargs) 15 | prompt = ChatPromptTemplate.from_template(CODE_TEMPLATE) 16 | chain = prompt | llm | StrOutputParser() 17 | response = chain.stream({"user_prompt": user_prompt}) 18 | return response 19 | 20 | def configure_generation(): 21 | """ 22 | Add sliders for temperature, top_p, top_k, and max_output_tokens 23 | """ 24 | # Add sliders for temperature, top_p, top_k, and max_output_tokens 25 | st.sidebar.header("Generation Configuration") 26 | temperature = st.sidebar.slider( 27 | "Temperature", min_value=0.0, max_value=1.0, value=0.7, step=0.01 28 | ) 29 | top_p = st.sidebar.slider( 30 | "Top P", min_value=0.0, max_value=1.0, value=0.9, step=0.01 31 | ) 32 | top_k = st.sidebar.slider("Top K", min_value=0, max_value=100, value=40, step=1) 33 | n_ctx = st.sidebar.slider( 34 | "Maximum Context Length", min_value=1, max_value=4096, value=2048, step=1 35 | ) 36 | 37 | generation_config = { 38 | "temperature": temperature, 39 | "top_p": top_p, 40 | "top_k": top_k, 41 | "num_ctx": n_ctx, 42 | } 43 | 44 | return generation_config --------------------------------------------------------------------------------