├── LICENSE ├── README.md ├── docs ├── img.png ├── img1.png ├── img3.png ├── img5.png ├── math.png ├── prompts.txt ├── strawberry.png └── tutor.mp4 ├── example.env ├── gradio ├── o1_groq.py └── requirements.txt ├── o1_groq.py ├── o1_ollama.py └── requirements.txt /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Charles Ching 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains 3 | 4 | [Video Demo](https://github.com/user-attachments/assets/db2a221f-f8eb-48c3-b5a7-8399c6300243) 5 | 6 | ## **Interface Updates (streamlit run o1_groq.py)** 7 | ![img.png](docs/img5.png) 8 | 9 | **仅仅 ```streamlit run o1_groq.py```更新了最新界面** 10 | 11 | This is an early prototype of using prompting strategies to improve the LLM's reasoning capabilities through o1-like reasoning chains. This allows the LLM to "think" and solve logical problems that usually otherwise stump leading models. Unlike o1, all the reasoning tokens are shown, and the app uses an open source model. 12 | 13 | 14 | g1 is experimental and being open sourced to help inspire the open source community to develop new strategies to produce o1-like reasoning. This experiment helps show the power of prompting reasoning in visualized steps, not a comparison to or full replication of o1, which uses different techniques. OpenAI's o1 is instead trained with large-scale reinforcement learning to reason using Chain of Thought, achieving state-of-the-art performance on complex PhD-level problems. 15 | 16 | g1 demonstrates the potential of prompting alone to overcome straightforward LLM logic issues like the Strawberry problem, allowing existing open source models to benefit from dynamic reasoning chains and an improved interface for exploring them. 17 | 18 | 19 | ### How it works 20 | 21 | g1 powered by Llama3.1-70b creates reasoning chains, in principle a dynamic Chain of Thought, that allows the LLM to "think" and solve some logical problems that usually otherwise stump leading models. 22 | 23 | At each step, the LLM can choose to continue to another reasoning step, or provide a final answer. Each step is titled and visible to the user. The system prompt also includes tips for the LLM. There is a full explanation under Prompt Breakdown, but a few examples are asking the model to “include exploration of alternative answers” and “use at least 3 methods to derive the answer”. 24 | 25 | The reasoning ability of the LLM is therefore improved through combining Chain-of-Thought with the requirement to try multiple methods, explore alternative answers, question previous draft solutions, and consider the LLM’s limitations. This alone, without any training, is sufficient to achieve ~70% accuracy on the Strawberry problem (n=10, "How many Rs are in strawberry?"). Without prompting, Llama-3.1-70b had 0% accuracy and ChatGPT-4o had 30% accuracy. 26 | 27 | 28 | ### Examples 29 | 30 | > [!IMPORTANT] 31 | > g1 is not perfect, but it can perform significantly better than LLMs out-of-the-box. From initial testing, g1 accurately solves simple logic problems 60-80% of the time that usually stump LLMs. However, accuracy has yet to be formally evaluated. See examples below. 32 | 33 | ### **Example of a Prompt Breakdown:** 34 | - **Prompt**: "Use at least 3 methods to find the number of Rs in the word 'strawberry'." 35 | - **Step 1**: Count characters manually. 36 | - **Step 2**: Check the result by comparing against other similar words. 37 | - **Step 3**: Analyze common patterns and mistakes in character counting. 38 | 39 | --- 40 | 41 | ## **Supported Models** 42 | 43 | 1. **Llama-3.1 70b on Groq**: The original implementation utilizes the **Llama-3.1 70b** model hosted on Groq, known for its capability to handle complex reasoning tasks. 44 | 45 | 2. **Ollama Local Models**: For those preferring to run models locally, o1 supports the **Ollama** environment, allowing the use of various open-source models on personal machines. 46 | 47 | --- 48 | 49 | ## **Installation & Setup** 50 | 51 | Here’s a step-by-step guide to setting up **o1** with both Streamlit and Gradio interfaces. 52 | 53 | ### **Quickstart (Streamlit UI on Groq)** 54 | 55 | 1. **Create a virtual environment**: 56 | ```bash 57 | python3 -m venv venv 58 | source venv/bin/activate 59 | ``` 60 | 61 | 2. **Install dependencies**: 62 | ```bash 63 | pip3 install -r requirements.txt 64 | ``` 65 | 66 | 3. **Set environment variable**: 67 | ```bash 68 | export GROQ_API_KEY=gsk... 69 | ``` 70 | 71 | 4. **Run the app**: 72 | ```bash 73 | streamlit run o1_groq.py 74 | ``` 75 | 76 | ### **Quickstart (Gradio UI)** 77 | 78 | For those preferring **Gradio**, follow these steps: 79 | 80 | 1. **Navigate to the Gradio folder**: 81 | ```bash 82 | cd gradio 83 | ``` 84 | 85 | 2. **Install dependencies**: 86 | ```bash 87 | pip3 install -r requirements.txt 88 | ``` 89 | 90 | 3. **Run the app**: 91 | ```bash 92 | python3 o1_groq.py 93 | ``` 94 | 95 | ### **For Ollama Local Models** 96 | 97 | 1. **Create a virtual environment**: 98 | ```bash 99 | python3 -m venv venv 100 | source venv/bin/activate 101 | ``` 102 | 103 | 2. **Install dependencies**: 104 | ```bash 105 | pip3 install -r requirements.txt 106 | ``` 107 | 108 | 3. **Configure your `.env` file**: 109 | Add the following environment variables: 110 | ``` 111 | OLLAMA_URL=your_ollama_url 112 | OLLAMA_MODEL=model_name 113 | ``` 114 | 115 | 4. **Run the app**: 116 | ```bash 117 | streamlit run o1_ollama.py 118 | ``` 119 | 120 | --- 121 | 122 | 123 | ### **Example: Is 3307 a Prime Number?** 124 | **Answer**: Yes. 125 | 126 | ![img.png](docs/img1.png) 127 | 128 | 129 | 130 | ## **Video Tutorial** 131 | 132 | Watch the step-by-step guide for setting up and using **o1-flow**: 133 | 134 | **`docs/tutor.mp4`** 135 | 136 | 140 | 141 | ### 其他例子在这：文章介绍 142 | https://mp.weixin.qq.com/s/RnDBwEHlGWEo01YROsHvmw 143 | 144 | ## **Forks** 145 | 146 | https://github.com/bklieger-groq/g1 147 | 148 | 149 | -------------------------------------------------------------------------------- /docs/img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/img.png -------------------------------------------------------------------------------- /docs/img1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/img1.png -------------------------------------------------------------------------------- /docs/img3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/img3.png -------------------------------------------------------------------------------- /docs/img5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/img5.png -------------------------------------------------------------------------------- /docs/math.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/math.png -------------------------------------------------------------------------------- /docs/prompts.txt: -------------------------------------------------------------------------------- 1 | You are an advanced AI reasoning assistant tasked with delivering a comprehensive analysis of a specific problem or question. Your goal is to outline your reasoning process in a structured and transparent manner, with each step reflecting a thorough examination of the issue at hand, culminating in a well-reasoned conclusion. 2 | 3 | ### Structure for Each Reasoning Step: 4 | 1. **Title**: Clearly label the phase of reasoning you are currently in. 5 | 2. **Content**: Provide a detailed account of your thought process, explaining your rationale and the steps taken to arrive at your conclusions. 6 | 3. **Next Action**: Decide whether to continue with further reasoning or if you are ready to provide a final answer. 7 | 8 | ### Response Format: 9 | Please return the results in the following JSON format: 10 | - `title`: A brief label for the current reasoning phase. 11 | - `content`: An in-depth explanation of your reasoning process for this step. 12 | - `next_action`: Choose `'continue'` to proceed with further reasoning or `'final_answer'` to conclude. 13 | 14 | ### Key Instructions: 15 | 1. Conduct **at least 5 distinct reasoning steps**, each building on the previous one. 16 | 2. **Acknowledge the limitations** inherent to AI, specifically what you can accurately assess and what you may struggle with. 17 | 3. **Adopt multiple reasoning frameworks** to resolve the problem or derive conclusions, such as: 18 | - **Deductive reasoning** (drawing specific conclusions from general principles) 19 | - **Inductive reasoning** (deriving broader generalizations from specific observations) 20 | - **Abductive reasoning** (choosing the best possible explanation for the given evidence) 21 | - **Analogical reasoning** (solving problems through comparisons and analogies) 22 | 4. **Critically analyze your reasoning** to identify potential flaws, biases, or gaps in logic. 23 | 5. When reviewing, apply a **fundamentally different perspective or approach** to enhance your analysis. 24 | 6. **Employ at least 2 distinct reasoning methods** to derive or verify the accuracy of your conclusions. 25 | 7. **Incorporate relevant domain knowledge** and **best practices** where applicable, ensuring your reasoning aligns with established standards. 26 | 8. **Quantify certainty levels** for each step and your final conclusion, where applicable. 27 | 9. Consider potential **edge cases or exceptions** that could impact the outcome of your reasoning. 28 | 10. Provide **clear justifications** for dismissing alternative hypotheses or solutions that arise during your analysis. 29 | 30 | ### Example JSON Output: 31 | 32 | ```json 33 | { 34 | "title": "Initial Problem Analysis", 35 | "content": "To approach this problem effectively, I'll first break down the given information into key components. This involves identifying... [detailed explanation]... By structuring the problem in this way, we can systematically address each aspect.", 36 | "next_action": "continue" 37 | } 38 | ``` -------------------------------------------------------------------------------- /docs/strawberry.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/strawberry.png -------------------------------------------------------------------------------- /docs/tutor.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Ai-trainee/o1-flow/6152ef59c6de2425cc66238265e6ceca8ff535e1/docs/tutor.mp4 -------------------------------------------------------------------------------- /example.env: -------------------------------------------------------------------------------- 1 | GROQ_API_KEY= 2 | 3 | OLLAMA_URL=http://localhost:11434 4 | OLLAMA_MODEL=llama3.1:8B 5 | 6 | -------------------------------------------------------------------------------- /gradio/o1_groq.py: -------------------------------------------------------------------------------- 1 | import gradio as gr 2 | import groq 3 | import os 4 | import json 5 | import time 6 | 7 | 8 | def make_api_call(client, messages, max_tokens, is_final_answer=False): 9 | for attempt in range(3): 10 | try: 11 | response = client.chat.completions.create( 12 | model="llama-3.1-70b-versatile", 13 | messages=messages, 14 | max_tokens=max_tokens, 15 | temperature=0.2, 16 | response_format={"type": "json_object"} 17 | ) 18 | return json.loads(response.choices[0].message.content) 19 | except Exception as e: 20 | if attempt == 2: 21 | if is_final_answer: 22 | return {"title": "Error", "content": f"Failed to generate final answer after 3 attempts. Error: {str(e)}"} 23 | else: 24 | return {"title": "Error", "content": f"Failed to generate step after 3 attempts. Error: {str(e)}", "next_action": "final_answer"} 25 | time.sleep(1) # Wait for 1 second before retrying 26 | 27 | def generate_response(client, prompt): 28 | messages = [ 29 | {"role": "system", "content": """You are an expert AI assistant that explains your reasoning step by step. For each step, provide a title that describes what you're doing in that step, along with the content. Decide if you need another step or if you're ready to give the final answer. Respond in JSON format with 'title', 'content', and 'next_action' (either 'continue' or 'final_answer') keys. USE AS MANY REASONING STEPS AS POSSIBLE. AT LEAST 3. BE AWARE OF YOUR LIMITATIONS AS AN LLM AND WHAT YOU CAN AND CANNOT DO. IN YOUR REASONING, INCLUDE EXPLORATION OF ALTERNATIVE ANSWERS. CONSIDER YOU MAY BE WRONG, AND IF YOU ARE WRONG IN YOUR REASONING, WHERE IT WOULD BE. FULLY TEST ALL OTHER POSSIBILITIES. YOU CAN BE WRONG. WHEN YOU SAY YOU ARE RE-EXAMINING, ACTUALLY RE-EXAMINE, AND USE ANOTHER APPROACH TO DO SO. DO NOT JUST SAY YOU ARE RE-EXAMINING. USE AT LEAST 3 METHODS TO DERIVE THE ANSWER. USE BEST PRACTICES. 30 | 31 | Example of a valid JSON response: 32 | ```json 33 | { 34 | "title": "Identifying Key Information", 35 | "content": "To begin solving this problem, we need to carefully examine the given information and identify the crucial elements that will guide our solution process. This involves...", 36 | "next_action": "continue" 37 | }``` 38 | """ }, 39 | {"role": "user", "content": prompt}, 40 | {"role": "assistant", "content": "Thank you! I will now think step by step following my instructions, starting at the beginning after decomposing the problem."} 41 | ] 42 | 43 | steps = [] 44 | step_count = 1 45 | total_thinking_time = 0 46 | 47 | while True: 48 | start_time = time.time() 49 | step_data = make_api_call(client, messages, 300) 50 | end_time = time.time() 51 | thinking_time = end_time - start_time 52 | total_thinking_time += thinking_time 53 | 54 | # Handle potential errors 55 | if step_data.get('title') == "Error": 56 | steps.append((f"Step {step_count}: {step_data.get('title')}", step_data.get('content'), thinking_time)) 57 | break 58 | 59 | step_title = f"Step {step_count}: {step_data.get('title', 'No Title')}" 60 | step_content = step_data.get('content', 'No Content') 61 | steps.append((step_title, step_content, thinking_time)) 62 | 63 | messages.append({"role": "assistant", "content": json.dumps(step_data)}) 64 | 65 | if step_data.get('next_action') == 'final_answer': 66 | break 67 | 68 | step_count += 1 69 | 70 | # Generate final answer 71 | messages.append({"role": "user", "content": "Please provide the final answer based on your reasoning above."}) 72 | 73 | start_time = time.time() 74 | final_data = make_api_call(client, messages, 200, is_final_answer=True) 75 | end_time = time.time() 76 | thinking_time = end_time - start_time 77 | total_thinking_time += thinking_time 78 | 79 | if final_data.get('title') == "Error": 80 | steps.append(("Final Answer", final_data.get('content'), thinking_time)) 81 | else: 82 | steps.append(("Final Answer", final_data.get('content', 'No Content'), thinking_time)) 83 | 84 | return steps, total_thinking_time 85 | 86 | def format_steps(steps, total_time): 87 | html_content = "" 88 | for title, content, thinking_time in steps: 89 | if title == "Final Answer": 90 | html_content += "

{}

".format(title) 91 | html_content += "

{}

".format(content.replace('\n', '
')) 92 | else: 93 | html_content += """ 94 |
95 | {} 96 |

{}

97 |

Thinking time for this step: {:.2f} seconds

98 |
99 |
100 | """.format(title, content.replace('\n', '
'), thinking_time) 101 | html_content += "Total thinking time: {:.2f} seconds".format(total_time) 102 | return html_content 103 | 104 | def main(api_key, user_query): 105 | if not api_key: 106 | return "Please enter your Groq API key to proceed.", "" 107 | 108 | if not user_query: 109 | return "Please enter a query to get started.", "" 110 | 111 | try: 112 | # Initialize the Groq client with the provided API key 113 | client = groq.Groq(api_key=api_key) 114 | except Exception as e: 115 | return f"Failed to initialize Groq client. Error: {str(e)}", "" 116 | 117 | try: 118 | steps, total_time = generate_response(client, user_query) 119 | formatted_steps = format_steps(steps, total_time) 120 | except Exception as e: 121 | return f"An error occurred during processing. Error: {str(e)}", "" 122 | 123 | return formatted_steps, "" 124 | 125 | # Define the Gradio interface 126 | with gr.Blocks() as demo: 127 | gr.Markdown("# 🧠 g1: Using Llama-3.1 70b on Groq to Create O1-like Reasoning Chains") 128 | 129 | gr.Markdown(""" 130 | This is an early prototype of using prompting to create O1-like reasoning chains to improve output accuracy. It is not perfect and accuracy has yet to be formally evaluated. It is powered by Groq so that the reasoning step is fast! 131 | 132 | Open source [repository here](https://github.com/bklieger-groq) 133 | """) 134 | 135 | with gr.Row(): 136 | with gr.Column(): 137 | api_input = gr.Textbox( 138 | label="Enter your Groq API Key:", 139 | placeholder="Your Groq API Key", 140 | type="password" 141 | ) 142 | user_input = gr.Textbox( 143 | label="Enter your query:", 144 | placeholder="e.g., How many 'R's are in the word strawberry?", 145 | lines=2 146 | ) 147 | submit_btn = gr.Button("Generate Response") 148 | 149 | with gr.Row(): 150 | with gr.Column(): 151 | output_html = gr.HTML() 152 | 153 | submit_btn.click(fn=main, inputs=[api_input, user_input], outputs=output_html) 154 | 155 | # Launch the Gradio app 156 | if __name__ == "__main__": 157 | demo.launch() 158 | 159 | ##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 -------------------------------------------------------------------------------- /gradio/requirements.txt: -------------------------------------------------------------------------------- 1 | groq 2 | gradio -------------------------------------------------------------------------------- /o1_groq.py: -------------------------------------------------------------------------------- 1 | # File path: g1_app_optimized_confirm.py 2 | import streamlit as st 3 | import groq 4 | import os 5 | import json 6 | import time 7 | from dotenv import load_dotenv 8 | 9 | # Load environment variables 10 | load_dotenv() 11 | 12 | # Centralized configuration 13 | CONFIG = { 14 | 'GROQ_MODEL': os.getenv('GROQ_MODEL', 'llama-3.1-70b-versatile'), 15 | 'DEFAULT_TEMPERATURE': float(os.getenv('DEFAULT_TEMPERATURE', 0.2)), 16 | 'DEFAULT_MAX_TOKENS': int(os.getenv('DEFAULT_MAX_TOKENS', 300)) 17 | } 18 | 19 | # Groq client 20 | client = groq.Groq() 21 | 22 | # Add caching to avoid redundant API calls 23 | @st.cache_data 24 | def make_api_call(messages, max_tokens, temperature, is_final_answer=False): 25 | for attempt in range(3): 26 | try: 27 | response = client.chat.completions.create( 28 | model=CONFIG['GROQ_MODEL'], 29 | messages=messages, 30 | max_tokens=max_tokens, 31 | temperature=temperature, 32 | response_format={"type": "json_object"} 33 | ) 34 | return json.loads(response.choices[0].message.content) 35 | except Exception as e: 36 | if attempt == 2: 37 | return { 38 | "title": "Error", 39 | "content": f"Failed to generate final answer after 3 attempts. Error: {str(e)}" if is_final_answer 40 | else f"Failed to generate step after 3 attempts. Error: {str(e)}", 41 | "next_action": "final_answer" if not is_final_answer else None 42 | } 43 | time.sleep(1) 44 | 45 | # Generate the reasoning response in steps 46 | def generate_response(prompt, max_tokens, temperature): 47 | messages = [ 48 | {"role": "system", "content": """You are an advanced AI reasoning assistant tasked with delivering a comprehensive analysis of a specific problem or question. Your goal is to outline your reasoning process in a structured and transparent manner, with each step reflecting a thorough examination of the issue at hand, culminating in a well-reasoned conclusion. 49 | 50 | ### Structure for Each Reasoning Step: 51 | 1. **Title**: Clearly label the phase of reasoning you are currently in. 52 | 2. **Content**: Provide a detailed account of your thought process, explaining your rationale and the steps taken to arrive at your conclusions. 53 | 3. **Next Action**: Decide whether to continue with further reasoning or if you are ready to provide a final answer. 54 | 55 | ### Response Format: 56 | Please return the results in the following JSON format: 57 | - `title`: A brief label for the current reasoning phase. 58 | - `content`: An in-depth explanation of your reasoning process for this step. 59 | - `next_action`: Choose `'continue'` to proceed with further reasoning or `'final_answer'` to conclude. 60 | 61 | ### Key Instructions: 62 | 1. Conduct **at least 5 distinct reasoning steps**, each building on the previous one. 63 | 2. **Acknowledge the limitations** inherent to AI, specifically what you can accurately assess and what you may struggle with. 64 | 3. **Adopt multiple reasoning frameworks** to resolve the problem or derive conclusions, such as: 65 | - **Deductive reasoning** (drawing specific conclusions from general principles) 66 | - **Inductive reasoning** (deriving broader generalizations from specific observations) 67 | - **Abductive reasoning** (choosing the best possible explanation for the given evidence) 68 | - **Analogical reasoning** (solving problems through comparisons and analogies) 69 | 4. **Critically analyze your reasoning** to identify potential flaws, biases, or gaps in logic. 70 | 5. When reviewing, apply a **fundamentally different perspective or approach** to enhance your analysis. 71 | 6. **Employ at least 2 distinct reasoning methods** to derive or verify the accuracy of your conclusions. 72 | 7. **Incorporate relevant domain knowledge** and **best practices** where applicable, ensuring your reasoning aligns with established standards. 73 | 8. **Quantify certainty levels** for each step and your final conclusion, where applicable. 74 | 9. Consider potential **edge cases or exceptions** that could impact the outcome of your reasoning. 75 | 10. Provide **clear justifications** for dismissing alternative hypotheses or solutions that arise during your analysis. 76 | 77 | ### Example JSON Output: 78 | 79 | ```json 80 | { 81 | "title": "Initial Problem Analysis", 82 | "content": "To approach this problem effectively, I'll first break down the given information into key components. This involves identifying... [detailed explanation]... By structuring the problem in this way, we can systematically address each aspect.", 83 | "next_action": "continue" 84 | } 85 | ```"""}, 86 | {"role": "user", "content": prompt}, 87 | {"role": "assistant", "content": "Thank you! I will now think step by step..."} 88 | ] 89 | 90 | steps = [] 91 | step_count = 1 92 | total_thinking_time = 0 93 | 94 | while True: 95 | start_time = time.time() 96 | step_data = make_api_call(messages, max_tokens, temperature) 97 | thinking_time = time.time() - start_time 98 | total_thinking_time += thinking_time 99 | 100 | steps.append((f"Step {step_count}: {step_data['title']}", step_data['content'], thinking_time)) 101 | messages.append({"role": "assistant", "content": json.dumps(step_data)}) 102 | 103 | if step_data['next_action'] == 'final_answer': 104 | break 105 | step_count += 1 106 | 107 | # Final answer step 108 | final_data = make_api_call(messages, 200, temperature, is_final_answer=True) 109 | steps.append(("Final Answer", final_data['content'], thinking_time)) 110 | 111 | return steps, total_thinking_time 112 | 113 | # Main Streamlit app 114 | def main(): 115 | st.set_page_config(page_title="g1 prototype", page_icon="🧠", layout="wide") 116 | 117 | # Sidebar settings 118 | with st.sidebar: 119 | st.title("Settings") 120 | st.markdown("Adjust the parameters below:") 121 | temperature = st.slider('Temperature', min_value=0.0, max_value=1.0, value=CONFIG['DEFAULT_TEMPERATURE'], key='temperature_slider') 122 | max_tokens = st.number_input('Max Tokens', min_value=50, max_value=20000, value=CONFIG['DEFAULT_MAX_TOKENS'], key='max_tokens_input') 123 | st.markdown("---") 124 | st.markdown("**Current Configuration**") 125 | st.markdown(f"**Model**: `{CONFIG['GROQ_MODEL']}`") 126 | st.markdown(f"**Temperature**: `{temperature}`") 127 | st.markdown(f"**Max Tokens**: `{max_tokens}`") 128 | 129 | st.title("g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains") 130 | 131 | # Custom CSS for improved UI aesthetics 132 | st.markdown(""" 133 | 150 | """, unsafe_allow_html=True) 151 | 152 | st.markdown(""" 153 | This is an early prototype of using prompting to create o1-like reasoning chains to improve output accuracy. 154 | It is not perfect, and accuracy has yet to be formally evaluated. 155 | It is powered by Groq to ensure fast reasoning steps! 156 | 157 | Forked from [bklieger-groq](https://github.com/bklieger-groq) 158 | Open source [repository here](https://github.com/Ai-trainee/o1-flow) 159 | """) 160 | 161 | # User query input 162 | user_query = st.text_input("Enter your query:", placeholder="e.g., How many 'R's are in the word strawberry?") 163 | 164 | # Add a button for confirmation of execution 165 | if st.button("Run Query"): 166 | if user_query.strip() == "": 167 | st.warning("Please enter a valid query.") 168 | else: 169 | st.write("Generating response...") 170 | 171 | # Add a loading spinner while generating responses 172 | with st.spinner('Processing your query...'): 173 | # Generate and display the response 174 | steps, total_thinking_time = generate_response(user_query, max_tokens, temperature) 175 | 176 | # Display response with progress 177 | progress_bar = st.progress(0) 178 | for idx, (title, content, thinking_time) in enumerate(steps): 179 | progress_bar.progress((idx + 1) / len(steps)) 180 | if title == "Final Answer": 181 | st.subheader(f"{title}") 182 | st.write(content.replace('\n', '
'), unsafe_allow_html=True) 183 | else: 184 | with st.expander(f"{title} ({thinking_time:.2f}s)", expanded=True): 185 | st.write(content.replace('\n', '
'), unsafe_allow_html=True) 186 | 187 | # Display total thinking time 188 | st.success(f"**Total thinking time: {total_thinking_time:.2f} seconds**") 189 | 190 | # Option to download results 191 | if st.button("Download Results"): 192 | json_data = json.dumps([{"title": title, "content": content} for title, content, _ in steps], indent=4) 193 | st.download_button("Download", json_data, file_name="reasoning_results.json", mime="application/json") 194 | 195 | 196 | if __name__ == "__main__": 197 | main() 198 | -------------------------------------------------------------------------------- /o1_ollama.py: -------------------------------------------------------------------------------- 1 | import streamlit as st 2 | import json 3 | import time 4 | import requests # Add this import for making HTTP requests to Ollama 5 | from dotenv import load_dotenv 6 | import os 7 | 8 | # Load environment variables 9 | load_dotenv() 10 | 11 | # Get configuration from .env file 12 | OLLAMA_URL = os.getenv('OLLAMA_URL', 'http://localhost:11434') 13 | OLLAMA_MODEL = os.getenv('OLLAMA_MODEL', 'llama3.1') 14 | 15 | 16 | def make_api_call(messages, max_tokens, is_final_answer=False): 17 | for attempt in range(3): 18 | try: 19 | response = requests.post( 20 | f"{OLLAMA_URL}/api/chat", 21 | json={ 22 | "model": OLLAMA_MODEL, 23 | "messages": messages, 24 | "stream": False, 25 | "options": { 26 | "num_predict": max_tokens, 27 | "temperature": 0.2 28 | } 29 | } 30 | ) 31 | response.raise_for_status() 32 | return json.loads(response.json()["message"]["content"]) 33 | except Exception as e: 34 | if attempt == 2: 35 | if is_final_answer: 36 | return {"title": "Error", 37 | "content": f"Failed to generate final answer after 3 attempts. Error: {str(e)}"} 38 | else: 39 | return {"title": "Error", "content": f"Failed to generate step after 3 attempts. Error: {str(e)}", 40 | "next_action": "final_answer"} 41 | time.sleep(1) # Wait for 1 second before retrying 42 | 43 | 44 | def generate_response(prompt): 45 | messages = [ 46 | {"role": "system", "content": """You are an advanced AI reasoning assistant tasked with delivering a comprehensive analysis of a specific problem or question. Your goal is to outline your reasoning process in a structured and transparent manner, with each step reflecting a thorough examination of the issue at hand, culminating in a well-reasoned conclusion. 47 | 48 | ### Structure for Each Reasoning Step: 49 | 1. **Title**: Clearly label the phase of reasoning you are currently in. 50 | 2. **Content**: Provide a detailed account of your thought process, explaining your rationale and the steps taken to arrive at your conclusions. 51 | 3. **Next Action**: Decide whether to continue with further reasoning or if you are ready to provide a final answer. 52 | 53 | ### Response Format: 54 | Please return the results in the following JSON format: 55 | - `title`: A brief label for the current reasoning phase. 56 | - `content`: An in-depth explanation of your reasoning process for this step. 57 | - `next_action`: Choose `'continue'` to proceed with further reasoning or `'final_answer'` to conclude. 58 | 59 | ### Key Instructions: 60 | 1. Conduct **at least 5 distinct reasoning steps**, each building on the previous one. 61 | 2. **Acknowledge the limitations** inherent to AI, specifically what you can accurately assess and what you may struggle with. 62 | 3. **Adopt multiple reasoning frameworks** to resolve the problem or derive conclusions, such as: 63 | - **Deductive reasoning** (drawing specific conclusions from general principles) 64 | - **Inductive reasoning** (deriving broader generalizations from specific observations) 65 | - **Abductive reasoning** (choosing the best possible explanation for the given evidence) 66 | - **Analogical reasoning** (solving problems through comparisons and analogies) 67 | 4. **Critically analyze your reasoning** to identify potential flaws, biases, or gaps in logic. 68 | 5. When reviewing, apply a **fundamentally different perspective or approach** to enhance your analysis. 69 | 6. **Employ at least 2 distinct reasoning methods** to derive or verify the accuracy of your conclusions. 70 | 7. **Incorporate relevant domain knowledge** and **best practices** where applicable, ensuring your reasoning aligns with established standards. 71 | 8. **Quantify certainty levels** for each step and your final conclusion, where applicable. 72 | 9. Consider potential **edge cases or exceptions** that could impact the outcome of your reasoning. 73 | 10. Provide **clear justifications** for dismissing alternative hypotheses or solutions that arise during your analysis. 74 | 75 | ### Example JSON Output: 76 | 77 | ```json 78 | { 79 | "title": "Initial Problem Analysis", 80 | "content": "To approach this problem effectively, I'll first break down the given information into key components. This involves identifying... [detailed explanation]... By structuring the problem in this way, we can systematically address each aspect.", 81 | "next_action": "continue" 82 | } 83 | ``` 84 | """}, 85 | {"role": "user", "content": prompt}, 86 | {"role": "assistant", "content": "Thank you! I will now think step by step following my instructions, starting at the beginning after decomposing the problem."} 87 | ] 88 | 89 | steps = [] 90 | step_count = 1 91 | total_thinking_time = 0 92 | 93 | while True: 94 | start_time = time.time() 95 | step_data = make_api_call(messages, 300) 96 | end_time = time.time() 97 | thinking_time = end_time - start_time 98 | total_thinking_time += thinking_time 99 | 100 | steps.append((f"Step {step_count}: {step_data['title']}", step_data['content'], thinking_time)) 101 | 102 | messages.append({"role": "assistant", "content": json.dumps(step_data)}) 103 | 104 | if step_data['next_action'] == 'final_answer': 105 | break 106 | 107 | step_count += 1 108 | 109 | # Yield after each step for Streamlit to update 110 | yield steps, None # We're not yielding the total time until the end 111 | 112 | # Generate final answer 113 | messages.append({"role": "user", "content": "Please provide the final answer based on your reasoning above."}) 114 | 115 | start_time = time.time() 116 | final_data = make_api_call(messages, 200, is_final_answer=True) 117 | end_time = time.time() 118 | thinking_time = end_time - start_time 119 | total_thinking_time += thinking_time 120 | 121 | steps.append(("Final Answer", final_data['content'], thinking_time)) 122 | 123 | yield steps, total_thinking_time 124 | 125 | 126 | def main(): 127 | st.set_page_config(page_title="ol1 prototype - Ollama version", page_icon="🧠", layout="wide") 128 | 129 | st.title("ol1: Using Ollama to create o1-like reasoning chains") 130 | 131 | st.markdown(""" 132 | This is an early prototype of using prompting to create o1-like reasoning chains to improve output accuracy. It is not perfect and accuracy has yet to be formally evaluated. It is powered by Ollama so that the reasoning step is local! 133 | 134 | Forked from [bklieger-groq](https://github.com/bklieger-groq) 135 | Open source [repository here](https://github.com/Ai-trainee/o1-flow) 136 | """) 137 | 138 | st.markdown(f"**Current Configuration:**") 139 | st.markdown(f"- Ollama URL: `{OLLAMA_URL}`") 140 | st.markdown(f"- Ollama Model: `{OLLAMA_MODEL}`") 141 | 142 | # Text input for user query 143 | user_query = st.text_input("Enter your query:", placeholder="e.g., How many 'R's are in the word strawberry?") 144 | 145 | if user_query: 146 | st.write("Generating response...") 147 | 148 | # Create empty elements to hold the generated text and total time 149 | response_container = st.empty() 150 | time_container = st.empty() 151 | 152 | # Generate and display the response 153 | for steps, total_thinking_time in generate_response(user_query): 154 | with response_container.container(): 155 | for i, (title, content, thinking_time) in enumerate(steps): 156 | if title.startswith("Final Answer"): 157 | st.markdown(f"### {title}") 158 | st.markdown(content.replace('\n', '
'), unsafe_allow_html=True) 159 | else: 160 | with st.expander(title, expanded=True): 161 | st.markdown(content.replace('\n', '
'), unsafe_allow_html=True) 162 | 163 | # Only show total time when it's available at the end 164 | if total_thinking_time is not None: 165 | time_container.markdown(f"**Total thinking time: {total_thinking_time:.2f} seconds**") 166 | 167 | 168 | if __name__ == "__main__": 169 | main() -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | streamlit 2 | groq 3 | python-dotenv 4 | requests 5 | blessed 6 | openai 7 | --------------------------------------------------------------------------------