{event.description}
59 | 60 | {new Date(event.timestamp).toLocaleTimeString()} 61 | 62 | tags, or inline) and surrounding text
8 |
9 | Your job is to:
10 | 1. Extract all code that is intended to be stored as a reusable skill (i.e., class or function definitions), as indicated by the message and its instructions.
11 | 2. If the code is not in Python, translate it to Python as faithfully as possible, using standard libraries and idiomatic Python where appropriate.
12 | 3. If the code is pseudo-code, incomplete, or contains small errors (e.g., typos, misspelled variable names, or incorrect standard library usage), correct these issues to produce valid, well-formed Python class/function definitions.
13 | 4. If the surrounding instructions request modifications (e.g., change argument names, add docstrings, use a different algorithm, add or adjust comments), make those changes in the final code.
14 | 5. If the message provides only a code body, or describes arguments or docstrings in natural language, construct the appropriate function or class definition, inferring argument names, types, and docstrings as needed from the context.
15 | 6. All class and function definitions should be type-annotated wherever possible, and include informative docstrings that describe their purpose, arguments, and return values.
16 | 7. Comments provided in the input should be preserved and adjusted as necessary. Add comments to the code wherever the operations performed are not easily understood at a glance.
17 | 8. If there are multiple class or function definitions, output them together as a single, coherent Python module.
18 | 9. Do not include any explanatory text, comments about your process, or formatting (such as markdown backticks or code tags) in your output—output only the final Python class and function definitions.
19 |
20 | Examples:
21 |
22 | Example 1:
23 | MESSAGE:
24 | Please store this function for future use:
25 | ```
26 | def add(a, b):
27 | return a + b
28 | ```
29 | Add type annotations and a docstring.
30 |
31 | OUTPUT:
32 | def add(a: int, b: int) -> int:
33 | """Add two integers and return the result."""
34 | return a + b
35 |
36 | Example 2:
37 | MESSAGE:
38 | Convert this JavaScript function to Python and make it reusable:
39 | ```
40 | function greet(name) {
41 | // Print a greeting
42 | console.log('Hello, ' + name);
43 | }
44 | ```
45 |
46 | OUTPUT:
47 | def greet(name: str) -> None:
48 | """Print a greeting to the given name."""
49 | # Print a greeting
50 | print(f"Hello, {name}")
51 |
52 | Example 3:
53 | MESSAGE:
54 | Here's a code body. Please turn it into a function that takes a list of numbers and returns their mean. Add type annotations and a docstring.
55 | ```
56 | total = 0
57 | for x in numbers:
58 | total += x
59 | return total / len(numbers)
60 | ```
61 |
62 | OUTPUT:
63 | def mean(numbers: list[float]) -> float:
64 | """Calculate and return the mean of a list of numbers."""
65 | total = 0
66 | for x in numbers:
67 | total += x
68 | return total / len(numbers)
69 |
70 | Example 4:
71 | MESSAGE:
72 | Hi, I will now teach you how to handle a person entity. Below is some code functionality for what I have in mind:
73 | ```
74 | class Person:
75 | def __init__(self, name: str, age: int) -> None:
76 | self.name = name # The person's name
77 | self.age = age # The person's age
78 |
79 | def greet(self) -> str:
80 | return f"Hello, my name is {self.name}."
81 | ```
82 |
83 | OUTPUT:
84 | class Person:
85 | """A class representing a person with a name and age."""
86 |
87 | def __init__(self, name: str, age: int) -> None:
88 | """Initialize a new Person with a name and age."""
89 | self.name = name # The person's name
90 | self.age = age # The person's age
91 |
92 | def greet(self) -> str:
93 | """Return a greeting message including the person's name."""
94 | return f"Hello, my name is {self.name}."
95 |
96 | Example 5:
97 | MESSAGE:
98 | Here is a new skill for you -- computing the kth fibonacci number.
99 |
100 | def fib(k: int) -> int:
101 | if k < 0:
102 | raise ValueError("k must be a non-negative integer")
103 | elif k == 0:
104 | return 0
105 | elif k == 1:
106 | return 1
107 | a, b = 0, 1
108 | for _ in range(2, k + 1):
109 | a, b = b, a + b
110 | return b
111 |
112 | OUTPUT:
113 | def fib(k: int) -> int:
114 | """Compute and return the k-th Fibonacci number (0-indexed)."""
115 | if k < 0:
116 | raise ValueError("k must be a non-negative integer")
117 | elif k == 0:
118 | return 0
119 | elif k == 1:
120 | return 1
121 | a, b = 0, 1
122 | for _ in range(2, k + 1):
123 | a, b = b, a + b
124 | return b
125 |
126 | Reminders:
127 | - Output only the final, properly formatted Python class and function definitions, with no extra text or formatting.
128 | - Add type annotations and docstrings wherever possible.
129 | - Preserve and add comments as appropriate.
130 | - If translation or correction is not possible, output the closest valid Python class/function definitions that fulfill the user's intent.
131 | - Do not output any surrounding backticks (```) or code tags—output just the code directly.
132 |
133 | This is the end of the classification instructions. Below is the input to process.
134 |
135 | MESSAGE:
136 | {message}
--------------------------------------------------------------------------------
/backend/setup.py:
--------------------------------------------------------------------------------
1 | # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2 | # SPDX-License-Identifier: Apache-2.0
3 | #
4 | # Licensed under the Apache License, Version 2.0 (the "License");
5 | # you may not use this file except in compliance with the License.
6 | # You may obtain a copy of the License at
7 | #
8 | # http://www.apache.org/licenses/LICENSE-2.0
9 | #
10 | # Unless required by applicable law or agreed to in writing, software
11 | # distributed under the License is distributed on an "AS IS" BASIS,
12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | # See the License for the specific language governing permissions and
14 | # limitations under the License.
15 | #!/usr/bin/env python3
16 | """
17 | Setup script for the Universal Deep Research Backend (UDR-B).
18 |
19 | This script helps users configure the backend by:
20 | 1. Creating necessary directories
21 | 2. Setting up environment configuration
22 | 3. Validating API key files
23 | 4. Installing dependencies
24 | """
25 |
26 | import os
27 | import subprocess
28 | import sys
29 | from pathlib import Path
30 |
31 |
32 | def print_header(title: str):
33 | """Print a formatted header."""
34 | print(f"\n{'='*60}")
35 | print(f" {title}")
36 | print(f"{'='*60}")
37 |
38 |
39 | def print_step(step: str):
40 | """Print a step message."""
41 | print(f"\n→ {step}")
42 |
43 |
44 | def check_python_version():
45 | """Check if Python version is compatible."""
46 | print_step("Checking Python version...")
47 | if sys.version_info < (3, 8):
48 | print("❌ Python 3.8 or higher is required")
49 | sys.exit(1)
50 | print(f"✅ Python {sys.version_info.major}.{sys.version_info.minor} detected")
51 |
52 |
53 | def create_directories():
54 | """Create necessary directories."""
55 | print_step("Creating directories...")
56 | directories = ["logs", "instances", "mock_instances"]
57 |
58 | for directory in directories:
59 | Path(directory).mkdir(exist_ok=True)
60 | print(f"✅ Created directory: {directory}")
61 |
62 |
63 | def setup_env_file():
64 | """Set up environment configuration file."""
65 | print_step("Setting up environment configuration...")
66 |
67 | env_file = Path(".env")
68 | env_example = Path("env.example")
69 |
70 | if env_file.exists():
71 | print("✅ .env file already exists")
72 | return
73 |
74 | if env_example.exists():
75 | # Copy example file
76 | with open(env_example, "r") as f:
77 | content = f.read()
78 |
79 | with open(env_file, "w") as f:
80 | f.write(content)
81 |
82 | print("✅ Created .env file from env.example")
83 | print("⚠️ Please edit .env file with your configuration")
84 | else:
85 | print("❌ env.example file not found")
86 | print("Creating basic .env file...")
87 |
88 | basic_env = """# Universal Deep Research Backend (UDR-B) - Environment Configuration
89 | HOST=0.0.0.0
90 | PORT=8000
91 | LOG_LEVEL=info
92 | FRONTEND_URL=http://localhost:3000
93 | DEFAULT_MODEL=llama-3.1-nemotron-253b
94 | LLM_API_KEY_FILE=nvdev_api.txt
95 | TAVILY_API_KEY_FILE=tavily_api.txt
96 | MAX_TOPICS=1
97 | MAX_SEARCH_PHRASES=1
98 | LOG_DIR=logs
99 | """
100 | with open(env_file, "w") as f:
101 | f.write(basic_env)
102 |
103 | print("✅ Created basic .env file")
104 | print("⚠️ Please edit .env file with your configuration")
105 |
106 |
107 | def check_api_keys():
108 | """Check for required API key files."""
109 | print_step("Checking API key files...")
110 |
111 | required_files = ["nvdev_api.txt", "tavily_api.txt"]
112 |
113 | missing_files = []
114 | for file in required_files:
115 | if Path(file).exists():
116 | print(f"✅ {file} found")
117 | else:
118 | print(f"❌ {file} missing")
119 | missing_files.append(file)
120 |
121 | if missing_files:
122 | print(f"\n⚠️ Missing API key files: {', '.join(missing_files)}")
123 | print("Please create these files with your API keys:")
124 | for file in missing_files:
125 | print(f" - {file}")
126 | print("\nExample:")
127 | print(" echo 'your-api-key-here' > nvdev_api.txt")
128 | print(" echo 'your-tavily-key-here' > tavily_api.txt")
129 |
130 |
131 | def install_dependencies():
132 | """Install Python dependencies."""
133 | print_step("Installing dependencies...")
134 |
135 | try:
136 | subprocess.run(
137 | [sys.executable, "-m", "pip", "install", "-r", "requirements.txt"],
138 | check=True,
139 | capture_output=True,
140 | text=True,
141 | )
142 | print("✅ Dependencies installed successfully")
143 | except subprocess.CalledProcessError as e:
144 | print(f"❌ Failed to install dependencies: {e}")
145 | print("Please run: pip install -r requirements.txt")
146 |
147 |
148 | def validate_setup():
149 | """Validate the setup."""
150 | print_step("Validating setup...")
151 |
152 | # Check if main modules can be imported
153 | try:
154 | import clients
155 | import config
156 | import main
157 | import scan_research
158 |
159 | print("✅ All modules can be imported")
160 | except ImportError as e:
161 | print(f"❌ Import error: {e}")
162 | return False
163 |
164 | # Check if directories exist
165 | required_dirs = ["logs", "instances"]
166 | for directory in required_dirs:
167 | if not Path(directory).exists():
168 | print(f"❌ Directory missing: {directory}")
169 | return False
170 |
171 | print("✅ Setup validation passed")
172 | return True
173 |
174 |
175 | def main():
176 | """Main setup function."""
177 | print_header("Universal Deep Research Backend (UDR-B) Setup")
178 |
179 | # Change to script directory
180 | script_dir = Path(__file__).parent
181 | os.chdir(script_dir)
182 |
183 | check_python_version()
184 | create_directories()
185 | setup_env_file()
186 | check_api_keys()
187 | install_dependencies()
188 |
189 | if validate_setup():
190 | print_header("Setup Complete!")
191 | print("🎉 Your Universal Deep Research Backend (UDR-B) is ready!")
192 | print("\nNext steps:")
193 | print("1. Edit .env file with your configuration")
194 | print("2. Create API key files (nvdev_api.txt, tavily_api.txt)")
195 | print("3. Run: ./launch_server.sh")
196 | print("4. Or run: uvicorn main:app --reload")
197 | print("\nFor more information, see README.md")
198 | else:
199 | print_header("Setup Failed")
200 | print("❌ Please fix the issues above and run setup again")
201 |
202 |
203 | if __name__ == "__main__":
204 | main()
205 |
--------------------------------------------------------------------------------
/frontend/README.md:
--------------------------------------------------------------------------------
1 | # NVR Universal Deep Research - Frontend
2 |
3 | A Next.js frontend application for the NVR Universal Deep Research system that provides an interactive interface for intelligent research and reporting capabilities.
4 |
5 | This software is provided exclusively for research and demonstration purposes. It is intended solely as a prototype to demonstrate research concepts and methodologies in artificial intelligence and automated research systems.
6 |
7 | - This software is not intended for production deployment, commercial use, or any real-world application where reliability, accuracy, or safety is required.
8 | - This software contains experimental features, unproven methodologies, and research-grade implementations that may contain bugs, security vulnerabilities, or other issues.
9 | - The software is provided "AS IS" without any warranties. Neither NVIDIA Corporation nor the authors shall be liable for any damages arising from the use of this software to the fullest extent permitted by law.
10 |
11 | By using this software, you acknowledge that you have read and understood the complete DISCLAIMER file and agree to be bound by its terms. For the complete legal disclaimer, please see the [DISCLAIMER](DISCLAIMER.txt) file in this directory.
12 |
13 | ## Features
14 |
15 | - Interactive research interface with real-time progress tracking
16 | - Configurable research strategies
17 | - Support for both V1 and V2 API endpoints
18 | - Dry run mode for testing
19 | - Real-time report generation and viewing
20 |
21 | ## Configuration
22 |
23 | The frontend application is highly configurable through environment variables. Copy the example environment file and customize it for your deployment:
24 |
25 | ```bash
26 | cp env.example .env.local
27 | ```
28 |
29 | ### Environment Variables
30 |
31 | #### Backend API Configuration
32 |
33 | - `NEXT_PUBLIC_BACKEND_BASE_URL`: The base URL of your backend server (default: `http://localhost`)
34 | - `NEXT_PUBLIC_BACKEND_PORT`: The port your backend server is running on (default: `8000`)
35 | - `NEXT_PUBLIC_API_VERSION`: API version to use - `v1` or `v2` (default: `v2`)
36 |
37 | #### Runtime Configuration
38 |
39 | - `NEXT_PUBLIC_DRY_RUN`: Enable dry run mode - `true` or `false` (default: `false`)
40 | - `NEXT_PUBLIC_ENABLE_V2_API`: Enable V2 API - `true` or `false` (default: `true`)
41 |
42 | #### Frontend Configuration
43 |
44 | - `NEXT_PUBLIC_FRONTEND_PORT`: The port for the frontend development server (default: `3000`)
45 | - `NEXT_PUBLIC_FRONTEND_HOST`: The host for the frontend development server (default: `localhost`)
46 |
47 | ### Example Configuration
48 |
49 | For a production deployment with a backend on a different server:
50 |
51 | ```bash
52 | NEXT_PUBLIC_BACKEND_BASE_URL=http://your-backend-server.com
53 | NEXT_PUBLIC_BACKEND_PORT=8000
54 | NEXT_PUBLIC_API_VERSION=v2
55 | NEXT_PUBLIC_DRY_RUN=false
56 | NEXT_PUBLIC_ENABLE_V2_API=true
57 | ```
58 |
59 | For local development:
60 |
61 | ```bash
62 | NEXT_PUBLIC_BACKEND_BASE_URL=http://localhost
63 | NEXT_PUBLIC_BACKEND_PORT=8000
64 | NEXT_PUBLIC_API_VERSION=v2
65 | NEXT_PUBLIC_DRY_RUN=true
66 | NEXT_PUBLIC_ENABLE_V2_API=true
67 | ```
68 |
69 | ## Getting Started
70 |
71 | 1. **Install dependencies:**
72 |
73 | ```bash
74 | npm install
75 | ```
76 |
77 | 2. **Configure environment variables:**
78 |
79 | ```bash
80 | cp env.example .env.local
81 | # Edit .env.local with your configuration
82 | ```
83 |
84 | 3. **Run the development server:**
85 |
86 | ```bash
87 | npm run dev
88 | ```
89 |
90 | 4. **Open your browser:**
91 | Navigate to [http://localhost:3000](http://localhost:3000) to see the application.
92 |
93 | ## Available Scripts
94 |
95 | - `npm run dev` - Start the development server with Turbopack
96 | - `npm run build` - Build the application for production
97 | - `npm run start` - Start the production server
98 | - `npm run lint` - Run ESLint
99 |
100 | ## Project Structure
101 |
102 | ```
103 | src/
104 | ├── app/ # Next.js app directory
105 | │ ├── page.tsx # Main application page
106 | │ ├── layout.tsx # Root layout
107 | │ └── globals.css # Global styles
108 | ├── components/ # React components
109 | │ ├── PromptBar.tsx # Main input interface
110 | │ ├── ResearchProgressList.tsx # Progress tracking
111 | │ ├── ReportViewer.tsx # Report display
112 | │ └── ... # Other UI components
113 | ├── config/ # Configuration management
114 | │ └── index.ts # App configuration
115 | └── types/ # TypeScript type definitions
116 | └── ApplicationState.ts
117 | ```
118 |
119 | ## Backend Requirements
120 |
121 | This frontend requires a compatible backend server running the NVR Universal Deep Research API. The backend should:
122 |
123 | - Support both `/api/research` (V1) and `/api/research2` (V2) endpoints
124 | - Accept POST requests with JSON payloads
125 | - Return Server-Sent Events (SSE) for real-time progress updates
126 | - Support the dry run mode parameter
127 |
128 | ## Deployment
129 |
130 | The application can be deployed to any platform that supports Next.js:
131 |
132 | 1. Build the application: `npm run build`
133 | 2. Start the production server: `npm run start`
134 | 3. Set the appropriate environment variables for your deployment
135 |
136 | ### Example Deployment Platforms
137 |
138 | - **Vercel**: Connect your Git repository and set environment variables in the dashboard
139 | - **Netlify**: Build and deploy with environment variable configuration
140 | - **Railway**: Deploy with automatic environment variable management
141 | - **Self-hosted**: Run on your own server with proper environment configuration
142 |
143 | ## Troubleshooting
144 |
145 | ### Common Issues
146 |
147 | 1. **Backend Connection Errors**: Ensure your backend server is running and the `NEXT_PUBLIC_BACKEND_BASE_URL` and `NEXT_PUBLIC_BACKEND_PORT` are correctly configured.
148 |
149 | 2. **CORS Errors**: Make sure your backend server allows requests from your frontend domain.
150 |
151 | 3. **API Version Issues**: If you're experiencing API compatibility issues, try switching between `v1` and `v2` using the `NEXT_PUBLIC_API_VERSION` environment variable.
152 |
153 | ### Development Tips
154 |
155 | - Use `NEXT_PUBLIC_DRY_RUN=true` during development to avoid making actual API calls
156 | - The application supports hot reloading - changes to your code will be reflected immediately
157 | - Check the browser console for detailed error messages
158 |
159 | ## Contributing
160 |
161 | 1. Fork the repository
162 | 2. Create a feature branch
163 | 3. Make your changes
164 | 4. Test thoroughly
165 | 5. Submit a pull request
166 |
167 | ## License and Disclaimer
168 |
169 | This software is provided for research and demonstration purposes only. Please refer to the [DISCLAIMER](DISCLAIMER.txt) file for complete terms and conditions regarding the use of this software. You can find the license in [LICENSE](LICENSE.txt).
170 |
171 | **Do not use this code in production.**
172 |
--------------------------------------------------------------------------------
/frontend/src/components/ReportViewer.module.css:
--------------------------------------------------------------------------------
1 | /*
2 | * SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3 | * SPDX-License-Identifier: Apache-2.0
4 | *
5 | * Licensed under the Apache License, Version 2.0 (the "License");
6 | * you may not use this file except in compliance with the License.
7 | * You may obtain a copy of the License at
8 | *
9 | * http://www.apache.org/licenses/LICENSE-2.0
10 | *
11 | * Unless required by applicable law or agreed to in writing, software
12 | * distributed under the License is distributed on an "AS IS" BASIS,
13 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 | * See the License for the specific language governing permissions and
15 | * limitations under the License.
16 | */
17 | .wrapper {
18 | width: 100%;
19 | height: 100%;
20 | display: flex;
21 | flex-direction: column;
22 | }
23 |
24 | .content {
25 | width: 100%;
26 | height: 100%;
27 | opacity: 0;
28 | transform: translateY(20px);
29 | animation: fadeInUp 0.5s ease-out forwards;
30 | }
31 |
32 | @keyframes fadeInUp {
33 | 0% {
34 | opacity: 0;
35 | transform: translateY(20px);
36 | }
37 | 100% {
38 | opacity: 1;
39 | transform: translateY(0);
40 | }
41 | }
42 |
43 | .container {
44 | width: 100%;
45 | max-height: 100%;
46 | min-height: 0; /* Allows container to shrink below max-height */
47 | display: flex;
48 | flex-direction: column;
49 | background-color: rgb(243 244 246);
50 | border-radius: 1rem;
51 | overflow: hidden;
52 | transition: all 0.3s ease-out;
53 | box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1),
54 | 0 2px 4px -1px rgba(0, 0, 0, 0.06);
55 | }
56 |
57 | .header {
58 | height: 3.5rem;
59 | border-bottom: 1px solid rgb(229 231 235);
60 | background-color: rgb(243 244 246);
61 | display: flex;
62 | align-items: center;
63 | justify-content: space-between;
64 | padding: 1rem;
65 | }
66 |
67 | .title {
68 | color: rgb(17 24 39);
69 | font-size: 1rem;
70 | font-weight: 600;
71 | }
72 |
73 | .reportContent {
74 | flex: 1;
75 | overflow-y: auto;
76 | padding: 1rem;
77 | scrollbar-width: none;
78 | -ms-overflow-style: none;
79 | transition: all 0.3s ease-out;
80 | }
81 |
82 | .reportContent::-webkit-scrollbar {
83 | display: none;
84 | }
85 |
86 | .reportContent:hover {
87 | scrollbar-width: thin;
88 | -ms-overflow-style: auto;
89 | }
90 |
91 | .reportContent:hover::-webkit-scrollbar {
92 | display: block;
93 | width: 8px;
94 | }
95 |
96 | .reportContent:hover::-webkit-scrollbar-track {
97 | background: rgb(229 231 235);
98 | border-radius: 4px;
99 | }
100 |
101 | .reportContent:hover::-webkit-scrollbar-thumb {
102 | background: rgb(156 163 175);
103 | border-radius: 4px;
104 | }
105 |
106 | /* Markdown styling */
107 | .reportContent h1 {
108 | font-size: 1.5rem;
109 | font-weight: 600;
110 | margin-top: 1.5rem;
111 | margin-bottom: 1rem;
112 | color: rgb(17 24 39);
113 | }
114 |
115 | .reportContent h2 {
116 | font-size: 1.25rem;
117 | font-weight: 600;
118 | margin-top: 1.25rem;
119 | margin-bottom: 0.75rem;
120 | color: rgb(17 24 39);
121 | }
122 |
123 | .reportContent h3 {
124 | font-size: 1.125rem;
125 | font-weight: 600;
126 | margin-top: 1rem;
127 | margin-bottom: 0.5rem;
128 | color: rgb(17 24 39);
129 | }
130 |
131 | .reportContent p {
132 | margin-bottom: 1rem;
133 | line-height: 1.6;
134 | color: rgb(55 65 81);
135 | }
136 |
137 | .reportContent ul,
138 | .reportContent ol {
139 | margin-bottom: 1rem;
140 | padding-left: 1.5rem;
141 | color: rgb(55 65 81);
142 | }
143 |
144 | .reportContent li {
145 | margin-bottom: 0.5rem;
146 | line-height: 1.6;
147 | }
148 |
149 | .reportContent a {
150 | color: rgb(79 70 229);
151 | text-decoration: none;
152 | }
153 |
154 | .reportContent a:hover {
155 | text-decoration: underline;
156 | }
157 |
158 | .reportContent code {
159 | background-color: rgb(229 231 235);
160 | padding: 0.2rem 0.4rem;
161 | border-radius: 0.25rem;
162 | font-family: monospace;
163 | font-size: 0.875rem;
164 | color: rgb(17 24 39);
165 | }
166 |
167 | .reportContent pre {
168 | background-color: rgb(17 24 39);
169 | padding: 1rem;
170 | border-radius: 0.5rem;
171 | overflow-x: auto;
172 | margin-bottom: 1rem;
173 | }
174 |
175 | .reportContent pre code {
176 | background-color: transparent;
177 | padding: 0;
178 | color: rgb(209 213 219);
179 | }
180 |
181 | .reportContent blockquote {
182 | border-left: 4px solid rgb(156 163 175);
183 | padding-left: 1rem;
184 | color: rgb(107 114 128);
185 | margin-bottom: 1rem;
186 | }
187 |
188 | .reportContent hr {
189 | border: 0;
190 | height: 1px;
191 | background-color: rgb(229 231 235);
192 | margin: 0.5rem 0;
193 | position: relative;
194 | }
195 |
196 | .reportContent hr::before {
197 | content: "";
198 | position: absolute;
199 | top: -0.125rem;
200 | left: 0;
201 | right: 0;
202 | height: 0.125rem;
203 | background: transparent;
204 | }
205 |
206 | .reportContent hr::after {
207 | content: "";
208 | position: absolute;
209 | bottom: -0.125rem;
210 | left: 0;
211 | right: 0;
212 | height: 0.125rem;
213 | background: transparent;
214 | }
215 |
216 | .reportContent table {
217 | width: 100%;
218 | border-collapse: collapse;
219 | margin-bottom: 1rem;
220 | }
221 |
222 | .reportContent th,
223 | .reportContent td {
224 | border: 1px solid rgb(229 231 235);
225 | padding: 0.5rem 0.75rem;
226 | text-align: left;
227 | }
228 |
229 | .reportContent th {
230 | background-color: rgb(243 244 246);
231 | font-weight: 600;
232 | }
233 |
234 | /* Dark mode styles */
235 | @media (prefers-color-scheme: dark) {
236 | .container,
237 | .header {
238 | background-color: rgb(31 41 55);
239 | }
240 |
241 | .header {
242 | border-color: rgb(55 65 81);
243 | }
244 |
245 | .title {
246 | color: rgb(243 244 246);
247 | }
248 |
249 | .reportContent h1,
250 | .reportContent h2,
251 | .reportContent h3 {
252 | color: rgb(243 244 246);
253 | }
254 |
255 | .reportContent p,
256 | .reportContent li {
257 | color: rgb(209 213 219);
258 | }
259 |
260 | .reportContent a {
261 | color: rgb(129 140 248);
262 | }
263 |
264 | .reportContent code {
265 | background-color: rgb(55 65 81);
266 | color: rgb(209 213 219);
267 | }
268 |
269 | .reportContent pre {
270 | background-color: rgb(17 24 39);
271 | }
272 |
273 | .reportContent pre code {
274 | color: rgb(243 244 246);
275 | }
276 |
277 | .reportContent blockquote {
278 | border-color: rgb(107 114 128);
279 | color: rgb(156 163 175);
280 | }
281 |
282 | .reportContent th,
283 | .reportContent td {
284 | border-color: rgb(55 65 81);
285 | }
286 |
287 | .reportContent th {
288 | background-color: rgb(31 41 55);
289 | }
290 |
291 | .reportContent:hover::-webkit-scrollbar-track {
292 | background: rgb(55 65 81);
293 | }
294 |
295 | .reportContent:hover::-webkit-scrollbar-thumb {
296 | background: rgb(107 114 128);
297 | }
298 |
299 | .reportContent hr {
300 | background-color: rgb(75 85 99);
301 | }
302 | }
303 |
--------------------------------------------------------------------------------
/backend/frame/errands4.py:
--------------------------------------------------------------------------------
1 | # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2 | # SPDX-License-Identifier: Apache-2.0
3 | #
4 | # Licensed under the Apache License, Version 2.0 (the "License");
5 | # you may not use this file except in compliance with the License.
6 | # You may obtain a copy of the License at
7 | #
8 | # http://www.apache.org/licenses/LICENSE-2.0
9 | #
10 | # Unless required by applicable law or agreed to in writing, software
11 | # distributed under the License is distributed on an "AS IS" BASIS,
12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | # See the License for the specific language governing permissions and
14 | # limitations under the License.
15 | import re
16 |
17 | from frame.clients import Client
18 |
19 |
20 | def namify_model_path(hf_model_path: str) -> str:
21 | return hf_model_path.replace("/", "_").replace("-", "_").replace(" ", "_")
22 |
23 |
24 | class Errand:
25 | def __init__(self, pre_prompt: str, prompt: str) -> None:
26 | self.pre_prompt = pre_prompt
27 | self.prompt = prompt
28 |
29 | def run(self, runner: Client, args: dict, completion_config: dict = {}) -> str:
30 | prompt_with_args = self.prompt
31 | for arg_name, arg_value in args.items():
32 | prompt_with_args = prompt_with_args.replace("{" + arg_name + "}", arg_value)
33 |
34 | ret = runner.run(self.pre_prompt, prompt_with_args, completion_config)
35 | return ret
36 |
37 |
38 | class FileErrand(
39 | Errand
40 | ): # NOTE: this is only for easier debug for now; production code should use errands directly
41 | def __init__(self, errand_path: str) -> None:
42 | with open(errand_path, "r") as f:
43 | errand = f.read()
44 |
45 | errand_parts = errand.split("===SEPARATOR===")
46 | if len(errand_parts) != 2:
47 | raise ValueError(
48 | f"Errand file {errand_path} is not valid. It should contain exactly one separator."
49 | )
50 | pre_prompt = errand_parts[0].strip()
51 | prompt = errand_parts[1].strip()
52 | super().__init__(pre_prompt, prompt)
53 |
54 |
55 | class MultipleChoiceErrand(FileErrand):
56 | def __init__(self, errand_path: str, choices: list[str]) -> None:
57 | super().__init__(errand_path)
58 | self.choices = choices
59 |
60 | def run(
61 | self, runner: Client, args: dict, completion_config: dict = {}
62 | ) -> str | None:
63 | raw_completion = super().run(runner, args, completion_config)
64 |
65 | for choice in self.choices:
66 | if choice in raw_completion:
67 | return choice
68 |
69 | return None
70 |
71 |
72 | import os
73 | from pathlib import Path
74 |
75 |
76 | class MessageTypeErrand(MultipleChoiceErrand):
77 | def __init__(self) -> None:
78 | errand_path = os.path.join(
79 | Path(__file__).resolve().parent, "errands/message_type.txt"
80 | )
81 | choices = [
82 | "code_skill",
83 | "routine_skill",
84 | "query_skill",
85 | "code",
86 | "routine",
87 | "query",
88 | "data",
89 | ]
90 | super().__init__(errand_path, choices)
91 |
92 |
93 | class MessageCodeProcessingErrand(FileErrand):
94 | def __init__(self) -> None:
95 | errand_path = os.path.join(
96 | Path(__file__).resolve().parent, "errands/message_code_processing.txt"
97 | )
98 | super().__init__(errand_path)
99 |
100 |
101 | class MessageCodeSkillProcessingErrand(FileErrand):
102 | def __init__(self) -> None:
103 | errand_path = os.path.join(
104 | Path(__file__).resolve().parent, "errands/message_code_skill_processing.txt"
105 | )
106 | super().__init__(errand_path)
107 |
108 |
109 | class MessageCodeCallErrand(FileErrand):
110 | def __init__(self) -> None:
111 | errand_path = os.path.join(
112 | Path(__file__).resolve().parent, "errands/message_code_call.txt"
113 | )
114 | super().__init__(errand_path)
115 |
116 |
117 | class MessageCodeVariablesErrand(FileErrand):
118 | def __init__(self) -> None:
119 | errand_path = os.path.join(
120 | Path(__file__).resolve().parent, "errands/message_code_variables.txt"
121 | )
122 | super().__init__(errand_path)
123 |
124 |
125 | class MessageRoutineProcessingErrand(FileErrand):
126 | def __init__(self) -> None:
127 | errand_path = os.path.join(
128 | Path(__file__).resolve().parent, "errands/message_routine_processing.txt"
129 | )
130 | super().__init__(errand_path)
131 |
132 |
133 | class MessageGeneratingRoutineProcessingErrand(FileErrand):
134 | def __init__(self) -> None:
135 | errand_path = os.path.join(
136 | Path(__file__).resolve().parent,
137 | "errands/message_generating_routine_processing.txt",
138 | )
139 | super().__init__(errand_path)
140 |
141 |
142 | class MessageRoutineCallErrand(FileErrand):
143 | def __init__(self) -> None:
144 | errand_path = os.path.join(
145 | Path(__file__).resolve().parent, "errands/message_routine_call.txt"
146 | )
147 | super().__init__(errand_path)
148 |
149 |
150 | class MessageGeneratingRoutineCallErrand(FileErrand):
151 | def __init__(self) -> None:
152 | errand_path = os.path.join(
153 | Path(__file__).resolve().parent,
154 | "errands/message_generating_routine_call.txt",
155 | )
156 | super().__init__(errand_path)
157 |
158 |
159 | class MessageRoutineVariablesErrand(FileErrand):
160 | def __init__(self) -> None:
161 | errand_path = os.path.join(
162 | Path(__file__).resolve().parent, "errands/message_routine_variables.txt"
163 | )
164 | super().__init__(errand_path)
165 |
166 |
167 | class MessageDataProcessingErrand(FileErrand):
168 | def __init__(self) -> None:
169 | errand_path = os.path.join(
170 | Path(__file__).resolve().parent, "errands/message_data_processing.txt"
171 | )
172 | super().__init__(errand_path)
173 |
174 |
175 | default_errand_profile: dict[str, type[Errand] | None] = {
176 | "message_type": MessageTypeErrand,
177 | "message_code_processing": MessageCodeProcessingErrand,
178 | "message_code_skill_processing": MessageCodeSkillProcessingErrand,
179 | "message_code_call": MessageCodeCallErrand,
180 | "message_code_variables": MessageCodeVariablesErrand,
181 | "message_routine_processing": MessageRoutineProcessingErrand,
182 | "message_generating_routine_processing": MessageGeneratingRoutineProcessingErrand,
183 | "message_routine_call": MessageRoutineCallErrand,
184 | "message_generating_routine_call": MessageGeneratingRoutineCallErrand,
185 | "message_routine_variables": MessageRoutineVariablesErrand,
186 | "message_data_processing": MessageDataProcessingErrand,
187 | "language_model": None, # on purpose, currently there is no errand but there might be a client profile # TODO: double-check if this is still needed in V4
188 | }
189 |
--------------------------------------------------------------------------------
/backend/frame/errands/message_code_processing.txt:
--------------------------------------------------------------------------------
1 | You are a helpful assistant for extracting and preparing executable Python code from user messages. Your task is to process the MESSAGE below, which may contain code (in any language or pseudo-code), natural language instructions, or a combination of both. Your goal is to output only the final, executable Python code, wrapped in a function named `code`, making any necessary modifications as described below. The function must return a tuple: (output: str, variables: dict[str, Any]), where the first element is the output string (what would be printed) and the second is a dictionary of all variables that have been modified in the original code segments/code modification instructions provided by the user.
2 |
3 | ===SEPARATOR===
4 | You are given a MESSAGE that may contain:
5 | - Python code, code in another language, or pseudo-code
6 | - Natural language instructions that may request modifications to code provided, corrections, or translation to Python
7 | - A mix of code blocks (e.g., in markdown backticks, tags, or inline) and surrounding text
8 |
9 | You are also given TIDINGS, which are variables, values, or context objects provided by the system. If you decide to use any of the tidings, the same name must also appear as an argument. Tidings are not globals -- they will be passed to the generated function in a function call as arguments. Each tiding is provided as a name, value, and description. If the MESSAGE requests code that uses a variable or value present in the tidings, use the tiding as an argument in the function. If the MESSAGE does not reference any tidings, you may ignore them.
10 |
11 | Your job is to:
12 | 1. Extract all code that is intended for execution, as indicated by the message and its instructions.
13 | 2. If the code is not in Python, translate it to Python as faithfully as possible, using standard libraries and idiomatic Python where appropriate.
14 | 3. If the code is pseudo-code or contains small errors (e.g., typos, misspelled variable names, or incorrect standard library usage), correct these issues to produce valid, executable Python code.
15 | 4. If the surrounding instructions request modifications (e.g., change a variable name, add a print statement, use a different algorithm), make those changes in the final code.
16 | 5. If there are multiple code blocks or fragments, combine them as needed to produce a single, coherent Python function that fulfills the user's intent.
17 | 6. Wrap all code in a function named `code`. The function must have a docstring describing what it does, based on the user's message and intent.
18 | 7. Any reference to external variables (not defined within the function) should be marked with a comment indicating that the variable is external and must be provided by the caller, unless it is provided as a tiding.
19 | 8. Any output that would be printed should instead be appended to a string accumulator (e.g., `__output`), and the function should return a tuple: (output, variables), where `output` is the string and `variables` is a dictionary of all variables that have been either modified in the original code segment present in the message or added/modified by the surrounding text.
20 | 9. Do not include any explanatory text, comments about your process, or formatting (such as markdown backticks) in your output—output only the final Python function definition.
21 |
22 | Examples:
23 |
24 | Example 1:
25 | MESSAGE:
26 | Please execute the following code:
27 | ```
28 | x = 5
29 | y = 10
30 | print(x + y)
31 | ```
32 | TIDINGS:
33 |
34 | OUTPUT:
35 | def code():
36 | """Execute the given code and return the output string and modified variables as a tuple."""
37 | output = ""
38 | try:
39 | x = 5
40 | y = 10
41 | output += str(x + y) + "\n"
42 | return output, { 'x': x, 'y': y }
43 | except Exception as e:
44 | return output + "\n\n" + str(e), { 'x': x, 'y': y }
45 |
46 | Example 2:
47 | MESSAGE:
48 | Run this JavaScript code, but in Python:
49 | ```
50 | let arr = [1, 2, 3];
51 | console.log(arr.map(x => x * 2));
52 | ```
53 | TIDINGS:
54 |
55 | OUTPUT:
56 | def code():
57 | """Double each element in the array and return the result string and newly defined / modified variables as a tuple."""
58 | output = ""
59 | try:
60 | arr = [1, 2, 3]
61 | doubled = [x * 2 for x in arr]
62 | output += str(doubled) + "\n"
63 | return output, { 'arr': arr, 'doubled': doubled }
64 | except Exception as e:
65 | return output + "\n\n" + str(e), { 'arr': arr, 'doubled': doubled }
66 |
67 | Example 3:
68 | MESSAGE:
69 | Here's some pseudo-code. Please execute it after translating to Python:
70 | - Set total to 0
71 | - For each number in [1,2,3,4], add it to total
72 | - Respond with total to me
73 | TIDINGS:
74 |
75 | OUTPUT:
76 | def code():
77 | """Sum the numbers in the list and return the total string and newly defined / modified variables as a tuple."""
78 | output = ""
79 | try:
80 | total = 0
81 | for number in [1, 2, 3, 4]:
82 | total += number
83 | output += str(total) + "\n"
84 | return output, { 'total': total }
85 | except Exception as e:
86 | return output + "\n\n" + str(e), { 'total': total }
87 |
88 | Example 4:
89 | MESSAGE:
90 | Execute this, but change the variable name from 'foo' to 'bar':
91 | ```
92 | foo = 42
93 | print(foo)
94 | ```
95 | TIDINGS:
96 |
97 | OUTPUT:
98 | def code():
99 | """Assign 42 to bar and return its value string and variables as a tuple."""
100 | output = ""
101 | try:
102 | bar = 42
103 | output += str(bar) + "\n"
104 | return output, { 'bar': bar }
105 | except Exception as e:
106 | return output + "\n\n" + str(e), { 'bar': bar }
107 |
108 | Example 5:
109 | MESSAGE:
110 | Please run this code (note: there is a typo in the function name):
111 | ```
112 | pritn('Hello, world!')
113 | ```
114 | TIDINGS:
115 |
116 | OUTPUT:
117 | def code():
118 | """Print 'Hello, world!' and return the output string and variables as a tuple."""
119 | output = ""
120 | try:
121 | output += 'Hello, world!\n'
122 | return output, {}
123 | except Exception as e:
124 | return output + "\n\n" + str(e), {}
125 |
126 | Example 6:
127 | MESSAGE:
128 | Use the provided user_name to greet the user.
129 | TIDINGS:
130 | user_name: Alice # The name of the user to greet
131 |
132 | OUTPUT:
133 | def code(user_name):
134 | """Greet the user by name using the provided user_name tiding, and return the output string and new/modified variables as a tuple."""
135 | output = ""
136 | try:
137 | output += f"Hello, {user_name}!\n"
138 | return output, { }
139 | except Exception as e:
140 | return output + "\n\n" + str(e), { }
141 |
142 | Reminders:
143 | - Output only the final, executable Python function definition, with no extra text or formatting.
144 | - Make reasonable corrections and modifications as requested or needed.
145 | - If translation or correction is not possible, output the closest valid Python function that fulfills the user's intent.
146 | - The function must always return a tuple: (output, variables), where output is a string and variables is a dictionary.
147 | - Do not output any surrounding backticks (```) or code tags—output just the code directly.
148 |
149 | This is the end of the classification instructions. Below is the input to classify.
150 |
151 | MESSAGE:
152 | {message}
153 | TIDINGS:
154 | {tidings}
--------------------------------------------------------------------------------
/backend/frame/errands/message_routine_processing.txt:
--------------------------------------------------------------------------------
1 | You are an expert Python code generator for an advanced AI system. Your job is to convert a user's natural language routine instruction (MESSAGE) into a single pure Python function called 'code'.
2 |
3 | ===SEPARATOR===
4 |
5 | You are an expert Python code generator for an advanced AI system. Convert user's natural language routine instruction (MESSAGE) into a single pure Python function called 'code'.
6 |
7 | You are provided with:
8 | - MESSAGE: The user's instruction, which may be a single short description of intended functionality or a multi-step list (bullets, numbered steps, etc.).
9 | - SKILLS: A list of available function signatures and docstrings you may call in your code.
10 | - TIDINGS: A list of available variables (with their values and descriptions) you may use as arguments or modify.
11 |
12 | Your function must:
13 | - Be named 'code' and use type hints for all arguments and the return value.
14 | - Accept as arguments any TIDINGS that are relevant to the MESSAGE.
15 | - Accumulate all user-facing output in a string variable called '__output' (do NOT use print). At the end, return this string as the first element of a tuple.
16 | - Return a tuple (output: str, modified_vars: dict[str, Any]), where 'output' is the accumulated output string, and 'modified_vars' is a dictionary of all variables that were modified by the function.
17 | - Have a high-quality docstring describing its purpose, arguments, and return value.
18 | - Use SKILLS as needed to accomplish the MESSAGE.
19 | - If the MESSAGE is a multi-step routine (bullets, numbered steps, or multiple sentences), interleave each step as a comment, followed by the code that implements that step. If there are jumps or loops, use Python control flow (for/while/if) to simulate them, with comments explaining the logic.
20 | - When in doubt, generate code that best accomplishes the user's intent.
21 | - Do not use print statements; all output must be accumulated in '__output'.
22 | - Modify variables that are listed in TIDINGS when referred to directly in natural language
23 | - Feel free to create new variables when the MESSAGE instructs to create new ones.
24 | - Sometimes you might want to create auxiliary/temporary variables. When that's the case, please prefix them by `__` (double underscore).
25 | - If the MESSAGE contains a payload text (for example, separated by a heading or clearly not an instruction), store that text without modification in a new variable with an appropriate name and a comment explaining its purpose. Do not process or change the payload text; just store it. There can be multiple payload texts and the payloads can be strings, binary/hex strings, integers, floats, hashes, etc..
26 |
27 | Example input:
28 | MESSAGE:
29 | """
30 | 1. Greet the user by name.
31 | 2. Add 10 to their score.
32 | 3. If their score is above 100, congratulate them.
33 | """
34 | SKILLS:
35 | - greet_user(name: str) -> str: Returns a greeting for the user.
36 | TIDINGS:
37 | - name = Alice # The user's name
38 | - score = 95 # The user's current score
39 |
40 | Example output:
41 | ```python
42 | def code(name: str, score: int) -> tuple[str, dict[str, Any]]:
43 | """
44 | Greets the user, updates their score, and congratulates them if their score exceeds 100.
45 | Args:
46 | name: The user's name.
47 | score: The user's current score.
48 | Returns:
49 | A tuple containing the output string and a dictionary of modified variables.
50 | """
51 | __output = ""
52 | # 1. Greet the user by name.
53 | __output += greet_user(name) + "\n"
54 | # 2. Add 10 to their score.
55 | score += 10
56 | # 3. If their score is above 100, congratulate them.
57 | if score > 100:
58 | __output += f"Congratulations, {name}! Your score is {score}.\n"
59 | __vars = {'score': score}
60 | return __output, __vars
61 | ```
62 |
63 | Example input:
64 | MESSAGE:
65 | """
66 | 1. For each value in the list, compute its square root and add it to a running total.
67 | 2. If the total exceeds 100, output a warning message.
68 | 3. Output the final total.
69 | """
70 | SKILLS:
71 | - format_warning(msg: str) -> str: Formats a warning message for the user.
72 | TIDINGS:
73 | - values = [4, 16, 25, 36, 49] # List of numbers
74 | - total = 0 # Running total
75 |
76 | Example output:
77 | ```python
78 | def code(values: list[float], total: float) -> tuple[str, dict[str, Any]]:
79 | """
80 | Computes the square root of each value, adds to total, warns if total > 100, and outputs the total.
81 | Args:
82 | values: List of numbers to process.
83 | total: Running total.
84 | Returns:
85 | A tuple containing the output string and a dictionary of modified variables.
86 | """
87 | import math
88 | __output = ""
89 | # 1. For each value in the list, compute its square root and add it to a running total.
90 | for __v in values:
91 | __sqrt = math.sqrt(__v)
92 | total += __sqrt
93 | # 2. If the total exceeds 100, output a warning message.
94 | if total > 100:
95 | __output += format_warning(f"Total exceeded 100: {total}") + "\n"
96 | # 3. Output the final total.
97 | __output += f"Final total: {total}\n"
98 | __vars = {'total': total}
99 | return __output, __vars
100 | ```
101 |
102 | Example input:
103 | MESSAGE:
104 | """
105 | 1. Check if the user is an admin.
106 | 2. If so, grant them access and log the event.
107 | 3. Otherwise, deny access and log the attempt.
108 | """
109 | SKILLS:
110 | - log_event(event: str) -> None: Logs an event string.
111 | - grant_access(user: str) -> str: Grants access to the user and returns a confirmation message.
112 | - deny_access(user: str) -> str: Denies access to the user and returns a denial message.
113 | TIDINGS:
114 | - user = bob # The username
115 | - is_admin = False # Whether the user is an admin
116 |
117 | Example output:
118 | ```python
119 | def code(user: str, is_admin: bool) -> tuple[str, dict[str, Any]]:
120 | """
121 | Checks admin status, grants or denies access, and logs the event.
122 | Args:
123 | user: The username.
124 | is_admin: Whether the user is an admin.
125 | Returns:
126 | A tuple containing the output string and a dictionary of modified variables.
127 | """
128 | __output = ""
129 | # 1. Check if the user is an admin.
130 | if is_admin:
131 | # 2. If so, grant them access and log the event.
132 | __output += grant_access(user) + "\n"
133 | log_event(f"Access granted to {user}")
134 | else:
135 | # 3. Otherwise, deny access and log the attempt.
136 | __output += deny_access(user) + "\n"
137 | log_event(f"Access denied to {user}")
138 | __vars = {}
139 | return __output, __vars
140 | ```
141 |
142 | Reminders:
143 | - Output a function called `code` relying on SKILLS and TIDINGS where needed
144 | - The function should be a pure function -- do not access global variables.
145 | - Prefer the functions described in SKILLS over external libraries
146 | - Output just the code of the function Do not output any other text.
147 |
148 | Now, generate the function as described above for the given MESSAGE, SKILLS, and TIDINGS.
149 |
150 | SKILLS:
151 | {skills}
152 |
153 | TIDINGS:
154 | {tidings}
155 |
156 | MESSAGE:
157 | {message}
158 |
159 | Reminders:
160 | - Output a function called `code` relying on SKILLS and TIDINGS where needed
161 | - The function should be a pure function -- do not access global variables.
162 | - Prefer the functions described in SKILLS over external libraries; these functions are available in the global scope and do not require imports.
163 | - Output just the code of the function Do not output any other text.
164 | - Remember to return two outputs: one for the aggregated function output, and one that is a dictionary of all variables that were modified by the function.
--------------------------------------------------------------------------------
/backend/frame/clients.py:
--------------------------------------------------------------------------------
1 | # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2 | # SPDX-License-Identifier: Apache-2.0
3 | #
4 | # Licensed under the Apache License, Version 2.0 (the "License");
5 | # you may not use this file except in compliance with the License.
6 | # You may obtain a copy of the License at
7 | #
8 | # http://www.apache.org/licenses/LICENSE-2.0
9 | #
10 | # Unless required by applicable law or agreed to in writing, software
11 | # distributed under the License is distributed on an "AS IS" BASIS,
12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | # See the License for the specific language governing permissions and
14 | # limitations under the License.
15 | from .trace import Trace
16 |
17 |
18 | class Client:
19 | def __init__(self, trace: Trace = None) -> None:
20 | self.trace = trace
21 |
22 | def run(self, pre_prompt: str, prompt: str, completion_config: dict = {}) -> str:
23 | self.trace_query(pre_prompt, prompt, completion_config)
24 |
25 | def run_messages(
26 | self,
27 | messages: list[dict],
28 | trace_input_messages: bool = True,
29 | completion_config: dict = {},
30 | ) -> list[dict]:
31 | if trace_input_messages:
32 | self.trace.write_separator()
33 | for message in messages:
34 | self.trace_message(message)
35 |
36 | def trace_message(self, message: dict) -> str:
37 | if self.trace is not None:
38 | self.trace(f"<<{message['role']}>>")
39 | self.trace(message["content"])
40 |
41 | def trace_query(
42 | self, pre_prompt: str, prompt: str, completion_config: dict = {}
43 | ) -> str:
44 | if self.trace is not None:
45 | self.trace.write_separator()
46 | self.trace("<>")
47 | self.trace(pre_prompt)
48 | self.trace("<>")
49 | self.trace(prompt)
50 |
51 | def trace_response(self, response: str) -> str:
52 | if self.trace is not None:
53 | self.trace("<>")
54 | self.trace(response)
55 |
56 |
57 | class HuggingFaceClient(Client):
58 | def __init__(
59 | self,
60 | model: str = None,
61 | pipeline_configuration: dict = {},
62 | seed: int = 42,
63 | api_key: str | None = None,
64 | trace: Trace = None,
65 | ) -> None:
66 | super().__init__(trace=trace)
67 |
68 | # to decrease the chance of hard-to-track bugs, we don't allow the 'model' key in pipeline_configuration
69 | if "model" in pipeline_configuration:
70 | raise ValueError(
71 | "The 'model' key is not allowed in pipeline_configuration. Use the 'model' parameter instead."
72 | )
73 |
74 | # data members
75 | self.model = model
76 | self.seed = seed
77 | if api_key is None:
78 | import os
79 |
80 | self.api_key = os.getenv("HUGGINGFACE_API_KEY")
81 | self.api_key = api_key
82 |
83 | # complete pipeline config
84 | from transformers import pipeline
85 |
86 | default_configuration = {
87 | "model": "meta-llama/Llama-3.1-8B-Instruct",
88 | "device": "cuda:0",
89 | "max_new_tokens": 1024,
90 | }
91 | merged_configuration = {**default_configuration, **pipeline_configuration}
92 |
93 | # model
94 | if self.model is not None:
95 | merged_configuration["model"] = self.model
96 |
97 | # seed
98 | from transformers import set_seed
99 |
100 | set_seed(self.seed)
101 |
102 | # tokenizer and pipeline for generation
103 | from transformers import AutoTokenizer
104 |
105 | self.tokenizer = AutoTokenizer.from_pretrained(
106 | default_configuration["model"], api_key=self.api_key
107 | )
108 | self.generator = pipeline("text-generation", **merged_configuration)
109 |
110 | def run(self, pre_prompt: str, prompt: str, completion_config: dict = {}) -> str:
111 | super().run(pre_prompt, prompt, completion_config)
112 | messages = [
113 | {"role": "system", "content": pre_prompt},
114 | {"role": "user", "content": prompt},
115 | ]
116 |
117 | completion = self.generator(
118 | messages,
119 | **completion_config,
120 | pad_token_id=self.generator.tokenizer.eos_token_id,
121 | )
122 |
123 | ret = completion[0]["generated_text"][-1]["content"]
124 | self.trace_response(ret)
125 | return ret
126 |
127 | def run_messages(
128 | self, messages, trace_input_messages: bool = True, completion_config={}
129 | ) -> list[dict]:
130 | super().run_messages(messages, trace_input_messages, completion_config)
131 | completion = self.generator(
132 | messages,
133 | **completion_config,
134 | pad_token_id=self.generator.tokenizer.eos_token_id,
135 | )
136 |
137 | ret = completion[0]["generated_text"]
138 | self.trace_message(ret[-1])
139 | return ret
140 |
141 |
142 | class OpenAIClient(Client):
143 | def __init__(
144 | self,
145 | base_url: str,
146 | model: str = None,
147 | seed: int = 42,
148 | api_key: str | None = None,
149 | trace: Trace = None,
150 | ) -> None:
151 | super().__init__(trace=trace)
152 |
153 | self.base_url = base_url
154 | self.model = (
155 | model
156 | if model is not None
157 | else "nvdev/nvidia/llama-3.1-nemotron-70b-instruct"
158 | )
159 | if api_key is None:
160 | import os
161 |
162 | api_key = os.getenv("NGC_API_KEY")
163 | self.api_key = api_key
164 | self.seed = seed
165 |
166 | from openai import OpenAI
167 |
168 | self.client = OpenAI(base_url=self.base_url, api_key=self.api_key)
169 |
170 | def _invoke(self, messages: list[dict], completion_config: dict = {}) -> str:
171 | default_settings = {
172 | "model": self.model,
173 | "top_p": 1,
174 | "temperature": 0.0,
175 | "max_tokens": 2048,
176 | "stream": True,
177 | "seed": self.seed,
178 | }
179 |
180 | merged_settings = {**default_settings, **completion_config}
181 | # go through the messages; if the role="ipython" rename it to "function"
182 | for message in messages:
183 | if message["role"] == "ipython":
184 | message["role"] = "function"
185 | completion = self.client.chat.completions.create(
186 | messages=messages, **merged_settings
187 | )
188 |
189 | ret: str = ""
190 | for chunk in completion:
191 | if chunk.choices[0].delta.content is not None:
192 | ret += chunk.choices[0].delta.content
193 |
194 | return ret
195 |
196 | def run(self, pre_prompt: str, prompt: str, completion_config: dict = {}) -> str:
197 | super().run(pre_prompt, prompt, completion_config)
198 |
199 | ret = self._invoke(
200 | messages=[
201 | {"role": "system", "content": pre_prompt},
202 | {"role": "user", "content": prompt},
203 | ],
204 | completion_config=completion_config,
205 | )
206 |
207 | self.trace_response(ret)
208 | return ret
209 |
210 | def run_messages(
211 | self, messages, trace_input_messages: bool = True, completion_config={}
212 | ) -> list[dict]:
213 | super().run_messages(messages, trace_input_messages, completion_config)
214 |
215 | ret_str: str = self._invoke(
216 | messages=messages, completion_config=completion_config
217 | )
218 | ret_msg: dict = {"role": "assistant", "content": ret_str}
219 |
220 | self.trace_message(ret_msg)
221 | return [*messages, ret_msg]
222 |
--------------------------------------------------------------------------------
/backend/frame/errands/message_type.txt:
--------------------------------------------------------------------------------
1 | You are a helpful assistant for detecting the type of the message send by the user. Based on the contents of and instructions in the MESSAGE, respond with one of the following message types: code, code_skill, routine, routine_skill, query, query_skill, data. Do not output any other text.
2 |
3 | ===SEPARATOR===
4 | You are deciding how the contents of MESSAGE will be handled by an AI system. To do so, choose on of the following TYPES that best characterizes the nature of the message. Every message can have only one type. Do not output any other text.
5 |
6 | TYPES:
7 | code
8 | ---
9 | code is the type of messages that contain code that is to be directly executed. The entire message can be
10 | - a directly pasted code snippet which is not just function definitions; or
11 | - it can be a human-friendly message with the code section to be executed enclosed in markdown single/triple backticks (e.g. `my one-line code` or ```my code```) or some other obvious separators (e.g.