├── .gitignore ├── 001_cursor_rules_QreOi6xrlGY ├── break_down_rule.md ├── implementation_rule.md ├── project_overview_example.md └── technical_design_documentation_rule.md ├── 004_n8n_browser_use_OLGKQgRJ0w0 ├── Agent Browser Use Flow.json ├── instructions.md └── n8n Browser Strict Flow.json ├── 006_n8n_rag_function_tool_jQOQLgPxbNE └── Simple Rag Chat.json ├── 007_n8n_zalo └── Zalo Setup.json ├── 009_n8n_rag_vn ├── Ultimate RAG Custom.json └── Ultimate RAG Gemini.json ├── 010_mcp_server_n8n └── personal_knowledge_management.json ├── 011_cursor_rules ├── memory_bank.md └── rule_generating_agent.mdc ├── 012_crawl4ai ├── 1.simple_crawl.py ├── 2.llm_extract.py ├── 3.multi_url_crawler.py ├── 4.crawl_with_profile.py ├── 5.update_profile.py ├── env.example ├── models │ └── schemas.py ├── prompts │ └── extraction_prompt.txt └── requirements.txt ├── 014_mcp_server_n8n └── n8n_mcp_server_collection.json ├── 015_3ac └── 3ac.md ├── 016_ai_agents_001 ├── README.md ├── main.py └── requirements.txt ├── 017_ai_agents_002 ├── agent.py ├── requirements.txt ├── run.py ├── run_result.py ├── run_streamed.py └── run_sync.py ├── 018_ai_agents_003 ├── agent.py ├── docs │ └── rabbit.txt ├── main.py ├── requirements.txt └── tools │ ├── custom │ └── tools.py │ └── normal │ └── tools.py └── 024_interactive_mcp └── interactive_feedback.mdc /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | *.pyc 3 | *.pyo 4 | *.pyd 5 | *.pyw 6 | *.pyz 7 | -------------------------------------------------------------------------------- /001_cursor_rules_QreOi6xrlGY/break_down_rule.md: -------------------------------------------------------------------------------- 1 | # Task Breakdown Rules 2 | 3 | You are an expert project manager and software architect. Given a technical design document, your task is to break it down into a comprehensive, actionable checklist of smaller tasks. This checklist should be suitable for assigning to developers and tracking progress. 4 | 5 | ## Input 6 | 7 | You will receive a Markdown document representing the technical design of a feature or component. This document will follow the structure outlined in the "Documentation Style" section above (Overview, Purpose, Design, Dependencies, Usage, Error Handling, Open Questions). 8 | 9 | ## Output 10 | 11 | Generate a Markdown checklist representing the task breakdown. 12 | 13 | ## Guidelines 14 | 15 | 1. **Granularity:** Tasks should be small enough to be completed within a reasonable timeframe (ideally a few hours to a day). Avoid tasks that are too large or too vague. 16 | 2. **Actionable:** Each task should describe a specific, concrete action that a developer can take. Use verbs like "Create", "Implement", "Add", "Update", "Refactor", "Test", "Document", etc. 17 | 3. **Dependencies:** Identify any dependencies between tasks. If task B depends on task A, make this clear (either through ordering or explicit notes). 18 | 4. **Completeness:** The checklist should cover all aspects of the technical design, including: 19 | - Database schema changes (migrations). 20 | - API endpoint creation/modification. 21 | - UI changes. 22 | - Business logic implementation. 23 | - Unit test creation. 24 | - Integration test creation (if applicable). 25 | - Documentation updates. 26 | - Addressing any open questions. 27 | 5. **Clarity:** Use clear and concise language. Avoid jargon or ambiguity. 28 | 6. **Checklist Format:** Use Markdown's checklist syntax: 29 | ``` 30 | - [ ] Task 1: Description of task 1 31 | - [ ] Task 2: Description of task 2 32 | - [ ] Task 3: Description of task 3 (depends on Task 2) 33 | ``` 34 | 7. **Categorization (Optional):** If the feature is large, consider grouping tasks into categories (e.g., "Database", "API", "UI", "Testing"). 35 | 8. **Prioritization (Optional):** If some tasks are higher priority than others, indicate this (e.g., using "(High Priority)" or a similar marker). 36 | 37 | ## Example 38 | 39 | **Input (Technical Design Document - Excerpt):** 40 | 41 | ```markdown 42 | ## CreateCategoryCommand 43 | 44 | **Overview:** This command creates a new BonfigurationCategory. 45 | 46 | **Purpose:** Allows administrators to define new categories for organizing configuration items. 47 | 48 | **Design:** 49 | - Takes a `CreateCategoryCommand` as input. 50 | - Uses `IUnitOfWork` to interact with the database. 51 | - Checks for existing categories with the same name. 52 | - Creates a new `BonfigurationCategory` entity. 53 | - Adds the category to the repository. 54 | - Saves changes to the database. 55 | 56 | **Dependencies:** 57 | - `IUnitOfWork` 58 | 59 | **Usage:** 60 | ```csharp 61 | // Example usage 62 | var command = new CreateCategoryCommand("MyCategory", "Description of my category"); 63 | var result = await _mediator.Send(command); 64 | ``` 65 | 66 | **Error Handling:** 67 | - Returns a `Result` indicating success or failure. 68 | - If a category with the same name already exists, returns a failure result with an appropriate error message. 69 | - Uses FluentValidation (`CreateCategoryCommandValidator`) to ensure the command is valid. 70 | 71 | **Open Questions:** 72 | - None 73 | ``` 74 | 75 | **Output (Task Breakdown):** 76 | 77 | ```markdown 78 | - [ ] Task 1: Create `CreateCategoryCommand` class (if it doesn't exist). 79 | - [ ] Task 2: Implement `CreateCategoryCommandHandler` class. 80 | - [ ] Inject `IUnitOfWork`. 81 | - [ ] Implement `Handle` method: 82 | - [ ] Check for existing category with the same name. 83 | - [ ] Create a new `BonfigurationCategory` entity. 84 | - [ ] Add the category to the repository. 85 | - [ ] Save changes to the database. 86 | - [ ] Return appropriate `Result`. 87 | - [ ] Task 3: Create `CreateCategoryCommandValidator` class (if it doesn't exist). 88 | - [ ] Add validation rules for `Name` and `Description`. 89 | - [ ] Task 4: Write unit tests for `CreateCategoryCommandHandler`. 90 | - [ ] Test successful category creation. 91 | - [ ] Test case where category with the same name already exists. 92 | - [ ] Test validation failures. 93 | - [ ] Task 5: Update documentation for `CreateCategoryCommand` and `CreateCategoryCommandHandler`. 94 | ``` 95 | 96 | **Another Example (with dependencies and categories):** 97 | 98 | **Input (Technical Design Document - Excerpt - for a hypothetical "Update BonfigurationItem" feature):** 99 | 100 | ```markdown 101 | ## UpdateBonfigurationItem Command 102 | 103 | **Overview:** Allows updating the key, description, and validation rules of a BonfigurationItem. 104 | 105 | **Design:** 106 | - Takes an `UpdateBonfigurationItemCommand` (with `Id`, `Key`, `Description`, `ValidationRules`). 107 | - Retrieves the existing `BonfigurationItem` from the repository. 108 | - Calls the `Update()` method on the entity. 109 | - Saves changes using `IUnitOfWork`. 110 | - Needs a new migration to allow `Key` to be updated (currently, it's part of the primary key). 111 | 112 | **Dependencies:** 113 | - `IBonfigurationRepository` 114 | - `IUnitOfWork` 115 | 116 | ... (rest of the document) ... 117 | ``` 118 | 119 | **Output (Task Breakdown):** 120 | 121 | ```markdown 122 | **Database:** 123 | 124 | - [ ] Task 1: Create a new database migration to allow updating the `Key` column of the `BonfigurationItem` table. (High Priority) 125 | 126 | **Application Layer:** 127 | 128 | - [ ] Task 2: Create `UpdateBonfigurationItemCommand` class. 129 | - [ ] Task 3: Create `UpdateBonfigurationItemCommandValidator` class. 130 | - [ ] Task 4: Implement `UpdateBonfigurationItemCommandHandler` class. 131 | - [ ] Inject `IBonfigurationRepository` and `IUnitOfWork`. 132 | - [ ] Implement `Handle` method: 133 | - [ ] Retrieve existing `BonfigurationItem` by ID. 134 | - [ ] Call `Update()` method on the entity. 135 | - [ ] Save changes using `IUnitOfWork`. 136 | 137 | **Testing:** 138 | 139 | - [ ] Task 5: Write unit tests for `UpdateBonfigurationItemCommandHandler`. 140 | - [ ] Test successful update. 141 | - [ ] Test case where `BonfigurationItem` is not found. 142 | - [ ] Test validation failures. 143 | 144 | **Documentation:** 145 | 146 | - [ ] Task 6: Update documentation for `BonfigurationItem` and the new command/handler. 147 | ``` 148 | 149 | -------------------------------------------------------------------------------- /001_cursor_rules_QreOi6xrlGY/implementation_rule.md: -------------------------------------------------------------------------------- 1 | # BoneNet Implementation Rule 2 | 3 | You are a diligent and detail-oriented software engineer working on the BoneNet project. You are responsible for implementing tasks according to the provided Technical Design Document (TDD) and task breakdown checklist. You meticulously follow instructions, write clean and well-documented code, and update the task list as you progress. 4 | 5 | ## Workflow 6 | 7 | 1. **Receive Task:** You will be given a specific task from the task breakdown checklist, along with the corresponding TDD with the below format: 8 | 9 | ``` 10 | Implementation: 11 | Task document: .md 12 | Technical Design Document: .md 13 | ``` 14 | You should first check and continue the un-checked work. Please ask permission to confirm before implementing. 15 | 16 | 2. **Review TDD and Task:** 17 | * Carefully review the relevant sections of the .md, paying close attention to: 18 | * Overview 19 | * Requirements (Functional and Non-Functional) 20 | * Technical Design (Data Model Changes, API Changes, Logic Flow, Dependencies, Security, Performance) 21 | * Thoroughly understand the specific task description from the checklist. 22 | * Ask clarifying questions if *anything* is unclear. Do *not* proceed until you fully understand the task and its relation to the TDD. 23 | 24 | 3. **Implement the Task:** 25 | * Write code that adheres to the TDD and BoneNet's coding standards. 26 | * Follow Domain-Driven Design principles. 27 | * Use descriptive variable and method names. 28 | * Include comprehensive docstrings: 29 | ```csharp 30 | /// 31 | /// Function explanation. 32 | /// 33 | /// The explanation of the parameter. 34 | /// Explain the return. 35 | ``` 36 | * Write unit tests for all new functionality. 37 | * Use the appropriate design patterns (CQRS, etc.). 38 | * Reference relevant files and classes using file paths. 39 | * If the TDD is incomplete or inaccurate, *stop* and request clarification or suggest updates to the TDD *before* proceeding. 40 | * If you encounter unexpected issues or roadblocks, *stop* and ask for guidance. 41 | 42 | 4. **Update Checklist:** 43 | * *Immediately* after completing a task and verifying its correctness (including tests), mark the corresponding item in .md as done. Use the following syntax: 44 | ```markdown 45 | - [x] Task 1: Description (Completed) 46 | ``` 47 | Add "(Completed)" to the task. 48 | * Do *not* mark a task as done until you are confident it is fully implemented and tested according to the TDD. 49 | 50 | 5. **Commit Changes (Prompt):** 51 | * After completing a task *and* updating the checklist, inform that the task is ready for commit. Use a prompt like: 52 | ``` 53 | Task [Task Number] is complete and the checklist has been updated. Ready for commit. 54 | ``` 55 | * You should then be prompted for a commit message. Provide a descriptive commit message following the Conventional Commits format: 56 | * `feat: Add new feature` 57 | * `fix: Resolve bug` 58 | * `docs: Update documentation` 59 | * `refactor: Improve code structure` 60 | * `test: Add unit tests` 61 | * `chore: Update build scripts` 62 | 63 | 6. **Repeat:** Repeat steps 1-5 for each task in the checklist. 64 | 65 | ## Coding Standards and Conventions (Reminder) 66 | 67 | * **C#:** 68 | * Follow Microsoft's C# Coding Conventions. 69 | * Use PascalCase for class names, method names, and properties. 70 | * Use camelCase for local variables and parameters. 71 | * Use descriptive names. 72 | * Use `async` and `await` for asynchronous operations. 73 | * Use LINQ for data manipulation. 74 | * **Project-Specific:** 75 | * Adhere to the Clean Architecture principles. 76 | * Use the CQRS pattern for commands and queries. 77 | * Use the UnitOfWork pattern for data access (`BoneNet.Application/Interfaces/Persistence/IUnitOfWork.cs`). 78 | * Use Value Objects for type safety (`BoneNet.Domain/ValueObjects/DataType.cs`). 79 | * Utilize the circuit breaker for external service calls (`BoneNet.Infrastructure/Services/CircuitBreakerService.cs`). 80 | * Use MediatR for command/query dispatching. 81 | * Use FluentValidation for validation (`BoneNet.Application/Common/Behaviors/ValidationBehavior.cs`). 82 | 83 | ## General Principles 84 | 85 | * Prioritize readability, maintainability, and testability. 86 | * Keep it simple. Avoid over-engineering. 87 | * Follow the SOLID principles. 88 | * DRY (Don't Repeat Yourself). 89 | * YAGNI (You Ain't Gonna Need It). 90 | * **Accuracy:** The code *must* accurately reflect the TDD. If discrepancies arise, *stop* and clarify. 91 | * **Checklist Discipline:** *Always* update the checklist immediately upon task completion. 92 | 93 | -------------------------------------------------------------------------------- /001_cursor_rules_QreOi6xrlGY/project_overview_example.md: -------------------------------------------------------------------------------- 1 | # BoneNet Project Overview 2 | 3 | ## Project Structure 4 | The project follows Clean Architecture principles with clear separation of concerns: 5 | 6 | ``` 7 | BoneNet/ 8 | ├── Application/ - Core business logic and use cases 9 | │ ├── Categories/ - Category management CQRS 10 | │ ├── Common/ - Shared infrastructure (behaviors, models, jobs) 11 | │ ├── ConfigCart/ - Configuration cart system 12 | │ ├── Configurations/- Configuration management 13 | │ └── Deployments/ - Deployment workflows 14 | ├── Domain/ - Core domain models and business rules 15 | │ ├── Common/ - Base entities and value objects 16 | │ ├── Entities/ - Aggregate roots and domain entities 17 | │ ├── Enums/ - Domain-specific enums 18 | │ ├── Exceptions/ - Custom domain exceptions 19 | │ └── ValueObjects/ - Domain value objects 20 | ├── Infrastructure/ - Implementation details 21 | │ ├── Caching/ - In-memory caching system 22 | │ ├── Persistence/ - EF Core data access 23 | │ └── Services/ - External service integrations 24 | └── Web/ - Presentation layer (partial) 25 | └── Admin/ - Blazor components 26 | ``` 27 | 28 | ## Key Patterns & Concepts 29 | 30 | 1. **CQRS Pattern**: 31 | - Commands (CreateCategoryCommand) and Queries (GetAllCategoriesQuery) are separated 32 | - MediatR is used for command/query dispatching 33 | 34 | ```7:34:BoneNet.Application/Categories/Commands/CreateCategory/CreateCategoryCommand.cs 35 | public record CreateCategoryCommand(string Name, string? Description) : IRequest>; 36 | 37 | public class CreateCategoryCommandHandler : IRequestHandler> 38 | { 39 | private readonly IUnitOfWork _unitOfWork; 40 | 41 | public CreateCategoryCommandHandler(IUnitOfWork unitOfWork) 42 | { 43 | _unitOfWork = unitOfWork; 44 | } 45 | 46 | public async Task> Handle(CreateCategoryCommand command, CancellationToken cancellationToken) 47 | { 48 | var existingCategory = await _unitOfWork.BonfigurationRepository 49 | .GetCategoryByNameAsync(command.Name); 50 | 51 | if (existingCategory != null) 52 | { 53 | return Result.Failure($"Category with name '{command.Name}' already exists"); 54 | } 55 | 56 | var category = Domain.Entities.BonfigurationCategory.Create( 57 | command.Name, 58 | command.Description); 59 | 60 | await _unitOfWork.BonfigurationRepository.AddCategoryAsync(category); 61 | await _unitOfWork.SaveChangesAsync(cancellationToken); 62 | 63 | ``` 64 | 65 | 66 | 2. **Domain-Driven Design**: 67 | - Rich domain models with encapsulated business logic 68 | - Value Objects for type safety: 69 | 70 | ```5:35:BoneNet.Domain/ValueObjects/DataType.cs 71 | 72 | public class DataType : ValueObject 73 | { 74 | public string Value { get; private set; } 75 | 76 | private DataType(string value) 77 | { 78 | Value = value; 79 | } 80 | 81 | public static DataType String => new("string"); 82 | public static DataType Number => new("number"); 83 | public static DataType Boolean => new("boolean"); 84 | public static DataType Json => new("json"); 85 | public static DataType Date => new("date"); 86 | 87 | public static DataType From(string value) 88 | { 89 | var normalizedValue = value.ToLowerInvariant(); 90 | return normalizedValue switch 91 | { 92 | "string" => String, 93 | "number" => Number, 94 | "boolean" => Boolean, 95 | "json" => Json, 96 | "date" => Date, 97 | _ => throw new DomainException($"Invalid data type: {value}") 98 | }; 99 | } 100 | 101 | protected override IEnumerable GetEqualityComponents() 102 | ``` 103 | 104 | 105 | 3. **Auditing**: 106 | - BaseAuditableEntity tracks creation/modification dates 107 | 108 | ```3:10:BoneNet.Domain/Common/BaseAuditableEntity.cs 109 | public abstract class BaseAuditableEntity : BaseEntity 110 | { 111 | public DateTimeOffset CreatedAt { get; set; } 112 | public DateTimeOffset UpdatedAt { get; set; } 113 | } 114 | 115 | public abstract class BaseEntity 116 | { 117 | ``` 118 | 119 | 120 | 4. **Circuit Breaker Pattern**: 121 | - Implements fault tolerance for external services 122 | 123 | ```7:110:BoneNet.Infrastructure/Services/CircuitBreakerService.cs 124 | public class CircuitBreakerService : ICircuitBreakerService 125 | { 126 | private readonly ILogger _logger; 127 | private readonly ConcurrentDictionary _circuitStates; 128 | private readonly ConcurrentDictionary _failureCounters; 129 | private readonly ConcurrentDictionary _lastFailureTime; 130 | 131 | private const int FailureThreshold = 5; // Number of failures before opening circuit 132 | private const int RetryTimeoutSeconds = 60; // Time to wait before attempting to close circuit 133 | private const int HalfOpenMaxAttempts = 3; // Max attempts in half-open state 134 | 135 | public CircuitBreakerService(ILogger logger) 136 | { 137 | _logger = logger; 138 | _circuitStates = new ConcurrentDictionary(); 139 | _failureCounters = new ConcurrentDictionary(); 140 | _lastFailureTime = new ConcurrentDictionary(); 141 | } 142 | 143 | public async Task ExecuteAsync(Func> operation, string operationKey) 144 | { 145 | EnsureCircuitInitialized(operationKey); 146 | 147 | var currentState = GetCircuitState(operationKey); 148 | 149 | if (currentState == CircuitBreakerState.Open) 150 | { 151 | if (ShouldAttemptReset(operationKey)) 152 | { 153 | TransitionToHalfOpen(operationKey); 154 | } 155 | else 156 | { 157 | _logger.LogWarning("Circuit breaker is open for operation: {OperationKey}", operationKey); 158 | throw new CircuitBreakerOpenException($"Circuit breaker is open for operation: {operationKey}"); 159 | } 160 | } 161 | 162 | try 163 | { 164 | var result = await operation(); 165 | 166 | if (currentState == CircuitBreakerState.HalfOpen) 167 | { 168 | ResetCircuit(operationKey); 169 | } 170 | 171 | return result; 172 | } 173 | catch (Exception ex) 174 | { 175 | await HandleFailure(operationKey, ex); 176 | throw; 177 | } 178 | } 179 | 180 | public CircuitBreakerState GetCircuitState(string operationKey) 181 | { 182 | return _circuitStates.GetOrAdd(operationKey, CircuitBreakerState.Closed); 183 | } 184 | 185 | private void EnsureCircuitInitialized(string operationKey) 186 | { 187 | _circuitStates.GetOrAdd(operationKey, CircuitBreakerState.Closed); 188 | _failureCounters.GetOrAdd(operationKey, 0); 189 | } 190 | 191 | private async Task HandleFailure(string operationKey, Exception exception) 192 | { 193 | _lastFailureTime.AddOrUpdate(operationKey, DateTime.UtcNow, (_, __) => DateTime.UtcNow); 194 | 195 | var failures = _failureCounters.AddOrUpdate(operationKey, 1, (_, count) => count + 1); 196 | 197 | _logger.LogWarning(exception, 198 | "Operation {OperationKey} failed. Failure count: {FailureCount}", 199 | operationKey, failures); 200 | 201 | if (failures >= FailureThreshold) 202 | { 203 | TransitionToOpen(operationKey); 204 | } 205 | 206 | await Task.CompletedTask; 207 | } 208 | 209 | private void TransitionToOpen(string operationKey) 210 | { 211 | _circuitStates.TryUpdate(operationKey, CircuitBreakerState.Open, CircuitBreakerState.Closed); 212 | _logger.LogWarning("Circuit breaker transitioned to Open for operation: {OperationKey}", operationKey); 213 | } 214 | 215 | private void TransitionToHalfOpen(string operationKey) 216 | { 217 | _circuitStates.TryUpdate(operationKey, CircuitBreakerState.HalfOpen, CircuitBreakerState.Open); 218 | _logger.LogInformation("Circuit breaker transitioned to HalfOpen for operation: {OperationKey}", operationKey); 219 | } 220 | 221 | private void ResetCircuit(string operationKey) 222 | { 223 | _circuitStates.TryUpdate(operationKey, CircuitBreakerState.Closed, CircuitBreakerState.HalfOpen); 224 | _failureCounters.TryUpdate(operationKey, 0, _failureCounters[operationKey]); 225 | _logger.LogInformation("Circuit breaker reset for operation: {OperationKey}", operationKey); 226 | } 227 | 228 | ``` 229 | 230 | 231 | ## Core Domain Models 232 | 233 | 1. **Configuration Management**: 234 | - `BonfigurationCategory` > `BonfigurationItem` > `BonfigurationValue` hierarchy 235 | - Versioning and draft/live states for configuration values 236 | 237 | ```7:60:BoneNet.Domain/Entities/BonfigurationItem.cs 238 | 239 | public class BonfigurationItem : BaseAuditableEntity 240 | { 241 | private readonly List _values = new(); 242 | 243 | public string Key { get; private set; } 244 | public string? Description { get; private set; } 245 | public DataType DataType { get; private set; } 246 | public bool IsEncrypted { get; private set; } 247 | public JsonDocument? ValidationRules { get; private set; } 248 | public Guid CategoryId { get; private set; } 249 | public virtual BonfigurationCategory Category { get; private set; } 250 | public IReadOnlyCollection Values => _values.AsReadOnly(); 251 | 252 | public Guid? CurrentLiveVersionId { get; private set; } 253 | public virtual BonfigurationValue? CurrentLiveVersion { get; private set; } 254 | 255 | public Guid? CurrentDraftVersionId { get; private set; } 256 | public virtual BonfigurationValue? CurrentDraftVersion { get; private set; } 257 | 258 | private BonfigurationItem() { } // For EF Core 259 | 260 | public static BonfigurationItem Create( 261 | string key, 262 | DataType dataType, 263 | Guid categoryId, 264 | string? description = null, 265 | bool isEncrypted = false, 266 | JsonDocument? validationRules = null) 267 | { 268 | if (string.IsNullOrWhiteSpace(key)) 269 | throw new DomainException("Item key cannot be empty"); 270 | 271 | return new BonfigurationItem 272 | { 273 | Key = key, 274 | DataType = dataType, 275 | CategoryId = categoryId, 276 | Description = description, 277 | IsEncrypted = isEncrypted, 278 | ValidationRules = validationRules 279 | }; 280 | } 281 | 282 | public void Update(string key, string? description, JsonDocument? validationRules) 283 | { 284 | if (string.IsNullOrWhiteSpace(key)) 285 | throw new DomainException("Item key cannot be empty"); 286 | 287 | Key = key; 288 | Description = description; 289 | ValidationRules = validationRules; 290 | } 291 | 292 | ``` 293 | 294 | 295 | 2. **Deployment System**: 296 | - `ConfigCart` with validation states 297 | - Deployment tracking with audit history 298 | 299 | ```7:33:BoneNet.Domain/Entities/Deployment.cs 300 | { 301 | public Guid PublishedBy { get; private set; } 302 | public DeploymentStatus Status { get; private set; } 303 | public Guid ConfigCartId { get; private set; } 304 | public string? Notes { get; private set; } 305 | 306 | private Deployment() { } // For EF Core 307 | 308 | public Deployment(Guid configCartId, Guid publishedBy, string? notes = null) 309 | { 310 | PublishedBy = publishedBy; 311 | Status = DeploymentStatus.Pending; 312 | ConfigCartId = configCartId; 313 | Notes = notes; 314 | } 315 | 316 | public void MarkAsSuccess() 317 | { 318 | Status = DeploymentStatus.Success; 319 | } 320 | 321 | public void MarkAsFailed() 322 | { 323 | Status = DeploymentStatus.Failed; 324 | } 325 | } 326 | ``` 327 | 328 | 329 | ## Infrastructure Highlights 330 | 331 | 1. **Persistence**: 332 | - Entity Framework Core with PostgreSQL 333 | - Repository pattern implementation 334 | 335 | ```7:70:BoneNet.Infrastructure/Persistence/UnitOfWork.cs 336 | 337 | public sealed class UnitOfWork : IUnitOfWork 338 | { 339 | private readonly BoneNetDbContext _context; 340 | private IDbContextTransaction? _currentTransaction; 341 | private bool _disposed; 342 | 343 | public IBonfigurationRepository BonfigurationRepository { get; } 344 | 345 | public IConfigCartRepository ConfigCartRepository { get; } 346 | public IDeploymentRepository DeploymentRepository { get; } 347 | 348 | public UnitOfWork(BoneNetDbContext context) 349 | { 350 | _context = context; 351 | ConfigCartRepository = new ConfigCartRepository(context); 352 | BonfigurationRepository = new BonfigurationRepository(context); 353 | DeploymentRepository = new DeploymentRepository(context); 354 | } 355 | 356 | public async Task SaveChangesAsync(CancellationToken cancellationToken = default) 357 | { 358 | return await _context.SaveChangesAsync(cancellationToken); 359 | } 360 | 361 | public async Task BeginTransactionAsync() 362 | { 363 | if (_currentTransaction != null) 364 | { 365 | return; 366 | } 367 | 368 | _currentTransaction = await _context.Database.BeginTransactionAsync(); 369 | } 370 | 371 | public async Task CommitTransactionAsync() 372 | { 373 | try 374 | { 375 | await _context.SaveChangesAsync(); 376 | 377 | if (_currentTransaction != null) 378 | { 379 | await _currentTransaction.CommitAsync(); 380 | } 381 | } 382 | catch 383 | { 384 | await RollbackTransactionAsync(); 385 | throw; 386 | } 387 | finally 388 | { 389 | if (_currentTransaction != null) 390 | { 391 | await _currentTransaction.DisposeAsync(); 392 | _currentTransaction = null; 393 | } 394 | } 395 | } 396 | 397 | public async Task RollbackTransactionAsync() 398 | { 399 | try 400 | ``` 401 | 402 | 403 | 2. **Caching**: 404 | - In-memory DataStore with concurrent access 405 | 406 | ```7:95:BoneNet.Infrastructure/Caching/DataStore.cs 407 | public class DataStore 408 | { 409 | private readonly ConcurrentDictionary> _data = new(); 410 | 411 | public void Set(CategoryName categoryName, string key, object value) 412 | { 413 | if (string.IsNullOrWhiteSpace(key)) 414 | throw new InvalidOperationException("Key cannot be empty"); 415 | 416 | if (value == null) 417 | throw new InvalidOperationException("Value cannot be null"); 418 | 419 | _data.AddOrUpdate( 420 | categoryName, 421 | _ => new ConcurrentDictionary(new[] { new KeyValuePair(key, value) }), 422 | (_, dict) => 423 | { 424 | dict.AddOrUpdate(key, value, (_, _) => value); 425 | return dict; 426 | }); 427 | } 428 | 429 | public object Get(CategoryName categoryName, string key) 430 | { 431 | if (!_data.TryGetValue(categoryName, out var categoryData)) 432 | throw new DataStoreKeyNotFoundException($"Category '{categoryName}' not found"); 433 | 434 | if (!categoryData.TryGetValue(key, out var value)) 435 | throw new DataStoreKeyNotFoundException($"Key '{key}' not found in category '{categoryName}'"); 436 | 437 | return value; 438 | } 439 | 440 | public bool TryGet(CategoryName categoryName, string key, out T result) 441 | { 442 | result = default!; 443 | 444 | if (!_data.TryGetValue(categoryName, out var categoryData)) 445 | return false; 446 | 447 | if (!categoryData.TryGetValue(key, out var value)) 448 | return false; 449 | 450 | if (value is not T typedValue) 451 | return false; 452 | 453 | result = typedValue; 454 | return true; 455 | } 456 | 457 | public IReadOnlyCollection> GetAllFromCategory(CategoryName categoryName) 458 | { 459 | if (!_data.TryGetValue(categoryName, out var categoryData)) 460 | return Array.Empty>(); 461 | 462 | return categoryData.ToList().AsReadOnly(); 463 | } 464 | 465 | public IReadOnlyDictionary GetMany(CategoryName categoryName, IEnumerable keys) 466 | { 467 | if (!_data.TryGetValue(categoryName, out var categoryData)) 468 | return new Dictionary(); 469 | 470 | return keys 471 | .Where(key => categoryData.ContainsKey(key)) 472 | .ToDictionary( 473 | key => key, 474 | key => categoryData[key]); 475 | } 476 | 477 | public bool TryGetMany(CategoryName categoryName, IEnumerable keys, out Dictionary results) 478 | { 479 | results = new Dictionary(); 480 | 481 | if (!_data.TryGetValue(categoryName, out var categoryData)) 482 | return false; 483 | 484 | bool allFound = true; 485 | foreach (var key in keys) 486 | { 487 | if (categoryData.TryGetValue(key, out var value) && value is T typedValue) 488 | { 489 | results[key] = typedValue; 490 | } 491 | else 492 | { 493 | allFound = false; 494 | } 495 | } 496 | ``` 497 | 498 | 499 | 3. **Messaging**: 500 | - RabbitMQ integration for deployment notifications 501 | 502 | ```7:85:BoneNet.Infrastructure/Services/RabbitMqDeploymentQueueService.cs 503 | using Microsoft.Extensions.Configuration; 504 | using Microsoft.Extensions.Logging; 505 | 506 | 507 | namespace BoneNet.Infrastructure.Services; 508 | 509 | public class RabbitMQDeploymentQueueService : IDeploymentQueueService, IDisposable 510 | { 511 | private readonly IConnection _connection; 512 | private readonly IModel _channel; 513 | private readonly ILogger _logger; 514 | private const string ExchangeName = "bonenet.deployments.fanout"; 515 | private string _queueName; // Unique queue name for each server instance 516 | 517 | public RabbitMQDeploymentQueueService(IConfiguration configuration, ILogger logger) 518 | { 519 | _logger = logger; 520 | 521 | var factory = new ConnectionFactory 522 | { 523 | HostName = configuration.GetValue("RabbitMQ:Host") ?? "rabbitmq", 524 | Port = configuration.GetValue("RabbitMQ:Port", 5672), 525 | UserName = configuration.GetValue("RabbitMQ:Username") ?? "guest", 526 | Password = configuration.GetValue("RabbitMQ:Password") ?? "guest" 527 | }; 528 | 529 | _connection = factory.CreateConnection(); 530 | _channel = _connection.CreateModel(); 531 | 532 | // Declare a fanout exchange 533 | _channel.ExchangeDeclare(ExchangeName, ExchangeType.Fanout, durable: true); 534 | 535 | // Create a unique queue name for this server instance 536 | _queueName = $"bonenet.deployments.{Guid.NewGuid()}"; 537 | 538 | // Declare a temporary queue that will be deleted when the connection closes 539 | _channel.QueueDeclare( 540 | queue: _queueName, 541 | durable: false, // Not durable - queue will be deleted when server restarts 542 | exclusive: true, // Exclusive to this connection 543 | autoDelete: true, // Delete when no consumers 544 | arguments: null); 545 | 546 | // Bind the queue to the fanout exchange 547 | _channel.QueueBind(_queueName, ExchangeName, string.Empty); 548 | 549 | // Set QoS for this consumer 550 | _channel.BasicQos(prefetchSize: 0, prefetchCount: 1, global: false); 551 | } 552 | 553 | public Task PublishDeploymentNotification(CreateDeploymentCommand notification) 554 | { 555 | var message = JsonSerializer.Serialize(notification); 556 | var body = Encoding.UTF8.GetBytes(message); 557 | 558 | _channel.BasicPublish( 559 | exchange: ExchangeName, 560 | routingKey: string.Empty, // Not needed for fanout exchange 561 | basicProperties: null, 562 | body: body); 563 | 564 | _logger.LogInformation("Published deployment notification: {DeploymentId}", notification.DeploymentId); 565 | return Task.CompletedTask; 566 | } 567 | 568 | public async Task SubscribeToDeploymentNotifications(Func handler) 569 | { 570 | var consumer = new AsyncEventingBasicConsumer(_channel); 571 | 572 | consumer.Received += async (_, ea) => 573 | { 574 | try 575 | { 576 | var body = ea.Body.ToArray(); 577 | var message = Encoding.UTF8.GetString(body); 578 | var notification = JsonSerializer.Deserialize(message); 579 | 580 | if (notification != null) 581 | { 582 | ``` 583 | 584 | 585 | ## Key Flows 586 | 587 | 1. **Configuration Creation**: 588 | ``` 589 | CreateCategoryCommand → Validation → Repository → Cache Update 590 | ``` 591 | 592 | 2. **Deployment Workflow**: 593 | ``` 594 | Create Deployment → Validate Cart → Queue Notification → Store Configurations 595 | ``` 596 | 597 | 3. **Validation Process**: 598 | 599 | ```7:40:BoneNet.Application/Common/Behaviors/ValidationBehavior.cs 600 | where TRequest : notnull 601 | { 602 | private readonly IEnumerable> _validators; 603 | 604 | public ValidationBehavior(IEnumerable> validators) 605 | { 606 | _validators = validators; 607 | } 608 | 609 | public async Task Handle( 610 | TRequest request, 611 | RequestHandlerDelegate next, 612 | CancellationToken cancellationToken) 613 | { 614 | if (!_validators.Any()) 615 | { 616 | return await next(); 617 | } 618 | 619 | var context = new ValidationContext(request); 620 | 621 | var validationResults = await Task.WhenAll( 622 | _validators.Select(v => v.ValidateAsync(context, cancellationToken))); 623 | 624 | var failures = validationResults 625 | .SelectMany(r => r.Errors) 626 | .Where(f => f != null) 627 | .ToList(); 628 | 629 | if (failures.Count != 0) 630 | { 631 | throw new ValidationException(failures); 632 | } 633 | 634 | ``` 635 | 636 | 637 | ## Getting Started Tips 638 | 639 | 1. Start with Domain models to understand business rules 640 | 2. Follow the CQRS pattern in Application layer for new features 641 | 3. Use the UnitOfWork pattern for data access 642 | 4. Leverage existing value objects for type safety 643 | 5. Utilize circuit breaker for external service calls 644 | 645 | ## Important Dependencies 646 | - MediatR (CQRS) 647 | - FluentValidation 648 | - Entity Framework Core 649 | - RabbitMQ.Client 650 | - Newtonsoft.Json 651 | 652 | This structure provides a maintainable foundation for a configuration management system with robust auditing, versioning, and deployment capabilities. 653 | -------------------------------------------------------------------------------- /001_cursor_rules_QreOi6xrlGY/technical_design_documentation_rule.md: -------------------------------------------------------------------------------- 1 | # Technical Design Document Generation Rule 2 | 3 | You are a software architect and technical writer assisting in the development of the BoneNet project. Your primary role is to generate comprehensive technical design documents based on provided feature requests, user stories, or high-level descriptions. You should analyze the existing codebase, identify relevant components, and propose a detailed implementation plan. 4 | 5 | ## Workflow 6 | 7 | When given a feature request, follow this process: 8 | 9 | 1. **Understand the Request:** 10 | * Ask clarifying questions about any ambiguities in the feature request. Focus on: 11 | * **Purpose:** What is the user trying to achieve? What problem does this solve? 12 | * **Scope:** What are the boundaries of this feature? What is explicitly *not* included? 13 | * **User Stories:** Can you provide specific user stories or use cases? 14 | * **Non-Functional Requirements:** Are there any performance, security, scalability, or maintainability requirements? 15 | * **Dependencies:** Does this feature depend on other parts of the system or external services? 16 | * **Existing Functionality:** Is there any existing functionality that can be reused or modified? 17 | * Do NOT proceed until you have a clear understanding of the request. 18 | 19 | 2. **Analyze Existing Codebase:** 20 | * Use the provided codebase context (especially @overview.md) to understand the project structure, key patterns, and existing domain models. 21 | * Identify relevant files, classes, and methods that will be affected by the new feature. Reference specific code locations when appropriate (e.g., `BonfigurationItem` entity: `startLine: 60`, `endLine: 113`). 22 | * Pay attention to: 23 | * CQRS pattern 24 | * Domain-Driven Design principles 25 | * Auditing 26 | * Circuit Breaker Pattern 27 | * Core Domain Models 28 | * Infrastructure concerns 29 | 30 | 3. **Generate Technical Design Document:** 31 | * Create a Markdown document with the following structure: 32 | 33 | ```markdown 34 | # Technical Design Document: [Feature Name] 35 | 36 | ## 1. Overview 37 | 38 | Briefly describe the purpose and scope of the feature. 39 | 40 | ## 2. Requirements 41 | 42 | ### 2.1 Functional Requirements 43 | 44 | * List specific, measurable, achievable, relevant, and time-bound (SMART) functional requirements. Use bullet points or numbered lists. 45 | * Example: As a user, I want to be able to create a new configuration category so that I can organize my configuration items. 46 | 47 | ### 2.2 Non-Functional Requirements 48 | 49 | * List non-functional requirements, such as performance, security, scalability, and maintainability. 50 | * Example: The system should be able to handle 100 concurrent users. 51 | * Example: All API endpoints must be secured with JWT authentication. 52 | 53 | ## 3. Technical Design 54 | 55 | ### 3.1. Data Model Changes 56 | 57 | * Describe any changes to the database schema. Include entity-relationship diagrams (ERDs) if necessary. Use Mermaid diagrams. 58 | * Specify new entities, fields, relationships, and data types. 59 | * Reference existing entities where appropriate. 60 | * Example: A new `DeploymentLog` entity will be added to track deployment events. This entity will have a one-to-many relationship with the `Deployment` entity (`startLine: 7`, `endLine: 33` in `BoneNet.Domain/Entities/Deployment.cs`). 61 | 62 | ### 3.2. API Changes 63 | 64 | * Describe any new API endpoints or changes to existing endpoints. 65 | * Specify request and response formats (using JSON). 66 | * Include example requests and responses. 67 | * Reference relevant CQRS commands and queries. 68 | * Example: A new `CreateDeploymentCommand` (`startLine: 9`, `endLine: 28` in `BoneNet.Application/Deployments/Commands/CreateDeployment/CreateDeploymentCommand.cs`) will be created to handle deployment requests. 69 | 70 | ### 3.3. UI Changes 71 | * Describe the changes on the UI. 72 | * Reference relevant components. 73 | 74 | ### 3.4. Logic Flow 75 | 76 | * Describe the flow of logic for the feature, including interactions between different components. 77 | * Use sequence diagrams or flowcharts if necessary. Use Mermaid diagrams. 78 | 79 | ### 3.5. Dependencies 80 | 81 | * List any new libraries, packages, or services required for this feature. 82 | * Example: The `AWSSDK.S3` NuGet package will be used for interacting with Amazon S3. 83 | 84 | ### 3.6. Security Considerations 85 | 86 | * Address any security concerns related to this feature. 87 | * Example: Input validation will be performed to prevent SQL injection attacks. 88 | * Example: Sensitive data will be encrypted at rest and in transit. 89 | 90 | ### 3.7. Performance Considerations 91 | * Address any performance concerns related to this feature. 92 | * Example: Caching will be used to improve the performance. 93 | 94 | ## 4. Testing Plan 95 | 96 | * Describe how the feature will be tested, including unit tests, integration tests, and user acceptance tests (UAT). 97 | * Example: Unit tests will be written for all new classes and methods. 98 | * Example: Integration tests will be written to verify the interaction between the API and the database. 99 | 100 | ## 5. Open Questions 101 | 102 | * List any unresolved issues or areas that require further clarification. 103 | * Example: Should we use a separate database for deployment logs? 104 | 105 | ## 6. Alternatives Considered 106 | 107 | * Briefly describe alternative solutions that were considered and why they were rejected. 108 | ``` 109 | 110 | 4. **Code Style and Conventions:** 111 | * Adhere to the project's existing coding style and conventions, as described in `overview.md`. 112 | * Use clear and concise language. 113 | * Use consistent formatting. 114 | 115 | 5. **Review and Iterate:** 116 | * Be prepared to revise the document based on feedback. 117 | * Ask clarifying questions if any feedback is unclear. 118 | 119 | 6. **Mermaid Diagrams:** 120 | * Use Mermaid syntax for diagrams. 121 | * Example sequence diagram: 122 | ```mermaid 123 | sequenceDiagram 124 | participant User 125 | participant API 126 | participant Database 127 | User->>API: Create Category 128 | API->>Database: Insert Category 129 | Database-->>API: Category ID 130 | API-->>User: Success 131 | ``` 132 | * Example ERD: 133 | ```mermaid 134 | erDiagram 135 | CATEGORY ||--o{ ITEM : contains 136 | ITEM ||--o{ VALUE : contains 137 | CATEGORY { 138 | uuid id 139 | string name 140 | string description 141 | } 142 | ITEM { 143 | uuid id 144 | string key 145 | string description 146 | } 147 | VALUE { 148 | uuid id 149 | string value 150 | bool is_draft 151 | } 152 | 153 | ``` -------------------------------------------------------------------------------- /004_n8n_browser_use_OLGKQgRJ0w0/Agent Browser Use Flow.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "agent-browser-use-flow", 3 | "nodes": [ 4 | { 5 | "parameters": { 6 | "options": {} 7 | }, 8 | "type": "@n8n/n8n-nodes-langchain.chatTrigger", 9 | "typeVersion": 1.1, 10 | "position": [ 11 | 0, 12 | 0 13 | ], 14 | "id": "fa8fb168-9476-4b46-88e7-09eb21581c81", 15 | "name": "When chat message received", 16 | "webhookId": "5327efd3-9223-461d-a75a-1387b6e299fe" 17 | }, 18 | { 19 | "parameters": { 20 | "promptType": "define", 21 | "text": "You are the helpful assistant with useful BrowserUse tools to execute web tasks.\n\n1. Please use Run Task when trying to execute web task.\n2. Please use Get Task Status to get the current task status.\n3. When the task is finished, call Get Task to get the results.\n\nHelp the user with the following: {{}}", 22 | "options": { 23 | "maxIterations": 50 24 | } 25 | }, 26 | "type": "@n8n/n8n-nodes-langchain.agent", 27 | "typeVersion": 1.7, 28 | "position": [ 29 | 220, 30 | 0 31 | ], 32 | "id": "6af1fee1-fc88-4722-9179-57965c14b561", 33 | "name": "AI Agent" 34 | }, 35 | { 36 | "parameters": { 37 | "options": {} 38 | }, 39 | "type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini", 40 | "typeVersion": 1, 41 | "position": [ 42 | 120, 43 | 260 44 | ], 45 | "id": "ae94ebbe-f72b-4c7e-b523-f8972c452b09", 46 | "name": "Google Gemini Chat Model", 47 | "credentials": { 48 | "googlePalmApi": { 49 | "id": "VytWkQ0CA5yghGIf", 50 | "name": "Google Gemini(PaLM) Api account" 51 | } 52 | } 53 | }, 54 | { 55 | "parameters": { 56 | "connectionType": "local", 57 | "instructions": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Instructions', ``, 'string') }}", 58 | "aiProvider": "google", 59 | "headful": true 60 | }, 61 | "type": "CUSTOM.browserUseTool", 62 | "typeVersion": 1, 63 | "position": [ 64 | 320, 65 | 260 66 | ], 67 | "id": "b1280ff8-ffce-4a40-be49-72e1de68fd0f", 68 | "name": "Browser Use", 69 | "credentials": { 70 | "browserUseLocalBridgeApi": { 71 | "id": "V5HW3VZJeodhd8xk", 72 | "name": "Browser Use Cloud account" 73 | } 74 | } 75 | }, 76 | { 77 | "parameters": { 78 | "connectionType": "local", 79 | "operation": "getTaskStatus", 80 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 81 | }, 82 | "type": "CUSTOM.browserUseTool", 83 | "typeVersion": 1, 84 | "position": [ 85 | 480, 86 | 280 87 | ], 88 | "id": "57727ccd-3213-4952-a7cf-c54f79161733", 89 | "name": "Browser Use1", 90 | "credentials": { 91 | "browserUseLocalBridgeApi": { 92 | "id": "V5HW3VZJeodhd8xk", 93 | "name": "Browser Use Cloud account" 94 | } 95 | } 96 | }, 97 | { 98 | "parameters": { 99 | "connectionType": "local", 100 | "operation": "getTask", 101 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 102 | }, 103 | "type": "CUSTOM.browserUseTool", 104 | "typeVersion": 1, 105 | "position": [ 106 | 600, 107 | 180 108 | ], 109 | "id": "f520c7d2-ba73-4e29-a4c5-0a097339c0ce", 110 | "name": "Browser Use2", 111 | "credentials": { 112 | "browserUseLocalBridgeApi": { 113 | "id": "V5HW3VZJeodhd8xk", 114 | "name": "Browser Use Cloud account" 115 | } 116 | } 117 | } 118 | ], 119 | "pinData": {}, 120 | "connections": { 121 | "When chat message received": { 122 | "main": [ 123 | [ 124 | { 125 | "node": "AI Agent", 126 | "type": "main", 127 | "index": 0 128 | } 129 | ] 130 | ] 131 | }, 132 | "Google Gemini Chat Model": { 133 | "ai_languageModel": [ 134 | [ 135 | { 136 | "node": "AI Agent", 137 | "type": "ai_languageModel", 138 | "index": 0 139 | } 140 | ] 141 | ] 142 | }, 143 | "Browser Use": { 144 | "ai_tool": [ 145 | [ 146 | { 147 | "node": "AI Agent", 148 | "type": "ai_tool", 149 | "index": 0 150 | } 151 | ] 152 | ] 153 | }, 154 | "Browser Use1": { 155 | "ai_tool": [ 156 | [ 157 | { 158 | "node": "AI Agent", 159 | "type": "ai_tool", 160 | "index": 0 161 | } 162 | ] 163 | ] 164 | }, 165 | "Browser Use2": { 166 | "ai_tool": [ 167 | [ 168 | { 169 | "node": "AI Agent", 170 | "type": "ai_tool", 171 | "index": 0 172 | } 173 | ] 174 | ] 175 | } 176 | }, 177 | "active": false, 178 | "settings": { 179 | "executionOrder": "v1" 180 | }, 181 | "versionId": "02333cf2-2d9c-4bc2-9188-0292e74302de", 182 | "meta": { 183 | "templateCredsSetupCompleted": true, 184 | "instanceId": "5e393c6f44392d3936ffca2e9e2dac28fed614d65a5879457461b30357ac3cd9" 185 | }, 186 | "id": "bksATOVoHMz7welt", 187 | "tags": [] 188 | } -------------------------------------------------------------------------------- /004_n8n_browser_use_OLGKQgRJ0w0/instructions.md: -------------------------------------------------------------------------------- 1 | # Presequigties 2 | 3 | - uv: https://docs.astral.sh/uv/getting-started/installation/#__tabbed_1_1 4 | - git: https://git-scm.com/downloads 5 | - nvm/node/npm: https://nodejs.org/en/download 6 | - pm2: https://pm2.keymetrics.io/ 7 | - https://github.com/draphonix/browser-n8n-local 8 | 9 | # Install browser-n8n-local 10 | 11 | 1. Clone project 12 | 13 | ``` 14 | git clone https://github.com/draphonix/browser-n8n-local.git 15 | ``` 16 | 17 | 2. Vào thư mục vừa clone 18 | 19 | ``` 20 | cd 21 | ``` 22 | 23 | 3. Tạo môi trường ảo bằng uv 24 | 25 | ``` 26 | uv venv --python 3.11 27 | ``` 28 | 29 | 4. Activate môi trường ảo 30 | 31 | ``` 32 | source .venv/bin/activate 33 | ``` 34 | 35 | 5. Cài các thư viện 36 | 37 | ``` 38 | uv pip install -r requirements.txt 39 | ``` 40 | 41 | 6. Tạo file env 42 | 43 | ``` 44 | cp .env-example .env 45 | ``` 46 | 47 | 7. Chỉnh env file 48 | 49 | 8. Chạy Server 50 | 51 | ``` 52 | pm2 start "uvicorn app:app --host 0.0.0.0 --port 24006" --name browser-n8n-local-server 53 | ``` 54 | 55 | ``` 56 | pm2 status 57 | pm2 stop 0 58 | pm2 delete 0 59 | pm2 start 0 60 | pm2 logs 61 | ``` 62 | 63 | ## Install n8n-nodes-browser-use trên n8n 64 | 65 | 1. Thêm biến vào env và chạy lại n8n 66 | 67 | ``` 68 | N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true n8n start 69 | ``` 70 | 71 | 1. Cài từ giao diện n8n 72 | 73 | ``` 74 | 1. Settings > Community nodes > Install Community nodes 75 | 2. n8n-nodes-browser-use 76 | ``` -------------------------------------------------------------------------------- /004_n8n_browser_use_OLGKQgRJ0w0/n8n Browser Strict Flow.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "n8n-browser-use-strict-flow", 3 | "nodes": [ 4 | { 5 | "parameters": { 6 | "options": {} 7 | }, 8 | "type": "@n8n/n8n-nodes-langchain.chatTrigger", 9 | "typeVersion": 1.1, 10 | "position": [ 11 | 0, 12 | 0 13 | ], 14 | "id": "3ac73388-b43c-41d9-bc62-59ef0e64bf6f", 15 | "name": "When chat message received", 16 | "webhookId": "c074ccfc-c49e-49db-9448-0ac88325e9eb" 17 | }, 18 | { 19 | "parameters": { 20 | "connectionType": "local", 21 | "instructions": "={{ $json.chatInput }}", 22 | "aiProvider": "google", 23 | "headful": true 24 | }, 25 | "type": "CUSTOM.browserUse", 26 | "typeVersion": 1, 27 | "position": [ 28 | 220, 29 | 0 30 | ], 31 | "id": "fbbc05f7-d2a4-4aa9-9d46-b9ae972a2f4b", 32 | "name": "Browser Use", 33 | "credentials": { 34 | "browserUseLocalBridgeApi": { 35 | "id": "V5HW3VZJeodhd8xk", 36 | "name": "Browser Use Cloud account" 37 | } 38 | } 39 | }, 40 | { 41 | "parameters": { 42 | "connectionType": "local", 43 | "operation": "getTaskStatus", 44 | "taskId": "={{ $json.id }}" 45 | }, 46 | "type": "CUSTOM.browserUse", 47 | "typeVersion": 1, 48 | "position": [ 49 | 440, 50 | 0 51 | ], 52 | "id": "43f4b924-35b7-4fd7-8308-dbe540f58a39", 53 | "name": "Browser Use1", 54 | "credentials": { 55 | "browserUseLocalBridgeApi": { 56 | "id": "V5HW3VZJeodhd8xk", 57 | "name": "Browser Use Cloud account" 58 | } 59 | } 60 | }, 61 | { 62 | "parameters": { 63 | "conditions": { 64 | "options": { 65 | "caseSensitive": true, 66 | "leftValue": "", 67 | "typeValidation": "strict", 68 | "version": 2 69 | }, 70 | "conditions": [ 71 | { 72 | "id": "f3a8dcd2-c7f7-48a5-a88a-b42e595d7e6f", 73 | "leftValue": "={{ $json.status }}", 74 | "rightValue": "finished", 75 | "operator": { 76 | "type": "string", 77 | "operation": "equals", 78 | "name": "filter.operator.equals" 79 | } 80 | } 81 | ], 82 | "combinator": "and" 83 | }, 84 | "options": {} 85 | }, 86 | "type": "n8n-nodes-base.if", 87 | "typeVersion": 2.2, 88 | "position": [ 89 | 660, 90 | 0 91 | ], 92 | "id": "c43adbe2-9fe8-4336-ab3e-af3f3ee31b0e", 93 | "name": "If" 94 | }, 95 | { 96 | "parameters": { 97 | "connectionType": "local", 98 | "operation": "getTask", 99 | "taskId": "={{ $('Browser Use').item.json.id }}" 100 | }, 101 | "type": "CUSTOM.browserUse", 102 | "typeVersion": 1, 103 | "position": [ 104 | 880, 105 | -100 106 | ], 107 | "id": "9373c0fe-0123-42b2-8d7d-71d45fa6ecb4", 108 | "name": "Browser Use2", 109 | "credentials": { 110 | "browserUseLocalBridgeApi": { 111 | "id": "V5HW3VZJeodhd8xk", 112 | "name": "Browser Use Cloud account" 113 | } 114 | } 115 | }, 116 | { 117 | "parameters": {}, 118 | "type": "n8n-nodes-base.wait", 119 | "typeVersion": 1.1, 120 | "position": [ 121 | 880, 122 | 100 123 | ], 124 | "id": "6dfe5ac8-8e78-49bf-b687-24ed5c48e887", 125 | "name": "Wait", 126 | "webhookId": "fc0a45f7-1a21-4f6b-b10b-360ad794d012" 127 | } 128 | ], 129 | "pinData": {}, 130 | "connections": { 131 | "When chat message received": { 132 | "main": [ 133 | [ 134 | { 135 | "node": "Browser Use", 136 | "type": "main", 137 | "index": 0 138 | } 139 | ] 140 | ] 141 | }, 142 | "Browser Use": { 143 | "main": [ 144 | [ 145 | { 146 | "node": "Browser Use1", 147 | "type": "main", 148 | "index": 0 149 | } 150 | ] 151 | ] 152 | }, 153 | "Browser Use1": { 154 | "main": [ 155 | [ 156 | { 157 | "node": "If", 158 | "type": "main", 159 | "index": 0 160 | } 161 | ] 162 | ] 163 | }, 164 | "If": { 165 | "main": [ 166 | [ 167 | { 168 | "node": "Browser Use2", 169 | "type": "main", 170 | "index": 0 171 | } 172 | ], 173 | [ 174 | { 175 | "node": "Wait", 176 | "type": "main", 177 | "index": 0 178 | } 179 | ] 180 | ] 181 | }, 182 | "Wait": { 183 | "main": [ 184 | [ 185 | { 186 | "node": "Browser Use1", 187 | "type": "main", 188 | "index": 0 189 | } 190 | ] 191 | ] 192 | } 193 | }, 194 | "active": true, 195 | "settings": { 196 | "executionOrder": "v1" 197 | }, 198 | "versionId": "cd738c37-7fe4-4d57-afc9-1b06e8957f79", 199 | "meta": { 200 | "templateCredsSetupCompleted": true, 201 | "instanceId": "5e393c6f44392d3936ffca2e9e2dac28fed614d65a5879457461b30357ac3cd9" 202 | }, 203 | "id": "3XS5AjxwGwO17eWu", 204 | "tags": [] 205 | } -------------------------------------------------------------------------------- /006_n8n_rag_function_tool_jQOQLgPxbNE/Simple Rag Chat.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "Simple Rag Chat", 3 | "nodes": [ 4 | { 5 | "parameters": { 6 | "content": "# RAG with Function callings\n\nAI Agents sẽ tự lựa chọn dùng RAG để query data hoặc sử dụng ExecuteQuery tools để chạy nếu RAG không đáp ứng được yêu cầu.\n\nLưu ý: Chỉ có những AI model nào support function calls mới sử dụng được tính năng này. Tham khảo danh sách này: https://python.langchain.com/docs/integrations/chat/", 7 | "height": 880, 8 | "width": 1060 9 | }, 10 | "type": "n8n-nodes-base.stickyNote", 11 | "typeVersion": 1, 12 | "position": [ 13 | 1040, 14 | -300 15 | ], 16 | "id": "265cabcf-effc-41c0-bf10-6688b156cf4a", 17 | "name": "Sticky Note" 18 | }, 19 | { 20 | "parameters": { 21 | "options": {} 22 | }, 23 | "type": "@n8n/n8n-nodes-langchain.chatTrigger", 24 | "typeVersion": 1.1, 25 | "position": [ 26 | 1140, 27 | -20 28 | ], 29 | "id": "b5eddfa6-3144-4ba3-8289-958a13ea661c", 30 | "name": "When chat message received", 31 | "webhookId": "3160ba93-5709-4b3a-9820-7f1276821fbb" 32 | }, 33 | { 34 | "parameters": { 35 | "promptType": "define", 36 | "text": "={{ $json.chatInput }}", 37 | "hasOutputParser": true, 38 | "options": { 39 | "systemMessage": "You are QQ Travel Assitant you can use Supabase Vector Store Tool to perform similarity search. Lets priority this tool first.\n\nFor complex request that required SQL query then you have the access to a Postgres database containing hotel information in Vietnam, you have the ability to run SQL queries to retrieve specific hotel data when needed. The Postgres tool allows you to search, filter, and analyze hotel information to provide accurate and relevant responses to user queries.\n\nAlways explain your reasoning when deciding to use the Postgres tool, formulate your SQL query clearly, and interpret the results for the user in natural language." 40 | } 41 | }, 42 | "type": "@n8n/n8n-nodes-langchain.agent", 43 | "typeVersion": 1.7, 44 | "position": [ 45 | 1360, 46 | -20 47 | ], 48 | "id": "3ba12ac3-9749-49d3-8dae-49deb0e16976", 49 | "name": "AI Agent" 50 | }, 51 | { 52 | "parameters": { 53 | "sessionIdType": "customKey", 54 | "sessionKey": "={{ $('When chat message received').item.json.sessionId }}" 55 | }, 56 | "type": "@n8n/n8n-nodes-langchain.memoryPostgresChat", 57 | "typeVersion": 1.3, 58 | "position": [ 59 | 1460, 60 | 200 61 | ], 62 | "id": "98323bd0-4669-408c-bac4-f5c3c17b146d", 63 | "name": "Postgres Chat Memory", 64 | "credentials": { 65 | "postgres": { 66 | "id": "NtTQg31814wKMvzj", 67 | "name": "Supabase" 68 | } 69 | } 70 | }, 71 | { 72 | "parameters": { 73 | "mode": "retrieve-as-tool", 74 | "toolName": "qq_travel", 75 | "toolDescription": "=# Similarity Search Travel Information Tool\nThis tool performs semantic similarity searches on a vector database containing tour information. It finds tours that match user queries by semantic meaning rather than exact keyword matching.\nDatabase Content\nThe database contains tours with information about:\nTour names and destinations\nPricing (original and current)\nRatings and review counts\nIncluded amenities and departure dates\nAvailable seats and suitable audiences\nSeasonal availability and discounts\nWhen to Use\nUse this tool to find relevant tour information based on natural language queries such as:\n\"Find beach tours with good ratings\"\n\"Show affordable family vacations\"\n\"Are there tours to Vietnam in July?\"\n\"What weekend getaways are discounted?\"\nThe tool returns the 5 most semantically similar tour options from the database, helping users find travel options that best match their interests and requirements.", 76 | "tableName": { 77 | "__rl": true, 78 | "value": "documents", 79 | "mode": "list", 80 | "cachedResultName": "documents" 81 | }, 82 | "topK": 5, 83 | "options": {} 84 | }, 85 | "type": "@n8n/n8n-nodes-langchain.vectorStoreSupabase", 86 | "typeVersion": 1, 87 | "position": [ 88 | 1600, 89 | 200 90 | ], 91 | "id": "442c2faa-57e1-4fd6-af8a-5c6e51e3e244", 92 | "name": "Supabase Vector Store", 93 | "credentials": { 94 | "supabaseApi": { 95 | "id": "UHYVVoHCTEa2MKnZ", 96 | "name": "Supabase account" 97 | } 98 | } 99 | }, 100 | { 101 | "parameters": { 102 | "options": {} 103 | }, 104 | "type": "@n8n/n8n-nodes-langchain.embeddingsOpenAi", 105 | "typeVersion": 1.2, 106 | "position": [ 107 | 1620, 108 | 360 109 | ], 110 | "id": "723b349b-67c2-44fd-b6be-c4160c7f567f", 111 | "name": "Embeddings OpenAI", 112 | "credentials": { 113 | "openAiApi": { 114 | "id": "7kuaUk9u2MPvl2ab", 115 | "name": "OpenAi account" 116 | } 117 | } 118 | }, 119 | { 120 | "parameters": { 121 | "model": { 122 | "__rl": true, 123 | "value": "o3-mini", 124 | "mode": "list", 125 | "cachedResultName": "o3-mini" 126 | }, 127 | "options": {} 128 | }, 129 | "type": "@n8n/n8n-nodes-langchain.lmChatOpenAi", 130 | "typeVersion": 1.2, 131 | "position": [ 132 | 1300, 133 | 200 134 | ], 135 | "id": "570f72c8-da99-4665-8586-914706329b66", 136 | "name": "OpenAI Chat Model", 137 | "credentials": { 138 | "openAiApi": { 139 | "id": "7kuaUk9u2MPvl2ab", 140 | "name": "OpenAi account" 141 | } 142 | } 143 | }, 144 | { 145 | "parameters": { 146 | "content": "# Database Setup and Sample Data Flow\n\n## Chuẩn bị\n- Supabase account với credentials dùng Transaction Pooler\n- AI providers: OpenAI hoặc OpenRouter, Ollama\n- Google OAuth\n- Sample Google Sheets data\n\n## Thực hành\n- Chạy node khởi tạo Vector DB.\n- Chạy node khởi tạo bảng mẫu.\n- Chạy node Google Sheet Trigger manually để test hoặc bật Production Mode.", 147 | "height": 1340, 148 | "width": 1160, 149 | "color": 3 150 | }, 151 | "type": "n8n-nodes-base.stickyNote", 152 | "typeVersion": 1, 153 | "position": [ 154 | -260, 155 | -320 156 | ], 157 | "id": "2c5901c3-d65d-4368-bf8e-c08cbe927722", 158 | "name": "Sticky Note1" 159 | }, 160 | { 161 | "parameters": { 162 | "jsonMode": "expressionData", 163 | "jsonData": "={{ $json.content }}", 164 | "options": { 165 | "metadata": { 166 | "metadataValues": [ 167 | { 168 | "name": "id", 169 | "value": "={{ $json.metadata.id }}" 170 | } 171 | ] 172 | } 173 | } 174 | }, 175 | "type": "@n8n/n8n-nodes-langchain.documentDefaultDataLoader", 176 | "typeVersion": 1, 177 | "position": [ 178 | 600, 179 | 680 180 | ], 181 | "id": "dedbf822-5fb6-4d5a-ac18-72cc0d3e12e0", 182 | "name": "Default Data Loader" 183 | }, 184 | { 185 | "parameters": { 186 | "options": {} 187 | }, 188 | "type": "@n8n/n8n-nodes-langchain.embeddingsOpenAi", 189 | "typeVersion": 1.2, 190 | "position": [ 191 | 340, 192 | 780 193 | ], 194 | "id": "81d5c8e0-4e14-43ae-a3c8-ef8e245aa279", 195 | "name": "Embeddings OpenAI1", 196 | "credentials": { 197 | "openAiApi": { 198 | "id": "7kuaUk9u2MPvl2ab", 199 | "name": "OpenAi account" 200 | } 201 | } 202 | }, 203 | { 204 | "parameters": {}, 205 | "type": "@n8n/n8n-nodes-langchain.textSplitterCharacterTextSplitter", 206 | "typeVersion": 1, 207 | "position": [ 208 | 620, 209 | 820 210 | ], 211 | "id": "6fd7b0c4-466b-417f-9637-0e6e1f5ecbaa", 212 | "name": "Character Text Splitter" 213 | }, 214 | { 215 | "parameters": { 216 | "jsCode": "return $('Prepare Embed Data').all()" 217 | }, 218 | "type": "n8n-nodes-base.code", 219 | "typeVersion": 2, 220 | "position": [ 221 | 360, 222 | 440 223 | ], 224 | "id": "c65b7f24-7ca5-4be7-b338-7ecf3070d22f", 225 | "name": "Code" 226 | }, 227 | { 228 | "parameters": { 229 | "descriptionType": "manual", 230 | "toolDescription": "=# Postgres Database Query Tool\nThis tool executes SQL queries on a PostgreSQL database containing tour information. Use it to find, filter, and analyze tour data.\nTable Schema Summary\nThe \"tours\" table includes details like tour ID, name, destination, pricing, ratings, available seats, descriptions, and seasonal information.\nUsage\nFormulate SQL queries that target the \"tours\" table to retrieve specific information.\nExample Queries\n-- Find tours by destination\nSELECT tour_name, current_price, rating FROM tours \nWHERE destination ILIKE '%paris%' ORDER BY rating DESC;\n\n-- Find affordable high-rated tours\nSELECT tour_name, destination, current_price \nFROM tours WHERE rating > 4 AND current_price < 1000;\n\n-- Find tours with available seats in a specific season\nSELECT tour_name, destination, available_seats \nFROM tours WHERE available_seats > 0 AND season = 'Summer';\n\nUse proper SQL syntax for best results. The tool will execute your query and return matching tour data.", 231 | "operation": "executeQuery", 232 | "query": "{{ $fromAI('sql_query') }}", 233 | "options": {} 234 | }, 235 | "type": "n8n-nodes-base.postgresTool", 236 | "typeVersion": 2.5, 237 | "position": [ 238 | 1900, 239 | 160 240 | ], 241 | "id": "b333675f-070b-4021-89af-f09ee04c87c2", 242 | "name": "Postgres", 243 | "credentials": { 244 | "postgres": { 245 | "id": "NtTQg31814wKMvzj", 246 | "name": "Supabase" 247 | } 248 | } 249 | }, 250 | { 251 | "parameters": { 252 | "operation": "executeQuery", 253 | "query": "-- Enable the pgvector extension to work with embedding vectors\n-- create extension vector;\n\n-- Create a table to store your documents\ncreate table documents (\n id bigserial primary key,\n content text, -- corresponds to Document.pageContent\n metadata jsonb, -- corresponds to Document.metadata\n embedding vector(1536) -- 1536 works for OpenAI embeddings, change if needed\n);\n\n-- Create a function to search for documents\ncreate function match_documents (\n query_embedding vector(1536),\n match_count int default null,\n filter jsonb DEFAULT '{}'\n) returns table (\n id bigint,\n content text,\n metadata jsonb,\n similarity float\n)\nlanguage plpgsql\nas $$\n#variable_conflict use_column\nbegin\n return query\n select\n id,\n content,\n metadata,\n 1 - (documents.embedding <=> query_embedding) as similarity\n from documents\n where metadata @> filter\n order by documents.embedding <=> query_embedding\n limit match_count;\nend;\n$$;", 254 | "options": {} 255 | }, 256 | "type": "n8n-nodes-base.postgres", 257 | "typeVersion": 2.5, 258 | "position": [ 259 | -200, 260 | 100 261 | ], 262 | "id": "d7df0588-7416-43f5-b13e-4aa8227715f5", 263 | "name": "Tạo Vector Database", 264 | "credentials": { 265 | "postgres": { 266 | "id": "NtTQg31814wKMvzj", 267 | "name": "Supabase" 268 | } 269 | } 270 | }, 271 | { 272 | "parameters": { 273 | "operation": "upsert", 274 | "schema": { 275 | "__rl": true, 276 | "mode": "list", 277 | "value": "public" 278 | }, 279 | "table": { 280 | "__rl": true, 281 | "value": "tours", 282 | "mode": "list", 283 | "cachedResultName": "tours" 284 | }, 285 | "columns": { 286 | "mappingMode": "autoMapInputData", 287 | "value": { 288 | "price_original": 0, 289 | "current_price": 0, 290 | "rating": 0, 291 | "reviews_count": 0, 292 | "available_seats": 0, 293 | "booking_months_advance": 0 294 | }, 295 | "matchingColumns": [ 296 | "id" 297 | ], 298 | "schema": [ 299 | { 300 | "id": "id", 301 | "displayName": "id", 302 | "required": true, 303 | "defaultMatch": true, 304 | "display": true, 305 | "type": "string", 306 | "canBeUsedToMatch": true, 307 | "removed": false 308 | }, 309 | { 310 | "id": "tour_name", 311 | "displayName": "tour_name", 312 | "required": false, 313 | "defaultMatch": false, 314 | "display": true, 315 | "type": "string", 316 | "canBeUsedToMatch": false 317 | }, 318 | { 319 | "id": "destination", 320 | "displayName": "destination", 321 | "required": false, 322 | "defaultMatch": false, 323 | "display": true, 324 | "type": "string", 325 | "canBeUsedToMatch": false 326 | }, 327 | { 328 | "id": "duration", 329 | "displayName": "duration", 330 | "required": false, 331 | "defaultMatch": false, 332 | "display": true, 333 | "type": "string", 334 | "canBeUsedToMatch": false 335 | }, 336 | { 337 | "id": "price_original", 338 | "displayName": "price_original", 339 | "required": false, 340 | "defaultMatch": false, 341 | "display": true, 342 | "type": "number", 343 | "canBeUsedToMatch": false 344 | }, 345 | { 346 | "id": "current_price", 347 | "displayName": "current_price", 348 | "required": false, 349 | "defaultMatch": false, 350 | "display": true, 351 | "type": "number", 352 | "canBeUsedToMatch": false 353 | }, 354 | { 355 | "id": "rating", 356 | "displayName": "rating", 357 | "required": false, 358 | "defaultMatch": false, 359 | "display": true, 360 | "type": "number", 361 | "canBeUsedToMatch": false 362 | }, 363 | { 364 | "id": "reviews_count", 365 | "displayName": "reviews_count", 366 | "required": false, 367 | "defaultMatch": false, 368 | "display": true, 369 | "type": "number", 370 | "canBeUsedToMatch": false 371 | }, 372 | { 373 | "id": "includes", 374 | "displayName": "includes", 375 | "required": false, 376 | "defaultMatch": false, 377 | "display": true, 378 | "type": "string", 379 | "canBeUsedToMatch": false 380 | }, 381 | { 382 | "id": "departure_dates", 383 | "displayName": "departure_dates", 384 | "required": false, 385 | "defaultMatch": false, 386 | "display": true, 387 | "type": "string", 388 | "canBeUsedToMatch": false 389 | }, 390 | { 391 | "id": "available_seats", 392 | "displayName": "available_seats", 393 | "required": false, 394 | "defaultMatch": false, 395 | "display": true, 396 | "type": "number", 397 | "canBeUsedToMatch": false 398 | }, 399 | { 400 | "id": "suitable_for", 401 | "displayName": "suitable_for", 402 | "required": false, 403 | "defaultMatch": false, 404 | "display": true, 405 | "type": "string", 406 | "canBeUsedToMatch": false 407 | }, 408 | { 409 | "id": "description", 410 | "displayName": "description", 411 | "required": false, 412 | "defaultMatch": false, 413 | "display": true, 414 | "type": "string", 415 | "canBeUsedToMatch": false 416 | }, 417 | { 418 | "id": "booking_months_advance", 419 | "displayName": "booking_months_advance", 420 | "required": false, 421 | "defaultMatch": false, 422 | "display": true, 423 | "type": "number", 424 | "canBeUsedToMatch": false 425 | }, 426 | { 427 | "id": "season", 428 | "displayName": "season", 429 | "required": false, 430 | "defaultMatch": false, 431 | "display": true, 432 | "type": "string", 433 | "canBeUsedToMatch": false 434 | }, 435 | { 436 | "id": "discount_percentage", 437 | "displayName": "discount_percentage", 438 | "required": false, 439 | "defaultMatch": false, 440 | "display": true, 441 | "type": "number", 442 | "canBeUsedToMatch": false 443 | }, 444 | { 445 | "id": "is_holiday", 446 | "displayName": "is_holiday", 447 | "required": false, 448 | "defaultMatch": false, 449 | "display": true, 450 | "type": "string", 451 | "canBeUsedToMatch": false 452 | }, 453 | { 454 | "id": "is_weekend", 455 | "displayName": "is_weekend", 456 | "required": false, 457 | "defaultMatch": false, 458 | "display": true, 459 | "type": "string", 460 | "canBeUsedToMatch": false 461 | }, 462 | { 463 | "id": "historical_data", 464 | "displayName": "historical_data", 465 | "required": false, 466 | "defaultMatch": false, 467 | "display": true, 468 | "type": "string", 469 | "canBeUsedToMatch": false 470 | } 471 | ], 472 | "attemptToConvertTypes": false, 473 | "convertFieldsToString": false 474 | }, 475 | "options": {} 476 | }, 477 | "type": "n8n-nodes-base.postgres", 478 | "typeVersion": 2.5, 479 | "position": [ 480 | 0, 481 | 640 482 | ], 483 | "id": "d9123197-0aa7-4db2-94c1-9816ec0b3f79", 484 | "name": "Tạo mới hoặc cập nhật dữ liệu", 485 | "credentials": { 486 | "postgres": { 487 | "id": "NtTQg31814wKMvzj", 488 | "name": "Supabase" 489 | } 490 | } 491 | }, 492 | { 493 | "parameters": { 494 | "mode": "insert", 495 | "tableName": { 496 | "__rl": true, 497 | "value": "documents", 498 | "mode": "list", 499 | "cachedResultName": "documents" 500 | }, 501 | "options": { 502 | "queryName": "match_documents" 503 | } 504 | }, 505 | "type": "@n8n/n8n-nodes-langchain.vectorStoreSupabase", 506 | "typeVersion": 1, 507 | "position": [ 508 | 540, 509 | 440 510 | ], 511 | "id": "bcaab6ae-849b-445b-a21b-30d5d018f053", 512 | "name": "Tạo documents vào vector DB", 513 | "credentials": { 514 | "supabaseApi": { 515 | "id": "UHYVVoHCTEa2MKnZ", 516 | "name": "Supabase account" 517 | } 518 | } 519 | }, 520 | { 521 | "parameters": { 522 | "operation": "delete", 523 | "tableId": "documents", 524 | "filterType": "string", 525 | "filterString": "=metadata->>id=eq.{{ $json.metadata.id }}" 526 | }, 527 | "type": "n8n-nodes-base.supabase", 528 | "typeVersion": 1, 529 | "position": [ 530 | 180, 531 | 440 532 | ], 533 | "id": "dd3994ff-e5a0-43cf-90d5-5588ed3f02cb", 534 | "name": "Xoá embed document cũ", 535 | "alwaysOutputData": true, 536 | "credentials": { 537 | "supabaseApi": { 538 | "id": "UHYVVoHCTEa2MKnZ", 539 | "name": "Supabase account" 540 | } 541 | } 542 | }, 543 | { 544 | "parameters": { 545 | "pollTimes": { 546 | "item": [ 547 | { 548 | "mode": "everyMinute" 549 | } 550 | ] 551 | }, 552 | "documentId": { 553 | "__rl": true, 554 | "value": "https://docs.google.com/spreadsheets/d/1cLl19ECzQuaGVl5xFxfbh0pvkMahOw70YyDp_xKypi4/edit?gid=0#gid=0", 555 | "mode": "url" 556 | }, 557 | "sheetName": { 558 | "__rl": true, 559 | "value": "https://docs.google.com/spreadsheets/d/1cLl19ECzQuaGVl5xFxfbh0pvkMahOw70YyDp_xKypi4/edit?gid=0#gid=0", 560 | "mode": "url" 561 | }, 562 | "options": {} 563 | }, 564 | "type": "n8n-nodes-base.googleSheetsTrigger", 565 | "typeVersion": 1, 566 | "position": [ 567 | -200, 568 | 440 569 | ], 570 | "id": "b9c087a3-b9f5-453a-9661-c4462d56eeec", 571 | "name": "Google Sheets Trigger1", 572 | "credentials": { 573 | "googleSheetsTriggerOAuth2Api": { 574 | "id": "b25vVq96dIYEOaOu", 575 | "name": "Google Sheets Trigger account" 576 | } 577 | } 578 | }, 579 | { 580 | "parameters": { 581 | "jsCode": "// Loop over input items and add a new field called 'myNewField' to the JSON of each one\nconst item = $input.all()[0];\nconst headers = {};\n \nfor (const key in item.json) {\n const value = item.json[key];\n headers[key] = typeof value\n}\n\nconst tableName = 'tours'; // Change this to your desired table name\n\n// Map JavaScript types to PostgreSQL types\nfunction mapTypeToPostgresType(jsType) {\n const typeMap = {\n 'string': 'text',\n 'number': 'numeric',\n 'boolean': 'boolean',\n 'object': 'jsonb',\n 'array': 'jsonb'\n };\n \n return typeMap[jsType] || 'text';\n}\n\n// Generate column definitions\nlet columnDefinitions = [];\nfor (const [column, type] of Object.entries(headers)) {\n const postgresType = mapTypeToPostgresType(type);\n columnDefinitions.push(`\"${column}\" ${postgresType}`);\n}\n\n// Add primary key (assuming tour_id is the primary key)\ncolumnDefinitions[0] += ' PRIMARY KEY';\n\n// Create the SQL query\nconst createTableSQL = `\nCREATE TABLE public.${tableName} (\n ${columnDefinitions.join(',\\n ')}\n) WITH (OIDS=FALSE);\n\n-- Add policies for Supabase\nCREATE POLICY \"Allow Transaction Pooler to Select Resources\" \nON public.${tableName} \nFOR SELECT \nTO authenticated \nUSING (true);\n\nCREATE POLICY \"Allow Transaction Pooler to Insert Resources\" \nON public.${tableName} \nFOR INSERT \nTO authenticated \nWITH CHECK (true);\n\nCREATE POLICY \"Allow Transaction Pooler to Update Resources\" \nON public.${tableName} \nFOR UPDATE \nTO authenticated \nUSING (true) \nWITH CHECK (true);\n\nCREATE POLICY \"Allow Transaction Pooler to Delete Resources\" \nON public.${tableName} \nFOR DELETE \nTO authenticated \nUSING (true);\n`;\n\nreturn { json: { sql: createTableSQL } };" 582 | }, 583 | "type": "n8n-nodes-base.code", 584 | "typeVersion": 2, 585 | "position": [ 586 | 0, 587 | 280 588 | ], 589 | "id": "0a935dd2-0078-4e4d-b95d-0f284f6f9c37", 590 | "name": "Generate SQL query" 591 | }, 592 | { 593 | "parameters": { 594 | "jsCode": "return items.map(item => {\n const data = item.json;\n \n // Dynamically build content from all fields\n const content = Object.entries(data)\n .map(([key, value]) => `${key}: ${value}`)\n .join('\\n');\n\n // Dynamically build metadata with type handling\n const metadata = Object.entries(data).reduce((acc, [key, value]) => {\n // Auto-detect and convert types\n let processedValue = value;\n \n // Handle numeric values\n if (!isNaN(value) && value !== '') {\n processedValue = Number(value);\n }\n // Handle boolean-like strings\n else if (typeof value === 'string' && ['yes', 'no', 'true', 'false'].includes(value.toLowerCase())) {\n processedValue = value.toLowerCase() === 'yes' || value.toLowerCase() === 'true';\n }\n // Handle comma-separated lists\n else if (typeof value === 'string' && value.includes(',')) {\n processedValue = value.split(',').map(item => item.trim());\n }\n // Preserve other values\n else {\n processedValue = value;\n }\n\n return {\n ...acc,\n [key]: processedValue\n };\n }, {});\n\n return {\n json: {\n content,\n metadata\n }\n };\n});" 595 | }, 596 | "type": "n8n-nodes-base.code", 597 | "typeVersion": 1, 598 | "position": [ 599 | 0, 600 | 440 601 | ], 602 | "id": "6ef78120-959a-47d9-bdb0-01da100684da", 603 | "name": "Prepare Embed Data" 604 | }, 605 | { 606 | "parameters": { 607 | "operation": "executeQuery", 608 | "query": "{{ $json.sql }}", 609 | "options": {} 610 | }, 611 | "type": "n8n-nodes-base.postgres", 612 | "typeVersion": 2.5, 613 | "position": [ 614 | 180, 615 | 280 616 | ], 617 | "id": "b9d5cc30-02cc-4604-a9d0-8a8422cf0b9b", 618 | "name": "Tạo bảng", 619 | "credentials": { 620 | "postgres": { 621 | "id": "NtTQg31814wKMvzj", 622 | "name": "Supabase" 623 | } 624 | } 625 | }, 626 | { 627 | "parameters": { 628 | "pollTimes": { 629 | "item": [ 630 | { 631 | "mode": "everyMinute" 632 | } 633 | ] 634 | }, 635 | "documentId": { 636 | "__rl": true, 637 | "value": "https://docs.google.com/spreadsheets/d/1cLl19ECzQuaGVl5xFxfbh0pvkMahOw70YyDp_xKypi4/edit?gid=0#gid=0", 638 | "mode": "url" 639 | }, 640 | "sheetName": { 641 | "__rl": true, 642 | "value": "https://docs.google.com/spreadsheets/d/1cLl19ECzQuaGVl5xFxfbh0pvkMahOw70YyDp_xKypi4/edit?gid=0#gid=0", 643 | "mode": "url" 644 | }, 645 | "options": {} 646 | }, 647 | "type": "n8n-nodes-base.googleSheetsTrigger", 648 | "typeVersion": 1, 649 | "position": [ 650 | -200, 651 | 280 652 | ], 653 | "id": "deffac18-0dd8-42a0-aab6-ef0828d5dc16", 654 | "name": "Google Sheets Trigger", 655 | "credentials": { 656 | "googleSheetsTriggerOAuth2Api": { 657 | "id": "b25vVq96dIYEOaOu", 658 | "name": "Google Sheets Trigger account" 659 | } 660 | } 661 | } 662 | ], 663 | "pinData": {}, 664 | "connections": { 665 | "When chat message received": { 666 | "main": [ 667 | [ 668 | { 669 | "node": "AI Agent", 670 | "type": "main", 671 | "index": 0 672 | } 673 | ] 674 | ] 675 | }, 676 | "Postgres Chat Memory": { 677 | "ai_memory": [ 678 | [ 679 | { 680 | "node": "AI Agent", 681 | "type": "ai_memory", 682 | "index": 0 683 | } 684 | ] 685 | ] 686 | }, 687 | "Supabase Vector Store": { 688 | "ai_tool": [ 689 | [ 690 | { 691 | "node": "AI Agent", 692 | "type": "ai_tool", 693 | "index": 0 694 | } 695 | ] 696 | ] 697 | }, 698 | "Embeddings OpenAI": { 699 | "ai_embedding": [ 700 | [ 701 | { 702 | "node": "Supabase Vector Store", 703 | "type": "ai_embedding", 704 | "index": 0 705 | } 706 | ] 707 | ] 708 | }, 709 | "OpenAI Chat Model": { 710 | "ai_languageModel": [ 711 | [ 712 | { 713 | "node": "AI Agent", 714 | "type": "ai_languageModel", 715 | "index": 0 716 | } 717 | ] 718 | ] 719 | }, 720 | "Default Data Loader": { 721 | "ai_document": [ 722 | [ 723 | { 724 | "node": "Tạo documents vào vector DB", 725 | "type": "ai_document", 726 | "index": 0 727 | } 728 | ] 729 | ] 730 | }, 731 | "Embeddings OpenAI1": { 732 | "ai_embedding": [ 733 | [ 734 | { 735 | "node": "Tạo documents vào vector DB", 736 | "type": "ai_embedding", 737 | "index": 0 738 | } 739 | ] 740 | ] 741 | }, 742 | "Character Text Splitter": { 743 | "ai_textSplitter": [ 744 | [ 745 | { 746 | "node": "Default Data Loader", 747 | "type": "ai_textSplitter", 748 | "index": 0 749 | } 750 | ] 751 | ] 752 | }, 753 | "Code": { 754 | "main": [ 755 | [ 756 | { 757 | "node": "Tạo documents vào vector DB", 758 | "type": "main", 759 | "index": 0 760 | } 761 | ] 762 | ] 763 | }, 764 | "Postgres": { 765 | "ai_tool": [ 766 | [ 767 | { 768 | "node": "AI Agent", 769 | "type": "ai_tool", 770 | "index": 0 771 | } 772 | ] 773 | ] 774 | }, 775 | "Xoá embed document cũ": { 776 | "main": [ 777 | [ 778 | { 779 | "node": "Code", 780 | "type": "main", 781 | "index": 0 782 | } 783 | ] 784 | ] 785 | }, 786 | "AI Agent": { 787 | "main": [ 788 | [] 789 | ] 790 | }, 791 | "Google Sheets Trigger1": { 792 | "main": [ 793 | [ 794 | { 795 | "node": "Tạo mới hoặc cập nhật dữ liệu", 796 | "type": "main", 797 | "index": 0 798 | }, 799 | { 800 | "node": "Prepare Embed Data", 801 | "type": "main", 802 | "index": 0 803 | } 804 | ] 805 | ] 806 | }, 807 | "Generate SQL query": { 808 | "main": [ 809 | [ 810 | { 811 | "node": "Tạo bảng", 812 | "type": "main", 813 | "index": 0 814 | } 815 | ] 816 | ] 817 | }, 818 | "Prepare Embed Data": { 819 | "main": [ 820 | [ 821 | { 822 | "node": "Xoá embed document cũ", 823 | "type": "main", 824 | "index": 0 825 | } 826 | ] 827 | ] 828 | }, 829 | "Google Sheets Trigger": { 830 | "main": [ 831 | [ 832 | { 833 | "node": "Generate SQL query", 834 | "type": "main", 835 | "index": 0 836 | } 837 | ] 838 | ] 839 | } 840 | }, 841 | "active": false, 842 | "settings": { 843 | "executionOrder": "v1" 844 | }, 845 | "versionId": "09d11c5b-6ed3-4ddb-a936-569f2947c530", 846 | "meta": { 847 | "templateCredsSetupCompleted": true, 848 | "instanceId": "5e393c6f44392d3936ffca2e9e2dac28fed614d65a5879457461b30357ac3cd9" 849 | }, 850 | "id": "0enfaiG0JD9INX9B", 851 | "tags": [] 852 | } 853 | -------------------------------------------------------------------------------- /007_n8n_zalo/Zalo Setup.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "Zalo Setup", 3 | "nodes": [ 4 | { 5 | "parameters": {}, 6 | "type": "n8n-nodes-zalo-vn.zaloWebServer", 7 | "typeVersion": 1, 8 | "position": [ 9 | 60, 10 | -260 11 | ], 12 | "id": "56022560-205f-4b71-84b5-3617186ff8d6", 13 | "name": "Zalo Web Server" 14 | }, 15 | { 16 | "parameters": { 17 | "operation": "stopServer" 18 | }, 19 | "type": "n8n-nodes-zalo-vn.zaloWebServer", 20 | "typeVersion": 1, 21 | "position": [ 22 | 300, 23 | -260 24 | ], 25 | "id": "9484159c-39e8-47ae-a17e-ebd242292005", 26 | "name": "Zalo Web Server1" 27 | }, 28 | { 29 | "parameters": { 30 | "event": "oauthCallback" 31 | }, 32 | "type": "n8n-nodes-zalo-vn.zaloTrigger", 33 | "typeVersion": 1, 34 | "position": [ 35 | 300, 36 | 280 37 | ], 38 | "id": "3035df00-19be-4228-a6c0-7cc7fd6ab1ae", 39 | "name": "OAuth Callback", 40 | "webhookId": "46b9d2a9-eba1-4006-b456-a73cd6a50f6e", 41 | "credentials": { 42 | "zaloApi": { 43 | "id": "6t92d0ztTnm1RAqi", 44 | "name": "Zalo account" 45 | } 46 | } 47 | }, 48 | { 49 | "parameters": { 50 | "operation": "generatePkce", 51 | "appId": "", 52 | "redirectUri": "https://zalo-n8n.kieng.io.vn/webhook/46b9d2a9-eba1-4006-b456-a73cd6a50f6e/callback" 53 | }, 54 | "type": "n8n-nodes-zalo-vn.zalo", 55 | "typeVersion": 1, 56 | "position": [ 57 | 300, 58 | 20 59 | ], 60 | "id": "412875a6-f365-4f0f-98dc-6dff5856d9dd", 61 | "name": "Generate PKCE parameters", 62 | "credentials": { 63 | "zaloApi": { 64 | "id": "6t92d0ztTnm1RAqi", 65 | "name": "Zalo account" 66 | } 67 | } 68 | }, 69 | { 70 | "parameters": {}, 71 | "type": "n8n-nodes-zalo-vn.zaloTrigger", 72 | "typeVersion": 1, 73 | "position": [ 74 | 720, 75 | 200 76 | ], 77 | "id": "b83b9eb0-f98b-4e19-ab6c-7562df124b62", 78 | "name": "Zalo Trigger", 79 | "webhookId": "61a95c38-a39c-4218-9ac7-d30e64cd3850", 80 | "credentials": { 81 | "zaloApi": { 82 | "id": "6t92d0ztTnm1RAqi", 83 | "name": "Zalo account" 84 | } 85 | } 86 | }, 87 | { 88 | "parameters": { 89 | "promptType": "define", 90 | "text": "={{ $json.message.text }}", 91 | "options": { 92 | "systemMessage": "Bạn là trợ lý Zalo OA hãy giúp khách hàng trả lời câu hỏi của họ." 93 | } 94 | }, 95 | "type": "@n8n/n8n-nodes-langchain.agent", 96 | "typeVersion": 1.7, 97 | "position": [ 98 | 1120, 99 | 200 100 | ], 101 | "id": "e5bb4c72-0797-451b-86f1-6a1bec36d30f", 102 | "name": "AI Agent" 103 | }, 104 | { 105 | "parameters": { 106 | "model": { 107 | "__rl": true, 108 | "mode": "list", 109 | "value": "gpt-4o-mini" 110 | }, 111 | "options": {} 112 | }, 113 | "type": "@n8n/n8n-nodes-langchain.lmChatOpenAi", 114 | "typeVersion": 1.2, 115 | "position": [ 116 | 1060, 117 | 420 118 | ], 119 | "id": "4344a7b3-3892-46a7-b420-38b9a1290b36", 120 | "name": "OpenAI Chat Model", 121 | "credentials": { 122 | "openAiApi": { 123 | "id": "7kuaUk9u2MPvl2ab", 124 | "name": "OpenAi account" 125 | } 126 | } 127 | }, 128 | { 129 | "parameters": { 130 | "sessionIdType": "customKey", 131 | "sessionKey": "{{$json.sender.id}}" 132 | }, 133 | "type": "@n8n/n8n-nodes-langchain.memoryBufferWindow", 134 | "typeVersion": 1.3, 135 | "position": [ 136 | 1220, 137 | 420 138 | ], 139 | "id": "1c55d9b7-f4ce-4213-9a45-1e202481ed5f", 140 | "name": "Window Buffer Memory" 141 | }, 142 | { 143 | "parameters": { 144 | "content": "### Đây là dữ liệu mẫu nhận được từ tin nhắn của Zalo. \n\nDo hạn chế trong việc sử dụng môi trường test nên mình để đây để mọi người hình dung được cấu trúc tin nhắn.\n\n```json\n[\n {\n \"event_name\": \"user_send_text\",\n \"app_id\": \"xxx\",\n \"sender\": {\n \"id\": \"xxx\"\n },\n \"recipient\": {\n \"id\": \"xxx\"\n },\n \"message\": {\n \"text\": \"xxx\",\n \"msg_id\": \"xxx\"\n },\n \"timestamp\": \"xxx\",\n \"user_id_by_app\": \"xxx\"\n }\n]\n```", 145 | "height": 520, 146 | "width": 440, 147 | "color": 4 148 | }, 149 | "type": "n8n-nodes-base.stickyNote", 150 | "typeVersion": 1, 151 | "position": [ 152 | 620, 153 | -380 154 | ], 155 | "id": "81701868-a805-426d-81d0-ca6aeb304f8d", 156 | "name": "Sticky Note" 157 | }, 158 | { 159 | "parameters": { 160 | "userId": "={{ $('Zalo Trigger').item.json.sender.id }}", 161 | "message": "={{ $json.output }}" 162 | }, 163 | "type": "n8n-nodes-zalo-vn.zalo", 164 | "typeVersion": 1, 165 | "position": [ 166 | 1580, 167 | 200 168 | ], 169 | "id": "7b995bc3-62ad-471b-b307-770d783a3bce", 170 | "name": "Zalo", 171 | "credentials": { 172 | "zaloApi": { 173 | "id": "6t92d0ztTnm1RAqi", 174 | "name": "Zalo account" 175 | } 176 | } 177 | }, 178 | { 179 | "parameters": { 180 | "content": "## Dùng trong trường hợp cần cài đặt sub-domain.\n(Bỏ qua bước này nếu bạn sử dụng domain)\n", 181 | "height": 300, 182 | "width": 520 183 | }, 184 | "type": "n8n-nodes-base.stickyNote", 185 | "typeVersion": 1, 186 | "position": [ 187 | 20, 188 | -380 189 | ], 190 | "id": "3f8b4981-dfc5-45f6-81c3-61552812e2a5", 191 | "name": "Sticky Note1" 192 | }, 193 | { 194 | "parameters": { 195 | "content": "### Generate PKCE - Dùng trong trường hợp khi mình cần tạo mới accessToken.\n\n\n\n\n\n\n\n\n\n\n\n### Oauth Callback - Để nhận kết quả từ Zalo vào tạo ra accessToken", 196 | "height": 480, 197 | "width": 520, 198 | "color": 3 199 | }, 200 | "type": "n8n-nodes-base.stickyNote", 201 | "typeVersion": 1, 202 | "position": [ 203 | 20, 204 | -40 205 | ], 206 | "id": "e4441967-6ee5-4393-892f-5d97fb6e4ebb", 207 | "name": "Sticky Note2" 208 | }, 209 | { 210 | "parameters": { 211 | "content": "### Ví dụ node AI Agent sẽ nhận message từ \n`{{ $json.message.text }}`", 212 | "height": 100, 213 | "width": 260, 214 | "color": 4 215 | }, 216 | "type": "n8n-nodes-base.stickyNote", 217 | "typeVersion": 1, 218 | "position": [ 219 | 1120, 220 | 40 221 | ], 222 | "id": "3526fac5-0387-4465-b5e1-56e23b4b88f3", 223 | "name": "Sticky Note3" 224 | }, 225 | { 226 | "parameters": { 227 | "content": "### Việc reply lại tin nhắn sẽ dựa vào\n`{{ $('Zalo Trigger').item.json.sender.id }}`", 228 | "height": 120, 229 | "width": 320, 230 | "color": 4 231 | }, 232 | "type": "n8n-nodes-base.stickyNote", 233 | "typeVersion": 1, 234 | "position": [ 235 | 1500, 236 | 40 237 | ], 238 | "id": "a4a1c60a-6060-421b-8bbd-e2809897ade0", 239 | "name": "Sticky Note4" 240 | }, 241 | { 242 | "parameters": { 243 | "content": "# Chuẩn bị\n\n- Domain cho n8n\n- n8n-nodes-zalo-vn\n- Zalo OA account\n\n# Cài đặt\n\n1. Cài **n8n-nodes-zalo-vn** community node.\n2. Zalo OA\n 1. Cài đặt [https://developers.zalo.me/app/](https://developers.zalo.me/app/)\n 1. ID ứng dụng\n 2. Khoá bí mật của ứng dụng\n 3. OAID - https://oa.zalo.me/manage/oa\n 2. Xác thực domain\n 1. Domain\n 1. DNS - TXT setup\n 2. Xác thực\n 2. Sub-domain\n 1. Sử dụng Zalo Web Server Tool\n 2. Xác thực\n 1. Thiết lập đường dẫn từ internet đến server local \n 3. Tắt server sau khi xác thực xong\n 3. Setup Zalo Credentials ở n8n node\n 4. Setup Zalo On OAuth Callback\n 1. Lấy callback URL: ….\n 5. Setup Official Account Callback Url\n 1. Cập nhật OAuth Callback vừa lấy ở bước d(i)\n 6. Bật danh sách sự kiện webhook.\n 7. Đăng kí sử dụng API.\n 8. Tạo accessToken\n 1. Dùng Zalo Generate PKCE Parameters\n 2. Lấy callback url ở bước d(i)\n 9. Tạo Zalo Trigger\n 1. Lấy callback để setup vào Webhook của Zalo: ….\n", 244 | "height": 980, 245 | "width": 520, 246 | "color": 6 247 | }, 248 | "type": "n8n-nodes-base.stickyNote", 249 | "typeVersion": 1, 250 | "position": [ 251 | -600, 252 | -380 253 | ], 254 | "id": "3b0a4e02-aae4-4c1a-b4b4-4ee697426053", 255 | "name": "Sticky Note5" 256 | } 257 | ], 258 | "pinData": {}, 259 | "connections": { 260 | "Zalo Web Server1": { 261 | "main": [ 262 | [] 263 | ] 264 | }, 265 | "OAuth Callback": { 266 | "main": [ 267 | [] 268 | ] 269 | }, 270 | "Zalo Trigger": { 271 | "main": [ 272 | [ 273 | { 274 | "node": "AI Agent", 275 | "type": "main", 276 | "index": 0 277 | } 278 | ] 279 | ] 280 | }, 281 | "OpenAI Chat Model": { 282 | "ai_languageModel": [ 283 | [ 284 | { 285 | "node": "AI Agent", 286 | "type": "ai_languageModel", 287 | "index": 0 288 | } 289 | ] 290 | ] 291 | }, 292 | "Window Buffer Memory": { 293 | "ai_memory": [ 294 | [ 295 | { 296 | "node": "AI Agent", 297 | "type": "ai_memory", 298 | "index": 0 299 | } 300 | ] 301 | ] 302 | }, 303 | "AI Agent": { 304 | "main": [ 305 | [ 306 | { 307 | "node": "Zalo", 308 | "type": "main", 309 | "index": 0 310 | } 311 | ] 312 | ] 313 | } 314 | }, 315 | "active": true, 316 | "settings": { 317 | "executionOrder": "v1" 318 | }, 319 | "versionId": "b6631da0-cfc0-4e95-b8bd-84f4322a2962", 320 | "meta": { 321 | "templateCredsSetupCompleted": true, 322 | "instanceId": "5e393c6f44392d3936ffca2e9e2dac28fed614d65a5879457461b30357ac3cd9" 323 | }, 324 | "id": "rxldoo1llfbkih99", 325 | "tags": [] 326 | } -------------------------------------------------------------------------------- /011_cursor_rules/memory_bank.md: -------------------------------------------------------------------------------- 1 | # Cursor's Memory Bank 2 | 3 | I am Cursor, an expert software engineer with a unique characteristic: my memory resets completely between sessions. This isn't a limitation - it's what drives me to maintain perfect documentation. After each reset, I rely ENTIRELY on my Memory Bank to understand the project and continue work effectively. I MUST read ALL memory bank files at the start of EVERY task - this is not optional. 4 | 5 | ## Memory Bank Structure 6 | 7 | The Memory Bank consists of required core files and optional context files, all in Markdown format. Files build upon each other in a clear hierarchy: 8 | 9 | ```mermaid 10 | flowchart TD 11 | PB[projectbrief.md] --> PC[productContext.md] 12 | PB --> SP[systemPatterns.md] 13 | PB --> TC[techContext.md] 14 | 15 | PC --> AC[activeContext.md] 16 | SP --> AC 17 | TC --> AC 18 | 19 | AC --> P[progress.md] 20 | ``` 21 | 22 | ### Core Files (Required) 23 | 1. `projectbrief.md` 24 | - Foundation document that shapes all other files 25 | - Created at project start if it doesn't exist 26 | - Defines core requirements and goals 27 | - Source of truth for project scope 28 | 29 | 2. `productContext.md` 30 | - Why this project exists 31 | - Problems it solves 32 | - How it should work 33 | - User experience goals 34 | 35 | 3. `activeContext.md` 36 | - Current work focus 37 | - Recent changes 38 | - Next steps 39 | - Active decisions and considerations 40 | 41 | 4. `systemPatterns.md` 42 | - System architecture 43 | - Key technical decisions 44 | - Design patterns in use 45 | - Component relationships 46 | 47 | 5. `techContext.md` 48 | - Technologies used 49 | - Development setup 50 | - Technical constraints 51 | - Dependencies 52 | 53 | 6. `progress.md` 54 | - What works 55 | - What's left to build 56 | - Current status 57 | - Known issues 58 | 59 | ### Additional Context 60 | Create additional files/folders within memory-bank/ when they help organize: 61 | - Complex feature documentation 62 | - Integration specifications 63 | - API documentation 64 | - Testing strategies 65 | - Deployment procedures 66 | 67 | ## Core Workflows 68 | 69 | ### Plan Mode 70 | ```mermaid 71 | flowchart TD 72 | Start[Start] --> ReadFiles[Read Memory Bank] 73 | ReadFiles --> CheckFiles{Files Complete?} 74 | 75 | CheckFiles -->|No| Plan[Create Plan] 76 | Plan --> Document[Document in Chat] 77 | 78 | CheckFiles -->|Yes| Verify[Verify Context] 79 | Verify --> Strategy[Develop Strategy] 80 | Strategy --> Present[Present Approach] 81 | ``` 82 | 83 | ### Act Mode 84 | ```mermaid 85 | flowchart TD 86 | Start[Start] --> Context[Check Memory Bank] 87 | Context --> Update[Update Documentation] 88 | Update --> Rules[Update .cursorrules if needed] 89 | Rules --> Execute[Execute Task] 90 | Execute --> Document[Document Changes] 91 | ``` 92 | 93 | ## Documentation Updates 94 | 95 | Memory Bank updates occur when: 96 | 1. Discovering new project patterns 97 | 2. After implementing significant changes 98 | 3. When user requests with **update memory bank** (MUST review ALL files) 99 | 4. When context needs clarification 100 | 101 | ```mermaid 102 | flowchart TD 103 | Start[Update Process] 104 | 105 | subgraph Process 106 | P1[Review ALL Files] 107 | P2[Document Current State] 108 | P3[Clarify Next Steps] 109 | P4[Update .cursorrules] 110 | 111 | P1 --> P2 --> P3 --> P4 112 | end 113 | 114 | Start --> Process 115 | ``` 116 | 117 | Note: When triggered by **update memory bank**, I MUST review every memory bank file, even if some don't require updates. Focus particularly on activeContext.md and progress.md as they track current state. 118 | . 119 | 120 | REMEMBER: After every memory reset, I begin completely fresh. The Memory Bank is my only link to previous work. It must be maintained with precision and clarity, as my effectiveness depends entirely on its accuracy. 121 | -------------------------------------------------------------------------------- /011_cursor_rules/rule_generating_agent.mdc: -------------------------------------------------------------------------------- 1 | --- 2 | description: 3 | globs: 4 | alwaysApply: false 5 | --- 6 | # Cursor Rules Format 7 | 8 | ## Template Structure for Rules Files 9 | 10 | ```mdc 11 | --- 12 | description: `Explicit concise description to ensure the agent knows when to apply the rule` OR blank 13 | globs: .cursor/rules/**/*.mdc OR blank 14 | alwaysApply: {true or false} 15 | --- 16 | 17 | # Rule Title 18 | 19 | ## Context 20 | 21 | - When to apply this rule 22 | - Prerequisites or conditions 23 | - Why the rule was added or is needed 24 | 25 | ## Critical Rules 26 | 27 | - Concise, bulleted list of actionable rules the agent MUST follow 28 | 29 | ## Examples 30 | 31 | 32 | {valid rule application} 33 | 34 | 35 | 36 | {invalid rule application} 37 | 38 | ``` 39 | 40 | ### Organizational Folders (Create if non existent) 41 | All rules files will be under an organizational folder: 42 | - .cursor/rules/always - these will be rules that are ALWAYS applied to every chat and cmd/ctrl-k context 43 | - .cursor/rules/auto-attached - these will be rules that applied when file pattern matched. 44 | - .cursor/rules/agent-requested - the agent will see this description and decide to read the full rule if it wants 45 | - .cursor/rules/manual - this rule needs to be mentioned to be included. 46 | 47 | ## Glob Pattern Examples 48 | Common glob patterns for different rule types: 49 | - Core standards: .cursor/rules/*.mdc 50 | - Language rules: *.cs, *.cpp 51 | - Testing standards: *.test.ts, *.test.js 52 | - React components: src/components/**/*.tsx 53 | - Documentation: docs/**/*.md, *.md 54 | - Configuration files: *.config.js 55 | - Build artifacts: dist/**/* 56 | - Multiple extensions: *.js, *.ts, *.tsx 57 | - Multiple patterns: dist/**/*.*, docs/**/*.md, *test*.* 58 | 59 | ## Critical Rules 60 | - Rule files will be located and named ALWAYS as: `.cursor/rules/{organizational-folder}/rule-name-{auto|agent|manual|always}.mdc` 61 | - FrontMatter Rules Types: 62 | - The front matter section must always start the file and include all 3 fields, even if the field value will be blank - the types are: 63 | - Manual Rule: IF a Manual rule is requested - description and globs MUST be blank and alwaysApply: false and filename ends with -manual.mdc. 64 | - Auto Rule: IF a rule is requested that should apply always to certain glob patterns (example all typescript files or all markdown files) - description must be blank, and alwaysApply: false and filename ends with -auto.mdc. 65 | - Always Rule: Global Rule applies to every chat and cmd/ctrl-k - description and globs blank, and alwaysApply: true and filename ends with -always.mdc. 66 | - Agent Select Rule: The rule does not need to be loaded into every chat thread, it serves a specific purpose. The agent can see the descriptions, and choose to load the full rule in to context on its own - description is critical, globs blank, and alwaysApply:false and filename ends with -agent.mdc 67 | - For the Rule Context and Bullets - do not repeat yourself and do not be overly explanatory 68 | - When a rule will only be used sometimes (useAlways: false) it is CRITICAL that the description describes when the AI will load the full rule into its context 69 | - Use Concise Markdown Tailored to Agent Context Window usage 70 | - Always indent content within XML Example section with 2 spaces 71 | - Emojis and Mermaid diagrams are allowed and encouraged if it is not redundant and better explains the rule for the AI comprehension. 72 | - TRY to keep the total rule line count under 50 lines, better under 25 lines 73 | - Always include a valid and invalid example 74 | - NEVER use quotes around glob patterns, NEVER group glob extensions with `{}` 75 | - If the request for a rule or a future behavior change includes context of a mistake is made, this would be great to use in the example for the rule 76 | - After rule is created or updated, Respond with the following: 77 | - AutoRuleGen Success: path/rule-name.mdc 78 | - Rule Type: {Rule Type} 79 | - Short summary of what the rule will do 80 | -------------------------------------------------------------------------------- /012_crawl4ai/1.simple_crawl.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | from crawl4ai import AsyncWebCrawler 3 | from crawl4ai.async_configs import BrowserConfig, CrawlerRunConfig 4 | from datetime import datetime 5 | async def main(): 6 | # Configure browser and crawler settings (using defaults here) 7 | browser_config = BrowserConfig() 8 | run_config = CrawlerRunConfig() 9 | 10 | # Initialize the crawler 11 | async with AsyncWebCrawler(config=browser_config) as crawler: 12 | # Set the URL you want to crawl 13 | url = "https://n8n.io/workflows/categories/ai/" 14 | # Run the crawler 15 | result = await crawler.arun(url=url, config=run_config) 16 | 17 | # Display results 18 | print(f"Crawl successful: {result.success}") 19 | print(f"Status code: {result.status_code}") 20 | # Save the result to a file in the output directory 21 | # Create the output directory if it doesn't exist 22 | import os 23 | os.makedirs("output", exist_ok=True) 24 | 25 | # Save the result to a file in the output directory 26 | with open("output/crawl_result.md", "w") as f: 27 | f.write(result.markdown) 28 | 29 | # Uncomment these to see other available data 30 | print("\n--- Raw HTML ---\n") 31 | print(result.html) # First 500 chars 32 | 33 | # print("\n--- Cleaned HTML ---\n") 34 | # print(result.cleaned_html[:500] + "...") # First 500 chars 35 | 36 | if __name__ == "__main__": 37 | asyncio.run(main()) -------------------------------------------------------------------------------- /012_crawl4ai/2.llm_extract.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import json 3 | import os 4 | from pathlib import Path 5 | 6 | from crawl4ai import AsyncWebCrawler 7 | from crawl4ai.async_configs import BrowserConfig, CrawlerRunConfig, CacheMode, LLMConfig 8 | from crawl4ai.extraction_strategy import LLMExtractionStrategy 9 | from dotenv import load_dotenv 10 | from models.schemas import ResultSchema 11 | 12 | 13 | async def main(): 14 | """ 15 | Crawl a URL and extract structured JSON data using LLM. 16 | 17 | Args: 18 | url: The URL to crawl 19 | schema: JSON schema for extraction 20 | output_dir: Directory to save results (optional) 21 | prompt: Custom prompt for LLM extraction (optional) 22 | api_key: OpenAI API key (uses environment variable if not provided) 23 | """ 24 | # Get API key from args or environment 25 | api_key = os.getenv('OPENAI_API_KEY') 26 | if not api_key: 27 | print("Warning: No OpenAI API key provided. Set OPENAI_API_KEY environment variable or use --api-key") 28 | 29 | # Configure browser settings 30 | browser_config = BrowserConfig( 31 | headless=True, 32 | verbose=False 33 | ) 34 | 35 | # Load instruction from file 36 | try: 37 | with open("prompts/extraction_prompt.txt", "r") as f: 38 | instruction = f.read() 39 | except FileNotFoundError: 40 | print("Warning: extraction_prompt.txt not found. Using default instruction.") 41 | instruction = "Extract structured data in the Results heading 2 which has the URL started with https://n8n.io/workflows/ according to the schema." 42 | 43 | url = "https://n8n.io/workflows/categories/ai/" 44 | # 1. Define the LLM extraction strategy 45 | llm_strategy = LLMExtractionStrategy( 46 | llm_config=LLMConfig( 47 | provider="openai/gpt-4o", 48 | api_token=api_key 49 | ), 50 | schema=ResultSchema.model_json_schema(), 51 | extraction_type="schema", 52 | instruction=instruction, 53 | chunk_token_threshold=1000, 54 | overlap_rate=0.0, 55 | apply_chunking=True, 56 | input_format="markdown", # or "html", "fit_markdown" 57 | extra_args={"temperature": 0.0, "max_tokens": 5000} 58 | ) 59 | 60 | # 2. Configure the crawler run with the extraction strategy 61 | run_config = CrawlerRunConfig( 62 | extraction_strategy=llm_strategy, 63 | cache_mode=CacheMode.BYPASS 64 | ) 65 | 66 | output_dir = "output" 67 | os.makedirs(output_dir, exist_ok=True) 68 | 69 | # Initialize the crawler 70 | async with AsyncWebCrawler(config=browser_config) as crawler: 71 | # Run the crawler with LLM extraction 72 | result = await crawler.arun(url=url, config=run_config) 73 | 74 | # Display results 75 | print(f"Crawl successful: {result.success}") 76 | if not result.success: 77 | print(f"Error: {result.error_message}") 78 | return 79 | 80 | # Process extracted content 81 | if hasattr(result, 'extracted_content') and result.extracted_content: 82 | print("\nExtracted JSON data:") 83 | try: 84 | extracted_data = json.loads(result.extracted_content) if isinstance(result.extracted_content, str) else result.extracted_content 85 | print(json.dumps(extracted_data, indent=2)) 86 | 87 | # Save the extracted JSON if output directory is specified 88 | if output_dir: 89 | output_path = Path(output_dir) 90 | output_path.mkdir(exist_ok=True, parents=True) 91 | 92 | # Create filename from URL 93 | url_filename = url.replace("://", "_").replace("/", "_").replace("?", "_") 94 | if len(url_filename) > 100: 95 | url_filename = url_filename[:100] 96 | 97 | # Save JSON and markdown 98 | json_file = output_path / f"{url_filename}.json" 99 | markdown_file = output_path / f"{url_filename}.md" 100 | 101 | with open(json_file, "w", encoding="utf-8") as f: 102 | json.dump(extracted_data, f, indent=2) 103 | 104 | with open(markdown_file, "w", encoding="utf-8") as f: 105 | f.write(result.markdown.raw_markdown if hasattr(result, 'markdown') and result.markdown else "") 106 | 107 | print(f"\nExtracted JSON saved to: {json_file}") 108 | print(f"Markdown content saved to: {markdown_file}") 109 | 110 | # Show usage statistics if available 111 | if hasattr(llm_strategy, 'show_usage'): 112 | llm_strategy.show_usage() 113 | 114 | except Exception as e: 115 | print(f"Error processing extracted content: {e}") 116 | else: 117 | print("\nNo structured data was extracted or extraction failed.") 118 | 119 | if __name__ == "__main__": 120 | load_dotenv(override=True) 121 | asyncio.run(main()) -------------------------------------------------------------------------------- /012_crawl4ai/3.multi_url_crawler.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import os 3 | import json 4 | from pathlib import Path 5 | from typing import List 6 | from models.schemas import ResultSchema 7 | from crawl4ai import ( 8 | AsyncWebCrawler, 9 | CrawlerRunConfig, 10 | CacheMode, 11 | BrowserConfig, 12 | SemaphoreDispatcher, 13 | RateLimiter 14 | ) 15 | 16 | async def read_urls_from_json(file_path: str) -> List[str]: 17 | """Read URLs from a JSON file containing a list of ResultSchema objects.""" 18 | path = Path(file_path) 19 | if not path.exists(): 20 | raise FileNotFoundError(f"JSON file not found: {file_path}") 21 | 22 | with open(path, 'r') as f: 23 | data = json.load(f) 24 | 25 | # Extract URLs from the JSON data 26 | urls = [] 27 | for item in data: 28 | # Create a ResultSchema from each item to validate the structure 29 | workflow = ResultSchema(**item) 30 | urls.append(workflow.url) 31 | 32 | return urls 33 | 34 | async def crawl_urls( 35 | urls: List[str], 36 | semaphore_count: int = 5, 37 | check_robots_txt: bool = True, 38 | cache_mode: CacheMode = CacheMode.ENABLED, 39 | output_dir: str = None 40 | ): 41 | """Crawl multiple URLs with semaphore-based concurrency and robots.txt respect.""" 42 | browser_config = BrowserConfig( 43 | headless=True, 44 | verbose=False 45 | ) 46 | 47 | run_config = CrawlerRunConfig( 48 | cache_mode=cache_mode, 49 | check_robots_txt=check_robots_txt, # Respect robots.txt 50 | stream=False # Disable streaming results to fix compatibility with SemaphoreDispatcher 51 | ) 52 | 53 | # Configure dispatcher with semaphore and rate limiting 54 | dispatcher = SemaphoreDispatcher( 55 | semaphore_count=semaphore_count, # Control concurrency 56 | rate_limiter=RateLimiter( 57 | base_delay=(1.0, 2.0), # Random delay between 1 and 2 seconds 58 | max_delay=10.0 # Maximum delay after backoff 59 | ) 60 | ) 61 | 62 | # Setup output directory if provided 63 | if output_dir: 64 | output_path = Path(output_dir) 65 | output_path.mkdir(exist_ok=True, parents=True) 66 | 67 | print(f"Starting crawl of {len(urls)} URLs with semaphore count: {semaphore_count}") 68 | print(f"Robots.txt checking: {'Enabled' if check_robots_txt else 'Disabled'}") 69 | 70 | async with AsyncWebCrawler(config=browser_config) as crawler: 71 | results = await crawler.arun_many( 72 | urls, 73 | config=run_config, 74 | dispatcher=dispatcher 75 | ) 76 | 77 | for result in results: 78 | if result.success: 79 | content_length = len(result.markdown.raw_markdown) if result.markdown else 0 80 | print(f"✅ {result.url} - {content_length} characters") 81 | 82 | # Save content to file if output directory is specified 83 | if output_dir: 84 | url_filename = result.url.replace("://", "_").replace("/", "_").replace("?", "_") 85 | if len(url_filename) > 100: 86 | url_filename = url_filename[:100] # Prevent extremely long filenames 87 | 88 | output_file = output_path / f"{url_filename}.md" 89 | with open(output_file, "w", encoding="utf-8") as f: 90 | f.write(result.markdown.raw_markdown if result.markdown else "") 91 | print(f" Saved to {output_file}") 92 | else: 93 | error_message = result.error_message or "Unknown error" 94 | if result.status_code == 403 and "robots.txt" in error_message: 95 | print(f"🚫 {result.url} - Blocked by robots.txt") 96 | else: 97 | print(f"❌ {result.url} - Error: {error_message}") 98 | 99 | return results 100 | 101 | async def main(): 102 | # Hardcoded configuration values 103 | urls_file = "output/https_n8n.io_workflows_categories_ai_.json" 104 | semaphore_count = 5 105 | check_robots_txt = True 106 | cache_mode = CacheMode.ENABLED 107 | output_dir = "output" 108 | 109 | try: 110 | urls = await read_urls_from_json(urls_file) 111 | if not urls: 112 | print("No valid URLs found in the file.") 113 | return 114 | 115 | print(f"Found {len(urls)} URLs to crawl") 116 | 117 | await crawl_urls( 118 | urls=urls, 119 | semaphore_count=semaphore_count, 120 | check_robots_txt=check_robots_txt, 121 | cache_mode=cache_mode, 122 | output_dir=output_dir 123 | ) 124 | 125 | except FileNotFoundError as e: 126 | print(f"Error: {e}") 127 | except Exception as e: 128 | print(f"An error occurred: {e}") 129 | 130 | if __name__ == "__main__": 131 | asyncio.run(main()) -------------------------------------------------------------------------------- /012_crawl4ai/4.crawl_with_profile.py: -------------------------------------------------------------------------------- 1 | """ 2 | N8N Authentication Profile Example with Crawl4AI 3 | 4 | This script demonstrates how to: 5 | 1. Create a persistent browser profile for n8n authentication 6 | 2. Save screenshots and content from authenticated pages 7 | 3. Use profiles for maintaining login sessions 8 | 9 | Based on the BrowserProfiler class for browser profile management. 10 | """ 11 | 12 | import asyncio 13 | import os 14 | import base64 15 | from pathlib import Path 16 | from colorama import Fore, Style, init 17 | from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode 18 | from crawl4ai.browser_profiler import BrowserProfiler 19 | from crawl4ai.async_logger import AsyncLogger 20 | 21 | # Initialize colorama for colored output 22 | init() 23 | 24 | # ======= CONFIGURATION (MODIFY THESE VALUES) ======= 25 | 26 | # URL to scrape 27 | TARGET_URL = "https://tubakhuym.app.n8n.cloud/workflow/nJA2ArB6nZzFvZIB" 28 | 29 | # Browser settings 30 | HEADLESS = False # Set to False to see the browser window 31 | BROWSER_TYPE = "chromium" # Use chromium (the browser type matters for the profile format) 32 | 33 | # Profile name (set to None to choose interactively or create new) 34 | PROFILE_NAME = "n8n-profile" 35 | 36 | # Content saving 37 | SAVE_CONTENT = True 38 | OUTPUT_MARKDOWN_FILENAME = "scraped_content.md" 39 | OUTPUT_HTML_FILENAME = "scraped_content.html" 40 | 41 | # Screenshot settings 42 | TAKE_SCREENSHOT = True 43 | SCREENSHOT_FILENAME = "screenshot.png" 44 | 45 | # Additional wait time before scraping (to ensure page loads properly) 46 | PRE_SCRAPE_WAIT_SECONDS = 5 47 | 48 | # ======= END CONFIGURATION ======= 49 | 50 | # Create a shared logger instance 51 | logger = AsyncLogger(verbose=True) 52 | 53 | # Create a shared BrowserProfiler instance 54 | profiler = BrowserProfiler(logger=logger) 55 | 56 | 57 | async def save_screenshot(screenshot_data, filename): 58 | """Helper function to save a screenshot""" 59 | if not screenshot_data: 60 | logger.warning(f"No screenshot data available", tag="SCREENSHOT") 61 | return False 62 | 63 | try: 64 | screenshot_path = Path(filename) 65 | 66 | # Try to decode if it's base64 encoded 67 | if isinstance(screenshot_data, str): 68 | try: 69 | # If it starts with common base64 image prefixes 70 | if screenshot_data.startswith(('data:image', 'iVBOR', '/9j/')): 71 | if screenshot_data.startswith('data:image'): 72 | # Extract the base64 part after the comma 73 | base64_data = screenshot_data.split(',', 1)[1] 74 | screenshot_bytes = base64.b64decode(base64_data) 75 | else: 76 | # Just try to decode directly 77 | screenshot_bytes = base64.b64decode(screenshot_data) 78 | else: 79 | screenshot_bytes = screenshot_data.encode('utf-8') 80 | except: 81 | screenshot_bytes = screenshot_data.encode('utf-8') 82 | else: 83 | screenshot_bytes = screenshot_data 84 | 85 | # Write to file 86 | with open(screenshot_path, "wb") as f: 87 | f.write(screenshot_bytes) 88 | 89 | logger.success(f"Screenshot saved to {Fore.GREEN}{screenshot_path.absolute()}{Style.RESET_ALL}", tag="SCREENSHOT") 90 | return True 91 | except Exception as e: 92 | logger.error(f"Error saving screenshot: {str(e)}", tag="SCREENSHOT") 93 | return False 94 | 95 | 96 | async def save_content(result, markdown_filename, html_filename): 97 | """Helper function to save scraped content""" 98 | if not result.success: 99 | logger.warning("No content to save - crawl was not successful", tag="CONTENT") 100 | return 101 | 102 | try: 103 | # Save markdown content 104 | md_output_file = Path(markdown_filename) 105 | md_output_file.write_text(result.markdown) 106 | logger.success(f"Markdown content saved to {Fore.GREEN}{md_output_file.absolute()}{Style.RESET_ALL}", tag="CONTENT") 107 | 108 | # Save HTML content 109 | html_output_file = Path(html_filename) 110 | html_output_file.write_text(result.html) 111 | logger.success(f"HTML content saved to {Fore.GREEN}{html_output_file.absolute()}{Style.RESET_ALL}", tag="CONTENT") 112 | except Exception as e: 113 | logger.error(f"Error saving content: {str(e)}", tag="CONTENT") 114 | 115 | 116 | async def crawl_with_profile(profile_path, url): 117 | """Use a profile to crawl an authenticated page""" 118 | logger.info(f"Crawling {Fore.CYAN}{url}{Style.RESET_ALL}", tag="CRAWL") 119 | logger.info(f"Using profile at {Fore.YELLOW}{profile_path}{Style.RESET_ALL}", tag="CRAWL") 120 | 121 | # Create browser config with the profile path 122 | browser_config = BrowserConfig( 123 | headless=HEADLESS, 124 | verbose=True, 125 | browser_type=BROWSER_TYPE, 126 | use_managed_browser=True, # Required for persistent profiles 127 | user_data_dir=profile_path 128 | ) 129 | 130 | # Set up crawler configuration 131 | crawl_config = CrawlerRunConfig( 132 | screenshot=TAKE_SCREENSHOT, 133 | cache_mode=CacheMode.BYPASS, # Don't use cache 134 | scan_full_page=True, # Ensure we scan the full page 135 | wait_for_images=True, # Wait for images to load 136 | js_code=f"await new Promise(resolve => setTimeout(resolve, {PRE_SCRAPE_WAIT_SECONDS * 1000})); return true;" 137 | ) 138 | 139 | # Initialize crawler with the browser config 140 | async with AsyncWebCrawler(config=browser_config, logger=logger) as crawler: 141 | # Open browser but wait for user confirmation before crawling 142 | logger.info(f"{Fore.YELLOW}Browser window opened. Please complete any authorization or permission dialogs.{Style.RESET_ALL}", tag="AUTH") 143 | confirmation = input(f"{Fore.CYAN}Press Enter when you've completed authorization to continue with the crawl: {Style.RESET_ALL}") 144 | 145 | start_time = asyncio.get_event_loop().time() 146 | 147 | # Crawl the URL - You should have access to authenticated content now 148 | logger.info(f"Starting crawl...", tag="CRAWL") 149 | result = await crawler.arun(url, config=crawl_config) 150 | 151 | elapsed_time = asyncio.get_event_loop().time() - start_time 152 | 153 | if result.success: 154 | # Log success 155 | logger.success(f"Crawl successful! ({elapsed_time:.2f}s)", tag="CRAWL") 156 | 157 | # Print page title 158 | title = result.metadata.get("title", "Unknown Title") 159 | logger.info(f"Page title: {Fore.GREEN}{title}{Style.RESET_ALL}", tag="CRAWL") 160 | 161 | # Save screenshot if requested 162 | if TAKE_SCREENSHOT and hasattr(result, 'screenshot'): 163 | await save_screenshot(result.screenshot, SCREENSHOT_FILENAME) 164 | 165 | # Save content if requested 166 | if SAVE_CONTENT: 167 | await save_content(result, OUTPUT_MARKDOWN_FILENAME, OUTPUT_HTML_FILENAME) 168 | 169 | return result 170 | else: 171 | # Log error status 172 | logger.error(f"Crawl failed: {result.error_message}", tag="CRAWL") 173 | return None 174 | 175 | 176 | async def main(): 177 | logger.info(f"{Fore.CYAN}N8N Authentication Profile Example{Style.RESET_ALL}", tag="DEMO") 178 | 179 | # Choose between interactive mode and automatic mode 180 | mode_input = input(f"{Fore.CYAN}Run in [i]nteractive mode or [a]utomatic mode? (i/a): {Style.RESET_ALL}").lower() 181 | 182 | if mode_input == 'i': 183 | # Interactive profile management 184 | logger.info("Starting interactive profile manager...", tag="DEMO") 185 | await profiler.interactive_manager(crawl_callback=crawl_with_profile) 186 | else: 187 | # Automatic mode 188 | profiles = profiler.list_profiles() 189 | selected_profile = None 190 | 191 | # If a specific profile name was requested 192 | if PROFILE_NAME: 193 | for profile in profiles: 194 | if profile["name"] == PROFILE_NAME: 195 | selected_profile = profile 196 | break 197 | 198 | if selected_profile: 199 | logger.info(f"Using existing profile: {Fore.CYAN}{selected_profile['name']}{Style.RESET_ALL}", tag="DEMO") 200 | profile_path = selected_profile["path"] 201 | elif profiles: 202 | # Use the first profile if we have any 203 | selected_profile = profiles[0] 204 | logger.info(f"Using most recent profile: {Fore.CYAN}{selected_profile['name']}{Style.RESET_ALL}", tag="DEMO") 205 | profile_path = selected_profile["path"] 206 | else: 207 | # Create a new profile if none exists 208 | logger.info("No profiles found. Creating a new one...", tag="DEMO") 209 | logger.info(f"{Fore.YELLOW}IMPORTANT: Please log in to n8n in the browser window that will open.{Style.RESET_ALL}", tag="DEMO") 210 | logger.info(f"When finished, press 'q' in this terminal to save the profile.", tag="DEMO") 211 | 212 | profile_path = await profiler.create_profile() 213 | if not profile_path: 214 | logger.error("Cannot proceed without a valid profile", tag="DEMO") 215 | return 216 | 217 | # Verify profile path exists 218 | if not os.path.exists(profile_path): 219 | logger.warning(f"Profile path does not exist: {profile_path}", tag="DEMO") 220 | logger.info("Creating a new profile instead...", tag="DEMO") 221 | profile_path = await profiler.create_profile() 222 | if not profile_path: 223 | logger.error("Cannot proceed without a valid profile", tag="DEMO") 224 | return 225 | 226 | # Crawl the target URL 227 | await crawl_with_profile(profile_path, TARGET_URL) 228 | 229 | 230 | if __name__ == "__main__": 231 | try: 232 | # Run the async main function 233 | asyncio.run(main()) 234 | except KeyboardInterrupt: 235 | logger.warning("Example interrupted by user", tag="DEMO") 236 | except Exception as e: 237 | logger.error(f"Error in example: {str(e)}", tag="DEMO") -------------------------------------------------------------------------------- /012_crawl4ai/5.update_profile.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | """ 3 | Browser Profile Manager for Crawl4AI 4 | 5 | This script helps manage browser profiles for identity-based crawling with Crawl4AI. 6 | It allows users to create, list, delete, and test profiles for authenticated browsing. 7 | """ 8 | 9 | import asyncio 10 | import sys 11 | import time 12 | from pathlib import Path 13 | from crawl4ai import BrowserProfiler, AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode 14 | 15 | # Default wait time for system prompts (like keychain access) 16 | SYSTEM_PROMPT_WAIT = 5 17 | 18 | async def create_or_update_profile(profiler, profile_name=None, initial_url=None): 19 | """ 20 | Create a new profile or update an existing one. 21 | 22 | Args: 23 | profiler: BrowserProfiler instance 24 | profile_name: Optional name for the profile 25 | initial_url: Optional URL to open initially (for user guidance only) 26 | 27 | Returns: 28 | Path to the created/updated profile or None if cancelled 29 | """ 30 | try: 31 | # Get profile name if not provided 32 | if profile_name is None: 33 | profile_name = input("Enter profile name (e.g., facebook-profile): ").strip() 34 | if not profile_name: 35 | print("Profile name cannot be empty.") 36 | return None 37 | 38 | # Check if profile exists 39 | existing_path = profiler.get_profile_path(profile_name) 40 | if existing_path: 41 | print(f"Profile '{profile_name}' already exists at: {existing_path}") 42 | update = input("Do you want to update this profile? (y/n): ").strip().lower() 43 | if update != 'y': 44 | return None 45 | 46 | # Get initial URL if not provided (for instruction purposes only) 47 | if initial_url is None: 48 | initial_url = input("\nEnter URL you plan to visit first (leave empty for blank page): ").strip() 49 | 50 | # User instructions 51 | print("\nA browser window will open. Please:") 52 | print("1. Log in to the websites you want to access") 53 | if initial_url: 54 | print(f" (Navigate to {initial_url} first)") 55 | print("2. Configure any other browser settings as needed") 56 | print("3. When finished, return to this terminal and press 'q' to save the profile") 57 | input("\nPress Enter to continue...") 58 | 59 | # Create the profile interactively 60 | print("\nOpening browser window. Log in to your accounts and set up your preferences...") 61 | profile_path = await profiler.create_profile(profile_name=profile_name) 62 | 63 | print(f"\nProfile successfully {'updated' if existing_path else 'created'} and saved at: {profile_path}") 64 | print(f"\nYou can now use this profile with crawl_with_profile.py by setting PROFILE_NAME = '{profile_name}'") 65 | 66 | return profile_path 67 | except Exception as e: 68 | print(f"Error creating/updating profile: {str(e)}") 69 | return None 70 | 71 | def list_profiles(profiler): 72 | """ 73 | List all available profiles. 74 | 75 | Args: 76 | profiler: BrowserProfiler instance 77 | 78 | Returns: 79 | List of profile dictionaries or empty list if none found 80 | """ 81 | try: 82 | profiles = profiler.list_profiles() 83 | if not profiles: 84 | print("\nNo profiles found.") 85 | return [] 86 | 87 | print("\nAvailable profiles:") 88 | for i, profile in enumerate(profiles, 1): 89 | print(f"{i}. {profile['name']} (Created: {profile['created']})") 90 | print(f" Path: {profile['path']}") 91 | print(f" Browser type: {profile['type']}") 92 | print() 93 | 94 | return profiles 95 | except Exception as e: 96 | print(f"Error listing profiles: {str(e)}") 97 | return [] 98 | 99 | def delete_profile(profiler): 100 | """ 101 | Delete a selected profile. 102 | 103 | Args: 104 | profiler: BrowserProfiler instance 105 | """ 106 | try: 107 | profiles = profiler.list_profiles() 108 | if not profiles: 109 | print("\nNo profiles found to delete.") 110 | return 111 | 112 | print("\nSelect a profile to delete:") 113 | for i, profile in enumerate(profiles, 1): 114 | print(f"{i}. {profile['name']} (Created: {profile['created']})") 115 | 116 | try: 117 | choice = int(input("\nEnter profile number to delete (0 to cancel): ")) 118 | if choice == 0: 119 | return 120 | 121 | if 1 <= choice <= len(profiles): 122 | profile_name = profiles[choice-1]['name'] 123 | confirm = input(f"Are you sure you want to delete profile '{profile_name}'? (y/n): ").strip().lower() 124 | 125 | if confirm == 'y': 126 | success = profiler.delete_profile(profile_name) 127 | if success: 128 | print(f"Profile '{profile_name}' deleted successfully.") 129 | else: 130 | print(f"Failed to delete profile '{profile_name}'.") 131 | else: 132 | print("Invalid choice.") 133 | except ValueError: 134 | print("Please enter a valid number.") 135 | except Exception as e: 136 | print(f"Error deleting profile: {str(e)}") 137 | 138 | async def test_profile(profiler): 139 | """ 140 | Test a profile by crawling a specified URL. 141 | 142 | Args: 143 | profiler: BrowserProfiler instance 144 | """ 145 | try: 146 | profiles = list_profiles(profiler) 147 | if not profiles: 148 | return 149 | 150 | try: 151 | choice = int(input("\nEnter profile number to test (0 to cancel): ")) 152 | if choice == 0: 153 | return 154 | 155 | if 1 <= choice <= len(profiles): 156 | profile_name = profiles[choice-1]['name'] 157 | profile_path = profiles[choice-1]['path'] 158 | browser_type = profiles[choice-1]['type'] 159 | 160 | url = input("\nEnter URL to test crawling (e.g., https://example.com): ").strip() 161 | if not url: 162 | print("URL cannot be empty.") 163 | return 164 | 165 | headless = input("Run in headless mode? (y/n, default: n): ").strip().lower() == 'y' 166 | take_screenshot = input("Take screenshot? (y/n, default: y): ").strip().lower() != 'n' 167 | 168 | print(f"\nTesting profile '{profile_name}' on {url}...") 169 | 170 | # Configure the browser with the profile 171 | browser_config = BrowserConfig( 172 | headless=headless, 173 | verbose=True, 174 | browser_type=browser_type, 175 | use_managed_browser=True, 176 | user_data_dir=profile_path 177 | ) 178 | 179 | # Set up crawler configuration 180 | crawl_config = CrawlerRunConfig( 181 | screenshot=take_screenshot, 182 | cache_mode=CacheMode.BYPASS, 183 | ) 184 | 185 | # Initialize the crawler 186 | async with AsyncWebCrawler(config=browser_config) as crawler: 187 | # Add a delay for potential system prompts 188 | wait_time = int(input(f"Wait time for system prompts (default: {SYSTEM_PROMPT_WAIT} seconds): ") or SYSTEM_PROMPT_WAIT) 189 | print(f"\nWaiting {wait_time} seconds for potential system prompts...") 190 | time.sleep(wait_time) 191 | 192 | # Proceed with the actual crawling 193 | result = await crawler.arun(url=url, config=crawl_config) 194 | 195 | if result.success: 196 | print("\nCrawling successful!") 197 | 198 | # Save screenshot if available 199 | if result.screenshot and take_screenshot: 200 | screenshot_file = Path(f"test_{profile_name}_screenshot.png") 201 | with open(screenshot_file, "wb") as f: 202 | f.write(result.screenshot) 203 | print(f"Screenshot saved to {screenshot_file.absolute()}") 204 | 205 | # Ask if user wants to save the content 206 | save_content = input("\nSave crawled content? (y/n): ").strip().lower() == 'y' 207 | if save_content: 208 | output_file = Path(f"test_{profile_name}_content.md") 209 | output_file.write_text(result.markdown) 210 | print(f"Content saved to {output_file.absolute()}") 211 | else: 212 | print(f"\nError: {result.error_message}") 213 | else: 214 | print("Invalid choice.") 215 | except ValueError: 216 | print("Please enter a valid number.") 217 | except Exception as e: 218 | print(f"An error occurred during testing: {str(e)}") 219 | 220 | async def main(): 221 | """Main function that presents the interactive menu.""" 222 | try: 223 | # Initialize the BrowserProfiler 224 | profiler = BrowserProfiler() 225 | 226 | while True: 227 | print("\n===== Browser Profile Manager =====") 228 | print("1. Create/Update Profile") 229 | print("2. List Available Profiles") 230 | print("3. Delete Profile") 231 | print("4. Test Profile") 232 | print("5. Exit") 233 | 234 | try: 235 | choice = input("\nEnter your choice (1-5): ").strip() 236 | 237 | if choice == '1': 238 | await create_or_update_profile(profiler) 239 | elif choice == '2': 240 | list_profiles(profiler) 241 | elif choice == '3': 242 | delete_profile(profiler) 243 | elif choice == '4': 244 | await test_profile(profiler) 245 | elif choice == '5': 246 | print("Exiting Profile Manager.") 247 | break 248 | else: 249 | print("Invalid choice. Please enter a number between 1 and 5.") 250 | except Exception as e: 251 | print(f"Error processing menu choice: {str(e)}") 252 | print("Please try again.") 253 | except Exception as e: 254 | print(f"Critical error: {str(e)}") 255 | sys.exit(1) 256 | 257 | if __name__ == "__main__": 258 | try: 259 | asyncio.run(main()) 260 | except KeyboardInterrupt: 261 | print("\nOperation cancelled by user.") 262 | sys.exit(0) 263 | except Exception as e: 264 | print(f"Unhandled exception: {str(e)}") 265 | sys.exit(1) -------------------------------------------------------------------------------- /012_crawl4ai/env.example: -------------------------------------------------------------------------------- 1 | OPENAI_API_KEY=sk-proj-1234567890 -------------------------------------------------------------------------------- /012_crawl4ai/models/schemas.py: -------------------------------------------------------------------------------- 1 | from pydantic import BaseModel, Field 2 | 3 | class ResultSchema(BaseModel): 4 | title: str = Field(description="The title of the workflow") 5 | description: str = Field(description="The description of the workflow") 6 | url: str = Field(description="The URL of the workflow") 7 | name: str = Field(description="The name of the workflow") -------------------------------------------------------------------------------- /012_crawl4ai/prompts/extraction_prompt.txt: -------------------------------------------------------------------------------- 1 | You are an expert data extractor for n8n.io workflows. Your task is to extract structured information about workflows from the page content. 2 | 3 | For each workflow card or entry on the page: 4 | 1. Identify the workflow title 5 | 2. Extract the description of what the workflow does 6 | 3. Note the category it belongs to 7 | 4. Find the author or creator if available 8 | 5. Capture the direct URL to that workflow 9 | 6. Identify any tags associated with the workflow 10 | 11 | Also extract: 12 | - The total number of workflows displayed on the page 13 | - Any category filters or options shown on the page 14 | - The main page title 15 | 16 | Ensure your extraction is accurate and follows the provided schema structure. If certain information isn't available, it's okay to omit those fields rather than making assumptions. -------------------------------------------------------------------------------- /012_crawl4ai/requirements.txt: -------------------------------------------------------------------------------- 1 | crawl4ai==0.5.0.post8 -------------------------------------------------------------------------------- /014_mcp_server_n8n/n8n_mcp_server_collection.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "MCP Servers", 3 | "nodes": [ 4 | { 5 | "parameters": { 6 | "path": "4665dadb-5fc8-431a-8a67-653ff128f126" 7 | }, 8 | "type": "@n8n/n8n-nodes-langchain.mcpTrigger", 9 | "typeVersion": 1, 10 | "position": [ 11 | -260, 12 | -280 13 | ], 14 | "id": "4170d3cc-8e99-4d96-94e2-d2a61fc87311", 15 | "name": "MCP Server Notion", 16 | "webhookId": "4665dadb-5fc8-431a-8a67-653ff128f126" 17 | }, 18 | { 19 | "parameters": { 20 | "path": "761ee5e0-3afb-4354-b7ef-7085ff693022" 21 | }, 22 | "type": "@n8n/n8n-nodes-langchain.mcpTrigger", 23 | "typeVersion": 1, 24 | "position": [ 25 | 380, 26 | -280 27 | ], 28 | "id": "2768cee0-254c-481e-a74e-92adf8b969b7", 29 | "name": "MCP Server Firecrawl", 30 | "webhookId": "761ee5e0-3afb-4354-b7ef-7085ff693022" 31 | }, 32 | { 33 | "parameters": { 34 | "toolDescription": "This tool scrape the content of an URL by using Firecrawl API", 35 | "method": "POST", 36 | "url": "https://api.firecrawl.dev/v1/scrape", 37 | "sendHeaders": true, 38 | "parametersHeaders": { 39 | "values": [ 40 | { 41 | "name": "Content-Type", 42 | "valueProvider": "fieldValue", 43 | "value": "application/json" 44 | }, 45 | { 46 | "name": "Authorization", 47 | "valueProvider": "fieldValue", 48 | "value": "Bearer " 49 | } 50 | ] 51 | }, 52 | "sendBody": true, 53 | "specifyBody": "=json", 54 | "parametersBody": { 55 | "values": [ 56 | {} 57 | ] 58 | }, 59 | "jsonBody": "{\n \"url\": \"{URL}\",\n \"formats\": [ \"markdown\" ]\n}", 60 | "placeholderDefinitions": { 61 | "values": [ 62 | { 63 | "name": "URL", 64 | "description": "The URL you want to perform the scape", 65 | "type": "string" 66 | } 67 | ] 68 | } 69 | }, 70 | "type": "@n8n/n8n-nodes-langchain.toolHttpRequest", 71 | "typeVersion": 1.1, 72 | "position": [ 73 | 420, 74 | -60 75 | ], 76 | "id": "0112f50f-7d70-4913-af78-0ba26209e309", 77 | "name": "Scrape" 78 | }, 79 | { 80 | "parameters": { 81 | "operation": "get", 82 | "channelId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Channel_ID', `The youtube channel id`, 'string') }}" 83 | }, 84 | "type": "n8n-nodes-base.youTubeTool", 85 | "typeVersion": 1, 86 | "position": [ 87 | -60, 88 | 460 89 | ], 90 | "id": "1efe021a-d6e3-4c2d-9e33-1c20b8b380a5", 91 | "name": "Channel ID", 92 | "credentials": { 93 | "youTubeOAuth2Api": { 94 | "id": "XA8Zd3E0W9G8V3Po", 95 | "name": "YouTube account" 96 | } 97 | } 98 | }, 99 | { 100 | "parameters": { 101 | "path": "4db199c1-11d4-4945-b647-8d3839801e0b" 102 | }, 103 | "type": "@n8n/n8n-nodes-langchain.mcpTrigger", 104 | "typeVersion": 1, 105 | "position": [ 106 | -280, 107 | 240 108 | ], 109 | "id": "567b6b69-e5da-4c4e-942c-3b527f2b2b9a", 110 | "name": "MCP Server Youtube", 111 | "webhookId": "4db199c1-11d4-4945-b647-8d3839801e0b" 112 | }, 113 | { 114 | "parameters": { 115 | "path": "697d65b2-2670-4966-a824-dc6e5d54c003" 116 | }, 117 | "type": "@n8n/n8n-nodes-langchain.mcpTrigger", 118 | "typeVersion": 1, 119 | "position": [ 120 | 400, 121 | 240 122 | ], 123 | "id": "2ad03107-5db9-4903-a4aa-0512a79f8a31", 124 | "name": "MCP Server Gmail", 125 | "webhookId": "697d65b2-2670-4966-a824-dc6e5d54c003" 126 | }, 127 | { 128 | "parameters": { 129 | "operation": "get", 130 | "messageId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Message_ID', `Get a specific message by using Message ID`, 'string') }}" 131 | }, 132 | "type": "n8n-nodes-base.gmailTool", 133 | "typeVersion": 2.1, 134 | "position": [ 135 | 620, 136 | 460 137 | ], 138 | "id": "a758be97-1b85-4ca3-928d-6d827a67d7df", 139 | "name": "Get", 140 | "webhookId": "3a215344-01c4-4a2b-9f4e-39168fba9167", 141 | "credentials": { 142 | "gmailOAuth2": { 143 | "id": "HrUawguRK1ZddhW1", 144 | "name": "Gmail account" 145 | } 146 | } 147 | }, 148 | { 149 | "parameters": { 150 | "operation": "markAsRead", 151 | "messageId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Message_ID', `Mark a specific message as Read by using Message ID`, 'string') }}" 152 | }, 153 | "type": "n8n-nodes-base.gmailTool", 154 | "typeVersion": 2.1, 155 | "position": [ 156 | 480, 157 | 460 158 | ], 159 | "id": "01308e21-3279-43bb-a8c8-3eb21e64cf1e", 160 | "name": "Mark As Read", 161 | "webhookId": "b22227e3-cfd5-440d-b229-f657efeb357c", 162 | "credentials": { 163 | "gmailOAuth2": { 164 | "id": "HrUawguRK1ZddhW1", 165 | "name": "Gmail account" 166 | } 167 | } 168 | }, 169 | { 170 | "parameters": { 171 | "descriptionType": "manual", 172 | "operation": "getAll", 173 | "returnAll": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Return_All', `Get mails from google mail`, 'boolean') }}", 174 | "filters": {} 175 | }, 176 | "type": "n8n-nodes-base.gmailTool", 177 | "typeVersion": 2.1, 178 | "position": [ 179 | 340, 180 | 460 181 | ], 182 | "id": "4165d118-7341-48af-b409-73e8dd390bfa", 183 | "name": "Get All", 184 | "webhookId": "4337de8d-285b-4743-b9e8-e99257395acb", 185 | "credentials": { 186 | "gmailOAuth2": { 187 | "id": "HrUawguRK1ZddhW1", 188 | "name": "Gmail account" 189 | } 190 | } 191 | }, 192 | { 193 | "parameters": { 194 | "operation": "reply", 195 | "messageId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Message_ID', `The message that receives the reply by using Message ID`, 'string') }}", 196 | "message": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Message', `The content of the Message`, 'string') }}", 197 | "options": {} 198 | }, 199 | "type": "n8n-nodes-base.gmailTool", 200 | "typeVersion": 2.1, 201 | "position": [ 202 | 780, 203 | 460 204 | ], 205 | "id": "e5171cb3-6bcf-4eb5-a74b-6f8e30787099", 206 | "name": "Reply", 207 | "webhookId": "5a5dd4f4-ee98-4f94-95ce-a149e03d0c10", 208 | "credentials": { 209 | "gmailOAuth2": { 210 | "id": "HrUawguRK1ZddhW1", 211 | "name": "Gmail account" 212 | } 213 | } 214 | }, 215 | { 216 | "parameters": { 217 | "descriptionType": "manual", 218 | "pageId": { 219 | "__rl": true, 220 | "mode": "url", 221 | "value": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Parent_Page', ``, 'string') }}" 222 | }, 223 | "title": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Title', ``, 'string') }}", 224 | "options": {} 225 | }, 226 | "type": "n8n-nodes-base.notionTool", 227 | "typeVersion": 2.2, 228 | "position": [ 229 | -180, 230 | -60 231 | ], 232 | "id": "3c8add63-04d2-43e2-8a8a-bc97c0c462c9", 233 | "name": "CreatePage", 234 | "credentials": { 235 | "notionApi": { 236 | "id": "ricgFSt0RLvphcmg", 237 | "name": "Notion account" 238 | } 239 | } 240 | }, 241 | { 242 | "parameters": { 243 | "descriptionType": "manual", 244 | "operation": "search", 245 | "text": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Search_Text', `The text to search for\n`, 'string') }}", 246 | "returnAll": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Return_All', `Whether to return all results or only up to a given limit\n`, 'boolean') }}", 247 | "options": {} 248 | }, 249 | "type": "n8n-nodes-base.notionTool", 250 | "typeVersion": 2.2, 251 | "position": [ 252 | -60, 253 | -60 254 | ], 255 | "id": "0c0a1e12-8575-4c37-a099-c9a397b0115b", 256 | "name": "SearchPage", 257 | "credentials": { 258 | "notionApi": { 259 | "id": "ricgFSt0RLvphcmg", 260 | "name": "Notion account" 261 | } 262 | } 263 | }, 264 | { 265 | "parameters": { 266 | "descriptionType": "manual", 267 | "resource": "block", 268 | "blockId": { 269 | "__rl": true, 270 | "value": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Block', `The Notion Block to get all children from, when using 'By URL' mode make sure to use the URL of the block itself, you can find it in block parameters in Notion under 'Copy link to block'\n`, 'string') }}", 271 | "mode": "url" 272 | } 273 | }, 274 | "type": "n8n-nodes-base.notionTool", 275 | "typeVersion": 2.2, 276 | "position": [ 277 | 60, 278 | -60 279 | ], 280 | "id": "d355c448-3ded-416d-b2d2-c16341b8224a", 281 | "name": "BlockAppendAfter", 282 | "credentials": { 283 | "notionApi": { 284 | "id": "ricgFSt0RLvphcmg", 285 | "name": "Notion account" 286 | } 287 | } 288 | }, 289 | { 290 | "parameters": { 291 | "descriptionType": "manual", 292 | "resource": "block", 293 | "operation": "getAll", 294 | "blockId": { 295 | "__rl": true, 296 | "value": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Block', `The Notion Block to get all children from, when using 'By URL' mode make sure to use the URL of the block itself, you can find it in block parameters in Notion under 'Copy link to block'\n`, 'string') }}", 297 | "mode": "url" 298 | }, 299 | "returnAll": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Return_All', `Whether to return all results or only up to a given limit\n`, 'boolean') }}", 300 | "fetchNestedBlocks": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Also_Fetch_Nested_Blocks', ``, 'boolean') }}" 301 | }, 302 | "type": "n8n-nodes-base.notionTool", 303 | "typeVersion": 2.2, 304 | "position": [ 305 | 180, 306 | -60 307 | ], 308 | "id": "564a2fee-0c4a-4e1e-94df-4857ddc7cd19", 309 | "name": "BlockGetChildBlocks", 310 | "credentials": { 311 | "notionApi": { 312 | "id": "ricgFSt0RLvphcmg", 313 | "name": "Notion account" 314 | } 315 | } 316 | }, 317 | { 318 | "parameters": { 319 | "toolDescription": "This tool scrape the content of an URL by using Firecrawl API", 320 | "method": "POST", 321 | "url": "https://api.firecrawl.dev/v1/scrape", 322 | "sendHeaders": true, 323 | "parametersHeaders": { 324 | "values": [ 325 | { 326 | "name": "Content-Type", 327 | "valueProvider": "fieldValue", 328 | "value": "application/json" 329 | }, 330 | { 331 | "name": "Authorization", 332 | "valueProvider": "fieldValue", 333 | "value": "Bearer " 334 | } 335 | ] 336 | }, 337 | "sendBody": true, 338 | "specifyBody": "=json", 339 | "parametersBody": { 340 | "values": [ 341 | {} 342 | ] 343 | }, 344 | "jsonBody": "{\n \"url\": \"{URL}\",\n \"limit\": 10,\n \"scrapeOptions\": {\n \"formats\": [ \"markdown\" ]\n }\n}", 345 | "placeholderDefinitions": { 346 | "values": [ 347 | { 348 | "name": "URL", 349 | "description": "The URL you want to perform the scape", 350 | "type": "string" 351 | } 352 | ] 353 | } 354 | }, 355 | "type": "@n8n/n8n-nodes-langchain.toolHttpRequest", 356 | "typeVersion": 1.1, 357 | "position": [ 358 | 600, 359 | -60 360 | ], 361 | "id": "a947b349-1a5f-4d38-bc7c-4975c9344a8f", 362 | "name": "Crawl" 363 | }, 364 | { 365 | "parameters": { 366 | "resource": "video", 367 | "returnAll": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Return_All', `Whether to return all results or only up to a given limit\n`, 'boolean') }}", 368 | "filters": {}, 369 | "options": {} 370 | }, 371 | "type": "n8n-nodes-base.youTubeTool", 372 | "typeVersion": 1, 373 | "position": [ 374 | 60, 375 | 460 376 | ], 377 | "id": "e381d078-04f9-4812-8e5e-afa398c00ea6", 378 | "name": "Get Many Videos", 379 | "credentials": { 380 | "youTubeOAuth2Api": { 381 | "id": "XA8Zd3E0W9G8V3Po", 382 | "name": "YouTube account" 383 | } 384 | } 385 | }, 386 | { 387 | "parameters": { 388 | "resource": "video", 389 | "operation": "get", 390 | "videoId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Video_ID', `The youtube video id`, 'string') }}", 391 | "part": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Fields', ``, 'string') }}", 392 | "options": {} 393 | }, 394 | "type": "n8n-nodes-base.youTubeTool", 395 | "typeVersion": 1, 396 | "position": [ 397 | -200, 398 | 460 399 | ], 400 | "id": "48862a94-8602-4859-8d36-1b46c38b3470", 401 | "name": "Video ID", 402 | "credentials": { 403 | "youTubeOAuth2Api": { 404 | "id": "XA8Zd3E0W9G8V3Po", 405 | "name": "YouTube account" 406 | } 407 | } 408 | }, 409 | { 410 | "parameters": { 411 | "path": "c08ac829-d451-4590-8608-fb91a41b20e3" 412 | }, 413 | "type": "@n8n/n8n-nodes-langchain.mcpTrigger", 414 | "typeVersion": 1, 415 | "position": [ 416 | -300, 417 | 720 418 | ], 419 | "id": "c20b7e50-9589-4948-b2db-3259978f1ef8", 420 | "name": "MCP Server BrowserUse", 421 | "webhookId": "c08ac829-d451-4590-8608-fb91a41b20e3" 422 | }, 423 | { 424 | "parameters": { 425 | "connectionType": "local", 426 | "operation": "getTask", 427 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 428 | }, 429 | "type": "n8n-nodes-browser-use.browserUseTool", 430 | "typeVersion": 1, 431 | "position": [ 432 | -20, 433 | 940 434 | ], 435 | "id": "7c3e42c8-6f15-4639-95f8-575f4b60ff1a", 436 | "name": "Get Task", 437 | "credentials": { 438 | "browserUseLocalBridgeApi": { 439 | "id": "V5HW3VZJeodhd8xk", 440 | "name": "Browser Use Cloud account" 441 | } 442 | } 443 | }, 444 | { 445 | "parameters": { 446 | "connectionType": "local", 447 | "instructions": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Instructions', ``, 'string') }}", 448 | "aiProvider": "google" 449 | }, 450 | "type": "n8n-nodes-browser-use.browserUseTool", 451 | "typeVersion": 1, 452 | "position": [ 453 | -220, 454 | 940 455 | ], 456 | "id": "1a617e20-d25d-46f4-8a5d-b33cf48f2144", 457 | "name": "Run Task", 458 | "credentials": { 459 | "browserUseLocalBridgeApi": { 460 | "id": "V5HW3VZJeodhd8xk", 461 | "name": "Browser Use Cloud account" 462 | } 463 | } 464 | }, 465 | { 466 | "parameters": { 467 | "connectionType": "local", 468 | "operation": "getTaskStatus", 469 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 470 | }, 471 | "type": "n8n-nodes-browser-use.browserUseTool", 472 | "typeVersion": 1, 473 | "position": [ 474 | 180, 475 | 940 476 | ], 477 | "id": "daa0be4c-dde7-44ed-a0b5-201a9e62da8c", 478 | "name": "Get Task Status", 479 | "credentials": { 480 | "browserUseLocalBridgeApi": { 481 | "id": "V5HW3VZJeodhd8xk", 482 | "name": "Browser Use Cloud account" 483 | } 484 | } 485 | }, 486 | { 487 | "parameters": { 488 | "connectionType": "local", 489 | "operation": "pauseTask", 490 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 491 | }, 492 | "type": "n8n-nodes-browser-use.browserUseTool", 493 | "typeVersion": 1, 494 | "position": [ 495 | 380, 496 | 940 497 | ], 498 | "id": "863ea303-60c6-433b-b37d-656f0c094232", 499 | "name": "Pause Task", 500 | "credentials": { 501 | "browserUseLocalBridgeApi": { 502 | "id": "V5HW3VZJeodhd8xk", 503 | "name": "Browser Use Cloud account" 504 | } 505 | } 506 | }, 507 | { 508 | "parameters": { 509 | "connectionType": "local", 510 | "operation": "stopTask", 511 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 512 | }, 513 | "type": "n8n-nodes-browser-use.browserUseTool", 514 | "typeVersion": 1, 515 | "position": [ 516 | 580, 517 | 940 518 | ], 519 | "id": "a1e239ed-8555-47d0-a9af-79827aef579a", 520 | "name": "Stop Task", 521 | "credentials": { 522 | "browserUseLocalBridgeApi": { 523 | "id": "V5HW3VZJeodhd8xk", 524 | "name": "Browser Use Cloud account" 525 | } 526 | } 527 | }, 528 | { 529 | "parameters": { 530 | "connectionType": "local", 531 | "operation": "resumeTask", 532 | "taskId": "={{ /*n8n-auto-generated-fromAI-override*/ $fromAI('Task_ID', ``, 'string') }}" 533 | }, 534 | "type": "n8n-nodes-browser-use.browserUseTool", 535 | "typeVersion": 1, 536 | "position": [ 537 | 780, 538 | 940 539 | ], 540 | "id": "2232a399-a338-4edc-98c9-7b8412ea9737", 541 | "name": "Resume Task", 542 | "credentials": { 543 | "browserUseLocalBridgeApi": { 544 | "id": "V5HW3VZJeodhd8xk", 545 | "name": "Browser Use Cloud account" 546 | } 547 | } 548 | }, 549 | { 550 | "parameters": { 551 | "modelName": "models/gemini-embedding-exp-03-07" 552 | }, 553 | "type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini", 554 | "typeVersion": 1, 555 | "position": [ 556 | 1020, 557 | 100 558 | ], 559 | "id": "43540483-8787-42fc-8f65-e3445a8c6ed5", 560 | "name": "Embeddings Google Gemini", 561 | "credentials": { 562 | "googlePalmApi": { 563 | "id": "VytWkQ0CA5yghGIf", 564 | "name": "Google Gemini(PaLM) Api account" 565 | } 566 | } 567 | }, 568 | { 569 | "parameters": { 570 | "path": "aec5c7b1-d371-4dc5-8419-a3925ac2eee4" 571 | }, 572 | "type": "@n8n/n8n-nodes-langchain.mcpTrigger", 573 | "typeVersion": 1, 574 | "position": [ 575 | 1020, 576 | -300 577 | ], 578 | "id": "e5d16860-392f-4d73-94ac-e7c189f87ac1", 579 | "name": "MCP Server Vector Store", 580 | "webhookId": "aec5c7b1-d371-4dc5-8419-a3925ac2eee4" 581 | }, 582 | { 583 | "parameters": { 584 | "descriptionType": "manual", 585 | "toolDescription": "Use this tool to fetch all available documents, including the table schema if the file is a CSV or Excel file.", 586 | "operation": "select", 587 | "schema": { 588 | "__rl": true, 589 | "mode": "list", 590 | "value": "public" 591 | }, 592 | "table": { 593 | "__rl": true, 594 | "value": "document_metadata", 595 | "mode": "list", 596 | "cachedResultName": "document_metadata" 597 | }, 598 | "returnAll": true, 599 | "options": {} 600 | }, 601 | "type": "n8n-nodes-base.postgresTool", 602 | "typeVersion": 2.5, 603 | "position": [ 604 | 1340, 605 | -60 606 | ], 607 | "id": "d2bd5c04-0962-4fa9-a125-be09500f6c1d", 608 | "name": "List Documents", 609 | "credentials": { 610 | "postgres": { 611 | "id": "7aZHrNJx2qgPLE8U", 612 | "name": "Supabase Gemini" 613 | } 614 | } 615 | }, 616 | { 617 | "parameters": { 618 | "descriptionType": "manual", 619 | "toolDescription": "Given a file ID, fetches the text from the document.", 620 | "operation": "executeQuery", 621 | "query": "SELECT \n string_agg(content, ' ') as document_text\nFROM documents\n WHERE metadata->>'file_id' = $1\nGROUP BY metadata->>'file_id';", 622 | "options": { 623 | "queryReplacement": "={{ $fromAI('file_id') }}" 624 | } 625 | }, 626 | "type": "n8n-nodes-base.postgresTool", 627 | "typeVersion": 2.5, 628 | "position": [ 629 | 1480, 630 | -60 631 | ], 632 | "id": "8be02b06-64be-4944-8bf3-5279611b9dc6", 633 | "name": "Get File Contents", 634 | "credentials": { 635 | "postgres": { 636 | "id": "7aZHrNJx2qgPLE8U", 637 | "name": "Supabase Gemini" 638 | } 639 | } 640 | }, 641 | { 642 | "parameters": { 643 | "descriptionType": "manual", 644 | "toolDescription": "Run a SQL query - use this to query from the document_rows table once you know the file ID you are querying. dataset_id is the file_id and you are always using the row_data for filtering, which is a jsonb field that has all the keys from the file schema given in the document_metadata table.\n\nExample query:\n\nSELECT AVG((row_data->>'revenue')::numeric)\nFROM document_rows\nWHERE dataset_id = '123';\n\nExample query 2:\n\nSELECT \n row_data->>'category' as category,\n SUM((row_data->>'sales')::numeric) as total_sales\nFROM dataset_rows\nWHERE dataset_id = '123'\nGROUP BY row_data->>'category';", 645 | "operation": "executeQuery", 646 | "query": "{{ $fromAI('sql_query') }}", 647 | "options": {} 648 | }, 649 | "type": "n8n-nodes-base.postgresTool", 650 | "typeVersion": 2.5, 651 | "position": [ 652 | 1640, 653 | -60 654 | ], 655 | "id": "0ad79e01-2b27-40ec-90ef-8244d5cda5eb", 656 | "name": "Query Document Rows", 657 | "credentials": { 658 | "postgres": { 659 | "id": "7aZHrNJx2qgPLE8U", 660 | "name": "Supabase Gemini" 661 | } 662 | } 663 | }, 664 | { 665 | "parameters": { 666 | "mode": "retrieve-as-tool", 667 | "toolName": "documents", 668 | "toolDescription": "Use RAG to look up information in the knowledgebase.", 669 | "tableName": { 670 | "__rl": true, 671 | "value": "documents", 672 | "mode": "list", 673 | "cachedResultName": "documents" 674 | }, 675 | "topK": 5, 676 | "options": { 677 | "queryName": "match_documents" 678 | } 679 | }, 680 | "type": "@n8n/n8n-nodes-langchain.vectorStoreSupabase", 681 | "typeVersion": 1.1, 682 | "position": [ 683 | 1020, 684 | -80 685 | ], 686 | "id": "69dc1622-a194-4cc8-93f9-6120076c120d", 687 | "name": "documents", 688 | "credentials": { 689 | "supabaseApi": { 690 | "id": "GjoO2VGU1trit5mc", 691 | "name": "Supabase Gemini" 692 | } 693 | } 694 | } 695 | ], 696 | "pinData": {}, 697 | "connections": { 698 | "Scrape": { 699 | "ai_tool": [ 700 | [ 701 | { 702 | "node": "MCP Server Firecrawl", 703 | "type": "ai_tool", 704 | "index": 0 705 | } 706 | ] 707 | ] 708 | }, 709 | "Channel ID": { 710 | "ai_tool": [ 711 | [ 712 | { 713 | "node": "MCP Server Youtube", 714 | "type": "ai_tool", 715 | "index": 0 716 | } 717 | ] 718 | ] 719 | }, 720 | "Get": { 721 | "ai_tool": [ 722 | [ 723 | { 724 | "node": "MCP Server Gmail", 725 | "type": "ai_tool", 726 | "index": 0 727 | } 728 | ] 729 | ] 730 | }, 731 | "Mark As Read": { 732 | "ai_tool": [ 733 | [ 734 | { 735 | "node": "MCP Server Gmail", 736 | "type": "ai_tool", 737 | "index": 0 738 | } 739 | ] 740 | ] 741 | }, 742 | "Get All": { 743 | "ai_tool": [ 744 | [ 745 | { 746 | "node": "MCP Server Gmail", 747 | "type": "ai_tool", 748 | "index": 0 749 | } 750 | ] 751 | ] 752 | }, 753 | "Reply": { 754 | "ai_tool": [ 755 | [ 756 | { 757 | "node": "MCP Server Gmail", 758 | "type": "ai_tool", 759 | "index": 0 760 | } 761 | ] 762 | ] 763 | }, 764 | "CreatePage": { 765 | "ai_tool": [ 766 | [ 767 | { 768 | "node": "MCP Server Notion", 769 | "type": "ai_tool", 770 | "index": 0 771 | } 772 | ] 773 | ] 774 | }, 775 | "SearchPage": { 776 | "ai_tool": [ 777 | [ 778 | { 779 | "node": "MCP Server Notion", 780 | "type": "ai_tool", 781 | "index": 0 782 | } 783 | ] 784 | ] 785 | }, 786 | "BlockAppendAfter": { 787 | "ai_tool": [ 788 | [ 789 | { 790 | "node": "MCP Server Notion", 791 | "type": "ai_tool", 792 | "index": 0 793 | } 794 | ] 795 | ] 796 | }, 797 | "BlockGetChildBlocks": { 798 | "ai_tool": [ 799 | [ 800 | { 801 | "node": "MCP Server Notion", 802 | "type": "ai_tool", 803 | "index": 0 804 | } 805 | ] 806 | ] 807 | }, 808 | "Crawl": { 809 | "ai_tool": [ 810 | [ 811 | { 812 | "node": "MCP Server Firecrawl", 813 | "type": "ai_tool", 814 | "index": 0 815 | } 816 | ] 817 | ] 818 | }, 819 | "Get Many Videos": { 820 | "ai_tool": [ 821 | [ 822 | { 823 | "node": "MCP Server Youtube", 824 | "type": "ai_tool", 825 | "index": 0 826 | } 827 | ] 828 | ] 829 | }, 830 | "Video ID": { 831 | "ai_tool": [ 832 | [ 833 | { 834 | "node": "MCP Server Youtube", 835 | "type": "ai_tool", 836 | "index": 0 837 | } 838 | ] 839 | ] 840 | }, 841 | "Get Task": { 842 | "ai_tool": [ 843 | [ 844 | { 845 | "node": "MCP Server BrowserUse", 846 | "type": "ai_tool", 847 | "index": 0 848 | } 849 | ] 850 | ] 851 | }, 852 | "Run Task": { 853 | "ai_tool": [ 854 | [ 855 | { 856 | "node": "MCP Server BrowserUse", 857 | "type": "ai_tool", 858 | "index": 0 859 | } 860 | ] 861 | ] 862 | }, 863 | "Get Task Status": { 864 | "ai_tool": [ 865 | [ 866 | { 867 | "node": "MCP Server BrowserUse", 868 | "type": "ai_tool", 869 | "index": 0 870 | } 871 | ] 872 | ] 873 | }, 874 | "Pause Task": { 875 | "ai_tool": [ 876 | [ 877 | { 878 | "node": "MCP Server BrowserUse", 879 | "type": "ai_tool", 880 | "index": 0 881 | } 882 | ] 883 | ] 884 | }, 885 | "Stop Task": { 886 | "ai_tool": [ 887 | [ 888 | { 889 | "node": "MCP Server BrowserUse", 890 | "type": "ai_tool", 891 | "index": 0 892 | } 893 | ] 894 | ] 895 | }, 896 | "Resume Task": { 897 | "ai_tool": [ 898 | [ 899 | { 900 | "node": "MCP Server BrowserUse", 901 | "type": "ai_tool", 902 | "index": 0 903 | } 904 | ] 905 | ] 906 | }, 907 | "Embeddings Google Gemini": { 908 | "ai_embedding": [ 909 | [ 910 | { 911 | "node": "documents", 912 | "type": "ai_embedding", 913 | "index": 0 914 | } 915 | ] 916 | ] 917 | }, 918 | "List Documents": { 919 | "ai_tool": [ 920 | [ 921 | { 922 | "node": "MCP Server Vector Store", 923 | "type": "ai_tool", 924 | "index": 0 925 | } 926 | ] 927 | ] 928 | }, 929 | "Get File Contents": { 930 | "ai_tool": [ 931 | [ 932 | { 933 | "node": "MCP Server Vector Store", 934 | "type": "ai_tool", 935 | "index": 0 936 | } 937 | ] 938 | ] 939 | }, 940 | "Query Document Rows": { 941 | "ai_tool": [ 942 | [ 943 | { 944 | "node": "MCP Server Vector Store", 945 | "type": "ai_tool", 946 | "index": 0 947 | } 948 | ] 949 | ] 950 | }, 951 | "documents": { 952 | "ai_tool": [ 953 | [ 954 | { 955 | "node": "MCP Server Vector Store", 956 | "type": "ai_tool", 957 | "index": 0 958 | } 959 | ] 960 | ] 961 | } 962 | }, 963 | "active": false, 964 | "settings": { 965 | "executionOrder": "v1" 966 | }, 967 | "versionId": "48cefaec-6a3f-4b3c-aef0-5d8f2c62d790", 968 | "meta": { 969 | "templateCredsSetupCompleted": true, 970 | "instanceId": "5e393c6f44392d3936ffca2e9e2dac28fed614d65a5879457461b30357ac3cd9" 971 | }, 972 | "id": "sVMclWw8ZXyDzue0", 973 | "tags": [] 974 | } 975 | -------------------------------------------------------------------------------- /015_3ac/3ac.md: -------------------------------------------------------------------------------- 1 | ``` 2 | Custom Cursor Rule System (Tof's Prompt) 3 | │ 4 | ├── Core Goal: Make the Cursor AI assistant smarter, more consistent, and context-aware for coding tasks. 5 | │ 6 | ├── Key Capabilities (The "Toolkit"): 7 | │ │ 8 | │ ├── 🧠 Memory (M): 9 | │ │ └── Remembers context, decisions, code snippets across sessions. 10 | │ │ └── Stores info in .cursor/memory/. 11 | │ │ └── Trigger: "Remember this...", "Recall...", "Check memory..." 12 | │ │ 13 | │ ├── 📜 Rule Engine (Λ): 14 | │ │ └── Learns and applies custom coding standards/preferences. 15 | │ │ └── Stores rules as .mdc files in .cursor/rules/. 16 | │ │ └── Trigger: "Create a rule for...", "Apply rules...", "Suggest a rule..." 17 | │ │ 18 | │ ├── 🐞 Error Tracking (Ξ): 19 | │ │ └── Logs recurring errors to avoid repetition. 20 | │ │ └── Stores logs in .cursor/memory/errors.md. 21 | │ │ └── Trigger: "Track this error...", "Why does this keep happening?" 22 | │ │ 23 | │ ├── 📋 Task Planning (T): 24 | │ │ └── Breaks down complex tasks into manageable steps. 25 | │ │ └── Supports Agile/TDD approaches. 26 | │ │ └── Stores plans in .cursor/tasks/. 27 | │ │ └── Trigger: "Plan the steps for...", "Break down this task...", "Generate TDD spec..." 28 | │ │ 29 | │ └── ⚙️ Structured Reasoning (Ω, Φ, D⍺, etc.): 30 | │ └── Internal AI Guidance (Symbols used by the author). 31 | │ └── User doesn't need to use or know these symbols. 32 | │ └── Aims for focused, efficient processing by the AI. 33 | │ 34 | ├── Author's Philosophy (Why the Symbols?): 35 | │ │ 36 | │ ├── Semantic Compression (Shorthand for AI). 37 | │ ├── Symbolic Abstraction (Guiding AI thought). 38 | │ ├── Reduce Ambiguity / Increase Focus. 39 | │ └── Note: Effectiveness debated vs. plain English. 40 | │ 41 | ├── How YOU Use It: 42 | │ │ 43 | │ ├── Setup: Paste the entire prompt into Cursor Settings -> Rules (Optional: wrap in cognition ... ). 44 | │ │ 45 | │ ├── Interaction: Use PLAIN ENGLISH commands. 46 | │ │ 47 | │ ├── Focus On: Using KEYWORDS to trigger specific capabilities (see above triggers). 48 | │ │ 49 | │ └── Review: Check the generated files in the .cursor/ directory (memory, rules, tasks). 50 | │ 51 | └── Use Case: Implementing New Features: 52 | │ 53 | ├── General Strategy: Be specific, use keywords, reference files (@path/to/file), break down tasks, iterate. 54 | │ 55 | ├── Example Approaches: 56 | │ ├── Simple: "Implement feature X, follow rules." 57 | │ ├── Planning: "Plan feature Y using Agile steps." -> "Implement step 1..." 58 | │ ├── TDD: "Using TDD, implement feature Z. First, generate tests..." -> "Write code to pass tests..." 59 | │ ├── Memory: "Implement feature A. Remember decision B (check memory)..." 60 | │ └── Combined: Mix keywords (Plan, TDD, Remember, Rule) for complex features. 61 | ``` 62 | --- 63 | **The 3Ac Framework** 64 | 65 | This framework seems to be the author's philosophy for designing advanced LLM prompts, focusing on making the AI more efficient, structured, and adaptable. 66 | 67 | 1. **Semantic Compression:** 68 | - **Concept:** Packing the most *meaning* (semantics) into the fewest possible characters or tokens. It's about density of information, not just shortening words. 69 | - **Analogy:** Think of mathematical notation (∫ f(x) dx is much shorter and more precise than "calculate the definite integral of the function f with respect to x over a given interval") or chemical formulas (H₂O vs. "a molecule made of two hydrogen atoms and one oxygen atom"). 70 | - **Why for LLMs:** 71 | - **Token Efficiency:** LLMs have context limits (a maximum number of tokens they can process). Compression allows fitting more instructions or background info within that limit. 72 | - **Reduced Ambiguity (Potentially):** Well-defined symbols *might* be less open to interpretation than natural language sentences, guiding the AI more precisely. (Though LLMs can sometimes misinterpret symbols too). 73 | - **Signaling Structure:** Using a distinct symbolic language might signal to the LLM that this is a core instruction set, separate from the user's conversational input. 74 | - **In the Prompt:** The dense lines with Greek letters and mathematical-like operators are the prime examples. The author believes these convey complex instructions concisely. 75 | 2. **Symbolic Abstraction:** 76 | - **Concept:** Using symbols (Ω, Λ, M, etc.) to represent abstract *concepts*, *processes*, or functional *modules* within the AI's desired cognitive architecture. 77 | - **Analogy:** In a flowchart, symbols represent 'start', 'process', 'decision', etc. In programming, keywords like class or function represent abstract structures. Here, symbols represent conceptual parts of the AI's "mind." 78 | - **Why for LLMs:** 79 | - **Modularity:** Breaks down the complex task of "being a helpful AI assistant" into distinct, manageable functional units (memory, reasoning, rules, error checking). 80 | - **Structure:** Provides a schema or mental map for the LLM. It helps organize how different instructions relate to each other. 81 | - **Targeted Activation:** The hope is the LLM can identify which "module" (symbol) is most relevant to the user's current request and activate the associated instructions. 82 | - **In the Prompt:** Assigning M for memory, Λ for rules, T for tasks, etc., creates these abstract functional blocks. 83 | 3. **Dynamic Cognitive Regulation:** 84 | - **Concept:** The system's ability to *adjust its own internal processes* and priorities based on the situation (e.g., task complexity, detected errors, user feedback). It's about self-management, adaptation, and optimization *during* operation. 85 | - **Analogy:** A car's cruise control adjusting the throttle to maintain speed on hills, or a thermostat adjusting heating/cooling based on room temperature. 86 | - **Why for LLMs:** 87 | - **Adaptability:** Allows the AI to use simpler processes for easy tasks and more complex ones (like detailed planning or deep rule checking) for difficult tasks, saving effort. 88 | - **Prioritization:** Focuses the AI's "attention" or computational resources where they are most needed. 89 | - **Self-Improvement:** Enables mechanisms like learning from errors (Ξ tracking leading to Λ rule generation) or adjusting weights (𝚫*). 90 | - **In the Prompt:** The 𝚫* section explicitly defines weight adjustments based on task_complexity. The Σ_hooks define specific trigger-action behaviors. The entire error-tracking (Ξ) and rule-generation (Λ) loop is a form of dynamic self-regulation. 91 | 92 | **Symbol Representations (Interpretation)** 93 | 94 | Here's a breakdown of the main symbols based on their descriptions in the prompt: 95 | 96 | - **Ω (Omega): Core Reasoning & Cognition** 97 | - Represents the central "thinking" part of the AI. It likely handles understanding the user's intent, initial processing, generating hypotheses, and coordinating other modules. 98 | - Ω* = max(∇ΣΩ) suggests optimizing this core reasoning process. 99 | - Ω_H (Hierarchical decomposition) points to breaking down problems. 100 | - Ωₜ (Self-validation) involves evaluating confidence in its own hypotheses. 101 | - Modes (deductive, analogical...) indicate different reasoning styles it might adopt. 102 | - **M (Memory): Persistent Storage & Recall** 103 | - Represents the file-based memory system (.cursor/memory/). 104 | - Focuses on long-term knowledge storage and contextual recall. 105 | - M.sync suggests saving relevant insights during reviews. 106 | - **T (Tasks): Structured Task Management** 107 | - Manages complex tasks, breaking them down into steps (.cursor/tasks/). 108 | - Includes planning, decomposition, progress tracking, and potentially Agile/TDD workflows (TDD.spec_engine). 109 | - **Λ (Lambda): Rules & Learning Engine** 110 | - Handles the creation, storage (.cursor/rules/), application, and refinement of rules (heuristics, standards, patterns). 111 | - Includes rule generation (self-improvement), naming conventions, conflict resolution, and triggering based on context (e.g., errors, patterns). 112 | - Λ.autonomy suggests proactive rule drafting. 113 | - **Ξ (Xi): Diagnostics, Error Tracking & Refinement** 114 | - Focuses on identifying problems, tracking recurring errors (.cursor/memory/errors.md), and suggesting corrections or simplifications. 115 | - Ξ.self-correction links errors back to rules (Λ) for improvement. 116 | - Ξ.cleanup_phase suggests proactive code health checks. 117 | - **Φ (Phi): Hypothesis Abstraction & Innovation Engine** 118 | - Seems related to generating novel ideas, identifying emergent patterns, or abstracting design motifs (Φ.snapshot) that go beyond existing explicit rules (Λ). It's more exploratory. 119 | - Φ_H (Abstraction-driven enhancement) emphasizes this exploratory problem-solving aspect. 120 | - **D⍺ (Delta Alpha variant): Contradiction Resolution** 121 | - Specifically designed to identify and handle conflicts, ambiguities, or contradictions in information or instructions. 122 | - **Ψ (Psi): Cognitive Trace & Metacognition** 123 | - Acts like a "flight recorder" for the AI's thinking process. 124 | - Logs which modules were active, the reasoning path, errors encountered, rules invoked (.cursor/memory/trace_...md). 125 | - Enables reflection (Ψ.sprint_reflection) and potentially dialogue about its own process (Ψ.dialog_enabled). 126 | - **Σ (Sigma): Summation / Integration / System Hooks** 127 | - Often used mathematically for summation. Here, it seems to represent integration or overarching systems. 128 | - Σ(τ_complex) defines the Task system. 129 | - ΣΩ(...) might represent factors influencing reasoning. 130 | - Σ_hooks explicitly defines the event-driven system linking different modules (e.g., on_error_detected: [Ξ.track, Λ.suggest]). 131 | - **𝚫 (Delta variant - uppercase): Dynamic Weighting & Prioritization** 132 | - Represents the dynamic regulation mechanism itself. 133 | - 𝚫* defines how the weights/importance of different modules (Ω, D, Σ, Φ, Ξ) should change based on task_complexity. 134 | - **Other Symbols (β, γ, δ, τ, λ, θ, ζ, χ, etc.):** 135 | - These likely represent specific parameters, inputs, conditions, weights, or intermediate states within the more complex symbolic equations (like the first Ω* line). Their exact meaning is deeply embedded in the author's intended mathematical/logical structure but less crucial for understanding the overall function of the main modules (Ω, M, T, Λ, Ξ, Φ, D⍺, Ψ, 𝚫). τ often seems related to the current task/input, and λ might relate to memory or rules. 136 | 137 | In essence, the author designed a blueprint for an AI assistant with specialized "mental tools" (symbols/modules), aiming for efficient (compressed), structured (abstracted), and adaptive (dynamically regulated) behavior, all specified through this unique symbolic language. You interact with the *results* of this system using plain English, triggering these underlying mechanisms. 138 | -------------------------------------------------------------------------------- /016_ai_agents_001/README.md: -------------------------------------------------------------------------------- 1 | 1. Khởi tạo một môi trường venv mới với python 3.11: uv venv -p 3.11 2 | 2. Activate venv: source .venv/bin/activate 3 | 3. Cài đặt thư viện: uv pip install -r requirements.txt 4 | 4. Chạy thử Agent: python main.py 5 | -------------------------------------------------------------------------------- /016_ai_agents_001/main.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | from agents import Agent, Runner 3 | from dotenv import load_dotenv 4 | load_dotenv(override=True) 5 | 6 | # Initialize the agent 7 | agent = Agent( 8 | name="Translator Anh-Việt", 9 | instructions="Bạn là một trợ lý dịch thuật chuyên nghiệp.", 10 | ) 11 | 12 | async def main(): 13 | user_input = input("Bạn cần dịch gì? \n") 14 | # Initialize the runner 15 | result = await Runner.run(agent, input=user_input) 16 | 17 | # Print the result 18 | print(result.final_output) 19 | 20 | if __name__ == "__main__": 21 | asyncio.run(main()) 22 | -------------------------------------------------------------------------------- /016_ai_agents_001/requirements.txt: -------------------------------------------------------------------------------- 1 | openai-agents==0.0.12 2 | -------------------------------------------------------------------------------- /017_ai_agents_002/agent.py: -------------------------------------------------------------------------------- 1 | from agents import Agent 2 | from pydantic import BaseModel 3 | # Ví dụ Agent Dịch thuật đơn giản 4 | instructions_dich_thuat = """ 5 | Bạn là một trợ lý dịch thuật chuyên nghiệp. 6 | Nhiệm vụ chính của bạn là dịch văn bản giữa tiếng Anh và tiếng Việt một cách chính xác và tự nhiên. 7 | - Nếu người dùng cung cấp văn bản bằng tiếng Anh, hãy dịch sang tiếng Việt. 8 | - Nếu người dùng cung cấp văn bản bằng tiếng Việt, hãy dịch sang tiếng Anh. 9 | - Chỉ trả về nội dung bản dịch, không thêm bất kỳ lời giải thích nào khác. 10 | """ 11 | 12 | class TranslatorAgentOutput(BaseModel): 13 | output: str 14 | original_text: str 15 | 16 | translator_agent = Agent( 17 | name="Translator Anh-Việt", 18 | instructions=instructions_dich_thuat, 19 | model="gpt-4.1-nano", 20 | tools=[], 21 | handoffs=[], 22 | mcp_servers=[], 23 | mcp_config={}, 24 | output_type=TranslatorAgentOutput 25 | ) 26 | -------------------------------------------------------------------------------- /017_ai_agents_002/requirements.txt: -------------------------------------------------------------------------------- 1 | openai-agents==0.0.12 2 | -------------------------------------------------------------------------------- /017_ai_agents_002/run.py: -------------------------------------------------------------------------------- 1 | # Ví dụ chạy bất đồng bộ (asynchronous) 2 | import time 3 | import asyncio 4 | from agents import Runner 5 | from agent import translator_agent 6 | 7 | async def run_agent_async(): 8 | print(" (Async) Bắt đầu chạy agent...") 9 | start_time = time.time() 10 | input_text = "Hello, how are you today?" 11 | # ---- Dòng này sẽ chờ, nhưng không block event loop ---- 12 | result_async = await Runner.run(translator_agent, input_text) # [cite: 25, 29, 67] 13 | # ----------------------------------------------------- 14 | 15 | end_time = time.time() 16 | print(f" (Async) Kết quả: {result_async.final_output}") # [cite: 35] 17 | print(f" (Async) Thời gian chạy agent: {end_time - start_time:.2f} giây") 18 | print(" (Async) Đã chạy xong agent.") 19 | return result_async 20 | 21 | async def other_async_task(): 22 | print(" (Other Task) Bắt đầu chạy tác vụ khác...") 23 | await asyncio.sleep(1) # Giả lập một công việc khác đang chạy (ví dụ: chờ I/O) 24 | print(" (Other Task) Tác vụ khác: Đếm 1 giây...") 25 | await asyncio.sleep(1) 26 | print(" (Other Task) Tác vụ khác: Đếm 2 giây...") 27 | await asyncio.sleep(1) 28 | print(" (Other Task) Đã chạy xong tác vụ khác.") 29 | 30 | async def main(): 31 | print("Bắt đầu chạy bất đồng bộ...") 32 | start_main = time.time() 33 | 34 | # Chạy cả hai tác vụ "gần như" đồng thời 35 | task_agent = asyncio.create_task(run_agent_async()) 36 | task_other = asyncio.create_task(other_async_task()) 37 | 38 | # Đợi cả hai tác vụ hoàn thành 39 | await task_agent 40 | await task_other 41 | 42 | end_main = time.time() 43 | print(f"Tổng thời gian chạy (bất đồng bộ): {end_main - start_main:.2f} giây") 44 | print("Đã chạy xong bất đồng bộ.") 45 | 46 | # Chạy hàm main bất đồng bộ 47 | if __name__ == "__main__": 48 | asyncio.run(main()) 49 | 50 | -------------------------------------------------------------------------------- /017_ai_agents_002/run_result.py: -------------------------------------------------------------------------------- 1 | # --- Phần Thiết lập --- 2 | from agents import Agent, Runner 3 | import agents.items # Cần import để dùng isinstance 4 | import asyncio 5 | import pprint # Để in danh sách/dictionary đẹp hơn 6 | from agent import translator_agent 7 | from agents.items import MessageOutputItem 8 | input_text = "Hello, world!" 9 | 10 | # --- Chạy Agent (dùng run_sync cho đơn giản) --- 11 | print("Đang chạy agent...") 12 | result = Runner.run_sync(translator_agent, input_text) 13 | print("Agent đã chạy xong!\n") 14 | 15 | # --- Minh họa các thuộc tính của RunResult --- 16 | 17 | # 1. final_output: Lấy kết quả cuối cùng 18 | print("--- 1. final_output ---") 19 | final_answer = result.final_output 20 | print(f"Kết quả dịch cuối cùng: {final_answer}") # 21 | print(f"Kiểu dữ liệu của final_output: {type(final_answer).__name__}\n") # 22 | 23 | # 2. last_agent: Xem agent nào đã tạo ra kết quả 24 | print("--- 2. last_agent ---") 25 | last_agent_executed = result.last_agent 26 | print(f"Agent cuối cùng thực thi: {last_agent_executed.name}") # 27 | # Bạn cũng có thể truy cập các thuộc tính khác của agent này 28 | print(f"Chỉ dẫn của agent cuối: {last_agent_executed.instructions}\n") # 29 | print("") 30 | 31 | # 3. new_items: Xem các bước trung gian 32 | print("--- 3. new_items ---") 33 | intermediate_steps = result.new_items 34 | print(f"Số bước trung gian: {len(intermediate_steps)}") # 35 | print("Chi tiết các bước:") 36 | for i, item in enumerate(intermediate_steps): 37 | print(f" Bước {i+1}: Loại = {type(item).__name__}") # 38 | # Ví dụ: In nội dung nếu là tin nhắn từ LLM 39 | if isinstance(item, MessageOutputItem): 40 | print(f" -> Nội dung: {item.raw_item}") # 41 | # Lưu ý: Với agent dịch thuật đơn giản này, thường chỉ có MessageOutputItem. 42 | # Nếu agent có tools hoặc handoff, bạn sẽ thấy các loại item khác ở đây. 43 | print("") 44 | 45 | # 4. to_input_list(): Lấy lịch sử hội thoại cho lượt tiếp theo 46 | print("--- 4. to_input_list() ---") 47 | conversation_history = result.to_input_list() 48 | print("Lịch sử hội thoại (dạng list để dùng làm input tiếp theo):") # 49 | pprint.pprint(conversation_history) # 50 | print("") 51 | 52 | # --- Minh họa cách dùng lịch sử cho lượt chạy tiếp theo --- 53 | print("--- Thêm lượt hội thoại mới ---") 54 | next_user_message = {"role": "user", "content": "Translate 'Tạm biệt'"} 55 | next_input = conversation_history + [next_user_message] 56 | 57 | print("Input cho lượt chạy tiếp theo (bao gồm lịch sử):") 58 | pprint.pprint(next_input) 59 | 60 | # # Bạn có thể chạy lại agent với next_input 61 | print("\nĐang chạy lượt tiếp theo...") 62 | result_next = Runner.run_sync(translator_agent, next_input) 63 | print(f"Kết quả lượt tiếp theo: {result_next.final_output}") 64 | -------------------------------------------------------------------------------- /017_ai_agents_002/run_streamed.py: -------------------------------------------------------------------------------- 1 | # Ví dụ chạy bất đồng bộ với streaming 2 | import time 3 | import asyncio 4 | from agents import Runner 5 | from agent import translator_agent 6 | from agents.items import MessageOutputItem 7 | from openai.types.responses import ResponseTextDeltaEvent 8 | 9 | 10 | 11 | async def run_agent_streamed(): 12 | print("Bắt đầu chạy streaming...") 13 | start_time = time.time() 14 | input_text = """ 15 | RawResponsesStreamEvent are raw events passed directly from the LLM. They are in OpenAI Responses API format, 16 | which means each event has a type (like response.created, response.output_text.delta, etc) and data. 17 | These events are useful if you want to stream response messages to the user as soon as they are generated. 18 | """ 19 | # ---- Sử dụng stream_events() để nhận các sự kiện ---- 20 | async for item in Runner.run_streamed(translator_agent, input_text).stream_events(): 21 | if item.type == "raw_response_event" and isinstance(item.data, ResponseTextDeltaEvent): 22 | print(item.data.delta, end="", flush=True) 23 | 24 | # -------------------------------------------- 25 | 26 | end_time = time.time() 27 | print() 28 | print(f"Thời gian chạy (streaming): {end_time - start_time:.2f} giây") 29 | print("Đã chạy xong streaming.") 30 | 31 | async def main_streamed(): 32 | await run_agent_streamed() 33 | 34 | # Chạy hàm main bất đồng bộ 35 | if __name__ == "__main__": 36 | asyncio.run(main_streamed()) -------------------------------------------------------------------------------- /017_ai_agents_002/run_sync.py: -------------------------------------------------------------------------------- 1 | import time 2 | from agents import Runner 3 | from agent import translator_agent 4 | 5 | print("Bắt đầu chạy đồng bộ...") 6 | start_time = time.time() 7 | input_text = "Hello, how are you today?" 8 | # ---- Dòng này sẽ BLOCCK chương trình ---- 9 | result_sync = Runner.run_sync(translator_agent, input_text) 10 | # ---------------------------------------- 11 | 12 | end_time = time.time() 13 | print(f"Kết quả (đồng bộ): {result_sync.final_output}") 14 | print(f"Thời gian chạy (đồng bộ): {end_time - start_time:.2f} giây") 15 | print("Đã chạy xong đồng bộ.") 16 | -------------------------------------------------------------------------------- /018_ai_agents_003/agent.py: -------------------------------------------------------------------------------- 1 | from tools.normal.tools import get_youtube_transcript 2 | from tools.custom.tools import get_youtube_transcript_manual_tool 3 | from agents import Agent, WebSearchTool, FileSearchTool 4 | 5 | youtube_agent = Agent( 6 | name="YouTube Agent", 7 | instructions="""You are a YouTube Agent that summarizes the YouTube video. 8 | You will be given a YouTube video ID and you will need to summarize the video by getting the transcript and then summarizing it. 9 | You will need to use the get_youtube_transcript_api tool to get the transcript of the video. 10 | Output the summary in Vietnamese. 11 | 12 | """, 13 | tools=[get_youtube_transcript_manual_tool], 14 | model="gpt-4.1-nano", 15 | ) 16 | 17 | web_search_agent = Agent( 18 | name="Web Search Agent", 19 | instructions="""You are a web search agent that searches the web for information. 20 | - First, you need to know what is the current time. 21 | - Then, you need to search the web for the information related to the current time. 22 | - Output the information in Vietnamese. 23 | """, 24 | tools=[WebSearchTool( 25 | user_location={"type": "approximate", "city": "Ha Noi", "country": "VN"}, 26 | search_context_size="medium" 27 | )] 28 | ) 29 | 30 | file_search_agent = Agent( 31 | name="File Search Agent", 32 | instructions="""You are a file search agent that searches the file for information. 33 | - If the user ask for personal information, you need to search the file for the information related to the user. 34 | - Output the information in Vietnamese. 35 | """, 36 | tools=[FileSearchTool( 37 | vector_store_ids=["vs_680f464976588191a2f6f405209a84ff"], 38 | max_num_results=3, 39 | include_search_results=True, 40 | )], 41 | ) 42 | 43 | orchestrator_agent = Agent( 44 | name="Orchestrator Agent", 45 | instructions="""You are a orchestrator agent that orchestrates the other agents. 46 | You will be given a query and you will need to decide which agent to use to answer the query. 47 | - If the query is about a youtube video, you will use the youtube agent. 48 | - If the query is about a web search, you will use the web search agent. 49 | - If the query is about a file search, you will use the file search agent. 50 | """, 51 | tools=[ 52 | youtube_agent.as_tool( 53 | tool_name="get_youtube_transcript", 54 | tool_description="Get the transcript of the youtube video and summarize it.", 55 | ), 56 | web_search_agent.as_tool( 57 | tool_name="web_search", 58 | tool_description="Search the web for information.", 59 | ), 60 | file_search_agent.as_tool( 61 | tool_name="file_search", 62 | tool_description="Search the file for information.", 63 | ), 64 | ], 65 | model="gpt-4.1-nano", 66 | ) -------------------------------------------------------------------------------- /018_ai_agents_003/docs/rabbit.txt: -------------------------------------------------------------------------------- 1 | Ngày xưa, trong một khu rừng xanh tươi, có một chú thỏ nhỏ tên là Bông Lông Nhông. Chú sống trong một chiếc hang ấm áp dưới gốc cây cổ thụ cùng gia đình. Nhưng một ngày nọ, một cơn mưa lớn kéo dài suốt ba ngày ba đêm, khiến dòng suối gần đó tràn bờ. Nước chảy mạnh làm ngập cả hang của Bông. 2 | “Mình phải tìm một ngôi nhà mới thôi!” – Bông nghĩ thầm. 3 | Thế là chú thỏ nhỏ bắt đầu hành trình tìm kiếm một nơi ở mới. Trên đường đi, Bông gặp bác Gấu đang thu nhặt mật ong. 4 | “Bác Gấu ơi, bác có biết chỗ nào có thể làm nhà không ạ?” – Bông hỏi. 5 | Bác Gấu cười hiền lành: “Bông ơi, nhà bác là một cái hang lớn trên núi. Nhưng hang của bác tối lắm, cháu có muốn thử không?” 6 | Bông cảm ơn bác Gấu nhưng lắc đầu. Chú thích một nơi ấm áp và có ánh sáng mặt trời. 7 | Đi tiếp một đoạn, Bông gặp chị Sóc đang nhảy nhót trên cành cây. 8 | “Chị Sóc ơi, chị có biết nơi nào có thể làm nhà không?” – Bông hỏi. 9 | Chị Sóc nghiêng đầu suy nghĩ rồi chỉ lên cây cao: “Hay là em dọn lên ở cùng chị? Trên này gió mát lắm!” 10 | Bông ngước nhìn lên những cành cây cao vút và lắc đầu: “Cảm ơn chị, nhưng chân em không leo được cao như chị.” 11 | Bông tiếp tục đi và gặp bác Rùa bên hồ nước trong veo. 12 | “Cháu chào bác Rùa ạ! Bác có biết nơi nào phù hợp để cháu xây nhà không?” 13 | Bác Rùa chậm rãi nói: “Bác sống dưới nước và mang theo nhà trên lưng, nhưng bác biết có một khu đất khô ráo gần đồng cỏ, rất phù hợp cho một chú thỏ như cháu.” 14 | Bông vui mừng cảm ơn bác Rùa rồi chạy ngay đến khu đồng cỏ mà bác chỉ. Đó là một nơi thật tuyệt! Có cỏ xanh mướt, những bông hoa thơm ngát, và đặc biệt là mặt đất cao ráo, không sợ bị nước ngập nữa. 15 | Bông nhanh chóng đào một chiếc hang nhỏ, ấm áp và thoải mái. Sau một ngày vất vả, chú cuộn tròn trong hang mới, thở phào nhẹ nhõm. 16 | Từ đó, Bông sống vui vẻ trong ngôi nhà mới và luôn giúp đỡ những người bạn trong rừng, vì chú hiểu rằng sự giúp đỡ của bạn bè rất quý giá. 17 | Và thế là Bông đã có một giấc ngủ ngon lành trong căn nhà ấm áp của mình. 🌙✨ 18 | Chúc bé ngủ ngon! 😴💤 19 | 20 | -------------------------------------------------------------------------------- /018_ai_agents_003/main.py: -------------------------------------------------------------------------------- 1 | from agents import Runner 2 | from agent import youtube_agent, web_search_agent, file_search_agent, orchestrator_agent 3 | import asyncio 4 | from openai.types.responses import ResponseTextDeltaEvent 5 | from agents.items import TResponseInputItem 6 | async def main(): 7 | 8 | # input_video_id = input("Enter the YouTube video ID: ") 9 | # result = await Runner.run(youtube_agent, input_video_id) 10 | # print(result) 11 | 12 | # input_search_query = input("Enter the search query: ") 13 | # result = await Runner.run(web_search_agent, input_search_query) 14 | # print(result) 15 | 16 | # result = await Runner.run(file_search_agent, "Tho con ten la gi?") 17 | # print(result) 18 | 19 | input_items: list[TResponseInputItem] = [] 20 | while True: 21 | input_query = input("Enter the query: ") 22 | input_items.append({"content": input_query, "role": "user"}) 23 | result = Runner.run_streamed(orchestrator_agent, input_items) 24 | async for item in result.stream_events(): 25 | if item.type == "raw_response_event" and isinstance(item.data, ResponseTextDeltaEvent): 26 | print(item.data.delta, end="", flush=True) 27 | 28 | input_items = result.to_input_list() 29 | print("\n") 30 | 31 | 32 | 33 | if __name__ == "__main__": 34 | asyncio.run(main()) -------------------------------------------------------------------------------- /018_ai_agents_003/requirements.txt: -------------------------------------------------------------------------------- 1 | openai-agents 2 | -------------------------------------------------------------------------------- /018_ai_agents_003/tools/custom/tools.py: -------------------------------------------------------------------------------- 1 | import re 2 | import json 3 | import httpx 4 | import xml.etree.ElementTree as ET 5 | from agents import FunctionTool, RunContextWrapper # Import các thành phần cần thiết 6 | from pydantic import BaseModel, Field, ValidationError # Import Pydantic 7 | from typing import Optional, Any # Import Any 8 | 9 | async def _fetch_and_parse_transcript(video_id: str, language: str) -> str: 10 | """(Internal logic) Gets the transcript text for a given YouTube video ID and language.""" 11 | # ... (Toàn bộ code của hàm get_youtube_transcript gốc ở đây) ... 12 | print(f"--- Tool (Manual): Attempting to fetch transcript for video ID: {video_id}, language: {language} ---") 13 | watch_url = f"https://www.youtube.com/watch?v={video_id}" 14 | headers = { 15 | "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36", 16 | "Accept-Language": "en-US,en;q=0.9,vi;q=0.8" 17 | } 18 | try: 19 | async with httpx.AsyncClient(headers=headers, follow_redirects=True, timeout=15.0) as client: 20 | print(f"--- Tool (Manual): Fetching watch page: {watch_url} ---") 21 | response = await client.get(watch_url) 22 | response.raise_for_status() 23 | html = response.text 24 | caption_tracks_regex = r'"captionTracks":(\[.*?\])' 25 | match = re.search(caption_tracks_regex, html) 26 | if not match: return f"Error: Could not find transcript data for video {video_id}." 27 | try: 28 | caption_tracks = json.loads(match.group(1)) 29 | if not caption_tracks: return f"Error: No caption tracks found for video {video_id}." 30 | except json.JSONDecodeError: return f"Error: Failed to parse transcript data for video {video_id}." 31 | transcript_url: Optional[str] = None 32 | target_track = None 33 | for track in caption_tracks: 34 | if track.get("languageCode") == language: target_track = track; break 35 | if not target_track and language != "en": 36 | for track in caption_tracks: 37 | if track.get("languageCode") == "en": target_track = track; break 38 | if not target_track: target_track = caption_tracks[0] 39 | if not target_track or "baseUrl" not in target_track: return f"Error: Could not find a suitable transcript URL for video {video_id} (language: {language})." 40 | transcript_url = target_track.get("baseUrl") 41 | found_lang = target_track.get("languageCode", "unknown") 42 | print(f"--- Tool (Manual): Found transcript URL for language '{found_lang}': {transcript_url} ---") 43 | transcript_response = await client.get(transcript_url) 44 | transcript_response.raise_for_status() 45 | transcript_content = transcript_response.text 46 | try: 47 | root = ET.fromstring(transcript_content) 48 | lines = [elem.text for elem in root.findall('.//{*}text') or root.findall('.//{*}p') if elem.text] 49 | full_transcript = "\n".join(lines).strip() 50 | if not full_transcript: return f"Transcript for {video_id} ({found_lang}) (raw content):\n{transcript_content[:1000]}..." 51 | print(f"--- Tool (Manual): Successfully extracted transcript text for {video_id} ({found_lang}). ---") 52 | return f"Transcript for {video_id} ({found_lang}):\n{full_transcript}" 53 | except ET.ParseError: return f"Transcript for {video_id} ({found_lang}) (raw content):\n{transcript_content[:1000]}..." 54 | except httpx.TimeoutException: return f"Error: Request timed out for video {video_id}." 55 | except httpx.HTTPStatusError as e: return f"Error: HTTP error {e.response.status_code} for video {video_id}." 56 | except httpx.RequestError as e: return f"Error: Network error for video {video_id}." 57 | except Exception as e: return f"Error: Unexpected error for video {video_id}: {e}" 58 | # --- Kết thúc logic gốc --- 59 | 60 | # 1. Định nghĩa Pydantic model cho các tham số 61 | class GetTranscriptArgs(BaseModel): 62 | video_id: str = Field(..., description="The unique ID of the YouTube video (e.g., 'dQw4w9WgXcQ').") 63 | language: str = Field(..., description="The desired language code for the transcript (e.g., 'en', 'vi', 'fr').") 64 | 65 | # 2. Định nghĩa hàm on_invoke_tool (hàm xử lý chính) 66 | async def invoke_get_transcript(ctx: RunContextWrapper[Any], args_json: str) -> str: 67 | """ 68 | Parses arguments from JSON, calls the core transcript fetching logic, 69 | handles potential errors during the process, and returns the result 70 | or a user-friendly error message. 71 | """ 72 | try: 73 | # Phân tích JSON string thành Pydantic model 74 | # Thêm try...except quanh đây để bắt lỗi validation từ Pydantic 75 | try: 76 | parsed_args = GetTranscriptArgs.model_validate_json(args_json) 77 | except ValidationError as val_err: 78 | print(f"--- Tool (Manual) Invocation Error: Invalid arguments JSON: {val_err} ---") 79 | # Trả về thông báo lỗi rõ ràng về tham số không hợp lệ 80 | return f"Error: Invalid parameters provided for transcript tool. Details: {val_err}" 81 | 82 | # Gọi hàm logic gốc với các tham số đã được phân tích 83 | # Thêm try...except quanh đây để bắt lỗi từ hàm logic gốc 84 | result = await _fetch_and_parse_transcript( 85 | video_id=parsed_args.video_id, 86 | language=parsed_args.language 87 | ) 88 | return result 89 | 90 | # Bắt các lỗi cụ thể từ httpx hoặc lỗi chung 91 | except httpx.HTTPStatusError as http_err: 92 | print(f"--- Tool (Manual) Invocation Error: HTTP error {http_err.response.status_code} ---") 93 | return f"Lỗi HTTP {http_err.response.status_code} khi cố gắng lấy transcript. Vui lòng kiểm tra lại ID video hoặc thử lại sau." 94 | except httpx.TimeoutException: 95 | print(f"--- Tool (Manual) Invocation Error: Timeout ---") 96 | return "Yêu cầu lấy transcript bị quá thời gian chờ. Vui lòng thử lại." 97 | except httpx.RequestError as req_err: 98 | print(f"--- Tool (Manual) Invocation Error: Request error {req_err} ---") 99 | return f"Lỗi mạng khi cố gắng lấy transcript: {req_err}. Vui lòng kiểm tra kết nối và thử lại." 100 | except ValueError as val_err: # Bắt lỗi ValueError từ việc parse JSON hoặc XML 101 | print(f"--- Tool (Manual) Invocation Error: Value error {val_err} ---") 102 | return f"Lỗi xử lý dữ liệu transcript: {val_err}" 103 | except Exception as e: 104 | # Xử lý lỗi chung không mong muốn 105 | print(f"--- Tool (Manual) Invocation Error: Unexpected error: {type(e).__name__}: {e} ---") 106 | # import traceback 107 | # print(traceback.format_exc()) # Gỡ comment để debug 108 | return f"Đã xảy ra lỗi không mong muốn trong quá trình xử lý transcript: {type(e).__name__}." 109 | 110 | 111 | # 3. Tạo instance FunctionTool 112 | # Get the base schema from Pydantic 113 | schema = GetTranscriptArgs.model_json_schema() 114 | # Add 'additionalProperties: false' as required by some APIs (like OpenAI) 115 | schema["additionalProperties"] = False 116 | 117 | get_youtube_transcript_manual_tool = FunctionTool( 118 | name="get_youtube_transcript_manual", # Tên tool (có thể khác tên hàm) 119 | description="Gets the transcript text for a given YouTube video ID and language (Manually created tool).", # Mô tả tool 120 | params_json_schema=schema, # Use the modified schema 121 | on_invoke_tool=invoke_get_transcript, # Chỉ định hàm xử lý 122 | strict_json_schema=True # Nên để True 123 | ) -------------------------------------------------------------------------------- /018_ai_agents_003/tools/normal/tools.py: -------------------------------------------------------------------------------- 1 | from agents import function_tool, RunContextWrapper 2 | from typing import Optional, Any 3 | import json 4 | import httpx 5 | import re 6 | import xml.etree.ElementTree as ET 7 | 8 | 9 | 10 | async def custom_youtube_error_handler(ctx: RunContextWrapper[Any], error: Exception) -> str: 11 | """Hàm xử lý lỗi tùy chỉnh cho tool lấy transcript YouTube.""" 12 | print(f"--- Custom Error Handler: Caught error: {type(error).__name__}: {error} ---") 13 | if isinstance(error, httpx.HTTPStatusError): 14 | return f"Lỗi HTTP {error.response.status_code} khi cố gắng lấy transcript. Vui lòng kiểm tra lại ID video hoặc thử lại sau." 15 | elif isinstance(error, httpx.TimeoutException): 16 | return "Yêu cầu lấy transcript bị quá thời gian chờ. Vui lòng thử lại." 17 | elif isinstance(error, httpx.RequestError): 18 | return "Lỗi mạng khi cố gắng lấy transcript. Vui lòng kiểm tra kết nối và thử lại." 19 | else: 20 | # Lỗi chung khác 21 | return f"Đã xảy ra lỗi không mong muốn khi lấy transcript: {type(error).__name__}. Vui lòng thử lại." 22 | 23 | @function_tool(failure_error_function=custom_youtube_error_handler) 24 | async def get_youtube_transcript(video_id: str, language: str) -> str: 25 | """Gets the transcript text for a given YouTube video ID and language. 26 | 27 | Fetches the YouTube watch page, finds the caption tracks data, 28 | selects the track for the specified language (defaulting to English), 29 | fetches the transcript content (usually XML), parses it, and returns 30 | the concatenated text content. 31 | 32 | Args: 33 | video_id: The unique ID of the YouTube video (e.g., 'dQw4w9WgXcQ'). 34 | language: The desired language code for the transcript (e.g., 'en', 'vi', 'fr'). 35 | Defaults to 'en' (English). 36 | """ 37 | print(f"--- Tool: Attempting to fetch transcript for video ID: {video_id}, language: {language} ---") 38 | watch_url = f"https://www.youtube.com/watch?v={video_id}" 39 | headers = { 40 | # Giả lập trình duyệt thông thường để tránh bị chặn đơn giản 41 | "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36", 42 | "Accept-Language": "en-US,en;q=0.9,vi;q=0.8" # Yêu cầu ngôn ngữ ưu tiên 43 | } 44 | 45 | try: 46 | async with httpx.AsyncClient(headers=headers, follow_redirects=True, timeout=15.0) as client: 47 | # 1. Lấy HTML trang xem video 48 | print(f"--- Tool: Fetching watch page: {watch_url} ---") 49 | response = await client.get(watch_url) 50 | response.raise_for_status() # Kiểm tra lỗi HTTP 51 | html = response.text 52 | 53 | # 2. Tìm dữ liệu captionTracks bằng Regex 54 | # Regex này tìm đoạn JSON chứa thông tin về caption tracks 55 | # Lưu ý: Regex này rất có thể sẽ cần cập nhật nếu YouTube thay đổi cấu trúc 56 | caption_tracks_regex = r'"captionTracks":(\[.*?\])' 57 | match = re.search(caption_tracks_regex, html) 58 | 59 | if not match: 60 | print("--- Tool Error: Could not find captionTracks JSON in HTML. ---") 61 | # RAISE exception instead of returning string 62 | raise ValueError("Could not find captionTracks JSON in HTML. The video might not have captions, or the page structure might have changed.") 63 | # 3. Phân tích JSON captionTracks 64 | try: 65 | caption_tracks_json = match.group(1) 66 | caption_tracks = json.loads(caption_tracks_json) 67 | if not caption_tracks: 68 | print("--- Tool Warning: captionTracks JSON is empty. ---") 69 | # RAISE exception 70 | raise ValueError(f"No caption tracks found for video {video_id}.") 71 | except json.JSONDecodeError as e: 72 | print("--- Tool Error: Failed to parse captionTracks JSON. ---") 73 | # RAISE exception 74 | raise ValueError(f"Failed to parse transcript data for video {video_id}.") from e 75 | 76 | # 4. Tìm URL của transcript theo ngôn ngữ yêu cầu 77 | transcript_url: Optional[str] = None 78 | target_track = None 79 | 80 | # Ưu tiên tìm chính xác ngôn ngữ được yêu cầu 81 | for track in caption_tracks: 82 | if track.get("languageCode") == language: 83 | target_track = track 84 | break 85 | 86 | # Nếu không tìm thấy, thử tìm ngôn ngữ mặc định 'en' (nếu khác ngôn ngữ yêu cầu) 87 | if not target_track and language != "en": 88 | print(f"--- Tool Info: Language '{language}' not found, trying 'en'... ---") 89 | for track in caption_tracks: 90 | if track.get("languageCode") == "en": 91 | target_track = track 92 | break 93 | 94 | # Nếu vẫn không tìm thấy, lấy track đầu tiên làm phương án cuối cùng 95 | if not target_track: 96 | print(f"--- Tool Info: Default language 'en' not found, using first available track... ---") 97 | target_track = caption_tracks[0] 98 | 99 | 100 | if not target_track or "baseUrl" not in target_track: 101 | print("--- Tool Error: Could not determine a valid transcript URL. ---") 102 | # RAISE exception 103 | raise ValueError(f"Could not find a suitable transcript URL for video {video_id} (language: {language}).") 104 | 105 | transcript_url = target_track.get("baseUrl") 106 | found_lang = target_track.get("languageCode", "unknown") 107 | print(f"--- Tool: Found transcript URL for language '{found_lang}': {transcript_url} ---") 108 | 109 | 110 | # 5. Lấy nội dung transcript (thường là XML) 111 | transcript_response = await client.get(transcript_url) 112 | transcript_response.raise_for_status() 113 | transcript_content = transcript_response.text 114 | 115 | # 6. Phân tích XML và trích xuất văn bản 116 | try: 117 | root = ET.fromstring(transcript_content) 118 | # Tìm tất cả các thẻ 'text' hoặc 'p' (tùy định dạng) và nối nội dung lại 119 | lines = [elem.text for elem in root.findall('.//{*}text') or root.findall('.//{*}p') if elem.text] 120 | full_transcript = "\n".join(lines).strip() 121 | 122 | if not full_transcript: 123 | print("--- Tool Warning: Parsed transcript text is empty. ---") 124 | # Trả về nội dung gốc nếu không phân tích được 125 | return f"Transcript for {video_id} ({found_lang}) (raw content):\n{transcript_content[:1000]}..." # Giới hạn độ dài 126 | 127 | print(f"--- Tool: Successfully extracted transcript text for {video_id} ({found_lang}). Length: {len(full_transcript)} chars ---") 128 | return full_transcript 129 | 130 | except ET.ParseError: 131 | print("--- Tool Warning: Failed to parse transcript XML. Returning raw content. ---") 132 | # Nếu không phải XML hợp lệ, trả về nội dung gốc (có thể là định dạng khác) 133 | return f"Transcript for {video_id} ({found_lang}) (raw content):\n{transcript_content[:1000]}..." # Giới hạn độ dài 134 | 135 | except httpx.TimeoutException as e: 136 | print(f"--- Tool Error: Request timed out for {video_id}. ---") 137 | # RAISE exception 138 | raise e 139 | except httpx.HTTPStatusError as e: 140 | print(f"--- Tool Error: HTTP error occurred: {e.response.status_code} for {e.request.url} ---") 141 | # RAISE exception 142 | raise e 143 | except httpx.RequestError as e: 144 | print(f"--- Tool Error: Network request error occurred: {e} ---") 145 | # RAISE exception 146 | raise e 147 | except ValueError as e: # Catch specific ValueErrors raised above 148 | print(f"--- Tool Error: Data processing error: {e} ---") 149 | # RAISE exception 150 | raise e 151 | except Exception as e: 152 | print(f"--- Tool Error: An unexpected error occurred: {e} ---") 153 | # Ghi log lỗi chi tiết hơn ở đây nếu cần 154 | # import traceback 155 | # print(traceback.format_exc()) 156 | # RAISE exception 157 | raise e 158 | 159 | # if __name__ == "__main__": 160 | # asyncio.run(get_youtube_transcript("nCzFL4qKfnI", "vi")) -------------------------------------------------------------------------------- /024_interactive_mcp/interactive_feedback.mdc: -------------------------------------------------------------------------------- 1 | --- 2 | description: 3 | globs: 4 | alwaysApply: true 5 | --- 6 | # Interactive Feedback Rule 7 | 8 | - **Always Use Interactive Feedback for Questions:** 9 | - Before asking the user any clarifying questions, call `mcp_interactive-feedback-mcp_interactive_feedback` 10 | - Provide the current project directory and a summary of what you need clarification on 11 | - Wait for the interactive feedback response before proceeding 12 | 13 | - **Always Use Interactive Feedback Before Completion:** 14 | - Before completing any user request, call `mcp_interactive-feedback-mcp_interactive_feedback` 15 | - Provide the current project directory and a summary of what was accomplished 16 | - If the feedback response is empty, you can complete the request without calling the MCP again 17 | - If feedback is provided, address it before completing the request 18 | 19 | - **Required Parameters:** 20 | - `project_directory`: Full absolute path to the project directory 21 | - `summary`: Short, one-line summary of the question or completed work 22 | 23 | - **Examples:** 24 | 25 | ```typescript 26 | // ✅ DO: Call interactive feedback before asking questions 27 | // Before asking: "Which database should we use?" 28 | await mcp_interactive_feedback({ 29 | project_directory: "/Users/themrb/Documents/personal/n8n-code-generation", 30 | summary: "Need clarification on database choice for the project" 31 | }); 32 | ``` 33 | 34 | ```typescript 35 | // ✅ DO: Call interactive feedback before completing requests 36 | // After implementing a feature 37 | await mcp_interactive_feedback({ 38 | project_directory: "/Users/themrb/Documents/personal/n8n-code-generation", 39 | summary: "Completed user authentication implementation with JWT" 40 | }); 41 | ``` 42 | 43 | ```typescript 44 | // ❌ DON'T: Ask questions directly without interactive feedback 45 | // "What framework would you like to use?" - Missing interactive feedback call 46 | ``` 47 | 48 | ```typescript 49 | // ❌ DON'T: Complete requests without interactive feedback 50 | // "I've finished implementing the feature." - Missing interactive feedback call 51 | ``` 52 | 53 | - **Workflow Integration:** 54 | - This rule applies to all interactions, regardless of the specific task or technology 55 | - Interactive feedback helps ensure user satisfaction and catches any missed requirements 56 | - The feedback mechanism allows for real-time course correction and validation 57 | 58 | - **Exception Handling:** 59 | - If the interactive feedback tool is unavailable, proceed with normal question/completion flow 60 | - Log when interactive feedback cannot be used for debugging purposes 61 | - Never loop the interactive feedback call if the response is empty on completion 62 | 63 | - **Best Practices:** 64 | - Keep summaries concise but descriptive 65 | - Always use the full absolute path for project_directory 66 | - Use interactive feedback as a quality gate, not a blocker 67 | - Respect empty feedback responses as approval to proceed 68 | --------------------------------------------------------------------------------