├── LICENSE ├── README.md ├── RIP-01.md ├── RIP-02.md ├── RIP-03.md ├── RIP-04.md └── RIP-05.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2025 routstr 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Routstr Improvement Protocols (RIPs) 2 | 3 | ![RIP](https://www.stuttgarter-zeitung.de/media.media.2418d951-7f77-41ba-a726-08d3a8043fd9.16x9_1024.jpg) 4 | 5 | Routstr Improvement Protocols (RIPs) are modular specification documents defining the events, behaviors, and interfaces for the decentralized AI inference network. They complement the core protocol by specifying discrete improvements, standards, and guidelines that implementations can adopt to enhance interoperability, security, and user experience. 6 | 7 | ## List of RIPs 8 | 9 | | ID | Title | Description | 10 | |--------|--------------------------------------------------|------------------------------------------------------------------------------| 11 | | RIP-01 | [OpenAI-API Proxy with Cashu Payments](RIP-01.md) | Proxy spec for OpenAI-compatible API requests, with per-request Cashu payment | 12 | | RIP-02 | [Node Listing](RIP-02.md) | Nostr event spec for announcing inference node presence and capabilities | 13 | | RIP-03 | [Frontend Discovery](RIP-03.md) | Specification for the discovery UI to browse and filter available nodes | 14 | | RIP-04 | [Evaluations & Quality Control](RIP-04.md) | Guidelines for randomized, anonymized evaluations to ensure provider quality | 15 | | RIP-05 | [Smart Clients & Token Management](RIP-05.md) | Client behaviors for token cycling, proxy/Tor usage, and provider optimization | 16 | 17 | Refer to each RIP file in this directory for detailed implementation guidance. 18 | -------------------------------------------------------------------------------- /RIP-01.md: -------------------------------------------------------------------------------- 1 | # RIP-01: OpenAI-API Proxy with Cashu Payments 2 | 3 | [x] - Implement HTTP proxy interface for OpenAI-compatible API requests 4 | 5 | - [x] Integrate per-request micropayment handling via Cashu tokens 6 | - [x] Support endpoints: 7 | - [x] POST /v1/{path:path} (proxy to upstream AI service) 8 | - [x] GET /v1/wallet (get Cashu token balance and API key) 9 | - [x] POST /v1/wallet/topup (top up wallet with Cashu token) 10 | - [x] POST /v1/wallet/refund (refund remaining balance as Cashu token) 11 | - [x] Authorization via Bearer API key or Cashu token 12 | - [x] API key and Cashu token validation and redemption 13 | - [x] Token-based pricing for chat/completions with MODEL_BASED_PRICING 14 | - [x] SSE and JSON cost reporting in responses 15 | - [ ] Tor hidden service deployment support 16 | - [ ] Admin dashboard for monitoring and settings 17 | 18 | Defines the HTTP proxy interface forwarding OpenAI-compatible API requests, with per-request micropayment handling via Cashu tokens. 19 | 20 | ## Endpoints 21 | 22 | ### POST /v1/{path:path} e.g. /v1/chat/completions 23 | 24 | Forward proxied requests to the upstream AI service at `UPSTREAM_BASE_URL`. 25 | This is usually your llamacpp or vllm server. 26 | 27 | ### GET /v1/wallet 28 | 29 | Get the balance of the Cashu token and the associated API key. 30 | 31 | ### POST /v1/wallet/topup 32 | 33 | Top up the balance of that temporary wallet with another Cashu token. 34 | 35 | ### POST /v1/wallet/refund 36 | 37 | Recieve the full amount of the remaining balance as a Cashu token. 38 | 39 | #### Authorization 40 | 41 | - Header: `Authorization: Bearer ` or `Bearer ` 42 | - API keys: prefixed `sk-`, validated against stored hashed keys. 43 | - Cashu tokens: prefixed `cashu`, redeemed via Cashu mint. 44 | 45 | #### Payment Flow 46 | 47 | 1. **Balance**: The cashu token is redeemed and the balance is credited internally (allows reuse of the same token for multiple requests). 48 | 2. **Api-Key**: Each balance has an associated API key that allowes for top-ups. 49 | 3. **Token based pricing**: For chat/completions when `MODEL_BASED_PRICING`=true: 50 | - Parse upstream `usage.prompt_tokens` and `usage.completion_tokens`. 51 | - Compute `input_msats` = prompt_tokens/1000 × model.prompt_price. 52 | - Compute `output_msats` = completion_tokens/1000 × model.completion_price. 53 | - Charge or refund the difference via balance adjustment. 54 | - Emit SSE event `data: {"cost": {...}}` appended to stream. 55 | 56 | #### Responses 57 | 58 | - **Streaming** (`text/event-stream`): Proxy SSE chunks and inject a final cost event. 59 | - **Non-Streaming** (`application/json`): Embed a top-level `"cost": {...}` field in JSON. 60 | 61 | #### Error Codes 62 | 63 | - `401 Unauthorized`: Missing or invalid API key/Cashu token. 64 | - `402 Payment Required`: Insufficient balance. 65 | - `502 Bad Gateway`: Upstream service unavailable. 66 | - `500 Internal Server Error`: Unexpected errors. 67 | 68 | ## Tor Routing 69 | 70 | Every node can be deployed as a Tor hidden service. 71 | 72 | ## Multi model support 73 | 74 | Each node can serve multiple models. 75 | These models are defined in the root `/` endpoint. 76 | Including node information and pricing. 77 | -------------------------------------------------------------------------------- /RIP-02.md: -------------------------------------------------------------------------------- 1 | # RIP-02: Node Listing 2 | 3 | - [ ] Create PR to the Nostr repo to add a new Kind 40500 event 4 | 5 | Nodes announce their presence and capabilities via a Nostr event for client discovery. 6 | 7 | **Kind**: 40500 8 | 9 | ```json 10 | { 11 | "kind": 40500, 12 | "created_at": , 13 | "tags": [ 14 | ["d", ""], // Unique node identifier 15 | ["p", ""], 16 | ["url", "https://..."], // Inference endpoint 17 | ["onion", ""], // Tor hidden service endpoint 18 | ], 19 | "content": "Human-readable description" 20 | } 21 | ``` 22 | 23 | Clients subscribe to Kind 40500, index by `d` tag, and override older listings with newer events. 24 | -------------------------------------------------------------------------------- /RIP-03.md: -------------------------------------------------------------------------------- 1 | # TODO 2 | 3 | - [x] 100% local frontend (no backend) 4 | - [x] UI components: search bar, filters panel, model card (description, models, price) 5 | - [x] Include chat application, discovery interface, landing page, and documentation 6 | - [x] Support Nostr-based login 7 | - [x] Cashu wallet integration 8 | - [ ] Filter nodes by supported models, region, price range, social proximity, and average rating 9 | - [ ] Allow users to publish personal node information and interact with nodes 10 | - [ ] List active nodes from Nostr Kind 40500 11 | - [ ] Subscribe to Nostr Kind 40500 & 31555 events 12 | - [ ] Maintain in-memory index and update on new events 13 | - [ ] Render node list with applied filters 14 | - [ ] Real-time UI updates on new events 15 | 16 | # RIP-03: Frontend Discovery 17 | 18 | This document defines a fully client-side, 100% local web or app interface for node discovery, chat, and interaction. There is no backend: all logic and data flow run in the user's browser or device. The frontend includes a chat application, discovery interface, landing page, and documentation. Authentication and filtering leverage Nostr for login and social graph-based filtering. Users can publish their own node information or interact with available nodes directly from the interface. 19 | 20 | ## Features 21 | 22 | - 100% local frontend: no backend, all data and logic run on the client. 23 | - Includes: 24 | - Chat application 25 | - Discovery interface 26 | - Landing page 27 | - Documentation 28 | - Nostr-based login and filtering (social graph proximity) 29 | - List active nodes from Nostr Kind 40500. 30 | - Filter by: 31 | - Supported model(s) 32 | - Region 33 | - Price range 34 | - Social network proximity (follow graph) 35 | - Average rating (from evals) 36 | - Publish personal node information or interact with nodes 37 | - Future: Toggle eval visibility once RIP-04 is live. 38 | 39 | ## UI Components 40 | 41 | - **Search Bar**: Free text for node-id or description. 42 | - **Filters Panel**: Sliders and checkboxes. 43 | - **Node Card**: 44 | - Description 45 | - Models 46 | - Price 47 | - Rating summary 48 | - Connect button (opens external client) 49 | 50 | ## Data Flow 51 | 52 | 1. Subscribe to Nostr Kind 40500 & 31555 events. 53 | 2. Maintain in-memory index, update on new events. 54 | 3. Join with evals to compute live scores. 55 | 4. Render node list with applied filters. 56 | 57 | ## Metrics 58 | 59 | - Query performance: ≤200 ms for 1 k nodes. 60 | - Real-time updates: UI refresh on new events. 61 | -------------------------------------------------------------------------------- /RIP-04.md: -------------------------------------------------------------------------------- 1 | # RIP-04: Evaluations & Quality Control 2 | 3 | - [ ] Implement independent evaluation (eval) services for LLM providers 4 | - [ ] Scan Nostr for all advertised providers 5 | - [ ] Run anonymized, randomized, proxied evaluation jobs 6 | - [ ] Publish benchmark reports for each provider (Kind 31555) 7 | - [ ] Support free and paid report tiers 8 | - [ ] Measure and report quality, latency, and cost metrics 9 | - [ ] Publish standard ratings to Nostr 10 | - [ ] Limit eval jobs to ≤5% of overall requests 11 | - [ ] Make public evals modular component of the protocol 12 | 13 | Specifies how independent evaluation (eval) services scan Nostr for all advertised LLM providers, run anonymized and randomized evals, and publish benchmark reports for each provider. 14 | 15 | ## Goals 16 | 17 | - Prevent provider bias by making evals indistinguishable from real inference calls. 18 | - Collect unbiased metrics on quality, latency, and cost. 19 | - Establish eval services as the primary source of trust and ranking in the marketplace. 20 | 21 | ## Eval Service Model 22 | 23 | - Evals are separate, continuously running services that monitor Nostr for all LLM providers advertising their capabilities. 24 | - For each discovered provider, the eval service runs anonymized, randomized, and proxied evaluation jobs to ensure providers cannot distinguish eval requests from normal user traffic. 25 | - Evals are designed to prevent providers from treating evals differently than real users, using strategies such as randomization, proxying, and ephemeral keys. 26 | - After running evals, the service publishes a report for each provider, containing benchmark scores and detailed metrics. 27 | 28 | ## Report Tiers & Monetization 29 | 30 | - There are multiple tiers of reports: 31 | - **Free Report:** All providers receive a basic report at no cost, summarizing essential metrics. 32 | - **Paid Reports:** Providers can pay the eval service (in sats) to receive more advanced, detailed reports. These paid reports offer deeper insights, improve user trust, and help the provider rank higher in the marketplace. 33 | - Eval services are a monetizable component of the ecosystem, earning fees for advanced reporting and acting as a trusted third party. 34 | 35 | ## Eval Flow 36 | 37 | 1. Eval service scans Nostr for all advertised providers. 38 | 2. For each provider: 39 | - Generate randomized inference parameters. 40 | - Send anonymized, proxied inference requests. 41 | - Measure latency, output quality against ground truth. 42 | 3. Assemble eval result and prepare Nostr event: 43 | - Use new random ephemeral Nostr key per eval. 44 | - Publish Kind 31555 with standard rating tags: 45 | - `d`: provider node-id 46 | - `rating` categories: `quality`, `latency`, `value` 47 | - No direct link to eval service's main pubkey. 48 | 4. Publish report (free or paid tier) with benchmark scores. 49 | 50 | ## Tags 51 | 52 | - `d`: provider node-id 53 | - `rating`: 54 | - `quality`: 0–1 match to ground truth 55 | - `latency`: normalized inverse latency score 56 | - `value`: cost vs performance 57 | - `content`: optional detailed notes 58 | 59 | ## Anonymity & Randomization 60 | 61 | - Ephemeral keys rotated per session. 62 | - Randomized send times & intervals. 63 | - All requests are proxied and randomized to prevent provider detection. 64 | 65 | ## Aggregation 66 | 67 | - Eval services can batch multiple evaluations in one event by repeating `rating` tags. 68 | 69 | ## Frequency 70 | 71 | - Limit eval jobs to ≤5% of overall requests to avoid load spikes. 72 | 73 | ## Trust & Benchmarking 74 | 75 | - Eval services are the source of trust in the marketplace, providing objective, third-party benchmarks. 76 | - Each report results in a set of benchmark scores, enabling transparent provider comparison and ranking. 77 | -------------------------------------------------------------------------------- /RIP-05.md: -------------------------------------------------------------------------------- 1 | # TODO 2 | 3 | - [ ] Develop multi-language SDKs (Python, TypeScript/JavaScript, Go, Rust, etc.) 4 | - [ ] Implement full-stack components: client, server, discovery, evaluation server, chat API 5 | - [ ] Provide unified interfaces for Cashu wallet management, API key rotation, proxy/anonymity 6 | - [ ] Expose high-level abstractions in SDKs (token management, API key rotation, provider selection, proxy/Tor integration, secure flows) 7 | - [ ] Smart clients: automated token cycling/top-up, per-request API key rotation, proxy/Tor circuit selection 8 | - [ ] Dynamic provider discovery and scoring (price, latency, rating, reliability) 9 | - [ ] Pluggable architecture for custom scoring, proxy, or wallet backends 10 | - [ ] Reference implementations for server, discovery, and evaluation endpoints in each language 11 | - [ ] Standardized APIs for provider registration, health, and metrics 12 | - [ ] Modular evaluation server for benchmarking and rating providers 13 | - [ ] Chat API reference implementations (real-time and batch) 14 | - [ ] Built-in proxy and Tor routing in all SDKs and clients 15 | - [ ] Automated rotation of API keys and tokens 16 | - [ ] Rate limiting and circuit rotation to prevent fingerprinting 17 | - [ ] Define core protocol interfaces and data models in each language 18 | - [ ] Maintain open-source SDKs and reference servers 19 | - [ ] Ensure feature parity and cross-language test suites 20 | - [ ] Document integration patterns and privacy best practices 21 | 22 | # RIP-05: SDKs & Smart Clients 23 | 24 | This RIP proposes a comprehensive suite of SDKs and Smart Client libraries across multiple programming languages, enabling seamless integration with the protocol and ensuring robust privacy, payment, and provider management. 25 | 26 | ## Scope 27 | 28 | - Multi-language SDKs: Python, TypeScript/JavaScript, Go, Rust, and others as needed. 29 | - Full-stack implementations for: 30 | - Client 31 | - Server 32 | - Discovery 33 | - Evaluation Server 34 | - Chat API 35 | - Unified interfaces for Cashu wallet management, API key rotation, and proxy/anonymity features. 36 | 37 | ## Components 38 | 39 | ### 1. SDKs 40 | 41 | - Provide idiomatic libraries in each language for all protocol operations. 42 | - Expose high-level abstractions for: 43 | - Token management (Cashu wallets) 44 | - API key rotation 45 | - Provider selection and scoring 46 | - Proxy/Tor integration for request anonymization 47 | - Secure, stateless request/response flows 48 | 49 | ### 2. Smart Clients 50 | 51 | - Automated token cycling and top-up from Cashu wallets. 52 | - Per-request API key rotation and proxy/Tor circuit selection. 53 | - Dynamic provider discovery and scoring (price, latency, rating, reliability). 54 | - Pluggable architecture for custom scoring, proxy, or wallet backends. 55 | 56 | ### 3. Server & Discovery 57 | 58 | - Reference implementations for server, discovery, and evaluation endpoints in each supported language. 59 | - Standardized APIs for provider registration, health, and metrics reporting. 60 | 61 | ### 4. Evaluation Server & Chat API 62 | 63 | - Modular evaluation server for benchmarking and rating providers. 64 | - Chat API reference implementations for real-time and batch inference. 65 | 66 | ## Privacy & Security 67 | 68 | - Built-in support for proxy and Tor routing in all SDKs and clients. 69 | - Automated rotation of API keys and tokens to minimize linkability. 70 | - Rate limiting and circuit rotation to prevent fingerprinting. 71 | 72 | ## Implementation Plan 73 | 74 | - Define core protocol interfaces and data models in each language. 75 | - Develop and maintain open-source SDKs and reference servers. 76 | - Ensure feature parity and cross-language test suites. 77 | - Document integration patterns and privacy best practices. 78 | 79 | --- 80 | --------------------------------------------------------------------------------