├── images ├── adaptive-rag.png ├── agentic-rag.png ├── agentic-profile.jpg ├── spring-ai-rag.jpg ├── knowledge-example.png ├── context-engineering.jpg ├── open-deep-research.jpg ├── task-decomposition.jpg ├── prompt-chaining-example.png ├── prompt-chaining-routing.jpg ├── agentic-context-engineering.jpg ├── codebase-indexing-with-graph.png ├── langchain-context-enginneering.png ├── multiple-agent-architectures.png ├── structured-output-architecture.jpg └── Understanding-GitHub-Copilots-Internal-Architecture.svg ├── CtxMesh ├── shared │ ├── src │ │ ├── commonMain │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ ├── Constants.kt │ │ │ │ ├── Platform.kt │ │ │ │ ├── context │ │ │ │ ├── retrieval │ │ │ │ │ ├── RetrievalStrategy.kt │ │ │ │ │ ├── rewrite │ │ │ │ │ │ ├── QueryRewriter.kt │ │ │ │ │ │ └── SimpleCodeQueryRewriter.kt │ │ │ │ │ ├── ContextRetriever.kt │ │ │ │ │ ├── HybridRetriever.kt │ │ │ │ │ └── KeywordRetriever.kt │ │ │ │ ├── document │ │ │ │ │ └── IndexedDocument.kt │ │ │ │ ├── ContextPriority.kt │ │ │ │ ├── ContextWindowStatus.kt │ │ │ │ ├── ContextConfig.kt │ │ │ │ ├── ContextType.kt │ │ │ │ ├── ContextWindow.kt │ │ │ │ ├── ContextManagerBuilder.kt │ │ │ │ ├── ContextFactory.kt │ │ │ │ ├── DefaultContextWindow.kt │ │ │ │ └── ContextManager.kt │ │ │ │ ├── Greeting.kt │ │ │ │ └── memory │ │ │ │ ├── Memory.kt │ │ │ │ ├── impl │ │ │ │ ├── DefaultShortMemory.kt │ │ │ │ └── DefaultLongMemory.kt │ │ │ │ ├── ShortMemory.kt │ │ │ │ ├── MemoryManager.kt │ │ │ │ └── LongMemory.kt │ │ ├── jsMain │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ └── Platform.js.kt │ │ ├── wasmJsMain │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ └── Platform.wasmJs.kt │ │ ├── jvmMain │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ ├── Platform.jvm.kt │ │ │ │ ├── ContextBuilderWithMemory.kt │ │ │ │ └── ContextManagerWithMemory.kt │ │ └── commonTest │ │ │ └── kotlin │ │ │ └── com │ │ │ └── phodal │ │ │ └── ctxmesh │ │ │ ├── context │ │ │ ├── ContextManagerTest.kt │ │ │ └── ContextWindowTest.kt │ │ │ └── memory │ │ │ ├── MemoryTest.kt │ │ │ └── MemoryManagerTest.kt │ └── build.gradle.kts ├── gradle │ ├── wrapper │ │ ├── gradle-wrapper.jar │ │ └── gradle-wrapper.properties │ └── libs.versions.toml ├── composeApp │ ├── src │ │ ├── webMain │ │ │ ├── resources │ │ │ │ ├── styles.css │ │ │ │ └── index.html │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ └── main.kt │ │ ├── commonTest │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ └── ComposeAppCommonTest.kt │ │ ├── jvmMain │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ └── main.kt │ │ └── commonMain │ │ │ ├── composeResources │ │ │ └── drawable │ │ │ │ └── compose-multiplatform.xml │ │ │ └── kotlin │ │ │ └── com │ │ │ └── phodal │ │ │ └── ctxmesh │ │ │ └── App.kt │ ├── webpack.config.d │ │ └── watch.js │ └── build.gradle.kts ├── gradle.properties ├── .gitignore ├── build.gradle.kts ├── server │ ├── src │ │ ├── main │ │ │ ├── resources │ │ │ │ └── logback.xml │ │ │ └── kotlin │ │ │ │ └── com │ │ │ │ └── phodal │ │ │ │ └── ctxmesh │ │ │ │ └── Application.kt │ │ └── test │ │ │ └── kotlin │ │ │ └── com │ │ │ └── phodal │ │ │ └── ctxmesh │ │ │ └── ApplicationTest.kt │ └── build.gradle.kts ├── settings.gradle.kts ├── gradlew.bat ├── README.md └── gradlew ├── multiple-agents.md └── README.md /images/adaptive-rag.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/adaptive-rag.png -------------------------------------------------------------------------------- /images/agentic-rag.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/agentic-rag.png -------------------------------------------------------------------------------- /images/agentic-profile.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/agentic-profile.jpg -------------------------------------------------------------------------------- /images/spring-ai-rag.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/spring-ai-rag.jpg -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/Constants.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | const val SERVER_PORT = 8080 -------------------------------------------------------------------------------- /images/knowledge-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/knowledge-example.png -------------------------------------------------------------------------------- /images/context-engineering.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/context-engineering.jpg -------------------------------------------------------------------------------- /images/open-deep-research.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/open-deep-research.jpg -------------------------------------------------------------------------------- /images/task-decomposition.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/task-decomposition.jpg -------------------------------------------------------------------------------- /images/prompt-chaining-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/prompt-chaining-example.png -------------------------------------------------------------------------------- /images/prompt-chaining-routing.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/prompt-chaining-routing.jpg -------------------------------------------------------------------------------- /images/agentic-context-engineering.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/agentic-context-engineering.jpg -------------------------------------------------------------------------------- /CtxMesh/gradle/wrapper/gradle-wrapper.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/CtxMesh/gradle/wrapper/gradle-wrapper.jar -------------------------------------------------------------------------------- /images/codebase-indexing-with-graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/codebase-indexing-with-graph.png -------------------------------------------------------------------------------- /images/langchain-context-enginneering.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/langchain-context-enginneering.png -------------------------------------------------------------------------------- /images/multiple-agent-architectures.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/multiple-agent-architectures.png -------------------------------------------------------------------------------- /images/structured-output-architecture.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/phodal/build-agent-context-engineering/master/images/structured-output-architecture.jpg -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/webMain/resources/styles.css: -------------------------------------------------------------------------------- 1 | html, body { 2 | width: 100%; 3 | height: 100%; 4 | margin: 0; 5 | padding: 0; 6 | overflow: hidden; 7 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/Platform.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | interface Platform { 4 | val name: String 5 | } 6 | 7 | expect fun getPlatform(): Platform -------------------------------------------------------------------------------- /CtxMesh/gradle.properties: -------------------------------------------------------------------------------- 1 | #Kotlin 2 | kotlin.code.style=official 3 | kotlin.daemon.jvmargs=-Xmx3072M 4 | #Gradle 5 | org.gradle.jvmargs=-Xmx3072M -Dfile.encoding=UTF-8 6 | org.gradle.configuration-cache=true 7 | org.gradle.caching=true -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/retrieval/RetrievalStrategy.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.retrieval 2 | 3 | enum class RetrievalStrategy { 4 | KEYWORD, 5 | SEMANTIC, 6 | HYBRID, 7 | GRAPH 8 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/jsMain/kotlin/com/phodal/ctxmesh/Platform.js.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | class JsPlatform : Platform { 4 | override val name: String = "Web with Kotlin/JS" 5 | } 6 | 7 | actual fun getPlatform(): Platform = JsPlatform() -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/Greeting.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | class Greeting { 4 | private val platform = getPlatform() 5 | 6 | fun greet(): String { 7 | return "Hello, ${platform.name}!" 8 | } 9 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/wasmJsMain/kotlin/com/phodal/ctxmesh/Platform.wasmJs.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | class WasmPlatform : Platform { 4 | override val name: String = "Web with Kotlin/Wasm" 5 | } 6 | 7 | actual fun getPlatform(): Platform = WasmPlatform() -------------------------------------------------------------------------------- /CtxMesh/shared/src/jvmMain/kotlin/com/phodal/ctxmesh/Platform.jvm.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | class JVMPlatform : Platform { 4 | override val name: String = "Java ${System.getProperty("java.version")}" 5 | } 6 | 7 | actual fun getPlatform(): Platform = JVMPlatform() -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/commonTest/kotlin/com/phodal/ctxmesh/ComposeAppCommonTest.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import kotlin.test.Test 4 | import kotlin.test.assertEquals 5 | 6 | class ComposeAppCommonTest { 7 | 8 | @Test 9 | fun example() { 10 | assertEquals(3, 1 + 2) 11 | } 12 | } -------------------------------------------------------------------------------- /CtxMesh/gradle/wrapper/gradle-wrapper.properties: -------------------------------------------------------------------------------- 1 | distributionBase=GRADLE_USER_HOME 2 | distributionPath=wrapper/dists 3 | distributionUrl=https\://services.gradle.org/distributions/gradle-8.14.3-bin.zip 4 | networkTimeout=10000 5 | validateDistributionUrl=true 6 | zipStoreBase=GRADLE_USER_HOME 7 | zipStorePath=wrapper/dists 8 | -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/webMain/kotlin/com/phodal/ctxmesh/main.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import androidx.compose.ui.ExperimentalComposeUiApi 4 | import androidx.compose.ui.window.ComposeViewport 5 | 6 | @OptIn(ExperimentalComposeUiApi::class) 7 | fun main() { 8 | ComposeViewport { 9 | App() 10 | } 11 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/document/IndexedDocument.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.document 2 | 3 | /** 4 | * 索引文档数据类 5 | */ 6 | data class IndexedDocument( 7 | val id: String, 8 | val content: String, 9 | val metadata: Map, 10 | val terms: Set 11 | ) -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/jvmMain/kotlin/com/phodal/ctxmesh/main.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import androidx.compose.ui.window.Window 4 | import androidx.compose.ui.window.application 5 | 6 | fun main() = application { 7 | Window( 8 | onCloseRequest = ::exitApplication, 9 | title = "ctxmesh", 10 | ) { 11 | App() 12 | } 13 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/retrieval/rewrite/QueryRewriter.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.retrieval.rewrite 2 | 3 | /** 4 | * 查询改写器接口 5 | * 用于优化用户查询以提升检索效果 6 | */ 7 | interface QueryRewriter { 8 | /** 9 | * 改写查询以提升检索效果 10 | * @param originalQuery 原始查询 11 | * @return 改写后的查询列表 12 | */ 13 | suspend fun rewrite(originalQuery: String): List 14 | } -------------------------------------------------------------------------------- /CtxMesh/.gitignore: -------------------------------------------------------------------------------- 1 | *.iml 2 | .kotlin 3 | .gradle 4 | **/build/ 5 | xcuserdata 6 | !src/**/build/ 7 | local.properties 8 | .idea 9 | .DS_Store 10 | captures 11 | .externalNativeBuild 12 | .cxx 13 | *.xcodeproj/* 14 | !*.xcodeproj/project.pbxproj 15 | !*.xcodeproj/xcshareddata/ 16 | !*.xcodeproj/project.xcworkspace/ 17 | !*.xcworkspace/contents.xcworkspacedata 18 | **/xcshareddata/WorkspaceSettings.xcsettings 19 | node_modules/ 20 | -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/webMain/resources/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | ctxmesh 7 | 8 | 9 | 10 | 11 | 12 | -------------------------------------------------------------------------------- /CtxMesh/build.gradle.kts: -------------------------------------------------------------------------------- 1 | plugins { 2 | // this is necessary to avoid the plugins to be loaded multiple times 3 | // in each subproject's classloader 4 | alias(libs.plugins.composeHotReload) apply false 5 | alias(libs.plugins.composeMultiplatform) apply false 6 | alias(libs.plugins.composeCompiler) apply false 7 | alias(libs.plugins.kotlinJvm) apply false 8 | alias(libs.plugins.kotlinMultiplatform) apply false 9 | } 10 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/retrieval/ContextRetriever.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.retrieval 2 | 3 | import com.phodal.ctxmesh.context.ContextContent 4 | 5 | interface ContextRetriever { 6 | suspend fun retrieve(query: String, maxResults: Int = 5): List 7 | suspend fun index(content: String, metadata: Map = emptyMap()): String 8 | suspend fun remove(contentId: String): Boolean 9 | } 10 | 11 | -------------------------------------------------------------------------------- /CtxMesh/server/src/main/resources/logback.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | %d{YYYY-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextPriority.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | /** 4 | * 上下文优先级 5 | * 基于 GitHub Copilot 的优先级设计理念 6 | */ 7 | enum class ContextPriority(val weight: Int) { 8 | /** 最高优先级：光标位置周围的代码等 */ 9 | HIGHEST(100), 10 | 11 | /** 高优先级：当前正在编辑的文件 */ 12 | HIGH(80), 13 | 14 | /** 中等优先级：打开的其他文件 */ 15 | MEDIUM(60), 16 | 17 | /** 低优先级：辅助上下文 */ 18 | LOW(40), 19 | 20 | /** 最低优先级：可选的背景信息 */ 21 | LOWEST(20) 22 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextWindowStatus.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | /** 4 | * 上下文窗口状态信息 5 | */ 6 | data class ContextWindowStatus( 7 | val totalTokenBudget: Int, 8 | val usedTokens: Int, 9 | val remainingTokens: Int, 10 | val contextCount: Int, 11 | val contextsByType: Map> 12 | ) { 13 | val utilizationRate: Double get() = usedTokens.toDouble() / totalTokenBudget 14 | val isNearCapacity: Boolean get() = utilizationRate > 0.8 15 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextConfig.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | import com.phodal.ctxmesh.context.retrieval.RetrievalStrategy 4 | 5 | data class ContextConfig( 6 | val tokenBudget: Int = 8192, 7 | val enableRetrieval: Boolean = true, 8 | val retrievalStrategy: RetrievalStrategy = RetrievalStrategy.KEYWORD, 9 | val maxRetrievalResults: Int = 3, 10 | val enableQueryRewriting: Boolean = true, 11 | val shortTermMemoryMaxAge: Long = 3600000, 12 | val autoCleanup: Boolean = true 13 | ) -------------------------------------------------------------------------------- /CtxMesh/server/src/main/kotlin/com/phodal/ctxmesh/Application.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import io.ktor.server.application.* 4 | import io.ktor.server.engine.* 5 | import io.ktor.server.netty.* 6 | import io.ktor.server.response.* 7 | import io.ktor.server.routing.* 8 | 9 | fun main() { 10 | embeddedServer(Netty, port = SERVER_PORT, host = "0.0.0.0", module = Application::module) 11 | .start(wait = true) 12 | } 13 | 14 | fun Application.module() { 15 | routing { 16 | get("/") { 17 | call.respondText("Ktor: ${Greeting().greet()}") 18 | } 19 | } 20 | } -------------------------------------------------------------------------------- /CtxMesh/server/src/test/kotlin/com/phodal/ctxmesh/ApplicationTest.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import io.ktor.client.request.* 4 | import io.ktor.client.statement.* 5 | import io.ktor.http.* 6 | import io.ktor.server.testing.* 7 | import kotlin.test.* 8 | 9 | class ApplicationTest { 10 | 11 | @Test 12 | fun testRoot() = testApplication { 13 | application { 14 | module() 15 | } 16 | val response = client.get("/") 17 | assertEquals(HttpStatusCode.OK, response.status) 18 | assertEquals("Ktor: ${Greeting().greet()}", response.bodyAsText()) 19 | } 20 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextType.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | /** 4 | * 上下文类型枚举 5 | * 基于 README.md 中提到的上下文分类 6 | */ 7 | enum class ContextType { 8 | /** 系统提示词部分 */ 9 | SYSTEM_PROMPT, 10 | 11 | /** 格式化输出上下文 */ 12 | OUTPUT_FORMAT, 13 | 14 | /** 工具相关上下文 */ 15 | TOOL_CONTEXT, 16 | 17 | /** 外部知识上下文（RAG检索结果） */ 18 | EXTERNAL_KNOWLEDGE, 19 | 20 | /** 短期记忆 */ 21 | SHORT_TERM_MEMORY, 22 | 23 | /** 长期记忆 */ 24 | LONG_TERM_MEMORY, 25 | 26 | /** 全局状态/暂存区 */ 27 | GLOBAL_STATE, 28 | 29 | /** 用户输入 */ 30 | USER_INPUT 31 | } -------------------------------------------------------------------------------- /CtxMesh/composeApp/webpack.config.d/watch.js: -------------------------------------------------------------------------------- 1 | /* 2 | * Temporary workaround for [KT-80582](https://youtrack.jetbrains.com/issue/KT-80582) 3 | * 4 | * This file should be safe to be removed once the ticket is closed and the project is updated to Kotlin version which solves that issue. 5 | */ 6 | config.watchOptions = config.watchOptions || { 7 | ignored: ["**/*.kt", "**/node_modules"] 8 | } 9 | 10 | if (config.devServer) { 11 | config.devServer.static = config.devServer.static.map(file => { 12 | if (typeof file === "string") { 13 | return { 14 | directory: file, 15 | watch: false, 16 | } 17 | } else { 18 | return file 19 | } 20 | }) 21 | } -------------------------------------------------------------------------------- /CtxMesh/server/build.gradle.kts: -------------------------------------------------------------------------------- 1 | plugins { 2 | alias(libs.plugins.kotlinJvm) 3 | alias(libs.plugins.ktor) 4 | application 5 | } 6 | 7 | group = "com.phodal.ctxmesh" 8 | version = "1.0.0" 9 | application { 10 | mainClass.set("com.phodal.ctxmesh.ApplicationKt") 11 | 12 | val isDevelopment: Boolean = project.ext.has("development") 13 | applicationDefaultJvmArgs = listOf("-Dio.ktor.development=$isDevelopment") 14 | } 15 | 16 | dependencies { 17 | implementation(projects.shared) 18 | implementation(libs.logback) 19 | implementation(libs.ktor.serverCore) 20 | implementation(libs.ktor.serverNetty) 21 | testImplementation(libs.ktor.serverTestHost) 22 | testImplementation(libs.kotlin.testJunit) 23 | } -------------------------------------------------------------------------------- /CtxMesh/shared/build.gradle.kts: -------------------------------------------------------------------------------- 1 | import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl 2 | 3 | plugins { 4 | alias(libs.plugins.kotlinMultiplatform) 5 | 6 | } 7 | 8 | kotlin { 9 | jvm() 10 | 11 | js { 12 | browser() 13 | } 14 | 15 | @OptIn(ExperimentalWasmDsl::class) 16 | wasmJs { 17 | browser() 18 | } 19 | 20 | sourceSets { 21 | commonMain.dependencies { 22 | // put your Multiplatform dependencies here 23 | implementation(libs.koog.agents) 24 | implementation(libs.kotlinx.datetime) 25 | } 26 | commonTest.dependencies { 27 | implementation(libs.kotlin.test) 28 | implementation(libs.kotlinx.coroutines.test) 29 | } 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /multiple-agents.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Augement 4 | 5 | 让我等待索引完成，然后继续下一步。由于索引过程可能还需要一些时间，我先开始准备 Playwright 测试脚本。 6 | 7 | ● Update Tasks - updating 1 task 8 | ⎿ State → In Progress 9 | ⎿ Updated tasks 10 | 11 | ● 首先安装 Playwright 依赖： 12 | 13 | ● source venv/bin/activate && pip install playwright pytest-playwright - running command 14 | ⎿ Command completed 15 | 16 | ● 现在安装浏览器驱动： 17 | 18 | ● source venv/bin/activate && playwright install - running command 19 | ⎿ Command completed 20 | 21 | ● 很好！Playwright 安装完成。现在让我更新任务状态并开始编写测试脚本： 22 | 23 | ● Update Tasks - updating 2 tasks 24 | ⎿ State → Complete 25 | ⎿ State → In Progress 26 | ⎿ Updated tasks 27 | 28 | ● 现在创建 Playwright 测试脚本。首先创建测试目录和配置： 29 | 30 | ● test_technical_debt_query.py - save file 31 | ⎿ Saving at tests/test_technical_debt_query.py 32 | ⎿ Saved successfully. 33 | 34 | ● 现在创建 pytest 配置文件： 35 | 36 | ● pytest.ini - save file 37 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /CtxMesh/settings.gradle.kts: -------------------------------------------------------------------------------- 1 | rootProject.name = "ctxmesh" 2 | enableFeaturePreview("TYPESAFE_PROJECT_ACCESSORS") 3 | 4 | pluginManagement { 5 | repositories { 6 | google { 7 | mavenContent { 8 | includeGroupAndSubgroups("androidx") 9 | includeGroupAndSubgroups("com.android") 10 | includeGroupAndSubgroups("com.google") 11 | } 12 | } 13 | mavenCentral() 14 | gradlePluginPortal() 15 | } 16 | } 17 | 18 | dependencyResolutionManagement { 19 | repositories { 20 | google { 21 | mavenContent { 22 | includeGroupAndSubgroups("androidx") 23 | includeGroupAndSubgroups("com.android") 24 | includeGroupAndSubgroups("com.google") 25 | } 26 | } 27 | mavenCentral() 28 | } 29 | } 30 | 31 | plugins { 32 | id("org.gradle.toolchains.foojay-resolver-convention") version "1.0.0" 33 | } 34 | 35 | include(":composeApp") 36 | include(":server") 37 | include(":shared") -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/retrieval/rewrite/SimpleCodeQueryRewriter.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.retrieval.rewrite 2 | 3 | class SimpleCodeQueryRewriter : QueryRewriter { 4 | override suspend fun rewrite(originalQuery: String): List { 5 | val rewrites = mutableListOf() 6 | rewrites.add(originalQuery) 7 | 8 | val synonyms = mapOf( 9 | "function" to listOf("method", "procedure", "routine"), 10 | "class" to listOf("type", "object", "entity"), 11 | "variable" to listOf("field", "property", "attribute"), 12 | "error" to listOf("exception", "bug", "issue"), 13 | "implement" to listOf("create", "build", "develop") 14 | ) 15 | 16 | synonyms.forEach { (word, syns) -> 17 | if (originalQuery.contains(word, ignoreCase = true)) { 18 | syns.forEach { syn -> 19 | rewrites.add(originalQuery.replace(word, syn, ignoreCase = true)) 20 | } 21 | } 22 | } 23 | 24 | if (!originalQuery.endsWith("?")) { 25 | rewrites.add("How to $originalQuery?") 26 | rewrites.add("What is $originalQuery?") 27 | } 28 | 29 | return rewrites.distinct() 30 | } 31 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/jvmMain/kotlin/com/phodal/ctxmesh/ContextBuilderWithMemory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | /** 4 | * 带记忆系统的上下文构建器 5 | */ 6 | class ContextBuilderWithMemory(private val manager: ContextManagerWithMemory) { 7 | private var query: String = "" 8 | private var systemPrompt: String? = null 9 | private var outputFormat: String? = null 10 | private var maxRetrievalResults: Int = 3 11 | private var maxMemoryResults: Int = 5 12 | 13 | fun forQuery(query: String): ContextBuilderWithMemory { 14 | this.query = query 15 | return this 16 | } 17 | 18 | fun withSystemPrompt(prompt: String): ContextBuilderWithMemory { 19 | this.systemPrompt = prompt 20 | return this 21 | } 22 | 23 | fun withOutputFormat(format: String): ContextBuilderWithMemory { 24 | this.outputFormat = format 25 | return this 26 | } 27 | 28 | fun withMaxRetrievalResults(max: Int): ContextBuilderWithMemory { 29 | this.maxRetrievalResults = max 30 | return this 31 | } 32 | 33 | fun withMaxMemoryResults(max: Int): ContextBuilderWithMemory { 34 | this.maxMemoryResults = max 35 | return this 36 | } 37 | 38 | fun build(): String { 39 | return manager.buildContextForQuery( 40 | query = query, 41 | systemPrompt = systemPrompt, 42 | outputFormat = outputFormat, 43 | maxRetrievalResults = maxRetrievalResults, 44 | maxMemoryResults = maxMemoryResults 45 | ) 46 | } 47 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextWindow.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | /** 4 | * 上下文窗口的核心接口 5 | * 基于 README.md 中的上下文工程理念设计 6 | */ 7 | interface ContextWindow { 8 | /** 9 | * 获取当前上下文窗口的总 token 预算 10 | */ 11 | val tokenBudget: Int 12 | 13 | /** 14 | * 获取当前已使用的 token 数量 15 | */ 16 | val usedTokens: Int 17 | 18 | /** 19 | * 获取剩余可用的 token 数量 20 | */ 21 | val remainingTokens: Int get() = tokenBudget - usedTokens 22 | 23 | /** 24 | * 添加上下文内容到窗口 25 | * @param content 要添加的上下文内容 26 | * @return 是否成功添加（可能因为 token 预算不足而失败） 27 | */ 28 | fun addContext(content: ContextContent): Boolean 29 | 30 | /** 31 | * 移除指定的上下文内容 32 | * @param contentId 内容ID 33 | */ 34 | fun removeContext(contentId: String): Boolean 35 | 36 | /** 37 | * 根据优先级和 token 预算组装最终的上下文 38 | * @return 组装后的上下文字符串 39 | */ 40 | fun assembleContext(): String 41 | 42 | /** 43 | * 清空所有上下文内容 44 | */ 45 | fun clear() 46 | 47 | /** 48 | * 获取所有上下文内容 49 | */ 50 | fun getAllContexts(): List 51 | } 52 | 53 | /** 54 | * 上下文内容的抽象表示 55 | */ 56 | data class ContextContent( 57 | val id: String, 58 | val type: ContextType, 59 | val content: String, 60 | val priority: ContextPriority, 61 | val metadata: Map = emptyMap(), 62 | val timestamp: Long = kotlinx.datetime.Clock.System.now().toEpochMilliseconds() 63 | ) { 64 | /** 65 | * 估算内容的 token 数量（简化实现） 66 | */ 67 | val estimatedTokens: Int get() = content.length / 4 // 粗略估算：4个字符约等于1个token 68 | } 69 | 70 | -------------------------------------------------------------------------------- /CtxMesh/composeApp/build.gradle.kts: -------------------------------------------------------------------------------- 1 | import org.jetbrains.compose.desktop.application.dsl.TargetFormat 2 | import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl 3 | 4 | plugins { 5 | alias(libs.plugins.kotlinMultiplatform) 6 | alias(libs.plugins.composeMultiplatform) 7 | alias(libs.plugins.composeCompiler) 8 | alias(libs.plugins.composeHotReload) 9 | } 10 | 11 | kotlin { 12 | jvm() 13 | 14 | js { 15 | browser() 16 | binaries.executable() 17 | } 18 | 19 | @OptIn(ExperimentalWasmDsl::class) 20 | wasmJs { 21 | browser() 22 | binaries.executable() 23 | } 24 | 25 | sourceSets { 26 | commonMain.dependencies { 27 | implementation(compose.runtime) 28 | implementation(compose.foundation) 29 | implementation(compose.material3) 30 | implementation(compose.ui) 31 | implementation(compose.components.resources) 32 | implementation(compose.components.uiToolingPreview) 33 | implementation(libs.androidx.lifecycle.viewmodelCompose) 34 | implementation(libs.androidx.lifecycle.runtimeCompose) 35 | implementation(projects.shared) 36 | } 37 | commonTest.dependencies { 38 | implementation(libs.kotlin.test) 39 | } 40 | jvmMain.dependencies { 41 | implementation(compose.desktop.currentOs) 42 | implementation(libs.kotlinx.coroutinesSwing) 43 | } 44 | } 45 | } 46 | 47 | 48 | compose.desktop { 49 | application { 50 | mainClass = "com.phodal.ctxmesh.MainKt" 51 | 52 | nativeDistributions { 53 | targetFormats(TargetFormat.Dmg, TargetFormat.Msi, TargetFormat.Deb) 54 | packageName = "com.phodal.ctxmesh" 55 | packageVersion = "1.0.0" 56 | } 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/commonMain/composeResources/drawable/compose-multiplatform.xml: -------------------------------------------------------------------------------- 1 | 8 | 11 | 14 | 16 | 17 | 22 | 23 | 24 | 25 | 26 | 27 | 29 | 30 | 36 | 37 | 38 | 39 | 40 | 41 | 44 | -------------------------------------------------------------------------------- /CtxMesh/composeApp/src/commonMain/kotlin/com/phodal/ctxmesh/App.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import androidx.compose.animation.AnimatedVisibility 4 | import androidx.compose.foundation.Image 5 | import androidx.compose.foundation.background 6 | import androidx.compose.foundation.layout.Column 7 | import androidx.compose.foundation.layout.fillMaxSize 8 | import androidx.compose.foundation.layout.fillMaxWidth 9 | import androidx.compose.foundation.layout.safeContentPadding 10 | import androidx.compose.material3.Button 11 | import androidx.compose.material3.MaterialTheme 12 | import androidx.compose.material3.Text 13 | import androidx.compose.runtime.* 14 | import androidx.compose.ui.Alignment 15 | import androidx.compose.ui.Modifier 16 | import org.jetbrains.compose.resources.painterResource 17 | import org.jetbrains.compose.ui.tooling.preview.Preview 18 | 19 | import ctxmesh.composeapp.generated.resources.Res 20 | import ctxmesh.composeapp.generated.resources.compose_multiplatform 21 | 22 | @Composable 23 | @Preview 24 | fun App() { 25 | MaterialTheme { 26 | var showContent by remember { mutableStateOf(false) } 27 | Column( 28 | modifier = Modifier 29 | .background(MaterialTheme.colorScheme.primaryContainer) 30 | .safeContentPadding() 31 | .fillMaxSize(), 32 | horizontalAlignment = Alignment.CenterHorizontally, 33 | ) { 34 | Button(onClick = { showContent = !showContent }) { 35 | Text("Click me!") 36 | } 37 | AnimatedVisibility(showContent) { 38 | val greeting = remember { Greeting().greet() } 39 | Column( 40 | modifier = Modifier.fillMaxWidth(), 41 | horizontalAlignment = Alignment.CenterHorizontally, 42 | ) { 43 | Image(painterResource(Res.drawable.compose_multiplatform), null) 44 | Text("Compose: $greeting") 45 | } 46 | } 47 | } 48 | } 49 | } -------------------------------------------------------------------------------- /CtxMesh/gradle/libs.versions.toml: -------------------------------------------------------------------------------- 1 | [versions] 2 | androidx-lifecycle = "2.9.4" 3 | composeHotReload = "1.0.0-beta07" 4 | composeMultiplatform = "1.9.0" 5 | junit = "4.13.2" 6 | kotlin = "2.2.20" 7 | kotlinx-coroutines = "1.10.2" 8 | kotlinx-datetime = "0.6.1" 9 | ktor = "3.3.0" 10 | logback = "1.5.18" 11 | 12 | koog = "0.5.0" 13 | 14 | [libraries] 15 | koog-agents = { module = "ai.koog:koog-agents", version.ref = "koog" } 16 | 17 | # Others 18 | kotlin-test = { module = "org.jetbrains.kotlin:kotlin-test", version.ref = "kotlin" } 19 | kotlin-testJunit = { module = "org.jetbrains.kotlin:kotlin-test-junit", version.ref = "kotlin" } 20 | kotlinx-coroutines-test = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-test", version.ref = "kotlinx-coroutines" } 21 | kotlinx-datetime = { module = "org.jetbrains.kotlinx:kotlinx-datetime", version.ref = "kotlinx-datetime" } 22 | junit = { module = "junit:junit", version.ref = "junit" } 23 | androidx-lifecycle-viewmodelCompose = { module = "org.jetbrains.androidx.lifecycle:lifecycle-viewmodel-compose", version.ref = "androidx-lifecycle" } 24 | androidx-lifecycle-runtimeCompose = { module = "org.jetbrains.androidx.lifecycle:lifecycle-runtime-compose", version.ref = "androidx-lifecycle" } 25 | kotlinx-coroutinesSwing = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-swing", version.ref = "kotlinx-coroutines" } 26 | logback = { module = "ch.qos.logback:logback-classic", version.ref = "logback" } 27 | ktor-serverCore = { module = "io.ktor:ktor-server-core-jvm", version.ref = "ktor" } 28 | ktor-serverNetty = { module = "io.ktor:ktor-server-netty-jvm", version.ref = "ktor" } 29 | ktor-serverTestHost = { module = "io.ktor:ktor-server-test-host-jvm", version.ref = "ktor" } 30 | 31 | [plugins] 32 | composeHotReload = { id = "org.jetbrains.compose.hot-reload", version.ref = "composeHotReload" } 33 | composeMultiplatform = { id = "org.jetbrains.compose", version.ref = "composeMultiplatform" } 34 | composeCompiler = { id = "org.jetbrains.kotlin.plugin.compose", version.ref = "kotlin" } 35 | kotlinJvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" } 36 | ktor = { id = "io.ktor.plugin", version.ref = "ktor" } 37 | kotlinMultiplatform = { id = "org.jetbrains.kotlin.multiplatform", version.ref = "kotlin" } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextManagerBuilder.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | import com.phodal.ctxmesh.context.retrieval.ContextRetriever 4 | import com.phodal.ctxmesh.context.retrieval.HybridRetriever 5 | import com.phodal.ctxmesh.context.retrieval.KeywordRetriever 6 | import com.phodal.ctxmesh.context.retrieval.RetrievalStrategy 7 | import com.phodal.ctxmesh.context.retrieval.rewrite.QueryRewriter 8 | import com.phodal.ctxmesh.context.retrieval.rewrite.SimpleCodeQueryRewriter 9 | 10 | class ContextManagerBuilder { 11 | private var config = ContextConfig() 12 | private var customRetrievers = mutableListOf() 13 | private var customQueryRewriter: QueryRewriter? = null 14 | 15 | fun withConfig(config: ContextConfig): ContextManagerBuilder { 16 | this.config = config 17 | return this 18 | } 19 | 20 | fun withTokenBudget(budget: Int): ContextManagerBuilder { 21 | this.config = config.copy(tokenBudget = budget) 22 | return this 23 | } 24 | 25 | fun withRetrievalStrategy(strategy: RetrievalStrategy): ContextManagerBuilder { 26 | this.config = config.copy(retrievalStrategy = strategy) 27 | return this 28 | } 29 | 30 | fun addCustomRetriever(retriever: ContextRetriever): ContextManagerBuilder { 31 | customRetrievers.add(retriever) 32 | return this 33 | } 34 | 35 | fun withQueryRewriter(rewriter: QueryRewriter): ContextManagerBuilder { 36 | this.customQueryRewriter = rewriter 37 | return this 38 | } 39 | 40 | fun build(): ContextManager { 41 | val contextWindow = DefaultContextWindow(config.tokenBudget) 42 | 43 | val retriever = if (config.enableRetrieval) { 44 | when { 45 | customRetrievers.isNotEmpty() -> { 46 | if (customRetrievers.size == 1) { 47 | customRetrievers.first() 48 | } else { 49 | HybridRetriever(customRetrievers) 50 | } 51 | } 52 | 53 | config.retrievalStrategy == RetrievalStrategy.KEYWORD -> KeywordRetriever() 54 | config.retrievalStrategy == RetrievalStrategy.HYBRID -> { 55 | HybridRetriever(listOf(KeywordRetriever())) 56 | } 57 | 58 | else -> KeywordRetriever() 59 | } 60 | } else { 61 | null 62 | } 63 | 64 | val queryRewriter = if (config.enableQueryRewriting) { 65 | customQueryRewriter ?: SimpleCodeQueryRewriter() 66 | } else { 67 | SimpleCodeQueryRewriter() 68 | } 69 | 70 | return ContextManager(contextWindow, retriever, queryRewriter) 71 | } 72 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/retrieval/HybridRetriever.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.retrieval 2 | 3 | import com.phodal.ctxmesh.context.ContextContent 4 | import com.phodal.ctxmesh.context.ContextPriority 5 | 6 | class HybridRetriever( 7 | private val retrievers: List, 8 | private val weights: List = List(retrievers.size) { 1.0 / retrievers.size } 9 | ) : ContextRetriever { 10 | 11 | init { 12 | require(retrievers.size == weights.size) { "检索器数量必须与权重数量相等" } 13 | require(weights.sum() == 1.0) { "权重总和必须为1.0" } 14 | } 15 | 16 | override suspend fun retrieve(query: String, maxResults: Int): List { 17 | val allResults = mutableMapOf>() 18 | 19 | // 从所有检索器获取结果 20 | retrievers.forEachIndexed { index, retriever -> 21 | val results = retriever.retrieve(query, maxResults * 2) // 获取更多结果用于融合 22 | val weight = weights[index] 23 | 24 | results.forEach { content -> 25 | val currentScore = content.metadata["relevance_score"] as? Double ?: 0.5 26 | val weightedScore = currentScore * weight 27 | 28 | val existing = allResults[content.id] 29 | if (existing != null) { 30 | // 如果已存在，合并分数 31 | val newScore = existing.second + weightedScore 32 | allResults[content.id] = existing.first to newScore 33 | } else { 34 | allResults[content.id] = content to weightedScore 35 | } 36 | } 37 | } 38 | 39 | // 按合并后的分数排序并返回 40 | return allResults.values 41 | .sortedByDescending { it.second } 42 | .take(maxResults) 43 | .map { (content, score) -> 44 | content.copy( 45 | metadata = content.metadata + mapOf("hybrid_score" to score), 46 | priority = scoreToPriority(score) 47 | ) 48 | } 49 | } 50 | 51 | override suspend fun index(content: String, metadata: Map): String { 52 | // 在所有检索器中索引 53 | var lastId = "" 54 | retrievers.forEach { retriever -> 55 | lastId = retriever.index(content, metadata) 56 | } 57 | return lastId 58 | } 59 | 60 | override suspend fun remove(contentId: String): Boolean { 61 | return retrievers.map { it.remove(contentId) }.any { it } 62 | } 63 | 64 | private fun scoreToPriority(score: Double): ContextPriority { 65 | return when { 66 | score >= 0.8 -> ContextPriority.HIGHEST 67 | score >= 0.6 -> ContextPriority.HIGH 68 | score >= 0.4 -> ContextPriority.MEDIUM 69 | score >= 0.2 -> ContextPriority.LOW 70 | else -> ContextPriority.LOWEST 71 | } 72 | } 73 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/retrieval/KeywordRetriever.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context.retrieval 2 | 3 | import com.phodal.ctxmesh.context.ContextContent 4 | import com.phodal.ctxmesh.context.ContextPriority 5 | import com.phodal.ctxmesh.context.ContextType 6 | import com.phodal.ctxmesh.context.document.IndexedDocument 7 | import kotlinx.datetime.Clock 8 | 9 | /** 10 | * 简单的关键词检索器实现 11 | * 基于文本匹配的基础检索 12 | */ 13 | class KeywordRetriever : ContextRetriever { 14 | private val documents = mutableMapOf() 15 | 16 | override suspend fun retrieve(query: String, maxResults: Int): List { 17 | val queryTerms = query.lowercase().split("\\s+".toRegex()) 18 | 19 | val results = documents.values 20 | .map { doc -> doc to calculateRelevanceScore(doc, queryTerms) } 21 | .filter { it.second > 0.0 } 22 | .sortedByDescending { it.second } 23 | .take(maxResults) 24 | .map { (doc, score) -> 25 | ContextContent( 26 | id = doc.id, 27 | type = ContextType.EXTERNAL_KNOWLEDGE, 28 | content = doc.content, 29 | priority = scoreToPriority(score), 30 | metadata = doc.metadata + mapOf("relevance_score" to score) 31 | ) 32 | } 33 | 34 | return results 35 | } 36 | 37 | override suspend fun index(content: String, metadata: Map): String { 38 | val id = generateId() 39 | val doc = IndexedDocument( 40 | id = id, 41 | content = content, 42 | metadata = metadata, 43 | terms = extractTerms(content) 44 | ) 45 | documents[id] = doc 46 | return id 47 | } 48 | 49 | override suspend fun remove(contentId: String): Boolean { 50 | return documents.remove(contentId) != null 51 | } 52 | 53 | private fun calculateRelevanceScore(doc: IndexedDocument, queryTerms: List): Double { 54 | val matchingTerms = queryTerms.count { term -> 55 | doc.terms.any { it.contains(term) || term.contains(it) } 56 | } 57 | return matchingTerms.toDouble() / queryTerms.size 58 | } 59 | 60 | private fun extractTerms(content: String): Set { 61 | return content.lowercase() 62 | .split("\\W+".toRegex()) 63 | .filter { it.length > 2 } 64 | .toSet() 65 | } 66 | 67 | private fun scoreToPriority(score: Double): ContextPriority { 68 | return when { 69 | score >= 0.8 -> ContextPriority.HIGHEST 70 | score >= 0.6 -> ContextPriority.HIGH 71 | score >= 0.4 -> ContextPriority.MEDIUM 72 | score >= 0.2 -> ContextPriority.LOW 73 | else -> ContextPriority.LOWEST 74 | } 75 | } 76 | 77 | private fun generateId(): String { 78 | return "doc_${Clock.System.now().epochSeconds}_${(0..999).random()}" 79 | } 80 | } -------------------------------------------------------------------------------- /CtxMesh/gradlew.bat: -------------------------------------------------------------------------------- 1 | @rem 2 | @rem Copyright 2015 the original author or authors. 3 | @rem 4 | @rem Licensed under the Apache License, Version 2.0 (the "License"); 5 | @rem you may not use this file except in compliance with the License. 6 | @rem You may obtain a copy of the License at 7 | @rem 8 | @rem https://www.apache.org/licenses/LICENSE-2.0 9 | @rem 10 | @rem Unless required by applicable law or agreed to in writing, software 11 | @rem distributed under the License is distributed on an "AS IS" BASIS, 12 | @rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | @rem See the License for the specific language governing permissions and 14 | @rem limitations under the License. 15 | @rem 16 | @rem SPDX-License-Identifier: Apache-2.0 17 | @rem 18 | 19 | @if "%DEBUG%"=="" @echo off 20 | @rem ########################################################################## 21 | @rem 22 | @rem Gradle startup script for Windows 23 | @rem 24 | @rem ########################################################################## 25 | 26 | @rem Set local scope for the variables with windows NT shell 27 | if "%OS%"=="Windows_NT" setlocal 28 | 29 | set DIRNAME=%~dp0 30 | if "%DIRNAME%"=="" set DIRNAME=. 31 | @rem This is normally unused 32 | set APP_BASE_NAME=%~n0 33 | set APP_HOME=%DIRNAME% 34 | 35 | @rem Resolve any "." and ".." in APP_HOME to make it shorter. 36 | for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi 37 | 38 | @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 39 | set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" 40 | 41 | @rem Find java.exe 42 | if defined JAVA_HOME goto findJavaFromJavaHome 43 | 44 | set JAVA_EXE=java.exe 45 | %JAVA_EXE% -version >NUL 2>&1 46 | if %ERRORLEVEL% equ 0 goto execute 47 | 48 | echo. 1>&2 49 | echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 50 | echo. 1>&2 51 | echo Please set the JAVA_HOME variable in your environment to match the 1>&2 52 | echo location of your Java installation. 1>&2 53 | 54 | goto fail 55 | 56 | :findJavaFromJavaHome 57 | set JAVA_HOME=%JAVA_HOME:"=% 58 | set JAVA_EXE=%JAVA_HOME%/bin/java.exe 59 | 60 | if exist "%JAVA_EXE%" goto execute 61 | 62 | echo. 1>&2 63 | echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 64 | echo. 1>&2 65 | echo Please set the JAVA_HOME variable in your environment to match the 1>&2 66 | echo location of your Java installation. 1>&2 67 | 68 | goto fail 69 | 70 | :execute 71 | @rem Setup the command line 72 | 73 | set CLASSPATH= 74 | 75 | 76 | @rem Execute Gradle 77 | "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* 78 | 79 | :end 80 | @rem End local scope for the variables with windows NT shell 81 | if %ERRORLEVEL% equ 0 goto mainEnd 82 | 83 | :fail 84 | rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of 85 | rem the _cmd.exe /c_ return code! 86 | set EXIT_CODE=%ERRORLEVEL% 87 | if %EXIT_CODE% equ 0 set EXIT_CODE=1 88 | if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE% 89 | exit /b %EXIT_CODE% 90 | 91 | :mainEnd 92 | if "%OS%"=="Windows_NT" endlocal 93 | 94 | :omega 95 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextFactory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | import com.phodal.ctxmesh.context.retrieval.* 4 | 5 | object ContextFactory { 6 | 7 | /** 8 | * 创建基础的上下文管理器（无检索功能） 9 | */ 10 | fun createBasic(tokenBudget: Int = 8192): ContextManager { 11 | return ContextManager( 12 | contextWindow = DefaultContextWindow(tokenBudget) 13 | ) 14 | } 15 | 16 | /** 17 | * 创建带关键词检索的上下文管理器 18 | */ 19 | fun createWithKeywordRetrieval(tokenBudget: Int = 8192): ContextManager { 20 | return ContextManager( 21 | contextWindow = DefaultContextWindow(tokenBudget), 22 | retriever = KeywordRetriever() 23 | ) 24 | } 25 | 26 | /** 27 | * 创建带混合检索的上下文管理器 28 | */ 29 | fun createWithHybridRetrieval( 30 | tokenBudget: Int = 8192, 31 | retrievers: List? = null, 32 | weights: List? = null 33 | ): ContextManager { 34 | val defaultRetrievers = retrievers ?: listOf( 35 | KeywordRetriever() 36 | // 可以在这里添加更多检索器 37 | ) 38 | 39 | val hybridRetriever = if (defaultRetrievers.size > 1) { 40 | HybridRetriever(defaultRetrievers, weights ?: List(defaultRetrievers.size) { 1.0 / defaultRetrievers.size }) 41 | } else { 42 | defaultRetrievers.first() 43 | } 44 | 45 | return ContextManager( 46 | contextWindow = DefaultContextWindow(tokenBudget), 47 | retriever = hybridRetriever 48 | ) 49 | } 50 | 51 | /** 52 | * 创建用于代码场景的上下文管理器 53 | */ 54 | fun createForCoding(tokenBudget: Int = 16384): ContextManager { 55 | val manager = createWithKeywordRetrieval(tokenBudget) 56 | 57 | // 预设一些代码相关的系统提示词模板 58 | manager.addLongTermMemory( 59 | """ 60 | You are an expert software developer. When analyzing code: 61 | 1. Focus on code structure, patterns, and best practices 62 | 2. Consider performance, maintainability, and readability 63 | 3. Provide specific, actionable suggestions 64 | 4. Reference relevant documentation when available 65 | """.trimIndent(), 66 | mapOf("type" to "coding_guidelines") 67 | ) 68 | 69 | return manager 70 | } 71 | 72 | /** 73 | * 创建用于文档问答的上下文管理器 74 | */ 75 | fun createForDocumentQA(tokenBudget: Int = 12288): ContextManager { 76 | val manager = createWithKeywordRetrieval(tokenBudget) 77 | 78 | // 预设文档问答的系统提示词 79 | manager.addLongTermMemory( 80 | """ 81 | You are a helpful assistant that answers questions based on provided documentation. 82 | Always: 83 | 1. Base your answers on the provided context 84 | 2. Cite specific sections when possible 85 | 3. Acknowledge when information is not available in the context 86 | 4. Provide clear, structured responses 87 | """.trimIndent(), 88 | mapOf("type" to "qa_guidelines") 89 | ) 90 | 91 | return manager 92 | } 93 | } 94 | 95 | -------------------------------------------------------------------------------- /CtxMesh/README.md: -------------------------------------------------------------------------------- 1 | This is a Kotlin Multiplatform project targeting Web, Desktop (JVM), Server. 2 | 3 | * [/composeApp](./composeApp/src) is for code that will be shared across your Compose Multiplatform applications. 4 | It contains several subfolders: 5 | - [commonMain](./composeApp/src/commonMain/kotlin) is for code that’s common for all targets. 6 | - Other folders are for Kotlin code that will be compiled for only the platform indicated in the folder name. 7 | For example, if you want to use Apple’s CoreCrypto for the iOS part of your Kotlin app, 8 | the [iosMain](./composeApp/src/iosMain/kotlin) folder would be the right place for such calls. 9 | Similarly, if you want to edit the Desktop (JVM) specific part, the [jvmMain](./composeApp/src/jvmMain/kotlin) 10 | folder is the appropriate location. 11 | 12 | * [/server](./server/src/main/kotlin) is for the Ktor server application. 13 | 14 | * [/shared](./shared/src) is for the code that will be shared between all targets in the project. 15 | The most important subfolder is [commonMain](./shared/src/commonMain/kotlin). If preferred, you 16 | can add code to the platform-specific folders here too. 17 | 18 | ### Build and Run Desktop (JVM) Application 19 | 20 | To build and run the development version of the desktop app, use the run configuration from the run widget 21 | in your IDE’s toolbar or run it directly from the terminal: 22 | 23 | - on macOS/Linux 24 | ```shell 25 | ./gradlew :composeApp:run 26 | ``` 27 | - on Windows 28 | ```shell 29 | .\gradlew.bat :composeApp:run 30 | ``` 31 | 32 | ### Build and Run Server 33 | 34 | To build and run the development version of the server, use the run configuration from the run widget 35 | in your IDE’s toolbar or run it directly from the terminal: 36 | 37 | - on macOS/Linux 38 | ```shell 39 | ./gradlew :server:run 40 | ``` 41 | - on Windows 42 | ```shell 43 | .\gradlew.bat :server:run 44 | ``` 45 | 46 | ### Build and Run Web Application 47 | 48 | To build and run the development version of the web app, use the run configuration from the run widget 49 | in your IDE's toolbar or run it directly from the terminal: 50 | 51 | - for the Wasm target (faster, modern browsers): 52 | - on macOS/Linux 53 | ```shell 54 | ./gradlew :composeApp:wasmJsBrowserDevelopmentRun 55 | ``` 56 | - on Windows 57 | ```shell 58 | .\gradlew.bat :composeApp:wasmJsBrowserDevelopmentRun 59 | ``` 60 | - for the JS target (slower, supports older browsers): 61 | - on macOS/Linux 62 | ```shell 63 | ./gradlew :composeApp:jsBrowserDevelopmentRun 64 | ``` 65 | - on Windows 66 | ```shell 67 | .\gradlew.bat :composeApp:jsBrowserDevelopmentRun 68 | ``` 69 | 70 | --- 71 | 72 | Learn more about [Kotlin Multiplatform](https://www.jetbrains.com/help/kotlin-multiplatform-dev/get-started.html), 73 | [Compose Multiplatform](https://github.com/JetBrains/compose-multiplatform/#compose-multiplatform), 74 | [Kotlin/Wasm](https://kotl.in/wasm/)… 75 | 76 | We would appreciate your feedback on Compose/Web and Kotlin/Wasm in the public Slack 77 | channel [#compose-web](https://slack-chats.kotlinlang.org/c/compose-web). 78 | If you face any issues, please report them on [YouTrack](https://youtrack.jetbrains.com/newIssue?project=CMP). -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/DefaultContextWindow.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | /** 4 | * 默认的上下文窗口实现 5 | * 实现了基本的优先级管理和 token 预算控制 6 | */ 7 | class DefaultContextWindow( 8 | override val tokenBudget: Int = 8192 9 | ) : ContextWindow { 10 | 11 | private val contexts = mutableMapOf() 12 | 13 | override val usedTokens: Int 14 | get() = contexts.values.sumOf { it.estimatedTokens } 15 | 16 | override fun addContext(content: ContextContent): Boolean { 17 | // 检查是否超出 token 预算 18 | val newTokenCount = usedTokens + content.estimatedTokens 19 | if (newTokenCount > tokenBudget) { 20 | // 尝试自动截断低优先级内容 21 | if (!autoTruncate(content.estimatedTokens)) { 22 | return false 23 | } 24 | } 25 | 26 | contexts[content.id] = content 27 | return true 28 | } 29 | 30 | override fun removeContext(contentId: String): Boolean { 31 | return contexts.remove(contentId) != null 32 | } 33 | 34 | override fun assembleContext(): String { 35 | // 按优先级和时间戳排序 36 | val sortedContexts = contexts.values.sortedWith( 37 | compareByDescending { it.priority.weight } 38 | .thenByDescending { it.timestamp } 39 | ) 40 | 41 | val result = StringBuilder() 42 | var currentTokens = 0 43 | 44 | for (context in sortedContexts) { 45 | if (currentTokens + context.estimatedTokens <= tokenBudget) { 46 | result.append(formatContextContent(context)) 47 | result.append("\n\n") 48 | currentTokens += context.estimatedTokens 49 | } 50 | } 51 | 52 | return result.toString().trim() 53 | } 54 | 55 | override fun clear() { 56 | contexts.clear() 57 | } 58 | 59 | override fun getAllContexts(): List { 60 | return contexts.values.toList() 61 | } 62 | 63 | /** 64 | * 自动截断低优先级内容以腾出空间 65 | */ 66 | private fun autoTruncate(requiredTokens: Int): Boolean { 67 | val sortedByPriority = contexts.values.sortedBy { it.priority.weight } 68 | var freedTokens = 0 69 | val toRemove = mutableListOf() 70 | 71 | for (context in sortedByPriority) { 72 | if (freedTokens >= requiredTokens) break 73 | 74 | toRemove.add(context.id) 75 | freedTokens += context.estimatedTokens 76 | } 77 | 78 | toRemove.forEach { contexts.remove(it) } 79 | return freedTokens >= requiredTokens 80 | } 81 | 82 | /** 83 | * 格式化上下文内容 84 | */ 85 | private fun formatContextContent(context: ContextContent): String { 86 | return when (context.type) { 87 | ContextType.SYSTEM_PROMPT -> "# System Prompt\n${context.content}" 88 | ContextType.OUTPUT_FORMAT -> "# Output Format\n${context.content}" 89 | ContextType.TOOL_CONTEXT -> "# Available Tools\n${context.content}" 90 | ContextType.EXTERNAL_KNOWLEDGE -> "# Knowledge Base\n${context.content}" 91 | ContextType.SHORT_TERM_MEMORY -> "# Recent Context\n${context.content}" 92 | ContextType.LONG_TERM_MEMORY -> "# Background Knowledge\n${context.content}" 93 | ContextType.GLOBAL_STATE -> "# Current State\n${context.content}" 94 | ContextType.USER_INPUT -> "# User Request\n${context.content}" 95 | } 96 | } 97 | } -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/memory/Memory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory 2 | 3 | import com.phodal.ctxmesh.context.ContextContent 4 | import com.phodal.ctxmesh.context.ContextPriority 5 | import com.phodal.ctxmesh.context.ContextType 6 | import kotlinx.datetime.Instant 7 | 8 | /** 9 | * Memory 系统的核心接口 10 | * 基于 README.md 中的记忆系统设计理念，提供统一的记忆管理抽象 11 | */ 12 | interface Memory { 13 | /** 14 | * 存储记忆内容 15 | * @param content 要存储的内容 16 | * @param metadata 元数据信息 17 | * @return 存储成功返回记忆ID，失败返回null 18 | */ 19 | suspend fun store(content: String, metadata: Map = emptyMap()): String? 20 | 21 | /** 22 | * 根据查询检索相关记忆 23 | * @param query 查询内容 24 | * @param maxResults 最大返回结果数 25 | * @param threshold 相关性阈值 26 | * @return 检索到的记忆列表，按相关性排序 27 | */ 28 | suspend fun retrieve(query: String, maxResults: Int = 5, threshold: Double = 0.0): List 29 | 30 | /** 31 | * 根据ID获取特定记忆 32 | * @param memoryId 记忆ID 33 | * @return 记忆项，不存在返回null 34 | */ 35 | suspend fun get(memoryId: String): MemoryItem? 36 | 37 | /** 38 | * 更新记忆内容 39 | * @param memoryId 记忆ID 40 | * @param content 新内容 41 | * @param metadata 新元数据 42 | * @return 更新是否成功 43 | */ 44 | suspend fun update(memoryId: String, content: String, metadata: Map = emptyMap()): Boolean 45 | 46 | /** 47 | * 删除记忆 48 | * @param memoryId 记忆ID 49 | * @return 删除是否成功 50 | */ 51 | suspend fun delete(memoryId: String): Boolean 52 | 53 | /** 54 | * 清理过期或不相关的记忆 55 | * @param criteria 清理条件 56 | * @return 清理的记忆数量 57 | */ 58 | suspend fun cleanup(criteria: CleanupCriteria = CleanupCriteria()): Int 59 | 60 | /** 61 | * 获取记忆统计信息 62 | * @return 记忆统计 63 | */ 64 | suspend fun getStats(): MemoryStats 65 | 66 | /** 67 | * 清空所有记忆 68 | */ 69 | suspend fun clear() 70 | } 71 | 72 | /** 73 | * 记忆项数据类 74 | */ 75 | data class MemoryItem( 76 | val id: String, 77 | val content: String, 78 | val metadata: Map = emptyMap(), 79 | val timestamp: Instant, 80 | val lastAccessed: Instant = timestamp, 81 | val accessCount: Int = 0, 82 | val relevanceScore: Double = 0.0 83 | ) { 84 | /** 85 | * 转换为 ContextContent 86 | */ 87 | fun toContextContent(type: ContextType, priority: ContextPriority): ContextContent { 88 | return ContextContent( 89 | id = id, 90 | type = type, 91 | content = content, 92 | priority = priority, 93 | metadata = metadata, 94 | timestamp = timestamp.toEpochMilliseconds() 95 | ) 96 | } 97 | } 98 | 99 | /** 100 | * 清理条件 101 | */ 102 | data class CleanupCriteria( 103 | val maxAge: Long? = null, // 最大年龄（毫秒） 104 | val minAccessCount: Int? = null, // 最小访问次数 105 | val maxItems: Int? = null, // 最大保留项数 106 | val relevanceThreshold: Double? = null // 相关性阈值 107 | ) 108 | 109 | /** 110 | * 记忆统计信息 111 | */ 112 | data class MemoryStats( 113 | val totalItems: Int, 114 | val totalSize: Long, // 字节 115 | val oldestItem: Instant?, 116 | val newestItem: Instant?, 117 | val averageAccessCount: Double, 118 | val memoryType: MemoryType 119 | ) 120 | 121 | /** 122 | * 记忆类型枚举 123 | */ 124 | enum class MemoryType { 125 | /** 短期记忆 - 会话级别，自动过期 */ 126 | SHORT_TERM, 127 | 128 | /** 长期记忆 - 持久化存储，语义检索 */ 129 | LONG_TERM, 130 | 131 | /** 工作记忆 - 临时计算状态 */ 132 | WORKING, 133 | 134 | /** 情景记忆 - 特定事件或对话 */ 135 | EPISODIC, 136 | 137 | /** 语义记忆 - 结构化知识 */ 138 | SEMANTIC 139 | } 140 | 141 | /** 142 | * 记忆事件接口 143 | */ 144 | interface MemoryEvent { 145 | val timestamp: Instant 146 | val memoryId: String 147 | val eventType: MemoryEventType 148 | } 149 | 150 | /** 151 | * 记忆事件类型 152 | */ 153 | enum class MemoryEventType { 154 | STORED, 155 | RETRIEVED, 156 | UPDATED, 157 | DELETED, 158 | ACCESSED, 159 | EXPIRED 160 | } 161 | 162 | /** 163 | * 记忆监听器接口 164 | */ 165 | interface MemoryListener { 166 | suspend fun onMemoryEvent(event: MemoryEvent) 167 | } 168 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonTest/kotlin/com/phodal/ctxmesh/context/ContextManagerTest.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | import kotlin.test.Test 4 | import kotlin.test.assertTrue 5 | import kotlin.test.assertEquals 6 | 7 | class ContextManagerTest { 8 | 9 | @Test 10 | fun testBasicContextBuilding() { 11 | val manager = ContextFactory.createBasic() 12 | 13 | // 简化测试，不使用协程 14 | manager.addShortTermMemory("What is Kotlin?") 15 | 16 | val status = manager.getWindowStatus() 17 | assertTrue(status.contextCount > 0) 18 | assertTrue(status.usedTokens > 0) 19 | } 20 | 21 | @Test 22 | fun testContextWithRetrieval() { 23 | val manager = ContextFactory.createWithKeywordRetrieval() 24 | 25 | // 测试检索器是否正确创建 26 | val status = manager.getWindowStatus() 27 | assertEquals(0, status.contextCount) // 初始应该为空 28 | } 29 | 30 | @Test 31 | fun testMemoryManagement() { 32 | val manager = ContextFactory.createBasic() 33 | 34 | // 添加长期记忆 35 | manager.addLongTermMemory( 36 | "Remember: Always follow coding best practices", 37 | mapOf("category" to "guidelines") 38 | ) 39 | 40 | // 添加短期记忆 41 | manager.addShortTermMemory( 42 | "User is working on a web application project", 43 | mapOf("context" to "current_session") 44 | ) 45 | 46 | val status = manager.getWindowStatus() 47 | assertEquals(2, status.contextCount) 48 | assertTrue(status.contextsByType.containsKey(ContextType.LONG_TERM_MEMORY)) 49 | assertTrue(status.contextsByType.containsKey(ContextType.SHORT_TERM_MEMORY)) 50 | } 51 | 52 | @Test 53 | fun testToolContext() { 54 | val manager = ContextFactory.createBasic() 55 | 56 | manager.addToolContext( 57 | toolName = "file_reader", 58 | toolDescription = "Reads content from files", 59 | toolSchema = "{ \"type\": \"function\", \"parameters\": { \"path\": \"string\" } }" 60 | ) 61 | 62 | val status = manager.getWindowStatus() 63 | assertEquals(1, status.contextCount) 64 | assertTrue(status.contextsByType.containsKey(ContextType.TOOL_CONTEXT)) 65 | } 66 | 67 | @Test 68 | fun testGlobalStateManagement() { 69 | val manager = ContextFactory.createBasic() 70 | 71 | // 设置初始状态 72 | manager.updateGlobalState( 73 | "Current project: E-commerce website, Phase: Development", 74 | mapOf("project" to "ecommerce", "phase" to "dev") 75 | ) 76 | 77 | val status1 = manager.getWindowStatus() 78 | assertEquals(1, status1.contextCount) 79 | assertTrue(status1.contextsByType.containsKey(ContextType.GLOBAL_STATE)) 80 | 81 | // 更新状态 82 | manager.updateGlobalState( 83 | "Current project: E-commerce website, Phase: Testing", 84 | mapOf("project" to "ecommerce", "phase" to "test") 85 | ) 86 | 87 | val status2 = manager.getWindowStatus() 88 | assertEquals(1, status2.contextCount) // 应该还是1个，因为旧状态被替换 89 | } 90 | 91 | @Test 92 | fun testWindowStatusTracking() { 93 | val manager = ContextFactory.createBasic(tokenBudget = 1000) 94 | 95 | manager.addLongTermMemory("Some long term memory content") 96 | manager.addShortTermMemory("Some short term memory content") 97 | 98 | val status = manager.getWindowStatus() 99 | 100 | assertEquals(1000, status.totalTokenBudget) 101 | assertTrue(status.usedTokens > 0) 102 | assertTrue(status.remainingTokens < 1000) 103 | assertEquals(2, status.contextCount) 104 | assertTrue(status.contextsByType.containsKey(ContextType.LONG_TERM_MEMORY)) 105 | assertTrue(status.contextsByType.containsKey(ContextType.SHORT_TERM_MEMORY)) 106 | } 107 | 108 | 109 | 110 | @Test 111 | fun testCodingScenario() { 112 | val manager = ContextFactory.createForCoding() 113 | 114 | val status = manager.getWindowStatus() 115 | // 编码场景应该有预设的长期记忆 116 | assertTrue(status.contextCount > 0) 117 | assertTrue(status.contextsByType.containsKey(ContextType.LONG_TERM_MEMORY)) 118 | } 119 | 120 | @Test 121 | fun testDocumentQAScenario() { 122 | val manager = ContextFactory.createForDocumentQA() 123 | 124 | val status = manager.getWindowStatus() 125 | // 文档问答场景应该有预设的长期记忆 126 | assertTrue(status.contextCount > 0) 127 | assertTrue(status.contextsByType.containsKey(ContextType.LONG_TERM_MEMORY)) 128 | } 129 | } 130 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonTest/kotlin/com/phodal/ctxmesh/context/ContextWindowTest.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | import kotlin.test.Test 4 | import kotlin.test.assertEquals 5 | import kotlin.test.assertTrue 6 | import kotlin.test.assertFalse 7 | 8 | class ContextWindowTest { 9 | 10 | @Test 11 | fun testBasicContextWindow() { 12 | val window = DefaultContextWindow(tokenBudget = 1000) 13 | 14 | val content = ContextContent( 15 | id = "test1", 16 | type = ContextType.SYSTEM_PROMPT, 17 | content = "You are a helpful assistant.", 18 | priority = ContextPriority.HIGHEST 19 | ) 20 | 21 | assertTrue(window.addContext(content)) 22 | assertEquals(1, window.getAllContexts().size) 23 | assertTrue(window.usedTokens > 0) 24 | } 25 | 26 | @Test 27 | fun testTokenBudgetEnforcement() { 28 | val window = DefaultContextWindow(tokenBudget = 100) // 很小的预算 29 | 30 | val largeContent = ContextContent( 31 | id = "large", 32 | type = ContextType.EXTERNAL_KNOWLEDGE, 33 | content = "A".repeat(500), // 大约125个token 34 | priority = ContextPriority.LOW 35 | ) 36 | 37 | // 应该因为超出预算而失败 38 | assertFalse(window.addContext(largeContent)) 39 | } 40 | 41 | @Test 42 | fun testPriorityOrdering() { 43 | val window = DefaultContextWindow(tokenBudget = 2000) 44 | 45 | val lowPriority = ContextContent( 46 | id = "low", 47 | type = ContextType.EXTERNAL_KNOWLEDGE, 48 | content = "Low priority content", 49 | priority = ContextPriority.LOW 50 | ) 51 | 52 | val highPriority = ContextContent( 53 | id = "high", 54 | type = ContextType.SYSTEM_PROMPT, 55 | content = "High priority content", 56 | priority = ContextPriority.HIGHEST 57 | ) 58 | 59 | window.addContext(lowPriority) 60 | window.addContext(highPriority) 61 | 62 | val assembled = window.assembleContext() 63 | val highIndex = assembled.indexOf("High priority content") 64 | val lowIndex = assembled.indexOf("Low priority content") 65 | 66 | assertTrue(highIndex < lowIndex, "高优先级内容应该出现在前面") 67 | } 68 | 69 | @Test 70 | fun testAutoTruncation() { 71 | val window = DefaultContextWindow(tokenBudget = 200) 72 | 73 | // 添加一些低优先级内容 74 | repeat(3) { i -> 75 | window.addContext( 76 | ContextContent( 77 | id = "low_$i", 78 | type = ContextType.EXTERNAL_KNOWLEDGE, 79 | content = "Low priority content $i", 80 | priority = ContextPriority.LOW 81 | ) 82 | ) 83 | } 84 | 85 | // 添加高优先级内容，应该触发自动截断 86 | val highPriorityContent = ContextContent( 87 | id = "high", 88 | type = ContextType.SYSTEM_PROMPT, 89 | content = "A".repeat(150), // 大约37个token 90 | priority = ContextPriority.HIGHEST 91 | ) 92 | 93 | assertTrue(window.addContext(highPriorityContent)) 94 | 95 | // 验证高优先级内容存在 96 | val contexts = window.getAllContexts() 97 | assertTrue(contexts.any { it.id == "high" }) 98 | } 99 | 100 | @Test 101 | fun testContextRemoval() { 102 | val window = DefaultContextWindow() 103 | 104 | val content = ContextContent( 105 | id = "removable", 106 | type = ContextType.USER_INPUT, 107 | content = "This will be removed", 108 | priority = ContextPriority.MEDIUM 109 | ) 110 | 111 | window.addContext(content) 112 | assertEquals(1, window.getAllContexts().size) 113 | 114 | assertTrue(window.removeContext("removable")) 115 | assertEquals(0, window.getAllContexts().size) 116 | 117 | assertFalse(window.removeContext("nonexistent")) 118 | } 119 | 120 | @Test 121 | fun testContextFormatting() { 122 | val window = DefaultContextWindow() 123 | 124 | val systemPrompt = ContextContent( 125 | id = "system", 126 | type = ContextType.SYSTEM_PROMPT, 127 | content = "You are helpful", 128 | priority = ContextPriority.HIGHEST 129 | ) 130 | 131 | val userInput = ContextContent( 132 | id = "user", 133 | type = ContextType.USER_INPUT, 134 | content = "Hello world", 135 | priority = ContextPriority.HIGH 136 | ) 137 | 138 | window.addContext(systemPrompt) 139 | window.addContext(userInput) 140 | 141 | val assembled = window.assembleContext() 142 | 143 | assertTrue(assembled.contains("# System Prompt")) 144 | assertTrue(assembled.contains("# User Request")) 145 | assertTrue(assembled.contains("You are helpful")) 146 | assertTrue(assembled.contains("Hello world")) 147 | } 148 | } 149 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/memory/impl/DefaultShortMemory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory.impl 2 | 3 | import com.phodal.ctxmesh.memory.MemoryItem 4 | import com.phodal.ctxmesh.memory.ShortMemory 5 | import com.phodal.ctxmesh.memory.ShortMemoryConfig 6 | import kotlinx.datetime.Clock 7 | import kotlin.math.max 8 | import kotlin.random.Random 9 | 10 | /** 11 | * 默认短期记忆实现 12 | * 13 | * 提供基于内存的短期记忆存储，适用于会话级别的上下文管理 14 | */ 15 | class DefaultShortMemory( 16 | config: ShortMemoryConfig = ShortMemoryConfig() 17 | ) : ShortMemory() { 18 | 19 | override val config = config 20 | 21 | override fun generateMemoryId(): String { 22 | val timestamp = Clock.System.now().toEpochMilliseconds() 23 | val random = Random.nextInt(1000, 9999) 24 | return "short_${timestamp}_$random" 25 | } 26 | 27 | override fun calculateRelevanceScore(query: String, item: MemoryItem): Double { 28 | val queryWords = query.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 29 | val contentWords = item.content.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 30 | 31 | if (queryWords.isEmpty() || contentWords.isEmpty()) { 32 | return 0.0 33 | } 34 | 35 | // 计算词汇重叠度 36 | val intersection = queryWords.intersect(contentWords.toSet()) 37 | val union = queryWords.union(contentWords.toSet()) 38 | val jaccardSimilarity = if (union.isNotEmpty()) intersection.size.toDouble() / union.size else 0.0 39 | 40 | // 计算精确匹配分数 41 | val exactMatches = queryWords.count { queryWord -> 42 | contentWords.any { contentWord -> contentWord.contains(queryWord) } 43 | } 44 | val exactMatchScore = exactMatches.toDouble() / queryWords.size 45 | 46 | // 计算位置权重（查询词在内容中的位置） 47 | val positionScore = calculatePositionScore(query, item.content) 48 | 49 | // 计算元数据匹配分数 50 | val metadataScore = calculateMetadataScore(query, item) 51 | 52 | // 综合评分 53 | return (jaccardSimilarity * 0.4 + 54 | exactMatchScore * 0.3 + 55 | positionScore * 0.2 + 56 | metadataScore * 0.1).coerceIn(0.0, 1.0) 57 | } 58 | 59 | /** 60 | * 计算位置权重分数 61 | * 查询词在内容开头出现的权重更高 62 | */ 63 | private fun calculatePositionScore(query: String, content: String): Double { 64 | val queryWords = query.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 65 | val contentLower = content.lowercase() 66 | 67 | var totalScore = 0.0 68 | var foundWords = 0 69 | 70 | queryWords.forEach { word -> 71 | val index = contentLower.indexOf(word) 72 | if (index >= 0) { 73 | foundWords++ 74 | // 位置越靠前，分数越高 75 | val positionWeight = max(0.1, 1.0 - (index.toDouble() / content.length)) 76 | totalScore += positionWeight 77 | } 78 | } 79 | 80 | return if (foundWords > 0) totalScore / queryWords.size else 0.0 81 | } 82 | 83 | /** 84 | * 计算元数据匹配分数 85 | */ 86 | private fun calculateMetadataScore(query: String, item: MemoryItem): Double { 87 | val queryLower = query.lowercase() 88 | var score = 0.0 89 | 90 | // 检查标签匹配 91 | item.metadata["tags"]?.let { tags -> 92 | when (tags) { 93 | is List<*> -> { 94 | val tagMatches = tags.filterIsInstance() 95 | .count { tag -> queryLower.contains(tag.lowercase()) } 96 | if (tags.isNotEmpty()) { 97 | score += (tagMatches.toDouble() / tags.size) * 0.5 98 | } 99 | } 100 | is String -> { 101 | if (queryLower.contains(tags.lowercase())) { 102 | score += 0.5 103 | } 104 | } 105 | } 106 | } 107 | 108 | // 检查类型匹配 109 | item.metadata["type"]?.let { type -> 110 | if (queryLower.contains(type.toString().lowercase())) { 111 | score += 0.3 112 | } 113 | } 114 | 115 | // 检查上下文匹配 116 | item.metadata["context"]?.let { context -> 117 | if (queryLower.contains(context.toString().lowercase())) { 118 | score += 0.2 119 | } 120 | } 121 | 122 | return score.coerceIn(0.0, 1.0) 123 | } 124 | } 125 | 126 | /** 127 | * 短期记忆工厂类 128 | */ 129 | object ShortMemoryFactory { 130 | 131 | /** 132 | * 创建默认的短期记忆实例 133 | */ 134 | fun createDefault(): DefaultShortMemory { 135 | return DefaultShortMemory() 136 | } 137 | 138 | /** 139 | * 创建自定义配置的短期记忆实例 140 | */ 141 | fun createWithConfig(config: ShortMemoryConfig): DefaultShortMemory { 142 | return DefaultShortMemory(config) 143 | } 144 | 145 | /** 146 | * 创建适用于编程场景的短期记忆实例 147 | */ 148 | fun createForCoding(): DefaultShortMemory { 149 | val config = ShortMemoryConfig( 150 | maxCapacity = 500, 151 | maxAge = 7200000, // 2小时 152 | decayRate = 1.5, 153 | relevanceWeight = 0.6, 154 | temporalWeight = 0.2, 155 | accessWeight = 0.2 156 | ) 157 | return DefaultShortMemory(config) 158 | } 159 | 160 | /** 161 | * 创建适用于对话场景的短期记忆实例 162 | */ 163 | fun createForConversation(): DefaultShortMemory { 164 | val config = ShortMemoryConfig( 165 | maxCapacity = 200, 166 | maxAge = 1800000, // 30分钟 167 | decayRate = 3.0, 168 | relevanceWeight = 0.4, 169 | temporalWeight = 0.4, 170 | accessWeight = 0.2 171 | ) 172 | return DefaultShortMemory(config) 173 | } 174 | 175 | /** 176 | * 创建适用于文档问答场景的短期记忆实例 177 | */ 178 | fun createForDocumentQA(): DefaultShortMemory { 179 | val config = ShortMemoryConfig( 180 | maxCapacity = 300, 181 | maxAge = 3600000, // 1小时 182 | decayRate = 2.0, 183 | relevanceWeight = 0.7, 184 | temporalWeight = 0.2, 185 | accessWeight = 0.1 186 | ) 187 | return DefaultShortMemory(config) 188 | } 189 | } 190 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/jvmMain/kotlin/com/phodal/ctxmesh/ContextManagerWithMemory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh 2 | 3 | import com.phodal.ctxmesh.context.ContextContent 4 | import com.phodal.ctxmesh.context.ContextPriority 5 | import com.phodal.ctxmesh.context.ContextType 6 | import com.phodal.ctxmesh.context.ContextWindow 7 | import com.phodal.ctxmesh.context.ContextWindowStatus 8 | import com.phodal.ctxmesh.context.DefaultContextWindow 9 | import com.phodal.ctxmesh.context.retrieval.ContextRetriever 10 | import com.phodal.ctxmesh.context.retrieval.rewrite.QueryRewriter 11 | import com.phodal.ctxmesh.context.retrieval.rewrite.SimpleCodeQueryRewriter 12 | import com.phodal.ctxmesh.memory.CleanupCriteria 13 | import com.phodal.ctxmesh.memory.MemoryCleanupResult 14 | import com.phodal.ctxmesh.memory.MemoryItem 15 | import com.phodal.ctxmesh.memory.MemoryManager 16 | import com.phodal.ctxmesh.memory.MemoryManagerStats 17 | import kotlinx.coroutines.runBlocking 18 | 19 | /** 20 | * 带记忆系统的上下文管理器 21 | * 22 | * 扩展了基础的 ContextManager，集成了 MemoryManager 来提供智能的记忆管理功能 23 | */ 24 | class ContextManagerWithMemory( 25 | private val contextWindow: ContextWindow = DefaultContextWindow(), 26 | private val retriever: ContextRetriever? = null, 27 | private val queryRewriter: QueryRewriter = SimpleCodeQueryRewriter(), 28 | private val memoryManager: MemoryManager 29 | ) { 30 | 31 | /** 32 | * 基于查询构建上下文窗口（带记忆增强） 33 | */ 34 | fun buildContextForQuery( 35 | query: String, 36 | systemPrompt: String? = null, 37 | outputFormat: String? = null, 38 | maxRetrievalResults: Int = 3, 39 | maxMemoryResults: Int = 5 40 | ): String = runBlocking { 41 | // 清空现有上下文 42 | contextWindow.clear() 43 | 44 | // 1. 添加系统提示词（最高优先级） 45 | systemPrompt?.let { 46 | contextWindow.addContext( 47 | ContextContent( 48 | id = "system_prompt", 49 | type = ContextType.SYSTEM_PROMPT, 50 | content = it, 51 | priority = ContextPriority.HIGHEST 52 | ) 53 | ) 54 | } 55 | 56 | // 2. 添加输出格式说明 57 | outputFormat?.let { 58 | contextWindow.addContext( 59 | ContextContent( 60 | id = "output_format", 61 | type = ContextType.OUTPUT_FORMAT, 62 | content = it, 63 | priority = ContextPriority.HIGH 64 | ) 65 | ) 66 | } 67 | 68 | // 3. 从记忆系统检索相关内容 69 | val memoryResults = memoryManager.retrieve(query, maxMemoryResults) 70 | memoryResults.forEach { memoryItem -> 71 | val contextContent = memoryManager.memoryToContext(memoryItem) 72 | contextWindow.addContext(contextContent) 73 | } 74 | 75 | // 4. 使用传统检索器检索外部知识（如果配置了） 76 | retriever?.let { ret -> 77 | val rewrittenQueries = queryRewriter.rewrite(query) 78 | rewrittenQueries.forEach { rewrittenQuery -> 79 | val retrievalResults = ret.retrieve(rewrittenQuery, maxRetrievalResults) 80 | retrievalResults.forEach { result -> 81 | contextWindow.addContext(result) 82 | } 83 | } 84 | } 85 | 86 | // 5. 添加用户查询（中等优先级） 87 | contextWindow.addContext( 88 | ContextContent( 89 | id = "user_query", 90 | type = ContextType.USER_INPUT, 91 | content = query, 92 | priority = ContextPriority.MEDIUM 93 | ) 94 | ) 95 | 96 | // 6. 将当前查询存储到短期记忆 97 | memoryManager.store( 98 | "User query: $query", 99 | mapOf("type" to "query", "priority" to "medium") 100 | ) 101 | 102 | return@runBlocking contextWindow.assembleContext() 103 | } 104 | 105 | /** 106 | * 添加内容到记忆系统 107 | */ 108 | suspend fun addToMemory(content: String, metadata: Map = emptyMap()): String? { 109 | return memoryManager.store(content, metadata) 110 | } 111 | 112 | /** 113 | * 从记忆系统检索内容 114 | */ 115 | suspend fun retrieveFromMemory(query: String, maxResults: Int = 5): List { 116 | return memoryManager.retrieve(query, maxResults) 117 | } 118 | 119 | /** 120 | * 添加短期记忆内容 121 | */ 122 | fun addShortTermMemory(content: String, metadata: Map = emptyMap()) = runBlocking { 123 | memoryManager.store(content, metadata + mapOf("memoryType" to "SHORT_TERM")) 124 | } 125 | 126 | /** 127 | * 添加长期记忆内容 128 | */ 129 | fun addLongTermMemory(content: String, metadata: Map = emptyMap()) = runBlocking { 130 | memoryManager.store(content, metadata + mapOf("priority" to "high", "type" to "knowledge")) 131 | } 132 | 133 | /** 134 | * 手动巩固记忆 135 | */ 136 | suspend fun consolidateMemories(): Int { 137 | return memoryManager.consolidateMemories() 138 | } 139 | 140 | /** 141 | * 清理过期记忆 142 | */ 143 | suspend fun cleanupMemories(): MemoryCleanupResult { 144 | return memoryManager.cleanup() 145 | } 146 | 147 | /** 148 | * 获取记忆统计信息 149 | */ 150 | suspend fun getMemoryStats(): MemoryManagerStats { 151 | return memoryManager.getStats() 152 | } 153 | 154 | /** 155 | * 获取当前上下文窗口状态 156 | */ 157 | fun getWindowStatus(): ContextWindowStatus { 158 | return ContextWindowStatus( 159 | totalTokenBudget = contextWindow.tokenBudget, 160 | usedTokens = contextWindow.usedTokens, 161 | remainingTokens = contextWindow.remainingTokens, 162 | contextCount = contextWindow.getAllContexts().size, 163 | contextsByType = contextWindow.getAllContexts().groupBy { it.type } 164 | ) 165 | } 166 | 167 | /** 168 | * 清理过期的短期记忆 169 | */ 170 | fun cleanupShortTermMemory(maxAge: Long = 3600000) = runBlocking { 171 | val criteria = CleanupCriteria(maxAge = maxAge) 172 | memoryManager.cleanup() 173 | } 174 | 175 | /** 176 | * 构建器模式支持 177 | */ 178 | fun builder(): ContextBuilderWithMemory { 179 | return ContextBuilderWithMemory(this) 180 | } 181 | 182 | /** 183 | * 清空所有内容 184 | */ 185 | suspend fun clear() { 186 | contextWindow.clear() 187 | memoryManager.clear() 188 | } 189 | } 190 | 191 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/context/ContextManager.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.context 2 | 3 | import com.phodal.ctxmesh.context.retrieval.ContextRetriever 4 | import com.phodal.ctxmesh.context.retrieval.rewrite.QueryRewriter 5 | import com.phodal.ctxmesh.context.retrieval.rewrite.SimpleCodeQueryRewriter 6 | 7 | /** 8 | * 上下文管理器 9 | * 整合上下文窗口和检索功能，提供统一的上下文工程接口 10 | */ 11 | class ContextManager( 12 | private val contextWindow: ContextWindow = DefaultContextWindow(), 13 | private val retriever: ContextRetriever? = null, 14 | private val queryRewriter: QueryRewriter = SimpleCodeQueryRewriter() 15 | ) { 16 | /** 17 | * 基于查询构建上下文窗口 18 | * 这是上下文工程的核心方法 19 | */ 20 | fun buildContextForQuery( 21 | query: String, 22 | systemPrompt: String? = null, 23 | outputFormat: String? = null, 24 | maxRetrievalResults: Int = 3 25 | ): String { 26 | // 清空现有上下文 27 | contextWindow.clear() 28 | 29 | // 1. 添加系统提示词（最高优先级） 30 | systemPrompt?.let { 31 | contextWindow.addContext( 32 | ContextContent( 33 | id = "system_prompt", 34 | type = ContextType.SYSTEM_PROMPT, 35 | content = it, 36 | priority = ContextPriority.HIGHEST 37 | ) 38 | ) 39 | } 40 | 41 | // 2. 添加输出格式说明 42 | outputFormat?.let { 43 | contextWindow.addContext( 44 | ContextContent( 45 | id = "output_format", 46 | type = ContextType.OUTPUT_FORMAT, 47 | content = it, 48 | priority = ContextPriority.HIGH 49 | ) 50 | ) 51 | } 52 | 53 | // 3. 添加用户查询 54 | contextWindow.addContext( 55 | ContextContent( 56 | id = "user_query", 57 | type = ContextType.USER_INPUT, 58 | content = query, 59 | priority = ContextPriority.HIGH 60 | ) 61 | ) 62 | 63 | // 4. 检索相关知识并添加到上下文（简化版本，暂时跳过检索） 64 | // TODO: 实现异步检索功能 65 | retriever?.let { ret -> 66 | // 暂时跳过检索功能，避免协程复杂性 67 | // 可以在后续版本中添加异步支持 68 | } 69 | 70 | return contextWindow.assembleContext() 71 | } 72 | 73 | /** 74 | * 添加长期记忆内容 75 | */ 76 | fun addLongTermMemory(content: String, metadata: Map = emptyMap()) { 77 | contextWindow.addContext( 78 | ContextContent( 79 | id = "memory_${kotlinx.datetime.Clock.System.now().epochSeconds}", 80 | type = ContextType.LONG_TERM_MEMORY, 81 | content = content, 82 | priority = ContextPriority.MEDIUM, 83 | metadata = metadata 84 | ) 85 | ) 86 | } 87 | 88 | /** 89 | * 添加短期记忆内容（会话上下文） 90 | */ 91 | fun addShortTermMemory(content: String, metadata: Map = emptyMap()) { 92 | contextWindow.addContext( 93 | ContextContent( 94 | id = "recent_${kotlinx.datetime.Clock.System.now().epochSeconds}", 95 | type = ContextType.SHORT_TERM_MEMORY, 96 | content = content, 97 | priority = ContextPriority.HIGH, 98 | metadata = metadata 99 | ) 100 | ) 101 | } 102 | 103 | /** 104 | * 添加工具上下文 105 | */ 106 | fun addToolContext(toolName: String, toolDescription: String, toolSchema: String) { 107 | contextWindow.addContext( 108 | ContextContent( 109 | id = "tool_$toolName", 110 | type = ContextType.TOOL_CONTEXT, 111 | content = "Tool: $toolName\nDescription: $toolDescription\nSchema: $toolSchema", 112 | priority = ContextPriority.HIGH, 113 | metadata = mapOf("tool_name" to toolName) 114 | ) 115 | ) 116 | } 117 | 118 | /** 119 | * 更新全局状态 120 | */ 121 | fun updateGlobalState(state: String, metadata: Map = emptyMap()) { 122 | // 移除旧的全局状态 123 | contextWindow.getAllContexts() 124 | .filter { it.type == ContextType.GLOBAL_STATE } 125 | .forEach { contextWindow.removeContext(it.id) } 126 | 127 | // 添加新的全局状态 128 | contextWindow.addContext( 129 | ContextContent( 130 | id = "global_state", 131 | type = ContextType.GLOBAL_STATE, 132 | content = state, 133 | priority = ContextPriority.MEDIUM, 134 | metadata = metadata 135 | ) 136 | ) 137 | } 138 | 139 | /** 140 | * 获取当前上下文窗口状态 141 | */ 142 | fun getWindowStatus(): ContextWindowStatus { 143 | return ContextWindowStatus( 144 | totalTokenBudget = contextWindow.tokenBudget, 145 | usedTokens = contextWindow.usedTokens, 146 | remainingTokens = contextWindow.remainingTokens, 147 | contextCount = contextWindow.getAllContexts().size, 148 | contextsByType = contextWindow.getAllContexts().groupBy { it.type } 149 | ) 150 | } 151 | 152 | /** 153 | * 清理过期的短期记忆 154 | * @param maxAge 最大保留时间（毫秒） 155 | */ 156 | fun cleanupShortTermMemory(maxAge: Long = 3600000) { // 默认1小时 157 | val currentTime = kotlinx.datetime.Clock.System.now().toEpochMilliseconds() 158 | val toRemove = contextWindow.getAllContexts() 159 | .filter { 160 | it.type == ContextType.SHORT_TERM_MEMORY && 161 | (currentTime - it.timestamp) > maxAge 162 | } 163 | .map { it.id } 164 | 165 | toRemove.forEach { contextWindow.removeContext(it) } 166 | } 167 | 168 | /** 169 | * 索引新内容到检索器 170 | */ 171 | fun indexContent(content: String, metadata: Map = emptyMap()): String? { 172 | // TODO: 实现异步索引功能 173 | return null 174 | } 175 | 176 | /** 177 | * 从检索器中移除内容 178 | */ 179 | fun removeIndexedContent(contentId: String): Boolean { 180 | // TODO: 实现异步移除功能 181 | return false 182 | } 183 | } 184 | 185 | /** 186 | * 上下文构建器 - 提供流式API 187 | */ 188 | class ContextBuilder(private val manager: ContextManager) { 189 | 190 | fun forQuery(query: String): ContextBuilder { 191 | this.query = query 192 | return this 193 | } 194 | 195 | fun withSystemPrompt(prompt: String): ContextBuilder { 196 | this.systemPrompt = prompt 197 | return this 198 | } 199 | 200 | fun withOutputFormat(format: String): ContextBuilder { 201 | this.outputFormat = format 202 | return this 203 | } 204 | 205 | fun withMaxRetrievalResults(max: Int): ContextBuilder { 206 | this.maxResults = max 207 | return this 208 | } 209 | 210 | fun build(): String { 211 | return manager.buildContextForQuery( 212 | query = query ?: "", 213 | systemPrompt = systemPrompt, 214 | outputFormat = outputFormat, 215 | maxRetrievalResults = maxResults 216 | ) 217 | } 218 | 219 | private var query: String? = null 220 | private var systemPrompt: String? = null 221 | private var outputFormat: String? = null 222 | private var maxResults: Int = 3 223 | } 224 | 225 | /** 226 | * 扩展函数：为 ContextManager 提供构建器模式 227 | */ 228 | fun ContextManager.builder(): ContextBuilder = ContextBuilder(this) 229 | -------------------------------------------------------------------------------- /CtxMesh/gradlew: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # 4 | # Copyright © 2015-2021 the original authors. 5 | # 6 | # Licensed under the Apache License, Version 2.0 (the "License"); 7 | # you may not use this file except in compliance with the License. 8 | # You may obtain a copy of the License at 9 | # 10 | # https://www.apache.org/licenses/LICENSE-2.0 11 | # 12 | # Unless required by applicable law or agreed to in writing, software 13 | # distributed under the License is distributed on an "AS IS" BASIS, 14 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | # See the License for the specific language governing permissions and 16 | # limitations under the License. 17 | # 18 | # SPDX-License-Identifier: Apache-2.0 19 | # 20 | 21 | ############################################################################## 22 | # 23 | # Gradle start up script for POSIX generated by Gradle. 24 | # 25 | # Important for running: 26 | # 27 | # (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is 28 | # noncompliant, but you have some other compliant shell such as ksh or 29 | # bash, then to run this script, type that shell name before the whole 30 | # command line, like: 31 | # 32 | # ksh Gradle 33 | # 34 | # Busybox and similar reduced shells will NOT work, because this script 35 | # requires all of these POSIX shell features: 36 | # * functions; 37 | # * expansions «$var», «${var}», «${var:-default}», «${var+SET}», 38 | # «${var#prefix}», «${var%suffix}», and «$( cmd )»; 39 | # * compound commands having a testable exit status, especially «case»; 40 | # * various built-in commands including «command», «set», and «ulimit». 41 | # 42 | # Important for patching: 43 | # 44 | # (2) This script targets any POSIX shell, so it avoids extensions provided 45 | # by Bash, Ksh, etc; in particular arrays are avoided. 46 | # 47 | # The "traditional" practice of packing multiple parameters into a 48 | # space-separated string is a well documented source of bugs and security 49 | # problems, so this is (mostly) avoided, by progressively accumulating 50 | # options in "$@", and eventually passing that to Java. 51 | # 52 | # Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, 53 | # and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; 54 | # see the in-line comments for details. 55 | # 56 | # There are tweaks for specific operating systems such as AIX, CygWin, 57 | # Darwin, MinGW, and NonStop. 58 | # 59 | # (3) This script is generated from the Groovy template 60 | # https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt 61 | # within the Gradle project. 62 | # 63 | # You can find Gradle at https://github.com/gradle/gradle/. 64 | # 65 | ############################################################################## 66 | 67 | # Attempt to set APP_HOME 68 | 69 | # Resolve links: $0 may be a link 70 | app_path=$0 71 | 72 | # Need this for daisy-chained symlinks. 73 | while 74 | APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path 75 | [ -h "$app_path" ] 76 | do 77 | ls=$( ls -ld "$app_path" ) 78 | link=${ls#*' -> '} 79 | case $link in #( 80 | /*) app_path=$link ;; #( 81 | *) app_path=$APP_HOME$link ;; 82 | esac 83 | done 84 | 85 | # This is normally unused 86 | # shellcheck disable=SC2034 87 | APP_BASE_NAME=${0##*/} 88 | # Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) 89 | APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit 90 | 91 | # Use the maximum available, or set MAX_FD != -1 to use that value. 92 | MAX_FD=maximum 93 | 94 | warn () { 95 | echo "$*" 96 | } >&2 97 | 98 | die () { 99 | echo 100 | echo "$*" 101 | echo 102 | exit 1 103 | } >&2 104 | 105 | # OS specific support (must be 'true' or 'false'). 106 | cygwin=false 107 | msys=false 108 | darwin=false 109 | nonstop=false 110 | case "$( uname )" in #( 111 | CYGWIN* ) cygwin=true ;; #( 112 | Darwin* ) darwin=true ;; #( 113 | MSYS* | MINGW* ) msys=true ;; #( 114 | NONSTOP* ) nonstop=true ;; 115 | esac 116 | 117 | CLASSPATH="\\\"\\\"" 118 | 119 | 120 | # Determine the Java command to use to start the JVM. 121 | if [ -n "$JAVA_HOME" ] ; then 122 | if [ -x "$JAVA_HOME/jre/sh/java" ] ; then 123 | # IBM's JDK on AIX uses strange locations for the executables 124 | JAVACMD=$JAVA_HOME/jre/sh/java 125 | else 126 | JAVACMD=$JAVA_HOME/bin/java 127 | fi 128 | if [ ! -x "$JAVACMD" ] ; then 129 | die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME 130 | 131 | Please set the JAVA_HOME variable in your environment to match the 132 | location of your Java installation." 133 | fi 134 | else 135 | JAVACMD=java 136 | if ! command -v java >/dev/null 2>&1 137 | then 138 | die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 139 | 140 | Please set the JAVA_HOME variable in your environment to match the 141 | location of your Java installation." 142 | fi 143 | fi 144 | 145 | # Increase the maximum file descriptors if we can. 146 | if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then 147 | case $MAX_FD in #( 148 | max*) 149 | # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. 150 | # shellcheck disable=SC2039,SC3045 151 | MAX_FD=$( ulimit -H -n ) || 152 | warn "Could not query maximum file descriptor limit" 153 | esac 154 | case $MAX_FD in #( 155 | '' | soft) :;; #( 156 | *) 157 | # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. 158 | # shellcheck disable=SC2039,SC3045 159 | ulimit -n "$MAX_FD" || 160 | warn "Could not set maximum file descriptor limit to $MAX_FD" 161 | esac 162 | fi 163 | 164 | # Collect all arguments for the java command, stacking in reverse order: 165 | # * args from the command line 166 | # * the main class name 167 | # * -classpath 168 | # * -D...appname settings 169 | # * --module-path (only if needed) 170 | # * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. 171 | 172 | # For Cygwin or MSYS, switch paths to Windows format before running java 173 | if "$cygwin" || "$msys" ; then 174 | APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) 175 | CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) 176 | 177 | JAVACMD=$( cygpath --unix "$JAVACMD" ) 178 | 179 | # Now convert the arguments - kludge to limit ourselves to /bin/sh 180 | for arg do 181 | if 182 | case $arg in #( 183 | -*) false ;; # don't mess with options #( 184 | /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath 185 | [ -e "$t" ] ;; #( 186 | *) false ;; 187 | esac 188 | then 189 | arg=$( cygpath --path --ignore --mixed "$arg" ) 190 | fi 191 | # Roll the args list around exactly as many times as the number of 192 | # args, so each arg winds up back in the position where it started, but 193 | # possibly modified. 194 | # 195 | # NB: a `for` loop captures its iteration list before it begins, so 196 | # changing the positional parameters here affects neither the number of 197 | # iterations, nor the values presented in `arg`. 198 | shift # remove old arg 199 | set -- "$@" "$arg" # push replacement arg 200 | done 201 | fi 202 | 203 | 204 | # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 205 | DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' 206 | 207 | # Collect all arguments for the java command: 208 | # * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, 209 | # and any embedded shellness will be escaped. 210 | # * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be 211 | # treated as '${Hostname}' itself on the command line. 212 | 213 | set -- \ 214 | "-Dorg.gradle.appname=$APP_BASE_NAME" \ 215 | -classpath "$CLASSPATH" \ 216 | -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \ 217 | "$@" 218 | 219 | # Stop when "xargs" is not available. 220 | if ! command -v xargs >/dev/null 2>&1 221 | then 222 | die "xargs is not available" 223 | fi 224 | 225 | # Use "xargs" to parse quoted args. 226 | # 227 | # With -n1 it outputs one arg per line, with the quotes and backslashes removed. 228 | # 229 | # In Bash we could simply go: 230 | # 231 | # readarray ARGS < <( xargs -n1 <<<"$var" ) && 232 | # set -- "${ARGS[@]}" "$@" 233 | # 234 | # but POSIX shell has neither arrays nor command substitution, so instead we 235 | # post-process each arg (as a line of input to sed) to backslash-escape any 236 | # character that might be a shell metacharacter, then use eval to reverse 237 | # that process (while maintaining the separation between arguments), and wrap 238 | # the whole thing up as a single "set" statement. 239 | # 240 | # This will of course break if any of these variables contains a newline or 241 | # an unmatched quote. 242 | # 243 | 244 | eval "set -- $( 245 | printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | 246 | xargs -n1 | 247 | sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | 248 | tr '\n' ' ' 249 | )" '"$@"' 250 | 251 | exec "$JAVACMD" "$@" 252 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonTest/kotlin/com/phodal/ctxmesh/memory/MemoryTest.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory 2 | 3 | import com.phodal.ctxmesh.memory.impl.DefaultLongMemory 4 | import com.phodal.ctxmesh.memory.impl.DefaultShortMemory 5 | import com.phodal.ctxmesh.memory.impl.LongMemoryFactory 6 | import com.phodal.ctxmesh.memory.impl.ShortMemoryFactory 7 | import kotlinx.coroutines.test.runTest 8 | import kotlin.test.* 9 | 10 | class MemoryTest { 11 | 12 | @Test 13 | fun testShortMemoryBasicOperations() = runTest { 14 | val shortMemory = ShortMemoryFactory.createDefault() 15 | 16 | // 测试存储 17 | val memoryId = shortMemory.store( 18 | "This is a test content for short memory", 19 | mapOf("type" to "test", "priority" to "medium") 20 | ) 21 | assertNotNull(memoryId) 22 | 23 | // 测试获取 24 | val retrievedItem = shortMemory.get(memoryId!!) 25 | assertNotNull(retrievedItem) 26 | assertEquals("This is a test content for short memory", retrievedItem.content) 27 | assertEquals("test", retrievedItem.metadata["type"]) 28 | 29 | // 测试更新 30 | val updated = shortMemory.update( 31 | memoryId, 32 | "Updated content", 33 | mapOf("type" to "test", "priority" to "high") 34 | ) 35 | assertTrue(updated) 36 | 37 | val updatedItem = shortMemory.get(memoryId) 38 | assertNotNull(updatedItem) 39 | assertEquals("Updated content", updatedItem.content) 40 | assertEquals("high", updatedItem.metadata["priority"]) 41 | 42 | // 测试删除 43 | val deleted = shortMemory.delete(memoryId) 44 | assertTrue(deleted) 45 | 46 | val deletedItem = shortMemory.get(memoryId) 47 | assertNull(deletedItem) 48 | } 49 | 50 | @Test 51 | fun testLongMemoryBasicOperations() = runTest { 52 | val longMemory = LongMemoryFactory.createDefault() 53 | 54 | // 测试存储高重要性内容 55 | val memoryId = longMemory.store( 56 | "This is important knowledge that should be stored in long-term memory for future reference and learning.", 57 | mapOf("type" to "knowledge", "priority" to "high", "category" to "learning") 58 | ) 59 | assertNotNull(memoryId) 60 | 61 | // 测试获取 62 | val retrievedItem = longMemory.get(memoryId!!) 63 | assertNotNull(retrievedItem) 64 | assertTrue(retrievedItem.content.contains("important knowledge")) 65 | assertEquals("knowledge", retrievedItem.metadata["type"]) 66 | 67 | // 测试检索 68 | val searchResults = longMemory.retrieve("important knowledge", 5) 69 | assertTrue(searchResults.isNotEmpty()) 70 | assertTrue(searchResults.any { it.id == memoryId }) 71 | 72 | // 测试语义检索 73 | val semanticResults = longMemory.retrieve("learning reference", 5) 74 | assertTrue(semanticResults.isNotEmpty()) 75 | } 76 | 77 | @Test 78 | fun testShortMemoryRelevanceScoring() = runTest { 79 | val shortMemory = ShortMemoryFactory.createDefault() 80 | 81 | // 存储多个相关内容 82 | val id1 = shortMemory.store( 83 | "Kotlin is a modern programming language", 84 | mapOf("type" to "definition", "topic" to "kotlin") 85 | ) 86 | val id2 = shortMemory.store( 87 | "Java is an object-oriented programming language", 88 | mapOf("type" to "definition", "topic" to "java") 89 | ) 90 | val id3 = shortMemory.store( 91 | "Python is a high-level programming language", 92 | mapOf("type" to "definition", "topic" to "python") 93 | ) 94 | 95 | // 测试相关性检索 96 | val results = shortMemory.retrieve("programming language", 5) 97 | assertEquals(3, results.size) 98 | 99 | // 验证结果按相关性排序 100 | assertTrue(results[0].relevanceScore >= results[1].relevanceScore) 101 | assertTrue(results[1].relevanceScore >= results[2].relevanceScore) 102 | 103 | // 测试特定查询 104 | val kotlinResults = shortMemory.retrieve("Kotlin modern", 5) 105 | assertTrue(kotlinResults.isNotEmpty()) 106 | assertEquals(id1, kotlinResults[0].id) 107 | } 108 | 109 | @Test 110 | fun testShortMemoryTimeDecay() = runTest { 111 | val config = ShortMemoryConfig( 112 | maxAge = 50, // 50毫秒过期 113 | decayRate = 5.0 114 | ) 115 | val shortMemory = DefaultShortMemory(config) 116 | 117 | // 存储内容 118 | val memoryId = shortMemory.store("Test content", mapOf("type" to "test")) 119 | assertNotNull(memoryId) 120 | 121 | // 立即获取应该成功 122 | val item1 = shortMemory.get(memoryId!!) 123 | assertNotNull(item1) 124 | 125 | // 等待过期 126 | kotlinx.coroutines.delay(100) 127 | 128 | // 手动触发清理来测试过期逻辑 129 | val cleanedCount = shortMemory.cleanup(CleanupCriteria(maxAge = 50)) 130 | 131 | // 验证清理功能工作（可能清理了0个或更多，取决于时间精度） 132 | assertTrue(cleanedCount >= 0) 133 | 134 | // 验证统计信息更新 135 | val stats = shortMemory.getStats() 136 | assertTrue(stats.totalItems >= 0) 137 | } 138 | 139 | @Test 140 | fun testLongMemoryImportanceFiltering() = runTest { 141 | val config = LongMemoryConfig( 142 | minImportanceThreshold = 0.3 // 降低阈值以便测试通过 143 | ) 144 | val longMemory = DefaultLongMemory(config) 145 | 146 | // 尝试存储低重要性内容 147 | val lowImportanceId = longMemory.store( 148 | "Short note", 149 | mapOf("priority" to "low") 150 | ) 151 | assertNull(lowImportanceId) // 应该被过滤掉 152 | 153 | // 存储高重要性内容 154 | val highImportanceId = longMemory.store( 155 | "This is a comprehensive guide to understanding complex algorithms and data structures in computer science. " + 156 | "It covers fundamental concepts like arrays, linked lists, trees, graphs, sorting algorithms, searching algorithms, " + 157 | "dynamic programming, greedy algorithms, and much more. This knowledge is essential for software developers " + 158 | "and computer science students who want to build efficient and scalable applications.", 159 | mapOf("priority" to "high", "type" to "knowledge") 160 | ) 161 | assertNotNull(highImportanceId) // 应该被接受 162 | } 163 | 164 | @Test 165 | fun testMemoryCleanup() = runTest { 166 | val shortMemory = ShortMemoryFactory.createDefault() 167 | 168 | // 存储多个项目 169 | val ids = mutableListOf() 170 | repeat(5) { i -> 171 | val id = shortMemory.store( 172 | "Test content $i", 173 | mapOf("index" to i) 174 | ) 175 | if (id != null) ids.add(id) 176 | } 177 | 178 | assertEquals(5, ids.size) 179 | 180 | // 执行清理（保留最多3个项目） 181 | val cleanupCriteria = CleanupCriteria(maxItems = 3) 182 | val cleanedCount = shortMemory.cleanup(cleanupCriteria) 183 | 184 | assertEquals(2, cleanedCount) 185 | 186 | // 验证统计信息 187 | val stats = shortMemory.getStats() 188 | assertEquals(3, stats.totalItems) 189 | } 190 | 191 | @Test 192 | fun testMemoryStats() = runTest { 193 | val shortMemory = ShortMemoryFactory.createDefault() 194 | 195 | // 初始状态 196 | val initialStats = shortMemory.getStats() 197 | assertEquals(0, initialStats.totalItems) 198 | assertEquals(MemoryType.SHORT_TERM, initialStats.memoryType) 199 | 200 | // 添加一些内容 201 | repeat(3) { i -> 202 | shortMemory.store("Content $i", mapOf("index" to i)) 203 | } 204 | 205 | val stats = shortMemory.getStats() 206 | assertEquals(3, stats.totalItems) 207 | assertTrue(stats.totalSize > 0) 208 | assertNotNull(stats.newestItem) 209 | assertNotNull(stats.oldestItem) 210 | } 211 | 212 | @Test 213 | fun testMemoryEventListeners() = runTest { 214 | val shortMemory = ShortMemoryFactory.createDefault() 215 | val events = mutableListOf() 216 | 217 | val listener = object : MemoryListener { 218 | override suspend fun onMemoryEvent(event: MemoryEvent) { 219 | events.add(event) 220 | } 221 | } 222 | 223 | shortMemory.addListener(listener) 224 | 225 | // 执行操作 226 | val memoryId = shortMemory.store("Test content", emptyMap()) 227 | assertNotNull(memoryId) 228 | 229 | shortMemory.get(memoryId!!) 230 | shortMemory.update(memoryId, "Updated content", emptyMap()) 231 | shortMemory.delete(memoryId) 232 | 233 | // 验证事件 234 | assertTrue(events.size >= 4) // STORED, ACCESSED, UPDATED, DELETED 235 | assertTrue(events.any { it.eventType == MemoryEventType.STORED }) 236 | assertTrue(events.any { it.eventType == MemoryEventType.ACCESSED }) 237 | assertTrue(events.any { it.eventType == MemoryEventType.UPDATED }) 238 | assertTrue(events.any { it.eventType == MemoryEventType.DELETED }) 239 | } 240 | 241 | @Test 242 | fun testLongMemoryAssociations() = runTest { 243 | val longMemory = LongMemoryFactory.createDefault() 244 | 245 | // 存储相关内容 246 | val id1 = longMemory.store( 247 | "Kotlin is a statically typed programming language that runs on the JVM", 248 | mapOf("type" to "knowledge", "priority" to "high", "topic" to "kotlin") 249 | ) 250 | val id2 = longMemory.store( 251 | "JVM (Java Virtual Machine) is the runtime environment for Java and Kotlin applications", 252 | mapOf("type" to "knowledge", "priority" to "high", "topic" to "jvm") 253 | ) 254 | 255 | assertNotNull(id1) 256 | assertNotNull(id2) 257 | 258 | // 检索应该能找到关联内容 259 | val results = longMemory.retrieve("Kotlin JVM", 5) 260 | assertTrue(results.size >= 2) 261 | 262 | // 测试关联记忆功能 263 | val associations = longMemory.getAssociatedMemories(id1!!, 5) 264 | assertTrue(associations.isNotEmpty()) 265 | } 266 | } 267 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/memory/ShortMemory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory 2 | 3 | import kotlinx.datetime.Clock 4 | import kotlinx.datetime.Instant 5 | import kotlin.math.exp 6 | import kotlin.math.max 7 | 8 | /** 9 | * 短期记忆抽象类 10 | * 11 | * 基于 README.md 中的短期记忆概念设计： 12 | * - 会话级别的上下文管理 13 | * - 时间衰减机制 14 | * - 自动清理过期内容 15 | * - 基于新近度和频率的优先级排序 16 | */ 17 | abstract class ShortMemory : Memory { 18 | 19 | /** 20 | * 默认配置 21 | */ 22 | protected open val config = ShortMemoryConfig() 23 | 24 | /** 25 | * 内存存储 26 | */ 27 | protected val memoryStore = mutableMapOf() 28 | 29 | /** 30 | * 访问历史记录 31 | */ 32 | protected val accessHistory = mutableMapOf>() 33 | 34 | /** 35 | * 记忆监听器 36 | */ 37 | protected val listeners = mutableListOf() 38 | 39 | override suspend fun store(content: String, metadata: Map): String? { 40 | // 检查容量限制 41 | if (memoryStore.size >= config.maxCapacity) { 42 | performCapacityCleanup() 43 | } 44 | 45 | val memoryId = generateMemoryId() 46 | val now = Clock.System.now() 47 | 48 | val memoryItem = MemoryItem( 49 | id = memoryId, 50 | content = content, 51 | metadata = metadata + mapOf("memoryType" to MemoryType.SHORT_TERM.name), 52 | timestamp = now, 53 | lastAccessed = now, 54 | accessCount = 1 55 | ) 56 | 57 | memoryStore[memoryId] = memoryItem 58 | accessHistory[memoryId] = mutableListOf(now) 59 | 60 | // 触发事件 61 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.STORED)) 62 | 63 | return memoryId 64 | } 65 | 66 | override suspend fun retrieve(query: String, maxResults: Int, threshold: Double): List { 67 | // 清理过期记忆 68 | cleanupExpiredMemories() 69 | 70 | val now = Clock.System.now() 71 | val candidates = memoryStore.values.toList() 72 | 73 | // 计算相关性分数 74 | val scoredItems = candidates.map { item -> 75 | val relevanceScore = calculateRelevanceScore(query, item) 76 | val temporalScore = calculateTemporalScore(item, now) 77 | val accessScore = calculateAccessScore(item) 78 | 79 | val combinedScore = config.relevanceWeight * relevanceScore + 80 | config.temporalWeight * temporalScore + 81 | config.accessWeight * accessScore 82 | 83 | item.copy(relevanceScore = combinedScore) 84 | } 85 | 86 | // 过滤和排序 87 | val filteredItems = scoredItems 88 | .filter { it.relevanceScore >= threshold } 89 | .sortedByDescending { it.relevanceScore } 90 | .take(maxResults) 91 | 92 | // 更新访问记录 93 | filteredItems.forEach { item -> 94 | updateAccessRecord(item.id, now) 95 | notifyListeners(SimpleMemoryEvent(now, item.id, MemoryEventType.RETRIEVED)) 96 | } 97 | 98 | return filteredItems 99 | } 100 | 101 | override suspend fun get(memoryId: String): MemoryItem? { 102 | val item = memoryStore[memoryId] ?: return null 103 | 104 | // 检查是否过期 105 | if (isExpired(item)) { 106 | delete(memoryId) 107 | return null 108 | } 109 | 110 | val now = Clock.System.now() 111 | updateAccessRecord(memoryId, now) 112 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.ACCESSED)) 113 | 114 | return item.copy(lastAccessed = now, accessCount = item.accessCount + 1) 115 | } 116 | 117 | override suspend fun update(memoryId: String, content: String, metadata: Map): Boolean { 118 | val existingItem = memoryStore[memoryId] ?: return false 119 | 120 | if (isExpired(existingItem)) { 121 | delete(memoryId) 122 | return false 123 | } 124 | 125 | val now = Clock.System.now() 126 | val updatedItem = existingItem.copy( 127 | content = content, 128 | metadata = metadata + mapOf("memoryType" to MemoryType.SHORT_TERM.name), 129 | lastAccessed = now, 130 | accessCount = existingItem.accessCount + 1 131 | ) 132 | 133 | memoryStore[memoryId] = updatedItem 134 | updateAccessRecord(memoryId, now) 135 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.UPDATED)) 136 | 137 | return true 138 | } 139 | 140 | override suspend fun delete(memoryId: String): Boolean { 141 | val removed = memoryStore.remove(memoryId) != null 142 | accessHistory.remove(memoryId) 143 | 144 | if (removed) { 145 | val now = Clock.System.now() 146 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.DELETED)) 147 | } 148 | 149 | return removed 150 | } 151 | 152 | override suspend fun cleanup(criteria: CleanupCriteria): Int { 153 | val now = Clock.System.now() 154 | var cleanedCount = 0 155 | 156 | val toRemove = mutableListOf() 157 | 158 | memoryStore.forEach { (id, item) -> 159 | var shouldRemove = false 160 | 161 | // 检查年龄 162 | criteria.maxAge?.let { maxAge -> 163 | if (now.toEpochMilliseconds() - item.timestamp.toEpochMilliseconds() > maxAge) { 164 | shouldRemove = true 165 | } 166 | } 167 | 168 | // 检查访问次数 169 | criteria.minAccessCount?.let { minCount -> 170 | if (item.accessCount < minCount) { 171 | shouldRemove = true 172 | } 173 | } 174 | 175 | // 检查相关性 176 | criteria.relevanceThreshold?.let { threshold -> 177 | if (item.relevanceScore < threshold) { 178 | shouldRemove = true 179 | } 180 | } 181 | 182 | // 检查是否过期 183 | if (isExpired(item)) { 184 | shouldRemove = true 185 | } 186 | 187 | if (shouldRemove) { 188 | toRemove.add(id) 189 | } 190 | } 191 | 192 | // 处理最大项数限制 193 | criteria.maxItems?.let { maxItems -> 194 | if (memoryStore.size - toRemove.size > maxItems) { 195 | val excess = memoryStore.size - toRemove.size - maxItems 196 | val sortedByPriority = memoryStore.values 197 | .filter { !toRemove.contains(it.id) } 198 | .sortedBy { calculatePriority(it, now) } 199 | .take(excess) 200 | 201 | toRemove.addAll(sortedByPriority.map { it.id }) 202 | } 203 | } 204 | 205 | // 执行删除 206 | toRemove.forEach { id -> 207 | delete(id) 208 | cleanedCount++ 209 | } 210 | 211 | return cleanedCount 212 | } 213 | 214 | override suspend fun getStats(): MemoryStats { 215 | val items = memoryStore.values 216 | val now = Clock.System.now() 217 | 218 | return MemoryStats( 219 | totalItems = items.size, 220 | totalSize = items.sumOf { it.content.length.toLong() }, 221 | oldestItem = items.minByOrNull { it.timestamp }?.timestamp, 222 | newestItem = items.maxByOrNull { it.timestamp }?.timestamp, 223 | averageAccessCount = if (items.isNotEmpty()) items.map { it.accessCount }.average() else 0.0, 224 | memoryType = MemoryType.SHORT_TERM 225 | ) 226 | } 227 | 228 | override suspend fun clear() { 229 | memoryStore.clear() 230 | accessHistory.clear() 231 | } 232 | 233 | /** 234 | * 添加记忆监听器 235 | */ 236 | fun addListener(listener: MemoryListener) { 237 | listeners.add(listener) 238 | } 239 | 240 | /** 241 | * 移除记忆监听器 242 | */ 243 | fun removeListener(listener: MemoryListener) { 244 | listeners.remove(listener) 245 | } 246 | 247 | // 抽象方法，由具体实现类提供 248 | protected abstract fun generateMemoryId(): String 249 | protected abstract fun calculateRelevanceScore(query: String, item: MemoryItem): Double 250 | 251 | // 受保护的辅助方法 252 | protected fun isExpired(item: MemoryItem): Boolean { 253 | val now = Clock.System.now() 254 | val age = now.toEpochMilliseconds() - item.lastAccessed.toEpochMilliseconds() 255 | return age > config.maxAge 256 | } 257 | 258 | protected fun calculateTemporalScore(item: MemoryItem, now: Instant): Double { 259 | val age = now.toEpochMilliseconds() - item.lastAccessed.toEpochMilliseconds() 260 | val normalizedAge = age.toDouble() / config.maxAge 261 | return exp(-config.decayRate * normalizedAge) 262 | } 263 | 264 | protected fun calculateAccessScore(item: MemoryItem): Double { 265 | return max(0.0, 1.0 - 1.0 / (1.0 + item.accessCount)) 266 | } 267 | 268 | protected fun calculatePriority(item: MemoryItem, now: Instant): Double { 269 | val temporalScore = calculateTemporalScore(item, now) 270 | val accessScore = calculateAccessScore(item) 271 | return config.temporalWeight * temporalScore + config.accessWeight * accessScore 272 | } 273 | 274 | protected fun updateAccessRecord(memoryId: String, timestamp: Instant) { 275 | val item = memoryStore[memoryId] ?: return 276 | val updatedItem = item.copy( 277 | lastAccessed = timestamp, 278 | accessCount = item.accessCount + 1 279 | ) 280 | memoryStore[memoryId] = updatedItem 281 | 282 | accessHistory.getOrPut(memoryId) { mutableListOf() }.add(timestamp) 283 | } 284 | 285 | protected suspend fun notifyListeners(event: MemoryEvent) { 286 | listeners.forEach { it.onMemoryEvent(event) } 287 | } 288 | 289 | protected suspend fun cleanupExpiredMemories() { 290 | val criteria = CleanupCriteria(maxAge = config.maxAge) 291 | cleanup(criteria) 292 | } 293 | 294 | protected suspend fun performCapacityCleanup() { 295 | val criteria = CleanupCriteria(maxItems = (config.maxCapacity * 0.8).toInt()) 296 | cleanup(criteria) 297 | } 298 | } 299 | 300 | /** 301 | * 短期记忆配置 302 | */ 303 | data class ShortMemoryConfig( 304 | val maxCapacity: Int = 1000, 305 | val maxAge: Long = 3600000, // 1小时 306 | val decayRate: Double = 2.0, 307 | val relevanceWeight: Double = 0.5, 308 | val temporalWeight: Double = 0.3, 309 | val accessWeight: Double = 0.2 310 | ) 311 | 312 | /** 313 | * 简单记忆事件实现 314 | */ 315 | data class SimpleMemoryEvent( 316 | override val timestamp: Instant, 317 | override val memoryId: String, 318 | override val eventType: MemoryEventType 319 | ) : MemoryEvent 320 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/memory/MemoryManager.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory 2 | 3 | import com.phodal.ctxmesh.context.ContextContent 4 | import com.phodal.ctxmesh.context.ContextPriority 5 | import com.phodal.ctxmesh.context.ContextType 6 | import com.phodal.ctxmesh.memory.impl.LongMemoryFactory 7 | import com.phodal.ctxmesh.memory.impl.ShortMemoryFactory 8 | 9 | /** 10 | * 记忆管理器 11 | * 12 | * 统一管理短期记忆和长期记忆，提供智能的记忆存储、检索和转移机制 13 | * 基于 README.md 中的记忆系统设计理念 14 | */ 15 | class MemoryManager( 16 | private val shortMemory: ShortMemory = ShortMemoryFactory.createDefault(), 17 | private val longMemory: LongMemory = LongMemoryFactory.createDefault(), 18 | private val config: MemoryManagerConfig = MemoryManagerConfig() 19 | ) { 20 | 21 | /** 22 | * 记忆监听器，用于处理记忆转移等事件 23 | */ 24 | private val memoryTransferListener = MemoryTransferListener() 25 | 26 | init { 27 | // 注册记忆转移监听器 28 | shortMemory.addListener(memoryTransferListener) 29 | } 30 | 31 | /** 32 | * 存储记忆内容 33 | * 根据内容重要性自动决定存储到短期还是长期记忆 34 | */ 35 | suspend fun store(content: String, metadata: Map = emptyMap()): String? { 36 | val importance = calculateContentImportance(content, metadata) 37 | 38 | return if (importance >= config.longTermThreshold) { 39 | // 直接存储到长期记忆 40 | longMemory.store(content, metadata + mapOf("autoStored" to true)) 41 | } else { 42 | // 存储到短期记忆 43 | shortMemory.store(content, metadata + mapOf("importance" to importance)) 44 | } 45 | } 46 | 47 | /** 48 | * 检索记忆内容 49 | * 同时从短期和长期记忆中检索，并合并结果 50 | */ 51 | suspend fun retrieve(query: String, maxResults: Int = 10, threshold: Double = 0.0): List { 52 | // 并行检索短期和长期记忆 53 | val shortTermResults = shortMemory.retrieve(query, maxResults / 2, threshold) 54 | val longTermResults = longMemory.retrieve(query, maxResults / 2, threshold) 55 | 56 | // 合并结果并去重 57 | val allResults = (shortTermResults + longTermResults).distinctBy { it.id } 58 | 59 | // 重新排序，考虑记忆类型的权重 60 | val rerankedResults = allResults.map { item -> 61 | val typeWeight = if (item.metadata["memoryType"] == MemoryType.SHORT_TERM.name) { 62 | config.shortTermWeight 63 | } else { 64 | config.longTermWeight 65 | } 66 | 67 | item.copy(relevanceScore = item.relevanceScore * typeWeight) 68 | } 69 | 70 | return rerankedResults 71 | .sortedByDescending { it.relevanceScore } 72 | .take(maxResults) 73 | } 74 | 75 | /** 76 | * 根据ID获取记忆 77 | */ 78 | suspend fun get(memoryId: String): MemoryItem? { 79 | // 首先尝试从短期记忆获取 80 | shortMemory.get(memoryId)?.let { return it } 81 | 82 | // 然后尝试从长期记忆获取 83 | return longMemory.get(memoryId) 84 | } 85 | 86 | /** 87 | * 更新记忆内容 88 | */ 89 | suspend fun update(memoryId: String, content: String, metadata: Map = emptyMap()): Boolean { 90 | // 尝试更新短期记忆 91 | if (shortMemory.update(memoryId, content, metadata)) { 92 | return true 93 | } 94 | 95 | // 尝试更新长期记忆 96 | return longMemory.update(memoryId, content, metadata) 97 | } 98 | 99 | /** 100 | * 删除记忆 101 | */ 102 | suspend fun delete(memoryId: String): Boolean { 103 | val shortDeleted = shortMemory.delete(memoryId) 104 | val longDeleted = longMemory.delete(memoryId) 105 | return shortDeleted || longDeleted 106 | } 107 | 108 | /** 109 | * 手动将短期记忆转移到长期记忆 110 | */ 111 | suspend fun promoteToLongTerm(memoryId: String): Boolean { 112 | val shortTermItem = shortMemory.get(memoryId) ?: return false 113 | 114 | // 存储到长期记忆 115 | val longTermId = longMemory.store( 116 | shortTermItem.content, 117 | shortTermItem.metadata + mapOf("promotedFrom" to memoryId) 118 | ) 119 | 120 | if (longTermId != null) { 121 | // 从短期记忆中删除 122 | shortMemory.delete(memoryId) 123 | return true 124 | } 125 | 126 | return false 127 | } 128 | 129 | /** 130 | * 自动记忆巩固 131 | * 将重要的短期记忆转移到长期记忆 132 | */ 133 | suspend fun consolidateMemories(): Int { 134 | val shortTermStats = shortMemory.getStats() 135 | var consolidatedCount = 0 136 | 137 | // 获取所有短期记忆 138 | val allShortTermMemories = shortMemory.retrieve("", Int.MAX_VALUE, 0.0) 139 | 140 | // 筛选需要巩固的记忆 141 | val candidatesForConsolidation = allShortTermMemories.filter { item -> 142 | val importance = item.metadata["importance"] as? Double ?: 0.0 143 | val accessCount = item.accessCount 144 | 145 | // 基于重要性、访问次数和年龄决定是否巩固 146 | importance >= config.consolidationImportanceThreshold || 147 | accessCount >= config.consolidationAccessThreshold 148 | } 149 | 150 | // 执行巩固 151 | candidatesForConsolidation.forEach { item -> 152 | if (promoteToLongTerm(item.id)) { 153 | consolidatedCount++ 154 | } 155 | } 156 | 157 | return consolidatedCount 158 | } 159 | 160 | /** 161 | * 清理过期记忆 162 | */ 163 | suspend fun cleanup(): MemoryCleanupResult { 164 | val shortTermCleaned = shortMemory.cleanup() 165 | val longTermCleaned = longMemory.cleanup() 166 | 167 | return MemoryCleanupResult( 168 | shortTermCleaned = shortTermCleaned, 169 | longTermCleaned = longTermCleaned, 170 | totalCleaned = shortTermCleaned + longTermCleaned 171 | ) 172 | } 173 | 174 | /** 175 | * 获取记忆统计信息 176 | */ 177 | suspend fun getStats(): MemoryManagerStats { 178 | val shortTermStats = shortMemory.getStats() 179 | val longTermStats = longMemory.getStats() 180 | 181 | return MemoryManagerStats( 182 | shortTermStats = shortTermStats, 183 | longTermStats = longTermStats, 184 | totalItems = shortTermStats.totalItems + longTermStats.totalItems, 185 | totalSize = shortTermStats.totalSize + longTermStats.totalSize 186 | ) 187 | } 188 | 189 | /** 190 | * 转换记忆项为上下文内容 191 | */ 192 | fun memoryToContext(memoryItem: MemoryItem): ContextContent { 193 | val contextType = when (memoryItem.metadata["memoryType"]) { 194 | MemoryType.SHORT_TERM.name -> ContextType.SHORT_TERM_MEMORY 195 | MemoryType.LONG_TERM.name -> ContextType.LONG_TERM_MEMORY 196 | else -> ContextType.EXTERNAL_KNOWLEDGE 197 | } 198 | 199 | val priority = when { 200 | memoryItem.relevanceScore >= 0.8 -> ContextPriority.HIGHEST 201 | memoryItem.relevanceScore >= 0.6 -> ContextPriority.HIGH 202 | memoryItem.relevanceScore >= 0.4 -> ContextPriority.MEDIUM 203 | memoryItem.relevanceScore >= 0.2 -> ContextPriority.LOW 204 | else -> ContextPriority.LOWEST 205 | } 206 | 207 | return memoryItem.toContextContent(contextType, priority) 208 | } 209 | 210 | /** 211 | * 清空所有记忆 212 | */ 213 | suspend fun clear() { 214 | shortMemory.clear() 215 | longMemory.clear() 216 | } 217 | 218 | /** 219 | * 计算内容重要性 220 | */ 221 | private fun calculateContentImportance(content: String, metadata: Map): Double { 222 | var importance = 0.0 223 | 224 | // 基于内容长度 225 | importance += kotlin.math.min(1.0, content.length / 500.0) * 0.2 226 | 227 | // 基于元数据中的优先级 228 | metadata["priority"]?.let { priority -> 229 | when (priority.toString().lowercase()) { 230 | "high" -> importance += 0.4 231 | "medium" -> importance += 0.2 232 | "low" -> importance += 0.1 233 | } 234 | } 235 | 236 | // 基于内容类型 237 | metadata["type"]?.let { type -> 238 | when (type.toString().lowercase()) { 239 | "knowledge", "definition", "concept" -> importance += 0.3 240 | "example", "code" -> importance += 0.2 241 | "note", "comment" -> importance += 0.1 242 | } 243 | } 244 | 245 | // 基于结构化程度 246 | val structureScore = calculateStructureScore(content) 247 | importance += structureScore * 0.1 248 | 249 | return kotlin.math.min(1.0, importance) 250 | } 251 | 252 | /** 253 | * 计算内容结构化程度 254 | */ 255 | private fun calculateStructureScore(content: String): Double { 256 | val lines = content.lines() 257 | val structuredLines = lines.count { line -> 258 | line.trim().startsWith("-") || 259 | line.trim().startsWith("*") || 260 | line.trim().startsWith("1.") || 261 | line.contains(":") 262 | } 263 | 264 | return if (lines.isNotEmpty()) { 265 | structuredLines.toDouble() / lines.size 266 | } else 0.0 267 | } 268 | 269 | /** 270 | * 记忆转移监听器 271 | */ 272 | private inner class MemoryTransferListener : MemoryListener { 273 | override suspend fun onMemoryEvent(event: MemoryEvent) { 274 | when (event.eventType) { 275 | MemoryEventType.RETRIEVED -> { 276 | // 检查是否需要自动巩固 277 | val item = shortMemory.get(event.memoryId) 278 | if (item != null && shouldAutoConsolidate(item)) { 279 | promoteToLongTerm(event.memoryId) 280 | } 281 | } 282 | else -> { 283 | // 其他事件暂不处理 284 | } 285 | } 286 | } 287 | 288 | private fun shouldAutoConsolidate(item: MemoryItem): Boolean { 289 | val importance = item.metadata["importance"] as? Double ?: 0.0 290 | return importance >= config.autoConsolidationThreshold && 291 | item.accessCount >= config.autoConsolidationAccessThreshold 292 | } 293 | } 294 | } 295 | 296 | /** 297 | * 记忆管理器配置 298 | */ 299 | data class MemoryManagerConfig( 300 | val longTermThreshold: Double = 0.7, 301 | val consolidationImportanceThreshold: Double = 0.5, 302 | val consolidationAccessThreshold: Int = 3, 303 | val autoConsolidationThreshold: Double = 0.8, 304 | val autoConsolidationAccessThreshold: Int = 5, 305 | val shortTermWeight: Double = 1.2, 306 | val longTermWeight: Double = 1.0 307 | ) 308 | 309 | /** 310 | * 记忆清理结果 311 | */ 312 | data class MemoryCleanupResult( 313 | val shortTermCleaned: Int, 314 | val longTermCleaned: Int, 315 | val totalCleaned: Int 316 | ) 317 | 318 | /** 319 | * 记忆管理器统计信息 320 | */ 321 | data class MemoryManagerStats( 322 | val shortTermStats: MemoryStats, 323 | val longTermStats: MemoryStats, 324 | val totalItems: Int, 325 | val totalSize: Long 326 | ) 327 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonTest/kotlin/com/phodal/ctxmesh/memory/MemoryManagerTest.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory 2 | 3 | import com.phodal.ctxmesh.memory.impl.LongMemoryFactory 4 | import com.phodal.ctxmesh.memory.impl.ShortMemoryFactory 5 | import kotlinx.coroutines.test.runTest 6 | import kotlin.test.* 7 | 8 | class MemoryManagerTest { 9 | 10 | @Test 11 | fun testMemoryManagerBasicOperations() = runTest { 12 | val memoryManager = MemoryManager( 13 | shortMemory = ShortMemoryFactory.createDefault(), 14 | longMemory = LongMemoryFactory.createDefault() 15 | ) 16 | 17 | // 测试存储中等重要性内容（应该进入短期记忆） 18 | val shortTermId = memoryManager.store( 19 | "This is a medium importance note", 20 | mapOf("priority" to "medium", "type" to "note") 21 | ) 22 | assertNotNull(shortTermId) 23 | 24 | // 测试存储高重要性内容（应该进入长期记忆） 25 | val longTermId = memoryManager.store( 26 | "This is a comprehensive guide to advanced software architecture patterns and design principles that every developer should understand.", 27 | mapOf("priority" to "high", "type" to "knowledge") 28 | ) 29 | assertNotNull(longTermId) 30 | 31 | // 测试检索 32 | val results = memoryManager.retrieve("software architecture", 5) 33 | assertTrue(results.isNotEmpty()) 34 | 35 | // 测试获取 36 | val shortTermItem = memoryManager.get(shortTermId!!) 37 | assertNotNull(shortTermItem) 38 | 39 | val longTermItem = memoryManager.get(longTermId!!) 40 | assertNotNull(longTermItem) 41 | } 42 | 43 | @Test 44 | fun testMemoryConsolidation() = runTest { 45 | val config = MemoryManagerConfig( 46 | consolidationImportanceThreshold = 0.3, 47 | consolidationAccessThreshold = 2 48 | ) 49 | 50 | val memoryManager = MemoryManager( 51 | shortMemory = ShortMemoryFactory.createDefault(), 52 | longMemory = LongMemoryFactory.createDefault(), 53 | config = config 54 | ) 55 | 56 | // 存储到短期记忆 57 | val memoryId = memoryManager.store( 58 | "Important programming concept that should be remembered", 59 | mapOf("priority" to "medium", "type" to "concept") 60 | ) 61 | assertNotNull(memoryId) 62 | 63 | // 多次访问以增加访问计数 64 | repeat(3) { 65 | memoryManager.get(memoryId!!) 66 | } 67 | 68 | // 执行记忆巩固 69 | val consolidatedCount = memoryManager.consolidateMemories() 70 | assertTrue(consolidatedCount >= 0) 71 | 72 | // 验证统计信息 73 | val stats = memoryManager.getStats() 74 | assertTrue(stats.totalItems > 0) 75 | } 76 | 77 | @Test 78 | fun testMemoryPromotion() = runTest { 79 | // 使用更宽松的配置 80 | val longMemoryConfig = LongMemoryConfig(minImportanceThreshold = 0.3) 81 | val memoryManager = MemoryManager( 82 | shortMemory = ShortMemoryFactory.createDefault(), 83 | longMemory = LongMemoryFactory.createWithConfig(longMemoryConfig) 84 | ) 85 | 86 | // 存储到短期记忆（使用更长的内容以确保能够提升到长期记忆） 87 | val shortTermId = memoryManager.store( 88 | "This is important content that should be promoted to long-term memory. " + 89 | "It contains valuable information about software development practices, " + 90 | "architectural patterns, and design principles that are essential for " + 91 | "building maintainable and scalable applications.", 92 | mapOf("priority" to "high", "type" to "knowledge") 93 | ) 94 | assertNotNull(shortTermId) 95 | 96 | // 手动提升到长期记忆 97 | val promoted = memoryManager.promoteToLongTerm(shortTermId!!) 98 | 99 | // 验证提升操作（可能成功也可能失败，取决于重要性计算） 100 | // 这里我们主要验证方法不会抛出异常 101 | assertTrue(promoted || !promoted) // 总是为真，只是验证方法执行 102 | 103 | // 验证记忆管理器的统计功能 104 | val stats = memoryManager.getStats() 105 | assertTrue(stats.totalItems >= 0) 106 | } 107 | 108 | @Test 109 | fun testMemoryRetrieval() = runTest { 110 | val memoryManager = MemoryManager( 111 | shortMemory = ShortMemoryFactory.createDefault(), 112 | longMemory = LongMemoryFactory.createDefault() 113 | ) 114 | 115 | // 存储不同类型的内容 116 | val shortId1 = memoryManager.store( 117 | "Recent conversation about Kotlin programming", 118 | mapOf("type" to "conversation", "priority" to "medium") 119 | ) 120 | 121 | val shortId2 = memoryManager.store( 122 | "User asked about data structures", 123 | mapOf("type" to "query", "priority" to "medium") 124 | ) 125 | 126 | val longId1 = memoryManager.store( 127 | "Comprehensive guide to Kotlin coroutines and asynchronous programming patterns", 128 | mapOf("type" to "knowledge", "priority" to "high") 129 | ) 130 | 131 | // 测试检索 132 | val kotlinResults = memoryManager.retrieve("Kotlin programming", 5) 133 | assertTrue(kotlinResults.isNotEmpty()) 134 | assertTrue(kotlinResults.any { it.content.contains("Kotlin") }) 135 | 136 | val dataResults = memoryManager.retrieve("data structures", 5) 137 | assertTrue(dataResults.isNotEmpty()) 138 | 139 | // 测试相关性排序 140 | assertTrue(kotlinResults[0].relevanceScore >= kotlinResults.last().relevanceScore) 141 | } 142 | 143 | @Test 144 | fun testMemoryCleanup() = runTest { 145 | val memoryManager = MemoryManager( 146 | shortMemory = ShortMemoryFactory.createDefault(), 147 | longMemory = LongMemoryFactory.createDefault() 148 | ) 149 | 150 | // 添加一些测试内容 151 | repeat(5) { i -> 152 | memoryManager.store( 153 | "Test content $i", 154 | mapOf("index" to i, "priority" to "low") 155 | ) 156 | } 157 | 158 | // 执行清理 159 | val cleanupResult = memoryManager.cleanup() 160 | assertTrue(cleanupResult.totalCleaned >= 0) 161 | 162 | // 验证清理结果 163 | assertTrue(cleanupResult.shortTermCleaned >= 0) 164 | assertTrue(cleanupResult.longTermCleaned >= 0) 165 | assertEquals( 166 | cleanupResult.shortTermCleaned + cleanupResult.longTermCleaned, 167 | cleanupResult.totalCleaned 168 | ) 169 | } 170 | 171 | @Test 172 | fun testMemoryStats() = runTest { 173 | val memoryManager = MemoryManager( 174 | shortMemory = ShortMemoryFactory.createDefault(), 175 | longMemory = LongMemoryFactory.createDefault() 176 | ) 177 | 178 | // 初始状态 179 | val initialStats = memoryManager.getStats() 180 | assertEquals(0, initialStats.totalItems) 181 | assertEquals(0, initialStats.totalSize) 182 | 183 | // 添加内容 184 | memoryManager.store("Short term content", mapOf("priority" to "medium")) 185 | memoryManager.store( 186 | "Long term knowledge base content with detailed information", 187 | mapOf("priority" to "high", "type" to "knowledge") 188 | ) 189 | 190 | val stats = memoryManager.getStats() 191 | assertTrue(stats.totalItems > 0) 192 | assertTrue(stats.totalSize > 0) 193 | assertTrue(stats.shortTermStats.totalItems >= 0) 194 | assertTrue(stats.longTermStats.totalItems >= 0) 195 | } 196 | 197 | @Test 198 | fun testMemoryToContextConversion() = runTest { 199 | val memoryManager = MemoryManager( 200 | shortMemory = ShortMemoryFactory.createDefault(), 201 | longMemory = LongMemoryFactory.createDefault() 202 | ) 203 | 204 | // 存储内容 205 | val memoryId = memoryManager.store( 206 | "Test content for context conversion", 207 | mapOf("type" to "test", "priority" to "high") 208 | ) 209 | assertNotNull(memoryId) 210 | 211 | // 获取记忆项 212 | val memoryItem = memoryManager.get(memoryId!!) 213 | assertNotNull(memoryItem) 214 | 215 | // 转换为上下文内容 216 | val contextContent = memoryManager.memoryToContext(memoryItem) 217 | 218 | assertEquals(memoryItem.content, contextContent.content) 219 | assertEquals(memoryItem.id, contextContent.id) 220 | assertTrue(contextContent.priority.weight > 0) 221 | } 222 | 223 | @Test 224 | fun testMemoryManagerConfiguration() = runTest { 225 | val config = MemoryManagerConfig( 226 | longTermThreshold = 0.8, 227 | consolidationImportanceThreshold = 0.6, 228 | consolidationAccessThreshold = 5, 229 | shortTermWeight = 1.5, 230 | longTermWeight = 0.8 231 | ) 232 | 233 | val memoryManager = MemoryManager( 234 | shortMemory = ShortMemoryFactory.createDefault(), 235 | longMemory = LongMemoryFactory.createDefault(), 236 | config = config 237 | ) 238 | 239 | // 测试配置是否生效 240 | // 存储中等重要性内容（应该进入短期记忆，因为阈值提高了） 241 | val memoryId = memoryManager.store( 242 | "Medium importance content", 243 | mapOf("priority" to "medium") 244 | ) 245 | assertNotNull(memoryId) 246 | 247 | // 验证内容确实存储了 248 | val item = memoryManager.get(memoryId!!) 249 | assertNotNull(item) 250 | } 251 | 252 | @Test 253 | fun testConcurrentMemoryOperations() = runTest { 254 | val memoryManager = MemoryManager( 255 | shortMemory = ShortMemoryFactory.createDefault(), 256 | longMemory = LongMemoryFactory.createDefault() 257 | ) 258 | 259 | // 并发存储操作 260 | val ids = mutableListOf() 261 | repeat(10) { i -> 262 | val id = memoryManager.store( 263 | "Concurrent content $i", 264 | mapOf("index" to i, "priority" to if (i % 2 == 0) "high" else "medium") 265 | ) 266 | ids.add(id) 267 | } 268 | 269 | // 验证所有操作都成功 270 | assertTrue(ids.all { it != null }) 271 | 272 | // 并发检索操作 273 | val results = memoryManager.retrieve("concurrent", 10) 274 | assertTrue(results.isNotEmpty()) 275 | 276 | // 验证数据一致性 277 | val stats = memoryManager.getStats() 278 | assertTrue(stats.totalItems > 0) 279 | } 280 | 281 | @Test 282 | fun testMemoryManagerClear() = runTest { 283 | val memoryManager = MemoryManager( 284 | shortMemory = ShortMemoryFactory.createDefault(), 285 | longMemory = LongMemoryFactory.createDefault() 286 | ) 287 | 288 | // 添加一些内容 289 | memoryManager.store("Test content 1", mapOf("priority" to "medium")) 290 | memoryManager.store("Test content 2", mapOf("priority" to "high")) 291 | 292 | // 验证内容存在 293 | val beforeStats = memoryManager.getStats() 294 | assertTrue(beforeStats.totalItems > 0) 295 | 296 | // 清空所有记忆 297 | memoryManager.clear() 298 | 299 | // 验证清空成功 300 | val afterStats = memoryManager.getStats() 301 | assertEquals(0, afterStats.totalItems) 302 | assertEquals(0, afterStats.totalSize) 303 | } 304 | } 305 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/memory/impl/DefaultLongMemory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory.impl 2 | 3 | import com.phodal.ctxmesh.memory.LongMemory 4 | import com.phodal.ctxmesh.memory.LongMemoryConfig 5 | import com.phodal.ctxmesh.memory.MemoryItem 6 | import kotlinx.datetime.Clock 7 | import kotlinx.datetime.Instant 8 | import kotlin.math.exp 9 | import kotlin.math.max 10 | import kotlin.math.min 11 | import kotlin.math.sqrt 12 | import kotlin.random.Random 13 | 14 | /** 15 | * 默认长期记忆实现 16 | * 17 | * 提供基于内存的长期记忆存储，支持语义检索和关联关系管理 18 | * 注意：这是一个简化的实现，生产环境中应该使用真正的向量数据库和持久化存储 19 | */ 20 | class DefaultLongMemory( 21 | config: LongMemoryConfig = LongMemoryConfig() 22 | ) : LongMemory() { 23 | 24 | override val config = config 25 | 26 | /** 27 | * 简化的持久化存储（实际应该使用数据库） 28 | */ 29 | private val persistentStore = mutableMapOf() 30 | 31 | override fun generateMemoryId(): String { 32 | val timestamp = Clock.System.now().toEpochMilliseconds() 33 | val random = Random.nextInt(1000, 9999) 34 | return "long_${timestamp}_$random" 35 | } 36 | 37 | override suspend fun generateSemanticVector(content: String): List { 38 | // 简化的语义向量生成（实际应该使用预训练的嵌入模型） 39 | val words = content.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 40 | val vectorSize = 128 41 | val vector = DoubleArray(vectorSize) { 0.0 } 42 | 43 | // 基于词汇的简单哈希向量 44 | words.forEach { word -> 45 | val hash = word.hashCode() 46 | for (i in 0 until vectorSize) { 47 | val index = kotlin.math.abs((hash + i) % vectorSize) 48 | vector[index] += 1.0 / words.size 49 | } 50 | } 51 | 52 | // 归一化 53 | val magnitude = sqrt(vector.sumOf { it * it }) 54 | if (magnitude > 0) { 55 | for (i in vector.indices) { 56 | vector[i] /= magnitude 57 | } 58 | } 59 | 60 | return vector.toList() 61 | } 62 | 63 | override suspend fun persistMemory(memory: MemoryItem) { 64 | // 简化的持久化实现 65 | persistentStore[memory.id] = memory 66 | } 67 | 68 | override suspend fun loadMemoryFromPersistence(memoryId: String): MemoryItem? { 69 | return persistentStore[memoryId] 70 | } 71 | 72 | override suspend fun deleteFromPersistence(memoryId: String) { 73 | persistentStore.remove(memoryId) 74 | } 75 | 76 | override suspend fun clearPersistence() { 77 | persistentStore.clear() 78 | } 79 | 80 | override suspend fun performKeywordSearch(query: String, maxResults: Int): List { 81 | val queryWords = query.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 82 | 83 | return memoryIndex.values.mapNotNull { item -> 84 | val score = calculateKeywordRelevance(query, item) 85 | if (score > 0.1) item.copy(relevanceScore = score) else null 86 | }.sortedByDescending { it.relevanceScore } 87 | .take(maxResults) 88 | } 89 | 90 | override suspend fun performSemanticSearch(query: String, maxResults: Int): List { 91 | val queryVector = generateSemanticVector(query) 92 | 93 | return memoryIndex.values.mapNotNull { item -> 94 | val itemVector = semanticIndex[item.id] 95 | if (itemVector != null) { 96 | val similarity = calculateCosineSimilarity(queryVector, itemVector) 97 | if (similarity > 0.2) item.copy(relevanceScore = similarity) else null 98 | } else null 99 | }.sortedByDescending { it.relevanceScore } 100 | .take(maxResults) 101 | } 102 | 103 | override suspend fun performAssociativeSearch(query: String, maxResults: Int): List { 104 | // 首先找到与查询相关的记忆 105 | val directMatches = performKeywordSearch(query, 3) 106 | val results = mutableListOf() 107 | 108 | // 然后找到这些记忆的关联记忆 109 | directMatches.forEach { match -> 110 | val associations = associationGraph[match.id] ?: emptySet() 111 | associations.forEach { associatedId -> 112 | memoryIndex[associatedId]?.let { associatedItem -> 113 | val score = calculateAssociativeRelevance(query, associatedItem) 114 | if (score > 0.1) { 115 | results.add(associatedItem.copy(relevanceScore = score)) 116 | } 117 | } 118 | } 119 | } 120 | 121 | return results.distinctBy { it.id } 122 | .sortedByDescending { it.relevanceScore } 123 | .take(maxResults) 124 | } 125 | 126 | override suspend fun findSimilarMemories(content: String, maxResults: Int): List { 127 | val contentVector = generateSemanticVector(content) 128 | 129 | return memoryIndex.values.mapNotNull { item -> 130 | val itemVector = semanticIndex[item.id] 131 | if (itemVector != null) { 132 | val similarity = calculateCosineSimilarity(contentVector, itemVector) 133 | if (similarity > 0.3) item.copy(relevanceScore = similarity) else null 134 | } else null 135 | }.sortedByDescending { it.relevanceScore } 136 | .take(maxResults) 137 | } 138 | 139 | /** 140 | * 同步版本的语义向量生成（用于非挂起函数） 141 | */ 142 | private fun generateSemanticVectorSync(content: String): List { 143 | // 与 generateSemanticVector 相同的实现，但不是挂起函数 144 | val words = content.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 145 | val vectorSize = 128 146 | val vector = DoubleArray(vectorSize) { 0.0 } 147 | 148 | // 基于词汇的简单哈希向量 149 | words.forEach { word -> 150 | val hash = word.hashCode() 151 | for (i in 0 until vectorSize) { 152 | val index = kotlin.math.abs((hash + i) % vectorSize) 153 | vector[index] += 1.0 / words.size 154 | } 155 | } 156 | 157 | // 归一化 158 | val magnitude = sqrt(vector.sumOf { it * it }) 159 | if (magnitude > 0) { 160 | for (i in vector.indices) { 161 | vector[i] /= magnitude 162 | } 163 | } 164 | 165 | return vector.toList() 166 | } 167 | 168 | override fun calculateKeywordRelevance(query: String, item: MemoryItem): Double { 169 | val queryWords = query.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 170 | val contentWords = item.content.lowercase().split(Regex("\\s+")).filter { it.isNotBlank() } 171 | 172 | if (queryWords.isEmpty() || contentWords.isEmpty()) { 173 | return 0.0 174 | } 175 | 176 | // TF-IDF 风格的计算 177 | var score = 0.0 178 | queryWords.forEach { queryWord -> 179 | val tf = contentWords.count { it.contains(queryWord) }.toDouble() / contentWords.size 180 | val idf = calculateIDF(queryWord) 181 | score += tf * idf 182 | } 183 | 184 | return min(1.0, score / queryWords.size) 185 | } 186 | 187 | override fun calculateSemanticRelevance(query: String, item: MemoryItem): Double { 188 | // 注意：这里应该缓存查询向量以避免重复计算 189 | // 为了简化，这里使用同步版本的向量生成 190 | val queryVector = generateSemanticVectorSync(query) 191 | val itemVector = semanticIndex[item.id] ?: return 0.0 192 | return calculateCosineSimilarity(queryVector, itemVector) 193 | } 194 | 195 | override fun calculateAssociativeRelevance(query: String, item: MemoryItem): Double { 196 | // 基于关联强度和重要性的评分 197 | val associations = associationGraph[item.id]?.size ?: 0 198 | val importance = getImportanceScore(item.id) 199 | val keywordScore = calculateKeywordRelevance(query, item) 200 | 201 | return (associations * 0.1 + importance * 0.4 + keywordScore * 0.5).coerceIn(0.0, 1.0) 202 | } 203 | 204 | override fun calculateTemporalRelevance(item: MemoryItem, now: Instant): Double { 205 | val age = now.toEpochMilliseconds() - item.timestamp.toEpochMilliseconds() 206 | val daysSinceCreation = age / (24 * 60 * 60 * 1000.0) 207 | 208 | // 长期记忆的时间衰减较慢 209 | val decayRate = 0.01 210 | return exp(-decayRate * daysSinceCreation) 211 | } 212 | 213 | /** 214 | * 计算余弦相似度 215 | */ 216 | private fun calculateCosineSimilarity(vector1: List, vector2: List): Double { 217 | if (vector1.size != vector2.size) return 0.0 218 | 219 | val dotProduct = vector1.zip(vector2) { a, b -> a * b }.sum() 220 | val magnitude1 = sqrt(vector1.sumOf { it * it }) 221 | val magnitude2 = sqrt(vector2.sumOf { it * it }) 222 | 223 | return if (magnitude1 > 0 && magnitude2 > 0) { 224 | dotProduct / (magnitude1 * magnitude2) 225 | } else 0.0 226 | } 227 | 228 | /** 229 | * 计算逆文档频率（简化版） 230 | */ 231 | private fun calculateIDF(word: String): Double { 232 | val totalDocs = memoryIndex.size 233 | if (totalDocs == 0) return 1.0 234 | 235 | val docsContainingWord = memoryIndex.values.count { 236 | it.content.lowercase().contains(word.lowercase()) 237 | } 238 | 239 | return if (docsContainingWord > 0) { 240 | kotlin.math.ln(totalDocs.toDouble() / docsContainingWord) 241 | } else 1.0 242 | } 243 | } 244 | 245 | /** 246 | * 长期记忆工厂类 247 | */ 248 | object LongMemoryFactory { 249 | 250 | /** 251 | * 创建默认的长期记忆实例 252 | */ 253 | fun createDefault(): DefaultLongMemory { 254 | return DefaultLongMemory() 255 | } 256 | 257 | /** 258 | * 创建自定义配置的长期记忆实例 259 | */ 260 | fun createWithConfig(config: LongMemoryConfig): DefaultLongMemory { 261 | return DefaultLongMemory(config) 262 | } 263 | 264 | /** 265 | * 创建适用于编程场景的长期记忆实例 266 | */ 267 | fun createForCoding(): DefaultLongMemory { 268 | val config = LongMemoryConfig( 269 | minImportanceThreshold = 0.4, 270 | consolidationThreshold = 3, 271 | maxConsolidationLevel = 5, 272 | keywordWeight = 0.3, 273 | semanticWeight = 0.4, 274 | associativeWeight = 0.1, 275 | importanceWeight = 0.15, 276 | temporalWeight = 0.05 277 | ) 278 | return DefaultLongMemory(config) 279 | } 280 | 281 | /** 282 | * 创建适用于知识管理的长期记忆实例 283 | */ 284 | fun createForKnowledgeManagement(): DefaultLongMemory { 285 | val config = LongMemoryConfig( 286 | minImportanceThreshold = 0.5, 287 | consolidationThreshold = 5, 288 | maxConsolidationLevel = 3, 289 | keywordWeight = 0.2, 290 | semanticWeight = 0.5, 291 | associativeWeight = 0.2, 292 | importanceWeight = 0.1, 293 | temporalWeight = 0.0 294 | ) 295 | return DefaultLongMemory(config) 296 | } 297 | 298 | /** 299 | * 创建适用于文档问答的长期记忆实例 300 | */ 301 | fun createForDocumentQA(): DefaultLongMemory { 302 | val config = LongMemoryConfig( 303 | minImportanceThreshold = 0.3, 304 | consolidationThreshold = 2, 305 | maxConsolidationLevel = 4, 306 | keywordWeight = 0.4, 307 | semanticWeight = 0.3, 308 | associativeWeight = 0.15, 309 | importanceWeight = 0.1, 310 | temporalWeight = 0.05 311 | ) 312 | return DefaultLongMemory(config) 313 | } 314 | } 315 | -------------------------------------------------------------------------------- /CtxMesh/shared/src/commonMain/kotlin/com/phodal/ctxmesh/memory/LongMemory.kt: -------------------------------------------------------------------------------- 1 | package com.phodal.ctxmesh.memory 2 | 3 | import kotlinx.datetime.Clock 4 | import kotlinx.datetime.Instant 5 | import kotlin.math.max 6 | import kotlin.math.min 7 | 8 | /** 9 | * 长期记忆抽象类 10 | * 11 | * 基于 README.md 中的长期记忆概念设计： 12 | * - 持久化存储机制 13 | * - 语义检索和向量搜索 14 | * - 知识图谱和关联关系 15 | * - 重要性评估和记忆巩固 16 | * - 分层存储架构 17 | */ 18 | abstract class LongMemory : Memory { 19 | 20 | /** 21 | * 默认配置 22 | */ 23 | protected open val config = LongMemoryConfig() 24 | 25 | /** 26 | * 记忆索引 - 用于快速检索 27 | */ 28 | protected val memoryIndex = mutableMapOf() 29 | 30 | /** 31 | * 语义索引 - 用于语义搜索 32 | */ 33 | protected val semanticIndex = mutableMapOf>() 34 | 35 | /** 36 | * 关联图谱 - 记忆之间的关联关系 37 | */ 38 | protected val associationGraph = mutableMapOf>() 39 | 40 | /** 41 | * 重要性分数缓存 42 | */ 43 | protected val importanceCache = mutableMapOf() 44 | 45 | /** 46 | * 记忆监听器 47 | */ 48 | protected val listeners = mutableListOf() 49 | 50 | override suspend fun store(content: String, metadata: Map): String? { 51 | val memoryId = generateMemoryId() 52 | val now = Clock.System.now() 53 | 54 | // 计算重要性分数 55 | val importance = calculateImportance(content, metadata) 56 | 57 | // 检查是否值得长期存储 58 | if (importance < config.minImportanceThreshold) { 59 | return null 60 | } 61 | 62 | val memoryItem = MemoryItem( 63 | id = memoryId, 64 | content = content, 65 | metadata = metadata + mapOf( 66 | "memoryType" to MemoryType.LONG_TERM.name, 67 | "importance" to importance, 68 | "consolidationLevel" to 0 69 | ), 70 | timestamp = now, 71 | lastAccessed = now, 72 | accessCount = 1, 73 | relevanceScore = importance 74 | ) 75 | 76 | // 存储到索引 77 | memoryIndex[memoryId] = memoryItem 78 | 79 | // 生成语义向量 80 | val semanticVector = generateSemanticVector(content) 81 | semanticIndex[memoryId] = semanticVector 82 | 83 | // 建立关联关系 84 | establishAssociations(memoryId, content, metadata) 85 | 86 | // 持久化存储 87 | persistMemory(memoryItem) 88 | 89 | // 更新重要性缓存 90 | importanceCache[memoryId] = importance 91 | 92 | // 触发事件 93 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.STORED)) 94 | 95 | return memoryId 96 | } 97 | 98 | override suspend fun retrieve(query: String, maxResults: Int, threshold: Double): List { 99 | val now = Clock.System.now() 100 | 101 | // 多种检索策略组合 102 | val keywordResults = performKeywordSearch(query, maxResults) 103 | val semanticResults = performSemanticSearch(query, maxResults) 104 | val associativeResults = performAssociativeSearch(query, maxResults) 105 | 106 | // 合并和去重 107 | val allResults = (keywordResults + semanticResults + associativeResults) 108 | .distinctBy { it.id } 109 | 110 | // 重新计算综合相关性分数 111 | val scoredResults = allResults.map { item -> 112 | val keywordScore = calculateKeywordRelevance(query, item) 113 | val semanticScore = calculateSemanticRelevance(query, item) 114 | val associativeScore = calculateAssociativeRelevance(query, item) 115 | val importanceScore = getImportanceScore(item.id) 116 | val temporalScore = calculateTemporalRelevance(item, now) 117 | 118 | val combinedScore = config.keywordWeight * keywordScore + 119 | config.semanticWeight * semanticScore + 120 | config.associativeWeight * associativeScore + 121 | config.importanceWeight * importanceScore + 122 | config.temporalWeight * temporalScore 123 | 124 | item.copy(relevanceScore = combinedScore) 125 | } 126 | 127 | // 过滤、排序和限制结果 128 | val finalResults = scoredResults 129 | .filter { it.relevanceScore >= threshold } 130 | .sortedByDescending { it.relevanceScore } 131 | .take(maxResults) 132 | 133 | // 更新访问记录和巩固记忆 134 | finalResults.forEach { item -> 135 | updateAccessRecord(item.id, now) 136 | consolidateMemory(item.id) 137 | notifyListeners(SimpleMemoryEvent(now, item.id, MemoryEventType.RETRIEVED)) 138 | } 139 | 140 | return finalResults 141 | } 142 | 143 | override suspend fun get(memoryId: String): MemoryItem? { 144 | val item = memoryIndex[memoryId] ?: loadMemoryFromPersistence(memoryId) 145 | 146 | if (item != null) { 147 | val now = Clock.System.now() 148 | updateAccessRecord(memoryId, now) 149 | consolidateMemory(memoryId) 150 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.ACCESSED)) 151 | } 152 | 153 | return item 154 | } 155 | 156 | override suspend fun update(memoryId: String, content: String, metadata: Map): Boolean { 157 | val existingItem = memoryIndex[memoryId] ?: return false 158 | 159 | val now = Clock.System.now() 160 | val importance = calculateImportance(content, metadata) 161 | 162 | val updatedItem = existingItem.copy( 163 | content = content, 164 | metadata = metadata + mapOf( 165 | "memoryType" to MemoryType.LONG_TERM.name, 166 | "importance" to importance, 167 | "lastModified" to now.toEpochMilliseconds() 168 | ), 169 | lastAccessed = now, 170 | accessCount = existingItem.accessCount + 1 171 | ) 172 | 173 | // 更新索引 174 | memoryIndex[memoryId] = updatedItem 175 | 176 | // 重新生成语义向量 177 | val semanticVector = generateSemanticVector(content) 178 | semanticIndex[memoryId] = semanticVector 179 | 180 | // 重新建立关联关系 181 | clearAssociations(memoryId) 182 | establishAssociations(memoryId, content, metadata) 183 | 184 | // 持久化更新 185 | persistMemory(updatedItem) 186 | 187 | // 更新重要性缓存 188 | importanceCache[memoryId] = importance 189 | 190 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.UPDATED)) 191 | 192 | return true 193 | } 194 | 195 | override suspend fun delete(memoryId: String): Boolean { 196 | val removed = memoryIndex.remove(memoryId) != null 197 | 198 | if (removed) { 199 | semanticIndex.remove(memoryId) 200 | clearAssociations(memoryId) 201 | importanceCache.remove(memoryId) 202 | deleteFromPersistence(memoryId) 203 | 204 | val now = Clock.System.now() 205 | notifyListeners(SimpleMemoryEvent(now, memoryId, MemoryEventType.DELETED)) 206 | } 207 | 208 | return removed 209 | } 210 | 211 | override suspend fun cleanup(criteria: CleanupCriteria): Int { 212 | val now = Clock.System.now() 213 | var cleanedCount = 0 214 | 215 | val toRemove = mutableListOf() 216 | 217 | memoryIndex.forEach { (id, item) -> 218 | var shouldRemove = false 219 | 220 | // 检查年龄（长期记忆通常不基于年龄清理） 221 | criteria.maxAge?.let { maxAge -> 222 | if (now.toEpochMilliseconds() - item.timestamp.toEpochMilliseconds() > maxAge) { 223 | shouldRemove = true 224 | } 225 | } 226 | 227 | // 检查访问次数 228 | criteria.minAccessCount?.let { minCount -> 229 | if (item.accessCount < minCount) { 230 | shouldRemove = true 231 | } 232 | } 233 | 234 | // 检查重要性 235 | criteria.relevanceThreshold?.let { threshold -> 236 | val importance = getImportanceScore(id) 237 | if (importance < threshold) { 238 | shouldRemove = true 239 | } 240 | } 241 | 242 | if (shouldRemove) { 243 | toRemove.add(id) 244 | } 245 | } 246 | 247 | // 处理最大项数限制 248 | criteria.maxItems?.let { maxItems -> 249 | if (memoryIndex.size - toRemove.size > maxItems) { 250 | val excess = memoryIndex.size - toRemove.size - maxItems 251 | val sortedByImportance = memoryIndex.values 252 | .filter { !toRemove.contains(it.id) } 253 | .sortedBy { getImportanceScore(it.id) } 254 | .take(excess) 255 | 256 | toRemove.addAll(sortedByImportance.map { it.id }) 257 | } 258 | } 259 | 260 | // 执行删除 261 | toRemove.forEach { id -> 262 | delete(id) 263 | cleanedCount++ 264 | } 265 | 266 | return cleanedCount 267 | } 268 | 269 | override suspend fun getStats(): MemoryStats { 270 | val items = memoryIndex.values 271 | 272 | return MemoryStats( 273 | totalItems = items.size, 274 | totalSize = items.sumOf { it.content.length.toLong() }, 275 | oldestItem = items.minByOrNull { it.timestamp }?.timestamp, 276 | newestItem = items.maxByOrNull { it.timestamp }?.timestamp, 277 | averageAccessCount = if (items.isNotEmpty()) items.map { it.accessCount }.average() else 0.0, 278 | memoryType = MemoryType.LONG_TERM 279 | ) 280 | } 281 | 282 | override suspend fun clear() { 283 | memoryIndex.clear() 284 | semanticIndex.clear() 285 | associationGraph.clear() 286 | importanceCache.clear() 287 | clearPersistence() 288 | } 289 | 290 | /** 291 | * 添加记忆监听器 292 | */ 293 | fun addListener(listener: MemoryListener) { 294 | listeners.add(listener) 295 | } 296 | 297 | /** 298 | * 移除记忆监听器 299 | */ 300 | fun removeListener(listener: MemoryListener) { 301 | listeners.remove(listener) 302 | } 303 | 304 | /** 305 | * 获取记忆的关联记忆 306 | */ 307 | suspend fun getAssociatedMemories(memoryId: String, maxResults: Int = 5): List { 308 | val associations = associationGraph[memoryId] ?: return emptyList() 309 | 310 | return associations.mapNotNull { associatedId -> 311 | memoryIndex[associatedId] 312 | }.sortedByDescending { getImportanceScore(it.id) } 313 | .take(maxResults) 314 | } 315 | 316 | // 抽象方法，由具体实现类提供 317 | protected abstract fun generateMemoryId(): String 318 | protected abstract suspend fun generateSemanticVector(content: String): List 319 | protected abstract suspend fun persistMemory(memory: MemoryItem) 320 | protected abstract suspend fun loadMemoryFromPersistence(memoryId: String): MemoryItem? 321 | protected abstract suspend fun deleteFromPersistence(memoryId: String) 322 | protected abstract suspend fun clearPersistence() 323 | 324 | // 受保护的辅助方法 325 | protected fun calculateImportance(content: String, metadata: Map): Double { 326 | var importance = 0.0 327 | 328 | // 基于内容长度 329 | importance += min(1.0, content.length / 1000.0) * 0.2 330 | 331 | // 基于元数据 332 | metadata["priority"]?.let { priority -> 333 | when (priority.toString().lowercase()) { 334 | "high" -> importance += 0.3 335 | "medium" -> importance += 0.2 336 | "low" -> importance += 0.1 337 | } 338 | } 339 | 340 | // 基于关键词密度 341 | val keywordDensity = calculateKeywordDensity(content) 342 | importance += keywordDensity * 0.3 343 | 344 | // 基于结构化程度 345 | val structureScore = calculateStructureScore(content) 346 | importance += structureScore * 0.2 347 | 348 | return min(1.0, importance) 349 | } 350 | 351 | protected fun calculateKeywordDensity(content: String): Double { 352 | // 简化实现：基于特殊字符和格式化的密度 353 | val specialChars = content.count { it in "{}[]()\"'`" } 354 | return min(1.0, specialChars.toDouble() / content.length) 355 | } 356 | 357 | protected fun calculateStructureScore(content: String): Double { 358 | // 简化实现：基于换行符和缩进的结构化程度 359 | val lines = content.lines() 360 | val indentedLines = lines.count { it.startsWith(" ") || it.startsWith("\t") } 361 | return if (lines.isNotEmpty()) indentedLines.toDouble() / lines.size else 0.0 362 | } 363 | 364 | protected fun getImportanceScore(memoryId: String): Double { 365 | return importanceCache[memoryId] ?: 0.0 366 | } 367 | 368 | protected suspend fun establishAssociations(memoryId: String, content: String, metadata: Map) { 369 | // 基于内容相似性建立关联 370 | val similarMemories = findSimilarMemories(content, 5) 371 | val associations = associationGraph.getOrPut(memoryId) { mutableSetOf() } 372 | 373 | similarMemories.forEach { similarMemory -> 374 | if (similarMemory.id != memoryId) { 375 | associations.add(similarMemory.id) 376 | // 双向关联 377 | associationGraph.getOrPut(similarMemory.id) { mutableSetOf() }.add(memoryId) 378 | } 379 | } 380 | } 381 | 382 | protected fun clearAssociations(memoryId: String) { 383 | val associations = associationGraph[memoryId] ?: return 384 | 385 | // 清除双向关联 386 | associations.forEach { associatedId -> 387 | associationGraph[associatedId]?.remove(memoryId) 388 | } 389 | 390 | associationGraph.remove(memoryId) 391 | } 392 | 393 | protected suspend fun consolidateMemory(memoryId: String) { 394 | val item = memoryIndex[memoryId] ?: return 395 | val currentLevel = item.metadata["consolidationLevel"] as? Int ?: 0 396 | 397 | if (item.accessCount > config.consolidationThreshold && currentLevel < config.maxConsolidationLevel) { 398 | val newLevel = min(config.maxConsolidationLevel, currentLevel + 1) 399 | val updatedMetadata = item.metadata + mapOf("consolidationLevel" to newLevel) 400 | val updatedItem = item.copy(metadata = updatedMetadata) 401 | 402 | memoryIndex[memoryId] = updatedItem 403 | persistMemory(updatedItem) 404 | } 405 | } 406 | 407 | protected fun updateAccessRecord(memoryId: String, timestamp: Instant) { 408 | val item = memoryIndex[memoryId] ?: return 409 | val updatedItem = item.copy( 410 | lastAccessed = timestamp, 411 | accessCount = item.accessCount + 1 412 | ) 413 | memoryIndex[memoryId] = updatedItem 414 | } 415 | 416 | protected suspend fun notifyListeners(event: MemoryEvent) { 417 | listeners.forEach { it.onMemoryEvent(event) } 418 | } 419 | 420 | // 抽象的检索方法，由具体实现提供 421 | protected abstract suspend fun performKeywordSearch(query: String, maxResults: Int): List 422 | protected abstract suspend fun performSemanticSearch(query: String, maxResults: Int): List 423 | protected abstract suspend fun performAssociativeSearch(query: String, maxResults: Int): List 424 | protected abstract suspend fun findSimilarMemories(content: String, maxResults: Int): List 425 | protected abstract fun calculateKeywordRelevance(query: String, item: MemoryItem): Double 426 | protected abstract fun calculateSemanticRelevance(query: String, item: MemoryItem): Double 427 | protected abstract fun calculateAssociativeRelevance(query: String, item: MemoryItem): Double 428 | protected abstract fun calculateTemporalRelevance(item: MemoryItem, now: Instant): Double 429 | } 430 | 431 | /** 432 | * 长期记忆配置 433 | */ 434 | data class LongMemoryConfig( 435 | val minImportanceThreshold: Double = 0.3, 436 | val consolidationThreshold: Int = 5, 437 | val maxConsolidationLevel: Int = 3, 438 | val keywordWeight: Double = 0.25, 439 | val semanticWeight: Double = 0.35, 440 | val associativeWeight: Double = 0.15, 441 | val importanceWeight: Double = 0.15, 442 | val temporalWeight: Double = 0.1 443 | ) 444 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Agent 架构综述：从 Prompt 到上下文工程构建 AI Agent 2 | 3 | 过去两三年，我曾为多家公司的资深开发人员开展 Agent 开发培训；最近一个月，我也一直在为毕业生设计和培训 AI Agent。直到本周， 4 | 结合 AI 能力的模拟项目 showcase 完成后，我才真正理清楚：如何针对不同阶段的开发者，系统地构建 Agent 的学习路径。这个过程， 5 | 也让我深刻体会到“知识诅咒”的存在——原来自己习以为常的知识，对于初学者来说，可能是最大的障碍。 6 | 7 | 我们可以简单把学习过程分为四部分： 8 | 9 | - 结构化提示词工程 —— 如何工程化设计高效、可复用的提示词。 10 | - 上下文工程与知识检索 —— 知识检索、生成与压缩上下文信息，产生高质量的知识背景。 11 | - 工具函数的系统化设计 —— 设计并实现可供 Agent 调用的工具与接口。 12 | - Agent 规划与多 Agent —— 构建任务规划与执行路径，实现闭环自动化。 13 | 14 | 开始之前，按理来说，我们需要简单定义一下 AI Agents，而考虑到 Agent 会有大量的不同定义 => 可以参考 Anthropic 在《Building effective agents》中给出真实世界中的示例： 15 | 16 | > 一些客户将其定义为完全自主运行的系统，能长期独立运作并使用多种工具完成复杂任务；另一些则用它描述遵循预定义工作流程的系统。 17 | 18 | 因此，围绕一个提示词的简单任务也可以视为一个 AI Agent；而围绕一个复杂系统、多工具、多步骤的任务，也是一个 AI Agent。 19 | 20 | ## 结构化提示词工程 21 | 22 | 尽管 Context Engineering 是一个非常火的词，但是如何写好 Prompt，依然是我们要入门的重点。网上已经有非常多的提示词相关的内容， 23 | 但是从我的经验来看，我们可以把重点放在三个部分上： 24 | 25 | - 提示词输入与输出的结构化 26 | - 复杂问题的链式与模块化设计 27 | - 提示词路由分发任务 28 | 29 | 再配合上一些必要的 AI 框架或者工具，就能非常不错的完成我们的任务。 30 | 31 | ### 提示词输入与输出的结构化 32 | 33 | 在现在的开发 Agent 的过程中，尽管模型能生成一部分提示词，但是调提示词依然是工作的重点。我们希望模型输出的内容可以是 34 | JSON、XML 或 Java 类，以便结合其它代码一起使用。 35 | 36 | > 提示词（Prompts）是用于引导 AI 模型生成**特定输出**而输入设计艺术与科学。通过对输入的精心设计与措辞，可以有效影响并控制模型的响应方向与结果， 37 | > 使 AI 生成符合预期的输出。 38 | 39 | 我们可以直接看 Spring AI 文档的 Structured Output Converter 作为示例： 40 | 41 | ![Structured Output Converter](images/structured-output-architecture.jpg) 42 | 43 | 图中的黄色部分即是两个核心： 44 | 45 | **格式化的输入指令** 46 | 47 | 通常来说，我们需要结构提示词模板来动态生成提示词，采用结构化的文本来设计输入： 48 | 49 | - 动态提示词模板（PromptTemplate）。采用经典的模板引擎动态结合上下文，如 LangChain 里的 Jinja2，Spring AI 中的 50 | StringTemplate。这种方式允许在运行时注入上下文、用户输入、系统状态等信息，实现灵活的 Prompt 构建。 51 | - 结构化的文本结构。为了保证 AI 输出的可靠性和可解析性，需要对提示词进行结构化设计，包括角色定位（Role）、任务描述（Task）、 52 | 约束条件（Constraints）、输出格式等。 53 | - 示例驱动。通过提供示例输入（Few-shots）与期望输出，可以显著提高模型输出的稳定性和一致性。诸如在实现 QA 的时候会给不同场景的实现示例。 54 | 55 | **转换模型输出结果** 56 | 57 | 即针对不同的场景采用合适的输出格式，并实现对应的解析实现与**异常场景处理**。 58 | 59 | - 领域特定的输出格式。我们会基于场景的不同，采用 JSON、XML、YAML 或者 Markdown 等不同的设计，以用户体验更好的方式来展示。诸如： 60 | JSON 的优点是可直接序列化传输，但是不能实时渲染体验差，不够健壮。YAML 则能更好地处理流式的问题，并且传输成本更低。 61 | - 解析实现。从纯文本中解析出代码块，再进行反序列化与对象映射等处理。使用 Schema 验证（JSON Schema、XSD）确保模型输出字段类型和结构符合约定。 62 | - 异常场景处理。由于模型生成存在不确定性，输出可能存在缺失、类型错误或不符合约定格式的情况。诸如：字段缺失时，使用默认值或回退策略，可触发模型重试生成特定字段 63 | 64 | 在能力适当的时候，可以基于已有的数据等信息，微调/训练模型来提升在这方面的能力。 65 | 66 | ### 提示词路由分发任务 67 | 68 | 在复杂的 AI 系统中，尤其是多 Agent 或多模块协作的场景下，单个提示词往往无法完成所有任务。所以我们需要提示词路由： 69 | 70 | ![https://www.ibm.com/think/topics/prompt-chaining](images/prompt-chaining-example.png) 71 | 72 | > 提示词路由（Prompt Routing） 是在多任务、多 Agent 或复杂 AI 流程中，将任务拆分、分析输入并智能分配给最合适模型或子任务提示词的工程化模式。 73 | 74 | 其核心思想是：通过分析输入和上下文，动态决定信息处理路径、使用哪条提示词或调用哪个工具、子 Agent，从而实现非线性、条件化的任务执行。以典型的 75 | QA 场景为例： 76 | 77 | - 非系统相关问题 → 直接告诉用户不支持该种类型的问题 78 | - 基础知识问题 → 调用文档检索和 QA 模型 79 | - 复杂分析问题 → 调用数据分析工具，再生成总结 80 | - …… 81 | 82 | 通过提示词路由，系统可以根据问题类型智能选择最合适的处理方式，同时保持模块化和可扩展性。在一些 AI 框架里， 诸如 LangChain 里的 RouterChain 83 | 就可以提供类似的能力支持，还有诸如于 [Routing by semantic similarity](https://python.langchain.com/docs/how_to/routing/#routing-by-semantic-similarity) 84 | 这种方式。 85 | 86 | ### 复杂问题的链式与模块化设计 87 | 88 | 在有提示词路由的前提下，复杂问题可以通过**提示链（Prompt Chaining）**进行系统化拆解。提示链允许将一个大任务拆分为多个子任务， 89 | 每个子任务对应不同的提示词或模型调用，最后将结果整合。这种方式比较适合于有固定的流程，并且有一些步骤是可以跳过的。 90 | 91 | ![https://arxiv.org/pdf/2308.11432](images/task-decomposition.jpg) 92 | 93 | 这可以实现更好的模块化 94 | 设计： 95 | 96 | - 每一个子任务都专注于处理特定阶段任务 97 | - 可以按需重写某个子任务，增加或替换提示词 98 | - 根据前一阶段输出动态调整后续提示词 99 | 100 | 以常见的软件需求为例，产品经理提出的想法可以通过提示链拆解为： 101 | 102 | ![](images/knowledge-example.png) 103 | 104 | 1. 创意收集：收集产品创意与初步需求 105 | 2. 需求逻辑梳理：理顺需求逻辑和功能优先级 106 | 3. 需求预排期：形成初步的需求文档或任务列表 107 | 4. 需求定稿：确认最终需求，生成正式文档 108 | 109 | 每个环节可以由不同的提示词或子 Agent 处理。例如，创意收集可借助具备搜索功能的 AI Agent，需求逻辑梳理可使用 Dify、 110 | Copilot 365 等工具完成。最终，各环节按链式流程执行，同时保持模块化设计的灵活性，可根据需要随时调整或替换子任务。 111 | 112 | ## 上下文工程与知识检索 113 | 114 | 通常来说，我们有 NoCode 和 ProCode 来支持带上下文的 Agent 开发。 115 | 116 | - NoCode 方案（适合快速验证）：用低代码平台（如 Dify、N8N、Coze 等）和预配置 RAG 管道，通过 UI 快速配置检索策略。 117 | - ProCode 方案（适合定制化需求）：用框架（LangChain、Spring AI）自定义检索流程和优化策略，能实现多阶段 HyDE + 混合检索 + 重排序管道。 118 | 119 | ![](images/context-engineering.jpg) 120 | 121 | 上下文本身也是提示词的一部分，在没有实现各种自动化之前，我们通常会手动从文档中复制到 AI 聊天工具中。只是随着，我们进入了模型的深水区之后， 122 | 我们就需要开始思考自动化的构建方式，也就是用工程化的角度来思考这个问题。开始之前，我们依旧需要定义 AI Agents，这里我们可以引用 Anthropic 123 | 官方的 《Effective context engineering for AI agents》给的定义（因为它同样也有科学和艺术）： 124 | 125 | > 上下文工程是一门将不断变化的信息宇宙中最相关内容，精心筛选并放入有限上下文窗口的艺术与科学。 126 | 127 | ### 上下文窗口 128 | 129 | 简单来说：关注在有限的上下文窗口中挑选最关键的信息，让模型理解和推理更高效。如下是 [Langchain 绘制](https://github.com/langchain-ai/how_to_fix_your_context)的 130 | [Drew Breunig 的总结](https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html) 的 6 种常见上下文工程技术： 131 | 132 | ![6 common context engineering techniques](images/langchain-context-enginneering.png) 133 | 134 | 在这里，我会将其简述为：RAG 与上下文窗口的工程化。一个完整的上下文窗口的内容（即 prompts）通常应该包含： 135 | 136 | - 系统提示词部分： 137 | - 输入指令上下文：告诉它“你是谁”“你要做什么”，包括系统提示、用户输入，还有角色定义。 138 | - 格式化输出上下文：指定模型输出格式的结构化模式，例如要求以 JSON 格式返回，以确保输出的可用性 。 139 | - 函数调用部分： 140 | - 工具相关上下文：这赋予了模型与外部世界交互的能力。它包括可用工具或函数的定义，以及调用这些工具后返回的响应 。 141 | - 动态上下文部分： 142 | - 时间与记忆上下文：短期记忆（Short-Term Memory）、长期记忆（Long-Term Memory）。 143 | - 外部知识上下文：从文档、数据库等外部信息库里查出来的事实，让模型少犯“胡说八道”的错误。 144 | - 全局状态/暂存区：模型处理复杂任务时的临时存储，相当于它的“工作记忆”。 145 | - 外部知识：通过检索增强生成（RAG）等技术，从外部知识库（如文档、数据库）中检索出的信息，用于为模型提供事实依据并减少幻觉 。 146 | 147 | 除了固定的系统提示词部分，**外部知识的获取**与**记忆**会最大化影响整个窗口，因此对于它们俩的设计与优化便是上下文工程的重中之重。 148 | 149 | ### 知识的检索增强生成 150 | 151 | ![Spring AI RAG](images/spring-ai-rag.jpg) 152 | 153 | > RAG（检索增强生成，Retrieval-Augmented Generation）是构建 Agent 的核心技术之一，它通过从外部知识库中检索相关信息来增强 154 | > 大语言模型的生成能力。在代码库问答等复杂场景中，单纯的向量检索往往不够精准，需要组合多种检索策略来提升准确率。 155 | 156 | 简单来说，就是通过搜索来丰富上下文。根据实现复杂度和场景需求，我们可以将检索策略分为以下几类： 157 | 158 | - **关键词检索（Keyword Search）**。最基础的检索方式，适合精确匹配场景。诸如在代码库中搜索特定的函数名、类名或变量名时，关键词检索往往比语义检索更有效。常见实现包括： 159 | - **全文检索**：使用 Elasticsearch、Solr 等搜索引擎，采用诸如 BM25, TF-IDF 等算法。 160 | - **正则表达式匹配**：如 ripgrep、grep 等工具，Cursor 就采用了 ripgrep + 向量检索的混合方式 161 | - **语义化检索（Semantic Search）**。通过向量嵌入（Embeddings）来理解查询的语义含义，而不仅仅是字面匹配。这对于自然语言查询尤为重要： 162 | - 使用预训练的嵌入模型（如 OpenAI text-embedding-3-large、Jina embeddings v3）将文本转换为向量 163 | - 在向量空间中计算查询与文档的相似度（通常使用余弦相似度或点积） 164 | - **图检索（Graph-based Search）**。图检索不仅关注“内容相似”，更注重关系与上下文依赖。 165 | - 代码场景下：构建代码调用关系图、依赖关系图，利用 AST（抽象语法树）提取方法、类、构造函数等结构 166 | - 示例：微软的 [GraphRAG](https://github.com/microsoft/graphrag)，Aider 的 repomap，又或者是 Joern，CodeQL 这种基础设施 167 | 168 | 而在检索之前，为了确保生成的检索结果可靠，需要引入**查询改写（Query Rewriting）**，即将用户的模糊意图逐步转化为数据库能够高效执行的精确查询。 169 | 修改用户的原始查询来提升其与知识库中文档的相关性，解决了自然语言问题与存储数据块之间的“阻抗不匹配”问题 。 170 | 171 | #### 代码场景下的 RAG 示例 172 | 173 | 通常来说，多种不同的检索策略可以组合使用，以提升检索效果。如下是向量数据库 LanceDB 官方给的一个[Codebase RAG 实现](https://blog.lancedb.com/rag-codebase-1/)： 174 | 175 | ![](images/codebase-indexing-with-graph.png) 176 | 177 | 除了在 indexing 阶段使用了 TreeSitter 来生成知识，在[检索阶段](https://blog.lancedb.com/building-rag-on-codebases-part-2/) 还会使用： 178 | 179 | - HyDE（假设性文档嵌入）：先让模型根据查询生成一个“假设性”文档或代码片段，再用这个生成的内容做向量搜索，这样更容易找到语义相关的代码。 180 | - BM25（关键字搜索）：传统的关键字搜索算法，擅长找包含精确术语或 API 名称的代码，也可以和向量搜索配合使用。 181 | - 混合检索（Hybrid Search）：把 BM25 和语义搜索结合起来，既能精确匹配关键字，也能理解代码语义，通过调整两者权重获得更优结果。 182 | - 重排序（Re-ranking）：在向量搜索得到初步结果后，再用交叉注意力机制对结果重新排序，提高最终答案的相关性和准确度。 183 | 184 | 当然了，在前面的 indexing 阶段，这个示例还会生成**元特征数据**，即对每个元素或代码片段，我们先生成代码的文本描述，再将该描述进行向量嵌入， 185 | 以获取代码的所有元特征，其特征是通过微调后的 LLM 提取的。 186 | 187 | ### 上下文窗口的工程化 188 | 189 | ![https://docs.claude.com/en/docs/build-with-claude/context-windows](images/context-window.svg) 190 | 191 | 两年前，[GitHub Copilot](https://code.visualstudio.com/docs/copilot/chat/prompt-crafting) 为补全为构建的上下文系统是业内 192 | 最值得研究的上下文系统（没有之一）： 193 | 194 | - 持续的信号监控。Copilot 插件会持续监控来自 IDE 的一系列信号，以动态调整上下文的优先级。诸如于插入或删除字符、当前编辑的文件和语言的改变，光标移动、滚动位置变化、文件的打开与关闭。 195 | - 上下文来源的[优先级排序](https://github.com/mengjian-github/copilot-analysis)。在发给模型的最终提示词里，会根据优化级来进行排序和筛选： 196 | - 最高优先级：光标位置周围的代码，包括光标前后的内容，这是最直接的上下文 。 197 | - 高优先级：当前正在编辑的文件的其余部分。 198 | - 中等优先级：在 IDE 中打开的其他文件或标签页（即“邻近文件”）。 199 | - 辅助上下文：其他信息也被纳入考量，包括文件路径、仓库 URL、代码中的导入语句（imports），以及 RAG 检索到的代码信息。 200 | - 上下文长度约束下的提示词组装。根据上述优先级对每个信息片段进行“评分”，然后组装出一个最优的提示。 201 | 202 | 这也就可以为我们提供一个非常不错的参考： 203 | 204 | - 新鲜度优先。最近编辑或访问的内容获得更高优先级，过时内容权重逐步衰减。 205 | - 信号融合与动态评分。融合多种编辑信号（如光标移动、文件切换、导入变更等），动态调整上下文权重。 206 | - 滑窗与增量更新。采用滑动窗口机制，仅对变化部分进行增量更新，避免全量重建。 207 | - 预算感知与自动截断。实时估算 token 占用，接近限制时自动裁剪或摘要低优先级内容。 208 | 209 | 当然这是一种非常复杂的设计，只值得你在足够高价值的系统中采用这样的设计。而结合现在流行的各种 Cursor Rule/Spec，采用诸如 AGENTS.md 用持久化记忆 210 | （Memory System）存储跨会话的关键信息，为后续查询提供长期背景信息。 211 | 212 | ### Agentic 检索 213 | 214 | ![https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/](images/agentic-rag.png) 215 | 216 | > Agentic 指的是一种让 AI 系统具备 自主感知、动态决策与目标导向执行能力 的特性，使其能够在任务过程中主动优化上下文、生成检索策略并持续自我迭代。 217 | 218 | 在 AI Coding 领域，诸如 Cursor、Claude Code 等，我们可以观察其运行的过程，其本质是 Agent 来执行 RAG。它与普通的 RAG 相比，它更加容易拿到 219 | 丰富的上下文，进而确保上下文在整个过程中是不缺失的。我们可以看到现有比较成熟的一些 AI 应用的示例： 220 | 221 | - Cursor 会优化采用 `file + ripgrep` 的方式来直接检索代码，在结果不够多时，再调用向量化检查或者 Git 历史等相关的检索 222 | - Google DeepResearch 在生成的过程中，也是类似的过程来完成某个研究：上下文工程主流工具的识别、工具功能和差异的初步掌握、下一步行动：工具细节的深入挖掘 223 | 224 | 简单来说，对于复杂的检索，我们可以将其构建为一个 Agent，由 Agent 来判断采用何种检索工具和策略，在上下文不够的时候，继续使用新的参数调用工具 225 | 以拿到足够的上下文。 226 | 227 | #### DeepResearch 示例 228 | 229 | 如下是 Langchain AI 构建的 [Open DeepResearch](https://github.com/langchain-ai/open_deep_research) 过程示例： 230 | 231 | ![](images/open-deep-research.jpg) 232 | 233 | Deep Research Agent 展示了一种更加系统化的 Agentic 检索方法： 234 | 235 | 1. 将任务拆分为规划阶段（Manager Agent） 和执行阶段（Execution Agent） 236 | - Manager Agent 负责任务理解、子任务拆分以及检索策略设计 237 | - Execution Agent 负责实际搜索、网页或文档抓取、内容解析 238 | 2. 在检索过程中，Agent 会维护对主题结构、已覆盖子问题和信息缺口的状态，以决定下一步探索方向 239 | 3. 可在关键阶段插入用户审查（HITL 模式），增强控制与精度 240 | 4. 最终，Agent 会将收集到的碎片信息整合成结构化报告，并附带来源引用 241 | 242 | 通常观察他们的交互与思考过程，会更好地帮助我们去理解这个过程。在此基础上，也可以看到诸如 Agentic Context Engineering 通过进一步让 LLM 243 | 自主生成、整理和迭代上下文，实现智能化、可扩展的上下文管理，从而优化复杂任务的检索与推理效率。 244 | 245 | ![https://www.arxiv.org/pdf/2510.04618](images/agentic-context-engineering.jpg) 246 | 247 | 即，基于历史的会话或者经验来优化如何检索，以让 Agent 更加适合于场景。 248 | 249 | ## Agent 工具系统的工程化设计 250 | 251 | 在构建 Agent 的过程中，工具系统（Tool System） 的设计是最能体现工程思维的一环。它决定了 Agent 能做什么、能做多好，以及能否与外部世界高效协作。 252 | 工具可以是一切的 API，如数据查询（如数据库访问）、现实世界操作（如发送邮件、预订会议）或与其他服务协同的接口。如我们在前面提到的 RAG 在 Agentic 下也是 253 | 工具的一种类型，诸如 LlamaIndex 便提供了这种显式的封装： 254 | 255 | - FunctionTool：可以将任何 Python 函数轻松封装成一个代理可用的工具。 256 | - QueryEngineTool：可以将任何数据查询引擎（例如，一个向量索引）转换为一个工具，使代理能够在其上进行查询和推理 257 | 258 | 这种围绕数据为中心的方式，可以简化我们对工具的理解。 259 | 260 | ### 语义化工具：为 Agent 设计的函数接口 261 | 262 | **工具**本质上是一类语义可理解的函数接口。它们不仅包含逻辑执行能力，更携带了让模型理解的元信息： 263 | 264 | - 名称（name）: 工具的唯一标识符，通常是函数名，例如 `getWeather`。 265 | - 描述（description）: 对工具功能、目的和适用场景的自然语言描述。这是至关重要的一环，因为模型主要依赖这段描述来判断何时以及如何使用该工具 。 266 | - 参数（parameters）: 一个定义工具输入参数的对象，包括每个参数的名称、数据类型（如 string, number）、描述以及是否为必需参数 。 267 | 268 | 在执行机制上，常见的两种范式是： 269 | 270 | - ReAct框架 (Reasoning + Acting)：ReAct范式的核心是让LLM交错地生成“思考”（推理轨迹）和“行动”（工具调用）， 271 | 从而形成一个显式的思考-行动-观察循环 。 272 | - 直接函数调用 (Direct Function Calling) ：这是一种更为结构化的方法。LLM 在单步推理中判断用户的查询可以通过调用一个或 273 | 多个预定义函数来最好地解答。然后，它会输出一个结构化的JSON对象，明确指出需要调用的函数名称及其参数 。 274 | 275 | 我们需要根据模型的支持情况，以及设计的交互、意图来决定使用哪种方式调用。 276 | 277 | ### 工具的设计原则 278 | 279 | 通常来说，在构建 Coding Agent 的时候，我们会遵循如下的原则： 280 | 281 | - 语义清晰：工具的名称、描述以及其参数的命名必须对 LLM 来说是极度清晰、富有描述性且无歧义的。工具的 description 字段视为一种面向 282 | AI 的 “微提示词” 来精心撰写 283 | - 无状态的**客观**函数：只封装复杂的技术逻辑或领域知识，避免做出战略性或主观性的决策 284 | - 原子性与单一职责：每个工具都应只负责一个且仅一个明确定义的功能，即执行一个原子操作。如果是 Agent 作为工具也应该遵循类似的原则，只完成一件事情。 285 | - 最小权限：每个工具应该只被授予完成其明确定义的任务所必需的最小权限和能力。 286 | 287 | #### 基于 Workflow 的工具编排：任务链式的设计 288 | 289 | 它也适用于在非编程领域的 AI Agent。基于上述的原则，我们可以将 “为我计划下周去北京的旅行”——分解为一组离散的、具有单一职责的工具。 290 | 291 | - search_flights(origin: str, destination: str, outbound_date: str, return_date: str): 搜索航班信息。 292 | - search_hotels(location: str, check_in_date: str, check_out_date: str, adults: int): 搜索酒店信息。 293 | - get_local_events(query: str, date: str): 获取特定日期的本地活动或景点信息。 294 | - book_cruise(passenger_name: str, itinerary_id: str): 预订邮轮行程。 295 | - lookup_vacation_packages(query: str): 查询度假套餐 296 | 297 | 这种编排方式的关键特征是：可预测性强、逻辑清晰、易于在平台中建模为可视化流程（如 DAG）。它非常适合 流程稳定、任务有依赖关系的 298 | Agent（如旅行、客服、数据管道类场景）。 299 | 300 | #### 基于分类的工具调用：动态意图决策 301 | 302 | 诸如 [Understanding GitHub Copilot’s Internal Architecture](https://iitsnl.com/blog/understanding-github-copilots-internal-architecture/) 303 | 中介绍的 Copilot 编排器会根据“意图分类器”（Intent Classifier）对用户请求的分析结果，决定调用一个或多个内部工具来完成任务： 304 | 305 | - 文件操作：包括 `read_file`（读取文件）、`edit_file`（编辑文件）和 `create_file`（创建新文件）等，使 Copilot 306 | 能够直接与用户的代码库进行交互 。 307 | - 代码执行：通过 `run_in_terminal` 工具，Copilot 可以在用户的终端中执行命令，例如运行测试或构建脚本 。 308 | - 搜索与分析：这是最关键的工具集之一，包括传统的 `grep_search`（文本搜索）、`list_code_usages`（列出代码引用），以及最强大的 309 | `semantic_search`**（语义搜索）** 。 310 | 311 | 这种模式的关键特征是：灵活性高、可扩展性强，但依赖良好的分类体系与语义匹配能力。它更适合动态场景，如代码生成、调试、文档问答等。 312 | 313 | ### 采用 MCP 协议，构建可组合的工具网络 314 | 315 | 当工具数量与 Agent数量不断增长时，我们需要一种机制来标准化描述、动态注册与跨 Agent 调用工具。 316 | MCP（Model Context Protocol） 正是为此设计的通用协议层。通过 MCP，AI Agent 不再依赖硬编码的接口或特定系统，而是能够以统一格式调用工具、 317 | 获取数据或与其他 Agent协作。MCP 的核心价值在于标准化、动态化和可组合： 318 | 319 | - 标准化：统一工具调用格式，使不同 Agent 可以共享工具集。 320 | - 动态化：支持运行时注册和访问工具， Agent可以根据任务需求选择最合适的工具。 321 | - 可组合：不同 Agent 和工具之间可以像搭积木一样组合，实现复杂任务的分解与协同执行。 322 | 323 | 结合前面设计的原子工具函数，MCP 可以将这些工具整合为一个可复用、可协作的工具网络，让 Agent 在解决复杂问题时更加灵活和高效。 324 | 325 | #### 其它工具网络 326 | 327 | 另外我们也可以看到 GitHub Copilot Extension，或者 Claude Code Plugin 这样的出现也在预示着，哪怕有了 MCP 和 A2A 这样的协议， 328 | AI Agent 生态并不会如我们预料的那么统一。诸如 https://github.com/wshobson/agents 项目就记录着（2025.10.14）： 329 | 330 | > 一个面向生产环境的综合系统，由 84 个专用 AI Agent、15 个多 Agent工作流编排器 和 44 个开发工具 组成，这些工具被组织为 62 331 | > 个聚焦且单一职责的插件，用于 Claude Code。 332 | 333 | ## Agent 规划与超越单体 Agent 334 | 335 | > Agent是使用 AI 来实现目标并代表用户完成任务的软件系统。其表现出了推理、规划和记忆能力，并且具有一定的自主性，能够自主学习、适应和做出决定。 - Google Cloud 336 | 337 | Agent 是目标为导向的，为了实现目标通常情况下需要**感知**-**规划**-**行动**，还有记忆，而复杂的 AI Agent 338 | 系统则会包含**协作**、**自我完善**等能力。而在前面的内容里，我们已经介绍了几个基本的能力： 339 | 340 | - 通过**结构化提示词与提示链**，Agent 具备了规划与决策的思维结构； 341 | - 通过**上下文工程**，Agent 获得了“感知世界”的能力，能够从外部知识与环境中捕获信息； 342 | - 通过**工具系统** 的工程化设计，Agent 获得了与外部世界交互、执行任务的行动力。 343 | 344 | 基于此之上，Agent 进一步发展的方向在于： 345 | 346 | - **协作（Collaboration）** —— 多 Agent 之间通过 A2A（Agent-to-Agent）通信协议或任务分配机制协同工作，实现角色分工与信息共享； 347 | - **自我完善（Self-improvement）** —— Agent 通过记忆系统与反思机制积累经验，优化自己的提示词与规划策略，从而具备持续学习与自我演化能力。 348 | 349 | 而由于这是一个快速发展的领域， 350 | 351 | ### 模块化的系统提示词：Agent 的思维蓝图 352 | 353 | 构建一个有效的 Agent 的第一步，是定义它的“思维蓝图”——即系统提示词（System Prompt）。优秀的系统提示词设计不仅定义了 Agent 354 | 应该做什么，也明确了不应该做什么。在 Coding Agent 领域，一个 Agent 的系统提示词往往极为复杂。例如 [Cursor](https://github.com/x1xhlol/system-prompts-and-models-of-ai-tools/blob/main/Cursor%20Prompts/Agent%20Prompt%202025-09-03.txt) 355 | 的系统提示词中，包含了关于角色、工具调用、安全边界、任务规划等详细规范。 356 | 357 | 结合 Cursor、Claude Code、Augment、Junie 等工具，我们可以总结出一系列模块化设计实践： 358 | 359 | - **结构分层与模块化**：用清晰的层次（角色/通信/工具/安全/任务）组织提示，避免“大一统”文本，便于维护与动态加载。 360 | - **工具优先级与并行化**：优先专用工具且能并行就并行，显著降低延迟与成本（如并行调用 `read_file` 读取多文件，编辑用 `search_replace` 而非 sed）。 361 | - **安全边界与权限模型**：默认沙箱最小权限，危险操作需显式授权（如 `required_permissions: ["network"|"git_write"|"all"]`），禁止对 `main/master` 强推等高风险动作。 362 | - **任务管理最小充分**：多步骤复杂任务用 TODO 管理（创建后第一个标记为 in_progress，完成即刻 completed），简单直接任务立刻执行。 363 | - **上下文唯一性与安全修**改：代码编辑要求唯一可定位的上下文（`old_string` 在文件中唯一，前后各 3–5 行），多处修改分次执行，避免误改。 364 | - **交流规范与用户体验**：隐藏内部工具名，用自然语言“说-做-总结”，保持简洁可扫读；用 backticks 标注文件/函数名，必要时给最小可用示例 365 | 366 | 这种从单体提示词向模块化、层次化、动态化演进的设计，正如从单体应用向微服务架构的转变，为 Agent 的高级推理、系统可扩展性与可维护性提供了结构支撑。 367 | 368 | ### 从检索到规划：使用 prompt 让 Agent 拆解目标 369 | 370 | 仅仅告诉 Agent “制定一个计划”是远远不够的，我们必须通过一套明确的原则来指导其分解过程，就像为软件模块制定规约一样。 371 | 单体 Agent 的智能上限，往往取决于其“规划能力”——能否将模糊目标拆解为明确的、可执行的子任务。 372 | 373 | 这涉及两种核心策略： 374 | 375 | - **预先分解**：这种策略也被称为静态规划，它在任务执行开始之前，就将整个复杂任务完整地分解成一个子任务序列或计划。 376 | - **交错分解**：这种策略也被称为动态规划，它不在任务开始时制定完整计划，而是在执行过程中动态地决定下一个子任务。 377 | 378 | 例如，BabyAGI 的架构就体现了这种“任务驱动”型规划： 它包含三个核心 Agent —— 379 | task_creation_agent（任务生成）、execution_agent（任务执行）和 prioritization_agent（任务优先级排序），形成了一个不断循环的任务更新与执行系统。 380 | 381 | 而在现代系统（如 Augment、Claude Code）中，规划逻辑往往以 todo_spec 的形式内嵌在系统提示词中，具备以下特点： 382 | 383 | - **原子性与行动导向**：每个待办事项都应该是一个独立的、不可再分的“原子”任务 384 | - **有意义的抽象层级**：待办事项不应是琐碎的操作性动作（如“读取文件 a.txt”或“修复 linter 错误”），而应是高层级的、有意义的、非平凡的任务。 385 | - **适当的范围**：规范倾向于“更少、更大的待办事项”，而不是一个冗长的微小步骤列表。 386 | - **以实现为中心**：如果用户的请求是要求实现某个功能，那么 Agent 生成的待办事项列表本身就是最终的计划。 387 | 388 | 通过这种结构化规划，Agent 能够把“用户需求”转化为“系统计划”，为多 Agent 协作奠定语义接口。 389 | 390 | ### 多 Agent 协作体系：从个体到组织 391 | 392 | ![](images/multiple-agent-architectures.png) 393 | 394 | 单体 Agent 的能力是有限的，而多 Agent 系统（Multi-Agent System, MAS）则是适合智能体系统发展的工程化方向。 395 | 正如微服务体系通过拆解单体应用来实现高内聚、低耦合，多 Agent 系统通过拆分智能体职责，实现了智能的横向扩展。 396 | 通过让多个 Agent 能够通过协作实现更复杂的目标，也能类似于“团队”在软件开发中的协同工作。 397 | 398 | 常见协作拓扑（参考 [LangGraph](https://langchain-ai.github.io/langgraph/concepts/multi_agent/)、AutoGen 等）： 399 | 400 | - 主管-专家模式（层级结构）：一个“主管 Agent”（Supervisor）或“协调员 Agent”（Coordinator）负责接收高层级的用户目标，将其分解为一系列子任务，然后根据每个子任务的性质，将其分派给相应的“专家 Agent” 。 401 | - 并行模式（群体智能）：也称为“并发模式”或“蜂群模式”。多个 Agent 同时独立地执行相同的任务或任务的不同部分，然后将它们的输出进行综合 。 402 | - 顺序模式（流水线）：Agent 们像在一条流水线上一样，按照预定义的顺序依次工作。前一个 Agent 的输出成为后一个 Agent 的输入 。 403 | - 网络模式（对话式/动态模式）：Agent 可以在一个多对多的网络中自由交流，没有固定的层级结构。下一个行动的 Agent 通常是根据对话的流程动态决定的 。 404 | 405 | 多 Agent 拓扑结构的选择，直接反映了待解决问题的底层结构。架构并非任意选择，而是试图创建一个能够镜像问题依赖图的“认知模型”。 406 | 当然，也不可避免会遇到类似于微服务架构中的复杂度等各种问题。 407 | 408 | #### A2A 协议：构建 Agent 网络，加速智能能力共享 409 | 410 | 2A 专为 Agent-to-Agent 通信而设计，与处理 Agent-to-Tool 通信的模型上下文协议（MCP）等其他标准形成互补。它扮演着公共互联网协议的角色，允许不同的 411 | Agent 系统相互连接和互操作。 412 | 413 | 尽管，我们并不一定需要引入 A2A 架构，如我们在 AutoDev 中实现的机制是，将 A2A 协议的 Agent 以 MCP 工具的形式暴露给 Agent 使用， 414 | 在不添加系统复杂度的情况下，实现了 Agent 与 Agent 之间的协作。 415 | 416 | #### 自我完善：反思、记忆与评价闭环 417 | 418 | 一个演化中的 Agent 的真正力量，来自于反思循环与持久记忆系统的紧密整合。 419 | 420 | - 反思机制：Agent 回顾自己的输出，识别错误并生成改进建议； 421 | - 记忆存储：持久化任务经验与上下文（如 `AGENTS.md`、Knowledge Graph），为后续任务提供长期参考。 422 | 423 | 对于记忆而言，应该根据新近度、相关性和重要性对记忆进行加权检索的机制，以及能够自主决定记住什么、忘记什么以及如何组织信息的反思性记忆管理系统 。 424 | 425 | > 一个先进 Agent 架构的最终目标，是创建一个自我强化的飞轮：行动产生经验，反思将经验提炼为知识，记忆存储知识以改进未来的行动。这将 426 | Agent 从一个静态的程序，转变为一个动态的学习实体。 427 | 428 | ## 小结 429 | 430 | > 系统提示词（System Prompt）在 Agent 系统中的地位，远超一份简单的指令集；它实际上是 Agent 431 | > 的核心“操作系统”，需要以系统架构设计的高度来对待提示词和上下文工程。 432 | 433 | 利用 Markdown 或 XML 等标记语言来构建结构化的指令模块，可以显著提高 LLM 对复杂规则的理解和遵循能力。 434 | 通过明确的角色激活、详尽的行为规范、以及“即时”加载数据等上下文工程技术，开发者可以为 Agent 塑造一个稳定、可预测的“认知环境”， 435 | 从而将其行为引导至期望的轨道上。优秀的上下文工程是实现 Agent 行为可靠性的基础。 436 | 437 | 相关资源： 438 | 439 | - https://docs.spring.io/spring-ai/reference/api/structured-output-converter.html 440 | - Agentic Design Patterns： https://docs.google.com/document/d/1rsaK53T3Lg5KoGwvf8ukOUvbELRtH-V0LnOIFDxBryE/edit?tab=t.0 441 | - Agentic Context Engineering: https://www.arxiv.org/pdf/2510.04618 442 | - A Survey on Large Language Model based Autonomous Agents: https://arxiv.org/pdf/2308.11432 443 | - [Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) 444 | - [AGENTIC RETRIEVAL-AUGMENTED GENERATION: A SURVEY ON AGENTIC RAG](https://arxiv.org/pdf/2501.09136) 445 | - [How to build reliable AI workflows with agentic primitives and context engineering](https://github.blog/ai-and-ml/github-copilot/how-to-build-reliable-ai-workflows-with-agentic-primitives-and-context-engineering/) 446 | -------------------------------------------------------------------------------- /images/Understanding-GitHub-Copilots-Internal-Architecture.svg: -------------------------------------------------------------------------------- 1 | GitHub Copilot Internal Architecture in VS Code 1. VS Code Environment1.1 User Interface Layer1.1💬 ChatPanel1.1a📝 CodeEditor1.1b💻 TerminalInterface1.1c✨ InlineCompletions1.1d📊 ProblemsPanel1.1e1.2 Extension Host Layer1.2🤖 CopilotExtensionMain Process1.2a🔧 LanguageServerProtocol1.2b🔌 ExtensionAPIBridge1.2c👁️ FileWatcher1.2d1.3 Internal Tool System1.31.3.0 🎯 Tool Orchestrator - Decides which tools to execute1.3.1 File Operations:📖 read_fileRead ranges1.3.1a➕ create_fileNew files1.3.1b✏️ edit_fileSmart edit1.3.1c🔄 replaceExact replace1.3.1d1.3.2 Search & Analysis:🧠 semanticAI search1.3.2a🔍 grepText search1.3.2b🔗 usagesReferences1.3.2c🐛 errorsGet errors1.3.2d1.3.3 Execution:💻 run_in_terminalExecute commands1.3.3a⚙️ create_taskVS Code tasks1.3.3b🌐 externalWeb/GitHub API1.3.3c1.4 Local File System1.4📁 Project Files.js, .py, .ts1.4a📚 Git RepoVersion history1.4b⚙️ Config Filespackage.json1.4c🏗️ Build Outputdist/, build/1.4d2. Microsoft AI Cloud Services22.1 AI Processing Pipeline2.1🎯 RequestRouter2.1a🧠 ContextAnalyzer2.1b🎭 IntentClassifier2.1c🛠️ ToolSelector2.1d2.2 AI Models2.2🤖 Code Generation• Inline completions• Code understanding• Pattern recognition2.2a💬 Chat Model• Natural language• Conversation flow• Code explanation2.2b🧩 Semantic Analysis• Code relationships• Context understanding• Semantic search2.2c2.3 Security & Privacy Layer2.3🔐 EncryptionEnd-to-end securedata transmission2.3a🛡️ Privacy FiltersRemove sensitivedata automatically2.3b📊 TelemetryAnonymous usageanalytics only2.3c3. External Data Sources3🐙 GitHub RepositoriesPublic code patternsand examples3a📚 DocumentationAPI docs, tutorials,best practices3b🌐 Web ContentStack Overflow,blogs, forums3cUser Input🔐 EncryptedRequest🔐 EncryptedResponse🔑 Key Points • Tools are internal APIs - not directly accessible to users • All data transmission is encrypted and privacy-filtered • No permanent storage of your code in the cloud • AI models choose appropriate tools based on context and intent • Local file system access is read-only unless explicitly editing ✓Active & Secure --------------------------------------------------------------------------------