├── LICENSE └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 David Shapiro 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Hierarchical Memory Consolidation System (HMCS) 2 | 3 | The Hierarchical Memory Consolidation System (HMCS) is a memory management and organization system designed to handle large-scale memory storage and retrieval efficiently. It is inspired by human memory processes and aims to mimic human-like memory organization and recall. The system is intended for use with autonomous cognitive entities (ACEs) to manage and optimize their internal memory. 4 | 5 | Other names that may be easier: 6 | 7 | - Adaptive Knowledge Archive (AKA) 8 | - Rolling Episodic Memory Organizer (REMO) 9 | - Summarized Knowledge Integration System (SKIS) 10 | - Efficient Memory Aggregation and Retrieval (EMAR) 11 | 12 | ## Overview 13 | 14 | The HMCS consists of several key components and processes: 15 | 16 | 1. Log-based memory: Records thoughts, inputs, and outputs as individual logs. 17 | 2. Rollup summaries: Periodically consolidates logs into higher-level summaries to reduce memory load. 18 | 3. KB articles: Organizes rollup summaries based on semantic similarity to create or update knowledge base (KB) articles. 19 | 4. Clustering or gating functions: Determines topical boundaries for efficient storage and retrieval of information. 20 | 5. Scalability: Adapts memory management to accommodate growing data volumes. 21 | 6. Periodic reindexing events: Optimizes memory by reorganizing and pruning the hierarchy as needed. 22 | 23 | ## Theory and Reasoning 24 | 25 | The Hierarchical Memory Consolidation System (HMCS) is built upon principles from cognitive science, information theory, and computational linguistics. The primary goal is to create a memory management system that resembles human-like memory organization and recall, allowing for efficient storage and retrieval of information. 26 | 27 | ### Cognitive Science 28 | 29 | Human memory is organized hierarchically, with information being stored and retrieved at different levels of abstraction. This hierarchical structure allows humans to recall information efficiently by navigating through various levels of detail, rather than searching through a large, unstructured database of memories. HMCS aims to replicate this hierarchical structure by creating a system that organizes memory in a similar fashion. 30 | 31 | ### Information Theory 32 | 33 | The concept of compression plays a crucial role in the design of the HMCS. By consolidating logs into rollup summaries, the system reduces the amount of data required to represent the information. This process is similar to data compression techniques used in information theory, where redundant or less important information is discarded to create a more compact representation. 34 | 35 | ### Computational Linguistics 36 | 37 | In order to organize the memory hierarchically, HMCS relies on computational linguistics techniques, such as semantic similarity, clustering, and gating functions. These methods enable the system to analyze the content of logs and rollup summaries, identify relationships between them, and organize them into meaningful knowledge base articles. 38 | 39 | ### Adaptability and Scalability 40 | 41 | One of the key features of the HMCS is its ability to adapt to growing data volumes. As the number of logs and rollup summaries increases, the system adjusts its hierarchical structure to maintain efficient storage and retrieval. This adaptability is essential for autonomous cognitive entities that need to continuously learn and update their knowledge. 42 | 43 | ### Periodic Reindexing 44 | 45 | To further optimize memory organization, the HMCS incorporates periodic reindexing events. These events involve reassessing the relationships between logs, rollup summaries, and knowledge base articles, and reorganizing the hierarchy as needed. This process ensures that the memory system remains efficient and up-to-date, even as new information is added and old information becomes less relevant. 46 | 47 | By integrating principles from cognitive science, information theory, and computational linguistics, the Hierarchical Memory Consolidation System creates a memory management system that is efficient, adaptable, and scalable, enabling autonomous cognitive entities to manage and optimize their internal memory effectively. 48 | 49 | ## Usage 50 | 51 | The HMCS is designed to be integrated into an ACE's memory system. By using a hierarchical approach, it allows efficient storage and retrieval of large amounts of data. To utilize the system, follow these steps: 52 | 53 | 1. Integrate log-based memory storage into the ACE's memory system. 54 | 2. Implement periodic consolidation of logs into rollup summaries. 55 | 3. Compare rollups based on semantic similarity to create or update KB articles. 56 | 4. Use clustering or gating functions to determine topical boundaries. 57 | 5. Adapt memory management to accommodate growing data volumes. 58 | 6. Incorporate periodic reindexing events to optimize memory organization. 59 | 60 | ## License 61 | 62 | This project is licensed under the MIT License. Please see the LICENSE file for more information. 63 | 64 | ## Contributing 65 | 66 | Contributions are welcome! To contribute, please submit a pull request with your proposed changes. All contributions should adhere to the project's coding standards and should include appropriate documentation and testing. 67 | 68 | Please note that this project adheres to a code of conduct. By participating in this project, you agree to abide by its terms. 69 | --------------------------------------------------------------------------------