├── src
├── Lumina.res
└── Lumina - Local Generative AI.groupproj
├── media
├── delphi.png
└── lumina.png
├── examples
└── testbed
│ ├── Testbed.dpr
│ ├── Testbed.res
│ ├── Testbed_Icon.ico
│ ├── UTestbed.pas
│ └── Testbed.dproj
├── virustotal.txt
├── .github
└── FUNDING.yml
├── LICENSE
├── .gitignore
└── README.md
/src/Lumina.res:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tinyBigGAMES/Lumina/HEAD/src/Lumina.res
--------------------------------------------------------------------------------
/media/delphi.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tinyBigGAMES/Lumina/HEAD/media/delphi.png
--------------------------------------------------------------------------------
/media/lumina.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tinyBigGAMES/Lumina/HEAD/media/lumina.png
--------------------------------------------------------------------------------
/examples/testbed/Testbed.dpr:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tinyBigGAMES/Lumina/HEAD/examples/testbed/Testbed.dpr
--------------------------------------------------------------------------------
/examples/testbed/Testbed.res:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tinyBigGAMES/Lumina/HEAD/examples/testbed/Testbed.res
--------------------------------------------------------------------------------
/examples/testbed/Testbed_Icon.ico:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/tinyBigGAMES/Lumina/HEAD/examples/testbed/Testbed_Icon.ico
--------------------------------------------------------------------------------
/virustotal.txt:
--------------------------------------------------------------------------------
1 | VirtusTotal report for Lumina Project
2 | =====================================
3 |
4 | Lumina.res: https://www.virustotal.com/gui/file/09dc6958d085c412dd8c6ec31f610700795f051f684fedff1cfe47096da70879
5 |
--------------------------------------------------------------------------------
/.github/FUNDING.yml:
--------------------------------------------------------------------------------
1 | # These are supported funding model platforms
2 |
3 | github: tinyBigGAMES # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
4 | patreon: # Replace with a single Patreon username
5 | open_collective: # Replace with a single Open Collective username
6 | ko_fi: # Replace with a single Ko-fi username
7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
9 | liberapay: # Replace with a single Liberapay username
10 | issuehunt: # Replace with a single IssueHunt username
11 | lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
12 | polar: # Replace with a single Polar username
13 | buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
14 | thanks_dev: # Replace with a single thanks.dev username
15 | custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
16 |
--------------------------------------------------------------------------------
/src/Lumina - Local Generative AI.groupproj:
--------------------------------------------------------------------------------
1 |
2 |
3 | {4EA3BFB9-BDA4-4113-883A-17BBCD36DA33}
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 | Default.Personality.12
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
32 |
33 |
34 |
35 |
36 |
37 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | BSD 3-Clause License
2 |
3 | Copyright (c) 2024-present, tinyBigGAMES LLC
4 |
5 | Redistribution and use in source and binary forms, with or without
6 | modification, are permitted provided that the following conditions are met:
7 |
8 | 1. Redistributions of source code must retain the above copyright notice, this
9 | list of conditions and the following disclaimer.
10 |
11 | 2. Redistributions in binary form must reproduce the above copyright notice,
12 | this list of conditions and the following disclaimer in the documentation
13 | and/or other materials provided with the distribution.
14 |
15 | 3. Neither the name of the copyright holder nor the names of its
16 | contributors may be used to endorse or promote products derived from
17 | this software without specific prior written permission.
18 |
19 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | # Uncomment these types if you want even more clean repository. But be careful.
2 | # It can make harm to an existing project source. Read explanations below.
3 | #
4 | # Resource files are binaries containing manifest, project icon and version info.
5 | # They can not be viewed as text or compared by diff-tools. Consider replacing them with .rc files.
6 | #*.res
7 | #
8 | # Type library file (binary). In old Delphi versions it should be stored.
9 | # Since Delphi 2009 it is produced from .ridl file and can safely be ignored.
10 | #*.tlb
11 | #
12 | # Diagram Portfolio file. Used by the diagram editor up to Delphi 7.
13 | # Uncomment this if you are not using diagrams or use newer Delphi version.
14 | #*.ddp
15 | #
16 | # Visual LiveBindings file. Added in Delphi XE2.
17 | # Uncomment this if you are not using LiveBindings Designer.
18 | #*.vlb
19 | #
20 | # Deployment Manager configuration file for your project. Added in Delphi XE2.
21 | # Uncomment this if it is not mobile development and you do not use remote debug feature.
22 | #*.deployproj
23 | #
24 | # C++ object files produced when C/C++ Output file generation is configured.
25 | # Uncomment this if you are not using external objects (zlib library for example).
26 | #*.obj
27 | #
28 |
29 | # Default Delphi compiler directories
30 | # Content of this directories are generated with each Compile/Construct of a project.
31 | # Most of the time, files here have not there place in a code repository.
32 | #Win32/
33 | #Win64/
34 | #OSX64/
35 | #OSXARM64/
36 | #Android/
37 | #Android64/
38 | #iOSDevice64/
39 | #Linux64/
40 |
41 | # Delphi compiler-generated binaries (safe to delete)
42 | *.exe
43 | *.dll
44 | *.bpl
45 | *.bpi
46 | *.dcp
47 | *.so
48 | *.apk
49 | *.drc
50 | *.map
51 | *.dres
52 | *.rsm
53 | *.tds
54 | *.dcu
55 | *.lib
56 | *.a
57 | *.o
58 | *.ocx
59 |
60 | # Delphi autogenerated files (duplicated info)
61 | *.cfg
62 | *.hpp
63 | *Resource.rc
64 |
65 | # Delphi local files (user-specific info)
66 | *.local
67 | *.identcache
68 | *.projdata
69 | *.tvsconfig
70 | *.dsk
71 |
72 | # Delphi history and backups
73 | __history/
74 | __recovery/
75 | *.~*
76 |
77 | # Castalia statistics file (since XE7 Castalia is distributed with Delphi)
78 | *.stat
79 |
80 | # Boss dependency manager vendor folder https://github.com/HashLoad/boss
81 | modules/
82 | zip_latest_commit.cmd
83 | Lumina.zip
84 | *.dsv
85 | examples/testbed/Testbed.dsv
86 | bin/example.db
87 | bin/test02.db
88 |
89 | bin/test04.db
90 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | 
2 | [](https://discord.gg/tPWjMwK)
3 | [](https://bsky.app/profile/tinybiggames.com)
4 |
5 | # 🌟 Lumina: Advanced Local Generative AI for Delphi Developers 💻🤖
6 |
7 | Lumina offers a cutting-edge 🛠️ for Delphi developers to seamlessly integrate advanced generative AI capabilities into their 📱. Built on the computational backbone of **llama.cpp** 🐪, Lumina prioritizes data privacy 🔒, performance ⚡, and a user-friendly API 📚, making it a powerful tool for local AI inference 🤖.
8 |
9 | ## 🧐 Why Choose Lumina?
10 |
11 | - **Localized Processing** 🏠: Operates entirely offline, ensuring sensitive data remains confidential 🛡️ while offering complete computational control 🧠.
12 | - **Broad Model Compatibility** 🌐: Supports **GGUF models** compliant with llama.cpp standards, granting access to diverse AI architectures 🧩.
13 | - **Intuitive Development Interface** 🎛️: A concise, flexible API simplifies model management 🗂️, inference execution 🧮, and callback customization 🎚️, minimizing implementation complexity.
14 | - **Future-Ready Scalability** 🚀: This release emphasizes stability 🏗️ and foundational features, with plans for multi-turn conversation 💬 and retrieval-augmented generation (RAG) 🔍 in future updates.
15 |
16 | ## 🛠️ Key Functionalities
17 |
18 | ### 🤖 Advanced AI Integration
19 |
20 | Lumina expands your development toolkit 🎒 with capabilities such as:
21 | - Dynamic chatbot creation 💬.
22 | - Automated text generation 📝 and summarization 📰.
23 | - Context-sensitive content generation ✍️.
24 | - Real-time inference for adaptive processes ⚡.
25 |
26 | ### 🔒 Privacy-Driven AI Execution
27 |
28 | - Operates independently of external networks 🛡️, guaranteeing data security.
29 | - Uses Vulkan 🖥️ for optional GPU acceleration to enhance performance.
30 |
31 | ### ⚙️ Performance Optimization
32 |
33 | - Configurable GPU utilization through the `AGPULayers` parameter 🧩.
34 | - Dynamic thread allocation based on hardware capabilities 🖥️ via `AMaxThreads`.
35 | - Comprehensive performance metrics 📊, offering insights into throughput 📈 and efficiency.
36 |
37 | ### 🔗 Streamlined Integration
38 |
39 | - Embedded dependencies eliminate the need for external libraries 📦.
40 | - Lightweight architecture (~2.5MB overhead) ensures broad deployment compatibility 🌍.
41 |
42 | ## 📥 Installation
43 |
44 | 1. **Download the Repository** 📦
45 | - [Download here](https://github.com/tinyBigGAMES/Lumina/archive/refs/heads/main.zip) and extract the files to your preferred directory 📂.
46 |
47 | 2. **Acquire a GGUF Model** 🧠
48 | - Obtain a model from [Hugging Face](https://huggingface.co), such as [Gemma 2.2B GGUF (Q8_0)](https://huggingface.co/bartowski/gemma-2-2b-it-abliterated-GGUF/resolve/main/gemma-2-2b-it-abliterated-Q8_0.gguf?download=true). Save it to a directory accessible to your application (e.g., `C:/LLM/GGUF`) 💾.
49 |
50 | 3. **Ensure GPU Compatibility** 🎮
51 | - Verify Vulkan compatibility for enhanced performance ⚡. Adjust `AGPULayers` as needed to accommodate VRAM limitations 📉.
52 |
53 | 4. **✨ TLumina Class**
54 | - 📜 Add `Lumina` to your `uses` section.
55 | - 🛠️ Create an instance of `TLumina`.
56 | - 🚀 All functionality will then be at your disposal. That simple! 🎉
57 |
58 | 5. **Explore Examples** 🔍
59 | - Check the `examples` directory for detailed usage demonstrations 📚.
60 |
61 | ## 🛠️ Usage
62 |
63 | ### 🔧 Basic Setup
64 |
65 | Integrate Lumina into your Delphi project 🖥️:
66 |
67 | ```delphi
68 | var
69 | Lumina: TLumina;
70 | begin
71 | Lumina := TLumina.Create;
72 | try
73 | if Lumina.LoadModel('C:\LLM\GGUF\gemma-2-2b-it-abliterated-Q8_0.gguf',
74 | '', 8192, -1, 8) then
75 | begin
76 | if Lumina.SimpleInference('What is the capital of Italy?') then
77 | WriteLn('Inference completed successfully.')
78 | else
79 | WriteLn('Error: ', Lumina.GetError);
80 | end;
81 | finally
82 | Lumina.Free;
83 | end;
84 | end;
85 | ```
86 |
87 | ### 🎚️ Customizing Callbacks
88 |
89 | Define custom behavior using Lumina’s callback functions 🛠️:
90 |
91 | ```delphi
92 | procedure NextTokenCallback(const AToken: string; const AUserData: Pointer);
93 | begin
94 | Write(AToken);
95 | end;
96 |
97 | Lumina.SetNextTokenCallback(NextTokenCallback, nil);
98 | ```
99 |
100 | ## 📖 API Reference
101 |
102 | ### 🧩 Core Methods
103 |
104 | - **LoadModel** 📂
105 | - Parameters:
106 | - `AModelFilename`: Path to the GGUF model file 📄.
107 | - `ATemplate`: Optional inference template 📝.
108 | - `AMaxContext`: Maximum context size (default: 512) 🧠.
109 | - `AGPULayers`: GPU layer configuration (-1 for maximum) 🎮.
110 | - `AMaxThreads`: Number of CPU threads allocated 🖥️.
111 | - Returns a boolean indicating success ✅.
112 |
113 | - **SimpleInference** 🧠
114 | - Accepts a single query for immediate processing 📝.
115 | - Returns a boolean indicating success ✅.
116 |
117 | - **SetNextTokenCallback** 💬
118 | - Assigns a handler to process tokens during inference 🧩.
119 |
120 | - **UnloadModel** ❌
121 | - Frees resources allocated during model loading 🗑️.
122 |
123 | - **GetPerformanceResult** 📊
124 | - Provides metrics, including token generation rates 📈.
125 |
126 | ## 🛠️ Advanced Configurations
127 |
128 | ### 🧠 Custom Inference Templates
129 |
130 | Lumina will use the template defined in the model's meta data by default, but you can also define custom templates to match your model’s requirements or change its behavor. These are some common model templates ✍️:
131 |
132 | ```delphi
133 | const
134 | CHATML_TEMPLATE = '<|im_start|>{role} {content}<|im_end|><|im_start|>assistant';
135 | GEMMA_TEMPLATE = '{role} {content}';
136 | PHI_TEMPLATE = '<|{role}|> {content}<|end|><|assistant|>';
137 | ```
138 |
139 | - **{role}** - will be replaced with the role (user, assistant, etc.)
140 | - **{content}** - will be replaced with the content sent to the model
141 |
142 | ### 🎮 GPU Optimization
143 |
144 | - `AGPULayers` values:
145 | - `-1`: Utilize all available layers (default) 🖥️.
146 | - `0`: CPU-only processing 🖥️.
147 | - Custom values for partial GPU utilization 🎛️.
148 |
149 | ### 📊 Performance Metrics
150 |
151 | Retrieve detailed operational metrics 📈:
152 |
153 | ```delphi
154 | var
155 | Perf: TLumina.PerformanceResult;
156 | begin
157 | Perf := Lumina.GetPerformanceResult;
158 | WriteLn('Tokens/Sec: ', Perf.TokensPerSecond);
159 | WriteLn('Input Tokens: ', Perf.TotalInputTokens);
160 | WriteLn('Output Tokens: ', Perf.TotalOutputTokens);
161 | end;
162 | ```
163 |
164 | ## 🎙️ Media
165 |
166 | ### 🌊 Deep Dive Podcast
167 | Discover in-depth discussions and insights about **Lumina** and its innovative features. 🚀✨
168 |
169 | https://github.com/user-attachments/assets/165e3dee-b29f-4478-b9ef-4fb6d2df2485
170 |
171 |
172 | ### 🛠️ Support and Resources
173 |
174 | - Report issues via the [Issue Tracker](https://github.com/tinyBigGAMES/Lumina/issues) 🐞.
175 | - Engage in discussions on the [Forum](https://github.com/tinyBigGAMES/Lumina/discussions) and [Discord](https://discord.gg/tPWjMwK) 💬.
176 | - Learn more at [Learn Delphi](https://learndelphi.org) 📚.
177 |
178 | ### 🤝 Contributing
179 |
180 | Contributions to **✨ Lumina** are highly encouraged! 🌟
181 | - 🐛 **Report Issues:** Submit issues if you encounter bugs or need help.
182 | - 💡 **Suggest Features:** Share your ideas to make **Lumina** even better.
183 | - 🔧 **Create Pull Requests:** Help expand the capabilities and robustness of the library.
184 |
185 | Your contributions make a difference! 🙌✨
186 |
187 | #### Contributors 👥🤝
188 |
189 |
190 |
191 |
192 |
193 |
194 | ### 📜 Licensing
195 |
196 | **Lumina** is distributed under the 🆓 **BSD-3-Clause License**, allowing for redistribution and use in both source and binary forms, with or without modification, under specific conditions. See the [LICENSE](https://github.com/tinyBigGAMES/Lumina?tab=BSD-3-Clause-1-ov-file#BSD-3-Clause-1-ov-file) file for more details.
197 |
198 | ---
199 |
200 | Advance your Delphi applications with Lumina 🌟 – a sophisticated solution for integrating local generative AI 🤖.
201 |
202 |