18 | Bloated NPM installs with 100+ dependencies and packages, pulling in
19 | 35MB+ of JS generates a bundle size that could lead to poor app
20 | performance in bandwith constrained apps. There is also a lack of
21 | support for other frameworks outside of React.
22 |
23 |
24 |
25 |
26 | Who Is This For?
27 |
28 |
29 | You want extremely low latency, client-side caching with persistent
30 | data storage. You want a high cache hit-rate from a lazy loading
31 | implementation to improve performance and save system resources. You
32 | want to set up ALL OF THIS in seconds.
33 |
34 |
35 |
36 |
37 |
38 |
39 | Tech Stack
40 |
41 |
42 | In order to provide the developer with very efficient caching for
43 | their GraphQL queries, LightQL implements a modern tech stack. Cache
44 | your queries with us to leverage these technologies and improve your
45 | application performance!
46 |
36 | LightQL is an open-source developer tool that leverages the
37 | pinpoint accuracy of GraphQL's queries and implements caching to
38 | improve your website's query efficiency.
39 |
40 |
41 |
42 |
43 |
Prerequisites:
44 |
45 |
GraphQL schemas setup with your database.
46 |
47 | Fullstack Application where frontend makes query request to
48 | backend.
49 |
50 |
51 |
52 |
53 |
54 |
55 |
Getting Started
56 |
57 | If this is your first time using LightQL, run the following
58 | command in your terminal:
59 |
60 |
61 |
npm install lightql-cache
62 |
72 |
73 |
74 |
75 | In your frontend app’s file (e.g. your ***filename.js*** file),
76 | you want to import our LightQL module to handle GraphQL requests
77 | using the ES6 module format. This can also be done in your React
78 | (.jsx), Typescript (.ts and .tsx), and similar file formats.
79 |
80 |
81 |
82 |
{importStr}
83 |
93 |
94 |
95 |
96 | Next, create an instance of a cache using the de-structured
97 | LRUCache object, passing in a capacity as the first argument.
98 | The capacity must be an integer and greater than zero. You must
99 | also pass in a valid GraphQL endpoint as a string as the second
100 | argument. We have set a capacity of 3 in our example below:
101 |
121 | Now, to make your first query to the cache, you create a GraphQL
122 | formatted query string based on your requirements, for example:
123 |
124 |
125 |
126 | {graphqlQueryStr}
127 |
128 |
129 |
130 |
131 | Next, we invoke the get function associated with the named
132 | variable you have for the LRUCache, and pass in your query
133 | string and associate variables if necessary. The get function
134 | always returns a promise, therefore it is best to generate an
135 | async function that leverages the await syntax to resolve the
136 | data returned from the cache.
137 |
138 |
139 |
140 |
{callTheCache}
141 |
153 |
154 |
155 |
156 | Now, you are properly set up and can use the data as you wish!
157 |
158 |
159 |
160 | A quick example: imagine you had a React app that included
161 | Chart.js functionality that you wanted to display on a specific
162 | page. You could import LightQL cache to effectively retrieve
163 | your data from your database, and then store it in your
164 | client-side LightQL caching solution. Then, every time you
165 | wanted to display the correct chart.js data, you could grab the
166 | correct information from your LightQL cache with extremely low
167 | latency time. Example code below:
168 |
169 |
170 |
171 |
172 |
173 |
174 |
175 |
Technology Stack
176 |
177 |
GraphQL
178 |
Typescript
179 |
Node/Express
180 |
AWS RDS
181 |
React
182 |
Chart.js
183 |
Jest/Supertest
184 |
Webpack
185 |
186 |
187 |
188 |
189 |
190 |
191 | >
192 | );
193 | };
194 |
195 | export default Docs;
196 |
197 | // changes made during ts transition
198 | // added React.FC type
--------------------------------------------------------------------------------
/npm-package/lightql.ts:
--------------------------------------------------------------------------------
1 | /*localForage is a fast and simple storage library for Javascript. localForage leverages asynchronous storage thorugh IndexedDB, a JS-based object-oriented database that runs in your browser, with a simple, localStorage-like API. LightQL leverages localForage and IndexedDB to persist cached data between sessions. Whenever you run a query through LightQL, the capacity, HashMap, Doubly-Linked List, and graphqlEndpoint are loaded into memory if available through the localForage method: localforage.setItem(). Additionally, before returning data to users, LighQL writes our cache data stuctures and graphqlEndpoint to our persistent memory in IndexedDB through the localForage method: localforage.getItem().
2 | */
3 | const localforage = require("localforage");
4 |
5 | //LRUCache function creates an instance of the LightQL cache. A capacity and graphQL endpoint are necessary arguments to pass in.
6 | function LRUCache(capacity, graphqlEndpoint) {
7 | this.capacity = Math.floor(capacity);
8 | this.map = new Map();
9 | this.dll = new DoublyLinkedList();
10 | this.graphqlEndpoint = graphqlEndpoint;
11 | }
12 |
13 | // Edge case handler to check if hashmap and dll are the same size:
14 | LRUCache.prototype.equalSize = function() {
15 | return this.map.size === this.dll.currCapacity;
16 | };
17 |
18 |
19 | //This function retrieves relevant data structures (this.map, this.dll, etc) stored in the IndexedDB browser storage and maps them with the corresponding structures in our LighQL cache solution using localForage.getItem()
20 | LRUCache.prototype.getIDBCache = async function () {
21 | await localforage.getItem('LightQL', (err, value) => {
22 | if(err) {
23 | return false;
24 | } else {
25 | if(!value) {
26 | return false;
27 | }
28 | else {
29 | this.capacity = value.capacity;
30 | this.graphqlEndpoint = value.graphqlEndpoint;
31 | //When data structures (this.dll, this.map) are saved to IndexedDB the underlying methods that allow these structures to function are not saved in IndexDB. As a result, a new instance of each structure must be created and updated with the data from IndexedDB
32 | this.map = new Map();
33 | value.map.forEach((val, query) => {
34 | this.map.set(query, val);
35 | })
36 | this.dll = new DoublyLinkedList();
37 | let currNode = value.dll.head;
38 | while(currNode){
39 | this.dll.add(currNode);
40 | currNode = currNode.next;
41 | }
42 | return true;
43 | }
44 | }
45 | })
46 | }
47 |
48 |
49 | //A function to save the current LighQL cache data structures to IndexedDB using localforage.setItem()
50 | LRUCache.prototype.saveIDBCache = async function (){
51 | let data = {
52 | capacity : this.capacity,
53 | map : this.map,
54 | dll : this.dll,
55 | graphqlEndpoint: this.graphqlEndpoint
56 | };
57 | await localforage.setItem('LightQL', data, (err, value) => {
58 | if (err) {
59 | return false;
60 | } else {
61 | return true;
62 | }
63 | });
64 | }
65 |
66 | //A function that allows the user to request data for a specific graphQL query. It implements our Cache's LRU eviction policy and Lazy-Loading caching pattern
67 | LRUCache.prototype.get = async function(query, variables) {
68 | //Writes LightQL Cache to IndexedDB
69 | await this.saveIDBCache();
70 | //If our Cache is not empty, pull relevant data structures stored in IndexedDB into our LightQL cache
71 | await this.getIDBCache();
72 |
73 | //Determines whether this.map and this.DLL have the same number of nodes and throws an error if not
74 | if (this.equalSize() === false) {
75 | throw new Error('Hashmap and linked list are out of sync and no longer have the same number of nodes');
76 | }
77 | //Checks if the grapQL endpoint is valid and throws an error if not
78 | if(!this.graphqlEndpoint){
79 | throw new Error('Graphql Endpoint Argument is invalid or missing')
80 | }
81 | //Checks capacity constraints : capacity must be an integer and greater than 0.
82 | if(this.capacity <= 0 || !this.capacity || typeof this.capacity !== 'number') {
83 | throw new Error('Capacity is invalid')
84 | }
85 |
86 | /* LAZY LOADING IMPLEMENTATION
87 | Lazy loading is a caching strategy that loads data into the cache only when necessary to improve performance and save system resources.
88 | When making a specific query, the application will hit the cache first; if the data for the query is present, the cache returns the data to your application. If the data is missing, then the cache will be responsible for pulling the data from the main database/data store. The returned data will then be written to the cache, and can quickly be retrieved the next time it's requested.
89 | */
90 | if (this.map.has(query)) {
91 | let currNode = this.map.get(query);
92 | this.dll.remove(currNode);
93 | this.dll.add(currNode);
94 | this.saveIDBCache();
95 | return currNode.value;
96 | } else {
97 | return new Promise ((resolve, reject) =>{
98 | fetch(this.graphqlEndpoint, {
99 | method: 'POST',
100 | headers: {'Content-type' : 'application/json',
101 | 'Accept' : 'application/json',
102 | },
103 | body: JSON.stringify({
104 | query: query,
105 | variables : {variables}
106 | })
107 | })
108 | .then((res) => res.json())
109 | .then((data) => {
110 | const actualData = data.data;
111 | this.put(query, actualData);
112 | this.saveIDBCache();
113 | resolve(actualData);
114 | })
115 | .catch((err) => console.log(`Error in data fetch: ` + err));
116 | });
117 | }
118 | };
119 |
120 |
121 | //A function that stores new queries and associated data into our LRU Cache. It maintains consistency between our underlying HashMap structure and Doubly-Linked List
122 | LRUCache.prototype.put = function (query, value) {
123 | if (this.equalSize() === false) {
124 | return console.log('Check hashmap and linked list');
125 | }
126 | if (this.map.has(query)) {
127 | //Get the current node from the list and save in a temp variable: currNode
128 | let currNode = this.map.get(query);
129 | //Delete the node from the Doubly-Linked list
130 | this.dll.remove(currNode);
131 | //Delete the node from the HashMap
132 | this.map.delete(query);
133 | //Update the currNode's value to reflect fresh data returned from the query
134 | currNode.value = value;
135 | //Add the node back to the DLL
136 | this.dll.add(currNode);
137 | //Add the node to the HashMap
138 | this.map.set(query, currNode);
139 | return;
140 | } else {
141 | // Case if the cache doesn't contain the requested Query
142 | // Case if the cache is at capacity: LRU Eviction is implemented here!
143 | if (this.map.size === this.capacity) {
144 | //Find the node at the tail of the DLL
145 | let tempTail = this.dll.tail;
146 | //Save the query from the tail DLL Node
147 | let tailToDeletequery = tempTail.query;
148 | //Delete the tail node from the DLL
149 | this.dll.remove(tempTail);
150 | //Remove the tail node from the HashMap
151 | this.map.delete(tailToDeletequery);
152 | //now we have free space in the HashMap and the DLL to add the new node
153 | }
154 | //Create a new node with the value and query passed in
155 | const newNode = new DLLNode(query, value);
156 | //Add the node to the DLL
157 | this.dll.add(newNode);
158 | //Add the node to the HashMap
159 | this.map.set(query, newNode);
160 | return;
161 | }
162 | };
163 |
164 | //Function to create a Doubly Linked-List node
165 | const DLLNode = function (query, value) {
166 | this.query = query;
167 | this.value = value;
168 | this.next = null;
169 | this.prev = null;
170 | };
171 |
172 | //Function that creates a Doubly Linked-List
173 | const DoublyLinkedList = function () {
174 | this.head = null;
175 | this.tail = null;
176 | this.currCapacity = 0;
177 | };
178 |
179 | //Function for adding nodes to the Doubly Linked-List
180 | DoublyLinkedList.prototype.add = function (node) {
181 | if (!this.head && !this.tail) {
182 | this.head = node;
183 | this.tail = node;
184 | } else {
185 | node.next = this.head;
186 | this.head.prev = node;
187 | this.head = node;
188 | }
189 | this.currCapacity += 1;
190 | return;
191 | };
192 |
193 | //Function for removing nodes from the Doubly Linked-List
194 | DoublyLinkedList.prototype.remove = function (nodeToRemove) {
195 | let curr = this.head;
196 | while (curr) {
197 | if ((curr.value === nodeToRemove.value)) {
198 | if (this.head.value === curr.value) {
199 | if (this.tail.value === this.head.value) {
200 | this.head = this.tail = null;
201 | this.currCapacity--;
202 | return;
203 | }
204 | this.head = curr.next;
205 | this.head.prev = null;
206 | this.currCapacity--;
207 | return;
208 | }
209 | if (this.tail.value === curr.value) {
210 | this.tail = curr.prev;
211 | this.tail.next = null;
212 | this.currCapacity--;
213 | return;
214 | }
215 | curr.prev.next = curr.next;
216 | curr.next.prev = curr.prev;
217 | this.currCapacity--;
218 | return;
219 | }
220 | curr = curr.next;
221 | }
222 | };
223 |
224 | module.exports = { LRUCache, DoublyLinkedList, DLLNode};
225 |
226 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
37 |
38 |
39 |
40 |
41 | ## Overview
42 |
43 | LightQL is an easy-to-use super fast and lightweight Javascript library providing a client-side caching solution for GraphQL. Use LightQL for extremely low latency retrieval of persistent cache data.
44 |
45 |
60 |
61 |
62 |
63 | ## Prerequisites
64 |
65 | - All developers will have to integrate GraphQL into their functionality. This means setting up your GraphQL schemas and resolvers as well as your database.
66 | - Our package is intended to work in tandem with a full-stack application where the frontend makes query requests to the backend, so please set this up as well before using our package.
67 |
68 | ## Demo
69 |
70 | - Head over to the home page of our website ([lightql.com](https://www.lightql.com/))
71 | - Using our demo is a simple 2-step process.
72 | - First, press the “Run the demo” button to see the resulting GraphQL query
73 | result. If you divert your attention to the metrics box down below you
74 | will see an uncached run time populate. This statistic represents the
75 | time it took to make a GraphQL fetch and receive the response data. You
76 | can also see the spike in time upwards on the graph provided on the
77 | right.
78 | - If you press the run query button again you will notice that your
79 | cached run time metric will now render. The graph on the right will also
80 | dive down as the response time has significantly decreased. The uncached run time should never change after this, as we are now retrieving your data from the cache every instance forward showing our super lightning speed of retrieval!
81 |
82 | \*A small disclaimer: It should be noted that your first query will have a significantly higher runtime than the other first queries because it is establishing a connection.
83 |
84 | ## Installation
85 |
86 | 1. If this is your first time using LightQL, run the following command in your terminal:
87 | 2. ```sh
88 | npm install lightql-cache
89 | ```
90 | 3. In your frontend app’s file (e.g. your filename.js file), you want to import our LightQL module to handle GraphQL requests using the ES6 module format. This can also be done in your React (.jsx), Typescript (.ts and .tsx), and similar file formats.
91 | ```js
92 | import { LRUCache, DoublyLinkedList, DLLNode } from ‘lightql-cache’;
93 | ```
94 | 4. Next, create an instance of a cache using the de-structured LRUCache object, passing in a capacity as the first argument. The capacity must be an integer and greater than zero. You must also pass in a valid GraphQL endpoint as a string as the second argument. We have set a capacity of 3 in our example below:
95 | ```js
96 | const cache = new LRUCache(3, 'http://localhost:3000/graphql');
97 | ```
98 | 5. Now, to make your first query to the cache, you create a GraphQL formatted query string based on your requirements, for example:
99 | ```js
100 | const graphqlQueryStr = ` {
101 | user {
102 | user_name,
103 | song_name,
104 | movie_name
105 | }
106 | }`;
107 | ```
108 | 6. Next, we invoke the get function associated with the named variable you have for the LRUCache, and pass in your query string and associate variables if necessary. The get function always returns a promise, therefore it is best to generate an async function that leverages the await syntax to resolve the data returned from the cache.
109 | ```js
110 | const callLightQL = async () => {
111 | const cacheGet = await cache.get(graphqlQueryStr, variables);
112 | };
113 | ```
114 | 7. Now, you are properly set up and can use the data as you wish!
115 | 8. A quick example: imagine you had a React app that included Chart.js functionality that you wanted to display on a specific page. You could import LightQL cache to effectively retrieve your data from your database, and then store it in your client-side LightQL caching solution. Then, every time you wanted to display the correct chart.js data, you could grab the correct information from your LightQL cache with extremely low latency time. Example code below:
116 |
117 |
118 |
119 |
120 |
121 |
122 |
123 | ## How LightQL works
124 |
125 | LightQL caches responses to GraphQL formatted queries as key-value object representations of the graph’s nodes, making it possible to satisfy GraphQL queries from a cached data store.
126 |
127 | When your application needs data, it checks the cache first to determine whether the data is available. If the data is available (a cache hit), the cached data is returned, and the response is issued to the user. If the data isn’t available (a cache miss), the database is queried for the data. The cache is then populated with the data that is retrieved from the database, and the data is returned to the user. The benefit of this strategy is that the cache contains only data that the application actually requests, keeping the cache size cost-effective. Further, this increases cache hits, reduces network calls, and significantly reduces the overall runtime and latency.
128 |
129 | LightQL’s LRUCache function creates an instance of the LightQL cache. A capacity and GraphQL endpoint are the two necessary arguments to pass in to this function. The LRUCache consists of a HashMap and Doubly-Linked List to store GraphQL query responses. The combination of these two data structures offer best-case scenario time complexity (O(1)) for insertion, deletion, and lookup.
130 |
131 | ```js
132 | function LRUCache(capacity, graphqlEndpoint) {
133 | this.capacity = Math.floor(capacity);
134 | this.map = new Map();
135 | this.dll = new DoublyLinkedList();
136 | this.graphqlEndpoint = graphqlEndpoint;
137 | }
138 | ```
139 |
140 | LightQL leverages localForage and IndexedDB to persist cached data between sessions. localForage is a fast and simple storage library for Javascript. localForage leverages asynchronous storage through IndexedDB, a JS-based object-oriented database that runs in your browser, with a simple, localStorage-like API. Whenever you run a query through LightQL, the capacity, HashMap, Doubly-Linked List, and graphqlEndpoint are loaded into memory if available through the localForage setItem method:
141 |
142 | ```js
143 | localforage.setItem();
144 | ```
145 |
146 | Additionally, before returning data to users, LightQL writes our cache data structures and graphqlEndpoint to our persistent memory in IndexedDB through the localForage getItem method:
147 |
148 | ```js
149 | localforage.getItem();
150 | ```
151 |
152 | Developers can entrust LightQL to handle their GraphQL caching needs simply and effectively, so they can focus on working what matters most.
153 |
154 |
167 |
168 |
169 |
170 | ## Contributing
171 |
172 | Here at LightQL we created our open-source project with the intention to further expand upon and improve this tool for years to come.
173 |
174 | That's where the community comes in! If you have an idea that might make LightQL better we always encourage contributions. Simply follow the steps below to submit the changes you would make.
175 |
176 | - Fork LightQL
177 | - Pull down our dev branch with command (`git pull origin dev`)
178 | - Create your own Feature Branch with the command (`git checkout -b `)
179 | - Add your changes with the command (`git add .`)
180 | - Stage and commit your changes with the command (`git commit -m ""`)
181 | - Merge your branch with the dev branch locally with the command (`git merge dev`)
182 | - Resolve any merge conflicts
183 | - Push up your branch with the command (`git push origin `)
184 | - Open a pull request
185 | - Don't forget to star this repo! We look forward to your contributions!
186 |
187 |