├── .gitignore ├── data └── README.md ├── LICENSE ├── README.md └── 7rolldb /.gitignore: -------------------------------------------------------------------------------- 1 | data/* 2 | !data/README.md 3 | -------------------------------------------------------------------------------- /data/README.md: -------------------------------------------------------------------------------- 1 | This directory is for the database storage. 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Thomas Backlund 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | #7RollDB - The Non Bloated Database Engine - in Bash 2 | A MongoDB replacement written in Bash. 3 | 4 | A novel database engine for courageous users. 5 | 6 | ##Features of 7RollDB 7 | * Free and open source 8 | * JSON document database engine 9 | * Strict Non Bloating Policy 10 | * Super lightweight - only ~7kb in size 11 | * Built in REST API 12 | * Written in Bash - the super popular shell language supported everywhere 13 | * Unlimited scalability - 14 | * Connect 7RollDB to your Dropbox and enjoy limitless scalability for life 15 | * MongoDB port compatible (uses 27017) 16 | * Growing community 17 | 18 | ##Non Bloating policy 19 | While MongoDB started out as a sleek shut-the-fcuk-up database engine, it has now 20 | more or less conformed to traditional database technologies. 21 | 22 | It used to be super lean without transaction logs or even write confirmations, 23 | but somewhere along the line of playing with the big boys n' girls, they completely lost the ball and their ways of NoSQL. 24 | 25 | 7RollDB is filling that vacuum for a novel database engine and its courageous users. 26 | 27 | 7RollDB adhere to our strict **Non Bloating Policy** of never getting bloated on the users expense. 28 | 29 | 30 | ##Comparing 7RollDB to MongoDB* 31 | | 7RollDB | MongoDB 32 | ----------- | --------- | ------------- 33 | Non Bloating Policy | Strict | None 34 | Sleak | Yes | No 35 | Bash | Yes | No 36 | REST API | Native | No (depends on 3rd party) 37 | RAM friendly| Yes | No (uses memory mapped files) 38 | Performant | Yes | Yes 39 | Scalable | Yes | Yes 40 | 41 | \* I have never actually used MongoDB 42 | 43 | ##HTTP REST API 44 | 7RollDB comes with a fully featured expandable REST API to handle your JSON documents. 45 | 46 | ###Databases 47 | Create new database: 48 | ```sh 49 | $ curl -X POST "http://127.0.0.1:27017/MyDB1" 50 | {status: "OK"} 51 | ``` 52 | List all databases: 53 | ```sh 54 | $ curl -X GET "http://127.0.0.1:27017/" 55 | [ 56 | "MyDB1" 57 | ] 58 | ``` 59 | 60 | ###Collections 61 | Create new collection within database: 62 | ```sh 63 | $ curl -X POST "http://127.0.0.1:27017/MyDB1/MyCollection1" 64 | {status: "OK"} 65 | ``` 66 | List all collections withinin specific database: 67 | ```sh 68 | $ curl -X GET "http://127.0.0.1:27017/MyDB1" 69 | [ 70 | "MyCollection1" 71 | ] 72 | ``` 73 | ###Documents 74 | Create new or update a document within a collection: 75 | ```sh 76 | $ curl -X POST "http://127.0.0.1:27017/MyDB1/MyCollection1/MyDoc1" -d \ 77 | '{"name": "Alexander McRaminov", "occupation": "Plain Cool"}' 78 | {status: "OK"} 79 | ``` 80 | List all documents within specific collection: 81 | ```sh 82 | $ curl -X GET "http://127.0.0.1:27017/MyDB1/MyCollection1" 83 | [ 84 | "MyDoc1" 85 | ] 86 | ``` 87 | Get a specific document: 88 | ```sh 89 | $ curl -X GET "http://127.0.0.1:27017/MyDB1/MyCollection1/MyDoc1" 90 | {"name": "Alexander McRaminov", "occupation": "Plain Cool"} 91 | 92 | ``` 93 | Delete a specific document: 94 | ```sh 95 | $ curl -X DELETE "http://127.0.0.1:27017/MyDB1/MyCollection1/MyDoc1" 96 | {status: "OK"} 97 | ``` 98 | 99 | ###Query and search 100 | 7RollDB uses a very down to earth and proved technique for searching through documents 101 | while still never sidestepping our strict 0 relations and 0 indexing Non Bloating Policy. 102 | 103 | Searching for documents in all collections within a database: 104 | ```sh 105 | $ curl -X GET "http://127.0.0.1:27017/MyDB1?Alexander" 106 | [ 107 | "MyCollection1/MyDoc1" 108 | ] 109 | ``` 110 | Searching for documents within a specific collection: 111 | ```sh 112 | $ curl -X GET "http://127.0.0.1:27017/MyDB1/MyCollection1?McRaminov" 113 | [ 114 | "MyDoc1" 115 | ] 116 | ``` 117 | Searching within a specific document for a match: 118 | ```sh 119 | $ curl -X GET "http://127.0.0.1:27017/MyDB1/MyCollection1/MyDoc1?Cool" 120 | [ 121 | "MyDoc1" 122 | ] 123 | ``` 124 | 125 | ##Installation 126 | 7RollDB depends on some GNU utils and *socat*. Socat is like Netcat++. 127 | ```sh 128 | $ sudo apt-get install socat 129 | or 130 | $ sudo pacman -S socat 131 | or 132 | $ brew install socat 133 | ``` 134 | Clone the 7RollDB repo: 135 | ```sh 136 | $ git clone https://github.com/thomasbacklund/7rolldb.git 137 | cd 7rolldb 138 | ./7rolldb 139 | ``` 140 | See `curl` examples above for how to connect to 7RollDB. 141 | 142 | All `database storage` is within the `./data/` directory. 143 | 144 | ##Contribute 145 | Contributions are welcome. 146 | There is still room for improvement within our strict Non Bloating Policy. 147 | 148 | ##Disclaimer 149 | You will be using 7RollDB at your own risk. 150 | 151 | ##Contact 152 | Get in touch with @ThomasBacklund on Twitter 153 | 154 | ##Donate 155 | You could support the further development of 7RollDB by donating some bitcoins. 156 | 157 | All donations will be used in coherence with our strict Non Bloating Policy. 158 | 159 | Bitcoin address: `17RoLLNFzN24CJCdbRNdA5pJqAWzRzXsze` 160 | -------------------------------------------------------------------------------- /7rolldb: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # 7RollDB - The Non Bloated Database Engine 4 | # github.com/thomasbacklund/7RollDB 5 | # Author: Thomas Backlund 6 | # Released under the MIT license. 7 | # MIT licensed. See LICENSE file for details. 8 | 9 | readonly port=27017 # Same port as MongoDB 10 | readonly version="0.1" 11 | readonly versionName="Sloof Slirpa" 12 | readonly dataDir="data" 13 | 14 | jsonArray() { 15 | a="$1" 16 | data="[]" 17 | if [ ! "$a" = "" ]; then 18 | data=$(echo -n "$a" | sed s/$/\"/ | sed s/^/\"/ | tr '\n' ',\n' | sed "\$a]" | sed "1 s/^\(.*\)/[\n\1/") 19 | fi 20 | response "200" "OK" "$data" 21 | } 22 | 23 | extractHeader() { 24 | method=$(echo -e $1 | head -1 | awk '{ print $1 };' 2>/dev/null) 25 | path=$(echo -e $1 | head -1 | awk '{ print $2 };' | sed 's/\.\.\//\./g' 2>/dev/null) 26 | query=$(echo -e $path | awk 'BEGIN {FS = "?" }; {print $2}') 27 | path=$(echo -e $path | awk 'BEGIN {FS = "?" }; {print $1}') 28 | db=$(echo -e $path | awk 'BEGIN {FS = "/" }; {print $2}') 29 | collection=$(echo -e $path | awk 'BEGIN {FS = "/" }; {print $3}') 30 | docid=$(echo -e $path | awk 'BEGIN {FS = "/" }; {print $4}') 31 | } 32 | 33 | listDatabases() { 34 | # TODO need a security check on the path. 35 | list=$(cd $dataDir 2&> /dev/null && find . -mindepth 1 -maxdepth 1 -type d | sed "s|^\./||") 36 | jsonArray "$list" 37 | } 38 | 39 | listCollections() { 40 | # TODO need a security check on the path. 41 | path=$dataDir/$1 42 | list=$(cd $path 2&> /dev/null && find . -mindepth 1 -maxdepth 1 -type d | sed "s|^\./||") 43 | jsonArray "$list" 44 | } 45 | 46 | listDocuments() { 47 | # TODO need a security check on the path. 48 | path=$dataDir/$1/$2 49 | list=$(cd $path 2&> /dev/null && find . -mindepth 1 -maxdepth 1 -type f -name '*.json' | sed "s|^\./||" | sed "s/\.json\$//") 50 | jsonArray "$list" 51 | } 52 | 53 | getDocument() { 54 | # TODO need a security check on the path. 55 | path=$dataDir/$1/$2/$3.json 56 | if [ -e $path ]; then 57 | echo -ne "HTTP/1.1 200 OK\r\nContent-Length: $(wc -c <$path)\r\n\r\n" 58 | cat $path 59 | else 60 | response "404" "Not Found" 61 | fi 62 | } 63 | 64 | queryDocuments() { 65 | # TODO need a security check on the path. 66 | db=$1 67 | collection=$2 68 | docid=$([ $3 ] && echo "$3.json") 69 | query=$4 70 | path=$dataDir 71 | if [ ! $db = "" ]; then 72 | path=$path/$db 73 | if [ ! $collection = "" ]; then 74 | path=$path/$collection 75 | fi 76 | fi 77 | list=$(cd $path; grep $query $docid -lsr --include="*.json" | sed "s/\.json\$//") 78 | jsonArray "$list" 79 | } 80 | 81 | createDB() { 82 | # TODO need a security check on the path. 83 | path=$dataDir/$1 84 | mkdir $path 2&> /dev/null 85 | response "200" "OK" '{status: "OK"}' 86 | } 87 | 88 | createCollection() { 89 | # TODO need a security check on the path. 90 | path=$dataDir/$1/$2 91 | mkdir $path 2&> /dev/null 92 | response "200" "OK" '{status: "OK"}' 93 | } 94 | 95 | function createDocument { 96 | # TODO need a security check on the path. 97 | path=$dataDir/$1/$2/$3.json 98 | dd bs=1 count="$length" conv=noerror iflag=nonblock >$path 2>/dev/null 99 | response "200" "OK" '{status: "OK"}' 100 | } 101 | 102 | function deleteDocument { 103 | # TODO need a security check on the path. 104 | path=$dataDir/$1/$2/$3.json 105 | if [ -e $path ]; then 106 | rm $path 107 | response "200" "OK" '{status: "OK"}' 108 | else 109 | response "404" "Not Found" '{status: "FAILED"}' 110 | fi 111 | } 112 | 113 | getContentLength() { 114 | length= 115 | while true 116 | do 117 | if read line ; then 118 | line=$(echo $line|tr -d '\r\n') 119 | if [[ "$line" == '' ]]; then 120 | break 121 | fi 122 | if [[ "$length" == "" ]]; then 123 | length=$(echo -e $line | head -1 | sed -n 's/Content-Length: \(.*\)/\1/p') 124 | fi 125 | fi 126 | done 127 | } 128 | 129 | response() { 130 | code=${1:-"200"} 131 | status=${2:-"OK"} 132 | data=${3:-} 133 | size=${#data} 134 | echo -ne "HTTP/1.1 $code $status\r\nContent-Length: $size\r\n\r\n$data"; 135 | } 136 | 137 | unknown() { 138 | response "418" "I'm a Teapot" "Can't do that." 139 | } 140 | 141 | exec7Roll() { 142 | if read line ; then 143 | extractHeader "$line" 144 | if [ "$method" = "GET" ]; then 145 | if [ ! "$query" = "" ]; then 146 | queryDocuments "$db" "$collection" "$docid" "$query" 147 | elif [ "$db" = "" ]; then 148 | listDatabases 149 | elif [ "$collection" = "" ]; then 150 | listCollections "$db" 151 | elif [ "$docid" = "" ]; then 152 | listDocuments "$db" "$collection" 153 | else 154 | getDocument "$db" "$collection" "$docid" 155 | fi 156 | elif [ "$method" = "POST" ]; then 157 | getContentLength 158 | if [ ! "$db" = "" ] && [ ! "$collection" = "" ] && [ ! "$docid" = "" ] ; then 159 | createDocument "$db" "$collection" "$docid" 160 | elif [ ! "$db" = "" ] && [ ! "$collection" = "" ] ; then 161 | createCollection "$db" "$collection" 162 | elif [ ! "$db" = "" ] ; then 163 | createDB "$db" 164 | else 165 | unknown 166 | fi 167 | elif [ "$method" = "PUT" ]; then 168 | getContentLength 169 | if [ ! "$docid" = "" ]; then 170 | updateDocument "$db" "$collection" "$docid" 171 | else 172 | unknown 173 | exit 1 174 | fi 175 | elif [ "$method" = "DELETE" ]; then 176 | if [ ! "$docid" = "" ]; then 177 | deleteDocument "$db" "$collection" "$docid" 178 | fi 179 | else 180 | unknown 181 | exit 1 182 | fi 183 | fi 184 | } 185 | 186 | log() { 187 | echo "[7RollDB] $1" 188 | } 189 | 190 | clearFrom7Rolls() { 191 | log "Checking system for other 7Roll instances..." 192 | somepid=`lsof -i:"$port" -t` 193 | if [[ $somepid > 0 ]]; then 194 | log "7Roll found, pid $somepid." 195 | log "Killing this old 7Roll, or MongoDB, or whoever is claiming port $port." 196 | kill -9 $somepid || { log "Aaargh...! We are not powerful enough to kill $somepid. Summon the Sudo Wizard to cast a kill -9 spell on $somepid !!!"; exit 1; } 197 | fi 198 | log "No 7Rolls found." 199 | } 200 | 201 | checkDeps() { 202 | command -v socat 2&> /dev/null || { log "Please install socat to run 7RollDB"; exit 1; } 203 | [ -d $dataDir ] || { log "Data directory not accessable, can not proceed. Make sure you run 7RollDB from its root directory and that the dataDir variable is set properly."; exit 1; } 204 | } 205 | 206 | start7Rolling() { 207 | checkDeps 208 | clearFrom7Rolls 209 | log "Server running as pid $$." 210 | log "Server listening on port $port." 211 | socat tcp-l:"$port",fork,reuseaddr exec:"$0 exec" 212 | } 213 | 214 | show7RollHelp() { 215 | read -d '' help << EOF 216 | 7RollDB version $version $versionName 217 | $0 Start the 7RollDB server 218 | $0 --help Shows this help. 219 | $0 --version Show version information 220 | EOF 221 | echo -e "$help" 222 | } 223 | 224 | if [ "$1" = "" ]; then 225 | start7Rolling 226 | elif [ $1 = "exec" ]; then 227 | exec7Roll 228 | elif [ $1 = "--help" ]; then 229 | show7RollHelp 230 | elif [ $1 = "--version" ]; then 231 | log "version $version $versionName " 232 | else 233 | show7RollHelp 234 | fi 235 | --------------------------------------------------------------------------------