├── LICENSE ├── README.md ├── internal ├── internal.pb.go └── internal.proto ├── main.slide ├── store.go └── store_test.go /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Ben Johnson 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so, 10 | subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 17 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 18 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 19 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 20 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Application Development Using BoltDB 2 | ==================================== 3 | 4 | ### Abstract 5 | 6 | We've been taught for decades that we need a complex database server to run our 7 | applications. However, these servers incur a huge performance hit for most 8 | queries and they are frequently misconfigured because of their operational 9 | complexity which can cause slowness or downtime. In this talk, I'll show you how 10 | to use a local, pure Go key/value store called BoltDB to build applications that 11 | are both simple and fast. We will see how development and deployment become a 12 | breeze once you ditch your complex database server. 13 | 14 | 15 | ### Introduction 16 | 17 | Software is too complex and too slow. We've seen the speed of CPUs increase 18 | by orders of magnitude in the past few decades yet our applications seem to 19 | require more hardware than ever. In the past several years I've used embedded 20 | key/value databases for my applications because of their simplicity and speed. 21 | Today I'm going to walk you through building a simple application using a 22 | pure Go key/value store I wrote called BoltDB. 23 | 24 | 25 | ### What is a embedded key/value database? 26 | 27 | Before we dive in, let's talk about what an embedded key/value store even is! 28 | "Embedded" refers to the database actually being compiled into your application 29 | instead of a database server which you connect to over a socket. A good example 30 | of an embedded database is SQLite. 31 | 32 | However, SQLite is a relational database so let's talk about what "key/value" 33 | means next. Key/value databases are extremely simple. They map a "key", which 34 | is just a unique set of bytes, to a "value", which is an arbitrary set of bytes. 35 | It helps to think of this in terms of relational databases. Your "key" would be 36 | your primary key and your value would be the encoded row. In fact, most database 37 | servers utilize a key/value database internally to store rows. Essentially, 38 | the key/value database is just a persisted map. 39 | 40 | Some key/value databases allow you to have multiple key/value mappings. In 41 | BoltDB, these are called "buckets". Every key is unique in a bucket and points 42 | to a value. Many times you can think of buckets like tables in a relational 43 | database. You may have a "users" bucket or a "products" bucket. 44 | 45 | Just as there are many database servers to choose from, there are also many 46 | types of embedded key/value databases with different trade offs. Sometimes 47 | you'll trade write performance for read performance or you'll trade 48 | transactional safety for performance. For example, BoltDB is read optimized and 49 | supports fully serializable ACID transactions. This makes it good for many read- 50 | heavy applications that require strong guarantees. 51 | 52 | 53 | ### Getting started with BoltDB 54 | 55 | One of the best things about using BoltDB is that installation process is so 56 | simple. You don't need to install a server or even configure it. You just use 57 | "go get" like any other Go package and it'll work on Windows, Mac, Linux, 58 | Raspberry Pi, and even iOS and Android. 59 | 60 | ``` 61 | $ go get github.com/boltdb/bolt 62 | ``` 63 | 64 | 65 | #### Object encoding 66 | 67 | One feature of relational databases that many of us take for granted is that 68 | they handle encoding rows into bytes on disk. Since Bolt only works with byte 69 | slices we'll need to handle that manually. Lucky for us, there are a LOT of 70 | options for object serialization. 71 | 72 | In Go, one of the most popular serialization libraries is Protocol Buffers 73 | (also called "protobufs"). One implementation, called gogoprotobuf, is also 74 | one of the fastest. With protobufs, we declare our serialization format and 75 | then generate Go files for doing this quickly. 76 | 77 | Let's take a look at an example application to see how we'd do this. This 78 | application shows how to do CRUD for a simple "User" data store but I've also 79 | built many other less traditional applications on Bolt such as message queues 80 | and analytics. 81 | 82 | 83 | #### Domain types 84 | 85 | In our app, we have a single domain type called "User" with two fields: ID 86 | & username. You can expand out to more types and nest objects but we'll stick 87 | with a single object to keep things simple. 88 | 89 | I like to separate out my domain types from my encoding types by placing my 90 | encoding types in a subpackage called "internal". I do this for two reasons. 91 | First, it keeps the generated protobufs code separate. And second, the 92 | "internal" package is inaccessible from other packages outside our app and it's 93 | hidden from godoc. 94 | 95 | Inside our protobuf definition we can see that it matches our domain type with 96 | a few exceptions. Since it's our binary representation we have to specify a size 97 | for our integer type. Also, you'll notice numbers on the right. These are 98 | essentially field IDs when it's encoded. When you add or remove fields you 99 | don't need to do a migration like with a relational database. You simply add a 100 | field with a higher number or delete a field. 101 | 102 | Back in our store.go we can add code to generate our protobufs. This line 103 | calls the protobuf compiler, "protoc", and will generate to 104 | `internal/internal.pb.go`. If we look in there we can see it's a bunch of ugly 105 | generated code. 106 | 107 | Our domain type will convert to and from this protobuf type but we can hide it 108 | all behind the `encoding.BinaryMarshaler` & `encoding.BinaryUnmarshaler` 109 | interfaces. Our `MarshalBinary()` simply copies our fields in and marshals them 110 | and the `UnmarshalBinary()` unmarshals the data and copies the fields out. This 111 | is a bit more work than in relational databases but it's easy to write, test, 112 | and migrate. 113 | 114 | 115 | #### Initializing the store 116 | 117 | Our `Store` type will be our application wrapper around our Bolt database. To 118 | open a `bolt.DB`, we simply need to pass in the file path and the file 119 | permissions to set if the file doesn't exist. This will use the umask so I 120 | typically set my permissions to `0666` and let users set the umask to filter 121 | that at runtime. 122 | 123 | Once the database is open, we'll start a writable transaction. The `Begin()` 124 | function is what starts a transaction. The `true` argument means that it's a 125 | "writable" transaction. Bolt can have as many read transactions as you want but 126 | only one write transaction gets processed at a time. This means it's important 127 | to keep updates small and break up really large updates into smaller chunks. All 128 | transactions operate under serializable isolation which means that all data will 129 | be a snapshot of exactly how it was when the transaction started -- even if 130 | other write transactions commit while the read transaction is in process. 131 | 132 | The deferred rollback can look odd since we want to commit the transaction at 133 | end. It's important in case you return an error early or your application 134 | panics. All transactions need rollback or commit when they're done or else they 135 | can block other operations later. 136 | 137 | Within our transaction we call `CreateBucketIfNotExists()` to create our bucket 138 | for our users. This is similar to a "CREATE TABLE IF NOT EXISTS" in SQL. If the 139 | bucket doesn't exist then it's created. Otherwise it's ignored. Calling this 140 | during initialization means that we won't have to check for it whenever we use 141 | the "users" bucket. It's guaranteed to be there. 142 | 143 | Finally, we commit our transaction and return the error, if any occurred while 144 | saving. Bolt does not allow partial transactions so if a disk error occurs then 145 | your entire transaction will be rolled back. The deferred rollback that we 146 | called earlier will be ignored for this transaction since we have successfully 147 | committed. 148 | 149 | Closing the store is a simple task. Simply call `Close()` on the Bolt database 150 | and it will release it's exclusive file lock and close the file descriptor. 151 | 152 | 153 | #### Creating a user 154 | 155 | Now that we have our database ready, let's create a user. In our `CreateUser()` 156 | method, we'll start by creating a writable transaction just like we did before. 157 | Then we'll grab the "Users" bucket from the transaction. We don't need to check 158 | if it exists because we created it during initialization. 159 | 160 | Next, we'll create a new ID for the user. Bolt has a nice feature called 161 | sequences which are transactionally safe autoincrementing integers for each 162 | bucket. Whenever we call `NextSequence()`, we'll get the bucket's next integer. 163 | Once we grab it, we assign it to our user's ID. 164 | 165 | Now our user is ready to be marshaled. We call `MarshalBinary()` and we get a 166 | set of bytes which represents our encoded user. Easy peasy! 167 | 168 | Since Bolt only works with bytes, we'll need to convert our ID to bytes. I 169 | recommend using the `binary.BigEndian.PutUint64()` function for this. I use 170 | big endian because it will sort our IDs in the right order. 171 | 172 | [show big endian vs little endian on slide] 173 | 174 | We'll use the bucket's `Put()` method to associate our encoded user with the 175 | encoded ID. Then we'll commit and our data is saved. 176 | 177 | 178 | #### Retrieving the user 179 | 180 | Creating a user is just a matter of converting objects to bytes so retrieving a 181 | user is simply converting bytes to objects. In our `User()` method we'll start 182 | a transaction but this time we'll pass in `false` to specify a read-only 183 | transaction. Again, read-only transactions can run completely in parallel so 184 | this scales really well across multiple CPU cores. 185 | 186 | Once we have our transaction, we call `Get()` on our bucket with an encoded 187 | user ID and we get back the encoded user bytes. We can call `UnmarshalBinary()` 188 | on a new `User` and decode the data. If the encoded bytes comes back as `nil` 189 | then we know that the user doesn't exist and we can simply return a `nil` user. 190 | 191 | 192 | #### Retrieving multiple users 193 | 194 | Reading one user is good but many times we want to return a list of all users. 195 | For this we'll need to use a Bolt cursor. A cursor is simply an object for 196 | iterating over a bucket in order. It has a handful of methods we can use to 197 | move forward, back or even jump around. 198 | 199 | In our `Users()` method we'll grab a read-only transaction and a cursor from 200 | our bucket. Then we'll iterate over every key/value in our bucket. We can 201 | collapse it all into a simple "for" loop where we call `First()` at the 202 | beginning and then `Next()` until we receive a `nil` key. For each value, we'll 203 | unmarshal the user and add it to our slice. 204 | 205 | If you need reverse sorting, you can call `Last()` and then `Prev()` on the 206 | cursor. You can also use the `Seek()` method to jump to a specific spot. For 207 | example, if we wanted to do pagination we could pass in an "options" object 208 | into the method and have an offset and limit. 209 | 210 | 211 | #### Updating a user 212 | 213 | Now that we've created a user, let's update it. Let's look at the 214 | `SetUsername()` method. This time we'll mix it up and use the `Update()` method 215 | instead of `Begin()`. This method works just like `Begin(true)` except that 216 | it executes a function in the context of the transaction. If the function 217 | returns `nil` then the transaction commits. Otherwise if it returns an error or 218 | panics then it will rollback the transaction. 219 | 220 | First we'll retrieve our user by ID and unmarshal. In this case we're combining 221 | the `Get()` and `UnmarshalBinary()` into a compound `if` block. I find it easier 222 | to read if I group these related types of calls together. Next we simply update 223 | the username on our user we just unmarshaled. 224 | 225 | Now that we have our updated user object, we can simply remarshal it and 226 | overwrite the previous value by calling `Put()` again. 227 | 228 | 229 | #### Deleting a user 230 | 231 | Finally, the last part of our CRUD store is the deletion. Delete is incredibly 232 | simply. Simply call the `Delete()` method on the bucket. That's it! 233 | 234 | 235 | 236 | ### Bolt in Practice 237 | 238 | That was the basics of doing CRUD operations with Bolt and we can talk about 239 | more advanced use cases in a minute but first let's look at what running Bolt 240 | in production looks like. 241 | 242 | Internally, Bolt structures itself as a B+tree of pages which requires a lot of 243 | random access at the file system so it's recommended that you run Bolt on an 244 | SSD. Other embedded databases such as LevelDB are optimized for spinning disks. 245 | 246 | Bolt also maps the database to memory using a read-only `mmap()` so byte slices 247 | returned from buckets cannot be updated (or else it will SEGFAULT) and the 248 | byte slices are only valid for the life of the transaction. The memory map 249 | provides two amazing benefits. First, it means that data is never copied from 250 | the database. You're accessing it directly from the operating system's page 251 | cache. Second, since it's in the OS page cache, your hot data will persist in 252 | memory across application restarts. 253 | 254 | 255 | #### Backup & restore 256 | 257 | From an operations standpoint, Bolt just uses a single file on disk so it's 258 | simple to manage. However, as a library, there's not a standard CLI command to 259 | backup your database but Bolt does provide a great option. 260 | 261 | Transactions in Bolt implement the `io.WriterTo` interface which means they 262 | can copy an entire snapshot of the database to an `io.Writer` with one line of 263 | code. Depending on your application you may wish to provide an HTTP endpoint so 264 | you can `curl` your backups or you can build an hourly snapshot to Amazon S3. 265 | 266 | Another option in the works is a streaming transaction log so that you can 267 | attach a process over the network to be an async replica. This is similar to 268 | how Postgres replication works. This is still early in development though and 269 | is not currently available. 270 | 271 | 272 | ### Performance 273 | 274 | While performance is not a primary goal of Bolt, it is an important feature to 275 | talk about. Bolt is read optimized so if your workload regularly consists of 276 | tens of thousands of writes per second then you may want to look at write 277 | optimized databases such as LevelDB. 278 | 279 | 280 | #### Benchmarks 281 | 282 | Benchmarks are not typically very useful but it's good to know a ballpark of 283 | what a database can handle. I typically tell people the following on a machine 284 | with an SSD. Expect to get up to 2,000 random writes/sec without optimization. 285 | If you're bulk loading and sorting data then you can get up to 400K+ writes/sec. 286 | Typically you're throttled by your hard drive speed. 287 | 288 | On the read side, it depends on if your data is in the page cache. Typical CRUD 289 | applications have maybe 10-20% of their data hot at any given time. That means 290 | that if you have a 20GB database then 2 - 4GB is hot and that will be resident 291 | in memory assuming you have that much RAM. For hot data, you can expect a read 292 | transaction and a single bucket read to take a 1-2µs. If you're iterating over 293 | hot data then you can see speeds of 20M keys/second. Again, all this data is 294 | in the page cache and there's no data copy so it's really fast. 295 | 296 | If your data is not hot then you'll be limited by the speed of your hard drive. 297 | Expect reads of cold data on an SSD to take hundreds of microseconds or a few 298 | milliseconds depending on the size of your database. 299 | 300 | 301 | ### Scaling with Bolt 302 | 303 | One of the biggest criticisms of embedded databases is that they are embedded 304 | and don't have a means of scaling or clustering. This is true, however, it's not 305 | necessarily a reason to avoid embedded databases. There are several strategies 306 | that can be used to obtain the safety required for your application while still 307 | using an embedded database. 308 | 309 | 310 | #### Vertical scaling 311 | 312 | The easiest way to scale is still, by far, by scaling vertically. If your 313 | bottleneck is with read queries then simply adding more CPU cores will scale 314 | your Bolt application near linearly. This answer may sound too simplistic but 315 | that's the beauty of it. It's really simple. 316 | 317 | 318 | #### Horizontal scaling via sharding 319 | 320 | Many applications -- especially SaaS applications -- can be partitioned by users 321 | or accounts. This means that you can either assign each account to its own 322 | database or use strategies like consistent hashing to group accounts into a 323 | number of partitions. This will allow you to add additional machines and 324 | rebalance the load of your application. 325 | 326 | 327 | #### Data integrity in the face of catastrophic failure 328 | 329 | Until streaming replication is ready for Bolt, windows of data loss are still 330 | an issue that needs consideration when building with an embedded database. Many 331 | cloud providers provide highly safe, redundant storage but even those can fail 332 | every great once in a while. 333 | 334 | For some applications, a daily backup may be reliable enough. For others, a 335 | standby machine taking snapshots every 10 minutes may be required. Many 336 | applications store critical data such as finances in a third party service like 337 | Stripe so a 10 minute window may be acceptable. 338 | 339 | Again, a Bolt database is simply a file and can be copied extremely quickly. 340 | Expect a copy to take 3-5 seconds per gigabyte on an SSD. 341 | 342 | 343 | 344 | ### Conclusion 345 | 346 | I believe that local key/value databases meet the requirements for many 347 | applications while providing a simple, fast development experience. I have 348 | shown you how to get up and running with Bolt and build a simple CRUD data 349 | store. The code is not just straightforward but also incredibly fast and 350 | can vertically scale for many workloads. Please consider using a local 351 | key/value database in your next project! 352 | 353 | 354 | -------------------------------------------------------------------------------- /internal/internal.pb.go: -------------------------------------------------------------------------------- 1 | // Code generated by protoc-gen-gogo. 2 | // source: internal/internal.proto 3 | // DO NOT EDIT! 4 | 5 | /* 6 | Package internal is a generated protocol buffer package. 7 | 8 | It is generated from these files: 9 | internal/internal.proto 10 | 11 | It has these top-level messages: 12 | User 13 | */ 14 | package internal 15 | 16 | import proto "github.com/gogo/protobuf/proto" 17 | import fmt "fmt" 18 | import math "math" 19 | 20 | // Reference imports to suppress errors if they are not otherwise used. 21 | var _ = proto.Marshal 22 | var _ = fmt.Errorf 23 | var _ = math.Inf 24 | 25 | // This is a compile-time assertion to ensure that this generated file 26 | // is compatible with the proto package it is being compiled against. 27 | const _ = proto.GoGoProtoPackageIsVersion1 28 | 29 | type User struct { 30 | ID *int64 `protobuf:"varint,1,opt,name=ID" json:"ID,omitempty"` 31 | Username *string `protobuf:"bytes,2,opt,name=Username" json:"Username,omitempty"` 32 | XXX_unrecognized []byte `json:"-"` 33 | } 34 | 35 | func (m *User) Reset() { *m = User{} } 36 | func (m *User) String() string { return proto.CompactTextString(m) } 37 | func (*User) ProtoMessage() {} 38 | func (*User) Descriptor() ([]byte, []int) { return fileDescriptorInternal, []int{0} } 39 | 40 | func (m *User) GetID() int64 { 41 | if m != nil && m.ID != nil { 42 | return *m.ID 43 | } 44 | return 0 45 | } 46 | 47 | func (m *User) GetUsername() string { 48 | if m != nil && m.Username != nil { 49 | return *m.Username 50 | } 51 | return "" 52 | } 53 | 54 | func init() { 55 | proto.RegisterType((*User)(nil), "internal.User") 56 | } 57 | 58 | var fileDescriptorInternal = []byte{ 59 | // 81 bytes of a gzipped FileDescriptorProto 60 | 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0xe2, 0x12, 0xcf, 0xcc, 0x2b, 0x49, 61 | 0x2d, 0xca, 0x4b, 0xcc, 0xd1, 0x87, 0x31, 0xf4, 0x0a, 0x8a, 0xf2, 0x4b, 0xf2, 0x85, 0x38, 0x60, 62 | 0x7c, 0x25, 0x15, 0x2e, 0x96, 0xd0, 0xe2, 0xd4, 0x22, 0x21, 0x2e, 0x2e, 0x26, 0x4f, 0x17, 0x09, 63 | 0x46, 0x05, 0x46, 0x0d, 0x66, 0x21, 0x01, 0x2e, 0x0e, 0x90, 0x58, 0x5e, 0x62, 0x6e, 0xaa, 0x04, 64 | 0x13, 0x50, 0x84, 0x13, 0x10, 0x00, 0x00, 0xff, 0xff, 0xc2, 0x20, 0xb9, 0x77, 0x49, 0x00, 0x00, 65 | 0x00, 66 | } 67 | -------------------------------------------------------------------------------- /internal/internal.proto: -------------------------------------------------------------------------------- 1 | package internal; 2 | 3 | message User { 4 | optional int64 ID = 1; 5 | optional string Username = 2; 6 | } 7 | -------------------------------------------------------------------------------- /main.slide: -------------------------------------------------------------------------------- 1 | App Development Using BoltDB 2 | 22 June 2016 3 | Tags: boltdb, databases 4 | 5 | Ben Johnson 6 | benbjohnson@yahoo.com 7 | @benbjohnson 8 | 9 | 10 | * Introduction 11 | 12 | * Introduction 13 | 14 | Software today is too complex and too slow 15 | 16 | CPUs are orders of magnitude faster but our applications are not 17 | 18 | We've used SQL databases for decades but is there a better way? 19 | 20 | 21 | * My History 22 | 23 | - Oracle DBA for years 24 | 25 | - Data visualization & front end developer for a while 26 | 27 | - Open source Go developer (BoltDB, go-raft, etc) 28 | 29 | 30 | 31 | * What is an embedded key/value store? 32 | 33 | 34 | * Embedded 35 | 36 | - Compiled into your application (e.g. SQLite) 37 | - No network connection 38 | 39 | 40 | * Key/Value 41 | 42 | - Maps a unique set of bytes ("key") to another set of bytes ("value") 43 | 44 | - Essentially, it is a persisted map: `[]byte` ➤ `[]byte` 45 | 46 | Some databases let you have multiple maps, sometimes called "buckets" 47 | 48 | 49 | * What embedded databases are there? 50 | 51 | Just like relational databases, there are TONS of embedded databases. 52 | 53 | *LSM* (Log-Structured Merge Tree) 54 | 55 | - Write optimized 56 | - No support for transactions 57 | - Examples: LevelDB, RocksDB 58 | 59 | *B+tree* 60 | 61 | - Read optimized 62 | - Supports fully serializable ACID transactions 63 | - Examples: LMDB, BoltDB 64 | 65 | 66 | * Getting started with BoltDB 67 | 68 | * Installation 69 | 70 | Just type: 71 | 72 | $ go get github.com/boltdb/bolt 73 | 74 | And you're ready! 75 | 76 | 🎉🎉🎉 77 | 78 | 79 | * Cross platform support 80 | 81 | Works on: 82 | 83 | - Windows 84 | - Mac OS X 85 | - Linux 86 | - ARM 87 | - iOS 88 | - Android 89 | 90 | 91 | * Object encoding 92 | 93 | * Object encoding 94 | 95 | Key/value stores only understand bytes so you must encode objects yourself 96 | 97 | Luckily, this is a solved problem: 98 | 99 | - Protocol Buffers 100 | - FlatBuffers 101 | - MessagePack 102 | - XML 103 | - JSON 104 | - YAML 105 | 106 | 107 | * Protocol Buffers 108 | 109 | Created by Google for RPC 110 | 111 | Uses a definition language to generate encoders & decoders 112 | 113 | .link https://developers.google.com/protocol-buffers/ 114 | 115 | I use `gogoprotobuf`. It's extremely fast and easy to use. 116 | 117 | .link https://github.com/gogo/protobuf 118 | 119 | 120 | * Example application 121 | 122 | * Domain Type 123 | 124 | We'll focus just on a simple data layer to perform CRUD operations on "users" 125 | 126 | // User represents a user in our system. 127 | type User struct { 128 | ID int 129 | Username string 130 | } 131 | 132 | 133 | * User serialization 134 | 135 | Separate out "domain types" from "encoding types" 136 | 137 | internal/ 138 | internal.proto // Definition file 139 | internal.pb.go // Generated file 140 | store.go 141 | store_test.go 142 | ... 143 | 144 | 145 | I use `internal` because it can only be used by my package and it's hidden from godoc. 146 | 147 | 148 | * Protobuf definition 149 | 150 | `internal.proto` 151 | 152 | package internal; 153 | 154 | message User { 155 | optional int64 ID = 1; 156 | optional string Username = 2; 157 | } 158 | 159 | - Must specify integer sizes 160 | - Field IDs provide versioning 161 | - No migration needed to add or remove fields 162 | 163 | 164 | * Go Generate 165 | 166 | Add this line to your main package: 167 | 168 | //go:generate protoc --gogo_out=. internal/internal.proto 169 | 170 | Automatically regenerate encoders/decoders: 171 | 172 | $ go generate 173 | 174 | 175 | * Ugly generated code 176 | 177 | package internal 178 | 179 | type User struct { 180 | ID *int64 `protobuf:"varint,1,opt,name=ID" json:"ID,omitempty"` 181 | Username *string `protobuf:"bytes,2,opt,name=Username" json:"Username,omitempty"` 182 | XXX_unrecognized []byte `json:"-"` 183 | } 184 | 185 | func (m *User) Reset() { *m = User{} } 186 | func (m *User) String() string { return proto.CompactTextString(m) } 187 | func (*User) ProtoMessage() {} 188 | func (*User) Descriptor() ([]byte, []int) { return fileDescriptorInternal, []int{0} } 189 | 190 | func (m *User) GetID() int64 { 191 | if m != nil && m.ID != nil { 192 | return *m.ID 193 | } 194 | return 0 195 | } 196 | 197 | ... 198 | 199 | 200 | * stdlib: encoding package 201 | 202 | We need to implement marshaling: 203 | 204 | type BinaryMarshaler interface { 205 | MarshalBinary() (data []byte, err error) 206 | } 207 | 208 | type BinaryUnmarshaler interface { 209 | UnmarshalBinary(data []byte) error 210 | } 211 | 212 | .link https://golang.org/pkg/encoding/ 213 | 214 | 215 | * BinaryMarshaler 216 | 217 | Encode a `*User` to a `[]byte`: 218 | 219 | // MarshalBinary encodes a user to binary format. 220 | func (u *User) MarshalBinary() ([]byte, error) { 221 | return proto.Marshal(&internal.User{ 222 | ID: proto.Int64(int64(u.ID)), 223 | Username: proto.String(u.Username), 224 | }) 225 | } 226 | 227 | `*User` ➤ `*internal.User` ➤ `[]byte` 228 | 229 | 230 | * BinaryUnmarshaler 231 | 232 | Decode from a `[]byte` to a `*User`: 233 | 234 | func (u *User) UnmarshalBinary(data []byte) error { 235 | var pb internal.User 236 | if err := proto.Unmarshal(data, &pb); err != nil { 237 | return err 238 | } 239 | 240 | u.ID = int(pb.GetID()) 241 | u.Username = pb.GetUsername() 242 | 243 | return nil 244 | } 245 | 246 | `[]byte` ➤ `*internal.User` ➤ `*User` 247 | 248 | 249 | * Data Store 250 | 251 | * Data Store 252 | 253 | // Store represents the data storage layer. 254 | type Store struct { 255 | // Filepath to the data file. 256 | Path string 257 | 258 | db *bolt.DB 259 | } 260 | 261 | * Opening & initialization 262 | 263 | // Open opens and initializes the store. 264 | func (s *Store) Open() error { 265 | // Open bolt database. 266 | db, err := bolt.Open(s.Path, 0666, nil) 267 | if err != nil { 268 | return err 269 | } 270 | s.db = db 271 | 272 | // Start a writable transaction. 273 | tx, err := s.db.Begin(true) 274 | if err != nil { 275 | return nil, err 276 | } 277 | defer tx.Rollback() 278 | 279 | // Initialize buckets to guarantee that they exist. 280 | tx.CreateBucketIfNotExists([]byte("Users")) 281 | 282 | // Commit the transaction. 283 | return tx.Commit() 284 | } 285 | 286 | * Open 287 | 288 | db, err := bolt.Open(s.Path, 0666, nil) 289 | if err != nil { 290 | return err 291 | } 292 | 293 | - Requires a file path & mode 294 | - Mode uses umask (`0022`) so it ends up being `0644` 295 | - Additional options available 296 | 297 | 298 | * Initialization 299 | 300 | // Start a writable transaction. 301 | tx, err := s.db.Begin(true) 302 | if err != nil { 303 | return nil, err 304 | } 305 | defer tx.Rollback() 306 | 307 | // Initialize buckets to guarantee that they exist. 308 | tx.CreateBucketIfNotExists([]byte("Users")) 309 | 310 | // Commit the transaction. 311 | return tx.Commit() 312 | 313 | - Everything must be in a transaction 314 | - Our "users" bucket will act like a table 315 | 316 | 317 | * Transactions 318 | 319 | Read-write transaction 320 | 321 | - Only one at a time 322 | - If rolled back, all changes are undone 323 | - Only committed if data written and fsync'd 324 | 325 | Read-only transactions 326 | 327 | - Unlimited number of concurrent transactions 328 | - Serializable isolation 329 | - You _must_ rollback all read-only transactions 330 | 331 | * Creating buckets 332 | 333 | func (*bolt.Tx) CreateBucketIfNotExists(name []byte) (*bolt.Bucket, error) 334 | 335 | - Creates the bucket if it doesn't exist 336 | - Returns the bucket if it does exist 337 | - Similiar to CREATE TABLE IF NOT EXISTS 338 | 339 | 340 | * Closing the store 341 | 342 | // Close shuts down the store. 343 | func (s *Store) Close() error { 344 | return s.db.Close() 345 | } 346 | 347 | - Waits for pending transactions to finish 348 | 349 | 350 | * CRUD 351 | 352 | * Creating a user 353 | 354 | * Creating a user 355 | 356 | - Start a transaction 357 | - Create a new id for our user 358 | - Marshal user to bytes 359 | - Save bytes to "Users" bucket 360 | - Commit transaction 361 | 362 | * Creating a user 363 | 364 | func (s *Store) CreateUser(u *User) error { 365 | tx, err := s.db.Begin(true) 366 | 367 | // Retrieve bucket. 368 | bkt := tx.Bucket([]byte("Users")) 369 | 370 | // The sequence is an autoincrementing integer that is transactionally safe. 371 | seq, _ := bkt.NextSequence() 372 | u.ID = int(seq) 373 | 374 | // Marshal our user into bytes. 375 | buf, err := u.MarshalBinary() 376 | if err != nil { 377 | return err 378 | } 379 | 380 | // Save user to the bucket. 381 | if err := bkt.Put(itob(u.ID), buf); err != nil { 382 | return err 383 | } 384 | return tx.Commit() 385 | } 386 | 387 | * Helper function 388 | 389 | // itob encodes v as a big endian integer. 390 | func itob(v int) []byte { 391 | buf := make([]byte, 8) 392 | binary.BigEndian.PutUint64(buf, uint64(v)) 393 | return buf 394 | } 395 | 396 | - Use big endian because it is byte sortable 397 | 398 | Encoding 1000: 399 | 400 | Big endian: 00 00 03 E8 401 | Little endian: E8 03 00 00 402 | 403 | 404 | * Autoincrementing sequences 405 | 406 | func (*bolt.Bucket) NextSequence() (uint64, error) 407 | 408 | - Starts at 1 409 | - Transactionally safe 410 | - Per-bucket sequence 411 | 412 | 413 | * Setting keys 414 | 415 | func (*bolt.Bucket) Put(key, value []byte) error 416 | 417 | - Assigns `value` to `key` 418 | - Creates the key if it doesn't exist 419 | - Overwrites it if it does exist 420 | 421 | 422 | * Retrieving a user 423 | 424 | * Retrieving a user 425 | 426 | - Start a read-only transaction 427 | - Read bytes from bucket for id 428 | - Unmarshal bytes to a `*User` 429 | - Return user 430 | - Rollback transaction 431 | 432 | * Retrieving a user 433 | 434 | func (s *Store) User(id int) (*User, error) { 435 | // Start a readable transaction. 436 | tx, err := s.db.Begin(false) 437 | if err != nil { 438 | return nil, err 439 | } 440 | defer tx.Rollback() 441 | 442 | // Read encoded user bytes. 443 | v := tx.Bucket([]byte("Users")).Get(itob(id)) 444 | if v == nil { 445 | return nil, nil 446 | } 447 | 448 | // Unmarshal bytes into a user. 449 | var u User 450 | if err := u.UnmarshalBinary(v); err != nil { 451 | return nil, err 452 | } 453 | 454 | return &u, nil 455 | } 456 | 457 | * Reading values 458 | 459 | func (*bolt.Bucket) Get(key []byte) []byte 460 | 461 | - Returns the value for a given key 462 | - Returns `nil` if key doesn't exist 463 | 464 | 465 | * Retrieving a list of users 466 | 467 | * Retrieving a list of users 468 | 469 | - Start a read-only transaction 470 | - Iterate over a bucket using a cursor 471 | - Unmarshal each value to a `*User` 472 | - Return users 473 | - Rollback transaction 474 | 475 | * Retrieving a list of users 476 | 477 | func (s *Store) Users() ([]*User, error) { 478 | // Start a readable transaction. 479 | tx, err := s.db.Begin(false) 480 | if err != nil { 481 | return nil, err 482 | } 483 | defer tx.Rollback() 484 | 485 | // Create a cursor on the user's bucket. 486 | c := tx.Bucket([]byte("Users")).Cursor() 487 | 488 | // Read all users into a slice. 489 | var a []*User 490 | for k, v := c.First(); k != nil; k, v = c.Next() { 491 | var u User 492 | if err := u.UnmarshalBinary(v); err != nil { 493 | return nil, err 494 | } 495 | a = append(a, &u) 496 | } 497 | 498 | return a, nil 499 | } 500 | 501 | * Iterating over buckets 502 | 503 | func (*bolt.Bucket) Cursor() *bolt.Cursor 504 | 505 | - Allows forward iteration: First(), Next() 506 | - Allows backward iteration: Last(), Prev() 507 | - Allows seeking to a key: Seek() 508 | - Always iterates keys in sorted order 509 | 510 | 511 | * Updating a user 512 | 513 | * Updating a user 514 | 515 | - Start a read-write transaction 516 | - Retrieving existing user value & unmarshal 517 | - Update user 518 | - Marshal new user value and save 519 | - Commit transaction 520 | 521 | * Updating a user 522 | 523 | func (s *Store) SetUsername(id int, username string) error { 524 | return s.db.Update(func(tx *bolt.Tx) error { 525 | bkt := tx.Bucket([]byte("Users")) 526 | 527 | // Retrieve encoded user and decode. 528 | var u User 529 | if v := bkt.Get(itob(id)); v == nil { 530 | return ErrUserNotFound 531 | } else if err := u.UnmarshalBinary(v); err != nil { 532 | return err 533 | } 534 | 535 | // Update user. 536 | u.Username = username 537 | 538 | // Encode and save user. 539 | if buf, err := u.MarshalBinary(); err != nil { 540 | return err 541 | } else if err := bkt.Put(itob(id), buf); err != nil { 542 | return err 543 | } 544 | 545 | return nil 546 | }) 547 | } 548 | 549 | * Transaction helper functions in Bolt 550 | 551 | func (*bolt.DB) Update(fn func(tx *bolt.Tx) error) error 552 | 553 | - Executes function in scope of a read-write transaction 554 | - If error is returned, transaction is rolled back 555 | 556 | func (*bolt.DB) View(fn func(tx *bolt.Tx) error) error 557 | 558 | - Executes function in scope of a read-only transaction 559 | 560 | 561 | * Deleting a user 562 | 563 | * Deleting a user 564 | 565 | - Start a read-write transaction 566 | - Delete key 567 | - Commit transaction 568 | 569 | * Deleting a user 570 | 571 | func (s *Store) DeleteUser(id int) error { 572 | return s.db.Update(func(tx *bolt.Tx) error { 573 | return tx.Bucket([]byte("Users")).Delete(itob(id)) 574 | }) 575 | } 576 | 577 | * Delete key 578 | 579 | func (*bolt.Bucket) Delete(key []byte) error 580 | 581 | - Removes a key from a bucket 582 | 583 | 584 | 585 | * BoltDB in Practice 586 | 587 | * BoltDB in Practice 588 | 589 | - Requires a lot of random access so SSDs are advised 590 | - Bolt returns byte slices pointing directly to a read-only mmap 591 | - Byte slices are only valid for the life of a transaction 592 | - Data is held in the OS page cache so it persists across application restarts 593 | 594 | 595 | * Backup & Restore 596 | 597 | - No built-in CLI command for backing up 598 | - Choose your own adventure 599 | - Streaming replication coming in the future 600 | 601 | 602 | * Backup & Restore 603 | 604 | Transactions implement `io.WriterTo`: 605 | 606 | type WriterTo interface { 607 | WriteTo(w Writer) (n int64, err error) 608 | } 609 | 610 | * HTTP Backup 611 | 612 | func (*Handler) ServeHTTP(w http.ResponseWriter, req *http.Request) { 613 | tx, err := db.Begin(false) 614 | if err != nil { 615 | http.Error(w, err.Error(), http.StatusInternalServerError) 616 | return 617 | } 618 | defer tx.Rollback() 619 | 620 | w.Header().Set("Content-Type", "application/octet-stream") 621 | w.Header().Set("Content-Disposition", `attachment; filename="my.db"`) 622 | w.Header().Set("Content-Length", strconv.Itoa(int(tx.Size()))) 623 | 624 | tx.WriteTo(w) 625 | } 626 | 627 | 628 | * Performance 629 | 630 | * Benchmarks 631 | 632 | These are ballpark estimates! 633 | 634 | 635 | * Write performance 636 | 637 | - 2,000 random writes/sec 638 | - 450,000 sequential, batched writes/sec 639 | 640 | * Read performance 641 | 642 | If data is hot in memory: 643 | 644 | - 1-2µs fetch time 645 | - Iterate over 20M keys/sec 646 | 647 | If data is not hot in memory: 648 | 649 | - Depends on your hard drive's speed 650 | 651 | 652 | * Scaling with Bolt 653 | 654 | * Scaling with Bolt 655 | 656 | - Biggest criticism of embedded databases 657 | 658 | 659 | * Vertical scaling 660 | 661 | - Really easy 662 | - Read transactions scale with number of CPU cores 663 | 664 | 665 | * Horizontal scaling 666 | 667 | - Many applications can be sharded (e.g. partitioned by account) 668 | - One database per partition 669 | - Possibly use consistent hashing 670 | - Allows you to simply add machines to rebalance load 671 | 672 | * Data integrity in the face of catastrophic failure! 673 | 674 | - Windows of data loss need consideration 675 | - Streaming replication will minimize this issue 676 | - Frequent backups (e.g. every hour or every 10m) 677 | - Many applications keep critical financial data in a separate service (e.g Stripe) 678 | 679 | * Data integrity in the face of catastrophic failure! 680 | 681 | - Backups are FAST! 682 | - It's just a file 683 | - 3 - 5 seconds per GB on an SSD 684 | 685 | 686 | * Other fun uses for Bolt 687 | 688 | - Message queue 689 | - Analytics 690 | - Buckets can be nested to provide interesting data structures 691 | 692 | 693 | * Conclusion 694 | 695 | * Conclusion 696 | 697 | - BoltDB fits many application use cases 698 | - Development experience is great 699 | - Testing experience is great 700 | - Deployment experience is great 701 | - Code is not only simple but extremely fast 702 | 703 | * Consider using BoltDB on your next project! 704 | -------------------------------------------------------------------------------- /store.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "encoding/binary" 5 | 6 | "github.com/benbjohnson/application-development-using-boltdb/internal" 7 | "github.com/boltdb/bolt" 8 | "github.com/gogo/protobuf/proto" 9 | ) 10 | 11 | //go:generate protoc --gogo_out=. internal/internal.proto 12 | 13 | // User represents a user in our system. 14 | type User struct { 15 | ID int 16 | Username string 17 | } 18 | 19 | // MarshalBinary encodes a user to binary format. 20 | func (u *User) MarshalBinary() ([]byte, error) { 21 | return proto.Marshal(&internal.User{ 22 | ID: proto.Int64(int64(u.ID)), 23 | Username: proto.String(u.Username), 24 | }) 25 | } 26 | 27 | // UnmarshalBinary decodes a user from binary data. 28 | func (u *User) UnmarshalBinary(data []byte) error { 29 | var pb internal.User 30 | if err := proto.Unmarshal(data, &pb); err != nil { 31 | return err 32 | } 33 | 34 | u.ID = int(pb.GetID()) 35 | u.Username = pb.GetUsername() 36 | 37 | return nil 38 | } 39 | 40 | // Store represents the data storage layer. 41 | type Store struct { 42 | // Filepath to the data file. 43 | Path string 44 | 45 | db *bolt.DB 46 | } 47 | 48 | // Open opens and initializes the store. 49 | func (s *Store) Open() error { 50 | // Open bolt database. 51 | db, err := bolt.Open(s.Path, 0666, nil) 52 | if err != nil { 53 | return err 54 | } 55 | s.db = db 56 | 57 | // Start a writable transaction. 58 | tx, err := s.db.Begin(true) 59 | if err != nil { 60 | return nil, err 61 | } 62 | defer tx.Rollback() 63 | 64 | // Initialize buckets to guarantee that they exist. 65 | tx.CreateBucketIfNotExists([]byte("Users")) 66 | 67 | // Commit the transaction. 68 | return tx.Commit() 69 | } 70 | 71 | // Close shuts down the store. 72 | func (s *Store) Close() error { 73 | return s.db.Close() 74 | } 75 | 76 | // User retrieves a user by ID. 77 | func (s *Store) User(id int) (*User, error) { 78 | // Start a readable transaction. 79 | tx, err := s.db.Begin(false) 80 | if err != nil { 81 | return nil, err 82 | } 83 | defer tx.Rollback() 84 | 85 | // Read encoded user bytes. 86 | v := tx.Bucket([]byte("Users")).Get(itob(id)) 87 | if v == nil { 88 | return nil, nil 89 | } 90 | 91 | // Unmarshal bytes into a user. 92 | var u User 93 | if err := u.UnmarshalBinary(v); err != nil { 94 | return nil, err 95 | } 96 | 97 | return &u, nil 98 | } 99 | 100 | // Users retrieves a list of all users. 101 | func (s *Store) Users() ([]*User, error) { 102 | // Start a readable transaction. 103 | tx, err := s.db.Begin(false) 104 | if err != nil { 105 | return nil, err 106 | } 107 | defer tx.Rollback() 108 | 109 | // Create a cursor on the user's bucket. 110 | c := tx.Bucket([]byte("Users")).Cursor() 111 | 112 | // Read all users into a slice. 113 | var a []*User 114 | for k, v := c.First(); k != nil; k, v = c.Next() { 115 | var u User 116 | if err := u.UnmarshalBinary(v); err != nil { 117 | return nil, err 118 | } 119 | a = append(a, &u) 120 | } 121 | 122 | return a, nil 123 | } 124 | 125 | // CreateUser creates a new user in the store. 126 | // The user's ID is set to u.ID on success. 127 | func (s *Store) CreateUser(u *User) error { 128 | // Start a writeable transaction. 129 | tx, err := s.db.Begin(true) 130 | if err != nil { 131 | return err 132 | } 133 | defer tx.Rollback() 134 | 135 | // Retrieve bucket. 136 | bkt := tx.Bucket([]byte("Users")) 137 | 138 | // The sequence is an autoincrementing integer that is transactionally safe. 139 | seq, _ := bkt.NextSequence() 140 | u.ID = int(seq) 141 | 142 | // Marshal our user into bytes. 143 | buf, err := u.MarshalBinary() 144 | if err != nil { 145 | return err 146 | } 147 | 148 | // Save user to the bucket. 149 | if err := bkt.Put(itob(u.ID), buf); err != nil { 150 | return err 151 | } 152 | 153 | // Commit transaction and exit. 154 | return tx.Commit() 155 | } 156 | 157 | // SetUsername updates the username for a user. 158 | func (s *Store) SetUsername(id int, username string) error { 159 | return s.db.Update(func(tx *bolt.Tx) error { 160 | bkt := tx.Bucket([]byte("Users")) 161 | 162 | // Retrieve encoded user and decode. 163 | var u User 164 | if v := bkt.Get(itob(id)); v == nil { 165 | return ErrUserNotFound 166 | } else if err := u.UnmarshalBinary(v); err != nil { 167 | return err 168 | } 169 | 170 | // Update user. 171 | u.Username = username 172 | 173 | // Encode and save user. 174 | if buf, err := u.MarshalBinary(); err != nil { 175 | return err 176 | } else if err := bkt.Put(itob(id), buf); err != nil { 177 | return err 178 | } 179 | 180 | return nil 181 | }) 182 | } 183 | 184 | // DeleteUser removes a user by id. 185 | func (s *Store) DeleteUser(id int) error { 186 | return s.db.Update(func(tx *bolt.Tx) error { 187 | return tx.Bucket([]byte("Users")).Delete(itob(id)) 188 | }) 189 | } 190 | 191 | // itob encodes v as a big endian integer. 192 | func itob(v int) []byte { 193 | buf := make([]byte, 8) 194 | binary.BigEndian.PutUint64(buf, uint64(v)) 195 | return buf 196 | } 197 | 198 | // User related errors. 199 | var ( 200 | ErrUserNotFound = Error("user not found") 201 | ) 202 | 203 | // Error represents an application error. 204 | type Error string 205 | 206 | func (e Error) Error() string { return string(e) } 207 | -------------------------------------------------------------------------------- /store_test.go: -------------------------------------------------------------------------------- 1 | package main_test 2 | 3 | import ( 4 | "io/ioutil" 5 | "os" 6 | "reflect" 7 | "testing" 8 | 9 | main "github.com/benbjohnson/application-development-using-boltdb" 10 | ) 11 | 12 | // Ensure store can create a new user. 13 | func TestStore_CreateUser(t *testing.T) { 14 | s := OpenStore() 15 | defer s.Close() 16 | 17 | // Create a new user. 18 | u := &main.User{Username: "susy"} 19 | if err := s.CreateUser(u); err != nil { 20 | t.Fatal(err) 21 | } else if u.ID != 1 { 22 | t.Fatalf("unexpected ID: %d", u.ID) 23 | } 24 | 25 | // Verify user can be retrieved. 26 | other, err := s.User(1) 27 | if err != nil { 28 | t.Fatal(err) 29 | } else if !reflect.DeepEqual(other, u) { 30 | t.Fatalf("unexpected user: %#v", other) 31 | } 32 | } 33 | 34 | // Ensure store can retrieve multiple users. 35 | func TestStore_Users(t *testing.T) { 36 | s := OpenStore() 37 | defer s.Close() 38 | 39 | // Create some users. 40 | if err := s.CreateUser(&main.User{Username: "susy"}); err != nil { 41 | t.Fatal(err) 42 | } else if err := s.CreateUser(&main.User{Username: "john"}); err != nil { 43 | t.Fatal(err) 44 | } 45 | 46 | // Verify users can be retrieved. 47 | if a, err := s.Users(); err != nil { 48 | t.Fatal(err) 49 | } else if !reflect.DeepEqual(a, []*main.User{ 50 | {ID: 1, Username: "susy"}, 51 | {ID: 2, Username: "john"}, 52 | }) { 53 | t.Fatalf("unexpected users: %#v", a) 54 | } 55 | } 56 | 57 | // Ensure store can update a user's username. 58 | func TestStore_SetUsername(t *testing.T) { 59 | s := OpenStore() 60 | defer s.Close() 61 | 62 | // Create a new user. 63 | if err := s.CreateUser(&main.User{Username: "susy"}); err != nil { 64 | t.Fatal(err) 65 | } 66 | 67 | // Update username. 68 | if err := s.SetUsername(1, "jimbo"); err != nil { 69 | t.Fatal(err) 70 | } 71 | 72 | // Verify username has changed. 73 | if u, err := s.User(1); err != nil { 74 | t.Fatal(err) 75 | } else if u.Username != "jimbo" { 76 | t.Fatalf("unexpected username: %s", u.Username) 77 | } 78 | } 79 | 80 | // Ensure store returns an error if user does not exist. 81 | func TestStore_SetUsername_ErrUserNotFound(t *testing.T) { 82 | s := OpenStore() 83 | defer s.Close() 84 | 85 | // Update username. 86 | if err := s.SetUsername(1, "jimbo"); err != main.ErrUserNotFound { 87 | t.Fatalf("unexpected error: %v", err) 88 | } 89 | } 90 | 91 | // Ensure store can remove a user. 92 | func TestStore_DeleteUser(t *testing.T) { 93 | s := OpenStore() 94 | defer s.Close() 95 | 96 | // Create a new user. 97 | if err := s.CreateUser(&main.User{Username: "susy"}); err != nil { 98 | t.Fatal(err) 99 | } 100 | 101 | // Delete the user. 102 | if err := s.DeleteUser(1); err != nil { 103 | t.Fatal(err) 104 | } 105 | 106 | // Verify user does not exist. 107 | if u, err := s.User(1); err != nil { 108 | t.Fatal(err) 109 | } else if u != nil { 110 | t.Fatalf("unexpected user: %#v", u) 111 | } 112 | } 113 | 114 | // Store is a test wrapper for main.Store. 115 | type Store struct { 116 | *main.Store 117 | } 118 | 119 | // NewStore returns a new instance of Store in a temporary path. 120 | func NewStore() *Store { 121 | f, err := ioutil.TempFile("", "appdevbolt-") 122 | if err != nil { 123 | panic(err) 124 | } 125 | f.Close() 126 | 127 | return &Store{ 128 | Store: &main.Store{ 129 | Path: f.Name(), 130 | }, 131 | } 132 | } 133 | 134 | // OpenStore returns a new, open instance of Store. 135 | func OpenStore() *Store { 136 | s := NewStore() 137 | if err := s.Open(); err != nil { 138 | panic(err) 139 | } 140 | return s 141 | } 142 | 143 | // Close closes the store and removes the underlying data file. 144 | func (s *Store) Close() error { 145 | defer os.Remove(s.Path) 146 | return s.Store.Close() 147 | } 148 | --------------------------------------------------------------------------------