├── .gitignore ├── LICENSE ├── README.md ├── cmd └── consulfs │ └── main.go ├── fs.go ├── go.mod └── go.sum /.gitignore: -------------------------------------------------------------------------------- 1 | /vendor 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2015 Benjamin Wester 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ConsulFS 2 | ======== 3 | 4 | ConsulFS implements a FUSE filesystem that is backed by a Consul Key-Value 5 | store. Each key in the key store is represented by a file. Read and write the 6 | file to get and put the key's value. "/" characters in a key name are used to 7 | break up the keys into different directories. 8 | 9 | This project should be considered alpha-quality. It works, but it might not 10 | handle everything you can throw at it. 11 | 12 | Installation 13 | ------------ 14 | ConsulFS uses [FUSE](http://fuse.sourceforge.net/) to implement a file system as 15 | a user-space process. Most popular Linux distributions will already include a 16 | FUSE module, so no further packages need to be installed. OS X systems will need 17 | to install a third-party file system package to mount FUSE volumes. [FUSE for OS 18 | X](https://osxfuse.github.io/) is the preferred FUSE package at the time of this 19 | writing. 20 | 21 | There are currently no binary packages available for ConsulFS, so you will need 22 | to build the package yourself. ConsulFS is written in Go, so install the Go 23 | toolchain available for your system. Look for a "golang" package in your 24 | system's package manager (Linux) or in [Homebrew](http://brew.sh/) (OS X). 25 | Binary packages can also be downloaded from [golang.org](http://golang.org/). 26 | 27 | Once FUSE and Go are installed, you can download, build, and install ConsulFS by 28 | running the following command: 29 | 30 | $ go get github.com/bwester/consulfs/cmd/consulfs 31 | 32 | That will create the binary `$GOPATH/bin/consulfs`, which you can call directly 33 | or copy to your preferred location for binaries. 34 | 35 | Command line usage 36 | ------------------ 37 | The `consulfs` command is used to mount a Consul Key-Value store onto a file 38 | system path. The basic form of the command is: 39 | 40 | $ consulfs [options] [consul_address] /mount/path 41 | 42 | There aren't many options, but you can run `consulfs --help` to see them. The 43 | address of a Consul agent is optional, and if you omit it, the local agent will 44 | be used. 45 | 46 | ConsulFS runs in the foreground, where it displays all error messages. If you 47 | interrupt the process with ^C or send it a SIGTERM or SIGINT, it will attempt to 48 | unmount the file system before exiting. Or, if the mount point is unmounted 49 | through other means ("umount" or "fusermount"), the process will exit. 50 | 51 | File system model 52 | ----------------- 53 | ConsulFS represents each key in Consul as a file in the file system. The slash 54 | character "/" is a key interpreted as a directory separator, and key names are 55 | broken up in the straightforward way. Reads and writes on a file will cause GETs 56 | and PUTs, respectively, to the key in order to fetch and update the key's value. 57 | 58 | Consul doesn't itself store directores; those are simply inferred from the keys. 59 | As such, Consul is not updated when creating or removing an empty directory. 60 | 61 | For example, the key "foo/bar" is exposed to the file system as the file "bar" 62 | in the directory "foo". When "bar" is read, a GET is performed for "foo/bar" and 63 | its value will be read. Writes to "bar" first GET "foo/bar", change the written 64 | bytes, then save them with a PUT to "foo/bar" containing the entire key's 65 | contents. In the directory "foo", if you write to a new file "baz", the key 66 | "foo/baz" will be written. 67 | 68 | ConsulFS does not currently support durable timestamps, owners, or mode 69 | bits--there isn't enough metadata in Consul to store these! The timestamps are 70 | faked, and owner/mode bits are fixed: attempts to change them will return an 71 | error. 72 | 73 | Consistency 74 | ----------- 75 | ConsulFS doesn't perform any local caching of key values (this was much easier 76 | to write!), so every read will always get the latest value. Writes to a file 77 | _atomically_ PUT the key's value. This might cause an occasional `write()` 78 | syscall to fail in the presence of concurrent writers. As you might imagine, 79 | ConsulFS is currently kinda slow when accessing remote file systems. Future 80 | versions may introduce a data cache. 81 | 82 | Directory listings are briefly cached, for about 1 second. The access patterns 83 | generated by FUSE repeatedly access this kind of metadata, so a small cache is 84 | required to get any kind of usable performance. 85 | -------------------------------------------------------------------------------- /cmd/consulfs/main.go: -------------------------------------------------------------------------------- 1 | // consulfs is a command for mounting Consul-FS to your filesystem. A basic execution, 2 | // 3 | // $ consulfs /mnt/kv 4 | // 5 | // will mount the local Consul agent's KV store to the `/mnt/kv` directory. 6 | // Optionally, a two-argument form allows you specify the location of the Consul agent to 7 | // contact: 8 | // 9 | // $ consulfs http://consul0.mydomain.com:5678 /mnt/kv 10 | // 11 | // For more information about the file system itself, refer to the package documentation 12 | // in the main "github.com/bwester/consulfs" package. 13 | package main 14 | 15 | import ( 16 | "flag" 17 | "fmt" 18 | "os" 19 | "os/signal" 20 | "path/filepath" 21 | "syscall" 22 | "time" 23 | 24 | "bazil.org/fuse" 25 | "bazil.org/fuse/fs" 26 | consul "github.com/hashicorp/consul/api" 27 | "github.com/sirupsen/logrus" 28 | "golang.org/x/net/context" 29 | 30 | "github.com/bwester/consulfs" 31 | ) 32 | 33 | // Default timeout for all requests. 60s is the default for OSXFUSE. 34 | const defaultTimeout = "60s" 35 | 36 | func init() { 37 | flag.Usage = func() { 38 | fmt.Fprintf( 39 | os.Stderr, 40 | "usage: %s [flags] [server_addr] mount_point\n\nAvailable flags:\n", 41 | filepath.Base(os.Args[0]), 42 | ) 43 | flag.PrintDefaults() 44 | os.Exit(1) 45 | } 46 | } 47 | 48 | func main() { 49 | allowOther := flag.Bool("allow-other", false, "allow all users access to the filesystem") 50 | allowRoot := flag.Bool("allow-root", false, "allow root to access the filesystem") 51 | debug := flag.Bool("debug", false, "enable debug output") 52 | gid := flag.Int("gid", os.Getgid(), "set the GID that should own all files") 53 | perm := flag.Int("perm", 0, "set the file permission flags for all files") 54 | ro := flag.Bool("ro", false, "mount the filesystem read-only") 55 | root := flag.String("root", "", "path in Consul to the root of the filesystem") 56 | timeout := flag.String("timeout", defaultTimeout, "timeout for Consul requests") 57 | tokenFile := flag.String("token-file", "", "use the ACL token from this file to connect to Consul") 58 | uid := flag.Int("uid", os.Getuid(), "set the UID that should own all files") 59 | flag.Parse() 60 | 61 | logger := logrus.New() 62 | if *debug { 63 | logger.Level = logrus.DebugLevel 64 | } 65 | 66 | consulConfig := consul.DefaultConfig() 67 | if *tokenFile != "" { 68 | consulConfig.TokenFile = *tokenFile 69 | } 70 | var mountPoint string 71 | switch flag.NArg() { 72 | case 1: 73 | mountPoint = flag.Arg(0) 74 | case 2: 75 | consulConfig.Address = flag.Arg(0) 76 | mountPoint = flag.Arg(1) 77 | default: 78 | flag.Usage() 79 | } 80 | 81 | // Initialize a Consul client. TODO: connection parameters 82 | client, err := consul.NewClient(consulConfig) 83 | if err != nil { 84 | logrus.NewEntry(logger).WithError(err).Error("could not initialize consul") 85 | os.Exit(1) 86 | } 87 | 88 | // Configure some mount options 89 | timeoutDuration, err := time.ParseDuration(*timeout) 90 | if err != nil { 91 | logrus.NewEntry(logger).WithError(err).Fatal("invalid -timeout value") 92 | } 93 | mountOptions := []fuse.MountOption{ 94 | fuse.DefaultPermissions(), 95 | fuse.DaemonTimeout(fmt.Sprint(int64(timeoutDuration.Seconds() + 1))), 96 | fuse.NoAppleDouble(), 97 | fuse.NoAppleXattr(), 98 | } 99 | if *allowOther { 100 | mountOptions = append(mountOptions, fuse.AllowOther()) 101 | } 102 | if *allowRoot { 103 | mountOptions = append(mountOptions, fuse.AllowRoot()) 104 | } 105 | if *ro { 106 | mountOptions = append(mountOptions, fuse.ReadOnly()) 107 | } 108 | 109 | // Mount the file system to start receiving FS events at the mount point. 110 | logger.WithField("location", mountPoint).Info("mounting kvfs") 111 | conn, err := fuse.Mount(mountPoint, mountOptions...) 112 | if err != nil { 113 | logrus.NewEntry(logger).WithError(err).Fatal("error mounting kvfs") 114 | } 115 | defer conn.Close() 116 | 117 | // Try to cleanly unmount the FS if SIGINT or SIGTERM is received 118 | sigs := make(chan os.Signal, 10) 119 | signal.Notify(sigs, os.Interrupt, syscall.SIGTERM) 120 | go func() { 121 | for sig := range sigs { 122 | logger.WithField("signal", sig).Info("attempting to unmount") 123 | err := fuse.Unmount(mountPoint) 124 | if err != nil { 125 | logrus.NewEntry(logger).WithError(err).Error("cannot unmount") 126 | } 127 | } 128 | }() 129 | 130 | // Create a file system object and start handing its requests 131 | server := fs.New(conn, &fs.Config{ 132 | Debug: func(m interface{}) { logger.Debug(m) }, 133 | WithContext: func(ctx context.Context, req fuse.Request) context.Context { 134 | // The returned cancel function doesn't matter: the request handler will 135 | // cancel the parent context at the end of the request. 136 | newCtx, _ := context.WithTimeout(ctx, timeoutDuration) 137 | return newCtx 138 | }, 139 | }) 140 | f := &consulfs.ConsulFS{ 141 | Client: client, 142 | Logger: logger, 143 | UID: uint32(*uid), 144 | GID: uint32(*gid), 145 | Perms: os.FileMode(*perm), 146 | RootPath: *root, 147 | } 148 | err = server.Serve(f) 149 | if err != nil { 150 | // Not sure what would cause Serve() to exit with an error 151 | logrus.NewEntry(logger).WithError(err).Error("error serving filesystem") 152 | } 153 | 154 | // Wait for the FUSE connection to end 155 | <-conn.Ready 156 | if conn.MountError != nil { 157 | logrus.NewEntry(logger).WithError(conn.MountError).Error("unmount error") 158 | os.Exit(1) 159 | } else { 160 | logger.Info("file system exiting normally") 161 | } 162 | } 163 | -------------------------------------------------------------------------------- /fs.go: -------------------------------------------------------------------------------- 1 | // Package consulfs implements a FUSE filesystem that is backed by a Consul Key-Value 2 | // store. 3 | // 4 | // API Usage 5 | // --------- 6 | // ConsulFS is implemented using the "bazil.org/fuse" package as a file system service. 7 | // Refer to the fuse package for complete documentation on how to create a new FUSE 8 | // connection that services a mount point. To have that mount point serve ConsulFS files, 9 | // create a new instance of `ConsulFS` and pass it to the 10 | // "bazil.org/fuse/fs".Server.Serve() method. The source code for the mounter at 11 | // "github.com/bwester/consulfs/cmd/consulfs" gives a full example of how to perform the 12 | // mounting. 13 | // 14 | // The `ConsulFS` instance contains common configuration data and is referenced by all 15 | // file and directory inodes. Notably, 'UID' and 'GID' set the uid and gid ownership of 16 | // all files. The `Consul` option is used to perform all Consul HTTP RPCs. The 17 | // `CancelConsulKV` struct is included in this package as a wrapper around the standard 18 | // Consul APIs. It is vital for system stability that the file system not get into an 19 | // uninteruptable sleep waiting for a remote RPC to complete, so CancelConsulKV will 20 | // abandon requests when needed. 21 | package consulfs 22 | 23 | import ( 24 | "os" 25 | "sort" 26 | "strings" 27 | "sync" 28 | "syscall" 29 | "time" 30 | 31 | "bazil.org/fuse" 32 | "bazil.org/fuse/fs" 33 | "bazil.org/fuse/fuseutil" 34 | consul "github.com/hashicorp/consul/api" 35 | "github.com/sirupsen/logrus" 36 | "golang.org/x/net/context" 37 | ) 38 | 39 | // Unsupported features: 40 | // * Changing file modes beyond the initial value 41 | // * Changing owners/groups beyond the initial value 42 | // * Querying Consul ACLs to determine access permissions 43 | // * Renaming directories 44 | // 45 | // Incomplete features: 46 | // * Renaming files 47 | // * Testing on Linux 48 | // * Testing 49 | // * Caching 50 | // 51 | // Known Bugs: 52 | // * Opening a file in append mode and writing to it doesn't append. Instead, data 53 | // is overwritten at the beginning of the file. Unfortunately, the kernel doesn't expose 54 | // the O_APPEND flag to FUSE open requests. It probably does the appends by issuing 55 | // writes to the "end" of the file, where the end is defined by the size, which is 56 | // faked to be zero. 57 | // * When using "allow_{root,other}" mount options on OS X, directory entries are not 58 | // refreshed. The immediate cause seems to be that the fuse.Server system caches old 59 | // dirents. But I don't yet understand why the unshared (default) option refrehses its 60 | // caches more frequently. 61 | // 62 | // With its current feature set, ConsulFS can be used for basic access with core POSIX tools. 63 | // More complex uses, like compiling and linking an executable, break horribly in strange 64 | // ways. 65 | 66 | // MaxWriteAttempts sets the number of time a write will be attempted before an 67 | // error is returned. 68 | const MaxWriteAttempts = 10 69 | 70 | // consulFile is a single file's inode in the filesystem. It is backed by a key in Consul. 71 | type consulFile struct { 72 | ConsulFS *ConsulFS 73 | Key string // The full keyname in Consul 74 | 75 | // Mutex guards all mutable metadata 76 | Mutex sync.Mutex 77 | Ctime time.Time // File attr 78 | Mtime time.Time // File attr 79 | Atime time.Time // File attr 80 | IsOpen bool // Is there an open handle to this file? 81 | Size uint64 // If the file is open, the expected file size 82 | Deleted bool // Whether this file has been deleted 83 | Buf []byte // If the file is deleted, buffers data locally 84 | } 85 | 86 | // deletedUnlocked returns the Deleted field to a caller that hasn't locked the file. 87 | func (file *consulFile) deletedUnlocked() bool { 88 | file.Mutex.Lock() 89 | defer file.Mutex.Unlock() 90 | return file.Deleted 91 | } 92 | 93 | // SetDeleted marks this file as deleted. 94 | // 95 | // If the file is open, Posix says those processes should continue to operate on the file 96 | // as if it exists, but when they close, it is removed. These semantics are preserved by 97 | // caching a copy of the file and operating on that copy, letting the key on Consul be 98 | // deleted eagerly. 99 | func (file *consulFile) SetDeleted(ctx context.Context) error { 100 | // If the file is already deleted, there is nothing more to do 101 | file.Mutex.Lock() 102 | if file.Deleted { 103 | file.Mutex.Unlock() 104 | file.ConsulFS.Logger.WithField("key", file.Key).Warning("SetDeleted() on deleted file") 105 | return fuse.ENOENT 106 | } 107 | if !file.IsOpen { 108 | file.Deleted = true 109 | file.Buf = make([]byte, 0) // just in case 110 | file.Mutex.Unlock() 111 | return nil 112 | } 113 | file.Mutex.Unlock() 114 | 115 | // Get a copy of the value to cache 116 | pair, _, err := file.ConsulFS.get(ctx, file.Key, nil) 117 | 118 | file.Mutex.Lock() 119 | defer file.Mutex.Unlock() 120 | if file.Deleted { 121 | file.ConsulFS.Logger.WithField("key", file.Key).Warning("SetDeleted() file became deleted mid-call") 122 | return fuse.ENOENT 123 | } 124 | if err == context.Canceled || err == context.DeadlineExceeded { 125 | return fuse.EINTR 126 | } 127 | if err != nil { 128 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 129 | "key": file.Key, 130 | logrus.ErrorKey: err, 131 | }).Error("consul read error") 132 | return fuse.EIO 133 | } 134 | 135 | file.Deleted = true 136 | if pair == nil { 137 | // Key must have been deleted externally. That data's gone, no way to preserve 138 | // Posix semantics now. But the local entry still needs to be deleted so the 139 | // file doesn't get opened again. 140 | file.Buf = make([]byte, 0) 141 | } else { 142 | file.Buf = pair.Value 143 | } 144 | return nil 145 | } 146 | 147 | // lockedAttr fills in an Attr struct for this file. Call when the file's mutex is locked. 148 | func (file *consulFile) lockedAttr(attr *fuse.Attr) { 149 | attr.Mode = file.ConsulFS.mode() 150 | if !file.Deleted { 151 | attr.Nlink = 1 152 | } 153 | // Timestamps aren't reflected in Consul, but it doesn't hurt to fake them 154 | attr.Ctime = file.Ctime 155 | attr.Mtime = file.Mtime 156 | attr.Atime = file.Atime 157 | attr.Uid = file.ConsulFS.UID 158 | attr.Gid = file.ConsulFS.GID 159 | if file.IsOpen { 160 | // Some applications like seeking... 161 | attr.Size = file.Size 162 | } 163 | } 164 | 165 | // Attr implements the Node interface. It is called when fetching the inode attribute for 166 | // this file (e.g., to service stat(2)). 167 | func (file *consulFile) Attr(ctx context.Context, attr *fuse.Attr) error { 168 | file.Mutex.Lock() 169 | defer file.Mutex.Unlock() 170 | file.lockedAttr(attr) 171 | return nil 172 | } 173 | 174 | // readSession is like readAll, except it returns the key's "Session" metadata instead of 175 | // its "Value". 176 | func (file *consulFile) readSession(ctx context.Context) ([]byte, error) { 177 | if file.deletedUnlocked() { 178 | return nil, nil 179 | } 180 | pair, _, err := file.ConsulFS.get(ctx, file.Key, nil) 181 | if file.deletedUnlocked() { 182 | return nil, nil 183 | } 184 | if err == context.Canceled || err == context.DeadlineExceeded { 185 | return nil, fuse.EINTR 186 | } 187 | if err != nil { 188 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 189 | "key": file.Key, 190 | logrus.ErrorKey: err, 191 | }).Error("consul read error") 192 | return nil, fuse.EIO 193 | } 194 | if pair == nil { 195 | return nil, fuse.ENOENT 196 | } 197 | return []byte(pair.Session), nil 198 | } 199 | 200 | // Getxattr fetches the contents a named extended attribute. The name doesn't have to have 201 | // been returned by a previous Listxattr(). 202 | func (file *consulFile) Getxattr( 203 | ctx context.Context, 204 | req *fuse.GetxattrRequest, 205 | resp *fuse.GetxattrResponse, 206 | ) error { 207 | var data []byte 208 | var err error 209 | switch req.Name { 210 | case "session": 211 | data, err = file.readSession(ctx) 212 | default: 213 | err = fuse.ErrNoXattr 214 | } 215 | if err != nil { 216 | return err 217 | } 218 | // OSXFUSE returns an "[Errno 34] Result too large" error the userspace process when 219 | // this method attempts to return an empty data set, so it's easier to pretend the 220 | // xattr never existed. 221 | if len(data) == 0 { 222 | return fuse.ErrNoXattr 223 | } 224 | if req.Position >= uint32(len(data)) { 225 | data = nil 226 | } else { 227 | data = data[req.Position:] 228 | } 229 | if req.Size != 0 && uint32(len(data)) > req.Size { 230 | data = data[:req.Size] 231 | } 232 | resp.Xattr = data 233 | return nil 234 | } 235 | 236 | // BufferRead returns locally-buffered file contents, which will only be used if the file 237 | // is deleted. 238 | func (file *consulFile) BufferRead() ([]byte, bool) { 239 | file.Mutex.Lock() 240 | defer file.Mutex.Unlock() 241 | if !file.Deleted { 242 | return nil, false 243 | } 244 | data := make([]byte, len(file.Buf)) 245 | copy(data, file.Buf) 246 | return data, true 247 | } 248 | 249 | // Read implements the HandleReader interface. It is called to handle every read request. 250 | // Because the file is opened in DirectIO mode, the kernel will not cache any file data. 251 | func (file *consulFile) Read( 252 | ctx context.Context, 253 | req *fuse.ReadRequest, 254 | resp *fuse.ReadResponse, 255 | ) error { 256 | data, err := file.readAll(ctx) 257 | if err != nil { 258 | return err 259 | } 260 | fuseutil.HandleRead(req, resp, data) 261 | return nil 262 | } 263 | 264 | // readAll handles every read request by fetching the key from the server. This leads to 265 | // simple consistency guarantees, as there is no caching, but performance may suffer in 266 | // distributed settings. It intentionally does not implement the fs.ReadAller interface to 267 | // avoid the caching inherent in that interface. 268 | func (file *consulFile) readAll(ctx context.Context) ([]byte, error) { 269 | // If the file has been removed from its directory, then data will come from the local 270 | // cache only. 271 | if data, ok := file.BufferRead(); ok { 272 | return data, nil 273 | } 274 | 275 | // Note that for complex caching semantics: the key has a 'CreateIndex' property that 276 | // could be used to distinguish a file's generation. 277 | 278 | // Query Consul for the full value for the file's key 279 | pair, _, err := file.ConsulFS.get(ctx, file.Key, nil) 280 | if data, ok := file.BufferRead(); ok { 281 | return data, nil 282 | } 283 | if err == context.Canceled || err == context.DeadlineExceeded { 284 | return nil, fuse.EINTR 285 | } else if err != nil { 286 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 287 | "key": file.Key, 288 | logrus.ErrorKey: err, 289 | }).Error("consul read error") 290 | return nil, fuse.EIO 291 | } 292 | if pair == nil { 293 | return nil, fuse.ENOENT 294 | } 295 | return pair.Value, nil 296 | } 297 | 298 | // doWrite does the buffer manipulation to perform a write. Data buffers are kept 299 | // contiguous. 300 | func doWrite( 301 | offset int64, 302 | writeData []byte, 303 | fileData []byte, 304 | ) []byte { 305 | fileEnd := int64(len(fileData)) 306 | writeEnd := offset + int64(len(writeData)) 307 | var buf []byte 308 | if writeEnd > fileEnd { 309 | buf = make([]byte, writeEnd) 310 | if fileEnd <= offset { 311 | copy(buf, fileData) 312 | } else { 313 | copy(buf, fileData[:offset]) 314 | } 315 | } else { 316 | buf = fileData 317 | } 318 | copy(buf[offset:writeEnd], writeData) 319 | return buf 320 | } 321 | 322 | func (file *consulFile) bufferWrite(req *fuse.WriteRequest, resp *fuse.WriteResponse) bool { 323 | file.Mutex.Lock() 324 | defer file.Mutex.Unlock() 325 | if !file.Deleted { 326 | return false 327 | } 328 | file.Buf = doWrite(req.Offset, req.Data, file.Buf) 329 | resp.Size = len(req.Data) 330 | return true 331 | } 332 | 333 | // Write implements the HandleWriter interface. It is called on *every* write (DirectIO 334 | // mode) to allow this module to handle consistency itself. Current strategy is to read 335 | // the file, change the written portions, then write it back atomically. If the key was 336 | // updated between the read and the write, try again. 337 | func (file *consulFile) Write( 338 | ctx context.Context, 339 | req *fuse.WriteRequest, 340 | resp *fuse.WriteResponse, 341 | ) error { 342 | for attempts := 0; attempts < MaxWriteAttempts; attempts++ { 343 | if file.bufferWrite(req, resp) { 344 | return nil 345 | } 346 | pair, _, err := file.ConsulFS.get(ctx, file.Key, nil) 347 | if file.bufferWrite(req, resp) { 348 | return nil 349 | } 350 | if err == context.Canceled || err == context.DeadlineExceeded { 351 | return fuse.EINTR 352 | } else if err != nil { 353 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 354 | "key": file.Key, 355 | logrus.ErrorKey: err, 356 | }).Error("consul read error") 357 | return fuse.EIO 358 | } 359 | if pair == nil { 360 | return fuse.ENOENT 361 | } 362 | 363 | pair.Value = doWrite(req.Offset, req.Data, pair.Value) 364 | 365 | // Write it back! 366 | success, _, err := file.ConsulFS.cas(ctx, pair, nil) 367 | if file.bufferWrite(req, resp) { 368 | return nil 369 | } 370 | if err == context.Canceled || err == context.DeadlineExceeded { 371 | return fuse.EINTR 372 | } else if err != nil { 373 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 374 | "key": file.Key, 375 | logrus.ErrorKey: err, 376 | }).Error("consul write error") 377 | return fuse.EIO 378 | } 379 | if success { 380 | resp.Size = len(req.Data) 381 | return nil 382 | } 383 | file.ConsulFS.Logger.WithField("key", file.Key).Warning("write did not succeed") 384 | } 385 | file.ConsulFS.Logger.WithField("key", file.Key).Error("unable to perform timely write; aborting") 386 | return fuse.EIO 387 | } 388 | 389 | // Fsync implements the NodeFsyncer interface. It is called to explicitly flush cached 390 | // data to storage (e.g., on a fsync(2) call). Since data is not cached, this is a no-op. 391 | func (file *consulFile) Fsync( 392 | ctx context.Context, 393 | req *fuse.FsyncRequest, 394 | ) error { 395 | return nil 396 | } 397 | 398 | // doTruncate implements the buffer manipulation needed to truncate a file's contents. 399 | func doTruncate(buf []byte, size uint64) ([]byte, bool) { 400 | bufLen := uint64(len(buf)) 401 | if bufLen == size { 402 | return buf, false 403 | } 404 | if bufLen > size { 405 | return buf[:size], true 406 | } 407 | if uint64(cap(buf)) >= size { 408 | buf = buf[:size] 409 | for i := bufLen; i < size; i++ { 410 | buf[i] = 0 411 | } 412 | return buf, true 413 | } 414 | newBuf := make([]byte, size) 415 | copy(newBuf, buf) 416 | return newBuf, true 417 | } 418 | 419 | func (file *consulFile) bufferTruncate(size uint64) bool { 420 | file.Mutex.Lock() 421 | defer file.Mutex.Unlock() 422 | if !file.Deleted { 423 | return false 424 | } 425 | file.Buf, _ = doTruncate(file.Buf, size) 426 | return true 427 | } 428 | 429 | // Truncate sets a key's value to the given size, stripping data off the end or adding \0 430 | // as needed. Note that a Consul Key-Value pair has two data segments, "value" and 431 | // "flags," and this operation only changes the value. So to preserve the flags, a full 432 | // read-modify-write must be done, even when the value is cleared entirely. 433 | func (file *consulFile) Truncate( 434 | ctx context.Context, 435 | size uint64, 436 | ) error { 437 | for attempts := 0; attempts < MaxWriteAttempts; attempts++ { 438 | if file.bufferTruncate(size) { 439 | return nil 440 | } 441 | // Read the contents of the key 442 | pair, _, err := file.ConsulFS.get(ctx, file.Key, nil) 443 | if file.bufferTruncate(size) { 444 | return nil 445 | } 446 | if err == context.Canceled || err == context.DeadlineExceeded { 447 | return fuse.EINTR 448 | } else if err != nil { 449 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 450 | "key": file.Key, 451 | logrus.ErrorKey: err, 452 | }).Error("consul read error") 453 | return fuse.EIO 454 | } 455 | if pair == nil { 456 | return fuse.ENOENT 457 | } 458 | 459 | var changed bool 460 | pair.Value, changed = doTruncate(pair.Value, size) 461 | if !changed { 462 | return nil 463 | } 464 | 465 | // Write the results back 466 | success, _, err := file.ConsulFS.cas(ctx, pair, nil) 467 | if file.bufferTruncate(size) { 468 | return nil 469 | } 470 | if err == context.Canceled || err == context.DeadlineExceeded { 471 | return fuse.EINTR 472 | } else if err != nil { 473 | file.ConsulFS.Logger.WithFields(logrus.Fields{ 474 | "key": file.Key, 475 | logrus.ErrorKey: err, 476 | }).Error("consul write error") 477 | return fuse.EIO 478 | } 479 | if success { 480 | return nil 481 | } 482 | file.ConsulFS.Logger.WithField("key", file.Key).Warning("truncate did not succeed") 483 | } 484 | return fuse.EINTR 485 | } 486 | 487 | // Setattr implements the fs.NodeSetattrer interface. This is used by the kernel to 488 | // request metadata changes, including the file's size (used by ftruncate(2) or by 489 | // open("...", O_TRUNC) to clear a file's content). 490 | func (file *consulFile) Setattr( 491 | ctx context.Context, 492 | req *fuse.SetattrRequest, 493 | resp *fuse.SetattrResponse, 494 | ) error { 495 | if req.Valid.Uid() || req.Valid.Gid() { 496 | return fuse.ENOTSUP 497 | } 498 | // Support only idempotent writes. This is needed so cp(1) can copy a file. 499 | if req.Valid.Mode() && req.Mode != file.ConsulFS.mode() { 500 | return fuse.EPERM 501 | } 502 | // The truncate operation could fail, so do it first before altering any other file 503 | // metadata in an attempt to keep this setattr request atomic. 504 | if req.Valid.Size() { 505 | err := file.Truncate(ctx, req.Size) 506 | if err != nil { 507 | return err 508 | } 509 | } 510 | file.Mutex.Lock() 511 | defer file.Mutex.Unlock() 512 | now := time.Now() 513 | if req.Valid.Atime() { 514 | file.Atime = req.Atime 515 | } 516 | if req.Valid.AtimeNow() { 517 | file.Atime = now 518 | } 519 | if req.Valid.Mtime() { 520 | file.Mtime = req.Mtime 521 | } 522 | if req.Valid.MtimeNow() { 523 | file.Mtime = now 524 | } 525 | file.lockedAttr(&resp.Attr) 526 | return nil 527 | } 528 | 529 | // Open implements the NodeOpener interface. It is called the first time a file is opened 530 | // by any process. Further opens or FD duplications will reuse this handle. When all FDs 531 | // have been closed, Release() will be called. 532 | func (file *consulFile) Open( 533 | ctx context.Context, 534 | req *fuse.OpenRequest, 535 | resp *fuse.OpenResponse, 536 | ) (fs.Handle, error) { 537 | // Using the DirectIO flag disables the kernel buffer cache. *Every* read and write 538 | // will be passed directly to this module. This gives the module greater control over 539 | // the file's consistency model. 540 | resp.Flags |= fuse.OpenDirectIO 541 | file.Mutex.Lock() 542 | defer file.Mutex.Unlock() 543 | file.IsOpen = true 544 | return file, nil 545 | } 546 | 547 | // Release implements the HandleReleaser interface. It is called when all file descriptors 548 | // to the file have been closed. 549 | func (file *consulFile) Release(ctx context.Context, req *fuse.ReleaseRequest) error { 550 | file.Mutex.Lock() 551 | defer file.Mutex.Unlock() 552 | file.IsOpen = false 553 | return nil 554 | } 555 | 556 | // consulDir represents a directory inode in VFS. Directories don't actually exist in 557 | // Consul. TODO: discuss the strategy used to fake dirs. 558 | type consulDir struct { 559 | ConsulFS *ConsulFS 560 | Prefix string 561 | Level uint 562 | 563 | // The mutex protects cached directory contents 564 | mux sync.Mutex 565 | expires time.Time 566 | readIndex uint64 567 | files map[string]*consulFile 568 | dirs map[string]*consulDir 569 | } 570 | 571 | func (dir *consulDir) newFile(key string) *consulFile { 572 | now := time.Now() 573 | return &consulFile{ 574 | ConsulFS: dir.ConsulFS, 575 | Key: key, 576 | Ctime: now, 577 | Mtime: now, 578 | Atime: now, 579 | } 580 | } 581 | 582 | func (dir *consulDir) newDir(prefix string) *consulDir { 583 | return &consulDir{ 584 | ConsulFS: dir.ConsulFS, 585 | Prefix: prefix, 586 | Level: dir.Level + 1, 587 | files: make(map[string]*consulFile), 588 | dirs: make(map[string]*consulDir), 589 | } 590 | } 591 | 592 | // Attr implements the Node interface. It is called when fetching the inode attribute for 593 | // this directory (e.g., to service stat(2)). 594 | func (dir *consulDir) Attr(ctx context.Context, attr *fuse.Attr) error { 595 | attr.Mode = dir.mode() 596 | // Nlink should technically include all the files in the directory, but VFS seems fine 597 | // with the constant "2". 598 | attr.Nlink = 2 599 | attr.Uid = dir.ConsulFS.UID 600 | attr.Gid = dir.ConsulFS.GID 601 | return nil 602 | } 603 | 604 | // Listxattr implements the NodeListxattrer interface to retrieve a list of xattrs. 605 | // Directories have no xattrs. 606 | func (dir *consulDir) Listxattr( 607 | ctx context.Context, 608 | req *fuse.ListxattrRequest, 609 | resp *fuse.ListxattrResponse, 610 | ) error { 611 | return nil 612 | } 613 | 614 | // Getxattr implements the NodeGetxattrer interface to retrieve the contents of a specific 615 | // xattr. Directories have no xattrs. 616 | func (dir *consulDir) Getxattr( 617 | ctx context.Context, 618 | req *fuse.GetxattrRequest, 619 | resp *fuse.GetxattrResponse, 620 | ) error { 621 | return fuse.ErrNoXattr 622 | } 623 | 624 | func (dir *consulDir) mode() os.FileMode { 625 | mode := dir.ConsulFS.mode() | os.ModeDir 626 | // Add ?+x if ?+r is present 627 | if mode&0400 == 0400 { 628 | mode |= 0100 629 | } 630 | if mode&0040 == 0040 { 631 | mode |= 0010 632 | } 633 | if mode&0004 == 0004 { 634 | mode |= 0001 635 | } 636 | return mode 637 | } 638 | 639 | // Lookup implements the NodeStringLookuper interface, to look up a directory entry by 640 | // name. This is called to get the inode for the given name. The name doesn't have to have 641 | // been returned by ReadDirAll() for a process to attempt to find it! 642 | func (dir *consulDir) Lookup(ctx context.Context, name string) (fs.Node, error) { 643 | err := dir.refresh(ctx) 644 | if err != nil { 645 | return nil, err 646 | } 647 | dir.mux.Lock() 648 | defer dir.mux.Unlock() 649 | // Search directories first. If there is a key that ends in a "/", allow it 650 | // to be masked. 651 | if d, ok := dir.dirs[name]; ok { 652 | return d, nil 653 | } else if f, ok := dir.files[name]; ok { 654 | return f, nil 655 | } 656 | return nil, fuse.ENOENT 657 | } 658 | 659 | func (dir *consulDir) refresh(ctx context.Context) error { 660 | // Check if the cached directory listing has expired 661 | now := time.Now() 662 | dir.mux.Lock() 663 | expires := dir.expires 664 | dir.mux.Unlock() 665 | if expires.After(now) { 666 | return nil 667 | } 668 | 669 | // Call Consul to get an updated listing. This could block for a while, so 670 | // do not hold the dir lock while calling. 671 | keys, meta, err := dir.ConsulFS.keys(ctx, dir.Prefix, "/", nil) 672 | if err == context.Canceled || err == context.DeadlineExceeded { 673 | return fuse.EINTR 674 | } else if err != nil { 675 | dir.ConsulFS.Logger.WithFields(logrus.Fields{ 676 | "prefix": dir.Prefix, 677 | logrus.ErrorKey: err, 678 | }).Error("consul list error") 679 | return fuse.EIO 680 | } 681 | // Reminder: if the directory is empty, `keys` could be `nil`. 682 | 683 | // If another, later update completed while waiting on Consul, ignore 684 | // these results. 685 | dir.mux.Lock() 686 | defer dir.mux.Unlock() 687 | if dir.readIndex > meta.LastIndex { 688 | return nil 689 | } 690 | dir.readIndex = meta.LastIndex 691 | dir.expires = time.Now().Add(1 * time.Second) 692 | 693 | // Add new files and directories 694 | prefixLen := len(dir.Prefix) 695 | fileNames := map[string]bool{} 696 | for _, key := range keys { 697 | if !strings.HasPrefix(key, dir.Prefix) { 698 | dir.ConsulFS.Logger.WithFields(logrus.Fields{ 699 | "prefix": dir.Prefix, 700 | "key": key, 701 | }).Warning("list included invalid key") 702 | continue 703 | } 704 | if key == dir.Prefix { 705 | // Pathological case: a key's full name ended in "/", making it look like 706 | // a directory, and now the OS is trying to list that directory. 707 | continue 708 | } 709 | name := key[prefixLen:] 710 | if strings.HasSuffix(name, "/") { 711 | // Probably a directory 712 | dirName := name[:len(name)-1] 713 | if _, ok := dir.dirs[dirName]; !ok { 714 | dir.dirs[dirName] = dir.newDir(key) 715 | } 716 | } else { 717 | // Data-holding key 718 | if _, ok := dir.files[name]; !ok { 719 | dir.files[name] = dir.newFile(key) 720 | } 721 | fileNames[name] = true 722 | } 723 | } 724 | 725 | // Remove any files that are not present anymore 726 | for name := range dir.files { 727 | if !fileNames[name] { 728 | delete(dir.files, name) 729 | } 730 | } 731 | 732 | return nil 733 | } 734 | 735 | // ReadDirAll returns the entire contents of the directory when the directory is being 736 | // listed (e.g., with "ls"). 737 | func (dir *consulDir) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) { 738 | // Get Consul to refresh the local cache of the directory's entries 739 | err := dir.refresh(ctx) 740 | if err != nil { 741 | return nil, err 742 | } 743 | dir.mux.Lock() 744 | defer dir.mux.Unlock() 745 | 746 | ents := make([]fuse.Dirent, 2, len(dir.files)+len(dir.dirs)+2) 747 | ents[0] = fuse.Dirent{Name: ".", Type: fuse.DT_Dir} 748 | ents[1] = fuse.Dirent{Name: "..", Type: fuse.DT_Dir} 749 | for fileName := range dir.files { 750 | ents = append(ents, fuse.Dirent{ 751 | Name: fileName, 752 | Type: fuse.DT_File, 753 | }) 754 | } 755 | for dirName := range dir.dirs { 756 | ents = append(ents, fuse.Dirent{ 757 | Name: dirName, 758 | Type: fuse.DT_Dir, 759 | }) 760 | } 761 | return ents, nil 762 | } 763 | 764 | // Create implements the NodeCreater interface. It is called to create and open a new 765 | // file. The kernel will first try to Lookup the name, and this method will only be called 766 | // if the name didn't exist. 767 | func (dir *consulDir) Create( 768 | ctx context.Context, 769 | req *fuse.CreateRequest, 770 | resp *fuse.CreateResponse, 771 | ) (fs.Node, fs.Handle, error) { 772 | // The filename can't contain the path separator. That would mess up how "directories" 773 | // are listed. 774 | if strings.Contains(req.Name, "/") { 775 | return nil, nil, fuse.EPERM 776 | } 777 | 778 | // Create the key first, then insert it into the directory structure 779 | key := dir.Prefix + req.Name 780 | pair := &consul.KVPair{ 781 | Key: key, 782 | ModifyIndex: 0, // Write will fail if the key already exists 783 | Flags: 0, 784 | Value: []byte{}, 785 | } 786 | success, _, err := dir.ConsulFS.cas(ctx, pair, nil) 787 | if err == context.Canceled || err == context.DeadlineExceeded { 788 | return nil, nil, fuse.EINTR 789 | } else if err != nil { 790 | dir.ConsulFS.Logger.WithFields(logrus.Fields{ 791 | "key": key, 792 | logrus.ErrorKey: err, 793 | }).Error("consul create error") 794 | return nil, nil, fuse.EIO 795 | } 796 | 797 | dir.mux.Lock() 798 | defer dir.mux.Unlock() 799 | // Success or failure, once the Consul CAS operation completes without an error, the key 800 | // exists in the store. Make sure it's in the local cache. 801 | var file *consulFile 802 | var ok bool 803 | if file, ok = dir.files[req.Name]; !ok { 804 | file = dir.newFile(key) 805 | dir.files[req.Name] = file 806 | } 807 | if !success { 808 | file.ConsulFS.Logger.WithField("key", key).Warning("create failed") 809 | return nil, nil, fuse.EEXIST 810 | } 811 | // Just like in File.Open() 812 | resp.OpenResponse.Flags |= fuse.OpenDirectIO 813 | return file, file, nil 814 | } 815 | 816 | // RemoveDir is called to remove a directory 817 | func (dir *consulDir) RemoveDir(ctx context.Context, req *fuse.RemoveRequest) error { 818 | // Look in the cache to find the child directory being removed. 819 | dir.mux.Lock() 820 | childDir, ok := dir.dirs[req.Name] 821 | if !ok { 822 | dir.mux.Unlock() 823 | return fuse.ENOENT 824 | } 825 | dir.mux.Unlock() 826 | 827 | // Don't delete a directory with files in it. To do that, we have to refresh the 828 | // file listing of the directory. 829 | err := childDir.refresh(ctx) 830 | if err != nil { 831 | return err 832 | } 833 | dir.mux.Lock() 834 | defer dir.mux.Unlock() 835 | childDir.mux.Lock() 836 | defer childDir.mux.Unlock() 837 | 838 | // State could have changed while the refresh was happening 839 | childDir2, ok := dir.dirs[req.Name] 840 | if !ok { 841 | return fuse.ENOENT 842 | } 843 | if childDir != childDir2 { 844 | // Many concurrent removes? Just give up. 845 | return fuse.EIO 846 | } 847 | 848 | // Only delete an empty directory 849 | if len(childDir.dirs) > 0 || len(childDir.files) > 0 { 850 | return fuse.Errno(syscall.ENOTEMPTY) 851 | } 852 | delete(dir.dirs, req.Name) 853 | return nil 854 | } 855 | 856 | // RemoveFile is called to unlink a file. 857 | func (dir *consulDir) RemoveFile(ctx context.Context, req *fuse.RemoveRequest) error { 858 | // Get a reference to the file 859 | dir.mux.Lock() 860 | file, ok := dir.files[req.Name] 861 | dir.mux.Unlock() 862 | if !ok { 863 | // Something else removed it first? 864 | return nil 865 | } 866 | 867 | // Mark the file as deleted. If the file is open, this may make a blocking 868 | // call to Consul to cache the file's contents. If this call fails, the entire 869 | // Remove operation can be aborted without changing any state. 870 | err := file.SetDeleted(ctx) 871 | if err != nil { 872 | return err 873 | } 874 | 875 | // Once the file is marked as deleted, remove its entry so it can't be found 876 | // locally. 877 | dir.mux.Lock() 878 | file2, ok := dir.files[req.Name] 879 | if file != file2 || !ok { 880 | // Something else has already removed and replaced the file?! 881 | dir.mux.Unlock() 882 | return nil 883 | } 884 | delete(dir.files, req.Name) 885 | dir.mux.Unlock() 886 | 887 | // Finally, remove the file's key from Consul. Errors at this step are horrible. 888 | // Any process that had the file open already will be working on its own forked 889 | // copy, but the key will still exist on the server. 890 | _, err = dir.ConsulFS.delete(ctx, file.Key, nil) 891 | if err == context.Canceled || err == context.DeadlineExceeded { 892 | dir.ConsulFS.Logger.WithField("key", file.Key).Error("delete interrupted at a bad time") 893 | return fuse.EINTR 894 | } else if err != nil { 895 | dir.ConsulFS.Logger.WithFields(logrus.Fields{ 896 | "key": file.Key, 897 | logrus.ErrorKey: err, 898 | }).Error("consul delete error") 899 | return fuse.EIO 900 | } 901 | return nil 902 | } 903 | 904 | // Remove implements the NodeRemover interface. It is called to remove files or directory 905 | // from a directory's contents. 906 | func (dir *consulDir) Remove(ctx context.Context, req *fuse.RemoveRequest) error { 907 | if req.Dir { 908 | return dir.RemoveDir(ctx, req) 909 | } 910 | return dir.RemoveFile(ctx, req) 911 | } 912 | 913 | // Mkdir implements the NodeMkdirer interface. It is called to make a new directory. 914 | func (dir *consulDir) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) { 915 | // Since directories don't exist in Consul, this is easy! Make sure the name doesn't 916 | // already exist, then add an entry for it. 917 | err := dir.refresh(ctx) 918 | if err != nil { 919 | return nil, err 920 | } 921 | dir.mux.Lock() 922 | defer dir.mux.Unlock() 923 | 924 | if _, ok := dir.dirs[req.Name]; ok { 925 | return nil, fuse.EEXIST 926 | } 927 | if _, ok := dir.files[req.Name]; ok { 928 | return nil, fuse.EEXIST 929 | } 930 | newDir := dir.newDir(dir.Prefix + req.Name + "/") 931 | dir.dirs[req.Name] = newDir 932 | return newDir, nil 933 | } 934 | 935 | // lockSet holds a set of locked Dirs so they can be easily unlocked. 936 | type lockSet []*consulDir 937 | 938 | // Len implements the sort.Interface interface. Returns the number of Dirs in the set. 939 | func (ls lockSet) Len() int { 940 | return len([]*consulDir(ls)) 941 | } 942 | 943 | // Less implements the sort.Interface interface. Equivalent to: ls[i] < ls[j]. 944 | func (ls lockSet) Less(i, j int) bool { 945 | return ls[i].Level < ls[j].Level || 946 | (ls[i].Level == ls[j].Level && ls[i].Prefix < ls[j].Prefix) 947 | } 948 | 949 | // Swap implements the sort.Interface interface. Swaps two elements. 950 | func (ls lockSet) Swap(i, j int) { 951 | ls[i], ls[j] = ls[j], ls[i] 952 | } 953 | 954 | // Unlock will release the mutexes on every Dir in a lockSet. 955 | func (ls lockSet) Unlock() { 956 | var last *consulDir 957 | for _, d := range ls { 958 | if d != last { 959 | d.mux.Unlock() 960 | } 961 | last = d 962 | } 963 | } 964 | 965 | // lockDirs acquires the locks on the given directores in the canonical order to prevent 966 | // deadlocks. Dirs are ordered by level in the tree, then by prefix. This lock 967 | // order allows a parent directory to always be able to lock one of its children without 968 | // needing to drop its own lock first. 969 | func lockDirs(dirs ...*consulDir) lockSet { 970 | ls := lockSet(dirs) 971 | sort.Sort(ls) 972 | var last *consulDir 973 | for _, d := range ls { 974 | if d != last { 975 | d.mux.Lock() 976 | } 977 | last = d 978 | } 979 | return ls 980 | } 981 | 982 | // Rename implements the NodeRenamer interface. It's called to rename a file from one name 983 | // to another, possibly in another directory. There is no plan to support renaming 984 | // directories at this time. Consul doesn't have a rename operation, so the new name is 985 | // written and the old one deleted as two separate actions. If the new name already exists 986 | // as a file, it is replaced atomically. 987 | func (dir *consulDir) Rename( 988 | ctx context.Context, 989 | req *fuse.RenameRequest, 990 | newDirNode fs.Node, 991 | ) error { 992 | newDir, ok := newDirNode.(*consulDir) 993 | if !ok { 994 | return fuse.ENOTSUP 995 | } 996 | var ndRefresh chan error 997 | if newDir != dir { 998 | ndRefresh = make(chan error) 999 | go func() { ndRefresh <- dir.refresh(ctx) }() 1000 | } 1001 | err := dir.refresh(ctx) 1002 | if err != nil { 1003 | return err 1004 | } 1005 | if ndRefresh != nil { 1006 | err = <-ndRefresh 1007 | if err != nil { 1008 | return err 1009 | } 1010 | } 1011 | 1012 | // TODO: finish me 1013 | 1014 | return fuse.ENOTSUP 1015 | } 1016 | 1017 | // ConsulFS is the main file system object that represents a Consul Key-Value store. 1018 | type ConsulFS struct { 1019 | // Client is used to make all calls to Consul. 1020 | Client *consul.Client 1021 | 1022 | // UID contains the UID that will own all the files in the file system. 1023 | UID uint32 1024 | 1025 | // GID contains the GID that will own all the files in the file system. 1026 | GID uint32 1027 | 1028 | // Perms sets the file permission flags for all files and directories. If zero, a 1029 | // default of 0600 will be used. 1030 | Perms os.FileMode 1031 | 1032 | // RootPath contains the path to the root of the filesystem in Consul. This 1033 | // string will be prefixed to all paths requested from Consul. A path 1034 | // separator will be added if needed. 1035 | RootPath string 1036 | 1037 | // Messages will be sent to this logger 1038 | Logger *logrus.Logger 1039 | } 1040 | 1041 | // Root implements the fs.FS interface. It is called once to get the root directory inode 1042 | // for the mount point. 1043 | func (f *ConsulFS) Root() (fs.Node, error) { 1044 | return &consulDir{ 1045 | ConsulFS: f, 1046 | Prefix: f.rootPath(), 1047 | Level: 0, 1048 | files: make(map[string]*consulFile), 1049 | dirs: make(map[string]*consulDir), 1050 | }, nil 1051 | } 1052 | 1053 | func (f *ConsulFS) mode() os.FileMode { 1054 | if f.Perms == 0 { 1055 | return 0600 1056 | } 1057 | return f.Perms & os.ModePerm 1058 | } 1059 | 1060 | func (f *ConsulFS) rootPath() string { 1061 | if f.RootPath == "" || strings.HasSuffix(f.RootPath, "/") { 1062 | return f.RootPath 1063 | } 1064 | return f.RootPath + "/" 1065 | } 1066 | 1067 | // cas performs a compare-and-swap on a key. 1068 | func (f *ConsulFS) cas( 1069 | ctx context.Context, 1070 | p *consul.KVPair, 1071 | q *consul.WriteOptions, 1072 | ) (bool, *consul.WriteMeta, error) { 1073 | f.Logger.WithField("key", p.Key).Debug(" => CAS") 1074 | success, meta, err := f.Client.KV().CAS(p, q.WithContext(ctx)) 1075 | f.Logger.WithFields(logrus.Fields{ 1076 | "key": p.Key, 1077 | "kv": p, 1078 | "success": success, 1079 | "meta": meta, 1080 | logrus.ErrorKey: err, 1081 | }).Debug(" <= CAS") 1082 | return success, meta, err 1083 | } 1084 | 1085 | // delete removes a key and its data. 1086 | func (f *ConsulFS) delete( 1087 | ctx context.Context, 1088 | key string, 1089 | w *consul.WriteOptions, 1090 | ) (*consul.WriteMeta, error) { 1091 | f.Logger.WithField("key", key).Debug(" => Delete") 1092 | meta, err := f.Client.KV().Delete(key, w.WithContext(ctx)) 1093 | f.Logger.WithFields(logrus.Fields{ 1094 | "key": key, 1095 | "options": w, 1096 | "meta": meta, 1097 | logrus.ErrorKey: err, 1098 | }).Debug(" <= Delete") 1099 | return meta, err 1100 | } 1101 | 1102 | // get returns the current value of a key. 1103 | func (f *ConsulFS) get( 1104 | ctx context.Context, 1105 | key string, 1106 | q *consul.QueryOptions, 1107 | ) (*consul.KVPair, *consul.QueryMeta, error) { 1108 | f.Logger.WithField("key", key).Debug(" => Get") 1109 | pair, meta, err := f.Client.KV().Get(key, q.WithContext(ctx)) 1110 | f.Logger.WithFields(logrus.Fields{ 1111 | "key": key, 1112 | "options": q, 1113 | "kv": pair, 1114 | "meta": meta, 1115 | logrus.ErrorKey: err, 1116 | }).Debug(" <= Get") 1117 | return pair, meta, err 1118 | } 1119 | 1120 | // keys lists all keys under a prefix 1121 | func (f *ConsulFS) keys( 1122 | ctx context.Context, 1123 | prefix string, 1124 | separator string, 1125 | q *consul.QueryOptions, 1126 | ) ([]string, *consul.QueryMeta, error) { 1127 | f.Logger.WithField("prefix", prefix).Debug(" => Keys") 1128 | keys, meta, err := f.Client.KV().Keys(prefix, separator, q.WithContext(ctx)) 1129 | f.Logger.WithFields(logrus.Fields{ 1130 | "prefix": prefix, 1131 | "options": q, 1132 | "keys": keys, 1133 | "meta": meta, 1134 | logrus.ErrorKey: err, 1135 | }).Debug(" <= Keys") 1136 | return keys, meta, err 1137 | } 1138 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/bwester/consulfs 2 | 3 | go 1.15 4 | 5 | require ( 6 | bazil.org/fuse v0.0.0-20200117220432-d6840b2e5583 7 | github.com/hashicorp/consul/api v1.7.0 8 | github.com/sirupsen/logrus v1.7.0 9 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b 10 | ) 11 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | bazil.org/fuse v0.0.0-20200117220432-d6840b2e5583 h1:Gs44vb0fXEpAcOKFwHdvi2zUe46jxgus/eVahufqpOo= 2 | bazil.org/fuse v0.0.0-20200117220432-d6840b2e5583/go.mod h1:FbcW6z/2VytnFDhZfumh8Ss8zxHE6qpMP5sHTRe0EaM= 3 | github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= 4 | github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da h1:8GUt8eRujhVEGZFFEjBj46YV4rDjvGrNxb0KMWYkL2I= 5 | github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= 6 | github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= 7 | github.com/armon/go-radix v1.0.0/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= 8 | github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= 9 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 10 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 11 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 12 | github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= 13 | github.com/fatih/color v1.9.0 h1:8xPHl4/q1VyqGIPif1F+1V3Y3lSmrq01EabUW3CoW5s= 14 | github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU= 15 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c h1:964Od4U6p2jUkFxvCydnIczKteheJEzHRToSGK3Bnlw= 16 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= 17 | github.com/hashicorp/consul/api v1.7.0 h1:tGs8Oep67r8CcA2Ycmb/8BLBcJ70St44mF2X10a/qPg= 18 | github.com/hashicorp/consul/api v1.7.0/go.mod h1:1NSuaUUkFaJzMasbfq/11wKYWSR67Xn6r2DXKhuDNFg= 19 | github.com/hashicorp/consul/sdk v0.6.0 h1:FfhMEkwvQl57CildXJyGHnwGGM4HMODGyfjGwNM1Vdw= 20 | github.com/hashicorp/consul/sdk v0.6.0/go.mod h1:fY08Y9z5SvJqevyZNy6WWPXiG3KwBPAvlcdx16zZ0fM= 21 | github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= 22 | github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= 23 | github.com/hashicorp/go-cleanhttp v0.5.1 h1:dH3aiDG9Jvb5r5+bYHsikaOUIpcM0xvgMXVoDkXMzJM= 24 | github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= 25 | github.com/hashicorp/go-hclog v0.12.0 h1:d4QkX8FRTYaKaCZBoXYY8zJX2BXjWxurN/GA2tkrmZM= 26 | github.com/hashicorp/go-hclog v0.12.0/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ= 27 | github.com/hashicorp/go-immutable-radix v1.0.0 h1:AKDB1HM5PWEA7i4nhcpwOrO2byshxBjXVn/J/3+z5/0= 28 | github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= 29 | github.com/hashicorp/go-msgpack v0.5.3 h1:zKjpN5BK/P5lMYrLmBHdBULWbJ0XpYR+7NGzqkZzoD4= 30 | github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= 31 | github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uPribsnS6o= 32 | github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= 33 | github.com/hashicorp/go-multierror v1.1.0 h1:B9UzwGQJehnUY1yNrnwREHc3fGbC2xefo8g4TbElacI= 34 | github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA= 35 | github.com/hashicorp/go-rootcerts v1.0.2 h1:jzhAVGtqPKbwpyCPELlgNWhE1znq+qwJtW5Oi2viEzc= 36 | github.com/hashicorp/go-rootcerts v1.0.2/go.mod h1:pqUvnprVnM5bf7AOirdbb01K4ccR319Vf4pU3K5EGc8= 37 | github.com/hashicorp/go-sockaddr v1.0.0 h1:GeH6tui99pF4NJgfnhp+L6+FfobzVW3Ah46sLo0ICXs= 38 | github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= 39 | github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= 40 | github.com/hashicorp/go-uuid v1.0.0 h1:RS8zrF7PhGwyNPOtxSClXXj9HA8feRnJzgnI1RJCSnM= 41 | github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 42 | github.com/hashicorp/go-uuid v1.0.1 h1:fv1ep09latC32wFoVwnqcnKJGnMSdBanPczbHAYm1BE= 43 | github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= 44 | github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo= 45 | github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= 46 | github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= 47 | github.com/hashicorp/mdns v1.0.1/go.mod h1:4gW7WsVCke5TE7EPeYliwHlRUyBtfCwuFwuMg2DmyNY= 48 | github.com/hashicorp/memberlist v0.2.2 h1:5+RffWKwqJ71YPu9mWsF7ZOscZmwfasdA8kbdC7AO2g= 49 | github.com/hashicorp/memberlist v0.2.2/go.mod h1:MS2lj3INKhZjWNqd3N0m3J+Jxf3DAOnAH9VT3Sh9MUE= 50 | github.com/hashicorp/serf v0.9.3 h1:AVF6JDQQens6nMHT9OGERBvK0f8rPrAGILnsKLr6lzM= 51 | github.com/hashicorp/serf v0.9.3/go.mod h1:UWDWwZeL5cuWDJdl0C6wrvrUwEqtQ4ZKBKKENpqIUyk= 52 | github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs= 53 | github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= 54 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= 55 | github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= 56 | github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= 57 | github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= 58 | github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= 59 | github.com/mattn/go-colorable v0.1.6 h1:6Su7aK7lXmJ/U79bYtBjLNaha4Fs1Rg9plHpcH+vvnE= 60 | github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= 61 | github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= 62 | github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= 63 | github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84= 64 | github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE= 65 | github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY= 66 | github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= 67 | github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= 68 | github.com/miekg/dns v1.1.26 h1:gPxPSwALAeHJSjarOs00QjVdV9QoBvc1D2ujQUr5BzU= 69 | github.com/miekg/dns v1.1.26/go.mod h1:bPDLeHnStXmXAq1m/Ch/hvfNHr14JKNPMBo3VZKjuso= 70 | github.com/mitchellh/cli v1.1.0/go.mod h1:xcISNoH86gajksDmfB23e/pu+B+GeFRMYmoHXxx3xhI= 71 | github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y= 72 | github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= 73 | github.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdIPrefOvVG1VZ96U0= 74 | github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= 75 | github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 76 | github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE= 77 | github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= 78 | github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c h1:Lgl0gzECD8GnQ5QCWA8o6BtfL6mDH5rQgM4/fX3avOs= 79 | github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= 80 | github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= 81 | github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 82 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 83 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 84 | github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= 85 | github.com/posener/complete v1.2.3/go.mod h1:WZIdtGGp+qx0sLrYKtIRAruyNpv6hFCicSgv7Sy7s/s= 86 | github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= 87 | github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 h1:nn5Wsu0esKSJiIVhscUtVbo7ada43DJhG55ua/hjS5I= 88 | github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= 89 | github.com/sirupsen/logrus v1.7.0 h1:ShrD1U9pZB12TX0cVy0DtePoCH97K8EtX+mg7ZARUtM= 90 | github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= 91 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 92 | github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w= 93 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 94 | github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= 95 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= 96 | github.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c h1:u6SKchux2yDvFQnDHS3lPnIRmfVJ5Sxy3ao2SIdysLQ= 97 | github.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c/go.mod h1:hzIxponao9Kjc7aWznkXaL4U4TWaDSs8zcsY4Ka08nM= 98 | golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= 99 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 100 | golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY= 101 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI= 102 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 103 | golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= 104 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 105 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 106 | golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 107 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b h1:uwuIcX0g4Yl1NC5XAz37xsr2lTtcqevgzYNVt49waME= 108 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= 109 | golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 h1:YUO/7uOKsKeq9UokNS62b8FYywz3ker1l1vDZRCRefw= 110 | golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 111 | golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU= 112 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 113 | golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 114 | golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 115 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 116 | golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 117 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 118 | golang.org/x/sys v0.0.0-20190922100055-0a153f010e69/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 119 | golang.org/x/sys v0.0.0-20190924154521-2837fb4f24fe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 120 | golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 121 | golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 122 | golang.org/x/sys v0.0.0-20191210023423-ac6580df4449 h1:gSbV7h1NRL2G1xTg/owz62CST1oJBmxy4QpMMregXVQ= 123 | golang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 124 | golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 125 | golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 126 | golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 127 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f h1:+Nyd8tzPX9R7BWHguqsrbFdRx3WQ/1ib8I44HXV5yTA= 128 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 129 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 130 | golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= 131 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= 132 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 133 | golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 134 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 135 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 136 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= 137 | gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 138 | gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 139 | gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= 140 | gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= 141 | --------------------------------------------------------------------------------