├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── cmd └── vfsgendev │ ├── generate.go │ ├── main.go │ └── parse.go ├── commentwriter.go ├── doc.go ├── generator.go ├── generator_test.go ├── go.mod ├── options.go ├── stringwriter.go └── test ├── doc.go ├── test_gen.go ├── test_test.go └── test_vfsdata_test.go /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | Contributing 2 | ============ 3 | 4 | vfsgen is open source, thanks for considering contributing! 5 | 6 | Please note that vfsgen aims to be simple and minimalistic, with as little to configure as possible. If you'd like to remove or simplify code (while having tests continue to pass), fix bugs, or improve code (e.g., add missing error checking, etc.), PRs and issues are welcome. 7 | 8 | However, if you'd like to add new functionality that increases complexity or scope, please make an issue and discuss your proposal first. I'm unlikely to accept such changes outright. It might be that your request is already a part of other similar packages, or it might fit in their scope better. See [Comparison and Alternatives](https://github.com/shurcooL/vfsgen/tree/README-alternatives-and-comparison-section#comparison) sections. 9 | 10 | Thank you! 11 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2015 Dmitri Shuralyov 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | vfsgen 2 | ====== 3 | 4 | [![Go Reference](https://pkg.go.dev/badge/github.com/shurcooL/vfsgen.svg)](https://pkg.go.dev/github.com/shurcooL/vfsgen) 5 | 6 | Package vfsgen takes an http.FileSystem (likely at `go generate` time) and 7 | generates Go code that statically implements the provided http.FileSystem. 8 | 9 | Features: 10 | 11 | - Efficient generated code without unneccessary overhead. 12 | 13 | - Uses gzip compression internally (selectively, only for files that compress well). 14 | 15 | - Enables direct access to internal gzip compressed bytes via an optional interface. 16 | 17 | - Outputs `gofmt`ed Go code. 18 | 19 | Installation 20 | ------------ 21 | 22 | ```sh 23 | go get github.com/shurcooL/vfsgen 24 | ``` 25 | 26 | Usage 27 | ----- 28 | 29 | Package `vfsgen` is a Go code generator library. It has a `Generate` function that takes an input filesystem (as a [`http.FileSystem`](https://godoc.org/net/http#FileSystem) type), and generates a Go code file that statically implements the contents of the input filesystem. 30 | 31 | For example, we can use [`http.Dir`](https://godoc.org/net/http#Dir) as a `http.FileSystem` implementation that uses the contents of the `/path/to/assets` directory: 32 | 33 | ```Go 34 | var fs http.FileSystem = http.Dir("/path/to/assets") 35 | ``` 36 | 37 | Now, when you execute the following code: 38 | 39 | ```Go 40 | err := vfsgen.Generate(fs, vfsgen.Options{}) 41 | if err != nil { 42 | log.Fatalln(err) 43 | } 44 | ``` 45 | 46 | An assets_vfsdata.go file will be generated in the current directory: 47 | 48 | ```Go 49 | // Code generated by vfsgen; DO NOT EDIT. 50 | 51 | package main 52 | 53 | import ... 54 | 55 | // assets statically implements the virtual filesystem provided to vfsgen.Generate. 56 | var assets http.FileSystem = ... 57 | ``` 58 | 59 | Then, in your program, you can use `assets` as any other [`http.FileSystem`](https://godoc.org/net/http#FileSystem), for example: 60 | 61 | ```Go 62 | file, err := assets.Open("/some/file.txt") 63 | if err != nil { 64 | return err 65 | } 66 | defer file.Close() 67 | ``` 68 | 69 | ```Go 70 | http.Handle("/assets/", http.FileServer(assets)) 71 | ``` 72 | 73 | `vfsgen` can be more useful when combined with build tags and go generate directives. This is described below. 74 | 75 | ### `go generate` Usage 76 | 77 | vfsgen is great to use with go generate directives. The code invoking `vfsgen.Generate` can go in an assets_generate.go file, which can then be invoked via "//go:generate go run assets_generate.go". The input virtual filesystem can read directly from disk, or it can be more involved. 78 | 79 | By using build tags, you can create a development mode where assets are loaded directly from disk via `http.Dir`, but then statically implemented for final releases. 80 | 81 | For example, suppose your source filesystem is defined in a package with import path "example.com/project/data" as: 82 | 83 | ```Go 84 | //go:build dev 85 | 86 | package data 87 | 88 | import "net/http" 89 | 90 | // Assets contains project assets. 91 | var Assets http.FileSystem = http.Dir("assets") 92 | ``` 93 | 94 | When built with the "dev" build tag, accessing `data.Assets` will read from disk directly via `http.Dir`. 95 | 96 | A generate helper file assets_generate.go can be invoked via "//go:generate go run -tags=dev assets_generate.go" directive: 97 | 98 | ```Go 99 | //go:build ignore 100 | 101 | package main 102 | 103 | import ( 104 | "log" 105 | 106 | "example.com/project/data" 107 | "github.com/shurcooL/vfsgen" 108 | ) 109 | 110 | func main() { 111 | err := vfsgen.Generate(data.Assets, vfsgen.Options{ 112 | PackageName: "data", 113 | BuildTags: "!dev", 114 | VariableName: "Assets", 115 | }) 116 | if err != nil { 117 | log.Fatalln(err) 118 | } 119 | } 120 | ``` 121 | 122 | Note that "dev" build tag is used to access the source filesystem, and the output file will contain "!dev" build tag. That way, the statically implemented version will be used during normal builds and `go get`, when custom builds tags are not specified. 123 | 124 | ### `vfsgendev` Usage 125 | 126 | `vfsgendev` is a binary that can be used to replace the need for the assets_generate.go file. 127 | 128 | Make sure it's installed and available in your PATH. 129 | 130 | ```bash 131 | go get -u github.com/shurcooL/vfsgen/cmd/vfsgendev 132 | ``` 133 | 134 | Then the "//go:generate go run -tags=dev assets_generate.go" directive can be replaced with: 135 | 136 | ``` 137 | //go:generate vfsgendev -source="example.com/project/data".Assets 138 | ``` 139 | 140 | vfsgendev accesses the source variable using "dev" build tag, and generates an output file with "!dev" build tag. 141 | 142 | ### Additional Embedded Information 143 | 144 | All compressed files implement [`httpgzip.GzipByter` interface](https://godoc.org/github.com/shurcooL/httpgzip#GzipByter) for efficient direct access to the internal compressed bytes: 145 | 146 | ```Go 147 | // GzipByter is implemented by compressed files for 148 | // efficient direct access to the internal compressed bytes. 149 | type GzipByter interface { 150 | // GzipBytes returns gzip compressed contents of the file. 151 | GzipBytes() []byte 152 | } 153 | ``` 154 | 155 | Files that have been determined to not be worth gzip compressing (their compressed size is larger than original) implement [`httpgzip.NotWorthGzipCompressing` interface](https://godoc.org/github.com/shurcooL/httpgzip#NotWorthGzipCompressing): 156 | 157 | ```Go 158 | // NotWorthGzipCompressing is implemented by files that were determined 159 | // not to be worth gzip compressing (the file size did not decrease as a result). 160 | type NotWorthGzipCompressing interface { 161 | // NotWorthGzipCompressing is a noop. It's implemented in order to indicate 162 | // the file is not worth gzip compressing. 163 | NotWorthGzipCompressing() 164 | } 165 | ``` 166 | 167 | Comparison 168 | ---------- 169 | 170 | vfsgen aims to be conceptually simple to use. The [`http.FileSystem`](https://godoc.org/net/http#FileSystem) abstraction is central to vfsgen. It's used as both input for code generation, and as output in the generated code. 171 | 172 | That enables great flexibility through orthogonality, since helpers and wrappers can operate on `http.FileSystem` without knowing about vfsgen. If you want, you can perform pre-processing, minifying assets, merging folders, filtering out files and otherwise modifying input via generic `http.FileSystem` middleware. 173 | 174 | It avoids unneccessary overhead by merging what was previously done with two distinct packages into a single package. 175 | 176 | It strives to be the best in its class in terms of code quality and efficiency of generated code. However, if your use goals are different, there are other similar packages that may fit your needs better. 177 | 178 | ### Alternatives 179 | 180 | - [`embed`](https://go.dev/pkg/embed) - Package embed provides access to files embedded in the running Go program. 181 | - [`go-bindata`](https://github.com/jteeuwen/go-bindata) - Reads from disk, generates Go code that provides access to data via a [custom API](https://github.com/jteeuwen/go-bindata#accessing-an-asset). 182 | - [`go-bindata-assetfs`](https://github.com/elazarl/go-bindata-assetfs) - Takes output of go-bindata and provides a wrapper that implements `http.FileSystem` interface (the same as what vfsgen outputs directly). 183 | - [`becky`](https://github.com/tv42/becky) - Embeds assets as string literals in Go source. 184 | - [`statik`](https://github.com/rakyll/statik) - Embeds a directory of static files to be accessed via `http.FileSystem` interface (sounds very similar to vfsgen); implementation sourced from [camlistore](https://camlistore.org). 185 | - [`go.rice`](https://github.com/GeertJohan/go.rice) - Makes working with resources such as HTML, JS, CSS, images and templates very easy. 186 | - [`esc`](https://github.com/mjibson/esc) - Embeds files into Go programs and provides `http.FileSystem` interfaces to them. 187 | - [`staticfiles`](https://github.com/bouk/staticfiles) - Allows you to embed a directory of files into your Go binary. 188 | - [`togo`](https://github.com/flazz/togo) - Generates a Go source file with a `[]byte` var containing the given file's contents. 189 | - [`fileb0x`](https://github.com/UnnoTed/fileb0x) - Simple customizable tool to embed files in Go. 190 | - [`embedfiles`](https://github.com/leighmcculloch/embedfiles) - Simple tool for embedding files in Go code as a map. 191 | - [`packr`](https://github.com/gobuffalo/packr) - Simple solution for bundling static assets inside of Go binaries. 192 | - [`rsrc`](https://github.com/akavel/rsrc) - Tool for embedding .ico & manifest resources in Go programs for Windows. 193 | 194 | Attribution 195 | ----------- 196 | 197 | This package was originally based on the excellent work by [@jteeuwen](https://github.com/jteeuwen) on [`go-bindata`](https://github.com/jteeuwen/go-bindata) and [@elazarl](https://github.com/elazarl) on [`go-bindata-assetfs`](https://github.com/elazarl/go-bindata-assetfs). 198 | 199 | Directories 200 | ----------- 201 | 202 | | Path | Synopsis | 203 | |------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| 204 | | [cmd/vfsgendev](https://pkg.go.dev/github.com/shurcooL/vfsgen/cmd/vfsgendev) | vfsgendev is a convenience tool for using vfsgen in a common development configuration. | 205 | 206 | License 207 | ------- 208 | 209 | - [MIT License](LICENSE) 210 | -------------------------------------------------------------------------------- /cmd/vfsgendev/generate.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "strconv" 5 | "text/template" 6 | ) 7 | 8 | type data struct { 9 | ImportPath string 10 | PackageName string 11 | BuildTags string 12 | VariableName string 13 | VariableComment string 14 | } 15 | 16 | var generateTemplate = template.Must(template.New("").Funcs(template.FuncMap{ 17 | "quote": strconv.Quote, 18 | }).Parse(`package main 19 | 20 | import ( 21 | "log" 22 | 23 | "github.com/shurcooL/vfsgen" 24 | 25 | sourcepkg {{.ImportPath | quote}} 26 | ) 27 | 28 | func main() { 29 | err := vfsgen.Generate(sourcepkg.{{.VariableName}}, vfsgen.Options{ 30 | PackageName: {{.PackageName | quote}}, 31 | BuildTags: {{.BuildTags | quote}}, 32 | VariableName: {{.VariableName | quote}}, 33 | VariableComment: {{.VariableComment | quote}}, 34 | }) 35 | if err != nil { 36 | log.Fatalln(err) 37 | } 38 | } 39 | `)) 40 | -------------------------------------------------------------------------------- /cmd/vfsgendev/main.go: -------------------------------------------------------------------------------- 1 | // vfsgendev is a convenience tool for using vfsgen in a common development configuration. 2 | package main 3 | 4 | import ( 5 | "bytes" 6 | "flag" 7 | "fmt" 8 | "go/build" 9 | "io" 10 | "log" 11 | "os" 12 | "os/exec" 13 | "path/filepath" 14 | ) 15 | 16 | var ( 17 | sourceFlag = flag.String("source", "", "Specifies the http.FileSystem variable to use as source.") 18 | tagFlag = flag.String("tag", "dev", "Specifies a single build tag to use for source. The output will include a negated version.") 19 | nFlag = flag.Bool("n", false, "Print the generated source but do not run it.") 20 | ) 21 | 22 | func usage() { 23 | fmt.Fprintln(os.Stderr, `Usage: vfsgendev [flags] -source="import/path".VariableName`) 24 | flag.PrintDefaults() 25 | } 26 | 27 | func main() { 28 | flag.Usage = usage 29 | flag.Parse() 30 | if flag.NArg() != 0 { 31 | flag.Usage() 32 | os.Exit(2) 33 | } 34 | importPath, variableName, err := parseSourceFlag(*sourceFlag) 35 | if err != nil { 36 | fmt.Fprintln(os.Stderr, "-source flag has invalid value:", err) 37 | fmt.Fprintln(os.Stderr) 38 | flag.Usage() 39 | os.Exit(2) 40 | } 41 | tag, err := parseTagFlag(*tagFlag) 42 | if err != nil { 43 | fmt.Fprintln(os.Stderr, "-tag flag has invalid value:", err) 44 | fmt.Fprintln(os.Stderr) 45 | flag.Usage() 46 | os.Exit(2) 47 | } 48 | 49 | err = run(importPath, variableName, tag) 50 | if err != nil { 51 | log.Fatalln(err) 52 | } 53 | } 54 | 55 | func run(importPath, variableName, tag string) error { 56 | bctx := build.Default 57 | bctx.BuildTags = []string{tag} 58 | packageName, variableComment, err := lookupNameAndComment(bctx, importPath, variableName) 59 | if err != nil { 60 | return err 61 | } 62 | 63 | var buf bytes.Buffer 64 | err = generateTemplate.Execute(&buf, data{ 65 | ImportPath: importPath, 66 | PackageName: packageName, 67 | BuildTags: "!" + tag, 68 | VariableName: variableName, 69 | VariableComment: variableComment, 70 | }) 71 | if err != nil { 72 | return err 73 | } 74 | 75 | if *nFlag { 76 | io.Copy(os.Stdout, &buf) 77 | return nil 78 | } 79 | 80 | err = goRun(buf.String(), tag) 81 | return err 82 | } 83 | 84 | // goRun runs Go code src with build tags. 85 | func goRun(src string, tags string) error { 86 | // Create a temp folder. 87 | tempDir, err := os.MkdirTemp("", "vfsgendev_") 88 | if err != nil { 89 | return err 90 | } 91 | defer func() { 92 | err := os.RemoveAll(tempDir) 93 | if err != nil { 94 | fmt.Fprintln(os.Stderr, "warning: error removing temp dir:", err) 95 | } 96 | }() 97 | 98 | // Write the source code file. 99 | tempFile := filepath.Join(tempDir, "generate.go") 100 | err = os.WriteFile(tempFile, []byte(src), 0600) 101 | if err != nil { 102 | return err 103 | } 104 | 105 | // Compile and run the program. 106 | cmd := exec.Command("go", "run", "-tags="+tags, tempFile) 107 | cmd.Stdout = os.Stdout 108 | cmd.Stderr = os.Stderr 109 | return cmd.Run() 110 | } 111 | -------------------------------------------------------------------------------- /cmd/vfsgendev/parse.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "go/ast" 7 | "go/build" 8 | "go/doc" 9 | "go/parser" 10 | "go/printer" 11 | "go/token" 12 | "os" 13 | "path/filepath" 14 | "strconv" 15 | "strings" 16 | ) 17 | 18 | // parseSourceFlag parses the "-source" flag value. It must have "import/path".VariableName format. 19 | // It returns an error if the parsed import path is relative. 20 | func parseSourceFlag(sourceFlag string) (importPath, variableName string, err error) { 21 | // Parse sourceFlag as a Go expression, albeit a strange one: 22 | // 23 | // "import/path".VariableName 24 | // 25 | e, err := parser.ParseExpr(sourceFlag) 26 | if err != nil { 27 | return "", "", fmt.Errorf("invalid format, failed to parse %q as a Go expression", sourceFlag) 28 | } 29 | se, ok := e.(*ast.SelectorExpr) 30 | if !ok { 31 | return "", "", fmt.Errorf("invalid format, expression %v is not a selector expression but %T", sourceFlag, e) 32 | } 33 | importPath, err = stringValue(se.X) 34 | if err != nil { 35 | return "", "", fmt.Errorf("invalid format, expression %v is not a properly quoted Go string: %v", stringifyAST(se.X), err) 36 | } 37 | if build.IsLocalImport(importPath) { 38 | // Generated code is executed in a temporary directory, 39 | // and can't use relative import paths. So disallow them. 40 | return "", "", fmt.Errorf("relative import paths are not supported") 41 | } 42 | variableName = se.Sel.Name 43 | return importPath, variableName, nil 44 | } 45 | 46 | // stringValue returns the string value of string literal e. 47 | func stringValue(e ast.Expr) (string, error) { 48 | lit, ok := e.(*ast.BasicLit) 49 | if !ok { 50 | return "", fmt.Errorf("not a string, but %T", e) 51 | } 52 | if lit.Kind != token.STRING { 53 | return "", fmt.Errorf("not a string, but %v", lit.Kind) 54 | } 55 | return strconv.Unquote(lit.Value) 56 | } 57 | 58 | // parseTagFlag parses the "-tag" flag value. It must be a single build tag. 59 | func parseTagFlag(tagFlag string) (tag string, err error) { 60 | tags := strings.Fields(tagFlag) 61 | if len(tags) != 1 { 62 | return "", fmt.Errorf("%q is not a valid single build tag, but %q", tagFlag, tags) 63 | } 64 | return tags[0], nil 65 | } 66 | 67 | // lookupNameAndComment imports package using provided build context, and 68 | // returns the package name and variable comment. 69 | func lookupNameAndComment(bctx build.Context, importPath, variableName string) (packageName, variableComment string, err error) { 70 | wd, err := os.Getwd() 71 | if err != nil { 72 | return "", "", err 73 | } 74 | bpkg, err := bctx.Import(importPath, wd, 0) 75 | if err != nil { 76 | return "", "", fmt.Errorf("can't import package %q: %v", importPath, err) 77 | } 78 | dpkg, err := computeDoc(bpkg) 79 | if err != nil { 80 | return "", "", fmt.Errorf("can't get godoc of package %q: %v", importPath, err) 81 | } 82 | for _, v := range dpkg.Vars { 83 | if len(v.Names) == 1 && v.Names[0] == variableName { 84 | variableComment = strings.TrimSuffix(v.Doc, "\n") 85 | break 86 | } 87 | } 88 | return bpkg.Name, variableComment, nil 89 | } 90 | 91 | func stringifyAST(node interface{}) string { 92 | var buf bytes.Buffer 93 | err := printer.Fprint(&buf, token.NewFileSet(), node) 94 | if err != nil { 95 | return "printer.Fprint error: " + err.Error() 96 | } 97 | return buf.String() 98 | } 99 | 100 | // computeDoc computes the package documentation for the given package. 101 | func computeDoc(bpkg *build.Package) (*doc.Package, error) { 102 | fset := token.NewFileSet() 103 | files := make(map[string]*ast.File) 104 | for _, file := range append(bpkg.GoFiles, bpkg.CgoFiles...) { 105 | f, err := parser.ParseFile(fset, filepath.Join(bpkg.Dir, file), nil, parser.ParseComments) 106 | if err != nil { 107 | return nil, err 108 | } 109 | files[file] = f 110 | } 111 | apkg := &ast.Package{ 112 | Name: bpkg.Name, 113 | Files: files, 114 | } 115 | return doc.New(apkg, bpkg.ImportPath, 0), nil 116 | } 117 | -------------------------------------------------------------------------------- /commentwriter.go: -------------------------------------------------------------------------------- 1 | package vfsgen 2 | 3 | import "io" 4 | 5 | // commentWriter writes a Go comment to the underlying io.Writer, 6 | // using line comment form (//). 7 | type commentWriter struct { 8 | W io.Writer 9 | wroteSlashes bool // Wrote "//" at the beginning of the current line. 10 | } 11 | 12 | func (c *commentWriter) Write(p []byte) (int, error) { 13 | var n int 14 | for i, b := range p { 15 | if !c.wroteSlashes { 16 | s := "//" 17 | if b != '\n' { 18 | s = "// " 19 | } 20 | if _, err := io.WriteString(c.W, s); err != nil { 21 | return n, err 22 | } 23 | c.wroteSlashes = true 24 | } 25 | n0, err := c.W.Write(p[i : i+1]) 26 | n += n0 27 | if err != nil { 28 | return n, err 29 | } 30 | if b == '\n' { 31 | c.wroteSlashes = false 32 | } 33 | } 34 | return len(p), nil 35 | } 36 | 37 | func (c *commentWriter) Close() error { 38 | if !c.wroteSlashes { 39 | if _, err := io.WriteString(c.W, "//"); err != nil { 40 | return err 41 | } 42 | c.wroteSlashes = true 43 | } 44 | return nil 45 | } 46 | -------------------------------------------------------------------------------- /doc.go: -------------------------------------------------------------------------------- 1 | /* 2 | Package vfsgen takes an http.FileSystem (likely at `go generate` time) and 3 | generates Go code that statically implements the provided http.FileSystem. 4 | 5 | Features: 6 | 7 | - Efficient generated code without unneccessary overhead. 8 | 9 | - Uses gzip compression internally (selectively, only for files that compress well). 10 | 11 | - Enables direct access to internal gzip compressed bytes via an optional interface. 12 | 13 | - Outputs `gofmt`ed Go code. 14 | */ 15 | package vfsgen 16 | -------------------------------------------------------------------------------- /generator.go: -------------------------------------------------------------------------------- 1 | package vfsgen 2 | 3 | import ( 4 | "bytes" 5 | "compress/gzip" 6 | "errors" 7 | "io" 8 | "net/http" 9 | "os" 10 | pathpkg "path" 11 | "sort" 12 | "strconv" 13 | "text/template" 14 | "time" 15 | 16 | "github.com/shurcooL/httpfs/vfsutil" 17 | ) 18 | 19 | // Generate Go code that statically implements input filesystem, 20 | // write the output to a file specified in opt. 21 | func Generate(input http.FileSystem, opt Options) error { 22 | opt.fillMissing() 23 | 24 | // Use an in-memory buffer to generate the entire output. 25 | buf := new(bytes.Buffer) 26 | 27 | err := t.ExecuteTemplate(buf, "Header", opt) 28 | if err != nil { 29 | return err 30 | } 31 | 32 | var toc toc 33 | err = findAndWriteFiles(buf, input, &toc) 34 | if err != nil { 35 | return err 36 | } 37 | 38 | err = t.ExecuteTemplate(buf, "DirEntries", toc.dirs) 39 | if err != nil { 40 | return err 41 | } 42 | 43 | err = t.ExecuteTemplate(buf, "Trailer", toc) 44 | if err != nil { 45 | return err 46 | } 47 | 48 | // Write output file (all at once). 49 | err = os.WriteFile(opt.Filename, buf.Bytes(), 0644) 50 | return err 51 | } 52 | 53 | type toc struct { 54 | dirs []*dirInfo 55 | 56 | HasCompressedFile bool // There's at least one compressedFile. 57 | HasFile bool // There's at least one uncompressed file. 58 | } 59 | 60 | // fileInfo is a definition of a file. 61 | type fileInfo struct { 62 | Path string 63 | Name string 64 | ModTime time.Time 65 | UncompressedSize int64 66 | } 67 | 68 | // dirInfo is a definition of a directory. 69 | type dirInfo struct { 70 | Path string 71 | Name string 72 | ModTime time.Time 73 | Entries []string 74 | } 75 | 76 | // findAndWriteFiles recursively finds all the file paths in the given directory tree. 77 | // They are added to the given map as keys. Values will be safe function names 78 | // for each file, which will be used when generating the output code. 79 | func findAndWriteFiles(buf *bytes.Buffer, fs http.FileSystem, toc *toc) error { 80 | walkFn := func(path string, fi os.FileInfo, r io.ReadSeeker, err error) error { 81 | if err != nil { 82 | // Consider all errors reading the input filesystem as fatal. 83 | return err 84 | } 85 | 86 | switch fi.IsDir() { 87 | case false: 88 | file := &fileInfo{ 89 | Path: path, 90 | Name: pathpkg.Base(path), 91 | ModTime: fi.ModTime().UTC(), 92 | UncompressedSize: fi.Size(), 93 | } 94 | 95 | marker := buf.Len() 96 | 97 | // Write CompressedFileInfo. 98 | err = writeCompressedFileInfo(buf, file, r) 99 | switch err { 100 | default: 101 | return err 102 | case nil: 103 | toc.HasCompressedFile = true 104 | // If compressed file is not smaller than original, revert and write original file. 105 | case errCompressedNotSmaller: 106 | _, err = r.Seek(0, io.SeekStart) 107 | if err != nil { 108 | return err 109 | } 110 | 111 | buf.Truncate(marker) 112 | 113 | // Write FileInfo. 114 | err = writeFileInfo(buf, file, r) 115 | if err != nil { 116 | return err 117 | } 118 | toc.HasFile = true 119 | } 120 | case true: 121 | entries, err := readDirPaths(fs, path) 122 | if err != nil { 123 | return err 124 | } 125 | 126 | dir := &dirInfo{ 127 | Path: path, 128 | Name: pathpkg.Base(path), 129 | ModTime: fi.ModTime().UTC(), 130 | Entries: entries, 131 | } 132 | 133 | toc.dirs = append(toc.dirs, dir) 134 | 135 | // Write DirInfo. 136 | err = t.ExecuteTemplate(buf, "DirInfo", dir) 137 | if err != nil { 138 | return err 139 | } 140 | } 141 | 142 | return nil 143 | } 144 | 145 | err := vfsutil.WalkFiles(fs, "/", walkFn) 146 | return err 147 | } 148 | 149 | // readDirPaths reads the directory named by dirname and returns 150 | // a sorted list of directory paths. 151 | func readDirPaths(fs http.FileSystem, dirname string) ([]string, error) { 152 | fis, err := vfsutil.ReadDir(fs, dirname) 153 | if err != nil { 154 | return nil, err 155 | } 156 | paths := make([]string, len(fis)) 157 | for i := range fis { 158 | paths[i] = pathpkg.Join(dirname, fis[i].Name()) 159 | } 160 | sort.Strings(paths) 161 | return paths, nil 162 | } 163 | 164 | // writeCompressedFileInfo writes CompressedFileInfo. 165 | // It returns errCompressedNotSmaller if compressed file is not smaller than original. 166 | func writeCompressedFileInfo(w io.Writer, file *fileInfo, r io.Reader) error { 167 | err := t.ExecuteTemplate(w, "CompressedFileInfo-Before", file) 168 | if err != nil { 169 | return err 170 | } 171 | sw := &stringWriter{Writer: w} 172 | gw, _ := gzip.NewWriterLevel(sw, gzip.BestCompression) 173 | _, err = io.Copy(gw, r) 174 | if err != nil { 175 | return err 176 | } 177 | err = gw.Close() 178 | if err != nil { 179 | return err 180 | } 181 | if sw.N >= file.UncompressedSize { 182 | return errCompressedNotSmaller 183 | } 184 | err = t.ExecuteTemplate(w, "CompressedFileInfo-After", file) 185 | return err 186 | } 187 | 188 | var errCompressedNotSmaller = errors.New("compressed file is not smaller than original") 189 | 190 | // Write FileInfo. 191 | func writeFileInfo(w io.Writer, file *fileInfo, r io.Reader) error { 192 | err := t.ExecuteTemplate(w, "FileInfo-Before", file) 193 | if err != nil { 194 | return err 195 | } 196 | sw := &stringWriter{Writer: w} 197 | _, err = io.Copy(sw, r) 198 | if err != nil { 199 | return err 200 | } 201 | err = t.ExecuteTemplate(w, "FileInfo-After", file) 202 | return err 203 | } 204 | 205 | var t = template.Must(template.New("").Funcs(template.FuncMap{ 206 | "quote": strconv.Quote, 207 | "comment": func(s string) (string, error) { 208 | var buf bytes.Buffer 209 | cw := &commentWriter{W: &buf} 210 | _, err := io.WriteString(cw, s) 211 | if err != nil { 212 | return "", err 213 | } 214 | err = cw.Close() 215 | return buf.String(), err 216 | }, 217 | }).Parse(`{{define "Header"}}// Code generated by vfsgen; DO NOT EDIT. 218 | 219 | {{with .BuildTags}}//go:build {{.}} 220 | 221 | {{end}}package {{.PackageName}} 222 | 223 | import ( 224 | "bytes" 225 | "compress/gzip" 226 | "fmt" 227 | "io" 228 | "net/http" 229 | "os" 230 | pathpkg "path" 231 | "time" 232 | ) 233 | 234 | {{comment .VariableComment}} 235 | var {{.VariableName}} = func() http.FileSystem { 236 | fs := vfsgen۰FS{ 237 | {{end}} 238 | 239 | 240 | 241 | {{define "CompressedFileInfo-Before"}} {{quote .Path}}: &vfsgen۰CompressedFileInfo{ 242 | name: {{quote .Name}}, 243 | modTime: {{template "Time" .ModTime}}, 244 | uncompressedSize: {{.UncompressedSize}}, 245 | {{/* This blank line separating compressedContent is neccessary to prevent potential gofmt issues. See issue #19. */}} 246 | compressedContent: []byte("{{end}}{{define "CompressedFileInfo-After"}}"), 247 | }, 248 | {{end}} 249 | 250 | 251 | 252 | {{define "FileInfo-Before"}} {{quote .Path}}: &vfsgen۰FileInfo{ 253 | name: {{quote .Name}}, 254 | modTime: {{template "Time" .ModTime}}, 255 | content: []byte("{{end}}{{define "FileInfo-After"}}"), 256 | }, 257 | {{end}} 258 | 259 | 260 | 261 | {{define "DirInfo"}} {{quote .Path}}: &vfsgen۰DirInfo{ 262 | name: {{quote .Name}}, 263 | modTime: {{template "Time" .ModTime}}, 264 | }, 265 | {{end}} 266 | 267 | 268 | 269 | {{define "DirEntries"}} } 270 | {{range .}}{{if .Entries}} fs[{{quote .Path}}].(*vfsgen۰DirInfo).entries = []os.FileInfo{{"{"}}{{range .Entries}} 271 | fs[{{quote .}}].(os.FileInfo),{{end}} 272 | } 273 | {{end}}{{end}} 274 | return fs 275 | }() 276 | {{end}} 277 | 278 | 279 | 280 | {{define "Trailer"}} 281 | type vfsgen۰FS map[string]interface{} 282 | 283 | func (fs vfsgen۰FS) Open(path string) (http.File, error) { 284 | path = pathpkg.Clean("/" + path) 285 | f, ok := fs[path] 286 | if !ok { 287 | return nil, &os.PathError{Op: "open", Path: path, Err: os.ErrNotExist} 288 | } 289 | 290 | switch f := f.(type) {{"{"}}{{if .HasCompressedFile}} 291 | case *vfsgen۰CompressedFileInfo: 292 | gr, err := gzip.NewReader(bytes.NewReader(f.compressedContent)) 293 | if err != nil { 294 | // This should never happen because we generate the gzip bytes such that they are always valid. 295 | panic("unexpected error reading own gzip compressed bytes: " + err.Error()) 296 | } 297 | return &vfsgen۰CompressedFile{ 298 | vfsgen۰CompressedFileInfo: f, 299 | gr: gr, 300 | }, nil{{end}}{{if .HasFile}} 301 | case *vfsgen۰FileInfo: 302 | return &vfsgen۰File{ 303 | vfsgen۰FileInfo: f, 304 | Reader: bytes.NewReader(f.content), 305 | }, nil{{end}} 306 | case *vfsgen۰DirInfo: 307 | return &vfsgen۰Dir{ 308 | vfsgen۰DirInfo: f, 309 | }, nil 310 | default: 311 | // This should never happen because we generate only the above types. 312 | panic(fmt.Sprintf("unexpected type %T", f)) 313 | } 314 | } 315 | {{if .HasCompressedFile}} 316 | // vfsgen۰CompressedFileInfo is a static definition of a gzip compressed file. 317 | type vfsgen۰CompressedFileInfo struct { 318 | name string 319 | modTime time.Time 320 | compressedContent []byte 321 | uncompressedSize int64 322 | } 323 | 324 | func (f *vfsgen۰CompressedFileInfo) Readdir(count int) ([]os.FileInfo, error) { 325 | return nil, fmt.Errorf("cannot Readdir from file %s", f.name) 326 | } 327 | func (f *vfsgen۰CompressedFileInfo) Stat() (os.FileInfo, error) { return f, nil } 328 | 329 | func (f *vfsgen۰CompressedFileInfo) GzipBytes() []byte { 330 | return f.compressedContent 331 | } 332 | 333 | func (f *vfsgen۰CompressedFileInfo) Name() string { return f.name } 334 | func (f *vfsgen۰CompressedFileInfo) Size() int64 { return f.uncompressedSize } 335 | func (f *vfsgen۰CompressedFileInfo) Mode() os.FileMode { return 0444 } 336 | func (f *vfsgen۰CompressedFileInfo) ModTime() time.Time { return f.modTime } 337 | func (f *vfsgen۰CompressedFileInfo) IsDir() bool { return false } 338 | func (f *vfsgen۰CompressedFileInfo) Sys() interface{} { return nil } 339 | 340 | // vfsgen۰CompressedFile is an opened compressedFile instance. 341 | type vfsgen۰CompressedFile struct { 342 | *vfsgen۰CompressedFileInfo 343 | gr *gzip.Reader 344 | grPos int64 // Actual gr uncompressed position. 345 | seekPos int64 // Seek uncompressed position. 346 | } 347 | 348 | func (f *vfsgen۰CompressedFile) Read(p []byte) (n int, err error) { 349 | if f.grPos > f.seekPos { 350 | // Rewind to beginning. 351 | err = f.gr.Reset(bytes.NewReader(f.compressedContent)) 352 | if err != nil { 353 | return 0, err 354 | } 355 | f.grPos = 0 356 | } 357 | if f.grPos < f.seekPos { 358 | // Fast-forward. 359 | _, err = io.CopyN(io.Discard, f.gr, f.seekPos-f.grPos) 360 | if err != nil { 361 | return 0, err 362 | } 363 | f.grPos = f.seekPos 364 | } 365 | n, err = f.gr.Read(p) 366 | f.grPos += int64(n) 367 | f.seekPos = f.grPos 368 | return n, err 369 | } 370 | func (f *vfsgen۰CompressedFile) Seek(offset int64, whence int) (int64, error) { 371 | switch whence { 372 | case io.SeekStart: 373 | f.seekPos = 0 + offset 374 | case io.SeekCurrent: 375 | f.seekPos += offset 376 | case io.SeekEnd: 377 | f.seekPos = f.uncompressedSize + offset 378 | default: 379 | panic(fmt.Errorf("invalid whence value: %v", whence)) 380 | } 381 | return f.seekPos, nil 382 | } 383 | func (f *vfsgen۰CompressedFile) Close() error { 384 | return f.gr.Close() 385 | } 386 | {{else}} 387 | // We already imported "compress/gzip" but ended up not using it. Avoid unused import error. 388 | var _ *gzip.Reader 389 | {{end}}{{if .HasFile}} 390 | // vfsgen۰FileInfo is a static definition of an uncompressed file (because it's not worth gzip compressing). 391 | type vfsgen۰FileInfo struct { 392 | name string 393 | modTime time.Time 394 | content []byte 395 | } 396 | 397 | func (f *vfsgen۰FileInfo) Readdir(count int) ([]os.FileInfo, error) { 398 | return nil, fmt.Errorf("cannot Readdir from file %s", f.name) 399 | } 400 | func (f *vfsgen۰FileInfo) Stat() (os.FileInfo, error) { return f, nil } 401 | 402 | func (f *vfsgen۰FileInfo) NotWorthGzipCompressing() {} 403 | 404 | func (f *vfsgen۰FileInfo) Name() string { return f.name } 405 | func (f *vfsgen۰FileInfo) Size() int64 { return int64(len(f.content)) } 406 | func (f *vfsgen۰FileInfo) Mode() os.FileMode { return 0444 } 407 | func (f *vfsgen۰FileInfo) ModTime() time.Time { return f.modTime } 408 | func (f *vfsgen۰FileInfo) IsDir() bool { return false } 409 | func (f *vfsgen۰FileInfo) Sys() interface{} { return nil } 410 | 411 | // vfsgen۰File is an opened file instance. 412 | type vfsgen۰File struct { 413 | *vfsgen۰FileInfo 414 | *bytes.Reader 415 | } 416 | 417 | func (f *vfsgen۰File) Close() error { 418 | return nil 419 | } 420 | {{else if not .HasCompressedFile}} 421 | // We already imported "bytes", but ended up not using it. Avoid unused import error. 422 | var _ = bytes.Reader{} 423 | {{end}} 424 | // vfsgen۰DirInfo is a static definition of a directory. 425 | type vfsgen۰DirInfo struct { 426 | name string 427 | modTime time.Time 428 | entries []os.FileInfo 429 | } 430 | 431 | func (d *vfsgen۰DirInfo) Read([]byte) (int, error) { 432 | return 0, fmt.Errorf("cannot Read from directory %s", d.name) 433 | } 434 | func (d *vfsgen۰DirInfo) Close() error { return nil } 435 | func (d *vfsgen۰DirInfo) Stat() (os.FileInfo, error) { return d, nil } 436 | 437 | func (d *vfsgen۰DirInfo) Name() string { return d.name } 438 | func (d *vfsgen۰DirInfo) Size() int64 { return 0 } 439 | func (d *vfsgen۰DirInfo) Mode() os.FileMode { return 0755 | os.ModeDir } 440 | func (d *vfsgen۰DirInfo) ModTime() time.Time { return d.modTime } 441 | func (d *vfsgen۰DirInfo) IsDir() bool { return true } 442 | func (d *vfsgen۰DirInfo) Sys() interface{} { return nil } 443 | 444 | // vfsgen۰Dir is an opened dir instance. 445 | type vfsgen۰Dir struct { 446 | *vfsgen۰DirInfo 447 | pos int // Position within entries for Seek and Readdir. 448 | } 449 | 450 | func (d *vfsgen۰Dir) Seek(offset int64, whence int) (int64, error) { 451 | if offset == 0 && whence == io.SeekStart { 452 | d.pos = 0 453 | return 0, nil 454 | } 455 | return 0, fmt.Errorf("unsupported Seek in directory %s", d.name) 456 | } 457 | 458 | func (d *vfsgen۰Dir) Readdir(count int) ([]os.FileInfo, error) { 459 | if d.pos >= len(d.entries) && count > 0 { 460 | return nil, io.EOF 461 | } 462 | if count <= 0 || count > len(d.entries)-d.pos { 463 | count = len(d.entries) - d.pos 464 | } 465 | e := d.entries[d.pos : d.pos+count] 466 | d.pos += count 467 | return e, nil 468 | } 469 | {{end}} 470 | 471 | 472 | 473 | {{define "Time"}} 474 | {{- if .IsZero -}} 475 | time.Time{} 476 | {{- else -}} 477 | time.Date({{.Year}}, {{printf "%d" .Month}}, {{.Day}}, {{.Hour}}, {{.Minute}}, {{.Second}}, {{.Nanosecond}}, time.UTC) 478 | {{- end -}} 479 | {{end}} 480 | `)) 481 | -------------------------------------------------------------------------------- /generator_test.go: -------------------------------------------------------------------------------- 1 | package vfsgen_test 2 | 3 | import ( 4 | "log" 5 | "net/http" 6 | "os" 7 | "os/exec" 8 | "path/filepath" 9 | "strings" 10 | "testing" 11 | 12 | "github.com/shurcooL/httpfs/union" 13 | "github.com/shurcooL/vfsgen" 14 | "golang.org/x/tools/godoc/vfs/httpfs" 15 | "golang.org/x/tools/godoc/vfs/mapfs" 16 | ) 17 | 18 | // This code will generate an assets_vfsdata.go file with 19 | // `var assets http.FileSystem = ...` 20 | // that statically implements the contents of "assets" directory. 21 | // 22 | // vfsgen is great to use with go generate directives. This code can go in an assets_gen.go file, which can 23 | // then be invoked via "//go:generate go run assets_gen.go". The input virtual filesystem can read directly 24 | // from disk, or it can be more involved. 25 | func Example() { 26 | var fs http.FileSystem = http.Dir("assets") 27 | 28 | err := vfsgen.Generate(fs, vfsgen.Options{}) 29 | if err != nil { 30 | log.Fatalln(err) 31 | } 32 | } 33 | 34 | // Verify that all possible combinations of {non-compressed,compressed} files build 35 | // successfully, and have no gofmt issues. 36 | func TestGenerate_buildAndGofmt(t *testing.T) { 37 | tempDir := t.TempDir() 38 | 39 | tests := []struct { 40 | filename string 41 | fs http.FileSystem 42 | wantError func(error) bool // Nil function means want nil error. 43 | }{ 44 | { 45 | // Empty. 46 | filename: "empty.go", 47 | fs: union.New(nil), 48 | }, 49 | { 50 | // Test that vfsgen.Generate returns an error when there is 51 | // an error reading from the input filesystem. 52 | filename: "notexist.go", 53 | fs: http.Dir("notexist"), 54 | wantError: os.IsNotExist, 55 | }, 56 | { 57 | // No compressed files. 58 | filename: "nocompressed.go", 59 | fs: httpfs.New(mapfs.New(map[string]string{ 60 | "not-compressable-file.txt": "Not compressable.", 61 | })), 62 | }, 63 | { 64 | // Only compressed files. 65 | filename: "onlycompressed.go", 66 | fs: httpfs.New(mapfs.New(map[string]string{ 67 | "compressable-file.txt": "This text compresses easily. " + strings.Repeat(" Go!", 128), 68 | })), 69 | }, 70 | { 71 | // Both non-compressed and compressed files. 72 | filename: "both.go", 73 | fs: httpfs.New(mapfs.New(map[string]string{ 74 | "not-compressable-file.txt": "Not compressable.", 75 | "compressable-file.txt": "This text compresses easily. " + strings.Repeat(" Go!", 128), 76 | })), 77 | }, 78 | } 79 | 80 | for _, test := range tests { 81 | filename := filepath.Join(tempDir, test.filename) 82 | 83 | err := vfsgen.Generate(test.fs, vfsgen.Options{ 84 | Filename: filename, 85 | PackageName: "test", 86 | }) 87 | switch { 88 | case test.wantError == nil && err != nil: 89 | t.Fatalf("%s: vfsgen.Generate returned non-nil error: %v", test.filename, err) 90 | case test.wantError != nil && !test.wantError(err): 91 | t.Fatalf("%s: vfsgen.Generate returned wrong error: %v", test.filename, err) 92 | } 93 | if test.wantError != nil { 94 | continue 95 | } 96 | 97 | if out, err := exec.Command("go", "build", filename).CombinedOutput(); err != nil { 98 | t.Errorf("err: %v\nout: %s", err, out) 99 | } 100 | if out, err := exec.Command("gofmt", "-d", "-s", filename).Output(); err != nil || len(out) != 0 { 101 | t.Errorf("gofmt issue\nerr: %v\nout: %s", err, out) 102 | } 103 | } 104 | } 105 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/shurcooL/vfsgen 2 | 3 | go 1.19 4 | -------------------------------------------------------------------------------- /options.go: -------------------------------------------------------------------------------- 1 | package vfsgen 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | ) 7 | 8 | // Options for vfsgen code generation. 9 | type Options struct { 10 | // Filename of the generated Go code output (including extension). 11 | // If left empty, it defaults to "{{toLower .VariableName}}_vfsdata.go". 12 | Filename string 13 | 14 | // PackageName is the name of the package in the generated code. 15 | // If left empty, it defaults to "main". 16 | PackageName string 17 | 18 | // BuildTags are the optional build tags in the generated code. 19 | // The build tags syntax is specified by the go tool. 20 | BuildTags string 21 | 22 | // VariableName is the name of the http.FileSystem variable in the generated code. 23 | // If left empty, it defaults to "assets". 24 | VariableName string 25 | 26 | // VariableComment is the comment of the http.FileSystem variable in the generated code. 27 | // If left empty, it defaults to "{{.VariableName}} statically implements the virtual filesystem provided to vfsgen.". 28 | VariableComment string 29 | } 30 | 31 | // fillMissing sets default values for mandatory options that are left empty. 32 | func (opt *Options) fillMissing() { 33 | if opt.PackageName == "" { 34 | opt.PackageName = "main" 35 | } 36 | if opt.VariableName == "" { 37 | opt.VariableName = "assets" 38 | } 39 | if opt.Filename == "" { 40 | opt.Filename = fmt.Sprintf("%s_vfsdata.go", strings.ToLower(opt.VariableName)) 41 | } 42 | if opt.VariableComment == "" { 43 | opt.VariableComment = fmt.Sprintf("%s statically implements the virtual filesystem provided to vfsgen.", opt.VariableName) 44 | } 45 | } 46 | -------------------------------------------------------------------------------- /stringwriter.go: -------------------------------------------------------------------------------- 1 | package vfsgen 2 | 3 | import ( 4 | "io" 5 | ) 6 | 7 | // stringWriter writes given bytes to underlying io.Writer as a Go interpreted string literal value, 8 | // not including double quotes. It tracks the total number of bytes written. 9 | type stringWriter struct { 10 | io.Writer 11 | N int64 // Total bytes written. 12 | } 13 | 14 | func (sw *stringWriter) Write(p []byte) (n int, err error) { 15 | const hex = "0123456789abcdef" 16 | buf := []byte{'\\', 'x', 0, 0} 17 | for _, b := range p { 18 | buf[2], buf[3] = hex[b/16], hex[b%16] 19 | _, err = sw.Writer.Write(buf) 20 | if err != nil { 21 | return n, err 22 | } 23 | n++ 24 | sw.N++ 25 | } 26 | return n, nil 27 | } 28 | -------------------------------------------------------------------------------- /test/doc.go: -------------------------------------------------------------------------------- 1 | // Package test contains tests for virtual filesystem implementation generated by vfsgen. 2 | package test 3 | -------------------------------------------------------------------------------- /test/test_gen.go: -------------------------------------------------------------------------------- 1 | //go:build generate 2 | 3 | package main 4 | 5 | import ( 6 | "log" 7 | "net/http" 8 | 9 | "github.com/shurcooL/vfsgen" 10 | "golang.org/x/tools/godoc/vfs/httpfs" 11 | "golang.org/x/tools/godoc/vfs/mapfs" 12 | ) 13 | 14 | func main() { 15 | var fs http.FileSystem = httpfs.New(mapfs.New(map[string]string{ 16 | "sample-file.txt": "This file compresses well. Blaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaah!", 17 | "not-worth-compressing-file.txt": "Its normal contents are here.", 18 | "folderA/file1.txt": "Stuff in /folderA/file1.txt.", 19 | "folderA/file2.txt": "Stuff in /folderA/file2.txt.", 20 | "folderB/folderC/file3.txt": "Stuff in /folderB/folderC/file3.txt.", 21 | // TODO: Empty folder somehow? 22 | //"folder-empty/": "", 23 | })) 24 | 25 | err := vfsgen.Generate(fs, vfsgen.Options{ 26 | Filename: "test_vfsdata_test.go", 27 | PackageName: "test_test", 28 | }) 29 | if err != nil { 30 | log.Fatalln(err) 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /test/test_test.go: -------------------------------------------------------------------------------- 1 | package test_test 2 | 3 | import ( 4 | "fmt" 5 | "io" 6 | "log" 7 | "net/http" 8 | "os" 9 | 10 | "github.com/shurcooL/httpfs/vfsutil" 11 | "github.com/shurcooL/httpgzip" 12 | ) 13 | 14 | //go:generate go run test_gen.go 15 | 16 | // Basic functionality test. 17 | func Example_basic() { 18 | var fs http.FileSystem = assets 19 | 20 | walkFn := func(path string, fi os.FileInfo, err error) error { 21 | if err != nil { 22 | log.Printf("can't stat file %s: %v\n", path, err) 23 | return nil 24 | } 25 | 26 | fmt.Println(path) 27 | if fi.IsDir() { 28 | return nil 29 | } 30 | 31 | b, err := vfsutil.ReadFile(fs, path) 32 | fmt.Printf("%q %v\n", string(b), err) 33 | return nil 34 | } 35 | 36 | err := vfsutil.Walk(fs, "/", walkFn) 37 | if err != nil { 38 | panic(err) 39 | } 40 | 41 | // Output: 42 | // / 43 | // /folderA 44 | // /folderA/file1.txt 45 | // "Stuff in /folderA/file1.txt." 46 | // /folderA/file2.txt 47 | // "Stuff in /folderA/file2.txt." 48 | // /folderB 49 | // /folderB/folderC 50 | // /folderB/folderC/file3.txt 51 | // "Stuff in /folderB/folderC/file3.txt." 52 | // /not-worth-compressing-file.txt 53 | // "Its normal contents are here." 54 | // /sample-file.txt 55 | // "This file compresses well. Blaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaah!" 56 | } 57 | 58 | func Example_compressed() { 59 | // Compressed file system. 60 | var fs http.FileSystem = assets 61 | 62 | walkFn := func(path string, fi os.FileInfo, err error) error { 63 | if err != nil { 64 | log.Printf("can't stat file %s: %v\n", path, err) 65 | return nil 66 | } 67 | 68 | fmt.Println(path) 69 | if fi.IsDir() { 70 | return nil 71 | } 72 | 73 | f, err := fs.Open(path) 74 | if err != nil { 75 | fmt.Printf("fs.Open(%q): %v\n", path, err) 76 | return nil 77 | } 78 | defer f.Close() 79 | 80 | b, err := io.ReadAll(f) 81 | fmt.Printf("%q %v\n", string(b), err) 82 | 83 | if gzipFile, ok := f.(httpgzip.GzipByter); ok { 84 | b := gzipFile.GzipBytes() 85 | fmt.Printf("%q\n", string(b)) 86 | } else { 87 | fmt.Println("") 88 | } 89 | return nil 90 | } 91 | 92 | err := vfsutil.Walk(fs, "/", walkFn) 93 | if err != nil { 94 | panic(err) 95 | } 96 | 97 | // Output: 98 | // / 99 | // /folderA 100 | // /folderA/file1.txt 101 | // "Stuff in /folderA/file1.txt." 102 | // 103 | // /folderA/file2.txt 104 | // "Stuff in /folderA/file2.txt." 105 | // 106 | // /folderB 107 | // /folderB/folderC 108 | // /folderB/folderC/file3.txt 109 | // "Stuff in /folderB/folderC/file3.txt." 110 | // 111 | // /not-worth-compressing-file.txt 112 | // "Its normal contents are here." 113 | // 114 | // /sample-file.txt 115 | // "This file compresses well. Blaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaah!" 116 | // "\x1f\x8b\b\x00\x00\x00\x00\x00\x02\xff\n\xc9\xc8,VH\xcb\xccIUH\xce\xcf-(J-.N-V(O\xcd\xc9\xd1Sp\xcaI\x1c\xd4 C\x11\x10\x00\x00\xff\xff\xe7G\x81:\xbd\x00\x00\x00" 117 | } 118 | 119 | func Example_readTwoOpenedCompressedFiles() { 120 | var fs http.FileSystem = assets 121 | 122 | f0, err := fs.Open("/sample-file.txt") 123 | if err != nil { 124 | panic(err) 125 | } 126 | defer f0.Close() 127 | _ = f0.(httpgzip.GzipByter) 128 | f1, err := fs.Open("/sample-file.txt") 129 | if err != nil { 130 | panic(err) 131 | } 132 | defer f1.Close() 133 | _ = f1.(httpgzip.GzipByter) 134 | 135 | _, err = io.CopyN(os.Stdout, f0, 9) 136 | if err != nil { 137 | panic(err) 138 | } 139 | _, err = io.CopyN(os.Stdout, f1, 9) 140 | if err != nil { 141 | panic(err) 142 | } 143 | 144 | // Output: 145 | // This fileThis file 146 | } 147 | 148 | func Example_readTwoOpenedUncompressedFiles() { 149 | var fs http.FileSystem = assets 150 | 151 | f0, err := fs.Open("/not-worth-compressing-file.txt") 152 | if err != nil { 153 | panic(err) 154 | } 155 | defer f0.Close() 156 | _ = f0.(httpgzip.NotWorthGzipCompressing) 157 | f1, err := fs.Open("/not-worth-compressing-file.txt") 158 | if err != nil { 159 | panic(err) 160 | } 161 | defer f1.Close() 162 | _ = f1.(httpgzip.NotWorthGzipCompressing) 163 | 164 | _, err = io.CopyN(os.Stdout, f0, 9) 165 | if err != nil { 166 | panic(err) 167 | } 168 | _, err = io.CopyN(os.Stdout, f1, 9) 169 | if err != nil { 170 | panic(err) 171 | } 172 | 173 | // Output: 174 | // Its normaIts norma 175 | } 176 | 177 | func Example_modTime() { 178 | var fs http.FileSystem = assets 179 | 180 | f, err := fs.Open("/sample-file.txt") 181 | if err != nil { 182 | panic(err) 183 | } 184 | defer f.Close() 185 | 186 | fi, err := f.Stat() 187 | if err != nil { 188 | panic(err) 189 | } 190 | 191 | fmt.Println(fi.ModTime()) 192 | 193 | // Output: 194 | // 0001-01-01 00:00:00 +0000 UTC 195 | } 196 | 197 | func Example_seek() { 198 | var fs http.FileSystem = assets 199 | 200 | f, err := fs.Open("/sample-file.txt") 201 | if err != nil { 202 | panic(err) 203 | } 204 | defer f.Close() 205 | 206 | _, err = io.CopyN(os.Stdout, f, 5) 207 | if err != nil { 208 | panic(err) 209 | } 210 | _, err = f.Seek(22, io.SeekCurrent) 211 | if err != nil { 212 | panic(err) 213 | } 214 | _, err = io.CopyN(os.Stdout, f, 10) 215 | if err != nil { 216 | panic(err) 217 | } 218 | fmt.Print("...") 219 | _, err = f.Seek(-4, io.SeekEnd) 220 | if err != nil { 221 | panic(err) 222 | } 223 | _, err = io.Copy(os.Stdout, f) 224 | if err != nil { 225 | panic(err) 226 | } 227 | _, err = f.Seek(3, io.SeekStart) 228 | if err != nil { 229 | panic(err) 230 | } 231 | _, err = f.Seek(1, io.SeekCurrent) 232 | if err != nil { 233 | panic(err) 234 | } 235 | _, err = io.CopyN(os.Stdout, f, 22) 236 | if err != nil { 237 | panic(err) 238 | } 239 | 240 | // Output: 241 | // This Blaaaaaaaa...aah! file compresses well. 242 | } 243 | 244 | type fisStringer []os.FileInfo 245 | 246 | func (fis fisStringer) String() string { 247 | var s = "[ " 248 | for _, fi := range fis { 249 | s += fi.Name() + " " 250 | } 251 | return s + "]" 252 | } 253 | 254 | func Example_seekDir1() { 255 | var fs http.FileSystem = assets 256 | 257 | f, err := fs.Open("/") 258 | if err != nil { 259 | panic(err) 260 | } 261 | defer f.Close() 262 | 263 | fis, err := f.Readdir(0) 264 | fmt.Println(fisStringer(fis), err) 265 | 266 | // Output: 267 | // [ folderA folderB not-worth-compressing-file.txt sample-file.txt ] 268 | } 269 | 270 | func Example_seekDir2() { 271 | var fs http.FileSystem = assets 272 | 273 | f, err := fs.Open("/") 274 | if err != nil { 275 | panic(err) 276 | } 277 | defer f.Close() 278 | 279 | fis, err := f.Readdir(2) 280 | fmt.Println(fisStringer(fis), err) 281 | fis, err = f.Readdir(1) 282 | fmt.Println(fisStringer(fis), err) 283 | _, err = f.Seek(0, io.SeekStart) 284 | fmt.Println(err) 285 | fis, err = f.Readdir(2) 286 | fmt.Println(fisStringer(fis), err) 287 | _, err = f.Seek(0, io.SeekStart) 288 | fmt.Println(err) 289 | fis, err = f.Readdir(1) 290 | fmt.Println(fisStringer(fis), err) 291 | fis, err = f.Readdir(10) 292 | fmt.Println(fisStringer(fis), err) 293 | fis, err = f.Readdir(10) 294 | fmt.Println(fisStringer(fis), err) 295 | 296 | // Output: 297 | // [ folderA folderB ] 298 | // [ not-worth-compressing-file.txt ] 299 | // 300 | // [ folderA folderB ] 301 | // 302 | // [ folderA ] 303 | // [ folderB not-worth-compressing-file.txt sample-file.txt ] 304 | // [ ] EOF 305 | } 306 | 307 | func Example_notExist() { 308 | var fs http.FileSystem = assets 309 | 310 | _, err := fs.Open("/does-not-exist") 311 | fmt.Println("os.IsNotExist:", os.IsNotExist(err)) 312 | fmt.Println(err) 313 | 314 | // Output: 315 | // os.IsNotExist: true 316 | // open /does-not-exist: file does not exist 317 | } 318 | 319 | func Example_pathCleaned() { 320 | var fs http.FileSystem = assets 321 | 322 | f, err := fs.Open("//folderB/../folderA/file1.txt") 323 | if err != nil { 324 | panic(err) 325 | } 326 | defer f.Close() 327 | 328 | fi, err := f.Stat() 329 | if err != nil { 330 | panic(err) 331 | } 332 | fmt.Println(fi.Name()) 333 | 334 | b, err := io.ReadAll(f) 335 | fmt.Printf("%q %v\n", string(b), err) 336 | 337 | // Output: 338 | // file1.txt 339 | // "Stuff in /folderA/file1.txt." 340 | } 341 | -------------------------------------------------------------------------------- /test/test_vfsdata_test.go: -------------------------------------------------------------------------------- 1 | // Code generated by vfsgen; DO NOT EDIT. 2 | 3 | package test_test 4 | 5 | import ( 6 | "bytes" 7 | "compress/gzip" 8 | "fmt" 9 | "io" 10 | "net/http" 11 | "os" 12 | pathpkg "path" 13 | "time" 14 | ) 15 | 16 | // assets statically implements the virtual filesystem provided to vfsgen. 17 | var assets = func() http.FileSystem { 18 | fs := vfsgen۰FS{ 19 | "/": &vfsgen۰DirInfo{ 20 | name: "/", 21 | modTime: time.Time{}, 22 | }, 23 | "/folderA": &vfsgen۰DirInfo{ 24 | name: "folderA", 25 | modTime: time.Time{}, 26 | }, 27 | "/folderA/file1.txt": &vfsgen۰FileInfo{ 28 | name: "file1.txt", 29 | modTime: time.Time{}, 30 | content: []byte("\x53\x74\x75\x66\x66\x20\x69\x6e\x20\x2f\x66\x6f\x6c\x64\x65\x72\x41\x2f\x66\x69\x6c\x65\x31\x2e\x74\x78\x74\x2e"), 31 | }, 32 | "/folderA/file2.txt": &vfsgen۰FileInfo{ 33 | name: "file2.txt", 34 | modTime: time.Time{}, 35 | content: []byte("\x53\x74\x75\x66\x66\x20\x69\x6e\x20\x2f\x66\x6f\x6c\x64\x65\x72\x41\x2f\x66\x69\x6c\x65\x32\x2e\x74\x78\x74\x2e"), 36 | }, 37 | "/folderB": &vfsgen۰DirInfo{ 38 | name: "folderB", 39 | modTime: time.Time{}, 40 | }, 41 | "/folderB/folderC": &vfsgen۰DirInfo{ 42 | name: "folderC", 43 | modTime: time.Time{}, 44 | }, 45 | "/folderB/folderC/file3.txt": &vfsgen۰FileInfo{ 46 | name: "file3.txt", 47 | modTime: time.Time{}, 48 | content: []byte("\x53\x74\x75\x66\x66\x20\x69\x6e\x20\x2f\x66\x6f\x6c\x64\x65\x72\x42\x2f\x66\x6f\x6c\x64\x65\x72\x43\x2f\x66\x69\x6c\x65\x33\x2e\x74\x78\x74\x2e"), 49 | }, 50 | "/not-worth-compressing-file.txt": &vfsgen۰FileInfo{ 51 | name: "not-worth-compressing-file.txt", 52 | modTime: time.Time{}, 53 | content: []byte("\x49\x74\x73\x20\x6e\x6f\x72\x6d\x61\x6c\x20\x63\x6f\x6e\x74\x65\x6e\x74\x73\x20\x61\x72\x65\x20\x68\x65\x72\x65\x2e"), 54 | }, 55 | "/sample-file.txt": &vfsgen۰CompressedFileInfo{ 56 | name: "sample-file.txt", 57 | modTime: time.Time{}, 58 | uncompressedSize: 189, 59 | 60 | compressedContent: []byte("\x1f\x8b\x08\x00\x00\x00\x00\x00\x02\xff\x0a\xc9\xc8\x2c\x56\x48\xcb\xcc\x49\x55\x48\xce\xcf\x2d\x28\x4a\x2d\x2e\x4e\x2d\x56\x28\x4f\xcd\xc9\xd1\x53\x70\xca\x49\x1c\xd4\x20\x43\x11\x10\x00\x00\xff\xff\xe7\x47\x81\x3a\xbd\x00\x00\x00"), 61 | }, 62 | } 63 | fs["/"].(*vfsgen۰DirInfo).entries = []os.FileInfo{ 64 | fs["/folderA"].(os.FileInfo), 65 | fs["/folderB"].(os.FileInfo), 66 | fs["/not-worth-compressing-file.txt"].(os.FileInfo), 67 | fs["/sample-file.txt"].(os.FileInfo), 68 | } 69 | fs["/folderA"].(*vfsgen۰DirInfo).entries = []os.FileInfo{ 70 | fs["/folderA/file1.txt"].(os.FileInfo), 71 | fs["/folderA/file2.txt"].(os.FileInfo), 72 | } 73 | fs["/folderB"].(*vfsgen۰DirInfo).entries = []os.FileInfo{ 74 | fs["/folderB/folderC"].(os.FileInfo), 75 | } 76 | fs["/folderB/folderC"].(*vfsgen۰DirInfo).entries = []os.FileInfo{ 77 | fs["/folderB/folderC/file3.txt"].(os.FileInfo), 78 | } 79 | 80 | return fs 81 | }() 82 | 83 | type vfsgen۰FS map[string]interface{} 84 | 85 | func (fs vfsgen۰FS) Open(path string) (http.File, error) { 86 | path = pathpkg.Clean("/" + path) 87 | f, ok := fs[path] 88 | if !ok { 89 | return nil, &os.PathError{Op: "open", Path: path, Err: os.ErrNotExist} 90 | } 91 | 92 | switch f := f.(type) { 93 | case *vfsgen۰CompressedFileInfo: 94 | gr, err := gzip.NewReader(bytes.NewReader(f.compressedContent)) 95 | if err != nil { 96 | // This should never happen because we generate the gzip bytes such that they are always valid. 97 | panic("unexpected error reading own gzip compressed bytes: " + err.Error()) 98 | } 99 | return &vfsgen۰CompressedFile{ 100 | vfsgen۰CompressedFileInfo: f, 101 | gr: gr, 102 | }, nil 103 | case *vfsgen۰FileInfo: 104 | return &vfsgen۰File{ 105 | vfsgen۰FileInfo: f, 106 | Reader: bytes.NewReader(f.content), 107 | }, nil 108 | case *vfsgen۰DirInfo: 109 | return &vfsgen۰Dir{ 110 | vfsgen۰DirInfo: f, 111 | }, nil 112 | default: 113 | // This should never happen because we generate only the above types. 114 | panic(fmt.Sprintf("unexpected type %T", f)) 115 | } 116 | } 117 | 118 | // vfsgen۰CompressedFileInfo is a static definition of a gzip compressed file. 119 | type vfsgen۰CompressedFileInfo struct { 120 | name string 121 | modTime time.Time 122 | compressedContent []byte 123 | uncompressedSize int64 124 | } 125 | 126 | func (f *vfsgen۰CompressedFileInfo) Readdir(count int) ([]os.FileInfo, error) { 127 | return nil, fmt.Errorf("cannot Readdir from file %s", f.name) 128 | } 129 | func (f *vfsgen۰CompressedFileInfo) Stat() (os.FileInfo, error) { return f, nil } 130 | 131 | func (f *vfsgen۰CompressedFileInfo) GzipBytes() []byte { 132 | return f.compressedContent 133 | } 134 | 135 | func (f *vfsgen۰CompressedFileInfo) Name() string { return f.name } 136 | func (f *vfsgen۰CompressedFileInfo) Size() int64 { return f.uncompressedSize } 137 | func (f *vfsgen۰CompressedFileInfo) Mode() os.FileMode { return 0444 } 138 | func (f *vfsgen۰CompressedFileInfo) ModTime() time.Time { return f.modTime } 139 | func (f *vfsgen۰CompressedFileInfo) IsDir() bool { return false } 140 | func (f *vfsgen۰CompressedFileInfo) Sys() interface{} { return nil } 141 | 142 | // vfsgen۰CompressedFile is an opened compressedFile instance. 143 | type vfsgen۰CompressedFile struct { 144 | *vfsgen۰CompressedFileInfo 145 | gr *gzip.Reader 146 | grPos int64 // Actual gr uncompressed position. 147 | seekPos int64 // Seek uncompressed position. 148 | } 149 | 150 | func (f *vfsgen۰CompressedFile) Read(p []byte) (n int, err error) { 151 | if f.grPos > f.seekPos { 152 | // Rewind to beginning. 153 | err = f.gr.Reset(bytes.NewReader(f.compressedContent)) 154 | if err != nil { 155 | return 0, err 156 | } 157 | f.grPos = 0 158 | } 159 | if f.grPos < f.seekPos { 160 | // Fast-forward. 161 | _, err = io.CopyN(io.Discard, f.gr, f.seekPos-f.grPos) 162 | if err != nil { 163 | return 0, err 164 | } 165 | f.grPos = f.seekPos 166 | } 167 | n, err = f.gr.Read(p) 168 | f.grPos += int64(n) 169 | f.seekPos = f.grPos 170 | return n, err 171 | } 172 | func (f *vfsgen۰CompressedFile) Seek(offset int64, whence int) (int64, error) { 173 | switch whence { 174 | case io.SeekStart: 175 | f.seekPos = 0 + offset 176 | case io.SeekCurrent: 177 | f.seekPos += offset 178 | case io.SeekEnd: 179 | f.seekPos = f.uncompressedSize + offset 180 | default: 181 | panic(fmt.Errorf("invalid whence value: %v", whence)) 182 | } 183 | return f.seekPos, nil 184 | } 185 | func (f *vfsgen۰CompressedFile) Close() error { 186 | return f.gr.Close() 187 | } 188 | 189 | // vfsgen۰FileInfo is a static definition of an uncompressed file (because it's not worth gzip compressing). 190 | type vfsgen۰FileInfo struct { 191 | name string 192 | modTime time.Time 193 | content []byte 194 | } 195 | 196 | func (f *vfsgen۰FileInfo) Readdir(count int) ([]os.FileInfo, error) { 197 | return nil, fmt.Errorf("cannot Readdir from file %s", f.name) 198 | } 199 | func (f *vfsgen۰FileInfo) Stat() (os.FileInfo, error) { return f, nil } 200 | 201 | func (f *vfsgen۰FileInfo) NotWorthGzipCompressing() {} 202 | 203 | func (f *vfsgen۰FileInfo) Name() string { return f.name } 204 | func (f *vfsgen۰FileInfo) Size() int64 { return int64(len(f.content)) } 205 | func (f *vfsgen۰FileInfo) Mode() os.FileMode { return 0444 } 206 | func (f *vfsgen۰FileInfo) ModTime() time.Time { return f.modTime } 207 | func (f *vfsgen۰FileInfo) IsDir() bool { return false } 208 | func (f *vfsgen۰FileInfo) Sys() interface{} { return nil } 209 | 210 | // vfsgen۰File is an opened file instance. 211 | type vfsgen۰File struct { 212 | *vfsgen۰FileInfo 213 | *bytes.Reader 214 | } 215 | 216 | func (f *vfsgen۰File) Close() error { 217 | return nil 218 | } 219 | 220 | // vfsgen۰DirInfo is a static definition of a directory. 221 | type vfsgen۰DirInfo struct { 222 | name string 223 | modTime time.Time 224 | entries []os.FileInfo 225 | } 226 | 227 | func (d *vfsgen۰DirInfo) Read([]byte) (int, error) { 228 | return 0, fmt.Errorf("cannot Read from directory %s", d.name) 229 | } 230 | func (d *vfsgen۰DirInfo) Close() error { return nil } 231 | func (d *vfsgen۰DirInfo) Stat() (os.FileInfo, error) { return d, nil } 232 | 233 | func (d *vfsgen۰DirInfo) Name() string { return d.name } 234 | func (d *vfsgen۰DirInfo) Size() int64 { return 0 } 235 | func (d *vfsgen۰DirInfo) Mode() os.FileMode { return 0755 | os.ModeDir } 236 | func (d *vfsgen۰DirInfo) ModTime() time.Time { return d.modTime } 237 | func (d *vfsgen۰DirInfo) IsDir() bool { return true } 238 | func (d *vfsgen۰DirInfo) Sys() interface{} { return nil } 239 | 240 | // vfsgen۰Dir is an opened dir instance. 241 | type vfsgen۰Dir struct { 242 | *vfsgen۰DirInfo 243 | pos int // Position within entries for Seek and Readdir. 244 | } 245 | 246 | func (d *vfsgen۰Dir) Seek(offset int64, whence int) (int64, error) { 247 | if offset == 0 && whence == io.SeekStart { 248 | d.pos = 0 249 | return 0, nil 250 | } 251 | return 0, fmt.Errorf("unsupported Seek in directory %s", d.name) 252 | } 253 | 254 | func (d *vfsgen۰Dir) Readdir(count int) ([]os.FileInfo, error) { 255 | if d.pos >= len(d.entries) && count > 0 { 256 | return nil, io.EOF 257 | } 258 | if count <= 0 || count > len(d.entries)-d.pos { 259 | count = len(d.entries) - d.pos 260 | } 261 | e := d.entries[d.pos : d.pos+count] 262 | d.pos += count 263 | return e, nil 264 | } 265 | --------------------------------------------------------------------------------