├── .gitignore ├── CONTRIBUTORS ├── testfiles ├── first.zip └── old_iris_favicon.ico ├── .github ├── FUNDING.yml ├── dependabot.yml ├── workflows │ └── ci.yml └── ISSUE_TEMPLATE │ ├── feature_request.md │ └── bug_report.md ├── go.mod ├── go.sum ├── LICENSE ├── installer_test.go ├── compression.go ├── http_test.go ├── installer.go ├── http.go ├── fs.go ├── updater.go └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .project 2 | .DS_STORE 3 | -------------------------------------------------------------------------------- /CONTRIBUTORS: -------------------------------------------------------------------------------- 1 | Gerasimos Maropoulos 2 | Brad Bumbalough 3 | -------------------------------------------------------------------------------- /testfiles/first.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kataras/go-fs/HEAD/testfiles/first.zip -------------------------------------------------------------------------------- /testfiles/old_iris_favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kataras/go-fs/HEAD/testfiles/old_iris_favicon.ico -------------------------------------------------------------------------------- /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: kataras 4 | # custom: http://iris-go.com/donate 5 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | updates: 3 | - package-ecosystem: "gomod" 4 | directory: "/" 5 | schedule: 6 | interval: "daily" -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/kataras/go-fs 2 | 3 | go 1.21 4 | 5 | require ( 6 | github.com/google/go-github v17.0.0+incompatible 7 | github.com/hashicorp/go-version v1.6.0 8 | github.com/klauspost/compress v1.17.4 9 | ) 10 | 11 | require github.com/google/go-querystring v1.1.0 // indirect 12 | -------------------------------------------------------------------------------- /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: CI 2 | 3 | on: 4 | push: 5 | branches: [master] 6 | pull_request: 7 | branches: [master] 8 | 9 | jobs: 10 | test: 11 | name: Test 12 | runs-on: ubuntu-latest 13 | 14 | strategy: 15 | matrix: 16 | go_version: [1.21.x] 17 | steps: 18 | 19 | - name: Set up Go 1.x 20 | uses: actions/setup-go@v5 21 | with: 22 | go-version: ${{ matrix.go_version }} 23 | 24 | - name: Check out code into the Go module directory 25 | uses: actions/checkout@v4 26 | 27 | - name: Test 28 | run: go test -v ./... 29 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for go-fs 4 | title: "[FEATURE REQUEST]" 5 | labels: enhancement 6 | assignees: kataras 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: "[BUG]" 5 | labels: bug 6 | assignees: kataras 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **To Reproduce** 14 | Steps to reproduce the behavior: 15 | 1. Go to '...' 16 | 2. Click on '....' 17 | 3. Scroll down to '....' 18 | 4. See error 19 | 20 | **Expected behavior** 21 | A clear and concise description of what you expected to happen. 22 | 23 | **Screenshots** 24 | If applicable, add screenshots to help explain your problem. 25 | 26 | **Desktop (please complete the following information):** 27 | - OS: [e.g. Windows] 28 | - Version [e.g. 10] 29 | 30 | **Additional context** 31 | Add any other context about the problem here. 32 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM= 2 | github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= 3 | github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY= 4 | github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ= 5 | github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8= 6 | github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU= 7 | github.com/hashicorp/go-version v1.6.0 h1:feTTfFNnjP967rlCxM/I9g701jU+RN74YKx2mOkIeek= 8 | github.com/hashicorp/go-version v1.6.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= 9 | github.com/klauspost/compress v1.17.4 h1:Ej5ixsIri7BrIjBkRZLTo6ghwrEtHFk7ijlczPW4fZ4= 10 | github.com/klauspost/compress v1.17.4/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM= 11 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016-2024 Gerasimos Maropoulos 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /installer_test.go: -------------------------------------------------------------------------------- 1 | package fs 2 | 3 | /// NOTE 4 | // Please be careful when running this test it will create a go-installer-test folder in your $HOME directory 5 | // At the end of these tests this should be removed 6 | // also this test may take some time if low download speed, it downloads real files and installs them 7 | // 8 | 9 | import ( 10 | "testing" 11 | ) 12 | 13 | var testInstalledDir = GetHomePath() + PathSeparator + "go-installer-test" + PathSeparator 14 | 15 | // remote file zip | expected output(installed) directory 16 | var filesToInstall = map[string]string{ 17 | "https://github.com/kataras/iris/archive/main.zip": testInstalledDir + "iris-main", 18 | "https://github.com/kataras/neffos/archive/master.zip": testInstalledDir + "neffos-master", 19 | "https://github.com/kataras/go-fs/archive/master.zip": testInstalledDir + "go-fs-master", 20 | "https://github.com/kataras/go-events/archive/master.zip": testInstalledDir + "go-events-master", 21 | } 22 | 23 | func TestInstallerFull(t *testing.T) { 24 | defer RemoveFile(testInstalledDir) 25 | myInstaller := NewInstaller(testInstalledDir) 26 | 27 | for remoteURI := range filesToInstall { 28 | myInstaller.Add(remoteURI) 29 | } 30 | 31 | installedDirs, err := myInstaller.Install() 32 | 33 | if err != nil { 34 | t.Fatal(err) 35 | } 36 | 37 | // check for created files 38 | for _, installedDir := range installedDirs { 39 | 40 | if !DirectoryExists(installedDir) { 41 | t.Logf("Failed: %s\n", installedDir) 42 | t.Fatalf("Error while installation completed: Directories were not created(expected len = %d but got %d), files are not unzipped correctly to the root destination path(Destination = %s)", len(filesToInstall), len(installedDirs), testInstalledDir) 43 | } 44 | } 45 | 46 | // check if any remote file remains to the installer, should be 0 47 | if len(myInstaller.RemoteFiles) > 0 { 48 | t.Fatalf("Error while installation completed: Some remote files are reaming to the installer instance, should be len of 0 but got %d", len(myInstaller.RemoteFiles)) 49 | } 50 | 51 | } 52 | 53 | func TestManualInstalls(t *testing.T) { 54 | // first check if already exists, from the previous test, if yes then remote the folder first 55 | RemoveFile(testInstalledDir) 56 | defer RemoveFile(testInstalledDir) 57 | for remoteURI, expectedInstalledDir := range filesToInstall { 58 | 59 | installedDir, err := Install(remoteURI, testInstalledDir, false) 60 | 61 | if err != nil { 62 | t.Fatal(err) 63 | } 64 | 65 | // check for created file 66 | if !DirectoryExists(installedDir) { 67 | t.Logf("Failed: %s\n", installedDir) 68 | } 69 | 70 | if expectedInstalledDir != installedDir { 71 | t.Fatalf("Expected installation dir to be: %s but got: %s", expectedInstalledDir, installedDir) 72 | } 73 | } 74 | 75 | } 76 | -------------------------------------------------------------------------------- /compression.go: -------------------------------------------------------------------------------- 1 | package fs 2 | 3 | import ( 4 | "io" 5 | "sync" 6 | 7 | "github.com/klauspost/compress/gzip" 8 | ) 9 | 10 | // writes gzip compressed content to an underline io.Writer. It uses sync.Pool to reduce memory allocations. 11 | // Better performance through klauspost/compress package which provides us a gzip.Writer which is faster than Go standard's gzip package's writer. 12 | 13 | // These constants are copied from the standard flate package 14 | // available Compressors 15 | const ( 16 | NoCompression = 0 17 | BestSpeed = 1 18 | BestCompression = 9 19 | DefaultCompression = -1 20 | ConstantCompression = -2 // Does only Huffman encoding 21 | ) 22 | 23 | // GzipPool is a wrapper of sync.Pool, to initialize a new gzip writer pool, just create a new instance of this iteral, GzipPool{} 24 | type GzipPool struct { 25 | sync.Pool 26 | Level int 27 | } 28 | 29 | // NewGzipPool returns a new gzip writer pool, ready to use 30 | func NewGzipPool(Level int) *GzipPool { 31 | return &GzipPool{Level: Level} 32 | } 33 | 34 | // DefaultGzipPool returns a new writer pool with Compressor's level setted to DefaultCompression 35 | func DefaultGzipPool() *GzipPool { 36 | return NewGzipPool(DefaultCompression) 37 | } 38 | 39 | // default writer pool with Compressor's level setted to DefaultCompression 40 | var defaultGzipWriterPool = DefaultGzipPool() 41 | 42 | // AcquireGzipWriter prepares a gzip writer and returns it 43 | // 44 | // see ReleaseGzipWriter 45 | func AcquireGzipWriter(w io.Writer) *gzip.Writer { 46 | return defaultGzipWriterPool.AcquireGzipWriter(w) 47 | } 48 | 49 | // AcquireGzipWriter prepares a gzip writer and returns it 50 | // 51 | // see ReleaseGzipWriter 52 | func (p *GzipPool) AcquireGzipWriter(w io.Writer) *gzip.Writer { 53 | v := p.Get() 54 | if v == nil { 55 | gzipWriter, err := gzip.NewWriterLevel(w, p.Level) 56 | if err != nil { 57 | return nil 58 | } 59 | return gzipWriter 60 | } 61 | gzipWriter := v.(*gzip.Writer) 62 | gzipWriter.Reset(w) 63 | return gzipWriter 64 | } 65 | 66 | // ReleaseGzipWriter called when flush/close and put the gzip writer back to the pool 67 | // 68 | // see AcquireGzipWriter 69 | func ReleaseGzipWriter(gzipWriter *gzip.Writer) { 70 | defaultGzipWriterPool.ReleaseGzipWriter(gzipWriter) 71 | } 72 | 73 | // ReleaseGzipWriter called when flush/close and put the gzip writer back to the pool 74 | // 75 | // see AcquireGzipWriter 76 | func (p *GzipPool) ReleaseGzipWriter(gzipWriter *gzip.Writer) { 77 | gzipWriter.Close() 78 | p.Put(gzipWriter) 79 | } 80 | 81 | // WriteGzip writes a compressed form of p to the underlying io.Writer. The 82 | // compressed bytes are not necessarily flushed until the Writer is closed 83 | func WriteGzip(w io.Writer, b []byte) (int, error) { 84 | return defaultGzipWriterPool.WriteGzip(w, b) 85 | } 86 | 87 | // WriteGzip writes a compressed form of p to the underlying io.Writer. The 88 | // compressed bytes are not necessarily flushed until the Writer is closed 89 | func (p *GzipPool) WriteGzip(w io.Writer, b []byte) (int, error) { 90 | gzipWriter := p.AcquireGzipWriter(w) 91 | n, err := gzipWriter.Write(b) 92 | p.ReleaseGzipWriter(gzipWriter) 93 | return n, err 94 | } 95 | -------------------------------------------------------------------------------- /http_test.go: -------------------------------------------------------------------------------- 1 | package fs 2 | 3 | import ( 4 | "bytes" 5 | "net/http" 6 | "net/http/httptest" 7 | "os" 8 | "path/filepath" 9 | "strings" 10 | "testing" 11 | ) 12 | 13 | func TestStaticContentHandler(t *testing.T) { 14 | req, err := http.NewRequest("GET", "/http.go", nil) 15 | if err != nil { 16 | t.Fatal(err) 17 | } 18 | contents, err := os.ReadFile("./http.go") 19 | if err != nil { 20 | t.Fatal(err) 21 | } 22 | 23 | h := StaticContentHandler(contents, "text/plain") 24 | res := httptest.NewRecorder() 25 | h.ServeHTTP(res, req) 26 | 27 | if status := res.Code; status != http.StatusOK { 28 | t.Errorf("handler returned wrong status code: got %v want %v", 29 | status, http.StatusOK) 30 | } 31 | 32 | if ctype := res.Header().Get("Content-Type"); ctype != "text/plain; charset=utf-8" { 33 | t.Errorf("handler returned wrong content type: got %v want %v", 34 | ctype, "text/plain; charset=utf-8") 35 | } 36 | 37 | body := res.Body.String() 38 | if !strings.HasPrefix(body, "package fs") { 39 | t.Errorf("handler returned wrong contents, got %v", body) 40 | } 41 | } 42 | 43 | func TestDirHandler(t *testing.T) { 44 | req, err := http.NewRequest("GET", "/http.go", nil) 45 | if err != nil { 46 | t.Fatal(err) 47 | } 48 | h := DirHandler("./", "") 49 | res := httptest.NewRecorder() 50 | h.ServeHTTP(res, req) 51 | 52 | if status := res.Code; status != http.StatusOK { 53 | t.Errorf("handler returned wrong status code: got %v want %v", 54 | status, http.StatusOK) 55 | } 56 | 57 | if ctype := res.Header().Get("Content-Type"); ctype != "text/x-go; charset=utf-8" { 58 | t.Errorf("handler returned wrong content type: got %v want %v", 59 | ctype, "text/x-go; charset=utf-8") 60 | } 61 | 62 | body := res.Body.String() 63 | if !strings.HasPrefix(body, "package fs") { 64 | t.Errorf("handler returned wrong contents") 65 | } 66 | } 67 | 68 | // TestFaviconHandler will test the FaviconHandler which calls the StaticContentHandler too 69 | func TestFaviconHandler(t *testing.T) { 70 | favPath := "./testfiles/old_iris_favicon.ico" 71 | 72 | req, err := http.NewRequest("GET", "/favicon.ico", nil) 73 | if err != nil { 74 | t.Fatal(err) 75 | } 76 | h := FaviconHandler(favPath) 77 | res := httptest.NewRecorder() 78 | h.ServeHTTP(res, req) 79 | 80 | if status := res.Code; status != http.StatusOK { 81 | t.Errorf("handler returned wrong status code: got %v want %v", 82 | status, http.StatusOK) 83 | } 84 | 85 | if ctype := res.Header().Get("Content-Type"); ctype != "image/vnd.microsoft.icon; charset=utf-8" { 86 | t.Errorf("handler returned wrong content type: got %v want %v", 87 | ctype, "image/vnd.microsoft.icon; charset=utf-8") 88 | } 89 | 90 | body := res.Body.Bytes() 91 | favContents, err := os.ReadFile(favPath) 92 | if err != nil { 93 | t.Fatal(err) 94 | } 95 | if !bytes.Equal(body, favContents) { 96 | t.Errorf("handler returned wrong contents") 97 | } 98 | } 99 | 100 | // TestSendStaticFileHandler will test the SendStaticFileHandler which calls the StaticFileHandler too 101 | func TestSendStaticFileHandler(t *testing.T) { 102 | sendFile := "./testfiles/first.zip" 103 | 104 | req, err := http.NewRequest("GET", "/first.zip", nil) 105 | if err != nil { 106 | t.Fatal(err) 107 | } 108 | h := SendStaticFileHandler(sendFile) 109 | res := httptest.NewRecorder() 110 | h.ServeHTTP(res, req) 111 | 112 | if status := res.Code; status != http.StatusOK { 113 | t.Errorf("handler returned wrong status code: got %v want %v", 114 | status, http.StatusOK) 115 | } 116 | 117 | if ctype := res.Header().Get("Content-Type"); ctype != "application/zip; charset=utf-8" { 118 | t.Errorf("handler returned wrong content type: got %v want %v", 119 | ctype, "application/zip; charset=utf-8") 120 | } 121 | 122 | // get the filename only, no the abs path 123 | _, filename := filepath.Split(sendFile) 124 | 125 | if attachment := res.Header().Get(contentDisposition); attachment != "attachment;filename="+filename { 126 | t.Errorf("handler returned wrong attachment: got %v want %v", 127 | attachment, "attachment;filename="+filename) 128 | } 129 | 130 | body := res.Body.Bytes() 131 | fileContents, err := os.ReadFile(sendFile) 132 | if err != nil { 133 | t.Fatal(err) 134 | } 135 | if !bytes.Equal(body, fileContents) { 136 | t.Errorf("handler returned wrong contents") 137 | } 138 | } 139 | -------------------------------------------------------------------------------- /installer.go: -------------------------------------------------------------------------------- 1 | package fs 2 | 3 | import ( 4 | "errors" 5 | "fmt" 6 | "io" 7 | "net/http" 8 | "os" 9 | "strings" 10 | "sync" 11 | "time" 12 | ) 13 | 14 | var ( 15 | // errNoZip describes the error when the file is not compressed. 16 | errNoZip = errors.New("file not a zip") 17 | // errFileDownload describes the error when downloading a file was failed. 18 | errFileDownload = errors.New("download file") 19 | ) 20 | 21 | // ShowIndicator shows a silly terminal indicator for a process, close of the finish channel is done here. 22 | func ShowIndicator(wr io.Writer, newLine bool) chan bool { 23 | finish := make(chan bool) 24 | go func() { 25 | if newLine { 26 | wr.Write([]byte("\n")) 27 | } 28 | wr.Write([]byte("|")) 29 | wr.Write([]byte("_")) 30 | wr.Write([]byte("|")) 31 | 32 | for { 33 | select { 34 | case v := <-finish: 35 | { 36 | if v { 37 | wr.Write([]byte("\010\010\010")) //remove the loading chars 38 | close(finish) 39 | return 40 | } 41 | } 42 | default: 43 | wr.Write([]byte("\010\010-")) 44 | time.Sleep(time.Second / 2) 45 | wr.Write([]byte("\010\\")) 46 | time.Sleep(time.Second / 2) 47 | wr.Write([]byte("\010|")) 48 | time.Sleep(time.Second / 2) 49 | wr.Write([]byte("\010/")) 50 | time.Sleep(time.Second / 2) 51 | wr.Write([]byte("\010-")) 52 | time.Sleep(time.Second / 2) 53 | wr.Write([]byte("|")) 54 | } 55 | } 56 | 57 | }() 58 | 59 | return finish 60 | } 61 | 62 | // DownloadZip downloads a zip file returns the downloaded filename and an error. 63 | func DownloadZip(zipURL string, newDir string, showOutputIndication bool) (string, error) { 64 | var err error 65 | var size int64 66 | if showOutputIndication { 67 | finish := ShowIndicator(os.Stdout, true) 68 | 69 | defer func() { 70 | finish <- true 71 | }() 72 | } 73 | 74 | os.MkdirAll(newDir, 0755) 75 | tokens := strings.Split(zipURL, "/") 76 | fileName := newDir + tokens[len(tokens)-1] 77 | if !strings.HasSuffix(fileName, ".zip") { 78 | return "", fmt.Errorf("filename: %s: %w", fileName, errNoZip) 79 | } 80 | 81 | output, err := os.Create(fileName) 82 | if err != nil { 83 | return "", fmt.Errorf("%w: %s", errFileCreate, err.Error()) 84 | } 85 | defer output.Close() 86 | response, err := http.Get(zipURL) 87 | if err != nil { 88 | return "", fmt.Errorf("%w: %s: %s", errFileDownload, zipURL, err.Error()) 89 | } 90 | defer response.Body.Close() 91 | 92 | size, err = io.Copy(output, response.Body) 93 | if err != nil { 94 | return "", fmt.Errorf("%w: %s", errFileCopy, err.Error()) 95 | } 96 | 97 | if showOutputIndication { 98 | print("OK ", size, " bytes downloaded") 99 | } 100 | 101 | return fileName, nil 102 | 103 | } 104 | 105 | // Install is just the flow of: downloadZip -> unzip -> removeFile(zippedFile) 106 | // accepts 3 parameters 107 | // 108 | // first parameter is the remote url file zip 109 | // second parameter is the target directory 110 | // third paremeter is a boolean which you can set to true to print out the progress 111 | // returns a string(installedDirectory) and an error 112 | // 113 | // (string) installedDirectory is the directory which the zip file had, this is the real installation path 114 | // the installedDirectory is not empty when the installation is succed, the targetDirectory is not already exists and no error happens 115 | // the installedDirectory is empty when the installation is already done by previous time or an error happens 116 | func Install(remoteFileZip string, targetDirectory string, showOutputIndication bool) (installedDirectory string, err error) { 117 | var zipFile string 118 | zipFile, err = DownloadZip(remoteFileZip, targetDirectory, showOutputIndication) 119 | if err == nil { 120 | installedDirectory, err = Unzip(zipFile, targetDirectory) 121 | if err == nil { 122 | RemoveFile(zipFile) 123 | } 124 | } 125 | return 126 | } 127 | 128 | // Installer is useful when you have single output-target directory and multiple zip files to install 129 | type Installer struct { 130 | // InstallDir is the directory which all zipped downloads will be extracted 131 | // defaults to $HOME path 132 | InstallDir string 133 | // Indicator when it's true it shows an indicator about the installation process 134 | // defaults to false 135 | Indicator bool 136 | // RemoteFiles is the list of the files which should be downloaded when Install() called 137 | RemoteFiles []string 138 | 139 | mu sync.Mutex 140 | } 141 | 142 | // NewInstaller returns an new Installer, it's just a builder to add remote files once and call Install to install all of them 143 | // first parameter is the installed directory, if empty then it uses the user's $HOME path 144 | // second parameter accepts optional remote zip files to be install 145 | func NewInstaller(installDir string, remoteFilesZip ...string) *Installer { 146 | if installDir == "" { 147 | installDir = GetHomePath() 148 | } 149 | return &Installer{InstallDir: installDir, Indicator: false, RemoteFiles: remoteFilesZip} 150 | } 151 | 152 | // Add adds a remote file(*.zip) to the list for download 153 | func (i *Installer) Add(remoteFilesZip ...string) { 154 | i.mu.Lock() 155 | i.RemoteFiles = append(i.RemoteFiles, remoteFilesZip...) 156 | i.mu.Unlock() 157 | } 158 | 159 | var errNoFilesToInstall = errors.New("no files to install, please use the .Add method to add remote zip files") 160 | 161 | // Install installs all RemoteFiles, when this function called then the RemoteFiles are being resseted 162 | // returns all installed paths and an error (if any) 163 | // it continues on errors and returns them when the operation completed 164 | func (i *Installer) Install() ([]string, error) { 165 | if len(i.RemoteFiles) == 0 { 166 | return nil, errNoFilesToInstall 167 | } 168 | 169 | var allErrors []string 170 | var installedDirectories []string // not strict to the remote file len because it continues on error 171 | 172 | // create a copy of the remote files 173 | remoteFiles := append([]string{}, i.RemoteFiles...) 174 | // clear the installers's remote files 175 | i.mu.Lock() 176 | i.RemoteFiles = nil 177 | 178 | for _, remoteFileZip := range remoteFiles { 179 | p, err := Install(remoteFileZip, i.InstallDir, i.Indicator) 180 | if err != nil { 181 | allErrors = append(allErrors, err.Error()) 182 | // add back the remote file if the install of this remote file has failed 183 | i.RemoteFiles = append(i.RemoteFiles, remoteFileZip) 184 | } 185 | 186 | installedDirectories = append(installedDirectories, p) 187 | } 188 | i.mu.Unlock() 189 | if len(allErrors) > 0 { 190 | return installedDirectories, fmt.Errorf(strings.Join(allErrors, ": ")) 191 | } 192 | 193 | return installedDirectories, nil 194 | } 195 | -------------------------------------------------------------------------------- /http.go: -------------------------------------------------------------------------------- 1 | package fs 2 | 3 | import ( 4 | "fmt" 5 | "net/http" 6 | "os" 7 | "path" 8 | "path/filepath" 9 | "strings" 10 | "time" 11 | ) 12 | 13 | var ( 14 | // TimeFormat default time format for any kind of datetime parsing 15 | TimeFormat = "Mon, 02 Jan 2006 15:04:05 GMT" 16 | // StaticCacheDuration expiration duration for INACTIVE file handlers 17 | StaticCacheDuration = 20 * time.Second 18 | // Charset the charset will be used to the Content-Type response header, if not given previously 19 | Charset = "utf-8" 20 | ) 21 | 22 | var ( 23 | // contentTypeHeader represents the header["Content-Type"] 24 | contentTypeHeader = "Content-Type" 25 | // contentLength represents the header["Content-Length"] 26 | contentLength = "Content-Length" 27 | // lastModified "Last-Modified" 28 | lastModified = "Last-Modified" 29 | // ifModifiedSince "If-Modified-Since" 30 | ifModifiedSince = "If-Modified-Since" 31 | // contentDisposition "Content-Disposition" 32 | contentDisposition = "Content-Disposition" 33 | // contentBinary header value for binary data. 34 | contentBinary = "application/octet-stream" 35 | ) 36 | 37 | func setContentType(res http.ResponseWriter, contentTypeValue string, alternative string) { 38 | // check if contnet type value is empty 39 | if contentTypeValue == "" && res.Header().Get("Content-Type") == "" { 40 | // if it's empty, then set it to alternative 41 | contentTypeValue = alternative 42 | } 43 | // check if charset part doesn't exists and the file is not binary form 44 | if !strings.Contains(contentTypeValue, ";charset=") && contentTypeValue != contentBinary { 45 | // if not, then add this to the value 46 | contentTypeValue += "; charset=" + Charset 47 | } 48 | // set the header 49 | res.Header().Set(contentTypeHeader, contentTypeValue) 50 | } 51 | 52 | func errorHandler(httpStatusCode int) http.Handler { 53 | return http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) { 54 | line := http.StatusText(httpStatusCode) 55 | if line == "" { 56 | line = http.StatusText(http.StatusBadRequest) 57 | } 58 | http.Error(res, line, httpStatusCode) 59 | }) 60 | } 61 | 62 | /* 63 | Note: 64 | If you want to be 100% compatible with http standars you have to put these handlers to both "GET" and "HEAD" HTTP Methods. 65 | */ 66 | 67 | // StaticContentHandler returns the net/http.Handler interface to handle raw binary data, 68 | // normally the data parameter was read by custom file reader or by variable 69 | func StaticContentHandler(data []byte, contentType string) http.Handler { 70 | if len(data) == 0 { 71 | return errorHandler(http.StatusNoContent) 72 | } 73 | modtime := time.Now() 74 | 75 | modtimeStr := modtime.UTC().Format(TimeFormat) 76 | return http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) { 77 | if t, err := time.Parse(TimeFormat, req.Header.Get(ifModifiedSince)); err == nil && modtime.Before(t.Add(StaticCacheDuration)) { 78 | res.Header().Del(contentTypeHeader) 79 | res.Header().Del(contentLength) 80 | res.WriteHeader(http.StatusNotModified) 81 | return 82 | } 83 | setContentType(res, contentType, contentBinary) 84 | res.Header().Set(lastModified, modtimeStr) 85 | res.Write(data) 86 | }) 87 | } 88 | 89 | // StaticFileHandler serves a static file such as css,js, favicons, static images 90 | // it stores the file contents to the memory, doesn't supports seek because we read all-in-one the file, but seek is supported by net/http.ServeContent 91 | func StaticFileHandler(filename string) http.Handler { 92 | fcontents, err := os.ReadFile(filename) // cache the contents of the file, this is the difference from net/http's impl, this is used only for static files, like favicons, css and so on 93 | if err != nil { 94 | return errorHandler(http.StatusBadRequest) 95 | } 96 | return StaticContentHandler(fcontents, TypeByExtension(filename)) 97 | } 98 | 99 | // SendStaticFileHandler sends a file for force-download to the client 100 | // it stores the file contents to the memory, doesn't supports seek because we read all-in-one the file, but seek is supported by net/http.ServeContent 101 | func SendStaticFileHandler(filename string) http.Handler { 102 | staticHandler := StaticFileHandler(filename) 103 | _, sendfilename := filepath.Split(filename) 104 | h := http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) { 105 | staticHandler.ServeHTTP(res, req) 106 | res.Header().Set(contentDisposition, "attachment;filename="+sendfilename) 107 | }) 108 | 109 | return h 110 | } 111 | 112 | // FaviconHandler receives the favicon path and serves the favicon 113 | func FaviconHandler(favPath string) http.Handler { 114 | f, err := os.Open(favPath) 115 | if err != nil { 116 | panic(fmt.Errorf("%w: %s: %s", errFileOpen, favPath, err.Error())) 117 | } 118 | defer f.Close() 119 | fi, _ := f.Stat() 120 | if fi.IsDir() { // if it's dir the try to get the favicon.ico 121 | fav := path.Join(favPath, "favicon.ico") 122 | f, err = os.Open(fav) 123 | if err != nil { 124 | //we try again with .png 125 | favPath = path.Join(favPath, "favicon.png") 126 | return FaviconHandler(favPath) 127 | } 128 | favPath = fav 129 | fi, _ = f.Stat() 130 | } 131 | 132 | cType := TypeByExtension(favPath) 133 | // copy the bytes here in order to cache and not read the ico on each request. 134 | cacheFav := make([]byte, fi.Size()) 135 | if _, err = f.Read(cacheFav); err != nil { 136 | panic(fmt.Errorf("%w: %s: %s", errFileRead, favPath, err.Error())) 137 | } 138 | 139 | return StaticContentHandler(cacheFav, cType) 140 | } 141 | 142 | // DirHandler serves a directory as web resource 143 | // accepts a system Directory (string), 144 | // a string which will be stripped off if not empty and 145 | // Note 1: this is a dynamic dir handler, means that if a new file is added to the folder it will be served 146 | // Note 2: it doesn't cache the system files, use it with your own risk, otherwise you can use the http.FileServer method, which is different of what I'm trying to do here. 147 | // example: 148 | // staticHandler := http.FileServer(http.Dir("static")) 149 | // http.Handle("/static/", http.StripPrefix("/static/", staticHandler)) 150 | // converted to -> 151 | // http.Handle("/static/", fs.DirHandler("./static", "/static/")) 152 | func DirHandler(dir string, strippedPrefix string) http.Handler { 153 | if dir == "" { 154 | return errorHandler(http.StatusNoContent) 155 | } 156 | 157 | dir = strings.Replace(dir, "/", PathSeparator, -1) 158 | 159 | h := http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) { 160 | reqpath := req.URL.Path 161 | if !strings.HasPrefix(reqpath, "/") { 162 | reqpath = PathSeparator + reqpath 163 | req.URL.Path = reqpath 164 | } 165 | reqpath = path.Clean(reqpath) 166 | fpath := reqpath 167 | relpath, err := filepath.Rel(dir, reqpath) 168 | if err != nil { 169 | abspath, err := filepath.Abs(dir + reqpath) 170 | if err == nil { 171 | fpath = abspath 172 | } 173 | } else { 174 | fpath = relpath 175 | } 176 | http.ServeFile(res, req, fpath) 177 | }) 178 | // the stripprefix handler checks for empty prefix so 179 | return http.StripPrefix(strippedPrefix, h) 180 | } 181 | -------------------------------------------------------------------------------- /fs.go: -------------------------------------------------------------------------------- 1 | // Package fs provides some common utilities which GoLang developers use when working with files, either system files or web files 2 | package fs 3 | 4 | import ( 5 | "archive/zip" 6 | "errors" 7 | "fmt" 8 | "io" 9 | "mime" 10 | "os" 11 | "path/filepath" 12 | "runtime" 13 | "strings" 14 | ) 15 | 16 | const ( 17 | // Version current version number 18 | Version = "0.0.5" 19 | ) 20 | 21 | // PathSeparator is the OS-specific path separator 22 | var PathSeparator = string(os.PathSeparator) 23 | 24 | // DirectoryExists returns true if a directory(or file) exists, otherwise false 25 | func DirectoryExists(dir string) bool { 26 | if _, err := os.Stat(dir); os.IsNotExist(err) { 27 | return false 28 | } 29 | return true 30 | } 31 | 32 | // GetHomePath returns the user's $HOME directory 33 | func GetHomePath() string { 34 | if runtime.GOOS == "windows" { 35 | return os.Getenv("HOMEDRIVE") + os.Getenv("HOMEPATH") 36 | } 37 | return os.Getenv("HOME") 38 | } 39 | 40 | // GetParentDir returns the parent directory(string) of the passed targetDirectory (string) 41 | func GetParentDir(targetDirectory string) string { 42 | lastSlashIndex := strings.LastIndexByte(targetDirectory, os.PathSeparator) 43 | //check if the slash is at the end , if yes then re- check without the last slash, we don't want /path/to/ , we want /path/to in order to get the /path/ which is the parent directory of the /path/to 44 | if lastSlashIndex == len(targetDirectory)-1 { 45 | lastSlashIndex = strings.LastIndexByte(targetDirectory[0:lastSlashIndex], os.PathSeparator) 46 | } 47 | 48 | parentDirectory := targetDirectory[0:lastSlashIndex] 49 | return parentDirectory 50 | } 51 | 52 | var ( 53 | // errFileOpen describes the error when opening a file. 54 | errFileOpen = errors.New("open file") 55 | // errFileCreate describes the error when creating a file. 56 | errFileCreate = errors.New("create file") 57 | // errFileRemove describes the error when deleting a file. 58 | errFileRemove = errors.New("delete file") 59 | // errFileCopy decsribes the error when copying a file. 60 | errFileCopy = errors.New("copy file") 61 | // errDirCreate describes the error when creating a directory. 62 | errDirCreate = errors.New("create directory") 63 | // errNotDir describes the error when a source file is not a directory. 64 | errNotDir = errors.New("source is not a directory") 65 | // errFileRead describes the error when reading a file. 66 | errFileRead = errors.New("read file") 67 | ) 68 | 69 | // RemoveFile removes a file or directory and returns an error, if any 70 | func RemoveFile(filePath string) error { 71 | err := os.RemoveAll(filePath) 72 | if err != nil { 73 | return fmt.Errorf("%s: %w", filePath, errFileRemove) 74 | } 75 | return nil 76 | } 77 | 78 | // RenameDir renames (moves) oldpath to newpath. 79 | // If newpath already exists, Rename replaces it. 80 | // OS-specific restrictions may apply when oldpath and newpath are in different directories. 81 | // If there is an error, it will be of type *LinkError. 82 | // 83 | // It's a copy of os.Rename 84 | func RenameDir(oldPath string, newPath string) error { 85 | return os.Rename(oldPath, newPath) 86 | } 87 | 88 | // CopyFile accepts full path of the source and full path of destination, if file exists it's overrides it 89 | // this function doesn't checks for permissions and all that, it returns an error 90 | func CopyFile(source string, destination string) error { 91 | reader, err := os.Open(source) 92 | 93 | if err != nil { 94 | return fmt.Errorf("%w: %s", errFileCopy, err.Error()) 95 | } 96 | 97 | defer reader.Close() 98 | 99 | writer, err := os.Create(destination) 100 | if err != nil { 101 | return fmt.Errorf("%w: %s", errFileCreate, err.Error()) 102 | } 103 | 104 | defer writer.Close() 105 | 106 | _, err = io.Copy(writer, reader) 107 | if err != nil { 108 | return fmt.Errorf("%w: %s", errFileCopy, err.Error()) 109 | } 110 | 111 | err = writer.Sync() 112 | if err != nil { 113 | return fmt.Errorf("%w: %s", errFileCopy, err.Error()) 114 | } 115 | 116 | return nil 117 | } 118 | 119 | // CopyDir recursively copies a directory tree, attempting to preserve permissions. 120 | // Source directory must exist. 121 | func CopyDir(source string, dest string) (err error) { 122 | 123 | // get properties of source dir 124 | fi, err := os.Stat(source) 125 | if err != nil { 126 | return err 127 | } 128 | 129 | if !fi.IsDir() { 130 | return fmt.Errorf("%s: %w", source, errNotDir) 131 | } 132 | 133 | // create dest dir 134 | 135 | err = os.MkdirAll(dest, fi.Mode()) 136 | if err != nil { 137 | return err 138 | } 139 | 140 | entries, err := os.ReadDir(source) 141 | 142 | for _, entry := range entries { 143 | 144 | sfp := source + PathSeparator + entry.Name() 145 | dfp := dest + PathSeparator + entry.Name() 146 | if entry.IsDir() { 147 | err = CopyDir(sfp, dfp) 148 | if err != nil { 149 | return 150 | } 151 | } else { 152 | // perform copy 153 | err = CopyFile(sfp, dfp) 154 | if err != nil { 155 | return 156 | } 157 | } 158 | 159 | } 160 | return 161 | } 162 | 163 | // Unzip extracts a zipped file to the target location 164 | // returns the path of the created folder (if any) and an error (if any) 165 | func Unzip(archive string, target string) (string, error) { 166 | reader, err := zip.OpenReader(archive) 167 | if err != nil { 168 | return "", err 169 | } 170 | 171 | if err := os.MkdirAll(target, 0755); err != nil { 172 | return "", fmt.Errorf("%w: %s", errDirCreate, err.Error()) 173 | } 174 | createdFolder := "" 175 | for _, file := range reader.File { 176 | path := filepath.Join(target, file.Name) 177 | if file.FileInfo().IsDir() { 178 | os.MkdirAll(path, file.Mode()) 179 | if createdFolder == "" { 180 | // this is the new directory that zip has 181 | createdFolder = path 182 | } 183 | continue 184 | } 185 | 186 | fileReader, err := file.Open() 187 | if err != nil { 188 | return "", fmt.Errorf("%w: %s", errFileOpen, err.Error()) 189 | } 190 | defer fileReader.Close() 191 | 192 | targetFile, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, file.Mode()) 193 | if err != nil { 194 | return "", fmt.Errorf("%w: %s", errFileOpen, err.Error()) 195 | } 196 | defer targetFile.Close() 197 | 198 | if _, err := io.Copy(targetFile, fileReader); err != nil { 199 | return "", fmt.Errorf("%w: %s", errFileCopy, err.Error()) 200 | } 201 | 202 | } 203 | 204 | reader.Close() 205 | return createdFolder, nil 206 | } 207 | 208 | // TypeByExtension returns the MIME type associated with the file extension ext. 209 | // The extension ext should begin with a leading dot, as in ".html". 210 | // When ext has no associated type, TypeByExtension returns "". 211 | // 212 | // Extensions are looked up first case-sensitively, then case-insensitively. 213 | // 214 | // The built-in table is small but on unix it is augmented by the local 215 | // system's mime.types file(s) if available under one or more of these 216 | // names: 217 | // 218 | // /etc/mime.types 219 | // /etc/apache2/mime.types 220 | // /etc/apache/mime.types 221 | // 222 | // On Windows, MIME types are extracted from the registry. 223 | // 224 | // Text types have the charset parameter set to "utf-8" by default. 225 | func TypeByExtension(fullfilename string) (t string) { 226 | ext := filepath.Ext(fullfilename) 227 | //these should be found by the windows(registry) and unix(apache) but on windows some machines have problems on this part. 228 | if t = mime.TypeByExtension(ext); t == "" { 229 | // no use of map here because we will have to lock/unlock it, by hand is better, no problem: 230 | if ext == ".json" { 231 | t = "application/json" 232 | } else if ext == ".js" { 233 | t = "application/javascript" 234 | } else if ext == ".zip" { 235 | t = "application/zip" 236 | } else if ext == ".3gp" { 237 | t = "video/3gpp" 238 | } else if ext == ".7z" { 239 | t = "application/x-7z-compressed" 240 | } else if ext == ".ace" { 241 | t = "application/x-ace-compressed" 242 | } else if ext == ".aac" { 243 | t = "audio/x-aac" 244 | } else if ext == ".ico" { // for any case 245 | t = "image/x-icon" 246 | } else if ext == ".png" { 247 | t = "image/png" 248 | } else { 249 | t = "application/octet-stream" 250 | } 251 | // mime.TypeByExtension returns as text/plain; | charset=utf-8 the static .js (not always) 252 | } else if t == "text/plain" || t == "text/plain; charset=utf-8" { 253 | if ext == ".js" { 254 | t = "application/javascript" 255 | } 256 | } 257 | return 258 | } 259 | -------------------------------------------------------------------------------- /updater.go: -------------------------------------------------------------------------------- 1 | package fs 2 | 3 | import ( 4 | "bufio" 5 | "context" 6 | "errors" 7 | "fmt" 8 | "io" 9 | "os" 10 | "os/exec" 11 | 12 | "github.com/google/go-github/github" 13 | "github.com/hashicorp/go-version" 14 | ) 15 | 16 | // updater.go Go app updater hosted on github, based on 'releases & tags', 17 | // unique and simple source code, no signs or other 'secure' methods. 18 | // 19 | // Note: the previous installed files(in $GOPATH) should not be edited before, if edited the go get tool will fail to upgrade these packages. 20 | // 21 | // tag name (github version) should be compatible with the Semantic Versioning 2.0.0 22 | // Read more about Semantic Versioning 2.0.0: http://semver.org/ 23 | // 24 | // quick example: 25 | // package main 26 | // 27 | // import ( 28 | // "github.com/kataras/go-fs" 29 | // "fmt" 30 | // ) 31 | // 32 | // func main(){ 33 | // fmt.Println("Current version is: 0.0.3") 34 | // 35 | // updater, err := fs.GetUpdater("kataras","rizla", "0.0.3") 36 | // if err !=nil{ 37 | // panic(err) 38 | // } 39 | // 40 | // updated := updater.Run() 41 | // _ = updated 42 | // } 43 | 44 | var ( 45 | errCantFetchRepo = errors.New("error while trying to fetch the remote repository") 46 | errAccessRepo = errors.New("couldn't access to the github repository, please make sure you're connected to the internet and you have access to read") 47 | ) 48 | 49 | // Updater is the base struct for the Updater feature 50 | type Updater struct { 51 | currentVersion *version.Version 52 | latestVersion *version.Version 53 | owner string 54 | repo string 55 | } 56 | 57 | // GetUpdater returns a new Updater based on a github repository and the latest local release version(string, example: "4.2.3" or "v4.2.3-rc1") 58 | func GetUpdater(owner string, repo string, currentReleaseVersion string) (*Updater, error) { 59 | client := github.NewClient(nil) // unuthenticated client, 60 req/hour 60 | ///TODO: rate limit error catching( impossible to same client checks 60 times for github updates, but we should do that check) 61 | 62 | // assign a new instace of context to support go-github's current API (https://github.com/google/go-github/issues/526) 63 | ctx := context.TODO() 64 | 65 | // get the latest release, delay depends on the user's internet connection's download speed 66 | latestRelease, response, err := client.Repositories.GetLatestRelease(ctx, owner, repo) 67 | if err != nil { 68 | return nil, fmt.Errorf("%w: %s:%s: %s", errCantFetchRepo, owner, repo, err.Error()) 69 | } 70 | 71 | if c := response.StatusCode; c != 200 && c != 201 && c != 202 && c != 301 && c != 302 && c == 304 { 72 | return nil, errAccessRepo 73 | } 74 | 75 | currentVersion, err := version.NewVersion(currentReleaseVersion) 76 | if err != nil { 77 | return nil, err 78 | } 79 | 80 | latestVersion, err := version.NewVersion(*latestRelease.TagName) 81 | if err != nil { 82 | return nil, err 83 | } 84 | 85 | u := &Updater{ 86 | currentVersion: currentVersion, 87 | latestVersion: latestVersion, 88 | owner: owner, 89 | repo: repo, 90 | } 91 | 92 | return u, nil 93 | } 94 | 95 | // HasUpdate returns true if a new update is available 96 | // the second output parameter is the latest ,full, version 97 | func (u *Updater) HasUpdate() (bool, string) { 98 | return u.currentVersion.LessThan(u.latestVersion), u.latestVersion.String() 99 | } 100 | 101 | var ( 102 | // DefaultUpdaterAlreadyInstalledMessage "\nThe latest version '%s' was already installed." 103 | DefaultUpdaterAlreadyInstalledMessage = "\nThe latest version '%s' was already installed." 104 | ) 105 | 106 | // Run runs the update, returns true if update has been found and installed, otherwise false 107 | func (u *Updater) Run(setters ...optionSetter) bool { 108 | opt := &Options{Stdin: os.Stdin, Stdout: os.Stdout, Stderr: os.Stderr, Silent: false} // default options 109 | 110 | for _, setter := range setters { 111 | setter.Set(opt) 112 | } 113 | 114 | writef := func(s string, a ...interface{}) { 115 | if !opt.Silent { 116 | opt.Stdout.Write([]byte(fmt.Sprintf(s, a...))) 117 | } 118 | } 119 | 120 | has, v := u.HasUpdate() 121 | if has { 122 | 123 | var scanner *bufio.Scanner 124 | if opt.Stdin != nil { 125 | scanner = bufio.NewScanner(opt.Stdin) 126 | } 127 | 128 | shouldProceedUpdate := func() bool { 129 | return shouldProceedUpdate(scanner) 130 | } 131 | 132 | writef("\nA newer version has been found[%s > %s].\n"+ 133 | "Release notes: %s\n"+ 134 | "Update now?[%s]: ", 135 | u.latestVersion.String(), u.currentVersion.String(), 136 | fmt.Sprintf("https://github.com/%s/%s/releases/latest", u.owner, u.repo), 137 | DefaultUpdaterYesInput[0]+"/n") 138 | 139 | if shouldProceedUpdate() { 140 | if !opt.Silent { 141 | finish := ShowIndicator(opt.Stdout, true) 142 | 143 | defer func() { 144 | finish <- true 145 | }() 146 | } 147 | // go get -u github.com/:owner/:repo 148 | cmd := exec.Command("go", "get", "-u", fmt.Sprintf("github.com/%s/%s", u.owner, u.repo)) 149 | cmd.Stdout = opt.Stdout 150 | cmd.Stderr = opt.Stderr 151 | 152 | if err := cmd.Run(); err != nil { 153 | writef("\nError while trying to get the package: %s.", err.Error()) 154 | } 155 | 156 | writef("\010\010\010") // remove the loading bars 157 | writef("Update has been installed, current version: %s. Please re-start your App.\n", u.latestVersion.String()) 158 | 159 | // TODO: normally, this should be in dev-mode machine, so a 'go build' and' & './$executable' on the current working path should be ok 160 | // for now just log a message to re-run the app manually 161 | //writef("\nUpdater was not able to re-build and re-run your updated App.\nPlease run your App again, by yourself.") 162 | return true 163 | } 164 | 165 | } else { 166 | writef(fmt.Sprintf(DefaultUpdaterAlreadyInstalledMessage, v)) 167 | } 168 | 169 | return false 170 | } 171 | 172 | // DefaultUpdaterYesInput the string or character which user should type to proceed the update, if !silent 173 | var DefaultUpdaterYesInput = [...]string{"y", "yes", "nai", "si"} 174 | 175 | func shouldProceedUpdate(sc *bufio.Scanner) bool { 176 | silent := sc == nil 177 | 178 | inputText := "" 179 | if !silent { 180 | if sc.Scan() { 181 | inputText = sc.Text() 182 | } 183 | } 184 | 185 | for _, s := range DefaultUpdaterYesInput { 186 | if inputText == s { 187 | return true 188 | } 189 | } 190 | // if silent, then return 'yes/true' always 191 | return silent 192 | } 193 | 194 | // Options the available options used iside the updater.Run func 195 | type Options struct { 196 | Silent bool 197 | // Stdin specifies the process's standard input. 198 | // If Stdin is nil, the process reads from the null device (os.DevNull). 199 | // If Stdin is an *os.File, the process's standard input is connected 200 | // directly to that file. 201 | // Otherwise, during the execution of the command a separate 202 | // goroutine reads from Stdin and delivers that data to the command 203 | // over a pipe. In this case, Wait does not complete until the goroutine 204 | // stops copying, either because it has reached the end of Stdin 205 | // (EOF or a read error) or because writing to the pipe returned an error. 206 | Stdin io.Reader 207 | 208 | // Stdout and Stderr specify the process's standard output and error. 209 | // 210 | // If either is nil, Run connects the corresponding file descriptor 211 | // to the null device (os.DevNull). 212 | // 213 | // If Stdout and Stderr are the same writer, at most one 214 | // goroutine at a time will call Write. 215 | Stdout io.Writer 216 | Stderr io.Writer 217 | } 218 | 219 | // Set implements the optionSetter 220 | func (o *Options) Set(main *Options) { 221 | main.Silent = o.Silent 222 | } 223 | 224 | type optionSetter interface { 225 | Set(*Options) 226 | } 227 | 228 | // OptionSet sets an option 229 | type OptionSet func(*Options) 230 | 231 | // Set implements the optionSetter 232 | func (o OptionSet) Set(main *Options) { 233 | o(main) 234 | } 235 | 236 | // Silent sets the Silent option to the 'val' 237 | func Silent(val bool) OptionSet { 238 | return func(o *Options) { 239 | o.Silent = val 240 | } 241 | } 242 | 243 | // Stdin specifies the process's standard input. 244 | // If Stdin is nil, the process reads from the null device (os.DevNull). 245 | // If Stdin is an *os.File, the process's standard input is connected 246 | // directly to that file. 247 | // Otherwise, during the execution of the command a separate 248 | // goroutine reads from Stdin and delivers that data to the command 249 | // over a pipe. In this case, Wait does not complete until the goroutine 250 | // stops copying, either because it has reached the end of Stdin 251 | // (EOF or a read error) or because writing to the pipe returned an error. 252 | func Stdin(val io.Reader) OptionSet { 253 | return func(o *Options) { 254 | o.Stdin = val 255 | } 256 | } 257 | 258 | // Stdout specify the process's standard output and error. 259 | // 260 | // If either is nil, Run connects the corresponding file descriptor 261 | // to the null device (os.DevNull). 262 | func Stdout(val io.Writer) OptionSet { 263 | return func(o *Options) { 264 | o.Stdout = val 265 | } 266 | } 267 | 268 | // Stderr specify the process's standard output and error. 269 | // 270 | // If Stdout and Stderr are the same writer, at most one 271 | // goroutine at a time will call Write. 272 | func Stderr(val io.Writer) OptionSet { 273 | return func(o *Options) { 274 | o.Stderr = val 275 | } 276 | } 277 | 278 | // simple way to compare version is to make them numbers 279 | // and remove any dots and 'v' or 'version' or 'release' 280 | // so 281 | // the v4.2.2 will be 422 282 | // which is bigger from v4.2.1 (which will be 421) 283 | // also a version could be something like: 1.0.0-beta+exp.sha.5114f85 284 | // so we should add a number of any alpha,beta,rc and so on 285 | // maybe this way is not the best but I think it will cover our needs 286 | // and the simplicity of source I keep to all of my packages. 287 | //var removeChars = [...]string{".","v","version","prerelease","pre-release","release","-","alpha","beta","rc"} 288 | // or just remove any non-numeric chars using regex... 289 | // ok.. just found a better way, to use a third-party package 'go-version' which will cover all version formats 290 | //func parseVersion(s string) int { 291 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Go FS 2 | 3 | Build Status 4 | License 5 | Releases 6 | Read me docs 7 | Build Status 8 | Built with GoLang 9 | Platforms 10 | 11 | The package go-fs provides some common utilities which GoLang developers use when working with files, either system files or web files. 12 | 13 | ## Installation 14 | 15 | The only requirement is the [Go Programming Language](https://golang.org/dl). 16 | 17 | ```bash 18 | $ go get github.com/kataras/go-fs 19 | ``` 20 | 21 | ## Documentation 22 | 23 | ### Local file system helpers 24 | 25 | ```go 26 | // DirectoryExists returns true if a directory(or file) exists, otherwise false 27 | DirectoryExists(dir string) bool 28 | 29 | // GetHomePath returns the user's $HOME directory 30 | GetHomePath() string 31 | 32 | // GetParentDir returns the parent directory of the passed targetDirectory 33 | GetParentDir(targetDirectory string) string 34 | 35 | // RemoveFile removes a file or a directory 36 | RemoveFile(filePath string) error 37 | 38 | // RenameDir renames (moves) oldpath to newpath. 39 | // If newpath already exists, Rename replaces it. 40 | // OS-specific restrictions may apply when oldpath and newpath are in different directories. 41 | // If there is an error, it will be of type *LinkError. 42 | // 43 | // It's a copy of os.Rename 44 | RenameDir(oldPath string, newPath string) error 45 | 46 | // CopyFile accepts full path of the source and full path of destination, if file exists it's overrides it 47 | // this function doesn't checks for permissions and all that, it returns an error 48 | CopyFile(source string, destination string) error 49 | 50 | // CopyDir recursively copies a directory tree, attempting to preserve permissions. 51 | // Source directory must exist. 52 | CopyDir(source string, dest string) error 53 | 54 | // Unzip extracts a zipped file to the target location 55 | // returns the path of the created folder (if any) and an error (if any) 56 | Unzip(archive string, target string) (string, error) 57 | 58 | // TypeByExtension returns the MIME type associated with the file extension ext. 59 | // The extension ext should begin with a leading dot, as in ".html". 60 | // When ext has no associated type, TypeByExtension returns "". 61 | // 62 | // Extensions are looked up first case-sensitively, then case-insensitively. 63 | // 64 | // The built-in table is small but on unix it is augmented by the local 65 | // system's mime.types file(s) if available under one or more of these 66 | // names: 67 | // 68 | // /etc/mime.types 69 | // /etc/apache2/mime.types 70 | // /etc/apache/mime.types 71 | // 72 | // On Windows, MIME types are extracted from the registry. 73 | // 74 | // Text types have the charset parameter set to "utf-8" by default. 75 | TypeByExtension(fullfilename string) string 76 | 77 | ``` 78 | 79 | ### Net/http handlers 80 | 81 | ```go 82 | // FaviconHandler receives the favicon path and serves the favicon 83 | FaviconHandler(favPath string) http.Handler 84 | 85 | // StaticContentHandler returns the net/http.Handler interface to handle raw binary data, 86 | // normally the data parameter was read by custom file reader or by variable 87 | StaticContentHandler(data []byte, contentType string) http.Handler 88 | 89 | // StaticFileHandler serves a static file such as css,js, favicons, static images 90 | // it stores the file contents to the memory, doesn't supports seek because we read all-in-one the file, but seek is supported by net/http.ServeContent 91 | StaticFileHandler(filename string) http.Handler 92 | 93 | // SendStaticFileHandler sends a file for force-download to the client 94 | // it stores the file contents to the memory, doesn't supports seek because we read all-in-one the file, but seek is supported by net/http.ServeContent 95 | SendStaticFileHandler(filename string) http.Handler 96 | 97 | // DirHandler serves a directory as web resource 98 | // accepts a system Directory (string), 99 | // a string which will be stripped off if not empty and 100 | // Note 1: this is a dynamic dir handler, means that if a new file is added to the folder it will be served 101 | // Note 2: it doesn't cache the system files, use it with your own risk, otherwise you can use the http.FileServer method, which is different of what I'm trying to do here. 102 | // example: 103 | // staticHandler := http.FileServer(http.Dir("static")) 104 | // http.Handle("/static/", http.StripPrefix("/static/", staticHandler)) 105 | // converted to -> 106 | // http.Handle("/static/", fs.DirHandler("./static", "/static/")) 107 | DirHandler(dir string, strippedPrefix string) http.Handler 108 | ``` 109 | 110 | Read the [http_test.go](https://github.com/kataras/go-fs/blob/master/http_test.go) for more. 111 | 112 | ### Gzip Writer 113 | Writes gzip compressed content to an underline io.Writer. It uses sync.Pool to reduce memory allocations. 114 | 115 | **Better performance** through klauspost/compress package which provides us a gzip.Writer which is faster than Go standard's gzip package's writer. 116 | 117 | ```go 118 | // NewGzipPool returns a new gzip writer pool, ready to use 119 | NewGzipPool(Level int) *GzipPool 120 | 121 | // DefaultGzipPool returns a new writer pool with Compressor's level setted to DefaultCompression 122 | DefaultGzipPool() *GzipPool 123 | 124 | // AcquireGzipWriter prepares a gzip writer and returns it 125 | // 126 | // see ReleaseGzipWriter 127 | AcquireGzipWriter(w io.Writer) *gzip.Writer 128 | 129 | // ReleaseGzipWriter called when flush/close and put the gzip writer back to the pool 130 | // 131 | // see AcquireGzipWriter 132 | ReleaseGzipWriter(gzipWriter *gzip.Writer) 133 | 134 | // WriteGzip writes a compressed form of p to the underlying io.Writer. The 135 | // compressed bytes are not necessarily flushed until the Writer is closed 136 | WriteGzip(w io.Writer, b []byte) (int, error) 137 | 138 | ``` 139 | 140 | 141 | - `AcquireGzipWriter` get a gzip writer, create new if no free writer available from inside the pool (sync.Pool). 142 | - `ReleaseGzipWriter` releases puts a gzip writer to the pool (sync.Pool). 143 | - `WriteGzip` gets a gzip writer, writes a compressed form of p to the underlying io.Writer. The 144 | compressed bytes are not necessarily flushed until the Writer is closed. Finally it Releases the particular gzip writer. 145 | 146 | > if these called from package level then the default gzip writer's pool is used to get/put and write 147 | 148 | - `NewGzipPool` receives a compression level and returns a new gzip writer pool 149 | - `DefaultGzipPool` returns a new gzip writer pool with DefaultCompression as the Compressor's Level 150 | 151 | > New & Default are optional, use them to create more than one sync.Pool, if you expect thousands of writers working together 152 | 153 | 154 | Using default pool's writer to compress & write content 155 | 156 | ```go 157 | import "github.com/kataras/go-fs" 158 | 159 | var writer io.Writer 160 | 161 | // ... using default package's Pool to get a gzip writer 162 | n, err := fs.WriteGzip(writer, []byte("Compressed data and content here")) 163 | ``` 164 | 165 | Using default Pool to get a gzip writer, compress & write content and finally release manually the gzip writer to the default Pool 166 | 167 | ```go 168 | import "github.com/kataras/go-fs" 169 | 170 | var writer io.Writer 171 | 172 | // ... using default writer's pool to get a gzip.Writer 173 | 174 | mygzipWriter := fs.AcquireGzipWriter(writer) // get a gzip.Writer from the default gzipwriter Pool 175 | 176 | n, err := mygzipWriter.WriteGzip([]byte("Compressed data and content here")) 177 | 178 | gzipwriter.ReleaseGzipWriter(mygzipWriter) // release this gzip.Writer to the default gzipwriter package's gzip writer Pool (sync.Pool) 179 | ``` 180 | 181 | Create and use a totally new gzip writer Pool 182 | 183 | ```go 184 | import "github.com/kataras/go-fs" 185 | 186 | var writer io.Writer 187 | var gzipWriterPool = fs.NewGzipPool(fs.DefaultCompression) 188 | 189 | // ... 190 | n, err := gzipWriterPool.WriteGzip(writer, []byte("Compressed data and content here")) 191 | ``` 192 | 193 | Get a gzip writer Pool with the default options(compressor's Level) 194 | 195 | ```go 196 | import "github.com/kataras/go-fs" 197 | 198 | var writer io.Writer 199 | var gzipWriterPool = fs.DefaultGzipPool() // returns a new default gzip writer pool 200 | 201 | // ... 202 | n, err := gzipWriterPool.WriteGzip(writer, []byte("Compressed data and content here")) 203 | ``` 204 | 205 | Acquire, Write and Release from a new(`.NewGzipPool/.DefaultGzipPool`) gzip writer Pool 206 | 207 | ```go 208 | import "github.com/kataras/go-fs" 209 | 210 | var writer io.Writer 211 | 212 | var gzipWriterPool = fs.DefaultGzipPool() // returns a new default gzip writer pool 213 | 214 | mygzipWriter := gzipWriterPool.AcquireGzipWriter(writer) // get a gzip.Writer from the new gzipWriterPool 215 | 216 | n, err := mygzipWriter.WriteGzip([]byte("Compressed data and content here")) 217 | 218 | gzipWriterPool.ReleaseGzipWriter(mygzipWriter) // release this gzip.Writer to the gzipWriterPool (sync.Pool) 219 | ``` 220 | 221 | 222 | ### Working with remote zip files 223 | 224 | ```go 225 | // DownloadZip downloads a zip file returns the downloaded filename and an error. 226 | DownloadZip(zipURL string, newDir string, showOutputIndication bool) (string, error) 227 | 228 | // Install is just the flow of: downloadZip -> unzip -> removeFile(zippedFile) 229 | // accepts 3 parameters 230 | // 231 | // first parameter is the remote url file zip 232 | // second parameter is the target directory 233 | // third paremeter is a boolean which you can set to true to print out the progress 234 | // returns a string(installedDirectory) and an error 235 | // 236 | // (string) installedDirectory is the directory which the zip file had, this is the real installation path 237 | // the installedDirectory is not empty when the installation is succed, the targetDirectory is not already exists and no error happens 238 | // the installedDirectory is empty when the installation is already done by previous time or an error happens 239 | Install(remoteFileZip string, targetDirectory string, showOutputIndication bool) (string, error) 240 | ``` 241 | 242 | > Install = DownloadZip -> Unzip to the destination folder, remove the downloaded .zip, copy the inside extracted folder to the destination 243 | 244 | Install many remote files(URI) to a single destination folder via installer instance 245 | 246 | ```go 247 | type Installer struct { 248 | // InstallDir is the directory which all zipped downloads will be extracted 249 | // defaults to $HOME path 250 | InstallDir string 251 | // Indicator when it's true it shows an indicator about the installation process 252 | // defaults to false 253 | Indicator bool 254 | // RemoteFiles is the list of the files which should be downloaded when Install() called 255 | RemoteFiles []string 256 | } 257 | 258 | // Add adds a remote file(*.zip) to the list for download 259 | Add(...string) 260 | 261 | // Install installs all RemoteFiles, when this function called then the RemoteFiles are being resseted 262 | // returns all installed paths and an error (if any) 263 | // it continues on errors and returns them when the operation completed 264 | Install() ([]string, error) 265 | ``` 266 | 267 | **Usage** 268 | 269 | ```go 270 | package main 271 | 272 | import "github.com/kataras/go-fs" 273 | 274 | var testInstalledDir = fs.GetHomePath() + fs.PathSeparator + "mydir" + fs.PathSeparator 275 | 276 | // remote file zip | expected output(installed) directory 277 | var filesToInstall = map[string]string{ 278 | "https://github.com/kataras/q/archive/master.zip": testInstalledDir + "q-master", 279 | "https://github.com/kataras/iris/archive/master.zip": testInstalledDir + "iris-master", 280 | "https://errors/archive/master.zip": testInstalledDir + "go-errors-master", 281 | "https://github.com/kataras/go-gzipwriter/archive/master.zip": testInstalledDir + "go-gzipwriter-master", 282 | "https://github.com/kataras/go-events/archive/master.zip": testInstalledDir + "go-events-master", 283 | } 284 | 285 | func main() { 286 | myInstaller := fs.NewInstaller(testInstalledDir) 287 | 288 | for remoteURI := range filesToInstall { 289 | myInstaller.Add(remoteURI) 290 | } 291 | 292 | installedDirs, err := myInstaller.Install() 293 | 294 | if err != nil { 295 | panic(err) 296 | } 297 | 298 | for _, installedDir := range installedDirs { 299 | println("New folder created: " + installedDir) 300 | } 301 | 302 | } 303 | 304 | ``` 305 | 306 | When you want to install different zip files to different destination directories. 307 | 308 | **Usage** 309 | 310 | ```go 311 | package main 312 | 313 | import "github.com/kataras/go-fs" 314 | 315 | var testInstalledDir = fs.GetHomePath() + fs.PathSeparator + "mydir" + fs.PathSeparator 316 | 317 | // remote file zip | expected output(installed) directory 318 | var filesToInstall = map[string]string{ 319 | "https://github.com/kataras/q/archive/master.zip": testInstalledDir + "q-master", 320 | "https://github.com/kataras/iris/archive/master.zip": testInstalledDir + "iris-master", 321 | "https://errors/archive/master.zip": testInstalledDir + "go-errors-master", 322 | "https://github.com/kataras/go-gzipwriter/archive/master.zip": testInstalledDir + "go-gzipwriter-master", 323 | "https://github.com/kataras/go-events/archive/master.zip": testInstalledDir + "go-events-master", 324 | } 325 | 326 | func main(){ 327 | for remoteURI, expectedInstalledDir := range filesToInstall { 328 | 329 | installedDir, err := fs.Install(remoteURI, testInstalledDir, false) 330 | 331 | if err != nil { 332 | panic(err) 333 | } 334 | println("Installed: "+installedDir) 335 | } 336 | } 337 | ``` 338 | 339 | Read the [installer_test.go](https://github.com/kataras/go-fs/blob/master/installer_test.go) for more. 340 | 341 | You do not need any other special explanations for this package, just navigate to the [godoc](https://godoc.org/github.com/kataras/go-fs) or the [source](https://github.com/kataras/go-fs/blob/master/http_test.go) [code](https://github.com/kataras/go-fs/blob/master/installer_test.go). 342 | 343 | ## FAQ 344 | 345 | Explore [these questions](https://github.com/kataras/go-fs/issues?go-fs=label%3Aquestion) or navigate to the [community chat][Chat]. 346 | 347 | ## Versioning 348 | 349 | Current: **v0.0.6** 350 | 351 | ## People 352 | 353 | The author of go-fs is [@kataras](https://github.com/kataras). 354 | 355 | If you're **willing to donate**, feel free to send **any** amount through paypal or stripe: https://iris-go.com/donate. 356 | 357 | ## Contributing 358 | 359 | If you are interested in contributing to the go-fs project, please make a PR. 360 | 361 | ## License 362 | 363 | This project is licensed under the MIT License. 364 | 365 | License can be found [here](LICENSE). 366 | 367 | [Travis Widget]: https://img.shields.io/travis/kataras/go-fs.svg?style=flat-square 368 | [Travis]: http://travis-ci.org/kataras/go-fs 369 | [License Widget]: https://img.shields.io/badge/license-MIT%20%20License%20-E91E63.svg?style=flat-square 370 | [License]: https://github.com/kataras/go-fs/blob/master/LICENSE 371 | [Release Widget]: https://img.shields.io/badge/release-v0.0.6-blue.svg?style=flat-square 372 | [Release]: https://github.com/kataras/go-fs/releases 373 | [Chat Widget]: https://img.shields.io/gitter/room/go-fs/community.svg?color=cc2b5e&logo=gitter&style=flat-square 374 | [Chat]: https://gitter.im/kataras/go-fs 375 | [ChatMain]: https://gitter.im/kataras/go-fs 376 | [Report Widget]: https://img.shields.io/badge/report%20card-A%2B-F44336.svg?style=flat-square 377 | [Report]: http://goreportcard.com/report/kataras/go-fs 378 | [Documentation Widget]: https://img.shields.io/badge/documentation-reference-5272B4.svg?style=flat-square 379 | [Documentation]: https://www.gitbook.com/book/kataras/go-fs/details 380 | [Language Widget]: https://img.shields.io/badge/powered_by-Go-3362c2.svg?style=flat-square 381 | [Language]: http://golang.org 382 | [Platform Widget]: https://img.shields.io/badge/platform-Any--OS-gray.svg?style=flat-square 383 | --------------------------------------------------------------------------------