├── LICENSE ├── PKGBUILD ├── README.md ├── go.mod ├── go.sum ├── megatools.PKGBUILD └── mego.go /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Lopez Benjamin 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /PKGBUILD: -------------------------------------------------------------------------------- 1 | # Maintainer: Benjamin Lopez 2 | 3 | pkgname=mego 4 | pkgver=1.3.0 5 | pkgrel=2 6 | pkgdesc="A simple megadl wrapper with auto-retry and download list" 7 | arch=('x86_64') 8 | url="https://github.com/scotow/${pkgname}" 9 | license=('MIT') 10 | depends=('megatools') 11 | makedepends=('go' 'git') 12 | source=("${pkgname}-${pkgver}.tar.gz::https://github.com/scotow/${pkgname}/archive/${pkgver}.tar.gz") 13 | sha256sums=('e06df26dd1a0cfc78793f35c925da69225b8fcb3b061d4c65816616e2e861595') 14 | 15 | prepare(){ 16 | mkdir -p src/github.com/scotow 17 | ln -rTsf "${pkgname}-${pkgver}" "src/github.com/scotow/${pkgname}" 18 | } 19 | 20 | build(){ 21 | export GOPATH="${srcdir}" 22 | cd "src/github.com/scotow/${pkgname}" 23 | go get -v ./... 24 | go install \ 25 | -gcflags "all=-trimpath=${GOPATH}/src" \ 26 | -asmflags "all=-trimpath=${GOPATH}/src" \ 27 | -ldflags "-extldflags ${LDFLAGS}" \ 28 | ./... 29 | } 30 | 31 | package(){ 32 | install -Dm755 "bin/${pkgname}" "${pkgdir}/usr/bin/${pkgname}" 33 | 34 | cd "${pkgname}-${pkgver}" 35 | install -Dm644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" 36 | } 37 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mego 💾 2 | 3 | Mego is a simple [megatools](https://megatools.megous.com) command wrapper, allowing you to use the `megatools dl` command with a list of links and adds an auto-try tool. 4 | 5 | ### Idea 6 | 7 | Megatools is a collection of programs used to access [mega.nz](https://mega.nz) services from the command line. 8 | 9 | #### Auto-retry 10 | 11 | While using the `megatools dl` command to download a bunch of large files, I often found myself being blocked by Mega for reaching the allowed quota (aka. error 509). 12 | 13 | Indeed, Mega allows users to download a few GB per day (apparently not fixed). 14 | 15 | By default `megatools dl` only retries 3 times when this error occurred, preventing the download of file during the night or while being away from the computer. To fix this problem, `mego` checks the error code returned by `megatools dl`, and retry if the command failed. 16 | 17 | #### List of files 18 | 19 | Another problem that I found while using `megatools dl` is the lack of options to download multiple files at once, and keeping track of the completed ones. 20 | 21 | To solve this problem, `mego` accepts as arguments a path of file(s) containing a list of Mega download links. `mego` will open the file and start downloading the files listed in it. Once the download of the file successfully terminated, `mego` will add a `#` before the link and write it in the file, preventing the next execution to re-download the file. `mego` will also mark invalid links it found with the `#-` string. 22 | 23 | #### Compatibility 24 | 25 | Because this script is a wrapper around the `megatools` command, it heavily depends on the outputs of the command. If you have problems using this script, be sure to use the version 1.11.0 (04.04.20) of `megatools`. You can download the latest version [here](https://megatools.megous.com/builds/experimental/) or use the [megatools.PKGBUILD](https://github.com/scotow/mego/blob/master/megatools.PKGBUILD) file available in this repo. 26 | 27 | ### Usage 28 | 29 | ``` 30 | Usage of mego: 31 | 32 | mego [-c COMMAND_PATH] [-s SPEED] [-p] [-r INTERVAL] LINK... LINK_PATH... 33 | 34 | Application Options: 35 | -s, --speed-limit=SPEED Speed limit passed to megatools dl as --limit-speed (default: 0) 36 | -p, --pipe-outputs Pipe megatools's stdout and stderr 37 | -r, --retry=INTERVAL Interval between two retries (default: 15min) 38 | -c, --command=COMMAND_PATH Path to the megatools command (default: megatools) 39 | ``` 40 | 41 | NB: The whole content of a list file is read and kept in memory. Every time a file is downloaded, the content of the list file will be overwritten. So please do not use a list file as a queue during execution. 42 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/scotow/mego 2 | 3 | go 1.12 4 | 5 | require github.com/jessevdk/go-flags v1.4.0 6 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/jessevdk/go-flags v1.4.0 h1:4IU2WS7AumrZ/40jfhf4QVDMsQwqA7VEHozFRrGARJA= 2 | github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= 3 | -------------------------------------------------------------------------------- /megatools.PKGBUILD: -------------------------------------------------------------------------------- 1 | # Maintainer: Benjamin Lopez 2 | 3 | pkgname=megatools 4 | pkgver=1.11.0 5 | pkgrel=20200404 6 | pkgdesc="Command line client application for Mega.nz" 7 | arch=('x86_64') 8 | url="http://megatools.megous.com" 9 | license=('GPL') 10 | depends=('curl' 'glib2' 'openssl') 11 | makedepends=('asciidoc') 12 | _archive="megatools-${pkgver}-git-${pkgrel}-linux-x86_64" 13 | source=("https://megatools.megous.com/builds/experimental/${_archive}.tar.gz") 14 | options=(!libtool) 15 | sha256sums=('e2795b6126ff9401a830dbae0006dbb77c48fd5dab788c8f1ffd1efc42db2f54') 16 | 17 | package() { 18 | cd "${_archive}" 19 | 20 | # Bin 21 | install -Dm755 "${pkgname}" "${pkgdir}/usr/bin/${pkgname}" 22 | 23 | # License 24 | install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" 25 | } 26 | -------------------------------------------------------------------------------- /mego.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "io" 7 | "io/ioutil" 8 | "log" 9 | "os" 10 | "os/exec" 11 | "regexp" 12 | "strings" 13 | "time" 14 | 15 | "github.com/jessevdk/go-flags" 16 | ) 17 | 18 | type options struct { 19 | Speed uint `short:"s" long:"speed-limit" description:"Speed limit passed to megatools dl as --limit-speed" default:"0" value-name:"SPEED"` 20 | Pipe bool `short:"p" long:"pipe-outputs" description:"Pipe megatools's stdout and stderr"` 21 | Retry time.Duration `short:"r" long:"retry" description:"Interval between two retries" default:"15min" value-name:"INTERVAL"` 22 | CommandPath string `short:"c" long:"command" description:"Path to the megatools command" default:"megatools" value-name:"COMMAND_PATH"` 23 | } 24 | 25 | var ( 26 | opts options 27 | linkRegex = regexp.MustCompile(`^(?:https?://)?mega\.nz/(?:(?:file|folder).+)?#.+$`) 28 | ) 29 | 30 | var ( 31 | outLogger = log.New(os.Stdout, "", log.LstdFlags) 32 | errLogger = log.New(os.Stderr, "", log.LstdFlags) 33 | ) 34 | 35 | func isValidLink(link string) bool { 36 | return linkRegex.MatchString(link) 37 | } 38 | 39 | func isAlreadyDownloadedError(line, link string) bool { 40 | if strings.HasPrefix(line, "ERROR: File already exists at ") { 41 | return true 42 | } 43 | // Typo in the original program. 44 | if strings.HasPrefix(line, fmt.Sprintf("ERROR: Download failed for '%s': Local file already exists:", link)) { 45 | return true 46 | } 47 | return false 48 | } 49 | 50 | func downloadRepeat(link string) { 51 | for !downloadCommand(link) { 52 | errLogger.Printf("Download of \"%s\" failed, waiting %s before retrying.\n", link, opts.Retry.String()) 53 | time.Sleep(opts.Retry) 54 | } 55 | 56 | outLogger.Printf("Download of \"%s\" done.\n", link) 57 | } 58 | 59 | func downloadCommand(link string) bool { 60 | cmd := exec.Command(opts.CommandPath, "dl", fmt.Sprintf("--limit-speed=%d", opts.Speed), link) 61 | 62 | var errBuff bytes.Buffer 63 | if opts.Pipe { 64 | cmd.Stdout = os.Stdout 65 | cmd.Stderr = io.MultiWriter(os.Stderr, &errBuff) 66 | } else { 67 | cmd.Stderr = &errBuff 68 | } 69 | 70 | err := cmd.Run() 71 | if err != nil { 72 | logLines := strings.Split(errBuff.String(), "\n") 73 | 74 | for _, line := range logLines { 75 | line = strings.TrimSpace(line) 76 | if len(line) == 0 { 77 | continue 78 | } 79 | if !isAlreadyDownloadedError(line, link) { 80 | return false 81 | } 82 | } 83 | } 84 | 85 | return true 86 | } 87 | 88 | func downloadFromFilesList(path string) { 89 | data, err := ioutil.ReadFile(path) 90 | if err != nil { 91 | errLogger.Printf("Cannot open file \"%s\". Skipping. (%s)\n", path, err.Error()) 92 | return 93 | } 94 | 95 | lines := strings.Split(string(data), "\n") 96 | links := make([]string, 0, len(lines)) 97 | 98 | // Parsing links in file. 99 | for _, line := range lines { 100 | line = strings.TrimSpace(line) 101 | if len(line) == 0 { 102 | continue 103 | } 104 | 105 | links = append(links, line) 106 | } 107 | 108 | // Download each links in list. 109 | for i, link := range links { 110 | if link[0] == '#' { 111 | errLogger.Printf("Skipping \"%s\".\n", link[1:]) 112 | continue 113 | } 114 | 115 | if !isValidLink(link) { 116 | errLogger.Printf("Invalid link %s. Skipping.\n", link) 117 | links[i] = fmt.Sprintf("#-%s", link) 118 | writeFilesList(path, links) 119 | continue 120 | } 121 | 122 | downloadRepeat(link) 123 | links[i] = fmt.Sprintf("#%s", link) 124 | writeFilesList(path, links) 125 | } 126 | } 127 | 128 | func writeFilesList(path string, links []string) { 129 | err := ioutil.WriteFile(path, []byte(strings.Join(links, "\n")), 0664) 130 | if err != nil { 131 | errLogger.Printf("Cannot write file \"%s\". (%s)\n", path, err.Error()) 132 | } 133 | } 134 | 135 | func commandExists(cmd string) bool { 136 | _, err := exec.LookPath(cmd) 137 | return err == nil 138 | } 139 | 140 | func main() { 141 | parser := flags.NewParser(&opts, flags.Default) 142 | parser.Usage = "[-c COMMAND_PATH] [-s SPEED] [-p] [-r INTERVAL] LINK... LINK_PATH..." 143 | 144 | args, err := parser.Parse() 145 | if err != nil { 146 | if flagsErr, ok := err.(*flags.Error); ok && flagsErr.Type == flags.ErrHelp { 147 | os.Exit(0) 148 | } else { 149 | os.Exit(1) 150 | } 151 | } 152 | 153 | if len(args) == 0 { 154 | parser.WriteHelp(os.Stdout) 155 | os.Exit(1) 156 | } 157 | 158 | if !commandExists(opts.CommandPath) { 159 | errLogger.Printf("Cannot find command \"%s\".\n", opts.CommandPath) 160 | os.Exit(1) 161 | } 162 | 163 | for _, arg := range args { 164 | if isValidLink(arg) { 165 | downloadRepeat(arg) 166 | } else { 167 | downloadFromFilesList(arg) 168 | } 169 | } 170 | 171 | outLogger.Println("All download(s) done.") 172 | } 173 | --------------------------------------------------------------------------------