├── LICENSE ├── README.md ├── cmd ├── genpkgup │ └── main.go └── obsdpkgup │ └── main.go ├── go.mod ├── go.sum ├── gzip └── gunzip.go ├── openbsd ├── index.go ├── quirks.go ├── signify.go ├── updatesignature.go └── version │ ├── dewey.go │ ├── version.go │ └── version_test.go └── update_indexes_go.sh /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Jeremy O'Brien 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # obsdpkgup 2 | 3 | ``` 4 | x1$ time obsdpkgup 5 | up to date 6 | 0m01.69s real 0m00.51s user 0m00.25s system 7 | ``` 8 | 9 | ## Pre-requisites 10 | 11 | - Go: `pkg_add go` 12 | 13 | ## Installation 14 | 15 | `go install github.com/neutralinsomniac/obsdpkgup/cmd/obsdpkgup@latest` 16 | 17 | ## Usage 18 | 19 | ### Check for upgrades using signatures: 20 | 21 | (see [The Same-Version Rebuild Problem](https://github.com/neutralinsomniac/obsdpkgup#the-same-version-rebuild-problem)): 22 | 23 | `PKGUP_URL=https://pintobyte.com/pkgup/ obsdpkgup` 24 | 25 | ### Force checking snapshot directory for upgrades: 26 | `obsdpkgup -s` 27 | 28 | ### Run and apply found package upgrades: 29 | `obsdpkgup |doas sh` 30 | 31 | ### Cron mode (don't output anything when packages are up-to-date): 32 | `obsdpkgup -c` 33 | 34 | ## Rationale 35 | 36 | OpenBSD's package tools are great. They've been battle-tested and designed to 37 | correctly handle package installation/upgrades even in the face of uncertain 38 | mirror conditions. They suffer from one problem however: with no central 39 | package index, in order to calculate an upgrade, the signature of *every* 40 | installed package must be checked against its signature on the configured 41 | mirror. This process can take anywhere from 1 to over 30 minutes to complete 42 | and uses a non-trivial amount of bandwidth, even if no updates are actually 43 | available. A simple solution to this would be to run package upgrades overnight 44 | through cron or some other mechanism, but not all OpenBSD machines are 45 | always-on computers, laptops being the prime example of this. 46 | 47 | **obsdpkgup** attempts to solve the slow `pkg_add -u` dilemma while maintaining 48 | the consistency safeguards built into the pkgtools. 49 | 50 | ## Theory of Operation 51 | 52 | The lack of a central-index in the packaging system is an intentional decision. 53 | When an upgrade is requested while a mirror is in a partially-synced state, any 54 | central index may inevitably be out-of-sync with the state of the files 55 | present. If this index is solely relied on as a source of truth, then Bad 56 | Things can happen when an upgrade is attempted. 57 | 58 | Because a central index may not accurately reflect the current state of a 59 | mirror, **obsdpkgup** uses its own index to seed a `pkg_add -u` call with a 60 | subset of packages it *thinks* have upgrades (with version numbers removed, to 61 | allow for flexibility in the mirror state). This prevents `pkg_add` from having 62 | to check every installed package individually, and instead enables it to check 63 | a much smaller list of potential upgrade candidates. This way, `pkg_add` will 64 | either do what it would've done with a full `pkg_add -u` call anyway (but with 65 | a much smaller list of packages to check), or in the case of a de-synced index, 66 | maybe report that a subset of the package candidate(s) are up-to-date, which 67 | will have no adverse effect on the system. 68 | 69 | ## The Same-Version Rebuild Problem 70 | 71 | Or, why can't we just compare version numbers in the mirror index? 72 | 73 | Currently, the only index-like file available for **obsdpkgup** to check is a 74 | mirror's `index.txt` file. This file contains the external-facing version 75 | numbers of the packages available on the mirror, but does not reveal the 76 | internal openbsd-specific version that can be incremented in certain 77 | situations. When packages are rebuilt, and their external-facing versions 78 | aren't incremented, **obsdpkgup** can't tell that an upgrade occurred. Thus, a 79 | new index file format was created that contains a hash of a package's 80 | "signature" (the same thing that the pkgtools themselves check) and stores it 81 | in a secondary index file (called index.pkgup.gz). This file can be easily 82 | generated from an existing mirror using the `genpkgup` command included in this repo. 83 | -------------------------------------------------------------------------------- /cmd/genpkgup/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | tar2 "archive/tar" 5 | "flag" 6 | "fmt" 7 | "github.com/neutralinsomniac/obsdpkgup/gzip" 8 | "github.com/neutralinsomniac/obsdpkgup/openbsd" 9 | "io/ioutil" 10 | "net/http" 11 | "os" 12 | "regexp" 13 | "strings" 14 | ) 15 | 16 | func getContentsFromPkgUrl(url string) []byte { 17 | resp, err := http.Get(url) 18 | if err != nil { 19 | fmt.Fprintf(os.Stderr, "Error downloading package %s: %s\n", url, err) 20 | goto Error 21 | } 22 | 23 | defer resp.Body.Close() 24 | 25 | switch resp.StatusCode { 26 | case 200: 27 | gz, err := gzip.NewReader(resp.Body) 28 | if err != nil { 29 | fmt.Fprintf(os.Stderr, "Error decompressing %s: %s\n", url, err) 30 | goto Error 31 | } 32 | 33 | tar := tar2.NewReader(gz) 34 | 35 | hdr, err := tar.Next() 36 | if err != nil { 37 | fmt.Fprintf(os.Stderr, "Error decompressing %s: %s\n", url, err) 38 | goto Error 39 | } 40 | for err == nil && hdr.Name != "+CONTENTS" { 41 | hdr, err = tar.Next() 42 | if err != nil { 43 | fmt.Fprintf(os.Stderr, "Error walking archive %s: %s\n", url, err) 44 | goto Error 45 | } 46 | } 47 | contents, _ := ioutil.ReadAll(tar) 48 | return contents 49 | case 404: 50 | fmt.Fprintf(os.Stderr, "404 while downloading package: %s\n", url) 51 | goto Error 52 | default: 53 | fmt.Fprintf(os.Stderr, "unexpected HTTP response (%d) while downloading package: %s\n", resp.StatusCode, url) 54 | goto Error 55 | } 56 | 57 | Error: 58 | return []byte{} 59 | } 60 | 61 | var mirror string 62 | var arch string 63 | var version string 64 | var showProgress bool 65 | 66 | var indexFormatVersion = 1 67 | 68 | var pkgpathRe = regexp.MustCompilePOSIX(`^@comment pkgpath=([^ ]+).*$`) 69 | 70 | func main() { 71 | flag.StringVar(&mirror, "m", "https://cdn.openbsd.org/pub/OpenBSD", "Mirror URL") 72 | flag.StringVar(&arch, "a", "", "Architecture") 73 | flag.StringVar(&version, "v", "", "Version") 74 | flag.BoolVar(&showProgress, "p", false, "Show progress") 75 | 76 | flag.Parse() 77 | 78 | if version == "" { 79 | fmt.Fprintf(os.Stderr, "Error: Must specify version (-v)\n") 80 | os.Exit(1) 81 | } 82 | 83 | if arch == "" { 84 | fmt.Fprintf(os.Stderr, "Error: Must specify arch (-a)\n") 85 | os.Exit(1) 86 | } 87 | 88 | var url string 89 | if version == "snapshots" { 90 | url = fmt.Sprintf("%s/%s/packages/%s/", mirror, version, arch) 91 | } else { 92 | url = fmt.Sprintf("%s/%s/packages-stable/%s/", mirror, version, arch) 93 | } 94 | 95 | // retrieve the index.txt first 96 | indexString, err := openbsd.GetIndexTxt(url) 97 | if err != nil { 98 | fmt.Fprintf(os.Stderr, "failed to retrieve index.txt at %s: %s\n", url, err) 99 | os.Exit(1) 100 | } 101 | 102 | // snag quirks for timestamp 103 | quirksSignifyBlock, err := openbsd.GetQuirksSignifyBlockFromIndex(url, indexString) 104 | if err != nil { 105 | fmt.Fprintf(os.Stderr, "%s\n", err) 106 | os.Exit(1) 107 | } 108 | quirksDate, err := openbsd.GetSignifyTimestampFromSignifyBlock(quirksSignifyBlock) 109 | if err != nil { 110 | fmt.Fprintf(os.Stderr, "%s\n", err) 111 | os.Exit(1) 112 | } 113 | 114 | // write index format version 115 | fmt.Println(indexFormatVersion) 116 | 117 | // write quirks date 118 | fmt.Println(quirksDate) 119 | 120 | lines := strings.Split(indexString, "\n") 121 | numPkgsToProcess := len(lines) 122 | for i, line := range lines { 123 | if showProgress { 124 | fmt.Fprintf(os.Stderr, "\r%d/%d", i, numPkgsToProcess) 125 | } 126 | if len(line) == 0 { 127 | continue 128 | } 129 | s := strings.Fields(line) 130 | pkgName := s[9] 131 | // doesn't look like a package to me 132 | if !strings.HasSuffix(pkgName, ".tgz") { 133 | continue 134 | } 135 | contents := getContentsFromPkgUrl(fmt.Sprintf("%s/%s", url, pkgName)) 136 | if len(contents) == 0 { 137 | // if we failed to get/decompress +CONTENTS, skip this package 138 | continue 139 | } 140 | 141 | signature := openbsd.GenerateSignatureFromContents(contents) 142 | 143 | pkgPath := pkgpathRe.FindSubmatch(contents)[1] 144 | 145 | fmt.Printf("%s %s %s\n", pkgName, signature, pkgPath) 146 | } 147 | } 148 | -------------------------------------------------------------------------------- /cmd/obsdpkgup/main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "compress/gzip" 6 | "flag" 7 | "fmt" 8 | "io/ioutil" 9 | "net/http" 10 | "os" 11 | "os/exec" 12 | "regexp" 13 | "sort" 14 | "strconv" 15 | "strings" 16 | "time" 17 | 18 | "github.com/neutralinsomniac/obsdpkgup/openbsd" 19 | version2 "github.com/neutralinsomniac/obsdpkgup/openbsd/version" 20 | 21 | "suah.dev/protect" 22 | ) 23 | 24 | // PkgVer represents an individual entry in our package index 25 | type PkgVer struct { 26 | name string 27 | fullName string 28 | version version2.Version 29 | flavor string 30 | signature string 31 | pkgpath string 32 | isBranch bool 33 | } 34 | 35 | func (p PkgVer) Equals(o PkgVer) bool { 36 | if p.signature != o.signature { 37 | return false 38 | } 39 | 40 | return true 41 | } 42 | 43 | func (p PkgVer) String() string { 44 | return p.fullName 45 | } 46 | 47 | // PkgList maps a package name to a list of PkgVer's 48 | type PkgList map[string][]PkgVer 49 | 50 | func checkAndExit(e error) { 51 | if e != nil { 52 | fmt.Fprintf(os.Stderr, "%s\n", e) 53 | os.Exit(1) 54 | } 55 | } 56 | 57 | var numRE = regexp.MustCompile(`^[0-9.]+.*$`) 58 | 59 | // returns: pkgVer struct, error 60 | func NewPkgVerFromString(pkgStr string) (PkgVer, error) { 61 | pkgFileSlice := strings.Split(pkgStr, "-") 62 | // pkgFileSlice: "[x, y, 1.2.3p4, flavor1, flavor2]" 63 | // walk backwards until we find the version 64 | for i := len(pkgFileSlice) - 1; i >= 0; i-- { 65 | // found version! 66 | if numRE.MatchString(pkgFileSlice[i]) { 67 | version := version2.NewVersionFromString(pkgFileSlice[i]) 68 | flavor := "" 69 | if len(pkgFileSlice[i:]) > 1 { 70 | flavor = strings.Join(pkgFileSlice[i+1:], "-") 71 | } 72 | return PkgVer{ 73 | fullName: pkgStr, 74 | version: version, 75 | flavor: flavor, 76 | name: strings.Join(pkgFileSlice[:i], "-"), 77 | }, nil 78 | } 79 | } 80 | return PkgVer{}, fmt.Errorf("couldn't find version in pkg: %q\n", pkgStr) 81 | } 82 | 83 | func parseLocalPkgInfoToPkgList() PkgList { 84 | pkgList := make(PkgList) 85 | 86 | pkgDbPath := "/var/db/pkg/" 87 | files, err := ioutil.ReadDir(pkgDbPath) 88 | checkAndExit(err) 89 | 90 | pkgpathRe := regexp.MustCompilePOSIX(`^@comment pkgpath=([^ ]+).*$`) 91 | isBranchRe := regexp.MustCompilePOSIX(`^@option is-branch$`) 92 | 93 | for _, file := range files { 94 | pkgdir := file.Name() 95 | pkgVer, err := NewPkgVerFromString(pkgdir) 96 | checkAndExit(err) 97 | 98 | f, err := os.Open(fmt.Sprintf("%s%s/+CONTENTS", pkgDbPath, pkgdir)) 99 | checkAndExit(err) 100 | 101 | contents, err := ioutil.ReadAll(f) 102 | checkAndExit(err) 103 | 104 | f.Close() 105 | 106 | pkgVer.signature = openbsd.GenerateSignatureFromContents(contents) 107 | pkgpath := pkgpathRe.FindSubmatch(contents)[1] 108 | pkgVer.pkgpath = string(pkgpath) 109 | if isBranchRe.Match(contents) { 110 | pkgVer.isBranch = true 111 | } 112 | 113 | pkgList[pkgVer.name] = append(pkgList[pkgVer.name], pkgVer) 114 | } 115 | return pkgList 116 | } 117 | 118 | func parseObsdPkgUpList(pkgup string) PkgList { 119 | pkgList := make(PkgList) 120 | 121 | for _, line := range strings.Split(pkgup, "\n") { 122 | if len(line) > 1 { 123 | tmp := strings.Fields(line) 124 | pkgFile := tmp[0] 125 | if !strings.HasSuffix(pkgFile, ".tgz") { 126 | continue 127 | } 128 | signature := tmp[1] 129 | pkgpath := tmp[2] 130 | pkgVer, err := NewPkgVerFromString(pkgFile[:len(pkgFile)-4]) 131 | checkAndExit(err) 132 | pkgVer.signature = signature 133 | pkgVer.pkgpath = pkgpath 134 | pkgList[pkgVer.name] = append(pkgList[pkgVer.name], pkgVer) 135 | } 136 | } 137 | 138 | return pkgList 139 | } 140 | 141 | type SysInfo struct { 142 | arch string 143 | version string 144 | snapshot bool 145 | } 146 | 147 | func getSystemInfo() SysInfo { 148 | var sysInfo SysInfo 149 | 150 | cmd := exec.Command("sysctl", "-n", "kern.version") 151 | output, err := cmd.Output() 152 | checkAndExit(err) 153 | 154 | if strings.Contains(string(output), "-current") || strings.Contains(string(output), "-beta") || forceSnapshot { 155 | sysInfo.snapshot = true 156 | } 157 | 158 | sysInfo.version = string(output[8:11]) 159 | 160 | cmd = exec.Command("arch", "-s") 161 | output, err = cmd.Output() 162 | checkAndExit(err) 163 | 164 | sysInfo.arch = strings.TrimSpace(string(output)) 165 | 166 | return sysInfo 167 | } 168 | 169 | var protocolRe = regexp.MustCompile(`://`) 170 | var repeatingSlashRe = regexp.MustCompile(`/+`) 171 | 172 | func replaceMirrorVars(mirror string, sysInfo SysInfo) string { 173 | mirror = strings.ReplaceAll(mirror, "%m", "/pub/OpenBSD/%c/packages/%a/") 174 | 175 | if sysInfo.snapshot { 176 | mirror = strings.ReplaceAll(mirror, "%c", "snapshots") 177 | } else { 178 | mirror = strings.ReplaceAll(mirror, "%c/packages", "%c/packages-stable") 179 | mirror = strings.ReplaceAll(mirror, "%c", sysInfo.version) 180 | } 181 | 182 | mirror = strings.ReplaceAll(mirror, "%a", sysInfo.arch) 183 | mirror = strings.ReplaceAll(mirror, "%v", sysInfo.version) 184 | 185 | // strip duplicate /'s to work around a bug on some of the mirrors 186 | s := protocolRe.Split(mirror, -1) 187 | s[1] = repeatingSlashRe.ReplaceAllString(s[1], "/") 188 | 189 | mirror = strings.Join(s, "://") 190 | 191 | return mirror 192 | } 193 | 194 | func getMirror() string { 195 | sysInfo := getSystemInfo() 196 | 197 | // TRUSTED_PKG_PATH env var is tested first 198 | trustedPkgPath := os.Getenv("TRUSTED_PKG_PATH") 199 | if trustedPkgPath != "" { 200 | return replaceMirrorVars(trustedPkgPath, sysInfo) 201 | } 202 | 203 | // PKG_PATH is tested next 204 | pkgPath := os.Getenv("PKG_PATH") 205 | if pkgPath != "" { 206 | return replaceMirrorVars(pkgPath, sysInfo) 207 | } 208 | 209 | // next, try /etc/installurl 210 | installurlBytes, err := ioutil.ReadFile("/etc/installurl") 211 | if err == nil { 212 | installurl := strings.TrimSpace(string(installurlBytes)) 213 | return replaceMirrorVars(fmt.Sprintf("%s/%%c/packages/%%a/", installurl), sysInfo) 214 | } 215 | 216 | // finally, fall back to cdn 217 | return replaceMirrorVars("https://cdn.openbsd.org/pub/OpenBSD/%c/packages/%a/", sysInfo) 218 | } 219 | 220 | var pkgpathVersionRE = regexp.MustCompile(`^.*/.*/([^ ,]+).*$`) 221 | 222 | var cronMode bool 223 | var forceSnapshot bool 224 | var verbose bool 225 | var debug bool 226 | 227 | var currentIndexFormatVersion = 1 228 | 229 | func main() { 230 | start := time.Now() 231 | _ = protect.Pledge("stdio unveil rpath wpath cpath flock dns inet tty proc exec") 232 | _ = protect.Unveil("/etc/resolv.conf", "r") 233 | _ = protect.Unveil("/etc/installurl", "r") 234 | _ = protect.Unveil("/etc/ssl/cert.pem", "r") 235 | _ = protect.Unveil("/sbin/sysctl", "rx") 236 | _ = protect.Unveil("/usr/bin/arch", "rx") 237 | _ = protect.Unveil("/bin/ls", "rx") 238 | _ = protect.Unveil("/var/db/pkg", "r") 239 | 240 | flag.BoolVar(&cronMode, "c", false, "Cron mode (only output when updates are available)") 241 | flag.BoolVar(&forceSnapshot, "s", false, "Force checking snapshot directory for upgrades") 242 | flag.BoolVar(&verbose, "v", false, "Show verbose logging information") 243 | flag.BoolVar(&debug, "d", false, "Show debug logging information") 244 | 245 | flag.Parse() 246 | 247 | var err error 248 | 249 | updateList := make(map[string]bool) // this is used as a set 250 | 251 | mirror := getMirror() 252 | 253 | var allPkgs PkgList 254 | var sysInfo SysInfo 255 | 256 | sysInfo = getSystemInfo() 257 | 258 | pkgUpBaseUrl := os.Getenv("PKGUP_URL") 259 | var resp *http.Response 260 | var pkgUpIndexUrl string 261 | if pkgUpBaseUrl != "" { 262 | pkgUpIndexUrl = replaceMirrorVars(fmt.Sprintf("%s/%%c/%%a/index.pkgup.gz", pkgUpBaseUrl), sysInfo) 263 | } else { 264 | pkgUpIndexUrl = fmt.Sprintf("%s/index.pkgup.gz", mirror) 265 | } 266 | 267 | // grab pkgup index 268 | var pkgUpQuirksDateString string 269 | resp, err = http.Get(pkgUpIndexUrl) 270 | checkAndExit(err) 271 | 272 | var pkgUpBytes []byte 273 | switch resp.StatusCode { 274 | case 200: 275 | // grab body 276 | r, err := gzip.NewReader(resp.Body) 277 | checkAndExit(err) 278 | pkgUpBytes, err = ioutil.ReadAll(r) 279 | checkAndExit(err) 280 | resp.Body.Close() 281 | case 404: 282 | fmt.Fprintf(os.Stderr, "unable to locate pkgup index at '%s'.\n", pkgUpIndexUrl) 283 | os.Exit(1) 284 | default: 285 | fmt.Fprintf(os.Stderr, "unexpected HTTP response: %d\n", resp.StatusCode) 286 | os.Exit(1) 287 | } 288 | 289 | // get + check version 290 | indexFormatVersionEndIndex := bytes.IndexByte(pkgUpBytes, '\n') 291 | indexFormatVersionStr := string(pkgUpBytes[:indexFormatVersionEndIndex]) 292 | pkgUpBytes = pkgUpBytes[indexFormatVersionEndIndex+1:] 293 | indexFormatVersion, err := strconv.Atoi(indexFormatVersionStr) 294 | if err != nil { 295 | fmt.Fprintf(os.Stderr, "expected index version %d, got: %s\n", currentIndexFormatVersion, indexFormatVersionStr) 296 | os.Exit(1) 297 | } 298 | 299 | if indexFormatVersion != currentIndexFormatVersion { 300 | fmt.Fprintf(os.Stderr, "expected index version %d, got: %d\n", currentIndexFormatVersion, indexFormatVersion) 301 | if currentIndexFormatVersion < indexFormatVersion { 302 | fmt.Fprintf(os.Stderr, "please update obsdpkgup and try again\n") 303 | } else { 304 | fmt.Fprintf(os.Stderr, "wait for remote mirror to update to the current pkgup index format and try again later\n") 305 | } 306 | 307 | os.Exit(1) 308 | } 309 | 310 | // get quirks timestamp from pkgUp 311 | quirksEndIndex := bytes.IndexByte(pkgUpBytes, '\n') 312 | pkgUpQuirksDateString = string(pkgUpBytes[:quirksEndIndex]) 313 | pkgUpBytes = pkgUpBytes[quirksEndIndex+1:] 314 | 315 | // now parse the actual package list 316 | allPkgs = parseObsdPkgUpList(string(pkgUpBytes)) 317 | 318 | // grab mirror quirks 319 | indexString, err := openbsd.GetIndexTxt(mirror) 320 | if err != nil { 321 | fmt.Fprintf(os.Stderr, "%s\n", err) 322 | os.Exit(1) 323 | } 324 | mirrorQuirksSignifyBlock, err := openbsd.GetQuirksSignifyBlockFromIndex(mirror, indexString) 325 | if err != nil { 326 | fmt.Fprintf(os.Stderr, "%s\n", err) 327 | os.Exit(1) 328 | } 329 | 330 | // and parse the quirks date 331 | mirrorQuirksDateString, err := openbsd.GetSignifyTimestampFromSignifyBlock(mirrorQuirksSignifyBlock) 332 | if err != nil { 333 | fmt.Fprintf(os.Stderr, "%s\n", err) 334 | os.Exit(1) 335 | } 336 | 337 | pkgUpQuirksDate, err := time.Parse(openbsd.SignifyTimeFormat, pkgUpQuirksDateString) 338 | if err != nil { 339 | fmt.Fprintf(os.Stderr, "error parsing pkgUp quirks date %s: %s\n", pkgUpQuirksDateString, err) 340 | os.Exit(1) 341 | } 342 | 343 | mirrorQuirksDate, err := time.Parse(openbsd.SignifyTimeFormat, mirrorQuirksDateString) 344 | if err != nil { 345 | fmt.Fprintf(os.Stderr, "error parsing mirror quirks date %s: %s\n", mirrorQuirksDateString, err) 346 | os.Exit(1) 347 | } 348 | 349 | if verbose { 350 | fmt.Fprintf(os.Stderr, "network took: %f seconds\n", float64(time.Now().Sub(start))/float64(time.Second)) 351 | start = time.Now() 352 | } 353 | 354 | installedPkgs := parseLocalPkgInfoToPkgList() 355 | var sortedInstalledPkgs []string 356 | for k := range installedPkgs { 357 | sortedInstalledPkgs = append(sortedInstalledPkgs, k) 358 | } 359 | sort.Strings(sortedInstalledPkgs) 360 | 361 | for _, name := range sortedInstalledPkgs { 362 | // quirks is treated specially; don't ever try to manually update it 363 | if name == "quirks" { 364 | continue 365 | } 366 | 367 | // if package name doesn't exist in remote, skip it 368 | if _, ok := allPkgs[name]; !ok { 369 | continue 370 | } 371 | 372 | installedVersions := installedPkgs[name] 373 | 374 | // check all versions to find upgrades 375 | for _, installedVersion := range installedVersions { 376 | bestVersionMatch := installedVersion 377 | NEXTVERSION: 378 | for _, remoteVersion := range allPkgs[name] { 379 | // verify flavor/pkgpath match first 380 | if remoteVersion.flavor != installedVersion.flavor || remoteVersion.pkgpath != installedVersion.pkgpath { 381 | continue NEXTVERSION 382 | } 383 | 384 | // check for version bump 385 | var versionComparisonResult = bestVersionMatch.version.Compare(remoteVersion.version) 386 | if versionComparisonResult == -1 { 387 | bestVersionMatch = remoteVersion 388 | } else if versionComparisonResult == 0 && installedVersion.signature != remoteVersion.signature { 389 | // check for same-version/different signature 390 | bestVersionMatch = remoteVersion 391 | } 392 | } 393 | 394 | // did we find an upgrade? 395 | if !bestVersionMatch.Equals(installedVersion) { 396 | var index string 397 | index = name 398 | if installedVersion.isBranch { 399 | res := pkgpathVersionRE.FindStringSubmatch(installedVersion.pkgpath) 400 | if len(res) == 2 { 401 | index = fmt.Sprintf("%s%%%s", name, res[1]) 402 | } 403 | } 404 | updateList[index] = true 405 | fmt.Fprintf(os.Stderr, "%s->%s", installedVersion.fullName, bestVersionMatch.version) 406 | if installedVersion.flavor != "" { 407 | fmt.Fprintf(os.Stderr, "-%s", installedVersion.flavor) 408 | } 409 | fmt.Fprintf(os.Stderr, "\n") 410 | } 411 | } 412 | } 413 | 414 | if verbose { 415 | fmt.Fprintf(os.Stderr, "parse took: %f seconds\n", float64(time.Now().Sub(start))/float64(time.Second)) 416 | } 417 | 418 | if !pkgUpQuirksDate.Equal(mirrorQuirksDate) { 419 | if pkgUpQuirksDate.After(mirrorQuirksDate) { 420 | fmt.Fprintf(os.Stderr, "WARNING: pkgup index appears to be newer than packages on configured mirror.\n") 421 | fmt.Fprintf(os.Stderr, "configured mirror: %s\n", mirrorQuirksDate.Format(time.RFC1123Z)) 422 | fmt.Fprintf(os.Stderr, "pkgup mirror: %s\n", pkgUpQuirksDate.Format(time.RFC1123Z)) 423 | } else { 424 | fmt.Fprintf(os.Stderr, "WARNING: pkgup index appears to be older than packages on configured mirror\n") 425 | fmt.Fprintf(os.Stderr, "pkgup mirror: %s\n", pkgUpQuirksDate.Format(time.RFC1123Z)) 426 | fmt.Fprintf(os.Stderr, "configured mirror: %s\n", mirrorQuirksDate.Format(time.RFC1123Z)) 427 | } 428 | } 429 | 430 | if len(updateList) == 0 { 431 | if !cronMode { 432 | fmt.Fprintf(os.Stderr, "up to date\n") 433 | } 434 | } else { 435 | fmt.Fprintf(os.Stderr, "\nto upgrade:\n") 436 | fmt.Printf("pkg_add -u") 437 | if sysInfo.snapshot == true { 438 | fmt.Printf(" -Dsnap") 439 | } 440 | var sortedUpdates []string 441 | for k := range updateList { 442 | sortedUpdates = append(sortedUpdates, k) 443 | } 444 | sort.Strings(sortedUpdates) 445 | for _, p := range sortedUpdates { 446 | fmt.Printf(" %s", p) 447 | } 448 | fmt.Printf("\n") 449 | } 450 | } 451 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/neutralinsomniac/obsdpkgup 2 | 3 | go 1.20 4 | 5 | require suah.dev/protect v1.2.3 6 | 7 | require golang.org/x/sys v0.11.0 // indirect 8 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= 2 | golang.org/x/sys v0.11.0 h1:eG7RXZHdqOJ1i+0lgLgCpSXAp6M3LYlAo6osgSi0xOM= 3 | golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= 4 | suah.dev/protect v1.2.3 h1:aHeoNwZ9YPp64hrYaN0g0djNE1eRujgH63CrfRrUKdc= 5 | suah.dev/protect v1.2.3/go.mod h1:n1R3XIbsnryKX7C1PO88i5Wgo0v8OTXm9K9FIKt4rfs= 6 | -------------------------------------------------------------------------------- /gzip/gunzip.go: -------------------------------------------------------------------------------- 1 | // Copyright 2009 The Go Authors. All rights reserved. 2 | // Use of this source code is governed by a BSD-style 3 | // license that can be found in the LICENSE file. 4 | 5 | // Package gzip implements reading and writing of gzip format compressed files, 6 | // as specified in RFC 1952. 7 | package gzip 8 | 9 | import ( 10 | "bufio" 11 | "compress/flate" 12 | "encoding/binary" 13 | "errors" 14 | "hash/crc32" 15 | "io" 16 | "time" 17 | ) 18 | 19 | const ( 20 | gzipID1 = 0x1f 21 | gzipID2 = 0x8b 22 | gzipDeflate = 8 23 | flagText = 1 << 0 24 | flagHdrCrc = 1 << 1 25 | flagExtra = 1 << 2 26 | flagName = 1 << 3 27 | flagComment = 1 << 4 28 | ) 29 | 30 | var ( 31 | // ErrChecksum is returned when reading GZIP data that has an invalid checksum. 32 | ErrChecksum = errors.New("gzip: invalid checksum") 33 | // ErrHeader is returned when reading GZIP data that has an invalid header. 34 | ErrHeader = errors.New("gzip: invalid header") 35 | ) 36 | 37 | var le = binary.LittleEndian 38 | 39 | // noEOF converts io.EOF to io.ErrUnexpectedEOF. 40 | func noEOF(err error) error { 41 | if err == io.EOF { 42 | return io.ErrUnexpectedEOF 43 | } 44 | return err 45 | } 46 | 47 | // The gzip file stores a header giving metadata about the compressed file. 48 | // That header is exposed as the fields of the Writer and Reader structs. 49 | // 50 | // Strings must be UTF-8 encoded and may only contain Unicode code points 51 | // U+0001 through U+00FF, due to limitations of the GZIP file format. 52 | type Header struct { 53 | Comment string // comment 54 | Extra []byte // "extra data" 55 | ModTime time.Time // modification time 56 | Name string // file name 57 | OS byte // operating system type 58 | } 59 | 60 | // A Reader is an io.Reader that can be read to retrieve 61 | // uncompressed data from a gzip-format compressed file. 62 | // 63 | // In general, a gzip file can be a concatenation of gzip files, 64 | // each with its own header. Reads from the Reader 65 | // return the concatenation of the uncompressed data of each. 66 | // Only the first header is recorded in the Reader fields. 67 | // 68 | // Gzip files store a length and checksum of the uncompressed data. 69 | // The Reader will return an ErrChecksum when Read 70 | // reaches the end of the uncompressed data if it does not 71 | // have the expected length or checksum. Clients should treat data 72 | // returned by Read as tentative until they receive the io.EOF 73 | // marking the end of the data. 74 | type Reader struct { 75 | Header // valid after NewReader or Reader.Reset 76 | r flate.Reader 77 | decompressor io.ReadCloser 78 | digest uint32 // CRC-32, IEEE polynomial (section 8) 79 | size uint32 // Uncompressed size (section 2.3.1) 80 | buf [512]byte 81 | commentBuf [4194304]byte 82 | err error 83 | multistream bool 84 | } 85 | 86 | // NewReader creates a new Reader reading the given reader. 87 | // If r does not also implement io.ByteReader, 88 | // the decompressor may read more data than necessary from r. 89 | // 90 | // It is the caller's responsibility to call Close on the Reader when done. 91 | // 92 | // The Reader.Header fields will be valid in the Reader returned. 93 | func NewReader(r io.Reader) (*Reader, error) { 94 | z := new(Reader) 95 | if err := z.Reset(r); err != nil { 96 | return nil, err 97 | } 98 | return z, nil 99 | } 100 | 101 | // Reset discards the Reader z's state and makes it equivalent to the 102 | // result of its original state from NewReader, but reading from r instead. 103 | // This permits reusing a Reader rather than allocating a new one. 104 | func (z *Reader) Reset(r io.Reader) error { 105 | *z = Reader{ 106 | decompressor: z.decompressor, 107 | multistream: true, 108 | } 109 | if rr, ok := r.(flate.Reader); ok { 110 | z.r = rr 111 | } else { 112 | z.r = bufio.NewReader(r) 113 | } 114 | z.Header, z.err = z.readHeader() 115 | return z.err 116 | } 117 | 118 | // Multistream controls whether the reader supports multistream files. 119 | // 120 | // If enabled (the default), the Reader expects the input to be a sequence 121 | // of individually gzipped data streams, each with its own header and 122 | // trailer, ending at EOF. The effect is that the concatenation of a sequence 123 | // of gzipped files is treated as equivalent to the gzip of the concatenation 124 | // of the sequence. This is standard behavior for gzip readers. 125 | // 126 | // Calling Multistream(false) disables this behavior; disabling the behavior 127 | // can be useful when reading file formats that distinguish individual gzip 128 | // data streams or mix gzip data streams with other data streams. 129 | // In this mode, when the Reader reaches the end of the data stream, 130 | // Read returns io.EOF. The underlying reader must implement io.ByteReader 131 | // in order to be left positioned just after the gzip stream. 132 | // To start the next stream, call z.Reset(r) followed by z.Multistream(false). 133 | // If there is no next stream, z.Reset(r) will return io.EOF. 134 | func (z *Reader) Multistream(ok bool) { 135 | z.multistream = ok 136 | } 137 | 138 | func (z *Reader) readComment() (string, error) { 139 | var err error 140 | needConv := false 141 | for i := 0; ; i++ { 142 | if i >= len(z.commentBuf) { 143 | return "", ErrHeader 144 | } 145 | z.commentBuf[i], err = z.r.ReadByte() 146 | if err != nil { 147 | return "", err 148 | } 149 | if z.commentBuf[i] > 0x7f { 150 | needConv = true 151 | } 152 | if z.commentBuf[i] == 0 { 153 | // Digest covers the NUL terminator. 154 | z.digest = crc32.Update(z.digest, crc32.IEEETable, z.commentBuf[:i+1]) 155 | 156 | // Strings are ISO 8859-1, Latin-1 (RFC 1952, section 2.3.1). 157 | if needConv { 158 | s := make([]rune, 0, i) 159 | for _, v := range z.commentBuf[:i] { 160 | s = append(s, rune(v)) 161 | } 162 | return string(s), nil 163 | } 164 | return string(z.commentBuf[:i]), nil 165 | } 166 | } 167 | } 168 | 169 | // readString reads a NUL-terminated string from z.r. 170 | // It treats the bytes read as being encoded as ISO 8859-1 (Latin-1) and 171 | // will output a string encoded using UTF-8. 172 | // This method always updates z.digest with the data read. 173 | func (z *Reader) readString() (string, error) { 174 | var err error 175 | needConv := false 176 | for i := 0; ; i++ { 177 | if i >= len(z.buf) { 178 | return "", ErrHeader 179 | } 180 | z.buf[i], err = z.r.ReadByte() 181 | if err != nil { 182 | return "", err 183 | } 184 | if z.buf[i] > 0x7f { 185 | needConv = true 186 | } 187 | if z.buf[i] == 0 { 188 | // Digest covers the NUL terminator. 189 | z.digest = crc32.Update(z.digest, crc32.IEEETable, z.buf[:i+1]) 190 | 191 | // Strings are ISO 8859-1, Latin-1 (RFC 1952, section 2.3.1). 192 | if needConv { 193 | s := make([]rune, 0, i) 194 | for _, v := range z.buf[:i] { 195 | s = append(s, rune(v)) 196 | } 197 | return string(s), nil 198 | } 199 | return string(z.buf[:i]), nil 200 | } 201 | } 202 | } 203 | 204 | // readHeader reads the GZIP header according to section 2.3.1. 205 | // This method does not set z.err. 206 | func (z *Reader) readHeader() (hdr Header, err error) { 207 | if _, err = io.ReadFull(z.r, z.buf[:10]); err != nil { 208 | // RFC 1952, section 2.2, says the following: 209 | // A gzip file consists of a series of "members" (compressed data sets). 210 | // 211 | // Other than this, the specification does not clarify whether a 212 | // "series" is defined as "one or more" or "zero or more". To err on the 213 | // side of caution, Go interprets this to mean "zero or more". 214 | // Thus, it is okay to return io.EOF here. 215 | return hdr, err 216 | } 217 | if z.buf[0] != gzipID1 || z.buf[1] != gzipID2 || z.buf[2] != gzipDeflate { 218 | return hdr, ErrHeader 219 | } 220 | flg := z.buf[3] 221 | if t := int64(le.Uint32(z.buf[4:8])); t > 0 { 222 | // Section 2.3.1, the zero value for MTIME means that the 223 | // modified time is not set. 224 | hdr.ModTime = time.Unix(t, 0) 225 | } 226 | // z.buf[8] is XFL and is currently ignored. 227 | hdr.OS = z.buf[9] 228 | z.digest = crc32.ChecksumIEEE(z.buf[:10]) 229 | 230 | if flg&flagExtra != 0 { 231 | if _, err = io.ReadFull(z.r, z.buf[:2]); err != nil { 232 | return hdr, noEOF(err) 233 | } 234 | z.digest = crc32.Update(z.digest, crc32.IEEETable, z.buf[:2]) 235 | data := make([]byte, le.Uint16(z.buf[:2])) 236 | if _, err = io.ReadFull(z.r, data); err != nil { 237 | return hdr, noEOF(err) 238 | } 239 | z.digest = crc32.Update(z.digest, crc32.IEEETable, data) 240 | hdr.Extra = data 241 | } 242 | 243 | var s string 244 | if flg&flagName != 0 { 245 | if s, err = z.readString(); err != nil { 246 | return hdr, err 247 | } 248 | hdr.Name = s 249 | } 250 | 251 | if flg&flagComment != 0 { 252 | if s, err = z.readComment(); err != nil { 253 | return hdr, err 254 | } 255 | hdr.Comment = s 256 | } 257 | 258 | if flg&flagHdrCrc != 0 { 259 | if _, err = io.ReadFull(z.r, z.buf[:2]); err != nil { 260 | return hdr, noEOF(err) 261 | } 262 | digest := le.Uint16(z.buf[:2]) 263 | if digest != uint16(z.digest) { 264 | return hdr, ErrHeader 265 | } 266 | } 267 | 268 | z.digest = 0 269 | if z.decompressor == nil { 270 | z.decompressor = flate.NewReader(z.r) 271 | } else { 272 | z.decompressor.(flate.Resetter).Reset(z.r, nil) 273 | } 274 | return hdr, nil 275 | } 276 | 277 | // Read implements io.Reader, reading uncompressed bytes from its underlying Reader. 278 | func (z *Reader) Read(p []byte) (n int, err error) { 279 | if z.err != nil { 280 | return 0, z.err 281 | } 282 | 283 | n, z.err = z.decompressor.Read(p) 284 | z.digest = crc32.Update(z.digest, crc32.IEEETable, p[:n]) 285 | z.size += uint32(n) 286 | if z.err != io.EOF { 287 | // In the normal case we return here. 288 | return n, z.err 289 | } 290 | 291 | // Finished file; check checksum and size. 292 | if _, err := io.ReadFull(z.r, z.buf[:8]); err != nil { 293 | z.err = noEOF(err) 294 | return n, z.err 295 | } 296 | digest := le.Uint32(z.buf[:4]) 297 | size := le.Uint32(z.buf[4:8]) 298 | if digest != z.digest || size != z.size { 299 | z.err = ErrChecksum 300 | return n, z.err 301 | } 302 | z.digest, z.size = 0, 0 303 | 304 | // File is ok; check if there is another. 305 | if !z.multistream { 306 | return n, io.EOF 307 | } 308 | z.err = nil // Remove io.EOF 309 | 310 | if _, z.err = z.readHeader(); z.err != nil { 311 | return n, z.err 312 | } 313 | 314 | // Read from next file, if necessary. 315 | if n > 0 { 316 | return n, nil 317 | } 318 | return z.Read(p) 319 | } 320 | 321 | // Close closes the Reader. It does not close the underlying io.Reader. 322 | // In order for the GZIP checksum to be verified, the reader must be 323 | // fully consumed until the io.EOF. 324 | func (z *Reader) Close() error { return z.decompressor.Close() } 325 | -------------------------------------------------------------------------------- /openbsd/index.go: -------------------------------------------------------------------------------- 1 | package openbsd 2 | 3 | import ( 4 | "fmt" 5 | "io/ioutil" 6 | "net/http" 7 | ) 8 | 9 | func GetIndexTxt(mirror string) (string, error) { 10 | indexUrl := fmt.Sprintf("%sindex.txt", mirror) 11 | resp, err := http.Get(indexUrl) 12 | if err != nil { 13 | return "", err 14 | } 15 | defer resp.Body.Close() 16 | 17 | var indexBytes []byte 18 | var indexString string 19 | switch resp.StatusCode { 20 | case 200: 21 | indexBytes, err = ioutil.ReadAll(resp.Body) 22 | if err != nil { 23 | return "", err 24 | } 25 | indexString = string(indexBytes) 26 | return indexString, nil 27 | case 404: 28 | return "", fmt.Errorf("404 encountered while downloading index: %s\n", indexUrl) 29 | default: 30 | return "", fmt.Errorf("unexpected HTTP response (%d) while downloading index: %s\n", resp.StatusCode, indexUrl) 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /openbsd/quirks.go: -------------------------------------------------------------------------------- 1 | package openbsd 2 | 3 | import ( 4 | "fmt" 5 | "github.com/neutralinsomniac/obsdpkgup/gzip" 6 | "net/http" 7 | "strings" 8 | ) 9 | 10 | func GetQuirksSignifyBlockFromIndex(baseUrl, index string) (string, error) { 11 | lines := strings.Split(index, "\n") 12 | for _, line := range lines { 13 | if len(line) == 0 { 14 | continue 15 | } 16 | s := strings.Fields(line) 17 | pkgName := s[9] 18 | if strings.HasPrefix(pkgName, "quirks-") { 19 | url := fmt.Sprintf("%s%s", baseUrl, pkgName) 20 | resp, err := http.Get(url) 21 | if err != nil { 22 | return "", fmt.Errorf("error fetching quirks (%s): %s", url, err.Error()) 23 | } 24 | defer resp.Body.Close() 25 | 26 | switch resp.StatusCode { 27 | case 200: 28 | gz, err := gzip.NewReader(resp.Body) 29 | if err != nil { 30 | return "", fmt.Errorf("error decompressing quirks %s: %s\n", url, err.Error()) 31 | } 32 | 33 | return gz.Comment, nil 34 | case 404: 35 | return "", fmt.Errorf("404 while downloading quirks: \"%s\"\n", url) 36 | default: 37 | return "", fmt.Errorf("unexpected HTTP response (%d) while downloading quirks: %s\n", resp.StatusCode, url) 38 | } 39 | } 40 | } 41 | 42 | return "", fmt.Errorf("couldn't find quirks package in index") 43 | } 44 | -------------------------------------------------------------------------------- /openbsd/signify.go: -------------------------------------------------------------------------------- 1 | package openbsd 2 | 3 | import ( 4 | "fmt" 5 | "strings" 6 | "time" 7 | ) 8 | 9 | var SignifyTimeFormat = time.RFC3339 10 | 11 | func GetSignifyTimestampFromSignifyBlock(signifyBlock string) (string, error) { 12 | lines := strings.Split(signifyBlock, "\n") 13 | for _, line := range lines { 14 | if strings.HasPrefix(line, "date=") { 15 | return line[5:], nil 16 | } 17 | } 18 | 19 | return "", fmt.Errorf("could not find date in signify block") 20 | } 21 | -------------------------------------------------------------------------------- /openbsd/updatesignature.go: -------------------------------------------------------------------------------- 1 | package openbsd 2 | 3 | import ( 4 | "fmt" 5 | "regexp" 6 | "sort" 7 | "strings" 8 | ) 9 | 10 | var nameRe = regexp.MustCompilePOSIX(`^@name (.*)$`) 11 | var dependRe = regexp.MustCompilePOSIX(`^@depend (.*)$`) 12 | var versionRe = regexp.MustCompilePOSIX(`^@version (.*)$`) 13 | var wantLibRe = regexp.MustCompilePOSIX(`^@wantlib (.*)$`) 14 | 15 | var numRe = regexp.MustCompile(`^\d+.*$`) 16 | 17 | func GenerateSignatureFromContents(contents []byte) string { 18 | var signatureParts []string 19 | // name 20 | matches := nameRe.FindAllSubmatch(contents, -1) 21 | for _, match := range matches { 22 | signatureParts = append(signatureParts, string(match[1])) 23 | } 24 | 25 | // version 26 | versionMatch := versionRe.FindSubmatch(contents) 27 | if len(versionMatch) > 0 { 28 | signatureParts = append(signatureParts, string(versionMatch[1])) 29 | } else { 30 | signatureParts = append(signatureParts, "0") 31 | } 32 | 33 | // depends 34 | var depends []string 35 | matches = dependRe.FindAllSubmatch(contents, -1) 36 | dependSet := make(map[string]bool) 37 | for _, match := range matches { 38 | dep := strings.Split(string(match[1]), ":")[2] 39 | dep = fmt.Sprintf("@%s", dep) 40 | dependSet[dep] = true 41 | } 42 | for dep, _ := range dependSet { 43 | // get the flavor part 44 | parts := strings.Split(dep, "-") 45 | flavors := make([]string, 0, len(parts)) 46 | var i int 47 | for i = len(parts) - 1; !numRe.MatchString(parts[i]); i-- { 48 | flavors = append(flavors, parts[i]) 49 | } 50 | // flavors are sorted in the signature even if they aren't in +CONTENTS 51 | sort.Strings(flavors) 52 | allParts := append(parts[:i+1], flavors...) 53 | dep = strings.Join(allParts, "-") 54 | depends = append(depends, dep) 55 | } 56 | sort.Strings(depends) 57 | 58 | signatureParts = append(signatureParts, depends...) 59 | 60 | // wantlib 61 | var wantlibs []string 62 | matches = wantLibRe.FindAllSubmatch(contents, -1) 63 | for _, match := range matches { 64 | wantlibs = append(wantlibs, string(match[1])) 65 | } 66 | sort.Strings(wantlibs) 67 | signatureParts = append(signatureParts, wantlibs...) 68 | 69 | return strings.Join(signatureParts, ",") 70 | } 71 | -------------------------------------------------------------------------------- /openbsd/version/dewey.go: -------------------------------------------------------------------------------- 1 | package version 2 | 3 | import ( 4 | "fmt" 5 | "regexp" 6 | "strconv" 7 | "strings" 8 | ) 9 | 10 | type Dewey struct { 11 | Deweys []string 12 | Suffix string 13 | SuffixValue string // this is a string because technically a suffix value doesn't need to be defined (eg 1.0rc) 14 | } 15 | 16 | var suffixRE = regexp.MustCompile(`^(\d+)(rc|alpha|beta|pre|pl)(\d*)$`) 17 | 18 | func NewDeweyFromString(deweyStr string) Dewey { 19 | dewey := Dewey{} 20 | 21 | dewey.Deweys = strings.Split(deweyStr, ".") 22 | 23 | suffixMatch := suffixRE.FindStringSubmatch(dewey.Deweys[len(dewey.Deweys)-1]) 24 | if len(suffixMatch) != 0 { 25 | if suffixMatch[2] != "" { 26 | dewey.Deweys[len(dewey.Deweys)-1] = suffixMatch[1] 27 | dewey.Suffix = suffixMatch[2] 28 | dewey.SuffixValue = suffixMatch[3] 29 | } 30 | } 31 | 32 | return dewey 33 | } 34 | 35 | func (d Dewey) String() string { 36 | deweyStr := strings.Join(d.Deweys, ".") 37 | if len(d.Suffix) != 0 { 38 | deweyStr = fmt.Sprintf("%s%s%s", deweyStr, d.Suffix, d.SuffixValue) 39 | } 40 | 41 | return deweyStr 42 | } 43 | 44 | func (a Dewey) suffixCompare(b Dewey) int { 45 | if a.Suffix == b.Suffix { 46 | aNum, _ := strconv.Atoi(a.SuffixValue) 47 | bNum, _ := strconv.Atoi(b.SuffixValue) 48 | if aNum < bNum { 49 | return -1 50 | } else if aNum > bNum { 51 | return 1 52 | } else { 53 | return 0 54 | } 55 | } 56 | if a.Suffix == "pl" { 57 | return 1 58 | } 59 | if b.Suffix == "pl" { 60 | return -1 61 | } 62 | 63 | if a.Suffix > b.Suffix { 64 | return -b.suffixCompare(a) 65 | } 66 | 67 | if a.Suffix == "" { 68 | return 1 69 | } 70 | if a.Suffix == "alpha" { 71 | return -1 72 | } 73 | if a.Suffix == "beta" { 74 | return -1 75 | } 76 | 77 | return 0 78 | } 79 | 80 | func (a Dewey) Compare(b Dewey) int { 81 | // numerical comparison 82 | for i := 0; ; i++ { 83 | if i >= len(a.Deweys) { 84 | if i >= len(b.Deweys) { 85 | break 86 | } else { 87 | return -1 88 | } 89 | } 90 | if i >= len(b.Deweys) { 91 | return 1 92 | } 93 | r := deweyCompare(a.Deweys[i], b.Deweys[i]) 94 | if r != 0 { 95 | return r 96 | } 97 | } 98 | 99 | return a.suffixCompare(b) 100 | } 101 | 102 | var versionRe = regexp.MustCompile(`^(\d+)([a-z]?)\.(\d+)([a-z]?)$`) 103 | 104 | func deweyCompare(a, b string) int { 105 | aNum, err1 := strconv.Atoi(a) 106 | bNum, err2 := strconv.Atoi(b) 107 | 108 | // pure numerical comparison 109 | if err1 == nil && err2 == nil { 110 | if aNum < bNum { 111 | return -1 112 | } else if aNum > bNum { 113 | return 1 114 | } else { 115 | return 0 116 | } 117 | } 118 | 119 | cmpString := fmt.Sprintf("%s.%s", a, b) 120 | matches := versionRe.FindStringSubmatch(cmpString) 121 | if len(matches) == 5 { 122 | anStr, al, bnStr, bl := matches[1], matches[2], matches[3], matches[4] 123 | an, _ := strconv.Atoi(anStr) 124 | bn, _ := strconv.Atoi(bnStr) 125 | 126 | if an != bn { 127 | if an > bn { 128 | return 1 129 | } else if an < bn { 130 | return -1 131 | } 132 | } else { 133 | return strings.Compare(al, bl) 134 | } 135 | } 136 | 137 | return strings.Compare(a, b) 138 | } 139 | -------------------------------------------------------------------------------- /openbsd/version/version.go: -------------------------------------------------------------------------------- 1 | package version 2 | 3 | import ( 4 | "fmt" 5 | "regexp" 6 | "strconv" 7 | ) 8 | 9 | type Version struct { 10 | Dewey Dewey 11 | P int // always init to -1 12 | V int // always init to -1 13 | } 14 | 15 | var vRE = regexp.MustCompile(`^(.*)v(\d+)$`) 16 | var pRE = regexp.MustCompile(`^(.*)p(\d+)$`) 17 | 18 | func NewVersionFromString(versionStr string) Version { 19 | var version Version 20 | version.P = -1 21 | version.V = -1 22 | 23 | matches := vRE.FindStringSubmatch(versionStr) 24 | if len(matches) > 0 { 25 | version.V, _ = strconv.Atoi(matches[2]) 26 | versionStr = matches[1] 27 | } 28 | 29 | matches = pRE.FindStringSubmatch(versionStr) 30 | if len(matches) > 0 { 31 | version.P, _ = strconv.Atoi(matches[2]) 32 | versionStr = matches[1] 33 | } 34 | 35 | version.Dewey = NewDeweyFromString(versionStr) 36 | 37 | return version 38 | } 39 | 40 | func (v Version) String() string { 41 | versionStr := v.Dewey.String() 42 | if v.P != -1 { 43 | versionStr = fmt.Sprintf("%sp%d", versionStr, v.P) 44 | } 45 | if v.V != -1 { 46 | versionStr = fmt.Sprintf("%sv%d", versionStr, v.V) 47 | } 48 | return versionStr 49 | } 50 | 51 | func (a Version) PnumCompare(b Version) int { 52 | if a.P < b.P { 53 | return -1 54 | } else if a.P > b.P { 55 | return 1 56 | } else { 57 | return 0 58 | } 59 | } 60 | 61 | func (a Version) Compare(b Version) int { 62 | // simple case: epoch number 63 | if a.V != b.V { 64 | if a.V < b.V { 65 | return -1 66 | } else if a.V > b.V { 67 | return 1 68 | } else { 69 | return 0 70 | } 71 | } 72 | // simple case: only p number differs 73 | if a.Dewey.Compare(b.Dewey) == 0 { 74 | return a.PnumCompare(b) 75 | } 76 | 77 | return a.Dewey.Compare(b.Dewey) 78 | } 79 | -------------------------------------------------------------------------------- /openbsd/version/version_test.go: -------------------------------------------------------------------------------- 1 | package version 2 | 3 | import ( 4 | debug2 "runtime/debug" 5 | "testing" 6 | ) 7 | 8 | func checkErr(err error) { 9 | if err != nil { 10 | panic("static parsing failed??") 11 | } 12 | } 13 | 14 | func checkVersion(t *testing.T, leftStr string, rightStr string, expected int) { 15 | var left, right Version 16 | 17 | left = NewVersionFromString(leftStr) 18 | right = NewVersionFromString(rightStr) 19 | res := left.Compare(right) 20 | if res != expected { 21 | t.Errorf("expected: %d, got: %d", expected, res) 22 | t.Log(string(debug2.Stack())) 23 | } 24 | } 25 | 26 | func TestVersionComparison(t *testing.T) { 27 | // these are ripped straight out of packages-specs(7) 28 | checkVersion(t, "foo-1.01", "foo-1.1", 0) 29 | 30 | checkVersion(t, "foo-1.001", "foo-1.002", -1) 31 | 32 | checkVersion(t, "foo-1.002", "foo-1.0010", -1) 33 | 34 | checkVersion(t, "foo-1.0rc2", "foo-1.0pre3", 0) 35 | 36 | checkVersion(t, "bar-1.0alpha5", "bar-1.0beta3", -1) 37 | 38 | checkVersion(t, "bar-1.0beta3", "bar-1.0rc1", -1) 39 | 40 | checkVersion(t, "baz-1.0", "baz-1.0pl1", -1) 41 | 42 | // these ones I made up 43 | checkVersion(t, "foo-80", "foo-81", -1) 44 | 45 | checkVersion(t, "foo-1.14.7v3", "foo-1.14.7p0v3", -1) 46 | 47 | checkVersion(t, "foo-1.0p2", "foo-1.0v2", -1) 48 | 49 | checkVersion(t, "foo-80.1", "foo-80.2", -1) 50 | 51 | checkVersion(t, "foo-80.0.0", "foo-80.0.1", -1) 52 | 53 | checkVersion(t, "foo-80.0.0", "foo-81.0.1", -1) 54 | 55 | checkVersion(t, "foo-80.1a", "foo-80.1b", -1) 56 | 57 | checkVersion(t, "foo-80.1b", "foo-80.1a", 1) 58 | 59 | checkVersion(t, "foo-80.1", "foo-80.1.1", -1) 60 | 61 | checkVersion(t, "foo-80.1aa", "foo-80.1b", -1) 62 | 63 | checkVersion(t, "foo-80.1p1", "foo-80.1p2", -1) 64 | 65 | checkVersion(t, "foo-80.1", "foo-80.1p1", -1) 66 | 67 | checkVersion(t, "foo-80.1a", "foo-80.0aa", 1) 68 | 69 | checkVersion(t, "foo-80.1.1", "foo-80.1", 1) 70 | 71 | checkVersion(t, "foo-80.1a", "foo-80.1aa", -1) 72 | 73 | checkVersion(t, "foo-80.1.1", "foo-80.1.1", 0) 74 | } 75 | -------------------------------------------------------------------------------- /update_indexes_go.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | pkgup_dir=/var/www/pkgup 4 | 5 | for arch in amd64 aarch64 6 | do 7 | for version in snapshots 6.8 8 | do 9 | echo $arch:$version 10 | if genpkgup -a $arch -v $version > index.pkgup 11 | then 12 | rm -f index.pkgup.gz 13 | gzip index.pkgup 14 | mv index.pkgup.gz $pkgup_dir/$version/$arch/ 15 | fi 16 | done 17 | done 18 | --------------------------------------------------------------------------------