├── LICENSE ├── README.md ├── go.mod ├── go.sum ├── ratt.go └── sbuild.go /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) Michael Stapelberg , 2015 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Description 2 | 3 | ratt (“Rebuild All The Things!”) operates on a Debian .changes file of a just-built package, identifies all reverse-build-dependencies and rebuilds them with the .debs from the .changes file. 4 | 5 | The intended use-case is, for example, to package a new snapshot of a Go library and verify that the new version does not break any other Go libraries/binaries. 6 | 7 | # Installation (from git, for hacking on ratt) 8 | 9 | Please install ratt from Debian. In case you want to hack on ratt, you can use the following commands to install Go, download ratt from git and compile/install it: 10 | 11 | ```bash 12 | sudo apt-get install golang-go 13 | git clone https://github.com/Debian/ratt 14 | cd ratt 15 | go install 16 | ``` 17 | 18 | Start the resulting binary in `~/go/bin/ratt`: 19 | 20 | ```bash 21 | ~/go/bin/ratt -help 22 | ``` 23 | 24 | After making changes to the code, to recompile and install it again, use: 25 | 26 | ```bash 27 | go install 28 | ``` 29 | 30 | # Usage 31 | 32 | Let’s assume you build a new version of a Go library, like so: 33 | 34 | ```bash 35 | debcheckout golang-github-jacobsa-gcloud-dev 36 | cd golang-github-jacobsa-gcloud-dev 37 | dch -i -m 'dummy new version' 38 | git commit -a -m 'dummy new version' 39 | gbp buildpackage --git-pbuilder 40 | ``` 41 | 42 | Now you can use ratt to identify and rebuild all reverse-build-dependencies: 43 | ``` 44 | $ ratt golang-github-jacobsa-gcloud_0.0\~git20150709-2_amd64.changes 45 | 2015/08/16 11:48:41 Loading changes file "golang-github-jacobsa-gcloud_0.0~git20150709-2_amd64.changes" 46 | 2015/08/16 11:48:41 - 1 binary packages: golang-github-jacobsa-gcloud-dev 47 | 2015/08/16 11:48:41 - corresponding .debs (will be injected when building): 48 | 2015/08/16 11:48:41 golang-github-jacobsa-gcloud-dev_0.0~git20150709-2_all.deb 49 | 2015/08/16 11:48:41 Loading sources index "/var/lib/apt/lists/ftp.ch.debian.org_debian_dists_sid_contrib_source_Sources" 50 | 2015/08/16 11:48:41 Loading sources index "/var/lib/apt/lists/ftp.ch.debian.org_debian_dists_sid_main_source_Sources" 51 | 2015/08/16 11:48:43 Loading sources index "/var/lib/apt/lists/ftp.ch.debian.org_debian_dists_sid_non-free_source_Sources" 52 | 2015/08/16 11:48:43 Building golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1 (commandline: [sbuild --arch-all --dist=sid --nolog golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1 --extra-package=golang-github-jacobsa-gcloud-dev_0.0~git20150709-2_all.deb]) 53 | 2015/08/16 11:49:19 Build results: 54 | 2015/08/16 11:49:19 PASSED: golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1 55 | ``` 56 | 57 | ratt uses `sbuild(1)` to build packages, see https://wiki.debian.org/sbuild for instructions on how to set up sbuild. Be sure to add `--components=main,contrib,non-free` to the sbuild-createchroot line in case you want to deal with packages outside of main as well. 58 | 59 | # Targeting a different suite 60 | 61 | Imagine you’re running Debian stable on your machine, but you’re working on a package for Debian unstable (“sid”). Unless you already have configured a corresponding `sources.list` entry for sid, you will encounter an error message like this: 62 | 63 | ``` 64 | $ ratt golang-google-grpc_1.11.0-1_amd64.changes 65 | 2019/01/19 10:44:34 Loading changes file "golang-google-grpc_1.11.0-1_amd64.changes" 66 | 2019/01/19 10:44:34 - 1 binary packages: golang-google-grpc-dev 67 | 2019/01/19 10:44:34 Corresponding .debs (will be injected when building): 68 | 2019/01/19 10:44:34 golang-google-grpc-dev_1.11.0-1_all.deb 69 | 2019/01/19 10:44:34 Setting -dist=sid (from .changes file) 70 | 2019/01/19 10:44:34 Could not find InRelease file for sid . Are you missing sid in your /etc/apt/sources.list? 71 | ``` 72 | 73 | The most direct solution is to add sid to your `/etc/apt/sources.list` file, then set `Default-Release` to stable, so that apt prefers the same packages as before your addition: 74 | 75 | ``` 76 | # echo 'deb http://deb.debian.org/debian sid main' >> /etc/apt/sources.list 77 | # echo 'deb-src http://deb.debian.org/debian sid main' >> /etc/apt/sources.list 78 | # echo 'APT::Default-Release "stable";' >> /etc/apt/apt.conf 79 | # apt update 80 | $ ratt ... 81 | ``` 82 | 83 | An alternative solution is to use `chdist(1)`, a tool that allows to create and maintain different apt trees for different suites. Assuming that you have a sid distribution ready in your `~/.chdist`, you can then use it by setting the environment variable `APT_CONFIG`: 84 | 85 | ``` 86 | APT_CONFIG=~/.chdist/sid/etc/apt/apt.conf ratt ... 87 | ``` 88 | 89 | # Restricting the set of packages that will be built 90 | 91 | In some cases there are many build dependencies; to focus on a smaller set of packages, invoke ratt with the `-include` / `-exclude` options. 92 | 93 | To only build selected packages, use: 94 | 95 | ``` 96 | ratt -recheck -include '^(hwloc|fltk1.3|wcslib|ccfits|qevercloud|libstxxl|caffe|frobby|starpu)$' ../doxygen_1.8.17-1_amd64.changes 97 | ``` 98 | 99 | To exclude certain packages (for example those with longer build times): 100 | 101 | ``` 102 | ratt -recheck -exclude '^(gcc-9|gcc-8|llvm-toolchain-10|libreoffice|trilinos|llvm-toolchain-9|llvm-toolchain-8|llvm-toolchain-7|gcc-snapshot|gcc-10|deal.ii|kodi|vg|qgis|openms|siconos|ball|gtg-trace|libsbml|dcmtk|gromacs|gudhi|kicad|libpwiz)$' ../doxygen_1.8.17-1_amd64.changes 103 | ``` 104 | 105 | **Note**: you need to escape the `+` sign in package names as in `dbus-c\+\+` to avoid messing up the regexp expression. 106 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/Debian/ratt 2 | 3 | go 1.18 4 | 5 | require pault.ag/go/debian v0.12.0 6 | 7 | require ( 8 | golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897 // indirect 9 | pault.ag/go/topsort v0.0.0-20160530003732-f98d2ad46e1a // indirect 10 | ) 11 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/DataDog/zstd v1.4.8/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw= 2 | github.com/kjk/lzma v0.0.0-20161016003348-3fd93898850d/go.mod h1:phT/jsRPBAEqjAibu1BurrabCBNTYiVI+zbmyCZJY6Q= 3 | github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos= 4 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 5 | golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897 h1:pLI5jrR7OSLijeIDcmRxNmw2api+jEfxLoykJVice/E= 6 | golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 7 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 8 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 9 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 10 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 11 | pault.ag/go/debian v0.12.0 h1:b8ctSdBSGJ98NE1VLn06aSx70EUpczlP2qqSHEiYYJA= 12 | pault.ag/go/debian v0.12.0/go.mod h1:UbnMr3z/KZepjq7VzbYgBEfz8j4+Pyrm2L5X1fzhy/k= 13 | pault.ag/go/topsort v0.0.0-20160530003732-f98d2ad46e1a h1:WwS7vlB5H2AtwKj1jsGwp2ZLud1x6WXRXh2fXsRqrcA= 14 | pault.ag/go/topsort v0.0.0-20160530003732-f98d2ad46e1a/go.mod h1:INqx0ClF7kmPAMk2zVTX8DRnhZ/yaA/Mg52g8KFKE7k= 15 | -------------------------------------------------------------------------------- /ratt.go: -------------------------------------------------------------------------------- 1 | // ratt operates on a Debian .changes file of a just-built package, identifies 2 | // all reverse-build-dependencies and rebuilds them with the .debs from the 3 | // .changes file. 4 | // 5 | // The intended use-case is, for example, to package a new snapshot of a Go 6 | // library and verify that the new version does not break any other Go 7 | // libraries/binaries. 8 | package main 9 | 10 | import ( 11 | "bufio" 12 | "bytes" 13 | "flag" 14 | "fmt" 15 | "io" 16 | "log" 17 | "os" 18 | "os/exec" 19 | "path/filepath" 20 | "regexp" 21 | "sort" 22 | "strings" 23 | 24 | "pault.ag/go/debian/control" 25 | "pault.ag/go/debian/version" 26 | ) 27 | 28 | func filter(m map[string][]version.Version, f func(string) bool) map[string][]version.Version { 29 | mf := make(map[string][]version.Version, 0) 30 | for k, v := range m { 31 | if f(k) { 32 | mf[k] = v 33 | } 34 | } 35 | return mf 36 | } 37 | 38 | type buildResult struct { 39 | src string 40 | version *version.Version 41 | err error 42 | recheckErr error 43 | logFile string 44 | recheckLogFile string 45 | } 46 | 47 | var ( 48 | logDir = flag.String("log_dir", 49 | "buildlogs", 50 | "Directory in which sbuild(1) logs for all reverse-build-dependencies are stored") 51 | 52 | dryRun = flag.Bool("dry_run", 53 | false, 54 | "Print sbuild command lines, but do not actually build the reverse-build-dependencies") 55 | 56 | include = flag.String("include", 57 | "", 58 | "Only build packages which match the supplied regex") 59 | 60 | exclude = flag.String("exclude", 61 | "", 62 | "Do not build packages which match the supplied regex") 63 | 64 | sbuildDist = flag.String("sbuild_dist", 65 | "", 66 | "sbuild --dist= value (e.g. \"sid\"). Defaults to the Distribution: entry from the specified .changes file") 67 | 68 | dist = flag.String("dist", 69 | "", 70 | "Distribution to look up reverse-build-dependencies from. Defaults to the Distribution: entry from the specified .changes file") 71 | 72 | recheck = flag.Bool("recheck", 73 | false, 74 | "Rebuild without new changes to check if the failures are really related") 75 | 76 | listsPrefixRe = regexp.MustCompile(`/([^/]*_dists_.*)_InRelease$`) 77 | ) 78 | 79 | func dependsOn(src control.SourceIndex, binaries map[string]bool) bool { 80 | buildDepends := src.GetBuildDepends() 81 | for _, possibility := range buildDepends.GetAllPossibilities() { 82 | if binaries[possibility.Name] { 83 | return true 84 | } 85 | } 86 | return false 87 | } 88 | 89 | func addReverseBuildDeps(sourcesPath string, binaries map[string]bool, rebuild map[string][]version.Version) error { 90 | log.Printf("Loading sources index %q\n", sourcesPath) 91 | catFile := exec.Command("/usr/lib/apt/apt-helper", 92 | "cat-file", 93 | sourcesPath) 94 | var s *bufio.Reader 95 | if lines, err := catFile.Output(); err == nil { 96 | s = bufio.NewReader(bytes.NewReader(lines)) 97 | } else { 98 | // Fallback for older versions of apt-get. See 99 | // <20160111171230.GA17291@debian.org> for context. 100 | o, err := os.Open(sourcesPath) 101 | if err != nil { 102 | return err 103 | } 104 | defer o.Close() 105 | s = bufio.NewReader(o) 106 | } 107 | idx, err := control.ParseSourceIndex(s) 108 | if err != nil && err != io.EOF { 109 | return err 110 | } 111 | 112 | for _, src := range idx { 113 | if dependsOn(src, binaries) { 114 | rebuild[src.Package] = append(rebuild[src.Package], src.Version) 115 | } 116 | } 117 | 118 | return nil 119 | } 120 | 121 | func fallback(sourcesPaths []string, binaries []string) (map[string][]version.Version, error) { 122 | bins := make(map[string]bool) 123 | for _, bin := range binaries { 124 | bins[bin] = true 125 | } 126 | 127 | rebuild := make(map[string][]version.Version) 128 | for _, sourcesPath := range sourcesPaths { 129 | if err := addReverseBuildDeps(sourcesPath, bins, rebuild); err != nil { 130 | return nil, err 131 | } 132 | } 133 | return rebuild, nil 134 | } 135 | 136 | func reverseBuildDeps(packagesPaths, sourcesPaths []string, binaries []string) (map[string][]version.Version, error) { 137 | if _, err := exec.LookPath("dose-ceve"); err != nil { 138 | log.Printf("dose-ceve(1) not found. Please install the dose-extra package for more accurate results. Falling back to interpreting Sources directly") 139 | return fallback(sourcesPaths, binaries) 140 | } 141 | 142 | archOut, err := exec.Command("dpkg-architecture", "--query=DEB_BUILD_ARCH").Output() 143 | if err != nil { 144 | log.Fatal(err) 145 | } 146 | arch := strings.TrimSpace(string(archOut)) 147 | 148 | // TODO: Cache this output based on the .changes file. dose-ceve takes quite a while. 149 | ceve := exec.Command( 150 | "dose-ceve", 151 | "--deb-native-arch="+arch, 152 | "-T", "debsrc", 153 | "-r", strings.Join(binaries, ","), 154 | "-G", "pkg") 155 | for _, packagesPath := range packagesPaths { 156 | ceve.Args = append(ceve.Args, "deb://"+packagesPath) 157 | } 158 | for _, sourcesPath := range sourcesPaths { 159 | ceve.Args = append(ceve.Args, "debsrc://"+sourcesPath) 160 | } 161 | ceve.Stderr = os.Stderr 162 | 163 | log.Printf("Figuring out reverse build dependencies using dose-ceve(1). This might take a while") 164 | out, err := ceve.Output() 165 | if err != nil { 166 | log.Printf("dose-ceve(1) failed (%v), falling back to interpreting Sources directly", err) 167 | return fallback(sourcesPaths, binaries) 168 | } 169 | var doseCeves []struct { 170 | Package string 171 | Version version.Version 172 | } 173 | r := bufio.NewReader(bytes.NewReader(out)) 174 | if err := control.Unmarshal(&doseCeves, r); err != nil { 175 | return nil, err 176 | } 177 | rebuild := make(map[string][]version.Version) 178 | for _, doseCeve := range doseCeves { 179 | rebuild[doseCeve.Package] = append(rebuild[doseCeve.Package], doseCeve.Version) 180 | } 181 | return rebuild, nil 182 | } 183 | 184 | func main() { 185 | flag.Parse() 186 | 187 | if flag.NArg() == 0 { 188 | log.Fatalf("Usage: %s [options] ...\n", os.Args[0]) 189 | } 190 | 191 | var debs []string 192 | var binaries []string 193 | var changesDist string 194 | for i, changesPath := range flag.Args() { 195 | log.Printf("Loading changes file %q\n", changesPath) 196 | c, err := os.Open(changesPath) 197 | if err != nil { 198 | log.Fatal(err) 199 | } 200 | defer c.Close() 201 | changes, err := control.ParseChanges(bufio.NewReader(c), changesPath) 202 | if err != nil && err != io.EOF { 203 | log.Fatal(err) 204 | } 205 | 206 | log.Printf(" - %d binary packages: %s\n", len(changes.Binaries), strings.Join(changes.Binaries, " ")) 207 | 208 | for _, file := range changes.Files { 209 | if filepath.Ext(file.Filename) == ".deb" { 210 | debs = append(debs, filepath.Join(filepath.Dir(changesPath), file.Filename)) 211 | } 212 | } 213 | binaries = append(binaries, changes.Binaries...) 214 | 215 | if i == 0 { 216 | changesDist = changes.Distribution 217 | } else if changesDist != changes.Distribution { 218 | log.Printf("%s has different distrution, but we will only consider %s\n", changes.Filename, changesDist) 219 | } 220 | } 221 | 222 | log.Printf("Corresponding .debs (will be injected when building):\n") 223 | for _, deb := range debs { 224 | log.Printf(" %s\n", deb) 225 | } 226 | 227 | if strings.TrimSpace(*dist) == "" { 228 | *dist = changesDist 229 | // Rewrite unstable to sid, which apt-get indextargets (below) requires. 230 | if *dist == "unstable" { 231 | *dist = "sid" 232 | } 233 | log.Printf("Setting -dist=%s (from .changes file)\n", *dist) 234 | } 235 | 236 | var sourcesPaths []string 237 | var packagesPaths []string 238 | indexTargets := exec.Command("apt-get", 239 | "indextargets", 240 | "--format", 241 | "$(FILENAME)", 242 | "Codename: "+*dist, 243 | "ShortDesc: Sources") 244 | if lines, err := indexTargets.Output(); err == nil { 245 | for _, line := range strings.Split(string(lines), "\n") { 246 | trimmed := strings.TrimSpace(line) 247 | if trimmed != "" { 248 | sourcesPaths = append(sourcesPaths, line) 249 | } 250 | } 251 | binaryIndexTargets := exec.Command( 252 | "apt-get", 253 | "indextargets", 254 | "--format", 255 | "$(FILENAME)", 256 | "Codename: "+*dist, 257 | "ShortDesc: Packages") 258 | lines, err = binaryIndexTargets.Output() 259 | if err != nil { 260 | log.Fatal("Could not get packages files using %+v: %v", binaryIndexTargets.Args, err) 261 | } 262 | for _, line := range strings.Split(string(lines), "\n") { 263 | trimmed := strings.TrimSpace(line) 264 | if trimmed != "" { 265 | packagesPaths = append(packagesPaths, line) 266 | } 267 | } 268 | } else { 269 | // Fallback for older versions of apt-get. See 270 | // https://bugs.debian.org/801594 for context. 271 | releaseMatches, err := filepath.Glob("/var/lib/apt/lists/*_InRelease") 272 | if err != nil { 273 | log.Fatal(err) 274 | } 275 | for _, releasepath := range releaseMatches { 276 | r, err := os.Open(releasepath) 277 | if err != nil { 278 | log.Fatal(err) 279 | } 280 | defer r.Close() 281 | var inRelease struct { 282 | Suite string 283 | } 284 | if err := control.Unmarshal(&inRelease, bufio.NewReader(r)); err != nil { 285 | log.Fatal(err) 286 | } 287 | 288 | listsPrefix := listsPrefixRe.FindStringSubmatch(releasepath) 289 | if len(listsPrefix) != 2 { 290 | log.Fatalf("release file path %q does not match regexp %q\n", releasepath, listsPrefixRe) 291 | } 292 | sourceMatches, err := filepath.Glob(fmt.Sprintf("/var/lib/apt/lists/%s_*_Sources", listsPrefix[1])) 293 | if err != nil { 294 | log.Fatal(err) 295 | } 296 | sourcesPaths = append(sourcesPaths, sourceMatches...) 297 | packagesMatches, err := filepath.Glob(fmt.Sprintf("/var/lib/apt/lists/%s_*_Packages", listsPrefix[1])) 298 | if err != nil { 299 | log.Fatal(err) 300 | } 301 | packagesPaths = append(packagesPaths, packagesMatches...) 302 | } 303 | } 304 | 305 | if len(sourcesPaths) == 0 { 306 | log.Fatal("Could not find InRelease file for " + *dist + " . Are you missing " + *dist + " in your /etc/apt/sources.list?") 307 | } 308 | 309 | rebuild, err := reverseBuildDeps(packagesPaths, sourcesPaths, binaries) 310 | if err != nil { 311 | log.Fatal(err) 312 | } 313 | log.Printf("Found %d reverse build dependencies\n", len(rebuild)) 314 | 315 | if *include != "" { 316 | filtered, err := regexp.Compile(*include) 317 | if err != nil { 318 | log.Fatal(err) 319 | } 320 | rebuild = filter(rebuild, func(v string) bool { 321 | return filtered.MatchString(v) 322 | }) 323 | log.Printf("Based on the supplied include filter, will only build %d reverse build dependencies\n", len(rebuild)) 324 | } 325 | if *exclude != "" { 326 | filtered, err := regexp.Compile(*exclude) 327 | if err != nil { 328 | log.Fatal(err) 329 | } 330 | rebuild = filter(rebuild, func(v string) bool { 331 | return !filtered.MatchString(v) 332 | }) 333 | log.Printf("Based on the supplied exclude filter, will only build %d reverse build dependencies\n", len(rebuild)) 334 | } 335 | 336 | // TODO: add -recursive flag to also cover dependencies which are not DIRECT dependencies. use http://godoc.org/pault.ag/go/debian/control#OrderDSCForBuild (topsort) to build dependencies in the right order (saving CPU time). 337 | 338 | // TODO: what’s a good integration method for doing this in more setups, e.g. on a cloud provider or something? mapreri from #debian-qa says jenkins.debian.net is suitable. 339 | 340 | if strings.TrimSpace(*sbuildDist) == "" { 341 | *sbuildDist = changesDist 342 | log.Printf("Setting -sbuild_dist=%s (from .changes file)\n", *sbuildDist) 343 | } 344 | 345 | if err := os.MkdirAll(*logDir, 0755); err != nil { 346 | log.Fatal(err) 347 | } 348 | 349 | builder := &sbuild{ 350 | dist: *sbuildDist, 351 | logDir: *logDir, 352 | dryRun: *dryRun, 353 | extraDebs: debs, 354 | } 355 | cnt := 1 356 | buildresults := make(map[string](*buildResult)) 357 | for src, versions := range rebuild { 358 | sort.Sort(sort.Reverse(version.Slice(versions))) 359 | newest := versions[0] 360 | log.Printf("Building package %d of %d: %s \n", cnt, len(rebuild), src) 361 | cnt++ 362 | result := builder.build(src, &newest) 363 | if result.err != nil { 364 | log.Printf("building %s failed: %v\n", src, result.err) 365 | } 366 | buildresults[src] = result 367 | } 368 | 369 | var toInclude []string 370 | for src, result := range buildresults { 371 | if result.err != nil { 372 | toInclude = append(toInclude, strings.ReplaceAll(src, "+", "\\+")) 373 | } 374 | } 375 | if len(toInclude) > 0 { 376 | log.Printf("%d packages failed the first pass; you can rerun ratt only for them passing the option -include '^(%s)$'\n", len(toInclude), strings.Join(toInclude, "|")) 377 | } 378 | 379 | if *dryRun { 380 | return 381 | } 382 | 383 | if *recheck { 384 | log.Printf("Begin to rebuild all failed packages without new changes\n") 385 | recheckBuilder := &sbuild{ 386 | dist: *sbuildDist, 387 | logDir: *logDir + "_recheck", 388 | dryRun: false, 389 | } 390 | if err := os.MkdirAll(recheckBuilder.logDir, 0755); err != nil { 391 | log.Fatal(err) 392 | } 393 | cnt := 1 394 | for src, result := range buildresults { 395 | if result.err == nil { 396 | continue 397 | } 398 | log.Printf("Rebuilding package %d of %d: %s \n", cnt, len(toInclude), src) 399 | cnt++ 400 | recheckResult := recheckBuilder.build(src, result.version) 401 | result.recheckErr = recheckResult.err 402 | result.recheckLogFile = recheckResult.logFile 403 | if recheckResult.err != nil { 404 | log.Printf("rebuilding %s without new changes failed: %v\n", src, recheckResult.err) 405 | } 406 | } 407 | } 408 | 409 | log.Printf("Build results:\n") 410 | // Print all successful builds first (not as interesting), then failed ones. 411 | for src, result := range buildresults { 412 | if result.err == nil { 413 | log.Printf("PASSED: %s\n", src) 414 | } 415 | } 416 | 417 | for src, result := range buildresults { 418 | if result.err != nil && result.recheckErr != nil { 419 | log.Printf("FAILED: %s, but maybe unrelated to new changes (see %s and %s)\n", 420 | src, result.logFile, result.recheckLogFile) 421 | } 422 | } 423 | 424 | failures := false 425 | for src, result := range buildresults { 426 | if result.err != nil && result.recheckErr == nil { 427 | log.Printf("FAILED: %s (see %s)\n", src, result.logFile) 428 | failures = true 429 | } 430 | } 431 | 432 | if failures { 433 | os.Exit(1) 434 | } 435 | } 436 | -------------------------------------------------------------------------------- /sbuild.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "log" 6 | "os" 7 | "os/exec" 8 | "path/filepath" 9 | 10 | "pault.ag/go/debian/version" 11 | ) 12 | 13 | type sbuild struct { 14 | dist string 15 | logDir string 16 | dryRun bool 17 | extraDebs []string 18 | } 19 | 20 | func (s *sbuild) build(sourcePackage string, version *version.Version) *buildResult { 21 | result := &buildResult{ 22 | src: sourcePackage, 23 | version: version, 24 | } 25 | target := fmt.Sprintf("%s_%s", sourcePackage, version) 26 | // TODO: discard resulting package immediately? 27 | args := []string{ 28 | "--arch-all", 29 | "--dist=" + s.dist, 30 | "--nolog", 31 | target, 32 | } 33 | for _, filename := range s.extraDebs { 34 | args = append(args, fmt.Sprintf("--extra-package=%s", filename)) 35 | } 36 | cmd := exec.Command("sbuild", args...) 37 | if s.dryRun { 38 | log.Printf(" commandline: %v\n", cmd.Args) 39 | return result 40 | } 41 | 42 | buildlog, err := os.Create(filepath.Join(s.logDir, target)) 43 | defer buildlog.Close() 44 | if err != nil { 45 | result.err = err 46 | return result 47 | } 48 | cmd.Stdout = buildlog 49 | cmd.Stderr = buildlog 50 | result.err = cmd.Run() 51 | result.logFile = buildlog.Name() 52 | return result 53 | } 54 | --------------------------------------------------------------------------------