├── .gitignore ├── .travis.yml ├── README.md ├── deploy.sh ├── docker └── Dockerfile ├── install.sh ├── makepkg.conf └── settings.conf /.gitignore: -------------------------------------------------------------------------------- 1 | repo/ 2 | *.pgp 3 | deploy_key* 4 | secrets.tar.gz 5 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: required 2 | 3 | services: 4 | - docker 5 | 6 | cache: 7 | directories: 8 | - repo/ 9 | 10 | before_install: 11 | # - openssl aes-256-cbc -K -iv -in secrets.tar.gz.enc -out secrets.tar.gz -d 12 | - tar xvfz secrets.tar.gz 13 | - chmod 600 deploy_key 14 | - docker build --build-arg USER_ID="$(id -u)" -t "$DOCKER_TAG" docker/ 15 | 16 | script: 17 | - docker run -e REPO_NAME="$REPO_NAME" -v "$PWD":/home/builder/aur-build "$DOCKER_TAG" 18 | 19 | deploy: 20 | provider: script 21 | script: ./deploy.sh 22 | skip_cleanup: true 23 | on: 24 | branch: master 25 | 26 | notifications: 27 | email: 28 | on_success: change 29 | on_failure: always 30 | 31 | branches: 32 | only: master 33 | 34 | env: 35 | global: 36 | - PUSH_BRANCH=gh-pages 37 | - DOCKER_TAG=aur-build 38 | - REPO_NAME=personal 39 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # aur-autobuild-template 2 | 3 | A template for automating your Arch AUR builds using Travis CI. By the way, I use Arch. 4 | 5 | Feel free to fork this template and use it for your own builds. Contributions are also welcome! 6 | 7 | ## Dependencies 8 | 9 | * [ruby-travis](https://aur.archlinux.org/packages/ruby-travis/) 10 | * [docker](https://www.archlinux.org/packages/community/x86_64/docker/) 11 | * Optional for local testing. 12 | 13 | ## Setup 14 | 15 | ### Signing your Packages (Optional but Highly Recommended!) 16 | 17 | Create a new PGP key using `gpg --gen-key`; follow the instructions given to create your key. Even 18 | though it is against normal best practices, you should use *no password* for your key because there 19 | is no security gained if you have to tell Travis CI your password anyway. Make sure to keep a copy 20 | of the revocation certificate in case your key becomes compromised. Also, make note of your key's 21 | fingerprint for the following steps. 22 | 23 | In `settings.conf`, add your key's fingerprint to `sign_key`. 24 | 25 | You'll also have to export your key for `pacman-key` so that it can sign it. This tells pacman to 26 | trust the key you've just created. Run the following to do that: 27 | 28 | ```bash 29 | gpg --export --output public-package-key.pgp 30 | sudo pacman-key -a public-package-key.pgp 31 | sudo pacman-key --lsign-key 32 | ``` 33 | 34 | Finally, you'll have to send the encrypted private key to Travis CI. Export your private key by 35 | running `gpg --export-secret-keys --output package-key.pgp `. 36 | 37 | **Warning!** Make sure that you do not accidentally commit your private key. That would be very, 38 | very bad. 39 | 40 | ### Local Repository 41 | 42 | First, you will need to get a list of all foreign packages. This list should be ordered with any 43 | dependencies coming before packages that depend on them. The following snippet will automate that 44 | for you: 45 | 46 | ```bash 47 | for p in $(pacman -Qmq) 48 | do 49 | pactree -ur $p 50 | done | tac | awk '!x[$0]++' | tac | grep -Fxvf <(pacman -Qnq) 51 | ``` 52 | 53 | In `settings.conf`, the `packages` array should be populated with this list. 54 | 55 | Next, you must create the git submodules. If all of your foreign packages were installed from the 56 | AUR, the following script will clone all the submodules: 57 | 58 | ```bash 59 | for p in $(pacman -Qmq) 60 | do 61 | git submodule add --branch master https://aur.archlinux.org/$p.git 62 | done 63 | ``` 64 | 65 | At this point, you should review all PKGBUILDs that have been cloned since there may be new commits 66 | since the last time your AUR packages were updated. 67 | 68 | If you trust all the PKGBUILDs and the `validpgpkeys` in each PKGBUILD, you'll have to add each key 69 | to `settings.conf` in the `trustedpgpkeys` array. 70 | 71 | A simple way of getting all PGP keys is this: 72 | 73 | ```bash 74 | git submodule foreach --quiet bash -c 'source PKGBUILD && 75 | if [ ! -z "${validpgpkeys[0]}" ] 76 | then 77 | printf "%s\n" "${validpgpkeys[@]}" 78 | fi' | sort -u 79 | ``` 80 | 81 | ### Travis CI 82 | 83 | In order to push releases, you'll have to generate a SSH keypair. To do this, run 84 | `ssh-keygen -f deploy_key`. Next, copy the contents of your public key, `deploy_key.pub`, over to 85 | `https://github.com///settings/keys`. 86 | 87 | Next, push your repository to GitHub. Then to enable the repository, you'll have to navigate to 88 | https://travis-ci.org/profile and turn on the switch corresponding to the repository you just 89 | pushed. 90 | 91 | At this point, it's time to encrypt your secret files. To do this, first create a tarball with all 92 | of the files you wish to encrypt using `tar cvfz secrets.tar.gz deploy_key package-key.pgp`. Omit 93 | the last parameter of that command if you chose not to sign your packages. Then, you'll have to push 94 | this package up to Travis CI. You can do this by running `travis encrypt-file secrets.tar.gz`. Make 95 | sure you put the line that was output under `before_install` in your `.travis.yml` file. 96 | 97 | Finally, if you would like, you may configure the repository name by changing 98 | the `env.global.REPO_NAME` section of your `.travis.yml` file. 99 | 100 | You may now push your changes to GitHub. This should automatically trigger a Travis CI build which, 101 | when complete, will push a release to `http(s)://.github.io//repo`. In order to 102 | maximise security, ensure you have "Enforce HTTPS" checked under 103 | `https://github.com///settings`, if possible. 104 | 105 | ### Pacman 106 | 107 | In `/etc/pacman.conf`, add the following section: 108 | 109 | ``` 110 | [$REPO_NAME] 111 | Server = $URL 112 | ``` 113 | 114 | Where `$REPO_NAME` should be replaced the value of `env.global.REPO_NAME` in `.travis.yml` and 115 | `$URL` should be replaced with the URL mentioned in the section above. 116 | 117 | If you do not plan on signing your packages, add `SigLevel = PackageOptional` as well. 118 | 119 | ### Finishing 120 | 121 | That's it, you're done! In order to use the newly created packages, simply run `sudo pacman -Syyu` 122 | to update the package databases (and also, your packages because Arch doesn't support partial 123 | upgrades). Finally, you can install all of your AUR packaages using a simple 124 | `sudo pacman -S `. 125 | 126 | ## Security Warning 127 | 128 | If you are signing your packages, please note that you are placing absolute trust in GitHub to not 129 | tamper with the source and Travis CI to not tamper with your packages. In addition to that, you are 130 | trusting Travis CI with PGP private key, so make sure that you save and protect your revocation 131 | certificate, just in case. 132 | 133 | ## Limitations 134 | 135 | * The `gh-pages` branch has a limit of 50 MB so, if you would like to host a repository larger than 136 | this, consider other hosting options. This can be configured in the `deploy` section of your 137 | `.travis.yml`. 138 | 139 | * Travis CI automatically kills builds which produce more than 4 MB of output so if your log output 140 | is very verbose, consider filtering out unnecessary lines using `grep -v` or logging into a file 141 | in your `repo/` directory. 142 | 143 | * Travis CI automatically kills builds which do not produce any output after ten minutes so ensure 144 | that your build produces some output within that timeframe. Alternatively, consider using 145 | [`travis_wait`](https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received). 146 | 147 | ## Contributing 148 | 149 | Pull-requests and suggestions are welcome! Please let me know if you have any suggestions or 150 | improvements! 151 | 152 | ## Acknowledgements 153 | 154 | * [/u/_ahrs](https://www.reddit.com/r/linuxmasterrace/comments/7aai76/i_am_using_archlinux/dp94r3s/) 155 | for the inspiration! 156 | 157 | * [Xin-Xin Wang](https://github.com/xinxinw1) for helping me review the project. 158 | -------------------------------------------------------------------------------- /deploy.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | git config user.name "$(git show -s --format='%an')" 4 | git config user.email "$(git show -s --format='%ae')" 5 | git remote add deploy "git@github.com:$TRAVIS_REPO_SLUG" 6 | git checkout --orphan "$PUSH_BRANCH" 7 | git rm --cached -rf . 8 | git add -f repo/ 9 | git commit -m "Repo from commit $TRAVIS_COMMIT" 10 | 11 | eval $(ssh-agent) 12 | ssh-add deploy_key 13 | git push -f deploy "$PUSH_BRANCH" 14 | -------------------------------------------------------------------------------- /docker/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM archlinux/base 2 | 3 | ARG USER_ID=1000 4 | ENV BUILD_USER builder 5 | ENV BUILD_HOME /home/"$BUILD_USER" 6 | ENV BUILD_DIR "$BUILD_HOME"/aur-build 7 | 8 | RUN pacman -Syyu --noconfirm --needed base base-devel git 9 | 10 | RUN mkdir -p "$BUILD_DIR" 11 | RUN useradd -u "$USER_ID" -s /bin/bash -d "$BUILD_HOME" -G wheel "$BUILD_USER" 12 | RUN chmod -R 777 $BUILD_HOME 13 | RUN chown -R $BUILD_USER $BUILD_HOME 14 | RUN echo '%wheel ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers 15 | 16 | VOLUME $BUILD_DIR 17 | WORKDIR $BUILD_DIR 18 | USER $BUILD_USER 19 | 20 | CMD ./install.sh 21 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | source settings.conf 6 | 7 | REPO_DIR="$PWD"/repo 8 | AUTO_GPG='gpg --batch --no-tty --yes' 9 | 10 | if [ -z "$REPO_NAME" ] 11 | then 12 | REPO_NAME=local 13 | fi 14 | 15 | export GPGKEY="$sign_key" 16 | export MAKEPKG_CONF="$PWD"/makepkg.conf 17 | export PACKAGER="$(git show -s --format='%an <%ae>')" 18 | 19 | sudo pacman -Syu --noconfirm 20 | 21 | if [ ! -z "$GPGKEY" ] 22 | then 23 | $AUTO_GPG --import package-key.pgp 24 | $AUTO_GPG --export --output public-package-key.pgp "$GPGKEY" 25 | 26 | sudo pacman-key --init 27 | sudo pacman-key -a public-package-key.pgp 28 | sudo pacman-key --lsign-key "$GPGKEY" 29 | 30 | SIGNING_ARGS="--sign" 31 | fi 32 | 33 | $AUTO_GPG --recv-keys "${trustedpgpkeys[@]}" 34 | for k in "${trustedpgpkeys[@]}" 35 | do 36 | $AUTO_GPG --lsign-key "$k" || : 37 | done 38 | 39 | mkdir -p "$REPO_DIR" 40 | for p in "${packages[@]}" 41 | do 42 | pushd "$p" 43 | makepkg -si $SIGNING_ARGS --noconfirm || : 44 | popd 45 | done 46 | 47 | repo-add -s "$REPO_DIR"/"$REPO_NAME".db.tar.xz "$REPO_DIR"/*.pkg.tar.xz 48 | -------------------------------------------------------------------------------- /makepkg.conf: -------------------------------------------------------------------------------- 1 | # 2 | # makepkg.conf 3 | # 4 | 5 | ######################################################################### 6 | # SOURCE ACQUISITION 7 | ######################################################################### 8 | # 9 | #-- The download utilities that makepkg should use to acquire sources 10 | # Format: 'protocol::agent' 11 | DLAGENTS=('ftp::/usr/bin/curl -fC - --ftp-pasv --retry 3 --retry-delay 3 -o %o %u' 12 | 'http::/usr/bin/curl -fLC - --retry 3 --retry-delay 3 -o %o %u' 13 | 'https::/usr/bin/curl -fLC - --retry 3 --retry-delay 3 -o %o %u' 14 | 'rsync::/usr/bin/rsync --no-motd -z %u %o' 15 | 'scp::/usr/bin/scp -C %u %o') 16 | 17 | # Other common tools: 18 | # /usr/bin/snarf 19 | # /usr/bin/lftpget -c 20 | # /usr/bin/wget 21 | 22 | #-- The package required by makepkg to download VCS sources 23 | # Format: 'protocol::package' 24 | VCSCLIENTS=('bzr::bzr' 25 | 'git::git' 26 | 'hg::mercurial' 27 | 'svn::subversion') 28 | 29 | ######################################################################### 30 | # ARCHITECTURE, COMPILE FLAGS 31 | ######################################################################### 32 | # 33 | CARCH="x86_64" 34 | CHOST="x86_64-pc-linux-gnu" 35 | 36 | #-- Compiler and Linker Flags 37 | # -march (or -mcpu) builds exclusively for an architecture 38 | # -mtune optimizes for an architecture, but builds for whole processor family 39 | CPPFLAGS="-D_FORTIFY_SOURCE=2" 40 | CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt" 41 | CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt" 42 | LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now" 43 | #-- Make Flags: change this for DistCC/SMP systems 44 | MAKEFLAGS="-j2" 45 | #-- Debugging flags 46 | DEBUG_CFLAGS="-g -fvar-tracking-assignments" 47 | DEBUG_CXXFLAGS="-g -fvar-tracking-assignments" 48 | 49 | ######################################################################### 50 | # BUILD ENVIRONMENT 51 | ######################################################################### 52 | # 53 | # Defaults: BUILDENV=(!distcc color !ccache check !sign) 54 | # A negated environment option will do the opposite of the comments below. 55 | # 56 | #-- distcc: Use the Distributed C/C++/ObjC compiler 57 | #-- color: Colorize output messages 58 | #-- ccache: Use ccache to cache compilation 59 | #-- check: Run the check() function if present in the PKGBUILD 60 | #-- sign: Generate PGP signature file 61 | # 62 | BUILDENV=(!distcc color !ccache check !sign) 63 | # 64 | #-- If using DistCC, your MAKEFLAGS will also need modification. In addition, 65 | #-- specify a space-delimited list of hosts running in the DistCC cluster. 66 | #DISTCC_HOSTS="" 67 | # 68 | #-- Specify a directory for package building. 69 | #BUILDDIR=/tmp/makepkg 70 | 71 | ######################################################################### 72 | # GLOBAL PACKAGE OPTIONS 73 | # These are default values for the options=() settings 74 | ######################################################################### 75 | # 76 | # Default: OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !optipng !upx !debug) 77 | # A negated option will do the opposite of the comments below. 78 | # 79 | #-- strip: Strip symbols from binaries/libraries 80 | #-- docs: Save doc directories specified by DOC_DIRS 81 | #-- libtool: Leave libtool (.la) files in packages 82 | #-- staticlibs: Leave static library (.a) files in packages 83 | #-- emptydirs: Leave empty directories in packages 84 | #-- zipman: Compress manual (man and info) pages in MAN_DIRS with gzip 85 | #-- purge: Remove files specified by PURGE_TARGETS 86 | #-- upx: Compress binary executable files using UPX 87 | #-- optipng: Optimize PNG images with optipng 88 | #-- debug: Add debugging flags as specified in DEBUG_* variables 89 | # 90 | OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !optipng !upx !debug) 91 | 92 | #-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512 93 | INTEGRITY_CHECK=(md5) 94 | #-- Options to be used when stripping binaries. See `man strip' for details. 95 | STRIP_BINARIES="--strip-all" 96 | #-- Options to be used when stripping shared libraries. See `man strip' for details. 97 | STRIP_SHARED="--strip-unneeded" 98 | #-- Options to be used when stripping static libraries. See `man strip' for details. 99 | STRIP_STATIC="--strip-debug" 100 | #-- Manual (man and info) directories to compress (if zipman is specified) 101 | MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info}) 102 | #-- Doc directories to remove (if !docs is specified) 103 | DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc}) 104 | #-- Files to be removed from all packages (if purge is specified) 105 | PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod) 106 | 107 | ######################################################################### 108 | # PACKAGE OUTPUT 109 | ######################################################################### 110 | # 111 | # Default: put built package and cached source in build directory 112 | # 113 | #-- Destination: specify a fixed directory where all packages will be placed 114 | PKGDEST=/home/builder/aur-build/repo 115 | #-- Source cache: specify a fixed directory where source files will be cached 116 | #SRCDEST=/home/sources 117 | #-- Source packages: specify a fixed directory where all src packages will be placed 118 | #SRCPKGDEST=/home/srcpackages 119 | #-- Log files: specify a fixed directory where all log files will be placed 120 | #LOGDEST=/home/makepkglogs 121 | #-- Packager: name/email of the person or organization building packages 122 | #PACKAGER="John Doe " 123 | #-- Specify a key to use for package signing 124 | #GPGKEY="" 125 | 126 | ######################################################################### 127 | # COMPRESSION DEFAULTS 128 | ######################################################################### 129 | # 130 | COMPRESSGZ=(gzip -c -f -n) 131 | COMPRESSBZ2=(bzip2 -c -f) 132 | COMPRESSXZ=(xz -c -z -) 133 | COMPRESSLRZ=(lrzip -q) 134 | COMPRESSLZO=(lzop -q) 135 | COMPRESSZ=(compress -c -f) 136 | 137 | ######################################################################### 138 | # EXTENSION DEFAULTS 139 | ######################################################################### 140 | # 141 | # WARNING: Do NOT modify these variables unless you know what you are 142 | # doing. 143 | # 144 | PKGEXT='.pkg.tar.xz' 145 | SRCEXT='.src.tar.gz' 146 | 147 | # vim: set ft=sh ts=2 sw=2 et: 148 | -------------------------------------------------------------------------------- /settings.conf: -------------------------------------------------------------------------------- 1 | packages=() 2 | trustedpgpkeys=() 3 | #sign_key= 4 | --------------------------------------------------------------------------------