├── README.pod ├── fakepkg └── man └── fakepkg.1 /README.pod: -------------------------------------------------------------------------------- 1 | =encoding utf8 2 | 3 | =head1 NAME 4 | 5 | fakepkg - reassemble Arch Linux packages 6 | 7 | =head1 SYNOPSIS 8 | 9 | Usage: B [ OPTIONS ] [ TARGETS ] 10 | 11 | =head1 DESCRIPTION 12 | 13 | Pacman usually stores all downloaded packages in its package cache to enable the user 14 | to downgrade each and every package. However if the package cache was removed either 15 | by accident or by purpose to e.g. save disk space, it may be useful to recreate the 16 | cache at a later point. The obvious choice would be to download all those packages again. 17 | However this is actually redundant since all files and folders from the packages should 18 | reside on your system. This is where fakepkg comes in handy. Its purpose is to recreate 19 | any given package as long as it is installed on your system. This probably saves you a 20 | lot of bandwith and time in comparison to building the package from source or downloading 21 | it from third parties. 22 | 23 | B The files from your system are taken as they are, hence any modifications 24 | done to them by you or any program will be present in the assembled package. This might 25 | be seen as either downside or benefit. However distributing the recreated package is 26 | therefore discouraged. See ABS and Arch Rollback Machine for alternatives. 27 | 28 | B Certain files and folders may only be read by root. 29 | Though fakepkg will warn you in those cases, consider running it as root. 30 | 31 | B Reassembling packages may take some time, so be patient. 32 | However you may speed things up a little by using B<-j> or B. 33 | 34 | =head1 OPTIONS 35 | 36 | =over 4 37 | 38 | =item B<-h>,B<--help> 39 | 40 | Display this help message and exit 41 | 42 | =item B<-v>,B<--verbose> 43 | 44 | Increase verbosity 45 | 46 | =item B<-j>,B<--jobs > 47 | 48 | Build in parallel - you may want to set XZ_OPT / ZSTD_NBTHREADS 49 | 50 | =item B<-o>,B<--out > 51 | 52 | Write output to 53 | 54 | =item B<-r>,B<--root > 55 | 56 | Use as an alternate root directory containing pacman db 57 | 58 | =back 59 | 60 | =head1 ENVIRONMENT VARIABLES 61 | 62 | B sets the file ending of the reassembled package and implicitly the 63 | compression format. 64 | 65 | B are honored by tar. Common usage 66 | scenarios are to utilize multicore compression. Maximum performace was achieved 67 | when used in conjunction with two jobs of fakepkg. However more than two active 68 | jobs may even lead to performance loss. 69 | 70 | =head1 EXAMPLES 71 | 72 | To reassemble e.g. gzip and binutils, use the following command: 73 | 74 | fakepkg gzip binutils 75 | 76 | You may as well specify a destination folder: 77 | 78 | fakepkg -o /var/cache/pacman/pkg tar xz 79 | 80 | Recreating multiple packages at once with only one thread can take some time. This 81 | can be sped up by utilizing multiple jobs, in this case 4: 82 | 83 | fakepkg -j 4 linux slurm-llnl virtualbox qemu 84 | 85 | Alternatively it may also make sense to set the tar environment variable for multi 86 | core compression. Hereby tar uses all the CPU power hence starting multiple jobs brings no performance gain: 87 | 88 | PKGEXT=".pkg.tar.xz" XZ_OPT="-T 0" fakepkg $(pacman -Qsq) 89 | 90 | =head1 AUTHOR 91 | 92 | Gordian Edenhofer 93 | 94 | =head1 LICENSE 95 | 96 | Unless otherwise stated, the files in this project may be distributed under the 97 | terms of the GNU General Public License as published by the Free Software Foundation; 98 | either version 2 of the License, or any later version. This work is distributed 99 | in the hope that it will be useful, but without any warranty; without even the 100 | implied warranty of merchantability or fitness for a particular purpose. See [version 2] 101 | (https://www.gnu.org/licenses/old-licenses/gpl-2.0.html) and [version 3] 102 | (https://www.gnu.org/copyleft/gpl-3.0.html) of the GNU General Public License for more details. 103 | -------------------------------------------------------------------------------- /fakepkg: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # fakepkg: Reassemble installed packages from its delivered files 4 | # 5 | 6 | # Strictly disallow uninitialized variables and exit on command failures (even in pipes) 7 | set -Eeuo pipefail 8 | 9 | unset LC_ALL 10 | export LANG=C 11 | 12 | declare -r myname='fakepkg' 13 | declare -r myver='1.42.2' 14 | 15 | # Directory in /tmp on which basis mktemp will create an unique tmp for each fakebuild 16 | # Make this variable something rather unique since on cleanup anything starting with this will be erased 17 | declare -r tmp_root='fakepkg' 18 | 19 | # Standard path where pacman db resides 20 | declare -r db_path='/var/lib/pacman' 21 | 22 | # Error codes 23 | declare -r ERROR_USAGE=1 24 | declare -r ERROR_OPERATION=2 25 | 26 | # Actual assembly function 27 | # $1: pkgname , $2: destination dir , $3 verbosity 28 | function fakebuild() { 29 | PKG_VER=$(pacman --dbpath "${ROOT_DIR:-/}${db_path}" -Qi "${1}" 2>/dev/null | awk '/^Version/{print $3}') 30 | # Setting global vars 31 | PKG_NAME="${1}-${PKG_VER}" 32 | PKG_DB_ENTRY="${ROOT_DIR:-/}${db_path}/local/${PKG_NAME}" 33 | PKGEXT="${PKGEXT:-.pkg.tar.zst}" # implicitly sets the compression format 34 | local _MTREE="${PKG_DB_ENTRY#/}/mtree" 35 | local _MIMETYPE 36 | 37 | # Skip if the package already exists in the destination dir 38 | if [[ -f "${2}/${PKG_NAME}${PKGEXT}" ]]; then 39 | >&2 echo -e "\e[39;1m${PKG_NAME}${PKGEXT}\e[0m already exists in ${2}! Skipping..." 40 | return ${ERROR_USAGE} 41 | fi 42 | 43 | # Create and enter a temporary dir 44 | TMPDIR=$(mktemp -q -p /tmp -t -d "${tmp_root}".XXX) 45 | cd "${TMPDIR}" || return ${ERROR_OPERATION} 46 | 47 | # Fetching the .PKGINFO, .Changelog, .MTREE and .INSTALL file 48 | if [[ -f "${PKG_DB_ENTRY}/desc" ]]; then 49 | cp -a "${PKG_DB_ENTRY}/desc" pre_PKGINFO 50 | else 51 | >&2 echo -e "Could not find a database entry for \e[39;1m${PKG_NAME}\e[0m" 52 | return ${ERROR_OPERATION} 53 | fi 54 | [[ -f "${PKG_DB_ENTRY}/install" ]] && cp -a "${PKG_DB_ENTRY}/install" .INSTALL 55 | 56 | if [[ -f "${_MTREE}" ]]; then 57 | _MIMETYPE=$(file --mime-type "${PKG_DB_ENTRY}/mtree") 58 | if [[ "${_MIMETYPE}" == *"application/gzip"* ]]; then 59 | gzip -dc "${_MTREE}" > .MTREE 60 | elif [[ "${_MIMETYPE}" == *"text/plain"* ]]; then 61 | cp "${_MTREE}" .MTREE 62 | fi 63 | fi 64 | [[ -f "${PKG_DB_ENTRY}/changelog" ]] && cp -a "${PKG_DB_ENTRY}/changelog" .Changelog 65 | 66 | # Hacking together the .PKGINFO 67 | sed -n -e '/%BACKUP%/,$p' "${PKG_DB_ENTRY}/files" | awk '{print $1}' >> pre_PKGINFO 68 | sed -i '/^$/d' pre_PKGINFO 69 | while read -r line; do 70 | if [[ $line =~ %[A-Z]+% ]]; then 71 | PKG_VAR=$(echo "${line,,}" | tr -d '%') 72 | case "$PKG_VAR" in 73 | "name") PKG_VAR=pkgname;; 74 | "version") PKG_VAR=pkgver;; 75 | "desc") PKG_VAR=pkgdesc;; 76 | "groups") PKG_VAR=group;; 77 | "depends") PKG_VAR=depend;; 78 | "optdepends") PKG_VAR=optdepend;; 79 | "conflicts") PKG_VAR=conflict;; 80 | esac 81 | else 82 | echo -e "$PKG_VAR = $line" >> .PKGINFO 83 | fi 84 | done < pre_PKGINFO 85 | sed -i '/installdate/d' .PKGINFO 86 | sed -i '/validation/d' .PKGINFO 87 | 88 | # Assembling a list of all files belonging to the package 89 | # The tar command is unable to handle very many arguments therefore create a file 90 | pacman --dbpath "${ROOT_DIR:-/}${db_path}" -Qlq "$1" | sed 's%^/%%' > ./package_file_list 91 | aux_files=(".PKGINFO") 92 | [[ -f ".INSTALL" ]] && aux_files+=(".INSTALL") 93 | [[ -f ".Changelog" ]] && aux_files+=(".Changelog") 94 | [[ -f ".MTREE" ]] && aux_files+=(".MTREE") 95 | 96 | # Taring things together 97 | if [[ $3 -ge "1" ]]; then 98 | #XZ_OPT="-T 0" \ 99 | tar --ignore-failed-read --owner=0 --group=0 --no-recursion -c -a \ 100 | --force-local -f "${PKG_NAME}${PKGEXT}" \ 101 | -C "${ROOT_DIR:-/}" -T ./package_file_list -C "$TMPDIR" \ 102 | "${aux_files[@]}" 103 | else 104 | #XZ_OPT="-T 0" \ 105 | if tar --ignore-failed-read --owner=0 --group=0 --no-recursion -c -a \ 106 | --force-local -f "${PKG_NAME}${PKGEXT}" \ 107 | -C "${ROOT_DIR:-/}" -T ./package_file_list -C "$TMPDIR" \ 108 | "${aux_files[@]}" 2>&1 >/dev/null \ 109 | | grep -i "permission denied" >/dev/null -; then 110 | 111 | >&2 echo -e "The permission to open one or more directories was denied for \e[39;1m${PKG_NAME}\e[0m. The package may be incomplete!" 112 | fi 113 | fi 114 | 115 | # Cleanup 116 | if [[ -f ${PKG_NAME}${PKGEXT} ]]; then 117 | mv "${PKG_NAME}${PKGEXT}" "$2" 118 | >&2 echo -e "Reassembled \e[39;1m${PKG_NAME}\e[0m successfully!" 119 | else 120 | >&2 echo -e "Internal \e[39;1merror\e[0m occurred while processing \e[39;1m${PKG_NAME}\e[0m!" 121 | fi 122 | rm -rf "${TMPDIR}" 123 | } 124 | 125 | # Run fakebuild in parallel with a limit of $MAX_JOBS jobs 126 | # By default only run one job 127 | MAX_JOBS=1 128 | function parallelize() { 129 | while [[ $# -gt 0 ]] ; do 130 | mapfile -t job_count < <(jobs -p) 131 | if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then 132 | fakebuild "$1" "$DEST_DIR" "$VERBOSITY" & 133 | shift 134 | fi 135 | done 136 | wait 137 | } 138 | 139 | # Print usage information, no arguments required 140 | function usage() { 141 | cat <<-EOF 142 | $myname $myver 143 | $myname reassembles installed packages from its delivered files. It comes in 144 | handy if there is no internet connection available and you have no access to 145 | an up-to-date package cache. 146 | 147 | Usage: $myname [-v] [-j ] [-o ] 148 | -h, --help Display this help message and exit 149 | -v, --verbose Increase verbosity 150 | -j, --jobs Build in parallel - you may want to set XZ_OPT 151 | -o, --out Write output to 152 | -r, --root Use an alternate root directory containing pacman db 153 | 154 | $myname honors the environment variable \`PKGEXT\`. Use it to set the 155 | compression scheme. 156 | 157 | Examples: # $myname slurm-llnl 158 | # $myname gzip munge binutils -o ~/Downloads 159 | # $myname -o /tmp -j 5 gzip munge binutils 160 | # PKGEXT=".pkg.tar.xz" $myname -o /tmp -r /old/root legacy-package 161 | # $myname \$(pacman -Qsq) 162 | 163 | Copyright (C) Gordian Edenhofer 164 | The script requires bash>=4.2, pacman, tar and gzip. 165 | EOF 166 | } 167 | 168 | function version() { 169 | echo "$myname $myver" 170 | echo "Copyright (C) Gordian Edenhofer " 171 | echo "Copyright (C) 2008-2016 Pacman Development Team " 172 | } 173 | 174 | # Clean up temporary dirs recursively 175 | function clean_up { 176 | rm -r /tmp/"${tmp_root}".* 177 | echo 178 | exit 179 | } 180 | 181 | # Trap termination signals 182 | trap clean_up SIGHUP SIGINT SIGTERM 183 | 184 | # Show usage and quit if invoked with no parameters 185 | [[ $# -eq 0 ]] && usage && exit ${ERROR_USAGE} 186 | for ARG in "$@"; do 187 | [[ $ARG == "-h" || $ARG == "--help" ]] && usage && exit 0 188 | done 189 | 190 | # Assembling PKG_LIST and DEST_DIR 191 | if ! PARAMS=$(getopt -o o:j:r:vV --long out:,jobs:,root:,verbose,version -n "$myname" -- "$@"); then 192 | >&2 echo "Try '$myname --help' for more information." 193 | exit ${ERROR_USAGE} 194 | fi 195 | eval set -- "$PARAMS" 196 | DEST_DIR=$PWD # Default value 197 | VERBOSITY=0 # Default value 198 | ROOT_DIR="" # Default value 199 | 200 | while true ; do 201 | case "$1" in 202 | -o|--out) 203 | if ! DEST_DIR="$(readlink -e "$2")"; then 204 | >&2 echo -e "The directory \e[39;1m$2\e[0m does not exist!" 205 | exit ${ERROR_USAGE} 206 | fi 207 | shift 208 | ;; 209 | 210 | -j|--jobs) 211 | if [[ $2 =~ ^-?[0-9]+$ ]]; then 212 | MAX_JOBS=$2 213 | else 214 | >&2 echo -e "\e[39;1m$2\e[0m is not a valid integer!" 215 | exit ${ERROR_USAGE} 216 | fi 217 | shift 218 | ;; 219 | 220 | -r|--root) 221 | ROOT_DIR=$2 222 | [[ ! -d "${ROOT_DIR}" ]] && >&2 echo -e "The specified alternate root directory does not exist!" && exit ${ERROR_USAGE} 223 | ROOT_DIR=$(readlink -e "${ROOT_DIR}") 224 | shift 225 | ;; 226 | 227 | -v|--verbose) 228 | VERBOSITY=$((VERBOSITY+1)) 229 | ;; 230 | 231 | -V|--version) 232 | version 233 | exit 0 234 | ;; 235 | 236 | --) 237 | shift 238 | break 239 | ;; 240 | 241 | *) 242 | usage 243 | exit ${ERROR_USAGE} 244 | ;; 245 | esac 246 | shift 247 | done 248 | PKG_LIST=("$@") 249 | 250 | # Checking the PKG_LIST 251 | PKG_NOTFOUND=() 252 | [[ ${#PKG_LIST[@]} == 0 ]] && usage && exit ${ERROR_USAGE} 253 | for package in "${PKG_LIST[@]}"; do 254 | ver=$(pacman --dbpath "${ROOT_DIR:-/}${db_path}" -Qi "${package}" 2>/dev/null | awk '/^Version/{print $3}') 255 | if [[ ! -d "${ROOT_DIR:-/}${db_path}/local/${package}-${ver}" ]]; then 256 | PKG_NOTFOUND+=("\"${package}\" ") 257 | fi 258 | done 259 | if [[ ${#PKG_NOTFOUND[@]} -ne 0 ]]; then 260 | >&2 echo -e "The following package(s) could not be found on your system: \e[39;1m${PKG_NOTFOUND[*]}\e[0m" 261 | >&2 echo -e "Keep in mind that the package(s) have to be installed!" 262 | exit ${ERROR_USAGE} 263 | fi 264 | 265 | # Assembling the packages 266 | parallelize "${PKG_LIST[@]}" 267 | 268 | exit 0 269 | -------------------------------------------------------------------------------- /man/fakepkg.1: -------------------------------------------------------------------------------- 1 | .TH FAKEPKG "1" "Aug 2021" "NONE" "User Commands" 2 | .SH NAME 3 | fakepkg \- reassemble Arch Linux packages 4 | .SH SYNOPSIS 5 | .B fakepkg 6 | [\fIOPTION\fR]... [\fIPACKAGE\fR]... 7 | .SH DESCRIPTION 8 | This script was designed to reassemble installed packages from its deliverd files. 9 | It comes in handy if there is no internet connection available and you have no 10 | access to a up-to-date package cache. 11 | .br 12 | Reassembling packages may take some time, so be patient. However you may speed 13 | things up a little by using 14 | .B \-j 15 | or 16 | .B XZ_OPT / ZSTD_NBTHREADS 17 | \ . 18 | .SH OPTIONS 19 | .TP 20 | .BR \-h , "\-\-help" 21 | Display this help message and exit 22 | .TP 23 | .BR \-v , "\-\-verbose" 24 | Increase verbosity 25 | .TP 26 | .BR \-j , "\-\-jobs " 27 | Build in parallel - you may want to set XZ_OPT / ZSTD_NBTHREADS 28 | .TP 29 | .BR \-o , "\-\-out " 30 | Write output to 31 | .TP 32 | .BR \-r , "\-\-root " 33 | Use as an alternate root directory containing pacman db 34 | .SH ENVIRONMENT VARIABLES 35 | .B PKGEXT 36 | sets the file ending of the reassembled package and implicitly the compression 37 | format. 38 | 39 | .B XZ_OPT, ZSTD_CLEVEL, ZSTD_NBTHREADS, ... 40 | are honored by tar. Common usage scenarios are to utilize multicore 41 | compression. Maximum performace was achieved when used in conjunction with two 42 | jobs of fakepkg. However more than two active jobs may even lead to performance 43 | loss. 44 | .SH EXAMPLES 45 | Reassamble the package named "tar" 46 | 47 | .ti 12 48 | .B fakepkg 49 | tar 50 | 51 | Reassamble the packages "gzip" and "pacman" and place the packages in ~/Downloads 52 | 53 | .ti 12 54 | .B fakepkg 55 | -o ~/Downloads gzip pacman 56 | 57 | Reassamble a bunch of packages in parallel with verbose output 58 | 59 | .ti 12 60 | .B fakepkg 61 | -j 2 -v gzip munge binutils 62 | 63 | Recreate all installed packages with xz's multicore compression 64 | 65 | .ti 12 66 | PKGEXT=".pkg.tar.xz" XZ_OPT="-T 0" 67 | .B fakepkg 68 | $(pacman -Qsq) 69 | 70 | .SH "BUGS" 71 | .sp 72 | Bugs? You must be kidding; there are no bugs in this software\&. 73 | But if I happen to be wrong, send me an email with as much detail as possible 74 | to gordian.edenhofer@gmail.com. 75 | 76 | .SH AUTHOR 77 | Written by Gordian Edenhofer. 78 | .SH COPYRIGHT 79 | Copyright \(co 2015 Gordian Edenhofer 80 | License GPLv2+: GNU GPL version 2 or later . 81 | 82 | .br 83 | This is free software: you are free to change and redistribute it. 84 | There is NO WARRANTY, to the extent permitted by law. 85 | .SH "SEE ALSO" 86 | .B pacman, 87 | .B tar, 88 | .B xz, 89 | .B zstd 90 | --------------------------------------------------------------------------------