├── .gitignore ├── LICENSE ├── README.md └── apkfastestmirror.sh /.gitignore: -------------------------------------------------------------------------------- 1 | # Created by .ignore support plugin (hsz.mobi) 2 | ### Example user template template 3 | ### Example user template 4 | 5 | # IntelliJ project files 6 | .idea 7 | *.iml 8 | out 9 | gen 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2015 Travis Wichert 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # APK Fastest Mirror 2 | *A Pure BusyBox APK mirror selector for Alpine Linux* 3 | 4 | MIT License (See: https://github.com/padthaitofuhot/apkfastestmirror) 5 | 6 | ## Installation 7 | ```text 8 | alpine:~# wget https://raw.githubusercontent.com/padthaitofuhot/apkfastestmirror/master/apkfastestmirror.sh 9 | alpine:~# ash ./apkfastestmirror.sh --install 10 | Installing to /usr/local/bin/apkfastestmirror 11 | Generating /etc/apk/apkfastestmirror.conf 12 | alpine:~# 13 | ``` 14 | 15 | ## Usage 16 | ```text 17 | Usage 18 | -h, --help Show usage and exit 19 | -q, --quiet Be quiet - only print results 20 | --shut-up Be more quiet - print nothing 21 | -v, --verbose Be verbose - print progress (default) 22 | --http-only Only use HTTP checks to determine mirror performance 23 | --icmp-only Only use ICMP checks to determine mirror performance 24 | -s, --samples Test performance n times before selecting mirror 25 | the default is two samples. 26 | -r, --replace Replace /etc/apk/repositories with fastest mirror 27 | -u, --update Update APK indexes (default, useful with --replace) 28 | --genconf Create /etc/apk/fastestmirror.conf if it does not 29 | already exist and exit 30 | --install Install as /usr/local/bin/apkfastestmirror and exit 31 | ``` 32 | 33 | ## Configuration 34 | All variables that can be set in the script header can also be set in the config file: `/etc/apk/apkfastestmirror.conf` 35 | 36 | ##### Behavior Variables 37 | 38 | | Variable | Default | Description 39 | | :--- | :---: | :--- | 40 | | `replace_apk_repositories` | true | Replace /etc/apk/repositories with output | 41 | | `update_indexes_after_replace` | false | (with `--replace`) Also update APK indexes | 42 | | `verbose` | true | Be verbose (and get a progress bar) | 43 | | `quiet` | false | Be quiet - only print results | 44 | | `shut_up` | false | Be more quiet - print nothing | 45 | 46 | ##### Mirror Variables 47 | 48 | | Variable | Default | Description 49 | | :--- | :---: | :--- | 50 | | `mirrors_custom` | *-* | Your custom HTTP mirrors, perhaps local mirrors | 51 | | `mirrors_published` | *see script* | Published mirrors that are usually available 52 | 53 | ##### Measurement Variables 54 | 55 | | Variable | Default | Description 56 | | :--- | :---: | :--- | 57 | | `http_only` | false | Got firewall problems? Maybe these can help. | 58 | | `icmp_only` | false | | 59 | | `sampling_rounds` | 2 | Sampling rounds before selecting mirror 60 | | `icmp_count` | 6 | Number of ICMP pings to send | 61 | | `http_timeout` | 1 | HTTP timeout in seconds (useful on laggy links) 62 | | `http_use_proxy` | on | "on" or "off"; defaults to "on", but does nothing unless the $http_proxy environment variable is set | 63 | 64 | ### Variable Precedence 65 | 66 | *command line > config file > script header* 67 | 68 | If the config file exists, its variables override the script. If arguments are given on the command line, they override the config file. 69 | 70 | ## Regarding Proxies 71 | 72 | When using proxies, you will want to set `sampling_rounds` to something > 1 to prime the proxy's cache. If you do not, a suboptimal mirror may be selected if, for example, the proxy gets a cache hit on a slower mirror and a cache miss on the mirror apkfastestmirror would have otherwise selected. 73 | -------------------------------------------------------------------------------- /apkfastestmirror.sh: -------------------------------------------------------------------------------- 1 | #!/bin/ash 2 | ############################################################################## 3 | # APK Fastest Mirror 0.9.6 - A Pure BusyBox APK mirror selector for Alpine 4 | # MIT License (See: https://github.com/padthaitofuhot/apkfastestmirror) 5 | # 6 | # Usage 7 | # -h, --help Show usage and exit 8 | # -q, --quiet Be quiet - only print results 9 | # --shut-up Be more quiet - print nothing 10 | # -v, --verbose Be verbose - print progress (default) 11 | # --http-only Only use HTTP checks to determine mirror performance 12 | # --icmp-only Only use ICMP checks to determine mirror performance 13 | # -s, --samples Test performance n times before selecting mirror 14 | # the default is two samples. 15 | # -r, --replace Replace /etc/apk/repositories with fastest mirror 16 | # -u, --update Update APK indexes (default, useful with --replace) 17 | # --genconf Create /etc/apk/fastestmirror.conf if it does not 18 | # already exist and exit 19 | # --install Install as /usr/local/bin/apkfastestmirror and exit 20 | #@############################################################################ 21 | # 22 | # All variables that can be set in the script header can also be set in 23 | # the config file: /etc/apk/apkfastestmirror.conf 24 | # 25 | # If the config file exists, its variables override the script. 26 | # If arguments are given on the command line, they override the config file. 27 | # 28 | ### Behavior Variables 29 | # 30 | # The contents of /etc/apk/repositories will be replaced whenever this script 31 | # runs if replace_apk_repositories is 'true'. 32 | replace_apk_repositories=true 33 | # 34 | # Run `apk update` if replace_apk_repositories=true or given the --replace 35 | # argument at runtime. 36 | update_indexes_after_replace=false 37 | # 38 | # Get a progress bar :) 39 | verbose=true 40 | # 41 | # Show the results 42 | show_results=false 43 | # 44 | ### Mirror Variables 45 | # 46 | # Your local HTTP mirrors 47 | mirrors_custom="" 48 | # 49 | # At least one published mirror that should generally be reachable 50 | mirrors_published=" 51 | http://dl-cdn.alpinelinux.org/alpine/ 52 | http://nl.alpinelinux.org/alpine/ 53 | http://uk.alpinelinux.org/alpine/ 54 | http://dl-2.alpinelinux.org/alpine/ 55 | http://dl-3.alpinelinux.org/alpine/ 56 | http://dl-4.alpinelinux.org/alpine/ 57 | http://dl-5.alpinelinux.org/alpine/ 58 | http://dl-8.alpinelinux.org/alpine/ 59 | http://mirror.yandex.ru/mirrors/alpine/ 60 | http://mirrors.gigenet.com/alpinelinux/ 61 | http://mirror1.hs-esslingen.de/pub/Mirrors/alpine/ 62 | http://mirror.leaseweb.com/alpine/ 63 | http://repository.fit.cvut.cz/mirrors/alpine/ 64 | http://alpine.mirror.far.fi/ 65 | http://alpine.mirror.wearetriple.com/ 66 | http://mirror.clarkson.edu/alpine/ 67 | http://linorg.usp.br/AlpineLinux/ 68 | http://ftp.yzu.edu.tw/Linux/alpine/ 69 | http://mirror.aarnet.edu.au/pub/alpine 70 | http://mirror.csclub.uwaterloo.ca/alpine 71 | http://ftp.acc.umu.se/mirror/alpinelinux.org 72 | http://ftp.halifax.rwth-aachen.de/alpine 73 | http://speglar.siminn.is/alpine 74 | http://mirrors.dotsrc.org/alpine 75 | http://ftp.tsukuba.wide.ad.jp/Linux/alpine 76 | http://mirror.rise.ph/alpine 77 | " 78 | # 79 | ### Measurement Variables 80 | # 81 | # Got firewall problems? Maybe these can help. 82 | icmp_only=false 83 | http_only=false 84 | # 85 | # Number of times to perform sampling before returning results 86 | sampling_rounds=2 87 | # 88 | # Number of times to ICMP ping each mirror host 89 | icmp_count=6 90 | # 91 | # Timeout in seconds when waiting for a mirror list (MIRRORS.txt) HTTP source 92 | # to respond before ignoring it for the current run 93 | http_timeout=1 94 | # 95 | # "on" or "off"; defaults to "on", but does nothing unless the $http_proxy 96 | # environment variable is set 97 | http_use_proxy=on 98 | # When using proxies, you will want to set `sampling_rounds` to something > 1 99 | # to prime the proxy's cache. If you do not, a suboptimal mirror may be 100 | # selected if, for example, the proxy gets a cache hit on a slower mirror and 101 | # a cache miss on the mirror apkfastestmirror would have otherwise selected. 102 | # 103 | #@@########################################################################### 104 | 105 | set -o pipefail 106 | 107 | if [ -f /etc/apk/apkfastestmirror.conf ]; then 108 | source /etc/apk/apkfastestmirror.conf 109 | fi 110 | 111 | usage() { 112 | sed '/^#@/{q}' $0 | egrep -v -E '#!|##|^$' 113 | } 114 | 115 | if ! O=$( 116 | getopt \ 117 | -l verbose,genconf,install,replace,update,help,quiet,samples:,shut-up,http-only,icmp-only \ 118 | -o vuhqrs: \ 119 | -n "${0}" \ 120 | -- ${@} 121 | ); then 122 | usage 123 | exit 1 124 | fi 125 | 126 | eval set -- "${O}" 127 | while true; do 128 | case "$1" in 129 | -v|--verbose) 130 | verbose=true 131 | shift 132 | ;; 133 | -q|--quiet) 134 | verbose=false 135 | show_results=true 136 | shift 137 | ;; 138 | --shut-up) 139 | verbose=false 140 | show_results=false 141 | shift 142 | ;; 143 | --genconf) 144 | if [ -f "/etc/apk/apkfastestmirror.conf" ]; then 145 | echo "Config file /etc/apk/apkfastestmirror.conf already exists." 146 | echo "Remove it and rerun with --genconf to regenerate." 147 | exit 1 148 | else 149 | echo "Generating /etc/apk/apkfastestmirror.conf" 150 | sed '/^#@@/{q}' $0 | egrep -v -E '#!|##|^$' >/etc/apk/apkfastestmirror.conf 151 | exit $? 152 | fi 153 | ;; 154 | -h|--help) 155 | usage 156 | exit 0 157 | ;; 158 | --install) 159 | echo "Installing to /usr/local/bin/apkfastestmirror" 160 | install -o root -g root -m 755 -cD -p "${0}" /usr/local/bin/apkfastestmirror || exit 1 161 | ash $0 --genconf 162 | exit $? 163 | ;; 164 | -r|--replace) 165 | replace_apk_repositories=true 166 | shift 167 | ;; 168 | -u|--update) 169 | update_indexes_after_replace=true 170 | shift 171 | ;; 172 | -s|--samples) 173 | sampling_rounds="${2}" 174 | shift 2 175 | ;; 176 | --icmp-only) 177 | icmp_only=true 178 | shift 179 | ;; 180 | --http-only) 181 | http_only=true 182 | shift 183 | ;; 184 | --) 185 | shift 186 | break 187 | ;; 188 | *) 189 | usage 190 | exit 1 191 | ;; 192 | esac 193 | done 194 | 195 | ############################################################################## 196 | 197 | # Assemble list of MIRRORS.txt HTTP sources 198 | mirrorlist_sources=" 199 | $( 200 | for mirror in ${mirrors_custom} ${mirrors_published}; do \ 201 | echo ${mirror}/MIRRORS.txt; \ 202 | done 203 | ) 204 | " 205 | 206 | # Fetch mirrorlists from sources 207 | $verbose && echo "Fetching mirror lists" 208 | mirrorlist=" 209 | ${mirrors_custom} 210 | $( 211 | for source in ${mirrorlist_sources}; do \ 212 | wget 2>/dev/null -O- -q -Y ${http_use_proxy} -T ${http_timeout} ${source}; \ 213 | done \ 214 | | sort \ 215 | | uniq \ 216 | | sed '/^$/d' 217 | ) 218 | " 219 | 220 | # Use time, $?, and wget -T to time http requests 221 | # to mirrors and filter those that don't respond. 222 | # Pipes into the icmp runner, providing the 223 | # url, host, and time wget took to access the mirror 224 | # in decimal seconds. 225 | parallel_http_ping() { 226 | url=$1 227 | 228 | if $icmp_only || fetchtime="$( 229 | time \ 230 | wget -O/dev/null -q \ 231 | -Y ${http_use_proxy} \ 232 | -T ${http_timeout} \ 233 | "${url}" \ 234 | 2>&1 \ 235 | )"; 236 | 237 | then 238 | $icmp_only && fetchtime=1 239 | echo -e "${url}\n${fetchtime}" | awk ' 240 | /^http/ { 241 | split($1, urlparts, "/") 242 | print $1 243 | print urlparts[3] 244 | } 245 | /^real/ { 246 | fetchtime = $2 * 60 + $3 247 | print fetchtime 248 | } 249 | ' 250 | $verbose && 1>&2 echo -n "#" 251 | else 252 | $verbose && 1>&2 echo -n "." 253 | return 1 254 | fi 255 | 256 | } 257 | 258 | # Pings the mirror a few times and calculates a weighted 259 | # speed and reliability metric from http fetchtime, packet 260 | # loss, and round-trip times. 261 | # 262 | # fetchtime = The time it took wget to contact a mirror, 263 | # request MIRRORS.txt, and receive it. 264 | # rtval = Round-trip min, max, and avg are summed to 265 | # represent jitter and link congestion. 266 | # plval = A dropped packet is weighted the same as the round- 267 | # trip sum, since that's roughly what would be lost 268 | # if TCP had to retransmit a packet 269 | # idx = The sum of rtval and plval, multiplied by 270 | # the fetchtime. It can happen that a host may respond 271 | # slowly to ICMP or TCP setup, but ultimately yields 272 | # a fast transfer. Conversely, local mirror on an 273 | # old 10baseT switch might ping and TCP handshake 274 | # fast, but may be slower to transfer than a remote 275 | # mirror accessible via a 10G OC transit. 276 | # By weighing against fetchtime, these scenarios are 277 | # passably accounted for. 278 | # 279 | # Once metrics are calculated for all reachable mirrors, 280 | # they are sorted by idx (lower is better) and the top 281 | # of the list is returned as the winning mirror. 282 | parallel_icmp_ping() { 283 | # since we're not using while-read, we should use 284 | # read -t here to avoid failure modes where we somehow 285 | # get called even though pipeline should have died. 286 | read -r -t 1 url 287 | read -r -t 1 host 288 | read -r -t 1 fetchtime 289 | 290 | if ( 291 | ( $http_only \ 292 | && echo -e "packet loss 0 0 0 0 0\nround-trip 0 0 0/0/1\n" 293 | ) \ 294 | || ping -q -c "${icmp_count}" -W 1 -w "${icmp_count}" "${host}" 2>/dev/null \ 295 | ) \ 296 | | awk -v url="${url}" \ 297 | -v host="${host}" \ 298 | -v fetchtime="${fetchtime}" ' 299 | /packet loss/ { 300 | loss = $7 301 | } 302 | /round-trip/ { 303 | split($4, s, "/") 304 | rtval = s[0] + s[1] + s[3] 305 | plval = loss * rtval 306 | idx = rtval + plval 307 | idx *= fetchtime 308 | print idx, host, url 309 | } 310 | '; 311 | then 312 | $verbose && 1>&2 echo -n "#" 313 | else 314 | $verbose && 1>&2 echo -n "." 315 | fi 316 | } 317 | 318 | # Look pretty 319 | mirror_count="$(set -- $mirrorlist; echo $#)" 320 | $verbose && echo "Determining fastest mirror" 321 | _foot=']' 322 | ## 323 | winners="$(mktemp)" 324 | for round in $(seq 1 "${sampling_rounds}"); do 325 | 326 | $verbose && _head="$(printf 'Sampling (%s/%s) [' ${round} ${sampling_rounds})" 327 | $verbose && _pad="$(( ${#_head} + ${#_foot} ))" 328 | $verbose && printf "%$(( ( mirror_count * 2 ) + _pad - 1 ))s%s\r%s" ' ' "${_foot}" "${_head}" 329 | 330 | reachable="$(mktemp)" 331 | 332 | for mirror in ${mirrorlist}; do 333 | parallel_http_ping ${mirror} | parallel_icmp_ping 1>>${reachable} & 334 | done 335 | wait 336 | 337 | grep -v '^0 $' "${reachable}" | sort -n -t ' ' -k 1 | head -1 >>"${winners}" 338 | 339 | rm -f "${reachable}" 340 | 341 | $verbose && printf "\r%$(( ( mirror_count * 2 ) + _pad + 1 ))s\r" 342 | done 343 | ## 344 | 345 | set -- $(sort -n -t ' ' -k 1 "${winners}" | head -1) 346 | rm -f "${winners}" 347 | 348 | if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then 349 | $verbose && echo "ERR: No hosts could be reached, not touching /etc/apk/repositories" 350 | exit 1 351 | fi 352 | 353 | # Spam to terminal and (if enabled) update the APK repo file 354 | output_results() { 355 | $replace_apk_repositories && new_repositories="$(mktemp)" 356 | while read -r line; do 357 | $replace_apk_repositories && echo "${line}" >>"${new_repositories}" 358 | $show_results && echo "${line}" 359 | done 360 | $replace_apk_repositories && mv -f "${new_repositories}" /etc/apk/repositories 361 | } 362 | 363 | v="$(source /etc/os-release; echo ${PRETTY_NAME} | cut -d ' ' -f 3)" 364 | echo "${3}" | awk -v v="${v}" ' 365 | { 366 | print $1 v "/main" 367 | print $1 v "/community" 368 | print "@edge_main " $1 "edge" "/main" 369 | print "@edge_community " $1 "edge" "/community" 370 | print "@edge_testing " $1 "edge" "/testing" 371 | } 372 | ' | output_results 373 | 374 | $verbose && echo "OK: ${2} (metric ${1})" 375 | 376 | $replace_apk_repositories && $update_indexes_after_replace && $verbose && apk update 377 | $replace_apk_repositories && $update_indexes_after_replace && ! $verbose && apk -q update 378 | 379 | exit 0 380 | 381 | # refs: 382 | # http://wiki.alpinelinux.org/wiki/Awk 383 | --------------------------------------------------------------------------------