├── .gitignore ├── README.md ├── custom.txt ├── gfw.sh └── util.sh /.gitignore: -------------------------------------------------------------------------------- 1 | release/*.pac 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | gfwlist2pac-shell 2 | ================= 3 | 4 | Translate the gfwlist in base64 to efficient pac file with X-level domain validation 5 | 6 | This project is mainly created by [@clowwindy](https://github.com/clowwindy/gfwlist2pac) via python 7 | 8 | This version is just an implementation of pure shell 9 | 10 | Generate O(1) PAC file from gfwlist. 11 | 12 | ### Dependency 13 | 14 | A socks5 proxy is needed to download the gfwlist. 15 | 16 | You can make one via `shadowsocks` or `ssh -D`, the `PROXY` will be used to generate the pac file as well, so it should be an commonly used local address. 17 | 18 | ```zsh 19 | # socks5 proxy ssh -D, shadowsocks or others 20 | PROXY="127.0.0.1:7070" 21 | ``` 22 | 23 | Beacuse of the limitation of shell, i migration to `zsh`, you can easily install with `apt-get`, `yum`, `pacman` in any kind of linux platform. 24 | 25 | The cli version of `curl`, `openssl` which was integrated into most popular linux version. 26 | 27 | You can specify the tools customly. 28 | 29 | ```zsh 30 | # curl & openssl cli command path 31 | CURL=/usr/bin/curl 32 | CURLOPT=(-s -x socks5://$PROXY) 33 | OPENSSL=/usr/bin/openssl 34 | ``` 35 | 36 | ### Custom 37 | You can add some domains to `custom.txt`, it will be added into the pac file automatically. 38 | One domain per-line. 39 | ``` 40 | github.com 41 | dropbox.com 42 | linode.com 43 | stackoverflow.com 44 | linost.com 45 | eigenlogik.com 46 | iceimg.com 47 | jetbrains.com 48 | instagram.com 49 | linkedin.com 50 | agilebits.com 51 | godaddy.com 52 | startssl.com 53 | btdigg.org 54 | digitalattackmap.com 55 | igvita.com 56 | apache.org 57 | jquery.com 58 | speedtest.net 59 | ``` 60 | 61 | ### Usage 62 | after all dependency ready 63 | 64 | Usage. `./gfw.sh [filename.pac]` 65 | 66 | Eg. `./gfw.sh release/proxy.pac` 67 | 68 | The proxy.pac will be generated to `release/proxy.pac` 69 | 70 | ### Performance 71 | 72 | An example of generated PAC file is [here] [1]. 73 | 74 | The PAC generated by GFWList2PAC is 1267x faster than SwitchySharp. 75 | 76 | Testing pac generated by gfwlist2pac 77 | total: 46.411584ms 78 | avg: 0.6706876300578034ns 79 | 80 | Testing pac generated by switchsharp 81 | total: 58828.813476ms 82 | avg: 850.1273623699423ns 83 | 84 | [1]: https://gist.github.com/cuber/7e1cb2864ec139236b59 85 | -------------------------------------------------------------------------------- /custom.txt: -------------------------------------------------------------------------------- 1 | github.com 2 | v2ex.com 3 | -------------------------------------------------------------------------------- /gfw.sh: -------------------------------------------------------------------------------- 1 | #! /bin/zsh 2 | 3 | # Generated by gfwlist2pac 4 | # created by @clowwindy via python 5 | # modified by @cube via native zsh 6 | # https://github.com/cuber/gfwlist2pac 7 | 8 | # url 9 | XLDURL="https://publicsuffix.org/list/effective_tld_names.dat" 10 | GFWURL="https://raw.githubusercontent.com/gfwlist/gfwlist/master/gfwlist.txt" 11 | 12 | # socks5 proxy ssh -D, shadowsocks or others 13 | PROXY="127.0.0.1:1080" 14 | 15 | # curl & openssl cli command path 16 | CURL=/usr/bin/curl 17 | CURLOPT=(-s -x socks5://$PROXY) 18 | OPENSSL=/usr/bin/openssl 19 | 20 | # get current dirname 21 | DIR=$(cd $(dirname $0); pwd) 22 | 23 | # load util functions 24 | source $DIR/util.sh 25 | typeset -A XLD 26 | typeset -A UNIQ 27 | for i in $($CURL $CURLOPT $XLDURL | grep -v -e '^\s*$' -e '^/'); do XLD[$i]=1; done 28 | 29 | # output pac file 30 | PAC=$1 31 | if [ -z $PAC ]; then 32 | echo "Usage. ./gfw.sh [filename.pac]" 33 | echo " Eg. ./gfw.sh release/proxy.pac" 34 | echo " The proxy.pac will be generated to release/proxy.pac" 35 | exit 1 36 | fi 37 | 38 | # make sure pac dirname exists 39 | PACDIR=$(dirname $PAC) 40 | if [ ! -d $PACDIR ]; then mkdir -p $PACDIR; fi 41 | 42 | cat > $PAC </dev/null \ 70 | | grep \ 71 | -e '\.' \ 72 | | grep -v \ 73 | -e '^\s*$' \ 74 | -e '^!' \ 75 | | valid 76 | 77 | DOMAINS=() 78 | for i in ${(k)UNIQ}; do DOMAINS+=(" '$i': 1"); done 79 | echo ${(j:,\n:)DOMAINS} >> $PAC 80 | 81 | cat >> $PAC <= 1); 98 | return direct; 99 | } 100 | EOF 101 | 102 | exit 0 103 | -------------------------------------------------------------------------------- /util.sh: -------------------------------------------------------------------------------- 1 | #! /bin/zsh 2 | 3 | function valid { 4 | local N 5 | local data 6 | local domain 7 | while read data; do 8 | domain=($(echo $data | tr . ' ')); 9 | N=$((${#domain[@]} - 1)) 10 | if ! exists ${(j:.:)domain:$((N--))}; then continue; fi 11 | for ((; N >= 0; N--)); do 12 | data=${(j:.:)domain:$N} 13 | if exists $data; then continue; fi 14 | UNIQ[$data]=1 15 | break 16 | done 17 | done 18 | } 19 | 20 | function exists { 21 | if [[ $XLD[$1] = 1 ]]; 22 | then return 0 23 | else return 1 24 | fi 25 | } 26 | 27 | function urlencode { 28 | local length="${#1}" 29 | for (( i = 0; i < length; i++ )); do 30 | local c="${1:i:1}" 31 | case $c in 32 | [a-zA-Z0-9.~_-]) printf "$c" ;; 33 | *) printf '%%%02X' "'$c" 34 | esac 35 | done 36 | } 37 | 38 | function urldecode { 39 | local data 40 | while read data; do 41 | local url_encoded="${data//+/ }" 42 | echo $(printf '%b' "${url_encoded//%/\x}") 43 | done 44 | } 45 | --------------------------------------------------------------------------------