├── test ├── uq_test_another_input ├── uq_test_input ├── chore │ ├── lint.sh │ ├── integration-test.sh │ └── bump-scripts-version.sh ├── self-installer.sh ├── my_unit_test_lib.sh ├── uq_test.sh └── parseOpts_test.sh ├── bin ├── console-text-color-themes ├── echo-args ├── xpf ├── tcp-connection-state-counter ├── a2l ├── taoc ├── coat ├── ap ├── c ├── xpl ├── rp ├── cp-into-docker-run ├── uq ├── find-in-jars ├── show-duplicate-java-classes └── show-busy-java-threads ├── docs ├── coat.png ├── logo.png ├── logo-social.png ├── script-icon.png ├── script-logo.png ├── logo-social-original.png ├── console-colorful-text.png ├── logo.meta.txt ├── install.md ├── developer-guide.md ├── shell.md └── java.md ├── .gitmodules ├── .editorconfig ├── .github ├── dependabot.yml └── workflows │ ├── lint.yaml │ └── ci.yaml ├── lib ├── console-text-color-themes.sh └── parseOpts.sh ├── .gitignore ├── README.md └── LICENSE /test/uq_test_another_input: -------------------------------------------------------------------------------- 1 | a 2 | m 3 | m 4 | x 5 | -------------------------------------------------------------------------------- /bin/console-text-color-themes: -------------------------------------------------------------------------------- 1 | ../lib/console-text-color-themes.sh -------------------------------------------------------------------------------- /test/uq_test_input: -------------------------------------------------------------------------------- 1 | c 2 | c 3 | v 4 | a 5 | a 6 | c 7 | c 8 | u 9 | -------------------------------------------------------------------------------- /docs/coat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/coat.png -------------------------------------------------------------------------------- /docs/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/logo.png -------------------------------------------------------------------------------- /docs/logo-social.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/logo-social.png -------------------------------------------------------------------------------- /docs/script-icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/script-icon.png -------------------------------------------------------------------------------- /docs/script-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/script-logo.png -------------------------------------------------------------------------------- /docs/logo-social-original.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/logo-social-original.png -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "test/shunit2"] 2 | path = test/shunit2-lib 3 | url = https://github.com/kward/shunit2.git 4 | -------------------------------------------------------------------------------- /docs/console-colorful-text.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oldratlee/useful-scripts/HEAD/docs/console-colorful-text.png -------------------------------------------------------------------------------- /docs/logo.meta.txt: -------------------------------------------------------------------------------- 1 | logo is created by https://www.logoly.pro 2 | 3 | font: Zilla Slab 4 | 5 | logo.fond-size: 60 6 | logo-social.fond-size: 160 7 | -------------------------------------------------------------------------------- /test/chore/lint.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -eEuo pipefail 3 | 4 | realpath() { 5 | [ -e "$1" ] && command realpath -- "$1" 6 | } 7 | 8 | # cd to the root of the project 9 | cd "$(dirname -- "$(realpath "${BASH_SOURCE[0]}")")"/../.. 10 | 11 | find bin lib -type f | 12 | grep -Pv '/show-duplicate-java-classes$' | 13 | grep -Pv '/\.editorconfig$' | 14 | xargs --verbose shellcheck --shell=bash 15 | -------------------------------------------------------------------------------- /.editorconfig: -------------------------------------------------------------------------------- 1 | root = true 2 | 3 | [*] 4 | charset = utf-8 5 | end_of_line = lf 6 | insert_final_newline = true 7 | tab_width = 4 8 | indent_style = space 9 | indent_size = 2 10 | trim_trailing_whitespace = true 11 | 12 | [*.{py,md,mkd,markdown}] 13 | indent_size = 4 14 | 15 | [*.xml] 16 | indent_style = tab 17 | 18 | [*.{md,mkd,markdown}] 19 | trim_trailing_whitespace = false 20 | 21 | # python files without extension 22 | [show-duplicate-java-classes] 23 | indent_size = 4 24 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | # To get started with Dependabot version updates, you'll need to specify which 2 | # package ecosystems to update and where the package manifests are located. 3 | # Please see the documentation for all configuration options: 4 | # https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates 5 | 6 | version: 2 7 | updates: 8 | - package-ecosystem: github-actions 9 | directory: / 10 | schedule: 11 | interval: daily 12 | -------------------------------------------------------------------------------- /test/self-installer.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | if [ ! -d "/tmp/useful-scripts-$USER" ]; then 4 | if type -P git &>/dev/null; then 5 | git clone https://github.com/oldratlee/useful-scripts.git "/tmp/useful-scripts-$USER" 6 | elif type -P svn &>/dev/null; then 7 | svn checkout https://github.com/oldratlee/useful-scripts/branches/release-3.x "/tmp/useful-scripts-$USER" 8 | else 9 | echo "fail to find command git/svn" 10 | return 1 11 | fi 12 | fi 13 | 14 | export PATH="$PATH:/tmp/useful-scripts-$USER/bin" 15 | -------------------------------------------------------------------------------- /.github/workflows/lint.yaml: -------------------------------------------------------------------------------- 1 | # Quickstart for GitHub Actions 2 | # https://docs.github.com/en/actions/quickstart 3 | 4 | name: Lint 5 | on: [ push, pull_request, workflow_dispatch ] 6 | 7 | jobs: 8 | 9 | test: 10 | runs-on: ubuntu-latest 11 | timeout-minutes: 5 12 | name: Lint 13 | 14 | steps: 15 | - uses: actions/checkout@v6 16 | with: 17 | submodules: recursive 18 | - run: test/chore/lint.sh 19 | # https://remarkablemark.org/blog/2017/10/12/check-git-dirty/ 20 | - name: Check git dirty 21 | run: | 22 | git status --short 23 | [ -z "$(git status --short)" ] 24 | -------------------------------------------------------------------------------- /bin/echo-args: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # print arguments in human and debugging friendly style. 4 | # 5 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-echo-args 6 | # @author Jerry Lee (oldratlee at gmail dot com) 7 | set -eEuo pipefail 8 | 9 | digitCount() { 10 | # argument 1(num) is always a non-negative integer in this script usage, 11 | # so NO argument validation logic. 12 | local num=$1 count=0 13 | while ((num != 0)); do 14 | ((++count)) 15 | ((num = num / 10)) 16 | done 17 | echo "$count" 18 | } 19 | 20 | digit_count=$(digitCount $#) 21 | readonly arg_count=$# digit_count 22 | 23 | readonly RED='\e[1;31m' BLUE='\e[1;36m' COLOR_RESET='\e[0m' 24 | printArg() { 25 | local idx=$1 value=$2 26 | 27 | # if stdout is a terminal, turn on color output. 28 | # '-t' check: is a terminal? 29 | # check isatty in bash https://stackoverflow.com/questions/10022323 30 | if [ -t 1 ]; then 31 | printf "%${digit_count}s/%s: ${RED}[${BLUE}%s${RED}]${COLOR_RESET}\n" "$idx" "$arg_count" "$value" 32 | else 33 | printf "%${digit_count}s/%s: [%s]\n" "$idx" "$arg_count" "$value" 34 | fi 35 | } 36 | 37 | printArg 0 "$0" 38 | idx=1 39 | for a; do 40 | printArg $((idx++)) "$a" 41 | done 42 | -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- 1 | # Quickstart for GitHub Actions 2 | # https://docs.github.com/en/actions/quickstart 3 | 4 | name: CI 5 | on: [ push, pull_request, workflow_dispatch ] 6 | 7 | jobs: 8 | 9 | test: 10 | runs-on: ${{ matrix.os }} 11 | timeout-minutes: 5 12 | strategy: 13 | matrix: 14 | # the OS supported by GitHub Actions 15 | # https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources 16 | os: [ ubuntu-latest, macos-latest ] 17 | fail-fast: false 18 | max-parallel: 64 19 | name: Test on ${{ matrix.os }} 20 | 21 | steps: 22 | - uses: actions/checkout@v6 23 | with: 24 | submodules: recursive 25 | - run: brew install coreutils gnu-sed 26 | # https://docs.github.com/en/actions/learn-github-actions/variables#detecting-the-operating-system 27 | # https://docs.github.com/en/actions/learn-github-actions/expressions 28 | if: runner.os == 'macOS' 29 | - run: test/chore/integration-test.sh 30 | # https://remarkablemark.org/blog/2017/10/12/check-git-dirty/ 31 | - name: Check git dirty 32 | run: | 33 | git status --short 34 | [ -z "$(git status --short)" ] 35 | -------------------------------------------------------------------------------- /docs/install.md: -------------------------------------------------------------------------------- 1 | 🐌 下载使用 2 | ==================================== 3 | 4 | 下载整个工程的脚本 5 | ------------------- 6 | 7 | ### 直接clone工程 8 | 9 | 使用简单、方便更新，不过要安装有`git`。 10 | 11 | ```bash 12 | git clone git://github.com/oldratlee/useful-scripts.git 13 | 14 | cd useful-scripts 15 | 16 | # 使用Release分支的内容 17 | git checkout release-3.x 18 | 19 | # 更新脚本 20 | git pull 21 | ``` 22 | 23 | 包含2个分支： 24 | 25 | - `dev-3.x`：开发分支 26 | - `release-3.x`：发布分支，功能稳定的脚本 27 | 28 | PS： 29 | 我的做法是把`useful-scripts` checkout到`$HOME/bin`目录下，再把`$HOME/bin/useful-scripts/bin`配置到`PATH`变量上，这样方便我本地使用所有的脚本。 30 | 31 | ### 打包下载 32 | 33 | 下载文件[release-3.x.zip](https://github.com/oldratlee/useful-scripts/archive/release-3.x.zip)： 34 | 35 | ```bash 36 | wget --no-check-certificate https://github.com/oldratlee/useful-scripts/archive/release-3.x.zip 37 | 38 | unzip release-3.x.zip 39 | ``` 40 | 41 | 下载和运行单个文件 42 | ------------------- 43 | 44 | 以[`show-busy-java-threads`](https://raw.github.com/oldratlee/useful-scripts/release-3.x/bin/show-busy-java-threads)为例。 45 | 46 | ### `curl`文件直接用`bash`运行 47 | 48 | ```bash 49 | curl -sLk 'https://raw.github.com/oldratlee/useful-scripts/release-3.x/bin/show-busy-java-threads' | bash 50 | ``` 51 | 52 | ### 下载单个文件 53 | 54 | ```bash 55 | wget --no-check-certificate https://raw.github.com/oldratlee/useful-scripts/release-3.x/bin/show-busy-java-threads 56 | chmod +x show-busy-java-threads 57 | 58 | ./show-busy-java-threads 59 | ``` 60 | -------------------------------------------------------------------------------- /bin/xpf: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Open file in file explorer, file is selected. 4 | # same as xpl --selected [file]... 5 | # 6 | # @Usage 7 | # $ ./xpf file 8 | # 9 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-xpl-and-xpf 10 | # @author Jerry Lee (oldratlee at gmail dot com) 11 | set -eEuo pipefail 12 | 13 | ################################################################################ 14 | # util functions 15 | ################################################################################ 16 | 17 | # `realpath` command exists on Linux and macOS, return resolved physical path 18 | # - realpath command on macOS do NOT support option `-e`; 19 | # combined `[ -e $file ]` to check file existence first. 20 | # - How can I get the behavior of GNU's readlink -f on a Mac? 21 | # https://stackoverflow.com/questions/1055671 22 | realpath() { 23 | [ -e "$1" ] && command realpath -- "$1" 24 | } 25 | 26 | ################################################################################ 27 | # biz logic 28 | ################################################################################ 29 | 30 | # DO NOT inline THIS_SCRIPT into BASE_DIR, because sub-shell: 31 | # BASE_DIR=$(dirname -- "$(realpath "${BASH_SOURCE[0]}")") 32 | THIS_SCRIPT=$(realpath "${BASH_SOURCE[0]}") 33 | BASE_DIR=$(dirname -- "$THIS_SCRIPT") 34 | 35 | # shellcheck disable=SC1091 36 | source "$BASE_DIR/xpl" "$@" 37 | -------------------------------------------------------------------------------- /test/chore/integration-test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -eEuo pipefail 3 | 4 | realpath() { 5 | [ -e "$1" ] && command realpath -- "$1" 6 | } 7 | 8 | cd "$(dirname -- "$(realpath "${BASH_SOURCE[0]}")")"/.. 9 | 10 | ################################################################################ 11 | # common util functions 12 | ################################################################################ 13 | 14 | colorEcho() { 15 | local color=$1 16 | shift 17 | # if stdout is a terminal, turn on color output. 18 | # '-t' check: is a terminal? 19 | # check isatty in bash https://stackoverflow.com/questions/10022323 20 | if [[ -t 1 || "${GITHUB_ACTIONS:-}" = true ]]; then 21 | printf '\e[1;%sm%s\e[0m\n' "$color" "$*" 22 | else 23 | printf '%s\n' "$*" 24 | fi 25 | } 26 | 27 | redEcho() { 28 | colorEcho 31 "$@" 29 | } 30 | 31 | yellowEcho() { 32 | colorEcho 33 "$@" 33 | } 34 | 35 | blueEcho() { 36 | colorEcho 36 "$@" 37 | } 38 | 39 | logAndRun() { 40 | local simple_mode=false 41 | [ "$1" == "-s" ] && { 42 | simple_mode=true 43 | shift 44 | } 45 | 46 | if $simple_mode; then 47 | echo "Run under work directory $PWD : $*" 48 | "$@" 49 | else 50 | # NOTE: $'foo' is the escape sequence syntax of bash 51 | local nl=$'\n' # new line 52 | blueEcho "Run under work directory $PWD :$nl$*" 53 | time "$@" 54 | fi 55 | } 56 | 57 | ################################################################################ 58 | # run *_test.sh unit test cases 59 | ################################################################################ 60 | 61 | for test_case in *_test.sh; do 62 | logAndRun ./"$test_case" 63 | done 64 | -------------------------------------------------------------------------------- /test/chore/bump-scripts-version.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -eEuo pipefail 3 | 4 | ################################################################################ 5 | # util functions 6 | ################################################################################ 7 | 8 | # NOTE: $'foo' is the escape sequence syntax of bash 9 | readonly NL=$'\n' # new line 10 | 11 | colorPrint() { 12 | local color=$1 13 | shift 14 | # if stdout is a terminal, turn on color output. 15 | # '-t' check: is a terminal? 16 | # check isatty in bash https://stackoverflow.com/questions/10022323 17 | if [ -t 1 ]; then 18 | printf '\e[1;%sm%s\e[0m\n' "$color" "$*" 19 | else 20 | printf '%s\n' "$*" 21 | fi 22 | } 23 | 24 | redPrint() { 25 | colorPrint 31 "$@" 26 | } 27 | 28 | yellowPrint() { 29 | colorPrint 33 "$@" 30 | } 31 | 32 | bluePrint() { 33 | colorPrint 36 "$@" 34 | } 35 | 36 | logAndRun() { 37 | local simple_mode=false 38 | [ "$1" = "-s" ] && { 39 | simple_mode=true 40 | shift 41 | } 42 | 43 | if $simple_mode; then 44 | echo "Run under work directory $PWD : $*" 45 | "$@" 46 | else 47 | bluePrint "Run under work directory $PWD :$NL$*" 48 | time "$@" 49 | fi 50 | } 51 | 52 | die() { 53 | redPrint "Error: $*" >&2 54 | exit 1 55 | } 56 | 57 | ################################################################################ 58 | # biz logic 59 | ################################################################################ 60 | 61 | (($# != 1)) && die "need only 1 argument for version!$NL${NL}usage:$NL $0 3.x.y" 62 | readonly bump_version=$1 63 | 64 | # adjust current dir to project dir 65 | # 66 | # Bash Pitfalls#5 67 | # http://mywiki.wooledge.org/BashPitfalls#cd_.24.28dirname_.22.24f.22.29 68 | cd -P -- "$(dirname -- "$0")"/.. 69 | 70 | # Bash Pitfalls#1 71 | # http://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29 72 | logAndRun find -D exec bin lib -type f -exec \ 73 | sed -ri "s/^(.*\bPROG_VERSION\s*=\s*')\S*('.*)$/\1$bump_version\2/" -- \ 74 | {} + 75 | -------------------------------------------------------------------------------- /bin/tcp-connection-state-counter: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # show count of tcp connection stat. 4 | # 5 | # @Usage 6 | # $ ./tcp-connection-state-counter 7 | # 8 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-tcp-connection-state-counter 9 | # @author Jerry Lee (oldratlee at gmail dot com) 10 | # @author @sunuslee (sunuslee at gmail dot com) 11 | set -eEuo pipefail 12 | 13 | readonly PROG=${0##*/} 14 | readonly PROG_VERSION='3.x-dev' 15 | 16 | ################################################################################ 17 | # parse options 18 | ################################################################################ 19 | 20 | usage() { 21 | cat <= 0; --idx)); do 44 | [[ "${args[idx]}" = -h || "${args[idx]}" = --help ]] && usage 45 | [[ "${args[idx]}" = -V || "${args[idx]}" = --version ]] && progVersion 46 | done 47 | unset args idx 48 | 49 | ################################################################################ 50 | # biz logic 51 | ################################################################################ 52 | 53 | # On MacOS, netstat need to using -p tcp to get only tcp output. 54 | UNAME=$(uname) 55 | [[ $UNAME = Darwin* ]] && option_for_mac=-ptcp 56 | 57 | # shellcheck disable=SC2086 58 | netstat -tna ${option_for_mac:-} | awk 'NR > 2 { 59 | ++s[$NF] 60 | } 61 | 62 | END { 63 | # get max length of stat and count 64 | for(v in s) { 65 | stat_len = length(v) 66 | if(stat_len > max_stat_len) max_stat_len = stat_len 67 | 68 | count_len = length(s[v]) 69 | if (count_len > max_count_len) max_count_len = count_len 70 | } 71 | 72 | for(v in s) { 73 | printf "%-" max_stat_len "s %" max_count_len "s\n", v, s[v] 74 | } 75 | }' | sort -nr -k2,2 76 | -------------------------------------------------------------------------------- /lib/console-text-color-themes.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # show all console text color themes. 4 | # 5 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-console-text-color-themessh 6 | # @author Jerry Lee (oldratlee at gmail dot com) 7 | 8 | colorEcho() { 9 | local combination=$1 10 | shift 1 11 | # if stdout is a terminal, turn on color output. 12 | # '-t' check: is a terminal? 13 | # check isatty in bash https://stackoverflow.com/questions/10022323 14 | if [ -t 1 ]; then 15 | printf '\e[%sm%s\e[0m\n' "$combination" "$*" 16 | else 17 | print '%s\n' "$*" 18 | fi 19 | } 20 | 21 | colorEchoWithoutNewLine() { 22 | local combination=$1 23 | shift 1 24 | 25 | if [ -t 1 ]; then 26 | printf '\e[%sm%s\e[0m' "$combination" "$*" 27 | else 28 | printf %s "$*" 29 | fi 30 | } 31 | 32 | # if source this script(use as lib), just export 2 helper functions, and do NOT print anything. 33 | # 34 | # if directly run this script, the length of array BASH_SOURCE is 1; 35 | # if source this script, the length of array BASH_SOURCE is grater than 1. 36 | ((${#BASH_SOURCE[@]} == 1)) || return 0 37 | 38 | for style in 0 1 2 3 4 5 6 7; do 39 | for fg in 30 31 32 33 34 35 36 37; do 40 | for bg in 40 41 42 43 44 45 46 47; do 41 | combination="${style};${fg};${bg}" 42 | colorEchoWithoutNewLine "$combination" "$combination" 43 | printf ' ' 44 | done 45 | echo 46 | done 47 | echo 48 | done 49 | 50 | echo 'Code sample to print color text:' 51 | 52 | printf %s ' echo -e "\e[' 53 | colorEchoWithoutNewLine '3;35;40' '1;36;41' 54 | printf %s m 55 | colorEchoWithoutNewLine '0;32;40' 'Sample Text' 56 | printf '%s\n' '\e[0m"' 57 | 58 | printf %s " echo \$'\e[" 59 | colorEchoWithoutNewLine '3;35;40' '1;36;41' 60 | printf %s "m'\"" 61 | colorEchoWithoutNewLine '0;32;40' 'Sample Text' 62 | printf '%s\n' "\"$'\e[0m'" 63 | printf '%s\n' " # NOTE: $'foo' is the escape sequence syntax of bash, safer escape" 64 | 65 | printf '%s\n' 'Output of above code:' 66 | printf %s ' ' 67 | colorEcho '1;36;41' 'Sample Text' 68 | echo 69 | echo 'If you are going crazy to write text in escapes string like me,' 70 | echo 'you can use colorEcho and colorEchoWithoutNewLine function in this script.' 71 | echo 72 | echo 'Code sample to print color text:' 73 | echo ' colorEcho "1;36;41" "Sample Text"' 74 | echo 'Output of above code:' 75 | echo -n ' ' 76 | colorEcho '1;36;41' 'Sample Text' 77 | -------------------------------------------------------------------------------- /bin/a2l: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # print each arguments on one line colorfully. 4 | # 5 | # @Usage 6 | # $ ./a2l arg1 arg2 7 | # $ ./a2l *.txt 8 | # 9 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-a2l 10 | # @author Jerry Lee (oldratlee at gmail dot com) 11 | set -eEuo pipefail 12 | 13 | readonly PROG=${0##*/} 14 | readonly PROG_VERSION='3.x-dev' 15 | 16 | ################################################################################ 17 | # parse options 18 | ################################################################################ 19 | 20 | usage() { 21 | cat < 0)); do 44 | case "$1" in 45 | -h | --help) 46 | usage 47 | ;; 48 | -V | --version) 49 | progVersion 50 | ;; 51 | --) 52 | shift 53 | args=(${args[@]:+"${args[@]}"} "$@") 54 | break 55 | ;; 56 | -*) 57 | # if unrecognized option, treat it and all follow arguments as args 58 | args=(${args[@]:+"${args[@]}"} "$@") 59 | break 60 | ;; 61 | *) 62 | # if not option, treat it and all follow arguments as args 63 | args=(${args[@]:+"${args[@]}"} "$@") 64 | break 65 | ;; 66 | esac 67 | done 68 | readonly args 69 | 70 | ################################################################################ 71 | # biz logic 72 | ################################################################################ 73 | 74 | readonly -a ROTATE_COLORS=(33 35 36 31 32 37 34) 75 | COLOR_INDEX=0 76 | rotateColorPrint() { 77 | local content=$* 78 | # - if stdout is a terminal, turn on color output. 79 | # '-t' check: is a terminal? 80 | # check isatty in bash https://stackoverflow.com/questions/10022323 81 | # - skip color for white space 82 | if [[ ! -t 1 || $content =~ ^[[:space:]]*$ ]]; then 83 | printf '%s\n' "$content" 84 | else 85 | local color=${ROTATE_COLORS[COLOR_INDEX++ % ${#ROTATE_COLORS[@]}]} 86 | printf '\e[1;%sm%s\e[0m\n' "$color" "$content" 87 | fi 88 | } 89 | 90 | for a in ${args[@]:+"${args[@]}"}; do 91 | rotateColorPrint "$a" 92 | done 93 | -------------------------------------------------------------------------------- /bin/taoc: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # tac lines colorfully. taoc means coat(*CO*lorful c*AT*) in reverse(last line first). 4 | # 5 | # @Usage 6 | # $ echo -e 'Hello\nWorld' | taoc 7 | # $ taoc /path/to/file1 8 | # $ taoc /path/to/file1 /path/to/file2 9 | # 10 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-coat 11 | # @author Jerry Lee (oldratlee at gmail dot com) 12 | set -eEuo pipefail 13 | 14 | readonly PROG=${0##*/} 15 | readonly PROG_VERSION='3.x-dev' 16 | 17 | ################################################################################ 18 | # parse options 19 | ################################################################################ 20 | 21 | usage() { 22 | cat <= 0; --idx)); do 46 | [ "${args[idx]}" = --help ] && usage 47 | [ "${args[idx]}" = --version ] && progVersion 48 | done 49 | unset args idx 50 | 51 | ################################################################################ 52 | # biz logic 53 | ################################################################################ 54 | 55 | # if stdout is not a terminal, use `tac` directly. 56 | # '-t' check: is a terminal? 57 | # check isatty in bash https://stackoverflow.com/questions/10022323 58 | [ -t 1 ] || exec tac "$@" 59 | 60 | readonly -a ROTATE_COLORS=(33 35 36 31 32 37 34) 61 | COLOR_INDEX=0 62 | # CAUTION: print content WITHOUT new line 63 | rotateColorPrint() { 64 | local content=$* 65 | # skip color for white space 66 | if [[ $content =~ ^[[:space:]]*$ ]]; then 67 | printf %s "$content" 68 | else 69 | local color=${ROTATE_COLORS[COLOR_INDEX++ % ${#ROTATE_COLORS[@]}]} 70 | printf '\e[1;%sm%s\e[0m' "$color" "$content" 71 | fi 72 | } 73 | 74 | rotateColorPrintln() { 75 | # NOTE: $'foo' is the escape sequence syntax of bash 76 | rotateColorPrint "$*"$'\n' 77 | } 78 | 79 | colorLines() { 80 | local line 81 | # Bash read line does not read leading spaces 82 | # https://stackoverflow.com/questions/29689172 83 | while IFS= read -r line; do 84 | rotateColorPrintln "$line" 85 | done 86 | # How to use `while read` (Bash) to read the last line in a file 87 | # if there’s no newline at the end of the file? 88 | # https://stackoverflow.com/questions/4165135 89 | [ -z "$line" ] || rotateColorPrint "$line" 90 | } 91 | 92 | tac "$@" | colorLines 93 | -------------------------------------------------------------------------------- /bin/coat: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # cat lines colorfully. coat means *CO*lorful c*AT*. 4 | # 5 | # @Usage 6 | # $ echo -e 'Hello\nWorld' | coat 7 | # $ coat /path/to/file1 8 | # $ coat /path/to/file1 /path/to/file2 9 | # 10 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-coat 11 | # @author Jerry Lee (oldratlee at gmail dot com) 12 | set -eEuo pipefail 13 | 14 | readonly PROG=${0##*/} 15 | readonly PROG_VERSION='3.x-dev' 16 | 17 | ################################################################################ 18 | # parse options 19 | ################################################################################ 20 | 21 | usage() { 22 | cat <= 0; --idx)); do 46 | [ "${args[idx]}" = --help ] && usage 47 | [ "${args[idx]}" = --version ] && progVersion 48 | done 49 | unset args idx 50 | 51 | ################################################################################ 52 | # biz logic 53 | ################################################################################ 54 | 55 | # if stdout is not a terminal, use `cat` directly. 56 | # '-t' check: is a terminal? 57 | # check isatty in bash https://stackoverflow.com/questions/10022323 58 | [ -t 1 ] || exec cat "$@" 59 | 60 | readonly -a ROTATE_COLORS=(33 35 36 31 32 37 34) 61 | COLOR_INDEX=0 62 | # CAUTION: print content WITHOUT new line 63 | rotateColorPrint() { 64 | local content=$* 65 | # skip color for white space 66 | if [[ $content =~ ^[[:space:]]*$ ]]; then 67 | printf %s "$content" 68 | else 69 | local color=${ROTATE_COLORS[COLOR_INDEX++ % ${#ROTATE_COLORS[@]}]} 70 | printf '\e[1;%sm%s\e[0m' "$color" "$content" 71 | fi 72 | } 73 | 74 | rotateColorPrintln() { 75 | # NOTE: $'foo' is the escape sequence syntax of bash 76 | rotateColorPrint "$*"$'\n' 77 | } 78 | 79 | colorLines() { 80 | local line 81 | # Bash read line does not read leading spaces 82 | # https://stackoverflow.com/questions/29689172 83 | while IFS= read -r line; do 84 | rotateColorPrintln "$line" 85 | done 86 | # How to use `while read` (Bash) to read the last line in a file 87 | # if there’s no newline at the end of the file? 88 | # https://stackoverflow.com/questions/4165135 89 | [ -z "$line" ] || rotateColorPrint "$line" 90 | } 91 | 92 | if (($# == 0)); then 93 | colorLines 94 | else 95 | cat "$@" | colorLines 96 | fi 97 | -------------------------------------------------------------------------------- /test/my_unit_test_lib.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # unit test lib 3 | 4 | ################################################# 5 | # commons functions 6 | ################################################# 7 | 8 | __ut_colorEcho() { 9 | local color=$1 10 | shift 11 | # if stdout is a terminal, turn on color output. 12 | # '-t' check: is a terminal? 13 | # check isatty in bash https://stackoverflow.com/questions/10022323 14 | if [[ -t 1 || "${GITHUB_ACTIONS:-}" = true ]]; then 15 | printf '\e[1;%sm%s\e[0m\n' "$color" "$*" 16 | else 17 | printf '%s\n' "$*" 18 | fi 19 | } 20 | 21 | redEcho() { 22 | __ut_colorEcho 31 "$@" 23 | } 24 | 25 | greenEcho() { 26 | __ut_colorEcho 32 "$@" 27 | } 28 | 29 | yellowEcho() { 30 | __ut_colorEcho 33 "$@" 31 | } 32 | 33 | blueEcho() { 34 | __ut_colorEcho 34 "$@" 35 | } 36 | 37 | fail() { 38 | redEcho "TEST FAIL: $*" 39 | exit 1 40 | } 41 | 42 | die() { 43 | redEcho "Error: $*" >&2 44 | exit 1 45 | } 46 | 47 | ################################################# 48 | # assertion functions 49 | ################################################# 50 | 51 | assertArrayEquals() { 52 | (($# == 2 || $# == 3)) || die "assertArrayEquals must 2 or 3 arguments!" 53 | local failMsg= 54 | (($# == 3)) && { 55 | failMsg=$1 56 | shift 57 | } 58 | 59 | local a1PlaceHolder="$1[@]" 60 | local a2PlaceHolder="$2[@]" 61 | local a1=("${!a1PlaceHolder}") 62 | local a2=("${!a2PlaceHolder}") 63 | 64 | ((${#a1[@]} == ${#a2[@]})) || fail "assertArrayEquals array length [${#a1[@]}] != [${#a2[@]}]${failMsg:+: $failMsg}" 65 | 66 | local i 67 | for ((i = 0; i < ${#a1[@]}; i++)); do 68 | [ "${a1[$i]}" = "${a2[$i]}" ] || fail "assertArrayEquals fail element $i: [${a1[$i]}] != [${a2[$i]}]${failMsg:+: $failMsg}" 69 | done 70 | } 71 | 72 | assertEquals() { 73 | (($# == 2 || $# == 3)) || die "assertEqual must 2 or 3 arguments!" 74 | local failMsg="" 75 | (($# == 3)) && { 76 | failMsg=$1 77 | shift 78 | } 79 | [ "$1" == "$2" ] || fail "assertEqual fail [$1] != [$2]${failMsg:+: $failMsg}" 80 | } 81 | 82 | readonly __ut_exclude_vars_builtin='^BASH_|^_=|^COLUMNS=|LINES=' 83 | readonly __ut_exclude_vars_ut_functions='^FUNCNAME=|^test_' 84 | 85 | assertAllVarsSame() { 86 | local test_afterVars 87 | test_afterVars=$(declare) 88 | 89 | diff \ 90 | <(echo "$test_beforeVars" | grep -Ev "$__ut_exclude_vars_builtin") \ 91 | <(echo "$test_afterVars" | grep -Ev "$__ut_exclude_vars_builtin|$__ut_exclude_vars_ut_functions") || 92 | fail "assertAllVarsSame: Unexpected extra global vars!" 93 | } 94 | 95 | assertAllVarsExcludeOptVarsSame() { 96 | local test_afterVars 97 | test_afterVars=$(declare) 98 | 99 | diff \ 100 | <(echo "$test_beforeVars" | grep -Ev "$__ut_exclude_vars_builtin") \ 101 | <(echo "$test_afterVars" | grep -Ev "$__ut_exclude_vars_builtin|$__ut_exclude_vars_ut_functions"'|^_OPT_|^_opts_index_name_') || 102 | fail "assertAllVarsExcludeOptVarsSame: Unexpected extra global vars!" 103 | } 104 | 105 | test_beforeVars=$(declare) 106 | -------------------------------------------------------------------------------- /bin/ap: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # convert to Absolute Path. 4 | # 5 | # @Usage 6 | # # print Absolute Path of current directory. 7 | # $ ./ap 8 | # # print Absolute Path of arguments. 9 | # $ ./ap a.txt ../dir1/b.txt 10 | # 11 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-ap-and-rp 12 | # @author Jerry Lee (oldratlee at gmail dot com) 13 | set -eEuo pipefail 14 | 15 | readonly PROG=${0##*/} 16 | readonly PROG_VERSION='3.x-dev' 17 | 18 | ################################################################################ 19 | # util functions 20 | ################################################################################ 21 | 22 | errorMsgPrint() { 23 | local errorMsg="$PROG: $*" 24 | # if stdout is a terminal, turn on color output. 25 | # '-t' check: is a terminal? 26 | # check isatty in bash https://stackoverflow.com/questions/10022323 27 | if [ -t 1 ]; then 28 | printf '\e[1;31m%s\e[0m\n' "$errorMsg" 29 | else 30 | printf '%s\n' "$errorMsg" 31 | fi 32 | } >&2 33 | 34 | die() { 35 | local prompt_help=false exit_status=2 36 | while (($# > 0)); do 37 | case "$1" in 38 | -h) 39 | prompt_help=true 40 | shift 41 | ;; 42 | -s) 43 | exit_status=$2 44 | shift 2 45 | ;; 46 | *) 47 | break 48 | ;; 49 | esac 50 | done 51 | 52 | (($# > 0)) && errorMsgPrint "$*" 53 | $prompt_help && echo "Try '$PROG --help' for more information." 54 | 55 | exit "$exit_status" 56 | } >&2 57 | 58 | # `realpath` command exists on Linux and macOS, return resolved physical path 59 | # - realpath command on macOS do NOT support option `-e`; 60 | # combined `[ -e $file ]` to check file existence first. 61 | # - How can I get the behavior of GNU's readlink -f on a Mac? 62 | # https://stackoverflow.com/questions/1055671 63 | realpath() { 64 | [ -e "$1" ] && command realpath -- "$1" 65 | } 66 | 67 | usage() { 68 | cat < 0)); do 95 | case "$1" in 96 | -h | --help) 97 | usage 98 | ;; 99 | -V | --version) 100 | progVersion 101 | ;; 102 | --) 103 | shift 104 | files=(${files[@]:+"${files[@]}"} "$@") 105 | break 106 | ;; 107 | -*) 108 | die -h "unrecognized option '$1'" 109 | ;; 110 | *) 111 | # if not option, treat all follow files as args 112 | files=(${files[@]:+"${files[@]}"} "$@") 113 | break 114 | ;; 115 | esac 116 | done 117 | 118 | # if files is empty, use "." 119 | readonly files=("${files[@]:-.}") 120 | 121 | ################################################################################ 122 | # biz logic 123 | ################################################################################ 124 | 125 | has_error=false 126 | 127 | for f in "${files[@]}"; do 128 | realpath "$f" || { 129 | has_error=true 130 | errorMsgPrint "$f: No such file or directory!" 131 | } 132 | done 133 | 134 | # set exit status 135 | ! $has_error 136 | -------------------------------------------------------------------------------- /bin/c: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Run command and put output to system clipper. 4 | # 5 | # @Usage 6 | # $ c ls -l 7 | # $ ls -l | c 8 | # $ c -q < ~/.ssh/id_rsa.pub 9 | # 10 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-c 11 | # @author Jerry Lee (oldratlee at gmail dot com) 12 | set -eEuo pipefail 13 | 14 | readonly PROG=${0##*/} 15 | readonly PROG_VERSION='3.x-dev' 16 | 17 | ################################################################################ 18 | # util functions 19 | ################################################################################ 20 | 21 | redPrint() { 22 | # if stdout is a terminal, turn on color output. 23 | # '-t' check: is a terminal? 24 | # check isatty in bash https://stackoverflow.com/questions/10022323 25 | if [ -t 1 ]; then 26 | printf '\e[1;31m%s\e[0m\n' "$*" 27 | else 28 | printf '%s\n' "$*" 29 | fi 30 | } 31 | 32 | die() { 33 | local prompt_help=false exit_status=2 34 | while (($# > 0)); do 35 | case "$1" in 36 | -h) 37 | prompt_help=true 38 | shift 39 | ;; 40 | -s) 41 | exit_status=$2 42 | shift 2 43 | ;; 44 | *) 45 | break 46 | ;; 47 | esac 48 | done 49 | 50 | (($# > 0)) && redPrint "$PROG: $*" 51 | $prompt_help && echo "Try '$PROG --help' for more information." 52 | 53 | exit "$exit_status" 54 | } >&2 55 | 56 | usage() { 57 | cat < 0)); do 90 | case "$1" in 91 | -k | --keep-eol) 92 | keep_eol=true 93 | shift 94 | ;; 95 | -q | --quiet) 96 | quiet=true 97 | shift 98 | ;; 99 | -h | --help) 100 | usage 101 | ;; 102 | -V | --version) 103 | progVersion 104 | ;; 105 | --) 106 | shift 107 | target_command=(${target_command[@]:+"${target_command[@]}"} "$@") 108 | break 109 | ;; 110 | -*) 111 | die -h "unrecognized option '$1'" 112 | ;; 113 | *) 114 | # if not option, treat all follow arguments as command 115 | target_command=(${target_command[@]:+"${target_command[@]}"} "$@") 116 | break 117 | ;; 118 | esac 119 | done 120 | 121 | readonly keep_eol quiet target_command 122 | 123 | if ((${#target_command[@]} > 0)) && ! type -P "${target_command[0]}" &>/dev/null; then 124 | die "command '${target_command[0]}' not found on PATH" 125 | fi 126 | 127 | ################################################################################ 128 | # biz logic 129 | ################################################################################ 130 | 131 | systemClip() { 132 | case "$(uname)" in 133 | Darwin*) 134 | pbcopy 135 | ;; 136 | CYGWIN* | MINGW*) 137 | clip 138 | ;; 139 | *) 140 | xsel -b 141 | ;; 142 | esac 143 | } 144 | 145 | bufferCopy() { 146 | local content 147 | content=$(cat) 148 | if $keep_eol; then 149 | printf '%s\n' "$content" 150 | else 151 | printf %s "$content" 152 | fi | systemClip 153 | } 154 | 155 | teeAndCopy() { 156 | if $quiet; then 157 | bufferCopy 158 | else 159 | tee >(bufferCopy) 160 | fi 161 | } 162 | 163 | if ((${#target_command[@]} == 0)); then 164 | teeAndCopy 165 | else 166 | command "${target_command[@]}" | teeAndCopy 167 | fi 168 | -------------------------------------------------------------------------------- /docs/developer-guide.md: -------------------------------------------------------------------------------- 1 | # 📚 `Shell`学习与开发的资料 2 | 3 | - 🛠️ 开发规范与工具 4 | - [`Google Shell Style Guide`](https://google.github.io/styleguide/shell.xml) | [中文版](https://zh-google-styleguide.readthedocs.io/en/latest/google-shell-styleguide/contents.html) 5 | - [`koalaman/shellcheck`](https://github.com/koalaman/shellcheck): `ShellCheck`, a static analysis tool for shell scripts 6 | - [`mvdan/sh(shfmt)`](https://github.com/mvdan/sh): `shfmt` formats shell programs 7 | - 👷 **`Bash/Shell`最佳实践与安全编程**文章 8 | - [Use the Unofficial Bash Strict Mode (Unless You Looove Debugging)](http://redsymbol.net/articles/unofficial-bash-strict-mode/) 9 | - Bash Pitfalls: 编程易犯的错误 - 团子的小窝：[Part 1](http://kodango.com/bash-pitfalls-part-1) | [Part 2](http://kodango.com/bash-pitfalls-part-2) | [Part 3](http://kodango.com/bash-pitfalls-part-3) | [Part 4](http://kodango.com/bash-pitfalls-part-4) | [英文原文：Bash Pitfalls](http://mywiki.wooledge.org/BashPitfalls) 10 | - [编写可靠shell脚本的八个建议 - xshell.net](https://www.xshell.net/shell/1577.html) 11 | - [Shell 编码风格 - 团子的小窝](http://kodango.com/shell-script-style) 12 | - [Bash 优良编程实践](https://www.techug.com/post/bash-practice.html) 13 | - [不要自己去指定`sh`的方式去执行脚本](https://github.com/oldratlee/useful-scripts/issues/57#issuecomment-326485965) 14 | - 🎶 **Tips** 15 | - [让你提升命令行效率的 Bash 快捷键 【完整版】](https://linuxtoy.org/archives/bash-shortcuts.html) 16 | 补充：`ctrl + x, ctrl + e` 就地打开文本编辑器来编辑当前命令行，对于复杂命令行特别有用 17 | - [应该知道的Linux技巧 | 酷 壳 - CoolShell](https://coolshell.cn/articles/8883.html) 18 | - 简洁的 Bash Programming 技巧 - 团子的小窝：[Part 1](http://kodango.com/simple-bash-programming-skills) | [Part 2](http://kodango.com/simple-bash-programming-skills-2) | [Part 3](http://kodango.com/simple-bash-programming-skills-3) 19 | - [Bash 测试和比较函数 — `test`、`[`、`[[`、`((`、和 `if-then-else` 解密](https://www.ibm.com/developerworks/cn/linux/l-bash-test.html) 20 | - [Filenames and Pathnames in Shell (bash, dash, ash, ksh, and so on): How to do it Correctly](https://dwheeler.com/essays/filenames-in-shell.html) 21 | - [理解 IFS - 团子的小窝](http://kodango.com/understand-ifs) 22 | - [shell中的IFS详解 – 笑遍世界](http://smilejay.com/2011/12/bash_ifs/) 23 | - [Bash脚本：怎样一行行地读文件（最好和最坏的方法）](http://blog.jobbole.com/72185/) 24 | - [Shell 脚本避免多次重复 source - 团子的小窝](http://kodango.com/avoid-repeated-source-in-shell) 25 | - [一个奇怪的 echo 结果 - 团子的小窝](http://kodango.com/a-strange-echo-result) 26 | - [浅谈 Shell 脚本配置文件格式 - 团子的小窝](http://kodango.com/config-file-format-in-shell) 27 | - [Bash function 还能这么玩 - 团子的小窝](http://kodango.com/bash-functions) 28 | - [Bash 获取当前函数名 - 团子的小窝](http://kodango.com/get-function-name-in-bash) 29 | - [Zsh和Bash，究竟有何不同 坑很深](https://www.xshell.net/shell/bash_zsh.html) 30 | - 💎 **系统学习** — 看文章、了解Tips完全不能替代系统学习才能真正理解并专业开发！ 31 | - [《Bash Pocket Reference》](https://book.douban.com/subject/26738258/) 32 | 力荐！说明简单直接结构体系的佳作，专业`Bash`编程必备！且16年的第二版更新到了新版的`Bash 4` 33 | - [《学习bash》](https://book.douban.com/subject/1241361/) 上面那本的展开版 34 | - 官方资料 35 | - [`bash man`](https://manned.org/bash) | [中文版](http://ahei.info/chinese-bash-man.htm) 36 | - [Bash Reference Manual - gnu.org](http://www.gnu.org/software/bash/manual/) | [中文版](https://yiyibooks.cn/Phiix/bash_reference_manual/bash%E5%8F%82%E8%80%83%E6%96%87%E6%A1%A3.html) 37 | Bash参考手册，讲得全面且有深度，比如会全面地讲解不同转义的区别、命令的解析过程，这有助统一深入的方式认识Bash整个执行方式和过程。这些内容在其它书中往往不会讲（因为复杂难于深入浅出的讲解），但却一通百通的关键。 38 | - [Advanced Bash-Scripting Guide](https://hangar118.sdf.org/p/bash-scripting-guide/index.html): An in-depth exploration of the art of shell scripting. 39 | - [命令行的艺术 - `jlevy/the-art-of-command-line`](https://github.com/jlevy/the-art-of-command-line/blob/master/README-zh.md) 40 | - [`awesome-lists/awesome-bash`](https://github.com/awesome-lists/awesome-bash): A curated list of delightful Bash scripts and resources. 41 | - [`alebcay/awesome-shell`](https://github.com/alebcay/awesome-shell): A curated list of awesome command-line frameworks, toolkits, guides and gizmos. 42 | - [wzb56/13_questions_of_shell: shell十三问 - shell教程](https://github.com/wzb56/13_questions_of_shell) 43 | - [实用 Shell 文档 - 团子的小窝](http://kodango.com/useful-documents-about-shell) 44 | - 更多书籍参见个人整理的[书籍豆列 **_`Bash/Shell`_**](https://www.douban.com/doulist/1779379/) 45 | -------------------------------------------------------------------------------- /bin/xpl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Open file in file explorer. 4 | # 5 | # @Usage 6 | # $ ./xpf [file [file ...] ] 7 | # 8 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-xpl-and-xpf 9 | # @author Jerry Lee (oldratlee at gmail dot com) 10 | set -eEuo pipefail 11 | 12 | readonly PROG=${0##*/} 13 | readonly PROG_VERSION='3.x-dev' 14 | 15 | ################################################################################ 16 | # util functions 17 | ################################################################################ 18 | 19 | redPrint() { 20 | # if stdout is a terminal, turn on color output. 21 | # '-t' check: is a terminal? 22 | # check isatty in bash https://stackoverflow.com/questions/10022323 23 | if [ -t 1 ]; then 24 | printf '\e[1;31m%s\e[0m\n' "$*" 25 | else 26 | printf '%s\n' "$*" 27 | fi 28 | } 29 | 30 | die() { 31 | local prompt_help=false exit_status=2 32 | while (($# > 0)); do 33 | case "$1" in 34 | -h) 35 | prompt_help=true 36 | shift 37 | ;; 38 | -s) 39 | exit_status=$2 40 | shift 2 41 | ;; 42 | *) 43 | break 44 | ;; 45 | esac 46 | done 47 | 48 | (($# > 0)) && redPrint "$PROG: $*" 49 | $prompt_help && echo "Try '$PROG --help' for more information." 50 | 51 | exit "$exit_status" 52 | } >&2 53 | 54 | usage() { 55 | cat < 0)); do 81 | case "$1" in 82 | -s | --selected) 83 | selected=true 84 | shift 85 | ;; 86 | -h | --help) 87 | usage 88 | ;; 89 | -V | --version) 90 | progVersion 91 | ;; 92 | --) 93 | shift 94 | files=(${files[@]:+"${files[@]}"} "$@") 95 | break 96 | ;; 97 | -*) 98 | die -h "unrecognized option '$1'" 99 | ;; 100 | *) 101 | files=(${files[@]:+"${files[@]}"} "$1") 102 | shift 103 | ;; 104 | esac 105 | done 106 | 107 | # if files is empty, use one element "." 108 | files=("${files[@]:-.}") 109 | 110 | # if program name is xpf, set option selected! 111 | [ "xpf" = "$PROG" ] && selected=true 112 | 113 | readonly files selected 114 | 115 | ################################################################################ 116 | # biz logic 117 | ################################################################################ 118 | 119 | # open one file 120 | openOneFile() { 121 | local file=$1 slt=$selected 122 | 123 | case "$(uname)" in 124 | Darwin*) 125 | [ -f "$file" ] && slt=true 126 | if $slt; then 127 | open -R "$file" 128 | else 129 | open "$file" 130 | fi 131 | ;; 132 | CYGWIN*) 133 | [ -f "$file" ] && slt=true 134 | if $slt; then 135 | explorer /select, "$(cygpath -w "$file")" 136 | else 137 | explorer "$(cygpath -w "$file")" 138 | fi 139 | ;; 140 | *) 141 | if [ -d "$file" ]; then 142 | nautilus "$(dirname -- "$file")" 143 | else 144 | if $slt; then 145 | nautilus "$file" 146 | else 147 | nautilus "$(dirname -- "$file")" 148 | fi 149 | fi 150 | ;; 151 | esac 152 | 153 | local selected_msg 154 | $slt && selected_msg='with selection' 155 | printf 'open %14s: %s\n' "$selected_msg" "$file" 156 | } 157 | 158 | has_error=false 159 | 160 | for file in "${files[@]}"; do 161 | [ -e "$file" ] || { 162 | has_error=true 163 | redPrint "$PROG: $file: No such file or directory!" >&2 164 | continue 165 | } 166 | 167 | openOneFile "$file" || has_error=true 168 | done 169 | 170 | # set exit status 171 | ! $has_error 172 | -------------------------------------------------------------------------------- /bin/rp: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # convert to Relative Path. 4 | # 5 | # @Usage 6 | # # if 1 argument, print relative path to current dir. 7 | # $ ./rp /etc/apache2/httpd.conf 8 | # # if more than 1 argument, print relative path to last argument. 9 | # $ ./rp a.txt ../b.txt /etc/passwd /etc/apache2 10 | # 11 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-ap-and-rp 12 | # @author Jerry Lee (oldratlee at gmail dot com) 13 | set -eEuo pipefail 14 | 15 | readonly PROG=${0##*/} 16 | readonly PROG_VERSION='3.x-dev' 17 | 18 | ################################################################################ 19 | # util functions 20 | ################################################################################ 21 | 22 | redPrint() { 23 | # if stdout is a terminal, turn on color output. 24 | # '-t' check: is a terminal? 25 | # check isatty in bash https://stackoverflow.com/questions/10022323 26 | if [ -t 1 ]; then 27 | printf '\e[1;31m%s\e[0m\n' "$*" 28 | else 29 | printf '%s\n' "$*" 30 | fi 31 | } 32 | 33 | die() { 34 | local prompt_help=false exit_status=2 35 | while (($# > 0)); do 36 | case "$1" in 37 | -h) 38 | prompt_help=true 39 | shift 40 | ;; 41 | -s) 42 | exit_status=$2 43 | shift 2 44 | ;; 45 | *) 46 | break 47 | ;; 48 | esac 49 | done 50 | 51 | (($# > 0)) && redPrint "$PROG: $*" 52 | $prompt_help && echo "Try '$PROG --help' for more information." 53 | 54 | exit "$exit_status" 55 | } >&2 56 | 57 | portableRelPath() { 58 | local file=$1 relTo=$2 uname 59 | 60 | uname=$(uname) 61 | case "$uname" in 62 | Linux* | CYGWIN* | MINGW*) 63 | realpath "$f" --relative-to="$relTo" 64 | ;; 65 | Darwin*) 66 | local py_args=(-c 'import os, sys; print(os.path.relpath(sys.argv[1], sys.argv[2]))' "$file" "$relTo") 67 | if type -P grealpath >/dev/null; then 68 | grealpath "$f" --relative-to="$relTo" 69 | elif type -P python3 >/dev/null; then 70 | python3 "${py_args[@]}" 71 | elif type -P python >/dev/null; then 72 | python "${py_args[@]}" 73 | else 74 | die "fail to find command(grealpath/python3/python) to get relative path!" 75 | fi 76 | ;; 77 | *) 78 | die "uname($uname) NOT support!" 79 | ;; 80 | esac 81 | } 82 | 83 | usage() { 84 | cat < 0)); do 112 | case "$1" in 113 | -h | --help) 114 | usage 115 | ;; 116 | -V | --version) 117 | progVersion 118 | ;; 119 | --) 120 | shift 121 | files=(${files[@]:+"${files[@]}"} "$@") 122 | break 123 | ;; 124 | -*) 125 | die -h "unrecognized option '$1'" 126 | ;; 127 | *) 128 | # if not option, treat all follow files as args 129 | files=(${files[@]:+"${files[@]}"} "$@") 130 | break 131 | ;; 132 | esac 133 | done 134 | 135 | ((${#files[@]} == 0)) && die -h "requires at least one argument!" 136 | 137 | if ((${#files[@]} == 1)); then 138 | relativeTo=. 139 | else 140 | argc=${#files[@]} 141 | 142 | # Get last argument 143 | relativeTo=${files[argc - 1]} 144 | files=("${files[@]:0:argc-1}") 145 | fi 146 | 147 | [ -f "$relativeTo" ] && relativeTo=$(dirname -- "$relativeTo") 148 | [ -e "$relativeTo" ] || die "relativeTo dir($relativeTo): No such file or directory!" 149 | 150 | readonly files relativeTo 151 | 152 | ################################################################################ 153 | # biz logic 154 | ################################################################################ 155 | 156 | has_error=false 157 | 158 | for f in "${files[@]}"; do 159 | if [ -e "$f" ]; then 160 | portableRelPath "$f" "$relativeTo" 161 | else 162 | redPrint "$PROG: $f: No such file or directory!" >&2 163 | has_error=true 164 | fi 165 | done 166 | 167 | # set exit status 168 | ! $has_error 169 | -------------------------------------------------------------------------------- /test/uq_test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | set -eEuo pipefail 3 | 4 | realpath() { 5 | [ -e "$1" ] && command realpath -- "$1" 6 | } 7 | 8 | BASE=$(dirname -- "$(realpath "${BASH_SOURCE[0]}")") 9 | cd "$BASE" 10 | 11 | ################################################# 12 | # commons and test data 13 | ################################################# 14 | 15 | readonly uq="../bin/uq" 16 | # NOTE: $'foo' is the escape sequence syntax of bash 17 | readonly nl=$'\n' # new line 18 | 19 | test_input=$(cat uq_test_input) 20 | 21 | ################################################# 22 | # test cases 23 | ################################################# 24 | 25 | test_uq_simple() { 26 | assertEquals "c${nl}v${nl}a${nl}u" \ 27 | "$(echo "$test_input" | "$uq")" 28 | assertEquals "c${nl}v${nl}a${nl}u" \ 29 | "$("$uq" uq_test_input)" 30 | 31 | assertEquals "c${nl}a" \ 32 | "$(echo "$test_input" | "$uq" -d)" 33 | assertEquals "c${nl}a" \ 34 | "$("$uq" -d uq_test_input)" 35 | 36 | assertEquals "v${nl}u" "$(echo "$test_input" | "$uq" -u)" 37 | assertEquals "v${nl}u" "$("$uq" -u uq_test_input)" 38 | } 39 | 40 | readonly test_output_uq_count=' 4 c 41 | 1 v 42 | 2 a 43 | 1 u' 44 | 45 | readonly test_output_uq_D_count=' 4 c 46 | 4 c 47 | 1 v 48 | 2 a 49 | 2 a 50 | 4 c 51 | 4 c 52 | 1 u' 53 | 54 | test_uq_count() { 55 | assertEquals "$test_output_uq_count" "$(echo "$test_input" | "$uq" -c)" 56 | assertEquals "$test_output_uq_count" "$("$uq" -c uq_test_input)" 57 | 58 | assertEquals "$test_output_uq_D_count" "$(echo "$test_input" | "$uq" -D -c)" 59 | assertEquals "$test_output_uq_D_count" "$("$uq" -D -c uq_test_input)" 60 | } 61 | 62 | test_uq_only_D_option__same_as_cat() { 63 | assertEquals "$test_input" "$(echo "$test_input" | "$uq" -D)" 64 | assertEquals "$test_input" "$("$uq" -D uq_test_input)" 65 | } 66 | 67 | test_multi_input_files__output_file() { 68 | local output_file="$SHUNIT_TMPDIR/uq_output_file_${$}_${RANDOM}_${RANDOM}" 69 | "$uq" uq_test_input uq_test_another_input "$output_file" 70 | assertEquals "c${nl}v${nl}a${nl}u${nl}m${nl}x" \ 71 | "$(cat "$output_file")" 72 | 73 | local output_file="$SHUNIT_TMPDIR/uq_output_file_${$}_${RANDOM}_${RANDOM}" 74 | "$uq" -d uq_test_input uq_test_another_input "$output_file" 75 | assertEquals "c${nl}a${nl}m" \ 76 | "$(cat "$output_file")" 77 | 78 | local output_file="$SHUNIT_TMPDIR/uq_output_file_${$}_${RANDOM}_${RANDOM}" 79 | "$uq" -u uq_test_input uq_test_another_input "$output_file" 80 | assertEquals "v${nl}u${nl}x" \ 81 | "$(cat "$output_file")" 82 | } 83 | 84 | test_multi_input_files__output_stdout() { 85 | assertEquals "c${nl}v${nl}a${nl}u${nl}m${nl}x" "$("$uq" uq_test_input uq_test_another_input -)" 86 | 87 | assertEquals "c${nl}a${nl}m" "$("$uq" -d uq_test_input uq_test_another_input -)" 88 | 89 | assertEquals "v${nl}u${nl}x" "$("$uq" -u uq_test_input uq_test_another_input -)" 90 | } 91 | 92 | test_ignore_case() { 93 | local input="a${nl}b${nl}A" 94 | 95 | assertEquals "a${nl}b${nl}A" "$(echo "$input" | "$uq")" 96 | assertEquals "a${nl}b" "$(echo "$input" | "$uq" -i)" 97 | } 98 | 99 | test_ignore_case__count() { 100 | local input="a${nl}b${nl}A" 101 | 102 | assertEquals " 1 a${nl} 1 b${nl} 1 A" \ 103 | "$(echo "$input" | "$uq" -c)" 104 | 105 | assertEquals " 2 a${nl} 1 b" \ 106 | "$(echo "$input" | "$uq" -i -c)" 107 | 108 | assertEquals " 2 a${nl} 1 b${nl} 2 A" \ 109 | "$(echo "$input" | "$uq" -i -D -c)" 110 | } 111 | 112 | test_max_input_check() { 113 | # shellcheck disable=SC2016 114 | assertTrue 'echo 123 | "$uq"' 115 | # shellcheck disable=SC2016 116 | assertTrue 'echo 123 | "$uq" -XM 4' 117 | # shellcheck disable=SC2016 118 | assertTrue 'echo 123 | "$uq" -XM 1k' 119 | # shellcheck disable=SC2016 120 | assertTrue 'echo 123 | "$uq" --max-input 1042k' 121 | # shellcheck disable=SC2016 122 | assertTrue 'echo 123 | "$uq" --max-input 1m' 123 | # shellcheck disable=SC2016 124 | assertTrue 'echo 123 | "$uq" --max-input 10420g' 125 | # shellcheck disable=SC2016 126 | assertTrue '"$uq" uq_test_input' 127 | # shellcheck disable=SC2016 128 | assertTrue '"$uq" uq_test_input -XM 42m' 129 | # shellcheck disable=SC2016 130 | assertTrue '"$uq" uq_test_input --max-input 1024000g' 131 | # shellcheck disable=SC2016 132 | assertTrue '"$uq" uq_test_input --max-input 1234567890g' 133 | 134 | # shellcheck disable=SC2016 135 | assertFalse 'should fail by -XM' 'echo -e 123 | "$uq" -XM 1' 136 | # shellcheck disable=SC2016 137 | assertFalse 'should fail by -XM' 'echo -e 123 | "$uq" -XM 3' 138 | # shellcheck disable=SC2016 139 | assertFalse 'should fail by --max-input' 'echo -e 123 | "$uq" --max-input 2' 140 | # shellcheck disable=SC2016 141 | assertFalse 'should fail by --max-input' '"$uq" --max-input 2 uq_test_input' 142 | 143 | # shellcheck disable=SC2016 144 | assertFalse 'should fail, number overflow!' '"$uq" uq_test_input --max-input 12345678901g' 145 | } 146 | 147 | ################################################# 148 | # Load and run shUnit2. 149 | ################################################# 150 | 151 | source "$BASE/shunit2-lib/shunit2" 152 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | !bin/ 2 | 3 | ################################################# 4 | # Programming Language 5 | ################################################# 6 | 7 | ### Python template 8 | # Byte-compiled / optimized / DLL files 9 | __pycache__/ 10 | *.py[cod] 11 | 12 | # C extensions 13 | *.so 14 | 15 | # Distribution / packaging 16 | .Python 17 | env/ 18 | build/ 19 | develop-eggs/ 20 | dist/ 21 | downloads/ 22 | eggs/ 23 | .eggs/ 24 | lib/ 25 | lib64/ 26 | parts/ 27 | sdist/ 28 | var/ 29 | *.egg-info/ 30 | .installed.cfg 31 | *.egg 32 | 33 | # PyInstaller 34 | # Usually these files are written by a python script from a template 35 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 36 | *.manifest 37 | *.spec 38 | 39 | # Installer logs 40 | pip-log.txt 41 | pip-delete-this-directory.txt 42 | 43 | # Unit test / coverage reports 44 | htmlcov/ 45 | .tox/ 46 | .coverage 47 | .coverage.* 48 | .cache 49 | nosetests.xml 50 | coverage.xml 51 | *,cover 52 | 53 | # Translations 54 | *.mo 55 | *.pot 56 | 57 | # Django stuff: 58 | *.log 59 | 60 | # Sphinx documentation 61 | docs/_build/ 62 | 63 | .ipynb_checkpoints/ 64 | 65 | ### Ruby template 66 | *.gem 67 | *.rbc 68 | /.config 69 | /coverage/ 70 | /InstalledFiles 71 | /pkg/ 72 | /spec/reports/ 73 | /test/tmp/ 74 | /test/version_tmp/ 75 | 76 | ## Specific to RubyMotion: 77 | .dat* 78 | .repl_history 79 | 80 | ## Documentation cache and generated files: 81 | /.yardoc/ 82 | /_yardoc/ 83 | /doc/ 84 | /rdoc/ 85 | 86 | ## Environment normalisation: 87 | /.bundle/ 88 | /vendor/bundle 89 | /lib/bundler/man/ 90 | 91 | # for a library or gem, you might want to ignore these files since the code is 92 | # intended to run in multiple environments; otherwise, check them in: 93 | # Gemfile.lock 94 | # .ruby-version 95 | # .ruby-gemset 96 | 97 | # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: 98 | .rvmrc 99 | 100 | 101 | ### Node template 102 | # Logs 103 | logs 104 | 105 | # Runtime data 106 | pids 107 | *.pid 108 | *.seed 109 | 110 | # Directory for instrumented libs generated by jscoverage/JSCover 111 | lib-cov 112 | 113 | # Coverage directory used by tools like istanbul 114 | coverage 115 | 116 | # Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) 117 | .grunt 118 | 119 | # node-waf configuration 120 | .lock-wscript 121 | 122 | # Compiled binary addons (http://nodejs.org/api/addons.html) 123 | build/Release 124 | 125 | # Dependency directory 126 | # https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git 127 | node_modules 128 | 129 | 130 | ################################################# 131 | # IDE/Editor 132 | ################################################# 133 | 134 | ### JetBrains template 135 | # Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm 136 | 137 | *.iml 138 | 139 | ## Directory-based project format: 140 | .idea/ 141 | # if you remove the above rule, at least ignore the following: 142 | 143 | # User-specific stuff: 144 | # .idea/workspace.xml 145 | # .idea/tasks.xml 146 | # .idea/dictionaries 147 | 148 | # Sensitive or high-churn files: 149 | # .idea/dataSources.ids 150 | # .idea/dataSources.xml 151 | # .idea/sqlDataSources.xml 152 | # .idea/dynamic.xml 153 | # .idea/uiDesigner.xml 154 | 155 | # Gradle: 156 | # .idea/gradle.xml 157 | # .idea/libraries 158 | 159 | # Mongo Explorer plugin: 160 | # .idea/mongoSettings.xml 161 | 162 | ## File-based project format: 163 | *.ipr 164 | *.iws 165 | 166 | ## Plugin-specific files: 167 | 168 | # IntelliJ 169 | /out/ 170 | 171 | # mpeltonen/sbt-idea plugin 172 | .idea_modules/ 173 | 174 | # JIRA plugin 175 | atlassian-ide-plugin.xml 176 | 177 | # Crashlytics plugin (for Android Studio and IntelliJ) 178 | com_crashlytics_export_strings.xml 179 | crashlytics.properties 180 | crashlytics-build.properties 181 | 182 | 183 | ### Eclipse template 184 | *.pydevproject 185 | .metadata 186 | .gradle 187 | tmp/ 188 | *.bak 189 | *.swp 190 | *~.nib 191 | .settings/ 192 | .loadpath 193 | 194 | # Eclipse Core 195 | .project 196 | 197 | # External tool builders 198 | .externalToolBuilders/ 199 | 200 | # Locally stored "Eclipse launch configurations" 201 | *.launch 202 | 203 | # CDT-specific 204 | .cproject 205 | 206 | # JDT-specific (Eclipse Java Development Tools) 207 | .classpath 208 | 209 | # PDT-specific 210 | .buildpath 211 | 212 | # sbteclipse plugin 213 | .target 214 | 215 | # TeXlipse plugin 216 | .texlipse 217 | 218 | .factorypath 219 | 220 | 221 | ### SublimeText template 222 | # cache files for sublime text 223 | *.tmlanguage.cache 224 | *.tmPreferences.cache 225 | *.stTheme.cache 226 | 227 | # workspace files are user-specific 228 | *.sublime-workspace 229 | 230 | # project files should be checked into the repository, unless a significant 231 | # proportion of contributors will probably not be using SublimeText 232 | # *.sublime-project 233 | 234 | # sftp configuration file 235 | sftp-config.json 236 | 237 | 238 | ### Vim template 239 | [._]*.s[a-w][a-z] 240 | [._]s[a-w][a-z] 241 | *.un~ 242 | Session.vim 243 | .netrwhist 244 | *~ 245 | 246 | 247 | ### vs code 248 | .vscode/ 249 | 250 | 251 | ################################################# 252 | # OS 253 | ################################################# 254 | 255 | ### OSX template 256 | .DS_Store 257 | .AppleDouble 258 | .LSOverride 259 | 260 | # Icon must end with two \r 261 | Icon 262 | 263 | 264 | ### Linux template 265 | # KDE directory preferences 266 | .directory 267 | 268 | # Linux trash folder which might appear on any partition or disk 269 | .Trash-* 270 | 271 | 272 | ### Windows template 273 | # Windows image file caches 274 | Thumbs.db 275 | ehthumbs.db 276 | ehthumbs_vista.db 277 | 278 | # Folder config file 279 | Desktop.ini 280 | 281 | # Recycle Bin used on file shares 282 | $RECYCLE.BIN/ 283 | 284 | # Windows Installer files 285 | *.cab 286 | *.msi 287 | *.msm 288 | *.msp 289 | 290 | # Windows shortcuts 291 | *.lnk 292 | -------------------------------------------------------------------------------- /test/parseOpts_test.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | BASE="$(dirname -- "${BASH_SOURCE[0]}")" 4 | 5 | source "$BASE/../lib/parseOpts.sh" 6 | 7 | source "$BASE/my_unit_test_lib.sh" 8 | 9 | ################################################# 10 | # Test 11 | ################################################# 12 | 13 | # ======================================== 14 | blueEcho "Test case: success parse" 15 | # ======================================== 16 | 17 | parseOpts "a,a-long|b,b-long:|c,c-long+|d,d-long+" aa -a -b bb -c c.sh -p pv -q qv cc \; bb --d-long d.sh -x xv d1 d2 d3 \; cc dd ee 18 | test_exitCode=$? 19 | _opts_showOptDescInfoList 20 | _opts_showOptValueInfoList 21 | 22 | ((test_exitCode == 0)) || fail "Wrong exit code!" 23 | ((${#_OPT_INFO_LIST_INDEX[@]} == 4)) || fail "Wrong _OPT_INFO_LIST_INDEX!" 24 | 25 | [[ $_OPT_VALUE_a == "true" && $_OPT_VALUE_a_long == "true" ]] || fail "Wrong option value of a!" 26 | [[ $_OPT_VALUE_b == "bb" && $_OPT_VALUE_b_long == "bb" ]] || fail "Wrong option value of b!" 27 | 28 | test_cArray=(c.sh -p pv -q qv cc) 29 | assertArrayEquals "Wrong option value of c!" test_cArray _OPT_VALUE_c 30 | assertArrayEquals "Wrong option value of c!" test_cArray _OPT_VALUE_c_long 31 | 32 | test_dArray=(d.sh -x xv d1 d2 d3) 33 | assertArrayEquals "Wrong option value of d!" test_dArray _OPT_VALUE_d 34 | assertArrayEquals "Wrong option value of d!" test_dArray _OPT_VALUE_d_long 35 | 36 | test_argArray=(aa bb cc dd ee) 37 | assertArrayEquals "Wrong args!" test_argArray _OPT_ARGS 38 | 39 | assertAllVarsExcludeOptVarsSame 40 | 41 | _opts_cleanOptValueInfoList 42 | assertAllVarsSame 43 | 44 | # ======================================== 45 | blueEcho "Test case: success parse with -- " 46 | # ======================================== 47 | 48 | parseOpts "a,a-long|b,b-long:|c,c-long+|d,d-long+" aa -a -b bb -c c.sh -p pv -q qv cc \; bb -- --d-long d.sh -x xv d1 d2 d3 \; cc dd ee 49 | test_exitCode=$? 50 | _opts_showOptDescInfoList 51 | _opts_showOptValueInfoList 52 | 53 | ((test_exitCode == 0)) || fail "Wrong exit code!" 54 | ((${#_OPT_INFO_LIST_INDEX[@]} == 4)) || fail "Wrong _OPT_INFO_LIST_INDEX!" 55 | 56 | [[ $_OPT_VALUE_a == "true" && $_OPT_VALUE_a_long == "true" ]] || fail "Wrong option value of a!" 57 | [[ $_OPT_VALUE_b == "bb" && $_OPT_VALUE_b_long == "bb" ]] || fail "Wrong option value of b!" 58 | 59 | test_cArray=(c.sh -p pv -q qv cc) 60 | assertArrayEquals "Wrong option value of c!" test_cArray _OPT_VALUE_c 61 | assertArrayEquals "Wrong option value of c!" test_cArray _OPT_VALUE_c_long 62 | 63 | [[ "$_OPT_VALUE_d" == "" && "$_OPT_VALUE_d_long" == "" ]] || fail "Wrong option value of d!" 64 | 65 | test_argArray=(aa bb --d-long d.sh -x xv d1 d2 d3 \; cc dd ee) 66 | assertArrayEquals "Wrong args!" test_argArray _OPT_ARGS 67 | 68 | assertAllVarsExcludeOptVarsSame 69 | 70 | _opts_cleanOptValueInfoList 71 | assertAllVarsSame 72 | 73 | # ======================================== 74 | blueEcho "Test case: illegal option x" 75 | # ======================================== 76 | 77 | parseOpts "a,a-long|b,b-long:|c,c-long+|d,d-long+" aa -a -b bb -x -c c.sh -p pv -q qv cc \; bb --d-long d.sh -x xv d1 d2 d3 \; cc -- dd ee 78 | test_exitCode=$? 79 | _opts_showOptDescInfoList 80 | _opts_showOptValueInfoList 81 | 82 | ((test_exitCode == 232)) || fail "Wrong exit code!" 83 | ((${#_OPT_INFO_LIST_INDEX[@]} == 0)) || fail "Wrong _OPT_INFO_LIST_INDEX!" 84 | [[ "$_OPT_VALUE_a" == "" && "$_OPT_VALUE_a_long" == "" ]] || fail "Wrong option value of a!" 85 | [[ "$_OPT_VALUE_b" == "" && "$_OPT_VALUE_b_long" == "" ]] || fail "Wrong option value of b!" 86 | [[ "$_OPT_VALUE_c" == "" && "$_OPT_VALUE_c_long" == "" ]] || fail "Wrong option value of c!" 87 | [[ "$_OPT_VALUE_d" == "" && "$_OPT_VALUE_d_long" == "" ]] || fail "Wrong option value of d!" 88 | [ "$_OPT_ARGS" == "" ] || fail "Wrong args!" 89 | 90 | # ======================================== 91 | blueEcho "Test case: empty options" 92 | # ======================================== 93 | 94 | parseOpts "a,a-long|b,b-long:|c,c-long+|d,d-long+" 95 | test_exitCode=$? 96 | _opts_showOptDescInfoList 97 | _opts_showOptValueInfoList 98 | 99 | ((test_exitCode == 0)) || fail "Wrong exit code!" 100 | ((${#_OPT_INFO_LIST_INDEX[@]} == 4)) || fail "Wrong _OPT_INFO_LIST_INDEX!" 101 | [[ "$_OPT_VALUE_a" == "" && "$_OPT_VALUE_a_long" == "" ]] || fail "Wrong option value of a!" 102 | [[ "$_OPT_VALUE_b" == "" && "$_OPT_VALUE_b_long" == "" ]] || fail "Wrong option value of b!" 103 | [[ "$_OPT_VALUE_c" == "" && "$_OPT_VALUE_c_long" == "" ]] || fail "Wrong option value of c!" 104 | [[ "$_OPT_VALUE_d" == "" && "$_OPT_VALUE_d_long" == "" ]] || fail "Wrong option value of d!" 105 | [ "$_OPT_ARGS" == "" ] || fail "Wrong args!" 106 | 107 | # ======================================== 108 | blueEcho "Test case: illegal option name" 109 | # ======================================== 110 | 111 | parseOpts "a,a-long|b,b-long:|c,c-long+|d,d-long+|#,z-long" aa -a -b bb -x -c c.sh -p pv -q qv cc \; bb -d d.sh -x xv d1 d2 d3 \; cc -- dd ee 112 | test_exitCode=$? 113 | _opts_showOptDescInfoList 114 | _opts_showOptValueInfoList 115 | 116 | ((test_exitCode == 221)) || fail "Wrong exit code!" 117 | ((${#_OPT_INFO_LIST_INDEX[@]} == 0)) || fail "Wrong _OPT_INFO_LIST_INDEX!" 118 | [[ "$_OPT_VALUE_a" == "" && "$_OPT_VALUE_a_long" == "" ]] || fail "Wrong option value of a!" 119 | [[ "$_OPT_VALUE_b" == "" && "$_OPT_VALUE_b_long" == "" ]] || fail "Wrong option value of b!" 120 | [[ "$_OPT_VALUE_c" == "" && "$_OPT_VALUE_c_long" == "" ]] || fail "Wrong option value of c!" 121 | [[ "$_OPT_VALUE_d" == "" && "$_OPT_VALUE_d_long" == "" ]] || fail "Wrong option value of d!" 122 | [ "$_OPT_ARGS" == "" ] || fail "Wrong args!" 123 | 124 | parseOpts "a,a-long|b,b-long:|c,c-long+|d,d-long+|z,z-#long" aa -a -b bb -x -c c.sh -p pv -q qv cc \; bb -d d.sh -x xv d1 d2 d3 \; cc -- dd ee 125 | test_exitCode=$? 126 | _opts_showOptDescInfoList 127 | _opts_showOptValueInfoList 128 | 129 | ((test_exitCode == 222)) || fail "Wrong exit code!" 130 | ((${#_OPT_INFO_LIST_INDEX[@]} == 0)) || fail "Wrong _OPT_INFO_LIST_INDEX!" 131 | [[ "$_OPT_VALUE_a" == "" && "$_OPT_VALUE_a_long" == "" ]] || fail "Wrong option value of a!" 132 | [[ "$_OPT_VALUE_b" == "" && "$_OPT_VALUE_b_long" == "" ]] || fail "Wrong option value of b!" 133 | [[ "$_OPT_VALUE_c" == "" && "$_OPT_VALUE_c_long" == "" ]] || fail "Wrong option value of c!" 134 | [[ "$_OPT_VALUE_d" == "" && "$_OPT_VALUE_d_long" == "" ]] || fail "Wrong option value of d!" 135 | [ "$_OPT_ARGS" == "" ] || fail "Wrong args!" 136 | 137 | assertAllVarsSame 138 | 139 | greenEcho "TEST SUCCESS!!!" 140 | -------------------------------------------------------------------------------- /bin/cp-into-docker-run: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Copy the command into docker container and run the command in container. 4 | # 5 | # Example: 6 | # cp-into-docker-run -c container_foo command_copied_into_container command_arg1 7 | # 8 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-cp-into-docker-run 9 | # @author Jerry Lee (oldratlee at gmail dot com) 10 | set -eEuo pipefail 11 | 12 | readonly PROG=${0##*/} 13 | readonly PROG_VERSION='3.x-dev' 14 | 15 | ################################################################################ 16 | # util functions 17 | ################################################################################ 18 | 19 | redPrint() { 20 | # if stdout is a terminal, turn on color output. 21 | # '-t' check: is a terminal? 22 | # check isatty in bash https://stackoverflow.com/questions/10022323 23 | if [ -t 1 ]; then 24 | printf '\e[1;31m%s\e[0m\n' "$*" 25 | else 26 | printf '%s\n' "$*" 27 | fi 28 | } 29 | 30 | die() { 31 | local prompt_help=false exit_staus=2 32 | while (($# > 0)); do 33 | case "$1" in 34 | -h) 35 | prompt_help=true 36 | shift 37 | ;; 38 | -s) 39 | exit_staus=$2 40 | shift 2 41 | ;; 42 | *) 43 | break 44 | ;; 45 | esac 46 | done 47 | 48 | (($# > 0)) && redPrint "$PROG: $*" 49 | $prompt_help && echo "Try '$PROG --help' for more information." 50 | 51 | exit "$exit_staus" 52 | } >&2 53 | 54 | isAbsolutePath() { 55 | [[ "$1" =~ ^/ ]] 56 | } 57 | 58 | # `realpath` command exists on Linux and macOS, return resolved physical path 59 | # - realpath command on macOS do NOT support option `-e`; 60 | # combined `[ -e $file ]` to check file existence first. 61 | # - How can I get the behavior of GNU's readlink -f on a Mac? 62 | # https://stackoverflow.com/questions/1055671 63 | realpath() { 64 | [ -e "$1" ] && command realpath -- "$1" 65 | } 66 | 67 | usage() { 68 | cat < 0)); do 118 | case "$1" in 119 | -c | --container) 120 | container_name=$2 121 | shift 2 122 | ;; 123 | -u | --docker-user) 124 | docker_user=$2 125 | shift 2 126 | ;; 127 | -w | --workdir) 128 | docker_workdir=$2 129 | shift 2 130 | ;; 131 | -t | --tmpdir) 132 | docker_tmpdir=$2 133 | shift 2 134 | ;; 135 | -p | --cp-path) 136 | docker_command_cp_path=$2 137 | shift 2 138 | ;; 139 | -v | --verbose) 140 | verbose=true 141 | shift 142 | ;; 143 | -h | --help) 144 | usage 145 | ;; 146 | -V | --version) 147 | progVersion 148 | ;; 149 | --) 150 | shift 151 | args=(${args[@]:+"${args[@]}"} "$@") 152 | break 153 | ;; 154 | -*) 155 | die -h "unrecognized option '$1'" 156 | ;; 157 | *) 158 | # if not option, treat all follow arguments as command 159 | args=(${args[@]:+"${args[@]}"} "$@") 160 | break 161 | ;; 162 | esac 163 | done 164 | 165 | readonly container_name docker_user docker_workdir docker_tmpdir docker_command_cp_path verbose args 166 | 167 | [ -n "$container_name" ] || 168 | die -h "requires destination docker container name, specified by option -c/--container!" 169 | 170 | if [ -n "$docker_workdir" ]; then 171 | isAbsolutePath "$docker_workdir" || 172 | die "docker workdir(-w/--workdir) must be absolute path: $docker_workdir" 173 | elif [ -n "$docker_command_cp_path" ]; then 174 | isAbsolutePath "$docker_command_cp_path" || 175 | die "when no docker workdir(-w/--workdir) is specified, the command path in docker to copy(-p/--cp-path) must be absolute path: $docker_command_cp_path" 176 | fi 177 | 178 | ################################################################################ 179 | # biz logic 180 | ################################################################################ 181 | 182 | ######################################## 183 | # check docker command existence 184 | ######################################## 185 | 186 | type -P docker &>/dev/null || die 'docker command not found!' 187 | 188 | ######################################## 189 | # prepare vars for docker operation 190 | ######################################## 191 | 192 | readonly specified_run_command=${args[0]} 193 | run_command=$specified_run_command 194 | if [ ! -f "$specified_run_command" ]; then 195 | type -P "$specified_run_command" &>/dev/null || 196 | die "specified command not exists and not found in PATH: $specified_run_command" 197 | 198 | run_command=$(type -P "$specified_run_command") 199 | fi 200 | 201 | run_command=$(realpath "$run_command") 202 | readonly run_command run_command_base_name=${run_command##*/} 203 | 204 | run_timestamp=$(date "+%Y%m%d_%H%M%S") 205 | readonly run_timestamp 206 | readonly uuid="${PROG}_${run_timestamp}_${$}_${RANDOM}" 207 | 208 | if [ -n "$docker_command_cp_path" ]; then 209 | if isAbsolutePath "$docker_command_cp_path"; then 210 | readonly run_command_in_docker=$docker_command_cp_path 211 | else 212 | readonly run_command_in_docker="${docker_workdir:+"$docker_workdir/"}$docker_command_cp_path" 213 | fi 214 | run_command_dir_in_docker=$(dirname -- "$run_command_in_docker") 215 | readonly run_command_dir_in_docker 216 | else 217 | readonly work_tmp_dir_in_docker=$docker_tmpdir/$uuid 218 | 219 | readonly run_command_in_docker="$work_tmp_dir_in_docker/$run_command_base_name" 220 | readonly run_command_dir_in_docker=$work_tmp_dir_in_docker 221 | fi 222 | 223 | cleanupWhenExit() { 224 | [ -n "${work_tmp_dir_in_docker:-}" ] || return 0 225 | 226 | # remove tmp dir in docker by root user 227 | docker exec "$container_name" rm -rf -- "$work_tmp_dir_in_docker" &>/dev/null 228 | } 229 | trap cleanupWhenExit EXIT 230 | 231 | ######################################## 232 | # docker operations 233 | ######################################## 234 | 235 | logAndRun() { 236 | $verbose && printf '%s\n' "[$PROG] $*" >&2 237 | "$@" 238 | } 239 | 240 | logAndRun docker exec ${docker_user:+"--user=$docker_user"} "$container_name" \ 241 | mkdir -p -- "$run_command_dir_in_docker" 242 | logAndRun docker cp "$run_command" "$container_name:$run_command_in_docker" 243 | logAndRun docker exec ${docker_user:+"--user=$docker_user"} "$container_name" \ 244 | chmod +x "$run_command_in_docker" 245 | 246 | logAndRun docker exec -i -t \ 247 | ${docker_user:+"--user=$docker_user"} \ 248 | ${docker_workdir:+"--workdir=$docker_workdir"} \ 249 | "$container_name" \ 250 | "$run_command_in_docker" "${args[@]:1:${#args[@]}}" 251 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | #
🐌 useful-scripts
2 | 3 |

4 | Github Workflow Build Status 5 | GitHub release 6 | License 7 | GitHub Stars 8 | GitHub Forks 9 | GitHub issues 10 | GitHub Contributors 11 | GitHub repo size 12 |

13 | 14 | 🐌 useful scripts for making developer's everyday life easier and happier, involved java, shell etc. 15 | 16 | 👉 平时有用的手动操作做成脚本，以便捷地使用，让开发的日常生活更轻松些。 💕 17 | 18 | 欢迎 👏 💖 19 | 20 | - 提问，[提交 Issue](https://github.com/oldratlee/useful-scripts/issues/new) 21 | - 分享平时自己常用但没有写成脚本的功能（即需求、想法），[提交Issue](https://github.com/oldratlee/useful-scripts/issues/new) 22 | - 优化改进，[Fork 后提通过 Pull Request 贡献代码](https://github.com/oldratlee/useful-scripts/fork) 23 | - 提供的自己好用脚本实现，[Fork 后提通过 Pull Request 提供](https://github.com/oldratlee/useful-scripts/fork) 24 | 25 | 本仓库的脚本（如`Java`相关脚本）在阿里等公司（如随身云，见[`awesome-scripts`仓库](https://github.com/Suishenyun/awesome-scripts)说明）的线上生产环境部署使用。 26 | 27 | 如果你的公司有部署使用，欢迎使用通过 [Issue：who's using | 用户反馈收集](https://github.com/oldratlee/useful-scripts/issues/96) 告知，方便互相交流反馈～ 💗 28 | 29 | repo-icon 30 | 31 | ---------------------- 32 | 33 | 34 | 35 | 36 | - [🔰 快速下载&使用](#-%E5%BF%AB%E9%80%9F%E4%B8%8B%E8%BD%BD%E4%BD%BF%E7%94%A8) 37 | - [📚 使用文档](#-%E4%BD%BF%E7%94%A8%E6%96%87%E6%A1%A3) 38 | - [☕ `Java`相关脚本](#-java%E7%9B%B8%E5%85%B3%E8%84%9A%E6%9C%AC) 39 | - [🐚 `Shell`相关脚本](#-shell%E7%9B%B8%E5%85%B3%E8%84%9A%E6%9C%AC) 40 | - [🎓 Developer Guide](#-developer-guide) 41 | - [🎯 面向开发者的目标](#-%E9%9D%A2%E5%90%91%E5%BC%80%E5%8F%91%E8%80%85%E7%9A%84%E7%9B%AE%E6%A0%87) 42 | - [关于`Shell`脚本](#%E5%85%B3%E4%BA%8Eshell%E8%84%9A%E6%9C%AC) 43 | - [🚦 开发约定](#-%E5%BC%80%E5%8F%91%E7%BA%A6%E5%AE%9A) 44 | - [📚 `Shell`学习与开发的资料](#-shell%E5%AD%A6%E4%B9%A0%E4%B8%8E%E5%BC%80%E5%8F%91%E7%9A%84%E8%B5%84%E6%96%99) 45 | 46 | 47 | 48 | ---------------------- 49 | 50 | 🔰 快速下载&使用 51 | ---------------------- 52 | 53 | ```bash 54 | source <(curl -fsSL https://raw.githubusercontent.com/oldratlee/useful-scripts/release-3.x/test/self-installer.sh) 55 | ``` 56 | 57 | 更多下载&使用方式，参见[下载使用](docs/install.md)。 58 | 59 | 📚 使用文档 60 | ---------------------- 61 | 62 | ### ☕ [`Java`相关脚本](docs/java.md) 63 | 64 | 1. [show-busy-java-threads](docs/java.md#-show-busy-java-threads) 65 | 用于快速排查`Java`的`CPU`性能问题(`top us`值过高)，自动查出运行的`Java`进程中消耗`CPU`多的线程，并打印出其线程栈，从而确定导致性能问题的方法调用。 66 | 1. [show-duplicate-java-classes](docs/java.md#-show-duplicate-java-classes) 67 | 找出`jar`文件和`class`目录中的重复类。用于排查`Java`类冲突问题。 68 | 1. [find-in-jars](docs/java.md#-find-in-jars) 69 | 在目录下所有`jar`文件里，查找类或资源文件。 70 | 71 | ### 🐚 [`Shell`相关脚本](docs/shell.md) 72 | 73 | `Shell`使用加强： 74 | 75 | 1. [c](docs/shell.md#-c) 76 | 原样命令行输出，并拷贝标准输出到系统剪贴板，省去`CTRL+C`操作，优化命令行与其它应用之间的操作流。 77 | 1. [coat and taoc](docs/shell.md#-coat) 78 | 彩色`cat`/`tac`出文件行，方便人眼区分不同的行。 79 | 1. [a2l](docs/shell.md#-a2l) 80 | 按行彩色输出参数，方便人眼查看。 81 | 1. [uq](docs/shell.md#-uq) 82 | 不重排序输入完成整个输入行的去重。相比系统的`uniq`命令加强的是可以跨行去重，不需要排序输入。 83 | 1. [ap and rp](docs/shell.md#-ap-and-rp) 84 | 批量转换文件路径为绝对路径/相对路径，会自动跟踪链接并规范化路径。 85 | 1. [cp-into-docker-run](docs/shell.md#-cp-into-docker-run) 86 | 一个`Docker`使用的便利脚本。拷贝本机的执行文件到指定的`docker container`中并在`docker container`中执行。 87 | 1. [tcp-connection-state-counter](docs/shell.md#-tcp-connection-state-counter) 88 | 统计各个`TCP`连接状态的个数。用于方便排查系统连接负荷问题。 89 | 1. [xpl and xpf](docs/shell.md#-xpl-and-xpf) 90 | 在命令行中快速完成 在文件浏览器中 打开/选中 指定的文件或文件夹的操作，优化命令行与其它应用之间的操作流。 91 | 92 | `Shell`开发/测试加强： 93 | 94 | 1. [echo-args](docs/shell.md#-echo-args) 95 | 输出脚本收到的参数，在控制台运行时，把参数值括起的括号显示成 **红色**，方便人眼查看。用于调试脚本参数输入。 96 | 1. [console-text-color-themes.sh](docs/shell.md#-console-text-color-themessh) 97 | 显示`Terminator`的全部文字彩色组合的效果及其打印方式，用于开发`Shell`的彩色输出。 98 | 1. [parseOpts.sh](docs/shell.md#-parseoptssh) 99 | 命令行选项解析库，加强支持选项有多个值（即数组）。 100 | 101 | ## 🎓 Developer Guide 102 | 103 | 为用户提供有用的功能，当然是这个库的首要的价值体现和存在理由。 104 | 105 | 但作为一个**开源**项目，每个人都可以看到源码实现，这个库或许能做得更多。 106 | 107 | ### 🎯 面向开发者的目标 108 | 109 | - 将`Shell/Bash`作为线上生产环境使用的专业编程语言。 110 | - 期望体现`Shell/Bash`脚本 生产环境级的严谨开发方式与最佳实践，进而有可能示例与改善在生产环境中`Shell`脚本的质量状况。 111 | 112 | PS： 113 | 114 | - 虽然上面是自己期望的目标，但自己在`Shell`语言上一定会有很多理解和使用上的问题、在这些实现脚本中也会很多需要的改进，可以一起学习、讨论与实践～ 💕 115 | - 这个库中脚本的实现也有使用`Python`。 116 | 117 | #### 关于`Shell`脚本 118 | 119 | 命令行（`CLI`）几乎是每个程序员每天都在使用的工具。相比图形界面工具（`GUI`），命令行有着自己不可替代的便利性和优越性。 120 | 121 | 命令行里写出来其实就是`Shell`脚本，可以说每个开发者会写`Shell`脚本（或多或少）。在生产环境的功能实现中，也常会看到`Shell`脚本（虽然不如主流语言那么常见）。 122 | 123 | 可能正因为上面所说的`Shell`脚本的便利性和大众性： 124 | 125 | - `Shell`脚本有不少是顺手实现的（包括生产环境用的`Shell`脚本）； 126 | - `Shell`脚本的实现常常可能质量不高，会引发线上严重的故障。 127 | 128 | ### 🚦 开发约定 129 | 130 | 在这个库中的`Shell`脚本： 131 | 132 | - 统一使用`Bash 3.2+`； 133 | - 面向生产环境，尽可能使用严谨安全的开发方式。 134 | 135 | `Shell`用`Bash`的原因是： 136 | 137 | - 目前仍然是主流的`Shell`，并且在不同环境基本上都缺省部署了。 138 | - 在[`Google`的`Shell`风格指南](https://zh-google-styleguide.readthedocs.io/en/latest/google-shell-styleguide/background.html#shell)中，明确说到了：`Bash`是**唯一**被允许执行的`shell`脚本语言。 139 | - 统一用`Bash`，可以避免不同`Shell`之间差异所带来的风险与没有收益的复杂性。 140 | - 有大量的`Shell`实现，`sh`、`bash`、`zsh`、`fish`、`csh`、`tcsh`、`ksh`、`ash`、`dash`…… 141 | - 不同的`Shell`有各种差异，深坑勿入。 142 | - 个人系统学习过的是`Bash`，比较理解熟悉。 143 | 144 | PS: 虽然交互`Shell`个人已经使用`Zsh` + [`oh-my-zsh`](https://ohmyz.sh/)，但在严谨的`Shell`脚本开发时还是使用`Bash`。 145 | 146 | ### 📚 `Shell`学习与开发的资料 147 | 148 | > 更多资料参见 [子文档](docs/developer-guide.md)。 149 | 150 | - 🛠️ 开发规范与工具 151 | - [`Google Shell Style Guide`](https://google.github.io/styleguide/shell.xml) | [中文版](https://zh-google-styleguide.readthedocs.io/en/latest/google-shell-styleguide/contents.html) 152 | - [`koalaman/shellcheck`](https://github.com/koalaman/shellcheck): `ShellCheck`, a static analysis tool for shell scripts 153 | - [`mvdan/sh(shfmt)`](https://github.com/mvdan/sh): `shfmt` formats shell programs 154 | - 👷 **`Bash/Shell`最佳实践与安全编程**文章 155 | - [Use the Unofficial Bash Strict Mode (Unless You Looove Debugging)](http://redsymbol.net/articles/unofficial-bash-strict-mode/) 156 | - Bash Pitfalls: 编程易犯的错误 - 团子的小窝：[Part 1](http://kodango.com/bash-pitfalls-part-1) | [Part 2](http://kodango.com/bash-pitfalls-part-2) | [Part 3](http://kodango.com/bash-pitfalls-part-3) | [Part 4](http://kodango.com/bash-pitfalls-part-4) | [英文原文：Bash Pitfalls](http://mywiki.wooledge.org/BashPitfalls) 157 | - [不要自己去指定`sh`的方式去执行脚本](https://github.com/oldratlee/useful-scripts/issues/57#issuecomment-326485965) 158 | - 🎶 **Tips** 159 | - [让你提升命令行效率的 Bash 快捷键 【完整版】](https://linuxtoy.org/archives/bash-shortcuts.html) 160 | 补充：`ctrl + x, ctrl + e` 就地打开文本编辑器来编辑当前命令行，对于复杂命令行特别有用 161 | - [应该知道的Linux技巧 | 酷 壳 - CoolShell](https://coolshell.cn/articles/8883.html) 162 | - 简洁的 Bash Programming 技巧 - 团子的小窝：[Part 1](http://kodango.com/simple-bash-programming-skills) | [Part 2](http://kodango.com/simple-bash-programming-skills-2) | [Part 3](http://kodango.com/simple-bash-programming-skills-3) 163 | - 💎 **系统学习** — 看文章、了解Tips完全不能替代系统学习才能真正理解并专业开发！ 164 | - [《Bash Pocket Reference》](https://book.douban.com/subject/26738258/) 165 | 力荐！说明简单直接结构体系的佳作，专业`Bash`编程必备！且16年的第二版更新到了新版的`Bash 4` 166 | - [《学习bash》](https://book.douban.com/subject/1241361/) 上面那本的展开版 167 | - 官方资料 168 | - [`bash man`](https://manned.org/bash) | [中文版](http://ahei.info/chinese-bash-man.htm) 169 | - [Bash Reference Manual - gnu.org](http://www.gnu.org/software/bash/manual/) | [中文版](https://yiyibooks.cn/Phiix/bash_reference_manual/bash%E5%8F%82%E8%80%83%E6%96%87%E6%A1%A3.html) 170 | Bash参考手册，讲得全面且有深度，比如会全面地讲解不同转义的区别、命令的解析过程，这有助统一深入的方式认识Bash整个执行方式和过程。这些内容在其它书中往往不会讲（因为复杂难于深入浅出的讲解），但却一通百通的关键。 171 | - [Advanced Bash-Scripting Guide](https://hangar118.sdf.org/p/bash-scripting-guide/index.html): An in-depth exploration of the art of shell scripting. 172 | - [命令行的艺术 - `jlevy/the-art-of-command-line`](https://github.com/jlevy/the-art-of-command-line/blob/master/README-zh.md) 173 | - [`awesome-lists/awesome-bash`](https://github.com/awesome-lists/awesome-bash): A curated list of delightful Bash scripts and resources. 174 | - [`alebcay/awesome-shell`](https://github.com/alebcay/awesome-shell): A curated list of awesome command-line frameworks, toolkits, guides and gizmos. 175 | - 更多书籍参见个人整理的[书籍豆列 **_`Bash/Shell`_**](https://www.douban.com/doulist/1779379/) 176 | -------------------------------------------------------------------------------- /bin/uq: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Filter lines from INPUT (or standard input), writing to OUTPUT (or standard output). 4 | # same as `uniq` command in core utils, 5 | # but detect repeated lines that are not adjacent, no sorting required. 6 | # 7 | # @Usage 8 | # uq [OPTION]... [INPUT [OUTPUT]] 9 | # 10 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-uq 11 | # @author Zava Xu (zava.kid at gmail dot com) 12 | # @author Jerry Lee (oldratlee at gmail dot com) 13 | set -eEuo pipefail 14 | 15 | readonly PROG=${0##*/} 16 | readonly PROG_VERSION='3.x-dev' 17 | 18 | ################################################################################ 19 | # util functions 20 | ################################################################################ 21 | 22 | # NOTE: $'foo' is the escape sequence syntax of bash 23 | readonly NL=$'\n' # new line 24 | 25 | redPrint() { 26 | # if stdout is a terminal, turn on color output. 27 | # '-t' check: is a terminal? 28 | # check isatty in bash https://stackoverflow.com/questions/10022323 29 | if [ -t 1 ]; then 30 | printf '\e[1;31m%s\e[0m\n' "$*" 31 | else 32 | printf '%s\n' "$*" 33 | fi 34 | } 35 | 36 | yellowPrint() { 37 | if [ -t 1 ]; then 38 | printf '\e[1;33m%s\e[0m\n' "$*" 39 | else 40 | printf '%s\n' "$*" 41 | fi 42 | } 43 | 44 | die() { 45 | local prompt_help=false exit_status=2 46 | while (($# > 0)); do 47 | case "$1" in 48 | -h) 49 | prompt_help=true 50 | shift 51 | ;; 52 | -s) 53 | exit_status=$2 54 | shift 2 55 | ;; 56 | *) 57 | break 58 | ;; 59 | esac 60 | done 61 | 62 | (($# > 0)) && redPrint "$PROG: $*" 63 | $prompt_help && echo "Try '$PROG --help' for more information." 64 | 65 | exit "$exit_status" 66 | } >&2 67 | 68 | convertHumanReadableSizeToSize() { 69 | local human_readable_size=$1 70 | 71 | [[ "$human_readable_size" =~ ^([0-9][0-9]*)([kmg]?)$ ]] || return 1 72 | 73 | local size=${BASH_REMATCH[1]} unit=${BASH_REMATCH[2]} 74 | case "$unit" in 75 | k) 76 | ((size *= 1024)) 77 | ;; 78 | m) 79 | ((size *= 1024 ** 2)) 80 | ;; 81 | g) 82 | ((size *= 1024 ** 3)) 83 | ;; 84 | esac 85 | 86 | echo "$size" 87 | } 88 | 89 | usage() { 90 | cat < 0)); do 146 | case "$1" in 147 | -c | --count) 148 | uq_opt_count=1 149 | shift 150 | ;; 151 | -d | --repeated) 152 | uq_opt_only_repeated=1 153 | shift 154 | ;; 155 | -D) 156 | uq_opt_all_repeated=1 157 | shift 158 | ;; 159 | --all-repeated=*) 160 | uq_opt_all_repeated=1 161 | 162 | uq_opt_repeated_method=${1#--all-repeated=} 163 | [[ $uq_opt_repeated_method = 'none' || $uq_opt_repeated_method = 'prepend' || $uq_opt_repeated_method = 'separate' ]] || 164 | die -h "invalid argument ‘$uq_opt_repeated_method’ for ‘--all-repeated’${NL}Valid arguments are:$NL - ‘none’$NL - ‘prepend’$NL - ‘separate’" 165 | 166 | shift 167 | ;; 168 | -u | --unique) 169 | uq_opt_only_unique=1 170 | shift 171 | ;; 172 | -i | --ignore-case) 173 | uq_opt_ignore_case=1 174 | shift 175 | ;; 176 | -z | --zero-terminated) 177 | uq_opt_zero_terminated=1 178 | shift 179 | ;; 180 | -XM | --max-input) 181 | uq_max_input_human_readable_size=$2 182 | shift 2 183 | ;; 184 | -h | --help) 185 | usage 186 | ;; 187 | -V | --version) 188 | progVersion 189 | ;; 190 | --) 191 | shift 192 | argv=(${argv[@]:+"${argv[@]}"} "$@") 193 | break 194 | ;; 195 | -) 196 | argv=(${argv[@]:+"${argv[@]}"} "$1") 197 | shift 198 | ;; 199 | -*) 200 | die -h "unrecognized option '$1'" 201 | ;; 202 | *) 203 | argv=(${argv[@]:+"${argv[@]}"} "$1") 204 | shift 205 | ;; 206 | esac 207 | done 208 | 209 | [[ $uq_opt_only_repeated = 1 && $uq_opt_only_unique = 1 ]] && 210 | die -h "printing duplicated lines(-d, --repeated) and unique lines(-u, --unique) is meaningless" 211 | [[ $uq_opt_all_repeated = 1 && $uq_opt_only_unique = 1 ]] && 212 | die -h "printing all duplicate lines(-D, --all-repeated) and unique lines(-u, --unique) is meaningless" 213 | 214 | [[ $uq_opt_all_repeated = 1 && $uq_opt_repeated_method = none && ($uq_opt_count = 0 && $uq_opt_only_repeated = 0) ]] && 215 | yellowPrint "WARN: -D/--all-repeated=none option without -c/-d option, just cat input simply!" >&2 216 | 217 | # DO NOT declare and assign var uq_max_input_size(as readonly) in ONE line! 218 | # more info see https://github.com/koalaman/shellcheck/wiki/SC2155 219 | uq_max_input_size=$(convertHumanReadableSizeToSize "$uq_max_input_human_readable_size") || 220 | die -h "illegal value of option -XM/--max-input: $uq_max_input_human_readable_size" 221 | 222 | readonly argc=${#argv[@]} argv uq_max_input_size 223 | 224 | if ((argc == 0)); then 225 | input_files=() 226 | output_file=/dev/stdout 227 | elif ((argc == 1)); then 228 | input_files=("${argv[0]}") 229 | output_file=/dev/stdout 230 | else 231 | input_files=("${argv[@]:0:argc-1}") 232 | output_file=${argv[argc - 1]} 233 | if [ "$output_file" = - ]; then 234 | output_file=/dev/stdout 235 | fi 236 | fi 237 | readonly output_file 238 | 239 | # Check input file 240 | for f in ${input_files[@]:+"${input_files[@]}"}; do 241 | # - is stdin, ok 242 | [ "$f" = - ] && continue 243 | 244 | [ -e "$f" ] || die "input file $f: No such file or directory!" 245 | [ ! -d "$f" ] || die "input file $f exists, but is a directory!" 246 | [ -f "$f" ] || die "input file $f exists, but is not a file!" 247 | [ -r "$f" ] || die "input file $f exists, but is not readable!" 248 | done 249 | unset f 250 | 251 | ################################################################################ 252 | # biz logic 253 | ################################################################################ 254 | 255 | # uq awk script 256 | # 257 | # edit in a separated file(eg: uq.awk) then copy here, 258 | # maybe more convenient(like good syntax highlight) 259 | 260 | # shellcheck disable=SC2016 261 | readonly uq_awk_script=' 262 | 263 | function printResult(for_lines) { 264 | for (idx = 0; idx < length(for_lines); idx++) { 265 | line = for_lines[idx] 266 | count = line_count_array[caseAwareLine(line)] 267 | #printf "DEBUG: %s %s, index: %s, uq_opt_only_repeated: %s\n", count, line, idx, uq_opt_only_repeated 268 | 269 | if (uq_opt_only_unique) { 270 | if (count == 1) printLine(count, line) 271 | } else { 272 | if (uq_opt_only_repeated && count <= 1) continue 273 | 274 | if (uq_opt_repeated_method == "prepend" || uq_opt_repeated_method == "separate" && previous_output) { 275 | if (line != previous_output) print "" 276 | } 277 | 278 | printLine(count, line) 279 | previous_output = line 280 | } 281 | } 282 | } 283 | 284 | function printLine(count, line) { 285 | if (uq_opt_count) printf "%7s %s%s", count, line, ORS 286 | else print line 287 | } 288 | 289 | function caseAwareLine(line) { 290 | if (IGNORECASE) return tolower(line) 291 | else return line 292 | } 293 | 294 | BEGIN { 295 | if (uq_opt_zero_terminated) ORS = RS = "\0" 296 | } 297 | 298 | { 299 | total_input_size += length + 1 300 | if (total_input_size > uq_max_input_size) { 301 | printf "%s: input size exceed max input size %s!\nuse option -XM/--max-input specify a REASONABLE larger value.\n", 302 | uq_PROG, uq_max_input_human_readable_size > "/dev/stderr" 303 | exit(1) 304 | } 305 | 306 | # use index to keep lines order 307 | original_lines[line_index++] = $0 308 | 309 | case_aware_line = caseAwareLine($0) 310 | # line_count_array: line content -> count 311 | if (++line_count_array[case_aware_line] == 1) { 312 | # use index to keep lines order 313 | deduplicated_lines[deduplicated_line_index++] = case_aware_line 314 | } 315 | } 316 | 317 | END { 318 | if (uq_opt_all_repeated) printResult(original_lines) 319 | else printResult(deduplicated_lines) 320 | } 321 | 322 | ' 323 | 324 | awk \ 325 | -v "uq_opt_count=$uq_opt_count" \ 326 | -v "uq_opt_only_repeated=$uq_opt_only_repeated" \ 327 | -v "uq_opt_all_repeated=$uq_opt_all_repeated" \ 328 | -v "uq_opt_repeated_method=$uq_opt_repeated_method" \ 329 | -v "uq_opt_only_unique=$uq_opt_only_unique" \ 330 | -v "IGNORECASE=$uq_opt_ignore_case" \ 331 | -v "uq_opt_zero_terminated=$uq_opt_zero_terminated" \ 332 | -v "uq_max_input_human_readable_size=$uq_max_input_human_readable_size" \ 333 | -v "uq_max_input_size=$uq_max_input_size" \ 334 | -v "uq_PROG=$PROG" \ 335 | -f <(printf "%s" "$uq_awk_script") \ 336 | -- ${input_files[@]:+"${input_files[@]}"} \ 337 | >"$output_file" 338 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | 3 | Apache License 4 | Version 2.0, January 2004 5 | http://www.apache.org/licenses/ 6 | 7 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 8 | 9 | 1. Definitions. 10 | 11 | "License" shall mean the terms and conditions for use, reproduction, 12 | and distribution as defined by Sections 1 through 9 of this document. 13 | 14 | "Licensor" shall mean the copyright owner or entity authorized by 15 | the copyright owner that is granting the License. 16 | 17 | "Legal Entity" shall mean the union of the acting entity and all 18 | other entities that control, are controlled by, or are under common 19 | control with that entity. For the purposes of this definition, 20 | "control" means (i) the power, direct or indirect, to cause the 21 | direction or management of such entity, whether by contract or 22 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 23 | outstanding shares, or (iii) beneficial ownership of such entity. 24 | 25 | "You" (or "Your") shall mean an individual or Legal Entity 26 | exercising permissions granted by this License. 27 | 28 | "Source" form shall mean the preferred form for making modifications, 29 | including but not limited to software source code, documentation 30 | source, and configuration files. 31 | 32 | "Object" form shall mean any form resulting from mechanical 33 | transformation or translation of a Source form, including but 34 | not limited to compiled object code, generated documentation, 35 | and conversions to other media types. 36 | 37 | "Work" shall mean the work of authorship, whether in Source or 38 | Object form, made available under the License, as indicated by a 39 | copyright notice that is included in or attached to the work 40 | (an example is provided in the Appendix below). 41 | 42 | "Derivative Works" shall mean any work, whether in Source or Object 43 | form, that is based on (or derived from) the Work and for which the 44 | editorial revisions, annotations, elaborations, or other modifications 45 | represent, as a whole, an original work of authorship. For the purposes 46 | of this License, Derivative Works shall not include works that remain 47 | separable from, or merely link (or bind by name) to the interfaces of, 48 | the Work and Derivative Works thereof. 49 | 50 | "Contribution" shall mean any work of authorship, including 51 | the original version of the Work and any modifications or additions 52 | to that Work or Derivative Works thereof, that is intentionally 53 | submitted to Licensor for inclusion in the Work by the copyright owner 54 | or by an individual or Legal Entity authorized to submit on behalf of 55 | the copyright owner. For the purposes of this definition, "submitted" 56 | means any form of electronic, verbal, or written communication sent 57 | to the Licensor or its representatives, including but not limited to 58 | communication on electronic mailing lists, source code control systems, 59 | and issue tracking systems that are managed by, or on behalf of, the 60 | Licensor for the purpose of discussing and improving the Work, but 61 | excluding communication that is conspicuously marked or otherwise 62 | designated in writing by the copyright owner as "Not a Contribution." 63 | 64 | "Contributor" shall mean Licensor and any individual or Legal Entity 65 | on behalf of whom a Contribution has been received by Licensor and 66 | subsequently incorporated within the Work. 67 | 68 | 2. Grant of Copyright License. Subject to the terms and conditions of 69 | this License, each Contributor hereby grants to You a perpetual, 70 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 71 | copyright license to reproduce, prepare Derivative Works of, 72 | publicly display, publicly perform, sublicense, and distribute the 73 | Work and such Derivative Works in Source or Object form. 74 | 75 | 3. Grant of Patent License. Subject to the terms and conditions of 76 | this License, each Contributor hereby grants to You a perpetual, 77 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 78 | (except as stated in this section) patent license to make, have made, 79 | use, offer to sell, sell, import, and otherwise transfer the Work, 80 | where such license applies only to those patent claims licensable 81 | by such Contributor that are necessarily infringed by their 82 | Contribution(s) alone or by combination of their Contribution(s) 83 | with the Work to which such Contribution(s) was submitted. If You 84 | institute patent litigation against any entity (including a 85 | cross-claim or counterclaim in a lawsuit) alleging that the Work 86 | or a Contribution incorporated within the Work constitutes direct 87 | or contributory patent infringement, then any patent licenses 88 | granted to You under this License for that Work shall terminate 89 | as of the date such litigation is filed. 90 | 91 | 4. Redistribution. You may reproduce and distribute copies of the 92 | Work or Derivative Works thereof in any medium, with or without 93 | modifications, and in Source or Object form, provided that You 94 | meet the following conditions: 95 | 96 | (a) You must give any other recipients of the Work or 97 | Derivative Works a copy of this License; and 98 | 99 | (b) You must cause any modified files to carry prominent notices 100 | stating that You changed the files; and 101 | 102 | (c) You must retain, in the Source form of any Derivative Works 103 | that You distribute, all copyright, patent, trademark, and 104 | attribution notices from the Source form of the Work, 105 | excluding those notices that do not pertain to any part of 106 | the Derivative Works; and 107 | 108 | (d) If the Work includes a "NOTICE" text file as part of its 109 | distribution, then any Derivative Works that You distribute must 110 | include a readable copy of the attribution notices contained 111 | within such NOTICE file, excluding those notices that do not 112 | pertain to any part of the Derivative Works, in at least one 113 | of the following places: within a NOTICE text file distributed 114 | as part of the Derivative Works; within the Source form or 115 | documentation, if provided along with the Derivative Works; or, 116 | within a display generated by the Derivative Works, if and 117 | wherever such third-party notices normally appear. The contents 118 | of the NOTICE file are for informational purposes only and 119 | do not modify the License. You may add Your own attribution 120 | notices within Derivative Works that You distribute, alongside 121 | or as an addendum to the NOTICE text from the Work, provided 122 | that such additional attribution notices cannot be construed 123 | as modifying the License. 124 | 125 | You may add Your own copyright statement to Your modifications and 126 | may provide additional or different license terms and conditions 127 | for use, reproduction, or distribution of Your modifications, or 128 | for any such Derivative Works as a whole, provided Your use, 129 | reproduction, and distribution of the Work otherwise complies with 130 | the conditions stated in this License. 131 | 132 | 5. Submission of Contributions. Unless You explicitly state otherwise, 133 | any Contribution intentionally submitted for inclusion in the Work 134 | by You to the Licensor shall be under the terms and conditions of 135 | this License, without any additional terms or conditions. 136 | Notwithstanding the above, nothing herein shall supersede or modify 137 | the terms of any separate license agreement you may have executed 138 | with Licensor regarding such Contributions. 139 | 140 | 6. Trademarks. This License does not grant permission to use the trade 141 | names, trademarks, service marks, or product names of the Licensor, 142 | except as required for reasonable and customary use in describing the 143 | origin of the Work and reproducing the content of the NOTICE file. 144 | 145 | 7. Disclaimer of Warranty. Unless required by applicable law or 146 | agreed to in writing, Licensor provides the Work (and each 147 | Contributor provides its Contributions) on an "AS IS" BASIS, 148 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 149 | implied, including, without limitation, any warranties or conditions 150 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 151 | PARTICULAR PURPOSE. You are solely responsible for determining the 152 | appropriateness of using or redistributing the Work and assume any 153 | risks associated with Your exercise of permissions under this License. 154 | 155 | 8. Limitation of Liability. In no event and under no legal theory, 156 | whether in tort (including negligence), contract, or otherwise, 157 | unless required by applicable law (such as deliberate and grossly 158 | negligent acts) or agreed to in writing, shall any Contributor be 159 | liable to You for damages, including any direct, indirect, special, 160 | incidental, or consequential damages of any character arising as a 161 | result of this License or out of the use or inability to use the 162 | Work (including but not limited to damages for loss of goodwill, 163 | work stoppage, computer failure or malfunction, or any and all 164 | other commercial damages or losses), even if such Contributor 165 | has been advised of the possibility of such damages. 166 | 167 | 9. Accepting Warranty or Additional Liability. While redistributing 168 | the Work or Derivative Works thereof, You may choose to offer, 169 | and charge a fee for, acceptance of support, warranty, indemnity, 170 | or other liability obligations and/or rights consistent with this 171 | License. However, in accepting such obligations, You may act only 172 | on Your own behalf and on Your sole responsibility, not on behalf 173 | of any other Contributor, and only if You agree to indemnify, 174 | defend, and hold each Contributor harmless for any liability 175 | incurred by, or claims asserted against, such Contributor by reason 176 | of your accepting any such warranty or additional liability. 177 | 178 | END OF TERMS AND CONDITIONS 179 | 180 | APPENDIX: How to apply the Apache License to your work. 181 | 182 | To apply the Apache License to your work, attach the following 183 | boilerplate notice, with the fields enclosed by brackets "[]" 184 | replaced with your own identifying information. (Don't include 185 | the brackets!) The text should be enclosed in the appropriate 186 | comment syntax for the file format. We also recommend that a 187 | file or class name and description of purpose be included on the 188 | same "printed page" as the copyright notice for easier 189 | identification within third-party archives. 190 | 191 | Copyright [yyyy] [name of copyright owner] 192 | 193 | Licensed under the Apache License, Version 2.0 (the "License"); 194 | you may not use this file except in compliance with the License. 195 | You may obtain a copy of the License at 196 | 197 | http://www.apache.org/licenses/LICENSE-2.0 198 | 199 | Unless required by applicable law or agreed to in writing, software 200 | distributed under the License is distributed on an "AS IS" BASIS, 201 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 202 | See the License for the specific language governing permissions and 203 | limitations under the License. 204 | -------------------------------------------------------------------------------- /lib/parseOpts.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # parse options lib, support multiple values for one option. 4 | # 5 | # @Usage 6 | # source this script to your script file, then use func parseOpts. 7 | # parseOpts func usage sample: 8 | # $ parseOpts "a,a-long|b,b-long:|c,c-long+" -a -b bv -c c.sh -p pv -q qv arg1 \; aa bb cc 9 | # then below global var is set: 10 | # _OPT_VALUE_a = true 11 | # _OPT_VALUE_a_long = true 12 | # _OPT_VALUE_b = bv 13 | # _OPT_VALUE_b_long = bv 14 | # _OPT_VALUE_c = (c.sh -p pv -q qv arg1) # Array type 15 | # _OPT_VALUE_c_long = (c.sh -p pv -q qv arg1) # Array type 16 | # _OPT_ARGS = (aa bb cc) # Array type 17 | # 18 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/shell.md#-parseoptssh 19 | # @author Jerry Lee (oldratlee at gmail dot com) 20 | 21 | ##################################################################### 22 | # Util Functions 23 | ##################################################################### 24 | 25 | # NOTE: $'foo' is the escape sequence syntax of bash 26 | readonly _opts_ec=$'\e' # escape char 27 | readonly _opts_eend=$'\e[0m' # escape end 28 | 29 | # shellcheck disable=SC2209 30 | 31 | if [ -z "${_opts_SED_CMD:-}" ]; then 32 | _opts_SED_CMD=sed 33 | if command -v gsed &>/dev/null; then 34 | _opts_SED_CMD=gsed 35 | fi 36 | readonly _opts_SED_CMD 37 | fi 38 | 39 | _opts_colorEcho() { 40 | local color=$1 41 | shift 42 | # if stdout is console, turn on color output. 43 | [ -t 1 ] && echo "${_opts_ec}[1;${color}m$*${_opts_eend}" || echo "$*" 44 | } 45 | 46 | _opts_redEcho() { 47 | _opts_colorEcho 31 "$@" 48 | } 49 | 50 | _opts_convertToVarName() { 51 | [ $# -ne 1 ] && { 52 | _opts_redEcho "NOT 1 arguments when call _opts_convertToVarName: $*" 53 | return 1 54 | } 55 | echo "$1" | $_opts_SED_CMD 's/-/_/g' 56 | } 57 | 58 | ##################################################################### 59 | # Parse Functions 60 | # 61 | # Use Globe Variable: 62 | # * _OPT_INFO_LIST_INDEX : Option info, data structure. 63 | # _OPT_INFO_LIST_INDEX ->* _a_a_long -> option value. 64 | # * _OPT_VALUE_* : value of option. is Array type for + mode option 65 | # * _OPT_ARGS : option arguments 66 | ##################################################################### 67 | 68 | _opts_findOptMode() { 69 | [ $# -ne 1 ] && { 70 | _opts_redEcho "NOT 1 arguments when call _opts_findOptMode: $*" 71 | return 1 72 | } 73 | 74 | local opt="$1" # like a, a-long 75 | local idxName 76 | for idxName in "${_OPT_INFO_LIST_INDEX[@]}"; do 77 | local idxNameArrayPlaceHolder="${idxName}[@]" 78 | local -a idxNameArray=("${!idxNameArrayPlaceHolder}") 79 | 80 | local mode="${idxNameArray[0]}" 81 | 82 | local optName 83 | # index from 1, skip mode 84 | for optName in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 85 | [ "$opt" == "${optName}" ] && { 86 | echo "$mode" 87 | return 88 | } 89 | done 90 | done 91 | 92 | echo "" 93 | } 94 | 95 | _opts_setOptBool() { 96 | [ $# -ne 2 ] && { 97 | _opts_redEcho "NOT 2 arguments when call _opts_setOptBool: $*" 98 | return 1 99 | } 100 | 101 | _opts_setOptValue "$@" 102 | } 103 | 104 | _opts_setOptValue() { 105 | [ $# -ne 2 ] && { 106 | _opts_redEcho "NOT 2 arguments when call _opts_setOptValue: $*" 107 | return 1 108 | } 109 | 110 | local opt="$1" # like a, a-long 111 | local value="$2" 112 | 113 | local idxName 114 | for idxName in "${_OPT_INFO_LIST_INDEX[@]}"; do 115 | local idxNameArrayPlaceHolder="${idxName}[@]" 116 | local -a idxNameArray=("${!idxNameArrayPlaceHolder}") 117 | 118 | local optName 119 | # index from 1, skip mode 120 | for optName in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 121 | [ "$opt" == "$optName" ] && { 122 | local optName2 123 | for optName2 in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 124 | local optValueVarName 125 | optValueVarName="_OPT_VALUE_$(_opts_convertToVarName "${optName2}")" 126 | # shellcheck disable=SC2016 127 | local from='"$value"' 128 | # set global var! 129 | eval "$optValueVarName=$from" 130 | done 131 | return 132 | } 133 | done 134 | done 135 | 136 | _opts_redEcho "NOT Found option $opt!" 137 | return 1 138 | } 139 | 140 | _opts_setOptArray() { 141 | local opt="$1" # like a, a-long 142 | shift 143 | 144 | local idxName 145 | for idxName in "${_OPT_INFO_LIST_INDEX[@]}"; do 146 | local idxNameArrayPlaceHolder="${idxName}[@]" 147 | local -a idxNameArray=("${!idxNameArrayPlaceHolder}") 148 | 149 | local optName 150 | # index from 1, skip mode 151 | for optName in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 152 | [ "$opt" == "$optName" ] && { 153 | # set _OPT_VALUE 154 | local optName2 155 | for optName2 in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 156 | local optValueVarName 157 | optValueVarName="_OPT_VALUE_$(_opts_convertToVarName "${optName2}")" 158 | local from='"$@"' 159 | eval "$optValueVarName=($from)" # set global var! 160 | done 161 | return 162 | } 163 | done 164 | done 165 | 166 | _opts_redEcho "NOT Found option $opt!" 167 | return 1 168 | } 169 | 170 | _opts_cleanOptValueInfoList() { 171 | local idxName 172 | for idxName in "${_OPT_INFO_LIST_INDEX[@]}"; do 173 | local idxNameArrayPlaceHolder="${idxName}[@]" 174 | local -a idxNameArray=("${!idxNameArrayPlaceHolder}") 175 | 176 | eval "unset $idxName" 177 | 178 | local optName 179 | # index from 1, skip mode 180 | for optName in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 181 | local optValueVarName 182 | optValueVarName="_OPT_VALUE_$(_opts_convertToVarName "$optName")" 183 | eval "unset $optValueVarName" 184 | done 185 | done 186 | 187 | unset _OPT_INFO_LIST_INDEX 188 | unset _OPT_ARGS 189 | } 190 | 191 | parseOpts() { 192 | local optsDescription="$1" # optsDescription LIKE a,a-long|b,b-long:|c,c-long+ 193 | shift 194 | 195 | _OPT_INFO_LIST_INDEX=() # set global var! 196 | 197 | local optDescLines 198 | optDescLines=$( 199 | echo "$optsDescription" | 200 | $_opts_SED_CMD -r 's/^\s+//;s/\s+$//' | # cut head and tail space 201 | awk -F '[\t ]*\\|[\t ]*' '{for(i=1; i<=NF; i++) print $i}' 202 | ) 203 | 204 | local optDesc # optDesc LIKE b,b-long: 205 | while read -r optDesc; do 206 | [ -z "$optDesc" ] && continue 207 | 208 | # LIKE : or + 209 | local mode="${optDesc:(-1)}" 210 | case "$mode" in 211 | + | : | -) 212 | # LIKE b,b-long 213 | optDesc="${optDesc:0:(${#optDesc} - 1)}" 214 | ;; 215 | *) 216 | mode="-" 217 | ;; 218 | esac 219 | 220 | local optLines # LIKE "a\na-long" 221 | optLines="$(echo "$optDesc" | awk -F '[\t ]*,[\t ]*' '{for(i=1; i<=NF; i++) print $i}')" 222 | 223 | [ "$(echo "$optLines" | wc -l)" -gt 2 ] && { 224 | _opts_redEcho "Illegal option description($optDesc), more than 2 opt name!" 1>&2 225 | _opts_cleanOptValueInfoList 226 | return 220 227 | } 228 | 229 | local -a optTuple=() 230 | local opt # opt LIKE a , a-long 231 | while read -r opt; do 232 | [ -z "$opt" ] && continue 233 | 234 | if [ ${#opt} -eq 1 ]; then 235 | echo "$opt" | grep -E '^[a-zA-Z0-9]$' -q || { 236 | _opts_redEcho "Illegal short option name($opt in $optDesc) in option description!" 1>&2 237 | _opts_cleanOptValueInfoList 238 | return 221 239 | } 240 | else 241 | echo "$opt" | grep -E '^[-a-zA-Z0-9]+$' -q || { 242 | _opts_redEcho "Illegal long option name($opt in $optDesc) in option description!" 1>&2 243 | _opts_cleanOptValueInfoList 244 | return 222 245 | } 246 | fi 247 | optTuple=("${optTuple[@]}" "$opt") 248 | done < <(echo "$optLines") 249 | 250 | [ ${#optTuple[@]} -gt 2 ] && { 251 | _opts_redEcho "more than 2 opt(${optTuple[*]}) in option description($optDesc)!" 1>&2 252 | _opts_cleanOptValueInfoList 253 | return 223 254 | } 255 | 256 | local idxName= 257 | local evalOpts= 258 | local o 259 | for o in "${optTuple[@]}"; do 260 | idxName="${idxName}_opts_index_name_$(_opts_convertToVarName "$o")" 261 | evalOpts="${evalOpts} $o" 262 | done 263 | 264 | eval "$idxName=($mode $evalOpts)" 265 | _OPT_INFO_LIST_INDEX=("${_OPT_INFO_LIST_INDEX[@]}" "$idxName") 266 | done < <(echo "$optDescLines") 267 | 268 | local -a args=() 269 | while true; do 270 | [ $# -eq 0 ] && break 271 | 272 | case "$1" in 273 | ---*) 274 | _opts_redEcho "Illegal option($1), more than 2 prefix '-'!" 1>&2 275 | _opts_cleanOptValueInfoList 276 | return 230 277 | ;; 278 | --) 279 | shift 280 | args=("${args[@]}" "$@") 281 | break 282 | ;; 283 | -*) # short & long option(-a, -a-long), use same read-in logic. 284 | local opt="$1" 285 | local optName 286 | optName=$(echo "$1" | $_opts_SED_CMD -r 's/^--?//') 287 | local mode 288 | mode=$(_opts_findOptMode "$optName") 289 | case "$mode" in 290 | -) 291 | _opts_setOptBool "$optName" "true" 292 | shift 293 | ;; 294 | :) 295 | [ $# -lt 2 ] && { 296 | _opts_redEcho "Option $opt has NO value!" 1>&2 297 | _opts_cleanOptValueInfoList 298 | return 231 299 | } 300 | _opts_setOptValue "$optName" "$2" 301 | shift 2 302 | ;; 303 | +) 304 | shift 305 | local -a valueArray=() 306 | local foundComma="" 307 | 308 | local value 309 | for value in "$@"; do 310 | [ ";" == "$value" ] && { 311 | foundComma=true 312 | break 313 | } || valueArray=("${valueArray[@]}" "$value") 314 | done 315 | [ "$foundComma" ] || { 316 | _opts_redEcho "value of option $opt no end comma, value = ${valueArray[*]}" 1>&2 317 | _opts_cleanOptValueInfoList 318 | return 231 319 | } 320 | shift "$((${#valueArray[@]} + 1))" 321 | _opts_setOptArray "$optName" "${valueArray[@]}" 322 | ;; 323 | *) 324 | _opts_redEcho "Undefined option $opt!" 1>&2 325 | _opts_cleanOptValueInfoList 326 | return 232 327 | ;; 328 | esac 329 | ;; 330 | *) 331 | args=("${args[@]}" "$1") 332 | shift 333 | ;; 334 | esac 335 | done 336 | # set global var! 337 | _OPT_ARGS=("${args[@]}") 338 | } 339 | 340 | ##################################################################### 341 | # Show parsed Option Info Functions 342 | ##################################################################### 343 | 344 | _opts_showOptDescInfoList() { 345 | echo "===============================================================================" 346 | echo "show option desc info list:" 347 | local idxName 348 | for idxName in "${_OPT_INFO_LIST_INDEX[@]}"; do 349 | local idxNameArrayPlaceHolder="${idxName}[@]" 350 | echo "$idxName = (${!idxNameArrayPlaceHolder})" 351 | done 352 | echo "===============================================================================" 353 | } 354 | 355 | _opts_showOptValueInfoList() { 356 | echo "===============================================================================" 357 | echo "show option value info list:" 358 | local idxName 359 | for idxName in "${_OPT_INFO_LIST_INDEX[@]}"; do 360 | local idxNameArrayPlaceHolder="${idxName}[@]" 361 | local -a idxNameArray=("${!idxNameArrayPlaceHolder}") 362 | 363 | local mode=${idxNameArray[0]} 364 | 365 | local optName 366 | # index from 1, skip mode 367 | for optName in "${idxNameArray[@]:1:${#idxNameArray[@]}}"; do 368 | local optValueVarName 369 | optValueVarName="_OPT_VALUE_$(_opts_convertToVarName "$optName")" 370 | case "$mode" in 371 | -) 372 | echo "$optValueVarName=${!optValueVarName}" 373 | ;; 374 | :) 375 | echo "$optValueVarName=${!optValueVarName}" 376 | ;; 377 | +) 378 | local optArrayValueArrayPlaceHolder="${optValueVarName}[@]" 379 | echo "$optValueVarName=(${!optArrayValueArrayPlaceHolder})" 380 | ;; 381 | esac 382 | done 383 | done 384 | echo "_OPT_ARGS=(${_OPT_ARGS[*]})" 385 | echo "===============================================================================" 386 | } 387 | -------------------------------------------------------------------------------- /bin/find-in-jars: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Find files in the jar files under specified directory, search jar files recursively(include subdirectory). 4 | # 5 | # @Usage 6 | # $ find-in-jars 'log4j\.properties' 7 | # # search file log4j.properties/log4j.xml at zip root 8 | # $ find-in-jars '^log4j\.(properties|xml)$' 9 | # $ find-in-jars 'log4j\.properties$' -d /path/to/find/directory 10 | # $ find-in-jars '\.properties$' -d /path/to/find/dir1 -d path/to/find/dir2 11 | # $ find-in-jars 'Service\.class$' -e jar -e zip 12 | # $ find-in-jars 'Mon[^$/]*Service\.class$' -s ' <-> ' 13 | # 14 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/java.md#-find-in-jars 15 | # @author Jerry Lee (oldratlee at gmail dot com) 16 | set -eEuo pipefail 17 | 18 | readonly PROG=${0##*/} 19 | readonly PROG_VERSION='3.x-dev' 20 | 21 | ################################################################################ 22 | # util functions 23 | ################################################################################ 24 | 25 | readonly COLOR_RESET='\e[0m' 26 | 27 | redPrint() { 28 | # if stdout is a terminal, turn on color output. 29 | # '-t' check: is a terminal? 30 | # check isatty in bash https://stackoverflow.com/questions/10022323 31 | if [ -t 1 ]; then 32 | printf "\e[1;31m%s$COLOR_RESET\n" "$*" 33 | else 34 | printf '%s\n' "$*" 35 | fi 36 | } 37 | 38 | # How to delete line with echo? 39 | # https://unix.stackexchange.com/questions/26576 40 | # 41 | # terminal escapes: http://ascii-table.com/ansi-escape-sequences.php 42 | # In particular, to clear from the cursor position to the beginning of the line: 43 | # echo -e "\033[1K" 44 | # Or everything on the line, regardless of cursor position: 45 | # echo -e "\033[2K" 46 | readonly LINE_CLEAR='\e[2K\r' 47 | 48 | # Getting console width using a bash script 49 | # https://unix.stackexchange.com/questions/299067 50 | [ -t 2 ] && COLUMNS=$(stty size | awk '{print $2}') 51 | 52 | printResponsiveMessage() { 53 | if ! $show_responsive || [ ! -t 2 ]; then 54 | return 55 | fi 56 | 57 | local content=$* 58 | # http://www.linuxforums.org/forum/red-hat-fedora-linux/142825-how-truncate-string-bash-script.html 59 | printf %b%s "$LINE_CLEAR" "${content:0:COLUMNS}" >&2 60 | } 61 | 62 | clearResponsiveMessage() { 63 | if ! $show_responsive || [ ! -t 2 ]; then 64 | return 65 | fi 66 | 67 | printf %b "$LINE_CLEAR" >&2 68 | } 69 | 70 | die() { 71 | local prompt_help=false exit_status=2 72 | while (($# > 0)); do 73 | case "$1" in 74 | -h) 75 | prompt_help=true 76 | shift 77 | ;; 78 | -s) 79 | exit_status=$2 80 | shift 2 81 | ;; 82 | *) 83 | break 84 | ;; 85 | esac 86 | done 87 | 88 | clearResponsiveMessage 89 | (($# > 0)) && redPrint "$PROG: $*" 90 | $prompt_help && echo "Try '$PROG --help' for more information." 91 | 92 | exit "$exit_status" 93 | } >&2 94 | 95 | usage() { 96 | cat < ' 111 | 112 | Find control: 113 | -d, --dir the directory that find jar files. 114 | default is current directory. this option can specify 115 | multiply times to find in multiply directories. 116 | -e, --extension set find file extension, default is jar. this option 117 | can specify multiply times to find multiply extension. 118 | -E, --extended-regexp PATTERN is an extended regular expression (*default*) 119 | -F, --fixed-strings PATTERN is a set of newline-separated strings 120 | -G, --basic-regexp PATTERN is a basic regular expression 121 | -P, --perl-regexp PATTERN is a Perl regular expression 122 | -i, --ignore-case ignore case distinctions 123 | 124 | Output control: 125 | -a, --absolute-path always print absolute path of jar file 126 | -s, --separator specify the separator between jar file and zip entry. 127 | default is \`!'. 128 | -L, --files-not-contained-found 129 | print only names of JAR FILEs NOT contained found 130 | -l, --files-contained-found 131 | print only names of JAR FILEs contained found 132 | -R, --no-find-progress do not display responsive find progress 133 | 134 | Miscellaneous: 135 | -h, --help display this help and exit 136 | -V, --version display version information and exit 137 | EOF 138 | 139 | exit 140 | } 141 | 142 | progVersion() { 143 | printf '%s\n' "$PROG $PROG_VERSION" 144 | exit 145 | } 146 | 147 | ################################################################################ 148 | # parse options 149 | ################################################################################ 150 | 151 | dirs=() 152 | extensions=() 153 | args=() 154 | 155 | separator='!' 156 | regex_mode=-E 157 | use_absolute_path=false 158 | show_responsive=true 159 | only_print_file_name=false 160 | 161 | while (($# > 0)); do 162 | case "$1" in 163 | -d | --dir) 164 | dirs=(${dirs[@]:+"${dirs[@]}"} "$2") 165 | shift 2 166 | ;; 167 | -e | --extension) 168 | extensions=(${extensions[@]:+"${extensions[@]}"} "$2") 169 | shift 2 170 | ;; 171 | -E | --extended-regexp) 172 | regex_mode=-E 173 | shift 174 | ;; 175 | -F | --fixed-strings) 176 | regex_mode=-F 177 | shift 178 | ;; 179 | -G | --basic-regexp) 180 | regex_mode=-G 181 | shift 182 | ;; 183 | -P | --perl-regexp) 184 | regex_mode=-P 185 | shift 186 | ;; 187 | -i | --ignore-case) 188 | ignore_case_option=-i 189 | shift 190 | ;; 191 | -a | --absolute-path) 192 | use_absolute_path=true 193 | shift 194 | ;; 195 | # support the legacy typo option name --seperator for compatibility 196 | -s | --separator | --seperator) 197 | separator=$2 198 | shift 2 199 | ;; 200 | -L | --files-not-contained-found) 201 | only_print_file_name=true 202 | print_matched_files=false 203 | shift 204 | ;; 205 | -l | --files-contained-found) 206 | only_print_file_name=true 207 | print_matched_files=true 208 | shift 209 | ;; 210 | -R | --no-find-progress) 211 | show_responsive=false 212 | shift 213 | ;; 214 | -h | --help) 215 | usage 216 | ;; 217 | -V | --version) 218 | progVersion 219 | ;; 220 | --) 221 | shift 222 | args=(${args[@]:+"${args[@]}"} "$@") 223 | break 224 | ;; 225 | -*) 226 | die -h "unrecognized option '$1'" 227 | ;; 228 | *) 229 | args=(${args[@]:+"${args[@]}"} "$1") 230 | shift 231 | ;; 232 | esac 233 | done 234 | 235 | readonly separator regex_mode ignore_case_option use_absolute_path only_print_file_name print_matched_files show_responsive args 236 | 237 | # shellcheck disable=SC2178 238 | dirs=${dirs:-.} 239 | # shellcheck disable=SC2178 240 | readonly extensions=${extensions:-jar} 241 | 242 | ((${#args[@]} == 0)) && die -h "requires file pattern!" 243 | ((${#args[@]} > 1)) && die -h "more than 1 file pattern: ${args[*]}" 244 | readonly pattern=${args[0]} 245 | 246 | tmp_dirs=() 247 | for d in "${dirs[@]}"; do 248 | [ -e "$d" ] || die "file $d(specified by option -d): No such file or directory!" 249 | [ -d "$d" ] || die "file $d(specified by option -d) exists but is not a directory!" 250 | [ -r "$d" ] || die "directory $d(specified by option -d) exists but is not readable!" 251 | 252 | # convert dirs to Absolute Path if has option -a, --absolute-path 253 | $use_absolute_path && tmp_dirs=(${tmp_dirs[@]:+"${tmp_dirs[@]}"} "$(cd "$d" && pwd)") 254 | done 255 | # set dirs to Absolute Path 256 | $use_absolute_path && dirs=("${tmp_dirs[@]}") 257 | readonly dirs 258 | unset d tmp_dirs 259 | 260 | # convert extensions to find -iname options 261 | find_iname_options=() 262 | for e in "${extensions[@]}"; do 263 | find_iname_options=(${find_iname_options[@]:+"${find_iname_options[@]}" -o} -iname "*.$e") 264 | done 265 | readonly find_iname_options 266 | unset e 267 | 268 | ################################################################################ 269 | # Check the existence of command for listing zip entry! 270 | ################################################################################ 271 | 272 | __prepareCommandToListZipEntries() { 273 | # `zipinfo -1`/`unzip -Z1` is ~25 times faster than `jar tf`, find zipinfo/unzip command first. 274 | # 275 | # How to list files in a zip without extra information in command line 276 | # https://unix.stackexchange.com/a/128304/136953 277 | 278 | if type -P zipinfo &>/dev/null; then 279 | command_to_list_zip_entries=(zipinfo -1) 280 | is_use_zip_cmd_to_list_zip_entries=true 281 | elif type -P unzip &>/dev/null; then 282 | command_to_list_zip_entries=(unzip -Z1) 283 | is_use_zip_cmd_to_list_zip_entries=true 284 | elif [ -n "$JAVA_HOME" ]; then 285 | # search jar command under JAVA_HOME 286 | if [ -f "$JAVA_HOME/bin/jar" ]; then 287 | [ -x "$JAVA_HOME/bin/jar" ] || die "found \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) is NOT executable!" 288 | command_to_list_zip_entries=("$JAVA_HOME/bin/jar" tf) 289 | elif [ -f "$JAVA_HOME/../bin/jar" ]; then 290 | [ -x "$JAVA_HOME/../bin/jar" ] || die "found \$JAVA_HOME/../bin/jar($JAVA_HOME/../bin/jar) is NOT executable!" 291 | command_to_list_zip_entries=("$JAVA_HOME/../bin/jar" tf) 292 | fi 293 | is_use_zip_cmd_to_list_zip_entries=false 294 | elif type -P jar &>/dev/null; then 295 | # search jar command under PATH 296 | command_to_list_zip_entries=(jar tf) 297 | is_use_zip_cmd_to_list_zip_entries=false 298 | else 299 | die "command to list zip entries NOT found : zipinfo, unzip or jar!" 300 | fi 301 | 302 | readonly command_to_list_zip_entries is_use_zip_cmd_to_list_zip_entries 303 | } 304 | __prepareCommandToListZipEntries 305 | 306 | listZipEntries() { 307 | local zip_file=$1 msg 308 | 309 | if $is_use_zip_cmd_to_list_zip_entries; then 310 | # How to check if zip file is empty in bash 311 | # https://superuser.com/questions/438878 312 | msg=$("${command_to_list_zip_entries[@]}" -t "$zip_file" 2>&1) || { 313 | # NOTE: 314 | # if list emtpy zip file by zipinfo/unzip command, 315 | # exit code is 1, and print 'Empty zipfile.' 316 | if [ "$msg" != 'Empty zipfile.' ]; then 317 | clearResponsiveMessage 318 | redPrint "fail to list zip entries of $zip_file, ignored: $msg" >&2 319 | fi 320 | return 321 | } 322 | fi 323 | 324 | "${command_to_list_zip_entries[@]}" "$zip_file" || { 325 | clearResponsiveMessage 326 | redPrint "fail to list zip entries of $zip_file, ignored!" >&2 327 | } 328 | } 329 | 330 | ################################################################################ 331 | # find logic 332 | ################################################################################ 333 | 334 | searchJarFiles() { 335 | printResponsiveMessage "searching jars under dir ${dirs[*]} , ..." 336 | 337 | local jar_files total_jar_count 338 | 339 | jar_files=$(find "${dirs[@]}" "${find_iname_options[@]}" -type f) 340 | [ -n "$jar_files" ] || die "${extensions[*]} file NOT found!" 341 | 342 | total_jar_count=$(printf '%s\n' "$jar_files" | wc -l) 343 | # remove white space, because the `wc -l` output on mac contains white space! 344 | total_jar_count=${total_jar_count//[[:space:]]/} 345 | 346 | echo "$total_jar_count" 347 | printf '%s\n' "$jar_files" 348 | } 349 | 350 | readonly JAR_COLOR='\e[1;35m' SEP_COLOR='\e[1;32m' 351 | __outputResultOfJarFile() { 352 | local jar_file=$1 file 353 | # shellcheck disable=SC2206 354 | local grep_opt_args=("$regex_mode" ${ignore_case_option:-} ${grep_color_option:-} -- "$pattern") 355 | 356 | if $only_print_file_name; then 357 | local matched=false 358 | # NOTE: Do NOT use -q flag with grep: 359 | # With the -q flag the grep program will stop immediately when the first line of data matches. 360 | # Normally you shouldn't use -q in a pipeline like this 361 | # unless you are sure the program at the other end can handle SIGPIPE. 362 | # more info see: 363 | # - https://stackoverflow.com/questions/19120263/why-exit-code-141-with-grep-q 364 | # - https://unix.stackexchange.com/questions/305547/broken-pipe-when-grepping-output-but-only-with-i-flag 365 | # - http://www.pixelbeat.org/programming/sigpipe_handling.html 366 | grep -c "${grep_opt_args[@]}" &>/dev/null && matched=true 367 | 368 | [ "$print_matched_files" != "$matched" ] && return 369 | 370 | clearResponsiveMessage 371 | if [ -t 1 ]; then 372 | printf "$JAR_COLOR%s$COLOR_RESET\n" "$jar_file" 373 | else 374 | printf '%s\n' "$jar_file" 375 | fi 376 | else 377 | { 378 | # Prevent grep from exiting in case of no match 379 | # https://unix.stackexchange.com/questions/330660 380 | grep "${grep_opt_args[@]}" || true 381 | } | while IFS= read -r file; do 382 | clearResponsiveMessage 383 | if [ -t 1 ]; then 384 | printf "$JAR_COLOR%s$SEP_COLOR%s$COLOR_RESET%s\n" "$jar_file" "$separator" "$file" 385 | else 386 | printf '%s\n' "$jar_file$separator$file" 387 | fi 388 | done 389 | fi 390 | } 391 | 392 | findInJarFiles() { 393 | [ -t 1 ] && local -r grep_color_option='--color=always' 394 | local counter=1 total_jar_count jar_file 395 | 396 | read -r total_jar_count 397 | while IFS= read -r jar_file; do 398 | printResponsiveMessage "finding in jar($((counter++))/$total_jar_count): $jar_file" 399 | listZipEntries "$jar_file" | __outputResultOfJarFile "$jar_file" 400 | done 401 | 402 | clearResponsiveMessage 403 | } 404 | 405 | searchJarFiles | findInJarFiles 406 | -------------------------------------------------------------------------------- /bin/show-duplicate-java-classes: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Function 4 | # Find duplicate classes among java lib dirs and class dirs. 5 | # 6 | # @Usage 7 | # $ show-duplicate-java-classes # search jars from current dir 8 | # $ show-duplicate-java-classes path/to/lib_dir1 /path/to/lib_dir2 9 | # $ show-duplicate-java-classes -c path/to/class_dir1 -c /path/to/class_dir2 10 | # $ show-duplicate-java-classes -c path/to/class_dir1 path/to/lib_dir1 11 | # $ show-duplicate-java-classes -L path/to/lib_dir1 # search jars in the subdirectories of lib dir 12 | # $ show-duplicate-java-classes -J path/to/lib_dir1 # search jars in the jar file 13 | # 14 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/java.md#-show-duplicate-java-classes 15 | # @author tg123 (farmer1992 at gmail dot com) 16 | # @author Jerry Lee (oldratlee at gmail dot com) 17 | 18 | __author__ = 'tg123' 19 | 20 | import os 21 | import re 22 | import sys 23 | from glob import glob 24 | from io import BytesIO 25 | from optparse import OptionParser 26 | from os import walk 27 | from os.path import exists, isdir, relpath 28 | from zipfile import BadZipfile, ZipFile 29 | 30 | ################################################################################ 31 | # utils functions 32 | ################################################################################ 33 | PROG_VERSION = '3.x-dev' 34 | 35 | # How to delete line with echo? 36 | # https://unix.stackexchange.com/questions/26576 37 | # 38 | # terminal escapes: http://ascii-table.com/ansi-escape-sequences.php 39 | # In particular, to clear from the cursor position to the beginning of the line: 40 | # echo -e "\033[1K" 41 | # Or everything on the line, regardless of cursor position: 42 | # echo -e "\033[2K" 43 | __clear_line = '\033[2K\r' 44 | __show_responsive = True 45 | 46 | 47 | def __get_terminal_columns_of_stderr(): 48 | """ 49 | Rewritten for stderr from 50 | """ 51 | try: 52 | columns, _ = os.get_terminal_size(sys.stderr.fileno()) 53 | except (AttributeError, ValueError, OSError): 54 | columns = 0 55 | 56 | return columns 57 | 58 | 59 | def print_responsive_message(msg): 60 | if not __show_responsive or not sys.stderr.isatty(): 61 | return 62 | columns = __get_terminal_columns_of_stderr() 63 | if columns <= 0: 64 | return 65 | 66 | print(__clear_line + msg[:columns], end='', file=sys.stderr) 67 | 68 | 69 | def clear_responsive_message(): 70 | if not __show_responsive or not sys.stderr.isatty(): 71 | return 72 | print(__clear_line, end='', file=sys.stderr) 73 | 74 | 75 | def print_error(msg): 76 | clear_responsive_message() 77 | print(msg, file=sys.stderr) 78 | 79 | 80 | def print_box_message(msg): 81 | print() 82 | print('=' * 80) 83 | print(msg) 84 | print('=' * 80) 85 | 86 | 87 | def str_len(x): 88 | return len(str(x)) 89 | 90 | 91 | # issue 32790: Keep trailing zeros in precision for string format option g - Python tracker 92 | # https://bugs.python.org/issue32790 93 | def percent_str(num): 94 | """ 95 | Input => Output 96 | 1.4545 / 10 **-1 => 1455% 97 | 1.4545 / 10 ** 0 => 145% 98 | 1.4545 / 10 ** 1 => 14.5% 99 | 1.4545 / 10 ** 2 => 1.45% 100 | 1.4545 / 10 ** 3 => 0.145% 101 | 1.4545 / 10 ** 4 => 0.015% 102 | 1.4545 / 10 ** 5 => 0.001% 103 | 1.4545 / 10 ** 6 => 0.000% 104 | 1.4545 / 10 ** 7 => 0.000% 105 | """ 106 | num = num * 100 107 | if num >= 100: 108 | return '%.0f%%' % num 109 | elif num >= 10: 110 | return '%.1f%%' % num 111 | elif num >= 1: 112 | return '%.2f%%' % num 113 | else: 114 | return '%.3f%%' % num 115 | 116 | 117 | def list_jar_file_under_lib_dirs(lib_dirs, recursive): 118 | jar_files = set() 119 | 120 | max_idx_str_len = str_len(len(lib_dirs)) 121 | for idx, lib_dir in enumerate(lib_dirs, start=1): 122 | print_responsive_message('list jar file under lib dir(%*s/%s): %s' % ( 123 | max_idx_str_len, idx, len(lib_dirs), lib_dir)) 124 | 125 | if not exists(lib_dir): 126 | print_error('WARN: lib dir %s not exists, ignored!' % lib_dir) 127 | continue 128 | 129 | if not isdir(lib_dir): 130 | jar_files.add(lib_dir) 131 | continue 132 | 133 | if not recursive: 134 | jar_files |= {p for p in glob(lib_dir + '/*.jar')} 135 | continue 136 | 137 | jar_files |= { 138 | dir_path + '/' + filename 139 | for dir_path, _, file_names in walk(lib_dir) 140 | for filename in file_names if filename.lower().endswith('.jar') 141 | } 142 | 143 | return jar_files 144 | 145 | 146 | def list_class_under_jar_file(jar_file, recursive, progress): 147 | """ 148 | :return: map: jar_jar_path('a.jar!b.jar!c.jar') -> classes 149 | """ 150 | index = 0 151 | 152 | def list_zip_in_zip(jar_jar_path, zf): 153 | nonlocal index 154 | index += 1 155 | index_marker = '' 156 | if recursive: 157 | index_marker = ' #%3s' % index 158 | print_responsive_message('list class under jar file(%*s/%s%s): %s' % ( 159 | str_len(progress[1]), progress[0], progress[1], index_marker, jar_jar_path)) 160 | 161 | ret = {} 162 | classes = {f for f in zf.namelist() if f.lower().endswith('.class')} 163 | ret[jar_jar_path] = classes 164 | if not recursive: 165 | return ret 166 | 167 | jars_in_jar = {f for f in zf.namelist() if f.lower().endswith('.jar')} 168 | for jar in jars_in_jar: 169 | next_jar_jar_path = jar_jar_path + '!' + jar 170 | try: 171 | with ZipFile(BytesIO(zf.read(jar))) as f: 172 | ret.update(list_zip_in_zip(next_jar_jar_path, f)) 173 | except BadZipfile as e: 174 | print_error('WARN: %s is bad zip file(%s), ignored!' % (next_jar_jar_path, e)) 175 | 176 | return ret 177 | 178 | try: 179 | with ZipFile(jar_file) as zip_file: 180 | return list_zip_in_zip(jar_file, zip_file) 181 | except BadZipfile as error: 182 | print_error('WARN: %s is bad zip file(%s), ignored!' % (jar_file, error)) 183 | return {} 184 | 185 | 186 | def list_class_under_class_dir(class_dir, progress): 187 | print_responsive_message('list class under class dir(%*s/%s): %s' % ( 188 | str_len(progress[1]), progress[0], progress[1], class_dir)) 189 | 190 | if not exists(class_dir): 191 | print_error('WARN: class dir %s not exists, ignored!' % class_dir) 192 | return {} 193 | if not isdir(class_dir): 194 | print_error('WARN: class dir %s is not dir, ignored!' % class_dir) 195 | return {} 196 | 197 | return {relpath(dir_path + '/' + filename, class_dir) 198 | for dir_path, _, file_names in walk(class_dir) 199 | for filename in file_names if filename.lower().endswith('.class')} 200 | 201 | 202 | def collect_class_path_to_classes(class_dirs, jar_files, recursive_jar): 203 | class_path_to_classes = {} 204 | total_count = len(jar_files) + len(class_dirs) 205 | index = 0 206 | 207 | # list all classes in jar files 208 | for jar_file in jar_files: 209 | index += 1 210 | class_path_to_classes.update( 211 | list_class_under_jar_file(jar_file, recursive=recursive_jar, progress=(index, total_count))) 212 | # list all classes in class dirs 213 | for class_dir in class_dirs: 214 | index += 1 215 | class_path_to_classes[class_dir] = list_class_under_class_dir(class_dir, progress=(index, total_count)) 216 | return class_path_to_classes 217 | 218 | 219 | def invert_as_class_to_class_paths(class_path_to_classes): 220 | class_to_class_paths = {} 221 | for class_path, classes in class_path_to_classes.items(): 222 | for clazz in classes: 223 | class_to_class_paths.setdefault(clazz, set()).add(class_path) 224 | return class_to_class_paths 225 | 226 | 227 | ################################################################################ 228 | # biz functions 229 | ################################################################################ 230 | 231 | __java9_module_file_pattern = re.compile(r'(^|.*/)module-info\.class$') 232 | 233 | 234 | def find_duplicate_classes(class_to_class_paths): 235 | class_paths_to_duplicate_classes = {} 236 | 237 | for clazz, class_paths in class_to_class_paths.items(): 238 | # skip java 9 module-info files 239 | if len(class_paths) == 1 or __java9_module_file_pattern.match(clazz): 240 | continue 241 | 242 | classes = class_paths_to_duplicate_classes.setdefault(frozenset(class_paths), set()) 243 | classes.add(clazz) 244 | 245 | return class_paths_to_duplicate_classes 246 | 247 | 248 | def print_duplicate_classes_info(class_paths_to_duplicate_classes, class_path_to_classes): 249 | if not class_paths_to_duplicate_classes: 250 | print('COOL! No duplicate classes found!') 251 | return 252 | 253 | duplicate_classes_total_count = sum(len(dcs) for dcs in class_paths_to_duplicate_classes.values()) 254 | class_paths_total_count = sum(len(cps) for cps in class_paths_to_duplicate_classes) 255 | print('Found %s duplicate classes in %s class paths and %s class path sets:' % ( 256 | duplicate_classes_total_count, class_paths_total_count, len(class_paths_to_duplicate_classes))) 257 | 258 | # sort key(class_paths) and value(duplicate_classes) 259 | class_paths_to_duplicate_classes = [(sorted(cps), sorted(dcs)) 260 | for cps, dcs in class_paths_to_duplicate_classes.items()] 261 | # sort kv pairs 262 | # 263 | # sort by multiple keys: 264 | # 1. class paths count, *descending*; aka. sort by len(item[0]) *reverse=True* 265 | # 2. duplicate classes count, *descending*; aka. sort by len(item[1]) *reverse=True* 266 | # 3. class paths, ascending; aka. sort by item[0] 267 | # sort also ensure output consistent for same input. 268 | # 269 | # How to sort objects by multiple keys in Python? 270 | # https://stackoverflow.com/questions/1143671 271 | # Sort a list by multiple attributes? 272 | # https://stackoverflow.com/questions/4233476 273 | # 274 | # use - operator of number key for reverse sort key 275 | class_paths_to_duplicate_classes.sort(key=lambda item: (-len(item[0]), -len(item[1]), item[0])) 276 | 277 | max_idx_str_len = str_len(len(class_paths_to_duplicate_classes)) 278 | for idx, (class_paths, classes) in enumerate(class_paths_to_duplicate_classes, start=1): 279 | duplicate_ratio = len(classes) / min((len(class_path_to_classes[cp]) for cp in class_paths)) 280 | print('[%*s] found %s(%s) duplicate classes in %s class paths:' % ( 281 | max_idx_str_len, idx, len(classes), percent_str(duplicate_ratio), len(class_paths))) 282 | 283 | max_class_path_idx_str_len = str_len(len(class_paths)) 284 | max_classes_count_str_len = str_len(max(len(class_path_to_classes[cp]) for cp in class_paths)) 285 | for i, cp in enumerate(class_paths, start=1): 286 | print(' %*s: (contain %*s classes) %s' % ( 287 | max_class_path_idx_str_len, i, max_classes_count_str_len, len(class_path_to_classes[cp]), cp)) 288 | 289 | print_box_message('Duplicate classes detail info:') 290 | for idx, (class_paths, classes) in enumerate(class_paths_to_duplicate_classes, start=1): 291 | print('[%*s] found %s duplicate classes in %s class paths %s :' % ( 292 | max_idx_str_len, idx, len(classes), len(class_paths), ' '.join(class_paths))) 293 | 294 | max_class_idx_str_len = str_len(len(classes)) 295 | for i, c in enumerate(classes, start=1): 296 | print(' %*s: %s' % (max_class_idx_str_len, i, c)) 297 | 298 | 299 | def print_class_paths_info(class_path_to_classes): 300 | if not class_path_to_classes: 301 | return 302 | 303 | max_idx_str_len = str_len(len(class_path_to_classes)) 304 | max_classes_count_str_len = str_len(max(len(classes) for classes in class_path_to_classes.values())) 305 | 306 | class_path_to_classes = sorted(class_path_to_classes.items(), key=lambda item: item[0]) 307 | print_box_message('Find in %s class paths:' % len(class_path_to_classes)) 308 | for idx, (cp, classes) in enumerate(class_path_to_classes, start=1): 309 | print('%*s: (contain %*s classes) %s' % ( 310 | max_idx_str_len, idx, max_classes_count_str_len, len(classes), cp)) 311 | 312 | 313 | def main(): 314 | option_parser = OptionParser( 315 | usage='%prog [OPTION]...' 316 | ' [-c class-dir1 [-c class-dir2] ...]' 317 | ' [lib-dir1|jar-file1 [lib-dir2|jar-file2] ...]' 318 | '\nFind duplicate classes among java lib dirs and class dirs.' 319 | '\n\nExamples:' 320 | '\n %prog # search jars from current dir' 321 | '\n %prog path/to/lib_dir1 /path/to/lib_dir2' 322 | '\n %prog -c path/to/class_dir1 -c /path/to/class_dir2' 323 | '\n %prog -c path/to/class_dir1 path/to/lib_dir1' 324 | '\n %prog -L path/to/lib_dir1' 325 | '\n %prog -J path/to/lib_dir1', 326 | version='%prog ' + PROG_VERSION) 327 | option_parser.add_option('-L', '--recursive-lib', dest='recursive_lib', default=False, 328 | action='store_true', help='search jars in the sub-directories of lib dir') 329 | option_parser.add_option('-J', '--recursive-jar', dest='recursive_jar', default=False, 330 | action='store_true', help='search jars in the jar file') 331 | option_parser.add_option('-c', '--class-dir', dest='class_dirs', default=[], 332 | action='append', help='add class dir') 333 | option_parser.add_option('-R', '--no-find-progress', dest='show_responsive', default=True, 334 | action='store_false', help='do not display responsive find progress') 335 | 336 | options, lib_dirs = option_parser.parse_args() 337 | class_dirs = options.class_dirs 338 | if not lib_dirs and not class_dirs: 339 | lib_dirs = ['.'] 340 | global __show_responsive 341 | __show_responsive = options.show_responsive 342 | 343 | jar_files = list_jar_file_under_lib_dirs(lib_dirs, recursive=options.recursive_lib) 344 | if not jar_files and not class_dirs: 345 | clear_responsive_message() 346 | print('search no jar files under lib dirs, and class dirs is absent.') 347 | return 0 348 | class_path_to_classes = collect_class_path_to_classes(class_dirs, jar_files, options.recursive_jar) 349 | if all(not classes for classes in class_path_to_classes.values()): 350 | clear_responsive_message() 351 | print('find no class files in jar files or class dirs.') 352 | return 0 353 | 354 | print_responsive_message('find duplicate classes...') 355 | class_to_class_paths = invert_as_class_to_class_paths(class_path_to_classes) 356 | class_paths_to_duplicate_classes = find_duplicate_classes(class_to_class_paths) 357 | 358 | clear_responsive_message() 359 | print_duplicate_classes_info(class_paths_to_duplicate_classes, class_path_to_classes) 360 | print_class_paths_info(class_path_to_classes) 361 | 362 | return int(bool(class_paths_to_duplicate_classes)) 363 | 364 | 365 | if __name__ == '__main__': 366 | exit(main()) 367 | -------------------------------------------------------------------------------- /docs/shell.md: -------------------------------------------------------------------------------- 1 | 🐌 `Shell`相关脚本 2 | ==================================== 3 | 4 | 5 | 6 | 7 | - [`Shell`使用加强](#shell%E4%BD%BF%E7%94%A8%E5%8A%A0%E5%BC%BA) 8 | - [🍺 c](#-c) 9 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B) 10 | - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99) 11 | - [🍺 coat and taoc](#-coat-and-taoc) 12 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-1) 13 | - [🍺 a2l](#-a2l) 14 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-2) 15 | - [🍺 uq](#-uq) 16 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-3) 17 | - [🍺 ap and rp](#-ap-and-rp) 18 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-4) 19 | - [🍺 cp-into-docker-run](#-cp-into-docker-run) 20 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-5) 21 | - [🍺 tcp-connection-state-counter](#-tcp-connection-state-counter) 22 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-6) 23 | - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85) 24 | - [🍺 xpl and xpf](#-xpl-and-xpf) 25 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-7) 26 | - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-1) 27 | - [`Shell`开发/测试加强](#shell%E5%BC%80%E5%8F%91%E6%B5%8B%E8%AF%95%E5%8A%A0%E5%BC%BA) 28 | - [🍺 echo-args](#-echo-args) 29 | - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B-8) 30 | - [使用方式](#%E4%BD%BF%E7%94%A8%E6%96%B9%E5%BC%8F) 31 | - [🍺 console-text-color-themes.sh](#-console-text-color-themessh) 32 | - [用法](#%E7%94%A8%E6%B3%95) 33 | - [示例](#%E7%A4%BA%E4%BE%8B) 34 | - [运行效果](#%E8%BF%90%E8%A1%8C%E6%95%88%E6%9E%9C) 35 | - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-2) 36 | - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99-1) 37 | - [🍺 parseOpts.sh](#-parseoptssh) 38 | - [用法](#%E7%94%A8%E6%B3%95-1) 39 | - [示例](#%E7%A4%BA%E4%BE%8B-1) 40 | - [兼容性](#%E5%85%BC%E5%AE%B9%E6%80%A7) 41 | - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-3) 42 | 43 | 44 | 45 | `Shell`使用加强 46 | ==================================== 47 | 48 | 🍺 [c](../bin/c) 49 | ---------------------- 50 | 51 | 原样命令行输出，并拷贝标准输出到系统剪贴板，省去`CTRL+C`操作，优化命令行与其它应用之间的操作流。 52 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 53 | 54 | 命令名`c`的意思是`Copy`，因为这个命令我平时非常常用，所以使用一个字符的命令名，方便快速键入。 55 | 56 | 更多说明参见[拷贝复制命令行输出放在系统剪贴板上](http://oldratlee.github.io/post/2012-12-23/command-output-to-clip)。 57 | 58 | ### 用法/示例 59 | 60 | 有3种使用风格，根据需要或是你的偏好选取。 61 | 62 | ```bash 63 | ############################################################ 64 | # 1. 前缀方式，后面跟上要运行的命令 65 | ############################################################ 66 | $ c pwd 67 | /Users/jerry 68 | $ c echo -e 'a\nb' 69 | a 70 | b 71 | # 这种使用方式，后面跟的命令不能是别名（alias），对于别名可以用下面的使用方式。 72 | 73 | ############################################################ 74 | # 2. 后缀方式，管道 75 | ############################################################ 76 | $ echo -e 'a\nb' | nl | c 77 | 1 a 78 | 2 b 79 | # gb是oh-my-zsh的别名，列出git的分支，需要后缀的方式的使用。 80 | $ gb | c 81 | 82 | ############################################################ 83 | # 3. 从标准输入读取内容。拷贝文件内容时这种方式最直接。 84 | ############################################################ 85 | $ c < ~/.ssh/id_rsa.pub 86 | ssh-rsa EAAAABIwAAAQEAz+ETZEgoLeIiC0rjWewdDs0sbo8c...== a@b.com 87 | 88 | ############################################################ 89 | # -q选项：拷贝但不输出。 90 | # 当输出内容比较多、又不关心输出内容和命令执行进展时，可以使用这个选项。 91 | ############################################################ 92 | $ c -q < ~/.ssh/id_rsa.pub 93 | 94 | # 帮助信息 95 | $ c --help 96 | Usage: c [OPTION]... [command [command_args ...]] 97 | Run command and put output to system clipper. 98 | If no command is specified, read from stdin(pipe). 99 | 100 | Example: 101 | c grep -i 'hello world' menu.h main.c 102 | set | c 103 | c -q < ~/.ssh/id_rsa.pub 104 | 105 | Options: 106 | -k, --keep-eol do not trim new line at end of file 107 | -q, --quiet suppress all normal output, default is false 108 | -h, --help display this help and exit 109 | -V, --version display version information and exit 110 | ``` 111 | 112 | ### 参考资料 113 | 114 | - [拷贝复制命令行输出放在系统剪贴板上](http://oldratlee.github.io/post/2012-12-23/command-output-to-clip)，给出了不同系统可用命令。 115 | - 关于文本文件最后的换行，参见[Why should text files end with a newline?](https://stackoverflow.com/questions/729692) 116 | 117 | 118 | 119 | 🍺 [coat](../bin/coat) and [taoc](../bin/taoc) 120 | ---------------------- 121 | 122 | 彩色`cat`/`tac`出文件行，方便人眼区分不同的行。 123 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 124 | 125 | 命令支持选项、功能和使用方式与[`cat`](https://manned.org/cat)/[`tac`](https://manned.org/tac)命令完全一样。 126 | 文件操作在实现上完全代理给了`cat`/`tac`命令。 127 | 128 | - 命令名`coat`的意思是`COlorful cAT`；同时单词`coat`是外套，而彩色的输出行就像件漂亮的外套～ 🌈 😆 129 | - 命令名`taoc`是`coat`倒序拼写；命名方式就像`tac`之于`cat`。 🐈 130 | 131 | ### 用法/示例 132 | 133 | ```bash 134 | $ echo Hello world | coat 135 | Hello world 136 | $ echo -e 'Hello\nWorld' | coat 137 | Hello 138 | World 139 | $ echo -e 'Hello\nWorld' | taoc 140 | World 141 | Hello 142 | $ echo -e 'Hello\nWorld' | nl | coat 143 | 1 Hello 144 | 2 World 145 | $ coat file1 file2.txt 146 | line1 of file1 147 | line2 of file1 148 | ... 149 | line1 of file2 150 | line2 of file2 151 | ... 152 | 153 | # 帮助信息 154 | # 可以看到本人机器上实现代理的`cat`/`tac`命令是GNU的实现。 155 | $ coat --help 156 | Usage: coat [OPTION]... [FILE]... 157 | cat lines colorfully. 158 | 159 | Support options: 160 | --help display this help and exit 161 | --version output version information and exit 162 | All other options and arguments are delegated to command cat, 163 | more info see the help/man of command cat(e.g. cat --help). 164 | cat executable: /usr/local/opt/coreutils/libexec/gnubin/cat 165 | 166 | $ taoc --help 167 | Usage: taoc [OPTION]... [FILE]... 168 | tac lines colorfully. 169 | 170 | Support options: 171 | --help display this help and exit 172 | --version output version information and exit 173 | All other options and arguments are delegated to command tac, 174 | more info see the help/man of command tac(e.g. tac --help). 175 | tac executable: /usr/local/opt/coreutils/libexec/gnubin/tac 176 | ``` 177 | 178 | 注：上面示例中，没有彩色；在控制台上运行可以看出彩色效果，如下： 179 | ![coat screenshot](../docs/coat.png) 180 | 181 | 🍺 [a2l](../bin/a2l) 182 | ---------------------- 183 | 184 | 按行彩色输出参数，方便人眼查看。 185 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 186 | 187 | 命令名`a2l`的意思是`Arguments to(2) Lines`。 188 | 189 | ### 用法/示例 190 | 191 | ```bash 192 | $ a2l *.java 193 | A.java 194 | B.java 195 | ... 196 | 197 | # zsh支持 **/* 跨目录glob，可以方便搜索，但是输出内容是空格分隔的不方便查看。 198 | # 把参数按行输出方便查看 或是 grep 199 | $ a2l **/*.sh 200 | lib/console-text-color-themes.sh 201 | test/parseOpts_test.sh 202 | test/self-installer.sh 203 | ... 204 | ``` 205 | 206 | 注：上面示例中，没有彩色；在控制台上运行可以看出彩色效果，和上面的`coat`命令一样。 207 | 208 | 🍺 [uq](../bin/uq) 209 | ---------------------- 210 | 211 | 不重排序输入完成整个输入行的去重。相比系统的`uniq`命令加强的是可以跨行去重，不需要排序输入。 212 | 使用方式与支持的选项 模仿系统的`uniq`命令。支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 213 | 214 | > ‼️ **_注意_**： 去重过程会在内存持有整个输入（因为全局去重）！ 215 | > 216 | > 对于输入大小较大的场景（如输入量有几G），需谨慎使用以避免占用过多内存；往往需要结合业务场景开发对应的优化实现。 217 | > 虽然平时的大部分场景输入量非常有限（如几M），一个简单没有充分优化的实现是快速够用的。 218 | > 219 | > `uq`处理的最大输入量缺省是 256m（字符数），超过了最大输入量则出错退出，以避免意外消耗了过大的内存； 220 | > 可以通过`-XM, --max-input`选项 设置 消耗更多内存可接受的合理最大输入量，如`uq --max-input 1g ...` 221 | 222 | 因为系统的`uniq`命令去重相邻的行，需要组合`sort`命令以对整个输入去重，并且有下面的问题： 223 | 224 | ```bash 225 | # 示例输入 226 | $ cat foo.txt 227 | c 228 | c 229 | b 230 | a 231 | a 232 | c 233 | c 234 | 235 | $ uniq foo.txt 236 | c 237 | b 238 | a 239 | c 240 | # c输出了2次，原因是第二个c与第一个c不是相邻的重复行 241 | 242 | # 可以通过 sort -u 来完成整个输入去重，但这样操作，顺序与输入行不一致 243 | $ sort -u foo.txt 244 | a 245 | b 246 | c 247 | # 输入行重排序了！ 248 | 249 | # 另外一个经典的用法 sort 与 uniq -c，输出重复次数 250 | $ sort foo.txt | uniq -c 251 | 2 a 252 | 1 b 253 | 4 c 254 | # 输入行重排序了！ 255 | ``` 256 | 257 | ### 用法/示例 258 | 259 | ```bash 260 | $ uq foo.txt # 输入是文件 261 | $ cat foo.txt | uq # 或是 标准输入/管道 262 | c 263 | b 264 | a 265 | # 对整个输入行去重，且顺序与输入行一致（保留第一次出现的位置） 266 | 267 | # -c 选项：输出重复次数 268 | $ uq -c foo.txt 269 | 4 c 270 | 1 b 271 | 2 a 272 | 273 | # -d, --repeated 选项：只输出 重复行 274 | $ uq -d foo.txt 275 | c 276 | a 277 | # -u, --unique 选项：只输出 唯一行（即不重复的行） 278 | $ uq -u foo.txt 279 | b 280 | 281 | # -D 选项：重复行都输出，即重复了几次就输出几次 282 | $ uq -D -c foo.txt 283 | 4 c 284 | 4 c 285 | 1 b 286 | 2 a 287 | 2 a 288 | 4 c 289 | 4 c 290 | 291 | # 有多个文件参数时，最后一个参数 是 输出文件 292 | $ uq in1.txt in2.txt out.txt 293 | # 当有多个输入文件时，但要输出到控制台时，指定输出文件（最后一个文件参数）为 `-` 即可 294 | $ uq in1.txt in2.txt - 295 | 296 | # 如果消耗更多内存可接受的合理的，可以通过 -XM, --max-input 选项设置更大的最大输入量（缺省是256m） 297 | $ uq -MI 768m large-file-input 298 | $ uq --max-input 10g huge-file-input 299 | 300 | # 帮助信息 301 | $ uq -h 302 | Usage: uq [OPTION]... [INPUT [OUTPUT]] 303 | Filter lines from INPUT (or standard input), writing to OUTPUT (or standard output). 304 | Same as `uniq` command in core utils, 305 | but detect repeated lines that are not adjacent, no sorting required. 306 | 307 | Example: 308 | # only one file, output to stdout 309 | uq in.txt 310 | # more than 1 file, last file argument is output file 311 | uq in.txt out.txt 312 | # when use - as output file, output to stdout 313 | uq in1.txt in2.txt - 314 | 315 | Options: 316 | -c, --count prefix lines by the number of occurrences 317 | -d, --repeated only print duplicate lines, one for each group 318 | -D print all duplicate lines 319 | combined with -c/-d option usually 320 | --all-repeated[=METHOD] like -D, but allow separating groups 321 | with an empty line; 322 | METHOD={none(default),prepend,separate} 323 | -u, --unique Only output unique lines 324 | that are not repeated in the input 325 | -i, --ignore-case ignore differences in case when comparing 326 | -z, --zero-terminated line delimiter is NUL, not newline 327 | -XM, --max-input max input size(count by char), support k,m,g postfix 328 | default is 256m 329 | avoid consuming large memory unexpectedly 330 | -h, --help display this help and exit 331 | -V, --version display version information and exit 332 | ``` 333 | 334 | 🍺 [ap](../bin/ap) and [rp](../bin/rp) 335 | ---------------------- 336 | 337 | 批量转换文件路径为绝对路径/相对路径，会自动跟踪链接并规范化路径。 338 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 339 | 340 | 命令名`ap`的意思是`Absolute Path`，`rp`是`Relative Path`。 341 | 342 | ### 用法/示例 343 | 344 | ```bash 345 | # ap缺省打印当前路径的绝对路径 346 | $ ap 347 | /home/admin/useful-scripts/test 348 | $ ap .. 349 | /home/admin/useful-scripts 350 | # 支持多个参数 351 | $ ap .. ../.. /etc /etc/../etc 352 | /home/admin/useful-scripts 353 | /home/admin 354 | /etc 355 | /etc 356 | 357 | # rp当一个参数时，打印相对于当前路径的相对路径 358 | $ rp /home 359 | ../.. 360 | # 多于一个参数时，打印相对于最后一个参数的相对路径 361 | $ rp /home /etc/../etc /home/admin 362 | .. 363 | ../../etc 364 | ``` 365 | 366 | 🍺 [cp-into-docker-run](../bin/cp-into-docker-run) 367 | ---------------------- 368 | 369 | 一个`Docker`使用的便利脚本。拷贝本机的执行文件到指定的`docker container`中并在`docker container`中执行。 370 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 371 | 372 | ### 用法/示例 373 | 374 | ```bash 375 | # 通过 -c 选项 指定 docker container 376 | $ cp-into-docker-run -c container_foo /path/to/command command_args... 377 | # 如果 指定的command 不是一个路径，会从 PATH 中查找 378 | $ cp-into-docker-run -c container_foo a2l command_arg1 command_arg2 379 | 380 | # 帮助信息 381 | $ cp-into-docker-run -h 382 | Usage: cp-into-docker-run [OPTION]... command [command-args]... 383 | 384 | Copy the command into docker container 385 | and run the command in container. 386 | 387 | Example: 388 | cp-into-docker-run -c container_foo command_copied_into_container command_arg1 389 | 390 | docker options: 391 | -c, --container destination docker container 392 | -u, --docker-user docker username or UID to run command 393 | optional, docker default is (maybe) root user 394 | -w, --workdir absolute working directory inside the container 395 | optional, docker default is (maybe) root dir 396 | -t, --tmpdir tmp dir in docker to copy command 397 | optional, default is /tmp 398 | -p, --cp-path destination path in docker of the command(including file name) 399 | if specified, command will be kept when run finished 400 | optional, default is under tmp dir and deleted when run finished 401 | 402 | run options: 403 | -v, --verbose show operation step infos 404 | 405 | miscellaneous: 406 | -h, --help display this help and exit 407 | -V, --version display version information and exit 408 | ``` 409 | 410 | 411 | 412 | 413 | 🍺 [tcp-connection-state-counter](../bin/tcp-connection-state-counter) 414 | ---------------------- 415 | 416 | 统计各个`TCP`连接状态的个数。 417 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 418 | 419 | 像`Nginx`、`Apache`的机器上需要查看，`TCP`连接的个数，以判定 420 | 421 | - 连接数、负荷 422 | - 是否有攻击，查看`SYN_RECV`数（`SYN`攻击） 423 | - `TIME_WAIT`数，太多会导致`TCP: time wait bucket table overflow`。 424 | 425 | ### 用法/示例 426 | 427 | ```bash 428 | $ tcp-connection-state-counter 429 | CLOSE_WAIT 584 430 | ESTABLISHED 493 431 | TIME_WAIT 112 432 | LISTEN 27 433 | SYN_SENT 7 434 | ``` 435 | 436 | ### 贡献者 437 | 438 | [sunuslee](https://github.com/sunuslee) 改进此脚本，增加对`MacOS`的支持。 [#56](https://github.com/oldratlee/useful-scripts/pull/56) 439 | 440 | 🍺 [xpl](../bin/xpl) and [xpf](../bin/xpf) 441 | ---------------------- 442 | 443 | 在命令行中快速完成 在文件浏览器中 打开/选中 指定的文件或文件夹的操作，优化命令行与其它应用之间的操作流。 444 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 445 | 446 | - `xpl`：在文件浏览器中打开指定的文件或文件夹。 447 | `xpl`是`explorer`的缩写。 448 | - `xpf`: 在文件浏览器中打开指定的文件或文件夹，并选中。 449 | `xpf`是`EXplorer and select File`的缩写。 450 | 451 | ### 用法/示例 452 | 453 | ```bash 454 | xpl 455 | # 缺省打开当前目录 456 | xpl <文件或是目录>... 457 | # 打开多个文件或目录 458 | 459 | xpf 460 | # 缺省打开当前目录 461 | xpf <文件或是目录>... 462 | # 打开多个文件或目录 463 | 464 | 465 | # 示例 466 | xpl /path/to/dir 467 | xpl /path/to/foo.txt 468 | xpl /path/to/dir1 /path/to/foo1.txt 469 | xpf /path/to/foo1.txt 470 | xpf /path/to/dir1 /path/to/foo1.txt 471 | ``` 472 | 473 | ### 贡献者 474 | 475 | - [Linhua Tan](https://github.com/toolchainX) 修复Linux的选定Bug。 476 | 477 | `Shell`开发/测试加强 478 | ==================================== 479 | 480 | 481 | 482 | 483 | 🍺 [echo-args](../bin/echo-args) 484 | ---------------------- 485 | 486 | 在编写脚本时，常常要确认输入参数是否是期望的：参数个数，参数值（可能包含有人眼不容易发现的空格问题）。 487 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 488 | 489 | 这个脚本输出脚本收到的参数。在控制台运行时，把参数值括起的括号显示成 **红色**，方便人眼查看。 490 | 491 | ### 用法/示例 492 | 493 | ```bash 494 | $ ./echo-args 1 " 2 foo " "3 3" 495 | 0/3: [./echo-args] 496 | 1/3: [1] 497 | 2/3: [ 2 foo ] 498 | 3/3: [3 3] 499 | ``` 500 | 501 | ### 使用方式 502 | 503 | 需要查看某个脚本（实际上也可以是其它的可执行程序）输出参数时，可以这么做： 504 | 505 | - 把要查看脚本重命名。 506 | - 建一个`echo-args`脚本的符号链接到要查看参数的脚本的位置，名字和查看脚本一样。 507 | 508 | 这样可以不改其它的程序，查看到输入参数的信息。 509 | 510 | 🍺 [console-text-color-themes.sh](../lib/console-text-color-themes.sh) 511 | ---------------------- 512 | 513 | 显示`Terminator`的全部文字彩色组合的效果及其打印方式。 514 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 515 | 516 | 脚本中，也给出了`colorEcho`和`colorEchoWithoutNewLine`函数更方便输出彩色文本 517 | 518 | ### 用法 519 | 520 | ```bash 521 | colorEcho <颜色样式> <要输出的文本>... 522 | colorEchoWithoutNewLine <颜色样式> <要输出的文本>... 523 | ``` 524 | 525 | ### 示例 526 | 527 | ```bash 528 | source console-text-color-themes.sh 529 | 530 | # 输出红色文本 531 | colorEcho "0;31;40" "Hello world!" 532 | # 输出黄色带下划线的文本 533 | colorEchoWithoutNewLine "4;33;40" "Hello world!" "Hello Hell!" 534 | ``` 535 | 536 | ### 运行效果 537 | 538 | ![console-text-color-themes.sh的运行效果图](console-colorful-text.png) 539 | 540 | ### 贡献者 541 | 542 | [姜太公](https://github.com/jzwlqx) 提供循环输出彩色组合的脚本。 543 | 544 | ### 参考资料 545 | 546 | - [utensil](https://github.com/utensil) 547 | 的[在Bash下输出彩色的文本](http://utensil.github.io/tech/2007/09/10/colorful-bash.html)，这是篇很有信息量很钻研的文章！ 548 | 549 | 🍺 [parseOpts.sh](../lib/parseOpts.sh) 550 | ---------------------- 551 | 552 | 命令行选项解析库，加强支持选项有多个值（即数组）。 553 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 554 | 555 | 自己写一个命令行选项解析函数，是因为[`bash`](https://manned.org/bash)的`builtin`命令[`getopts`](https://manned.org/man/getopts.1)和加强版本命令[`getopt`](https://manned.org/getopt)都不支持数组的值。 556 | 557 | 指定选项的多个值（即数组）的风格模仿[`find`](https://manned.org/find)命令的`-exec`选项： 558 | 559 | ```bash 560 | $ find . -name \*.txt -exec echo "find file: " {} \; 561 | find file: foo.txt 562 | find file: bar.txt 563 | ... 564 | ``` 565 | 566 | ### 用法 567 | 568 | `parseOpts`函数的第一个参数是要解析的选项说明，后面跟实际要解析的输入参数。 569 | 570 | 选项说明可以长选项和短选项，用逗号分隔，如`a,a-long`。不同选项的说明间用竖号分隔，如`a,a-long|b,b-long:`。 571 | 572 | 选项说明最后可以有选项类型说明： 573 | 574 | - `-`： 无参数的选项。既有选项则把值设置成`true`。这是 ***缺省*** 的类型。 575 | - `:`： 有参数的选项，值只有一个。 576 | - `+`： 有多个参数值的选项。值列表要以`;`表示结束。 577 | 注意，`;`是`Bash`的元字符（用于一行中多个命令分隔），所以加上转义写成`\;`（当然也可以按你的喜好写成`";"`或`';'`）。 578 | 579 | 实际要解析的输入参数往往是你的脚本参数，这样`parseOpts`函数调用一般是： 580 | 581 | ```bash 582 | parseOpts "a,a-long|b,b-long:|c,c-long+" "$@" 583 | # "$@" 即是回放你的脚本参数 584 | ``` 585 | 586 | 通过约定的全局变量来获取选项和参数： 587 | 588 | - 选项名为`a`，通过全局变量`_OPT_VALUE_a`来获取选项的值。 589 | - 选项名为`a-long`，通过全局变量`_OPT_VALUE_a_long`来获取选项的值。 590 | 即，把选项名的`-`转`_`，再加上前缀`_OPT_VALUE_`对应的全局变量来获得选项值。 591 | - 除了选项剩下的参数，通过全局变量`_OPT_ARGS`来获取。 592 | 593 | 按照惯例，输入参数中如果有`--`表示之后参数中不再有选项，即之后都是参数。 594 | 595 | ### 示例 596 | 597 | ```bash 598 | # 导入parseOpts.sh 599 | source /path/to/parseOpts.sh 600 | 601 | parseOpts "a,a-long|b,b-long:|c,c-long+" -a -b bv --c-long c.sh -p pv -q qv arg1 \; aa bb cc 602 | # 可以通过下面全局变量来获得解析的参数值： 603 | # _OPT_VALUE_a = true 604 | # _OPT_VALUE_a_long = true 605 | # _OPT_VALUE_b = bv 606 | # _OPT_VALUE_b_long = bv 607 | # _OPT_VALUE_c = (c.sh -p pv -q qv arg1) ，数组类型 608 | # _OPT_VALUE_c_long = (c.sh -p pv -q qv arg1) ，数组类型 609 | # _OPT_ARGS = (aa bb cc) ，数组类型 610 | ``` 611 | 612 | `--`的使用效果示例： 613 | 614 | ```bash 615 | # 导入parseOpts.sh 616 | source /path/to/parseOpts.sh 617 | 618 | parseOpts "a,a-long|b,b-long:|c,c-long+" -a -b bv -- --c-long c.sh -p pv -q qv arg1 \; aa bb cc 619 | # 可以通过下面全局变量来获得解析的参数值： 620 | # _OPT_VALUE_a = true 621 | # _OPT_VALUE_a_long = true 622 | # _OPT_VALUE_b = bv 623 | # _OPT_VALUE_b_long = bv 624 | # _OPT_VALUE_c 没有设置过 625 | # _OPT_VALUE_c_long 没有设置过 626 | # _OPT_ARGS = (--c-long c.sh -p pv -q qv arg1 ';' aa bb cc) ，数组类型 627 | ``` 628 | 629 | ### 兼容性 630 | 631 | 这个脚本比较复杂，测试过的环境有： 632 | 633 | 1. `bash --version` 634 | `GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)` 635 | `uname -a` 636 | `Linux foo-host 2.6.32-41-generic #94-Ubuntu SMP Fri Jul 6 18:00:34 UTC 2012 x86_64 GNU/Linux` 637 | 1. `bash --version` 638 | `GNU bash, version 3.2.53(1)-release (x86_64-apple-darwin14)` 639 | `uname -a` 640 | `Darwin foo-host 14.0.0 Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 x86_64 i386 MacBookPro10,1 Darwin` 641 | 1. `bash --version` 642 | `GNU bash, version 3.00.15(1)-release (i386-redhat-linux-gnu)` 643 | `uname -a` 644 | `Linux foo-host 2.6.9-103.ELxenU #1 SMP Wed Mar 14 16:31:15 CST 2012 i686 i686 i386 GNU/Linux` 645 | 646 | ### 贡献者 647 | 648 | - [Khotyn Huang](https://github.com/khotyn) 指出`bash` `3.0`下使用有问题，并提供`bash` `3.0`的测试机器。 649 | 650 | -------------------------------------------------------------------------------- /bin/show-busy-java-threads: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # @Function 3 | # Find out the highest cpu consumed threads of java processes, and print the stack of these threads. 4 | # 5 | # @Usage 6 | # $ ./show-busy-java-threads 7 | # 8 | # @online-doc https://github.com/oldratlee/useful-scripts/blob/dev-3.x/docs/java.md#-show-busy-java-threads 9 | # @author Jerry Lee (oldratlee at gmail dot com) 10 | # @author superhj1987 (superhj1987 at 126 dot com) 11 | 12 | readonly PROG=${0##*/} 13 | readonly PROG_VERSION='3.x-dev' 14 | # choosing between $0 and BASH_SOURCE 15 | # https://stackoverflow.com/a/35006505/922688 16 | # How can I get the source directory of a Bash script from within the script itself? 17 | # https://stackoverflow.com/questions/59895 18 | # Will $0 always include the path to the script? 19 | # https://unix.stackexchange.com/questions/119929 20 | readonly -a COMMAND_LINE=("${BASH_SOURCE[0]}" "$@") 21 | # CAUTION: env var $USER is not reliable! 22 | # $USER may be overwritten; if run command by `sudo -u`, may is not `root`. 23 | # more info see https://www.baeldung.com/linux/get-current-user 24 | # 25 | # DO NOT declare and assign var(as readonly) in ONE line! 26 | # more info see https://github.com/koalaman/shellcheck/wiki/SC2155 27 | WHOAMI=$(whoami) 28 | UNAME=$(uname) 29 | readonly WHOAMI UNAME 30 | 31 | ################################################################################ 32 | # util functions 33 | ################################################################################ 34 | 35 | # NOTE: $'foo' is the escape sequence syntax of bash 36 | readonly NL=$'\n' # new line 37 | 38 | colorPrint() { 39 | local color=$1 40 | shift 41 | 42 | # if stdout is a terminal, turn on color output. 43 | # '-t' check: is a terminal? 44 | # check isatty in bash https://stackoverflow.com/questions/10022323 45 | if [ -t 1 ]; then 46 | printf '\e[1;%sm%s\e[0m\n' "$color" "$*" 47 | else 48 | printf '%s\n' "$*" 49 | fi 50 | } 51 | 52 | __appendToFile() { 53 | [[ -n "$append_file" && -w "$append_file" ]] && printf '%s\n' "$*" >>"$append_file" 54 | [[ -n "$store_dir" && -w "$store_dir" ]] && printf '%s\n' "$*" >>"$store_file_prefix$PROG" 55 | } 56 | 57 | colorOutput() { 58 | local color=$1 59 | shift 60 | 61 | colorPrint "$color" "$*" 62 | __appendToFile "$*" 63 | } 64 | 65 | # shellcheck disable=SC2120 66 | normalOutput() { 67 | printf '%s\n' "$*" 68 | __appendToFile "$*" 69 | } 70 | 71 | redOutput() { 72 | colorOutput 31 "$*" 73 | } 74 | 75 | greenOutput() { 76 | colorOutput 32 "$*" 77 | } 78 | 79 | yellowOutput() { 80 | colorOutput 33 "$*" 81 | } 82 | 83 | blueOutput() { 84 | colorOutput 36 "$*" 85 | } 86 | 87 | die() { 88 | local prompt_help=false exit_status=2 89 | while (($# > 0)); do 90 | case "$1" in 91 | -h) 92 | prompt_help=true 93 | shift 94 | ;; 95 | -s) 96 | exit_status=$2 97 | shift 2 98 | ;; 99 | *) 100 | break 101 | ;; 102 | esac 103 | done 104 | 105 | (($# > 0)) && colorPrint "1;31" "$PROG: $*" 106 | $prompt_help && echo "Try '$PROG --help' for more information." 107 | 108 | exit "$exit_status" 109 | } >&2 110 | 111 | logAndRun() { 112 | printf '%s\n' "$*" 113 | echo 114 | "$@" 115 | } 116 | 117 | logAndCat() { 118 | printf '%s\n' "$*" 119 | echo 120 | cat 121 | } 122 | 123 | # Bash RegEx to check floating point numbers from user input 124 | # https://stackoverflow.com/questions/13790763 125 | isNonNegativeFloatNumber() { 126 | [[ "$1" =~ ^[+]?[0-9]+\.?[0-9]*$ ]] 127 | } 128 | 129 | isNaturalNumber() { 130 | [[ "$1" =~ ^[+]?[0-9]+$ ]] 131 | } 132 | 133 | isNaturalNumberList() { 134 | [[ "$1" =~ ^([0-9]+)(,[0-9]+)*$ ]] 135 | } 136 | 137 | # print calling(quoted) command line which is able to copy and paste to rerun safely 138 | # 139 | # How to get the complete calling command of a BASH script from inside the script (not just the arguments) 140 | # https://stackoverflow.com/questions/36625593 141 | printCallingCommandLine() { 142 | local arg isFirst=true 143 | for arg in "${COMMAND_LINE[@]}"; do 144 | if $isFirst; then 145 | isFirst=false 146 | else 147 | printf ' ' 148 | fi 149 | printf '%q' "$arg" 150 | done 151 | echo 152 | } 153 | 154 | usage() { 155 | cat < find out the highest cpu consumed threads from 167 | the specified java process. 168 | support pid list(eg: 42,47). 169 | default from all java process. 170 | -c, --count set the thread count to show, default is 5. 171 | set count 0 to show all threads. 172 | -a, --append-file specifies the file to append output as log. 173 | -S, --store-dir specifies the directory for storing 174 | the intermediate files, and keep files. 175 | default store intermediate files at tmp dir, 176 | and auto remove after run. use this option to keep 177 | files so as to review jstack/top/ps output later. 178 | delay the delay between updates in seconds. 179 | count the number of updates. 180 | delay/count arguments imitates the style of 181 | vmstat command. 182 | 183 | jstack control: 184 | -s, --jstack-path specifies the path of jstack command. 185 | -F, --force set jstack to force a thread dump. 186 | use when jstack does not respond (process is hung). 187 | -m, --mix-native-frames set jstack to print both java and 188 | native frames (mixed mode). 189 | -l, --lock-info set jstack with long listing. 190 | prints additional information about locks. 191 | 192 | CPU usage calculation control: 193 | -i, --cpu-sample-interval specifies the delay between cpu samples to get 194 | thread cpu usage percentage during this interval. 195 | default is 0.5 (second). 196 | set interval 0 to get the percentage of time spent 197 | running during the *entire lifetime* of a process. 198 | 199 | Miscellaneous: 200 | -h, --help display this help and exit. 201 | -V, --version display version information and exit. 202 | EOF 203 | 204 | exit 205 | } 206 | 207 | progVersion() { 208 | printf '%s\n' "$PROG $PROG_VERSION" 209 | exit 210 | } 211 | 212 | ################################################################################ 213 | # check os support 214 | ################################################################################ 215 | 216 | [[ $UNAME = Linux* ]] || die "only support Linux, not support $UNAME yet!" 217 | 218 | ################################################################################ 219 | # parse options 220 | ################################################################################ 221 | 222 | # DO NOT declare and assign var ARGS(as readonly) in ONE line! 223 | ARGS=$( 224 | getopt -n "$PROG" -a -o c:p:a:s:S:i:Pd:FmlhV \ 225 | -l count:,pid:,append-file:,jstack-path:,store-dir:,cpu-sample-interval:,use-ps,top-delay:,force,mix-native-frames,lock-info,help,version \ 226 | -- "$@" 227 | ) || die -h 228 | eval set -- "$ARGS" 229 | unset ARGS 230 | 231 | count=5 232 | cpu_sample_interval=0.5 233 | 234 | while true; do 235 | case "$1" in 236 | -c | --count) 237 | count=$2 238 | shift 2 239 | ;; 240 | -p | --pid) 241 | pid_list=$2 242 | shift 2 243 | ;; 244 | -a | --append-file) 245 | append_file=$2 246 | shift 2 247 | ;; 248 | -s | --jstack-path) 249 | jstack_path=$2 250 | shift 2 251 | ;; 252 | -S | --store-dir) 253 | store_dir=$2 254 | shift 2 255 | ;; 256 | # support the legacy option name -P,--use-ps for compatibility 257 | -P | --use-ps) 258 | cpu_sample_interval=0 259 | shift 260 | ;; 261 | # support the legacy option name -d,--top-delay for compatibility 262 | -i | --cpu-sample-interval | -d | --top-delay) 263 | cpu_sample_interval=$2 264 | shift 2 265 | ;; 266 | -F | --force) 267 | force=-F 268 | shift 269 | ;; 270 | -m | --mix-native-frames) 271 | mix_native_frames=-m 272 | shift 273 | ;; 274 | -l | --lock-info) 275 | lock_info=-l 276 | shift 277 | ;; 278 | -h | --help) 279 | usage 280 | ;; 281 | -V | --version) 282 | progVersion 283 | ;; 284 | --) 285 | shift 286 | break 287 | ;; 288 | esac 289 | done 290 | 291 | readonly count cpu_sample_interval force mix_native_frames lock_info 292 | readonly update_delay=${1:-0} 293 | isNonNegativeFloatNumber "$update_delay" || die "update delay($update_delay) is not a non-negative float number!" 294 | 295 | [ -z "$1" ] && update_count=1 || update_count=${2:-0} 296 | isNaturalNumber "$update_count" || die "update count($update_count) is not a natural number!" 297 | readonly update_count 298 | 299 | if [ -n "$pid_list" ]; then 300 | pid_list=${pid_list//[[:space:]]/} # delete white space 301 | isNaturalNumberList "$pid_list" || die "pid(s)($pid_list) is illegal! example: 42 or 42,99,67" 302 | fi 303 | readonly pid_list 304 | 305 | # check the directory of append-file(-a) mode, create if not existed. 306 | if [ -n "$append_file" ]; then 307 | if [ -e "$append_file" ]; then 308 | [ -f "$append_file" ] || die "$append_file(specified by option -a, for storing run output files) exists but is not a file!" 309 | [ -w "$append_file" ] || die "file $append_file(specified by option -a, for storing run output files) exists but is not writable!" 310 | else 311 | append_file_dir=$(dirname -- "$append_file") 312 | if [ -e "$append_file_dir" ]; then 313 | [ -d "$append_file_dir" ] || die "directory $append_file_dir(specified by option -a, for storing run output files) exists but is not a directory!" 314 | [ -w "$append_file_dir" ] || die "directory $append_file_dir(specified by option -a, for storing run output files) exists but is not writable!" 315 | else 316 | mkdir -p "$append_file_dir" || die "fail to create directory $append_file_dir(specified by option -a, for storing run output files)!" 317 | fi 318 | fi 319 | fi 320 | readonly append_file 321 | 322 | # check store directory(-S) mode, create directory if not existed. 323 | if [ -n "$store_dir" ]; then 324 | if [ -e "$store_dir" ]; then 325 | [ -d "$store_dir" ] || die "$store_dir(specified by option -S, for storing output files) exists but is not a directory!" 326 | [ -w "$store_dir" ] || die "directory $store_dir(specified by option -S, for storing output files) exists but is not writable!" 327 | else 328 | mkdir -p "$store_dir" || die "fail to create directory $store_dir(specified by option -S, for storing output files)!" 329 | fi 330 | fi 331 | readonly store_dir 332 | 333 | isNonNegativeFloatNumber "$cpu_sample_interval" || die "cpu sample interval($cpu_sample_interval) is not a non-negative float number!" 334 | 335 | ################################################################################ 336 | # search/check the existence of jstack command 337 | # 338 | # search order/priority: 339 | # 1. from -s option 340 | # 2. from under env var JAVA_HOME 341 | # 3. from under env var PATH 342 | ################################################################################ 343 | 344 | if [ -n "$jstack_path" ]; then 345 | # 1. check jstack_path set by -s option 346 | [ -f "$jstack_path" ] || die "$jstack_path (set by -s option) is NOT found!" 347 | [ -x "$jstack_path" ] || die "$jstack_path (set by -s option) is NOT executable!" 348 | elif [ -n "$JAVA_HOME" ]; then 349 | # 2. search jstack under JAVA_HOME 350 | if [ -f "$JAVA_HOME/bin/jstack" ]; then 351 | [ -x "$JAVA_HOME/bin/jstack" ] || die -h "found \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) is NOT executable!${NL}Use -s option set jstack path manually." 352 | jstack_path="$JAVA_HOME/bin/jstack" 353 | elif [ -f "$JAVA_HOME/../bin/jstack" ]; then 354 | [ -x "$JAVA_HOME/../bin/jstack" ] || die -h "found \$JAVA_HOME/../bin/jstack($JAVA_HOME/../bin/jstack) is NOT executable!${NL}Use -s option set jstack path manually." 355 | jstack_path="$JAVA_HOME/../bin/jstack" 356 | fi 357 | elif type -P jstack &>/dev/null; then 358 | # 3. search jstack under PATH 359 | jstack_path=$(type -P jstack) 360 | [ -x "$jstack_path" ] || die -h "found $jstack_path from PATH is NOT executable!${NL}Use -s option set jstack path manually." 361 | else 362 | die -h "jstack NOT found by JAVA_HOME(${JAVA_HOME:-not set}) setting and PATH!${NL}Use -s option set jstack path manually." 363 | fi 364 | readonly jstack_path 365 | 366 | ################################################################################ 367 | # biz logic 368 | ################################################################################ 369 | 370 | # DO NOT declare and assign var run_timestamp(as readonly) in ONE line! 371 | run_timestamp=$(date "+%Y-%m-%d_%H:%M:%S.%N") 372 | readonly run_timestamp 373 | readonly uuid="${PROG}_${run_timestamp}_${$}_${RANDOM}" 374 | 375 | readonly tmp_store_dir="/tmp/$uuid" 376 | if [ -n "$store_dir" ]; then 377 | readonly store_file_prefix="$store_dir/${run_timestamp}_" 378 | else 379 | readonly store_file_prefix="$tmp_store_dir/${run_timestamp}_" 380 | fi 381 | mkdir -p "$tmp_store_dir" 382 | 383 | cleanupWhenExit() { 384 | rm -rf "$tmp_store_dir" &>/dev/null 385 | } 386 | trap cleanupWhenExit EXIT 387 | 388 | headInfo() { 389 | local timestamp=$1 390 | colorPrint "0;34;42" ================================================================================ 391 | printf '%s\n' "$timestamp [$((update_round_num + 1))/$update_count]: $(printCallingCommandLine)" 392 | colorPrint "0;34;42" ================================================================================ 393 | echo 394 | } 395 | 396 | if [ -n "$pid_list" ]; then 397 | readonly ps_process_select_options="-p $pid_list" 398 | else 399 | readonly ps_process_select_options="-C java -C jsvc" 400 | fi 401 | 402 | __die_when_no_java_process_found() { 403 | if [ -n "$pid_list" ]; then 404 | die "process($pid_list) is not running, or not java process!" 405 | else 406 | die 'No java process found!' 407 | fi 408 | } 409 | 410 | # output field: pid, thread id(lwp), pcpu, user 411 | # order by pcpu(percentage of cpu usage) 412 | # 413 | # NOTE: 414 | # use ps command to find busy thread(cpu usage) 415 | # cpu usage of ps command is expressed as 416 | # the percentage of time spent running during the *entire lifetime* of a process, 417 | # this is not ideal in general. 418 | findBusyJavaThreadsByPs() { 419 | # 1. sort by %cpu by ps option `--sort -pcpu` 420 | # unfortunately, ps from `procps-ng 3.3.12`, `--sort` does not work properly with other options, 421 | # use 422 | # ps 423 | # combined 424 | # sort -k3,3nr 425 | # instead of 426 | # ps --sort -pcpu 427 | # 2. use wide output(unlimited width) by ps option `-ww` 428 | # avoid trunk user column to username_fo+ or $uid alike 429 | 430 | # shellcheck disable=SC2206 431 | local -a ps_cmd_line=(ps $ps_process_select_options -wwLo 'pid,lwp,pcpu,user' --no-headers) 432 | # DO NOT combine var ps_out declaration and assignment in ONE line! 433 | # more info see https://github.com/koalaman/shellcheck/wiki/SC2155 434 | local ps_out 435 | ps_out=$("${ps_cmd_line[@]}" | sort -k3,3nr) 436 | [ -n "$ps_out" ] || __die_when_no_java_process_found 437 | 438 | if [ -n "$store_dir" ]; then 439 | printf '%s\n' "$ps_out" | logAndCat "${ps_cmd_line[*]} | sort -k3,3nr" >"$store_file_prefix$((update_round_num + 1))_ps" 440 | fi 441 | 442 | if ((count > 0)); then 443 | printf '%s\n' "$ps_out" | head -n "$count" 444 | else 445 | printf '%s\n' "$ps_out" 446 | fi 447 | } 448 | 449 | # top with output field: thread id, %cpu 450 | __top_threadId_cpu() { 451 | # DO NOT combine var java_pid_list declaration and assignment in ONE line! 452 | local java_pid_list 453 | # shellcheck disable=SC2086 454 | java_pid_list=$(ps $ps_process_select_options -o pid --no-headers) 455 | [ -n "$java_pid_list" ] || __die_when_no_java_process_found 456 | # shellcheck disable=SC2086 457 | java_pid_list=$(echo $java_pid_list | tr ' ' ,) # join with , 458 | 459 | # 1. sort by %cpu by top option `-o %CPU` 460 | # unfortunately, top version 3.2 does not support -o option(supports from top version 3.3+), 461 | # use 462 | # HOME=$tmp_store_dir top -H -b -n 1 463 | # combined 464 | # sort 465 | # instead of 466 | # HOME=$tmp_store_dir top -H -b -n 1 -o %CPU 467 | # 2. change HOME env var when run top, 468 | # so as to prevent top command output format being change by .toprc user config file unexpectedly 469 | # 3. use option `-d 0.5`(update interval 0.5 second) and `-n 2`(update 2 times), 470 | # and use second time update data to get cpu percentage of thread in 0.5 second interval 471 | # 4. top v3.3, there is 1 black line between 2 update; 472 | # but top v3.2, there is 2 blank lines between 2 update! 473 | local -a top_cmd_line=(top -H -b -d "$cpu_sample_interval" -n 2 -p "$java_pid_list") 474 | # DO NOT combine var top_out declaration and assignment in ONE line! 475 | local top_out 476 | top_out=$(HOME=$tmp_store_dir "${top_cmd_line[@]}") 477 | if [ -n "$store_dir" ]; then 478 | printf '%s\n' "$top_out" | logAndCat "${top_cmd_line[@]}" >"$store_file_prefix$((update_round_num + 1))_top" 479 | fi 480 | 481 | # DO NOT combine var result_threads_top_info declaration and assignment in ONE line! 482 | local result_threads_top_info 483 | result_threads_top_info=$(printf '%s\n' "$top_out" | awk '{ 484 | # from text line to empty line, increase block index 485 | if (previousLine && !$0) blockIndex++ 486 | # only print 4th text block(blockIndex == 3), aka. process info of second top update 487 | if (blockIndex == 3 && $1 ~ /^[0-9]+$/) 488 | print $1, $9 # $1 is thread id field, $9 is %cpu field 489 | previousLine = $0 490 | }') 491 | [ -n "$result_threads_top_info" ] || __die_when_no_java_process_found 492 | 493 | printf '%s\n' "$result_threads_top_info" | sort -k2,2nr 494 | } 495 | 496 | __complete_pid_user_by_ps() { 497 | # ps output field: pid, thread id(lwp), user 498 | # shellcheck disable=SC2206 499 | local -a ps_cmd_line=(ps $ps_process_select_options -wwLo 'pid,lwp,user' --no-headers) 500 | # DO NOT combine var ps_out declaration and assignment in ONE line! 501 | local ps_out 502 | ps_out=$("${ps_cmd_line[@]}") 503 | if [ -n "$store_dir" ]; then 504 | printf '%s\n' "$ps_out" | logAndCat "${ps_cmd_line[@]}" >"$store_file_prefix$((update_round_num + 1))_ps" 505 | fi 506 | 507 | local idx=0 threadId pcpu output_fields 508 | while read -r threadId pcpu; do 509 | ((count <= 0 || idx < count)) || break 510 | 511 | # output field: pid, threadId, pcpu, user 512 | output_fields=$(printf '%s\n' "$ps_out" | awk -v "threadId=$threadId" -v "pcpu=$pcpu" '$2==threadId { 513 | print $1, threadId, pcpu, $3; exit 514 | }') 515 | if [ -n "$output_fields" ]; then 516 | ((idx++)) 517 | printf '%s\n' "$output_fields" 518 | fi 519 | done 520 | } 521 | 522 | # output format is same as function findBusyJavaThreadsByPs 523 | findBusyJavaThreadsByTop() { 524 | __top_threadId_cpu | __complete_pid_user_by_ps 525 | } 526 | 527 | printStackOfThreads() { 528 | local idx=0 pid threadId pcpu user threadId0x 529 | while read -r pid threadId pcpu user; do 530 | printf -v threadId0x '%#x' "$threadId" 531 | 532 | ((idx++ > 0)) && normalOutput 533 | local jstackFile="$store_file_prefix$((update_round_num + 1))_jstack_$pid" 534 | [ -f "$jstackFile" ] || { 535 | # shellcheck disable=SC2206 536 | local -a jstack_cmd_line=("$jstack_path" $force $mix_native_frames $lock_info $pid) 537 | if [ "$user" = "$WHOAMI" ]; then 538 | # run without sudo, when java process user is current user 539 | logAndRun "${jstack_cmd_line[@]}" >"$jstackFile" 540 | elif ((UID == 0)); then 541 | # if java process user is not current user, must run jstack with sudo 542 | logAndRun sudo -u "$user" "${jstack_cmd_line[@]}" >"$jstackFile" 543 | else 544 | # current user is not root user, so can not run with sudo; print error message and rerun suggestion 545 | redOutput "[$idx] Fail to jstack busy($pcpu%) thread($threadId/$threadId0x) stack of java process($pid) under user($user)." 546 | redOutput "User of java process($user) is not current user($WHOAMI), need sudo to rerun:" 547 | yellowOutput " sudo $(printCallingCommandLine)" 548 | continue 549 | fi || { 550 | redOutput "[$idx] Fail to jstack busy($pcpu%) thread($threadId/$threadId0x) stack of java process($pid) under user($user)." 551 | rm "$jstackFile" &>/dev/null 552 | continue 553 | } 554 | } 555 | 556 | blueOutput "[$idx] Busy($pcpu%) thread($threadId/$threadId0x) stack of java process($pid) under user($user):" 557 | 558 | if [ -n "$mix_native_frames" ]; then 559 | local sed_script="/--------------- $threadId ---------------/,/^---------------/ { 560 | /--------------- $threadId ---------------/b # skip first separator line 561 | /^---------------/d # delete second separator line 562 | p 563 | }" 564 | elif [ -n "$force" ]; then 565 | local sed_script="/^Thread $threadId:/,/^$/ { 566 | /^$/d; p # delete end separator line 567 | }" 568 | else 569 | local sed_script="/ nid=($threadId0x|$threadId) /,/^$/ { 570 | /^$/d; p # delete end separator line 571 | }" 572 | fi 573 | sed "$sed_script" -n -r "$jstackFile" | tee ${append_file:+-a "$append_file"} ${store_dir:+-a "$store_file_prefix$PROG"} 574 | done 575 | } 576 | 577 | main() { 578 | local update_round_num timestamp 579 | # if update_count <= 0, infinite loop till user interrupted (eg: CTRL+C) 580 | for ((update_round_num = 0; update_count <= 0 || update_round_num < update_count; ++update_round_num)); do 581 | ((update_round_num > 0)) && { 582 | sleep "$update_delay" 583 | normalOutput 584 | } 585 | 586 | timestamp=$(date "+%Y-%m-%d %H:%M:%S.%N") 587 | [[ -n "$append_file" || -n "$store_dir" ]] && headInfo "$timestamp" | 588 | tee ${append_file:+-a "$append_file"} ${store_dir:+-a "$store_file_prefix$PROG"} >/dev/null 589 | ((update_count != 1)) && headInfo "$timestamp" 590 | 591 | if [ "$cpu_sample_interval" = 0 ]; then 592 | findBusyJavaThreadsByPs 593 | else 594 | findBusyJavaThreadsByTop 595 | fi | printStackOfThreads 596 | done 597 | } 598 | 599 | main 600 | -------------------------------------------------------------------------------- /docs/java.md: -------------------------------------------------------------------------------- 1 | 🐌 `Java`相关脚本 2 | ==================================== 3 | 4 | 5 | 6 | 7 | - [🍺 show-busy-java-threads](#-show-busy-java-threads) 8 | - [用法](#%E7%94%A8%E6%B3%95) 9 | - [示例](#%E7%A4%BA%E4%BE%8B) 10 | - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85) 11 | - [🍺 show-duplicate-java-classes](#-show-duplicate-java-classes) 12 | - [用法](#%E7%94%A8%E6%B3%95-1) 13 | - [`JDK`开发场景使用说明](#jdk%E5%BC%80%E5%8F%91%E5%9C%BA%E6%99%AF%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E) 14 | - [对于一般的工程](#%E5%AF%B9%E4%BA%8E%E4%B8%80%E8%88%AC%E7%9A%84%E5%B7%A5%E7%A8%8B) 15 | - [对于`Web`工程](#%E5%AF%B9%E4%BA%8Eweb%E5%B7%A5%E7%A8%8B) 16 | - [`Android`开发场景使用说明](#android%E5%BC%80%E5%8F%91%E5%9C%BA%E6%99%AF%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E) 17 | - [示例](#%E7%A4%BA%E4%BE%8B-1) 18 | - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-1) 19 | - [🍺 find-in-jars](#-find-in-jars) 20 | - [用法](#%E7%94%A8%E6%B3%95-2) 21 | - [示例](#%E7%A4%BA%E4%BE%8B-2) 22 | - [运行效果](#%E8%BF%90%E8%A1%8C%E6%95%88%E6%9E%9C) 23 | - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99) 24 | 25 | 26 | 27 | ------------------------------- 28 | 29 | 关于`Java`排错与诊断，力荐️`Arthas`： ❤️ 30 | 31 | - `Arthas`用户文档： https://arthas.aliyun.com/doc/quick-start.html 32 | - GitHub Repo： [alibaba/arthas: Alibaba Java诊断利器](https://github.com/alibaba/arthas) 33 | 34 | `Arthas`功能异常(😜)强劲，且在阿里巴巴线上支持使用多年。我自己也常用，一定要看看用用！ 35 | 36 | `Arthas`是通过`Agent`方式来连接运行的`Java`进程、主要通过交互式来完成功能，与之对应的脚本方式也有其优势，如： 37 | 38 | 1. 可以在进程不能启动的情况下完成诊断（如依赖中的重复类分析、`ClassPath`上的资源或类查找） 39 | 1. 开销少；简单少依赖（就纯文本的一个脚本文件） 40 | 1. 方便与（已有的）工具（如`awk`、`sed`、`cron`）、流程或设施集成，进一步编程/自动化 41 | 42 | 请按需按场景选用。 43 | 44 | ------------------------------- 45 | 46 | 47 | 48 | 49 | 🍺 [show-busy-java-threads](../bin/show-busy-java-threads) 50 | ---------------------- 51 | 52 | 用于快速排查`Java`的`CPU`性能问题(`top us`值过高)，自动查出运行的`Java`进程中消耗`CPU`多的线程，并打印出其线程栈，从而确定导致性能问题的方法调用。 53 | 目前只支持`Linux`。原因是`Mac`、`Windows`的`ps`命令不支持列出进程的线程`id`，更多信息参见 [#33](https://github.com/oldratlee/useful-scripts/issues/33)，欢迎提供解法。 54 | 55 | PS，如何操作可以参见[`@bluedavy`](http://weibo.com/bluedavy)的[《分布式Java应用》](https://book.douban.com/subject/4848587/)的【5.1.1 `CPU`消耗分析】一节，说得很详细： 56 | 57 | 1. `top`命令找出消耗`CPU`高的`Java`进程及其线程`id`： 58 | 1. 开启线程显示模式（`top -H`，或是打开`top`后按`H`） 59 | 1. 按`CPU`使用率排序（`top`缺省是按`CPU`使用降序，已经合要求；打开`top`后按`P`可以显式指定按`CPU`使用降序） 60 | 1. 记下`Java`进程`id`及其`CPU`高的线程`id` 61 | 1. 查看消耗`CPU`高的线程栈： 62 | 1. 用进程`id`作为参数，`jstack`出有问题的`Java`进程 63 | 1. 手动转换线程`id`成十六进制（可以用`printf %x 1234`） 64 | 1. 在`jstack`输出中查找十六进制的线程`id`（可以用`vim`的查找功能`/0x1234`，或是`grep 0x1234 -A 20`） 65 | 1. 查看对应的线程栈，分析问题 66 | 67 | 查问题时，会要多次上面的操作以分析确定问题，这个过程**太繁琐太慢了**。 68 | 期望整合上面的过程成一个脚本，这样一行命令就可以自动化地搞定。 69 | 70 | ### 用法 71 | 72 | ```bash 73 | show-busy-java-threads 74 | # 从所有运行的Java进程中找出最消耗CPU的线程（缺省5个），打印出其线程栈 75 | 76 | # 缺省会自动从所有的Java进程中找出最消耗CPU的线程，这样用更方便 77 | # 当然你可以通过 -p 选项 手动指定要分析的Java进程Id，以保证只会显示你关心的那个Java进程的信息 78 | show-busy-java-threads -p <指定的Java进程Id> 79 | show-busy-java-threads -p 42 80 | show-busy-java-threads -p 42,47 81 | 82 | show-busy-java-threads -c <要展示示的线程栈个数> 83 | 84 | show-busy-java-threads <重复执行的间隔秒数> [<重复执行的次数>] 85 | # 多次执行；这2个参数的使用方式类似vmstat命令 86 | 87 | show-busy-java-threads -a <运行输出的记录到的文件> 88 | # 记录到文件以方便回溯查看 89 | 90 | show-busy-java-threads -S <存储jstack输出文件的目录> 91 | # 指定jstack输出文件的存储目录，方便记录以后续分析 92 | 93 | ############################## 94 | # 注意： 95 | ############################## 96 | # 如果Java进程的用户 与 执行脚本的当前用户 不同，则jstack不了这个Java进程 97 | # 为了能切换到Java进程的用户，需要加sudo来执行，即可以解决： 98 | sudo show-busy-java-threads 99 | 100 | show-busy-java-threads -s <指定jstack命令的全路径> 101 | # 对于sudo方式的运行，JAVA_HOME环境变量不能传递给root， 102 | # 而root用户往往没有配置JAVA_HOME且不方便配置，不能找到jstack命令。 103 | # 这时显式指定jstack命令的路径就反而显得更方便了 104 | 105 | # -m 选项：执行jstack命令时加上 -m 选项，显示上Native的栈帧，一般应用排查不需要使用 106 | show-busy-java-threads -m 107 | # -F 选项：执行jstack命令时加上 -F 选项（如果直接jstack无响应时，用于强制jstack），一般情况不需要使用 108 | show-busy-java-threads -F 109 | # -l 选项：执行jstack命令时加上 -l 选项，显示上更多相关锁的信息，一般情况不需要使用 110 | # 注意：和 -m -F 选项一起使用时，可能会大大增加jstack操作的耗时 111 | show-busy-java-threads -l 112 | 113 | # 帮助信息 114 | $ show-busy-java-threads -h 115 | Usage: show-busy-java-threads [OPTION]... [delay [count]] 116 | Find out the highest cpu consumed threads of java processes, 117 | and print the stack of these threads. 118 | 119 | Example: 120 | show-busy-java-threads # show busy java threads info 121 | show-busy-java-threads 1 # update every 1 second, (stop by eg: CTRL+C) 122 | show-busy-java-threads 3 10 # update every 3 seconds, update 10 times 123 | 124 | Output control: 125 | -p, --pid find out the highest cpu consumed threads from 126 | the specified java process. 127 | support pid list(eg: 42,47). 128 | default from all java process. 129 | -c, --count set the thread count to show, default is 5. 130 | set count 0 to show all threads. 131 | -a, --append-file specifies the file to append output as log. 132 | -S, --store-dir specifies the directory for storing 133 | the intermediate files, and keep files. 134 | default store intermediate files at tmp dir, 135 | and auto remove after run. use this option to keep 136 | files so as to review jstack/top/ps output later. 137 | delay the delay between updates in seconds. 138 | count the number of updates. 139 | delay/count arguments imitates the style of 140 | vmstat command. 141 | 142 | jstack control: 143 | -s, --jstack-path specifies the path of jstack command. 144 | -F, --force set jstack to force a thread dump. 145 | use when jstack does not respond (process is hung). 146 | -m, --mix-native-frames set jstack to print both java and 147 | native frames (mixed mode). 148 | -l, --lock-info set jstack with long listing. 149 | prints additional information about locks. 150 | 151 | CPU usage calculation control: 152 | -i, --cpu-sample-interval specifies the delay between cpu samples to get 153 | thread cpu usage percentage during this interval. 154 | default is 0.5 (second). 155 | set interval 0 to get the percentage of time spent 156 | running during the *entire lifetime* of a process. 157 | 158 | Miscellaneous: 159 | -h, --help display this help and exit. 160 | -V, --version display version information and exit. 161 | ``` 162 | 163 | ### 示例 164 | 165 | ```bash 166 | $ show-busy-java-threads 167 | [1] Busy(57.0%) thread(23355/0x5b3b) stack of java process(23269) under user(admin): 168 | "pool-1-thread-1" prio=10 tid=0x000000005b5c5000 nid=0x5b3b runnable [0x000000004062c000] 169 | java.lang.Thread.State: RUNNABLE 170 | at java.text.DateFormat.format(DateFormat.java:316) 171 | at com.xxx.foo.services.common.DateFormatUtil.format(DateFormatUtil.java:41) 172 | at com.xxx.foo.shared.monitor.schedule.AppMonitorDataAvgScheduler.run(AppMonitorDataAvgScheduler.java:127) 173 | at com.xxx.foo.services.common.utils.AliTimer$2.run(AliTimer.java:128) 174 | at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 175 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 176 | at java.lang.Thread.run(Thread.java:662) 177 | 178 | [2] Busy(26.1%) thread(24018/0x5dd2) stack of java process(23269) under user(admin): 179 | "pool-1-thread-2" prio=10 tid=0x000000005a968800 nid=0x5dd2 runnable [0x00000000420e9000] 180 | java.lang.Thread.State: RUNNABLE 181 | at java.util.Arrays.copyOf(Arrays.java:2882) 182 | at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) 183 | at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572) 184 | at java.lang.StringBuffer.append(StringBuffer.java:320) 185 | - locked <0x00000007908d0030> (a java.lang.StringBuffer) 186 | at java.text.SimpleDateFormat.format(SimpleDateFormat.java:890) 187 | at java.text.SimpleDateFormat.format(SimpleDateFormat.java:869) 188 | at java.text.DateFormat.format(DateFormat.java:316) 189 | at com.xxx.foo.services.common.DateFormatUtil.format(DateFormatUtil.java:41) 190 | at com.xxx.foo.shared.monitor.schedule.AppMonitorDataAvgScheduler.run(AppMonitorDataAvgScheduler.java:126) 191 | at com.xxx.foo.services.common.utils.AliTimer$2.run(AliTimer.java:128) 192 | at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 193 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 194 | at java.lang.Thread.run(Thread.java:662) 195 | 196 | ...... 197 | ``` 198 | 199 | 上面的线程栈可以看出，`CPU`消耗最高的2个线程都在执行`java.text.DateFormat.format`，业务代码对应的方法是`shared.monitor.schedule.AppMonitorDataAvgScheduler.run`。可以基本确定： 200 | 201 | - `AppMonitorDataAvgScheduler.run`调用`DateFormat.format`次数比较频繁。 202 | - `DateFormat.format`比较慢。（这个可以由`DateFormat.format`的实现确定。） 203 | 204 | 多执行几次`show-busy-java-threads`，如果上面情况高概率出现，则可以确定上面的判定。 205 | 因为调用越少代码执行越快，则出现在线程栈的概率就越低。 206 | 脚本有自动多次执行的功能，指定 重复执行的间隔秒数/重复执行的次数 参数。 207 | 208 | 分析`shared.monitor.schedule.AppMonitorDataAvgScheduler.run`实现逻辑和调用方式，以优化实现解决问题。 209 | 210 | ### 贡献者 211 | 212 | - [silentforce](https://github.com/silentforce) 改进此脚本，增加对环境变量`JAVA_HOME`的判断。 [#15](https://github.com/oldratlee/useful-scripts/pull/15) 213 | - [liuyangc3](https://github.com/liuyangc3) 214 | - 发现并解决`jstack`非当前用户`Java`进程的问题。 [#50](https://github.com/oldratlee/useful-scripts/pull/50) 215 | - 优化性能，通过`read -a`简化反复的`awk`操作。 [#51](https://github.com/oldratlee/useful-scripts/pull/51) 216 | - [superhj1987](https://github.com/superhj1987) / [lirenzuo](https://github.com/lirenzuo) 217 | - 提出/实现了多次执行的功能 [superhj1987/awesome-scripts#1](https://github.com/superhj1987/awesome-scripts/issues/1) 218 | - [xiongchen2012](https://github.com/xiongchen2012) 提出并解决了长用户名截断的Bug [#62](https://github.com/oldratlee/useful-scripts/pull/62) 219 | - [qsLI](https://github.com/qsLI) / [sdslnmd](https://github.com/sdslnmd) 220 | - 发现并提交Issue：show-busy-java-threads支持top来获取cpu占用率，ps的cpu占用率非实时 [#67](https://github.com/oldratlee/useful-scripts/issues/67) 221 | - [geekMessi](https://github.com/geekMessi) 222 | - 发现并提交Issue：在`top v3.2`下提取不正确的Bug [#71](https://github.com/oldratlee/useful-scripts/issues/71) 223 | - 发现并提交Issue：support command name jsvc to find java process [#72](https://github.com/oldratlee/useful-scripts/issues/72) 224 | 225 | 🍺 [show-duplicate-java-classes](../bin/show-duplicate-java-classes) 226 | ---------------------- 227 | 228 | 找出`Java Lib`（`Java`库，即`Jar`文件）或`Class`目录（类目录）中的重复类。 229 | 全系统支持（`Python 3`实现，安装`Python 3`即可），如`Linux`、`Mac`、`Windows`。 230 | 231 | `Java`开发的一个麻烦的问题是`Jar`冲突（即多个版本的`Jar`），或者说重复类。会出`NoSuchMethod`等的问题，还不见得当时出问题。找出有重复类的`Jar`，可以防患未然。 232 | 233 | ### 用法 234 | 235 | - 通过脚本参数 指定 `Libs`目录，查找目录下`Jar`文件，收集`Jar`文件中`Class`文件以分析重复类。可以指定多个`Libs`目录。 236 | - 缺省只会查找指定`Lib`目录下`Jar`文件，不会收集`Lib`目录的子目录下`Jar`文件。 237 | - 因为`Libs`目录一般不会用子目录再放`Jar`，也避免把去查找不期望的`Jar`文件。 238 | - 可以通过 `-L`选项 设置 收集`Lib`子目录下的`Jar`文件；这样可以简化`Lib`目录的设置，不需要指定完整的`Lib`目录路径。 239 | - 对于找到的`Jar`文件，缺省不会进一步收集包含在`Jar`文件中的`Jar`。 240 | - 即`FatJar`/`UberJar`的场景，随着像`SpringBoot`的广泛使用，`FatJar`/`UberJar`也比较常见。 241 | - 可以通过 `-J`选项 设置 收集包含在`Jar`文件中的`Jar`。 242 | - 通过`-c`选项 指定 `Class`目录，直接收集这个目录下的`Class`文件以分析重复类。可以多次指定多个`Class`目录。 243 | 244 | ```bash 245 | # 查找当前目录下所有Jar中的重复类 246 | show-duplicate-java-classes 247 | 248 | # 查找多个指定目录下所有Jar中的重复类 249 | show-duplicate-java-classes path/to/lib_dir1 /path/to/lib_dir2 250 | # 通过 -L 选项，收集子目录中的Jar文件 251 | show-duplicate-java-classes -L path/to/lib_dir1 252 | # 通过 -J 选项，收集包含在Jar文件中的Jar文件（即 收集包含在FatJar/UberJar中的Jar） 253 | show-duplicate-java-classes -J path/to/lib_dir1 254 | 255 | # 查找多个指定Class目录下的重复类。 Class目录 通过 -c 选项指定 256 | show-duplicate-java-classes -c path/to/class_dir1 -c /path/to/class_dir2 257 | 258 | # 查找指定Class目录和指定目录下所有Jar中的重复类的Jar 259 | show-duplicate-java-classes path/to/lib_dir1 /path/to/lib_dir2 -c path/to/class_dir1 -c path/to/class_dir2 260 | 261 | # 帮助信息 262 | $ show-duplicate-java-classes -h 263 | Usage: show-duplicate-java-classes [OPTION]... [-c class-dir1 [-c class-dir2] ...] [lib-dir1|jar-file1 [lib-dir2|jar-file2] ...] 264 | Find duplicate classes among java lib dirs and class dirs. 265 | 266 | Examples: 267 | show-duplicate-java-classes # search jars from current dir 268 | show-duplicate-java-classes path/to/lib_dir1 /path/to/lib_dir2 269 | show-duplicate-java-classes -c path/to/class_dir1 -c /path/to/class_dir2 270 | show-duplicate-java-classes -c path/to/class_dir1 path/to/lib_dir1 271 | show-duplicate-java-classes -L path/to/lib_dir1 272 | show-duplicate-java-classes -J path/to/lib_dir1 273 | 274 | Options: 275 | --version show program's version number and exit 276 | -h, --help show this help message and exit 277 | -L, --recursive-lib search jars in the sub-directories of lib dir 278 | -J, --recursive-jar search jars in the jar file 279 | -c CLASS_DIRS, --class-dir=CLASS_DIRS 280 | add class dir 281 | -R, --no-find-progress 282 | do not display responsive find progress 283 | ``` 284 | 285 | #### `JDK`开发场景使用说明 286 | 287 | 以`Maven`作为构建工程示意过程。 288 | 289 | ##### 对于一般的工程 290 | 291 | ```sh 292 | # 在项目模块目录下执行，拷贝依赖Jar到目录target/dependency下 293 | $ mvn dependency:copy-dependencies -DincludeScope=runtime 294 | ... 295 | # 检查重复类 296 | $ show-duplicate-java-classes target/dependency 297 | ... 298 | ``` 299 | 300 | ##### 对于`Web`工程 301 | 302 | 对于`Web`工程，即`war` `maven`模块，会打包生成`war`文件。 303 | 304 | ```sh 305 | # 在war模块目录下执行，生成war文件 306 | $ mvn install 307 | ... 308 | # 解压war文件，war文件中包含了应用的依赖的Jar文件 309 | $ unzip target/*.war -d target/war 310 | ... 311 | # 检查重复类 312 | $ show-duplicate-java-classes -c target/war/WEB-INF/classes target/war/WEB-INF/lib 313 | ... 314 | ``` 315 | 316 | #### `Android`开发场景使用说明 317 | 318 | `Android`开发，有重复类在编译打包时会报`[Dex Loader] Unable to execute dex: Multiple dex files define Lorg/foo/xxx/Yyy`。 319 | 320 | 但只会给出一个重复类名，如果重复类比较多时，上面打包/报错/排查会要进行多次，而`Android`的打包比较费时，这个过程比较麻烦，希望可以一次把所有重复类都列出来，一起排查掉。 321 | 322 | 以`Gradle`作为构建工程示意过程。 323 | 324 | 在`App`的`build.gradle`中添加拷贝库到目录`build/dependencies`下。 325 | 326 | ```groovy 327 | task copyDependencies(type: Copy) { 328 | def dest = new File(buildDir, "dependencies") 329 | 330 | // clean dir 331 | dest.deleteDir() 332 | dest.mkdirs() 333 | 334 | // fill dir with dependencies 335 | from configurations.compile into dest 336 | } 337 | ``` 338 | 339 | ```sh 340 | # 拷贝依赖 341 | $ ./gradlew app:copyDependencies 342 | ... 343 | # 检查重复类 344 | $ show-duplicate-java-classes app/build/dependencies 345 | ... 346 | ``` 347 | 348 | ### 示例 349 | 350 | ```bash 351 | $ show-duplicate-java-classes WEB-INF/lib 352 | COOL! No duplicate classes found! 353 | 354 | ================================================================================ 355 | Find in 150 class paths: 356 | ================================================================================ 357 | 1: (contain 9 classes) WEB-INF/lib/aopalliance-1.0.jar 358 | 2: (contain 25 classes) WEB-INF/lib/asm-5.0.4.jar 359 | 3: (contain 313 classes) WEB-INF/lib/aviator-5.0.0.jar 360 | 4: (contain 687 classes) WEB-INF/lib/cassandra-0.6.1.jar 361 | ... 362 | 363 | $ show-duplicate-java-classes -c WEB-INF/classes WEB-INF/lib 364 | Found 1272 duplicate classes in 345 class paths and 9 class path sets: 365 | [1] found 188(100%) duplicate classes in 3 class paths: 366 | 1: (contain 188 classes) WEB-INF/lib/jdom-2.0.2.jar 367 | 2: (contain 195 classes) WEB-INF/lib/jdom2-2.0.6.jar 368 | 3: (contain 195 classes) WEB-INF/lib/jdom2-2.0.8.jar 369 | [2] found 150(33.8%) duplicate classes in 2 class paths: 370 | 1: (contain 1385 classes) WEB-INF/lib/netty-all-4.0.35.Final.jar 371 | 2: (contain 444 classes) WEB-INF/lib/netty-common-4.1.31.Final.jar 372 | [3] found 148(55.4%) duplicate classes in 2 class paths: 373 | 1: (contain 1385 classes) WEB-INF/lib/netty-all-4.0.35.Final.jar 374 | 2: (contain 267 classes) WEB-INF/lib/netty-handler-4.1.31.Final.jar 375 | [4] found 103(82.4%) duplicate classes in 2 class paths: 376 | 1: (contain 125 classes) WEB-INF/lib/hessian-3.0.14.bugfix.jar 377 | 2: (contain 275 classes) WEB-INF/lib/hessian-4.0.38.jar 378 | ... 379 | 380 | ================================================================================ 381 | Duplicate classes detail info: 382 | ================================================================================ 383 | [1] found 188 duplicate classes in 3 class paths WEB-INF/lib/jdom-2.0.2.jar WEB-INF/lib/jdom2-2.0.6.jar WEB-INF/lib/jdom2-2.0.8.jar : 384 | 1: org/jdom2/Attribute.class 385 | 2: org/jdom2/AttributeList$1.class 386 | 3: org/jdom2/AttributeList$ALIterator.class 387 | 4: org/jdom2/AttributeList.class 388 | 5: org/jdom2/AttributeType.class 389 | ... 390 | [2] found 150 duplicate classes in 2 class paths WEB-INF/lib/netty-all-4.0.35.Final.jar WEB-INF/lib/netty-common-4.1.31.Final.jar : 391 | 1: io/netty/util/AbstractReferenceCounted.class 392 | 2: io/netty/util/Attribute.class 393 | 3: io/netty/util/AttributeKey.class 394 | 4: io/netty/util/AttributeMap.class 395 | 5: io/netty/util/CharsetUtil.class 396 | ... 397 | ... 398 | 399 | ================================================================================ 400 | Find in 232 class paths: 401 | ================================================================================ 402 | 1: (contain 42 classes) WEB-INF/classes 403 | 2: (contain 70 classes) WEB-INF/lib/HikariCP-2.7.8.jar 404 | 3: (contain 13 classes) WEB-INF/lib/accessors-smart-1.2.jar 405 | 4: (contain 9 classes) WEB-INF/lib/aopalliance-1.0.jar 406 | 5: (contain 25 classes) WEB-INF/lib/asm-5.0.4.jar 407 | 6: (contain 313 classes) WEB-INF/lib/aviator-5.0.0.jar 408 | ... 409 | ``` 410 | 411 | ### 贡献者 412 | 413 | [tgic](https://github.com/tg123) 提供此脚本。友情贡献者的链接 [commandlinefu.cn](http://commandlinefu.cn/) | [微博linux命令行精选](http://weibo.com/u/2674868673) 414 | 415 | 416 | 417 | 418 | 🍺 [find-in-jars](../bin/find-in-jars) 419 | ---------------------- 420 | 421 | 在当前目录下所有`jar`文件里，查找类或资源文件。 422 | 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 423 | 424 | ### 用法 425 | 426 | ```bash 427 | # 在当前目录下所有`jar`文件里，查找类或资源文件。 428 | find-in-jars 'log4j\.properties' 429 | find-in-jars 'log4j\.xml$' 430 | find-in-jars log4j\\.xml$ # 和上面命令一样，Shell转义的不同写法而已 431 | find-in-jars 'log4j\.(properties|xml)$' 432 | 433 | # -d选项 指定 查找目录（覆盖缺省的当前目录） 434 | find-in-jars 'log4j\.properties$' -d /path/to/find/directory 435 | # 支持多个查找目录，多次指定这个选项即可 436 | find-in-jars 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 437 | 438 | # -e选项 指定 查找`zip`文件的扩展名，缺省是`jar` 439 | find-in-jars 'log4j\.properties' -e zip 440 | # 支持多种查找扩展名，多次指定这个选项即可 441 | find-in-jars 'log4j\.properties' -e jar -e zip 442 | 443 | # -a选项 指定 查找结果中的Jar文件使用绝对路径 444 | # 分享给别人时，Jar文件路径是完整的，方便别人找到文件 445 | find-in-jars 'log4j\.properties' -a 446 | 447 | # -s选项 指定 查找结果中的Jar文件和Jar文件里的查找Entry间分隔符，缺省是『!』 448 | # 方便你喜欢的人眼查看，或是与工具脚本如`awk`的处理 449 | find-in-jars 'log4j\.properties' -s ' <-> ' 450 | find-in-jars 'log4j\.properties' -s ' ' | awk '{print $2}' 451 | 452 | # -l选项 指定 只列出Jar文件，不显示Jar文件内匹配的文件列表 453 | # 列出 包含log4j.xml文件的Jar文件： 454 | find-in-jars -l 'log4j\.xml$' 455 | 456 | # 帮助信息 457 | $ find-in-jars -h 458 | Usage: find-in-jars [OPTION]... PATTERN 459 | 460 | Find files in the jar files under specified directory, 461 | search jar files recursively(include subdirectory). 462 | The pattern default is *extended* regex. 463 | 464 | Example: 465 | find-in-jars 'log4j\.properties' 466 | # search file log4j.properties/log4j.xml at zip root 467 | find-in-jars '^log4j\.(properties|xml)$' 468 | find-in-jars 'log4j\.properties$' -d /path/to/find/directory 469 | find-in-jars '\.properties$' -d /path/to/find/dir1 -d path/to/find/dir2 470 | find-in-jars 'Service\.class$' -e jar -e zip 471 | find-in-jars 'Mon[^$/]*Service\.class$' -s ' <-> ' 472 | 473 | Find control: 474 | -d, --dir the directory that find jar files. 475 | default is current directory. this option can specify 476 | multiply times to find in multiply directories. 477 | -e, --extension set find file extension, default is jar. this option 478 | can specify multiply times to find multiply extension. 479 | -E, --extended-regexp PATTERN is an extended regular expression (*default*) 480 | -F, --fixed-strings PATTERN is a set of newline-separated strings 481 | -G, --basic-regexp PATTERN is a basic regular expression 482 | -P, --perl-regexp PATTERN is a Perl regular expression 483 | -i, --ignore-case ignore case distinctions 484 | 485 | Output control: 486 | -a, --absolute-path always print absolute path of jar file 487 | -s, --separator specify the separator between jar file and zip entry. 488 | default is `!'. 489 | -L, --files-not-contained-found 490 | print only names of JAR FILEs NOT contained found 491 | -l, --files-contained-found 492 | print only names of JAR FILEs contained found 493 | -R, --no-find-progress do not display responsive find progress 494 | 495 | Miscellaneous: 496 | -h, --help display this help and exit 497 | -V, --version display version information and exit 498 | ``` 499 | 500 | 注意，Pattern缺省是`grep`的 **扩展**正则表达式。 501 | 502 | ### 示例 503 | 504 | ```bash 505 | # 在当前目录下的所有Jar文件中，查找出 log4j.properties文件 506 | $ find-in-jars 'log4j\.properties$' 507 | ./hadoop-core-0.20.2-cdh3u3.jar!log4j.properties 508 | ...... 509 | 510 | # 查找出 以Service结尾的类，Jar文件路径输出成绝对路径 511 | $ find-in-jars 'Service.class$' -a 512 | /home/foo/deploy/app/WEB-INF/libs/spring-2.5.6.SEC03.jar!org/springframework/stereotype/Service.class 513 | /home/foo/deploy/app/WEB-INF/libs/rpc-hello-0.0.1-SNAPSHOT.jar!com/taobao/biz/HelloService.class 514 | ...... 515 | 516 | # 在指定的多个目录的Jar文件中，查找出 properties文件 517 | $ find-in-jars '\.properties$' -d WEB-INF/lib -d ../deploy/lib | grep -v '/pom\.properties$' 518 | WEB-INF/lib/aspectjtools-1.6.2.jar!org/aspectj/ajdt/ajc/messages.properties 519 | WEB-INF/lib/aspectjweaver-1.8.8.jar!org/aspectj/weaver/XlintDefault.properties 520 | ../deploy/lib/groovy-all-1.1-rc-1.jar!groovy/ui/InteractiveShell.properties 521 | ../deploy/lib/httpcore-4.3.3.jar!org/apache/http/version.properties 522 | ../deploy/lib/javax.servlet-api-3.0.1.jar!javax/servlet/http/LocalStrings_es.properties 523 | ...... 524 | 525 | # 列出 包含properties文件的Jar文件 526 | $ find-in-jars '\.properties$' -l -d WEB-INF/lib 527 | WEB-INF/lib/aspectjtools-1.6.2.jar 528 | WEB-INF/lib/aspectjweaver-1.8.8.jar 529 | WEB-INF/lib/javax.servlet-api-3.0.1.jar 530 | ...... 531 | ``` 532 | 533 | ### 运行效果 534 | 535 | 支持彩色输出，文件名中的匹配部分以`grep`的高亮方式显示。 536 | 537 | ![find-in-jar screenshot](https://user-images.githubusercontent.com/1063891/33545067-9eb66072-d8a2-11e7-8a77-d815c0979e5e.gif) 538 | 539 | ### 参考资料 540 | 541 | [在多个Jar(Zip)文件查找Log4J配置文件的Shell命令行](http://oldratlee.github.io/458/tech/shell/find-file-in-jar-zip-files.html) 542 | --------------------------------------------------------------------------------