├── 0_Helloworld.sh ├── 1_2_traveling_all_arguments.sh ├── 1_using_arguments.sh ├── LICENSE ├── README.md ├── Shell Scripting tutorials.md ├── cycle_through_an_array.sh ├── dev_env_installation ├── docker-batch-internal-app.sh └── install—_mysql_1-master-and-2-slaves.sh ├── file_handle.sh ├── funnyGames └── BashSnake.sh ├── lnmp-full-example └── include │ ├── init.sh │ └── lnmp ├── loops.sh ├── mongodb ├── backup.sh ├── export.sh ├── import.sh └── restore.sh ├── mysql ├── mysql_dump_every_dbs.sh └── mysql_dump_every_dbs_struc.sh ├── scripts └── auto_clean_logs.sh ├── test.sh ├── using_brace_to_range.sh ├── utils ├── file │ ├── basename.sh │ ├── count_file.sh │ ├── count_lines.sh │ ├── create_new_file.sh │ ├── dirname.sh │ ├── extract_file_between_two_marker.sh │ ├── head.sh │ └── tail.sh ├── is_hex_color.sh ├── is_null.sh ├── lowercase.sh ├── lstrip.sh ├── random_array_element.sh ├── regex.sh ├── remove_array_dups.sh ├── reverse_array.sh ├── rstrip.sh ├── split.sh ├── trim.sh ├── uppercase.sh ├── urldecode.sh └── urlencode.sh ├── variable_usage.sh └── 系统性能分析__笔记_《性能之巅：洞悉系统_企业与云计算》.md /0_Helloworld.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | echo "Hello world!" 3 | -------------------------------------------------------------------------------- /1_2_traveling_all_arguments.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | # travel through all input arguments 4 | echo -e "Now total arguments is $#.\nAnd every one of them is the following:" 5 | count=1;step=1 6 | while (($# > 0)) # 注意这里循环体，while或者for ，这里的判断体要用两层括号包围来写，因为单层括号会被解释成【提高运算符优先级的普通括号】 7 | do 8 | echo -e "Argument $count : $1" 9 | count=`expr $count + $step` 10 | shift 11 | done 12 | -------------------------------------------------------------------------------- /1_using_arguments.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | #example of using arguments to a script 4 | echo "My first argument is $1" 5 | echo "My second argument is $2" 6 | echo "My third argument is $3" 7 | echo "My other arguments are $4 $5 $6 $7 $8 $9 ${10} ${11}" 8 | echo "Total amount of arguments is $#" 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 刘洋 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Shell 2 | Some sample scripts developed with shell. Also including some beginner's tutorials. 3 | -------------------------------------------------------------------------------- /Shell Scripting tutorials.md: -------------------------------------------------------------------------------- 1 | # Shell Scripting tutorials 2 | 3 | ----------------------------------- 4 | 5 | ## Category 6 | 7 | ### Part1 About the Ingredients 8 | 9 | #### 1. getting started 10 | 11 | #### 2. variables 12 | 13 | #### 3. wildcard expansion 14 | 15 | #### 4. conditional execution 16 | 17 | #### 5. flow control using loops 18 | 19 | #### 6. variables continued 20 | 21 | #### 7. functions and libraries 22 | 23 | #### 8. arrays 24 | 25 | #### 9. processes 26 | 27 | #### 10. choosing and using shells 28 | 29 | ### Part2 Recipes for Using and Extending system tools 30 | 31 | #### 11. file manipulation 32 | 33 | #### 12. text manipulation 34 | 35 | #### 13. tools for system administration 36 | 37 | ### Part3 Recipes for System administration 38 | 39 | #### 14. shell features 40 | 41 | #### 15. system administration 42 | 43 | #### 16. presentation 44 | 45 | #### 17. data storage and retrieval 46 | 47 | #### 18. numbers 48 | 49 | #### 19. processes 50 | 51 | #### 20. internationalization 52 | 53 | ### Part4 Reference 54 | 55 | #### Appendix. Further Reading 56 | 57 | #### Glossary 58 | 59 | ______________________________________ 60 | 61 | ### Part1 About the Ingredients 62 | 63 | #### 1.getting started 64 | 65 | 66 | 67 | ### Part2 Recipes for Using and Extending system tools 68 | 69 | ### Part3 Recipes for System administration 70 | 71 | ### Part4 Reference 72 | 73 | -------------------------------------------------------------------------------- /cycle_through_an_array.sh: -------------------------------------------------------------------------------- 1 | arr=(a b c d) 2 | 3 | cycle() { 4 | printf '%s ' "${arr[${i:=0}]}" 5 | ((i=i>=${#arr[@]}-1?0:++i)) 6 | } -------------------------------------------------------------------------------- /dev_env_installation/docker-batch-internal-app.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # docker批量 start|stop|restart|status 其中的应用 4 | 5 | function start_docker_internal_app() 6 | { 7 | docker_name = $1 8 | app_name = $2 9 | app_ops_operation = $3 10 | abc = $(docker exec -it $docker_name su -work -c "/home/user/app/$app_name/bin/console $app_ops_operation") 11 | } 12 | 13 | function get_docker_app_name() 14 | { 15 | ops_oper = $1 16 | for i in `docker ps -a|awk '{if ($8 == "Up") print $NF}' | grep -v "CONTAINER" | grep -v "^$"` 17 | do 18 | result = $(docker exec -it $i ls -lrt /home/user/app/ | tail -1 | awk '{print $9}' | tr -d "\r") 19 | start_docker_internal_app $i $result $ops_oper 20 | done 21 | } 22 | 23 | case "$1" in 24 | start) 25 | echo -e "\033[32m 应用启动 \033[0m" 26 | ops_operation = 'start' 27 | get_docker_app_name $ops_operation 28 | $1 29 | ;; 30 | stop) 31 | echo -e "\033[32m 应用停止 \033[0m" 32 | ops_operation = 'stop' 33 | get_docker_app_name $ops_operation 34 | $1 35 | ;; 36 | restart) 37 | echo -e "\033[32m 应用重启 \033[0m" 38 | ops_operation = 'restart' 39 | get_docker_app_name $ops_operation 40 | $1 41 | ;; 42 | status) 43 | echo -e "\033[32m 检查程序 \033[0m" 44 | ops_operation = 'status' 45 | get_docker_app_name $ops_operation 46 | ;; 47 | *) 48 | echo -e '\033[31m Docker内部应用管理，请输入 "$"Usage: $0 {status|start|stop|restart}" \033[0m' 49 | exit 2 50 | esac 51 | -------------------------------------------------------------------------------- /dev_env_installation/install—_mysql_1-master-and-2-slaves.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # under centos7.5 3 | 4 | slave_user = 'slave1' 5 | slave_password = '123456' 6 | slave_ipaddr = '192.168.1.102' 7 | master_ipaddr = '192.168.1.104' 8 | yum -y install openssh-clients 9 | yum -y install mysql mysql-server mysql-devel 10 | sed -i '/^\[mysqld\]$/a\server-id=1' /etc/my.cnf 11 | sed -i '/^\[mysqld\]$/a\log-bin=mysql-bin' /etc/my.cnf 12 | 13 | /etc/init.d/mysqld restart 14 | 15 | # set mysql password 16 | mysqladmin -u root password '123456' 17 | mysql -uroot -p123456 -e "grant replication slave, super, reload on *.* to '$slave_user'@'$slave_ipaddr' identified by '$slave_password';" 18 | 19 | master_status = `mysql -uroot -p123456 -e "show mastaer status;"` 20 | echo "$master_status" 21 | binlogname = `echo "$master_status" | grep bin | awk '{print $1}'` 22 | echo '$binlogname' 23 | position = `echo "$master_status" | grep bin | awk '{print $2}'` 24 | echo '$position' 25 | 26 | # set slave 27 | ssh root@$slave_ipaddr > /dev/null 2>&1 << eeooff 28 | yum -y install mysql mysql-server mysql-devel 29 | sed -i '/^\[mysqld\]$/a\server-id=2' /etc/my.cnf 30 | /etc/init.d/mysqld restart 31 | mysqladmin -u root password '123456' 32 | mysql -uroot -p123456 -e 'stop slave;' 33 | mysql -uroot -p123456 -e "change master to master_host = '$master_ipaddr', master_user = '$slave_user', master_password='$slave_password', master_log_file = '$binlogname', master_log_pos = '$position';" 34 | mysql -uroot -p123456 -e 'start slave;' 35 | 36 | # 判断slave 37 | slave_status = mysql -uroot -p123456 -e "show slave status\G;" | grep yes | wc -l 38 | if [$slave_status == 2] then 39 | echo "slave is ok." 40 | else 41 | echo "slave fail." 42 | fi 43 | exit 44 | eeooff -------------------------------------------------------------------------------- /file_handle.sh: -------------------------------------------------------------------------------- 1 | # read a file to string 2 | file_data = "$(<"file")" 3 | 4 | # read a file to an array by line 5 | # if Bash version<4 6 | IFS = $'\n' read -d "" -ra file_data < "file" 7 | # if Bash version>=4 8 | mapfile -t file_data < "file" 9 | -------------------------------------------------------------------------------- /funnyGames/BashSnake.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | IFS='' 4 | 5 | declare -i height=$(($(tput lines)-5)) width=$(($(tput cols)-2)) 6 | 7 | # row and column number of head 8 | declare -i head_r head_c tail_r tail_c 9 | 10 | declare -i alive 11 | declare -i length 12 | declare body 13 | 14 | declare -i direction delta_dir 15 | declare -i score=0 16 | 17 | border_color="\e[30;43m" 18 | snake_color="\e[32;42m" 19 | food_color="\e[34;44m" 20 | text_color="\e[31;43m" 21 | no_color="\e[0m" 22 | 23 | #signals 24 | SIG_UP=USR1 25 | SIG_RIGHT=USR2 26 | SIG_DOWN=URG 27 | SIG_LEFT=IO 28 | SIG_QUIT=WINCH 29 | SIG_DEAD=HUP 30 | 31 | # direction arrays: 0 = up, 1 = right, 2 = down, 3 = left 32 | move_r=([0]=-1 [1]=0 [2]=1 [3]=0) 33 | move_c=([0]=0 [1]=1 [2]=0 [3]=-1) 34 | 35 | init_game() { 36 | clear 37 | echo -ne "\e[?251" 38 | stty -echo 39 | for((i=0; i < height; i++)); do 40 | for((j=0; j < width; j++)); do 41 | eval "arr$i[$j]=' '" 42 | done 43 | done 44 | } 45 | 46 | move_and_draw() { 47 | echo -ne "\e[${1};${2}H$3" 48 | } 49 | 50 | draw_board() { 51 | move_and_draw 1 1 "$border_color+$no_color" 52 | for((i=2; i<=width+1; i++)); do 53 | move_and_draw 1 $i "$border_color-$no_color" 54 | done 55 | move_and_draw 1 $((width+2)) "$border_color-$no_color" 56 | echo 57 | 58 | for((i=0; i> /etc/hosts 83 | fi 84 | pingresult = `ping -c1 lnmp.org 2>&1` 85 | echo "${pingresult}" 86 | if echo "${pingresult}" | grep -q "unknown host"; then 87 | echo "DNS fail" 88 | echo "Writing nameserver to /etc/resolv.conf..." 89 | echo -e "nameserver 208.67.220.220\nnameserver 114.114.114.114" > /etc/resolv.conf 90 | else 91 | echo "Dns...ok" 92 | fi 93 | } 94 | 95 | RHEL_Modify_Source() 96 | { 97 | GET_RHEL_Version 98 | \cp $(cur_dir)/conf/CentOS-Base-163.repo /etc/yum.repos.d/CentOS-Base-163.repo 99 | sed -i "s/RPM-GPG-KEY-CentOS-6/RPM-GPG-KEY-CentOS-${RHEL_Ver}/g" /etc/yum.repos.d/CentOS-Base-163.repo 100 | yum clean all 101 | yum makecache 102 | } 103 | 104 | Ubuntu_Modify_Source() 105 | { 106 | if ["${country}" = "CN"]; then 107 | OldReleasesURL = 'http://mirrors.ustc.edu.cn/ubuntu-old-releases/ubuntu/' 108 | else 109 | OldReleasesURL = 'http://old-releases.ubuntu.com/ubuntu/' 110 | fi 111 | CodeName = '' 112 | if grep -Eqi "10.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^10.10'; then 113 | CodeName='maverick' 114 | elif grep -Eqi "11.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^11.04'; then 115 | CodeName='natty' 116 | elif grep -Eqi "11.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^11.10'; then 117 | CodeName='oneiric' 118 | elif grep -Eqi "12.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^12.10'; then 119 | CodeName='quantal' 120 | elif grep -Eqi "13.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^13.04'; then 121 | CodeName='raring' 122 | elif grep -Eqi "13.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^13.10'; then 123 | CodeName='saucy' 124 | elif grep -Eqi "10.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^10.04'; then 125 | CodeName='lucid' 126 | elif grep -Eqi "14.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^14.10'; then 127 | CodeName='utopic' 128 | elif grep -Eqi "15.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^15.04'; then 129 | CodeName='vivid' 130 | elif grep -Eqi "12.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^12.04'; then 131 | CodeName='precise' 132 | elif grep -Eqi "15.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^15.10'; then 133 | CodeName='wily' 134 | elif grep -Eqi "16.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^16.10'; then 135 | CodeName='yakkety' 136 | elif grep -Eqi "14.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^14.04'; then 137 | Ubuntu_Deadline trusty 138 | elif grep -Eqi "17.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^17.04'; then 139 | CodeName='zesty' 140 | elif grep -Eqi "17.10" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^17.10'; then 141 | Ubuntu_Deadline artful 142 | elif grep -Eqi "16.04" /etc/*-release || echo "${Ubuntu_Version}" | grep -Eqi '^16.04'; then 143 | Ubuntu_Deadline xenial 144 | fi 145 | if [ "${CodeName}" != "" ]; then 146 | \cp /etc/apt/sources.list /etc/apt/sources.list.$(date +"%Y%m%d") 147 | cat > /etc/apt/sources.list<&1 | awk '/^HTTP/{print $2}'` 165 | if [ ${OR_Status} != '404']; then 166 | echo "Ubuntu old-releases status: ${OR_Status}"; 167 | codeName = $1 168 | fi 169 | } 170 | 171 | Ubuntu_Deadline() 172 | { 173 | trusty_deadline = `date-d "2019-7-22 00:00:00" +%s` 174 | artful_deadline = `date-d "2018-7-31 00:00:00" +%s` 175 | xenial_deadline = `date-d "2021-4-30 00:00:00" +%s` 176 | cur_time = `date +%s` 177 | case "$1" in 178 | trusty) 179 | if [ ${cur_time} -gt ${trusty_deadline} ]; then 180 | echo "${cur_time} > ${trusty_deadline}" 181 | Check_Old_Releases_URL trusty 182 | fi 183 | ;; 184 | artful) 185 | if [ ${cur_time} -gt ${artful_deadline} ]; then 186 | echo "${cur_time} > ${artful_deadline}" 187 | Check_Old_Releases_URL artful 188 | fi 189 | ;; 190 | xenial) 191 | if [ ${cur_time} -gt ${xenial_deadline} ]; then 192 | echo "${cur_time} > ${xenial_deadline}" 193 | Check_Old_Releases_URL xenial 194 | fi 195 | ;; 196 | esac 197 | } -------------------------------------------------------------------------------- /lnmp-full-example/include/lnmp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OneCodeMonkey/Shell/0c12acd36511f836a58354a7564d1c1817768025/lnmp-full-example/include/lnmp -------------------------------------------------------------------------------- /loops.sh: -------------------------------------------------------------------------------- 1 | # loop over a range of numbers 2 | # loop from 0-100(no variable support) 3 | for i in {0..100}; do 4 | printf '%s\n' "$i" 5 | done 6 | 7 | # loop over a variable range of numbers 8 | # loop from 0-VAR 9 | VAR=50 10 | for (( i = 0; i <= VAR; i++)); do 11 | printf '%s\n' "$i" 12 | done 13 | 14 | # loop over an array 15 | arr = (apples oranges tomatoes) 16 | # just elements 17 | for element in "${arr[@]}"; do 18 | printf '%s\n' "$element" 19 | done 20 | 21 | # loop over an array with an index 22 | arr = (apples oranges tomatoes) 23 | # elements and index. 24 | for i in "${!arr[@]}"; do 25 | printf '%s\n' "${arr[i]}" 26 | done 27 | # another alternative 28 | for (( i = 0; i < ${#arr[@]}; i++)); do 29 | printf '%s\n' "${arr[i]}" 30 | done 31 | 32 | # loop over the contents of a file 33 | while read -r line; do 34 | printf '%s\n' "$line" 35 | done 36 | 37 | # loop over files and directories without command "ls" 38 | # greedy alternative 39 | for file in *; do 40 | printf '%s\n' "$file" 41 | done 42 | # PNG files in dir. 43 | for file in ~/images/*.png; do 44 | printf '%s\n' "$file" 45 | done 46 | # Iterator over directories 47 | for dir in ~/folder_name/*/; do 48 | printf '%s\n' "$dir" 49 | done 50 | # Brace Expansion 51 | for file in /path/to/parentdir/{file1, file2, subdir/file3}; do 52 | printf '%s\n' "$file" 53 | done 54 | # Iterator recursively 55 | shopt -s globstar 56 | for file in ~/images/**/*; do 57 | printf '%s\n' "$file" 58 | done 59 | shopt -u globstar -------------------------------------------------------------------------------- /mongodb/backup.sh: -------------------------------------------------------------------------------- 1 | HOST=$1 2 | PORT=$2 3 | MONGODB_DBNAME=$3 4 | PATH=$4 5 | 6 | sudo mongodump -h "$HOST:$PORT" -d $MONGODB_DBNAME -o $PATH 7 | -------------------------------------------------------------------------------- /mongodb/export.sh: -------------------------------------------------------------------------------- 1 | MONGODB_DBNAME=$1 2 | COLLECTION_NAME=$2 3 | EXPORT_PATH=$3 4 | TYPE=$4 5 | FIELDS1=$5 6 | FIELDS2=$6 7 | FIELDS3=$7 8 | 9 | sudo mongoexport -d $MONGODB_DBNAME -c $COLLECTION_NAME -o $EXPORT_PATH --type $TYPE -f "$FIELDS1,$FIELDS2,$FIELDS3" 10 | -------------------------------------------------------------------------------- /mongodb/import.sh: -------------------------------------------------------------------------------- 1 | MONGODB_DBNAME=$1 2 | COLLECTION_NAME=$2 3 | IMPORT_PATH=$3 4 | TYPE=$4 5 | FIELDS1=$5 6 | FIELDS2=$6 7 | FIELDS3=$7 8 | 9 | sudo mongoimport -d $MONGODB_DBNAME -c $COLLECTION_NAME --file $IMPORT_PATH --headerline --type $TYPE -f "$FIELDS1,$FIELDS2,$FIELDS3" 10 | -------------------------------------------------------------------------------- /mongodb/restore.sh: -------------------------------------------------------------------------------- 1 | HOST=$1 2 | PORT=$2 3 | MONGODB_DBNAME=$3 4 | PATH=$4 5 | 6 | sudo mongorestpre -h "$HOST:$PORT" -d $MONGODB_DBNAME --dir $PATH 7 | -------------------------------------------------------------------------------- /mysql/mysql_dump_every_dbs.sh: -------------------------------------------------------------------------------- 1 | TIMESTAMP=$(date +"%F") 2 | BACKUP_DIR="./sql/" 3 | MYSQL_USER=$1 4 | MYSQL=/usr/bin/mysql 5 | MYSQL_PASSWORD=$2 6 | MYSQL_HOST=$3 7 | MYSQLDUMP=/usr/bin/mysqldump 8 | 9 | mkdir -p "$BACKUP_DIR" 10 | 11 | if [ -z "$MYSQL_HOST" ] 12 | then 13 | MYSQL_HOST="127.0.0.1" 14 | fi 15 | 16 | databases=`$MYSQL --user=$MYSQL_USER -p$MYSQL_PASSWORD -h$MYSQL_HOST -e "SHOW DATABASES;" | grep -Ev "(Database|information_schema|performance_schema)"` 17 | 18 | for loop in 5 4 3 2 1 19 | do 20 | echo "will begin dumping in $loop s..." 21 | sleep 1 22 | done 23 | 24 | count=0; 25 | for db in $databases; do 26 | echo "dumping database: $db..." 27 | $MYSQLDUMP --force --opt --user=$MYSQL_USER -p$MYSQL_PASSWORD --databases $db > "$BACKUP_DIR/$db.sql" 28 | count=`expr $count + 1` 29 | echo "dumping database $db completed!" 30 | done 31 | echo "finished dumping $count databases." -------------------------------------------------------------------------------- /mysql/mysql_dump_every_dbs_struc.sh: -------------------------------------------------------------------------------- 1 | TIMESTAMP=$(date +"%F") 2 | BACKUP_DIR="./sql_only_structure/" 3 | MYSQL_USER=$1 4 | MYSQL=/usr/bin/mysql 5 | MYSQL_PASSWORD=$2 6 | MYSQL_HOST=$3 7 | MYSQLDUMP=/usr/bin/mysqldump 8 | 9 | mkdir -p "$BACKUP_DIR" 10 | 11 | if [ -z "$MYSQL_HOST" ] 12 | then 13 | MYSQL_HOST="127.0.0.1" 14 | fi 15 | 16 | databases=`$MYSQL --user=$MYSQL_USER -p$MYSQL_PASSWORD -h$MYSQL_HOST -e "SHOW DATABASES;" | grep -Ev "(Database|information_schema|performance_schema)"` 17 | 18 | for loop in 5 4 3 2 1 19 | do 20 | echo "will begin dumping in $loop s..." 21 | sleep 1 22 | done 23 | 24 | count=0; 25 | for db in $databases; do 26 | echo "dumping database: $db..." 27 | $MYSQLDUMP --force --opt --no-data --user=$MYSQL_USER -p$MYSQL_PASSWORD --databases $db > "$BACKUP_DIR/$db.sql" 28 | count=`expr $count + 1` 29 | echo "dumping database $db completed!" 30 | done 31 | echo "finished dumping $count databases." -------------------------------------------------------------------------------- /scripts/auto_clean_logs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | before_days=30 # 间隔天数，删除指定天数之前存的log 4 | LOG_DIR=/home/XXXX/runtimes/logs/ # 目录是示意，替换成目标log存放目录即可 5 | 6 | # 遍历删除 LOG_DIR 下的log 7 | # 原理是找到 LOG_DIR/{A,B,C, ...} 的各个目录，根据目录下的文件 {date}_info.log 或者 {date}_error.log 中的 date 与当前日期天数之差是否大于指定天数，满足则删除 date_{info,error}.log 文件 8 | echo "Now will clean folder $LOG_DIR ..." 9 | for((i = 5; i > 0; i--)); do 10 | echo $i 11 | sleep 1 12 | done 13 | 14 | dir_log_dir=`ls $LOG_DIR` 15 | for i in $dir_log_dir; do 16 | SUB_DIR=$LOG_DIR$i 17 | # echo $SUB_DIR 18 | content=$(date +%Y-%m-%d --date "$before_days days ago") 19 | 20 | cd $SUB_DIR 21 | # 删除 {data_error}.log 文件 22 | DIR_NUM=$(find -name "$content"_error.log|wc -l) 23 | if [ "$DIR_NUM" -gt 0 ];then # 如果文件存在 24 | rm -f $SUB_DIR/$content""_error.log 25 | echo "log file deleted: [$SUB_DIR/$content""_error.log]" 26 | echo `date "+%Y-%m-%d %H:%M:%S"`" -- deleted file: "$SUB_DIR/$content""_error.log >> $LOG_DIR/auto_clean.log 27 | fi 28 | 29 | # 删除 {data_info}.log 文件 30 | DIR_NUM=$(find -name "$content"_info.log|wc -l) 31 | if [ "$DIR_NUM" -gt 0 ];then 32 | rm -f $SUB_DIR/$content""_info.log 33 | echo "log file deleted: [$SUB_DIR/$content""_info.log]" 34 | echo `date "+%Y-%m-%d %H:%M:%S"`" -- deleted file: "$SUB_DIR/$content""_info.log >> $LOG_DIR/auto_clean.log 35 | fi 36 | done 37 | echo -e "Completed! folder $LOG_DIR cleaned\n" -------------------------------------------------------------------------------- /test.sh: -------------------------------------------------------------------------------- 1 | echo $*; 2 | -------------------------------------------------------------------------------- /using_brace_to_range.sh: -------------------------------------------------------------------------------- 1 | # for ranges 2 | 3 | # print numbers 1-100 4 | echo {1..100} 5 | 6 | # print range of floats 1.1 - 1.9 7 | echo 1.{1..9} 8 | 9 | # print chars a to z 10 | echo {a..z} 11 | echo {A..Z} 12 | 13 | # nesting 14 | echo {A..Z}{0..9} 15 | 16 | # print zero-padded numbers 001,002,..,010,011,..,100 17 | echo {01..100} 18 | 19 | # change increment amount 20 | # for example, increment by 2 21 | echo {1..10..2} 22 | 23 | # array 24 | echo {apples,pears,bananas} 25 | 26 | # remove series of file or folder 27 | rm -rf ~/foldername/{a.txt,b.txt,c.txt} -------------------------------------------------------------------------------- /utils/file/basename.sh: -------------------------------------------------------------------------------- 1 | # get base-name of a path 2 | basename() { 3 | # Usage: basename "path" 4 | : "${1%/}" 5 | printf '%s\n' "${_##*/}" 6 | } 7 | # >> basename ~/images/a.jpg 8 | # >> a.jpg 9 | # >> basename ~/images/ 10 | # >> images -------------------------------------------------------------------------------- /utils/file/count_file.sh: -------------------------------------------------------------------------------- 1 | # count files or directories in directory 2 | count_file_or_dir() { 3 | # Usage: count_file_or_dir /path/to/dir/* 4 | # count_file_or_dir /path/to/dir/*/ 5 | printf '%s\n' "$#" 6 | } -------------------------------------------------------------------------------- /utils/file/count_lines.sh: -------------------------------------------------------------------------------- 1 | # get the count of lines of a file 2 | line_count() { 3 | # Usage: line_count "file" 4 | mapfile -tn 0 lines < "$1" 5 | printf '%s\n' "${#lines[@]}" 6 | } 7 | 8 | # another alternative in bash<=3 9 | line_count_2() { 10 | # Usage: line_count_2 "file" 11 | count = 0 12 | while IFS= read -r _; do 13 | ((count++)) 14 | done < "$1" 15 | printf '%s\n' "$count" 16 | } -------------------------------------------------------------------------------- /utils/file/create_new_file.sh: -------------------------------------------------------------------------------- 1 | # solution 1 2 | >filename 3 | 4 | # solution 2 5 | :>filename 6 | 7 | # set file content 8 | echo 'aawerewa' >filename -------------------------------------------------------------------------------- /utils/file/dirname.sh: -------------------------------------------------------------------------------- 1 | dirname() { 2 | # Usage: dirname "path" 3 | printf '%s\n' "${1%/*}/" 4 | } -------------------------------------------------------------------------------- /utils/file/extract_file_between_two_marker.sh: -------------------------------------------------------------------------------- 1 | # extract file between two marker 2 | extract() { 3 | # Usage extract filename "opening marker" "closing marker" 4 | while IFS=$'\n' read -r line; do 5 | [[ $extract && $line != "$3" ]] && 6 | print '%s\n' "$line" 7 | [[ $line == "$2" ]] && extract = 1 8 | [[ $line == "$3" ]] && extract = 9 | done < "$1" 10 | } -------------------------------------------------------------------------------- /utils/file/head.sh: -------------------------------------------------------------------------------- 1 | # get the first N lines of a file 2 | head() { 3 | # Usage: head "n" "file" 4 | mapfile -tn "$1" line < "$2" 5 | printf '%s\n' "${line[@]}" 6 | } -------------------------------------------------------------------------------- /utils/file/tail.sh: -------------------------------------------------------------------------------- 1 | # get the last N lines of a file 2 | tail() { 3 | # Usage: tail "n" "file" 4 | mapfile -tn 0 line < "$2" 5 | printf '%s\n' "${line[@]: -$1}" 6 | } -------------------------------------------------------------------------------- /utils/is_hex_color.sh: -------------------------------------------------------------------------------- 1 | is_hex_color() { 2 | if [[ $1 =~ ^(#?([a-fA-F0-9]{6}|[a-fA-F0-9]{3}))$ ]]; then 3 | printf '%s\n' "${BASH_REMATCH[1]}" 4 | else 5 | printf '%s\n' "error: $1 is an invalid color!" 6 | return 1 7 | fi 8 | } 9 | read -r color 10 | is_hex_color "$color" || color='#FFFFFF' -------------------------------------------------------------------------------- /utils/is_null.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | is_null() { 3 | var=$1 4 | if [ ! $var ]; then 5 | echo "is null" 6 | else 7 | echo "not null" 8 | fi 9 | } 10 | -------------------------------------------------------------------------------- /utils/lowercase.sh: -------------------------------------------------------------------------------- 1 | lowercase() { 2 | # Usage: lower "string" 3 | printf '%s\n' "${1,,}" 4 | } -------------------------------------------------------------------------------- /utils/lstrip.sh: -------------------------------------------------------------------------------- 1 | lstrip() { 2 | # Usage: lstrip "string" "pattern" 3 | printf '%s\n' "${1##$2}" 4 | } -------------------------------------------------------------------------------- /utils/random_array_element.sh: -------------------------------------------------------------------------------- 1 | random_array_element() { 2 | # Usage random_array_element "array" 3 | local arr=("$@") 4 | printf '%s\n' "${arr[RANDOM % $#]}" 5 | } -------------------------------------------------------------------------------- /utils/regex.sh: -------------------------------------------------------------------------------- 1 | regex() { 2 | # Usage: regex "string" "regex" 3 | [[ $1 =~ $2 ]] && printf '%s\n' "${BASH_REMATCH[1]}" 4 | } -------------------------------------------------------------------------------- /utils/remove_array_dups.sh: -------------------------------------------------------------------------------- 1 | # Usage: remove duplicate array elements 2 | remove_array_dups() { 3 | # Usage: remove_array_dups "array" 4 | declare -A tmp_array 5 | for i in "$@"; do 6 | [[ $i ]] && IFS=" " tmp_array["${i:- }"]=1 7 | done 8 | 9 | printf '%s\n' "${!tmp_array[@]}" 10 | } -------------------------------------------------------------------------------- /utils/reverse_array.sh: -------------------------------------------------------------------------------- 1 | reverse_array() { 2 | # Usage: reverse_array "array" 3 | shopt -s extdebug 4 | f()(printf '%s\n' "${BASH_ARGV[@]}"); f "$@" 5 | shopt -u extdebug 6 | } -------------------------------------------------------------------------------- /utils/rstrip.sh: -------------------------------------------------------------------------------- 1 | rstrip() { 2 | # Usage: rstrip "string" "pattern" 3 | printf '%s\n' "${1%%$2}" 4 | } -------------------------------------------------------------------------------- /utils/split.sh: -------------------------------------------------------------------------------- 1 | split() { 2 | # Usage: split "string" "delimiter" 3 | IFS=$'\n' read -d "" -ra arr <<< "${1//$2/$'\n'}" 4 | printf '%s\n' "${arr[@]}" 5 | } -------------------------------------------------------------------------------- /utils/trim.sh: -------------------------------------------------------------------------------- 1 | trim_string() { 2 | # Usage: trim_string " example string " 3 | # >> example string 4 | : "${1#"${1%%[![:space:]]*}"}" 5 | : "${_%"${_##*[![:space:]]}"}" 6 | printf '%s\n' "$_" 7 | } 8 | 9 | trim_all() { 10 | # Usage: trim_all " example string " 11 | # >> example string 12 | set -f 13 | set -- $* 14 | printf '%s\n' "$*" 15 | set +f 16 | } -------------------------------------------------------------------------------- /utils/uppercase.sh: -------------------------------------------------------------------------------- 1 | uppercase() { 2 | # Usage: upper "string" 3 | printf '%s\n' "${1^^}" 4 | } -------------------------------------------------------------------------------- /utils/urldecode.sh: -------------------------------------------------------------------------------- 1 | urldecode() { 2 | # Usage: urldecode "https%3A%2F%2Fgithub.com" 3 | # https://github.com 4 | : "${1//+/ }" 5 | printf '%b\n' "${_//%/\\x}" 6 | } -------------------------------------------------------------------------------- /utils/urlencode.sh: -------------------------------------------------------------------------------- 1 | urlencode() { 2 | # Usage: urlencode "https://github.com" 3 | # https%3A%2F%2Fgithub.com 4 | local LC_ALL=C 5 | for(( i = 0; i < ${#1}; i++ )); do 6 | : "${1:i:1}" 7 | case "$_" in 8 | [a-zA-Z0-9.~_-]) 9 | printf '%s' "$_" 10 | ;; 11 | 12 | *) 13 | printf '%%%02X' "'$_" 14 | ;; 15 | esac 16 | done 17 | printf '\n' 18 | } -------------------------------------------------------------------------------- /variable_usage.sh: -------------------------------------------------------------------------------- 1 | # assign and access a variable using another variable 2 | hello_world="value" 3 | # create the variable name 4 | var="world" 5 | ref="hello_$var" 6 | # print the value of ref 7 | printf '%s\n' "${!ref}" 8 | # >> value 9 | 10 | # name a variable based on another variable 11 | var="world" 12 | declare "hello_$var=value" 13 | printf '%s\n' "$hello_world" 14 | # >> value -------------------------------------------------------------------------------- /系统性能分析__笔记_《性能之巅：洞悉系统_企业与云计算》.md: -------------------------------------------------------------------------------- 1 | # 系统性能分析--读书笔记--《性能之巅：洞悉系统，企业和云计算》 2 | 3 | 4 | 5 | #### 推荐诊断 CheckLists: 6 | 7 | ![img](https://img-blog.csdnimg.cn/20181219214641376.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 8 | 9 | 10 | 11 | ## 1. 绪论 12 | 13 | 系统性能是对整个系统的研究，包括了所有的硬件组件和整个软件栈。所有数据路径上和软硬件上所发生的事情集合，因为这些都会影响性能。 14 | 15 | ### 1.1 通用系统软件栈 16 | 17 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219205301572.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 18 | 19 | ### 1.2 性能的事情列表 20 | 21 | | 性能事件 | 22 | | ------------------------------------------ | 23 | | 设置性能目标和建立性能模型 | 24 | | 基于软件和硬件原型，归纳出性能特征 | 25 | | 对开发代码进行性能分析（软件整合之前） | 26 | | 执行软件非回归性测试（软件发布前或发布后） | 27 | | 针对软件发布版本的基准测试 | 28 | | 目标环境中的概念验证测试 | 29 | | 生产环境部署的配置优化 | 30 | | 监控生产环境中运行的软件 | 31 | | 特定问题的性能分析 | 32 | 33 | ### 1.3 两种性能分析视觉：资源分析和负载分析 34 | 35 | 性能术语**“容量规划”（capacity planning）**指的是一系列事前行动。在设计阶段，包括通过研究开发软件的资源占用情况，来得知原有设计在多大程度上能满足目标需求。在部署后，包括监控资源的使用情况，这样问题在出现之前就能预测到。 36 | 37 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219205613470.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 38 | 39 | - 资源分析指标：IPOS，吞吐量，使用率，饱和度 40 | - 工作负载指标：吞吐量和延时。 41 | 42 | 性能分析必须量化问题的重要与否程度，有一个指标很合适，叫做延时（latency） 43 | 44 | 动态跟踪技术把所有的软件变得可监控，而且能用于真实的生产环境中。这项技术利用内存中的 CPU 指令并在这些指令上动态构建检测数据。这样能从任何运行的软件中获取定制化的性能统计数据，从而提供远超系统自带统计所能给予的观测性。 45 | 46 | DTrace 对用户态和内核态的软件都提供了静态跟踪和动态跟踪，并且数据是实时产生的。 47 | 48 | ### 1.4 受测系统 49 | 50 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219204638349.jpg) 51 | 52 | 扰动（perturbation）是会影响结果的。扰动包括定时执行的系统活动，系统的其他用户以及其他的工作负载。现代环境的另一个困难是系统很可能由若干个网络化的组件组成，都用于处理输入工作负载，包括负载均衡，Web服务器，数据库服务器，应用程序服务器，以及存储系统。 53 | 54 | ### 1.5 排队系统 55 | 56 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219204719439.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 57 | 58 | ### 1.6 排队理论及使用率响应时间关系 59 | 60 | ![平均响应时间随使用率变化曲线](https://img-blog.csdnimg.cn/20181219213450339.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 61 | 62 | ![平均响应时间随使用率变化曲线](https://img-blog.csdnimg.cn/20181219213516845.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 63 | 64 | ### 1.7 方法 65 | 66 | > 在取得数据之前就把事情理论化，是一个严重的错误。不理智的人歪曲事实来适应自己的理论，而不是通过修正理论来适应事实。 67 | 68 | 面对一个性能不佳，且有一定复杂程度的系统环境时，首先要知道的就是从什么地方开始分析，开始收集什么样的数据，以及如何分析数据。 69 | 70 | #### 1.7.1 通用的性能分析方法 71 | 72 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219212719529.png) 73 | 74 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219212744573.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 75 | 76 | - 街灯讹方法：在熟悉的工具或流程中试错，比较盲目 77 | - Ad Hoc 核对清单法：保证所有人知道如何检查最糟糕的问题，覆盖全面，但必须保持清单及时更新 78 | - 诊断循环：假设——仪器检验——数据——假设 79 | - USE 方法：本书重点要讲的方法：对于所有资源，我们依次查看它的使用率，饱和度，错误。 80 | 81 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219213001360.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 82 | 83 | 84 | 85 | ## 2. 操作系统 86 | 87 | 了解操作系统和内核，对于系统性能分析是至关重要的。我们经常针对系统行为来开发和测试，如系统调用是如何执行的，CPU 是如何调度线程的，有限大小的内存是如何影响性能的，或者文件系统是如何处理 I/O 的。 88 | 89 | 这部分书中介绍了如下操作系统相关基本概念： 90 | 91 | - 内核：内核执行，时钟周期，内核态/用户态 92 | - 栈：用户栈和内核栈 93 | - 中断和中断线程 94 | - 中断优先级 95 | - 进程：进程创建，进程的生命周期，进程环境 96 | - 系统调用 97 | - 虚拟内存 98 | - 内存管理 99 | - 调度器 100 | - 文件系统：VFS, I/O 栈 101 | - 缓存（括号内表示例子）： 102 | - 应用程序缓存 103 | - 服务器缓存（Apache缓存） 104 | - 缓存服务器（Redis缓存） 105 | - 数据库缓存（MySQL缓冲区高速缓存） 106 | - 目录缓存（DNLC） 107 | - 文件元数据缓存（inode缓存） 108 | - 操作系统缓冲区高速缓存（segvn） 109 | - 文件系统主缓存（ZFS ARC） 110 | - 文件系统次级缓存（ZFS L2ARC） 111 | - 设备缓存（ZFS vdev） 112 | - 块缓存（缓冲区高速缓存） 113 | - 磁盘控制器缓存（RAID卡缓存） 114 | - 存储阵列缓存 115 | - 磁盘内置缓存 116 | - 网络 117 | - 设备驱动 118 | - 多处理器：CPU 交叉调用 119 | - 抢占 / 非抢占 120 | - 资源管理 121 | - 观测性 122 | 123 | 124 | 125 | ## 3. 观测工具 126 | 127 | 性能观测工具可以按照系统级别和进程级别来分类，多数工具要么基于计数器，要么基于跟踪： 128 | 129 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219204808393.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 130 | 131 | 132 | 133 | ### 3.1 计数器 134 | 135 | 操作系统内核中维护了各种统计数据，称为计数器，用于对事件计数。通常计数器实现为 unsigned int，发生事件时递增。 136 | 137 | 系统级的计数器有： 138 | 139 | - vmstat：虚拟内存和物理内存统计，系统级别 140 | - mpstat：每个 CPU 的使用情况 141 | - iostat：每个磁盘 I /O 的使用情况，由块设备接口报告 142 | - netstat：网络接口的统计，TCP/IP 栈的统计，以及每个连接的一些统计信息 143 | - sar：各种各样的统计，能归档历史数据 144 | 145 | 进程级别的计数器： 146 | 147 | - ps：进程状态，显示进程的各种统计信息，包括内存和 CPU 使用 148 | - top：按一个统计数据排序，显示排名高的进程 149 | - pmap：将进程的内存段和使用统计一起列出 150 | 151 | 一般来说，上述工具是从 `/proc` 文件系统里读取统计信息的 152 | 153 | ### 3.2 跟踪 154 | 155 | 跟踪收集每一个事件的数据以供分析。跟踪框架一般默认是不启用的，因为跟踪捕获数据会有 CPU 开销，另外还需要不小的存储空间来存放数据。 156 | 157 | 系统级别： 158 | 159 | - tcpdump：网络包跟踪（libpcap lib) 160 | - blktrace：块 I/O 跟踪 161 | - DTrace：跟踪内核的内部活动和所有资源的使用情况，支持静态和动态的跟踪 162 | - SystemTap：作用 = DTrace 163 | - perf：Linux 性能事件，跟踪动态和静态的指针 164 | 165 | 进程级别： 166 | 167 | - strace：系统调用跟踪 168 | - gbd：源码级别的调试器 169 | 170 | 171 | 172 | ## 4. 应用程序 173 | 174 | > 性能调整离工作具体执行的地方越近越好，最好在应用程序里，包括 Web 服务器，应用服务器，负载均衡器，文件服务器等。 175 | 176 | 设立性能目标能为我们分析性能指明方向，并帮助我们做出选择。没有清晰的目标，性能分析容易沦为“一通乱撞” 177 | 178 | 常见指标： 179 | 180 | - 延时（latency） 181 | - 吞吐量 182 | - 资源使用率 183 | 184 | 应用程序性能技术：选择 I/O 尺寸，缓存，缓冲区，轮询（epoll），并发和并行，非阻塞 I/O，处理器绑定 185 | 186 | 187 | 188 | ## 5. CPU分析 189 | 190 | ### 5.1 CPU分析工具 191 | 192 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219214148400.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 193 | 194 | ### 5.2 CPU调优 195 | 196 | - 当前有多少 CPU 可用？是核吗？是硬件的线程？ 197 | - CPU 架构是单架构还是多处理器架构 198 | - CPU 缓存大小是多少？是共享 CPU 吗？ 199 | - CPU 的时钟频率是多少？是动态时钟吗？是否基于 BIOS ？ 200 | - CPU 有其它特性？ 201 | - CPU 勘误表上有硬件上的 bug 吗? 202 | - BIOS 版本有 bug 吗？ 203 | - CPU 有使用软件的限制吗 ？ 204 | 205 | ## 6. 内存分析 206 | 207 | ### 6.1 内存分析工具 208 | 209 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219214309641.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 210 | 211 | ### 6.2 内存调优 212 | 213 | - 主内存有多少？ 214 | - 配置允许应用程序使用的内存大小？ 215 | - 使用哪个分配器？ 216 | - 主内存 I/O 速度？ 217 | - 系统架构？ NUMA（共享存储器，物理上分布在所有处理机的本地存储器上的存储器） ? UMA（均匀存储器存取） ? 218 | - 内存总线大小？ 219 | - 是否配置使用了大页面？ 220 | - 是否支持和配置过度提交？ 221 | - 使用了哪些内存可调参数？ 222 | - 软件强制内存限制？ 223 | 224 | 225 | 226 | ## 7. 文件系统分析 227 | 228 | ### 7.1 文件系统分析工具 229 | 230 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219214347604.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 231 | 232 | ### 7.2 文件系统调优 233 | 234 | - 当前挂载并使用的文件系统数量？ 235 | - 文件系统记录大小？ 236 | - 是否启用访问时间戳？ 237 | - 是否有其他参数（压缩？加密？） 238 | - 缓存的大小？二级缓存？三级缓存？ 239 | - 存储设备数量？配置如何？是否RAID ？ 240 | - 何种文件系统？版本？ 241 | - 是否启用 I/O 控制？ 242 | 243 | ## 8. 磁盘分析 244 | 245 | ### 8.1 磁盘分析工具 246 | 247 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219214445810.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 248 | 249 | ### 8.2 磁盘性能调优 250 | 251 | - 磁盘数量？类型？ 252 | - 磁盘的固件版本？ 253 | - 多少个磁盘控制器？版本？接口类型？ 254 | - 是否为调速插槽？ 255 | - 是否配置了 RAID ? 256 | - 是否启用多路径？ 257 | - 磁盘设备驱动？ 258 | - 是否启用 I/O 控制？ 259 | 260 | 261 | 262 | ## 9. 网络 263 | 264 | 网络通信是由一组协议栈组成的，其中每一层实现一个特定的目标： 265 | 266 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219204937410.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 267 | 268 | 一些网络通信和网络性能相关的概念： 269 | 270 | - 网络和路由 271 | - 协议 272 | - 封装 273 | - 包的长度 274 | - 延时 275 | - DNS 解析延时 276 | - ping 延时 277 | - 连接延时 278 | - 首字节延时 279 | - 往返时间 280 | - 连接生命周期 281 | - 缓冲 282 | - 连接积压队列 283 | - 接口协商 284 | - 使用率 285 | - 本地连接 286 | 287 | 对于日常检测来说，最常用的还是各种工具，比如： 288 | 289 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219214552456.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 290 | 291 | - netstat -s：查找高流量的重新传输和乱序数据包 292 | - nestat -i：检查接口的错误计数器 293 | - ifconfig：检查“错误”，“丢弃”，“超限” 294 | - ip(8)：检查传输和接收的字节率 295 | - tcpdump/snoop：尽管需要大量的 CPU 开销，短期使用可能就足以发现谁在使用网络并定位和消除不必要的网络 I/O 开销 296 | - dtrace / stap /perf：用来检查包括内核状态在内的应用程序与线路间选中的数据 297 | 298 | 299 | 300 | ## 10. 云计算 & 基准测试 301 | 302 | 略。 303 | 304 | ## 11 术语 305 | 306 | ### 11.1 方法 307 | 308 | - IOPS：每秒发生的输入 / 输出操作的次数。是度量数据传输的一个指标，对磁盘的读写，IOPS 指的是每秒读写的次数 309 | - 吞吐量：评价工作执行效率的指标。尤其在数据传输方面，吞吐量用于描述数据传输速度。在某些情况下比如如数据库，吞吐量指的是操作的速度（每秒操作数或每秒业务数） 310 | - 响应事件：一次操作完成的事件。包括用于等待和服务的事件，也包括用来返回结果的时间。 311 | - 延时（latency）：描述操作里用来等待服务的事件。在某些情况下，它可以指整个操作时间，等同于响应时间 312 | - 使用率：对于服务器所请求的资源，使用率描述在所给定的时间区间内，资源使用的繁忙程度。对于存储资源来说，使用率指的是所消耗的存储容量（例如内存使用率） 313 | - 饱和度：指的是某一资源无法满足服务的排队工作量 314 | - 瓶颈：在系统性能力，瓶颈指的是限制系统性能的那个资源。分辨和移除系统瓶颈是性能分析的重要工作 315 | - 工作负载：系统的输入或对系统施加的负载叫做工作负载。对于数据库来说，工作负载指的是客户端发出的数据库请求和命令 316 | - 缓存：用于复制或缓冲一定量数据的高速存储区域。目的是为了避免对较慢的存储层级的直接访问。从而提高性能。 317 | 318 | ### 11.2 操作系统 319 | 320 | - 操作系统：这里指的是安装在系统上的软件和文件，使得系统可以启动和运行程序。操作系统包括内核，管理工具以及系统库 321 | - 内核：内核是管理系统的程序，包括设备（硬件），内核和 CPU 调度。它运行在 CPU 的特权模式，允许直接访问硬件，成为内核态 322 | - 进程：是一个操作系统的抽象概念，是用来执行程序的环境。程序通常运行在用户模式，通过系统调用或陷阱来进入内核模式 323 | - 线程：可被调度的，运行在 CPU 上的可执行上下文。内核有多个线程，一个进程包含一或多个线程 324 | - 任务：一个 Linux 的可运行实体。可以指一个进程（包括单线程进程），一个多线程进程中的一个线程，或内核线程 325 | - 内核空间：内核的内存地址空间 326 | - 用户空间：进程的内存地址空间 327 | - 上下文切换：内核程序切换 CPU ，让其在不同的地址空间上做操作 328 | - 系统调用：一套明确定义的协议，为用户程序请求内核执行特权操作，包括设备 I/O 329 | - 陷阱：信号发送到内核，请求执行一段系统程序（特权操作）。陷阱类型包括系统调用，处理器异常，中断。 330 | - 中断：由物理设备发送给内核的信号，通常是请求 I/O 服务。 331 | 332 | ## 12 USE 法 Linux 检查清单 333 | 334 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219215221376.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 335 | 336 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219215241512.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 337 | 338 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219215259616.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 339 | 340 | 341 | 342 | ### 12.1 软件资源 343 | 344 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/20181219215314746.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70) 345 | 346 | ![在这里插入图片描述](https://img-blog.csdnimg.cn/2018121921532815.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3p1b3pld2Vp,size_16,color_FFFFFF,t_70)sh --------------------------------------------------------------------------------