├── README.md └── successrate.sh /README.md: -------------------------------------------------------------------------------- 1 | # Success Rate Tool 2 | 3 | This script calculates success rates for audits, downloads, uploads and repair traffic on Linux systems. 4 | 5 | Alexey built a similar [script for windows](https://github.com/AlexeyALeonov/success_rate) users. 6 | 7 | ## How to use it 8 | 9 | If your operating system can run _bash shell scripts_ you can run the script directly. Otherwise you can run it with the official [_bash_ docker image](https://hub.docker.com/_/bash). 10 | 11 | The script accepts one optional argument, which is: 12 | 13 | - a) The name of the docker storage node docker container. 14 | - b) The path to a storage node log file. 15 | 16 | When the argument isn't provided, then it defauls to option (a) using _storagenode_ as a container name. 17 | 18 | ### Examples running it directly 19 | 20 | Passing a docker container name: `./successrate.sh storjv3` 21 | 22 | Passing a log file: `./successrate.sh /data/storagenode.log` 23 | 24 | 25 | ### Examples running it through docker 26 | 27 | Unfortunately, running it through a docker container you can only run it with option (b), passing a log file. 28 | 29 | ``` 30 | docker run --rm --mount "type=bind,source=<>,target=/tools,readonly" --mount "type=bind,source=<>,target=/data,readonly" bash /tools/successrate.sh /data/storagenode.log 31 | ``` 32 | 33 | Remember to update the "<< path...>>" place holders in the above instruction to your correct local paths. 34 | 35 | 36 | ## Locale error fix 37 | If you see errors like `./successrate.sh: line 68: printf: 99.8048: invalid number` try running the script with 38 | ``` 39 | LC_ALL=C ./successrate.sh 40 | ``` 41 | 42 | Example output: 43 | ``` 44 | ========== AUDIT ============== 45 | Critically failed: 0 46 | Critical Fail Rate: 0.000% 47 | Recoverable failed: 0 48 | Recoverable Fail Rate: 0.000% 49 | Successful: 14 50 | Success Rate: 100.000% 51 | ========== DOWNLOAD =========== 52 | Failed: 0 53 | Fail Rate: 0.000% 54 | Canceled: 3 55 | Cancel Rate: 1.000% 56 | Successful: 297 57 | Success Rate: 99.000% 58 | ========== UPLOAD ============= 59 | Rejected: 0 60 | Acceptance Rate: 100.000% 61 | ---------- accepted ----------- 62 | Failed: 0 63 | Fail Rate: 0.000% 64 | Canceled: 553 65 | Cancel Rate: 28.417% 66 | Successful: 1393 67 | Success Rate: 71.583% 68 | ========== REPAIR DOWNLOAD ==== 69 | Failed: 0 70 | Fail Rate: 0.000% 71 | Canceled: 0 72 | Cancel Rate: 0.000% 73 | Successful: 0 74 | Success Rate: 0.000% 75 | ========== REPAIR UPLOAD ====== 76 | Failed: 0 77 | Fail Rate: 0.000% 78 | Canceled: 0 79 | Cancel Rate: 0.000% 80 | Successful: 2 81 | Success Rate: 100.000% 82 | ========== DELETE ============= 83 | Failed: 0 84 | Fail Rate: 0.000% 85 | Successful: 145 86 | Success Rate: 100.000% 87 | ``` 88 | -------------------------------------------------------------------------------- /successrate.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | #A StorJ node monitor script: Contains code contributed by: BrightSilence, turbostorjdsk / KernelPanick, Alexey 3 | 4 | LOG_SOURCE="$*" 5 | 6 | if [ -e "${1}" ] 7 | then 8 | # the first argument is passed and it's an existing log file 9 | if [[ "${LOG_SOURCE}" == *.gz ]] 10 | then 11 | LOG="zcat ${LOG_SOURCE}" 12 | else 13 | LOG="cat ${LOG_SOURCE}" 14 | fi 15 | else 16 | # assumes your docker container is named 'storagenode'. If not, pass it as the first argument, e.g.: 17 | # bash successrate.sh mynodename 18 | DOCKER_NODE_NAME="${*:-storagenode}" 19 | LOG="docker logs $DOCKER_NODE_NAME" 20 | fi 21 | 22 | PRINTF=$(which printf) 23 | 24 | #Node Success Rates 25 | echo -e "\e[96m========== AUDIT ============== \e[0m" 26 | #count of successful audits 27 | audit_success=$($LOG 2>&1 | grep GET_AUDIT | grep downloaded -c) 28 | #count of recoverable failed audits 29 | audit_failed_warn=$($LOG 2>&1 | grep GET_AUDIT | grep failed | grep -v exist -c) 30 | #count of unrecoverable failed audits 31 | audit_failed_crit=$($LOG 2>&1 | grep GET_AUDIT | grep failed | grep exist -c) 32 | #Ratio of Successful to Failed Audits 33 | if [ $(($audit_success+$audit_failed_crit+$audit_failed_warn)) -ge 1 ] 34 | then 35 | audit_successrate=$($PRINTF '%.3f\n' $(echo -e "$audit_success $audit_failed_crit $audit_failed_warn" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 36 | else 37 | audit_successrate=0.000% 38 | fi 39 | if [ $(($audit_success+$audit_failed_crit+$audit_failed_warn)) -ge 1 ] 40 | then 41 | audit_recfailrate=$($PRINTF '%.3f\n' $(echo -e "$audit_failed_warn $audit_success $audit_failed_crit" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 42 | else 43 | audit_recfailrate=0.000% 44 | fi 45 | if [ $(($audit_success+$audit_failed_crit+$audit_failed_warn)) -ge 1 ] 46 | then 47 | audit_failrate=$($PRINTF '%.3f\n' $(echo -e "$audit_failed_crit $audit_failed_warn $audit_success" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 48 | else 49 | audit_failrate=0.000% 50 | fi 51 | echo -e "\e[91mCritically failed: $audit_failed_crit \e[0m" 52 | echo -e "Critical Fail Rate: $audit_failrate" 53 | echo -e "\e[33mRecoverable failed: $audit_failed_warn \e[0m" 54 | echo -e "Recoverable Fail Rate: $audit_recfailrate" 55 | echo -e "\e[92mSuccessful: $audit_success \e[0m" 56 | echo -e "Success Rate: $audit_successrate" 57 | 58 | echo -e "\e[96m========== DOWNLOAD =========== \e[0m" 59 | #count of successful downloads 60 | dl_success=$($LOG 2>&1 | grep '"GET"' | grep downloaded -c) 61 | #canceled Downloads from your node 62 | dl_canceled=$($LOG 2>&1 | grep '"GET"' | grep 'download canceled' -c) 63 | #Failed Downloads from your node 64 | dl_failed=$($LOG 2>&1 | grep '"GET"' | grep 'download failed' -c) 65 | #Ratio of canceled Downloads 66 | if [ $(($dl_success+$dl_failed+$dl_canceled)) -ge 1 ] 67 | then 68 | dl_canratio=$($PRINTF '%.3f\n' $(echo -e "$dl_canceled $dl_success $dl_failed" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 69 | else 70 | dl_canratio=0.000% 71 | fi 72 | #Ratio of Failed Downloads 73 | if [ $(($dl_success+$dl_failed+$dl_canceled)) -ge 1 ] 74 | then 75 | dl_failratio=$($PRINTF '%.3f\n' $(echo -e "$dl_failed $dl_success $dl_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 76 | else 77 | dl_failratio=0.000% 78 | fi 79 | #Ratio of Successful Downloads 80 | if [ $(($dl_success+$dl_failed+$dl_canceled)) -ge 1 ] 81 | then 82 | dl_ratio=$($PRINTF '%.3f\n' $(echo -e "$dl_success $dl_failed $dl_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 83 | else 84 | dl_ratio=0.000% 85 | fi 86 | echo -e "\e[91mFailed: $dl_failed \e[0m" 87 | echo -e "Fail Rate: $dl_failratio" 88 | echo -e "\e[33mCanceled: $dl_canceled \e[0m" 89 | echo -e "Cancel Rate: $dl_canratio" 90 | echo -e "\e[92mSuccessful: $dl_success \e[0m" 91 | echo -e "Success Rate: $dl_ratio" 92 | 93 | echo -e "\e[96m========== UPLOAD ============= \e[0m" 94 | #count of successful uploads to your node 95 | put_success=$($LOG 2>&1 | grep '"PUT"' | grep uploaded -c) 96 | #count of rejected uploads to your node 97 | put_rejected=$($LOG 2>&1 | grep 'upload rejected' -c) 98 | #count of canceled uploads to your node 99 | put_canceled=$($LOG 2>&1 | grep '"PUT"' | grep 'upload canceled' -c) 100 | #count of failed uploads to your node 101 | put_failed=$($LOG 2>&1 | grep '"PUT"' | grep 'upload failed' -c) 102 | #Ratio of Rejections 103 | if [ $(($put_success+$put_rejected+$put_canceled+$put_failed)) -ge 1 ] 104 | then 105 | put_accept_ratio=$($PRINTF '%.3f\n' $(echo -e "$put_rejected $put_success $put_canceled $put_failed" | awk '{print ( ($2 + $3 + $4) / ( $1 + $2 + $3 + $4 )) * 100 }'))% 106 | else 107 | put_accept_ratio=0.000% 108 | fi 109 | #Ratio of Failed 110 | if [ $(($put_success+$put_rejected+$put_canceled+$put_failed)) -ge 1 ] 111 | then 112 | put_fail_ratio=$($PRINTF '%.3f\n' $(echo -e "$put_failed $put_success $put_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 113 | else 114 | put_fail_ratio=0.000% 115 | fi 116 | #Ratio of canceled 117 | if [ $(($put_success+$put_rejected+$put_canceled+$put_failed)) -ge 1 ] 118 | then 119 | put_cancel_ratio=$($PRINTF '%.3f\n' $(echo -e "$put_canceled $put_failed $put_success" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 120 | else 121 | put_cancel_ratio=0.000% 122 | fi 123 | #Ratio of Success 124 | if [ $(($put_success+$put_rejected+$put_canceled+$put_failed)) -ge 1 ] 125 | then 126 | put_ratio=$($PRINTF '%.3f\n' $(echo -e "$put_success $put_failed $put_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 127 | else 128 | put_ratio=0.000% 129 | fi 130 | echo -e "\e[33mRejected: $put_rejected \e[0m" 131 | echo -e "Acceptance Rate: $put_accept_ratio" 132 | echo -e "\e[96m---------- accepted ----------- \e[0m" 133 | echo -e "\e[91mFailed: $put_failed \e[0m" 134 | echo -e "Fail Rate: $put_fail_ratio" 135 | echo -e "\e[33mCanceled: $put_canceled \e[0m" 136 | echo -e "Cancel Rate: $put_cancel_ratio" 137 | echo -e "\e[92mSuccessful: $put_success \e[0m" 138 | echo -e "Success Rate: $put_ratio" 139 | 140 | echo -e "\e[96m========== REPAIR DOWNLOAD ==== \e[0m" 141 | #count of successful downloads of pieces for repair process 142 | get_repair_success=$($LOG 2>&1 | grep GET_REPAIR | grep downloaded -c) 143 | #count of failed downloads of pieces for repair process 144 | get_repair_failed=$($LOG 2>&1 | grep GET_REPAIR | grep 'download failed' -c) 145 | #count of canceled downloads of pieces for repair process 146 | get_repair_canceled=$($LOG 2>&1 | grep GET_REPAIR | grep 'download canceled' -c) 147 | #Ratio of Fail GET_REPAIR 148 | if [ $(($get_repair_success+$get_repair_failed+$get_repair_canceled)) -ge 1 ] 149 | then 150 | get_repair_failratio=$($PRINTF '%.3f\n' $(echo -e "$get_repair_failed $get_repair_success $get_repair_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 151 | else 152 | get_repair_failratio=0.000% 153 | fi 154 | #Ratio of Cancel GET_REPAIR 155 | if [ $(($get_repair_success+$get_repair_failed+$get_repair_canceled)) -ge 1 ] 156 | then 157 | get_repair_canratio=$($PRINTF '%.3f\n' $(echo -e "$get_repair_canceled $get_repair_success $get_repair_failed" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 158 | else 159 | get_repair_canratio=0.000% 160 | fi 161 | #Ratio of Success GET_REPAIR 162 | if [ $(($get_repair_success+$get_repair_failed+$get_repair_canceled)) -ge 1 ] 163 | then 164 | get_repair_ratio=$($PRINTF '%.3f\n' $(echo -e "$get_repair_success $get_repair_failed $get_repair_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 165 | else 166 | get_repair_ratio=0.000% 167 | fi 168 | echo -e "\e[91mFailed: $get_repair_failed \e[0m" 169 | echo -e "Fail Rate: $get_repair_failratio" 170 | echo -e "\e[33mCanceled: $get_repair_canceled \e[0m" 171 | echo -e "Cancel Rate: $get_repair_canratio" 172 | echo -e "\e[92mSuccessful: $get_repair_success \e[0m" 173 | echo -e "Success Rate: $get_repair_ratio" 174 | 175 | echo -e "\e[96m========== REPAIR UPLOAD ====== \e[0m" 176 | #count of successful uploads of repaired pieces 177 | put_repair_success=$($LOG 2>&1 | grep PUT_REPAIR | grep uploaded -c) 178 | #count of canceled uploads repaired pieces 179 | put_repair_canceled=$($LOG 2>&1 | grep PUT_REPAIR | grep 'upload canceled' -c) 180 | #count of failed uploads repaired pieces 181 | put_repair_failed=$($LOG 2>&1 | grep PUT_REPAIR | grep 'upload failed' -c) 182 | #Ratio of Fail PUT_REPAIR 183 | if [ $(($put_repair_success+$put_repair_failed+$put_repair_canceled)) -ge 1 ] 184 | then 185 | put_repair_failratio=$($PRINTF '%.3f\n' $(echo -e "$put_repair_failed $put_repair_success $put_repair_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 186 | else 187 | put_repair_failratio=0.000% 188 | fi 189 | #Ratio of Cancel PUT_REPAIR 190 | if [ $(($put_repair_success+$put_repair_failed+$put_repair_canceled)) -ge 1 ] 191 | then 192 | put_repair_canratio=$($PRINTF '%.3f\n' $(echo -e "$put_repair_canceled $put_repair_success $put_repair_failed" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 193 | else 194 | put_repair_canratio=0.000% 195 | fi 196 | #Ratio of Success PUT_REPAIR 197 | if [ $(($put_repair_success+$put_repair_failed+$put_repair_canceled)) -ge 1 ] 198 | then 199 | put_repair_ratio=$($PRINTF '%.3f\n' $(echo -e "$put_repair_success $put_repair_failed $put_repair_canceled" | awk '{print ( $1 / ( $1 + $2 + $3 )) * 100 }'))% 200 | else 201 | put_repair_ratio=0.000% 202 | fi 203 | echo -e "\e[91mFailed: $put_repair_failed \e[0m" 204 | echo -e "Fail Rate: $put_repair_failratio" 205 | echo -e "\e[33mCanceled: $put_repair_canceled \e[0m" 206 | echo -e "Cancel Rate: $put_repair_canratio" 207 | echo -e "\e[92mSuccessful: $put_repair_success \e[0m" 208 | echo -e "Success Rate: $put_repair_ratio" 209 | 210 | echo -e "\e[96m========== DELETE ============= \e[0m" 211 | #count of successful deletes 212 | delete_success=$($LOG 2>&1 | grep -E 'deleted|delete piece' -c) 213 | #count of failed deletes 214 | delete_failed=$($LOG 2>&1 | grep 'delete failed' -c) 215 | #Ratio of Fail delete 216 | if [ $(($delete_success+$delete_failed)) -ge 1 ] 217 | then 218 | delete_failratio=$($PRINTF '%.3f\n' $(echo -e "$delete_failed $delete_success" | awk '{print ( $1 / ( $1 + $2 )) * 100 }'))% 219 | else 220 | delete_failratio=0.000% 221 | fi 222 | #Ratio of Success delete 223 | if [ $(($delete_success+$delete_failed)) -ge 1 ] 224 | then 225 | delete_ratio=$($PRINTF '%.3f\n' $(echo -e "$delete_success $delete_failed" | awk '{print ( $1 / ( $1 + $2 )) * 100 }'))% 226 | else 227 | delete_ratio=0.000% 228 | fi 229 | echo -e "\e[33mFailed: $delete_failed \e[0m" 230 | echo -e "Fail Rate: $delete_failratio" 231 | echo -e "\e[92mSuccessful: $delete_success \e[0m" 232 | echo -e "Success Rate: $delete_ratio" 233 | --------------------------------------------------------------------------------