├── .gitignore ├── README.md ├── backup └── .gitkeep ├── bin ├── analyse.py ├── monitor_apache.py ├── monitor_clean.sh ├── monitor_memcached.py ├── monitor_nginx.py ├── monitor_start.sh ├── monitor_stop.sh ├── monitor_tomcat.py ├── plot.py ├── report.py ├── templates │ ├── base.html │ └── report.html └── util.py ├── conf ├── config.ini └── report.ini ├── example.png ├── init.sh ├── lib └── ukai.ttc ├── log └── .gitkeep ├── requirements.txt ├── result └── .gitkeep └── sample ├── test-v1.0-api_performanceTest_monitor_statistical_data_201511192006.html ├── test-v1.0-api_performanceTest_server_block_201511192006.txt ├── test-v1.0-api_performanceTest_server_cpu_201511192006-iowait.png ├── test-v1.0-api_performanceTest_server_cpu_201511192006-system.png ├── test-v1.0-api_performanceTest_server_cpu_201511192006-used.png ├── test-v1.0-api_performanceTest_server_cpu_201511192006-user.png ├── test-v1.0-api_performanceTest_server_cpu_201511192006.txt ├── test-v1.0-api_performanceTest_server_eth0_201511192006-rx(MB).png ├── test-v1.0-api_performanceTest_server_eth0_201511192006-tx(MB).png ├── test-v1.0-api_performanceTest_server_eth0_201511192006.txt ├── test-v1.0-api_performanceTest_server_inode_201511192006.txt ├── test-v1.0-api_performanceTest_server_io_rate_201511192006-read(MB).png ├── test-v1.0-api_performanceTest_server_io_rate_201511192006-tps.png ├── test-v1.0-api_performanceTest_server_io_rate_201511192006-wrtn(MB).png ├── test-v1.0-api_performanceTest_server_io_rate_201511192006.txt ├── test-v1.0-api_performanceTest_server_memory_201511192006-buffers(MB).png ├── test-v1.0-api_performanceTest_server_memory_201511192006-cached(MB).png ├── test-v1.0-api_performanceTest_server_memory_201511192006-memfree(MB).png ├── test-v1.0-api_performanceTest_server_memory_201511192006-memused(MB).png ├── test-v1.0-api_performanceTest_server_memory_201511192006-memused--.png ├── test-v1.0-api_performanceTest_server_memory_201511192006-memused.png ├── test-v1.0-api_performanceTest_server_memory_201511192006.txt ├── test-v1.0-api_performanceTest_server_network_201511192006.txt ├── test-v1.0-api_performanceTest_server_paging_201511192006.txt ├── test-v1.0-api_performanceTest_server_queue_load_201511192006-ldavg-1.png ├── test-v1.0-api_performanceTest_server_queue_load_201511192006-ldavg-15.png ├── test-v1.0-api_performanceTest_server_queue_load_201511192006-ldavg-5.png ├── test-v1.0-api_performanceTest_server_queue_load_201511192006-plist-sz.png ├── test-v1.0-api_performanceTest_server_queue_load_201511192006.txt ├── test-v1.0-api_performanceTest_server_socket_201511192006-tcpsck.png ├── test-v1.0-api_performanceTest_server_socket_201511192006-totsck.png ├── test-v1.0-api_performanceTest_server_socket_201511192006-udpsck.png ├── test-v1.0-api_performanceTest_server_socket_201511192006.txt └── test-v1.0-api_performanceTest_server_swapping_201511192006.txt /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.out 3 | result/* 4 | venv/* 5 | log/* 6 | backup/* 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Performance Monitor 2 | performance-monitor为开源的服务器资源监控工具,采用shelll + python语言开发,具有灵活的监控配置 3 | 4 | ## 报告样例 5 |  6 | 7 | ## 统计资源 8 | performance monitor用于监控linux服务器中的资源情况,包括: 9 | * server_cpu -- 服务器CPU资源情况(sar) 10 | * `iowait(%) -- iowait占用CPU百分比,一般不应该超过30% ` 11 | * `system(%) -- 内核空间占用CPU百分比` 12 | * `used(%) -- 100%-idle(%)` 13 | * `user(%) -- 用户空间占用CPU百分比` 14 | * server_eth0 -- 服务器网卡eth0资源情况(sar -n DEV) 15 | * `rx/s(MB/s) -- 接收速度` 16 | * `tx/s(MB/s) -- 发送速度` 17 | * server_eth1 -- 服务器网卡eth1资源情况(sar -n DEV) 18 | * `rx/s(MB/s) -- 接收速度` 19 | * `tx/s(MB/s) -- 发送速度` 20 | * server_io_rate -- 服务器IO资源情况(sar -b) 21 | * `read/s(MB) -- 每秒物理设备读取数据总量` 22 | * `wrtn/s(MB) -- 每秒物理设备写入数据总量` 23 | * `tps -- 每秒物理设备IO操作次数(Total number of transfers per second that were issued to physical devices)` 24 | * server_memory -- 服务器内存资源情况(sar -r) 25 | * `buffers(MB) -- 内核缓冲区占用内存大小` 26 | * `cached(MB) -- 内核高速缓存数据占用内存大小` 27 | * `memfree(MB) -- 空闲内存大小` 28 | * `memused(MB) -- 已使用内存大小` 29 | * `memused(%) -- 已使用内存百分比` 30 | * `memused--(%) -- (memused-cached-buffers)*100%/(memfree+memused)` 31 | * server_queue_load -- 服务器CPU负载情况(sar -q) 32 | * `ldavg-1 -- 1分钟load平均值` 33 | * `ldavg-5 -- 5分钟load平均值` 34 | * `ldavg-15 -- 15分钟load平均值` 35 | * `plist-ze -- 队列中的进程和线程数量` 36 | * server_socket -- 服务器socket资源情况(sar -n SOCK) 37 | * `totsck -- 总的socket数量` 38 | * `tcpsck -- tcp连接socket数量` 39 | * `udpsck -- udp连接socket数量` 40 | * process_xxx -- 进程占用资源情况(top, ps -mP) 41 | * `CPU(%) -- 占用CPU百分比(每个核占用百分比之和)` 42 | * `MEM(%) -- 占用物理内存百分比` 43 | * `Threads -- 进程对应线程数` 44 | * mysql -- mysql数据库连接数(show global status like 'Threads_connected';) 45 | * `Threads_connected ` 46 | * redis -- redis信息(redis info) 47 | * `connected_clients -- 客户端连接数` 48 | * `hit_rate -- 命中率(keyspace_hits*100%/(keyspace_hits+keyspace_misses))` 49 | * `instantaneous_ops_per_sec -- 每秒处理请求数量` 50 | * `keyspace_hits -- 总命中数量` 51 | * `keyspace_misses -- 总不命中数量` 52 | * `total_commands_processed -- 总的请求数量` 53 | * `use_memory(MB) -- 使用内存大小` 54 | * `use_memory_peak(MB) -- 使用的最大内存大小` 55 | * memcached --memcached状态(client.get_stats()) 56 | * `accepting_conns -- 接收的请求数` 57 | * `bytes -- 处理的字节数量` 58 | * `bytes_read -- 读操作的字节数量` 59 | * `bytes_written -- 写操作的字节数量` 60 | * `cmd_flush -- flush命令总请求数量` 61 | * `cmd_get -- get命令总请求数量` 62 | * `cmd_set -- set命令总请求数量` 63 | * `curr_connections -- 当前打开这的连接数` 64 | * `curr_items -- 当前存储的items数量` 65 | * `evictions -- 为了获取空闲内存而删除的items数` 66 | * `get_hits -- 总命中次数` 67 | * `get_misses -- 总不命中次数` 68 | * `hit_rate -- 命中率(get_hits*100%/(get_hits+get_misses))` 69 | * `limit_maxbytes -- 分配给memcached内存大小` 70 | * `threads -- 当前线程数` 71 | * `total_items -- 从服务启动以后存储的items总数量` 72 | * mongodb -- mongodb信息(mongostat) 73 | * `conn -- 当前连接数` 74 | * `delete -- 每秒删除次数` 75 | * `faults -- 每秒访问失败数` 76 | * `flushes -- 每秒执行fsync将数据写入硬盘的次数` 77 | * `getmore -- 每秒执行getmore次数` 78 | * `insert -- 每秒插入次数` 79 | * `mapped(MB) -- 所有的被mmap的数据量` 80 | * `non-mapped(MB) -- MongoDB's internal data structures and threads'stacks, essentially anything not backed by files on disk` 81 | * `query -- 每秒查询数量` 82 | * `res(MB) -- 物理内存使用量` 83 | * `update -- 每秒更新次数` 84 | * `vsize(MB) -- 虚拟内存使用量` 85 | * apache --apache状态(status页面) 86 | * `closing_connection -- 关闭连接状态(C)` 87 | * `currently_processed -- 近期处理的请求数量` 88 | * `dns_lookup -- 正在查找DNS状态(D)` 89 | * `gracefully_finishing -- 进入正常结束程序中状态(G)` 90 | * `idle_cleanup_of_worker -- 处理闲置状态(I)` 91 | * `idle_worker -- 空闲线程` 92 | * `keepalive_read -- 处于保持联机的状态(K)` 93 | * `logging -- 正在写入日志文件` 94 | * `open_slot_with_no_current_process` 95 | * `reading_request -- 正在读取请求状态(R)` 96 | * `sending_reply -- 正在发送回应(W)` 97 | * `starting_up -- 启动中(S)` 98 | * `waiting_for_connection -- 等待连接中(_)` 99 | * tomcat --tomcat状态(status页面) 100 | * `free_memory -- 空闲内存` 101 | * `total_memory -- 总的使用内存` 102 | * `%ps_eden_space -- 年轻代(Eden Space)使用百分比` 103 | * `%ps_old_gen -- 年老代使用百分比` 104 | * `%ps_survivor_space -- 年轻代(Survivor Space)使用百分比` 105 | * `max_threads -- 最大线程数量` 106 | * `current_thread_count -- 近期线程数量` 107 | * `current_thread_busy -- 近期忙线程数量` 108 | * `max_processing_time -- 最大处理时间` 109 | * `processing_time -- 处理时间` 110 | * nginx -- nginx状态(status页面) 111 | * `active_connections -- 激活的连接数` 112 | * `handled_connections -- 处理的连接数` 113 | * `handled_handshake -- 处理的握手数` 114 | * `handled_requests -- 处理的请求数` 115 | * `reading -- 读取操作` 116 | * `waiting -- 等待操作` 117 | * `writing -- 写入操作` 118 | 119 | --- 120 | 121 | ## 安装步骤 122 | 123 | - apt-get install build-essential 124 | - apt-get install libssl-dev 125 | - 自行安装python2.7+,setuptools和pip 126 | - apt-get install libfreetype* 127 | - apt-get install libpng* 128 | - apt-get install sysstat 129 | - apt-get install libmysqld-dev 130 | - apt-get install zip 131 | - pip2.7 install virtualenv (可选,若需要在python虚拟环境中运行,则需安装) 132 | - pip2.7 install -r requirements.txt 133 | 134 | --- 135 | 136 | ## 使用方法 137 | 138 | 配置config目录下的config.ini和report.ini 139 | 140 | cd performance_monitor/bin 141 | 142 | ./monitor_start.sh -i interval -c count [-t time] -f prefix_name 143 | 144 | 参数说明: 145 | -i 采样间隔 146 | -c 采样次数 147 | -t 采样时长(如果你使用了-t设置时长 ,脚本将会忽略-c采样次数) 148 | -f 数据文件名前缀 例子: test-v1.0-api (使用'-'作为分隔符,请勿使用'_' ) 149 | 结束后可到result目录中查看对应的输出表单和数据 150 | 151 | --- 152 | 153 | -------------------------------------------------------------------------------- /backup/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JeffXue/performance-monitor/f9f7e29885c0a4a21a78664654d3abf86d03a590/backup/.gitkeep -------------------------------------------------------------------------------- /bin/analyse.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | import sys 3 | import platform 4 | import ConfigParser 5 | import util 6 | import plot 7 | import report 8 | 9 | 10 | class MonitorConfig(): 11 | """get config from the ini file""" 12 | 13 | def __init__(self, config_ini): 14 | self.config_file = config_ini 15 | config = ConfigParser.ConfigParser() 16 | with open(self.config_file, "r") as cfg_file: 17 | config.readfp(cfg_file) 18 | self.res_dir = config.get("common", "resDir") 19 | self.granularity = int(config.get("plot", "granularity")) 20 | self.server_cpu_types = config.get("plot", "serverCPU").split(",") 21 | self.server_memory_types = config.get("plot", "serverMemory").split(",") 22 | self.server_io_rate_types = config.get("plot", "serverIORate").split(",") 23 | self.server_load_types = config.get("plot", "serverQueueLoad").split(",") 24 | self.server_sock_types = config.get("plot", "serverSock").split(",") 25 | server_platform = platform.platform() 26 | if server_platform.find("Ubuntu") != -1: 27 | self.server_eth_types = config.get("plot", "ubuntuEth").split(",") 28 | if server_platform.find("debian") != -1: 29 | self.server_eth_types = config.get("plot", "ubuntuEth").split(",") 30 | if server_platform.find("centos") != -1: 31 | self.server_eth_types = config.get("plot", "ubuntuEth").split(",") 32 | if server_platform.find("redhat") != -1: 33 | self.server_eth_types = config.get("plot", "redhatEth").split(",") 34 | self.mysql_connections_types = config.get("plot", "mysql").split(",") 35 | self.tcp_port_types = config.get("plot", "TCPPort").split(",") 36 | self.process_types = config.get("plot", "processStatus").split(",") 37 | self.redis_types = config.get("plot", "redisStatus").split(",") 38 | self.memcached_types = config.get("plot", "memcachedStatus").split(",") 39 | self.mongodb_types = config.get("plot", "mongodbStatus").split(",") 40 | self.apache_types = config.get("plot", 'apacheStatus').split(",") 41 | self.tomcat7_types = config.get("plot", 'tomcat7Status').split(",") 42 | self.tomcat6_types = config.get("plot", 'tomcat6Status').split(",") 43 | self.nginx_types = config.get("plot", 'nginxStatus').split(",") 44 | self.socket_stat_types = config.get("plot", 'socketStat').split(",") 45 | 46 | 47 | def main(): 48 | # get the avg 49 | parameter_lists = util.get_parameter_lists(sys.argv) 50 | if len(parameter_lists): 51 | result_prefix = parameter_lists[0] 52 | else: 53 | result_prefix = "test" 54 | if len(parameter_lists) > 1: 55 | end_time = parameter_lists[1] 56 | else: 57 | end_time = "N/A" 58 | 59 | # read the monitor config 60 | config = MonitorConfig("../conf/report.ini") 61 | 62 | # analyse file 63 | files = util.get_dir_files(config.res_dir) 64 | for datafile in files: 65 | if datafile.find("txt") != -1: 66 | if datafile.find("server_cpu") != -1: 67 | cpu_resource = plot.CPUResource(config.server_cpu_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 68 | cpu_resource.work() 69 | if datafile.find("server_memory") != -1: 70 | memory_resource = plot.MemoryResource(config.server_memory_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 71 | memory_resource.work() 72 | if datafile.find("server_io_rate") != -1: 73 | io_rate_resource = plot.IOResource(config.server_io_rate_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 74 | io_rate_resource.work() 75 | if datafile.find("server_eth0") != -1: 76 | eth0_resource = plot.EthResource(config.server_eth_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 77 | eth0_resource.work() 78 | if datafile.find("server_eth1") != -1: 79 | eth1_resource = plot.EthResource(config.server_eth_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 80 | eth1_resource.work() 81 | if datafile.find("server_queue_load") != -1: 82 | load_resource = plot.LoadResource(config.server_load_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 83 | load_resource.work() 84 | if datafile.find("server_socket") != -1: 85 | sock_resource = plot.SockResource(config.server_sock_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 86 | sock_resource.work() 87 | if datafile.find("mysql") != -1 and datafile.find('threads') != -1: 88 | mysql_resource = plot.MySQLResource(config.mysql_connections_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 89 | mysql_resource.work() 90 | if datafile.find("TCPPort") != -1: 91 | tcp_port_resource = plot.TCPPortResource(config.tcp_port_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 92 | tcp_port_resource.work() 93 | if datafile.find("process") != -1 and datafile.find('mysql') == -1: 94 | process_resource = plot.ProcessResource(config.process_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 95 | process_resource.work() 96 | if datafile.find("redis") != -1: 97 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 98 | redis_resource = plot.RedisResource(config.redis_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 99 | redis_resource.work() 100 | if datafile.find('memcached') != -1: 101 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 102 | memcached_resource = plot.MemcachedResource(config.memcached_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 103 | memcached_resource.work() 104 | if datafile.find('mongodb') != -1: 105 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 106 | mongodb_resource = plot.MongodbResource(config.mongodb_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 107 | mongodb_resource.work() 108 | if datafile.find('apache') != -1: 109 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 110 | apache_resource = plot.ApacheResource(config.apache_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 111 | apache_resource.work() 112 | if datafile.find('tomcat6') != -1: 113 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 114 | tomcat_resource = plot.TomcatResource(config.tomcat6_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 115 | tomcat_resource.work() 116 | if datafile.find('tomcat7') != -1: 117 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 118 | tomcat_resource = plot.TomcatResource(config.tomcat7_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 119 | tomcat_resource.work() 120 | if datafile.find('nginx') != -1: 121 | if datafile.find('process') == -1 and datafile.find('thread') == -1: 122 | nginx_resource = plot.TomcatResource(config.nginx_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 123 | nginx_resource.work() 124 | if datafile.find('SocketStat') != -1: 125 | socket_stat_resource = plot.SocketStatResource(config.socket_stat_types, result_prefix, config.res_dir+"/"+datafile, config.granularity) 126 | socket_stat_resource.work() 127 | 128 | # generate sum report 129 | report.Report.end_time = end_time 130 | resource_sum_report = report.Report(config.res_dir) 131 | resource_sum_report.work() 132 | 133 | if __name__ == "__main__": 134 | main() 135 | 136 | 137 | 138 | 139 | -------------------------------------------------------------------------------- /bin/monitor_apache.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | import sys 3 | import time 4 | import requests 5 | 6 | from util import get_parameter_lists 7 | 8 | 9 | class MonitorApache(): 10 | 11 | def __init__(self, url, interval, end_time, filename, user, password): 12 | self.url = url 13 | self.interval = int(interval) 14 | self.end_time = float(end_time) 15 | self.filename = filename 16 | self.user = user 17 | self.password = password 18 | 19 | def work(self): 20 | while time.time() < self.end_time: 21 | if self.user and self.password: 22 | r = requests.get(self.url, auth=(self.user, self.password)) 23 | else: 24 | r = requests.get(self.url) 25 | if r.status_code == 200: 26 | currently_processed = r.text.split('requests currently being processed')[0].split('
')[1].split('')[0].replace('\n', '') 29 | waiting_for_connection = str(key.count('_')) 30 | starting_up = str(key.count('S')) 31 | reading_request = str(key.count('R')) 32 | sending_reply = str(key.count('W')) 33 | keepalive_read = str(key.count('K')) 34 | dns_lookup = str(key.count('D')) 35 | closing_connection = str(key.count('C')) 36 | logging = str(key.count('L')) 37 | gracefully_finishing = str(key.count('G')) 38 | idle_cleanup_of_worker = str(key.count('I')) 39 | open_slot_with_no_current_process = str(key.count('.')) 40 | stats_time = time.strftime('%H:%M:%S %p', time.localtime(time.time())) 41 | stats_list = [stats_time, currently_processed, idle_worker, waiting_for_connection, starting_up, 42 | reading_request, sending_reply, keepalive_read, dns_lookup, closing_connection, 43 | logging, gracefully_finishing, idle_cleanup_of_worker, open_slot_with_no_current_process] 44 | with open(self.filename, 'a') as f: 45 | f.write(' '.join(stats_list)+'\n') 46 | time.sleep(self.interval) 47 | 48 | 49 | def main(): 50 | parameters = get_parameter_lists(sys.argv) 51 | if len(parameters) == 4: 52 | url = parameters[0] 53 | interval = parameters[1] 54 | end_time = parameters[2] 55 | filename = parameters[3] 56 | user = None 57 | password = None 58 | if len(parameters) == 6: 59 | url = parameters[0] 60 | interval = parameters[1] 61 | end_time = parameters[2] 62 | filename = parameters[3] 63 | user = parameters[4] 64 | password = parameters[5] 65 | monitor_apache_status = MonitorApache(url, interval, end_time, filename, user, password) 66 | monitor_apache_status.work() 67 | 68 | if __name__ == "__main__": 69 | main() -------------------------------------------------------------------------------- /bin/monitor_clean.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | #get the config 4 | source ../conf/config.ini 5 | 6 | systemType=`uname` 7 | case $systemType in 8 | Linux) 9 | files=`ls -lt $resDir | wc -l` 10 | 11 | if [ $files -gt 1 ];then 12 | newDir=`cat $startTimeRecord` 13 | mkdir $backupDir/$newDir 14 | mv $resDir/* $backupDir/$newDir 15 | 16 | fi 17 | 18 | echo "--------------------------------------------------" 19 | echo "|Finish Backup ! |" 20 | echo "--------------------------------------------------" 21 | exit 0 22 | ;; 23 | *) 24 | echo Unknow platform,please add this platform monitor into the script. 25 | exit 0 26 | ;; 27 | esac 28 | exit 0 29 | -------------------------------------------------------------------------------- /bin/monitor_memcached.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | 3 | import sys 4 | import time 5 | import bmemcached 6 | 7 | from util import get_parameter_lists 8 | 9 | 10 | class MonitorMemcachedStat: 11 | 12 | def __init__(self, ip, port, interval, end_time, filename, user, passwd): 13 | if user != '' and passwd != '': 14 | self.client = bmemcached.Client(('%s:%s' % (ip, port),), user, passwd) 15 | else: 16 | self.client = bmemcached.Client(('%s:%s' % (ip, port),)) 17 | self.ip = ip 18 | self.port = port 19 | self.interval = int(interval) 20 | self.end_time = float(end_time) 21 | self.filename = filename 22 | 23 | def work(self): 24 | while time.time() < self.end_time: 25 | stats = self.client.stats()['%s:%s' % (self.ip, self.port)] 26 | curr_connections = stats.get('curr_connections') 27 | cmd_get = stats.get('cmd_get') 28 | cmd_set = stats.get('cmd_set') 29 | cmd_flush = stats.get('cmd_flush') 30 | get_hits = stats.get('get_hits') 31 | get_misses = stats.get('get_misses') 32 | get_total = float(int(get_hits)+int(get_misses)) 33 | if int(get_hits) != 0 or int(get_total) != 0: 34 | hit_rate = '%0.2f' % float(float(int(get_hits)*100)/get_total) 35 | else: 36 | hit_rate = '0' 37 | bytes_read = stats.get('bytes_read') 38 | bytes_written = stats.get('bytes_written') 39 | limit_maxbytes = stats.get('limit_maxbytes') 40 | accepting_conns = stats.get('accepting_conns') 41 | threads = stats.get('threads') 42 | bytes = stats.get('bytes') 43 | curr_items = stats.get('curr_items') 44 | total_items = stats.get('total_items') 45 | evictions = stats.get('evictions') 46 | stats_time = time.strftime('%H:%M:%S %p', time.localtime(time.time())) 47 | stats_list = [stats_time, curr_connections, cmd_get, cmd_set, cmd_flush, get_hits, 48 | get_misses, str(hit_rate), bytes_read, bytes_written, limit_maxbytes, 49 | accepting_conns, threads, bytes, curr_items, total_items, evictions] 50 | with open(self.filename, 'a') as f: 51 | f.write(' '.join(stats_list)+'\n') 52 | time.sleep(self.interval) 53 | 54 | 55 | def main(): 56 | parameters = get_parameter_lists(sys.argv) 57 | if len(parameters) == 5: 58 | ip = parameters[0] 59 | port = parameters[1] 60 | interval = parameters[2] 61 | end_time = parameters[3] 62 | filename = parameters[4] 63 | user = '' 64 | passwd = '' 65 | monitor_memcached_stat = MonitorMemcachedStat(ip, port, interval, end_time, filename, user, passwd) 66 | monitor_memcached_stat.work() 67 | if len(parameters) == 7: 68 | ip = parameters[0] 69 | port = parameters[1] 70 | interval = parameters[2] 71 | end_time = parameters[3] 72 | filename = parameters[4] 73 | user = parameters[5] 74 | passwd = parameters[6] 75 | monitor_memcached_stat = MonitorMemcachedStat(ip, port, interval, end_time, filename, user, passwd) 76 | monitor_memcached_stat.work() 77 | 78 | if __name__ == "__main__": 79 | main() 80 | -------------------------------------------------------------------------------- /bin/monitor_nginx.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | import sys 3 | import time 4 | import requests 5 | 6 | from util import get_parameter_lists 7 | 8 | 9 | class MonitorNginx(): 10 | 11 | def __init__(self, url, interval, end_time, filename): 12 | self.url = url 13 | self.interval = int(interval) 14 | self.end_time = float(end_time) 15 | self.filename = filename 16 | 17 | def work(self): 18 | while time.time() < self.end_time: 19 | r = requests.get(self.url) 20 | if r.status_code == 200: 21 | active_connections = r.text.split('Active connections:')[-1].split()[0] 22 | handled_connections = r.text.split('server accepts handled requests')[-1].split()[0] 23 | handled_handshake = r.text.split('server accepts handled requests')[-1].split()[1] 24 | handled_requests = r.text.split('server accepts handled requests')[-1].split()[2] 25 | reading = r.text.split('Reading:')[-1].split()[0] 26 | writing = r.text.split('Writing:')[-1].split()[0] 27 | waiting = r.text.split('Waiting:')[-1].split()[0] 28 | stats_time = time.strftime('%H:%M:%S %p', time.localtime(time.time())) 29 | stats_list = [stats_time, active_connections, handled_connections, handled_handshake, handled_requests, 30 | reading, writing, waiting] 31 | with open(self.filename, 'a') as f: 32 | f.write(' '.join(stats_list)+'\n') 33 | time.sleep(self.interval) 34 | 35 | 36 | def main(): 37 | parameters = get_parameter_lists(sys.argv) 38 | if len(parameters) == 4: 39 | url = parameters[0] 40 | interval = parameters[1] 41 | end_time = parameters[2] 42 | filename = parameters[3] 43 | 44 | monitor_nginx_status = MonitorNginx(url, interval, end_time, filename) 45 | monitor_nginx_status.work() 46 | 47 | if __name__ == "__main__": 48 | main() -------------------------------------------------------------------------------- /bin/monitor_start.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | #导入配置参数 4 | source ../conf/config.ini 5 | 6 | #记录PID号 7 | echo $$ > $pidRecord 8 | 9 | function envError(){ 10 | echo "---------------------Error------------------------" 11 | echo "--------------------------------------------------" 12 | echo "|performance_monitor can not start ... |" 13 | echo "|make sure system has installed: |" 14 | echo "|1.sysstat |" 15 | echo "|2.python |" 16 | echo "|3.python-numpy |" 17 | echo "|4.python-Matplotlib |" 18 | echo "|5.python-memcached |" 19 | echo "|6.python-requests |" 20 | echo "|7.MySQL-python |" 21 | echo "--------------------------------------------------" 22 | exit 1 23 | } 24 | 25 | #判断是否已安装sar和python 26 | if [ ! -f "/usr/bin/sar" ];then 27 | envError 28 | elif [ ! -f "/usr/bin/python" ];then 29 | envError 30 | fi 31 | 32 | #操作记录 33 | echo "`date`: $0 $@ " >> $historyLog 34 | 35 | function paraError(){ 36 | echo "-----------------------------Error-----------------------------------" 37 | echo "---------------------------------------------------------------------" 38 | echo "Version: $version " 39 | echo "parameter error " 40 | echo "./monitor_start.sh -i interval -c count [-t time] -f prefix_name " 41 | echo "parameter: " 42 | echo " -i monitoring interval " 43 | echo " -c monitoring count " 44 | echo " -t monitoring time(minute) " 45 | echo " (If you set the -t ,the script ignore the -c parameter) " 46 | echo " -f the prefix of the recording file " 47 | echo " format: test-v1.0-dir (use '-' for separator, not '_' ) " 48 | echo "---------------------------------------------------------------------" 49 | exit 1 50 | } 51 | 52 | #解析输入参数 53 | while getopts :i:c:t:f: args 54 | do 55 | case $args in 56 | i)interval=$OPTARG 57 | ;; 58 | c)count=$OPTARG 59 | ;; 60 | t)monitorTime=$OPTARG 61 | ;; 62 | f)prefix=$OPTARG 63 | ;; 64 | \?)paraError 65 | exit; 66 | esac 67 | done 68 | 69 | #判断传入参数是否正确 70 | if [ -z $interval ];then 71 | paraError 72 | fi 73 | if [ -z $prefix ];then 74 | paraError 75 | fi 76 | if [[ -z $count && -z $monitorTime ]];then 77 | paraError 78 | fi 79 | 80 | #-f 传入参数不能带下划线_ 81 | hostname=`hostname | sed 's/\//_/'` 82 | fJudge=$(echo $prefix | grep "_") 83 | if [[ "$fJudge" != "" ]];then 84 | paraError 85 | else 86 | filename=$prefix"_"$hostname 87 | fi 88 | 89 | #若使用-t指定时长,将忽略-c参数 90 | if [ ! -z $monitorTime ];then 91 | echo "using -t parameter, ignore -c parameter" 92 | count=$(($monitorTime*60/$interval)) 93 | fi 94 | 95 | time=`date +%Y%m%d%H%M` 96 | formatted_start_time=`date '+%Y-%m-%d %H:%M:%S'` 97 | 98 | echo "--------------------------------------------------" 99 | echo "|performance_monitor is going to start ... |" 100 | echo "--------------------------------------------------" 101 | ################################################# 102 | #backup the old data 103 | ################################################# 104 | echo "--------------------------------------------------" 105 | echo "|Auto Backup the old result |" 106 | echo "--------------------------------------------------" 107 | ./monitor_clean.sh 108 | 109 | echo $filename"_"$time > $startTimeRecord 110 | startTime=`date +%s` 111 | endTime=$(($startTime+($count*$interval))) 112 | 113 | #监控进程时用于输出top监控菜单文件头 114 | function monitorProcessTitle(){ 115 | if [ $1 -eq 1 ];then 116 | title=`top -n 1 -b | grep "PID USER"` 117 | echo `date +%H:%M:%S` `date +%P` $title Threads>> $resDir/$filename"_process_"$2"_"$time.txt 118 | else 119 | echo "[ERROR]Cat not monitor the $2, there are several pids of $2. monitor will exit" 120 | exit 0 121 | fi 122 | } 123 | 124 | #监控进程时用于输出监控数据,来源top数据和ps -mP 125 | function monitorProcessData(){ 126 | mpPsData=$(eval `eval echo '$'"processCommand$mpCount"`) 127 | mpPID=`echo $mpPsData |awk '{print $2}'` 128 | mpPIDNumber=`echo $mpPsData |awk '{print $2}' |wc -l` 129 | mpTopName=$(eval echo '$'"processTopName$mpCount") 130 | mpSelfName=$(eval echo '$'"processSelfName$mpCount") 131 | if [ $mpPIDNumber -eq 1 ];then 132 | processData=`top -p $mpPID -n 1 -b |grep -w $mpTopName ` 133 | threadData=`ps -mP $mpPID |wc -l` 134 | if [[ "$processData" != "" ]];then 135 | echo `date +%H:%M:%S` `date +%P` $processData $threadData >> $resDir/$filename"_process_"$mpSelfName"_"$time.txt 136 | fi 137 | else 138 | exit 0 139 | fi 140 | } 141 | 142 | #监控进程占用资源函数 143 | function monitorProcess(){ 144 | processNumber=$(($processNumber+1)) 145 | 146 | #check the pid whether correct and create the file to record 147 | mpCount=1 148 | while [ $mpCount -lt $processNumber ] 149 | do 150 | mpPsData=$(eval `eval echo '$'"processCommand$mpCount"`) 151 | mpPID=`echo $mpPsData |awk '{print $2}'` 152 | mpPIDNumber=`echo $mpPsData |awk '{print $2}' |wc -l` 153 | mpTopName=$(eval echo '$'"processTopName$mpCount") 154 | mpSelfName=$(eval echo '$'"processSelfName$mpCount") 155 | monitorProcessTitle $mpPIDNumber $mpSelfName 156 | mpCount=$(($mpCount+1)) 157 | done 158 | 159 | #monitor the pid 160 | nowTime=`date +%s` 161 | while [ $nowTime -lt $endTime ] 162 | do 163 | mpCount=1 164 | while [ $mpCount -lt $processNumber ] 165 | do 166 | monitorProcessData $mpCount & 167 | mpCount=$(($mpCount+1)) 168 | done 169 | sleep $interval 170 | nowTime=`date +%s` 171 | done 172 | } 173 | 174 | 175 | function dumpMysqlSlowLog(){ 176 | $mysqlPath -h $mysqlIP -P $mysqlPort -u$mysqlUser -p$mysqlPassword -e"select * from mysql.slow_log where start_time >= '$formatted_start_time'" > $resDir/$filename"_mysql_slow_log_"$time.txt 177 | } 178 | 179 | function dumpMysqlProcessList(){ 180 | $mysqlPath -h $mysqlIP -P $mysqlPort -u$mysqlUser -p$mysqlPassword -e"SELECT * FROM information_schema.processlist" > $resDir/$filename"_mysql_process_list_"$time.txt 181 | } 182 | 183 | function killMysqlProcessList(){ 184 | killSql=`$mysqlPath -h $mysqlIP -P $mysqlPort -u$mysqlUser -p$mysqlPassword -e"SELECT CONCAT('KILL ',id,';') FROM information_schema.processlist WHERE Command='Query'" | grep -v CONCAT` 185 | $mysqlPath -h $mysqlIP -P $mysqlPort -u$mysqlUser -p$mysqlPassword mysql -e"$killSql" 186 | } 187 | 188 | 189 | function dumpMysqlInnodeLock(){ 190 | $mysqlPath -h $mysqlIP -P $mysqlPort -u$mysqlUser -p$mysqlPassword mysql -e"SELECT a.*, b.* FROM information_schema.INNODB_LOCK_WAITS a JOIN information_schema.INNODB_TRX c ON a.blocking_trx_id = c.trx_id JOIN information_schema.PROCESSLIST b ON c.trx_mysql_thread_id = b.ID ORDER BY TIME DESC ;" >> $resDir/$filename"_mysql_innode_lock_"$time.txt 191 | } 192 | 193 | #监控mysql线程函数 194 | function monitorMysql(){ 195 | echo `date +%H:%M:%S` `date +%P` Threads_connected > $resDir/$filename"_mysql_threads_"$time.txt 196 | nowTime=`date +%s` 197 | while [ $nowTime -lt $endTime ] 198 | do 199 | threads=`$mysqlPath -h $mysqlIP -P $mysqlPort -u$mysqlUser -p$mysqlPassword $mysqlDatabase -e"show global status like 'Threads_connected';" | grep Threads_connected |awk '{print $2}'` 200 | if [ ! -z $threads ];then 201 | echo `date +%H:%M:%S` `date +%P` $threads >> $resDir/$filename"_mysql_threads_"$time.txt 202 | fi 203 | sleep $interval 204 | nowTime=`date +%s` 205 | 206 | if [ $mysqlInnodeLockWaitsFlag -eq 1 ];then 207 | dumpMysqlInnodeLock 208 | fi 209 | done 210 | dumpMysqlSlowLog 211 | dumpMysqlProcessList 212 | killMysqlProcessList 213 | } 214 | 215 | 216 | 217 | #监控端口稳定连接数函数 218 | function monitorNetstat(){ 219 | echo `date +%H:%M:%S` `date +%P` ESTABLISHED > $resDir/$filename"_TCPPort_"$netstatPort"_"$time.txt 220 | nowTime=`date +%s` 221 | while [ $nowTime -lt $endTime ] 222 | do 223 | netstatData=`netstat -lan |grep ":$netstatPort" |grep ESTABLISHED |wc -l` 224 | echo `date +%H:%M:%S` `date +%P` $netstatData >> $resDir/$filename"_TCPPort_"$netstatPort"_"$time.txt 225 | sleep $interval 226 | nowTime=`date +%s` 227 | done 228 | } 229 | 230 | #监控socket不同状态数量函数 231 | function monitorSocketStat(){ 232 | echo `date +%H:%M:%S` `date +%P` LISTEN SYN-RECV ESTAB CLOSE-WAIT LAST_ACK FIN-WAIT-1 FIN-WAIT-2 CLOSING TIME_WAIT > $resDir/$filename"_SocketStat_"$time.txt 233 | nowTime=`date +%s` 234 | while [ $nowTime -lt $endTime ] 235 | do 236 | ss -t -a |awk '{print $1}' |sort | uniq -c |sed 's/^[ \t]*//g' > socket_stat.txt 237 | listenNum=`grep LISTEN socket_stat.txt| awk '{print $1}'` 238 | synRecvNum=`grep SYN-RECV socket_stat.txt| awk '{print $1}'` 239 | estabNum=`grep ESTAB socket_stat.txt| awk '{print $1}'` 240 | closeWaitNum=`grep CLOSE-WAIT socket_stat.txt| awk '{print $1}'` 241 | lastAckNum=`grep LAST_ACK socket_stat.txt| awk '{print $1}'` 242 | finWait1Num=`grep FIN-WAIT-1 socket_stat.txt| awk '{print $1}'` 243 | finWait2Num=`grep FIN-WAIT-2 socket_stat.txt| awk '{print $1}'` 244 | closingNum=`grep CLOSING socket_stat.txt| awk '{print $1}'` 245 | timeWaitNum=`grep TIME_WAIT socket_stat.txt| awk '{print $1}'` 246 | 247 | if [ -z $listenNum ];then 248 | listenNum=0 249 | fi 250 | if [ -z $synRecvNum ];then 251 | synRecvNum=0 252 | fi 253 | if [ -z $estabNum ];then 254 | estabNum=0 255 | fi 256 | if [ -z $closeWaitNum ];then 257 | closeWaitNum=0 258 | fi 259 | if [ -z $lastAckNum ];then 260 | lastAckNum=0 261 | fi 262 | if [ -z $finWait1Num ];then 263 | finWait1Num=0 264 | fi 265 | if [ -z $finWait2Num ];then 266 | finWait2Num=0 267 | fi 268 | if [ -z $closingNum ];then 269 | closingNum=0 270 | fi 271 | if [ -z $timeWaitNum ];then 272 | timeWaitNum=0 273 | fi 274 | 275 | echo `date +%H:%M:%S` `date +%P` $listenNum $synRecvNum $estabNum $closeWaitNum $lastAckNum $finWait1Num $finWait2Num $closingNum $timeWaitNum >> $resDir/$filename"_SocketStat_"$time.txt 276 | sleep $interval 277 | nowTime=`date +%s` 278 | done 279 | rm -rf socket_stat.txt 280 | } 281 | 282 | 283 | #监控redis info函数 284 | function monitorRedis(){ 285 | echo `date +%H:%M:%S` `date +%P` connected_clients used_memory used_memory_peak total_commands_processed keyspace_hits keyspace_misses hit_rate instantaneous_ops_per_sec > $resDir/$filename"_redis_"$time.txt 286 | nowTime=`date +%s` 287 | while [ $nowTime -lt $endTime ] 288 | do 289 | if [ -z $redisPassword ];then 290 | $redisPath/redis-cli -h $redisIP -p $redisPort info > redis_info.txt 291 | else 292 | $redisPath/redis-cli -h $redisIP -p $redisPort -a $redisPassword info > redis_info.txt 293 | fi 294 | connectedClients=`grep connected_clients redis_info.txt|cut -f2 -d':'|sed 's/\r//g'` 295 | usedMemory=`grep used_memory: redis_info.txt|cut -f2 -d":" |sed 's/\r//g'` 296 | usedMemoryPeak=`grep used_memory_peak: redis_info.txt|cut -f2 -d":" |sed 's/\r//g'` 297 | totalCommandsProcessed=`grep total_commands_processed redis_info.txt|cut -f2 -d":" |sed 's/\r//g'` 298 | keyspaceHits=`grep keyspace_hits redis_info.txt|cut -f2 -d":" |sed 's/\r//g'` 299 | keyspaceMisses=`grep keyspace_misses redis_info.txt|cut -f2 -d":" |sed 's/\r//g'` 300 | instantaneousOpsPerSec=`grep instantaneous_ops_per_sec redis_info.txt|cut -f2 -d":" |sed 's/\r//g'` 301 | 302 | oldRecord=`tail -1 $resDir/$filename"_redis_"$time.txt |grep -v connected` 303 | if [ ! -z "$oldRecord" ];then 304 | oldKeyspaceHits=`echo $oldRecord |cut -f7 -d" "` 305 | oldKeyspaceMisses=`echo $oldRecord |cut -f8 -d" "` 306 | realKeyspaceHits=$(($keyspaceHits-$oldKeyspaceHits)) 307 | realKeyspaceMisses=$(($keyspaceMisses-$oldKeyspaceMisses)) 308 | else 309 | realKeyspaceHits=$keyspaceHits 310 | realKeyspaceMisses=$keyspaceMisses 311 | fi 312 | keyspace=$(($realKeyspaceHits+$realKeyspaceMisses)) 313 | if [ $realKeyspaceHits -eq 0 -o $keyspace -eq 0 ];then 314 | hitRate=0 315 | else 316 | hitRate=$(($realKeyspaceHits*100/$keyspace)) 317 | fi 318 | echo `date +%H:%M:%S` `date +%P` $connectedClients $usedMemory $usedMemoryPeak $totalCommandsProcessed $keyspaceHits $keyspaceMisses $hitRate $instantaneousOpsPerSec >> $resDir/$filename"_redis_"$time.txt 319 | sleep $interval 320 | nowTime=`date +%s` 321 | done 322 | rm -rf redis_info.txt 323 | } 324 | 325 | #监控memcached stat函数 326 | function monitorMemcached(){ 327 | echo `date +%H:%M:%S` `date +%P` curr_connections cmd_get cmd_set cmd_flush get_hits get_misses hit_rate bytes_read bytes_written limit_maxbytes accepting_conns threads bytes curr_items total_items evictions > $resDir/$filename"_memcached_"$time.txt 328 | python monitor_memcached.py $memcachedIP $memcachedPort $interval $endTime $resDir/$filename"_memcached_"$time.txt $memcachedUser $memcachedPasswd 329 | } 330 | 331 | #监控mongodb stat函数 332 | function monitorMongoDB(){ 333 | echo `date +%H:%M:%S` `date +%P` `$mongodbPath/mongostat -h $mongodbIP --port $mongodbPort -u $mongodbUser -p $mongodbPassword --authenticationDatabase $authenticationDatabase --all -n 1 1 |grep -w insert |grep -v connected` > $resDir/$filename"_mongodb_"$time.txt 334 | nowTime=`date +%s` 335 | while [ $nowTime -lt $endTime ] 336 | do 337 | echo `date +%H:%M:%S` `date +%P` `$mongodbPath/mongostat -h $mongodbIP --port $mongodbPort -u $mongodbUser -p $mongodbPassword --authenticationDatabase $authenticationDatabase --noheaders --all -n 1 |grep -v connected` >> $resDir/$filename"_mongodb_"$time.txt & 338 | sleep $interval 339 | nowTime=`date +%s` 340 | done 341 | } 342 | 343 | #监控apache状态页面函数 344 | function monitorApache(){ 345 | echo `date +%H:%M:%S` `date +%P` currently_processed idle_worker waiting_for_connection starting_up reading_request sending_reply keepalive_read dns_lookup closing_connection logging gracefully_finishing idle_cleanup_of_worker open_slot_with_no_current_process > $resDir/$filename"_apache_"$time.txt 346 | python monitor_apache.py $apacheURL $interval $endTime $resDir/$filename"_apache_"$time.txt $apacheUser $apachePassword 347 | } 348 | 349 | #监控tomcat状态页面函数 350 | function monitorTomcat(){ 351 | tomcatNumber=$(($tomcatNumber+1)) 352 | mtCount=1 353 | while [ $mtCount -lt $tomcatNumber ] 354 | do 355 | tomcatType=$(eval echo '$'"tomcatType$mtCount") 356 | tomcatURL=$(eval echo '$'"tomcatURL$mtCount") 357 | tomcatMonitorSign=$(eval echo '$'"tomcatMonitorSign$mtCount") 358 | tomcatUser=$(eval echo '$'"tomcatUser$mtCount") 359 | tomcatPassword=$(eval echo '$'"tomcatPassword$mtCount") 360 | tomcatSelfName=$(eval echo '$'"tomcatSelfName$mtCount") 361 | if [[ "$tomcatType" == "tomcat6" ]];then 362 | tomcatResFile=$resDir/$filename"_tomcat6_"$tomcatSelfName"_"$time.txt 363 | echo `date +%H:%M:%S` `date +%P` free_memory total_memory max_threads current_thread_count current_thread_busy max_processing_time processing_time > $tomcatResFile 364 | python monitor_tomcat.py $tomcatURL $interval $endTime $tomcatResFile $tomcatMonitorSign $tomcatType $tomcatSelfName $tomcatUser $tomcatPassword & 365 | fi 366 | if [[ "$tomcatType" == "tomcat7" ]];then 367 | tomcatResFile=$resDir/$filename"_tomcat7_"$tomcatSelfName"_"$time.txt 368 | echo `date +%H:%M:%S` `date +%P` free_memory total_memory %ps_eden_space %ps_old_gen %ps_survivor_space max_threads current_thread_count current_thread_busy max_processing_time processing_time > $tomcatResFile 369 | python monitor_tomcat.py $tomcatURL $interval $endTime $tomcatResFile $tomcatMonitorSign $tomcatType $tomcatSelfName $tomcatUser $tomcatPassword & 370 | fi 371 | mtCount=$(($mtCount+1)) 372 | done 373 | } 374 | 375 | #监控nginx状态页面函数 376 | function monitorNginx(){ 377 | echo `date +%H:%M:%S` `date +%P` active_connections handled_connections handled_handshake handled_requests reading writing waiting > $resDir/$filename"_nginx_"$time.txt 378 | python monitor_nginx.py $nginxURL $interval $endTime $resDir/$filename"_nginx_"$time.txt 379 | } 380 | 381 | systemType=`uname` 382 | case $systemType in 383 | Linux) 384 | echo "--------------------------------------------------" 385 | echo "|Starting Linux monitor...... |" 386 | echo "--------------------------------------------------" 387 | echo "please don't kill the monitor process which running in the background" 388 | 389 | ############################################# 390 | #monitor common resource 391 | ############################################# 392 | if [ $serverSourceFlag -eq 1 ];then 393 | sar $interval $count > $resDir/$filename"_server_cpu_"$time.txt & 394 | sar $interval $count -r > $resDir/$filename"_server_memory_"$time.txt & 395 | sar -B $interval $count > $resDir/$filename"_server_paging_"$time.txt & 396 | sar -dp $interval $count > $resDir/$filename"_server_block_"$time.txt & 397 | sar -n DEV $interval $count > $resDir/$filename"_server_network_"$time.txt & 398 | sar -n SOCK $interval $count > $resDir/$filename"_server_socket_"$time.txt & 399 | sar -W $interval $count > $resDir/$filename"_server_swapping_"$time.txt & 400 | sar -b $interval $count > $resDir/$filename"_server_io_rate_"$time.txt & 401 | sar -v $interval $count > $resDir/$filename"_server_inode_"$time.txt & 402 | sar -q $interval $count > $resDir/$filename"_server_queue_load_"$time.txt & 403 | fi 404 | 405 | if [ $netstatFlag -eq 1 ];then 406 | monitorNetstat & 407 | fi 408 | 409 | if [ $socketFlag -eq 1 ];then 410 | monitorSocketStat & 411 | fi 412 | 413 | if [ $processFlag -eq 1 ];then 414 | monitorProcess & 415 | fi 416 | 417 | if [ $mysqlFlag -eq 1 ];then 418 | monitorMysql & 419 | fi 420 | 421 | if [ $redisFlag -eq 1 ];then 422 | monitorRedis & 423 | fi 424 | 425 | if [ $memcachedFlag -eq 1 ];then 426 | monitorMemcached & 427 | fi 428 | 429 | if [ $mongodbFlag -eq 1 ];then 430 | monitorMongoDB & 431 | fi 432 | 433 | if [ $apacheFlag -eq 1 ];then 434 | monitorApache & 435 | fi 436 | 437 | if [ $tomcatFlag -eq 1 ];then 438 | monitorTomcat & 439 | fi 440 | 441 | if [ $nginxFlag -eq 1 ];then 442 | monitorNginx & 443 | fi 444 | ############################################# 445 | #wait for monitor finish 446 | ############################################# 447 | delay=$(($interval*$count)) 448 | sleep $delay 449 | runFlag=1 450 | while [ $runFlag -eq 1 ] 451 | do 452 | sleep $interval 453 | checkFlag=`ps -ef |grep -w $$ |grep -v grep |wc -l` 454 | if [ $checkFlag -lt 3 ];then 455 | runFlag=0 456 | else 457 | runFlag=1 458 | fi 459 | done 460 | 461 | 462 | ############################################# 463 | #split the network information into eth0/1 464 | ############################################# 465 | if [ $serverSourceFlag -eq 1 ];then 466 | if [ ! -f $interface1 ];then 467 | cat $resDir/$filename"_server_network_"$time.txt | head -3 > $resDir/$interface1.txt 468 | cat $resDir/$filename"_server_network_"$time.txt | grep -w $interface1 >> $resDir/$interface1.txt 469 | mv $resDir/$interface1.txt $resDir/$filename"_server_"$interface1"_"$time.txt 470 | fi 471 | 472 | if [ ! -f $interface2 ];then 473 | cat $resDir/$filename"_server_network_"$time.txt | head -3 > $resDir/$interface2.txt 474 | cat $resDir/$filename"_server_network_"$time.txt | grep -w $interface2 >> $resDir/$interface2.txt 475 | mv $resDir/$interface2.txt $resDir/$filename"_server_"$interface2"_"$time.txt 476 | fi 477 | fi 478 | 479 | 480 | ############################################# 481 | #analyse data 482 | ############################################# 483 | echo "Start analyse the data" 484 | endTime=`date +%Y%m%d%H%M` 485 | python analyse.py $filename $endTime 486 | 487 | echo "Finish analyse the data" 488 | echo "Check the data and graph in result directory" 489 | echo "Finish monitor at `date`" >> $historyLog 490 | exit 0 491 | ;; 492 | *) 493 | echo Unknow platform,please add this platform monitor into the script. 494 | exit 0 495 | ;; 496 | 497 | esac 498 | exit 0 499 | 500 | -------------------------------------------------------------------------------- /bin/monitor_stop.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | #get the config 4 | source ../conf/config.ini 5 | 6 | systemType=`uname` 7 | case $systemType in 8 | Linux) 9 | PID=`cat $pidRecord` 10 | echo $PID 11 | PIDS=`ps -ef |grep $PID |grep -v grep |awk '{print $2}'` 12 | echo $PIDS 13 | PIDSNumber=`ps -ef |grep $PID |grep -v grep |awk '{print $2}'|wc -l` 14 | while [ $PIDSNumber -gt 0 ] 15 | do 16 | kill -9 $PIDS 17 | PIDSNumber=`ps -ef |grep $PID |grep -v grep |awk '{print $2}'|wc -l` 18 | done 19 | echo The performance monitor have stopped! 20 | exit 0 21 | ;; 22 | *) 23 | echo Unknow platform,please add this platform monitor into the script. 24 | exit 0 25 | ;; 26 | esac 27 | exit 0 28 | -------------------------------------------------------------------------------- /bin/monitor_tomcat.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | import sys 3 | import time 4 | import requests 5 | 6 | from util import get_parameter_lists 7 | 8 | 9 | class MonitorTomcat(): 10 | 11 | def __init__(self, url, interval, end_time, filename, monitor_sign, tomcat_type, tomcat_self_name, user, password): 12 | self.url = url 13 | self.interval = int(interval) 14 | self.end_time = float(end_time) 15 | self.filename = filename 16 | self.monitor_sign = monitor_sign 17 | self.tomcat_type = tomcat_type 18 | self.tomcat_self_name = tomcat_self_name 19 | self.user = user 20 | self.password = password 21 | 22 | def work(self): 23 | while time.time() < self.end_time: 24 | if self.user and self.password: 25 | r = requests.get(self.url, auth=(self.user, self.password)) 26 | else: 27 | r = requests.get(self.url) 28 | if r.status_code == 200: 29 | free_memory = r.text.split('Free memory:')[-1].split('Total memory:')[0].strip().replace(' ', '') 30 | total_memory = r.text.split('Total memory:')[-1].split('Max memory:')[0].strip().replace(' ', '') 31 | if self.tomcat_type == 'tomcat6': 32 | pass 33 | elif self.tomcat_type == 'tomcat7': 34 | ps_eden_space = r.text.split('PS Eden Space')[-1].split(r'%')[0].split('(')[-1] 35 | ps_old_gen = r.text.split('PS Old Gen')[-1].split(r'%')[0].split('(')[-1] 36 | ps_survivor_space = r.text.split('PS Survivor Space')[-1].split(r'%')[0].split('(')[-1] 37 | split_sign = self.monitor_sign 38 | max_threads = r.text.split(split_sign)[-1].split('Max threads:')[1].split('Current thread count:')[0].strip() 39 | current_thread_count = r.text.split(split_sign)[-1].split('Current thread count:')[1].split('Current thread busy:')[0].strip() 40 | current_thread_busy = r.text.split(split_sign)[-1].split('Current thread busy:')[1].split('
服务器系统信息
7 |监控时间段 | 10 |{{ duration }} | 11 |
---|---|
主机名 | 14 |{{ system_information['hostname'] }} | 15 |
内核版本 | 18 |{{ system_information['kernel'] }} | 19 |
CPU | 22 |{{ system_information['cpuinfo'] }} | 23 |
内存 | 26 |{{ system_information['meminfo'] }} | 27 |
服务器资源使用情况汇总 ({{ datafile_prefix }})
32 |Item | 35 |Type | 36 |Min | 37 |Max | 38 |Avg | 39 |90%小于 | 40 |
---|---|---|---|---|---|
{{ item }} | 46 |{{ sub_type }} | 47 |{{ sum_value[0] }} | 48 |{{ sum_value[1] }} | 49 |{{ sum_value[2] }} | 50 |{{ sum_value[3] }} | 51 |
{{ sub_type }} | 55 |{{ sum_value[0] }} | 56 |{{ sum_value[1] }} | 57 |{{ sum_value[2] }} | 58 |{{ sum_value[3] }} | 59 |
服务器系统信息
21 |监控时间段 | 24 |2015-11-19_20:06~2015-11-19_20:08 | 25 |
---|---|
主机名 | 28 |performanceTest | 29 |
内核版本 | 32 |Linux version 2.6.32-5-amd64 | 33 |
CPU | 36 |2 Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz | 37 |
内存 | 40 |4048224 kB | 41 |
服务器资源使用情况汇总 (test-v1.0-api)
46 |Item | 49 |Type | 50 |Min | 51 |Max | 52 |Avg | 53 |90%小于 | 54 |
---|---|---|---|---|---|
server_cpu | 58 |iowait(%) | 59 |0.0 | 60 |7.5 | 61 |0.07 | 62 |0.0 | 63 |
system(%) | 67 |12.5 | 68 |21.11 | 69 |16.42 | 70 |18.09 | 71 ||
used(%) | 75 |13.43 | 76 |26.34 | 77 |17.34 | 78 |19.31 | 79 ||
user(%) | 83 |0.0 | 84 |8.06 | 85 |0.85 | 86 |1.98 | 87 ||
server_eth0 | 91 |rx/s(MB) | 92 |0.0 | 93 |0.0 | 94 |0.0 | 95 |0.0 | 96 |
tx/s(MB) | 100 |0.0 | 101 |0.01 | 102 |0.0 | 103 |0.01 | 104 ||
server_io_rate | 108 |read/s(MB) | 109 |0.0 | 110 |0.0 | 111 |0.0 | 112 |0.0 | 113 |
tps | 117 |0.0 | 118 |43.0 | 119 |4.24 | 120 |15.0 | 121 ||
wrtn/s(MB) | 125 |0.0 | 126 |0.2 | 127 |0.02 | 128 |0.09 | 129 ||
server_memory | 133 |memused(%) | 134 |97.66 | 135 |97.69 | 136 |97.67 | 137 |97.68 | 138 |
memused--(%) | 142 |82.48 | 143 |82.5 | 144 |82.49 | 145 |82.5 | 146 ||
buffers(MB) | 150 |35.62 | 151 |35.82 | 152 |35.72 | 153 |35.81 | 154 ||
cached(MB) | 158 |564.32 | 159 |564.43 | 160 |564.37 | 161 |564.41 | 162 ||
memfree(MB) | 166 |91.51 | 167 |92.48 | 168 |91.94 | 169 |92.18 | 170 ||
memused(MB) | 174 |3860.86 | 175 |3861.84 | 176 |3861.4 | 177 |3861.74 | 178 ||
server_queue_load | 182 |ldavg-1 | 183 |0.02 | 184 |0.19 | 185 |0.08 | 186 |0.15 | 187 |
ldavg-15 | 191 |0.01 | 192 |0.02 | 193 |0.01 | 194 |0.02 | 195 ||
ldavg-5 | 199 |0.03 | 200 |0.06 | 201 |0.05 | 202 |0.05 | 203 ||
plist-sz | 207 |623.0 | 208 |633.0 | 209 |632.92 | 210 |633.0 | 211 ||
server_socket | 215 |tcpsck | 216 |38.0 | 217 |38.0 | 218 |38.0 | 219 |38.0 | 220 |
totsck | 224 |573.0 | 225 |579.0 | 226 |578.95 | 227 |579.0 | 228 ||
udpsck | 232 |6.0 | 233 |6.0 | 234 |6.0 | 235 |6.0 | 236 |