├── .gitignore ├── LICENSE ├── README.md ├── data └── .gitkeep ├── py_src ├── plot_results.py ├── start_tests.py ├── test_cpu.py ├── test_disks.py ├── test_networks.py ├── test_pi_cam.py └── test_ram.py └── setup └── install_tool.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.csv 2 | *.png 3 | *.json -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 NASA/Sam Pedrotty 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # System Monitor for Radiation Testing (SMRT) 2 | 3 | SMRT is a Python®-based software package intended to monitor computer system performance during radiation testing. 4 | 5 | ---- 6 | 7 | **Table of Contents** 8 | 9 | [[_TOC_]] 10 | 11 | ---- 12 | 13 | # Purpose 14 | 15 | This tool is intended to provide a comprehensive, cross-platform (Windows®/Linux®, Intel®/ARM®, etc.) test suite geared towards assessing computer performance as they go through radiation testing. It's hoped that this will be widely adopted by the aerospace avionics community, creating a common standard for radiation survivability of computers (especially commercial-off-the-shelf single board computers) and that the community will continually improve this tool. 16 | 17 | # Questions/Comments/Concerns? 18 | 19 | The best way to get these things addressed is to open an issue on GitHub® and/or email the developer (samuel.m.pedrotty@nasa.gov). Don't hesitate to reach out-- it's likely others also have your questions/comments/concerns. We encourage users to add capability and fix bugs and then submit merge/pull requests to have their improvements brought back to the repository for all to benefit from. 20 | 21 | # Basic Use 22 | 23 | 1. move the repository to the target computer 24 | 2. install Python 3 25 | 3. run the install script in the "setup" directory 26 | 4. run the start_tests.py script in the py_src directory 27 | * the test scripts will start assessing the system and logging data to a folder named with the start time in the "data" folder 28 | 5. stop the script or shutdown the SBC to terminate the software 29 | 30 | # Data Analysis 31 | 1. after running the tool, run the "plot_results.py" script in the "py_src" folder to load, concatenate, and plot the data. 32 | * in some cases, it may be best to transfer the data to a faster computer for analysis and plotting 33 | 34 | # Detailed Use 35 | 36 | It's assumed that all of the interfaces of the computer under test will be connected appropriately to allow their monitoring (e.g. displays, ethernet, USB devices, etc.). 37 | 38 | ## Pre-test 39 | 40 | 1. appropriately install the Operating System on the computer (this software should support Linux and Windows) 41 | 2. move the repository to the computer 42 | 3. install Python 3 43 | 4. run the install script in the "setup" directory 44 | 5. update the user input section of the "start_tests.py" script in the "py_src" directory 45 | * set the *ram_pct_to_use*, the RAM usage parameter with units of percents (< 95% is recommended to provide the OS some headroom) 46 | * set the *data_save_interval*, the parameter that determines the time between data file saving with units of seconds (if a high particle flux is to be used, an interval of < 3 seconds is recommended) 47 | * set the *test_cycle_time*, the parameter that sets the run rate of the test scripts with units of seconds (0.1 is recommended. Note that some tests will run slower than the rate as their functions take a set time that may be larger than some user-input cycle times) 48 | 6. configure the computer as it will be when under test (software and hardware) 49 | 7. run the "start_tests.py" script in the "py_src" directory for a couple minutes to verify functionality 50 | * it's recommended to run the script as 'sudo' on Linux and as administrator on Windows 51 | 8. transfer the data to a faster computer that also has Python 3 installed and run the "plot_results.py" script to verify the tool (and computer) functioned as intended 52 | 9. delete the pre-test data from the computer to save disk space, keep the set that was moved onto the faster computer as a pre-test artifact 53 | 54 | ## Test use 55 | 56 | 10. appropriately deploy the computer in the test environment, connecting all interfaces as briefly mentioned above 57 | 11. run the "start_tests.py" script in the "py_src" directory and wait until you see the terminal print stating RAM allocation is complete, and that the cpu, disk, and network monitoring functions are operating 58 | 12. start the radiation test (e.g., start the particle flow and other test support equipment) 59 | 13. stop the test and/or make note when any off nominal behavior is observed, including unresponsive interfaces, crashed software, or prints stating off-nominal conditions were detected (including RAM changes, disk disconnections, and network adapter disconnections) 60 | * noting the flux/fluence and failure type will provide insight into the MTBF of that computer subsystem and will inform fault recovery and mitigation design 61 | 14. when the test is complete, assuming the device is still functional, transfer the data off and save it as a test artifact 62 | 15. plot the data with "plot_results.py" script in the "py_src" directory and review the data for other indicators of faults to inform MTBF analysis 63 | 64 | 65 | # Detailed Function Description 66 | 67 | ## install_tool.py 68 | 69 | This script automatically installs all of the necessary software requirements for Windows and Linux platforms after Python 3 is installed by the user. The packages required for data plotting are optional and user-prompted as they can be memory-intensive for small, cheap, computers with limited RAM. 70 | 71 | ## start_tests.py 72 | 73 | This script starts all of the test scripts and provides a regular, 1 Hz print to the terminal to provide a "heartbeat" to indicate if the computer/interface is still functioning. This script also contains the aforementioned user inputs. 74 | 75 | ## test_ram.py 76 | 77 | This script starts writing 1s to a struct until a user-defined amount of RAM is consumed. Each cycle, the script will check the struct to make sure it's consistent. If it's not, it will log an 'upset' and will print *RAM STATE CHANGE DETECTED*. The RAM will not be overwritten to reset it. Sometimes restarting the script can reset upsets, but sometimes the whole device must be restarted. At other times, especially after the unit under test has been subject to a considerable particle flux, upsets may trigger after restarting even when the particle beam is off. The script also records the percentage of RAM used. 78 | 79 | ## test_cpu.py 80 | 81 | This script records the percentage of CPU used, current frequency in MHz, and CPU temperature in C (temperature is Linux-only). 82 | 83 | ## test_disks.py 84 | 85 | This script records the number of disks/drives detected and will print *NUMBER OF DISKS HAS CHANGED!* if a change in the number of drives is detected. This script also records some information regarding the drives. 86 | 87 | ## test_networks.py 88 | 89 | This script records the number of network adapters detected and will print *NUMBER OF NETWORK ADAPTERS HAS CHANGED!* if a change in the number of adapters is detected. This script also records some information regarding the adapters. 90 | 91 | ## plot_results.py 92 | 93 | This script concatenates and loads all data in the most recently created data folder. It then plots the data and, depending on a user-set flag, can also save the plots and pngs. 94 | 95 | # Known Issues and Forward Work 96 | 1. The CPU temperature code only works on Linux operating systems. Windows CPU temp values are hard-coded to 9999 97 | 2. The test_disks and test_networks scripts leave much to the user to parse in post-processing. This should be intelligently handled by the tool. 98 | 3. The CPU frequency code only works on Linux operating systems. Windows CPU frequency values are fixed to the system's base clock speed. 99 | 100 | -------------------------------------------------------------------------------- /data/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nasa/System_Monitor_for_Radiation_Testing/af3461e07632104be7296bca76f76855c92be4e6/data/.gitkeep -------------------------------------------------------------------------------- /py_src/plot_results.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | plot_results.py 6 | 7 | This script is intended to plot 8 | the data captured by the test 9 | scripts 10 | ''' 11 | 12 | ############################################################# 13 | #IMPORT MODULES 14 | ############################################################# 15 | import os 16 | import csv 17 | import sys 18 | import time 19 | import pandas as pd 20 | import matplotlib.pyplot as plt 21 | 22 | 23 | ############################################################# 24 | # USER INPUT 25 | ############################################################# 26 | title_prefix = 'Rad Test ' 27 | plot_prefix = 'rad_test_' 28 | save_plot = True 29 | 30 | 31 | ############################################################# 32 | # SUPPORT FUNCTIONS 33 | ############################################################# 34 | 35 | def find_and_load_data(keyword): 36 | 37 | print("\nLoading: " + keyword) 38 | 39 | # list data 40 | dir_list = os.listdir('../data') 41 | dir_list.sort() 42 | if len(dir_list) > 1: 43 | try: dir_list.remove('demo') 44 | except: pass 45 | data_dir = dir_list[-1] 46 | 47 | print("Using folder: " + str(os.path.join('..','data',data_dir))) 48 | data_file_list = os.listdir(os.path.join('..','data',data_dir)) 49 | 50 | the_files = [] 51 | for filename in data_file_list: 52 | if keyword in filename: the_files+=[filename] 53 | 54 | the_data = [] 55 | for filename in the_files: 56 | df = pd.read_csv(os.path.join('..','data',data_dir,filename)) 57 | the_data+= [df] 58 | 59 | result = pd.concat(the_data) 60 | result.sort_values(by=['time'], inplace=True) 61 | 62 | return result 63 | 64 | 65 | ############################################################# 66 | # MAIN CODE 67 | ############################################################# 68 | 69 | # Load and concatenate data 70 | try: 71 | ram_data = find_and_load_data("ram") 72 | ram_data_status = True 73 | except: 74 | ram_data_status = False 75 | print('Error: failed to load RAM data!') 76 | try: 77 | net_data = find_and_load_data("net") 78 | net_data_status = True 79 | except: 80 | net_data_status = False 81 | print('Error: failed to load network data!') 82 | try: 83 | cpu_data = find_and_load_data("cpu") 84 | cpu_data_status = True 85 | except: 86 | cpu_data_status = False 87 | print('Error: failed to load CPU data!') 88 | try: 89 | disk_data = find_and_load_data("disk") 90 | disk_data_status = True 91 | except: 92 | disk_data_status = False 93 | print('Error: failed to load disk data!') 94 | 95 | 96 | # Plot data 97 | n=0 98 | 99 | if ram_data_status: 100 | #Plot RAM upsets and use 101 | n+=1 102 | plt.figure(n) 103 | ax1=plt.subplot(211) 104 | plt.title(title_prefix+'RAM Upsets and % Use') 105 | plt.plot(ram_data['time'], ram_data['ram_pct_used'], '-o', markersize=3) 106 | plt.ylabel('% Used') 107 | 108 | plt.subplot(212, sharex=ax1) 109 | plt.plot(ram_data['time'], ram_data['upsets'], '-o', markersize=3) 110 | plt.ylabel('Upsets') 111 | plt.xlabel('System time (s)') 112 | if save_plot == True: 113 | plt.savefig(plot_prefix+'ram_combined.png') 114 | 115 | if cpu_data_status: 116 | #Plot CPU use and temp 117 | n+=1 118 | plt.figure(n) 119 | ax1=plt.subplot(311) 120 | plt.plot(cpu_data['time'], cpu_data['cpu_pct_used'], '-o', markersize=3) 121 | plt.ylabel('% Used') 122 | plt.title(title_prefix+'CPU Temp and % Use') 123 | 124 | plt.subplot(312, sharex=ax1) 125 | plt.plot(cpu_data['time'], cpu_data['cpu_freq'], '-o', markersize=3) 126 | plt.ylabel('Freq (MHz)') 127 | 128 | plt.subplot(313, sharex=ax1) 129 | plt.plot(cpu_data['time'], cpu_data['cpu_temp'], '-o', markersize=3) 130 | plt.ylabel('Temp (C)') 131 | plt.xlabel('System time (s)') 132 | if save_plot == True: 133 | plt.savefig(plot_prefix+'cpu_combined.png') 134 | 135 | if disk_data_status: 136 | #Plot num disks 137 | n+=1 138 | plt.figure(n) 139 | ax1=plt.subplot(111) 140 | plt.plot(disk_data['time'], disk_data['num_detected_disks'], '-o', markersize=3) 141 | plt.ylabel('#') 142 | plt.title(title_prefix+'Number of Disks Detected') 143 | plt.xlabel('System time (s)') 144 | if save_plot == True: 145 | plt.savefig(plot_prefix+'disk_combined.png') 146 | 147 | 148 | if net_data_status: 149 | #Plot num adapters 150 | n+=1 151 | plt.figure(n) 152 | ax1=plt.subplot(111) 153 | plt.plot(net_data['time'], net_data['num_detected_adapters'], '-o', markersize=3) 154 | plt.ylabel('#') 155 | plt.title(title_prefix+'Number of Network Adapters Detected') 156 | plt.xlabel('System time (s)') 157 | if save_plot == True: 158 | plt.savefig(plot_prefix+'net_combined.png') 159 | 160 | 161 | 162 | plt.show() #show plots 163 | 164 | 165 | 166 | 167 | 168 | -------------------------------------------------------------------------------- /py_src/start_tests.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | start_tests.py 6 | 7 | This script is intended to trigger and monitor 8 | scripts related to radiation testing 9 | ''' 10 | 11 | ############################################################# 12 | #IMPORT MODULES 13 | ############################################################# 14 | import os 15 | import sys 16 | import time 17 | import json 18 | import subprocess 19 | from datetime import datetime 20 | 21 | 22 | ############################################################# 23 | # USER INPUT 24 | ############################################################# 25 | ram_pct_to_use = 85 #%, RAM to be consumed and monitored by test program 26 | test_cycle_time = 0.1 #seconds, delay between system data checks 27 | data_save_interval = 5 #seconds, length of time of each data file 28 | 29 | 30 | ############################################################# 31 | # MAIN CODE 32 | ############################################################# 33 | 34 | #create directory for data 35 | data_dir = '../data' 36 | if os.path.exists(data_dir): 37 | pass 38 | else: 39 | os.mkdir(data_dir) 40 | 41 | init_time = str(datetime.now()) 42 | init_time = init_time .split('.') 43 | init_time = init_time [0] 44 | init_time = init_time .replace(' ','_') 45 | init_time = init_time .replace(':','-') 46 | the_dir = os.path.join(data_dir, init_time ) 47 | os.mkdir(the_dir) 48 | 49 | #save off parameters that the test scripts use 50 | input_data = {'ram_pct_to_use':ram_pct_to_use, 'test_cycle_time': test_cycle_time, 'data_save_interval':data_save_interval} 51 | 52 | with open('data.json', 'w', encoding='utf-8') as f: 53 | json.dump(input_data, f, ensure_ascii=False, indent=4) 54 | 55 | with open(os.path.join(the_dir,'data.json'), 'w', encoding='utf-8') as f: 56 | json.dump(input_data, f, ensure_ascii=False, indent=4) 57 | 58 | 59 | #save off the code state 60 | home = os.getcwd() 61 | os.chdir(os.path.dirname(os.path.abspath(__file__))) 62 | git_show = str(subprocess.check_output(["git", "show"])) 63 | git_status = str(subprocess.check_output(["git", "status"])) 64 | git_dict = {'time':init_time,'git show':git_show,'git status':git_status} 65 | 66 | with open(os.path.join(the_dir,'git_status.json'),"w") as statfile: 67 | json.dump(git_dict, statfile) 68 | 69 | os.chdir(home) 70 | 71 | #start logging processes 72 | subprocess.Popen([sys.executable,os.path.abspath('test_ram.py'),str(the_dir)], stdin=None, stdout=None, stderr=None) 73 | time.sleep(1) 74 | 75 | subprocess.Popen([sys.executable,os.path.abspath('test_cpu.py'),str(the_dir)], stdin=None, stdout=None, stderr=None) 76 | time.sleep(1) 77 | 78 | subprocess.Popen([sys.executable,os.path.abspath('test_disks.py'),str(the_dir)], stdin=None, stdout=None, stderr=None) 79 | time.sleep(1) 80 | 81 | subprocess.Popen([sys.executable,os.path.abspath('test_networks.py'),str(the_dir)], stdin=None, stdout=None, stderr=None) 82 | time.sleep(1) 83 | 84 | run_cams = False #adds the ability to test attached cameras. Feature incomplete. 85 | if 'linux' in sys.platform: 86 | if run_cams == True: 87 | subprocess.Popen([sys.executable,os.path.abspath('test_pi_cam.py'),str(the_dir)], stdin=None, stdout=None, stderr=None) 88 | time.sleep(1) 89 | 90 | 91 | while True: 92 | time.sleep(1) 93 | print("%.2f (heartbeat)" % time.time())#+' (heartbeat)') 94 | 95 | 96 | 97 | -------------------------------------------------------------------------------- /py_src/test_cpu.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | test_cpu.py 6 | 7 | This script is intended to 8 | monitor CPU usage 9 | ''' 10 | 11 | ############################################################# 12 | #IMPORT MODULES 13 | ############################################################# 14 | import os 15 | import sys 16 | import csv 17 | import time 18 | import json 19 | import psutil 20 | from datetime import datetime 21 | 22 | 23 | ############################################################# 24 | # SUPPORT FUNCTIONS 25 | ############################################################# 26 | def cpu_test(data_dirname): 27 | 28 | #load inputs 29 | with open('data.json') as f: 30 | data = json.load(f) 31 | data_save_interval = data['data_save_interval'] 32 | test_cycle_time = data['test_cycle_time'] 33 | 34 | #define vars 35 | ttime=[] 36 | upsets=[] 37 | cpu_pct_used=[] 38 | cpu_temp=[] 39 | cpu_freq=[] 40 | 41 | print(str(time.time()) + ': starting CPU monitor!') 42 | 43 | #one day we might have user-configurable CPU usage here 44 | 45 | start = time.time() 46 | while True: 47 | 48 | time.sleep(test_cycle_time) 49 | end = time.time() 50 | 51 | ttime+=[time.time()] 52 | cpu_pct_used+=[psutil.cpu_percent()] 53 | cpu_freq+=[psutil.cpu_freq(percpu=False)[0]] #NOTE that as of 12/2020 this is the rated val on windows, not current 54 | 55 | 56 | if 'linux' in sys.platform: 57 | cpu_temp+=[psutil.sensors_temperatures()['cpu-thermal'][0][1]] 58 | else: 59 | cpu_temp+=[9999] #TODO figure out how to do this on Windows 60 | 61 | 62 | if end-start > data_save_interval: 63 | time1 = time.time() 64 | 65 | data = {'time':ttime,'cpu_pct_used':cpu_pct_used,'cpu_temp':cpu_temp,'cpu_freq':cpu_freq} 66 | 67 | now = str(datetime.now()) 68 | now = now.split('.') 69 | now = now[0] 70 | now = now.replace(' ','_') 71 | now = now.replace(':','-') 72 | 73 | #write stuff 74 | keys=sorted(data.keys()) 75 | with open(os.path.join(data_dirname, now+'cpu_log.csv'),'w', newline='') as csv_file: 76 | writer=csv.writer(csv_file) 77 | writer.writerow(keys) 78 | writer.writerows(zip(*[data[key] for key in keys])) 79 | 80 | #reset vars 81 | ttime=[] 82 | cpu_temp=[] 83 | cpu_pct_used=[] 84 | cpu_freq=[] 85 | 86 | #reset time 87 | start = time.time() 88 | 89 | 90 | ############################################################# 91 | # MAIN CODE 92 | ############################################################# 93 | if __name__ == '__main__': 94 | 95 | try: 96 | data_dirname = sys.argv[1] 97 | except: 98 | data_dirname = '../data/demo' 99 | if os.path.exists(os.path.join(data_dirname)): 100 | pass 101 | else: 102 | os.makedirs(os.path.join(data_dirname)) 103 | #print(data_dirname) 104 | cpu_test(data_dirname) 105 | 106 | 107 | 108 | 109 | -------------------------------------------------------------------------------- /py_src/test_disks.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | test_disks.py 6 | 7 | This script is intended to poll disks 8 | and record related data 9 | ''' 10 | 11 | ############################################################# 12 | #IMPORT MODULES 13 | ############################################################# 14 | import os 15 | import sys 16 | import csv 17 | import time 18 | import json 19 | import psutil 20 | from datetime import datetime 21 | 22 | 23 | ############################################################# 24 | # SUPPORT FUNCTIONS 25 | ############################################################# 26 | def disk_test(data_dirname): 27 | 28 | #load inputs 29 | with open('data.json') as f: 30 | data = json.load(f) 31 | data_save_interval = data['data_save_interval'] 32 | test_cycle_time = data['test_cycle_time'] 33 | 34 | #define vars 35 | ttime=[] 36 | num_detected_disks=[] 37 | disk_space_total=[] 38 | disk_space_used=[] 39 | disk_space_free=[] 40 | disk_space_used_pct=[] 41 | disk_info=[] 42 | 43 | print(str(time.time()) + ': starting disk monitor!') 44 | 45 | disks = psutil.disk_partitions() 46 | old_num_detected_disks = len(disks) 47 | 48 | #probably create list of drives here for data storage 49 | 50 | start = time.time() 51 | while True: 52 | 53 | time.sleep(test_cycle_time) 54 | end = time.time() 55 | 56 | ttime+=[time.time()] 57 | disks = psutil.disk_partitions() 58 | num_detected_disks+=[len(disks)] 59 | # how to deal with variable number of disks? 60 | 61 | disk_space_total+=[psutil.disk_usage(disks[1][0])[0]] 62 | disk_space_used+=[psutil.disk_usage(disks[1][0])[1]] 63 | disk_space_free+=[psutil.disk_usage(disks[1][0])[2]] 64 | disk_space_used_pct+=[psutil.disk_usage(disks[1][0])[3]] 65 | disk_info+=[disks] 66 | 67 | if num_detected_disks[-1] != old_num_detected_disks: 68 | print('\n\n NUMBER OF DISKS HAS CHANGED!\n\n') 69 | print(old_num_detected_disks) 70 | print(num_detected_disks) 71 | old_num_detected_disks = num_detected_disks 72 | 73 | if end-start > data_save_interval: 74 | time1 = time.time() 75 | 76 | data = {'time':ttime,'num_detected_disks':num_detected_disks, 'disk_info':disk_info} 77 | 78 | now = str(datetime.now()) 79 | now = now.split('.') 80 | now = now[0] 81 | now = now.replace(' ','_') 82 | now = now.replace(':','-') 83 | 84 | #write stuff 85 | keys=sorted(data.keys()) 86 | with open(os.path.join(data_dirname, now+'disk_log.csv'),'w', newline='') as csv_file: 87 | writer=csv.writer(csv_file) 88 | writer.writerow(keys) 89 | writer.writerows(zip(*[data[key] for key in keys])) 90 | 91 | #reset vars 92 | ttime=[] 93 | num_detected_disks=[] 94 | disk_space_total=[] 95 | disk_space_used=[] 96 | disk_space_free=[] 97 | disk_space_used_pct=[] 98 | disk_info=[] 99 | 100 | 101 | #reset time 102 | start = time.time() 103 | 104 | 105 | ############################################################# 106 | # MAIN CODE 107 | ############################################################# 108 | if __name__ == '__main__': 109 | 110 | try: 111 | data_dirname = sys.argv[1] 112 | except: 113 | data_dirname = '../data/demo' 114 | if os.path.exists(os.path.join(data_dirname)): 115 | pass 116 | else: 117 | os.makedirs(os.path.join(data_dirname)) 118 | #print(data_dirname) 119 | disk_test(data_dirname) 120 | 121 | 122 | 123 | -------------------------------------------------------------------------------- /py_src/test_networks.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | test_networks.py 6 | 7 | This script is intended to 8 | check and record network information 9 | ''' 10 | 11 | ############################################################# 12 | #IMPORT MODULES 13 | ############################################################# 14 | import os 15 | import sys 16 | import csv 17 | import time 18 | import json 19 | import psutil 20 | from datetime import datetime 21 | 22 | 23 | ############################################################# 24 | # SUPPORT FUNCTIONS 25 | ############################################################# 26 | def network_test(data_dirname): 27 | 28 | #load inputs 29 | with open('data.json') as f: 30 | data = json.load(f) 31 | data_save_interval = data['data_save_interval'] 32 | test_cycle_time = data['test_cycle_time'] 33 | 34 | #define vars 35 | ttime=[] 36 | net_info=[] 37 | num_detected_adapters=[] 38 | 39 | print(str(time.time()) + ': starting network monitor!') 40 | 41 | onetime_net_info=psutil.net_io_counters(pernic=True) 42 | old_num_detected_adapters=len(onetime_net_info) 43 | 44 | start = time.time() 45 | while True: 46 | 47 | time.sleep(test_cycle_time) 48 | 49 | end = time.time() 50 | 51 | #grab network information 52 | ttime+=[time.time()] 53 | net_info+=[psutil.net_io_counters(pernic=True)] 54 | num_detected_adapters+=[len(net_info[-1])] 55 | 56 | 57 | if num_detected_adapters[-1] != old_num_detected_adapters: 58 | print('\n\n NUMBER OF NETWORK ADAPTERS HAS CHANGED!\n\n') 59 | print(old_num_detected_adapters) 60 | print(num_detected_adapters) 61 | old_num_detected_adapters = num_detected_adapters[-1] 62 | 63 | 64 | if end-start > data_save_interval: 65 | time1 = time.time() 66 | 67 | data = {'time':ttime,'net_info':net_info,'num_detected_adapters':num_detected_adapters} 68 | 69 | now = str(datetime.now()) 70 | now = now.split('.') 71 | now = now[0] 72 | now = now.replace(' ','_') 73 | now = now.replace(':','-') 74 | 75 | #write stuff 76 | keys=sorted(data.keys()) 77 | with open(os.path.join(data_dirname, now+'net_log.csv'),'w', newline='') as csv_file: 78 | writer=csv.writer(csv_file) 79 | writer.writerow(keys) 80 | writer.writerows(zip(*[data[key] for key in keys])) 81 | 82 | #reset vars 83 | ttime=[] 84 | net_info=[] 85 | num_detected_adapters=[] 86 | 87 | #reset time 88 | start = time.time() 89 | 90 | 91 | ############################################################# 92 | # MAIN CODE 93 | ############################################################# 94 | if __name__ == '__main__': 95 | 96 | try: 97 | data_dirname = sys.argv[1] 98 | except: 99 | data_dirname = '../data/demo' 100 | if os.path.exists(os.path.join(data_dirname)): 101 | pass 102 | else: 103 | os.makedirs(os.path.join(data_dirname)) 104 | #print(data_dirname) 105 | network_test(data_dirname) 106 | 107 | 108 | 109 | -------------------------------------------------------------------------------- /py_src/test_pi_cam.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | test_cam.py 6 | 7 | This script is intended to acquire 8 | images from a Raspberry Pi 9 | camera, save them, and report faults 10 | 11 | NOTE: THIS IS NOT YET SUPPORTED AND MAY 12 | REQUIRE MODIFICATION TO BE FUNCTIONAL 13 | ''' 14 | 15 | ############################################################# 16 | #IMPORT MODULES 17 | ############################################################# 18 | import os 19 | import sys 20 | import csv 21 | import time 22 | import psutil 23 | import cv2 24 | from datetime import datetime 25 | from picamera.array import PiRGBArray 26 | from picamera import PiCamera 27 | 28 | 29 | ############################################################# 30 | # SUPPORT FUNCTIONS 31 | ############################################################# 32 | def cam_test(data_dirname): 33 | 34 | #define inputs 35 | time_interval = 1 #sec 36 | ttime=[] 37 | 38 | #allocate available mem 39 | print('starting cam monitor!') 40 | 41 | upsets=[] 42 | 43 | start = time.time() 44 | # initialize the camera 45 | #cam = cv2.VideoCapture(0) # 0 -> index of camera 46 | camera = PiCamera() 47 | 48 | while True: 49 | 50 | end = time.time() 51 | 52 | ttime+=[time.time()] 53 | 54 | if end-start > time_interval: 55 | 56 | now = str(datetime.now()) 57 | now = now.split('.') 58 | now = now[0] 59 | now = now.replace(' ','_') 60 | now = now.replace(':','-') 61 | 62 | try: 63 | camera.start_preview(fullscreen=False, window=(100,200,800,800)) 64 | time.sleep(1) 65 | camera.capture(os.path.join(data_dirname , now+"_img.jpg")) #save image 66 | print("image captured") 67 | camera.stop_preview() 68 | upsets+=[0] 69 | except: 70 | print("cam not read") 71 | upsets+=[1] 72 | 73 | ''' 74 | #s, img = cam.read() 75 | if True: 76 | cv2.namedWindow("cam-test") 77 | cv2.imshow("cam-test",img) 78 | waitKey(0) 79 | print("image captured") 80 | time.sleep(1) 81 | destroyWindow("cam-test") 82 | 83 | imwrite(os.path.join(data_dirname , now+"_img.jpg",img)) #save image 84 | upsets+=[0] 85 | else : 86 | print("cam not read") 87 | upsets+=[1] 88 | ''' 89 | 90 | time1 = time.time() 91 | 92 | data = {'time':ttime,'upsets':upsets} 93 | 94 | now = str(datetime.now()) 95 | now = now.split('.') 96 | now = now[0] 97 | now = now.replace(' ','_') 98 | now = now.replace(':','-') 99 | 100 | #write stuff 101 | 102 | with open(os.path.join(data_dirname , now+'cam_log.csv'),'w', newline='') as csv_file: 103 | writer=csv.writer(csv_file) 104 | writer.writerow(keys) 105 | writer.writerows(zip(*[data[key] for key in keys])) 106 | 107 | #reset vars 108 | ttime=[] 109 | upsets=[] 110 | 111 | 112 | #reset time 113 | start = time.time() 114 | 115 | 116 | ############################################################# 117 | # MAIN CODE 118 | ############################################################# 119 | if __name__ == '__main__': 120 | 121 | try: 122 | data_dirname = sys.argv[1] 123 | except: 124 | data_dirname = '../data/demo' 125 | if os.path.exists(os.path.join(data_dirname)): 126 | pass 127 | else: 128 | os.makedirs(os.path.join(data_dirname)) 129 | #print(data_dirname) 130 | cam_test(data_dirname) 131 | 132 | 133 | 134 | 135 | -------------------------------------------------------------------------------- /py_src/test_ram.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | test_ram.py 6 | 7 | This script is intended to consume RAM 8 | and monitor it for changed values 9 | ''' 10 | 11 | ############################################################# 12 | #IMPORT MODULES 13 | ############################################################# 14 | import os 15 | import sys 16 | import csv 17 | import time 18 | import json 19 | import psutil 20 | from datetime import datetime 21 | 22 | 23 | ############################################################# 24 | # SUPPORT FUNCTIONS 25 | ############################################################# 26 | def ram_test(data_dirname): 27 | 28 | #load inputs 29 | with open('data.json') as f: 30 | data = json.load(f) 31 | data_save_interval = data['data_save_interval'] 32 | test_cycle_time = data['test_cycle_time'] 33 | ram_pct_to_use = data['ram_pct_to_use'] 34 | 35 | #define vars 36 | ttime=[] 37 | upsets=[] 38 | ram_pct_used=[] 39 | 40 | print(str(time.time()) + ': starting RAM monitor!') 41 | 42 | #allocate available mem 43 | ram_soaker = [] 44 | ram_info = psutil.virtual_memory() 45 | ram_pct = int(ram_info[2]) 46 | old_pct = ram_pct 47 | 48 | print('allocating ram...') 49 | while ram_pct < ram_pct_to_use: 50 | 51 | ram_soaker += ['1' * 512000] 52 | ram_info = psutil.virtual_memory() 53 | ram_pct = int(ram_info[2]) 54 | if ram_pct != old_pct: 55 | print(ram_pct) 56 | old_pct = ram_pct 57 | 58 | 59 | print('...RAM allocation complete!') 60 | ram_info = psutil.virtual_memory() 61 | ram_pct = int(ram_info[2]) 62 | print(ram_pct) 63 | print(len(ram_soaker)) 64 | 65 | print('\n\nWatching for changed RAM vals...') 66 | start = time.time() 67 | while True: 68 | 69 | time.sleep(test_cycle_time) 70 | 71 | outliers = [i for i in range(1,len(ram_soaker)) if ram_soaker[i]!=ram_soaker[i-1] ] 72 | if len(outliers) > 0: 73 | print('\n\n RAM STATE CHANGE DETECTED!\n\n') 74 | upsets+=[1] 75 | else: 76 | upsets+=[0] 77 | 78 | end = time.time() 79 | 80 | ttime+=[time.time()] 81 | ram_info = psutil.virtual_memory() 82 | ram_pct = ram_info[2] 83 | ram_pct_used+=[ram_pct] 84 | 85 | if end-start > data_save_interval: 86 | time1 = time.time() 87 | 88 | data = {'time':ttime,'ram_pct_used':ram_pct_used,'upsets':upsets} 89 | 90 | now = str(datetime.now()) 91 | now = now.split('.') 92 | now = now[0] 93 | now = now.replace(' ','_') 94 | now = now.replace(':','-') 95 | 96 | #write stuff 97 | keys=sorted(data.keys()) 98 | with open(os.path.join(data_dirname, now+'ram_log.csv'),'w', newline='') as csv_file: 99 | writer=csv.writer(csv_file) 100 | writer.writerow(keys) 101 | writer.writerows(zip(*[data[key] for key in keys])) 102 | 103 | #reset vars 104 | ttime=[] 105 | upsets=[] 106 | ram_pct_used=[] 107 | 108 | #reset time 109 | start = time.time() 110 | 111 | 112 | ############################################################# 113 | # MAIN CODE 114 | ############################################################# 115 | if __name__ == '__main__': 116 | 117 | try: 118 | data_dirname = sys.argv[1] 119 | except: 120 | data_dirname = '../data/demo' 121 | if os.path.exists(os.path.join(data_dirname)): 122 | pass 123 | else: 124 | os.makedirs(os.path.join(data_dirname)) 125 | #print(data_dirname) 126 | ram_test(data_dirname) 127 | 128 | 129 | 130 | -------------------------------------------------------------------------------- /setup/install_tool.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # -*- coding: utf-8 -*- 3 | 4 | ''' 5 | install_tool.py 6 | 7 | This script is intended to automatically 8 | install the rad test script dependencies 9 | ''' 10 | 11 | ############################################################# 12 | #IMPORT MODULES 13 | ############################################################# 14 | import os 15 | import sys 16 | 17 | 18 | 19 | ############################################################# 20 | # MAIN CODE 21 | ############################################################# 22 | 23 | print("\nInstalling...\n") 24 | 25 | if 'linux' in sys.platform: pip_cmd = 'sudo pip3' 26 | else: pip_cmd = 'pip3' 27 | 28 | os.system(pip_cmd+' install --upgrade pip') 29 | os.system(pip_cmd+' install opencv-python') 30 | os.system(pip_cmd+' install psutil') 31 | 32 | if 'linux' in sys.platform: os.system('sudo apt-get install libatlas-base-dev --yes') 33 | 34 | print("\n\n...required install complete.") 35 | response = input("\nInstall data visualization tools (not recommended for systems with < 1 GB RAM)? [y/n]: ") 36 | 37 | print(response) 38 | if response == 'y' or response == 'yes' or response == 'YES' or response == 'yeet' or response == 'Y': 39 | print("\ninstalling data visualization tools...\n") 40 | os.system(pip_cmd+' install pandas') 41 | os.system(pip_cmd+' --no-cache-dir install matplotlib') 42 | print("\n ...visualization tool install complete!") 43 | 44 | print("\nComplete. Exiting...\n") 45 | 46 | 47 | --------------------------------------------------------------------------------