├── LICENSE ├── README.md ├── app.py ├── doc ├── .keep ├── CONTRIB.md ├── INSTALL.md └── LICENSE.md ├── images ├── OpenSVP.png ├── closed.gif ├── closed_128.gif ├── closed_32.gif ├── closed_64.gif ├── closed_64s.gif ├── closed_96.gif ├── complete.gif ├── directory.gif ├── error.gif ├── excel.png ├── none.png ├── open.gif ├── open_64s.gif ├── open_x.gif ├── page.png ├── page_copy.png ├── pass.gif ├── pause_0.gif ├── pause_128.gif ├── pause_144.gif ├── pause_160.gif ├── pause_192.gif ├── pause_192_255.gif ├── pause_192_255.ico ├── pause_224.gif ├── pause_224_0.gif ├── pause_224_0.ico ├── pause_32.gif ├── pause_64.gif ├── pause_96.gif ├── report.png ├── result.gif ├── result.png ├── result_dir.gif ├── result_dir.png ├── result_dir_p.gif ├── results.png ├── results_1.png ├── results_2.png ├── results_27000.gif ├── results_3.png ├── results_5400.gif ├── results_grey.gif ├── results_p.gif ├── run_0.gif ├── run_128.gif ├── run_160.gif ├── run_192.gif ├── run_224.gif ├── run_32.gif ├── run_64.gif ├── run_96.gif ├── running.gif ├── sandia.gif ├── script.gif ├── script_32.gif ├── script_dir.gif ├── scripts.gif ├── stop_0.gif ├── stop_128.gif ├── stop_160.gif ├── stop_192.gif ├── stop_224.gif ├── stop_32.gif ├── stop_64.gif ├── stop_96.gif ├── stopped.gif ├── suite.gif ├── suite_32.gif ├── suite_complete.gif ├── suite_dir.gif ├── suites.gif ├── sunspec_16.gif ├── sunspec_32.gif ├── sunspec_x.gif ├── sunspec_x.png ├── test.gif ├── test_32.gif ├── test_dir.gif ├── tests.gif └── working_dir.gif ├── result.py ├── script.py ├── sunspec_x.ico ├── svp_requirements.txt ├── svptreectrl.py ├── testing └── .keep └── ui.py /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![SVP Logo](images/OpenSVP.png?raw=true) 2 | 3 | --- 4 | 5 | [Sunspec Alliance](https://sunspec.org/) is an alliance of over 100 solar and storage distributed energy industry participants, together pursuing information standards to enable “plug & play” system interoperability. 6 | 7 | [CanmetENERGY-Varennes](https://www.nrcan.gc.ca/science-data/research-centres-labs/canmetenergy-research-centres/varennes-qc-research-centre/5761) is a research center designing and implementing clean energy solutions, and build on knowledge that helps produce and use energy in ways that are more efficient and sustainable. 8 | 9 | This repository contains all of the open-source OpenSVP components written in Python 3.7 10 | 11 | 12 | ## Contribution 13 | 14 | For the contribution list, please refer to [Contribution section](doc/CONTRIB.md) 15 | 16 | ## Installation 17 | 18 | Please refer to the [Install section](doc/INSTALL.md) for detailed instruction 19 | 20 | ## Scripts, Tests and Suites 21 | 22 | ### UL1741 SA standard 23 | Here is the repo [UL1741 SA repository][1741SA-url] make sure to use the "dev" branch 24 | 25 | ### IEEE 1547.1 standard 26 | The latest version is 1.4.2 and here is the repo [IEEE 1547.1 repository][1547-1-url]. Make sure to use the library (p1547.py) with the same version. 27 | 28 | ### DR_AS-NZS 4777.2 standard 29 | The latest version is 1.0.1 and here is the repo [DR_AS-NZS 4777.2 repository][4777-2-url]. Make sure to use the library (pAus4777.py) with the same version. 30 | 31 | ## Drivers 32 | ### SVPELAB 33 | The SVPELAB is the repository use by the SIRFN members. It tries to assemble all the equipment from different manufacturer(Grid simulator, PV simulator, Data acquisition system, HILs, etc.) in a single repo [SVPELAB repository][svpelab-url]. 34 | 35 | ### Support 36 | 37 | For any bugs/issues, please refer to the [bug tracker][bug-tracker-url] section. 38 | 39 | 40 | 41 | [bug-tracker-url]: https://github.com/sunspec/svp/issues 42 | [1547-1-url]: https://github.com/jayatsandia/svp_1547.1/tree/master3.7 43 | [1741SA-url]: https://github.com/sunspec/svp_UL1741SA/tree/dev/UL1741%20SA 44 | [4777-2-url]: https://github.com/BuiMCanmet/DR_AS-NZS-Scripts/tree/master 45 | [svpelab-url]: https://github.com/sunspec/svp_energy_lab/tree/dev37 46 | 47 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | 4 | Copyright 2018, SunSpec Alliance 5 | 6 | Licensed under the Apache License, Version 2.0 (the "License"); 7 | you may not use this file except in compliance with the License. 8 | You may obtain a copy of the License at 9 | 10 | http://www.apache.org/licenses/LICENSE-2.0 11 | 12 | Unless required by applicable law or agreed to in writing, software 13 | distributed under the License is distributed on an "AS IS" BASIS, 14 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | See the License for the specific language governing permissions and 16 | limitations under the License. 17 | 18 | """ 19 | 20 | 21 | 22 | 23 | import os 24 | import sys 25 | 26 | import multiprocessing 27 | import importlib 28 | import datetime 29 | import copy 30 | import imp 31 | 32 | import time 33 | import xml.etree.ElementTree as ET 34 | 35 | import result as rslt 36 | import script 37 | 38 | extended_path_list = [] 39 | 40 | class SVPError(Exception): 41 | pass 42 | 43 | def script_update(path, old_name, new_name): 44 | try: 45 | files = os.listdir(path) 46 | for f in files: 47 | try: 48 | file_path = os.path.join(path, f) 49 | name, ext = os.path.splitext(f) 50 | if ext == TEST_EXT: 51 | config = script.ScriptConfig(filename=file_path) 52 | if os.path.normcase(config.script) == os.path.normcase(old_name): 53 | if new_name is not None: 54 | config.script = new_name 55 | config.to_xml_file() 56 | else: 57 | # remove file 58 | pass 59 | else: 60 | if os.path.isdir(file_path): 61 | script_update(file_path, old_name, new_name) 62 | except Exception as e: 63 | pass 64 | except Exception as e: 65 | raise SVPError('Error on script update - directory %s: %s' % (path, str(e))) 66 | 67 | SUITE_EXT = '.ste' 68 | TEST_EXT = '.tst' 69 | SCRIPT_EXT = '.py' 70 | RESULTS_EXT = '.rlt' 71 | LOG_EXT = '.log' 72 | CSV_EXT = '.csv' 73 | 74 | SUITES_DIR = 'Suites' 75 | TESTS_DIR = 'Tests' 76 | SCRIPTS_DIR = 'Scripts' 77 | RESULTS_DIR = 'Results' 78 | LIB_DIR = 'Lib' 79 | FILES_DIR = 'Files' 80 | 81 | def test_to_file(name): 82 | if not is_test_file(name): 83 | return name + TEST_EXT 84 | return name 85 | 86 | def file_to_test(name): 87 | if is_test_file(name): 88 | return name[:-4] 89 | return name 90 | 91 | def is_test_file(name): 92 | return name.endswith(TEST_EXT) 93 | 94 | def file_to_script(name): 95 | if is_script_file(name): 96 | return name[:-3] 97 | return name 98 | 99 | def script_to_file(name): 100 | if not is_script_file(name): 101 | return name + SCRIPT_EXT 102 | return name 103 | 104 | def is_script_file(name): 105 | return name.endswith(SCRIPT_EXT) 106 | 107 | def is_suite_file(name): 108 | return name.endswith(SUITE_EXT) 109 | 110 | def suite_to_file(name): 111 | if not is_suite_file(name): 112 | return name + SUITE_EXT 113 | return name 114 | 115 | def file_to_suite(name): 116 | if is_suite_file(name): 117 | return name[:-4] 118 | return name 119 | 120 | def is_log_file(name): 121 | return name.endswith(LOG_EXT) 122 | 123 | ''' 124 | class _Popen(multiprocessing.forking.Popen): 125 | def __init__(self, *args, **kw): 126 | if hasattr(sys, 'frozen'): 127 | # We have to set original _MEIPASS2 value from sys._MEIPASS 128 | # to get --onefile mode working. 129 | os.putenv('_MEIPASS2', sys._MEIPASS) 130 | try: 131 | super(_Popen, self).__init__(*args, **kw) 132 | finally: 133 | if hasattr(sys, 'frozen'): 134 | # On some platforms (e.g. AIX) 'os.unsetenv()' is not 135 | # available. In those cases we cannot delete the variable 136 | # but only set it to the empty string. The bootloader 137 | # can handle this case. 138 | if hasattr(os, 'unsetenv'): 139 | os.unsetenv('_MEIPASS2') 140 | else: 141 | os.putenv('_MEIPASS2', '') 142 | 143 | 144 | class MultiProcess(multiprocessing.Process): 145 | _Popen = _Popen 146 | ''' 147 | 148 | class MultiProcess(multiprocessing.Process): 149 | pass 150 | 151 | # cmd_line_target_dirs = [SUITES_DIR, TESTS_DIR, SCRIPTS_DIR] 152 | 153 | def process_run(filename, env, config, params, lib_path, conn): 154 | name = script_path = None 155 | try: 156 | sys.stdout = sys.stderr = open(os.path.join(trace_dir(), 'sunssvp_script.log'), "w") 157 | script_path, name = os.path.split(filename) 158 | name, ext = os.path.splitext(name) 159 | if lib_path is not None: 160 | sys.path.insert(0, lib_path) 161 | sys.path.insert(0, script_path) 162 | try: 163 | m = importlib.import_module(name) 164 | info = m.script_info() 165 | test_script = RunScript(env=env, info=info, config=config, config_file=None, params=params, conn=conn) 166 | m.run(test_script) 167 | except Exception as e: 168 | raise e 169 | finally: 170 | if name in sys.modules: 171 | del sys.modules[name] 172 | if sys.path[0] == script_path: 173 | del sys.path[0] 174 | if lib_path is not None and sys.path[0] == lib_path: 175 | del sys.path[0] 176 | 177 | 178 | class LogEntry(object): 179 | def __init__(self, message, level=script.INFO, timestamp=None): 180 | self.message = message 181 | self.level = level 182 | self.timestamp = timestamp 183 | 184 | if self.timestamp is None: 185 | self.timestamp = datetime.datetime.now() 186 | 187 | def timestamp_str(self): 188 | return self.timestamp.strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] 189 | 190 | def __str__(self): 191 | return '%s %s %s' % (self.timestamp_str(), self.level, self.message) 192 | 193 | class Directory(object): 194 | def __init__(self, path=None, working=False): 195 | self.path = path 196 | self.working = working 197 | 198 | ''' 199 | Suite 200 | ''''' 201 | 202 | # suite xml elements and attributes 203 | SUITE_ROOT = 'suite' 204 | SUITE_ATTR_NAME = 'name' 205 | SUITE_ATTR_TYPE = 'type' 206 | SUITE_ATTR_GLOBALS = 'globals' 207 | SUITE_PARAMS = 'params' 208 | SUITE_PARAM = 'param' 209 | SUITE_MEMBERS = 'members' 210 | SUITE_MEMBER = 'member' 211 | SUITE_TESTS = 'tests' 212 | SUITE_SUITE = 'suite' 213 | SUITE_TEST = 'test' 214 | 215 | def member_update(path, old_name, new_name): 216 | try: 217 | files = os.listdir(path) 218 | for f in files: 219 | try: 220 | file_path = os.path.join(path, f) 221 | name, ext = os.path.splitext(f) 222 | if ext == SUITE_EXT: 223 | suite = Suite(filename=file_path) 224 | suite.member_update(old_name, new_name) 225 | else: 226 | if os.path.isdir(file_path): 227 | member_update(file_path, old_name, new_name) 228 | except Exception as e: 229 | pass 230 | except Exception as e: 231 | raise SVPError('Error on member update - directory %s: %s' % (path, str(e))) 232 | 233 | class Suite(object): 234 | def __init__(self, name=None, desc=None, filename=None, parent=None): 235 | self.name = name 236 | self.globals = True 237 | self.desc = desc 238 | self.filename = filename 239 | self.members = [] 240 | self.params = {} 241 | self.param_defs = script.ScriptParamGroupDef(name=script.SCRIPT_PARAM_ROOT, qname=script.SCRIPT_PARAM_ROOT) 242 | self.scripts = [] 243 | self.logos = [] 244 | self.parent = parent 245 | self.member_index = 0 246 | self.result_dir = None 247 | self.active_params = None 248 | self.result = None 249 | 250 | if filename: 251 | self.from_xml(filename=filename) 252 | 253 | def next_member(self): 254 | if self.member_index >= len(self.members): 255 | if self.parent is not None: 256 | return self.parent.next_member() 257 | else: 258 | member = self.members[self.member_index] 259 | self.member_index += 1 260 | return member 261 | 262 | def member_update(self, old_name, new_name): 263 | # print 'member_update: %s %s %s' % (self.filename, old_name, new_name) 264 | updated = False 265 | members = [] 266 | for i in range(len(self.members)): 267 | # print 'compare: %s %s' % (os.path.normcase(self.members[i]), os.path.normcase(old_name)) 268 | if os.path.normcase(self.members[i]) == os.path.normcase(old_name): 269 | if new_name is not None: 270 | members.append(new_name) 271 | updated = True 272 | # print 'updated = %s' % (updated) 273 | else: 274 | members.append(self.members[i]) 275 | # print 'update check = %s' % (updated) 276 | if updated: 277 | # print self.members 278 | self.members = members 279 | self.to_xml_file() 280 | 281 | def merge_suite(self, suite, working_dir): 282 | for m in suite.members: 283 | try: 284 | if is_suite_file(m): 285 | filename = os.path.join(working_dir, SUITES_DIR, os.path.normpath(m)) 286 | member_suite = Suite(filename=filename) 287 | self.merge_suite(member_suite, working_dir) 288 | elif is_test_file(m): 289 | filename = os.path.join(working_dir, TESTS_DIR, os.path.normpath(m)) 290 | # print 'merge file: ', filename 291 | script_config = script.ScriptConfig(filename=filename) 292 | if script_config.script not in self.scripts: 293 | self.scripts.append(script_config.script) 294 | script_path = os.path.join(working_dir, SCRIPTS_DIR, os.path.normpath(script_config.script)) 295 | lib_path = os.path.join(working_dir, LIB_DIR) 296 | test_script = script.load_script(script_path, lib_path, path_list = extended_path_list) 297 | if test_script.param_defs is not None: 298 | for group in test_script.param_defs.param_groups: 299 | if test_script.param_is_global(group.name): 300 | if self.param_defs.param_group_find(group.name) is None: 301 | self.param_defs.param_groups.append(group) 302 | 303 | # merge logos while we are at it 304 | for logo in test_script.info.logos: 305 | if logo not in self.logos: 306 | self.logos.append(logo) 307 | else: 308 | ### log 309 | pass 310 | except Exception as e: 311 | print ("{}".format(e)) 312 | 313 | def merge_param_defs(self, working_dir): 314 | # print 'working_dir =', working_dir 315 | self.param_defs = script.ScriptParamGroupDef(name=script.SCRIPT_PARAM_ROOT, qname=script.SCRIPT_PARAM_ROOT) 316 | self.scripts = [] 317 | self.merge_suite(self, working_dir) 318 | self.param_defs.resolve_active(self.param_defs, self.param_get) 319 | # print self.scripts 320 | 321 | def contains_suite(self, working_dir, filename): 322 | if self.filename == filename: 323 | return True 324 | for m in self.members: 325 | if is_suite_file(m): 326 | member_filename = os.path.join(working_dir, SUITES_DIR, os.path.normpath(m)) 327 | if filename == member_filename: 328 | return True 329 | else: 330 | suite = Suite(filename=member_filename) 331 | if suite.contains_suite(working_dir, filename): 332 | return True 333 | return False 334 | 335 | def param_get(self, name): 336 | value = None 337 | if self.globals is True and self.params is not None: 338 | value = self.params.get(name) 339 | if value is None: 340 | if self.param_defs is not None: 341 | value = self.param_defs.param_value(name, self.param_defs, self.param_get) 342 | return value 343 | 344 | def param_value(self, name): 345 | value = None 346 | if self.globals is True and self.params is not None: 347 | value = self.params.get(name) 348 | if value is None: 349 | if self.param_defs is not None: 350 | value = self.param_defs.param_value(name, self.param_defs, self.param_value) 351 | return value 352 | 353 | def from_xml(self, element=None, filename=None): 354 | if element is None and filename is not None: 355 | element = ET.ElementTree(file=filename).getroot() 356 | if element is None: 357 | raise SVPError('No xml document element') 358 | 359 | if element.tag != SUITE_ROOT: 360 | raise SVPError('Unexpected test suite root element %s' % (element.tag)) 361 | 362 | self.name = element.attrib.get(SUITE_ATTR_NAME) 363 | if self.name is None: 364 | raise SVPError('Suite name missing') 365 | # self.desc = element.attrib.get(TEST_CFG_ATTR_DESC) 366 | globals = element.attrib.get(SUITE_ATTR_GLOBALS) 367 | if globals == 'False': 368 | self.globals = False 369 | 370 | for e in element.findall('*'): 371 | if e.tag == SUITE_MEMBERS: 372 | for e_test in e.findall('*'): 373 | if e_test.tag == SUITE_MEMBER: 374 | name = e_test.attrib.get(SUITE_ATTR_NAME) 375 | if name: 376 | self.members.append(name) 377 | 378 | script.params_from_xml(self.params, element) 379 | 380 | def to_xml(self, parent=None, filename=None): 381 | attr = {} 382 | if self.name: 383 | attr[SUITE_ATTR_NAME] = self.name 384 | 385 | if self.globals is not None: 386 | attr[SUITE_ATTR_GLOBALS] = str(self.globals) 387 | 388 | if parent is not None: 389 | e = ET.SubElement(parent, SUITE_ROOT, attrib=attr) 390 | else: 391 | e = ET.Element(SUITE_ROOT, attrib=attr) 392 | 393 | e_members = ET.SubElement(e, SUITE_MEMBERS) 394 | for m in self.members: 395 | attr = {SUITE_ATTR_NAME: m} 396 | ET.SubElement(e_members, SUITE_MEMBER, attrib=attr) 397 | 398 | script.params_to_xml(self.params, e) 399 | 400 | return e 401 | 402 | def to_xml_str(self, pretty_print=False): 403 | e = self.to_xml() 404 | 405 | if pretty_print: 406 | script.xml_indent(e) 407 | 408 | return ET.tostring(e, encoding='unicode') 409 | 410 | def to_xml_file(self, filename=None, pretty_print=True, replace_existing=True): 411 | xml = self.to_xml_str(pretty_print) 412 | if filename is None and self.filename is not None: 413 | filename = self.filename 414 | 415 | if filename is not None: 416 | if replace_existing is False and os.path.exists(filename): 417 | raise SVPError('File %s already exists' % (filename)) 418 | f = open(filename, 'w') 419 | f.write(xml) 420 | f.close() 421 | else: 422 | print (xml) 423 | 424 | RUN_MSG_ALERT = 'alert' 425 | RUN_MSG_CONFIRM = 'confirm' 426 | RUN_MSG_LOG = 'log' 427 | RUN_MSG_RESULT = 'result' 428 | RUN_MSG_RESULT_FILE = 'result_file' 429 | RUN_MSG_STATUS = 'status' 430 | RUN_MSG_CMD = 'cmd' 431 | 432 | RUN_MSG_CMD_PAUSE = 'pause' 433 | RUN_MSG_CMD_RESUME = 'resume' 434 | RUN_MSG_CMD_STOP = 'stop' 435 | 436 | class RunScript(script.Script): 437 | def __init__(self, env=None, info=None, config=None, config_file=None, params=None, conn=None): 438 | script.Script.__init__(self, env=env, info=info, config=config, config_file=config_file, params=params) 439 | 440 | self._conn = conn 441 | self._files_dir = env.get('files_dir', '') 442 | self._results_dir = env.get('results_dir', '') 443 | self._result_dir = env.get('result_dir', '') 444 | self._log_file = os.path.join(self._results_dir, env.get('result_log_file')) 445 | 446 | def conn_msg(self): 447 | msg = None 448 | try: 449 | if self._conn: 450 | if self._conn.poll() is True: 451 | msg = self._conn.recv() 452 | except Exception as e: 453 | raise SVPError('Conn msg error: {}'.format (e)) 454 | 455 | return msg 456 | 457 | def alert(self, message): 458 | self._conn.send({'op': RUN_MSG_ALERT, 459 | 'message': message}) 460 | 461 | def confirm(self, message): 462 | result = False 463 | 464 | self._conn.send({'op': RUN_MSG_CONFIRM, 465 | 'message': message}) 466 | 467 | seconds = 1000 468 | while seconds > 0: 469 | msg = self.conn_msg() 470 | if msg is None: 471 | time.sleep(.1) 472 | seconds -= .1 473 | elif isinstance(msg, dict): 474 | if msg.get('op') == RUN_MSG_CONFIRM: 475 | result = msg.get('result', False) 476 | break 477 | 478 | return result 479 | 480 | def log(self, message, level=script.INFO): 481 | entry = LogEntry(message, level=level) 482 | if self._log_file is not None: 483 | log_file = open(self._log_file, 'a') 484 | log_file.write('%s\n' % (str(entry))) 485 | log_file.close() 486 | if self._conn: 487 | self._conn.send({'op': RUN_MSG_LOG, 488 | 'timestamp': entry.timestamp_str(), 489 | 'level': entry.level, 490 | 'message': entry.message}) 491 | 492 | def result(self, status=None, params=None): 493 | self.log('Test result - %s' % (script.result_str(status))) 494 | 495 | self._conn.send({'op': RUN_MSG_RESULT, 496 | 'status': status, 497 | 'params': params}) 498 | 499 | # wait for confirmation of completion 500 | seconds = 5 501 | while seconds > 0: 502 | msg = self.conn_msg() 503 | if msg is None: 504 | time.sleep(.1) 505 | seconds -= .1 506 | elif isinstance(msg, dict): 507 | if msg.get('op') == RUN_MSG_RESULT: 508 | break 509 | 510 | def result_file(self, name=None, status=None, params=None): 511 | 512 | self._conn.send({'op': RUN_MSG_RESULT_FILE, 513 | 'name': name, 514 | 'status': status, 515 | 'params': params}) 516 | 517 | # wait for confirmation of completion 518 | seconds = 5 519 | while seconds > 0: 520 | msg = self.conn_msg() 521 | if msg is None: 522 | time.sleep(.1) 523 | seconds -= .1 524 | elif isinstance(msg, dict): 525 | if msg.get('op') == RUN_MSG_RESULT_FILE: 526 | break 527 | 528 | def result_file_path(self, name): 529 | return os.path.join(self._results_dir, self._result_dir, name) 530 | 531 | def sleep(self, seconds): 532 | if self.callback is True: 533 | raise script.ScriptError('Can not call sleep() from callback function') 534 | current_time = time.time() 535 | wake_time = current_time + seconds 536 | while wake_time > current_time: 537 | sleep_time = .5 538 | # service timers 539 | timers = list(self.timers) 540 | for t in timers: 541 | next = round(t.next_timeout - current_time, 3) 542 | if next <= 0: 543 | self.callback = True 544 | t.callback(t.arg) 545 | self.callback = False 546 | if t.repeating is False: 547 | self.timer_cancel(t) 548 | else: 549 | t.next_timeout += t.period 550 | next = round(t.next_timeout - current_time, 3) 551 | if next < sleep_time: 552 | sleep_time = next 553 | 554 | # service messages 555 | msg = self.conn_msg() 556 | if msg is not None: 557 | if isinstance(msg, dict): 558 | if msg.get('op') == RUN_MSG_CMD: 559 | self.log('message: %s' % msg) 560 | if msg.get('cmd') == RUN_MSG_CMD_STOP: 561 | raise script.ScriptError('Commanded stop') 562 | elif msg.get('cmd') == RUN_MSG_CMD_PAUSE: 563 | self._conn.send(msg) 564 | paused = True 565 | while paused: 566 | msg = self.conn_msg() 567 | if msg is None: 568 | time.sleep(.1) 569 | # seconds -= .1 570 | elif msg.get('cmd') == RUN_MSG_CMD_RESUME: 571 | self._conn.send(msg) 572 | break 573 | # elif isinstance(msg, Confirm): 574 | # return msg 575 | # elif isinstance(msg, Alert): 576 | # return msg 577 | 578 | if sleep_time > 0: 579 | time.sleep(sleep_time) 580 | current_time = time.time() 581 | 582 | ''' 583 | PROC_STATE_RUNNING = 1 584 | PROC_STATE_COMPLETE = 2 585 | PROC_STATE_RUNNING_PAUSE = 3 586 | PROC_STATE_PAUSED = 4 587 | PROC_STATE_RUNNING_STOP = 5 588 | PROC_STATE_STOPPED = 6 589 | ''' 590 | 591 | def makedirs(path): 592 | try: 593 | os.makedirs(path) 594 | except OSError: 595 | if not os.path.isdir(path): 596 | raise 597 | 598 | def result_file_name(name): 599 | return name.replace(script.PATH_SEP, '__') 600 | 601 | PERIODIC_RECV_LIMIT = 10 602 | 603 | class RunContext(object): 604 | 605 | def __init__(self, svp_dir, svp_file=None, results=None, results_name=None): 606 | self.active = False 607 | self.results_tree = results 608 | if svp_dir is None or not os.path.isdir(svp_dir): 609 | raise SVPError('Unknown run context directory: {}'.format(svp_dir)) 610 | self.svp_dir = svp_dir 611 | self.files_dir = None 612 | self.results_dir = None 613 | self.results = None 614 | self.results_name = results_name 615 | self.results_id = None 616 | self.results_file = None 617 | self.lib_path = os.path.normpath(os.path.join(svp_dir, LIB_DIR)) 618 | self.env = {} 619 | 620 | self.svp_file = svp_file 621 | self.process = None 622 | self.log_file = None 623 | self.test_conn = None 624 | self.app_conn = None 625 | self.suites = [] 626 | self.suite = None 627 | self.suite_params = None 628 | self.suite_result_dir = '' 629 | self.result_dir = '' 630 | self.active_result = None 631 | self.status = None 632 | 633 | def is_alive(self): 634 | if self.process is not None: 635 | return self.process.is_alive() 636 | return False 637 | 638 | def run(self): 639 | self.active = True 640 | # set root result entry 641 | d = datetime.datetime.now() 642 | self.results_id = '%d-%02d-%02d_%02d-%02d-%02d-%03d' % (d.year, d.month, d.day, d.hour, d.minute, d.second, 643 | d.microsecond/1000) 644 | # create results directory and results file 645 | if self.results_name is not None: 646 | self.results_id += '__' + result_file_name(self.results_name) 647 | self.results_dir = os.path.join(self.svp_dir, RESULTS_DIR, self.results_id) 648 | makedirs(self.results_dir) 649 | self.results_file = os.path.join(self.results_dir, self.results_id + RESULTS_EXT) 650 | self.files_dir = os.path.join(self.svp_dir, FILES_DIR) 651 | if self.results_tree: 652 | self.results = self.results_tree 653 | self.active_result = self.results 654 | self.update_result(name=self.results_id) 655 | self.active_result.results_index = 0 656 | self.svp_file = None 657 | result = self.active_result.next_result() 658 | if result is not None: 659 | self.svp_file = result.file() 660 | self.active_result = result 661 | else: 662 | self.results = rslt.Result(name=self.results_id, type=rslt.RESULT_TYPE_RESULT) 663 | self.active_result = self.results 664 | self.update_result() 665 | 666 | # start 667 | self.run_next() 668 | 669 | def run_next(self): 670 | while self.suite: 671 | self.active_result = self.suite.result 672 | if self.results_tree: 673 | self.svp_file = None 674 | result = self.active_result.next_result() 675 | if result is not None: 676 | self.active_result = result 677 | self.svp_file = result.file() 678 | else: 679 | self.svp_file = self.suite.next_member() 680 | if self.svp_file is None: 681 | self.suite = None if not self.suites else self.suites.pop() 682 | if self.suite is not None: 683 | self.suite_params = self.suite.active_params 684 | self.suite_result_dir = self.suite.result_dir 685 | else: 686 | self.suite_params = None 687 | self.suite_result_dir = self.results_dir 688 | else: 689 | break 690 | 691 | if self.svp_file is not None and self.status != rslt.RESULT_STOPPED: 692 | name, ext = os.path.splitext(self.svp_file) 693 | if ext == TEST_EXT: 694 | filename = os.path.normpath(os.path.join(self.svp_dir, TESTS_DIR, self.svp_file)) 695 | script_config = script.ScriptConfig(filename=filename) 696 | script_filename = os.path.normpath(os.path.join(self.svp_dir, SCRIPTS_DIR, script_config.script)) 697 | self.result_dir = os.path.join(self.suite_result_dir, result_file_name(name)) 698 | makedirs(os.path.join(self.results_dir, self.result_dir)) 699 | log_file = os.path.join(self.result_dir, result_file_name(name) + LOG_EXT) 700 | if self.results_tree: 701 | self.update_result(status=rslt.RESULT_RUNNING, filename=log_file) 702 | else: 703 | result = rslt.Result(name=name, type=rslt.RESULT_TYPE_TEST, filename=log_file) 704 | self.active_result.add_result(result) 705 | self.active_result = result 706 | self.update_result(status=rslt.RESULT_RUNNING) 707 | env = {'files_dir': self.files_dir, 708 | 'results_dir': self.results_dir, 709 | 'result_dir': self.result_dir, 710 | 'results_id': self.results_id, 711 | 'result_log_file': log_file} 712 | self.start(script_filename, env, config=script_config, params=self.suite_params) 713 | elif ext == SUITE_EXT: 714 | filename = os.path.normpath(os.path.join(self.svp_dir, SUITES_DIR, self.svp_file)) 715 | suite = Suite(filename=filename, parent=self.suite) 716 | # use parent suite params if present 717 | if self.suite is not None: 718 | self.suites.append(self.suite) 719 | if self.suite_params: 720 | suite.active_params = self.suite_params 721 | elif suite.globals: 722 | self.suite_params = suite.active_params = suite.params 723 | suite.result_dir = os.path.join(self.suite_result_dir, result_file_name(name)) 724 | self.suite = suite 725 | self.suite_result_dir = suite.result_dir 726 | if self.results_tree: 727 | suite.result = self.active_result 728 | else: 729 | suite.result = rslt.Result(name=name, type=rslt.RESULT_TYPE_SUITE) 730 | self.active_result.add_result(suite.result) 731 | self.active_result = suite.result 732 | self.update_result() 733 | self.run_next() 734 | elif ext == SCRIPT_EXT: 735 | script_filename = os.path.normpath(os.path.join(self.svp_dir, SCRIPTS_DIR, self.svp_file)) 736 | self.result_dir = os.path.join(self.suite_result_dir, result_file_name(name)) 737 | makedirs(os.path.join(self.results_dir, self.result_dir)) 738 | log_file = os.path.join(self.result_dir, result_file_name(name) + LOG_EXT) 739 | if self.results_tree: 740 | self.update_result(status=rslt.RESULT_RUNNING, filename=log_file) 741 | else: 742 | result = rslt.Result(name=name, type=rslt.RESULT_TYPE_SCRIPT, filename=log_file) 743 | self.active_result.add_result(result) 744 | self.active_result = result 745 | self.update_result(status=rslt.RESULT_RUNNING) 746 | env = {'files_dir': self.files_dir, 747 | 'results_dir': self.results_dir, 748 | 'result_dir': self.result_dir, 749 | 'result_log_file': log_file} 750 | self.start(script_filename, env, config=None, params=None) 751 | else: 752 | if ext: 753 | raise SVPError('Unknown target file extension: %s' % (ext)) 754 | else: 755 | raise SVPError('Target file missing extension') 756 | self.svp_file = None 757 | else: 758 | self.active = False 759 | self.complete() 760 | 761 | def complete(self): 762 | pass 763 | 764 | def pause(self): 765 | pass 766 | ''' 767 | try: 768 | if self.process and self.app_conn and self.state == PROC_STATE_RUNNING: 769 | self.app_conn.send({'op': RUN_MSG_CMD, 770 | 'cmd': RUN_MSG_CMD_PAUSE}) 771 | self.state = PROC_STATE_RUNNING_PAUSE 772 | except Exception, e: 773 | raise 774 | ''' 775 | 776 | def resume(self): 777 | pass 778 | ''' 779 | try: 780 | if self.process and self.app_conn and self.state == PROC_STATE_PAUSED: 781 | self.app_conn.send({'op': RUN_MSG_CMD, 782 | 'cmd': RUN_MSG_CMD_RESUME}) 783 | except Exception, e: 784 | raise 785 | ''' 786 | 787 | def start(self, filename, env=None, config=None, params=None): 788 | if self.process is not None: 789 | raise SVPError('Execution context process already running') 790 | 791 | try: 792 | if self.test_conn is not None: 793 | self.test_conn.close() 794 | self.test_conn = None 795 | if self.app_conn is not None: 796 | self.app_conn.close() 797 | self.app_conn = None 798 | except Exception as e: 799 | pass 800 | 801 | try: 802 | self.test_conn, self.app_conn = multiprocessing.Pipe() 803 | except Exception as e: 804 | print ('Error creating execution context pipe: {}'.format(e)) 805 | 806 | try: 807 | if config is not None: 808 | script_config = copy.deepcopy(config) 809 | else: 810 | script_config = None 811 | self.process = MultiProcess(name='svp_process', target=process_run, args=(filename, env, script_config, 812 | params, self.lib_path, 813 | self.test_conn)) 814 | self.process.start() 815 | except Exception as e: 816 | # raise 817 | print ('Error creating execution context process: {}'.format(e)) 818 | try: 819 | if self.process: 820 | self.process.terminate() 821 | # self.process.join(timeout=0) 822 | except Exception as e: 823 | pass 824 | 825 | self.process = None 826 | 827 | def terminate(self): 828 | if self.process and self.process.is_alive(): 829 | # ### send stop signal to process, stop forcefully for now 830 | try: 831 | self.process.terminate() 832 | except Exception as e: 833 | print ('Process termination error: {}'.format(e)) 834 | self.status = script.RESULT_FAIL 835 | self.clean_up() 836 | 837 | def stop(self): 838 | try: 839 | if self.process and self.app_conn and self.status == rslt.RESULT_RUNNING: 840 | self.update_result(status=rslt.RESULT_STOPPED) 841 | self.app_conn.send({'op': RUN_MSG_CMD, 842 | 'cmd': RUN_MSG_CMD_STOP}) 843 | except Exception as e: 844 | raise e 845 | 846 | def clean_up(self): 847 | try: 848 | if self.test_conn is not None: 849 | self.test_conn.close() 850 | self.test_conn = None 851 | if self.app_conn is not None: 852 | self.app_conn.close() 853 | self.app_conn = None 854 | except Exception as e: 855 | pass 856 | 857 | try: 858 | if self.process: 859 | self.process.join(timeout=0) 860 | except Exception as e: 861 | pass 862 | 863 | if self.process and self.process.exitcode != 0: 864 | if self.status != rslt.RESULT_STOPPED: 865 | self.update_result(status=script.RESULT_FAIL) 866 | 867 | self.process = None 868 | 869 | def periodic(self): 870 | if self.app_conn: 871 | count = 0 872 | msg = None 873 | while count < PERIODIC_RECV_LIMIT: 874 | if self.app_conn.poll() is True: 875 | try: 876 | msg = self.app_conn.recv() 877 | if isinstance(msg, dict): 878 | op = msg.get('op') 879 | if op == RUN_MSG_LOG: 880 | timestamp = msg.get('timestamp') 881 | level = msg.get('level') 882 | message = msg.get('message') 883 | self.log(timestamp, level, message) 884 | elif op == RUN_MSG_ALERT: 885 | message = msg.get('message') 886 | self.alert(message) 887 | elif op == RUN_MSG_CONFIRM: 888 | message = msg.get('message') 889 | msg['result'] = self.confirm(message) 890 | self.app_conn.send(msg) 891 | elif op == RUN_MSG_RESULT: 892 | status = msg.get('status') 893 | filename = msg.get('filename') 894 | params = msg.get('params') 895 | self.update_result(status=status, filename=filename, params=params) 896 | self.app_conn.send(msg) 897 | elif op == RUN_MSG_RESULT_FILE: 898 | filename = None 899 | status = msg.get('status') 900 | name = msg.get('name') 901 | params = msg.get('params') 902 | if name is not None: 903 | filename = os.path.join(self.result_dir, name) 904 | if self.active_result is not None: 905 | result = rslt.Result(name=name, type=rslt.RESULT_TYPE_FILE, status=status, 906 | filename=filename, params=params) 907 | self.add_result(result) 908 | self.app_conn.send(msg) 909 | elif op == RUN_MSG_STATUS: 910 | pass 911 | elif op == RUN_MSG_CMD: 912 | cmd = msg.get('cmd') 913 | pass 914 | ''' 915 | if cmd == RUN_MSG_CMD_PAUSE and self.process.state == PROC_STATE_RUNNING_PAUSE: 916 | self.state = PROC_STATE_PAUSED 917 | elif cmd == RUN_MSG_CMD_RESUME and self.process.state == PROC_STATE_PAUSED: 918 | self.state = PROC_STATE_RUNNING 919 | ''' 920 | else: 921 | raise SVPError('Unknown run message type: %s' % (type(msg))) 922 | except Exception as e: 923 | entry = LogEntry('Error processing app connection for type {}: {}'.format(type(msg), str(e)), 924 | level=script.ERROR) 925 | self.log(entry.timestamp_str(), entry.level, entry.message) 926 | 927 | count += 1 928 | else: 929 | if self.process and not self.process.is_alive(): 930 | self.clean_up() 931 | break 932 | 933 | if self.active and self.process is None: 934 | self.run_next() 935 | 936 | def add_result(self, result): 937 | self.active_result.add_result(result) 938 | self.results.to_xml_file(self.results_file) 939 | 940 | def update_result(self, name=None, status=None, filename=None, params=None): 941 | print ('update_result: name={} status={} filename={} params={}'.format((name), (status), (filename), 942 | (params))) 943 | self.status = status 944 | if self.active_result is not None: 945 | if name is not None: 946 | self.active_result.name = name 947 | if status is not None: 948 | self.active_result.status = status 949 | if filename is not None: 950 | self.active_result.filename = filename 951 | if params is not None: 952 | self.active_result.params = params 953 | print ('writing results: {}'.format(self.results_file)) 954 | self.results.to_xml_file(self.results_file) 955 | 956 | def alert(self, message): 957 | print ("{}".format(message)) 958 | 959 | def confirm(self, message): 960 | pass 961 | 962 | def log(self, timestamp, level, message): 963 | print ('%s %s %s'.format(timestamp, level, message)) 964 | 965 | 966 | ######################################################################################################### 967 | 968 | SVP_DIR_CONFIG_FILE = '.svp' 969 | 970 | # SVP general configuration file 971 | CONFIG_DIR_ROOT = '.sunspec' 972 | CONFIG_FILE_EXT = '.xml' 973 | 974 | APP_CFG = 'appConfig' 975 | APP_CFG_ATTR_NAME = 'name' 976 | APP_CFG_ATTR_TYPE = 'type' 977 | APP_CFG_DIRS = 'dirs' 978 | APP_CFG_DIR = 'dir' 979 | APP_CFG_ATTR_WORKING = 'working' 980 | APP_CFG_ATTR_VAL_TRUE = 'true' 981 | APP_CFG_REG_PARAMS = 'reg_params' 982 | APP_CFG_PARAM = 'param' 983 | 984 | app_cfg_type = {'str': str, 'int': int, 'float': float, str: 'str', int: 'int', float: 'float'} 985 | 986 | """ 987 | 988 | 989 | C:\\Users\\Fred\\SunSpecTestTool 990 | C:\\Users\Fred\\SomeOtherDir 991 | 992 | 993 | 994 | """ 995 | 996 | def trace_dir(): 997 | user_dir = os.path.expanduser('~') 998 | 999 | # make sure user home directory exists 1000 | if not os.path.exists(user_dir): 1001 | raise SVPError('User home directory %s does not exist' % (user_dir)) 1002 | 1003 | dir = os.path.join(user_dir, CONFIG_DIR_ROOT) 1004 | 1005 | # create base directory if it does not exist 1006 | try: 1007 | os.makedirs(dir) 1008 | except OSError: 1009 | if not os.path.isdir(dir): 1010 | raise 1011 | 1012 | return dir 1013 | 1014 | def config_filename(app_name): 1015 | user_dir = os.path.expanduser('~') 1016 | 1017 | # make sure user home directory exists 1018 | if not os.path.exists(user_dir): 1019 | raise SVPError('User home directory %s does not exist' % (user_dir)) 1020 | 1021 | config_dir = os.path.join(user_dir, CONFIG_DIR_ROOT, app_name) 1022 | 1023 | # create config directory if it does not exist 1024 | try: 1025 | os.makedirs(config_dir) 1026 | except OSError: 1027 | if not os.path.isdir(config_dir): 1028 | raise 1029 | 1030 | return os.path.join(config_dir, app_name + CONFIG_FILE_EXT) 1031 | 1032 | class SVP(object): 1033 | 1034 | def __init__(self, app_id): 1035 | self.app_id = app_id 1036 | self.name = None 1037 | self.dirs = [] 1038 | self.config_file = config_filename('SVP') 1039 | self.run_context = None 1040 | 1041 | # command line related attributes 1042 | self.running = True 1043 | self.dir = None 1044 | self.lib_path = None 1045 | self.result_name = None 1046 | self.suite_params = None 1047 | self.suite = None 1048 | self.env = {} 1049 | 1050 | self.reg_params = {'name': '', 1051 | 'company': '', 1052 | 'email': '', 1053 | 'id': self.app_id, 1054 | 'key': ''} 1055 | 1056 | try: 1057 | self.from_xml(filename=self.config_file) 1058 | except Exception as e: 1059 | pass 1060 | 1061 | def run(self, args=None): 1062 | 1063 | self.run_target(args.get('svp_dir'), args.get('svp_file')) 1064 | 1065 | while self.run_context and self.run_context.active: 1066 | self.run_context.periodic() 1067 | time.sleep(.2) 1068 | 1069 | def run_target(self, svp_dir, svp_file): 1070 | 1071 | if self.run_context is not None: 1072 | raise SVPError('Run context already active') 1073 | 1074 | self.run_context = RunContext(svp_dir, svp_file) 1075 | self.run_context.run() 1076 | 1077 | def from_xml(self, element=None, filename=None): 1078 | if element is None and filename is not None: 1079 | element = ET.ElementTree(file=filename).getroot() 1080 | 1081 | if element is None: 1082 | raise SVPError('No xml document element') 1083 | 1084 | if element.tag != APP_CFG: 1085 | raise SVPError('Unexpected app config root element %s' % (element.tag)) 1086 | 1087 | # self.name = element.attrib.get(APP_CFG_ATTR_NAME) 1088 | 1089 | for e in element.findall('*'): 1090 | if e.tag == APP_CFG_REG_PARAMS: 1091 | for p in e.findall('*'): 1092 | if p.tag == APP_CFG_PARAM: 1093 | k = p.attrib.get(APP_CFG_ATTR_NAME) 1094 | t = app_cfg_type.get(p.attrib.get(APP_CFG_ATTR_TYPE, 'str'), str) 1095 | try: 1096 | v = t(p.text) 1097 | except ValueError: 1098 | pass 1099 | if k and v: 1100 | self.reg_params[k] = v 1101 | 1102 | elif e.tag == APP_CFG_DIRS: 1103 | for d in e.findall('*'): 1104 | if d.tag == APP_CFG_DIR: 1105 | if d.text: 1106 | working = d.attrib.get(APP_CFG_ATTR_WORKING) 1107 | if working == APP_CFG_ATTR_VAL_TRUE: 1108 | working = True 1109 | else: 1110 | working = False 1111 | self.dirs.append(Directory(d.text, working)) 1112 | 1113 | def to_xml(self, parent=None, filename=None): 1114 | attr = {} 1115 | if self.name: 1116 | attr[APP_CFG_ATTR_NAME] = self.name 1117 | 1118 | if parent is not None: 1119 | e = ET.SubElement(parent, APP_CFG, attrib=attr) 1120 | else: 1121 | e = ET.Element(APP_CFG, attrib=attr) 1122 | 1123 | # registration params 1124 | e_reg_params = ET.SubElement(e, APP_CFG_REG_PARAMS) 1125 | for k, v in list(self.reg_params.items()): 1126 | if v: 1127 | attr = {'name': k, 'type': app_cfg_type.get(type(v), 'str')} 1128 | e_param = ET.SubElement(e_reg_params, APP_CFG_PARAM, attrib=attr) 1129 | e_param.text = str(v) 1130 | e_dirs = ET.SubElement(e, APP_CFG_DIRS) 1131 | 1132 | for d in self.dirs: 1133 | attr = {} 1134 | if d.working is True: 1135 | attr[APP_CFG_ATTR_WORKING] = APP_CFG_ATTR_VAL_TRUE 1136 | e_dir = ET.SubElement(e_dirs, APP_CFG_DIR, attrib=attr) 1137 | e_dir.text = d.path 1138 | 1139 | return e 1140 | 1141 | def to_xml_str(self, pretty_print=False): 1142 | e = self.to_xml() 1143 | 1144 | if pretty_print: 1145 | script.xml_indent(e) 1146 | 1147 | return ET.tostring(e, encoding='unicode') 1148 | 1149 | def to_xml_file(self, filename=None, pretty_print=True, replace_existing=True): 1150 | xml = self.to_xml_str(pretty_print) 1151 | 1152 | if filename is not None: 1153 | if replace_existing is False and os.path.exists(filename): 1154 | raise SVPError('File %s already exists' % (filename)) 1155 | f = open(filename, 'w') 1156 | f.write(xml) 1157 | f.close() 1158 | else: 1159 | print (xml) 1160 | 1161 | def config_file_update(self): 1162 | if self.config_file: 1163 | self.to_xml_file(self.config_file) 1164 | 1165 | def add_directory(self, path): 1166 | paths = self.get_directory_paths() 1167 | if path not in paths: 1168 | d = Directory(path) 1169 | self.dirs.append(d) 1170 | self.config_file_update() 1171 | 1172 | def remove_directory(self, path): 1173 | try: 1174 | dir = None 1175 | for d in self.dirs: 1176 | if d.path == path: 1177 | dir = d 1178 | break 1179 | if dir is not None: 1180 | self.dirs.remove(dir) 1181 | self.config_file_update() 1182 | except Exception as e: 1183 | pass 1184 | 1185 | def get_directory_paths(self): 1186 | paths = [] 1187 | for d in self.dirs: 1188 | paths.append(d.path) 1189 | return paths 1190 | 1191 | 1192 | SVP_PROG_NAME = 'SVP' 1193 | 1194 | if __name__ == "__main__": 1195 | # On Windows calling this function is necessary. 1196 | multiprocessing.freeze_support() 1197 | 1198 | app = SVP(1) 1199 | app.run({'svp_dir': 'c:/users/bob/pycharmprojects/svp test/', 1200 | 'svp_file': 'suite_a.ste'}) 1201 | 1202 | 1203 | -------------------------------------------------------------------------------- /doc/.keep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/doc/.keep -------------------------------------------------------------------------------- /doc/CONTRIB.md: -------------------------------------------------------------------------------- 1 | 2 | ## Contribution 3 | 4 | OpenSVP is free and open-source. 5 | The platform has been developped by Sunspec, Sandia and CanmetENERGY -------------------------------------------------------------------------------- /doc/INSTALL.md: -------------------------------------------------------------------------------- 1 | ### Published packages 2 | 3 | | Package | Version | Description | 4 | | ----------------------------------------------------------- | ------- | ---------------------------------------------------------------------- | 5 | | [`@sunspec/pysunspec`][pysunspec-url] | 2.0.0 | Custom library packages needed to run OpenSVP | 6 | ### Software requirements 7 | * Python 3.7+ [Link](https://www.python.org/downloads/) 8 | * Git [Link](https://git-scm.com/download/win) 9 | 10 | ### Install dependencies 11 | 12 | 1. Install Python 3.7+ on your computer or python environment if it hasn't been done yet 13 | 2. Install Git on your computer if not done yet 14 | 3. Download/Clone this OpenSVP repository on your computer 15 | 4. Install missing packages for python with the command displayed: 16 | 17 | The list packages required are included in ./svp_requirements.txt 18 | 19 | Open folder location in an Winders Explorer folder and type cmd in address bar. 20 | This should open a command prompt 21 | Execute the following command: 22 | 23 | ```bash 24 | pip install -r svp_requirements.txt 25 | ``` 26 | 27 | ### Execute OpenSVP 28 | 29 | You will need to create a project folder and create a folder named Lib. 30 | 31 | You will need to copy/clone the svpelab drivers into that folder Lib: [svpelab][svpelab-url] 32 | 33 | After installing all the dependencies, you can execute OpenSVP through commands: 34 | 35 | ```bash 36 | cd /opensvp_dir 37 | python ui.py 38 | ``` 39 | 40 | 41 | [pysunspec-url]: https://github.com/sunspec/pysunspec 42 | [svpelab-url]: https://github.com/BuiMCanmet/svp_energy_lab/tree/dev_canmet_python37 43 | -------------------------------------------------------------------------------- /doc/LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright 2018, SunSpec Alliance 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | -------------------------------------------------------------------------------- /images/OpenSVP.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/OpenSVP.png -------------------------------------------------------------------------------- /images/closed.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/closed.gif -------------------------------------------------------------------------------- /images/closed_128.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/closed_128.gif -------------------------------------------------------------------------------- /images/closed_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/closed_32.gif -------------------------------------------------------------------------------- /images/closed_64.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/closed_64.gif -------------------------------------------------------------------------------- /images/closed_64s.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/closed_64s.gif -------------------------------------------------------------------------------- /images/closed_96.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/closed_96.gif -------------------------------------------------------------------------------- /images/complete.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/complete.gif -------------------------------------------------------------------------------- /images/directory.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/directory.gif -------------------------------------------------------------------------------- /images/error.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/error.gif -------------------------------------------------------------------------------- /images/excel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/excel.png -------------------------------------------------------------------------------- /images/none.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/none.png -------------------------------------------------------------------------------- /images/open.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/open.gif -------------------------------------------------------------------------------- /images/open_64s.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/open_64s.gif -------------------------------------------------------------------------------- /images/open_x.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/open_x.gif -------------------------------------------------------------------------------- /images/page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/page.png -------------------------------------------------------------------------------- /images/page_copy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/page_copy.png -------------------------------------------------------------------------------- /images/pass.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pass.gif -------------------------------------------------------------------------------- /images/pause_0.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_0.gif -------------------------------------------------------------------------------- /images/pause_128.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_128.gif -------------------------------------------------------------------------------- /images/pause_144.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_144.gif -------------------------------------------------------------------------------- /images/pause_160.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_160.gif -------------------------------------------------------------------------------- /images/pause_192.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_192.gif -------------------------------------------------------------------------------- /images/pause_192_255.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_192_255.gif -------------------------------------------------------------------------------- /images/pause_192_255.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_192_255.ico -------------------------------------------------------------------------------- /images/pause_224.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_224.gif -------------------------------------------------------------------------------- /images/pause_224_0.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_224_0.gif -------------------------------------------------------------------------------- /images/pause_224_0.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_224_0.ico -------------------------------------------------------------------------------- /images/pause_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_32.gif -------------------------------------------------------------------------------- /images/pause_64.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_64.gif -------------------------------------------------------------------------------- /images/pause_96.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/pause_96.gif -------------------------------------------------------------------------------- /images/report.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/report.png -------------------------------------------------------------------------------- /images/result.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/result.gif -------------------------------------------------------------------------------- /images/result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/result.png -------------------------------------------------------------------------------- /images/result_dir.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/result_dir.gif -------------------------------------------------------------------------------- /images/result_dir.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/result_dir.png -------------------------------------------------------------------------------- /images/result_dir_p.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/result_dir_p.gif -------------------------------------------------------------------------------- /images/results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results.png -------------------------------------------------------------------------------- /images/results_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_1.png -------------------------------------------------------------------------------- /images/results_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_2.png -------------------------------------------------------------------------------- /images/results_27000.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_27000.gif -------------------------------------------------------------------------------- /images/results_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_3.png -------------------------------------------------------------------------------- /images/results_5400.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_5400.gif -------------------------------------------------------------------------------- /images/results_grey.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_grey.gif -------------------------------------------------------------------------------- /images/results_p.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/results_p.gif -------------------------------------------------------------------------------- /images/run_0.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_0.gif -------------------------------------------------------------------------------- /images/run_128.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_128.gif -------------------------------------------------------------------------------- /images/run_160.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_160.gif -------------------------------------------------------------------------------- /images/run_192.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_192.gif -------------------------------------------------------------------------------- /images/run_224.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_224.gif -------------------------------------------------------------------------------- /images/run_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_32.gif -------------------------------------------------------------------------------- /images/run_64.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_64.gif -------------------------------------------------------------------------------- /images/run_96.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/run_96.gif -------------------------------------------------------------------------------- /images/running.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/running.gif -------------------------------------------------------------------------------- /images/sandia.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/sandia.gif -------------------------------------------------------------------------------- /images/script.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/script.gif -------------------------------------------------------------------------------- /images/script_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/script_32.gif -------------------------------------------------------------------------------- /images/script_dir.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/script_dir.gif -------------------------------------------------------------------------------- /images/scripts.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/scripts.gif -------------------------------------------------------------------------------- /images/stop_0.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_0.gif -------------------------------------------------------------------------------- /images/stop_128.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_128.gif -------------------------------------------------------------------------------- /images/stop_160.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_160.gif -------------------------------------------------------------------------------- /images/stop_192.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_192.gif -------------------------------------------------------------------------------- /images/stop_224.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_224.gif -------------------------------------------------------------------------------- /images/stop_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_32.gif -------------------------------------------------------------------------------- /images/stop_64.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_64.gif -------------------------------------------------------------------------------- /images/stop_96.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stop_96.gif -------------------------------------------------------------------------------- /images/stopped.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/stopped.gif -------------------------------------------------------------------------------- /images/suite.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/suite.gif -------------------------------------------------------------------------------- /images/suite_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/suite_32.gif -------------------------------------------------------------------------------- /images/suite_complete.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/suite_complete.gif -------------------------------------------------------------------------------- /images/suite_dir.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/suite_dir.gif -------------------------------------------------------------------------------- /images/suites.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/suites.gif -------------------------------------------------------------------------------- /images/sunspec_16.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/sunspec_16.gif -------------------------------------------------------------------------------- /images/sunspec_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/sunspec_32.gif -------------------------------------------------------------------------------- /images/sunspec_x.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/sunspec_x.gif -------------------------------------------------------------------------------- /images/sunspec_x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/sunspec_x.png -------------------------------------------------------------------------------- /images/test.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/test.gif -------------------------------------------------------------------------------- /images/test_32.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/test_32.gif -------------------------------------------------------------------------------- /images/test_dir.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/test_dir.gif -------------------------------------------------------------------------------- /images/tests.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/tests.gif -------------------------------------------------------------------------------- /images/working_dir.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/images/working_dir.gif -------------------------------------------------------------------------------- /result.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | 4 | Copyright 2018, SunSpec Alliance 5 | 6 | Licensed under the Apache License, Version 2.0 (the "License"); 7 | you may not use this file except in compliance with the License. 8 | You may obtain a copy of the License at 9 | 10 | http://www.apache.org/licenses/LICENSE-2.0 11 | 12 | Unless required by applicable law or agreed to in writing, software 13 | distributed under the License is distributed on an "AS IS" BASIS, 14 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 | See the License for the specific language governing permissions and 16 | limitations under the License. 17 | 18 | """ 19 | 20 | 21 | 22 | import os 23 | import xml.etree.ElementTree as ET 24 | import csv 25 | import math 26 | import xlsxwriter 27 | import natsort 28 | 29 | RESULT_TYPE_RESULT = 'result' 30 | RESULT_TYPE_SUITE = 'suite' 31 | RESULT_TYPE_TEST = 'test' 32 | RESULT_TYPE_SCRIPT = 'script' 33 | RESULT_TYPE_FILE = 'file' 34 | 35 | type_ext = {RESULT_TYPE_SUITE: '.ste', 36 | RESULT_TYPE_TEST: '.tst', 37 | RESULT_TYPE_SCRIPT: '.py'} 38 | 39 | RESULT_RUNNING = 'Running' 40 | RESULT_STOPPED = 'Stopped' 41 | RESULT_COMPLETE = 'Complete' 42 | RESULT_PASS = 'Pass' 43 | RESULT_FAIL = 'Fail' 44 | 45 | PARAM_TYPE_STR = 'string' 46 | PARAM_TYPE_INT = 'int' 47 | PARAM_TYPE_FLOAT = 'float' 48 | PARAM_TYPE_BOOL = 'bool' 49 | 50 | param_types = {'int': int, 'float': float, 'string': str, 'bool': bool, 51 | int: 'int', float: 'float', str: 'string', bool: 'bool'} 52 | 53 | RESULT_TAG = 'result' 54 | RESULT_ATTR_NAME = 'name' 55 | RESULT_ATTR_TYPE = 'type' 56 | RESULT_ATTR_STATUS = 'status' 57 | RESULT_ATTR_FILENAME = 'filename' 58 | RESULT_PARAMS = 'params' 59 | RESULT_PARAM = 'param' 60 | RESULT_PARAM_ATTR_NAME = 'name' 61 | RESULT_PARAM_ATTR_TYPE = 'type' 62 | RESULT_RESULTS = 'results' 63 | 64 | INDEX_COL_FILE = 0 65 | INDEX_COL_DESC = 1 66 | INDEX_COL_NOTES = 2 67 | 68 | index_hdr = [('File', 30), 69 | ('Description', 80), 70 | ('Notes', 80)] 71 | 72 | XL_COL_WIDTH_DEFAULT = 10 73 | 74 | def xl_col(index): 75 | return chr(index + 65) 76 | 77 | def find_result(results_dir, result_dir): 78 | r_target = None 79 | rlt_name = os.path.split(results_dir)[1] 80 | rlt_file = os.path.join(results_dir, rlt_name) + '.rlt' 81 | path = os.path.normpath(result_dir) 82 | path = path.split(os.sep) 83 | r = Result() 84 | r.from_xml(filename=rlt_file) 85 | r_target = r.find(path) 86 | return r_target 87 | 88 | def result_workbook(file, results_dir, result_dir, index=True): 89 | r = find_result(results_dir, result_dir) 90 | if r is not None: 91 | r.to_xlsx(filename=os.path.join(results_dir, result_dir, file), results_dir=results_dir, index=index, 92 | index_row=0) 93 | else: 94 | raise ResultError('Error creating summary workbook - resource not found: %s %s' % (results_dir, result_dir)) 95 | 96 | 97 | class ResultError(Exception): 98 | pass 99 | 100 | 101 | class Result(object): 102 | 103 | def __init__(self, name=None, type=None, status=None, filename=None, params=None, result_path=None): 104 | self.name = name 105 | self.type = type 106 | self.status = status 107 | self.filename = filename 108 | self.params = [] 109 | self.result_path = result_path 110 | self.ref = None 111 | self.results_index = 0 112 | if params is not None: 113 | self.params = params 114 | else: 115 | self.params = {} 116 | self.results = [] 117 | 118 | def __str__(self): 119 | return self.to_str() 120 | 121 | def find(self, path): 122 | result = None 123 | for r in self.results: 124 | if r.name == path[0]: 125 | if len(path) > 1: 126 | result = r.find(path[1:]) 127 | else: 128 | result = r 129 | return result 130 | 131 | def next_result(self): 132 | if self.results_index < len(self.results): 133 | result = self.results[self.results_index] 134 | self.results_index += 1 135 | return result 136 | 137 | def add_result(self, result): 138 | self.results.append(result) 139 | 140 | def file(self): 141 | return self.name + type_ext.get(self.type, '') 142 | 143 | def to_str(self, indent=''): 144 | s = '%sname = %s type = %s status = %s filename = %s\n%s params = %s\n%s results = \n ' % ( 145 | indent, self.name, self.type, self.status, self.filename, indent, self.params, indent 146 | ) 147 | indent += ' ' 148 | for r in self.results: 149 | s += '%s' % (r.to_str(indent=indent)) 150 | return s 151 | 152 | def from_xml(self, element=None, filename=None): 153 | if element is None and filename is not None: 154 | element = ET.ElementTree(file=filename).getroot() 155 | self.result_path, file = os.path.split(filename) 156 | if element is None: 157 | raise ResultError('No xml document element') 158 | if element.tag != RESULT_TAG: 159 | raise ResultError('Unexpected result root element: %s' % (element.tag)) 160 | self.name = element.attrib.get(RESULT_ATTR_NAME) 161 | self.type = element.attrib.get(RESULT_ATTR_TYPE) 162 | self.status = element.attrib.get(RESULT_ATTR_STATUS) 163 | self.filename = element.attrib.get(RESULT_ATTR_FILENAME) 164 | self.params = {} 165 | self.results = [] 166 | if self.name is None: 167 | raise ResultError('Result name missing') 168 | 169 | for e in element.findall('*'): 170 | if e.tag == RESULT_PARAMS: 171 | for e_param in e.findall('*'): 172 | if e_param.tag == RESULT_PARAM: 173 | name = e_param.attrib.get(RESULT_PARAM_ATTR_NAME) 174 | param_type = e_param.attrib.get(RESULT_PARAM_ATTR_TYPE) 175 | if name: 176 | vtype = param_types.get(param_type, str) 177 | self.params[name] = vtype(e_param.text) 178 | elif e.tag == RESULT_RESULTS: 179 | for e_param in e.findall('*'): 180 | if e_param.tag == RESULT_TAG: 181 | result = Result(result_path=self.result_path) 182 | self.results.append(result) 183 | result.from_xml(e_param) 184 | 185 | def to_xml(self, parent=None, filename=None): 186 | attr = {} 187 | if self.name: 188 | attr[RESULT_ATTR_NAME] = self.name 189 | if self.type: 190 | attr[RESULT_ATTR_TYPE] = self.type 191 | if self.status: 192 | attr[RESULT_ATTR_STATUS] = self.status 193 | if self.filename: 194 | attr[RESULT_ATTR_FILENAME] = self.filename 195 | if parent is not None: 196 | e = ET.SubElement(parent, RESULT_TAG, attrib=attr) 197 | else: 198 | e = ET.Element(RESULT_TAG, attrib=attr) 199 | 200 | e_params = ET.SubElement(e, RESULT_PARAMS) 201 | 202 | params = natsort.natsorted(self.params, key=self.params.get) 203 | for p in params: 204 | value_type = None 205 | value_str = None 206 | attr = {RESULT_PARAM_ATTR_NAME: p} 207 | value = self.params.get(p) 208 | if value is not None: 209 | value_type = param_types.get(type(value), PARAM_TYPE_STR) 210 | value_str = str(value) 211 | 212 | if value_type is not None: 213 | attr[RESULT_PARAM_ATTR_TYPE] = value_type 214 | 215 | e_param = ET.SubElement(e_params, RESULT_PARAM, attrib=attr) 216 | if value_str is not None: 217 | e_param.text = value_str 218 | 219 | e_results = ET.SubElement(e, RESULT_RESULTS) 220 | for r in self.results: 221 | r.to_xml(e_results) 222 | 223 | return e 224 | 225 | def to_xml_str(self, pretty_print=False): 226 | e = self.to_xml() 227 | 228 | if pretty_print: 229 | xml_indent(e) 230 | 231 | return ET.tostring(e, encoding='unicode') 232 | 233 | def to_xml_file(self, filename=None, pretty_print=True, replace_existing=True): 234 | xml = self.to_xml_str(pretty_print) 235 | if filename is None and self.filename is not None: 236 | filename = self.filename 237 | 238 | if filename is not None: 239 | if replace_existing is False and os.path.exists(filename): 240 | raise ResultError('File %s already exists' % (filename)) 241 | f = open(filename, 'w') 242 | f.write(xml) 243 | f.close() 244 | else: 245 | print (xml) 246 | 247 | def to_xlsx(self, wb=None, filename=None, results_dir=None, index=True, index_row=0): 248 | print ('to_xlsx: {} {}'.format (wb, filename)) 249 | result_wb = wb 250 | if result_wb is None: 251 | result_wb = ResultWorkbook(filename=filename) 252 | if index: 253 | result_wb.add_index() 254 | index_row = 1 255 | if self.type == RESULT_TYPE_FILE: 256 | name, ext = os.path.splitext(self.filename) 257 | if ext == '.csv': 258 | index_row = result_wb.add_csv_file(os.path.join(results_dir, self.filename), self.name, 259 | relative_value_names = ['TIME'], params=self.params, 260 | index_row=index_row) 261 | print ('results = {}'.format(self.results)) 262 | for r in self.results: 263 | print ('result in: {}'.format(self.filename)) 264 | index_row = r.to_xlsx(wb=result_wb, results_dir=results_dir, index=index, index_row=index_row) 265 | print ('result out: {}'.format(self.filename)) 266 | if wb is None: 267 | result_wb.close() 268 | 269 | return index_row 270 | 271 | 272 | class ResultWorkbook(object): 273 | 274 | def __init__(self, filename): 275 | self.wb = xlsxwriter.Workbook(filename) 276 | self.ws_index = None 277 | self.hdr_format = self.wb.add_format() 278 | self.link_format = self.wb.add_format({'color': 'blue', 'underline': 1}) 279 | 280 | self.hdr_format.set_text_wrap() 281 | self.hdr_format.set_align('center') 282 | self.hdr_format.set_align('vcenter') 283 | self.hdr_format.set_bold() 284 | 285 | self.link_format.set_align('center') 286 | self.link_format.set_align('vcenter') 287 | 288 | def add_index(self): 289 | print ('add_index') 290 | self.ws_index = self.wb.add_worksheet('Index') 291 | col = 0 292 | for i in range(len(index_hdr)): 293 | width = index_hdr[i][1] 294 | if width: 295 | self.ws_index.set_column(i, i, width) 296 | self.ws_index.write(0, col,index_hdr[i][0], self.hdr_format) 297 | col += 1 298 | 299 | def add_index_entry(self, title, index_row, desc=None, notes=None): 300 | print ('add_index_entry: {}'.format(title)) 301 | self.ws_index.write_url(index_row, INDEX_COL_FILE, 'internal:%s!A1' % (title), 302 | string=title) 303 | if desc is not None: 304 | self.ws_index.write(index_row, INDEX_COL_DESC, desc) 305 | if notes is not None: 306 | self.ws_index.write(index_row, INDEX_COL_NOTES, notes) 307 | return index_row + 1 308 | 309 | 310 | def add_chart(self, ws, params=None, index_row=None): 311 | print ('add chart') 312 | # get fieldnames in first row of worksheet 313 | colors = ['blue', 'green', 'purple', 'orange', 'red', 'brown', 'yellow'] 314 | color_idx = 0 315 | point_names = params.get('plot.point_names', []) 316 | 317 | x_points = [] 318 | y_points = [] 319 | y2_points = [] 320 | if params is not None: 321 | points = params.get('plot.x.points') 322 | if points is not None: 323 | x_points = [x.strip() for x in points.split(',')] 324 | points = params.get('plot.y.points') 325 | if points is not None: 326 | y_points = [x.strip() for x in points.split(',')] 327 | points = params.get('plot.y2.points') 328 | if points is not None: 329 | y2_points = [x.strip() for x in points.split(',')] 330 | 331 | title = params.get('plot.title', '') 332 | # chartsheet = self.wb.add_chartsheet(title) 333 | ws_chart = self.wb.add_worksheet(title) 334 | if index_row is not None: 335 | index_row = self.add_index_entry(title, index_row) 336 | 337 | chart = self.wb.add_chart({'type': 'scatter', 'subtype': 'straight'}) 338 | # chartsheet.set_chart(chart) 339 | ws_chart.insert_chart('A1', chart, {'x_offset': 25, 'y_offset': 10}) 340 | 341 | chart.set_title({'name': title}) 342 | chart.set_size({'width': 1200, 'height': 600}) 343 | chart.set_x_axis({'name': params.get('plot.x.title', '')}) 344 | chart.set_y_axis({'name': params.get('plot.y.title', '')}) 345 | chart.set_y2_axis({'name': params.get('plot.y2.title', '')}) 346 | chart.set_style(2) 347 | print ('ws name = {}'.format(ws.get_name())) 348 | 349 | # chart.x_axis.title = params.get('plot.x.title', '') 350 | # chart.y_axis.title = params.get('plot.y.title', '') 351 | 352 | count = params.get('plot.point_value_count', 1) 353 | ws_name = ws.get_name() 354 | categories = [] 355 | 356 | if len(x_points) > 0: 357 | # only support one x point for now 358 | name = x_points[0] 359 | try: 360 | # col = point_names.index(name) + 1 361 | # categories = [ws_name, 2, 0, count + 1, 0] 362 | col_index = point_names.index(name) 363 | col = xl_col(col_index) 364 | categories = '=%s!$%s$%s:$%s$%s' % (ws_name, col, 2, col, count + 1) 365 | except ValueError: 366 | print ('Value error for x point: {}'.format(name)) 367 | 368 | if len(y_points) > 0: 369 | for name in y_points: 370 | try: 371 | min_error = params.get('plot.%s.min_error' % name) 372 | max_error = params.get('plot.%s.max_error' % name) 373 | print ('min_error, max_error = {} {}'.format(min_error, max_error)) 374 | col_index = point_names.index(name) 375 | col = xl_col(col_index) 376 | line_color = params.get('plot.%s.color' % name, colors[color_idx]) 377 | point = params.get('plot.%s.point' % name, 'False') 378 | if point == 'True': 379 | marker = {'type': 'circle', 380 | 'size': 5, 381 | # 'fill': {'color': line_color} 382 | } 383 | else: 384 | marker = {} 385 | series = { 386 | 'name': name, 387 | 'categories': categories, 388 | # 'values': [ws_name, 2, col, count, col], 389 | 'values': '=%s!$%s$%s:$%s$%s' % (ws_name, col, 2, col, count + 1), 390 | # 'line': {'color': line_color, 'width': 1.5}, 391 | 'line': {'width': 1.5}, 392 | 'marker' : marker, 393 | } 394 | if min_error and max_error: 395 | min_col = xl_col(point_names.index(min_error)) 396 | max_col = xl_col(point_names.index(max_error)) 397 | min_values = '=%s!$%s$%s:$%s$%s' % (ws_name, min_col, 2, min_col, count + 1) 398 | max_values = '=%s!$%s$%s:$%s$%s' % (ws_name, max_col, 2, max_col, count + 1) 399 | print ('min_values = %s'.format(min_values)) 400 | print ('max_values = %s'.format(max_values)) 401 | series['y_error_bars'] = { 402 | 'type': 'custom', 403 | 'direction': 'both', 404 | # 'value': 10 405 | 'plus_values': max_values, 406 | 'minus_values': min_values 407 | } 408 | print ('series = {}'.format(series)) 409 | chart.add_series(series) 410 | color_idx += 1 411 | 412 | except ValueError: 413 | print ('Value error for y1 point: %s'.format (name)) 414 | 415 | if len(y2_points) > 0: 416 | for name in y2_points: 417 | try: 418 | col = point_names.index(name) 419 | line_color = params.get('plot.%s.color' % name, colors[color_idx]) 420 | point = params.get('plot.%s.point' % name, 'False') 421 | if point == 'True': 422 | marker = {'type': 'circle', 423 | 'size': 5, 424 | # 'fill': {'color': line_color} 425 | } 426 | else: 427 | marker = {} 428 | chart.add_series({ 429 | 'name': name, 430 | 'categories': categories, 431 | 'values': [ws_name, 2, col, count, col], 432 | # 'line': {'color': line_color, 'width': 1.5}, 433 | 'line': {'width': 1.5}, 434 | 'marker' : marker, 435 | 'y2_axis': 1 436 | }) 437 | 438 | except ValueError: 439 | print ('Value error for y2 point: {}'.format(name)) 440 | 441 | return index_row 442 | 443 | def add_csv_file(self, filename, title, relative_value_names=None, params=None, index_row=None): 444 | print ('add_csv_file: {}'.format (title)) 445 | col_width = [] 446 | line = 1 447 | ws = self.wb.add_worksheet(title) 448 | if index_row is not None: 449 | index_row = self.add_index_entry(title, index_row) 450 | f = None 451 | relative_value_index = [] 452 | relative_value_start = [] 453 | if relative_value_names is None: 454 | relative_value_names = [] 455 | if params is None: 456 | params = {} 457 | try: 458 | f = open(filename) 459 | ''' 460 | reader = csv.reader(f, skipinitialspace=True) 461 | print 'reader = %s %s' % (filename, reader) 462 | for row in reader: 463 | ''' 464 | print ('filename = {} {}'.format (filename, f)) 465 | for rec in f: 466 | row = [x.strip() for x in rec.split(',')] 467 | # print 'row = %s' % (row) 468 | for i in range(len(row)): 469 | try: 470 | v = float(row[i]) 471 | if math.isnan(v) or math.isinf(v): 472 | row[i] = '' 473 | else: 474 | row[i] = v 475 | except ValueError: 476 | pass 477 | # adjust column width if necessary 478 | width = len(str(row[i])) + 4 479 | if width < XL_COL_WIDTH_DEFAULT: 480 | width = XL_COL_WIDTH_DEFAULT 481 | try: 482 | curr_width = col_width[i] 483 | except IndexError: 484 | curr_width = 0 485 | if width > curr_width: 486 | col_width.insert(i, width) 487 | ws.set_column(i, i, width) 488 | # find fields to be treated as relative value 489 | if line == 1: 490 | params['plot.point_names'] = row 491 | for i in range(len(row)): 492 | width = len(row[i]) + 4 493 | if width < XL_COL_WIDTH_DEFAULT: 494 | width = XL_COL_WIDTH_DEFAULT 495 | ws.set_column(i, i, width) 496 | if relative_value_names is not None: 497 | for name in relative_value_names: 498 | try: 499 | index = row.index(name) 500 | relative_value_index.append(index) 501 | except ValueError: 502 | print ('Value error for relative value name: {}'.format (name)) 503 | # get initial value for relative value fields 504 | elif line == 2: 505 | for index in relative_value_index: 506 | relative_value_start.append(row[index]) 507 | row[index] = 0 508 | else: 509 | for index in relative_value_index: 510 | row[index] = row[index] - relative_value_start[index] 511 | ws.write_row(line - 1, 0, row) 512 | line += 1 513 | params['plot.point_value_count'] = line - 1 514 | 515 | if title[-4:] == '.csv': 516 | chart_title = title[:-4] 517 | else: 518 | chart_title = title + '_chart' 519 | 520 | print ('params - plot: {} - {}'.format(params, params.get('plot.title'))) 521 | if params is not None and params.get('plot.title') is not None: 522 | index_row = self.add_chart(ws, params=params, index_row=index_row) 523 | 524 | except Exception as e: 525 | print ('add_csv_file error: {}'.format(e)) 526 | raise 527 | finally: 528 | if f: 529 | f.close() 530 | 531 | return index_row 532 | 533 | def save(self, filename=None): 534 | pass 535 | 536 | def close(self): 537 | if self.wb is not None: 538 | self.wb.close() 539 | 540 | """ Simple XML pretty print support function 541 | 542 | """ 543 | def xml_indent(elem, level=0): 544 | i = "\n" + level*" " 545 | if len(elem): 546 | if not elem.text or not elem.text.strip(): 547 | elem.text = i + " " 548 | if not elem.tail or not elem.tail.strip(): 549 | elem.tail = i 550 | for elem in elem: 551 | xml_indent(elem, level+1) 552 | if not elem.tail or not elem.tail.strip(): 553 | elem.tail = i 554 | else: 555 | if level and (not elem.tail or not elem.tail.strip()): 556 | elem.tail = i 557 | 558 | if __name__ == "__main__": 559 | 560 | result = Result(name='Result', type='suite') 561 | result1 = Result(name='Result 1', type='test', status='complete') 562 | result1.results.append(Result(name='Result 1 Log', type='log', filename='log/file/name/1')) 563 | result2 = Result(name='Result 2', type='test', status='complete', params={'param 1': 'param 1 value'}) 564 | result2.results.append(Result(name='Result 2 Log', type='log', filename='log/file/name/2')) 565 | result.results.append(result1) 566 | result.results.append(result2) 567 | 568 | xml_str = result.to_xml_str(pretty_print=True) 569 | print (xml_str) 570 | print (result) 571 | print ('-------------------') 572 | result_xml = Result() 573 | root = ET.fromstring(xml_str) 574 | result_xml.from_xml(root) 575 | print (result_xml) 576 | 577 | -------------------------------------------------------------------------------- /script.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | 4 | 5 | """ 6 | 7 | Copyright 2018, SunSpec Alliance 8 | 9 | Licensed under the Apache License, Version 2.0 (the "License"); 10 | you may not use this file except in compliance with the License. 11 | You may obtain a copy of the License at 12 | 13 | http://www.apache.org/licenses/LICENSE-2.0 14 | 15 | Unless required by applicable law or agreed to in writing, software 16 | distributed under the License is distributed on an "AS IS" BASIS, 17 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | See the License for the specific language governing permissions and 19 | limitations under the License. 20 | 21 | """ 22 | 23 | ''' 24 | - update param to param_def 25 | - support index attribute (use dict) 26 | 27 | ''' 28 | 29 | ''' 30 | 31 | An indexed parameter is a parameter that contains multiple values that are referenced by numeric index. 32 | The parameter index has a start value and count. The index start is the index associated with the first 33 | parameter value. The index count is the number of values the parameter contains. 34 | 35 | The parameter values are encoded as a dict with each parameter value represented as a entry where the key 36 | is an integer index and the value is the value corresponding to the index. The dict contains two additional keys: 37 | 'index_start' and 'index_count' whose values are the integer values for the start index and number of 38 | parameter values, respectively. 39 | 40 | 41 | ''' 42 | 43 | 44 | from builtins import input 45 | from builtins import range 46 | 47 | 48 | 49 | 50 | import sys 51 | import os 52 | import time 53 | import datetime 54 | import importlib 55 | import xml.etree.ElementTree as ET 56 | import shlex 57 | import natsort 58 | 59 | import multiprocessing 60 | 61 | try: 62 | # Python 3.4+ 63 | if sys.platform.startswith('win'): 64 | import multiprocessing.popen_spawn_win32 as forking 65 | else: 66 | import multiprocessing.popen_fork as forking 67 | except ImportError: 68 | import multiprocessing.forking as forking 69 | 70 | 71 | version = '1.5.9' 72 | 73 | # log levels 74 | ERROR = 'E' 75 | WARNING = 'W' 76 | INFO = 'I' 77 | DEBUG = 'D' 78 | 79 | # test result codes (should be the same as result.py) 80 | RESULT_COMPLETE = 'Complete' 81 | RESULT_PASS = 'Pass' 82 | RESULT_FAIL = 'Fail' 83 | 84 | PARAM_TYPE_STR = 'string' 85 | PARAM_TYPE_INT = 'int' 86 | PARAM_TYPE_FLOAT = 'float' 87 | 88 | param_types = {'int': int, 'float': float, 'string': str, 89 | int: 'int', float: 'float', str: 'string'} 90 | 91 | param_default = {int: 0, float: 0., str: ''} 92 | 93 | PTYPE_DIR = 'dir' 94 | PTYPE_FILE = 'file' 95 | 96 | PARAM_SEP = '.' 97 | PATH_SEP = '/' 98 | 99 | SCRIPT_PARAM_ROOT = '_root_' 100 | 101 | class _Popen(forking.Popen): 102 | def __init__(self, *args, **kw): 103 | if hasattr(sys, 'frozen'): 104 | # We have to set original _MEIPASS2 value from sys._MEIPASS 105 | # to get --onefile mode working. 106 | os.putenv('_MEIPASS2', sys._MEIPASS) 107 | try: 108 | super(_Popen, self).__init__(*args, **kw) 109 | finally: 110 | if hasattr(sys, 'frozen'): 111 | # On some platforms (e.g. AIX) 'os.unsetenv()' is not 112 | # available. In those cases we cannot delete the variable 113 | # but only set it to the empty string. The bootloader 114 | # can handle this case. 115 | if hasattr(os, 'unsetenv'): 116 | os.unsetenv('_MEIPASS2') 117 | else: 118 | os.putenv('_MEIPASS2', '') 119 | 120 | class Process(multiprocessing.Process): 121 | _Popen = _Popen 122 | 123 | 124 | class ScriptFail(Exception): 125 | pass 126 | 127 | 128 | class ScriptError(Exception): 129 | pass 130 | 131 | 132 | class ScriptParamError(Exception): 133 | pass 134 | 135 | 136 | class ScriptConfigError(Exception): 137 | pass 138 | 139 | 140 | def result_str(result): 141 | return result 142 | 143 | def is_sequence(arg): 144 | return (not hasattr(arg, 'strip') and 145 | hasattr(arg, '__getitem__') or 146 | hasattr(arg, '__iter__') and 147 | not isinstance(arg, str)) 148 | 149 | """ Simple XML pretty print support function 150 | 151 | """ 152 | def xml_indent(elem, level=0): 153 | i = "\n" + level*" " 154 | if len(elem): 155 | if not elem.text or not elem.text.strip(): 156 | elem.text = i + " " 157 | if not elem.tail or not elem.tail.strip(): 158 | elem.tail = i 159 | for elem in elem: 160 | xml_indent(elem, level+1) 161 | if not elem.tail or not elem.tail.strip(): 162 | elem.tail = i 163 | else: 164 | if level and (not elem.tail or not elem.tail.strip()): 165 | elem.tail = i 166 | 167 | def load_script(path, lib_path, path_list = None): 168 | if path_list is not None: 169 | for p in path_list: 170 | sys.path.insert(0, p) 171 | if lib_path is not None: 172 | sys.path.insert(0, lib_path) 173 | script_path, name = os.path.split(path) 174 | name, ext = os.path.splitext(name) 175 | sys.path.insert(0, script_path) 176 | try: 177 | m = importlib.import_module(name) 178 | try: 179 | try: 180 | info = m.script_info() 181 | s = Script(info=info) 182 | except Exception as e: 183 | raise e 184 | # raise ScriptError('%s does not appear to be a script: %s' % (path, str(e))) 185 | finally: 186 | if name in sys.modules: 187 | del sys.modules[name] 188 | if sys.path[0] == script_path: 189 | del sys.path[0] 190 | if lib_path is not None and sys.path[0] == lib_path: 191 | del sys.path[0] 192 | except Exception as e: 193 | raise e 194 | # raise ScriptError('Error importing module %s: %s' % (path, str(e))) 195 | return s 196 | 197 | def check_active_value(value, active_value): 198 | if is_sequence(value): 199 | values = value 200 | else: 201 | values = [value] 202 | for v in values: 203 | if is_sequence(active_value): 204 | if v in active_value: 205 | return v 206 | else: 207 | if v == active_value: 208 | return v 209 | 210 | def param_get_active(param_defs, entry, param_value): 211 | if entry is not None: 212 | if entry.active is not None: 213 | if param_is_active(param_defs, entry.active, param_value) is None: 214 | return 215 | value = param_value(entry.active) 216 | if check_active_value(value, entry.active_value) is None: 217 | # check other entries if present 218 | active_entry = entry.active_entry(value) 219 | if active_entry is None: 220 | return 221 | entry = active_entry 222 | if entry.parent is None or param_is_active(param_defs, entry.parent.qname, param_value): 223 | return entry 224 | 225 | def param_is_active(param_defs, name, param_value): 226 | entry = param_defs._param_get(param_defs, name, param_value) 227 | if entry is not None: 228 | return param_get_active(param_defs, entry, param_value) 229 | 230 | def param_update_ref_values(param_defs, name, value, param_value): 231 | index_count = index_start = None 232 | entry = param_defs._param_get(param_defs, name, param_value) 233 | if entry is not None: 234 | if entry.index_count is not None: 235 | if type(entry.index_count) == str: 236 | index_count = param_value(entry.index_count) 237 | ''' 238 | if index_count is None: 239 | raise ScriptParamError('Unable to resolve param name %s referenced in param %s' % (entry.index_count, 240 | entry.qname)) 241 | ''' 242 | 243 | else: 244 | index_count = entry.index_count 245 | if entry.index_start is not None: 246 | if type(entry.index_start) == str: 247 | index_start = param_value(entry.index_start) 248 | ''' 249 | if index_start is None: 250 | raise ScriptParamError('Unable to resolve param name %s referenced in param %s' % (entry.index_count, 251 | entry.qname)) 252 | ''' 253 | else: 254 | index_start = entry.index_start 255 | 256 | if index_count is not None and index_start is not None: 257 | entry.index_update(index_count, index_start) 258 | 259 | class ScriptInfo(object): 260 | 261 | def __init__(self, name=None, label=None, desc=None, run=None, version=None): 262 | self.name = name 263 | self.label = label 264 | self.desc = desc 265 | self.run = run 266 | self.version = '1.0.0' 267 | self.param_defs = ScriptParamGroupDef(name=SCRIPT_PARAM_ROOT, qname=SCRIPT_PARAM_ROOT) 268 | self.logos = [] 269 | 270 | if not self.label: 271 | self.label = self.name 272 | if version is not None: 273 | self.version = version 274 | 275 | def logo(self, filename): 276 | self.logos.append(filename) 277 | 278 | def param_group(self, name, group=None, label=None, desc=None, active=None, active_value=None, glob=False, 279 | index_count=None, index_start=None): 280 | return self.param_defs.param_group_add(group=group, name=name, label=label, desc=desc, active=active, 281 | active_value=active_value, glob=glob, index_count=index_count, 282 | index_start=index_start) 283 | 284 | def param(self, name, group=None, label=None, default=None, desc=None, values=None, active=None, glob=False, 285 | active_value=None, ptype=None, width=None, index_count=None, index_start=None): 286 | return self.param_defs.param_add(group=group, name=name, label=label, default=default, desc=desc, values=values, 287 | active=active, active_value=active_value, glob=glob, ptype=ptype, width=width, 288 | index_count=index_count, index_start=index_start) 289 | 290 | def param_add_value(self, name, value, sorted=True): 291 | param = self.param_defs.param_def_get(name, self.param_defs, self.param_defs.param_value, active=False) 292 | if param is None: 293 | raise ScriptParamError('param_add_value - unknown param name: %s' % name) 294 | if value not in param.values: 295 | param.values.append(value) 296 | if sorted is True: 297 | param.values.sort() 298 | 299 | 300 | class Script(object): 301 | def __init__(self, env=None, info=None, config=None, config_file=None, params=None): 302 | self.name = None 303 | self.desc = None 304 | self.param_defs = None 305 | self.info = info 306 | self.config = config 307 | self.timers = [] 308 | self.callback = False 309 | if params is None: 310 | self.params = {} 311 | else: 312 | self.params = params 313 | 314 | self._results_dir = '' 315 | self._result_dir = '' 316 | if env is None: 317 | self.env = {} 318 | else: 319 | self.env = env 320 | self._files_dir = env.get('files_dir', '') 321 | self._results_dir = env.get('results_dir', '') 322 | self._result_dir = env.get('result_dir', '') 323 | 324 | # resolve active and references in param defs 325 | if self.info is not None: 326 | self.name = self.info.name 327 | self.desc = self.info.desc 328 | self.param_defs = self.info.param_defs 329 | if self.info.param_defs is not None: 330 | self.info.param_defs.resolve_active(self.info.param_defs, self.param_value) 331 | self.info.param_defs.resolve_refs(self.info.param_defs, self.param_value) 332 | 333 | try: 334 | if config_file is not None: 335 | self.config = ScriptConfig(filename=config_file) 336 | except Exception as e: 337 | self.config = config 338 | self.log('Error loading script config file: %s' % str(e)) 339 | 340 | def alert(self, message): 341 | print (message) 342 | 343 | def config_name(self): 344 | name = '' 345 | if self.config is not None: 346 | name = self.config.name 347 | return name 348 | 349 | def confirm(self, message): 350 | while True: 351 | c = input("%s\nType 'Y' to confirm or 'N' to cancel: " % (str(message))).rstrip('\r\n').lower() 352 | if c == 'y': 353 | return True 354 | elif c == 'n': 355 | return False 356 | 357 | def fail(self, reason): 358 | raise ScriptFail(reason) 359 | 360 | def files_dir(self): 361 | return self._files_dir 362 | 363 | def group_params(self, group=None, params=None): 364 | if params is None: 365 | params = {} 366 | if group is not None: 367 | param_group = self.param_defs.param_group_get(group) 368 | else: 369 | param_group = self.param_defs 370 | if param_group is not None: 371 | for param in param_group.params: 372 | params[param.name] = self.param_value(param.qname) 373 | return params 374 | 375 | def log(self, message, level=INFO): 376 | print ('{} {} {}'.format (datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f'), level, message)) 377 | 378 | def log_active_params(self, param_group=None, config=None, level=0): 379 | if param_group is None: 380 | param_group = self.param_defs 381 | if config is None: 382 | config = self.config 383 | for param in param_group.params: 384 | if param_is_active(self.param_defs, param.qname, self.param_value): 385 | self.log('%s%s = %s' % (' ' * (level * 4), param.label, self.param_value(param.qname))) 386 | for group in param_group.param_groups: 387 | if param_is_active(self.param_defs, group.qname, self.param_value): 388 | self.log('%s%s:' % (' ' * (level * 4), group.label)) 389 | self.log_active_params(param_group=group, config=config, level=level+1) 390 | 391 | def log_debug(self, message): 392 | self.log(message, level=DEBUG) 393 | 394 | def log_error(self, message): 395 | self.log(message, level=ERROR) 396 | 397 | def log_warning(self, message): 398 | self.log(message, level=WARNING) 399 | 400 | def param_get(self, name, param_defs=None, param_value=None): 401 | return self.param_value(name, param_defs, param_value) 402 | 403 | def param_is_global(self, name): 404 | if self.param_defs is not None: 405 | param = self.param_defs.find(name) 406 | if param is not None: 407 | return param.glob 408 | return False 409 | 410 | def param_value(self, name, param_defs=None, param_value=None): 411 | if param_defs is None: 412 | param_defs = self.param_defs 413 | if param_value is None: 414 | param_value = self.param_value 415 | value = self.params.get(name) 416 | if value is None: 417 | if self.config is not None: 418 | value = self.config.param_value(name, param_defs, param_value) 419 | if value is None: 420 | if self.param_defs is not None: 421 | value = self.param_defs.param_value(name, param_defs, param_value) 422 | return value 423 | 424 | def result(self, status=None, params=None): 425 | s = 'Test result' 426 | if status is not None: 427 | s += ' - Status: %s' % (status) 428 | if params is not None: 429 | s += ' - Params: %s' % params 430 | print (s) 431 | 432 | def result_dir(self): 433 | return self._result_dir 434 | 435 | def results_dir(self): 436 | return self._results_dir 437 | 438 | def result_file(self, name=None, status=None, params=None): 439 | s = 'Test result file' 440 | if name is not None: 441 | s += ' - %s' % (name) 442 | if status is not None: 443 | s += ' - Status: %s' % (status) 444 | if params is not None: 445 | s += ' - Params: %s' % (params) 446 | print (s) 447 | 448 | def result_file_path(self, name): 449 | return os.path.join(self.result_dir(), name) 450 | 451 | def resolve_active(self): 452 | if self.param_defs: 453 | self.param_defs.resolve_active(self.param_defs, self.param_value) 454 | 455 | def resolve_refs(self): 456 | if self.param_defs: 457 | self.param_defs.resolve_refs(self.param_defs, self.param_value) 458 | 459 | ''' 460 | Every entry is considered before any entry gets another chance 461 | A complete pass thorough the timeout list is performed for any sleep call 462 | sleep() can not be called in a timer callback routine 463 | ''' 464 | def sleep(self, seconds): 465 | if self.callback is True: 466 | raise ScriptError('Can not call sleep() from callback function') 467 | current_time = time.time() 468 | wake_time = current_time + seconds 469 | sleep_time = wake_time - current_time 470 | while sleep_time > 0: 471 | timers = self.timers[:] 472 | for t in timers: 473 | next = t.next_timeout - current_time 474 | if next <= 0: 475 | self.callback = True 476 | t.callback(t.arg) 477 | self.callback = False 478 | if t.repeating is False: 479 | self.timer_cancel(t) 480 | else: 481 | t.next_timeout += t.period 482 | next = t.next_timeout - current_time 483 | if next < sleep_time: 484 | sleep_time = next 485 | time.sleep(sleep_time) 486 | current_time = time.time() 487 | sleep_time = wake_time - current_time 488 | 489 | def timer_cancel(self, timer): 490 | self.timers.remove(timer) 491 | 492 | def timer_start(self, period, callback, arg=None, repeating=False): 493 | timer = ScriptTimer(period, callback, arg=arg, repeating=repeating) 494 | self.timers.append(timer) 495 | return timer 496 | 497 | def svp_version(self, required=None): 498 | if required is not None: 499 | required_version = required.split('.') 500 | if len(required_version) > 3: 501 | raise ScriptError('Invalid version format for required version: %s' % (required)) 502 | current_version = version.split('.') 503 | for i in range(len(required_version)): 504 | if int(current_version[i]) > int(required_version[i]): 505 | break 506 | elif int(current_version[i]) < int(required_version[i]): 507 | raise ScriptError('Current SVP version %s is older than required version %s' % (version, required)) 508 | return version 509 | 510 | 511 | class ScriptTimer(object): 512 | def __init__(self, period, callback, arg, repeating=False): 513 | self.period = period 514 | self.callback = callback 515 | self.arg = arg 516 | self.repeating = repeating 517 | self.count = 1 518 | self.next_timeout = time.time() + period 519 | 520 | 521 | class ScriptParamDef(object): 522 | 523 | def __init__(self, parent=None, name=None, qname=None, label=None, default=None, desc=None, values=None, 524 | active=None, active_value=None, glob=False, ptype=None, width=None, index_count=None, 525 | index_start=None): 526 | self.parent = parent 527 | self.name = name 528 | self.qname = qname 529 | self.label = label 530 | self.desc = desc 531 | self.active = active 532 | self.active_value = active_value 533 | self.default = default 534 | self.values = [] 535 | if values is not None: 536 | if isinstance(values, list): 537 | self.values = values 538 | else: 539 | self.values = [values] 540 | self.glob = glob 541 | self.referenced = False 542 | self.ptype = ptype 543 | self.width = width 544 | self.vtype = None 545 | self.index_count = index_count 546 | self.index_start = index_start 547 | self.value = self.default 548 | self.entries = [] 549 | 550 | # default index start to 0 if indexing active 551 | if self.index_count is not None: 552 | if self.index_start is None: 553 | self.index_start = 0 554 | if type(self.index_count) != str and type(self.index_start) != str: 555 | self.value = {'index_count': self.index_count, 'index_start': self.index_start} 556 | for i in range(self.index_start, self.index_start + self.index_count): 557 | if type(self.default) == dict: 558 | value = self.default.get(i) 559 | if value is not None: 560 | if self.vtype is None: 561 | self.vtype = type(value) 562 | else: 563 | if self.vtype != type(value): 564 | ### error - multiple value types in parameter 565 | pass 566 | else: 567 | value = param_default.get(self.vtype) 568 | self.value[i] = value 569 | else: 570 | self.value[i] = self.default 571 | if self.vtype is None: 572 | self.vtype = type(self.default) 573 | else: 574 | self.value = {} 575 | else: 576 | self.vtype = type(self.default) 577 | 578 | def active_entry(self, value): 579 | active_entry = self 580 | if self.active and check_active_value(value, self.active_value) is None: 581 | # check other entries if present 582 | active_entry = None 583 | for e in self.entries: 584 | if check_active_value(value, e.active_value) is not None: 585 | active_entry = e 586 | break 587 | return active_entry 588 | 589 | def index_update(self, index_count, index_start): 590 | self.value = {'index_count': index_count, 'index_start': index_start} 591 | for i in range(index_start, index_start + index_count): 592 | if type(self.default) == dict: 593 | value = self.default.get(i) 594 | if value is not None: 595 | if self.vtype is None: 596 | self.vtype = type(value) 597 | else: 598 | if self.vtype != type(value): 599 | ### error - multiple value types in parameter 600 | pass 601 | else: 602 | value = param_default.get(self.vtype) 603 | self.value[i] = value 604 | else: 605 | self.value[i] = self.default 606 | if self.vtype is None: 607 | self.vtype = type(self.default) 608 | 609 | def dump(self, indent=''): 610 | return '%sparam - name: %s label: %s default: %s active: %s active_value: %s desc: %s values: %s referenced: %s ptype: %s width: %s' % (indent, 611 | self.name, self.label, str(self.default), self.active, self.active_value, self.desc, str(self.values), str(self.referenced), 612 | str(self.ptype), str(self.width)) 613 | 614 | def __str__(self): 615 | 616 | return 'name: %s label: %s default: %s desc: %s values: %s referenced: %s ptype: %s width: %s' % (self.name, self.label, str(self.default), self.desc, str(self.values), 617 | str(self.referenced), str(self.ptype), str(self.width)) 618 | 619 | class ScriptParamGroupDef(object): 620 | 621 | def __init__(self, parent=None, name=None, qname=None, label=None, desc=None, active=None, active_value=None, 622 | glob=False, index_count=None, index_start=None): 623 | self.parent = parent 624 | self.name = name 625 | self.qname = qname 626 | self.label = label 627 | self.desc = desc 628 | self.active = active 629 | self.active_value = active_value 630 | self.glob = glob 631 | self.entries = [] 632 | self.param_groups = [] 633 | self.params = [] 634 | self.index_count = index_count 635 | self.index_start = index_start 636 | 637 | # default index start to 0 if indexing active 638 | if self.index_count is not None: 639 | if self.index_start is None: 640 | self.index_start = 0 641 | 642 | def param_group_find(self, name): 643 | if name == self.name: 644 | return self 645 | for g in self.param_groups: 646 | if g.name == name: 647 | return g 648 | 649 | def param_find(self, name): 650 | for p in self.params: 651 | if p.name == name: 652 | return p 653 | 654 | def find(self, name): 655 | e = self.param_group_find(name) 656 | if e is None: 657 | e = self.param_find(name) 658 | return e 659 | 660 | def param_group_find_active(self, param_defs, name, param_value): 661 | group = None 662 | if name == self.name: 663 | group = self 664 | else: 665 | for g in self.param_groups: 666 | if g.name == name: 667 | group = g 668 | if group: 669 | return param_get_active(param_defs, group, param_value) 670 | 671 | def param_find_active(self, param_defs, name, param_value): 672 | param = None 673 | for p in self.params: 674 | if p.name == name: 675 | param = p 676 | if param: 677 | return param_get_active(param_defs, param, param_value) 678 | 679 | def find_active(self, param_defs, name, param_value): 680 | e = self.param_group_find_active(param_defs, name, param_value) 681 | if e is None: 682 | e = self.param_find_active(param_defs, name, param_value) 683 | return e 684 | 685 | def param_group_get(self, name): 686 | if name is None: 687 | return 688 | group = self 689 | path = name.split(PARAM_SEP) 690 | for i in range(len(path) - 1): 691 | group = group.param_group_find(path[i]) 692 | if group is None: 693 | return 694 | return group.param_group_find(path[-1]) 695 | 696 | #def param_def_get(self, name, param_defs, param_value): 697 | def param_def_get(self, name, param_defs, param_value=None, active=True): 698 | # print 'param_def_get: %s' % name 699 | if name is None: 700 | return 701 | group = self 702 | path = name.split(PARAM_SEP) 703 | for i in range(len(path) - 1): 704 | if active: 705 | group = group.param_group_find_active(param_defs, path[i], param_value) 706 | else: 707 | group = group.param_group_find(path[i]) 708 | if group is None: 709 | return 710 | # print 'returning %s %s' % (path[-1], group.param_find_active(param_defs, path[-1], param_value)) 711 | if active: 712 | param_def = group.param_find_active(param_defs, path[-1], param_value) 713 | else: 714 | param_def = group.param_find(path[-1]) 715 | return param_def 716 | 717 | def _param_get(self, param_defs, name, param_value): 718 | if name is None: 719 | return 720 | group = self 721 | path = name.split(PARAM_SEP) 722 | for i in range(len(path) - 1): 723 | group = group.param_group_find_active(param_defs, path[i], param_value) 724 | if group is None: 725 | return 726 | return group.find_active(param_defs, path[-1], param_value) 727 | 728 | def param_value(self, name, param_defs=None, param_value=None): 729 | if param_defs is None: 730 | param_defs = self 731 | if param_value is None: 732 | param_value = self.param_value 733 | param_def = self.param_def_get(name, param_defs, param_value) 734 | if param_def is not None: 735 | return param_def.value 736 | 737 | def param_group_add(self, group=None, name=None, label=None, desc=None, active=None, active_value=None, glob=False, 738 | index_count=None, index_start=None): 739 | if name is None: 740 | raise ScriptParamError('Missing parameter group name') 741 | group = self 742 | path = name.split(PARAM_SEP) 743 | group_name = path[-1] 744 | if not group_name: 745 | raise ScriptParamError('Missing parameter group name') 746 | for i in range(len(path) - 1): 747 | group = group.param_group_find(path[i]) 748 | if group is None: 749 | raise ScriptParamError('Parameter group not found: %s' % PARAM_SEP.join(path[:i+1])) 750 | if group.find(group_name): 751 | raise ScriptParamError('Duplicate parameter name: %s' % (name)) 752 | g = ScriptParamGroupDef(parent=group, name=group_name, qname=name, label=label, desc=desc, active=active, 753 | active_value=active_value, glob=glob, index_count=index_count, index_start=index_start) 754 | group.param_groups.append(g) 755 | 756 | def param_add(self, group=None, parent=None, name=None, label=None, default=None, desc=None, values=None, 757 | active=None, active_value=None, glob=False, ptype=None, width=None, index_count=None, 758 | index_start=None): 759 | if name is None: 760 | raise ScriptParamError('Missing parameter name') 761 | group = self 762 | path = name.split(PARAM_SEP) 763 | param_name = path[-1] 764 | if not param_name: 765 | raise ScriptParamError('Missing parameter name') 766 | for i in range(len(path) - 1): 767 | group = group.param_group_find(path[i]) 768 | if group is None: 769 | raise ScriptParamError('Parameter group not found: %s' % PARAM_SEP.join(path[:i+1])) 770 | if group.find(param_name): 771 | raise ScriptParamError('Duplicate parameter name: %s' % (name)) 772 | if group.index_count is not None: 773 | index_count = group.index_count 774 | index_start = group.index_start 775 | g = ScriptParamDef(parent=group, name=param_name, qname=name, label=label, default=default, desc=desc, 776 | values=values, active=active, active_value=active_value, glob=glob, ptype=ptype, 777 | width=width, index_count=index_count, index_start=index_start) 778 | group.params.append(g) 779 | 780 | def resolve_active(self, param_defs, param_value): 781 | for g in self.param_groups: 782 | g.resolve_active(param_defs, param_value) 783 | 784 | for p in self.params: 785 | if p.active is not None: 786 | param = param_defs.param_def_get(p.active, param_defs, param_value) 787 | if param is not None: 788 | param.referenced = True 789 | ''' 790 | else: 791 | raise ScriptParamError('Unable to resolve param name %s referenced in param %s' % (p.active, p.qname)) 792 | ''' 793 | 794 | def resolve_refs(self, param_defs, param_value): 795 | for g in self.param_groups: 796 | g.resolve_refs(param_defs, param_value) 797 | 798 | for p in self.params: 799 | param = None 800 | if type(p.index_count) == str: 801 | param = param_defs.param_def_get(p.index_count, param_defs=param_defs, param_value=param_value) 802 | if param is not None: 803 | param.referenced = True 804 | if type(p.index_start) == str: 805 | param = param_defs.param_def_get(p.index_start, param_defs=param_defs, param_value=param_value) 806 | if param is not None: 807 | param.referenced = True 808 | if param is not None: 809 | param_update_ref_values(param_defs, p.qname, p.value, param_defs.param_value) 810 | 811 | def active_entry(self, value): 812 | active_entry = self 813 | if self.active and check_active_value(value, self.active_value) is None: 814 | # check other entries if present 815 | active_entry = None 816 | for e in self.entries: 817 | if check_active_value(value, e.active_value) is not None: 818 | active_entry = e 819 | break 820 | return active_entry 821 | 822 | def dump(self, indent=''): 823 | s = '%sparam group - name: %s label: %s desc: %s active: %s active_value: %s\n' % (indent, 824 | self.name, self.label, self.desc, self.active, str(self.active_value)) 825 | indent += ' ' 826 | for group in self.param_groups: 827 | s += '%s\n' % group.dump(indent) 828 | for param in self.params: 829 | s += '%s\n' % param.dump(indent) 830 | return s 831 | 832 | def params_from_xml(params, element): 833 | for e in element.findall('*'): 834 | if e.tag == SCRIPT_CFG_PARAMS: 835 | for e_param in e.findall('*'): 836 | if e_param.tag == SCRIPT_PARAM: 837 | name = e_param.attrib.get(SCRIPT_PARAM_ATTR_NAME) 838 | param_type = e_param.attrib.get(SCRIPT_PARAM_ATTR_TYPE) 839 | count = e_param.attrib.get(SCRIPT_PARAM_INDEX_COUNT) 840 | start = e_param.attrib.get(SCRIPT_PARAM_INDEX_START) 841 | if name: 842 | vtype = param_types.get(param_type, str) 843 | if count is not None and start is not None: 844 | count = int(count) 845 | start = int(start) 846 | value = {'index_count': count, 'index_start': start} 847 | values = shlex.split(e_param.text) 848 | i = start 849 | for v in values: 850 | value[i] = vtype(v) 851 | i += 1 852 | if len(values) != count: 853 | ### count/value mismatch 854 | pass 855 | else: 856 | value = vtype(e_param.text) 857 | params[name] = value 858 | 859 | def params_to_xml(params, parent=None): 860 | if parent is not None: 861 | e_params = ET.SubElement(parent, SCRIPT_CFG_PARAMS) 862 | else: 863 | e_params = ET.Element(SCRIPT_CFG_PARAMS) 864 | sorted_params = natsort.natsorted(params, key=params.get) 865 | 866 | for p in sorted_params: 867 | value_type = None 868 | value_str = None 869 | attr = {SCRIPT_PARAM_ATTR_NAME: p} 870 | value = params.get(p) 871 | if type(value) == dict: 872 | start = value.get('index_start') 873 | count = value.get('index_count') 874 | attr['index_start'] = str(start) 875 | attr['index_count'] = str(count) 876 | if count is not None and start is not None: 877 | value_str = '' 878 | value_type = None 879 | for i in range(start, start + count): 880 | v = value.get(i) 881 | if value_type is None and v is not None: 882 | value_type = param_types.get(type(v), PARAM_TYPE_STR) 883 | v_str = str(v) 884 | if ' ' in v_str: 885 | v_str = '"%s"' % (v_str) 886 | value_str += '%s ' % (v_str) 887 | else: 888 | ### error unknown dict value type 889 | pass 890 | else: 891 | if value is not None: 892 | value_type = param_types.get(type(value), PARAM_TYPE_STR) 893 | value_str = str(value) 894 | 895 | if value_type is not None: 896 | attr[SCRIPT_PARAM_ATTR_TYPE] = value_type 897 | 898 | e_param = ET.SubElement(e_params, SCRIPT_PARAM, attrib=attr) 899 | if value_str is not None: 900 | e_param.text = value_str 901 | 902 | return e_params 903 | 904 | # script config xml elements and attributes 905 | SCRIPT_CFG = 'scriptConfig' 906 | SCRIPT_CFG_ATTR_NAME = 'name' 907 | SCRIPT_CFG_ATTR_SCRIPT = 'script' 908 | SCRIPT_CFG_DESC = 'desc' 909 | SCRIPT_CFG_PARAMS = 'params' 910 | SCRIPT_PARAM = 'param' 911 | SCRIPT_PARAM_ATTR_NAME = 'name' 912 | SCRIPT_PARAM_ATTR_LABEL = 'label' 913 | SCRIPT_PARAM_ATTR_TYPE = 'type' 914 | SCRIPT_PARAM_DESC = 'desc' 915 | SCRIPT_PARAM_INDEX_COUNT = 'index_count' 916 | SCRIPT_PARAM_INDEX_START = 'index_start' 917 | 918 | class ScriptConfig(object): 919 | 920 | def __init__(self, name=None, script=None, desc=None, params=None, filename=None): 921 | self.name = name 922 | self.script = script 923 | self.desc = desc 924 | self.params = None 925 | self.filename = filename 926 | 927 | if params is None: 928 | self.params = {} 929 | else: 930 | self.params = params.copy() 931 | 932 | if filename: 933 | try: 934 | self.from_xml(filename=filename) 935 | except Exception as e: 936 | raise ScriptConfigError('Error scanning script configuration file {}: {}'.format(filename, str(e))) 937 | 938 | def param_value(self, name, param_defs=None, param_value=None): 939 | return self.params.get(name) 940 | 941 | def param_add_default(self, script, param_group_def): 942 | for g in param_group_def.param_groups: 943 | self.param_add_default(script, g) 944 | for p in param_group_def.params: 945 | if param_is_active(script.param_defs, p.qname, script.param_value): 946 | #if script.param_is_active(p.qname): 947 | self.params[p.qname] = p.value 948 | 949 | def from_xml(self, element=None, filename=None): 950 | if element is None and filename is not None: 951 | element = ET.ElementTree(file=filename).getroot() 952 | if element is None: 953 | raise ScriptConfigError('No xml document element') 954 | if element.tag != SCRIPT_CFG: 955 | raise ScriptConfigError('Unexpected script config root element %s' % (element.tag)) 956 | self.name = element.attrib.get(SCRIPT_CFG_ATTR_NAME) 957 | self.script = element.attrib.get(SCRIPT_CFG_ATTR_SCRIPT) 958 | if self.name is None: 959 | raise ScriptConfigError('Script configuration name missing') 960 | if self.script is None: 961 | raise ScriptConfigError('Script name missing') 962 | # self.desc = element.attrib.get(TEST_CFG_ATTR_DESC) 963 | 964 | params_from_xml(self.params, element) 965 | 966 | def params_to_xml(self, parent=None): 967 | if parent is not None: 968 | e_params = ET.SubElement(parent, SCRIPT_CFG_PARAMS) 969 | else: 970 | e_params = ET.Element(SCRIPT_CFG_PARAMS) 971 | params = natsort.natsorted(self.params, key= self.params.get) 972 | 973 | for p in params: 974 | value_type = None 975 | value_str = None 976 | attr = {SCRIPT_PARAM_ATTR_NAME: p} 977 | value = self.params.get(p) 978 | if type(value) == dict: 979 | start = value.get('index_start') 980 | count = value.get('index_count') 981 | attr['index_start'] = str(start) 982 | attr['index_count'] = str(count) 983 | if count is not None and start is not None: 984 | value_str = '' 985 | value_type = None 986 | for i in range(start, start + count): 987 | v = value.get(i) 988 | if value_type is None and v is not None: 989 | value_type = param_types.get(type(v), PARAM_TYPE_STR) 990 | v_str = str(v) 991 | if ' ' in v_str: 992 | v_str = '"%s"' % (v_str) 993 | value_str += '%s ' % (v_str) 994 | else: 995 | raise ScriptConfigError('Script configuration error: count = %s start = %s' % (count, start)) 996 | else: 997 | if value is not None: 998 | value_type = param_types.get(type(value), PARAM_TYPE_STR) 999 | value_str = str(value) 1000 | 1001 | if value_type is not None: 1002 | attr[SCRIPT_PARAM_ATTR_TYPE] = value_type 1003 | 1004 | e_param = ET.SubElement(e_params, SCRIPT_PARAM, attrib=attr) 1005 | if value_str is not None: 1006 | e_param.text = value_str 1007 | 1008 | return e_params 1009 | 1010 | def to_xml(self, parent=None, filename=None): 1011 | attr = {} 1012 | if self.name: 1013 | attr[SCRIPT_CFG_ATTR_NAME] = self.name 1014 | if self.script: 1015 | attr[SCRIPT_CFG_ATTR_SCRIPT] = self.script 1016 | if parent is not None: 1017 | e = ET.SubElement(parent, SCRIPT_CFG, attrib=attr) 1018 | else: 1019 | e = ET.Element(SCRIPT_CFG, attrib=attr) 1020 | 1021 | params_to_xml(self.params, e) 1022 | 1023 | return e 1024 | 1025 | def to_xml_str(self, pretty_print=False): 1026 | e = self.to_xml() 1027 | 1028 | if pretty_print: 1029 | xml_indent(e) 1030 | 1031 | return ET.tostring(e, encoding='unicode') 1032 | 1033 | def to_xml_file(self, filename=None, pretty_print=True, replace_existing=True): 1034 | xml = self.to_xml_str(pretty_print) 1035 | if filename is None and self.filename is not None: 1036 | filename = self.filename 1037 | 1038 | if filename is not None: 1039 | if replace_existing is False and os.path.exists(filename): 1040 | raise ScriptConfigError('File %s already exists' % (filename)) 1041 | f = open(filename, 'w') 1042 | f.write(xml) 1043 | f.close() 1044 | else: 1045 | print(xml) 1046 | -------------------------------------------------------------------------------- /sunspec_x.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/sunspec_x.ico -------------------------------------------------------------------------------- /svp_requirements.txt: -------------------------------------------------------------------------------- 1 | numpy 2 | pandas 3 | cycler 4 | et-xmlfile 5 | future 6 | iso8601 7 | jdcal 8 | kiwisolver 9 | matplotlib 10 | openpyxl 11 | Pillow 12 | prettytable 13 | PyDAQmx 14 | pymodbus 15 | pyparsing 16 | pyserial 17 | pysunspec 18 | pysunspec2 19 | python-dateutil 20 | pytz 21 | PyVISA 22 | PyYAML 23 | serial 24 | six 25 | typing 26 | wxmplot 27 | wxPython==4.0.7.post2 28 | XlsxWriter 29 | natsort 30 | -------------------------------------------------------------------------------- /testing/.keep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sunspec/svp/d2a56c31b13da10c15ee4fe4613eed18fa31f80e/testing/.keep --------------------------------------------------------------------------------