├── .gitignore ├── LICENSE ├── MANIFEST.in ├── README.md ├── SikuliXLibrary ├── __init__.py ├── requirements.txt ├── sikulixapp.py ├── sikulixdebug.py ├── sikuliximagepath.py ├── sikulixjclass.py ├── sikulixlibrary.py ├── sikulixlogger.py ├── sikulixpy4j.py ├── sikulixregion.py ├── sikulixsettings.py └── version.py ├── migrate ├── ImageHorizonLibraryMigration.py ├── SikuliLibraryMigration.py └── SikuliXCustomLibrary.py ├── setup.py └── test ├── __init__.robot ├── debug test.py ├── img ├── MacOS │ ├── NewDoc.png │ ├── TextEdit edited.png │ ├── TextEdit menu.png │ ├── TextEdit mod.png │ ├── TextEdit typed.png │ ├── TextEdit window.png │ ├── TextEdit window2.png │ ├── TextEdit.png │ └── TextEdit2.png ├── Ubuntu │ ├── Leafpad menu.png │ ├── Leafpad mod.png │ ├── Leafpad typed.png │ ├── Leafpad.png │ └── Leafpad2.png └── Windows │ ├── iNotepad menu.PNG │ ├── iNotepad mod.PNG │ ├── iNotepad typed.PNG │ ├── iNotepad.PNG │ └── iNotepad2.PNG ├── test.py ├── test_defaultlibrary_osx.robot ├── test_defaultlibrary_ubuntu.robot ├── test_defaultlibrary_win.robot ├── test_imagehorizonlibrarymigration_win.robot ├── test_sikulilibrarymigration_win.robot ├── test_sikulixcustomlibrary_win.robot ├── testlibrary_osx.py ├── testlibrary_ubuntu.py └── testlibrary_win.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | results/ 3 | build/ 4 | robotframework_sikulixlibrary.egg-info/ 5 | dist/ 6 | .vscode/ -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Adrian V. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include LICENSE 2 | include README.md 3 | 4 | recursive-exclude test * -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # robotframework-sikulixlibrary 2 | The all new, modern, SikuliX Robot Framework library for Python 3.x, based on JPype or Py4J Python modules. 3 | 4 | It can be used with e.g. Robot Framework projects (https://robotframework.org), Robocorp projects (https://robocorp.com) - the easiest approach for beginners or with pure Python projects. Either for Test Automation type of projects, or for open source RPA (Robot Process Automation) projects. 5 | 6 | [JPype](https://github.com/jpype-project/jpype) is a Python module to provide full access to Java from within Python. 7 | 8 | [Py4J](https://github.com/bartdag/py4j) enables Python programs running in a Python interpreter to dynamically access Java objects in a JVM. 9 | 10 | This library is a wrapper to SikuliX that is exposing Java functions as Robot Framework keywords, and it can be enabled to use by 11 | choice any of the JPype or Py4J modules. This is done by creating SIKULI_PY4J environment variable and setting to 1. When not defined or 12 | set to 0, JPype is used instead. Please note that on MacOS, only Py4J can be used, while on Windows or Ubuntu, any of them is working. 13 | 14 | While in the past the only approach to use Sikuli functionality within Robot Framework was through Remote Server with XML RPC interface ([SikuliLibrary](https://github.com/rainmanwy/robotframework-SikuliLibrary/)), the aim 15 | of this library is to replace that approach and make it a lot easier to use SikuliX within Robot Framework projects with a simple Library statement 16 | (i.e. no need to start remote server and so on). 17 | 18 | Also with this implementation is very easy to extend the library with new custom keywords, for example with the purpose to 19 | create migration classes to help migrate from current Sikuli libraries or other image recognition alternatives. For practical examples check migrate folder. 20 | 21 | See [keyword documentation](https://adrian-evo.github.io/SikuliXLibrary.html). 22 | 23 | # Installation instructions (Windows) 24 | 25 | 1. Python 3.8 or newer, as supported by JPype or Py4J 26 | 2. JPype 1.4.1 or newer and JPype project dependencies as explained on project page: https://github.com/jpype-project/jpype 27 | - Install Java 8 or newer 28 | - While not mentioned on JPype page, on a new Windows 10 machine also Visual C++ Redistributable 2015 and newer are needed (e.g. vc_redist.x64.exe) 29 | 3. or Py4J 0.10.9.7 or newer 30 | 4. SikuliX as a standalone jar from project page: https://raiman.github.io/SikuliX1/downloads.html 31 | - Put jar file in any local directory (e.g. C:\sikulix\sikulix.jar) 32 | - Install Tesseract 4 for using OCR functionality 33 | - Py4J server is enabled from SikuliX 2.0.5 onward and currently advertised as experimental. However, this library is working as expected with Py4J. 34 | - Recommended to use environment variable SIKULI_HOME that point to sikulix local directory 35 | 5. `pip install robotframework-sikulixlibrary` 36 | 6. Check Known issues section below if libspec is not generated within your environment (RF IDE tool) and thus library keywords are not recognised 37 | 38 | While JPype JVM is always started automatically, Py4J JVM can be started manually or automatically. To start manually, use the command: 39 | 40 | `java -jar sikulix.jar -p` (to start Py4J server) or 41 | `java -jar -DsikuliDebug=3 sikulixide.jar -p` (useful e.g. for checking sikulix debug info) 42 | 43 | # Installation instructions (MacOS) 44 | 45 | Mainly the same steps as for Windows above. However, installing Tesseract 4 seems to be challenging since e.g. `brew install tesseract` will install Tesseract 5. But at least the following method from [stackoverflow](https://stackoverflow.com/questions/3987683/homebrew-install-specific-version-of-formula/7787703#7787703) for installing Tesseract 4.1.3 is working. 46 | 47 | # Examples 48 | 49 | ### Testing with [Robot Framework](https://robotframework.org) 50 | ```RobotFramework 51 | *** Settings *** 52 | Library SikuliXLibrary sikuli_path=sikulixide-2.0.5.jar 53 | 54 | *** Test Cases *** 55 | Example Test 56 | imagePath add ${my_path} 57 | settings set MinSimilarity ${0.9} 58 | app open C:/Windows/System32/notepad.exe 59 | region wait iNotepad.PNG 60 | region paste Welcome! 61 | ``` 62 | 63 | ### Testing with [Python](https://python.org). 64 | ```python 65 | from SikuliXLibrary import SikuliXLibrary 66 | sikuli_path = 'sikulixide-2.0.5.jar' 67 | lib = SikuliXLibrary(sikuli_path) 68 | lib.imagePath_add('my_path') 69 | lib.settings_set('MinSimilarity', float(0.9)) 70 | lib.app_open("C:\\Windows\\System32\\notepad.exe") 71 | lib.region_wait('iNotepad') 72 | lib.region_paste('Welcome!) 73 | ``` 74 | 75 | # Testing 76 | Git clone, and if not using pip install for this library, then just point PYTHONPATH to local robotframework-sikulixlibrary folder and execute: 77 | 78 | `python testlibrary_win.py` (or any .py file from under test directory and for OS of choice 79 | 80 | `robot --outputdir results/default test_defaultlibrary_win.robot` (or any .robot file from under test directory and for OS of choice) 81 | 82 | or maybe on MacOS 83 | 84 | `python3 -m robot --outputdir results/default test_defaultlibrary_osx.robot` 85 | 86 | Obviously, image files from test/img/MacOS, Ubuntu or Windows might not work on specific environment and would need to be regenerated. Also for these tests SIKULI_PATH is defined and the name of SikuliX is `sikulixide-2.0.5.jar` 87 | 88 | Additionally, debugging with some RF supported tools is also possible with this library, for both Robot Framework and Pyton code. Python library debugging was tested with Visual Studio Code with Robot Framework Language Server by Robocorp, by using `debug test.py` file. Also Robot Framework test code from within test directory was tested with debugging, with the same tool, by creating a specific configuration within launch.json file (VSCode specific file). 89 | 90 | - Tested on Windows 10 with 1920x1080, Windows Dark mode, Light app mode. Also limited testing with 4k. 91 | - Tested on MacOS Big Sur with 1440x900, Dark Appearance. Also limited testing with 4k. Text Edit should be put in Plain Text mode. 92 | - Tested on Ubuntu with 1920x1080 and with Leafpad application. 93 | 94 | # Supported Operating Systems 95 | 96 | 1. Windows 10 97 | - supported, tested with both JPype and Py4J 98 | 99 | 2. OSX 100 | - SikuliX works under OSX, however currently there are issues with JPype generally working under OSX: https://github.com/jpype-project/jpype/issues/911 101 | - Py4J tested and working under MacOS and always enabled without definding SIKULI_PY4J environment variable. However, forcing JPype for experimental purpose is 102 | possible with environment variable SIKULI_PY4J=0. 103 | 104 | 3. Linux 105 | - supported, tested with Ubuntu 20.04 and Leafpad application. Tested with both JPype and Py4J. 106 | - due to https://github.com/RaiMan/SikuliX1/issues/438, openApp is not currently working with SikuliX 2.0.5, thus it is disabled in the test .py and .robot code for Ubuntu. 107 | This means you have to start the test and open manually Leafpad app in order for tests to succeed. 108 | - tested with: python3.8, default-jre (openjdk-11-jre), libopencv4.2-java as explained on SikuliX support page, gnome-panel, `pip install robotframework-sikulixlibrary` 109 | - start the tests with e.g. `python -m robot --outputdir results/ubuntu test_defaultlibrary_ubuntu.robot` or `python testlibrary_ubuntu.py` 110 | 111 | # Known Issues 112 | 113 | - When using Py4J, libdoc will not generate library documentation within IDE, e.g. Visual Studio Code with Robocorp extension (Robot Framework Language Server). The workaround is to start manually SikuliX Py4J server (see above or run `java -jar sikulix.jar -p`), then reload the library to generate the keyword documentation. If not using pip install for this library, then the extension settings.json need to have pythonpath set to local robotframework-sikulixlibrary folder. 114 | 115 | - When generating library documentation within IDE, the library is instantiated with no arguments and it will look for sikulix.jar file within SIKULI_HOME environment variable defined directory, so make sure this file is there even if it is a duplicate of e.g. sikulixide-2.0.5.jar. Otherwise, it is also possible to manually generate the libspec with the following command, and copy it under project folder or PYTHONPATH: 116 | 117 | `python -m robot.libdoc SikuliXLibrary::sikuli_path=path\to\sikulixide-2.0.5.jar SikuliXLibrary.libspec` 118 | 119 | See this documentation for reference: 120 | https://github.com/robocorp/robotframework-lsp/blob/master/robotframework-ls/docs/faq.md 121 | -------------------------------------------------------------------------------- /SikuliXLibrary/__init__.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixlibrary import SikuliXLibrary 4 | 5 | from .sikulixapp import SikuliXApp 6 | from .sikuliximagepath import SikuliXImagePath 7 | from .sikulixjclass import SikuliXJClass 8 | from .sikulixlogger import SikuliXLogger 9 | from .sikulixregion import SikuliXRegion 10 | from .sikulixdebug import SikuliXDebug 11 | from .sikulixsettings import SikuliXSettings 12 | 13 | 14 | from .version import __version__ as VERSION 15 | 16 | __version__ = VERSION 17 | -------------------------------------------------------------------------------- /SikuliXLibrary/requirements.txt: -------------------------------------------------------------------------------- 1 | robotframework >=3.2.2 2 | jpype1 >=1.2.0 3 | py4j >=0.10.9.2 -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixapp.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixjclass import * 4 | 5 | 6 | class SikuliXApp(SikuliXJClass): 7 | ''' 8 | SikuliX Application class (App) methods 9 | ''' 10 | @keyword 11 | def app_open(self, application): 12 | ''' 13 | Open the specified application (e.g. notepad.exe). Check https://sikulix-2014.readthedocs.io/en/latest/appclass.html for more 14 | 15 | The string application must allow the system to locate the application in the system specific manner. 16 | If this is not possible you might try the full path to an application executable. 17 | 18 | Optionally you might add parameters, that will be given to the application at time of open. 19 | 20 | There are 2 options: 21 | - put the application string in apostrophes and the rest following the second apostrophes will be taken as parameter string 22 | - put `` -- `` (space 2 hyphens! space) between the applications name or path (no apostrophes!) and the parameter string. 23 | 24 | | App Open | C:/Windows/System32/notepad.exe | 25 | | App Open | "C:/Windows/System32/notepad.exe"path_to_my_txt_file | 26 | | App Open | C:/Windows/System32/notepad.exe -- path_to_my_txt_file | 27 | ''' 28 | #SikuliXJClass.App.open(application) 29 | SikuliXJClass.App(application).open() 30 | 31 | @keyword 32 | def app_focus(self, title): 33 | ''' 34 | Switch the input focus to a running application, having a front-most window with a matching title. 35 | 36 | | App Focus | Notepad | 37 | ''' 38 | #SikuliXJClass.App.focus(title) 39 | SikuliXJClass.App(title).focus() 40 | 41 | @keyword 42 | def app_close(self, app): 43 | ''' 44 | It closes the running application matching the given string. 45 | 46 | | App Close | Notepad | 47 | ''' 48 | #SikuliXJClass.App.close(app) 49 | SikuliXJClass.App(app).close() 50 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixdebug.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixjclass import * 4 | 5 | 6 | class SikuliXDebug(SikuliXJClass): 7 | ''' 8 | SikuliX Debug class 9 | ''' 10 | @keyword 11 | def set_debug(self, value): 12 | ''' 13 | Sets the debug level of the SikuliX core engine. This data is logged to the console (stdout). 14 | Default is 0, more output is generated using level 3. Higher values may give more output. 15 | 16 | Example 17 | | Set Debug | 3 | 18 | ''' 19 | SikuliXJClass.Debug.setGlobalDebug(int(value)) 20 | 21 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikuliximagepath.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixjclass import * 4 | 5 | 6 | class SikuliXImagePath(SikuliXJClass): 7 | ''' 8 | SikuliX ImagePath class, handling the locations (paths) from where to load the reference images to search for 9 | ''' 10 | @not_keyword 11 | def __init__(self, image_path=''): 12 | if image_path != '': 13 | SikuliXJClass.ImagePath.add(image_path) 14 | 15 | libLogger.debug('SikuliXImagePath init') 16 | 17 | @keyword 18 | def imagePath_add(self, path): 19 | ''' 20 | Used usually in any suite setup. Will add to SikuliX ImagePath a new directory where to find reference images 21 | Note: paths must be specified using the correct path separators (slash on Mac and Unix and double blackslashes 22 | on Windows). In Robot Framework you can use the `${/}` construct as universal separator. 23 | 24 | | ImagePath Add | path | 25 | ''' 26 | SikuliXJClass.ImagePath.add(path) 27 | 28 | @keyword 29 | def imagePath_remove(self, path): 30 | ''' 31 | Will remove from SikuliX ImagePath the given path 32 | 33 | | ImagePath Remove | path | 34 | ''' 35 | SikuliXJClass.ImagePath.remove(path) 36 | 37 | @keyword 38 | def imagePath_reset(self): 39 | ''' 40 | Will reset the SikuliX ImagePath and thereby remove all previous entries 41 | 42 | | ImagePath Reset | 43 | ''' 44 | SikuliXJClass.ImagePath.reset() 45 | 46 | @keyword 47 | def imagePath_dump(self): 48 | ''' 49 | Retrieves the full list of image paths and logs these as trace messagesin the log file. 50 | ''' 51 | imgPath = list(SikuliXJClass.ImagePath.getPaths()) 52 | for p in imgPath: 53 | logger.trace("Image PATH: " + str(p)) 54 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixjclass.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | import os, time, subprocess, logging, sys 4 | from py4j.java_gateway import GatewayParameters 5 | 6 | # Check which Python Java bridge to use between JPype and Py4J. When SIKULI_PY4J environment variable is defined with value 1 7 | # use Py4J, otherwise if not defined or has value 0, use JPype 8 | useJpype = True 9 | if os.getenv('SIKULI_PY4J') == '1': 10 | useJpype = False 11 | 12 | # On MacOs always use Py4J, unless SIKULI_PY4J is set to 0 (e.g. for experiments) 13 | if sys.platform.startswith('darwin'): 14 | useJpype = False 15 | 16 | # Override MacOS Py4J forcing, e.g. for experiments 17 | if os.getenv('SIKULI_PY4J') == '0': 18 | useJpype = True 19 | 20 | 21 | if useJpype: 22 | import jpype 23 | import jpype.imports 24 | from jpype.types import * 25 | else: 26 | from py4j.java_gateway import ( 27 | JavaGateway, Py4JNetworkError, get_field, get_method, get_java_class) 28 | 29 | from robot.api.deco import * 30 | from robot.api import logger 31 | 32 | libLogger = logging.getLogger(__name__) 33 | libLogger.setLevel(level=logging.INFO) 34 | logging.getLogger("py4j").setLevel(logging.ERROR) 35 | 36 | 37 | class SikuliXJClass(): 38 | ''' 39 | Main class holding JPype JClasses or Py4J gateway classes used by SikuliX library 40 | ''' 41 | Initialized = False 42 | 43 | Screen = None 44 | Region = None 45 | Pattern = None 46 | Match = None 47 | Key = None 48 | KeyModifier = None 49 | App = None 50 | FindFailed = None 51 | FindFailedResponse = None 52 | ImagePath = None 53 | Settings = None 54 | Debug = None 55 | JavaGW = None 56 | Py4JProcess = None 57 | 58 | 59 | @not_keyword 60 | def __init__(self, sikuli_path=''): 61 | self._init_python_console_logger() 62 | libLogger.debug('PY4J env variable: %s' % os.getenv('SIKULI_PY4J')) 63 | if not SikuliXJClass.Initialized: 64 | if useJpype: 65 | self._jvm_sikuli_init(sikuli_path) 66 | else: 67 | self._py4j_sikuli_init(sikuli_path) 68 | SikuliXJClass.Initialized = True 69 | 70 | libLogger.debug('SikuliXJClass init') 71 | 72 | @not_keyword 73 | def _init_python_console_logger(self): 74 | logFormatter = logging.Formatter("%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s") 75 | consoleHandler = logging.StreamHandler() 76 | consoleHandler.setFormatter(logFormatter) 77 | libLogger.addHandler(consoleHandler) 78 | 79 | @not_keyword 80 | def _handle_sikuli_path(self, sikuli_path): 81 | # Check type of sikuli_path: empty for path from SIKULI_HOME + sikulix.jar, not empty might be either jar name or full path 82 | if (sikuli_path == '') or not os.path.isabs(sikuli_path): 83 | sikuli_home = os.getenv('SIKULI_HOME') 84 | if not sikuli_home: 85 | raise Exception("SIKULI_HOME environment variable not defined or full SikuliX path is missing.") 86 | if (sikuli_path == ''): 87 | sikuli_path = os.path.join(sikuli_home, 'sikulix.jar') 88 | else: 89 | sikuli_path = os.path.join(sikuli_home, sikuli_path) 90 | 91 | if not os.path.isfile(sikuli_path): 92 | raise FileNotFoundError(sikuli_path) 93 | libLogger.debug('Use SikuliX file: %s' % sikuli_path) 94 | 95 | return sikuli_path 96 | 97 | @not_keyword 98 | def _jvm_sikuli_init(self, sikuli_path): 99 | libLogger.info('JPype init') 100 | sikuli_path = self._handle_sikuli_path(sikuli_path) 101 | # Launch the JVM 102 | try: 103 | #java_path = jpype.getDefaultJVMPath() 104 | jpype.addClassPath(sikuli_path) 105 | jpype.startJVM() 106 | #jpype.startJVM(java_path, "-ea", "-Djava.class.path=%s" % sikuli_path) 107 | except: 108 | raise Exception("Fail to start JVM. Check Java and SikuliX paths.") 109 | 110 | if not jpype.isJVMStarted(): 111 | raise Exception("Fail to start JVM. Check Java and SikuliX paths.") 112 | 113 | SikuliXJClass.ImagePath = JClass("org.sikuli.script.ImagePath") 114 | SikuliXJClass.Screen = JClass("org.sikuli.script.Screen") 115 | SikuliXJClass.Region = JClass("org.sikuli.script.Region") 116 | SikuliXJClass.Pattern = JClass('org.sikuli.script.Pattern') 117 | SikuliXJClass.Match = JClass('org.sikuli.script.Match') 118 | 119 | SikuliXJClass.Key = JClass('org.sikuli.script.Key') 120 | SikuliXJClass.KeyModifier = JClass('org.sikuli.script.KeyModifier') 121 | SikuliXJClass.App = JClass('org.sikuli.script.App') 122 | SikuliXJClass.FindFailed = JClass('org.sikuli.script.FindFailed') 123 | SikuliXJClass.FindFailedResponse = JClass('org.sikuli.script.FindFailedResponse') 124 | SikuliXJClass.Settings = JClass('org.sikuli.basics.Settings') 125 | SikuliXJClass.Debug = JClass('org.sikuli.basics.Debug') 126 | 127 | @not_keyword 128 | def _py4j_sikuli_init(self, sikuli_path): 129 | libLogger.info('Py4J init') 130 | sikuli_path = self._handle_sikuli_path(sikuli_path) 131 | 132 | # wait for gateway 133 | def wait_for_gateway(func, max_tries, sleep_time): 134 | for _ in range(0, max_tries): 135 | try: 136 | f = func() 137 | print(f) 138 | return f 139 | except: 140 | libLogger.info('Gateway not ready. Waiting.') 141 | time.sleep(sleep_time) 142 | raise Exception("Fail to start Py4J. SikuliX not running") 143 | 144 | # Check if already running 145 | manuallyStarted = False 146 | try: 147 | JavaGW = JavaGateway(gateway_parameters=GatewayParameters(eager_load=True, auto_field=True)) 148 | libLogger.info("JVM accepting connection") 149 | manuallyStarted = True 150 | except Py4JNetworkError: 151 | libLogger.debug("No JVM listening") 152 | except Exception: 153 | libLogger.error("Other JVM exception") 154 | 155 | # Launch the JVM 156 | if not manuallyStarted: 157 | SikuliXJClass.Py4JProcess = subprocess.Popen(['java', '-jar', sikuli_path, '-p'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) 158 | libLogger.info('JVM started: %s' % SikuliXJClass.Py4JProcess) 159 | JavaGW = JavaGateway() 160 | wait_for_gateway(lambda : JavaGW.jvm.System.getProperty("java.runtime.name"), 20, 0.5) 161 | 162 | SikuliXJClass.JavaGW = JavaGW 163 | 164 | SikuliXJClass.ImagePath = JavaGW.jvm.org.sikuli.script.ImagePath 165 | SikuliXJClass.Screen = JavaGW.jvm.org.sikuli.script.Screen 166 | SikuliXJClass.Region = JavaGW.jvm.org.sikuli.script.Region 167 | SikuliXJClass.Pattern = JavaGW.jvm.org.sikuli.script.Pattern 168 | SikuliXJClass.Match = JavaGW.jvm.org.sikuli.script.Match 169 | 170 | SikuliXJClass.Key = JavaGW.jvm.org.sikuli.script.Key 171 | SikuliXJClass.KeyModifier = JavaGW.jvm.org.sikuli.script.KeyModifier 172 | SikuliXJClass.App = JavaGW.jvm.org.sikuli.script.App 173 | SikuliXJClass.FindFailed = JavaGW.jvm.org.sikuli.script.FindFailed 174 | SikuliXJClass.FindFailedResponse = JavaGW.jvm.org.sikuli.script.FindFailedResponse 175 | SikuliXJClass.Settings = JavaGW.jvm.org.sikuli.basics.Settings 176 | SikuliXJClass.Debug = JavaGW.jvm.org.sikuli.basics.Debug 177 | 178 | @keyword 179 | def log_java_bridge(self): 180 | ''' 181 | Log within Robot Framework which java bridge was used 182 | ''' 183 | if not SikuliXJClass.JavaGW == None: 184 | if SikuliXJClass.Py4JProcess: 185 | logger.info('Using Py4J, started automatically') 186 | else: 187 | logger.info('Using Py4J, started manually') 188 | else: 189 | logger.info('Using JPype') 190 | 191 | @keyword 192 | def destroy_vm(self): 193 | ''' 194 | Shutdown the Java Virtual Machine used by JPype or JavaGateway from Py4J 195 | ''' 196 | SikuliXJClass.Initialized = False 197 | if useJpype: 198 | jpype.shutdownJVM() 199 | elif SikuliXJClass.Py4JProcess: 200 | SikuliXJClass.JavaGW.shutdown() 201 | SikuliXJClass.Py4JProcess.kill() 202 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixlibrary.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .version import __version__ as VERSION 4 | 5 | from .sikulixregion import * 6 | from .sikulixapp import * 7 | from .sikuliximagepath import * 8 | from .sikulixsettings import * 9 | from .sikulixdebug import * 10 | 11 | 12 | @library(scope='GLOBAL', version=VERSION) 13 | class SikuliXLibrary(SikuliXRegion, 14 | SikuliXApp, 15 | SikuliXImagePath, 16 | SikuliXSettings, 17 | SikuliXDebug): 18 | 19 | ''' The all new, modern, SikuliX Robot Framework library for Python 3.x, based on JPype or Py4J Python modules. 20 | 21 | It can be enabled to use by choice any of the JPype or Py4J modules. This is done by creating SIKULI_PY4J environment variable 22 | and setting to 1 for using Py4J. When not defined or set to 0, JPype is used instead. 23 | Please note that on MacOS, only Py4J can be used, while on Windows or Ubuntu, any of them is working. 24 | 25 | So far, the only approach to use SikuliX Java library within Robot Framework was through Remote library and Jython 2.7. 26 | The existing ``robotframework-SikuliLibrary`` and other known custom implementations (e.g. mostly based on old 27 | http://blog.mykhailo.com/2011/02/how-to-sikuli-and-robot-framework.html) are using Remote library approach only, which is now obsolete. 28 | 29 | In addition, also other popular libraries like ``ImageHorizonLibrary`` (built on top of pyautoguy), that is used currently due easier 30 | usage in comparison with previous SikuliX remote server implementations, can now be easily switched to this new library. 31 | 32 | With the help of this new library, SikuliX implementation can be used now natively with Robot Framework and Python 3.x: 33 | - robotremoteserver and Remote library are not needed anymore 34 | - debugging with some RF supporting tools 35 | - very easy to extend the library with new keywords, or overwrite existing keywords and methods by extending the main class, e.g. 36 | | class ImageHorizonLibraryMigration(SikuliXLibrary): 37 | | def click_image(self, reference_image): 38 | | self.region_click(target, 0, 0, False) 39 | | 40 | | class SikuliLibraryMigration(SikuliXLibrary): 41 | | def click(self, image, xOffset, yOffset): 42 | | self.region_click(image, xOffset, yOffset, False) 43 | | 44 | | class SikuliXCustomLibrary(SikuliXLibrary): 45 | | def _passed(self, msg): 46 | | logger.info('MY PASS MESSAGE: ' + msg) 47 | 48 | This library is using: 49 | | [https://github.com/RaiMan/SikuliX1] 50 | | [https://github.com/jpype-project/jpype] 51 | | [https://github.com/bartdag/py4j] 52 | 53 | The keywords are matching as much as possible the original SikuliX functions so that it is easier to understand them from 54 | the official documentation: https://sikulix-2014.readthedocs.io/en/latest/index.html 55 | E.g. ``SikuliX class Region.find(PS)`` function is translated into Python and Robot keyword as 56 | ``region_find(target, onScreen)`` 57 | 58 | ``region_find = Region.find(PS)``, where PS is a Pattern or String that define the path to an image file 59 | 60 | Pattern will need the following parameters, provided as arguments to this keyword 61 | - target - a string naming an image file from known image paths (with or without .png extension) 62 | - similar - minimum similarity. If not given, the default is used. Can be set as ``img=similarity`` 63 | - mask - an image with transparent or black parts or 0 for default masked black parts. Should be set as img:mask, img:0, img:mask=similarity or img:0=similarity 64 | - onScreen - reset the region to the whole screen, otherwise it will search on a region defined previously with set parameters keywords 65 | e.g. `Region SetRect` where the parameters can be from a previous match or known dimension, etc. 66 | 67 | Compared with other libraries, the import parameter ``centerMode`` will allow using click coordinates relative to center of the image, 68 | otherwise the click coordinates are relative to upper left corner (default). 69 | With this approach, it is very easy to capture a screenshot, open it e.g. in Paint in Windows and the coordinates shown in the lower left 70 | corner are the click coordinates that should be given to the click keyword: 71 | 72 | ``region_click = Region.click(PSMRL[, modifiers])``, where PSMRL is a pattern, a string, a match, a region or a location that evaluates to a click point. 73 | 74 | Currently only String, together with parameters that define a pattern will be accepted. 75 | Pattern will need the following parameters, provided as arguments to this keyword 76 | - target - a string naming an image file from known image paths (with or without .png extension) 77 | - similar - minimum similarity. If not given, the default is used. Can be set as img=similarity 78 | - mask - an image with transparent or black parts or 0 for default masked black parts. Should be set as img:mask, img:0, img:mask=similarity or img:0=similarity 79 | - dx, dy - define click point, either relative to center or relative to upper left corner (default with set_offsetCenterMode) 80 | Note: within RF, coordinates can be given both as string or numbers, for any keyword that needs coordinates, e.g.: 81 | ``Region Click 10 10`` or ``Region Click ${10} ${10}`` 82 | - useLastMatch - if True, will assume the LastMatch can be used otherwise SikuliX will do a find on the target image and click in the center of it. 83 | 84 | If implicit find operation is needed, assume the region is the whole screen. 85 | 86 | Region Click with no arguments will either click the center of the last used Region or the lastMatch, if any is available. 87 | = Debugging = 88 | When writing test cases and keywords it is important to understand the precise effect of the code written. 89 | The following tools can help to understand what's going on, in order of detail level: 90 | - Robot Framework's own 91 | [https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#log-levels|`Set Log Level`] 92 | - Vizualisation tools offered by SikuliXLibrary as `Settings Set Show Actions` and `Region Highlight` 93 | - Additional logging of the SikuliX core engine, enabled by the keyword `Set Debug`. 94 | - Once logging of the SikuliX core engine is enabled, more logging sections can be enabled using the 95 | `DebugLogs`, `ProfileLogs` and `TraceLogs` switches, see `Settings Set`. 96 | ''' 97 | @not_keyword 98 | def __init__(self, sikuli_path='', image_path='', logImages=True, centerMode=False): 99 | ''' 100 | | sikuli_path | Path to sikulix.jar file. If empty, it will try to use SIKULI_HOME environment variable. | 101 | | image_path | Initial path to image library. More paths can be added later with the keyword `ImagePath Add` | 102 | | logImages | Default True, if screen captures of found images and whole screen if not found, are logged in the final result log.html file | 103 | | centerMode | Default False, if should calculate the click offset relative to center of the image or relative to upper left corner. | 104 | ''' 105 | SikuliXJClass.__init__(self, sikuli_path) 106 | SikuliXImagePath.__init__(self, image_path) 107 | SikuliXRegion.__init__(self, logImages, centerMode) 108 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixlogger.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixjclass import * 4 | from os.path import relpath 5 | import datetime, shutil 6 | 7 | 8 | class SikuliXLogger(): 9 | ''' 10 | Class handling the logging of source images, matches and screenshots within robot log.html file 11 | ''' 12 | resultDir: str = '.' 13 | 14 | @not_keyword 15 | def __init__(self, logImages=True): 16 | self.passedLogImages = False 17 | self.failedLogImages = False 18 | self.notFoundLogImages = False 19 | 20 | if logImages: 21 | self.passedLogImages = True 22 | self.failedLogImages = True 23 | 24 | #libLogger.debug('SikuliXLogger init') 25 | 26 | @keyword 27 | def set_sikuli_resultDir(self, path): 28 | ''' 29 | Used to set the directory where to save the screenshots for the log file 30 | 31 | | Set Sikuli ResultDir | path | 32 | ''' 33 | SikuliXLogger.resultDir = path 34 | 35 | @keyword 36 | def set_passedLogImages(self, mode): 37 | ''' 38 | Enable or disable logging of the images when keyword passes 39 | 40 | | Set PassedLogImages | ${True} | 41 | ''' 42 | scr = self.passedLogImages 43 | self.passedLogImages = mode 44 | return scr 45 | 46 | @keyword 47 | def set_failedLogImages(self, mode): 48 | ''' 49 | Enable or disable logging of the images when keyword fails 50 | 51 | | Set FailedLogImages | ${True} | 52 | ''' 53 | scr = self.failedLogImages 54 | self.failedLogImages = mode 55 | return scr 56 | 57 | @keyword 58 | def set_notFoundLogImages(self, mode): 59 | ''' 60 | Enable or disable logging of the images when the image is not found (for keywords that does not throw exception) 61 | 62 | | Set NotFoundLogImages | ${True} | 63 | ''' 64 | scr = self.notFoundLogImages 65 | self.notFoundLogImages = mode 66 | return scr 67 | 68 | @keyword 69 | def log_warning(self, msg): 70 | ''' 71 | Print text in the log with the label WARNING: 72 | 73 | | Log Warning | msg | 74 | ''' 75 | logger.warn("WARNING: %s" % msg) 76 | 77 | @not_keyword 78 | def _screenshot(self, folder="/screenshots/", region=None): 79 | # generate unique name for screenshot filename 80 | if region == None: 81 | #br = SikuliXJClass.Screen().getBottomRight() 82 | #region = (0, 0, br.x, br.y) 83 | x = SikuliXJClass.Screen().getW() 84 | y = SikuliXJClass.Screen().getH() 85 | region = (0, 0, x, y) 86 | logger.trace('Screenshot Size {} {}'.format(x, y)) 87 | 88 | name = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S-%f') + ".png" 89 | img_src = str(self.appScreen.capture(*region).getFile()) 90 | full_folder = SikuliXLogger.resultDir + folder 91 | 92 | if img_src == None: 93 | return 'Screen capture failed (check resolution)' 94 | 95 | try: 96 | logger.trace("Screenshot: " + img_src) 97 | logger.trace("Matches: " + full_folder + name) 98 | shutil.copy(img_src, full_folder + name) 99 | except IOError: 100 | logger.error('FAIL: Capture screenshot path: ' + full_folder + name) 101 | 102 | return full_folder + name 103 | 104 | @not_keyword 105 | def _passed(self, msg, mode=None): 106 | libLogger.debug('PASS %s' % msg) 107 | logger.info('PASS: ' + msg) 108 | 109 | # matched image 110 | last_match: SikuliXJClass.Match = self.appRegion.getLastMatch() 111 | # score of match 112 | score: float = float(last_match.getScore()) 113 | 114 | if self.passedLogImages: 115 | if mode == None: 116 | # source image 117 | src_img: str = str(self.appPattern.getFilename()) 118 | 119 | # get relative path from result directory (log.html) 120 | rel_path = relpath(src_img, SikuliXLogger.resultDir) 121 | logger.debug('Source Image: ' % rel_path, True) 122 | 123 | # screenshot of matched image 124 | region = (last_match.getX(), last_match.getY(), last_match.getW(), last_match.getH()) 125 | name = self._screenshot("/matches/", region) 126 | rel_path = relpath(name, SikuliXLogger.resultDir) 127 | logger.debug('Best Match: ' % rel_path, True) 128 | 129 | logger.info("Matched with score: %s" % score) 130 | 131 | @not_keyword 132 | def _failed(self, msg, seconds, mode=None): 133 | libLogger.debug('FAIL %s' % msg) 134 | logger.error('FAIL: ' + msg) 135 | 136 | if self.failedLogImages: 137 | if mode == None: 138 | # source image 139 | src_img: str = str(self.appPattern.getFilename()) 140 | rel_path = relpath(src_img, SikuliXLogger.resultDir) 141 | logger.info('Source Image: ' % rel_path, True, True) 142 | 143 | # screenshot 144 | name = self._screenshot("/screenshots/") 145 | rel_path = relpath(name, SikuliXLogger.resultDir) 146 | logger.info('No Match: ' % rel_path, True, True) 147 | 148 | if mode == None: 149 | wait: float = float(self.appRegion.getAutoWaitTimeout()) 150 | if seconds > 0: 151 | wait: float = seconds 152 | logger.debug('Image not visible after ' + str(wait) + ' seconds') 153 | raise Exception(msg) 154 | 155 | @not_keyword 156 | def _notfound(self, msg, seconds, mode=None): 157 | libLogger.debug('NOT FOUND %s' % msg) 158 | if self.notFoundLogImages: 159 | if mode == None: 160 | # source image 161 | src_img: str = str(self.appPattern.getFilename()) 162 | rel_path = relpath(src_img, SikuliXLogger.resultDir) 163 | logger.info('Source Image: ' % rel_path, True, True) 164 | 165 | # screenshot 166 | name = self._screenshot("/screenshots/") 167 | rel_path = relpath(name, SikuliXLogger.resultDir) 168 | logger.info('Not Found: ' % rel_path, True, True) 169 | 170 | if mode == None: 171 | wait: float = float(self.appRegion.getAutoWaitTimeout()) 172 | if seconds > 0: 173 | wait: float = seconds 174 | logger.debug('Image not visible after ' + str(wait) + ' seconds') 175 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixpy4j.py: -------------------------------------------------------------------------------- 1 | from .sikulixjclass import useJpype, SikuliXJClass 2 | 3 | if not useJpype: 4 | def JBoolean(x): 5 | return bool(x) 6 | def JInt(x): 7 | return int(x) 8 | def JFloat(x): 9 | return SikuliXJClass.JavaGW.jvm.java.lang.Double(float(x)).floatValue() 10 | def JDouble(x): 11 | return float(x) -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixregion.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixjclass import * 4 | from .sikulixlogger import * 5 | 6 | if not useJpype: 7 | from .sikulixpy4j import * 8 | 9 | class SikuliXRegion(SikuliXJClass, SikuliXLogger): 10 | ''' 11 | SikuliX Region class and all interactions with the region 12 | ''' 13 | @not_keyword 14 | def __init__(self, logImages=True, centerMode=False): 15 | SikuliXLogger.__init__(self, logImages) 16 | 17 | self.appScreen = SikuliXJClass.Screen() 18 | #br = self.appScreen.getBottomRight() 19 | #appCoordinates = (0, 0, br.x, br.y) 20 | x = self.appScreen.getW() 21 | y = self.appScreen.getH() 22 | appCoordinates = (0, 0, x, y) 23 | self.appScreen = SikuliXJClass.Screen() 24 | self.appRegion = SikuliXJClass.Region(*appCoordinates) 25 | self.userDefined = appCoordinates 26 | self.appPattern = SikuliXJClass.Pattern() 27 | self.appMatch = SikuliXJClass.Match() 28 | 29 | self.offsetCenterMode = centerMode 30 | self.defaultRegionSelectMode = None 31 | 32 | libLogger.debug('SikuliXRegion init') 33 | 34 | # Region - Set operations 35 | @keyword 36 | def set_offsetCenterMode(self, mode): 37 | ''' 38 | Set to use click coordinates relative to center of the image (True) or relative to upper left corner (default False). 39 | 40 | With this approach, it is very easy to capture a screenshot, open it e.g. in Paint in Windows and the 41 | coordinates shown in lower left corner are the click coordinates that should be given to the mouse action keywords. 42 | 43 | | Set OffsetCenterMode | ${True} | 44 | ''' 45 | self.offsetCenterMode = mode 46 | 47 | @keyword 48 | def region_setDefaultSelectMode(self, mode=None): 49 | ''' 50 | Set the default mode to select the active region for image and text searches. 51 | - `UserDefined` will use the values of the last `Region Set Rect` call 52 | - `FullScreen` selects the entire screen 53 | - `LastMatch` uses the result of the previous search action 54 | - `None` uses the legacy way of selecting the active region 55 | 56 | The default selection can be overriden at every keyword call, by specifying the value for `regionSelect` 57 | 58 | | Region SetDefaultSelectMode | FullScreen | 59 | ''' 60 | self.defaultRegionSelectMode = mode 61 | 62 | @keyword 63 | def region_setAutoWait(self, seconds): 64 | ''' 65 | Set the maximum waiting time for all subsequent find operations in that Region. 66 | 67 | | Region SetAutoWait | ${5} | 68 | ''' 69 | self.appRegion.setAutoWaitTimeout(float(seconds)) 70 | 71 | @keyword 72 | def region_getAutoWait(self): 73 | ''' 74 | Get the current value of the maximum waiting time for find operation in this region. 75 | 76 | | ${wait} | Region GetAutoWait | 77 | ''' 78 | return self.appRegion.getAutoWaitTimeout() 79 | 80 | @keyword 81 | def region_setFindFailedResponse(self, val): 82 | ''' 83 | Define the response if SikuliX cannot find the image. 84 | 85 | Check https://sikulix-2014.readthedocs.io/en/latest/region.html for response options. 86 | - PROMPT will ask user for the next action 87 | - ABORT the execution of test 88 | - SKIP the step 89 | - RETRY to search again for image 90 | 91 | | Region SetFindFailedResponse | SKIP | 92 | ''' 93 | if useJpype: 94 | jVal = SikuliXJClass.FindFailedResponse.class_.getDeclaredField(val).get(None) 95 | else: 96 | jVal = get_java_class(SikuliXJClass.FindFailedResponse).getDeclaredField(val).get(None) 97 | self.appRegion.setFindFailedResponse(jVal) 98 | 99 | @keyword 100 | def region_getFindFailedResponse(self): 101 | ''' 102 | Return the response set if SikuliX cannot find the image, see `Region SetFindFailedResponse`. 103 | 104 | | ${val} | Region GetFindFailedResponse | 105 | ''' 106 | return self.appRegion.getFindFailedResponse() 107 | 108 | @keyword 109 | def region_setRect(self, x=0, y=0, w=0, h=0, dx=0, dy=0, mode=None): 110 | ''' 111 | Set position and dimension of the current region to new values, in one of the following manners. 112 | | =Mode= | =Example= | =Effect= | 113 | | mode parameter is not specified | ``Region SetRect 0 0 1920 1080`` | the first 2 parameters \ 114 | specify the upper left corner of the region, the sencond 2 parameters specify the size of the \ 115 | region (default behavior) | 116 | | mode=left-upper | ``Region SetRect 0 0 1920 1080 left-upper`` | the first 2 parameters are \ 117 | redundant and ignored, the region has the left upper corner at 0,0, the size of the region \ 118 | is 1920x1080 | 119 | | mode=left-upper | ``Region SetRect w=1920 h=1080 mode=left-upper`` | the region has the left \ 120 | upper corner at 0,0, the size of the region is 1920x1080 | 121 | | mode=right-upper | ``Region SetRect w=800 h=600 mode=right-upper`` | the region has the right \ 122 | upper corner at the right upper corner of the screen, the size of the region is 800x600 | 123 | | mode=center | ``Region SetRect w=800 h=600 mode=center`` | the region is centered to the \ 124 | center of the screen, the size of the region is 800x600 | 125 | 126 | Next to `left-uppper` and `right-upper`, also `left-lower` and `right-lower` modes are supported, which will 127 | align the region to the corresponding corner of the screen. With the `dx` and `dy` parameters, a shift 128 | for the region is specified. 129 | 130 | ``Region SetRect w=800 h=600 dx=100 dy=100 mode=left-upper`` is equivalent of 131 | ``Region SetRect 100 100 800 600`` 132 | ''' 133 | if mode == 'left-upper': 134 | x = dx 135 | y = dy 136 | elif mode == 'right-upper': 137 | x = self.appScreen.w - w + dx 138 | y = dy 139 | elif mode == 'left-lower': 140 | x = dx 141 | y = self.appScreen.h - h + dy 142 | elif mode == 'right-lower': 143 | x = self.appScreen.w - w + dx 144 | y = self.appScreen.h - h + dy 145 | elif mode == 'center': 146 | x = (self.appScreen.w - w) // 2 + dx 147 | y = (self.appScreen.h - h) // 2 + dy 148 | elif mode == None: 149 | pass 150 | else: 151 | logger.error('Unsupported mode: {}'.format(mode)) 152 | 153 | self.appRegion.setRect(JInt(x), JInt(y), JInt(w), JInt(h)) 154 | self.userDefined = (int(x), int(y), int(w), int(h)) 155 | logger.trace('Region Set Rect {} {} {} {}'.format(x, y, w, h)) 156 | 157 | # Region - find operations 158 | @not_keyword 159 | def _prepare_pattern(self, target, dx=0, dy=0): 160 | # target can be img, img=similar, img:mask, img:0, img:mask=similar or img:0=similar 161 | img = target 162 | mask = -1 163 | sim = 0 164 | if ":" in target: 165 | text = target.split(':') 166 | img = text[0] 167 | mask = text[1] 168 | if "=" in mask: 169 | text = mask.split('=') 170 | mask=text[0] 171 | sim=float(text[1]) 172 | elif "=" in target: 173 | text = target.split('=') 174 | img = text[0] 175 | sim=float(text[1]) 176 | else: 177 | img = target 178 | 179 | logger.trace("Prepare pattern with image: %s" % img) 180 | pattern = SikuliXJClass.Pattern(img) 181 | if mask == '0': 182 | logger.trace("Prepare pattern with mask: default black") 183 | pattern.mask() 184 | elif mask != -1: 185 | logger.trace("Prepare pattern with mask: %s" % mask) 186 | pattern.mask(mask) 187 | if sim != 0: 188 | logger.trace("Prepare pattern with similarity: %s" % sim) 189 | pattern.similar(sim) 190 | 191 | # if dx and dy are not given, no target offset is given and click is center of image 192 | if dx == 0 and dy == 0: 193 | return pattern 194 | 195 | # calculate offset relative to upper left corner. 196 | if not self.offsetCenterMode: 197 | dx -= pattern.getImage().getW() / 2 198 | dy -= pattern.getImage().getH() / 2 199 | 200 | return pattern.targetOffset(JInt(dx), JInt(dy)) 201 | 202 | @not_keyword 203 | def _set_active_region(self, onScreen, regionSelect): 204 | # selects the proper region for the next operation, using one of many modes 205 | if regionSelect == None: 206 | regionSelect = self.defaultRegionSelectMode 207 | 208 | if regionSelect == 'UserDefined': 209 | self.appRegion.setRect(SikuliXJClass.Region(*self.userDefined)) 210 | elif regionSelect == 'LastMatch': 211 | self.appRegion.setRect(self.appRegion.getLastMatch()) 212 | elif regionSelect == 'FullScreen': 213 | self.appRegion.setRect(self.appScreen) 214 | else: 215 | # legacy modes 216 | if onScreen == True: 217 | self.appRegion.setRect(self.appScreen) 218 | 219 | logger.info('Active area {} {}, {}x{}'.format(self.appRegion.x, self.appRegion.y, self.appRegion.w, self.appRegion.h)) 220 | 221 | @not_keyword 222 | def _prepare_lastMatch(self, dx, dy): 223 | # calculate offset relative to upper left corner. 224 | self.appMatch = self.appRegion.getLastMatch() 225 | 226 | # if dx and dy are not given, no target offset is given and click is center of image 227 | if dx == 0 and dy == 0: 228 | return 229 | 230 | if not self.offsetCenterMode: 231 | dx -= self.appMatch.getW() / 2 232 | dy -= self.appMatch.getH() / 2 233 | 234 | self.appMatch.setTargetOffset(JInt(dx), JInt(dy)) 235 | 236 | @not_keyword 237 | def _region_findOperation(self, type, target, seconds, onScreen, regionSelect): 238 | logger.trace('{} on target ()'.format(type, target)) 239 | 240 | self._set_active_region(onScreen, regionSelect) 241 | 242 | self.appPattern = self._prepare_pattern(target) 243 | try: 244 | if seconds == 0: 245 | logger.trace("Call findOperation with arguments: %s" % type) 246 | logger.trace('Region: ' + str(self.appRegion) + '; Pattern: ' + str(self.appPattern)) 247 | if useJpype: 248 | res = SikuliXJClass.Region.class_.getDeclaredMethod(type, JObject).invoke(self.appRegion, self.appPattern) 249 | else: 250 | #print(self.appRegion) 251 | #print(get_java_class(SikuliXJClass.Region)) 252 | #print(get_method(self.appRegion, type)) 253 | res = get_method(self.appRegion, type)(self.appPattern) 254 | else: 255 | logger.trace("Call findOperation with arguments: %s, %s seconds" % (type, seconds)) 256 | logger.trace('Region: ' + str(self.appRegion) + '; Pattern: ' + str(self.appPattern)) 257 | if useJpype: 258 | res = SikuliXJClass.Region.class_.getDeclaredMethod(type, JObject, JDouble).invoke(self.appRegion, 259 | self.appPattern, JDouble(seconds)) 260 | else: 261 | res = get_method(self.appRegion, type)(self.appPattern, JDouble(seconds)) 262 | 263 | except: # except should happen only for find or wait 264 | self._failed("Image not visible on screen: " + target, seconds) 265 | raise Exception("_Find text method Failed") 266 | 267 | if res: 268 | if type == 'waitVanish': 269 | logger.info('PASS: ' + 'Image vanished from screen') 270 | else: 271 | self._passed("Image visible on screen") 272 | else: 273 | self._notfound("Image not visible on screen: " + target, seconds) 274 | 275 | return res 276 | 277 | @keyword 278 | def region_find(self, target, onScreen=True, regionSelect=None): 279 | ''' 280 | Find a particular pattern, which is the given image. It searches within the region and returns the best match, 281 | that shows a similarity greater than the minimum similarity given by the pattern. If no similarity was set for 282 | the pattern by e.g. `Settings Set` before, a default minimum similarity of 0.7 is set automatically. 283 | 284 | From SikuliX documentation: Region.find(PS), where PS is a Pattern or String that define the path to an image file 285 | Pattern will need the following parameters, provided as arguments on this keyword 286 | - target - a string naming an image file from known image paths (with or without .png extension) 287 | - similar - minimum similarity. If not given, the default is used. Can be set as img=similarity 288 | - mask - an image with transparent or black parts or 0 for default masked black parts. Should be set as \ 289 | img:mask, img:0, img:mask=similarity or img:0=similarity 290 | - onScreen - reset the region to the whole screen, otherwise will search on a region defined previously \ 291 | with set parameters keywords 292 | e.g. `Region SetRect` where the parameters can be from a previous match or known dimensions, etc. 293 | 294 | `Region Find` does not wait for the appearance until timeout expires and throws `FindFailed` if not found. 295 | 296 | `Region Find` returns a SikuliX match object containing the location parameters of the result. These parameters 297 | include the `x `and `y` coordinates of the top-left corner and the `w` and `h` dimensions of the 298 | found object. Example usage: 299 | | ${object} Region Find target-image.png 300 | | Log Image found at ${object.x}, ${object.y} 301 | | Log Image has width ${object.w} and height ${object.h} 302 | 303 | | Region Find | image.png=0.7 | 304 | | Region Find | image | onScreen=${False} | 305 | 306 | ''' 307 | return self._region_findOperation('find', target, 0, onScreen, regionSelect) 308 | 309 | @keyword 310 | def region_wait(self, target, seconds=0, onScreen=True, regionSelect=None): 311 | ''' 312 | Wait until the particular pattern, which is the given image appears in the current region. See `Region Find` 313 | for more details. 314 | 315 | Region Wait repeat search until timeout expires and throws FindFailed if not found. 316 | 317 | seconds: granularity is milliseconds. If not specified, the auto wait timeout value set by `Region SetAutoWait` 318 | is used 319 | 320 | | Region Wait | image.png=0.7 | 10s | 321 | | Region Wait | image | onScreen=${False} | 322 | 323 | ''' 324 | return self._region_findOperation('wait', target, seconds, onScreen, regionSelect) 325 | 326 | @keyword 327 | def region_waitVanish(self, target, seconds=0, onScreen=True, regionSelect=None): 328 | ''' 329 | Wait until the particular pattern, which is the given image vanishes the current screen. See `Region Find` 330 | for more details. 331 | 332 | Region WaitVanish repeat search until timeout expires and does not throw exception. 333 | 334 | | Region WaitVanish | image | 10s | 335 | ''' 336 | return self._region_findOperation('waitVanish', target, seconds, onScreen, regionSelect) 337 | 338 | @keyword 339 | def region_exists(self, target, seconds=0, onScreen=True, regionSelect=None): 340 | ''' 341 | Wait until the particular pattern, which is the given image appears in the current region. See `Region Find` 342 | for more details. 343 | 344 | Region Exists repeat search until timeout expires but does not throws FindFailed if not found. 345 | 346 | seconds: granularity is milliseconds. If not specified, the auto wait timeout value set by `Region SetAutoWait` is used 347 | 348 | | Region Exists | image.png=0.7 | 10s | 349 | | Region Exists | image | onScreen=${False} | 350 | 351 | ''' 352 | return self._region_findOperation('exists', target, seconds, onScreen, regionSelect) 353 | 354 | @keyword 355 | def region_has(self, target, seconds=0, onScreen=True, regionSelect=None): 356 | ''' 357 | Similar with `Region Exists` as convenience wrapper intended to be used in logical expressions. 358 | ''' 359 | return self._region_findOperation('has', target, seconds, onScreen, regionSelect) 360 | 361 | # Region - mouse actions 362 | @not_keyword 363 | def _region_mouseAction(self, action='click', target=None, dx=0, dy=0, useLastMatch=False): 364 | logger.trace('{} on target {} with offsets {},{}'.format(action, target, dx, dy)) 365 | 366 | # 1st case, target none - click on default 367 | if target == None: 368 | logger.trace('Region ' + str(self.appRegion)) 369 | if useJpype: 370 | return SikuliXJClass.Region.class_.getDeclaredMethod(action).invoke(self.appRegion) 371 | else: 372 | return get_method(self.appRegion, action)() 373 | #return self.appRegion.click() 374 | 375 | # 2nd case, define a Pattern from image name - implicit find operation is processed first. 376 | if not useLastMatch: 377 | self._set_active_region(None, None) 378 | pattern = self._prepare_pattern(target, JInt(dx), JInt(dy)) 379 | logger.trace('Region ' + str(self.appRegion) + '; Pattern ' + str(pattern)) 380 | if useJpype: 381 | return SikuliXJClass.Region.class_.getDeclaredMethod(action, JObject).invoke(self.appRegion, pattern) 382 | else: 383 | return get_method(self.appRegion, action)(pattern) 384 | 385 | # 3rd case, match can be given only as lastMatch. Target offset can be null or specified. 386 | if useLastMatch: 387 | self._prepare_lastMatch(JInt(dx), JInt(dy)) 388 | logger.trace('Region ' + str(self.appRegion) + '; Match ' + str(self.appMatch)) 389 | if useJpype: 390 | return SikuliXJClass.Region.class_.getDeclaredMethod(action, JObject).invoke(self.appRegion, self.appMatch) 391 | else: 392 | return get_method(self.appRegion, action)(self.appMatch) 393 | 394 | # 4th case, region - not implemented 395 | # 5th case, location - not implemented 396 | 397 | @keyword 398 | def region_click(self, target=None, dx=0, dy=0, useLastMatch=False): 399 | ''' 400 | Perform a mouse click on the click point using the left button. 401 | 402 | From SikuliX documentation: Region.click(PSMRL[, modifiers]), where PSMRL is a pattern, a string, a match, a region or a location that evaluates to a click point. 403 | 404 | Currently only String, together with parameters that define a pattern will be accepted. 405 | Pattern will need the following parameters, provided as arguments on this keyword 406 | - target - a string naming an image file from known image paths (with or without .png extension) 407 | - similar - minimum similarity. If not given, the default is used. Can be set as img=similarity 408 | - mask - an image with transparent or black parts or 0 for default masked black parts. Should be set as img:mask, img:0, img:mask=similarity or img:0=similarity 409 | - dx, dy - define click point, either relative to center or relative to upper left corner (default with `Set OffsetCenterMode`) 410 | - useLastMatch - if True, will assume the LastMatch can be used otherwise SikuliX will do a find on the target image and click in the center of it. 411 | If implicit find operation is needed, assume the region is the whole screen. 412 | 413 | Region Click with no arguments will either click the center of the last used Region or the lastMatch, if any is available. 414 | 415 | | Region Click | image.png=0.7 | dx | dy | 416 | | Region Click | image.png=0.7 | ${dx} | ${dy} | 417 | | Region Click | image | dx | dy | useLastMatch=${True} | 418 | ''' 419 | return self._region_mouseAction('click', target, dx, dy, useLastMatch) 420 | 421 | @keyword 422 | def region_doubleClick(self, target=None, dx=0, dy=0, useLastMatch=False): 423 | ''' 424 | Perform a mouse double-click on the click point using the left button. See `Region Click` for details. 425 | 426 | | Region DoubleClick | image | dx | dy | 427 | ''' 428 | return self._region_mouseAction('doubleClick', target, dx, dy, useLastMatch) 429 | 430 | @keyword 431 | def region_rightClick(self, target=None, dx=0, dy=0, useLastMatch=False): 432 | ''' 433 | Perform a mouse click on the click point using the right button. See `Region Click` for details. 434 | 435 | | Region RightClick | image | dx | dy | 436 | ''' 437 | return self._region_mouseAction('rightClick', target, dx, dy, useLastMatch) 438 | 439 | @keyword 440 | def region_hover(self, target=None, dx=0, dy=0, useLastMatch=False): 441 | ''' 442 | Move the mouse cursor to hover above a click point defined by a target image and coordinates, 443 | i.e. to display a tooltip. See `Region Click` for details. 444 | 445 | | Region Hover | image | dx | dy | 446 | ''' 447 | return self._region_mouseAction('hover', target, dx, dy, useLastMatch) 448 | 449 | @keyword 450 | def region_mouseMove(self, xoff, yoff): 451 | ''' 452 | Move the mouse pointer from it’s current position to the position given by the offset values 453 | (<0: left, up; >0: right, down) 454 | 455 | | Region MouseMove | x | y | 456 | ''' 457 | return self.appScreen.mouseMove(JInt(xoff), JInt(yoff)) 458 | 459 | # Region - highlights operations 460 | @keyword 461 | def region_highlight(self, seconds=0, useLastMatch=True): 462 | ''' 463 | Highlight toggle (switched on if off and vice versa) for the current region (defined with `Region Set Rect`) 464 | or last match region. 465 | 466 | For last match to be used, a last match operation needs to be performed first (e.g. find, wait, existsText and so on). 467 | 468 | | Region Highlight | 10 | 469 | ''' 470 | if useLastMatch: 471 | self._prepare_lastMatch(0, 0) 472 | if self.appMatch == None: 473 | return 0 474 | logger.trace(self.appMatch) 475 | if seconds == 0: 476 | return self.appMatch.highlight() 477 | else: 478 | return self.appMatch.highlight(float(seconds)) 479 | else: 480 | logger.trace(self.appRegion) 481 | if seconds == 0: 482 | return self.appRegion.highlight() 483 | else: 484 | return self.appRegion.highlight(float(seconds)) 485 | 486 | @keyword 487 | def region_highlightAllOff(self): 488 | ''' 489 | Switch off all currently active highlights. 490 | 491 | | Region HighlightAllOff | 492 | ''' 493 | return self.appScreen.highlightAllOff() 494 | 495 | # Region - keyboard operations 496 | @keyword 497 | def region_paste(self, text, target=None, dx=0, dy=0): 498 | ''' 499 | Paste the text at a click point defined by a target image and coordinates. See `Region Click` for more details. 500 | 501 | From SikuliX documentation: Region.click([PSMRL,] text), where PSMRL is a pattern, a string, a match, a region or a location that evaluates to a click point. 502 | 503 | Currently only String, together with parameters that define a pattern will be accepted. 504 | Pattern will need the following parameters, provided as arguments on this keyword 505 | - target - a string naming an image file from known image paths (with or without .png extension) 506 | - similar - minimum similarity. If not given, the default is used. Can be set as img=similarity 507 | - mask - an image with transparent or black parts or 0 for default masked black parts. 508 | Should be set as img:mask, img:0, img:mask=similarity or img:0=similarity 509 | - dx, dy - define click point, either relative to center or relative to upper left corner 510 | (default with `Set Offset Center Mode`) 511 | 512 | If target is omitted, it performs the paste on the current focused component (normally an input field). 513 | 514 | | Region Paste | text | image.png=0.7 | dx | dy | 515 | | Region Paste | text | dx | dy | 516 | ''' 517 | # 1st case, target none - click on default 518 | if target == None: 519 | return self.appScreen.paste(text) 520 | 521 | # 2nd case, define a Pattern from image name - implicit find operation is processed first. 522 | pattern = self._prepare_pattern(target, dx, dy) 523 | self.appRegion.setRect(self.appScreen) 524 | return self.appRegion.paste(pattern, text) 525 | 526 | @keyword 527 | def region_type(self, text, target=None, dx=0, dy=0, modifier=None): 528 | ''' 529 | Type the text at the current focused input field or at a click point specified by target image. 530 | 531 | From SikuliX documentation: Region.type([PSMRL,] text[, modifiers]), where PSMRL is a pattern, 532 | a string, a match, a region or a location that evaluates to a click point. 533 | 534 | Special keys (ENTER, TAB, BACKSPACE) can be incorporated into text using the constants defined in 535 | Class Key using the format SikuliXJClass.Key.Key_String 536 | e.g. SikuliXJClass.Key.ENTER for both key and modifier. Key modifiers can be ALT, CTRL, etc. 537 | 538 | Best Practice: As a general guideline, the best choice is to use `Region Paste` for readable text and 539 | Region Type for action keys like TAB, ENTER, ESC. Use one Region Type for each key or key combination 540 | and be aware, that in some cases a short wait after a type might be necessary to give the target 541 | application some time to react and be prepared for the next SikuliX action. 542 | 543 | | Region Type | text=A | modifier=SikuliXJClass.Key.CTRL | 544 | | Region Type | SikuliXJClass.Key.DELETE | 545 | 546 | ''' 547 | key = text 548 | mod = None 549 | 550 | if "SikuliXJClass.Key" in text: 551 | s_key = text.split(".")[2] 552 | try: 553 | #key = SikuliXJClass.Key.class_.getDeclaredField(s_key).get(None) 554 | key = SikuliXJClass.Key().getClass().getDeclaredField(s_key).get(None) 555 | except: 556 | key = s_key 557 | if modifier and "SikuliXJClass.Key" in modifier: 558 | s_key = modifier.split(".")[2] 559 | #mod = SikuliXJClass.Key.class_.getDeclaredField(s_key).get(None) 560 | mod = SikuliXJClass.Key().getClass().getDeclaredField(s_key).get(None) 561 | 562 | # 1st case, target none - click on default 563 | if target == None: 564 | if modifier == None: 565 | return self.appScreen.type(key) 566 | else: 567 | return self.appScreen.type(key, mod) 568 | 569 | # 2nd case, define a Pattern from image name - implicit find operation is processed first. 570 | pattern = self._prepare_pattern(target, dx, dy) 571 | self.appRegion.setRect(self.appScreen) 572 | if modifier == None: 573 | return self.appRegion.type(pattern, key) 574 | else: 575 | return self.appRegion.type(pattern, key, mod) 576 | 577 | @keyword 578 | def region_dragDrop(self, target1, target2, dx1=0, dy1=0, dx2=0, dy2=0, useLastMatch=False): 579 | ''' 580 | Perform a drag-and-drop operation from a starting click point to the target click point indicated 581 | by the two target images respectively. 582 | 583 | From SikuliX documentation: Region.dragDrop(PSMRL, PSMRL[, modifiers]), where PSMRL is a pattern, 584 | a string, a match, a region or a location that evaluates to a click point. 585 | 586 | Currently only String, together with parameters that define a pattern will be accepted. 587 | Pattern will need the following parameters, provided as arguments on this keyword 588 | - target - a string path to an image file 589 | - similar - minimum similarity. If not given, the default is used. Can be set as img=similarity 590 | - mask - an image with transparent or black parts or 0 for default masked black parts. Should be 591 | set as img:mask, img:0, img:mask=similarity or img:0=similarity 592 | - dx, dy - define click point, either relative to center or relative to upper left corner 593 | (default with `Set Offset Center Mode`) 594 | - useLastMatch - if True, will assume the LastMatch can be used otherwise SikuliX will do a \ 595 | find on the target image and click in the center of it. 596 | 597 | if implicit find operation is needed, assume the region is the whole screen. 598 | 599 | target1 and target2 can be the same image with different click points or separate images 600 | 601 | | Region DragDrop | image1=0.7 | image2 | dx1 | dy1 | dx2 | dy2 | 602 | ''' 603 | # define a Pattern from second image name - implicit find operation is processed first. 604 | pattern2 = self._prepare_pattern(target2, dx2, dy2) 605 | logger.trace(pattern2) 606 | 607 | # match can be given only as lastMatch. Target offset can be null or specified. 608 | if useLastMatch: 609 | self._prepare_lastMatch(dx1, dy1) 610 | self.appRegion.setRect(self.appScreen) 611 | #return SikuliXJClass.Region.class_.getDeclaredMethod("dragDrop", JObject, JObject).invoke(self.appRegion, self.appMatch, pattern2) 612 | self.appRegion.dragDrop(self.appMatch, pattern2) 613 | # define a Pattern from first image name - implicit find operation is processed first. 614 | if not useLastMatch: 615 | pattern1 = self._prepare_pattern(target1, dx1, dy1) 616 | logger.trace(pattern1) 617 | self.appRegion.setRect(self.appScreen) 618 | #return SikuliXJClass.Region.class_.getDeclaredMethod("dragDrop", JObject, JObject).invoke(self.appRegion, pattern1, pattern2) 619 | self.appRegion.dragDrop(pattern1, pattern2) 620 | 621 | # Region - find text operations 622 | def _region_findTextOperation(self, type, text, seconds, onScreen, regionSelect): 623 | self._set_active_region(onScreen, regionSelect) 624 | 625 | try: 626 | if seconds == 0: 627 | logger.trace("Call findTextOperation with arguments: %s" % type) 628 | logger.trace(self.appRegion) 629 | if useJpype: 630 | res = SikuliXJClass.Region.class_.getDeclaredMethod(type, JString).invoke(self.appRegion, text) 631 | else: 632 | res = get_method(self.appRegion, type)(text) 633 | else: 634 | logger.trace("Call findTextOperation with arguments: %s, %s seconds" % (type, seconds)) 635 | logger.trace(self.appRegion) 636 | if useJpype: 637 | res = SikuliXJClass.Region.class_.getDeclaredMethod(type, JString, JDouble).invoke(self.appRegion, 638 | text, JDouble(seconds)) 639 | else: 640 | res = get_method(self.appRegion, type)(text, JDouble(seconds)) 641 | 642 | except: # except should happen only for find or wait 643 | self._failed("Text not visible on screen: " + text, seconds, mode='text') 644 | raise Exception("_Find text method Failed") 645 | 646 | if res: 647 | if type == 'waitVanish': 648 | logger.info('PASS: ' + 'Text vanished from screen') 649 | else: 650 | self._passed("Text visible on screen", mode='text') 651 | else: 652 | self._notfound("Text not visible on screen: " + text, seconds, mode='text') 653 | 654 | return res 655 | 656 | @keyword 657 | def region_findText(self, text, onScreen=True, regionSelect=None): 658 | ''' 659 | Search for given text on screen or within current region. Does not repeat search and throws `FindFailed` if not found. 660 | 661 | From SikuliX documentation: Region.findText(String), where text is the string to search for on screen. 662 | The `Region SetAutoWait` is used. 663 | onScreen - reset the region to the whole screen, otherwise will search on a region defined previously with 664 | set parameters keywords e.g. `Region Set Rect` where the parameters can be from a previous match or known 665 | dimension, etc. 666 | 667 | Be aware other than the image search functions, the text search functions search from top left to bottom right. 668 | So if there is more than one possible match in a region, always the top left match is found. With image search it 669 | is still so, that it cannot be foreseen, which of the possible matches is returned as the result. In doubt you have 670 | to use the functions, that return all matches in a region and then filter the result to your needs. 671 | 672 | `Region FindText` returns the found text. 673 | 674 | | Region FindText | text | 675 | ''' 676 | return self._region_findTextOperation('findText', text, 0, onScreen, regionSelect) 677 | 678 | @keyword 679 | def region_waitText(self, text, seconds=0, onScreen=True, regionSelect=None): 680 | ''' 681 | Wait for the given text to appear on screen or current region. Repeat search and throws if not found during the given 682 | timeout in seconds or as `Region SetAutoWait` previously. 683 | 684 | | Region WaitText | text | ${10} | 685 | ''' 686 | return self._region_findTextOperation('waitText', text, seconds, onScreen, regionSelect) 687 | 688 | @keyword 689 | def region_waitVanishText(self, text, seconds=0, onScreen=True, regionSelect=None): 690 | ''' 691 | According to SikuliX documentation, not implemented yet. 692 | ''' 693 | return self._region_findTextOperation('waitVanishText', text, seconds, onScreen, regionSelect) 694 | 695 | @keyword 696 | def region_existsText(self, text, seconds=0, onScreen=True, regionSelect=None): 697 | ''' 698 | Wait for the given text to appear on screen or current region. Repeat search and does not throws error 699 | if not found during the given timeout in seconds or as `Region SetAutoWait` previously. 700 | 701 | | Region ExistsText | text | ${10} | 702 | ''' 703 | return self._region_findTextOperation('existsText', text, seconds, onScreen, regionSelect) 704 | 705 | @keyword 706 | def region_hasText(self, text, seconds=0, onScreen=True, regionSelect=None): 707 | ''' 708 | Search for given text on screen or within current region. Does not repeat search and does not throws 709 | `FindFailed` if not found. 710 | 711 | | Region HasText | text | 712 | ''' 713 | return self._region_findTextOperation('hasText', text, seconds, onScreen, regionSelect) 714 | 715 | # Region - read text by OCR operations 716 | @keyword 717 | def region_getText(self, onScreen=True, regionSelect=None): 718 | ''' 719 | Captures text from screen or within current region. Returns that text. 720 | ''' 721 | self._set_active_region(onScreen, regionSelect) 722 | 723 | text = self.appRegion.text() 724 | logger.trace('Text read: {}'.format(text)) 725 | return text 726 | 727 | # Region - read text by OCR operations 728 | @keyword 729 | def region_text(self, img): 730 | ''' 731 | Extract and return text from the given found image on screen (by using OCR) 732 | 733 | | Region Text | image.png | 734 | ''' 735 | self.region_find(img) 736 | self.appMatch = self.appRegion.getLastMatch() 737 | text = self.appMatch.text() 738 | return str(text) 739 | 740 | @keyword 741 | def region_screenshot(self, onScreen=True, regionSelect=None): 742 | ''' 743 | Take a screenshot of the specified region and add that to the log file. 744 | ''' 745 | self._set_active_region(onScreen, regionSelect) 746 | region = (self.appRegion.x, self.appRegion.y, self.appRegion.w, self.appRegion.h) 747 | name = self._screenshot("/matches/", region) 748 | rel_path = relpath(name, SikuliXLogger.resultDir) 749 | logger.info('Screenshot: ' % rel_path, True) 750 | 751 | -------------------------------------------------------------------------------- /SikuliXLibrary/sikulixsettings.py: -------------------------------------------------------------------------------- 1 | # MIT license 2 | 3 | from .sikulixjclass import * 4 | 5 | if not useJpype: 6 | from .sikulixpy4j import * 7 | 8 | 9 | class SikuliXSettings(SikuliXJClass): 10 | ''' 11 | SikuliX Settings class 12 | ''' 13 | @keyword 14 | def settings_set(self, variable, value): 15 | ''' 16 | Set the value for any public Settings class variable, see http://doc.sikuli.org/globals.html. There are 17 | many documented and even more undocumented settings that you can manipulate to alter SikuliX' behavior. 18 | These settings include: 19 | 20 | | | Name | type | default | Usage | 21 | | *Behavior* | 22 | | | ThrowException | boolean | true | Throw FindFailed exception | 23 | | | WheelNatural | boolean | true | Setting to false reverses the wheel direction | 24 | | | checkMousePosition | boolean | true | Setting to false supresses error message in RobotDesktop | 25 | | | ActionLogs | boolean | true | Switched to show or hide the respective action message in the log file | 26 | | | InfoLogs | boolean | true | Switched to show or hide the respective info message type in the file | 27 | | | DebugLogs | boolean | false | Switched to show or hide the respective debug message in the log file | 28 | | | ProfileLogs | boolean | false | - | 29 | | | TraceLogs | boolean | false | - | 30 | | | LogTime | boolean | false | - | 31 | | *Timing* | 32 | | | AutoWaitTimeout | float | 3.0 | Timeout for Range Wait operations in seconds | 33 | | | WaitScanRate | float | 3.0 | Specify the number of times actual search operations are performed per \ 34 | second while waiting for a pattern to appear or vanish. | 35 | | | ObserveScanRate | float | 3.0 | Specify the number of times actual search operations are performed per \ 36 | second while waiting for a pattern to appear or vanish. | 37 | | | RepeatWaitTime | int | 1 | Seconds for visual to vanish after action | 38 | | | DelayBeforeMouseDown | double | 0.3 | Delay time for mouse interaction | 39 | | | DelayAfterDrag | double | 0.3 | Specifies the waiting time after mouse down at the source location \ 40 | as a decimal value (seconds). | 41 | | | DelayBeforeDrag | double | -0.3 | Delay time for mouse interaction | 42 | | | DelayBeforeDrop | double | 0.3 | Specifies the waiting time before mouse up at the target location \ 43 | as a decimal value (seconds). | 44 | | | TypeDelay | double | 0.0 | Delay time between two characters, must be < 1 second | 45 | | | ClickDelay | double | 0.0 | Delay time between two mouse down and mouse up, must be < 1 second. \ 46 | Note: ClickDelay is reset to 0.0 on every mouse click | 47 | | | SlowMotionDelay | float | 2.0 | Control the duration of the visual effect (seconds). | 48 | | | MoveMouseDelay | float | 0.5 | Control the time taken for mouse movement to a target location by \ 49 | setting this value to a decimal value (default 0.5). The unit is seconds. Setting it to 0 will \ 50 | switch off any animation (the mouse will “jump” to the target location). | 51 | | *Show Actions* | 52 | | | ShowActions | boolean | false | Use `setShowActions` to change the value of this setting | 53 | | | Highlight | boolean | false | Highlight every match (show red rectangle around) | 54 | | | DefaultHighlightTime | float | 2.0 | Time in seconds to show highlighting rectangle | 55 | | | DefaultHighlightColor | String | "RED" | Color for highlighting rectangle | 56 | | | HighlightTransparent | boolean | false | - | 57 | | | WaitAfterHighlight | double | 0.3 | - | 58 | | *Image Recognition and OCR* | 59 | | | MinSimilarity | double | 0.7 | Similarity required for a positive match 0.0...1.0 | 60 | | | InputFontMono | boolean | false | - | 61 | | | InputFontSize | int | 14 | - | 62 | | | OcrLanguageDefault | string | "eng" | OCR expected language | 63 | 64 | Example usage 65 | | ${prev} | Settings Set | MinSimilarity | ${0.9} | 66 | | Settings Set | Highlight | ${True} | 67 | ''' 68 | if useJpype: 69 | target = SikuliXJClass.Settings.class_.getDeclaredField(variable) 70 | else: 71 | target = get_java_class(SikuliXJClass.Settings).getDeclaredField(variable) 72 | 73 | previous = target.get(None) 74 | variable_type = str(target.getGenericType()) 75 | logger.trace('Setting {}({}) to {}'.format(variable, variable_type, value)) 76 | if variable_type == 'int': 77 | target.set(None, JInt(value)) 78 | elif variable_type == 'float': 79 | target.set(value, JFloat(value)) 80 | elif variable_type == 'double': 81 | target.set(None, JDouble(value)) 82 | elif variable_type == 'boolean': 83 | target.set(None, JBoolean(value)) 84 | else: 85 | target.set(None, value) 86 | 87 | return previous 88 | 89 | @keyword 90 | def settings_get(self, variable): 91 | ''' 92 | Return the value for any public Settings class variable. See http://doc.sikuli.org/globals.html for details and 93 | different variable names that can be set: MinSimilarity, Highlight, ActionLogs, MoveMouseDelay and so on. 94 | 95 | | ${val} | Settings Get | MinSimilarity | 96 | ''' 97 | if useJpype: 98 | return SikuliXJClass.Settings.class_.getDeclaredField(variable).get(None) 99 | else: 100 | #return SikuliXJClass.Settings().getClass().getDeclaredField(variable).get(None) 101 | return get_java_class(SikuliXJClass.Settings).getDeclaredField(variable).get(None) 102 | 103 | @keyword 104 | def settings_setShowActions(self, mode): 105 | ''' 106 | If set to True, when a script is run, SikuliX shows a visual effect (a blinking double lined red circle) 107 | on the spot where the action will take place before executing actions. Default False 108 | 109 | | Settings SetShowAction | ${True} | 110 | ''' 111 | SikuliXJClass.Settings.setShowActions(mode) 112 | 113 | @keyword 114 | def settings_isShowActions(self): 115 | ''' 116 | Return show action mode 117 | 118 | | ${val} | Settings IsShowAction | 119 | ''' 120 | return SikuliXJClass.Settings.isShowActions() 121 | -------------------------------------------------------------------------------- /SikuliXLibrary/version.py: -------------------------------------------------------------------------------- 1 | __version__ = "2.0.0" 2 | -------------------------------------------------------------------------------- /migrate/ImageHorizonLibraryMigration.py: -------------------------------------------------------------------------------- 1 | """ 2 | Description: SikuliX sample custom library, based on the new, SikuliX JPype or Py4J based library, implementing 3 | keywords from ImageHorizonLibrary for convenience migration. 4 | Check https://eficode.github.io/robotframework-imagehorizonlibrary/doc/ImageHorizonLibrary.html for details 5 | """ 6 | 7 | from SikuliXLibrary import SikuliXLibrary 8 | 9 | from robot.api.deco import * 10 | from robot.api import logger 11 | 12 | 13 | @library(scope='GLOBAL', version='0.1') 14 | class ImageHorizonLibraryMigration(SikuliXLibrary): 15 | 16 | @not_keyword 17 | def __init__(self, sikuli_path='', image_path='', logImages=True, centerMode=False): 18 | super().__init__(sikuli_path, image_path, logImages, centerMode) 19 | 20 | @keyword 21 | def click_image(self, target): 22 | # always click on center of the target image 23 | self.region_click(target, 0, 0, False) 24 | 25 | @keyword 26 | def set_confidence(self, new_confidence): 27 | self.settings_set('MinSimilarity', float(new_confidence)) 28 | 29 | @keyword 30 | def wait_for(self, reference_image, timeout=10): 31 | return self.region_wait(reference_image, timeout, True) 32 | -------------------------------------------------------------------------------- /migrate/SikuliLibraryMigration.py: -------------------------------------------------------------------------------- 1 | """ 2 | Description: SikuliX sample custom library, based on the new SikuliX JPype or Py4J library, implementing 3 | keywords from SikuliLibrary (robotframework-sikulilibrary) for convenience migration. 4 | Check http://rainmanwy.github.io/robotframework-SikuliLibrary/doc/SikuliLibrary.html for details 5 | """ 6 | 7 | from SikuliXLibrary import SikuliXLibrary 8 | 9 | from robot.api.deco import * 10 | from robot.api import logger 11 | 12 | 13 | @library(scope='GLOBAL', version='0.1') 14 | class SikuliLibraryMigration(SikuliXLibrary): 15 | 16 | @not_keyword 17 | def __init__(self, sikuli_path='', image_path='', logImages=True, centerMode=False): 18 | super().__init__(sikuli_path, image_path, logImages, centerMode) 19 | 20 | @keyword 21 | def add_image_path(self, path): 22 | self.imagePath_add(path) 23 | 24 | @keyword 25 | def click_(self, image, xOffset=0, yOffset=0): 26 | # this library is using offset center mode 27 | self.offsetCenterMode = True 28 | self.region_click(image, xOffset, yOffset, False) 29 | 30 | @keyword 31 | def exists(self, image, timeout=0): 32 | return self.region_exists(image, timeout, True) 33 | 34 | @keyword 35 | def get_text(self, img): 36 | return self.region_text(img) 37 | 38 | @keyword 39 | def wait_until_screen_contain(self, image, timeout): 40 | self.region_wait(image, timeout, True) 41 | 42 | @keyword 43 | def wait_until_screen_not_contain(self, image, timeout): 44 | self.region_waitVanish(image, timeout, True) 45 | -------------------------------------------------------------------------------- /migrate/SikuliXCustomLibrary.py: -------------------------------------------------------------------------------- 1 | """ 2 | Description: SikuliX sample custom library, based on the new, SikuliX JPype or Py4J based library, implementing custom 3 | keywords (e.g. oneOfTheRegionsShouldExist) and/or override existing keywords for custom functionality (e.g. _passed). 4 | """ 5 | 6 | 7 | from SikuliXLibrary import SikuliXLibrary 8 | 9 | from robot.api.deco import * 10 | from robot.api import logger 11 | 12 | 13 | @library(scope='GLOBAL', version='0.1') 14 | class SikuliXCustomLibrary(SikuliXLibrary): 15 | 16 | @not_keyword 17 | def __init__(self, sikuli_path='', image_path='', logImages=True, centerMode=False): 18 | super().__init__(sikuli_path, image_path, logImages, centerMode) 19 | 20 | # Custom _passed method 21 | @not_keyword 22 | def _passed(self, msg): 23 | #logger.info('PASS: ' + msg) 24 | super()._passed(msg) 25 | 26 | # Custom _failed method 27 | @not_keyword 28 | def _failed(self, msg, seconds): 29 | #logger.info('FAILED: ' + msg) 30 | super()._failed(msg, 0) 31 | 32 | # Custom _notfound method 33 | @not_keyword 34 | def _notfound(self, msg, seconds): 35 | #logger.info('NOT FOUND: ' + msg) 36 | super()._notfound(msg, 0) 37 | 38 | # Sample custom keyword that will wait for the image to appear on screen and if not will try 39 | # again a second search with a decreased similarity with 0.1 value (if repeat flag is set to true) 40 | @keyword 41 | def regionWaitRepeat(self, img, timeout=0, repeatFindWithLowerSimilar=False): 42 | ''' 43 | Check the given image exists on screen, then try again with decreased similarity 44 | ''' 45 | if not repeatFindWithLowerSimilar: 46 | found = self.region_wait(img, timeout) 47 | else: 48 | found = self.region_exists(img, timeout) 49 | if not found and repeatFindWithLowerSimilar: 50 | logger.warn('WARNING: Use decreased similiarity for a new search') 51 | # check the same image again by decreasing the minimum similarity 52 | minSimilar = self.settings_get('MinSimilarity') 53 | target = img + '=' + str(minSimilar - float(0.1)) 54 | self.region_wait(target, timeout) 55 | 56 | # Sample custom keyword that will wait for first given image to appear on screen and if not will wait 57 | # for the second given image to appear on screen. 58 | @keyword 59 | def oneOfTheRegionsShouldExist(self, img1, img2, timeout=0): 60 | ''' 61 | One of the two given images should be found on screen 62 | ''' 63 | if not self.region_exists(img1, timeout): 64 | logger.warn('WARNING: First image not found, wait for the second.') 65 | self.region_wait(img2, timeout) 66 | 67 | # Sample custom keyword that will simply rename the original SikuliXLibrary keyword for convenience 68 | # and easy of understanding the code 69 | @keyword 70 | def waitUntilScreenContains(self, target, timeout=0, onScreen=True): 71 | self.region_wait(target, timeout, onScreen) 72 | 73 | @keyword 74 | def waitUntilScreenDoesNotContain(self, target, timeout=0, onScreen=True): 75 | self.region_waitVanish(target, timeout, onScreen) 76 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import os 3 | from setuptools import setup, find_packages # type: ignore 4 | import sys 5 | 6 | # get __version__ number 7 | exec(compile(open('SikuliXLibrary/version.py', "rb").read(), 'SikuliXLibrary/version.py', 'exec')) 8 | 9 | with open("README.md", encoding="utf-8") as f: 10 | long_description = f.read() 11 | 12 | install_requires = open(os.path.join("SikuliXLibrary", "requirements.txt")).readlines() 13 | 14 | setup_kwargs = { 15 | "name": "robotframework-sikulixlibrary", 16 | "version": __version__, 17 | "description": "Robot Framework SiluliX library powered by SikuliX Java library and JPype or Py4J Python modules.", 18 | "long_description": long_description, 19 | "long_description_content_type": "text/markdown", 20 | "author": "Adrian V.", 21 | "author_email": "avaidos@gmail.com", 22 | "maintainer": None, 23 | "maintainer_email": None, 24 | "url": "https://github.com/adrian-evo/robotframework-sikulixlibrary", 25 | "packages": find_packages(exclude=["test"]), 26 | "include_package_data" : True, 27 | "install_requires": install_requires, 28 | "python_requires": ">=3.7,<4.0", 29 | "classifiers": [ 30 | "Development Status :: 5 - Production/Stable", 31 | "License :: OSI Approved :: MIT License", 32 | "Operating System :: OS Independent", 33 | "Programming Language :: Python :: 3", 34 | "Topic :: Software Development :: Testing", 35 | "Framework :: Robot Framework", 36 | "Framework :: Robot Framework :: Library", 37 | ], 38 | } 39 | 40 | setup(**setup_kwargs) 41 | -------------------------------------------------------------------------------- /test/__init__.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Suite setup in case all tests from this directory should be run as a suite 3 | ... 4 | 5 | Suite Setup SikuliX Suite Setup 6 | Suite Teardown SikuliX Suite Teardown 7 | 8 | 9 | *** Keywords *** 10 | SikuliX Suite Setup 11 | Log SikuliX suite setup 12 | 13 | 14 | SikuliX Suite Teardown 15 | Log SikuliX suite teardown -------------------------------------------------------------------------------- /test/debug test.py: -------------------------------------------------------------------------------- 1 | #!python 2 | 3 | """ 4 | Description: Utility that will allow to debug and break into e.g. custom python libraries. Add a configuration within Visual Studio Code 5 | to run Python file and then execute this file. Set a breakpoint in the Python library as needed. 6 | """ 7 | 8 | from pathlib import Path 9 | 10 | import robot 11 | 12 | if __name__ == "__main__": 13 | robot.run('test/test_defaultlibrary_win.robot', outputdir='test/results/default') 14 | -------------------------------------------------------------------------------- /test/img/MacOS/NewDoc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/NewDoc.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit edited.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit edited.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit menu.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit mod.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit mod.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit typed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit typed.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit window.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit window.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit window2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit window2.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit.png -------------------------------------------------------------------------------- /test/img/MacOS/TextEdit2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/MacOS/TextEdit2.png -------------------------------------------------------------------------------- /test/img/Ubuntu/Leafpad menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Ubuntu/Leafpad menu.png -------------------------------------------------------------------------------- /test/img/Ubuntu/Leafpad mod.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Ubuntu/Leafpad mod.png -------------------------------------------------------------------------------- /test/img/Ubuntu/Leafpad typed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Ubuntu/Leafpad typed.png -------------------------------------------------------------------------------- /test/img/Ubuntu/Leafpad.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Ubuntu/Leafpad.png -------------------------------------------------------------------------------- /test/img/Ubuntu/Leafpad2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Ubuntu/Leafpad2.png -------------------------------------------------------------------------------- /test/img/Windows/iNotepad menu.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Windows/iNotepad menu.PNG -------------------------------------------------------------------------------- /test/img/Windows/iNotepad mod.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Windows/iNotepad mod.PNG -------------------------------------------------------------------------------- /test/img/Windows/iNotepad typed.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Windows/iNotepad typed.PNG -------------------------------------------------------------------------------- /test/img/Windows/iNotepad.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Windows/iNotepad.PNG -------------------------------------------------------------------------------- /test/img/Windows/iNotepad2.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adrian-evo/robotframework-sikulixlibrary/7a7d5db6abab010111b70f2c73fe16b5e023d58c/test/img/Windows/iNotepad2.PNG -------------------------------------------------------------------------------- /test/test.py: -------------------------------------------------------------------------------- 1 | # Sample test file from JPype project to test library functionality under any OS 2 | 3 | import jpype 4 | import jpype.imports 5 | 6 | jpype.startJVM() 7 | import java 8 | import javax 9 | from javax.swing import * 10 | 11 | def createAndShowGUI(): 12 | frame = JFrame("HelloWorldSwing") 13 | frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE) 14 | label = JLabel("Hello World") 15 | frame.getContentPane().add(label) 16 | frame.pack() 17 | frame.setVisible(True) 18 | 19 | # Start an event loop thread to handling gui events 20 | @jpype.JImplements(java.lang.Runnable) 21 | class Launch: 22 | @jpype.JOverride 23 | def run(self): 24 | createAndShowGUI() 25 | javax.swing.SwingUtilities.invokeLater(Launch()) 26 | -------------------------------------------------------------------------------- /test/test_defaultlibrary_osx.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Test case to demonstrate SikuliX library keywords usage 3 | ... Install first with 'pip install robotframework-sikulixlibrary' 4 | ... The reference images from img directory are generated on 1440 x 900 MacOS screen, Dark mode. Regenerate them for different environment 5 | ... 6 | 7 | # Initialize library with sikuli_path or use SIKULI_HOME environment variable (recommended) 8 | Library SikuliXLibrary sikuli_path=sikulixide-2.0.5.jar 9 | #Library SikuliXLibrary sikuli_path=sikulixide-2.0.5.jar image_path= logImages=${True} centerMode=${False} 10 | Library OperatingSystem 11 | 12 | 13 | *** Variables *** 14 | ${IMAGE_DIR} ${CURDIR}/img/MacOS 15 | ${DEFAULT_WAIT} ${15} 16 | 17 | 18 | *** Test Cases *** 19 | Test TextEdit With SikuliX 20 | Set Log Level TRACE 21 | log java bridge 22 | 23 | # local path for image files. 24 | imagePath add ${IMAGE_DIR} 25 | 26 | set sikuli resultDir ${OUTPUT DIR} 27 | Create Directory ${OUTPUT DIR}/matches 28 | Create Directory ${OUTPUT DIR}/screenshots 29 | 30 | region setAutoWait ${DEFAULT_WAIT} 31 | # coordinates relative to upper left corner 32 | set offsetCenterMode ${False} 33 | set notFoundLogImages ${True} 34 | 35 | # for demo purpose 36 | settings setShowActions ${True} 37 | ${prev} settings set Highlight ${True} 38 | ${prev} settings set WaitAfterHighlight ${0.5} 39 | 40 | # default min similarity 41 | ${prev} settings set MinSimilarity ${0.9} 42 | 43 | # step 1 44 | log Step1: open TextEdit 45 | app open TextEdit 46 | region wait NewDoc 47 | region click 48 | region wait TextEdit 1 49 | 50 | #exit here 51 | 52 | # step 2 53 | # message is TextEdit2 not found after removing path 54 | log Step2: TextEdit2 image should not be found 55 | imagePath remove ${IMAGE_DIR} 56 | region exists TextEdit2 57 | 58 | # step 3 59 | log Step3: TextEdit2 should be found after adding image path 60 | imagePath add ${IMAGE_DIR} 61 | region exists TextEdit2 62 | 63 | region paste Welcome to the all new SikuliX RF library 64 | sleep 3 65 | 66 | # step 4 67 | log Step4: delete all typed text and type new one 68 | region type text=A modifier=SikuliXJClass.Key.CTRL 69 | region type SikuliXJClass.Key.DELETE 70 | 71 | region paste Welcome to the all new SikuliX RF libraryy TextEdit edited=0.7 14 60 72 | region wait TextEdit typed 73 | region type SikuliXJClass.Key.BACKSPACE 74 | 75 | # step 5 76 | log Step5: get all region text by OCR 77 | ${text} region text TextEdit typed 78 | log ${text} 79 | 80 | # step 6 81 | log Step6: type new line and new text 82 | region type SikuliXJClass.Key.ENTER 83 | 84 | ${py4j} Get Environment Variable SIKULI_PY4J default=0 85 | Run Keyword If '${py4j}' == '1' 86 | ... region paste Based on Py4J Python module 87 | ... ELSE region paste Based on JPype Python module 88 | 89 | #${prev} settings set Highlight ${False} 90 | 91 | # step 7 92 | log Step7: search and highlight Untitled text 93 | ${found} region existsText Format 94 | log ${found} 95 | region highlight 3 96 | region highlightAllOff 97 | 98 | # step 8 99 | log Step8: right click on TextEdit - will fail if SKIP is not used next 100 | region setFindFailedResponse SKIP 101 | region rightClick TextEdit 48 14 102 | sleep 3 103 | 104 | # step 9 105 | log Step9: use correct image for right click 106 | region setFindFailedResponse ABORT 107 | ${prev} settings set MinSimilarity ${0.7} 108 | #region rightClick TextEdit typed 48 14 109 | 110 | # coordinates relative to upper left corner of the image 111 | region click TextEdit menu 338 8 112 | region click TextEdit window 48 10 113 | 114 | region click TextEdit menu 338 8 115 | 116 | # step 10 117 | # coordinates relative to center of the image 118 | log Step10: same as step 9, with coordinates relative to center 119 | set offsetCenterMode ${True} 120 | region click TextEdit window2 -128 116 121 | 122 | # step 11 123 | log Step11: dragDrop TextEdit 124 | set offsetCenterMode ${False} 125 | ${prev} settings set DelayBeforeDrop ${2.0} 126 | region dragDrop TextEdit typed TextEdit typed 60 12 110 12 127 | 128 | # step 12 129 | log Step12: delete everything and close TextEdit 130 | region type text=A modifier=SikuliXJClass.Key.CTRL 131 | region type SikuliXJClass.Key.DELETE 132 | 133 | app close TextEdit 134 | destroy vm 135 | 136 | *** Keywords *** 137 | Exit Here 138 | app close TextEdit 139 | destroy vm 140 | pass execution . 141 | -------------------------------------------------------------------------------- /test/test_defaultlibrary_ubuntu.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Test case to demonstrate SikuliX library keywords usage 3 | ... Install first with 'pip install robotframework-sikulixlibrary' 4 | ... The reference images from img directory are generated on 1440 x 900 virtual screen, regenerate them for different environment 5 | ... 6 | 7 | # Library SikuliXLibrary 8 | # Initialize library with sikuli_path or use SIKULI_HOME environment variable (recommended) 9 | Library SikuliXLibrary sikuli_path=sikulixide-2.0.5.jar 10 | #Library SikuliXLibrary sikuli_path=/Home/eclipse/sikulix/sikulix.jar image_path= logImages=${True} centerMode=${False} 11 | Library OperatingSystem 12 | 13 | 14 | *** Variables *** 15 | ${IMAGE_DIR} ${CURDIR}/img/Ubuntu 16 | ${DEFAULT_WAIT} ${15} 17 | 18 | 19 | *** Test Cases *** 20 | Test Leafpad With SikuliX 21 | Set Log Level TRACE 22 | log java bridge 23 | 24 | # local path for image files. 25 | imagePath add ${IMAGE_DIR} 26 | 27 | set sikuli resultDir ${OUTPUT DIR} 28 | Create Directory ${OUTPUT DIR}/matches 29 | Create Directory ${OUTPUT DIR}/screenshots 30 | 31 | region setAutoWait ${DEFAULT_WAIT} 32 | # coordinates relative to upper left corner 33 | set offsetCenterMode ${False} 34 | set notFoundLogImages ${True} 35 | 36 | # for demo purpose 37 | settings setShowActions ${True} 38 | ${prev} settings set Highlight ${True} 39 | ${prev} settings set WaitAfterHighlight ${0.9} 40 | 41 | # default min similarity 42 | ${prev} settings set MinSimilarity ${0.9} 43 | 44 | # step 1 45 | log Step1: open Leafpad 46 | 47 | # TODO: Enable after https://github.com/RaiMan/SikuliX1/issues/438 is fixed 48 | #app open leafpad 49 | region wait Leafpad 50 | 51 | #pass execution . 52 | 53 | # step 2 54 | # message is Leafpad2 not found after removing path 55 | log Step2: Leafpad2 image should not be found 56 | imagePath remove ${IMAGE_DIR} 57 | region exists Leafpad2 58 | 59 | # step 3 60 | log Step3: Leafpad2 should be found after adding image path 61 | imagePath add ${IMAGE_DIR} 62 | region exists Leafpad2 63 | 64 | region paste Welcome to the all new SikuliX RF library 65 | sleep 3 66 | 67 | # step 4 68 | log Step4: delete all typed text and type new one 69 | region type text=A modifier=SikuliXJClass.Key.CTRL 70 | region type SikuliXJClass.Key.DELETE 71 | 72 | region paste Welcome to the all new SikuliX RF libraryy Leafpad=0.7 14 60 73 | region wait Leafpad typed 74 | region type SikuliXJClass.Key.BACKSPACE 75 | 76 | # step 5 77 | log Step5: get all region text by OCR 78 | ${text} region text Leafpad typed 79 | log ${text} 80 | 81 | # step 6 82 | log Step6: type new line and new text 83 | region type SikuliXJClass.Key.ENTER 84 | 85 | ${py4j} Get Environment Variable SIKULI_PY4J default=0 86 | Run Keyword If '${py4j}' == '1' 87 | ... region paste Based on Py4J Python module 88 | ... ELSE region paste Based on JPype Python module 89 | 90 | #${prev} settings set Highlight ${False} 91 | 92 | # step 7 93 | log Step7: search and highlight Untitled text 94 | ${found} region existsText *Untitled 95 | log ${found} 96 | region highlight 3 97 | region highlightAllOff 98 | 99 | # step 8 100 | log Step8: right click on Leafpad - will fail if SKIP is not used next 101 | region setFindFailedResponse SKIP 102 | region rightClick Leafpad 48 14 103 | sleep 3 104 | 105 | # step 9 106 | log Step9: use correct image for right click 107 | region setFindFailedResponse ABORT 108 | ${prev} settings set MinSimilarity ${0.7} 109 | region rightClick Leafpad typed 48 14 110 | 111 | # coordinates relative to upper left corner of the image 112 | #region click Leafpad menu 54 80 113 | #app focus Leafpad 114 | 115 | #region click Leafpad typed 50 12 116 | # use previous found region 117 | #region rightClick None 0 0 True 118 | 119 | # step 10 120 | #log Step10: same as step 9, with coordinates relative to center 121 | # coordinates relative to center of the image 122 | #set offsetCenterMode ${True} 123 | #region click Leafpad menu -10 10 124 | #app focus Leafpad 125 | 126 | # step 11 127 | log Step11: dragDrop Leafpad 128 | set offsetCenterMode ${False} 129 | ${prev} settings set DelayBeforeDrop ${2.0} 130 | region dragDrop Leafpad typed Leafpad typed 50 12 100 12 131 | 132 | # step 12 133 | log Step12: delete everything and close Leafpad 134 | region type text=A modifier=SikuliXJClass.Key.CTRL 135 | region type SikuliXJClass.Key.DELETE 136 | 137 | app close leafpad 138 | destroy vm 139 | -------------------------------------------------------------------------------- /test/test_defaultlibrary_win.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Test case to demonstrate SikuliX library keywords usage 3 | ... Install first with 'pip install robotframework-sikulixlibrary' 4 | ... The reference images from img directory are generated on 1920 x 1080 screen, regenerate them for different resolutions 5 | ... 6 | 7 | Library SikuliXLibrary sikuli_path=sikulixide-2.0.5.jar 8 | # Initialize library with sikuli_path or use SIKULI_HOME environment variable (recommended) 9 | #Library SikuliXLibrary sikuli_path=C:/sikuli/sikulix.jar image_path= logImages=${True} centerMode=${False} 10 | Library OperatingSystem 11 | 12 | 13 | *** Variables *** 14 | ${IMAGE_DIR} ${CURDIR}/img/Windows 15 | ${DEFAULT_WAIT} ${5} 16 | 17 | 18 | *** Test Cases *** 19 | Test Notepad With SikuliX 20 | Set Log Level TRACE 21 | log java bridge 22 | 23 | # local path for image files. images below have iNotepad prefix so that to differentiate from text Notepad 24 | imagePath add ${IMAGE_DIR} 25 | 26 | set sikuli resultDir ${OUTPUT DIR} 27 | Create Directory ${OUTPUT DIR}/matches 28 | Create Directory ${OUTPUT DIR}/screenshots 29 | 30 | region setAutoWait ${DEFAULT_WAIT} 31 | # coordinates relative to upper left corner 32 | set offsetCenterMode ${False} 33 | set notFoundLogImages ${True} 34 | 35 | # for demo purpose 36 | settings setShowActions ${True} 37 | ${prev} settings set Highlight ${True} 38 | ${prev} settings set WaitAfterHighlight ${0.9} 39 | 40 | # default min similarity 41 | ${prev} settings set MinSimilarity ${0.9} 42 | 43 | region setRect 0 0 1920 1080 44 | region setRect ${0} ${0} ${1920} ${1080} 45 | 46 | # step 1 47 | log Step1: open Notepad 48 | app open C:/Windows/System32/notepad.exe 49 | #region wait iNotepad mod.PNG 50 | 51 | # different mask and similarity options given 52 | region wait iNotepad.PNG 53 | #region wait iNotepad.PNG:0 54 | #region wait iNotepad.PNG:iNotepad2 55 | #region wait iNotepad.PNG:0=0.69 56 | #region wait iNotepad.PNG:iNotepad2=0.71 57 | #region wait iNotepad:iNotepad2=0.71 58 | #region wait iNotepad.PNG:iNotepad2.PNG=0.71 59 | 60 | region click iNotepad ${50} ${50} 61 | region mouseMove 100 100 62 | region click iNotepad 50 50 63 | 64 | #pass execution . 65 | 66 | # step 2 67 | # message is Notepad2 not found after removing path 68 | log Step2: iNotepad2 image should not be found 69 | imagePath remove ${IMAGE_DIR} 70 | #region wait iNotepad2.PNG 71 | region exists iNotepad2.PNG 72 | 73 | # step 3 74 | log Step3: iNotepad2 should be found after adding image path 75 | imagePath add ${IMAGE_DIR} 76 | region exists iNotepad2 77 | 78 | region paste Welcome to the all new SikuliX RF library 79 | sleep 3 80 | 81 | # step 4 82 | log Step4: delete all typed text and type new one 83 | region type text=A modifier=SikuliXJClass.Key.CTRL 84 | region type SikuliXJClass.Key.DELETE 85 | 86 | region paste Welcome to the all new SikuliX RF libraryy iNotepad=0.7 14 60 87 | region wait iNotepad typed 88 | region type SikuliXJClass.Key.BACKSPACE 89 | 90 | # step 5 91 | log Step5: get all region text by OCR 92 | ${text} region text iNotepad typed 93 | log ${text} 94 | 95 | # step 6 96 | log Step6: type new line and new text 97 | region type SikuliXJClass.Key.ENTER 98 | 99 | ${py4j} Get Environment Variable SIKULI_PY4J default=0 100 | Run Keyword If '${py4j}' == '1' 101 | ... region paste Based on Py4J Python module 102 | ... ELSE region paste Based on JPype Python module 103 | 104 | #${prev} settings set Highlight ${False} 105 | 106 | # step 7 107 | log Step7: search and highlight Untitled text 108 | ${found} region existsText *Untitled 109 | log ${found} 110 | region highlight 3 111 | region highlightAllOff 112 | 113 | # step 8 114 | log Step8: right click on iNotepad - will fail if SKIP is not used next 115 | region setFindFailedResponse SKIP 116 | region rightClick iNotepad 48 14 117 | sleep 3 118 | 119 | # step 9 120 | log Step9: use correct image for right click 121 | region setFindFailedResponse ABORT 122 | ${prev} settings set MinSimilarity ${0.7} 123 | region rightClick iNotepad typed 48 14 124 | 125 | # coordinates relative to upper left corner of the image 126 | region click iNotepad menu 54 80 127 | app focus Notepad 128 | 129 | region click iNotepad typed 50 12 130 | # use previous found region 131 | region rightClick None 0 0 True 132 | 133 | # step 10 134 | log Step10: same as step 9, with coordinates relative to center 135 | # coordinates relative to center of the image 136 | set offsetCenterMode ${True} 137 | region click iNotepad menu -10 10 138 | app focus Notepad 139 | 140 | # step 11 141 | log Step11: dragDrop Notepad 142 | set offsetCenterMode ${False} 143 | ${prev} settings set DelayBeforeDrop ${2.0} 144 | region dragDrop iNotepad typed iNotepad typed 50 12 100 12 145 | 146 | # step 12 147 | log Step12: delete everything and close Notepad 148 | region type text=A modifier=SikuliXJClass.Key.CTRL 149 | region type SikuliXJClass.Key.DELETE 150 | 151 | app close Notepad 152 | destroy vm 153 | -------------------------------------------------------------------------------- /test/test_imagehorizonlibrarymigration_win.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Test case to demonstrate ImageHorizonLibraryMigration keywords usage 3 | ... Details: 4 | ... 5 | ... 6 | 7 | Library ${CURDIR}/../migrate/ImageHorizonLibraryMigration.py sikuli_path=sikulixide-2.0.5.jar 8 | Library OperatingSystem 9 | 10 | 11 | *** Variables *** 12 | ${IMAGE_DIR} ${CURDIR}/img/Windows 13 | ${DEFAULT_WAIT} ${5} 14 | 15 | 16 | *** Test Cases *** 17 | Test Notepad With ImageHorizonLibraryMigration 18 | Set Log Level TRACE 19 | log java bridge 20 | 21 | # local path for image files. images below have iNotepad prefix so that to differentiate from text Notepad 22 | imagePath add ${IMAGE_DIR} 23 | 24 | set sikuli resultDir ${OUTPUT DIR} 25 | Create Directory ${OUTPUT DIR}/matches 26 | Create Directory ${OUTPUT DIR}/screenshots 27 | 28 | region setAutoWait ${DEFAULT_WAIT} 29 | # coordinates relative to upper left corner 30 | #set offsetCenterMode ${False} 31 | set notFoundLogImages ${True} 32 | 33 | # for demo purpose 34 | settings setShowActions ${True} 35 | ${prev} settings set Highlight ${True} 36 | ${prev} settings set WaitAfterHighlight ${0.9} 37 | 38 | # default min similarity 39 | #${prev} settings set MinSimilarity ${0.7} 40 | set confidence ${0.7} 41 | 42 | app open C:/Windows/System32/notepad.exe 43 | 44 | wait for iNotepad ${DEFAULT_WAIT} 45 | click image iNotepad 46 | 47 | set confidence ${0.5} 48 | wait for iNotepad mod 49 | click image iNotepad 50 | 51 | app close Notepad 52 | destroy vm 53 | -------------------------------------------------------------------------------- /test/test_sikulilibrarymigration_win.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Test case to demonstrate SikuliLibraryMigration keywords usage 3 | ... Details: 4 | ... 5 | ... 6 | 7 | Library ${CURDIR}/../migrate/SikuliLibraryMigration.py sikuli_path=sikulixide-2.0.5.jar 8 | Library OperatingSystem 9 | 10 | 11 | *** Variables *** 12 | ${IMAGE_DIR} ${CURDIR}/img/Windows 13 | ${DEFAULT_WAIT} ${5} 14 | 15 | 16 | *** Test Cases *** 17 | Test Notepad With SikuliLibraryMigration 18 | Set Log Level TRACE 19 | log java bridge 20 | 21 | # local path for image files. images below have iNotepad prefix so that to differentiate from text Notepad 22 | add image path ${IMAGE_DIR} 23 | 24 | set sikuli resultDir ${OUTPUT DIR} 25 | Create Directory ${OUTPUT DIR}/matches 26 | Create Directory ${OUTPUT DIR}/screenshots 27 | 28 | region setAutoWait ${DEFAULT_WAIT} 29 | # coordinates relative to upper left corner 30 | #set offsetCenterMode ${False} 31 | set notFoundLogImages ${True} 32 | 33 | # for demo purpose 34 | settings setShowActions ${True} 35 | ${prev} settings set Highlight ${True} 36 | ${prev} settings set WaitAfterHighlight ${0.9} 37 | 38 | # default min similarity 39 | ${prev} settings set MinSimilarity ${0.7} 40 | 41 | app open C:/Windows/System32/notepad.exe 42 | 43 | wait until screen contain iNotepad ${DEFAULT_WAIT} 44 | click iNotepad 45 | wait until screen not contain iNotepad mod ${DEFAULT_WAIT} 46 | 47 | exists iNotepad 48 | exists iNotepad mod=0.6 49 | click iNotepad 50 | 51 | ${text} get text iNotepad 52 | 53 | app close Notepad 54 | destroy vm 55 | -------------------------------------------------------------------------------- /test/test_sikulixcustomlibrary_win.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Documentation Test case to demonstrate SikuliXCustomLibrary keywords usage 3 | ... Details: 4 | ... 5 | ... 6 | 7 | Library ${CURDIR}/../migrate/SikuliXCustomLibrary.py sikuli_path=sikulixide-2.0.5.jar 8 | Library OperatingSystem 9 | 10 | 11 | *** Variables *** 12 | ${IMAGE_DIR} ${CURDIR}/img/Windows 13 | ${DEFAULT_WAIT} ${5} 14 | 15 | 16 | *** Test Cases *** 17 | Test Notepad With SikuliXCustomLibrary 18 | Set Log Level TRACE 19 | log java bridge 20 | 21 | # local path for image files. images below have iNotepad prefix so that to differentiate from text Notepad 22 | imagePath add ${IMAGE_DIR} 23 | 24 | set sikuli resultDir ${OUTPUT DIR} 25 | Create Directory ${OUTPUT DIR}/matches 26 | Create Directory ${OUTPUT DIR}/screenshots 27 | 28 | region setAutoWait ${DEFAULT_WAIT} 29 | # coordinates relative to upper left corner 30 | set offsetCenterMode ${False} 31 | set notFoundLogImages ${True} 32 | 33 | # for demo purpose 34 | settings setShowActions ${True} 35 | ${prev} settings set Highlight ${True} 36 | ${prev} settings set WaitAfterHighlight ${0.9} 37 | 38 | # default min similarity 39 | ${prev} settings set MinSimilarity ${0.7} 40 | 41 | app open C:/Windows/System32/notepad.exe 42 | 43 | wait until screen contains iNotepad 44 | wait until screen does not contain iNotepad mod 45 | 46 | #region wait iNotepad mod.PNG 47 | #region wait repeat iNotepad mod.PNG 48 | region wait repeat iNotepad mod.PNG repeatFindWithLowerSimilar=${True} 49 | 50 | one of the regions should exist iNotepad mod iNotepad 51 | 52 | app close Notepad 53 | destroy vm 54 | -------------------------------------------------------------------------------- /test/testlibrary_osx.py: -------------------------------------------------------------------------------- 1 | # Python test case to demonstrate SikuliX library keywords usage with Python 2 | 3 | 4 | from SikuliXLibrary import SikuliXLibrary 5 | import time, os, sys 6 | 7 | if __name__ == "__main__": 8 | start_time = time.time() 9 | # local path for image files. 10 | img_path = os.getcwd() + '/img/MacOS' 11 | print(img_path) 12 | if not os.path.exists(img_path): 13 | print("Wrong image path") 14 | sys.exit() 15 | 16 | # sikuli_path: empty for path from SIKULI_HOME + sikulix.jar, not empty might be either SIKULI_HOME + given jar name or full path 17 | sikuli_path = 'sikulixide-2.0.5.jar' 18 | 19 | lib = SikuliXLibrary(sikuli_path, img_path, logImages=False) 20 | lib.log_java_bridge() 21 | 22 | lib.set_sikuli_resultDir('.') 23 | pre = lib.region_getAutoWait() 24 | lib.region_setAutoWait(5) 25 | print('Region AutWaitTimeout (3.0 -> 5.0): %s -> %s' % (pre, lib.region_getAutoWait())) 26 | # coordinates relative to upper left corner 27 | lib.set_offsetCenterMode(False) 28 | 29 | # for demo purpose 30 | pre = lib.settings_isShowActions() 31 | lib.settings_setShowActions(True) 32 | print('Settings ShowActions (False -> True): %s -> %s' % (pre, lib.settings_isShowActions())) 33 | 34 | pre = lib.settings_set('Highlight', True) 35 | print('Settings Highlight (False -> True): %s -> %s' % (pre, lib.settings_get('Highlight'))) 36 | 37 | pre = lib.settings_set('WaitAfterHighlight', 0.5) 38 | print('Settings WaitAfterHighlight (0.3 -> 0.5): %s -> %s' % (pre, lib.settings_get('WaitAfterHighlight'))) 39 | 40 | # default min similarity 41 | pre = lib.settings_set('MinSimilarity', float(0.9)) 42 | print('Settings MinSimilarity (0.7 -> 0.9): %s -> %s' % (pre, lib.settings_get('MinSimilarity'))) 43 | 44 | print('=======Step: open TextEdit') 45 | lib.app_open('TextEdit') 46 | lib.region_wait('NewDoc.png') 47 | lib.region_click() 48 | print('Wait with timeout: 1 second') 49 | lib.region_wait('TextEdit.png', 1) 50 | 51 | def exit_here(): 52 | lib.app_close('TextEdit') 53 | print("done") 54 | lib.destroy_vm() 55 | print('Run time: %s seconds' % (time.time() - start_time)) 56 | sys.exit() 57 | 58 | #exit_here() 59 | 60 | # message is TextEdit2 not found after removing path 61 | print('=======Step: TextEdit2 image should not be found') 62 | lib.imagePath_remove(img_path) 63 | res = lib.region_exists('TextEdit2.PNG') 64 | print('Exists (None): ', res) 65 | 66 | print('=======Step: TextEdit2 should be found after adding image path') 67 | lib.imagePath_add(img_path) 68 | res = lib.region_has('TextEdit2', 10) 69 | print('Exist - Has (True): ', res) 70 | 71 | #sys.exit() 72 | 73 | lib.region_paste('Welcome to the all new SikuliX RF library') 74 | time.sleep(3) 75 | 76 | print('=======Step: waitVanish - use SKIP to avoid exception') 77 | pre = lib.region_getFindFailedResponse() 78 | lib.region_setFindFailedResponse('SKIP') 79 | print('FindFailed (ABORT -> SKIP): %s -> %s' % (pre, lib.region_getFindFailedResponse())) 80 | res = lib.region_waitVanish('TextEdit typed', 5) 81 | print('Not vanished (False): ', res) 82 | pre = lib.region_getFindFailedResponse() 83 | lib.region_setFindFailedResponse('ABORT') 84 | print('FindFailed (SKIP -> ABORT): %s -> %s' % (pre, lib.region_getFindFailedResponse())) 85 | 86 | print('=======Step: delete all typed text') 87 | lib.region_type(text='A', modifier='SikuliXJClass.Key.CTRL') 88 | lib.region_type('SikuliXJClass.Key.DELETE') 89 | 90 | print('=======Step: waitVanish after delete') 91 | res = lib.region_waitVanish('TextEdit typed', 5) 92 | print('Vanished (True): ', res) 93 | 94 | lib.region_paste('Welcome to the all new SikuliX RF library ', 'TextEdit edited=0.7', 14, 60) 95 | lib.region_wait('TextEdit typed', 10) 96 | lib.region_type('SikuliXJClass.Key.BACKSPACE') 97 | 98 | print('=======Step: get all region text by OCR') 99 | text = lib.region_text('TextEdit typed') 100 | print('OCR text: ', text) 101 | 102 | print('=======Step: type new line and new text') 103 | lib.region_type('SikuliXJClass.Key.ENTER') 104 | if os.getenv('SIKULI_PY4J') == '1': 105 | lib.region_paste('Based on Py4J Python module') 106 | else: 107 | lib.region_paste('Based on JPype Python module') 108 | 109 | print('=======Step: search and highlight Format text') 110 | found = lib.region_existsText('Format') 111 | print('Text found (Match): ', found) 112 | lib.region_highlight(3) 113 | lib.region_highlightAllOff() 114 | 115 | print ('=======Step: double click found text, by using last match') 116 | lib.region_doubleClick(useLastMatch=True) 117 | time.sleep(3) 118 | lib.region_waitText('Format', 5) 119 | lib.region_doubleClick(useLastMatch=True) 120 | 121 | print('=======Step: right click on TextEdit - will fail if SKIP is not used next') 122 | lib.region_setFindFailedResponse('SKIP') 123 | print('FindFailed (SKIP): ', lib.region_getFindFailedResponse()) 124 | lib.region_rightClick('TextEdit', 48, 14) 125 | 126 | lib.region_setRect(1, 2, 400, 200) 127 | ret = lib.region_find('TextEdit', onScreen=False) 128 | print('Find on screen should fail (None): ', ret) 129 | 130 | print ('=======Step: use correct image for right click') 131 | lib.region_setFindFailedResponse('ABORT') 132 | print('FindFailed (ABORT): ', lib.region_getFindFailedResponse()) 133 | 134 | pre = lib.settings_set('MinSimilarity', float(0.7)) 135 | print('Settings MinSimilarity (0.9 -> 0.7): %s -> %s' % (pre, lib.settings_get('MinSimilarity'))) 136 | 137 | res = lib.region_has('TextEdit typed=0.99') 138 | print('Has with 0.99 similiarity: (False)', res) 139 | 140 | #lib.region_rightClick('TextEdit typed', 312, 8) 141 | 142 | # coordinates relative to upper left corner of the image 143 | lib.region_click('TextEdit menu', 338, 8) 144 | lib.region_click('TextEdit window', 48, 10) 145 | #lib.app_focus('TextEdit') 146 | 147 | lib.region_click('TextEdit menu', 338, 8) 148 | # use previous found region 149 | #lib.region_rightClick(None, 0, 0, True) 150 | 151 | print('=======Step: click with coordinates relative to center') 152 | # coordinates relative to center of the image 153 | lib.set_offsetCenterMode(True) 154 | lib.region_click('TextEdit window2', -128, 116) 155 | #lib.app_focus('TextEdit') 156 | 157 | print('=======Step: dragDrop TextEdit') 158 | lib.set_offsetCenterMode(False) 159 | prev = lib.settings_set('DelayBeforeDrop', float(2.0)) 160 | lib.region_dragDrop('TextEdit typed', 'TextEdit typed', 60, 12, 110, 12) 161 | 162 | print('=======Step: delete everything') 163 | lib.region_find('TextEdit typed') 164 | lib.region_type(text='A', modifier='SikuliXJClass.Key.CTRL') 165 | lib.region_type('SikuliXJClass.Key.DELETE') 166 | 167 | print('=======Step: hover, click, double click') 168 | lib.region_hover('TextEdit typed', dx=10, dy=10) 169 | lib.region_mouseMove(100, 200) 170 | lib.region_doubleClick('TextEdit typed', 100, 300) 171 | 172 | exit_here() 173 | -------------------------------------------------------------------------------- /test/testlibrary_ubuntu.py: -------------------------------------------------------------------------------- 1 | # Python test case to demonstrate SikuliX library keywords usage with Python 2 | 3 | 4 | from SikuliXLibrary import SikuliXLibrary 5 | import time, os, sys 6 | 7 | if __name__ == "__main__": 8 | start_time = time.time() 9 | # local path for image files. 10 | img_path = os.getcwd() + '/img/Ubuntu' 11 | print(img_path) 12 | if not os.path.exists(img_path): 13 | print("Wrong image path") 14 | sys.exit() 15 | 16 | # sikuli_path: empty for path from SIKULI_HOME + sikulix.jar, not empty might be either SIKULI_HOME + given jar name or full path 17 | sikuli_path = 'sikulixide-2.0.5.jar' 18 | 19 | lib = SikuliXLibrary(sikuli_path, img_path, logImages=False) 20 | lib.log_java_bridge() 21 | 22 | lib.set_sikuli_resultDir('.') 23 | pre = lib.region_getAutoWait() 24 | lib.region_setAutoWait(5) 25 | print('Region AutWaitTimeout (3.0 -> 5.0): %s -> %s' % (pre, lib.region_getAutoWait())) 26 | # coordinates relative to upper left corner 27 | lib.set_offsetCenterMode(False) 28 | 29 | # for demo purpose 30 | pre = lib.settings_isShowActions() 31 | lib.settings_setShowActions(True) 32 | print('Settings ShowActions (False -> True): %s -> %s' % (pre, lib.settings_isShowActions())) 33 | 34 | pre = lib.settings_set('Highlight', True) 35 | print('Settings Highlight (False -> True): %s -> %s' % (pre, lib.settings_get('Highlight'))) 36 | 37 | pre = lib.settings_set('WaitAfterHighlight', 0.5) 38 | print('Settings WaitAfterHighlight (0.3 -> 0.5): %s -> %s' % (pre, lib.settings_get('WaitAfterHighlight'))) 39 | 40 | # default min similarity 41 | pre = lib.settings_set('MinSimilarity', float(0.9)) 42 | print('Settings MinSimilarity (0.7 -> 0.9): %s -> %s' % (pre, lib.settings_get('MinSimilarity'))) 43 | 44 | print('=======Step: open Leafpad') 45 | 46 | # TODO: Enable after https://github.com/RaiMan/SikuliX1/issues/438 is fixed 47 | #lib.app_open("leafpad") 48 | 49 | lib.region_wait('Leafpad') 50 | print('Wait with timeout: 1 second') 51 | lib.region_wait('Leafpad', 1) 52 | 53 | def exit_here(): 54 | lib.app_close('Leafpad') 55 | print("done") 56 | lib.destroy_vm() 57 | print('Run time: %s seconds' % (time.time() - start_time)) 58 | sys.exit() 59 | 60 | #exit_here() 61 | 62 | # message is Leafpad2 not found after removing path 63 | print('=======Step: Leafpad2 image should not be found') 64 | lib.imagePath_remove(img_path) 65 | res = lib.region_exists('Leafpad2') 66 | print('Exists (None): ', res) 67 | 68 | print('=======Step: Leafpad2 should be found after adding image path') 69 | lib.imagePath_add(img_path) 70 | res = lib.region_has('Leafpad2', 10) 71 | print('Exist - Has (True): ', res) 72 | 73 | #sys.exit() 74 | 75 | lib.region_paste('Welcome to the all new SikuliX RF library') 76 | time.sleep(3) 77 | 78 | print('=======Step: waitVanish - use SKIP to avoid exception') 79 | pre = lib.region_getFindFailedResponse() 80 | lib.region_setFindFailedResponse('SKIP') 81 | print('FindFailed (ABORT -> SKIP): %s -> %s' % (pre, lib.region_getFindFailedResponse())) 82 | res = lib.region_waitVanish('Leafpad typed', 5) 83 | print('Not vanished: ', res) 84 | pre = lib.region_getFindFailedResponse() 85 | lib.region_setFindFailedResponse('ABORT') 86 | print('FindFailed (SKIP -> ABORT): %s -> %s' % (pre, lib.region_getFindFailedResponse())) 87 | 88 | print('=======Step: delete all typed text') 89 | lib.region_type(text='A', modifier='SikuliXJClass.Key.CTRL') 90 | lib.region_type('SikuliXJClass.Key.DELETE') 91 | 92 | print('=======Step: waitVanish after delete') 93 | res = lib.region_waitVanish('Leafpad typed', 5) 94 | print('Vanished: ', res) 95 | 96 | lib.region_paste('Welcome to the all new SikuliX RF library ', 'Leafpad=0.7', 14, 60) 97 | lib.region_wait('Leafpad typed', 10) 98 | lib.region_type('SikuliXJClass.Key.BACKSPACE') 99 | 100 | print('=======Step: get all region text by OCR') 101 | text = lib.region_text('Leafpad typed') 102 | print('OCR text: ', text) 103 | 104 | print('=======Step: type new line and new text') 105 | lib.region_type('SikuliXJClass.Key.ENTER') 106 | if os.getenv('SIKULI_PY4J') == '1': 107 | lib.region_paste('Based on Py4J Python module') 108 | else: 109 | lib.region_paste('Based on JPype Python module') 110 | 111 | print('=======Step: search and highlight Untitled text') 112 | found = lib.region_existsText('Untitled') 113 | print('Text found (Match): ', found) 114 | lib.region_highlight(3) 115 | lib.region_highlightAllOff() 116 | 117 | print ('=======Step: double click found text, by using last match') 118 | lib.region_doubleClick(useLastMatch=True) 119 | time.sleep(3) 120 | lib.region_waitText('Untitled', 5) 121 | lib.region_doubleClick(useLastMatch=True) 122 | 123 | print('=======Step: right click on Leafpad - will fail if SKIP is not used next') 124 | lib.region_setFindFailedResponse('SKIP') 125 | print('FindFailed (SKIP): ', lib.region_getFindFailedResponse()) 126 | lib.region_rightClick('Leafpad', 48, 14) 127 | 128 | lib.region_setRect(1, 2, 500, 300) 129 | ret = lib.region_find('Leafpad', onScreen=False) 130 | print('Find on screen should fail: ', ret) 131 | 132 | print ('=======Step: use correct image for right click') 133 | lib.region_setFindFailedResponse('ABORT') 134 | print('FindFailed (ABORT): ', lib.region_getFindFailedResponse()) 135 | 136 | pre = lib.settings_set('MinSimilarity', float(0.7)) 137 | print('Settings MinSimilarity (0.9 -> 0.7): %s -> %s' % (pre, lib.settings_get('MinSimilarity'))) 138 | 139 | res = lib.region_has('Leafpad typed=0.99') 140 | print('Has with 0.99 similiarity: (None)', res) 141 | 142 | #lib.region_rightClick('Leafpad typed', 48, 14) 143 | 144 | # coordinates relative to upper left corner of the image 145 | #lib.region_click('Leafpad menu', 54, 80) 146 | #lib.app_focus('Leafpad') 147 | 148 | #lib.region_click('Leafpad typed', 50, 12) 149 | # use previous found region 150 | #lib.region_rightClick(None, 0, 0, True) 151 | 152 | #print('=======Step: click with coordinates relative to center') 153 | # coordinates relative to center of the image 154 | #lib.set_offsetCenterMode(True) 155 | #lib.region_click('Leafpad menu', -10, 10) 156 | #lib.app_focus('Leafpad') 157 | 158 | #print('=======Step: dragDrop Leapfad') 159 | #lib.set_offsetCenterMode(False) 160 | #prev = lib.settings_set('DelayBeforeDrop', float(2.0)) 161 | #lib.region_dragDrop('Leafpad typed', 'Leafpad typed', 50, 12, 100, 12) 162 | 163 | #print('=======Step: delete everything') 164 | #lib.region_type(text='A', modifier='SikuliXJClass.Key.CTRL') 165 | #lib.region_type('SikuliXJClass.Key.DELETE') 166 | 167 | print('=======Step: hover, click, double click') 168 | lib.region_hover('Leafpad typed', dx=10, dy=10) 169 | lib.region_mouseMove(100, 200) 170 | lib.region_doubleClick('Leafpad', 100, 300) 171 | 172 | exit_here() 173 | -------------------------------------------------------------------------------- /test/testlibrary_win.py: -------------------------------------------------------------------------------- 1 | # Python test case to demonstrate SikuliX library keywords usage with Python 2 | 3 | 4 | from SikuliXLibrary import SikuliXLibrary 5 | 6 | import time, os, sys 7 | from ntpath import abspath 8 | 9 | if __name__ == "__main__": 10 | start_time = time.time() 11 | # local path for image files. Images below have iNotepad prefix so that to differentiate from text Notepad 12 | img_path = abspath('./img/Windows') 13 | print(img_path) 14 | if not os.path.exists(img_path): 15 | print("Wrong image path") 16 | #sys.exit() 17 | 18 | # sikuli_path: empty for path from SIKULI_HOME + sikulix.jar, not empty might be either SIKULI_HOME + given jar name or full path 19 | sikuli_path = 'sikulixide-2.0.5.jar' 20 | 21 | lib = SikuliXLibrary(sikuli_path, img_path, logImages=False) 22 | lib.log_java_bridge() 23 | 24 | lib.set_sikuli_resultDir('.') 25 | pre = lib.region_getAutoWait() 26 | lib.region_setAutoWait(5) 27 | print('Region AutWaitTimeout (3.0 -> 5.0): %s -> %s' % (pre, lib.region_getAutoWait())) 28 | # coordinates relative to upper left corner 29 | lib.set_offsetCenterMode(False) 30 | 31 | # for demo purpose 32 | pre = lib.settings_isShowActions() 33 | lib.settings_setShowActions(True) 34 | print('Settings ShowActions (False -> True): %s -> %s' % (pre, lib.settings_isShowActions())) 35 | 36 | pre = lib.settings_set('Highlight', True) 37 | print('Settings Highlight (False -> True): %s -> %s' % (pre, lib.settings_get('Highlight'))) 38 | 39 | pre = lib.settings_set('WaitAfterHighlight', 0.5) 40 | print('Settings WaitAfterHighlight (0.3 -> 1): %s -> %s' % (pre, lib.settings_get('WaitAfterHighlight'))) 41 | 42 | # default min similarity 43 | pre = lib.settings_set('MinSimilarity', float(0.9)) 44 | print('Settings MinSimilarity (0.7 -> 0.9): %s -> %s' % (pre, lib.settings_get('MinSimilarity'))) 45 | 46 | print('=======Step: open Notepad') 47 | lib.app_open("C:\\Windows\\System32\\notepad.exe") 48 | lib.region_wait('iNotepad.PNG') 49 | print('Wait with timeout: 1 second') 50 | lib.region_wait('iNotepad.PNG', 1) 51 | 52 | def exit_here(): 53 | lib.app_close('Notepad') 54 | print("done") 55 | lib.destroy_vm() 56 | print('Run time: %s seconds' % (time.time() - start_time)) 57 | sys.exit() 58 | 59 | #exit_here() 60 | 61 | # message is Notepad2 not found after removing path 62 | print('=======Step: iNotepad2 image should not be found') 63 | lib.imagePath_remove(img_path) 64 | res = lib.region_exists('iNotepad2.PNG') 65 | print('Exists (None): ', res) 66 | 67 | print('=======Step: iNotepad2 should be found after adding image path') 68 | lib.imagePath_add(img_path) 69 | res = lib.region_has('iNotepad2', 10) 70 | print('Exist - Has (True): ', res) 71 | 72 | lib.region_paste('Welcome to the all new SikuliX RF library') 73 | time.sleep(3) 74 | 75 | print('=======Step: waitVanish - use SKIP to avoid exception') 76 | pre = lib.region_getFindFailedResponse() 77 | lib.region_setFindFailedResponse('SKIP') 78 | print('FindFailed (ABORT -> SKIP): %s -> %s' % (pre, lib.region_getFindFailedResponse())) 79 | res = lib.region_waitVanish('iNotepad typed', 5) 80 | print('Not vanished: ', res) 81 | pre = lib.region_getFindFailedResponse() 82 | lib.region_setFindFailedResponse('ABORT') 83 | print('FindFailed (SKIP -> ABORT): %s -> %s' % (pre, lib.region_getFindFailedResponse())) 84 | 85 | print('=======Step: delete all typed text') 86 | lib.region_type(text='A', modifier='SikuliXJClass.Key.CTRL') 87 | lib.region_type('SikuliXJClass.Key.DELETE') 88 | 89 | print('=======Step: waitVanish after delete') 90 | res = lib.region_waitVanish('iNotepad typed', 5) 91 | print('Vanished: ', res) 92 | 93 | lib.region_paste('Welcome to the all new SikuliX RF library ', 'iNotepad=0.7', 14, 60) 94 | lib.region_wait('iNotepad typed', 10) 95 | lib.region_type('SikuliXJClass.Key.BACKSPACE') 96 | 97 | print('=======Step: get all region text by OCR') 98 | text = lib.region_text('iNotepad typed') 99 | print('OCR text: ', text) 100 | 101 | print('=======Step: type new line and new text') 102 | lib.region_type('SikuliXJClass.Key.ENTER') 103 | if os.getenv('SIKULI_PY4J') == '1': 104 | lib.region_paste('Based on Py4J Python module') 105 | else: 106 | lib.region_paste('Based on JPype Python module') 107 | 108 | print('=======Step: search and highlight Untitled text') 109 | found = lib.region_existsText('Untitled') 110 | print('Text found (Match): ', found) 111 | lib.region_highlight(3) 112 | lib.region_highlightAllOff() 113 | 114 | print ('=======Step: double click found text, by using last match') 115 | lib.region_doubleClick(useLastMatch=True) 116 | time.sleep(3) 117 | lib.region_waitText('Untitled', 5) 118 | lib.region_doubleClick(useLastMatch=True) 119 | 120 | print('=======Step: right click on iNotepad - will fail if SKIP is not used next') 121 | lib.region_setFindFailedResponse('SKIP') 122 | print('FindFailed (SKIP): ', lib.region_getFindFailedResponse()) 123 | lib.region_rightClick('iNotepad', 48, 14) 124 | 125 | lib.region_setRect(1, 2, 300, 200) 126 | ret = lib.region_find('iNotepad', onScreen=False) 127 | print('Find on screen should fail: ', ret) 128 | 129 | print ('=======Step: use correct image for right click') 130 | lib.region_setFindFailedResponse('ABORT') 131 | print('FindFailed (ABORT): ', lib.region_getFindFailedResponse()) 132 | 133 | pre = lib.settings_set('MinSimilarity', float(0.7)) 134 | print('Settings MinSimilarity (0.9 -> 0.7): %s -> %s' % (pre, lib.settings_get('MinSimilarity'))) 135 | 136 | res = lib.region_has('iNotepad typed=0.99') 137 | print('Has with 0.99 similiarity: (None)', res) 138 | 139 | lib.region_rightClick('iNotepad typed', 48, 14) 140 | 141 | # coordinates relative to upper left corner of the image 142 | lib.region_click('iNotepad menu', 54, 80) 143 | lib.app_focus('Notepad') 144 | 145 | lib.region_click('iNotepad typed', 50, 12) 146 | # use previous found region 147 | lib.region_rightClick(None, 0, 0, True) 148 | 149 | print('=======Step: click with coordinates relative to center') 150 | # coordinates relative to center of the image 151 | lib.set_offsetCenterMode(True) 152 | lib.region_click('iNotepad menu', -10, 10) 153 | lib.app_focus('Notepad') 154 | 155 | print('=======Step: dragDrop Notepad') 156 | lib.set_offsetCenterMode(False) 157 | prev = lib.settings_set('DelayBeforeDrop', float(2.0)) 158 | lib.region_dragDrop('iNotepad typed', 'iNotepad typed', 50, 12, 100, 12) 159 | 160 | print('=======Step: delete everything') 161 | lib.region_type(text='A', modifier='SikuliXJClass.Key.CTRL') 162 | lib.region_type('SikuliXJClass.Key.DELETE') 163 | 164 | print('=======Step: hover, click, double click') 165 | lib.region_hover('iNotepad', dx=10, dy=10) 166 | lib.region_mouseMove(100, 200) 167 | lib.region_doubleClick('iNotepad', 100, 300) 168 | 169 | exit_here() 170 | --------------------------------------------------------------------------------