├── .gitignore ├── CONTRIBUTING.md ├── LICENSE.txt ├── README.md ├── examples ├── README.md ├── helloworld.py ├── run_voter.sh └── voter.py └── python_client_pycharm_project.jpg /.gitignore: -------------------------------------------------------------------------------- 1 | # Global directories 2 | __pycache__/ 3 | 4 | # Global files 5 | *.py[cod] 6 | *.dll 7 | *.so 8 | *.log 9 | *.swp 10 | 11 | # Root directories 12 | /.cache/ 13 | /.idea/ 14 | /.pytest_cache/ 15 | /.vscode/ 16 | 17 | .DS_Store 18 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Thank you for your interest in contributing. VoltDB uses GitHub to manage reviews of pull requests. We welcome your contributions to VoltDB or to any of the related libraries and tools. 4 | 5 | VoltDB uses the standard github workflow, meaning we make use of forking and pull requests. 6 | 7 | * If you have a trivial fix or improvement, go ahead and create a pull request. 8 | 9 | * If you are interested in contributing something more involved, feel free to discuss your ideas in [VoltDB Public](http://chat.voltdb.com/) on Slack. 10 | 11 | ## Contributor License Agreement 12 | 13 | In order to contribute code to VoltDB, you must first sign the [VoltDB Contributor License Agreement (CLA)](https://www.voltdb.com/contributor-license-agreement/) and email an image or PDF of the document including your hand-written signature to [support@voltdb.com](mailto:support@voltdb.com). VoltDB will sign and return the final copy of the agreement for your records. 14 | 15 | ## How to submit code 16 | 17 | The workflow is essentially the following: 18 | 19 | 1. Fork the VoltDB project 20 | 2. Make a branch. Commit your changes to this branch. (See note below) 21 | 3. Issue a pull request on the VoltDB repository. 22 | 23 | Once you have signed the CLA, a VoltDB engineer will review your pull request. 24 | 25 | Note: 26 | 27 | It will be easier to keep your work merge-able (conflict-free) if you don't work directly on your master branch in your VoltDB fork. Rather, keep your master branch in sync with the VoltDB repository and apply your changes on a branch of your fork. 28 | 29 | For further reading: 30 | 31 | * [How to fork a GitHub repository](https://help.github.com/articles/fork-a-repo) 32 | * [Using pull requests](https://help.github.com/articles/using-pull-requests/) 33 | 34 | ## Additional Resources 35 | 36 | * [VoltDB Wiki](https://github.com/VoltDB/voltdb/wiki) on Github 37 | * [VoltDB Public](http://chat.voltdb.com/) on Slack 38 | * [VoltDB on Stack Overflow](https://stackoverflow.com/questions/tagged/voltdb) 39 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (C) 2008-2021 VoltDB Inc. 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining 4 | a copy of this software and associated documentation files (the 5 | "Software"), to deal in the Software without restriction, including 6 | without limitation the rights to use, copy, modify, merge, publish, 7 | distribute, sublicense, and/or sell copies of the Software, and to 8 | permit persons to whom the Software is furnished to do so, subject to 9 | the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be 12 | included in all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 17 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 18 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 19 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 20 | OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Description 2 | 3 | The VoltDB Python client library is a native Python implementation of the VoltDB 4 | wire protocol. It provides the functionality of connecting to a VoltDB server, 5 | authenticating the client, and making procedure calls. The client supports 6 | Python 3.6 or later. 7 | 8 | The Python client library is a single file, voltdbclient.py, contained in the VoltDB distribution 9 | tarball. This pypi package is provided for the convenience of users who don't need the 10 | full VoltDB distribution. 11 | 12 | ------- 13 | 14 | The core of the Python client library is the FastSerializer class. It manages 15 | the connection to the server and serializes/deserializes the VoltDB primitive 16 | types. There are higher level classes which wraps around the FastSerializer 17 | class to handle compound objects, namely procedure, response, table, and 18 | exception. VoltProcedure, VoltResponse, VoltTable, and VoltException classes 19 | handles them, respectively. Note that none of the classes in the Python client 20 | library is thread safe. 21 | 22 | Each VoltDB primitive type is mapped to a Python primitive type. The following 23 | table shows the mapping. 24 | 25 | |VoltDB |Python | 26 | |---------|--------| 27 | |NULL |None | 28 | |TINYINT |integer | 29 | |SMALLINT |integer | 30 | |INTEGER |integer | 31 | |BIGINT |integer | 32 | |FLOAT |float | 33 | |STRING |string | 34 | |DECIMAL |decimal | 35 | 36 | Although Python does not distinguish between the different integer types, the 37 | size of the integer being serialized needs to fit the corresponding type. 38 | 39 | The common work flow is to create a FastSerializer object and connect to the 40 | server. Then create a VoltProcedure object for each stored procedure you want to 41 | call, passing the FastSerializer object to it. For each call, you will get a 42 | VoltResponse object, which may or may not contain a list of VoltTable 43 | objects. In case of failure, a VoltException object may be included. 44 | 45 | 46 | API 47 | 48 | FastSerializer.VOLTTYPE_NULL 49 | FastSerializer.VOLTTYPE_TINYINT 50 | FastSerializer.VOLTTYPE_SMALLINT 51 | FastSerializer.VOLTTYPE_INTEGER 52 | FastSerializer.VOLTTYPE_BIGINT 53 | FastSerializer.VOLTTYPE_FLOAT 54 | FastSerializer.VOLTTYPE_STRING 55 | FastSerializer.VOLTTYPE_TIMESTAMP 56 | FastSerializer.VOLTTYPE_DECIMAL 57 | FastSerializer.VOLTTYPE_DECIMAL_STRING 58 | FastSerializer.VOLTTYPE_VOLTTABLE 59 | VoltDB types. 60 | 61 | FastSerializer(host, port, username, password, dump_file) 62 | Create a connection to host (string) on port (integer). If username (string) 63 | and password (string) is given, authenticate the client using them. If 64 | dump_file (string) is given, all the data received from and sent to the 65 | server will be written into the file pointed to by dump_file. 66 | 67 | FastSerializer.close() 68 | Closes the connection. No further use of the object is valid. 69 | 70 | VoltProcedure(fser, name, paramtypes) 71 | Create a procedure object which can be used to call the stored procedure 72 | name (string) with parameters of types paramtypes (list). The parameter 73 | types are defined in the FastSerializer class as VoltDB types. For parameter 74 | of type array, you specify the type in the same way as primitive types. fser 75 | is the FastSerializer object with a valid connection to the server. 76 | 77 | VoltProcedure.call(params, response, timeout) 78 | Make a stored procedure invocation with the parameters params (list). The 79 | parameters has to match the types defined in the VoltProcedure 80 | constructor. If a parameter is an array, pass it in as a list. If response 81 | (bool) is given and False, the invocation will return None. If timeout 82 | (float) is given, the invocation will wait for timeout seconds at most if 83 | the server does not respond. A socket.timeout exception will be raised if 84 | timeout seconds has elapsed. A successful invocation will return a 85 | VoltReponse object. 86 | 87 | VoltResponse.status 88 | The status code (integer) for a stored procedure invocation. For a list of 89 | status code, please refer to the VoltDB documentation. 90 | 91 | VoltResponse.statusString 92 | A human-friendly string of the meaning of the status code. 93 | 94 | VoltResponse.roundtripTime 95 | The round-trip time (integer) of the invocation in milliseconds. 96 | 97 | VoltResponse.exception 98 | The VoltException object in case of failure, may be None. 99 | 100 | VoltResponse.tables 101 | A list of VoltTable objects as the result of the invocation. May be None. 102 | 103 | VoltException.type 104 | The type of the VoltDB exception. Can be the following values, 105 | VOLTEXCEPTION_NONE, VOLTEXCEPTION_EEEXCEPTION, VOLTEXCEPTION_SQLEXCEPTION, 106 | VOLTEXCEPTION_CONSTRAINTFAILURE, VOLTEXCEPTION_GENERIC. 107 | 108 | VoltException.message 109 | A string explaining the exception. 110 | 111 | VoltTable.columns 112 | A list of VoltColumn objects, representing the columns in the table. 113 | 114 | VoltTable.tuples 115 | A list of rows in the table. A row a list of values deserialized in Python 116 | types. 117 | 118 | VoltColumn.type 119 | The type of the column. A list of types is defined in the FastSerializer 120 | class. 121 | 122 | VoltColumn.name 123 | The name of the column as a string. 124 | 125 | 126 | Example 127 | 128 | The following example shows how to make a connection to a VoltDB server instance 129 | running on port 21212 on localhost, and make a single call to the stored 130 | procedure named "Select", which takes a single parameter of type string. Assume 131 | the server is not secure, so that it does not require authentication. 132 | 133 | >>> from voltdbclient import * 134 | >>> client = FastSerializer("localhost", 21212) 135 | >>> proc = VoltProcedure(client, "@SystemInformation", [FastSerializer.VOLTTYPE_STRING]) 136 | >>> response = proc.call(["OVERVIEW"]) 137 | >>> client.close() 138 | >>> print (response) 139 | 140 | The output will be a human-readable form of the VoltResponse object. 141 | 142 | ## Installation 143 | 144 | You can install the package via **PyPI** or from **source**. 145 | 146 | ### Install from PyPI system-wide 147 | 148 | ```bash 149 | pip install voltdbclient 150 | ``` 151 | 152 | ### Install in a Virtual Environment 153 | 154 | It is recommended to use isolated self-contained virtual env per individual Python projects. 155 | Below example show how to set up such an env using the _**venv**_ library that is included in 156 | python3 distribution. 157 | 158 | Pre-requisites and naming conventions: 159 | * you have python3 installed 160 | * to run provided examples - you have the 'voldb-client-python' project downloaded locally 161 | * CLIENT_HOME refers to the path of your local 'voldb-client-python' project 162 | * 'voltdb_venv_p3' is the name of your venv environment - can be anything 163 | 164 | Commands below will create a new virtual env 'voltdb_venv_p3', 165 | which will result in a subdirectory of that name created in the CLIENT_HOME dir, activate the env, 166 | and install the voltdbclient in that venv 167 | 168 | ```bash 169 | cd CLIENT_HOME 170 | python3 -m venv voltdb_venv_p3 171 | source voltdb_venv_p3/bin/activate 172 | pip install voltdbclient 173 | ``` 174 | 175 | Once the ven is created and the client is installed - you can create a new project in a PyCharm 176 | (or your IDE of choice) and use the new venv as the Python Interpreter for the project. 177 | Here is an example of what this might look like in PyCharm: 178 | 179 | ![python_client_pycharm_project](python_client_pycharm_project.jpg) -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | The Hello World and Voter examples included here are intended to 2 | be used with example projects shipped with the main distribution. 3 | You will need to have a server running with the appropriate catalog 4 | or ddl loaded for these examples to work. 5 | 6 | You will need to copy voltdbclient.py to the examples directory for 7 | the examples to run. 8 | 9 | The examples require that the command 'python3' points to Python 10 | 3.6 or later. We use 'python3' rather than the bare 'python' 11 | command to ensure we use the appropriate version. The VoltDB 12 | client library no longer supports Python version 2. 13 | 14 | # Hello World 15 | 16 | First, follow the instructions in the VoltDB kit's doc/tutorials/helloworld 17 | folder to start the database and load the schema. 18 | 19 | Then, after copying the voltdbclient.py to the examples directory, run the 20 | following command to start the python helloworld.py client. This requires 21 | no arguments and connects to localhost. 22 | 23 | ./helloworld.py 24 | 25 | # Voter 26 | 27 | First, follow the instructions in the VoltDB kit's examples/voter 28 | folder to start the database and load the schema. 29 | 30 | Then, after copying the voltdbclient.py to the examples directory, run the 31 | ./voter.py command with arguments to start the python voter.py client. 32 | 33 | The voter.py client has seven arguments: 34 | [number of contestants] 35 | [votes per phone number] 36 | [transactions per second] 37 | [client feedback interval (seconds)] 38 | [test duration (seconds)] 39 | [lag record delay (seconds)] 40 | [server list (comma separated)] 41 | 42 | A reasonable default invocation that connects to localhost is: 43 | 44 | ./voter.py 6 2 100000 5 120 3 localhost 45 | -------------------------------------------------------------------------------- /examples/helloworld.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # This file is part of VoltDB. 4 | # Copyright (C) 2008-2021 VoltDB Inc. 5 | # 6 | # Permission is hereby granted, free of charge, to any person obtaining 7 | # a copy of this software and associated documentation files (the 8 | # "Software"), to deal in the Software without restriction, including 9 | # without limitation the rights to use, copy, modify, merge, publish, 10 | # distribute, sublicense, and/or sell copies of the Software, and to 11 | # permit persons to whom the Software is furnished to do so, subject to 12 | # the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be 15 | # included in all copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 18 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 19 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 20 | # IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 21 | # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 22 | # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 23 | # OTHER DEALINGS IN THE SOFTWARE. 24 | 25 | from voltdbclient import * 26 | 27 | client = FastSerializer("localhost", 21212) 28 | proc = VoltProcedure( client, "Insert", [FastSerializer.VOLTTYPE_STRING, FastSerializer.VOLTTYPE_STRING, FastSerializer.VOLTTYPE_STRING] ) 29 | 30 | proc.call([ "English", "Hello", "World" ]) 31 | proc.call([ "French" , "Bonjour", "Monde" ]) 32 | proc.call([ "Spanish", "Hola", "Mundo" ]) 33 | proc.call([ "Danish", "Hej", "Verden" ]) 34 | proc.call([ "Italian", "Ciao", "Mondo" ]) 35 | 36 | proc = VoltProcedure( client, "Select", [FastSerializer.VOLTTYPE_STRING]) 37 | response = proc.call( ["Spanish"] ) 38 | 39 | for x in response.tables: 40 | print(x) 41 | -------------------------------------------------------------------------------- /examples/run_voter.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | python3 voter.py 6 2 100000 5 120 3 localhost 3 | -------------------------------------------------------------------------------- /examples/voter.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # This file is part of VoltDB. 4 | # Copyright (C) 2008-2021 VoltDB Inc. 5 | # 6 | # Permission is hereby granted, free of charge, to any person obtaining 7 | # a copy of this software and associated documentation files (the 8 | # "Software"), to deal in the Software without restriction, including 9 | # without limitation the rights to use, copy, modify, merge, publish, 10 | # distribute, sublicense, and/or sell copies of the Software, and to 11 | # permit persons to whom the Software is furnished to do so, subject to 12 | # the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be 15 | # included in all copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 18 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 19 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 20 | # IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 21 | # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 22 | # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 23 | # OTHER DEALINGS IN THE SOFTWARE. 24 | 25 | import sys 26 | import time 27 | import random 28 | import threading 29 | from voltdbclient import * 30 | 31 | # main method of voter client 32 | def main(): 33 | # prints required command line arguments if these were not passed in correctly 34 | if len(sys.argv) < 8: 35 | print("ClientVoter [number of contestants] [votes per phone number] [transactions per second] [client feedback interval (seconds)] [test duration (seconds)] [lag record delay (seconds)] [server list (comma separated)]") 36 | exit(1) 37 | 38 | # checks for validity of 1st command line argument 39 | # NOTE: 0th command line argument is the file name of this python program 40 | global max_contestant 41 | max_contestant = int(sys.argv[1]) 42 | if max_contestant < 1 or max_contestant > 12: 43 | print("Number of contestants must be between 1 and 12") 44 | exit(1) 45 | 46 | # sets up global variables, including: 47 | global results_lock # a lock for manipulating result data 48 | global params_lock # a lock for manipulating the parameter list 49 | global requestPermit # control rate at which requests are generated 50 | global availableThreads # Allow a thread to indicate it is available to service a request 51 | global shouldContinue 52 | global min_execution_secs # minimum number of seconds used to execute stored procedure 53 | global max_execution_secs # maximum number of seconds used to execute stored procedure 54 | global tot_execution_secs # total number of seconds used to execute stored procedures 55 | global tot_executions # total number of executed stored procedures 56 | global tot_executions_latency # total number of executed stored procedures as a measure for latency 57 | global check_latency # boolean value: if false, latency is not yet being measures; if true, latency is being measured 58 | global latency_counter # array used to show how many stored procedures fell into various time ranges for latency (e.g.: 200 stored procedures had latency in range between 25 and 50 milliseconds 59 | global vote_result_counter # array used to show how many votes got (0) Accepted, (1) Rejected due to invalid contestant, (2) Rejected due to voter being over the limit (of phone calls) 60 | global invocation_params 61 | results_lock = threading.Lock() 62 | params_lock = threading.Lock() 63 | requestPermit = threading.Semaphore(0) 64 | availableThreads = threading.Semaphore(0) 65 | invocation_params = [] 66 | shouldContinue = True 67 | min_execution_secs = 999999 68 | max_execution_secs = -1 69 | tot_execution_secs = 0 70 | tot_executions = 0 71 | tot_executions_latency = 0 72 | check_latency = False 73 | latency_counter = [0, 0, 0, 0, 0, 0, 0, 0, 0] 74 | vote_result_counter = [0, 0, 0] 75 | 76 | # assigns values to other variables using command line arguments and creativity 77 | max_votes_per_phone_number = int(sys.argv[2]) 78 | transactions_per_sec = int(sys.argv[3]) 79 | transactions_per_milli = transactions_per_sec / float(1000) # uses millis, not secs 80 | client_feedback_interval_secs = int(sys.argv[4]) 81 | test_duration_secs = int(sys.argv[5]) 82 | lag_latency_secs = int(sys.argv[6]) 83 | server_list = sys.argv[7] 84 | this_outstanding = 0 85 | last_outstanding = 0 86 | contestant_names = "Edwina Burnam,Tabatha Gehling,Kelly Clauss,Jessie Alloway,Alana Bregman,Jessie Eichman,Allie Rogalski,Nita Coster,Kurt Walser,Ericka Dieter,Loraine Nygren,Tania Mattioli" 87 | 88 | print(("Allowing %d votes per phone number" % max_votes_per_phone_number)) 89 | print(("Submitting %d SP calls/sec" % transactions_per_sec)) 90 | print(("Feedback interval = %d second(s)" % client_feedback_interval_secs)) 91 | print(("Running for %d second(s)" % test_duration_secs)) 92 | print(("Latency not recorded for %d second(s)" % lag_latency_secs)) 93 | 94 | phone_number = None 95 | contestant_number = None 96 | transactions_this_sec = 0 97 | last_milli = time.time() * 1000 # uses millis, not secs 98 | this_milli = time.time() * 1000 # uses millis, not secs 99 | 100 | # parses the list of servers specified at command line and creates corresponding URL for each, adding these to the dictionary 101 | volt_servers = server_list.rsplit(",") 102 | 103 | # invokes the stored procedure 'Initialize' to set up database with contestant names/numbers 104 | # uses quick parse hack to process the response of the invocation 105 | # contestant names/numbers entered into database if this is the first client to connect; otherwise, existing configuration info retrieved 106 | client = FastSerializer(volt_servers[0]) 107 | initprocedure = VoltProcedure( client, "Initialize", [ FastSerializer.VOLTTYPE_INTEGER, FastSerializer.VOLTTYPE_STRING ]) 108 | 109 | response = initprocedure.call( [max_contestant, contestant_names ] ) 110 | 111 | # sets up start and end times of the voting process (and of latency measurements) based on command line-specified duration and delay values 112 | start_time = time.time() 113 | end_time = start_time + test_duration_secs 114 | current_time = start_time 115 | last_feedback_time = start_time 116 | num_sp_calls = 0 117 | start_recording_latency = start_time + lag_latency_secs 118 | 119 | thread_list = [] 120 | 121 | for x in range(5): 122 | thread = doQueries(volt_servers[x % len(volt_servers)]) 123 | thread.setDaemon(True) 124 | thread.start() 125 | 126 | # main while loop of voter client, used to invoke stored procedure 'Vote' repeatedly 127 | while end_time > current_time: 128 | availableThreads.acquire() 129 | num_sp_calls = num_sp_calls + 1 130 | 131 | # generates random 10-digit 'phone number' and not entirely random contestant number 132 | # the contestant number (as generated below) is most likely to be 2 133 | # NOTE: every 100 votes, the contestant number is made to be potentially invalid 134 | phone_number = random.randint(1000000000, 9999999999) 135 | contestant_number = (int(random.random() * max_contestant) * int(random.random() * max_contestant)) % max_contestant + 1 136 | if num_sp_calls % 100 == 0: 137 | contestant_number = (int(random.random() * max_contestant) + 1) * 2 138 | 139 | params_lock.acquire() 140 | invocation_params.append([ phone_number, contestant_number ]) 141 | params_lock.release() 142 | requestPermit.release() 143 | 144 | # if more votes per second are happening than the command line argument allows: waits until enough time has passed to resume voting 145 | # this block uses millis, not secs 146 | transactions_this_sec = transactions_this_sec + 1 147 | if transactions_this_sec >= transactions_per_milli: 148 | this_milli = time.time() * 1000 149 | while this_milli <= last_milli: 150 | this_milli = time.time() * 1000 151 | time.sleep(0) #yield to other threads 152 | last_milli = this_milli 153 | transactions_this_sec = 0 154 | 155 | current_time = time.time() 156 | 157 | if not check_latency and current_time >= start_recording_latency: 158 | check_latency = True 159 | 160 | # if enough time has passed since last status report: reports current voting status (prints some data to console) 161 | if current_time >= (last_feedback_time + client_feedback_interval_secs): 162 | elapsed_time_secs_2 = time.time() - start_time 163 | last_feedback_time = current_time 164 | run_time_secs = end_time - start_time 165 | if tot_executions_latency == 0: 166 | tot_executions_latency = 1 167 | 168 | percent_complete = (float(elapsed_time_secs_2) / float(run_time_secs)) * 100 169 | if percent_complete > 100: 170 | percent_complete = 100 171 | 172 | # lock necessary, because global variables manipulated in this block may also be used by other threads (those responsible for invoking stored procedure 'Vote') 173 | # execution times are printed in millis, not secs 174 | results_lock.acquire() 175 | this_outstanding = num_sp_calls - tot_executions 176 | avg_latency = float(tot_execution_secs) * 1000 / float(tot_executions_latency) 177 | print(("%f%% Complete | SP Calls: %d at %f SP/sec | outstanding = %d (%d) | min = %d | max = %d | avg = %f" % (percent_complete, num_sp_calls, (float(num_sp_calls) / float(elapsed_time_secs_2)), this_outstanding, (this_outstanding - last_outstanding), (min_execution_secs * 1000), (max_execution_secs * 1000), avg_latency))) 178 | last_outstanding = this_outstanding 179 | results_lock.release() 180 | shouldContinue = False 181 | # joins outstanding threads (those responsible for invoking stored procedure 'Vote') 182 | for thread in thread_list: 183 | if thread.isAlive(): 184 | thread.join() 185 | 186 | elapsed_time_secs = time.time() - start_time 187 | 188 | # prints statistics about the numbers of accepted/rejected votes 189 | print() 190 | print("****************************************************************************") 191 | print("Voting Results") 192 | print("****************************************************************************") 193 | print((" - Accepted votes = %d" % vote_result_counter[0])) 194 | print((" - Rejected votes (invalid contestant) = %d" % vote_result_counter[1])) 195 | print((" - Rejected votes (voter over limit) = %d" % vote_result_counter[2])) 196 | print() 197 | 198 | winner_name = "<>" 199 | winner_votes = -1 200 | 201 | # invokes the stored procedure 'Results' to retrieve all stored tuples in database 202 | # uses quick parse hack to process the response of the invocation 203 | # analyzes the processed data to determine number of votes per contestant, winner, and number of votes for winner 204 | resultsprocedure = VoltProcedure( client, "Results", []) 205 | response = resultsprocedure.call([]) 206 | table = response.tables[0] 207 | if len(table.tuples) == 0: 208 | print(" - No results to report.") 209 | else: 210 | for row in table.tuples: 211 | result_name = row[0] 212 | result_votes = row[2] 213 | print((" - Contestant %s received %d vote(s)" % (result_name, result_votes))) 214 | 215 | if result_votes > winner_votes: 216 | winner_votes = result_votes 217 | winner_name = result_name 218 | 219 | # prints winner data 220 | # prints statistics about average latency and distribution of stored procedures across ranges in latency 221 | print() 222 | print((" - Contestant %s was the winner with %d vote(s)" % (winner_name, winner_votes))) 223 | print() 224 | print("****************************************************************************") 225 | print("System Statistics") 226 | print("****************************************************************************") 227 | print((" - Ran for %f second(s)" % elapsed_time_secs)) 228 | print((" - Performed %d Stored Procedure call(s)" % num_sp_calls)) 229 | print((" - At %f call(s) per second" % (num_sp_calls / elapsed_time_secs))) 230 | print((" - Average Latency = %f ms" % (float(tot_execution_secs) * 1000 / float(tot_executions_latency)))) 231 | print((" - Latency 0ms - 25ms = %d" % latency_counter[0])) 232 | print((" - Latency 25ms - 50ms = %d" % latency_counter[1])) 233 | print((" - Latency 50ms - 75ms = %d" % latency_counter[2])) 234 | print((" - Latency 75ms - 100ms = %d" % latency_counter[3])) 235 | print((" - Latency 100ms - 125ms = %d" % latency_counter[4])) 236 | print((" - Latency 125ms - 150ms = %d" % latency_counter[5])) 237 | print((" - Latency 150ms - 175ms = %d" % latency_counter[6])) 238 | print((" - Latency 175ms - 200ms = %d" % latency_counter[7])) 239 | print((" - Latency 200ms+ = %d" % latency_counter[8])) 240 | 241 | # class, whose objects run in separate threads 242 | # responsible for invoking stored procedure 'Vote' and processing results (updating statistics) 243 | class doQueries(threading.Thread): 244 | # accepts url used to connect to server as parameter 245 | # accepts data used to invoke stored procedure 'Vote' as parameter 246 | def __init__ (self, server): 247 | threading.Thread.__init__(self) 248 | self.client = FastSerializer( server ) 249 | self.proc = VoltProcedure( self.client, "Vote", [ FastSerializer.VOLTTYPE_BIGINT, FastSerializer.VOLTTYPE_TINYINT, FastSerializer.VOLTTYPE_BIGINT ]) 250 | 251 | # the method that gets called when this new thread is started 252 | def run(self): 253 | global vote_result_counter 254 | global tot_executions 255 | global tot_executions_latency 256 | global tot_execution_secs 257 | global check_latency 258 | global latency_counter 259 | global min_execution_secs 260 | global max_execution_secs 261 | global requestPermit 262 | global shouldContinue 263 | global max_contestant 264 | global results_lock 265 | global params_lock 266 | global availableThreads 267 | global invocation_params 268 | num_sp_calls = 0 269 | while shouldContinue: 270 | #coordinate so that threads can always block on either semaphores or a request, indicate availability and then wait for a permit 271 | #to generate a request 272 | availableThreads.release(); 273 | requestPermit.acquire() 274 | num_sp_calls = num_sp_calls + 1 275 | 276 | params_lock.acquire(); 277 | params = invocation_params.pop() 278 | params_lock.release() 279 | 280 | # invokes the stored procedure 'Vote' to cast a vote for some contestant by voter with some phone number 281 | # times the invocation 282 | # uses quick parse hack to process the response of the invocation 283 | # analyzes the processed data to determine if invocation was successful (if not, exits) 284 | time_before = time.time() 285 | response = self.proc.call( [params[0], params[1], max_contestant] ) 286 | time_after = time.time() 287 | if response.status != 1: 288 | print("Failed to execute!!!") 289 | exit(-1) 290 | else: 291 | # updates statistics about execution count, execution times, vote successes/failures, and latency 292 | # lock necessary, because any other thread may need to access the same global variables, which could result in inconsistencies without proper thread safety 293 | results_lock.acquire() 294 | tot_executions = tot_executions + 1 295 | table = response.tables[0] 296 | vote_result = table.tuples[0][0] 297 | vote_result_counter[vote_result] = vote_result_counter[vote_result] + 1 298 | if check_latency: 299 | execution_time = time_after - time_before 300 | tot_executions_latency = tot_executions_latency + 1 301 | tot_execution_secs = tot_execution_secs + execution_time 302 | 303 | if execution_time < min_execution_secs: 304 | min_execution_secs = execution_time 305 | if execution_time > max_execution_secs: 306 | max_execution_secs = execution_time 307 | 308 | latency_bucket = int(execution_time * 40) 309 | if latency_bucket > 8: 310 | latency_bucket = 8 311 | latency_counter[latency_bucket] = latency_counter[latency_bucket] + 1 312 | results_lock.release() 313 | 314 | # used to call main method of voter client 315 | if __name__ == "__main__": 316 | main() 317 | -------------------------------------------------------------------------------- /python_client_pycharm_project.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VoltDB/voltdb-client-python/b71bb70f29044d2b1bf0999430c94329f49f7077/python_client_pycharm_project.jpg --------------------------------------------------------------------------------