├── __init__.py ├── setup.py ├── install.sh ├── LICENSE ├── .gitignore ├── rover.py ├── UDPComms.py └── README.md /__init__.py: -------------------------------------------------------------------------------- 1 | from .UDPComms import Publisher 2 | from .UDPComms import Subscriber 3 | from .UDPComms import timeout 4 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from distutils.core import setup 4 | 5 | setup(name='UDPComms', 6 | version='1.1dev', 7 | py_modules=['UDPComms'], 8 | description='Simple library for sending messages over UDP', 9 | author='Michal Adamkiewicz', 10 | author_email='mikadam@stanford.edu', 11 | url='https://github.com/stanfordroboticsclub/UDP-Comms', 12 | ) 13 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | #Uses python magic vs realpath so it also works on a mac 4 | FOLDER=$(python -c "import os; print(os.path.dirname(os.path.realpath('$0')))") 5 | cd $FOLDER 6 | 7 | yes | sudo pip3 install msgpack 8 | yes | sudo pip install msgpack 9 | 10 | #The `clean --all` removes the build directory automatically which makes reinstalling new versions possible with the same command. 11 | python3 setup.py clean --all install 12 | 13 | # used for rover command 14 | yes | sudo pip3 install pexpect 15 | 16 | # Install rover command 17 | sudo ln -s $FOLDER/rover.py /usr/local/bin/rover 18 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Michal Adamkiewicz 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Mac folder files 2 | .DS_Store 3 | 4 | # Byte-compiled / optimized / DLL files 5 | __pycache__/ 6 | *.py[cod] 7 | *$py.class 8 | 9 | # C extensions 10 | *.so 11 | 12 | # Distribution / packaging 13 | .Python 14 | build/ 15 | develop-eggs/ 16 | dist/ 17 | downloads/ 18 | eggs/ 19 | .eggs/ 20 | lib/ 21 | lib64/ 22 | parts/ 23 | sdist/ 24 | var/ 25 | wheels/ 26 | *.egg-info/ 27 | .installed.cfg 28 | *.egg 29 | MANIFEST 30 | 31 | # PyInstaller 32 | # Usually these files are written by a python script from a template 33 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 34 | *.manifest 35 | *.spec 36 | 37 | # Installer logs 38 | pip-log.txt 39 | pip-delete-this-directory.txt 40 | 41 | # Unit test / coverage reports 42 | htmlcov/ 43 | .tox/ 44 | .nox/ 45 | .coverage 46 | .coverage.* 47 | .cache 48 | nosetests.xml 49 | coverage.xml 50 | *.cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | 63 | # Flask stuff: 64 | instance/ 65 | .webassets-cache 66 | 67 | # Scrapy stuff: 68 | .scrapy 69 | 70 | # Sphinx documentation 71 | docs/_build/ 72 | 73 | # PyBuilder 74 | target/ 75 | 76 | # Jupyter Notebook 77 | .ipynb_checkpoints 78 | 79 | # IPython 80 | profile_default/ 81 | ipython_config.py 82 | 83 | # pyenv 84 | .python-version 85 | 86 | # celery beat schedule file 87 | celerybeat-schedule 88 | 89 | # SageMath parsed files 90 | *.sage.py 91 | 92 | # Environments 93 | .env 94 | .venv 95 | env/ 96 | venv/ 97 | ENV/ 98 | env.bak/ 99 | venv.bak/ 100 | 101 | # Spyder project settings 102 | .spyderproject 103 | .spyproject 104 | 105 | # Rope project settings 106 | .ropeproject 107 | 108 | # mkdocs documentation 109 | /site 110 | 111 | # mypy 112 | .mypy_cache/ 113 | .dmypy.json 114 | dmypy.json 115 | -------------------------------------------------------------------------------- /rover.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import sys 4 | import argparse 5 | import json 6 | import time 7 | import select 8 | import pexpect 9 | 10 | import UDPComms 11 | import msgpack 12 | 13 | def peek_func(port): 14 | sub = UDPComms.Subscriber(port, timeout = 10) 15 | while 1: 16 | try: 17 | data = sub.recv() 18 | print( json.dumps(data) ) 19 | except UDPComms.timeout: 20 | exit() 21 | 22 | def poke_func(port, rate): 23 | pub = UDPComms.Publisher(port) 24 | data = None 25 | 26 | while 1: 27 | if select.select([sys.stdin], [], [], 0)[0]: 28 | line = sys.stdin.readline() 29 | # detailed behaviour 30 | # reading from file: -ignores empty lines -repeats last line forever 31 | # reading from terminal: -repeats last command 32 | if line.rstrip(): 33 | data = line.rstrip() 34 | elif len(line) == 0: 35 | # exit() #uncomment to quit on end of file 36 | pass 37 | else: 38 | continue 39 | 40 | if data != None: 41 | pub.send( json.loads(data) ) 42 | time.sleep( rate/1000 ) 43 | 44 | def call_func(command, ssh = True): 45 | child = pexpect.spawn(command) 46 | 47 | if ssh: 48 | i = 1 49 | while i == 1: 50 | try: 51 | i = child.expect(['password:', 52 | 'Are you sure you want to continue connecting', 53 | 'Welcome'], timeout=20) 54 | except pexpect.EOF: 55 | print("Can't connect to device") 56 | exit() 57 | except pexpect.TIMEOUT: 58 | print("Interaction with device failed") 59 | exit() 60 | 61 | if i == 1: 62 | child.sendline('yes') 63 | if i == 0: 64 | child.sendline('raspberry') 65 | else: 66 | try: 67 | child.expect('robot:', timeout=1) 68 | child.sendline('hello') 69 | except pexpect.TIMEOUT: 70 | pass 71 | 72 | child.interact() 73 | 74 | if __name__ == '__main__': 75 | parser = argparse.ArgumentParser() 76 | subparsers = parser.add_subparsers(dest='subparser') 77 | 78 | peek = subparsers.add_parser("peek") 79 | peek.add_argument('port', help="UDP port to subscribe to", type=int) 80 | 81 | poke = subparsers.add_parser("poke") 82 | poke.add_argument('port', help="UDP port to publish the data to", type=int) 83 | poke.add_argument('rate', help="how often to republish (ms)", type=float) 84 | 85 | peek = subparsers.add_parser("discover") 86 | 87 | commands = ['status', 'log', 'start', 'stop', 'restart', 'enable', 'disable'] 88 | for command in commands: 89 | status = subparsers.add_parser(command) 90 | status.add_argument('host', help="Which device to look for this program on") 91 | status.add_argument('unit', help="The unit whose status we want to know", 92 | nargs='?', default=None) 93 | 94 | connect = subparsers.add_parser('connect') 95 | connect.add_argument('host', help="Which device to log into") 96 | 97 | args = parser.parse_args() 98 | 99 | if args.subparser == 'peek': 100 | peek_func(args.port) 101 | elif args.subparser == 'poke': 102 | poke_func(args.port, args.rate) 103 | elif args.subparser == 'connect': 104 | call_func("ssh pi@"+args.host+".local") 105 | elif args.subparser == 'discover': 106 | call_func("nmap -sP 10.0.0.0/24", ssh=False) 107 | 108 | elif args.subparser in commands: 109 | if args.unit is None: 110 | args.unit = args.host 111 | 112 | if args.host == 'local': 113 | prefix = "" 114 | ssh = False 115 | else: 116 | prefix = "ssh pi@"+args.host+".local " 117 | ssh = True 118 | 119 | if args.subparser == 'status': 120 | call_func(prefix + "sudo systemctl status "+args.unit, ssh) 121 | elif args.subparser == 'log': 122 | call_func(prefix + "sudo journalctl -f -u "+args.unit, ssh) 123 | elif args.subparser == 'start': 124 | call_func(prefix + "sudo systemctl start "+args.unit, ssh) 125 | elif args.subparser == 'stop': 126 | call_func(prefix + "sudo systemctl stop "+args.unit, ssh) 127 | elif args.subparser == 'restart': 128 | call_func(prefix + "sudo systemctl restart "+args.unit, ssh) 129 | elif args.subparser == 'enable': 130 | call_func(prefix + "sudo systemctl enable "+args.unit, ssh) 131 | elif args.subparser == 'disable': 132 | call_func(prefix + "sudo systemctl disable "+args.unit, ssh) 133 | else: 134 | parser.print_help() 135 | 136 | -------------------------------------------------------------------------------- /UDPComms.py: -------------------------------------------------------------------------------- 1 | 2 | """ 3 | This is a simple library to enable communication between different processes (potentially on different machines) over a network using UDP. It's goals a simplicity and easy of understanding and reliability 4 | 5 | mikadam@stanford.edu 6 | """ 7 | from __future__ import absolute_import 8 | from __future__ import division 9 | from __future__ import print_function 10 | 11 | import socket 12 | import struct 13 | from collections import namedtuple 14 | 15 | import msgpack 16 | 17 | from sys import version_info 18 | 19 | USING_PYTHON_2 = (version_info[0] < 3) 20 | if USING_PYTHON_2: 21 | from time import time as monotonic 22 | else: 23 | from time import monotonic 24 | 25 | timeout = socket.timeout 26 | 27 | MAX_SIZE = 65507 28 | 29 | DEFAULT_IP = "10.0.0.255" 30 | 31 | class Publisher: 32 | def __init__(self, port, ip = DEFAULT_IP): 33 | """ Create a Publisher Object 34 | 35 | Arguments: 36 | port -- the port to publish the messages on 37 | ip -- the ip to send the messages to 38 | """ 39 | self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) 40 | 41 | self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1) 42 | self.broadcast_ip = ip 43 | 44 | self.sock.settimeout(0.2) 45 | self.sock.connect((self.broadcast_ip, port)) 46 | 47 | self.port = port 48 | 49 | def send(self, obj): 50 | """ Publish a message. The obj can be any nesting of standard python types """ 51 | msg = msgpack.dumps(obj, use_bin_type=False) 52 | assert len(msg) < MAX_SIZE, "Encoded message too big!" 53 | self.sock.send(msg) 54 | 55 | def __del__(self): 56 | self.sock.close() 57 | 58 | 59 | class Subscriber: 60 | def __init__(self, port, timeout=0.2): 61 | """ Create a Subscriber Object 62 | 63 | Arguments: 64 | port -- the port to listen to messages on 65 | timeout -- how long to wait before a message is considered out of date 66 | """ 67 | self.max_size = MAX_SIZE 68 | 69 | self.port = port 70 | self.timeout = timeout 71 | 72 | self.last_data = None 73 | self.last_time = float('-inf') 74 | 75 | self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP 76 | self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1) 77 | self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) 78 | if hasattr(socket, "SO_REUSEPORT"): 79 | self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1) 80 | 81 | self.sock.settimeout(timeout) 82 | self.sock.bind(("", port)) 83 | 84 | def recv(self): 85 | """ Receive a single message from the socket buffer. It blocks for up to timeout seconds. 86 | If no message is received before timeout it raises a UDPComms.timeout exception""" 87 | 88 | try: 89 | self.last_data, address = self.sock.recvfrom(self.max_size) 90 | except BlockingIOError: 91 | raise socket.timeout("no messages in buffer and called with timeout = 0") 92 | 93 | self.last_time = monotonic() 94 | return msgpack.loads(self.last_data, raw=USING_PYTHON_2) 95 | 96 | def get(self): 97 | """ Returns the latest message it can without blocking. If the latest massage is 98 | older then timeout seconds it raises a UDPComms.timeout exception""" 99 | try: 100 | self.sock.settimeout(0) 101 | while True: 102 | self.last_data, address = self.sock.recvfrom(self.max_size) 103 | self.last_time = monotonic() 104 | except socket.error: 105 | pass 106 | finally: 107 | self.sock.settimeout(self.timeout) 108 | 109 | current_time = monotonic() 110 | if (current_time - self.last_time) < self.timeout: 111 | return msgpack.loads(self.last_data, raw=USING_PYTHON_2) 112 | else: 113 | raise socket.timeout("timeout=" + str(self.timeout) + \ 114 | ", last message time=" + str(self.last_time) + \ 115 | ", current time=" + str(current_time)) 116 | 117 | def get_list(self): 118 | """ Returns list of messages, in the order they were received""" 119 | msg_bufer = [] 120 | try: 121 | self.sock.settimeout(0) 122 | while True: 123 | self.last_data, address = self.sock.recvfrom(self.max_size) 124 | self.last_time = monotonic() 125 | msg = msgpack.loads(self.last_data, raw=USING_PYTHON_2) 126 | msg_bufer.append(msg) 127 | except socket.error: 128 | pass 129 | finally: 130 | self.sock.settimeout(self.timeout) 131 | 132 | return msg_bufer 133 | 134 | def __del__(self): 135 | self.sock.close() 136 | 137 | 138 | if __name__ == "__main__": 139 | msg = 'very important data' 140 | 141 | a = Publisher(1000) 142 | a.send( {"text": "magic", "number":5.5, "bool":False} ) 143 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # UDPComms 2 | 3 | This is a simple library to enable communication between different processes (potentially on different machines) over a network using UDP. It's goals a simplicity and easy of understanding and reliability. It works for devices on the `10.0.0.X` subnet although this can easiliy be changed. 4 | 5 | Currently it works in python 2 and 3 but it should be relatively simple to extend it to other languages such as C (to run on embeded devices) or Julia (to interface with faster solvers). 6 | 7 | This new verison of the library automatically determines the type of the message and trasmits it along with it, so the subscribers can decode it correctly. While faster to prototype with then systems with explicit type declaration (such as ROS) its easy to shoot yourself in the foot if types are mismatched between publisher and subscriber. 8 | 9 | ### To Send Messages 10 | ``` 11 | >>> from UDPComms import Publisher 12 | >>> a = Publisher(5500) 13 | >>> a.send({"name":"Bob", "age": 20, "height": 180.5, "mass": 70.1}) 14 | ``` 15 | 16 | ### To Receive Messages 17 | 18 | #### recv Method 19 | 20 | 21 | Note: before using the `Subsciber.recv()` method read about the `Subsciber.get()` and understand the difference between them. The `Subsciber.recv()` method will pull a message from the socket buffer and it won't necessary be the most recent message. If you are calling it too slowly and there is a lot of messages you will be getting old messages. The `Subsciber.recv()` can also block for up to `timeout` seconds messing up timing. 22 | 23 | ``` 24 | >>> from UDPComms import Subscriber 25 | >>> a = Subscriber(5500) 26 | >>> message = a.recv() 27 | >>> message['age'] 28 | 20 29 | >>> message['height'] 30 | 180.5 31 | >>> message['name'] 32 | "Bob" 33 | >>> message 34 | {"name":"Bob", "age": 20, "height": 180.5, "mass": 70.1} 35 | ``` 36 | 37 | #### get Method 38 | The preferred way of accessing messages is the `Subsciber.get()` method (as opposed to the `recv()` method). It is guaranteed to be nonblocking so it can be used in places without messing with timing. It checks for any new messages and returns the newest one. 39 | 40 | If the newest message is older then `timeout` seconds it raises the `UDPComms.timeout` exception. **This is an important safety feature!** Make sure to catch the timeout using `try: ... except UDPComms.timeout: ...` and put the robot in a safe configuration (e.g. turn off motors, when the joystick stop sending messages) 41 | 42 | Note that if you call `.get` immediately after creating a subscriber it is possible its hasn't received any messages yet and it will timeout. In general it is better to have a short timeout and gracefully catch timeouts then to have long timeouts 43 | 44 | ``` 45 | >>> from UDPComms import Subscriber, timout 46 | >>> a = Subscriber(5500) 47 | >>> while 1: 48 | >>> try: 49 | >>> message = a.get() 50 | >>> print("got", message) 51 | >>> except timeout: 52 | >>> print("safing robot") 53 | ``` 54 | 55 | #### get_list Method 56 | Although UDPComms isn't ideal for commands that need to be processed in order (as the underlying UDP protocol has no guarantees of deliverry) it can be used as such in a pinch. The `Subsciber.get_list()` method will return all the messages we haven't seen yet in a list 57 | 58 | ``` 59 | >>> from UDPComms import Subscriber, timout 60 | >>> a = Subscriber(5500) 61 | >>> messages = a.get_list() 62 | >>> for message in messages: 63 | >>> print("got", message) 64 | ``` 65 | 66 | ### Publisher Arguments 67 | - `port` 68 | The port the messages will be sent on. If you are part of Stanford Student Robotics make sure there isn't any port conflicts by checking the `UDP Ports` sheet of the [CS Comms System](https://docs.google.com/spreadsheets/d/1pqduUwYa1_sWiObJDrvCCz4Al3pl588ytE4u-Dwa6Pw/edit?usp=sharing) document. If you are not I recommend keep track of your port numbers somewhere. It's possible that in the future UDPComms will have a system of naming (with a string) as opposed to numbering publishers. 69 | - `ip` By default UDPComms sends to the `10.0.0.X` subnet, but can be changed to a different ip using this argument. Set to localhost (`127.0.0.1`) for development on the same computer. 70 | 71 | ### Subscriber Arguments 72 | 73 | - `port` 74 | The port the subscriber will be listen on. 75 | - `timeout` 76 | If the `recv()` method don't get a message in `timeout` seconds it throws a `UDPComms.timeout` exception 77 | 78 | ### Rover 79 | 80 | The library also comes with the `rover` command that can be used to interact with the messages manually. 81 | 82 | 83 | | Command | Descripion | 84 | |---------|------------| 85 | | `rover peek port` | print messages sent on port `port` | 86 | | `rover poke port rate` | send messages to `port` once every `rate` milliseconds. Type message in json format and press return | 87 | 88 | There are more commands used for starting and stoping services described in [this repo](https://github.com/stanfordroboticsclub/RPI-Setup/blob/master/README.md) 89 | 90 | ### To Install 91 | 92 | ``` 93 | $git clone https://github.com/stanfordroboticsclub/UDPComms.git 94 | $sudo bash UDPComms/install.sh 95 | ``` 96 | 97 | ### To Update 98 | 99 | ``` 100 | $cd UDPComms 101 | $git pull 102 | $sudo bash install.sh 103 | ``` 104 | 105 | ### Developing without hardware 106 | 107 | Because this library expects you to be connected to the robot (`10.0.0.X`) network you won't be able to send messages between two programs on your computer without any other hardware connected. You can get around this by forcing your (unused) ethernet interface to get an ip on the rover network without anything being connected to it. On my computer you can do this using this command: 108 | 109 | `sudo ifconfig en1 10.0.0.52 netmask 255.255.255.0` 110 | 111 | Note that the exact command depends which interface on your computer is unused and what ip you want. So only use this if you know what you are doing. 112 | 113 | If you have internet access a slightly cleaner way to do it is to setup [RemoteVPN](https://github.com/stanfordroboticsclub/RemoteVPN) on your development computer and simply connect to a development network (given if you are the only computer there) 114 | 115 | ### Known issues: 116 | 117 | - Macs have issues sending large messages. They are fine receiving them. I think it is related to [this issue](https://github.com/BanTheRewind/Cinder-Asio/issues/9). I wonder does it work on Linux by chance (as the packets happen to be in order) but so far we didn't have issues. 118 | 119 | - Messages over the size of one MTU (typically 1500 bytes) will be split up into multiple frames which reduces their chance of getting to their destination on wireless networks. 120 | 121 | --------------------------------------------------------------------------------