├── .gitignore ├── .idea ├── directKiwi.iml ├── inspectionProfiles │ └── Project_Default.xml ├── misc.xml ├── modules.xml └── vcs.xml ├── README.md ├── directKiwi.bat ├── directKiwi.blank.cfg ├── directKiwi.cfg ├── directKiwi.py ├── directKiwi_server_list.db ├── directKiwi_static_server_list.db ├── icon.gif ├── kiwiclient ├── kiwi │ ├── __init__.py │ ├── client.py │ ├── wavreader.py │ ├── worker.py │ └── wsclient.py ├── kiwirecorder.py ├── microkiwi_waterfall.py └── mod_pywebsocket │ ├── __init__.py │ ├── _stream_base.py │ ├── _stream_hixie75.py │ ├── _stream_hybi.py │ ├── common.py │ ├── extensions.py │ ├── http_header_util.py │ ├── stream.py │ └── util.py ├── maps ├── How to find the other maps.txt ├── directKiwi_map_grayscale_with_sea.jpg └── directTDoA_map.jpg ├── setup.bat └── setup.sh /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | *.py[cod] 3 | *$py.class 4 | 5 | # direct Kiwi 6 | directKiwi_server_list.db.bak -------------------------------------------------------------------------------- /.idea/directKiwi.iml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 11 | -------------------------------------------------------------------------------- /.idea/inspectionProfiles/Project_Default.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 20 | -------------------------------------------------------------------------------- /.idea/misc.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 7 | -------------------------------------------------------------------------------- /.idea/modules.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /.idea/vcs.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # directKiwi v7.21 2 | 3 | This piece of software is JUST a GUI written for Python 2/3 designed to fast connect audio socket to KiwiSDR servers around the world using modified versions @ https://github.com/dev-zzo/kiwiclient or related fork @ https://github.com/jks-prv/kiwiclient 4 | 5 | Thanks to Pierre Ynard (linkfanel) for the listing of available KiwiSDR nodes used as source for the TDoA map update process (http://rx.linkfanel.net) 6 | 7 | ## Stuff required to run the software: 8 | 9 | * Tk 10 | * numpy (https://pypi.org/project/numpy/) 11 | * sounddevice (https://pypi.org/project/sounddevice/) 12 | * libsamplerate (https://pypi.org/project/samplerate/) 13 | * requests (https://pypi.org/project/requests/) 14 | * pillow (https://pypi.org/project/Pillow/) 15 | * matplotlib (https://pypi.org/project/matplotlib/) 16 | 17 | ## INSTALL AND RUN (on WINDOWS) 18 | 19 | Install python 2 or 3 20 | 21 | If you don't have git for windows installed, just download `https://github.com/llinkz/directKiwi/archive/master.zip` and unzip the package somewhere 22 | 23 | Else in 'Git Bash' type : `git clone --recursive https://github.com/llinkz/directKiwi` 24 | 25 | Double-click on `setup.bat` (this script will install python modules) 26 | 27 | Double-click on `directKiwi.bat` 28 | 29 | 30 | ## INSTALL AND RUN (on LINUX) Thanks Daniel E. for the install procedure 31 | 32 | Install python 2 or 3 33 | 34 | Install python-pip (search for the right package for your distro) 35 | 36 | `git clone --recursive https://github.com/llinkz/directKiwi` 37 | 38 | `cd directKiwi` 39 | 40 | `./setup.sh` (this script will install python modules, it may fail, if so, install modules manually using your package manager) 41 | 42 | `./directKiwi.py` (note: check the shebang if it fails on your system. On my Archlinux it should be "#!/usr/bin/python2" for example) 43 | 44 | 45 | ## INSTALL AND RUN (on MAC OS) Thanks Nicolas M. for the install procedure 46 | 47 | * REQUIREMENT Xcode + Homebrew (https://brew.sh/index_fr) 48 | 49 | Install Homebrew, in terminal : `/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"` 50 | 51 | Install Python 2 or 3, in terminal : `brew install python@#` 52 | 53 | `git clone --recursive https://github.com/llinkz/directKiwi` 54 | 55 | `cd directKiwi` 56 | 57 | `./setup.sh` (this script will install python modules, it may fail, if so, install modules manually using your package manager) 58 | 59 | `./directKiwi.py` 60 | 61 | 62 | ## TIP 63 | * Zoom function is actually disabled because it has some bugs, but if you need the feature active just uncomment 64 | 65 | `line 459 for Windows` or `lines 460 + 461 for Linux (and MacOS ?)` 66 | 67 | ## LICENSE 68 | * This python GUI code has been written and released under the "do what the f$ck you want with it" license 69 | 70 | ## WARNING 71 | * This code has been tested on Raspberry Pi B+ but there was some issues with a third-party USB sound card, so it was not working here. For testers : feedback appreciated, thanks. 72 | 73 | ## TODO LIST 74 | * enable IQ mode with direct recording to file ? 75 | * real-time management of the KiwiSDR (freq change, demodulation change, AGC/MGC change.. work in progress) 76 | 77 | ## CHANGE LOG 78 | * v1.00 : first working version, basic, only freq/mode/bw, connect/disconnect 79 | * v1.10 : design modification + console output 80 | * v1.1beta : adding the smeter 81 | * v1.20 : adding the spectrum extension 82 | * v1.30 : adding the recorder extension 83 | * v1.31 : dealing with processes & sub-processes + code clean-up 84 | * v1.32 : changed LP & HP from QSlider to QScrollBar style for 100Hz steps clicks + some comments added + code clean-up 85 | * v1.33 : AM modulation has fixed BW (10kHz) - CW has frequency offset -1kHz BW set to 200Hz centered on desired freq 86 | * v1.40 : most "update proc" bugs has been removed (major concerns wrong user entries in their kiwisdr setup/reg) 87 | * v1.40beta : The KiwiSDR manual server listing update has been added - note: requires pygeoip module, still some bugs 88 | * v1.41 : adding the possibility to manually enter IP:PORT/HOSTNAME:PORT to connect to a specific KiwiSDR server 89 | * v1.42 : adding some regexp checks to validate frequencies and IP:PORT/HOSTNAME:PORT in both top inputboxes 90 | * v2.00 : BUG SOLVED: "QScrollbar & TextEdit refresh still not working with direct program lunch" 91 | * v2.01 : KiwiSDRclient.py cleaned + S-meter bars removed 92 | * v2.10 : adding the AGC control (manual or auto) 93 | * v2.20 : spectrum & recording modules has been removed, too many issues with client-side sound card settings 94 | * v2.30 : it's now possible to switch from node to node directly by double-clicking on the listing lines (prev forgiven) 95 | * v3.00 : GUI based on TK now, no more PyQT stuff needed 96 | * v3.10 : adding labels, node listing and console scrollbars, cancel connect possibility + all CRLF converted to LF 97 | * v3.11 : some modifications in the console output log format + an update process bug has been solved 98 | * v3.20 : update process finally re-written and still using GeoIP - node listing now redrawing itself, no more restart needed 99 | * v3.30 : host ports are checked before filling the node listing, normally all nodes listed are reachable now 100 | * v3.40 : adding top bar menus for colors changing and saving directKiwi.cfg configuration (colors + geometry) 101 | * v3.50 : adding default lowpass, highpass, modulation to directKiwi.cfg configuration file 102 | * v3.60 : adding default agc/mgc, mgc gain, hang, agc threshold, agc slope, agc decay to directKiwi.cfg configuration file 103 | * v3.61 : agc/mgc listbox now (previously checkbox) + xxx.proxy.kiwisdr.com hosts (out of USA) locations fixed + autosorting list at start 104 | * v3.62 : fixed a TK issue than was caused by python 2.7.15 version (code was written under 2.7.14 105 | * v3.63 : fixed an issue with the India located remote that has few informations in kiwisdr.com/public listing page (source for updates) 106 | * v4.00 : the GUI is now using directTDoA design, nodes are displayed on a World map instead of a table 107 | * v5.00 : code clean up + cfg file now in json format + faster way to switch between nodes (left click only) - no more s-meter - icon size change possible 108 | * v5.10 : no more pygame & scipy (for MacOS), sounddevice python module + libsamplerate instead -> works with 20kHz KiwiSDRs + no audio compression set by default 109 | * v6.00 : IS0KYB microkiwi_waterfall script added (SNR measurement + waterfall display), IS0KYB SNR website source is not used anymore + code clean up 110 | * v6.10 : adding a line to open Web Browser with pre-set TDoA extension loaded (requested by user) 111 | * v7.00 : directKiwi now uses same GUI as directTDoA - python 3 compatibility added - bug fixed on the map update due to some format modifications on the website sources 112 | * v7.10 : Restart GUI routine modified + directKiwi.bat & setup.bat added for Windows users + bug fix that caused the map to move suddenly far away when selecting a node (problem only noticed on Windows OS) + added functionality to manage the overlapping of icons on the map. Now when you click near a cluster of multiple nodes, a menu will appear and allow you to choose the one you really want (to listen or to display node menu) + bug fixes 113 | * v7.20 : Bug fixes for demodulation bandpass filters + modification on microkiwi_waterfall.py, to deal with nodes that have blocked frequency ranges 114 | * v7.21 : Bug fixes 115 | 116 | Enjoy 117 | 118 | linkz 119 | 120 | October 2020 update 121 | -------------------------------------------------------------------------------- /directKiwi.bat: -------------------------------------------------------------------------------- 1 | start /affinity 1 pythonw directKiwi.py -------------------------------------------------------------------------------- /directKiwi.blank.cfg: -------------------------------------------------------------------------------- 1 | { 2 | "demod": { 3 | "slope": 6, 4 | "decay": 1000, 5 | "agc": 1, 6 | "hang": 1, 7 | "lp_cut": 100, 8 | "mgain": 10, 9 | "hp_cut": 3000, 10 | "thres": -75 11 | }, 12 | "map": { 13 | "std": "#00ff00", 14 | "blk": "#ff0000", 15 | "iconsize": 2, 16 | "icontype": 1, 17 | "mapfl": 0, 18 | "hlt": "#ffff00", 19 | "fav": "#ffff00", 20 | "file": "maps/directKiwi_map_grayscale_with_sea.jpg", 21 | "y1": "1264.0", 22 | "y0": "259.0", 23 | "x0": "162.0", 24 | "x1": "2078.0" 25 | }, 26 | "guicolors": { 27 | "main_b": "#d9d9d9", 28 | "stat_b": "#ffffff", 29 | "main_f": "#000000", 30 | "stat_f": "#000000", 31 | "cons_b": "#000000", 32 | "grad": 10, 33 | "cons_f": "#00ff00", 34 | "thres": 180 35 | }, 36 | "nodes": { 37 | "blacklist": [], 38 | "whitelist": [] 39 | } 40 | } 41 | -------------------------------------------------------------------------------- /directKiwi.cfg: -------------------------------------------------------------------------------- 1 | { 2 | "demod": { 3 | "slope": 6, 4 | "decay": 1000, 5 | "agc": 1, 6 | "hang": 1, 7 | "lp_cut": 100, 8 | "mgain": 10, 9 | "hp_cut": 3000, 10 | "thres": -75 11 | }, 12 | "map": { 13 | "std": "#00ff00", 14 | "blk": "#ff0000", 15 | "iconsize": 2, 16 | "icontype": 1, 17 | "mapfl": 0, 18 | "hlt": "#ffff00", 19 | "fav": "#ffff00", 20 | "file": "maps/directKiwi_map_grayscale_with_sea.jpg", 21 | "y1": "1264.0", 22 | "y0": "259.0", 23 | "x0": "162.0", 24 | "x1": "2078.0" 25 | }, 26 | "guicolors": { 27 | "main_b": "#d9d9d9", 28 | "stat_b": "#ffffff", 29 | "main_f": "#000000", 30 | "stat_f": "#000000", 31 | "cons_b": "#000000", 32 | "grad": 10, 33 | "cons_f": "#00ff00", 34 | "thres": 180 35 | }, 36 | "nodes": { 37 | "blacklist": [], 38 | "whitelist": [] 39 | } 40 | } 41 | -------------------------------------------------------------------------------- /directKiwi_static_server_list.db: -------------------------------------------------------------------------------- 1 | [] 2 | -------------------------------------------------------------------------------- /icon.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/llinkz/directKiwi/c18ef280edbb9c04d06c2a45e004e4a3bf1f8698/icon.gif -------------------------------------------------------------------------------- /kiwiclient/kiwi/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | ## -*- python -*- 3 | 4 | #from .client import KiwiSDRStream 5 | #from .client import KiwiError,KiwiTooBusyError,KiwiDownError,KiwiBadPasswordError,KiwiTimeLimitError,KiwiServerTerminatedConnection,KiwiUnknownModulation 6 | from .client import * 7 | from .worker import KiwiWorker 8 | from .wavreader import * 9 | -------------------------------------------------------------------------------- /kiwiclient/kiwi/client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import array 4 | import logging 5 | import socket 6 | import struct 7 | import time 8 | import numpy as np 9 | try: 10 | import urllib.parse as urllib 11 | except ImportError: 12 | import urllib 13 | 14 | import sys 15 | if sys.version_info > (3,): 16 | buffer = memoryview 17 | def bytearray2str(b): 18 | return b.decode('ascii') 19 | else: 20 | def bytearray2str(b): 21 | return str(b) 22 | 23 | import json 24 | import mod_pywebsocket.common 25 | from mod_pywebsocket._stream_base import ConnectionTerminatedException 26 | from mod_pywebsocket.stream import Stream, StreamOptions 27 | from .wsclient import ClientHandshakeProcessor, ClientRequest 28 | 29 | # 30 | # IMAADPCM decoder 31 | # 32 | 33 | stepSizeTable = ( 34 | 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 23, 25, 28, 31, 34, 35 | 37, 41, 45, 50, 55, 60, 66, 73, 80, 88, 97, 107, 118, 130, 143, 36 | 157, 173, 190, 209, 230, 253, 279, 307, 337, 371, 408, 449, 494, 37 | 544, 598, 658, 724, 796, 876, 963, 1060, 1166, 1282, 1411, 1552, 38 | 1707, 1878, 2066, 2272, 2499, 2749, 3024, 3327, 3660, 4026, 39 | 4428, 4871, 5358, 5894, 6484, 7132, 7845, 8630, 9493, 10442, 40 | 11487, 12635, 13899, 15289, 16818, 18500, 20350, 22385, 24623, 41 | 27086, 29794, 32767) 42 | 43 | indexAdjustTable = [ 44 | -1, -1, -1, -1, # +0 - +3, decrease the step size 45 | 2, 4, 6, 8, # +4 - +7, increase the step size 46 | -1, -1, -1, -1, # -0 - -3, decrease the step size 47 | 2, 4, 6, 8 # -4 - -7, increase the step size 48 | ] 49 | 50 | 51 | def clamp(x, xmin, xmax): 52 | if x < xmin: 53 | return xmin 54 | if x > xmax: 55 | return xmax 56 | return x 57 | 58 | class ImaAdpcmDecoder(object): 59 | def __init__(self): 60 | self.index = 0 61 | self.prev = 0 62 | 63 | def _decode_sample(self, code): 64 | step = stepSizeTable[self.index] 65 | self.index = clamp(self.index + indexAdjustTable[code], 0, len(stepSizeTable) - 1) 66 | difference = step >> 3 67 | if ( code & 1 ): 68 | difference += step >> 2 69 | if ( code & 2 ): 70 | difference += step >> 1 71 | if ( code & 4 ): 72 | difference += step 73 | if ( code & 8 ): 74 | difference = -difference 75 | sample = clamp(self.prev + difference, -32768, 32767) 76 | self.prev = sample 77 | return sample 78 | 79 | def decode(self, data): 80 | fcn = ord if isinstance(data, str) else lambda x : x 81 | samples = array.array('h') 82 | for b in map(fcn, data): 83 | sample0 = self._decode_sample(b & 0x0F) 84 | sample1 = self._decode_sample(b >> 4) 85 | samples.append(sample0) 86 | samples.append(sample1) 87 | return samples 88 | 89 | # 90 | # KiwiSDR WebSocket client 91 | # 92 | 93 | class KiwiError(Exception): 94 | pass 95 | class KiwiTooBusyError(KiwiError): 96 | pass 97 | class KiwiDownError(KiwiError): 98 | pass 99 | class KiwiBadPasswordError(KiwiError): 100 | pass 101 | class KiwiTimeLimitError(KiwiError): 102 | pass 103 | class KiwiServerTerminatedConnection(KiwiError): 104 | pass 105 | class KiwiUnknownModulation(KiwiError): 106 | pass 107 | 108 | class KiwiSDRStreamBase(object): 109 | """KiwiSDR WebSocket stream base client.""" 110 | 111 | def __init__(self): 112 | self._socket = None 113 | self._decoder = None 114 | self._sample_rate = None 115 | self._isIQ = False 116 | self._version_major = None 117 | self._version_minor = None 118 | self._modulation = None 119 | self._stream = None 120 | 121 | def connect(self, host, port): 122 | # self._prepare_stream(host, port, 'SND') 123 | pass 124 | 125 | def _process_message(self, tag, body): 126 | logging.warn('Unknown message tag: %s' % tag) 127 | logging.warn(repr(body)) 128 | 129 | def _prepare_stream(self, host, port, which): 130 | self._stream_name = which 131 | self._socket = socket.create_connection(address=(host, port), timeout=self._options.socket_timeout) 132 | uri = '/%d/%s' % (self._options.timestamp, which) 133 | handshake = ClientHandshakeProcessor(self._socket, host, port) 134 | handshake.handshake(uri) 135 | 136 | request = ClientRequest(self._socket) 137 | request.ws_version = mod_pywebsocket.common.VERSION_HYBI13 138 | 139 | stream_option = StreamOptions() 140 | stream_option.mask_send = True 141 | stream_option.unmask_receive = False 142 | 143 | self._stream = Stream(request, stream_option) 144 | 145 | def _send_message(self, msg): 146 | if msg != 'SET keepalive': 147 | logging.debug("send SET (%s) %s", self._stream_name, msg) 148 | self._stream.send_message(msg) 149 | 150 | def _set_auth(self, client_type, password='', tlimit_password=''): 151 | if tlimit_password != '': 152 | self._send_message('SET auth t=%s p=%s ipl=%s' % (client_type, password, tlimit_password)) 153 | else: 154 | self._send_message('SET auth t=%s p=%s' % (client_type, password)) 155 | 156 | def set_name(self, name): 157 | self._send_message('SET ident_user=%s' % (name)) 158 | 159 | def set_geo(self, geo): 160 | self._send_message('SET geo=%s' % (geo)) 161 | 162 | def _set_keepalive(self): 163 | self._send_message('SET keepalive') 164 | 165 | def _process_ws_message(self, message): 166 | tag = bytearray2str(message[0:3]) 167 | body = message[3:] 168 | self._process_message(tag, body) 169 | 170 | 171 | class KiwiSDRStream(KiwiSDRStreamBase): 172 | """KiwiSDR WebSocket stream client.""" 173 | 174 | def __init__(self, *args, **kwargs): 175 | super(KiwiSDRStream, self).__init__() 176 | self._decoder = ImaAdpcmDecoder() 177 | self._sample_rate = None 178 | self._version_major = None 179 | self._version_minor = None 180 | self._modulation = None 181 | self._compression = True 182 | self._gps_pos = [0,0] 183 | self._s_meter_avgs = self._s_meter_cma = 0 184 | self._s_meter_valid = False 185 | self._tot_meas_count = self._meas_count = 0 186 | self._stop = False 187 | 188 | self.MAX_FREQ = 30e3 ## in kHz 189 | self.MAX_ZOOM = 14 190 | self.WF_BINS = 1024 191 | 192 | def connect(self, host, port): 193 | self._prepare_stream(host, port, self._type) 194 | 195 | def set_mod(self, mod, lc, hc, freq): 196 | mod = mod.lower() 197 | self._modulation = mod 198 | if lc == None or hc == None: 199 | if mod == 'am' or mod == 'sam': 200 | lc = -6000 if lc == None else lc 201 | hc = 6000 if hc == None else hc 202 | elif mod == 'sau': 203 | lc = -30 if lc == None else lc 204 | hc = 6000 if hc == None else hc 205 | elif mod == 'sal': 206 | lc = -6000 if lc == None else lc 207 | hc = 30 if hc == None else hc 208 | elif mod == 'lsb': 209 | lc = -3000 if lc == None else lc 210 | hc = -100 if hc == None else hc 211 | elif mod == 'usb': 212 | lc = 100 if lc == None else lc 213 | hc = 3000 if hc == None else hc 214 | elif mod == 'cw': 215 | lc = 500 if lc == None else lc 216 | hc = 1000 if hc == None else hc 217 | elif mod == 'nbfm': 218 | lc = -6000 if lc == None else lc 219 | hc = 6000 if hc == None else hc 220 | elif mod == 'iq': 221 | lc = -5000 if lc == None else lc 222 | hc = 5000 if hc == None else hc 223 | else: 224 | raise KiwiUnknownModulation('"%s"' % mod) 225 | self._send_message('SET mod=%s low_cut=%d high_cut=%d freq=%.3f' % (mod, lc, hc, freq)) 226 | 227 | def set_agc(self, on=False, hang=False, thresh=-100, slope=6, decay=1000, gain=50): 228 | self._send_message('SET agc=%d hang=%d thresh=%d slope=%d decay=%d manGain=%d' % (on, hang, thresh, slope, decay, gain)) 229 | 230 | def set_squelch(self, sq, thresh): 231 | self._send_message('SET squelch=%d max=%d' % (sq, thresh)) 232 | 233 | def set_autonotch(self, val): 234 | self._send_message('SET lms_autonotch=%d' % (val)) 235 | 236 | def set_noise_blanker(self, gate, thresh): 237 | self._send_message('SET nb=%d th=%d' % (gate, thresh)) 238 | 239 | def _set_ar_ok(self, ar_in, ar_out): 240 | self._send_message('SET AR OK in=%d out=%d' % (ar_in, ar_out)) 241 | 242 | def _set_gen(self, freq, attn): 243 | self._send_message('SET genattn=%d' % (attn)) 244 | self._send_message('SET gen=%d mix=%d' % (freq, -1)) 245 | 246 | def _set_zoom_cf(self, zoom, cf): 247 | major = self._version_major if self._version_major != None else -1 248 | minor = self._version_minor if self._version_minor != None else -1 249 | if major >= 2 or (major == 1 and minor >= 329): 250 | self._send_message('SET zoom=%d cf=%f' % (zoom, cf)) 251 | else: 252 | counter = start_frequency_to_counter(cf - zoom_to_span(zoom)/2) 253 | self._send_message('SET zoom=%d start=%f' % (0, counter)) 254 | 255 | def zoom_to_span(self, zoom): 256 | """return frequency span in kHz for a given zoom level""" 257 | assert(zoom >=0 and zoom <= self.MAX_ZOOM) 258 | return self.MAX_FREQ/2**zoom 259 | 260 | def start_frequency_to_counter(self, start_frequency): 261 | """convert a given start frequency in kHz to the counter value used in _set_zoom_start""" 262 | assert(start_frequency >= 0 and start_frequency <= self.MAX_FREQ) 263 | counter = round(start_frequency/self.MAX_FREQ * 2**self.MAX_ZOOM * self.WF_BINS) 264 | ## actual start frequency 265 | start_frequency = counter * self.MAX_FREQ / self.WF_BINS / 2**self.MAX_ZOOM 266 | return counter,start_frequency 267 | 268 | def _set_zoom_start(self, zoom, start): 269 | self._send_message('SET zoom=%d start=%f' % (zoom, start)) 270 | 271 | def _set_maxdb_mindb(self, maxdb, mindb): 272 | self._send_message('SET maxdb=%d mindb=%d' % (maxdb, mindb)) 273 | 274 | def _set_snd_comp(self, comp): 275 | self._compression = comp 276 | self._send_message('SET compression=%d' % (1 if comp else 0)) 277 | 278 | def _set_wf_comp(self, comp): 279 | self._compression = comp 280 | self._send_message('SET wf_comp=%d' % (1 if comp else 0)) 281 | 282 | def _set_wf_speed(self, wf_speed): 283 | self._send_message('SET wf_speed=%d' % wf_speed) 284 | 285 | def _process_msg_param(self, name, value): 286 | if name == 'load_cfg': 287 | logging.debug("load_cfg: (cfg info not printed)") 288 | d = json.loads(urllib.unquote(value)) 289 | self._gps_pos = [float(x) for x in urllib.unquote(d['rx_gps'])[1:-1].split(",")[0:2]] 290 | if self._options.idx == 0: 291 | logging.info("GNSS position: lat,lon=[%+6.2f, %+7.2f]" % (self._gps_pos[0], self._gps_pos[1])) 292 | self._on_gnss_position(self._gps_pos) 293 | else: 294 | logging.debug("recv MSG (%s) %s: %s", self._stream_name, name, value) 295 | # Handle error conditions 296 | if name == 'too_busy': 297 | raise KiwiTooBusyError('%s: all %s client slots taken' % (self._options.server_host, value)) 298 | if name == 'badp' and value == '1': 299 | raise KiwiBadPasswordError('%s: bad password' % self._options.server_host) 300 | if name == 'down': 301 | raise KiwiDownError('%s: server is down atm' % self._options.server_host) 302 | # Handle data items 303 | if name == 'audio_rate': 304 | self._set_ar_ok(int(value), 44100) 305 | elif name == 'sample_rate': 306 | self._sample_rate = float(value) 307 | self._on_sample_rate_change() 308 | # Optional, but is it?.. 309 | self.set_squelch(0, 0) 310 | self.set_autonotch(0) 311 | self._set_gen(0, 0) 312 | # Required to get rolling 313 | self._setup_rx_params() 314 | # Also send a keepalive 315 | self._set_keepalive() 316 | elif name == 'wf_setup': 317 | # Required to get rolling 318 | self._setup_rx_params() 319 | # Also send a keepalive 320 | self._set_keepalive() 321 | elif name == 'version_maj': 322 | self._version_major = int(value) 323 | if self._options.idx == 0 and self._version_major is not None and self._version_minor is not None: 324 | logging.info("Server version: %d.%d", self._version_major, self._version_minor) 325 | elif name == 'version_min': 326 | self._version_minor = int(value) 327 | if self._options.idx == 0 and self._version_major is not None and self._version_minor is not None: 328 | logging.info("Server version: %d.%d", self._version_major, self._version_minor) 329 | 330 | def _process_message(self, tag, body): 331 | if tag == 'MSG': 332 | self._process_msg(bytearray2str(body[1:])) ## skip 1st byte 333 | elif tag == 'SND': 334 | try: 335 | self._process_aud(body) 336 | except Exception as e: 337 | logging.error(e) 338 | # Ensure we don't get kicked due to timeouts 339 | self._set_keepalive() 340 | elif tag == 'W/F': 341 | self._process_wf(body[1:]) ## skip 1st byte 342 | # Ensure we don't get kicked due to timeouts 343 | self._set_keepalive() 344 | else: 345 | logging.warn("unknown tag %s" % tag) 346 | pass 347 | 348 | def _process_msg(self, body): 349 | for pair in body.split(' '): 350 | if '=' in pair: 351 | name, value = pair.split('=', 1) 352 | self._process_msg_param(name, value) 353 | else: 354 | name = pair 355 | self._process_msg_param(name, None) 356 | 357 | def _process_aud(self, body): 358 | flags,seq, = struct.unpack('H', buffer(body[5:7])) 360 | data = body[7:] 361 | rssi = 0.1*smeter - 127 362 | ##logging.info("SND flags %2d seq %6d RSSI %6.1f len %d" % (flags, seq, rssi, len(data))) 363 | if self._options.ADC_OV and (flags & 2): 364 | print(" ADC OV") 365 | 366 | # first rssi is no good because first audio buffer is leftover from last time this channel was used 367 | if self._options.S_meter >= 0 and not self._s_meter_valid: 368 | # tlimit in effect if streaming RSSI 369 | self._start_time = time.time() 370 | self._start_sm_ts = time.gmtime() 371 | self._s_meter_valid = True 372 | if not self._options.sound: 373 | return 374 | else: 375 | 376 | # streaming 377 | if self._options.S_meter == 0 and self._options.sdt == 0: 378 | self._meas_count += 1 379 | self._tot_meas_count += 1 380 | ts = time.strftime('%d-%b-%Y %H:%M:%S UTC ', time.gmtime()) if self._options.tstamp else '' 381 | print("%sRSSI: %6.1f %d" % (ts, rssi, self._options.tstamp)) 382 | if not self._options.sound: 383 | return 384 | else: 385 | 386 | # averaging with optional dt 387 | if self._options.S_meter >= 0: 388 | self._s_meter_cma = (self._s_meter_cma * self._s_meter_avgs) + rssi 389 | self._s_meter_avgs += 1 390 | self._s_meter_cma /= self._s_meter_avgs 391 | self._meas_count += 1 392 | self._tot_meas_count += 1 393 | now = time.gmtime() 394 | sec_of_day = lambda x: 3600*x.tm_hour + 60*x.tm_min + x.tm_sec 395 | if self._options.sdt != 0: 396 | interval = (self._start_sm_ts is not None) and (sec_of_day(now)//self._options.sdt != sec_of_day(self._start_sm_ts)//self._options.sdt) 397 | meas_sec = float(self._meas_count)/self._options.sdt 398 | else: 399 | interval = False 400 | if self._s_meter_avgs == self._options.S_meter or interval: 401 | ts = time.strftime('%d-%b-%Y %H:%M:%S UTC ', now) if self._options.tstamp else '' 402 | if self._options.stats and self._options.sdt: 403 | print("%sRSSI: %6.1f %.1f meas/sec" % (ts, self._s_meter_cma, meas_sec)) 404 | else: 405 | print("%sRSSI: %6.1f" % (ts, self._s_meter_cma)) 406 | if interval: 407 | self._start_sm_ts = time.gmtime() 408 | if self._options.sdt == 0: 409 | self._stop = True 410 | else: 411 | self._s_meter_avgs = self._s_meter_cma = 0 412 | self._meas_count = 0 413 | if not self._options.sound: 414 | return 415 | 416 | if self._modulation == 'iq': 417 | gps = dict(zip(['last_gps_solution', 'dummy', 'gpssec', 'gpsnsec'], struct.unpack(' tlimit 516 | if time_limit or self._stop: 517 | if self._options.stats and self._tot_meas_count > 0 and self._start_time != None: 518 | print("%.1f meas/sec" % (float(self._tot_meas_count) / (time.time() - self._start_time))) 519 | raise KiwiTimeLimitError('time limit reached') 520 | 521 | # EOF 522 | -------------------------------------------------------------------------------- /kiwiclient/kiwi/wavreader.py: -------------------------------------------------------------------------------- 1 | # -*- python -*- 2 | 3 | import collections 4 | import struct 5 | import numpy as np 6 | from chunk import Chunk 7 | 8 | class KiwiIQWavError(Exception): 9 | pass 10 | 11 | class KiwiIQWavReader(collections.Iterator): 12 | def __init__(self, f): 13 | super(KiwiIQWavReader, self).__init__() 14 | self._frame_counter = 0 15 | self._last_gpssec = -1 16 | try: 17 | self._f = open(f, 'rb') 18 | self._initfp(self._f) 19 | except: 20 | if self._f: 21 | self._f.close() 22 | raise 23 | 24 | def __del__(self): 25 | if self._f: 26 | self._f.close() 27 | 28 | def _initfp(self, file): 29 | self._file = Chunk(file, bigendian = 0) 30 | if self._file.getname() != b'RIFF': 31 | raise KiwiIQWavError('file does not start with RIFF id') 32 | if self._file.read(4) != b'WAVE': 33 | raise KiwiIQWavError('not a WAVE file') 34 | 35 | chunk = Chunk(self._file, bigendian = 0) 36 | if chunk.getname() != b'fmt ': 37 | raise KiwiIQWavError('fmt chunk is missing') 38 | 39 | self._proc_chunk_fmt(chunk) 40 | chunk.skip() 41 | 42 | ## for python3 43 | def __next__(self): 44 | return self.next() 45 | 46 | ## for python2 47 | def next(self): 48 | try: 49 | chunk = Chunk(self._file, bigendian = 0) 50 | if chunk.getname() != b'kiwi': 51 | raise KiwiIQWavError('missing KiwiSDR GNSS time stamp') 52 | 53 | self._proc_chunk_kiwi(chunk) 54 | chunk.skip() 55 | 56 | chunk = Chunk(self._file, bigendian = 0) 57 | if chunk.getname() != b'data': 58 | raise KiwiIQWavError('missing WAVE data chunk') 59 | 60 | tz = self._proc_chunk_data(chunk) 61 | chunk.skip() 62 | return tz 63 | except EOFError: 64 | raise StopIteration 65 | 66 | def process_iq_samples(self, t,z): 67 | ## print(len(t), len(z)) 68 | pass 69 | 70 | def get_samplerate(self): 71 | return self._samplerate 72 | 73 | def _proc_chunk_fmt(self, chunk): 74 | wFormatTag, nchannels, self._samplerate, dwAvgBytesPerSec, wBlockAlign = struct.unpack('= 0: 86 | if self._frame_counter < 3: 87 | self._samplerate = n/(self.gpssec - self._last_gpssec) 88 | else: 89 | self._samplerate = 0.9*self._samplerate + 0.1*n/(self.gpssec - self._last_gpssec) 90 | 91 | if self._frame_counter >= 2: 92 | t = np.arange(start = self.gpssec, 93 | stop = self.gpssec + (n-0.5)/self._samplerate, 94 | step = 1/self._samplerate, 95 | dtype = np.float64) 96 | ##t = self.gpssec + np.array(range(n))/self._samplerate 97 | self.process_iq_samples(t,z) 98 | 99 | self._last_gpssec = self.gpssec 100 | self._frame_counter += (self._frame_counter < 3) 101 | return t,z 102 | 103 | def read_kiwi_iq_wav(filename): 104 | t = [] 105 | z = [] 106 | for _t,_z in KiwiIQWavReader(filename): 107 | if _t is None: 108 | continue 109 | t.append(_t) 110 | z.append(_z) 111 | return np.concatenate(t), np.concatenate(z) 112 | 113 | if __name__ == '__main__': 114 | import sys 115 | if len(sys.argv) != 2: 116 | print('Usage: ...') 117 | 118 | [t,z]=read_kiwi_iq_wav(sys.argv[1]) 119 | print (len(t),len(z), t[-1], z[-1], (t[-1]-t[-2])*1e6) 120 | -------------------------------------------------------------------------------- /kiwiclient/kiwi/worker.py: -------------------------------------------------------------------------------- 1 | ## -*- python -*- 2 | 3 | import logging 4 | import threading 5 | from traceback import print_exc 6 | 7 | from .client import KiwiTooBusyError, KiwiTimeLimitError, KiwiServerTerminatedConnection 8 | 9 | class KiwiWorker(threading.Thread): 10 | def __init__(self, group=None, target=None, name=None, args=(), kwargs=None): 11 | super(KiwiWorker, self).__init__(group=group, target=target, name=name) 12 | self._recorder, self._options, self._run_event = args 13 | self._recorder._reader = True 14 | self._event = threading.Event() 15 | 16 | def _do_run(self): 17 | return self._run_event.is_set() 18 | 19 | def run(self): 20 | self.connect_count = self._options.connect_retries 21 | 22 | while self._do_run(): 23 | try: 24 | self._recorder.connect(self._options.server_host, self._options.server_port) 25 | except Exception as e: 26 | logging.info("Failed to connect, sleeping and reconnecting error='%s'" %e) 27 | if self._options.is_kiwi_tdoa: 28 | self._options.status = 1 29 | break 30 | self.connect_count -= 1 31 | if self._options.connect_retries > 0 and self.connect_count == 0: 32 | break 33 | if self._options.connect_timeout > 0: 34 | self._event.wait(timeout = self._options.connect_timeout) 35 | continue 36 | 37 | try: 38 | self._recorder.open() 39 | while self._do_run(): 40 | self._recorder.run() 41 | except KiwiServerTerminatedConnection as e: 42 | if self._options.no_api: 43 | msg = '' 44 | else: 45 | msg = ' Reconnecting after 5 seconds' 46 | logging.info("%s:%s %s.%s" % (self._options.server_host, self._options.server_port, e, msg)) 47 | self._recorder.close() 48 | if self._options.no_api: ## don't retry 49 | break 50 | self._recorder._start_ts = None ## this makes the recorder open a new file on restart 51 | self._event.wait(timeout=5) 52 | continue 53 | except KiwiTooBusyError: 54 | logging.info("%s:%d too busy now. Reconnecting after 15 seconds" 55 | % (self._options.server_host, self._options.server_port)) 56 | if self._options.is_kiwi_tdoa: 57 | self._options.status = 2 58 | break 59 | self._event.wait(timeout=15) 60 | continue 61 | except KiwiTimeLimitError: 62 | break 63 | except Exception as e: 64 | if self._options.is_kiwi_tdoa: 65 | self._options.status = 1 66 | print_exc() 67 | break 68 | 69 | self._run_event.clear() # tell all other threads to stop 70 | self._recorder.close() 71 | -------------------------------------------------------------------------------- /kiwiclient/kiwi/wsclient.py: -------------------------------------------------------------------------------- 1 | """ 2 | Modified echo client from the pywebsocket examples 3 | """ 4 | 5 | import base64 6 | import logging 7 | import os 8 | import re 9 | import socket 10 | 11 | from mod_pywebsocket import common 12 | from mod_pywebsocket.extensions import DeflateFrameExtensionProcessor 13 | from mod_pywebsocket.extensions import PerMessageDeflateExtensionProcessor 14 | from mod_pywebsocket.extensions import _PerMessageDeflateFramer 15 | from mod_pywebsocket.extensions import _parse_window_bits 16 | from mod_pywebsocket import util 17 | 18 | 19 | _TIMEOUT_SEC = 10 20 | _UNDEFINED_PORT = -1 21 | 22 | _UPGRADE_HEADER = 'Upgrade: websocket\r\n' 23 | _UPGRADE_HEADER_HIXIE75 = 'Upgrade: WebSocket\r\n' 24 | _CONNECTION_HEADER = 'Connection: Upgrade\r\n' 25 | 26 | 27 | class ClientHandshakeError(Exception): 28 | pass 29 | 30 | 31 | def _build_method_line(resource): 32 | return ('GET %s HTTP/1.1\r\n' % resource).encode() 33 | 34 | 35 | def _origin_header(header, origin): 36 | # 4.1 13. concatenation of the string "Origin:", a U+0020 SPACE character, 37 | # and the /origin/ value, converted to ASCII lowercase, to /fields/. 38 | return '%s: %s\r\n' % (header, origin.lower()) 39 | 40 | 41 | def _format_host_header(host, port, secure): 42 | # 4.1 9. Let /hostport/ be an empty string. 43 | # 4.1 10. Append the /host/ value, converted to ASCII lowercase, to 44 | # /hostport/ 45 | hostport = host.lower() 46 | # 4.1 11. If /secure/ is false, and /port/ is not 80, or if /secure/ 47 | # is true, and /port/ is not 443, then append a U+003A COLON character 48 | # (:) followed by the value of /port/, expressed as a base-ten integer, 49 | # to /hostport/ 50 | if ((not secure and port != common.DEFAULT_WEB_SOCKET_PORT) or 51 | (secure and port != common.DEFAULT_WEB_SOCKET_SECURE_PORT)): 52 | hostport += ':' + str(port) 53 | # 4.1 12. concatenation of the string "Host:", a U+0020 SPACE 54 | # character, and /hostport/, to /fields/. 55 | return '%s: %s\r\n' % (common.HOST_HEADER, hostport) 56 | 57 | 58 | def _receive_bytes(socket, length): 59 | bytes = [] 60 | remaining = length 61 | while remaining > 0: 62 | received_bytes = socket.recv(remaining) 63 | if not received_bytes: 64 | raise IOError( 65 | 'Connection closed before receiving requested length ' 66 | '(requested %d bytes but received only %d bytes)' % 67 | (length, length - remaining)) 68 | bytes.append(received_bytes) 69 | remaining -= len(received_bytes) 70 | return bytearray().join(bytes).decode('utf-8') 71 | 72 | 73 | def _get_mandatory_header(fields, name): 74 | """Gets the value of the header specified by name from fields. 75 | 76 | This function expects that there's only one header with the specified name 77 | in fields. Otherwise, raises an ClientHandshakeError. 78 | """ 79 | 80 | values = fields.get(name.lower()) 81 | if values is None or len(values) == 0: 82 | raise ClientHandshakeError( 83 | '%s header not found: %r' % (name, values)) 84 | if len(values) > 1: 85 | raise ClientHandshakeError( 86 | 'Multiple %s headers found: %r' % (name, values)) 87 | return values[0] 88 | 89 | 90 | def _validate_mandatory_header(fields, name, 91 | expected_value, case_sensitive=False): 92 | """Gets and validates the value of the header specified by name from 93 | fields. 94 | 95 | If expected_value is specified, compares expected value and actual value 96 | and raises an ClientHandshakeError on failure. You can specify case 97 | sensitiveness in this comparison by case_sensitive parameter. This function 98 | expects that there's only one header with the specified name in fields. 99 | Otherwise, raises an ClientHandshakeError. 100 | """ 101 | 102 | value = _get_mandatory_header(fields, name) 103 | 104 | if ((case_sensitive and value != expected_value) or 105 | (not case_sensitive and value.lower() != expected_value.lower())): 106 | raise ClientHandshakeError( 107 | 'Illegal value for header %s: %r (expected) vs %r (actual)' % 108 | (name, expected_value, value)) 109 | 110 | 111 | class ClientHandshakeBase(object): 112 | """A base class for WebSocket opening handshake processors for each 113 | protocol version. 114 | """ 115 | 116 | def __init__(self): 117 | self._logger = util.get_class_logger(self) 118 | 119 | def _read_fields(self): 120 | # 4.1 32. let /fields/ be a list of name-value pairs, initially empty. 121 | fields = {} 122 | while True: # "Field" 123 | # 4.1 33. let /name/ and /value/ be empty byte arrays 124 | name = '' 125 | value = '' 126 | # 4.1 34. read /name/ 127 | name = self._read_name() 128 | if name is None: 129 | break 130 | # 4.1 35. read spaces 131 | # TODO(tyoshino): Skip only one space as described in the spec. 132 | ch = self._skip_spaces() 133 | # 4.1 36. read /value/ 134 | value = self._read_value(ch) 135 | # 4.1 37. read a byte from the server 136 | ch = _receive_bytes(self._socket, 1) 137 | if ch != '\n': # 0x0A 138 | raise ClientHandshakeError( 139 | 'Expected LF but found %r while reading value %r for ' 140 | 'header %r' % (ch, value, name)) 141 | self._logger.debug('Received %r header', name) 142 | # 4.1 38. append an entry to the /fields/ list that has the name 143 | # given by the string obtained by interpreting the /name/ byte 144 | # array as a UTF-8 stream and the value given by the string 145 | # obtained by interpreting the /value/ byte array as a UTF-8 byte 146 | # stream. 147 | fields.setdefault(name, []).append(value) 148 | # 4.1 39. return to the "Field" step above 149 | return fields 150 | 151 | def _read_name(self): 152 | # 4.1 33. let /name/ be empty byte arrays 153 | name = '' 154 | while True: 155 | # 4.1 34. read a byte from the server 156 | ch = _receive_bytes(self._socket, 1) 157 | if ch == '\r': # 0x0D 158 | return None 159 | elif ch == '\n': # 0x0A 160 | raise ClientHandshakeError( 161 | 'Unexpected LF when reading header name %r' % name) 162 | elif ch == ':': # 0x3A 163 | return name 164 | elif ch >= 'A' and ch <= 'Z': # Range 0x31 to 0x5A 165 | ch = chr(ord(ch) + 0x20) 166 | name += ch 167 | else: 168 | name += ch 169 | 170 | def _skip_spaces(self): 171 | # 4.1 35. read a byte from the server 172 | while True: 173 | ch = _receive_bytes(self._socket, 1) 174 | if ch == ' ': # 0x20 175 | continue 176 | return ch 177 | 178 | def _read_value(self, ch): 179 | # 4.1 33. let /value/ be empty byte arrays 180 | value = '' 181 | # 4.1 36. read a byte from server. 182 | while True: 183 | if ch == '\r': # 0x0D 184 | return value 185 | elif ch == '\n': # 0x0A 186 | raise ClientHandshakeError( 187 | 'Unexpected LF when reading header value %r' % value) 188 | else: 189 | value += ch 190 | ch = _receive_bytes(self._socket, 1) 191 | 192 | 193 | def _get_permessage_deflate_framer(extension_response): 194 | """Validate the response and return a framer object using the parameters in 195 | the response. This method doesn't accept the server_.* parameters. 196 | """ 197 | 198 | client_max_window_bits = None 199 | client_no_context_takeover = None 200 | 201 | client_max_window_bits_name = ( 202 | PerMessageDeflateExtensionProcessor. 203 | _CLIENT_MAX_WINDOW_BITS_PARAM) 204 | client_no_context_takeover_name = ( 205 | PerMessageDeflateExtensionProcessor. 206 | _CLIENT_NO_CONTEXT_TAKEOVER_PARAM) 207 | 208 | # We didn't send any server_.* parameter. 209 | # Handle those parameters as invalid if found in the response. 210 | 211 | for param_name, param_value in extension_response.get_parameters(): 212 | if param_name == client_max_window_bits_name: 213 | if client_max_window_bits is not None: 214 | raise ClientHandshakeError( 215 | 'Multiple %s found' % client_max_window_bits_name) 216 | 217 | parsed_value = _parse_window_bits(param_value) 218 | if parsed_value is None: 219 | raise ClientHandshakeError( 220 | 'Bad %s: %r' % 221 | (client_max_window_bits_name, param_value)) 222 | client_max_window_bits = parsed_value 223 | elif param_name == client_no_context_takeover_name: 224 | if client_no_context_takeover is not None: 225 | raise ClientHandshakeError( 226 | 'Multiple %s found' % client_no_context_takeover_name) 227 | 228 | if param_value is not None: 229 | raise ClientHandshakeError( 230 | 'Bad %s: Has value %r' % 231 | (client_no_context_takeover_name, param_value)) 232 | client_no_context_takeover = True 233 | 234 | if client_no_context_takeover is None: 235 | client_no_context_takeover = False 236 | 237 | return _PerMessageDeflateFramer(client_max_window_bits, 238 | client_no_context_takeover) 239 | 240 | 241 | class ClientHandshakeProcessor(ClientHandshakeBase): 242 | """WebSocket opening handshake processor for 243 | draft-ietf-hybi-thewebsocketprotocol-06 and later. 244 | """ 245 | 246 | def __init__(self, socket, host, port, origin=None, deflate_frame=False, use_permessage_deflate=False): 247 | super(ClientHandshakeProcessor, self).__init__() 248 | 249 | self._socket = socket 250 | self._host = host 251 | self._port = port 252 | self._origin = origin 253 | self._deflate_frame = deflate_frame 254 | self._use_permessage_deflate = use_permessage_deflate 255 | 256 | self._logger = util.get_class_logger(self) 257 | 258 | def handshake(self, resource): 259 | """Performs opening handshake on the specified socket. 260 | 261 | Raises: 262 | ClientHandshakeError: handshake failed. 263 | """ 264 | 265 | request_line = _build_method_line(resource) 266 | self._logger.debug('Client\'s opening handshake Request-Line: %r', request_line) 267 | 268 | fields = [] 269 | fields.append(_format_host_header(self._host, self._port, False)) 270 | fields.append(_UPGRADE_HEADER) 271 | fields.append(_CONNECTION_HEADER) 272 | if self._origin is not None: 273 | fields.append(_origin_header(common.ORIGIN_HEADER, self._origin)) 274 | 275 | original_key = os.urandom(16) 276 | self._key = base64.b64encode(original_key) 277 | self._logger.debug('%s: %r (%s)', common.SEC_WEBSOCKET_KEY_HEADER, self._key, util.hexify(original_key)) 278 | fields.append('%s: %s\r\n' % (common.SEC_WEBSOCKET_KEY_HEADER, self._key.decode())) 279 | fields.append('%s: %d\r\n' % (common.SEC_WEBSOCKET_VERSION_HEADER, common.VERSION_HYBI_LATEST)) 280 | extensions_to_request = [] 281 | 282 | if self._deflate_frame: 283 | extensions_to_request.append(common.ExtensionParameter(common.DEFLATE_FRAME_EXTENSION)) 284 | 285 | if self._use_permessage_deflate: 286 | extension = common.ExtensionParameter(common.PERMESSAGE_DEFLATE_EXTENSION) 287 | # Accept the client_max_window_bits extension parameter by default. 288 | extension.add_parameter(PerMessageDeflateExtensionProcessor._CLIENT_MAX_WINDOW_BITS_PARAM, None) 289 | extensions_to_request.append(extension) 290 | 291 | if len(extensions_to_request) != 0: 292 | fields.append('%s: %s\r\n' % (common.SEC_WEBSOCKET_EXTENSIONS_HEADER, common.format_extensions(extensions_to_request))) 293 | 294 | self._socket.sendall(request_line) 295 | for field in fields: 296 | self._socket.sendall(field.encode()) 297 | self._socket.sendall(b'\r\n') 298 | 299 | self._logger.debug('Sent client\'s opening handshake headers: %r', fields) 300 | self._logger.debug('Start reading Status-Line') 301 | 302 | status_line = '' 303 | while True: 304 | ch = _receive_bytes(self._socket, 1) 305 | status_line += ch 306 | if ch == '\n': 307 | break 308 | 309 | m = re.match('HTTP/\\d+\.\\d+ (\\d\\d\\d) .*\r\n', status_line) 310 | if m is None: 311 | raise ClientHandshakeError('Wrong status line format: %r' % status_line) 312 | status_code = m.group(1) 313 | if status_code != '101': 314 | self._logger.debug('Unexpected status code %s with following headers: %r', status_code, self._read_fields()) 315 | raise ClientHandshakeError('Expected HTTP status code 101 but found %r' % status_code) 316 | 317 | self._logger.debug('Received valid Status-Line') 318 | self._logger.debug('Start reading headers until we see an empty line') 319 | 320 | fields = self._read_fields() 321 | 322 | ch = _receive_bytes(self._socket, 1) 323 | if ch != '\n': # 0x0A 324 | raise ClientHandshakeError( 325 | 'Expected LF but found %r while reading value %r for header ' 326 | 'name %r' % (ch, value, name)) 327 | 328 | self._logger.debug('Received an empty line') 329 | self._logger.debug('Server\'s opening handshake headers: %r', fields) 330 | 331 | _validate_mandatory_header(fields, common.UPGRADE_HEADER, common.WEBSOCKET_UPGRADE_TYPE, False) 332 | _validate_mandatory_header(fields, common.CONNECTION_HEADER, common.UPGRADE_CONNECTION_TYPE, False) 333 | 334 | accept = _get_mandatory_header(fields, common.SEC_WEBSOCKET_ACCEPT_HEADER) 335 | # Validate 336 | try: 337 | binary_accept = base64.b64decode(accept) 338 | except TypeError as e: 339 | raise HandshakeError( 340 | 'Illegal value for header %s: %r' % 341 | (common.SEC_WEBSOCKET_ACCEPT_HEADER, accept)) 342 | 343 | if len(binary_accept) != 20: 344 | raise ClientHandshakeError( 345 | 'Decoded value of %s is not 20-byte long' % 346 | common.SEC_WEBSOCKET_ACCEPT_HEADER) 347 | 348 | self._logger.debug('Response for challenge : %r (%s)', accept, util.hexify(binary_accept)) 349 | 350 | binary_expected_accept = util.sha1_hash(self._key + common.WEBSOCKET_ACCEPT_UUID.encode()).digest() 351 | expected_accept = base64.b64encode(binary_expected_accept) 352 | self._logger.debug( 353 | 'Expected response for challenge: %r (%s)', 354 | expected_accept, util.hexify(binary_expected_accept)) 355 | 356 | if accept.encode() != expected_accept: 357 | raise ClientHandshakeError( 358 | 'Invalid %s header: %r (expected: %s)' % 359 | (common.SEC_WEBSOCKET_ACCEPT_HEADER, accept, expected_accept)) 360 | 361 | deflate_frame_accepted = False 362 | permessage_deflate_accepted = False 363 | 364 | extensions_header = fields.get(common.SEC_WEBSOCKET_EXTENSIONS_HEADER.lower()) 365 | accepted_extensions = [] 366 | if extensions_header is not None and len(extensions_header) != 0: 367 | accepted_extensions = common.parse_extensions(extensions_header[0]) 368 | 369 | # TODO(bashi): Support the new style perframe compression extension. 370 | for extension in accepted_extensions: 371 | extension_name = extension.name() 372 | if (extension_name == common.DEFLATE_FRAME_EXTENSION and self._deflate_frame): 373 | deflate_frame_accepted = True 374 | processor = DeflateFrameExtensionProcessor(extension) 375 | unused_extension_response = processor.get_extension_response() 376 | self._deflate_frame = processor 377 | continue 378 | elif (extension_name == common.PERMESSAGE_DEFLATE_EXTENSION and self._use_permessage_deflate): 379 | permessage_deflate_accepted = True 380 | framer = _get_permessage_deflate_framer(extension) 381 | framer.set_compress_outgoing_enabled(True) 382 | self._use_permessage_deflate = framer 383 | continue 384 | 385 | raise ClientHandshakeError('Unexpected extension %r' % extension_name) 386 | 387 | if (self._deflate_frame and not deflate_frame_accepted): 388 | raise ClientHandshakeError('Requested %s, but the server rejected it' % common.DEFLATE_FRAME_EXTENSION) 389 | if (self._use_permessage_deflate and not permessage_deflate_accepted): 390 | raise ClientHandshakeError('Requested %s, but the server rejected it' % common.PERMESSAGE_DEFLATE_EXTENSION) 391 | 392 | # TODO(tyoshino): Handle Sec-WebSocket-Protocol 393 | # TODO(tyoshino): Handle Cookie, etc. 394 | 395 | 396 | class ClientConnection(object): 397 | """A wrapper for socket object to provide the mp_conn interface. 398 | mod_pywebsocket library is designed to be working on Apache mod_python's 399 | mp_conn object. 400 | """ 401 | 402 | def __init__(self, socket): 403 | self._socket = socket 404 | 405 | def write(self, data): 406 | try: 407 | self._socket.sendall(data) 408 | except Exception as e: 409 | logging.debug('ClientConnection write error: "%s"' % e) 410 | 411 | def read(self, n): 412 | return self._socket.recv(n) 413 | 414 | def get_remote_addr(self): 415 | return self._socket.getpeername() 416 | remote_addr = property(get_remote_addr) 417 | 418 | 419 | class ClientRequest(object): 420 | """A wrapper class just to make it able to pass a socket object to 421 | functions that expect a mp_request object. 422 | """ 423 | 424 | def __init__(self, socket): 425 | self._logger = util.get_class_logger(self) 426 | 427 | self._socket = socket 428 | self.connection = ClientConnection(socket) 429 | 430 | 431 | -------------------------------------------------------------------------------- /kiwiclient/kiwirecorder.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | ## -*- python -*- 3 | 4 | import array, logging, os, struct, sys, time, copy, threading, os 5 | import gc 6 | import math 7 | import numpy as np 8 | from copy import copy 9 | from traceback import print_exc 10 | from kiwi import KiwiSDRStream, KiwiWorker 11 | from optparse import OptionParser 12 | from optparse import OptionGroup 13 | import sounddevice 14 | 15 | stream = sounddevice.OutputStream(12000, 2048, channels=1, dtype='int16') 16 | stream.start() 17 | 18 | HAS_RESAMPLER = True 19 | try: 20 | ## if available use libsamplerate for resampling 21 | from samplerate import Resampler 22 | except ImportError: 23 | ## otherwise linear interpolation is used 24 | HAS_RESAMPLER = False 25 | 26 | 27 | def _write_wav_header(fp, filesize, samplerate, num_channels, is_kiwi_wav): 28 | fp.write(struct.pack('<4sI4s', b'RIFF', filesize - 8, b'WAVE')) 29 | bits_per_sample = 16 30 | byte_rate = samplerate * num_channels * bits_per_sample // 8 31 | block_align = num_channels * bits_per_sample // 8 32 | fp.write(struct.pack('<4sIHHIIHH', b'fmt ', 16, 1, num_channels, int(samplerate+0.5), byte_rate, block_align, bits_per_sample)) 33 | if not is_kiwi_wav: 34 | fp.write(struct.pack('<4sI', b'data', filesize - 12 - 8 - 16 - 8)) 35 | 36 | class RingBuffer(object): 37 | def __init__(self, len): 38 | self._array = np.zeros(len, dtype='float64') 39 | self._index = 0 40 | self._is_filled = False 41 | 42 | def insert(self, sample): 43 | self._array[self._index] = sample 44 | self._index += 1 45 | if self._index == len(self._array): 46 | self._is_filled = True 47 | self._index = 0 48 | 49 | def is_filled(self): 50 | return self._is_filled 51 | 52 | def applyFn(self, fn): 53 | return fn(self._array) 54 | 55 | def max_abs(self): 56 | return np.max(np.abs(self._array)) 57 | 58 | class GNSSPerformance(object): 59 | def __init__(self): 60 | self._last_solution = -1 61 | self._last_ts = -1 62 | self._num_frames = 0 63 | self._buffer_dt_per_frame = RingBuffer(10) 64 | self._buffer_num_frames = RingBuffer(10) 65 | 66 | def analyze(self, filename, gps): 67 | ## gps = {'last_gps_solution': 1, 'dummy': 0, 'gpsnsec': 886417795, 'gpssec': 466823} 68 | self._num_frames += 1 69 | if gps['last_gps_solution'] == 0 and self._last_solution != 0: 70 | ts = gps['gpssec'] + 1e-9 * gps['gpsnsec'] 71 | msg_gnss_drift = '' 72 | if self._last_ts != -1: 73 | dt = ts - self._last_ts 74 | if dt < -12*3600*7: 75 | dt += 24*3600*7 76 | if abs(dt) < 10: 77 | self._buffer_dt_per_frame.insert(dt / self._num_frames) 78 | self._buffer_num_frames.insert(self._num_frames) 79 | if self._buffer_dt_per_frame.is_filled(): 80 | std_dt_per_frame = self._buffer_dt_per_frame.applyFn(np.std) 81 | mean_num_frames = self._buffer_num_frames.applyFn(np.mean) 82 | msg_gnss_drift = 'std(clk drift)= %5.1f m' % (3e8 * std_dt_per_frame * mean_num_frames) 83 | 84 | logging.info('%s: (%2d,%3d) t_gnss= %16.9f %s' 85 | % (filename, self._last_solution, self._num_frames, ts, msg_gnss_drift)) 86 | self._num_frames = 0 87 | self._last_ts = ts 88 | 89 | self._last_solution = gps['last_gps_solution'] 90 | 91 | 92 | class Squelch(object): 93 | def __init__(self, options): 94 | self._status_msg = not options.quiet 95 | self._threshold = options.thresh 96 | self._squelch_tail = options.squelch_tail ## in seconds 97 | self._ring_buffer = RingBuffer(65) 98 | self._squelch_on_seq = None 99 | self.set_sample_rate(12000.0) ## default setting 100 | 101 | def set_sample_rate(self, fs): 102 | self._tail_delay = round(self._squelch_tail*fs/512) ## seconds to number of buffers 103 | 104 | def process(self, seq, rssi): 105 | if not self._ring_buffer.is_filled() or self._squelch_on_seq is None: 106 | self._ring_buffer.insert(rssi) 107 | if not self._ring_buffer.is_filled(): 108 | return False 109 | median_nf = self._ring_buffer.applyFn(np.median) 110 | rssi_thresh = median_nf + self._threshold 111 | is_open = self._squelch_on_seq is not None 112 | if is_open: 113 | rssi_thresh -= 6 114 | rssi_green = rssi >= rssi_thresh 115 | if rssi_green: 116 | self._squelch_on_seq = seq 117 | is_open = True 118 | if self._status_msg: 119 | sys.stdout.write('\r Median: %6.1f Thr: %6.1f %s' % (median_nf, rssi_thresh, ("s", "S")[is_open])) 120 | sys.stdout.flush() 121 | if not is_open: 122 | return False 123 | if seq > self._squelch_on_seq + self._tail_delay: 124 | logging.info("\nSquelch closed") 125 | self._squelch_on_seq = None 126 | return False 127 | return is_open 128 | 129 | class KiwiSoundRecorder(KiwiSDRStream): 130 | def __init__(self, options): 131 | super(KiwiSoundRecorder, self).__init__() 132 | self._options = options 133 | self._type = 'SND' 134 | freq = options.frequency 135 | #logging.info("%s:%s freq=%d" % (options.server_host, options.server_port, freq)) 136 | self._freq = freq 137 | self._start_ts = None 138 | self._start_time = None 139 | self._squelch = Squelch(self._options) if options.thresh is not None else None 140 | self._num_channels = 2 if options.modulation == 'iq' else 1 141 | self._last_gps = dict(zip(['last_gps_solution', 'dummy', 'gpssec', 'gpsnsec'], [0,0,0,0])) 142 | self._resampler = None 143 | self._gnss_performance = GNSSPerformance() 144 | 145 | def _setup_rx_params(self): 146 | if self._options.no_api: 147 | if self._options.user != 'kiwirecorder.py': 148 | self.set_name(self._options.user) 149 | return 150 | self.set_name(self._options.user) 151 | mod = self._options.modulation 152 | lp_cut = self._options.lp_cut 153 | hp_cut = self._options.hp_cut 154 | if mod == 'am': 155 | # For AM, ignore the low pass filter cutoff 156 | lp_cut = -hp_cut if hp_cut is not None else hp_cut 157 | self.set_mod(mod, lp_cut, hp_cut, self._freq) 158 | if self._options.agc_gain is not None: 159 | self.set_agc(on=False, gain=self._options.agc_gain[0], hang=self._options.agc_gain[1], 160 | thresh=self._options.agc_gain[2], slope=self._options.agc_gain[3], 161 | decay=self._options.agc_gain[4]) 162 | else: 163 | self.set_agc(on=True) 164 | if self._options.compression is False: 165 | self._set_snd_comp(False) 166 | if self._options.nb is True: 167 | gate = self._options.nb_gate 168 | if gate < 100 or gate > 5000: 169 | gate = 100 170 | thresh = self._options.nb_thresh 171 | if thresh < 0 or thresh > 100: 172 | thresh = 50 173 | self.set_noise_blanker(gate, thresh) 174 | self._output_sample_rate = self._sample_rate 175 | if self._squelch: 176 | self._squelch.set_sample_rate(self._sample_rate) 177 | if self._options.resample > 0: 178 | self._output_sample_rate = self._options.resample 179 | self._ratio = float(self._output_sample_rate)/self._sample_rate 180 | logging.info('resampling from %g to %d Hz (ratio=%f)' % (self._sample_rate, self._options.resample, self._ratio)) 181 | if not HAS_RESAMPLER: 182 | logging.info("libsamplerate not available: linear interpolation is used for low-quality resampling. " 183 | "(pip install samplerate)") 184 | 185 | def _process_audio_samples(self, seq, samples, rssi): 186 | if self._options.quiet is False: 187 | sys.stdout.write('\rBlock: %08x, RSSI: %6.1f' % (seq, rssi)) 188 | sys.stdout.flush() 189 | 190 | if self._squelch: 191 | is_open = self._squelch.process(seq, rssi) 192 | if not is_open: 193 | self._start_ts = None 194 | self._start_time = None 195 | return 196 | 197 | if self._options.resample > 0: 198 | if HAS_RESAMPLER: 199 | ## libsamplerate resampling 200 | if self._resampler is None: 201 | self._resampler = Resampler(converter_type='sinc_best') 202 | samples = np.round(self._resampler.process(samples, ratio=self._ratio)).astype(np.int16) 203 | else: 204 | ## resampling by linear interpolation 205 | n = len(samples) 206 | xa = np.arange(round(n*self._ratio))/self._ratio 207 | xp = np.arange(n) 208 | samples = np.round(np.interp(xa,xp,samples)).astype(np.int16) 209 | 210 | self._write_samples(samples, {}) 211 | 212 | def _process_iq_samples(self, seq, samples, rssi, gps): 213 | if self._squelch: 214 | is_open = self._squelch.process(seq, rssi) 215 | if not is_open: 216 | self._start_ts = None 217 | self._start_time = None 218 | return 219 | 220 | ##print gps['gpsnsec']-self._last_gps['gpsnsec'] 221 | self._last_gps = gps 222 | ## convert list of complex numbers into an array 223 | s = np.zeros(2*len(samples), dtype=np.int16) 224 | s[0::2] = np.real(samples).astype(np.int16) 225 | s[1::2] = np.imag(samples).astype(np.int16) 226 | 227 | if self._options.resample > 0: 228 | if HAS_RESAMPLER: 229 | ## libsamplerate resampling 230 | if self._resampler is None: 231 | self._resampler = Resampler(channels=2, converter_type='sinc_best') 232 | s = self._resampler.process(s.reshape(len(samples),2), ratio=self._ratio) 233 | s = np.round(s.reshape(1, np.size(s))).astype(np.int16) 234 | else: 235 | ## resampling by linear interpolation 236 | n = len(samples) 237 | m = int(round(n*self._ratio)) 238 | xa = np.arange(m)/self._ratio 239 | xp = np.arange(n) 240 | s = np.zeros(2*m, dtype=np.int16) 241 | s[0::2] = np.round(np.interp(xa,xp,np.real(samples))).astype(np.int16) 242 | s[1::2] = np.round(np.interp(xa,xp,np.imag(samples))).astype(np.int16) 243 | 244 | self._write_samples(s, gps) 245 | 246 | # no GPS or no recent GPS solution 247 | last = gps['last_gps_solution'] 248 | if last == 255 or last == 254: 249 | self._options.status = 3 250 | 251 | def _get_output_filename(self): 252 | if self._options.test_mode: 253 | return os.devnull 254 | station = '' if self._options.station is None else '_'+ self._options.station 255 | 256 | # if multiple connections specified but not distinguished via --station then use index 257 | if self._options.multiple_connections and self._options.station is None: 258 | station = '_%d' % self._options.idx 259 | if self._options.filename != '': 260 | filename = '%s%s.wav' % (self._options.filename, station) 261 | else: 262 | ts = time.strftime('%Y%m%dT%H%M%SZ', self._start_ts) 263 | filename = '%s_%d%s_%s.wav' % (ts, int(self._freq * 1000), station, self._options.modulation) 264 | if self._options.dir is not None: 265 | filename = '%s/%s' % (self._options.dir, filename) 266 | return filename 267 | 268 | def _update_wav_header(self): 269 | with open(self._get_output_filename(), 'r+b') as fp: 270 | fp.seek(0, os.SEEK_END) 271 | filesize = fp.tell() 272 | fp.seek(0, os.SEEK_SET) 273 | 274 | # fp.tell() sometimes returns zero. _write_wav_header writes filesize - 8 275 | if filesize >= 8: 276 | _write_wav_header(fp, filesize, int(self._output_sample_rate), self._num_channels, self._options.is_kiwi_wav) 277 | 278 | def _write_samples(self, samples, *args): 279 | """Output to a file on the disk OR give me sound ! """ 280 | if self._options.audio: 281 | stream.write(np.array(samples, dtype=np.int16)) 282 | else: 283 | now = time.gmtime() 284 | sec_of_day = lambda x: 3600*x.tm_hour + 60*x.tm_min + x.tm_sec 285 | dt_reached = self._options.dt != 0 and self._start_ts is not None and sec_of_day(now)//self._options.dt != sec_of_day(self._start_ts)//self._options.dt 286 | if self._start_ts is None or (self._options.filename == '' and dt_reached): 287 | self._start_ts = now 288 | self._start_time = time.time() 289 | # Write a static WAV header 290 | with open(self._get_output_filename(), 'wb') as fp: 291 | _write_wav_header(fp, 100, int(self._output_sample_rate), self._num_channels, self._options.is_kiwi_wav) 292 | if self._options.is_kiwi_tdoa: 293 | # NB: MUST be a print (i.e. not a logging.info) 294 | print("file=%d %s" % (self._options.idx, self._get_output_filename())) 295 | else: 296 | logging.info("Started a new file: %s" % self._get_output_filename()) 297 | with open(self._get_output_filename(), 'ab') as fp: 298 | if self._options.is_kiwi_wav: 299 | gps = args[0] 300 | self._gnss_performance.analyze(self._get_output_filename(), gps) 301 | fp.write(struct.pack('<4sIBBII', b'kiwi', 10, gps['last_gps_solution'], 0, gps['gpssec'], gps['gpsnsec'])) 302 | sample_size = samples.itemsize * len(samples) 303 | fp.write(struct.pack('<4sI', b'data', sample_size)) 304 | # TODO: something better than that 305 | samples.tofile(fp) 306 | self._update_wav_header() 307 | 308 | def _on_gnss_position(self, pos): 309 | pos_record = False 310 | if self._options.dir is not None: 311 | pos_dir = self._options.dir 312 | pos_record = True 313 | else: 314 | if os.path.isdir('gnss_pos'): 315 | pos_dir = 'gnss_pos' 316 | pos_record = True 317 | if pos_record: 318 | station = 'kiwi_noname' if self._options.station is None else self._options.station 319 | pos_filename = pos_dir +'/'+ station + '.txt' 320 | with open(pos_filename, 'w') as f: 321 | station = station.replace('-', '_') # since Octave var name 322 | f.write("d.%s = struct('coord', [%f,%f], 'host', '%s', 'port', %d);\n" 323 | % (station, 324 | pos[0], pos[1], 325 | self._options.server_host, 326 | self._options.server_port)) 327 | 328 | class KiwiWaterfallRecorder(KiwiSDRStream): 329 | def __init__(self, options): 330 | super(KiwiWaterfallRecorder, self).__init__() 331 | self._options = options 332 | self._type = 'W/F' 333 | freq = options.frequency 334 | #logging.info "%s:%s freq=%d" % (options.server_host, options.server_port, freq) 335 | self._freq = freq 336 | self._start_ts = None 337 | self._start_time = None 338 | 339 | self._num_channels = 2 if options.modulation == 'iq' else 1 340 | self._last_gps = dict(zip(['last_gps_solution', 'dummy', 'gpssec', 'gpsnsec'], [0,0,0,0])) 341 | 342 | def _setup_rx_params(self): 343 | self._set_zoom_cf(self._options.zoom, self._freq) 344 | self._set_maxdb_mindb(-10, -110) # needed, but values don't matter 345 | #self._set_wf_comp(True) 346 | self._set_wf_comp(False) 347 | self._set_wf_speed(1) # 1 Hz update 348 | self.set_name(self._options.user) 349 | self._start_time = time.time() 350 | 351 | def _process_waterfall_samples(self, seq, samples): 352 | nbins = len(samples) 353 | bins = nbins-1 354 | max = -1 355 | min = 256 356 | bmax = bmin = 0 357 | i = 0 358 | for s in samples: 359 | if s > max: 360 | max = s 361 | bmax = i 362 | if s < min: 363 | min = s 364 | bmin = i 365 | i += 1 366 | span = 30000 367 | logging.info("wf samples %d bins %d..%d dB %.1f..%.1f kHz rbw %d kHz" 368 | % (nbins, min-255, max-255, span*bmin/bins, span*bmax/bins, span/bins)) 369 | 370 | def options_cross_product(options): 371 | """build a list of options according to the number of servers specified""" 372 | def _sel_entry(i, l): 373 | """if l is a list, return the element with index i, else return l""" 374 | return l[min(i, len(l)-1)] if type(l) == list else l 375 | 376 | l = [] 377 | multiple_connections = 0 378 | for i,s in enumerate(options.server_host): 379 | opt_single = copy(options) 380 | opt_single.server_host = s 381 | opt_single.status = 0 382 | 383 | # time() returns seconds, so add pid and host index to make timestamp unique per connection 384 | opt_single.timestamp = int(time.time() + os.getpid() + i) & 0xffffffff 385 | for x in ['server_port', 'password', 'tlimit_password', 'frequency', 'filename', 'station', 'user']: 386 | opt_single.__dict__[x] = _sel_entry(i, opt_single.__dict__[x]) 387 | l.append(opt_single) 388 | multiple_connections = i 389 | return multiple_connections,l 390 | 391 | def get_comma_separated_args(option, opt, value, parser, fn): 392 | values = [fn(v.strip()) for v in value.split(',')] 393 | setattr(parser.values, option.dest, values) 394 | ## setattr(parser.values, option.dest, map(fn, value.split(','))) 395 | 396 | def join_threads(snd, wf): 397 | [r._event.set() for r in snd] 398 | [r._event.set() for r in wf] 399 | [t.join() for t in threading.enumerate() if t is not threading.currentThread()] 400 | 401 | def main(): 402 | parser = OptionParser() 403 | parser.add_option('-s', '--server-host', 404 | dest='server_host', type='string', 405 | default='localhost', help='Server host (can be a comma-delimited list)', 406 | action='callback', 407 | callback_args=(str,), 408 | callback=get_comma_separated_args) 409 | parser.add_option('-p', '--server-port', 410 | dest='server_port', type='string', 411 | default=8073, help='Server port, default 8073 (can be a comma delimited list)', 412 | action='callback', 413 | callback_args=(int,), 414 | callback=get_comma_separated_args) 415 | parser.add_option('--pw', '--password', 416 | dest='password', type='string', default='', 417 | help='Kiwi login password (if required, can be a comma delimited list)', 418 | action='callback', 419 | callback_args=(str,), 420 | callback=get_comma_separated_args) 421 | parser.add_option('--tlimit-pw', '--tlimit-password', 422 | dest='tlimit_password', type='string', default='', 423 | help='Connect time limit exemption password (if required, can be a comma delimited list)', 424 | action='callback', 425 | callback_args=(str,), 426 | callback=get_comma_separated_args) 427 | parser.add_option('-u', '--user', 428 | dest='user', type='string', default='kiwirecorder.py', 429 | help='Kiwi connection user name', 430 | action='callback', 431 | callback_args=(str,), 432 | callback=get_comma_separated_args) 433 | parser.add_option('--station', 434 | dest='station', 435 | type='string', default=None, 436 | help='Station ID to be appended to filename (can be a comma-separated list)', 437 | action='callback', 438 | callback_args=(str,), 439 | callback=get_comma_separated_args) 440 | parser.add_option('--log', '--log-level', '--log_level', type='choice', 441 | dest='log_level', default='warn', 442 | choices=['debug', 'info', 'warn', 'error', 'critical'], 443 | help='Log level: debug|info|warn(default)|error|critical') 444 | parser.add_option('-q', '--quiet', 445 | dest='quiet', 446 | default=False, 447 | action='store_true', 448 | help='Don\'t print progress messages') 449 | parser.add_option('-d', '--dir', 450 | dest='dir', 451 | type='string', default=None, 452 | help='Optional destination directory for files') 453 | parser.add_option('--fn', '--filename', 454 | dest='filename', 455 | type='string', default='', 456 | help='Use fixed filename instead of generated filenames (optional station ID(s) will apply)', 457 | action='callback', 458 | callback_args=(str,), 459 | callback=get_comma_separated_args) 460 | parser.add_option('--tlimit', '--time-limit', 461 | dest='tlimit', 462 | type='float', default=None, 463 | help='Record time limit in seconds. Ignored when --dt-sec used.') 464 | parser.add_option('--dt-sec', 465 | dest='dt', 466 | type='int', default=0, 467 | help='Start a new file when mod(sec_of_day,dt) == 0') 468 | parser.add_option('--launch-delay', '--launch_delay', 469 | dest='launch_delay', 470 | type='int', default=0, 471 | help='Delay (secs) in launching multiple connections') 472 | parser.add_option('--connect-retries', '--connect_retries', 473 | dest='connect_retries', type='int', default=0, 474 | help='Number of retries when connecting to host (retries forever by default)') 475 | parser.add_option('--connect-timeout', '--connect_timeout', 476 | dest='connect_timeout', type='int', default=15, 477 | help='Retry timeout(sec) connecting to host') 478 | parser.add_option('-k', '--socket-timeout', '--socket_timeout', 479 | dest='socket_timeout', type='int', default=10, 480 | help='Socket timeout(sec) during data transfers') 481 | parser.add_option('--OV', 482 | dest='ADC_OV', 483 | default=False, 484 | action='store_true', 485 | help='Print "ADC OV" message when Kiwi ADC is overloaded') 486 | parser.add_option('--ts', '--tstamp', '--timestamp', 487 | dest='tstamp', 488 | default=False, 489 | action='store_true', 490 | help='Add timestamps to output. Applies only to S-meter mode currently.') 491 | parser.add_option('--stats', 492 | dest='stats', 493 | default=False, 494 | action='store_true', 495 | help='Print additional statistics. Applies only to S-meter mode currently.') 496 | parser.add_option('--no-api', 497 | dest='no_api', 498 | default=False, 499 | action='store_true', 500 | help='Simulate connection to Kiwi using improper/incomplete API') 501 | 502 | group = OptionGroup(parser, "Audio connection options", "") 503 | group.add_option('-f', '--freq', 504 | dest='frequency', 505 | type='string', default=1000, 506 | help='Frequency to tune to, in kHz (can be a comma-separated list)', 507 | action='callback', 508 | callback_args=(float,), 509 | callback=get_comma_separated_args) 510 | group.add_option('-m', '--modulation', 511 | dest='modulation', 512 | type='string', default='am', 513 | help='Modulation; one of am, lsb, usb, cw, nbfm, iq (default passband if -L/-H not specified)') 514 | group.add_option('--ncomp', '--no_compression', 515 | dest='compression', 516 | default=True, 517 | action='store_false', 518 | help='Don\'t use audio compression') 519 | group.add_option('-L', '--lp-cutoff', 520 | dest='lp_cut', 521 | type='float', default=None, 522 | help='Low-pass cutoff frequency, in Hz') 523 | group.add_option('-H', '--hp-cutoff', 524 | dest='hp_cut', 525 | type='float', default=None, 526 | help='High-pass cutoff frequency, in Hz') 527 | group.add_option('-r', '--resample', 528 | dest='resample', 529 | type='int', default=0, 530 | help='Resample output file to new sample rate in Hz. The resampling ratio has to be in the range [1/256,256]') 531 | group.add_option('-T', '--squelch-threshold', 532 | dest='thresh', 533 | type='float', default=None, 534 | help='Squelch threshold, in dB.') 535 | group.add_option('--squelch-tail', 536 | dest='squelch_tail', 537 | type='float', default=1, 538 | help='Time for which the squelch remains open after the signal is below threshold.') 539 | group.add_option('-g', '--agc-gain', 540 | dest='agc_gain', 541 | type='string', 542 | default=None, 543 | help='AGC gain; if set, AGC is turned off (can be a comma-separated list)', 544 | action='callback', 545 | callback_args=(float,), 546 | callback=get_comma_separated_args) 547 | group.add_option('--nb', 548 | dest='nb', 549 | action='store_true', default=False, 550 | help='Enable noise blanker with default parameters.') 551 | group.add_option('--nb-gate', 552 | dest='nb_gate', 553 | type='int', default=100, 554 | help='Noise blanker gate time in usec (100 to 5000, default 100)') 555 | group.add_option('--nb-th', '--nb-thresh', 556 | dest='nb_thresh', 557 | type='int', default=50, 558 | help='Noise blanker threshold in percent (0 to 100, default 50)') 559 | group.add_option('-w', '--kiwi-wav', 560 | dest='is_kiwi_wav', 561 | default=False, 562 | action='store_true', 563 | help='In the wav file include KIWI header containing GPS time-stamps (only for IQ mode)') 564 | group.add_option('--kiwi-tdoa', 565 | dest='is_kiwi_tdoa', 566 | default=False, 567 | action='store_true', 568 | help='Used when called by Kiwi TDoA extension') 569 | group.add_option('--test-mode', 570 | dest='test_mode', 571 | default=False, 572 | action='store_true', 573 | help='Write wav data to /dev/null (Linux) or NUL (Windows)') 574 | group.add_option('--snd', '--sound', 575 | dest='sound', 576 | default=False, 577 | action='store_true', 578 | help='Also process sound data when in waterfall or S-meter mode (sound connection options above apply)') 579 | group.add_option('-a', '--audio', 580 | dest='audio', 581 | default=False, 582 | action='store_true', 583 | help='Get audio output instead of writing to disk (mod by linkz)') 584 | parser.add_option_group(group) 585 | 586 | group = OptionGroup(parser, "S-meter mode options", "") 587 | group.add_option('--S-meter', '--s-meter', 588 | dest='S_meter', 589 | type='int', default=-1, 590 | help='Report S-meter (RSSI) value after S_METER number of averages. S_METER=0 does no averaging and reports each RSSI value received. Options --ts and --stats apply.') 591 | parser.add_option('--sdt-sec', 592 | dest='sdt', 593 | type='int', default=0, 594 | help='S-meter measurement interval') 595 | parser.add_option_group(group) 596 | 597 | group = OptionGroup(parser, "Waterfall connection options", "") 598 | group.add_option('--wf', 599 | dest='waterfall', 600 | default=False, 601 | action='store_true', 602 | help='Process waterfall data instead of audio') 603 | group.add_option('-z', '--zoom', 604 | dest='zoom', type='int', default=0, 605 | help='Zoom level 0-14') 606 | parser.add_option_group(group) 607 | 608 | (options, unused_args) = parser.parse_args() 609 | 610 | ## clean up OptionParser which has cyclic references 611 | parser.destroy() 612 | 613 | FORMAT = '%(asctime)-15s pid %(process)5d %(message)s' 614 | logging.basicConfig(level=logging.getLevelName(options.log_level.upper()), format=FORMAT) 615 | if options.log_level.upper() == 'DEBUG': 616 | gc.set_debug(gc.DEBUG_SAVEALL | gc.DEBUG_LEAK | gc.DEBUG_UNCOLLECTABLE) 617 | 618 | run_event = threading.Event() 619 | run_event.set() 620 | 621 | if options.S_meter >= 0: 622 | if options.S_meter > 0 and options.sdt != 0: 623 | raise Exception('Options --S-meter > 0 and --sdt-sec != 0 are incompatible. Did you mean to use --S-meter=0 ?') 624 | options.quiet = True 625 | 626 | if options.tlimit is not None and options.dt != 0: 627 | print('Warning: --tlimit ignored when --dt-sec option used') 628 | 629 | options.raw = False 630 | gopt = options 631 | multiple_connections,options = options_cross_product(options) 632 | 633 | snd_recorders = [] 634 | if not gopt.waterfall or (gopt.waterfall and gopt.sound): 635 | for i,opt in enumerate(options): 636 | opt.multiple_connections = multiple_connections 637 | opt.idx = i 638 | snd_recorders.append(KiwiWorker(args=(KiwiSoundRecorder(opt),opt,run_event))) 639 | 640 | wf_recorders = [] 641 | if gopt.waterfall: 642 | for i,opt in enumerate(options): 643 | opt.multiple_connections = multiple_connections 644 | opt.idx = i 645 | wf_recorders.append(KiwiWorker(args=(KiwiWaterfallRecorder(opt),opt,run_event))) 646 | 647 | try: 648 | for i,r in enumerate(snd_recorders): 649 | if opt.launch_delay != 0 and i != 0 and options[i-1].server_host == options[i].server_host: 650 | time.sleep(opt.launch_delay) 651 | r.start() 652 | #logging.info("started sound recorder %d, timestamp=%d" % (i, options[i].timestamp)) 653 | logging.info("started sound recorder %d" % i) 654 | 655 | for i,r in enumerate(wf_recorders): 656 | if i!=0 and options[i-1].server_host == options[i].server_host: 657 | time.sleep(opt.launch_delay) 658 | r.start() 659 | logging.info("started waterfall recorder %d" % i) 660 | 661 | while run_event.is_set(): 662 | time.sleep(.1) 663 | 664 | except KeyboardInterrupt: 665 | run_event.clear() 666 | join_threads(snd_recorders, wf_recorders) 667 | print("KeyboardInterrupt: threads successfully closed") 668 | except Exception as e: 669 | print_exc() 670 | run_event.clear() 671 | join_threads(snd_recorders, wf_recorders) 672 | print("Exception: threads successfully closed") 673 | 674 | if gopt.is_kiwi_tdoa: 675 | for i,opt in enumerate(options): 676 | # NB: MUST be a print (i.e. not a logging.info) 677 | print("status=%d,%d" % (i, opt.status)) 678 | 679 | logging.debug('gc %s' % gc.garbage) 680 | 681 | 682 | if __name__ == '__main__': 683 | #import faulthandler 684 | #faulthandler.enable() 685 | main() 686 | # EOF 687 | -------------------------------------------------------------------------------- /kiwiclient/microkiwi_waterfall.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | """ Here is the IS0KYB microkiwi_waterfall script, modified to use python instead of jupyther notebook """ 4 | 5 | # python 2/3 compatibility 6 | from __future__ import print_function 7 | from __future__ import division 8 | 9 | from optparse import OptionParser 10 | 11 | import io 12 | import socket 13 | import struct 14 | import time 15 | import numpy as np 16 | import matplotlib.pyplot as plt 17 | from matplotlib.colors import LinearSegmentedColormap 18 | 19 | from kiwi import wsclient 20 | 21 | import mod_pywebsocket.common 22 | from mod_pywebsocket.stream import Stream 23 | from mod_pywebsocket.stream import StreamOptions 24 | 25 | 26 | processfailed = False 27 | 28 | cdict1 = { 29 | 'red': ((0.0, 0.0, 0.0), 30 | (0.2, 0.0, 0.0), 31 | (0.4, 1.0, 1.0), 32 | (0.6, 1.0, 1.0), 33 | (0.8, 1.0, 1.0), 34 | (1.0, 1.0, 1.0)), 35 | 36 | 'green': ((0.0, 0.0, 0.0), 37 | (0.2, 0.0, 0.0), 38 | (0.4, 1.0, 1.0), 39 | (0.6, 0.0, 0.0), 40 | (0.8, 0.0, 0.0), 41 | (1.0, 0.764, 0.764)), 42 | 43 | 'blue': ((0.0, 0.0, 0.0), 44 | (0.2, 1.0, 1.0), 45 | (0.4, 0.0, 0.0), 46 | (0.6, 0.0, 0.0), 47 | (0.8, 1.0, 1.0), 48 | (1.0, 1.0, 1.0)), 49 | } 50 | 51 | cmap = LinearSegmentedColormap('SAColorMap', cdict1, 1024) 52 | 53 | parser = OptionParser() 54 | parser.add_option("-s", "--server", type=str, 55 | help="server name", dest="server", default='192.168.1.82') 56 | parser.add_option("-p", "--port", type=int, 57 | help="port number", dest="port", default=8073) 58 | parser.add_option("-l", "--length", type=int, 59 | help="how many samples to draw from the server", dest="length", default=200) 60 | parser.add_option("-z", "--zoom", type=int, 61 | help="zoom factor", dest="zoom", default=0) 62 | parser.add_option("-o", "--offset", type=int, 63 | help="start frequency in kHz", dest="start", default=0) 64 | parser.add_option("-v", "--verbose", type=int, 65 | help="whether to print progress and debug info", dest="verbosity", default=0) 66 | 67 | options = vars(parser.parse_args()[0]) 68 | host = options['server'] 69 | port = options['port'] 70 | 71 | # the default number of bins is 1024 72 | bins = 1024 73 | zoom = options['zoom'] 74 | offset_khz = options['start'] # this is offset in kHz 75 | full_span = 30000 # for a 30MHz kiwiSDR 76 | if zoom > 0: 77 | span = full_span // 2.**zoom 78 | else: 79 | span = full_span 80 | rbw = span//bins 81 | if offset_khz > 0: 82 | offset = (offset_khz+100)//(full_span//bins)*2**4*1000. 83 | offset = max(0, offset) 84 | else: 85 | offset = 0 86 | center_freq = span//2+offset_khz 87 | now = b'(datetime.now()' 88 | header = [center_freq, span, now] 89 | header_bin = struct.pack("II26s", *header) 90 | 91 | try: 92 | mysocket = socket.socket() 93 | mysocket.connect((host, port)) 94 | except: 95 | print("Failed to connect") 96 | exit() 97 | 98 | uri = '/%d/%s' % (int(time.time()), 'W/F') 99 | handshake = wsclient.ClientHandshakeProcessor(mysocket, host, port) 100 | handshake.handshake(uri) 101 | request = wsclient.ClientRequest(mysocket) 102 | request.ws_version = mod_pywebsocket.common.VERSION_HYBI13 103 | stream_option = StreamOptions() 104 | stream_option.mask_send = True 105 | stream_option.unmask_receive = False 106 | mystream = Stream(request, stream_option) 107 | 108 | # send a sequence of messages to the server, hardcoded for now 109 | # max wf speed, no compression 110 | msg_list = ['SET auth t=kiwi p=', 'SET zoom=%d start=%d' % (zoom, offset), 111 | 'SET maxdb=0 mindb=-100', 'SET wf_speed=4', 'SET wf_comp=0'] 112 | for msg in msg_list: 113 | mystream.send_message(msg) 114 | # number of samples to draw from server 115 | length = options['length'] 116 | # create a numpy array to contain the waterfall data 117 | wf_data = np.zeros((length, bins)) 118 | binary_wf_list = [] 119 | time = 0 120 | 121 | while time < length: 122 | # receive one msg from server 123 | try: 124 | tmp = mystream.receive_message() 125 | except: 126 | processfailed = True 127 | break 128 | if b'W/F' in tmp: # this is one waterfall line 129 | tmp = tmp[16:].replace(b"7", b"\xa0") # remove some header from each msg & bug fix for blocked freq ranges 130 | print("received sample") 131 | if options['verbosity']: 132 | print(time), 133 | spectrum = np.array(struct.unpack('%dB' % len(tmp), tmp)) # convert from binary data to uint8 134 | # spectrum = np.ndarray(len(tmp), dtype='B', buffer=tmp) # convert from binary data to uint8 135 | binary_wf_list.append(tmp) # append binary data to be saved to file 136 | # wf_data[time, :] = spectrum-255 # mirror dBs 137 | wf_data[time, :] = spectrum 138 | wf_data[time, :] = -(255 - wf_data[time, :]) # dBm 139 | wf_data[time, :] = wf_data[time, :] - 13 # typical Kiwi wf cal 140 | time += 1 141 | else: # this is chatter between client and server 142 | pass 143 | 144 | try: 145 | mystream.close_connection(mod_pywebsocket.common.STATUS_GOING_AWAY) 146 | mysocket.close() 147 | except Exception as e: 148 | print("exception: %s" % e) 149 | 150 | avg_wf = np.mean(wf_data, axis=0) # average over time 151 | p95 = np.percentile(avg_wf, 95) 152 | median = np.percentile(avg_wf, 50) 153 | maxsig = np.max(wf_data) 154 | minsig = np.min(wf_data) 155 | 156 | # print "Waterfall with %d bins: median= %f dB, p95= %f dB - SNR= %f rbw= %f kHz" % (bins, median, p95,p95-median, rbw) 157 | print("SNR: %i dB [median: %i dB, p95: %i dB, high: %i dBm, low: %i dBm]" % (p95 - median, median, p95, maxsig, minsig)) 158 | 159 | fd = io.BytesIO() # no more file to write on disk 160 | fd.write(header_bin) # write the header info at the top 161 | for line in binary_wf_list: 162 | fd.write(line) 163 | 164 | fd.seek(0) 165 | buff = fd.read() 166 | header_len = 8 + 26 # 2 unsigned int for center_freq and span: 8 bytes PLUS 26 bytes for datetime 167 | length = len(buff[header_len:]) 168 | n_t = length // bins 169 | header = struct.unpack('2I26s', buff[:header_len]) 170 | data = struct.unpack('%dB' % length, buff[header_len:]) 171 | waterfall_array = np.reshape(np.array(data[:]), (n_t, bins)) 172 | waterfall_array -= 255 173 | plt.figure(figsize=(14, 5)) 174 | plt.yticks(np.arange(0, 0, step=1)) 175 | plt.xlabel('MHz') 176 | plt.xticks(np.linspace(0, bins, 31), np.linspace(0, 30, num=31, endpoint=True, dtype=int)) 177 | plt.pcolormesh(waterfall_array[:, 1:], cmap=cmap, vmin=minsig+30, vmax=maxsig+30) 178 | if processfailed: 179 | plt.title("Sorry, measurement failed on this Kiwi, no slot available, try later") 180 | else: 181 | plt.title("HF waterfall @ " + str(host) + " - [SNR: %i dB" % (p95 - median) + "]") 182 | clb = plt.colorbar() 183 | clb.ax.set_title('dBm') 184 | plt.show() 185 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright 2011, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """WebSocket extension for Apache HTTP Server. 32 | 33 | mod_pywebsocket is a WebSocket extension for Apache HTTP Server 34 | intended for testing or experimental purposes. mod_python is required. 35 | 36 | 37 | Installation 38 | ============ 39 | 40 | 0. Prepare an Apache HTTP Server for which mod_python is enabled. 41 | 42 | 1. Specify the following Apache HTTP Server directives to suit your 43 | configuration. 44 | 45 | If mod_pywebsocket is not in the Python path, specify the following. 46 | is the directory where mod_pywebsocket is installed. 47 | 48 | PythonPath "sys.path+['']" 49 | 50 | Always specify the following. is the directory where 51 | user-written WebSocket handlers are placed. 52 | 53 | PythonOption mod_pywebsocket.handler_root 54 | PythonHeaderParserHandler mod_pywebsocket.headerparserhandler 55 | 56 | To limit the search for WebSocket handlers to a directory 57 | under , configure as follows: 58 | 59 | PythonOption mod_pywebsocket.handler_scan 60 | 61 | is useful in saving scan time when 62 | contains many non-WebSocket handler files. 63 | 64 | If you want to allow handlers whose canonical path is not under the root 65 | directory (i.e. symbolic link is in root directory but its target is not), 66 | configure as follows: 67 | 68 | PythonOption mod_pywebsocket.allow_handlers_outside_root_dir On 69 | 70 | Example snippet of httpd.conf: 71 | (mod_pywebsocket is in /websock_lib, WebSocket handlers are in 72 | /websock_handlers, port is 80 for ws, 443 for wss.) 73 | 74 | 75 | PythonPath "sys.path+['/websock_lib']" 76 | PythonOption mod_pywebsocket.handler_root /websock_handlers 77 | PythonHeaderParserHandler mod_pywebsocket.headerparserhandler 78 | 79 | 80 | 2. Tune Apache parameters for serving WebSocket. We'd like to note that at 81 | least TimeOut directive from core features and RequestReadTimeout 82 | directive from mod_reqtimeout should be modified not to kill connections 83 | in only a few seconds of idle time. 84 | 85 | 3. Verify installation. You can use example/console.html to poke the server. 86 | 87 | 88 | Writing WebSocket handlers 89 | ========================== 90 | 91 | When a WebSocket request comes in, the resource name 92 | specified in the handshake is considered as if it is a file path under 93 | and the handler defined in 94 | /_wsh.py is invoked. 95 | 96 | For example, if the resource name is /example/chat, the handler defined in 97 | /example/chat_wsh.py is invoked. 98 | 99 | A WebSocket handler is composed of the following three functions: 100 | 101 | web_socket_do_extra_handshake(request) 102 | web_socket_transfer_data(request) 103 | web_socket_passive_closing_handshake(request) 104 | 105 | where: 106 | request: mod_python request. 107 | 108 | web_socket_do_extra_handshake is called during the handshake after the 109 | headers are successfully parsed and WebSocket properties (ws_location, 110 | ws_origin, and ws_resource) are added to request. A handler 111 | can reject the request by raising an exception. 112 | 113 | A request object has the following properties that you can use during the 114 | extra handshake (web_socket_do_extra_handshake): 115 | - ws_resource 116 | - ws_origin 117 | - ws_version 118 | - ws_location (HyBi 00 only) 119 | - ws_extensions (HyBi 06 and later) 120 | - ws_deflate (HyBi 06 and later) 121 | - ws_protocol 122 | - ws_requested_protocols (HyBi 06 and later) 123 | 124 | The last two are a bit tricky. See the next subsection. 125 | 126 | 127 | Subprotocol Negotiation 128 | ----------------------- 129 | 130 | For HyBi 06 and later, ws_protocol is always set to None when 131 | web_socket_do_extra_handshake is called. If ws_requested_protocols is not 132 | None, you must choose one subprotocol from this list and set it to 133 | ws_protocol. 134 | 135 | For HyBi 00, when web_socket_do_extra_handshake is called, 136 | ws_protocol is set to the value given by the client in 137 | Sec-WebSocket-Protocol header or None if 138 | such header was not found in the opening handshake request. Finish extra 139 | handshake with ws_protocol untouched to accept the request subprotocol. 140 | Then, Sec-WebSocket-Protocol header will be sent to 141 | the client in response with the same value as requested. Raise an exception 142 | in web_socket_do_extra_handshake to reject the requested subprotocol. 143 | 144 | 145 | Data Transfer 146 | ------------- 147 | 148 | web_socket_transfer_data is called after the handshake completed 149 | successfully. A handler can receive/send messages from/to the client 150 | using request. mod_pywebsocket.msgutil module provides utilities 151 | for data transfer. 152 | 153 | You can receive a message by the following statement. 154 | 155 | message = request.ws_stream.receive_message() 156 | 157 | This call blocks until any complete text frame arrives, and the payload data 158 | of the incoming frame will be stored into message. When you're using IETF 159 | HyBi 00 or later protocol, receive_message() will return None on receiving 160 | client-initiated closing handshake. When any error occurs, receive_message() 161 | will raise some exception. 162 | 163 | You can send a message by the following statement. 164 | 165 | request.ws_stream.send_message(message) 166 | 167 | 168 | Closing Connection 169 | ------------------ 170 | 171 | Executing the following statement or just return-ing from 172 | web_socket_transfer_data cause connection close. 173 | 174 | request.ws_stream.close_connection() 175 | 176 | close_connection will wait 177 | for closing handshake acknowledgement coming from the client. When it 178 | couldn't receive a valid acknowledgement, raises an exception. 179 | 180 | web_socket_passive_closing_handshake is called after the server receives 181 | incoming closing frame from the client peer immediately. You can specify 182 | code and reason by return values. They are sent as a outgoing closing frame 183 | from the server. A request object has the following properties that you can 184 | use in web_socket_passive_closing_handshake. 185 | - ws_close_code 186 | - ws_close_reason 187 | 188 | 189 | Threading 190 | --------- 191 | 192 | A WebSocket handler must be thread-safe if the server (Apache or 193 | standalone.py) is configured to use threads. 194 | 195 | 196 | Configuring WebSocket Extension Processors 197 | ------------------------------------------ 198 | 199 | See extensions.py for supported WebSocket extensions. Note that they are 200 | unstable and their APIs are subject to change substantially. 201 | 202 | A request object has these extension processing related attributes. 203 | 204 | - ws_requested_extensions: 205 | 206 | A list of common.ExtensionParameter instances representing extension 207 | parameters received from the client in the client's opening handshake. 208 | You shouldn't modify it manually. 209 | 210 | - ws_extensions: 211 | 212 | A list of common.ExtensionParameter instances representing extension 213 | parameters to send back to the client in the server's opening handshake. 214 | You shouldn't touch it directly. Instead, call methods on extension 215 | processors. 216 | 217 | - ws_extension_processors: 218 | 219 | A list of loaded extension processors. Find the processor for the 220 | extension you want to configure from it, and call its methods. 221 | """ 222 | 223 | 224 | # vi:sts=4 sw=4 et tw=72 225 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/_stream_base.py: -------------------------------------------------------------------------------- 1 | # Copyright 2011, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """Base stream class. 32 | """ 33 | 34 | 35 | # Note: request.connection.write/read are used in this module, even though 36 | # mod_python document says that they should be used only in connection 37 | # handlers. Unfortunately, we have no other options. For example, 38 | # request.write/read are not suitable because they don't allow direct raw bytes 39 | # writing/reading. 40 | 41 | 42 | import socket 43 | 44 | from mod_pywebsocket import util 45 | 46 | 47 | # Exceptions 48 | 49 | 50 | class ConnectionTerminatedException(Exception): 51 | """This exception will be raised when a connection is terminated 52 | unexpectedly. 53 | """ 54 | 55 | pass 56 | 57 | 58 | class InvalidFrameException(ConnectionTerminatedException): 59 | """This exception will be raised when we received an invalid frame we 60 | cannot parse. 61 | """ 62 | 63 | pass 64 | 65 | 66 | class BadOperationException(Exception): 67 | """This exception will be raised when send_message() is called on 68 | server-terminated connection or receive_message() is called on 69 | client-terminated connection. 70 | """ 71 | 72 | pass 73 | 74 | 75 | class UnsupportedFrameException(Exception): 76 | """This exception will be raised when we receive a frame with flag, opcode 77 | we cannot handle. Handlers can just catch and ignore this exception and 78 | call receive_message() again to continue processing the next frame. 79 | """ 80 | 81 | pass 82 | 83 | 84 | class InvalidUTF8Exception(Exception): 85 | """This exception will be raised when we receive a text frame which 86 | contains invalid UTF-8 strings. 87 | """ 88 | 89 | pass 90 | 91 | 92 | class StreamBase(object): 93 | """Base stream class.""" 94 | 95 | def __init__(self, request): 96 | """Construct an instance. 97 | 98 | Args: 99 | request: mod_python request. 100 | """ 101 | 102 | self._logger = util.get_class_logger(self) 103 | 104 | self._request = request 105 | 106 | def _read(self, length): 107 | """Reads length bytes from connection. In case we catch any exception, 108 | prepends remote address to the exception message and raise again. 109 | 110 | Raises: 111 | ConnectionTerminatedException: when read returns empty string. 112 | """ 113 | 114 | try: 115 | read_bytes = self._request.connection.read(length) 116 | if not read_bytes: 117 | raise ConnectionTerminatedException( 118 | 'Receiving %d byte failed. Peer (%r) closed connection' % 119 | (length, (self._request.connection.remote_addr,))) 120 | return read_bytes 121 | except socket.error as e: 122 | # Catch a socket.error. Because it's not a child class of the 123 | # IOError prior to Python 2.6, we cannot omit this except clause. 124 | # Use %s rather than %r for the exception to use human friendly 125 | # format. 126 | raise ConnectionTerminatedException( 127 | 'Receiving %d byte failed. socket.error (%s) occurred' % 128 | (length, e)) 129 | except IOError as e: 130 | # Also catch an IOError because mod_python throws it. 131 | raise ConnectionTerminatedException( 132 | 'Receiving %d byte failed. IOError (%s) occurred' % 133 | (length, e)) 134 | 135 | def _write(self, bytes_to_write): 136 | """Writes given bytes to connection. In case we catch any exception, 137 | prepends remote address to the exception message and raise again. 138 | """ 139 | 140 | try: 141 | self._request.connection.write(bytes_to_write) 142 | except Exception as e: 143 | util.prepend_message_to_exception( 144 | 'Failed to send message to %r: ' % 145 | (self._request.connection.remote_addr,), 146 | e) 147 | raise 148 | 149 | def receive_bytes(self, length): 150 | """Receives multiple bytes. Retries read when we couldn't receive the 151 | specified amount. 152 | 153 | Raises: 154 | ConnectionTerminatedException: when read returns empty string. 155 | """ 156 | 157 | read_bytes = [] 158 | while length > 0: 159 | new_read_bytes = self._read(length) 160 | read_bytes.append(new_read_bytes) 161 | length -= len(new_read_bytes) 162 | if length!=0 and type(read_bytes[0]) == str: 163 | return ''.join(read_bytes) 164 | else: 165 | return bytearray().join(read_bytes) 166 | 167 | def _read_until(self, delim_char): 168 | """Reads bytes until we encounter delim_char. The result will not 169 | contain delim_char. 170 | 171 | Raises: 172 | ConnectionTerminatedException: when read returns empty string. 173 | """ 174 | 175 | read_bytes = [] 176 | while True: 177 | ch = self._read(1) 178 | if ch == delim_char: 179 | break 180 | read_bytes.append(ch) 181 | return ''.join(read_bytes) 182 | 183 | 184 | # vi:sts=4 sw=4 et 185 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/_stream_hixie75.py: -------------------------------------------------------------------------------- 1 | # Copyright 2011, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """This file provides a class for parsing/building frames of the WebSocket 32 | protocol version HyBi 00 and Hixie 75. 33 | 34 | Specification: 35 | - HyBi 00 http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-00 36 | - Hixie 75 http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-75 37 | """ 38 | 39 | 40 | from mod_pywebsocket import common 41 | from mod_pywebsocket._stream_base import BadOperationException 42 | from mod_pywebsocket._stream_base import ConnectionTerminatedException 43 | from mod_pywebsocket._stream_base import InvalidFrameException 44 | from mod_pywebsocket._stream_base import StreamBase 45 | from mod_pywebsocket._stream_base import UnsupportedFrameException 46 | from mod_pywebsocket import util 47 | 48 | 49 | class StreamHixie75(StreamBase): 50 | """A class for parsing/building frames of the WebSocket protocol version 51 | HyBi 00 and Hixie 75. 52 | """ 53 | 54 | def __init__(self, request, enable_closing_handshake=False): 55 | """Construct an instance. 56 | 57 | Args: 58 | request: mod_python request. 59 | enable_closing_handshake: to let StreamHixie75 perform closing 60 | handshake as specified in HyBi 00, set 61 | this option to True. 62 | """ 63 | 64 | StreamBase.__init__(self, request) 65 | 66 | self._logger = util.get_class_logger(self) 67 | 68 | self._enable_closing_handshake = enable_closing_handshake 69 | 70 | self._request.client_terminated = False 71 | self._request.server_terminated = False 72 | 73 | def send_message(self, message, end=True, binary=False): 74 | """Send message. 75 | 76 | Args: 77 | message: unicode string to send. 78 | binary: not used in hixie75. 79 | 80 | Raises: 81 | BadOperationException: when called on a server-terminated 82 | connection. 83 | """ 84 | 85 | if not end: 86 | raise BadOperationException( 87 | 'StreamHixie75 doesn\'t support send_message with end=False') 88 | 89 | if binary: 90 | raise BadOperationException( 91 | 'StreamHixie75 doesn\'t support send_message with binary=True') 92 | 93 | if self._request.server_terminated: 94 | raise BadOperationException( 95 | 'Requested send_message after sending out a closing handshake') 96 | 97 | self._write(''.join(['\x00', message.encode('utf-8'), '\xff'])) 98 | 99 | def _read_payload_length_hixie75(self): 100 | """Reads a length header in a Hixie75 version frame with length. 101 | 102 | Raises: 103 | ConnectionTerminatedException: when read returns empty string. 104 | """ 105 | 106 | length = 0 107 | while True: 108 | b_str = self._read(1) 109 | b = ord(b_str) 110 | length = length * 128 + (b & 0x7f) 111 | if (b & 0x80) == 0: 112 | break 113 | return length 114 | 115 | def receive_message(self): 116 | """Receive a WebSocket frame and return its payload an unicode string. 117 | 118 | Returns: 119 | payload unicode string in a WebSocket frame. 120 | 121 | Raises: 122 | ConnectionTerminatedException: when read returns empty 123 | string. 124 | BadOperationException: when called on a client-terminated 125 | connection. 126 | """ 127 | 128 | if self._request.client_terminated: 129 | raise BadOperationException( 130 | 'Requested receive_message after receiving a closing ' 131 | 'handshake') 132 | 133 | while True: 134 | # Read 1 byte. 135 | # mp_conn.read will block if no bytes are available. 136 | # Timeout is controlled by TimeOut directive of Apache. 137 | frame_type_str = self.receive_bytes(1) 138 | frame_type = ord(frame_type_str) 139 | if (frame_type & 0x80) == 0x80: 140 | # The payload length is specified in the frame. 141 | # Read and discard. 142 | length = self._read_payload_length_hixie75() 143 | if length > 0: 144 | _ = self.receive_bytes(length) 145 | # 5.3 3. 12. if /type/ is 0xFF and /length/ is 0, then set the 146 | # /client terminated/ flag and abort these steps. 147 | if not self._enable_closing_handshake: 148 | continue 149 | 150 | if frame_type == 0xFF and length == 0: 151 | self._request.client_terminated = True 152 | 153 | if self._request.server_terminated: 154 | self._logger.debug( 155 | 'Received ack for server-initiated closing ' 156 | 'handshake') 157 | return None 158 | 159 | self._logger.debug( 160 | 'Received client-initiated closing handshake') 161 | 162 | self._send_closing_handshake() 163 | self._logger.debug( 164 | 'Sent ack for client-initiated closing handshake') 165 | return None 166 | else: 167 | # The payload is delimited with \xff. 168 | bytes = self._read_until('\xff') 169 | # The WebSocket protocol section 4.4 specifies that invalid 170 | # characters must be replaced with U+fffd REPLACEMENT 171 | # CHARACTER. 172 | message = bytes.decode('utf-8', 'replace') 173 | if frame_type == 0x00: 174 | return message 175 | # Discard data of other types. 176 | 177 | def _send_closing_handshake(self): 178 | if not self._enable_closing_handshake: 179 | raise BadOperationException( 180 | 'Closing handshake is not supported in Hixie 75 protocol') 181 | 182 | self._request.server_terminated = True 183 | 184 | # 5.3 the server may decide to terminate the WebSocket connection by 185 | # running through the following steps: 186 | # 1. send a 0xFF byte and a 0x00 byte to the client to indicate the 187 | # start of the closing handshake. 188 | self._write('\xff\x00') 189 | 190 | def close_connection(self, unused_code='', unused_reason=''): 191 | """Closes a WebSocket connection. 192 | 193 | Raises: 194 | ConnectionTerminatedException: when closing handshake was 195 | not successfull. 196 | """ 197 | 198 | if self._request.server_terminated: 199 | self._logger.debug( 200 | 'Requested close_connection but server is already terminated') 201 | return 202 | 203 | if not self._enable_closing_handshake: 204 | self._request.server_terminated = True 205 | self._logger.debug('Connection closed') 206 | return 207 | 208 | self._send_closing_handshake() 209 | self._logger.debug('Sent server-initiated closing handshake') 210 | 211 | # TODO(ukai): 2. wait until the /client terminated/ flag has been set, 212 | # or until a server-defined timeout expires. 213 | # 214 | # For now, we expect receiving closing handshake right after sending 215 | # out closing handshake, and if we couldn't receive non-handshake 216 | # frame, we take it as ConnectionTerminatedException. 217 | message = self.receive_message() 218 | if message is not None: 219 | raise ConnectionTerminatedException( 220 | 'Didn\'t receive valid ack for closing handshake') 221 | # TODO: 3. close the WebSocket connection. 222 | # note: mod_python Connection (mp_conn) doesn't have close method. 223 | 224 | def send_ping(self, body): 225 | raise BadOperationException( 226 | 'StreamHixie75 doesn\'t support send_ping') 227 | 228 | 229 | # vi:sts=4 sw=4 et 230 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/_stream_hybi.py: -------------------------------------------------------------------------------- 1 | # Copyright 2012, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """This file provides classes and helper functions for parsing/building frames 32 | of the WebSocket protocol (RFC 6455). 33 | 34 | Specification: 35 | http://tools.ietf.org/html/rfc6455 36 | """ 37 | 38 | 39 | from collections import deque 40 | import logging 41 | import os 42 | import struct 43 | import time 44 | 45 | import sys 46 | if sys.version_info > (3,): 47 | buffer = memoryview 48 | 49 | from mod_pywebsocket import common 50 | from mod_pywebsocket import util 51 | from mod_pywebsocket._stream_base import BadOperationException 52 | from mod_pywebsocket._stream_base import ConnectionTerminatedException 53 | from mod_pywebsocket._stream_base import InvalidFrameException 54 | from mod_pywebsocket._stream_base import InvalidUTF8Exception 55 | from mod_pywebsocket._stream_base import StreamBase 56 | from mod_pywebsocket._stream_base import UnsupportedFrameException 57 | 58 | 59 | _NOOP_MASKER = util.NoopMasker() 60 | 61 | 62 | class Frame(object): 63 | 64 | def __init__(self, fin=1, rsv1=0, rsv2=0, rsv3=0, 65 | opcode=None, payload=''): 66 | self.fin = fin 67 | self.rsv1 = rsv1 68 | self.rsv2 = rsv2 69 | self.rsv3 = rsv3 70 | self.opcode = opcode 71 | self.payload = payload 72 | 73 | 74 | # Helper functions made public to be used for writing unittests for WebSocket 75 | # clients. 76 | 77 | 78 | def create_length_header(length, mask): 79 | """Creates a length header. 80 | 81 | Args: 82 | length: Frame length. Must be less than 2^63. 83 | mask: Mask bit. Must be boolean. 84 | 85 | Raises: 86 | ValueError: when bad data is given. 87 | """ 88 | 89 | if mask: 90 | mask_bit = 1 << 7 91 | else: 92 | mask_bit = 0 93 | 94 | if length < 0: 95 | raise ValueError('length must be non negative integer') 96 | elif length <= 125: 97 | return struct.pack('B', mask_bit | length) 98 | elif length < (1 << 16): 99 | return struct.pack('B', mask_bit | 126) + struct.pack('!H', length) 100 | elif length < (1 << 63): 101 | return struct.pack('B', mask_bit | 127) + struct.pack('!Q', length) 102 | else: 103 | raise ValueError('Payload is too big for one frame') 104 | 105 | 106 | def create_header(opcode, payload_length, fin, rsv1, rsv2, rsv3, mask): 107 | """Creates a frame header. 108 | 109 | Raises: 110 | Exception: when bad data is given. 111 | """ 112 | 113 | if opcode < 0 or 0xf < opcode: 114 | raise ValueError('Opcode out of range') 115 | 116 | if payload_length < 0 or (1 << 63) <= payload_length: 117 | raise ValueError('payload_length out of range') 118 | 119 | if (fin | rsv1 | rsv2 | rsv3) & ~1: 120 | raise ValueError('FIN bit and Reserved bit parameter must be 0 or 1') 121 | 122 | header = bytearray() 123 | 124 | first_byte = ((fin << 7) 125 | | (rsv1 << 6) | (rsv2 << 5) | (rsv3 << 4) 126 | | opcode) 127 | header += struct.pack('B', first_byte) 128 | header += create_length_header(payload_length, mask) 129 | 130 | return header 131 | 132 | 133 | def _build_frame(header, body, mask): 134 | if not mask: 135 | return header + body 136 | 137 | masking_nonce = os.urandom(4) 138 | masker = util.RepeatedXorMasker(masking_nonce) 139 | return header + masking_nonce + masker.mask(body) 140 | 141 | 142 | def _filter_and_format_frame_object(frame, mask, frame_filters): 143 | for frame_filter in frame_filters: 144 | frame_filter.filter(frame) 145 | 146 | header = create_header( 147 | frame.opcode, len(frame.payload), frame.fin, 148 | frame.rsv1, frame.rsv2, frame.rsv3, mask) 149 | return _build_frame(header, frame.payload, mask) 150 | 151 | 152 | def create_binary_frame( 153 | message, opcode=common.OPCODE_BINARY, fin=1, mask=False, frame_filters=[]): 154 | """Creates a simple binary frame with no extension, reserved bit.""" 155 | 156 | frame = Frame(fin=fin, opcode=opcode, payload=message) 157 | return _filter_and_format_frame_object(frame, mask, frame_filters) 158 | 159 | 160 | def create_text_frame( 161 | message, opcode=common.OPCODE_TEXT, fin=1, mask=False, frame_filters=[]): 162 | """Creates a simple text frame with no extension, reserved bit.""" 163 | 164 | encoded_message = message.encode('utf-8') 165 | return create_binary_frame(encoded_message, opcode, fin, mask, 166 | frame_filters) 167 | 168 | 169 | def parse_frame(receive_bytes, logger=None, 170 | ws_version=common.VERSION_HYBI_LATEST, 171 | unmask_receive=True): 172 | """Parses a frame. Returns a tuple containing each header field and 173 | payload. 174 | 175 | Args: 176 | receive_bytes: a function that reads frame data from a stream or 177 | something similar. The function takes length of the bytes to be 178 | read. The function must raise ConnectionTerminatedException if 179 | there is not enough data to be read. 180 | logger: a logging object. 181 | ws_version: the version of WebSocket protocol. 182 | unmask_receive: unmask received frames. When received unmasked 183 | frame, raises InvalidFrameException. 184 | 185 | Raises: 186 | ConnectionTerminatedException: when receive_bytes raises it. 187 | InvalidFrameException: when the frame contains invalid data. 188 | """ 189 | if not logger: 190 | logger = logging.getLogger() 191 | 192 | logger.log(common.LOGLEVEL_FINE, 'Receive the first 2 octets of a frame') 193 | 194 | received = receive_bytes(2) 195 | if type(received[0]) is int: 196 | received = [x for x in received] 197 | else: 198 | received = map(ord, received); 199 | 200 | first_byte = (received[0]) 201 | fin = (first_byte >> 7) & 1 202 | rsv1 = (first_byte >> 6) & 1 203 | rsv2 = (first_byte >> 5) & 1 204 | rsv3 = (first_byte >> 4) & 1 205 | opcode = first_byte & 0xf 206 | 207 | second_byte = (received[1]) 208 | mask = (second_byte >> 7) & 1 209 | payload_length = second_byte & 0x7f 210 | 211 | logger.log(common.LOGLEVEL_FINE, 212 | 'FIN=%s, RSV1=%s, RSV2=%s, RSV3=%s, opcode=%s, ' 213 | 'Mask=%s, Payload_length=%s', 214 | fin, rsv1, rsv2, rsv3, opcode, mask, payload_length) 215 | 216 | if (mask == 1) != unmask_receive: 217 | raise InvalidFrameException( 218 | 'Mask bit on the received frame did\'nt match masking ' 219 | 'configuration for received frames') 220 | 221 | # The HyBi and later specs disallow putting a value in 0x0-0xFFFF 222 | # into the 8-octet extended payload length field (or 0x0-0xFD in 223 | # 2-octet field). 224 | valid_length_encoding = True 225 | length_encoding_bytes = 1 226 | if payload_length == 127: 227 | logger.log(common.LOGLEVEL_FINE, 228 | 'Receive 8-octet extended payload length') 229 | 230 | extended_payload_length = receive_bytes(8) 231 | payload_length = struct.unpack( 232 | '!Q', buffer(extended_payload_length))[0] 233 | if payload_length > 0x7FFFFFFFFFFFFFFF: 234 | raise InvalidFrameException( 235 | 'Extended payload length >= 2^63') 236 | if ws_version >= 13 and payload_length < 0x10000: 237 | valid_length_encoding = False 238 | length_encoding_bytes = 8 239 | 240 | logger.log(common.LOGLEVEL_FINE, 241 | 'Decoded_payload_length=%s', payload_length) 242 | elif payload_length == 126: 243 | logger.log(common.LOGLEVEL_FINE, 244 | 'Receive 2-octet extended payload length') 245 | 246 | extended_payload_length = receive_bytes(2) 247 | payload_length = struct.unpack( 248 | '!H', buffer(extended_payload_length))[0] 249 | if ws_version >= 13 and payload_length < 126: 250 | valid_length_encoding = False 251 | length_encoding_bytes = 2 252 | 253 | logger.log(common.LOGLEVEL_FINE, 254 | 'Decoded_payload_length=%s', payload_length) 255 | 256 | if not valid_length_encoding: 257 | logger.warning( 258 | 'Payload length is not encoded using the minimal number of ' 259 | 'bytes (%d is encoded using %d bytes)', 260 | payload_length, 261 | length_encoding_bytes) 262 | 263 | if mask == 1: 264 | logger.log(common.LOGLEVEL_FINE, 'Receive mask') 265 | 266 | masking_nonce = receive_bytes(4) 267 | masker = util.RepeatedXorMasker(masking_nonce) 268 | 269 | logger.log(common.LOGLEVEL_FINE, 'Mask=%r', masking_nonce) 270 | else: 271 | masker = _NOOP_MASKER 272 | 273 | logger.log(common.LOGLEVEL_FINE, 'Receive payload data') 274 | if logger.isEnabledFor(common.LOGLEVEL_FINE): 275 | receive_start = time.time() 276 | 277 | raw_payload_bytes = receive_bytes(payload_length) 278 | 279 | if logger.isEnabledFor(common.LOGLEVEL_FINE): 280 | logger.log( 281 | common.LOGLEVEL_FINE, 282 | 'Done receiving payload data at %s MB/s', 283 | payload_length / (time.time() - receive_start) / 1000 / 1000) 284 | logger.log(common.LOGLEVEL_FINE, 'Unmask payload data') 285 | 286 | if logger.isEnabledFor(common.LOGLEVEL_FINE): 287 | unmask_start = time.time() 288 | 289 | unmasked_bytes = masker.mask(raw_payload_bytes) 290 | 291 | if logger.isEnabledFor(common.LOGLEVEL_FINE): 292 | logger.log( 293 | common.LOGLEVEL_FINE, 294 | 'Done unmasking payload data at %s MB/s', 295 | payload_length / (time.time() - unmask_start) / 1000 / 1000) 296 | 297 | return opcode, unmasked_bytes, fin, rsv1, rsv2, rsv3 298 | 299 | 300 | class FragmentedFrameBuilder(object): 301 | """A stateful class to send a message as fragments.""" 302 | 303 | def __init__(self, mask, frame_filters=[], encode_utf8=True): 304 | """Constructs an instance.""" 305 | 306 | self._mask = mask 307 | self._frame_filters = frame_filters 308 | # This is for skipping UTF-8 encoding when building text type frames 309 | # from compressed data. 310 | self._encode_utf8 = encode_utf8 311 | 312 | self._started = False 313 | 314 | # Hold opcode of the first frame in messages to verify types of other 315 | # frames in the message are all the same. 316 | self._opcode = common.OPCODE_TEXT 317 | 318 | def build(self, payload_data, end, binary): 319 | if binary: 320 | frame_type = common.OPCODE_BINARY 321 | else: 322 | frame_type = common.OPCODE_TEXT 323 | if self._started: 324 | if self._opcode != frame_type: 325 | raise ValueError('Message types are different in frames for ' 326 | 'the same message') 327 | opcode = common.OPCODE_CONTINUATION 328 | else: 329 | opcode = frame_type 330 | self._opcode = frame_type 331 | 332 | if end: 333 | self._started = False 334 | fin = 1 335 | else: 336 | self._started = True 337 | fin = 0 338 | 339 | if binary or not self._encode_utf8: 340 | return create_binary_frame( 341 | payload_data, opcode, fin, self._mask, self._frame_filters) 342 | else: 343 | return create_text_frame( 344 | payload_data, opcode, fin, self._mask, self._frame_filters) 345 | 346 | 347 | def _create_control_frame(opcode, body, mask, frame_filters): 348 | frame = Frame(opcode=opcode, payload=body) 349 | 350 | for frame_filter in frame_filters: 351 | frame_filter.filter(frame) 352 | 353 | if len(frame.payload) > 125: 354 | raise BadOperationException( 355 | 'Payload data size of control frames must be 125 bytes or less') 356 | 357 | header = create_header( 358 | frame.opcode, len(frame.payload), frame.fin, 359 | frame.rsv1, frame.rsv2, frame.rsv3, mask) 360 | return _build_frame(header, frame.payload, mask) 361 | 362 | 363 | def create_ping_frame(body, mask=False, frame_filters=[]): 364 | return _create_control_frame(common.OPCODE_PING, body, mask, frame_filters) 365 | 366 | 367 | def create_pong_frame(body, mask=False, frame_filters=[]): 368 | return _create_control_frame(common.OPCODE_PONG, body, mask, frame_filters) 369 | 370 | 371 | def create_close_frame(body, mask=False, frame_filters=[]): 372 | return _create_control_frame( 373 | common.OPCODE_CLOSE, body, mask, frame_filters) 374 | 375 | 376 | def create_closing_handshake_body(code, reason): 377 | body = '' 378 | if code is not None: 379 | if (code > common.STATUS_USER_PRIVATE_MAX or 380 | code < common.STATUS_NORMAL_CLOSURE): 381 | raise BadOperationException('Status code is out of range') 382 | if (code == common.STATUS_NO_STATUS_RECEIVED or 383 | code == common.STATUS_ABNORMAL_CLOSURE or 384 | code == common.STATUS_TLS_HANDSHAKE): 385 | raise BadOperationException('Status code is reserved pseudo ' 386 | 'code') 387 | encoded_reason = reason.encode('utf-8') 388 | body = struct.pack('!H', code) + encoded_reason 389 | return body 390 | 391 | 392 | class StreamOptions(object): 393 | """Holds option values to configure Stream objects.""" 394 | 395 | def __init__(self): 396 | """Constructs StreamOptions.""" 397 | 398 | # Filters applied to frames. 399 | self.outgoing_frame_filters = [] 400 | self.incoming_frame_filters = [] 401 | 402 | # Filters applied to messages. Control frames are not affected by them. 403 | self.outgoing_message_filters = [] 404 | self.incoming_message_filters = [] 405 | 406 | self.encode_text_message_to_utf8 = True 407 | self.mask_send = False 408 | self.unmask_receive = True 409 | 410 | 411 | class Stream(StreamBase): 412 | """A class for parsing/building frames of the WebSocket protocol 413 | (RFC 6455). 414 | """ 415 | 416 | def __init__(self, request, options): 417 | """Constructs an instance. 418 | 419 | Args: 420 | request: mod_python request. 421 | """ 422 | 423 | StreamBase.__init__(self, request) 424 | 425 | self._logger = util.get_class_logger(self) 426 | 427 | self._options = options 428 | 429 | self._request.client_terminated = False 430 | self._request.server_terminated = False 431 | 432 | # Holds body of received fragments. 433 | self._received_fragments = [] 434 | # Holds the opcode of the first fragment. 435 | self._original_opcode = None 436 | 437 | self._writer = FragmentedFrameBuilder( 438 | self._options.mask_send, self._options.outgoing_frame_filters, 439 | self._options.encode_text_message_to_utf8) 440 | 441 | self._ping_queue = deque() 442 | 443 | def _receive_frame(self): 444 | """Receives a frame and return data in the frame as a tuple containing 445 | each header field and payload separately. 446 | 447 | Raises: 448 | ConnectionTerminatedException: when read returns empty 449 | string. 450 | InvalidFrameException: when the frame contains invalid data. 451 | """ 452 | 453 | def _receive_bytes(length): 454 | return self.receive_bytes(length) 455 | 456 | return parse_frame(receive_bytes=_receive_bytes, 457 | logger=self._logger, 458 | ws_version=self._request.ws_version, 459 | unmask_receive=self._options.unmask_receive) 460 | 461 | def _receive_frame_as_frame_object(self): 462 | opcode, unmasked_bytes, fin, rsv1, rsv2, rsv3 = self._receive_frame() 463 | 464 | return Frame(fin=fin, rsv1=rsv1, rsv2=rsv2, rsv3=rsv3, 465 | opcode=opcode, payload=unmasked_bytes) 466 | 467 | def receive_filtered_frame(self): 468 | """Receives a frame and applies frame filters and message filters. 469 | The frame to be received must satisfy following conditions: 470 | - The frame is not fragmented. 471 | - The opcode of the frame is TEXT or BINARY. 472 | 473 | DO NOT USE this method except for testing purpose. 474 | """ 475 | 476 | frame = self._receive_frame_as_frame_object() 477 | if not frame.fin: 478 | raise InvalidFrameException( 479 | 'Segmented frames must not be received via ' 480 | 'receive_filtered_frame()') 481 | if (frame.opcode != common.OPCODE_TEXT and 482 | frame.opcode != common.OPCODE_BINARY): 483 | raise InvalidFrameException( 484 | 'Control frames must not be received via ' 485 | 'receive_filtered_frame()') 486 | 487 | for frame_filter in self._options.incoming_frame_filters: 488 | frame_filter.filter(frame) 489 | for message_filter in self._options.incoming_message_filters: 490 | frame.payload = message_filter.filter(frame.payload) 491 | return frame 492 | 493 | def send_message(self, message, end=True, binary=False): 494 | """Send message. 495 | 496 | Args: 497 | message: text in unicode or binary in str to send. 498 | binary: send message as binary frame. 499 | 500 | Raises: 501 | BadOperationException: when called on a server-terminated 502 | connection or called with inconsistent message type or 503 | binary parameter. 504 | """ 505 | 506 | if self._request.server_terminated: 507 | raise BadOperationException( 508 | 'Requested send_message after sending out a closing handshake') 509 | 510 | if binary and isinstance(message, unicode): 511 | raise BadOperationException( 512 | 'Message for binary frame must be instance of str') 513 | 514 | for message_filter in self._options.outgoing_message_filters: 515 | message = message_filter.filter(message, end, binary) 516 | 517 | try: 518 | # Set this to any positive integer to limit maximum size of data in 519 | # payload data of each frame. 520 | MAX_PAYLOAD_DATA_SIZE = -1 521 | 522 | if MAX_PAYLOAD_DATA_SIZE <= 0: 523 | self._write(self._writer.build(message, end, binary)) 524 | return 525 | 526 | bytes_written = 0 527 | while True: 528 | end_for_this_frame = end 529 | bytes_to_write = len(message) - bytes_written 530 | if (MAX_PAYLOAD_DATA_SIZE > 0 and 531 | bytes_to_write > MAX_PAYLOAD_DATA_SIZE): 532 | end_for_this_frame = False 533 | bytes_to_write = MAX_PAYLOAD_DATA_SIZE 534 | 535 | frame = self._writer.build( 536 | message[bytes_written:bytes_written + bytes_to_write], 537 | end_for_this_frame, 538 | binary) 539 | self._write(frame) 540 | 541 | bytes_written += bytes_to_write 542 | 543 | # This if must be placed here (the end of while block) so that 544 | # at least one frame is sent. 545 | if len(message) <= bytes_written: 546 | break 547 | except ValueError as e: 548 | raise BadOperationException(e) 549 | 550 | def _get_message_from_frame(self, frame): 551 | """Gets a message from frame. If the message is composed of fragmented 552 | frames and the frame is not the last fragmented frame, this method 553 | returns None. The whole message will be returned when the last 554 | fragmented frame is passed to this method. 555 | 556 | Raises: 557 | InvalidFrameException: when the frame doesn't match defragmentation 558 | context, or the frame contains invalid data. 559 | """ 560 | 561 | if frame.opcode == common.OPCODE_CONTINUATION: 562 | if not self._received_fragments: 563 | if frame.fin: 564 | raise InvalidFrameException( 565 | 'Received a termination frame but fragmentation ' 566 | 'not started') 567 | else: 568 | raise InvalidFrameException( 569 | 'Received an intermediate frame but ' 570 | 'fragmentation not started') 571 | 572 | if frame.fin: 573 | # End of fragmentation frame 574 | self._received_fragments.append(frame.payload) 575 | message = ''.join(self._received_fragments) 576 | self._received_fragments = [] 577 | return message 578 | else: 579 | # Intermediate frame 580 | self._received_fragments.append(frame.payload) 581 | return None 582 | else: 583 | if self._received_fragments: 584 | if frame.fin: 585 | raise InvalidFrameException( 586 | 'Received an unfragmented frame without ' 587 | 'terminating existing fragmentation') 588 | else: 589 | raise InvalidFrameException( 590 | 'New fragmentation started without terminating ' 591 | 'existing fragmentation') 592 | 593 | if frame.fin: 594 | # Unfragmented frame 595 | 596 | self._original_opcode = frame.opcode 597 | return frame.payload 598 | else: 599 | # Start of fragmentation frame 600 | 601 | if common.is_control_opcode(frame.opcode): 602 | raise InvalidFrameException( 603 | 'Control frames must not be fragmented') 604 | 605 | self._original_opcode = frame.opcode 606 | self._received_fragments.append(frame.payload) 607 | return None 608 | 609 | def _process_close_message(self, message): 610 | """Processes close message. 611 | 612 | Args: 613 | message: close message. 614 | 615 | Raises: 616 | InvalidFrameException: when the message is invalid. 617 | """ 618 | 619 | self._request.client_terminated = True 620 | 621 | # Status code is optional. We can have status reason only if we 622 | # have status code. Status reason can be empty string. So, 623 | # allowed cases are 624 | # - no application data: no code no reason 625 | # - 2 octet of application data: has code but no reason 626 | # - 3 or more octet of application data: both code and reason 627 | if len(message) == 0: 628 | self._logger.debug('Received close frame (empty body)') 629 | self._request.ws_close_code = ( 630 | common.STATUS_NO_STATUS_RECEIVED) 631 | elif len(message) == 1: 632 | raise InvalidFrameException( 633 | 'If a close frame has status code, the length of ' 634 | 'status code must be 2 octet') 635 | elif len(message) >= 2: 636 | self._request.ws_close_code = struct.unpack( 637 | '!H', buffer(message[0:2]))[0] 638 | self._request.ws_close_reason = message[2:].decode( 639 | 'utf-8', 'replace') 640 | self._logger.debug( 641 | 'Received close frame (code=%d, reason=%r)', 642 | self._request.ws_close_code, 643 | self._request.ws_close_reason) 644 | 645 | # As we've received a close frame, no more data is coming over the 646 | # socket. We can now safely close the socket without worrying about 647 | # RST sending. 648 | 649 | if self._request.server_terminated: 650 | self._logger.debug( 651 | 'Received ack for server-initiated closing handshake') 652 | return 653 | 654 | self._logger.debug( 655 | 'Received client-initiated closing handshake') 656 | 657 | code = common.STATUS_NORMAL_CLOSURE 658 | reason = '' 659 | if hasattr(self._request, '_dispatcher'): 660 | dispatcher = self._request._dispatcher 661 | code, reason = dispatcher.passive_closing_handshake( 662 | self._request) 663 | if code is None and reason is not None and len(reason) > 0: 664 | self._logger.warning( 665 | 'Handler specified reason despite code being None') 666 | reason = '' 667 | if reason is None: 668 | reason = '' 669 | self._send_closing_handshake(code, reason) 670 | self._logger.debug( 671 | 'Acknowledged closing handshake initiated by the peer ' 672 | '(code=%r, reason=%r)', code, reason) 673 | 674 | def _process_ping_message(self, message): 675 | """Processes ping message. 676 | 677 | Args: 678 | message: ping message. 679 | """ 680 | 681 | try: 682 | handler = self._request.on_ping_handler 683 | if handler: 684 | handler(self._request, message) 685 | return 686 | except AttributeError as e: 687 | pass 688 | self._send_pong(message) 689 | 690 | def _process_pong_message(self, message): 691 | """Processes pong message. 692 | 693 | Args: 694 | message: pong message. 695 | """ 696 | 697 | # TODO(tyoshino): Add ping timeout handling. 698 | 699 | inflight_pings = deque() 700 | 701 | while True: 702 | try: 703 | expected_body = self._ping_queue.popleft() 704 | if expected_body == message: 705 | # inflight_pings contains pings ignored by the 706 | # other peer. Just forget them. 707 | self._logger.debug( 708 | 'Ping %r is acked (%d pings were ignored)', 709 | expected_body, len(inflight_pings)) 710 | break 711 | else: 712 | inflight_pings.append(expected_body) 713 | except IndexError as e: 714 | # The received pong was unsolicited pong. Keep the 715 | # ping queue as is. 716 | self._ping_queue = inflight_pings 717 | self._logger.debug('Received a unsolicited pong') 718 | break 719 | 720 | try: 721 | handler = self._request.on_pong_handler 722 | if handler: 723 | handler(self._request, message) 724 | except AttributeError as e: 725 | pass 726 | 727 | def receive_message(self): 728 | """Receive a WebSocket frame and return its payload as a text in 729 | unicode or a binary in str. 730 | 731 | Returns: 732 | payload data of the frame 733 | - as unicode instance if received text frame 734 | - as str instance if received binary frame 735 | or None iff received closing handshake. 736 | Raises: 737 | BadOperationException: when called on a client-terminated 738 | connection. 739 | ConnectionTerminatedException: when read returns empty 740 | string. 741 | InvalidFrameException: when the frame contains invalid 742 | data. 743 | UnsupportedFrameException: when the received frame has 744 | flags, opcode we cannot handle. You can ignore this 745 | exception and continue receiving the next frame. 746 | """ 747 | 748 | if self._request.client_terminated: 749 | raise BadOperationException( 750 | 'Requested receive_message after receiving a closing ' 751 | 'handshake') 752 | 753 | while True: 754 | # mp_conn.read will block if no bytes are available. 755 | # Timeout is controlled by TimeOut directive of Apache. 756 | 757 | frame = self._receive_frame_as_frame_object() 758 | 759 | # Check the constraint on the payload size for control frames 760 | # before extension processes the frame. 761 | # See also http://tools.ietf.org/html/rfc6455#section-5.5 762 | if (common.is_control_opcode(frame.opcode) and 763 | len(frame.payload) > 125): 764 | raise InvalidFrameException( 765 | 'Payload data size of control frames must be 125 bytes or ' 766 | 'less') 767 | 768 | for frame_filter in self._options.incoming_frame_filters: 769 | frame_filter.filter(frame) 770 | 771 | if frame.rsv1 or frame.rsv2 or frame.rsv3: 772 | raise UnsupportedFrameException( 773 | 'Unsupported flag is set (rsv = %d%d%d)' % 774 | (frame.rsv1, frame.rsv2, frame.rsv3)) 775 | 776 | message = self._get_message_from_frame(frame) 777 | if message is None: 778 | continue 779 | 780 | for message_filter in self._options.incoming_message_filters: 781 | message = message_filter.filter(message) 782 | 783 | if self._original_opcode == common.OPCODE_TEXT: 784 | # The WebSocket protocol section 4.4 specifies that invalid 785 | # characters must be replaced with U+fffd REPLACEMENT 786 | # CHARACTER. 787 | try: 788 | return message.decode('utf-8') 789 | except UnicodeDecodeError as e: 790 | raise InvalidUTF8Exception(e) 791 | elif self._original_opcode == common.OPCODE_BINARY: 792 | return message 793 | elif self._original_opcode == common.OPCODE_CLOSE: 794 | self._process_close_message(message) 795 | return None 796 | elif self._original_opcode == common.OPCODE_PING: 797 | self._process_ping_message(message) 798 | elif self._original_opcode == common.OPCODE_PONG: 799 | self._process_pong_message(message) 800 | else: 801 | raise UnsupportedFrameException( 802 | 'Opcode %d is not supported' % self._original_opcode) 803 | 804 | def _send_closing_handshake(self, code, reason): 805 | body = create_closing_handshake_body(code, reason) 806 | frame = create_close_frame( 807 | body, mask=self._options.mask_send, 808 | frame_filters=self._options.outgoing_frame_filters) 809 | 810 | self._request.server_terminated = True 811 | 812 | self._write(frame) 813 | 814 | def close_connection(self, code=common.STATUS_NORMAL_CLOSURE, reason='', 815 | wait_response=True): 816 | """Closes a WebSocket connection. Note that this method blocks until 817 | it receives acknowledgement to the closing handshake. 818 | 819 | Args: 820 | code: Status code for close frame. If code is None, a close 821 | frame with empty body will be sent. 822 | reason: string representing close reason. 823 | wait_response: True when caller want to wait the response. 824 | Raises: 825 | BadOperationException: when reason is specified with code None 826 | or reason is not an instance of both str and unicode. 827 | """ 828 | 829 | if self._request.server_terminated: 830 | self._logger.debug( 831 | 'Requested close_connection but server is already terminated') 832 | return 833 | 834 | # When we receive a close frame, we call _process_close_message(). 835 | # _process_close_message() immediately acknowledges to the 836 | # server-initiated closing handshake and sets server_terminated to 837 | # True. So, here we can assume that we haven't received any close 838 | # frame. We're initiating a closing handshake. 839 | 840 | if code is None: 841 | if reason is not None and len(reason) > 0: 842 | raise BadOperationException( 843 | 'close reason must not be specified if code is None') 844 | reason = '' 845 | else: 846 | if not isinstance(reason, str) and not isinstance(reason, unicode): 847 | raise BadOperationException( 848 | 'close reason must be an instance of str or unicode') 849 | 850 | self._send_closing_handshake(code, reason) 851 | self._logger.debug( 852 | 'Initiated closing handshake (code=%r, reason=%r)', 853 | code, reason) 854 | 855 | if (code == common.STATUS_GOING_AWAY or 856 | code == common.STATUS_PROTOCOL_ERROR) or not wait_response: 857 | # It doesn't make sense to wait for a close frame if the reason is 858 | # protocol error or that the server is going away. For some of 859 | # other reasons, it might not make sense to wait for a close frame, 860 | # but it's not clear, yet. 861 | return 862 | 863 | # TODO(ukai): 2. wait until the /client terminated/ flag has been set, 864 | # or until a server-defined timeout expires. 865 | # 866 | # For now, we expect receiving closing handshake right after sending 867 | # out closing handshake. 868 | message = self.receive_message() 869 | if message is not None: 870 | raise ConnectionTerminatedException( 871 | 'Didn\'t receive valid ack for closing handshake') 872 | # TODO: 3. close the WebSocket connection. 873 | # note: mod_python Connection (mp_conn) doesn't have close method. 874 | 875 | def send_ping(self, body=''): 876 | frame = create_ping_frame( 877 | body, 878 | self._options.mask_send, 879 | self._options.outgoing_frame_filters) 880 | self._write(frame) 881 | 882 | self._ping_queue.append(body) 883 | 884 | def _send_pong(self, body): 885 | frame = create_pong_frame( 886 | body, 887 | self._options.mask_send, 888 | self._options.outgoing_frame_filters) 889 | self._write(frame) 890 | 891 | def get_last_received_opcode(self): 892 | """Returns the opcode of the WebSocket message which the last received 893 | frame belongs to. The return value is valid iff immediately after 894 | receive_message call. 895 | """ 896 | 897 | return self._original_opcode 898 | 899 | 900 | # vi:sts=4 sw=4 et 901 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/common.py: -------------------------------------------------------------------------------- 1 | # Copyright 2012, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """This file must not depend on any module specific to the WebSocket protocol. 32 | """ 33 | 34 | 35 | from mod_pywebsocket import http_header_util 36 | 37 | 38 | # Additional log level definitions. 39 | LOGLEVEL_FINE = 9 40 | 41 | # Constants indicating WebSocket protocol version. 42 | VERSION_HIXIE75 = -1 43 | VERSION_HYBI00 = 0 44 | VERSION_HYBI01 = 1 45 | VERSION_HYBI02 = 2 46 | VERSION_HYBI03 = 2 47 | VERSION_HYBI04 = 4 48 | VERSION_HYBI05 = 5 49 | VERSION_HYBI06 = 6 50 | VERSION_HYBI07 = 7 51 | VERSION_HYBI08 = 8 52 | VERSION_HYBI09 = 8 53 | VERSION_HYBI10 = 8 54 | VERSION_HYBI11 = 8 55 | VERSION_HYBI12 = 8 56 | VERSION_HYBI13 = 13 57 | VERSION_HYBI14 = 13 58 | VERSION_HYBI15 = 13 59 | VERSION_HYBI16 = 13 60 | VERSION_HYBI17 = 13 61 | 62 | # Constants indicating WebSocket protocol latest version. 63 | VERSION_HYBI_LATEST = VERSION_HYBI13 64 | 65 | # Port numbers 66 | DEFAULT_WEB_SOCKET_PORT = 80 67 | DEFAULT_WEB_SOCKET_SECURE_PORT = 443 68 | 69 | # Schemes 70 | WEB_SOCKET_SCHEME = 'ws' 71 | WEB_SOCKET_SECURE_SCHEME = 'wss' 72 | 73 | # Frame opcodes defined in the spec. 74 | OPCODE_CONTINUATION = 0x0 75 | OPCODE_TEXT = 0x1 76 | OPCODE_BINARY = 0x2 77 | OPCODE_CLOSE = 0x8 78 | OPCODE_PING = 0x9 79 | OPCODE_PONG = 0xa 80 | 81 | # UUIDs used by HyBi 04 and later opening handshake and frame masking. 82 | WEBSOCKET_ACCEPT_UUID = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11' 83 | 84 | # Opening handshake header names and expected values. 85 | UPGRADE_HEADER = 'Upgrade' 86 | WEBSOCKET_UPGRADE_TYPE = 'websocket' 87 | WEBSOCKET_UPGRADE_TYPE_HIXIE75 = 'WebSocket' 88 | CONNECTION_HEADER = 'Connection' 89 | UPGRADE_CONNECTION_TYPE = 'Upgrade' 90 | HOST_HEADER = 'Host' 91 | ORIGIN_HEADER = 'Origin' 92 | SEC_WEBSOCKET_ORIGIN_HEADER = 'Sec-WebSocket-Origin' 93 | SEC_WEBSOCKET_KEY_HEADER = 'Sec-WebSocket-Key' 94 | SEC_WEBSOCKET_ACCEPT_HEADER = 'Sec-WebSocket-Accept' 95 | SEC_WEBSOCKET_VERSION_HEADER = 'Sec-WebSocket-Version' 96 | SEC_WEBSOCKET_PROTOCOL_HEADER = 'Sec-WebSocket-Protocol' 97 | SEC_WEBSOCKET_EXTENSIONS_HEADER = 'Sec-WebSocket-Extensions' 98 | SEC_WEBSOCKET_DRAFT_HEADER = 'Sec-WebSocket-Draft' 99 | SEC_WEBSOCKET_KEY1_HEADER = 'Sec-WebSocket-Key1' 100 | SEC_WEBSOCKET_KEY2_HEADER = 'Sec-WebSocket-Key2' 101 | SEC_WEBSOCKET_LOCATION_HEADER = 'Sec-WebSocket-Location' 102 | 103 | # Extensions 104 | DEFLATE_FRAME_EXTENSION = 'deflate-frame' 105 | PERMESSAGE_DEFLATE_EXTENSION = 'permessage-deflate' 106 | X_WEBKIT_DEFLATE_FRAME_EXTENSION = 'x-webkit-deflate-frame' 107 | MUX_EXTENSION = 'mux_DO_NOT_USE' 108 | 109 | # Status codes 110 | # Code STATUS_NO_STATUS_RECEIVED, STATUS_ABNORMAL_CLOSURE, and 111 | # STATUS_TLS_HANDSHAKE are pseudo codes to indicate specific error cases. 112 | # Could not be used for codes in actual closing frames. 113 | # Application level errors must use codes in the range 114 | # STATUS_USER_REGISTERED_BASE to STATUS_USER_PRIVATE_MAX. The codes in the 115 | # range STATUS_USER_REGISTERED_BASE to STATUS_USER_REGISTERED_MAX are managed 116 | # by IANA. Usually application must define user protocol level errors in the 117 | # range STATUS_USER_PRIVATE_BASE to STATUS_USER_PRIVATE_MAX. 118 | STATUS_NORMAL_CLOSURE = 1000 119 | STATUS_GOING_AWAY = 1001 120 | STATUS_PROTOCOL_ERROR = 1002 121 | STATUS_UNSUPPORTED_DATA = 1003 122 | STATUS_NO_STATUS_RECEIVED = 1005 123 | STATUS_ABNORMAL_CLOSURE = 1006 124 | STATUS_INVALID_FRAME_PAYLOAD_DATA = 1007 125 | STATUS_POLICY_VIOLATION = 1008 126 | STATUS_MESSAGE_TOO_BIG = 1009 127 | STATUS_MANDATORY_EXTENSION = 1010 128 | STATUS_INTERNAL_ENDPOINT_ERROR = 1011 129 | STATUS_TLS_HANDSHAKE = 1015 130 | STATUS_USER_REGISTERED_BASE = 3000 131 | STATUS_USER_REGISTERED_MAX = 3999 132 | STATUS_USER_PRIVATE_BASE = 4000 133 | STATUS_USER_PRIVATE_MAX = 4999 134 | # Following definitions are aliases to keep compatibility. Applications must 135 | # not use these obsoleted definitions anymore. 136 | STATUS_NORMAL = STATUS_NORMAL_CLOSURE 137 | STATUS_UNSUPPORTED = STATUS_UNSUPPORTED_DATA 138 | STATUS_CODE_NOT_AVAILABLE = STATUS_NO_STATUS_RECEIVED 139 | STATUS_ABNORMAL_CLOSE = STATUS_ABNORMAL_CLOSURE 140 | STATUS_INVALID_FRAME_PAYLOAD = STATUS_INVALID_FRAME_PAYLOAD_DATA 141 | STATUS_MANDATORY_EXT = STATUS_MANDATORY_EXTENSION 142 | 143 | # HTTP status codes 144 | HTTP_STATUS_BAD_REQUEST = 400 145 | HTTP_STATUS_FORBIDDEN = 403 146 | HTTP_STATUS_NOT_FOUND = 404 147 | 148 | 149 | def is_control_opcode(opcode): 150 | return (opcode >> 3) == 1 151 | 152 | 153 | class ExtensionParameter(object): 154 | 155 | """This is exchanged on extension negotiation in opening handshake.""" 156 | 157 | def __init__(self, name): 158 | self._name = name 159 | # TODO(tyoshino): Change the data structure to more efficient one such 160 | # as dict when the spec changes to say like 161 | # - Parameter names must be unique 162 | # - The order of parameters is not significant 163 | self._parameters = [] 164 | 165 | def name(self): 166 | """Return the extension name.""" 167 | return self._name 168 | 169 | def add_parameter(self, name, value): 170 | """Add a parameter.""" 171 | self._parameters.append((name, value)) 172 | 173 | def get_parameters(self): 174 | """Return the parameters.""" 175 | return self._parameters 176 | 177 | def get_parameter_names(self): 178 | """Return the names of the parameters.""" 179 | return [name for name, unused_value in self._parameters] 180 | 181 | def has_parameter(self, name): 182 | """Test if a parameter exists.""" 183 | for param_name, param_value in self._parameters: 184 | if param_name == name: 185 | return True 186 | return False 187 | 188 | def get_parameter_value(self, name): 189 | """Get the value of a specific parameter.""" 190 | for param_name, param_value in self._parameters: 191 | if param_name == name: 192 | return param_value 193 | 194 | 195 | class ExtensionParsingException(Exception): 196 | 197 | """Exception to handle errors in extension parsing.""" 198 | 199 | def __init__(self, name): 200 | super(ExtensionParsingException, self).__init__(name) 201 | 202 | 203 | def _parse_extension_param(state, definition): 204 | param_name = http_header_util.consume_token(state) 205 | 206 | if param_name is None: 207 | raise ExtensionParsingException('No valid parameter name found') 208 | 209 | http_header_util.consume_lwses(state) 210 | 211 | if not http_header_util.consume_string(state, '='): 212 | definition.add_parameter(param_name, None) 213 | return 214 | 215 | http_header_util.consume_lwses(state) 216 | 217 | # TODO(tyoshino): Add code to validate that parsed param_value is token 218 | param_value = http_header_util.consume_token_or_quoted_string(state) 219 | if param_value is None: 220 | raise ExtensionParsingException( 221 | 'No valid parameter value found on the right-hand side of ' 222 | 'parameter %r' % param_name) 223 | 224 | definition.add_parameter(param_name, param_value) 225 | 226 | 227 | def _parse_extension(state): 228 | extension_token = http_header_util.consume_token(state) 229 | if extension_token is None: 230 | return None 231 | 232 | extension = ExtensionParameter(extension_token) 233 | 234 | while True: 235 | http_header_util.consume_lwses(state) 236 | 237 | if not http_header_util.consume_string(state, ';'): 238 | break 239 | 240 | http_header_util.consume_lwses(state) 241 | 242 | try: 243 | _parse_extension_param(state, extension) 244 | except ExtensionParsingException as e: 245 | raise ExtensionParsingException( 246 | 'Failed to parse parameter for %r (%r)' % 247 | (extension_token, e)) 248 | 249 | return extension 250 | 251 | 252 | def parse_extensions(data): 253 | """Parse Sec-WebSocket-Extensions header value. 254 | 255 | Returns a list of ExtensionParameter objects. 256 | Leading LWSes must be trimmed. 257 | """ 258 | state = http_header_util.ParsingState(data) 259 | 260 | extension_list = [] 261 | while True: 262 | extension = _parse_extension(state) 263 | if extension is not None: 264 | extension_list.append(extension) 265 | 266 | http_header_util.consume_lwses(state) 267 | 268 | if http_header_util.peek(state) is None: 269 | break 270 | 271 | if not http_header_util.consume_string(state, ','): 272 | raise ExtensionParsingException( 273 | 'Failed to parse Sec-WebSocket-Extensions header: ' 274 | 'Expected a comma but found %r' % 275 | http_header_util.peek(state)) 276 | 277 | http_header_util.consume_lwses(state) 278 | 279 | if len(extension_list) == 0: 280 | raise ExtensionParsingException( 281 | 'No valid extension entry found') 282 | 283 | return extension_list 284 | 285 | 286 | def format_extension(extension): 287 | """Format an ExtensionParameter object.""" 288 | formatted_params = [extension.name()] 289 | for param_name, param_value in extension.get_parameters(): 290 | if param_value is None: 291 | formatted_params.append(param_name) 292 | else: 293 | quoted_value = http_header_util.quote_if_necessary(param_value) 294 | formatted_params.append('%s=%s' % (param_name, quoted_value)) 295 | return '; '.join(formatted_params) 296 | 297 | 298 | def format_extensions(extension_list): 299 | """Format a list of ExtensionParameter objects.""" 300 | formatted_extension_list = [] 301 | for extension in extension_list: 302 | formatted_extension_list.append(format_extension(extension)) 303 | return ', '.join(formatted_extension_list) 304 | 305 | 306 | # vi:sts=4 sw=4 et 307 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/extensions.py: -------------------------------------------------------------------------------- 1 | # Copyright 2012, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | from mod_pywebsocket import common 32 | from mod_pywebsocket import util 33 | from mod_pywebsocket.http_header_util import quote_if_necessary 34 | 35 | 36 | # The list of available server side extension processor classes. 37 | _available_processors = {} 38 | _compression_extension_names = [] 39 | 40 | 41 | class ExtensionProcessorInterface(object): 42 | 43 | def __init__(self, request): 44 | self._logger = util.get_class_logger(self) 45 | 46 | self._request = request 47 | self._active = True 48 | 49 | def request(self): 50 | return self._request 51 | 52 | def name(self): 53 | return None 54 | 55 | def check_consistency_with_other_processors(self, processors): 56 | pass 57 | 58 | def set_active(self, active): 59 | self._active = active 60 | 61 | def is_active(self): 62 | return self._active 63 | 64 | def _get_extension_response_internal(self): 65 | return None 66 | 67 | def get_extension_response(self): 68 | if not self._active: 69 | self._logger.debug('Extension %s is deactivated', self.name()) 70 | return None 71 | 72 | response = self._get_extension_response_internal() 73 | if response is None: 74 | self._active = False 75 | return response 76 | 77 | def _setup_stream_options_internal(self, stream_options): 78 | pass 79 | 80 | def setup_stream_options(self, stream_options): 81 | if self._active: 82 | self._setup_stream_options_internal(stream_options) 83 | 84 | 85 | def _log_outgoing_compression_ratio( 86 | logger, original_bytes, filtered_bytes, average_ratio): 87 | # Print inf when ratio is not available. 88 | ratio = float('inf') 89 | if original_bytes != 0: 90 | ratio = float(filtered_bytes) / original_bytes 91 | 92 | logger.debug('Outgoing compression ratio: %f (average: %f)' % 93 | (ratio, average_ratio)) 94 | 95 | 96 | def _log_incoming_compression_ratio( 97 | logger, received_bytes, filtered_bytes, average_ratio): 98 | # Print inf when ratio is not available. 99 | ratio = float('inf') 100 | if filtered_bytes != 0: 101 | ratio = float(received_bytes) / filtered_bytes 102 | 103 | logger.debug('Incoming compression ratio: %f (average: %f)' % 104 | (ratio, average_ratio)) 105 | 106 | 107 | def _parse_window_bits(bits): 108 | """Return parsed integer value iff the given string conforms to the 109 | grammar of the window bits extension parameters. 110 | """ 111 | 112 | if bits is None: 113 | raise ValueError('Value is required') 114 | 115 | # For non integer values such as "10.0", ValueError will be raised. 116 | int_bits = int(bits) 117 | 118 | # First condition is to drop leading zero case e.g. "08". 119 | if bits != str(int_bits) or int_bits < 8 or int_bits > 15: 120 | raise ValueError('Invalid value: %r' % bits) 121 | 122 | return int_bits 123 | 124 | 125 | class _AverageRatioCalculator(object): 126 | """Stores total bytes of original and result data, and calculates average 127 | result / original ratio. 128 | """ 129 | 130 | def __init__(self): 131 | self._total_original_bytes = 0 132 | self._total_result_bytes = 0 133 | 134 | def add_original_bytes(self, value): 135 | self._total_original_bytes += value 136 | 137 | def add_result_bytes(self, value): 138 | self._total_result_bytes += value 139 | 140 | def get_average_ratio(self): 141 | if self._total_original_bytes != 0: 142 | return (float(self._total_result_bytes) / 143 | self._total_original_bytes) 144 | else: 145 | return float('inf') 146 | 147 | 148 | class DeflateFrameExtensionProcessor(ExtensionProcessorInterface): 149 | """deflate-frame extension processor. 150 | 151 | Specification: 152 | http://tools.ietf.org/html/draft-tyoshino-hybi-websocket-perframe-deflate 153 | """ 154 | 155 | _WINDOW_BITS_PARAM = 'max_window_bits' 156 | _NO_CONTEXT_TAKEOVER_PARAM = 'no_context_takeover' 157 | 158 | def __init__(self, request): 159 | ExtensionProcessorInterface.__init__(self, request) 160 | self._logger = util.get_class_logger(self) 161 | 162 | self._response_window_bits = None 163 | self._response_no_context_takeover = False 164 | self._bfinal = False 165 | 166 | # Calculates 167 | # (Total outgoing bytes supplied to this filter) / 168 | # (Total bytes sent to the network after applying this filter) 169 | self._outgoing_average_ratio_calculator = _AverageRatioCalculator() 170 | 171 | # Calculates 172 | # (Total bytes received from the network) / 173 | # (Total incoming bytes obtained after applying this filter) 174 | self._incoming_average_ratio_calculator = _AverageRatioCalculator() 175 | 176 | def name(self): 177 | return common.DEFLATE_FRAME_EXTENSION 178 | 179 | def _get_extension_response_internal(self): 180 | # Any unknown parameter will be just ignored. 181 | 182 | window_bits = None 183 | if self._request.has_parameter(self._WINDOW_BITS_PARAM): 184 | window_bits = self._request.get_parameter_value( 185 | self._WINDOW_BITS_PARAM) 186 | try: 187 | window_bits = _parse_window_bits(window_bits) 188 | except ValueError as e: 189 | return None 190 | 191 | no_context_takeover = self._request.has_parameter( 192 | self._NO_CONTEXT_TAKEOVER_PARAM) 193 | if (no_context_takeover and 194 | self._request.get_parameter_value( 195 | self._NO_CONTEXT_TAKEOVER_PARAM) is not None): 196 | return None 197 | 198 | self._rfc1979_deflater = util._RFC1979Deflater( 199 | window_bits, no_context_takeover) 200 | 201 | self._rfc1979_inflater = util._RFC1979Inflater() 202 | 203 | self._compress_outgoing = True 204 | 205 | response = common.ExtensionParameter(self._request.name()) 206 | 207 | if self._response_window_bits is not None: 208 | response.add_parameter( 209 | self._WINDOW_BITS_PARAM, str(self._response_window_bits)) 210 | if self._response_no_context_takeover: 211 | response.add_parameter( 212 | self._NO_CONTEXT_TAKEOVER_PARAM, None) 213 | 214 | self._logger.debug( 215 | 'Enable %s extension (' 216 | 'request: window_bits=%s; no_context_takeover=%r, ' 217 | 'response: window_wbits=%s; no_context_takeover=%r)' % 218 | (self._request.name(), 219 | window_bits, 220 | no_context_takeover, 221 | self._response_window_bits, 222 | self._response_no_context_takeover)) 223 | 224 | return response 225 | 226 | def _setup_stream_options_internal(self, stream_options): 227 | 228 | class _OutgoingFilter(object): 229 | 230 | def __init__(self, parent): 231 | self._parent = parent 232 | 233 | def filter(self, frame): 234 | self._parent._outgoing_filter(frame) 235 | 236 | class _IncomingFilter(object): 237 | 238 | def __init__(self, parent): 239 | self._parent = parent 240 | 241 | def filter(self, frame): 242 | self._parent._incoming_filter(frame) 243 | 244 | stream_options.outgoing_frame_filters.append( 245 | _OutgoingFilter(self)) 246 | stream_options.incoming_frame_filters.insert( 247 | 0, _IncomingFilter(self)) 248 | 249 | def set_response_window_bits(self, value): 250 | self._response_window_bits = value 251 | 252 | def set_response_no_context_takeover(self, value): 253 | self._response_no_context_takeover = value 254 | 255 | def set_bfinal(self, value): 256 | self._bfinal = value 257 | 258 | def enable_outgoing_compression(self): 259 | self._compress_outgoing = True 260 | 261 | def disable_outgoing_compression(self): 262 | self._compress_outgoing = False 263 | 264 | def _outgoing_filter(self, frame): 265 | """Transform outgoing frames. This method is called only by 266 | an _OutgoingFilter instance. 267 | """ 268 | 269 | original_payload_size = len(frame.payload) 270 | self._outgoing_average_ratio_calculator.add_original_bytes( 271 | original_payload_size) 272 | 273 | if (not self._compress_outgoing or 274 | common.is_control_opcode(frame.opcode)): 275 | self._outgoing_average_ratio_calculator.add_result_bytes( 276 | original_payload_size) 277 | return 278 | 279 | frame.payload = self._rfc1979_deflater.filter( 280 | frame.payload, bfinal=self._bfinal) 281 | frame.rsv1 = 1 282 | 283 | filtered_payload_size = len(frame.payload) 284 | self._outgoing_average_ratio_calculator.add_result_bytes( 285 | filtered_payload_size) 286 | 287 | _log_outgoing_compression_ratio( 288 | self._logger, 289 | original_payload_size, 290 | filtered_payload_size, 291 | self._outgoing_average_ratio_calculator.get_average_ratio()) 292 | 293 | def _incoming_filter(self, frame): 294 | """Transform incoming frames. This method is called only by 295 | an _IncomingFilter instance. 296 | """ 297 | 298 | received_payload_size = len(frame.payload) 299 | self._incoming_average_ratio_calculator.add_result_bytes( 300 | received_payload_size) 301 | 302 | if frame.rsv1 != 1 or common.is_control_opcode(frame.opcode): 303 | self._incoming_average_ratio_calculator.add_original_bytes( 304 | received_payload_size) 305 | return 306 | 307 | frame.payload = self._rfc1979_inflater.filter(frame.payload) 308 | frame.rsv1 = 0 309 | 310 | filtered_payload_size = len(frame.payload) 311 | self._incoming_average_ratio_calculator.add_original_bytes( 312 | filtered_payload_size) 313 | 314 | _log_incoming_compression_ratio( 315 | self._logger, 316 | received_payload_size, 317 | filtered_payload_size, 318 | self._incoming_average_ratio_calculator.get_average_ratio()) 319 | 320 | 321 | _available_processors[common.DEFLATE_FRAME_EXTENSION] = ( 322 | DeflateFrameExtensionProcessor) 323 | _compression_extension_names.append(common.DEFLATE_FRAME_EXTENSION) 324 | 325 | _available_processors[common.X_WEBKIT_DEFLATE_FRAME_EXTENSION] = ( 326 | DeflateFrameExtensionProcessor) 327 | _compression_extension_names.append(common.X_WEBKIT_DEFLATE_FRAME_EXTENSION) 328 | 329 | 330 | class PerMessageDeflateExtensionProcessor(ExtensionProcessorInterface): 331 | """permessage-deflate extension processor. 332 | 333 | Specification: 334 | http://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-08 335 | """ 336 | 337 | _SERVER_MAX_WINDOW_BITS_PARAM = 'server_max_window_bits' 338 | _SERVER_NO_CONTEXT_TAKEOVER_PARAM = 'server_no_context_takeover' 339 | _CLIENT_MAX_WINDOW_BITS_PARAM = 'client_max_window_bits' 340 | _CLIENT_NO_CONTEXT_TAKEOVER_PARAM = 'client_no_context_takeover' 341 | 342 | def __init__(self, request): 343 | """Construct PerMessageDeflateExtensionProcessor.""" 344 | 345 | ExtensionProcessorInterface.__init__(self, request) 346 | self._logger = util.get_class_logger(self) 347 | 348 | self._preferred_client_max_window_bits = None 349 | self._client_no_context_takeover = False 350 | 351 | def name(self): 352 | # This method returns "deflate" (not "permessage-deflate") for 353 | # compatibility. 354 | return 'deflate' 355 | 356 | def _get_extension_response_internal(self): 357 | for name in self._request.get_parameter_names(): 358 | if name not in [self._SERVER_MAX_WINDOW_BITS_PARAM, 359 | self._SERVER_NO_CONTEXT_TAKEOVER_PARAM, 360 | self._CLIENT_MAX_WINDOW_BITS_PARAM]: 361 | self._logger.debug('Unknown parameter: %r', name) 362 | return None 363 | 364 | server_max_window_bits = None 365 | if self._request.has_parameter(self._SERVER_MAX_WINDOW_BITS_PARAM): 366 | server_max_window_bits = self._request.get_parameter_value( 367 | self._SERVER_MAX_WINDOW_BITS_PARAM) 368 | try: 369 | server_max_window_bits = _parse_window_bits( 370 | server_max_window_bits) 371 | except ValueError as e: 372 | self._logger.debug('Bad %s parameter: %r', 373 | self._SERVER_MAX_WINDOW_BITS_PARAM, 374 | e) 375 | return None 376 | 377 | server_no_context_takeover = self._request.has_parameter( 378 | self._SERVER_NO_CONTEXT_TAKEOVER_PARAM) 379 | if (server_no_context_takeover and 380 | self._request.get_parameter_value( 381 | self._SERVER_NO_CONTEXT_TAKEOVER_PARAM) is not None): 382 | self._logger.debug('%s parameter must not have a value: %r', 383 | self._SERVER_NO_CONTEXT_TAKEOVER_PARAM, 384 | server_no_context_takeover) 385 | return None 386 | 387 | # client_max_window_bits from a client indicates whether the client can 388 | # accept client_max_window_bits from a server or not. 389 | client_client_max_window_bits = self._request.has_parameter( 390 | self._CLIENT_MAX_WINDOW_BITS_PARAM) 391 | if (client_client_max_window_bits and 392 | self._request.get_parameter_value( 393 | self._CLIENT_MAX_WINDOW_BITS_PARAM) is not None): 394 | self._logger.debug('%s parameter must not have a value in a ' 395 | 'client\'s opening handshake: %r', 396 | self._CLIENT_MAX_WINDOW_BITS_PARAM, 397 | client_client_max_window_bits) 398 | return None 399 | 400 | self._rfc1979_deflater = util._RFC1979Deflater( 401 | server_max_window_bits, server_no_context_takeover) 402 | 403 | # Note that we prepare for incoming messages compressed with window 404 | # bits upto 15 regardless of the client_max_window_bits value to be 405 | # sent to the client. 406 | self._rfc1979_inflater = util._RFC1979Inflater() 407 | 408 | self._framer = _PerMessageDeflateFramer( 409 | server_max_window_bits, server_no_context_takeover) 410 | self._framer.set_bfinal(False) 411 | self._framer.set_compress_outgoing_enabled(True) 412 | 413 | response = common.ExtensionParameter(self._request.name()) 414 | 415 | if server_max_window_bits is not None: 416 | response.add_parameter( 417 | self._SERVER_MAX_WINDOW_BITS_PARAM, 418 | str(server_max_window_bits)) 419 | 420 | if server_no_context_takeover: 421 | response.add_parameter( 422 | self._SERVER_NO_CONTEXT_TAKEOVER_PARAM, None) 423 | 424 | if self._preferred_client_max_window_bits is not None: 425 | if not client_client_max_window_bits: 426 | self._logger.debug('Processor is configured to use %s but ' 427 | 'the client cannot accept it', 428 | self._CLIENT_MAX_WINDOW_BITS_PARAM) 429 | return None 430 | response.add_parameter( 431 | self._CLIENT_MAX_WINDOW_BITS_PARAM, 432 | str(self._preferred_client_max_window_bits)) 433 | 434 | if self._client_no_context_takeover: 435 | response.add_parameter( 436 | self._CLIENT_NO_CONTEXT_TAKEOVER_PARAM, None) 437 | 438 | self._logger.debug( 439 | 'Enable %s extension (' 440 | 'request: server_max_window_bits=%s; ' 441 | 'server_no_context_takeover=%r, ' 442 | 'response: client_max_window_bits=%s; ' 443 | 'client_no_context_takeover=%r)' % 444 | (self._request.name(), 445 | server_max_window_bits, 446 | server_no_context_takeover, 447 | self._preferred_client_max_window_bits, 448 | self._client_no_context_takeover)) 449 | 450 | return response 451 | 452 | def _setup_stream_options_internal(self, stream_options): 453 | self._framer.setup_stream_options(stream_options) 454 | 455 | def set_client_max_window_bits(self, value): 456 | """If this option is specified, this class adds the 457 | client_max_window_bits extension parameter to the handshake response, 458 | but doesn't reduce the LZ77 sliding window size of its inflater. 459 | I.e., you can use this for testing client implementation but cannot 460 | reduce memory usage of this class. 461 | 462 | If this method has been called with True and an offer without the 463 | client_max_window_bits extension parameter is received, 464 | - (When processing the permessage-deflate extension) this processor 465 | declines the request. 466 | - (When processing the permessage-compress extension) this processor 467 | accepts the request. 468 | """ 469 | 470 | self._preferred_client_max_window_bits = value 471 | 472 | def set_client_no_context_takeover(self, value): 473 | """If this option is specified, this class adds the 474 | client_no_context_takeover extension parameter to the handshake 475 | response, but doesn't reset inflater for each message. I.e., you can 476 | use this for testing client implementation but cannot reduce memory 477 | usage of this class. 478 | """ 479 | 480 | self._client_no_context_takeover = value 481 | 482 | def set_bfinal(self, value): 483 | self._framer.set_bfinal(value) 484 | 485 | def enable_outgoing_compression(self): 486 | self._framer.set_compress_outgoing_enabled(True) 487 | 488 | def disable_outgoing_compression(self): 489 | self._framer.set_compress_outgoing_enabled(False) 490 | 491 | 492 | class _PerMessageDeflateFramer(object): 493 | """A framer for extensions with per-message DEFLATE feature.""" 494 | 495 | def __init__(self, deflate_max_window_bits, deflate_no_context_takeover): 496 | self._logger = util.get_class_logger(self) 497 | 498 | self._rfc1979_deflater = util._RFC1979Deflater( 499 | deflate_max_window_bits, deflate_no_context_takeover) 500 | 501 | self._rfc1979_inflater = util._RFC1979Inflater() 502 | 503 | self._bfinal = False 504 | 505 | self._compress_outgoing_enabled = False 506 | 507 | # True if a message is fragmented and compression is ongoing. 508 | self._compress_ongoing = False 509 | 510 | # Calculates 511 | # (Total outgoing bytes supplied to this filter) / 512 | # (Total bytes sent to the network after applying this filter) 513 | self._outgoing_average_ratio_calculator = _AverageRatioCalculator() 514 | 515 | # Calculates 516 | # (Total bytes received from the network) / 517 | # (Total incoming bytes obtained after applying this filter) 518 | self._incoming_average_ratio_calculator = _AverageRatioCalculator() 519 | 520 | def set_bfinal(self, value): 521 | self._bfinal = value 522 | 523 | def set_compress_outgoing_enabled(self, value): 524 | self._compress_outgoing_enabled = value 525 | 526 | def _process_incoming_message(self, message, decompress): 527 | if not decompress: 528 | return message 529 | 530 | received_payload_size = len(message) 531 | self._incoming_average_ratio_calculator.add_result_bytes( 532 | received_payload_size) 533 | 534 | message = self._rfc1979_inflater.filter(message) 535 | 536 | filtered_payload_size = len(message) 537 | self._incoming_average_ratio_calculator.add_original_bytes( 538 | filtered_payload_size) 539 | 540 | _log_incoming_compression_ratio( 541 | self._logger, 542 | received_payload_size, 543 | filtered_payload_size, 544 | self._incoming_average_ratio_calculator.get_average_ratio()) 545 | 546 | return message 547 | 548 | def _process_outgoing_message(self, message, end, binary): 549 | if not binary: 550 | message = message.encode('utf-8') 551 | 552 | if not self._compress_outgoing_enabled: 553 | return message 554 | 555 | original_payload_size = len(message) 556 | self._outgoing_average_ratio_calculator.add_original_bytes( 557 | original_payload_size) 558 | 559 | message = self._rfc1979_deflater.filter( 560 | message, end=end, bfinal=self._bfinal) 561 | 562 | filtered_payload_size = len(message) 563 | self._outgoing_average_ratio_calculator.add_result_bytes( 564 | filtered_payload_size) 565 | 566 | _log_outgoing_compression_ratio( 567 | self._logger, 568 | original_payload_size, 569 | filtered_payload_size, 570 | self._outgoing_average_ratio_calculator.get_average_ratio()) 571 | 572 | if not self._compress_ongoing: 573 | self._outgoing_frame_filter.set_compression_bit() 574 | self._compress_ongoing = not end 575 | return message 576 | 577 | def _process_incoming_frame(self, frame): 578 | if frame.rsv1 == 1 and not common.is_control_opcode(frame.opcode): 579 | self._incoming_message_filter.decompress_next_message() 580 | frame.rsv1 = 0 581 | 582 | def _process_outgoing_frame(self, frame, compression_bit): 583 | if (not compression_bit or 584 | common.is_control_opcode(frame.opcode)): 585 | return 586 | 587 | frame.rsv1 = 1 588 | 589 | def setup_stream_options(self, stream_options): 590 | """Creates filters and sets them to the StreamOptions.""" 591 | 592 | class _OutgoingMessageFilter(object): 593 | 594 | def __init__(self, parent): 595 | self._parent = parent 596 | 597 | def filter(self, message, end=True, binary=False): 598 | return self._parent._process_outgoing_message( 599 | message, end, binary) 600 | 601 | class _IncomingMessageFilter(object): 602 | 603 | def __init__(self, parent): 604 | self._parent = parent 605 | self._decompress_next_message = False 606 | 607 | def decompress_next_message(self): 608 | self._decompress_next_message = True 609 | 610 | def filter(self, message): 611 | message = self._parent._process_incoming_message( 612 | message, self._decompress_next_message) 613 | self._decompress_next_message = False 614 | return message 615 | 616 | self._outgoing_message_filter = _OutgoingMessageFilter(self) 617 | self._incoming_message_filter = _IncomingMessageFilter(self) 618 | stream_options.outgoing_message_filters.append( 619 | self._outgoing_message_filter) 620 | stream_options.incoming_message_filters.append( 621 | self._incoming_message_filter) 622 | 623 | class _OutgoingFrameFilter(object): 624 | 625 | def __init__(self, parent): 626 | self._parent = parent 627 | self._set_compression_bit = False 628 | 629 | def set_compression_bit(self): 630 | self._set_compression_bit = True 631 | 632 | def filter(self, frame): 633 | self._parent._process_outgoing_frame( 634 | frame, self._set_compression_bit) 635 | self._set_compression_bit = False 636 | 637 | class _IncomingFrameFilter(object): 638 | 639 | def __init__(self, parent): 640 | self._parent = parent 641 | 642 | def filter(self, frame): 643 | self._parent._process_incoming_frame(frame) 644 | 645 | self._outgoing_frame_filter = _OutgoingFrameFilter(self) 646 | self._incoming_frame_filter = _IncomingFrameFilter(self) 647 | stream_options.outgoing_frame_filters.append( 648 | self._outgoing_frame_filter) 649 | stream_options.incoming_frame_filters.append( 650 | self._incoming_frame_filter) 651 | 652 | stream_options.encode_text_message_to_utf8 = False 653 | 654 | 655 | _available_processors[common.PERMESSAGE_DEFLATE_EXTENSION] = ( 656 | PerMessageDeflateExtensionProcessor) 657 | # TODO(tyoshino): Reorganize class names. 658 | _compression_extension_names.append('deflate') 659 | 660 | 661 | class MuxExtensionProcessor(ExtensionProcessorInterface): 662 | """WebSocket multiplexing extension processor.""" 663 | 664 | _QUOTA_PARAM = 'quota' 665 | 666 | def __init__(self, request): 667 | ExtensionProcessorInterface.__init__(self, request) 668 | self._quota = 0 669 | self._extensions = [] 670 | 671 | def name(self): 672 | return common.MUX_EXTENSION 673 | 674 | def check_consistency_with_other_processors(self, processors): 675 | before_mux = True 676 | for processor in processors: 677 | name = processor.name() 678 | if name == self.name(): 679 | before_mux = False 680 | continue 681 | if not processor.is_active(): 682 | continue 683 | if before_mux: 684 | # Mux extension cannot be used after extensions 685 | # that depend on frame boundary, extension data field, or any 686 | # reserved bits which are attributed to each frame. 687 | if (name == common.DEFLATE_FRAME_EXTENSION or 688 | name == common.X_WEBKIT_DEFLATE_FRAME_EXTENSION): 689 | self.set_active(False) 690 | return 691 | else: 692 | # Mux extension should not be applied before any history-based 693 | # compression extension. 694 | if (name == 'deflate' or 695 | name == common.DEFLATE_FRAME_EXTENSION or 696 | name == common.X_WEBKIT_DEFLATE_FRAME_EXTENSION): 697 | self.set_active(False) 698 | return 699 | 700 | def _get_extension_response_internal(self): 701 | self._active = False 702 | quota = self._request.get_parameter_value(self._QUOTA_PARAM) 703 | if quota is not None: 704 | try: 705 | quota = int(quota) 706 | except ValueError as e: 707 | return None 708 | if quota < 0 or quota >= 2 ** 32: 709 | return None 710 | self._quota = quota 711 | 712 | self._active = True 713 | return common.ExtensionParameter(common.MUX_EXTENSION) 714 | 715 | def _setup_stream_options_internal(self, stream_options): 716 | pass 717 | 718 | def set_quota(self, quota): 719 | self._quota = quota 720 | 721 | def quota(self): 722 | return self._quota 723 | 724 | def set_extensions(self, extensions): 725 | self._extensions = extensions 726 | 727 | def extensions(self): 728 | return self._extensions 729 | 730 | 731 | _available_processors[common.MUX_EXTENSION] = MuxExtensionProcessor 732 | 733 | 734 | def get_extension_processor(extension_request): 735 | """Given an ExtensionParameter representing an extension offer received 736 | from a client, configures and returns an instance of the corresponding 737 | extension processor class. 738 | """ 739 | 740 | processor_class = _available_processors.get(extension_request.name()) 741 | if processor_class is None: 742 | return None 743 | return processor_class(extension_request) 744 | 745 | 746 | def is_compression_extension(extension_name): 747 | return extension_name in _compression_extension_names 748 | 749 | 750 | # vi:sts=4 sw=4 et 751 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/http_header_util.py: -------------------------------------------------------------------------------- 1 | # Copyright 2011, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """Utilities for parsing and formatting headers that follow the grammar defined 32 | in HTTP RFC http://www.ietf.org/rfc/rfc2616.txt. 33 | """ 34 | 35 | try: 36 | from urllib.parse import urlparse 37 | except ImportError: 38 | import urlparse 39 | 40 | _SEPARATORS = '()<>@,;:\\"/[]?={} \t' 41 | 42 | 43 | def _is_char(c): 44 | """Returns true iff c is in CHAR as specified in HTTP RFC.""" 45 | 46 | return ord(c) <= 127 47 | 48 | 49 | def _is_ctl(c): 50 | """Returns true iff c is in CTL as specified in HTTP RFC.""" 51 | 52 | return ord(c) <= 31 or ord(c) == 127 53 | 54 | 55 | class ParsingState(object): 56 | 57 | def __init__(self, data): 58 | self.data = data 59 | self.head = 0 60 | 61 | 62 | def peek(state, pos=0): 63 | """Peeks the character at pos from the head of data.""" 64 | 65 | if state.head + pos >= len(state.data): 66 | return None 67 | 68 | return state.data[state.head + pos] 69 | 70 | 71 | def consume(state, amount=1): 72 | """Consumes specified amount of bytes from the head and returns the 73 | consumed bytes. If there's not enough bytes to consume, returns None. 74 | """ 75 | 76 | if state.head + amount > len(state.data): 77 | return None 78 | 79 | result = state.data[state.head:state.head + amount] 80 | state.head = state.head + amount 81 | return result 82 | 83 | 84 | def consume_string(state, expected): 85 | """Given a parsing state and a expected string, consumes the string from 86 | the head. Returns True if consumed successfully. Otherwise, returns 87 | False. 88 | """ 89 | 90 | pos = 0 91 | 92 | for c in expected: 93 | if c != peek(state, pos): 94 | return False 95 | pos += 1 96 | 97 | consume(state, pos) 98 | return True 99 | 100 | 101 | def consume_lws(state): 102 | """Consumes a LWS from the head. Returns True if any LWS is consumed. 103 | Otherwise, returns False. 104 | 105 | LWS = [CRLF] 1*( SP | HT ) 106 | """ 107 | 108 | original_head = state.head 109 | 110 | consume_string(state, '\r\n') 111 | 112 | pos = 0 113 | 114 | while True: 115 | c = peek(state, pos) 116 | if c == ' ' or c == '\t': 117 | pos += 1 118 | else: 119 | if pos == 0: 120 | state.head = original_head 121 | return False 122 | else: 123 | consume(state, pos) 124 | return True 125 | 126 | 127 | def consume_lwses(state): 128 | """Consumes *LWS from the head.""" 129 | 130 | while consume_lws(state): 131 | pass 132 | 133 | 134 | def consume_token(state): 135 | """Consumes a token from the head. Returns the token or None if no token 136 | was found. 137 | """ 138 | 139 | pos = 0 140 | 141 | while True: 142 | c = peek(state, pos) 143 | if c is None or c in _SEPARATORS or _is_ctl(c) or not _is_char(c): 144 | if pos == 0: 145 | return None 146 | 147 | return consume(state, pos) 148 | else: 149 | pos += 1 150 | 151 | 152 | def consume_token_or_quoted_string(state): 153 | """Consumes a token or a quoted-string, and returns the token or unquoted 154 | string. If no token or quoted-string was found, returns None. 155 | """ 156 | 157 | original_head = state.head 158 | 159 | if not consume_string(state, '"'): 160 | return consume_token(state) 161 | 162 | result = [] 163 | 164 | expect_quoted_pair = False 165 | 166 | while True: 167 | if not expect_quoted_pair and consume_lws(state): 168 | result.append(' ') 169 | continue 170 | 171 | c = consume(state) 172 | if c is None: 173 | # quoted-string is not enclosed with double quotation 174 | state.head = original_head 175 | return None 176 | elif expect_quoted_pair: 177 | expect_quoted_pair = False 178 | if _is_char(c): 179 | result.append(c) 180 | else: 181 | # Non CHAR character found in quoted-pair 182 | state.head = original_head 183 | return None 184 | elif c == '\\': 185 | expect_quoted_pair = True 186 | elif c == '"': 187 | return ''.join(result) 188 | elif _is_ctl(c): 189 | # Invalid character %r found in qdtext 190 | state.head = original_head 191 | return None 192 | else: 193 | result.append(c) 194 | 195 | 196 | def quote_if_necessary(s): 197 | """Quotes arbitrary string into quoted-string.""" 198 | 199 | quote = False 200 | if s == '': 201 | return '""' 202 | 203 | result = [] 204 | for c in s: 205 | if c == '"' or c in _SEPARATORS or _is_ctl(c) or not _is_char(c): 206 | quote = True 207 | 208 | if c == '"' or _is_ctl(c): 209 | result.append('\\' + c) 210 | else: 211 | result.append(c) 212 | 213 | if quote: 214 | return '"' + ''.join(result) + '"' 215 | else: 216 | return ''.join(result) 217 | 218 | 219 | def parse_uri(uri): 220 | """Parse absolute URI then return host, port and resource.""" 221 | 222 | parsed = urlparse.urlsplit(uri) 223 | if parsed.scheme != 'wss' and parsed.scheme != 'ws': 224 | # |uri| must be a relative URI. 225 | # TODO(toyoshim): Should validate |uri|. 226 | return None, None, uri 227 | 228 | if parsed.hostname is None: 229 | return None, None, None 230 | 231 | port = None 232 | try: 233 | port = parsed.port 234 | except ValueError as e: 235 | # port property cause ValueError on invalid null port description like 236 | # 'ws://host:/path'. 237 | return None, None, None 238 | 239 | if port is None: 240 | if parsed.scheme == 'ws': 241 | port = 80 242 | else: 243 | port = 443 244 | 245 | path = parsed.path 246 | if not path: 247 | path += '/' 248 | if parsed.query: 249 | path += '?' + parsed.query 250 | if parsed.fragment: 251 | path += '#' + parsed.fragment 252 | 253 | return parsed.hostname, port, path 254 | 255 | 256 | #try: 257 | # urlparse.uses_netloc.index('ws') 258 | # urlparse.uses_netloc.index('ws') 259 | #except ValueError as e: 260 | # # urlparse in Python2.5.1 doesn't have 'ws' and 'wss' entries. 261 | # urlparse.uses_netloc.append('ws') 262 | # urlparse.uses_netloc.append('wss') 263 | 264 | 265 | # vi:sts=4 sw=4 et 266 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/stream.py: -------------------------------------------------------------------------------- 1 | # Copyright 2011, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """This file exports public symbols.""" 32 | 33 | 34 | from mod_pywebsocket._stream_base import BadOperationException 35 | from mod_pywebsocket._stream_base import ConnectionTerminatedException 36 | from mod_pywebsocket._stream_base import InvalidFrameException 37 | from mod_pywebsocket._stream_base import InvalidUTF8Exception 38 | from mod_pywebsocket._stream_base import UnsupportedFrameException 39 | from mod_pywebsocket._stream_hixie75 import StreamHixie75 40 | from mod_pywebsocket._stream_hybi import Frame 41 | from mod_pywebsocket._stream_hybi import Stream 42 | from mod_pywebsocket._stream_hybi import StreamOptions 43 | 44 | # These methods are intended to be used by WebSocket client developers to have 45 | # their implementations receive broken data in tests. 46 | from mod_pywebsocket._stream_hybi import create_close_frame 47 | from mod_pywebsocket._stream_hybi import create_header 48 | from mod_pywebsocket._stream_hybi import create_length_header 49 | from mod_pywebsocket._stream_hybi import create_ping_frame 50 | from mod_pywebsocket._stream_hybi import create_pong_frame 51 | from mod_pywebsocket._stream_hybi import create_binary_frame 52 | from mod_pywebsocket._stream_hybi import create_text_frame 53 | from mod_pywebsocket._stream_hybi import create_closing_handshake_body 54 | 55 | 56 | # vi:sts=4 sw=4 et 57 | -------------------------------------------------------------------------------- /kiwiclient/mod_pywebsocket/util.py: -------------------------------------------------------------------------------- 1 | # Copyright 2011, Google Inc. 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are 6 | # met: 7 | # 8 | # * Redistributions of source code must retain the above copyright 9 | # notice, this list of conditions and the following disclaimer. 10 | # * Redistributions in binary form must reproduce the above 11 | # copyright notice, this list of conditions and the following disclaimer 12 | # in the documentation and/or other materials provided with the 13 | # distribution. 14 | # * Neither the name of Google Inc. nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | 30 | 31 | """WebSocket utilities.""" 32 | 33 | 34 | import array 35 | import errno 36 | 37 | # Import hash classes from a module available and recommended for each Python 38 | # version and re-export those symbol. Use sha and md5 module in Python 2.4, and 39 | # hashlib module in Python 2.6. 40 | try: 41 | import hashlib 42 | md5_hash = hashlib.md5 43 | sha1_hash = hashlib.sha1 44 | except ImportError: 45 | import md5 46 | import sha 47 | md5_hash = md5.md5 48 | sha1_hash = sha.sha 49 | 50 | try: 51 | from StringIO import StringIO 52 | except ImportError: 53 | from io import StringIO 54 | 55 | import logging 56 | import os 57 | import re 58 | import socket 59 | import traceback 60 | import zlib 61 | 62 | try: 63 | from mod_pywebsocket import fast_masking 64 | except ImportError: 65 | pass 66 | 67 | 68 | def get_stack_trace(): 69 | """Get the current stack trace as string. 70 | 71 | This is needed to support Python 2.3. 72 | TODO: Remove this when we only support Python 2.4 and above. 73 | Use traceback.format_exc instead. 74 | """ 75 | out = StringIO.StringIO() 76 | traceback.print_exc(file=out) 77 | return out.getvalue() 78 | 79 | 80 | def prepend_message_to_exception(message, exc): 81 | """Prepend message to the exception.""" 82 | exc.args = (message + str(exc),) 83 | return 84 | 85 | 86 | def __translate_interp(interp, cygwin_path): 87 | """Translate interp program path for Win32 python to run cygwin program 88 | (e.g. perl). Note that it doesn't support path that contains space, 89 | which is typically true for Unix, where #!-script is written. 90 | For Win32 python, cygwin_path is a directory of cygwin binaries. 91 | 92 | Args: 93 | interp: interp command line 94 | cygwin_path: directory name of cygwin binary, or None 95 | Returns: 96 | translated interp command line. 97 | """ 98 | if not cygwin_path: 99 | return interp 100 | m = re.match('^[^ ]*/([^ ]+)( .*)?', interp) 101 | if m: 102 | cmd = os.path.join(cygwin_path, m.group(1)) 103 | return cmd + m.group(2) 104 | return interp 105 | 106 | 107 | def get_script_interp(script_path, cygwin_path=None): 108 | r"""Get #!-interpreter command line from the script. 109 | 110 | It also fixes command path. When Cygwin Python is used, e.g. in WebKit, 111 | it could run "/usr/bin/perl -wT hello.pl". 112 | When Win32 Python is used, e.g. in Chromium, it couldn't. So, fix 113 | "/usr/bin/perl" to "\perl.exe". 114 | 115 | Args: 116 | script_path: pathname of the script 117 | cygwin_path: directory name of cygwin binary, or None 118 | Returns: 119 | #!-interpreter command line, or None if it is not #!-script. 120 | """ 121 | fp = open(script_path) 122 | line = fp.readline() 123 | fp.close() 124 | m = re.match('^#!(.*)', line) 125 | if m: 126 | return __translate_interp(m.group(1), cygwin_path) 127 | return None 128 | 129 | 130 | def wrap_popen3_for_win(cygwin_path): 131 | """Wrap popen3 to support #!-script on Windows. 132 | 133 | Args: 134 | cygwin_path: path for cygwin binary if command path is needed to be 135 | translated. None if no translation required. 136 | """ 137 | __orig_popen3 = os.popen3 138 | 139 | def __wrap_popen3(cmd, mode='t', bufsize=-1): 140 | cmdline = cmd.split(' ') 141 | interp = get_script_interp(cmdline[0], cygwin_path) 142 | if interp: 143 | cmd = interp + ' ' + cmd 144 | return __orig_popen3(cmd, mode, bufsize) 145 | 146 | os.popen3 = __wrap_popen3 147 | 148 | 149 | def hexify(s): 150 | r= ' '.join(map(lambda x: '%02x' % x, bytearray(s))) 151 | return r 152 | 153 | 154 | def get_class_logger(o): 155 | """Return the logging class information.""" 156 | return logging.getLogger( 157 | '%s.%s' % (o.__class__.__module__, o.__class__.__name__)) 158 | 159 | 160 | class NoopMasker(object): 161 | """A NoOp masking object. 162 | 163 | This has the same interface as RepeatedXorMasker but just returns 164 | the string passed in without making any change. 165 | """ 166 | 167 | def __init__(self): 168 | """NoOp.""" 169 | pass 170 | 171 | def mask(self, s): 172 | """NoOp.""" 173 | return s 174 | 175 | 176 | class RepeatedXorMasker(object): 177 | 178 | """A masking object that applies XOR on the string. 179 | 180 | Applies XOR on the string given to mask method with the masking bytes 181 | given to the constructor repeatedly. This object remembers the position 182 | in the masking bytes the last mask method call ended and resumes from 183 | that point on the next mask method call. 184 | """ 185 | 186 | def __init__(self, masking_key): 187 | self._masking_key = masking_key 188 | self._masking_key_index = 0 189 | 190 | def _mask_using_swig(self, s): 191 | """Perform the mask via SWIG.""" 192 | masked_data = fast_masking.mask( 193 | s, self._masking_key, self._masking_key_index) 194 | self._masking_key_index = ( 195 | (self._masking_key_index + len(s)) % len(self._masking_key)) 196 | return masked_data 197 | 198 | def _mask_using_array(self, s): 199 | """Perform the mask via python.""" 200 | result = array.array('B') 201 | result.fromstring(bytes(s)) 202 | 203 | # Use temporary local variables to eliminate the cost to access 204 | # attributes 205 | if type(self._masking_key[0]) is int: 206 | masking_key = [x for x in self._masking_key] 207 | else: 208 | masking_key = map(ord, self._masking_key) 209 | masking_key_size = len(masking_key) 210 | masking_key_index = self._masking_key_index 211 | 212 | for i in range(len(result)): 213 | result[i] ^= masking_key[masking_key_index] 214 | masking_key_index = (masking_key_index + 1) % masking_key_size 215 | 216 | self._masking_key_index = masking_key_index 217 | 218 | return result.tostring() 219 | 220 | if 'fast_masking' in globals(): 221 | mask = _mask_using_swig 222 | else: 223 | mask = _mask_using_array 224 | 225 | 226 | # By making wbits option negative, we can suppress CMF/FLG (2 octet) and 227 | # ADLER32 (4 octet) fields of zlib so that we can use zlib module just as 228 | # deflate library. DICTID won't be added as far as we don't set dictionary. 229 | # LZ77 window of 32K will be used for both compression and decompression. 230 | # For decompression, we can just use 32K to cover any windows size. For 231 | # compression, we use 32K so receivers must use 32K. 232 | # 233 | # Compression level is Z_DEFAULT_COMPRESSION. We don't have to match level 234 | # to decode. 235 | # 236 | # See zconf.h, deflate.cc, inflate.cc of zlib library, and zlibmodule.c of 237 | # Python. See also RFC1950 (ZLIB 3.3). 238 | 239 | 240 | class _Deflater(object): 241 | 242 | def __init__(self, window_bits): 243 | self._logger = get_class_logger(self) 244 | 245 | self._compress = zlib.compressobj( 246 | zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -window_bits) 247 | 248 | def compress(self, bytes): 249 | compressed_bytes = self._compress.compress(bytes) 250 | self._logger.debug('Compress input %r', bytes) 251 | self._logger.debug('Compress result %r', compressed_bytes) 252 | return compressed_bytes 253 | 254 | def compress_and_flush(self, bytes): 255 | compressed_bytes = self._compress.compress(bytes) 256 | compressed_bytes += self._compress.flush(zlib.Z_SYNC_FLUSH) 257 | self._logger.debug('Compress input %r', bytes) 258 | self._logger.debug('Compress result %r', compressed_bytes) 259 | return compressed_bytes 260 | 261 | def compress_and_finish(self, bytes): 262 | compressed_bytes = self._compress.compress(bytes) 263 | compressed_bytes += self._compress.flush(zlib.Z_FINISH) 264 | self._logger.debug('Compress input %r', bytes) 265 | self._logger.debug('Compress result %r', compressed_bytes) 266 | return compressed_bytes 267 | 268 | 269 | class _Inflater(object): 270 | 271 | def __init__(self, window_bits): 272 | self._logger = get_class_logger(self) 273 | self._window_bits = window_bits 274 | 275 | self._unconsumed = '' 276 | 277 | self.reset() 278 | 279 | def decompress(self, size): 280 | if not (size == -1 or size > 0): 281 | raise Exception('size must be -1 or positive') 282 | 283 | data = '' 284 | 285 | while True: 286 | if size == -1: 287 | data += self._decompress.decompress(self._unconsumed) 288 | # See Python bug http://bugs.python.org/issue12050 to 289 | # understand why the same code cannot be used for updating 290 | # self._unconsumed for here and else block. 291 | self._unconsumed = '' 292 | else: 293 | data += self._decompress.decompress( 294 | self._unconsumed, size - len(data)) 295 | self._unconsumed = self._decompress.unconsumed_tail 296 | if self._decompress.unused_data: 297 | # Encountered a last block (i.e. a block with BFINAL = 1) and 298 | # found a new stream (unused_data). We cannot use the same 299 | # zlib.Decompress object for the new stream. Create a new 300 | # Decompress object to decompress the new one. 301 | # 302 | # It's fine to ignore unconsumed_tail if unused_data is not 303 | # empty. 304 | self._unconsumed = self._decompress.unused_data 305 | self.reset() 306 | if size >= 0 and len(data) == size: 307 | # data is filled. Don't call decompress again. 308 | break 309 | else: 310 | # Re-invoke Decompress.decompress to try to decompress all 311 | # available bytes before invoking read which blocks until 312 | # any new byte is available. 313 | continue 314 | else: 315 | # Here, since unused_data is empty, even if unconsumed_tail is 316 | # not empty, bytes of requested length are already in data. We 317 | # don't have to "continue" here. 318 | break 319 | 320 | if data: 321 | self._logger.debug('Decompressed %r', data) 322 | return data 323 | 324 | def append(self, data): 325 | self._logger.debug('Appended %r', data) 326 | self._unconsumed += data 327 | 328 | def reset(self): 329 | self._logger.debug('Reset') 330 | self._decompress = zlib.decompressobj(-self._window_bits) 331 | 332 | 333 | # Compresses/decompresses given octets using the method introduced in RFC1979. 334 | 335 | 336 | class _RFC1979Deflater(object): 337 | """A compressor class that applies DEFLATE to given byte sequence and 338 | flushes using the algorithm described in the RFC1979 section 2.1. 339 | """ 340 | 341 | def __init__(self, window_bits, no_context_takeover): 342 | self._deflater = None 343 | if window_bits is None: 344 | window_bits = zlib.MAX_WBITS 345 | self._window_bits = window_bits 346 | self._no_context_takeover = no_context_takeover 347 | 348 | def filter(self, bytes, end=True, bfinal=False): 349 | if self._deflater is None: 350 | self._deflater = _Deflater(self._window_bits) 351 | 352 | if bfinal: 353 | result = self._deflater.compress_and_finish(bytes) 354 | # Add a padding block with BFINAL = 0 and BTYPE = 0. 355 | result = result + chr(0) 356 | self._deflater = None 357 | return result 358 | 359 | result = self._deflater.compress_and_flush(bytes) 360 | if end: 361 | # Strip last 4 octets which is LEN and NLEN field of a 362 | # non-compressed block added for Z_SYNC_FLUSH. 363 | result = result[:-4] 364 | 365 | if self._no_context_takeover and end: 366 | self._deflater = None 367 | 368 | return result 369 | 370 | 371 | class _RFC1979Inflater(object): 372 | """A decompressor class a la RFC1979. 373 | 374 | A decompressor class for byte sequence compressed and flushed following 375 | the algorithm described in the RFC1979 section 2.1. 376 | """ 377 | 378 | def __init__(self, window_bits=zlib.MAX_WBITS): 379 | self._inflater = _Inflater(window_bits) 380 | 381 | def filter(self, bytes): 382 | # Restore stripped LEN and NLEN field of a non-compressed block added 383 | # for Z_SYNC_FLUSH. 384 | self._inflater.append(bytes + '\x00\x00\xff\xff') 385 | return self._inflater.decompress(-1) 386 | 387 | 388 | class DeflateSocket(object): 389 | """A wrapper class for socket object to intercept send and recv to perform 390 | deflate compression and decompression transparently. 391 | """ 392 | 393 | # Size of the buffer passed to recv to receive compressed data. 394 | _RECV_SIZE = 4096 395 | 396 | def __init__(self, socket): 397 | self._socket = socket 398 | 399 | self._logger = get_class_logger(self) 400 | 401 | self._deflater = _Deflater(zlib.MAX_WBITS) 402 | self._inflater = _Inflater(zlib.MAX_WBITS) 403 | 404 | def recv(self, size): 405 | """Receives data from the socket specified on the construction up 406 | to the specified size. Once any data is available, returns it even 407 | if it's smaller than the specified size. 408 | """ 409 | 410 | # TODO(tyoshino): Allow call with size=0. It should block until any 411 | # decompressed data is available. 412 | if size <= 0: 413 | raise Exception('Non-positive size passed') 414 | while True: 415 | data = self._inflater.decompress(size) 416 | if len(data) != 0: 417 | return data 418 | 419 | read_data = self._socket.recv(DeflateSocket._RECV_SIZE) 420 | if not read_data: 421 | return '' 422 | self._inflater.append(read_data) 423 | 424 | def sendall(self, bytes): 425 | self.send(bytes) 426 | 427 | def send(self, bytes): 428 | self._socket.sendall(self._deflater.compress_and_flush(bytes)) 429 | return len(bytes) 430 | 431 | 432 | # vi:sts=4 sw=4 et 433 | -------------------------------------------------------------------------------- /maps/How to find the other maps.txt: -------------------------------------------------------------------------------- 1 | Additional World maps are stored here: 2 | http://81.93.247.141/~linkz/directTDoA/maps/ 3 | 4 | download them to /maps directory and they will be available in the "Browse map folder" -------------------------------------------------------------------------------- /maps/directKiwi_map_grayscale_with_sea.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/llinkz/directKiwi/c18ef280edbb9c04d06c2a45e004e4a3bf1f8698/maps/directKiwi_map_grayscale_with_sea.jpg -------------------------------------------------------------------------------- /maps/directTDoA_map.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/llinkz/directKiwi/c18ef280edbb9c04d06c2a45e004e4a3bf1f8698/maps/directTDoA_map.jpg -------------------------------------------------------------------------------- /setup.bat: -------------------------------------------------------------------------------- 1 | :: directKiwi setup file for windows 2 | @echo off 3 | python -m pip install --upgrade pip 4 | python -m pip install numpy pillow future requests sounddevice samplerate matplotlib 5 | echo The setup is now finished. To start the software, double_click on directKiwi.bat 6 | pause -------------------------------------------------------------------------------- /setup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | python -m pip install --upgrade pip 3 | python -m pip install numpy pillow future requests sounddevice samplerate matplotlib 4 | echo -e "The setup is now finished.\nTo start the software from console, type ./directKiwi.py" 5 | sleep 5 --------------------------------------------------------------------------------