$ pip install PySocks
219 |
220 | Then make sure your code (like the example) has import socks.
221 |
222 | ### Run the tor connect example
223 |
224 | Just simply run it from the terminal window:
225 |
226 | ```python3
227 | python tor-example.py
228 | ```
229 |
230 | This will return the DuckDuckGo .onion html as proof that it is working.
231 |
232 | ## Google Search Example
233 |
234 | The Google seach portion of the pattern library was very useful. This example shows you that you can compare the popularity of phrases or sets of terms together using percentages and the sort() command. It selects 10 random words to search on from the imported included dictionary list that is in the assets folder. This doesn't work anymore. Thanks for nothing Google.
235 |
236 | ### Run the example
237 |
238 | ```python3
239 | python3 pattern-example-google.py -c 'sexy'
240 | ```
241 |
242 | Returns:
243 |
244 | ```python3
245 | 89.13% "sexy seemed"
246 | 2.17% "sexy impassive"
247 | 1.09% "sexy spiegels"
248 | 1.09% "sexy slumping"
249 | 1.09% "sexy quietuses"
250 | 1.09% "sexy noncooperation"
251 | 1.09% "sexy miriness"
252 | 1.09% "sexy incompliancy"
253 | 1.09% "sexy evaporators"
254 | 1.09% "sexy cudgeler"
255 | ```
256 |
257 | ## hug example
258 |
259 | hug is a great easy-to-use api to help route things on your web app
260 |
261 | ### Run the example
262 |
263 | ```python3
264 | python3 hug_api_example.py
265 | ```
266 |
267 | This will output hug and start a listener process on 127.0.0.1:8000
268 |
269 | Then you can go to http://localhost:8000/happy_birthday?name=Hug&age=1 and see the output.
270 |
271 | ## base64 Example
272 |
273 | Converting data to base64 ensure a nice obsfuscation layer for data transport.
274 |
275 | ### Run the base64 example
276 |
277 | ```python3
278 | python3 base64_example.py
279 | ```
280 |
281 | This will output a html string that is encoded into base64.
282 |
283 | ## Html to Text Example
284 |
285 | Beautiful Soup is a great library to parse and select html or iterate through the DOM.
286 | For this example to work you need to install Beautiful Soup:
287 |
288 | ```bash
289 | uv add beautifulsoup4
290 | ```
291 |
292 | ### Run the Example
293 |
294 | ```python3
295 | python3 example-html2plaintext.py
296 | ```
297 |
298 | #### Returns
299 |
300 | ```python3
301 | [*-*]Before html with text:
302 | ------------------
303 |
304 |
305 | I hope you enjoy this example.
311 | ------------------ 312 | 313 | 314 | 315 | [*-*]After cleanMe() function: 316 | ------------------- 317 | THIS IS AN EXAMPLE by @jamescampbell 318 | Hello World 319 | I hope you enjoy this example. 320 | ------------------- 321 | ``` 322 | 323 | ## FuzzyWuzzy Example 324 | 325 | This example searches for 'cowboy' and returns Dallas Cowboys as the closest match from the list available. 326 | 327 | ### Run the Example 328 | 329 | ```python3 330 | python3 fuzzywuzzy-example.py 331 | ``` 332 | 333 | #### Returns 334 | 335 | ```python3 336 | Dallas Cowboys, 90 337 | ``` 338 | 339 | ## Google Mask Example 340 | 341 | This example used to do three things, 1. sets your search term, 2 . set your number of mask search terms, and 3. selects a random user agent for each search query. Google killed their API for this, so byebye. 342 | 343 | ### Run the Example 344 | 345 | ```python3 346 | doesn't work anymore because google killed their API 347 | ``` 348 | 349 | ## Server Example 350 | 351 | This example starts an http server on localhost:10010 and returns data when you visit the page 352 | 353 | ### Run the Example 354 | 355 | ```python3 356 | python3 server-example.py 357 | ``` 358 | 359 | ## Scrapy Spider Example 360 | 361 | This example gets the list of all internal links for any domain by following all internal homepage links and their links. 362 | 363 | ### Run the Example 364 | 365 | ```python3 366 | python3 spider.py -u jamescampbell.us 367 | ``` 368 | 369 | ## Bitcoin Wallet Example 370 | 371 | This example queries the blockchain.info API for an example wallet address and returns the ip address and dates for the transactions as well as the final wallet balance. 372 | 373 | ### Run the Example 374 | 375 | ```python3 376 | python3 bitcoin-example-1.py 377 | ``` 378 | 379 | ## Exifread Example 380 | 381 | This example gets the exif data from an image file 382 | 383 | ### Run the Example 384 | 385 | ```python3 386 | python3 exif-reader.py assets/cat.jpg 387 | ``` 388 | 389 | #### Output 390 | 391 | ```python3 392 | Total tags found: 66 393 | Key: Interoperability InteroperabilityVersion, value [48, 49, 48, 48] 394 | Key: EXIF InteroperabilityOffset, value 36724 395 | Key: Image Software, value SLT-A57 v1.02 396 | Key: EXIF UserComment, value [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 397 | Key: Image Orientation, value Horizontal (normal) 398 | Key: Thumbnail JPEGInterchangeFormat, value 37012 399 | Key: Interoperability InteroperabilityIndex, value R98 400 | Key: Image ResolutionUnit, value Pixels/Inch 401 | Key: EXIF ExifImageWidth, value 4912 402 | Key: EXIF ComponentsConfiguration, value YCbCr 403 | Key: EXIF FNumber, value 28/5 404 | Key: Thumbnail Software, value SLT-A57 v1.02 405 | Key: EXIF DateTimeDigitized, value 2013:04:07 14:13:38 406 | Key: EXIF ExposureProgram, value Aperture Priority 407 | Key: EXIF DateTimeOriginal, value 2013:04:07 14:13:38 408 | Key: EXIF Sharpness, value Normal 409 | Key: EXIF RecommendedExposureIndex, value 3200 410 | Key: EXIF MakerNote, value [83, 79, 78, 89, 32, 68, 83, 67, 32, 0, 0, 0, 78, 0, 3, 16, 4, 0, 16, 0, ... ] 411 | Key: EXIF CustomRendered, value Normal 412 | Key: EXIF Saturation, value Normal 413 | Key: EXIF ExposureTime, value 1/80 414 | Key: Image Make, value SONY 415 | Key: EXIF ExifImageLength, value 3264 416 | Key: EXIF DigitalZoomRatio, value 1 417 | Key: Image Model, value SLT-A57 418 | Key: EXIF Contrast, value Normal 419 | Key: EXIF SensitivityType, value Recommended Exposure Index 420 | Key: Thumbnail Orientation, value Horizontal (normal) 421 | Key: Thumbnail YResolution, value 72 422 | Key: Thumbnail Model, value SLT-A57 423 | Key: Image PrintIM, value [80, 114, 105, 110, 116, 73, 77, 0, 48, 51, 48, 48, 0, 0, 3, 0, 2, 0, 1, 0, ... ] 424 | Key: Thumbnail Make, value SONY 425 | Key: EXIF CompressedBitsPerPixel, value 2 426 | Key: EXIF MeteringMode, value Pattern 427 | Key: EXIF MaxApertureValue, value 49/32 428 | Key: Image YCbCrPositioning, value Co-sited 429 | Key: EXIF BrightnessValue, value 303/320 430 | Key: EXIF FlashPixVersion, value 0100 431 | Key: EXIF WhiteBalance, value Auto 432 | Key: EXIF LensModel, value 50mm F1.7 433 | Key: Thumbnail YCbCrPositioning, value Co-sited 434 | Key: Image DateTime, value 2013:04:07 14:13:38 435 | Key: EXIF ExifVersion, value 0230 436 | Key: Thumbnail ImageDescription, value 437 | Key: Image ExifOffset, value 360 438 | Key: Thumbnail JPEGInterchangeFormatLength, value 7654 439 | Key: EXIF ExposureMode, value Auto Bracket 440 | Key: EXIF SceneType, value Directly Photographed 441 | Key: EXIF LensSpecification, value [50, 50, 17/10, 17/10] 442 | Key: Image XResolution, value 350 443 | Key: EXIF ExposureBiasValue, value 0 444 | Key: EXIF ColorSpace, value sRGB 445 | Key: EXIF ISOSpeedRatings, value 3200 446 | Key: EXIF SceneCaptureType, value Standard 447 | Key: EXIF FocalLengthIn35mmFilm, value 75 448 | Key: Image YResolution, value 350 449 | Key: Thumbnail DateTime, value 2013:04:07 14:13:38 450 | Key: EXIF FocalLength, value 50 451 | Key: Thumbnail Compression, value JPEG (old-style) 452 | Key: EXIF FileSource, value Digital Camera 453 | Key: EXIF Flash, value Flash did not fire, compulsory flash mode 454 | Key: Image ImageDescription, value 455 | Key: Thumbnail XResolution, value 72 456 | Key: Thumbnail ResolutionUnit, value Pixels/Inch 457 | Key: EXIF LightSource, value Unknown 458 | ``` 459 | 460 | ## Sentiment Example 461 | 462 | This example takes a test list of tweets and returns positive or negative. It works in Python 3. 463 | 464 | ### Run the Example 465 | 466 | ```python3 467 | python3 sentiment-analysis-nltk-example.py testtweets.txt 468 | ``` 469 | 470 | ### Output 471 | 472 | ```python3 473 | negative 474 | positive 475 | negative 476 | positive 477 | negative 478 | Positive count: 2 479 | Negative count: 3 480 | ``` 481 | 482 | ## hashlib example 483 | 484 | The hashlib package generates hashes from strings. This example uses the sha256 hash algorithm. 485 | 486 | ### Run the Example 487 | 488 | ```python3 489 | python3 hashlib_example.py 490 | ``` 491 | 492 | ## Proxymanager Example 493 | 494 | This example uses urllib3 in Python 3 to connect through a privoxy connection and return status, headers, and content. 495 | 496 | ### Run the Example 497 | 498 | ```python3 499 | python3 urllib3-proxymanager-example.py 500 | ``` 501 | 502 | #### Output 503 | 504 | ```python3 505 | 200 506 | HTTPHeaderDict({'Content-Length': '5255', 'Proxy-Connection': 'keep-alive', 'ETag': '"564e8118-1487"', 'Server': 'nginx', 'Cache-Control': 'no-cache', 'Expires': 'Fri, 20 Nov 2015 02:15:59 GMT', 'Accept-Ranges': 'bytes', 'Content-Type': 'text/html; charset=UTF-8', 'Connection': 'keep-alive', 'Date': 'Fri, 20 Nov 2015 02:16:00 GMT'}) 507 | 508 | 509 | 510 | 511 | 512 | 513 | 514 | 515 | 516 | 517 | 518 | 519 | 520 | 521 | 522 | 523 | 524 | 525 | 526 | 527 | 528 | 529 | 530 | 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | 540 | 541 | 542 | 543 |Normal text here
9 | 10 | 11 | -------------------------------------------------------------------------------- /python-examples/assets/iptc-example.jpg~: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/james-see/python-examples/24ef8e164738827e25a049dfebcd9e2e56d8dc1d/python-examples/assets/iptc-example.jpg~ -------------------------------------------------------------------------------- /python-examples/assets/test.xls: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/james-see/python-examples/24ef8e164738827e25a049dfebcd9e2e56d8dc1d/python-examples/assets/test.xls -------------------------------------------------------------------------------- /python-examples/assets/testimage.jpg~: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/james-see/python-examples/24ef8e164738827e25a049dfebcd9e2e56d8dc1d/python-examples/assets/testimage.jpg~ -------------------------------------------------------------------------------- /python-examples/assets/testtweets.txt: -------------------------------------------------------------------------------- 1 | this is not good. 2 | i like all of this. 3 | this is a good test. 4 | having a good day 5 | raining outside, but I have to go out 6 | -------------------------------------------------------------------------------- /python-examples/audio_waveform-example.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | import numpy as np 3 | import wave 4 | # to install pyaudio on osx: brew install portaudio then pip 5 | # install --allow-external pyaudio --allow-unverified pyaudio pyaudio 6 | import pyaudio 7 | import speech_recognition as sr # pip install speechrecognition 8 | 9 | r = sr.Recognizer() 10 | with sr.Microphone() as source: # use the default microphone as the audio source 11 | audio = r.listen(source) 12 | 13 | CHUNK = 1024 14 | FORMAT = pyaudio.paInt16 # paInt8 15 | CHANNELS = 1 16 | RATE = 44100 # sample rate 17 | RECORD_SECONDS = 5 18 | WAVE_OUTPUT_FILENAME = "output.wav" 19 | 20 | p = pyaudio.PyAudio() 21 | 22 | stream = p.open( 23 | format=FORMAT, channels=CHANNELS, rate=RATE, input=True, frames_per_buffer=CHUNK 24 | ) # buffer 25 | 26 | print("* recording") 27 | 28 | frames = [] 29 | 30 | for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)): 31 | data = stream.read(CHUNK) 32 | frames.append(data) # 2 bytes(16 bits) per channel 33 | wf = wave.open("temp.wav", "wb") 34 | wf.setnchannels(CHANNELS) 35 | wf.setsampwidth(p.get_sample_size(FORMAT)) 36 | wf.setframerate(RATE) 37 | wf.writeframes(b"".join(frames)) 38 | wf.close() 39 | print("* done recording") 40 | stream.stop_stream() 41 | stream.close() 42 | p.terminate() 43 | spf = wave.open("temp.wav", "r") 44 | signal = spf.readframes(-1) 45 | signal = np.fromstring(signal, "Int16") 46 | plt.figure(1) 47 | plt.title("Signal Wave...") 48 | plt.plot(signal) 49 | plt.show() 50 | exit() 51 | -------------------------------------------------------------------------------- /python-examples/bitcoin-example-1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # Author: James Campbell 3 | # Date: 2015-11-22 4 | # Date Updated: 2019-06-19 5 | # What: Accesses the blockchain module and queries some data as example 6 | import sys 7 | from sys import exit 8 | import datetime 9 | from blockchain import blockexplorer 10 | 11 | # example address test 12 | address = blockexplorer.get_address("1SDude3hVWoAT2sFxy3JkH2VrcUXPM4PA") 13 | if len(sys.argv) > 1: # you can pass a bitcoin address in from terminal 14 | print(sys.argv[1]) 15 | address = blockexplorer.get_address(sys.argv[1]) 16 | print(address) 17 | # final balance 18 | print(f"\nFinal balance of wallet: {address.final_balance}") # add decimal after first 19 | transactions = address.transactions 20 | print(f"\nList of {len(transactions)} transactions: \n -----------------------------") 21 | for trans in transactions: 22 | print( 23 | f"Ip address of relayed transaction: {trans.relayed_by}" 24 | ) # print ip address of the relayed transaction 25 | print(f"Hash of the transaction: {trans.hash}") # hash of the transaction 26 | print(f"Time of the transaction: {trans.time}") # time of the transaction 27 | timestamp = trans.time 28 | fixedtime = datetime.datetime.fromtimestamp(timestamp) 29 | fixer = fixedtime.strftime("%Y-%m-%d %H:%M:%S") 30 | print(f"Converted time: {fixedtime.strftime('%Y-%m-%d %H:%M:%S')}\n") 31 | # inventory only works for transactions up to 1 hour old 32 | # inv = blockexplorere.get_inventory_data(trans.hash) 33 | # print('Time: %s Initial IP: %s' % (fixer, inv.initial_ip)) 34 | exit() 35 | -------------------------------------------------------------------------------- /python-examples/browser-example.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | """ 3 | Date Updated: 2019-06-19 4 | Author: James Campbell 5 | What: Loads CNN.com using mechanize and saves all the links to assets/cnn.txt 6 | """ 7 | import mechanize 8 | 9 | base_url = "http://www.cnn.com" 10 | title = "cnn" 11 | 12 | 13 | def crawl(site): 14 | seed_url = site 15 | br = mechanize.Browser() 16 | 17 | br.set_handle_robots(False) 18 | br.set_handle_equiv(False) 19 | 20 | br.open(seed_url) 21 | 22 | link_bank = [] 23 | 24 | for link in br.links(): 25 | if link.url[0:4] == "http": 26 | link_bank.append(link.url) 27 | if link.url[0] == "/": 28 | url = link.url 29 | if url.find(".com") == -1: 30 | if url.find(".org") == -1: 31 | link_bank.append(base_url + link.url) 32 | else: 33 | link_bank.append(link.url) 34 | else: 35 | link_bank.append(link.url) 36 | 37 | if link.url[0] == "#": 38 | link_bank.append(base_url + link.url) 39 | 40 | link_bank = list(set(link_bank)) 41 | my_file = open(f"./assets/{title}.txt", "w") 42 | for link in link_bank: 43 | my_file.write(link + "\n") 44 | my_file.close() 45 | return link_bank 46 | 47 | 48 | crawl(base_url) 49 | raise SystemExit(0) 50 | -------------------------------------------------------------------------------- /python-examples/bs4_email_regex-example.py: -------------------------------------------------------------------------------- 1 | """ 2 | What: bs4 get email from beautiful soup object 3 | Author: James Campbell 4 | Date: 2015-10-09 5 | Updated Date: 3 July 2019 6 | """ 7 | from bs4 import BeautifulSoup 8 | import re 9 | import sys 10 | 11 | 12 | def get_emails(soupcontent): 13 | """ 14 | soupcontent: expected to be a bs4 object 15 | Description: This functions gets emails from bs4 object and returns a list 16 | """ 17 | emaillist = [] 18 | soupere = soupcontent.find_all( 19 | text=re.compile( 20 | r"\S[a-zA-Z0-9._%+-].*?[a-zA-Z0-9_%+-]+@[a-zA-Z0-9.-]+?\.[a-zA-Z]{2,4}?\b" 21 | ) 22 | ) 23 | for soupe in soupere: 24 | soupe = soupe.strip() 25 | if soupe == "": 26 | continue 27 | emaillist.append(soupe) 28 | return emaillist 29 | 30 | 31 | htmlcontent = """ 32 |This is an email: james@jamescampbell.us and this is not: james.
""" 33 | soupobject = BeautifulSoup(htmlcontent, "html.parser") 34 | finallist = get_emails(soupobject) 35 | 36 | # prove that it worked 37 | for email in finallist: 38 | print(email) 39 | 40 | sys.exit() 41 | -------------------------------------------------------------------------------- /python-examples/chrome-headless-example.py: -------------------------------------------------------------------------------- 1 | from selenium import webdriver 2 | from webdriver_manager.chrome import ChromeDriverManager 3 | instauser = "default" 4 | 5 | 6 | def getuser(): 7 | """Get instagram username.""" 8 | instauser = input('What instagram user to get information about? ["q" for quit]: ') 9 | return instauser 10 | 11 | 12 | def getcontent(instauser): 13 | """Headless chrome gets the content we need.""" 14 | options = webdriver.ChromeOptions() 15 | options.add_argument('--headless') 16 | options.add_argument('--no-sandbox') 17 | driver = webdriver.Chrome(ChromeDriverManager().install(), options=options) 18 | driver.get('https://www.instagram.com/{}/'.format(instauser)) 19 | print(driver.title) 20 | userdata = driver.execute_script("return _sharedData.entry_data.ProfilePage[0].graphql.user") 21 | # print all of the data 22 | print(f"User ID: {userdata['id']}\n\ 23 | Biography: {userdata['biography']}\n\ 24 | Friends: {userdata['edge_followed_by']['count']}\n\ 25 | Following: {userdata['edge_follow']['count']}") 26 | 27 | 28 | def main(): 29 | """Run the program.""" 30 | instauser = getuser() 31 | if instauser != 'q': 32 | getcontent(instauser) 33 | else: 34 | print('Thanks for playing.') 35 | 36 | 37 | if __name__ == "__main__": 38 | main() 39 | -------------------------------------------------------------------------------- /python-examples/circles-example.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | x = [1, 2, 3, 4, 5] 3 | y = [10, 20, 30, 40, 50] 4 | r = [100, 80, 60, 40, 20] # in points, not data units 5 | fig, ax = plt.subplots(1, 1) 6 | ax.scatter(x, y, s=r) 7 | fig.show() 8 | -------------------------------------------------------------------------------- /python-examples/cvlib_example.py: -------------------------------------------------------------------------------- 1 | """Example using cvlib.""" 2 | import cvlib as cv 3 | from cvlib.object_detection import draw_bbox 4 | import cv2 5 | 6 | img = image = cv2.imread("assets/sv.jpg") 7 | bbox, label, conf = cv.detect_common_objects(img, model="largess") 8 | print(label) 9 | 10 | output_image = draw_bbox(img, bbox, label, conf) 11 | cv2.imwrite("cvlib-example-out.jpg", output_image) 12 | -------------------------------------------------------------------------------- /python-examples/djvu-pdf-example.py: -------------------------------------------------------------------------------- 1 | """djvu to pdf script example.""" 2 | # Date Updated: 1 July 2019 3 | # pre-req osx: djvu2pdf (brew install djvu2pdf with homebrew installed) 4 | # pre-req Ubuntu / Debian: sudo apt-get install djvulibre-bin ghostscript 5 | import fnmatch 6 | import os 7 | import subprocess 8 | # global variables (change to suit your needs) 9 | inputfolderpath = '~' # set to import folder path 10 | outputpath = '~' # set to output folder (must exist) 11 | operationtype = input('Input from folder (1) or single file (2)?: ') 12 | 13 | 14 | def find_files(directory, pattern): 15 | """Find specific files in a directory and sub directories.""" 16 | for _, _, files in os.walk(directory): 17 | for basename in files: 18 | if fnmatch.fnmatch(basename, pattern): 19 | filename = basename 20 | yield filename 21 | 22 | 23 | if operationtype == '1': 24 | i = 0 25 | print(f"Input dir & sub directory underneath set as {inputfolderpath}") 26 | for filename in find_files(inputfolderpath, '*.djvu'): 27 | print(f"[*] Processing DJVU to PDF for {filename}...") 28 | i = i + 1 29 | inputfull = inputfolderpath+filename 30 | outputfilename = filename[:-4]+i+'pdf' # make filename unique 31 | outputfilepath = outputpath 32 | p = subprocess.Popen(["djvu2pdf", inputfull], stdout=subprocess.PIPE) 33 | output, err = p.communicate() 34 | subprocess.call(["mv", outputfilename, outputfilepath]) 35 | print('[-] Processing finished for %s' % filename) 36 | print(f"[--] processed {i} file(s) [--]") 37 | exit('\n\"Sanity is madness put to good uses.\" - George Santayana\n') 38 | 39 | elif operationtype == '2': 40 | filename = input('What filename to process? (leave blank for example): ') 41 | if 'djvu' in filename: 42 | print('Processing DJVU to PDF...') 43 | p = subprocess.Popen(["djvu2pdf", filename], stdout=subprocess.PIPE) 44 | output, err = p.communicate() 45 | print('Processing finished') 46 | exit('Completed sucessfully') 47 | else: 48 | print('No djvu file to process, running sample') 49 | print('Processing DJVU to PDF...') 50 | p = subprocess.Popen(["djvu2pdf", "assets/example.djvu"], 51 | stdout=subprocess.PIPE) 52 | output, err = p.communicate() 53 | print('Processing finished') 54 | exit('Completed sucessfully') 55 | 56 | 57 | elif operationtype == '': 58 | exit('You hit enter without inputing anything, nice work, exiting.') 59 | -------------------------------------------------------------------------------- /python-examples/exif_reader-example.py: -------------------------------------------------------------------------------- 1 | """Show how to get exif data and iptc data from various libraries.""" 2 | import sys 3 | 4 | from PIL import Image 5 | 6 | try: 7 | import exifread 8 | except ModuleNotFoundError as e: 9 | print(e, "pip3 install this!") 10 | exit(1) 11 | try: 12 | import iptcinfo3 13 | except ModuleNotFoundError as e: 14 | print(e, "pip3 install this!") 15 | exit(1) 16 | 17 | # Open image file for reading (binary mode) 18 | path_name = 'assets/cat.jpg' 19 | f = sys.argv[1] # check to see if image in command line 20 | f = open(f, 'rb') 21 | 22 | # Return Exif tags 23 | tags = exifread.process_file(f) 24 | totaltags = len(tags) 25 | print('-------EXIF DATA FOUND-------') 26 | print(f"Total EXIF tags found: {totaltags}") 27 | for tag in tags.keys(): 28 | print("Key: %s, value %s" % (tag, tags[tag])) 29 | print('-----------------END EXIF DATA-------') 30 | 31 | im = Image.open(sys.argv[1]) 32 | try: 33 | info = iptcinfo3.IPTCInfo(sys.argv[1]) 34 | print('-------IPTC DATA FOUND-------') 35 | for k, v in info._data.items(): 36 | print(k, v) 37 | info['city'] = '#magistræde #🇩🇰' 38 | info.save() 39 | except Exception as e: 40 | if str(e) != "No IPTC data found.": 41 | raise 42 | -------------------------------------------------------------------------------- /python-examples/flask-example.py: -------------------------------------------------------------------------------- 1 | """Flask example.""" 2 | import os 3 | from http import HTTPStatus 4 | 5 | from flask import Flask, request 6 | from flask import jsonify 7 | 8 | from werkzeug.utils import secure_filename 9 | """ 10 | Author: James Campbell 11 | Date: Mon May 23 16:26:36 2016 12 | Date Updated: 2 July 2019 13 | What is this code: An example Flask connection 14 | Why?: For me to remember later 15 | """ 16 | 17 | app = Flask(__name__) 18 | ALLOWED_EXTENSIONS = ["zip", "gz", "bz2"] 19 | 20 | 21 | def allowed_filename(filename: str) -> bool: 22 | """Define allowed file extensions.""" 23 | return "." in filename and filename.rsplit(".", 1)[1] in ALLOWED_EXTENSIONS 24 | 25 | 26 | @app.route("/") 27 | def hello_world(): 28 | """Hello world example.""" 29 | return """\ 30 |curl -X POST localhost:6969/upload\ 33 | -F file=@"assets/archive_name.tar.gz" -ito test from\ 34 | same folder you executed
python3\ 35 | flask-example.py36 | """ 37 | 38 | 39 | @app.route("/upload", methods=["POST"]) 40 | def upload_csv() -> str: 41 | """Upload CSV example.""" 42 | submitted_file = request.files["file"] 43 | if submitted_file and allowed_filename(submitted_file.filename): 44 | filename = secure_filename(submitted_file.filename) 45 | directory = os.path.join(app.config["UPLOAD_FOLDER"]) 46 | if not os.path.exists(directory): 47 | os.mkdir(directory) 48 | basedir = os.path.abspath(os.path.dirname(__file__)) 49 | submitted_file.save( 50 | os.path.join(basedir, app.config["UPLOAD_FOLDER"], filename) 51 | ) 52 | out = { 53 | "status": HTTPStatus.OK, 54 | "filename": filename, 55 | "message": f"{filename} saved successful.", 56 | } 57 | return jsonify(out) 58 | 59 | 60 | if __name__ == "__main__": 61 | app.config["UPLOAD_FOLDER"] = "flaskme/" 62 | app.run(port=6969, debug=True) 63 | 64 | # curl -X POST localhost:6969/upload -F file=@"assets/archive_name.tar.gz" -i 65 | -------------------------------------------------------------------------------- /python-examples/fuzzywuzzy-example.py: -------------------------------------------------------------------------------- 1 | """Example to use fuzzywuzzy which does a fuzzy match fast.""" 2 | # Author: James Campbell 3 | # Date: August 11th 2016 4 | # Date Updated: 2 July 2019 5 | 6 | from fuzzywuzzy import process 7 | 8 | choices = ["Atlanta Falcons", "New York Jets", "Dallas Cowboys"] 9 | hello = process.extractOne("cowboys", choices) 10 | print(hello) # should print Dallas Cowboys 11 | -------------------------------------------------------------------------------- /python-examples/geoname-example.py: -------------------------------------------------------------------------------- 1 | """Shows how to use geotext.""" 2 | from geotext import GeoText 3 | 4 | places = GeoText("London is a great city") 5 | places.cities 6 | # "London" 7 | 8 | # filter by country code 9 | result = GeoText('I loved Rio de Janeiro and Havana', 'BR').cities 10 | print(result) 11 | # 'Rio de Janeiro' 12 | 13 | print(GeoText('New York, Texas, and also China').country_mentions) 14 | # OrderedDict([(u'US', 2), (u'CN', 1)]) 15 | -------------------------------------------------------------------------------- /python-examples/get_geo-example.py: -------------------------------------------------------------------------------- 1 | """Google maps to shodan .""" 2 | print("google api killed this example.") 3 | -------------------------------------------------------------------------------- /python-examples/html2plaintext-example.py: -------------------------------------------------------------------------------- 1 | """Example on how to get plaintext from html using python's beautiful soup.""" 2 | # Author: James Campbell 3 | # Date: 2015 05 19 4 | # Date Updated: 2 July 2019 5 | from bs4 import BeautifulSoup 6 | 7 | 8 | def cleanMe(html): 9 | """Clean html into text only for-real.""" 10 | soup = BeautifulSoup(html, "lxml") # create a new bs4 object from html 11 | for script in soup(["script", "style"]): # remove all javascript & css 12 | script.extract() 13 | # get text 14 | text = soup.get_text() 15 | # break into lines and remove leading and trailing space on each 16 | lines = (line.strip() for line in text.splitlines()) 17 | # break multi-headlines into a line each 18 | chunks = (phrase.strip() for line in lines for phrase in line.split(" ")) 19 | # drop blank lines 20 | text = '\n'.join(chunk for chunk in chunks if chunk) 21 | return text 22 | 23 | 24 | testhtml = """ 25 | \n\n
I hope you enjoy this example. 28 |
29 | """ 30 | 31 | print('\n\n[*-*]Before html with text:\n------------------') 32 | print(testhtml) 33 | print('------------------\n\n\n\n[*-*]After cleanMe() function:\n------------') 34 | print(cleanMe(testhtml)) 35 | print('-------------------\n\n') 36 | -------------------------------------------------------------------------------- /python-examples/http.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 |This is a test.
", "utf-8")) 27 | # you can use if else to check path and do custom things based on path accessed 28 | if self.path == '/win': 29 | self.wfile.write(bytes("YOU WIN! @ path %s
" % self.path, "utf-8")) 30 | else: 31 | self.wfile.write(bytes("You accessed path: %s
" % self.path, "utf-8")) 32 | self.wfile.write(bytes("", "utf-8")) 33 | 34 | 35 | myServer = HTTPServer((hostName, hostPort), MyServer) 36 | print(time.asctime(), "Server Starts - %s:%s" % (hostName, hostPort)) 37 | 38 | # continue to serve until a keypress in terminal 39 | try: 40 | myServer.serve_forever() 41 | except KeyboardInterrupt: 42 | pass 43 | 44 | myServer.server_close() 45 | print(time.asctime(), "Server Stops - %s:%s" % (hostName, hostPort)) 46 | # python3 -m http.server 10010 --bind 127.0.0.1 47 | -------------------------------------------------------------------------------- /python-examples/setup.py: -------------------------------------------------------------------------------- 1 | """ 2 | This is a setup.py script generated by py2applet 3 | 4 | Usage: 5 | python setup.py py2app 6 | """ 7 | 8 | from setuptools import setup 9 | 10 | APP = ['tkinter-example.py'] 11 | DATA_FILES = [] 12 | OPTIONS = {'argv_emulation': True} 13 | 14 | setup( 15 | app=APP, 16 | data_files=DATA_FILES, 17 | options={'py2app': OPTIONS}, 18 | setup_requires=['py2app'], 19 | ) 20 | -------------------------------------------------------------------------------- /python-examples/shodan-example.py: -------------------------------------------------------------------------------- 1 | """Example on using shodan api, requires shodan api key.""" 2 | # Author: James Campbell 3 | # Date: June 23rd 2016 4 | # Date Updated: 3 July 2019 5 | # What: Shodan example 6 | from configs import globalshodankey # have a configs.py file with shodan api key 7 | import shodan 8 | 9 | shodan_api_key = globalshodankey # set in configs.py 10 | try: 11 | api = shodan.Shodan(shodan_api_key) 12 | except Exception: 13 | exit('make sure you have the shodan key setup in configs.py as globalshodankey = "yourkey"') 14 | try: 15 | results = api.count('country:GB city:Glasgow Nginx') 16 | if int(results['total']) == 0: 17 | print('NONE FOUND! TRY AGAIN') 18 | else: 19 | print(f"Results found for NGINX in Glasgow: {results['total']}") 20 | except Exception: 21 | exit('failed') 22 | -------------------------------------------------------------------------------- /python-examples/smash-site-example.py: -------------------------------------------------------------------------------- 1 | import urllib 2 | import urllib.request 3 | import sys 4 | from concurrent.futures import ThreadPoolExecutor 5 | 6 | # globals 7 | if len(sys.argv) < 2: 8 | link = 'https://jamescampbell.us' 9 | else: 10 | link = sys.argv[1] 11 | 12 | headers = {} 13 | headers['User-Agent'] = "Mozilla/5.0 (X11; Linux i686)" 14 | p = 1 15 | 16 | 17 | def getyou(linked=link): 18 | global p 19 | f = urllib.request.Request(linked, headers=headers) 20 | urllib.request.urlopen(f) 21 | p = p + 1 22 | print('IM P %d' % (p,)) 23 | 24 | 25 | executor = ThreadPoolExecutor(max_workers=100) 26 | futures = [] 27 | i = 1 28 | while i < 500: 29 | a = executor.submit(getyou, link) 30 | futures.append(a) 31 | i = i + 1 32 | 33 | -------------------------------------------------------------------------------- /python-examples/speech-example.py: -------------------------------------------------------------------------------- 1 | # speech recognition example 2 | # author: James Campbell 3 | # date: 2015-05-26 4 | # to install pyaudio on osx: brew install portaudio then 5 | # pip install --allow-external pyaudio --allow-unverified pyaudio pyaudio 6 | import speech_recognition as sr # pip install speechrecognition 7 | from termcolor import colored 8 | r = sr.Recognizer() 9 | with sr.Microphone() as source: # use the default microphone as the audio source 10 | # listen for the first phrase and extract it into audio data 11 | audio = r.listen(source) 12 | 13 | try: 14 | # recognize speech using Google Speech Recognition 15 | print("You said " + colored(r.recognize(audio), 'yellow')) 16 | if r.recognize(audio) == 'exit': 17 | exit('goodbye') 18 | except LookupError: # speech is unintelligible 19 | print("Could not understand audio") 20 | -------------------------------------------------------------------------------- /python-examples/spider.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | # getting all links example crawling jamescampbell.us 3 | # author: James Campbell 4 | # Date Created: 2015 05 22 5 | # Date Updated: 2 July 2019 6 | import argparse 7 | from scrapy.spiders import CrawlSpider, Rule 8 | from scrapy.linkextractors.lxmlhtml import LxmlLinkExtractor 9 | from scrapy.item import Item, Field 10 | 11 | # terminal arguments parser globals - do not change 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument('-u', action='store', dest='url', 14 | help='Domain to crawl') 15 | parser.add_argument('-c', action='store_const', dest='constant_value', 16 | const='value-to-store', 17 | help='Store a constant value') 18 | parser.add_argument('--version', action='version', version='%(prog)s 1.0') 19 | results = parser.parse_args() 20 | 21 | # setup the default search terms 22 | domainer = 'jamescampbell.us' # default search term if none set is a random term from a dict 23 | if results.url is not None: # if search terms set then change from default to that 24 | domainer = results.url # set from argparse above in globals section 25 | 26 | 27 | DOMAIN = domainer 28 | URL = 'https://%s' % DOMAIN 29 | 30 | 31 | class MyItem(Item): 32 | url = Field() 33 | 34 | 35 | class someSpider(CrawlSpider): 36 | name = 'crawltest' 37 | allowed_domains = ['jamescampbell.us'] 38 | start_urls = ['https://jamescampbell.us'] 39 | rules = (Rule(LxmlLinkExtractor(allow=()), callback='parse_obj', follow=True),) 40 | 41 | def parse_obj(self, response): 42 | item = MyItem() 43 | item['url'] = [] 44 | for link in LxmlLinkExtractor(allow=(), deny=self.allowed_domains).extract_links(response): 45 | item['url'].append(link.url) 46 | print(link.url) 47 | return item 48 | 49 | 50 | someSpider() 51 | -------------------------------------------------------------------------------- /python-examples/spotify-example.py: -------------------------------------------------------------------------------- 1 | # Author: James Campbell 2 | # What: uses spotipy to get example data 3 | import sys 4 | import spotipy 5 | import spotipy.util as util 6 | 7 | scope = 'user-library-read' 8 | 9 | if len(sys.argv) > 1: 10 | username = sys.argv[1] 11 | else: 12 | print("Usage: %s username" % (sys.argv[0],)) 13 | sys.exit() 14 | 15 | token = util.prompt_for_user_token(username, scope) 16 | 17 | if token: 18 | sp = spotipy.Spotify(auth=token) 19 | results = sp.current_user_saved_tracks() 20 | for item in results['items']: 21 | track = item['track'] 22 | print(track['name'] + ' - ' + track['artists'][0]['name']) 23 | else: 24 | print("Can't get token for", username) 25 | -------------------------------------------------------------------------------- /python-examples/stem_tor-example.py: -------------------------------------------------------------------------------- 1 | import os 2 | import shutil 3 | # configs.py file you need to create 4 | from configs import stempass 5 | from stem.control import Controller 6 | from flask import Flask 7 | 8 | app = Flask(__name__) 9 | 10 | 11 | @app.route('/') 12 | def index(): 13 | global result 14 | hoster = result.hostname 15 | return "\ 16 |Hi Grandma! {}
{}".format(hoster)
17 |
18 |
19 | print(' * Connecting to tor')
20 |
21 | with Controller.from_port() as controller:
22 | controller.authenticate(password=stempass)
23 |
24 | # All hidden services have a directory on disk. Lets put ours in tor's data
25 | # directory.
26 |
27 | hidden_service_dir = os.path.join(controller.get_conf('DataDirectory', '/tmp'), 'hello_world')
28 |
29 | # Create a hidden service where visitors of port 80 get redirected to local
30 | # port 5000 (this is where Flask runs by default).
31 |
32 | print(" * Creating our hidden service in %s" % hidden_service_dir)
33 | result = controller.create_hidden_service(hidden_service_dir, 80, target_port=5000)
34 |
35 | # The hostname is only available when we can read the hidden service
36 | # directory. This requires us to be running with the same user as tor.
37 |
38 | if result.hostname:
39 | print(" * Our service is available at %s, press ctrl+c to quit" % result.hostname)
40 | else:
41 | print(" * Unable to read the hidden service directory")
42 |
43 | try:
44 | app.run()
45 | finally:
46 | # Shut down the hidden service and clean it off disk. Note that you *don't*
47 | # want to delete the hidden service directory if you'd like to have this
48 | # same *.onion address in the future.
49 |
50 | print(" * Shutting down our hidden service")
51 | controller.remove_hidden_service(hidden_service_dir)
52 | shutil.rmtree(hidden_service_dir)
53 |
--------------------------------------------------------------------------------
/python-examples/test_tor-example.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 | import socks
3 | import socket
4 |
5 |
6 | def set_socks_default():
7 | # TOR SETUP GLOBAL Vars
8 | SOCKS_PORT = 9050 # TOR proxy port that is default from torrc, change to whatever torrc
9 |
10 | socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", SOCKS_PORT)
11 | socket.socket = socks.socksocket
12 |
13 | # Perform DNS resolution through the socket
14 | def getaddrinfo(*args):
15 | return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))]
16 | socket.getaddrinfo = getaddrinfo
17 | return "success"
18 |
19 |
20 | def test_socks():
21 | assert set_socks_default() == "success"
22 |
--------------------------------------------------------------------------------
/python-examples/textract-example.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 | # Author: James Campbell
3 | # Date: June 3rd 2019
4 | # What: get text from a PDF
5 | import sys
6 | import textract
7 |
8 | text = textract.process(f"{sys.argv[1]}")
9 | print(text)
10 |
--------------------------------------------------------------------------------
/python-examples/tika-example-too.py:
--------------------------------------------------------------------------------
1 | import requests
2 | import json
3 | from pprint import pprint
4 |
5 | import os
6 |
7 | # globals
8 |
9 | filelist = os.listdir('assets') # from configurator
10 |
11 | for filepath in filelist:
12 | payload = open(filepath, 'rb').read()
13 | filename = filepath.rsplit('/')[0]
14 | print(filename)
15 | r = requests.put('http://localhost:9998/rmeta', data=payload)
16 | print(r.text)
17 | rdict = json.loads(r.text)[0]
18 | pprint(rdict['meta:page-count'])
19 | exit()
20 |
--------------------------------------------------------------------------------
/python-examples/tika-example.py:
--------------------------------------------------------------------------------
1 | """Tika example get metadata."""
2 | from tika import parser
3 | parsed = parser.from_file("temp.wav")
4 | print(parsed["metadata"])
5 | print(parsed["content"])
6 |
--------------------------------------------------------------------------------
/python-examples/tika-get-text-example.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 | # Author: James Campbell
3 | # Date: June 3rd 2019
4 | # What: get text from a PDF
5 | from tika import parser
6 | import sys
7 |
8 | text = parser.from_file(f"{sys.argv[1]}")
9 | print(text)
10 |
--------------------------------------------------------------------------------
/python-examples/tkinter-example.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python3
2 | # Author: James Campbell
3 | # What: Example GUI with Tkinter
4 |
5 | import sys
6 | if sys.version_info < (3, 0):
7 | # Python 2
8 | import Tkinter as tk
9 | else:
10 | # Python 3
11 | import tkinter as tk
12 | root = tk.Tk()
13 | root.title("Sandwich")
14 | tk.Button(root, text="Make me a Sandwich").pack()
15 | tk.mainloop()
16 |
--------------------------------------------------------------------------------
/python-examples/tor-example.py:
--------------------------------------------------------------------------------
1 | # tor connect example code
2 | # author: James Campbell
3 | # date: 2015 05 17
4 | # date updated: 2016 09 18 confirmed working (make sure to not use privoxy settings or will break)
5 |
6 | import urllib
7 | import urllib.request
8 | import socks
9 | import socket
10 | import argparse
11 |
12 |
13 | # terminal arguments parser globals - do not change
14 | parser = argparse.ArgumentParser()
15 | parser.add_argument('-o', action='store', dest='onion',
16 | help='put in onion site to load (with http & quotes)')
17 | results = parser.parse_args()
18 |
19 | # Global Vars
20 | # set the default onion site to visit to test, in this case DuckDuckGo
21 | onionsite = 'http://3g2upl4pq6kufc4m.onion'
22 | if results.onion is not None: # if search terms set in terminal then change from default to that
23 | onionsite = results.onion # set from argparse above in globals section
24 |
25 | # TOR SETUP GLOBAL Vars
26 | SOCKS_PORT = 9050 # TOR proxy port that is default from torrc, change to whatever torrc
27 |
28 | socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", SOCKS_PORT)
29 | socket.socket = socks.socksocket
30 |
31 | # Perform DNS resolution through the socket
32 |
33 |
34 | def getaddrinfo(*args):
35 | return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))]
36 |
37 |
38 | socket.getaddrinfo = getaddrinfo
39 |
40 | # test connect to DuckDuckGo .onion site
41 | headers = {'User-Agent': 'JAMES CAMPBELL jamescampbell.us SEARCH BOT! I FOUND YOU!!!!'}
42 | req = urllib.request.Request(onionsite, None, headers)
43 | response = urllib.request.urlopen(req) # new python 3 code -jc
44 | status = 'loaded successfully'
45 | try:
46 | sitehtml = response.read()
47 | print(sitehtml)
48 | except urllib.error.URLError as e:
49 | html = e.read().decode("utf8", 'ignore')
50 | status = 'failed reading'
51 | html = 'none'
52 | currenturl = 'none'
53 | print(status)
54 |
--------------------------------------------------------------------------------
/python-examples/triplot-example.py:
--------------------------------------------------------------------------------
1 | """
2 | Creating and plotting unstructured triangular grids.
3 | """
4 | import matplotlib.pyplot as plt
5 | import matplotlib.tri as tri
6 | import numpy as np
7 | import math
8 |
9 | # Creating a Triangulation without specifying the triangles results in the
10 | # Delaunay triangulation of the points.
11 |
12 | # First create the x and y coordinates of the points.
13 | n_angles = 36
14 | n_radii = 8
15 | min_radius = 0.25
16 | radii = np.linspace(min_radius, 0.95, n_radii)
17 |
18 | angles = np.linspace(0, 2*math.pi, n_angles, endpoint=False)
19 | angles = np.repeat(angles[..., np.newaxis], n_radii, axis=1)
20 | angles[:, 1::2] += math.pi/n_angles
21 |
22 | x = (radii*np.cos(angles)).flatten()
23 | y = (radii*np.sin(angles)).flatten()
24 |
25 | # Create the Triangulation; no triangles so Delaunay triangulation created.
26 | triang = tri.Triangulation(x, y)
27 |
28 | # Mask off unwanted triangles.
29 | xmid = x[triang.triangles].mean(axis=1)
30 | ymid = y[triang.triangles].mean(axis=1)
31 | mask = np.where(xmid*xmid + ymid*ymid < min_radius*min_radius, 1, 0)
32 | triang.set_mask(mask)
33 |
34 | # Plot the triangulation.
35 | plt.figure()
36 | plt.gca().set_aspect('equal')
37 | plt.triplot(triang, 'bo-')
38 | plt.title('triplot of Delaunay triangulation')
39 |
40 |
41 | # You can specify your own triangulation rather than perform a Delaunay
42 | # triangulation of the points, where each triangle is given by the indices of
43 | # the three points that make up the triangle, ordered in either a clockwise or
44 | # anticlockwise manner.
45 |
46 | xy = np.asarray([
47 | [-0.101, 0.872], [-0.080, 0.883], [-0.069, 0.888], [-0.054, 0.890],
48 | [-0.045, 0.897], [-0.057, 0.895], [-0.073, 0.900], [-0.087, 0.898],
49 | [-0.090, 0.904], [-0.069, 0.907], [-0.069, 0.921], [-0.080, 0.919],
50 | [-0.073, 0.928], [-0.052, 0.930], [-0.048, 0.942], [-0.062, 0.949],
51 | [-0.054, 0.958], [-0.069, 0.954], [-0.087, 0.952], [-0.087, 0.959],
52 | [-0.080, 0.966], [-0.085, 0.973], [-0.087, 0.965], [-0.097, 0.965],
53 | [-0.097, 0.975], [-0.092, 0.984], [-0.101, 0.980], [-0.108, 0.980],
54 | [-0.104, 0.987], [-0.102, 0.993], [-0.115, 1.001], [-0.099, 0.996],
55 | [-0.101, 1.007], [-0.090, 1.010], [-0.087, 1.021], [-0.069, 1.021],
56 | [-0.052, 1.022], [-0.052, 1.017], [-0.069, 1.010], [-0.064, 1.005],
57 | [-0.048, 1.005], [-0.031, 1.005], [-0.031, 0.996], [-0.040, 0.987],
58 | [-0.045, 0.980], [-0.052, 0.975], [-0.040, 0.973], [-0.026, 0.968],
59 | [-0.020, 0.954], [-0.006, 0.947], [0.003, 0.935], [0.006, 0.926],
60 | [0.005, 0.921], [0.022, 0.923], [0.033, 0.912], [0.029, 0.905],
61 | [0.017, 0.900], [0.012, 0.895], [0.027, 0.893], [0.019, 0.886],
62 | [0.001, 0.883], [-0.012, 0.884], [-0.029, 0.883], [-0.038, 0.879],
63 | [-0.057, 0.881], [-0.062, 0.876], [-0.078, 0.876], [-0.087, 0.872],
64 | [-0.030, 0.907], [-0.007, 0.905], [-0.057, 0.916], [-0.025, 0.933],
65 | [-0.077, 0.990], [-0.059, 0.993]])
66 | x = xy[:, 0]*180/3.14159
67 | y = xy[:, 1]*180/3.14159
68 |
69 | triangles = np.asarray([
70 | [67, 66, 1], [65, 2, 66], [1, 66, 2], [64, 2, 65], [63, 3, 64],
71 | [60, 59, 57], [2, 64, 3], [3, 63, 4], [0, 67, 1], [62, 4, 63],
72 | [57, 59, 56], [59, 58, 56], [61, 60, 69], [57, 69, 60], [4, 62, 68],
73 | [6, 5, 9], [61, 68, 62], [69, 68, 61], [9, 5, 70], [6, 8, 7],
74 | [4, 70, 5], [8, 6, 9], [56, 69, 57], [69, 56, 52], [70, 10, 9],
75 | [54, 53, 55], [56, 55, 53], [68, 70, 4], [52, 56, 53], [11, 10, 12],
76 | [69, 71, 68], [68, 13, 70], [10, 70, 13], [51, 50, 52], [13, 68, 71],
77 | [52, 71, 69], [12, 10, 13], [71, 52, 50], [71, 14, 13], [50, 49, 71],
78 | [49, 48, 71], [14, 16, 15], [14, 71, 48], [17, 19, 18], [17, 20, 19],
79 | [48, 16, 14], [48, 47, 16], [47, 46, 16], [16, 46, 45], [23, 22, 24],
80 | [21, 24, 22], [17, 16, 45], [20, 17, 45], [21, 25, 24], [27, 26, 28],
81 | [20, 72, 21], [25, 21, 72], [45, 72, 20], [25, 28, 26], [44, 73, 45],
82 | [72, 45, 73], [28, 25, 29], [29, 25, 31], [43, 73, 44], [73, 43, 40],
83 | [72, 73, 39], [72, 31, 25], [42, 40, 43], [31, 30, 29], [39, 73, 40],
84 | [42, 41, 40], [72, 33, 31], [32, 31, 33], [39, 38, 72], [33, 72, 38],
85 | [33, 38, 34], [37, 35, 38], [34, 38, 35], [35, 37, 36]])
86 |
87 | # Rather than create a Triangulation object, can simply pass x, y and triangles
88 | # arrays to triplot directly. It would be better to use a Triangulation object
89 | # if the same triangulation was to be used more than once to save duplicated
90 | # calculations.
91 | plt.figure()
92 | plt.gca().set_aspect('equal')
93 | plt.triplot(x, y, triangles, 'go-')
94 | plt.title('triplot of user-specified triangulation')
95 | plt.xlabel('Longitude (degrees)')
96 | plt.ylabel('Latitude (degrees)')
97 |
98 | plt.show()
99 |
--------------------------------------------------------------------------------
/python-examples/tuple-example.py:
--------------------------------------------------------------------------------
1 | # tuple sort example
2 | # author: James Campbell
3 | # date: 2015-05-28
4 | # Date Updated: 2 July 2019
5 | valued = []
6 | lettered = []
7 | plusone = []
8 | listed = [(('d', 0), ('g', 0)), (('d', 0), ('d', 1)), (('i', 0), ('g', 0))]
9 | for (x, y) in listed:
10 | for subx, suby in x, y:
11 | valued.append(int(suby))
12 | lettered.append(subx)
13 | for value in valued:
14 | value = value + 1
15 | plusone.append(int(value))
16 | # print plusone
17 | coolness = zip(lettered, plusone)
18 | print(coolness)
19 |
20 | print(map(list, zip(lettered, plusone)))
21 |
--------------------------------------------------------------------------------
/python-examples/urllib3_proxymanager-example.py:
--------------------------------------------------------------------------------
1 | """Using urllib3 ProxyManager and tor example."""
2 | # author: James Campbell
3 | # date: 2015 11 19
4 | # Date Updated: 2 July 2019
5 | import urllib3 # use with python 3 only
6 | import argparse
7 | from bs4 import BeautifulSoup
8 |
9 | # terminal arguments parser globals - do not change
10 | parser = argparse.ArgumentParser()
11 | parser.add_argument('-o', action='store', dest='onion',
12 | help='put in onion site to load (with http & quotes)')
13 | results = parser.parse_args()
14 |
15 | # Global Vars
16 | # set the default onion site to visit to test, in this case DuckDuckGo
17 | onionsite = 'http://3g2upl4pq6kufc4m.onion'
18 | if results.onion is not None: # if search terms set in terminal then change from default to that
19 | onionsite = results.onion # set from argparse above in globals section
20 |
21 | # TOR SETUP GLOBAL Vars
22 | # TOR proxy port that is default from torrc, change to whatever torrc is configured to
23 | SOCKS_PORT = 9050
24 |
25 |
26 | header = {'User-Agent': 'JAMES CAMPBELL jamescampbell.us SEARCH BOT! I FOUND YOU!!!!'}
27 | # using this with privoxy and forwarding to tor
28 | proxy = urllib3.ProxyManager('http://127.0.0.1:8119/')
29 | r1 = proxy.request('GET', onionsite, headers=header)
30 | print(r1.status) # status code
31 | print(r1.headers) # header data
32 | print(r1.data.decode('utf8')) # html raw output
33 | souper = BeautifulSoup(r1.data, "html.parser")
34 | soupera = souper.find_all('a') # get all a href's
35 | for eachone in soupera:
36 | print('This is a link: \n', eachone.text)
37 | exit()
38 | # test connect to DuckDuckGo .onion site
39 |
--------------------------------------------------------------------------------
/python-examples/websockify-example.py:
--------------------------------------------------------------------------------
1 | """Example using websockify."""
2 | from websockify import auth_plugins as auth
3 | from websockify import websocket
4 | import select
5 | import signal
6 | import socket
7 | import optparse
8 | import time
9 | import os
10 | import sys
11 | import subprocess
12 | import logging
13 | import errno
14 |
15 | from socketserver import ForkingMixIn
16 | from http.server import HTTPServer
17 | from urllib.parse import parse_qs, urlparse
18 | '''
19 | A WebSocket to TCP socket proxy with support for "wss://" encryption.
20 | Copyright 2011 Joel Martin
21 | Licensed under LGPL version 3 (see docs/LICENSE.LGPL-3)
22 |
23 | You can make a cert/key with openssl using:
24 | openssl req -new -x509 -days 365 -nodes -out self.pem -keyout self.pem
25 | as taken from http://docs.python.org/dev/library/ssl.html#certificates
26 |
27 | '''
28 |
29 |
30 | class ProxyRequestHandler(websocket.WebSocketRequestHandler):
31 |
32 | traffic_legend = """
33 | Traffic Legend:
34 | } - Client receive
35 | }. - Client receive partial
36 | { - Target receive
37 |
38 | > - Target send
39 | >. - Target send partial
40 | < - Client send
41 | <. - Client send partial
42 | """
43 |
44 | def send_auth_error(self, ex):
45 | self.send_response(ex.code, ex.msg)
46 | self.send_header('Content-Type', 'text/html')
47 | for name, val in ex.headers.items():
48 | self.send_header(name, val)
49 |
50 | self.end_headers()
51 |
52 | def validate_connection(self):
53 | if self.server.token_plugin:
54 | host, port = self.get_target(self.server.token_plugin, self.path)
55 | if host == 'unix_socket':
56 | self.server.unix_target = port
57 |
58 | else:
59 | self.server.target_host = host
60 | self.server.target_port = port
61 |
62 | if self.server.auth_plugin:
63 | try:
64 | self.server.auth_plugin.authenticate(
65 | headers=self.headers, target_host=self.server.target_host,
66 | target_port=self.server.target_port)
67 | except auth.AuthenticationError:
68 | ex = sys.exc_info()[1]
69 | self.send_auth_error(ex)
70 | raise
71 |
72 | def new_websocket_client(self):
73 | """
74 | Called after a new WebSocket connection has been established.
75 | """
76 | # Checking for a token is done in validate_connection()
77 |
78 | # Connect to the target
79 | if self.server.wrap_cmd:
80 | msg = "connecting to command: '%s' (port %s)" % (
81 | " ".join(self.server.wrap_cmd), self.server.target_port)
82 | elif self.server.unix_target:
83 | msg = "connecting to unix socket: %s" % self.server.unix_target
84 | else:
85 | msg = "connecting to: %s:%s" % (
86 | self.server.target_host, self.server.target_port)
87 |
88 | if self.server.ssl_target:
89 | msg += " (using SSL)"
90 | self.log_message(msg)
91 |
92 | tsock = websocket.WebSocketServer.socket(self.server.target_host,
93 | self.server.target_port,
94 | connect=True,
95 | use_ssl=self.server.ssl_target,
96 | unix_socket=self.server.unix_target)
97 |
98 | self.request.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
99 | if not self.server.wrap_cmd and not self.server.unix_target:
100 | tsock.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
101 |
102 | self.print_traffic(self.traffic_legend)
103 |
104 | # Start proxying
105 | try:
106 | self.do_proxy(tsock)
107 | except Exception:
108 | if tsock:
109 | tsock.shutdown(socket.SHUT_RDWR)
110 | tsock.close()
111 | if self.verbose:
112 | self.log_message("%s:%s: Closed target",
113 | self.server.target_host,
114 | self.server.target_port)
115 | raise
116 |
117 | def get_target(self, target_plugin, path):
118 | """
119 | Parses the path, extracts a token, and looks up a target
120 | for that token using the token plugin. Sets
121 | target_host and target_port if successful
122 | """
123 | # The files in targets contain the lines
124 | # in the form of token: host:port
125 |
126 | # Extract the token parameter from url
127 | args = parse_qs(urlparse(path)[4]) # 4 is the query from url
128 |
129 | if 'token' not in args or not len(args['token']):
130 | raise self.server.EClose("Token not present")
131 |
132 | token = args['token'][0].rstrip('\n')
133 |
134 | result_pair = target_plugin.lookup(token)
135 |
136 | if result_pair is not None:
137 | return result_pair
138 | else:
139 | raise self.server.EClose("Token '%s' not found" % token)
140 |
141 | def do_proxy(self, target):
142 | """
143 | Proxy client WebSocket to normal target socket.
144 | """
145 | cqueue = []
146 | c_pend = 0
147 | tqueue = []
148 | rlist = [self.request, target]
149 |
150 | if self.server.heartbeat:
151 | now = time.time()
152 | self.heartbeat = now + self.server.heartbeat
153 | else:
154 | self.heartbeat = None
155 |
156 | while True:
157 | wlist = []
158 |
159 | if self.heartbeat is not None:
160 | now = time.time()
161 | if now > self.heartbeat:
162 | self.heartbeat = now + self.server.heartbeat
163 | self.send_ping()
164 |
165 | if tqueue:
166 | wlist.append(target)
167 | if cqueue or c_pend:
168 | wlist.append(self.request)
169 | try:
170 | ins, outs, excepts = select.select(rlist, wlist, [], 1)
171 | except (select.error, OSError):
172 | exc = sys.exc_info()[1]
173 | if hasattr(exc, 'errno'):
174 | err = exc.errno
175 | else:
176 | err = exc[0]
177 |
178 | if err != errno.EINTR:
179 | raise
180 | else:
181 | continue
182 |
183 | if excepts:
184 | raise Exception("Socket exception")
185 |
186 | if self.request in outs:
187 | # Send queued target data to the client
188 | c_pend = self.send_frames(cqueue)
189 |
190 | cqueue = []
191 |
192 | if self.request in ins:
193 | # Receive client data, decode it, and queue for target
194 | bufs, closed = self.recv_frames()
195 | tqueue.extend(bufs)
196 |
197 | if closed:
198 | # TODO: What about blocking on client socket?
199 | if self.verbose:
200 | self.log_message("%s:%s: Client closed connection",
201 | self.server.target_host, self.server.target_port)
202 | raise self.CClose(closed['code'], closed['reason'])
203 |
204 | if target in outs:
205 | # Send queued client data to the target
206 | dat = tqueue.pop(0)
207 | sent = target.send(dat)
208 | if sent == len(dat):
209 | self.print_traffic(">")
210 | else:
211 | # requeue the remaining data
212 | tqueue.insert(0, dat[sent:])
213 | self.print_traffic(".>")
214 |
215 | if target in ins:
216 | # Receive target data, encode it and queue for client
217 | buf = target.recv(self.buffer_size)
218 | if len(buf) == 0:
219 | if self.verbose:
220 | self.log_message("%s:%s: Target closed connection",
221 | self.server.target_host, self.server.target_port)
222 | raise self.CClose(1000, "Target closed")
223 |
224 | cqueue.append(buf)
225 | self.print_traffic("{")
226 |
227 |
228 | class WebSocketProxy(websocket.WebSocketServer):
229 | """
230 | Proxy traffic to and from a WebSockets client to a normal TCP
231 | socket server target. All traffic to/from the client is base64
232 | encoded/decoded to allow binary data to be sent/received to/from
233 | the target.
234 | """
235 |
236 | buffer_size = 65536
237 |
238 | def __init__(self, RequestHandlerClass=ProxyRequestHandler, *args, **kwargs):
239 | # Save off proxy specific options
240 | self.target_host = kwargs.pop('target_host', None)
241 | self.target_port = kwargs.pop('target_port', None)
242 | self.wrap_cmd = kwargs.pop('wrap_cmd', None)
243 | self.wrap_mode = kwargs.pop('wrap_mode', None)
244 | self.unix_target = kwargs.pop('unix_target', None)
245 | self.ssl_target = kwargs.pop('ssl_target', None)
246 | self.heartbeat = kwargs.pop('heartbeat', None)
247 |
248 | self.token_plugin = kwargs.pop('token_plugin', None)
249 | self.auth_plugin = kwargs.pop('auth_plugin', None)
250 |
251 | # Last 3 timestamps command was run
252 | self.wrap_times = [0, 0, 0]
253 |
254 | if self.wrap_cmd:
255 | wsdir = os.path.dirname(sys.argv[0])
256 | rebinder_path = [os.path.join(wsdir, "..", "lib"),
257 | os.path.join(wsdir, "..", "lib", "websockify"),
258 | wsdir]
259 | self.rebinder = None
260 |
261 | for rdir in rebinder_path:
262 | rpath = os.path.join(rdir, "rebind.so")
263 | if os.path.exists(rpath):
264 | self.rebinder = rpath
265 | break
266 |
267 | if not self.rebinder:
268 | raise Exception(
269 | "rebind.so not found, perhaps you need to run make")
270 | self.rebinder = os.path.abspath(self.rebinder)
271 |
272 | self.target_host = "127.0.0.1" # Loopback
273 | # Find a free high port
274 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
275 | sock.bind(('', 0))
276 | self.target_port = sock.getsockname()[1]
277 | sock.close()
278 |
279 | os.environ.update({
280 | "LD_PRELOAD": self.rebinder,
281 | "REBIND_OLD_PORT": str(kwargs['listen_port']),
282 | "REBIND_NEW_PORT": str(self.target_port)})
283 |
284 | websocket.WebSocketServer.__init__(
285 | self, RequestHandlerClass, *args, **kwargs)
286 |
287 | def run_wrap_cmd(self):
288 | self.msg("Starting '%s'", " ".join(self.wrap_cmd))
289 | self.wrap_times.append(time.time())
290 | self.wrap_times.pop(0)
291 | self.cmd = subprocess.Popen(
292 | self.wrap_cmd, env=os.environ, preexec_fn=_subprocess_setup)
293 | self.spawn_message = True
294 |
295 | def started(self):
296 | """
297 | Called after Websockets server startup (i.e. after daemonize)
298 | """
299 | # Need to call wrapped command after daemonization so we can
300 | # know when the wrapped command exits
301 | if self.wrap_cmd:
302 | dst_string = "'%s' (port %s)" % (
303 | " ".join(self.wrap_cmd), self.target_port)
304 | elif self.unix_target:
305 | dst_string = self.unix_target
306 | else:
307 | dst_string = "%s:%s" % (self.target_host, self.target_port)
308 |
309 | if self.token_plugin:
310 | msg = " - proxying from %s:%s to targets generated by %s" % (
311 | self.listen_host, self.listen_port, type(self.token_plugin).__name__)
312 | else:
313 | msg = " - proxying from %s:%s to %s" % (
314 | self.listen_host, self.listen_port, dst_string)
315 |
316 | if self.ssl_target:
317 | msg += " (using SSL)"
318 |
319 | self.msg("%s", msg)
320 |
321 | if self.wrap_cmd:
322 | self.run_wrap_cmd()
323 |
324 | def poll(self):
325 | # If we are wrapping a command, check it's status
326 |
327 | if self.wrap_cmd and self.cmd:
328 | ret = self.cmd.poll()
329 | if ret is not None:
330 | self.vmsg(
331 | "Wrapped command exited (or daemon). Returned %s" % ret)
332 | self.cmd = None
333 |
334 | if self.wrap_cmd and self.cmd is None:
335 | # Response to wrapped command being gone
336 | if self.wrap_mode == "ignore":
337 | pass
338 | elif self.wrap_mode == "exit":
339 | sys.exit(ret)
340 | elif self.wrap_mode == "respawn":
341 | now = time.time()
342 | avg = sum(self.wrap_times)/len(self.wrap_times)
343 | if (now - avg) < 10:
344 | # 3 times in the last 10 seconds
345 | if self.spawn_message:
346 | self.warn("Command respawning too fast")
347 | self.spawn_message = False
348 | else:
349 | self.run_wrap_cmd()
350 |
351 |
352 | def _subprocess_setup():
353 | # Python installs a SIGPIPE handler by default. This is usually not what
354 | # non-Python successfulbprocesses expect.
355 | signal.signal(signal.SIGPIPE, signal.SIG_DFL)
356 |
357 |
358 | def logger_init():
359 | logger = logging.getLogger(WebSocketProxy.log_prefix)
360 | logger.propagate = False
361 | logger.setLevel(logging.INFO)
362 | h = logging.StreamHandler()
363 | h.setLevel(logging.DEBUG)
364 | h.setFormatter(logging.Formatter("%(message)s"))
365 | logger.addHandler(h)
366 |
367 |
368 | def websockify_init():
369 | logger_init()
370 |
371 | usage = "\n %prog [options]"
372 | usage += " [source_addr:]source_port [target_addr:target_port]"
373 | usage += "\n %prog [options]"
374 | usage += " [source_addr:]source_port -- WRAP_COMMAND_LINE"
375 | parser = optparse.OptionParser(usage=usage)
376 | parser.add_option("--verbose", "-v", action="store_true",
377 | help="verbose messages")
378 | parser.add_option("--traffic", action="store_true",
379 | help="per frame traffic")
380 | parser.add_option("--record",
381 | help="record sessions to FILE.[session_number]", metavar="FILE")
382 | parser.add_option("--daemon", "-D",
383 | dest="daemon", action="store_true",
384 | help="become a daemon (background process)")
385 | parser.add_option("--run-once", action="store_true",
386 | help="handle a single WebSocket connection and exit")
387 | parser.add_option("--timeout", type=int, default=0,
388 | help="after TIMEOUT seconds exit when not connected")
389 | parser.add_option("--idle-timeout", type=int, default=0,
390 | help="server exits after TIMEOUT seconds if there are no "
391 | "active connections")
392 | parser.add_option("--cert", default="self.pem",
393 | help="SSL certificate file")
394 | parser.add_option("--key", default=None,
395 | help="SSL key file (if separate from cert)")
396 | parser.add_option("--ssl-only", action="store_true",
397 | help="disallow non-encrypted client connections")
398 | parser.add_option("--ssl-target", action="store_true",
399 | help="connect to SSL target as SSL client")
400 | parser.add_option("--unix-target",
401 | help="connect to unix socket target", metavar="FILE")
402 | parser.add_option("--web", default=None, metavar="DIR",
403 | help="run webserver on same port. Serve files from DIR.")
404 | parser.add_option("--wrap-mode", default="exit", metavar="MODE",
405 | choices=["exit", "ignore", "respawn"],
406 | help="action to take when the wrapped program exits "
407 | "or daemonizes: exit (default), ignore, respawn")
408 | parser.add_option("--prefer-ipv6", "-6",
409 | action="store_true", dest="source_is_ipv6",
410 | help="prefer IPv6 when resolving source_addr")
411 | parser.add_option("--libserver", action="store_true",
412 | help="use Python library SocketServer engine")
413 | parser.add_option("--target-config", metavar="FILE",
414 | dest="target_cfg",
415 | help="Configuration file containing valid targets "
416 | "in the form 'token: host:port' or, alternatively, a "
417 | "directory containing configuration files of this form "
418 | "(DEPRECATED: use `--token-plugin TokenFile --token-source "
419 | " path/to/token/file` instead)")
420 | parser.add_option("--token-plugin", default=None, metavar="PLUGIN",
421 | help="use the given Python class to process tokens "
422 | "into host:port pairs")
423 | parser.add_option("--token-source", default=None, metavar="ARG",
424 | help="an argument to be passed to the token plugin"
425 | "on instantiation")
426 | parser.add_option("--auth-plugin", default=None, metavar="PLUGIN",
427 | help="use the given Python class to determine if "
428 | "a connection is allowed")
429 | parser.add_option("--auth-source", default=None, metavar="ARG",
430 | help="an argument to be passed to the auth plugin"
431 | "on instantiation")
432 | parser.add_option("--auto-pong", action="store_true",
433 | help="Automatically respond to ping frames with a pong")
434 | parser.add_option("--heartbeat", type=int, default=0,
435 | help="send a ping to the client every HEARTBEAT seconds")
436 | parser.add_option("--log-file", metavar="FILE",
437 | dest="log_file",
438 | help="File where logs will be saved")
439 |
440 | (opts, args) = parser.parse_args()
441 |
442 | if opts.log_file:
443 | opts.log_file = os.path.abspath(opts.log_file)
444 | handler = logging.FileHandler(opts.log_file)
445 | handler.setLevel(logging.DEBUG)
446 | handler.setFormatter(logging.Formatter("%(message)s"))
447 | logging.getLogger(WebSocketProxy.log_prefix).addHandler(handler)
448 |
449 | del opts.log_file
450 |
451 | if opts.verbose:
452 | logging.getLogger(WebSocketProxy.log_prefix).setLevel(logging.DEBUG)
453 |
454 | if opts.token_source and not opts.token_plugin:
455 | parser.error("You must use --token-plugin to use --token-source")
456 |
457 | if opts.auth_source and not opts.auth_plugin:
458 | parser.error("You must use --auth-plugin to use --auth-source")
459 |
460 | # Transform to absolute path as daemon may chdir
461 | if opts.target_cfg:
462 | opts.target_cfg = os.path.abspath(opts.target_cfg)
463 |
464 | if opts.target_cfg:
465 | opts.token_plugin = 'TokenFile'
466 | opts.token_source = opts.target_cfg
467 |
468 | del opts.target_cfg
469 |
470 | # Sanity checks
471 | if len(args) < 2 and not (opts.token_plugin or opts.unix_target):
472 | parser.error("Too few arguments")
473 | if sys.argv.count('--'):
474 | opts.wrap_cmd = args[1:]
475 | else:
476 | opts.wrap_cmd = None
477 | if len(args) > 2:
478 | parser.error("Too many arguments")
479 |
480 | if not websocket.ssl and opts.ssl_target:
481 | parser.error("SSL target requested and Python SSL module not loaded.")
482 |
483 | if opts.ssl_only and not os.path.exists(opts.cert):
484 | parser.error("SSL only and %s not found" % opts.cert)
485 |
486 | # Parse host:port and convert ports to numbers
487 | if args[0].count(':') > 0:
488 | opts.listen_host, opts.listen_port = args[0].rsplit(':', 1)
489 | opts.listen_host = opts.listen_host.strip('[]')
490 | else:
491 | opts.listen_host, opts.listen_port = '', args[0]
492 |
493 | try:
494 | opts.listen_port = int(opts.listen_port)
495 | except Exception:
496 | parser.error("Error parsing listen port")
497 |
498 | if opts.wrap_cmd or opts.unix_target or opts.token_plugin:
499 | opts.target_host = None
500 | opts.target_port = None
501 | else:
502 | if args[1].count(':') > 0:
503 | opts.target_host, opts.target_port = args[1].rsplit(':', 1)
504 | opts.target_host = opts.target_host.strip('[]')
505 | else:
506 | parser.error("Error parsing target")
507 | try:
508 | opts.target_port = int(opts.target_port)
509 | except Exception:
510 | parser.error("Error parsing target port")
511 |
512 | if opts.token_plugin is not None:
513 | if '.' not in opts.token_plugin:
514 | opts.token_plugin = (
515 | 'websockify.token_plugins.%s' % opts.token_plugin)
516 |
517 | token_plugin_module, token_plugin_cls = opts.token_plugin.rsplit(
518 | '.', 1)
519 |
520 | __import__(token_plugin_module)
521 | token_plugin_cls = getattr(
522 | sys.modules[token_plugin_module], token_plugin_cls)
523 |
524 | opts.token_plugin = token_plugin_cls(opts.token_source)
525 |
526 | del opts.token_source
527 |
528 | if opts.auth_plugin is not None:
529 | if '.' not in opts.auth_plugin:
530 | opts.auth_plugin = 'websockify.auth_plugins.%s' % opts.auth_plugin
531 |
532 | auth_plugin_module, auth_plugin_cls = opts.auth_plugin.rsplit('.', 1)
533 |
534 | __import__(auth_plugin_module)
535 | auth_plugin_cls = getattr(
536 | sys.modules[auth_plugin_module], auth_plugin_cls)
537 |
538 | opts.auth_plugin = auth_plugin_cls(opts.auth_source)
539 |
540 | del opts.auth_source
541 |
542 | # Create and start the WebSockets proxy
543 | libserver = opts.libserver
544 | del opts.libserver
545 | if libserver:
546 | # Use standard Python SocketServer framework
547 | server = LibProxyServer(**opts.__dict__)
548 | server.serve_forever()
549 | else:
550 | # Use internal service framework
551 | server = WebSocketProxy(**opts.__dict__)
552 | server.start_server()
553 |
554 |
555 | class LibProxyServer(ForkingMixIn, HTTPServer):
556 | """
557 | Just like WebSocketProxy, but uses standard Python SocketServer
558 | framework.
559 | """
560 |
561 | def __init__(self, RequestHandlerClass=ProxyRequestHandler, **kwargs):
562 | # Save off proxy specific options
563 | self.target_host = kwargs.pop('target_host', None)
564 | self.target_port = kwargs.pop('target_port', None)
565 | self.wrap_cmd = kwargs.pop('wrap_cmd', None)
566 | self.wrap_mode = kwargs.pop('wrap_mode', None)
567 | self.unix_target = kwargs.pop('unix_target', None)
568 | self.ssl_target = kwargs.pop('ssl_target', None)
569 | self.token_plugin = kwargs.pop('token_plugin', None)
570 | self.auth_plugin = kwargs.pop('auth_plugin', None)
571 | self.heartbeat = kwargs.pop('heartbeat', None)
572 |
573 | self.token_plugin = None
574 | self.auth_plugin = None
575 | self.daemon = False
576 |
577 | # Server configuration
578 | listen_host = kwargs.pop('listen_host', '')
579 | listen_port = kwargs.pop('listen_port', None)
580 | web = kwargs.pop('web', '')
581 |
582 | # Configuration affecting base request handler
583 | self.only_upgrade = not web
584 | self.verbose = kwargs.pop('verbose', False)
585 | record = kwargs.pop('record', '')
586 | if record:
587 | self.record = os.path.abspath(record)
588 | self.run_once = kwargs.pop('run_once', False)
589 | self.handler_id = 0
590 |
591 | for arg in kwargs.keys():
592 | print("warning: option %s ignored when using --libserver" % arg)
593 |
594 | if web:
595 | os.chdir(web)
596 |
597 | HTTPServer.__init__(self, (listen_host, listen_port),
598 | RequestHandlerClass)
599 |
600 | def process_request(self, request, client_address):
601 | """Override process_request to implement a counter"""
602 | self.handler_id += 1
603 | ForkingMixIn.process_request(self, request, client_address)
604 |
605 |
606 | if __name__ == '__main__':
607 | websockify_init()
608 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | beautifulsoup4>=4.12.3
2 | pysocks>=1.7.1
3 | rethinkdb>=2.4.10
4 | quandl>=3.7.0
5 | nltk>=3.9.1
6 | exifread>=3.0.0
7 | blockchain>=1.4.4
8 | websockify>=0.11.0
9 | shodan>=1.31.0
10 | urllib3>=2.2.1
11 | fuzzywuzzy>=0.18.0
12 | scrapy>=2.12.0
13 | pytest>=8.3.0
14 | termcolor>=2.4.0
15 | pycld2>=0.41
16 | polyglot>=16.7.4
17 | tika>=2.6.0
18 | pyzillow>=0.7.0
19 | geotext>=0.4.0
20 | tabulate>=0.9.0
21 | tqdm>=4.66.4
22 | redis>=5.0.4
23 | psycopg2-binary>=2.9.9
24 | pypdf2>=3.0.1
25 | pinboard>=2.1.9
26 | webdriver-manager>=4.0.2
27 | scapy>=2.5.0
28 | matplotlib>=3.9.0
29 | iptcinfo3>=2.1.4
30 | requests>=2.31.0
31 | lxml>=5.2.0
32 | pillow>=10.3.0
33 | numpy>=1.26.0
34 |
--------------------------------------------------------------------------------
/setup.cfg:
--------------------------------------------------------------------------------
1 | [flake8]
2 | max-line-length = 100
--------------------------------------------------------------------------------