├── .gitignore ├── Dockerfile ├── Makefile ├── README.md ├── root ├── .profile ├── .tmux.conf ├── .tmuxinator │ └── server-example.yml └── src │ ├── server.py │ └── server.rb └── slides ├── assets ├── chart.sketch └── kernelpatch.pxm ├── portsharding.key ├── portsharding.pdf └── preview.png /.gitignore: -------------------------------------------------------------------------------- 1 | docker.built 2 | .viminfo 3 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | 3 | RUN apt-get update -y 4 | RUN apt-get install -y python 5 | RUN apt-get install -y ruby 6 | RUN apt-get install -y vim 7 | RUN apt-get install -y tmux 8 | RUN apt-get install -y netcat-openbsd 9 | RUN gem install tmuxinator 10 | 11 | EXPOSE 1234 12 | VOLUME /root 13 | 14 | CMD ["tmuxinator", "start", "server-example"] 15 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | run: docker.built 2 | docker run -it --volume=$(shell pwd)/root:/root -p 1234:1234 server-example 3 | .PHONY: run 4 | 5 | docker.built: 6 | @rm -f $@ 7 | docker build -t server-example . 8 | @touch $@ 9 | 10 | clean: 11 | rm -f docker.built 12 | .PHONY: clean 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Linux port sharding demo 2 | ======================== 3 | 4 | Demo showing how to use the `SO_REUSEPORT` linux socket option, allowing multiple processes to listen on the same TCP or UDP port. 5 | 6 | Example available in Ruby and Python (both do the same thing): 7 | - [server.py](root/src/server.py) 8 | - [server.rb](root/src/server.rb) 9 | 10 | What is this? 11 | ------------- 12 | 13 | ![SO_REUSEPORT sharding](slides/preview.png) 14 | 15 | See my really quick [Linux Port Sharding](https://speakerdeck.com/joewalnes/linux-port-sharding) walkthrough first. 16 | 17 | Setup 18 | ----- 19 | 20 | - If you're on Linux with kernel 3.9 or later (check with `uname -v`), you can just run the Python or Ruby scripts directly. 21 | - If you're on Mac OSX, fret not, install Docker and run `make`, which will install a working Docker image and drop you into a ready to go tmux session, where you can call the scripts. 22 | 23 | Usage 24 | ----- 25 | 26 | Run the server, it listens on port 1234. Connect to it with netcat or telnet to see it do something: `nc localhost 1234`. 27 | 28 | Now run another server process, and a few more. You'll see the kernel allows these all to listen on the *same* port. 29 | 30 | Connect to the port. You'll see the kernel picks one of the processes and allows it to handle it. Next time it may be another process. 31 | 32 | You can add and remove processes. You can even run both the Python and Ruby processes at the same time and the kernel will share the workload. However, note that queued connections which have not been accepted will be lost when stopping a process. 33 | 34 | More 35 | ---- 36 | 37 | * [The SO_REUSEPORT socket option](https://lwn.net/Articles/542629/) 38 | * [Using SO_REUSEPORT with Python](http://www.giantflyingsaucer.com/blog/?p=4684) 39 | -------------------------------------------------------------------------------- /root/.profile: -------------------------------------------------------------------------------- 1 | export EDITOR=vim 2 | export SHELL=bash 3 | export PS1='$ ' 4 | unset HISTFILE 5 | -------------------------------------------------------------------------------- /root/.tmux.conf: -------------------------------------------------------------------------------- 1 | # Remap C-b to C-a 2 | set-option -g prefix C-a 3 | bind-key C-a last-window 4 | bind-key a send-prefix 5 | 6 | # Allows for faster key repetition 7 | set -s escape-time 0 8 | 9 | # Mouse 10 | set -g mode-mouse on 11 | set -g mouse-resize-pane on 12 | set -g mouse-select-pane on 13 | set -g mouse-select-window on 14 | -------------------------------------------------------------------------------- /root/.tmuxinator/server-example.yml: -------------------------------------------------------------------------------- 1 | name: server-example 2 | root: ~/src 3 | 4 | windows: 5 | - main: 6 | layout: main-vertical 7 | panes: 8 | - # 9 | - # 10 | - # 11 | - # 12 | - # 13 | - # 14 | -------------------------------------------------------------------------------- /root/src/server.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from os import getpid 4 | from socket import * 5 | 6 | port = 1234 7 | 8 | server = socket(AF_INET, SOCK_STREAM) 9 | server.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1) 10 | server.bind(('', port)) 11 | server.listen(0) 12 | 13 | print('[python pid:{}] listening on port {}...'.format(getpid(), port)) 14 | 15 | while True: 16 | client, addr = server.accept() 17 | print('[python pid:{}] got connection from {}'.format( 18 | getpid(), client.getpeername())) 19 | client.send('Hello from Python process {}\n'.format(getpid())) 20 | client.close() 21 | -------------------------------------------------------------------------------- /root/src/server.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | require "socket" 4 | 5 | port = 1234 6 | 7 | server = Socket.new Socket::AF_INET, Socket::SOCK_STREAM 8 | server.setsockopt Socket::SOL_SOCKET, Socket::SO_REUSEPORT, true 9 | server.bind Addrinfo.tcp '', port 10 | server.listen 0 11 | 12 | puts "[ruby pid:#{Process.pid}] listening on port #{port}..." 13 | 14 | loop do 15 | client, addr = server.accept 16 | puts "[ruby pid:#{Process.pid}] got connection from #{addr.ip_unpack}" 17 | client.send "Hello from Ruby process #{Process.pid}\n", 0 18 | client.close 19 | end 20 | -------------------------------------------------------------------------------- /slides/assets/chart.sketch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/joewalnes/port-sharding/4ff08d4c55ba200990a42293ebac7c93ea9068a2/slides/assets/chart.sketch -------------------------------------------------------------------------------- /slides/assets/kernelpatch.pxm: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/joewalnes/port-sharding/4ff08d4c55ba200990a42293ebac7c93ea9068a2/slides/assets/kernelpatch.pxm -------------------------------------------------------------------------------- /slides/portsharding.key: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/joewalnes/port-sharding/4ff08d4c55ba200990a42293ebac7c93ea9068a2/slides/portsharding.key -------------------------------------------------------------------------------- /slides/portsharding.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/joewalnes/port-sharding/4ff08d4c55ba200990a42293ebac7c93ea9068a2/slides/portsharding.pdf -------------------------------------------------------------------------------- /slides/preview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/joewalnes/port-sharding/4ff08d4c55ba200990a42293ebac7c93ea9068a2/slides/preview.png --------------------------------------------------------------------------------