├── .gitignore
├── CHANGELOG
├── LICENSE
├── MANIFEST.in
├── README.md
├── docs
    ├── Makefile
    ├── _templates
    │   └── empty
    ├── _theme
    │   └── nature
    │   │   ├── static
    │   │       ├── nature.css_t
    │   │       └── pygments.css
    │   │   └── theme.conf
    ├── actions.rst
    ├── conf.py
    ├── configuration_file.rst
    ├── configuring_rules.rst
    ├── guides
    │   └── simple.rst
    ├── index.rst
    └── rest_api.rst
├── examples
    └── default.conf
├── mantrid
    ├── __init__.py
    ├── actions.py
    ├── cli.py
    ├── client.py
    ├── config.py
    ├── greenbody.py
    ├── loadbalancer.py
    ├── management.py
    ├── socketmeld.py
    ├── static
    │   ├── no-hosts.http
    │   ├── test.http
    │   ├── timeout.http
    │   └── unknown.http
    ├── stats_socket.py
    └── tests
    │   ├── __init__.py
    │   ├── actions.py
    │   ├── client.py
    │   └── loadbalancer.py
└── setup.py


/.gitignore:
--------------------------------------------------------------------------------
 1 | *.wpr
 2 | *.bundle
 3 | *.pyc
 4 | *.swp
 5 | *.swo
 6 | *.un~
 7 | *.egg-info
 8 | */pip-delete-this-directory.txt
 9 | build/*
10 | dist/*
11 | docs/_build
12 | 


--------------------------------------------------------------------------------
/CHANGELOG:
--------------------------------------------------------------------------------
 1 | 1.0.3
 2 | -----
 3 | 
 4 |  * Fixed traceback when more than one port being bound was unavailable
 5 | 
 6 | 
 7 | 1.0.2
 8 | -----
 9 | 
10 |  * Fixed issue with MANIFEST.in including the wrong static files
11 | 
12 | 
13 | 1.0.1
14 | -----
15 | 
16 |  * Fixed issue with setting backend ports via mantrid-client
17 |  * Changed error constants to use the errno module
18 | 
19 | 
20 | 1.0
21 | ---
22 | 
23 |  * Initial release
24 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | Copyright (c) 2011, Epio Limited
 2 | All rights reserved.
 3 | 
 4 | Redistribution and use in source and binary forms, with or without
 5 | modification, are permitted provided that the following conditions are met:
 6 |     * Redistributions of source code must retain the above copyright
 7 |       notice, this list of conditions and the following disclaimer.
 8 |     * Redistributions in binary form must reproduce the above copyright
 9 |       notice, this list of conditions and the following disclaimer in the
10 |       documentation and/or other materials provided with the distribution.
11 |     * Neither the name of the <organization> nor the
12 |       names of its contributors may be used to endorse or promote products
13 |       derived from this software without specific prior written permission.
14 | 
15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
16 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
18 | DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
19 | DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
22 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
25 | 


--------------------------------------------------------------------------------
/MANIFEST.in:
--------------------------------------------------------------------------------
1 | include mantrid/static/*.http
2 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | Mantrid
 2 | =======
 3 | 
 4 | Mantrid is the HTTP load balancer used for [Epio](https://www.ep.io). It is designed with high availability and simplicity in mind: it is configured at runtime with JSON over HTTP and can temporarily hold open connections while backend servers restart. It monitors bandwidth and connection statistics and is ideal for serving large numbers of hostnames.
 5 | 
 6 | It trades some raw speed for flexibility, but is still designed to be fast. Its aim is to have latency of no more than 10ms, and have no more than a 10% reduction in throughput.
 7 | 
 8 | Compatibility
 9 | -------------
10 | 
11 | Mantrid is designed to work with Python 2.6 or 2.7, and requires a Python implementation that supports greenlets (so either CPython or PyPy 1.7 and up).
12 | 
13 | Quick start
14 | -----------
15 | 
16 | Install Mantrid:
17 | 
18 |     $ sudo python setup.py install
19 | 
20 | Launch Mantrid with the default settings (listening on port 80, management on 8042):
21 | 
22 |     $ sudo mantrid
23 | 
24 | Add a host:
25 | 
26 |     $ mantrid-client set localhost static false type=test
27 | 
28 | Then visit http://localhost/ to see the test page.
29 | 
30 | 
31 | Configuration
32 | -------------
33 | 
34 | Mantrid is partially configured using a small configuration file (`/etc/mantrid/mantrid.conf`) which sets up basic things like ports to listen on. The hostnames and load balancing rules are configured at runtime using a REST API, and persisted to a state file on disk so they survive restarts.
35 | 
36 | A command-line interface, `mantrid-client`, ships with Mantrid to make simple interactions with the API easier.
37 | 
38 | Contributing
39 | ------------
40 | 
41 | Mantrid is released under a BSD (3-clause) license. Contributions are very welcome - come and chat with us in #epio on freenode! If you discover a security issue, please email <security@ep.io>.
42 | 
43 | 


--------------------------------------------------------------------------------
/docs/Makefile:
--------------------------------------------------------------------------------
 1 | # Makefile for Sphinx documentation
 2 | #
 3 | 
 4 | # You can set these variables from the command line.
 5 | SPHINXOPTS    =
 6 | SPHINXBUILD   = sphinx-build
 7 | PAPER         =
 8 | 
 9 | # Internal variables.
10 | PAPEROPT_a4     = -D latex_paper_size=a4
11 | PAPEROPT_letter = -D latex_paper_size=letter
12 | ALLSPHINXOPTS   = -d _build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
13 | 
14 | .PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest
15 | 
16 | help:
17 | 	@echo "Please use \`make <target>' where <target> is one of"
18 | 	@echo "  html       to make standalone HTML files"
19 | 	@echo "  dirhtml    to make HTML files named index.html in directories"
20 | 	@echo "  pickle     to make pickle files"
21 | 	@echo "  json       to make JSON files"
22 | 	@echo "  htmlhelp   to make HTML files and a HTML help project"
23 | 	@echo "  qthelp     to make HTML files and a qthelp project"
24 | 	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
25 | 	@echo "  changes    to make an overview of all changed/added/deprecated items"
26 | 	@echo "  linkcheck  to check all external links for integrity"
27 | 	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
28 | 	@echo "  websupport to generate data for WebSupport"
29 | 
30 | clean:
31 | 	-rm -rf _build/*
32 | 
33 | html:
34 | 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) _build/html
35 | 	@echo
36 | 	@echo "Build finished. The HTML pages are in _build/html."
37 | 
38 | dirhtml:
39 | 	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) _build/dirhtml
40 | 	@echo
41 | 	@echo "Build finished. The HTML pages are in _build/dirhtml."
42 | 
43 | pickle:
44 | 	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) _build/pickle
45 | 	@echo
46 | 	@echo "Build finished; now you can process the pickle files."
47 | 
48 | json:
49 | 	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) _build/json
50 | 	@echo
51 | 	@echo "Build finished; now you can process the JSON files."
52 | 
53 | htmlhelp:
54 | 	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) _build/htmlhelp
55 | 	@echo
56 | 	@echo "Build finished; now you can run HTML Help Workshop with the" \
57 | 	      ".hhp project file in _build/htmlhelp."
58 | 
59 | qthelp:
60 | 	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) _build/qthelp
61 | 	@echo
62 | 	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
63 | 	      ".qhcp project file in _build/qthelp, like this:"
64 | 	@echo "# qcollectiongenerator _build/qthelp/South.qhcp"
65 | 	@echo "To view the help file:"
66 | 	@echo "# assistant -collectionFile _build/qthelp/South.qhc"
67 | 
68 | latex:
69 | 	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) _build/latex
70 | 	@echo
71 | 	@echo "Build finished; the LaTeX files are in _build/latex."
72 | 	@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
73 | 	      "run these through (pdf)latex."
74 | 
75 | changes:
76 | 	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) _build/changes
77 | 	@echo
78 | 	@echo "The overview file is in _build/changes."
79 | 
80 | linkcheck:
81 | 	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) _build/linkcheck
82 | 	@echo
83 | 	@echo "Link check complete; look for any errors in the above output " \
84 | 	      "or in _build/linkcheck/output.txt."
85 | 
86 | doctest:
87 | 	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) _build/doctest
88 | 	@echo "Testing of doctests in the sources finished, look at the " \
89 | 	      "results in _build/doctest/output.txt."
90 | 
91 | websupport:
92 | 	mkdir -p _build/websupport/
93 | 	touch index.rst
94 | 	touch quickstart/index.rst
95 | 	touch guides/index.rst
96 | 	python -c "from sphinx.websupport import WebSupport; support = WebSupport(srcdir='.', builddir='_build/websupport/'); support.build()"
97 | 


--------------------------------------------------------------------------------
/docs/_templates/empty:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/epio/mantrid/1c699f1a4b33888b533c19cb6d025173f2160576/docs/_templates/empty


--------------------------------------------------------------------------------
/docs/_theme/nature/static/nature.css_t:
--------------------------------------------------------------------------------
  1 | /**
  2 |  * Sphinx stylesheet -- default theme
  3 |  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  4 |  */
  5 |  
  6 | @import url("basic.css");
  7 |  
  8 | /* -- page layout ----------------------------------------------------------- */
  9 |  
 10 | body {
 11 |     font-family: Arial, sans-serif;
 12 |     font-size: 100%;
 13 |     background-color: #111;
 14 |     color: #555;
 15 |     margin: 0;
 16 |     padding: 0;
 17 | }
 18 | 
 19 | div.documentwrapper {
 20 |     float: left;
 21 |     width: 100%;
 22 | }
 23 | 
 24 | div.bodywrapper {
 25 |     margin: 0 0 0 230px;
 26 | }
 27 | 
 28 | hr{
 29 |     border: 1px solid #B1B4B6;
 30 | }
 31 |  
 32 | div.document {
 33 |     background-color: #eee;
 34 | }
 35 |  
 36 | div.body {
 37 |     background-color: #ffffff;
 38 |     color: #3E4349;
 39 |     padding: 0 30px 30px 30px;
 40 |     font-size: 0.8em;
 41 | }
 42 |  
 43 | div.footer {
 44 |     color: #555;
 45 |     width: 100%;
 46 |     padding: 13px 0;
 47 |     text-align: center;
 48 |     font-size: 75%;
 49 | }
 50 |  
 51 | div.footer a {
 52 |     color: #444;
 53 |     text-decoration: underline;
 54 | }
 55 |  
 56 | div.related {
 57 |     background-color: #6BA81E;
 58 |     line-height: 32px;
 59 |     color: #fff;
 60 |     text-shadow: 0px 1px 0 #444;
 61 |     font-size: 0.80em;
 62 | }
 63 |  
 64 | div.related a {
 65 |     color: #E2F3CC;
 66 | }
 67 |  
 68 | div.sphinxsidebar {
 69 |     font-size: 0.75em;
 70 |     line-height: 1.5em;
 71 | }
 72 | 
 73 | div.sphinxsidebarwrapper{
 74 |     padding: 0 0;
 75 | }
 76 |  
 77 | div.sphinxsidebar h3,
 78 | div.sphinxsidebar h4 {
 79 |     font-family: Arial, sans-serif;
 80 |     color: #222;
 81 |     font-size: 1.2em;
 82 |     font-weight: normal;
 83 |     margin: 0;
 84 |     padding: 5px 10px;
 85 |     background-color: #ddd;
 86 |     text-shadow: 1px 1px 0 white
 87 | }
 88 | 
 89 | div.sphinxsidebar h4{
 90 |     font-size: 1.1em;
 91 | }
 92 |  
 93 | div.sphinxsidebar h3 a {
 94 |     color: #444;
 95 | }
 96 |  
 97 |  
 98 | div.sphinxsidebar p {
 99 |     color: #888;
100 |     padding: 5px 20px;
101 | }
102 |  
103 | div.sphinxsidebar p.topless {
104 | }
105 |  
106 | div.sphinxsidebar ul {
107 |     margin: 10px 20px;
108 |     padding: 0;
109 |     color: #000;
110 | }
111 |  
112 | div.sphinxsidebar a {
113 |     color: #444;
114 | }
115 |  
116 | div.sphinxsidebar input {
117 |     border: 1px solid #ccc;
118 |     font-family: sans-serif;
119 |     font-size: 1em;
120 | }
121 | 
122 | div.sphinxsidebar input[type=text]{
123 |     margin-left: 20px;
124 | }
125 |  
126 | /* -- body styles ----------------------------------------------------------- */
127 |  
128 | a {
129 |     color: #005B81;
130 |     text-decoration: none;
131 | }
132 |  
133 | a:hover {
134 |     color: #E32E00;
135 |     text-decoration: underline;
136 | }
137 |  
138 | div.body h1,
139 | div.body h2,
140 | div.body h3,
141 | div.body h4,
142 | div.body h5,
143 | div.body h6 {
144 |     font-family: Arial, sans-serif;
145 |     background-color: #BED4EB;
146 |     font-weight: normal;
147 |     color: #212224;
148 |     margin: 30px 0px 10px 0px;
149 |     padding: 5px 0 5px 10px;
150 |     text-shadow: 0px 1px 0 white
151 | }
152 |  
153 | div.body h1 { border-top: 20px solid white; margin-top: 0; font-size: 200%; }
154 | div.body h2 { font-size: 150%; background-color: #C8D5E3; }
155 | div.body h3 { font-size: 120%; background-color: #D8DEE3; }
156 | div.body h4 { font-size: 110%; background-color: #D8DEE3; }
157 | div.body h5 { font-size: 100%; background-color: #D8DEE3; }
158 | div.body h6 { font-size: 100%; background-color: #D8DEE3; }
159 |  
160 | a.headerlink {
161 |     color: #c60f0f;
162 |     font-size: 0.8em;
163 |     padding: 0 4px 0 4px;
164 |     text-decoration: none;
165 | }
166 |  
167 | a.headerlink:hover {
168 |     background-color: #c60f0f;
169 |     color: white;
170 | }
171 |  
172 | div.body p, div.body dd, div.body li {
173 |     line-height: 1.5em;
174 | }
175 |  
176 | div.admonition p.admonition-title + p {
177 |     display: inline;
178 | }
179 | 
180 | div.highlight{
181 |     background-color: white;
182 | }
183 | 
184 | div.note {
185 |     background-color: #eee;
186 |     border: 1px solid #ccc;
187 | }
188 |  
189 | div.seealso {
190 |     background-color: #ffc;
191 |     border: 1px solid #ff6;
192 | }
193 |  
194 | div.topic {
195 |     background-color: #eee;
196 | }
197 |  
198 | div.warning {
199 |     background-color: #ffe4e4;
200 |     border: 1px solid #f66;
201 | }
202 |  
203 | p.admonition-title {
204 |     display: inline;
205 | }
206 |  
207 | p.admonition-title:after {
208 |     content: ":";
209 | }
210 |  
211 | pre {
212 |     padding: 10px;
213 |     background-color: White;
214 |     color: #222;
215 |     line-height: 1.2em;
216 |     border: 1px solid #C6C9CB;
217 |     font-size: 1.2em;
218 |     margin: 1.5em 0 1.5em 0;
219 |     -webkit-box-shadow: 1px 1px 1px #d8d8d8;
220 |     -moz-box-shadow: 1px 1px 1px #d8d8d8;
221 | }
222 |  
223 | tt {
224 |     background-color: #ecf0f3;
225 |     color: #222;
226 |     padding: 1px 2px;
227 |     font-size: 1.2em;
228 |     font-family: monospace;
229 | }
230 | 


--------------------------------------------------------------------------------
/docs/_theme/nature/static/pygments.css:
--------------------------------------------------------------------------------
 1 | .c { color: #999988; font-style: italic } /* Comment */
 2 | .k { font-weight: bold } /* Keyword */
 3 | .o { font-weight: bold } /* Operator */
 4 | .cm { color: #999988; font-style: italic } /* Comment.Multiline */
 5 | .cp { color: #999999; font-weight: bold } /* Comment.preproc */
 6 | .c1 { color: #999988; font-style: italic } /* Comment.Single */
 7 | .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */
 8 | .ge { font-style: italic } /* Generic.Emph */
 9 | .gr { color: #aa0000 } /* Generic.Error */
10 | .gh { color: #999999 } /* Generic.Heading */
11 | .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */
12 | .go { color: #111 } /* Generic.Output */
13 | .gp { color: #555555 } /* Generic.Prompt */
14 | .gs { font-weight: bold } /* Generic.Strong */
15 | .gu { color: #aaaaaa } /* Generic.Subheading */
16 | .gt { color: #aa0000 } /* Generic.Traceback */
17 | .kc { font-weight: bold } /* Keyword.Constant */
18 | .kd { font-weight: bold } /* Keyword.Declaration */
19 | .kp { font-weight: bold } /* Keyword.Pseudo */
20 | .kr { font-weight: bold } /* Keyword.Reserved */
21 | .kt { color: #445588; font-weight: bold } /* Keyword.Type */
22 | .m { color: #009999 } /* Literal.Number */
23 | .s { color: #bb8844 } /* Literal.String */
24 | .na { color: #008080 } /* Name.Attribute */
25 | .nb { color: #999999 } /* Name.Builtin */
26 | .nc { color: #445588; font-weight: bold } /* Name.Class */
27 | .no { color: #ff99ff } /* Name.Constant */
28 | .ni { color: #800080 } /* Name.Entity */
29 | .ne { color: #990000; font-weight: bold } /* Name.Exception */
30 | .nf { color: #990000; font-weight: bold } /* Name.Function */
31 | .nn { color: #555555 } /* Name.Namespace */
32 | .nt { color: #000080 } /* Name.Tag */
33 | .nv { color: purple } /* Name.Variable */
34 | .ow { font-weight: bold } /* Operator.Word */
35 | .mf { color: #009999 } /* Literal.Number.Float */
36 | .mh { color: #009999 } /* Literal.Number.Hex */
37 | .mi { color: #009999 } /* Literal.Number.Integer */
38 | .mo { color: #009999 } /* Literal.Number.Oct */
39 | .sb { color: #bb8844 } /* Literal.String.Backtick */
40 | .sc { color: #bb8844 } /* Literal.String.Char */
41 | .sd { color: #bb8844 } /* Literal.String.Doc */
42 | .s2 { color: #bb8844 } /* Literal.String.Double */
43 | .se { color: #bb8844 } /* Literal.String.Escape */
44 | .sh { color: #bb8844 } /* Literal.String.Heredoc */
45 | .si { color: #bb8844 } /* Literal.String.Interpol */
46 | .sx { color: #bb8844 } /* Literal.String.Other */
47 | .sr { color: #808000 } /* Literal.String.Regex */
48 | .s1 { color: #bb8844 } /* Literal.String.Single */
49 | .ss { color: #bb8844 } /* Literal.String.Symbol */
50 | .bp { color: #999999 } /* Name.Builtin.Pseudo */
51 | .vc { color: #ff99ff } /* Name.Variable.Class */
52 | .vg { color: #ff99ff } /* Name.Variable.Global */
53 | .vi { color: #ff99ff } /* Name.Variable.Instance */
54 | .il { color: #009999 } /* Literal.Number.Integer.Long */


--------------------------------------------------------------------------------
/docs/_theme/nature/theme.conf:
--------------------------------------------------------------------------------
1 | [theme]
2 | inherit = basic
3 | stylesheet = nature.css
4 | pygments_style = tango
5 | 


--------------------------------------------------------------------------------
/docs/actions.rst:
--------------------------------------------------------------------------------
  1 | Actions
  2 | =======
  3 | 
  4 | Actions are what you build rules with in Mantrid - they cover all the tasks from returning error pages to proxying requests through to a backend (as you'd expect a load balancer to do).
  5 | 
  6 | Each action has zero or more configuration options - where no table of options is shown, the action in question takes no configuration options.
  7 | 
  8 | 
  9 | empty
 10 | -----
 11 | 
 12 | .. table:: 
 13 | 
 14 |     ========  ========  ===========
 15 |     Argument  Required  Description
 16 |     ========  ========  ===========
 17 |     code      Yes       The (numeric) HTTP code to send as a response
 18 |     ========  ========  ===========
 19 | 
 20 | Sends a HTTP response with a zero-length body (literally just the status code and response headers).
 21 | 
 22 | 
 23 | proxy
 24 | -----
 25 | 
 26 | .. table:: 
 27 | 
 28 |     ========    ========  ===========
 29 |     Argument    Required  Description
 30 |     ========    ========  ===========
 31 |     backends    Yes       A list of backend servers to use
 32 |     attempts    No        How many times a connection is attempted to the backends. Defaults to 1.
 33 |     delay       No        Delay between attempts, in seconds. Defaults to 1.
 34 |     ========    ========  ===========
 35 | 
 36 | Proxies the request through to a backend server. Will randomly choose a server from those provided as "backends"; provides no session stickiness.
 37 | 
 38 | If a connection to a backend drops, it can optionally retry several times with a delay until it gets a response. If no connection is ever accomplished, will send the ``timeout`` static page.
 39 | 
 40 | 
 41 | redirect
 42 | --------
 43 | 
 44 | .. table:: 
 45 | 
 46 |     ===========    ========  ===========
 47 |     Argument       Required  Description
 48 |     ===========    ========  ===========
 49 |     redirect_to    Yes       The URL to redirect to.
 50 |     ===========    ========  ===========
 51 | 
 52 | Sends a HTTP 302 Found response redirecting the user to a different URL. If the URL specified has the protocol part included, they will be forced onto that protocol; otherwise, they will be forwarded with the same protocol (http or https) that they are currently using.
 53 | 
 54 | Note that the use of HTTPS is detected by the presence of an ``X-Forwarded-Proto`` or ``X-Forwarded-Protocol`` header on a ``bind_internal`` interface. Mantrid cannot do SSL termination itself.
 55 | 
 56 | 
 57 | spin
 58 | ----
 59 | 
 60 | .. table:: 
 61 | 
 62 |     ==============  ========  ===========
 63 |     Argument        Required  Description
 64 |     ==============  ========  ===========
 65 |     timeout         No        How long to wait before giving up, in seconds. Defaults to 120.
 66 |     check_interval  No        Delay between rule checks, in seconds. Defaults to 1.
 67 |     ==============  ========  ===========
 68 | 
 69 | Holds the incoming request open, checking Mantrid's rules table periodically for a match. If a new rule is added matching the hostname while the request is being held open, Mantrid will then pass control over to the new action. If no new rule is added before the timeout expires, sends the ``timeout`` static response.
 70 | 
 71 | This is particularly useful for webservers that are being started or restarted; you can set the site to ``spin``, restart the webserver (knowing that your requests are being held), and then set the rule back to ``proxy`` again and all the requests will continue as normal.
 72 | 
 73 | 
 74 | static
 75 | ------
 76 | 
 77 | .. table:: 
 78 | 
 79 |     ========  ========  ===========
 80 |     Argument  Required  Description
 81 |     ========  ========  ===========
 82 |     type      Yes       The name of the static response to send, without the ``.http`` extension.
 83 |     ========  ========  ===========
 84 | 
 85 | Sends a HTTP response that is already saved as a file on disk. Mantrid ships with several default responses, but you can provide your own in the directory specified by the ``static_dir`` configuration option.
 86 | 
 87 | Default responses:
 88 | 
 89 |  * ``no-hosts``, used by the ``no_hosts`` action (short message for a fresh mantrid install)
 90 |  * ``test``, a short test page that says "Congratulations!...".
 91 |  * ``timeout``, used by the ``spin`` and ``proxy`` actions after a timeout.
 92 |  * ``unknown``, used by the ``unknown`` action.
 93 | 
 94 | 
 95 | no_hosts
 96 | --------
 97 | 
 98 | Sends a predefined response pointing out that the load balancer has no hosts configured at all. The default action if the server is completely devoid of rules (as it would be on a fresh install).
 99 | 
100 | 
101 | unknown
102 | -------
103 | 
104 | Sends a predefined response that says "The site you have tried to access is not known to this server". The default action for any unknown host; takes no arguments. It is unlikely you would want to set this as part of a rule.
105 | 
106 | 
107 | 


--------------------------------------------------------------------------------
/docs/conf.py:
--------------------------------------------------------------------------------
  1 | # -*- coding: utf-8 -*-
  2 | #
  3 | # This file is execfile()d with the current directory set to its containing dir.
  4 | #
  5 | # Note that not all possible configuration values are present in this
  6 | # autogenerated file.
  7 | #
  8 | # All configuration values have a default; values that are commented out
  9 | # serve to show the default.
 10 | 
 11 | import sys, os
 12 | 
 13 | # If extensions (or modules to document with autodoc) are in another directory,
 14 | # add these directories to sys.path here. If the directory is relative to the
 15 | # documentation root, use os.path.abspath to make it absolute, like shown here.
 16 | #sys.path.append(os.path.abspath('.'))
 17 | 
 18 | # -- General configuration -----------------------------------------------------
 19 | 
 20 | # Add any Sphinx extension module names here, as strings. They can be extensions
 21 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 22 | extensions = []
 23 | 
 24 | # Add any paths that contain templates here, relative to this directory.
 25 | templates_path = ['_templates']
 26 | 
 27 | # The suffix of source filenames.
 28 | source_suffix = '.rst'
 29 | 
 30 | # The encoding of source files.
 31 | #source_encoding = 'utf-8'
 32 | 
 33 | # The master toctree document.
 34 | master_doc = 'index'
 35 | 
 36 | # General information about the project.
 37 | project = u'Mantrid'
 38 | copyright = u'2011 Epio Limited'
 39 | 
 40 | # The version info for the project you're documenting, acts as replacement for
 41 | # |version| and |release|, also used in various other places throughout the
 42 | # built documents.
 43 | #
 44 | # The short X.Y version.
 45 | version = '1.0'
 46 | # The full version, including alpha/beta/rc tags.
 47 | release = '1.0'
 48 | 
 49 | # The language for content autogenerated by Sphinx. Refer to documentation
 50 | # for a list of supported languages.
 51 | #language = None
 52 | 
 53 | # There are two options for replacing |today|: either, you set today to some
 54 | # non-false value, then it is used:
 55 | #today = ''
 56 | # Else, today_fmt is used as the format for a strftime call.
 57 | #today_fmt = '%B %d, %Y'
 58 | 
 59 | # List of documents that shouldn't be included in the build.
 60 | #unused_docs = []
 61 | 
 62 | # List of directories, relative to source directory, that shouldn't be searched
 63 | # for source files.
 64 | exclude_trees = ['_build']
 65 | 
 66 | # The reST default role (used for this markup: `text`) to use for all documents.
 67 | #default_role = None
 68 | 
 69 | # If true, '()' will be appended to :func: etc. cross-reference text.
 70 | #add_function_parentheses = True
 71 | 
 72 | # If true, the current module name will be prepended to all description
 73 | # unit titles (such as .. function::).
 74 | #add_module_names = True
 75 | 
 76 | # If true, sectionauthor and moduleauthor directives will be shown in the
 77 | # output. They are ignored by default.
 78 | #show_authors = False
 79 | 
 80 | # The name of the Pygments (syntax highlighting) style to use.
 81 | pygments_style = 'sphinx'
 82 | 
 83 | # A list of ignored prefixes for module index sorting.
 84 | #modindex_common_prefix = []
 85 | 
 86 | 
 87 | # -- Options for HTML output ---------------------------------------------------
 88 | 
 89 | # The theme to use for HTML and HTML Help pages.  Major themes that come with
 90 | # Sphinx are currently 'default' and 'sphinxdoc'.
 91 | html_theme = 'nature'
 92 | 
 93 | # Theme options are theme-specific and customize the look and feel of a theme
 94 | # further.  For a list of options available for each theme, see the
 95 | # documentation.
 96 | #html_theme_options = {}
 97 | 
 98 | # Add any paths that contain custom themes here, relative to this directory.
 99 | html_theme_path = ["_theme"]
100 | 
101 | # The name for this set of Sphinx documents.  If None, it defaults to
102 | # "<project> v<release> documentation".
103 | #html_title = None
104 | 
105 | # A shorter title for the navigation bar.  Default is the same as html_title.
106 | #html_short_title = None
107 | 
108 | # The name of an image file (relative to this directory) to place at the top
109 | # of the sidebar.
110 | #html_logo = "_static/logo.png"
111 | 
112 | # The name of an image file (within the static path) to use as favicon of the
113 | # docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
114 | # pixels large.
115 | #html_favicon = None
116 | 
117 | # Add any paths that contain custom static files (such as style sheets) here,
118 | # relative to this directory. They are copied after the builtin static files,
119 | # so a file named "default.css" will overwrite the builtin "default.css".
120 | html_static_path = ['_static']
121 | 
122 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
123 | # using the given strftime format.
124 | #html_last_updated_fmt = '%b %d, %Y'
125 | 
126 | # If true, SmartyPants will be used to convert quotes and dashes to
127 | # typographically correct entities.
128 | #html_use_smartypants = True
129 | 
130 | # Custom sidebar templates, maps document names to template names.
131 | #html_sidebars = {}
132 | 
133 | # Additional templates that should be rendered to pages, maps page names to
134 | # template names.
135 | #html_additional_pages = {}
136 | 
137 | # If false, no module index is generated.
138 | #html_use_modindex = True
139 | 
140 | # If false, no index is generated.
141 | #html_use_index = True
142 | 
143 | # If true, the index is split into individual pages for each letter.
144 | #html_split_index = False
145 | 
146 | # If true, links to the reST sources are added to the pages.
147 | #html_show_sourcelink = True
148 | 
149 | # If true, an OpenSearch description file will be output, and all pages will
150 | # contain a <link> tag referring to it.  The value of this option must be the
151 | # base URL from which the finished HTML is served.
152 | #html_use_opensearch = ''
153 | 
154 | # If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
155 | #html_file_suffix = ''
156 | 
157 | # Output file base name for HTML help builder.
158 | htmlhelp_basename = 'mantriddoc'
159 | 
160 | 
161 | # -- Options for LaTeX output --------------------------------------------------
162 | 
163 | # The paper size ('letter' or 'a4').
164 | #latex_paper_size = 'letter'
165 | 
166 | # The font size ('10pt', '11pt' or '12pt').
167 | #latex_font_size = '10pt'
168 | 
169 | # Grouping the document tree into LaTeX files. List of tuples
170 | # (source start file, target name, title, author, documentclass [howto/manual]).
171 | latex_documents = [
172 |   ('index', 'mantrid.tex', u'Mantrid Documentation',
173 |    u'Epio Limited', 'manual'),
174 | ]
175 | 
176 | # The name of an image file (relative to this directory) to place at the top of
177 | # the title page.
178 | #latex_logo = None
179 | 
180 | # For "manual" documents, if this is true, then toplevel headings are parts,
181 | # not chapters.
182 | #latex_use_parts = False
183 | 
184 | # Additional stuff for the LaTeX preamble.
185 | #latex_preamble = ''
186 | 
187 | # Documents to append as an appendix to all manuals.
188 | #latex_appendices = []
189 | 
190 | # If false, no module index is generated.
191 | #latex_use_modindex = True
192 | 


--------------------------------------------------------------------------------
/docs/configuration_file.rst:
--------------------------------------------------------------------------------
 1 | The configuration file
 2 | ======================
 3 | 
 4 | This file is looked for at either ``/etc/mantrid/mantrid.conf`` or the location passed on the command line using the ``-c`` switch.
 5 | 
 6 | The default settings would look like this::
 7 | 
 8 |     bind = *:80
 9 |     bind_management = *:8042
10 |     state_file = /var/lib/mantrid/state.json
11 |     uid = 4321
12 |     gid = 4321
13 |     static_dir = /etc/mantrid/static/
14 | 
15 | 
16 | Address formats
17 | ---------------
18 | 
19 | Mantrid supports both IPv4 and IPv6, so bind addresses can be supplied in both formats::
20 | 
21 |     bind = 10.0.0.45:80
22 |     bind = 0.0.0.0:80
23 |     bind = [fe32::54ab:12cc]:80
24 |     bind = [::]:80
25 |     bind = *:80
26 | 
27 | The special address ``*`` will bind to all connections on either IPv4 or IPv6.
28 | 
29 | 
30 | Options
31 | -------
32 | 
33 | bind
34 | ~~~~
35 | 
36 | Tells Mantrid to bind to the given address and port to serve external users. Use the address ``*`` to listen on all available addresses.
37 | 
38 | This option may be specified more than once to listen on multiple ports or addresses.
39 | 
40 | 
41 | bind_internal
42 | ~~~~~~~~~~~~~
43 | 
44 | Tells Mantrid to bind to the given address and port to serve internal proxies. Use the address ``*`` to listen on all available addresses.
45 | 
46 | Requests from internal proxies will not have their ``X-Forwarded-For`` and ``X-Forwarded-Protocol`` headers removed. 'Internal' bind addresses are for use behind an SSL terminator, which should add these headers itself.
47 | 
48 | This option may be specified more than once to listen on multiple ports or addresses.
49 | 
50 | 
51 | bind_management
52 | ~~~~~~~~~~~~~~~
53 | 
54 | Tells Mantrid to bind to the given address and port to serve management API requests. Use the address ``*`` to listen on all available addresses.
55 | 
56 | Note that there is no authentication on the Mantrid management API; anyone who can access the port can wipe your loadbalancer. We suggest you limit it to either local connections only or a secure subnet.
57 | 
58 | This option may be specified more than once to listen on multiple ports or addresses.
59 | 
60 | 
61 | state_file
62 | ~~~~~~~~~~
63 | 
64 | Specifies the location where Mantrid stores its state between restarts. Defaults to ``/var/lib/mantrid/state.json``. Should be writable by the user Mantrid drops priviledges to; it will attempt to make that possible if it has root access when it is launched.
65 | 
66 | 
67 | uid
68 | ~~~
69 | 
70 | The UID to drop to once Mantrid has started. Defaults to 4321.
71 | 
72 | 
73 | gid
74 | ~~~
75 | 
76 | The GID to drop to once Mantrid has started. Defaults to 4321.
77 | 
78 | 
79 | static_dir
80 | ~~~~~~~~~~
81 | 
82 | The directory which Mantrid will look in for static response files (ending in ``.http``) used by the ``static`` action. Defaults to ``/etc/mantrid/static/``.
83 | 
84 | 
85 | 


--------------------------------------------------------------------------------
/docs/configuring_rules.rst:
--------------------------------------------------------------------------------
 1 | Configuring rules
 2 | =================
 3 | 
 4 | All configuration of Mantrid's load-balancing rules is done at runtime using a REST API (either directly, or via the ``mantrid-client`` command line client). For simplicity, this document will just demonstrate using the command line client.
 5 | 
 6 | Mantrid only works on the basis of hostnames; for each incoming request, it will take its hostname and attempt to resolve it to an *action*. It will first try and find an exact match to the hostname, and if no match is found it will then keep removing the first part of the hostname (up to the first ``.``) until it has consumed the entire hostname.
 7 | 
 8 | Partial matches will only occur if the domain that is eventually partially matched allows subdomain matches.
 9 | 
10 | For example, if we asked for the host "www.andrew.example.com", Mantrid would try to find rules matching these hostnames (in order)::
11 | 
12 |     www.andrew.example.com
13 |     andrew.example.com
14 |     example.com
15 |     com
16 | 
17 | If there was an entry for ``andrew.example.com`` with subdomain matches allowed, this would match. If only exact matches were allowed, however, this would not match that entry.
18 | 
19 | Each rule is made up of three parts: an :doc:`action name <actions>`, arguments for that action (as keywords), and the "are subdomain matches allowed" flag. You can read about the :doc:`actions` and see what options you have.
20 | 
21 | All changes made via the API take effect immediately, for all future requests.
22 | 
23 | 
24 | Adding and updating rules
25 | -------------------------
26 | 
27 | Adding and updating a rule is the same operation, called 'set'; if there's a previous record for the hostname you're setting, it will be overwritten. To add a rule that just returns an empty 403 Forbidden to everyone requesting "top-secret.com", or any subdomains, you would call::
28 | 
29 |     $ mantrid-client set top-secret.com empty true code=403
30 | 
31 | The arguments are, in order, the host name (``top-secret.com``), the action name (``empty``), the subdomains_allowed flag (``true``), and the arguments (``code=403``, to tell the empty action what status code to use).
32 | 
33 | 
34 | Deleting a rule
35 | ---------------
36 | 
37 | Deleting a rule is pretty simple::
38 | 
39 |     $ mantrid-client delete top-secret.com
40 | 
41 | 
42 | Listing rules
43 | -------------
44 | 
45 | You can get a human-readable list of rules using::
46 | 
47 |     $ mantrid-client list
48 | 
49 | This produces something like the following::
50 | 
51 |     HOST                                ACTION                    SUBDOMS 
52 |     top-secret.com                      empty<403>                True    
53 |     www.forever.com                     spin                      True
54 | 
55 | 
56 | 


--------------------------------------------------------------------------------
/docs/guides/simple.rst:
--------------------------------------------------------------------------------
 1 | Guide: A Simple Setup
 2 | =====================
 3 | 
 4 | This guide will show you how to get a very simple Mantrid install working - we'll have one host, which we proxy through to a backend, and then we'll show you how to put it into "spin" mode (where it will hold open incoming connections) and then back into proxy mode, which will then let all the pending connections through.
 5 | 
 6 | First of all, install Mantrid; instructions on how to do this are on :doc:`the main page </index>`. Once you have it installed, you need to start Mantrid::
 7 | 
 8 |     sudo mantrid
 9 | 
10 | Now Mantrid should be listening on port 80, and listening for management connections on localhost:8042. If you go to http://localhost/ now, you should get a simple page telling you that you currently have no hosts.
11 | 
12 | A single host
13 | -------------
14 | 
15 | Let's add an example host - we'll just use "localhost" for now, and tell it to proxy to ``google.com``::
16 | 
17 |     mantrid-client set localhost proxy true backends=localhost:8000
18 | 
19 | That tells the client to set a new rule, for the domain ``localhost``, using the action ``proxy``, handling subdomains as well (``true``), and then specifies the one backend we're using - in this case, we're presuming you're running something on port 8000 locally - change that as required.
20 | 
21 | If you now go to http://localhost/, you should see the application you redirected to appear.
22 | 
23 | Holding back connections
24 | ------------------------
25 | 
26 | Now, let's change localhost to 'spin' incoming connections::
27 | 
28 |     mantrid-client set localhost spin true
29 | 
30 | (spin takes no arguments, so there is nothing after ``true``).
31 | 
32 | If you now visit http://localhost/, your browser will just sit and try and load the page - Mantrid is holding open connections (this is useful if, for example, you are restarting your web servers). Now, you can set it back to proxy mode::
33 | 
34 |     mantrid-client set localhost proxy true backends=localhost:8000
35 | 
36 | Your open connection will then successfully go through and serve the page you saw before.
37 | 
38 | Multiple backends
39 | -----------------
40 | 
41 | You can also set more than one backend; if we set::
42 | 
43 |     mantrid-client set localhost proxy true backends=localhost:8000,localhost:8001
44 | 
45 | then hitting http://localhost/ will randomly connect you through to one of the two ports.
46 | 


--------------------------------------------------------------------------------
/docs/index.rst:
--------------------------------------------------------------------------------
 1 | 
 2 | .. _index:
 3 | 
 4 | Mantrid Documentation
 5 | =====================
 6 | 
 7 | Mantrid is the HTTP load balancer used for `Epio <https://www.ep.io>`_. It is designed with high availability and simplicity in mind: it is configured at runtime with JSON over HTTP and can temporarily hold open connections while backend servers restart. It monitors bandwidth and connection statistics and is ideal for serving large numbers of hostnames.
 8 | 
 9 | It trades some raw speed for flexibility, but is still designed to be fast. Its aim is to have latency of no more than 10ms, and have no more than a 10% reduction in throughput.
10 | 
11 | It is available on `GitHub <https://github.com/epio/mantrid>`_.
12 | 
13 | 
14 | Installation
15 | ------------
16 | 
17 | If you haven't got `pip <http://www.pip-installer.org/>`_ installed, install it from a system package (``python-pip`` on Ubuntu and Debian) or run::
18 | 
19 |     $ sudo easy_install pip
20 | 
21 | Then run::
22 | 
23 |     $ sudo pip install mantrid
24 | 
25 | You can improve performance by using PyPy 1.7 or greater. Just use the pypy-specific pip to install it. At the time of writing, PyPy 1.7 is not yet released, but a nightly build will work.
26 | 
27 | 
28 | Quick start
29 | -----------
30 | 
31 | To run Mantrid with a default configuration, just run::
32 | 
33 |     $ sudo mantrid
34 | 
35 | (Or run ``mantrid`` as root.) Mantrid needs root in order to bind to port 80 and set its resource limits. It automatically drops to a less privileged user once it has started up.
36 | 
37 | The default configuration is to serve external clients on port 80 (from all available addresses), and to have management on port 8042 bound to localhost.
38 | 
39 | See the :doc:`guides/simple` article for a walkthrough of an initial, simple installation.
40 | 
41 | 
42 | Configuration
43 | -------------
44 | 
45 | Mantrid will look for startup configuration in ``/etc/mantrid/mantrid.conf`` by default. You can specify an alternative location on the command line::
46 | 
47 |     $ mantrid -c /home/andrew/mantrid.conf
48 | 
49 | The configuration file is in the format ``variable_name = value`` and comments are denoted by starting them with a ``#``. For available configuration options, see the :doc:`configuration_file` page.
50 | 
51 | Note that the configuration file only tells Mantrid how to start up; configuring hostnames and Mantrid's responses are done via the REST API. You can use the included ``mantrid-client`` tool to interact with the REST API. For more information, read :doc:`configuring_rules`.
52 | 
53 | Running as a normal user
54 | ------------------------
55 | 
56 | If you only make Mantrid listen on port 1024 or greater, there is no need to run it as root. Mantrid won't be able to automatically change resource limits as a normal user, but you can do it manually with things like ``ulimit`` or ``pam_limits``.
57 | 
58 | 
59 | Table of contents
60 | -----------------
61 | 
62 | .. toctree::
63 |     :maxdepth: 2
64 | 
65 |     configuration_file
66 |     configuring_rules
67 |     actions
68 |     rest_api
69 |     guides/simple
70 | 
71 | 


--------------------------------------------------------------------------------
/docs/rest_api.rst:
--------------------------------------------------------------------------------
 1 | REST API
 2 | ========
 3 | 
 4 | Mantrid is configured mainly via a REST API, available on port 8042 by default. All changes are done using HTTP with JSON responses.
 5 | 
 6 | Note that a *rule* is always formatted as a triple of ``[action_name, kwargs, match_subdomains]``, where ``action_name`` is a string, ``kwargs`` is a mapping of strings to strings or integers, and ``match_subdomains`` is a boolean.
 7 | 
 8 | Statistics are returned as a dictionary with four entries: ``open_requests``, ``completed_requests``, ``bytes_sent``, and ``bytes_received``. The names are reasonably self-explanatory, but note that the two byte measurements are only updated once a request is completed.
 9 | 
10 | 
11 | /hostname/
12 | ----------
13 | 
14 | GET
15 | ~~~
16 | 
17 | Returns a dictionary with all hostnames and their rules.
18 | 
19 | PUT
20 | ~~~
21 | 
22 | Accepts a dictionary in the same format that GET produces (hostname: rule)
23 | 
24 | 
25 | /hostname/www.somesite.com/
26 | ---------------------------
27 | 
28 | GET
29 | ~~~
30 | 
31 | Returns the rule for this hostname, or None if there is no rule for it currently.
32 | 
33 | PUT
34 | ~~~
35 | 
36 | Accepts a rule triple to be used for this hostname.
37 | 
38 | DELETE
39 | ~~~~~~
40 | 
41 | Removes the rule for this hostname.
42 | 
43 | 
44 | /stats/
45 | -------
46 | 
47 | GET
48 | ~~~
49 | 
50 | Returns a dictionary with all hostnames and their statistics.
51 | 
52 | 
53 | /stats/www.somesite.com/
54 | ------------------------
55 | 
56 | GET
57 | ~~~
58 | 
59 | Returns the statistics for just the specified hostname.
60 | 
61 | 
62 | 


--------------------------------------------------------------------------------
/examples/default.conf:
--------------------------------------------------------------------------------
 1 | # This configuration file is equivalent to the default Mantrid configuration.
 2 | 
 3 | # Bind to all address on port 80 for untrusted connections
 4 | bind = *:80
 5 | 
 6 | # Don't bind to any addresses for trusted connections
 7 | # bind_internal = [::1]:81
 8 | 
 9 | # Bind to loopback only on port 8042 for management
10 | bind_management = [::1]:8042      # IPv6
11 | bind_management = 127.0.0.1:8042  # IPv4
12 | 
13 | # Save state to /var/lib
14 | state_file = /var/lib/mantrid/state.json
15 | 
16 | # Some hopefully-unused UIDs to drop privs to
17 | uid = 4321
18 | gid = 4321
19 | 
20 | # Default place to look for extra static pages
21 | static_dir = /etc/mantrid/static/
22 | 


--------------------------------------------------------------------------------
/mantrid/__init__.py:
--------------------------------------------------------------------------------
1 | __version__ = "1.0.3"
2 | 


--------------------------------------------------------------------------------
/mantrid/actions.py:
--------------------------------------------------------------------------------
  1 | """
  2 | Contains Mantrid's built-in actions.
  3 | """
  4 | 
  5 | import errno
  6 | import os
  7 | import random
  8 | import eventlet
  9 | from eventlet.green import socket
 10 | from httplib import responses
 11 | from .socketmeld import SocketMelder
 12 | 
 13 | 
 14 | class Action(object):
 15 |     "Base action. Doesn't do anything."
 16 | 
 17 |     def __init__(self, balancer, host, matched_host):
 18 |         self.host = host
 19 |         self.balancer = balancer
 20 |         self.matched_host = matched_host
 21 | 
 22 |     def handle(self, sock, read_data, path, headers):
 23 |         raise NotImplementedError("You must use an Action subclass")
 24 | 
 25 | 
 26 | class Empty(Action):
 27 |     "Sends a code-only HTTP response"
 28 | 
 29 |     code = None
 30 | 
 31 |     def __init__(self, balancer, host, matched_host, code):
 32 |         super(Empty, self).__init__(balancer, host, matched_host)
 33 |         self.code = code
 34 | 
 35 |     def handle(self, sock, read_data, path, headers):
 36 |         "Sends back a static error page."
 37 |         try:
 38 |             sock.sendall("HTTP/1.0 %s %s\r\nConnection: close\r\nContent-length: 0\r\n\r\n" % (self.code, responses.get(self.code, "Unknown")))
 39 |         except socket.error, e:
 40 |             if e.errno != errno.EPIPE:
 41 |                 raise
 42 | 
 43 | 
 44 | class Static(Action):
 45 |     "Sends a static HTTP response"
 46 | 
 47 |     type = None
 48 | 
 49 |     def __init__(self, balancer, host, matched_host, type=None):
 50 |         super(Static, self).__init__(balancer, host, matched_host)
 51 |         if type is not None:
 52 |             self.type = type
 53 | 
 54 |     # Try to get sendfile() using ctypes; otherwise, fall back
 55 |     try:
 56 |         import ctypes
 57 |         _sendfile = ctypes.CDLL("libc.so.6").sendfile
 58 |         _sendfile.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.c_long, ctypes.c_size_t]
 59 |         _sendfile.restype = ctypes.c_ssize_t
 60 |     except Exception:
 61 |         _sendfile = None
 62 | 
 63 |     def handle(self, sock, read_data, path, headers):
 64 |         "Sends back a static error page."
 65 |         assert self.type is not None
 66 |         try:
 67 |             # Get the correct file
 68 |             try:
 69 |                 fh = open(os.path.join(self.balancer.static_dir, "%s.http" % self.type))
 70 |             except IOError:
 71 |                 fh = open(os.path.join(os.path.dirname(__file__), "static", "%s.http" % self.type))
 72 |             # Send it, using sendfile if poss. (no fileno() means we're probably using mock sockets)
 73 |             try:
 74 |                 self._sendfile(sock.fileno(), fh.fileno(), 0, os.fstat(fh.fileno()).st_size)
 75 |             except (TypeError, AttributeError):
 76 |                 sock.sendall(fh.read())
 77 |             # Close the file and socket
 78 |             fh.close()
 79 |             sock.close()
 80 |         except socket.error, e:
 81 |             if e.errno != errno.EPIPE:
 82 |                 raise
 83 | 
 84 | 
 85 | class Unknown(Static):
 86 |     "Standard class for 'nothing matched'"
 87 | 
 88 |     type = "unknown"
 89 | 
 90 | 
 91 | class NoHosts(Static):
 92 |     "Standard class for 'there are no host entries at all'"
 93 | 
 94 |     type = "no-hosts"
 95 | 
 96 | 
 97 | class Redirect(Action):
 98 |     "Sends a redirect"
 99 | 
100 |     type = None
101 | 
102 |     def __init__(self, balancer, host, matched_host, redirect_to):
103 |         super(Redirect, self).__init__(balancer, host, matched_host)
104 |         self.redirect_to = redirect_to
105 | 
106 |     def handle(self, sock, read_data, path, headers):
107 |         "Sends back a static error page."
108 |         if "://" not in self.redirect_to:
109 |             destination = "http%s://%s" % (
110 |                 "s" if headers.get('X-Forwarded-Protocol', headers.get('X-Forwarded-Proto', "")).lower() in ("https", "ssl") else "",
111 |                 self.redirect_to
112 |             )
113 |         else:
114 |             destination = self.redirect_to
115 |         try:
116 |             sock.sendall("HTTP/1.0 302 Found\r\nLocation: %s/%s\r\n\r\n" % (
117 |                 destination.rstrip("/"),
118 |                 path.lstrip("/"),
119 |             ))
120 |         except socket.error, e:
121 |             if e.errno != errno.EPIPE:
122 |                 raise
123 | 
124 | 
125 | class Proxy(Action):
126 |     "Proxies them through to a server. What loadbalancers do."
127 | 
128 |     attempts = 1
129 |     delay = 1
130 | 
131 |     def __init__(self, balancer, host, matched_host, backends, attempts=None, delay=None):
132 |         super(Proxy, self).__init__(balancer, host, matched_host)
133 |         self.backends = backends
134 |         assert self.backends
135 |         if attempts is not None:
136 |             self.attempts = int(attempts)
137 |         if delay is not None:
138 |             self.delay = float(delay)
139 | 
140 |     def handle(self, sock, read_data, path, headers):
141 |         "Sends back a static error page."
142 |         for i in range(self.attempts):
143 |             try:
144 |                 server_sock = eventlet.connect(
145 |                     tuple(random.choice(self.backends)),
146 |                 )
147 |             except socket.error:
148 |                 eventlet.sleep(self.delay)
149 |                 continue
150 |             # Function to help track data usage
151 |             def send_onwards(data):
152 |                 server_sock.sendall(data)
153 |                 return len(data)
154 |             try:
155 |                 size = send_onwards(read_data)
156 |                 size += SocketMelder(sock, server_sock).run()
157 |             except socket.error, e:
158 |                 if e.errno != errno.EPIPE:
159 |                     raise
160 | 
161 | 
162 | class Spin(Action):
163 |     """
164 |     Just holds the request open until either the timeout expires, or
165 |     another action becomes available.
166 |     """
167 | 
168 |     timeout = 120
169 |     check_interval = 1
170 | 
171 |     def __init__(self, balancer, host, matched_host, timeout=None, check_interval=None):
172 |         super(Spin, self).__init__(balancer, host, matched_host)
173 |         if timeout is not None:
174 |             self.timeout = int(timeout)
175 |         if check_interval is not None:
176 |             self.check_interval = int(check_interval)
177 | 
178 |     def handle(self, sock, read_data, path, headers):
179 |         "Just waits, and checks for other actions to replace us"
180 |         for i in range(self.timeout // self.check_interval):
181 |             # Sleep first
182 |             eventlet.sleep(self.check_interval)
183 |             # Check for another action
184 |             action = self.balancer.resolve_host(self.host)
185 |             if not isinstance(action, Spin):
186 |                 return action.handle(sock, read_data, path, headers)
187 |         # OK, nothing happened, so give up.
188 |         action = Static(self.balancer, self.host, self.matched_host, type="timeout")
189 |         return action.handle(sock, read_data, path, headers)
190 | 


--------------------------------------------------------------------------------
/mantrid/cli.py:
--------------------------------------------------------------------------------
  1 | import sys
  2 | from .client import MantridClient
  3 | 
  4 | 
  5 | class MantridCli(object):
  6 |     """Command line interface to Mantrid"""
  7 | 
  8 |     def __init__(self, base_url):
  9 |         self.client = MantridClient(base_url)
 10 | 
 11 |     @classmethod
 12 |     def main(cls):
 13 |         cli = cls("http://localhost:8042")
 14 |         cli.run(sys.argv)
 15 | 
 16 |     @property
 17 |     def action_names(self):
 18 |         for method_name in dir(self):
 19 |             if method_name.startswith("action_") \
 20 |                and method_name != "action_names":
 21 |                 yield method_name[7:]
 22 |         
 23 |     def run(self, argv):
 24 |         # Work out what action we're doing
 25 |         try:
 26 |             action = argv[1]
 27 |         except IndexError:
 28 |             sys.stderr.write(
 29 |                 "Please provide an action (%s).\n" % (
 30 |                     ", ".join(self.action_names),
 31 |                 )
 32 |             )
 33 |             sys.exit(1)
 34 |         if action not in list(self.action_names):
 35 |             sys.stderr.write(
 36 |                 "Action %s does not exist.\n" % (
 37 |                     action,
 38 |                 )
 39 |             )
 40 |             sys.exit(1)
 41 |         # Run it
 42 |         getattr(self, "action_%s" % action)(*argv[2:])
 43 |     
 44 |     def action_list(self):
 45 |         "Lists all hosts on the LB"
 46 |         format = "%-35s %-25s %-8s"
 47 |         print format % ("HOST", "ACTION", "SUBDOMS")
 48 |         for host, details in sorted(self.client.get_all().items()):
 49 |             if details[0] in ("proxy", "mirror"):
 50 |                 action = "%s<%s>" % (
 51 |                     details[0],
 52 |                     ",".join(
 53 |                         "%s:%s" % (host, port)
 54 |                         for host, port in details[1]['backends']
 55 |                     )
 56 |                 )
 57 |             elif details[0] == "static":
 58 |                 action = "%s<%s>" % (
 59 |                     details[0],
 60 |                     details[1]['type'],
 61 |                 )
 62 |             elif details[0] == "redirect":
 63 |                 action = "%s<%s>" % (
 64 |                     details[0],
 65 |                     details[1]['redirect_to'],
 66 |                 )
 67 |             elif details[0] == "empty":
 68 |                 action = "%s<%s>" % (
 69 |                     details[0],
 70 |                     details[1]['code'],
 71 |                 )
 72 |             else:
 73 |                 action = details[0]
 74 |             print format % (host, action, details[2])
 75 |     
 76 |     def action_set(self, hostname=None, action=None, subdoms=None, *args):
 77 |         "Adds a hostname to the LB, or alters an existing one"
 78 |         usage = "set <hostname> <action> <subdoms> [option=value, ...]"
 79 |         if hostname is None:
 80 |             sys.stderr.write("You must supply a hostname.\n")
 81 |             sys.stderr.write("Usage: %s\n" % usage)
 82 |             sys.exit(1)
 83 |         if action is None:
 84 |             sys.stderr.write("You must supply an action.\n")
 85 |             sys.stderr.write("Usage: %s\n" % usage)
 86 |             sys.exit(1)
 87 |         if subdoms is None or subdoms.lower() not in ("true", "false"):
 88 |             sys.stderr.write("You must supply True or False for the subdomains flag.\n")
 89 |             sys.stderr.write("Usage: %s\n" % usage)
 90 |             sys.exit(1)
 91 |         # Grab options
 92 |         options = {}
 93 |         for arg in args:
 94 |             if "=" not in arg:
 95 |                 sys.stderr.write("%s is not a valid option (no =)\n" % (
 96 |                     arg
 97 |                 ))
 98 |                 sys.exit(1)
 99 |             key, value = arg.split("=", 1)
100 |             options[key] = value
101 |         # Sanity-check options
102 |         if action in ("proxy, mirror") and "backends" not in options:
103 |             sys.stderr.write("The %s action requires a backends option.\n" % action)
104 |             sys.exit(1)
105 |         if action == "static" and "type" not in options:
106 |             sys.stderr.write("The %s action requires a type option.\n" % action)
107 |             sys.exit(1)
108 |         if action == "redirect" and "redirect_to" not in options:
109 |             sys.stderr.write("The %s action requires a redirect_to option.\n" % action)
110 |             sys.exit(1)
111 |         if action == "empty" and "code" not in options:
112 |             sys.stderr.write("The %s action requires a code option.\n" % action)
113 |             sys.exit(1)
114 |         # Expand some options from text to datastructure
115 |         if "backends" in options:
116 |             options['backends'] = [
117 |                 (lambda x: (x[0], int(x[1])))(bit.split(":", 1))
118 |                 for bit in options['backends'].split(",")
119 |             ]
120 |         # Set!
121 |         self.client.set(
122 |             hostname,
123 |             [action, options, subdoms.lower() == "true"]
124 |         )
125 |     
126 |     def action_delete(self, hostname):
127 |         "Deletes the hostname from the LB."
128 |         self.client.delete(
129 |             hostname,
130 |         )
131 |     
132 |     def action_stats(self, hostname=None):
133 |         "Shows stats (possibly limited by hostname)"
134 |         format = "%-35s %-11s %-11s %-11s %-11s"
135 |         print format % ("HOST", "OPEN", "COMPLETED", "BYTES IN", "BYTES OUT")
136 |         for host, details in sorted(self.client.stats(hostname).items()):
137 |             print format % (
138 |                 host,
139 |                 details.get("open_requests", 0),
140 |                 details.get("completed_requests", 0),
141 |                 details.get("bytes_received", 0),
142 |                 details.get("bytes_sent", 0),
143 |             )
144 | 


--------------------------------------------------------------------------------
/mantrid/client.py:
--------------------------------------------------------------------------------
 1 | try:
 2 |     import eventlet
 3 |     httplib2 = eventlet.import_patched("httplib2")
 4 | except ImportError:
 5 |     import httplib2
 6 | import json
 7 | 
 8 | 
 9 | class MantridClient(object):
10 |     """
11 |     Class encapsulating Mantrid client operations.
12 |     """
13 | 
14 |     def __init__(self, base_url):
15 |         self.base_url = base_url.rstrip("/")
16 | 
17 |     def _request(self, path, method, body=None):
18 |         "Base request function"
19 |         h = httplib2.Http()
20 |         resp, content = h.request(
21 |             self.base_url + path,
22 |             method,
23 |             body = json.dumps(body),
24 |         )
25 |         if resp['status'] == "200":
26 |             return json.loads(content)
27 |         else:
28 |             raise IOError(
29 |                 "Got %s reponse from server (%s)" % (
30 |                     resp['status'],
31 |                     content,
32 |                 )
33 |             )
34 |     
35 |     def get_all(self):
36 |         "Returns all endpoints"
37 |         return self._request("/hostname/", "GET")
38 |     
39 |     def set_all(self, data):
40 |         "Sets all endpoints"
41 |         return self._request("/hostname/", "PUT", data)
42 |     
43 |     def set(self, hostname, entry):
44 |         "Sets endpoint for a single hostname"
45 |         return self._request("/hostname/%s/" % hostname, "PUT", entry)
46 |     
47 |     def delete(self, hostname):
48 |         "Deletes a single hostname"
49 |         return self._request("/hostname/%s/" % hostname, "DELETE")
50 | 
51 |     def stats(self, hostname=None):
52 |         if hostname:
53 |             return self._request("/stats/%s/" % hostname, "GET")
54 |         else:
55 |             return self._request("/stats/", "GET")
56 | 


--------------------------------------------------------------------------------
/mantrid/config.py:
--------------------------------------------------------------------------------
 1 | import socket
 2 | 
 3 | 
 4 | class SimpleConfig(object):
 5 |     "Simple configuration file parser"
 6 | 
 7 |     def __init__(self, filename):
 8 |         self.filename = filename
 9 |         self.load()
10 |     
11 |     def load(self):
12 |         items = {}
13 |         with open(self.filename) as fh:
14 |             for line in fh:
15 |                 # Clean up line, remove comments
16 |                 line = line.strip()
17 |                 if "#" in line:
18 |                     line = line[:line.index("#")].strip()
19 |                 # Get the values
20 |                 if line:
21 |                     try:
22 |                         variable, value = line.split("=", 1)
23 |                     except ValueError:
24 |                         raise ValueError("Bad config line (no = and not a comment): %s" % line)
25 |                     items.setdefault(variable.strip().lower(), set()).add(value.strip())
26 |         # Save to ourselves
27 |         self.items = items
28 |     
29 |     def __getitem__(self, item):
30 |         values = self.items[item]
31 |         if len(values) > 1:
32 |             raise ValueError("More than one value specified for %s" % item)
33 |         return list(values)[0]
34 |     
35 |     def get(self, item, default=None):
36 |         values = self.items.get(item, set())
37 |         if len(values) == 0:
38 |             return default
39 |         if len(values) > 1:
40 |             raise ValueError("More than one value specified for %s" % item)
41 |         return list(values)[0]
42 |     
43 |     def get_int(self, item, default):
44 |         return int(self.get(item, default))
45 |     
46 |     def get_all(self, item):
47 |         return self.items.get(item, set())
48 |     
49 |     def get_all_addresses(self, item, default=None):
50 |         addresses = set()
51 |         for value in self.get_all(item):
52 |             try:
53 |                 address, port = value.rsplit(":", 1)
54 |                 family = socket.AF_INET
55 |             except ValueError:
56 |                 raise ValueError("Invalid address (no port found): %s" % value)
57 |             if address[0] == "[":
58 |                 address = address.strip("[]")
59 |                 family = socket.AF_INET6
60 |             if address == "*":
61 |                 address = "::"
62 |                 family = socket.AF_INET6
63 |             addresses.add(((address, int(port)), family))
64 |         if not addresses:
65 |             addresses = default or set()
66 |         return addresses
67 | 


--------------------------------------------------------------------------------
/mantrid/greenbody.py:
--------------------------------------------------------------------------------
 1 | from eventlet.greenpool import GreenPool
 2 | from eventlet.event import Event
 3 | 
 4 | 
 5 | class GreenBody(GreenPool):
 6 |     """
 7 |     Special subclass of GreenPool which has a wait() method,
 8 |     that will return when any greenthread inside the pool exits.
 9 |     """
10 | 
11 |     def __init__(self, *args, **kwargs):
12 |         super(GreenBody, self).__init__(*args, **kwargs)
13 |         self.one_exited = Event()
14 | 
15 |     def wait(self):
16 |         return self.one_exited.wait()
17 |     
18 |     def _spawn_done(self, coro):
19 |         super(GreenBody, self)._spawn_done(coro)
20 |         if not self.one_exited.ready():
21 |             self.one_exited.send(coro.wait())
22 | 


--------------------------------------------------------------------------------
/mantrid/loadbalancer.py:
--------------------------------------------------------------------------------
  1 | import eventlet
  2 | import errno
  3 | import logging
  4 | import traceback
  5 | import mimetools
  6 | import resource
  7 | import json
  8 | import os
  9 | import sys
 10 | import argparse
 11 | from eventlet import wsgi
 12 | from eventlet.green import socket
 13 | from .actions import Unknown, Proxy, Empty, Static, Redirect, NoHosts, Spin
 14 | from .config import SimpleConfig
 15 | from .management import ManagementApp
 16 | from .stats_socket import StatsSocket
 17 | from .greenbody import GreenBody
 18 | 
 19 | 
 20 | class Balancer(object):
 21 |     """
 22 |     Main loadbalancer class.
 23 |     """
 24 | 
 25 |     nofile = 102400
 26 |     save_interval = 10
 27 |     action_mapping = {
 28 |         "proxy": Proxy,
 29 |         "empty": Empty,
 30 |         "static": Static,
 31 |         "redirect": Redirect,
 32 |         "unknown": Unknown,
 33 |         "spin": Spin,
 34 |         "no_hosts": NoHosts,
 35 |     }
 36 | 
 37 |     def __init__(self, external_addresses, internal_addresses, management_addresses, state_file, uid=None, gid=65535, static_dir="/etc/mantrid/static/"):
 38 |         """
 39 |         Constructor.
 40 | 
 41 |         Takes one parameter, the dict of ports to listen on.
 42 |         The key in this dict is the port number, and the value
 43 |         is if it's an internal endpoint or not.
 44 |         Internal endpoints do not have X-Forwarded-* stripped;
 45 |         other ones do, and have X-Forwarded-For added.
 46 |         """
 47 |         self.external_addresses = external_addresses
 48 |         self.internal_addresses = internal_addresses
 49 |         self.management_addresses = management_addresses
 50 |         self.state_file = state_file
 51 |         self.uid = uid
 52 |         self.gid = gid
 53 |         self.static_dir = static_dir
 54 | 
 55 |     @classmethod
 56 |     def main(cls):
 57 |         # Parse command-line args
 58 |         parser = argparse.ArgumentParser(description='The Mantrid load balancer')
 59 |         parser.add_argument('--debug', dest='debug', action='store_const', const=True, help='Enable debug logging')
 60 |         parser.add_argument('-c', '--config', dest='config', default=None, metavar="PATH", help='Path to the configuration file')
 61 |         args = parser.parse_args()
 62 |         # Set up logging
 63 |         logger = logging.getLogger()
 64 |         logger.setLevel(logging.DEBUG if args.debug else logging.INFO)
 65 |         # Output to stderr, always
 66 |         sh = logging.StreamHandler()
 67 |         sh.setFormatter(logging.Formatter(
 68 |             fmt = "%(asctime)s - %(levelname)8s: %(message)s",
 69 |             datefmt="%Y-%m-%d %H:%M:%S",
 70 |         ))
 71 |         sh.setLevel(logging.DEBUG)
 72 |         logger.addHandler(sh)
 73 |         # Check they have root access
 74 |         try:
 75 |             resource.setrlimit(resource.RLIMIT_NOFILE, (cls.nofile, cls.nofile))
 76 |         except (ValueError, resource.error):
 77 |             logging.warning("Cannot raise resource limits (run as root/change ulimits)")
 78 |         # Load settings from the config file
 79 |         if args.config is None:
 80 |             if os.path.exists("/etc/mantrid/mantrid.conf"):
 81 |                 args.config = "/etc/mantrid/mantrid.conf"
 82 |                 logging.info("Using configuration file %s" % args.config)
 83 |             else:
 84 |                 args.config = "/dev/null"
 85 |                 logging.info("No configuration file found - using defaults.")
 86 |         else:
 87 |             logging.info("Using configuration file %s" % args.config)
 88 |         config = SimpleConfig(args.config)
 89 |         balancer = cls(
 90 |             config.get_all_addresses("bind", set([(("::", 80), socket.AF_INET6)])),
 91 |             config.get_all_addresses("bind_internal"),
 92 |             config.get_all_addresses("bind_management", set([(("127.0.0.1", 8042), socket.AF_INET), (("::1", 8042), socket.AF_INET6)])),
 93 |             config.get("state_file", "/var/lib/mantrid/state.json"),
 94 |             config.get_int("uid", 4321),
 95 |             config.get_int("gid", 4321),
 96 |             config.get("static_dir", "/etc/mantrid/static/"),
 97 |         )
 98 |         balancer.run()
 99 | 
100 |     def load(self):
101 |         "Loads the state from the state file"
102 |         try:
103 |             if os.path.getsize(self.state_file) <= 1:
104 |                 raise IOError("File is empty.")
105 |             with open(self.state_file) as fh:
106 |                 state = json.load(fh)
107 |                 assert isinstance(state, dict)
108 |                 self.hosts = state['hosts']
109 |                 self.stats = state['stats']
110 |             for key in self.stats:
111 |                 self.stats[key]['open_requests'] = 0
112 |         except (IOError, OSError):
113 |             # There is no state file; start empty.
114 |             self.hosts = {}
115 |             self.stats = {}
116 | 
117 |     def save(self):
118 |         "Saves the state to the state file"
119 |         with open(self.state_file, "w") as fh:
120 |             json.dump({
121 |                 "hosts": self.hosts,
122 |                 "stats": self.stats,
123 |             }, fh)
124 | 
125 |     def run(self):
126 |         # First, initialise the process
127 |         self.load()
128 |         self.running = True
129 |         # Try to ensure the state file is readable
130 |         state_dir = os.path.dirname(self.state_file)
131 |         if not os.path.isdir(state_dir):
132 |             os.makedirs(state_dir)
133 |         if self.uid is not None:
134 |             try:
135 |                 os.chown(state_dir, self.uid, -1)
136 |             except OSError:
137 |                 pass
138 |             try:
139 |                 os.chown(self.state_file, self.uid, -1)
140 |             except OSError:
141 |                 pass
142 |         # Then, launch the socket loops
143 |         pool = GreenBody(
144 |             len(self.external_addresses) +
145 |             len(self.internal_addresses) +
146 |             len(self.management_addresses) +
147 |             1
148 |         )
149 |         pool.spawn(self.save_loop)
150 |         for address, family in self.external_addresses:
151 |             pool.spawn(self.listen_loop, address, family, internal=False)
152 |         for address, family in self.internal_addresses:
153 |             pool.spawn(self.listen_loop, address, family, internal=True)
154 |         for address, family in self.management_addresses:
155 |             pool.spawn(self.management_loop, address, family)
156 |         # Give the other threads a chance to open their listening sockets
157 |         eventlet.sleep(0.5)
158 |         # Drop to the lesser UID/GIDs, if supplied
159 |         if self.gid:
160 |             try:
161 |                 os.setegid(self.gid)
162 |                 os.setgid(self.gid)
163 |             except OSError:
164 |                 logging.error("Cannot change to GID %i (probably not running as root)" % self.gid)
165 |             else:
166 |                 logging.info("Dropped to GID %i" % self.gid)
167 |         if self.uid:
168 |             try:
169 |                 os.seteuid(0)
170 |                 os.setuid(self.uid)
171 |                 os.seteuid(self.uid)
172 |             except OSError:
173 |                 logging.error("Cannot change to UID %i (probably not running as root)" % self.uid)
174 |             else:
175 |                 logging.info("Dropped to UID %i" % self.uid)
176 |         # Ensure we can save to the state file, or die hard.
177 |         try:
178 |             open(self.state_file, "a").close()
179 |         except (OSError, IOError):
180 |             logging.critical("Cannot write to state file %s" % self.state_file)
181 |             sys.exit(1)
182 |         # Wait for one to exit, or for a clean/forced shutdown
183 |         try:
184 |             pool.wait()
185 |         except (KeyboardInterrupt, StopIteration, SystemExit):
186 |             pass
187 |         except:
188 |             logging.error(traceback.format_exc())
189 |         # We're done
190 |         self.running = False
191 |         logging.info("Exiting")
192 | 
193 |     ### Management ###
194 | 
195 |     def save_loop(self):
196 |         """
197 |         Saves the state if it has changed.
198 |         """
199 |         last_hash = hash(repr(self.hosts))
200 |         while self.running:
201 |             eventlet.sleep(self.save_interval)
202 |             next_hash = hash(repr(self.hosts))
203 |             if next_hash != last_hash:
204 |                 self.save()
205 |                 last_hash = next_hash
206 | 
207 |     def management_loop(self, address, family):
208 |         """
209 |         Accepts management requests.
210 |         """
211 |         try:
212 |             sock = eventlet.listen(address, family)
213 |         except socket.error, e:
214 |             logging.critical("Cannot listen on (%s, %s): %s" % (address, family, e))
215 |             return
216 |         # Sleep to ensure we've dropped privileges by the time we start serving
217 |         eventlet.sleep(0.5)
218 |         # Actually serve management
219 |         logging.info("Listening for management on %s" % (address, ))
220 |         management_app = ManagementApp(self)
221 |         try:
222 |             with open("/dev/null", "w") as log_dest:
223 |                 wsgi.server(
224 |                     sock,
225 |                     management_app.handle,
226 |                     log = log_dest,
227 |                 )
228 |         finally:
229 |             sock.close()
230 | 
231 |     ### Client handling ###
232 | 
233 |     def listen_loop(self, address, family, internal=False):
234 |         """
235 |         Accepts incoming connections.
236 |         """
237 |         try:
238 |             sock = eventlet.listen(address, family)
239 |         except socket.error, e:
240 |             if e.errno == errno.EADDRINUSE:
241 |                 logging.critical("Cannot listen on (%s, %s): already in use" % (address, family))
242 |                 raise
243 |             elif e.errno == errno.EACCES and address[1] <= 1024:
244 |                 logging.critical("Cannot listen on (%s, %s) (you might need to launch as root)" % (address, family))
245 |                 return
246 |             logging.critical("Cannot listen on (%s, %s): %s" % (address, family, e))
247 |             return
248 |         # Sleep to ensure we've dropped privileges by the time we start serving
249 |         eventlet.sleep(0.5)
250 |         # Start serving
251 |         logging.info("Listening for requests on %s" % (address, ))
252 |         try:
253 |             eventlet.serve(
254 |                 sock,
255 |                 lambda sock, addr: self.handle(sock, addr, internal),
256 |                 concurrency = 10000,
257 |             )
258 |         finally:
259 |             sock.close()
260 | 
261 |     def resolve_host(self, host, protocol="http"):
262 |         # Special case for empty hosts dict
263 |         if not self.hosts:
264 |             return NoHosts(self, host, "unknown")
265 |         # Check for an exact or any subdomain matches
266 |         bits = host.split(".")
267 |         for i in range(len(bits)):
268 |             for prefix in ["%s://" % protocol, ""]:
269 |                 subhost = prefix + (".".join(bits[i:]))
270 |                 if subhost in self.hosts:
271 |                     action, kwargs, allow_subs = self.hosts[subhost]
272 |                     if allow_subs or i == 0:
273 |                         action_class = self.action_mapping[action]
274 |                         return action_class(
275 |                             balancer = self,
276 |                             host = host,
277 |                             matched_host = subhost,
278 |                             **kwargs
279 |                         )
280 |         return Unknown(self, host, "unknown")
281 | 
282 |     def handle(self, sock, address, internal=False):
283 |         """
284 |         Handles an incoming HTTP connection.
285 |         """
286 |         try:
287 |             sock = StatsSocket(sock)
288 |             rfile = sock.makefile('rb', 4096)
289 |             # Read the first line
290 |             first = rfile.readline().strip("\r\n")
291 |             words = first.split()
292 |             # Ensure it looks kind of like HTTP
293 |             if not (2 <= len(words) <= 3):
294 |                 sock.sendall("HTTP/1.0 400 Bad Request\r\nConnection: close\r\nContent-length: 0\r\n\r\n")
295 |                 return
296 |             path = words[1]
297 |             # Read the headers
298 |             headers = mimetools.Message(rfile, 0)
299 |             # Work out the host
300 |             try:
301 |                 host = headers['Host']
302 |             except KeyError:
303 |                 host = "unknown"
304 |             headers['Connection'] = "close"
305 |             if not internal:
306 |                 headers['X-Forwarded-For'] = address[0]
307 |                 headers['X-Forwarded-Protocol'] = ""
308 |                 headers['X-Forwarded-Proto'] = ""
309 |             # Make sure they're not using odd encodings
310 |             if "Transfer-Encoding" in headers:
311 |                 sock.sendall("HTTP/1.0 411 Length Required\r\nConnection: close\r\nContent-length: 0\r\n\r\n")
312 |                 return
313 |             # Match the host to an action
314 |             protocol = "http"
315 |             if headers.get('X-Forwarded-Protocol', headers.get('X-Forwarded-Proto', "")).lower() in ("ssl", "https"):
316 |                 protocol = "https"
317 |             action = self.resolve_host(host, protocol)
318 |             # Record us as an open connection
319 |             stats_dict = self.stats.setdefault(action.matched_host, {})
320 |             stats_dict['open_requests'] = stats_dict.get('open_requests', 0) + 1
321 |             # Run the action
322 |             try:
323 |                 rfile._rbuf.seek(0)
324 |                 action.handle(
325 |                     sock = sock,
326 |                     read_data = first + "\r\n" + str(headers) + "\r\n" + rfile._rbuf.read(),
327 |                     path = path,
328 |                     headers = headers,
329 |                 )
330 |             finally:
331 |                 stats_dict['open_requests'] -= 1
332 |                 stats_dict['completed_requests'] = stats_dict.get('completed_requests', 0) + 1
333 |                 stats_dict['bytes_sent'] = stats_dict.get('bytes_sent', 0) + sock.bytes_sent
334 |                 stats_dict['bytes_received'] = stats_dict.get('bytes_received', 0) + sock.bytes_received
335 |         except socket.error, e:
336 |             if e.errno not in (errno.EPIPE, errno.ETIMEDOUT, errno.ECONNRESET):
337 |                 logging.error(traceback.format_exc())
338 |         except:
339 |             logging.error(traceback.format_exc())
340 |             try:
341 |                 sock.sendall("HTTP/1.0 500 Internal Server Error\r\n\r\nThere has been an internal error in the load balancer.")
342 |             except socket.error, e:
343 |                 if e.errno != errno.EPIPE:
344 |                     raise
345 |         finally:
346 |             try:
347 |                 sock.close()
348 |                 rfile.close()
349 |             except:
350 |                 logging.error(traceback.format_exc())
351 | 
352 | if __name__ == "__main__":
353 |     Balancer.main()
354 | 


--------------------------------------------------------------------------------
/mantrid/management.py:
--------------------------------------------------------------------------------
  1 | import json
  2 | import re
  3 | 
  4 | 
  5 | class HttpNotFound(Exception):
  6 |     "Exception raised to pass on a 404 error."
  7 |     pass
  8 | 
  9 | 
 10 | class HttpMethodNotAllowed(Exception):
 11 |     "Exception raised for a valid path but invalid method."
 12 |     pass
 13 | 
 14 | 
 15 | class HttpBadRequest(Exception):
 16 |     "Exception raised for an invalidly formed host entry."
 17 |     pass
 18 | 
 19 | 
 20 | class ManagementApp(object):
 21 |     """
 22 |     Management WSGI app for the Mantrid loadbalancer.
 23 |     Allows endpoints to be changed via HTTP requests to
 24 |     the management port.
 25 |     """
 26 | 
 27 |     host_regex = re.compile(r"^/hostname/([^/]+)/?$")
 28 |     stats_host_regex = re.compile(r"^/stats/([^/]+)/?$")
 29 | 
 30 |     def __init__(self, balancer):
 31 |         self.balancer = balancer
 32 | 
 33 |     def handle(self, environ, start_response):
 34 |         "Main entry point"
 35 |         # Pass off to the router
 36 |         try:
 37 |             handler = self.route(
 38 |                 environ['PATH_INFO'].lower(),
 39 |                 environ['REQUEST_METHOD'].lower(),
 40 |             )
 41 |             if handler is None:
 42 |                 raise HttpNotFound()
 43 |         # Handle errors
 44 |         except HttpNotFound:
 45 |             start_response('404 Not Found', [('Content-Type', 'application/json')])
 46 |             return [json.dumps({"error": "not_found"})]
 47 |         except HttpMethodNotAllowed:
 48 |             start_response('405 Method Not Allowed', [('Content-Type', 'application/json')])
 49 |             return [json.dumps({"error": "method_not_allowed"})]
 50 |         # Dispatch to the named method
 51 |         body = environ['wsgi.input'].read()
 52 |         if body:
 53 |             body = json.loads(body)
 54 |         response = handler(
 55 |             environ['PATH_INFO'].lower(),
 56 |             body,
 57 |         )
 58 |         # Send the response
 59 |         start_response('200 OK', [('Content-Type', 'application/json')])
 60 |         return [json.dumps(response)]
 61 | 
 62 |     def route(self, path, method):
 63 |         # Simple routing for paths
 64 |         if path == "/":
 65 |             raise HttpMethodNotAllowed()
 66 |         elif path == "/stats/":
 67 |             if method == "get":
 68 |                 return self.get_all_stats
 69 |             else:
 70 |                 raise HttpMethodNotAllowed()
 71 |         elif self.stats_host_regex.match(path):
 72 |             if method == "get":
 73 |                 return self.get_single_stats
 74 |             else:
 75 |                 raise HttpMethodNotAllowed()
 76 |         elif path == "/hostname/":
 77 |             if method == "get":
 78 |                 return self.get_all
 79 |             elif method == "put":
 80 |                 return self.set_all
 81 |             else:
 82 |                 raise HttpMethodNotAllowed()
 83 |         elif self.host_regex.match(path):
 84 |             if method == "get":
 85 |                 return self.get_single
 86 |             elif method == "put":
 87 |                 return self.set_single
 88 |             elif method == "delete":
 89 |                 return self.delete_single
 90 |             else:
 91 |                 raise HttpMethodNotAllowed()
 92 |         else:
 93 |             raise HttpNotFound()
 94 | 
 95 |     ### Handling methods ###
 96 | 
 97 |     def host_errors(self, hostname, details):
 98 |         """
 99 |         Validates the format of a host entry
100 |         Returns an error string, or None if it is valid.
101 |         """
102 |         if not hostname or not isinstance(hostname, basestring):
103 |             return "hostname_invalid"
104 |         if not isinstance(details, list):
105 |             return "host_details_not_list"
106 |         if len(details) != 3:
107 |             return "host_details_wrong_length"
108 |         if details[0] not in self.balancer.action_mapping:
109 |             return "host_action_invalid:%s" % details[0]
110 |         if not isinstance(details[1], dict):
111 |             return "host_kwargs_not_dict"
112 |         if not isinstance(details[2], bool):
113 |             return "host_match_subdomains_not_bool"
114 |         return None
115 | 
116 |     def get_all(self, path, body):
117 |         return self.balancer.hosts
118 | 
119 |     def set_all(self, path, body):
120 |         "Replaces the hosts list with the provided input"
121 |         # Do some error checking
122 |         if not isinstance(body, dict):
123 |             raise HttpBadRequest("body_not_a_dict")
124 |         for hostname, details in body.items():
125 |             error = self.host_errors(hostname, details)
126 |             if error:
127 |                 raise HttpBadRequest("%s:%s" % (hostname, error))
128 |         # Replace
129 |         old_hostnames = set(self.balancer.hosts.keys())
130 |         new_hostnames = set(body.keys())
131 |         self.balancer.hosts = body
132 |         # Clean up stats dict
133 |         for hostname in new_hostnames - old_hostnames:
134 |             self.balancer.stats[hostname] = {}
135 |         for hostname in old_hostnames - new_hostnames:
136 |             try:
137 |                 del self.balancer.stats[hostname]
138 |             except KeyError:
139 |                 pass
140 |         return {"ok": True}
141 | 
142 |     def get_single(self, path, body):
143 |         host = self.host_regex.match(path).group(1)
144 |         if host in self.balancer.hosts:
145 |             return self.balancer.hosts[host]
146 |         else:
147 |             return None
148 | 
149 |     def set_single(self, path, body):
150 |         host = self.host_regex.match(path).group(1)
151 |         error = self.host_errors(host, body)
152 |         if error:
153 |             raise HttpBadRequest("%s:%s" % (host, error))
154 |         self.balancer.hosts[host] = body
155 |         self.balancer.stats[host] = {}
156 |         return {"ok": True}
157 | 
158 |     def delete_single(self, path, body):
159 |         host = self.host_regex.match(path).group(1)
160 |         try:
161 |             del self.balancer.hosts[host]
162 |         except KeyError:
163 |             pass
164 |         try:
165 |             del self.balancer.stats[host]
166 |         except KeyError:
167 |             pass
168 |         return {"ok": True}
169 | 
170 |     def get_all_stats(self, path, body):
171 |         return self.balancer.stats
172 | 
173 |     def get_single_stats(self, path, body):
174 |         host = self.stats_host_regex.match(path).group(1)
175 |         return self.balancer.stats.get(host, {})
176 | 


--------------------------------------------------------------------------------
/mantrid/socketmeld.py:
--------------------------------------------------------------------------------
 1 | import eventlet
 2 | import greenlet
 3 | from eventlet.green import socket
 4 | 
 5 | 
 6 | class SocketMelder(object):
 7 |     """
 8 |     Takes two sockets and directly connects them together.
 9 |     """
10 | 
11 |     def __init__(self, client, server):
12 |         self.client = client
13 |         self.server = server
14 |         self.data_handled = 0
15 | 
16 |     def piper(self, in_sock, out_sock, out_addr, onkill):
17 |         "Worker thread for data reading"
18 |         try:
19 |             while True:
20 |                 written = in_sock.recv(32768)
21 |                 if not written:
22 |                     try:
23 |                         out_sock.shutdown(socket.SHUT_WR)
24 |                     except socket.error:
25 |                         self.threads[onkill].kill()
26 |                     break
27 |                 try:
28 |                     out_sock.sendall(written)
29 |                 except socket.error:
30 |                     pass
31 |                 self.data_handled += len(written)
32 |         except greenlet.GreenletExit:
33 |             return
34 | 
35 |     def run(self):
36 |         self.threads = {
37 |             "ctos": eventlet.spawn(self.piper, self.server, self.client, "client", "stoc"),
38 |             "stoc": eventlet.spawn(self.piper, self.client, self.server, "server", "ctos"),
39 |         }
40 |         try:
41 |             self.threads['stoc'].wait()
42 |         except (greenlet.GreenletExit, socket.error):
43 |             pass
44 |         try:
45 |             self.threads['ctos'].wait()
46 |         except (greenlet.GreenletExit, socket.error):
47 |             pass
48 |         self.server.close()
49 |         self.client.close()
50 |         return self.data_handled
51 | 


--------------------------------------------------------------------------------
/mantrid/static/no-hosts.http:
--------------------------------------------------------------------------------
 1 | HTTP/1.0 503 Service Unavailable
 2 | Cache-Control: no-cache
 3 | Connection: close
 4 | Content-Type: text/html
 5 | 
 6 | <html>
 7 | <head>
 8 |     <title>No Hosts</title>
 9 |     <style>
10 |         body {
11 |             font-family: sans-serif;
12 |         }
13 |         .footer {
14 |             color: #aaa;
15 |             border-top: 1px solid #aaa;
16 |             padding: 5px 0 0 0;
17 |             margin: 20px 0 0 0;
18 |             font-size: 70%;
19 |         }
20 |     </style>
21 | </head>
22 | <body>
23 |     <h1>No Hosts</h1>
24 |     <p>
25 |     The site you have tried to access is not available; this load balancer has no known hosts
26 |     at all. If you are the administrator, you may want to add some.
27 |     </p>
28 |     <div class="footer">Powered by Mantrid</div>
29 | </body>
30 | </html>
31 | 


--------------------------------------------------------------------------------
/mantrid/static/test.http:
--------------------------------------------------------------------------------
 1 | HTTP/1.0 200 OK
 2 | Cache-Control: no-cache
 3 | Connection: close
 4 | Content-Type: text/html
 5 | 
 6 | <html>
 7 | <head>
 8 |     <title>Mantrid Test Page</title>
 9 |     <style>
10 |         body {
11 |             font-family: sans-serif;
12 |         }
13 |         .footer {
14 |             color: #aaa;
15 |             border-top: 1px solid #aaa;
16 |             padding: 5px 0 0 0;
17 |             margin: 20px 0 0 0;
18 |             font-size: 70%;
19 |         }
20 |     </style>
21 | </head>
22 | <body>
23 |     <h1>Mantrid Test Page</h1>
24 |     <p>
25 |     Congratulations; if you can see this, Mantrid is successfully serving requests.
26 |     </p>
27 |     <div class="footer">Powered by Mantrid</div>
28 | </body>
29 | </html>
30 | 


--------------------------------------------------------------------------------
/mantrid/static/timeout.http:
--------------------------------------------------------------------------------
 1 | HTTP/1.0 502 Gateway Timeout
 2 | Cache-Control: no-cache
 3 | Connection: close
 4 | Content-Type: text/html
 5 | 
 6 | <html>
 7 | <head>
 8 |     <title>Server Timeout</title>
 9 |     <style>
10 |         body {
11 |             font-family: sans-serif;
12 |         }
13 |         .footer {
14 |             color: #aaa;
15 |             border-top: 1px solid #aaa;
16 |             padding: 5px 0 0 0;
17 |             margin: 20px 0 0 0;
18 |             font-size: 70%;
19 |         }
20 |     </style>
21 | </head>
22 | <body>
23 |     <h1>Server Timeout</h1>
24 |     <p>
25 |     The site you have tried to access is not responding. Please try again later.
26 |     </p>
27 |     <div class="footer">Powered by Mantrid</div>
28 | </body>
29 | </html>
30 | 


--------------------------------------------------------------------------------
/mantrid/static/unknown.http:
--------------------------------------------------------------------------------
 1 | HTTP/1.0 503 Service Unavailable
 2 | Cache-Control: no-cache
 3 | Connection: close
 4 | Content-Type: text/html
 5 | 
 6 | <html>
 7 | <head>
 8 |     <title>Unknown Host</title>
 9 |     <style>
10 |         body {
11 |             font-family: sans-serif;
12 |         }
13 |         .footer {
14 |             color: #aaa;
15 |             border-top: 1px solid #aaa;
16 |             padding: 5px 0 0 0;
17 |             margin: 20px 0 0 0;
18 |             font-size: 70%;
19 |         }
20 |     </style>
21 | </head>
22 | <body>
23 |     <h1>Unknown Host</h1>
24 |     <p>
25 |     The site you have tried to access is not known to this server.
26 |     </p>
27 |     <div class="footer">Powered by Mantrid</div>
28 | </body>
29 | </html>
30 | 


--------------------------------------------------------------------------------
/mantrid/stats_socket.py:
--------------------------------------------------------------------------------
 1 | class StatsSocket(object):
 2 |     """
 3 |     Wrapper around a socket that measures how many bytes
 4 |     have been sent and received.
 5 |     """
 6 | 
 7 |     def __init__(self, sock):
 8 |         self.sock = sock
 9 |         self.bytes_sent = 0
10 |         self.bytes_received = 0
11 | 
12 |     def __getattr__(self, attr):
13 |         return getattr(self.sock, attr)
14 |     
15 |     def sendall(self, data):
16 |         self.bytes_sent += len(data)
17 |         self.sock.sendall(data)
18 |     
19 |     def send(self, data):
20 |         sent = self.sock.send(data)
21 |         self.bytes_sent += sent
22 |         return sent
23 |     
24 |     def recv(self, length):
25 |         recvd = self.sock.recv(length)
26 |         self.bytes_received += len(recvd)
27 |         return recvd
28 | 
29 |     def makefile(self, *args, **kwargs):
30 |         fh = self.sock.makefile(*args, **kwargs)
31 |         fh._sock = self
32 |         return fh
33 | 


--------------------------------------------------------------------------------
/mantrid/tests/__init__.py:
--------------------------------------------------------------------------------
1 | from .actions import ActionTests, LiveActionTests
2 | from .loadbalancer import BalancerTests
3 | from .client import ClientTests
4 | 


--------------------------------------------------------------------------------
/mantrid/tests/actions.py:
--------------------------------------------------------------------------------
  1 | import os
  2 | import errno
  3 | import socket
  4 | import time
  5 | import eventlet
  6 | import unittest
  7 | httplib2 = eventlet.import_patched("httplib2")
  8 | from eventlet.timeout import Timeout
  9 | from ..loadbalancer import Balancer
 10 | from ..actions import Empty, Static, Unknown, NoHosts, Redirect, Proxy, Spin
 11 | 
 12 | 
 13 | class MockBalancer(object):
 14 |     "Fake Balancer class for testing."
 15 | 
 16 |     def __init__(self, fixed_action=None):
 17 |         self.fixed_action = None
 18 |         self.static_dir = "/tmp/"
 19 | 
 20 |     def resolve_host(self, host):
 21 |         return self.fixed_action
 22 | 
 23 | 
 24 | class MockSocket(object):
 25 |     "Fake Socket class that remembers what was sent. Doesn't implement sendfile."
 26 | 
 27 |     def __init__(self):
 28 |         self.data = ""
 29 | 
 30 |     def send(self, data):
 31 |         self.data += data
 32 |         return len(data)
 33 | 
 34 |     def sendall(self, data):
 35 |         self.data += data
 36 | 
 37 |     def close(self):
 38 |         pass
 39 | 
 40 | 
 41 | class MockErrorSocket(object):
 42 |     "Fake Socket class that raises a specific error message on use."
 43 | 
 44 |     def __init__(self, error_code):
 45 |         self.error_code = error_code
 46 | 
 47 |     def _error(self, *args, **kwargs):
 48 |         raise socket.error(self.error_code, os.strerror(self.error_code))
 49 |     sendall = _error
 50 | 
 51 | 
 52 | class ActionTests(unittest.TestCase):
 53 |     "Tests the various actions"
 54 | 
 55 |     def test_empty(self):
 56 |         "Tests the Empty action"
 57 |         action = Empty(MockBalancer(), "zomg-lol.com", "zomg-lol.com", code=500)
 58 |         sock = MockSocket()
 59 |         action.handle(sock, "", "/", {})
 60 |         self.assertEqual(
 61 |             "HTTP/1.0 500 Internal Server Error\r\nConnection: close\r\nContent-length: 0\r\n\r\n",
 62 |             sock.data,
 63 |         )
 64 | 
 65 |     def test_handle(self):
 66 |         "Tests the Static action"
 67 |         action = Static(MockBalancer(), "kittens.net", "kittens.net", type="timeout")
 68 |         sock = MockSocket()
 69 |         action.handle(sock, "", "/", {})
 70 |         self.assertEqual(
 71 |             open(os.path.join(os.path.dirname(__file__), "..", "static", "timeout.http")).read(),
 72 |             sock.data,
 73 |         )
 74 | 
 75 |     def test_unknown(self):
 76 |         "Tests the Unknown action"
 77 |         action = Unknown(MockBalancer(), "firefly.org", "firefly.org")
 78 |         sock = MockSocket()
 79 |         action.handle(sock, "", "/", {})
 80 |         self.assertEqual(
 81 |             open(os.path.join(os.path.dirname(__file__), "..", "static", "unknown.http")).read(),
 82 |             sock.data,
 83 |         )
 84 | 
 85 |     def test_nohosts(self):
 86 |         "Tests the NoHosts action"
 87 |         action = NoHosts(MockBalancer(), "thevoid.local", "thevoid.local")
 88 |         sock = MockSocket()
 89 |         action.handle(sock, "", "/", {})
 90 |         self.assertEqual(
 91 |             open(os.path.join(os.path.dirname(__file__), "..", "static", "no-hosts.http")).read(),
 92 |             sock.data,
 93 |         )
 94 | 
 95 |     def test_redirect(self):
 96 |         "Tests the Redirect action"
 97 |         action = Redirect(MockBalancer(), "lions.net", "lions.net", redirect_to="http://tigers.net")
 98 |         # Test with root path
 99 |         sock = MockSocket()
100 |         action.handle(sock, "", "/", {})
101 |         self.assertEqual(
102 |             "HTTP/1.0 302 Found\r\nLocation: http://tigers.net/\r\n\r\n",
103 |             sock.data,
104 |         )
105 |         # Test with non-root path
106 |         sock = MockSocket()
107 |         action.handle(sock, "", "/bears/", {})
108 |         self.assertEqual(
109 |             "HTTP/1.0 302 Found\r\nLocation: http://tigers.net/bears/\r\n\r\n",
110 |             sock.data,
111 |         )
112 |         # Test with https
113 |         action = Redirect(MockBalancer(), "oh-my.com", "oh-my.com", redirect_to="https://meme-overload.com")
114 |         sock = MockSocket()
115 |         action.handle(sock, "", "/bears2/", {})
116 |         self.assertEqual(
117 |             "HTTP/1.0 302 Found\r\nLocation: https://meme-overload.com/bears2/\r\n\r\n",
118 |             sock.data,
119 |         )
120 |         # Test with same-protocol
121 |         action = Redirect(MockBalancer(), "example.com", "example.com", redirect_to="example.net")
122 |         sock = MockSocket()
123 |         action.handle(sock, "", "/test/", {})
124 |         self.assertEqual(
125 |             "HTTP/1.0 302 Found\r\nLocation: http://example.net/test/\r\n\r\n",
126 |             sock.data,
127 |         )
128 |         sock = MockSocket()
129 |         action.handle(sock, "", "/test/", {"X-Forwarded-Protocol": "SSL"})
130 |         self.assertEqual(
131 |             "HTTP/1.0 302 Found\r\nLocation: https://example.net/test/\r\n\r\n",
132 |             sock.data,
133 |         )
134 | 
135 |     def test_proxy(self):
136 |         "Tests the Proxy action"
137 |         # Check failure with no backends
138 |         self.assertRaises(
139 |             AssertionError,
140 |             lambda: Proxy(MockBalancer(), "khaaaaaaaaaaaaan.xxx", "khaaaaaaaaaaaaan.xxx", backends=[]),
141 |         )
142 |         # TODO: launch local server, proxy to that
143 | 
144 |     def test_spin(self):
145 |         "Tests the Spin action"
146 |         # Set the balancer up to return a Spin
147 |         balancer = MockBalancer()
148 |         action = Spin(balancer, "aeracode.org", "aeracode.org", timeout=2, check_interval=1)
149 |         balancer.fixed_action = action
150 |         # Ensure it times out
151 |         sock = MockSocket()
152 |         try:
153 |             with Timeout(2.2):
154 |                 start = time.time()
155 |                 action.handle(sock, "", "/", {})
156 |                 duration = time.time() - start
157 |         except Timeout:
158 |             self.fail("Spin lasted for too long")
159 |         self.assert_(
160 |             duration >= 1,
161 |             "Spin did not last for long enough"
162 |         )
163 |         self.assertEqual(
164 |             open(os.path.join(os.path.dirname(__file__), "..", "static", "timeout.http")).read(),
165 |             sock.data,
166 |         )
167 |         # Now, ensure it picks up a change
168 |         sock = MockSocket()
169 |         try:
170 |             with Timeout(2):
171 |                 def host_changer():
172 |                     eventlet.sleep(0.7)
173 |                     balancer.fixed_action = Empty(balancer, "aeracode.org", "aeracode.org", code=402)
174 |                 eventlet.spawn(host_changer)
175 |                 action.handle(sock, "", "/", {})
176 |         except Timeout:
177 |             self.fail("Spin lasted for too long")
178 |         self.assertEqual(
179 |             "HTTP/1.0 402 Payment Required\r\nConnection: close\r\nContent-length: 0\r\n\r\n",
180 |             sock.data,
181 |         )
182 | 
183 |     def test_socket_errors(self):
184 |         for action in [
185 |             Empty(MockBalancer(), "", "", code=500),
186 |             Unknown(MockBalancer(), "", ""),
187 |             Redirect(MockBalancer(), "", "", redirect_to="http://pypy.org/"),
188 |         ]:
189 |             sock = MockErrorSocket(errno.EPIPE)
190 |             # Doesn't error
191 |             action.handle(sock, "", "/", {})
192 |             sock = MockErrorSocket(errno.EBADF)
193 |             with self.assertRaises(socket.error) as cm:
194 |                 action.handle(sock, "", "/", {})
195 |             self.assertEqual(cm.exception.errno, errno.EBADF)
196 | 
197 | 
198 | class LiveActionTests(unittest.TestCase):
199 |     """
200 |     Tests that the client/API work correctly.
201 |     """
202 | 
203 |     next_port = 30300
204 | 
205 |     def setUp(self):
206 |         self.__class__.next_port += 3
207 |         self.balancer = Balancer(
208 |             [(("0.0.0.0", self.next_port), socket.AF_INET)],
209 |             [(("0.0.0.0", self.next_port + 1), socket.AF_INET)],
210 |             [(("0.0.0.0", self.next_port + 2), socket.AF_INET)],
211 |             "/tmp/mantrid-test-state-2",
212 |         )
213 |         self.balancer_thread = eventlet.spawn(self.balancer.run)
214 |         eventlet.sleep(0.1)
215 |         self.balancer.hosts = {
216 |             "test-host.com": ["static", {"type": "test"}, True],
217 |         }
218 |     
219 |     def tearDown(self):
220 |         self.balancer.running = False
221 |         self.balancer_thread.kill()
222 |         eventlet.sleep(0.1)
223 | 
224 |     def test_unknown(self):
225 |         # Send a HTTP request to the balancer, ensure the response
226 |         # is the same as the "unknown" template
227 |         h = httplib2.Http()
228 |         resp, content = h.request(
229 |             "http://127.0.0.1:%i" % self.next_port,
230 |             "GET",
231 |         )
232 |         self.assertEqual(
233 |             '503',
234 |             resp['status'],
235 |         )
236 |         expected_content = open(os.path.join(os.path.dirname(__file__), "..", "static", "unknown.http")).read()
237 |         expected_content = expected_content[expected_content.index("\r\n\r\n") + 4:]
238 |         self.assertEqual(
239 |             expected_content,
240 |             content,
241 |         )
242 | 


--------------------------------------------------------------------------------
/mantrid/tests/client.py:
--------------------------------------------------------------------------------
  1 | import unittest
  2 | import eventlet
  3 | import socket
  4 | from ..loadbalancer import Balancer
  5 | from ..client import MantridClient
  6 | 
  7 | 
  8 | class MockSocket(object):
  9 |     "Fake Socket class that remembers what was sent"
 10 | 
 11 |     def __init__(self):
 12 |         self.data = ""
 13 | 
 14 |     def send(self, data):
 15 |         self.data += data
 16 |         return len(data)
 17 |     
 18 |     def sendall(self, data):
 19 |         self.data += data
 20 | 
 21 | 
 22 | class ClientTests(unittest.TestCase):
 23 |     """
 24 |     Tests that the client/API work correctly.
 25 |     """
 26 | 
 27 |     next_port = 30200
 28 | 
 29 |     def setUp(self):
 30 |         self.__class__.next_port += 3
 31 |         self.balancer = Balancer(
 32 |             [(("0.0.0.0", self.next_port), socket.AF_INET)],
 33 |             [(("0.0.0.0", self.next_port + 1), socket.AF_INET)],
 34 |             [(("0.0.0.0", self.next_port + 2), socket.AF_INET)],
 35 |             "/tmp/mantrid-test-state",
 36 |         )
 37 |         self.balancer_thread = eventlet.spawn(self.balancer.run)
 38 |         eventlet.sleep(0.1)
 39 |         self.client = MantridClient("http://127.0.0.1:%i" % (self.next_port + 2))
 40 |     
 41 |     def tearDown(self):
 42 |         self.balancer.running = False
 43 |         eventlet.sleep(0.1)
 44 | 
 45 |     def test_set_single(self):
 46 |         "Sets a single host"
 47 |         # Check we start empty
 48 |         self.assertEqual(
 49 |             {},
 50 |             self.balancer.hosts,
 51 |         )
 52 |         self.assertEqual(
 53 |             {},
 54 |             self.balancer.stats,
 55 |         )
 56 |         # Add a single host
 57 |         self.client.set("test-host.com", ["spin", {}, False])
 58 |         # See if we got it
 59 |         self.assertEqual(
 60 |             {"test-host.com": ["spin", {}, False]},
 61 |             self.balancer.hosts,
 62 |         )
 63 |         self.assertEqual(
 64 |             {"test-host.com": {}},
 65 |             self.balancer.stats,
 66 |         )
 67 |         # Override with new settings
 68 |         self.client.set("test-host.com", ["unknown", {}, True])
 69 |         self.assertEqual(
 70 |             {"test-host.com": ["unknown", {}, True]},
 71 |             self.balancer.hosts,
 72 |         )
 73 |         self.assertEqual(
 74 |             {"test-host.com": {}},
 75 |             self.balancer.stats,
 76 |         )
 77 |         # Try a wrong setting
 78 |         self.assertRaises(
 79 |             IOError,
 80 |             self.client.set, "test-host.com", ["do-da-be-dee", {}, "bruce"],
 81 |         )
 82 |         # Delete it
 83 |         self.client.delete("test-host.com")
 84 |         self.assertEqual(
 85 |             {},
 86 |             self.balancer.hosts,
 87 |         )
 88 |         self.assertEqual(
 89 |             {},
 90 |             self.balancer.stats,
 91 |         )
 92 | 
 93 |     def test_set_multiple(self):
 94 |         "Sets a single host"
 95 |         # Check we start empty
 96 |         self.assertEqual(
 97 |             {},
 98 |             self.balancer.hosts,
 99 |         )
100 |         self.assertEqual(
101 |             {},
102 |             self.balancer.stats,
103 |         )
104 |         # Add multiple hosts
105 |         hosts = {
106 |             "kittens.com": ["spin", {}, False],
107 |             "khaaaaaaaaaan.com": ["unknown", {}, True],
108 |         }
109 |         self.client.set_all(hosts)
110 |         self.assertEqual(
111 |             hosts,
112 |             self.balancer.hosts,
113 |         )
114 |         self.assertEqual(
115 |             {"kittens.com": {}, "khaaaaaaaaaan.com": {}},
116 |             self.balancer.stats,
117 |         )
118 |         # Change to a different set of hosts
119 |         hosts = {
120 |             "ceilingcat.net": ["spin", {}, False],
121 |             "khaaaaaaaaaan.com": ["unknown", {}, True],
122 |         }
123 |         self.client.set_all(hosts)
124 |         self.assertEqual(
125 |             hosts,
126 |             self.balancer.hosts,
127 |         )
128 |         self.assertEqual(
129 |             {"ceilingcat.net": {}, "khaaaaaaaaaan.com": {}},
130 |             self.balancer.stats,
131 |         )
132 | 


--------------------------------------------------------------------------------
/mantrid/tests/loadbalancer.py:
--------------------------------------------------------------------------------
 1 | from unittest import TestCase
 2 | from ..loadbalancer import Balancer
 3 | from ..actions import Empty, Unknown, Redirect, Spin, Proxy
 4 | 
 5 | 
 6 | class BalancerTests(TestCase):
 7 |     "Tests the main load balancer class itself"
 8 | 
 9 |     def test_resolution(self):
10 |         "Tests name resolution"
11 |         balancer = Balancer(None, None, None, None)
12 |         balancer.hosts = {
13 |             "localhost": [
14 |                 "empty",
15 |                 {"code": 402},
16 |                 False,
17 |             ],
18 |             "local.ep.io": [
19 |                 "spin",
20 |                 {},
21 |                 True,
22 |             ],
23 |             "http://ep.io": [
24 |                 "redirect",
25 |                 {"redirect_to": "https://www.ep.io"},
26 |                 True,
27 |             ],
28 |             "ep.io": [
29 |                 "proxy",
30 |                 {"backends": ["0.0.0.0:0"]},
31 |                 True,
32 |             ],
33 |         }
34 |         # Test direct name resolution
35 |         self.assertEqual(
36 |             balancer.resolve_host("localhost").__class__,
37 |             Empty,
38 |         )
39 |         self.assertEqual(
40 |             balancer.resolve_host("local.ep.io").__class__,
41 |             Spin,
42 |         )
43 |         self.assertEqual(
44 |             balancer.resolve_host("ep.io").__class__,
45 |             Redirect,
46 |         )
47 |         self.assertEqual(
48 |             balancer.resolve_host("ep.io", "https").__class__,
49 |             Proxy,
50 |         )
51 |         # Test subdomain resolution
52 |         self.assertEqual(
53 |             balancer.resolve_host("subdomain.localhost").__class__,
54 |             Unknown,
55 |         )
56 |         self.assertEqual(
57 |             balancer.resolve_host("subdomain.local.ep.io").__class__,
58 |             Spin,
59 |         )
60 |         self.assertEqual(
61 |             balancer.resolve_host("subdomain.ep.io").__class__,
62 |             Redirect,
63 |         )
64 |         self.assertEqual(
65 |             balancer.resolve_host("multi.level.subdomain.local.ep.io").__class__,
66 |             Spin,
67 |         )
68 |         # Test nonexistent base name
69 |         self.assertEqual(
70 |             balancer.resolve_host("i-love-bees.com").__class__,
71 |             Unknown,
72 |         )
73 | 


--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/python
 2 | 
 3 | # Use setuptools if we can
 4 | try:
 5 |     from setuptools import setup
 6 | except ImportError:
 7 |     from distutils import setup
 8 | 
 9 | from mantrid import __version__
10 | 
11 | setup(
12 |     name = 'mantrid',
13 |     version = __version__,
14 |     author = "Epio Limited",
15 |     author_email= "team@ep.io",
16 |     url = "http://github.com/epio/mantrid/",
17 |     description = 'A pure-Python loadbalancer.',
18 |     long_description = """
19 |         A Python load balancer that is also configurable at runtime.
20 | 
21 |         Development version: https://github.com/epio/mantrid/tarball/master#egg=mantrid-dev
22 |     """,
23 |     packages = [
24 |         "mantrid",
25 |         "mantrid.tests",
26 |     ],
27 |     classifiers = [
28 |         'Development Status :: 5 - Production/Stable',
29 |         'License :: OSI Approved :: BSD License',
30 |         'Programming Language :: Python',
31 |     ],
32 |     entry_points = """
33 |         [console_scripts]
34 |         mantrid = mantrid.loadbalancer:Balancer.main
35 |         mantrid-client = mantrid.cli:MantridCli.main
36 |     """,
37 |     package_data = {
38 |         "mantrid": ["static/*.http"],
39 |     },
40 |     install_requires = [
41 |         "httplib2",
42 |         "argparse",
43 |         "eventlet>=0.9.16",
44 |     ],
45 | )
46 | 


--------------------------------------------------------------------------------