├── .gitignore ├── README.md ├── docker-compose.yml ├── docker ├── config-server │ ├── Dockerfile │ ├── config-render.py │ ├── kick-update.py │ └── templates │ │ └── default ├── ebconfig │ ├── Dockerfile │ ├── ebconfig.py │ └── example.json ├── netstack-setup │ ├── Dockerfile │ └── start.py ├── portconfig │ ├── Dockerfile │ ├── example.json │ └── portconfig.py ├── route-server │ ├── Dockerfile │ ├── config-render.py │ ├── daemons │ ├── ipsec.conf │ └── templates │ │ ├── frr.conf.template │ │ └── ipsec.secrets.template └── routing │ ├── Dockerfile │ ├── config-render.py │ ├── daemons │ ├── ipsec.conf │ └── templates │ ├── frr.conf.template │ └── ipsec.secrets.template ├── nante-wan.conf └── start.py /.gitignore: -------------------------------------------------------------------------------- 1 | .*.swp 2 | myconf 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | Nante-WAN: なんちゃってSD-WAN 3 | ============================== 4 | 5 | Nante-WAN is yet another SD-WAN solution by open source software: 6 | Linux and [FRRouting](https://frrouting.org/). Nante-WAN provides 7 | NAT-traversal, Multipoint, Encrypted Layer-2 overlay networks using 8 | [DMVPN (Dynamic Multipoint 9 | VPN)](https://www.cisco.com/c/en/us/products/security/dynamic-multipoint-vpn-dmvpn/index.html), 10 | [VXLAN](https://datatracker.ietf.org/doc/rfc7348/) and 11 | [EVPN](https://datatracker.ietf.org/doc/draft-ietf-bess-dci-evpn-overlay/). 12 | 13 | 14 | The data plane of Nante-WAN is VXLAN over DMVPN/IPsec overlay network. 15 | DMVPN provides multipoint layer-3 overlay, and IPsec provides packet 16 | encryption and NAT-traversal. Moreover, VXLAN encapsulates Ethernet 17 | frames in IP headers, which are inner IP headers of the DMVPN overlay. 18 | 19 | The control plane of Nante-WAN is composed of FRRouting that is a fork 20 | of Quagga. Nante-WAN uses EVPN for exchanging VXLAN FDB, and NBMA 21 | Nexthop Resolution Protocol (NHRP) for DMVPN. 22 | 23 | 24 | Nante-WAN components are packaged as docker containers, and they run 25 | on Ubuntu 17.10. So, you can (easily?) test and deploy this yet 26 | another SD-WAN in your both physical and virtual machine environments. 27 | 28 | 29 | 30 | ## Nante-WAN components 31 | 32 | #### Fig.1 Overview of Nante-Wan overlay 33 | ![Overview of Nante-WAN Overlay](https://raw.githubusercontent.com/wiki/upa/nante-wan/fig/nante-wan-overlay.png) 34 | 35 | 36 | As shown in Fig.1, a Nante-WAN overlay comprises customer edge (CE) 37 | nodes, Route Server, and Config Server. 38 | 39 | - **CE node**: CE nodes accommodate edge networks and deliver Ethernet 40 | frames from the edge networks to distant and proper destination CE 41 | nodes with VXLAN and IPsec encapsulation. 42 | 43 | - **Route Server**: Route Server is BGP Route Reflector and NHRP 44 | Next-Hop Server. A route server establishes iBGP connections with 45 | all CE nodes, and exchange EVPN routes as a control plane for VXLAN 46 | overlays. Moreover, CE nodes use the route server as a Next Hop 47 | Server to resolve underlay IP addresses associateding IP addresses 48 | on the DMVPN overlay (NHRP). 49 | 50 | - **Config Server**: Config Server is an HTTP server. CE nodes 51 | regurarly fetch bridge interface configurations from the config 52 | server across the DMVPN overlay. Moreover, when a bridge config file 53 | for a CE node is changed, the config server notifies the CE. 54 | 55 | 56 | CE nodes constructs VXLAN over DMVPN overlay using Route Server as 57 | IPsec anchor, Next Hop Resolation on DMVPN, and RR for EVPN. Bridge 58 | interface configurations on CE nodes are centralized in Config 59 | Server. Under this control, CE nodes deliver Ethernet frames from edge 60 | networks to proper CE nodes across the Internet. 61 | 62 | 63 | 64 | 65 | ## Example setup 66 | 67 | 68 | This section describes an example setup for testing Nante-WAN. Please 69 | make four Ubuntu 17.10 VMs and setup a topology shown in Fig.2 using 70 | hypervisor software you like. 71 | 72 | 73 | #### Fig.2 Example test environment. 74 | 75 | ![Example test environment](https://raw.githubusercontent.com/wiki/upa/nante-wan/fig/nante-wan-test-env.png) 76 | 77 | 78 | In the example test environment shown in Fig.2, there are three VMs as 79 | CE nodes (CE 1 ~ 3) and one VM for the route and config server 80 | roles. All nodes are connected to the network 192.168.0.0/24 through 81 | Ethernet interface eth0. This network performs an underlay network, 82 | e.g, the Internet. 83 | 84 | Each node has a *gre1* interface. The gre1 interface is an entry point 85 | to a DMVPN overlay network. Each gre1 interface has a unique /32 IP 86 | address. Those /32 IP addresses on the DMVPN overlay are used for 87 | messaging between CE nodes, route and config servers. **Note**: 88 | nante-wan start script creates gre1 interface. In this step, you don't 89 | need to make gre1 interface by your hand. 90 | 91 | 92 | In the example environment, instead of physical ports, we use network 93 | namespace and veth interface to emulate edge network (172.16.0.0/24, 94 | depicted as orange boxes in Fig.2). After the Nante-WAN overlay is 95 | Up, all namespaces will be connected as a single layer-2 segment 96 | across the underlay network. Then you can ping from any namespaces to 97 | others. 98 | 99 | 100 | How to make an edge network namespace is shown below. 101 | ```shell-session 102 | # change edge_addr accordance with nodes. 103 | export edge_addr=172.16.0.1/24 104 | 105 | export ns=edge-network 106 | 107 | ip link add vetha type veth peer name vethb 108 | ip netns add $ns 109 | ip link set dev vethb netns $ns 110 | 111 | ip link set dev vetha up 112 | ip netns exec $ns ip link set dev vethb up 113 | ip netns exec $ns ip link set dev lo up 114 | ip netns exec $ns ip addr add dev vethb $edge_addr 115 | ``` 116 | 117 | #### List of IP Addresses in this example test environment. 118 | | Node | eth0 | gre1 | vethb | 119 | |:--------------------|:----------------|:-------------|:--------------| 120 | | CE1 | 192.168.0.1/24 | 10.0.0.1/32 | 172.16.0.1/24 | 121 | | CE2 | 192.168.0.2/24 | 10.0.0.2/32 | 172.16.0.2/24 | 122 | | CE3 | 192.168.0.3/24 | 10.0.0.3/32 | 172.16.0.3/24 | 123 | | Route/Config Server | 192.168.0.10/24 | 10.0.0.10/32 | none | 124 | 125 | 126 | 127 | ### 1. Edit nante-wan.conf 128 | 129 | First of all, clone Nante-WAN repository and edit 130 | [nante-wan.conf](https://github.com/upa/nante-wan/blob/master/nante-wan.conf) 131 | 132 | ```shell-session 133 | # at all nodes, 134 | ce1:$ git clone https://github.com/upa/nante-wan.git 135 | ce1:$ cd nante-wan 136 | ce1:$ vim nante-wan.conf 137 | ``` 138 | 139 | nante-wan.conf is configuration file for Nante-WAN. Alhtough dozens of 140 | parameters exist, only a few are important. The nante-wan.conf for 141 | this example environment is shown below. 142 | 143 | ``` 144 | # Nante-WAN example config file 145 | 146 | [general] 147 | dmvpn_addr = 10.0.0.X 148 | 149 | [config_fetch] 150 | timeout = 5 151 | interval = 3600 152 | failed_interval = 5 153 | 154 | [routing] 155 | wan_interface = eth0 156 | dmvpn_interface = gre1 157 | as_number = 65000 158 | nhs_nbma_addr = 192.168.0.10 159 | rr_addr = 10.0.0.10 160 | bgp_range = 10.0.0.0/16 161 | ipsec_secret = hogehogemogamoga 162 | gre_key = 1 163 | gre_ttl = 64 164 | 165 | 166 | [portconfig] 167 | br_interface = bridge 168 | json_url_prefix = http://10.0.0.10/portconfig 169 | bind_port = 8080 170 | 171 | [ebconfig] 172 | br_interface = bridge 173 | json_url_prefix = http://10.0.0.10/ebconfig 174 | bind_port = 8081 175 | ``` 176 | 177 | The most important parameter is **dmvpn_addr**. dmvpn_addr is an IP 178 | address assigned to a gre1 interface of a node, and it is used for 179 | messaging, iBGP (EVPN), and VXLAN encapsulation. Namely, dmvpn_addr 180 | is node's IP address on a DMVPN overlay. Therefore, dmvpn_addr must be 181 | different from other nodes. In this example, it is 10.0.0.1 on CE1, 182 | and 10.0.0.10 on the route/config server. **wan_interface** should be 183 | changed for a proper interface name according as machine 184 | environment. If an interface connecting to underlay (public networks) 185 | is enp1s0 in your environment, *wan_interface* should be enp1s0. 186 | 187 | 188 | Other parameters are identical among all nodes regardless of node 189 | types (CE, route or config server). 190 | 191 | 192 | #### Note 193 | 194 | **nhs_nbma_addr** and **nhs_addr** are NHRP configurations. They 195 | indicate IP addresses of a route server node on underlay (eth0) and 196 | DMVPN overlay (gre1). **rr_addr** is an IP address that iBGP on CE 197 | nodes connect to. Thus, rr_addr is also an IP address of a route 198 | server on DMVPN overlay. 199 | 200 | 201 | 202 | 203 | ### 2. Pull containers 204 | 205 | At CE nodes, clone routing and portconfig containers. The routing 206 | container contains FRRrouting and StrongSwan. The portconfig container 207 | contains a portconfig daemon that configure bridge interfaces. 208 | 209 | ```shell-session 210 | ce1:$ docker pull upaa/nante-wan-routing 211 | ce1:$ docker pull upaa/nante-wan-portconfig 212 | ``` 213 | 214 | At the route and config server, 215 | ```shell-session 216 | server:$ docker pull upaa/nante-wan-route-server 217 | server:$ docker pull upaa/nante-wan-config-serger 218 | ``` 219 | 220 | 221 | All containers contains config rendering scripts. These scripts 222 | generate specific configuration files, for example, frr.conf and IPsec 223 | configuration, from nante-wan.conf. Therefore, you can run containers 224 | with nante-wan.conf to start Nante-WAN without editting such specific 225 | configurations. 226 | 227 | 228 | 229 | ### 3. Run containers 230 | 231 | nante-wan/start.py does all things to start Nante-WAN at nodes. 232 | 233 | * create GRE interface 234 | * create Bridge interface 235 | * setup NFLOG for NHRP redirect/shortcut 236 | * setup TCP MSS clamping (1340 byte) 237 | * run containers 238 | 239 | 240 | At the route/config server node, 241 | ```shell-session 242 | # make a directory to store CEs' bridge configuration files. 243 | server:$ mkdir html 244 | server:$ sudo ./start.py --route-server --config-server --config-dir html nante-wan.conf 245 | ``` 246 | 247 | At CE nodes, 248 | ``` 249 | ce1:$ sudo ./start.py nante-wan.conf 250 | ``` 251 | 252 | 253 | start.py shows executing commands like below. 254 | 255 | ```shell-session 256 | ce2:$ sudo ./start.py nante-wan.conf 257 | # Setup GRE Interface 258 | # wan_interface : eth0 259 | # dmvpn_interface : gre1 260 | # dmvpn_addr : 10.0.0.2 261 | modprobe af_key 262 | /bin/ip tunnel add gre1 mode gre key 1 ttl 64 dev eth0 263 | /bin/ip addr flush gre1 264 | /bin/ip addr add 10.0.0.2/32 dev gre1 265 | /bin/ip link set gre1 up 266 | # Setup Bridge Interface 267 | # br_interface : bridge 268 | /bin/ip link add bridge type bridge vlan_filtering 1 269 | /bin/ip link set dev bridge up 270 | # Setup NFLOG 271 | /sbin/iptables -A FORWARD -i gre1 -o gre1 -m hashlimit --hashlimit-upto 4/minute --hashlimit-burst 1 --hashlimit-mode srcip,dstip --hashlimit-srcmask 16 --hashlimit-name loglimit-0 -j NFLOG --nflog-group 1 --nflog-size 128 272 | /sbin/iptables -P FORWARD ACCEPT 273 | # Setup TCP MSS Clamp 274 | /sbin/iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1340 275 | # Start Nante-WAN Docker Containers 276 | /usr/bin/docker run -dt --privileged --net=host -v /home/upa/work/nante-wan/nante-wan.conf:/etc/nante-wan.conf -v /dev/log:/dev/log upaa/nante-wan-routing 277 | /usr/bin/docker run -dt --privileged --net=host -v /home/upa/work/nante-wan/nante-wan.conf:/etc/nante-wan.conf -v /dev/log:/dev/log upaa/nante-wan-portconfig 278 | ``` 279 | 280 | And, you can verify EVPN and IPsec status like following. 281 | 282 | ```shell-session 283 | ce2:$ docker ps 284 | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 285 | b6b4fdd4e57e upaa/nante-wan-portconfig "/bin/sh -c 'bash ..." 2 seconds ago Up 1 second hardcore_bell 286 | 7a461646c69b upaa/nante-wan-routing "/bin/sh -c 'bash ..." 2 seconds ago Up 1 second compassionate_bohr 287 | $ docker exec -it 7a vtysh 288 | ce2# show bgp l2vpn evpn summary 289 | BGP router identifier 10.0.0.2, local AS number 65000 vrf-id 0 290 | BGP table version 0 291 | RIB entries 5, using 760 bytes of memory 292 | Peers 1, using 19 KiB of memory 293 | Peer groups 1, using 64 bytes of memory 294 | 295 | Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/P 296 | fxRcd 297 | 10.0.0.10 4 65000 8 6 0 0 0 00:01:18 298 | 2 299 | 300 | Total number of neighbors 1 301 | ce2:# exit 302 | $ docker exec -it 7a ipsec status 303 | Security Associations (1 up, 0 connecting): 304 | dmvpn[1]: ESTABLISHED 2 minutes ago, 192.168.0.2[192.168.0.2]...192.168.0.10[192.168.0.10] 305 | dmvpn{1}: INSTALLED, TRANSPORT, reqid 1, ESP SPIs: c86df829_i cf9c192e_o 306 | dmvpn{1}: 192.168.0.2/32[gre] === 192.168.0.10/32[gre] 307 | 308 | $ 309 | ``` 310 | 311 | 312 | 313 | ### 4. Put configuration files at config server 314 | 315 | After containers run on all nodes, DMVPN/IPsec overlay is established, 316 | and BGP EVPN starts VXLAN FDB exchange. Next step is distribuitng 317 | bridge configuration files to CE nodes. 318 | 319 | CE nodes try to fetch their configuration files from URL specified by 320 | **json_url_prefix** on [portconfig] section in nante-wan.conf. The URL 321 | is **[json_url_prefix]/[dmvpn_addr].json**. For example, CE1 accesses 322 | *http://10.0.0.10/portconfig/10.0.0.1.json*, and CE2 accesses 323 | *http://10.0.0.10/portconfig/10.0.0.2.json* (10.0.0.10 is dmvpn_addr 324 | of config server). 325 | 326 | The DocumentRoot of config server container is the directory specified 327 | by **--config-dir** option of start.py. So, *html* in this case (see 328 | step 3). 329 | 330 | 331 | An example for bridge configuraiton file for this environment is shown 332 | below. As you can see, this file indicates that the port 'vetha' is 333 | untagged port and it belongs to vlan 99. If an CE has multiple ports 334 | or you want to configure a port as tagged, please modify the json as 335 | you might have guessed. 336 | 337 | ```json 338 | { 339 | "name" : "bridge", 340 | "ports" : [ 341 | { 342 | "name" : "vetha", 343 | "tagged" : false, 344 | "vlans" : [ 99 ] 345 | } 346 | ] 347 | } 348 | ``` 349 | 350 | After place bridge configuration files in *html/portconfig* directory, 351 | bridge interfaces on all CE nodes are configured automatically. 352 | 353 | At config server, 354 | ```shell-session 355 | server:$ cat << EOF > example.json 356 | heredoc% { "name" : "bridge", "ports" : [ { "name": "vetha", "tagged": false, "vlans": [ 99 ] } ] } 357 | heredoc% EOF 358 | server:$ cp example.json html/portconfig/10.0.0.1.json 359 | server:$ cp example.json html/portconfig/10.0.0.2.json 360 | server:$ cp example.json html/portconfig/10.0.0.3.json 361 | ``` 362 | 363 | 364 | At CE nodes, verify that vlan 99 is created and vetha is configured as 365 | tagged vlan 99. 366 | 367 | ```shell-session 368 | ce1:$ bridge vlan show 369 | port vlan ids 370 | docker0 1 PVID Egress Untagged 371 | 372 | vetha 1 Egress Untagged 373 | 99 PVID Egress Untagged 374 | 375 | bridge 1 PVID Egress Untagged 376 | 99 377 | 378 | vxlan99 1 Egress Untagged 379 | 99 PVID Egress Untagged 380 | ``` 381 | 382 | 383 | Then, you can ping from any edge network namespaces from others across 384 | VXLAN over DMVPN overlay. 385 | 386 | ```shell-session 387 | ce1:$ sudo ip netns exec edge-network bash 388 | ce1:# ifconfig 389 | lo: flags=73 mtu 65536 390 | inet 127.0.0.1 netmask 255.0.0.0 391 | inet6 ::1 prefixlen 128 scopeid 0x10 392 | loop txqueuelen 1000 (Local Loopback) 393 | RX packets 0 bytes 0 (0.0 B) 394 | RX errors 0 dropped 0 overruns 0 frame 0 395 | TX packets 0 bytes 0 (0.0 B) 396 | TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 397 | 398 | vethb: flags=4163 mtu 1500 399 | inet 172.16.0.1 netmask 255.255.255.0 broadcast 0.0.0.0 400 | inet6 fe80::845c:d0ff:fe82:2cc6 prefixlen 64 scopeid 0x20 401 | ether 86:5c:d0:82:2c:c6 txqueuelen 1000 (Ethernet) 402 | RX packets 859 bytes 55960 (55.9 KB) 403 | RX errors 0 dropped 0 overruns 0 frame 0 404 | TX packets 1295 bytes 115470 (115.4 KB) 405 | TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 406 | 407 | ce1:# ping 172.16.0.2 408 | PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data. 409 | 64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=1.39 ms 410 | ^C 411 | --- 172.16.0.2 ping statistics --- 412 | 1 packets transmitted, 1 received, 0% packet loss, time 0ms 413 | rtt min/avg/max/mdev = 1.390/1.390/1.390/0.000 ms 414 | ce1# ping 172.16.0.3 415 | PING 172.16.0.3 (172.16.0.3) 56(84) bytes of data. 416 | ^C 417 | --- 172.16.0.3 ping statistics --- 418 | 2 packets transmitted, 0 received, 100% packet loss, time 1005ms 419 | 420 | ce1:# ping 172.16.0.3 421 | PING 172.16.0.3 (172.16.0.3) 56(84) bytes of data. 422 | 64 bytes from 172.16.0.3: icmp_seq=1 ttl=64 time=2.03 ms 423 | 64 bytes from 172.16.0.3: icmp_seq=2 ttl=64 time=0.875 ms 424 | ^C 425 | --- 172.16.0.3 ping statistics --- 426 | 2 packets transmitted, 2 received, 0% packet loss, time 1001ms 427 | rtt min/avg/max/mdev = 0.875/1.456/2.038/0.582 ms 428 | ce1# 429 | ``` 430 | 431 | 432 | 433 | 434 | ### 5. Next Step 435 | 436 | #### Adding new CE node 437 | 438 | It is very easy. copy the nante-wan.conf to new node, change 439 | **dmvpn_addr**, and `start.py nante-wan.conf`. 440 | 441 | 442 | #### Changing bridge configuration of CE nodes 443 | 444 | Edit portconfig/[CE's dmvpn_addr].json in the config directory in 445 | your config server. 446 | 447 | 448 | #### Web interface 449 | 450 | Nante-WAN does not have any web interfaces. But, it is easy to make 451 | your own web interface. What your web interface must do is, putting 452 | and updating config json files. 453 | 454 | 455 | #### Firewall 456 | 457 | The ebconfig container provides L4 ACL and MAC address filtering 458 | functions. 459 | 460 | #### Redundancy 461 | 462 | Multiple RRs and NHSes are supported. use `nhs_nbma_addrs\d+` and 463 | `rr_addrs\d+` in your nante-wan.conf 464 | 465 | ## Contact 466 | 467 | upa at haeena.net 468 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | # docker-compose-yml for nante-wan edge devices 2 | # 3 | version: '2' 4 | services: 5 | routing: 6 | container_name: routing 7 | image: upaa/nante-wan-routing 8 | volumes: 9 | - /etc/nante-wan.conf:/etc/nante-wan.conf 10 | - /dev/log:/dev/log 11 | privileged: true 12 | network_mode: "host" 13 | tty: true 14 | 15 | portconfig: 16 | container_name: portconfig 17 | image: upaa/nante-wan-portconfig 18 | volumes: 19 | - /etc/nante-wan.conf:/etc/nante-wan.conf 20 | - /dev/log:/dev/log 21 | privileged: true 22 | network_mode: "host" 23 | 24 | -------------------------------------------------------------------------------- /docker/config-server/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:17.10 2 | 3 | ARG workdir="/root" 4 | 5 | RUN apt-get update && apt-get install -y \ 6 | nginx \ 7 | python3-jinja2 \ 8 | python3-pyinotify \ 9 | python3-requests \ 10 | && rm -rf /etc/nginx/sites-enabled/default 11 | 12 | # add Config Render 13 | ADD templates ${workdir}/templates 14 | ADD config-render.py ${workdir}/config-render.py 15 | 16 | # add kick-update 17 | ADD kick-update.py ${workdir}/kick-update.py 18 | 19 | CMD bash -c "/root/config-render.py /etc/nante-wan.conf && nginx && /root/kick-update.py -c /etc/nante-wan.conf -d /var/www/html" 20 | -------------------------------------------------------------------------------- /docker/config-server/config-render.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import sys 5 | import socket 6 | import fcntl 7 | import struct 8 | import configparser 9 | from optparse import OptionParser 10 | from jinja2 import Environment, FileSystemLoader 11 | 12 | 13 | 14 | def config_render_nginx(config, of) : 15 | 16 | d = { "dmvpn_addr" : config.get("general", "dmvpn_addr")} 17 | 18 | tmp = os.path.join(os.path.dirname(__file__), "templates") 19 | env = Environment(loader = FileSystemLoader(tmp, encoding = "utf-8")) 20 | tpl = env.get_template("default") 21 | 22 | nginx_conf = tpl.render(d) 23 | print(nginx_conf, file = of) 24 | 25 | 26 | if __name__ == '__main__' : 27 | 28 | desc = "usage: %prog [options] configfile" 29 | parser = OptionParser(desc) 30 | parser.add_option('-s', '--stdout', action = "store_true", 31 | default = None, dest = "stdout", 32 | help = "output to stdout") 33 | options, args = parser.parse_args() 34 | try : 35 | ini_file = args.pop() 36 | except : 37 | print("config file is not specified") 38 | sys.exit() 39 | 40 | config = configparser.ConfigParser() 41 | config.read(ini_file) 42 | 43 | if options.stdout : 44 | of_nginx = sys.stdout 45 | else : 46 | of_nginx = open("/etc/nginx/sites-enabled/default", "w") 47 | 48 | 49 | config_render_nginx(config, of_nginx) 50 | -------------------------------------------------------------------------------- /docker/config-server/kick-update.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import re 4 | import sys 5 | import time 6 | import pyinotify 7 | import configparser 8 | 9 | from optparse import OptionParser 10 | 11 | import requests 12 | from requests.packages.urllib3.exceptions import InsecureRequestWarning 13 | requests.packages.urllib3.disable_warnings(InsecureRequestWarning) 14 | 15 | from logging import getLogger, DEBUG, StreamHandler, Formatter 16 | from logging.handlers import SysLogHandler 17 | logger = getLogger(__name__) 18 | logger.setLevel(DEBUG) 19 | stream = StreamHandler() 20 | syslog = SysLogHandler(address = "/dev/log") 21 | syslog.setFormatter(Formatter("kick-update: %(message)s")) 22 | logger.addHandler(stream) 23 | logger.addHandler(syslog) 24 | logger.propagate = False 25 | 26 | 27 | 28 | class NotifyHandler(pyinotify.ProcessEvent): 29 | 30 | def __init__(self, configfile, dryrun) : 31 | 32 | self.dryrun = dryrun 33 | 34 | config = configparser.ConfigParser() 35 | config.read(configfile) 36 | 37 | pt_prefix = config.get("portconfig", "json_url_prefix") 38 | self.portconfig_path = "/".join(pt_prefix.split("/")[3:]) 39 | self.portconfig_port = config.getint("portconfig", "bind_port") 40 | 41 | eb_prefix = config.get("ebconfig", "json_url_prefix") 42 | self.ebconfig_path = "/".join(eb_prefix.split("/")[3:]) 43 | self.ebconfig_port = config.getint("ebconfig", "bind_port") 44 | 45 | logger.info("sub-directory for portconfig: %s" % self.portconfig_path) 46 | logger.info("sub-directory for ebconfig: %s" % self.ebconfig_path) 47 | logger.info("Port number of portconfig REST: %d" % self.portconfig_port) 48 | logger.info("Port number of ebconfig REST: %d" % self.ebconfig_port) 49 | 50 | 51 | def kick_update(self, name, port) : 52 | 53 | # name is X.X.X.X.json 54 | ipaddr = name[0:len(name) - 5] 55 | url = "http://%s:%d/update-kick" % (ipaddr, port) 56 | 57 | if self.dryrun : 58 | logger.info("dryrun: kick url is %s" % url) 59 | return 60 | 61 | for x in range (5) : 62 | try : 63 | res = requests.get(url, timeout = 5) 64 | if res.status_code == 200 : 65 | logger.error("kick %s success" % url) 66 | break 67 | else : 68 | logger.error("Failed to kick %s, %d" % 69 | (url, res.status_code)) 70 | 71 | except Exception as e : 72 | logger.error("Failed to kick %s :%s:%s" % 73 | (url, e.__class__.__name__, e)) 74 | 75 | 76 | 77 | def process_IN_CLOSE_WRITE(self, event) : 78 | 79 | name = event.name 80 | path = event.path 81 | 82 | if re.match(r'^(\d{1,3}\.){3,3}\d{1,3}\.json$', name) : 83 | 84 | if self.portconfig_path in path : 85 | port = self.portconfig_port 86 | elif self.ebconfig_path in path : 87 | port = self.ebconfig_port 88 | else : 89 | return 90 | 91 | self.kick_update(name, port) 92 | 93 | 94 | 95 | if __name__ == "__main__" : 96 | 97 | desc = "usage : %prog [options]" 98 | parser = OptionParser(desc) 99 | 100 | parser.add_option( 101 | "-c", "--config", type = "string", default = None, dest = "configfile", 102 | help = "nante-wan config file" 103 | ) 104 | parser.add_option( 105 | "-d", "--directory", type = "string", default = None, dest = "targetdir", 106 | help = "watch target directry (DocumentRoot of config serve)" 107 | ) 108 | parser.add_option( 109 | "-n", "--no-execute", action = "store_true", default = False, 110 | dest = "dryrun", help = "no kick the RERST GWs (dryrun)" 111 | ) 112 | 113 | (options, args) = parser.parse_args() 114 | 115 | if not options.configfile : 116 | logger.error("config file (-c option) is required") 117 | sys.exit(1) 118 | 119 | if not options.targetdir : 120 | logger.rror("target directory (-d option) is required") 121 | sys.exit(1) 122 | 123 | 124 | wm = pyinotify.WatchManager() 125 | nh = NotifyHandler(options.configfile, options.dryrun) 126 | notifier = pyinotify.Notifier(wm, nh) 127 | added = wm.add_watch(options.targetdir, pyinotify.IN_CLOSE_WRITE, rec = True) 128 | 129 | notifier.loop() 130 | 131 | 132 | -------------------------------------------------------------------------------- /docker/config-server/templates/default: -------------------------------------------------------------------------------- 1 | server { 2 | listen {{ dmvpn_addr }}:80 default_server; 3 | 4 | root /var/www/html; 5 | index index.html index.htm; 6 | 7 | server_name af-graft-nginx; 8 | index index.html index.htm index.nginx-debian.html; 9 | 10 | location / { 11 | # First attempt to serve request as file, then 12 | # as directory, then fall back to displaying a 404. 13 | try_files $uri $uri/ =404; 14 | # Uncomment to enable naxsi on this location 15 | # include /etc/nginx/naxsi.rules 16 | } 17 | } 18 | -------------------------------------------------------------------------------- /docker/ebconfig/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:17.10 2 | 3 | ARG workdir="/root" 4 | 5 | RUN apt-get update && apt-get install -y \ 6 | iproute2 python3 python3-requests python3-flask ebtables 7 | 8 | # setup ebconfig 9 | ADD ebconfig.py ${workdir}/ebconfig.py 10 | 11 | CMD bash -c "/root/ebconfig.py /etc/nante-wan.conf" 12 | -------------------------------------------------------------------------------- /docker/ebconfig/ebconfig.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import sys 5 | import json 6 | import asyncio 7 | import threading 8 | import subprocess 9 | import configparser 10 | 11 | 12 | from logging import getLogger, DEBUG, StreamHandler, Formatter 13 | from logging.handlers import SysLogHandler 14 | 15 | import requests 16 | from requests.packages.urllib3.exceptions import InsecureRequestWarning 17 | requests.packages.urllib3.disable_warnings(InsecureRequestWarning) 18 | 19 | logger = getLogger(__name__) 20 | logger.setLevel(DEBUG) 21 | stream = StreamHandler() 22 | syslog = SysLogHandler(address = "/dev/log") 23 | syslog.setFormatter(Formatter("ebconfig: %(message)s")) 24 | logger.addHandler(stream) 25 | logger.addHandler(syslog) 26 | logger.propagate = False 27 | 28 | ebcmd = "/sbin/ebtables" 29 | 30 | 31 | 32 | from flask import Flask 33 | app = Flask(__name__) 34 | 35 | 36 | # EbConfig instance 37 | 38 | def run_cmds(cmds): 39 | for cmd in cmds : 40 | logger.debug(" ".join(list(map(str, cmd)))) 41 | subprocess.check_output(list(map(str, cmd))) 42 | 43 | 44 | class EbConfig() : 45 | 46 | def __init__ (self, configfile) : 47 | 48 | config = configparser.ConfigParser() 49 | config.read(configfile) 50 | 51 | self.dmvpn_addr = config.get("general", "dmvpn_addr") 52 | 53 | self.timeout = config.getint("config_fetch", "timeout") 54 | self.interval = config.getint("config_fetch", "interval") 55 | self.failed_interval = config.getint("config_fetch", "failed_interval") 56 | 57 | self.url = "%s/%s.json" % (config.get("ebconfig", "json_url_prefix"), 58 | config.get("general", "dmvpn_addr")) 59 | self.bind_port = config.getint("ebconfig", "bind_port") 60 | if config.has_option("ebconfig", "bind_addr") : 61 | self.bind_addr = config.get("ebconfig", "bind_addr") 62 | else : 63 | self.bind_addr = config.get("general", "dmvpn_addr") 64 | 65 | if config.has_option("ebconfig", "json_config") : 66 | self.json_config = config.get("ebconfig", "json_config") 67 | else : 68 | self.json_config = None 69 | 70 | 71 | def fetch(self) : 72 | # fetch the config json from confg server 73 | 74 | if self.json_config : 75 | with open(self.json_config, "r") as f : 76 | return json.load(f) 77 | 78 | try : 79 | res = requests.get(self.url, timeout = self.timeout) 80 | if res.status_code != 200 : 81 | logger.error("Failed to GET '%s', status_code %d" % 82 | (self.url, res.status_code)) 83 | return False 84 | except requests.exceptions.RequestException as e : 85 | logger.error("Failed to GET '%s': %s" % (self.url, e)) 86 | return False 87 | 88 | except Exception as e : 89 | logger.error("Failed to GET '%s': %s" % (self.url, e)) 90 | return False 91 | 92 | try : 93 | return res.json() 94 | except Exception as e : 95 | logger.error("Invalid JSON from '%s': %s" % (self.url, e)) 96 | return False 97 | 98 | 99 | 100 | def execute(self) : 101 | 102 | jsonconfig = self.fetch() 103 | if not jsonconfig : 104 | return False 105 | 106 | self.flush_ebtables() 107 | for netconfig in jsonconfig : 108 | self.insert_ip_filter(netconfig) 109 | self.insert_mac_filter(netconfig) 110 | 111 | return True 112 | 113 | 114 | 115 | def flush_ebtables(self) : 116 | 117 | cmds = [ 118 | [ ebcmd, "-X" ], 119 | [ ebcmd, "-F" ] 120 | ] 121 | 122 | run_cmds(cmds) 123 | 124 | 125 | def insert_ip_filter(self, netconfig) : 126 | 127 | if (not "vlan" in netconfig or 128 | not "ip-filter" in netconfig or 129 | not "rules" in netconfig["ip-filter"]) : 130 | return 131 | 132 | vlan = netconfig["vlan"] 133 | default = netconfig["ip-filter"]["default"] 134 | rules = netconfig["ip-filter"]["rules"] 135 | interface = "vxlan%d" % vlan 136 | 137 | if default == "permit" : default = "ACCEPT" 138 | elif default == "deny" : default = "DROP" 139 | else : 140 | raise RuntimeError("invalid default action '%s'" % default) 141 | 142 | cmds = [] 143 | 144 | for rule in rules : 145 | 146 | proto = rule["proto"] 147 | src_ip = rule["src-ip"] if "src-ip" in rule else "0.0.0.0/0" 148 | dst_ip = rule["dst-ip"] if "dst-ip" in rule else "0.0.0.0/0" 149 | src_port = rule["src-port"] if "src-port" in rule else "any" 150 | dst_port = rule["dst-port"] if "dst-port" in rule else "any" 151 | 152 | action = rule["action"] if "action" in rule else "deny" 153 | if action == "permit" : action = "ACCEPT" 154 | elif action == "deny" : action = "DROP" 155 | else : 156 | raise RuntimeError("invalid action '%s'" % action) 157 | 158 | 159 | cmd = [ ebcmd, "-A", "FORWARD", "-o", interface, 160 | "-p", "IPv4", "--ip-src", src_ip, "--ip-dst", dst_ip, 161 | "--ip-proto", proto 162 | ] 163 | 164 | if src_port != "any" : 165 | cmd += [ "--ip-sport", src_port ] 166 | if dst_port != "any" : 167 | cmd += [ "--ip-dport", dst_port ] 168 | 169 | cmd += [ "-j", action ] 170 | 171 | cmds.append(cmd) 172 | 173 | cmds.append([ 174 | ebcmd, "-A", "FORWARD", "-o", interface, 175 | "-p", "IPv4", "-j", default 176 | ]) 177 | 178 | run_cmds(cmds) 179 | 180 | 181 | def insert_mac_filter(self, netconfig) : 182 | 183 | if (not "vlan" in netconfig or 184 | not "mac-filter" in netconfig or 185 | not "rules" in netconfig["mac-filter"]) : 186 | return 187 | 188 | vlan = netconfig["vlan"] 189 | default = netconfig["mac-filter"]["default"] 190 | rules = netconfig["mac-filter"]["rules"] 191 | interface = "vxlan%d" % vlan 192 | 193 | if default == "permit" : default = "ACCEPT" 194 | elif default == "deny" : default = "DROP" 195 | else : 196 | raise RuntimeError("invalid default action '%s'" % default) 197 | 198 | cmds = [] 199 | 200 | for rule in rules : 201 | 202 | mac = rule["mac"] 203 | action = rule["action"] if "action" in rule else "deny" 204 | if action == "permit" : action = "ACCEPT" 205 | elif action == "deny" : action = "DROP" 206 | else : 207 | raise RuntimeError("invalid action '%s'" % action) 208 | 209 | cmd = [ ebcmd, "-A", "FORWARD", "-o", interface, 210 | "--src", mac, "-j", action 211 | ] 212 | 213 | cmds.append(cmd) 214 | 215 | cmds.append([ 216 | ebcmd, "-A", "FORWARD", "-o", interface, "-j", default 217 | ]) 218 | 219 | run_cmds(cmds) 220 | 221 | 222 | 223 | def fetch_loop(self, loop, once) : 224 | 225 | try : 226 | ret = self.execute() 227 | if ret : 228 | if once : 229 | loop.stop() 230 | return True 231 | loop.call_later(self.interval, self.fetch_loop, loop, once) 232 | else : 233 | loop.call_later(self.failed_interval, 234 | self.fetch_loop, loop, once) 235 | 236 | except KeyboardInterrupt: 237 | logger.info("Keyboard interrupt. stop ebconfig loop.") 238 | loop.stop() 239 | 240 | except Exception as e: 241 | logger.error("ebconfig error occurd:%s: %s" % 242 | (e.__class__.__name__, e)) 243 | loop.call_later(self.failed_interval, self.fetch_loop, loop, once) 244 | 245 | 246 | def start_loop(self) : 247 | loop = asyncio.get_event_loop() 248 | loop.call_soon(self.fetch_loop, loop, False) 249 | loop.run_forever() 250 | loop.close() 251 | 252 | 253 | def execute_once(self) : 254 | loop = asyncio.new_event_loop() 255 | loop.call_soon(self.fetch_loop, loop, True) 256 | loop.run_forever() 257 | loop.close() 258 | 259 | 260 | @app.route("/update-kick", methods = [ "GET", "POST" ]) 261 | def ebconfig_rest_update_from_url() : 262 | logger.info("Update trigger. start to obtain JSON") 263 | 264 | global ebconfig 265 | ebconfig.execute_once() 266 | return "Start Update", 200 267 | 268 | 269 | def ebconfig_rest_start(bind_addr, bind_port) : 270 | logger.info("Start REST Gateway on %s:%d" % (bind_addr, 271 | bind_port)) 272 | th = threading.Thread(name = "rest_gw", target = app.run, 273 | kwargs = { 274 | "threaded" : True, 275 | "host" : bind_addr, 276 | "port" : bind_port, 277 | }) 278 | th.start() 279 | 280 | 281 | 282 | if __name__ == "__main__" : 283 | ebconfig = EbConfig(sys.argv[1]) 284 | ebconfig_rest_start(ebconfig.bind_addr, ebconfig.bind_port) 285 | ebconfig.start_loop() 286 | -------------------------------------------------------------------------------- /docker/ebconfig/example.json: -------------------------------------------------------------------------------- 1 | 2 | [ 3 | { 4 | "vlan" : 10, 5 | "ip-filter" : { 6 | "default" : "permit", 7 | "rules" : [ 8 | { 9 | "proto" : "tcp", 10 | "src-ip" : "0.0.0.0/0", "src-port" : "any", 11 | "dst-ip" : "10.0.0.1/32", "dst-port" : "any", 12 | "action" : "deny" 13 | }, 14 | { 15 | "proto" : "tcp", 16 | "src-ip" : "1.1.1.1", "src-port" : "any", 17 | "dst-ip" : "2.2.2.2", "dst-port" : "any", 18 | "action" : "deny" 19 | }, 20 | { 21 | "proto" : "udp", 22 | "src-ip" : "0.0.0.0/30", "src-port" : "any", 23 | "dst-ip" : "192.168.0.0/16", "dst-port" : 80, 24 | "action" : "permit" 25 | } 26 | ] 27 | }, 28 | "mac-filter" : { 29 | "default" : "deny", 30 | "rules" : [ 31 | { "mac" : "0a:00:27:00:00:01", "action" : "permit" }, 32 | { "mac" : "0a:00:27:00:00:02", "action" : "permit" }, 33 | { "mac" : "0a:00:27:00:00:03", "action" : "permit" } 34 | ] 35 | } 36 | }, 37 | { 38 | } 39 | ] 40 | -------------------------------------------------------------------------------- /docker/netstack-setup/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:17.04 2 | 3 | ARG workdir="/root" 4 | 5 | RUN apt-get update && apt-get install -y \ 6 | python3 iproute2 iptables kmod 7 | 8 | ADD start.py ${workdir}/start.py 9 | 10 | CMD [ "python3", "/root/start.py", "--network-only", "/etc/nante-wan.conf" ] 11 | -------------------------------------------------------------------------------- /docker/netstack-setup/start.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import sys 5 | import subprocess 6 | import configparser 7 | 8 | from optparse import OptionParser 9 | 10 | from logging import getLogger, DEBUG, StreamHandler, Formatter 11 | from logging.handlers import SysLogHandler 12 | logger = getLogger(__name__) 13 | logger.setLevel(DEBUG) 14 | stream = StreamHandler() 15 | syslog = SysLogHandler(address = "/dev/log") 16 | syslog.setFormatter(Formatter("nanate-wan: %(message)s")) 17 | logger.addHandler(stream) 18 | logger.addHandler(syslog) 19 | logger.propagate = False 20 | 21 | 22 | ipcmd = "/bin/ip" 23 | iwcmd = "/sbin/iw" 24 | iptables = "/sbin/iptables" 25 | docker = "/usr/bin/docker" 26 | 27 | def run_cmds(cmds): 28 | 29 | for cmd in cmds : 30 | logger.info(" ".join(list(map(str, cmd)))) 31 | subprocess.check_output(list(map(str, cmd))) 32 | 33 | 34 | def setup_gre(config) : 35 | 36 | wan_interface = config.get("routing", "wan_interface") 37 | dmvpn_interface = config.get("routing", "dmvpn_interface") 38 | dmvpn_addr = config.get("general", "dmvpn_addr") 39 | gre_key = config.get("routing", "gre_key") 40 | gre_ttl = config.get("routing", "gre_ttl") 41 | 42 | logger.info("# Setup GRE Interface") 43 | logger.info("# wan_interface : %s" % wan_interface) 44 | logger.info("# dmvpn_interface : %s" % dmvpn_interface) 45 | logger.info("# dmvpn_addr : %s" % dmvpn_addr) 46 | 47 | cmds = [ 48 | [ "modprobe", "af_key" ], 49 | [ ipcmd, "tunnel", "add", dmvpn_interface , "mode", "gre", 50 | "key", gre_key, "ttl", gre_ttl, "dev", wan_interface ], 51 | [ ipcmd, "addr", "flush", dmvpn_interface ], 52 | [ ipcmd, "addr", "flush", dmvpn_interface ], 53 | [ ipcmd, "addr", "add", "%s/32" % dmvpn_addr, "dev", dmvpn_interface ], 54 | [ ipcmd, "link", "set", dmvpn_interface, "up" ], 55 | ] 56 | # XXX: ip addr flush gre1 sometimes does not flush addr, so, try twice 57 | 58 | if os.path.exists("/sys/class/net/%s" % dmvpn_interface) : 59 | logger.error("# '%s' exists. delete and recreate." % dmvpn_interface) 60 | cmds.insert(0, [ ipcmd, "tunnel", "del", dmvpn_interface]) 61 | 62 | 63 | run_cmds(cmds) 64 | 65 | def setup_bridge(config) : 66 | 67 | br_interface = config.get("portconfig", "br_interface") 68 | 69 | logger.info("# Setup Bridge Interface") 70 | logger.info("# br_interface : %s" % br_interface) 71 | 72 | cmds = [ 73 | [ ipcmd, "link", "add", br_interface, "type", "bridge", 74 | "vlan_filtering", 1 ], 75 | [ ipcmd, "link", "set", "dev", br_interface, "up" ] 76 | ] 77 | 78 | if os.path.exists("/sys/class/net/%s" % br_interface) : 79 | logger.error("# '%s' exists. delete and recreate." % br_interface) 80 | cmds.insert(0, [ ipcmd, "link", "del", "dev", br_interface ]) 81 | 82 | run_cmds(cmds) 83 | 84 | 85 | def iptables_ver() : 86 | return subprocess.getoutput(["%s -V" % iptables]).strip().split(" ")[1] 87 | 88 | def setup_nflog(config) : 89 | 90 | logger.info("# Setup NFLOG") 91 | 92 | dmvpn_interface = config.get("routing", "dmvpn_interface") 93 | 94 | cmds = [ 95 | [ 96 | iptables, "-A", "FORWARD", 97 | "-i", dmvpn_interface, "-o", dmvpn_interface, 98 | "-m", "hashlimit", 99 | "--hashlimit-upto", "4/minute", 100 | "--hashlimit-burst", 1, 101 | "--hashlimit-mode", "srcip,dstip", 102 | "--hashlimit-srcmask", 16, 103 | "--hashlimit-name", "loglimit-0", 104 | "-j", "NFLOG", "--nflog-group", 1, 105 | "--nflog-size" if iptables_ver() > "v1.6.0" else "--nflog-range", 106 | 128 107 | ], 108 | [ 109 | iptables, "-P", "FORWARD", "ACCEPT" 110 | ] 111 | ] 112 | 113 | nflog = "iptables -nL --line-numbers | grep NFLOG" 114 | for line in subprocess.getoutput([nflog]).split("\n") : 115 | if not line : continue 116 | logger.error("# NFLOG rule '%s' exists. delete it." % line) 117 | rulenum = line.split()[0] 118 | cmds.insert(0, [ iptables, "-D", "FORWARD", rulenum]) 119 | 120 | run_cmds(cmds) 121 | 122 | def setup_mss_clamp(config) : 123 | 124 | logger.info("# Setup TCP MSS Clamp") 125 | 126 | cmds = [ 127 | [ 128 | iptables, "-A", "FORWARD", "-p", "tcp", 129 | "--tcp-flags", "SYN,RST", "SYN", 130 | "-j", "TCPMSS", "--set-mss", 1340 131 | ], 132 | ] 133 | 134 | nflog = "iptables -nL --line-numbers | grep TCPMSS" 135 | for line in subprocess.getoutput([nflog]).split("\n") : 136 | if not line : continue 137 | logger.error("# TCPMSS rule '%s' exists. delete it." % line) 138 | rulenum = line.split()[0] 139 | cmds.insert(0, [ iptables, "-D", "FORWARD", rulenum]) 140 | 141 | run_cmds(cmds) 142 | 143 | 144 | def run_containers(option, config, configpath) : 145 | 146 | logger.info("# Start Nante-WAN Docker Containers") 147 | 148 | cmds = [] 149 | 150 | if option.route_server : 151 | # run route server container 152 | cmds += [ 153 | [ docker, "run", "-dt", "--privileged", "--net=host", 154 | "-v", "%s:/etc/nante-wan.conf" % configpath, 155 | "-v", "/dev/log:/dev/log", 156 | "upaa/nante-wan-route-server" 157 | ] 158 | ] 159 | 160 | 161 | else : 162 | # run as edge device 163 | cmds += [ 164 | [ docker, "run", "-dt", "--privileged", "--net=host", 165 | "-v", "%s:/etc/nante-wan.conf" % configpath, 166 | "-v", "/dev/log:/dev/log", 167 | "upaa/nante-wan-routing" 168 | ], 169 | [ docker, "run", "-dt", "--privileged", "--net=host", 170 | "-v", "%s:/etc/nante-wan.conf" % configpath, 171 | "-v", "/dev/log:/dev/log", 172 | "upaa/nante-wan-portconfig" 173 | ], 174 | ] 175 | 176 | if option.config_server : 177 | # run config server container 178 | cmds += [ 179 | [ docker, "run", "-dt", "--net=host", 180 | "-v", "%s:/etc/nante-wan.conf" % configpath, 181 | "-v", "/dev/log:/dev/log", 182 | "-v", "%s:/var/www/html" % os.path.abspath(option.config_dir), 183 | "upaa/nante-wan-config-server" 184 | ] 185 | ] 186 | 187 | 188 | if option.enable_ebconfig : 189 | # run ebconfig container 190 | cmds += [ 191 | [ docker, "run", "-dt", "--privileged", "--net=host", 192 | "-v", "%s:/etc/nante-wan.conf" % configpath, 193 | "-v", "/dev/log:/dev/log", 194 | "upaa/nante-wan-ebconfig" 195 | ] 196 | ] 197 | 198 | dockerps = "docker ps | grep upaa/nante-wan" 199 | for line in subprocess.getoutput([dockerps]).split("\n") : 200 | if not line : continue 201 | c_id = line.split()[0] 202 | c_name = line.split()[1] 203 | logger.error("%s is working as %s. delete and re-run" % (c_name, c_id)) 204 | cmds.insert(0, [ docker, "rm", "-f", c_id ]) 205 | 206 | run_cmds(cmds) 207 | 208 | 209 | if __name__ == "__main__" : 210 | 211 | desc = "usage: %prog [options] nante-wan.conf" 212 | parser = OptionParser(desc) 213 | 214 | parser.add_option( 215 | "--route-server", action = "store_true", default = False, 216 | dest = "route_server", 217 | help = "Run as a route server (BGP RR and NHRP NHS)" 218 | ) 219 | parser.add_option( 220 | "--config-server", action = "store_true", default = False, 221 | dest = "config_server", 222 | help = "Run as a config server (HTTP server)" 223 | ) 224 | parser.add_option( 225 | "--config-dir", type = "string", default = False, 226 | dest = "config_dir", 227 | help = "config directory for config server (DocumentRoot)" 228 | ) 229 | parser.add_option( 230 | "--container-only", action = "store_true", default = False, 231 | dest = "container_only", 232 | help = "(stop and) start containers (not config network stack)" 233 | ) 234 | parser.add_option( 235 | "--network-only", action = "store_true", default = False, 236 | dest = "network_only", 237 | help = "reset network configuration (not start containers)" 238 | ) 239 | parser.add_option( 240 | "--enable-ebconfig", action = "store_true", default = False, 241 | dest = "enable_ebconfig", 242 | help = "run ebconfig container for firewalling" 243 | ) 244 | 245 | (option, args) = parser.parse_args() 246 | 247 | if not args : 248 | print("Usage: %s [Nante-WAN Config]" % sys.argv[0]) 249 | sys.exit(1) 250 | 251 | if option.config_server and not option.config_dir : 252 | print("--config-dir is required to run as config-server") 253 | sys.exit(1) 254 | 255 | config = configparser.ConfigParser() 256 | config.read(args[0]) 257 | 258 | if not option.container_only : 259 | setup_gre(config) 260 | setup_bridge(config) 261 | setup_nflog(config) 262 | setup_mss_clamp(config) 263 | 264 | if not option.network_only : 265 | run_containers(option, config, os.path.abspath(args[0])) 266 | -------------------------------------------------------------------------------- /docker/portconfig/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:17.10 2 | 3 | ARG workdir="/root" 4 | 5 | # install required packages 6 | RUN apt-get update && apt-get install -y \ 7 | iproute2 python3 python3-requests python3-flask 8 | 9 | 10 | # setup sdwconfig 11 | ADD portconfig.py ${workdir}/portconfig.py 12 | 13 | CMD bash -c "/root/portconfig.py /etc/nante-wan.conf" 14 | -------------------------------------------------------------------------------- /docker/portconfig/example.json: -------------------------------------------------------------------------------- 1 | { 2 | "name" : "bridge", 3 | "ports" : [ 4 | { 5 | "name" : "vetha", 6 | "tagged" : false, 7 | "vlans" : [ 10, 11 ] 8 | }, 9 | { 10 | "name" : "vethc", 11 | "tagged" : false, 12 | "vlans" : [ 10 ] 13 | } 14 | ] 15 | } 16 | -------------------------------------------------------------------------------- /docker/portconfig/portconfig.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | 4 | import os 5 | import sys 6 | import json 7 | import time 8 | import struct 9 | import asyncio 10 | import functools 11 | import threading 12 | import subprocess 13 | import configparser 14 | 15 | import requests 16 | from requests.packages.urllib3.exceptions import InsecureRequestWarning 17 | requests.packages.urllib3.disable_warnings(InsecureRequestWarning) 18 | 19 | from logging import getLogger, DEBUG, StreamHandler, Formatter 20 | from logging.handlers import SysLogHandler 21 | logger = getLogger(__name__) 22 | logger.setLevel(DEBUG) 23 | stream = StreamHandler() 24 | syslog = SysLogHandler(address = "/dev/log") 25 | syslog.setFormatter(Formatter("portconfig: %(message)s")) 26 | logger.addHandler(stream) 27 | logger.addHandler(syslog) 28 | logger.propagate = False 29 | 30 | ipcmd = "/bin/ip" 31 | brcmd = "/sbin/bridge" 32 | 33 | 34 | 35 | # PortConfig Instance 36 | global portconfig 37 | portconfig = None 38 | 39 | 40 | from flask import Flask 41 | app = Flask(__name__) 42 | 43 | class Port() : 44 | 45 | def __init__(self, name, master = None, vlans = None, tagged = False) : 46 | self.name = name 47 | self.master = master 48 | self.tagged = tagged 49 | if vlans : 50 | self.vlans = vlans 51 | else : 52 | self.vlans = [] 53 | 54 | if tagged is not True and tagged is not False : 55 | raise ValueError("invalid tagged value '%s'. true or false." 56 | % tagged) 57 | 58 | 59 | def __str__(self) : 60 | return "" % \ 61 | (self.name, self.master, self.tagged, self.vlans) 62 | 63 | def add_vlan(self, vlan) : 64 | self.vlans.append(vlan) 65 | 66 | def set_tagged(self, tagged) : 67 | self.tagged = tagged 68 | 69 | def diff_check(self, port) : 70 | # check parameters of 'self' and 'port' are same? 71 | if (self.name == port.name and 72 | self.master == port.master and 73 | set(self.vlans) == set(port.vlans) and 74 | self.tagged == port.tagged) : 75 | return False 76 | else: 77 | return True 78 | 79 | def destroy(self) : 80 | # destroy this port. physical ports cannot be removed, 81 | # so, this function actually remove only vxlan interface 82 | # and remove the interface from the associated bridge 83 | 84 | cmd = [ipcmd, "link", "set", "dev", self.name, "nomaster"] 85 | subprocess.check_output(cmd) 86 | 87 | 88 | def setup(self) : 89 | 90 | # ok, setup vlans 91 | 92 | cmds = [ 93 | [ipcmd, "link", "set", "dev", self.name, "master", self.master], 94 | [ipcmd, "link", "set", "up", "dev", self.name], 95 | ] 96 | 97 | for vlan in self.vlans : 98 | cmd = [ brcmd, "vlan", "add", "vid", vlan, "dev", self.name ] 99 | if not self.tagged : 100 | cmd += [ "untagged", "pvid" ] 101 | cmds.append(cmd) 102 | 103 | for cmd in cmds : 104 | subprocess.check_output(list(map(str, cmd))) 105 | 106 | class Bridge() : 107 | 108 | def __init__(self, name, dmvpn_addr) : 109 | self.name = name 110 | self.dmvpn_addr = dmvpn_addr 111 | self.ports = {} # key is name, value is class Port 112 | self.vlans = set() # vlan set 113 | 114 | def __str__(self) : 115 | ports = ' \n'.join(map(str, self.ports.values())) 116 | return "" % (self.name, ports) 117 | 118 | def add_port(self, port): 119 | if port.name in self.ports : 120 | raise RuntimeError("port '%s' already exists on bridge '%s'" 121 | % (port.name, self.name)) 122 | self.ports[port.name] = port 123 | 124 | def list_ports(self) : 125 | return self.ports.values() 126 | 127 | def find_port(self, port_name) : 128 | if port_name in self.ports : 129 | return self.ports[port_name] 130 | else : 131 | return None 132 | 133 | def update_vlan_set(self) : 134 | self.vlans = set() 135 | for port in self.ports.values() : 136 | for vlan in port.vlans : 137 | self.vlans.add(vlan) 138 | 139 | def validate_vlan_set(self) : 140 | # check this vlan has vxlan interface 141 | rem = [] 142 | for vlan in self.vlans : 143 | if not os.path.exists("/sys/class/net/vxlan%d" % vlan) : 144 | rem.append(vlan) 145 | for v in rem : 146 | self.vlans.remove(v) 147 | 148 | def add_vxlan(self, vlan) : 149 | # add new vxlan interface 150 | vxlan_name = "vxlan%d" % vlan 151 | 152 | cmds = [] 153 | 154 | # check does vxlan interface exist. if not, make it. 155 | if not os.path.exists("/sys/class/net/%s" % vxlan_name) : 156 | cmds.append( 157 | [ipcmd, "link", "add", vxlan_name, 158 | "type", "vxlan", "nolearning", "dstport", 4789, 159 | "id", vlan, "local", self.dmvpn_addr] 160 | ) 161 | 162 | cs = [ 163 | [ipcmd, "link", "set", "dev", vxlan_name, "master", self.name], 164 | [brcmd, "vlan", "add", "vid", vlan, "dev", self.name, "self"], 165 | [brcmd, "vlan", "add", "vid", vlan, "dev", vxlan_name, 166 | "untagged", "pvid"], 167 | [ipcmd, "link", "set", "up", "dev", vxlan_name] 168 | ] 169 | for c in cs : 170 | cmds.append(c) 171 | 172 | for cmd in cmds : 173 | subprocess.check_output(list(map(str, cmd))) 174 | 175 | def delete_vxlan(self, vlan) : 176 | # remove vxlan interface 177 | vxlan_name = "vxlan%d" % vlan 178 | 179 | if not os.path.exists("/sys/class/net/%s" % vxlan_name) : 180 | return 181 | 182 | cmds = [ 183 | [ipcmd, "link", "set", "down", "dev", vxlan_name], 184 | [ipcmd, "link", "del", "dev", vxlan_name], 185 | [brcmd, "vlan", "del", "vid", vlan, "dev", self.name, "self"] 186 | ] 187 | for cmd in cmds : 188 | subprocess.check_output(list(map(str, cmd))) 189 | 190 | def load_from_json(self, jsondata) : 191 | """ 192 | json format is 193 | { 194 | 'name' : 'bridge_name', 195 | 'ports' : [ 196 | { 'name' : 'port1_name', 'vlan' : VLANID, 'tagged' : true }, 197 | { 'name' : 'port1_name', 'vlan' : VLANID, 'tagged' : false }, 198 | { 'name' : 'port1_name', 'vlan' : VLANID 'tagged' : false }, 199 | ] 200 | } 201 | 202 | """ 203 | 204 | try : 205 | self.name = jsondata["name"] 206 | for jport in jsondata["ports"] : 207 | 208 | if not os.path.exists("/sys/class/net/%s" % jport["name"]) : 209 | logger.error("Port '%s' does not exist" % jport["name"]) 210 | continue 211 | 212 | self.add_port(Port(jport["name"], 213 | master = self.name, 214 | vlans = jport["vlans"], 215 | tagged = jport["tagged"])) 216 | 217 | # update vlan set of this bridge 218 | self.update_vlan_set() 219 | 220 | except Exception as e: 221 | logger.error("JSON load failed :%s: %s" % 222 | (e.__class__.__name__, e)) 223 | return False 224 | 225 | return True 226 | 227 | def load_from_os(self) : 228 | 229 | # check associated ports 230 | brout = subprocess.check_output([brcmd, "link", "show"]) 231 | brout = brout.decode("utf-8") 232 | for line in brout.split('\n') : 233 | if "vxlan" in line : continue 234 | s = line.split(' ') 235 | for n in range(len(s)) : 236 | if s[n] == "master" and s[n + 1] == self.name : 237 | port = Port(s[1], master = self.name) 238 | self.add_port(port) 239 | 240 | # check vlan id of associated ports 241 | brout = subprocess.check_output([brcmd, "-json", "vlan", "show"]) 242 | brout = brout.decode("utf-8") 243 | vlanshow = json.loads(brout) 244 | for port in self.list_ports() : 245 | if not port.name in vlanshow : continue 246 | 247 | port.set_tagged(False) 248 | 249 | vlans = vlanshow[port.name] 250 | for vlan in vlans : 251 | if vlan["vlan"] == 1 : continue 252 | port.add_vlan(vlan["vlan"]) 253 | 254 | if (not "flags" in vlan or 255 | not "Egress Untagged" in vlan["flags"]) : 256 | # tagged vlan object does not have flag Untagged 257 | port.set_tagged(True) 258 | 259 | 260 | # update vlan set of this bridge 261 | self.update_vlan_set() 262 | self.validate_vlan_set() 263 | 264 | 265 | class PortConfig() : 266 | 267 | def __init__(self, configfile) : 268 | 269 | config = configparser.ConfigParser() 270 | config.read(configfile) 271 | 272 | self.dmvpn_addr = config.get("general", "dmvpn_addr") 273 | 274 | self.timeout = config.getint("config_fetch", "timeout") 275 | self.interval = config.getint("config_fetch", "interval") 276 | self.failed_interval = config.getint("config_fetch", "failed_interval") 277 | 278 | self.br_name = config.get("portconfig", "br_interface") 279 | self.url = "%s/%s.json" % (config.get("portconfig", "json_url_prefix"), 280 | config.get("general", "dmvpn_addr")) 281 | self.bind_port = config.getint("portconfig", "bind_port") 282 | if config.has_option("portconfig", "bind_addr") : 283 | self.bind_addr = config.get("portconfig", "bind_addr") 284 | else : 285 | self.bind_addr = config.get("general", "dmvpn_addr") 286 | 287 | if config.has_option("portconfig", "json_config") : 288 | self.json_config = config.get("portconfig", "json_config") 289 | else : 290 | self.json_config = None 291 | 292 | 293 | def fetch(self) : 294 | # fetch the config json 295 | 296 | if self.json_config : 297 | with open(self.json_config, "r") as f : 298 | return json.load(f) 299 | 300 | try : 301 | res = requests.get(self.url, timeout = self.timeout) 302 | if res.status_code != 200 : 303 | logger.error("Failed to GET '%s', status_code %d" % 304 | (self.url, res.status_code)) 305 | return False 306 | except requests.exceptions.RequestException as e : 307 | logger.error("Failed to GET '%s': %s" % (self.url, e)) 308 | return False 309 | 310 | except Exception as e : 311 | logger.error("Failed to GET '%s': %s" % (self.url, e)) 312 | return False 313 | 314 | try : 315 | return res.json() 316 | except Exception as e : 317 | logger.error("Invalid JSON from '%s': %s" % (self.url, e)) 318 | return False 319 | 320 | 321 | 322 | def execute(self) : 323 | 324 | jsonconfig = self.fetch() 325 | if not jsonconfig : 326 | return False 327 | 328 | 329 | bridge_now = Bridge(self.br_name, self.dmvpn_addr) 330 | bridge_now.load_from_os() 331 | 332 | bridge_new = Bridge(self.br_name, self.dmvpn_addr) 333 | if not bridge_new.load_from_json(jsonconfig) : 334 | return False 335 | 336 | logger.info("Start to configure bridge '%s'" % bridge_now.name) 337 | 338 | # 1. remove unnecessary vlans 339 | for vlan in bridge_now.vlans - bridge_new.vlans : 340 | logger.debug("- remove vlan %d", vlan) 341 | bridge_new.delete_vxlan(vlan) 342 | 343 | # 2. create new vlans 344 | for vlan in bridge_new.vlans - bridge_now.vlans : 345 | logger.debug("- add vlan %d", vlan) 346 | bridge_new.add_vxlan(vlan) 347 | 348 | 349 | # 3. check removed or changed ports and remove them 350 | for now_port in bridge_now.list_ports() : 351 | if not bridge_new.find_port(now_port.name) : 352 | logger.debug("- destroy %s" % now_port) 353 | now_port.destroy() 354 | 355 | elif now_port.diff_check(bridge_new.find_port(now_port.name)) : 356 | now_port.destroy() 357 | 358 | # 4. check new or changed ports and setup them 359 | for new_port in bridge_new.list_ports() : 360 | if not bridge_now.find_port(new_port.name) : 361 | logger.debug("- setup %s" % new_port) 362 | new_port.setup() 363 | 364 | elif new_port.diff_check(bridge_now.find_port(new_port.name)) : 365 | now_port = bridge_now.find_port(new_port.name) 366 | logger.debug("- setup %s" % new_port) 367 | logger.debug("- current: %s" % now_port) 368 | 369 | 370 | new_port.setup() 371 | 372 | return True 373 | 374 | 375 | def fetch_loop(self, loop, once) : 376 | 377 | try: 378 | ret = self.execute() 379 | if ret : 380 | if once : 381 | loop.stop() 382 | return True 383 | loop.call_later(self.interval, self.fetch_loop, loop, once) 384 | else : 385 | loop.call_later(self.failed_interval, 386 | self.fetch_loop, loop, once) 387 | except KeyboardInterrupt: 388 | logger.info("Keyboard interrupt. stop portconfig loop.") 389 | loop.stop() 390 | 391 | except Exception as e: 392 | logger.error("portconfig error occurd:%s: %s" % 393 | (e.__class__.__name__, e )) 394 | loop.call_later(self.failed_interval, self.fetch_loop, loop, once) 395 | 396 | 397 | def start_loop(self) : 398 | 399 | loop = asyncio.get_event_loop() 400 | loop.call_soon(self.fetch_loop, loop, False) 401 | loop.run_forever() 402 | loop.close() 403 | 404 | 405 | def execute_once(self) : 406 | loop = asyncio.new_event_loop() 407 | loop.call_soon(self.fetch_loop, loop, True) 408 | loop.run_forever() 409 | loop.close() 410 | 411 | 412 | 413 | @app.route("/update-kick", methods = [ "GET", "POST" ]) 414 | def portconfig_rest_update_from_url() : 415 | logger.info("Update trigger. start to obtain JSON") 416 | 417 | global portconfig 418 | portconfig.execute_once() 419 | return "Start Update", 200 420 | 421 | 422 | def portconfig_rest_start(bind_addr, bind_port) : 423 | logger.info("Start REST Gateway on %s:%d" % (bind_addr, 424 | bind_port)) 425 | th = threading.Thread(name = "rest_gw", target = app.run, 426 | kwargs = { 427 | "threaded" : True, 428 | "host" : bind_addr, 429 | "port" : bind_port, 430 | }) 431 | th.start() 432 | 433 | 434 | 435 | if __name__ == "__main__" : 436 | portconfig = PortConfig(sys.argv[1]) 437 | portconfig_rest_start(portconfig.bind_addr, portconfig.bind_port) 438 | portconfig.start_loop() 439 | -------------------------------------------------------------------------------- /docker/route-server/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:17.10 2 | 3 | ARG workdir="/root" 4 | 5 | # install required packages and usuful applications 6 | RUN apt-get update && apt-get install -y \ 7 | wget iputils-ping iproute2 kmod \ 8 | git autoconf automake libtool make gawk libreadline-dev \ 9 | texinfo dejagnu pkg-config libpam0g-dev libjson-c-dev bison flex \ 10 | python-pytest libc-ares-dev python3-dev libsystemd-dev \ 11 | libgmp-dev openssl gperf python3-jinja2 12 | 13 | # setup FRRouting with the cumulus extension for EVPN/VXLAN 14 | RUN cd ${workdir} \ 15 | && groupadd -g 92 frr \ 16 | && groupadd -r -g 85 frrvty \ 17 | && adduser --system --ingroup frr --home /var/run/frr/ \ 18 | --gecos "FRR suite" --shell /sbin/nologin frr \ 19 | && usermod -a -G frrvty frr \ 20 | && git clone https://github.com/frrouting/frr.git frr \ 21 | && cd frr \ 22 | && git checkout -b itworks 67c0a9206ce9b50dacb6561e7dccdc0ae8e7fc43 \ 23 | && ./bootstrap.sh \ 24 | && ./configure \ 25 | --prefix=/usr \ 26 | --enable-exampledir=/usr/share/doc/frr/examples/ \ 27 | --localstatedir=/var/run/frr \ 28 | --sbindir=/usr/lib/frr \ 29 | --sysconfdir=/etc/frr \ 30 | --enable-watchfrr \ 31 | --enable-multipath=64 \ 32 | --enable-user=frr \ 33 | --enable-group=frr \ 34 | --enable-vty-group=frrvty \ 35 | --enable-configfile-mask=0640 \ 36 | --enable-logfile-mask=0640 \ 37 | --enable-systemd=yes \ 38 | --with-pkg-git-version \ 39 | --with-pkg-extra-version=-Nante-WAN \ 40 | --enable-cumulus \ 41 | && make -j 4\ 42 | && make install \ 43 | && install -m 755 -o frr -g frr -d /var/log/frr \ 44 | && install -m 775 -o frr -g frrvty -d /etc/frr \ 45 | && install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf \ 46 | && install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf \ 47 | && install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf \ 48 | && install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf \ 49 | && install -m 644 tools/frr.service /etc/systemd/system/frr.service \ 50 | && install -m 644 tools/etc/default/frr /etc/default/frr \ 51 | && install -m 644 tools/etc/frr/daemons /etc/frr/daemons \ 52 | && install -m 644 tools/etc/frr/daemons.conf /etc/frr/daemons.conf \ 53 | && install -m 644 tools/etc/frr/frr.conf /etc/frr/frr.conf \ 54 | && install -m 644 -o frr -g frr tools/etc/frr/vtysh.conf \ 55 | /etc/frr/vtysh.conf \ 56 | && rm -f /etc/frr/daemons 57 | 58 | ADD daemons /etc/frr/daemons 59 | 60 | # setup StrongSwan 61 | RUN cd ${workdir} \ 62 | && git clone -b tteras --depth=1 \ 63 | git://git.alpinelinux.org/user/tteras/strongswan 64 | RUN cd ${workdir}/strongswan \ 65 | && autoreconf -i || true \ 66 | && autoconf \ 67 | && autoreconf -i \ 68 | && ./configure \ 69 | && make -j 4 || true \ 70 | && make -j 4 || true \ 71 | && make -j 4 \ 72 | && make install \ 73 | && rm -f /usr/local/etc/ipsec.conf 74 | ADD ipsec.conf /usr/local/etc/ipsec.conf 75 | 76 | # setup Config Render 77 | ADD templates ${workdir}/templates 78 | ADD config-render.py ${workdir}/config-render.py 79 | 80 | 81 | CMD bash -c "/root/config-render.py /etc/nante-wan.conf && /usr/lib/frr/frr start && ipsec start && bash" 82 | -------------------------------------------------------------------------------- /docker/route-server/config-render.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import sys 5 | import socket 6 | import fcntl 7 | import struct 8 | import configparser 9 | from optparse import OptionParser 10 | from jinja2 import Environment, FileSystemLoader 11 | 12 | 13 | 14 | def config_render_frr_conf(config, of) : 15 | 16 | d = { "dmvpn_addr" : config.get("general", "dmvpn_addr")} 17 | 18 | for param in config["routing"] : 19 | d[param] = config["routing"][param] 20 | 21 | tmp = os.path.join(os.path.dirname(__file__), "templates") 22 | env = Environment(loader = FileSystemLoader(tmp, encoding = "utf-8")) 23 | tpl = env.get_template("frr.conf.template") 24 | 25 | frr_conf = tpl.render(d) 26 | print(frr_conf, file = of) 27 | 28 | 29 | def config_render_ipsec_secrets(config, of) : 30 | 31 | d = { 32 | "ipsec_secret" : config.get("routing", "ipsec_secret") 33 | } 34 | 35 | tmp = os.path.join(os.path.dirname(__file__), "templates") 36 | env = Environment(loader = FileSystemLoader(tmp, encoding = "utf-8")) 37 | tpl = env.get_template("ipsec.secrets.template") 38 | 39 | ipsec_secrets = tpl.render(d) 40 | print(ipsec_secrets, file = of) 41 | 42 | 43 | 44 | if __name__ == '__main__' : 45 | 46 | desc = "usage: %prog [options] configfile" 47 | parser = OptionParser(desc) 48 | parser.add_option('-s', '--stdout', action = "store_true", 49 | default = None, dest = "stdout", 50 | help = "output to stdout") 51 | options, args = parser.parse_args() 52 | try : 53 | ini_file = args.pop() 54 | except : 55 | print("config file is not specified") 56 | sys.exit() 57 | 58 | config = configparser.ConfigParser() 59 | config.read(ini_file) 60 | 61 | if options.stdout : 62 | of_frr = sys.stdout 63 | of_ipsec_secrets = sys.stdout 64 | else : 65 | of_frr = open("/etc/frr/frr.conf", "w") 66 | of_ipsec_secrets = open("/usr/local/etc/ipsec.secrets", "w") 67 | 68 | 69 | config_render_frr_conf(config, of_frr) 70 | config_render_ipsec_secrets(config, of_ipsec_secrets) 71 | -------------------------------------------------------------------------------- /docker/route-server/daemons: -------------------------------------------------------------------------------- 1 | # This file tells the frr package which daemons to start. 2 | # 3 | # Entries are in the format: =(yes|no|priority) 4 | # 0, "no" = disabled 5 | # 1, "yes" = highest priority 6 | # 2 .. 10 = lower priorities 7 | # Read /usr/share/doc/frr/README.Debian for details. 8 | # 9 | # Sample configurations for these daemons can be found in 10 | # /usr/share/doc/frr/examples/. 11 | # 12 | # ATTENTION: 13 | # 14 | # When activation a daemon at the first time, a config file, even if it is 15 | # empty, has to be present *and* be owned by the user and group "frr", else 16 | # the daemon will not be started by /etc/init.d/frr. The permissions should 17 | # be u=rw,g=r,o=. 18 | # When using "vtysh" such a config file is also needed. It should be owned by 19 | # group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too. 20 | # 21 | # The watchfrr daemon is always started. Per default in monitoring-only but 22 | # that can be changed via /etc/frr/daemons.conf. 23 | # 24 | zebra=yes 25 | bgpd=yes 26 | ospfd=no 27 | ospf6d=no 28 | ripd=no 29 | ripngd=no 30 | isisd=no 31 | pimd=no 32 | ldpd=no 33 | nhrpd=yes 34 | eigrpd=no 35 | babeld=no 36 | -------------------------------------------------------------------------------- /docker/route-server/ipsec.conf: -------------------------------------------------------------------------------- 1 | 2 | # IPsec configuration for dmvpn 3 | 4 | config setup 5 | 6 | conn dmvpn 7 | left=%any 8 | right=%any 9 | leftprotoport=gre 10 | rightprotoport=gre 11 | type=transport 12 | authby=secret 13 | auto=add 14 | keyingtries=%forever 15 | -------------------------------------------------------------------------------- /docker/route-server/templates/frr.conf.template: -------------------------------------------------------------------------------- 1 | frr defaults traditional 2 | ! 3 | nhrp nflog-group 1 4 | ! 5 | service integrated-vtysh-config 6 | ! 7 | log file /var/log/frr/frr.log 8 | ! 9 | interface gre1 10 | description dmvpn 11 | ip address {{ dmvpn_addr }}/32 12 | ip nhrp network-id 1 13 | ip nhrp redirect 14 | tunnel protection vici profile dmvpn 15 | tunnel source {{ wan_interface }} 16 | ! 17 | router bgp {{ as_number }} 18 | bgp router-id {{ dmvpn_addr }} 19 | no bgp default ipv4-unicast 20 | bgp cluster-id {{ dmvpn_addr }} 21 | neighbor evpn-peer peer-group 22 | neighbor evpn-peer remote-as {{ as_number }} 23 | neighbor evpn-peer update-source {{ dmvpn_interface }} 24 | neighbor evpn-peer capability extended-nexthop 25 | bgp listen range {{ bgp_range }} peer-group evpn-peer 26 | ! 27 | address-family ipv4 unicast 28 | redistribute nhrp 29 | neighbor evpn-peer activate 30 | neighbor evpn-peer route-reflector-client 31 | exit-address-family 32 | ! 33 | address-family l2vpn evpn 34 | neighbor evpn-peer activate 35 | neighbor evpn-peer route-reflector-client 36 | advertise-all-vni 37 | exit-address-family 38 | vnc defaults 39 | response-lifetime 3600 40 | exit-vnc 41 | ! 42 | line vty 43 | ! 44 | end 45 | -------------------------------------------------------------------------------- /docker/route-server/templates/ipsec.secrets.template: -------------------------------------------------------------------------------- 1 | 2 | # ipsec secret 3 | : PSK "{{ ipsec_secret }}" 4 | -------------------------------------------------------------------------------- /docker/routing/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:17.10 2 | 3 | ARG workdir="/root" 4 | 5 | # install required packages and usuful applications 6 | RUN apt-get update && apt-get install -y \ 7 | wget iputils-ping iproute2 kmod \ 8 | git autoconf automake libtool make gawk libreadline-dev \ 9 | texinfo dejagnu pkg-config libpam0g-dev libjson-c-dev bison flex \ 10 | python-pytest libc-ares-dev python3-dev libsystemd-dev \ 11 | libgmp-dev openssl gperf python3-jinja2 12 | 13 | # setup FRRouting with the cumulus extension for EVPN/VXLAN 14 | RUN cd ${workdir} \ 15 | && groupadd -g 92 frr \ 16 | && groupadd -r -g 85 frrvty \ 17 | && adduser --system --ingroup frr --home /var/run/frr/ \ 18 | --gecos "FRR suite" --shell /sbin/nologin frr \ 19 | && usermod -a -G frrvty frr \ 20 | && git clone https://github.com/frrouting/frr.git frr \ 21 | && cd frr \ 22 | && git checkout -b itworks 67c0a9206ce9b50dacb6561e7dccdc0ae8e7fc43 \ 23 | && ./bootstrap.sh \ 24 | && ./configure \ 25 | --prefix=/usr \ 26 | --enable-exampledir=/usr/share/doc/frr/examples/ \ 27 | --localstatedir=/var/run/frr \ 28 | --sbindir=/usr/lib/frr \ 29 | --sysconfdir=/etc/frr \ 30 | --enable-watchfrr \ 31 | --enable-multipath=64 \ 32 | --enable-user=frr \ 33 | --enable-group=frr \ 34 | --enable-vty-group=frrvty \ 35 | --enable-configfile-mask=0640 \ 36 | --enable-logfile-mask=0640 \ 37 | --enable-systemd=yes \ 38 | --with-pkg-git-version \ 39 | --with-pkg-extra-version=-Nante-WAN \ 40 | --enable-cumulus \ 41 | && make -j 4\ 42 | && make install \ 43 | && install -m 755 -o frr -g frr -d /var/log/frr \ 44 | && install -m 775 -o frr -g frrvty -d /etc/frr \ 45 | && install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf \ 46 | && install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf \ 47 | && install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf \ 48 | && install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf \ 49 | && install -m 644 tools/frr.service /etc/systemd/system/frr.service \ 50 | && install -m 644 tools/etc/default/frr /etc/default/frr \ 51 | && install -m 644 tools/etc/frr/daemons /etc/frr/daemons \ 52 | && install -m 644 tools/etc/frr/daemons.conf /etc/frr/daemons.conf \ 53 | && install -m 644 tools/etc/frr/frr.conf /etc/frr/frr.conf \ 54 | && install -m 644 -o frr -g frr tools/etc/frr/vtysh.conf \ 55 | /etc/frr/vtysh.conf \ 56 | && rm -f /etc/frr/daemons 57 | 58 | ADD daemons /etc/frr/daemons 59 | 60 | # setup StrongSwan 61 | RUN cd ${workdir} \ 62 | && git clone -b tteras --depth=1 \ 63 | git://git.alpinelinux.org/user/tteras/strongswan 64 | RUN cd ${workdir}/strongswan \ 65 | && autoreconf -i || true \ 66 | && autoconf \ 67 | && autoreconf -i \ 68 | && ./configure \ 69 | && make -j 4 || true \ 70 | && make -j 4 || true \ 71 | && make -j 4 \ 72 | && make install \ 73 | && rm -f /usr/local/etc/ipsec.conf 74 | ADD ipsec.conf /usr/local/etc/ipsec.conf 75 | 76 | # setup Config Render 77 | ADD templates ${workdir}/templates 78 | ADD config-render.py ${workdir}/config-render.py 79 | 80 | 81 | CMD bash -c "/root/config-render.py /etc/nante-wan.conf && /usr/lib/frr/frr start && ipsec start && bash" 82 | -------------------------------------------------------------------------------- /docker/routing/config-render.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import re 4 | import os 5 | import sys 6 | import socket 7 | import fcntl 8 | import struct 9 | import configparser 10 | from optparse import OptionParser 11 | from jinja2 import Environment, FileSystemLoader 12 | 13 | 14 | 15 | def config_render_frr_conf(config, of) : 16 | 17 | d = { "dmvpn_addr" : config.get("general", "dmvpn_addr")} 18 | 19 | rr_addrs = [] 20 | nhs_nbma_addrs = [] 21 | 22 | for param in config["routing"] : 23 | d[param] = config["routing"][param] 24 | if re.match(r"rr_addr\d*", param) : 25 | rr_addrs.append(config["routing"][param]) 26 | 27 | if re.match(r"nhs_nbma_addr\d*", param) : 28 | nhs_nbma_addrs.append(config["routing"][param]) 29 | 30 | d["rr_addrs"] = rr_addrs 31 | d["nhs_nbma_addrs"] = nhs_nbma_addrs 32 | 33 | tmp = os.path.join(os.path.dirname(__file__), "templates") 34 | env = Environment(loader = FileSystemLoader(tmp, encoding = "utf-8")) 35 | tpl = env.get_template("frr.conf.template") 36 | 37 | frr_conf = tpl.render(d) 38 | print(frr_conf, file = of) 39 | 40 | 41 | def config_render_ipsec_secrets(config, of) : 42 | 43 | d = { 44 | "ipsec_secret" : config.get("routing", "ipsec_secret") 45 | } 46 | 47 | tmp = os.path.join(os.path.dirname(__file__), "templates") 48 | env = Environment(loader = FileSystemLoader(tmp, encoding = "utf-8")) 49 | tpl = env.get_template("ipsec.secrets.template") 50 | 51 | ipsec_secrets = tpl.render(d) 52 | print(ipsec_secrets, file = of) 53 | 54 | 55 | 56 | if __name__ == '__main__' : 57 | 58 | desc = "usage: %prog [options] configfile" 59 | parser = OptionParser(desc) 60 | parser.add_option('-s', '--stdout', action = "store_true", 61 | default = None, dest = "stdout", 62 | help = "output to stdout") 63 | options, args = parser.parse_args() 64 | try : 65 | ini_file = args.pop() 66 | except : 67 | print("config file is not specified") 68 | sys.exit() 69 | 70 | config = configparser.ConfigParser() 71 | config.read(ini_file) 72 | 73 | if options.stdout : 74 | of_frr = sys.stdout 75 | of_ipsec_secrets = sys.stdout 76 | else : 77 | of_frr = open("/etc/frr/frr.conf", "w") 78 | of_ipsec_secrets = open("/usr/local/etc/ipsec.secrets", "w") 79 | 80 | 81 | config_render_frr_conf(config, of_frr) 82 | config_render_ipsec_secrets(config, of_ipsec_secrets) 83 | -------------------------------------------------------------------------------- /docker/routing/daemons: -------------------------------------------------------------------------------- 1 | # This file tells the frr package which daemons to start. 2 | # 3 | # Entries are in the format: =(yes|no|priority) 4 | # 0, "no" = disabled 5 | # 1, "yes" = highest priority 6 | # 2 .. 10 = lower priorities 7 | # Read /usr/share/doc/frr/README.Debian for details. 8 | # 9 | # Sample configurations for these daemons can be found in 10 | # /usr/share/doc/frr/examples/. 11 | # 12 | # ATTENTION: 13 | # 14 | # When activation a daemon at the first time, a config file, even if it is 15 | # empty, has to be present *and* be owned by the user and group "frr", else 16 | # the daemon will not be started by /etc/init.d/frr. The permissions should 17 | # be u=rw,g=r,o=. 18 | # When using "vtysh" such a config file is also needed. It should be owned by 19 | # group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too. 20 | # 21 | # The watchfrr daemon is always started. Per default in monitoring-only but 22 | # that can be changed via /etc/frr/daemons.conf. 23 | # 24 | zebra=yes 25 | bgpd=yes 26 | ospfd=no 27 | ospf6d=no 28 | ripd=no 29 | ripngd=no 30 | isisd=no 31 | pimd=no 32 | ldpd=no 33 | nhrpd=yes 34 | eigrpd=no 35 | babeld=no 36 | -------------------------------------------------------------------------------- /docker/routing/ipsec.conf: -------------------------------------------------------------------------------- 1 | 2 | # IPsec configuration for dmvpn 3 | 4 | config setup 5 | 6 | conn dmvpn 7 | left=%any 8 | right=%any 9 | leftprotoport=gre 10 | rightprotoport=gre 11 | type=transport 12 | authby=secret 13 | auto=add 14 | keyingtries=%forever 15 | -------------------------------------------------------------------------------- /docker/routing/templates/frr.conf.template: -------------------------------------------------------------------------------- 1 | frr defaults traditional 2 | ! 3 | nhrp nflog-group 1 4 | ! 5 | service integrated-vtysh-config 6 | ! 7 | log file /var/log/frr/frr.log 8 | ! 9 | interface gre1 10 | description dmvpn 11 | ip address {{ dmvpn_addr }}/32 12 | ip nhrp network-id 1 13 | {% for nhs_nbma_addr in nhs_nbma_addrs %} 14 | ip nhrp nhs dynamic nbma {{ nhs_nbma_addr }} 15 | {% endfor %} 16 | ip nhrp registration no-unique 17 | ip nhrp shortcut 18 | tunnel protection vici profile dmvpn 19 | tunnel source {{ wan_interface }} 20 | ! 21 | router bgp {{ as_number }} 22 | bgp router-id {{ dmvpn_addr }} 23 | no bgp default ipv4-unicast 24 | neighbor evpn-peer peer-group 25 | neighbor evpn-peer remote-as {{ as_number }} 26 | neighbor evpn-peer update-source {{ dmvpn_interface }} 27 | neighbor evpn-peer capability extended-nexthop 28 | {% for rr_addr in rr_addrs %} 29 | neighbor {{ rr_addr }} peer-group evpn-peer 30 | {% if bgp_password %}neighbor {{ rr_addr }} password {{ bgp_password }} 31 | {% endif %} 32 | {% endfor %}! 33 | address-family ipv4 unicast 34 | neighbor evpn-peer activate 35 | neighbor evpn-peer soft-reconfiguration inbound 36 | exit-address-family 37 | ! 38 | address-family l2vpn evpn 39 | neighbor evpn-peer activate 40 | advertise-all-vni 41 | exit-address-family 42 | vnc defaults 43 | response-lifetime 3600 44 | exit-vnc 45 | ! 46 | line vty 47 | ! 48 | end 49 | -------------------------------------------------------------------------------- /docker/routing/templates/ipsec.secrets.template: -------------------------------------------------------------------------------- 1 | 2 | # ipsec secret 3 | : PSK "{{ ipsec_secret }}" 4 | -------------------------------------------------------------------------------- /nante-wan.conf: -------------------------------------------------------------------------------- 1 | # Nante-WAN config file 2 | 3 | ### General Setting 4 | [general] 5 | # @dmvpn_addr: IP address of DMVPN interface 6 | dmvpn_addr = 10.0.0.10 7 | 8 | 9 | ### Config Fetcher Configuration 10 | [config_fetch] 11 | 12 | # @timeout: timeout for fetching config files (sec) 13 | # @interval: interval for regular config file fetch (sec) 14 | # @failed_interval: interval when config fetch fails (sec) 15 | timeout = 5 16 | interval = 3600 17 | failed_interval = 5 18 | 19 | 20 | 21 | ### Routing Container Configuration 22 | [routing] 23 | 24 | # @wan_interface: name of interface which is connected to uplink 25 | # @vpn_interface: name of DMVPN interface 26 | wan_interface = eth0 27 | dmvpn_interface = gre1 28 | 29 | # @nhs_nbma_addr\d+: Nexthop Server's IP address on the DMVPN overlay. 30 | # NHS addr is resolved by 'dynamic' 31 | nhs_nbma_addr1 = 192.168.0.10 32 | nhs_nbma_addr2 = 192.168.0.11 33 | 34 | # @as_number: AS number used in iBGP configuration for EVPN. 35 | # @rr_addr\d+: BGP Route Reflector address which this host connect to. 36 | # @bgp_password: BGP password (optional) 37 | as_number = 65000 38 | rr_addr1 = 10.0.0.100 39 | rr_addr2 = 10.0.0.101 40 | bgp_password = bgp_password 41 | 42 | # @ipsec_secret: IPsec shared secret. 43 | ipsec_secret = hogehogemogamoga 44 | 45 | # @gre_key: GRE key field value 46 | # @gre_ttl: TTL of GRE encapsulated packets 47 | gre_key = 1 48 | gre_ttl = 64 49 | 50 | # @bgp_range: Prefix where BGP connections come from (for Route Server) 51 | bgp_range = 10.0.0.0/16 52 | 53 | 54 | 55 | ### Port Config Container Configuration 56 | [portconfig] 57 | 58 | # @br_interface: bridge interface name to be configured 59 | # @json_url_prefix: Prefix for config file URL. URL is PREFIX/[dmvpn_addr].json 60 | br_interface = bridge 61 | json_url_prefix = http://10.0.0.100/portconfig 62 | 63 | # @bind_port: Port number of portconfig REST API for update trigger 64 | # @bind_addr: IP Address of REST API for update trigger. default dmvpn_addr 65 | bind_port = 8080 66 | #bind_addr = 10.0.0.10 67 | 68 | # @json_config: port config json config file (for test) 69 | #json_config = /tmp/portconfig-test.json 70 | 71 | 72 | 73 | ### ebtables-based ACL Configuration 74 | [ebconfig] 75 | 76 | # @br_interface: bridge interface name to be configured 77 | # @json_url_prefix: Prefix for config file URL. URL is PREFIX/[dmvpn_addr].json 78 | br_interface = bridge 79 | json_url_prefix = http://10.0.0.0/ebconfig 80 | 81 | # @bind_port: Port number for portconfig REST API for trigger update 82 | # @bind_addr: address for REST API for trigger update. default dmvpn_addr 83 | bind_port = 8081 84 | bind_addr = 10.0.0.10 85 | 86 | # @json_config: port config json config file (for test) 87 | #json_config = /tmp/ebconfig-test.json 88 | -------------------------------------------------------------------------------- /start.py: -------------------------------------------------------------------------------- 1 | docker/netstack-setup/start.py --------------------------------------------------------------------------------