├── .gitignore ├── BLOG.md ├── BLOG2.md ├── LICENSE ├── README.md ├── diagrams ├── clustered_k8s_nodeport.png ├── clustered_k8s_traefik.png ├── end_to_end_singleserver.png └── multi_server_proxied.png ├── k8s ├── README.md ├── redis │ ├── redis-master-deployment.yaml │ ├── redis-master-service.yaml │ ├── redis-slave-deployment.yaml │ └── redis-slave-service.yaml ├── traefik │ ├── crd.yaml │ ├── ingress.yaml │ ├── rbac.yaml │ └── traefik.yaml └── wsk │ ├── wsk-deployment.yaml │ └── wsk-service.yaml └── stack ├── .dockerignore ├── Dockerfile ├── README.md ├── client_socket.js ├── docker-compose.yml ├── haproxy.cfg ├── package-lock.json ├── package.json └── server_socket.js /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules 2 | .DS_Store -------------------------------------------------------------------------------- /BLOG.md: -------------------------------------------------------------------------------- 1 | # Scaling Websockets in the Cloud (Part 1). From a robust solution leveraging Socket.io and Redis to a distributed and containerazed architecture with Docker and Kubernetes 2 | 3 | In this post we will go through several configuration of Websockets. Let's say from zero infrastructure to quite complex cluster configurations. 4 | 5 | ## Intro 6 | 7 | Websocket is a wide spread protocol allowing full-duplex communication over TCP. There are several libraries implementing that protocol, one of the most robust and well-known is `Socket.io` on the Javascript side, which allows to quickly create real-time communication patterns. 8 | 9 | ## Simple End-to-End Communication 10 | 11 | Creating a single server in `NodeJS` where multiple clients can connect is fairly simple and straightforward using any example of the `Socket.io` library itself. 12 | 13 | | ![End-to_end Architecture](diagrams/end_to_end_singleserver.png) | 14 | | :--: | 15 | 16 | One of the main feature of the library is that it can wrap the websocket within an http server. 17 | For example, a first approach can be that as soon as a client connects to the websocket, the server application logs the message and may wait for a specific `topic` on it: 18 | 19 | ```Javascript 20 | // Server 21 | const socketServer = require('http').createServer(); 22 | const io = require('socket.io')(socketServer, {}); 23 | const socketPort = 5000; 24 | 25 | io.on('connection', client => { 26 | console.log('New incoming Connection from', client.id); 27 | client.on('testMsg', function(message) { 28 | console.log('Message from the client:',client.id,'->',message); 29 | }) 30 | }); 31 | socketServer.listen(socketPort); 32 | 33 | // Client 34 | const io = require('socket.io-client'); 35 | const client = io('http://localhost:5000', { 36 | transports: ['websocket'] 37 | }); 38 | 39 | client.on('connect', function(){ 40 | console.log("Connected!"); 41 | client.emit('testMsg', 'a message'); 42 | }); 43 | ``` 44 | 45 | The server after client connection will wait for messages of type `testMsg`. 46 | 47 | The client is configured to employ native websockets, without attempting naïves solutions as http polling. 48 | 49 | ```Javascript 50 | { transports: ['websocket'] } 51 | ``` 52 | 53 | The server is also able to broadcast messages to all the clients connected to the websocket. 54 | 55 | ```Javascript 56 | io.emit("brdcst", 'A broadcast message'); 57 | ``` 58 | 59 | >`Tip`: those messages will be received by ALL the clients connected. 60 | 61 | ## Multi-Server to Multi-Client communication 62 | 63 | The previous example was simple and fun. We can play with it effectively. But creating a scalable multiserver architecture is another story and is not as immediate as the previous case. 64 | 65 | There are 2 main issues to take into account: 66 | 67 | 1. multiple websocket servers involved should coordinate among themselves. Once a server receives a message from a client, it should ensure that any clients connected to any servers receive this message. 68 | 2. when a client handshakes and establishes a connection with a server, all its future messages should pass through the same server, otherwise another server will refuse further messages upon receiving them. 69 | 70 | Luckily the two problems are not so difficult as they appear. 71 | 72 | The first problem can be natively addressed by employing an `Adapter`. This `Socket.io` mechanism, which natively is *in-memory*, allows to pass messages between processes (servers) and to broadcast events to all clients. 73 | The most suitable adapter for a multi-server scenario is [socket.io-redis](https://github.com/socketio/socket.io-redis), leveraging the pub/sub paradigm of `Redis`. 74 | As anticipated the configuration is simple and smooth, just a small piece of code is needed. 75 | 76 | ```Javascript 77 | const redisAdapter = require('socket.io-redis'); 78 | const redisHost = process.env.REDIS_HOST || 'localhost'; 79 | io.adapter(redisAdapter({ host: redisHost, port: 6379 })); 80 | ``` 81 | 82 | The second problem of keeping a session initialized by one client with the same origin server can be addressed without particular pain. The trick resides in creating `sticky` connections so that when a client connects to a specific server, it begins a session that is effectively bound to the same server. 83 | 84 | This cannot be achieved directly but we should place *something* in front of the NodeJS server application. 85 | This may typically be a `Reverse Proxy`, like NGINX, HAProxy, Apache Httpd. In case of HAProxy, imagining two replicas of the websocket server application, the configuration of the *backend* part would be: 86 | 87 | ```Bash 88 | cookie io prefix indirect nocache 89 | server ws0 socket-server:5000 check cookie ws0 90 | server ws1 socket-server2:5000 check cookie ws1 91 | ``` 92 | 93 | which sets the *io* session cookie upon handshake. 94 | 95 | ### Multi-Server recap 96 | 97 | To summarize what you should expect with this configuration: 98 | 99 | 1. Each client will establish a connection with a specific websocket application server instance. 100 | 2. Any message sent from the client is always passing through the same server with whom the session was initialized. 101 | 3. Upon receiving a message, the server may broadcast it. The adapter is in charge of advertising all the other servers that will in turn forward the message to all the clients which have established a connection with them. 102 | 103 | ## Containerizing everything: a Docker Stack of Microservices 104 | 105 | What we seen so far can be summarized in the following diagram: 106 | 107 | | ![Multi Server Proxied Architecture](diagrams/multi_server_proxied.png) | 108 | | :--: | 109 | 110 | The natural evolution is a bunch of microservices containerized as Docker images. Our previous scenario holds 3 building blocks that can be containerized: 111 | 112 | * Websocket server application (1+ replicas) 113 | * Redis (pub/sub adapter) 114 | * HAProxy (as Reverse Proxy handling Sticky Connections) 115 | 116 | For the first service I created a predefined multi-staged [Docker image](https://hub.docker.com/r/sw360cab/wsk-base) starting from the `NodeJS` native image in its `alpine` variant. The other two microservices will work with native images. 117 | The 3 services can leverage `docker-compose` to be deployed. Their configuration can be the following: 118 | 119 | ```yaml 120 | version: '3.2' 121 | services: 122 | socket-server: 123 | image: sw360cab/wsk-base:0.1.1 124 | container_name: socket-server 125 | restart: always 126 | deploy: 127 | replicas: 2 128 | environment: 129 | - "REDIS_HOST=redis" 130 | 131 | proxy: 132 | image: haproxy:1 133 | container_name: proxy 134 | ports: 135 | - 5000:80 136 | volumes: 137 | - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg 138 | depends_on: 139 | - socket-server 140 | 141 | redis: 142 | image: redis:5.0 143 | container_name: redis 144 | ``` 145 | 146 | The `deploy` part will only work in a `Swarm` stack deployment. Assuming that you have a `Swarm` up and ready, the deploy can be started with: 147 | 148 | ```bash 149 | docker stack deploy --compose-file stack/docker-compose.yml wsk 150 | ``` 151 | 152 | However the previous option (*deploy*) is not valid outside a `Swarm`, for example within a local development environment. 153 | 154 | In this case you may use the `docker-compose` basic command. An extra replica of the websocket server application may be manually added. 155 | 156 | ```yaml 157 | socket-server2: 158 | image: sw360cab/wsk-base:0.1.1 159 | container_name: socket-server2 160 | restart: always 161 | environment: 162 | - "REDIS_HOST=redis" 163 | ``` 164 | 165 | >`Tip`: another elegant way to achieve that is by using `docker-compose up -d --scale socket-server=2 socket-server` 166 | 167 | Now to bring everything up is sufficient to execute: 168 | 169 | ```bash 170 | docker-compose up -d 171 | ``` 172 | 173 | >`Tip`: when using `HAProxy` remember to define the correct hostnames of the endpoints involved in the configuration file. 174 | 175 | ```bash 176 | backend bk_socket 177 | http-check expect status 200 178 | cookie io prefix indirect nocache 179 | # Using the `io` cookie set upon handshake 180 | server ws0 socket-server:5000 check cookie ws0 181 | # Uncomment following line for non-Swarm usage 182 | #server ws1 socket-server2:5000 check cookie ws1 183 | ``` 184 | 185 | >`Tip`: After introducing a *Reverse Proxy* remember to tune the endpoint hostname and port which the client will attempt to connect to. 186 | 187 | ## Orchestrating more and more: a Kubernetes cluster in action 188 | 189 | So far, we have already addressed our problems for a multi-server case. We also achieved a containerized stack of microservices in Docker. 190 | Now we are able to add a further step, we can orchestrate all the microservices transforming them in `Kubernetes` (*K8s*) resources and running in a cluster. 191 | 192 | The shift from Docker is really easy. First of all we can drop the Reverse Proxy service, we will see later why. Then we will transform each service into a pair of service and deployment. The service will be the interface for receiving requests, whereas the deployment will maintain a replica set of pods (if you don't know K8s terminology, imagine each pod as a container) running the application: in this case either the Websocket application or Redis. 193 | Each request received by the service will be forwarded to the replica set defined by the deployment. 194 | 195 | Supposing that you have a K8s cluster setup (I suggest [K3s](https://k3s.io/ "K3s: Lightweight Kubernetes") or [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/ "kind") for development purpose), we will have: 196 | 197 | * Redis service and deployment (to add more sugar in the [repo](https://github.com/sw360cab/websockets-scaling) I employed a Master-Slave Redis solution) 198 | * Websocket Application service and deployment (the latter is composed of 3 replicas of the custom image of the NodeJS application) 199 | 200 | The new architecture we will achieve can be summarized as follow: 201 | 202 | | ![Clustered K8s Architecture](diagrams/clustered_k8s_nodeport.png) | 203 | | :--: | 204 | 205 | ## Websocket Kubernetes Service 206 | 207 | As we seen before in this shift towards Kubernetes we skipped the Reverse Proxy part (*HAProxy*). 208 | This is completely on purpose, because creating a Kubernetes Service enables directly such a form of proxing. 209 | 210 | Although a Kubernetes Service can be exposed outside the cluster in many ways (NodePort, LoadBalancer, Ingress, check [here](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types)), in this case I decided for a simple way. Indeed using `NodePort` allows to expose a specific port outside the cluster, where the clients will connect to. 211 | 212 | ```yaml 213 | apiVersion: v1 214 | kind: Service 215 | metadata: 216 | name: wsk-svc 217 | spec: 218 | selector: 219 | app: wsk-base 220 | sessionAffinity: ClientIP 221 | sessionAffinityConfig: 222 | clientIP: 223 | timeoutSeconds: 10 224 | ports: 225 | - protocol: TCP 226 | port: 3000 227 | targetPort: 5000 228 | nodePort: 30000 229 | type: NodePort 230 | ``` 231 | 232 | The key point of the service is `sessionAffinity: ClientIP` which will mimic Sticky connection for the service. 233 | 234 | >`Tip`: In general `Round Robin` among Pods is not guaranteed using a simple solution like *NodePort*. In order to mimic Round Robin policy, this part of the previous configuration was employed: 235 | 236 | ```yaml 237 | sessionAffinityConfig: 238 | clientIP: 239 | timeoutSeconds: 10 240 | ``` 241 | 242 | The cluster can be established with simple `kubectl` commands: 243 | 244 | ```bash 245 | # Launch Redis Master/Slave Deployment and Service 246 | kubectl create -f k8s/redis 247 | # Launch Websocket Deployment and Service 248 | kubectl create -f k8s/wsk 249 | ``` 250 | 251 | ### Client Gotchas 252 | 253 | Dealing with this new architecture will lead to slight changes to client: 254 | 255 | * the endpoint should be tuned according to cluster configuration. In the case above with local cluster having a NodePort service, the endpoint will be `http://localhost:30000` 256 | 257 | * relying on the K8s orchestration means that pods can be transparently rescheduled. So a pod can be suddenly terminated and this can give a hard time to the client. 258 | However if we add `reconnection` policies to the client, as soon as the connection is lost, it will reconnect to the first available pod in the replicaset maintained by the deployment. 259 | 260 | Here is the new client configuration: 261 | 262 | ```Javascript 263 | const io = require('socket.io-client') 264 | const client = io('http://localhost:30000', { 265 | reconnection: true, 266 | reconnectionDelay: 500, 267 | transports: ['websocket'] 268 | }); 269 | ``` 270 | 271 | ## Conclusions 272 | 273 | In this part we started from scratch and we landed in a fully clustered architecture. In a following chapter we will see of we can achieve an even more sophisticated solution. 274 | 275 | >`Note`: This part will correspond in the repository to the `haproxy` branch 276 | 277 | ## References 278 | 279 | * [sw360cab/websockets-scaling: A tutorial to scale Websockets both via Docker Swarm and Kubernetes](https://github.com/sw360cab/websockets-scaling) 280 | * [Scaling Websockets Tutorial](https://hackernoon.com/scaling-websockets-9a31497af051) 281 | * [Socket.io - Using Multiple Nodes](https://socket.io/docs/using-multiple-nodes/) 282 | * [Scaling Node.js Socket Server with Nginx and Redis](https://blog.jscrambler.com/scaling-node-js-socket-server-with-nginx-and-redis/) 283 | * [Deploying a real time notification system on Kubernetes](https://medium.com/@faiyaz26/deploying-a-real-time-notification-system-on-kubernetes-part-2-1a28b4321dfc) 284 | -------------------------------------------------------------------------------- /BLOG2.md: -------------------------------------------------------------------------------- 1 | # Scaling Websockets in the Cloud (Part 2). Introducing Traefik: the all-in-one dynamic and flexible solution for Docker Stacks and Kubernetes clusters 2 | 3 | ## Recap 4 | 5 | In [Part 1](BLOG.md) we journeyed from a simple websocket application to a quite advanced Kubernetes configuration. 6 | The final architecture looks like the following diagram. 7 | 8 | | ![Clustered K8s Architecture](diagrams/clustered_k8s_nodeport.png) | 9 | | :--: | 10 | 11 | You may remember that one of the key point in the whole multi-server pattern was leveraging an element which acts as `Reverse Proxy`. 12 | In the case of Docker configuration we defined a service in *docker-compose*, whereas in Kubernetes it was easier to employ a simple *Service Resource*, in that case exposed as a `NodePort`. 13 | 14 | It would be nice to have a solution that is suitable for both cases maybe reusing some configurations. 15 | Moreover Kubernetes may require even more attention since resources can be rescheduled quite often, so the configuration should be flexible enough to keep up with this dynamic pattern. 16 | 17 | ## Traefik: one service addressing many issues 18 | 19 | > `Traefik` is an open-source *Edge Router* that makes publishing your services a fun and easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them. 20 | 21 | Traefik has a multipurpose structure, hence it can be employed as a standalone service but, thanks to the concepts of `Providers`, it works smoothly either in a fully `Docker` environment or as a `Kubernetes` resource, likely with many others third party. 22 | 23 | | ![Traefik K8s Basic](https://docs.traefik.io/assets/img/entrypoints.png) | 24 | | :--: | 25 | 26 | The above image explain the basic blocks of `Traefik`: 27 | 28 | * Entrypoints (network points of access to the infrastructure) 29 | * Routers (forwarders of requests to the corresponding service) 30 | * Middlewares (chainable modifiers of requests before they are forwarded or responses before they are forwarded back to requestors) 31 | * Services (abstract representations of the services that will effectively handle all the requests) 32 | * Providers (orchestrators, container engines, cloud providers, or key-value stores whose APIs Traefik is able to query to find relevant information about routing). 33 | 34 | >`Tip`: take a moment to understand basic concepts and different kind and blocks of configuration in `Traefik`, you will become immediately friends. Otherwise it may happen that it gives you hard time. That's part of the game, deal with it at the end you will cope and win together. 35 | 36 | Traefik is now at version 2.2. and it comes *batteries included*! Indeed *out-of-the-box* you can leverage its `Dashboard` that provides usage statistics but above all a visual overview of the setup configured. 37 | 38 | | ![Traefik Dashboard](https://docs.traefik.io/assets/img/webui-dashboard.png) | 39 | | :--: | 40 | 41 | This is particularly useful when you'll need to troubleshoot your configuration, since you may visually find which component is not correctly working or whether a dynamic configuration is not actively recognizing the expected element. 42 | 43 | ### Configuring Traefik 44 | 45 | Traefik can have either a `static` or a `dynamic` configuration. Most of the option blocks in the static configuration are available seamlessly in the dynamic one. 46 | 47 | In a static configuration the Traefik elements are defined at startup and they are not expect to change over time. 48 | The configuration can be defined either as: 49 | 50 | * Traefik CLI parameters (referred as `CLI` in doc), 51 | * Environment variables, 52 | * Text File (either in `yaml` or `toml` format). 53 | 54 | What is more interesting is `dynamic` configuration. This can change and is seamlessly hot-reloaded, without any request interruption or connection loss. Depending on the provider Traefik will listen and adapt to any change in configuration. 55 | 56 | Static and dynamic configurations can be mixed and work together without any problem. It is quite common to launch Traefik CLI with some options and then allow dynamic parts to complete the configuration. Indeed you will usually declare the provider (e.g. Kubernetes, Docker, File) in Traefik CLI and then leave all the rest of configuration as dynamic, using different patterns, according to each specific provider: 57 | 58 | * `Labels` for Docker provider 59 | * `CRD` or `Ingress` for Kubernetes provider 60 | * `TOML` or `YAML` format for plain File provider 61 | 62 | Going back to what explained at the beginning, this philosopy is particularly useful: 63 | 64 | * it adapts to a K8s cluster where resources can be quickly scheduled or terminated, 65 | * it is reusable and adaptable, since the same configuration elements (entrypoints, routers, middlewares) are present either if we are working in Docker or in Kubernetes. 66 | 67 | >`Tip`: Traefik has a vast and deep documentation. Many examples are included, most of the time they are referred with multiple providers variant. Sometimes only in the `File Provider` fashion. Since you will likely employ another provider, don't be scared, the transition from File provider to others is simple enough. 68 | 69 | ### TLS & HTTPS 70 | 71 | One of the key feature of Traefik is its ability to natively handle `HTTPS` traffic and `TLS` connections. Traefik can handle very easily HTTP to HTTPS redirection and TLS certificates support. The latter include a very powerful management of certificates through [Let's Encrypt (ACME)](https://docs.traefik.io/https/acme/), which is able to generate and renew ACME certificates by itself. 72 | 73 | This part is out of the scope of this post but it is a fundamental feature when you will deploy Traefik in production. 74 | 75 | ## Traefik in Docker 76 | 77 | Traefik can be defined as a single service in a *docker-compose* file. 78 | 79 | Interested services will be dynamically configured and reachable using the concept of [Docker Labels](https://docs.docker.com/compose/compose-file/#labels). This list of labels will keep all the definitions allowing Traefik to discover the involved services. 80 | 81 | >`Tip`: If any sort of troubleshooting is needed enable access logging and debug level detail by adding these options to Traefik CLI service: 82 | 83 | ```yaml 84 | - "--accesslog" 85 | - "--log.level=DEBUG" 86 | ``` 87 | 88 | ### Docker Stack in action 89 | 90 | The new stack will swap *HAProxy* service (or any other Reverse Proxy) with a Traefik container, whose CLI will declare any static configuration part. 91 | 92 | ```yaml 93 | traefik-reverse-proxy: 94 | image: traefik:v2.2 95 | command: 96 | - "--api.insecure=true" 97 | - "--accesslog" 98 | ports: 99 | - "80:80" 100 | # The Web UI (enabled by --api.insecure=true) 101 | - "8080:8080" 102 | volumes: 103 | # allow Traefik to listen to the Docker events 104 | - /var/run/docker.sock:/var/run/docker.sock 105 | ``` 106 | 107 | As you see the key part is mapping `Docker Daemon` entry point for Docker APIs into Traefik, hence it will be able to listen to any configuration change in the stack. 108 | 109 | All the necessary configuration parts will be dynamic and left to the `Docker Provider`. They will placed in the websocket application service as `label` items of the service configuration: 110 | 111 | ```yaml 112 | socket-server: 113 | image: sw360cab/wsk-base:0.1.1 114 | restart: always 115 | deploy: 116 | mode: replicated 117 | replicas: 2 118 | environment: 119 | - "REDIS_HOST=redis" 120 | labels: 121 | - "traefik.http.routers.socket-router.rule=PathPrefix(`/wsk`)" 122 | - "traefik.http.services.service01.loadbalancer.server.port=5000" 123 | - "traefik.http.services.service01.loadbalancer.sticky.cookie=true" 124 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.name=io" 125 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.httponly=true" 126 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.secure=true" 127 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.samesite=io" 128 | - "traefik.http.middlewares.socket-replaceprefix.replacepath.path=/" 129 | - "traefik.http.routers.socket-router.middlewares=socket-replaceprefix@docker" 130 | ``` 131 | 132 | >Note: in this configuration I added another step of complexity. We expect the websocket application to be exposed at the `/wsk` path instead of root path. This will allow us to see the `middleware` components of Traefik in action. 133 | 134 | These *labels* define: 135 | 136 | * a router and its rules, which handles the requests to the `PathPrefix` */wsk* 137 | * a service, which references the port exposed by websocket application service (*5000*), and addresses the issue of retrieving `sticky` sessions 138 | * a middleware, which will translate all the requests received at the `/wsk` path, into what the websocket service is expecting: `/`. 139 | 140 | In this line: 141 | 142 | ```yaml 143 | - "traefik.http.routers.socket-router.middlewares=socket-replaceprefix@docker" 144 | ``` 145 | 146 | The middleware defined is dynamically associated with the router by a list of formatted strings with the convention `@`. 147 | 148 | >`Note`: the names of router (`socket-router`), service (`service01`) and middleware (`socket-replaceprefix`) do not follow any convention and are absolutely up to you. 149 | 150 | At this point you just need to launch the stack again, with: 151 | 152 | ```bash 153 | docker stack deploy --compose-file stack/docker-compose.yml wsk 154 | ``` 155 | 156 | Then check out the Traefik Dashboard exposed by default at and if no errors are shown, get your websocket client ready to connect. Don't forget to tune its configuration: now to reach the websocket service, the client will contact directly Traefik, which in turn is now exposing the service at path `/wsk`. 157 | 158 | ```Javascript 159 | const io = require('socket.io-client'); 160 | const client = io('http://localhost:80', { 161 | path: '/wsk' 162 | transports: ['websocket'] 163 | }); 164 | ``` 165 | 166 | And the magic should happen! The communication among clients and websocket services keeps on working smoothly! 167 | 168 | ## Traefik in Kubernetes 169 | 170 | Dynamic configuration of Traefik in Kubernetes may act as an `Ingress` resource: an entrypoint to a specific service, that can be bound to a domain name and port. In version 2.2 Traefik improved its terminology and added the `KubernetesCRD` Provider, hence the state of the art is actually defining an `Ingress Route` through a Kubernetes `CRD` (Custom Resource Definition). 171 | 172 | Here is the new architecture introducing `Traefik` as first point of access into the cluster: 173 | 174 | | ![Clustered K8s Architecture with Traefik](diagrams/clustered_k8s_traefik.png) | 175 | | :--: | 176 | 177 | In this environment Traefik requires the definition of the following resources: 178 | 179 | * CRD 180 | * RBAC (Role-Based Access Control) defining these items: 181 | * ServiceAccount 182 | * ClusterRole (with access to specific api groups in K8s) 183 | * ClusterRoleBinding (binding the previous two items) 184 | * Deployment (Traefik Pod definition) 185 | 186 | ```yaml 187 | serviceAccountName: traefik-ingress-controller 188 | containers: 189 | - name: traefik 190 | image: traefik:v2.2 191 | args: 192 | - --api.insecure 193 | - --accesslog 194 | - --entrypoints.web.address=:80 195 | - --entrypoints.websecure.address=:443 196 | - --providers.kubernetescr 197 | ``` 198 | 199 | * Service (exposing the entrypoint and the Dashboard to the external world) 200 | * the `IngressRoute`. The core of our configuration, specifying the router rules, the service to be bound, the entrypoint of reference and eventually extra-rules for middlewares. 201 | 202 | ```yaml 203 | kind: IngressRoute 204 | apiVersion: traefik.containo.us/v1alpha1 205 | metadata: 206 | name: simpleingressroute 207 | namespace: default 208 | spec: 209 | entryPoints: 210 | - web 211 | routes: 212 | - match: PathPrefix(`/`) 213 | kind: Rule 214 | services: 215 | - name: wsk-svc 216 | port: 3000 217 | ``` 218 | 219 | The previous configuration define an IngressRoute related to the entrypoint `web` (which was defined above in Traefik deployment), and a rule which match everything having that `PathPrefix` to a specific Kubernetes service resource. 220 | 221 | >`Tip`: whether your service will have access from a predefined domain name, the router *match* rules will be similar to: 222 | 223 | ```yaml 224 | - match: Host(`your.example.com`) && PathPrefix(`/`) 225 | ``` 226 | 227 | ### Replacing NodePort for default service and run 228 | 229 | In the first Kubernetes configuration we used to expose the websocket service as `NodePort`. Introducing Traefik we don't need to expose the previous service anymore, since the task of exposing services outside the cluster will be in charge to Traefik service itself. 230 | 231 | The `NodePort` part should be removed from websocket service resource. The service won't need to be exposed in any way, it will simply look like that: 232 | 233 | ```yaml 234 | apiVersion: v1 235 | kind: Service 236 | metadata: 237 | name: wsk-svc 238 | spec: 239 | selector: 240 | app: wsk-base 241 | ports: 242 | - protocol: TCP 243 | port: 3000 244 | targetPort: 5000 245 | ``` 246 | 247 | Then we will transfer the access to our cluster to Traefik service. For sake of simplicity I reused a `NodePort` form: 248 | 249 | ```yaml 250 | apiVersion: v1 251 | kind: Service 252 | metadata: 253 | name: traefik 254 | spec: 255 | selector: 256 | app: traefik 257 | ports: 258 | - name: web 259 | protocol: TCP 260 | port: 80 261 | targetPort: 80 262 | nodePort: 30080 263 | - name: admin 264 | protocol: TCP 265 | port: 8080 266 | targetPort: 8080 267 | nodePort: 30808 268 | type: NodePort 269 | ``` 270 | 271 | The configuration above will expose Traefik `web` entrypoint on port **30080** and Dashboard on port **30808**. 272 | When a client send a request through port 30080 (supposing on localhost), the latter will be forwarded to the entrypoint, then through the `IngressRoute` custom resource, which in turn, according to `router` rules, will allow the request to be received by the corresponding service. Finally the service response will be forwarded back through the same path and back to the client outside the cluster. 273 | 274 | Now we are ready to launch K8s new resources. First off all let's define the new CRD, then we launch Traefik resources and finally we may redeploy websocket service (supposing it has been already scheduled). 275 | 276 | ```bash 277 | # Launch Traefik CRD 278 | kubectl create -f k8s/traefik/crd.yaml 279 | # Lauch Traefik RBAC,deployment/service,ingressRoute 280 | kubectl create -f k8s/traefik/rbac.yaml,k8s/traefik/traefik.yaml,k8s/traefik/ingress.yaml 281 | # Reschedule websocket Service 282 | kubectl apply -f k8s/wsk/wsk-service.yaml 283 | ``` 284 | 285 | ### A Traefik Middleware 286 | 287 | `Middlewares` are a key concept in Traefik as they add very powerful manipulation possibilities to incoming requests. There is a long list of natively available ones in the [official doc](https://docs.traefik.io/middlewares/overview/). 288 | 289 | As for the Docker configuration also for Kubernetes we may expose our websocket service at path `/wsk`. Respect to the previous configuration, this shift requires only a slight change by: 290 | 291 | 1. introducing a `StripPrefix` Middleware resource (using another Traefik CRD), 292 | 2. adding the previous middleware reference to the list of middlewares in the Router within the IngressRoute configuration. 293 | 294 | The final result is the following: 295 | 296 | ```yaml 297 | kind: Middleware 298 | apiVersion: traefik.containo.us/v1alpha1 299 | metadata: 300 | name: socket-replaceprefix 301 | spec: 302 | replacePath: 303 | path: / 304 | --- 305 | kind: IngressRoute 306 | apiVersion: traefik.containo.us/v1alpha1 307 | metadata: 308 | name: simpleingressroute 309 | namespace: default 310 | spec: 311 | entryPoints: 312 | - web 313 | routes: 314 | - match: PathPrefix(`/wsk`) 315 | kind: Rule 316 | services: 317 | - name: wsk-svc 318 | port: 3000 319 | middlewares: 320 | - name: socket-replaceprefix 321 | ``` 322 | 323 | As you may notice the previous steps are basically the same required while we performed this exact change in Docker configuration above. 324 | 325 | ## Path Prefix Gotchas 326 | 327 | `Socket.io` by default will expose the default websocket endpoint adding a `/socket.io` suffix at the end. This is quite strange when you start reaching your websocket in a complex environment. 328 | 329 | The solution is fixing the path of the websocket in the application. 330 | 331 | ```Javascript 332 | const socketServer = require('http').createServer(); 333 | const io = require('socket.io')(socketServer, { 334 | path: '/' 335 | }) 336 | ``` 337 | 338 | ## Conclusions 339 | 340 | In this part we introduced `Traefik` which is an impressive tool. It is powerful and straightforward to configure. 341 | However you know that `from great power comes great responsibility`, so my suggestion is to: 342 | 343 | * understand Traefik basic concepts, 344 | * have clear in mind what is the final result you expect. 345 | 346 | Otherwise you will end up trying again and again to adjust your configuration, which will keep not working (that comes from a true story...mine). 347 | 348 | If you have clear where you want to go, Traefik will be a good friend. As you may have noticed it solved both the use cases in Docker and Kubernetes in one-shot. Amazing! 349 | 350 | ## References 351 | 352 | * [sw360cab/websockets-scaling: A tutorial to scale Websockets both via Docker Swarm and Kubernetes](https://github.com/sw360cab/websockets-scaling) 353 | * [Concepts - Traefik](https://docs.traefik.io/getting-started/concepts/ "Concepts - Traefik") 354 | * [Docker - Traefik](https://docs.traefik.io/providers/docker/ "Docker - Traefik") 355 | * [Docker Configuration Reference - Traefik](https://docs.traefik.io/reference/dynamic-configuration/docker/ "Docker Configuration Reference - Traefik") 356 | * [Kubernetes IngressRoute - Traefik](https://docs.traefik.io/routing/providers/kubernetes-crd/ "Kubernetes IngressRoute - Traefik") 357 | * [Middlewares - Traefik](https://docs.traefik.io/middlewares/overview/ "Middlewares - Traefik") 358 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Sergio Maria Matone 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Scaling websockets 2 | 3 | ## Intro 4 | 5 | Purpose of this is achieving a scalable environment when WebSocket are involved. 6 | The pattern to reproduce is a Multi-Server to Multi-client communication via websocket. 7 | 8 | The result should be a client connected to a specific server host and keeping its connection bound to it (`sticky` connection). 9 | Whereas the server hosts will broadcast messages in turn and all the connected clients should receive them. 10 | The latter will be achieved leveraging the pub/sub paradigm of `Redis`. 11 | 12 | The application is made of server and client part. Both are based `socket.io` library. 13 | 14 | ### Server 15 | 16 | * it employs a dedicated adapter implementing native Redis support. 17 | * an instance will advertise periodically its private ip. **NOTE that this message should be received by ALL the clients connected.** 18 | 19 | ### Client 20 | 21 | * it is configured to employ native websockets, without attempting to use naïfes solutions as http polling. 22 | 23 | { transports: ['websocket'] } 24 | 25 | * it will send a message to a single server instance at connection time. **NOTE that this message should be received only by a single server instance.** 26 | 27 | ## Stack of Microservices Scenario 28 | 29 | See [Stack](stack/README.md) configuration. 30 | 31 | ## Kubernates Scenario 32 | 33 | See [Kubernates](k8s/README.md) configuration. 34 | 35 | ## References 36 | 37 | * [Scaling Websockets Tutorial](https://hackernoon.com/scaling-websockets-9a31497af051) 38 | * [Socket.io - Using Multiple Nodes](https://socket.io/docs/using-multiple-nodes/) 39 | * [Scaling Node.js Socket Server with Nginx and Redis](https://blog.jscrambler.com/scaling-node-js-socket-server-with-nginx-and-redis/) 40 | * [Deploying a real time notification system on Kubernetes](https://medium.com/@faiyaz26/deploying-a-real-time-notification-system-on-kubernetes-part-2-1a28b4321dfc) 41 | -------------------------------------------------------------------------------- /diagrams/clustered_k8s_nodeport.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sw360cab/websockets-scaling/51e65e2d85702b285bccde080ec90b84ef595bf0/diagrams/clustered_k8s_nodeport.png -------------------------------------------------------------------------------- /diagrams/clustered_k8s_traefik.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sw360cab/websockets-scaling/51e65e2d85702b285bccde080ec90b84ef595bf0/diagrams/clustered_k8s_traefik.png -------------------------------------------------------------------------------- /diagrams/end_to_end_singleserver.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sw360cab/websockets-scaling/51e65e2d85702b285bccde080ec90b84ef595bf0/diagrams/end_to_end_singleserver.png -------------------------------------------------------------------------------- /diagrams/multi_server_proxied.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sw360cab/websockets-scaling/51e65e2d85702b285bccde080ec90b84ef595bf0/diagrams/multi_server_proxied.png -------------------------------------------------------------------------------- /k8s/README.md: -------------------------------------------------------------------------------- 1 | # K8S version 2 | 3 | This configuration will use a service interfacing a deployment with 3 replicas of socket-server application. 4 | The back part of Redis is handled by a classic Redis Master/Slave configuration. 5 | 6 | ## Websocket Deployment/Service (folder wsk/) 7 | 8 | The service is packed and exposed from static version via a Docker image deployed on private registry. 9 | The exposed port of the container / pod is 5000. The service can be exposed by a Traefik ingress or via a `NodePort` on port 30000. 10 | 11 | The key point of the service is `sessionAffinity: ClientIP` which will mimic Sticky connection for the service. 12 | 13 | * Note: In general Round Robin among PODs is not guaranteed using a simple solution like NodePort. In order to mimic Round Robin policy, this configuration should be added: 14 | 15 | sessionAffinityConfig: 16 | clientIP: 17 | timeoutSeconds: 10 18 | 19 | ## Redis Deployment/Service (folder redis/) 20 | 21 | A Master-Slave K8S for Redis solution that is maintaining in-sync websockets through Pub/Sub, using the endpoint: 22 | 23 | * **redis-master.default.svc.cluster.local`** 24 | 25 | ## Traefik Ingress Route (folder traefik/) 26 | 27 | A Traefik `Ingress Route` can be used to proxy request towards websocket service. 28 | 29 | ## Setup and Run 30 | 31 | ### microservices 32 | 33 | # Launch Redis Deployment and Services 34 | cd redis/ 35 | kubectl create -f redis-master-deployment.yaml, redis-slave-deployment.yaml 36 | kubectl create -f redis-master-service.yaml, redis-slave-service.yaml 37 | 38 | # Launch Websocket Deployment and Services 39 | cd ../wsk 40 | kubectl create -f wsk-deployment.yaml 41 | kubectl create -f wsk-service.yaml 42 | 43 | # Launch Traefik CRD, RBAC and Ingress 44 | cd ../traefik 45 | kubectl create -f crd.yaml 46 | kubectl create -f rbac.yaml,traefik.yaml,ingress.yaml 47 | 48 | ### client 49 | 50 | The client configuration is different depending on cluster setup: 51 | 52 | * `NodePort Service`: ** 53 | * `Traefik Ingress`: ** (also path is `wsk/`) 54 | 55 | For any new client open a new terminal and run: 56 | 57 | node ../stack/client_socket.js 58 | -------------------------------------------------------------------------------- /k8s/redis/redis-master-deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: redis-master 5 | labels: 6 | app: redis 7 | spec: 8 | selector: 9 | matchLabels: 10 | app: redis 11 | role: master 12 | replicas: 1 13 | template: 14 | metadata: 15 | labels: 16 | app: redis 17 | role: master 18 | spec: 19 | containers: 20 | - name: master 21 | image: k8s.gcr.io/redis 22 | resources: 23 | requests: 24 | cpu: 100m 25 | memory: 100Mi 26 | ports: 27 | - containerPort: 6379 28 | -------------------------------------------------------------------------------- /k8s/redis/redis-master-service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: redis-master 5 | labels: 6 | app: redis 7 | role: master 8 | spec: 9 | ports: 10 | - port: 6379 11 | targetPort: 6379 12 | selector: 13 | app: redis 14 | role: master -------------------------------------------------------------------------------- /k8s/redis/redis-slave-deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: redis-slave 5 | labels: 6 | app: redis 7 | spec: 8 | selector: 9 | matchLabels: 10 | app: redis 11 | role: slave 12 | replicas: 3 13 | template: 14 | metadata: 15 | labels: 16 | app: redis 17 | role: slave 18 | spec: 19 | containers: 20 | - name: slave 21 | image: k8s.gcr.io/redis-slave:v2 22 | resources: 23 | requests: 24 | cpu: 100m 25 | memory: 100Mi 26 | env: 27 | - name: GET_HOSTS_FROM 28 | value: dns 29 | # Using `GET_HOSTS_FROM=dns` requires your cluster to 30 | # provide a dns service. As of Kubernetes 1.3, DNS is a built-in 31 | # service launched automatically. However, if the cluster you are using 32 | # does not have a built-in DNS service, you can instead 33 | # access an environment variable to find the master 34 | # service's host. To do so, comment out the 'value: dns' line above, and 35 | # uncomment the line below: 36 | # value: env 37 | ports: 38 | - containerPort: 6379 39 | -------------------------------------------------------------------------------- /k8s/redis/redis-slave-service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: redis-slave 5 | labels: 6 | app: redis 7 | role: slave 8 | spec: 9 | ports: 10 | - port: 6379 11 | targetPort: 6379 12 | selector: 13 | app: redis 14 | role: slave -------------------------------------------------------------------------------- /k8s/traefik/crd.yaml: -------------------------------------------------------------------------------- 1 | kind: CustomResourceDefinition 2 | apiVersion: apiextensions.k8s.io/v1beta1 3 | metadata: 4 | name: ingressroutes.traefik.containo.us 5 | spec: 6 | group: traefik.containo.us 7 | version: v1alpha1 8 | names: 9 | kind: IngressRoute 10 | plural: ingressroutes 11 | singular: ingressroute 12 | scope: Namespaced 13 | 14 | --- 15 | kind: CustomResourceDefinition 16 | apiVersion: apiextensions.k8s.io/v1beta1 17 | metadata: 18 | name: middlewares.traefik.containo.us 19 | spec: 20 | group: traefik.containo.us 21 | version: v1alpha1 22 | names: 23 | kind: Middleware 24 | plural: middlewares 25 | singular: middleware 26 | scope: Namespaced 27 | 28 | --- 29 | kind: CustomResourceDefinition 30 | apiVersion: apiextensions.k8s.io/v1beta1 31 | metadata: 32 | name: ingressroutetcps.traefik.containo.us 33 | spec: 34 | group: traefik.containo.us 35 | version: v1alpha1 36 | names: 37 | kind: IngressRouteTCP 38 | plural: ingressroutetcps 39 | singular: ingressroutetcp 40 | scope: Namespaced 41 | 42 | --- 43 | kind: CustomResourceDefinition 44 | apiVersion: apiextensions.k8s.io/v1beta1 45 | metadata: 46 | name: ingressrouteudps.traefik.containo.us 47 | spec: 48 | group: traefik.containo.us 49 | version: v1alpha1 50 | names: 51 | kind: IngressRouteUDP 52 | plural: ingressrouteudps 53 | singular: ingressrouteudp 54 | scope: Namespaced 55 | 56 | --- 57 | kind: CustomResourceDefinition 58 | apiVersion: apiextensions.k8s.io/v1beta1 59 | metadata: 60 | name: tlsoptions.traefik.containo.us 61 | spec: 62 | group: traefik.containo.us 63 | version: v1alpha1 64 | names: 65 | kind: TLSOption 66 | plural: tlsoptions 67 | singular: tlsoption 68 | scope: Namespaced 69 | 70 | --- 71 | kind: CustomResourceDefinition 72 | apiVersion: apiextensions.k8s.io/v1beta1 73 | metadata: 74 | name: tlsstores.traefik.containo.us 75 | spec: 76 | group: traefik.containo.us 77 | version: v1alpha1 78 | names: 79 | kind: TLSStore 80 | plural: tlsstores 81 | singular: tlsstore 82 | scope: Namespaced 83 | 84 | --- 85 | kind: CustomResourceDefinition 86 | apiVersion: apiextensions.k8s.io/v1beta1 87 | metadata: 88 | name: traefikservices.traefik.containo.us 89 | spec: 90 | group: traefik.containo.us 91 | version: v1alpha1 92 | names: 93 | kind: TraefikService 94 | plural: traefikservices 95 | singular: traefikservice 96 | scope: Namespaced 97 | -------------------------------------------------------------------------------- /k8s/traefik/ingress.yaml: -------------------------------------------------------------------------------- 1 | kind: Middleware 2 | apiVersion: traefik.containo.us/v1alpha1 3 | metadata: 4 | name: socket-replaceprefix 5 | spec: 6 | replacePath: 7 | path: / 8 | --- 9 | kind: IngressRoute 10 | apiVersion: traefik.containo.us/v1alpha1 11 | metadata: 12 | name: simpleingressroute 13 | namespace: default 14 | spec: 15 | entryPoints: 16 | - web 17 | routes: 18 | # - match: Host(`your.example.com`) && PathPrefix(`/notls`) 19 | - match: PathPrefix(`/wsk`) 20 | kind: Rule 21 | services: 22 | - name: wsk-svc 23 | port: 3000 24 | sticky: 25 | cookie: 26 | httpOnly: true 27 | name: io 28 | secure: true 29 | sameSite: lax 30 | strategy: RoundRobin 31 | middlewares: 32 | - name: socket-replaceprefix 33 | -------------------------------------------------------------------------------- /k8s/traefik/rbac.yaml: -------------------------------------------------------------------------------- 1 | kind: ServiceAccount 2 | apiVersion: v1 3 | metadata: 4 | name: traefik-ingress-controller 5 | --- 6 | kind: ClusterRole 7 | apiVersion: rbac.authorization.k8s.io/v1beta1 8 | metadata: 9 | name: traefik-ingress-controller 10 | rules: 11 | - apiGroups: 12 | - "" 13 | resources: 14 | - services 15 | - endpoints 16 | - secrets 17 | verbs: 18 | - get 19 | - list 20 | - watch 21 | - apiGroups: 22 | - extensions 23 | - networking.k8s.io 24 | resources: 25 | - ingresses 26 | verbs: 27 | - get 28 | - list 29 | - watch 30 | - apiGroups: 31 | - extensions 32 | - networking.k8s.io 33 | resources: 34 | - ingresses/status 35 | verbs: 36 | - update 37 | - apiGroups: 38 | - traefik.containo.us 39 | resources: 40 | - middlewares 41 | - ingressroutes 42 | - traefikservices 43 | - ingressroutetcps 44 | - ingressrouteudps 45 | - tlsoptions 46 | - tlsstores 47 | verbs: 48 | - get 49 | - list 50 | - watch 51 | --- 52 | kind: ClusterRoleBinding 53 | apiVersion: rbac.authorization.k8s.io/v1beta1 54 | metadata: 55 | name: traefik-ingress-controller 56 | roleRef: 57 | apiGroup: rbac.authorization.k8s.io 58 | kind: ClusterRole 59 | name: traefik-ingress-controller 60 | subjects: 61 | - kind: ServiceAccount 62 | name: traefik-ingress-controller 63 | namespace: default 64 | -------------------------------------------------------------------------------- /k8s/traefik/traefik.yaml: -------------------------------------------------------------------------------- 1 | kind: Deployment 2 | apiVersion: apps/v1 3 | metadata: 4 | name: traefik 5 | labels: 6 | app: traefik 7 | spec: 8 | replicas: 1 9 | selector: 10 | matchLabels: 11 | app: traefik 12 | template: 13 | metadata: 14 | labels: 15 | app: traefik 16 | spec: 17 | serviceAccountName: traefik-ingress-controller 18 | containers: 19 | - name: traefik 20 | image: traefik:v2.2 21 | args: 22 | - --api.insecure 23 | - --accesslog 24 | # - --log.level=DEBUG 25 | - --entrypoints.web.address=:80 26 | - --entrypoints.websecure.address=:443 27 | - --providers.kubernetescrd 28 | - --certificatesresolvers.myresolver.acme.tlschallenge 29 | - --certificatesresolvers.myresolver.acme.email=foo@you.com 30 | - --certificatesresolvers.myresolver.acme.storage=acme.json 31 | # Please note that this is the staging Let's Encrypt server. 32 | # Once you get things working, you should remove that whole line altogether. 33 | - --certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory 34 | ports: 35 | - name: web 36 | containerPort: 80 37 | - name: websecure 38 | containerPort: 4443 39 | - name: admin 40 | containerPort: 8080 41 | --- 42 | apiVersion: v1 43 | kind: Service 44 | metadata: 45 | name: traefik 46 | spec: 47 | selector: 48 | app: traefik 49 | ports: 50 | - name: web 51 | protocol: TCP 52 | port: 80 53 | targetPort: 80 54 | nodePort: 30080 55 | - name: admin 56 | protocol: TCP 57 | port: 8080 58 | targetPort: 8080 59 | nodePort: 30808 60 | - name: webtls 61 | protocol: TCP 62 | port: 443 63 | targetPort: 443 64 | nodePort: 30443 65 | type: NodePort 66 | -------------------------------------------------------------------------------- /k8s/wsk/wsk-deployment.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: wsk-deploy 5 | spec: 6 | replicas: 3 7 | selector: 8 | matchLabels: 9 | app: wsk-base 10 | template: 11 | metadata: 12 | labels: 13 | app: wsk-base 14 | track: stable 15 | version: 0.1.1 16 | spec: 17 | containers: 18 | - name: websocket-base 19 | image: "sw360cab/wsk-base:0.1.1" 20 | imagePullPolicy: IfNotPresent 21 | env: 22 | - name: REDIS_HOST 23 | value: "redis-master.default.svc" 24 | ports: 25 | - name: websocket 26 | containerPort: 5000 27 | -------------------------------------------------------------------------------- /k8s/wsk/wsk-service.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: v1 2 | kind: Service 3 | metadata: 4 | name: wsk-svc 5 | spec: 6 | selector: 7 | app: wsk-base 8 | sessionAffinity: ClientIP 9 | sessionAffinityConfig: 10 | clientIP: 11 | timeoutSeconds: 10 12 | ports: 13 | - protocol: TCP 14 | port: 3000 15 | targetPort: 5000 16 | # nodePort: 30000 17 | # type: NodePort 18 | -------------------------------------------------------------------------------- /stack/.dockerignore: -------------------------------------------------------------------------------- 1 | node_modules/ 2 | Dockerfile 3 | docker-compose* 4 | README.md 5 | *.cfg 6 | client* 7 | -------------------------------------------------------------------------------- /stack/Dockerfile: -------------------------------------------------------------------------------- 1 | # The instructions for the first stage 2 | FROM node:13-alpine as node-compiled 3 | 4 | ARG NODE_ENV=prod 5 | ENV NODE_ENV=${NODE_ENV} 6 | 7 | # dependecies for npm compiling 8 | RUN apk --no-cache add python make g++ 9 | 10 | COPY package*.json ./ 11 | RUN npm install 12 | 13 | # The instructions for second stage 14 | FROM node:13-alpine 15 | 16 | WORKDIR /usr/src/app 17 | COPY . . 18 | COPY --from=node-compiled node_modules node_modules 19 | 20 | # EXPOSE 5000 21 | 22 | CMD [ "npm", "run", "start" ] -------------------------------------------------------------------------------- /stack/README.md: -------------------------------------------------------------------------------- 1 | # Static Microservices Scenario 2 | 3 | This leverages **docker-compose** and deploys the following services: 4 | 5 | * redis (pub/sub) 6 | * socket-server application (2 instances) 7 | * Traefik ad reverse proxy (with dynamic configuration) 8 | 9 | ## Traefik 10 | 11 | Traefik is employed as proxy for Websockets connections. 12 | It dynamically configures itself leveraging `Docker Labels` on websocket service image 13 | 14 | ## Setup and Run 15 | 16 | ### Microservices 17 | 18 | docker-compose up -d 19 | # scaling websocket service 20 | docker-compose up -d --scale socket-server=2 socket-server 21 | 22 | ### Swarm Mode 23 | 24 | docker stack deploy --compose-file docker-compose.yml wsk 25 | 26 | ### client 27 | 28 | The client endpoint is ``, path is `wsk/`. 29 | 30 | For any new client open a new terminale and run: 31 | 32 | node client_socket.js 33 | -------------------------------------------------------------------------------- /stack/client_socket.js: -------------------------------------------------------------------------------- 1 | const uuid = require('uuid'); 2 | const io = require('socket.io-client'); 3 | const client = io('http://localhost:30080', { 4 | // port: 5000 - when using direct connection 5 | // port: 80 - when using Haproxy or Treafik Reverse Proxy 6 | // port: 30000 - when using plain k8s with NodePort 7 | // port: 30080, when using Traefik Ingress in k8s 8 | path: '/wsk', // when using Traefik Ingress 9 | // path: '/', // for all other scenarios 10 | reconnection: true, 11 | reconnectionDelay: 500, 12 | transports: ['websocket'] 13 | }); 14 | 15 | const clientId = uuid.v4(); 16 | let disconnectTimer; 17 | 18 | client.on('connect', function(){ 19 | console.log("Connected!", clientId); 20 | setTimeout(function() { 21 | console.log('Sending first message'); 22 | client.emit('test000', clientId); 23 | }, 500); 24 | // clear disconnection timeout 25 | clearTimeout(disconnectTimer); 26 | }); 27 | 28 | client.on('okok', function(message) { 29 | console.log('The server has a message for you:', message); 30 | }) 31 | 32 | client.on('disconnect', function(){ 33 | console.log("Disconnected!"); 34 | disconnectTimer = setTimeout(function() { 35 | console.log('Not reconnecting in 30s. Exiting...'); 36 | process.exit(0); 37 | }, 10000); 38 | }); 39 | 40 | client.on('error', function(err){ 41 | console.error(err); 42 | process.exit(1); 43 | }); 44 | 45 | setInterval(function() { 46 | console.log('Sending repeated message'); 47 | client.emit('test000', clientId); 48 | }, 5000); -------------------------------------------------------------------------------- /stack/docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '3.2' 2 | services: 3 | socket-server: 4 | # build: . 5 | image: sw360cab/wsk-base:0.1.1 6 | restart: always 7 | deploy: 8 | mode: replicated 9 | replicas: 2 10 | environment: 11 | - "REDIS_HOST=redis" 12 | labels: 13 | - "traefik.enable=true" 14 | - "traefik.http.routers.socket-router.rule=PathPrefix(`/wsk`)" 15 | - "traefik.http.services.service01.loadbalancer.server.port=5000" 16 | - "traefik.http.services.service01.loadbalancer.sticky.cookie=true" 17 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.name=io" 18 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.httponly=true" 19 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.secure=true" 20 | - "traefik.http.services.service01.loadbalancer.sticky.cookie.samesite=lax" 21 | # Replace prefix /wsk 22 | - "traefik.http.middlewares.socket-replaceprefix.replacepath.path=/" 23 | # Apply the middleware named `socket-replaceprefix` to the router named `scoker-router` 24 | - "traefik.http.routers.socket-router.middlewares=socket-replaceprefix@docker" 25 | 26 | traefik-reverse-proxy: 27 | image: traefik:v2.2 28 | command: 29 | - "--api.insecure=true" 30 | - "--providers.docker.exposedByDefault=false" 31 | - "--accesslog" 32 | # - "--entryPoints.web.address=:80" 33 | # - "--entryPoints.web.forwardedHeaders.insecure=true" 34 | ports: 35 | - "80:80" 36 | # The Web UI (enabled by --api.insecure=true) 37 | - "8080:8080" 38 | volumes: 39 | # allow Traefik to listen to the Docker events 40 | - /var/run/docker.sock:/var/run/docker.sock 41 | 42 | redis: 43 | image: redis:5.0 44 | -------------------------------------------------------------------------------- /stack/haproxy.cfg: -------------------------------------------------------------------------------- 1 | global 2 | log /dev/log local0 3 | #log /dev/log local1 notice 4 | #user haproxy 5 | #group haproxy 6 | daemon 7 | 8 | defaults 9 | mode http 10 | log global 11 | option httplog 12 | option http-server-close 13 | option dontlognull 14 | option redispatch 15 | option contstats 16 | retries 3 17 | backlog 10000 18 | timeout client 25s 19 | timeout connect 5s 20 | timeout server 25s 21 | timeout tunnel 3600s 22 | timeout http-keep-alive 1s 23 | timeout http-request 15s 24 | timeout queue 30s 25 | timeout tarpit 60s 26 | option forwardfor 27 | 28 | frontend ft_socket 29 | bind *:80 30 | default_backend bk_socket 31 | 32 | backend bk_socket 33 | http-check expect status 200 34 | cookie io prefix indirect nocache # using the `io` cookie set upon handshake 35 | server ws0 socket-server:5000 check cookie ws0 36 | # Uncomment following line for non-Swarm usage 37 | #server ws1 socket-server2:5000 check cookie ws1 38 | -------------------------------------------------------------------------------- /stack/package-lock.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "static-websocker-scaler", 3 | "version": "0.1.1", 4 | "lockfileVersion": 1, 5 | "requires": true, 6 | "dependencies": { 7 | "accepts": { 8 | "version": "1.3.8", 9 | "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz", 10 | "integrity": "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw==", 11 | "requires": { 12 | "mime-types": "~2.1.34", 13 | "negotiator": "0.6.3" 14 | } 15 | }, 16 | "after": { 17 | "version": "0.8.2", 18 | "resolved": "https://registry.npmjs.org/after/-/after-0.8.2.tgz", 19 | "integrity": "sha1-/ts5T58OAqqXaOcCvaI7UF+ufh8=" 20 | }, 21 | "arraybuffer.slice": { 22 | "version": "0.0.7", 23 | "resolved": "https://registry.npmjs.org/arraybuffer.slice/-/arraybuffer.slice-0.0.7.tgz", 24 | "integrity": "sha512-wGUIVQXuehL5TCqQun8OW81jGzAWycqzFF8lFp+GOM5BXLYj3bKNsYC4daB7n6XjCqxQA/qgTJ+8ANR3acjrog==" 25 | }, 26 | "async-limiter": { 27 | "version": "1.0.0", 28 | "resolved": "https://registry.npmjs.org/async-limiter/-/async-limiter-1.0.0.tgz", 29 | "integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg==" 30 | }, 31 | "backo2": { 32 | "version": "1.0.2", 33 | "resolved": "https://registry.npmjs.org/backo2/-/backo2-1.0.2.tgz", 34 | "integrity": "sha1-MasayLEpNjRj41s+u2n038+6eUc=" 35 | }, 36 | "base64-arraybuffer": { 37 | "version": "0.1.5", 38 | "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.5.tgz", 39 | "integrity": "sha1-c5JncZI7Whl0etZmqlzUv5xunOg=" 40 | }, 41 | "base64id": { 42 | "version": "2.0.0", 43 | "resolved": "https://registry.npmjs.org/base64id/-/base64id-2.0.0.tgz", 44 | "integrity": "sha512-lGe34o6EHj9y3Kts9R4ZYs/Gr+6N7MCaMlIFA3F1R2O5/m7K06AxfSeO5530PEERE6/WyEg3lsuyw4GHlPZHog==" 45 | }, 46 | "better-assert": { 47 | "version": "1.0.2", 48 | "resolved": "https://registry.npmjs.org/better-assert/-/better-assert-1.0.2.tgz", 49 | "integrity": "sha1-QIZrnhueC1W0gYlDEeaPr/rrxSI=", 50 | "requires": { 51 | "callsite": "1.0.0" 52 | } 53 | }, 54 | "blob": { 55 | "version": "0.0.5", 56 | "resolved": "https://registry.npmjs.org/blob/-/blob-0.0.5.tgz", 57 | "integrity": "sha512-gaqbzQPqOoamawKg0LGVd7SzLgXS+JH61oWprSLH+P+abTczqJbhTR8CmJ2u9/bUYNmHTGJx/UEmn6doAvvuig==" 58 | }, 59 | "callsite": { 60 | "version": "1.0.0", 61 | "resolved": "https://registry.npmjs.org/callsite/-/callsite-1.0.0.tgz", 62 | "integrity": "sha1-KAOY5dZkvXQDi28JBRU+borxvCA=" 63 | }, 64 | "component-bind": { 65 | "version": "1.0.0", 66 | "resolved": "https://registry.npmjs.org/component-bind/-/component-bind-1.0.0.tgz", 67 | "integrity": "sha1-AMYIq33Nk4l8AAllGx06jh5zu9E=" 68 | }, 69 | "component-emitter": { 70 | "version": "1.2.1", 71 | "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.2.1.tgz", 72 | "integrity": "sha1-E3kY1teCg/ffemt8WmPhQOaUJeY=" 73 | }, 74 | "component-inherit": { 75 | "version": "0.0.3", 76 | "resolved": "https://registry.npmjs.org/component-inherit/-/component-inherit-0.0.3.tgz", 77 | "integrity": "sha1-ZF/ErfWLcrZJ1crmUTVhnbJv8UM=" 78 | }, 79 | "cookie": { 80 | "version": "0.4.2", 81 | "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.4.2.tgz", 82 | "integrity": "sha512-aSWTXFzaKWkvHO1Ny/s+ePFpvKsPnjc551iI41v3ny/ow6tBG5Vd+FuqGNhh1LxOmVzOlGUriIlOaokOvhaStA==" 83 | }, 84 | "debug": { 85 | "version": "4.1.1", 86 | "resolved": "https://registry.npmjs.org/debug/-/debug-4.1.1.tgz", 87 | "integrity": "sha512-pYAIzeRo8J6KPEaJ0VWOh5Pzkbw/RetuzehGM7QRRX5he4fPHx2rdKMB256ehJCkX+XRQm16eZLqLNS8RSZXZw==", 88 | "requires": { 89 | "ms": "^2.1.1" 90 | } 91 | }, 92 | "denque": { 93 | "version": "1.5.0", 94 | "resolved": "https://registry.npmjs.org/denque/-/denque-1.5.0.tgz", 95 | "integrity": "sha512-CYiCSgIF1p6EUByQPlGkKnP1M9g0ZV3qMIrqMqZqdwazygIA/YP2vrbcyl1h/WppKJTdl1F85cXIle+394iDAQ==" 96 | }, 97 | "engine.io": { 98 | "version": "3.6.0", 99 | "resolved": "https://registry.npmjs.org/engine.io/-/engine.io-3.6.0.tgz", 100 | "integrity": "sha512-Kc8fo5bbg8F4a2f3HPHTEpGyq/IRIQpyeHu3H1ThR14XDD7VrLcsGBo16HUpahgp8YkHJDaU5gNxJZbuGcuueg==", 101 | "requires": { 102 | "accepts": "~1.3.4", 103 | "base64id": "2.0.0", 104 | "cookie": "~0.4.1", 105 | "debug": "~4.1.0", 106 | "engine.io-parser": "~2.2.0", 107 | "ws": "~7.4.2" 108 | }, 109 | "dependencies": { 110 | "base64-arraybuffer": { 111 | "version": "0.1.4", 112 | "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.4.tgz", 113 | "integrity": "sha512-a1eIFi4R9ySrbiMuyTGx5e92uRH5tQY6kArNcFaKBUleIoLjdjBg7Zxm3Mqm3Kmkf27HLR/1fnxX9q8GQ7Iavg==" 114 | }, 115 | "engine.io-parser": { 116 | "version": "2.2.1", 117 | "resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.2.1.tgz", 118 | "integrity": "sha512-x+dN/fBH8Ro8TFwJ+rkB2AmuVw9Yu2mockR/p3W8f8YtExwFgDvBDi0GWyb4ZLkpahtDGZgtr3zLovanJghPqg==", 119 | "requires": { 120 | "after": "0.8.2", 121 | "arraybuffer.slice": "~0.0.7", 122 | "base64-arraybuffer": "0.1.4", 123 | "blob": "0.0.5", 124 | "has-binary2": "~1.0.2" 125 | } 126 | }, 127 | "ws": { 128 | "version": "7.4.6", 129 | "resolved": "https://registry.npmjs.org/ws/-/ws-7.4.6.tgz", 130 | "integrity": "sha512-YmhHDO4MzaDLB+M9ym/mDA5z0naX8j7SIlT8f8z+I0VtzsRbekxEutHSme7NPS2qE8StCYQNUnfWdXta/Yu85A==" 131 | } 132 | } 133 | }, 134 | "engine.io-client": { 135 | "version": "3.5.3", 136 | "resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-3.5.3.tgz", 137 | "integrity": "sha512-qsgyc/CEhJ6cgMUwxRRtOndGVhIu5hpL5tR4umSpmX/MvkFoIxUTM7oFMDQumHNzlNLwSVy6qhstFPoWTf7dOw==", 138 | "requires": { 139 | "component-emitter": "~1.3.0", 140 | "component-inherit": "0.0.3", 141 | "debug": "~3.1.0", 142 | "engine.io-parser": "~2.2.0", 143 | "has-cors": "1.1.0", 144 | "indexof": "0.0.1", 145 | "parseqs": "0.0.6", 146 | "parseuri": "0.0.6", 147 | "ws": "~7.4.2", 148 | "xmlhttprequest-ssl": "~1.6.2", 149 | "yeast": "0.1.2" 150 | }, 151 | "dependencies": { 152 | "base64-arraybuffer": { 153 | "version": "0.1.4", 154 | "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.4.tgz", 155 | "integrity": "sha512-a1eIFi4R9ySrbiMuyTGx5e92uRH5tQY6kArNcFaKBUleIoLjdjBg7Zxm3Mqm3Kmkf27HLR/1fnxX9q8GQ7Iavg==" 156 | }, 157 | "component-emitter": { 158 | "version": "1.3.0", 159 | "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.3.0.tgz", 160 | "integrity": "sha512-Rd3se6QB+sO1TwqZjscQrurpEPIfO0/yYnSin6Q/rD3mOutHvUrCAhJub3r90uNb+SESBuE0QYoB90YdfatsRg==" 161 | }, 162 | "debug": { 163 | "version": "3.1.0", 164 | "resolved": "https://registry.npmjs.org/debug/-/debug-3.1.0.tgz", 165 | "integrity": "sha512-OX8XqP7/1a9cqkxYw2yXss15f26NKWBpDXQd0/uK/KPqdQhxbPa994hnzjcE2VqQpDslf55723cKPUOGSmMY3g==", 166 | "requires": { 167 | "ms": "2.0.0" 168 | } 169 | }, 170 | "engine.io-parser": { 171 | "version": "2.2.1", 172 | "resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.2.1.tgz", 173 | "integrity": "sha512-x+dN/fBH8Ro8TFwJ+rkB2AmuVw9Yu2mockR/p3W8f8YtExwFgDvBDi0GWyb4ZLkpahtDGZgtr3zLovanJghPqg==", 174 | "requires": { 175 | "after": "0.8.2", 176 | "arraybuffer.slice": "~0.0.7", 177 | "base64-arraybuffer": "0.1.4", 178 | "blob": "0.0.5", 179 | "has-binary2": "~1.0.2" 180 | } 181 | }, 182 | "ms": { 183 | "version": "2.0.0", 184 | "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", 185 | "integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==" 186 | }, 187 | "parseqs": { 188 | "version": "0.0.6", 189 | "resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.6.tgz", 190 | "integrity": "sha512-jeAGzMDbfSHHA091hr0r31eYfTig+29g3GKKE/PPbEQ65X0lmMwlEoqmhzu0iztID5uJpZsFlUPDP8ThPL7M8w==" 191 | }, 192 | "parseuri": { 193 | "version": "0.0.6", 194 | "resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.6.tgz", 195 | "integrity": "sha512-AUjen8sAkGgao7UyCX6Ahv0gIK2fABKmYjvP4xmy5JaKvcbTRueIqIPHLAfq30xJddqSE033IOMUSOMCcK3Sow==" 196 | }, 197 | "ws": { 198 | "version": "7.4.6", 199 | "resolved": "https://registry.npmjs.org/ws/-/ws-7.4.6.tgz", 200 | "integrity": "sha512-YmhHDO4MzaDLB+M9ym/mDA5z0naX8j7SIlT8f8z+I0VtzsRbekxEutHSme7NPS2qE8StCYQNUnfWdXta/Yu85A==" 201 | }, 202 | "xmlhttprequest-ssl": { 203 | "version": "1.6.3", 204 | "resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.6.3.tgz", 205 | "integrity": "sha512-3XfeQE/wNkvrIktn2Kf0869fC0BN6UpydVasGIeSm2B1Llihf7/0UfZM+eCkOw3P7bP4+qPgqhm7ZoxuJtFU0Q==" 206 | } 207 | } 208 | }, 209 | "engine.io-parser": { 210 | "version": "2.1.3", 211 | "resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.1.3.tgz", 212 | "integrity": "sha512-6HXPre2O4Houl7c4g7Ic/XzPnHBvaEmN90vtRO9uLmwtRqQmTOw0QMevL1TOfL2Cpu1VzsaTmMotQgMdkzGkVA==", 213 | "requires": { 214 | "after": "0.8.2", 215 | "arraybuffer.slice": "~0.0.7", 216 | "base64-arraybuffer": "0.1.5", 217 | "blob": "0.0.5", 218 | "has-binary2": "~1.0.2" 219 | } 220 | }, 221 | "has-binary2": { 222 | "version": "1.0.3", 223 | "resolved": "https://registry.npmjs.org/has-binary2/-/has-binary2-1.0.3.tgz", 224 | "integrity": "sha512-G1LWKhDSvhGeAQ8mPVQlqNcOB2sJdwATtZKl2pDKKHfpf/rYj24lkinxf69blJbnsvtqqNU+L3SL50vzZhXOnw==", 225 | "requires": { 226 | "isarray": "2.0.1" 227 | } 228 | }, 229 | "has-cors": { 230 | "version": "1.1.0", 231 | "resolved": "https://registry.npmjs.org/has-cors/-/has-cors-1.1.0.tgz", 232 | "integrity": "sha1-XkdHk/fqmEPRu5nCPu9J/xJv/zk=" 233 | }, 234 | "indexof": { 235 | "version": "0.0.1", 236 | "resolved": "https://registry.npmjs.org/indexof/-/indexof-0.0.1.tgz", 237 | "integrity": "sha1-gtwzbSMrkGIXnQWrMpOmYFn9Q10=" 238 | }, 239 | "isarray": { 240 | "version": "2.0.1", 241 | "resolved": "https://registry.npmjs.org/isarray/-/isarray-2.0.1.tgz", 242 | "integrity": "sha1-o32U7ZzaLVmGXJ92/llu4fM4dB4=" 243 | }, 244 | "mime-db": { 245 | "version": "1.52.0", 246 | "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", 247 | "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==" 248 | }, 249 | "mime-types": { 250 | "version": "2.1.35", 251 | "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", 252 | "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", 253 | "requires": { 254 | "mime-db": "1.52.0" 255 | } 256 | }, 257 | "ms": { 258 | "version": "2.1.3", 259 | "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", 260 | "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==" 261 | }, 262 | "negotiator": { 263 | "version": "0.6.3", 264 | "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz", 265 | "integrity": "sha512-+EUsqGPLsM+j/zdChZjsnX51g4XrHFOIXwfnCVPGlQk/k5giakcKsuxCObBRu6DSm9opw/O6slWbJdghQM4bBg==" 266 | }, 267 | "notepack.io": { 268 | "version": "2.2.0", 269 | "resolved": "https://registry.npmjs.org/notepack.io/-/notepack.io-2.2.0.tgz", 270 | "integrity": "sha512-9b5w3t5VSH6ZPosoYnyDONnUTF8o0UkBw7JLA6eBlYJWyGT1Q3vQa8Hmuj1/X6RYvHjjygBDgw6fJhe0JEojfw==" 271 | }, 272 | "object-component": { 273 | "version": "0.0.3", 274 | "resolved": "https://registry.npmjs.org/object-component/-/object-component-0.0.3.tgz", 275 | "integrity": "sha1-8MaapQ78lbhmwYb0AKM3acsvEpE=" 276 | }, 277 | "parseqs": { 278 | "version": "0.0.5", 279 | "resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.5.tgz", 280 | "integrity": "sha1-1SCKNzjkZ2bikbouoXNoSSGouJ0=", 281 | "requires": { 282 | "better-assert": "~1.0.0" 283 | } 284 | }, 285 | "parseuri": { 286 | "version": "0.0.5", 287 | "resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.5.tgz", 288 | "integrity": "sha1-gCBKUNTbt3m/3G6+J3jZDkvOMgo=", 289 | "requires": { 290 | "better-assert": "~1.0.0" 291 | } 292 | }, 293 | "redis": { 294 | "version": "3.1.2", 295 | "resolved": "https://registry.npmjs.org/redis/-/redis-3.1.2.tgz", 296 | "integrity": "sha512-grn5KoZLr/qrRQVwoSkmzdbw6pwF+/rwODtrOr6vuBRiR/f3rjSTGupbF90Zpqm2oenix8Do6RV7pYEkGwlKkw==", 297 | "requires": { 298 | "denque": "^1.5.0", 299 | "redis-commands": "^1.7.0", 300 | "redis-errors": "^1.2.0", 301 | "redis-parser": "^3.0.0" 302 | }, 303 | "dependencies": { 304 | "redis-commands": { 305 | "version": "1.7.0", 306 | "resolved": "https://registry.npmjs.org/redis-commands/-/redis-commands-1.7.0.tgz", 307 | "integrity": "sha512-nJWqw3bTFy21hX/CPKHth6sfhZbdiHP6bTawSgQBlKOVRG7EZkfHbbHwQJnrE4vsQf0CMNE+3gJ4Fmm16vdVlQ==" 308 | } 309 | } 310 | }, 311 | "redis-errors": { 312 | "version": "1.2.0", 313 | "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz", 314 | "integrity": "sha1-62LSrbFeTq9GEMBK/hUpOEJQq60=" 315 | }, 316 | "redis-parser": { 317 | "version": "3.0.0", 318 | "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz", 319 | "integrity": "sha1-tm2CjNyv5rS4pCin3vTGvKwxyLQ=", 320 | "requires": { 321 | "redis-errors": "^1.0.0" 322 | } 323 | }, 324 | "socket.io": { 325 | "version": "2.5.0", 326 | "resolved": "https://registry.npmjs.org/socket.io/-/socket.io-2.5.0.tgz", 327 | "integrity": "sha512-gGunfS0od3VpwDBpGwVkzSZx6Aqo9uOcf1afJj2cKnKFAoyl16fvhpsUhmUFd4Ldbvl5JvRQed6eQw6oQp6n8w==", 328 | "requires": { 329 | "debug": "~4.1.0", 330 | "engine.io": "~3.6.0", 331 | "has-binary2": "~1.0.2", 332 | "socket.io-adapter": "~1.1.0", 333 | "socket.io-client": "2.5.0", 334 | "socket.io-parser": "~3.4.0" 335 | }, 336 | "dependencies": { 337 | "base64-arraybuffer": { 338 | "version": "0.1.4", 339 | "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-0.1.4.tgz", 340 | "integrity": "sha512-a1eIFi4R9ySrbiMuyTGx5e92uRH5tQY6kArNcFaKBUleIoLjdjBg7Zxm3Mqm3Kmkf27HLR/1fnxX9q8GQ7Iavg==" 341 | }, 342 | "component-emitter": { 343 | "version": "1.3.0", 344 | "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.3.0.tgz", 345 | "integrity": "sha512-Rd3se6QB+sO1TwqZjscQrurpEPIfO0/yYnSin6Q/rD3mOutHvUrCAhJub3r90uNb+SESBuE0QYoB90YdfatsRg==" 346 | }, 347 | "engine.io-parser": { 348 | "version": "2.2.1", 349 | "resolved": "https://registry.npmjs.org/engine.io-parser/-/engine.io-parser-2.2.1.tgz", 350 | "integrity": "sha512-x+dN/fBH8Ro8TFwJ+rkB2AmuVw9Yu2mockR/p3W8f8YtExwFgDvBDi0GWyb4ZLkpahtDGZgtr3zLovanJghPqg==", 351 | "requires": { 352 | "after": "0.8.2", 353 | "arraybuffer.slice": "~0.0.7", 354 | "base64-arraybuffer": "0.1.4", 355 | "blob": "0.0.5", 356 | "has-binary2": "~1.0.2" 357 | } 358 | }, 359 | "ms": { 360 | "version": "2.0.0", 361 | "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", 362 | "integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==" 363 | }, 364 | "parseqs": { 365 | "version": "0.0.6", 366 | "resolved": "https://registry.npmjs.org/parseqs/-/parseqs-0.0.6.tgz", 367 | "integrity": "sha512-jeAGzMDbfSHHA091hr0r31eYfTig+29g3GKKE/PPbEQ65X0lmMwlEoqmhzu0iztID5uJpZsFlUPDP8ThPL7M8w==" 368 | }, 369 | "parseuri": { 370 | "version": "0.0.6", 371 | "resolved": "https://registry.npmjs.org/parseuri/-/parseuri-0.0.6.tgz", 372 | "integrity": "sha512-AUjen8sAkGgao7UyCX6Ahv0gIK2fABKmYjvP4xmy5JaKvcbTRueIqIPHLAfq30xJddqSE033IOMUSOMCcK3Sow==" 373 | }, 374 | "socket.io-client": { 375 | "version": "2.5.0", 376 | "resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-2.5.0.tgz", 377 | "integrity": "sha512-lOO9clmdgssDykiOmVQQitwBAF3I6mYcQAo7hQ7AM6Ny5X7fp8hIJ3HcQs3Rjz4SoggoxA1OgrQyY8EgTbcPYw==", 378 | "requires": { 379 | "backo2": "1.0.2", 380 | "component-bind": "1.0.0", 381 | "component-emitter": "~1.3.0", 382 | "debug": "~3.1.0", 383 | "engine.io-client": "~3.5.0", 384 | "has-binary2": "~1.0.2", 385 | "indexof": "0.0.1", 386 | "parseqs": "0.0.6", 387 | "parseuri": "0.0.6", 388 | "socket.io-parser": "~3.3.0", 389 | "to-array": "0.1.4" 390 | }, 391 | "dependencies": { 392 | "debug": { 393 | "version": "3.1.0", 394 | "resolved": "https://registry.npmjs.org/debug/-/debug-3.1.0.tgz", 395 | "integrity": "sha512-OX8XqP7/1a9cqkxYw2yXss15f26NKWBpDXQd0/uK/KPqdQhxbPa994hnzjcE2VqQpDslf55723cKPUOGSmMY3g==", 396 | "requires": { 397 | "ms": "2.0.0" 398 | } 399 | }, 400 | "socket.io-parser": { 401 | "version": "3.3.3", 402 | "resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.3.3.tgz", 403 | "integrity": "sha512-qOg87q1PMWWTeO01768Yh9ogn7chB9zkKtQnya41Y355S0UmpXgpcrFwAgjYJxu9BdKug5r5e9YtVSeWhKBUZg==", 404 | "requires": { 405 | "component-emitter": "~1.3.0", 406 | "debug": "~3.1.0", 407 | "isarray": "2.0.1" 408 | } 409 | } 410 | } 411 | }, 412 | "ws": { 413 | "version": "7.4.6", 414 | "resolved": "https://registry.npmjs.org/ws/-/ws-7.4.6.tgz", 415 | "integrity": "sha512-YmhHDO4MzaDLB+M9ym/mDA5z0naX8j7SIlT8f8z+I0VtzsRbekxEutHSme7NPS2qE8StCYQNUnfWdXta/Yu85A==" 416 | }, 417 | "xmlhttprequest-ssl": { 418 | "version": "1.6.3", 419 | "resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.6.3.tgz", 420 | "integrity": "sha512-3XfeQE/wNkvrIktn2Kf0869fC0BN6UpydVasGIeSm2B1Llihf7/0UfZM+eCkOw3P7bP4+qPgqhm7ZoxuJtFU0Q==" 421 | } 422 | } 423 | }, 424 | "socket.io-adapter": { 425 | "version": "1.1.2", 426 | "resolved": "https://registry.npmjs.org/socket.io-adapter/-/socket.io-adapter-1.1.2.tgz", 427 | "integrity": "sha512-WzZRUj1kUjrTIrUKpZLEzFZ1OLj5FwLlAFQs9kuZJzJi5DKdU7FsWc36SNmA8iDOtwBQyT8FkrriRM8vXLYz8g==" 428 | }, 429 | "socket.io-client": { 430 | "version": "2.2.0", 431 | "resolved": "https://registry.npmjs.org/socket.io-client/-/socket.io-client-2.2.0.tgz", 432 | "integrity": "sha512-56ZrkTDbdTLmBIyfFYesgOxsjcLnwAKoN4CiPyTVkMQj3zTUh0QAx3GbvIvLpFEOvQWu92yyWICxB0u7wkVbYA==", 433 | "requires": { 434 | "backo2": "1.0.2", 435 | "base64-arraybuffer": "0.1.5", 436 | "component-bind": "1.0.0", 437 | "component-emitter": "1.2.1", 438 | "debug": "~3.1.0", 439 | "engine.io-client": "~3.3.1", 440 | "has-binary2": "~1.0.2", 441 | "has-cors": "1.1.0", 442 | "indexof": "0.0.1", 443 | "object-component": "0.0.3", 444 | "parseqs": "0.0.5", 445 | "parseuri": "0.0.5", 446 | "socket.io-parser": "~3.3.0", 447 | "to-array": "0.1.4" 448 | }, 449 | "dependencies": { 450 | "debug": { 451 | "version": "3.1.0", 452 | "resolved": "https://registry.npmjs.org/debug/-/debug-3.1.0.tgz", 453 | "integrity": "sha512-OX8XqP7/1a9cqkxYw2yXss15f26NKWBpDXQd0/uK/KPqdQhxbPa994hnzjcE2VqQpDslf55723cKPUOGSmMY3g==", 454 | "requires": { 455 | "ms": "2.0.0" 456 | } 457 | }, 458 | "engine.io-client": { 459 | "version": "3.3.3", 460 | "resolved": "https://registry.npmjs.org/engine.io-client/-/engine.io-client-3.3.3.tgz", 461 | "integrity": "sha512-PXIgpzb1brtBzh8Q6vCjzCMeu4nfEPmaDm+L3Qb2sVHwLkxC1qRiBMSjOB0NJNjZ0hbPNUKQa+s8J2XxLOIEeQ==", 462 | "requires": { 463 | "component-emitter": "1.2.1", 464 | "component-inherit": "0.0.3", 465 | "debug": "~3.1.0", 466 | "engine.io-parser": "~2.1.1", 467 | "has-cors": "1.1.0", 468 | "indexof": "0.0.1", 469 | "parseqs": "0.0.5", 470 | "parseuri": "0.0.5", 471 | "ws": "~6.1.0", 472 | "xmlhttprequest-ssl": "~1.6.3", 473 | "yeast": "0.1.2" 474 | } 475 | }, 476 | "ms": { 477 | "version": "2.0.0", 478 | "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", 479 | "integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g=" 480 | }, 481 | "socket.io-parser": { 482 | "version": "3.3.3", 483 | "resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.3.3.tgz", 484 | "integrity": "sha512-qOg87q1PMWWTeO01768Yh9ogn7chB9zkKtQnya41Y355S0UmpXgpcrFwAgjYJxu9BdKug5r5e9YtVSeWhKBUZg==", 485 | "requires": { 486 | "component-emitter": "~1.3.0", 487 | "debug": "~3.1.0", 488 | "isarray": "2.0.1" 489 | }, 490 | "dependencies": { 491 | "component-emitter": { 492 | "version": "1.3.0", 493 | "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.3.0.tgz", 494 | "integrity": "sha512-Rd3se6QB+sO1TwqZjscQrurpEPIfO0/yYnSin6Q/rD3mOutHvUrCAhJub3r90uNb+SESBuE0QYoB90YdfatsRg==" 495 | } 496 | } 497 | }, 498 | "xmlhttprequest-ssl": { 499 | "version": "1.6.3", 500 | "resolved": "https://registry.npmjs.org/xmlhttprequest-ssl/-/xmlhttprequest-ssl-1.6.3.tgz", 501 | "integrity": "sha512-3XfeQE/wNkvrIktn2Kf0869fC0BN6UpydVasGIeSm2B1Llihf7/0UfZM+eCkOw3P7bP4+qPgqhm7ZoxuJtFU0Q==" 502 | } 503 | } 504 | }, 505 | "socket.io-parser": { 506 | "version": "3.4.2", 507 | "resolved": "https://registry.npmjs.org/socket.io-parser/-/socket.io-parser-3.4.2.tgz", 508 | "integrity": "sha512-QFZBaZDNqZXeemwejc7D39jrq2eGK/qZuVDiMPKzZK1hLlNvjGilGt4ckfQZeVX4dGmuPzCytN9ZW1nQlEWjgA==", 509 | "requires": { 510 | "component-emitter": "1.2.1", 511 | "debug": "~4.1.0", 512 | "isarray": "2.0.1" 513 | } 514 | }, 515 | "socket.io-redis": { 516 | "version": "5.4.0", 517 | "resolved": "https://registry.npmjs.org/socket.io-redis/-/socket.io-redis-5.4.0.tgz", 518 | "integrity": "sha512-yCQm/Sywd3d08WXUfZRxt6O+JV2vWoPgWK6GVjiM0GkBtq5cpLOk8oILRPKbzTv1VEtSYmK41q0xzcgDinMbmQ==", 519 | "requires": { 520 | "debug": "~4.1.0", 521 | "notepack.io": "~2.2.0", 522 | "redis": "^3.0.0", 523 | "socket.io-adapter": "~1.1.0", 524 | "uid2": "0.0.3" 525 | } 526 | }, 527 | "to-array": { 528 | "version": "0.1.4", 529 | "resolved": "https://registry.npmjs.org/to-array/-/to-array-0.1.4.tgz", 530 | "integrity": "sha1-F+bBH3PdTz10zaek/zI46a2b+JA=" 531 | }, 532 | "uid2": { 533 | "version": "0.0.3", 534 | "resolved": "https://registry.npmjs.org/uid2/-/uid2-0.0.3.tgz", 535 | "integrity": "sha1-SDEm4Rd03y9xuLY53NeZw3YWK4I=" 536 | }, 537 | "uuid": { 538 | "version": "3.3.3", 539 | "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.3.3.tgz", 540 | "integrity": "sha512-pW0No1RGHgzlpHJO1nsVrHKpOEIxkGg1xB+v0ZmdNH5OAeAwzAVrCnI2/6Mtx+Uys6iaylxa+D3g4j63IKKjSQ==" 541 | }, 542 | "ws": { 543 | "version": "6.1.4", 544 | "resolved": "https://registry.npmjs.org/ws/-/ws-6.1.4.tgz", 545 | "integrity": "sha512-eqZfL+NE/YQc1/ZynhojeV8q+H050oR8AZ2uIev7RU10svA9ZnJUddHcOUZTJLinZ9yEfdA2kSATS2qZK5fhJA==", 546 | "requires": { 547 | "async-limiter": "~1.0.0" 548 | } 549 | }, 550 | "yeast": { 551 | "version": "0.1.2", 552 | "resolved": "https://registry.npmjs.org/yeast/-/yeast-0.1.2.tgz", 553 | "integrity": "sha1-AI4G2AlDIMNy28L47XagymyKxBk=" 554 | } 555 | } 556 | } 557 | -------------------------------------------------------------------------------- /stack/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "static-websocker-scaler", 3 | "version": "0.1.1", 4 | "description": "A simple example of static Socket.io client/server", 5 | "repository": { 6 | "type": "git", 7 | "url": "https://github.com/sw360cab/websockets-scaling.git" 8 | }, 9 | "main": "server_socket.js", 10 | "scripts": { 11 | "test": "echo \"Error: no test specified\" && exit 1", 12 | "start": "node server_socket" 13 | }, 14 | "author": "Sergio Matone", 15 | "license": "MIT", 16 | "dependencies": { 17 | "socket.io": "^2.5.0", 18 | "socket.io-client": "^2.2.0", 19 | "socket.io-redis": "^5.3.8", 20 | "uuid": "^3.3.3" 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /stack/server_socket.js: -------------------------------------------------------------------------------- 1 | const os = require('os'); 2 | const ifaces = os.networkInterfaces(); 3 | 4 | const privateIp = (() => { 5 | return Object.values(ifaces).flat().find(val => { 6 | return (val.family == 'IPv4' && val.internal == false); 7 | }).address; 8 | })(); 9 | 10 | const randomOffset = Math.floor(Math.random() * 10); 11 | const intervalOffset = (30+randomOffset) * Math.pow(10,3); 12 | 13 | // WebSocket Server 14 | const socketPort = 5000; 15 | const socketServer = require('http').createServer(); 16 | const io = require('socket.io')(socketServer, { 17 | path: '/' 18 | }); 19 | // Redis Adapter 20 | const redisAdapter = require('socket.io-redis'); 21 | const redisHost = process.env.REDIS_HOST || 'localhost'; // 'redis-master.default.svc'; 22 | io.adapter(redisAdapter({ host: redisHost, port: 6379 })); 23 | // Handlers 24 | io.on('connection', client => { 25 | console.log('New incoming Connection from', client.id); 26 | client.on('test000', function(message) { 27 | console.log('Message from the client:',client.id,'->',message); 28 | }) 29 | }); 30 | setInterval(() => { 31 | let log0 = `I am the host: ${privateIp}. I am healty.`; 32 | console.log(log0); 33 | io.emit("okok", log0); 34 | }, intervalOffset); 35 | 36 | // Web Socket listen 37 | socketServer.listen(socketPort); --------------------------------------------------------------------------------