├── alb-nginx_ingress.png
├── nginx-values.yaml
├── alb-ingress-connect-nginx .yaml
├── README.md
└── alb-nginx-controller.html


/alb-nginx_ingress.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/rajeshwrn/alb-nginx-controller/HEAD/alb-nginx_ingress.png


--------------------------------------------------------------------------------
/nginx-values.yaml:
--------------------------------------------------------------------------------
 1 | controller: 
 2 |   extraArgs: 
 3 |     http-port: 8080 
 4 |     https-port: 8443 
 5 |   containerPort: 
 6 |     http: 8080 
 7 |     https: 8443 
 8 |   service: 
 9 |     ports: 
10 |       http: 80 
11 |       https: 443 
12 |     targetPorts: 
13 |       http: 8080 
14 |       https: 8443 
15 |   image: 
16 |     allowPrivilegeEscalation: false
17 | 


--------------------------------------------------------------------------------
/alb-ingress-connect-nginx .yaml:
--------------------------------------------------------------------------------
 1 | apiVersion: extensions/v1beta1 
 2 | kind: Ingress 
 3 | metadata: 
 4 |   annotations: 
 5 |     #alb.ingress.kubernetes.io/certificate-arn: <CERTIFICATE_ARN> 
 6 |     alb.ingress.kubernetes.io/healthcheck-path: /healthz 
 7 |     alb.ingress.kubernetes.io/scheme: internet-facing 
 8 |     alb.ingress.kubernetes.io/target-type: ip 
 9 |     alb.ingress.kubernetes.io/subnets: <subnets> 
10 |     kubernetes.io/ingress.class: alb  
11 |   name: alb-ingress-connect-nginx 
12 |   namespace: kube-system 
13 | spec: 
14 |   rules: 
15 |     - http: 
16 |         paths: 
17 |           - backend: 
18 |               serviceName: nginx-ingress-controller 
19 |               servicePort: 8080 
20 |             path: /*
21 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # Single ALB for multiple ingress in AWS EKS fargate 
  2 | 
  3 | Documentation/Steps to create an alb in EKS fargate for multiple ingress/service running different namespace. Option to reuse an existing ALB instead of creating a new ALB per Ingress.
  4 | 
  5 | ## Requirement 
  6 | - Need to deploy multiple service in kubernetes with single alb. 
  7 | - Service/Ingress running on different fargate profile should communicate each other
  8 | 
  9 | ### What is an Application Load Balancer?
 10 | An Application Load Balancer or ALB is a bridge between inbound traffic and several targets (for example several pods for one application). The objective is to have applications with high availability.
 11 | 
 12 | 
 13 | ### Limitations
 14 | 
 15 |   - Each service/ingress running in kubernetes with different namespaces/ fargate profile will create new alb
 16 |   - More cost and more public IP needed
 17 | 
 18 | ### Solution
 19 |  We will now going to achieve this with 2 ingress controllers: the ALB ingress controller and the Nginx Ingress controller. 
 20 |  #### Why do we have to use 2 ingress controllers? 
 21 |  Because if we use the nginx ingress controller, we can not connect it directly to an ALB and if we only use the ALB ingress controller, you will have an ALB instance for every ingress resource in the cluster. 
 22 |  
 23 | But the requirement is one load balancer all services running in cluster. In the case in which we have both: it is important to know the nginx ingress controller will manage all ingresses resources of your applications in your EKS cluster and the ALB ingress controller will manage the life cycle of the Application Load Balancer instance.
 24 | 
 25 | ![alt text](https://github.com/rajeshwrn/alb-nginx-controller/blob/master/alb-nginx_ingress.png?raw=true "alb-architecture")
 26 | 
 27 | To create alb ingress in aws eks fargate use the below aws doc,
 28 | https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
 29 | 
 30 | creating second ingress controller - nginx
 31 | Before that, understand currently the eks fargate will not support nginx official helm chart. so we need to do small tweak on the helm values.
 32 | 
 33 | - Port type should be NodePort
 34 | - allowPrivilegeEscalation should be false to enable the node creation in fargate profile
 35 | - Override defalut container ports 80 to 8080 and 443 to 8443
 36 | - Traffic policy should be local
 37 | 
 38 | ### Installation
 39 | 
 40 | Use the below helm command to deploy nginx controller in your cluster.
 41 | 
 42 | ```
 43 | helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
 44 | ```
 45 | 
 46 | ```
 47 | helm install nginx-ingress ingress-nginx/ingress-nginx --set-string controller.service.externalTrafficPolicy=Local --set-string controller.service.type=NodePort --set controller.publishService.enabled=true --set serviceAccount.create=true --set rbac.create=true --set-string controller.config.server-tokens=false --set-string controller.config.use-proxy-protocol=false --set-string controller.config.compute-full-forwarded-for=true --set-string controller.config.use-forwarded-headers=true --set controller.metrics.enabled=true --set controller.autoscaling.maxReplicas=1 --set controller.autoscaling.minReplicas=1 --set controller.autoscaling.enabled=true --namespace kube-system -f nginx-values.yaml 
 48 | ```
 49 | 
 50 | For more customization on ingress-nginx helm chart 
 51 | https://artifacthub.io/packages/helm/ingress-nginx/ingress-nginx
 52 | 
 53 | Use below value file for helm deployment
 54 | > nginx-values.yaml 
 55 | 
 56 | ```
 57 | controller: 
 58 |   extraArgs: 
 59 |     http-port: 8080 
 60 |     https-port: 8443 
 61 |   containerPort: 
 62 |     http: 8080 
 63 |     https: 8443 
 64 |   service: 
 65 |     ports: 
 66 |       http: 80 
 67 |       https: 443 
 68 |     targetPorts: 
 69 |       http: 8080 
 70 |       https: 8443 
 71 |   image: 
 72 |     allowPrivilegeEscalation: false
 73 | ```
 74 | 
 75 | ### Connect the ALB to the Nginx Ingress controller
 76 | 
 77 | To connect the ALB to the nginx ingress controller, we need to create a kubernetes ingress resource in the namespace kube-system with the following configuration:
 78 | 
 79 | > kubectl apply -f alb-ingress-connect-nginx.yaml 
 80 | 
 81 | ```
 82 | apiVersion: extensions/v1beta1 
 83 | kind: Ingress 
 84 | metadata: 
 85 |   annotations: 
 86 |     #alb.ingress.kubernetes.io/certificate-arn: <CERTIFICATE_ARN> 
 87 |     alb.ingress.kubernetes.io/healthcheck-path: /healthz 
 88 |     alb.ingress.kubernetes.io/scheme: internet-facing 
 89 |     alb.ingress.kubernetes.io/target-type: ip 
 90 |     alb.ingress.kubernetes.io/subnets: <subnets> 
 91 |     kubernetes.io/ingress.class: alb  
 92 |   name: alb-ingress-connect-nginx 
 93 |   namespace: kube-system 
 94 | spec: 
 95 |   rules: 
 96 |     - http: 
 97 |         paths: 
 98 |           - backend: 
 99 |               serviceName: nginx-ingress-controller 
100 |               servicePort: 8080 
101 |             path: /* 
102 | ```
103 | 
104 | Now we are ready with alb it will communicate to the nginx ingress controller.
105 | 
106 | Validate the changes by accessing the ingress address, it will be the public facing dns address and we can access the services running in cluster this dns.
107 | > kubectl get ingress -n kube-system
108 | 
109 | 
110 | 
111 | Deploy your application with following ingress annonation and service port should be 'ClusterIP'
112 |  ```
113 |  annotations:
114 |     kubernetes.io/ingress.class: "nginx"
115 | ```
116 | 
117 | 


--------------------------------------------------------------------------------
/alb-nginx-controller.html:
--------------------------------------------------------------------------------
  1 | <!DOCTYPE html>
  2 | <html>
  3 |    <head>
  4 |       <meta property="article:published_time" content="2020-12-04T10:00:00.000Z" />
  5 |    </head>
  6 |    <body>
  7 |       <main>
  8 |          <article>
  9 |             <section>
 10 |                <div class="pw-post-title">
 11 |                   <h1 id="single-alb-for-multiple-ingress-in-aws-eks-fargate">Single ALB for multiple ingress in AWS EKS fargate</h1>
 12 |                </div>
 13 |                <p>Documentation/Steps to create an alb in EKS fargate for multiple ingress/service running different namespace. Option to reuse an existing ALB instead of creating a new ALB per Ingress.</p>
 14 |                <h2 id="requirement">Requirement</h2>
 15 |                <ul>
 16 |                   <li>Need to deploy multiple service in kubernetes with single alb. </li>
 17 |                   <li>Service/Ingress running on different fargate profile should communicate each other</li>
 18 |                </ul>
 19 |                <h3 id="what-is-an-application-load-balancer-">What is an Application Load Balancer?</h3>
 20 |                <p>An Application Load Balancer or ALB is a bridge between inbound traffic and several targets (for example several pods for one application). The objective is to have applications with high availability.</p>
 21 |                <h3 id="limitations">Limitations</h3>
 22 |                <ul>
 23 |                   <li>Each service/ingress running in kubernetes with different namespaces/ fargate profile will create new alb</li>
 24 |                   <li>More cost and more public IP needed</li>
 25 |                </ul>
 26 |                <h3 id="solution">Solution</h3>
 27 |                <p> We will now going to achieve this with 2 ingress controllers: the ALB ingress controller and the Nginx Ingress controller. </p>
 28 |                <h4 id="why-do-we-have-to-use-2-ingress-controllers-">Why do we have to use 2 ingress controllers?</h4>
 29 |                <p> Because if we use the nginx ingress controller, we can not connect it directly to an ALB and if we only use the ALB ingress controller, you will have an ALB instance for every ingress resource in the cluster. </p>
 30 |                <p>But the requirement is one load balancer all services running in cluster. In the case in which we have both: it is important to know the nginx ingress controller will manage all ingresses resources of your applications in your EKS cluster and the ALB ingress controller will manage the life cycle of the Application Load Balancer instance.</p>
 31 |                <p><img src="https://github.com/rajeshwrn/alb-nginx-controller/blob/master/alb-nginx_ingress.png?raw=true" alt="alt text" title="alb-architecture"></p>
 32 |                <p>To create alb ingress in aws eks fargate use the below aws doc,
 33 |                   <a href="https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html">https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html</a>
 34 |                </p>
 35 |                <p>creating second ingress controller - nginx
 36 |                   Before that, understand currently the eks fargate will not support nginx official helm chart. so we need to do small tweak on the helm values.
 37 |                </p>
 38 |                <ul>
 39 |                   <li>Port type should be NodePort</li>
 40 |                   <li>allowPrivilegeEscalation should be false to enable the node creation in fargate profile</li>
 41 |                   <li>Override defalut container ports 80 to 8080 and 443 to 8443</li>
 42 |                   <li>Traffic policy should be local</li>
 43 |                </ul>
 44 |                <h3 id="installation">Installation</h3>
 45 |                <p>Use the below helm command to deploy nginx controller in your cluster.</p>
 46 |                <pre><code>helm repo <span class="hljs-keyword">add</span> ingress-nginx https:<span class="hljs-comment">//kubernetes.github.io/ingress-nginx</span>
 47 | </code></pre>
 48 |                <pre><code>helm install nginx-ingress ingress-nginx/ingress-nginx --set-string controller<span class="hljs-selector-class">.service</span><span class="hljs-selector-class">.externalTrafficPolicy</span>=Local --set-string controller<span class="hljs-selector-class">.service</span><span class="hljs-selector-class">.type</span>=NodePort --set controller<span class="hljs-selector-class">.publishService</span><span class="hljs-selector-class">.enabled</span>=true --set serviceAccount.create=true --set rbac.create=true --set-string controller<span class="hljs-selector-class">.config</span><span class="hljs-selector-class">.server-tokens</span>=false --set-string controller<span class="hljs-selector-class">.config</span><span class="hljs-selector-class">.use-proxy-protocol</span>=false --set-string controller<span class="hljs-selector-class">.config</span><span class="hljs-selector-class">.compute-full-forwarded-for</span>=true --set-string controller<span class="hljs-selector-class">.config</span><span class="hljs-selector-class">.use-forwarded-headers</span>=true --set controller<span class="hljs-selector-class">.metrics</span><span class="hljs-selector-class">.enabled</span>=true --set controller<span class="hljs-selector-class">.autoscaling</span><span class="hljs-selector-class">.maxReplicas</span>=<span class="hljs-number">1</span> --set controller<span class="hljs-selector-class">.autoscaling</span><span class="hljs-selector-class">.minReplicas</span>=<span class="hljs-number">1</span> --set controller<span class="hljs-selector-class">.autoscaling</span><span class="hljs-selector-class">.enabled</span>=true --namespace kube-system -f nginx-values.yaml
 49 | </code></pre>
 50 |                <p>For more customization on ingress-nginx helm chart 
 51 |                   <a href="https://artifacthub.io/packages/helm/ingress-nginx/ingress-nginx">https://artifacthub.io/packages/helm/ingress-nginx/ingress-nginx</a>
 52 |                </p>
 53 |                <p>Use below value file for helm deployment</p>
 54 |                <blockquote>
 55 |                   <p>nginx-values.yaml </p>
 56 |                </blockquote>
 57 |                <pre><code><span class="hljs-attr">controller:</span> 
 58 | <span class="hljs-attr">  extraArgs:</span> 
 59 | <span class="hljs-attr">    http-port:</span> <span class="hljs-number">8080</span> 
 60 | <span class="hljs-attr">    https-port:</span> <span class="hljs-number">8443</span> 
 61 | <span class="hljs-attr">  containerPort:</span> 
 62 | <span class="hljs-attr">    http:</span> <span class="hljs-number">8080</span> 
 63 | <span class="hljs-attr">    https:</span> <span class="hljs-number">8443</span> 
 64 | <span class="hljs-attr">  service:</span> 
 65 | <span class="hljs-attr">    ports:</span> 
 66 | <span class="hljs-attr">      http:</span> <span class="hljs-number">80</span> 
 67 | <span class="hljs-attr">      https:</span> <span class="hljs-number">443</span> 
 68 | <span class="hljs-attr">    targetPorts:</span> 
 69 | <span class="hljs-attr">      http:</span> <span class="hljs-number">8080</span> 
 70 | <span class="hljs-attr">      https:</span> <span class="hljs-number">8443</span> 
 71 | <span class="hljs-attr">  image:</span> 
 72 | <span class="hljs-attr">    allowPrivilegeEscalation:</span> <span class="hljs-literal">false</span>
 73 | </code></pre>
 74 |                <h3 id="connect-the-alb-to-the-nginx-ingress-controller">Connect the ALB to the Nginx Ingress controller</h3>
 75 |                <p>To connect the ALB to the nginx ingress controller, we need to create a kubernetes ingress resource in the namespace kube-system with the following configuration:</p>
 76 |                <blockquote>
 77 |                   <p>kubectl apply -f alb-ingress-connect-nginx.yaml </p>
 78 |                </blockquote>
 79 |                <pre><code>apiVersion: extensions/v1beta1 
 80 | kind: Ingress 
 81 | metadata: 
 82 |   annotations: 
 83 |     <span class="hljs-selector-id">#alb</span><span class="hljs-selector-class">.ingress</span><span class="hljs-selector-class">.kubernetes</span><span class="hljs-selector-class">.io</span>/certificate-arn: &lt;CERTIFICATE_ARN&gt; 
 84 |     alb<span class="hljs-selector-class">.ingress</span><span class="hljs-selector-class">.kubernetes</span><span class="hljs-selector-class">.io</span>/healthcheck-path: /healthz 
 85 |     alb<span class="hljs-selector-class">.ingress</span><span class="hljs-selector-class">.kubernetes</span><span class="hljs-selector-class">.io</span>/scheme: internet-facing 
 86 |     alb<span class="hljs-selector-class">.ingress</span><span class="hljs-selector-class">.kubernetes</span><span class="hljs-selector-class">.io</span>/target-type: ip 
 87 |     alb<span class="hljs-selector-class">.ingress</span><span class="hljs-selector-class">.kubernetes</span><span class="hljs-selector-class">.io</span>/subnets: &lt;subnets&gt; 
 88 |     kubernetes.io/ingress<span class="hljs-selector-class">.class</span>: alb  
 89 |   name: alb-ingress-connect-nginx 
 90 |   namespace: kube-system 
 91 | spec: 
 92 |   rules: 
 93 |     - http: 
 94 |         paths: 
 95 |           - backend: 
 96 |               serviceName: nginx-ingress-controller 
 97 |               servicePort: <span class="hljs-number">8080</span> 
 98 |             path: <span class="hljs-comment">/*</span>
 99 | </code></pre>
100 |                <p>Now we are ready with alb it will communicate to the nginx ingress controller.</p>
101 |                <p>Validate the changes by accessing the ingress address, it will be the public facing dns address and we can access the services running in cluster this dns.</p>
102 |                <blockquote>
103 |                   <p>kubectl get ingress -n kube-system</p>
104 |                </blockquote>
105 |                <p>Deploy your application with following ingress annonation and service port should be &#39;ClusterIP&#39;</p>
106 |                <pre><code> <span class="hljs-attribute">annotations</span>:
107 |     kubernetes.io/ingress.<span class="hljs-attribute">class</span>: <span class="hljs-string">"nginx"</span>
108 | </code></pre>
109 |             </section>
110 |          </article>
111 |       </main>
112 |    </body>
113 | </html>
114 | 


--------------------------------------------------------------------------------