├── .gitignore ├── images ├── logs.jpg ├── eb-init.jpg ├── log-title.jpg ├── RDS │ ├── region.jpg │ ├── db-settings.jpg │ ├── create-database.jpg │ ├── database-endpoint.jpg │ └── rds-dashboard-create.jpg ├── S3 │ ├── IAM-users.jpg │ ├── create-group.jpg │ ├── S3-buckets-create.jpg │ └── programmatic-access.jpg ├── createEbApp2.jpg ├── health-check.jpg ├── configuration.jpg ├── eb-successful.jpg ├── recent-events.jpg ├── AWSinterface │ ├── s3.jpg │ ├── search.jpg │ ├── aws_console.jpg │ ├── ec2_console.jpg │ ├── RDS_dashboard.jpg │ └── elastic_branstalk.jpg ├── ElastiCache │ ├── rules.jpg │ ├── elasticache.jpg │ ├── redis-create.png │ ├── loadbalancer-config.jpg │ └── create-https-loadbalancer.png ├── degraded-health.jpg ├── disallowed_host.jpg ├── environment-info.jpg ├── select-env-tire.jpg ├── CreateEBApplication.jpg ├── configuring-eb-env.jpg ├── create-environment.jpg ├── environment-properties.jpg ├── HttpsRedirect │ ├── addlistener.jpg │ ├── https-redirect.jpg │ └── redirect_to_Https.jpg └── security-groups │ ├── security_group.jpg │ └── security-group-create.png ├── LICENSE ├── redirectHttps.md ├── SecurityGroups.md ├── IAM.md ├── Route53.md ├── debugging-tips.md ├── AWS_Interface.md ├── s3buckets.md ├── readme.md ├── UsingEnvironment.md ├── django-channels.md ├── connecting_RDS.md └── deploying_django.md /.gitignore: -------------------------------------------------------------------------------- 1 | temp -------------------------------------------------------------------------------- /images/logs.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/logs.jpg -------------------------------------------------------------------------------- /images/eb-init.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/eb-init.jpg -------------------------------------------------------------------------------- /images/log-title.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/log-title.jpg -------------------------------------------------------------------------------- /images/RDS/region.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/RDS/region.jpg -------------------------------------------------------------------------------- /images/S3/IAM-users.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/S3/IAM-users.jpg -------------------------------------------------------------------------------- /images/createEbApp2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/createEbApp2.jpg -------------------------------------------------------------------------------- /images/health-check.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/health-check.jpg -------------------------------------------------------------------------------- /images/configuration.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/configuration.jpg -------------------------------------------------------------------------------- /images/eb-successful.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/eb-successful.jpg -------------------------------------------------------------------------------- /images/recent-events.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/recent-events.jpg -------------------------------------------------------------------------------- /images/AWSinterface/s3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/AWSinterface/s3.jpg -------------------------------------------------------------------------------- /images/ElastiCache/rules.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/ElastiCache/rules.jpg -------------------------------------------------------------------------------- /images/RDS/db-settings.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/RDS/db-settings.jpg -------------------------------------------------------------------------------- /images/S3/create-group.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/S3/create-group.jpg -------------------------------------------------------------------------------- /images/degraded-health.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/degraded-health.jpg -------------------------------------------------------------------------------- /images/disallowed_host.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/disallowed_host.jpg -------------------------------------------------------------------------------- /images/environment-info.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/environment-info.jpg -------------------------------------------------------------------------------- /images/select-env-tire.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/select-env-tire.jpg -------------------------------------------------------------------------------- /images/AWSinterface/search.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/AWSinterface/search.jpg -------------------------------------------------------------------------------- /images/CreateEBApplication.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/CreateEBApplication.jpg -------------------------------------------------------------------------------- /images/RDS/create-database.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/RDS/create-database.jpg -------------------------------------------------------------------------------- /images/configuring-eb-env.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/configuring-eb-env.jpg -------------------------------------------------------------------------------- /images/create-environment.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/create-environment.jpg -------------------------------------------------------------------------------- /images/RDS/database-endpoint.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/RDS/database-endpoint.jpg -------------------------------------------------------------------------------- /images/S3/S3-buckets-create.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/S3/S3-buckets-create.jpg -------------------------------------------------------------------------------- /images/S3/programmatic-access.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/S3/programmatic-access.jpg -------------------------------------------------------------------------------- /images/environment-properties.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/environment-properties.jpg -------------------------------------------------------------------------------- /images/AWSinterface/aws_console.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/AWSinterface/aws_console.jpg -------------------------------------------------------------------------------- /images/AWSinterface/ec2_console.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/AWSinterface/ec2_console.jpg -------------------------------------------------------------------------------- /images/ElastiCache/elasticache.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/ElastiCache/elasticache.jpg -------------------------------------------------------------------------------- /images/ElastiCache/redis-create.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/ElastiCache/redis-create.png -------------------------------------------------------------------------------- /images/RDS/rds-dashboard-create.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/RDS/rds-dashboard-create.jpg -------------------------------------------------------------------------------- /images/AWSinterface/RDS_dashboard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/AWSinterface/RDS_dashboard.jpg -------------------------------------------------------------------------------- /images/HttpsRedirect/addlistener.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/HttpsRedirect/addlistener.jpg -------------------------------------------------------------------------------- /images/HttpsRedirect/https-redirect.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/HttpsRedirect/https-redirect.jpg -------------------------------------------------------------------------------- /images/AWSinterface/elastic_branstalk.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/AWSinterface/elastic_branstalk.jpg -------------------------------------------------------------------------------- /images/ElastiCache/loadbalancer-config.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/ElastiCache/loadbalancer-config.jpg -------------------------------------------------------------------------------- /images/HttpsRedirect/redirect_to_Https.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/HttpsRedirect/redirect_to_Https.jpg -------------------------------------------------------------------------------- /images/security-groups/security_group.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/security-groups/security_group.jpg -------------------------------------------------------------------------------- /images/ElastiCache/create-https-loadbalancer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/ElastiCache/create-https-loadbalancer.png -------------------------------------------------------------------------------- /images/security-groups/security-group-create.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PaulleDemon/AWS-deployment/HEAD/images/security-groups/security-group-create.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Paul 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /redirectHttps.md: -------------------------------------------------------------------------------- 1 | # Redirecting traffic from Http to Https: 2 | 3 | If you don't have an SSL certificate for your domain, you can request one from the certificate manager in the AWS console. There are a lot of tutorials so I won't be doing it 4 | 5 | Chances are that you don't want your users to connect through HTTP you instead want them to be connected using HTTPS: 6 | 7 | So for that go to EC2 console then in the left nav panel scroll down to load balancer -> click on listeners tab 8 | 9 | ![Load balancer](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/HttpsRedirect/https-redirect.jpg) 10 | 11 | If you don't see HTTPS:443 under listeners then click on "add listeners" 12 | 13 | then under protocol click on HTTPS port no 443: 14 | Then under default action dropdown click on "forward" to: select your target group where your EC2 resides. 15 | 16 | Now scroll down and select your SSL certificate: 17 | 18 | and click on "add" 19 | 20 | ![https](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/HttpsRedirect/addlistener.jpg) 21 | 22 | Now go to listeners and select the HTTP listener and click on edit: 23 | 24 | From the "Default action" drop-down select "Redirect" to -> select HTTPS protocol and port 443 then save changes. 25 | 26 | ![https-redirect](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/HttpsRedirect/redirect_to_Https.jpg) 27 | 28 | That's it save the changes and now all http protocols will be redirected to https. 29 | 30 | 31 | ### Additional information 32 | 33 | Additionally for django users, you can also set `SECURE_SSL_REDIRECT=True` in settings.py to redirect http to https connections. 34 | 35 | reference: https://docs.djangoproject.com/en/2.2/topics/security/#ssl-https -------------------------------------------------------------------------------- /SecurityGroups.md: -------------------------------------------------------------------------------- 1 | # Security Groups 2 | 3 | A security group acts as a virtual firewall for your EC2 instances to control the incoming traffic and outgoing traffic. You can control the incoming traffic by setting inbound rules and the outgoing traffic by setting the Outbound rules. 4 | 5 | By default, there is a security group called as Default security group. You can also create your custom security group. 6 | 7 | 8 | ## creating custom security group: 9 | 10 | Go to EC2 console -> Scroll down to Security groups in the side navigation panel -> security group. 11 | 12 | Your security groups will appear here. You can create a new security group, modify inbound/outbound rules for existing security groups from here. 13 | ![security group](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/security-groups/security_group.jpg) 14 | 15 | 1. Click on create security group. 16 | 17 | ![Create security group](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/security-groups/security-group-create.png) 18 | 19 | Give it a security group name, give an optional description. You can choose a default VPC. 20 | 21 | When you click on add rule below are common things you will have to fill for both 22 | 23 | ### Common functions for inbound and outbound 24 | The Type contains many Types like custom TCP, All traffic, All TCP, HTTP, HTTPS, PostgresSQL. 25 | 26 | Some of the predefined Types such as PostgresSQL will automatically fill the port number to 5432 and protocol to TCP. HTTPS will assign 443 as port and protocol to TCP. 27 | 28 | In source, you can choose who should be allowed. 29 | Sources include Anywhere-IPv4, Anywhere-IPv6, custom and My IP. 30 | 31 | Now be careful while choosing My Ip. If you are connected through a modem it's likely that your IP address might keep changing. So you might have to update my IP in the secuity group each time. 32 | 33 | 34 | You can add as many rules as you want in a security group. 35 | 36 | ### Inbound rules: 37 | 38 | This is what controls the incoming traffic. Your rules define who should be allowed access and who shouldn't be. 39 | 40 | ### Outbound rules: 41 | 42 | This controls the outgoing traffic. This rule checks on the outgoing access. 43 | 44 | You can now assign this to your EC2 instance or even modify the security groups of EC2, RDS S3 buckets, etc. 45 | -------------------------------------------------------------------------------- /IAM.md: -------------------------------------------------------------------------------- 1 | # IAM 2 | 3 | Identity and access management helps in ensuring that the right users have appropriate access to the AWS resources. 4 | 5 | If you are an employer with root access to AWS and want to allow your employees to have access to the AWS resources, you shouldn't give away your root account mail id and password, instead, you should use your account to create a User Group and Users from IAM console. You can then have control on what aws resources your employee can have access to from here. 6 | 7 | ### User: 8 | 9 | > An AWS Identity and Access Management (IAM) user is an entity that you create in AWS to represent the person or application that uses it to interact with AWS. A user in AWS consists of a name and credentials. 10 | From the docs: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html 11 | 12 | To add a new user: 13 | 14 | IAM console -> User (left nav panel) -> Add user 15 | 16 | 1. Create a username check on the access type you want. If you are using from application to access resources you might want to go with "Access key - Programmatic access". Programmatic access provides you with an access key and secret key. If you simply want your employee to have access to resources you might want to go with "Password - AWS Management Console access". You can also choose to check on both the checkboxes(programmatic access and AWS management console access). Now click next. 17 | 2. You can choose to add users to a group(discussed below). If you don't have any group you can create group or skip this and add it later, click next. 18 | 3. You can choose to add key-value such as user's email for example. The key would be **Email** and value would be **person@mail.com**. You can also choose to skip this. Click on next. 19 | 4. Now review your choice and click on create. 20 | 21 | ### User Group: 22 | 23 | > An IAM user group is a collection of IAM users. User groups let you specify permissions for multiple users, which can make it easier to manage the permissions for those users. 24 | from the docs: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html 25 | 26 | You can create a security group by heading to IAM console -> User groups(left nav panel) -> Create group. 27 | 28 | 1. Now give it a group name for example: Database access. 29 | 30 | 2. Add users to the group all the users in this group will have access to resources you specify in attach permissions policy for eg: all users in this group will have access to database resources. 31 | 32 | 3. Attach permission policy by simply clicking on the checkboxes. You can check on multiple checkboxes. For example, you can check on "AmazonS3FullAccess" to allow users of this group to have full access to S3 resources. 33 | 34 | 4. Click on create group. 35 | -------------------------------------------------------------------------------- /Route53.md: -------------------------------------------------------------------------------- 1 | # Route53 2 | 3 | **What is DNS(domain name system)?** 4 | 5 | Every device connected on the internet has a unique IP address, which other machines use to find the device. 6 | The DNS eliminates the need for us humans to memorize IP addresses, such as 192.164.2.1(IPv4) and instead help us use more friendly names such as 7 | [peckspace.com](peckspace.com) to browse websites. 8 |
9 | The process of DNS resolution helps in converting hostnames back into computer-friendly IP address such as 192.164.2.1. 10 | 11 | ## Route 53: 12 | 13 | from AWS: https://aws.amazon.com/route53/ 14 | 15 | *"Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. Amazon Route 53 effectively connects user requests to infrastructure running in AWS – such as Amazon EC2 instances, Elastic Load Balancing load balancers, or Amazon S3 buckets."* 16 | 17 | ## connecting Elasticbeanstalk to custom domain 18 | 19 | Once you have purchased a domain name. Search for Route53 from the search bar. 20 | 21 | 1. From the left sidebar click on hosted zones, and then click on create hosted zone. 22 | 2. In the hosted zone configuration page, under domain name give your domain name, and give an optional description. 23 | 3. Now thats done, You will see two rows, One with Type(column type) NS(name server) and other with type SOA. Copy all the values(the last column)from the NS row. 24 | 4. Go to the website you purchased the domain from and find DNS management console and update all the NS values there to the one given by the AWS. 25 | 6. now create a record and under record name give it a subdomain name like www or about or just leave it blank. (Note: if you use *, it will act as a wild card, meaning all the subdomains will be captured) 26 | 7. Let the record type be "A-Routes traffic to an IPv4 address and some AWS resource". 27 | 8. Now turn on the Alias near to the value, You will be shown a Route traffic to dropdown. 28 | 9. From the drop-down choose "Alias to ElasticBeanstalk environment". 29 | 10. Choose the region you have your ElasticBeanstalk environment, then choose your correct environment. Now click on create record. 30 | 31 | Thats it will now be able to use your custom domain. 32 | 33 | ## Email configuration 34 | 35 | If you have purchased email for your domain from some other website and have hosted your domain in Route53, 36 | you will not be able to send emails unless you create the records thats asked by email provider. 37 | 38 | They usually ask you to create MX record, TXT record and couple of CNAME. 39 | 40 | Creating the records are same as above, you just need to change record type to MX or TXT depending what your mail provider asks you to. 41 | 42 | Then just copy the values and create those records. 43 | 44 | 45 | ## Hosting static websites 46 | 47 | Sometimes you might need to take a maintenance break for your main website, and you probably don't want the users to write or read your website. 48 | 49 | In the mean time you can serve a static website from s3 bucket telling your users that the website is currently under maintenance. 50 | 51 | To do so, follow the following steps. 52 | 53 | 1. Go to s3 bucket and create a bucket matching your domain, for example if your domain is `example.com` then name your bucket `example.com` 54 | 2. once you have created that bucket, click on the bucket and under properties tab, scroll down and enable static website hosting. 55 | 3. Upload all your static website files into that bucket 56 | 3. Specify your index document and click on save changes 57 | 4. Under bucket policy in the permissions tab make it read only by editing it and adding the following. 58 | 59 | ```json 60 | { 61 | "Version": "2012-10-17", 62 | "Statement": [ 63 | { 64 | "Sid": "PublicRead", 65 | "Effect": "Allow", 66 | "Principal": "*", 67 | "Action": [ 68 | "s3:GetObject", 69 | "s3:GetObjectVersion" 70 | ], 71 | "Resource": "arn:aws:s3:::example.com/*" 72 | } 73 | ] 74 | } 75 | ``` 76 | Replace example.com with your bucket name. 77 | 78 | 5. Go back to Route53 and find the subdomain that matches your bucket name, in my case it was `example.com`. Click on edit values. 79 | 6. Now under route traffic to dropdown(this will appear if you have enable alias), click on "Alias to s3 website endpoint". 80 | 7. Choose the region and your bucket from the dropdown then click on save. 81 | 82 | That it now you can server your static website from s3 bucket 83 | 84 | -------------------------------------------------------------------------------- /debugging-tips.md: -------------------------------------------------------------------------------- 1 | # Debugging tips: 2 | 3 | There is a high chance that you may not be able to get your application running the first time you deploy it. So here are some of the debugging tips you can use to understand the cause of your error. 4 | 5 | ### Check list 6 | 7 | 1. Ensure you have not added `.ebextensions` and `requirements.txt` to `.gitignore` 8 | 9 | 2. Check if all the endpoints to RDS, S3 are correct. 10 | 11 | ### debugging tips: 12 | 13 | 1. Check the logs in the elastic beanstalk. 14 | ![Logs](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/logs.jpg). 15 | You would usually need to download only the last 100 lines. In logs, Most likely your errors would lie under the `/var/log/web.stdout.log`. 16 | 17 | If you get `ModuleNotFoundError: No module named 'application'` then it's likely your wsgi path was not set correctly during deployment. Try to stage changes using `git add .` and redeploy the staged changes `eb deploy --staged` 18 | 19 | If you get `ModuleNotFoundError: No module named 'xyz'` make sure you have added the module xyz in the `requirments.txt`(here xyz is a placeholder, this isn't an actual library). 20 | 21 | Check if your database is connected by connecting to Ec2 instance using `eb shh`. 22 | ```sh 23 | eb ssh 24 | Enter passphrase: 25 | 26 | -------------- Elastic beanstalk----------------- 27 | source /var/app/venv/*/bin/activate 28 | cd /var/app/current/ 29 | python manage.py check --database default 30 | ``` 31 | This should result in `System check identified no issues (0 silenced).` if there is no connection problem. 32 | 33 | 34 | ## Some commonly used commands: 35 | 36 | ### 1. EB CLI COMMANDS: 37 | 38 | Things in square brackets are optional. 39 | | Command | description | usage | 40 | |---------------------------------------|--------------------------------------------------------------------------------------------------|------------------------------------------------| 41 | | eb init [-p platform applicationname] | sets default values for Elastic Beanstalk application prompting you with a series of questions | eb init, eb init -p python-3.8 ApplicationName | 42 | | eb create [environmnet name] | creates an environment | eb create, eb create sample-env | 43 | | eb ssh --setup | change or create key pair(ssh) assigned to an instance | eb ssh --setup | 44 | | eb ssh [environment name] | Connects to Amazon Linux EC2 instance in your environment using SSH | eb ssh, eb ssh sample-env | 45 | | eb deploy | deploys the application bundle from the initialized project directory to the running application | eb deploy | 46 | | eb deploy --staged | deploy staged changes | eb deploy --staged | 47 | 48 | 49 | ### Some Linux commands: 50 | 51 | | Command | description | usage | 52 | |---------------------------------------------------------|-------------------------------------------------------------------------------------|--------------------------------------------| 53 | | cd | change directory | cd /var/app/current/ | 54 | | ls | list directories and files under a specified directory | ls /var/app/current/ | 55 | | source | reads and executes the contents of a file | source /var/app/venv/*/bin/activate | 56 | | exit | exit the shell where its running currently | exit | 57 | | netstat -antpl | returns listening ports along with application | netstat -antpl | 58 | | sudo | helps you run programs with security privileges of another user | sudo netstat -antpl | 59 | | lsof -i:portno | returns the application thats running in specific port | sudo lsof -i:5000 | 60 | | psql -h [HOST] -p[PORTNO] -U[USER NAME] [DATABASE NAME] | connect to postgres SQL (this command is available only if you have installed psql) | psql -h localhost -p 5432 -U user postgres | 61 | 62 | Running daphne: 63 | ``` 64 | daphne -b 0.0.0.0 -p 5000 project.asgi:application 65 | ``` 66 | 67 | Checking database connection in python using manage.py: 68 | ``` 69 | python manage.py check --database default 70 | ``` 71 | -------------------------------------------------------------------------------- /AWS_Interface.md: -------------------------------------------------------------------------------- 1 | # Introduction to AWS console interface 2 | 3 | If you are new to AWS you are very much likely to find it difficult to navigate through the AWS interface. 4 | 5 | In this blog we'll only be focusing on navigating in AWS interface, we'll also talk about some of the commonly used services. 6 | 7 | Once you create an account on AWS you will mostly be working on their console. 8 | 9 | You should be able to find their console just by typing AWS console in the browser. 10 | 11 | ![AWS console](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/AWSinterface/aws_console.jpg) 12 | 13 | This is how the console looks as of Jan-31-2022 14 | 15 | Here you will be able to see the recently visited items. You should be able to find their services just by typing the name of the service in the search bar located in the header. 16 | 17 | ## Commonly used services: 18 | 19 | 1. **Elastic Beanstalk**: This is one of their service which makes it easier of deploying our web application. It combines many of the other services and helps you to easily create a database, ec2 instance, loadbalancer, etc. while still giving you a good control of the resources you use. 20 | 21 | 2. **RDS (Relational database service)** : 22 | This is a relational database service offered by AWS. This is likely where you will be creating your database. 23 | 24 | 3. **S3(simple storage service)**: You will likely be using this service mostly to service static files and media files directly from amazon. 25 | 26 | 4. **EC2(Elastic computer cloud)**: This is the service that provides you with the hardware necessary to run your application on the cloud. 27 | 28 | 5. **Route 53**: 29 | from AWS: https://aws.amazon.com/route53/ 30 | > Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. Amazon Route 53 effectively connects user requests to infrastructure running in AWS – such as Amazon EC2 instances, Elastic Load Balancing load balancers, or Amazon S3 buckets. 31 | 32 | 6. **Certificate manager**: This provides you SSL/TSL certificate you would like to use in AWS, after validating your domain ownership. 33 | 34 | 7. **IAM(Identity and access management)**: If you are working as a team or you have employee's you want to give access to AWS services. You will use the root account(your account) that has access to all the AWS services to setup an account for an employee using IAM so that you have control over what they have access to in the AWS services. 35 | 36 | 8. **ElastiCache** This is used to start an in-memory database such as Redis. Useful when you have a websocket and need an in-memory database(databases in RAM). 37 | 38 | 9. **Billing**: This is self-explanatory 39 | 40 | ## Some abbreviations you should know: 41 | 42 | 1. **eb**: Elastic beanstalk 43 | 2. **sg**: Security group 44 | 45 | 46 | ## Navigating in AWS Console: 47 | 48 | ### search: 49 | 50 | Search is easy you can just type in the service you want and services click a service you intend to use. 51 | 52 | ![search](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/AWSinterface/search.jpg) 53 | 54 | ### Elastic beanstalk: 55 | 56 | If you had typed Elastic beanstalk in the search bar, you would be presented with the below page: 57 | 58 | ![elastic_beanstalk](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/AWSinterface/elastic_branstalk.jpg) 59 | 60 | Now, you can create a new application by clicking on Application on the left navigation bar. You can also quickly visit your environments from the left navigation panel. An application can have many environments. Each environment should have a unique name. You will know about this as you start with deployment. 61 | 62 | ### EC2: 63 | 64 | This is a very important page and you will often come back here so please note the items in the left navigation panel. 65 | 66 | ![EC2 Console](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/AWSinterface/ec2_console.jpg): 67 | 68 | >Since not all the items are not visible I'll be writing it down important items that you will be using below 69 | 70 | **The important items in the left navigation panels are as follows** 71 | 72 | ``` 73 | . 74 | ├── EC2 Dashboard 75 | ├── EC2 Global View 76 | ├── Events 77 | ├── ... 78 | ├── Instances: 79 | │ ├── Instances (**important** Your current instances that you have created) 80 | │ ├── Instance Type (List of all the instance types like: t1.micro, t2nano, a1.xlarge) 81 | │ └── ... 82 | ├── Images: 83 | │ └── ... 84 | ├── Elastic Block Store: 85 | │ └── ... 86 | ├── Network & Security: 87 | │ ├── Security Groups(**important** A security group acts as a virtual firewall for your EC2 instances to control inbound and outbound traffic) 88 | │ │ 89 | │ └── Key Pairs (A Key pair, consists of a public key and a private key, is a set of security credentials that you use to prove your identity when connecting to an Amazon EC2 instance.) 90 | ├ 91 | ├── Load Balancing: 92 | │ ├── Load Balancers(**important** Displays all your loadbalancers) 93 | │ └── Target Groups(A target group tells a load balancer where to direct traffic to: EC2 instances, fixed IP addresses; or AWS Lambda functions, amongst others.) 94 | ├ 95 | └── Auto Scaling 96 | └── ... 97 | ``` 98 | 99 | 100 | ### RDS Console: 101 | 102 | The most important item in the left panel is Databases highlighted in orange color 103 | 104 | ![RDS console](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/AWSinterface/RDS_dashboard.jpg) 105 | 106 | ### S3: 107 | 108 | The most important item in the left panel is the buckets themselves. 109 | 110 | ![S3 Bucket](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/AWSinterface/s3.jpg) 111 | 112 | The others aren't that difficult so I'll leave it here. 113 | -------------------------------------------------------------------------------- /s3buckets.md: -------------------------------------------------------------------------------- 1 | # Using S3(Simple storage service) to save media files and other files: 2 | 3 | If your users are going to be uploading images such as profiles etc, you can also serve static files from S3 but in this, we will focus on how to serve media files. You should definitely use S3 to store them and serve them directly from S3 instead of Django serving those files. 4 | 5 | ### To set up S3 follow the below instructions: 6 | 7 | 1. Install the following packages and put them in `requirements.txt` file: 8 | ``` 9 | pip install boto3 10 | pip install django-storages 11 | ``` 12 | 13 | `requirements.txt` 14 | ``` 15 | boto3==1.20.26 16 | django-storages==1.12.3 17 | ``` 18 | 19 | add storages in installed apps: 20 | 21 | ```python 22 | INSTALLED_APPS = [ 23 | 'django.contrib.admin', 24 | 'django.contrib.auth', 25 | 'django.contrib.contenttypes', 26 | 'django.contrib.sessions', 27 | 'django.contrib.messages', 28 | 'django.contrib.staticfiles', 29 | 30 | 'storages', # <----- here 31 | 'my-app' 32 | ] 33 | ``` 34 | 35 | django-storages is a library to manage storage backends like Amazon S3, OneDrive, etc. 36 | 37 | boto3 is a public API client to access the AWS resources such as AWS S3. 38 | 39 | 2. Go to S3 by searching for it on AWS console: 40 | 41 | 3. Click on create bucket 42 | 43 | ![Create-bucket](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/S3/S3-buckets-create.jpg) 44 | 45 | 4. Now give the bucket a name of your choice and uncheck block all public access. (make sure the region you are in is the same as in the EC2 instance) 46 | 47 | 5. Now scroll down and click on create bucket. 48 | 49 | ### Creating IAM Role: 50 | 51 | You ideally don't want your django-app to be able to access all the services by AWS for security reasons. So you create an account that has access only to the S3 buckets from the IAM role. 52 | 53 | Now search for IAM in the search bar: 54 | 55 | 1. Click on Users from the left navbar and click on "Add users". 56 | 57 | ![IAM](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/S3/IAM-users.jpg) 58 | 59 | 2. Type in a user name and click on Access-key: programmatic access: 60 | ![programmatic-access](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/S3/programmatic-access.jpg) 61 | 62 | 3. Click on next and under "Add user to group" 63 | click on create group and search for S3 full access, check the check-box. 64 | ![Create-group](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/S3/create-group.jpg) 65 | 66 | 4. Click on review and if everything seems correct then you can click on create user. 67 | 68 | 5. Take note of all the information including the User, Access Key and the Secret Access Key. We will be using this to give Django permission to access the S3 buckets. 69 | 70 | 71 | Now in your `settings.py` add the below(you can add this anywhere in your settings I'll be adding it at the bottom): 72 | 73 | ```python 74 | AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID') 75 | AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY') 76 | AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME') 77 | 78 | AWS_S3_REGION_NAME = os.environ.get('AWS_S3_Region') 79 | AWS_QUERYSTRING_AUTH = False 80 | AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com' 81 | AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'} 82 | AWS_DEFAULT_ACL = 'public-read' 83 | ``` 84 | 85 | You should add the access key id etc in `.env` file and in production you should add them as environment variables. (read the Hiding sensitive information topic). 86 | 87 | Now in the same directory as `settings.py` create a file called `storages.py` and add the following. 88 | 89 | ```py 90 | from storages.backends.s3boto3 import S3Boto3Storage 91 | 92 | class MediaStore(S3Boto3Storage): 93 | location = 'media' 94 | file_overwrite = False 95 | ``` 96 | 97 | now back in `settings.py` add 98 | 99 | ```py 100 | DEFAULT_FILE_STORAGE = '.storage.MediaStore' 101 | ``` 102 | replace `` with your project name. 103 | The above will work if the storages.py is in the same directory as `settings.py` If you have it somewhere else provide the path to that eg: `app.storages.MediaStore`. 104 | 105 | 106 | ### Creating bucket policy 107 | 108 | Once you are done with the setup, go back to s3 in aws console. Click on your bucket -> permissions tab. Now set "block all public access" to off. Now edit the bucket policy and set the below code to allow only read access 109 | ```json 110 | { 111 | "Version": "2012-10-17", 112 | "Statement": [ 113 | { 114 | "Effect": "Allow", 115 | "Principal": "*", 116 | "Action": [ 117 | "s3:GetObject", 118 | "s3:GetObjectAcl" 119 | ], 120 | "Resource": [ 121 | "arn:aws:s3:::media-name", 122 | "arn:aws:s3:::media-name/*" 123 | ] 124 | } 125 | ] 126 | } 127 | ``` 128 | 129 | replace `arn:aws:s3:::media-name` with the arn you get when you click on edit bucket policy. 130 | 131 | ### Cross-origin resource sharing (CORS) 132 | 133 | Sometimes the browser might report cors error so scoll down in the permission tab to the CORS policy and add the following: 134 | ```json 135 | [ 136 | { 137 | "AllowedHeaders": [ 138 | "Authorization" 139 | ], 140 | "AllowedMethods": [ 141 | "GET", 142 | "HEAD" 143 | ], 144 | "AllowedOrigins": [ 145 | "http://mydomain.com", 146 | "https://mydomain.com" 147 | ], 148 | "ExposeHeaders": [ 149 | "Access-Control-Allow-Origin" 150 | ] 151 | } 152 | ] 153 | ``` 154 | 155 | That's it, it should now be up and running. 156 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # Deploying a web application to AWS: Django, Django Channels 2 | 3 | Deploying your project to AWS can be difficult and frustrating if you are new to deploying a project. Neither is AWS user interface easy to work with nor is their documentation easy to understand. 4 | 5 | I recently had to redeploy a Django application that made use of WebSockets and it took me around 8 days to get it properly up and running. Most of the errors occurred because I didn't understand their system well. 6 | 7 | In this blog, I wish not only to help you deploy your Django project but also to help you understand AWS better. I will also talk about some of the common errors that I faced during the deployment. 8 | 9 | While I'll be using Django, most of the steps followed will remain the same across different frameworks and programming languages. 10 | 11 | >Note: If think some steps are missing or is incorrect please create a new issue on this GitHub repository or create a new pull request. 12 | 13 | > **Note:** Since the release of Amazon Linux 2023 (python 3.9 and above), amazon has removed some of the packages, you don't need those packages any longer. So can skip those in the configuration files. 14 | 15 |
16 | 17 | ## Help me deploy to AWS? 18 | Since many people have been asking me to help them in deployment, I have created it as a service, If you want me to help you deploy Django channels, Please Create a [discussion](https://github.com/PaulleDemon/AWS-deployment/discussions) or 19 | Send a message on the [Discord server](https://discord.gg/VtyfrAhD), I'll DM you there, (Standard charges apply) 20 | 21 | **Note**: 22 | For me to help you will have to provide me necessary permission through IAM access, if your company policy doesn't allow you to provide access, I have also created a video recording, you can check it out [here](https://www.buymeacoffee.com/artpaul/e/241689) 23 | 24 |
25 | 26 | ### Table of contents: 27 | 28 | 1. [Introduction to AWS interface](https://github.com/PaulleDemon/AWS-deployment/blob/master/AWS_Interface.md) 29 | 30 | 1. Introduction 31 | 2. Commonly used services 32 | 3. Some abbreviation you should know 33 | 4. Navigating in AWS Console 34 |
35 | 1. Search 36 | 2. Elastic beanstalk 37 | 3. EC2 38 | 4. RDS 39 | 5. S3 40 | 41 | 2. [Deploying our Django application](https://github.com/PaulleDemon/AWS-deployment/blob/master/deploying_django.md) 42 | 43 | 1. Deployment flow 44 | 2. Multiple ways to deploy an application 45 | 3. Setup 46 | 4. Creating an application using the AWS console. 47 | 5. Using EB CLI to deploy an application 48 |
49 | 1. Creating an application 50 | 2. Creating an environment 51 | 6. Debugging errors 52 | 53 | 54 | 3. [Using RDS to create database](https://github.com/PaulleDemon/AWS-deployment/blob/master/connecting_RDS.md) 55 | 56 | 1. Introduction 57 | 2. setting up RDS 58 | 3. Connecting to pgAdmin 59 | 4. Setting up RDS in our project 60 | 5. Testing connection on an EC2 instance (DEBUGGING) 61 | 6. Migrations 62 | 63 | 64 | 4. [Using Environment variables to hide our sensitive info's](https://github.com/PaulleDemon/AWS-deployment/blob/master/UsingEnvironment.md) 65 | 66 | 1. Introduction. 67 | 2. Hiding your sensitive information in development 68 | 3. Setting environment variables in production: 69 | 70 | 71 | 5. [Using S3 storage to store static and media files.](https://github.com/PaulleDemon/AWS-deployment/blob/master/s3buckets.md) 72 | 1. Introduction. 73 | 2. Setting up a project for S3 and creating a bucket. 74 | 3. Creating IAM Role 75 | 76 | 77 | 6. [Redirecting Http to Https](https://github.com/PaulleDemon/AWS-deployment/blob/master/redirectHttps.md) 78 | 79 | 7. [Deploying Django channels application](https://github.com/PaulleDemon/AWS-deployment/blob/master/django-channels.md) 80 | 81 | 1. Setup 82 | 2. Connecting Redis: 83 | 84 | 85 | 8. [Using Route53 to connect to custom domain](https://github.com/PaulleDemon/AWS-deployment/blob/master/Route53.md) 86 | 87 | 1. Connecting your custom domain to elasticbeanstalk using Route53. 88 | 2. Configuring your custom domain email to send mails 89 | 3. Hosting a static website 90 | 91 | 92 | 9. [some debugging tips](https://github.com/PaulleDemon/AWS-deployment/blob/master/debugging-tips.md) 93 | 1. Introduction 94 | 2. Checklist 95 | 3. Debugging tips 96 | 4. Commonly used commands. 97 | 1. Eb cli commands 98 | 2. Linux commands 99 | 100 | ### Additional information: 101 | 102 | 1. [Security Groups](https://github.com/PaulleDemon/AWS-deployment/blob/master/SecurityGroups.md) 103 | 2. [IAM](https://github.com/PaulleDemon/AWS-deployment/blob/master/IAM.md) 104 | 105 |
106 | 107 | References: 108 | 1. https://docs.aws.amazon.com/ 109 | 2. StackOverflow answers 110 | 3. https://blog.zishanahmad.dev/how-to-deploy-django-channels-2x-on-aws-elastic-beanstalk-amazon-linux-2#comments-list 111 | 4. https://medium.com/@elspanishgeek/how-to-deploy-django-channels-2-x-on-aws-elastic-beanstalk-8621771d4ff0 112 | 113 | 114 | **Support open-source:** 115 | 116 | Like the article? consider a one-time donation 117 | 118 | [Donate](https://www.buymeacoffee.com/ArtPaul) 119 | 120 | [](https://www.buymeacoffee.com/ArtPaul) 121 | 122 | 123 | **Keywords** 124 | 1. How to deploy Django to AWS? 125 | 2. How to deploy Django Channels to AWS? 126 | 3. How to deploy Django to cloud? 127 | 4. AWS deployment help. 128 | 5. Redis connection for Django. 129 | 6. How to host Django WebSockets? 130 | 7. AWS Deployment help 131 | 8. Django AWS deployment help 132 | 9. Django Channels AWS deployment help 133 | -------------------------------------------------------------------------------- /UsingEnvironment.md: -------------------------------------------------------------------------------- 1 | # Setting you allowed host and other sensitive information in environment variables. 2 | 3 | If you are working in production or even in development you don't want to expose your hostname, allowed hosts, passwords, etc, by placing them inside your settings. 4 | 5 | ## Hiding your sensitive information in development 6 | 7 | In development you will create a `.env` file where you will store all the important passwords, endpoints to the database, etc. You will add this file to `.gitignore` and will not commit this file or put this up in a public repository. 8 | 9 | The `.env` file should look something like the one below 10 | ``` 11 | RDS_HOST=xyz.region-xyq.aws.database.com 12 | RDS_PASSWORD=xyzwie129 13 | RDS_PORT=5432 14 | 15 | AWS_KEY=wewefwef 16 | AWS_SECRET_KEY=wefwfef 17 | ``` 18 | You can change the names of the keys and values as per your requirements 19 | 20 | > Note: This `.env` file should be in the same directory level as other app directories. 21 | eg: 22 | >``` 23 | >. 24 | >└── sampleproject 25 | > ├── .ebextensions 26 | > ├── .env <-------- here 27 | > ├── app1 28 | > ├── sampleproject 29 | > │ ├── __init__.py 30 | > │ ├── settings.py 31 | > │ ├── urls.py 32 | > │ ├── wsgi.py 33 | > │ └── asgi.py 34 | > ├── app2 35 | > └── ... 36 | >``` 37 | 38 | Now to use go to settings.py copy all the sensitive information such as RDS Host, password, username, etc, and add it to `.env` file in the above-shown format. 39 | 40 | Now use pip install and install `django-dotenv` and add it to `requirements.txt` 41 | ``` 42 | pip install django-dotenv 43 | ``` 44 | 45 | Now open manage.py file and add the following line under main `dotenv.read_dotenv()` don't forget to import dotenv 46 | 47 | ```python 48 | import os 49 | import sys 50 | import dotenv 51 | 52 | def main(): 53 | """Run administrative tasks.""" 54 | dotenv.read_dotenv() # <----------------- here 55 | os.environ.setdefault('DJANGO_SETTINGS_MODULE', '.settings') 56 | try: 57 | from django.core.management import execute_from_command_line 58 | ... 59 | ``` 60 | 61 | now go to settings and replace all the sensitive information with `os.getenv(KEY)` or `os.environ.get(KEY)`. `os.environ` return a dictionary of key-value pairs `os.getenv` is simply a wrapper around `os.environ.get(KEY)` Here the key is the name you assigned on the Left-hand side of the = in `.env` file 62 | 63 | .env 64 | ``` 65 | RDS_HOST=aws.wedkwed.com 66 | ``` 67 | So here RDS_HOST is your key. 68 | 69 | ## Setting environment variables in production: 70 | 71 | When you are uploading to production you are less likely to upload your `.env` file, you will instead set your RDS_HOST, RDS_PASSWORD, and other sensitive information as an environment variable. 72 | 73 | There are multiple ways to set the environment variable. I'll be showing you two ways. 74 | 75 | ### 1. First Way 76 | 77 | The first way is to set the environment variable directly in the EC2 instance. 78 | You can do so by logging into your EC2 instance using `eb ssh` 79 | 80 | Then you can set the environment variables using the Linux commands. 81 | Eg: 82 | ```sh 83 | export Key1=value1 Key2=value2 KeyN=valueN 84 | ``` 85 | 86 | I personally haven't used this method myself. 87 | 88 | ### 2. Second Way: 89 | 90 | GO to ElasticBeanstalk on AWS console -> Configurations -> Software -> Edit 91 | 92 | Now scroll down until you see "Environment properties": 93 | 94 | ![Environment properties](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/environment-properties.jpg) 95 | 96 | Now you can start filling in the name and value as I have shown above. (Please use your own hostname, password, etc) 97 | 98 | Once you have set all the information you can click on Apply. 99 | 100 | It will take some time to apply. If everything went right your status should be ok. If something went wrong during the process of updating. It will reset back to the previous configuration. You can check the events under Recent events. 101 | 102 | 103 | You can also set this using EB CLI using the below command. 104 | ``` 105 | eb setenv Key1=value1 Key2=value2 KeyN=valueN 106 | ``` 107 | ### Note: 108 | 109 | If you are using the second method it's likely that your environment variables are not set as you expect. 110 | 111 | So add the below code in your `settings.py`: 112 | 113 | ```python 114 | import os 115 | import ast 116 | import subprocess 117 | from pathlib import Path 118 | 119 | BASE_DIR = Path(__file__).resolve().parent.parent 120 | 121 | ... 122 | 123 | # SECURITY WARNING: don't run with debug turned to True in production! 124 | DEBUG = False 125 | 126 | def get_environ_vars(): # This is because the environment variables are set in this path in elasticbeanstalk 127 | # This method returns a dictionary of key value pairs 128 | completed_process = subprocess.run( 129 | ['/opt/elasticbeanstalk/bin/get-config', 'environment'], 130 | stdout=subprocess.PIPE, 131 | text=True, 132 | check=True 133 | ) 134 | 135 | return ast.literal_eval(completed_process.stdout) 136 | 137 | 138 | ENV_VARS = None 139 | 140 | if not DEBUG: 141 | 142 | ENV_VARS = get_environ_vars() 143 | 144 | ... 145 | ``` 146 | Source for the above code: https://stackoverflow.com/a/64528198/15993687 147 | 148 | > Note: `get_enviton_vars()` returns dictinoary of key value pairs so you can use any dictinoary methods such as `.get()` on it. 149 | 150 | Example usage in `settings.py`: 151 | 152 | ```python 153 | DATABASES = { 154 | 'default': { 155 | 'ENGINE': 'django.db.backends.postgresql', 156 | 'NAME': 'postgres', 157 | 'USER': ENV_VARS['USER'], 158 | 'PASSWORD': ENV_VARS['RDS_PASSWORD], 159 | 'HOST': ENV_VARS['RDS_HOST'], 160 | 'PORT': ENV_VARS['RDS_PORT'] 161 | } 162 | } 163 | ``` 164 | 165 | > Note: The above is needed only if the environment variable is not set as expected. Otherwise use `os.getenv()`. 166 | -------------------------------------------------------------------------------- /django-channels.md: -------------------------------------------------------------------------------- 1 | # Django-channels: 2 | 3 | If you are a beginner please make sure you have visited the uploading Django. There are important concepts you need to know before proceeding. 4 | 5 | Make sure you are using an Application load balancer and not a classic loadbalancer. 6 | Classic Load balancers don't support WebSockets. 7 | 8 | You can check your loadbalancer type under LoadBalancer from configurations in the environment. 9 | 10 | If you are using classic loadbalancer you will have to create a new environment with an application loadbalancer as there is no way you can upgrade from classic loadbalancer to application loadbalancer as of Feb-2-2022. 11 | 12 | ## setup 13 | 14 | 1. First add the following to your `requiremet.txt`: 15 | ``` 16 | gunicorn==20.0.4 17 | daphne==3.0.1 18 | supervisor==4.2.2 19 | ``` 20 | 21 | > Daphne is a HTTP, HTTP2 and WebSocket protocol server for ASGI and ASGI-HTTP, developed to power Django Channels. we'll be using this to run our ASGI application 22 | 23 | > The Gunicorn "Green Unicorn" is a Python Web Server Gateway Interface HTTP server. 24 | 25 | 2. create a file named `Procfile` and add the following: 26 | ``` 27 | web: gunicorn --bind :8000 --workers 3 --threads 2 .wsgi:application 28 | websocket: daphne -b 0.0.0.0 -p 5000 .asgi:application 29 | ``` 30 | replace the project with your project name. 31 | 32 | Now your structure should look below. 33 | ``` 34 | . 35 | └── sampleproject 36 | ├── .ebextensions 37 | │ └── django.config 38 | ├── app1 39 | ├── app2 40 | ├── sampleproject 41 | │ ├── __init__.py 42 | │ ├── settings.py 43 | │ ├── urls.py 44 | │ ├── wsgi.py 45 | │ └── asgi.py 46 | ├── app3 47 | ├── Procfile #<------------here 48 | └── .gitignore 49 | ``` 50 | 51 | 52 | 3. Goto elasticbeanstalk and in sidebar click on configuration. If you have an appication loadbalancer. Go to environment -> loadbalancer -> edit: 53 | 54 | 4. If you want to use HTTPS and have an SSL certificate then click on add listener and fill in the port number to be 443 choose your SSL certificate and choose any SSL policy from the dropdown. 55 | 56 | ![HTTPS](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/ElastiCache/create-https-loadbalancer.png) 57 | 58 | 5. Then go ahead and add processes and match the below settings. (make sure you enable stickiness in port 5000) 59 | ![process](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/ElastiCache/loadbalancer-config.jpg) 60 | 61 | Your health check might fail with "100% of GET request failing with 4xx". That's why I gave a different health check path that returns 200 Ok status. 62 | 63 | 6. Now under Rules match the below (if you aren't using HTTPS then change the listener port to 80) the process is the name you gave, this will make sure to redirect all the path that has `/ws/` is redirected to the websocket process : 64 | ![rules](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/ElastiCache/rules.jpg) 65 | 66 | >Note: If you are using HTTPS protocol then you should use wss protocol eg: wss://ws/xys otherwise use ws://ws/pqr 67 | 68 | ### Way 2: 69 | If you haven't created your environment yet you can just add the following in `01_loadbalancer.config` under `.ebextensions` 70 | Note you should also follow till step 2 from the above. 71 | ``` 72 | option_settings: 73 | aws:elbv2:listener:443: 74 | ListenerEnabled: 'true' 75 | SSLCertificateArns: arn: 76 | Protocol: HTTPS 77 | Rules: ws 78 | aws:elbv2:listenerrule:ws: 79 | PathPatterns: /ws/* 80 | Process: websocket 81 | Priority: 1 82 | aws:elasticbeanstalk:environment:process:websocket: 83 | Port: '5000' 84 | Protocol: HTTP 85 | ``` 86 | 87 | If you don't want to use HTTPS you can remove the 88 | ``` 89 | aws:elbv2:listener:443: 90 | ListenerEnabled: 'true' 91 | SSLCertificateArns: arn: 92 | Protocol: HTTPS 93 | Rules: ws 94 | ``` 95 | 96 | If you are using HTTPS and have an SSL certificate replace the SSL certificate arn with your SSL certificate ARN. 97 | 98 | 99 | ## Connecting Redis: 100 | 101 | It's highly likely you want an in-memory database such as Redis. 102 | 103 | ### 1. Creating Redis through AWS console. 104 | 105 | 1. Search for ElastiCache and in elasticcache dashboard click on create. 106 | ![elastic cache](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/ElastiCache/elasticache.jpg) 107 | 108 | 2. Let cluster engine be Redis. Now under Redis settings give your cluster a name and select a node type(note: only `cache.t2.micro` and `cache.t3.micro` comes under free tire (if you have a free tire account)) 109 | ![redis create](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/ElastiCache/redis-create.png) 110 | 111 | > If you are eligible for free tire make sure you have set the node type to `cache.t2.micro` or `cache.t3.micro`. Set number of replicas to 0 disable multi A-Z, disable automatic backups. 112 | 113 | Now click on create. 114 | 115 | Once created copy the primary endpoint and in settings.py: 116 | 117 | ```py 118 | ... 119 | CHANNEL_LAYERS = { 120 | 'default': { 121 | 'BACKEND': 'channels_redis.core.RedisChannelLayer', 122 | 'CONFIG': { 123 | "hosts": [('', 6379)], 124 | }, 125 | }, 126 | } 127 | ... 128 | ``` 129 | You will also have to create a [security group](https://github.com/PaulleDemon/AWS-deployment/blob/master/SecurityGroups.md) and associate it with the ec2 instance. 130 | 131 | 1. Go to Ec2 > security groups > click on create security group. 132 | 133 | 2. Now give it a name say, _redis, add a small description.(optional) 134 | 135 | 3. Now under inbound rules, click on add inbound rule. 136 | 137 | 4. set type to `custom TCP`, port to 6379 (redis port), set source to custom and search for your environment security group and set it to that. 138 | 139 | 5. Go to the bottom of this page to see how to test redis connection. 140 | 141 | ### 2. Connecting through config file(recommended) 142 | 143 | create a file called `elastic_cache.config` in `.ebextensions` folder and paste the below code: 144 | 145 | ``` 146 | Resources: 147 | MyCacheSecurityGroup: 148 | Type: "AWS::EC2::SecurityGroup" 149 | Properties: 150 | GroupDescription: "Lock cache down to webserver access only" 151 | SecurityGroupIngress : 152 | - IpProtocol : "tcp" 153 | FromPort : 154 | Fn::GetOptionSetting: 155 | OptionName : "CachePort" 156 | DefaultValue: "6379" 157 | ToPort : 158 | Fn::GetOptionSetting: 159 | OptionName : "CachePort" 160 | DefaultValue: "6379" 161 | SourceSecurityGroupName: 162 | Ref: "AWSEBSecurityGroup" 163 | MyElastiCache: 164 | Type: "AWS::ElastiCache::CacheCluster" 165 | Properties: 166 | CacheNodeType: 167 | Fn::GetOptionSetting: 168 | OptionName : "CacheNodeType" 169 | DefaultValue : "cache.t2.micro" 170 | NumCacheNodes: 171 | Fn::GetOptionSetting: 172 | OptionName : "NumCacheNodes" 173 | DefaultValue : "1" 174 | Engine: 175 | Fn::GetOptionSetting: 176 | OptionName : "Engine" 177 | DefaultValue : "redis" 178 | VpcSecurityGroupIds: 179 | - 180 | Fn::GetAtt: 181 | - MyCacheSecurityGroup 182 | - GroupId 183 | 184 | Outputs: 185 | ElastiCache: 186 | Description: "ID of ElastiCache Cache Cluster with Redis Engine" 187 | Value : 188 | Ref : "MyElastiCache" 189 | ``` 190 | 191 | Now as shown above copy the primary endpoint and replace it under host. 192 | 193 | 194 | ### Debugging tips: 195 | 196 | To test the connection first ssh into your instance. 197 | 198 | then type 199 | ``` 200 | source /var/app/venv/*/bin/activate 201 | cd /var/app/current/ 202 | ``` 203 | now type `python manage.py shell` and give the following command 204 | 205 | ``` 206 | >>> import channels.layers 207 | >>> from asgiref.sync import async_to_sync 208 | >>> channel_layer = channels.layers.get_channel_layer() 209 | >>> async_to_sync(channel_layer.send)('test_channel', {'foo': 'bar'}) 210 | >>> async_to_sync(channel_layer.receive)('test_channel') 211 | >>> {'foo': 'bar'} # <---------- you should receive this as output if everything went well 212 | ``` 213 | 214 | If you receive connection timeout, its most likely related to your security group. 215 | 216 | Daphne will run your WebSockets connection on port 5000. you can manually start this by going to your current directory activating your environment in your EC2 instance and running 217 | ``` 218 | daphne -b 0.0.0.0 -p 5000 projectname.asgi:application 219 | ``` 220 | replace projectname with your project name. 221 | 222 | full steps: 223 | ```sh 224 | eb ssh 225 | Enter your passphrase: 226 | 227 | ---- Elastic Beanstalk---- 228 | 229 | source /var/app/venv/*/bin/activate 230 | cd /var/app/current/ 231 | daphne -b 0.0.0.0 -p 5000 projectname.asgi:application 232 | ``` 233 | -------------------------------------------------------------------------------- /connecting_RDS.md: -------------------------------------------------------------------------------- 1 | # Connecting to RDS(Relational Database Service): 2 | 3 | If you are going to be working in production you definitely want to ditch the SQLite database that you have been working with and use a database such as Postgre SQL or MySQL. 4 | 5 | RDS provides you with a simple and easy setup to connect to an external database. This database won't be inside our project directory but outside our application. We will connect to the database using the endpoint that is provided by AWS. 6 | 7 | We will be using the AWS console to set up our RDS. 8 | 9 | ## Setting up RDS: 10 | 11 | Now go to AWS console and search for RDS 12 | 13 | If you are currently in the RDS dashboard(see the left nav panel) 14 | 15 | Before creating a Database make sure you are in the same region as the region you selected when creating your application using `eb init` or it's in the same region as the environment region. 16 | ![region](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/RDS/region.jpg) 17 | 18 | In my case it's Us-west-2(oregon). If it's not in the same region click on the drop-down select the correct region. 19 | 20 | you can scroll down to Create database and click on create database. 21 | 22 | ![dashboard-create](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/RDS/rds-dashboard-create.jpg) 23 | 24 | If you are currently in databases you can simply click on create database if you see that. 25 | 26 | ![Create database](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/RDS/create-database.jpg) 27 | 28 | 1. Choose Standard create under database creation method. 29 | 2. Choose your Engine type, I'll be going with postgresSQL 12.8. 30 | 3. Choose Template. I'll be choosing free tire. You can choose production or dev/test if you don't have free tire(free database usage for 1st year). 31 | 32 | Scroll down to settings. 33 | ![settings](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/RDS/db-settings.jpg) 34 | 35 | Fill in database identifier. You can choose a name of your choice. 36 | 37 | Choose a password and username you can remember. I'll be leaving the username as the default(Postgres). 38 | 39 | 4. Now scroll down and under connectivity section -> Public access -> click on Yes. 40 | 41 | You won't ideally need to keep public access to yes. But if you want to access your database from your computer or from PgAdmin on your computer, you will have to keep this yes. You can change this later using [security groups](https://github.com/PaulleDemon/AWS-deployment/blob/master/SecurityGroups.md). 42 | 43 | Don't change anything else. You can scroll down and click on create-database. Your database creation can take a few minutes to create. 44 | 45 | Once created the status will be changed to available(view this in databases panel). 46 | 47 | Now click on your database and copy the endpoint 48 | 49 | ![Endpoint](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/RDS/database-endpoint.jpg) 50 | 51 | Don't share this endpoint. 52 | 53 | ### Connecting to pgAdmin: 54 | 55 | You can skip this if you don't want to access your database through pg-admin. 56 | 57 | Open your pgAdmin, I am using pgAdmin4. 58 | 59 | Now in pg-admin: 60 | 61 | 1. object(on the top header) -> create -> server group -> give a name for the server group. 62 | 2. Right-click on your server group -> create -> server. 63 | 3. A create server dialog should pop up. Specify a name for the server, then in the same dialog click on "connection" tab on the top. 64 | 4. Now paste the endpoint you copied to the hostname/address. Give the user name and password you created when creating the database. (Make sure you are in port 5432. If you had chosen another port when creating the database then change the port in the pgadmin too). 65 | 5. That's it you now have your database connected to pgadmin. You can now view your tables directly from here. 66 | 67 | ### Errors: 68 | If you get timeout errors even after having a proper internet connection. It's likely you haven't given public access database. You can change this later in the security group of the database. 69 | 70 | 71 | > Note: Public access to the database is quite insecure. If someone has your endpoint, username and password. They will be able to connect to your database from any part of the world. We will later update the security group to allow access only to particular IP addresses in [`security groups`](https://github.com/PaulleDemon/AWS-deployment/blob/master/SecurityGroups.md). 72 | 73 | Now that your pgadmin is up and running lets now setup the database in our project: 74 | 75 | ## Setting up RDS in our project: 76 | 77 | First add `psycopg2-binary==2.9.2` to `requirements.txt`. 78 | 79 | Open your settings.py and under databases change the below accordingly. 80 | 81 | ```python 82 | DATABASES = { 83 | 'default': { 84 | 'ENGINE': 'django.db.backends.postgresql', 85 | 'NAME': 'postgres', 86 | 'USER': '', 87 | 'PASSWORD': '', 88 | 'HOST': '', 89 | 'PORT': '5432' 90 | } 91 | } 92 | ``` 93 | Fill in the username password and host. Please don't commit this to pubic repositories yet. There are other ways to hide your password, host and port using `.env` file which we will be discussing later. 94 | 95 | > Tip: If you want to continue using SQLite as your devlopment database. You can try something as shown below in settings.py 96 | >```python 97 | > if DEBUG: 98 | > DATABASES = { 99 | > 'default': { 100 | > 'ENGINE': 'django.db.backends.sqlite3', 101 | > 'NAME': BASE_DIR / 'db.sqlite3', 102 | > } 103 | > } 104 | > else: 105 | > DATABASES = { 106 | > 'default': { 107 | > 'ENGINE': 'django.db.backends.postgresql', 108 | > 'NAME': 'postgres', 109 | > 'USER': '', 110 | > 'PASSWORD': '', 111 | > 'HOST': '', 112 | > 'PORT': '5432' 113 | > } 114 | > } 115 | >``` 116 | 117 | That's it now you can save the `settings.py` and in your EB CLI `eb deploy` (make sure your eb cli's path is in the project directory eg: `(eb-env) c:\\path\PhotoProject>`) 118 | 119 | 120 | ## Testing connection on EC2 instance (DEBUGGING): 121 | 122 | If you ever want to check if your project is being able to connect to the database. 123 | 124 | using eb cli connect to your EC2 instance using 125 | 126 | ``` 127 | eb ssh 128 | ``` 129 | It will then ask for a passphrase. Give the passphrase you set up during ssh creation. If you didn't set up ssh you can create it using `eb ssh --setup`. 130 | 131 | now navigate to your project directory in the Ec2 using: 132 | ``` 133 | cd /var/app/current/ 134 | ``` 135 | 136 | activate your virtual environment using 137 | ``` 138 | source /var/app/venv/*/bin/activate 139 | ``` 140 | 141 | > Note: these paths might change in the future if AWS team decides to change them. 142 | 143 | Now to check if it is connected just use: 144 | ``` 145 | python manage.py check --database default 146 | ``` 147 | 148 | If its successful you should get 149 | ``` 150 | System check identified no issues (0 silenced). 151 | ``` 152 | 153 | ### Errors 154 | 155 | If you get 156 | ``` 157 | Traceback (most recent call last): 158 | File "", line 1, in 159 | psycopg2.OperationalError: could not connect to server: No such file or directory 160 | Is the server running locally and accepting 161 | connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? 162 | ``` 163 | It means that the database was not added correctly or you haven't added hostname. Try rechecking the hostname. 164 | 165 | In python, you can check the hostname by using the below code in your shell 166 | 167 | ```python 168 | >>> from django.conf import settings 169 | >>> settings.DATABASES['default']['HOST'] 170 | >>> 'awed.aws.com' 171 | >>> settings.DATABASES['default']['PORT'] 172 | >>> '5432' 173 | ``` 174 | 175 | 176 | ## Migrations: 177 | 178 | If had set public access you should be simply able to run : 179 | ``` 180 | python manage.py migrate 181 | ``` 182 | directly from your development environment. (make sure if you have set the right database in `settings.py`) 183 | 184 | You can also log in to EC2 instance using ssh and perform migration(not recommended). 185 | 186 | If the database is not accessible from your development environment follow the recommended way for migration as stated below. 187 | 188 | ### Recommened migration steps: 189 | 190 | In your `.ebextensions` folder create a file called `db-migrate.config`(the name could be whatever you want). 191 | 192 | Then paste the following command 193 | ``` 194 | container_commands: 195 | 01_migrate: 196 | command: "source /var/app/venv/*/bin/activate && python3 manage.py migrate" 197 | leader_only: true 198 | option_settings: 199 | aws:elasticbeanstalk:application:environment: 200 | DJANGO_SETTINGS_MODULE: PhotoProject.settings 201 | ``` 202 | 203 | In the above replace the `PhotoProject` with your application name where the settings.py resides. 204 | 205 | This will run `python manage.py migrate` every time you deploy on EC2 instance. 206 | 207 | 208 | ### Additional information for django users. 209 | 210 | loaddata and dumpdata can be useful when you need to quickly load and unload data from your database from a specifed format such as JSON, XML etc. 211 | 212 | **dump data** 213 | If you ever need to serialize the data from the database to JSON or any other format you can use the following commands provided by django 214 | 215 | `python -Xutf8 manage.py dumpdata > mydata.json --indent 4` 216 | 217 | The above command will serialize all the data from the database to JSON format with 4 indent, the Xutf8 specifies the encoding. 218 | 219 | You can also be more specific on which apps you would like to serialize by specifying the appe name eg: 220 | 221 | `python -Xutf8 manage.py dumpdata app1 > mydata.json --indent 4` 222 | 223 | The above will serialize models only from the app1. 224 | 225 | > Note: If you get the error `CommandError: Unable to serialize database: cursor "_django_curs_xxxxxx_66" does not exist` it usually means that the table in the database and model doesn't match. 226 | > Django uses your models to serialize data, so if you had made any changes to your model revert back and the dumpdata again. 227 | 228 | **Load data** 229 | 230 | You can use the below command to provide an initial data to the database. 231 | 232 | `python manage.py loaddata mydata.json` 233 | 234 | The loaddata will deserialize the data and insert it to your database. 235 | 236 | >Note: If you already have some data in your data base, then make sure to comment out post_save methods, if you are using that to create additional fields. Otherwise it might sometimes create duplicate elements and your loaddata will fail. 237 | 238 | 239 | Additional reference: 240 | 241 | `dumpdata`: https://docs.djangoproject.com/en/4.0/ref/django-admin/#dumpdata 242 | 243 | `loaddata`: https://docs.djangoproject.com/en/4.0/ref/django-admin/#loaddata -------------------------------------------------------------------------------- /deploying_django.md: -------------------------------------------------------------------------------- 1 | # Deploying Django to AWS using ElasticBeanstalk (EB): 2 | 3 | There are many services provided by AWS and it can become challenging, in the beginning, to figure out which of them to use. So to make deploying web applications easier AWS put together some of the common services that are used while deploying a web app and brought it under Elastic beanstalk. 4 | 5 | When using Elastic beanstalk we first create an Application in Elastic beanstalk, upload our project in the form of an application bundle, the Elastic beanstalk then takes care of creating our environment and creates configurations required to run our code. 6 | 7 | We will be using the same to deploy our web application. 8 | 9 | > Note: Before you move on please set up an Access by logging in as root user from the IAM console. This will be used when you are using EB CLI to allow EB CLI (command line interface) to manage the access. You can follow this guide to set this up: https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys 10 | > or you can refer to any other tutorial. 11 | 12 | ## Deployment flow: 13 | 14 | 1. Create an Application in Elastic beanstalk. 15 | 2. Create an environment under the above-created application. 16 | 3. Upload your application. 17 | 18 | ## There are multiple ways to deploy an application (some are stated below): 19 | 1. Using the AWS console. 20 | 2. Using the command-line interface (recommended) 21 | 22 | Before proceeding I suggest you download EB CLI by following the instruction from here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install-advanced.html 23 | 24 | This will help us run important commands in the future. 25 | 26 | I'll be using eb-cli to deploy but will also show you how to use AWS console, 27 | 28 | The project I'll be deploying is called PhotoApp. It is basically a simple web app that allows users to upload photos. 29 | 30 | Here is the link to that project you can refer to this at later point in time: [PhotoApp](https://github.com/PaulleDemon/DjangoPhotoApp) 31 | 32 | Before you deploy I am assuming your project directory structure looks like something shown below: 33 | 34 | ``` 35 | . 36 | └── sampleproject 37 | ├── app1 38 | ├── app2 39 | ├── sampleproject 40 | │ ├── __init__.py 41 | │ ├── settings.py 42 | │ ├── urls.py 43 | │ ├── wsgi.py 44 | │ └── asgi.py 45 | ├── app3 46 | └── ... 47 | ``` 48 | 49 | ### Setup(Common for both AWS console and eb-cli): 50 | 51 | Before deploying there are some more things we need to add to our project. 52 | 53 | Create a directory named `.ebextensions` inside our project folder and add the following script inside the `.ebextensions` folder. You can name the file as `django.config` as an example. (note it should have .config extension) 54 | ``` 55 | option_settings: 56 | aws:elasticbeanstalk:container:python: 57 | WSGIPath: sampleproject.wsgi:application 58 | ``` 59 | Replace the sampleproject with the name of your project. 60 | 61 | This setting, WSGIPath, specifies the location of the WSGI script that Elastic Beanstalk uses to start your application. 62 | 63 | Now if you haven't created a `requirements.txt` create it and use `pip freeze > requirements.txt` to copy the package names from your local virtual environment. Elastic Beanstalk uses this file to determine which packages to install on EC2 instance running your application 64 | 65 | Make sure that these are not added to `.gitignore` 66 | 67 | So now your directory should look like this: 68 | 69 | ``` 70 | . 71 | └── sampleproject 72 | ├── .ebextensions 73 | │ └── django.config 74 | ├── app1 75 | ├── app2 76 | ├── sampleproject 77 | │ ├── __init__.py 78 | │ ├── settings.py 79 | │ ├── urls.py 80 | │ ├── wsgi.py 81 | │ └── asgi.py 82 | ├── app3 83 | ├── requirements.txt 84 | └── .gitignore 85 | ``` 86 | **Setting up static files for production:** 87 | As soon as you set `DEBUG=False`, django will no longer serve your static files(css, js files). So before you send it to AWS there are a couple of things you need to configure in settings.py. 88 | 89 | 1. set `STATIC_ROOT = BASE_DIR.joinpath('static')` 90 | 2. set `STATIC_URL = '/static/'` 91 | 3. set `STATICFILES_DIRS = []`, in the list mention all the paths to your static files that are outside static folder in your apps; as by default django looks into static folder inside your django app you don't need to mention it in this list. 92 | 93 | Whats happening? Well all that the above steps does is that django will collect and place all the static files into a folder mentioned in `STATIC_ROOT = BASE_DIR.joinpath('static')`, so that it doesn't have to keep searhing every folder for your static files in your project during production. 94 | 95 | Now with that done you still need to add one more thing to `django.config` file 96 | ``` 97 | aws:elasticbeanstalk:environment:proxy:staticfiles: 98 | /static: static 99 | value: static/ 100 | ``` 101 | 102 | So now your `django.config` file should look as such. 103 | ``` 104 | option_settings: 105 | aws:elasticbeanstalk:environment:proxy:staticfiles: 106 | /static: static 107 | value: static/ 108 | 109 | aws:elasticbeanstalk:container:python: 110 | WSGIPath: sampleproject.wsgi:application 111 | ``` 112 | 113 | 114 | > Note you can choose to add a file called `.ebignore` in which you can type in all the files and folders that you don't want to be uploaded when using the command-line interface to upload. If you have `.gitignore` you won't require `.ebignore`, but if you do add `.ebignore` your `.gitignore` won't be read and only the files and directories specified in `.ebignore` will be ignored. 115 | 116 | ## 1. Creating an application using the AWS console. 117 | This is one of the easiest way to set your Django project up and running. 118 | 119 | Go to AWS console and search for ElasticBeanstalk: 120 | 121 | ![Create Application](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/CreateEBApplication.jpg) 122 | 123 | 1. Click on "Create a new application" on the top right-hand corner next to actions. 124 | 125 | ![Type application name](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/createEbApp2.jpg) 126 | 2. Type in an application name and description(optional) and then click on create. 127 | 128 | I'll name it sample application and give a sample description. 129 | 130 | That's it, your new application has been created. 131 | 132 | But it's not done yet. You would need to create an environment to deploy and run our project. 133 | 134 | ![Create environment](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/create-environment.jpg) 135 | 136 | 3. Now click on "create a new environment". You will be presented with the below screen. 137 | 138 | ![select env tire](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/select-env-tire.jpg) 139 | 140 | 4. Choose Web server environment and click on select. 141 | 5. You will now be presented with the following screen. 142 | 143 | ![Environment info](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/environment-info.jpg) 144 | 145 | Here you can change the name of the environment or you can keep the default. You can create as many environments as you want but all their names must be unique. 146 | 147 | You can choose to enter a Domain or leave it blank, if you leave it blank the name will be auto-generated. 148 | 149 | Now under the platform section choose Python other dropdowns will be auto-filled. 150 | 151 | You can now choose to upload your code by clicking on upload your code and choose the project from your local computer(max upload size: 512MB). 152 | 153 | Or you can even choose to leave that and later upload the code through eb-cli. 154 | 155 | 6. Now click on "configure more options". 156 | 157 | **Note: If you are deploying a Django application that uses channels for WebSockets you need to pay close attention to the below.** 158 | 159 | Now you will be presented with the below screen: 160 | 161 | ![Configuring environment](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/configuring-eb-env.jpg) 162 | 163 | 7. under presets click on custom. Then under the Load balancer section click on edit. 164 | 165 | If you are not using channels or WebSockets. You can go ahead with the classic load balancer. 166 |
167 | 168 | **Webscockets**: 169 | If you are using WebSockets click on Application load balancer. Only Application load balancer allows WebSocket connections. 170 | 171 | Now go to the bottom of the page and click on save. 172 |
173 | 174 | > Note: you will not be able to change the load balancer later. So if you are using WebSockets use Application load balancer. 175 | 176 | >Note: If you are ever required to change the load balancer later you will have to create a new environment and copy your existing environment settings. 177 | 178 | 179 | > Note: Don't click on edit database under Database section, as we will be creating a database from the RDS console. If you are ever required to modify database settings you will find it difficult to modify. So don't create the database when creating the environment. 180 | 181 | > Note: for productions, I suggest you add your local (sqlite3/any other) database to `.gitignore` and not upload it EB environment. 182 | 183 | 8. Now if you are happy with the settings you can click in create environment at the bottom of the page. 184 | 185 | 186 | ## 2. Using EB CLI to deploy the application. 187 | 188 | 189 | To use this you must have EB CLI installed as stated above and added the path to environment variables. Please follow the steps as provided in the guide to install EB CLI. 190 | 191 | ### generating access key and configuring AWS CLI: 192 | 193 | first go to IAM console by searching for it in the search bar. Now click on users, click on your account name > under summary click on "security credentials" tab > click on create access key. 194 | 195 | Now follow the steps in this documentation to configure your account with cli: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html 196 | 197 | ---- 198 | 199 | Now open the command prompt or terminal and change the current directory to the project directory. 200 | 201 | In Windows 202 | ```sh 203 | cd `path_to_project _directory` 204 | ``` 205 | now type `eb` and EB CLI should pop up. If it doesn't open in a new window, no problem. Make sure you have deactivated the virtual environment you are currently working in when using the EB CLI or EB commands. 206 | 207 | ### 1. Creating an application: 208 | 209 | ```sh 210 | eb init 211 | ``` 212 | 213 | or 214 | 215 | ``` 216 | eb init -p python-3.8 PhotoProject 217 | ``` 218 | In the above replace the python version and PhotoApp with a version and name of your choice. This just creates an application named PhotoApp in AWS Elastic Beanstalk. 219 | 220 | The 2nd option will directly create an application without prompting you for more options (this one is quicker). 221 | 222 | >I'll go with 1st option `eb init` 223 | 224 | 225 | If you went with the first option you will be prompted to select a region you can leave this at default or choose a region. 226 | 227 | ![region-selection](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/eb-init.jpg) 228 | 229 | > Note: In the future, You will have to keep all the services such as RDS, S3 under the same region for it to work properly. 230 | 231 | I'll leave this at default which is option-3 (US west (Oregon)) 232 | 233 | Next, you will be asked to choose an application or create a new 234 | 235 | ```sh 236 | Select an application to use 237 | 1) sampleapp 238 | 2) [ Create new Application ] 239 | (default is 2): 2 240 | ``` 241 | select `2` 242 | 243 | If you haven't previously set up provided a secret key and access key you will be asked to give it. If you haven't set this up please follow their guide to set up 244 | 245 | Now you will be asked to give a name to the application 246 | ``` 247 | Enter Application Name 248 | (default is "PhotoProject"): 249 | ``` 250 | You can choose to give a name or choose to leave it as default. 251 | 252 | Now the application is created. 253 | 254 | ``` 255 | Enter Application Name 256 | (default is "PhotoProject"): 257 | Application PhotoProject has been created. 258 | 259 | It appears you are using Python. Is this correct? 260 | (Y/n): 261 | Select a platform branch. 262 | 1) Python 3.8 running on 64bit Amazon Linux 2 263 | 2) Python 3.7 running on 64bit Amazon Linux 2 264 | 3) Python 3.6 running on 64bit Amazon Linux (Deprecated) 265 | (default is 1): 266 | ``` 267 | choose the version, I'll be going with the first option 268 | 269 | ``` 270 | Do you wish to continue with CodeCommit? (Y/n): n 271 | Do you want to set up SSH for your instances? 272 | (Y/n): y 273 | 274 | Select a keypair. 275 | 1) existing-pair 276 | 2) [ Create new KeyPair ] 277 | (default is 1): 278 | ``` 279 | 280 | In the above, we choose `n` for codecommit. You can choose `y`. 281 | >AWS CodeCommit is a fully managed service for securely hosting private Git repositories. CodeCommit now supports pull requests, which allows repository users to review, comment upon, and interactively iterate on code changes. Used as a collaboration tool between team members, pull requests help you to review potential changes to a CodeCommit repository before merging those changes into the repository 282 | > above is taken from docs: https://aws.amazon.com/blogs/devops/using-aws-codecommit-pull-requests-to-request-code-reviews-and-discuss-code/ 283 | 284 | 285 | We choose `y` for SSH 286 | > The Secure Shell Protocol (SSH) is a cryptographic network protocol for operating network services securely over an unsecured network. 287 | 288 | We will be using SSH to connect to our EC2 instance so we must give `y`. If you haven't set this you can later set this using `eb ssh --setup` 289 | 290 | The output for the above command will look something like this. 291 | ``` 292 | Type a keypair name. 293 | (Default is aws-eb): 294 | Generating public/private RSA key pair. 295 | Enter passphrase (empty for no passphrase): 296 | Enter the same passphrase again: 297 | Your identification has been saved in C:\path\.ssh\. 298 | Your public key has been saved in C:\path\.ssh\.pub. 299 | The key fingerprint is: 300 | SHA***:G**4A****9******3ps*** 301 | The key's randomart image is: 302 | +---[RSA ***]----+ 303 | 304 | +----[SHA***]-----+ 305 | Enter passphrase: 306 | WARNING: Uploaded SSH public key for "***" into EC2 for region *****. 307 | ``` 308 | You can choose `\` of your choice. 309 | Enter a passphrase you can remember. you can also choose to save the passphrase, everything including fingerprint in a file(shouldn't be uploaded to a public repo). 310 | 311 | ### 2. Creating an environment: 312 | 313 | Environment will create an instance of EC2 where your project will run. 314 | 315 | To set this up type 316 | ``` 317 | eb create app-env 318 | ``` 319 | You can replace "app-env" with a name of your choice. This simply creates an environment with the specified name and uploads the project zip file to the S3 buckets service (it's a simple storage service provided by AWS). You can also host your media files here and media files will directly be served from here. 320 | 321 | This might take from a few seconds up to a few minutes to complete. After completion, you should see something as shown below at the end. 322 | 323 | ``` 324 | 2022-01-31 16:09:00 INFO Instance deployment successfully generated a 'Procfile'. 325 | 2022-01-31 16:09:03 INFO Instance deployment completed successfully. 326 | 2022-01-31 16:10:06 INFO Successfully launched environment: app-env 327 | 328 | ``` 329 | 330 | 331 | Now head back to AWS Console and type elastic beanstalk -> left panel Applications -> choose the application you just created in my case it was called PhotoProject. 332 | Now click on the environment in my case it was called app-env. 333 | 334 | Now if it was successfully deployed you should see the health ok and a green checkmark. 335 | 336 | ![eb-successful](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/eb-successful.jpg) 337 | 338 | Now What I have highlighted in red is your domain. Click on it and you will see. 339 | 340 | ![disallowed host](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/disallowed_host.jpg). 341 | 342 | Now all you have to do is head back to your settings.py in your project and under 343 | ALLOWED HOST type in the domain. 344 | 345 | In my case, it was 'app-env.eba-zprspr6p.us-west-2.elasticbeanstalk.com' 346 | ``` 347 | DEBUG = True 348 | 349 | ALLOWED_HOSTS = ['app-env.eba-zprspr6p.us-west-2.elasticbeanstalk.com'] 350 | ``` 351 | > Note: In production don't forget to set `DEBUG=FALSE` 352 | 353 | Now save it again head back to your EB CLI and type 354 | ```sh 355 | eb deploy 356 | ``` 357 | If you make any changes to your application you can use `eb deploy` to deploy the application. 358 | 359 | You can also upload only the staged changes by using 360 | ``` 361 | eb deploy --staged 362 | ``` 363 | 364 | you can stage changes by using: 365 | ```git 366 | git add . 367 | ``` 368 | 369 | Now you should be able to see your application output 370 | 371 | 372 | ### Unsuccessful deployment: 373 | 374 |
375 | 376 | ![eb-unsuccessful](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/degraded-health.jpg) 377 | 378 | If you see this that means that there was an error while uploading or the health check is failing with a 4xx error. Follow the debugging errors below to debug the errors 379 | ### Debugging errors: 380 | 381 | If you see health as degraded you should head over to logs in the left navigation panel. 382 | 383 | ![Logs](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/logs.jpg) 384 | 385 | Here click on request logs and click on last 100 lines. 386 | 387 | Now download the logs: 388 | 389 | When you view that page: 390 | You should notice the title: 391 | Eg: 392 | ![log-title](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/log-title.jpg) 393 | 394 | The above is logs for eb-engine. 395 | 396 | Now scroll down till you find 397 | ``` 398 | ---------------------------------------- 399 | /var/log/web.stdout.log 400 | ---------------------------------------- 401 | ``` 402 | 403 | Under this, you should be able to see what the error is. 404 | 405 | 406 | If you want to connect to Linux AWS servers use. You can also debug from there: 407 | 408 | ``` 409 | eb ssh 410 | ``` 411 | You should enter your passphrase and be able to login. You can now type Linux commands 412 | 413 | To activate your environment in Linux-2 414 | ``` 415 | source /var/app/venv/*/bin/activate 416 | ``` 417 | to change our project directory use: 418 | ``` 419 | cd /var/app/current/ 420 | ``` 421 | 422 | Note the above paths might change if AWS team decides to change them in the future. You can always find the new paths in the migrations documentation they release. 423 | 424 | The above is helpful when you want to find out what's going wrong. 425 | You can also use 426 | ``` 427 | ls /var/app/current/ 428 | ``` 429 | to list all the files and folders under the current directory. 430 | 431 | 432 | Errors: 433 | ### ModuleNotFoundError: No module named 'application': 434 | 435 | If you see this error that means that django.config file was not uploaded successfully 436 | 437 | You can confirm this by heading over to the configuration tab from the left panel. 438 | 439 | ![configuration](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/configuration.jpg). 440 | 441 | Now under software click on edit: 442 | 443 | Now under the Container options Section, you can see the WSGI path if WSGI Path contains only application and not "PhotoProject.wsgi:application" then it means that the django.config file was not uploaded correctly. 444 | 445 | 1. You can try to stage the changes using: 446 | ``` 447 | git add . 448 | ``` 449 | and using 450 | ``` 451 | eb deploy --staged 452 | ``` 453 | 2. Make sure you haven't add .ebextensions to git ignore. 454 | 455 | If it still doesn't work only as a last resort 456 | you can head elastic beanstalk configurations -> software -> edit -> WSGI path fill the path like so: PhotoProject.wsgi:application 457 | 458 | ### ModuleNotFoundError: No module named 'XYZ': 459 | If you find module not found then make sure you have included them in `requirements.txt` 460 | 461 | ### Failing health checks: 462 | If you don't see any errors in the log file then head over to the link that was underlined in red in the above image(the image where the health was green). 463 | 464 | Click on the link if you can see your app running or the Django's debug error page then it means that your upload was successful 465 | 466 | You should head over to the environment page and check out the recent events. 467 | 468 | There are three types of Type: 469 | 1. INFO - it's just info and there are no errors 470 | 2. WARN - A warning. 471 | 3. ERROR - an error occurred 472 | 473 | These types will be visible under the Type column under Recent events section as shown below. 474 | 475 | ![recent-events](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/recent-events.jpg) 476 | 477 | If you don't see any error click on show all. 478 | 479 | If under detail you see x% (eg: 50%) of health checks are failing with 4xx. This means that only the health checks are failing. 480 | 481 | By default, the health checks are performed to the `/` endpoint. So when AWS makes get requests to `http://myhost/` your application should return 200 Ok or anything in the 200 range. 482 | 483 | If your application is not returning 200 then you can create a dedicated health check endpoint. 484 | 485 | Eg: 486 | urls.py 487 | ``` 488 | urlpatterns = [ 489 | path('health/', views.health_check), 490 | ... 491 | ] 492 | ``` 493 | 494 | views.py: 495 | ``` 496 | from django.http import HttpResponse 497 | def health_check(request): 498 | return HttpResponse(status=200) 499 | ``` 500 | 501 | Now head over to the environment configuration in ealsticbeanstalk -> left panel -> Configuration -> scroll down -> Load balancer. 502 | 503 | Now under load balancer click on edit. 504 | 505 | Scroll down to health check: 506 | 507 | ![health-check](https://github.com/PaulleDemon/AWS-deployment/blob/master/images/health-check.jpg) 508 | Now under health check path provide the path to your health check eg:`/health` as shown above. 509 | 510 | then scroll down and click on apply and this could take several minutes. You can head over to events and check if the health check is failing 511 | --------------------------------------------------------------------------------