└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Algotrading as Code: *a practical tutorial how to automate freqtrade - a crypto-currency algorithmic trading software* 2 | 3 | ## Disclaimer 4 | 5 | Use everything at your own risk. I am not a financial advisor, and not responsible for anything. 6 | 7 | ## Premise 8 | 9 | This documentation is intended to provide necessary information, how to automatically spin up instances via Terraform and configure all necessary bits & pieces via Ansible, so at the end you have a ready to run machine with Freqtrade installed. If this is all very much new to you, please follow the excellent documentations for the forementioned software: 10 | 11 | - [Freqtrade](https://www.freqtrade.io/en/stable/) 12 | - [Terraform](https://www.terraform.io/docs/index.html) 13 | - [Ansible](https://docs.ansible.com/) 14 | 15 | ## Requirements 16 | 17 | The example workflow is described on an Debian/Ubuntu based system, your mileage may vary, if you intend to use CentOS, ArchLinux, etc. please check the documentation how to install Ansible & Terraform on your preferred OS-flavour. Usage from Windows or MacOS systems is absolutely possible but not covered inhere. The probably easiest way to get everything up & running when your primary OS is Windows is WSL (Windows Subsystem for Linux) which easily allows you to run a virtual Linux environment of your choice. Please check https://wiki.ubuntu.com/WSL for more information, how to get an Ubuntu WSL instance running. 18 | 19 | ### Terraform 20 | 21 | Terraform must be installed on the system from which the project will be executed. Start by getting Terraform from their [download page](https://www.terraform.io/downloads.html) and select the appropriate package for your system, for example: 22 | 23 | wget https://releases.hashicorp.com/terraform/0.15.3/terraform_0.15.3_linux_amd64.zip 24 | 25 | 26 | Extract the binaries into a suitable location such as `/usr/local/bin` and make sure it is included in your PATH environmental variable. 27 | 28 | sudo unzip terraform_0.15.3_linux_amd64.zip -d /usr/local/bin 29 | 30 | Test if terraform is accessible by checking the version number in the terminal via the command: 31 | 32 | terraform -v 33 | Terraform v0.15.3 34 | 35 | ### Ansible 36 | 37 | The installation of Ansible on various *nix based operating systems is described in [the official Ansible documentation](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-on-specific-operating-systems), please follow the guide for your operating system. 38 | 39 | Installing Ansible on Window OS can be a bit cumbersome, either you use Cygwin (described [here](https://phoenixnap.com/kb/install-ansible-on-windows)), or probably a better option is to use WSL as mentioned above. 40 | 41 | 42 | ## Workflow 43 | 44 | The automated deployment is broken into two parts: 45 | 46 | 1. Build the necessary infrastructure via Terraform 47 | 2. Install Freqtrade with dependencies via Ansible 48 | 49 | To outline it a bit more detailed: A terraform project will build the necessary linux instances, create SSH keys & logins and create a `hosts.cfg` file, which will be the base for Ansible to kick off the software installation. 50 | 51 | ## Step by Step Guide 52 | 53 | ### 1. Terraform Project 54 | 55 | At the moment one Terraform Project has been published to create instances and generate all necessary files for Ansible. This project runs on Vultr infrastructure. The reason for Vultr is that they offer instances in Tokyo/Japan which have very low latency to the largest crypto exchange (Binance - also hosted in Japan). Beside currently the pricing of Vultr is more competetive than Azure or AWS. More Terraform Projects for Azure, AWS, GCP are in the making. 56 | 57 | On your working host, where Terraform and Ansible have been installed checkout the Vultr Terraform Project via git 58 | 59 | git clone https://github.com/nightshift2k/terraform-ansible-ubuntu-vultr.git 60 | 61 | Follow the instructions in the `README.md` to setup API access to Vultr, then rename `terraform.tfvars.example` to `terraform.tfvars` and adjust all variables. 62 | 63 | Afterwards perform 64 | 65 | terraform init 66 | terraform plan 67 | terraform apply 68 | 69 | If everything went smooth proceed to the Ansible part. 70 | 71 | 72 | ### 2. Ansible 73 | 74 | Depending which installation option you like, you can choose between a native or a docker containerized version of Freqtrade. Docker can be great if you want to run multiple containers with different versions but be a bit cumbersome for development. A parallel usage is also possible. 75 | 76 | #### 2.1 Ansible Requirements 77 | 78 | To create a playbook to install Freqtrade you will need the appropriate roles directly from GitHub or Ansible Galaxy. Furthermore also a community collection for Docker operations will be needed. 79 | 80 | Install all prerequisites with `ansible-galaxy`: 81 | 82 | ansible-galaxy collection install community.docker 83 | ansible-galaxy install geerlingguy.docker 84 | ansible-galaxy install nightshift2k.freqtrade 85 | ansible-galaxy install nightshift2k.freqtrade_docker 86 | 87 | #### 2.2 Create an Ansible Playbook for native Freqtrade installation 88 | 89 | Please check the [`README.md`](https://github.com/nightshift2k/ansible-role-freqtrade/blob/master/README.md) of the Ansible Role `nightshift2.freqtrade` how to create and configure a playbook using this role. 90 | 91 | In short, create a file called `freqtrade-playbook.yml` with the following content - please be aware, this is an example, your requirements may vary: 92 | 93 | - hosts: instances 94 | roles: 95 | - role: nightshift2k.freqtrade 96 | - vars: 97 | freqtrade_installation_directory: "/opt/freqtrade" 98 | freqtrade_branch: "develop" 99 | freqtrade_install_dev_deps: false 100 | freqtrade_install_plot_deps: true 101 | freqtrade_install_hyperopt_deps: true 102 | freqtrade_install_additional_pips: 103 | - finta 104 | become: yes 105 | 106 | #### 2.3 Create an Ansible Playbook for a dockerized Freqtrade installation. 107 | 108 | This is very much similar to the above, but instead of installing freqtrade natively docker containers will be pulled from the registry or build - depending on your configuration. 109 | 110 | Please check the [`README.md`](https://github.com/nightshift2k/ansible-role-freqtrade-docker/blob/master/README.md) of the Ansible Role `nightshift2.freqtrade_docker` how to create and configure a playbook using this role. 111 | 112 | In short, create a file called `freqtrade-playbook.yml` with the following content - please be aware, this is an example, your requirements may vary: 113 | 114 | - hosts: instances 115 | roles: 116 | - role: nightshift2k.freqtrade_docker 117 | vars: 118 | freqtrade_branch: "develop" 119 | docker_build_directory: "/tmp/build/docker" 120 | build_freqtrade: true 121 | build_freqtrade_custom: true 122 | freqtrade_custom_prefix : "nightshift2k" 123 | freqtrade_custom_commands: 124 | - "RUN pip install --upgrade pip" 125 | freqtrade_custom_pips: 126 | - finta 127 | - ta 128 | - technical 129 | 130 | After saving the playbook yml file you are ready to run ansible against your instance(s) by calling `ansible-playbook` and providing the `hosts.cfg` from the Terraform project. 131 | 132 | 133 | #### 2.4 Run the Playbook on your instance(s) 134 | 135 | This is basically tying it all together: `ansible-playbook` will now take the `hosts.cfg` file from Terraform and utilize the information about the host (IP's) and the credential (username + SSH Key) and run the playbook to install freqtrade in the chosen flavor (native or dockerized). 136 | 137 | Please take note of the full path of your `hosts.cfg` as it will be needed for the command. So if your Terraform project has been executed in `/home/dev/terraform-project` the full filename you will need will very likely be `/home/dev/terraform-project/output/hosts.cfg` 138 | 139 | **WARNING:** If you have to move the files inside the output folder, you will have to adjust the variable `ansible_ssh_private_key_file` as it contains an absolute path to the SSH Key! 140 | 141 | Run the playbook with the following command (with filename from the above explanation): 142 | 143 | 144 | ansible-playbook -i /home/dev/terraform-project/output/hosts.cfg freqtrade-playbook.yml 145 | 146 | Depending on your configuration and the instance size this might take from 5m to somwhere around 30m - especially if you run it 147 | 148 | 149 | 150 | ## License 151 | 152 | MIT / BSD --------------------------------------------------------------------------------