├── .gitattributes
├── .gitignore
├── README.md
├── app.py
├── bot.py
├── data
    ├── Binance_ADAUSDT_1h.csv
    ├── Binance_BCHUSDT_1h.csv
    ├── Binance_BNBUSDT_1h.csv
    ├── Binance_BTCUSDT_1h.csv
    ├── Binance_BTTUSDT_1h.csv
    ├── Binance_DASHUSDT_1h.csv
    ├── Binance_DOGEUSDT_1h.csv
    ├── Binance_ETHUSDT_1h.csv
    ├── Binance_LTCUSDT_1h.csv
    ├── Binance_NANOUSDT_1h.csv
    ├── Binance_XLMUSDT_1h.csv
    └── Binance_XMRUSDT_1h.csv
├── models
    └── model
    │   └── LSTM-2021-08-23.tflite
├── requirements_rpi.txt
├── requirements_win.txt
├── screenshot.png
├── setup.py
├── src
    ├── config.py
    ├── data_backup.py
    ├── data_update.py
    ├── orders.py
    ├── predict.py
    ├── process_logfile.py
    └── utils.py
├── static
    ├── Chart.bundle.min.js
    ├── bootstrap.min.css
    ├── favicon.ico
    ├── main.css
    └── script.js
├── templates
    └── index.html
├── tests.py
├── tools
    ├── data_add_new_pair.py
    └── simulate.py
├── training-google-colab
    ├── convert_model.py
    ├── model3.ipynb
    └── model3_kfold.ipynb
├── whl
    ├── tflite_runtime-2.3.1-cp37-cp37m-linux_armv6l.whl
    └── tflite_runtime-2.5.0-cp37-cp37m-linux_armv7l.whl
└── wsgi.py


/.gitattributes:
--------------------------------------------------------------------------------
1 | training-google-colab/* linguist-documentation
2 | 


--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
 1 | *.rar
 2 | *.zip
 3 | 
 4 | old/
 5 | __pycache__/
 6 | src/__pycache__/
 7 | tools/__pycache__/
 8 | training-google-colab/__pycache__/
 9 | venv/
10 | .idea/
11 | 
12 | secrets.ini
13 | config.ini
14 | logfile.log
15 | logfile.csv
16 | 
17 | todos.txt
18 | t.py
19 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | 
  2 | # crypto-bot
  3 | 
  4 | ![Screenshot](screenshot.png)
  5 | 
  6 | ## Contents
  7 | 1. <a href="#about">About</a>
  8 | 2. <a href="#crypto-bot-botpy">crypto-bot (bot.py)</a>
  9 | 	- <a href="#installation">Installation</a>
 10 | 	- <a href="#setup">Setup</a>
 11 | 	- <a href="#running">Running</a>
 12 | 	- <a href="#optional-github-backups">[OPTIONAL] GitHub backups</a>
 13 | 3. <a href="#info-site-apppy">info-site (app.py)</a>
 14 | 4. <a href="#usage">Usage</a>
 15 | 5. <a href="#training">Training</a>
 16 | 6. <a href="#tools-and-tests">Tools and tests</a>
 17 | 7. <a href="#tools-and-tests">Issues</a>
 18 | 
 19 | ## About
 20 | This archive contains the source code of "crypto-bot", which is a cryptocurrency trading bot (`bot.py`) together with info-site (`app.py`) created with Flask.
 21 | 
 22 | It is designed to work in 1h intervals (data gathering, making predictions, placing orders) on Binance, the biggest cryptocurrency exchange. It is configured to predict BTC, basing on hourly data of 12 selected cryptocurrencies. Its features:
 23 | 
 24 | - data gathering and trading using binance.com API
 25 | - neural network (LSTM) predictor trained on Google Colab
 26 | - automatic data backups using Git
 27 | - info-site with logs and stats using Flask
 28 | - implemented on Raspberry Pi Zero W (armv6l)
 29 | 
 30 | 
 31 | 
 32 | ## crypto-bot (`bot.py`)
 33 | 
 34 | ### Installation
 35 | 1. Install **Raspberry PI OS Lite** (tested on <a href="http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2020-02-14/">2020-02-14 release</a>) and setup internet access. Make sure to have **Python 3.7** and **git** installed. The default working directory is assumed to be ```/home/pi/```.
 36 | 2. Clone this repository
 37 | 	```sh
 38 | 	git clone https://github.com/kamsec/crypto-bot.git
 39 | 	```
 40 | 3. Install requirements
 41 | 	```sh
 42 | 	cd crypto-bot
 43 | 	```
 44 | 	Linux:
 45 | 	```
 46 | 	pip3 install -r requirements_rpi.txt
 47 | 	```
 48 | 	Windows:
 49 | 	```
 50 | 	pip install -r requirements_win.txt
 51 | 	```
 52 | ### Setup
 53 | 4. Run `setup.py` with command:
 54 | 	```sh
 55 | 	python3 setup.py
 56 | 	```
 57 | 	It will create the files `config.ini`, `secrets.ini`, `logfile.log`.
 58 | 5. In order to check the balances and operate Binance account, bot needs to have access to API keys. In `secrets.ini` you will find the following content:
 59 | 	```
 60 | 	[DEFAULT]
 61 | 	testnet_api_key = ***
 62 | 	testnet_api_secret = ***
 63 | 	mainnet_api_key = ***
 64 | 	mainnet_api_secret = ***
 65 | 	```
 66 | 	In place of `***` placeholders provide your binance.com API keys  (testnet or/and mainnet) without any quotation marks. Visit https://www.binance.com/en/support/faq/360002502072 for more information.
 67 | 	[OPTIONAL] To enable orders, in `config.ini` you can set:
 68 | 	```
 69 | 	orders_allowed = True
 70 | 	```
 71 | 	In `config.ini` you can also change working binance.com enviroment between `testnet` and `mainnet` according to provided keys.
 72 | 	```
 73 | 	enviroment = testnet  # can be changed to mainnet
 74 | 	```
 75 | ### Running
 76 | 6. Run the bot with:
 77 | 	```sh
 78 | 	python3 bot.py
 79 | 	```
 80 | 	If everything works correctly it will produce the output in logfile.log of the following format:
 81 | 	```sh
 82 | 	2021-08-23 09:32:45 [INFO] Bot started
 83 | 	2021-08-23 09:32:47 [INFO] [BALANCE] 0.29 USD, 0.0010283 BTC
 84 | 	2021-08-23 09:32:47 [INFO] [PREDICTION] DOWN: 42%, UP: 58%
 85 | 	```
 86 | 	If an hour haven't passed from last record in `data/` folder, the message
 87 | 	```sh
 88 | 	"[WARNING] [UPDATE] No new records available yet. Try again in: 0:47:50.464589" 
 89 | 	```
 90 | 	will appear and `[PREDICTION]` will not be made this hour, but this is correct. Bot will also create `logfile.csv` which is used only by Flask info-site (`app.py`).
 91 | 7. In order to make the bot working at the beginning of every hour (1st minute), add the cron job. Enter the command:
 92 | 	```
 93 | 	crontab -e
 94 | 	```
 95 | 	and at the end of the opened file add the following line:
 96 | 	```sh
 97 | 	1 * * * * cd /home/pi/crypto-bot && python3 /home/pi/crypto-bot/bot.py
 98 | 	```
 99 | 
100 | ### [OPTIONAL] GitHub backups
101 | 8. In order to create GitHub backups bot requires a remote repository and SSH key for authentication. Having remote repository defined, create new ssh key:
102 | 	```sh
103 | 	ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
104 | 	```
105 | 	When the prompts appear, press enter three times so the file will be saved in default location `/home/pi/.ssh/` under default name `id_rsa` with no passphrase.
106 | 	Open the ssh agent with command
107 | 	```shell
108 | 	eval "$(ssh-agent -s)"
109 | 	```
110 | 	and add created key
111 | 	```shell
112 | 	ssh-add ~/.ssh/id_rsa
113 | 	```
114 | 	Now add the SSH key to your account on GitHub, and create repository for this bot. For more information, see https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account.
115 | 	In `config.ini` set:
116 | 	```
117 | 	backups_allowed = True
118 | 	```
119 | Instructions from this section will make `bot.py` script running once in the first minute of every hour.
120 | 
121 | ## info-site (`app.py`)
122 | After running crypto-bot for the first time, you can view the results on Flask info-site, by running:
123 | ```sh
124 | python3 app.py
125 | ```
126 | and visiting the `http://IP:5000/` in your browser (port is always 5000, but you need to check the IP assigned to your Raspberry Pi), for example http://192.168.0.107:5000/.
127 | If you want to host the Flask info site on Raspbery Pi as service, you can use Gunicorn with nginx. To do this, take the folliwing steps:
128 | 1. Install nginx.
129 | 	```sh
130 | 	sudo apt-get install nginx
131 | 	```
132 | 2.  Create `crypto-bot` file in `/etc/nginx/sites-available/` with the following comand:
133 | 	```sh
134 | 	sudo nano /etc/nginx/sites-available/crypto-bot
135 | 	```
136 | 	and write the following content (assumed localhost and port 5000):
137 | 	```sh
138 | 	server {
139 | 		listen 80;
140 | 		server_name 123.123.123.123;
141 | 
142 | 		location / {
143 | 			proxy_pass http://127.0.0.1:5000;
144 | 			proxy_set_header Host $host;
145 | 			proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
146 | 			proxy_set_header X-Scheme $scheme;
147 | 		}
148 | 	}
149 | 	```
150 | 3.  Create `crypto-bot.service` file in `/etc/systemd/system/` with the following comand:
151 | 	```sh
152 | 	sudo nano /etc/systemd/system/crypto-bot.service
153 | 	```
154 | 	and write the following content (assumed localhost and port 5000):
155 | 	```sh
156 | 	[Unit]
157 | 		Description=Gunicorn instance to serve crypto-bot
158 | 		After=network.target
159 | 
160 | 	[Service]
161 | 		User=pi
162 | 		Group=pi
163 | 		WorkingDirectory=/home/pi/crypto-bot
164 | 		# Environment="PATH=/home/pi/crypto-bot/myprojectenv/bin" # if using virtual env
165 | 		ExecStart=/home/pi/.local/bin/gunicorn --bind 0.0.0.0:5000 --workers 3 wsgi:app
166 | 		Restart=always
167 | 
168 | 	[Install]
169 | 		WantedBy=multi-user.target
170 | 	```
171 | 	Add and start the service with commands:
172 | 	```
173 | 	sudo systemctl daemon-reload
174 | 	sudo systemctl enable crypto-bot.service
175 | 	sudo systemctl start crypto-bot.service
176 | 	```
177 | You can now visit information site on your RaspberryPi IP on port 5000, e.g.  http://192.168.0.107:5000/, and this site will be available whenever Raspberry Pi is running and has internet connection.
178 | To access the site outside the local network, you should perform port forwarding in your router settings.
179 | 
180 | ## Usage
181 | - To control the configuration of the bot, you can edit constants in`config.ini`, for example:
182 | 	```sh
183 | 	status_active = True  # set to False to make the bot inactive
184 | 	orders_allowed = False  # set to True to allow placing orders
185 | 	backups_allowed = False  # set to True to allow GitHub backups
186 | 	enviroment = testnet  # set to mainnet to change binance.com enviroment
187 | 	```
188 | 	
189 | - To stop the bot completely, type:
190 | 	```sh
191 | 	crontab -e
192 | 	```
193 | 	and remove or comment the line that defines cron job:
194 | 	```sh
195 | 	1 * * * * cd /home/pi/crypto-bot && python3 /home/pi/crypto-bot/bot.py
196 | 	```
197 | - To restart the bot (erase log history and reset balances), delete `logfile.log`, `config.ini`, `logfile.csv` (if exists) and run :
198 | 	```sh
199 | 	python3 setup.py
200 | 	python3 bot.py
201 | 	```
202 | - To control Flask info-site, you can use commands:
203 | 	```sh
204 | 	sudo systemctl stop crypto-bot.service  # to stop the app
205 | 	sudo systemctl restart crypto-bot.service  # to restart the app
206 | 	```
207 | - To change the prediction model place new model in `models/model/`, and in `config.cfg` set:
208 | 	```sh
209 | 	model_path = models/model/***
210 | 	```
211 |     and in `setup.py` set the variable
212 | 	```sh
213 | 	MODEL_NAME = '***'
214 | 	```
215 |     with `***` as new model name.
216 | 
217 | ## Training
218 | Training neural network models is performed manually on Google Colab, and files used for it are located in `training-google-colab` directory.
219 | 
220 | ## Tools and tests
221 | Tools for adding new data pairs and running bot simulation (placing orders during using past data) are located in `tools/` directory.
222 | In main directory, file `tests.py` contains some manual tests and allows to run&test individial functions from the project.
223 | 
224 | ## Issues
225 | - Recent version of Firefox sometimes display parts of info-page incorrectly - just use Chrome in that case.
226 | - If you encounter Binance error "Timestamp for this request is 1000ms ahead of the server's" you should sync your system time with Binance. More information: https://github.com/ccxt/ccxt/issues/773
227 | 
228 | 
229 | 
230 | 


--------------------------------------------------------------------------------
/app.py:
--------------------------------------------------------------------------------
  1 | import importlib
  2 | import logging
  3 | import csv
  4 | from json import dumps
  5 | from flask import Flask, render_template
  6 | from datetime import datetime, timedelta
  7 | 
  8 | import src.config  # also reloaded with every page refresh
  9 | from src.data_backup import get_last_commit_time
 10 | from src.utils import shift_hour_exact, UTC_OFFSET
 11 | 
 12 | # disabling Flask werkzeug logger because in config we use basicConfig for loggers, which is propagated to werkzeug,
 13 | # and we don't want app.run() logs in logfile.log
 14 | flask_logger = logging.getLogger("werkzeug")
 15 | flask_logger.disabled = True
 16 | 
 17 | app = Flask(__name__)
 18 | 
 19 | @app.route("/")
 20 | def generate_page():
 21 |     # all hours are given in UTC+0. Times are adjusted in process_logfile.py -> logfile.csv
 22 |     # logfile.log timestamps also are in UTC+0
 23 | 
 24 |     # config needs to be reloaded on every page refresh, so importing constants here
 25 |     importlib.reload(src.config)
 26 |     from src.config import CONFIG_NAME, cfg, STATUS_ACTIVE, ORDERS_ALLOWED, ENVIROMENT, LOG_FILE_ABS, MODEL_PATH, \
 27 |         STARTING_TIMESTAMP, STARTING_BALANCE_USD, STARTING_BALANCE_BTC, CURRENT_BALANCE_USD, CURRENT_BALANCE_BTC, \
 28 |         BACKUPS_ALLOWED
 29 | 
 30 |     # for Log textarea
 31 |     with open(LOG_FILE_ABS, 'rt') as f:
 32 |         logs_data = f.read()
 33 |         if logs_data[-1] == '\n':
 34 |             logs_data = logs_data[:-1]   # stripping ending blank line, if it's there
 35 | 
 36 |     # time from line '2021-08-12 16:40:49 [INFO] Bot started.'
 37 |     exact_starting_timestamp = logs_data[:19]  # '2021-08-12 16:40:49'
 38 | 
 39 |     def interpret_string_from_csv(x):
 40 |         if x == 'True':
 41 |             return True
 42 |         elif x == 'False':
 43 |             return False
 44 |         elif x == '':
 45 |             return None
 46 |         else:
 47 |             return x
 48 | 
 49 |     columns = {'date': [], 'BTCUSDT_close': [], 'future': [], 'target': [],
 50 |                'predictions_shifted': [], 'accurate_prediction': [], 'orders_shifted': [],
 51 |                'balance_USD': [], 'balance_BTC': []}
 52 |     with open('logfile.csv', 'r') as file:
 53 |         reader = csv.reader(file, delimiter=',')
 54 |         next(reader)  # skipping header
 55 |         for row in reader:
 56 |             for item, key in zip(row, columns):
 57 |                 columns[key].append(interpret_string_from_csv(item))
 58 | 
 59 |     # accuracy from start
 60 |     accuracy_from_start = 50  # default value if no predictions were made
 61 |     num_of_true_preds = columns['accurate_prediction'].count(True)
 62 |     num_of_false_preds = columns['accurate_prediction'].count(False)  # None predictions are not counted
 63 |     if num_of_true_preds + num_of_false_preds > 0:  # prevents zero division
 64 |         accuracy_from_start = f'{(num_of_true_preds / (num_of_true_preds + num_of_false_preds) * 100):.1f}'
 65 | 
 66 |     # slicing last 72 hours for chart
 67 |     columns_72h = {}
 68 |     for key in columns:
 69 |         columns_72h[key] = columns[key][-72:]
 70 | 
 71 |     # accuracy from last 72h
 72 |     accuracy_72h = 50  # default value if no predictions were made
 73 |     num_of_true_preds_72h = columns_72h['accurate_prediction'].count(True)
 74 |     num_of_false_preds_72h = columns_72h['accurate_prediction'].count(False)  # None predictions are not counted
 75 |     if num_of_true_preds_72h + num_of_false_preds_72h > 0:  # prevents zero division
 76 |         accuracy_72h = f'{(num_of_true_preds_72h / (num_of_true_preds_72h + num_of_false_preds_72h) * 100):.1f}'
 77 | 
 78 |     # values directly used to plot graph
 79 |     data_values_72h = columns_72h['BTCUSDT_close']
 80 |     data_index_72h = [x[:-6] for x in columns_72h['date']]  # [:-6] makes time in format "%Y-%m-%d %H"
 81 |     predictions_72h = dumps(columns_72h['predictions_shifted'])  # also converts python None to js null
 82 |     orders_72h = dumps(columns_72h['orders_shifted'])  # also converts python None to js null
 83 | 
 84 |     # lifetime profit string
 85 |     profit_from_start_text = '+0 USD (0.00%)'  # default text
 86 |     # [-1] because we want to see what is our starting balance worth currently
 87 |     starting_total_value_USD = STARTING_BALANCE_USD + STARTING_BALANCE_BTC * float(columns['BTCUSDT_close'][-1])
 88 |     if columns['BTCUSDT_close'][-1] is not None:
 89 |         current_total_value_USD = CURRENT_BALANCE_USD + CURRENT_BALANCE_BTC * float(columns['BTCUSDT_close'][-1])
 90 |         # these are TOTAL values (USD + BTC * BTCUSD_close)
 91 |         # Profit from start. Displayed string: Runtime accuracy: 	52.8%, +1.62 USD (+1.6%)
 92 |         profit_from_start = current_total_value_USD - starting_total_value_USD
 93 |         profit_from_start_pct = ((current_total_value_USD / starting_total_value_USD) - 1) * 100
 94 |         if profit_from_start > 0:
 95 |             profit_from_start_text = f'+{profit_from_start:.2f} USD (+{profit_from_start_pct:.2f}%)'
 96 |         elif profit_from_start < 0:
 97 |             profit_from_start_text = f'{profit_from_start:.2f} USD ({profit_from_start_pct:.2f}%)'
 98 | 
 99 |     # 72h profit string
100 |     _72h_profit_text = '+0 USD (0.00%)'  # default text
101 |     # these are TOTAL values (USD + BTC * BTCUSD_close)
102 |     # if we just started bot, all balances in columns_72h['balance_USD'] will be None, we want to avoid that
103 |     if (columns_72h['balance_USD'][0] is not None) and (columns_72h['balance_BTC'][0] is not None):
104 |         starting_timestamp_dt = datetime.strptime(STARTING_TIMESTAMP, '%Y-%m-%d %H:%M:%S')
105 |         seconds_from_starting_timestamp = (datetime.utcnow() - starting_timestamp_dt).total_seconds()
106 |         # if 72h havent passed, the profits are the same
107 |         # +5 minutes just to be sure
108 |         if seconds_from_starting_timestamp < timedelta(hours=72, minutes=5).total_seconds():
109 |             _72h_profit_text = profit_from_start_text  # if 72h have not passed yet, both strings are the same
110 |         else:
111 |             _72h_total_value_USD = float(columns_72h['balance_USD'][0]) + float(columns_72h['balance_BTC'][0]) * float(columns_72h['BTCUSDT_close'][0])
112 |             _72h_profit = current_total_value_USD - _72h_total_value_USD
113 |             _72h_profit_pct = ((current_total_value_USD / _72h_total_value_USD) - 1) * 100
114 |             if _72h_profit > 0:
115 |                 _72h_profit_text = f'+{_72h_profit:.2f} USD (+{_72h_profit_pct:.2f}%)'
116 |             elif _72h_profit < 0:
117 |                 _72h_profit_text = f'{_72h_profit:.2f} USD ({_72h_profit_pct:.2f}%)'
118 | 
119 |     runtime_price_change_text = f'0.00% , +0.00 USD/BTC'
120 |     runtime_price_change = float(columns['BTCUSDT_close'][-1]) - float(columns['BTCUSDT_close'][71])
121 |     runtime_price_change_pct = (float(columns['BTCUSDT_close'][-1]) / float(columns['BTCUSDT_close'][71]) - 1) * 100
122 |     if runtime_price_change > 0:
123 |         runtime_price_change_text = f'+{runtime_price_change_pct:.2f}% (+{runtime_price_change:.2f} USD/BTC)'
124 |     elif runtime_price_change < 0:
125 |         runtime_price_change_text = f'{runtime_price_change_pct:.2f}% ({runtime_price_change:.2f} USD/BTC)'
126 | 
127 |     # conversion from booleans to displayed strings
128 |     if STATUS_ACTIVE is True:
129 |         STATUS_ACTIVE_text = 'Active'
130 |     else:
131 |         STATUS_ACTIVE_text = 'Disabled'
132 |     if ORDERS_ALLOWED is True:
133 |         ORDERS_ALLOWED_text = 'Allowed'
134 |     else:
135 |         ORDERS_ALLOWED_text = 'Disallowed'
136 |     if BACKUPS_ALLOWED is True:
137 |         BACKUPS_ALLOWED_text = 'Allowed'
138 |     else:
139 |         BACKUPS_ALLOWED_text = 'Disallowed'
140 | 
141 |     # Github backup time, shifted to UTC+0. Offset sometimes is -1 and sometimes -2, but it is returned by that command
142 |     last_commit_time = shift_hour_exact(*get_last_commit_time())  # unpacking string timestamp and integer offset
143 | 
144 |     return render_template('index.html',
145 |                            data_values=data_values_72h, data_index=data_index_72h,
146 |                            predictions=predictions_72h, orders=orders_72h,
147 |                            logs_data=logs_data,
148 |                            exact_starting_timestamp=exact_starting_timestamp,
149 |                            STATUS_ACTIVE_text=STATUS_ACTIVE_text,
150 |                            ORDERS_ALLOWED_text=ORDERS_ALLOWED_text,
151 |                            BACKUPS_ALLOWED_text=BACKUPS_ALLOWED_text,
152 |                            ENVIROMENT=ENVIROMENT,
153 |                            MODEL_PATH=MODEL_PATH,  # relative, just name
154 |                            STARTING_BALANCE_USD=f'{STARTING_BALANCE_USD:.2f}',
155 |                            STARTING_BALANCE_BTC=f'{STARTING_BALANCE_BTC:.7f}',
156 |                            starting_total_value_USD=f'{starting_total_value_USD:.2f}',
157 |                            CURRENT_BALANCE_USD=f'{CURRENT_BALANCE_USD:.2f}',
158 |                            CURRENT_BALANCE_BTC=f'{CURRENT_BALANCE_BTC:.7f}',
159 |                            current_total_value_USD=f'{current_total_value_USD:.2f}',
160 |                            accuracy_from_start=accuracy_from_start,
161 |                            profit_from_start_text=profit_from_start_text,
162 |                            accuracy_72h=accuracy_72h,
163 |                            _72h_profit_text=_72h_profit_text,
164 |                            runtime_price_change_text=runtime_price_change_text,
165 |                            last_commit_time=last_commit_time
166 |                            )
167 | 
168 | 
169 | if __name__ == '__main__':
170 |     app.run(host='0.0.0.0', port=5000, debug=True)  # on RPi it's set by gunicorn, which gets app from wsgi.py
171 | 


--------------------------------------------------------------------------------
/bot.py:
--------------------------------------------------------------------------------
 1 | 
 2 | from src.config import STATUS_ACTIVE, ORDERS_ALLOWED, BACKUPS_ALLOWED, set_bot_config, logger, LOG_FILE_ABS,\
 3 |                         STARTING_BALANCE_USD, STARTING_BALANCE_BTC
 4 | from src.data_update import update_all_data
 5 | from src.predict import predict_next
 6 | from src.data_backup import backup_data_from_RPi, pull_changes_from_remote
 7 | from src.orders import place_order, get_account_balances, convert_balances_to_USD
 8 | from src.process_logfile import process_logfile
 9 | from src.utils import shift_hour_trunc
10 | from datetime import datetime
11 | 
12 | 
13 | # bot.py is independent from app.py, however app.py requires processed logfile (done by bot.py) to draw the chart
14 | def main():
15 |     # Performed on PC:
16 |     # training and evaluation of the model
17 |     # simulation
18 | 
19 |     # Performed on RPi:
20 |     if STATUS_ACTIVE is True:
21 | 
22 |         # Runs once at start (when logfile.log is empty), writes first entries in logfile.log from config.ini
23 |         with open(LOG_FILE_ABS, 'rt') as f:
24 |             logfile_content = f.read().strip('\n').strip(' ')
25 |             if logfile_content == '':
26 |                 logger.info(f'Bot started')
27 |                 starting_balances = get_account_balances()  # this has try except inside when sending requests
28 |                 current_utc_timestamp = datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')
29 |                 if starting_balances is not False:
30 |                     set_bot_config(STARTING_BALANCE_USD=starting_balances['USD'],
31 |                                    STARTING_BALANCE_BTC=starting_balances['BTC'],
32 |                                    CURRENT_BALANCE_USD=starting_balances['USD'],
33 |                                    CURRENT_BALANCE_BTC=starting_balances['BTC'],
34 |                                    STARTING_TIMESTAMP=shift_hour_trunc(current_utc_timestamp, h=0)  # ..17:03:58 -> ..17:00:00
35 |                                    )
36 |                     logger.info(f'[BALANCE] {starting_balances["USD"]:.2f} USD, {starting_balances["BTC"]:.7f} BTC')
37 |                 else:
38 |                     print('Cannot load starting balances. Check the provided API keys in secrets.ini')
39 |                     return
40 | 
41 |         # missing values happens once per few months, it's better to check manually, so fill_missing=False
42 |         # if missing values appear, error will be raised and logged, and bot will be set inactive
43 |         update_successful = update_all_data(fill_missing=True)
44 |         prediction_result = 'EVEN'  # default value if no prediction is made
45 |         if update_successful is True:  # False in case if no new records available
46 |             prediction_result = predict_next()  # 'UP' 'DOWN' 'EVEN'
47 | 
48 |         if ORDERS_ALLOWED is True:
49 |             balances = get_account_balances()  # returns a dict with balances or False
50 |             if balances is not False:
51 |                 # bot buys and sells alternately, and side depends from if he has more BTC or USD - checking that here
52 |                 balances_in_USD = convert_balances_to_USD(balances)
53 |                 if (balances_in_USD['USD'] > balances_in_USD['BTC']) and (prediction_result == 'UP'):
54 |                     quantity = round(balances['USD'] * 0.995, 6)  # fixes "no sufficient amount", "precision err"
55 |                     order_filled = place_order(side='BUY', quantity=quantity, symbol='BTCUSDT', order_type='MARKET')
56 |                     if order_filled is not False:
57 |                         balances = get_account_balances()
58 |                         logger.info(f'[BALANCE] {balances["USD"]:.2f} USD, {balances["BTC"]:.7f} BTC')
59 |                         set_bot_config(CURRENT_BALANCE_USD=balances['USD'], CURRENT_BALANCE_BTC=balances['BTC'])
60 | 
61 |                 elif (balances_in_USD['USD'] <= balances_in_USD['BTC']) and (prediction_result == 'DOWN'):
62 |                     quantity = round(balances['BTC'] * 0.995, 6)
63 |                     order_filled = place_order(side='SELL', quantity=quantity, symbol='BTCUSDT', order_type='MARKET')
64 |                     if order_filled is not False:
65 |                         balances = get_account_balances()
66 |                         logger.info(f'[BALANCE] {balances["USD"]:.2f} USD, {balances["BTC"]:.7f} BTC')
67 |                         set_bot_config(CURRENT_BALANCE_USD=balances['USD'], CURRENT_BALANCE_BTC=balances['BTC'])
68 |                 else:
69 |                     pass  # bot alternates between BUY/SELL state so no need more orders conditions
70 | 
71 |         process_logfile()  # process logfile to get accuracies and information for chart and save data as logfile.csv
72 | 
73 |         if (update_successful is True) and (BACKUPS_ALLOWED is True):  # False in case if no new records available
74 |             pull_changes_from_remote()
75 |             backup_data_from_RPi()  # use with ssh key for github in default location without passphrase
76 | 
77 |     else:  # when STATUS_ACTIVE is False, bot does nothing
78 |         pass
79 | 
80 | 
81 | if __name__ == "__main__":
82 |     main()
83 | 
84 | 


--------------------------------------------------------------------------------
/models/model/LSTM-2021-08-23.tflite:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kamsec/crypto-bot/4d247efe76a71ca7a8e7a638f48f791a06f428b5/models/model/LSTM-2021-08-23.tflite


--------------------------------------------------------------------------------
/requirements_rpi.txt:
--------------------------------------------------------------------------------
 1 | aiohttp==3.7.4.post0
 2 | async-timeout==3.0.1
 3 | attrs==21.2.0
 4 | backports.zoneinfo==0.2.1
 5 | certifi==2021.5.30
 6 | chardet==4.0.0
 7 | charset-normalizer==2.0.4
 8 | click==8.0.1
 9 | dateparser==1.0.0
10 | Flask==2.0.1
11 | gunicorn==20.1.0
12 | idna==3.2
13 | importlib-metadata==4.6.4
14 | itsdangerous==2.0.1
15 | Jinja2==3.0.1
16 | joblib==1.0.1
17 | MarkupSafe==2.0.1
18 | multidict==5.1.0
19 | numpy==1.21.2
20 | pandas==1.3.2
21 | pybind11==2.7.1
22 | python-binance==1.0.12
23 | python-dateutil==2.8.2
24 | pytz==2021.1
25 | regex==2021.8.3
26 | requests==2.26.0
27 | scikit-learn==0.24.2
28 | scipy==1.7.1
29 | six==1.16.0
30 | threadpoolctl==2.2.0
31 | typing-extensions==3.10.0.0
32 | tzlocal==3.0
33 | ujson==4.1.0
34 | urllib3==1.26.6
35 | websockets==9.1
36 | Werkzeug==2.0.1
37 | yarl==1.6.3
38 | zipp==3.5.0
39 | ./whl/tflite_runtime-2.3.1-cp37-cp37m-linux_armv6l.whl; platform_machine == 'armv6l'
40 | ./whl/tflite_runtime-2.5.0-cp37-cp37m-linux_armv7l.whl; platform_machine == 'armv7l'
41 | 


--------------------------------------------------------------------------------
/requirements_win.txt:
--------------------------------------------------------------------------------
 1 | --extra-index-url=https://google-coral.github.io/py-repo/
 2 | aiohttp==3.7.4.post0
 3 | async-timeout==3.0.1
 4 | attrs==21.2.0
 5 | backports.zoneinfo==0.2.1
 6 | certifi==2021.5.30
 7 | chardet==4.0.0
 8 | charset-normalizer==2.0.4
 9 | click==8.0.1
10 | dateparser==1.0.0
11 | Flask==2.0.1
12 | gunicorn==20.1.0
13 | idna==3.2
14 | importlib-metadata==4.6.4
15 | itsdangerous==2.0.1
16 | Jinja2==3.0.1
17 | joblib==1.0.1
18 | MarkupSafe==2.0.1
19 | multidict==5.1.0
20 | numpy==1.21.2
21 | pandas==1.3.2
22 | pybind11==2.7.1
23 | python-binance==1.0.12
24 | python-dateutil==2.8.2
25 | pytz==2021.1
26 | regex==2021.8.3
27 | requests==2.26.0
28 | scikit-learn==0.24.2
29 | scipy==1.7.1
30 | six==1.16.0
31 | threadpoolctl==2.2.0
32 | typing-extensions==3.10.0.0
33 | tzlocal==3.0
34 | ujson==4.1.0
35 | urllib3==1.26.6
36 | websockets==9.1
37 | Werkzeug==2.0.1
38 | yarl==1.6.3
39 | zipp==3.5.0
40 | tflite_runtime==2.5.0; platform_machine == 'AMD64'
41 | 


--------------------------------------------------------------------------------
/screenshot.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kamsec/crypto-bot/4d247efe76a71ca7a8e7a638f48f791a06f428b5/screenshot.png


--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
 1 | 
 2 | import configparser
 3 | import os
 4 | from datetime import datetime
 5 | 
 6 | from src.utils import shift_hour_trunc
 7 | 
 8 | 
 9 | SECRETS_FILE = 'secrets.ini'
10 | CONFIG_NAME = 'config.ini'
11 | LOG_FILE = 'logfile.log'
12 | 
13 | MODEL_NAME = 'LSTM-2021-08-23.tflite'
14 | PROJECT_PATH_ABS = os.path.abspath(os.path.join(os.path.dirname(__file__)))
15 | 
16 | def setup():
17 |     # SECRETS
18 |     cfg_secrets = configparser.ConfigParser()
19 |     if not os.path.exists(SECRETS_FILE):
20 |         cfg_secrets['DEFAULT'] = {'TESTNET_API_KEY': '***',
21 |                                   'TESTNET_API_SECRET': '***',
22 |                                   'MAINNET_API_KEY': '***',
23 |                                   'MAINNET_API_SECRET': '***',
24 |                                   }
25 |         with open(SECRETS_FILE, 'w') as secretsfile:
26 |             cfg_secrets.write(secretsfile)
27 |             print('Created secrets.ini.')
28 | 
29 |     # CONFIG
30 |     cfg = configparser.ConfigParser()
31 |     if not os.path.exists(CONFIG_NAME):
32 |         current_utc_timestamp = datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')
33 |         cfg['DEFAULT'] = {'SEQ_LEN': '48',
34 |                           'RATIOS': '["BTCUSDT", "LTCUSDT", "ETHUSDT", "BNBUSDT", "ADAUSDT", "BTTUSDT", "DASHUSDT",'
35 |                                     '"XMRUSDT", "NANOUSDT", "DOGEUSDT", "XLMUSDT", "BCHUSDT"]',
36 |                           'RATIO_TO_PREDICT': 'BTCUSDT',
37 |                           'MODEL_PATH': f'models/model/{MODEL_NAME}',
38 |                           'DATA_PATH': 'data/',
39 |                           'STATUS_ACTIVE': True,
40 |                           'ORDERS_ALLOWED': False,
41 |                           'BACKUPS_ALLOWED  ': False,
42 |                           'ENVIROMENT': 'testnet',
43 |                           'STARTING_BALANCE_USD': 0,
44 |                           'STARTING_BALANCE_BTC': 0,
45 |                           'STARTING_TIMESTAMP': shift_hour_trunc(current_utc_timestamp, h=0),  # ..17:03:58 ->..17:00:00
46 |                           'CURRENT_BALANCE_USD': 0,
47 |                           'CURRENT_BALANCE_BTC': 0
48 |                           }
49 |         with open(CONFIG_NAME, 'w') as configfile:
50 |             cfg.write(configfile)
51 |             print('Created config.ini')
52 | 
53 |     # LOGGING
54 |     if not os.path.exists(LOG_FILE):
55 |         with open(LOG_FILE, 'w') as f:
56 |             f.write('')
57 |             print('Created logfile.log')
58 | 
59 | 
60 | if __name__ == "__main__":
61 |     setup()
62 | 
63 | 


--------------------------------------------------------------------------------
/src/config.py:
--------------------------------------------------------------------------------
 1 | 
 2 | import time
 3 | import json
 4 | import logging
 5 | import configparser
 6 | import os
 7 | 
 8 | from setup import CONFIG_NAME, LOG_FILE, SECRETS_FILE, PROJECT_PATH_ABS
 9 | 
10 | # assumed config.py in crypto-bot/src, config.ini in crypto-bot/, setup.py in crypto-bot/
11 | CONFIG_PATH_ABS = os.path.abspath(os.path.join(PROJECT_PATH_ABS, CONFIG_NAME))
12 | 
13 | cfg = configparser.ConfigParser()
14 | 
15 | cfg.read(CONFIG_PATH_ABS)
16 | 
17 | SEQ_LEN = cfg['DEFAULT'].getint('SEQ_LEN')
18 | RATIOS = json.loads(cfg['DEFAULT']['RATIOS'])
19 | RATIO_TO_PREDICT = cfg['DEFAULT']['RATIO_TO_PREDICT']
20 | 
21 | # for tensorflow 2.5 'models/model', for TFLite you need to specify 'models/model/model.tflite'
22 | MODEL_PATH = cfg['DEFAULT']['MODEL_PATH']
23 | MODEL_PATH_ABS = os.path.abspath(os.path.join(PROJECT_PATH_ABS, MODEL_PATH))
24 | 
25 | DATA_PATH = cfg['DEFAULT']['DATA_PATH']
26 | DATA_PATH_ABS = os.path.abspath(os.path.join(PROJECT_PATH_ABS, DATA_PATH)) + os.path.sep  # ending with /
27 | 
28 | STATUS_ACTIVE = cfg['DEFAULT'].getboolean('STATUS_ACTIVE')
29 | ORDERS_ALLOWED = cfg['DEFAULT'].getboolean('ORDERS_ALLOWED')
30 | BACKUPS_ALLOWED = cfg['DEFAULT'].getboolean('BACKUPS_ALLOWED')
31 | ENVIROMENT = cfg['DEFAULT']['ENVIROMENT']  # testnet or mainnet
32 | 
33 | # logging
34 | # Level Numeric value
35 | # CRITICAL 50
36 | # ERROR 40
37 | # WARNING 30
38 | # INFO 20
39 | # DEBUG 10
40 | # NOTSET 0
41 | LOG_FILE_ABS = os.path.abspath(os.path.join(PROJECT_PATH_ABS, LOG_FILE))
42 | 
43 | logging.basicConfig(filename=LOG_FILE_ABS,
44 |                     level=logging.INFO,  # DEBUG is the lowest - the most information
45 |                     format="%(asctime)s [%(levelname)s] %(message)s",
46 |                     datefmt='%Y-%m-%d %H:%M:%S')
47 | logging.Formatter.converter = time.gmtime  # logs will be in gmt (UTC+0)
48 | logger = logging.getLogger(__name__)
49 | 
50 | STARTING_BALANCE_USD = cfg['DEFAULT'].getfloat('starting_balance_USD')
51 | STARTING_BALANCE_BTC = cfg['DEFAULT'].getfloat('starting_balance_BTC')
52 | STARTING_TIMESTAMP = cfg['DEFAULT']['starting_timestamp']
53 | 
54 | CURRENT_BALANCE_USD = cfg['DEFAULT'].getfloat('current_balance_USD')
55 | CURRENT_BALANCE_BTC = cfg['DEFAULT'].getfloat('current_balance_BTC')
56 | 
57 | def set_bot_config(**kwargs):  # e.g. set_bot_config(STATUS_ACTIVE=False)
58 |     for key, value in kwargs.items():
59 |         if key in cfg['DEFAULT']:
60 |             cfg.set('DEFAULT', key, str(value))
61 |             if (key == 'STATUS_ACTIVE') or (key == 'ORDERS_ALLOWED') or (key == 'BACKUPS_ALLOWED'):
62 |                 logger.info(f'[CONFIG] {key} set to {value}')
63 |         else:
64 |             raise Exception(f'Config has no {key} setting')
65 |     with open(CONFIG_PATH_ABS, 'w') as configfile:
66 |         cfg.write(configfile)
67 | 


--------------------------------------------------------------------------------
/src/data_backup.py:
--------------------------------------------------------------------------------
 1 | 
 2 | import subprocess
 3 | from src.config import logger, set_bot_config
 4 | 
 5 | 
 6 | def git_error_check(proc):
 7 |     proc_returncode = proc.returncode
 8 |     if proc_returncode is not 0:
 9 |         proc_stderr = proc.stderr
10 |         error = str(proc_stderr)[2:-3].replace(r'\n', '\n')  # 2 removes b', -3 removes \n'
11 |         logger.error(f'[BACKUP] Git backup error. Message: {error}')
12 |         set_bot_config(BACKUPS_ALLOWED=False)
13 |         raise Exception(f'Git backup error. Message: {error}')
14 | 
15 | def pull_changes_from_remote():
16 |     # git pull
17 | 
18 |     proc = subprocess.run(["git", "pull"], capture_output=True)
19 |     git_error_check(proc)
20 | 
21 | def backup_data_from_RPi():
22 |     # git add .
23 |     # git commit -m "data update"
24 |     # git push
25 | 
26 |     proc = subprocess.run(["git", "add", "."], capture_output=True)
27 |     git_error_check(proc)
28 |     proc = subprocess.run(["git", "commit", "-m", "Data update by RPi"], capture_output=True)
29 |     git_error_check(proc)
30 |     proc = subprocess.run(["git", "push"], capture_output=True)
31 |     git_error_check(proc)
32 |     return
33 | 
34 | # commit, no push, but whatever
35 | def get_last_commit_time():
36 |     # git --no-pager log -1 --format=%ai
37 | 
38 |     proc = subprocess.run(["git", "--no-pager", "log", "-1", "--format=%ai"], capture_output=True)
39 |     # output is b'2021-08-23 19:27:51' +0200 or and sometimes +0100, soi need to grab that offset as well
40 |     last_commit_time = str(proc.stdout)[2:21]
41 |     last_commit_offset = -int(str(proc.stdout)[24])  # -2 or -1 so can be passed directly as h= utils functions
42 |     return last_commit_time, last_commit_offset
43 | 
44 | 


--------------------------------------------------------------------------------
/src/data_update.py:
--------------------------------------------------------------------------------
  1 | import time
  2 | import pandas as pd
  3 | import numpy as np
  4 | from datetime import datetime, timedelta
  5 | from requests import get
  6 | from json import loads
  7 | 
  8 | from src.utils import UTC_OFFSET, unix_to_datetime, datetime_to_unix
  9 | from src.config import RATIOS, logger, DATA_PATH_ABS, set_bot_config
 10 | 
 11 | 
 12 | def update_ratio(ratio, filename, request_records=500, fill_missing=False):
 13 |     old_df = pd.read_csv(filename, nrows=None, skiprows=None, parse_dates=['date'], usecols=[0, 1, 2])
 14 |     old_df.set_index("date", inplace=True)
 15 |     last_record_dt = old_df.tail(1).index.item()  # get last record (1 element of tail) in old csv that is being updated
 16 |     last_record_unix = datetime_to_unix(str(last_record_dt), h=-UTC_OFFSET)  # -UTC_OFFSET because of timezones
 17 | 
 18 |     '''
 19 |     logger.debug(f'Existing records: {len(old_df)}   |   '
 20 |                  f'Last close: {old_df.tail(1).index.item()} {old_df[f"{ratio}_close"][-1]}')
 21 |     '''
 22 |     seconds_from_last_update = (datetime.utcnow() - last_record_dt).total_seconds()
 23 |     # e.g. 18:00 close is like 18:59:59 so we can get 18:00 close price (UTC+0) at 18:59:59 so 19:01
 24 |     # so we can request new price after constant 2 hours passed from last record (+ few seconds just to be sure)
 25 |     # also thats why cron job is set to 1 * * * *
 26 |     seconds_remaining = timedelta(hours=2, seconds=10).seconds - seconds_from_last_update
 27 |     if seconds_remaining > 0:
 28 |         logger.warning(f'[UPDATE] No new records available yet. Try again in: {str(timedelta(seconds=seconds_remaining))}')
 29 |         return False
 30 | 
 31 |     # example:
 32 |     # curl -X GET "https://api.binance.com/api/v1/klines?&symbol=BTCUSDT&interval=3m"
 33 |     # https://binance-docs.github.io/apidocs/spot/en/#kline-candlestick-data
 34 |     times_repeat_download = int((-seconds_remaining // (request_records * 3600))) + 1
 35 |     new_df = pd.DataFrame(columns=[f'{ratio}_close', f'{ratio}_volume'])
 36 |     for i in range(0, times_repeat_download):
 37 |         temp_df = pd.DataFrame(columns=['date', f'{ratio}_close', f'{ratio}_volume'])
 38 |         response = get('https://api.binance.com/api/v1/klines',
 39 |                        params={'symbol': ratio,
 40 |                                'interval': '1h',
 41 |                                'startTime': last_record_unix + (i * request_records * 3600 * 1000),
 42 |                                'limit': request_records})  # last_record_unix also limits number of records, if it's recent
 43 |         raw_data = loads(response.content)
 44 |         raw_data_df = pd.DataFrame(raw_data)
 45 | 
 46 |         # we receive data in UTC+2 timezone, so need to set h=-2, so we pass list of [-2] as second argument
 47 |         temp_df['date'] = list(map(unix_to_datetime, raw_data_df[0], [UTC_OFFSET] * len(raw_data_df[0])))
 48 |         temp_df[f'{ratio}_close'] = list(map(float, raw_data_df[4]))  # df[4]  # Close
 49 |         temp_df[f'{ratio}_volume'] = list(map(float, raw_data_df[7]))  # df[7]  # Quote asset volume (second assets unit, so in BTC/USDT its volume_USDT)  # noqa
 50 |         temp_df.set_index("date", inplace=True)
 51 |         if i == 0:  # in first iteration we drop temp_df[0] because its the same as in old_df[-1]
 52 |             temp_df = temp_df[1:]
 53 |         new_df = pd.concat([new_df, temp_df])
 54 |     # if we send request at 15:27 (UTC-0 13:27) it will return data with last entry 13:00 which will change over time
 55 |     new_df = new_df[:-1]  # dropping that entry
 56 | 
 57 |     '''
 58 |     logger.debug(f'Received {len(new_df)} records.   |   '
 59 |                  f'Last: {new_df.head(1).index.item()} {new_df[f"{ratio}_close"][0]}   |   '
 60 |                  f'First: {new_df.tail(1).index.item()} {new_df[f"{ratio}_close"][-1]}')
 61 |     # python floats (and interpolation) might cause that old_df[-1] is not equal new_df[0] but thats ok,
 62 |     # e.g.: old_df[-1] is not equal new_df[0]: 0.23510999999999999 != 0.23511 but this won't be a problem
 63 |     '''
 64 | 
 65 |     # check for gaps in timestamps and interpolate if found
 66 |     new_df.index = pd.to_datetime(new_df.index)
 67 |     expected_range = pd.date_range(start=new_df.index.min(), end=new_df.index.max(), freq='H')
 68 | 
 69 |     missing_ts = list(expected_range.difference(new_df.index))
 70 |     if missing_ts and (fill_missing is True):
 71 |         new_df = new_df.resample('60min').mean()
 72 |         new_df.replace(to_replace=0, value=np.nan, inplace=True)  # converting 0s to NaNs
 73 |         new_df.interpolate(method='linear', inplace=True)  # interpolating missing NaNs
 74 | 
 75 |         logger.warning(f'[UPDATE] Missing {len(missing_ts)} timestamps in {ratio} pair. Performed interpolation.')
 76 | 
 77 |         logger.debug(f'{ratio} updated succesfully. All new records: {len(new_df)}   |   '
 78 |                      f'First: {new_df.tail(1).index.item()} {new_df[f"{ratio}_close"][-1]}\n')
 79 |         return new_df
 80 |     elif missing_ts and (fill_missing is False):
 81 |         logger.error(f'[UPDATE] Missing {len(missing_ts)} timestamps. Update skipped, check manually.')
 82 |         set_bot_config(STATUS_ACTIVE=False)
 83 |         raise Exception(f'Missing {len(missing_ts)} timestamps. Update skipped, bot set inactive.')
 84 |     else:
 85 |         return new_df
 86 | 
 87 | 
 88 | def update_all_data(fill_missing=False):
 89 |     dict_name_ratio_df = {}
 90 |     for ratio in RATIOS:
 91 |         filename = f"{DATA_PATH_ABS}Binance_{ratio}_1h.csv"
 92 |         time.sleep(0.2)  # to not spam the exchange
 93 |         # 336 hourly records = 14 days, if files havent been updated for longer time,
 94 |         # just update them again until "No new records available yet." encountered
 95 |         new_df = update_ratio(ratio, filename, request_records=336, fill_missing=fill_missing)
 96 |         dict_name_ratio_df[f'{filename}'] = new_df
 97 |         if new_df is False:  # some new_df will be False if not enough time have passed from last update
 98 |             return False
 99 | 
100 |     for filename, _df in dict_name_ratio_df.items():
101 |         _df.to_csv(filename, mode='a', header=False)
102 |     return True
103 | 
104 | 
105 | if __name__ == "__main__":
106 |     update_all_data()
107 | 


--------------------------------------------------------------------------------
/src/orders.py:
--------------------------------------------------------------------------------
  1 | 
  2 | import configparser
  3 | import os
  4 | import time
  5 | from json import loads
  6 | from requests import get
  7 | from binance.client import Client
  8 | 
  9 | from src.config import RATIOS, logger, set_bot_config, ENVIROMENT, PROJECT_PATH_ABS, DATA_PATH_ABS, SECRETS_FILE
 10 | 
 11 | # doing it here since only functions in this file use authentication on Binance. E.g. Flask doesnt need that
 12 | cfg_secrets = configparser.ConfigParser()
 13 | SECRETS_FILE_ABS = os.path.abspath(os.path.join(PROJECT_PATH_ABS, SECRETS_FILE))
 14 | cfg_secrets.read(SECRETS_FILE_ABS)
 15 | 
 16 | if ENVIROMENT == 'testnet':
 17 |     TESTNET_API_KEY = cfg_secrets['DEFAULT']['TESTNET_API_KEY']
 18 |     TESTNET_API_SECRET = cfg_secrets['DEFAULT']['TESTNET_API_SECRET']
 19 |     client = Client(TESTNET_API_KEY, TESTNET_API_SECRET, testnet=True)
 20 | elif ENVIROMENT == 'mainnet':
 21 |     MAINNET_API_KEY = cfg_secrets['DEFAULT']['MAINNET_API_KEY']
 22 |     MAINNET_API_SECRET = cfg_secrets['DEFAULT']['MAINNET_API_SECRET']
 23 |     client = Client(MAINNET_API_KEY, MAINNET_API_SECRET, testnet=False)
 24 | 
 25 | def get_account_balances():
 26 |     try:
 27 |         BTC = client.get_asset_balance(asset='BTC')['free']
 28 |         USD = client.get_asset_balance(asset='USDT')['free']  # assumed 1:1 USD/USDT ratio, using 'USD' name
 29 |     except Exception as e:
 30 |         logger.error(f'[get_account_balances] {e}')
 31 |         return False
 32 |     # print(f'{USD} USD, {BTC} BTC')
 33 |     return {'BTC': float(BTC), 'USD': float(USD)}
 34 | 
 35 | 
 36 | def get_last_price():
 37 |     try:
 38 |         # 'https://testnet.binance.vision/api/v3/ticker/price?symbol=BTCUSDT'  # for testnet
 39 |         # 'https://api.binance.com/api/v3/ticker/price?symbol=BTCUSDT'  # for mainnet
 40 |         if ENVIROMENT == 'testnet':
 41 |             url = 'https://testnet.binance.vision/api/v3/ticker/price'
 42 |         elif ENVIROMENT == 'mainnet':
 43 |             url = 'https://api.binance.com/api/v3/ticker/price'
 44 |         response = get(url, params={'symbol': 'BTCUSDT'})
 45 |     except Exception as e:
 46 |         logger.error(f'[get_last_price] {e}')
 47 |         return False
 48 |     return float(loads(response.text)['price'])
 49 | 
 50 | 
 51 | def get_last_data_price():
 52 |     with open(f"{DATA_PATH_ABS}Binance_BTCUSDT_1h.csv", 'r') as file:
 53 |         last_data_price = float(file.readlines()[-1].split(',')[1])  # -1 bcs last line and 1 bcs (date, close, volume)
 54 |     return last_data_price
 55 | 
 56 | 
 57 | def convert_balances_to_USD(balances_dict):
 58 |     new_balances_dict = balances_dict.copy()
 59 |     last_data_price = get_last_data_price()
 60 |     new_balances_dict['BTC'] = balances_dict['BTC'] * last_data_price
 61 |     new_balances_dict['USD'] = balances_dict['USD']
 62 |     return new_balances_dict  # {'BTC': BTC, 'USD': USD} and USD is not changed
 63 | 
 64 | 
 65 | # quantity is translated to quantity or quoteOrderQty. quantity is always known, so no additional calculations needed
 66 | def place_order(side, quantity, symbol, order_type):  # side='SELL', quantity=0.1, symbol='BTCUSDT', order_type='MARKET'
 67 | 
 68 |     # checking actual price, to prevent error (if no enough liquidity in the market, etc)
 69 |     last_data_price = get_last_data_price()  # last price from data/ folder
 70 |     max_diff = last_data_price * 0.02  # for 45000 it is 900, very unlikely to make 2% change in few minutes
 71 |     price_confirmed = False
 72 |     for i in range(0, 10):
 73 |         last_price = get_last_price()  # last price from the exchange ticker
 74 |         if abs(last_data_price - last_price) > max_diff:
 75 |             time.sleep(5)  # sleep 5 seconds, wait for price change
 76 |             continue
 77 |         else:
 78 |             price_confirmed = True
 79 |             break
 80 |     if price_confirmed is False:  # function returns False in this case
 81 |         logger.warning(f'[ORDER CANCELLED] Market price changed too much, {last_data_price:.7f} -> {last_price:.7f}')
 82 |         return False
 83 | 
 84 |     # placing order
 85 |     if side == 'BUY':  # e.g buy x BTC for 80 USD so quoteOrderQty=80
 86 |         try:
 87 |             order = client.create_order(side=side, quoteOrderQty=quantity, symbol=symbol, type=order_type)
 88 |         except Exception as e:
 89 |             set_bot_config(ORDERS_ALLOWED=False)
 90 |             logger.error(f'[place_order] side={side}, quoteOrderQty={quantity} {e}')
 91 |             return False
 92 |     elif side == 'SELL':  # e.g sell 0.0003BTC for 80 USD so quantity=0.0003
 93 |         try:
 94 |             order = client.create_order(side=side, quantity=quantity, symbol=symbol, type=order_type)
 95 |         except Exception as e:
 96 |             set_bot_config(ORDERS_ALLOWED=False)
 97 |             logger.error(f'[place_order] side={side}, quantity={quantity} {e}')
 98 |             return False
 99 | 
100 |     # e.g. in BTCUSDT, BTC is base asset and USDT is quote asset
101 |     quote_amount = float(order["cummulativeQuoteQty"])  # USD (USDT but we assume 1:1)
102 |     base_amount = float(order['executedQty'])  # BTC
103 |     fills = order["fills"]
104 |     status = order["status"]
105 |     # fee = 0.001  # not dealing with fees, we check the balances later from the exchange anyway
106 | 
107 |     # order that is partially filled is still in order book and waiting for filling so no need to renew it
108 |     if len(fills) > 0:  # its better than status == 'FILLED': because EXPIRED can also be partially filled :/
109 |         avg_price = quote_amount / base_amount
110 |         # format of logs e.g.:
111 |         # 2021-07-27 05:05:51 [INFO] [ORDER] [FILLED] BUY 0.0003523 BTC for 23.12 USD at 33123.12 USD/BTC in 1 FILL
112 |         # 2021-07-27 09:05:53 [INFO] [ORDER] [FILLED] SELL 0.0003523 BTC for 23.89 USD at 33242.43 USD/BTC in 2 FILLS
113 |         if side == 'BUY':
114 |             logger.info(f'[ORDER] [{status}] {side} {base_amount:.7f} BTC for {quote_amount:.2f} USD'
115 |                         f' at {avg_price:.2f} USD/BTC in {len(fills)} FILL{"S" if len(fills) > 1 else ""}')
116 |         elif side == 'SELL':
117 |             logger.info(f'[ORDER] [{status}] {side} {base_amount:.7f} BTC for {quote_amount:.2f} USD'
118 |                         f' at {avg_price:.2f} USD/BTC in {len(fills)} FILL{"S" if len(fills) > 1 else ""}')
119 |     else:
120 |         logger.warning(f'[ORDER] [{status}] Response: {order}')
121 |         set_bot_config(ORDERS_ALLOWED=False)
122 |         return False
123 |     return True
124 | 


--------------------------------------------------------------------------------
/src/predict.py:
--------------------------------------------------------------------------------
  1 | 
  2 | import numpy as np
  3 | import pandas as pd
  4 | import tflite_runtime.interpreter as tflite
  5 | from sklearn import preprocessing
  6 | from collections import deque
  7 | from src.config import RATIOS, RATIO_TO_PREDICT, SEQ_LEN, MODEL_PATH_ABS, DATA_PATH_ABS, logger
  8 | 
  9 | # windows: C:\Users\Admin\Desktop\python37\Scripts>pip3 install --index-url https://google-coral.gith
 10 | # rpi: tflite_runtime-2.3.1-cp37-cp37m-linux_armv6l.whl
 11 | 
 12 | 
 13 | def construct_df_for_pred(ratios):
 14 |     main_df = pd.DataFrame()
 15 |     # 'unix,date,symbol,open,high,low,close,Volume LTC,Volume USDT,tradecount'
 16 |     for ratio in ratios:
 17 |         dataset = f"{DATA_PATH_ABS}Binance_{ratio}_1h.csv"
 18 |         df = pd.read_csv(dataset, nrows=None, skiprows=None, parse_dates=['date'], usecols=[0, 1, 2])
 19 |         df.set_index("date", inplace=True)
 20 | 
 21 |         # later we use pct_change so we lose 1 row and need 1 more (-49 for single pred)
 22 |         # but there is no future and target column, so we dont waste one last row, so removing "+1"
 23 |         num_of_preds = 1
 24 |         short_df = df[-SEQ_LEN - num_of_preds:]
 25 | 
 26 |         if len(main_df) == 0:
 27 |             main_df = short_df
 28 |         else:
 29 |             main_df = main_df.join(short_df)
 30 | 
 31 |     # in this case it reduces main_df length to the length of pair with the smallest data
 32 |     main_df.dropna(how='any', inplace=True)
 33 |     return main_df
 34 | 
 35 | 
 36 | # percentage change and scaling
 37 | def preprocess_data_for_pred(df):
 38 |     df = df.copy()  # fixes random SettingWithCopyWarning appearing with pct_change and preprocessing.scale
 39 |     for col in df.columns:
 40 |         # here we lose 1 row
 41 |         df[col] = df[col].pct_change()  # converted to percentage change
 42 |         # example:
 43 |         # hour 1:00 close = 93.13, hour 2:00 close = 92.41, so
 44 |         # 93.13 + 93.13 * x = 92.41, x = -0.007731
 45 |         # so percentage change of 1.0 means doubling the price
 46 |         # first row (the oldest values) is being filled with NaN percentage, but we don't want to drop whole row yet
 47 |         # so we wait until loop ends (we will have whole row filled with nans)
 48 |     df.dropna(inplace=True)
 49 | 
 50 |     for col in df.columns:
 51 |         if col != "target":
 52 |             # scaling should be used after splitting to training and test set - leaks data from test set to training
 53 |             df[col] = preprocessing.scale(df[col].values)  # normalizes to [-1, +1]
 54 |             # df[col] = preprocessing.StandardScaler().fit_transform(df[col].values.reshape(-1, 1)) # it does the same
 55 |             # scales the data so it has mean = 0, standard deviation = 1, variance = 1
 56 |             # it can be checked before and after with e.g. print(df['BTCUSDT_close'].mean())  .std()  .var()
 57 | 
 58 |     # here we always will have sequential_data with one array 1x48x24
 59 |     prev_days = deque(maxlen=SEQ_LEN)  # list of SEQ_LEN items, we append to it, as it exceed, it pops old values
 60 |     sequential_data = []
 61 |     for i in df.values:
 62 |         prev_days.append([n for n in i])  # set of price and volume features added (-1 so without target)
 63 |         if len(prev_days) == SEQ_LEN:
 64 |             sequential_data.append(np.array([prev_days]))  # [] is necessary to be compatible with model input
 65 |     # the sequences are like this ([a, b, 1] is single prev_days):
 66 |     # [([[a, b] [c, d], [e, f]], 1)   ([[c, d] [e, f] [g, h]], 1) ... ([[g, h], [i, j], [k, l]], 0)]
 67 |     return sequential_data
 68 | 
 69 | 
 70 | def predict_next():
 71 |     main_df = construct_df_for_pred(RATIOS)
 72 |     preprocessed_sequences = preprocess_data_for_pred(main_df)
 73 | 
 74 |     interpreter = tflite.Interpreter(model_path=MODEL_PATH_ABS)
 75 |     interpreter.allocate_tensors()
 76 |     input_details = interpreter.get_input_details()
 77 |     output_details = interpreter.get_output_details()
 78 |     data_for_tflite = np.array(preprocessed_sequences[0], dtype=np.float32)  # doesnt work without dtype=np.float32
 79 |     interpreter.set_tensor(input_details[0]['index'], data_for_tflite)
 80 |     interpreter.invoke()
 81 | 
 82 |     predictions = interpreter.get_tensor(output_details[0]['index'])
 83 |     probabilities = predictions[0]
 84 | 
 85 |     def classifier(prob):
 86 |         if round(prob, 2) > 0.50:
 87 |             return 'UP'
 88 |         elif round(prob, 2) == 0.50:
 89 |             # when bot is inactive it can be treated as 'EVEN' prediction as well, because it has no influence on orders
 90 |             return 'EVEN'
 91 |         elif round(prob, 2) < 0.50:
 92 |             return 'DOWN'
 93 | 
 94 |     prediction = classifier(probabilities[1])
 95 |     logger.info(f'[PREDICTION] DOWN: {probabilities[0]*100:.0f}%, UP: {probabilities[1]*100:.0f}%')
 96 |     return prediction
 97 | 
 98 | 
 99 | if __name__ == "__main__":
100 |     predict_next()
101 | 


--------------------------------------------------------------------------------
/src/process_logfile.py:
--------------------------------------------------------------------------------
 1 | 
 2 | import numpy as np
 3 | import pandas as pd
 4 | from src.utils import datetime_to_unix, unix_to_datetime, shift_hour_trunc
 5 | from src.config import LOG_FILE_ABS, DATA_PATH_ABS, STARTING_TIMESTAMP, STARTING_BALANCE_USD, STARTING_BALANCE_BTC
 6 | 
 7 | 
 8 | def process_logfile():
 9 |     def classifier(current, future):
10 |         if current < future:
11 |             return 'UP'
12 |         elif current == future:
13 |             return np.nan
14 |         elif current > future:
15 |             return 'DOWN'
16 | 
17 |     df = pd.read_csv(f'{DATA_PATH_ABS}Binance_BTCUSDT_1h.csv', parse_dates=['date'], usecols=[0, 1])  # volume not required
18 |     df.set_index("date", inplace=True)
19 |     # shifted by 72h for 72h chart, so if we pass current date as STARTING_TIMESTAMP, it will still plot full chart
20 |     df = df.truncate(before=shift_hour_trunc(STARTING_TIMESTAMP, h=-72))
21 |     df['future'] = df[f"BTCUSDT_close"].shift(-1)
22 |     df['target'] = list(map(classifier, df[f"BTCUSDT_close"], df["future"]))
23 |     # df.drop('future', 1, inplace=True)  # future is not necessary from now but no need to drop it
24 | 
25 |     df['predictions_shifted'] = None
26 |     df['accurate_prediction'] = None
27 |     df['orders_shifted'] = None
28 |     df['balance_USD'] = np.nan  # np.nan instead of None so we can use interpolate (fill nans with nearest vals) later
29 |     df['balance_BTC'] = np.nan
30 | 
31 |     balance_USD = STARTING_BALANCE_USD
32 |     balance_BTC = STARTING_BALANCE_BTC
33 |     with open(LOG_FILE_ABS, 'rt') as f:
34 |         logs_data_lines = [line for line in f.readlines() if line.strip()]  # strips empty lines
35 |         for line in logs_data_lines:
36 |             if line[-1] == '\n':
37 |                 line = line[:-1]  # [:-1] removes /n
38 |             timestamp = line[:19]
39 |             # using "at" vs "loc" should be faster, but for some reason here it isn't
40 |             if '[PREDICTION]' in line:
41 |                 if 'UP: ' in line:  # should be more optimized than with "and"
42 |                     up_percentage = int(line.partition(f'UP: ')[2][:-1])  # 2 bcs ('.', 'UP: ', '52%) and -1 to remove %
43 |                     if up_percentage > 50:
44 |                         df.at[shift_hour_trunc(timestamp, h=-1), 'predictions_shifted'] = 'UP'
45 |                     elif up_percentage < 50:
46 |                         df.at[shift_hour_trunc(timestamp, h=-1), 'predictions_shifted'] = 'DOWN'
47 |             elif '[ORDER]' in line:
48 |                 if 'BUY' in line:
49 |                     df.at[shift_hour_trunc(timestamp, h=-1), 'orders_shifted'] = 'BUY'
50 |                 elif 'SELL' in line:
51 |                     df.at[shift_hour_trunc(timestamp, h=-1), 'orders_shifted'] = 'SELL'
52 |             elif '[BALANCE]' in line:  # balance is not shifted
53 |                 balance_USD = float(line.split(' ')[4])
54 |                 balance_BTC = float(line.split(' ')[6])
55 |             # h=0, h=1 because it's current balance (e.g. 14:00) and last index on chart BTC_sclose will be (13:00)
56 |             # df.index doesnt need trunc, but it needs added h=1
57 |             # we want to omit last balance record, because we don't have BTC_close price for it yet
58 |             if shift_hour_trunc(timestamp, h=0) != shift_hour_trunc(str(df.index[-1]), h=1):
59 |                 df.at[shift_hour_trunc(timestamp, h=0), 'balance_USD'] = balance_USD
60 |                 df.at[shift_hour_trunc(timestamp, h=0), 'balance_BTC'] = balance_BTC
61 | 
62 |     # [None, 1, 2, None, None, 3, None, None, 2, None, 1, 2, None, 1] -> [1, 1, 2, 2, 2, 3, 3, 3, 2, 2, 1, 2, 2, 1]
63 |     # here there are values like 0.002341 or 94,82 instead of 1 2 3, but it shows the idea
64 |     # firstly forward fill
65 |     df['balance_USD'].interpolate(method='ffill', inplace=True)
66 |     # now assume that before starting timestamp balance was the same that last entry
67 |     df['balance_USD'].interpolate(method='bfill', inplace=True)
68 | 
69 |     df['balance_BTC'].interpolate(method='ffill', inplace=True)  # doing the same for BTC
70 |     df['balance_BTC'].interpolate(method='bfill', inplace=True)
71 | 
72 |     def check_prediction(x):
73 |         # target, predictions_shifted -> accurate_prediction
74 |         # UP, UP -> True
75 |         # UP, DOWN -> False
76 |         # UP, NONE -> None
77 |         # np.nan, None -> None
78 |         # None, None -> None
79 |         if type(x['target']) == str and type(x['predictions_shifted']) == str:
80 |             if x['target'] == x['predictions_shifted']:
81 |                 return True
82 |             else:
83 |                 return False
84 |         else:
85 |             return None
86 | 
87 |     df['accurate_prediction'] = df.apply(check_prediction, axis=1)
88 |     df.to_csv(f'{LOG_FILE_ABS[:-4]}.csv', header=True)  # truncating extension .log and saving as .csv
89 |     # logfile.log is still nedeed, it will be displayed in Log textarea
90 | 
91 | 
92 | if __name__ == "__main__":
93 |     process_logfile()
94 | 


--------------------------------------------------------------------------------
/src/utils.py:
--------------------------------------------------------------------------------
 1 | 
 2 | import time
 3 | from datetime import datetime, timedelta
 4 | 
 5 | # dealing with timezones and summer time
 6 | IS_DST = time.daylight and (time.localtime().tm_isdst > 0)  # daylight saving time
 7 | UTC_OFFSET = int((time.altzone if IS_DST else time.timezone) / 3600)  # PC=-2 RPi=-1, Warsaw time in summer
 8 | 
 9 | def unix_to_datetime(x: int, h: int = 0) -> str:  # use with h=UTC_OFFSET
10 |     return (datetime.fromtimestamp(int(x) / 1000) + timedelta(hours=h)).strftime('%Y-%m-%d %H:%M:%S')
11 | 
12 | def datetime_to_unix(x: str, h: int = 0) -> int:  # use with h=-UTC_OFFSET
13 |     return int(datetime.timestamp(datetime.strptime(x, '%Y-%m-%d %H:%M:%S') + timedelta(hours=h))) * 1000
14 | 
15 | 
16 | def shift_hour_exact(timestamp, h):  # '2021-07-31 13:13:23', h=-1 -> '2021-07-31 12:13:23'
17 |     return unix_to_datetime(datetime_to_unix(timestamp), h=h)
18 | 
19 | def shift_hour_trunc(timestamp, h):  # '2021-07-31 13:13:23', h=-1 -> '2021-07-31 12:00:00'
20 |     return unix_to_datetime(datetime_to_unix(timestamp), h=h)[:-6] + ':00:00'
21 | 
22 | def shift_hour_trunc_short(timestamp, h):  # '2021-07-31 13:13:23', h=-1 -> '2021-07-31 12'
23 |     return unix_to_datetime(datetime_to_unix(timestamp), h=h)[:-6]
24 | 
25 | 


--------------------------------------------------------------------------------
/static/favicon.ico:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kamsec/crypto-bot/4d247efe76a71ca7a8e7a638f48f791a06f428b5/static/favicon.ico


--------------------------------------------------------------------------------
/static/main.css:
--------------------------------------------------------------------------------
  1 | 
  2 | canvas{
  3 |     margin: 0 auto;
  4 | }
  5 | 
  6 | :root{
  7 |     --bg_dark: #1c1b17;
  8 |     --bg_light: #38362e;
  9 |     --gray_light: #999999;
 10 |     --gray_dark: #333333;
 11 | 
 12 |     --green_light: #00ff00;
 13 |     --green_dark: #33cc33;
 14 |     --red_light: #ff0000;
 15 |     --red_dark: #990000;
 16 | }
 17 | 
 18 | body{
 19 |     height:100vh;
 20 |     background: var(--bg_dark);
 21 |     color: var(--gray_light);
 22 |     display: flex;
 23 | }
 24 | 
 25 | .main-wrapper{
 26 |     width: 100%;
 27 |     margin-top: auto;
 28 |     margin-bottom: auto;
 29 | }
 30 | 
 31 | .container{
 32 |     width: 98%;
 33 |     max-width: 1780px;
 34 |     border-bottom: 1px solid var(--gray_dark);
 35 | }
 36 | 
 37 | textarea{
 38 |     font-family:"Times New Roman", Times, serif;
 39 |     background-color: var(--bg_dark);
 40 |     white-space: pre;
 41 |     overflow-wrap: normal;
 42 |     border: none;
 43 |     border: solid 1px var(--gray_dark);
 44 |     color: var(--gray_light);
 45 |     font-size: 15px;
 46 |     -webkit-box-sizing: border-box;
 47 |     -moz-box-sizing: border-box;
 48 |     box-sizing: border-box;
 49 |     resize: none;
 50 | }
 51 | 
 52 | .textarea-container {
 53 |     position: relative;
 54 |     height: 90%;
 55 |     min-height:270px;
 56 | }
 57 | 
 58 | .textarea-container textarea {
 59 |     width: 100%;
 60 |     height: 100%;
 61 |     box-sizing: border-box;
 62 | }
 63 | 
 64 | .textarea-container button {
 65 |     background-color: var(--bg_dark);
 66 |     border-radius: 5px;
 67 |     color: var(--gray_light);
 68 |     border: solid 2px var(--green_light);
 69 |     position: absolute;
 70 |     top: 3px;
 71 |     right: 22px;
 72 | }
 73 | 
 74 | .textarea-container button:hover {
 75 |     background-color: var(--bg_light);
 76 | }
 77 | 
 78 | .button-clicked{
 79 |     border: solid 1px var(--gray_light) !important;
 80 |     top: 4px !important;
 81 |     right: 23px !important;
 82 | }
 83 | 
 84 | /* removes browser focus outline */
 85 | :focus {
 86 |     outline:none !important;
 87 | }
 88 | 
 89 | /* dots for chart legend */
 90 | .green-small-dot, .green-big-dot, .red-small-dot, .red-big-dot{
 91 |     margin-right: 3px;
 92 |     border-radius: 50%;
 93 |     display: inline-block;
 94 | }
 95 | 
 96 | .green-small-dot, .red-small-dot{
 97 |     margin-bottom: 2px;
 98 |     height: 5px;
 99 |     width: 5px;
100 | }
101 | 
102 | .green-big-dot, .red-big-dot{
103 |     margin-top: 8px;
104 |     height: 8px;
105 |     width: 8px;
106 | }
107 | 
108 | .red-small-dot, .red-big-dot{
109 |     background-color: var(--red_light);
110 | }
111 | 
112 | .green-small-dot, .green-big-dot{
113 |     background-color: var(--green_light);
114 | }
115 | 
116 | .tile {
117 |     text-align: center;
118 |     display: block;
119 |     border: 1px solid var(--gray_dark);
120 |     border-radius: 5px;
121 |     min-height: 65px;
122 |     max-height: 65px;
123 |     padding: 0px 10px;
124 |     margin-left: 5px;
125 |     margin-right: 5px;
126 |     margin-bottom: 8px;
127 | }
128 | 
129 | /* firefox only but without that it is fine too */
130 | .tile:nth-child(4n) {
131 |     break-after: always; /* New syntax */
132 | }
133 | 
134 | .tile-header{
135 |     border-bottom: 1px solid var(--gray_dark);
136 | }
137 | 
138 | .tile-content{
139 |     line-height: 18px;
140 |     margin-top: 0px;
141 | }
142 | 


--------------------------------------------------------------------------------
/static/script.js:
--------------------------------------------------------------------------------
  1 | 
  2 | // access colors defined in css
  3 | var vars_from_css = getComputedStyle(document.body)
  4 | // .substring(1) removes first space in " #ff0000"
  5 | var gray_light = vars_from_css.getPropertyValue('--gray_light').substring(1);
  6 | var gray_dark = vars_from_css.getPropertyValue('--gray_dark').substring(1);
  7 | var green_light = vars_from_css.getPropertyValue('--green_light').substring(1);
  8 | var green_dark = vars_from_css.getPropertyValue('--green_dark').substring(1);
  9 | var red_light = vars_from_css.getPropertyValue('--red_light').substring(1);
 10 | var red_dark = vars_from_css.getPropertyValue('--red_dark').substring(1);
 11 | 
 12 | // points colors on the chart - UP/DOWN
 13 | var points_colors = [];
 14 | for (var x of predictions) {
 15 |     if (x == 'UP') {
 16 |         points_colors[points_colors.length] = green_light;
 17 |     } else if (x == 'DOWN') {
 18 |         points_colors[points_colors.length] = red_light;
 19 |     } else if (x == null) {
 20 |         points_colors[points_colors.length] = gray_light;
 21 |     }
 22 | }
 23 | 
 24 | // points borders colors on the chart
 25 | var points_borders = [];
 26 | for (var x of orders) {
 27 |     if (x == 'BUY') {
 28 |         points_borders[points_borders.length] = green_light;
 29 |     } else if (x == 'SELL') {
 30 |         points_borders[points_borders.length] = red_light;
 31 |     } else if (x == null) {
 32 |         points_borders[points_borders.length] = "transparent";
 33 |     }
 34 | }
 35 | 
 36 | new Chart("BTCUSD_chart", {
 37 |   type: "line",
 38 |   data: {
 39 |     labels: data_index,
 40 |     datasets: [{
 41 |       label: "Binance BTC/USDT_close_1h",
 42 |       fill: false,
 43 |       lineTension: 0,
 44 |       pointStyle: "circle",
 45 |       pointBackgroundColor: points_colors,
 46 |       pointStrokeColor: points_borders,
 47 |       pointBorderColor: points_borders,
 48 |       pointBorderWidth: 5,
 49 |       pointRadius: 2,
 50 |       borderWidth: 1,
 51 |       backgroundColor: gray_light,
 52 |       borderColor: gray_light,
 53 |       data: data_values
 54 |     }],
 55 |   },
 56 |   options: {
 57 |     responsive: true,
 58 |     legend: {
 59 |       display: false,
 60 |     },
 61 |     scales: {
 62 |       xAxes: [{
 63 |         gridLines: {
 64 |           color: gray_dark
 65 |         },
 66 |         ticks: {
 67 |             fontSize: 12,
 68 |             autoSkip: true,
 69 |             maxRotation: 90,
 70 |             minRotation: 70
 71 |         }
 72 |       }],
 73 |       yAxes: [{
 74 |         gridLines: {
 75 |           color: gray_dark
 76 |         },
 77 |         ticks: {
 78 |             fontSize: 12
 79 |         }
 80 |       }],
 81 |     },
 82 |     /*
 83 |     animation: {
 84 |       duration: 0,
 85 |     }
 86 |     */
 87 |   }
 88 | });
 89 | 
 90 | 
 91 | // texts colors
 92 | function set_tile_text_color(elementId, text_green, text_red)
 93 | {
 94 |     let element = document.getElementById(elementId)
 95 |     let content = element.textContent;
 96 |     if (content == text_green) {
 97 |         document.getElementById(elementId).style.color = green_light;
 98 |     }
 99 |     else if (content == text_red)
100 |     {
101 |         document.getElementById(elementId).style.color = red_light;
102 |     }
103 | }
104 | set_tile_text_color("status", "Active", "Disabled");
105 | set_tile_text_color("orders", "Allowed", "Disallowed");
106 | set_tile_text_color("backups", "Allowed", "Disallowed");
107 | 
108 | function set_color_acc_pct(elementId)
109 | {
110 |     let element = document.getElementById(elementId)
111 |     let content = element.textContent.slice(0, -1);  // removing % from 52.1%
112 |     if (content > 50.0) {
113 |         document.getElementById(elementId).style.color = green_light;
114 |     }
115 |     else if (content < 50.0)
116 |     {
117 |         document.getElementById(elementId).style.color = red_light;
118 |     }
119 | }
120 | set_color_acc_pct("accuracy_pct");
121 | set_color_acc_pct("accuracy_72h_pct");
122 | 
123 | function set_color_acc_text(elementId)
124 | {
125 |     let element = document.getElementById(elementId)
126 |     let first_char = element.textContent.charAt(0);
127 |     if (first_char == "+") {
128 |         document.getElementById(elementId).style.color = green_light;
129 |     }
130 |     else if (first_char == "-")
131 |     {
132 |         document.getElementById(elementId).style.color = red_light;
133 |     }
134 | }
135 | set_color_acc_text("accuracy_text");
136 | set_color_acc_text("accuracy_72h_text");
137 | set_color_acc_text("runtime_price_change_text");
138 | 
139 | 
140 | // making logfile textarea scrolled to bottom by default
141 | var textarea = document.getElementById('log_textarea');
142 | textarea.scrollTop = textarea.scrollHeight;
143 | 
144 | 
145 | // add breaks in textarea with some regex
146 | var lines_broken = true;
147 | var log_data = textarea.textContent;  // imported above
148 | log_data_new = log_data.replace(/FILLED] /g,"FILLED]\n  ");  // /text/g flag makes replace work for all instances
149 | log_data_new = log_data_new.replace(/ Try again/g,"\n  Try again");
150 | function toggle_line_break() {
151 |     if(lines_broken == false) {
152 |         textarea.value = log_data;
153 |         lines_broken = true;
154 |         textarea.scrollTop = textarea.scrollHeight;
155 |         document.getElementById("break_line_button").classList.add("button-clicked");
156 |     }
157 |     else if (lines_broken == true) {
158 |         textarea.value = log_data_new;
159 |         lines_broken = false;
160 |         textarea.scrollTop = textarea.scrollHeight;
161 |         document.getElementById("break_line_button").classList.remove("button-clicked");
162 |     }
163 | }
164 | toggle_line_break()
165 | 


--------------------------------------------------------------------------------
/templates/index.html:
--------------------------------------------------------------------------------
  1 | <!DOCTYPE html>
  2 | <html>
  3 | 
  4 | <head>
  5 |     <meta charset="utf-8">
  6 |     <meta name="viewport" content="width=device-width, initial-scale=1">
  7 |     <meta name="author" content="github.com/kamsec">
  8 | 
  9 |     <!-- Bootstrap CSS -->
 10 |     <!-- <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.0-beta1/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-giJF6kkoqNQ00vy+HMDP7azOuL0xtbfIcaT9wjKHr8RbDVddVHyTfAAsrekwKmP1" crossorigin="anonymous"> -->
 11 |     <link href="{{ url_for('static', filename='bootstrap.min.css') }}" rel="stylesheet" integrity="sha384-giJF6kkoqNQ00vy+HMDP7azOuL0xtbfIcaT9wjKHr8RbDVddVHyTfAAsrekwKmP1" crossorigin="anonymous">
 12 | 
 13 |     <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='main.css') }}">
 14 |     <link rel="shortcut icon" href="{{ url_for('static', filename='favicon.ico') }}">
 15 | 
 16 |     <title>crypto-bot</title>
 17 | </head>
 18 | 
 19 | 
 20 | <body>
 21 |     <div class="main-wrapper">
 22 |         <header class="site-header">
 23 |             <div class="container">
 24 |                 <div class="row">
 25 |                     <div class="col-lg-12">
 26 |                         <h1 class="text-center">crypto-bot</h1>
 27 |                     </div>
 28 |                 </div>
 29 |             </div>
 30 |         </header>
 31 | 
 32 |         <main>
 33 |             <div class="container my-1 py-1">
 34 |                 <div class="row">
 35 | 
 36 |                     <div class="col-md-6 mb-1">
 37 |                         <h5 class="text-center">Binance BTCUSDT_close_1h - 72h chart</h5>
 38 |                         <canvas id="BTCUSD_chart"></canvas>
 39 | 
 40 |                         <div style="display: flex; justify-content: space-around; width: 95%; margin: auto">
 41 |                             <div><span class="green-small-dot"></span>Prediction UP</div>
 42 |                             <div><span class="red-small-dot"></span>Prediction DOWN</div>
 43 |                             <div><span class="green-big-dot"></span>Bought BTC</div>
 44 |                             <div><span class="red-big-dot"></span>Sold BTC</div>
 45 |                         </div>
 46 |                     </div>
 47 | 
 48 | 
 49 |                     <div class="col-md-6 mb-1">
 50 |                         <h5 class="text-center" >Log</h5>
 51 |                         <div class="textarea-container">
 52 |                             <textarea disabled id="log_textarea">{{ logs_data }}</textarea>
 53 |                             <button id="break_line_button" onclick="toggle_line_break()">Break lines</button>
 54 |                         </div>
 55 |                     </div>
 56 | 
 57 |                 </div>
 58 |             </div>
 59 | 
 60 | 
 61 |             <div class="container py-1" style="border: none;">
 62 |                 <div class="row">
 63 | 
 64 |                     <div class ="col-lg-6 mb-2 mt-0" style="display: flex; justify-content: space-between;
 65 |                                                   margin: auto; flex-wrap: wrap;">
 66 |                         <table style="width:100%">
 67 |                             <tr class="tile-header">
 68 |                                 <td>Current balance:</td>
 69 |                                 <td>{{ CURRENT_BALANCE_USD }} USD , {{ CURRENT_BALANCE_BTC }} BTC , <span title="current_USD + current_BTC * current_BTCUSD_close">TOTAL: {{ current_total_value_USD }} USD</span></td>
 70 |                             </tr>
 71 |                             <tr class="tile-header">
 72 |                                 <td><div title="at {{ exact_starting_timestamp }}">Starting balance:</div></td>
 73 |                                 <td>{{ STARTING_BALANCE_USD }} USD , {{ STARTING_BALANCE_BTC }} BTC , <span title="starting_USD + starting_BTC * current_BTCUSD_close">TOTAL: {{ starting_total_value_USD }} USD</span></td>
 74 |                             </tr>
 75 |                             <tr class="tile-header">
 76 |                                 <td>Runtime accuracy:</td>
 77 |                                 <td><span id="accuracy_pct">{{ accuracy_from_start }}%</span> , TOTAL: <span id="accuracy_text">{{ profit_from_start_text }}</span></td>
 78 |                             </tr>
 79 |                             <tr class="tile-header">
 80 |                                 <td>72h accuracy:</td>
 81 |                                 <td><span id="accuracy_72h_pct">{{ accuracy_72h }}%</span> , TOTAL: <span id="accuracy_72h_text">{{ _72h_profit_text }}</span></td>
 82 |                             </tr>
 83 |                             <tr class="tile-header">
 84 |                                 <td>Runtime price change:</td>
 85 |                                 <td><span id="runtime_price_change_text">{{ runtime_price_change_text }}</span></td>
 86 |                             </tr>
 87 |                         </table>
 88 |                     </div>
 89 | 
 90 |                     <div class ="col-lg-6 mt-1 px-3" style="display: flex; justify-content: center; margin: auto; flex-wrap: wrap;">
 91 |                         <div class="tile">
 92 |                             <div class="tile-header">Started</div>
 93 |                             <div class="tile-content">{{ exact_starting_timestamp[:10] }}<br>{{ exact_starting_timestamp[11:] }}</div>
 94 |                         </div>
 95 |                         <div class="tile">
 96 |                             <div class="tile-header">Enviroment</div>
 97 |                             <div class="tile-content">binance.com<br>{{ ENVIROMENT }}</div>
 98 |                         </div>
 99 |                         <div class="tile">
100 |                             <div class="tile-header">Last backup</div>
101 |                             <div class="tile-content">{{ last_commit_time[:10] }}<br>{{ last_commit_time[11:] }}</div>
102 |                         </div>
103 |                         <div class="tile">
104 |                             <div class="tile-header">Prediction model</div>
105 |                             <div class="tile-content mt-2">{{ MODEL_PATH.split("/")[2] }}</div>
106 |                         </div>
107 |                         <div class="tile">
108 |                             <div class="tile-header">Status</div>
109 |                             <div id="status" class="tile-content mt-2">{{ STATUS_ACTIVE_text }}</div>
110 |                         </div>
111 | 
112 |                         <div class="tile">
113 |                             <div class="tile-header">Orders</div>
114 |                             <div id="orders" class="tile-content mt-2">{{ ORDERS_ALLOWED_text }}</div>
115 |                         </div>
116 |                         <div class="tile">
117 |                             <div class="tile-header">GitHub backups</div>
118 |                             <div id="backups" class="tile-content mt-2">{{ BACKUPS_ALLOWED_text }}</div>
119 |                         </div>
120 |                     </div>
121 | 
122 |                 </div>
123 |             </div>
124 | 
125 |             <div class="text-center">All timestamps are given in UTC+0</div>
126 |         </main>
127 |     </div>
128 | 
129 |     <!-- Chart.js -->
130 |     <!-- <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.9.3/Chart.bundle.min.js" integrity="sha512-vBmx0N/uQOXznm/Nbkp7h0P1RfLSj0HQrFSzV8m7rOGyj30fYAOKHYvCNez+yM8IrfnW0TCodDEjRqf6fodf/Q==" crossorigin="anonymous" referrerpolicy="no-referrer"></script> -->
131 |     <script src="{{ url_for('static', filename='Chart.bundle.min.js') }}"></script>
132 | 
133 |     <script> // these wont work in external .js file, the easiest way is to do it here
134 |     {% autoescape false %} // removes quotes denoted as &#39, e.g. [&#39;BUY&#39;, &#39;SELL&#39;]
135 |         var data_index = {{ data_index }};
136 |         var data_values = {{ data_values }};
137 |         var predictions = {{ predictions }};
138 |         var orders = {{ orders }};
139 |     {% endautoescape %}
140 |     </script>
141 | 
142 |     <script src="{{ url_for('static', filename='script.js') }}"></script>
143 | 
144 | </body>
145 | </html>
146 | 


--------------------------------------------------------------------------------
/tests.py:
--------------------------------------------------------------------------------
 1 | 
 2 | # this file is for testing functions manually and executing tools from tools/ that are not used by the bot
 3 | # it is not used by the bot.py or app.py
 4 | 
 5 | def test_simulation():
 6 |     from tools.simulate import simulation
 7 |     simulation()
 8 | # test_simulation()
 9 | 
10 | 
11 | def test_prediction():
12 |     # watch out because executing it will print output to logfile.log and might have unexpected behaviour for app.py
13 |     from src.predict import predict_next
14 |     predict_next()
15 | # test_prediction()
16 | 
17 | 
18 | def my_test_orders():
19 |     from binance.client import Client
20 |     import configparser
21 | 
22 |     cfg_secrets = configparser.ConfigParser()
23 |     cfg_secrets.read('secrets.ini')
24 |     TESTNET_API_KEY = cfg_secrets['DEFAULT']['TESTNET_API_KEY']
25 |     TESTNET_API_SECRET = cfg_secrets['DEFAULT']['TESTNET_API_SECRET']
26 | 
27 |     client = Client(TESTNET_API_KEY, TESTNET_API_SECRET, testnet=True)
28 | 
29 |     BTC = client.get_asset_balance(asset='BTC')['free']
30 |     USDT = client.get_asset_balance(asset='USDT')['free']
31 |     ETH = client.get_asset_balance(asset='ETH')['free']
32 |     print(f'BTC: {BTC}, USDT: {USDT}, ETH: {ETH}')
33 | 
34 |     # https://github.com/binance/binance-spot-api-docs/blob/master/rest-api.md#lot_size
35 |     # quantity= or quoteOrderQty=
36 |     # order = client.create_order(side='SELL', quantity=round(49.963952), symbol='BTCUSDT', type='MARKET')
37 |     # order = client.create_order(side='BUY', quoteOrderQty=round(50, 6), symbol='ETHUSDT', type='MARKET')
38 |     # print(order)
39 | 
40 |     BTC = client.get_asset_balance(asset='BTC')['free']
41 |     USDT = client.get_asset_balance(asset='USDT')['free']
42 |     ETH = client.get_asset_balance(asset='ETH')['free']
43 |     print(f'BTC: {BTC}, USDT: {USDT}, ETH: {ETH}')
44 |     # exit()
45 | 
46 |     # request:
47 |     # BTC = client.get_asset_balance(asset='BTC')['free']
48 |     # resulted in:
49 |     # binance.exceptions.BinanceAPIException: APIError(code=-1021): Timestamp for this request was 1000ms ahead of the server's time.
50 |     # windows fix:
51 |     # from control panel > date and time > internet time
52 |     # change the server to >>>> time.nist.gov
53 | # my_test_orders()
54 | 
55 | 


--------------------------------------------------------------------------------
/tools/data_add_new_pair.py:
--------------------------------------------------------------------------------
 1 | # this file is not used by the bot or app
 2 | 
 3 | import pandas as pd
 4 | import numpy as np
 5 | import time
 6 | from datetime import datetime
 7 | from requests import get
 8 | from json import loads
 9 | 
10 | from src.utils import unix_to_datetime, datetime_to_unix, UTC_OFFSET
11 | from src.data_update import update_ratio
12 | from src.config import DATA_PATH_ABS
13 | 
14 | def download_data(starting_time, ratio, filename):
15 |     try:
16 |         open(filename, "r")
17 |         raise Exception('File already exists, you can only update it.')
18 |     except FileNotFoundError:
19 |         pass
20 |     last_record_unix = datetime_to_unix(str(starting_time), h=-UTC_OFFSET)  # -2 because of timezones
21 | 
22 |     response = get('https://api.binance.com/api/v1/klines',
23 |                 params={'symbol': ratio,
24 |                         'interval': '1h',
25 |                         'startTime': last_record_unix,
26 |                         'limit': 1000})
27 |     raw_data = loads(response.content)
28 |     raw_data_df = pd.DataFrame(raw_data)
29 | 
30 |     new_df = pd.DataFrame(columns=['date', f'{ratio}_close', f'{ratio}_volume'])
31 |     # we receive data in UTC+2 timezone, so need to set h=-2, so we pass list of [-2] as second argument
32 |     new_df['date'] = list(map(unix_to_datetime, raw_data_df[0], [UTC_OFFSET] * len(raw_data_df[0])))
33 |     new_df[f'{ratio}_close'] = list(map(float, raw_data_df[4]))  # df[4]  # Close
34 |     new_df[f'{ratio}_volume'] = list(map(float, raw_data_df[7]))  # df[7]  # Quote asset volume (second assets unit, so in BTC/USDT its volume_USDT)  # noqa
35 |     new_df.set_index("date", inplace=True)
36 |     # if we send request at 15:27 (UTC-0 13:27) it will return data with last entry 13:00 which will change over time
37 |     new_df = new_df[:-1]  # dropping that entry
38 | 
39 |     new_df.index = pd.to_datetime(new_df.index)
40 |     expected_range = pd.date_range(start=new_df.index.min(), end=new_df.index.max(), freq='H')
41 |     missing_ts = list(expected_range.difference(new_df.index))
42 |     if missing_ts:
43 |         # inp = input(f'Missing timestamps: {missing_ts}.\n'
44 |         #             f'{len(expected_range) - len(new_df.index)} missing timestamps. y - interpolate, n - exit\n')
45 |         new_df = new_df.resample('60min').mean()
46 |         new_df.replace(to_replace=0, value=np.nan, inplace=True)  # converting 0s to NaNs
47 |         new_df.interpolate(method='linear', inplace=True)  # interpolating missing nan values
48 |         print(f'Missing timestamps: {missing_ts}.\n'
49 |               f'{len(missing_ts)} missing timestamps. auto interpolate')
50 | 
51 |     return new_df
52 | 
53 | 
54 | def data_add_new_pair():
55 |     ratio = ['XRPUSDT']  # ["BCHUSDT"]
56 |     starting_time = datetime.strptime('2017-01-01 00:00:00', '%Y-%m-%d %H:%M:%S')
57 |     # it only downloads first 500 records, to get all data use update_data() on new pair .csv file
58 |     filename = f"{DATA_PATH_ABS}Binance_{ratio}_1h.csv"
59 |     df_first = download_data(starting_time, ratio, filename)
60 |     df_first.to_csv(filename)
61 |     while True:  # added data_update from src.data_update.py
62 |         filename = f"{DATA_PATH_ABS}Binance_{ratio}_1h.csv"
63 |         time.sleep(0.3)
64 |         new_df = update_ratio(ratio, filename, request_records=1000, fill_missing=True)
65 |         new_df.to_csv(filename, mode='a', header=False)
66 | 
67 | 
68 | if __name__ == "__main__":
69 |     data_add_new_pair()
70 | 


--------------------------------------------------------------------------------
/tools/simulate.py:
--------------------------------------------------------------------------------
  1 | # this file is not used by the bot or app
  2 | 
  3 | import numpy as np
  4 | import pandas as pd
  5 | from sklearn import preprocessing
  6 | from collections import deque
  7 | from src.config import RATIOS, RATIO_TO_PREDICT, SEQ_LEN, MODEL_PATH_ABS, DATA_PATH_ABS, logger
  8 | import tflite_runtime.interpreter as tflite
  9 | # windows: C:\Users\Admin\Desktop\python37\Scripts>pip3 install --index-url https://google-coral.gith
 10 | # rpi: tflite_runtime-2.3.1-cp37-cp37m-linux_armv6l.whl
 11 | 
 12 | pd.set_option('display.max_columns', None)
 13 | pd.set_option('display.expand_frame_repr', False)
 14 | 
 15 | # simulation removes the latest record from data,
 16 | # because in this case its not possible to verify real price and "target"
 17 | 
 18 | NUM_OF_PREDS = 1000  # tu inaczej niz w single prediction
 19 | FUTURE_PERIOD_PREDICT = 1
 20 | 
 21 | def construct_df_for_multiple_pred(ratios):
 22 |     main_df = pd.DataFrame()
 23 |     # 'unix,date,symbol,open,high,low,close,Volume LTC,Volume USDT,tradecount'
 24 |     for ratio in ratios:
 25 |         # drive_path = 'drive/MyDrive/Colab Notebooks/crypto_prediction/model3/data/'
 26 |         dataset = f"{DATA_PATH_ABS}Binance_{ratio}_1h.csv"
 27 |         df = pd.read_csv(dataset, nrows=None, skiprows=None, parse_dates=['date'], usecols=[0, 1, 2])
 28 |         df.set_index("date", inplace=True)
 29 | 
 30 |         # later we use pct_change so we lose 1 row and need 1 more (-49 for single pred)
 31 |         # "+ 1" is because the newest row is removed cause target would be NAN, so
 32 |         # "+ 1" has to be here for training, and cant be here for prediction
 33 |         # but "+ 1" has to be here for simulation of multiple predictions
 34 |         short_df = df[-(SEQ_LEN + 1) - NUM_OF_PREDS:]
 35 | 
 36 |         if len(main_df) == 0:
 37 |             main_df = short_df
 38 |         else:
 39 |             main_df = main_df.join(short_df)
 40 | 
 41 |     # in this case it reduces main_df length to the length of pair with the smallest data
 42 |     main_df.dropna(how='any', inplace=True)
 43 | 
 44 |     def classify(current, future):
 45 |         if float(future) > float(current):
 46 |             return 1
 47 |         else:
 48 |             return 0
 49 |     # we determine targets by making ['future'] column with shifted rows,
 50 |     # and then if its price is higher than current day setting ['target'] to 1, and 0 otherwise
 51 |     main_df['future'] = main_df[f"{RATIO_TO_PREDICT}_close"].shift(-FUTURE_PERIOD_PREDICT)
 52 |     main_df.dropna(how='any', inplace=True)  # todo check if it shouldnt be replaced everywhere
 53 |     main_df['target'] = list(map(classify, main_df[f"{RATIO_TO_PREDICT}_close"], main_df["future"]))
 54 |     main_df.drop('future', 1, inplace=True)  # future column is no longer needed, used only to determine target column
 55 |     # rows with future = NaN also are dropped here
 56 | 
 57 |     return main_df
 58 | 
 59 | 
 60 | # percentage change and scaling
 61 | def preprocess_data_for_pred(df):
 62 |     df = df.copy()  # fixes random SettingWithCopyWarning appearing with pct_change and preprocessing.scale
 63 |     for col in df.columns:
 64 |         # todo here we lose 1 row
 65 |         df[col] = df[col].pct_change()  # converted to percentage change
 66 |         # example:
 67 |         # hour 1:00 close = 93.13, hour 2:00 close = 92.41, so
 68 |         # 93.13 + 93.13 * x = 92.41, x = -0.007731
 69 |         # so percentage change of 1.0 means doubling the price
 70 |         # first row (the oldest values) is being filled with NaN percentage, but we don't want to drop whole row yet
 71 |         # so we wait until loop ends (we will have whole row filled with nans)
 72 |     df.dropna(inplace=True)
 73 |     # print('b', df.shape, df.isnull().values.any())
 74 |     for col in df.columns:
 75 |         if col != "target":
 76 |             # scaling should be used after splitting to training and test set - leaks data from test set to training
 77 |             df[col] = preprocessing.scale(df[col].values)  # normalizes to [-1, +1]
 78 |             # df[col] = preprocessing.StandardScaler().fit_transform(df[col].values.reshape(-1, 1)) # it does the same
 79 |             # scales the data so it has mean = 0, standard deviation = 1, variance = 1
 80 |             # it can be checked before and after with e.g. print(df['BTCUSDT_close'].mean())  .std()  .var()
 81 | 
 82 |     prev_days = deque(maxlen=SEQ_LEN)  # list of SEQ_LEN items, we append to it, as it exceed, it pops old values
 83 |     sequential_data = []
 84 |     for i in df.values:
 85 |         prev_days.append([n for n in i])  # set of price and volume features added (-1 so without target)
 86 |         if len(prev_days) == SEQ_LEN:
 87 |             sequential_data.append(np.array([prev_days]))
 88 |     # print(len(sequential_data))
 89 |     # ok so the sequences are like this ([a, b, 1] is single prev_days) (* and ^ just to point where it goes):
 90 |     # [([[a, b] [c, d], [e, f]], 1*)   ([[c, d] [e, f] [g, h]], 1) ... ([[g, h], [i, j], [k, l]], 0^)]
 91 | 
 92 |     return sequential_data
 93 | 
 94 | 
 95 | balance_USD = float(100)
 96 | balance_BTC = float(0)
 97 | order_counter = 0
 98 | fee = 0.001
 99 | def simulation():
100 |     main_df = construct_df_for_multiple_pred(RATIOS)
101 |     real_y = main_df['target'][SEQ_LEN:]
102 |     main_df.drop('target', 1, inplace=True)
103 |     print(main_df.shape)
104 | 
105 |     predict_x = preprocess_data_for_pred(main_df)
106 |     print(predict_x[0].shape)
107 | 
108 |     def classifier(prob):
109 |         return 1 if prob > 0.5 else 0
110 | 
111 |     predictions = []
112 |     interpreter = tflite.Interpreter(model_path=MODEL_PATH_ABS)
113 |     interpreter.allocate_tensors()
114 |     input_details = interpreter.get_input_details()
115 |     output_details = interpreter.get_output_details()
116 |     for x, y in zip(predict_x, real_y.values):
117 |         data_for_tflite = np.array(x, dtype=np.float32)
118 |         interpreter.set_tensor(input_details[0]['index'], data_for_tflite)
119 |         interpreter.invoke()
120 |         predict_y = interpreter.get_tensor(output_details[0]['index'])
121 |         predictions.append(classifier(predict_y[0][1]))
122 | 
123 |     predictions = np.array(predictions)
124 |     real_y = np.array(real_y.values)
125 |     print(f'pred 0s: {(predictions == 0).sum()}, 1s: {(predictions == 1).sum()}')
126 |     print(f'real 0s: {(real_y == 0).sum()}, 1s: {(real_y == 1).sum()}')
127 |     print(f'accuracy: {np.mean(predictions == real_y)}')
128 | 
129 |     # zakres [Number
130 |     real_BTC_prices = main_df['BTCUSDT_close'][SEQ_LEN:].values.tolist()
131 |     predictions = predictions.tolist()
132 |     global balance_USD
133 |     global balance_BTC
134 |     global order_counter
135 | 
136 |     def buy_BTC(money, USD_BTC_price):
137 |         global balance_USD
138 |         global order_counter
139 |         balance_USD = 0
140 |         order_counter += 1
141 |         print(f'ORDER: {order_counter} | HOUR {i}: Bought {money / USD_BTC_price}BTC for {money}USD at the price of {USD_BTC_price}USD/BTC')
142 |         return money / USD_BTC_price - fee * (money / USD_BTC_price)
143 | 
144 |     def sell_BTC(money, USD_BTC_price):
145 |         global balance_BTC
146 |         global order_counter
147 |         balance_BTC = 0
148 |         order_counter += 1
149 |         print(f'ORDER: {order_counter} | HOUR {i}: Sold {money}BTC for {money * USD_BTC_price}USD at the price of {USD_BTC_price}USD/BTC')
150 |         return money * USD_BTC_price - fee * (money * USD_BTC_price)
151 | 
152 |     last_buy_price = 0
153 |     print(f'Start balance_BTC: {balance_BTC}')
154 |     print(f'Start balance_USD: {balance_USD}')
155 |     for i in range(1, len(real_BTC_prices)):  # todo removed -2 here, check again if everything is correct
156 |         if (balance_USD > 0) and (predictions[i] == 1):
157 |             balance_BTC = buy_BTC(balance_USD, real_BTC_prices[i])
158 |             continue
159 |         # if (balance_BTC > 0) and (real_BTC_prices[i] > real_BTC_prices[i - 1]) and predictions[i] == 0:
160 |         if (balance_BTC > 0) and (predictions[i] == 0):
161 |             balance_USD = sell_BTC(balance_BTC, real_BTC_prices[i])
162 |         # elif (balance_BTC > 0) and (real_BTC_prices[i] > real_BTC_prices[i - 1]) and predictions[i] == 1:
163 |         elif (balance_BTC > 0) and predictions[i] == 1:
164 |             continue
165 | 
166 |     if balance_BTC > 0:
167 |         balance_USD = sell_BTC(balance_BTC, real_BTC_prices[-1])
168 |     print(f'Final balance_BTC: {balance_BTC}')
169 |     print(f'Final balance_USD: {balance_USD}')
170 | 
171 | 
172 | if __name__ == "__main__":
173 |     simulation()


--------------------------------------------------------------------------------
/training-google-colab/convert_model.py:
--------------------------------------------------------------------------------
 1 | # this file is not used by the bot or app
 2 | # it shows how to convert tensorflow model to TFlite to be suitable for Raspberry Pi
 3 | # it can be done on Google Colab and downloaded
 4 | 
 5 | import tensorflow as tf
 6 | from config import MODEL_PATH
 7 | 
 8 | # Convert the model
 9 | converter = tf.lite.TFLiteConverter.from_saved_model(MODEL_PATH)  # path to the SavedModel directory
10 | tflite_model = converter.convert()
11 | 
12 | # Save it
13 | with open('model2.tflite', 'wb') as f:
14 |     print("Started writing")
15 |     f.write(tflite_model)
16 |     print("Finished")
17 | 


--------------------------------------------------------------------------------
/training-google-colab/model3.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "nbformat": 4,
  3 |   "nbformat_minor": 0,
  4 |   "metadata": {
  5 |     "colab": {
  6 |       "name": "model3.ipynb",
  7 |       "provenance": [],
  8 |       "collapsed_sections": []
  9 |     },
 10 |     "kernelspec": {
 11 |       "name": "python3",
 12 |       "display_name": "Python 3"
 13 |     },
 14 |     "language_info": {
 15 |       "name": "python"
 16 |     },
 17 |     "accelerator": "GPU"
 18 |   },
 19 |   "cells": [
 20 |     {
 21 |       "cell_type": "code",
 22 |       "metadata": {
 23 |         "id": "sfwZbBSN6dnu"
 24 |       },
 25 |       "source": [
 26 |         "from sklearn import preprocessing\n",
 27 |         "from collections import deque\n",
 28 |         "from datetime import datetime\n",
 29 |         "import random\n",
 30 |         "import time\n",
 31 |         "import numpy as np\n",
 32 |         "import pandas as pd\n",
 33 |         "import tensorflow as tf\n",
 34 |         "from tensorflow import keras\n",
 35 |         "from tensorflow.keras.models import Sequential\n",
 36 |         "from tensorflow.keras.layers import Dense, Dropout, LSTM, BatchNormalization\n",
 37 |         "from tensorflow.python.keras.layers import CuDNNLSTM\n",
 38 |         "from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping\n",
 39 |         "\n",
 40 |         "drive_path = f'drive/MyDrive/Colab Notebooks/crypto_prediction/model3/'"
 41 |       ],
 42 |       "execution_count": 1,
 43 |       "outputs": []
 44 |     },
 45 |     {
 46 |       "cell_type": "code",
 47 |       "metadata": {
 48 |         "id": "SYa7ACGv6xuU"
 49 |       },
 50 |       "source": [
 51 |         "pd.set_option('display.max_columns', None)\n",
 52 |         "pd.set_option('display.expand_frame_repr', False)\n",
 53 |         "\n",
 54 |         "# ratios = [\"BTCUSDT\", \"ETHUSDT\", \"NANOUSDT\"]\n",
 55 |         "ratios = [\"BTCUSDT\", \"LTCUSDT\", \"ETHUSDT\", \"BNBUSDT\",\n",
 56 |         "          \"ADAUSDT\", \"BTTUSDT\", \"DASHUSDT\", \"XMRUSDT\",\n",
 57 |         "          \"NANOUSDT\", \"DOGEUSDT\", \"XLMUSDT\", \"BCHUSDT\"]\n",
 58 |         "RATIO_TO_PREDICT = \"BTCUSDT\"\n",
 59 |         "\n",
 60 |         "SEQ_LEN = 48\n",
 61 |         "FUTURE_PERIOD_PREDICT = 1\n",
 62 |         "EPOCHS = 40\n",
 63 |         "BATCH_SIZE = 64\n",
 64 |         "current_time = datetime.utcnow().strftime('%Y-%m-%d %H-%M-%S')\n",
 65 |         "NAME = f\"LSTM-{current_time}\"\n",
 66 |         "\n",
 67 |         "def construct_main_df(ratios):\n",
 68 |         "    main_df = pd.DataFrame()\n",
 69 |         "    # 'unix,date,symbol,open,high,low,close,Volume LTC,Volume USDT,tradecount'\n",
 70 |         "    for ratio in ratios:\n",
 71 |         "        dataset = f\"{drive_path}data/Binance_{ratio}_1h.csv\"\n",
 72 |         "        df = pd.read_csv(dataset, nrows=None, skiprows=None, parse_dates=['date'], usecols=[0, 1, 2])  # todo\n",
 73 |         "        df.set_index(\"date\", inplace=True)\n",
 74 |         "\n",
 75 |         "        if len(main_df) == 0:\n",
 76 |         "            main_df = df\n",
 77 |         "        else:\n",
 78 |         "            main_df = main_df.join(df)\n",
 79 |         "\n",
 80 |         "    # in this case it reduces main_df length to the length of pair with the smallest data\n",
 81 |         "    main_df.dropna(how='any', inplace=True)\n",
 82 |         "\n",
 83 |         "    # we determine targets by making ['future'] column with shifted rows,\n",
 84 |         "    # and then if its price is higher than current day setting ['target'] to 1, and 0 otherwise\n",
 85 |         "    main_df['future'] = main_df[f\"{RATIO_TO_PREDICT}_close\"].shift(-FUTURE_PERIOD_PREDICT)\n",
 86 |         "    main_df['target'] = list(map(classify, main_df[f\"{RATIO_TO_PREDICT}_close\"], main_df[\"future\"]))\n",
 87 |         "    main_df.drop('future', 1, inplace=True)  # future column is no longer needed, used only to determine target column\n",
 88 |         "    # rows with future = NaN also are dropped here\n",
 89 |         "    return main_df\n",
 90 |         "\n",
 91 |         "def classify(current, future):\n",
 92 |         "    return 1 if float(future) > float(current) else 0\n",
 93 |         "\n",
 94 |         "def split_main_df(main_df, train_split, validation_split, test_split):\n",
 95 |         "    # [train train train train_index val val val val_index test test test_index]\n",
 96 |         "    train_index = int(train_split * len(main_df))  # last indexes\n",
 97 |         "    validation_index = int(train_index + validation_split * len(main_df))\n",
 98 |         "    test_index = test_split  # we don't really need to use it, its always the last index\n",
 99 |         "\n",
100 |         "    train_df = main_df.iloc[:train_index]\n",
101 |         "    validation_df = main_df.iloc[train_index:validation_index]\n",
102 |         "    test_df = main_df.iloc[validation_index:]\n",
103 |         "    return train_df, validation_df, test_df\n",
104 |         "\n",
105 |         "# percentage change and scaling\n",
106 |         "def preprocess_data(df):\n",
107 |         "    df = df.copy()  # fixes random SettingWithCopyWarning appearing with pct_change and preprocessing.scale\n",
108 |         "    for col in df.columns:\n",
109 |         "        if col != \"target\":\n",
110 |         "            df[col] = df[col].pct_change()  # converted to percentage change\n",
111 |         "            # example:\n",
112 |         "            # hour 1:00 close = 93.13, hour 2:00 close = 92.41, so\n",
113 |         "            # 93.13 + 93.13 * x = 92.41, x = -0.007731\n",
114 |         "            # so percentage change of 1.0 means doubling the price\n",
115 |         "            # first row (the oldest values) is being filled with NaN percentage, but we don't want to drop whole row yet\n",
116 |         "            # so we wait until loop ends (we will have whole row filled with nans)\n",
117 |         "    df.dropna(inplace=True)\n",
118 |         "    # print('b', df.shape, df.isnull().values.any())\n",
119 |         "    for col in df.columns:\n",
120 |         "        if col != \"target\":\n",
121 |         "            # scaling should be used after splitting to training and test set - leaks data from test set to training\n",
122 |         "            df[col] = preprocessing.scale(df[col].values)  # normalizes to [-1, +1]\n",
123 |         "            # df[col] = preprocessing.StandardScaler().fit_transform(df[col].values.reshape(-1, 1)) # it does the same\n",
124 |         "            # scales the data so it has mean = 0, standard deviation = 1, variance = 1\n",
125 |         "            # it can be checked before and after with e.g. print(df['BTCUSDT_close'].mean())  .std()  .var()\n",
126 |         "\n",
127 |         "    sequential_data = []\n",
128 |         "    prev_days = deque(maxlen=SEQ_LEN)  # list of SEQ_LEN items, we append to it, as it exceed, it pops old values\n",
129 |         "\n",
130 |         "    for i in df.values:\n",
131 |         "        prev_days.append([n for n in i[:-1]])  # set of price and volume features added (-1 so without target)\n",
132 |         "        if len(prev_days) == SEQ_LEN:\n",
133 |         "            sequential_data.append([np.array(prev_days), i[-1]])\n",
134 |         "    # print(len(sequential_data))\n",
135 |         "    # ok so the sequences are like this ([a, b, 1] is single prev_days) (* and ^ just to point where it goes):\n",
136 |         "    # [([[a, b] [c, d], [e, f]], 1*)   ([[c, d] [e, f] [g, h]], 1) ... ([[g, h], [i, j], [k, l]], 0^)]\n",
137 |         "    random.shuffle(sequential_data)\n",
138 |         "\n",
139 |         "\n",
140 |         "    buys = []  # list that will store our buy sequences and targets\n",
141 |         "    sells = []  # list that will store our sell sequences and targets\n",
142 |         "\n",
143 |         "    for seq, target in sequential_data:  # iterate over the sequential data\n",
144 |         "        if target == 0:  # if it's a \"not buy\"\n",
145 |         "            sells.append([seq, target])  # append to sells list\n",
146 |         "        elif target == 1:  # otherwise if the target is a 1...\n",
147 |         "            buys.append([seq, target])  # it's a buy!\n",
148 |         "\n",
149 |         "    random.shuffle(buys)  # shuffle the buys\n",
150 |         "    random.shuffle(sells)  # shuffle the sells\n",
151 |         "\n",
152 |         "    lower = min(len(buys), len(sells))  # what's the shorter length?\n",
153 |         "\n",
154 |         "    buys = buys[:lower]  # make sure both lists are only up to the shortest length.\n",
155 |         "    sells = sells[:lower]  # make sure both lists are only up to the shortest length.\n",
156 |         "\n",
157 |         "    sequential_data = buys + sells  # concatenate lists\n",
158 |         "    random.shuffle(\n",
159 |         "        sequential_data)  # another shuffle, so the model doesn't get confused with all 1 class then the other.\n",
160 |         "\n",
161 |         "    x = []\n",
162 |         "    y = []\n",
163 |         "\n",
164 |         "    for seq, target in sequential_data:\n",
165 |         "        x.append(seq)\n",
166 |         "        y.append(target)\n",
167 |         "    return np.array(x), np.array(y)"
168 |       ],
169 |       "execution_count": 2,
170 |       "outputs": []
171 |     },
172 |     {
173 |       "cell_type": "code",
174 |       "metadata": {
175 |         "id": "FY9cXsAF64Uq",
176 |         "colab": {
177 |           "base_uri": "https://localhost:8080/"
178 |         },
179 |         "outputId": "d7263d26-d264-4fec-92a6-d99f22a2bc5b"
180 |       },
181 |       "source": [
182 |         "time0 = time.perf_counter()\n",
183 |         "main_df = construct_main_df(ratios)\n",
184 |         "print(main_df.shape)\n",
185 |         "time1 = time.perf_counter()\n",
186 |         "print(time1 - time0)\n",
187 |         "\n",
188 |         "train_df, validation_df, test_df = split_main_df(main_df, train_split=0.6, validation_split=0.2, test_split=0.2)\n",
189 |         "print(train_df.shape, validation_df.shape, test_df.shape)\n",
190 |         "time2 = time.perf_counter()\n",
191 |         "print(time2 - time1)\n",
192 |         "\n",
193 |         "train_x, train_y = preprocess_data(train_df)\n",
194 |         "validation_x, validation_y = preprocess_data(validation_df)\n",
195 |         "test_x, test_y = preprocess_data(test_df)\n",
196 |         "time3 = time.perf_counter()\n",
197 |         "print(train_x.shape, train_y.shape, validation_x.shape, validation_y.shape, test_x.shape, test_y.shape,)\n",
198 |         "print(time3 - time2)"
199 |       ],
200 |       "execution_count": 3,
201 |       "outputs": [
202 |         {
203 |           "output_type": "stream",
204 |           "text": [
205 |             "(15737, 25)\n",
206 |             "0.4711706650000451\n",
207 |             "(9442, 25) (3147, 25) (3148, 25)\n",
208 |             "0.0005700589999833028\n",
209 |             "(9134, 48, 24) (9134,) (2940, 48, 24) (2940,) (3066, 48, 24) (3066,)\n",
210 |             "2.1966656359999774\n"
211 |           ],
212 |           "name": "stdout"
213 |         }
214 |       ]
215 |     },
216 |     {
217 |       "cell_type": "code",
218 |       "metadata": {
219 |         "id": "JJKJcltc7kDx",
220 |         "colab": {
221 |           "base_uri": "https://localhost:8080/"
222 |         },
223 |         "outputId": "382dadb1-3de1-44fc-c440-aba94c06f01f"
224 |       },
225 |       "source": [
226 |         "def build_model():\n",
227 |         "    model = Sequential()\n",
228 |         "    model.add(LSTM(128, input_shape=(train_x.shape[1:]), return_sequences=True))\n",
229 |         "    # i cannot save CuDNNLSTM due to some error, not using it\n",
230 |         "    # model.add(CuDNNLSTM(128, input_shape=(train_x.shape[1:]), return_sequences=True))\n",
231 |         "    model.add(Dropout(0.2))\n",
232 |         "    model.add(BatchNormalization())\n",
233 |         "\n",
234 |         "    model.add(LSTM(128, input_shape=(train_x.shape[1:]), return_sequences=True))\n",
235 |         "    model.add(Dropout(0.1))\n",
236 |         "    model.add(BatchNormalization())\n",
237 |         "\n",
238 |         "    model.add(LSTM(128, input_shape=(train_x.shape[1:])))\n",
239 |         "    model.add(Dropout(0.2))\n",
240 |         "    model.add(BatchNormalization())\n",
241 |         "\n",
242 |         "    model.add(Dense(32, activation=\"relu\"))\n",
243 |         "    model.add(Dropout(0.2))\n",
244 |         "\n",
245 |         "    model.add(Dense(2, activation=\"softmax\"))\n",
246 |         "\n",
247 |         "    opt = tf.keras.optimizers.Adam(learning_rate=0.001, decay=1e-6)\n",
248 |         "    # opt = tf.keras.optimizers.Adam()\n",
249 |         "\n",
250 |         "    model.compile(loss = \"sparse_categorical_crossentropy\", \n",
251 |         "                  optimizer=opt,\n",
252 |         "                  metrics=['accuracy'])\n",
253 |         "\n",
254 |         "    print(model.summary())\n",
255 |         "    return model\n",
256 |         "model = build_model()"
257 |       ],
258 |       "execution_count": 4,
259 |       "outputs": [
260 |         {
261 |           "output_type": "stream",
262 |           "text": [
263 |             "Model: \"sequential\"\n",
264 |             "_________________________________________________________________\n",
265 |             "Layer (type)                 Output Shape              Param #   \n",
266 |             "=================================================================\n",
267 |             "lstm (LSTM)                  (None, 48, 128)           78336     \n",
268 |             "_________________________________________________________________\n",
269 |             "dropout (Dropout)            (None, 48, 128)           0         \n",
270 |             "_________________________________________________________________\n",
271 |             "batch_normalization (BatchNo (None, 48, 128)           512       \n",
272 |             "_________________________________________________________________\n",
273 |             "lstm_1 (LSTM)                (None, 48, 128)           131584    \n",
274 |             "_________________________________________________________________\n",
275 |             "dropout_1 (Dropout)          (None, 48, 128)           0         \n",
276 |             "_________________________________________________________________\n",
277 |             "batch_normalization_1 (Batch (None, 48, 128)           512       \n",
278 |             "_________________________________________________________________\n",
279 |             "lstm_2 (LSTM)                (None, 128)               131584    \n",
280 |             "_________________________________________________________________\n",
281 |             "dropout_2 (Dropout)          (None, 128)               0         \n",
282 |             "_________________________________________________________________\n",
283 |             "batch_normalization_2 (Batch (None, 128)               512       \n",
284 |             "_________________________________________________________________\n",
285 |             "dense (Dense)                (None, 32)                4128      \n",
286 |             "_________________________________________________________________\n",
287 |             "dropout_3 (Dropout)          (None, 32)                0         \n",
288 |             "_________________________________________________________________\n",
289 |             "dense_1 (Dense)              (None, 2)                 66        \n",
290 |             "=================================================================\n",
291 |             "Total params: 347,234\n",
292 |             "Trainable params: 346,466\n",
293 |             "Non-trainable params: 768\n",
294 |             "_________________________________________________________________\n",
295 |             "None\n"
296 |           ],
297 |           "name": "stdout"
298 |         }
299 |       ]
300 |     },
301 |     {
302 |       "cell_type": "code",
303 |       "metadata": {
304 |         "colab": {
305 |           "base_uri": "https://localhost:8080/"
306 |         },
307 |         "id": "HwV1fg6uttcy",
308 |         "outputId": "007144f2-b485-4dc2-d998-666f4781a6a9"
309 |       },
310 |       "source": [
311 |         "checkpoint = ModelCheckpoint(filepath=f\"models/{NAME}\", monitor='val_accuracy', verbose=1, save_best_only=True, mode='max') # saves only the best ones\n",
312 |         "earlystopping = EarlyStopping(monitor='val_loss', patience=4)\n",
313 |         "cb = [checkpoint, earlystopping]\n",
314 |         "\n",
315 |         "history = model.fit(\n",
316 |         "        train_x, train_y,\n",
317 |         "        batch_size=BATCH_SIZE,\n",
318 |         "        epochs=EPOCHS,\n",
319 |         "        validation_data=(validation_x, validation_y),\n",
320 |         "        callbacks=cb)\n"
321 |       ],
322 |       "execution_count": 5,
323 |       "outputs": [
324 |         {
325 |           "output_type": "stream",
326 |           "text": [
327 |             "Epoch 1/40\n",
328 |             "143/143 [==============================] - 13s 23ms/step - loss: 0.7907 - accuracy: 0.5310 - val_loss: 0.6946 - val_accuracy: 0.4980\n",
329 |             "\n",
330 |             "Epoch 00001: val_accuracy improved from -inf to 0.49796, saving model to models/LSTM-2021-08-23 20-05-44\n"
331 |           ],
332 |           "name": "stdout"
333 |         },
334 |         {
335 |           "output_type": "stream",
336 |           "text": [
337 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
338 |           ],
339 |           "name": "stderr"
340 |         },
341 |         {
342 |           "output_type": "stream",
343 |           "text": [
344 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
345 |           ],
346 |           "name": "stdout"
347 |         },
348 |         {
349 |           "output_type": "stream",
350 |           "text": [
351 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
352 |           ],
353 |           "name": "stderr"
354 |         },
355 |         {
356 |           "output_type": "stream",
357 |           "text": [
358 |             "Epoch 2/40\n",
359 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.6907 - accuracy: 0.5473 - val_loss: 0.7004 - val_accuracy: 0.5177\n",
360 |             "\n",
361 |             "Epoch 00002: val_accuracy improved from 0.49796 to 0.51769, saving model to models/LSTM-2021-08-23 20-05-44\n"
362 |           ],
363 |           "name": "stdout"
364 |         },
365 |         {
366 |           "output_type": "stream",
367 |           "text": [
368 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
369 |           ],
370 |           "name": "stderr"
371 |         },
372 |         {
373 |           "output_type": "stream",
374 |           "text": [
375 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
376 |           ],
377 |           "name": "stdout"
378 |         },
379 |         {
380 |           "output_type": "stream",
381 |           "text": [
382 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
383 |           ],
384 |           "name": "stderr"
385 |         },
386 |         {
387 |           "output_type": "stream",
388 |           "text": [
389 |             "Epoch 3/40\n",
390 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.6819 - accuracy: 0.5651 - val_loss: 0.6283 - val_accuracy: 0.6435\n",
391 |             "\n",
392 |             "Epoch 00003: val_accuracy improved from 0.51769 to 0.64354, saving model to models/LSTM-2021-08-23 20-05-44\n"
393 |           ],
394 |           "name": "stdout"
395 |         },
396 |         {
397 |           "output_type": "stream",
398 |           "text": [
399 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
400 |           ],
401 |           "name": "stderr"
402 |         },
403 |         {
404 |           "output_type": "stream",
405 |           "text": [
406 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
407 |           ],
408 |           "name": "stdout"
409 |         },
410 |         {
411 |           "output_type": "stream",
412 |           "text": [
413 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
414 |           ],
415 |           "name": "stderr"
416 |         },
417 |         {
418 |           "output_type": "stream",
419 |           "text": [
420 |             "Epoch 4/40\n",
421 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.6284 - accuracy: 0.6156 - val_loss: 0.6218 - val_accuracy: 0.7037\n",
422 |             "\n",
423 |             "Epoch 00004: val_accuracy improved from 0.64354 to 0.70374, saving model to models/LSTM-2021-08-23 20-05-44\n"
424 |           ],
425 |           "name": "stdout"
426 |         },
427 |         {
428 |           "output_type": "stream",
429 |           "text": [
430 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
431 |           ],
432 |           "name": "stderr"
433 |         },
434 |         {
435 |           "output_type": "stream",
436 |           "text": [
437 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
438 |           ],
439 |           "name": "stdout"
440 |         },
441 |         {
442 |           "output_type": "stream",
443 |           "text": [
444 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
445 |           ],
446 |           "name": "stderr"
447 |         },
448 |         {
449 |           "output_type": "stream",
450 |           "text": [
451 |             "Epoch 5/40\n",
452 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.6344 - accuracy: 0.6056 - val_loss: 0.5704 - val_accuracy: 0.7007\n",
453 |             "\n",
454 |             "Epoch 00005: val_accuracy did not improve from 0.70374\n",
455 |             "Epoch 6/40\n",
456 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.5978 - accuracy: 0.6362 - val_loss: 0.5423 - val_accuracy: 0.7170\n",
457 |             "\n",
458 |             "Epoch 00006: val_accuracy improved from 0.70374 to 0.71701, saving model to models/LSTM-2021-08-23 20-05-44\n"
459 |           ],
460 |           "name": "stdout"
461 |         },
462 |         {
463 |           "output_type": "stream",
464 |           "text": [
465 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
466 |           ],
467 |           "name": "stderr"
468 |         },
469 |         {
470 |           "output_type": "stream",
471 |           "text": [
472 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
473 |           ],
474 |           "name": "stdout"
475 |         },
476 |         {
477 |           "output_type": "stream",
478 |           "text": [
479 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
480 |           ],
481 |           "name": "stderr"
482 |         },
483 |         {
484 |           "output_type": "stream",
485 |           "text": [
486 |             "Epoch 7/40\n",
487 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.5774 - accuracy: 0.6511 - val_loss: 0.5440 - val_accuracy: 0.7323\n",
488 |             "\n",
489 |             "Epoch 00007: val_accuracy improved from 0.71701 to 0.73231, saving model to models/LSTM-2021-08-23 20-05-44\n"
490 |           ],
491 |           "name": "stdout"
492 |         },
493 |         {
494 |           "output_type": "stream",
495 |           "text": [
496 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
497 |           ],
498 |           "name": "stderr"
499 |         },
500 |         {
501 |           "output_type": "stream",
502 |           "text": [
503 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
504 |           ],
505 |           "name": "stdout"
506 |         },
507 |         {
508 |           "output_type": "stream",
509 |           "text": [
510 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
511 |           ],
512 |           "name": "stderr"
513 |         },
514 |         {
515 |           "output_type": "stream",
516 |           "text": [
517 |             "Epoch 8/40\n",
518 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.5530 - accuracy: 0.6632 - val_loss: 0.5340 - val_accuracy: 0.7252\n",
519 |             "\n",
520 |             "Epoch 00008: val_accuracy did not improve from 0.73231\n",
521 |             "Epoch 9/40\n",
522 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.5499 - accuracy: 0.6642 - val_loss: 0.5367 - val_accuracy: 0.7330\n",
523 |             "\n",
524 |             "Epoch 00009: val_accuracy improved from 0.73231 to 0.73299, saving model to models/LSTM-2021-08-23 20-05-44\n"
525 |           ],
526 |           "name": "stdout"
527 |         },
528 |         {
529 |           "output_type": "stream",
530 |           "text": [
531 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
532 |           ],
533 |           "name": "stderr"
534 |         },
535 |         {
536 |           "output_type": "stream",
537 |           "text": [
538 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
539 |           ],
540 |           "name": "stdout"
541 |         },
542 |         {
543 |           "output_type": "stream",
544 |           "text": [
545 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
546 |           ],
547 |           "name": "stderr"
548 |         },
549 |         {
550 |           "output_type": "stream",
551 |           "text": [
552 |             "Epoch 10/40\n",
553 |             "143/143 [==============================] - 3s 18ms/step - loss: 0.5401 - accuracy: 0.6751 - val_loss: 0.5112 - val_accuracy: 0.7333\n",
554 |             "\n",
555 |             "Epoch 00010: val_accuracy improved from 0.73299 to 0.73333, saving model to models/LSTM-2021-08-23 20-05-44\n"
556 |           ],
557 |           "name": "stdout"
558 |         },
559 |         {
560 |           "output_type": "stream",
561 |           "text": [
562 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
563 |           ],
564 |           "name": "stderr"
565 |         },
566 |         {
567 |           "output_type": "stream",
568 |           "text": [
569 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
570 |           ],
571 |           "name": "stdout"
572 |         },
573 |         {
574 |           "output_type": "stream",
575 |           "text": [
576 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
577 |           ],
578 |           "name": "stderr"
579 |         },
580 |         {
581 |           "output_type": "stream",
582 |           "text": [
583 |             "Epoch 11/40\n",
584 |             "143/143 [==============================] - 3s 19ms/step - loss: 0.5340 - accuracy: 0.6775 - val_loss: 0.6029 - val_accuracy: 0.7456\n",
585 |             "\n",
586 |             "Epoch 00011: val_accuracy improved from 0.73333 to 0.74558, saving model to models/LSTM-2021-08-23 20-05-44\n"
587 |           ],
588 |           "name": "stdout"
589 |         },
590 |         {
591 |           "output_type": "stream",
592 |           "text": [
593 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
594 |           ],
595 |           "name": "stderr"
596 |         },
597 |         {
598 |           "output_type": "stream",
599 |           "text": [
600 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
601 |           ],
602 |           "name": "stdout"
603 |         },
604 |         {
605 |           "output_type": "stream",
606 |           "text": [
607 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
608 |           ],
609 |           "name": "stderr"
610 |         },
611 |         {
612 |           "output_type": "stream",
613 |           "text": [
614 |             "Epoch 12/40\n",
615 |             "143/143 [==============================] - 3s 18ms/step - loss: 0.5349 - accuracy: 0.6846 - val_loss: 0.5104 - val_accuracy: 0.7388\n",
616 |             "\n",
617 |             "Epoch 00012: val_accuracy did not improve from 0.74558\n",
618 |             "Epoch 13/40\n",
619 |             "143/143 [==============================] - 3s 18ms/step - loss: 0.5291 - accuracy: 0.6833 - val_loss: 0.5159 - val_accuracy: 0.7378\n",
620 |             "\n",
621 |             "Epoch 00013: val_accuracy did not improve from 0.74558\n",
622 |             "Epoch 14/40\n",
623 |             "143/143 [==============================] - 3s 18ms/step - loss: 0.5258 - accuracy: 0.6866 - val_loss: 0.5121 - val_accuracy: 0.7422\n",
624 |             "\n",
625 |             "Epoch 00014: val_accuracy did not improve from 0.74558\n",
626 |             "Epoch 15/40\n",
627 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.5268 - accuracy: 0.6813 - val_loss: 0.5140 - val_accuracy: 0.7357\n",
628 |             "\n",
629 |             "Epoch 00015: val_accuracy did not improve from 0.74558\n",
630 |             "Epoch 16/40\n",
631 |             "143/143 [==============================] - 2s 17ms/step - loss: 0.5207 - accuracy: 0.6915 - val_loss: 0.5407 - val_accuracy: 0.7466\n",
632 |             "\n",
633 |             "Epoch 00016: val_accuracy improved from 0.74558 to 0.74660, saving model to models/LSTM-2021-08-23 20-05-44\n"
634 |           ],
635 |           "name": "stdout"
636 |         },
637 |         {
638 |           "output_type": "stream",
639 |           "text": [
640 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
641 |           ],
642 |           "name": "stderr"
643 |         },
644 |         {
645 |           "output_type": "stream",
646 |           "text": [
647 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
648 |           ],
649 |           "name": "stdout"
650 |         },
651 |         {
652 |           "output_type": "stream",
653 |           "text": [
654 |             "INFO:tensorflow:Assets written to: models/LSTM-2021-08-23 20-05-44/assets\n"
655 |           ],
656 |           "name": "stderr"
657 |         }
658 |       ]
659 |     },
660 |     {
661 |       "cell_type": "code",
662 |       "metadata": {
663 |         "id": "Eye28PNHttoz",
664 |         "colab": {
665 |           "base_uri": "https://localhost:8080/",
666 |           "height": 556
667 |         },
668 |         "outputId": "b0babe6b-b99f-4306-9c33-c12d2ed431a2"
669 |       },
670 |       "source": [
671 |         "def plot_history(history):\n",
672 |         "        hist_df = pd.DataFrame(history.history)\n",
673 |         "        print(f'Trained model:')\n",
674 |         "        print(hist_df.tail(1), '\\n')\n",
675 |         "        hist_df_short = hist_df[['accuracy', 'val_accuracy', 'val_loss']]\n",
676 |         "        hist_df_short.plot(figsize=(8,8), grid=True)\n",
677 |         "\n",
678 |         "plot_history(history)"
679 |       ],
680 |       "execution_count": 7,
681 |       "outputs": [
682 |         {
683 |           "output_type": "stream",
684 |           "text": [
685 |             "Trained model:\n",
686 |             "        loss  accuracy  val_loss  val_accuracy\n",
687 |             "15  0.520694  0.691482  0.540717      0.746599 \n",
688 |             "\n"
689 |           ],
690 |           "name": "stdout"
691 |         },
692 |         {
693 |           "output_type": "display_data",
694 |           "data": {
695 |             "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAHSCAYAAAAjRIj6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUZdrH8e+ZyaSS3iAJKdRACGFCEyw0aYIUFcUurrqW1VVXVyzv6iq6uqK7uuuq2FB0dRVEBSkSISCCAglIILSQQkJJSCGVlEnO+8cJIZRAEmbmzIT7c125ksyccp+g+eU55ymKqqoIIYQQwjEZ9C5ACCGEEC2ToBZCCCEcmAS1EEII4cAkqIUQQggHJkEthBBCODAJaiGEEMKBuehdwOmCgoLU6Ohoqx6zsrISLy8vqx7TEch1ORe5Luci1+V8nPnaUlJSClVVDT7bew4X1NHR0WzZssWqx0xOTmbkyJFWPaYjkOtyLnJdzkWuy/k487UpipLT0nty61sIIYRwYBLUQgghhAOToBZCCCEcmMM9oz6buro68vLyqK6ubtf+vr6+7Nq1y8pV6U+v63J3dyciIgKTyWT3cwshxMXGKYI6Ly8Pb29voqOjURSlzfuXl5fj7e1tg8r0pcd1qapKUVEReXl5xMTE2PXcQghxMXKKW9/V1dUEBga2K6SFdSmKQmBgYLvvbgghhGgbpwhqQELagci/hRBC2I/TBLUQQghxMZKgdjAWi0XvEoQQQjiQVgW1oigTFEXZoyhKhqIos8/y/j8URdnW+LFXUZRjzd6rb/bed9Ys3t6mTZvGwIEDiYuLY968eQCsWLGCxMREEhISGDNmDAAVFRXMmjWL+Ph4+vfvz6JFiwDo1KlT07EWLlzIHXfcAcAdd9zBvffey9ChQ/nzn//Mpk2bGDZsGGazmeHDh7Nnzx4A6uvreeyxx+jXrx/9+/fnnXfeYfXq1UybNq3puKtWrWL69On2+HEIIYSwg/P2+lYUxQi8BYwF8oDNiqJ8p6pq+oltVFV9pNn2DwLmZoc4rqrqAGsV/NclO0k/VNamferr6zEajS2+3zfMh2evjjvvcT788EMCAgI4fvw4gwcPZurUqdx9992sW7eOmJgYiouLAXjhhRfw9fUlLS0NgJKSkvMeOy8vjw0bNmA0GikrK+Onn37CxcWFpKQknnrqKRYtWsS8efPIzs5m27ZtuLi4kJOTQ2RkJPfffz9Hjx4lODiYjz76iDvvvLOVPxkhhBCOrjXDs4YAGaqqZgIoivIFMBVIb2H7G4FnrVOeY3nzzTdZvHgxALm5ucybN48rrriiaZhSQEAAAElJSXzxxRdN+/n7+5/32DNmzGj6Y6K0tJTbb7+dffv2oSgKdXV1Tce99957cXFxaTqfoijceuutfPrpp8yaNYuNGzfyySefWO+ihRBC6Ko1QR0O5Db7Pg8YerYNFUWJAmKA1c1edlcUZQtgAV5WVfWbs+x3D3APQGhoKMnJyae87+vrS3l5OQCPjoxsRcmnOl+LGmg6fkt++uknVq5cyQ8//ICnpydXXXUVvXv3Ji0t7Yx9GxoaqKioOON1RVGaXispKaGuro7y8nLq6uowGAxN782ePZthw4bxySefkJOTw6RJkygvL8disVBVVdW0XX19PeXl5cyYMYMbbrgBgKlTp3L8+PHW/3Daqbq6+ox/J2upqKiw2bH1JNflXOS6nE9HvTZrT3gyE1ioqmp9s9eiVFU9qChKN2C1oihpqqrub76TqqrzgHkAgwYNUk9f/WTXrl0XNLGHNSYGqaurIygoiNDQUHbv3s3mzZtRFIWNGzdSWFjYdOs7ICCA8ePH8/HHH/PPf/4T0ELZ39+f0NBQ8vLy6N27NytWrMDb2xtvb29MJhMeHh5NNVZVVdG9e3e8vb1ZuHAhiqLg7e3NxIkTWbBgAZMmTWq69R0VFYW3tzcRERHMnTuXpKQku0yC4u7ujtlsPv+G7eDMK+Cci1yXc5Hrcj4d9dpa05nsINC12fcRja+dzUzg8+YvqKp6sPFzJpDMqc+vncaECROwWCz06dOH2bNnc8kllxAcHMy8efO45pprSEhIaGrVPvPMM5SUlNCvXz8SEhJYs2YNAC+//DKTJ09m+PDhdOnSpcVz/fnPf+bJJ5/EbDaf0gv8rrvuIjIykv79+5OQkMBXX33V9N7NN99M165d6dOnj41+AkIIIfTQmhb1ZqCnoigxaAE9E7jp9I0URYkF/IGNzV7zB6pUVa1RFCUIuBT4uzUKtzc3NzeWL19+1vcmTpx4yvedOnXi448/PmO76667juuuu+6M1+fPn3/K98OGDWPv3r1N38+ZMwcAFxcXXn/9dV5//XXg1Nv169ev5+67727dxQghhHAa5w1qVVUtiqL8AVgJGIEPVVXdqSjK88AWVVVPDLmaCXyhqqrabPc+wLuKojSgtd5fbt5bXFjHwIED8fLy4rXXXtO7FCGE6PiKM8E/Buw0S2OrnlGrqroMWHbaa3857fvnzrLfBiD+AuoTrZCSkqJ3CUII0fHVHYfkl2HDv2DGfOg7xS6ndYrVs4QQwq5K8wgu+BmqzeDuq3c1whFkrYMlf9Ra0+ZbIeZyu51agloIIZrb8TUseZi4mlLY92/odw0MnAXhA+12q1M4kOMl8MP/wdYF2u3u276DbiPsWoIEtRBCANRUwIonYOunED6Q7QFX0d90ANIWaq+F9oOBd0D/66WVfTFQVUj/FpY9DlVFcOnDMHI2mDzsXoosyiGEEIe2wrtXwNbP4PI/wZ0rKQ4cBFPehD/thsn/AMUAyx6D12Lhmwcgb4v2y1x0PKUH4Yub4KvbwacL3LMGxv5Vl5AGaVELIS5mDQ2w8d/w4/PgFQy3Lznz2aO7Dwy6U/s4mAop87VW9jZpZXc4DQ2Q8iGseg4aLDBuDgy9D4z6RqUEtY106tSJiooKvcsQQrSk/Ags/j1kJkOfq+HqN8Ez4Nz7hCdqH+PmwI6FsOUjrZX9w/9Bv2thkDzLPoWqOs/P4uge+O4hyP0Fuo3S7qIExOhdFSBB3eFZLJamRTyEEI32LIdvH4DaKrj6DUi8vW2BIq3ss6sph5wNkLlW6yVduAd6jAXzzdBzHBhNeld4JkstrP8H/DQXXL1g2juQMNOh/sBwvt/gy2fDkbQ27eJRbzn3rYvO8TDx5XMeY/bs2XTt2pUHHngAgOeeew4XFxfWrFnTtMDGnDlzmDp16nnrqaioYOrUqWfd75NPPmHu3LkoikL//v1ZsGAB+fn53HvvvWRmZgLw9ttvExYWxlVXXUV6ujZ/zNy5c6moqOC5555j5MiRDBgwgPXr13PjjTfSq1cv5syZQ21tLYGBgXz22WeEhoZSUVHBgw8+yJYtW1AUhWeffZbS0lK2b9/eNE/5e++9R3p6Ov/4xz/Oe11COLy647DqL7Bpnvb//bUfQnCvCzvmiVb2+Bch7auLq5VdVw15m04G88EUUOvB6AaRQ2HAzbD7e9jzvfZoof8N2muhffWuXJO7SWtFH90F/a6DCS9Dp2C9qzqD8wW1Tm644QYefvjhpqD+8ssvWblyJQ899BA+Pj4UFhZyySWXMGXKFJTz/A/p7u7O4sWLz9gvPT2dOXPmsGHDBoKCgprWt37ooYcYMWIEixcvpr6+noqKivOucV1bW8uWLVsAbVGQX375BUVReP/99/n73//Oa6+9dtZ1s00mEy+++CKvvvoqJpOJjz76iHffffdCf3xC6C8/HRb9DgrS4ZIH4MpnwcXNesd38z7Zyj60VQvs01vZ8TPAw89657S3egsc3qY9LshaB7m/gqUaFKP2x8plD0PMCOg65GTHq6tehYwkref8r+9ofQLCErVWdr/r9Pl51JTDjy9of7D5hMNNX0Kv8favo5WcL6jP0/I9m+NWWD3LbDZTUFDAoUOHOHr0KP7+/nTu3JlHHnmEdevWYTAYOHjwIPn5+XTu3Pmcx1JVlaeeeuqM/VavXs2MGTMICgoCTq5vvXr16qY1po1GI76+vucN6hMLhADk5eVxww03cPjwYWpra5vWz25p3ezRo0ezdOlS+vTpQ11dHfHxMrmccGKqCpvfh5VPa7esb14EPa+07TnDzDDF3HIre+AdEDHI8VvZqqr9YZO1Tms15/wMNWXae6H9tD9KYkZA1HDtZ3s2RhP0nqh9VBbC9i+10P7+T7DiKa1/gPlmiBkJBjsMRNq7EpY+CmUHYcg9MOb/tD+yHJjzBbWOZsyYwcKFCzly5Ag33HADn332GUePHiUlJQWTyUR0dDTV1dXnPU5792vOxcWFhoaGpu9P39/Ly6vp6wcffJBHH32UKVOmkJyczHPPPXfOY99111289NJLxMbGMmvWrDbVJYRDqSzSnkXvXa49K532H+gUYr/zO1srW1WhJOtkMGetg6pC7T3/GG3yl5grIPqK9t0i9gqCYffDJfdpLfOtn2l/yOxYCD4RMOAm7cMWnbgqCmDFbNixCIL7wO9WQdfB1j+PDUhQt8ENN9zA3XffTWFhIWvXruXLL78kJCQEk8nEmjVryMnJadVxSktLz7rf6NGjmT59Oo8++iiBgYFN61uPGTOGt99+m4cffrjp1ndoaChHjx6lqKiITp06sXTpUiZMmNDi+cLDwwFOWdVr7NixvPXWW2esmz106FByc3NJTU1l+/btF/IjE0I/+9fA4nvheDFMeAWG/l7fFuwpreyFkNKsld17Anh30TqfuflordMzvvbVvrZ2h6yyw5D9U2Mwr4XSXO31Tp2hxxgtmGOuAL9I651TUbSfR5hZ60G/Z5nWyl73Kqz7O0RdprWy+07VOnhdCFWFbf+FlU9BXRWMegYu/SO4uFrnWuxAgroN4uLiKC8vJzw8nC5dunDzzTdz9dVXEx8fz6BBg4iNjW3VcVraLy4ujqeffpoRI0ZgNBoxm83Mnz+fN954g3vuuYcPPvgAo9HI22+/zbBhw3jiiScYMmQI4eHh5zz3c889x4wZM/D392f06NFkZWUB2rrZDzzwAP369cNoNPLss89yzTXXAHD99dezbdu2ptvhQjgNSy2sfgE2vAlBveGWhVrHMUfh5q11MBs0S2tlp8yHjB/h+DGoLT/v7pg8zxHmJ772a3Ebl7oy2LXk1J7ZoO0Tc7kWYjEjIKinff6wMblrLfV+12gTjfz2OWz7DL65T5sVLG46mG+BrkPbXk9xJix5WPsDJHKYNgTvQjsP6kBRHWxmnUGDBqknOkGdsGvXLvr06dPuY5Zb4Rm1I7LldU2ePJlHHnmEMWPGnPX9C/03OZfk5GRGjhxpk2PrSa7LDgoztA5jh7dpt5vHvQiunu06lC7X1VCvPQOuLoPq0ha+PnbubSytfIxm8tTCq9sIrcXcuT8YjLa9vtZSVTiwUbs1vnMx1FVCYA/ttnjCjeATdtbdmv7N6i3wy39gzUtgcNFmFRs4yz7PwNtJUZQUVVUHne09aVGLUxw7dowhQ4aQkJDQYkgL4XBUVWuFLfuzdkvzhs+gz2S9q2o7gxE8/LWP9rLUaMFd0xjep4X5/oy9dB8xE8IHOe7tX0XROqhFDYeJr2hzbm/9VJtBbvUc6D5GuzXe+6oze+4f/g2+e1D73HsSTJrbYrA7CwlqG0pLS+PWW2895TU3Nzd+/fVXnSo6Pz8/P/bu3at3GUK03vFjsPRhreUVfTlcM8/pfzFfEBc3raNXC529cmuT6R413M5FXQC3Tloom2+Gov3a8+bfPoev7tD+oIm/XnsvsCfd9n8Ma7/VOq1d/wn0meL4PetbQYLahuLj49m2bZveZQjRceVshK/vhvLDMOZZ7fmqo9y+FdYX2F0bTjXqKchco90aT5kPm94F105E1lZA4m0w9vkLuyvhYCSohRDOp95ysoewXxTc+QNEDNS7KmEvBiP0uFL7OF6i9aI/sJFtxgEMmPKQ3tVZnQS1EMK5HDsAi+7WFk9IuFGb+crBJ6wQNuThD0PuhiF3cyw5We9qbEKCWgjhPHYsgiWPACpc8z70n6F3RULYnAS1EKL9Dm0jIvc7WL8NGuq0W9INdVBfp63nW18H9bUnvz79vVP2qT37/s23qauCiCFw7XvgH6331QthFxLUNnKu9aizs7OZPHkyO3bssHNVQjcl2XiX7QF1hPP3QlVV2P8j/PwmZK2lB8D+Zu8rRm32LINJW7XO6Hrya4Op8T2XZtuYtJ7KRtezvHfaPv7R2njYc62GJ0QHI/+1C2FrORvg85kMrC6FgwtgyF3a/M4XOjWivdXXaUOgfn4T8tO0KS/HvsDPlZFcOnKsFqgGF4eeVEIIZ+R0Qf3KplfYXby7TfvU19djNLY8ZCM2IJYnhjxxzmNYcz3q5qqrq7nvvvvYsmULLi4uvP7664waNYqdO3cya9YsamtraWhoYNGiRYSFhXH99deTl5dHfX09jz32GHfccUebzifsbNdSWHgn+EWyL2IGPct/gSV/1NZEHnALDP6dNuTEkdVUQOon2kxPpbkQHAtT/6P9seHiSl1ysvP90SGEE3G6oNaLNdejbu6tt95CURTS0tLYvXs348aNY+/evbzzzjv88Y9/5Oabb6a2tpb6+nqWLVtGWFgY33//PaAtXykc2OYPtEUXwhLhpi85uDmNniPmalMjbnpPG/v5y1vaqk5D7taGmjjSGOCKAm394M0faNNWRl0Kk17T6pVWsxB243RBfb6W79lYY05sa65H3dz69et58MEHAYiNjSUqKoq9e/cybNgwXnzxRfLy8rjmmmvo2bMn8fHx/OlPf+KJJ55g8uTJDBgw4IKuSdiIqkLy32DtK9BzPMz46GSLs/nUiOVHtMkatnwE/71eGw88+C5tAQLPAP3qL8yAjf+CbZ9rHbz6XK1NJBJx1mmIhRA2Jn8Wt8GJ9aj/97//nbEe9bZt2wgNDW3zutItuemmm/juu+/w8PDgqquuYvXq1fTq1YvU1FTi4+N55plnePnll61yLmFF9Rbt1vbaV7Rb2zM/a/m2sHdnGDkbHtkB130EvhGw6v/g9T7aGsqH7DyrXe5m+OJm+PcgLaQH3AQPpsANCySkhdCR07Wo9WSt9aibu/zyy/nss88YPXo0e/fu5cCBA/Tu3ZvMzEy6devGQw89xIEDB9i+fTuxsbEEBARwyy234OfnxzvvvGODqxTtVndcex69Zxlc/hiMfqZ1PbyNppPL/OXv1G6Lb/+ftghBxBDttnjfqWcuPmANDQ2wb6XWQezABm2pwysegyH3QKcQ659PCNFmEtRtYK31qJu7//77ue+++4iPj8fFxYX58+fj5ubGl19+yYIFCzCZTHTu3JmnnnqKzZs38/jjj2MwGDCZTMydO9cGVynapaoYPp8JuZvgqrlauLZHaBxc/U+48jlt4YFN72lzWa98ChJv19Yw9o248HotNZD2lRbQhXvAtytMeBnMt2qLIAghHIYEdRulpaU1fR0UFMTGjRvPul1LY6gBoqOjm8ZQu7u789FHH52xzezZs5k9e/Ypr40fP57x48c3fV9e3opF5oXtlebBgmugJAtmzIe4aRd+TA8/uOQ+GPJ7bfGBze/DT6/B+n9A7FVaizf68raPya4u1Z6J//I2VByB0Hhthq+4aVrLXgjhcCSohbgQ+enw6bVQWwG3Loboy6x7fIMBeozRPkpyYMuH2lCpXUu0YVKD74KEmeef67rskDa8ast8qC2HbiNh2n+g+2jnn4BFiA5OgtqGnHE9atEGjROZYPKEWcuhcz/bns8/Csb+FUY+CTu/hk3ztOFfSX+FATdqoR3c+9R9CnbBhn/B9i9BrYe46TD8IQiTEQNCOAsJahuS9ag7sF1LYOHvwC8Sbv1a+2wvJnetR/aAmyAvBTa/17gm7zyIuUK7Le7hrwX03hXg4gGD7oRh98v82EI4IacJalVV2zSRiLAdVVX1LkFfJyYyCR8IN32p75jniIHax7g52i3xLR/C/27R3vMMhJFPaS1tr0D9ahRCXBCnCGp3d3eKiooIDAyUsNaZqqoUFRXh7u6udyn213wik14TtLHPrp56V6XxCoLLH9UmJtm7UptJrO80x6lPCNFuThHUERER5OXlcfTo0XbtX11d3SGDRa/rcnd3JyLCCkOEnEm9Bb5/RGu1mm+ByW845gpOBqPWK1wI0WE44G+aM5lMJmJiYtq9f3JyMmaz2YoVOYaOel0Op7YKFv1Om8jkisdh1NPSU1oIYTdOEdRC6MZaE5kIIUQ7SVAL0ZJjudoY6ZJsuP5jbRpPIYSwMwlqIc6maSKTSm34lbUnMhFCiFaSoBbidNk/w+c3aj2m71yuzb8thBA6kWUuhWgu/TtYMB28Q+F3P0hICyF0J0EtxAmb34cvb4MuCXDnSvvONiaEEC2QW99CqCqseQnW/R16TYTrPpSJQoQQDkOCWlzcTpnI5FaY/E/HnMhECHHRkt9I4uJVWwUL74S9y+GKP8Oop2QiEyGEw5GgFhen6lL47HrI/RUmvaYtXCGEEA5IglpcfKqK4dNr4EgazPhIW6NZCCEclAS1uLhUFMAn06AoA2b+F3qN17siIYQ4JwlqcfEoPQifTIWyg3Dzl9BtpN4VCSHEeUlQi4tDSTZ8PEW77X3L1xA1TO+KhBCiVSSoRcdXmAGfTNHm7b79WwgfqHdFQgjRahLUomPLT9dud6sNcMf30Lmf3hUJIUSbSFCLjuvQNm3ebhc3uO17CO6ld0VCCNFmMte36JhyN2nPpF07waxlEtJCCKclQS06nqx12hAsr0BtmcqAbnpXJIQQ7SZBLTqWfUnw2Qzw6wqzloNvhN4VCSHEBZGgFh3HrqXw+UwI6gV3LAPvznpXJIQQF0yCWnQMaQu1taTDBsDtS7Tb3kII0QFIUAvnl7oAFt0FUcPh1sXg4ad3RUIIYTUS1MK5/ToPvvsDdB8NN30Jbt56VySEEFYlQS2c189vwPLHofckuPFzcPXUuyIhhLA6mfBEOB9VhbWvQPLfoN+1MP1dMJr0rkoIIWxCglo4F1WFVX+BDW/CgFtgyptgMOpdlRBC2IwEtXAeDQ2w/M+w+T0YfDdM/DsY5OmNEKJjk6AWzqGhHr57CLZ9CsMfhLEvgKLoXZUQQticBLVwfPV1sPj3sGMRjJgNI2dLSAshLhoS1OL8vv49HNkOoXGNH/20z95dbB+YlhpYeCfsXgpX/hUue9i25xNCCAcjQS3OrbIItv8PAnvAgV8g7auT73n4nwztEx/Bfaw3TKq2Cr68FTKSYOKrMPQe6xxXCCGciAS1OLfMNYCqDYGKGAjHS6BgF+TvhCNp2ufUT6CuStteMUBA91Nb3qFx4BfZttZ3TTl8fiNkr4cp/4bEW21yeUII4ehaFdSKokwA3gCMwPuqqr582vv/AEY1fusJhKiq6tf43u3AM43vzVFV9WNrFC7sJCMJPAK0ObRBa0VHDdc+TmhogJIsLbTzd0L+Djj8G6R/c3IbNx8I6Xva7fO+Z51JzKWuAhZMh4OpcO37EH+djS9SCCEc13mDWlEUI/AWMBbIAzYrivKdqqrpJ7ZRVfWRZts/CJgbvw4AngUGASqQ0rhviVWvQthGQ4MW1D3GnHusssEAgd21j75TTr5eU9HY+t5xMsTTFsKWD05u4xd1suXduR/4R5Pw2/9BVS5c/zH0udp21yeEEE6gNS3qIUCGqqqZAIqifAFMBdJb2P5GtHAGGA+sUlW1uHHfVcAE4PMLKdrRNKgNbC3YytrctYyPHk9cUJzeJVnHke1QeRR6XNm+/d06QdfB2scJqgqleSdb3icCfO9yUBsA8DS4alOC9hxrhYsQQgjn1pqgDgdym32fBww924aKokQBMcDqc+wb3vYyHY+qqqQXpbMsaxkrsldQUFUAwKHKQ8wdMVfn6qwkI0n73H209Y6pKODXVfvoPeHk63XH4egeKEgnNa+WwRLSQggBWL8z2Uxgoaqq9W3ZSVGUe4B7AEJDQ0lOTrZqURUVFVY75qHaQ6RWppJSlUKhpRAjRvp69GVi0ES2Vm7ll9xfWLNmDYodxvla87rOZsDWRRg7dSdlSzot30CxtjAqsO116cXW/156ketyLh31uqDjXltrgvog0LXZ9xGNr53NTOCB0/Ydedq+yafvpKrqPGAewKBBg9SRI0eevskFSU5O5kKOeaDsACuyV7A8azkZxzIwKAaGdh7KQzEPMTpyNL5uvgB8sfsLXvz1RXoO6kmEd4SVqm/ZhV7XOR0/Bmv3wGWP2O4cLbDpdelIrsu5yHU5n456ba0J6s1AT0VRYtCCdyZw0+kbKYoSC/gDG5u9vBJ4SVEU/8bvxwFPXlDFdnKk8ggrs1eyPGs5O4t2ApAYksjTQ5/myqgrCfIIOmOfxNBEAFILUu0S1DaVmQxqffufTwshhLCK8wa1qqoWRVH+gBa6RuBDVVV3KoryPLBFVdXvGjedCXyhqqrabN9iRVFeQAt7gOdPdCxzREXHi1iVs4rlWctJLUgFIC4wjscGPcb46PF09up8zv17+PXA29Wb1PxUpnSfcs5tHV5GErj5QsTg828rhBDCZlr1jFpV1WXAstNe+8tp3z/Xwr4fAh+2sz6bK6st48ecH1metZxfj/xKg9pAd9/u/GHAH5gQM4Eon6hWH8ugGDCHmJtC3mmpKmT8CN1HglHmxBFCCD1dlL+Fq+qqSM5NZnn2cn4++DN1DXVEdIrgd/1+x8SYifT079nuY5tDzKzLW0dxdTEB7gFWrNqOCtKh/JDc9hZCCAdw0QR1TX0N6w+uZ0XWCtbmreW45TghniHcGHsjE2MmEhcYZ5We2okh2nPqbQXbGB1pxWFN9nRiWJYEtRBC6K5DB7WlwcKvh39lQeECnvzfk1TUVeDv5s+U7lOYED2BxNBEDIrBqufsF9QPV4Mrqfmpzh3UIXHgE6Z3JUIIcdHr0EFdZaniwdUP4qK6ML77eCZGT2RIlyG4GGx32a5GV/oF9WNrwVabncOmasohZyNccp/elQghhKCDB7WPqw8fTfiIgh0FjL3UfjNdJYYmMn/HfKrqqvA0WWnJR3vJ+gka6mT6TiGEcBDWve/rgBKCEzApJrue0xxixqJa2FG4w67ntYqMJDB5QclR20wAACAASURBVNdL9K5ECCEEF0FQ62FAyAAUFOcbpqWqkLEKuo0AF1e9qxFCCIEEtU34uPrQ078nqflOFtRFGXDsgPT2FkKIFlTVWvj36n1U1ljsds4O/YxaT+YQM0v2L8HSYLFp5zWrkmFZQghxVqqqsnLnEV5YuouDx44TGejFlAT7jIyRFrWNDAwdSJWlij0le/QupfX2rYKgXuDf+tnYhBCio8s8WsFtH27i3k9T8XZ34at7h9ktpEFa1DZjDjEDsDV/K3GBcTpX0wp1xyHnZxh0p96VCCGEQ6iqtfCv1Rm8/1Mm7i5Gnru6L7dcEoWL0b5tXAlqG+ns1ZkwrzBSC1K5pe8tepdzftk/g6UaeozRuxIhhNCVqqosSzvCnO/TOVxazbWJEcyeGEuwt5su9UhQ21BiaCIbD21EVVWrTE9qUxmrwMUDoi7TuxIhhNBNRkEFz323k/UZhfTt4sO/bzIzMErfdRskqG3IHGJmaeZScstzifSJ1Lucc8tIgujLwOSudyVCCGF3lTUW3ly9jw/XZ+FuMvLXKXHcPDTS7re5z0aC2oYGhg4EICU/xbGDujhLG5o1+G69KxFCCLtSVZXv0w4zZ+kujpRVM2NgBE9MjCWokz63uc9GgtqGYnxj8HXzZWvBVqb3nK53OS07MSxLpg0VQlygyhoLuw6XseNgKYdKqxkcHcBlPYLwcDXqXdoZMgrK+cu3O9mwv4i4MB/eujmRgVH+epd1BglqGzIoBszBZsdfoCPjR/CPhoBuelcihHAixZW17DxUys5DZdrHwVKyiipRVe19F4PCvHWZuJsMXN4zmLF9QxkTG0Kgzq3VihoLb/6o3eb2dDXywrR+3DQkEqPBMfsSSVDbWGJoIsl5yRQeLyTII0jvcs5kqYGsdTDgRnD0Dm9CCF2oqsqRsmp2Hixjx4lgbmwxnxDu50FcmA/TzOHEhfnQL9wXf09Xfs0qYlV6Pknp+axKz8egwMAof8b2DWVs387EBHnZ9TqWbD/Mi9+nk19Www2DuvLnCb11/8PhfCSobezEeOptBdu4MsoBZ/w6sBHqKqGH3PYWQg+7j5SxIaMIb3cXArxc8fdyJcDTFX9PV7zdXTDYuZXX0KCSU1zFzkOl7DhY1tRiLq6sBbS/57sFeTEoOoB+4T7EhfnSt4sP/l5nXx/g8p7BXN4zmL9OiWPnoTJ+aAzsl5bt5qVlu+kR0qkxtEMZEOFns+vdm1/Os9/uZGNmEfHhvrxzy0DMkY53m/tsJKhtLC4wDjejGyn5KY4Z1BlJYHTVenwLIeyi1tLAyp1HWLAxh03ZxS1uZzQo+Hua8G8Mbn8vkxbmnq4EeLni5+lKgJfplO993F1aPRy0rr6BjIIKdh7SnimnHyoj/XAZFY3zWJuMCj1DvLmyTwhxYb70C/chtrMPXm5tjw5FUegX7ku/cF8eHduL3OIqknZpoT1vXSZvJ+8n2NuNK/uEMK5vZ4Z1D8TddOHPtcur63gjaR/zN2Tj5ebCi9P7MXOw497mPhsJahszGU3EB8U77nPqfUkQOQzcOuldiRAdXn5ZNZ/9eoDPNx3gaHkNkQGePHVVLFcnhGGpVymurKW4qpZjVbUUV9ZR0vh9SWUtJVW1ZBdWkXrgGCWVtVga1LOew8WgNAW4n2dj69zrZKBn59Sx8uvt7DxUxu4j5dRaGgDwMBnp08WbaxK1W9dxYb70DO2Em4ttOoF1DfBk1qUxzLo0htKqOtbsKWBVej7fbTvE55ty8XQ1MqKX9lx7dGwIfp5tW9FPVVW+3XaIl5bt4mhFDTMHd+Xx8bEEtNDyd2QS1HZgDjHz4Y4PqaqrwtPkqXc5J5XmwdFdYL5Z70qE6LBUVeXXrGIWbMxh5c4j1KsqI3sFc9uwaEb0Cj7lVm/XgNb9flBVlfIaC8cq65qCvLgxzEtOC/n9RysoyamjpKqW+sZw9/U4QlyYD3cMj24MZR9igjrp1sr09TQxzRzONHM4NZZ6Nu5vfK69K5/lO45gNCgMjvZnbN/OjOsbet6f0+4jZfzl251syiqmf4Qv7902iISufna6GuuToLaDgaEDeS/tPX47+hvDwobpXc5JGT9qn2W1LCGsrqLGwuKtB1mwMZu9+RX4epi487IYbh4aSVTghXWgUhQFH3cTPu4mIgNbH+5l1RbWrP2JqeNHOexsiW4uRkb2DmFk7xBemNqPtIOlrErP54f0I7ywNJ0XlqYT29m76bl2fLhv07VU1ak8vySdjzdm4+3uwkvT47lhcFenus19NhLUdpAQnIBBMbC1YKuDBfUq8AmH4Fi9KxGiw8goKGfBxhwWpR6kosZCv3Af/n5df6YkhFnlmWt7KYqCr4cJP3eDw4b06QwGhYSufiR09eOx8b3JKapsDO183lqTwb9WZ9DZx50r+4bQLagTb6w/TlltFjcOieTxcb1b7ODmbCSo7aCTayd6+/cmNT9V71JOqq+DzLUQN12GZQlxgSz1DSTtyueTjTls2F+Eq9HApP5duHVYFOaufk4TjI4uKtCLuy7vxl2Xd6O4spbVuwtYlX6ERSkHOV5XTzdfAwvuHk7/COe9zX02EtR2Yg4xszhjMXUNdZgMJr3LgbzNUFMmt72FuABHy2v4YtMB/rvpAIdLqwn38+Dx8b25YXBXh5qCsiMK8HLluoERXDcwguq6ejKPVnJkT0qHC2mQoLYbc6iZ/+7+L3uK99AvqJ/e5cC+VWBwgW4j9K5ECKeiqiqpB0r4ZGMOy9IOU1evcnnPIP46JY4xfUKd/nmoM3I3Gekb5kPB3o75s5egtpPEkERAW6DDIYI6Iwm6DgV3X70rEcIpHK+t59ttB/lkYw7ph8vwdnfhlkuiuOWSKLoHy/BGYTsS1HYS4hlCRKcIthZs5fa42/UtpjwfjmyHMX/Rtw4hnEB2YSULfsnhqy25lFVbiO3szUvT45lmDsPTVX6FCtuT/8rsKDE0kfUH16Oqqr6dS/afGJYl04YKcTZVtRa2Flj48MNNrNt7FBeDwsT4Ltw2LIpBUf7SOUzYlQS1HSWGJPLd/u/ILssmxjdGv0IykqBTKHSO168GIXRWXl1HTlEVOUVVZBdVkl1Y2fR1QXkNAKE+ZTw6thczh3QlxNtd54rFxUqC2o7ModoCHVsLtuoX1A31sH819Joow7JEh1d6vI6cokqyi6rIKdQ+ZxdVklNUSWFF7Snbhni7ER3kxcjewUQFelF3NJsHrh2NyWjQqXohNBLUdhTjE4O/mz8p+Slc0/MafYo4mArHS6CnDMsSHcOxqlotgAsrG0O4qunziRWfTuji605UoCdX9gklKtCLmCBPogK9iAr0PON5c3JynoS0cAgS1HakKArmELO+C3RkJIFigG6j9KtBiHY4VqVNcHEilE+0lEuP1zVtoygQ5utBVKAn4+M6NwVxdKAXkQGeeLjqNzOYEO0lQW1niaGJrM5dzdGqowR7Btu/gIxVED4QPAPsf24h2uHEKkjPL02nuLIWgwJhfh5EB3pxdUIXogO9GsPYk64BnrpO0ymELUhQ29mJ8dSpBamMjx5v35NXFmm3vkc+ad/zCtFOucVVPPPNDtbuPcqArn68f/sg4sJ8bLb0ohCOSILazmIDY/Fw8WBrwVb7B3XmGkCVaUOFw7PUNzB/Qzav/bAXgwLPXd2XW4dFy6xf4qIkQW1nJoOJ/kH99VmgY98q8AiAsAH2P7cQrbTzUCmzF6WRdrCU0bEhvDCtH+F+HnqXJYRuJKh1YA41M2/7PCpqK+jkaqepBxsatIlOeowBg9w2FI6nuq6efybt472fMvH3NPGvG81M7t9FJhcRFz0Jah0khiTSoDaw/eh2hocPt89Jj2yHyqNy21s4pJ8zCnlqcRo5RVVcPyiCp67qg59nx1hLWIgLJUGtg/7B/TEqRlIKUuwX1BmrtM/dx9jnfEK0QkllLS8u28XClDyiAz35791DGd49SO+yhHAoEtQ68DJ50Tugt33HU2f8CF0GQCcdhoQJcRpVVfnut0M8vySd0uN13D+yOw+N6SlDq4Q4CwlqnSSGJPLV3q+oq6/DZDTZ9mTHj0HuJrjsEdueR4hWyCvRhlwl7zlKQoQvn941lD5dfPQuSwiHJfPj6SQxNJGa+hrSi9Ntf7LMZFDroaesliX0U9+g8sH6LMb9Yx2bsor5y+S+fH3/pRLSQpyHtKh1Yg5pXKAjfysJwQm2PVlGErj5Qvgg255HiBakHyrjya+381teKaN6B/PCtH5E+HvqXZYQTkGCWidBHkFE+USRUpDCHdxhuxOpqvZ8uvtIMMo/t7Cv6rp63vhxH++ty8TXw8QbMwcwJSFMhlwJ0Qbym1tH5hAzybnJNKgNGBQbPYUoSIfyQ9BDbnsL+9rQOOQqu6iK6wZG8PRVffD3kiFXQrSVPKPWUWJIIsdqjpFVmmW7k2QkaZ97yLAsYR/Hqmp5/KvfuOn9X1GBz+4aytwZCRLSQrSTtKh1lBh6coGO7n7dbXOSjCQIiQOfMNscX4hGqqqydPth/rpkJyVVddw7ojsPXylDroS4UNKi1lGkdySB7oFszbfReOqacsjZCD1lNjJhWwePHed3H2/hwc+3EubnwXd/uJTZE2MlpIWwAmlR60hRFBJDE0ktsNECHVk/QUOdTBsqbEJVVapq61mVXccDq9fSoMIzk/ow69IYWeVKCCuSoNaZOcTMqpxVHKk8QmevztY9eMYqcO0EXS+x7nGF02toUCmvsVBRY6G8uo7y6uafLad93/i55tTXK2os1DeoAIzoFcycaf3oGiBDroSwNglqnZ14Tr21YCsTYyZa78Cqqj2fjhkBLtKJ52KyObuYpPR8yk4L1eZBXFFjOe9xXAwK3u4ueLub6OTmgre7C+F+Hvi4e5983d2F6vws/jhjsAy5EsJGJKh11tu/N54unqTmp1o3qIsy4NgBuPRh6x1TOLRaSwOvr9rLu+v2YzIY8PEwNQaq9hHcqdMpAevT9J7plED2afza3WRoVfgmJ+dKSAthQxLUOnMxuJAQnGD9BTr2Na6WJc+nLwqZRyv44xfbSDtYyo1DuvJ/k/vi6Sr/ewvREUivbwdgDjWzt2QvZbVl1jtoRhIE9QL/KOsdUzgcVVX5YtMBJr25ntySKt65ZSB/u6a/hLQQHYgEtQNIDElEReW3gt+sc8C645Dzs7SmO7hjVbXc/1kqs79Owxzpx4o/XsGEflbukCiE0J382e0A4oPicVFcSC1I5fKIyy/8gNnrwVItQd2BbdhfyKP/+42iyhqenBjL3Zd3wyBDooTokCSoHYCnyZM+gX1IzbfSeOqMJHDxgKhLrXM84TCadxiLCfTivdsuJT7CV++yhBA2JEHtIBJDEvl89+fU1tfiarzA4VQZSRB9GZjcrVOccAjSYUyIi5M8o3YQ5lAztQ217CzaeWEHKs7Shmb1lNWyOgrpMCbExU3+T3cQ5hAzAKn5qU1ft0vTalnyfLojOFZVy5Nfp7F8xxGGdw/k9esH0NlX7pQIcTGRFrWDCHAPIMY35sLn/c74EfyjIaCbVeoS+tmwv5AJ//yJpF35PDkxlk9/N1RCWoiLkLSoHUhiSCI/5PxAg9qAQWnH31CWGshaBwNuApkpymlJhzEhRHPSonYgiaGJlNeWs//Y/vYd4MBGqKuU295OLPNoBde+vYF31u5n5uCuLH3oMglpIS5y0qJ2IM2fU/f079n2A2QkgdEVYqwwFlvYlaqq/G9zLn9dko6bycA7twyUyUuEEIC0qB1KRKcIgj2C2/+cel8SRA0HVy/rFiZsSmYYE0Kci7SoHYiiKCSGJrZvgY7SPDi6C8w3W78wYTMyw5gQ4nykRe1gzCFmDlce5nDF4bbtKMOynEqtpYGXl+/m5vd/xdPVyNf3XcrvR3SXkBZCnEFa1A5mYOhAAFILUpnUaVLrd8xIAp8ICI61UWXCWmSGMSFEW0iL2sH09OuJl8mrbfN+19dB5lroMUaGZTkwmWFMCNEerfoNoSjKBOANwAi8r6rqy2fZ5nrgOUAFflNV9abG1+uBtMbNDqiqOsUKdXdYRoORAcED2tahLHcT1JTJbW8Hdqyqlre21bAlP01mGBNCtMl5g1pRFCPwFjAWyAM2K4rynaqq6c226Qk8CVyqqmqJoighzQ5xXFXVAVauu0NLDE3kX1v/RWlNKb5urRhDm5EEBhfoNsL2xYk2yzxawR0fbeZgSb10GBNCtFlrbn0PATJUVc1UVbUW+AKYeto2dwNvqapaAqCqaoF1y7y4nBhPva1gW+t2yEiCrkPBXSbGcDQpOcVc+/YGKmosPDnEXTqMCSHarDW3vsOB3Gbf5wFDT9umF4CiKD+j3R5/TlXVFY3vuSuKsgWwAC+rqvrN6SdQFOUe4B6A0NBQkpOT23IN51VRUWH1Y9pSbUMtRox8s+Ub1P1qi9tVVFSwYeVihh/ZTmbMrRxwoms8F2f792rJ5iMW3t1eQ6C7wp8GuuPZUNUhrut0HeXf63RyXc6no16btXqxuAA9gZFABLBOUZR4VVWPAVGqqh5UFKUbsFpRlDRVVU+ZI1NV1XnAPIBBgwapI0eOtFJZmuTkZKx9TFv7ZNknFFJ4zrqTk5MZ7ncIgG7j7qFbl/52qs62nPHfqzlVVflgfRb/+W0XiZH+vHfbIAK8XJ3+uloi1+VcOup1Qce9ttbc+j4IdG32fUTja83lAd+pqlqnqmoWsBctuFFV9WDj50wgGbiANRwvHokhiewo2kG1pfrcG2YkQadQ6Bxvn8LEOdU3qPx1STpzvt/FhLjOfHbXUAK8XPUuSwjhxFoT1JuBnoqixCiK4grMBL47bZtv0FrTKIoShHYrPFNRFH9FUdyavX4pkI44r8TQRCwNFnYW7Wx5I7Ue9q/WenvLsCzdHa+t575PU5i/IZu7LovhrZsScTcZ9S5LCOHkzhvUqqpagD8AK4FdwJeqqu5UFOV5RVFODLVaCRQpipIOrAEeV1W1COgDbFEU5bfG119u3ltctGxAsNZR/lzjqX3KMuB4iTZ+WuiqsKKGG9/7hVW78nn26r48M7mvdBoTQlhFq55Rq6q6DFh22mt/afa1Cjza+NF8mw2A3JNtBz93P3r49TjneOqA4lRQDNBtlB0rE6c7Mfwqv6yad24ZyPg4WVBDCGE9MjOZAzOHmNlWsI36hvqzvh9QnALhg8AzwM6ViROaD7/6/J5LJKSFEFYnQe3AzCFmKuoqyDiWceablUV4l2fIbGQ6Wp52mBvf+xU/T1cW3z+cxEh/vUsSQnRAEtQOrPkCHWfIXIOCKkGtA1VVef+nTO7/byrx4b4sum84UYGyBrgQwjYkqB1YF68uhHqGnr1D2b5V1Jp8IExGu9mTDL8SQtibLNvjwBRFITE0kZQjKaiqitJ8CFbWOkr8Ewg1yN9a9nK8tp4/frGVH9LzueuyGJ66qo/07BZC2Jz8lndwiSGJFBwv4GBFszlmaiuh/BCVXlH6FXaRkeFXQgi9SIvawZ1YoGNrwVYivCO0F0uyAah2D9WpqouLDL8SQuhJWtQOrqd/T7xN3qd2KGsM6uMeXfQp6iIiw6+EEHqTFrWDMygGBoQMOLVDWXEWAMc9pEVtS8vSDvPw/7YR7ufB/FmDpWe3EEIX0qJ2AomhiWSWZlJSXaK9UJINbr5YXLx1raujOjH86gEZfiWEcAAS1E4gMSQR0J5TA1CSBQHRshCHDcjwKyGEo5GgdgJxQXGYDKaTQV2cBf7RutbUEcnqV0IIRyTPqJ2Am9GN+KB4rUNZQz0cOwB9rta7rA6lsKKGuz7ewm95x3j26r7MujRG75KEEAKQFrXTMIeYSS9M53jxfmiokxa1FWUereCa/2xg1+Ey3rlloIS0EMKhSFA7icTQRCyqhR2567QXAiRMrEGGXwkhHJ0EtZNICE5AQSHlyBbtBX8J6gu1Kj1fVr8SQjg8CWon4evmSw//Hmwt3QcGF/AJ17skp3asqpbHvvqN3qHeMvxKCOHQJKidSGJIIttqCrH4dQWj9AO8EP9anUF5dR2vzugvw6+EEA5NgtqJJIYkUkUDe/1k6tALkVNUyScbs5kxsCuxnX30LkcIIc5JgtqJJIY2TnziIbdpL8QrK3ZjMhr407heepcihBDnJUHtRDob3Airs5BiqNW7FKe1JbuYZWlH+P0V3Qnxcde7HCGEOC8JamdSnIW5poatNYWoqqp3NU5HVVXmfL+LUB837r5Ces0LIZyDBLUzKckmsbqGwrpy8srz9K7G6Szdfphtucf407jeeLpKZzwhhHOQoHYmJVkkVtcAkFKQonMxzqXGUs8rK3YT29mbaxMj9C5HCCFaTYLamZRk083VD18335MLdIhW+XhDNnklx3lmUl+MBll1TAjhPCSonUlxFgb/GMzBZlLzU/WuxmkUV9byr9UZjOodzGU9g/QuRwgh2kSC2pmUZIN/NOZQM9ll2ZTXl+tdkVN488d9VNZYePKqPnqXIoQQbSZB7SwstVCaBwExJIZo46kzazJ1LsrxZR6t4NNfcpg5JJJeod56lyOEEG0mQe0sjh0AVPCPIS4wDjejG0uPLeWrvV9RXist65a8smI3bi4GHrlSJjcRQjgnCWpnUZKlffaPxmQ0MeeyOQA8v/F5Rn85mqd+eorNRzbToDboWKRj+TWziJU787lvZHeCvd30LkcIIdpFBpM6i5Js7XPjOtQToifgluVGUL8gFmcsZnnWcpZkLiGiUwTTekxjao+pdPa6eNdWbmhQeXHZLrr4uvO7y7rpXY4QQrSbBLWzKM4CFw/oFNr0kqIoxAfHEx8cz+ODHycpJ4lvM77l39v+zVvb3mJ42HCm9ZjGqMhRuBkvrhblku2H2J5XymszEvBwNepdjhBCtJsEtbNo7PGNcvYxwB4uHlzd/Wqu7n41eeV5fLv/W77J+IbH1z2Oj6sPk7pNYnqP6fQJ7Pg9n6vr6vn7ij3Ehfkw3SzrdgshnJsEtbMoyWq67X0+Ed4RPDDgAe7tfy+/HvmVb/Z9w6K9i/h89+fEBsQyrcc0JsVMws/dz8ZF6+Ojn7M5eOw4r87oj0EmNxFCODkJamegqlqLutvINu1mNBgZHjac4WHDKa0pZVnWMr7J+IaXN73Ma1teY1TXUUzvOZ1hXYZhNHSM28NFFTX8Z00GV/YJYXh3mdxECOH8JKidQUUB1FWBf/tXfPJ18+XG2Bu5MfZG9hTv4ZuMb1iauZQfcn4g1DOUKd2nMK3HNCJ9Iq1YuP39M2kfVXX1zJ7Y8W/xCyEuDjI8yxk0G5plDb0DevPEkCf4ccaPvDbiNXr59+KDHR8wafEkZq2YxbcZ31JVV2WVc9lTRkEF/910gJuGRNIjpJPe5QghhFVIi9oZnDY0y1pcja6Mix7HuOhx5FfmsyRzCYv3LeaZn5/hb5v+xoToCUzrMY2E4ASUFjqxOZKXl+/Cw2Tk4St76l2KEEJYjbSonUFxFqCAn+1uS4d6hXJX/F0snb6U+RPmc2XklSzLWsaty29l6rdT2Xhoo83ObQ0b9heStKuA+0d1J7DTxTUUTQjRsUlQO4OSLPAJBxfbB5CiKAwMHcicy+aw5vo1PD/8eWosNby65VWbn7u9GhpUXvx+F+F+Htx5qXXvOgghhN4kqJ1BSbbVb3u3hpfJi+k9p3Nb3G3sK9lHVmmW3WtojcVbD7LzUBl/ntAbd1PH6L0uhBAnSFA7g+Is8I/S7fRjIscA8EP2D7rV0JLjtfXM/WEP/SN8ubp/mN7lCCGE1UlQO7raSqgsuKChWReqs1dnBgQP4IccxwvqD9Zncri0mqev6iOTmwghOiQJakdnox7fbTUuehx7S/aSXZqtax3NFZRX83byfsb1DWVot0C9yxFCCJuQoHZ0xdYdQ91eY6PGArAqZ5WudTT3z6R91FgamD0xVu9ShBDCZiSoHd2JFrWOt75Bu/2dEJzgMLe/9+aX88WmA9xySRTdgmVyEyFExyVB7ehKssDdFzwD9K6EsVFj2V28mwNlB/Quhb8t24WXmwsPjZHJTYQQHZsEtaMrztL9tvcJ46LGAejeql6/r5A1e47y4OgeBHi56lqLEELYmgS1oyvJ1v229+ur9vLSsl14GoPoH9Rf12Fa9Q0qc75PJ8Lfg9uGRetWhxBC2IsEtSNrqIdjB3RtURdW1PDv1fuYty6TUXOTCTYMZlfxLnLLc3WpZ1FqHruPlPPEhFiZ3EQIcVGQoHZkZQehoU7XoVkrdhyhQYXXr0+gW5AX3/6srfH80bZv7V5LVa2FuSv3MKCrH5P7d7H7+YUQQg8S1I6saWiWfkH9/fbDdA/2Yro5nK/uHcY/rxuFoTaSL3Yu5ZH/bSO/rNputcxbl0lBeQ3/N7mPU6zmJYQQ1iBB7cisvA51WxWUV/NrVhGT+4ehKAqKojB1QDj3D74Go8dBvk/fyai5ybydvJ8aS71taymr5t21mVwV35mBUfr3gBdCCHuRoHZkJdlgcAHfCF1Of+K296TTbjNP6j4egHsnVTC8exCvrNjN+H+sY/XufJvV8toPe7E0NPDEBJncRAhxcZGgdmTFWdoa1AZ9Ok0t3X6YXqGd6BXqfcrrEd4RxAXGsbkgmfdvH8T8WYMxGBTunL+FWR9tIvNohVXr2HW4jC9TcrltWDRRgV5WPbYQQjg6CWpHpuPQrPyyajZnFzO5hRWpxkaNZUfRDg5WHGRk7xBW/PEKnr6qD5uzSxj/z3X8bfkuKmosVqnlpWW78HE38eDoHlY5nhBCOBMJakdWkqVbj+9laYdRVbgq/uy9q8dFa5OfrMrW5v52dTFw9xXdWP3YCKYNCOfdtdpwrkUpeTQ0qO2uI3lPAT/tK+TB0T3w85TJTYQQFx8JakdVVQzVpbp1JPt++2FiO3vTI+Ts82h39e5Kn4A+ZyzSEeLtzqszElh8/3DC/Dz401e/ce07G9ied6zNzyWT5wAAIABJREFUNVjqG3hp2S6iAj1lchMhxEVLgtpR6bgYx6Fjx9mSU3LescrjosexvXA7hysOn/GeOdKfxfcN59Xr+pNbXMXUt37miYXbKayoaXUdX6XksTe/gicmxOLqIv+pCiEuTvLbz1HpODRrWZoWvJNaeD59wvnm/jYYFGYM6srqx0Zy12UxLErNY9TcZD5Yn0VdfcM5j11ZY+G1H/YyKMqfif06t+MqhBCiY5CgdlRNLepou5/6+7TDxIX5EBN07h7WkT6RxAbEnneRDh93E09P6suKh6/AHOnPC0vTmfjGT6zfV9jiPu+u3U9hRQ1PT5LJTYQQFzcJakdVnAVeIeBm37WW80qq2Hrg2Bljp1syLmoc249u50jlkfNu2yOkEx/PGsz7tw2i1tLALR/8yu8XbCG3uOqU7YqrG5j3UyaT+3fBHOnfrusQQoiOQoLaUZVk63rbe3L8uW97n9DU+/u0TmUtURSFK/uG8sMjV/D4+N6s21vImNfX8toPe6iq1YZzfb2vjoYGZHITIYRAgtpxlWTrMjTr++2H6R/hS2SgZ6u2j/KJord/7zYvfeluMvLAqB6sfmwEE/t15l+rMxjz2lreTt7Pzwct3HFpNF0DWleDEEJ0ZBLUjshSA6V5du/xnVtcxW95pUxqYex0S8ZGjWXb0W2tuv19ui6+Hrwx08xX9w7D39OVV1bsxssED4ySyU2EEAIkqB3TsVxAtfut76XbT/T2bltQn7j9nZST1O5zD44OYMmDl/H69Qn8weyOr4ep3ccSQoiORILaEZ0YmmXnW9/fpx1iQFc/Ivzbdss5xjeGnv49z9v7+3yMBoVrEv+/vTuPj7K6Fz/+OZN9z2QhQIBsJBP2VZbaKhWMWBVttZa6/LBWsbe2tva27lesVmutt+29alstorUuSJdrVVoVUOrGKvuSCZAESAgkYYaQPZnM+f0xmTREAiF5Zp6Zyff9euWVZDLzPN/H4Hxzvs/3nDOCwhRz1jYXQohAJIk6EDn8P4e6vLaRXZUnz7rISW+KsorYWr2VY42+20FLCCEGI0nUgchZDhGxEJ/ht1Ou7Oz27m1t77PpKn8f6n/5W4hAUdtcy9bGrWaHIQQgiTowOcs8o2k/LvTx9o4qpmVZGZ4c06/X5yblMjp59Dl3fwsRiF7d+yrLapdR29z7ojxC+EufErVSar5Syq6U2q+UuqeX51yrlNqjlNqtlHq12+OLlFL7Oj8WGRV4SHOU+bXsfaCmgb1VJ8+527snb/m7pqnGoMiEMMdex14A7A67yZEI0YdErZQKA54BLgXGAt9USo3t8Zx84F7gfK31OOCHnY+nAEuAmcAMYIlSSpaaOhOt/b4P9codVSjV/7K3V1F2ERrd58VPhAhUJY4SAOxOSdTCfH0ZUc8A9mutS7XWbcBy4Moez7kVeEZr7QTQWld3Pn4JsEpr7ej82SpgvjGhh6iGY+Bq9uuIeuWOKs7LSmFoUvSAjpOXnEdeUp4kahHUHC0Oqps9b2HFjmKToxGib4k6Ezjc7fuKzse6KwAKlFKfKKXWK6Xmn8NrRXfezTj8NDVr37F67Mfqz3nudG+Ksov47Nhncm9PBC1vuTvWEts1shbCTOEGHicfmAOMAD5USk3o64uVUouBxQAZGRmsXbvWoLA8GhoaDD+mr2QcfZ8xwIZ91TRXrj3jc424rv/b14YCkk6WsnZt+YCOBZDcloxG89vVv+WChAv6dYxg+n2dC7mu4LCmbg0AEyImsKluE6s+WEWECp0FeELt99VdqF5bXxJ1JTCy2/cjOh/rrgLYoLVuB8qUUiV4EnclnuTd/bVre55Aa/0c8BzA9OnT9Zw5c3o+ZUDWrl2L0cf0mQ8+hWLFzKJrIDzqjE8d6HVprfnZlg+ZmZvAVfNn9/s4PY+5/O/LKY8s58E5D/brGEH1+zoHcl3B4d2P3iWjLYNxsePY0LqB4ROHMy51nNlhGSbUfl/dheq19aX0vQnIV0rlKKUigYXAmz2e8wadCVkplYanFF4KvAsUKaWsnU1kRZ2Pid44yyFpxFmTtBFKjjWwv7qByyb2baesvlBKUZQl5W8RvIodxRSmFJIZ6blLJ+VvYbazJmqttQv4Hp4EuxdYobXerZR6WCm1oPNp7wLHlVJ7gA+An2itj2utHcAjeJL9JuDhzsdEb/w4NevtHUewKJg/bqihxy3KLsKt3bx/6H1DjyuEr7V2tFJeV06BtYC08DRiwmOk81uYrk/3qLXW/wD+0eOxB7t9rYEfdX70fO0yYNnAwhxEnOVQcInPT6O1ZuWOKmblppKeYOzoPT85n+zEbN4rf49rbdcaemwhfOnAiQO4tIvClEIsJy3kW/NlLrUwnaxMFkhaG6Cx2i8d33ur6imtbeRyA8veXkopLs66mE3HNnG8+bjhxxfCV7xJ2ZZi83y22rA77XjGIkKYQxJ1IPFOzfJD6fvtHUcIsyguGeeb9cQvyb4Et3az5tAanxxfCF+wO+3EhMcwMsHTP2uz2qhvq6eqscrkyMRgJok6kHQlat+OqLXWrNxZxRfyUkmN903TWoG1gKzELFn8RASVYkcxNqsNi/K8NXpH1lL+FmaSRB1I/LQP9e4jJzl4vKnfW1r2hbf7e9PRTThbnD47jxBG0VpT4ijpSs4A+dZ8QJYSFeaSRB1InOUQnQQxvl0O/e0dVYRbFEVjje327unirIvp0B1S/hZB4UjjEerb609J1HERcYxMGEmJU6ZoCfNIog4kjjK/lL3f3nGE80enYY2L9Om5ClMKGZkwUra+FEGhq5HMajvlcZvVJqVvYSpJ1IHE6fs51Dsq6qhwNvu07O3lLX9vPLpRyt8i4NkddizK0lXu9ipIKeBw/WGa2ptMikwMdpKoA4W7A04c8vn96ZU7q4gI833Z26sou4gO3cEHhz/wy/mE6C+7086ohFHEhMec8rjNakOjpfwtTCOJOlDUVYDb5dPSt3eRky/lp5MU659NBsakjGFE/Agpf4uA5106tCfvY5KohVkkUQcKP8yh3nr4BJUnmrlsgu/L3l5KKS7OvpgNVRuoa63z23mFOBf1bfVUNlSe0kjmNSxuGAmRCbI3tTCNJOpA4YepWSt3VBEZZuFiHy1y0ptLsi7BpV2y9rcIWN7Rcs9GMvD8sVlgLZApWsI0kqgDhaMMLBGQmOmTw7vdnrL3BQXpJEb7d2/dsaljyYzP5N2DsnGaCEze0fLpRtTgSeD7nPtwa7c/wxICkEQdOJzlkDwKLGE+OfyWQ06OnmzxS7d3T97u7w1HpPwtAlOJs4SU6BTSY9JP+3Nbio1mVzOH6w/7OTIhJFEHDmeZT8veb++oIjLcwryx/i17e12cdTEu7ZLubxGQvEuHKqVO+3NvSVzmUwszSKIOBFqDo9xnjWQdbs0/dlbxZVs68VF92tnUcOPTxjM8brh0f4uA43K72O/c32vZGyAvOQ+Lssh9amEKSdSBoNkJrXU+m5q1udxBdX0rl/lgS8u+8m59ua5qHSfbTpoWhxA9ldeV0+ZuO2Oijg6PJjsxmxKHTNES/ieJOhB4O759NKJeubOK6AgLcwuH+OT4fVWUXYTL7WLt4bWmxiFEd95R8uk6vruzpdhkRC1MIYk6EHjnUPvgHrWn7H2UiwqHEGdS2dtrQtoEhsYNlfK3CCh2h51ISyTZSdlnfJ7NaqOqsUoaIoXfSaIOBA7fjag3lB2ntqGVyyaYV/b28pa/Pz3yKfVt9WaHIwTgGVHnJecRYTnztEVvaVxWKBP+Jok6EDjLIG4IRMYZfuiVO6qIiQjjy4Wnn3bib0VZRbS726X8LQKC1rrXpUN7ks5vYRZJ1IHAedAnZW9Xh5t3dh1l7pghxEaaW/b2mpg+kYzYDCl/i4BQ21yLo8VxxkYyr7SYNFKiU+Q+tfA7SdSBwEf7UK8vdXC8sc2URU56Y1EWLs66mE+OfEJDW4PZ4YhBrq+NZNBtKVEZUQs/k0RtNlcrnKz0yf3plTuPEBcZxhybud3ePV2SfYmn/F2x1uxQxCDnXTq0IKWgT8+3WW0cOHEAl9vly7CEOIUkarOdOARow0vf7Z1l73ljM4iO8M2ypP01MX0iQ2KHSPlbmK7EUUJmfCaJkYl9er4txUabu43yunLfBiZEN5Kozeajju9PDxzH2dTu1y0t+6qr/F0p5W9hrmJncZ/K3l7ee9lyn1r4kyRqs3XtQ23siHrljiMkRIVzQUFgdHv3VJRVRJu7jX9V/MvsUMQg1exq5uDJg31qJPPKScohwhIhiVr4lSRqsznLICIW4o27j9zmcvPu7mNcHIBlb6/JQyaTHpPOqoOrzA5FDFL7nftxa/c5JeoISwR5yXnSUCb8ShK12ZzlnrJ3L7v29Mcn+2upa27nsgDq9u7JoizMy5rHx5Uf09TeZHY4YhA6l47v7qTzW/ibJGqz+WBq1ts7qkiIDueL+WmGHtdoRVlFtHa0SvlbmKLYUUx8RDyZ8Znn9Dqb1cbxluPUNtf6KDIhTiWJ2kxa/3tEbZBWVwfv7TlK0dihRIUHZtnba8qQKaTFpEn3tzBFibOEAmtBr3tQ96ZrKVHZSUv4iSRqMzUcA1ezoVOzPiqppb7FxeWTArfs7RVmCWPeqHl8VPmRlL+FX7m1G7vD3qelQ3vqWkpUGsqEn0iiNlPX1CzjEvXKnVUkxURwfl5gl729irI95e8PKz80OxQxiFTUV9DkajqnRjKv5OhkhsQOkUQt/EYStZkM3oe6pb2DVXuOccm4DCLDg+NXO3XIVFKjU6X8Lfyqq5GsH4kaPKNqaSgT/hIc7+ahylkOKEgeZcjh/lVSQ0Ori8snmr+lZV+FWcKYlzWPjyqk/C38p9hRTJgKY3Ty6H69vjClkPK6clo7Wg2OTIjPk0RtJkcZJI2A8EhDDrdyRxXW2Ahm56Uacjx/KcoqoqWjhY8qPzI7FDFIlDhKyEnKISosql+vL0gpwKVdHDhxwODIhPg8SdRmMrDju6W9g9V7jzF//FAiwoLr1zotYxop0Smy+Inwm2Jncb/L3iB7Uwv/Cq539FDjLDOs4/uD4mqa2jqCquztFWYJY+6ouXxY8SHNrmazwxEhrq61jqONR895oZPuRiWMIjosmhKnTNESvieJ2iyt9dBYY9iI+u2dVaTGRTIzJ8WQ4/lbUXYRza5mPq782OxQRIjzjoIHMqIOs4SRb82Xzm/hF5KozeI86PlswNSspjYX7++tZv74oYQHWdnba3rGdKxRVun+Fj7X36VDe/IuJaq1NiIsIXoVnO/qocDAqVnvF1fT3N4R0Gt7n024JZy5WXP5V8W/aHO3mR2OCGHFjmLSY9JJjRlY06UtxcbJtpMcazpmUGRCnJ4karN4t7c04B71yh1VpMVHMTMnuLq9eyrK8pS/97bsNTsUEcJKnCUUpBQM+DjSUCb8RRK1WRxlEJ0MMdYBHaax1cX7xdV8ZcJQwizG7cBlhvOGnkdyVDIbGjaYHYoIUe0d7ew/sZ9C67kvHdpTgdWT7OU+tfA1SdRmcZYZUvZeU1xNq8vNZROCt+ztFW4J57rC69jZvJP1VevNDkeEoNK6Ulxu14AaybziI+MZET+CYkexAZEJ0TtJ1GZxlhtS9n57+xGGJERxXnZwdnv39K3x3yItPI1H1z9KW4fcqxbGGujSoT3ZUmwyRUv4nCRqM3S44MShAXd8N7s0a0tq+MqEYViCvOztFR0ezTUp11B+spyX9rxkdjgixBQ7iokOiyYrIcuQ49msNg6dPCTL3wqfkkRthpOV4HYNuPS9tbqDNpeby4O42/t0xsWM46KRF/Hs9mc50nDE7HBECClxlJBvzSfMYsxe7QUpBWg0+07sM+R4QpyOJGozeKdmDbD0vbHKxbCkaKaOGlhDWiC6e8bdAPxi4y9MjkSECq01xc7iriYwI0jnt/AHSdRmcAx8DnVdczu7ajtCquzd3fD44dw26TbeP/w+H1bIXtVi4I41HaOutY7ClIF3fHtlxmcSHxEv96mFT0miNoOzHCwRkJjZ70Os3nMMlyaoFzk5m0VjF5GdmM3PN/ycFleL2eGIIOcd9RqZqJVSXSuUCeErkqjN4CwDaxYM4D7Zm9uPkBqtmDIy2cDAAktEWAT3z7qfioYKlu1aZnY4Ish5O77zrfmGHrfAWkCJswS3dht6XCG8JFGbYYDbW/527X7+VVLDl0aEo1Tolb27mzVsFpdmX8rzO5/n8MnDZocjglixo5hRCaOIi4gz9LiFKYU0uZqoqK8w9LhCeEmi9jetwVHe76lZf/y0nCfesXPFpOEsyIswNrYA9ePzfkxEWASPbXxMNkAQ/WZ32A2bP92d95iyQpnwFUnU/tbshNa6fo2oV2w+zJI3dzNvTAa/unYSlhAfTXsNiR3Cdyd9l48rP+b9Q++bHY4IQo3tjRyuPzzgHbNOZ3TyaCzKIvephc9Iova3fk7Nemv7Ee756w6+lJ/G09dNISJIt7Psr+vGXEe+NZ/HNz0ui0uIc7bPuQ+N9smIOjo8mqzELBlRC58ZXO/2gaBralbfE/WqPce48/VtTM9K4bkbpxMdYcxiDcEk3BLOAzMf4GjjUZ7b8ZzZ4Ygg44uO7+5sVhslDpmiJXxDErW/de1D3bclDD/eV8vtr2xh7PBEnr9pOjGRgy9Je03NmMqCvAX8cfcfKT1RanY4IogUO4tJjEwkIzbDJ8e3pdg40niEk20nfXJ8MbhJovY3ZznEZ0Dk2TtPN5U7uPWlzeSmx/HSzTNIiB4czWNn8qNpPyImIoZHNzwqjWWiz0ocJRSmFPpsloR3tTMZVQtfkETtb47yPjWS7ag4wc0vbGJYUjR/+vZMkmMjfR5aMEiNSeWOKXew8ehG/ln2T7PDEUGgw91BibPE0KVDe+paSlTuUwsfkETtb87ys96fth+t5/8t20hiTAQv3zKT9IQo/8QWJL5e8HXGpo7lyc1P0tDWYHY4IsAdqj9ES0eLz+5Pg2dmgjXKKp3fwickUfuTq9Wzc9YZOr7Lahu5fukGIsMsvHrrTIYnx/gxwOAQZgnjgZkPUNtcyzPbnjE7HBHgfN1IBp1LiaYUyIha+IQkan9yHgR0r6XvCmcT1/9hPW6teeWWmWSlGruCUiiZkD6Bawqu4bXi12QUI87I7rQTbgknNynXp+exWW3sd+7H5Xb59Dxi8JFE7U/Ocs/n05S+q0+2cP3SDdS3unjp5hnkZyT4N7Yg9IOpPyAxMpFHNzwq6yyLXhU7islLyiMizLfNmLYUG23uNg6ePOjT84jBRxK1P/Wy2ImjsY3rl26gpr6VF781g/GZSSYEF3ySopK4c9qdbK3eypsH3jQ7HBGgfLV0aE+yN7XwFUnU/uQsh4g4iEvveqiuuZ0bn9/AIUcTSxdNZ1qW1bz4gtCVo69kUvokfv3Zr6lrrTM7HBFgjjcfp6a5xidLh/aUm5RLuCVc7lMLw0mi9idHmef+dOdczsZWFze/uImSY/X8/oZpfCEvzdz4gpBFWXhg1gOcaD3BU1ufMjscEWC8SdMfI+qIsAhyk3IlUQvDSaL2J2dZVyNZS3sHt760ma2HnPzvwil8uXCIubEFscKUQr5Z+E1W2Fewu3a32eGIAOJdgMQfI2rveWTRE2E0SdT+orWn9J2SQ5vLzXdf2cKnB47z5NcncemEYWZHF/Run3w7qTGp/Gz9z+hwd5gdjggQxc5iMmIzSI5O9sv5bCk2apprON583C/nE4ODJGp/qT8Krhbcydnc+fo23i+u5mdXjedrU0eYHVlISIhM4D+n/ye7ju/ir/v+anY4IkDYHXafzp/uSfamFr4gidpfOqdmLd2tWbmzivu/MoYbZvVtYw7RN5flXMZ5Q8/jf7b8D44Wh9nhCJO1drRSVlfm06VDe/KW2KX8LYwkidpPtMOz29Or+yz8cF4+t17g28UXBiOlFPfPvJ+m9iZ+/dmvzQ5HmOzAiQN06A6/jqit0VaGxAyREbUwlCRqP9Ba88mmzXRoxfzzZ/CDuflmhxSy8pLzuHHcjbyx/w22Vm81OxxhIn8sHXo6spSoMFqfErVSar5Syq6U2q+Uuuc0P79JKVWjlNrW+XFLt591dHt8UK5K8dT7+6k5bOdkZAZ3XzbBZ1vtCY/vTPwOGbEZ/Gz9z2Q5x0HM7rQTGx7LiAT/9oHYrDbKTpTR1tHm1/OK0HXWRK2UCgOeAS4FxgLfVEqNPc1TX9daT+78WNrt8eZujy8wJuzgsfSjUn61qoQp8U6SMwskSftBbEQsd8+4mxJnCcuLl5sdjjBJsaOYAmsBFuXfwqEtxYZLuyitK/XreUXo6su/4BnAfq11qda6DVgOXOnbsELDqxsO8bOVe7l0/FCyVDUqJdvskAaNeaPmcf7w83l629PUNNWYHY7wM601JY4Svyx00pMsJSqMFt6H52QCh7t9XwHMPM3zrlZKXQCUAHdqrb2viVZKbQZcwONa6zd6vlAptRhYDJCRkcHatWv7fgV90NDQYPgxz+bTIy7+sKOVielhXDvkGGp/LaVOzSED4zDjuvzBqOu6iIvY0L6Bu1bexaL0RQMPbIDk9+U/x13HqW+vR9WofsfW3+vq0B1EqAhW71hNUkXgrdsfiL8vo4TstWmtz/gBXAMs7fb9jcDTPZ6TCkR1fn0b8H63n2V2fs4FyoG8M51v2rRp2mgffPCB4cc8k3/uPKJz712pFz67Tje3ubSu2qH1kkStd/7V0PP4+7r8xcjremrLU3r8i+P1hiMbDDtmf8nvy39WH1ytx784Xm+v3t7vYwzkuha+tVDf/M7N/X69LwXi78sowXxtwGbdS17sS+m7EhjZ7fsRnY91T/bHtdatnd8uBaZ1+1ll5+dSYC0wpY9/QwSltfZqvv/aViaNSGLpoulER4T9e3vLlM9vbyl865YJt5AZn8mjGx6lvaPd7HCEn5Q4SrAoC/lWc2ZY2FJs2J1272BFiAHpS6LeBOQrpXKUUpHAQuCU7m2lVPc1MBcAezsftyqlojq/TgPOB/YYEXggWl96nNv+9Bn5QxJ44VsziIvqvLPg6Nze8jT7UAvfig6P5t4Z91JaV8qf9v7J7HCEnxQ7ihmVMIqY8BhTzl9gLaCutY5jTcdMOb8ILWdN1FprF/A94F08CXiF1nq3UuphpZS3i/sOpdRupdR24A7gps7HxwCbOx//AM896pBM1OsOHOfmFzcxMiWWP317Bkkx3Tapd5ZBdDLE+Ge9YXGqC0deyJyRc/j99t9ztPGo2eEIP7A7/bt0aE/eJrYSp6xQJgauT/MWtNb/0FoXaK3ztNaPdj72oNb6zc6v79Vaj9NaT9Jaf1lrXdz5+Kda6wmdj0/QWj/vu0sxz1p7NTe9sJHM5BhevWUmqfFRpz6hczMOYZ57ZtyD1ponNj1hdijCx+rb6qlsqDSl49vLu2ypdH4LI8jKZAP0zq6j3PrSZvLS41m+eBZDEqM//yRHmZS9TZYZn8niiYtZdXAVH1d+bHY4woe8o1h/bW15OgmRCWTGZ8oKZcIQkqgH4O/bKrn91S2MG57Ea7fO+vxIGqDDBXWHu/ahFuZZNG4R2YnZPLbhMVo7Ws/+AhGUih3FgP+XDu2pwFogI2phCEnU/fT6pkP88PVtTM+y8vItM0mKjTj9E09WgNslpe8AEBkWyX0z7+Nw/WGW7VpmdjjCR+wOOynRKaTFpJkahy3FxsGTB2lqbzI1DhH8+rLgiejhxU/KeOitPVxQkM6zN0wjJjKs9yd3dXxn+yU2cWazh8/mkuxLeH7n81yeezkjE0ae/UV90NjeSG1z7Rk/IpojGHViFLnJsnOaL9mddmxWm+nL9RZaC9Fo9p/Yz8T0iabGIoKbJOpz9Lu1B/jFO8UUjc3gqeumEBV+hiQN/55DLfeoA8ZPpv+Ejyo+4vGNj/P0RU/3+obucrtwtDjOmoBrm2tpdjV/7vXhKpyUGM/ILj0mnc3OzVz95tV8c8w3+c6k75AYmejrSx10XG4X+537uW7MdWaHQkFKZ0OZ0y6JWgyIJOo+0lrz69X7+N81+7hi0nB+de0kIsL6cOfAWQZhkZA43PdBij7JiMvgu5O/y5Obn+TZHc8SFxF32uTrbHGi+fyCFQmRCV3Jd3zaeNJi0rq+T41J7fo6KSrplA0h3lrzFltit/DynpdZWbqSO6bcwVWjryLMcpY/9kSfldeV0+ZuM7Xj2yszPpO4iDi5Ty0GTBJ1H2iteewfe/nDR2VcO30EP//aRMIsfSyrOcsheRTIm3FAuW7Mdfz9wN95ZtszAIRbwrsS7PD44UxMn3ja5Jsak0pU2GmaBvsgISyBJbOXcG3BtTy+8XEeWvcQr9tf596Z9zJlSEgv2Oc3xU5PI5mZHd9eFmWhwFogc6nFgEmiPgu3W/Pgm7t4ef0hFs3OYskV47D0NUmDTM0KUBGWCF6c/yLVjdWkx6aTGJnot3uaY1LH8OL8F3mn/B2e3Pwk/++f/4+v5HyFH037ERlxGX6JIVSVOEqItESSnZRtdiiAp/P77dK3cWu337fbFKFD/uWcQYdbc9dfd/Dy+kPcdmEuDy04xySttWdELY1kASkxMpHR1tEkRSX5vfFIKcWlOZfy1lVvsXjiYlYfXM0Vb1zBczuek6ljA1DsKCYvOY8ISy+zMPzMlmKjsb2RyobKsz9ZBIUTLSdY9M9F7KzZ6bdzSqLuRXuHmx8s38pfPqvgznkF3DO/8NzfzJud0HpSpmaJXsVGxPL9Kd/n71f9nfOHn89TW5/iyjeuZM2hNbKhwznSWpu+dGhP3hJ8iUPK36Hil5t/yY6aHUSGRfrtnJKoT6OlvYP/eHkLb++o4r6vFPKDefn9G3HJZhyij0YkjODXX/41fyj6AzHhMfzwgx+yeNViDpw4YHZoQaO2uRZHiyMgGsm8RiePRqFkhbIQse7IOt488CbfGv8tv/47k0TdQ3PubUCEAAAZZklEQVRbB7e+tJnVe4/xyJXjWHxBXv8P5pQ51OLczBo2iz9f8WfunXEvu4/v5uo3r+bxjY9T11pndmgBz5sMA6GRzCs2IpasxKyu1dJE8Gp2NfPwuofJSszitkm3+fXckqi7qW9pZ9GyjXyyv5YnrpnIjbOzB3ZASdSiH8It4Vw35jpWfnUlV+dfzWvFr3HF/13BCvsKOtwdZocXsLzJMJBG1OCJRzq/g9/vtv2OioYKlsxe0u+ZH/0librTiaY2bnh+I58dcvKbhVO4droBK1Y5yiE+AyJjB34sMehYo6381+z/YsXlK8hNzuWR9Y+wcOVCPjv2mdmhBSS7w05mfCYJkQlmh3IKm9VGZUMl9W31Zoci+mnv8b28tOclrs6/mvOGnuf380uiBmobWvnmHzaw98hJfnf9VBZMMmhxEme53J8WA2ZLsfHCJS/wywt/yYnWE9z0zk3c9a+7TN1bu8PdweH6w3xY8SGlLaWmxdGdd+nQQCN7Uwc3l9vFkk+XkByVzJ3T7jQlhkE/j/rYyRau+8N6KpzN/GHRdC4sSDfu4M4yyLnAuOOJQUspxfzs+Vw44kKW7VrGC7teYG3FWm4efzM3jbuJ6PDTbK9qgNaOVsrryimrK6OsrozSulLK6sooP1neNY3MgoUvOL5gard1s6uZgycPMj97vmkx9Kb73tTTMqaZHI04V6/sfYW9jr08eeGTJEUlmRLDoE7UFc4mrl+6gdr6Vv548wxm5aYad/D2Fjh5RO5PC0PFhMdw++Tb+eror/Lk5id5ZtszvLH/DX48/cfMHTW33/PB61rrTknEpXWllJ4opbKhsmsZVYUiMz6TnKQcZg2bRW5yLpnxmfxozY+4/+P7WX7ZciLCzJm/vN+5H7d2B9z9aYCM2AySopJkRB2EDtcf5umtTzNn5ByKsopMi2PQJuqy2kau/8N6Glpd/OmWmUwdZTX2BCcOAVpK38InhscP51dzfsXGqo08vulx7lx7JzOHzuSuGXd1jeB60lpztPHoKcnY+9nR4uh6nndlr3Fp47gi7wpyk3LJScohKzHrtCP3hSkLea7mOZ7b+Ry3T77dZ9d8JoG0dGhPSilsVpus+R1ktNY8su4Rwixh3D/zflN3YxuUibrkWD3XL91Ah1vz2uJZjBvug3KGt+NbFjsRPjRj2AxWXL6Cv5T8hae3Pc3X3/o61xZcy9fyv0ZFQ8Upo+Pyk+Wn7PKVGJlIblIuc0bOIScxh9xkT0IeHjf8nDYKmRA7gctzL2fpjqVcNPIixqSO8cWlnpHdYSc+Ip7M+Ey/n7svCqwF/Lnkz7jcLsItg/JtN+i8Xfo266rWcd/M+xgaN9TUWAbdv5hdlXXc+PwGIsIsvL54FvkZPuoQ7dreMts3xxeiU7glnIWFC5mfPZ9ntj3DipIVLLcv7/r5sLhh5CTlMC1jGjlJOeQk5ZCblEtKdIpho4R7ZtzD+qr1PPDJA6aUwO0OOwXWAtP3oO6NLcVGa0crh04ekv3Ig4CjxcETm55gUvokvmH7htnhDK5EveWQk0XLNpIYHcErt8wkOy3OdydzlEFEHMQZ2JwmxBkkRydz/6z7WVi4ELvDTlZSFjmJOcRG+H56YFJUEktmL+H773/f7yVwt3ZT4izhqtFX+e2c58rbaGd32iVRB4EnNj1BQ3sDD81+KCA2UzE/Aj9ZX3qcG5duICUuktdvm+XbJA2e0rc1GwL0L3wRuvKS8/hK7lcYlzrOL0naa87IOVyRewVLdyxl7/G9fjtvRX0FTa6mgGwk88pNyiVchct96iDwSeUnrCxdyS0TbmG0dbTZ4QCDJFHvrHGxaNlGhiXHsOK22Yyw+uHNy1ku96fFoHP3jLuxRlt54JMHaO9o98s5u5YODeBEHRkWSU5yjqz5HeCa2pt4ZP0j5CTlcOuEW80Op0vIJ+r3dh/lf7a0kpcez+uLZ5GR6Jv5pqdwu2V7SzEoJUUl8eDsBylxlvDczuf8cs5iRzFhKozRyYEx+umNzWqTXbQC3DPbnqGyoZKHZj/k192xziakE/XRuha+99pWRiVaeO3WWaTG+2l91oZj4GqRRC0GJX+XwO0OOzlJOX5ff/lc2aw2qpurcbY4zQ5FnMau2l28vPdlri24lqkZU80O5xQhnaiHJkXz7A3T+Ml50STF+rELVaZmiUHOnyVwu9Me0GVvr4KUzhXKpPwdcNrd7Tz06UOkRafxw2k/NDuczwnpRA3w5cIhxIT7uaFL9qEWg1z3EvizO5712XnqWus42ng0IBc66ckbozSUBZ6Xdr+E3Wnnvpn3BdymLjAIErUpnOWgLJBkwA5cQgSprhL4zqXsOb7HJ+fwJr1gGFGnxqSSFpMmiTrAHDp5iN9t/x1zR81lbtZcs8M5LUnUvuAsg6QREB44zQhCmOHuGXeTEp3Cf33yXz4pgXftQR0EI2rw/EEhpe/AobXm4XUPE2GJ4L6Z95kdTq8kUfuCdHwLAfx7IRRflcDtTjvpMemkxhi4oY4P2aw2SutK/TZ1TZzZG/vfYMPRDdw57U6GxA4xO5xeSaL2BUeZ3J8WotOFIy9kQd4Cn5TA7Q57V5NWMLBZbbjcLkrrAmMP78GstrmWJzc/ydQhU7mm4BqzwzkjSdRGa62HploZUQvRzV3n3UVKdIqhXeDtHe0cqDtAodW8fbDPlfdeupS/zfeLjb+g2dXMki8sCYhlQs8ksKMLRt7NOGRqlhBdvCXwfc59hpXAS+tKcbldQdFI5pWVmEWkJVIaykz2YcWHvFP+DosnLiY3KfDXXpdEbTSZmiXEaRldAg+GpUN7CreEM9o6WkbUJmpsb+SR9Y8wOnk03x7/bbPD6RNJ1EbzLnYipW8hPueu8+4iNTrVkBJ4saOY6LBoshKyDIrOP7xLiWqtzQ5lUHpq61McazzGktlL/L4da39JojaasxxirBCTbHYkQgScpKgklnzBUwL//Y7fD+hYdoedfGs+YZYwg6LzD1uKDWerk+qmarNDGXS212zn1b2v8g3bN5g8ZLLZ4fSZJGqjSce3EGd0wYgLWJC3gOd3Pt/vErjWOmiWDu2pwCpLiZqhvcOzTOiQ2CH8YOoPzA7nnEiiNprMoRbirAZaAj/WdIy61rqgWeikO+8fFyVO2UnLn17Y/QL7T+zngVkPEB8Zb3Y450QStZE6XFB3WDq+hTiLgZbAvV3ThSnBMzXLKzEykeFxw6Xz24/K6sp4dvuzFGUVMWfkHLPDOWeSqI1UdxjcLhlRC9EH3Uvgu4/vPqfXepcOzbfm+yI0nytIKZDSt5+4tZufrvspUeFR3DvzXrPD6RdJ1EbyzqGWe9RC9ElXCfzjB2jraOvz6+xOO6MSRhEXEefD6HzHZrVx8ORBWlwtZocS8v627298duwzfjz9x6TFpJkdTr9IojaS7EMtxDnxlsD3n9h/Tguh2B3B2UjmZUux4dZu9p/Yb3YoIa2mqYZfbf4V5w09j6+O/qrZ4fSbJGojOcshLBIShpkdiRBB41xL4I3tjRyuPxyUjWResje1f/x8489p7WhlyewlKKXMDqffJFEbyVEGyVkQZPM6hTDb3TPu7nMJfJ9zHxod1CPqEQkjiA2P7brXLoz3/qH3WXVwFd+Z9B2yEoNrUZyeJFEbyVkmjWRC9ENiZGJXCfz328/cBe5NbsHY8e1lURbyrfkyRctHGtoaeHTDo+Rb87lp/E1mhzNgkqiN4iiF2v2Qmmd2JEIEpQtGXMCVeVeybNcydtf2XgK3O+0kRiaSEZvhx+iMV5hSSIlTlhL1hd9s+Q01TTX8dPZPibAExzKhZyKJ2ghNDnjl6xARAzMWmx2NEEHrrhn/XgiltxK43WGnMKUwqO85gmeFsob2BiobKs0OJaRsrd7KCvsKrh9zPRPSJ5gdjiEkUQ+UqxWWXw8nDsHCV2VELcQAnK0E3uHuYJ9zX1Dfn/YyY2/qZlczFW0VITstrF17lgkdGjeU70/5vtnhGCbc7ACCmtbw99vh0Kdw9fOQNdvsiIQIet1L4HNHzWVc2riunx2sP0hLR0tQd3x75Sfno1CUOEqYO2quz87T0NbAhxUfsvrQaj6u/JhmVzP//dp/MyZlDJPSJzFpyCQmp09maNxQn8XgL6vqVlFaV8pv5/6W2IhYs8MxjCTqgfjgMdj5Z7joAZhwjdnRCBEy7ppxF+uOrOOBTx7g9ctfJzIsEoASh6f5KpgbybxiI2IZlTjKJyPqEy0n+ODwB6w+tJp1R9bR7m4nLSaNBXkLiKqNImJYBNtqtvHnkj/z8t6XAciIzWDykMlMTp/MpPRJFKYUBs02kAAHThzgvbr3uDTnUr404ktmh2MoSdT9te1V+PAJmHIDfOnHZkcjREjxlsBvX3M7v9/+e+6Yegfg6fgOt4STm5RrcoTGKLAWsPf4XkOOVdNU45mSdGgVm49upkN3MCxuGAsLFzJv1DwmpU8izBLG2rVrmTNtDgDt7nZKHCVsq9nGtuptbK/Zzrvl7wIQFRbFuNRxXSPuSemTSI1JNSRWIzW0NVDVWMXD6x4myhLF3efdbXZIhpNE3R9lH8Kbd0DOhXD5byDIm1qECESnK4HbnXbykvKCaqR3JjarjVUHV9HQ1tCvHZ0qGypZfXA1aw6tYVv1NjSa7MRsvjX+W8zLmsfYlLFnbLqLsEQwLm0c49LGcf2Y6wE41niM7TXb2VbjSdx/2vMnXnC/AMDIhJFMSvck7slDJjM6ebRP9wNv72jnaNNRjjb++6OqsYqqxqqu7xvaG7qef0PqDQH5x8RASaI+VzV2WH6Dp2ns2pcgRN4whAhEd824i3VV/y6B2x12Zg8PnV6Q7lteTs2Y2qfXlNWVsfrgalYdXMVeh2c0brPa+I/J/8HFoy4mLzlvQB3xGXEZFMUVUZRdBEBrRyt7ju9he7Unea87so63S98GIDY8lgnpE7qS98T0iSRFJfXpPFprjrccPyUBd/98tPEotc21aE6dvmaNsjI0bigjE0YyY+gMhsYNZVjcMHKScqjaUdXv6w5kkqjPRUM1vHINhEfBdSsgJtnsiIQIaYmRiTw0+yG+u+a7PLbhMWqaa0KikcyraylRp73XRK21xu60s/rgalYfXM2BugMATEybyI+m/Yi5o+YyKnGUz2KMCotiypApTBkypSueyoZKz4i7ejvba7azdOdS3NoNQG5SLpOHeErlBdYCTrSe+FwCrmqs4ljjMdrcp07Biw6LZmjcUIbGDeWLmV9kWNywru+HxQ0jIy6DmPCYXmOtQhL14NbWBK8thIYa+NZKsAb3knRCBIsvjfgSV42+ir/u+ytASEzN8hoaN5TEyMTPrfnt1m521u5kzcE1rDq4ioqGCizKwtQhU7nHdg9zR801rUtbKcWIhBGMSBjB5bmXA9DU3sSu2l1d5fI1h9bwt31/O+V1FmUhPSadYXHDGJc6jnmj5pERl8GwuGFdCTk5Kjno58f7giTqvnC74f8WQ+UW+MbLkDnN7IiEGFR+ct5P+PTIp1Q3VYfUiFophS3FRomzhA53B1uqt7Dq4CrWHFpDdVM14SqcmcNm8u0J3+bLI78csPdfYyNimTFsBjOGzQA8f2iUnyyn9EQpKdEpDIsbRlpsWkisEmYGSdR9sfpB2PsWXPIYjLnc7GiEGHQSIxP57wv/m/VV60mODq1bTjarjeX25Vz054twtDiICovi/OHnM2/qPC4ceSGJkYlmh3jOLMpCblJuyHTnm00S9dlseh4+fQrOuxVmfdfsaIQYtCYP8XQah5ovZn6Rf5b9kxlDZzAvax5fzPxiSC3WIQZOEvWZ7FsF//gx5F8C8x+XaVhCCMOdn3k+a7+x1uwwRACTtb57c3Qn/PkmyBgH1yyDMPmbRgghhP9Joj6dk0fglWshKtEzDSvq3BciEEIIIYwgw8SeWuvh1Wuh9STc/A4kDjc7IiGEEIOYJOruOlzwl5vh2B7PSHpoaOxlKoQQInhJovbSGt65G/a9B5f/GvLnmR2REEIIIfeou6x7BjYthS98H6bfbHY0QgghBCCJ2mPvW/DeAzBmAcx72OxohBBCiC6SqCs+g7/e6lkW9GvPgUX+kwghhAgcgzsrOQ/Ca9+A+CHwzeUQ0fuuLEIIIYQZBm8zWfMJeOXr0NEGN62E+HSzIxJCCCE+Z3AmalcbrLgRHKVw498gPXR24xFCCBFa+lT6VkrNV0rZlVL7lVL3nObnNymlapRS2zo/bun2s0VKqX2dH4uMDL5ftIa3fwhlH8KC/4WcC8yOSAghhOjVWUfUSqkw4BngYqAC2KSUelNrvafHU1/XWn+vx2tTgCXAdEADn3W+1mlI9P3x0ZOw7RW48G6YfJ1pYQghhBB90ZcR9Qxgv9a6VGvdBiwHruzj8S8BVmmtHZ3JeRUwv3+hGmDnX+D9n8HEb8Cce00LQwghhOirviTqTOBwt+8rOh/r6Wql1A6l1F+UUiPP8bW+d/BTeOM/IOt8WPCUbFkphBAiKBjVTPYW8JrWulUpdRvwR+Civr5YKbUYWAyQkZHB2rVrDQrLQ9fuo/3jn9IemcaWEf+B6+N1hh7fLA0NDYb/twoEcl3BRa4ruITqdUHoXltfEnUlMLLb9yM6H+uitT7e7dulwBPdXjunx2vX9jyB1vo54DmA6dOn6zlz5vR8Sv81Hqfp6e8QERlJxC0r+WJKrnHHNtnatWsx9L9VgJDrCi5yXcElVK8LQvfa+lL63gTkK6VylFKRwELgze5PUEoN6/btAmBv59fvAkVKKatSygoUdT7mH+0tsPw6oltqYeFrEEJJWgghxOBw1hG11tqllPoengQbBizTWu9WSj0MbNZavwncoZRaALgAB3BT52sdSqlH8CR7gIe11g4fXMfptTeBdrN3zJ2MGzXTb6cVQgghjNKne9Ra638A/+jx2IPdvr4XOG0btdZ6GbBsADH2X2wK3PwONR9+ZMrphRBCiIEK/bW+LWFmRyCEEEL0W+gnaiGEECKISaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKASaIWQgghApgkaiGEECKAKa212TGcQilVAxw0+LBpQK3BxwwEcl3BRa4ruMh1BZ9gvrYsrXX66X4QcInaF5RSm7XW082Ow2hyXcFFriu4yHUFn1C9Nil9CyGEEAFMErUQQggRwAZLon7O7AB8RK4ruMh1BRe5ruATktc2KO5RCyGEEMFqsIyohRBCiKAU0olaKTVfKWVXSu1XSt1jdjxGUEqNVEp9oJTao5TarZT6gdkxGUkpFaaU2qqUetvsWIyklEpWSv1FKVWslNqrlJptdkxGUErd2fnvcJdS6jWlVLTZMfWHUmqZUqpaKbWr22MpSqlVSql9nZ+tZsbYH71c1y87/x3uUEr9n1Iq2cwY++N019XtZ/+plNJKqTQzYvOFkE3USqkw4BngUmAs8E2l1FhzozKEC/hPrfVYYBZwe4hcl9cPgL1mB+ED/wO8o7UuBCYRAteolMoE7gCma63HA2HAQnOj6rcXgfk9HrsHWKO1zgfWdH4fbF7k89e1ChivtZ4IlAD3+jsoA7zI568LpdRIoAg45O+AfClkEzUwA9ivtS7VWrcBy4ErTY5pwLTWVVrrLZ1f1+N5w880NypjKKVGAJcBS82OxUhKqSTgAuB5AK11m9b6hLlRGSYciFFKhQOxwBGT4+kXrfWHgKPHw1cCf+z8+o/AVX4NygCnuy6t9Xtaa1fnt+uBEX4PbIB6+X0B/Bq4Cwip5qtQTtSZwOFu31cQIgnNSymVDUwBNpgbiWF+g+d/MrfZgRgsB6gBXugs6y9VSsWZHdRAaa0rgSfxjF6qgDqt9XvmRmWoDK11VefXR4EMM4PxkZuBf5odhBGUUlcClVrr7WbHYrRQTtQhTSkVD/wV+KHW+qTZ8QyUUupyoFpr/ZnZsfhAODAV+J3WegrQSHCWUU/Rec/2Sjx/iAwH4pRSN5gblW9oz/SYkBqlKaXux3Mr7RWzYxkopVQscB/woNmx+EIoJ+pKYGS370d0Phb0lFIReJL0K1rrv5kdj0HOBxYopcrx3Ka4SCn1srkhGaYCqNBaeysff8GTuIPdPKBMa12jtW4H/gZ8weSYjHRMKTUMoPNztcnxGEYpdRNwOXC9Do05unl4/mDc3vkeMgLYopQaampUBgnlRL0JyFdK5SilIvE0ubxpckwDppRSeO517tVa/8rseIyitb5Xaz1Ca52N53f1vtY6JEZnWuujwGGllK3zobnAHhNDMsohYJZSKrbz3+VcQqBJrps3gUWdXy8C/m5iLIZRSs3Hc4tpgda6yex4jKC13qm1HqK1zu58D6kApnb+vxf0QjZRdzZLfA94F8+bxwqt9W5zozLE+cCNeEac2zo/vmJ2UOKsvg+8opTaAUwGHjM5ngHrrBD8BdgC7MTzfhKUK0MppV4D1gE2pVSFUurbwOPAxUqpfXiqB4+bGWN/9HJdTwMJwKrO94/fmxpkP/RyXSFLViYTQgghAljIjqiFEEKIUCCJWgghhAhgkqiFEEKIACaJWgghhAhgkqiFEEKIACaJWgghhAhgkqiFEEKIACaJWgghhAhg/x+1y/KN3paPfwAAAABJRU5ErkJggg==\n",
696 |             "text/plain": [
697 |               "<Figure size 576x576 with 1 Axes>"
698 |             ]
699 |           },
700 |           "metadata": {
701 |             "needs_background": "light"
702 |           }
703 |         }
704 |       ]
705 |     },
706 |     {
707 |       "cell_type": "code",
708 |       "metadata": {
709 |         "id": "suv_B7cOvYq8",
710 |         "colab": {
711 |           "base_uri": "https://localhost:8080/"
712 |         },
713 |         "outputId": "63ed1bbf-4905-4cfb-9654-ac44a6a76a54"
714 |       },
715 |       "source": [
716 |         "model_path = f'{drive_path}models/{NAME}'\n",
717 |         "model.save(model_path)\n",
718 |         "\n",
719 |         "model = keras.models.load_model(model_path)\n",
720 |         "score = model.evaluate(test_x, test_y, verbose=10)  # evaluate against unseen test data\n",
721 |         "print(\"TEST LOSS:\", score[0])\n",
722 |         "print(\"TEST ACCURACY:\", score[1])"
723 |       ],
724 |       "execution_count": 8,
725 |       "outputs": [
726 |         {
727 |           "output_type": "stream",
728 |           "text": [
729 |             "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
730 |           ],
731 |           "name": "stderr"
732 |         },
733 |         {
734 |           "output_type": "stream",
735 |           "text": [
736 |             "INFO:tensorflow:Assets written to: drive/MyDrive/Colab Notebooks/crypto_prediction/model3/models/LSTM-2021-08-23 20-05-44/assets\n"
737 |           ],
738 |           "name": "stdout"
739 |         },
740 |         {
741 |           "output_type": "stream",
742 |           "text": [
743 |             "INFO:tensorflow:Assets written to: drive/MyDrive/Colab Notebooks/crypto_prediction/model3/models/LSTM-2021-08-23 20-05-44/assets\n"
744 |           ],
745 |           "name": "stderr"
746 |         },
747 |         {
748 |           "output_type": "stream",
749 |           "text": [
750 |             "TEST LOSS: 0.6933889985084534\n",
751 |             "TEST ACCURACY: 0.5228310227394104\n"
752 |           ],
753 |           "name": "stdout"
754 |         }
755 |       ]
756 |     },
757 |     {
758 |       "cell_type": "code",
759 |       "metadata": {
760 |         "id": "xd9UqtJ1hmSD"
761 |       },
762 |       "source": [
763 |         "\n"
764 |       ],
765 |       "execution_count": null,
766 |       "outputs": []
767 |     }
768 |   ]
769 | }


--------------------------------------------------------------------------------
/training-google-colab/model3_kfold.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |   "nbformat": 4,
  3 |   "nbformat_minor": 0,
  4 |   "metadata": {
  5 |     "colab": {
  6 |       "name": "model3_kfold.ipynb",
  7 |       "provenance": [],
  8 |       "collapsed_sections": []
  9 |     },
 10 |     "kernelspec": {
 11 |       "name": "python3",
 12 |       "display_name": "Python 3"
 13 |     },
 14 |     "language_info": {
 15 |       "name": "python"
 16 |     },
 17 |     "accelerator": "GPU"
 18 |   },
 19 |   "cells": [
 20 |     {
 21 |       "cell_type": "code",
 22 |       "metadata": {
 23 |         "id": "sfwZbBSN6dnu"
 24 |       },
 25 |       "source": [
 26 |         "from sklearn import preprocessing\n",
 27 |         "from sklearn.model_selection import KFold\n",
 28 |         "from collections import deque\n",
 29 |         "from datetime import datetime\n",
 30 |         "import random\n",
 31 |         "import time\n",
 32 |         "import numpy as np\n",
 33 |         "import pandas as pd\n",
 34 |         "import tensorflow as tf\n",
 35 |         "from tensorflow import keras\n",
 36 |         "from tensorflow.keras.models import Sequential\n",
 37 |         "from tensorflow.keras.layers import Dense, Dropout, LSTM, BatchNormalization\n",
 38 |         "from tensorflow.python.keras.layers import CuDNNLSTM\n",
 39 |         "from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping\n",
 40 |         "\n",
 41 |         "drive_path = f'drive/MyDrive/Colab Notebooks/crypto_prediction/model3/'"
 42 |       ],
 43 |       "execution_count": null,
 44 |       "outputs": []
 45 |     },
 46 |     {
 47 |       "cell_type": "code",
 48 |       "metadata": {
 49 |         "id": "SYa7ACGv6xuU"
 50 |       },
 51 |       "source": [
 52 |         "pd.set_option('display.max_columns', None)\n",
 53 |         "pd.set_option('display.expand_frame_repr', False)\n",
 54 |         "\n",
 55 |         "# ratios = [\"BTCUSDT\", \"ETHUSDT\", \"NANOUSDT\"]\n",
 56 |         "ratios = [\"BTCUSDT\", \"LTCUSDT\", \"ETHUSDT\", \"BNBUSDT\",\n",
 57 |         "          \"ADAUSDT\", \"BTTUSDT\", \"DASHUSDT\", \"XMRUSDT\",\n",
 58 |         "          \"NANOUSDT\", \"DOGEUSDT\", \"XLMUSDT\", \"BCHUSDT\"]\n",
 59 |         "RATIO_TO_PREDICT = \"BTCUSDT\"\n",
 60 |         "\n",
 61 |         "NUM_FOLDS = 5  # for 10 we got val (input[test] of 10-fold) acc ~0.70 and test acc ~0.52. Overfiting to \"val\" set. Use 5, so its like 20% used for val\n",
 62 |         "SEQ_LEN = 48\n",
 63 |         "FUTURE_PERIOD_PREDICT = 1\n",
 64 |         "EPOCHS = 40 # 12 # 40\n",
 65 |         "BATCH_SIZE = 64\n",
 66 |         "current_time = datetime.utcnow().strftime('%Y-%m-%d %H-%M-%S')\n",
 67 |         "NAME = f\"LSTM-{current_time}\"\n",
 68 |         "\n",
 69 |         "acc_per_fold = []\n",
 70 |         "loss_per_fold = []\n",
 71 |         "kfold = KFold(n_splits=NUM_FOLDS, shuffle=True)\n",
 72 |         "\n",
 73 |         "def construct_main_df(ratios):\n",
 74 |         "    main_df = pd.DataFrame()\n",
 75 |         "    # 'unix,date,symbol,open,high,low,close,Volume LTC,Volume USDT,tradecount'\n",
 76 |         "    for ratio in ratios:\n",
 77 |         "        dataset = f\"{drive_path}data/Binance_{ratio}_1h.csv\"\n",
 78 |         "        df = pd.read_csv(dataset, nrows=None, skiprows=None, parse_dates=['date'], usecols=[0, 1, 2])  # todo\n",
 79 |         "        df.set_index(\"date\", inplace=True)\n",
 80 |         "\n",
 81 |         "        if len(main_df) == 0:\n",
 82 |         "            main_df = df\n",
 83 |         "        else:\n",
 84 |         "            main_df = main_df.join(df)\n",
 85 |         "\n",
 86 |         "    # in this case it reduces main_df length to the length of pair with the smallest data\n",
 87 |         "    main_df.dropna(how='any', inplace=True)\n",
 88 |         "\n",
 89 |         "    # we determine targets by making ['future'] column with shifted rows,\n",
 90 |         "    # and then if its price is higher than current day setting ['target'] to 1, and 0 otherwise\n",
 91 |         "    main_df['future'] = main_df[f\"{RATIO_TO_PREDICT}_close\"].shift(-FUTURE_PERIOD_PREDICT)\n",
 92 |         "    main_df['target'] = list(map(classify, main_df[f\"{RATIO_TO_PREDICT}_close\"], main_df[\"future\"]))\n",
 93 |         "    main_df.drop('future', 1, inplace=True)  # future column is no longer needed, used only to determine target column\n",
 94 |         "    # rows with future = NaN also are dropped here\n",
 95 |         "    return main_df\n",
 96 |         "\n",
 97 |         "def classify(current, future):\n",
 98 |         "    return 1 if float(future) > float(current) else 0\n",
 99 |         "\n",
100 |         "\n",
101 |         "# in k-fold we dont need validation set\n",
102 |         "def split_main_df(main_df, train_split, test_split):\n",
103 |         "    # [train train train train_index val val val val_index test test test_index]\n",
104 |         "    train_index = int(train_split * len(main_df))  # last indexes\n",
105 |         "    test_index = test_split  # we don't really need to use it, its always the last index\n",
106 |         "\n",
107 |         "    train_df = main_df.iloc[:train_index]\n",
108 |         "    test_df = main_df.iloc[train_index:]\n",
109 |         "    return train_df, test_df\n",
110 |         "\n",
111 |         "# percentage change and scaling\n",
112 |         "def preprocess_data(df):\n",
113 |         "    df = df.copy()  # fixes random SettingWithCopyWarning appearing with pct_change and preprocessing.scale\n",
114 |         "    for col in df.columns:\n",
115 |         "        if col != \"target\":\n",
116 |         "            df[col] = df[col].pct_change()  # converted to percentage change\n",
117 |         "            # example:\n",
118 |         "            # hour 1:00 close = 93.13, hour 2:00 close = 92.41, so\n",
119 |         "            # 93.13 + 93.13 * x = 92.41, x = -0.007731\n",
120 |         "            # so percentage change of 1.0 means doubling the price\n",
121 |         "            # first row (the oldest values) is being filled with NaN percentage, but we don't want to drop whole row yet\n",
122 |         "            # so we wait until loop ends (we will have whole row filled with nans)\n",
123 |         "    df.dropna(inplace=True)\n",
124 |         "    # print('b', df.shape, df.isnull().values.any())\n",
125 |         "    for col in df.columns:\n",
126 |         "        if col != \"target\":\n",
127 |         "            # scaling should be used after splitting to training and test set - leaks data from test set to training\n",
128 |         "            df[col] = preprocessing.scale(df[col].values)  # normalizes to [-1, +1]\n",
129 |         "            # df[col] = preprocessing.StandardScaler().fit_transform(df[col].values.reshape(-1, 1)) # it does the same\n",
130 |         "            # scales the data so it has mean = 0, standard deviation = 1, variance = 1\n",
131 |         "            # it can be checked before and after with e.g. print(df['BTCUSDT_close'].mean())  .std()  .var()\n",
132 |         "\n",
133 |         "    sequential_data = []\n",
134 |         "    prev_days = deque(maxlen=SEQ_LEN)  # list of SEQ_LEN items, we append to it, as it exceed, it pops old values\n",
135 |         "\n",
136 |         "    for i in df.values:\n",
137 |         "        prev_days.append([n for n in i[:-1]])  # set of price and volume features added (-1 so without target)\n",
138 |         "        if len(prev_days) == SEQ_LEN:\n",
139 |         "            sequential_data.append([np.array(prev_days), i[-1]])\n",
140 |         "    # print(len(sequential_data))\n",
141 |         "    # ok so the sequences are like this ([a, b, 1] is single prev_days) (* and ^ just to point where it goes):\n",
142 |         "    # [([[a, b] [c, d], [e, f]], 1*)   ([[c, d] [e, f] [g, h]], 1) ... ([[g, h], [i, j], [k, l]], 0^)]\n",
143 |         "    random.shuffle(sequential_data)\n",
144 |         "\n",
145 |         "\n",
146 |         "    buys = []  # list that will store our buy sequences and targets\n",
147 |         "    sells = []  # list that will store our sell sequences and targets\n",
148 |         "\n",
149 |         "    for seq, target in sequential_data:  # iterate over the sequential data\n",
150 |         "        if target == 0:  # if it's a \"not buy\"\n",
151 |         "            sells.append([seq, target])  # append to sells list\n",
152 |         "        elif target == 1:  # otherwise if the target is a 1...\n",
153 |         "            buys.append([seq, target])  # it's a buy!\n",
154 |         "\n",
155 |         "    random.shuffle(buys)  # shuffle the buys\n",
156 |         "    random.shuffle(sells)  # shuffle the sells\n",
157 |         "\n",
158 |         "    lower = min(len(buys), len(sells))  # what's the shorter length?\n",
159 |         "\n",
160 |         "    buys = buys[:lower]  # make sure both lists are only up to the shortest length.\n",
161 |         "    sells = sells[:lower]  # make sure both lists are only up to the shortest length.\n",
162 |         "\n",
163 |         "    sequential_data = buys + sells  # concatenate lists\n",
164 |         "    random.shuffle(\n",
165 |         "        sequential_data)  # another shuffle, so the model doesn't get confused with all 1 class then the other.\n",
166 |         "\n",
167 |         "    x = []\n",
168 |         "    y = []\n",
169 |         "\n",
170 |         "    for seq, target in sequential_data:\n",
171 |         "        x.append(seq)\n",
172 |         "        y.append(target)\n",
173 |         "    return np.array(x), np.array(y)"
174 |       ],
175 |       "execution_count": null,
176 |       "outputs": []
177 |     },
178 |     {
179 |       "cell_type": "code",
180 |       "metadata": {
181 |         "id": "FY9cXsAF64Uq",
182 |         "colab": {
183 |           "base_uri": "https://localhost:8080/"
184 |         },
185 |         "outputId": "b566df64-2e8b-4938-c8f4-4437ec503987"
186 |       },
187 |       "source": [
188 |         "time0 = time.perf_counter()\n",
189 |         "main_df = construct_main_df(ratios)\n",
190 |         "print(main_df.shape)\n",
191 |         "time1 = time.perf_counter()\n",
192 |         "# print(time1 - time0)\n",
193 |         "\n",
194 |         "train_df, test_df = split_main_df(main_df, train_split=0.8, test_split=0.2)\n",
195 |         "print(train_df.shape, test_df.shape)\n",
196 |         "time2 = time.perf_counter()\n",
197 |         "# print(time2 - time1)\n",
198 |         "\n",
199 |         "train_x, train_y = preprocess_data(train_df)\n",
200 |         "test_x, test_y = preprocess_data(test_df)\n",
201 |         "time3 = time.perf_counter()\n",
202 |         "print(train_x.shape, train_y.shape, test_x.shape, test_y.shape,)\n",
203 |         "# print(time3 - time2)"
204 |       ],
205 |       "execution_count": null,
206 |       "outputs": [
207 |         {
208 |           "output_type": "stream",
209 |           "text": [
210 |             "(15737, 25)\n",
211 |             "(12589, 25) (3148, 25)\n",
212 |             "(12440, 48, 24) (12440,) (3066, 48, 24) (3066,)\n"
213 |           ],
214 |           "name": "stdout"
215 |         }
216 |       ]
217 |     },
218 |     {
219 |       "cell_type": "code",
220 |       "metadata": {
221 |         "id": "JJKJcltc7kDx"
222 |       },
223 |       "source": [
224 |         "def build_model():\n",
225 |         "    model = Sequential()\n",
226 |         "    model.add(LSTM(128, input_shape=(train_x.shape[1:]), return_sequences=True))\n",
227 |         "    # i cannot save CuDNNLSTM due to some error, not using it\n",
228 |         "    # model.add(CuDNNLSTM(128, input_shape=(train_x.shape[1:]), return_sequences=True))\n",
229 |         "    model.add(Dropout(0.2))\n",
230 |         "    model.add(BatchNormalization())\n",
231 |         "\n",
232 |         "    model.add(LSTM(128, input_shape=(train_x.shape[1:]), return_sequences=True))\n",
233 |         "    model.add(Dropout(0.1))\n",
234 |         "    model.add(BatchNormalization())\n",
235 |         "\n",
236 |         "    model.add(LSTM(128, input_shape=(train_x.shape[1:])))\n",
237 |         "    model.add(Dropout(0.2))\n",
238 |         "    model.add(BatchNormalization())\n",
239 |         "\n",
240 |         "    model.add(Dense(32, activation=\"relu\"))\n",
241 |         "    model.add(Dropout(0.2))\n",
242 |         "\n",
243 |         "    model.add(Dense(2, activation=\"softmax\"))\n",
244 |         "\n",
245 |         "    opt = tf.keras.optimizers.Adam(learning_rate=0.001, decay=1e-6)\n",
246 |         "    # opt = tf.keras.optimizers.Adam()\n",
247 |         "\n",
248 |         "    model.compile(loss = \"sparse_categorical_crossentropy\", \n",
249 |         "                  optimizer=opt,\n",
250 |         "                  metrics=['accuracy'])\n",
251 |         "    # print(model.summary())\n",
252 |         "    return model\n"
253 |       ],
254 |       "execution_count": null,
255 |       "outputs": []
256 |     },
257 |     {
258 |       "cell_type": "code",
259 |       "metadata": {
260 |         "colab": {
261 |           "base_uri": "https://localhost:8080/"
262 |         },
263 |         "id": "HwV1fg6uttcy",
264 |         "outputId": "ed7bbc35-1e42-4a17-c82b-df0cd53bb575"
265 |       },
266 |       "source": [
267 |         "print(train_x.shape)\n",
268 |         "\n",
269 |         "\n",
270 |         "FINAL_PRODUCTION_TRAINING = True  # set manually!\n",
271 |         "\n",
272 |         "if FINAL_PRODUCTION_TRAINING is False:\n",
273 |         "  # this way we can use test_x and test_y as final evaluation after k-fold\n",
274 |         "  inputs = train_x\n",
275 |         "  targets = train_y\n",
276 |         "elif FINAL_PRODUCTION_TRAINING is True:\n",
277 |         "  # this way we cant make that final evaluation, and rely on average score from k-fold\n",
278 |         "  inputs = np.concatenate((train_x, test_x), axis=0)\n",
279 |         "  targets = np.concatenate((train_y, test_y), axis=0)\n",
280 |         "\n",
281 |         "# production model training should be performed with train_x with 100% of main_df. we can expect its results close to average score from k-fold evals\n",
282 |         "fold_no = 1\n",
283 |         "for train, test in kfold.split(inputs, targets):\n",
284 |         "    model = build_model()\n",
285 |         "\n",
286 |         "    checkpoint = ModelCheckpoint(filepath=f\"models/{NAME}\", monitor='val_accuracy', verbose=1, save_best_only=True, mode='max') # saves only the best ones\n",
287 |         "    earlystopping = EarlyStopping(monitor='accuracy', patience=4)  # todo val_loss not available for EarlyStopping callback\n",
288 |         "    cb = [earlystopping]\n",
289 |         "    print('------------------------------------------------------------------------')\n",
290 |         "    print(f'Training for fold {fold_no} ...')\n",
291 |         "    history = model.fit(\n",
292 |         "            inputs[train], targets[train],\n",
293 |         "            batch_size=BATCH_SIZE,\n",
294 |         "            epochs=EPOCHS,\n",
295 |         "            verbose=0,\n",
296 |         "            # validation_data=(validation_x, validation_y),\n",
297 |         "            callbacks=cb)\n",
298 |         "    \n",
299 |         "    scores = model.evaluate(inputs[test], targets[test], verbose=0)  # input[test] is like validation set, its part of training set\n",
300 |         "    print(f'Score for fold {fold_no}: {model.metrics_names[0]} of {scores[0]}; {model.metrics_names[1]} of {scores[1]*100}%')\n",
301 |         "    acc_per_fold.append(scores[1] * 100)\n",
302 |         "    loss_per_fold.append(scores[0])\n",
303 |         "\n",
304 |         "    # Increase fold number\n",
305 |         "    fold_no = fold_no + 1\n",
306 |         "\n",
307 |         "\n"
308 |       ],
309 |       "execution_count": null,
310 |       "outputs": [
311 |         {
312 |           "output_type": "stream",
313 |           "text": [
314 |             "(12440, 48, 24)\n",
315 |             "------------------------------------------------------------------------\n",
316 |             "Training for fold 1 ...\n",
317 |             "Score for fold 1: loss of 1.3445900678634644; accuracy of 63.7653112411499%\n",
318 |             "------------------------------------------------------------------------\n",
319 |             "Training for fold 2 ...\n",
320 |             "Score for fold 2: loss of 1.2719135284423828; accuracy of 64.75330591201782%\n",
321 |             "------------------------------------------------------------------------\n",
322 |             "Training for fold 3 ...\n",
323 |             "Score for fold 3: loss of 1.3596887588500977; accuracy of 65.07578492164612%\n",
324 |             "------------------------------------------------------------------------\n",
325 |             "Training for fold 4 ...\n",
326 |             "Score for fold 4: loss of 1.2974392175674438; accuracy of 62.59270906448364%\n",
327 |             "------------------------------------------------------------------------\n",
328 |             "Training for fold 5 ...\n",
329 |             "Score for fold 5: loss of 1.2682578563690186; accuracy of 65.68848490715027%\n"
330 |           ],
331 |           "name": "stdout"
332 |         }
333 |       ]
334 |     },
335 |     {
336 |       "cell_type": "code",
337 |       "metadata": {
338 |         "colab": {
339 |           "base_uri": "https://localhost:8080/"
340 |         },
341 |         "id": "naOrFdKs7TRw",
342 |         "outputId": "51cb67dd-c6fe-4262-c4db-c9cf310233c5"
343 |       },
344 |       "source": [
345 |         "# == Provide average scores ==\n",
346 |         "print('------------------------------------------------------------------------')\n",
347 |         "print('Score per fold')\n",
348 |         "for i in range(0, len(acc_per_fold)):\n",
349 |         "  print('------------------------------------------------------------------------')\n",
350 |         "  print(f'> Fold {i+1} - Loss: {loss_per_fold[i]} - Accuracy: {acc_per_fold[i]}%')\n",
351 |         "print('------------------------------------------------------------------------')\n",
352 |         "print('Average scores for all folds:')\n",
353 |         "print(f'> Accuracy: {np.mean(acc_per_fold)} (+- {np.std(acc_per_fold)})')\n",
354 |         "print(f'> Loss: {np.mean(loss_per_fold)}')\n",
355 |         "print('------------------------------------------------------------------------')\n"
356 |       ],
357 |       "execution_count": null,
358 |       "outputs": [
359 |         {
360 |           "output_type": "stream",
361 |           "text": [
362 |             "------------------------------------------------------------------------\n",
363 |             "Score per fold\n",
364 |             "------------------------------------------------------------------------\n",
365 |             "> Fold 1 - Loss: 1.3445900678634644 - Accuracy: 63.7653112411499%\n",
366 |             "------------------------------------------------------------------------\n",
367 |             "> Fold 2 - Loss: 1.2719135284423828 - Accuracy: 64.75330591201782%\n",
368 |             "------------------------------------------------------------------------\n",
369 |             "> Fold 3 - Loss: 1.3596887588500977 - Accuracy: 65.07578492164612%\n",
370 |             "------------------------------------------------------------------------\n",
371 |             "> Fold 4 - Loss: 1.2974392175674438 - Accuracy: 62.59270906448364%\n",
372 |             "------------------------------------------------------------------------\n",
373 |             "> Fold 5 - Loss: 1.2682578563690186 - Accuracy: 65.68848490715027%\n",
374 |             "------------------------------------------------------------------------\n",
375 |             "Average scores for all folds:\n",
376 |             "> Accuracy: 64.37511920928955 (+- 1.0869902263795579)\n",
377 |             "> Loss: 1.3083778858184814\n",
378 |             "------------------------------------------------------------------------\n"
379 |           ],
380 |           "name": "stdout"
381 |         }
382 |       ]
383 |     },
384 |     {
385 |       "cell_type": "code",
386 |       "metadata": {
387 |         "id": "Eye28PNHttoz",
388 |         "colab": {
389 |           "base_uri": "https://localhost:8080/",
390 |           "height": 556
391 |         },
392 |         "outputId": "a38c3822-9b2d-4a31-b534-f63468fa653d"
393 |       },
394 |       "source": [
395 |         "def plot_history(history):\n",
396 |         "        hist_df = pd.DataFrame(history.history)\n",
397 |         "        print(f'Trained model:')\n",
398 |         "        print(hist_df.tail(1), '\\n')\n",
399 |         "        hist_df_short = hist_df[['accuracy', 'loss']]\n",
400 |         "        hist_df_short.plot(figsize=(8,8), grid=True)\n",
401 |         "\n",
402 |         "plot_history(history)"
403 |       ],
404 |       "execution_count": null,
405 |       "outputs": [
406 |         {
407 |           "output_type": "stream",
408 |           "text": [
409 |             "Trained model:\n",
410 |             "        loss  accuracy\n",
411 |             "39  0.124177  0.950181 \n",
412 |             "\n"
413 |           ],
414 |           "name": "stdout"
415 |         },
416 |         {
417 |           "output_type": "display_data",
418 |           "data": {
419 |             "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAHSCAYAAADfUaMwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3wUdeL/8dek9xDSSKND6AFCEVCaoHRQsKJSFM7zvObdeXqWu+/p/fT0qp4NPezIYUelCAJSpAtBOiGUNCAJJCG97Pz+GEREkACbzGb3/Xw89pHs7mTyHlDfzuxnPh/DNE1ERETEPl52BxAREfF0KmMRERGbqYxFRERspjIWERGxmcpYRETEZipjERERm/nY9YujoqLMli1bOm1/paWlBAcHO21/dtPxuDYdj+tzt2PS8bi2uhzP5s2b803TjD7Xe7aVccuWLdm0aZPT9rdixQoGDx7stP3ZTcfj2nQ8rs/djknH49rqcjyGYRw633u6TC0iImIzlbGIiIjNVMYiIiI2s+0z43Oprq4mKyuLioqKi/7Z8PBwdu3aVQ+p7HEpxxMQEEBiYiK+vr71lEpEROqDS5VxVlYWoaGhtGzZEsMwLupnT548SWhoaD0la3gXezymaVJQUEBWVhatWrWqx2QiIuJsLnWZuqKigsjIyIsuYgHDMIiMjLykqwoiImIvlypjQEV8GfRnJyLSOLlcGYuIiHgalbFNampq7I4gIiIuQmV8DhMmTCA1NZXOnTsza9YsABYtWkTPnj1JSUnh6quvBqCkpIRp06bRtWtXunXrxvvvvw9ASEjI6X299957TJ06FYCpU6dy991307dvX+6//342bNhAv3796NGjB/3792fPnj0A1NbW8tBDD9GlSxe6devGs88+y7Jly5gwYcLp/S5ZsoTrrruuIf44RESknrnUaOoz/d8nO9iZU1zn7Wtra/H29v7RbTrFh/HHsZ0vuK/Zs2fTtGlTysvL6d27N+PHj2fGjBmsXLmSVq1acfz4cQAee+wxwsPD+eabbwA4ceLEBfedlZXFV199hbe3N8XFxaxatQofHx+WLl3KH/7wB95//31mzZrF4cOH2bp1Kz4+Phw/fpyIiAjuuece8vLyiI6O5tVXX2X69Ol1+JMRERFX57JlbKdnnnmGDz/8EIDMzExmzZrFwIEDT98y1LRpUwCWLl3K3LlzT/9cRETEBfd9ww03nP6fhqKiIqZMmcK+ffswDIPq6urT+502bRo+Pj7f+3233347b731FtOmTWPt2rW88cYbTjpiERGxk8uWcV3OYM/krPuMV6xYwdKlS1m7di1BQUEMHjyY7t27s3v37jrv48xRzWffanTmqh6PPPIIQ4YM4cMPP+TgwYMXnGR82rRpjB07loCAAG644YbTZS0iIo2bPjM+S1FREREREQQFBbF7927WrVtHRUUFK1eu5MCBAwCnL1MPHz6c55577vTPfnuZOjY2ll27duFwOE6fYZ/vdyUkJADw2muvnX59+PDhvPrqq6cHeX37++Lj44mPj+fxxx9n2rRpzjtoERGxlcr4LCNGjKCmpoaOHTvywAMPcMUVVxAdHc2sWbO4/vrrSUlJ4aabbgLg4Ycf5sSJE3Tp0oWUlBSWL18OwJNPPsmYMWPo378/cXFx5/1d999/Pw8++CA9evT43ujqu+66i8TERLp160ZKSgpz5sw5/d7kyZNJSkqiY8eO9fQnICIiDU3XOc/i7+/PwoULz/neyJEjv/c8JCSE119//QfbTZo0iUmTJv3g9TPPfgH69evH3r17Tz9//PHHAfDx8eGJJ54452X31atXM2PGjAseh4iINB46M25EUlNT2bZtG7fddpvdUURE3JppmmQXljfY79OZcSOyefNmuyOIiLi13UeKmb81h/lpOVTWOFj34NV4e9X/VMMqYxER8WiHC8qYn5bN/LQc9h4twdvLYEDbKMalxFPrMFXGIiIi9eFYcQWfbstlfloOWzMLAejVIoLHxndmZNc4okL8GzSPylhERDxC5vEyVu3L57Nvcli7vwCHCZ3iwnhgZAfGdIsjMSLItmwqYxERcUvHiitYm1HAmvR8vtpfQNYJa0BWy8gg7h3ajnEpcbSNufzJopxBZXyWkJAQSkpK7I4hIiIXqbCsinUZx1m7P581+wtIP2b9tzw80Jd+rSOZObA1/dtE0iY6xOXWf1cZi4hIo1NRXcvuIyfZmVPMztwi0jKL2J5ThGlCoK83fVo15YbURAa0jaJjXFiDDMK6HCrj8zBNk/vvv5+FCxdiGAYPP/wwN910E7m5udx0000UFxdTU1PDCy+8QP/+/bnzzjvZtGkThmEwffp0fv3rX9t9CCIibuFEaRU7c4vZkVPEzpxiduQUsz+vBIdpvR8a4EPn+DB+eXU7BrSNIiWxCX4+jWsaDdct44UPwJFv6rx5YG0NeF/gcJp1hZFP1ml/H3zwAVu3biUtLY38/Hx69+7NwIEDmTNnDtdeey0PPfQQtbW1lJWVsXXrVrKzs9m+fTsAhYWFdc4tIiLfcThM9h0rYe3+fNZlHGdbViE5Rd8tuBMXHkCnuDBGdmlGp/gwOseHkxgR6HKXnS+W65axzVavXs0tt9yCt7c3sbGxDBo0iI0bN9K7d2+mT59OdXU1EyZMoHv37rRu3ZqMjAx+/vOfM3r0aK655hq744uINAqmaZJ+rIS1GQWsyyhgXcZxjpdWAZAYEUjvVk3pHB9Gp7hwOsWH0TTYz+bE9cN1y7iOZ7DfKnfSEooXMnDgQFauXMlnn33G1KlTue+++7jjjjtIS0tj8eLFvPjii8ybN4/Zs2fXexYRkcbG4TDJyC9h2eFq3p3zNeszCsgvsco3PjyAwcnR9GsdyRWtI0lqat+tRg3NdcvYZldddRUvvfQSU6ZM4fjx46xcuZKnn36aQ4cOkZiYyIwZM6isrOTrr79m1KhR+Pn5MXHiRJKTkzV3tIjIKUVl1WzNKmTL4RNsOVzI1sxCisqrAWgWdoKr2kVzReum9GsdRVLTxn+5+VKpjM/juuuuY+3ataSkpGAYBk899RTNmjXj9ddf5+mnn8bX15eQkBDeeOMNsrOzmTZtGg6HA4AnnnjC5vQiIg2vptbB3qMlbMm0infL4RPszysFwDCgfUwoI7s0o0fzJnAsnRtHDfHY8j2byvgs395jbBgGTz/9NE8//fT33p8yZQpTpkz5wc99/fXXDZJPRMRVmKbJ3qMlrNqXx8p9+Ww6eJyyqloAmgb70bN5E67vmUiPpCZ0TQwnNMD39M+uWJGhIj6DylhEROqsoKSS1en5rNqXz6p9eRwtrgSgbUwIk1ITSW0RQY+kCI++5HwpVMYiInJeVTUONh86wap9eazal396Yo3wQF+ubBfFwHZRXNkumoQmgXZHbdRUxiIi8gPHS6t4bc0BXl97iKLyary9DHo2b8J9w9pzVftouiaEu/ysVo2Jy5WxaZq6tHGJTNO0O4KINHLZheW8vDKDuRsPU1Ht4JpOsUxMTaR/m8jvfeYrzuVSZRwQEEBBQQGRkZEq5ItkmiYFBQUEBATYHUVEGqH0Yyd5YUUGH2/NBmB89wTuHtSadrGusaqRu3OpMk5MTCQrK4u8vLyL/tmKigq3KqJLOZ6AgAASExPrKZGIuKOtmYW8sCKdz3cexd/Hi9uuaMFdV7WydW1fT+RSZezr60urVq0u6WdXrFhBjx49nJzIPu52PCLiOkzTZHV6Pi+s2M9X+wsIC/Dh50PaMqV/SyJD/O2O55FcqoxFRKR+1DpMNh86waLtR1i84wjZheXEhPrz0KiO3NK3OSH+qgM76U9fRMRNVdc6WJdRcKqAj5JfUomfjxcD20Vx3/D2jEmJw9/H2+6YgspYRMStVFTXsnpfPot2HGHprqMUllUT5OfNkOQYru3SjKEdYnQW7IL0NyIi0siZpsm6jOPM25TJ5zuOUFpVS2iAD8M7xnJtl2YMah9NgK/OgF2ZylhEpJE6WlzBe5uzmLcpk0MFZYQG+DA2JZ4RXZrRv00Ufj5edkeUOlIZi4g0IjW1DpbvyeN/Gw+zfE8etQ6Tvq2a8qth7RjZJU5nwI2UylhEpBE4Uurgr4t2897mLPJOVhId6s/Mga25sVcSraKC7Y4nl0llLCLiohwOk6W7jjJ7zQHWZZTjZexnaIcYburdnMHJ0fh66zK0u1AZi4i4mMqaWj7aks1LKzPIyCsloUkgE9v58rtJA2kW7j4zDcp3VMYiIi6iqLyat9cf4tU1B8k7WUnn+DD+fXN3RneNY/WqlSpiN6YyFhGxWU5hObNXH+CdDYcprarlqnZR/PPG7gxoq0VzPIXKWETEJruPFDPrywzmp+VgAmO6xTHjqtZ0SQi3O5o0MJWxiEgD23L4BM8uS2fZ7mME+npze78W3HmlVkryZCpjEZEGsvHgcZ75Yh+r9uXTJMiX+4a3545+LWgS5Gd3NLGZylhEpB6ZpsnajAKe/SKdtRkFRAb78cDIDtx2RQvNES2n6Z8EEZF6YJomq/bl88wX+9h06AQxof48MqYTt/ZpTqCfZsmS71MZi4g4kWmaLNt9jGeWpZOWWUhceAB/Ht+ZG3slaapKOS+VsYiIEzgcJp/vPMKzy9LZkVNMYkQg/++6rkxMTdCawXJBKmMRkctQ6zD5dFsOzy1PZ+/RElpGBvHUpG5c1yNB01VKnamMRUQuQXWtg4+2ZPP8iv0cyC+lfWwI/765O2O6xePtpYk65OKojEVELkJlTS3vbsrihRX7yS4sp3N8GC/e1pNrOjXDSyUsl0hlLCJSB+VVtbyz4TAvrdzP0eJKejRvwuMTujA4OVpTVsplUxmLiPyI/JJK5qw/zBtrD5JfUkXfVk35x43d6d9G80aL86iMRUTOYUdOEa+uOcj8tByqahwMTo7mnsFt6dOqqd3RxA2pjEVETql1mCzZeYTZaw6y4cBxAn29ualXElP6t6RtTIjd8cSNqYxFxOMVlVXzv02Hef2rQ2QXlpPQJJCHRnXkxt5JhAf62h1PPIDKWEQ81sH8Ul5ZncH7m7Mpr66lT6umPDKmI8M6xuKje4SlAamMRcTjmKbJG2sP8ZcFu8CEcd3jmTagJZ3jtY6w2ENlLCIepaCkkvvf28YXu48xODmapyZ2IyYswO5Y4uFUxiLiMVbty+O+eWkUlVXzx7GdmNq/pW5PEpegMhYRt1dV4+Bvn+9h1soM2saE8Pq0PnSKD7M7lshpKmMRcWv780r45dwtbM8uZnLf5jw8upPWExaXozIWEbdkmib/23iYP83fib+vFy/dnsq1nZvZHUvknFTGIuJ2isqqeW5rJZuOfkO/1pH886buNAvXIC1xXSpjEXErS3ce5dGPt3O0uJbfj+jAzIGttaShuDyVsYi4hYP5pfz5050s232MtjEhPHRFANMHt7E7lkidqIxFpFErr6rl+RXpvPRlBr7eBg+N6sjUAS1Zs2ql3dFE6kxlLCKNkmmaLN5xlMc+3Ul2YTnju8fzh1EdidUEHtIIqYxFpNHJyCvhT5/sZOXePDo0C+V/M6+gb+tIu2OJXDKVsYg0GmVVNTy7LJ1XVmUQ4OPNo2M6cUe/FlrUQRo9lbGIuLzKmlrmb83hH0v2kltUwcSeifx+ZDIxobokLe5BZSwiLuvYyQreWneYOesPkV9SRef4MJ69pQe9Wja1O5qIU6mMRcTlbMsq5NU1B/l0Ww41DpOhyTFMG9CKAW0jtbCDuCWVsYi4hOpaB4t3HOHVNQfZfOgEwX7eTO7bgqn9W9IyKtjueCL1SmUsIrY6UVrFOxsP8+baQ+QWVdC8aRCPjunEDb0SCQ3wtTueSINQGYuILQrLqnhpZQavrTlIeXUt/dtE8ufxXRjaIUbTV4rHURmLSIMqrqhm9uoD/HfVAUqqahjbLZ57hrShQzOtLyyeS2UsIg2irKqG1746yKyVGRSWVTOiczN+Pbw9yc1C7Y4mYrs6lbFhGCOAfwPewCumaT551vvNgdeBJqe2ecA0zQVOzioijVBFdS1vrz/MCyvSyS+pYkhyNPcNT6ZrYrjd0URcxgXL2DAMb+A5YDiQBWw0DGO+aZo7z9jsYWCeaZovGIbRCVgAtKyHvCLSSFTVOJi3KZP/LEvnSHEF/dtE8tLt7UltoXuERc5WlzPjPkC6aZoZAIZhzAXGA2eWsQl8+4FPOJDjzJAi0ng4HCbz03L4+5I9ZB4vJ7VFBP+4KYX+baLsjibisupSxglA5hnPs4C+Z23zJ+BzwzB+DgQDw5ySTkQalXUZBfzls118k11E5/gwXpvWhUHtozVRh8gFGKZp/vgGhjEJGGGa5l2nnt8O9DVN894ztrnv1L7+bhhGP+C/QBfTNB1n7WsmMBMgNjY2de7cuU47kJKSEkJCQpy2P7vpeFybjuf7cksczNtbxZZjtTQNMJjYzpd+8T542VjC+jtybZ54PEOGDNlsmmavc75pmuaPPoB+wOIznj8IPHjWNjuApDOeZwAxP7bf1NRU05mWL1/u1P3ZTcfj2nQ8lvyTFeYjH31jtn7wM7Pzo4vM/yzbZ5ZX1Tg33CXS35Fr88TjATaZ5+nEulym3gi0MwyjFZAN3AzcetY2h4GrgdcMw+gIBAB5ddi3iDRCFdW1vLrmIM8vT6esupZb+iTxq2HtiQrxtzuaSKN0wTI2TbPGMIx7gcVYty3NNk1zh2EYf8Zq+fnAb4CXDcP4NdZgrqmn/i9ARNyIw2HyybYcnlq0h+zCcq7uEMODozrQNkb3CotcjjrdZ2xa9wwvOOu1R8/4ficwwLnRRMSVfLU/nycX7mZbljU46+lJ3ejfViOkRZxBM3CJyI/alVvMkwt38+XePOLDA/jbDSlc3yMBL80fLeI0KmMROaesE2X8Y8lePtySTViAL38Y1YE7+rUkwNfb7mgibkdlLCLfU1hWxXPL03l97SEAZg5szT2D2hIepOUMReqLylhEgDNGSK9Ip6Syhkk9E/n18PbENwm0O5qI21MZi3g4h8NkVVY1D/5tBblFFVzdIYb7R3TQakoiDUhlLOLBjp2s4Dfz0li1r4ruSU34503duaJ1pN2xRDyOyljEQ63cm8d987ZSUlnD1M5+/PG2/ppDWsQmXnYHEJGGVVXj4IkFu7hj9gYig/355N4rGZzkqyIWsZHOjEU8yOGCMn7+ztekZRUxuW9zHhnTiQBfb7J32Z1MxLOpjEU8xMdbs3now+14GfDC5J6M7BpndyQROUVlLOLmyqpq+OPHO3h3cxa9WkTwr5u7kxgRZHcsETmDyljEje3IKeLn72zhQH4pvxjall9c3Q4fbw0VEXE1KmMRN3SooJR3N2Uxa2UGEcG+vH1XX/q30aIOIq5KZSziJoorqlmwLZf3v85i48ETGAaM7NKMxyd0pWmwn93xRORHqIxFGrFah8mqfXm8/3U2n+84QmWNgzbRwdw/IpnreiQQF66pLEUaA5WxSCO09+hJ3t+cxYdbsjl2spLwQF9u7JXExNREUhLDdc+wSCOjMhZpJKprHXy6LYfX1hwkLasIHy+DwcnRTOyZyNCOMfj7aGlDkcZKZSzi4orKq3lnw2FeW3OQI8UVtIkO5pExnRjfPZ6oEH+744mIE7hHGe9ZSM/ND0P/leAXbHcaEafIPF7G7DUHmLcxk9KqWvq3ieSJ67syqH00Xl66DC3iTtyjjDEIO5kOOVug5ZV2hxG5LFsOn+CVVQdYuD0XL8NgbEo8d17Zii4J4XZHE5F64h5lnNjb+pq5QWUsjVKtw2TJzqO8siqDTYdOEBrgw4yBrZnav6VGRIt4APco4+BIygLjCcraaHcSkYtSXFHNvI2ZvL72IJnHy0lqGsgfx3bixl5JBPu7x7+eInJhbvNve3FYB4IyN4Bpgm7rEBeXkVfC618d5L3NWZRW1dK7ZQQPjuzItZ2b4a3Pg0U8jtuUcVF4Ms2OLoPjGRDZxu44Ij9gmiar9uXz6poDLN+Th5+3F2NS4pg+QJ8Hi3g6tynj4rAO1jdZG1XG4lLKqmr44OtsXvvqIOnHSogK8edXw9oxuW8LokN1a5KIuFEZlwYngV+oNYgr5Wa744hQWVPLc8vSeX3tIYrKq+maEM4/bkxhdLc4TdAhIt/jNmWM4Q2JqZC1we4kIuzKLebX/9vK7iMnGdmlGXde2YrUFhGaplJEzsl9yhggsQ+s+htUngT/ULvTiAdyOExeWZ3B3xbvJSzQl9lTezG0Q6zdsUTExblXGSf1BdMB2V9D60F2pxEPk3WijN++m8a6jONc0ymWJ67vSqSmqxSROnCvMk5Mtb5mbVAZS4MxTZOPtmbz6Ec7cJgmT03qxg2pibokLSJ15l5lHBgBUcmQqck/pGEUllXx0Ifb+eybXHq1iOCfN3UnqWmQ3bFEpJFxrzIGSOoNuz/T5B9S71bty+O376ZxvLSK+0ck85OBbTRhh4hcEi+7AzhdUl8oPwEF6XYnETdUUV1LWmYhb+6s5Pb/biAswJcP7xnAPYPbqohF5JK535lxYh/ra+YGiGpnbxZp1I6drGBnTjG7ck+yM7eYXbnFZOSV4DCt96cNaMnvR3QgwFf3DIvI5XG/Mo5qDwHh1iCuHpPtTiMuyuEwKSqvpqC0iuOlVRwvraSgtIrDBWWnize/pOr09glNAukYF8aoLs3oFB9GSeYuJo3sbOMRiIg7cb8y9vKChF7WmbF4vIKSSl5edYDM42UUlFaeKt4qTpRVU/vtKe4Z/Ly9aBcbwuDkGDrFhdExLoxOcWGEB/l+b7sV+Xsa6hBExAO4XxmD9bnxiiegosg6SxaPY5omH27J5rFPd3KyoobmkUFEBvvRKiqY1BZNiQz2o2mwH5EhfkQEffd9VIg/vt7uN5RCRFybm5Zxb8CE7M3QZqjdaaSBZR4v4w8ffsOqffn0aN6Ev07sRvtYzcgmIq7LPcs4oRdgWPcbq4w9Rq3D5LWvDvK3xXvwMuD/xnXmtitaaJSziLg89yzjgDCI6QiZ6+1OIg1kV24xD7y/jbSsIoYkR/P4dV1JaBJodywRkTpxzzIGSOoD2z8Eh8Ma1CVuqaK6lv8sS+fFL/cTHujLM7f0YGy3OE1FKSKNivuWcWIf2Pwa5O+FmA52p5F6sOHAcR74YBsZeaVc3zOBR0Z3IiLYz+5YIiIXzX3LOOnU5B9ZG1TGjUBFdS0r9hxj0fYjHDtZSU2tSbXDQa3DpLrWpKb21PcOBzW1JjUOk7yTlSRGBPLG9D4MbB9t9yGIiFwy9y3jyLbWwhGZ66HnHXankXOoqK5l5d48Pvsml6U7j1JaVUvTYD9aRwXj7WUQ7OeDj7eBj5cXvt4G3l4Gvt5e+HgZ+Hh7kRgRyLQBLQnyc99/jEXEM7jvf8UMw7pUrRWcXEpVjYPV6Xl8mpbLkp1HOVlZQ5MgX8Z1j2d013iuaN0UH93nKyIexn3LGKz7jfctthaOCIywO43Hqq51sC2vhs/eTWPxjiMUV9QQFuDDiC7NGJMST/82kZpoQ0Q8mnuX8beLRmRthnbD7M3iYSpralm9L58F3xxh6a6jFJVXExpwhGs6NWNMtzgGtI3Cz0cFLCIC7l7GCalgeFmfG6uM6115VS1f7j3Gwu1H+GLXMUoqrTPg4Z2akUge91w/BH8frXAkInI29y5j/xCI7WyNqJZ6UVJZw/Ldx1i4PZflu/Mor7YGYY3pFsfIrnH0ax2Jn48XK1asUBGLiJyHe5cxWJeqt80DRy14qQycJf1YCf9Zto8F249QVeMgOtSfSamJjOzSjD6tNAhLRORiuH8ZJ/WBTf+FvN3WWbJclvRjJTy7bB/z03II9PXmlt5JjEmJp2fzCM0BLSJyidy/jBN7W18z16uML8PZJXz3oDbMuKo1TTXjlYjIZXP/Mm7aGoKirPuNe023O02joxIWEal/7l/GhmFdqnbTQVz7jp7kUEEZpVU1lFXVUlpZQ2llLWVVNZRUfvdaWVUtALFhAcSFBxDX5NTX8EDiwgMID/T93uIK334mPD8thwBfb34ysA0zrmpFZIi/XYcqIuK23L+MwbpUvWcBlBZAcKTdaZxie3YR/1q6l6W7jp3zfX8fL0L8fQjy9ybYz4dgfx8cpsna/fkcPVlJrcP83vaBvt6nS9rX24uVe/MI8PVmpkpYRKTeeUYZn140YiMkj7A3y2XalVvMv5buZfGOo4QF+PCb4e0ZlBxNkJ8Pwf7eBPv7EOTr/aOjmWtqHeSXVJFTVM6RogpyiyrILSwnt9j6mlNWoRIWEWlAnlHG8T3B8LYuVTfSMt579CT/XrqPz77JJdTfh18Na8f0K1sRFuB70fvy8faiWXgAzcID6iGpiIhcLM8oY78gaNYVMhvf58bpx0p45ot9fLIthyBfb34+tC13Xdma8KCLL2EREXFNnlHGYF2q3vIW1NaAt+sf9tFSB/f9bysfbc0m4NQo5plXtSZCo5hFRNyO67eSsyT2gQ2z4NgOiEuxO805Vdc6WL77GPM2ZbFsdzl+PrncdVVrZg5sTZQ+uxURcVueU8bfDuLK3OByZbzv6Enmbcrkwy3Z5JdUER3qz4iWvvzp1oHEhOpzXRERd+c5ZdykOYTEWiOq+8ywOw3FFdV8mpbLvE2ZbM0sxMfLYFjHWG7olcig9tGsXrVSRSwi4iE8p4wNw7rfOHO9bREcDpN1Bwp4d1MWC7fnUlHtoH1sCA+P7siEHgm6FC0i4qE8p4zBulS9+1MoyYOQ6Ab7tUVl1by7OZO31h3iYEEZof4+TOyZyI29kuiWGP69ma9ERMTzeFgZ97W+Zm2ADqPr/ddtzy7izbWH+Dgtm4pqB71aRPDLYe0Y0TmOQD8t5ygiIhbPKuO47uDtBwfX1FsZV9U4WLg9lzfWHmLzoRME+HpxXY8EbruiBZ3jw+vld4qISOPmWWXsGwCtB8OuT+Dav1ifIztJTmE5c9YfZu7Gw+SXVNEyMoiHR3fkhtQkTdAhIiI/yrPKGKDjONj3OeRuhfgel7Wr46VVrNhzjEXbj/DF7mM4TJOhyTHc3q8FA9tF4+Wlz4JFROTCPK+MO4yGT34JO+dfdBmbpsmOnGKW7z7Gsj3H2JpZiGlCVIg/d13Vitv6tl7Sah8AACAASURBVCCpaVA9BRcREXfleWUc1BRaXgm75sPVj17wUnVpZQ2r0/NZvvsYy/cc42hxJQDdEsP5xdB2DO0QQ9eEcJ0Fi4jIJfO8MgboNA4++w0c2wWxnc65yY6cIp5cuJv1GcepqnUQ6u/DVe2jGJIcw6DkaE3IISIiTuOZZdxhLHz2W+vs+Dxl/JfPdrEzt5gp/VswpEMMvVo0xc/n/GsEi4iIXCrPbJfQWGh+hfW58TkcK65gbUYBd/RryUOjO9G/TZSKWERE6o3nNkzHcdYKTgX7f/DWp9tyMU0YlxJvQzAREfE0HlzGY62vOz/+wVvz03LoFBdG25iQBg4lIiKeyHPLuEkSxPe0Pjc+w+GCMrZmFjKuu86KRUSkYXhuGYM1qjpnCxQePv3SJ9tyABirS9QiItJAPLuMO46zvu765PRLn6Tl0KtFBAlNAm0KJSIinsazyziyDcR2OT2qeu/Rk+w+clJnxSIi0qA8u4zBOjvOXA8njzB/aw5eBozqGmd3KhER8SAq407jARNz1yfMT8thQNsookP97U4lIiIeRGUc0wGi2lOy5QMOHy/TJWoREWlwKmOAjuMIzl1PrHcJ13ZuZncaERHxMCpjoLbDWLyo5Z64PYQH+todR0REPIxnLhRxlg3liSQ4ohnpvcHuKCIi4oF0ZgzM35bLUuMKovPWQXmh3XFERMTDeHwZV9U4WLg9l8KWIzAc1bB3sd2RRETEw9SpjA3DGGEYxh7DMNINw3jgPNvcaBjGTsMwdhiGMce5MevP6vQ8CsuqSel7NYTG/2CuahERkfp2wc+MDcPwBp4DhgNZwEbDMOabprnzjG3aAQ8CA0zTPGEYRkx9BXa2+VtzCA/05ar2sXBwLHz9OlSWgL9WbBIRkYZRlzPjPkC6aZoZpmlWAXOB8WdtMwN4zjTNEwCmaR5zbsz6UV5Vy+c7jzKqazP8fLyshSNqKiB9id3RRETEgximaf74BoYxCRhhmuZdp57fDvQ1TfPeM7b5CNgLDAC8gT+ZprnoHPuaCcwEiI2NTZ07d66zjoOSkhJCQi7ubHbDkRqe31rJ73sH0DHSG8xa+n81jcImXdjZ+X6nZbsUl3I8rkzH49rc7XjA/Y5Jx+Pa6nI8Q4YM2WyaZq9zveesW5t8gHbAYCARWGkYRlfTNL83NNk0zVnALIBevXqZgwcPdtKvhxUrVnCx+3vnzU3EhBYy87qheHsZ1osl1xGz7V1iBvQFX/tWbrqU43FlOh7X5m7HA+53TDoe13a5x1OXy9TZQNIZzxNPvXamLGC+aZrVpmkewDpLbnfJqRpAcUU1y/fkMbpb3HdFDNbCEdWlsH+ZfeFERMSj1KWMNwLtDMNoZRiGH3AzcPaQ44+wzooxDCMKaA9kODGn0y3efoSqGgfjzp6LutVACGhyellFERGR+nbBMjZNswa4F1gM7ALmmaa5wzCMPxuGMe7UZouBAsMwdgLLgd+ZpllQX6GdYX5aDs2bBtE9qcn33/D2heRRsGch1FTZE05ERDxKnT4zNk1zAbDgrNcePeN7E7jv1MPl5ZdU8tX+Au4e1BrDMH64QadxkDYHDqyEdsMaPqCIiHgUj5yBa8E3udQ6TMalJJx7g9ZDwC8Edn3csMFERMQjeWQZf5KWQ3JsKMnNQs+9gW8AJI+EbfNg3QvgqG3YgCIi4lE8royzC8vZePAE47rH//iG1zwOLa+CRQ/AK1dD7raGCSgiIh7H48r407QcAMZ0i/vxDUObweR3YdJsKMqCWYPh80egqqz+Q4qIiEfxuDKen5ZDSlITWkQGX3hjw4AuE+HejdDjNvjqGXj+CkhfWv9BRUTEY3hUGe/IKWJHTjETLnSJ+myBETDuGZi6ALz94K2J8P4MKMmrn6AiIuJRPKqM56w/jL+PF9f3SLy0HbQcAD9dA4MegB0fwnO9YctbcIH5vUVERH6Mx5RxaWUNH2/NYXS3OMKDfC99Rz7+MORBq5SjO8DHP4PXx1ozdlWedF5gERHxGM5aKMLlfZKWQ0llDZP7NnfODqOTrcvWW96Apf8H824HL19o0R/aXQPtr4XIttbnziIiIj/CY8p4zobDJMeG0rN5hPN26uUFqVOh+2TIXA97F8O+z+Hzh6xHRCurlNtdAy2vtM6qRUREzuIRZbw9u4htWUX837jO557+8nJ5+1pl2/JKuOYxOHHIKuV9n8Pm12D9i+AbDK0HW5OJdBgNQU2dn0NERBoljyjjt9cfJsDXiwk9zjP9pbNFtIA+M6xHVRkcXPXdWfOez+DTX1lTbna+DjqMskZri4iIx3L7Mi6prGH+1mzGdosnPPAyBm5dKr8g61J1+2utUdc5W6yR2Ds/go/vgU98oc2pYk4eBYFNLrxPERFxK25fxh9vzaa0qpZbnTVw63IYBiT0tB7D/ww5X1vFvONj2PdTawBYm6HQ+Tp8qs8zb7aIiLgdty5j0zSZs/4wHePCfrhusd0MAxJSrcfwxyD7a9jxAez4CPYtZgBesKctxHaBZl2sr7FdICxeI7RFRNyMW5fxtixrxq3HxtfTwC1nMQxITLUe1zwOWZs4tPRlWgaUQPYmq6S/FRjxXTGfWdLebv1XKSLi1tz6v+Bz1h8m0Neb8Q01cMsZDAOSenOwVSktBw+2XqsogqM74Mh2OHrqsfk1qCm33g+Ohk4ToOskSOxj3XIlIiKNhtuW8cmKauan5TAuJZ6wABsGbjlTQLg1mUiL/t+95qiF4xmQmwa7PoEtb8LGlyE8CbpcD10mQbOuuqQtItIIuG0Zf7Q1h/JqFxm4VR+8vCGqnfXoOgkqimHPAvjmPfjqP7Dm3xDV3irlrpMgso3diUVE5Dzcsoy/HbjVOT6MbonhdsdpGAFhkHKz9SgtsG6d+uY9WPH/rEd8D+h8vXX7VFRbu9OKiMgZ3LKMt2YWsiu3mL9c18W1B27Vl+BI6H2n9SjKgu0fwPb3YMkj1iOyLbQfYT2aX2HNICYiIrZxyzKes/4wwX7ejO/eiAZu1ZfwRBjwC+tx4pA1E9jehbBhFqz9j/V5dNvhVjG3G6bZwEREbOB2ZVxUXs0n23K4rkciIf5ud3iXJ6IF9J1pPSpPwv7lsHeRVdDb3wPDG5r3g+QR1ujsJkl2JxYR8Qhu11YfbcmmotrhvKUS3ZV/KHQaZz0ctZC92SrmPYvg84fh80eshS9SboaO46zPpEVEpF64VRl/O3Cra0I4XRI8ZOCWM3h5Q1If63H1o9YtU9vehW1z4eOfwWe/sVaa6nazNV2nJhgREXEqt/qv6teHT7Dn6EmeuL6r3VEat6atYfDvYdD9kLUR0uZas4Btfx+CY6xbpbrdBHEpuo9ZRMQJ3KqM56zPJMTfh3Ep8XZHcQ+G8d0Z84gnrSUg096BDS/DuuchugN0mQjthkOzFM38JSJyidymjEurTT7dlsOk1ESCNXDL+Xz8oOMY61F23Fptatv/YPlfrEdwDLQdZo3IbjNUo7JFRC6C27TWV9k1VNY43HfGLVcS1PS7+5hLjkH6F5C+xJoBLG0OGF7WHNnthlm3TTXrZndiERGX5hZlbJomy7OqSUlqQud4DdxqUCEx0P0W6/HtqOx9n8O+JbDscesRHENyaBeIr4LWg62zbBEROc0tynjToRPklJj86lqdFdvqzFHZQx8+dda8FPYtIXr3YpizDAKaQMex1mIWLQdqZLaICG5SxoVl1SSFejEmJc7uKHKmkBjofit0v5U1y5YwKKHGmppzx4fWKlNBUdZ9zp2vt1ak8vK2O7GIiC3cooyHd4rF91ggQX5ucThuyfTyheThkDwSqsuty9g7PrBum9o0G0KaQafx1hmz1mQWEQ+j9pKG5xv43exfVaXWzF/bP4DNr8GGl6wz5taDoPUQ6zNmTcspIm5OZSz28gu27lXuMtFak3nvIutz5owV1iQjYK0y1XqwVc6trrIWtxARcSMqY3EdAWHQ7UbrYZpwbJdVyhnLYes7sPEV67aphFSrmJNHQkJPu1OLiFw2lbG4JsOA2E7Wo989UFNlTc35bTmv+husfApSboFrHofgKLsTi4hcMpWxNA4+ftBygPUY+hCUF8JXz8Kaf1uXtq95HLpP1lzZItIoaciqNE6BTeDqR+DuVdYc2R//DF4bA3l77U4mInLRVMbSuMV0hKkLYOwzcPQbeKE/LP9/UF1hdzIRkTpTGUvj5+UFqVPg3k3Q+Tr48q9WKWd8aXcyEZE6URmL+wiJgYkvw+0fgumAN8bBh3dDab7dyUREfpQGcIn7aTMU7lkLq/4Oq/8FexZCbBfrnma/IPAN/u57v+BTz4PAL8S6pzkuRQPBRKRBqYzFPfkGWotVdJlk3QJ18giUHLVm/Koug6oS63tHzQ9/NrqjtQpVt5sgtFnDZxcRj6MyFvcW0wEmzT7/+zVVUF1qFXNVKRxaY00wsuRRWPonaDvMupc5eRT4BjRYbBHxLCpj8Ww+ftYjMMJ6Hp0MvaZD/j5Ie8dayOK9adYUnF0mWvcyJ6Tam1lE3I7KWORcotrB1Y/CkIfgwErYOsc6Y940G6La0zy0L5zsCKGxdicVETegMhb5MV7e0GaI9agohp0fwdY5tD7wJvzzHeg41jqTbnmVBn2JyCVTGYvUVUAY9LwDet7B+gVv09d7J2x5C3Z8CJHtrFLufst3l7xFROpI9xmLXILyoAS49i/wm90w4UVres7FD8LfO8BH90DWJmvlKRGROtCZscjl8A20zoa73wK526zPlLfNg61vQ7Nu1tlyt5us+5hFRM5DZ8YizhLXDcb+yzpbHv0PaxawT38FrwyDoiy704mIC1MZizhbQBj0vhPuXg23zoOiTHj5asjZYncyEXFRKmOR+mIY0P5auPNz8PaFV0dZU3OKiJxFZSxS32I6wl1fWBOKzL0V1r9kdyIRcTEqY5GGEBoLUz+D9iNh4f2w8PfgqLU7lYi4CJWxSEPxC4ab3oQr7oH1L8L/brPmwxYRj6cyFmlIXt4w4gkY+TTsXWR9jnzyiN2pRMRmKmMRO/SdCTe/Yy1I8cowOLrT7kQiYiOVsYhdkkfAtAVQWw2zr4X0L+xOJCI2URmL2Cm+O8z4AsKT4K2J1sCuypN2pxKRBqYyFrFbeCLcuRj6zLBue3ruCti72O5UItKAVMYirsA/FEY9DdMXg38IzLkR3psOJXl2JxORBqAyFnElzfvCT1bC4D/Ark/gud6wdY5WgBJxcypjEVfj4w+Df2/NbR2VDB/9FN6cAMcz7E4mIvVEZSziqqKTYdpCGP13yNoMz/eHNf+G2hq7k4mIk6mMRVyZlxf0vgt+th7aDIUlj8LLQ2Djf+HINypmETfhY3cAEamD8AS4+W3YNR8WPwSf3We97hsMCT0hqQ8k9rYewVH2ZhWRi6YyFmksDAM6jYeO4+DEQcjaBFkbIHODdfnaceosOaLVd+Xc/lpo0tzW2CJyYSpjkcbGMKBpK+vR7QbrtaoyyN1qFXPWRti/HLb9D774M9zxsXX2LCIuS2Us4g78gqBFf+sB1q1QeXtgzg3WSOwpn0Bcir0ZReS8NIBLxB0ZBsR0gCmfgl8ovDEBju6wO5WInIfKWMSdRbSAqZ9Y9y6/Pg6O7bY7kYicg8pYxN01bW2dIXt5w+tjrWUbRcSlqIxFPEFUW+tzY0yrkAv2251IRM6gMhbxFNHJcMd8qKm0LlmfOGh3IhE5RWUs4kliO1m3OlWVwGtjoTDT7kQigspYxPPEdYM7PoKKInh9DBRl251IxOOpjEU8UXwPuP1DKC2A18fiV1lgdyIRj6YyFvFUialw2/tQcpSUtEeh8LDdiUQ8lspYxJM17wuT38W/8gS8NBD2LbU7kYhHUhmLeLoW/dmc+ncIS4C3J8GKJ8HhsDuViEdRGYsI5UFxcOcSSLkZVjxhzWlddtzuWCIeQ2UsIha/IJjwAoz5FxxYaV22zt5sdyoRj6AyFpHvGAb0mgbTFwMGzB4Bm2Zbq0CJSL1RGYvIDyX0hJ98Ca0Gwqe/ho9+aq2ZLCL1ok5lbBjGCMMw9hiGkW4YxgM/st1EwzBMwzB6OS+iiNgiqCnc+i4MfhDS5sIrwzSntUg9uWAZG4bhDTwHjAQ6AbcYhtHpHNuFAr8E1js7pIjYxMsLBj8Ak9+DkzkwazDs+tTuVCJupy5nxn2AdNM0M0zTrALmAuPPsd1jwF+BCifmExFX0G4Y/GQlRLaB/02GpX+C2hq7U4m4jbqUcQJw5mzyWadeO80wjJ5Akmmanzkxm4i4kibNYdoiSJ0Kq/8Jb10Ppfl2pxJxC4Z5gVGShmFMAkaYpnnXqee3A31N07z31HMvYBkw1TTNg4ZhrAB+a5rmpnPsayYwEyA2NjZ17ty5TjuQkpISQkJCnLY/u+l4XJunH0+z3KW03/siVX7h7Oj8e06Gta/HdJfG0/+OXJ0nHs+QIUM2m6Z57jFVpmn+6APoByw+4/mDwINnPA8H8oGDpx4VQA7Q68f2m5qaajrT8uXLnbo/u+l4XJuOxzTN7K9N8x9dTPPPUaa5cbZpOhxOz3U59Hfk2jzxeIBN5nk6sS6XqTcC7QzDaGUYhh9wMzD/jDIvMk0zyjTNlqZptgTWAePMc5wZi4gbie9h3f7U8ir49Ffw8b1QXW53KpFG6YJlbJpmDXAvsBjYBcwzTXOHYRh/NgxjXH0HFBEXFtQUJr8LA++HrW/Bf6+BEwftTiXS6PjUZSPTNBcAC8567dHzbDv48mOJSKPh5Q1DH4KEVPhgJrw0CCa+Au2G251MpNHQDFwi4hzJI+AnKyA8Cd6+AZY/ATVVdqcSaRRUxiLiPE1bw52fW6s/ffkkvDgAMlbYnUrE5amMRcS5/ILguhfh1nlQWw1vjId5U6Aoy+5kIi5LZSwi9aP9tXDPOhjyMOxdDP/pDSv/BjWVdicTcTkqYxGpP74BMOh3cO8GaHs1LHsMnu8H+5banUzEpaiMRaT+NWkON70Ft31grZn89kR451bdBiVyispYRBpO26vhp2th2J+sgV3P9YUVT+rStXg8lbGINCwfP7jy13DvRkgeBSuegHdugWot+CaeS2UsIvYIT4AbXoVxz8L+L+DdKbovWTyWylhE7NXzDhj9D9i7CN6bZt0OJeJhVMYiYr/ed8KIv8LuT60pNWtr7E4k0qDqNDe1iEi9u+JuqK2CJY+Atx9MeN6a91rEA6iMRcR1DPgF1FbCssfB2wfGPgteuoAn7k9lLCKuZeDvrM+Nv/yrdYY8+h/WvckibkxlLCKuZ/CD1r3Ha/5lFfKIJ1XI4tZUxiLiegzDmhiktgrWPQ/evjD8MRWyuC2VsYi4JsOAa/+fVchfPQve/nD1I3anEqkXKmMRcV2GASOftgp51d/AUQNDHrJm8RJxIypjEXFtXl4w5t/W92v+BXsWwMinoM0Qe3OJOJHuGRAR1+flZU2beeu71kjrNyfAvDugKMvuZCJOoTIWkcaj/TVwzzoY8jDs/Rz+0xtW/V2rPkmjpzIWkcbFNwAG/Q7u3QBthsIXf4bn+0H6UruTiVwylbGINE5NmsPNb8Pk9wET3poIcydD4WG7k4lcNJWxiDRu7YZZl66HPgLpX8B/+sDKpzEctXYnE6kzlbGINH4+/jDwt3DvRmg3HJY9Tvu9z4Np2p1MpE5UxiLiPpokwU1vwlW/Je7IUlj+F7sTidSJylhE3M/Qh8ltNgxWPg0bXrY7jcgFadIPEXE/hsHe9vcQF+YLC34HwdHQeYLdqUTOS2fGIuKWTC9vmDQbkvrABzPgwCq7I4mcl8pYRNyXXxDcMheatoa5t8KRb+xOJHJOKmMRcW9BTeG298E/1LoX+cQhuxOJ/IDKWETcX3iiVcg1FfDW9VCab3cike9RGYuIZ4jpCLfOsxaXmHMjVJXanUjkNJWxiHiO5lfApFchZwvMm2KtACXiAlTGIuJZOoyCMf+C9CUw/+eapUtcgu4zFhHPkzoFSo5aM3T5BsKIv4KPn92pxIOpjEXEMw38HVSVwJp/w9GdcMNrEBZndyrxULpMLSKeyTBg+J9h4n+t+49fGggHV9udSjyUylhEPFvXSTDjCwgIh9fHwVfP6nNkaXAqYxGRmI4wY5k1uOvzh+HdKVB50u5U4kFUxiIiAAFhcOObMPwx2PUJvDwU8vbYnUo8hMpYRORbhgEDfgF3zIfyEzBrCGz/wO5U4gFUxiIiZ2t1FfxkJTTrAu9Ng0V/0AQhUq9UxiIi5xIWD1M+hb53w7rnrMFdxTl2pxI3pTIWETkfHz8Y+Vfr9qfcNHjxSti31O5U4oZUxiIiF9J1EsxcAaFx8PZEWPJHXbYWp1IZi4jURXR7uGsppE6DNf+C10ZDYabdqcRNqIxFROrKNxDG/gsmzbam0HzxSti9wO5U4gZUxiIiF6vLRPjJlxDRAubeYo22rqmyO5U0YipjEZFLEdkG7lwCfX5ijbaefS2cOGh3KmmkVMYiIpfKxx9GPQU3vQXH98OLA2Hnx3ankkZIZSwicrk6joWfrIKotjDvjlOThNTYnUoaEZWxiIgzRLSAaYu+u2w950YoL7Q7lTQSKmMREWfx8bMuW499Bg6shFeGQX663amkEVAZi4g4W+oUmDIfyo/DK0Mh/Qu7E4mLUxmLiNSHFv1hxnIIS4S3J8G6F8A07U4lLkplLCJSXyJawJ2fQ/IoWPQAzP+57keWc1IZi4jUJ/8QuPFNGPg72PImvDEOSvLsTiUuRmUsIlLfvLxg6MPWNJo5W+DloXDkG7tTiQtRGYuINJQuE2H6InDUwH+vhV2f2p1IXITKWESkIcX3gJnLIaaDNUHIrk/sTiQuQGUsItLQQpvBHfMhoSe8N123PonKWETEFv4hMPldiEqGuZPh0Fq7E4mNVMYiInYJjIDbP4TwRGv6zJwtdicSm6iMRUTsFBINd3wEAU3gzevh2C67E4kNVMYiInYLT4QpH4O3H7wxAY5n2J1IGpjKWETEFTRtbZ0h11bB6+OhKNvuRNKAVMYiIq4ipiPc/gGUn4A3xmumLg+iMhYRcSXxPWDyPCjKgjevs4pZ3J7KWETE1bToDze/Bfl74O0boPKk3YmknqmMRURcUdth1lzW2V/DO7fgXVNudyKpRypjERFX1XEsTHgeDq6m16ZfwsHVdieSeqIyFhFxZSk3w7QFgAGvjYaFD0BVmd2pxMlUxiIirq5Ffzb2/jf0ngHrX4AXr4TD6+1OJU6kMhYRaQQc3gEw+m/WAhO11fDqCPj8EaiusDuaOIHKWESkMWk9CH66BnrcDl89A7MGWYO8pFFTGYuINDYBYTDuGZj8PlQUwyvDYNnjUFNldzK5RCpjEZHGqt0wuGctdLsRVj4NLw+B3DS7U8klUBmLiDRmgU3guhfh5neg5BjMGgwLfgflhXYnk4ugMhYRcQcdRsG9G6DXnbDxFXg2Fba8BQ6H3cmkDlTGIiLuIjDCGnE9c4W1CtTHP4PZ1+rSdSOgMhYRcTdxKTB9MYx/3lobedZg+Oy3WnTChamMRUTckZcX9JgMP99sTRay6b/wbC/4+k1dunZBKmMREXcW2ARGPQUzv4TItjD/Xph9DeRstTuZnEFlLCLiCeK6wfRFMOFFOHHQug1q0R+gssTuZILKWETEcxgGdL8F7t0EqdNg3XPwfD/Yt9TuZB5PZSwi4mkCm8CYf8C0ReAbAG9PhPfvgtJ8u5N5rDqVsWEYIwzD2GMYRrphGA+c4/37DMPYaRjGNsMwvjAMo4Xzo4qIiFO16Ad3r4ZBD8COj+A/vWDrHDBNu5N5nAuWsWEY3sBzwEigE3CLYRidztpsC9DLNM1uwHvAU84OKiIi9cDHH4Y8aJVyVHv46Kfw5gTrlihpMHU5M+4DpJummWGaZhUwFxh/5gamaS43TfPb1a7XAYnOjSkiIvUqpoN12Xr03yFrMzzfH9b8G2pr7E7mEepSxglA5hnPs069dj53AgsvJ5SIiNjAywt63wU/Ww9thsKSR61R10e2253M7RnmBT4bMAxjEjDCNM27Tj2/Hehrmua959j2NuBeYJBpmpXneH8mMBMgNjY2de7cuZd/BKeUlJQQEhLitP3ZTcfj2nQ8rs/djqnBj8c0icpfS7t9s/CtLiG97XRy4kdaI7KdwBP/foYMGbLZNM1e53zTNM0ffQD9gMVnPH8QePAc2w0DdgExF9qnaZqkpqaazrR8+XKn7s9uOh7XpuNxfe52TLYdz8ljpvnmRNP8Y5hpzp1smmXHnbJbT/z7ATaZ5+nEulym3gi0MwyjlWEYfsDNwPwzNzAMowfwEjDONM1jddiniIg0BiHR/P/27jw4yvqO4/j7m8SIglcQrRoVDxwLakFqiudQ5Pa2eCBa1IoyVaR423asRWztqOBRq7XIYVUUj7ao4AFkitpOqkKsoCKo2JaikapFbT2Qb//4PUx3UpIm7JLf/jaf10wmu0+W3e93vpP95Pk9D89y2kwYOAGWzoE7Doe/1MWuquT83zB297WEpecnCXu+M919iZmNN7Njs4ddD3QCHjSzejOb1cTTiYhIasrK4JAxcPZTYGUwdQg8c6OucV1AFS15kLvPBmY32nZVzu3+Ba5LRESKTXVvGP0MPDoW5o2HtxbACXfCVjvGrix5ugKXiIi0XIdtYNhUOOaWsFx9x6GwfF7sqpKnMBYRkdYxg94j4dxa2HJ7uOdEmHs1fPlF7MqSpTAWEZGNs8NXYdR86H0mPDsJJveHFc/FripJCmMREdl4lVvCMTfDSdPh4waYNhRmDIf3lsauLCkKYxERyV+P42HMi3DkVbDiWfhFH5h1IaxZFbuyJCiMRUSkMCq3hMMvhgvroea88AlQt/SC+RPg0zWxqytqCmMRESmsjp1hyHVwwfOw71BYcH0I5bo7Ye3nsasrSgpjERHZNKr2gGFTYFRtONlrzqVwWw0sfgRcFwzJpTAWEZFNSPSjTAAAB+NJREFUa5cDYeSjMOIh2GwLeOgsvlF3Hiy4AT56J3Z1RUFhLCIim54ZdBsAo5+FYVP5tMNXYP41MLE73D8Cls2FdV/GrjKaFl0OU0REpCDKymG/E3lpdRV9998VFk6HRffCa4/BNrvBgd+GXqfD1jvFrrRNac9YRETi6LwXDBgPF70KJ02DzntC7QSY1ANmnAavP9Vu9pa1ZywiInFVVEKPE8LX+2/Cwrth0T2w9HHYdnfo813oNQI23yp2pZuM9oxFRKR4VO0J/a+Gca+Eq3ptvTM8cTlM7AFPXwX/XBm7wk1CYSwiIsWnojJc1evsJ+Cc+bB3P/jDrXDzAfDwOfD3+tgVFpSWqUVEpLhV9w7HlD94G+p+GZaxX34Quh4OB58P3QZBWdr7lmlXLyIi7cd2u8Pgn8BFS2DgBHj/LZhxKtx2EDx/F3zx79gVbjSFsYiIpKXDNnDIGBj7EnzrrnBi1+MXhbOwa38Kn6yOXWGrKYxFRCRN5RWw/7Bwuc0zZ0N1Dfz+uhDKj42D1ctjV9hiOmYsIiJpM4Ouh4av916HP/48XEjkhamw71FhL3q3PrGrbJb2jEVEpHR02QeOvQXGLYYjLoG3n4Mpg2DyAHhlVtFeRERhLCIipafTDtDvhzBuCQy9AT5pgJlnwK29w5W9iozCWERESldlR6gZBWMWwsl3h0+NmnFK+C9SRURhLCIipa+sHLofB+fMhX2GwJzLYM4VRbNsrTAWEZH2o7IjnPJrOPgCqLs9fHzjZx/HrkphLCIi7UxZOQy6NhxLXvYkTBsKa1bFLSnqq4uIiMRSMwqGPwD/eAMmHwnvLI5WisJYRETar30GwllzwB2mDIZlc6OUoTAWEZH2bacDYNQ8qOoK950crnPdxhTGIiIiW+8MZz0Be/cP17l+8gewbl2bvbzCWEREBGDzTnDqfVBzbrik5swz4PN/tclL69rUIiIi65VXwNDroWovWPo4lLVNTCqMRUREGuszOpxtXVbeJi+nZWoREZENaaMgBoWxiIhIdApjERGRyBTGIiIikSmMRUREIlMYi4iIRKYwFhERiUxhLCIiEpnCWEREJDKFsYiISGQKYxERkcgUxiIiIpEpjEVERCJTGIuIiESmMBYREYlMYSwiIhKZwlhERCQyhbGIiEhk5u5xXtjsPeDtAj7l9sDqAj5fbOqnuKmf4ldqPamf4taSfnZ39y4b+kG0MC40M3vB3b8eu45CUT/FTf0Uv1LrSf0Ut3z70TK1iIhIZApjERGRyEopjO+MXUCBqZ/ipn6KX6n1pH6KW179lMwxYxERkVSV0p6xiIhIkkoijM1ssJktNbPlZnZF7HryZWYrzOxlM6s3sxdi19NaZjbFzBrMbHHOtioze9rMlmXft4tZY2s00c/VZrYym1G9mQ2NWWNrmNmuZlZrZq+Y2RIzG5ttT3JGzfST5IzMrIOZ/cnMXsr6+XG2fQ8zq8ve5x4ws8rYtbZEM/1MM7O3cubTM3atrWFm5Wa2yMwey+7nNZ/kw9jMyoHbgCFAd2C4mXWPW1VBfNPdeyZ66v80YHCjbVcA89y9GzAvu5+KafxvPwCTshn1dPfZbVxTPtYCF7t7d6APcH72O5PqjJrqB9Kc0WdAP3f/GtATGGxmfYCfEfrZG/gA+E7EGlujqX4ALs2ZT328EjfKWODVnPt5zSf5MAZqgOXu/qa7fw7cDxwXuaZ2zd0XAO832nwcMD27PR04vk2LykMT/STL3Ve5+8Ls9keEN5RdSHRGzfSTJA8+zu5uln050A94KNue0nya6idZZlYNHAVMzu4bec6nFMJ4F+CvOff/RsK/iBkHnjKzF83s3NjFFMiO7r4qu/0OsGPMYgrkAjP7c7aMncSSbmNm1hXoBdRRAjNq1A8kOqNsCbQeaACeBt4APnT3tdlDknqfa9yPu6+fz7XZfCaZ2eYRS2ytm4DLgHXZ/c7kOZ9SCONSdJi7H0hYej/fzI6IXVAheTiFP+m/jIHbgb0Iy26rgBvjltN6ZtYJeBj4nruvyf1ZijPaQD/Jzsjdv3T3nkA1YfVv38gl5aVxP2a2H3Aloa+DgCrg8ogltpiZHQ00uPuLhXzeUgjjlcCuOfers23JcveV2fcG4DeEX8bUvWtmOwFk3xsi15MXd383e4NZB/yKxGZkZpsRguted38k25zsjDbUT+ozAnD3D4Fa4GBgWzOryH6U5PtcTj+Ds8ML7u6fAVNJZz6HAsea2QrCYdF+wM3kOZ9SCOPngW7ZmWyVwKnArMg1bTQz62hmW62/DQwEFjf/r5IwCxiZ3R4J/C5iLXlbH1qZE0hoRtnxrbuAV919Ys6PkpxRU/2kOiMz62Jm22a3twAGEI6D1wLDsoelNJ8N9fNazh9+Rji+msR83P1Kd692966EvJnv7iPIcz4lcdGP7L8s3ASUA1Pc/drIJW00M9uTsDcMUAHcl1o/ZjYD6Ev4FJN3gR8BvwVmArsRPq3rZHdP4qSoJvrpS1j+dGAFcF7O8daiZmaHAc8AL/PfY17fJxxnTW5GzfQznARnZGYHEE4AKifsMM109/HZe8P9hCXdRcDp2V5lUWumn/lAF8CAemB0zoleSTCzvsAl7n50vvMpiTAWERFJWSksU4uIiCRNYSwiIhKZwlhERCQyhbGIiEhkCmMREZHIFMYiIiKRKYxFREQiUxiLiIhE9h+I2B6SCYo2bgAAAABJRU5ErkJggg==\n",
420 |             "text/plain": [
421 |               "<Figure size 576x576 with 1 Axes>"
422 |             ]
423 |           },
424 |           "metadata": {
425 |             "needs_background": "light"
426 |           }
427 |         }
428 |       ]
429 |     },
430 |     {
431 |       "cell_type": "code",
432 |       "metadata": {
433 |         "id": "suv_B7cOvYq8",
434 |         "colab": {
435 |           "base_uri": "https://localhost:8080/"
436 |         },
437 |         "outputId": "8f339af5-0531-4546-a2eb-f6a69b5c8697"
438 |       },
439 |       "source": [
440 |         "model_path = f'{drive_path}models/{NAME}'\n",
441 |         "model.save(model_path)\n",
442 |         "\n",
443 |         "model = keras.models.load_model(model_path)\n",
444 |         "score = model.evaluate(test_x, test_y, verbose=10)  # evaluate against unseen test data\n",
445 |         "print(\"TEST LOSS:\", score[0])\n",
446 |         "print(\"TEST ACCURACY:\", score[1])"
447 |       ],
448 |       "execution_count": null,
449 |       "outputs": [
450 |         {
451 |           "output_type": "stream",
452 |           "text": [
453 |             "WARNING:absl:Found untraced functions such as lstm_cell_15_layer_call_and_return_conditional_losses, lstm_cell_15_layer_call_fn, lstm_cell_16_layer_call_and_return_conditional_losses, lstm_cell_16_layer_call_fn, lstm_cell_17_layer_call_and_return_conditional_losses while saving (showing 5 of 15). These functions will not be directly callable after loading.\n"
454 |           ],
455 |           "name": "stderr"
456 |         },
457 |         {
458 |           "output_type": "stream",
459 |           "text": [
460 |             "INFO:tensorflow:Assets written to: drive/MyDrive/Colab Notebooks/crypto_prediction/model3/models/LSTM-2021-08-23 20-19-28/assets\n"
461 |           ],
462 |           "name": "stdout"
463 |         },
464 |         {
465 |           "output_type": "stream",
466 |           "text": [
467 |             "INFO:tensorflow:Assets written to: drive/MyDrive/Colab Notebooks/crypto_prediction/model3/models/LSTM-2021-08-23 20-19-28/assets\n"
468 |           ],
469 |           "name": "stderr"
470 |         },
471 |         {
472 |           "output_type": "stream",
473 |           "text": [
474 |             "TEST LOSS: 0.3757164478302002\n",
475 |             "TEST ACCURACY: 0.8891063332557678\n"
476 |           ],
477 |           "name": "stdout"
478 |         }
479 |       ]
480 |     },
481 |     {
482 |       "cell_type": "code",
483 |       "metadata": {
484 |         "id": "xd9UqtJ1hmSD"
485 |       },
486 |       "source": [
487 |         "converter = tf.lite.TFLiteConverter.from_saved_model(model_path)  # path to the SavedModel directory\n",
488 |         "tflite_model = converter.convert()"
489 |       ],
490 |       "execution_count": null,
491 |       "outputs": []
492 |     },
493 |     {
494 |       "cell_type": "code",
495 |       "metadata": {
496 |         "colab": {
497 |           "base_uri": "https://localhost:8080/"
498 |         },
499 |         "id": "8q4GjSyVy5em",
500 |         "outputId": "d93c7889-ea51-4b1e-d3ce-a83e02a37528"
501 |       },
502 |       "source": [
503 |         "with open(f'{model_path}.tflite', 'wb') as f:\n",
504 |         "    f.write(tflite_model)\n",
505 |         "    print(\"Finished\")\n",
506 |         "    f.close()"
507 |       ],
508 |       "execution_count": null,
509 |       "outputs": [
510 |         {
511 |           "output_type": "stream",
512 |           "text": [
513 |             "Finished\n"
514 |           ],
515 |           "name": "stdout"
516 |         }
517 |       ]
518 |     },
519 |     {
520 |       "cell_type": "code",
521 |       "metadata": {
522 |         "id": "epvKcKD1zX0f"
523 |       },
524 |       "source": [
525 |         ""
526 |       ],
527 |       "execution_count": null,
528 |       "outputs": []
529 |     }
530 |   ]
531 | }


--------------------------------------------------------------------------------
/whl/tflite_runtime-2.3.1-cp37-cp37m-linux_armv6l.whl:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kamsec/crypto-bot/4d247efe76a71ca7a8e7a638f48f791a06f428b5/whl/tflite_runtime-2.3.1-cp37-cp37m-linux_armv6l.whl


--------------------------------------------------------------------------------
/whl/tflite_runtime-2.5.0-cp37-cp37m-linux_armv7l.whl:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/kamsec/crypto-bot/4d247efe76a71ca7a8e7a638f48f791a06f428b5/whl/tflite_runtime-2.5.0-cp37-cp37m-linux_armv7l.whl


--------------------------------------------------------------------------------
/wsgi.py:
--------------------------------------------------------------------------------
1 | 
2 | from app import app
3 | 
4 | if __name__ == "__main__":
5 |     app.run()
6 | 


--------------------------------------------------------------------------------