├── Graph ├── CV_Best_Model.png ├── P_L.png ├── Price_B1A1.png ├── Quantity.png ├── depth.png ├── pipline.png └── prediction.png ├── HFT_factors.ipynb ├── README.md ├── SGX-FTSE-China-A50-Index-Futures.pdf ├── data ├── CN_Futures_2014.01.02.csv └── order_book_3_2014_1_2.csv ├── data_process.ipynb ├── data_visualization.ipynb ├── feature_engineering.ipynb ├── images ├── CV_result.png ├── best_CV_result.png ├── best_CV_result_all.png ├── best_bid_ask.png ├── bid1ask1.png ├── depth_0000_0900_w.png ├── depth_0000_0915.png ├── depth_0900_1130_w.png ├── depth_0915_1130.png ├── depth_1130_1300.png ├── depth_1130_1300_w.png ├── depth_1300_1600.png ├── depth_1300_1600_w.png ├── prediction.png ├── rise_0000_0915_w.png ├── rise_0915_1130_w.png ├── rise_1130_1300_w.png ├── rise_1300_1600_w.png └── single_day_accuracy.png ├── model_fitting.ipynb ├── order_book_3_2014_1_2.csv └── order_book_4_2014_1_2.csv /Graph/CV_Best_Model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/CV_Best_Model.png -------------------------------------------------------------------------------- /Graph/P_L.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/P_L.png -------------------------------------------------------------------------------- /Graph/Price_B1A1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/Price_B1A1.png -------------------------------------------------------------------------------- /Graph/Quantity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/Quantity.png -------------------------------------------------------------------------------- /Graph/depth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/depth.png -------------------------------------------------------------------------------- /Graph/pipline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/pipline.png -------------------------------------------------------------------------------- /Graph/prediction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/Graph/prediction.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## High Frequency Trading Framework with Machine/Deep Learning 2 | 3 | In this project, we provide a framework/pipeline for high frequency trading using machine/deep learning techniques. More advanced feature engineering (with depth trade and quote data) and models (such as pre-trained models) can be applied in this framework. 4 | 5 | ### Target 6 | - Extract trading signals from level-II orderbook data 7 | - Predict orderbook dynamics using machine learning and deep learning techniques 8 | 9 | ### Data 10 | The SGX FTSE CHINA A50 INDEX Futures (新加坡交易所FTSE中国A50指数期货) tick depth data are used. 11 | 12 | ### Strategy Pipline 13 | 14 | 15 | ### Orderbook Signals 16 | We use limit orderbook data to develop trading signals, including **Depth Ratio**, **Rise Ratio**, and **Orderbook Imbalance (OBI)**. 17 | 18 | 19 | 20 | ### Price Series 21 | 22 | 23 | 24 | ### Feature Engineering & HFT Factors Design 25 | - Simple average depth ratio and OBI: 26 | 27 | 28 | 29 | 30 | - Weighted average depth ratio, OBI, and rise ratio: 31 | 32 | 33 | 34 | ### Model Fitting 35 | - Basic Models: 36 | * RandomForestClassifier 37 | * ExtraTreesClassifier 38 | * AdaBoostClassifier 39 | * GradientBoostingClassifier 40 | * Support Vector Machines 41 | * Other classifiers: Softmax, KNN, MLP, LSTM, etc. 42 | 43 | - Hyperparameters: 44 | * Training window: 30min 45 | * Test window: 10sec 46 | * Prediction label: 15min forward 47 | 48 | ### Performance Metrics 49 | - Prediction accuracy: 50 | 51 | 52 | 53 | - Prediction Accuracy Series: 54 | 55 | 56 | - Cross Validation Mean Accuracy: 57 | 58 | 59 | 60 | - Best Model: 61 | 62 | 63 | 64 | 65 | ### PnL Visualization 66 | 67 | 68 | ### Improvements 69 | 70 | **Feature Engineering** 71 | 72 | Other potentially useful signals: 73 | - volume imbalance signal 74 | - trade imbalance signal 75 | - technical indicators of bid and ask series (RSI, MACD...) 76 | - WAP/WPR, weighted average price, VWAP, TWAP 77 | - ..... 78 | 79 | Signal generating techniques: 80 | - consider different weights on different level of orderbook data for a particular signal 81 | - consider moving average with period n (hyperparameter) 82 | - consider weighted average of signals, such as weighted average of trade imbalance and orderbook imbalance 83 | - Lasso regression, genetic programming 84 | - ..... 85 | 86 | **Models** 87 | 88 | This project only provides a baseline. More advanced models are welcomed: 89 | - CNN 90 | - GRU/LSTM 91 | - XGBoost, AdaBoost, GBDT, LightGBM 92 | - Attention, Auto-encoder 93 | - TabNet 94 | - Pre-trained models 95 | - ..... 96 | 97 | **Performance Metrics** 98 | 99 | The performance metrics are subject to amendment, including the PnL calculation, commission fee consideration, etc. 100 | 101 | -------------------------------------------------------------------------------- /SGX-FTSE-China-A50-Index-Futures.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/SGX-FTSE-China-A50-Index-Futures.pdf -------------------------------------------------------------------------------- /data_process.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### **High Frequency Trading Strategies Design using ML and DL**" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# -*- coding: utf-8 -*-\n", 17 | "\"\"\"\n", 18 | "Created on Sun Mar 13 22:40:00 2022\n", 19 | "\n", 20 | "@author: Bradley\n", 21 | "\n", 22 | "HFT: Machine Learning Techniques on Full Orderbook Tick Data\n", 23 | "\"\"\"" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 2, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "import numpy as np\n", 33 | "import pandas as pd\n", 34 | "import matplotlib.pyplot as plt\n", 35 | "import seaborn as sns\n", 36 | "import statsmodels.api as sm\n", 37 | "from datetime import datetime\n", 38 | "import csv\n", 39 | "\n", 40 | "plt.rcParams['font.family'] = ['sans-serif']\n", 41 | "plt.rcParams['font.sans-serif'] = ['SimHei']\n", 42 | "plt.rcParams['axes.unicode_minus'] = False\n", 43 | "plt.rcParams['figure.figsize'] = (8,5) #提前设置图片形状大小\n", 44 | "\n", 45 | "%config InlineBackend.figure_format = 'svg'\n", 46 | "\n", 47 | "%matplotlib inline\n", 48 | "\n", 49 | "import warnings\n", 50 | "warnings.filterwarnings('ignore') # 忽略一些warnings\n", 51 | "\n", 52 | "from IPython.core.interactiveshell import InteractiveShell\n", 53 | "InteractiveShell.ast_node_interactivity = \"all\"\n", 54 | "\n", 55 | "from IPython.display import display\n", 56 | "pd.set_option('expand_frame_repr', False)\n", 57 | "pd.set_option('display.unicode.ambiguous_as_wide', True)\n", 58 | "pd.set_option('display.unicode.east_asian_width', True)\n", 59 | "pd.set_option('display.width', 180)" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "#### **Data Pre-processing**" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "**Goal: Transform from full orderbook data (`CN_Futures_2014.01.02.csv`) to organized tick data (`order_book_3_2014_1_2.csv`)**" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "**我们分析的对象是SGX China A50 Futures在2014年1月2日的高频订单簿数据。目标是根据其订单簿数据挖掘因子并实现使用机器学习算法滚动预测买卖时点。**" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": 7, 86 | "metadata": {}, 87 | "outputs": [], 88 | "source": [ 89 | "\n", 90 | "def order_book_tranform(year,month,day,path,best_price_number,series):\n", 91 | " ## read file\n", 92 | " def read_file(year,month,day,path,series):\n", 93 | " data = []\n", 94 | " if len(str(month)) == 1:\n", 95 | " month_ = '0' + str(month)\n", 96 | " else:\n", 97 | " month_ = str(month)\n", 98 | " if len(str(day)) == 1:\n", 99 | " day_ = '0' + str(day)\n", 100 | " else:\n", 101 | " day_ = str(day)\n", 102 | " datapath = str(path) + str(year) + '.' + str(month_) + '.' + str(day_) + '.csv'\n", 103 | " data = pd.read_csv(datapath)\n", 104 | " data = data[data.Series == series]\n", 105 | " return data.reset_index(drop = True)\n", 106 | "\n", 107 | " def insert(order_book_data,data_to_insert,ob_position):\n", 108 | " top = order_book_data[0:ob_position]\n", 109 | " bottom = order_book_data[ob_position:]\n", 110 | " return pd.concat((top,data_to_insert,bottom)).reset_index(drop = True)\n", 111 | "\n", 112 | " def draw_out(order_book_data,ob_position):\n", 113 | " top = order_book_data[0:ob_position]\n", 114 | " bottom = order_book_data[ob_position + 1:]\n", 115 | " return pd.concat((top,bottom)).reset_index(drop = True)\n", 116 | "\n", 117 | " def order_book_to_csv(order_book_bid,order_book_ask,data,i):\n", 118 | " \n", 119 | " order_book_bid_sum = order_book_bid[['Price','QuantityDifference']].groupby(by = ['Price'],as_index = False,sort = False).sum()\n", 120 | " order_book_ask_sum = order_book_ask[['Price','QuantityDifference']].groupby(by = ['Price'],as_index = False).sum()\n", 121 | " order_book_bid_sum = order_book_bid_sum[order_book_bid_sum.QuantityDifference != 0.0].reset_index(drop = True)\n", 122 | " order_book_ask_sum = order_book_ask_sum[order_book_ask_sum.QuantityDifference != 0.0].reset_index(drop = True)\n", 123 | " order_book_bid_ask = pd.concat([order_book_bid_sum[['Price','QuantityDifference']],order_book_ask_sum[['Price','QuantityDifference']]],axis = 1) \n", 124 | " \n", 125 | " with open('order_book_'+str(best_price_number)+'_'+str(year)+'_'+str(month)+'_'+str(day)+'.csv','a') as f:\n", 126 | " order_book = csv.writer(f)\n", 127 | " order_book.writerow([\"TimeStamp\",data.TimeStamp[i-1:i].iloc[0]])\n", 128 | " order_book = csv.writer(f,delimiter=',')\n", 129 | " for i in range(0,min(len(order_book_bid_ask),best_price_number),1):\n", 130 | " order_book.writerow(order_book_bid_ask[i:i+1].values.tolist()[0])\n", 131 | " return order_book_bid_sum,order_book_ask_sum\n", 132 | "\n", 133 | " data = read_file(year,month,day,path,series)\n", 134 | " \n", 135 | " with open('order_book_' + str(best_price_number) + '_' + str(year) + '_' + str(month) + '_' + str(day) + '.csv', 'wb') as csvfile:\n", 136 | " f = csv.writer(csvfile) \n", 137 | "\n", 138 | " data[['QuantityDifference']] = data[['QuantityDifference']].astype(float)\n", 139 | " data['QuantityDifference_'] = data['QuantityDifference']\n", 140 | " data_ask = data[(data.BidOrAsk == 'A')].reset_index(drop=True)\n", 141 | " data_bid = data[(data.BidOrAsk == 'B')].reset_index(drop=True)\n", 142 | " order_book_bid = []\n", 143 | " order_book_ask = []\n", 144 | " x1 = data[(data.BidOrAsk == 'A')].TimeStamp.unique()\n", 145 | " x2 = data[(data.BidOrAsk == 'B')].TimeStamp.unique()\n", 146 | " temp_ask = 0\n", 147 | " temp_bid = 0\n", 148 | " \n", 149 | " def first_order_create(index_,data):\n", 150 | " timestamp = data.TimeStamp.unique()[index_]\n", 151 | " print('timestamp = %s'%(timestamp)) \n", 152 | " bid = []\n", 153 | " ask = []\n", 154 | " timestamp_ = []\n", 155 | " index_find = data[data['TimeStamp'].str.contains(timestamp)].index[-1]\n", 156 | " y = data[:index_find + 1]\n", 157 | " bid.append(y[(y.BidOrAsk == 'B')][[\"Price\",\"OrderNumber\",\"QuantityDifference\",\"QuantityDifference_\"]]) # bid\n", 158 | " ask.append(y[(y.BidOrAsk == 'A')][[\"Price\",\"OrderNumber\",\"QuantityDifference\",\"QuantityDifference_\"]]) # ask\n", 159 | " a = bid[0].sort_values(by=['Price'], ascending = [False])\n", 160 | " b = ask[0].sort_values(by=['Price'], ascending = [True])\n", 161 | " # a = bid[0].sort(['Price'],ascending = [False])\n", 162 | " # b = ask[0].sort(['Price'],ascending = [True])\n", 163 | " order_book_bid = a[a.QuantityDifference != 0].reset_index(drop = True)\n", 164 | " order_book_ask = b[b.QuantityDifference != 0].reset_index(drop = True)\n", 165 | " order_book_bid_sum = order_book_bid[['Price','QuantityDifference']].groupby(by = ['Price'],as_index = False,sort = False).sum()\n", 166 | " order_book_ask_sum = order_book_ask[['Price','QuantityDifference']].groupby(by = ['Price'],as_index = False).sum()\n", 167 | " \n", 168 | " if len(order_book_bid_sum[order_book_bid_sum.QuantityDifference == 0.0]) != 0 and len(order_book_ask_sum[order_book_ask_sum.QuantityDifference == 0.0]) != 0:\n", 169 | " \n", 170 | " print('Exist Bid Ask Order Book Price = Zero')\n", 171 | " price_bid_zero = order_book_bid_sum[order_book_bid_sum.QuantityDifference == 0.0]['Price'][0]\n", 172 | " price_ask_zero = order_book_ask_sum[order_book_ask_sum.QuantityDifference == 0.0]['Price'][0]\n", 173 | " order_book_bid = order_book_bid[order_book_bid.Price != price_bid_zero]\n", 174 | " order_book_ask = order_book_ask[order_book_ask.Price != price_ask_zero]\n", 175 | " elif len(order_book_bid_sum[order_book_bid_sum.QuantityDifference == 0.0]) != 0 and len(order_book_ask_sum[order_book_ask_sum.QuantityDifference == 0.0]) == 0:\n", 176 | " \n", 177 | " print('Exist Bid Order Book Price = Zero')\n", 178 | " price_bid_zero = order_book_bid_sum[order_book_bid_sum.QuantityDifference == 0.0]['Price'][0]\n", 179 | " order_book_bid = order_book_bid[order_book_bid.Price != price_bid_zero]\n", 180 | "\n", 181 | " elif len(order_book_bid_sum[order_book_bid_sum.QuantityDifference == 0.0]) == 0 and len(order_book_ask_sum[order_book_ask_sum.QuantityDifference == 0.0]) != 0:\n", 182 | " \n", 183 | " print('Exist Ask Order Book Price = Zero')\n", 184 | " price_ask_zero = order_book_ask_sum[order_book_ask_sum.QuantityDifference == 0.0]['Price'][0]\n", 185 | " order_book_ask = order_book_ask[order_book_ask.Price != price_ask_zero]\n", 186 | "\n", 187 | " order_book_bid_sum = order_book_bid_sum[order_book_bid_sum.QuantityDifference != 0].reset_index(drop = True)\n", 188 | " order_book_ask_sum = order_book_ask_sum[order_book_ask_sum.QuantityDifference != 0].reset_index(drop = True)\n", 189 | " order_book_bid_ask = pd.concat([order_book_bid_sum[['Price','QuantityDifference']],order_book_ask_sum[['Price','QuantityDifference']]],axis = 1) \n", 190 | " \n", 191 | " return order_book_bid, order_book_ask, order_book_bid_ask, timestamp, y, index_find\n", 192 | " \n", 193 | " def with_first_order_book(best_price_number,year,month,day,timestamp,order_book_bid_ask,index_):\n", 194 | " with open('order_book_'+str(best_price_number)+'_'+str(year)+'_'+str(month)+'_'+str(day)+'.csv','a') as f: \n", 195 | " order_book = csv.writer(f)\n", 196 | " if index_ == 0:\n", 197 | " order_book.writerow([\"Bid\",\"Bid_Quantity\",\"Ask\",\"Ask_Quantity\"])\n", 198 | " order_book.writerow([\"TimeStamp\",timestamp])\n", 199 | " order_book = csv.writer(f,delimiter=',')\n", 200 | " for i in range(0,min(len(order_book_bid_ask),best_price_number),1):\n", 201 | " order_book.writerow(order_book_bid_ask[i:i+1].values.tolist()[0]) \n", 202 | " \n", 203 | " # 建立初始委託簿\n", 204 | " first_order_book_data_lenth = 0\n", 205 | " order_book_bid_time = 0\n", 206 | " order_book_ask_time = 0\n", 207 | " \n", 208 | " for time in range(0,1000,1):\n", 209 | " index_ = time\n", 210 | " order_book_bid, order_book_ask, order_book_bid_ask,\\\n", 211 | " timestamp, y, index_find = first_order_create(index_, data) \n", 212 | "\n", 213 | " if len(order_book_bid) != 0 and len(order_book_ask) != 0:\n", 214 | " with_first_order_book(best_price_number,year,month,day,timestamp,order_book_bid_ask,index_) \n", 215 | " break\n", 216 | " elif len(order_book_bid) == 0 and len(order_book_ask) != 0: \n", 217 | " with_first_order_book(best_price_number,year,month,day,timestamp,order_book_bid_ask,index_)\n", 218 | " temp_ask +=1\n", 219 | " elif len(order_book_bid) != 0 and len(order_book_ask) == 0:\n", 220 | " with_first_order_book(best_price_number,year,month,day,timestamp,order_book_bid_ask,index_)\n", 221 | " temp_bid +=1\n", 222 | " \n", 223 | " print('-------------------------------------------')\n", 224 | " print('index_find = %s'%(index_find))\n", 225 | " \n", 226 | " # 这里作为举例，只取了100条，可以取全部length的data，生成的结果就是\"order_book_3_2014_1_2.csv\"文件\n", 227 | " for i in range(index_find + 1,100,1):#len(data), 1):\n", 228 | " print('---------------------------------')\n", 229 | " print(data[['Price','QuantityDifference','BidOrAsk','TimeStamp']][i:i+1])\n", 230 | " print(i ,temp_bid, temp_ask)\n", 231 | " print(data.TimeStamp[i], x2[temp_bid], x1[temp_ask])\n", 232 | " time_second = int(data[i:i+1].TimeStamp.iloc[0][18]) + int(data[i:i+1].TimeStamp.iloc[0][17])*10 +\\\n", 233 | " int(data[i:i+1].TimeStamp.iloc[0][15])*60 + int(data[i:i+1].TimeStamp.iloc[0][14])*600 +\\\n", 234 | " int(data[i:i+1].TimeStamp.iloc[0][12])*3600 + int(data[i:i+1].TimeStamp.iloc[0][11])*36000\n", 235 | "\n", 236 | " if time_second > 57600:\n", 237 | " break\n", 238 | " if time_second == 32400 and time_second >= 57300:\n", 239 | " order_book_bid = order_book_bid.sort(['Price'],ascending = [False]).reset_index(drop = True)\n", 240 | " order_book_ask = order_book_ask.sort(['Price'],ascending = [True]).reset_index(drop = True)\n", 241 | " pass \n", 242 | " \n", 243 | " if data.BidOrAsk[i] == 'A':\n", 244 | " data_ask_Quantity = data.BestQuantity[i]\n", 245 | " if int(data[['QuantityDifference']][i:i+1].values) > 0 :\n", 246 | " if order_book_bid.Price[0] >= data[i:i+1].Price.iloc[0] and time_second < 32400:\n", 247 | " for k in range(0,len(order_book_bid)):\n", 248 | " diff = order_book_bid.QuantityDifference_[k] - data[i:i+1].QuantityDifference_.iloc[0] \n", 249 | " if order_book_bid.Price[k] >= data[i:i+1].Price.iloc[0] and diff >= 0:\n", 250 | " order_book_bid.QuantityDifference_[k] = diff\n", 251 | " data[i:i+1].QuantityDifference_.iloc[0] = 0\n", 252 | " break\n", 253 | " elif order_book_bid.Price[k] >= data[i:i+1].Price.iloc[0] and diff < 0: \n", 254 | " order_book_bid.QuantityDifference_[k] = 0\n", 255 | " data[i:i+1].QuantityDifference_.iloc[0] = -diff\n", 256 | " pass\n", 257 | " else:\n", 258 | " break\n", 259 | " if data.TimeStamp[i] == x1[temp_ask]:\n", 260 | " \n", 261 | " position_ = int(data[['OrderBookPosition']][i:i+1].iloc[0]) - 1 \n", 262 | " order_book_ask = insert(order_book_ask,data[['Price','OrderNumber','QuantityDifference','QuantityDifference_']][i:i+1],position_)\n", 263 | " if time_second > 32400 and time_second < 57300: \n", 264 | " if position_ == 0 and len(order_book_ask) > 1: \n", 265 | " if order_book_ask[position_ + 1:position_ + 1 + 1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 266 | " print('Some error1(Ask & Q>0 & timestamp not change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 267 | " break\n", 268 | " else:\n", 269 | " pass\n", 270 | " elif 0 < position_< (len(order_book_ask)-1):\n", 271 | " if order_book_ask[position_ + 1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_ask[position_ - 1:position_ - 1 + 1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]:\n", 272 | " print('Some error1(Ask & Q>0 & timestamp not change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 273 | " break\n", 274 | " else:\n", 275 | " pass\n", 276 | " elif position_ == len(order_book_ask)-1: \n", 277 | " if order_book_ask[position_ - 1:position_ - 1 + 1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 278 | " print('Some error1(Ask & Q>0 & timestamp not change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 279 | " break\n", 280 | " else:\n", 281 | " pass\n", 282 | " elif position_ == 0 and len(order_book_ask) == 1:\n", 283 | " pass\n", 284 | " else:\n", 285 | " pass\n", 286 | " elif data.TimeStamp[i] != x1[temp_ask]:\n", 287 | " if temp_ask == 0:\n", 288 | " temp_ask = temp_ask + 1\n", 289 | " best_price = data[i:(i+1)]['BestPrice']\n", 290 | " position_ = int(data[['OrderBookPosition']][i:i+1].iloc[0]) - 1 \n", 291 | " order_book_ask = insert(order_book_ask,data[['Price','OrderNumber','QuantityDifference','QuantityDifference_']][i:i+1],position_) \n", 292 | "\n", 293 | " if time_second > 32400 and time_second < 57300:\n", 294 | " if position_ == 0 and len(order_book_ask) > 1:\n", 295 | " if order_book_ask[position_+1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]:\n", 296 | " print('Some error2(Ask & Q>0 & timestamp change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 297 | " break\n", 298 | " else:\n", 299 | " pass\n", 300 | " elif 0 < position_< len(order_book_ask)-1:\n", 301 | " if order_book_ask[position_+ 1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_ask[position_-1:position_-1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 302 | " print('Some error2(Ask & Q>0 & timestamp change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 303 | " break\n", 304 | " else:\n", 305 | " pass\n", 306 | " elif position_ == len(order_book_ask)-1: \n", 307 | " if order_book_ask[position_-1:position_-1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 308 | " print('Some error2(Ask & Q>0 & timestamp change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 309 | " break\n", 310 | " else:\n", 311 | " pass\n", 312 | " elif position_ == 0 and len(order_book_ask) == 1:\n", 313 | " pass\n", 314 | " else:\n", 315 | " pass\n", 316 | " else:\n", 317 | " order_book_bid_sum,order_book_ask_sum = order_book_to_csv(order_book_bid,order_book_ask,data,i)\n", 318 | " if time_second > 32400 and time_second < 57300:\n", 319 | " if round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) > 0.03 or\\\n", 320 | " round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) < 0:\n", 321 | " if data[i-1:i].BidOrAsk.iloc[0] == 'A':\n", 322 | " if order_book_ask_sum[0:1].values.tolist()[0][1] == data[i-1:i].BestQuantity.iloc[0]:\n", 323 | " pass\n", 324 | " else:\n", 325 | " # print 'Best ask quantity is false'\n", 326 | " pass\n", 327 | " #break \n", 328 | " else:\n", 329 | " j = i - 1\n", 330 | " while j >= 1:\n", 331 | " if data[j-1:j].BidOrAsk.iloc[0] == 'A':\n", 332 | " if order_book_ask_sum[0:1].values.tolist()[0][1] == data[j-1:j].BestQuantity.iloc[0]:\n", 333 | " break\n", 334 | " else:\n", 335 | " j = j - 1\n", 336 | " pass\n", 337 | " else:\n", 338 | " pass\n", 339 | " else:\n", 340 | " pass\n", 341 | " position_ = int(data[['OrderBookPosition']][i:i+1].iloc[0]) - 1 \n", 342 | " temp_ask = temp_ask + 1\n", 343 | " \n", 344 | " order_book_ask = insert(order_book_ask,data[['Price','OrderNumber','QuantityDifference','QuantityDifference_']][i:i+1],position_)\n", 345 | " if time_second > 32400 and time_second < 57300:\n", 346 | " if position_ == 0:\n", 347 | " \n", 348 | " if order_book_ask[position_+1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 349 | " print('Some error3(Ask & Q>0 & timestamp change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 350 | " break\n", 351 | " else:\n", 352 | " pass\n", 353 | " elif 0 < position_< len(order_book_ask)-1:\n", 354 | " if order_book_ask[position_+1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 355 | " print('Some error3(Ask & Q>0 & timestamp change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 356 | " break\n", 357 | " else:\n", 358 | " pass\n", 359 | " elif position_ == len(order_book_ask)-1: \n", 360 | " if order_book_ask[position_-1:position_-1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 361 | " print('Some error3(Ask & Q>0 & timestamp change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 362 | " break\n", 363 | " else:\n", 364 | " pass\n", 365 | " elif position_ == 0 and len(order_book_ask[0]) == 1:\n", 366 | " pass\n", 367 | " else:\n", 368 | " pass\n", 369 | " elif int(data[['QuantityDifference']][i:i+1].values) < 0: \n", 370 | " if data.TimeStamp[i] == x1[temp_ask]: \n", 371 | " order_number_ = data['OrderNumber'][i:i + 1].iloc[0]\n", 372 | " position_ = order_book_ask[order_book_ask.OrderNumber == order_number_].index[0] \n", 373 | " price_ = data[i:i+1]['Price'].iloc[0]\n", 374 | " if time_second > 32400 and time_second < 57300:\n", 375 | " if position_ == 0 and len(order_book_ask) > 1:\n", 376 | " if order_book_ask[position_+1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 377 | " print('Some error4(Ask & Q<0 & timestamp not change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 378 | " break\n", 379 | " else: \n", 380 | " pass\n", 381 | " elif 0 < position_< len(order_book_ask)-1:\n", 382 | " if order_book_ask[position_+1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 383 | " print('Some error4(Ask & Q<0 & timestamp not change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 384 | " break\n", 385 | " else: \n", 386 | " pass \n", 387 | " elif position_ == len(order_book_ask)-1:\n", 388 | " if position_ > 0 and order_book_ask[position_-1:position_-1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 389 | " print('Some error4(Ask & Q<0 & timestamp not change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 390 | " break\n", 391 | " elif position_ == 0:\n", 392 | " pass \n", 393 | " else: \n", 394 | " pass\n", 395 | " elif position_ == 0 and len(order_book_ask) == 1: \n", 396 | " pass\n", 397 | " else:\n", 398 | " pass\n", 399 | " if order_book_ask[(order_book_ask.OrderNumber == order_number_)&(order_book_ask.Price == price_)]['QuantityDifference'].iloc[0] == abs(data[i:i+1]['QuantityDifference'].iloc[0]):\n", 400 | " order_book_ask = order_book_ask.drop(order_book_ask.index[[position_]]).reset_index(drop = True) \n", 401 | " else:\n", 402 | " order_book_ask['QuantityDifference'][order_book_ask.OrderNumber == order_number_] = order_book_ask['QuantityDifference'][order_book_ask.OrderNumber == order_number_] + data[i:i+1]['QuantityDifference'].iloc[0]\n", 403 | "\n", 404 | " elif data.TimeStamp[i] != x1[temp_ask]:\n", 405 | " \n", 406 | " order_book_bid_sum,order_book_ask_sum = order_book_to_csv(order_book_bid,order_book_ask,data,i)\n", 407 | " if time_second > 32400 and time_second < 57300:\n", 408 | " if round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) > 0.03 or\\\n", 409 | " round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) < 0:\n", 410 | " if data[i-1:i].BidOrAsk.iloc[0] == 'A':\n", 411 | " if order_book_ask_sum[0:1].values.tolist()[0][1] == data[i-1:i].BestQuantity.iloc[0]:\n", 412 | " pass\n", 413 | " else:\n", 414 | " print('Best ask quantity is false')\n", 415 | " #break\n", 416 | " pass\n", 417 | " else:\n", 418 | " j = i - 1\n", 419 | " while j >= 1:\n", 420 | " if data[j-1:j].BidOrAsk.iloc[0] == 'A':\n", 421 | " if order_book_ask_sum[0:1].values.tolist()[0][1] == data[j-1:j].BestQuantity.iloc[0]:\n", 422 | " break\n", 423 | " else:\n", 424 | " j = j - 1\n", 425 | " pass\n", 426 | " else:\n", 427 | " pass\n", 428 | " order_number_ = data['OrderNumber'][i : i + 1].iloc[0]\n", 429 | " position_ = order_book_ask[order_book_ask.OrderNumber == order_number_].index[0]\n", 430 | " price_ = data[i:i+1]['Price'].iloc[0]\n", 431 | " temp_ask = temp_ask + 1\n", 432 | " if time_second > 32400 and time_second < 57300:\n", 433 | " if position_ == 0 and len(order_book_ask) > 1:\n", 434 | " if order_book_ask[position_ + 1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]:\n", 435 | " print('Some error5(Ask & Q<0 & timestamp change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 436 | " break\n", 437 | " else:\n", 438 | " pass\n", 439 | " elif 0 < position_< len(order_book_ask)-1: \n", 440 | " if order_book_ask[position_ + 1:position_+1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 441 | " print('Some error5(Ask & Q<0 & timestamp change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 442 | " break\n", 443 | " else:\n", 444 | " pass\n", 445 | " elif position_ == len(order_book_ask)-1:\n", 446 | " if position_ > 0 and order_book_ask[position_-1:position_-1 + 1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 447 | " print('Some error5(Ask & Q<0 & timestamp change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 448 | " break\n", 449 | " elif position_ == 0:\n", 450 | " pass \n", 451 | " else: \n", 452 | " pass\n", 453 | " elif position_ == 0 and len(order_book_ask) == 1:\n", 454 | " pass \n", 455 | " else:\n", 456 | " pass\n", 457 | " if order_book_ask[(order_book_ask.OrderNumber == order_number_)&(order_book_ask.Price == price_)]['QuantityDifference'].iloc[0] == abs(data[i:i+1]['QuantityDifference'].iloc[0]): \n", 458 | " order_book_ask = order_book_ask.drop(order_book_ask.index[[position_]]).reset_index(drop = True)\n", 459 | " else:\n", 460 | " order_book_ask['QuantityDifference'][order_book_ask.OrderNumber == order_number_] = order_book_ask['QuantityDifference'][order_book_ask.OrderNumber == order_number_] + data[i:i+1]['QuantityDifference'].iloc[0]\n", 461 | "\n", 462 | " elif data.BidOrAsk[i] == 'B':\n", 463 | " data_bid_Quantity = data.BestQuantity[i]\n", 464 | " if int(data[['QuantityDifference']][i:i+1].values) > 0: \n", 465 | " \n", 466 | " if order_book_ask.Price[0] <= data[i:i+1].Price.iloc[0] and time_second < 32400:\n", 467 | " for k in range(0,len(order_book_ask)):\n", 468 | " diff = order_book_ask.QuantityDifference_[k] - data[i:i+1].QuantityDifference_.iloc[0]\n", 469 | " if order_book_ask.Price[k] <= data[i:i+1].Price.iloc[0] and diff >= 0:\n", 470 | " order_book_ask.QuantityDifference_[k] = diff\n", 471 | " data[i:i+1].QuantityDifference_.iloc[0] = 0\n", 472 | " break\n", 473 | " elif order_book_ask.Price[k] <= data[i:i+1].Price.iloc[0] and diff < 0:\n", 474 | " order_book_ask.QuantityDifference_[k] = 0\n", 475 | " data[i:i+1].QuantityDifference_.iloc[0] = - diff\n", 476 | " pass\n", 477 | " else:\n", 478 | " break\n", 479 | " if data.TimeStamp[i] == x2[temp_bid]:\n", 480 | " position_ = int(data[['OrderBookPosition']][i:i+1].iloc[0]) - 1 \n", 481 | " order_book_bid = insert(order_book_bid,data[['Price','OrderNumber','QuantityDifference','QuantityDifference_']][i:i+1],position_) \n", 482 | " if time_second > 32400 and time_second < 57300:\n", 483 | " if position_ == 0 and len(order_book_bid) > 1: \n", 484 | " if order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 485 | " print('Some error1(Bid & Q>0 & timestamp not change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 486 | " break\n", 487 | " else: \n", 488 | " pass \n", 489 | " elif 0 < position_< len(order_book_bid)-1: \n", 490 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 491 | " print('Some error1(Bid & Q>0 & timestamp not change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 492 | " break \n", 493 | " else: \n", 494 | " pass \n", 495 | " elif position_ == len(order_book_bid)-1 and len(order_book_bid) > 1: \n", 496 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 497 | " print('Some error1(Bid & Q>0 & timestamp not change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 498 | " break\n", 499 | " else: \n", 500 | " pass\n", 501 | " elif position_ == 0 and len(order_book_bid[temp_bid]) == 1:\n", 502 | " pass \n", 503 | " else:\n", 504 | " pass\n", 505 | " elif data.TimeStamp[i] != x2[temp_bid]:\n", 506 | " if temp_bid == 0:\n", 507 | " best_price = data[i:(i+1)]['BestPrice'] \n", 508 | " position_ = int(data[['OrderBookPosition']][i:i+1].iloc[0]) - 1 \n", 509 | " temp_bid = temp_bid + 1 \n", 510 | " order_book_bid = insert(order_book_bid,data[['Price','OrderNumber','QuantityDifference','QuantityDifference_']][i:i+1],position_) \n", 511 | " if time_second > 32400 and time_second < 57300:\n", 512 | " if position_ == 0 and len(order_book_bid) > 1:\n", 513 | " if order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0] or order_book_bid['Price'][0:1].iloc[0] != data['BestPrice'][i]: \n", 514 | " print('Some error2(Bid & Q>0 & timestamp change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0]) + data['OrderNumber'][i:i+1].iloc[0])\n", 515 | " break\n", 516 | " else: \n", 517 | " pass \n", 518 | " elif 0 < position_< len(order_book_bid)-1: \n", 519 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 520 | " print('Some error2(Bid & Q>0 & timestamp change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 521 | " break\n", 522 | " else: \n", 523 | " pass\n", 524 | " elif position_ == len(order_book_bid)-1:\n", 525 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid['Price'][0:1].iloc[0] != data['BestPrice'][i]: \n", 526 | " print('Some error2(Bid & Q>0 & timestamp change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 527 | " pass\n", 528 | " else:\n", 529 | " pass\n", 530 | " elif position_ == 0 and len(order_book_bid) == 1: \n", 531 | " pass\n", 532 | " else:\n", 533 | " pass\n", 534 | " else:\n", 535 | " if time_second > 32400 and time_second < 57300:\n", 536 | " if round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) > 0.03 or\\\n", 537 | " round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) < 0:\n", 538 | " order_book_bid_sum,order_book_ask_sum = order_book_to_csv(order_book_bid,order_book_ask,data,i) \n", 539 | " if data[i-1:i].BidOrAsk.iloc[0] == 'B':\n", 540 | " if order_book_bid_sum[0:1].values.tolist()[0][1] == data[i-1:i].BestQuantity.iloc[0]:\n", 541 | " pass\n", 542 | " else:\n", 543 | " print('Best bid quantity is false')\n", 544 | " #break\n", 545 | " pass\n", 546 | " else:\n", 547 | " j = i - 1\n", 548 | " while j >= 1:\n", 549 | " if data[j-1:j].BidOrAsk.iloc[0] == 'B':\n", 550 | " if order_book_bid_sum[0:1].values.tolist()[0][1] == data[j-1:j].BestQuantity.iloc[0]:\n", 551 | " break\n", 552 | " else:\n", 553 | " print('Best bid quantity is false')\n", 554 | " #break\n", 555 | " pass\n", 556 | " else:\n", 557 | " j = j - 1\n", 558 | " pass\n", 559 | " else:\n", 560 | " pass\n", 561 | " position_ = int(data[['OrderBookPosition']][i:i+1].iloc[0]) - 1 \n", 562 | " temp_bid = temp_bid + 1 \n", 563 | " order_book_bid = insert(order_book_bid,data[['Price','OrderNumber','QuantityDifference','QuantityDifference_']][i:i+1],position_) \n", 564 | " if time_second > 32400 and time_second < 57300:\n", 565 | " if position_ == 0 and len(order_book_bid) > 1:\n", 566 | " if order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0] or order_book_bid['Price'][0:1].iloc[0] != data['BestPrice'][i]: \n", 567 | " print('Some error3(Bid & Q>0 & timestamp change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 568 | " break\n", 569 | " else: \n", 570 | " pass\n", 571 | " elif 0 < position_< len(order_book_bid)-1: \n", 572 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 573 | " print('Some error3(Bid & Q>0 & timestamp change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 574 | " break\n", 575 | " else: \n", 576 | " pass\n", 577 | " elif position_ == len(order_book_bid)-1: \n", 578 | " if order_book_bid[position_-1:position_-1 + 1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid['Price'][0:1].iloc[0] != data['BestPrice'][i]: \n", 579 | " print('Some error3(Bid & Q>0 & timestamp change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 580 | " break\n", 581 | " else: \n", 582 | " pass\n", 583 | " elif position_ == 0 and len(order_book_bid[0]) == 1:\n", 584 | " pass\n", 585 | " else:\n", 586 | " pass\n", 587 | " elif int(data[['QuantityDifference']][i:i+1].values) < 0: \n", 588 | " if data.TimeStamp[i] == x2[temp_bid]: \n", 589 | " order_number_ = data['OrderNumber'][i : i + 1].iloc[0]\n", 590 | " position_ = order_book_bid[order_book_bid.OrderNumber == order_number_].index[0] \n", 591 | " price_ = data[i:i+1]['Price'].iloc[0]\n", 592 | " if time_second > 32400 and time_second < 57300:\n", 593 | " if position_ == 0 and len(order_book_bid) > 1: \n", 594 | " if order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 595 | " print('Some error4(Bid & Q<0 & timestamp not change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 596 | " break\n", 597 | " else: \n", 598 | " pass \n", 599 | " elif 0 < position_< len(order_book_bid)-1: \n", 600 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 601 | " print('Some error4(Bid & Q<0 & timestamp not change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 602 | " break\n", 603 | " else: \n", 604 | " pass\n", 605 | " elif position_ == len(order_book_bid)-1: \n", 606 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 607 | " print('Some error4(Bid & Q<0 & timestamp not change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0])) \n", 608 | " break\n", 609 | " elif position_ == 0:\n", 610 | " pass\n", 611 | " else: \n", 612 | " pass\n", 613 | " elif position_ == 0 and len(order_book_bid) == 1:\n", 614 | " pass\n", 615 | " else:\n", 616 | " pass\n", 617 | " if order_book_bid[(order_book_bid.OrderNumber == order_number_)&(order_book_bid.Price == price_)]['QuantityDifference'].iloc[0] == abs(data[i:i+1]['QuantityDifference'].iloc[0]): \n", 618 | " order_book_bid = order_book_bid.drop(order_book_bid.index[[position_]]).reset_index(drop = True) \n", 619 | " else:\n", 620 | " order_book_bid['QuantityDifference'][order_book_bid.OrderNumber == order_number_] = order_book_bid['QuantityDifference'][order_book_bid.OrderNumber == order_number_] + data[i:i+1]['QuantityDifference'].iloc[0] \n", 621 | " elif data.TimeStamp[i] != x2[temp_bid]:\n", 622 | " if time_second > 32400 and time_second < 57300:\n", 623 | " if round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) > 0.03 or\\\n", 624 | " round(float(data[i:i+1].TimeStamp.iloc[0][18:29]) - float(data[i-1:i].TimeStamp.iloc[0][18:28]),4) < 0:\n", 625 | " order_book_bid_sum,order_book_ask_sum = order_book_to_csv(order_book_bid,order_book_ask,data,i)\n", 626 | " if data[i-1:i].BidOrAsk.iloc[0] == 'B':\n", 627 | " if order_book_bid_sum[0:1].values.tolist()[0][1] == data[i-1:i].BestQuantity.iloc[0]:\n", 628 | " pass\n", 629 | " else:\n", 630 | " print('Best bid quantity is false')\n", 631 | " #break\n", 632 | " pass\n", 633 | " else:\n", 634 | " j = i - 1\n", 635 | " while j >= 1:\n", 636 | " if data[j-1:j].BidOrAsk.iloc[0] == 'B':\n", 637 | " if order_book_bid_sum[0:1].values.tolist()[0][1] == data[j-1:j].BestQuantity.iloc[0]:\n", 638 | " break\n", 639 | " else:\n", 640 | " print('Best bid quantity is false')\n", 641 | " #break\n", 642 | " pass\n", 643 | " else:\n", 644 | " j = j - 1\n", 645 | " pass\n", 646 | " else:\n", 647 | " pass\n", 648 | " order_number_ = data['OrderNumber'][i:i+1].iloc[0]\n", 649 | " position_ = order_book_bid[order_book_bid.OrderNumber == order_number_].index[0]\n", 650 | " price_ = data[i:i+1]['Price'].iloc[0]\n", 651 | " temp_bid = temp_bid + 1\n", 652 | " if time_second > 32400 and time_second < 57300:\n", 653 | " if position_ == 0 and len(order_book_bid) > 1: \n", 654 | " if order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 655 | " print('Some error5(Bid & Q<0 & timestamp change & 1),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 656 | " break\n", 657 | " else: \n", 658 | " pass \n", 659 | " elif 0 < position_< len(order_book_bid)-1: \n", 660 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0] or order_book_bid[position_+1:position_+1+1][\"Price\"].iloc[0] > data['Price'][i:i+1].iloc[0]: \n", 661 | " print('Some error5(Bid & Q<0 & timestamp change & 2),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 662 | " break\n", 663 | " else: \n", 664 | " pass\n", 665 | " elif position_ == len(order_book_bid)-1: \n", 666 | " if order_book_bid[position_-1:position_-1+1][\"Price\"].iloc[0] < data['Price'][i:i+1].iloc[0]: \n", 667 | " print('Some error5(Bid & Q<0 & timestamp change & 3),position = %d,index = %d,price = %d,OrderNumber = %s'%(position_,i,data['Price'][i:i+1].iloc[0],data['OrderNumber'][i:i+1].iloc[0]))\n", 668 | " break\n", 669 | " elif position_ == 0:\n", 670 | " pass\n", 671 | " else: \n", 672 | " pass \n", 673 | " elif position_ == 0 and len(order_book_bid) == 1:\n", 674 | " pass \n", 675 | " else:\n", 676 | " pass\n", 677 | " if order_book_bid[(order_book_bid.OrderNumber == order_number_)&(order_book_bid.Price == price_)]['QuantityDifference'].iloc[0] == abs(data[i:i+1]['QuantityDifference'].iloc[0]):\n", 678 | " order_book_bid = order_book_bid.drop(order_book_bid.index[[position_]]).reset_index(drop = True) \n", 679 | " else:\n", 680 | " order_book_bid['QuantityDifference'][order_book_bid.OrderNumber == order_number_] = order_book_bid['QuantityDifference'][order_book_bid.OrderNumber == order_number_] + data[i:i+1]['QuantityDifference'].iloc[0]\n", 681 | " return data, order_book_bid, order_book_ask, order_book_bid_sum, order_book_ask_sum" 682 | ] 683 | }, 684 | { 685 | "cell_type": "code", 686 | "execution_count": 8, 687 | "metadata": {}, 688 | "outputs": [ 689 | { 690 | "name": "stdout", 691 | "output_type": "stream", 692 | "text": [ 693 | "2\n", 694 | "timestamp = 2014-01-02D04:19:51.857166800\n", 695 | "-------------------------------------------\n", 696 | "index_find = 16\n", 697 | "---------------------------------\n", 698 | " Price QuantityDifference BidOrAsk TimeStamp\n", 699 | "17 0 210.0 A 2014-01-02D08:45:00.130582600\n", 700 | "17 0 0\n", 701 | "2014-01-02D08:45:00.130582600 2014-01-02D04:19:51.857166800 2014-01-02D04:19:51.857166800\n", 702 | "---------------------------------\n", 703 | " Price QuantityDifference BidOrAsk TimeStamp\n", 704 | "18 700500 -2.0 B 2014-01-02D08:45:00.131582600\n", 705 | "18 0 1\n", 706 | "2014-01-02D08:45:00.131582600 2014-01-02D04:19:51.857166800 2014-01-02D08:45:00.130582600\n", 707 | "---------------------------------\n", 708 | " Price QuantityDifference BidOrAsk TimeStamp\n", 709 | "19 682000 2.0 B 2014-01-02D08:45:00.131582600\n", 710 | "19 1 1\n", 711 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 712 | "---------------------------------\n", 713 | " Price QuantityDifference BidOrAsk TimeStamp\n", 714 | "20 699000 -2.0 B 2014-01-02D08:45:00.131582600\n", 715 | "20 1 1\n", 716 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 717 | "---------------------------------\n", 718 | " Price QuantityDifference BidOrAsk TimeStamp\n", 719 | "21 682000 2.0 B 2014-01-02D08:45:00.131582600\n", 720 | "21 1 1\n", 721 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 722 | "---------------------------------\n", 723 | " Price QuantityDifference BidOrAsk TimeStamp\n", 724 | "22 697500 -1.0 B 2014-01-02D08:45:00.131582600\n", 725 | "22 1 1\n", 726 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 727 | "---------------------------------\n", 728 | " Price QuantityDifference BidOrAsk TimeStamp\n", 729 | "23 682000 1.0 B 2014-01-02D08:45:00.131582600\n", 730 | "23 1 1\n", 731 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 732 | "---------------------------------\n", 733 | " Price QuantityDifference BidOrAsk TimeStamp\n", 734 | "24 690500 -1.0 B 2014-01-02D08:45:00.131582600\n", 735 | "24 1 1\n", 736 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 737 | "---------------------------------\n", 738 | " Price QuantityDifference BidOrAsk TimeStamp\n", 739 | "25 682000 1.0 B 2014-01-02D08:45:00.131582600\n", 740 | "25 1 1\n", 741 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 742 | "---------------------------------\n", 743 | " Price QuantityDifference BidOrAsk TimeStamp\n", 744 | "26 690000 -2.0 B 2014-01-02D08:45:00.131582600\n", 745 | "26 1 1\n", 746 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 747 | "---------------------------------\n", 748 | " Price QuantityDifference BidOrAsk TimeStamp\n", 749 | "27 682000 2.0 B 2014-01-02D08:45:00.131582600\n", 750 | "27 1 1\n", 751 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 752 | "---------------------------------\n", 753 | " Price QuantityDifference BidOrAsk TimeStamp\n", 754 | "28 685000 -2.0 B 2014-01-02D08:45:00.131582600\n", 755 | "28 1 1\n", 756 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 757 | "---------------------------------\n", 758 | " Price QuantityDifference BidOrAsk TimeStamp\n", 759 | "29 682000 2.0 B 2014-01-02D08:45:00.131582600\n", 760 | "29 1 1\n", 761 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 762 | "---------------------------------\n", 763 | " Price QuantityDifference BidOrAsk TimeStamp\n", 764 | "30 683500 -1.0 B 2014-01-02D08:45:00.131582600\n", 765 | "30 1 1\n", 766 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 767 | "---------------------------------\n", 768 | " Price QuantityDifference BidOrAsk TimeStamp\n", 769 | "31 682000 1.0 B 2014-01-02D08:45:00.131582600\n", 770 | "31 1 1\n", 771 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 772 | "---------------------------------\n", 773 | " Price QuantityDifference BidOrAsk TimeStamp\n", 774 | "32 0 -210.0 A 2014-01-02D08:45:00.131582600\n", 775 | "32 1 1\n", 776 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.130582600\n", 777 | "---------------------------------\n", 778 | " Price QuantityDifference BidOrAsk TimeStamp\n", 779 | "33 682000 210.0 A 2014-01-02D08:45:00.131582600\n", 780 | "33 1 2\n", 781 | "2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600\n", 782 | "---------------------------------\n", 783 | " Price QuantityDifference BidOrAsk TimeStamp\n", 784 | "34 682000 210.0 B 2014-01-02D08:45:00.296582600\n", 785 | "34 1 2\n", 786 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600 2014-01-02D08:45:00.131582600\n", 787 | "---------------------------------\n", 788 | " Price QuantityDifference BidOrAsk TimeStamp\n", 789 | "35 682000 -210.0 B 2014-01-02D08:45:00.296582600\n", 790 | "35 2 2\n", 791 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 792 | "---------------------------------\n", 793 | " Price QuantityDifference BidOrAsk TimeStamp\n", 794 | "36 708000 210.0 B 2014-01-02D08:45:00.296582600\n", 795 | "36 2 2\n", 796 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 797 | "---------------------------------\n", 798 | " Price QuantityDifference BidOrAsk TimeStamp\n", 799 | "37 682000 -2.0 B 2014-01-02D08:45:00.296582600\n", 800 | "37 2 2\n", 801 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 802 | "---------------------------------\n", 803 | " Price QuantityDifference BidOrAsk TimeStamp\n", 804 | "38 700500 2.0 B 2014-01-02D08:45:00.296582600\n", 805 | "38 2 2\n", 806 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 807 | "---------------------------------\n", 808 | " Price QuantityDifference BidOrAsk TimeStamp\n", 809 | "39 682000 -2.0 B 2014-01-02D08:45:00.296582600\n", 810 | "39 2 2\n", 811 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 812 | "---------------------------------\n", 813 | " Price QuantityDifference BidOrAsk TimeStamp\n", 814 | "40 699000 2.0 B 2014-01-02D08:45:00.296582600\n", 815 | "40 2 2\n", 816 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 817 | "---------------------------------\n", 818 | " Price QuantityDifference BidOrAsk TimeStamp\n", 819 | "41 682000 -1.0 B 2014-01-02D08:45:00.296582600\n", 820 | "41 2 2\n", 821 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 822 | "---------------------------------\n", 823 | " Price QuantityDifference BidOrAsk TimeStamp\n", 824 | "42 697500 1.0 B 2014-01-02D08:45:00.296582600\n", 825 | "42 2 2\n", 826 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 827 | "---------------------------------\n", 828 | " Price QuantityDifference BidOrAsk TimeStamp\n", 829 | "43 682000 -1.0 B 2014-01-02D08:45:00.296582600\n", 830 | "43 2 2\n", 831 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 832 | "---------------------------------\n", 833 | " Price QuantityDifference BidOrAsk TimeStamp\n", 834 | "44 690500 1.0 B 2014-01-02D08:45:00.296582600\n", 835 | "44 2 2\n", 836 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 837 | "---------------------------------\n", 838 | " Price QuantityDifference BidOrAsk TimeStamp\n", 839 | "45 682000 -2.0 B 2014-01-02D08:45:00.296582600\n", 840 | "45 2 2\n", 841 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 842 | "---------------------------------\n", 843 | " Price QuantityDifference BidOrAsk TimeStamp\n", 844 | "46 690000 2.0 B 2014-01-02D08:45:00.296582600\n", 845 | "46 2 2\n", 846 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 847 | "---------------------------------\n", 848 | " Price QuantityDifference BidOrAsk TimeStamp\n", 849 | "47 682000 -2.0 B 2014-01-02D08:45:00.296582600\n", 850 | "47 2 2\n", 851 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 852 | "---------------------------------\n", 853 | " Price QuantityDifference BidOrAsk TimeStamp\n", 854 | "48 685000 2.0 B 2014-01-02D08:45:00.296582600\n", 855 | "48 2 2\n", 856 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 857 | "---------------------------------\n", 858 | " Price QuantityDifference BidOrAsk TimeStamp\n", 859 | "49 682000 -1.0 B 2014-01-02D08:45:00.296582600\n", 860 | "49 2 2\n", 861 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 862 | "---------------------------------\n", 863 | " Price QuantityDifference BidOrAsk TimeStamp\n", 864 | "50 683500 1.0 B 2014-01-02D08:45:00.296582600\n", 865 | "50 2 2\n", 866 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 867 | "---------------------------------\n", 868 | " Price QuantityDifference BidOrAsk TimeStamp\n", 869 | "51 682000 -210.0 A 2014-01-02D08:45:00.296582600\n", 870 | "51 2 2\n", 871 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.131582600\n", 872 | "---------------------------------\n", 873 | " Price QuantityDifference BidOrAsk TimeStamp\n", 874 | "52 708000 210.0 A 2014-01-02D08:45:00.296582600\n", 875 | "52 2 3\n", 876 | "2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600\n", 877 | "---------------------------------\n", 878 | " Price QuantityDifference BidOrAsk TimeStamp\n", 879 | "53 695500 1.0 B 2014-01-02D08:45:00.398582300\n", 880 | "53 2 3\n", 881 | "2014-01-02D08:45:00.398582300 2014-01-02D08:45:00.296582600 2014-01-02D08:45:00.296582600\n", 882 | "---------------------------------\n", 883 | " Price QuantityDifference BidOrAsk TimeStamp\n", 884 | "54 748000 5.0 A 2014-01-02D08:45:13.857414600\n", 885 | "54 3 3\n", 886 | "2014-01-02D08:45:13.857414600 2014-01-02D08:45:00.398582300 2014-01-02D08:45:00.296582600\n", 887 | "---------------------------------\n", 888 | " Price QuantityDifference BidOrAsk TimeStamp\n", 889 | "55 738000 4.0 A 2014-01-02D08:45:14.337414600\n", 890 | "55 3 4\n", 891 | "2014-01-02D08:45:14.337414600 2014-01-02D08:45:00.398582300 2014-01-02D08:45:13.857414600\n", 892 | "---------------------------------\n", 893 | " Price QuantityDifference BidOrAsk TimeStamp\n", 894 | "56 683500 3.0 B 2014-01-02D08:45:14.977402600\n", 895 | "56 3 5\n", 896 | "2014-01-02D08:45:14.977402600 2014-01-02D08:45:00.398582300 2014-01-02D08:45:14.337414600\n", 897 | "---------------------------------\n", 898 | " Price QuantityDifference BidOrAsk TimeStamp\n", 899 | "57 759000 5.0 A 2014-01-02D08:45:15.557390600\n", 900 | "57 4 5\n", 901 | "2014-01-02D08:45:15.557390600 2014-01-02D08:45:14.977402600 2014-01-02D08:45:14.337414600\n", 902 | "---------------------------------\n", 903 | " Price QuantityDifference BidOrAsk TimeStamp\n", 904 | "58 653500 3.0 B 2014-01-02D08:45:17.727366600\n", 905 | "58 4 6\n", 906 | "2014-01-02D08:45:17.727366600 2014-01-02D08:45:14.977402600 2014-01-02D08:45:15.557390600\n", 907 | "---------------------------------\n", 908 | " Price QuantityDifference BidOrAsk TimeStamp\n", 909 | "59 663500 3.0 B 2014-01-02D08:45:18.317366600\n", 910 | "59 5 6\n", 911 | "2014-01-02D08:45:18.317366600 2014-01-02D08:45:17.727366600 2014-01-02D08:45:15.557390600\n", 912 | "---------------------------------\n", 913 | " Price QuantityDifference BidOrAsk TimeStamp\n", 914 | "60 725000 4.0 A 2014-01-02D08:45:30.059222600\n", 915 | "60 6 6\n", 916 | "2014-01-02D08:45:30.059222600 2014-01-02D08:45:18.317366600 2014-01-02D08:45:15.557390600\n", 917 | "---------------------------------\n", 918 | " Price QuantityDifference BidOrAsk TimeStamp\n", 919 | "61 731500 1.0 A 2014-01-02D08:45:36.809138600\n", 920 | "61 6 7\n", 921 | "2014-01-02D08:45:36.809138600 2014-01-02D08:45:18.317366600 2014-01-02D08:45:30.059222600\n", 922 | "---------------------------------\n", 923 | " Price QuantityDifference BidOrAsk TimeStamp\n", 924 | "62 736000 1.0 A 2014-01-02D08:45:52.185958600\n", 925 | "62 6 8\n", 926 | "2014-01-02D08:45:52.185958600 2014-01-02D08:45:18.317366600 2014-01-02D08:45:36.809138600\n", 927 | "---------------------------------\n", 928 | " Price QuantityDifference BidOrAsk TimeStamp\n", 929 | "63 690000 75.0 B 2014-01-02D08:46:00.014862600\n", 930 | "63 6 9\n", 931 | "2014-01-02D08:46:00.014862600 2014-01-02D08:45:18.317366600 2014-01-02D08:45:52.185958600\n", 932 | "---------------------------------\n", 933 | " Price QuantityDifference BidOrAsk TimeStamp\n", 934 | "64 712000 2.0 A 2014-01-02D08:46:00.018862600\n", 935 | "64 7 9\n", 936 | "2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.014862600 2014-01-02D08:45:52.185958600\n", 937 | "---------------------------------\n", 938 | " Price QuantityDifference BidOrAsk TimeStamp\n", 939 | "65 708000 -210.0 B 2014-01-02D08:46:00.018862600\n", 940 | "65 7 10\n", 941 | "2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.014862600 2014-01-02D08:46:00.018862600\n", 942 | "---------------------------------\n", 943 | " Price QuantityDifference BidOrAsk TimeStamp\n", 944 | "66 706500 210.0 B 2014-01-02D08:46:00.018862600\n", 945 | "66 8 10\n", 946 | "2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600\n", 947 | "---------------------------------\n", 948 | " Price QuantityDifference BidOrAsk TimeStamp\n", 949 | "67 708000 -210.0 A 2014-01-02D08:46:00.018862600\n", 950 | "67 8 10\n", 951 | "2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600\n", 952 | "---------------------------------\n", 953 | " Price QuantityDifference BidOrAsk TimeStamp\n", 954 | "68 706500 210.0 A 2014-01-02D08:46:00.018862600\n", 955 | "68 8 10\n", 956 | "2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600\n", 957 | "---------------------------------\n", 958 | " Price QuantityDifference BidOrAsk TimeStamp\n", 959 | "69 770000 6.0 A 2014-01-02D08:46:00.018862600\n", 960 | "69 8 10\n", 961 | "2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600\n", 962 | "---------------------------------\n", 963 | " Price QuantityDifference BidOrAsk TimeStamp\n", 964 | "70 754000 3.0 A 2014-01-02D08:46:00.021862600\n", 965 | "70 8 10\n", 966 | "2014-01-02D08:46:00.021862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.018862600\n", 967 | "---------------------------------\n", 968 | " Price QuantityDifference BidOrAsk TimeStamp\n", 969 | "71 760000 15.0 A 2014-01-02D08:46:00.022862600\n", 970 | "71 8 11\n", 971 | "2014-01-02D08:46:00.022862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.021862600\n", 972 | "---------------------------------\n", 973 | " Price QuantityDifference BidOrAsk TimeStamp\n", 974 | "72 770000 6.0 A 2014-01-02D08:46:00.024862600\n", 975 | "72 8 12\n", 976 | "2014-01-02D08:46:00.024862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.022862600\n", 977 | "---------------------------------\n", 978 | " Price QuantityDifference BidOrAsk TimeStamp\n", 979 | "73 770000 2.0 A 2014-01-02D08:46:00.025862600\n", 980 | "73 8 13\n", 981 | "2014-01-02D08:46:00.025862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.024862600\n", 982 | "---------------------------------\n", 983 | " Price QuantityDifference BidOrAsk TimeStamp\n", 984 | "74 770000 6.0 A 2014-01-02D08:46:00.029862600\n", 985 | "74 8 14\n", 986 | "2014-01-02D08:46:00.029862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.025862600\n", 987 | "---------------------------------\n", 988 | " Price QuantityDifference BidOrAsk TimeStamp\n", 989 | "75 770000 6.0 A 2014-01-02D08:46:00.029862600\n", 990 | "75 8 15\n", 991 | "2014-01-02D08:46:00.029862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.029862600\n", 992 | "---------------------------------\n", 993 | " Price QuantityDifference BidOrAsk TimeStamp\n", 994 | "76 702000 1.0 B 2014-01-02D08:46:00.033862600\n", 995 | "76 8 15\n", 996 | "2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.018862600 2014-01-02D08:46:00.029862600\n", 997 | "---------------------------------\n", 998 | " Price QuantityDifference BidOrAsk TimeStamp\n", 999 | "77 706500 -210.0 B 2014-01-02D08:46:00.033862600\n", 1000 | "77 9 15\n", 1001 | "2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.029862600\n", 1002 | "---------------------------------\n", 1003 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1004 | "78 707000 210.0 B 2014-01-02D08:46:00.033862600\n", 1005 | "78 9 15\n", 1006 | "2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.029862600\n", 1007 | "---------------------------------\n", 1008 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1009 | "79 706500 -210.0 A 2014-01-02D08:46:00.033862600\n", 1010 | "79 9 15\n", 1011 | "2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.029862600\n", 1012 | "---------------------------------\n", 1013 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1014 | "80 707000 210.0 A 2014-01-02D08:46:00.033862600\n", 1015 | "80 9 16\n", 1016 | "2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600\n", 1017 | "---------------------------------\n", 1018 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1019 | "81 673000 8.0 B 2014-01-02D08:46:00.033862600\n", 1020 | "81 9 16\n", 1021 | "2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600\n", 1022 | "---------------------------------\n", 1023 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1024 | "82 748000 5.0 A 2014-01-02D08:46:00.036862600\n", 1025 | "82 9 16\n", 1026 | "2014-01-02D08:46:00.036862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.033862600\n", 1027 | "---------------------------------\n", 1028 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1029 | "83 725000 1.0 A 2014-01-02D08:46:00.037862600\n", 1030 | "83 9 17\n", 1031 | "2014-01-02D08:46:00.037862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.036862600\n", 1032 | "---------------------------------\n", 1033 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1034 | "84 744000 1.0 A 2014-01-02D08:46:00.039862600\n", 1035 | "84 9 18\n", 1036 | "2014-01-02D08:46:00.039862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.037862600\n", 1037 | "---------------------------------\n", 1038 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1039 | "85 701000 1.0 B 2014-01-02D08:46:00.040862600\n", 1040 | "85 9 19\n", 1041 | "2014-01-02D08:46:00.040862600 2014-01-02D08:46:00.033862600 2014-01-02D08:46:00.039862600\n", 1042 | "---------------------------------\n", 1043 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1044 | "86 707000 1.0 B 2014-01-02D08:46:00.043862600\n", 1045 | "86 10 19\n", 1046 | "2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.040862600 2014-01-02D08:46:00.039862600\n", 1047 | "---------------------------------\n", 1048 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1049 | "87 707000 -210.0 B 2014-01-02D08:46:00.043862600\n", 1050 | "87 11 19\n", 1051 | "2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.039862600\n", 1052 | "---------------------------------\n", 1053 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1054 | "88 709500 210.0 B 2014-01-02D08:46:00.043862600\n", 1055 | "88 11 19\n", 1056 | "2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.039862600\n", 1057 | "---------------------------------\n", 1058 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1059 | "89 707000 -210.0 A 2014-01-02D08:46:00.043862600\n", 1060 | "89 11 19\n", 1061 | "2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.039862600\n", 1062 | "---------------------------------\n", 1063 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1064 | "90 709500 210.0 A 2014-01-02D08:46:00.043862600\n", 1065 | "90 11 20\n", 1066 | "2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.043862600\n", 1067 | "---------------------------------\n", 1068 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1069 | "91 747000 2.0 A 2014-01-02D08:46:00.045862600\n", 1070 | "91 11 20\n", 1071 | "2014-01-02D08:46:00.045862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.043862600\n", 1072 | "---------------------------------\n", 1073 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1074 | "92 712000 2.0 A 2014-01-02D08:46:00.048862600\n", 1075 | "92 11 21\n", 1076 | "2014-01-02D08:46:00.048862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.045862600\n", 1077 | "---------------------------------\n", 1078 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1079 | "93 736000 1.0 A 2014-01-02D08:46:00.048862600\n", 1080 | "93 11 22\n", 1081 | "2014-01-02D08:46:00.048862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.048862600\n", 1082 | "---------------------------------\n", 1083 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1084 | "94 756000 1.0 A 2014-01-02D08:46:00.052862600\n", 1085 | "94 11 22\n", 1086 | "2014-01-02D08:46:00.052862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.048862600\n", 1087 | "---------------------------------\n", 1088 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1089 | "95 746000 1.0 A 2014-01-02D08:46:00.056862600\n", 1090 | "95 11 23\n", 1091 | "2014-01-02D08:46:00.056862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.052862600\n", 1092 | "---------------------------------\n", 1093 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1094 | "96 720000 2.0 A 2014-01-02D08:46:00.061862600\n", 1095 | "96 11 24\n", 1096 | "2014-01-02D08:46:00.061862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.056862600\n", 1097 | "---------------------------------\n", 1098 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1099 | "97 760000 4.0 A 2014-01-02D08:46:00.065862600\n", 1100 | "97 11 25\n", 1101 | "2014-01-02D08:46:00.065862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.061862600\n", 1102 | "---------------------------------\n", 1103 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1104 | "98 680000 2.0 B 2014-01-02D08:46:00.069862600\n", 1105 | "98 11 26\n", 1106 | "2014-01-02D08:46:00.069862600 2014-01-02D08:46:00.043862600 2014-01-02D08:46:00.065862600\n", 1107 | "---------------------------------\n", 1108 | " Price QuantityDifference BidOrAsk TimeStamp\n", 1109 | "99 675000 2.0 B 2014-01-02D08:46:00.069862600\n", 1110 | "99 12 26\n", 1111 | "2014-01-02D08:46:00.069862600 2014-01-02D08:46:00.069862600 2014-01-02D08:46:00.065862600\n", 1112 | "Total time = 1.131267\n" 1113 | ] 1114 | } 1115 | ], 1116 | "source": [ 1117 | "import time\n", 1118 | "\n", 1119 | "if __name__ == '__main__':\n", 1120 | " path = './data/CN_Futures_'\n", 1121 | " year = 2014\n", 1122 | " # 决定生成的每一个tick中有多少组bid和ask\n", 1123 | " best_price_number = 3\n", 1124 | " \n", 1125 | " # Future Delivery Months 近月合約代碼 : series\n", 1126 | " # 2014 :CNF14:1月, CNG14:2月, CNH14:3月, CNJ14:4月, CNK14:5月, CNM14:6月,\n", 1127 | " # CNN14:7月, CNQ14:8月, CNU14:9月, CNV14:10月, CNX14:11月, CNZ14:12月\n", 1128 | " # 2015 :CNF15:1月, CNG15:2月, CNH15:3月, CNJ15:4月, CNK15:5月, CNM15:6月,\n", 1129 | " # CNN15:7月, CNQ15:8月, CNU15:9月, CNV15:10月, CNX15:11月, CNZ15:12月\n", 1130 | " \n", 1131 | " series = 'CNF14' \n", 1132 | " month = 1\n", 1133 | " day_ = [2]\n", 1134 | " for i in day_:\n", 1135 | " print(i)\n", 1136 | " start = time.time()\n", 1137 | " data, order_book_bid, order_book_ask, order_book_bid_sum, order_book_ask_sum = order_book_tranform(year,month,i,path,best_price_number,series)\n", 1138 | " end = time.time()\n", 1139 | " print(\"Total time = %f\"%(end - start)) " 1140 | ] 1141 | }, 1142 | { 1143 | "cell_type": "code", 1144 | "execution_count": 9, 1145 | "metadata": {}, 1146 | "outputs": [ 1147 | { 1148 | "data": { 1149 | "text/html": [ 1150 | "
\n", 1151 | "\n", 1164 | "\n", 1165 | " \n", 1166 | " \n", 1167 | " \n", 1168 | " \n", 1169 | " \n", 1170 | " \n", 1171 | " \n", 1172 | " \n", 1173 | " \n", 1174 | " \n", 1175 | " \n", 1176 | " \n", 1177 | " \n", 1178 | " \n", 1179 | " \n", 1180 | " \n", 1181 | " \n", 1182 | " \n", 1183 | " \n", 1184 | " \n", 1185 | " \n", 1186 | " \n", 1187 | " \n", 1188 | " \n", 1189 | " \n", 1190 | " \n", 1191 | " \n", 1192 | " \n", 1193 | " \n", 1194 | " \n", 1195 | " \n", 1196 | " \n", 1197 | " \n", 1198 | " \n", 1199 | " \n", 1200 | " \n", 1201 | " \n", 1202 | " \n", 1203 | " \n", 1204 | " \n", 1205 | " \n", 1206 | " \n", 1207 | " \n", 1208 | " \n", 1209 | " \n", 1210 | " \n", 1211 | " \n", 1212 | " \n", 1213 | " \n", 1214 | " \n", 1215 | " \n", 1216 | " \n", 1217 | " \n", 1218 | " \n", 1219 | " \n", 1220 | " \n", 1221 | " \n", 1222 | " \n", 1223 | " \n", 1224 | " \n", 1225 | " \n", 1226 | " \n", 1227 | " \n", 1228 | " \n", 1229 | " \n", 1230 | " \n", 1231 | " \n", 1232 | " \n", 1233 | " \n", 1234 | " \n", 1235 | " \n", 1236 | " \n", 1237 | " \n", 1238 | " \n", 1239 | " \n", 1240 | " \n", 1241 | " \n", 1242 | " \n", 1243 | " \n", 1244 | " \n", 1245 | " \n", 1246 | " \n", 1247 | " \n", 1248 | " \n", 1249 | " \n", 1250 | " \n", 1251 | " \n", 1252 | " \n", 1253 | " \n", 1254 | " \n", 1255 | " \n", 1256 | " \n", 1257 | " \n", 1258 | " \n", 1259 | " \n", 1260 | " \n", 1261 | " \n", 1262 | " \n", 1263 | " \n", 1264 | " \n", 1265 | " \n", 1266 | " \n", 1267 | " \n", 1268 | " \n", 1269 | " \n", 1270 | " \n", 1271 | " \n", 1272 | " \n", 1273 | " \n", 1274 | " \n", 1275 | " \n", 1276 | " \n", 1277 | " \n", 1278 | " \n", 1279 | " \n", 1280 | " \n", 1281 | " \n", 1282 | " \n", 1283 | " \n", 1284 | " \n", 1285 | " \n", 1286 | " \n", 1287 | " \n", 1288 | " \n", 1289 | " \n", 1290 | " \n", 1291 | " \n", 1292 | " \n", 1293 | " \n", 1294 | " \n", 1295 | " \n", 1296 | " \n", 1297 | " \n", 1298 | " \n", 1299 | " \n", 1300 | " \n", 1301 | " \n", 1302 | " \n", 1303 | " \n", 1304 | " \n", 1305 | " \n", 1306 | " \n", 1307 | " \n", 1308 | " \n", 1309 | " \n", 1310 | " \n", 1311 | " \n", 1312 | " \n", 1313 | " \n", 1314 | " \n", 1315 | " \n", 1316 | " \n", 1317 | " \n", 1318 | " \n", 1319 | " \n", 1320 | " \n", 1321 | " \n", 1322 | " \n", 1323 | " \n", 1324 | " \n", 1325 | " \n", 1326 | " \n", 1327 | " \n", 1328 | " \n", 1329 | " \n", 1330 | " \n", 1331 | " \n", 1332 | " \n", 1333 | " \n", 1334 | " \n", 1335 | " \n", 1336 | " \n", 1337 | " \n", 1338 | " \n", 1339 | " \n", 1340 | " \n", 1341 | " \n", 1342 | " \n", 1343 | " \n", 1344 | " \n", 1345 | " \n", 1346 | " \n", 1347 | " \n", 1348 | " \n", 1349 | "
SeriesSequenceNumberTimeStampOrderNumberOrderBookPositionPriceQuantityDifferenceTradeBidOrAskBestPriceBestQuantityQuantityDifference_
0CNF1412014-01-02D04:19:51.857166800642F9A8039E51EE517570003.0A71550023.0
1CNF1422014-01-02D04:19:51.857166800490D725B88E56DE5175500010.0A715500210.0
2CNF1432014-01-02D04:19:51.857166800E1C9F25394A679A617195001.0A71550021.0
3CNF1442014-01-02D04:19:51.857166800ECA71A88142AF92B47685001.0A71550021.0
4CNF1452014-01-02D04:19:51.857166800E325782FEB07CE6717005002.0B70050022.0
.......................................
97475CNF14974762014-01-03D02:20:00.060617300E3B755234111255113770000-6.0A7085002-6.0
97476CNF14974772014-01-03D02:20:00.060617300473C5923A5D088F013770000-2.0A7085002-2.0
97477CNF14974782014-01-03D02:20:00.060617300633F552361CA460A13770000-6.0A7085002-6.0
97478CNF14974792014-01-03D02:20:00.06061730047BC592368224B4213770000-6.0A7085002-6.0
97479CNF14974802014-01-03D02:20:00.060617300E3BFD52B096BEDAC13770000-6.0A7085002-6.0
\n", 1350 | "

97480 rows × 12 columns

\n", 1351 | "
" 1352 | ], 1353 | "text/plain": [ 1354 | " Series SequenceNumber TimeStamp OrderNumber OrderBookPosition Price QuantityDifference Trade BidOrAsk BestPrice BestQuantity QuantityDifference_\n", 1355 | "0 CNF14 1 2014-01-02D04:19:51.857166800 642F9A8039E51EE5 1 757000 3.0 A 715500 2 3.0\n", 1356 | "1 CNF14 2 2014-01-02D04:19:51.857166800 490D725B88E56DE5 1 755000 10.0 A 715500 2 10.0\n", 1357 | "2 CNF14 3 2014-01-02D04:19:51.857166800 E1C9F25394A679A6 1 719500 1.0 A 715500 2 1.0\n", 1358 | "3 CNF14 4 2014-01-02D04:19:51.857166800 ECA71A88142AF92B 4 768500 1.0 A 715500 2 1.0\n", 1359 | "4 CNF14 5 2014-01-02D04:19:51.857166800 E325782FEB07CE67 1 700500 2.0 B 700500 2 2.0\n", 1360 | "... ... ... ... ... ... ... ... ... ... ... ... ...\n", 1361 | "97475 CNF14 97476 2014-01-03D02:20:00.060617300 E3B7552341112551 13 770000 -6.0 A 708500 2 -6.0\n", 1362 | "97476 CNF14 97477 2014-01-03D02:20:00.060617300 473C5923A5D088F0 13 770000 -2.0 A 708500 2 -2.0\n", 1363 | "97477 CNF14 97478 2014-01-03D02:20:00.060617300 633F552361CA460A 13 770000 -6.0 A 708500 2 -6.0\n", 1364 | "97478 CNF14 97479 2014-01-03D02:20:00.060617300 47BC592368224B42 13 770000 -6.0 A 708500 2 -6.0\n", 1365 | "97479 CNF14 97480 2014-01-03D02:20:00.060617300 E3BFD52B096BEDAC 13 770000 -6.0 A 708500 2 -6.0\n", 1366 | "\n", 1367 | "[97480 rows x 12 columns]" 1368 | ] 1369 | }, 1370 | "execution_count": 9, 1371 | "metadata": {}, 1372 | "output_type": "execute_result" 1373 | } 1374 | ], 1375 | "source": [ 1376 | "data" 1377 | ] 1378 | }, 1379 | { 1380 | "cell_type": "code", 1381 | "execution_count": 10, 1382 | "metadata": {}, 1383 | "outputs": [ 1384 | { 1385 | "data": { 1386 | "text/html": [ 1387 | "
\n", 1388 | "\n", 1401 | "\n", 1402 | " \n", 1403 | " \n", 1404 | " \n", 1405 | " \n", 1406 | " \n", 1407 | " \n", 1408 | " \n", 1409 | " \n", 1410 | " \n", 1411 | " \n", 1412 | " \n", 1413 | " \n", 1414 | " \n", 1415 | " \n", 1416 | " \n", 1417 | " \n", 1418 | " \n", 1419 | " \n", 1420 | " \n", 1421 | " \n", 1422 | " \n", 1423 | " \n", 1424 | " \n", 1425 | " \n", 1426 | " \n", 1427 | " \n", 1428 | " \n", 1429 | " \n", 1430 | " \n", 1431 | " \n", 1432 | " \n", 1433 | " \n", 1434 | " \n", 1435 | " \n", 1436 | " \n", 1437 | " \n", 1438 | " \n", 1439 | " \n", 1440 | " \n", 1441 | " \n", 1442 | " \n", 1443 | " \n", 1444 | " \n", 1445 | " \n", 1446 | " \n", 1447 | " \n", 1448 | " \n", 1449 | " \n", 1450 | " \n", 1451 | " \n", 1452 | " \n", 1453 | " \n", 1454 | " \n", 1455 | " \n", 1456 | " \n", 1457 | " \n", 1458 | " \n", 1459 | " \n", 1460 | " \n", 1461 | " \n", 1462 | " \n", 1463 | " \n", 1464 | " \n", 1465 | " \n", 1466 | " \n", 1467 | " \n", 1468 | " \n", 1469 | " \n", 1470 | " \n", 1471 | " \n", 1472 | " \n", 1473 | " \n", 1474 | " \n", 1475 | " \n", 1476 | " \n", 1477 | " \n", 1478 | " \n", 1479 | " \n", 1480 | " \n", 1481 | " \n", 1482 | " \n", 1483 | " \n", 1484 | " \n", 1485 | " \n", 1486 | " \n", 1487 | " \n", 1488 | " \n", 1489 | " \n", 1490 | " \n", 1491 | " \n", 1492 | " \n", 1493 | " \n", 1494 | " \n", 1495 | " \n", 1496 | " \n", 1497 | " \n", 1498 | " \n", 1499 | " \n", 1500 | " \n", 1501 | " \n", 1502 | " \n", 1503 | " \n", 1504 | " \n", 1505 | " \n", 1506 | " \n", 1507 | " \n", 1508 | " \n", 1509 | " \n", 1510 | " \n", 1511 | " \n", 1512 | " \n", 1513 | " \n", 1514 | " \n", 1515 | " \n", 1516 | " \n", 1517 | " \n", 1518 | " \n", 1519 | " \n", 1520 | " \n", 1521 | " \n", 1522 | " \n", 1523 | " \n", 1524 | " \n", 1525 | " \n", 1526 | " \n", 1527 | " \n", 1528 | " \n", 1529 | " \n", 1530 | " \n", 1531 | " \n", 1532 | " \n", 1533 | " \n", 1534 | " \n", 1535 | " \n", 1536 | " \n", 1537 | " \n", 1538 | " \n", 1539 | " \n", 1540 | " \n", 1541 | " \n", 1542 | " \n", 1543 | " \n", 1544 | " \n", 1545 | " \n", 1546 | " \n", 1547 | " \n", 1548 | " \n", 1549 | " \n", 1550 | " \n", 1551 | " \n", 1552 | " \n", 1553 | "
PriceOrderNumberQuantityDifferenceQuantityDifference_
0709500C7C3FA1363164816210.00.0
170700043E3747FD245B5651.01.0
2702000C36B74779D8980A91.01.0
3701000EF3D516B3FA423E41.01.0
4700500E325782FEB07CE672.02.0
56990004959765357F03CF02.02.0
669750043A57467BFB59F551.01.0
7695500E70A765B175EFC1F1.01.0
8690500B3F5144E612645E61.01.0
9690000E27C9AD83BDC20DC2.02.0
1069000043E374773034135475.075.0
11685000CACB3B94B2C197812.02.0
12683500336D9C4E88C46D841.01.0
13683500E14CDA075D093F493.03.0
146820001335141E47F92CF9100.00.0
156800006FBDD1633B411F812.02.0
16675000C3EB747F49582C782.02.0
176730006F3D516B2DF712378.08.0
1866350069C45A07D1B9B3F93.03.0
19653500E1CCDA0FA8338A733.03.0
\n", 1554 | "
" 1555 | ], 1556 | "text/plain": [ 1557 | " Price OrderNumber QuantityDifference QuantityDifference_\n", 1558 | "0 709500 C7C3FA1363164816 210.0 0.0\n", 1559 | "1 707000 43E3747FD245B565 1.0 1.0\n", 1560 | "2 702000 C36B74779D8980A9 1.0 1.0\n", 1561 | "3 701000 EF3D516B3FA423E4 1.0 1.0\n", 1562 | "4 700500 E325782FEB07CE67 2.0 2.0\n", 1563 | "5 699000 4959765357F03CF0 2.0 2.0\n", 1564 | "6 697500 43A57467BFB59F55 1.0 1.0\n", 1565 | "7 695500 E70A765B175EFC1F 1.0 1.0\n", 1566 | "8 690500 B3F5144E612645E6 1.0 1.0\n", 1567 | "9 690000 E27C9AD83BDC20DC 2.0 2.0\n", 1568 | "10 690000 43E3747730341354 75.0 75.0\n", 1569 | "11 685000 CACB3B94B2C19781 2.0 2.0\n", 1570 | "12 683500 336D9C4E88C46D84 1.0 1.0\n", 1571 | "13 683500 E14CDA075D093F49 3.0 3.0\n", 1572 | "14 682000 1335141E47F92CF9 100.0 0.0\n", 1573 | "15 680000 6FBDD1633B411F81 2.0 2.0\n", 1574 | "16 675000 C3EB747F49582C78 2.0 2.0\n", 1575 | "17 673000 6F3D516B2DF71237 8.0 8.0\n", 1576 | "18 663500 69C45A07D1B9B3F9 3.0 3.0\n", 1577 | "19 653500 E1CCDA0FA8338A73 3.0 3.0" 1578 | ] 1579 | }, 1580 | "execution_count": 10, 1581 | "metadata": {}, 1582 | "output_type": "execute_result" 1583 | } 1584 | ], 1585 | "source": [ 1586 | "order_book_bid" 1587 | ] 1588 | }, 1589 | { 1590 | "cell_type": "code", 1591 | "execution_count": 11, 1592 | "metadata": {}, 1593 | "outputs": [ 1594 | { 1595 | "data": { 1596 | "text/html": [ 1597 | "
\n", 1598 | "\n", 1611 | "\n", 1612 | " \n", 1613 | " \n", 1614 | " \n", 1615 | " \n", 1616 | " \n", 1617 | " \n", 1618 | " \n", 1619 | " \n", 1620 | " \n", 1621 | " \n", 1622 | " \n", 1623 | " \n", 1624 | " \n", 1625 | " \n", 1626 | " \n", 1627 | " \n", 1628 | " \n", 1629 | " \n", 1630 | " \n", 1631 | " \n", 1632 | " \n", 1633 | " \n", 1634 | " \n", 1635 | " \n", 1636 | " \n", 1637 | " \n", 1638 | " \n", 1639 | " \n", 1640 | " \n", 1641 | " \n", 1642 | " \n", 1643 | " \n", 1644 | " \n", 1645 | " \n", 1646 | " \n", 1647 | " \n", 1648 | " \n", 1649 | " \n", 1650 | " \n", 1651 | " \n", 1652 | " \n", 1653 | " \n", 1654 | " \n", 1655 | " \n", 1656 | " \n", 1657 | " \n", 1658 | " \n", 1659 | " \n", 1660 | " \n", 1661 | " \n", 1662 | " \n", 1663 | " \n", 1664 | " \n", 1665 | " \n", 1666 | " \n", 1667 | " \n", 1668 | " \n", 1669 | " \n", 1670 | " \n", 1671 | " \n", 1672 | " \n", 1673 | " \n", 1674 | " \n", 1675 | " \n", 1676 | " \n", 1677 | " \n", 1678 | " \n", 1679 | " \n", 1680 | " \n", 1681 | " \n", 1682 | " \n", 1683 | " \n", 1684 | " \n", 1685 | " \n", 1686 | " \n", 1687 | " \n", 1688 | " \n", 1689 | " \n", 1690 | " \n", 1691 | " \n", 1692 | " \n", 1693 | " \n", 1694 | " \n", 1695 | " \n", 1696 | " \n", 1697 | " \n", 1698 | " \n", 1699 | " \n", 1700 | " \n", 1701 | " \n", 1702 | " \n", 1703 | " \n", 1704 | " \n", 1705 | " \n", 1706 | " \n", 1707 | " \n", 1708 | " \n", 1709 | " \n", 1710 | " \n", 1711 | " \n", 1712 | " \n", 1713 | " \n", 1714 | " \n", 1715 | " \n", 1716 | " \n", 1717 | " \n", 1718 | " \n", 1719 | " \n", 1720 | " \n", 1721 | " \n", 1722 | " \n", 1723 | " \n", 1724 | " \n", 1725 | " \n", 1726 | " \n", 1727 | " \n", 1728 | " \n", 1729 | " \n", 1730 | " \n", 1731 | " \n", 1732 | " \n", 1733 | " \n", 1734 | " \n", 1735 | " \n", 1736 | " \n", 1737 | " \n", 1738 | " \n", 1739 | " \n", 1740 | " \n", 1741 | " \n", 1742 | " \n", 1743 | " \n", 1744 | " \n", 1745 | " \n", 1746 | " \n", 1747 | " \n", 1748 | " \n", 1749 | " \n", 1750 | " \n", 1751 | " \n", 1752 | " \n", 1753 | " \n", 1754 | " \n", 1755 | " \n", 1756 | " \n", 1757 | " \n", 1758 | " \n", 1759 | " \n", 1760 | " \n", 1761 | " \n", 1762 | " \n", 1763 | " \n", 1764 | " \n", 1765 | " \n", 1766 | " \n", 1767 | " \n", 1768 | " \n", 1769 | " \n", 1770 | " \n", 1771 | " \n", 1772 | " \n", 1773 | " \n", 1774 | " \n", 1775 | " \n", 1776 | " \n", 1777 | " \n", 1778 | " \n", 1779 | " \n", 1780 | " \n", 1781 | " \n", 1782 | " \n", 1783 | " \n", 1784 | " \n", 1785 | " \n", 1786 | " \n", 1787 | " \n", 1788 | " \n", 1789 | " \n", 1790 | " \n", 1791 | " \n", 1792 | " \n", 1793 | " \n", 1794 | " \n", 1795 | " \n", 1796 | " \n", 1797 | " \n", 1798 | " \n", 1799 | " \n", 1800 | " \n", 1801 | " \n", 1802 | " \n", 1803 | " \n", 1804 | " \n", 1805 | " \n", 1806 | " \n", 1807 | " \n", 1808 | " \n", 1809 | " \n", 1810 | " \n", 1811 | " \n", 1812 | " \n", 1813 | " \n", 1814 | " \n", 1815 | " \n", 1816 | " \n", 1817 | " \n", 1818 | " \n", 1819 | " \n", 1820 | " \n", 1821 | " \n", 1822 | " \n", 1823 | " \n", 1824 | " \n", 1825 | " \n", 1826 | " \n", 1827 | " \n", 1828 | " \n", 1829 | " \n", 1830 | " \n", 1831 | " \n", 1832 | " \n", 1833 | " \n", 1834 | " \n", 1835 | " \n", 1836 | " \n", 1837 | " \n", 1838 | " \n", 1839 | " \n", 1840 | " \n", 1841 | " \n", 1842 | " \n", 1843 | " \n", 1844 | " \n", 1845 | " \n", 1846 | " \n", 1847 | " \n", 1848 | " \n", 1849 | " \n", 1850 | " \n", 1851 | " \n", 1852 | " \n", 1853 | " \n", 1854 | " \n", 1855 | " \n", 1856 | " \n", 1857 | " \n", 1858 | " \n", 1859 | " \n", 1860 | " \n", 1861 | "
PriceOrderNumberQuantityDifferenceQuantityDifference_
070950047C3FA130EB2F3B3210.00.0
17120006F35516BC930AD702.02.0
2712000C363747FE0C9C3E92.02.0
3715500C365FC27C2C8A6282.02.0
4717500CBADF86F77EA578A1.01.0
5719500E1C9F25394A679A61.01.0
6720000EFB5D16396077A472.02.0
77250006BF7787F751753F74.04.0
87250006FBD516B9E4382831.01.0
9731500E14C560FA31A855A1.01.0
107360006144D60F9AB47CF41.01.0
117360006F35D1636FCB540B1.01.0
1273800061C4DA0743E526254.04.0
13740000E8ABBA801F3C015C2.02.0
147440004363747FF224D5441.01.0
15745000C90D7253BB41A04110.010.0
16746000436B747F53EA370A1.01.0
17747000EFBD516BFFA9E3E92.02.0
18748000E14C5A0F8D8C6FCC5.05.0
19748000C3EB7477400523255.05.0
207540006FB5516B20B104F13.03.0
21755000490D725B88E56DE510.010.0
227560006FB5D1635CBF40FF1.01.0
23757000642F9A8039E51EE53.03.0
2475700064A79A88F278D7781.01.0
2575900061C4DA0FFA35DC755.05.0
26760000C3E3747718A3FBC415.015.0
277600006F3DD163510F354F4.04.0
28768500ECA71A88142AF92B1.01.0
29770000C3637477EF36D2566.06.0
30770000EF35516B374E1B8E6.06.0
31770000436B747707D5EAF62.02.0
32770000EFB5516B13DFF7E06.06.0
3377000043EB7477305D137D6.06.0
\n", 1862 | "
" 1863 | ], 1864 | "text/plain": [ 1865 | " Price OrderNumber QuantityDifference QuantityDifference_\n", 1866 | "0 709500 47C3FA130EB2F3B3 210.0 0.0\n", 1867 | "1 712000 6F35516BC930AD70 2.0 2.0\n", 1868 | "2 712000 C363747FE0C9C3E9 2.0 2.0\n", 1869 | "3 715500 C365FC27C2C8A628 2.0 2.0\n", 1870 | "4 717500 CBADF86F77EA578A 1.0 1.0\n", 1871 | "5 719500 E1C9F25394A679A6 1.0 1.0\n", 1872 | "6 720000 EFB5D16396077A47 2.0 2.0\n", 1873 | "7 725000 6BF7787F751753F7 4.0 4.0\n", 1874 | "8 725000 6FBD516B9E438283 1.0 1.0\n", 1875 | "9 731500 E14C560FA31A855A 1.0 1.0\n", 1876 | "10 736000 6144D60F9AB47CF4 1.0 1.0\n", 1877 | "11 736000 6F35D1636FCB540B 1.0 1.0\n", 1878 | "12 738000 61C4DA0743E52625 4.0 4.0\n", 1879 | "13 740000 E8ABBA801F3C015C 2.0 2.0\n", 1880 | "14 744000 4363747FF224D544 1.0 1.0\n", 1881 | "15 745000 C90D7253BB41A041 10.0 10.0\n", 1882 | "16 746000 436B747F53EA370A 1.0 1.0\n", 1883 | "17 747000 EFBD516BFFA9E3E9 2.0 2.0\n", 1884 | "18 748000 E14C5A0F8D8C6FCC 5.0 5.0\n", 1885 | "19 748000 C3EB747740052325 5.0 5.0\n", 1886 | "20 754000 6FB5516B20B104F1 3.0 3.0\n", 1887 | "21 755000 490D725B88E56DE5 10.0 10.0\n", 1888 | "22 756000 6FB5D1635CBF40FF 1.0 1.0\n", 1889 | "23 757000 642F9A8039E51EE5 3.0 3.0\n", 1890 | "24 757000 64A79A88F278D778 1.0 1.0\n", 1891 | "25 759000 61C4DA0FFA35DC75 5.0 5.0\n", 1892 | "26 760000 C3E3747718A3FBC4 15.0 15.0\n", 1893 | "27 760000 6F3DD163510F354F 4.0 4.0\n", 1894 | "28 768500 ECA71A88142AF92B 1.0 1.0\n", 1895 | "29 770000 C3637477EF36D256 6.0 6.0\n", 1896 | "30 770000 EF35516B374E1B8E 6.0 6.0\n", 1897 | "31 770000 436B747707D5EAF6 2.0 2.0\n", 1898 | "32 770000 EFB5516B13DFF7E0 6.0 6.0\n", 1899 | "33 770000 43EB7477305D137D 6.0 6.0" 1900 | ] 1901 | }, 1902 | "execution_count": 11, 1903 | "metadata": {}, 1904 | "output_type": "execute_result" 1905 | } 1906 | ], 1907 | "source": [ 1908 | "order_book_ask" 1909 | ] 1910 | }, 1911 | { 1912 | "cell_type": "code", 1913 | "execution_count": 15, 1914 | "metadata": {}, 1915 | "outputs": [ 1916 | { 1917 | "data": { 1918 | "text/html": [ 1919 | "
\n", 1920 | "\n", 1933 | "\n", 1934 | " \n", 1935 | " \n", 1936 | " \n", 1937 | " \n", 1938 | " \n", 1939 | " \n", 1940 | " \n", 1941 | " \n", 1942 | " \n", 1943 | " \n", 1944 | " \n", 1945 | " \n", 1946 | " \n", 1947 | " \n", 1948 | " \n", 1949 | " \n", 1950 | " \n", 1951 | " \n", 1952 | " \n", 1953 | " \n", 1954 | " \n", 1955 | " \n", 1956 | " \n", 1957 | " \n", 1958 | " \n", 1959 | " \n", 1960 | " \n", 1961 | " \n", 1962 | " \n", 1963 | " \n", 1964 | " \n", 1965 | " \n", 1966 | " \n", 1967 | " \n", 1968 | " \n", 1969 | " \n", 1970 | " \n", 1971 | " \n", 1972 | " \n", 1973 | " \n", 1974 | " \n", 1975 | " \n", 1976 | " \n", 1977 | " \n", 1978 | " \n", 1979 | " \n", 1980 | " \n", 1981 | " \n", 1982 | " \n", 1983 | " \n", 1984 | " \n", 1985 | " \n", 1986 | " \n", 1987 | " \n", 1988 | " \n", 1989 | " \n", 1990 | " \n", 1991 | " \n", 1992 | " \n", 1993 | " \n", 1994 | " \n", 1995 | " \n", 1996 | " \n", 1997 | " \n", 1998 | " \n", 1999 | " \n", 2000 | " \n", 2001 | " \n", 2002 | " \n", 2003 | " \n", 2004 | " \n", 2005 | " \n", 2006 | " \n", 2007 | " \n", 2008 | " \n", 2009 | " \n", 2010 | " \n", 2011 | " \n", 2012 | " \n", 2013 | " \n", 2014 | " \n", 2015 | " \n", 2016 | " \n", 2017 | " \n", 2018 | " \n", 2019 | " \n", 2020 | " \n", 2021 | " \n", 2022 | " \n", 2023 | " \n", 2024 | " \n", 2025 | " \n", 2026 | " \n", 2027 | " \n", 2028 | " \n", 2029 | " \n", 2030 | " \n", 2031 | " \n", 2032 | " \n", 2033 | " \n", 2034 | " \n", 2035 | " \n", 2036 | " \n", 2037 | " \n", 2038 | " \n", 2039 | " \n", 2040 | " \n", 2041 | " \n", 2042 | " \n", 2043 | " \n", 2044 | " \n", 2045 | " \n", 2046 | " \n", 2047 | " \n", 2048 | " \n", 2049 | " \n", 2050 | " \n", 2051 | " \n", 2052 | " \n", 2053 | " \n", 2054 | " \n", 2055 | " \n", 2056 | " \n", 2057 | " \n", 2058 | " \n", 2059 | " \n", 2060 | " \n", 2061 | " \n", 2062 | " \n", 2063 | "
PriceQuantityDifference
0709500210.0
17120004.0
27155002.0
37175001.0
47195001.0
57200002.0
67250005.0
77315001.0
87360002.0
97380004.0
107400002.0
117440001.0
1274500010.0
137460001.0
147470002.0
1574800010.0
167540003.0
1775500010.0
187560001.0
197570004.0
207590005.0
2176000015.0
227685001.0
2377000026.0
\n", 2064 | "
" 2065 | ], 2066 | "text/plain": [ 2067 | " Price QuantityDifference\n", 2068 | "0 709500 210.0\n", 2069 | "1 712000 4.0\n", 2070 | "2 715500 2.0\n", 2071 | "3 717500 1.0\n", 2072 | "4 719500 1.0\n", 2073 | "5 720000 2.0\n", 2074 | "6 725000 5.0\n", 2075 | "7 731500 1.0\n", 2076 | "8 736000 2.0\n", 2077 | "9 738000 4.0\n", 2078 | "10 740000 2.0\n", 2079 | "11 744000 1.0\n", 2080 | "12 745000 10.0\n", 2081 | "13 746000 1.0\n", 2082 | "14 747000 2.0\n", 2083 | "15 748000 10.0\n", 2084 | "16 754000 3.0\n", 2085 | "17 755000 10.0\n", 2086 | "18 756000 1.0\n", 2087 | "19 757000 4.0\n", 2088 | "20 759000 5.0\n", 2089 | "21 760000 15.0\n", 2090 | "22 768500 1.0\n", 2091 | "23 770000 26.0" 2092 | ] 2093 | }, 2094 | "execution_count": 15, 2095 | "metadata": {}, 2096 | "output_type": "execute_result" 2097 | } 2098 | ], 2099 | "source": [ 2100 | "order_book_ask_sum" 2101 | ] 2102 | }, 2103 | { 2104 | "cell_type": "code", 2105 | "execution_count": 13, 2106 | "metadata": {}, 2107 | "outputs": [ 2108 | { 2109 | "data": { 2110 | "text/html": [ 2111 | "
\n", 2112 | "\n", 2125 | "\n", 2126 | " \n", 2127 | " \n", 2128 | " \n", 2129 | " \n", 2130 | " \n", 2131 | " \n", 2132 | " \n", 2133 | " \n", 2134 | " \n", 2135 | " \n", 2136 | " \n", 2137 | " \n", 2138 | " \n", 2139 | " \n", 2140 | " \n", 2141 | " \n", 2142 | " \n", 2143 | " \n", 2144 | " \n", 2145 | " \n", 2146 | " \n", 2147 | " \n", 2148 | " \n", 2149 | " \n", 2150 | " \n", 2151 | " \n", 2152 | " \n", 2153 | " \n", 2154 | " \n", 2155 | " \n", 2156 | " \n", 2157 | " \n", 2158 | " \n", 2159 | " \n", 2160 | " \n", 2161 | " \n", 2162 | " \n", 2163 | " \n", 2164 | " \n", 2165 | " \n", 2166 | " \n", 2167 | " \n", 2168 | " \n", 2169 | " \n", 2170 | " \n", 2171 | " \n", 2172 | " \n", 2173 | " \n", 2174 | " \n", 2175 | " \n", 2176 | " \n", 2177 | " \n", 2178 | " \n", 2179 | " \n", 2180 | " \n", 2181 | " \n", 2182 | " \n", 2183 | " \n", 2184 | " \n", 2185 | " \n", 2186 | " \n", 2187 | " \n", 2188 | " \n", 2189 | " \n", 2190 | " \n", 2191 | " \n", 2192 | " \n", 2193 | " \n", 2194 | " \n", 2195 | " \n", 2196 | " \n", 2197 | " \n", 2198 | " \n", 2199 | " \n", 2200 | " \n", 2201 | " \n", 2202 | " \n", 2203 | " \n", 2204 | " \n", 2205 | " \n", 2206 | " \n", 2207 | " \n", 2208 | " \n", 2209 | " \n", 2210 | " \n", 2211 | " \n", 2212 | " \n", 2213 | " \n", 2214 | " \n", 2215 | "
PriceQuantityDifference
0709500210.0
17070001.0
27020001.0
37010001.0
47005002.0
56990002.0
66975001.0
76955001.0
86905001.0
969000077.0
106850002.0
116835004.0
12682000100.0
136730008.0
146635003.0
156535003.0
\n", 2216 | "
" 2217 | ], 2218 | "text/plain": [ 2219 | " Price QuantityDifference\n", 2220 | "0 709500 210.0\n", 2221 | "1 707000 1.0\n", 2222 | "2 702000 1.0\n", 2223 | "3 701000 1.0\n", 2224 | "4 700500 2.0\n", 2225 | "5 699000 2.0\n", 2226 | "6 697500 1.0\n", 2227 | "7 695500 1.0\n", 2228 | "8 690500 1.0\n", 2229 | "9 690000 77.0\n", 2230 | "10 685000 2.0\n", 2231 | "11 683500 4.0\n", 2232 | "12 682000 100.0\n", 2233 | "13 673000 8.0\n", 2234 | "14 663500 3.0\n", 2235 | "15 653500 3.0" 2236 | ] 2237 | }, 2238 | "execution_count": 13, 2239 | "metadata": {}, 2240 | "output_type": "execute_result" 2241 | } 2242 | ], 2243 | "source": [ 2244 | "order_book_bid_sum" 2245 | ] 2246 | }, 2247 | { 2248 | "cell_type": "code", 2249 | "execution_count": null, 2250 | "metadata": {}, 2251 | "outputs": [], 2252 | "source": [] 2253 | } 2254 | ], 2255 | "metadata": { 2256 | "interpreter": { 2257 | "hash": "ed225720166559e7176d3793db16a2fd8d295f725007103b21ac3099d2a89ee8" 2258 | }, 2259 | "kernelspec": { 2260 | "display_name": "Python 3.9.7 ('base')", 2261 | "language": "python", 2262 | "name": "python3" 2263 | }, 2264 | "language_info": { 2265 | "codemirror_mode": { 2266 | "name": "ipython", 2267 | "version": 3 2268 | }, 2269 | "file_extension": ".py", 2270 | "mimetype": "text/x-python", 2271 | "name": "python", 2272 | "nbconvert_exporter": "python", 2273 | "pygments_lexer": "ipython3", 2274 | "version": "3.9.7" 2275 | }, 2276 | "orig_nbformat": 4 2277 | }, 2278 | "nbformat": 4, 2279 | "nbformat_minor": 2 2280 | } 2281 | -------------------------------------------------------------------------------- /feature_engineering.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### **High Frequency Trading Strategies Design using ML and DL**" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# -*- coding: utf-8 -*-\n", 17 | "\"\"\"\n", 18 | "Created on Sun Mar 13 22:40:00 2022\n", 19 | "\n", 20 | "@author: Bradley\n", 21 | "\n", 22 | "HFT: Machine Learning Techniques on Full Orderbook Tick Data\n", 23 | "\"\"\"" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 18, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "import numpy as np\n", 33 | "import pandas as pd\n", 34 | "import matplotlib.pyplot as plt\n", 35 | "import seaborn as sns\n", 36 | "import statsmodels.api as sm\n", 37 | "from datetime import datetime\n", 38 | "import csv\n", 39 | "\n", 40 | "plt.rcParams['font.family'] = ['sans-serif']\n", 41 | "plt.rcParams['font.sans-serif'] = ['SimHei']\n", 42 | "plt.rcParams['axes.unicode_minus'] = False\n", 43 | "plt.rcParams['figure.figsize'] = (8,5) #提前设置图片形状大小\n", 44 | "\n", 45 | "%config InlineBackend.figure_format = 'svg'\n", 46 | "\n", 47 | "%matplotlib inline\n", 48 | "\n", 49 | "import warnings\n", 50 | "warnings.filterwarnings('ignore') # 忽略一些warnings\n", 51 | "\n", 52 | "# from IPython.core.interactiveshell import InteractiveShell\n", 53 | "# InteractiveShell.ast_node_interactivity = \"all\"\n", 54 | "\n", 55 | "from IPython.display import display\n", 56 | "pd.set_option('expand_frame_repr', False)\n", 57 | "pd.set_option('display.unicode.ambiguous_as_wide', True)\n", 58 | "pd.set_option('display.unicode.east_asian_width', True)\n", 59 | "pd.set_option('display.width', 180)" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "### **Feature Engineering**" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": 19, 72 | "metadata": {}, 73 | "outputs": [], 74 | "source": [ 75 | "# read the data\n", 76 | "def order_book(month,day):\n", 77 | " datapath = './data/order_book_3_2014' + '_' + str(month) + '_' + str(day) + '.csv'\n", 78 | " order = pd.read_csv(datapath,sep=',') #隔一个取一个\n", 79 | " bid_price_1 = np.array(order['Bid'][1::4].astype(\"float64\"))/100 #bid1\n", 80 | " bid_price_2 = np.array(order['Bid'][2::4].astype(\"float64\"))/100 #bid2\n", 81 | " bid_price_3 = np.array(order['Bid'][3::4].astype(\"float64\"))/100 #bid3\n", 82 | " timestamp = np.array(order['Bid_Quantity'][0::4]) #每个tick data的时间\n", 83 | " bid_quantity_1 = np.array(order['Bid_Quantity'][1::4].astype(\"float64\"))/100 #bidqty1\n", 84 | " bid_quantity_2 = np.array(order['Bid_Quantity'][2::4].astype(\"float64\"))/100 #bidqty2\n", 85 | " bid_quantity_3 = np.array(order['Bid_Quantity'][3::4].astype(\"float64\"))/100 #bidqty3\n", 86 | " ask_price_1 = np.array(order['Ask'][1::4].astype(\"float64\"))/100 #ask1\n", 87 | " ask_price_2 = np.array(order['Ask'][2::4].astype(\"float64\"))/100 #ask2\n", 88 | " ask_price_3 = np.array(order['Ask'][3::4].astype(\"float64\"))/100 #ask3\n", 89 | " ask_quantity_1 = np.array(order['Ask_Quantity'][1::4].astype(\"float64\"))/100 #askqty1\n", 90 | " ask_quantity_2 = np.array(order['Ask_Quantity'][2::4].astype(\"float64\"))/100 #askqty1\n", 91 | " ask_quantity_3 = np.array(order['Ask_Quantity'][3::4].astype(\"float64\"))/100 #askqty1\n", 92 | " \n", 93 | " # 填充缺失值为0\n", 94 | " bid_quantity_1[np.isnan(bid_quantity_1)] = 0\n", 95 | " bid_quantity_2[np.isnan(bid_quantity_2)] = 0\n", 96 | " bid_quantity_3[np.isnan(bid_quantity_3)] = 0\n", 97 | " ask_quantity_1[np.isnan(ask_quantity_1)] = 0\n", 98 | " ask_quantity_2[np.isnan(ask_quantity_2)] = 0\n", 99 | " ask_quantity_3[np.isnan(ask_quantity_3)] = 0\n", 100 | " \n", 101 | " return timestamp,order,bid_price_1,bid_price_2,bid_price_3,bid_quantity_1,\\\n", 102 | " bid_quantity_2,bid_quantity_3,ask_price_1,ask_price_2,ask_price_3,ask_quantity_1,\\\n", 103 | " ask_quantity_2,ask_quantity_3" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 20, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [ 112 | "# time_second是将timestamp数据格式转换为全按分钟计算\n", 113 | "# time_second_basic是讲上面的转换结果减去一个32400\n", 114 | "# 作用是使得上午的交易时间[0, 9000]对应09:00 ~ 11:30，下午的交易时间[14400, 25200]对应13:00 ~ 16:00，借以将上下午分开\n", 115 | "def time_transform(timestamp_time):\n", 116 | " time_second_basic = []\n", 117 | " time_second = []\n", 118 | " for i in range(0,len(timestamp_time),1):\n", 119 | " second = float(timestamp_time[i][11])*36000 + float(timestamp_time[i][12])*3600+\\\n", 120 | " float(timestamp_time[i][14])*600 + float(timestamp_time[i][15])*60+\\\n", 121 | " float(timestamp_time[i][17])*10 + float(timestamp_time[i][18]) \n", 122 | " time_second_basic.append(second - 32400.0)\n", 123 | " time_second.append(second)\n", 124 | " return np.array(time_second),np.array(time_second_basic)" 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": 21, 130 | "metadata": {}, 131 | "outputs": [], 132 | "source": [ 133 | "# rise factor\n", 134 | "def rise_ask(Ask1,timestamp_time_second,before_time):\n", 135 | " Ask1[Ask1 == 0] = np.mean(Ask1)\n", 136 | " rise_ratio = []\n", 137 | " index = np.where(timestamp_time_second >= before_time)[0][0]\n", 138 | " #open first before_time mins\n", 139 | " for i in range(0, index, 1):\n", 140 | " rise_ratio_ = round((Ask1[i] - Ask1[0])*(1.0)/Ask1[0]*100,5)\n", 141 | " rise_ratio.append(rise_ratio_)\n", 142 | " for i in range(index,len(Ask1),1):\n", 143 | " #print np.where(timestamp_time_second[:i] >= timestamp_time_second[i] - before_time)\n", 144 | " #print timestamp_time_second[i],timestamp_time_second[i] - before_time\n", 145 | " index_start = np.where(timestamp_time_second[:i] >= timestamp_time_second[i] - before_time)[0][0]\n", 146 | " rise_ratio_ = round((Ask1[i] - Ask1[index_start])*(1.0)/Ask1[index_start]*100,5)\n", 147 | " rise_ratio.append(rise_ratio_)\n", 148 | " return np.array(rise_ratio)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 22, 154 | "metadata": {}, 155 | "outputs": [], 156 | "source": [ 157 | "# level-3 OBI and Depth Ratio\n", 158 | "def weight_pecentage(w1,w2,w3,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3):\n", 159 | " Weight_Ask = (w1 * ask_quantity_1 + w2 * ask_quantity_2 + w3 * ask_quantity_3)\n", 160 | " Weight_Bid = (w1 * bid_quantity_1 + w2 * bid_quantity_2 + w3 * bid_quantity_3)\n", 161 | "\n", 162 | " W_AB = Weight_Ask/Weight_Bid\n", 163 | " W_A_B = (Weight_Ask - Weight_Bid)/(Weight_Ask + Weight_Bid)\n", 164 | " return W_AB, W_A_B" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": 23, 170 | "metadata": {}, 171 | "outputs": [], 172 | "source": [ 173 | "# 定义每个second的要predict的label，以及对应second级别的各类输入factor/feature，由于是second级别，输出的dataframe会比原来小\n", 174 | "# 可以根据需要更改成tick级别的label和feature数据，这样输出的dataframe和原来大小差不多\n", 175 | "def traded_label_one_second(time1,time2,time_second_basic,bid_price_1,ask_price_1, bid_quantity_1, ask_quantity_1, traded_time,\\\n", 176 | " rise_ratio_ask_1,rise_ratio_ask_2,rise_ratio_ask_3,rise_ratio_ask_4,\\\n", 177 | " rise_ratio_ask_5,rise_ratio_ask_6,rise_ratio_ask_7,rise_ratio_ask_8,\\\n", 178 | " rise_ratio_ask_9,rise_ratio_ask_10,rise_ratio_ask_11,rise_ratio_ask_12,\\\n", 179 | " rise_ratio_ask_13,rise_ratio_ask_14,rise_ratio_ask_15,rise_ratio_ask_16,\\\n", 180 | " rise_ratio_ask_17,rise_ratio_ask_18,rise_ratio_ask_19,rise_ratio_ask_20,\\\n", 181 | " rise_ratio_ask_21,rise_ratio_ask_22,rise_ratio_ask_23,rise_ratio_ask_24,\\\n", 182 | " rise_ratio_ask_25,rise_ratio_ask_26,rise_ratio_ask_27,rise_ratio_ask_28,\\\n", 183 | " rise_ratio_ask_29,rise_ratio_ask_30,W_AB_100, W_A_B_100, W_AB_010, W_A_B_010,\\\n", 184 | " W_AB_001,W_A_B_001, W_AB_910, W_A_B_910, W_AB_820, W_A_B_820, W_AB_730,\\\n", 185 | " W_A_B_730,W_AB_640, W_A_B_640, W_AB_550, W_A_B_550,W_AB_721, W_A_B_721,\\\n", 186 | " W_AB_532,W_A_B_532, W_AB_111, W_A_B_111, W_AB_190, W_A_B_190, W_AB_280,\\\n", 187 | " W_A_B_280,W_AB_370, W_A_B_370, W_AB_460, W_A_B_460, W_AB_127, W_A_B_127,\\\n", 188 | " W_AB_235, W_A_B_235):\n", 189 | " global index\n", 190 | " \n", 191 | " traded = []\n", 192 | " index_ = []\n", 193 | " \n", 194 | " rise_ratio_second_1 = []\n", 195 | " rise_ratio_second_2 = [] \n", 196 | " rise_ratio_second_3 = []\n", 197 | " rise_ratio_second_4 = []\n", 198 | " rise_ratio_second_5 = []\n", 199 | " rise_ratio_second_6 = []\n", 200 | " rise_ratio_second_7 = []\n", 201 | " rise_ratio_second_8 = []\n", 202 | " rise_ratio_second_9 = []\n", 203 | " rise_ratio_second_10 = []\n", 204 | " rise_ratio_second_11 = []\n", 205 | " rise_ratio_second_12 = []\n", 206 | " rise_ratio_second_13 = []\n", 207 | " rise_ratio_second_14 = []\n", 208 | " rise_ratio_second_15 = []\n", 209 | " rise_ratio_second_16 = []\n", 210 | " rise_ratio_second_17 = []\n", 211 | " rise_ratio_second_18 = []\n", 212 | " rise_ratio_second_19 = []\n", 213 | " rise_ratio_second_20 = []\n", 214 | " rise_ratio_second_21 = []\n", 215 | " rise_ratio_second_22 = []\n", 216 | " rise_ratio_second_23 = []\n", 217 | " rise_ratio_second_24 = []\n", 218 | " rise_ratio_second_25 = []\n", 219 | " rise_ratio_second_26 = []\n", 220 | " rise_ratio_second_27 = []\n", 221 | " rise_ratio_second_28 = []\n", 222 | " rise_ratio_second_29 = []\n", 223 | " rise_ratio_second_30 = []\n", 224 | "\n", 225 | " w_divid_100 = []\n", 226 | " w_diff_100 = []\n", 227 | " w_divid_010 = []\n", 228 | " w_diff_010 = []\n", 229 | " w_divid_001 = []\n", 230 | " w_diff_001 = []\n", 231 | " w_divid_910 = []\n", 232 | " w_diff_910 = []\n", 233 | " w_divid_820 = []\n", 234 | " w_diff_820 = []\n", 235 | " w_divid_730 = []\n", 236 | " w_diff_730 = []\n", 237 | " w_divid_640 = []\n", 238 | " w_diff_640 = []\n", 239 | " w_divid_550 = []\n", 240 | " w_diff_550 = []\n", 241 | " w_divid_721 = []\n", 242 | " w_diff_721 = []\n", 243 | " w_divid_532 = []\n", 244 | " w_diff_532 = []\n", 245 | " w_divid_111 = []\n", 246 | " w_diff_111 = []\n", 247 | " w_divid_190 = []\n", 248 | " w_diff_190 = []\n", 249 | " w_divid_280 = []\n", 250 | " w_diff_280 = []\n", 251 | " w_divid_370 = []\n", 252 | " w_diff_370 = []\n", 253 | " w_divid_460 = []\n", 254 | " w_diff_460 = []\n", 255 | " w_divid_127 = []\n", 256 | " w_diff_127 = []\n", 257 | " w_divid_235 = []\n", 258 | " w_diff_235 = []\n", 259 | " spread = []\n", 260 | " ask = []\n", 261 | " bid = []\n", 262 | " # micsecond = time_second_basic[np.where((time_second_basic >= time1) & (time_second_basic <= time2))]\n", 263 | " # for i in range(0,len(micsecond),1):\n", 264 | " \n", 265 | " # index = np.where(time_second_basic <= micsecond[i])[0][-1]\n", 266 | " # if i == 0:\n", 267 | " # index_.append(index)\n", 268 | " # if i == len(micsecond) - 1:\n", 269 | " # index_.append(index)\n", 270 | " # if micsecond[i] <= 25200 - traded_time:\n", 271 | " # index_min = np.where(time_second_basic <= micsecond[i] + traded_time)[0][-1]\n", 272 | " # if bid_price_1[index] > min(ask_price_1[index:index_min]):\n", 273 | " # traded.append(1)\n", 274 | " # else:\n", 275 | " # traded.append(0)\n", 276 | " # elif micsecond[i] > 25200 - traded_time:\n", 277 | " # if bid_price_1[index] > ask_price_1[-1]:\n", 278 | " # traded.append(1)\n", 279 | " # else:\n", 280 | " # traded.append(0)\n", 281 | "\n", 282 | " if time1 == 0:\n", 283 | " index_one = np.where(time_second_basic <= 0)[0][-1]\n", 284 | " elif time1 == 14400:\n", 285 | " index_one = np.where(time_second_basic <= 14400)[0][-1]\n", 286 | " \n", 287 | " for i in range(time1, time2, 1):\n", 288 | " if i == 0 or i == 14400:\n", 289 | " index_array = np.where(time_second_basic <= i)[-1]\n", 290 | " else:\n", 291 | " index_array = np.where((time_second_basic < i+1) & (time_second_basic >= i))[-1]\n", 292 | " \n", 293 | " if len(index_array) > 0:\n", 294 | " index = index_array[-1]\n", 295 | " if i == time1:\n", 296 | " index_.append(index)\n", 297 | " if i == time2 - 1:\n", 298 | " index_.append(index)\n", 299 | " if i < 25200 - traded_time:\n", 300 | " index_min = np.where(time_second_basic <= i + traded_time)[0][-1]\n", 301 | " traded_min = ask_price_1[index:index_min]\n", 302 | " if bid_price_1[index] > min(traded_min):\n", 303 | " traded.append(1)\n", 304 | " else:\n", 305 | " traded.append(0)\n", 306 | " elif i >= 25200 - traded_time:\n", 307 | " if bid_price_1[index] > ask_price_1[-1]:\n", 308 | " traded.append(1)\n", 309 | " else:\n", 310 | " traded.append(0)\n", 311 | " ask0 = ask_price_1[index]\n", 312 | " bid0 = bid_price_1[index]\n", 313 | " ask.append(ask0)\n", 314 | " bid.append(bid0)\n", 315 | " spread.append(10000*(ask0-bid0)*2/(ask0+bid0))\n", 316 | " rise_ratio_second_1.append(rise_ratio_ask_1[(index - index_one)])\n", 317 | " rise_ratio_second_2.append(rise_ratio_ask_2[(index - index_one)])\n", 318 | " rise_ratio_second_3.append(rise_ratio_ask_3[(index - index_one)])\n", 319 | " rise_ratio_second_4.append(rise_ratio_ask_4[(index - index_one)])\n", 320 | " rise_ratio_second_5.append(rise_ratio_ask_5[(index - index_one)])\n", 321 | " rise_ratio_second_6.append(rise_ratio_ask_6[(index - index_one)])\n", 322 | " rise_ratio_second_7.append(rise_ratio_ask_7[(index - index_one)])\n", 323 | " rise_ratio_second_8.append(rise_ratio_ask_8[(index - index_one)])\n", 324 | " rise_ratio_second_9.append(rise_ratio_ask_9[(index - index_one)])\n", 325 | " rise_ratio_second_10.append(rise_ratio_ask_10[(index - index_one)])\n", 326 | " rise_ratio_second_11.append(rise_ratio_ask_11[(index - index_one)])\n", 327 | " rise_ratio_second_12.append(rise_ratio_ask_12[(index - index_one)])\n", 328 | " rise_ratio_second_13.append(rise_ratio_ask_13[(index - index_one)])\n", 329 | " rise_ratio_second_14.append(rise_ratio_ask_14[(index - index_one)])\n", 330 | " rise_ratio_second_15.append(rise_ratio_ask_15[(index - index_one)])\n", 331 | " rise_ratio_second_16.append(rise_ratio_ask_16[(index - index_one)])\n", 332 | " rise_ratio_second_17.append(rise_ratio_ask_17[(index - index_one)])\n", 333 | " rise_ratio_second_18.append(rise_ratio_ask_18[(index - index_one)])\n", 334 | " rise_ratio_second_19.append(rise_ratio_ask_19[(index - index_one)])\n", 335 | " rise_ratio_second_20.append(rise_ratio_ask_20[(index - index_one)])\n", 336 | " rise_ratio_second_21.append(rise_ratio_ask_21[(index - index_one)])\n", 337 | " rise_ratio_second_22.append(rise_ratio_ask_22[(index - index_one)])\n", 338 | " rise_ratio_second_23.append(rise_ratio_ask_23[(index - index_one)])\n", 339 | " rise_ratio_second_24.append(rise_ratio_ask_24[(index - index_one)])\n", 340 | " rise_ratio_second_25.append(rise_ratio_ask_25[(index - index_one)])\n", 341 | " rise_ratio_second_26.append(rise_ratio_ask_26[(index - index_one)])\n", 342 | " rise_ratio_second_27.append(rise_ratio_ask_27[(index - index_one)])\n", 343 | " rise_ratio_second_28.append(rise_ratio_ask_28[(index - index_one)])\n", 344 | " rise_ratio_second_29.append(rise_ratio_ask_29[(index - index_one)])\n", 345 | " rise_ratio_second_30.append(rise_ratio_ask_30[(index - index_one)])\n", 346 | " \n", 347 | " w_divid_100.append(W_AB_100[index_one + (index - index_one)])\n", 348 | " w_diff_100.append(W_A_B_100[index_one + (index - index_one)])\n", 349 | " w_divid_010.append(W_AB_010[index_one + (index - index_one)])\n", 350 | " w_diff_010.append(W_A_B_010[index_one + (index - index_one)])\n", 351 | " w_divid_001.append(W_AB_001[index_one + (index - index_one)])\n", 352 | " w_diff_001.append(W_A_B_001[index_one + (index - index_one)])\n", 353 | " w_divid_910.append(W_AB_910[index_one + (index - index_one)])\n", 354 | " w_diff_910.append(W_A_B_910[index_one + (index - index_one)])\n", 355 | " w_divid_820.append(W_AB_820[index_one + (index - index_one)])\n", 356 | " w_diff_820.append(W_A_B_820[index_one + (index - index_one)])\n", 357 | " w_divid_730.append(W_AB_730[index_one + (index - index_one)])\n", 358 | " w_diff_730.append(W_A_B_730[index_one + (index - index_one)])\n", 359 | " w_divid_640.append(W_AB_640[index_one + (index - index_one)])\n", 360 | " w_diff_640.append(W_A_B_640[index_one + (index - index_one)])\n", 361 | " w_divid_550.append(W_AB_550[index_one + (index - index_one)])\n", 362 | " w_diff_550.append(W_A_B_550[index_one + (index - index_one)])\n", 363 | " w_divid_721.append(W_AB_721[index_one + (index - index_one)])\n", 364 | " w_diff_721.append(W_A_B_721[index_one + (index - index_one)])\n", 365 | " w_divid_532.append(W_AB_532[index_one + (index - index_one)])\n", 366 | " w_diff_532.append(W_A_B_532[index_one + (index - index_one)])\n", 367 | " w_divid_111.append(W_AB_111[index_one + (index - index_one)])\n", 368 | " w_diff_111.append(W_A_B_111[index_one + (index - index_one)])\n", 369 | " w_divid_190.append(W_AB_190[index_one + (index - index_one)])\n", 370 | " w_diff_190.append(W_A_B_190[index_one + (index - index_one)])\n", 371 | " w_divid_280.append(W_AB_280[index_one + (index - index_one)])\n", 372 | " w_diff_280.append(W_A_B_280[index_one + (index - index_one)])\n", 373 | " w_divid_370.append(W_AB_370[index_one + (index - index_one)])\n", 374 | " w_diff_370.append(W_A_B_370[index_one + (index - index_one)])\n", 375 | " w_divid_460.append(W_AB_460[index_one + (index - index_one)])\n", 376 | " w_diff_460.append(W_A_B_460[index_one + (index - index_one)])\n", 377 | " w_divid_127.append(W_AB_127[index_one + (index - index_one)])\n", 378 | " w_diff_127.append(W_A_B_127[index_one + (index - index_one)])\n", 379 | " w_divid_235.append(W_AB_235[index_one + (index - index_one)])\n", 380 | " w_diff_235.append(W_A_B_235[index_one + (index - index_one)])\n", 381 | " \n", 382 | " elif len(index_array) == 0:\n", 383 | " if i < 25200 - traded_time:\n", 384 | " index_min = np.where(time_second_basic <= i + traded_time)[0][-1]\n", 385 | " traded_min = ask_price_1[index:index_min]\n", 386 | " if bid_price_1[index] > min(traded_min):\n", 387 | " traded.append(1)\n", 388 | " else:\n", 389 | " traded.append(0)\n", 390 | " elif i >= 25200 - traded_time:\n", 391 | " if bid_price_1[index] > ask_price_1[-1]:\n", 392 | " traded.append(1)\n", 393 | " else:\n", 394 | " traded.append(0)\n", 395 | "\n", 396 | " ask0 = ask_price_1[index]\n", 397 | " bid0 = bid_price_1[index]\n", 398 | " ask.append(ask0)\n", 399 | " bid.append(bid0)\n", 400 | " spread.append(10000*(ask0-bid0)*2/(ask0+bid0))\n", 401 | " rise_ratio_second_1.append(rise_ratio_second_1[-1])\n", 402 | " rise_ratio_second_2.append(rise_ratio_second_2[-1])\n", 403 | " rise_ratio_second_3.append(rise_ratio_second_3[-1])\n", 404 | " rise_ratio_second_4.append(rise_ratio_second_4[-1])\n", 405 | " rise_ratio_second_5.append(rise_ratio_second_5[-1])\n", 406 | " rise_ratio_second_6.append(rise_ratio_second_6[-1])\n", 407 | " rise_ratio_second_7.append(rise_ratio_second_7[-1])\n", 408 | " rise_ratio_second_8.append(rise_ratio_second_8[-1])\n", 409 | " rise_ratio_second_9.append(rise_ratio_second_9[-1])\n", 410 | " rise_ratio_second_10.append(rise_ratio_second_10[-1])\n", 411 | " rise_ratio_second_11.append(rise_ratio_second_11[-1])\n", 412 | " rise_ratio_second_12.append(rise_ratio_second_12[-1])\n", 413 | " rise_ratio_second_13.append(rise_ratio_second_13[-1])\n", 414 | " rise_ratio_second_14.append(rise_ratio_second_14[-1])\n", 415 | " rise_ratio_second_15.append(rise_ratio_second_15[-1])\n", 416 | " rise_ratio_second_16.append(rise_ratio_second_16[-1])\n", 417 | " rise_ratio_second_17.append(rise_ratio_second_17[-1])\n", 418 | " rise_ratio_second_18.append(rise_ratio_second_18[-1])\n", 419 | " rise_ratio_second_19.append(rise_ratio_second_19[-1])\n", 420 | " rise_ratio_second_20.append(rise_ratio_second_20[-1])\n", 421 | " rise_ratio_second_21.append(rise_ratio_second_21[-1])\n", 422 | " rise_ratio_second_22.append(rise_ratio_second_22[-1])\n", 423 | " rise_ratio_second_23.append(rise_ratio_second_23[-1])\n", 424 | " rise_ratio_second_24.append(rise_ratio_second_24[-1])\n", 425 | " rise_ratio_second_25.append(rise_ratio_second_25[-1])\n", 426 | " rise_ratio_second_26.append(rise_ratio_second_26[-1])\n", 427 | " rise_ratio_second_27.append(rise_ratio_second_27[-1])\n", 428 | " rise_ratio_second_28.append(rise_ratio_second_28[-1])\n", 429 | " rise_ratio_second_29.append(rise_ratio_second_29[-1])\n", 430 | " rise_ratio_second_30.append(rise_ratio_second_30[-1])\n", 431 | " \n", 432 | " w_divid_100.append(w_divid_100[-1])\n", 433 | " w_diff_100.append(w_diff_100[-1])\n", 434 | " w_divid_010.append(w_divid_010[-1])\n", 435 | " w_diff_010.append(w_diff_010[-1])\n", 436 | " w_divid_001.append(w_divid_001[-1])\n", 437 | " w_diff_001.append(w_diff_001[-1])\n", 438 | " w_divid_910.append(w_divid_910[-1])\n", 439 | " w_diff_910.append(w_diff_910[-1])\n", 440 | " w_divid_820.append(w_divid_820[-1])\n", 441 | " w_diff_820.append(w_diff_820[-1])\n", 442 | " w_divid_730.append(w_divid_730[-1])\n", 443 | " w_diff_730.append(w_diff_730[-1])\n", 444 | " w_divid_640.append(w_divid_640[-1])\n", 445 | " w_diff_640.append(w_diff_640[-1])\n", 446 | " w_divid_550.append(w_divid_550[-1])\n", 447 | " w_diff_550.append(w_diff_550[-1])\n", 448 | " w_divid_721.append(w_divid_721[-1])\n", 449 | " w_diff_721.append(w_diff_721[-1])\n", 450 | " w_divid_532.append(w_divid_532[-1])\n", 451 | " w_diff_532.append(w_diff_532[-1])\n", 452 | " w_divid_111.append(w_divid_111[-1])\n", 453 | " w_diff_111.append(w_diff_111[-1])\n", 454 | " w_divid_190.append(w_divid_190[-1])\n", 455 | " w_diff_190.append(w_diff_190[-1])\n", 456 | " w_divid_280.append(w_divid_280[-1])\n", 457 | " w_diff_280.append(w_diff_280[-1])\n", 458 | " w_divid_370.append(w_divid_370[-1])\n", 459 | " w_diff_370.append(w_diff_370[-1])\n", 460 | " w_divid_460.append(w_divid_460[-1])\n", 461 | " w_diff_460.append(w_diff_460[-1])\n", 462 | " w_divid_127.append(w_divid_127[-1])\n", 463 | " w_diff_127.append(w_diff_127[-1])\n", 464 | " w_divid_235.append(w_divid_235[-1])\n", 465 | " w_diff_235.append(w_diff_235[-1])\n", 466 | " \n", 467 | " return traded,index_,rise_ratio_second_1,rise_ratio_second_2,rise_ratio_second_3,\\\n", 468 | " rise_ratio_second_4,rise_ratio_second_5,rise_ratio_second_6,rise_ratio_second_7,\\\n", 469 | " rise_ratio_second_8,rise_ratio_second_9,rise_ratio_second_10,rise_ratio_second_11,\\\n", 470 | " rise_ratio_second_12,rise_ratio_second_13,rise_ratio_second_14,rise_ratio_second_15,\\\n", 471 | " rise_ratio_second_16,rise_ratio_second_17,rise_ratio_second_18,rise_ratio_second_19,\\\n", 472 | " rise_ratio_second_20,rise_ratio_second_21,rise_ratio_second_22,rise_ratio_second_23,\\\n", 473 | " rise_ratio_second_24,rise_ratio_second_25,rise_ratio_second_26,rise_ratio_second_27,\\\n", 474 | " rise_ratio_second_28,rise_ratio_second_29,rise_ratio_second_30,w_divid_100,w_diff_100,\\\n", 475 | " w_divid_010,w_diff_010,w_divid_001,w_diff_001,w_divid_910,w_diff_910,w_divid_820,w_diff_820,\\\n", 476 | " w_divid_730,w_diff_730,w_divid_640,w_diff_640,w_divid_550,w_diff_550,w_divid_721,w_diff_721,\\\n", 477 | " w_divid_532,w_diff_532,w_divid_111,w_diff_111,w_divid_190,w_diff_190,w_divid_280,w_diff_280,\\\n", 478 | " w_divid_370,w_diff_370,w_divid_460,w_diff_460,w_divid_127,w_diff_127,w_divid_235,w_diff_235, spread, ask, bid" 479 | ] 480 | }, 481 | { 482 | "cell_type": "code", 483 | "execution_count": 24, 484 | "metadata": {}, 485 | "outputs": [], 486 | "source": [ 487 | "# 根据输入时间，分别得到上下午的因子值\n", 488 | "\n", 489 | "# # 上午：只取09:00 ~ 11:30作为交易区间，对应：\n", 490 | "# time1 = 0\n", 491 | "# time2 = 9000\n", 492 | "\n", 493 | "# # 下午：只取13:00 ~ 16:00作为交易区间，对应：\n", 494 | "# time1 = 14400\n", 495 | "# time2 = 25200\n", 496 | "\n", 497 | "\n", 498 | "def get_feature(time1, time2, traded_time,time_second_basic,bid_price_1,ask_price_1, bid_quantity_1, ask_quantity_1, rise_ratio_ask_1,\\\n", 499 | " rise_ratio_ask_2,rise_ratio_ask_3,rise_ratio_ask_4,rise_ratio_ask_5,\\\n", 500 | " rise_ratio_ask_6,rise_ratio_ask_7,rise_ratio_ask_8,rise_ratio_ask_9,\\\n", 501 | " rise_ratio_ask_10,rise_ratio_ask_11,rise_ratio_ask_12,rise_ratio_ask_13,\\\n", 502 | " rise_ratio_ask_14,rise_ratio_ask_15,rise_ratio_ask_16,rise_ratio_ask_17,\\\n", 503 | " rise_ratio_ask_18,rise_ratio_ask_19,rise_ratio_ask_20,rise_ratio_ask_21,\\\n", 504 | " rise_ratio_ask_22,rise_ratio_ask_23,rise_ratio_ask_24,rise_ratio_ask_25,\\\n", 505 | " rise_ratio_ask_26,rise_ratio_ask_27,rise_ratio_ask_28,rise_ratio_ask_29,\\\n", 506 | " rise_ratio_ask_30,W_AB_100, W_A_B_100, W_AB_010, W_A_B_010, W_AB_001,\\\n", 507 | " W_A_B_001, W_AB_910, W_A_B_910, W_AB_820, W_A_B_820, W_AB_730 , W_A_B_730,\\\n", 508 | " W_AB_640, W_A_B_640, W_AB_550, W_A_B_550,W_AB_721, W_A_B_721, W_AB_532,\\\n", 509 | " W_A_B_532, W_AB_111, W_A_B_111, W_AB_190, W_A_B_190, W_AB_280 , W_A_B_280,\\\n", 510 | " W_AB_370, W_A_B_370, W_AB_460, W_A_B_460, W_AB_127, W_A_B_127, W_AB_235, W_A_B_235):\n", 511 | "\n", 512 | " traded,index_,rise_ratio_second_1,rise_ratio_second_2,rise_ratio_second_3,\\\n", 513 | " rise_ratio_second_4,rise_ratio_second_5,rise_ratio_second_6,rise_ratio_second_7,\\\n", 514 | " rise_ratio_second_8,rise_ratio_second_9,rise_ratio_second_10,rise_ratio_second_11,\\\n", 515 | " rise_ratio_second_12,rise_ratio_second_13,rise_ratio_second_14,rise_ratio_second_15,\\\n", 516 | " rise_ratio_second_16,rise_ratio_second_17,rise_ratio_second_18,rise_ratio_second_19,\\\n", 517 | " rise_ratio_second_20,rise_ratio_second_21,rise_ratio_second_22,rise_ratio_second_23,\\\n", 518 | " rise_ratio_second_24,rise_ratio_second_25,rise_ratio_second_26,rise_ratio_second_27,\\\n", 519 | " rise_ratio_second_28,rise_ratio_second_29,rise_ratio_second_30,w_divid_100,w_diff_100,\\\n", 520 | " w_divid_010,w_diff_010,w_divid_001,w_diff_001,w_divid_910,w_diff_910,w_divid_820,w_diff_820,\\\n", 521 | " w_divid_730,w_diff_730,w_divid_640,w_diff_640,w_divid_550,w_diff_550,w_divid_721,w_diff_721,\\\n", 522 | " w_divid_532,w_diff_532,w_divid_111,w_diff_111,w_divid_190,w_diff_190,w_divid_280,w_diff_280,\\\n", 523 | " w_divid_370,w_diff_370,w_divid_460,w_diff_460,w_divid_127,w_diff_127,w_divid_235,w_diff_235, spread, ask, bid=\\\n", 524 | " traded_label_one_second(time1,time2,time_second_basic,bid_price_1,ask_price_1, bid_quantity_1, ask_quantity_1, traded_time,\\\n", 525 | " rise_ratio_ask_1,rise_ratio_ask_2,rise_ratio_ask_3,rise_ratio_ask_4,\\\n", 526 | " rise_ratio_ask_5,rise_ratio_ask_6,rise_ratio_ask_7,rise_ratio_ask_8,\\\n", 527 | " rise_ratio_ask_9,rise_ratio_ask_10,rise_ratio_ask_11,rise_ratio_ask_12,\\\n", 528 | " rise_ratio_ask_13,rise_ratio_ask_14,rise_ratio_ask_15,rise_ratio_ask_16,\\\n", 529 | " rise_ratio_ask_17,rise_ratio_ask_18,rise_ratio_ask_19,rise_ratio_ask_20,\\\n", 530 | " rise_ratio_ask_21,rise_ratio_ask_22,rise_ratio_ask_23,rise_ratio_ask_24,\\\n", 531 | " rise_ratio_ask_25,rise_ratio_ask_26,rise_ratio_ask_27,rise_ratio_ask_28,\\\n", 532 | " rise_ratio_ask_29,rise_ratio_ask_30,W_AB_100, W_A_B_100, W_AB_010, W_A_B_010,\\\n", 533 | " W_AB_001,W_A_B_001, W_AB_910, W_A_B_910, W_AB_820, W_A_B_820, W_AB_730,\\\n", 534 | " W_A_B_730,W_AB_640, W_A_B_640, W_AB_550, W_A_B_550,W_AB_721, W_A_B_721,\\\n", 535 | " W_AB_532,W_A_B_532, W_AB_111, W_A_B_111, W_AB_190, W_A_B_190, W_AB_280,\\\n", 536 | " W_A_B_280,W_AB_370, W_A_B_370, W_AB_460, W_A_B_460, W_AB_127, W_A_B_127,\\\n", 537 | " W_AB_235, W_A_B_235)\n", 538 | " \n", 539 | " data = np.array([traded,rise_ratio_second_1,rise_ratio_second_2,rise_ratio_second_3,\\\n", 540 | " rise_ratio_second_4,rise_ratio_second_5,rise_ratio_second_6,rise_ratio_second_7,\\\n", 541 | " rise_ratio_second_8,rise_ratio_second_9,rise_ratio_second_10,rise_ratio_second_11,\\\n", 542 | " rise_ratio_second_12,rise_ratio_second_13,rise_ratio_second_14,rise_ratio_second_15,\\\n", 543 | " rise_ratio_second_16,rise_ratio_second_17,rise_ratio_second_18,rise_ratio_second_19,\\\n", 544 | " rise_ratio_second_20,rise_ratio_second_21,rise_ratio_second_22,rise_ratio_second_23,\\\n", 545 | " rise_ratio_second_24,rise_ratio_second_25,rise_ratio_second_26,rise_ratio_second_27,\\\n", 546 | " rise_ratio_second_28,rise_ratio_second_29,rise_ratio_second_30,w_divid_100,w_diff_100,\\\n", 547 | " w_divid_010,w_diff_010,w_divid_001,w_diff_001,w_divid_910,w_diff_910,w_divid_820,w_diff_820,\\\n", 548 | " w_divid_730,w_diff_730,w_divid_640,w_diff_640,w_divid_550,w_diff_550,w_divid_721,w_diff_721,\\\n", 549 | " w_divid_532,w_diff_532,w_divid_111,w_diff_111,w_divid_190,w_diff_190,w_divid_280,w_diff_280,\\\n", 550 | " w_divid_370,w_diff_370,w_divid_460,w_diff_460,w_divid_127,w_diff_127,w_divid_235,w_diff_235, spread, ask, bid]).T\n", 551 | "\n", 552 | " return pd.DataFrame(data)#,traded_1 #, columns = ['label', 'rise', 'depth_divid', 'depth_diff'])\n" 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": 27, 558 | "metadata": {}, 559 | "outputs": [], 560 | "source": [ 561 | "def data(month,day,traded_time):\n", 562 | " \n", 563 | " # 读取所有bid1-3, ask1-3, bidqty1-3, askqty1-3，订单时间的序列\n", 564 | " # order_book_此处没有用到，是读取的dataframe本身\n", 565 | " timestamp, order_book_ ,bid_price_1, bid_price_2, bid_price_3, bid_quantity_1, bid_quantity_2, bid_quantity_3, ask_price_1, ask_price_2, ask_price_3, ask_quantity_1, ask_quantity_2, ask_quantity_3 = order_book(month,day)\n", 566 | " \n", 567 | "\n", 568 | " # 得到计算的时间戳\n", 569 | " time_second, time_second_basic = time_transform(timestamp)\n", 570 | " # Ask1是上午9:00之后卖一价\n", 571 | " Ask1 = ask_price_1[np.where(time_second_basic <= 0.0)[0][-1]:]\n", 572 | "\n", 573 | " # 设置不同权重创建一系列rise ratio factor作为输入feature\n", 574 | " before_time = 60.0 * 6\n", 575 | " rise_ratio_ask_1 = rise_ask(Ask1, time_second_basic, before_time)\n", 576 | " before_time = 60.0 * 6 + 30 \n", 577 | " rise_ratio_ask_2 = rise_ask(Ask1, time_second_basic, before_time)\n", 578 | " before_time = 60.0 * 7\n", 579 | " rise_ratio_ask_3 = rise_ask(Ask1, time_second_basic, before_time)\n", 580 | " before_time = 60.0 * 7 + 30 \n", 581 | " rise_ratio_ask_4 = rise_ask(Ask1, time_second_basic, before_time)\n", 582 | " before_time = 60.0 * 8\n", 583 | " rise_ratio_ask_5 = rise_ask(Ask1, time_second_basic, before_time)\n", 584 | " before_time = 60.0 * 8 + 30\n", 585 | " rise_ratio_ask_6 = rise_ask(Ask1, time_second_basic, before_time)\n", 586 | " before_time = 60.0 * 9\n", 587 | " rise_ratio_ask_7 = rise_ask(Ask1, time_second_basic, before_time)\n", 588 | " before_time = 60.0 * 9 + 30\n", 589 | " rise_ratio_ask_8 = rise_ask(Ask1, time_second_basic, before_time)\n", 590 | " before_time = 60.0 * 10\n", 591 | " rise_ratio_ask_9 = rise_ask(Ask1, time_second_basic, before_time)\n", 592 | " before_time = 60.0 * 10 + 30\n", 593 | " rise_ratio_ask_10 = rise_ask(Ask1, time_second_basic, before_time)\n", 594 | " before_time = 60.0 * 11\n", 595 | " rise_ratio_ask_11 = rise_ask(Ask1, time_second_basic, before_time)\n", 596 | " before_time = 60.0 * 11 + 30\n", 597 | " rise_ratio_ask_12 = rise_ask(Ask1, time_second_basic, before_time)\n", 598 | " before_time = 60.0 * 12\n", 599 | " rise_ratio_ask_13 = rise_ask(Ask1, time_second_basic, before_time)\n", 600 | " before_time = 60.0 * 12 + 30\n", 601 | " rise_ratio_ask_14 = rise_ask(Ask1, time_second_basic, before_time)\n", 602 | " before_time = 60.0 * 13\n", 603 | " rise_ratio_ask_15 = rise_ask(Ask1, time_second_basic, before_time)\n", 604 | " before_time = 60.0 * 13 + 30\n", 605 | " rise_ratio_ask_16 = rise_ask(Ask1, time_second_basic, before_time)\n", 606 | " before_time = 60.0 * 14\n", 607 | " rise_ratio_ask_17 = rise_ask(Ask1, time_second_basic, before_time)\n", 608 | " before_time = 60.0 * 14 + 30 \n", 609 | " rise_ratio_ask_18 = rise_ask(Ask1, time_second_basic, before_time)\n", 610 | " before_time = 60.0 * 15\n", 611 | " rise_ratio_ask_19 = rise_ask(Ask1, time_second_basic, before_time)\n", 612 | " before_time = 60.0 * 15 + 30\n", 613 | " rise_ratio_ask_20 = rise_ask(Ask1, time_second_basic, before_time)\n", 614 | " before_time = 60.0 * 16\n", 615 | " rise_ratio_ask_21 = rise_ask(Ask1, time_second_basic, before_time)\n", 616 | " before_time = 60.0 * 16 + 30\n", 617 | " rise_ratio_ask_22 = rise_ask(Ask1, time_second_basic, before_time)\n", 618 | " before_time = 60.0 * 17\n", 619 | " rise_ratio_ask_23 = rise_ask(Ask1, time_second_basic, before_time)\n", 620 | " before_time = 60.0 * 17 + 30\n", 621 | " rise_ratio_ask_24 = rise_ask(Ask1, time_second_basic, before_time)\n", 622 | " before_time = 60.0 * 18\n", 623 | " rise_ratio_ask_25 = rise_ask(Ask1, time_second_basic, before_time)\n", 624 | " before_time = 60.0 * 18 + 30\n", 625 | " rise_ratio_ask_26 = rise_ask(Ask1, time_second_basic, before_time)\n", 626 | " before_time = 60.0 * 19\n", 627 | " rise_ratio_ask_27 = rise_ask(Ask1, time_second_basic, before_time)\n", 628 | " before_time = 60.0 * 19 + 30\n", 629 | " rise_ratio_ask_28 = rise_ask(Ask1, time_second_basic, before_time)\n", 630 | " before_time = 60.0 * 20 \n", 631 | " rise_ratio_ask_29 = rise_ask(Ask1, time_second_basic, before_time)\n", 632 | " before_time = 60.0 * 20 + 30\n", 633 | " rise_ratio_ask_30 = rise_ask(Ask1, time_second_basic, before_time)\n", 634 | " \n", 635 | " # 设置不同权重创建一系列depth ratio factor、OBI signal作为输入feature\n", 636 | " W_AB_100 , W_A_B_100 = weight_pecentage(100, 0.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 637 | " W_AB_010 , W_A_B_010 = weight_pecentage(0.0, 100.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 638 | " W_AB_001 , W_A_B_001 = weight_pecentage(0.0, 0.0, 100.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 639 | " W_AB_910 , W_A_B_910 = weight_pecentage(90.0, 10.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 640 | " W_AB_820 , W_A_B_820 = weight_pecentage(80.0, 20.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 641 | " W_AB_730 , W_A_B_730 = weight_pecentage(70.0, 30.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 642 | " W_AB_640 , W_A_B_640 = weight_pecentage(60.0, 40.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 643 | " W_AB_550 , W_A_B_550 = weight_pecentage(50.0, 50.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 644 | " W_AB_721 , W_A_B_721 = weight_pecentage(70.0, 20.0, 10.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 645 | " W_AB_532 , W_A_B_532 = weight_pecentage(50.0, 30.0, 20.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 646 | " W_AB_111 , W_A_B_111 = weight_pecentage(1.0, 1.0, 1.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 647 | " W_AB_190 , W_A_B_190 = weight_pecentage(10.0, 90.0, 1.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 648 | " W_AB_280 , W_A_B_280 = weight_pecentage(20.0, 80.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 649 | " W_AB_370 , W_A_B_370 = weight_pecentage(30.0, 70.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 650 | " W_AB_460 , W_A_B_460 = weight_pecentage(40.0, 60.0, 0.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 651 | " W_AB_127 , W_A_B_127 = weight_pecentage(10.0, 20.0, 70.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 652 | " W_AB_235 , W_A_B_235 = weight_pecentage(20.0, 30.0, 50.0,ask_quantity_1,ask_quantity_2,ask_quantity_3, bid_quantity_1,bid_quantity_2,bid_quantity_3)\n", 653 | " \n", 654 | " # 上午：只取09:00 ~ 11:30作为交易区间，对应：\n", 655 | " time1 = 0\n", 656 | " time2 = 9000\n", 657 | "\n", 658 | " # 分别计算交易日上午的输入feature\n", 659 | " data_2014_UP =\\\n", 660 | " get_feature(time1, time2, traded_time,time_second_basic,bid_price_1,ask_price_1, bid_quantity_1, ask_quantity_1, rise_ratio_ask_1,\\\n", 661 | " rise_ratio_ask_2,rise_ratio_ask_3,rise_ratio_ask_4,rise_ratio_ask_5,\\\n", 662 | " rise_ratio_ask_6,rise_ratio_ask_7,rise_ratio_ask_8,rise_ratio_ask_9,\\\n", 663 | " rise_ratio_ask_10,rise_ratio_ask_11,rise_ratio_ask_12,rise_ratio_ask_13,\\\n", 664 | " rise_ratio_ask_14,rise_ratio_ask_15,rise_ratio_ask_16,rise_ratio_ask_17,\\\n", 665 | " rise_ratio_ask_18,rise_ratio_ask_19,rise_ratio_ask_20,rise_ratio_ask_21,\\\n", 666 | " rise_ratio_ask_22,rise_ratio_ask_23,rise_ratio_ask_24,rise_ratio_ask_25,\\\n", 667 | " rise_ratio_ask_26,rise_ratio_ask_27,rise_ratio_ask_28,rise_ratio_ask_29,\\\n", 668 | " rise_ratio_ask_30,W_AB_100, W_A_B_100, W_AB_010, W_A_B_010, W_AB_001,\\\n", 669 | " W_A_B_001, W_AB_910, W_A_B_910, W_AB_820, W_A_B_820, W_AB_730 , W_A_B_730,\\\n", 670 | " W_AB_640, W_A_B_640, W_AB_550, W_A_B_550,W_AB_721, W_A_B_721, W_AB_532,\\\n", 671 | " W_A_B_532, W_AB_111, W_A_B_111, W_AB_190, W_A_B_190, W_AB_280 , W_A_B_280,\\\n", 672 | " W_AB_370, W_A_B_370, W_AB_460, W_A_B_460, W_AB_127, W_A_B_127, W_AB_235, W_A_B_235)\n", 673 | "\n", 674 | " # 下午：只取13:00 ~ 16:00作为交易区间，对应：\n", 675 | " time1 = 14400\n", 676 | " time2 = 25200\n", 677 | "\n", 678 | " # 计算交易日下午的输入feature\n", 679 | " data_2014_DOWN =\\\n", 680 | " get_feature(time1, time2, traded_time,time_second_basic,bid_price_1,ask_price_1, bid_quantity_1, ask_quantity_1, rise_ratio_ask_1,\\\n", 681 | " rise_ratio_ask_2,rise_ratio_ask_3,rise_ratio_ask_4,rise_ratio_ask_5,\\\n", 682 | " rise_ratio_ask_6,rise_ratio_ask_7,rise_ratio_ask_8,rise_ratio_ask_9,\\\n", 683 | " rise_ratio_ask_10,rise_ratio_ask_11,rise_ratio_ask_12,rise_ratio_ask_13,\\\n", 684 | " rise_ratio_ask_14,rise_ratio_ask_15,rise_ratio_ask_16,rise_ratio_ask_17,\\\n", 685 | " rise_ratio_ask_18,rise_ratio_ask_19,rise_ratio_ask_20,rise_ratio_ask_21,\\\n", 686 | " rise_ratio_ask_22,rise_ratio_ask_23,rise_ratio_ask_24,rise_ratio_ask_25,\\\n", 687 | " rise_ratio_ask_26,rise_ratio_ask_27,rise_ratio_ask_28,rise_ratio_ask_29,\\\n", 688 | " rise_ratio_ask_30,W_AB_100, W_A_B_100, W_AB_010, W_A_B_010, W_AB_001,\\\n", 689 | " W_A_B_001, W_AB_910, W_A_B_910, W_AB_820, W_A_B_820, W_AB_730 , W_A_B_730,\\\n", 690 | " W_AB_640, W_A_B_640, W_AB_550, W_A_B_550,W_AB_721, W_A_B_721, W_AB_532,\\\n", 691 | " W_A_B_532, W_AB_111, W_A_B_111, W_AB_190, W_A_B_190, W_AB_280 , W_A_B_280,\\\n", 692 | " W_AB_370, W_A_B_370, W_AB_460, W_A_B_460, W_AB_127, W_A_B_127, W_AB_235, W_A_B_235)\n", 693 | "\n", 694 | " return data_2014_UP,data_2014_DOWN, len(W_AB_111) #trade_1, trade_2, timestamp" 695 | ] 696 | }, 697 | { 698 | "cell_type": "code", 699 | "execution_count": 28, 700 | "metadata": {}, 701 | "outputs": [ 702 | { 703 | "name": "stdout", 704 | "output_type": "stream", 705 | "text": [ 706 | "2\n" 707 | ] 708 | } 709 | ], 710 | "source": [ 711 | "# 将上午和下午计算的因子分别存档\n", 712 | "# traded_time为产生人工label的参数，一般取15min，对应300s，详见HFT_factor.ipynb\n", 713 | "\n", 714 | "def train_test_to_csv(month,day,traded_time):\n", 715 | " data_UP,data_DOWN, len_ = data(month,day,traded_time)\n", 716 | " data_UP.to_csv('./processed_data/order_book_3_2014_new'+'_'+str(month)+'_'+str(day)+'_'+'UP'+'.csv', index = False)\n", 717 | " data_DOWN.to_csv('./processed_data/order_book_3_2014_new'+'_'+str(month)+'_'+str(day)+'_'+'DOWN'+'.csv', index = False) \n", 718 | "\n", 719 | "if __name__ == \"__main__\":\n", 720 | " # 以一天为例，有多天可以循环产生文件，后续model fit部分可以append起来\n", 721 | " month = 1\n", 722 | " day_ = [2]\n", 723 | " traded_time = 600\n", 724 | " for i in day_:\n", 725 | " print(i)\n", 726 | " train_test_to_csv(month,i,600) " 727 | ] 728 | }, 729 | { 730 | "cell_type": "code", 731 | "execution_count": null, 732 | "metadata": {}, 733 | "outputs": [], 734 | "source": [] 735 | } 736 | ], 737 | "metadata": { 738 | "interpreter": { 739 | "hash": "ed225720166559e7176d3793db16a2fd8d295f725007103b21ac3099d2a89ee8" 740 | }, 741 | "kernelspec": { 742 | "display_name": "Python 3.9.7 ('base')", 743 | "language": "python", 744 | "name": "python3" 745 | }, 746 | "language_info": { 747 | "codemirror_mode": { 748 | "name": "ipython", 749 | "version": 3 750 | }, 751 | "file_extension": ".py", 752 | "mimetype": "text/x-python", 753 | "name": "python", 754 | "nbconvert_exporter": "python", 755 | "pygments_lexer": "ipython3", 756 | "version": "3.9.7" 757 | }, 758 | "orig_nbformat": 4 759 | }, 760 | "nbformat": 4, 761 | "nbformat_minor": 2 762 | } 763 | -------------------------------------------------------------------------------- /images/CV_result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/CV_result.png -------------------------------------------------------------------------------- /images/best_CV_result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/best_CV_result.png -------------------------------------------------------------------------------- /images/best_CV_result_all.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/best_CV_result_all.png -------------------------------------------------------------------------------- /images/best_bid_ask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/best_bid_ask.png -------------------------------------------------------------------------------- /images/bid1ask1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/bid1ask1.png -------------------------------------------------------------------------------- /images/depth_0000_0900_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_0000_0900_w.png -------------------------------------------------------------------------------- /images/depth_0000_0915.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_0000_0915.png -------------------------------------------------------------------------------- /images/depth_0900_1130_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_0900_1130_w.png -------------------------------------------------------------------------------- /images/depth_0915_1130.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_0915_1130.png -------------------------------------------------------------------------------- /images/depth_1130_1300.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_1130_1300.png -------------------------------------------------------------------------------- /images/depth_1130_1300_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_1130_1300_w.png -------------------------------------------------------------------------------- /images/depth_1300_1600.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_1300_1600.png -------------------------------------------------------------------------------- /images/depth_1300_1600_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/depth_1300_1600_w.png -------------------------------------------------------------------------------- /images/prediction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/prediction.png -------------------------------------------------------------------------------- /images/rise_0000_0915_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/rise_0000_0915_w.png -------------------------------------------------------------------------------- /images/rise_0915_1130_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/rise_0915_1130_w.png -------------------------------------------------------------------------------- /images/rise_1130_1300_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/rise_1130_1300_w.png -------------------------------------------------------------------------------- /images/rise_1300_1600_w.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/rise_1300_1600_w.png -------------------------------------------------------------------------------- /images/single_day_accuracy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bradleyboyuyang/ML-HFT/5bcd376d61a3a21b70f2937ed771d01e07b90123/images/single_day_accuracy.png -------------------------------------------------------------------------------- /order_book_3_2014_1_2.csv: -------------------------------------------------------------------------------- 1 | Bid,Bid_Quantity,Ask,Ask_Quantity 2 | TimeStamp,2014-01-02D04:19:51.857166800 3 | 700500.0,2.0,715500.0,2.0 4 | 699000.0,2.0,717500.0,1.0 5 | 697500.0,1.0,719500.0,1.0 6 | TimeStamp,2014-01-02D08:45:00.131582600 7 | 682000.0,111.0,0.0,210.0 8 | nan,nan,715500.0,2.0 9 | nan,nan,717500.0,1.0 10 | TimeStamp,2014-01-02D08:45:00.296582600 11 | 708000.0,210.0,682000.0,210.0 12 | 700500.0,2.0,715500.0,2.0 13 | 699000.0,2.0,717500.0,1.0 14 | TimeStamp,2014-01-02D08:45:00.398582300 15 | 708000.0,210.0,708000.0,210.0 16 | 700500.0,2.0,715500.0,2.0 17 | 699000.0,2.0,717500.0,1.0 18 | TimeStamp,2014-01-02D08:45:13.857414600 19 | 708000.0,210.0,708000.0,210.0 20 | 700500.0,2.0,715500.0,2.0 21 | 699000.0,2.0,717500.0,1.0 22 | TimeStamp,2014-01-02D08:45:14.977402600 23 | 708000.0,210.0,708000.0,210.0 24 | 700500.0,2.0,715500.0,2.0 25 | 699000.0,2.0,717500.0,1.0 26 | TimeStamp,2014-01-02D08:45:18.317366600 27 | 708000.0,210.0,708000.0,210.0 28 | 700500.0,2.0,715500.0,2.0 29 | 699000.0,2.0,717500.0,1.0 30 | TimeStamp,2014-01-02D08:45:30.059222600 31 | 708000.0,210.0,708000.0,210.0 32 | 700500.0,2.0,715500.0,2.0 33 | 699000.0,2.0,717500.0,1.0 34 | TimeStamp,2014-01-02D08:45:36.809138600 35 | 708000.0,210.0,708000.0,210.0 36 | 700500.0,2.0,715500.0,2.0 37 | 699000.0,2.0,717500.0,1.0 38 | TimeStamp,2014-01-02D08:46:00.014862600 39 | 708000.0,210.0,708000.0,210.0 40 | 700500.0,2.0,715500.0,2.0 41 | 699000.0,2.0,717500.0,1.0 42 | TimeStamp,2014-01-02D08:46:00.018862600 43 | 706500.0,210.0,706500.0,210.0 44 | 700500.0,2.0,712000.0,2.0 45 | 699000.0,2.0,715500.0,2.0 46 | TimeStamp,2014-01-02D08:46:00.021862600 47 | 706500.0,210.0,706500.0,210.0 48 | 700500.0,2.0,712000.0,2.0 49 | 699000.0,2.0,715500.0,2.0 50 | TimeStamp,2014-01-02D08:46:00.022862600 51 | 706500.0,210.0,706500.0,210.0 52 | 700500.0,2.0,712000.0,2.0 53 | 699000.0,2.0,715500.0,2.0 54 | TimeStamp,2014-01-02D08:46:00.024862600 55 | 706500.0,210.0,706500.0,210.0 56 | 700500.0,2.0,712000.0,2.0 57 | 699000.0,2.0,715500.0,2.0 58 | TimeStamp,2014-01-02D08:46:00.025862600 59 | 706500.0,210.0,706500.0,210.0 60 | 700500.0,2.0,712000.0,2.0 61 | 699000.0,2.0,715500.0,2.0 62 | TimeStamp,2014-01-02D08:46:00.033862600 63 | 707000.0,210.0,706500.0,210.0 64 | 702000.0,1.0,712000.0,2.0 65 | 700500.0,2.0,715500.0,2.0 66 | TimeStamp,2014-01-02D08:46:00.033862600 67 | 707000.0,210.0,707000.0,210.0 68 | 702000.0,1.0,712000.0,2.0 69 | 700500.0,2.0,715500.0,2.0 70 | TimeStamp,2014-01-02D08:46:00.036862600 71 | 707000.0,210.0,707000.0,210.0 72 | 702000.0,1.0,712000.0,2.0 73 | 700500.0,2.0,715500.0,2.0 74 | TimeStamp,2014-01-02D08:46:00.037862600 75 | 707000.0,210.0,707000.0,210.0 76 | 702000.0,1.0,712000.0,2.0 77 | 700500.0,2.0,715500.0,2.0 78 | TimeStamp,2014-01-02D08:46:00.043862600 79 | 709500.0,210.0,707000.0,210.0 80 | 707000.0,1.0,712000.0,2.0 81 | 702000.0,1.0,715500.0,2.0 82 | TimeStamp,2014-01-02D08:46:00.043862600 83 | 709500.0,210.0,709500.0,210.0 84 | 707000.0,1.0,712000.0,2.0 85 | 702000.0,1.0,715500.0,2.0 86 | TimeStamp,2014-01-02D08:46:00.045862600 87 | 709500.0,210.0,709500.0,210.0 88 | 707000.0,1.0,712000.0,2.0 89 | 702000.0,1.0,715500.0,2.0 90 | TimeStamp,2014-01-02D08:46:00.048862600 91 | 709500.0,210.0,709500.0,210.0 92 | 707000.0,1.0,712000.0,4.0 93 | 702000.0,1.0,715500.0,2.0 94 | TimeStamp,2014-01-02D08:46:00.052862600 95 | 709500.0,210.0,709500.0,210.0 96 | 707000.0,1.0,712000.0,4.0 97 | 702000.0,1.0,715500.0,2.0 98 | TimeStamp,2014-01-02D08:46:00.056862600 99 | 709500.0,210.0,709500.0,210.0 100 | 707000.0,1.0,712000.0,4.0 101 | 702000.0,1.0,715500.0,2.0 102 | TimeStamp,2014-01-02D08:46:00.061862600 103 | 709500.0,210.0,709500.0,210.0 104 | 707000.0,1.0,712000.0,4.0 105 | 702000.0,1.0,715500.0,2.0 106 | -------------------------------------------------------------------------------- /order_book_4_2014_1_2.csv: -------------------------------------------------------------------------------- 1 | Bid,Bid_Quantity,Ask,Ask_Quantity 2 | TimeStamp,2014-01-02D04:19:51.857166800 3 | 700500.0,2.0,715500.0,2.0 4 | 699000.0,2.0,717500.0,1.0 5 | 697500.0,1.0,719500.0,1.0 6 | 690500.0,1.0,740000.0,2.0 7 | TimeStamp,2014-01-02D08:45:00.131582600 8 | 682000.0,111.0,0.0,210.0 9 | nan,nan,715500.0,2.0 10 | nan,nan,717500.0,1.0 11 | nan,nan,719500.0,1.0 12 | TimeStamp,2014-01-02D08:45:00.296582600 13 | 708000.0,210.0,682000.0,210.0 14 | 700500.0,2.0,715500.0,2.0 15 | 699000.0,2.0,717500.0,1.0 16 | 697500.0,1.0,719500.0,1.0 17 | TimeStamp,2014-01-02D08:45:00.398582300 18 | 708000.0,210.0,708000.0,210.0 19 | 700500.0,2.0,715500.0,2.0 20 | 699000.0,2.0,717500.0,1.0 21 | 697500.0,1.0,719500.0,1.0 22 | TimeStamp,2014-01-02D08:45:13.857414600 23 | 708000.0,210.0,708000.0,210.0 24 | 700500.0,2.0,715500.0,2.0 25 | 699000.0,2.0,717500.0,1.0 26 | 697500.0,1.0,719500.0,1.0 27 | TimeStamp,2014-01-02D08:45:14.977402600 28 | 708000.0,210.0,708000.0,210.0 29 | 700500.0,2.0,715500.0,2.0 30 | 699000.0,2.0,717500.0,1.0 31 | 697500.0,1.0,719500.0,1.0 32 | TimeStamp,2014-01-02D08:45:18.317366600 33 | 708000.0,210.0,708000.0,210.0 34 | 700500.0,2.0,715500.0,2.0 35 | 699000.0,2.0,717500.0,1.0 36 | 697500.0,1.0,719500.0,1.0 37 | TimeStamp,2014-01-02D08:45:30.059222600 38 | 708000.0,210.0,708000.0,210.0 39 | 700500.0,2.0,715500.0,2.0 40 | 699000.0,2.0,717500.0,1.0 41 | 697500.0,1.0,719500.0,1.0 42 | TimeStamp,2014-01-02D08:45:36.809138600 43 | 708000.0,210.0,708000.0,210.0 44 | 700500.0,2.0,715500.0,2.0 45 | 699000.0,2.0,717500.0,1.0 46 | 697500.0,1.0,719500.0,1.0 47 | TimeStamp,2014-01-02D08:46:00.014862600 48 | 708000.0,210.0,708000.0,210.0 49 | 700500.0,2.0,715500.0,2.0 50 | 699000.0,2.0,717500.0,1.0 51 | 697500.0,1.0,719500.0,1.0 52 | TimeStamp,2014-01-02D08:46:00.018862600 53 | 706500.0,210.0,706500.0,210.0 54 | 700500.0,2.0,712000.0,2.0 55 | 699000.0,2.0,715500.0,2.0 56 | 697500.0,1.0,717500.0,1.0 57 | TimeStamp,2014-01-02D08:46:00.021862600 58 | 706500.0,210.0,706500.0,210.0 59 | 700500.0,2.0,712000.0,2.0 60 | 699000.0,2.0,715500.0,2.0 61 | 697500.0,1.0,717500.0,1.0 62 | TimeStamp,2014-01-02D08:46:00.022862600 63 | 706500.0,210.0,706500.0,210.0 64 | 700500.0,2.0,712000.0,2.0 65 | 699000.0,2.0,715500.0,2.0 66 | 697500.0,1.0,717500.0,1.0 67 | TimeStamp,2014-01-02D08:46:00.024862600 68 | 706500.0,210.0,706500.0,210.0 69 | 700500.0,2.0,712000.0,2.0 70 | 699000.0,2.0,715500.0,2.0 71 | 697500.0,1.0,717500.0,1.0 72 | TimeStamp,2014-01-02D08:46:00.025862600 73 | 706500.0,210.0,706500.0,210.0 74 | 700500.0,2.0,712000.0,2.0 75 | 699000.0,2.0,715500.0,2.0 76 | 697500.0,1.0,717500.0,1.0 77 | TimeStamp,2014-01-02D08:46:00.033862600 78 | 707000.0,210.0,706500.0,210.0 79 | 702000.0,1.0,712000.0,2.0 80 | 700500.0,2.0,715500.0,2.0 81 | 699000.0,2.0,717500.0,1.0 82 | TimeStamp,2014-01-02D08:46:00.033862600 83 | 707000.0,210.0,707000.0,210.0 84 | 702000.0,1.0,712000.0,2.0 85 | 700500.0,2.0,715500.0,2.0 86 | 699000.0,2.0,717500.0,1.0 87 | TimeStamp,2014-01-02D08:46:00.036862600 88 | 707000.0,210.0,707000.0,210.0 89 | 702000.0,1.0,712000.0,2.0 90 | 700500.0,2.0,715500.0,2.0 91 | 699000.0,2.0,717500.0,1.0 92 | TimeStamp,2014-01-02D08:46:00.037862600 93 | 707000.0,210.0,707000.0,210.0 94 | 702000.0,1.0,712000.0,2.0 95 | 700500.0,2.0,715500.0,2.0 96 | 699000.0,2.0,717500.0,1.0 97 | TimeStamp,2014-01-02D08:46:00.043862600 98 | 709500.0,210.0,707000.0,210.0 99 | 707000.0,1.0,712000.0,2.0 100 | 702000.0,1.0,715500.0,2.0 101 | 701000.0,1.0,717500.0,1.0 102 | TimeStamp,2014-01-02D08:46:00.043862600 103 | 709500.0,210.0,709500.0,210.0 104 | 707000.0,1.0,712000.0,2.0 105 | 702000.0,1.0,715500.0,2.0 106 | 701000.0,1.0,717500.0,1.0 107 | TimeStamp,2014-01-02D08:46:00.045862600 108 | 709500.0,210.0,709500.0,210.0 109 | 707000.0,1.0,712000.0,2.0 110 | 702000.0,1.0,715500.0,2.0 111 | 701000.0,1.0,717500.0,1.0 112 | TimeStamp,2014-01-02D08:46:00.048862600 113 | 709500.0,210.0,709500.0,210.0 114 | 707000.0,1.0,712000.0,4.0 115 | 702000.0,1.0,715500.0,2.0 116 | 701000.0,1.0,717500.0,1.0 117 | TimeStamp,2014-01-02D08:46:00.052862600 118 | 709500.0,210.0,709500.0,210.0 119 | 707000.0,1.0,712000.0,4.0 120 | 702000.0,1.0,715500.0,2.0 121 | 701000.0,1.0,717500.0,1.0 122 | TimeStamp,2014-01-02D08:46:00.056862600 123 | 709500.0,210.0,709500.0,210.0 124 | 707000.0,1.0,712000.0,4.0 125 | 702000.0,1.0,715500.0,2.0 126 | 701000.0,1.0,717500.0,1.0 127 | TimeStamp,2014-01-02D08:46:00.061862600 128 | 709500.0,210.0,709500.0,210.0 129 | 707000.0,1.0,712000.0,4.0 130 | 702000.0,1.0,715500.0,2.0 131 | 701000.0,1.0,717500.0,1.0 132 | --------------------------------------------------------------------------------