├── LICENSE ├── README.md ├── abilene ├── .gitignore ├── A ├── downloadlink.txt ├── links ├── readme.txt └── topo-2003-04-10.txt ├── code └── full-pipeline │ ├── .gitignore │ ├── 001_generateSFCRTrace.R │ ├── 002_sfcrTrace2CPLEX.py │ ├── 003_runCPLEXPar.py │ ├── 004_getPlacementInfoFromCPLEXLogs.py │ ├── 005_processSolverResults.R │ ├── 006a_generateTrainingData.R │ ├── 006b_mergeTrainingData.R │ ├── 007_trainAutoMLModel.R │ ├── abilene-preproc.R │ ├── abilene-week1.csv │ ├── common.R │ ├── data │ └── .gitignore │ ├── feature-description.txt │ ├── inet2 │ ├── mec │ ├── middlebox-spec │ └── runwk_template.sh └── solver ├── patch_e4bd6e8_to_ni-version.patch └── readme.txt /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2019 Stanislav Lange 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all 11 | copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 19 | SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Predicting VNF Deployment Decisions under Dynamically Changing Network Conditions 2 | 3 | ## General Idea 4 | 5 | Use existing placement algorithms to generate (problem, solution) pairs for networks with dynamically arriving SFC requests. Then, learn from the solutions in order to make decisions regarding the deployment / shutdown of VNF instances. 6 | 7 | ## Methodology 8 | 9 | 1. Generation of SFC requests that are characterized by their arrival time, duration, source and destination nodes, bandwidth constraint, and requested VNF chain. 10 | 2. Generation of instances of the VNF placement problem that represent the system state at each request arrival event. 11 | 3. Calculation of optimal placements for the above mentioned problem instances. 12 | 4. Extraction of optimal placement information from solver log files. 13 | 5. Consolidation of solver results. 14 | 6. Generation of training data for deployment decisions by using solver results to extract features and deployment decisions. A detailed overview of extracted features is provided in `code/full-pipeline/feature-description.txt`. 15 | 7. Training of supervised machine learning models using the H2O framework [7]. 16 | 17 | ## Components / Structure 18 | 19 | * Code 20 | * Python-based scripts for parts of SFCR generation and invocation of placement algorithm. 21 | * R-based scripts for statistical evaluation, parts of SFCR generation, training data generation, and model training. 22 | * Data 23 | * Traffic matrices from a dataset used in [2]. For extensions, there's also a dataset from [3] which is available at [4], as well as dynamic instances available at [5]. 24 | * Solver 25 | * ILP-based placement algorithm using CPLEX-based implementation from [1]. 26 | 27 | ## Dependencies and Usage Instructions 28 | 29 | ### Traffix Matrix Data 30 | 31 | * Download traffic matrix data (file names X01 through X24) from [2] into the `abilene` folder. 32 | 33 | ### ILP Solver 34 | 35 | * Prepare dependencies for CPLEX-based ILP solver: Java, CPLEX (we used v12.8), environment variables such as `JAVA_HOME` and `CPLEX_ROOT`. 36 | * Clone repository [6] into `solver/middlebox-placement`. 37 | * Apply patch file `patch_e4bd6e8_to_ni-version.patch` according to instructions in `solver/readme.txt`. 38 | * Compile by running `make dbg` in `solver/middlebox-placement/src`. 39 | * Create log file folder `solver/middlebox-placement/src/logs` or change the corresponding parameter (`LOGFILEPATH`) that is passed in steps 2-4. 40 | 41 | ### R / H2O 42 | 43 | * Install the H2O ML framework / R package [8]. 44 | * Install required packages: data.table, dplyr, forcats, ggplot2, optparse, purrr, RColorBrewer, RcppRoll, readr, tidyr (some are part of the tidyverse package and can therefore be skipped if tidyverse is already installed). 45 | 46 | ### Misc. 47 | * GNU parallel [9]. 48 | 49 | ### Usage 50 | * `code/full-pipeline/runwk_template.sh`: exemplary bash script that covers steps 1 to 6, i.e., from generation of requests to extraction of training data. 51 | * `code/full-pipeline/007_trainAutoMLModel.R`: example for training an H2O AutoML model. 52 | 53 | ### Publication 54 | 55 | Lange, S., Kim, H.-G. et al. "*Predicting VNF Deployment Decisions under Dynamically Changing Network Conditions.*" CNSM 2019. 56 | 57 | BibTeX entry 58 | 59 | ``` 60 | @inproceedings{lange2019predicting, 61 | title={{Predicting VNF Deployment Decisions under Dynamically Changing Network Conditions}}, 62 | author={Lange, Stanislav and Kim, Hee-Gon and Jeong, Se-Yeon and Choi, Heeyoul and Yoo, Jae-Hyung and Hong, James Won-Ki}, 63 | booktitle={International Conference on Network and Service Management (CNSM)}, 64 | year={2019} 65 | } 66 | ``` 67 | 68 | ### References 69 | 70 | [1] Bari, M. F. et al. "*On orchestrating virtual network functions.*" CNSM 2015. 71 | 72 | [2] http://www.cs.utexas.edu/~yzhang/research/AbileneTM/ 73 | 74 | [3] Benson, T. et al. "*Network traffic characteristics of data centers in the wild.*" SIGCOMM 2010. 75 | 76 | [4] http://pages.cs.wisc.edu/~tbenson/IMC10_Data.html 77 | 78 | [5] http://sndlib.zib.de/dynamicmatrices.overview.action 79 | 80 | [6] https://github.com/srcvirus/middlebox-placement/ 81 | 82 | [7] https://www.h2o.ai/ 83 | 84 | [8] http://h2o-release.s3.amazonaws.com/h2o/latest_stable.html 85 | 86 | [9] Tange, O. "*GNU Parallel - The Command-Line Power Tool*" ;login: The USENIX Magazine 2011. 87 | -------------------------------------------------------------------------------- /abilene/.gitignore: -------------------------------------------------------------------------------- 1 | X* 2 | -------------------------------------------------------------------------------- /abilene/A: -------------------------------------------------------------------------------- 1 | # routing matrix 2 | # link(x,y) dmd(s,d) link_index dmd_index frac 3 | *,ATLA-M5 ATLA-M5,ATLA-M5 31 1 1 4 | *,ATLA-M5 ATLA-M5,ATLAng 31 2 1 5 | *,ATLA-M5 ATLA-M5,CHINng 31 3 1 6 | *,ATLA-M5 ATLA-M5,DNVRng 31 4 1 7 | *,ATLA-M5 ATLA-M5,HSTNng 31 5 1 8 | *,ATLA-M5 ATLA-M5,IPLSng 31 6 1 9 | *,ATLA-M5 ATLA-M5,KSCYng 31 7 1 10 | *,ATLA-M5 ATLA-M5,LOSAng 31 8 1 11 | *,ATLA-M5 ATLA-M5,NYCMng 31 9 1 12 | *,ATLA-M5 ATLA-M5,SNVAng 31 10 1 13 | *,ATLA-M5 ATLA-M5,STTLng 31 11 1 14 | *,ATLA-M5 ATLA-M5,WASHng 31 12 1 15 | *,ATLAng ATLAng,ATLA-M5 33 13 1 16 | *,ATLAng ATLAng,ATLAng 33 14 1 17 | *,ATLAng ATLAng,CHINng 33 15 1 18 | *,ATLAng ATLAng,DNVRng 33 16 1 19 | *,ATLAng ATLAng,HSTNng 33 17 1 20 | *,ATLAng ATLAng,IPLSng 33 18 1 21 | *,ATLAng ATLAng,KSCYng 33 19 1 22 | *,ATLAng ATLAng,LOSAng 33 20 1 23 | *,ATLAng ATLAng,NYCMng 33 21 1 24 | *,ATLAng ATLAng,SNVAng 33 22 1 25 | *,ATLAng ATLAng,STTLng 33 23 1 26 | *,ATLAng ATLAng,WASHng 33 24 1 27 | *,CHINng CHINng,ATLA-M5 35 25 1 28 | *,CHINng CHINng,ATLAng 35 26 1 29 | *,CHINng CHINng,CHINng 35 27 1 30 | *,CHINng CHINng,DNVRng 35 28 1 31 | *,CHINng CHINng,HSTNng 35 29 1 32 | *,CHINng CHINng,IPLSng 35 30 1 33 | *,CHINng CHINng,KSCYng 35 31 1 34 | *,CHINng CHINng,LOSAng 35 32 1 35 | *,CHINng CHINng,NYCMng 35 33 1 36 | *,CHINng CHINng,SNVAng 35 34 1 37 | *,CHINng CHINng,STTLng 35 35 1 38 | *,CHINng CHINng,WASHng 35 36 1 39 | *,DNVRng DNVRng,ATLA-M5 37 37 1 40 | *,DNVRng DNVRng,ATLAng 37 38 1 41 | *,DNVRng DNVRng,CHINng 37 39 1 42 | *,DNVRng DNVRng,DNVRng 37 40 1 43 | *,DNVRng DNVRng,HSTNng 37 41 1 44 | *,DNVRng DNVRng,IPLSng 37 42 1 45 | *,DNVRng DNVRng,KSCYng 37 43 1 46 | *,DNVRng DNVRng,LOSAng 37 44 1 47 | *,DNVRng DNVRng,NYCMng 37 45 1 48 | *,DNVRng DNVRng,SNVAng 37 46 1 49 | *,DNVRng DNVRng,STTLng 37 47 1 50 | *,DNVRng DNVRng,WASHng 37 48 1 51 | *,HSTNng HSTNng,ATLA-M5 39 49 1 52 | *,HSTNng HSTNng,ATLAng 39 50 1 53 | *,HSTNng HSTNng,CHINng 39 51 1 54 | *,HSTNng HSTNng,DNVRng 39 52 1 55 | *,HSTNng HSTNng,HSTNng 39 53 1 56 | *,HSTNng HSTNng,IPLSng 39 54 1 57 | *,HSTNng HSTNng,KSCYng 39 55 1 58 | *,HSTNng HSTNng,LOSAng 39 56 1 59 | *,HSTNng HSTNng,NYCMng 39 57 1 60 | *,HSTNng HSTNng,SNVAng 39 58 1 61 | *,HSTNng HSTNng,STTLng 39 59 1 62 | *,HSTNng HSTNng,WASHng 39 60 1 63 | *,IPLSng IPLSng,ATLA-M5 41 61 1 64 | *,IPLSng IPLSng,ATLAng 41 62 1 65 | *,IPLSng IPLSng,CHINng 41 63 1 66 | *,IPLSng IPLSng,DNVRng 41 64 1 67 | *,IPLSng IPLSng,HSTNng 41 65 1 68 | *,IPLSng IPLSng,IPLSng 41 66 1 69 | *,IPLSng IPLSng,KSCYng 41 67 1 70 | *,IPLSng IPLSng,LOSAng 41 68 1 71 | *,IPLSng IPLSng,NYCMng 41 69 1 72 | *,IPLSng IPLSng,SNVAng 41 70 1 73 | *,IPLSng IPLSng,STTLng 41 71 1 74 | *,IPLSng IPLSng,WASHng 41 72 1 75 | *,KSCYng KSCYng,ATLA-M5 43 73 1 76 | *,KSCYng KSCYng,ATLAng 43 74 1 77 | *,KSCYng KSCYng,CHINng 43 75 1 78 | *,KSCYng KSCYng,DNVRng 43 76 1 79 | *,KSCYng KSCYng,HSTNng 43 77 1 80 | *,KSCYng KSCYng,IPLSng 43 78 1 81 | *,KSCYng KSCYng,KSCYng 43 79 1 82 | *,KSCYng KSCYng,LOSAng 43 80 1 83 | *,KSCYng KSCYng,NYCMng 43 81 1 84 | *,KSCYng KSCYng,SNVAng 43 82 1 85 | *,KSCYng KSCYng,STTLng 43 83 1 86 | *,KSCYng KSCYng,WASHng 43 84 1 87 | *,LOSAng LOSAng,ATLA-M5 45 85 1 88 | *,LOSAng LOSAng,ATLAng 45 86 1 89 | *,LOSAng LOSAng,CHINng 45 87 1 90 | *,LOSAng LOSAng,DNVRng 45 88 1 91 | *,LOSAng LOSAng,HSTNng 45 89 1 92 | *,LOSAng LOSAng,IPLSng 45 90 1 93 | *,LOSAng LOSAng,KSCYng 45 91 1 94 | *,LOSAng LOSAng,LOSAng 45 92 1 95 | *,LOSAng LOSAng,NYCMng 45 93 1 96 | *,LOSAng LOSAng,SNVAng 45 94 1 97 | *,LOSAng LOSAng,STTLng 45 95 1 98 | *,LOSAng LOSAng,WASHng 45 96 1 99 | *,NYCMng NYCMng,ATLA-M5 47 97 1 100 | *,NYCMng NYCMng,ATLAng 47 98 1 101 | *,NYCMng NYCMng,CHINng 47 99 1 102 | *,NYCMng NYCMng,DNVRng 47 100 1 103 | *,NYCMng NYCMng,HSTNng 47 101 1 104 | *,NYCMng NYCMng,IPLSng 47 102 1 105 | *,NYCMng NYCMng,KSCYng 47 103 1 106 | *,NYCMng NYCMng,LOSAng 47 104 1 107 | *,NYCMng NYCMng,NYCMng 47 105 1 108 | *,NYCMng NYCMng,SNVAng 47 106 1 109 | *,NYCMng NYCMng,STTLng 47 107 1 110 | *,NYCMng NYCMng,WASHng 47 108 1 111 | *,SNVAng SNVAng,ATLA-M5 49 109 1 112 | *,SNVAng SNVAng,ATLAng 49 110 1 113 | *,SNVAng SNVAng,CHINng 49 111 1 114 | *,SNVAng SNVAng,DNVRng 49 112 1 115 | *,SNVAng SNVAng,HSTNng 49 113 1 116 | *,SNVAng SNVAng,IPLSng 49 114 1 117 | *,SNVAng SNVAng,KSCYng 49 115 1 118 | *,SNVAng SNVAng,LOSAng 49 116 1 119 | *,SNVAng SNVAng,NYCMng 49 117 1 120 | *,SNVAng SNVAng,SNVAng 49 118 1 121 | *,SNVAng SNVAng,STTLng 49 119 1 122 | *,SNVAng SNVAng,WASHng 49 120 1 123 | *,STTLng STTLng,ATLA-M5 51 121 1 124 | *,STTLng STTLng,ATLAng 51 122 1 125 | *,STTLng STTLng,CHINng 51 123 1 126 | *,STTLng STTLng,DNVRng 51 124 1 127 | *,STTLng STTLng,HSTNng 51 125 1 128 | *,STTLng STTLng,IPLSng 51 126 1 129 | *,STTLng STTLng,KSCYng 51 127 1 130 | *,STTLng STTLng,LOSAng 51 128 1 131 | *,STTLng STTLng,NYCMng 51 129 1 132 | *,STTLng STTLng,SNVAng 51 130 1 133 | *,STTLng STTLng,STTLng 51 131 1 134 | *,STTLng STTLng,WASHng 51 132 1 135 | *,WASHng WASHng,ATLA-M5 53 133 1 136 | *,WASHng WASHng,ATLAng 53 134 1 137 | *,WASHng WASHng,CHINng 53 135 1 138 | *,WASHng WASHng,DNVRng 53 136 1 139 | *,WASHng WASHng,HSTNng 53 137 1 140 | *,WASHng WASHng,IPLSng 53 138 1 141 | *,WASHng WASHng,KSCYng 53 139 1 142 | *,WASHng WASHng,LOSAng 53 140 1 143 | *,WASHng WASHng,NYCMng 53 141 1 144 | *,WASHng WASHng,SNVAng 53 142 1 145 | *,WASHng WASHng,STTLng 53 143 1 146 | *,WASHng WASHng,WASHng 53 144 1 147 | ATLA-M5,* ATLA-M5,ATLA-M5 32 1 1 148 | ATLA-M5,* ATLAng,ATLA-M5 32 13 1 149 | ATLA-M5,* CHINng,ATLA-M5 32 25 1 150 | ATLA-M5,* DNVRng,ATLA-M5 32 37 1 151 | ATLA-M5,* HSTNng,ATLA-M5 32 49 1 152 | ATLA-M5,* IPLSng,ATLA-M5 32 61 1 153 | ATLA-M5,* KSCYng,ATLA-M5 32 73 1 154 | ATLA-M5,* LOSAng,ATLA-M5 32 85 1 155 | ATLA-M5,* NYCMng,ATLA-M5 32 97 1 156 | ATLA-M5,* SNVAng,ATLA-M5 32 109 1 157 | ATLA-M5,* STTLng,ATLA-M5 32 121 1 158 | ATLA-M5,* WASHng,ATLA-M5 32 133 1 159 | ATLA-M5,ATLAng ATLA-M5,ATLAng 1 2 1 160 | ATLA-M5,ATLAng ATLA-M5,CHINng 1 3 1 161 | ATLA-M5,ATLAng ATLA-M5,DNVRng 1 4 1 162 | ATLA-M5,ATLAng ATLA-M5,HSTNng 1 5 1 163 | ATLA-M5,ATLAng ATLA-M5,IPLSng 1 6 1 164 | ATLA-M5,ATLAng ATLA-M5,KSCYng 1 7 1 165 | ATLA-M5,ATLAng ATLA-M5,LOSAng 1 8 1 166 | ATLA-M5,ATLAng ATLA-M5,NYCMng 1 9 1 167 | ATLA-M5,ATLAng ATLA-M5,SNVAng 1 10 1 168 | ATLA-M5,ATLAng ATLA-M5,STTLng 1 11 1 169 | ATLA-M5,ATLAng ATLA-M5,WASHng 1 12 1 170 | ATLAng,* ATLA-M5,ATLAng 34 2 1 171 | ATLAng,* ATLAng,ATLAng 34 14 1 172 | ATLAng,* CHINng,ATLAng 34 26 1 173 | ATLAng,* DNVRng,ATLAng 34 38 1 174 | ATLAng,* HSTNng,ATLAng 34 50 1 175 | ATLAng,* IPLSng,ATLAng 34 62 1 176 | ATLAng,* KSCYng,ATLAng 34 74 1 177 | ATLAng,* LOSAng,ATLAng 34 86 1 178 | ATLAng,* NYCMng,ATLAng 34 98 1 179 | ATLAng,* SNVAng,ATLAng 34 110 1 180 | ATLAng,* STTLng,ATLAng 34 122 1 181 | ATLAng,* WASHng,ATLAng 34 134 1 182 | ATLAng,ATLA-M5 ATLAng,ATLA-M5 2 13 1 183 | ATLAng,ATLA-M5 CHINng,ATLA-M5 2 25 1 184 | ATLAng,ATLA-M5 DNVRng,ATLA-M5 2 37 1 185 | ATLAng,ATLA-M5 HSTNng,ATLA-M5 2 49 1 186 | ATLAng,ATLA-M5 IPLSng,ATLA-M5 2 61 1 187 | ATLAng,ATLA-M5 KSCYng,ATLA-M5 2 73 1 188 | ATLAng,ATLA-M5 LOSAng,ATLA-M5 2 85 1 189 | ATLAng,ATLA-M5 NYCMng,ATLA-M5 2 97 1 190 | ATLAng,ATLA-M5 SNVAng,ATLA-M5 2 109 1 191 | ATLAng,ATLA-M5 STTLng,ATLA-M5 2 121 1 192 | ATLAng,ATLA-M5 WASHng,ATLA-M5 2 133 1 193 | ATLAng,HSTNng ATLA-M5,HSTNng 3 5 1 194 | ATLAng,HSTNng ATLA-M5,LOSAng 3 8 1 195 | ATLAng,HSTNng ATLAng,HSTNng 3 17 1 196 | ATLAng,HSTNng ATLAng,LOSAng 3 20 1 197 | ATLAng,HSTNng NYCMng,HSTNng 3 101 1 198 | ATLAng,HSTNng WASHng,HSTNng 3 137 1 199 | ATLAng,HSTNng WASHng,LOSAng 3 140 1 200 | ATLAng,IPLSng ATLA-M5,CHINng 4 3 1 201 | ATLAng,IPLSng ATLA-M5,DNVRng 4 4 1 202 | ATLAng,IPLSng ATLA-M5,IPLSng 4 6 1 203 | ATLAng,IPLSng ATLA-M5,KSCYng 4 7 1 204 | ATLAng,IPLSng ATLA-M5,SNVAng 4 10 1 205 | ATLAng,IPLSng ATLA-M5,STTLng 4 11 1 206 | ATLAng,IPLSng ATLAng,CHINng 4 15 1 207 | ATLAng,IPLSng ATLAng,DNVRng 4 16 1 208 | ATLAng,IPLSng ATLAng,IPLSng 4 18 1 209 | ATLAng,IPLSng ATLAng,KSCYng 4 19 1 210 | ATLAng,IPLSng ATLAng,SNVAng 4 22 1 211 | ATLAng,IPLSng ATLAng,STTLng 4 23 1 212 | ATLAng,WASHng ATLA-M5,NYCMng 5 9 1 213 | ATLAng,WASHng ATLA-M5,WASHng 5 12 1 214 | ATLAng,WASHng ATLAng,NYCMng 5 21 1 215 | ATLAng,WASHng ATLAng,WASHng 5 24 1 216 | ATLAng,WASHng HSTNng,NYCMng 5 57 1 217 | ATLAng,WASHng HSTNng,WASHng 5 60 1 218 | ATLAng,WASHng LOSAng,WASHng 5 96 1 219 | CHINng,* ATLA-M5,CHINng 36 3 1 220 | CHINng,* ATLAng,CHINng 36 15 1 221 | CHINng,* CHINng,CHINng 36 27 1 222 | CHINng,* DNVRng,CHINng 36 39 1 223 | CHINng,* HSTNng,CHINng 36 51 1 224 | CHINng,* IPLSng,CHINng 36 63 1 225 | CHINng,* KSCYng,CHINng 36 75 1 226 | CHINng,* LOSAng,CHINng 36 87 1 227 | CHINng,* NYCMng,CHINng 36 99 1 228 | CHINng,* SNVAng,CHINng 36 111 1 229 | CHINng,* STTLng,CHINng 36 123 1 230 | CHINng,* WASHng,CHINng 36 135 1 231 | CHINng,IPLSng CHINng,ATLA-M5 6 25 1 232 | CHINng,IPLSng CHINng,ATLAng 6 26 1 233 | CHINng,IPLSng CHINng,DNVRng 6 28 1 234 | CHINng,IPLSng CHINng,HSTNng 6 29 1 235 | CHINng,IPLSng CHINng,IPLSng 6 30 1 236 | CHINng,IPLSng CHINng,KSCYng 6 31 1 237 | CHINng,IPLSng CHINng,LOSAng 6 32 1 238 | CHINng,IPLSng CHINng,SNVAng 6 34 1 239 | CHINng,IPLSng CHINng,STTLng 6 35 1 240 | CHINng,IPLSng NYCMng,DNVRng 6 100 1 241 | CHINng,IPLSng NYCMng,IPLSng 6 102 1 242 | CHINng,IPLSng NYCMng,KSCYng 6 103 1 243 | CHINng,IPLSng NYCMng,LOSAng 6 104 1 244 | CHINng,IPLSng NYCMng,SNVAng 6 106 1 245 | CHINng,IPLSng NYCMng,STTLng 6 107 1 246 | CHINng,IPLSng WASHng,DNVRng 6 136 1 247 | CHINng,IPLSng WASHng,IPLSng 6 138 1 248 | CHINng,IPLSng WASHng,KSCYng 6 139 1 249 | CHINng,IPLSng WASHng,SNVAng 6 142 1 250 | CHINng,IPLSng WASHng,STTLng 6 143 1 251 | CHINng,NYCMng CHINng,NYCMng 7 33 1 252 | CHINng,NYCMng CHINng,WASHng 7 36 1 253 | CHINng,NYCMng DNVRng,NYCMng 7 45 1 254 | CHINng,NYCMng DNVRng,WASHng 7 48 1 255 | CHINng,NYCMng IPLSng,NYCMng 7 69 1 256 | CHINng,NYCMng IPLSng,WASHng 7 72 1 257 | CHINng,NYCMng KSCYng,NYCMng 7 81 1 258 | CHINng,NYCMng KSCYng,WASHng 7 84 1 259 | CHINng,NYCMng LOSAng,NYCMng 7 93 1 260 | CHINng,NYCMng SNVAng,NYCMng 7 117 1 261 | CHINng,NYCMng SNVAng,WASHng 7 120 1 262 | CHINng,NYCMng STTLng,NYCMng 7 129 1 263 | CHINng,NYCMng STTLng,WASHng 7 132 1 264 | DNVRng,* ATLA-M5,DNVRng 38 4 1 265 | DNVRng,* ATLAng,DNVRng 38 16 1 266 | DNVRng,* CHINng,DNVRng 38 28 1 267 | DNVRng,* DNVRng,DNVRng 38 40 1 268 | DNVRng,* HSTNng,DNVRng 38 52 1 269 | DNVRng,* IPLSng,DNVRng 38 64 1 270 | DNVRng,* KSCYng,DNVRng 38 76 1 271 | DNVRng,* LOSAng,DNVRng 38 88 1 272 | DNVRng,* NYCMng,DNVRng 38 100 1 273 | DNVRng,* SNVAng,DNVRng 38 112 1 274 | DNVRng,* STTLng,DNVRng 38 124 1 275 | DNVRng,* WASHng,DNVRng 38 136 1 276 | DNVRng,KSCYng DNVRng,ATLA-M5 8 37 1 277 | DNVRng,KSCYng DNVRng,ATLAng 8 38 1 278 | DNVRng,KSCYng DNVRng,CHINng 8 39 1 279 | DNVRng,KSCYng DNVRng,HSTNng 8 41 1 280 | DNVRng,KSCYng DNVRng,IPLSng 8 42 1 281 | DNVRng,KSCYng DNVRng,KSCYng 8 43 1 282 | DNVRng,KSCYng DNVRng,NYCMng 8 45 1 283 | DNVRng,KSCYng DNVRng,WASHng 8 48 1 284 | DNVRng,KSCYng LOSAng,CHINng 8 87 1 285 | DNVRng,KSCYng LOSAng,IPLSng 8 90 1 286 | DNVRng,KSCYng LOSAng,KSCYng 8 91 1 287 | DNVRng,KSCYng LOSAng,NYCMng 8 93 1 288 | DNVRng,KSCYng SNVAng,ATLA-M5 8 109 1 289 | DNVRng,KSCYng SNVAng,ATLAng 8 110 1 290 | DNVRng,KSCYng SNVAng,CHINng 8 111 1 291 | DNVRng,KSCYng SNVAng,IPLSng 8 114 1 292 | DNVRng,KSCYng SNVAng,KSCYng 8 115 1 293 | DNVRng,KSCYng SNVAng,NYCMng 8 117 1 294 | DNVRng,KSCYng SNVAng,WASHng 8 120 1 295 | DNVRng,KSCYng STTLng,ATLA-M5 8 121 1 296 | DNVRng,KSCYng STTLng,ATLAng 8 122 1 297 | DNVRng,KSCYng STTLng,CHINng 8 123 1 298 | DNVRng,KSCYng STTLng,IPLSng 8 126 1 299 | DNVRng,KSCYng STTLng,KSCYng 8 127 1 300 | DNVRng,KSCYng STTLng,NYCMng 8 129 1 301 | DNVRng,KSCYng STTLng,WASHng 8 132 1 302 | DNVRng,SNVAng ATLA-M5,SNVAng 9 10 1 303 | DNVRng,SNVAng ATLAng,SNVAng 9 22 1 304 | DNVRng,SNVAng CHINng,LOSAng 9 32 1 305 | DNVRng,SNVAng CHINng,SNVAng 9 34 1 306 | DNVRng,SNVAng DNVRng,LOSAng 9 44 1 307 | DNVRng,SNVAng DNVRng,SNVAng 9 46 1 308 | DNVRng,SNVAng IPLSng,LOSAng 9 68 1 309 | DNVRng,SNVAng IPLSng,SNVAng 9 70 1 310 | DNVRng,SNVAng KSCYng,LOSAng 9 80 1 311 | DNVRng,SNVAng KSCYng,SNVAng 9 82 1 312 | DNVRng,SNVAng NYCMng,LOSAng 9 104 1 313 | DNVRng,SNVAng NYCMng,SNVAng 9 106 1 314 | DNVRng,SNVAng WASHng,SNVAng 9 142 1 315 | DNVRng,STTLng ATLA-M5,STTLng 10 11 1 316 | DNVRng,STTLng ATLAng,STTLng 10 23 1 317 | DNVRng,STTLng CHINng,STTLng 10 35 1 318 | DNVRng,STTLng DNVRng,STTLng 10 47 1 319 | DNVRng,STTLng IPLSng,STTLng 10 71 1 320 | DNVRng,STTLng KSCYng,STTLng 10 83 1 321 | DNVRng,STTLng NYCMng,STTLng 10 107 1 322 | DNVRng,STTLng WASHng,STTLng 10 143 1 323 | HSTNng,* ATLA-M5,HSTNng 40 5 1 324 | HSTNng,* ATLAng,HSTNng 40 17 1 325 | HSTNng,* CHINng,HSTNng 40 29 1 326 | HSTNng,* DNVRng,HSTNng 40 41 1 327 | HSTNng,* HSTNng,HSTNng 40 53 1 328 | HSTNng,* IPLSng,HSTNng 40 65 1 329 | HSTNng,* KSCYng,HSTNng 40 77 1 330 | HSTNng,* LOSAng,HSTNng 40 89 1 331 | HSTNng,* NYCMng,HSTNng 40 101 1 332 | HSTNng,* SNVAng,HSTNng 40 113 1 333 | HSTNng,* STTLng,HSTNng 40 125 1 334 | HSTNng,* WASHng,HSTNng 40 137 1 335 | HSTNng,ATLAng HSTNng,ATLA-M5 11 49 1 336 | HSTNng,ATLAng HSTNng,ATLAng 11 50 1 337 | HSTNng,ATLAng HSTNng,NYCMng 11 57 1 338 | HSTNng,ATLAng HSTNng,WASHng 11 60 1 339 | HSTNng,ATLAng LOSAng,ATLA-M5 11 85 1 340 | HSTNng,ATLAng LOSAng,ATLAng 11 86 1 341 | HSTNng,ATLAng LOSAng,WASHng 11 96 1 342 | HSTNng,KSCYng HSTNng,CHINng 12 51 1 343 | HSTNng,KSCYng HSTNng,DNVRng 12 52 1 344 | HSTNng,KSCYng HSTNng,IPLSng 12 54 1 345 | HSTNng,KSCYng HSTNng,KSCYng 12 55 1 346 | HSTNng,LOSAng ATLA-M5,LOSAng 13 8 1 347 | HSTNng,LOSAng ATLAng,LOSAng 13 20 1 348 | HSTNng,LOSAng HSTNng,LOSAng 13 56 1 349 | HSTNng,LOSAng HSTNng,SNVAng 13 58 1 350 | HSTNng,LOSAng HSTNng,STTLng 13 59 1 351 | HSTNng,LOSAng WASHng,LOSAng 13 140 1 352 | IPLSng,* ATLA-M5,IPLSng 42 6 1 353 | IPLSng,* ATLAng,IPLSng 42 18 1 354 | IPLSng,* CHINng,IPLSng 42 30 1 355 | IPLSng,* DNVRng,IPLSng 42 42 1 356 | IPLSng,* HSTNng,IPLSng 42 54 1 357 | IPLSng,* IPLSng,IPLSng 42 66 1 358 | IPLSng,* KSCYng,IPLSng 42 78 1 359 | IPLSng,* LOSAng,IPLSng 42 90 1 360 | IPLSng,* NYCMng,IPLSng 42 102 1 361 | IPLSng,* SNVAng,IPLSng 42 114 1 362 | IPLSng,* STTLng,IPLSng 42 126 1 363 | IPLSng,* WASHng,IPLSng 42 138 1 364 | IPLSng,ATLAng CHINng,ATLA-M5 14 25 1 365 | IPLSng,ATLAng CHINng,ATLAng 14 26 1 366 | IPLSng,ATLAng DNVRng,ATLA-M5 14 37 1 367 | IPLSng,ATLAng DNVRng,ATLAng 14 38 1 368 | IPLSng,ATLAng IPLSng,ATLA-M5 14 61 1 369 | IPLSng,ATLAng IPLSng,ATLAng 14 62 1 370 | IPLSng,ATLAng KSCYng,ATLA-M5 14 73 1 371 | IPLSng,ATLAng KSCYng,ATLAng 14 74 1 372 | IPLSng,ATLAng SNVAng,ATLA-M5 14 109 1 373 | IPLSng,ATLAng SNVAng,ATLAng 14 110 1 374 | IPLSng,ATLAng STTLng,ATLA-M5 14 121 1 375 | IPLSng,ATLAng STTLng,ATLAng 14 122 1 376 | IPLSng,CHINng ATLA-M5,CHINng 15 3 1 377 | IPLSng,CHINng ATLAng,CHINng 15 15 1 378 | IPLSng,CHINng DNVRng,CHINng 15 39 1 379 | IPLSng,CHINng DNVRng,NYCMng 15 45 1 380 | IPLSng,CHINng DNVRng,WASHng 15 48 1 381 | IPLSng,CHINng HSTNng,CHINng 15 51 1 382 | IPLSng,CHINng IPLSng,CHINng 15 63 1 383 | IPLSng,CHINng IPLSng,NYCMng 15 69 1 384 | IPLSng,CHINng IPLSng,WASHng 15 72 1 385 | IPLSng,CHINng KSCYng,CHINng 15 75 1 386 | IPLSng,CHINng KSCYng,NYCMng 15 81 1 387 | IPLSng,CHINng KSCYng,WASHng 15 84 1 388 | IPLSng,CHINng LOSAng,CHINng 15 87 1 389 | IPLSng,CHINng LOSAng,NYCMng 15 93 1 390 | IPLSng,CHINng SNVAng,CHINng 15 111 1 391 | IPLSng,CHINng SNVAng,NYCMng 15 117 1 392 | IPLSng,CHINng SNVAng,WASHng 15 120 1 393 | IPLSng,CHINng STTLng,CHINng 15 123 1 394 | IPLSng,CHINng STTLng,NYCMng 15 129 1 395 | IPLSng,CHINng STTLng,WASHng 15 132 1 396 | IPLSng,KSCYng ATLA-M5,DNVRng 16 4 1 397 | IPLSng,KSCYng ATLA-M5,KSCYng 16 7 1 398 | IPLSng,KSCYng ATLA-M5,SNVAng 16 10 1 399 | IPLSng,KSCYng ATLA-M5,STTLng 16 11 1 400 | IPLSng,KSCYng ATLAng,DNVRng 16 16 1 401 | IPLSng,KSCYng ATLAng,KSCYng 16 19 1 402 | IPLSng,KSCYng ATLAng,SNVAng 16 22 1 403 | IPLSng,KSCYng ATLAng,STTLng 16 23 1 404 | IPLSng,KSCYng CHINng,DNVRng 16 28 1 405 | IPLSng,KSCYng CHINng,HSTNng 16 29 1 406 | IPLSng,KSCYng CHINng,KSCYng 16 31 1 407 | IPLSng,KSCYng CHINng,LOSAng 16 32 1 408 | IPLSng,KSCYng CHINng,SNVAng 16 34 1 409 | IPLSng,KSCYng CHINng,STTLng 16 35 1 410 | IPLSng,KSCYng IPLSng,DNVRng 16 64 1 411 | IPLSng,KSCYng IPLSng,HSTNng 16 65 1 412 | IPLSng,KSCYng IPLSng,KSCYng 16 67 1 413 | IPLSng,KSCYng IPLSng,LOSAng 16 68 1 414 | IPLSng,KSCYng IPLSng,SNVAng 16 70 1 415 | IPLSng,KSCYng IPLSng,STTLng 16 71 1 416 | IPLSng,KSCYng NYCMng,DNVRng 16 100 1 417 | IPLSng,KSCYng NYCMng,KSCYng 16 103 1 418 | IPLSng,KSCYng NYCMng,LOSAng 16 104 1 419 | IPLSng,KSCYng NYCMng,SNVAng 16 106 1 420 | IPLSng,KSCYng NYCMng,STTLng 16 107 1 421 | IPLSng,KSCYng WASHng,DNVRng 16 136 1 422 | IPLSng,KSCYng WASHng,KSCYng 16 139 1 423 | IPLSng,KSCYng WASHng,SNVAng 16 142 1 424 | IPLSng,KSCYng WASHng,STTLng 16 143 1 425 | KSCYng,* ATLA-M5,KSCYng 44 7 1 426 | KSCYng,* ATLAng,KSCYng 44 19 1 427 | KSCYng,* CHINng,KSCYng 44 31 1 428 | KSCYng,* DNVRng,KSCYng 44 43 1 429 | KSCYng,* HSTNng,KSCYng 44 55 1 430 | KSCYng,* IPLSng,KSCYng 44 67 1 431 | KSCYng,* KSCYng,KSCYng 44 79 1 432 | KSCYng,* LOSAng,KSCYng 44 91 1 433 | KSCYng,* NYCMng,KSCYng 44 103 1 434 | KSCYng,* SNVAng,KSCYng 44 115 1 435 | KSCYng,* STTLng,KSCYng 44 127 1 436 | KSCYng,* WASHng,KSCYng 44 139 1 437 | KSCYng,DNVRng ATLA-M5,DNVRng 17 4 1 438 | KSCYng,DNVRng ATLA-M5,SNVAng 17 10 1 439 | KSCYng,DNVRng ATLA-M5,STTLng 17 11 1 440 | KSCYng,DNVRng ATLAng,DNVRng 17 16 1 441 | KSCYng,DNVRng ATLAng,SNVAng 17 22 1 442 | KSCYng,DNVRng ATLAng,STTLng 17 23 1 443 | KSCYng,DNVRng CHINng,DNVRng 17 28 1 444 | KSCYng,DNVRng CHINng,LOSAng 17 32 1 445 | KSCYng,DNVRng CHINng,SNVAng 17 34 1 446 | KSCYng,DNVRng CHINng,STTLng 17 35 1 447 | KSCYng,DNVRng HSTNng,DNVRng 17 52 1 448 | KSCYng,DNVRng IPLSng,DNVRng 17 64 1 449 | KSCYng,DNVRng IPLSng,LOSAng 17 68 1 450 | KSCYng,DNVRng IPLSng,SNVAng 17 70 1 451 | KSCYng,DNVRng IPLSng,STTLng 17 71 1 452 | KSCYng,DNVRng KSCYng,DNVRng 17 76 1 453 | KSCYng,DNVRng KSCYng,LOSAng 17 80 1 454 | KSCYng,DNVRng KSCYng,SNVAng 17 82 1 455 | KSCYng,DNVRng KSCYng,STTLng 17 83 1 456 | KSCYng,DNVRng NYCMng,DNVRng 17 100 1 457 | KSCYng,DNVRng NYCMng,LOSAng 17 104 1 458 | KSCYng,DNVRng NYCMng,SNVAng 17 106 1 459 | KSCYng,DNVRng NYCMng,STTLng 17 107 1 460 | KSCYng,DNVRng WASHng,DNVRng 17 136 1 461 | KSCYng,DNVRng WASHng,SNVAng 17 142 1 462 | KSCYng,DNVRng WASHng,STTLng 17 143 1 463 | KSCYng,HSTNng CHINng,HSTNng 18 29 1 464 | KSCYng,HSTNng DNVRng,HSTNng 18 41 1 465 | KSCYng,HSTNng IPLSng,HSTNng 18 65 1 466 | KSCYng,HSTNng KSCYng,HSTNng 18 77 1 467 | KSCYng,IPLSng DNVRng,ATLA-M5 19 37 1 468 | KSCYng,IPLSng DNVRng,ATLAng 19 38 1 469 | KSCYng,IPLSng DNVRng,CHINng 19 39 1 470 | KSCYng,IPLSng DNVRng,IPLSng 19 42 1 471 | KSCYng,IPLSng DNVRng,NYCMng 19 45 1 472 | KSCYng,IPLSng DNVRng,WASHng 19 48 1 473 | KSCYng,IPLSng HSTNng,CHINng 19 51 1 474 | KSCYng,IPLSng HSTNng,IPLSng 19 54 1 475 | KSCYng,IPLSng KSCYng,ATLA-M5 19 73 1 476 | KSCYng,IPLSng KSCYng,ATLAng 19 74 1 477 | KSCYng,IPLSng KSCYng,CHINng 19 75 1 478 | KSCYng,IPLSng KSCYng,IPLSng 19 78 1 479 | KSCYng,IPLSng KSCYng,NYCMng 19 81 1 480 | KSCYng,IPLSng KSCYng,WASHng 19 84 1 481 | KSCYng,IPLSng LOSAng,CHINng 19 87 1 482 | KSCYng,IPLSng LOSAng,IPLSng 19 90 1 483 | KSCYng,IPLSng LOSAng,NYCMng 19 93 1 484 | KSCYng,IPLSng SNVAng,ATLA-M5 19 109 1 485 | KSCYng,IPLSng SNVAng,ATLAng 19 110 1 486 | KSCYng,IPLSng SNVAng,CHINng 19 111 1 487 | KSCYng,IPLSng SNVAng,IPLSng 19 114 1 488 | KSCYng,IPLSng SNVAng,NYCMng 19 117 1 489 | KSCYng,IPLSng SNVAng,WASHng 19 120 1 490 | KSCYng,IPLSng STTLng,ATLA-M5 19 121 1 491 | KSCYng,IPLSng STTLng,ATLAng 19 122 1 492 | KSCYng,IPLSng STTLng,CHINng 19 123 1 493 | KSCYng,IPLSng STTLng,IPLSng 19 126 1 494 | KSCYng,IPLSng STTLng,NYCMng 19 129 1 495 | KSCYng,IPLSng STTLng,WASHng 19 132 1 496 | LOSAng,* ATLA-M5,LOSAng 46 8 1 497 | LOSAng,* ATLAng,LOSAng 46 20 1 498 | LOSAng,* CHINng,LOSAng 46 32 1 499 | LOSAng,* DNVRng,LOSAng 46 44 1 500 | LOSAng,* HSTNng,LOSAng 46 56 1 501 | LOSAng,* IPLSng,LOSAng 46 68 1 502 | LOSAng,* KSCYng,LOSAng 46 80 1 503 | LOSAng,* LOSAng,LOSAng 46 92 1 504 | LOSAng,* NYCMng,LOSAng 46 104 1 505 | LOSAng,* SNVAng,LOSAng 46 116 1 506 | LOSAng,* STTLng,LOSAng 46 128 1 507 | LOSAng,* WASHng,LOSAng 46 140 1 508 | LOSAng,HSTNng LOSAng,ATLA-M5 20 85 1 509 | LOSAng,HSTNng LOSAng,ATLAng 20 86 1 510 | LOSAng,HSTNng LOSAng,HSTNng 20 89 1 511 | LOSAng,HSTNng LOSAng,WASHng 20 96 1 512 | LOSAng,HSTNng SNVAng,HSTNng 20 113 1 513 | LOSAng,HSTNng STTLng,HSTNng 20 125 1 514 | LOSAng,SNVAng HSTNng,SNVAng 21 58 1 515 | LOSAng,SNVAng HSTNng,STTLng 21 59 1 516 | LOSAng,SNVAng LOSAng,CHINng 21 87 1 517 | LOSAng,SNVAng LOSAng,DNVRng 21 88 1 518 | LOSAng,SNVAng LOSAng,IPLSng 21 90 1 519 | LOSAng,SNVAng LOSAng,KSCYng 21 91 1 520 | LOSAng,SNVAng LOSAng,NYCMng 21 93 1 521 | LOSAng,SNVAng LOSAng,SNVAng 21 94 1 522 | LOSAng,SNVAng LOSAng,STTLng 21 95 1 523 | NYCMng,* ATLA-M5,NYCMng 48 9 1 524 | NYCMng,* ATLAng,NYCMng 48 21 1 525 | NYCMng,* CHINng,NYCMng 48 33 1 526 | NYCMng,* DNVRng,NYCMng 48 45 1 527 | NYCMng,* HSTNng,NYCMng 48 57 1 528 | NYCMng,* IPLSng,NYCMng 48 69 1 529 | NYCMng,* KSCYng,NYCMng 48 81 1 530 | NYCMng,* LOSAng,NYCMng 48 93 1 531 | NYCMng,* NYCMng,NYCMng 48 105 1 532 | NYCMng,* SNVAng,NYCMng 48 117 1 533 | NYCMng,* STTLng,NYCMng 48 129 1 534 | NYCMng,* WASHng,NYCMng 48 141 1 535 | NYCMng,CHINng NYCMng,CHINng 22 99 1 536 | NYCMng,CHINng NYCMng,DNVRng 22 100 1 537 | NYCMng,CHINng NYCMng,IPLSng 22 102 1 538 | NYCMng,CHINng NYCMng,KSCYng 22 103 1 539 | NYCMng,CHINng NYCMng,LOSAng 22 104 1 540 | NYCMng,CHINng NYCMng,SNVAng 22 106 1 541 | NYCMng,CHINng NYCMng,STTLng 22 107 1 542 | NYCMng,CHINng WASHng,CHINng 22 135 1 543 | NYCMng,CHINng WASHng,DNVRng 22 136 1 544 | NYCMng,CHINng WASHng,IPLSng 22 138 1 545 | NYCMng,CHINng WASHng,KSCYng 22 139 1 546 | NYCMng,CHINng WASHng,SNVAng 22 142 1 547 | NYCMng,CHINng WASHng,STTLng 22 143 1 548 | NYCMng,WASHng CHINng,WASHng 23 36 1 549 | NYCMng,WASHng DNVRng,WASHng 23 48 1 550 | NYCMng,WASHng IPLSng,WASHng 23 72 1 551 | NYCMng,WASHng KSCYng,WASHng 23 84 1 552 | NYCMng,WASHng NYCMng,ATLA-M5 23 97 1 553 | NYCMng,WASHng NYCMng,ATLAng 23 98 1 554 | NYCMng,WASHng NYCMng,HSTNng 23 101 1 555 | NYCMng,WASHng NYCMng,WASHng 23 108 1 556 | NYCMng,WASHng SNVAng,WASHng 23 120 1 557 | NYCMng,WASHng STTLng,WASHng 23 132 1 558 | SNVAng,* ATLA-M5,SNVAng 50 10 1 559 | SNVAng,* ATLAng,SNVAng 50 22 1 560 | SNVAng,* CHINng,SNVAng 50 34 1 561 | SNVAng,* DNVRng,SNVAng 50 46 1 562 | SNVAng,* HSTNng,SNVAng 50 58 1 563 | SNVAng,* IPLSng,SNVAng 50 70 1 564 | SNVAng,* KSCYng,SNVAng 50 82 1 565 | SNVAng,* LOSAng,SNVAng 50 94 1 566 | SNVAng,* NYCMng,SNVAng 50 106 1 567 | SNVAng,* SNVAng,SNVAng 50 118 1 568 | SNVAng,* STTLng,SNVAng 50 130 1 569 | SNVAng,* WASHng,SNVAng 50 142 1 570 | SNVAng,DNVRng LOSAng,CHINng 24 87 1 571 | SNVAng,DNVRng LOSAng,DNVRng 24 88 1 572 | SNVAng,DNVRng LOSAng,IPLSng 24 90 1 573 | SNVAng,DNVRng LOSAng,KSCYng 24 91 1 574 | SNVAng,DNVRng LOSAng,NYCMng 24 93 1 575 | SNVAng,DNVRng SNVAng,ATLA-M5 24 109 1 576 | SNVAng,DNVRng SNVAng,ATLAng 24 110 1 577 | SNVAng,DNVRng SNVAng,CHINng 24 111 1 578 | SNVAng,DNVRng SNVAng,DNVRng 24 112 1 579 | SNVAng,DNVRng SNVAng,IPLSng 24 114 1 580 | SNVAng,DNVRng SNVAng,KSCYng 24 115 1 581 | SNVAng,DNVRng SNVAng,NYCMng 24 117 1 582 | SNVAng,DNVRng SNVAng,WASHng 24 120 1 583 | SNVAng,LOSAng CHINng,LOSAng 25 32 1 584 | SNVAng,LOSAng DNVRng,LOSAng 25 44 1 585 | SNVAng,LOSAng IPLSng,LOSAng 25 68 1 586 | SNVAng,LOSAng KSCYng,LOSAng 25 80 1 587 | SNVAng,LOSAng NYCMng,LOSAng 25 104 1 588 | SNVAng,LOSAng SNVAng,HSTNng 25 113 1 589 | SNVAng,LOSAng SNVAng,LOSAng 25 116 1 590 | SNVAng,LOSAng STTLng,HSTNng 25 125 1 591 | SNVAng,LOSAng STTLng,LOSAng 25 128 1 592 | SNVAng,STTLng HSTNng,STTLng 26 59 1 593 | SNVAng,STTLng LOSAng,STTLng 26 95 1 594 | SNVAng,STTLng SNVAng,STTLng 26 119 1 595 | STTLng,* ATLA-M5,STTLng 52 11 1 596 | STTLng,* ATLAng,STTLng 52 23 1 597 | STTLng,* CHINng,STTLng 52 35 1 598 | STTLng,* DNVRng,STTLng 52 47 1 599 | STTLng,* HSTNng,STTLng 52 59 1 600 | STTLng,* IPLSng,STTLng 52 71 1 601 | STTLng,* KSCYng,STTLng 52 83 1 602 | STTLng,* LOSAng,STTLng 52 95 1 603 | STTLng,* NYCMng,STTLng 52 107 1 604 | STTLng,* SNVAng,STTLng 52 119 1 605 | STTLng,* STTLng,STTLng 52 131 1 606 | STTLng,* WASHng,STTLng 52 143 1 607 | STTLng,DNVRng STTLng,ATLA-M5 27 121 1 608 | STTLng,DNVRng STTLng,ATLAng 27 122 1 609 | STTLng,DNVRng STTLng,CHINng 27 123 1 610 | STTLng,DNVRng STTLng,DNVRng 27 124 1 611 | STTLng,DNVRng STTLng,IPLSng 27 126 1 612 | STTLng,DNVRng STTLng,KSCYng 27 127 1 613 | STTLng,DNVRng STTLng,NYCMng 27 129 1 614 | STTLng,DNVRng STTLng,WASHng 27 132 1 615 | STTLng,SNVAng STTLng,HSTNng 28 125 1 616 | STTLng,SNVAng STTLng,LOSAng 28 128 1 617 | STTLng,SNVAng STTLng,SNVAng 28 130 1 618 | WASHng,* ATLA-M5,WASHng 54 12 1 619 | WASHng,* ATLAng,WASHng 54 24 1 620 | WASHng,* CHINng,WASHng 54 36 1 621 | WASHng,* DNVRng,WASHng 54 48 1 622 | WASHng,* HSTNng,WASHng 54 60 1 623 | WASHng,* IPLSng,WASHng 54 72 1 624 | WASHng,* KSCYng,WASHng 54 84 1 625 | WASHng,* LOSAng,WASHng 54 96 1 626 | WASHng,* NYCMng,WASHng 54 108 1 627 | WASHng,* SNVAng,WASHng 54 120 1 628 | WASHng,* STTLng,WASHng 54 132 1 629 | WASHng,* WASHng,WASHng 54 144 1 630 | WASHng,ATLAng NYCMng,ATLA-M5 29 97 1 631 | WASHng,ATLAng NYCMng,ATLAng 29 98 1 632 | WASHng,ATLAng NYCMng,HSTNng 29 101 1 633 | WASHng,ATLAng WASHng,ATLA-M5 29 133 1 634 | WASHng,ATLAng WASHng,ATLAng 29 134 1 635 | WASHng,ATLAng WASHng,HSTNng 29 137 1 636 | WASHng,ATLAng WASHng,LOSAng 29 140 1 637 | WASHng,NYCMng ATLA-M5,NYCMng 30 9 1 638 | WASHng,NYCMng ATLAng,NYCMng 30 21 1 639 | WASHng,NYCMng HSTNng,NYCMng 30 57 1 640 | WASHng,NYCMng WASHng,CHINng 30 135 1 641 | WASHng,NYCMng WASHng,DNVRng 30 136 1 642 | WASHng,NYCMng WASHng,IPLSng 30 138 1 643 | WASHng,NYCMng WASHng,KSCYng 30 139 1 644 | WASHng,NYCMng WASHng,NYCMng 30 141 1 645 | WASHng,NYCMng WASHng,SNVAng 30 142 1 646 | WASHng,NYCMng WASHng,STTLng 30 143 1 647 | -------------------------------------------------------------------------------- /abilene/downloadlink.txt: -------------------------------------------------------------------------------- 1 | http://www.cs.utexas.edu/~yzhang/research/AbileneTM/ 2 | -------------------------------------------------------------------------------- /abilene/links: -------------------------------------------------------------------------------- 1 | # link(x,y) link_index link_type 2 | # link_type: 0: internal, 1: inbound, 2: outbound 3 | ATLA-M5,ATLAng 1 0 4 | ATLAng,ATLA-M5 2 0 5 | ATLAng,HSTNng 3 0 6 | ATLAng,IPLSng 4 0 7 | ATLAng,WASHng 5 0 8 | CHINng,IPLSng 6 0 9 | CHINng,NYCMng 7 0 10 | DNVRng,KSCYng 8 0 11 | DNVRng,SNVAng 9 0 12 | DNVRng,STTLng 10 0 13 | HSTNng,ATLAng 11 0 14 | HSTNng,KSCYng 12 0 15 | HSTNng,LOSAng 13 0 16 | IPLSng,ATLAng 14 0 17 | IPLSng,CHINng 15 0 18 | IPLSng,KSCYng 16 0 19 | KSCYng,DNVRng 17 0 20 | KSCYng,HSTNng 18 0 21 | KSCYng,IPLSng 19 0 22 | LOSAng,HSTNng 20 0 23 | LOSAng,SNVAng 21 0 24 | NYCMng,CHINng 22 0 25 | NYCMng,WASHng 23 0 26 | SNVAng,DNVRng 24 0 27 | SNVAng,LOSAng 25 0 28 | SNVAng,STTLng 26 0 29 | STTLng,DNVRng 27 0 30 | STTLng,SNVAng 28 0 31 | WASHng,ATLAng 29 0 32 | WASHng,NYCMng 30 0 33 | *,ATLA-M5 31 1 34 | ATLA-M5,* 32 2 35 | *,ATLAng 33 1 36 | ATLAng,* 34 2 37 | *,CHINng 35 1 38 | CHINng,* 36 2 39 | *,DNVRng 37 1 40 | DNVRng,* 38 2 41 | *,HSTNng 39 1 42 | HSTNng,* 40 2 43 | *,IPLSng 41 1 44 | IPLSng,* 42 2 45 | *,KSCYng 43 1 46 | KSCYng,* 44 2 47 | *,LOSAng 45 1 48 | LOSAng,* 46 2 49 | *,NYCMng 47 1 50 | NYCMng,* 48 2 51 | *,SNVAng 49 1 52 | SNVAng,* 50 2 53 | *,STTLng 51 1 54 | STTLng,* 52 2 55 | *,WASHng 53 1 56 | WASHng,* 54 2 57 | -------------------------------------------------------------------------------- /abilene/readme.txt: -------------------------------------------------------------------------------- 1 | * Format of X*.gz 2 | 3 | Each file contains 12x24x7 = 2016 5-min traffic matrices. 4 | 5 | Unit: (100 bytes / 5 minutes) // 100 is the pkt sampling rate 6 | 7 | Each line belongs to one TM; and each line contains 144x5=720 values 8 | organized as follows: 9 | 10 | \ 11 | \ 12 | \ 13 | \ 14 | \ 15 | 16 | ... 17 | 18 | \ 19 | \ 20 | \ 21 | \ 22 | \ 23 | 24 | where simpleGravity means the independence model: p(s,d) = p(s)p(d), 25 | whereas generalGravity means the conditional independence model, 26 | which treats outbound traffic differently. simpleTomogravity and 27 | generalTomogravity are the tomogravity estimates with either 28 | simpleGravity or generalGravity as the prior. 29 | 30 | For Abilene, simpleGravity and generalGravity gives similar results. 31 | But in other networks, generalTomogravity often performs much 32 | better. 33 | 34 | * Format of A 35 | 36 | A is the routing matrix. it should be self-explanatory. 37 | 38 | * Dates (the first day of each week) 39 | 40 | 2004-03-01 X01 41 | 2004-03-08 X02 42 | 2004-04-02 X03 43 | 2004-04-09 X04 44 | 2004-04-22 X05 45 | 2004-05-01 X06 46 | 2004-05-08 X07 47 | 2004-05-15 X08 48 | 2004-05-22 X09 49 | 2004-05-29 X10 50 | 2004-06-05 X11 51 | 2004-06-12 X12 52 | 2004-06-19 X13 53 | 2004-06-26 X14 54 | 2004-07-03 X15 55 | 2004-07-10 X16 56 | 2004-07-17 X17 57 | 2004-07-24 X18 58 | 2004-07-31 X19 59 | 2004-08-07 X20 60 | 2004-08-13 X21 61 | 2004-08-21 X22 62 | 2004-08-28 X23 63 | 2004-09-04 X24 64 | 65 | * Collection methodology 66 | 67 | The idea is to directly work with destination AS numbers (which 68 | themselves are computed by the routers using the BGP tables when each 69 | netflow record is generated). 70 | 71 | (1) I examine the netflow leaving the egress links and build a set of 72 | destination AS numbers that egress at each router 73 | 74 | (2) from (1), for each destination AS number, I can find all the egress 75 | routers correspond to this AS. 76 | 77 | (3) At each ingress router, for each flow record, I can map its 78 | destination AS number to a set of egress routers using results 79 | of (2). I then simulate OSPF to pick one egress router that is 80 | the closest. 81 | 82 | The above works as long as the destination AS is not Abilene (AS11537). 83 | 84 | When the destination is Abilene, in principle you could do the same 85 | thing on destination prefixes. Unfortunately, the Abilene netflow 86 | mask out the last 11 bits of all the IP addresses, so it is impossible 87 | to get the true destination prefixes for Abilene. As a result, the 88 | TMs I generated do not include any traffic destined for AS11537 89 | (fortunately they don't account for too much traffic). 90 | -------------------------------------------------------------------------------- /abilene/topo-2003-04-10.txt: -------------------------------------------------------------------------------- 1 | router 2 | #name city latitude longitude 3 | ATLA-M5 Atlanta_GA 33.750000 -84.383300 4 | ATLAng Atlanta_GA 33.750000 -84.383300 5 | CHINng Chicago_IL 41.833300 -87.616700 6 | DNVRng Denver_CO 40.750000 -105.000000 7 | HSTNng Houston_TX 29.770031 -95.517364 8 | IPLSng Indianapolis_IN 39.780622 -86.159535 9 | KSCYng Kansas_City_MO 38.961694 -96.596704 10 | LOSAng Los_Angeles_CA 34.050000 -118.250000 11 | NYCMng New_York_NY 40.783300 -73.966700 12 | SNVAng Sunnyvale_CA 37.38575 -122.02553 13 | STTLng Seattle_WA 47.600000 -122.300000 14 | WASHng Washington_DC 38.897303 -77.026842 15 | link 16 | #capacity on 4/10/2003 (see page 20 of http://www.internet2.edu/presentations/spring03/20030410-Abilene-Corbato.pdf) 17 | #OSPF weight on 04/10/2003 (see http://www.abilene.iu.edu/images/Ab-IGP-topo.jpg) 18 | #x y capacity(kbps) OSPF 19 | ATLA-M5 ATLAng 9920000 1 20 | ATLAng ATLA-M5 9920000 1 21 | ATLAng HSTNng 9920000 1176 22 | ATLAng IPLSng 2480000 587 23 | ATLAng WASHng 9920000 846 24 | CHINng IPLSng 9920000 260 25 | CHINng NYCMng 9920000 700 26 | DNVRng KSCYng 9920000 639 27 | DNVRng SNVAng 9920000 1295 28 | DNVRng STTLng 9920000 2095 29 | HSTNng ATLAng 9920000 1176 30 | HSTNng KSCYng 9920000 902 31 | HSTNng LOSAng 9920000 1893 32 | IPLSng ATLAng 2480000 587 33 | IPLSng CHINng 9920000 260 34 | IPLSng KSCYng 9920000 548 35 | KSCYng DNVRng 9920000 639 36 | KSCYng HSTNng 9920000 902 37 | KSCYng IPLSng 9920000 548 38 | LOSAng HSTNng 9920000 1893 39 | LOSAng SNVAng 9920000 366 40 | NYCMng CHINng 9920000 700 41 | NYCMng WASHng 9920000 233 42 | SNVAng DNVRng 9920000 1295 43 | SNVAng LOSAng 9920000 366 44 | SNVAng STTLng 9920000 861 45 | STTLng DNVRng 9920000 2095 46 | STTLng SNVAng 9920000 861 47 | WASHng ATLAng 9920000 846 48 | WASHng NYCMng 9920000 233 49 | -------------------------------------------------------------------------------- /code/full-pipeline/.gitignore: -------------------------------------------------------------------------------- 1 | tmp* 2 | *.RData 3 | *.RHist 4 | *.Rhistory 5 | .Rproj.user 6 | *.Rproj 7 | sfcrhist*.txt 8 | *.rds 9 | requests-*.txt 10 | h2omodels-* 11 | xgb-grid/* 12 | data/* 13 | h2o-10h-* 14 | nodeinfo*.txt 15 | routeinfo*.txt 16 | nvnfs*.txt 17 | -------------------------------------------------------------------------------- /code/full-pipeline/001_generateSFCRTrace.R: -------------------------------------------------------------------------------- 1 | ####################################################### 2 | ### Step 001 - Generation of SFC requests. ### 3 | ### Output: SFC request trace with information on ### 4 | ### arrival time, duration, involved nodes, ### 5 | ### VNF chain, and constraints. ### 6 | ####################################################### 7 | 8 | source("common.R") 9 | 10 | ################# 11 | ### CLI Opts. ### 12 | ################# 13 | 14 | optlist <- list( 15 | make_option( 16 | "--rdsOutfile", type = "character", default = "requests.rds", 17 | help = "Name of RDS output file containing the SFC request trace. [default is %default]."), 18 | make_option( 19 | "--txtOutfile", type = "character", default = "requests.txt", 20 | help = "Name of txt output file containing the SFC request trace. [default is %default]."), 21 | make_option( 22 | "--topoFile", type = "character", default = "inet2", 23 | help = "Name of the topology file to use [default is %default]."), 24 | make_option( 25 | "--reqsPerMin", type = "double", default = "2", 26 | help = "Number of requests per minute to generate. [default is %default]."), 27 | make_option( 28 | "--iatDist", type = "character", default = "negexp", 29 | help = "Distribution to use for generating request interarrival times. [default is negative exponential]."), 30 | make_option( 31 | "--nMinutes", type = "integer", default = "10000", 32 | help = "Duration of result trace in minutes. [default is %default (one week)]."), 33 | make_option( 34 | "--meanDuration", type = "integer", default = "1000", 35 | help = "Mean duration of SFC requests. Durations are composed of 100 + NegExp with mean = meanDuration - 100. [default is %default]."), 36 | make_option( 37 | "--durDist", type = "character", default = "negexp", 38 | help = "Distribution to use for generating request durations. [default is negative exponential]."), 39 | make_option( 40 | "--seed", type = "integer", default = "42", 41 | help = "RNG seed for reproducibility. [default is %default].")) 42 | 43 | optparser <- OptionParser(option_list = optlist) 44 | 45 | opt <- parse_args(optparser) 46 | 47 | set.seed(opt$seed) 48 | 49 | ############# 50 | ### Main. ### 51 | ############# 52 | 53 | # Use abilene-preproc.R to generate abilene-week1.csv. 54 | # nvolume refers to normalized traffic volume. 55 | abilene <- 56 | fread("abilene-week1.csv", col.names = c("minute", "nvolume")) %>% 57 | mutate(second = minute * 60) 58 | 59 | # 1 SFC request per minute during peak hours. 60 | # Note: lowest rate value leads to highest number of requests! 61 | # 1 / rate = mean interarrival time. 62 | reqsPerMin <- opt$reqsPerMin 63 | minrate <- reqsPerMin / 60 64 | 65 | iatDist <- opt$iatDist 66 | durDist <- opt$durDist 67 | 68 | # Calculate resulting rate. 69 | abilene <- 70 | abilene %>% 71 | mutate(rate = minrate * nvolume) 72 | 73 | # Number of requests to generate. 1 week ~ 10k minutes, reqsPerMin per minute => ~10k * reqsPerMin requests in a week. 74 | nrequests <- opt$nMinutes * reqsPerMin 75 | 76 | 77 | # SFC catalog with VNF chains and associated probabilities (= fraction of total traffic volume). 78 | sfccat <- data.frame( 79 | sfc = I(c( 80 | list(c("nat", "firewall", "ids")), 81 | list(c("nat", "proxy")), 82 | list(c("nat", "wano")), 83 | list(c("nat", "firewall", "wano", "ids")))), 84 | p = c(.6, .1, .2, .1)) %>% 85 | mutate(sfcid = 1:nrow(.)) 86 | 87 | requests <- data.frame() 88 | 89 | # Generate an arrival process for each SFC in the catalog. 90 | for (i in 1:nrow(sfccat)) { 91 | curNRequests <- round(sfccat$p[i] * nrequests) 92 | curMinrate <- sfccat$p[i] * minrate 93 | curArrivaltimes <- switch (iatDist, 94 | "negexp" = cumsum(rexp(n = curNRequests, rate = curMinrate)), 95 | "norm" = cumsum(rnorm(n = curNRequests, mean = 1 / curMinrate, sd = 7.5))) 96 | requests <- rbind(requests, data.frame(arrivaltime = curArrivaltimes, sfcid = sfccat$sfcid[i])) 97 | } 98 | 99 | # Remove requests based on the desired rate at the corresponding arrival time. 100 | requests <- 101 | requests %>% 102 | arrange(arrivaltime) %>% 103 | mutate(probkeep = abilene$nvolume[as.numeric(cut(arrivaltime, abilene$second))]) %>% 104 | mutate(keep = runif(n = n()) < probkeep) %>% 105 | filter(keep == T) 106 | 107 | # VNF catalog. 108 | vnfcat <- 109 | fread("middlebox-spec", col.names = c("name", "cpu", "delay", "capacity"), select = 1:4) %>% 110 | arrange(name) 111 | saveRDS(vnfcat, "vnfcat.rds") 112 | vnfs <- vnfcat$name 113 | 114 | topofile <- opt$topoFile 115 | topoinfo <- readTopo(topofile) 116 | 117 | # Number of nodes in the network. 118 | nnodes <- topoinfo[[1]][1] 119 | 120 | # Keep mean duration of 1000, but make it an RV. Rounding since CPLEX solver expects integer duration. 121 | duration <- switch (durDist, 122 | "negexp" = round(100 + rexp(nrow(requests), 1 / (opt$meanDuration - 100))), 123 | "norm" = round(100 + rnorm(nrow(requests), mean = opt$meanDuration - 100, sd = opt$meanDuration / 4))) 124 | 125 | requests$duration <- duration 126 | requests$src <- 0 127 | requests$dst <- 0 128 | 129 | # Allowed / occurring chain lengths. 130 | allowedchainlengths <- 1:4 131 | colnames <- sprintf("vnf%d", allowedchainlengths) 132 | requests[, colnames] <- NA 133 | 134 | # Constant used in original paper, required solver input. 135 | requests$penalty <- sprintf("%.8f", 0.00000010) 136 | 137 | # Control average number of requests per instance and latency restrictions via these parameters. 138 | requests$traffic <- round(70000 + runif(n = nrow(requests)) * 50000) 139 | requests$maxlat <- round(700 + runif(n = nrow(requests)) * 50) 140 | 141 | for (i in 1:nrow(requests)) { 142 | # Random source and destination pairs. 143 | # EXTENSION: probability of choosing a pair could be based on the TM. 144 | 145 | allowednodes <- switch (topofile, 146 | # All nodes are allowed in the case of Internet2. 147 | "inet2" = 1:nnodes, 148 | # FIXME: hardcoded for now. Only leaves are allowed as source / destination node in the case 149 | # of the MEC topology. 150 | "mec" = topoinfo[[3]] %>% filter(cpu == 40) %>% pull(v) 151 | ) 152 | 153 | # FIXME: general / at least for MEC: it's not unreasonable to allow having instances with src == dst. 154 | fromto <- sample(allowednodes, 2) 155 | 156 | # -1 since CPLEX uses 0-indexing. 157 | requests$src[i] <- min(fromto) - 1 158 | requests$dst[i] <- max(fromto) - 1 159 | } 160 | 161 | # Explicitly add VNF information for each position in the chain. 162 | for (i in 1:nrow(sfccat)) { 163 | curChain <- sfccat$sfc[i][[1]] 164 | affectedReqs <- which(requests$sfcid == sfccat$sfcid[i]) 165 | for (j in 1:length(curChain)) { 166 | requests[affectedReqs, sprintf("vnf%d", j)] <- curChain[j] 167 | } 168 | } 169 | 170 | # Convert into a format that is compatible with traffic files used by the solver. 171 | requests <- 172 | requests %>% 173 | mutate(arrivaltime = round(arrivaltime)) %>% 174 | select(arrivaltime, duration, src, dst, traffic, maxlat, penalty, colnames, sfcid) %>% 175 | mutate(arrivaltime = sprintf("%d", arrivaltime), traffic = sprintf("%d", traffic)) 176 | 177 | saveRDS(requests, file = opt$rdsOutfile) 178 | 179 | write.table(requests %>% select(-one_of("sfcid")), file = opt$txtOutfile, sep = ",", col.names = F, row.names = F, quote = F) 180 | -------------------------------------------------------------------------------- /code/full-pipeline/002_sfcrTrace2CPLEX.py: -------------------------------------------------------------------------------- 1 | ############################################################# 2 | ### Step 002 - Generation of placement problem instances. ### 3 | ### Output: files with traffic lists that are compatible ### 4 | ### with the CPLEX-based solver. ### 5 | ############################################################# 6 | 7 | import argparse 8 | from builtins import range 9 | import functools 10 | import glob 11 | import multiprocessing 12 | import random 13 | import re 14 | import subprocess 15 | 16 | # Given a SFC request history, generate traffic lists for CPLEX-based solver. 17 | # For each request arrival, these traffic lists contain a list of requests 18 | # that are active at that time and represent an instance of the VNF placement problem. 19 | # Detailed information regarding individual parameters can be found below (parser.add_argument(...)). 20 | def sfcrTrace2cplex(sfcrTraceFile, outFilePath, suffix): 21 | with open(sfcrTraceFile) as f: 22 | content = f.readlines() 23 | nlines = len(content) 24 | for i in range(0, nlines): 25 | if i % 100 == 0: 26 | print('%05d/%05d' % (i, nlines)) 27 | partsI = content[i].split(',') 28 | (arrI, durI) = (int(partsI[0]), int(partsI[1])) 29 | with open(outFilePath + '/traf-%s-%05d' % (suffix, i + 1), 'w') as fout: 30 | # i + 1 to take into account the current request 31 | for j in range(0, i + 1): 32 | partsJ = content[j].split(',') 33 | (arrJ, durJ) = (int(partsJ[0]), int(partsJ[1])) 34 | # Ignore requests that are not active anymore when the i-th request arrives. 35 | if arrJ + durJ < arrI: 36 | continue 37 | # Replace arrival time with 0, remove duration, and place everything at once. 38 | # WARNING: don't use \r\n here, since then middleman (the CPLEX-based solver) will ignore the last VNF in the chain due to parsing its name as $name^M. 39 | fout.write('0,' + ','.join([pt.strip() for pt in partsJ[2:] if not pt.startswith('NA')]) + '\n') 40 | 41 | if __name__ == '__main__': 42 | 43 | parser = argparse.ArgumentParser() 44 | 45 | parser.add_argument( 46 | "--sfcrTraceFile", 47 | help = "Text file with the entire SFC request trace. [default is requests.txt].", 48 | type = str, 49 | default = "requests.txt") 50 | 51 | parser.add_argument( 52 | "--outFilePath", 53 | help = "Folder in which to store requests that are compatible with the CPLEX-based solver. [default is ../../solver/middlebox-placement/src/logs].", 54 | type = str, 55 | default = "../../solver/middlebox-placement/src/logs") 56 | 57 | parser.add_argument( 58 | "--suffix", 59 | help = "Suffix to add to output log files. The resulting file names have the following structure: traf-$suffix-$requestID. [default is the empty string].", 60 | type = str, 61 | default = "") 62 | 63 | args = parser.parse_args() 64 | 65 | sfcrTrace2cplex( 66 | sfcrTraceFile = args.sfcrTraceFile, 67 | outFilePath = args.outFilePath, 68 | suffix = args.suffix) 69 | -------------------------------------------------------------------------------- /code/full-pipeline/003_runCPLEXPar.py: -------------------------------------------------------------------------------- 1 | ######################################################### 2 | ### Step 003 - Calculation of optimal placements. ### 3 | ### Output: log files with placement and solver data. ### 4 | ######################################################### 5 | 6 | import argparse 7 | from builtins import range 8 | import functools 9 | import glob 10 | import multiprocessing 11 | import random 12 | import re 13 | import subprocess 14 | 15 | # Run CPLEX-based solver on all traffic files that match the supplied pattern. 16 | # Detailed information regarding individual parameters can be found below (parser.add_argument(...)). 17 | def runCPLEXPar(trafficListPath, topoFile, suffix, ncores): 18 | trafficListPattern = 'traf-%s-*' % suffix 19 | trafficListPrefix = 'traf-%s-' % suffix 20 | trafficLists = glob.glob(trafficListPath + '/' + trafficListPattern) 21 | nFiles = len(trafficLists) 22 | theRange = range(1, nFiles) 23 | # Support for continuing a cancelled run - make sure to remove the 24 | # last ncores log files since they might be incomplete. 25 | existingLogfiles = glob.glob('%s/cplex-full-%s-*' % (trafficListPath, suffix)) 26 | completedIDs = [int(x.split('-')[-1].split('.')[0]) for x in existingLogfiles] 27 | remainingIndices = list(set(theRange) - set(completedIDs)) 28 | # Single-argument version of runCPLEXSingleInstance. 29 | cplexSingle = functools.partial(runCPLEXSingleInstance, trafficListPath = trafficListPath, topoFile = topoFile, suffix = suffix) 30 | pool = multiprocessing.Pool(processes = ncores) 31 | res = pool.map(cplexSingle, remainingIndices) 32 | 33 | # Just like runCPLEX, but performs only the i-th iteration. 34 | def runCPLEXSingleInstance(i, trafficListPath, topoFile, suffix): 35 | trafficListPattern = 'traf-%s-*' % suffix 36 | trafficListPrefix = 'traf-%s-' % suffix 37 | trafficLists = glob.glob(trafficListPath + '/' + trafficListPattern) 38 | nFiles = len(trafficLists) 39 | print('%05d/%05d' % (i, nFiles)) 40 | outfile = open('%s/cplex-full-%s-%05d.log' % (trafficListPath, suffix, i), 'w') 41 | runcommand = './middleman --per_core_cost=0.01 --per_bit_transit_cost=3.626543209876543e-7 --topology_file=%s --middlebox_spec_file=middlebox-spec --traffic_request_file=logs/%s%05d --max_time=7200 --algorithm=cplex' % (topoFile, trafficListPrefix, i) 42 | subprocess.call( 43 | runcommand.split(), 44 | cwd = '../../solver/middlebox-placement/src', stdout = outfile) 45 | 46 | if __name__ == '__main__': 47 | 48 | parser = argparse.ArgumentParser() 49 | 50 | parser.add_argument( 51 | "--trafficListPath", 52 | help = "Folder in which to look for request lists that are compatible with the CPLEX-based solver. [default is ../../solver/middlebox-placement/src/logs].", 53 | type = str, 54 | default = "../../solver/middlebox-placement/src/logs") 55 | 56 | parser.add_argument( 57 | "--topoFile", 58 | help = "Name of the topology file to use. [default is inet2].", 59 | type = str, 60 | default = "inet2") 61 | 62 | parser.add_argument( 63 | "--suffix", 64 | help = "Suffix to add to output log files. The resulting file names have the following structure: cplex-full-$suffix-$requestID.log. [default is the empty string].", 65 | type = str, 66 | default = "") 67 | 68 | parser.add_argument( 69 | "--ncores", 70 | help = "Number of subprocesses to launch. Each will launch a separate CPLEX instance. [default is 18].", 71 | type = int, 72 | default = 18) 73 | 74 | args = parser.parse_args() 75 | 76 | runCPLEXPar( 77 | trafficListPath = args.trafficListPath, 78 | topoFile = args.topoFile, 79 | suffix = args.suffix, 80 | ncores = args.ncores) 81 | -------------------------------------------------------------------------------- /code/full-pipeline/004_getPlacementInfoFromCPLEXLogs.py: -------------------------------------------------------------------------------- 1 | ############################################################### 2 | ### Step 004 - Extraction of optimal placement information. ### 3 | ### Output: placement characteristics, i.e., number and ### 4 | ### location of VNF instances. ### 5 | ############################################################### 6 | 7 | import argparse 8 | from builtins import range 9 | import functools 10 | import glob 11 | import multiprocessing 12 | import random 13 | import re 14 | import subprocess 15 | 16 | # For each log CPLEX-produced log file that matches the provided pattern, 17 | # extract information regarding the number and location of placed VNF instances. 18 | # Detailed information regarding individual parameters can be found below (parser.add_argument(...)). 19 | def getPlacementInfoFromCPLEXLogs(logFilePath, suffix, vnfTypes): 20 | logFilePattern = 'cplex-full-%s-*.log' % (suffix) 21 | logFiles = glob.glob(logFilePath + '/' + logFilePattern) 22 | nFiles = len(logFiles) 23 | with open('%s/nvnfs-%s.txt' % (logFilePath, suffix), 'w') as fout1,\ 24 | open('%s/nodeinfo-%s.txt' % (logFilePath, suffix), 'w') as fout2,\ 25 | open('%s/routeinfo-%s.txt' % (logFilePath, suffix), 'w') as fout3: 26 | for i in range(0, nFiles): 27 | if i % 100 == 0: 28 | print('file %05d of %05d' % (i, nFiles - 1)) 29 | with open(logFiles[i]) as f: 30 | curlogfile = logFiles[i] 31 | curreqid = int(curlogfile.split('-')[-1].split('.')[0]) 32 | 33 | # Go through logfile once, extracting relevant information. 34 | vnflocs = [] 35 | routeinfo = [] 36 | for line in f.readlines(): 37 | if 'vnfloc' in line: 38 | vnflocs.append(line) 39 | if 'routeinfo' in line: 40 | routeinfo.append(line) 41 | 42 | ######################################## 43 | ### Aggregated placement information ### 44 | ######################################## 45 | 46 | if len(vnflocs) == 0: 47 | print('[ WARN ] no vnfloc tag in file %s - skipping.' % logFiles[i]) 48 | continue 49 | # Last line gives final number of deployed middleboxes. 50 | nMiddleBoxes = vnflocs[-1] 51 | nMiddleBoxes = int(nMiddleBoxes.split(' ')[3]) 52 | # Given the number of middleboxes, extract their location and type. 53 | middleboxLocations = [] 54 | for line in vnflocs[-(nMiddleBoxes + 1):-1]: 55 | (location, vnfType) = (int(line.split()[5]), line.split()[3][:-1]) 56 | middleboxLocations.append((location, vnfType)) 57 | # Aggregate to get number of vnfs per type. -- FIXME: make more compact / readable. 58 | nPerType = [(vnfType, [item[1] for item in middleboxLocations].count(vnfType)) for vnfType in vnfTypes] 59 | # FIXME: generalize for the case of an arbitrary number of types. 60 | fout1.write('%d,%d,%s\r\n' % (curreqid, nMiddleBoxes, ','.join([str(item[1]) for item in nPerType]))) 61 | 62 | # Number of instaces of a VNF per node. 63 | nodeinfo = [(item[0], item[1], middleboxLocations.count(item)) for item in set(middleboxLocations)] 64 | for line in nodeinfo: 65 | fout2.write('%d,%d,%s,%d\r\n' % (curreqid, line[0], line[1], line[2])) 66 | 67 | ###################################### 68 | ### Detailed placement information ### 69 | ###################################### 70 | 71 | if len(routeinfo) == 0: 72 | print('[ WARN ] no routeinfo tag in file %s - skipping.' % logFiles[i]) 73 | continue 74 | # Find index of last occurrence of " ... [routeinfo] start". 75 | startIdx = next(i for i in reversed(range(len(routeinfo))) if 'start' in routeinfo[i]) 76 | routeinfo = routeinfo[(startIdx + 1):] 77 | currentprefix = '%d,' % (curreqid) 78 | for line in routeinfo: 79 | currentinfo = re.sub(r'.*demand (\d+); mbox (\d+); node (\d+)', r'\1,\2,\3', line) 80 | fout3.write(currentprefix + currentinfo.strip() + '\r\n') 81 | 82 | 83 | if __name__ == '__main__': 84 | parser = argparse.ArgumentParser() 85 | 86 | parser.add_argument( 87 | "--logFilePath", 88 | help = "Folder in which to look for CPLEX log files and to which to write output placement information logs. [default is ../../solver/middlebox-placement/src/logs].", 89 | type = str, 90 | default = "../../solver/middlebox-placement/src/logs") 91 | 92 | parser.add_argument( 93 | "--suffix", 94 | help = "Suffix to add to output log files. The resulting file names have the following structure: cplex-full-$suffix-$requestID.log. [default is the empty string].", 95 | type = str, 96 | default = "") 97 | 98 | args = parser.parse_args() 99 | 100 | # FIXME: Make this a parameter as well and / or supply a config file with the vnf catalogue. 101 | vnfTypes = ['firewall', 'proxy', 'ids', 'nat', 'wano'] 102 | vnfTypes.sort() 103 | 104 | getPlacementInfoFromCPLEXLogs( 105 | logFilePath = args.logFilePath, 106 | suffix = args.suffix, 107 | vnfTypes = vnfTypes) 108 | -------------------------------------------------------------------------------- /code/full-pipeline/005_processSolverResults.R: -------------------------------------------------------------------------------- 1 | #################################################### 2 | ### Step 005 - Consolidation of solver results. ### 3 | ### Output: single file with information on all ### 4 | ### solver results for a given scenario. ### 5 | #################################################### 6 | 7 | source("common.R") 8 | 9 | ################# 10 | ### CLI Opts. ### 11 | ################# 12 | 13 | optlist <- list( 14 | make_option( 15 | "--requestsFile", type = "character", default = "requests.rds", 16 | help = "Name of RDS file containing the SFC request trace. [default is %default]."), 17 | make_option( 18 | "--nvnfsFile", type = "character", default = "../../solver/middlebox-placement/src/logs/nvnfs.txt", 19 | help = "Path to the txt output file containing placement information regarding the number of vnf instances. [default is %default]."), 20 | make_option( 21 | "--routeinfoFile", type = "character", default = "../../solver/middlebox-placement/src/logs/routeinfo.txt", 22 | help = "Path to the txt output file containing placement information w.r.t. routing. [default is %default]."), 23 | make_option( 24 | "--nodeinfoFile", type = "character", default = "../../solver/middlebox-placement/src/logs/nodeinfo.txt", 25 | help = "Path to the txt output file containing placement information regarding the status of individual nodes. [default is %default]."), 26 | make_option( 27 | "--solverResultsOutFile", type = "character", default = "solver-results.rds", 28 | help = "Name of output RDS file containing the aggregated solver results. [default is %default].")) 29 | 30 | optparser <- OptionParser(option_list = optlist) 31 | 32 | opt <- parse_args(optparser) 33 | 34 | ####################### 35 | ### General config. ### 36 | ####################### 37 | 38 | requestsFile <- opt$requestsFile 39 | nvnfsFile <- opt$nvnfsFile 40 | 41 | routeinfoFile <- opt$routeinfoFile 42 | nodeinfoFile <- opt$nodeinfoFile 43 | 44 | solverResultsOutFile <- opt$solverResultsOutFile 45 | 46 | ############# 47 | ### Main. ### 48 | ############# 49 | 50 | # Requests for matching. 51 | requests <- readRDS(requestsFile) 52 | requests$rid <- 1:nrow(requests) 53 | 54 | # Extract maximum chain length from requests. 55 | maxchainlength <- as.numeric(gsub("vnf", "", tail(names(requests[grepl("vnf", names(requests))]), n = 1))) 56 | 57 | # VNF catalogue for utilization / allocation information. 58 | vnfcat <- readRDS("vnfcat.rds") 59 | 60 | # Add column with all VNFs in each request. 61 | requests <- 62 | requests %>% 63 | unite(nfchain, sprintf("vnf%d", 1:maxchainlength), remove = F) %>% 64 | mutate(arrivaltime = as.numeric(arrivaltime), 65 | traffic = as.numeric(traffic), 66 | penalty = as.numeric(penalty)) 67 | 68 | # Add one column per VNF type and extract the amount of requested traffic. 69 | bwcolnames <- sprintf("bw%s", vnfcat$name) 70 | for (i in 1:nrow(vnfcat)) { 71 | requests <- 72 | requests %>% 73 | rowwise() %>% 74 | mutate(!!bwcolnames[i] := ifelse(grepl(vnfcat$name[i], nfchain), traffic, 0)) %>% 75 | ungroup() 76 | } 77 | 78 | # Returned by Python script that extracts VNF location data. 79 | nvnfs <- 80 | fread(nvnfsFile, col.names = c("rid", "nvnfs", sprintf("n%s", vnfcat$name))) %>% 81 | as_tibble() %>% 82 | arrange(rid) 83 | 84 | nvnfs <- 85 | nvnfs %>% 86 | # Include request information. 87 | inner_join(requests %>% select(rid, sfcid, arrivaltime, duration, traffic)) %>% 88 | mutate(ncpus = 0) 89 | 90 | for (i in 1:nrow(vnfcat)){ 91 | # Alternative: sweep(nvnfs[, sprintf("n%s", vnfcat$name)], MARGIN = 2, vnfcat$cpu, `*`) 92 | nvnfs <- 93 | nvnfs %>% 94 | # Multiply number of VNF instances with number of consumed cores. 95 | mutate(ncpus = ncpus + !!sym(sprintf("n%s", vnfcat$name[i])) * vnfcat$cpu[i]) %>% 96 | # Determine total amount of bandwidth that is allocated per VNF type. 97 | mutate(!!sprintf("bwalloc%s", vnfcat$name[i]) := !!sym(sprintf("n%s", vnfcat$name[i])) * vnfcat$capacity[i]) 98 | } 99 | 100 | # Determine number of active requests after each arrival and calculate corresponding statistics. 101 | nvnfs$nactive <- 0 102 | nvnfs$totalbw <- 0 103 | nvnfs[, sprintf("bw%s", vnfcat$name)] <- 0 104 | nvnfs[, sprintf("nactive%s", vnfcat$name)] <- 0 105 | nvnfs$reqlist <- 0 106 | 107 | # Column names for bandwidth quantile information. 108 | quantileCols <- expand.grid(q = seq(0, 1000, 125), nf = vnfcat$name) %>% mutate(colname = sprintf("q%03d%s", q, nf)) 109 | 110 | nvnfs[, quantileCols$colname] <- NA 111 | 112 | for (i in 1:nrow(nvnfs)) { 113 | 114 | # Using the current arrival time, determine all active requests for that time. 115 | curArr <- nvnfs$arrivaltime[i] 116 | curID <- nvnfs$rid[i] 117 | activereqs <- subset(requests, (rid <= curID) & (arrivaltime <= curArr) & (arrivaltime + duration >= curArr)) 118 | 119 | # Number of active SFC requests after the i-th arrival. 120 | nvnfs$nactive[i] <- nrow(activereqs) 121 | # Total requested bandwidth, as well as total requested bandwidth per VNF type. 122 | nvnfs$totalbw[i] <- sum(activereqs$traffic) 123 | 124 | # Add information on total requested bandwidth per VNF type as well as quantiles of active requests. 125 | for (j in 1:nrow(vnfcat)) { 126 | curNFName <- vnfcat$name[j] 127 | curColName <- sprintf("bw%s", curNFName) 128 | nvnfs[i, curColName] <- sum(activereqs[, curColName]) 129 | nvnfs[i, sprintf("nactive%s", curNFName)] <- sum(activereqs[, curColName] != 0) 130 | # Quantiles of per-request, per-type bandwidth demands. 131 | curQuantileColNames <- (quantileCols %>% filter(nf == curNFName))$colname 132 | curVec <- activereqs[[curColName]] 133 | curVec <- curVec[which(curVec != 0)] 134 | nvnfs[i, curQuantileColNames] <- as.numeric(quantile(curVec, seq(0, 1, .125))) / vnfcat$capacity[j] 135 | } 136 | 137 | # List of active requests for routeinfo matching. 138 | nvnfs$reqlist[i] <- list(activereqs$rid) 139 | } 140 | 141 | for (i in 1:nrow(vnfcat)) { 142 | nfutil <- nvnfs[, sprintf("bw%s", vnfcat$name[i])] / nvnfs[, sprintf("bwalloc%s", vnfcat$name[i])] 143 | ngt1 <- length(which(nfutil > 1)) 144 | if (ngt1 > 0) { 145 | cat(sprintf("WARNING: NF %s has %d utilization values > 1.\n", vnfcat$name[i], ngt1)) 146 | } 147 | } 148 | 149 | ### Part 2 - enrich solver results with node-level information. 150 | 151 | # Helper functions for later(using purrr::pmap instead of rowwise()). 152 | req2rid <- function(reqlist, req, ...) {return(reqlist[req])} 153 | nf2fname <- function(nf, nfchain, ...) {return(strsplit(nfchain, "_")[[1]][nf])} 154 | 155 | routeinfo <- 156 | fread(routeinfoFile, col.names = c("trafid", "req", "nf", "node")) %>% 157 | # Translate locally scoped request number to global request id. 158 | left_join(nvnfs, by = c("trafid" = "rid")) %>% 159 | select(trafid, req, nf, node, reqlist) %>% 160 | # Missing route info results in NULL entries. 161 | drop_na() %>% 162 | mutate(rid = pmap_dbl(., req2rid)) %>% 163 | select(trafid, rid, nf, node) %>% 164 | # Get traffic volume and middlebox data based on global request id. 165 | left_join(requests) %>% 166 | select(one_of("trafid", "rid", "nf", "node", c(sprintf("vnf%d", 1:maxchainlength), "traffic"))) %>% 167 | # Translate local nf id within the request to the actual nf name. 168 | unite(nfchain, sprintf("vnf%d", 1:maxchainlength), remove = F) %>% 169 | mutate(nf = pmap_chr(., nf2fname)) %>% 170 | select(trafid, rid, nf, node, traffic) %>% 171 | # Add arrivaltime. 172 | left_join(requests %>% select(rid, arrivaltime), by = c("trafid" = "rid")) 173 | 174 | nodetraffic <- 175 | routeinfo %>% 176 | # Summarize statistics per (trafid, node, nf) tuple. 177 | group_by(trafid, arrivaltime, node, nf) %>% 178 | summarise(totaltraffic = sum(traffic)) %>% 179 | ungroup() 180 | 181 | # Given the number of instantiated VNFs per node, determine the total capacity of each node per NF type. 182 | nodecapacity <- 183 | fread(nodeinfoFile, col.names = c("trafid", "node", "nf", "ninst")) %>% 184 | left_join(vnfcat, by = c("nf" = "name")) %>% 185 | rowwise() %>% 186 | mutate(capacity = ninst * capacity, usedcpu = ninst * cpu) %>% 187 | ungroup() %>% 188 | select(trafid, node, nf, capacity, usedcpu, ninst) 189 | 190 | remainingCapacityPerNode <- 191 | (nodecapacity %>% select(trafid, node, nf, capacity)) %>% 192 | left_join(nodetraffic %>% select(trafid, node, nf, totaltraffic), by = c("trafid", "node", "nf")) %>% 193 | mutate(remainingcapacity = capacity - totaltraffic) 194 | 195 | newfeatures <- 196 | remainingCapacityPerNode %>% 197 | # Calculate statistics regarding the remaining capacity per VNF type at each arrival event. 198 | group_by(trafid, nf) %>% 199 | summarise( 200 | minremcap = min(remainingcapacity), 201 | meanremcap = mean(remainingcapacity), 202 | maxremcap = max(remainingcapacity), 203 | tottraf = sum(totaltraffic)) %>% 204 | ungroup() %>% 205 | # Create separate rows for each entry. 206 | tidyr::gather(feature, value, minremcap:tottraf) %>% 207 | # Naming consistent with other nvnfs-columns. 208 | mutate(fname = sprintf("%s%s", feature, nf)) %>% 209 | # Create one column per feature. 210 | select(-one_of("feature", "nf")) %>% 211 | tidyr::spread(fname, value) 212 | 213 | nvnfs.extended <- 214 | nvnfs %>% 215 | left_join(newfeatures, by = c("rid" = "trafid")) 216 | 217 | naRows <- is.na(nvnfs.extended$tottraffirewall) 218 | 219 | if (!assertthat::are_equal(nvnfs$bwfirewall[!naRows], nvnfs.extended$tottraffirewall[!naRows])) { 220 | warning(sprintf("[ processSolverResults ] %s: equality check failed - proceed with caution!", solverResultsOutFile)) 221 | } else { 222 | cat(sprintf("[ processSolverResults ] %s: equality check passed.\n", solverResultsOutFile)) 223 | } 224 | 225 | nvnfs.extended <- 226 | nvnfs.extended %>% 227 | drop_na() 228 | 229 | cat(sprintf("[ processSolverResults ] %s: extending introduced %d NAs.\n", solverResultsOutFile, nrow(nvnfs) - nrow(nvnfs.extended))) 230 | 231 | saveRDS(nvnfs.extended, solverResultsOutFile) 232 | -------------------------------------------------------------------------------- /code/full-pipeline/006a_generateTrainingData.R: -------------------------------------------------------------------------------- 1 | ########################################################################### 2 | ### Step 006a - Generation of training data for deployment decisions. ### 3 | ### Output: labeled training data for a given scenario & configuration. ### 4 | ### Note: parallelism requires step 006b for merging slices. ### 5 | ########################################################################### 6 | 7 | source("common.R") 8 | 9 | ################# 10 | ### CLI Opts. ### 11 | ################# 12 | 13 | optlist <- list( 14 | make_option( 15 | "--requestsFile", type = "character", default = "requests.rds", 16 | help = "Name of RDS file containing the SFC request trace. [default is %default]."), 17 | make_option( 18 | "--solverResultsFile", type = "character", default = "solver-results.rds", 19 | help = "Name of RDS file containing the aggregated solver results. [default is %default]."), 20 | make_option( 21 | "--trainingDataOutFile", type = "character", default = "training.rds", 22 | help = "Name of output RDS file containing the training data [default is %default]."), 23 | make_option( 24 | "--aWidth", type = "integer", default = "600", 25 | help = "Width of aggregation window in seconds. [default is %default]."), 26 | make_option( 27 | "--tPred", type = "integer", default = "200", 28 | help = "Prediction time. [default is %default]."), 29 | make_option( 30 | "--predMode", type = "character", default = "tail", 31 | help = "Prediction mode, i.e., whether the goal is to predict the value at exactly t + tpred (tail) or use the maximum in [t; t + tpred] (max). [default is %default]."), 32 | make_option( 33 | "--nSlices", type = "integer", default = "1", 34 | help = "Number of slices to subdivide the work into for parallel processing. [default is %default]."), 35 | make_option( 36 | "--sliceID", type = "integer", default = "1", 37 | help = "Slice number to be handled by this worker (first is 1). [default is %default]."), 38 | make_option( 39 | "--seed", type = "integer", default = "42", 40 | help = "RNG seed for reproducibility. [default is %default].")) 41 | 42 | optparser <- OptionParser(option_list = optlist) 43 | 44 | opt <- parse_args(optparser) 45 | 46 | ################################################################## 47 | ### Import solver results from previous step + general config. ### 48 | ################################################################## 49 | 50 | requestsFile <- opt$requestsFile 51 | solverResultsFile <- opt$solverResultsFile 52 | trainingDataOutFile <- opt$trainingDataOutFile 53 | 54 | nvnfs <- 55 | readRDS(solverResultsFile) %>% 56 | arrange(rid) 57 | rngSeed <- opt$seed 58 | 59 | # Width of aggregation window, e.g., 10 minutes. 60 | awidth <- opt$aWidth 61 | # Prediction time, e.g., 200 seconds (five average iats). 62 | tpred <- opt$tPred 63 | # Prediction mode, i.e., whether to predict value at exactly t + tpred ("tail") or use the maximum in [t; t + tpred] ("max"). 64 | predMode <- opt$predMode 65 | 66 | ## Extract aggregated statistics from aggregation windows that end with the arrival of a request and whose width is equal to awidth. 67 | 68 | allowedEndtimes <- (min(nvnfs$arrivaltime) + awidth):(max(nvnfs$arrivaltime) - tpred - 1) 69 | 70 | vnfcat <- readRDS("vnfcat.rds") 71 | 72 | ### Extract more events / statistics from solver results (request departure times, VNF addition / removal times). 73 | reqs <- readRDS(requestsFile) %>% 74 | mutate( 75 | rid = 1:nrow(.), 76 | arrivaltime = as.numeric(arrivaltime), 77 | traffic = as.numeric(traffic), 78 | penalty = as.numeric(penalty)) 79 | 80 | departureData <- data.frame( 81 | time = nvnfs$arrivaltime + nvnfs$duration, 82 | rid = nvnfs$rid, 83 | sfcid = nvnfs$sfcid, 84 | type = "departure") %>% 85 | arrange(time) 86 | 87 | arrivalData <- data.frame( 88 | time = nvnfs$arrivaltime, 89 | rid = nvnfs$rid, 90 | sfcid = nvnfs$sfcid, 91 | type = "arrival") 92 | 93 | # Extract event (arrival / departure) data on a per-sfcid and per-VNF-type basis. 94 | eventData <- rbind(departureData, arrivalData) %>% 95 | inner_join(reqs %>% select(rid, starts_with("vnf"))) %>% 96 | tidyr::gather(pos, nf, starts_with("vnf")) %>% 97 | drop_na() %>% 98 | arrange(time) 99 | 100 | # Threshold to filter out endtimes that are too low to contain enough eventData for 101 | # later extraction of arrival / departure features (to avoid "timesincelast*" features to be NA). 102 | endtimeThresh <- eventData %>% head(1) %>% pull(time) 103 | nCombinations <- nrow(unique(eventData %>% select(sfcid, type, nf))) 104 | while (nrow(unique(eventData %>% filter(time < endtimeThresh) %>% select(sfcid, type, nf))) != nCombinations) { 105 | endtimeThresh <- endtimeThresh * 2 106 | } 107 | 108 | # Deployment / shutdown actions and changes in total / per-type bandwidth. 109 | churnData <- data.frame( 110 | time = nvnfs$arrivaltime[2:nrow(nvnfs)], 111 | changeN = diff(nvnfs$nvnfs), 112 | changeB = diff(nvnfs$totalbw), 113 | type = "global") 114 | 115 | for (i in 1:nrow(vnfcat)) { 116 | curVNFName <- vnfcat$name[i] 117 | churnDataCurVNF <- data.frame( 118 | time = nvnfs$arrivaltime[2:nrow(nvnfs)], 119 | changeN = diff(nvnfs[[sprintf("n%s", curVNFName)]]), 120 | changeB = diff(nvnfs[[sprintf("bw%s", curVNFName)]]), 121 | type = sprintf("%s", curVNFName) 122 | ) 123 | churnData[, sprintf("changeN%s", curVNFName)] <- churnDataCurVNF$changeN 124 | churnData[, sprintf("changeB%s", curVNFName)] <- churnDataCurVNF$changeB 125 | } 126 | 127 | ############# 128 | ### Main. ### 129 | ############# 130 | 131 | set.seed(rngSeed) 132 | 133 | # Place one endtime between each pair of consecutive arrivals. 134 | endtimes <- 135 | nvnfs %>% 136 | select(arrivaltime) %>% 137 | mutate(nextarrival = lead(arrivaltime), u = runif(nrow(nvnfs))) %>% 138 | mutate(endtime = arrivaltime + u * (nextarrival - arrivaltime)) %>% 139 | drop_na() %>% 140 | filter(endtime > endtimeThresh) %>% 141 | pull(endtime) 142 | 143 | nsamples <- length(endtimes) 144 | 145 | nslices <- opt$nSlices 146 | sliceID <- opt$sliceID 147 | 148 | indices <- c(0, round(nsamples / nslices * 1:nslices)) 149 | 150 | # Avoid using the same sample in different slices by going left-exclusive and right-inclusive. 151 | currentIndices <- c(indices[sliceID] + 1, indices[sliceID + 1]) 152 | 153 | aggstats <- data.frame(endtime = endtimes[currentIndices[1]:currentIndices[2]]) 154 | 155 | # Number of arrivals and departures. 156 | aggstats$narrivals <- 0 157 | aggstats$ndepartures <- 0 158 | 159 | # Arrival and departure rates. 160 | aggstats$meanarrrate <- 0 161 | aggstats$meandeprate <- 0 162 | 163 | # Change of total / per-type number of VNFs and requested bandwidth. 164 | aggstats$changeN <- 0 165 | aggstats$changeB <- 0 166 | for (i in 1:nrow(vnfcat)) { 167 | aggstats[, sprintf("changeN%s", vnfcat$name[i])] <- 0 168 | aggstats[, sprintf("changeB%s", vnfcat$name[i])] <- 0 169 | } 170 | 171 | # Last actions and time since last action / event per type. 172 | for (i in 1:nrow(vnfcat)) { 173 | for (k in 1:5) { 174 | aggstats[, sprintf("lastaction%s%d", vnfcat$name[i], k)] <- 0 175 | } 176 | aggstats[, sprintf("timesincelastaction%s", vnfcat$name[i])] <- 0 177 | aggstats[, sprintf("timesincelastarrival%s", vnfcat$name[i])] <- 0 178 | aggstats[, sprintf("timesincelastdeparture%s", vnfcat$name[i])] <- 0 179 | } 180 | 181 | # Time since last event - global and per SFC type. 182 | nSFCs <- length(unique(eventData$sfcid)) 183 | for (i in 1:nSFCs) { 184 | aggstats[, sprintf("timesincelastarrivalsfc%d", i)] <- 0 185 | aggstats[, sprintf("timesincelastdeparturesfc%d", i)] <- 0 186 | } 187 | 188 | aggstats$timesincelastarrival <- 0 189 | aggstats$timesincelastdeparture <- 0 190 | 191 | # Names of features in nvnfs that are used for "tail"-type features in aggstats. 192 | tailFeatureNames <- 193 | rbind( 194 | expand.grid( 195 | fname = c("n", "bw", "bwalloc", 196 | "bwtocap", "bwtoshut", "bwtocapn", 197 | "bwtoshutn", "util", "nactive", 198 | "maxremcap", "meanremcap", "minremcap"), 199 | vnfname = vnfcat$name) %>% 200 | transmute(featurename = paste(fname, vnfname, sep = "")), 201 | data.frame(featurename = c("nvnfs", "totalbw", "nactive")))$featurename 202 | # Same as "tail"-type features, but covering the most recent 5 values. 203 | tailFeatureNamesN <- 204 | (expand.grid(n = 1:5, fname = tailFeatureNames) %>% 205 | transmute(featurename = paste(fname, n, sep = "")))$featurename 206 | 207 | aggstats[, tailFeatureNamesN] <- 0 208 | 209 | # Columns from nvnfs that represent the quantiles of per-type, per-request bw demands. 210 | quantileCols <- expand.grid(q = seq(0, 1000, 125), nf = vnfcat$name) %>% mutate(colname = sprintf("q%03d%s", q, nf)) 211 | aggstats[, quantileCols$colname] <- 0 212 | 213 | # Arrival rate in first and second half of aggregation window. 214 | aggstats$rate1 <- 0 215 | aggstats$rate2 <- 0 216 | 217 | # Features in terms of statistics and fields. 218 | commonstats <- c( 219 | "mean", 220 | "median", 221 | "sd", 222 | "cv", 223 | "min", 224 | "max", 225 | "slope") 226 | 227 | fields <- c( 228 | "nvnfs", 229 | "totalbw", 230 | sprintf("n%s", vnfcat$name), 231 | sprintf("bw%s", vnfcat$name), 232 | sprintf("bwalloc%s", vnfcat$name), 233 | sprintf("util%s", vnfcat$name), 234 | sprintf("bwtocap%s", vnfcat$name), 235 | sprintf("bwtocapn%s", vnfcat$name), 236 | sprintf("bwtoshut%s", vnfcat$name), 237 | sprintf("bwtoshutn%s", vnfcat$name)) 238 | 239 | featurelist <- expand.grid(stat = commonstats, field = fields) %>% 240 | mutate( 241 | name = paste(stat, field, sep = "."), 242 | stat = as.character(stat), 243 | field = as.character(field)) 244 | 245 | utilfnames <- sprintf("util%s", vnfcat$name) 246 | # Capacity-related features. 247 | # Amount of bandwidth until reaching utilization 1. 248 | # Amount of bandwidth until a VNF can be shut down (not accounting for bin packing issues etc.). 249 | # Both features can be expressed in terms of absolute values or normalized. 250 | # Note, however, that due to above reasons, negative values and values > 1 are possible, e.g., when more instances are required than sum(requests) / capacity might suggest. 251 | bwtocapfnames <- sprintf("bwtocap%s", vnfcat$name) 252 | bwtocapnfnames <- sprintf("bwtocapn%s", vnfcat$name) 253 | bwtoshutfnames <- sprintf("bwtoshut%s", vnfcat$name) 254 | bwtoshutnfnames <- sprintf("bwtoshutn%s", vnfcat$name) 255 | 256 | for (i in 1:length(utilfnames)) { 257 | nvnfs[[utilfnames[i]]] <- nvnfs[[sprintf("bw%s", vnfcat$name)[i]]] / nvnfs[[sprintf("bwalloc%s", vnfcat$name)[i]]] 258 | nvnfs[[bwtocapfnames[i]]] <- nvnfs[[sprintf("bwalloc%s", vnfcat$name)[i]]] - nvnfs[[sprintf("bw%s", vnfcat$name)[i]]] 259 | nvnfs[[bwtoshutfnames[i]]] <- vnfcat$capacity[i] - nvnfs[[bwtocapfnames[i]]] 260 | nvnfs[[bwtocapnfnames[i]]] <- nvnfs[[bwtocapfnames[i]]] / vnfcat$capacity[i] 261 | nvnfs[[bwtoshutnfnames[i]]] <- nvnfs[[bwtoshutfnames[i]]] / vnfcat$capacity[i] 262 | } 263 | 264 | aggstats[, featurelist$name] <- 0 265 | 266 | # Deployment action regarding VNFs - per type. 267 | # -i: shut down i instances. 268 | # 0: keep current number of instances. 269 | # i: deploy i instances. 270 | aggstats[, sprintf("%saction", vnfcat$name)] <- 0 271 | 272 | getSlope <- function(vec1, vec2) { 273 | return(mean(vec2) - mean(vec1)) 274 | } 275 | 276 | # Coefficient of variation. 277 | cv <- function(vec) { 278 | return(sd(vec) / mean(vec)) 279 | } 280 | 281 | skippedRows <- c() 282 | 283 | nagg <- nrow(aggstats) 284 | 285 | for (i in 1:nagg) { 286 | 287 | if (i %% 100 == 0) { 288 | cat(sprintf("%05d/%05d\n", i, nagg)) 289 | } 290 | 291 | endtime <- aggstats$endtime[i] 292 | starttime <- endtime - awidth 293 | 294 | # Skip incomplete intervals. 295 | if ((starttime < 0) || (endtime + tpred > max(nvnfs$arrivaltime))) { 296 | warning("[ generateTrainingData ] Incomplete interval detected - skipping.") 297 | aggstats[i, featurelist$name] <- NA 298 | skippedRows <- c(skippedRows, i) 299 | next 300 | } 301 | 302 | # Extract relevant slice of arrivals. 303 | curslice <- 304 | nvnfs %>% 305 | filter( 306 | arrivaltime >= starttime, 307 | arrivaltime <= endtime) 308 | 309 | # Skip intervals with less than five arrival events (~99% have more) to allow assessing "tail"-features. 310 | if (nrow(curslice) < 5) { 311 | warning("[ generateTrainingData ] Interval with less than 5 arrivals detected - skipping.") 312 | aggstats[i, featurelist$name] <- NA 313 | skippedRows <- c(skippedRows, i) 314 | next 315 | } 316 | 317 | cursliceDep <- 318 | departureData %>% 319 | filter( 320 | time >= starttime, 321 | time <= endtime) 322 | 323 | cursliceChurn <- 324 | churnData %>% 325 | filter( 326 | time >= starttime, 327 | time <= endtime) 328 | 329 | firsthalf <- curslice %>% filter(arrivaltime <= starttime + awidth / 2) 330 | secondhalf <- curslice %>% filter(arrivaltime > starttime + awidth / 2) 331 | 332 | if (nrow(firsthalf) == 0 || nrow(secondhalf) == 0) { 333 | warning(sprintf("[ generateTrainingData ] [ i = %05d ] One of the halves does not contain an event - skipping.", i)) 334 | aggstats[i, featurelist$name] <- NA 335 | skippedRows <- c(skippedRows, i) 336 | next 337 | } 338 | 339 | # Extract number of events and arrival rates. 340 | 341 | aggstats$narrivals[i] <- nrow(curslice) 342 | aggstats$ndepartures[i] <- nrow(cursliceDep) 343 | aggstats$meanarrrate[i] <- nrow(curslice) / awidth 344 | aggstats$meandeprate[i] <- nrow(cursliceDep) / awidth 345 | aggstats$changeN[i] <- sum(cursliceChurn$changeN) 346 | aggstats$changeB[i] <- sum(cursliceChurn$changeB) 347 | for (j in 1:nrow(vnfcat)) { 348 | curVNFName <- vnfcat$name[j] 349 | aggstats[i, sprintf("changeN%s", curVNFName)] <- sum(cursliceChurn[[sprintf("changeN%s", curVNFName)]]) 350 | aggstats[i, sprintf("changeB%s", curVNFName)] <- sum(cursliceChurn[[sprintf("changeB%s", curVNFName)]]) 351 | lastactions <- tail((churnData %>% filter(time <= endtime, !!sym(sprintf("changeN%s", curVNFName)) != 0)), 5) 352 | if (nrow(lastactions) == 5) { 353 | for (k in 1:5) { 354 | aggstats[i, sprintf("lastaction%s%d", curVNFName, k)] <- sign(lastactions[[5 - k + 1, sprintf("changeN%s", curVNFName)]]) 355 | } 356 | # EXTENSION: Could normalize this /w awidth for robustness. 357 | aggstats[i, sprintf("timesincelastaction%s", curVNFName)] <- endtime - lastactions[5, ]$time 358 | } else { 359 | warning(sprintf("[ generateTrainingData ] [ i = %05d ] Did not find valid last actions for VNF %s.", i, curVNFName)) 360 | } 361 | } 362 | 363 | cursliceEvents <- 364 | eventData %>% 365 | filter(time <= endtime) 366 | 367 | if (length(unique(cursliceEvents$type)) != 2) { 368 | warning(sprintf("[ generateTrainingData ] [ i = %05d ] Event data does not contain at least one arrival and one departure - skipping.", i)) 369 | aggstats[i, featurelist$name] <- NA 370 | skippedRows <- c(skippedRows, i) 371 | next 372 | } 373 | 374 | for (j in 1:nrow(vnfcat)) { 375 | curVNFName <- vnfcat$name[j] 376 | aggstats[i, sprintf("timesincelastarrival%s", curVNFName)] <- endtime - cursliceEvents %>% filter(type == "arrival", nf == curVNFName) %>% tail(1) %>% pull(time) 377 | aggstats[i, sprintf("timesincelastdeparture%s", curVNFName)] <- endtime - cursliceEvents %>% filter(type == "departure", nf == curVNFName) %>% tail(1) %>% pull(time) 378 | } 379 | 380 | for (j in 1:nSFCs) { 381 | aggstats[i, sprintf("timesincelastarrivalsfc%d", j)] <- endtime - cursliceEvents %>% filter(type == "arrival", sfcid == j) %>% tail(1) %>% pull(time) 382 | aggstats[i, sprintf("timesincelastdeparturesfc%d", j)] <- endtime - cursliceEvents %>% filter(type == "departure", sfcid == j) %>% tail(1) %>% pull(time) 383 | } 384 | 385 | aggstats$timesincelastarrival[i] <- endtime - (cursliceEvents %>% filter(type == "arrival") %>% tail(1))$time 386 | aggstats$timesincelastdeparture[i] <- endtime - (cursliceEvents %>% filter(type == "departure") %>% tail(1))$time 387 | 388 | aggstats$rate1[i] <- nrow(firsthalf) / (awidth / 2) 389 | aggstats$rate2[i] <- nrow(secondhalf) / (awidth / 2) 390 | 391 | ### Extraction of "tail"-type features. 392 | cursliceTail <- curslice %>% tail(n = 5) 393 | for (j in 1:length(tailFeatureNames)) { 394 | for (k in 1:5) { 395 | aggstats[i, sprintf("%s%d", tailFeatureNames[j], 5 - k + 1)] <- cursliceTail[k, tailFeatureNames[j]] 396 | } 397 | } 398 | 399 | ### Extraction of bw quantile data. 400 | 401 | aggstats[i, quantileCols$colname] <- (curslice %>% tail(n = 1))[, quantileCols$colname] 402 | 403 | ### Statistics calculation. 404 | 405 | for (j in 1:nrow(featurelist)) { 406 | curfieldname <- featurelist$field[j] 407 | curstatname <- featurelist$stat[j] 408 | curfeaturename <- featurelist$name[j] 409 | 410 | if (curstatname != "slope") { 411 | aggstats[i, curfeaturename] <- get(curstatname)(curslice[[curfieldname]]) 412 | } else { 413 | aggstats[i, curfeaturename] <- getSlope(secondhalf[[curfieldname]], firsthalf[[curfieldname]]) 414 | aggstats[i, gsub("slope", "slopei", curfeaturename)] <- discretizeSlope(getSlope(secondhalf[[curfieldname]], firsthalf[[curfieldname]])) 415 | } 416 | } 417 | 418 | ### Action identification. 419 | 420 | for (j in 1:nrow(vnfcat)) { 421 | currentnnf <- (nvnfs %>% filter(arrivaltime <= endtime) %>% tail(n = 1))[[sprintf("n%s", vnfcat$name[j])]] 422 | 423 | futurennf <- switch (predMode, 424 | # Predict amount of VNFs that are needed to serve all demands that are active at t + p. 425 | "tail" = (nvnfs %>% filter(arrivaltime <= endtime + tpred) %>% tail(n = 1))[[sprintf("n%s", vnfcat$name[j])]], 426 | # Determine maximum number of VNFs that are needed to serve all demands until t + p. 427 | "max" = max((nvnfs %>% filter(arrivaltime > endtime, arrivaltime <= endtime + tpred))[[sprintf("n%s", vnfcat$name[j])]])) 428 | 429 | aggstats[i, sprintf("%saction", vnfcat$name[j])] <- futurennf - currentnnf 430 | } 431 | 432 | } 433 | 434 | aggstats$meangrowthrate <- aggstats$meanarrrate - aggstats$meandeprate 435 | 436 | aggstats <- 437 | aggstats %>% 438 | filter(endtime >= awidth, !is.na(mean.totalbw)) 439 | 440 | for (j in 1:nrow(vnfcat)) { 441 | # Discretize actions based on their sign. 442 | aggstats[, sprintf("%sactionD", vnfcat$name[j])] <- discretizeSlope(aggstats[[sprintf("%saction", vnfcat$name[j])]]) 443 | } 444 | 445 | table(aggstats$firewallaction) / nrow(aggstats) 446 | table(aggstats$firewallactionD) / nrow(aggstats) 447 | 448 | aggstats$rateslope <- aggstats$rate2 - aggstats$rate1 449 | aggstats$rateslopei <- discretizeSlope(aggstats$rateslope) 450 | 451 | cat(sprintf("%d total, %d w/o duplicates.\n", nrow(aggstats), nrow(unique(aggstats[,names(aggstats)[-1]])))) 452 | 453 | saveRDS(aggstats, file = trainingDataOutFile) 454 | -------------------------------------------------------------------------------- /code/full-pipeline/006b_mergeTrainingData.R: -------------------------------------------------------------------------------- 1 | # This is only needed in the case of using 006a with nSlices > 1. 2 | # To be used in conjunction with 006a_generateTrainingData.R in order to combine the individual slices into one rds file. 3 | 4 | source("common.R") 5 | 6 | ################# 7 | ### CLI Opts. ### 8 | ################# 9 | 10 | optlist <- list( 11 | make_option( 12 | "--trainingDataFilePattern", type = "character", default = "training-slice-*", 13 | help = "File pattern of rds files to join [default is %default]."), 14 | make_option( 15 | "--trainingDataOutFile", type = "character", default = "training.rds", 16 | help = "Name of output RDS file containing the entire training data [default is %default].")) 17 | 18 | optparser <- OptionParser(option_list = optlist) 19 | 20 | opt <- parse_args(optparser) 21 | 22 | ############# 23 | ### Main. ### 24 | ############# 25 | 26 | allfiles <- list.files(path = ".", pattern = opt$trainingDataFilePattern, recursive = T) 27 | 28 | trainingdata <- data.frame() 29 | 30 | for (i in 1:length(allfiles)) { 31 | trainingdata <- rbind(trainingdata, readRDS(allfiles[i])) 32 | } 33 | 34 | saveRDS(trainingdata, opt$trainingDataOutFile) 35 | -------------------------------------------------------------------------------- /code/full-pipeline/007_trainAutoMLModel.R: -------------------------------------------------------------------------------- 1 | ####################################################### 2 | ### Step 007 - Training an AutoML model via H2O. ### 3 | ### Output: AutoML object containing performance ### 4 | ### indicators and model parameters. ### 5 | ####################################################### 6 | 7 | source("common.R") 8 | library(h2o) 9 | 10 | # Either create a local H2O cluster or connect to a remote one. 11 | useRemoteCluster <- F 12 | remoteClusterIP <- "1.2.3.4" 13 | remoteClusterPort <- 1234 14 | 15 | if (useRemoteCluster) { 16 | h2o.connect(ip = remoteClusterIP, port = remoteClusterPort) 17 | } else { 18 | h2o.init() 19 | } 20 | 21 | # VNF for which deployment actions are to be predicted. 22 | targetVNF <- "firewall" 23 | 24 | # The name of the network topology is used to derive corresponding training data files. 25 | topo <- "inet2" 26 | 27 | # Example for combining three weeks into one labeled data set. 28 | fulldataset <- createDataset( 29 | c(sprintf("data/training-wk1_sfccat_norm_%s-p60.rds", topo), 30 | sprintf("data/training-wk2_sfccat_norm_%s-p60.rds", topo), 31 | sprintf("data/training-wk3_sfccat_norm_%s-p60.rds", topo) 32 | ), 33 | extractEndtimes = T, 34 | saveTimeInfo = T, 35 | targetVNF = targetVNF) 36 | 37 | printActionInfo(fulldataset) 38 | 39 | # Add weight column that is used to counteract class imbalance during model training. 40 | fulldataset.weighted <- generateWeightedDataSet(fulldataset) 41 | 42 | # Identifiers for h2o objects representing training / testing sets. 43 | trainName <- sprintf("a-train-%s-%s", topo, targetVNF) 44 | testName <- sprintf("a-test-%s-%s", topo, targetVNF) 45 | 46 | # Upload data set to h2o cluster. 47 | fulldataset.weighted.h2o <- h2o.splitFrame( 48 | data = as.h2o(fulldataset.weighted), 49 | ratios = .75, 50 | destination_frames = c(trainName, testName), 51 | seed = 191) 52 | 53 | # Run AutoML for 10 hours. 54 | amlResult <- h2o.automl( 55 | y = "action", 56 | weights_column = "weight", 57 | training_frame = trainName, 58 | validation_frame = testName, 59 | # This is only used to determine the order of results that are reported in the leaderboard. 60 | leaderboard_frame = testName, 61 | seed = 42, 62 | max_runtime_secs = 36000, 63 | max_models = 0, 64 | nfolds = 9, 65 | balance_classes = F, 66 | keep_cross_validation_predictions = T, 67 | keep_cross_validation_models = T, 68 | keep_cross_validation_fold_assignment = F, 69 | project_name = sprintf("aml-%s-%s", topo, targetVNF), 70 | export_checkpoints_dir = sprintf("data/aml-%s-%s-checkpoints", topo, targetVNF)) 71 | 72 | saveRDS(amlResult, sprintf("data/aml-%s-%s.rds", topo, targetVNF)) 73 | -------------------------------------------------------------------------------- /code/full-pipeline/abilene-preproc.R: -------------------------------------------------------------------------------- 1 | ################################################################################################### 2 | ### Preprocess the traffic matrix data that is used for modulating the request arrival process. ### 3 | ################################################################################################### 4 | 5 | source("common.R") 6 | 7 | # From http://www.cs.utexas.edu/~yzhang/research/AbileneTM. 8 | # 1 file = 1 week in 5min intervals. 9 | # 1 line = 1 TM with 5 different types of data: x_1,1 x_1,2 .. x_1,5 .. x_144,5. 10 | abilene.prefix <- "../../abilene/X" 11 | abilene.nfiles <- 24 12 | abilene.df <- data.frame() 13 | 14 | # Here, we use the first type of TM data, i.e., x_*,1 in the above notation. 15 | # Summing over all entries in a given TM, we get the total traffic volume for the corresponding 5-minute interval. 16 | for (i in 1:abilene.nfiles) { 17 | abilene.curweek <- fread(paste(abilene.prefix, sprintf("%02d", i), sep = ""), select = seq(1, 720, 5)) %>% 18 | rowSums() 19 | abilene.df <- rbind(abilene.df, 20 | data.frame(minute = seq(0, 5 * (length(abilene.curweek) - 1), 5), 21 | totalvolume = abilene.curweek, 22 | week = i)) 23 | } 24 | 25 | # Overview plot. 26 | ggplot(abilene.df, aes(x = minute, y = totalvolume)) + 27 | geom_point() + 28 | facet_wrap(week ~ ., scales = "free") 29 | 30 | # Using only week 1, normalize rate in order to scale it according to our needs later on. 31 | # Apply moving average in order to smooth the curve. 32 | abilene.week.1.normalized <- 33 | abilene.df %>% 34 | filter(week == 1) %>% 35 | mutate(normvolume = totalvolume / max(totalvolume)) %>% 36 | mutate(smoothvolume = roll_mean(normvolume, 30, align = "center", fill = c(first(normvolume), 0, last(normvolume)))) %>% 37 | mutate(smoothvolume = smoothvolume / max(smoothvolume)) 38 | 39 | ggplot(abilene.week.1.normalized, aes(x = minute / 60, y = smoothvolume)) + 40 | geom_point() + 41 | xlab("Hour") + 42 | ylab("Normalized Traffic Volume") + 43 | theme_bw(base_size = 22) 44 | 45 | # Save data to csv for later use in SFCR trace generation. 46 | write.table( 47 | abilene.week.1.normalized %>% 48 | select(minute, smoothvolume), 49 | file = "abilene-week1.csv", 50 | sep = ",", 51 | col.names = F, 52 | row.names = F) 53 | -------------------------------------------------------------------------------- /code/full-pipeline/abilene-week1.csv: -------------------------------------------------------------------------------- 1 | 0,0.619123161533192 2 | 5,0.619123161533192 3 | 10,0.619123161533192 4 | 15,0.619123161533192 5 | 20,0.619123161533192 6 | 25,0.619123161533192 7 | 30,0.619123161533192 8 | 35,0.619123161533192 9 | 40,0.619123161533192 10 | 45,0.619123161533192 11 | 50,0.619123161533192 12 | 55,0.619123161533192 13 | 60,0.619123161533192 14 | 65,0.619123161533192 15 | 70,0.61925511650225 16 | 75,0.620152075982564 17 | 80,0.621554110553964 18 | 85,0.623014966360124 19 | 90,0.624031082748987 20 | 95,0.62590349329394 21 | 100,0.626871649671515 22 | 105,0.628158988839785 23 | 110,0.629029002469391 24 | 115,0.630524170181338 25 | 120,0.633023697257066 26 | 125,0.636246027415183 27 | 130,0.638606904937884 28 | 135,0.641526402929076 29 | 140,0.642445179120753 30 | 145,0.644234922265216 31 | 150,0.645700731297525 32 | 155,0.645955368534204 33 | 160,0.644888846363254 34 | 165,0.645767719599653 35 | 170,0.647433306925201 36 | 175,0.648342405447664 37 | 180,0.649070249215051 38 | 185,0.649922613351991 39 | 190,0.652093912524074 40 | 195,0.653375773278701 41 | 200,0.6538897530997 42 | 205,0.654146878931605 43 | 210,0.654932918635557 44 | 215,0.655134349307374 45 | 220,0.656033343744432 46 | 225,0.655880285146851 47 | 230,0.655096868719686 48 | 235,0.65395177794922 49 | 240,0.653907026584472 50 | 245,0.653134452502553 51 | 250,0.653298783608256 52 | 255,0.652795947840194 53 | 260,0.652866510515997 54 | 265,0.653119539990419 55 | 270,0.652199460821163 56 | 275,0.65046475863301 57 | 280,0.64863403412149 58 | 285,0.646537363973013 59 | 290,0.645676574446539 60 | 295,0.644601404564648 61 | 300,0.643338827020239 62 | 305,0.642922392360004 63 | 310,0.643725697820059 64 | 315,0.643267950183995 65 | 320,0.643020263373761 66 | 325,0.642826485966672 67 | 330,0.642891127794204 68 | 335,0.641733785762038 69 | 340,0.640135665897997 70 | 345,0.63973450967613 71 | 350,0.639374487548507 72 | 355,0.639830430549403 73 | 360,0.63883894906681 74 | 365,0.638884366780387 75 | 370,0.638833276911542 76 | 375,0.639214045465619 77 | 380,0.640521766165805 78 | 385,0.640688127286972 79 | 390,0.640272523974505 80 | 395,0.639763740316881 81 | 400,0.638585946977578 82 | 405,0.637750806767809 83 | 410,0.637659125422444 84 | 415,0.637121661886762 85 | 420,0.636454116362515 86 | 425,0.636268593897942 87 | 430,0.636254790531263 88 | 435,0.635991547665974 89 | 440,0.635783532693045 90 | 445,0.635046105629466 91 | 450,0.634385177311999 92 | 455,0.634095057647225 93 | 460,0.633365110613968 94 | 465,0.632981889664521 95 | 470,0.632061606943481 96 | 475,0.632065082346752 97 | 480,0.631067098573818 98 | 485,0.630325587714106 99 | 490,0.630574762465077 100 | 495,0.630908212107295 101 | 500,0.630510385638583 102 | 505,0.629953338886385 103 | 510,0.629732307350623 104 | 515,0.62867661266488 105 | 520,0.626661395013562 106 | 525,0.625444069767493 107 | 530,0.62290179920627 108 | 535,0.622160140216751 109 | 540,0.621101126637809 110 | 545,0.621028416061715 111 | 550,0.622415690196572 112 | 555,0.622896386658262 113 | 560,0.622660181555112 114 | 565,0.622457152098603 115 | 570,0.622897215254841 116 | 575,0.622387953650533 117 | 580,0.621779064036308 118 | 585,0.621235400841072 119 | 590,0.619701957628824 120 | 595,0.619348492706722 121 | 600,0.618249773896109 122 | 605,0.617044935989094 123 | 610,0.616198290252736 124 | 615,0.615635517986684 125 | 620,0.61564488093436 126 | 625,0.614912422952466 127 | 630,0.614683328478219 128 | 635,0.613644946825589 129 | 640,0.611445801041808 130 | 645,0.611251748262423 131 | 650,0.60979022945592 132 | 655,0.60850150767039 133 | 660,0.607663506332688 134 | 665,0.606327985320417 135 | 670,0.60773074537311 136 | 675,0.609184051174629 137 | 680,0.609010027756871 138 | 685,0.608013874049502 139 | 690,0.607035646437891 140 | 695,0.604849767774239 141 | 700,0.601394347727687 142 | 705,0.599386264054178 143 | 710,0.598784437610846 144 | 715,0.59850638261562 145 | 720,0.596968225259165 146 | 725,0.59659856242238 147 | 730,0.597711680630284 148 | 735,0.59807706153103 149 | 740,0.599998357844261 150 | 745,0.601132385233167 151 | 750,0.603554160848704 152 | 755,0.605649082507988 153 | 760,0.607208707826405 154 | 765,0.608930803650381 155 | 770,0.610235342057522 156 | 775,0.612282139991674 157 | 780,0.615441361114616 158 | 785,0.62067446760005 159 | 790,0.625956337896452 160 | 795,0.629144093337836 161 | 800,0.634414885428836 162 | 805,0.639277890950642 163 | 810,0.643767562065377 164 | 815,0.648048556075638 165 | 820,0.649754673482752 166 | 825,0.651916655880074 167 | 830,0.656752257858306 168 | 835,0.662671351503978 169 | 840,0.668229990325161 170 | 845,0.67448176789562 171 | 850,0.682339019511646 172 | 855,0.690232757902642 173 | 860,0.695761909317504 174 | 865,0.701418727810508 175 | 870,0.707431463675286 176 | 875,0.7127429734174 177 | 880,0.716899441736728 178 | 885,0.722754038462888 179 | 890,0.727315428277692 180 | 895,0.732497585874617 181 | 900,0.737335789771722 182 | 905,0.742194261659897 183 | 910,0.747608364505844 184 | 915,0.75338538242702 185 | 920,0.759169215369082 186 | 925,0.764425578777109 187 | 930,0.769114820651316 188 | 935,0.774317888388538 189 | 940,0.780157881966523 190 | 945,0.785748409511405 191 | 950,0.791484464093051 192 | 955,0.798061774469857 193 | 960,0.805361313944152 194 | 965,0.814413238855629 195 | 970,0.824286106791975 196 | 975,0.832020547467996 197 | 980,0.839315784680033 198 | 985,0.846374935289146 199 | 990,0.85371890130689 200 | 995,0.860108751730489 201 | 1000,0.86494894234803 202 | 1005,0.869733070123558 203 | 1010,0.8755294588043 204 | 1015,0.880641165564875 205 | 1020,0.886725370097324 206 | 1025,0.892429102932608 207 | 1030,0.899003576779282 208 | 1035,0.904519420783576 209 | 1040,0.911995615187883 210 | 1045,0.918337488948259 211 | 1050,0.9228014585461 212 | 1055,0.927500899928529 213 | 1060,0.931778352687982 214 | 1065,0.935579722023756 215 | 1070,0.938505817691906 216 | 1075,0.941249180945001 217 | 1080,0.944274802141278 218 | 1085,0.946098525962855 219 | 1090,0.949098730697994 220 | 1095,0.951453563116422 221 | 1100,0.954226241817479 222 | 1105,0.955824583604729 223 | 1110,0.957504249108155 224 | 1115,0.959119302232631 225 | 1120,0.960777497638304 226 | 1125,0.96597259986895 227 | 1130,0.972290001658604 228 | 1135,0.977419385550433 229 | 1140,0.979664696321396 230 | 1145,0.981109110008652 231 | 1150,0.9829777789909 232 | 1155,0.983332739347109 233 | 1160,0.984129170273715 234 | 1165,0.984894027288662 235 | 1170,0.985473626441876 236 | 1175,0.98611081393519 237 | 1180,0.98531702867498 238 | 1185,0.98411092936592 239 | 1190,0.981318000221673 240 | 1195,0.979655362804203 241 | 1200,0.980334757445961 242 | 1205,0.979967678953019 243 | 1210,0.980671505437961 244 | 1215,0.983347936372003 245 | 1220,0.984945875662117 246 | 1225,0.98647682257762 247 | 1230,0.987187770352321 248 | 1235,0.988298595697614 249 | 1240,0.98886445582939 250 | 1245,0.990853426133351 251 | 1250,0.993094251353141 252 | 1255,0.992344965615289 253 | 1260,0.990109024470874 254 | 1265,0.987263143029546 255 | 1270,0.984971985504325 256 | 1275,0.980577256889384 257 | 1280,0.975315079377409 258 | 1285,0.969117743637561 259 | 1290,0.966832047439652 260 | 1295,0.965573231901816 261 | 1300,0.964014902873054 262 | 1305,0.963410699854947 263 | 1310,0.961173150857776 264 | 1315,0.964264522447842 265 | 1320,0.965294455579231 266 | 1325,0.964947266843176 267 | 1330,0.964290295414201 268 | 1335,0.964658681319203 269 | 1340,0.973919066932423 270 | 1345,0.983330952268871 271 | 1350,0.988032997715014 272 | 1355,0.986508122393028 273 | 1360,0.983278992128364 274 | 1365,0.976895272421695 275 | 1370,0.97368058205291 276 | 1375,0.971049425870168 277 | 1380,0.974499048600454 278 | 1385,0.970640423384046 279 | 1390,0.966002650264466 280 | 1395,0.960068994222607 281 | 1400,0.953225850645593 282 | 1405,0.949837904467923 283 | 1410,0.947382513124109 284 | 1415,0.94397201561332 285 | 1420,0.938918033829198 286 | 1425,0.933906848852958 287 | 1430,0.929607944720127 288 | 1435,0.926448025148134 289 | 1440,0.929135452150108 290 | 1445,0.925998861818967 291 | 1450,0.926585602968257 292 | 1455,0.931877360404334 293 | 1460,0.948244996984793 294 | 1465,0.950416268264888 295 | 1470,0.945667625225196 296 | 1475,0.945750939381344 297 | 1480,0.943508635877128 298 | 1485,0.941481602946112 299 | 1490,0.930777510827902 300 | 1495,0.919238645675051 301 | 1500,0.910837710220822 302 | 1505,0.910099333110151 303 | 1510,0.910757708016551 304 | 1515,0.912457411623869 305 | 1520,0.912820487082555 306 | 1525,0.912458528280794 307 | 1530,0.90579271387419 308 | 1535,0.906252511146039 309 | 1540,0.90561314404136 310 | 1545,0.905332574350813 311 | 1550,0.90446953141957 312 | 1555,0.903028460698417 313 | 1560,0.90260251744039 314 | 1565,0.902890130953007 315 | 1570,0.904759288216998 316 | 1575,0.905433848259533 317 | 1580,0.905183444016689 318 | 1585,0.905442481228084 319 | 1590,0.899077122160231 320 | 1595,0.898570436293249 321 | 1600,0.893695786572261 322 | 1605,0.884511505181034 323 | 1610,0.864972708711181 324 | 1615,0.855113908708154 325 | 1620,0.854775669128283 326 | 1625,0.850799718290637 327 | 1630,0.850983681835607 328 | 1635,0.849401134271307 329 | 1640,0.848339780516544 330 | 1645,0.848788019229017 331 | 1650,0.848559731595236 332 | 1655,0.847406643118422 333 | 1660,0.84489860357441 334 | 1665,0.841512048828663 335 | 1670,0.839530497531119 336 | 1675,0.838279351828494 337 | 1680,0.836823634338341 338 | 1685,0.834514355183847 339 | 1690,0.833239294535767 340 | 1695,0.83203001063497 341 | 1700,0.831666670700145 342 | 1705,0.831988026785331 343 | 1710,0.830752721234223 344 | 1715,0.827943095648991 345 | 1720,0.824640874619256 346 | 1725,0.822860118867074 347 | 1730,0.820346905911295 348 | 1735,0.817993829819328 349 | 1740,0.815567463971688 350 | 1745,0.813729556847854 351 | 1750,0.81272069157953 352 | 1755,0.811183775244602 353 | 1760,0.810200990216055 354 | 1765,0.809320071036256 355 | 1770,0.807712255258275 356 | 1775,0.806194898103158 357 | 1780,0.803327384551625 358 | 1785,0.800574600666386 359 | 1790,0.797598425413042 360 | 1795,0.793816911027994 361 | 1800,0.791258617813686 362 | 1805,0.788159254942747 363 | 1810,0.786669146196725 364 | 1815,0.785916477528131 365 | 1820,0.785336078521023 366 | 1825,0.784108572156258 367 | 1830,0.782345350075305 368 | 1835,0.781214199181171 369 | 1840,0.778438098100581 370 | 1845,0.775896523409169 371 | 1850,0.773538079313115 372 | 1855,0.771703620044463 373 | 1860,0.769561193749965 374 | 1865,0.769295673841402 375 | 1870,0.769259055081761 376 | 1875,0.768828572778225 377 | 1880,0.769297763948625 378 | 1885,0.768936750451776 379 | 1890,0.768451584556898 380 | 1895,0.767071361321183 381 | 1900,0.766604829663624 382 | 1905,0.76649654290181 383 | 1910,0.764125058351019 384 | 1915,0.761900146849901 385 | 1920,0.75933400188706 386 | 1925,0.757566207710131 387 | 1930,0.757533827446703 388 | 1935,0.757921791174442 389 | 1940,0.758402587095234 390 | 1945,0.758718897355043 391 | 1950,0.758831514434002 392 | 1955,0.759615023206997 393 | 1960,0.75899271271376 394 | 1965,0.758031899784455 395 | 1970,0.755365583162277 396 | 1975,0.753893280563792 397 | 1980,0.753792246497212 398 | 1985,0.753061024396465 399 | 1990,0.754493606450738 400 | 1995,0.754781954150714 401 | 2000,0.754607325652381 402 | 2005,0.754756606957999 403 | 2010,0.755233700086131 404 | 2015,0.753748088375686 405 | 2020,0.751686992168538 406 | 2025,0.749654588889495 407 | 2030,0.746439819451177 408 | 2035,0.74385979034168 409 | 2040,0.740689258791267 410 | 2045,0.738775835548283 411 | 2050,0.736575389538113 412 | 2055,0.734228297028766 413 | 2060,0.734067740572435 414 | 2065,0.733446933947871 415 | 2070,0.732541557060092 416 | 2075,0.730296121001431 417 | 2080,0.726550255038913 418 | 2085,0.723656167662582 419 | 2090,0.719734661198587 420 | 2095,0.716203663420731 421 | 2100,0.712934209196476 422 | 2105,0.710075897050767 423 | 2110,0.708679954516751 424 | 2115,0.707263152051636 425 | 2120,0.705291747048528 426 | 2125,0.702730215938196 427 | 2130,0.699337105290124 428 | 2135,0.696704333173008 429 | 2140,0.692786160389852 430 | 2145,0.690227601884906 431 | 2150,0.687640304459939 432 | 2155,0.684527571226885 433 | 2160,0.682614036289249 434 | 2165,0.682780615134443 435 | 2170,0.683529304624485 436 | 2175,0.68351050480903 437 | 2180,0.684699854228019 438 | 2185,0.686418718128948 439 | 2190,0.688366152366424 440 | 2195,0.689557429337188 441 | 2200,0.690098803868996 442 | 2205,0.691420930011849 443 | 2210,0.692565995985415 444 | 2215,0.694048463973759 445 | 2220,0.695795843097706 446 | 2225,0.698527154224503 447 | 2230,0.702453773556007 448 | 2235,0.705429002961441 449 | 2240,0.710791408161079 450 | 2245,0.715606242952241 451 | 2250,0.720108045670056 452 | 2255,0.725088819039578 453 | 2260,0.727518235738591 454 | 2265,0.731511120099122 455 | 2270,0.7363888040648 456 | 2275,0.741083274827051 457 | 2280,0.745892252608386 458 | 2285,0.751062277697235 459 | 2290,0.757758501828165 460 | 2295,0.763790537355994 461 | 2300,0.770197149018318 462 | 2305,0.775218026650512 463 | 2310,0.780055455888821 464 | 2315,0.784685076420975 465 | 2320,0.789194889279787 466 | 2325,0.794146509041849 467 | 2330,0.798085040641512 468 | 2335,0.803366525898157 469 | 2340,0.807810239704277 470 | 2345,0.81363881850871 471 | 2350,0.819581615348785 472 | 2355,0.824617172438448 473 | 2360,0.82922134769438 474 | 2365,0.833661968331497 475 | 2370,0.839936499266467 476 | 2375,0.845895177399194 477 | 2380,0.850756449907245 478 | 2385,0.85517028549341 479 | 2390,0.857105709234422 480 | 2395,0.860132551053607 481 | 2400,0.863079528807166 482 | 2405,0.865139546186238 483 | 2410,0.869939396427554 484 | 2415,0.872665191927257 485 | 2420,0.876013328509073 486 | 2425,0.879313209851685 487 | 2430,0.883176002087692 488 | 2435,0.886228601587846 489 | 2440,0.887296820826094 490 | 2445,0.889030321812561 491 | 2450,0.891461937489862 492 | 2455,0.895473767080665 493 | 2460,0.897688040693317 494 | 2465,0.899957801534091 495 | 2470,0.902092945141776 496 | 2475,0.904959161743006 497 | 2480,0.908780392237782 498 | 2485,0.910578259878584 499 | 2490,0.912899018966369 500 | 2495,0.914583902805582 501 | 2500,0.916579605802909 502 | 2505,0.918169728069202 503 | 2510,0.919360450358639 504 | 2515,0.919196856571581 505 | 2520,0.91907165685745 506 | 2525,0.918889810958742 507 | 2530,0.919986249268995 508 | 2535,0.921879421154742 509 | 2540,0.925996327811023 510 | 2545,0.928073171860058 511 | 2550,0.931006541943354 512 | 2555,0.934394336316932 513 | 2560,0.935637078132738 514 | 2565,0.936873544957092 515 | 2570,0.939442053915258 516 | 2575,0.942510519648925 517 | 2580,0.94428177815197 518 | 2585,0.944818711504577 519 | 2590,0.947220490519355 520 | 2595,0.949537438142516 521 | 2600,0.951350067251558 522 | 2605,0.951221348621929 523 | 2610,0.952521682180622 524 | 2615,0.95271048936573 525 | 2620,0.952640849623329 526 | 2625,0.951737937312173 527 | 2630,0.950996615777701 528 | 2635,0.950507766511385 529 | 2640,0.950327699980346 530 | 2645,0.948951254344292 531 | 2650,0.949283797288744 532 | 2655,0.949524298382526 533 | 2660,0.9504960107171 534 | 2665,0.952215273938628 535 | 2670,0.952073427359214 536 | 2675,0.953218966401533 537 | 2680,0.952842249923467 538 | 2685,0.952449995472174 539 | 2690,0.950764962534809 540 | 2695,0.949372737816393 541 | 2700,0.94731755119031 542 | 2705,0.945207561069916 543 | 2710,0.943266665938032 544 | 2715,0.94356862170647 545 | 2720,0.941498468720969 546 | 2725,0.938949107467913 547 | 2730,0.939078024011199 548 | 2735,0.941833648245655 549 | 2740,0.941527296294435 550 | 2745,0.938745416558605 551 | 2750,0.933739695181218 552 | 2755,0.929953460443688 553 | 2760,0.926251372359419 554 | 2765,0.923671527697233 555 | 2770,0.920665590950366 556 | 2775,0.917422798824255 557 | 2780,0.914529664336706 558 | 2785,0.911156585385152 559 | 2790,0.908231060425815 560 | 2795,0.905357993780338 561 | 2800,0.900057128732263 562 | 2805,0.895327326703461 563 | 2810,0.890717714111671 564 | 2815,0.886115200012781 565 | 2820,0.882787521426024 566 | 2825,0.878401593683612 567 | 2830,0.873797969642871 568 | 2835,0.869709500479469 569 | 2840,0.865374189835796 570 | 2845,0.861381898075941 571 | 2850,0.856989168607038 572 | 2855,0.852057936748066 573 | 2860,0.847818399227958 574 | 2865,0.842401802755005 575 | 2870,0.838789550882345 576 | 2875,0.835719508509248 577 | 2880,0.829583456978938 578 | 2885,0.821952665591817 579 | 2890,0.816273953127908 580 | 2895,0.812801154653205 581 | 2900,0.810761405364753 582 | 2905,0.808214804920783 583 | 2910,0.804957952047024 584 | 2915,0.800962899116084 585 | 2920,0.798133600592996 586 | 2925,0.796221315229101 587 | 2930,0.793521540909113 588 | 2935,0.791612661725317 589 | 2940,0.789629161680983 590 | 2945,0.788520360088579 591 | 2950,0.787740526922153 592 | 2955,0.786974305480374 593 | 2960,0.785734876213743 594 | 2965,0.784162023938936 595 | 2970,0.782447098138775 596 | 2975,0.780906217838283 597 | 2980,0.778936639724202 598 | 2985,0.776866925091795 599 | 2990,0.774482727794503 600 | 2995,0.772649458804231 601 | 3000,0.771924145877301 602 | 3005,0.77182694208117 603 | 3010,0.771359611004114 604 | 3015,0.770430224110062 605 | 3020,0.768316356876606 606 | 3025,0.76556787385993 607 | 3030,0.764680408189767 608 | 3035,0.76389952234966 609 | 3040,0.762512978284427 610 | 3045,0.761932882052906 611 | 3050,0.761532409827297 612 | 3055,0.7612807964007 613 | 3060,0.761513986834232 614 | 3065,0.762140172856787 615 | 3070,0.761591693361354 616 | 3075,0.761363929657123 617 | 3080,0.761791988416572 618 | 3085,0.760909048017874 619 | 3090,0.760773534657377 620 | 3095,0.760318571206455 621 | 3100,0.759761464724497 622 | 3105,0.759358572049889 623 | 3110,0.758584676383667 624 | 3115,0.758084792188876 625 | 3120,0.757088260211124 626 | 3125,0.756245542575322 627 | 3130,0.756058487698491 628 | 3135,0.755929753050427 629 | 3140,0.755598737251154 630 | 3145,0.754379827646035 631 | 3150,0.75326106817944 632 | 3155,0.7517500794657 633 | 3160,0.750338911290804 634 | 3165,0.748878808025328 635 | 3170,0.746759872920783 636 | 3175,0.744470358163079 637 | 3180,0.742275466332849 638 | 3185,0.740153061381711 639 | 3190,0.738279946351381 640 | 3195,0.735276394631956 641 | 3200,0.732353782150046 642 | 3205,0.730327643129198 643 | 3210,0.728764117312614 644 | 3215,0.727162432396361 645 | 3220,0.72542553895859 646 | 3225,0.723360384657205 647 | 3230,0.721037602106129 648 | 3235,0.719026275522917 649 | 3240,0.716458676339504 650 | 3245,0.714054689314575 651 | 3250,0.711780644999269 652 | 3255,0.710354490898933 653 | 3260,0.709999581113922 654 | 3265,0.709683140661335 655 | 3270,0.707605119899816 656 | 3275,0.706049599681147 657 | 3280,0.704008424308461 658 | 3285,0.702461981112913 659 | 3290,0.70185939637312 660 | 3295,0.701244518832294 661 | 3300,0.699622985390789 662 | 3305,0.698682466226852 663 | 3310,0.698064333305472 664 | 3315,0.698157831476305 665 | 3320,0.69845486006439 666 | 3325,0.699096365313499 667 | 3330,0.699023228791812 668 | 3335,0.699238524994509 669 | 3340,0.699535545292827 670 | 3345,0.700134638194614 671 | 3350,0.700546609620887 672 | 3355,0.701047518660721 673 | 3360,0.700572013846473 674 | 3365,0.700120698671231 675 | 3370,0.700908457289093 676 | 3375,0.701021239158842 677 | 3380,0.701008607002269 678 | 3385,0.701744226706641 679 | 3390,0.70176464244742 680 | 3395,0.701552814181774 681 | 3400,0.700027667641425 682 | 3405,0.698031203080599 683 | 3410,0.694703935543195 684 | 3415,0.693471585275842 685 | 3420,0.693173033904661 686 | 3425,0.691604588274394 687 | 3430,0.691663794078312 688 | 3435,0.691769736085023 689 | 3440,0.691674039361251 690 | 3445,0.691673995668026 691 | 3450,0.691609424629311 692 | 3455,0.689859365195705 693 | 3460,0.688592426256576 694 | 3465,0.688194190358454 695 | 3470,0.687199800851016 696 | 3475,0.685923513588952 697 | 3480,0.684180071543126 698 | 3485,0.682451135530313 699 | 3490,0.681264493769973 700 | 3495,0.680767967075526 701 | 3500,0.680326191385601 702 | 3505,0.678840232708598 703 | 3510,0.67745385711749 704 | 3515,0.676069163923689 705 | 3520,0.674233384790227 706 | 3525,0.672969747703214 707 | 3530,0.670673713261139 708 | 3535,0.668600774244303 709 | 3540,0.666660540338969 710 | 3545,0.665275292436692 711 | 3550,0.665648469510143 712 | 3555,0.664821347269589 713 | 3560,0.664832744151364 714 | 3565,0.662871002518527 715 | 3570,0.661988922717079 716 | 3575,0.661619118212162 717 | 3580,0.660236228929532 718 | 3585,0.658037968521803 719 | 3590,0.655168292608137 720 | 3595,0.653699893584809 721 | 3600,0.653465372661809 722 | 3605,0.65419206263122 723 | 3610,0.654637286547375 724 | 3615,0.653938422356646 725 | 3620,0.654536057435976 726 | 3625,0.655138770232755 727 | 3630,0.657082439124223 728 | 3635,0.657621108296073 729 | 3640,0.658063101176078 730 | 3645,0.658320194916811 731 | 3650,0.658901068974615 732 | 3655,0.660729442558957 733 | 3660,0.662632463520231 734 | 3665,0.664454715419507 735 | 3670,0.666145420575489 736 | 3675,0.668636845653088 737 | 3680,0.671877691729298 738 | 3685,0.675147281811609 739 | 3690,0.678065239915272 740 | 3695,0.680721591197039 741 | 3700,0.683445055661384 742 | 3705,0.688179399414483 743 | 3710,0.695205192961382 744 | 3715,0.7028871111955 745 | 3720,0.708426437466483 746 | 3725,0.713694053907111 747 | 3730,0.719946136569947 748 | 3735,0.725865381965411 749 | 3740,0.731331581343868 750 | 3745,0.737301956228918 751 | 3750,0.742866761387689 752 | 3755,0.748404297974045 753 | 3760,0.752781038138979 754 | 3765,0.758155815218233 755 | 3770,0.763942120411288 756 | 3775,0.771552293139343 757 | 3780,0.780128048616288 758 | 3785,0.788411413593459 759 | 3790,0.795114765457723 760 | 3795,0.801784894941333 761 | 3800,0.809048474647195 762 | 3805,0.815124432762194 763 | 3810,0.821591851836717 764 | 3815,0.829215534785027 765 | 3820,0.836265994677769 766 | 3825,0.840781698501872 767 | 3830,0.845587159847401 768 | 3835,0.85052882450456 769 | 3840,0.855827675790906 770 | 3845,0.859776393260497 771 | 3850,0.864311439065282 772 | 3855,0.867558001578232 773 | 3860,0.869968419615987 774 | 3865,0.871713577281537 775 | 3870,0.874346905202176 776 | 3875,0.877978066349602 777 | 3880,0.881686816383671 778 | 3885,0.889147561232896 779 | 3890,0.897700352752674 780 | 3895,0.903206670439347 781 | 3900,0.905729004491571 782 | 3905,0.909128365877192 783 | 3910,0.916061238812893 784 | 3915,0.922862540416549 785 | 3920,0.925406337960053 786 | 3925,0.92544961982789 787 | 3930,0.92370795039846 788 | 3935,0.923258802336582 789 | 3940,0.924593369772273 790 | 3945,0.925455036430282 791 | 3950,0.925227154298226 792 | 3955,0.925860697968443 793 | 3960,0.925994255324079 794 | 3965,0.925001640134429 795 | 3970,0.925561361657707 796 | 3975,0.927542740336244 797 | 3980,0.931244041725265 798 | 3985,0.933199687230133 799 | 3990,0.935830646277516 800 | 3995,0.938478318309217 801 | 4000,0.938935425606046 802 | 4005,0.938936281497317 803 | 4010,0.938003219231993 804 | 4015,0.937407978519667 805 | 4020,0.937611981408375 806 | 4025,0.937902726986741 807 | 4030,0.937728139050345 808 | 4035,0.934618295163011 809 | 4040,0.931631475776727 810 | 4045,0.929291678171643 811 | 4050,0.930301587968681 812 | 4055,0.930692105722885 813 | 4060,0.929363458809665 814 | 4065,0.928447893590369 815 | 4070,0.933164903866486 816 | 4075,0.934275920202214 817 | 4080,0.935301306867124 818 | 4085,0.935799473261149 819 | 4090,0.936745493083541 820 | 4095,0.937879955259854 821 | 4100,0.938570825364222 822 | 4105,0.939596133403047 823 | 4110,0.941254740763069 824 | 4115,0.943302855756543 825 | 4120,0.943795848331553 826 | 4125,0.944965127754577 827 | 4130,0.944370824853793 828 | 4135,0.944653112209357 829 | 4140,0.944245594623184 830 | 4145,0.945523592672291 831 | 4150,0.947428669540976 832 | 4155,0.950206709042347 833 | 4160,0.953643003112751 834 | 4165,0.956463025477316 835 | 4170,0.958918664464337 836 | 4175,0.961256844035456 837 | 4180,0.961395226967502 838 | 4185,0.96056401703028 839 | 4190,0.960002608865262 840 | 4195,0.960211752784967 841 | 4200,0.959823231452762 842 | 4205,0.9592177695847 843 | 4210,0.957963155302476 844 | 4215,0.95553793362365 845 | 4220,0.95121028480597 846 | 4225,0.951182232723896 847 | 4230,0.95006484563 848 | 4235,0.94915772249742 849 | 4240,0.946928665769015 850 | 4245,0.944563270558719 851 | 4250,0.942647967674468 852 | 4255,0.940353915807947 853 | 4260,0.937987325513741 854 | 4265,0.93620861063698 855 | 4270,0.933118203483449 856 | 4275,0.932411668724461 857 | 4280,0.938520743389838 858 | 4285,0.935499179093202 859 | 4290,0.932954915693765 860 | 4295,0.929809377844484 861 | 4300,0.92595584687981 862 | 4305,0.921473221768186 863 | 4310,0.916864425763673 864 | 4315,0.913339397758192 865 | 4320,0.910520061570928 866 | 4325,0.906486181517247 867 | 4330,0.905053981054694 868 | 4335,0.905028203355205 869 | 4340,0.906394140209575 870 | 4345,0.910629163861174 871 | 4350,0.917365174601049 872 | 4355,0.913480879596919 873 | 4360,0.910261393046006 874 | 4365,0.91081243988945 875 | 4370,0.912853445267617 876 | 4375,0.910507406119748 877 | 4380,0.908806964795586 878 | 4385,0.907951839080313 879 | 4390,0.907647332326199 880 | 4395,0.907481627055904 881 | 4400,0.907619922520051 882 | 4405,0.907093403177907 883 | 4410,0.9062880646073 884 | 4415,0.904048775591703 885 | 4420,0.903125414018702 886 | 4425,0.900903890317206 887 | 4430,0.890718047720483 888 | 4435,0.890004170544922 889 | 4440,0.888709825817828 890 | 4445,0.887392151080635 891 | 4450,0.88841442504672 892 | 4455,0.889111988486162 893 | 4460,0.888421676158209 894 | 4465,0.887568147671035 895 | 4470,0.885459048374771 896 | 4475,0.883815637840544 897 | 4480,0.88153251038434 898 | 4485,0.878746248601772 899 | 4490,0.873930719515493 900 | 4495,0.866362016562937 901 | 4500,0.855812479200411 902 | 4505,0.855824270868753 903 | 4510,0.854528937107322 904 | 4515,0.849446031910619 905 | 4520,0.842693029049344 906 | 4525,0.839782380402286 907 | 4530,0.838516429284798 908 | 4535,0.836416405147362 909 | 4540,0.834041872224889 910 | 4545,0.831881902177502 911 | 4550,0.828902723313596 912 | 4555,0.827794886791224 913 | 4560,0.825274886453147 914 | 4565,0.823199939493709 915 | 4570,0.821935958118926 916 | 4575,0.819415337912634 917 | 4580,0.818772385465724 918 | 4585,0.818114877998348 919 | 4590,0.817395520997348 920 | 4595,0.816050449142055 921 | 4600,0.812806315485521 922 | 4605,0.810003338419943 923 | 4610,0.807474721946248 924 | 4615,0.804416140601067 925 | 4620,0.802321405606334 926 | 4625,0.799881611513011 927 | 4630,0.798139833103918 928 | 4635,0.796549175369371 929 | 4640,0.795942740394909 930 | 4645,0.795056939809473 931 | 4650,0.794209406796571 932 | 4655,0.793293240542033 933 | 4660,0.792424959346984 934 | 4665,0.79223696454092 935 | 4670,0.791001955810522 936 | 4675,0.790872600896043 937 | 4680,0.788664026081752 938 | 4685,0.788116009220465 939 | 4690,0.787872014125735 940 | 4695,0.786574208294312 941 | 4700,0.786682718916029 942 | 4705,0.784836541175142 943 | 4710,0.784441540811003 944 | 4715,0.784352442524387 945 | 4720,0.783461840915093 946 | 4725,0.782213382200999 947 | 4730,0.780198291176773 948 | 4735,0.778513089122736 949 | 4740,0.776450233494039 950 | 4745,0.775032833821818 951 | 4750,0.775333205520804 952 | 4755,0.776295549504873 953 | 4760,0.778139834174348 954 | 4765,0.779637597660241 955 | 4770,0.780899413668302 956 | 4775,0.7814875750284 957 | 4780,0.78098231600585 958 | 4785,0.78067099388695 959 | 4790,0.779337582674625 960 | 4795,0.778355070411985 961 | 4800,0.777773616740204 962 | 4805,0.776847001578543 963 | 4810,0.777295254789058 964 | 4815,0.777414075581954 965 | 4820,0.778198842064609 966 | 4825,0.778103848966467 967 | 4830,0.779233079440197 968 | 4835,0.77905064954733 969 | 4840,0.77877504586148 970 | 4845,0.779865419537638 971 | 4850,0.779526177520434 972 | 4855,0.779707227456732 973 | 4860,0.77927855570704 974 | 4865,0.779694518013468 975 | 4870,0.781393981832948 976 | 4875,0.782895987643189 977 | 4880,0.783983230647294 978 | 4885,0.784532184053128 979 | 4890,0.786135552751333 980 | 4895,0.786246262821935 981 | 4900,0.785087725561059 982 | 4905,0.783781719322201 983 | 4910,0.780981932024939 984 | 4915,0.779339391219377 985 | 4920,0.777823480049365 986 | 4925,0.777387925241158 987 | 4930,0.776852130971285 988 | 4935,0.775591038924123 989 | 4940,0.773957594318555 990 | 4945,0.773521814057652 991 | 4950,0.772961107500122 992 | 4955,0.77158419218905 993 | 4960,0.768330973129128 994 | 4965,0.765191874781539 995 | 4970,0.762757827016986 996 | 4975,0.761056399509226 997 | 4980,0.759209806574103 998 | 4985,0.757371230350251 999 | 4990,0.755698565482468 1000 | 4995,0.753612023673299 1001 | 5000,0.752558491910787 1002 | 5005,0.751303986336728 1003 | 5010,0.749870588319624 1004 | 5015,0.747529884795071 1005 | 5020,0.744309156281435 1006 | 5025,0.741743717360565 1007 | 5030,0.739369706910597 1008 | 5035,0.737418202922531 1009 | 5040,0.735246474202969 1010 | 5045,0.735292430262689 1011 | 5050,0.736666362289418 1012 | 5055,0.73661231387883 1013 | 5060,0.737030164206792 1014 | 5065,0.73700295992532 1015 | 5070,0.736658859326488 1016 | 5075,0.736311949709413 1017 | 5080,0.736698721719829 1018 | 5085,0.736949956604543 1019 | 5090,0.736916236584484 1020 | 5095,0.736878365364163 1021 | 5100,0.737488697723605 1022 | 5105,0.73980522274966 1023 | 5110,0.742939000496872 1024 | 5115,0.745755906253028 1025 | 5120,0.747919688253737 1026 | 5125,0.750431355620511 1027 | 5130,0.753385469849669 1028 | 5135,0.756780486420103 1029 | 5140,0.759913217213211 1030 | 5145,0.763353856388401 1031 | 5150,0.766453400439614 1032 | 5155,0.771003903111851 1033 | 5160,0.775507055667957 1034 | 5165,0.779541911805475 1035 | 5170,0.784056241718385 1036 | 5175,0.788365678316654 1037 | 5180,0.793642882506057 1038 | 5185,0.799293881492284 1039 | 5190,0.80472509141227 1040 | 5195,0.808442508741403 1041 | 5200,0.811675827130182 1042 | 5205,0.817345366216006 1043 | 5210,0.823574537773464 1044 | 5215,0.829477251998381 1045 | 5220,0.835599505854438 1046 | 5225,0.841837250913136 1047 | 5230,0.847757939651115 1048 | 5235,0.853710129919144 1049 | 5240,0.861373755443474 1050 | 5245,0.867492587924543 1051 | 5250,0.873297304635309 1052 | 5255,0.879027131484414 1053 | 5260,0.885144877716437 1054 | 5265,0.890042283869874 1055 | 5270,0.89449815525247 1056 | 5275,0.898947428795209 1057 | 5280,0.904070840760771 1058 | 5285,0.908960280339736 1059 | 5290,0.914741356463747 1060 | 5295,0.918948580780658 1061 | 5300,0.922876528429639 1062 | 5305,0.925593035227567 1063 | 5310,0.929884769083185 1064 | 5315,0.932638245091551 1065 | 5320,0.935347575340641 1066 | 5325,0.938305705961803 1067 | 5330,0.940862391359556 1068 | 5335,0.94282843768004 1069 | 5340,0.94441872040219 1070 | 5345,0.947049187475444 1071 | 5350,0.950029243660934 1072 | 5355,0.951740461573306 1073 | 5360,0.95359684046951 1074 | 5365,0.956403384071494 1075 | 5370,0.958784373509591 1076 | 5375,0.961456333309053 1077 | 5380,0.964313779952431 1078 | 5385,0.966323977968966 1079 | 5390,0.966409112842067 1080 | 5395,0.967128170379767 1081 | 5400,0.968187854108523 1082 | 5405,0.969147648627309 1083 | 5410,0.970940645722431 1084 | 5415,0.973216784905242 1085 | 5420,0.977597723058751 1086 | 5425,0.979699214883104 1087 | 5430,0.98153306704399 1088 | 5435,0.982203715113002 1089 | 5440,0.982152368822485 1090 | 5445,0.983452074186995 1091 | 5450,0.985707891133252 1092 | 5455,0.988092360517703 1093 | 5460,0.989143690514538 1094 | 5465,0.991576811852404 1095 | 5470,0.9943692799553 1096 | 5475,0.995547647044203 1097 | 5480,0.996910167663965 1098 | 5485,0.99757722963055 1099 | 5490,0.998379586907805 1100 | 5495,1 1101 | 5500,0.999842509526582 1102 | 5505,0.998101345266127 1103 | 5510,0.996257456025768 1104 | 5515,0.994018774407537 1105 | 5520,0.992818781271906 1106 | 5525,0.99145538555685 1107 | 5530,0.990470834427882 1108 | 5535,0.990379296053845 1109 | 5540,0.992135526965416 1110 | 5545,0.99270291699356 1111 | 5550,0.992180985946828 1112 | 5555,0.991099699809849 1113 | 5560,0.988410810768135 1114 | 5565,0.985807754460325 1115 | 5570,0.982150215275016 1116 | 5575,0.97935694883862 1117 | 5580,0.977794166168858 1118 | 5585,0.976738270573919 1119 | 5590,0.976019708014936 1120 | 5595,0.974366439264718 1121 | 5600,0.972911628997229 1122 | 5605,0.970933417914598 1123 | 5610,0.969318833342944 1124 | 5615,0.967134556359973 1125 | 5620,0.964885400802419 1126 | 5625,0.963468959485157 1127 | 5630,0.960203589292334 1128 | 5635,0.956978090379916 1129 | 5640,0.954313747265229 1130 | 5645,0.950551795661878 1131 | 5650,0.947969147691977 1132 | 5655,0.947370769574424 1133 | 5660,0.945807511030419 1134 | 5665,0.943253837894994 1135 | 5670,0.940382383337656 1136 | 5675,0.937330506477106 1137 | 5680,0.932973598529389 1138 | 5685,0.928458749763959 1139 | 5690,0.922577829480698 1140 | 5695,0.917412583695795 1141 | 5700,0.913589819594166 1142 | 5705,0.910256322885276 1143 | 5710,0.907335065581135 1144 | 5715,0.905120655829879 1145 | 5720,0.902740310839698 1146 | 5725,0.901318016576043 1147 | 5730,0.897047908079356 1148 | 5735,0.892885229448668 1149 | 5740,0.887719679132485 1150 | 5745,0.883464290139356 1151 | 5750,0.878586130109386 1152 | 5755,0.875230284207367 1153 | 5760,0.872071183485142 1154 | 5765,0.868754188425818 1155 | 5770,0.865540102114962 1156 | 5775,0.861797636422916 1157 | 5780,0.860365877824842 1158 | 5785,0.858691296934938 1159 | 5790,0.856638526766821 1160 | 5795,0.854723880312639 1161 | 5800,0.851931407865762 1162 | 5805,0.848659919263959 1163 | 5810,0.845802578025509 1164 | 5815,0.843651403547371 1165 | 5820,0.841945700881996 1166 | 5825,0.84108159334922 1167 | 5830,0.840582136041218 1168 | 5835,0.84100358702395 1169 | 5840,0.842300751375843 1170 | 5845,0.842319537163854 1171 | 5850,0.842317114705763 1172 | 5855,0.84188330568986 1173 | 5860,0.841316897094082 1174 | 5865,0.839369275205607 1175 | 5870,0.836758838453276 1176 | 5875,0.83363492763717 1177 | 5880,0.832289152545395 1178 | 5885,0.831796604586053 1179 | 5890,0.83240507191087 1180 | 5895,0.831441942308499 1181 | 5900,0.829916481671059 1182 | 5905,0.827477741789911 1183 | 5910,0.825670075065581 1184 | 5915,0.824102290969563 1185 | 5920,0.822649653942264 1186 | 5925,0.821580821152675 1187 | 5930,0.819256067428999 1188 | 5935,0.818228177073523 1189 | 5940,0.817438403318337 1190 | 5945,0.816841460416433 1191 | 5950,0.817427976945623 1192 | 5955,0.817413296727957 1193 | 5960,0.818213158767313 1194 | 5965,0.818716372208461 1195 | 5970,0.818846630363619 1196 | 5975,0.817352983009379 1197 | 5980,0.815486290340111 1198 | 5985,0.812209517880785 1199 | 5990,0.80820327036191 1200 | 5995,0.806494557497599 1201 | 6000,0.803421412751684 1202 | 6005,0.800120741564438 1203 | 6010,0.797604099324003 1204 | 6015,0.797111544405238 1205 | 6020,0.796401792474644 1206 | 6025,0.796153395061258 1207 | 6030,0.794877778745405 1208 | 6035,0.792040593902924 1209 | 6040,0.788899711644584 1210 | 6045,0.787819123567507 1211 | 6050,0.786155767094219 1212 | 6055,0.784415320912186 1213 | 6060,0.781211591796121 1214 | 6065,0.779165102085633 1215 | 6070,0.777458820729377 1216 | 6075,0.775709470867228 1217 | 6080,0.77486022992707 1218 | 6085,0.773798191307252 1219 | 6090,0.772665338169241 1220 | 6095,0.770973241545135 1221 | 6100,0.768491816387859 1222 | 6105,0.766933733237561 1223 | 6110,0.764208829466986 1224 | 6115,0.761241310811649 1225 | 6120,0.757644207465128 1226 | 6125,0.755506431693591 1227 | 6130,0.754468985825312 1228 | 6135,0.753637494995316 1229 | 6140,0.753703555151264 1230 | 6145,0.753522064210234 1231 | 6150,0.752852560310036 1232 | 6155,0.752623706220925 1233 | 6160,0.75109966646399 1234 | 6165,0.748845006953795 1235 | 6170,0.748304588315356 1236 | 6175,0.74807025103422 1237 | 6180,0.747146309720543 1238 | 6185,0.747746941731562 1239 | 6190,0.748124924272404 1240 | 6195,0.747477763284661 1241 | 6200,0.7478893562686 1242 | 6205,0.748172821241645 1243 | 6210,0.748846621467324 1244 | 6215,0.749042552077984 1245 | 6220,0.748058269732938 1246 | 6225,0.747482730194355 1247 | 6230,0.7451947498942 1248 | 6235,0.742425800294049 1249 | 6240,0.740342423999293 1250 | 6245,0.738219044240361 1251 | 6250,0.737242430563082 1252 | 6255,0.736503292060862 1253 | 6260,0.735915849991881 1254 | 6265,0.736004355723072 1255 | 6270,0.736344621888472 1256 | 6275,0.736296395410051 1257 | 6280,0.735531884791516 1258 | 6285,0.73550297486338 1259 | 6290,0.733939643078017 1260 | 6295,0.732010375665333 1261 | 6300,0.731037566936703 1262 | 6305,0.730275782149144 1263 | 6310,0.730032994048077 1264 | 6315,0.729158120877547 1265 | 6320,0.727344153514169 1266 | 6325,0.725601593912206 1267 | 6330,0.724576572449533 1268 | 6335,0.72252255613761 1269 | 6340,0.719407430347545 1270 | 6345,0.71639009021198 1271 | 6350,0.713476064082259 1272 | 6355,0.711481053389056 1273 | 6360,0.708757287796219 1274 | 6365,0.705681473166204 1275 | 6370,0.702605249097731 1276 | 6375,0.700007184894715 1277 | 6380,0.699074988638521 1278 | 6385,0.698827662650341 1279 | 6390,0.697115731021629 1280 | 6395,0.694674491449999 1281 | 6400,0.691436937261896 1282 | 6405,0.687863764354368 1283 | 6410,0.684711149161951 1284 | 6415,0.681555374355697 1285 | 6420,0.678627526180171 1286 | 6425,0.67458817440077 1287 | 6430,0.672305961678701 1288 | 6435,0.669396995781899 1289 | 6440,0.666893951908365 1290 | 6445,0.663638198840499 1291 | 6450,0.660413701994365 1292 | 6455,0.656682004673163 1293 | 6460,0.653072075772344 1294 | 6465,0.650756690498721 1295 | 6470,0.647578609830563 1296 | 6475,0.645778427733937 1297 | 6480,0.644523722734877 1298 | 6485,0.643278574443216 1299 | 6490,0.644459617642594 1300 | 6495,0.64452293366854 1301 | 6500,0.64580667223634 1302 | 6505,0.646059818618194 1303 | 6510,0.647749660136137 1304 | 6515,0.649109668123969 1305 | 6520,0.649978697787219 1306 | 6525,0.650354338052648 1307 | 6530,0.649830849217558 1308 | 6535,0.650608507985399 1309 | 6540,0.652228565812218 1310 | 6545,0.655800774084112 1311 | 6550,0.659703669351996 1312 | 6555,0.663037083597616 1313 | 6560,0.66698675582933 1314 | 6565,0.671246000079858 1315 | 6570,0.675982453804851 1316 | 6575,0.682064890861284 1317 | 6580,0.685487083893601 1318 | 6585,0.689818586348737 1319 | 6590,0.694377340596904 1320 | 6595,0.700536877244642 1321 | 6600,0.706363235622373 1322 | 6605,0.712230078252654 1323 | 6610,0.719198525256384 1324 | 6615,0.726234821449091 1325 | 6620,0.734532045595018 1326 | 6625,0.740977747200079 1327 | 6630,0.744950930296367 1328 | 6635,0.750154285731937 1329 | 6640,0.754564914861652 1330 | 6645,0.761407148392413 1331 | 6650,0.76749529347817 1332 | 6655,0.774036547061883 1333 | 6660,0.780130956442034 1334 | 6665,0.786351190400111 1335 | 6670,0.794180090108765 1336 | 6675,0.801305839219967 1337 | 6680,0.809190389635286 1338 | 6685,0.815677110164753 1339 | 6690,0.825777269732072 1340 | 6695,0.838971397215451 1341 | 6700,0.845466449865526 1342 | 6705,0.851541639457605 1343 | 6710,0.855708435052543 1344 | 6715,0.858832215377223 1345 | 6720,0.861157786077325 1346 | 6725,0.862219163108594 1347 | 6730,0.865475312655372 1348 | 6735,0.867611603490759 1349 | 6740,0.870887282442968 1350 | 6745,0.873618474417942 1351 | 6750,0.876423612777758 1352 | 6755,0.878575154069026 1353 | 6760,0.880061740207165 1354 | 6765,0.880448977530516 1355 | 6770,0.880743328053334 1356 | 6775,0.880485297623254 1357 | 6780,0.882434131365222 1358 | 6785,0.883659739210496 1359 | 6790,0.885552763654234 1360 | 6795,0.886471877264759 1361 | 6800,0.886945712568847 1362 | 6805,0.887297478523158 1363 | 6810,0.887534997174244 1364 | 6815,0.887786399428988 1365 | 6820,0.886629839530886 1366 | 6825,0.88639807500964 1367 | 6830,0.886265753969549 1368 | 6835,0.886121230356563 1369 | 6840,0.882471916303189 1370 | 6845,0.874978508089571 1371 | 6850,0.874570073271475 1372 | 6855,0.875611168537629 1373 | 6860,0.879225378107492 1374 | 6865,0.880669854655796 1375 | 6870,0.882067087569841 1376 | 6875,0.88520523273905 1377 | 6880,0.886606024569431 1378 | 6885,0.887333362280946 1379 | 6890,0.887070319039756 1380 | 6895,0.886502443625857 1381 | 6900,0.884727180586262 1382 | 6905,0.88435038826045 1383 | 6910,0.884821758662117 1384 | 6915,0.885124951108098 1385 | 6920,0.88551296929852 1386 | 6925,0.885548545538539 1387 | 6930,0.885318724080705 1388 | 6935,0.885420181088291 1389 | 6940,0.883447125516598 1390 | 6945,0.880734996475724 1391 | 6950,0.877558936211969 1392 | 6955,0.874478169260803 1393 | 6960,0.871675284740154 1394 | 6965,0.87048291252468 1395 | 6970,0.868286791998975 1396 | 6975,0.865127763187762 1397 | 6980,0.861850312343155 1398 | 6985,0.858070398317493 1399 | 6990,0.854305846460463 1400 | 6995,0.850803640870716 1401 | 7000,0.846040552631867 1402 | 7005,0.840700933177509 1403 | 7010,0.833460766967661 1404 | 7015,0.83031221263877 1405 | 7020,0.826771123735057 1406 | 7025,0.822243820771541 1407 | 7030,0.819194089222992 1408 | 7035,0.816625011709904 1409 | 7040,0.813495751758885 1410 | 7045,0.810072089899355 1411 | 7050,0.807148759655772 1412 | 7055,0.80360381564851 1413 | 7060,0.799095440050637 1414 | 7065,0.7952662900955 1415 | 7070,0.789906564010925 1416 | 7075,0.785068276546456 1417 | 7080,0.779988087931665 1418 | 7085,0.774671212429855 1419 | 7090,0.77105622230062 1420 | 7095,0.768022337220462 1421 | 7100,0.765249375336991 1422 | 7105,0.76117444747142 1423 | 7110,0.757730937303837 1424 | 7115,0.752590697093617 1425 | 7120,0.748967030058257 1426 | 7125,0.746569177723192 1427 | 7130,0.743200516837231 1428 | 7135,0.739643613031401 1429 | 7140,0.735976907065429 1430 | 7145,0.731893252781381 1431 | 7150,0.729191184567086 1432 | 7155,0.725821526881921 1433 | 7160,0.72211921973423 1434 | 7165,0.716796541558576 1435 | 7170,0.712263926049151 1436 | 7175,0.707135038638332 1437 | 7180,0.701901418404557 1438 | 7185,0.697144701177274 1439 | 7190,0.693309241677924 1440 | 7195,0.689755844208142 1441 | 7200,0.68752534673456 1442 | 7205,0.685585342354407 1443 | 7210,0.684378061590682 1444 | 7215,0.681691211053299 1445 | 7220,0.68068784380084 1446 | 7225,0.679938761669072 1447 | 7230,0.679212485373503 1448 | 7235,0.677670744864909 1449 | 7240,0.674613924316871 1450 | 7245,0.671460307492569 1451 | 7250,0.668274008606647 1452 | 7255,0.666229620867886 1453 | 7260,0.662907913365748 1454 | 7265,0.660562791866101 1455 | 7270,0.658708416595667 1456 | 7275,0.655589395511164 1457 | 7280,0.652999283221678 1458 | 7285,0.651302059177722 1459 | 7290,0.649980512250647 1460 | 7295,0.648687347819822 1461 | 7300,0.645457475259033 1462 | 7305,0.643074989409433 1463 | 7310,0.640947124814276 1464 | 7315,0.639299699535139 1465 | 7320,0.638644555737289 1466 | 7325,0.638147896316074 1467 | 7330,0.637265888968636 1468 | 7335,0.635762950306234 1469 | 7340,0.633819071998606 1470 | 7345,0.632352910144411 1471 | 7350,0.630789246460679 1472 | 7355,0.629650110638747 1473 | 7360,0.627013575329513 1474 | 7365,0.625230517067458 1475 | 7370,0.621934589002403 1476 | 7375,0.618848262405118 1477 | 7380,0.616310511631081 1478 | 7385,0.614453721576925 1479 | 7390,0.612976680652731 1480 | 7395,0.611643818429305 1481 | 7400,0.610033467937482 1482 | 7405,0.607623126281421 1483 | 7410,0.606119083740642 1484 | 7415,0.604205426429396 1485 | 7420,0.601480918884332 1486 | 7425,0.600750265483306 1487 | 7430,0.599608464175556 1488 | 7435,0.598087239645139 1489 | 7440,0.595782793569475 1490 | 7445,0.593244221094931 1491 | 7450,0.592067335032592 1492 | 7455,0.590503164225092 1493 | 7460,0.590560574624806 1494 | 7465,0.589706209758084 1495 | 7470,0.587655327285701 1496 | 7475,0.585624635771098 1497 | 7480,0.583996162636406 1498 | 7485,0.5830192063637 1499 | 7490,0.581666923497307 1500 | 7495,0.579835730975963 1501 | 7500,0.577607404923529 1502 | 7505,0.574750865394216 1503 | 7510,0.573213705669563 1504 | 7515,0.572023707639673 1505 | 7520,0.571425645890754 1506 | 7525,0.570775403191262 1507 | 7530,0.569678476635465 1508 | 7535,0.567648138558146 1509 | 7540,0.566480621561258 1510 | 7545,0.565638276747763 1511 | 7550,0.564831408805685 1512 | 7555,0.564041566741372 1513 | 7560,0.563230772201031 1514 | 7565,0.562055384866986 1515 | 7570,0.561554608562971 1516 | 7575,0.559624220565677 1517 | 7580,0.558663022397516 1518 | 7585,0.557105176555367 1519 | 7590,0.556075651070544 1520 | 7595,0.556022775951555 1521 | 7600,0.55495116992728 1522 | 7605,0.553660131857834 1523 | 7610,0.552834558437123 1524 | 7615,0.552039278122905 1525 | 7620,0.55096720814319 1526 | 7625,0.551005132903097 1527 | 7630,0.55096313346942 1528 | 7635,0.551404342097668 1529 | 7640,0.552666632776326 1530 | 7645,0.552624359617446 1531 | 7650,0.552921484551445 1532 | 7655,0.552982789096252 1533 | 7660,0.552409707907916 1534 | 7665,0.551887448094101 1535 | 7670,0.55077170186723 1536 | 7675,0.549430677952114 1537 | 7680,0.54866619701746 1538 | 7685,0.547893454787215 1539 | 7690,0.547573348990718 1540 | 7695,0.547016859480987 1541 | 7700,0.546655408816589 1542 | 7705,0.54588316172796 1543 | 7710,0.544865997407597 1544 | 7715,0.544528871507212 1545 | 7720,0.543789107742998 1546 | 7725,0.543184012145068 1547 | 7730,0.541821672180681 1548 | 7735,0.541844058551062 1549 | 7740,0.542093660333622 1550 | 7745,0.541693706489935 1551 | 7750,0.542178397732315 1552 | 7755,0.54312710921302 1553 | 7760,0.541833461278834 1554 | 7765,0.541482933178561 1555 | 7770,0.541330607338445 1556 | 7775,0.539748754892808 1557 | 7780,0.536924220487827 1558 | 7785,0.53471736384535 1559 | 7790,0.532023698160379 1560 | 7795,0.530414880261814 1561 | 7800,0.528690785680948 1562 | 7805,0.526869436569854 1563 | 7810,0.525662931460421 1564 | 7815,0.524575889275011 1565 | 7820,0.524606638083377 1566 | 7825,0.525060361797868 1567 | 7830,0.524186965545218 1568 | 7835,0.524003970735515 1569 | 7840,0.522877517351755 1570 | 7845,0.521717810509767 1571 | 7850,0.520226354459445 1572 | 7855,0.519632569071786 1573 | 7860,0.519033236707958 1574 | 7865,0.518214406578122 1575 | 7870,0.518683141684651 1576 | 7875,0.519298232587422 1577 | 7880,0.519181331545232 1578 | 7885,0.518172073761881 1579 | 7890,0.516482111607922 1580 | 7895,0.51545256597788 1581 | 7900,0.513920247332225 1582 | 7905,0.5120211315027 1583 | 7910,0.510316314394375 1584 | 7915,0.508279728819394 1585 | 7920,0.507146244229213 1586 | 7925,0.506769510130868 1587 | 7930,0.507991351924236 1588 | 7935,0.508064707255946 1589 | 7940,0.508751116506492 1590 | 7945,0.509638659499544 1591 | 7950,0.510084798801431 1592 | 7955,0.510251268884161 1593 | 7960,0.508921186628594 1594 | 7965,0.507662452368482 1595 | 7970,0.506006191790755 1596 | 7975,0.505067027080589 1597 | 7980,0.504529106811337 1598 | 7985,0.504050136394929 1599 | 7990,0.505403753515575 1600 | 7995,0.507299860576058 1601 | 8000,0.509674446857117 1602 | 8005,0.511348805298913 1603 | 8010,0.512758846481825 1604 | 8015,0.514124128010222 1605 | 8020,0.513576729677756 1606 | 8025,0.512753246581468 1607 | 8030,0.512434056387901 1608 | 8035,0.512803399634267 1609 | 8040,0.51379294757401 1610 | 8045,0.513634958139891 1611 | 8050,0.514101125102012 1612 | 8055,0.514764997770879 1613 | 8060,0.515767262357656 1614 | 8065,0.516815279434269 1615 | 8070,0.516918565783424 1616 | 8075,0.516866182258464 1617 | 8080,0.51652774116305 1618 | 8085,0.516574086715109 1619 | 8090,0.515948421617516 1620 | 8095,0.516134722415169 1621 | 8100,0.516328623118965 1622 | 8105,0.517653247160407 1623 | 8110,0.520380965741202 1624 | 8115,0.522785698048744 1625 | 8120,0.524626230042158 1626 | 8125,0.525281394202426 1627 | 8130,0.52691837777821 1628 | 8135,0.527938529328701 1629 | 8140,0.527068501671652 1630 | 8145,0.525911976181513 1631 | 8150,0.524653055709347 1632 | 8155,0.525078453182162 1633 | 8160,0.525389465991401 1634 | 8165,0.525518371158882 1635 | 8170,0.526808881488744 1636 | 8175,0.528030044770131 1637 | 8180,0.530155544826965 1638 | 8185,0.531496179285941 1639 | 8190,0.532405281862789 1640 | 8195,0.533362454625183 1641 | 8200,0.534379768233747 1642 | 8205,0.535807334291324 1643 | 8210,0.537803403721683 1644 | 8215,0.539983633922873 1645 | 8220,0.541196502749602 1646 | 8225,0.541784986759007 1647 | 8230,0.543012093097276 1648 | 8235,0.544601432722705 1649 | 8240,0.546897521120664 1650 | 8245,0.547917806755418 1651 | 8250,0.550283667423448 1652 | 8255,0.551986871339538 1653 | 8260,0.553570292505887 1654 | 8265,0.554105529044594 1655 | 8270,0.55458467977248 1656 | 8275,0.556004476689593 1657 | 8280,0.557230669995121 1658 | 8285,0.559282647639815 1659 | 8290,0.563087844187151 1660 | 8295,0.565447478588734 1661 | 8300,0.56818815172201 1662 | 8305,0.570272266186107 1663 | 8310,0.572681137367862 1664 | 8315,0.575614318306915 1665 | 8320,0.577587845707517 1666 | 8325,0.579315067105152 1667 | 8330,0.580463853337807 1668 | 8335,0.581964353252184 1669 | 8340,0.583267687670249 1670 | 8345,0.585788333560339 1671 | 8350,0.588018227880988 1672 | 8355,0.589669969033528 1673 | 8360,0.591056314424906 1674 | 8365,0.592425128642846 1675 | 8370,0.595229482533706 1676 | 8375,0.597909923868676 1677 | 8380,0.599269289435783 1678 | 8385,0.599958767619529 1679 | 8390,0.600085420142438 1680 | 8395,0.601175680259651 1681 | 8400,0.601899463326412 1682 | 8405,0.601911303774059 1683 | 8410,0.602829446287276 1684 | 8415,0.604666933836691 1685 | 8420,0.607424793339789 1686 | 8425,0.60895554860421 1687 | 8430,0.609747602425359 1688 | 8435,0.610533033211551 1689 | 8440,0.609962906763973 1690 | 8445,0.610944901893586 1691 | 8450,0.610746589133373 1692 | 8455,0.610062554177012 1693 | 8460,0.60927150654893 1694 | 8465,0.608910227652845 1695 | 8470,0.609644902550054 1696 | 8475,0.610877295623734 1697 | 8480,0.610847174303653 1698 | 8485,0.609311166711782 1699 | 8490,0.608853304973776 1700 | 8495,0.607470488814853 1701 | 8500,0.605959496336328 1702 | 8505,0.604533443541286 1703 | 8510,0.602172102714742 1704 | 8515,0.598435612025951 1705 | 8520,0.595193794707633 1706 | 8525,0.593363453842176 1707 | 8530,0.591551129825512 1708 | 8535,0.58936766634378 1709 | 8540,0.588446042544815 1710 | 8545,0.587112500976812 1711 | 8550,0.585846149090729 1712 | 8555,0.584696434892893 1713 | 8560,0.581039316277498 1714 | 8565,0.577462555059351 1715 | 8570,0.574186772828956 1716 | 8575,0.571446671580988 1717 | 8580,0.569511360349125 1718 | 8585,0.567591745446755 1719 | 8590,0.565931533148066 1720 | 8595,0.563699626531782 1721 | 8600,0.56284862774477 1722 | 8605,0.561431029389645 1723 | 8610,0.560699365867867 1724 | 8615,0.558997031436092 1725 | 8620,0.556286213210065 1726 | 8625,0.553709558868243 1727 | 8630,0.553193659902885 1728 | 8635,0.55383231164413 1729 | 8640,0.553674011941056 1730 | 8645,0.553920363990239 1731 | 8650,0.554597633229915 1732 | 8655,0.555526937190101 1733 | 8660,0.558161023271451 1734 | 8665,0.560209817428503 1735 | 8670,0.561702254639693 1736 | 8675,0.561970019391759 1737 | 8680,0.562606831456378 1738 | 8685,0.564057435536024 1739 | 8690,0.564291910286109 1740 | 8695,0.564638273212974 1741 | 8700,0.563905019793568 1742 | 8705,0.564506726270581 1743 | 8710,0.565930365385996 1744 | 8715,0.567155566706899 1745 | 8720,0.568258926121714 1746 | 8725,0.568926217595379 1747 | 8730,0.571045696801843 1748 | 8735,0.572335972248946 1749 | 8740,0.571566723956298 1750 | 8745,0.571585755342225 1751 | 8750,0.570301626865787 1752 | 8755,0.570469408505461 1753 | 8760,0.570220285049684 1754 | 8765,0.570048209763318 1755 | 8770,0.570584673821006 1756 | 8775,0.571179315570533 1757 | 8780,0.57125255206717 1758 | 8785,0.571106085010672 1759 | 8790,0.571705709905528 1760 | 8795,0.57088282050496 1761 | 8800,0.570116675481075 1762 | 8805,0.569747301989731 1763 | 8810,0.568746085931751 1764 | 8815,0.5699252649472 1765 | 8820,0.57002772899855 1766 | 8825,0.571150744608788 1767 | 8830,0.571426851635521 1768 | 8835,0.572568550497681 1769 | 8840,0.573887688233911 1770 | 8845,0.574463122304217 1771 | 8850,0.575953978559143 1772 | 8855,0.576328938846497 1773 | 8860,0.575799806092027 1774 | 8865,0.575792809474568 1775 | 8870,0.575673272286543 1776 | 8875,0.575934146360026 1777 | 8880,0.573007208067853 1778 | 8885,0.570883958194001 1779 | 8890,0.570531044432789 1780 | 8895,0.570575836051776 1781 | 8900,0.570887775649701 1782 | 8905,0.570203626084599 1783 | 8910,0.56980755293776 1784 | 8915,0.568723960381324 1785 | 8920,0.567305083972222 1786 | 8925,0.565754817239398 1787 | 8930,0.562960441639448 1788 | 8935,0.560738278377015 1789 | 8940,0.558430442003939 1790 | 8945,0.557376455697949 1791 | 8950,0.555839633410243 1792 | 8955,0.55441792760238 1793 | 8960,0.552753203788172 1794 | 8965,0.551136585599168 1795 | 8970,0.550375268224135 1796 | 8975,0.54838349435906 1797 | 8980,0.547160674952411 1798 | 8985,0.545568766802475 1799 | 8990,0.543715571737812 1800 | 8995,0.541954759807 1801 | 9000,0.540255422469813 1802 | 9005,0.538465080163452 1803 | 9010,0.537661147106512 1804 | 9015,0.537383154918034 1805 | 9020,0.538311581719535 1806 | 9025,0.538086126054704 1807 | 9030,0.538843963356684 1808 | 9035,0.540122808500496 1809 | 9040,0.540267923944911 1810 | 9045,0.540321842995317 1811 | 9050,0.5410747140935 1812 | 9055,0.541813467773162 1813 | 9060,0.541595669391243 1814 | 9065,0.542735260969349 1815 | 9070,0.544550137039585 1816 | 9075,0.545921041600047 1817 | 9080,0.548043055592753 1818 | 9085,0.549114290043616 1819 | 9090,0.549941386482723 1820 | 9095,0.55150896136168 1821 | 9100,0.552857600694575 1822 | 9105,0.553502141536819 1823 | 9110,0.553771186333855 1824 | 9115,0.553515700264879 1825 | 9120,0.553818584469096 1826 | 9125,0.554576037129517 1827 | 9130,0.555184128536941 1828 | 9135,0.555129737458527 1829 | 9140,0.555522889820072 1830 | 9145,0.556198819339317 1831 | 9150,0.557214964525165 1832 | 9155,0.558929474669542 1833 | 9160,0.560207069542766 1834 | 9165,0.560634594879252 1835 | 9170,0.559124368317258 1836 | 9175,0.558842026987711 1837 | 9180,0.559714574426161 1838 | 9185,0.559742355371322 1839 | 9190,0.560531547195741 1840 | 9195,0.560522172066815 1841 | 9200,0.560441823267032 1842 | 9205,0.559960671230168 1843 | 9210,0.56023285093464 1844 | 9215,0.559174600756339 1845 | 9220,0.556941377044534 1846 | 9225,0.555816650194754 1847 | 9230,0.554635778709388 1848 | 9235,0.554310132452696 1849 | 9240,0.552453182214182 1850 | 9245,0.550204163473979 1851 | 9250,0.548317931770118 1852 | 9255,0.547579776854156 1853 | 9260,0.547895905101094 1854 | 9265,0.547199755193587 1855 | 9270,0.545844864978344 1856 | 9275,0.544036467976241 1857 | 9280,0.542105679228453 1858 | 9285,0.540298895382385 1859 | 9290,0.538110113363669 1860 | 9295,0.53590528072748 1861 | 9300,0.532971655199908 1862 | 9305,0.530542303153835 1863 | 9310,0.528863017133488 1864 | 9315,0.527077558078956 1865 | 9320,0.525871211054289 1866 | 9325,0.523780964117569 1867 | 9330,0.521845608649484 1868 | 9335,0.520928859504882 1869 | 9340,0.518999820060784 1870 | 9345,0.516994481567073 1871 | 9350,0.515348825320546 1872 | 9355,0.515248780885599 1873 | 9360,0.514676306551631 1874 | 9365,0.515184698291526 1875 | 9370,0.515495795247328 1876 | 9375,0.515617454416549 1877 | 9380,0.515977646212891 1878 | 9385,0.515481155863099 1879 | 9390,0.516623432818842 1880 | 9395,0.516500571923037 1881 | 9400,0.516231267138643 1882 | 9405,0.515239506772367 1883 | 9410,0.513483399736699 1884 | 9415,0.513006821877303 1885 | 9420,0.511864423109048 1886 | 9425,0.511863275446947 1887 | 9430,0.511741707899559 1888 | 9435,0.512660019809997 1889 | 9440,0.514481174256372 1890 | 9445,0.515015644154552 1891 | 9450,0.516754168214788 1892 | 9455,0.517336358245917 1893 | 9460,0.517289500067713 1894 | 9465,0.517711971451955 1895 | 9470,0.51734533271485 1896 | 9475,0.518056800337568 1897 | 9480,0.518056763848114 1898 | 9485,0.51749695396099 1899 | 9490,0.518116862358554 1900 | 9495,0.518468508274147 1901 | 9500,0.517529939160194 1902 | 9505,0.515630150683065 1903 | 9510,0.514707782687482 1904 | 9515,0.514228089613371 1905 | 9520,0.513960558549468 1906 | 9525,0.512720755184485 1907 | 9530,0.511104824231627 1908 | 9535,0.510461408679974 1909 | 9540,0.510352447134764 1910 | 9545,0.511985374010007 1911 | 9550,0.512827434147843 1912 | 9555,0.513229690791 1913 | 9560,0.514094324669667 1914 | 9565,0.514967746135515 1915 | 9570,0.517730782416965 1916 | 9575,0.519750389735091 1917 | 9580,0.52221147851977 1918 | 9585,0.523253233763579 1919 | 9590,0.523666103371602 1920 | 9595,0.524887926431545 1921 | 9600,0.525322334319748 1922 | 9605,0.526635135832472 1923 | 9610,0.527949487857382 1924 | 9615,0.528906875538315 1925 | 9620,0.531918013788036 1926 | 9625,0.533680274473241 1927 | 9630,0.534961184311979 1928 | 9635,0.536256769264203 1929 | 9640,0.53746756259035 1930 | 9645,0.539190780845127 1931 | 9650,0.542728150050848 1932 | 9655,0.544821152411924 1933 | 9660,0.54727992424299 1934 | 9665,0.549067033250709 1935 | 9670,0.551925039815343 1936 | 9675,0.554506803676187 1937 | 9680,0.557696457908561 1938 | 9685,0.559846039955081 1939 | 9690,0.561396483433696 1940 | 9695,0.562621195133462 1941 | 9700,0.565071802503548 1942 | 9705,0.566778411196993 1943 | 9710,0.567273158487571 1944 | 9715,0.56744147266331 1945 | 9720,0.56749530358517 1946 | 9725,0.568703808700387 1947 | 9730,0.569462839674483 1948 | 9735,0.570172834560404 1949 | 9740,0.572203838841753 1950 | 9745,0.574339393010093 1951 | 9750,0.576726432837642 1952 | 9755,0.579396977838174 1953 | 9760,0.58172829388965 1954 | 9765,0.583602206800982 1955 | 9770,0.584339770555213 1956 | 9775,0.586178354960232 1957 | 9780,0.588560405035979 1958 | 9785,0.590674826489213 1959 | 9790,0.592736972166356 1960 | 9795,0.594600748738211 1961 | 9800,0.596318513738244 1962 | 9805,0.599172561350995 1963 | 9810,0.600826330166008 1964 | 9815,0.602587314344779 1965 | 9820,0.603853925991971 1966 | 9825,0.606447205117165 1967 | 9830,0.607980101965003 1968 | 9835,0.610214037365955 1969 | 9840,0.611670002191617 1970 | 9845,0.61320616640216 1971 | 9850,0.614797273648405 1972 | 9855,0.617746740811688 1973 | 9860,0.62170874977296 1974 | 9865,0.625375188240103 1975 | 9870,0.6284403278044 1976 | 9875,0.630547844732606 1977 | 9880,0.633632413881234 1978 | 9885,0.636689324603111 1979 | 9890,0.637323640488633 1980 | 9895,0.638734906222138 1981 | 9900,0.639663385965926 1982 | 9905,0.64059338894531 1983 | 9910,0.64249467164385 1984 | 9915,0.644584894018968 1985 | 9920,0.646391931843218 1986 | 9925,0.646472910991074 1987 | 9930,0.647309181125016 1988 | 9935,0.64828882031377 1989 | 9940,0.649391490175648 1990 | 9945,0.64993311577155 1991 | 9950,0.648703482448109 1992 | 9955,0.647616221697025 1993 | 9960,0.647273797965972 1994 | 9965,0.646975985460979 1995 | 9970,0.645754567251164 1996 | 9975,0.644386901292623 1997 | 9980,0.64443510853082 1998 | 9985,0.644456025187092 1999 | 9990,0.645290417037261 2000 | 9995,0.644319953516233 2001 | 10000,0.642730619519881 2002 | 10005,0.585859389060967 2003 | 10010,0.585859389060967 2004 | 10015,0.585859389060967 2005 | 10020,0.585859389060967 2006 | 10025,0.585859389060967 2007 | 10030,0.585859389060967 2008 | 10035,0.585859389060967 2009 | 10040,0.585859389060967 2010 | 10045,0.585859389060967 2011 | 10050,0.585859389060967 2012 | 10055,0.585859389060967 2013 | 10060,0.585859389060967 2014 | 10065,0.585859389060967 2015 | 10070,0.585859389060967 2016 | 10075,0.585859389060967 2017 | -------------------------------------------------------------------------------- /code/full-pipeline/common.R: -------------------------------------------------------------------------------- 1 | ########################################################################### 2 | ### Package imports and functions that are used throughout the project. ### 3 | ########################################################################### 4 | 5 | library(data.table) 6 | library(dplyr) 7 | library(forcats) 8 | library(ggplot2) 9 | library(h2o) 10 | library(optparse) 11 | library(purrr) 12 | library(RColorBrewer) 13 | library(RcppRoll) 14 | library(readr) 15 | library(tidyr) 16 | 17 | # Read topology file like the one used by Bari et al. 18 | readTopo <- function(topofile = "inet2") { 19 | ptr <- 1 20 | # First line: number of nodes and edges. 21 | topoinfo <- read_lines(topofile, skip = 0, n_max = 1) 22 | n.nodes <- as.numeric(strsplit(topoinfo, split = " ")[[1]][1]) 23 | n.edges <- as.numeric(strsplit(topoinfo, split = " ")[[1]][2]) 24 | # Next n.nodes lines: node info, i.e., ID and number of CPU cores. 25 | nodeinfo <- data.frame() 26 | for (i in 1:n.nodes) { 27 | curnode <- strsplit(read_lines(topofile, skip = ptr, n_max = 1), split = " ")[[1]] 28 | nodeinfo <- rbind(nodeinfo, data.frame(v = i, 29 | cpu = as.numeric(curnode[2]))) 30 | ptr <- ptr + 1 31 | } 32 | # Next n.edges lines: link information including source, destination, capacity, and imposed delay. 33 | edgeinfo <- data.frame() 34 | for (i in 1:n.edges) { 35 | curedge <- strsplit(read_lines(topofile, skip = ptr, n_max = 1), split = " ")[[1]] 36 | edgeinfo <- rbind(edgeinfo, data.frame(from = as.numeric(curedge[1]) + 1, 37 | to = as.numeric(curedge[2]) + 1, 38 | bw = as.numeric(curedge[3]), 39 | delay = as.numeric(curedge[4]))) 40 | ptr <- ptr + 1 41 | } 42 | return(list(n.nodes, n.edges, nodeinfo, edgeinfo)) 43 | } 44 | 45 | # Given a slope value / vector of slope values, map them to -1, 0, or 1, based on the sign. 46 | discretizeSlope <- function(theslope) { 47 | return(case_when( 48 | theslope > 0 ~ 1, 49 | theslope == 0 ~ 0, 50 | theslope < 0 ~ -1 51 | )) 52 | } 53 | 54 | # Given a vector of observations, calculate the width of the confidence interval. 55 | getCI <- function(vec) { 56 | return(qt(.95, length(vec) - 1) * sd(vec) / sqrt(length(vec))) 57 | } 58 | 59 | # Generate copper colormap with n distinct colors. 60 | copper <- function(n) { 61 | fromColor <- c(0, 0, 0) * 255 62 | toColor <- c(1.00, 0.7812, 0.4975) * 255 63 | # Equidistant colors between fromColor and toColor. 64 | cols <- matrix( 65 | c( 66 | seq(fromColor[1], toColor[1], length.out = n), 67 | seq(fromColor[2], toColor[2], length.out = n), 68 | seq(fromColor[3], toColor[3], length.out = n) 69 | ), nrow = n) 70 | # Conversion to hex by rowwise application of rgb2hex. 71 | cols <- apply(cols, 1, rgb2hex) 72 | return(cols) 73 | } 74 | 75 | # Convert vector of rgb values to a hex string that is prepended with #. 76 | rgb2hex <- function(rgbVec) { 77 | paste(c("#", lapply(rgbVec, function(x) return(sprintf("%02x", round(x))))), collapse = "") 78 | } 79 | 80 | # Create data set that can be used for training & testing. 81 | # trainFileNames: List of training-*.rds filenames (with path) that should be aggregated. 82 | # targetVNF: VNF type for which action labels are created. 83 | # extractEndtimes: Whether to include the timestamp of each prediction event as a feature. 84 | # saveWeekInfo: Whether to include information on the week. 85 | # saveTimeInfo: Whether to add information on weekday, hour, minute. 86 | createDataset <- function(trainFileNames, targetVNF = "firewall", extractEndtimes = F, saveWeekInfo = F, saveTimeInfo = F) { 87 | 88 | fulldataset <- tibble() 89 | 90 | for (i in 1:length(trainFileNames)) { 91 | curData <- readRDS(trainFileNames[i]) 92 | if (saveWeekInfo) { 93 | curData <- curData %>% mutate(week = i) #trainFileNames[i]) 94 | } 95 | fulldataset <- rbind(fulldataset, curData) 96 | } 97 | 98 | vnfcat <- readRDS("vnfcat.rds") 99 | 100 | # Disambiguate between columns that represent features for the given use case and those that represent labels. 101 | relevantFeatures <- c() 102 | predFeatures <- c() 103 | for (i in 1:nrow(vnfcat)) { 104 | curvnfname <- vnfcat$name[i] 105 | predFeatures <- c(predFeatures, sprintf("%saction", curvnfname), sprintf("%sactionD", curvnfname)) 106 | } 107 | if (saveTimeInfo) { 108 | fulldataset <- 109 | fulldataset %>% 110 | mutate( 111 | minute = floor(endtime / 60) %% 60, 112 | hour = floor(endtime / 3600) %% 24, 113 | day = floor(endtime / (3600 * 24))) 114 | } 115 | fnames <- names(fulldataset) 116 | if (extractEndtimes) { 117 | idxFrom <- 1 118 | } else { 119 | idxFrom <- 2 120 | } 121 | for (i in idxFrom:length(fnames)) { 122 | if (fnames[i] %in% predFeatures) { 123 | next 124 | } 125 | relevantFeatures <- c(relevantFeatures, fnames[i]) 126 | } 127 | 128 | relevantFeatures <- relevantFeatures[!(grepl("cv|util", relevantFeatures) & grepl(paste(subset(vnfcat, name != targetVNF)$name, collapse = "|"), relevantFeatures))] 129 | 130 | removeActionRelatedFeatures <- F 131 | if (removeActionRelatedFeatures) { 132 | relevantFeatures <- names(fulldataset)[!grepl("action", names(fulldataset))][-1] 133 | } 134 | 135 | predField <- paste(targetVNF, "actionD", sep = "") 136 | 137 | fulldataset <- fulldataset[, c(relevantFeatures, predField)] 138 | names(fulldataset)[length(names(fulldataset))] <- "action" 139 | fulldataset <- 140 | fulldataset %>% 141 | mutate(action = factor(action)) %>% 142 | mutate(action = fct_recode(action, remove = "-1", doNothing = "0", add = "1")) 143 | 144 | # Drop NAs, report amount. 145 | nbefore <- nrow(fulldataset) 146 | 147 | fulldataset <- 148 | fulldataset %>% 149 | drop_na() 150 | 151 | nafter <- nrow(fulldataset) 152 | 153 | cat(sprintf("Dropped rows that contained at least one NA entry, went from %d rows to %d.\n", nbefore, nafter)) 154 | 155 | fulldataset <- unique(fulldataset) 156 | 157 | nafterdups <- nrow(fulldataset) 158 | 159 | cat(sprintf("Dropped duplicate rows, went from %d rows to %d.\n", nafter, nafterdups)) 160 | 161 | return(fulldataset) 162 | 163 | } 164 | 165 | # Extract the union of the top n most relevant features from the leaderboard of an automl object. 166 | # Considers up to 5 best performing models in the leaderboard. 167 | aml2topfeat <- function(theAML, topn) { 168 | aml.lb <- as_tibble(theAML@leaderboard) 169 | alltopn <- c() 170 | for (i in 1:min(5, nrow(aml.lb))) { 171 | curmodid <- aml.lb$model_id[i] 172 | curvimp <- as_tibble(h2o.varimp(h2o.getModel(curmodid))) 173 | curtopn <- head(curvimp$variable, topn) 174 | alltopn <- union(alltopn, curtopn) 175 | } 176 | return(alltopn) 177 | } 178 | 179 | # Downsample a data set by removing rows until the number of occurences per class diverges by a maximum factor. 180 | downsample <- function(thedf, classcolname, maxImbalance = 1) { 181 | nperclass <- table(thedf[[classcolname]]) 182 | minocc <- min(nperclass) 183 | classestodownsample <- nperclass[which(nperclass > maxImbalance * minocc)] 184 | if (length(classestodownsample) == 0) { 185 | cat("[ downsample ] Input data set already balanced - doing nothing.\n") 186 | return(thedf) 187 | } 188 | removerows <- c() 189 | for (i in 1:length(classestodownsample)) { 190 | curclassname <- names(classestodownsample[i]) 191 | downsampleby <- as.numeric(classestodownsample[i]) - round(maxImbalance * minocc) 192 | currowids <- which(thedf[[classcolname]] == curclassname) 193 | currem <- sample(currowids, downsampleby) 194 | removerows <- c(removerows, currem) 195 | } 196 | outdf <- thedf[-removerows, ] 197 | return(outdf) 198 | } 199 | 200 | # Quick comparison of data frames by means of pairwise application of KS test to all numeric columns. 201 | comparedfs <- function(df1, df2) { 202 | res <- data.frame() 203 | cnames <- names(df1) 204 | for (i in 1:length(cnames)) { 205 | curname <- cnames[i] 206 | if (!is.numeric(df1[[curname]])) { 207 | next 208 | } 209 | curstat <- as.numeric(ks.test(df1[[curname]], df2[[curname]])$statistic) 210 | res <- rbind(res, data.frame(name = curname, d = curstat)) 211 | } 212 | return(res) 213 | } 214 | 215 | # Assign class weights, so that the sum of weights per class is equal. 216 | # fulldata %>% group_by(action) %>% summarise(wsum = sum(weight)) 217 | generateWeightedDataSet <- function(dataset) { 218 | 219 | classweights <- round(table(dataset$action) / nrow(dataset), 2) 220 | classweights <- 221 | data.frame(action = as.character(names(classweights)), weight = 1 / as.numeric(classweights)) %>% 222 | # For the three-class case, this normalization asserts that only the majority class receives a weight < 1. 223 | mutate(weight = weight / median(weight)) 224 | 225 | dataset.w <- 226 | dataset %>% 227 | full_join(classweights, by = "action") %>% 228 | mutate(action = factor(action)) 229 | 230 | return(dataset.w) 231 | 232 | } 233 | 234 | # Print information on the class probability distribution of a data set. 235 | printActionInfo <- function(traindf) { 236 | obscount <- traindf %>% group_by(action) %>% summarise(nobs = n()) 237 | cat(sprintf("%s - %.2f\t", obscount$action, obscount$nobs / sum(obscount$nobs))) 238 | cat("\n") 239 | } 240 | -------------------------------------------------------------------------------- /code/full-pipeline/data/.gitignore: -------------------------------------------------------------------------------- 1 | # Ignore everything in this directory 2 | * 3 | # Except this file 4 | !.gitignore 5 | 6 | -------------------------------------------------------------------------------- /code/full-pipeline/feature-description.txt: -------------------------------------------------------------------------------- 1 | "narrivals": Number of arrivals during aggregation interval. 2 | "ndepartures": Number of departures during aggregation interval (departure occurs at t = arrivaltime + duration). 3 | "meanarrrate": Mean arrival rate, i.e., number of arrivals / duration of aggreagtion interval. 4 | "meandeprate": Mean departure rate - cf. arrival rate. 5 | "changeN": Change in the number of instantiated VNFs (type-independent), i.e., n_vnfs at the end of the aggregation interval - n_vnfs at its beginning. 6 | "changeB": Change in the total requested bandwidth (type-independent). Unit: kbps. 7 | "changeNproxy": Change in the number of instantiated VNFs (type-dependent). In this case, proxy. 8 | "changeBproxy": Change in the total requested bandwidth (type-dependent). Unit: kbps. 9 | "lastactionproxy1": Last action that wasn't "doNothing" performed for VNF (type-dependent). Either 1 or -1, indicating that the action was "add" / "remove". 10 | "lastactionproxy2": Action before the last one (type-dependent). 11 | "lastactionproxy3": 3rd recent action (type-dependent). 12 | "lastactionproxy4": 4th recent action (type-dependent). 13 | "lastactionproxy5": 5th recent action (type-dependent). 14 | "timesincelastactionproxy": Time (in seconds) since last action (type-dependent), i.e., t_end_of_aggregation_interval - t_last_action. 15 | "nfirewall1": Last five values of the number of VNF instances (type-dependent). In this case, firewall. nfirewall1 = most recent one. 16 | "nfirewall2": See above. 17 | "nfirewall3": See above. 18 | "nfirewall4": See above. 19 | "nfirewall5": See above. 20 | "bwfirewall1": Last five values of the amount of requested bandwidth (type-dependent). Unit: kbps. 21 | "bwfirewall2": See above. 22 | "bwfirewall3": See above. 23 | "bwfirewall4": See above. 24 | "bwfirewall5": See above. 25 | "bwallocfirewall1": Last five values of the amount of allocated bandwidth (type-dependent). Unit: kbps. This is calculated by multiplying the number of active VNF instances with the VNF capacity. 26 | "bwallocfirewall2": See above. 27 | "bwallocfirewall3": See above. 28 | "bwallocfirewall4": See above. 29 | "bwallocfirewall5": See above. 30 | "bwtocapfirewall1": Last five values of the difference between the amount of allocated and requested bandwidth (type-dependent). 31 | "bwtocapfirewall2": See above. 32 | "bwtocapfirewall3": See above. 33 | "bwtocapfirewall4": See above. 34 | "bwtocapfirewall5": See above. 35 | "bwtoshutfirewall1": Last five values of the difference between the requested amount of bandwidth and the bandwidth level that corresponds to allocating one instance less than is currently active. For example, if we have 3 firewalls with a capacity of 900Mbps each (resulting in 2700Mbps of allocated bandwidth) and the bandwidth sum of all firewall-related requests equals, 2500Mbps, bwtoshutfirewall equals 2500Mbps - 2 * 900Mbps = 700Mbps. 36 | "bwtoshutfirewall2": See above. 37 | "bwtoshutfirewall3": See above. 38 | "bwtoshutfirewall4": See above. 39 | "bwtoshutfirewall5": See above. 40 | "bwtocapnfirewall1": Same as bwtocapfirewall1, but normalized with the capacity. 41 | "bwtocapnfirewall2": See above. 42 | "bwtocapnfirewall3": See above. 43 | "bwtocapnfirewall4": See above. 44 | "bwtocapnfirewall5": See above. 45 | "bwtoshutnfirewall1": Same as bwtoshutfirewall1, but normalized with the capacity. 46 | "bwtoshutnfirewall2": See above. 47 | "bwtoshutnfirewall3": See above. 48 | "bwtoshutnfirewall4": See above. 49 | "bwtoshutnfirewall5": See above. 50 | "utilfirewall1": Last five values of VNF utilization (type-dependent). Utilization is calculated as ratio between requested and allocation bandwidth. 51 | "utilfirewall2": See above. 52 | "utilfirewall3": See above. 53 | "utilfirewall4": See above. 54 | "utilfirewall5": See above. 55 | [The following fields are identical to the previous ones, but with other VNFs (currently considered VNFs: firewall, proxy, ids, nat).] 56 | "nids1" 57 | "nids2" 58 | "nids3" 59 | "nids4" 60 | "nids5" 61 | "bwids1" 62 | "bwids2" 63 | "bwids3" 64 | "bwids4" 65 | "bwids5" 66 | "bwallocids1" 67 | "bwallocids2" 68 | "bwallocids3" 69 | "bwallocids4" 70 | "bwallocids5" 71 | "bwtocapids1" 72 | "bwtocapids2" 73 | "bwtocapids3" 74 | "bwtocapids4" 75 | "bwtocapids5" 76 | "bwtoshutids1" 77 | "bwtoshutids2" 78 | "bwtoshutids3" 79 | "bwtoshutids4" 80 | "bwtoshutids5" 81 | "bwtocapnids1" 82 | "bwtocapnids2" 83 | "bwtocapnids3" 84 | "bwtocapnids4" 85 | "bwtocapnids5" 86 | "bwtoshutnids1" 87 | "bwtoshutnids2" 88 | "bwtoshutnids3" 89 | "bwtoshutnids4" 90 | "bwtoshutnids5" 91 | "nnat1" 92 | "nnat2" 93 | "nnat3" 94 | "nnat4" 95 | "nnat5" 96 | "bwnat1" 97 | "bwnat2" 98 | "bwnat3" 99 | "bwnat4" 100 | "bwnat5" 101 | "bwallocnat1" 102 | "bwallocnat2" 103 | "bwallocnat3" 104 | "bwallocnat4" 105 | "bwallocnat5" 106 | "bwtocapnat1" 107 | "bwtocapnat2" 108 | "bwtocapnat3" 109 | "bwtocapnat4" 110 | "bwtocapnat5" 111 | "bwtoshutnat1" 112 | "bwtoshutnat2" 113 | "bwtoshutnat3" 114 | "bwtoshutnat4" 115 | "bwtoshutnat5" 116 | "bwtocapnnat1" 117 | "bwtocapnnat2" 118 | "bwtocapnnat3" 119 | "bwtocapnnat4" 120 | "bwtocapnnat5" 121 | "bwtoshutnnat1" 122 | "bwtoshutnnat2" 123 | "bwtoshutnnat3" 124 | "bwtoshutnnat4" 125 | "bwtoshutnnat5" 126 | "nproxy1" 127 | "nproxy2" 128 | "nproxy3" 129 | "nproxy4" 130 | "nproxy5" 131 | "bwproxy1" 132 | "bwproxy2" 133 | "bwproxy3" 134 | "bwproxy4" 135 | "bwproxy5" 136 | "bwallocproxy1" 137 | "bwallocproxy2" 138 | "bwallocproxy3" 139 | "bwallocproxy4" 140 | "bwallocproxy5" 141 | "bwtocapproxy1" 142 | "bwtocapproxy2" 143 | "bwtocapproxy3" 144 | "bwtocapproxy4" 145 | "bwtocapproxy5" 146 | "bwtoshutproxy1" 147 | "bwtoshutproxy2" 148 | "bwtoshutproxy3" 149 | "bwtoshutproxy4" 150 | "bwtoshutproxy5" 151 | "bwtocapnproxy1" 152 | "bwtocapnproxy2" 153 | "bwtocapnproxy3" 154 | "bwtocapnproxy4" 155 | "bwtocapnproxy5" 156 | "bwtoshutnproxy1" 157 | "bwtoshutnproxy2" 158 | "bwtoshutnproxy3" 159 | "bwtoshutnproxy4" 160 | "bwtoshutnproxy5" 161 | "nvnfs1": Last five values of the number of VNFs (type-independent). 162 | "nvnfs2": See above. 163 | "nvnfs3": See above. 164 | "nvnfs4": See above. 165 | "nvnfs5": See above. 166 | "totalbw1": Last five values of the total amount of requested bandwidth (type-independent). 167 | "totalbw2": See above. 168 | "totalbw3": See above. 169 | "totalbw4": See above. 170 | "totalbw5": See above. 171 | "rate1": Arrival rate in the first half of the aggregation interval. 172 | "rate2": Arrival rate in the second half of the aggregation interval. 173 | "mean.nvnfs": Mean number of VNF instances during the aggregation interval (type-independent). 174 | "median.nvnfs": Median of the number of VNF instances during the aggregation interval (type-independent). 175 | "sd.nvnfs": Standard deviation of the number of VNF instances during the aggregation interval (type-independent). 176 | "cv.nvnfs": Coefficient of variation of the number of VNF instances during the aggregation interval (type-independent). 177 | "min.nvnfs": Minimum number of VNF instances during the aggregation interval (type-independent). 178 | "max.nvnfs": Maximum number of VNF instances during the aggregation interval (type-independent). 179 | "slope.nvnfs": "Slope" of . This is calculated as the difference between the mean value in the second half of the aggregation interval and the mean value in the first one. 180 | "mean.totalbw": Mean amount of requested bandwidth during the aggregation interval (type-independent). Below statistics analogous to *.nvnfs. 181 | "median.totalbw" 182 | "sd.totalbw" 183 | "cv.totalbw" 184 | "min.totalbw" 185 | "max.totalbw" 186 | "slope.totalbw" 187 | "mean.nfirewall": Mean number of VNF instances during the aggregation interval (per VNF type). Below statistics analogous to *.nvnfs. 188 | "median.nfirewall" 189 | "sd.nfirewall" 190 | "cv.nfirewall" 191 | "min.nfirewall" 192 | "max.nfirewall" 193 | "slope.nfirewall" 194 | "mean.nids" 195 | "median.nids" 196 | "sd.nids" 197 | "min.nids" 198 | "max.nids" 199 | "slope.nids" 200 | "mean.nnat" 201 | "median.nnat" 202 | "sd.nnat" 203 | "min.nnat" 204 | "max.nnat" 205 | "slope.nnat" 206 | "mean.nproxy" 207 | "median.nproxy" 208 | "sd.nproxy" 209 | "min.nproxy" 210 | "max.nproxy" 211 | "slope.nproxy" 212 | "mean.bwfirewall" 213 | "median.bwfirewall" 214 | "sd.bwfirewall" 215 | "cv.bwfirewall" 216 | "min.bwfirewall" 217 | "max.bwfirewall" 218 | "slope.bwfirewall" 219 | "mean.bwids" 220 | "median.bwids" 221 | "sd.bwids" 222 | "min.bwids" 223 | "max.bwids" 224 | "slope.bwids" 225 | "mean.bwnat" 226 | "median.bwnat" 227 | "sd.bwnat" 228 | "min.bwnat" 229 | "max.bwnat" 230 | "slope.bwnat" 231 | "mean.bwproxy" 232 | "median.bwproxy" 233 | "sd.bwproxy" 234 | "min.bwproxy" 235 | "max.bwproxy" 236 | "slope.bwproxy" 237 | "mean.bwallocfirewall" 238 | "median.bwallocfirewall" 239 | "sd.bwallocfirewall" 240 | "cv.bwallocfirewall" 241 | "min.bwallocfirewall" 242 | "max.bwallocfirewall" 243 | "slope.bwallocfirewall" 244 | "mean.bwallocids" 245 | "median.bwallocids" 246 | "sd.bwallocids" 247 | "min.bwallocids" 248 | "max.bwallocids" 249 | "slope.bwallocids" 250 | "mean.bwallocnat" 251 | "median.bwallocnat" 252 | "sd.bwallocnat" 253 | "min.bwallocnat" 254 | "max.bwallocnat" 255 | "slope.bwallocnat" 256 | "mean.bwallocproxy" 257 | "median.bwallocproxy" 258 | "sd.bwallocproxy" 259 | "min.bwallocproxy" 260 | "max.bwallocproxy" 261 | "slope.bwallocproxy" 262 | "mean.utilfirewall" 263 | "median.utilfirewall" 264 | "sd.utilfirewall" 265 | "cv.utilfirewall" 266 | "min.utilfirewall" 267 | "max.utilfirewall" 268 | "slope.utilfirewall" 269 | "mean.bwtocapfirewall" 270 | "median.bwtocapfirewall" 271 | "sd.bwtocapfirewall" 272 | "cv.bwtocapfirewall" 273 | "min.bwtocapfirewall" 274 | "max.bwtocapfirewall" 275 | "slope.bwtocapfirewall" 276 | "mean.bwtocapids" 277 | "median.bwtocapids" 278 | "sd.bwtocapids" 279 | "min.bwtocapids" 280 | "max.bwtocapids" 281 | "slope.bwtocapids" 282 | "mean.bwtocapnat" 283 | "median.bwtocapnat" 284 | "sd.bwtocapnat" 285 | "min.bwtocapnat" 286 | "max.bwtocapnat" 287 | "slope.bwtocapnat" 288 | "mean.bwtocapproxy" 289 | "median.bwtocapproxy" 290 | "sd.bwtocapproxy" 291 | "min.bwtocapproxy" 292 | "max.bwtocapproxy" 293 | "slope.bwtocapproxy" 294 | "mean.bwtocapnfirewall" 295 | "median.bwtocapnfirewall" 296 | "sd.bwtocapnfirewall" 297 | "cv.bwtocapnfirewall" 298 | "min.bwtocapnfirewall" 299 | "max.bwtocapnfirewall" 300 | "slope.bwtocapnfirewall" 301 | "mean.bwtocapnids" 302 | "median.bwtocapnids" 303 | "sd.bwtocapnids" 304 | "min.bwtocapnids" 305 | "max.bwtocapnids" 306 | "slope.bwtocapnids" 307 | "mean.bwtocapnnat" 308 | "median.bwtocapnnat" 309 | "sd.bwtocapnnat" 310 | "min.bwtocapnnat" 311 | "max.bwtocapnnat" 312 | "slope.bwtocapnnat" 313 | "mean.bwtocapnproxy" 314 | "median.bwtocapnproxy" 315 | "sd.bwtocapnproxy" 316 | "min.bwtocapnproxy" 317 | "max.bwtocapnproxy" 318 | "slope.bwtocapnproxy" 319 | "mean.bwtoshutfirewall" 320 | "median.bwtoshutfirewall" 321 | "sd.bwtoshutfirewall" 322 | "cv.bwtoshutfirewall" 323 | "min.bwtoshutfirewall" 324 | "max.bwtoshutfirewall" 325 | "slope.bwtoshutfirewall" 326 | "mean.bwtoshutids" 327 | "median.bwtoshutids" 328 | "sd.bwtoshutids" 329 | "min.bwtoshutids" 330 | "max.bwtoshutids" 331 | "slope.bwtoshutids" 332 | "mean.bwtoshutnat" 333 | "median.bwtoshutnat" 334 | "sd.bwtoshutnat" 335 | "min.bwtoshutnat" 336 | "max.bwtoshutnat" 337 | "slope.bwtoshutnat" 338 | "mean.bwtoshutproxy" 339 | "median.bwtoshutproxy" 340 | "sd.bwtoshutproxy" 341 | "min.bwtoshutproxy" 342 | "max.bwtoshutproxy" 343 | "slope.bwtoshutproxy" 344 | "mean.bwtoshutnfirewall" 345 | "median.bwtoshutnfirewall" 346 | "sd.bwtoshutnfirewall" 347 | "cv.bwtoshutnfirewall" 348 | "min.bwtoshutnfirewall" 349 | "max.bwtoshutnfirewall" 350 | "slope.bwtoshutnfirewall" 351 | "mean.bwtoshutnids" 352 | "median.bwtoshutnids" 353 | "sd.bwtoshutnids" 354 | "min.bwtoshutnids" 355 | "max.bwtoshutnids" 356 | "slope.bwtoshutnids" 357 | "mean.bwtoshutnnat" 358 | "median.bwtoshutnnat" 359 | "sd.bwtoshutnnat" 360 | "min.bwtoshutnnat" 361 | "max.bwtoshutnnat" 362 | "slope.bwtoshutnnat" 363 | "mean.bwtoshutnproxy" 364 | "median.bwtoshutnproxy" 365 | "sd.bwtoshutnproxy" 366 | "min.bwtoshutnproxy" 367 | "max.bwtoshutnproxy" 368 | "slope.bwtoshutnproxy" 369 | "changeNfirewall": Change in the number of instantiated VNFs (per VNF type), i.e., n_vnfs at the end of the aggregation interval - n_vnfs at its beginning. 370 | "changeBfirewall": Change in the total requested bandwidth (per VNF type). Unit: kbps. 371 | "changeNids" 372 | "changeBids" 373 | "changeNnat" 374 | "changeBnat" 375 | "slopei.nvnfs": Sign of slope*-features from above, i.e., 1 for values > 0, 0 for values == 0, and -1 for values < 0. 376 | "slopei.totalbw" 377 | "slopei.nfirewall" 378 | "slopei.nids" 379 | "slopei.nnat" 380 | "slopei.nproxy" 381 | "slopei.bwfirewall" 382 | "slopei.bwids" 383 | "slopei.bwnat" 384 | "slopei.bwproxy" 385 | "slopei.bwallocfirewall" 386 | "slopei.bwallocids" 387 | "slopei.bwallocnat" 388 | "slopei.bwallocproxy" 389 | "slopei.utilfirewall" 390 | "slopei.bwtocapfirewall" 391 | "slopei.bwtocapids" 392 | "slopei.bwtocapnat" 393 | "slopei.bwtocapproxy" 394 | "slopei.bwtocapnfirewall" 395 | "slopei.bwtocapnids" 396 | "slopei.bwtocapnnat" 397 | "slopei.bwtocapnproxy" 398 | "slopei.bwtoshutfirewall" 399 | "slopei.bwtoshutids" 400 | "slopei.bwtoshutnat" 401 | "slopei.bwtoshutproxy" 402 | "slopei.bwtoshutnfirewall" 403 | "slopei.bwtoshutnids" 404 | "slopei.bwtoshutnnat" 405 | "slopei.bwtoshutnproxy" 406 | "lastactionfirewall1": See lastactionproxy1. 407 | "lastactionfirewall2" 408 | "lastactionfirewall3" 409 | "lastactionfirewall4" 410 | "lastactionfirewall5" 411 | "timesincelastactionfirewall" 412 | "lastactionids1" 413 | "lastactionids2" 414 | "lastactionids3" 415 | "lastactionids4" 416 | "lastactionids5" 417 | "timesincelastactionids" 418 | "lastactionnat1" 419 | "lastactionnat2" 420 | "lastactionnat3" 421 | "lastactionnat4" 422 | "lastactionnat5" 423 | "timesincelastactionnat" 424 | "meangrowthrate": Difference between the mean arrival rate and the mean departure rate. 425 | "rateslope": Slope of the arrival rate. 426 | "rateslopei": Sign of the slope of the arrival rate. 427 | "action": Action regarding the firewall VNF. Takes on values "add", "remove", and "doNothing" - indicating that the number of instantiated VNFs between the current time t_cur and the time (t_cur + p) increased, decreased, or did not change, respectively. In the previous statement, p denotes the prediction time (how "far" we predict). 428 | -------------------------------------------------------------------------------- /code/full-pipeline/inet2: -------------------------------------------------------------------------------- 1 | 12 15 2 | 0 0 3 | 1 160 4 | 2 0 5 | 3 160 6 | 4 160 7 | 5 160 8 | 6 0 9 | 7 0 10 | 8 0 11 | 9 160 12 | 10 160 13 | 11 160 14 | 0 1 10000000 1 15 | 1 4 10000000 117 16 | 1 5 10000000 58 17 | 1 11 10000000 84 18 | 2 5 10000000 26 19 | 2 8 10000000 70 20 | 3 6 10000000 63 21 | 3 9 10000000 129 22 | 3 10 10000000 209 23 | 4 6 10000000 90 24 | 4 7 10000000 189 25 | 5 6 10000000 54 26 | 7 9 10000000 36 27 | 8 11 10000000 23 28 | 9 10 10000000 86 29 | -------------------------------------------------------------------------------- /code/full-pipeline/mec: -------------------------------------------------------------------------------- 1 | 16 15 2 | 0 0 3 | 1 0 4 | 2 0 5 | 3 0 6 | 4 0 7 | 5 0 8 | 6 40 9 | 7 40 10 | 8 40 11 | 9 40 12 | 10 40 13 | 11 40 14 | 12 40 15 | 13 40 16 | 14 0 17 | 15 400 18 | 0 2 10000000 30 19 | 0 3 10000000 30 20 | 1 4 10000000 30 21 | 1 5 10000000 30 22 | 0 14 100000000 50 23 | 1 14 100000000 50 24 | 2 6 10000000 10 25 | 2 7 10000000 10 26 | 3 8 10000000 10 27 | 3 9 10000000 10 28 | 4 10 10000000 10 29 | 4 11 10000000 10 30 | 5 12 10000000 10 31 | 5 13 10000000 10 32 | 14 15 100000000 80 33 | -------------------------------------------------------------------------------- /code/full-pipeline/middlebox-spec: -------------------------------------------------------------------------------- 1 | firewall,4,45,900000,0.0 2 | proxy,4,40,900000,0.0 3 | ids,8,1,600000,0.0 4 | nat,1,10,900000,0.0 5 | wano,4,5,400000,0.0 6 | -------------------------------------------------------------------------------- /code/full-pipeline/runwk_template.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Number of CPU cores to use. 4 | NCORES=18 5 | 6 | # Topology file. 7 | # Currently available options: inet2, mec. 8 | TOPOFILE="inet2" 9 | 10 | # Distribution of request interarrival time and flow duration. 11 | # Currently available options: exp, norm. 12 | DIST="norm" 13 | 14 | # Week identifier, used as seed (one run is typically used to generate 15 | # requests over the span of one week ~ 10k minutes). 16 | WEEK=1 17 | NMINUTES=10000 18 | 19 | # Suffix used for consistent file naming scheme. 20 | SUFFIX="wk$WEEK""_sfccat_$DIST""_$TOPOFILE" 21 | 22 | # Location of CPLEX log files. 23 | LOGFILEPATH="../../solver/middlebox-placement/src/logs/" 24 | 25 | # File names of generated output files (requests, logs, solver results). 26 | REQSRDS="requests-$SUFFIX.rds" 27 | REQSTXT="requests-$SUFFIX.txt" 28 | NVNFSFILE="$LOGFILEPATH""nvnfs-$SUFFIX.txt" 29 | ROUTEINFOFILE="$LOGFILEPATH""routeinfo-$SUFFIX.txt" 30 | NODEINFOFILE="$LOGFILEPATH""nodeinfo-$SUFFIX.txt" 31 | SOLVERRESOUTFILE="solver-results-$SUFFIX.rds" 32 | 33 | # Run the individual steps from generating the request trace to obtaining solver results. 34 | Rscript 001_generateSFCRTrace.R --rdsOutfile $REQSRDS --txtOutfile $REQSTXT --topoFile $TOPOFILE --nMinutes $NMINUTES --seed $WEEK --iatDist $DIST --durDist $DIST 35 | python3 002_sfcrTrace2CPLEX.py --sfcrTraceFile $REQSTXT --outFilePath $LOGFILEPATH --suffix $SUFFIX 36 | python3 003_runCPLEXPar.py --trafficListPath $LOGFILEPATH --topoFile $TOPOFILE --suffix $SUFFIX --ncores $NCORES 37 | python3 004_getPlacementInfoFromCPLEXLogs.py --logFilePath $LOGFILEPATH --suffix $SUFFIX 38 | Rscript 005_processSolverResults.R --requestsFile $REQSRDS --nvnfsFile $NVNFSFILE --routeinfoFile $ROUTEINFOFILE --nodeinfoFile $NODEINFOFILE --solverResultsOutFile $SOLVERRESOUTFILE 39 | 40 | # Generation of training data. 41 | 42 | # Prediction horizon. 43 | TPRED="60" 44 | # Tag for training files (since the same scenario / week can be used 45 | # to generate different training data sets, TAG is different from SUFFIX). 46 | TAG="p$TPRED" 47 | TRAININGDATAOUTFILE="data/training-$SUFFIX-$TAG.rds" 48 | 49 | # Export statements to make variables available in the subshells that are opened via parallel. 50 | export SOLVERRESOUTFILE 51 | export SUFFIX 52 | export TPRED 53 | export WEEK 54 | export NCORES 55 | export REQSRDS 56 | 57 | parallel -j $NCORES 'SLICEID={}; TRAININGDATAOUTFILE="training-$SUFFIX-slice-$SLICEID.rds"; Rscript 006a_generateTrainingData.R --tPred $TPRED --requestsFile $REQSRDS --solverResultsFile $SOLVERRESOUTFILE --trainingDataOutFile $TRAININGDATAOUTFILE --seed $WEEK --nSlices $NCORES --sliceID $SLICEID' ::: `seq 1 1 $NCORES` 58 | Rscript 006b_mergeTrainingData.R --trainingDataFilePattern "training-$SUFFIX-slice-*" --trainingDataOutFile $TRAININGDATAOUTFILE 59 | 60 | # Slices can be removed upon completion, i.e., "rm training-$SUFFIX-slice-*". 61 | -------------------------------------------------------------------------------- /solver/patch_e4bd6e8_to_ni-version.patch: -------------------------------------------------------------------------------- 1 | From 0104ed6ec7b69e0b78e967d9d329c6be63f70569 Mon Sep 17 00:00:00 2001 2 | From: stas 3 | Date: Mon, 1 Jul 2019 11:29:56 +0900 4 | Subject: [PATCH] Changes for deployment module. 5 | 6 | --- 7 | .gitignore | 8 +++- 8 | data-generator/traffic-generator-internet2.cc | 2 +- 9 | patch_e4bd6e8_to_ni-version.patch | 0 10 | src/CMakeLists.txt | 20 +++++++++ 11 | src/Makefile | 5 ++- 12 | .../CMakeFiles/clion-log.txt | 1 + 13 | src/cplex5-pz.h | 43 +++++++++++++++---- 14 | src/inet2 | 28 ++++++++++++ 15 | src/mec | 32 ++++++++++++++ 16 | src/middlebox-spec | 9 ++-- 17 | src/middleman.cc | 4 +- 18 | src/util.h | 16 +++++++ 19 | src/viterbi.h | 14 ++++++ 20 | 13 files changed, 163 insertions(+), 19 deletions(-) 21 | create mode 100644 patch_e4bd6e8_to_ni-version.patch 22 | create mode 100644 src/CMakeLists.txt 23 | create mode 100644 src/cmake-build-debug/CMakeFiles/clion-log.txt 24 | create mode 100644 src/inet2 25 | create mode 100644 src/mec 26 | 27 | diff --git a/.gitignore b/.gitignore 28 | index 998250b..a2bd5ff 100644 29 | --- a/.gitignore 30 | +++ b/.gitignore 31 | @@ -1,3 +1,7 @@ 32 | -src/*.o 33 | +*.log 34 | +src/.idea 35 | +src/log* 36 | +src/nvnfs.txt 37 | +src/sfcr* 38 | +src/traf-* 39 | src/middleman 40 | - 41 | diff --git a/data-generator/traffic-generator-internet2.cc b/data-generator/traffic-generator-internet2.cc 42 | index ffde8d5..62a3ce3 100644 43 | --- a/data-generator/traffic-generator-internet2.cc 44 | +++ b/data-generator/traffic-generator-internet2.cc 45 | @@ -67,7 +67,7 @@ int main(int argc, char *argv[]) { 46 | FILE *ofp = fopen("traffic-request", "w"); 47 | for (int i = 1, current_time = 0; i <= kMaxTrafficFileIndex; ++i) { 48 | std::string kTrafficFileName = 49 | - "/home/sr2chowd/UW/middlebox-placement/traffic-data-yzhang/X"; 50 | + "/mnt/e/data/abilene/X"; 51 | kTrafficFileName += (i <= 9 ? "0" : "") + std::to_string(i); 52 | printf("Openning %s\n", kTrafficFileName.c_str()); 53 | FILE *ifp = fopen(kTrafficFileName.c_str(), "r"); 54 | diff --git a/patch_e4bd6e8_to_ni-version.patch b/patch_e4bd6e8_to_ni-version.patch 55 | new file mode 100644 56 | index 0000000..e69de29 57 | diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt 58 | new file mode 100644 59 | index 0000000..0817bfc 60 | --- /dev/null 61 | +++ b/src/CMakeLists.txt 62 | @@ -0,0 +1,20 @@ 63 | +# cmake_minimum_required(VERSION ) 64 | +project(src) 65 | + 66 | +set(CMAKE_CXX_STANDARD 14) 67 | + 68 | +include_directories(.) 69 | + 70 | +add_executable(src 71 | + cplex4-hb.h 72 | + cplex4-hw.h 73 | + cplex4-pz.h 74 | + cplex4.h 75 | + cplex5-pz.h 76 | + cplex5.h 77 | + datastructure.h 78 | + io.h 79 | + log_processor.cc 80 | + middleman.cc 81 | + util.h 82 | + viterbi.h) 83 | diff --git a/src/Makefile b/src/Makefile 84 | index cec99b0..901e96d 100644 85 | --- a/src/Makefile 86 | +++ b/src/Makefile 87 | @@ -1,7 +1,8 @@ 88 | -LIB_PATHS = -L$(CPLEX_ROOT)/cplex/lib/x86-64_sles10_4.1/static_pic -L$(CPLEX_ROOT)/concert/lib/x86-64_sles10_4.1/static_pic 89 | +#LIB_PATHS = -L$(CPLEX_ROOT)/cplex/lib/x86-64_sles10_4.1/static_pic -L$(CPLEX_ROOT)/concert/lib/x86-64_sles10_4.1/static_pic 90 | +LIB_PATHS = -L$(CPLEX_ROOT)/cplex/lib/x86-64_linux/static_pic -L$(CPLEX_ROOT)/concert/lib/x86-64_linux/static_pic 91 | 92 | INCLUDE_PATHS = -I$(CPLEX_ROOT)/cplex/include -I$(CPLEX_ROOT)/concert/include 93 | -LIBS = -lilocplex -lconcert -lcplex -lm -lpthread 94 | +LIBS = -lilocplex -lconcert -lcplex -lm -lpthread -ldl 95 | DFLAGS = -DIL_STD -DCPLEX_PZ 96 | FILES = middleman.cc 97 | 98 | diff --git a/src/cmake-build-debug/CMakeFiles/clion-log.txt b/src/cmake-build-debug/CMakeFiles/clion-log.txt 99 | new file mode 100644 100 | index 0000000..c7ba5e8 101 | --- /dev/null 102 | +++ b/src/cmake-build-debug/CMakeFiles/clion-log.txt 103 | @@ -0,0 +1 @@ 104 | +Toolchains are not configured Configure 105 | diff --git a/src/cplex5-pz.h b/src/cplex5-pz.h 106 | index 8c6c809..f9c025b 100644 107 | --- a/src/cplex5-pz.h 108 | +++ b/src/cplex5-pz.h 109 | @@ -80,20 +80,22 @@ void run_cplex(std::vector traffic_requests, double &opex, 110 | string topology_filename) { 111 | 112 | bool atleast_one_solution_found = false; 113 | - for (int time_limit = 10; time_limit < 90; time_limit += 10) { 114 | + for (int time_limit = 80000; time_limit <= 320000; time_limit *= 2) { 115 | // for (int time_limit = 100; time_limit < 180; time_limit += 10) { 116 | auto solution_start_time = std::chrono::high_resolution_clock::now(); 117 | 118 | int lower_bound = 1; 119 | - int upper_bound = traffic_requests.size() * 3; 120 | + int upper_bound = traffic_requests.size() * 4; 121 | 122 | int max_vnf = ceil((lower_bound + upper_bound) / 2.0); 123 | + // Avoid skipping some values during binary search. 124 | + double max_vnf_dbl = (lower_bound + upper_bound) / 2.0; 125 | int last_max_vnf = 0; 126 | 127 | int is_feasible = 0; // 0 = Infeasible, 1 = feasible 128 | 129 | while (max_vnf != last_max_vnf) { 130 | - // cout << "Run Max VNF = " << max_vnf << endl; 131 | + // cout << "Run Max VNF = " << max_vnf << " Time Limit = " << time_limit << endl; 132 | last_max_vnf = max_vnf; 133 | run_cplex_opt(traffic_requests, opex, opex_breakdown, running_time, 134 | sequence, path, all_edges, delays, utilization, 135 | @@ -103,9 +105,12 @@ void run_cplex(std::vector traffic_requests, double &opex, 136 | upper_bound = max_vnf; 137 | atleast_one_solution_found = true; 138 | } else { 139 | - lower_bound = max_vnf; 140 | + // lower_bound = max_vnf; 141 | + lower_bound = floor(max_vnf_dbl); 142 | } 143 | max_vnf = ceil((lower_bound + upper_bound) / 2.0); 144 | + max_vnf_dbl = (lower_bound + upper_bound) / 2.0; 145 | + // cout << "[bounds] Max VNF " << max_vnf << " || lb = " << lower_bound << " || ub = " << upper_bound << endl; 146 | } 147 | 148 | if (atleast_one_solution_found) { 149 | @@ -1135,9 +1140,9 @@ void run_cplex_opt(std::vector traffic_requests, double &opex, 150 | const IloInt timeLimit = time_limit; // in seconds 151 | const IloNum relativeGap = 152 | 0.001; // find Integer solution within 0.1% of optimal 153 | - cplex.setParam(IloCplex::TiLim, timeLimit); 154 | - // cplex.setParam(IloCplex::EpGap, relativeGap); 155 | - // cplex.setParam(IloCplex::Threads, 2); 156 | + cplex.setParam(IloCplex::DetTiLim, timeLimit); 157 | + //cplex.setParam(IloCplex::EpGap, relativeGap); 158 | + cplex.setParam(IloCplex::Threads, 1); 159 | cplex.setParam(IloCplex::MemoryEmphasis, true); 160 | cplex.setParam(IloCplex::PreDual, true); 161 | if (!cplex.solve()) { 162 | @@ -1193,6 +1198,9 @@ void run_cplex_opt(std::vector traffic_requests, double &opex, 163 | cplex.getValues(ym, ym_vals); 164 | std: 165 | string type = ""; 166 | + // 167 | + int nmboxes = 0; 168 | + // 169 | for (int m = 0, sw; m < kMboxCount; ++m) { 170 | if (fabs(ym_vals[m] - 1) < EPS) { 171 | if (mboxType[m] == 0) { 172 | @@ -1207,11 +1215,22 @@ void run_cplex_opt(std::vector traffic_requests, double &opex, 173 | sw = switch4mbox[m]; 174 | DEBUG("Middlebox %d (%s) is active on switch %d\n", m, type.c_str(), 175 | sw); 176 | + 177 | + // 178 | + if (mboxType[m] > 1) { 179 | + DEBUG("[vnfloc] mbox %s; node %d\n", type.c_str(), sw + 1); 180 | + nmboxes++; 181 | + } 182 | + // 183 | } 184 | } 185 | + // 186 | + DEBUG("[vnfloc] nmboxes: %d\n", nmboxes); 187 | + // 188 | 189 | // print ztn_n 190 | // cout << endl; 191 | + DEBUG("[routeinfo] start\n"); 192 | for (int t = 0; t < kTrafficCount; ++t) { 193 | for (int n = 0; n < trafficNodeCount[t]; ++n) { 194 | IloNumArray ztn_n_vals(env, kSwitchCount); 195 | @@ -1222,7 +1241,15 @@ void run_cplex_opt(std::vector traffic_requests, double &opex, 196 | "Traffic %d node %d provisioned on switch %d pseudo-switch " 197 | "%d\n", 198 | t, n, _s, _s); 199 | - } 200 | + // 201 | + if (n > 0 && n < trafficNodeCount[t] - 1) { 202 | + // t + 1 for mapping to our requestIDs whose index starts with 1. 203 | + // n for mbox number is okay since it actually starts with node 0 for the request's source node. 204 | + // _s + 1 for mapping to our nodeIDs whose index starts with 1. 205 | + DEBUG("[routeinfo] demand %d; mbox %d; node %d\n", t + 1, n, _s + 1); 206 | + } 207 | + // 208 | +} 209 | } 210 | } 211 | } 212 | diff --git a/src/inet2 b/src/inet2 213 | new file mode 100644 214 | index 0000000..9302b65 215 | --- /dev/null 216 | +++ b/src/inet2 217 | @@ -0,0 +1,28 @@ 218 | +12 15 219 | +0 0 220 | +1 160 221 | +2 0 222 | +3 160 223 | +4 160 224 | +5 160 225 | +6 0 226 | +7 0 227 | +8 0 228 | +9 160 229 | +10 160 230 | +11 160 231 | +0 1 10000000 1 232 | +1 4 10000000 117 233 | +1 5 10000000 58 234 | +1 11 10000000 84 235 | +2 5 10000000 26 236 | +2 8 10000000 70 237 | +3 6 10000000 63 238 | +3 9 10000000 129 239 | +3 10 10000000 209 240 | +4 6 10000000 90 241 | +4 7 10000000 189 242 | +5 6 10000000 54 243 | +7 9 10000000 36 244 | +8 11 10000000 23 245 | +9 10 10000000 86 246 | diff --git a/src/mec b/src/mec 247 | new file mode 100644 248 | index 0000000..c691ddc 249 | --- /dev/null 250 | +++ b/src/mec 251 | @@ -0,0 +1,32 @@ 252 | +16 15 253 | +0 0 254 | +1 0 255 | +2 0 256 | +3 0 257 | +4 0 258 | +5 0 259 | +6 40 260 | +7 40 261 | +8 40 262 | +9 40 263 | +10 40 264 | +11 40 265 | +12 40 266 | +13 40 267 | +14 0 268 | +15 400 269 | +0 2 10000000 30 270 | +0 3 10000000 30 271 | +1 4 10000000 30 272 | +1 5 10000000 30 273 | +0 14 100000000 50 274 | +1 14 100000000 50 275 | +2 6 10000000 10 276 | +2 7 10000000 10 277 | +3 8 10000000 10 278 | +3 9 10000000 10 279 | +4 10 10000000 10 280 | +4 11 10000000 10 281 | +5 12 10000000 10 282 | +5 13 10000000 10 283 | +14 15 100000000 80 284 | diff --git a/src/middlebox-spec b/src/middlebox-spec 285 | index 35b97d6..7486340 100644 286 | --- a/src/middlebox-spec 287 | +++ b/src/middlebox-spec 288 | @@ -1,4 +1,5 @@ 289 | -firewall,4,45,900000,0.0 290 | -proxy,4,40,900000,0.0 291 | -nat,1,10,900000,0.0 292 | -ids,8,1,600000,0.0 293 | +firewall,4,45,900000,0.0 294 | +proxy,4,40,900000,0.0 295 | +ids,8,1,600000,0.0 296 | +nat,1,10,900000,0.0 297 | +wano,4,5,400000,0.0 298 | diff --git a/src/middleman.cc b/src/middleman.cc 299 | index 79c6140..0243c91 100644 300 | --- a/src/middleman.cc 301 | +++ b/src/middleman.cc 302 | @@ -277,8 +277,8 @@ int main(int argc, char *argv[]) { 303 | elapsed_time += solution_time; 304 | UpdateResources(result.get(), traffic_requests[i]); 305 | RefreshServerStats(current_time); 306 | - // printf("i = %d, %s\n", i, 307 | - // traffic_requests[i].GetDebugString().c_str()); 308 | + printf("i = %d, %s\n", i, 309 | + traffic_requests[i].GetDebugString().c_str()); 310 | // Progress bar 311 | if (i % 500 == 0) { 312 | double percentage_completed = 100.0 * static_cast(i) / 313 | diff --git a/src/util.h b/src/util.h 314 | index 650c9f1..c383c08 100644 315 | --- a/src/util.h 316 | +++ b/src/util.h 317 | @@ -212,12 +212,28 @@ int UsedMiddleboxIndex(int current_node, const middlebox &m_box, 318 | 319 | void UpdateMiddleboxInstances(int current_node, const middlebox *m_box, 320 | const traffic_request &t_request) { 321 | + 322 | + // m_box.middlebox_name.c_str() 323 | + DEBUG("[xxx] Placing request (arrivalTime: %d, src: %d, dst: %d, minBw: %d, maxDelay: %d, SFC: [%d, %d, %d])\n", t_request.arrival_time, 324 | + t_request.source, t_request.destination, t_request.min_bandwidth, t_request.max_delay, 325 | + t_request.middlebox_sequence[0], t_request.middlebox_sequence[1], t_request.middlebox_sequence[2]); 326 | + DEBUG("[UpdateMiddleboxInstances] node: %d, mbox: %s\n", current_node, m_box->middlebox_name.c_str()); 327 | + // 328 | + 329 | int used_middlebox_index = 330 | UsedMiddleboxIndex(current_node, *m_box, t_request); 331 | if (used_middlebox_index != NIL) { 332 | + // 333 | + DEBUG("[UpdateMiddleboxInstances] using existing mbox\n"); 334 | + DEBUG("[xxx] Using existing %s VNF instance on node %d.\n", m_box->middlebox_name.c_str(), current_node); 335 | + // 336 | deployed_mboxes[current_node][used_middlebox_index].residual_capacity -= 337 | t_request.min_bandwidth; 338 | } else { 339 | + // 340 | + DEBUG("[UpdateMiddleboxInstances] placing new mbox\n"); 341 | + DEBUG("[xxx] Placing new %s VNF instance on node %d.\n", m_box->middlebox_name.c_str(), current_node); 342 | + // 343 | deployed_mboxes[current_node].emplace_back( 344 | m_box, m_box->processing_capacity - t_request.min_bandwidth); 345 | ReduceNodeCapacity(current_node, *m_box); 346 | diff --git a/src/viterbi.h b/src/viterbi.h 347 | index 4fd86ca..4fa7e61 100644 348 | --- a/src/viterbi.h 349 | +++ b/src/viterbi.h 350 | @@ -81,6 +81,9 @@ std::unique_ptr > ViterbiCompute( 351 | if (new_middlebox_deployed) { 352 | current_vector[current_node].cpu_cores[current_node] -= 353 | m_box.cpu_requirement; 354 | + // or current_vector[current_node].*? 355 | + // DEBUG("[stas] ? placing new middlebox %s at node %d\n", m_box.middlebox_name.c_str(), current_node); 356 | + // 357 | } 358 | } else { 359 | current_vector[current_node].cpu_cores.clear(); 360 | @@ -118,6 +121,17 @@ std::unique_ptr > ViterbiCompute( 361 | return_vector->push_back(t_request.source); 362 | std::reverse(return_vector->begin(), return_vector->end()); 363 | return_vector->push_back(t_request.destination); 364 | + 365 | + // 366 | + DEBUG("[stas] vector content\n"); 367 | + for (int i = 0; i < return_vector->size(); ++i) { 368 | + //if (i != 0) DEBUG(","); 369 | + DEBUG("[stas] %d\n", return_vector->at(i)); 370 | + } 371 | + DEBUG("[stas] vector content end\n"); 372 | + //DEBUG("Vector: ", return_vector->); 373 | + // 374 | + 375 | return std::move(return_vector); 376 | } 377 | 378 | -- 379 | 2.17.1 380 | 381 | -------------------------------------------------------------------------------- /solver/readme.txt: -------------------------------------------------------------------------------- 1 | Original repository at https://github.com/srcvirus/middlebox-placement 2 | Minor adaptations, e.g., for extended logging, deterministic time constraints. 3 | 4 | After cloning the above repository, change into the middlebox-placement folder, and apply the patch: 5 | 6 | * git apply --stat ../patch_e4bd6e8_to_ni-version.patch 7 | * git apply --check ../patch_e4bd6e8_to_ni-version.patch 8 | * git am --signoff < ../patch_e4bd6e8_to_ni-version.patch 9 | 10 | Compile by running "make dbg" in middlebox-placement/src. 11 | --------------------------------------------------------------------------------