├── LICENSE ├── README.md ├── python ├── check-best-perf.py ├── dsp-eval.py ├── estimator-eval.py ├── lryzx.py └── rtb-test.py ├── results ├── rtb.results.1458.best.perf.tsv └── rtb.results.1458.tsv └── scripts ├── check-best-perf.sh ├── demo.sh ├── run-lr.sh └── run-rtb-test.sh /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Optimal Real-Time Bidding for Display Advertising 2 | =========== 3 | 4 | A benchmarking framework supporting the experiments of real-time bidding optimisation. A description of such problem and benchmarking experiment is [here](http://arxiv.org/abs/1407.7073). 5 | 6 | In the current version, we implemented the feature engineering for logistic regression CTR estimator and standard bidding functions as described in the above reference. The state-of-the-art bidding function proposed in KDD'14 paper [Optimal Real-Time Bidding for Display Advertising](http://www0.cs.ucl.ac.uk/staff/w.zhang/papers/ortb-kdd.pdf) is planned to be published here in Dec. 2014. 7 | 8 | For any problems, please report the issues here or contact [Weinan Zhang](http://www0.cs.ucl.ac.uk/staff/w.zhang/). 9 | 10 | ### Feature Engineering Code 11 | Please check our GitHub project [make-ipinyou-data](https://github.com/wnzhang/make-ipinyou-data). After downloading the dataset, by simplying `make all` you can generate the standardised data which will be used in the bid optimisation tasks. 12 | 13 | ### Run Bid Optimisation 14 | After successfully generating the data, let's suppose `make-ipinyou-data` project is placed in the same folder of `optimal-rtb` project, just like this: 15 | ``` 16 | weinan@ZHANG:~/Project$ ls 17 | optimal-rtb make-ipinyou-data 18 | ``` 19 | Then under `optimal-rtb/scripts/` you can simply run `bash demo.sh` to train a logistic regression CTR estimator for one campaign data generated in `make-ipinyou-data` and then perform several bid optimisation algorithms with different parameters. 20 | 21 | After running the demo, you could find a file `results/rtb.results.1458.best.perf.tsv` containing the best performance for each bidding algorithm. 22 | ``` 23 | prop clks bids imps budget spend algo para 24 | 16 51 227472 97784 2826028 2826032 const 55 25 | 16 482 614638 89408 2826028 2804895 lin 130 26 | 16 473 614638 38858 2826028 816763 mcpc 1 27 | 16 50 199209 77751 2826028 2826041 rand 100 28 | 64 16 472727 57030 706507 706508 const 20 29 | 64 471 614638 31908 706507 628971 lin 60 30 | 64 330 424901 32372 706507 706520 mcpc 1 31 | 64 14 614638 52079 706507 689611 rand 30 32 | ``` 33 | Here 'prop' means the proportion between the original cost in the test data and the set budget in the test environment. See [our benchmarking paper](http://arxiv.org/abs/1407.7073) for details. 34 | 35 | ### Misc 36 | The code implementation of generating features has a little difference between that in [our benchmarking paper](http://arxiv.org/abs/1407.7073): in the orginal dataset, there could be multiple click events for one impression event. In the benchmarking paper, the multiple clicks are directly counted as the number is. In the `make-ipinyou-data` project, we only count for 1 click even if there are more than one clicks. 37 | -------------------------------------------------------------------------------- /python/check-best-perf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import sys 3 | import random 4 | 5 | 6 | if len(sys.argv) < 2: 7 | print 'Usage: python check-best-perf.py rtb.result.1458.txt' 8 | exit(-1) 9 | 10 | setting_row = {} 11 | setting_perf = {} 12 | 13 | # setting is (proportion, algorithm) 14 | 15 | fi = open(sys.argv[1], 'r') # rtb.result.1458.txt 16 | fo = open(sys.argv[1].replace('.tsv', '.best.perf.tsv'), 'w') 17 | first = True 18 | for line in fi: 19 | line = line.strip() 20 | s = line.split('\t') 21 | if first: 22 | first = False 23 | fo.write(line + '\n') 24 | continue 25 | algo = s[6] 26 | prop = s[0] 27 | perf = int(s[1]) 28 | setting = (prop, algo) 29 | if setting in setting_perf and perf > setting_perf[setting] or setting not in setting_perf: 30 | setting_perf[setting] = perf 31 | setting_row[setting] = line 32 | fi.close() 33 | for setting in sorted(setting_perf): 34 | fo.write(setting_row[setting] + '\n') 35 | fo.close() -------------------------------------------------------------------------------- /python/dsp-eval.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import sys 3 | import random 4 | 5 | random.seed(10) 6 | 7 | def const(bid): 8 | return bid 9 | 10 | def rand(upper): 11 | return int(random.random() * upper) 12 | 13 | def mcpc(ecpc, pctr): 14 | return int(ecpc * pctr) 15 | 16 | def lin(pctr, basectr, basebid): 17 | return int(pctr * basebid / basectr) 18 | 19 | def win(ccfme, bid): 20 | return bid >= ccfme[2] and bid > ccfme[3] 21 | 22 | if len(sys.argv) < 6: 23 | print 'Usage: train.log.txt test.log.txt test.lr.txt.pred test.gbrt.txt.pred rtb-result.txt' 24 | exit(-1) 25 | 26 | ccfm = [] # clk cnv floor market 27 | lrpctrs = [] 28 | gbrtpctrs = [] 29 | totalcost = 0 30 | tecpc = 0. 31 | tctr = 0. 32 | 33 | # read in train data for tecpc and tctr 34 | fi = open(sys.argv[1], 'r') 35 | first = True 36 | num = 0 37 | for line in fi: 38 | s = line.split('\t') 39 | if first: 40 | first = False 41 | continue 42 | clk = int(s[0]) 43 | cost = int(s[23]) 44 | num += 1 45 | tctr += clk 46 | tecpc += cost 47 | fi.close() 48 | tecpc /= tctr 49 | tctr /= num 50 | 51 | # read in test data 52 | fi = open(sys.argv[2], 'r') 53 | first = True 54 | for line in fi: 55 | s = line.split('\t') 56 | if first: 57 | first = False 58 | continue 59 | clk = int(s[27]) 60 | cnv = int(s[28]) 61 | floorprice = int(s[20]) 62 | marketprice = int(s[23]) 63 | ccfm.append((clk, cnv, floorprice, marketprice)) 64 | totalcost+= marketprice 65 | fi.close() 66 | 67 | # read in lr pctr 68 | fi = open(sys.argv[3], 'r') 69 | for line in fi: 70 | lrpctrs.append(float(line.strip())) 71 | fi.close() 72 | 73 | # read in gbrt pctr 74 | fi = open(sys.argv[4], 'r') 75 | for line in fi: 76 | gbrtpctrs.append(float(line.strip())) 77 | fi.close() 78 | 79 | # rock! 80 | budgetProportions = [32, 8, 2] 81 | constParas = range(2, 20, 2) + range(20, 100, 5) + range(100, 301, 10) #[2, 4, 6, 8, 10, 20, 30, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 260, 280, 300] 82 | randParas = range(2, 20, 2) + range(20, 100, 5) + range(100, 501, 10) #[5, 10, 20, 30, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 260, 280, 300, 350, 400, 450, 500] 83 | mcpcParas = [1] 84 | linParas = range(2, 20, 2) + range(20, 200, 5) + range(200, 300, 10) + range(300, 501, 25) # [10, 15, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 220, 250, 270, 300, 350, 400, 450, 500] 85 | algoParas = {"const":constParas, "rand":randParas, "mcpc-lr":mcpcParas, "mcpc-gbrt":mcpcParas, "lin-lr":linParas, "lin-gbrt":linParas} 86 | 87 | # output format 88 | # budgetProportion clk cnv bid imp budget spend para 89 | def simulateABiddingStategyWithParameter(cases, tcost, proportion, algo, para): 90 | budget = int(tcost / proportion) 91 | cost = 0 92 | clks = 0 93 | cnvs = 0 94 | bids = 0 95 | imps = 0 96 | for idx in range(0, len(cases)): 97 | bid = 0 98 | if algo == "const": 99 | bid = const(para) 100 | elif algo == "rand": 101 | bid = rand(para) 102 | elif algo == "mcpc-lr": 103 | lrpctr = lrpctrs[idx] 104 | bid = mcpc(tecpc, lrpctr) 105 | elif algo == "mcpc-gbrt": 106 | gbrtpctr = gbrtpctrs[idx] 107 | bid = mcpc(tecpc, gbrtpctr) 108 | elif algo == "lin-lr": 109 | lrpctr = lrpctrs[idx] 110 | bid = lin(lrpctr, tctr, para) 111 | elif algo == "lin-gbrt": 112 | gbrtpctr = gbrtpctrs[idx] 113 | bid = lin(gbrtpctr, tctr, para) 114 | else: 115 | print 'wrong algo' 116 | sys.exit(-1) 117 | bids += 1 118 | case = cases[idx] 119 | if win(case, bid): 120 | imps += 1 121 | clks += case[0] 122 | cnvs += case[1] 123 | cost += case[3] 124 | if cost > budget: 125 | break 126 | return str(proportion) + '\t' + str(clks) + '\t' + str(cnvs) + '\t' + str(bids) + '\t' + \ 127 | str(imps) + '\t' + str(budget) + '\t' + str(cost) + '\t' + algo + '\t'+ str(para) 128 | 129 | def simulateABiddingStrategy(cases, tcost, proportion, algo, writer): 130 | paras = algoParas[algo] 131 | for para in paras: 132 | res = simulateABiddingStategyWithParameter(cases, tcost, proportion, algo, para) 133 | print res 134 | writer.write(res + '\n') 135 | 136 | fo = open(sys.argv[5], 'w') 137 | header = "prop\tclk\tcnv\tbid\timp\tbudget\tspend\talgo\tpara" 138 | fo.write(header + "\n") 139 | print header 140 | for bp in budgetProportions: 141 | for algo in algoParas: 142 | simulateABiddingStrategy(ccfm, totalcost, bp, algo, fo) 143 | fo.close() 144 | -------------------------------------------------------------------------------- /python/estimator-eval.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import sys 3 | import random 4 | import math 5 | import operator 6 | from sklearn.metrics import roc_auc_score 7 | from sklearn.metrics import mean_squared_error 8 | 9 | def ints(s): 10 | res = [] 11 | for ss in s: 12 | res.append(int(ss)) 13 | return res 14 | 15 | def sigmoid(p): 16 | return 1.0 / (1.0 + math.exp(-p)) 17 | 18 | def estimator_lr(feats): 19 | pred = 0.0 20 | for feat in feats: 21 | if feat in featWeight: 22 | pred += featWeight[feat] 23 | pred = sigmoid(pred) 24 | return pred 25 | 26 | random.seed(10) 27 | 28 | if len(sys.argv) < 3: 29 | print 'Usage: test.yzx.txt train.yzx.txt.lr.weight' 30 | exit(-1) 31 | 32 | y = [] 33 | yp = [] 34 | featWeight = {} 35 | #initialize the lr 36 | 37 | fi = open(sys.argv[2], 'r') 38 | for line in fi: 39 | s = line.strip().split() 40 | feat = int(s[0]) 41 | weight = float(s[1]) 42 | featWeight[feat] = weight 43 | fi.close() 44 | 45 | fi = open(sys.argv[1], 'r') 46 | for line in fi: 47 | data = ints(line.strip().replace(":1", "").split()) 48 | clk = data[0] 49 | mp = data[1] 50 | fsid = 2 # feature start id 51 | feats = data[fsid:] 52 | pred = estimator_lr(feats) 53 | y.append(clk) 54 | yp.append(pred) 55 | fi.close() 56 | 57 | # evaluation 58 | auc = roc_auc_score(y, yp) 59 | rmse = math.sqrt(mean_squared_error(y, yp)) 60 | print "algo\tauc\trmse" 61 | print "lr" + '\t' + str(auc) + '\t' + str(rmse) 62 | 63 | -------------------------------------------------------------------------------- /python/lryzx.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import sys 3 | import random 4 | import math 5 | import operator 6 | from sklearn.metrics import roc_auc_score 7 | from sklearn.metrics import mean_squared_error 8 | 9 | 10 | bufferCaseNum = 1000000 11 | eta = 0.01 12 | lamb = 1E-6 13 | featWeight = {} 14 | trainRounds = 10 15 | random.seed(10) 16 | initWeight = 0.05 17 | 18 | def nextInitWeight(): 19 | return (random.random() - 0.5) * initWeight 20 | 21 | def ints(s): 22 | res = [] 23 | for ss in s: 24 | res.append(int(ss)) 25 | return res 26 | 27 | def sigmoid(p): 28 | return 1.0 / (1.0 + math.exp(-p)) 29 | 30 | 31 | if len(sys.argv) < 3: 32 | print 'Usage: train.yzx.txt test.yzx.txt' 33 | exit(-1) 34 | 35 | 36 | for round in range(0, trainRounds): 37 | # train for this round 38 | fi = open(sys.argv[1], 'r') 39 | lineNum = 0 40 | trainData = [] 41 | for line in fi: 42 | lineNum = (lineNum + 1) % bufferCaseNum 43 | trainData.append(ints(line.replace(":1", "").split())) 44 | if lineNum == 0: 45 | for data in trainData: 46 | clk = data[0] 47 | mp = data[1] 48 | fsid = 2 # feature start id 49 | # predict 50 | pred = 0.0 51 | for i in range(fsid, len(data)): 52 | feat = data[i] 53 | if feat not in featWeight: 54 | featWeight[feat] = nextInitWeight() 55 | pred += featWeight[feat] 56 | pred = sigmoid(pred) 57 | # start to update weight 58 | # w_i = w_i + learning_rate * [ (y - p) * x_i - lamb * w_i ] 59 | for i in range(fsid, len(data)): 60 | feat = data[i] 61 | featWeight[feat] = featWeight[feat] * (1 - lamb) + eta * (clk - pred) 62 | trainData = [] 63 | 64 | if len(trainData) > 0: 65 | for data in trainData: 66 | clk = data[0] 67 | mp = data[1] 68 | fsid = 2 # feature start id 69 | # predict 70 | pred = 0.0 71 | for i in range(fsid, len(data)): 72 | feat = data[i] 73 | if feat not in featWeight: 74 | featWeight[feat] = nextInitWeight() 75 | pred += featWeight[feat] 76 | pred = sigmoid(pred) 77 | # start to update weight 78 | # w_i = w_i + learning_rate * [ (y - p) * x_i - lamb * w_i ] 79 | for i in range(fsid, len(data)): 80 | feat = data[i] 81 | featWeight[feat] = featWeight[feat] * (1 - lamb) + eta * (clk - pred) 82 | fi.close() 83 | 84 | # test for this round 85 | y = [] 86 | yp = [] 87 | fi = open(sys.argv[2], 'r') 88 | for line in fi: 89 | data = ints(line.replace(":1", "").split()) 90 | clk = data[0] 91 | mp = data[1] 92 | fsid = 2 # feature start id 93 | pred = 0.0 94 | for i in range(fsid, len(data)): 95 | feat = data[i] 96 | if feat in featWeight: 97 | pred += featWeight[feat] 98 | pred = sigmoid(pred) 99 | y.append(clk) 100 | yp.append(pred) 101 | fi.close() 102 | auc = roc_auc_score(y, yp) 103 | rmse = math.sqrt(mean_squared_error(y, yp)) 104 | print str(round) + '\t' + str(auc) + '\t' + str(rmse) 105 | 106 | # output the weights 107 | fo = open(sys.argv[1] + '.lr.weight', 'w') 108 | featvalue = sorted(featWeight.iteritems(), key=operator.itemgetter(0)) 109 | for fv in featvalue: 110 | fo.write(str(fv[0]) + '\t' + str(fv[1]) + '\n') 111 | fo.close() 112 | 113 | 114 | # output the prediction 115 | fi = open(sys.argv[2], 'r') 116 | fo = open(sys.argv[2] + '.lr.pred', 'w') 117 | 118 | for line in fi: 119 | data = ints(line.replace(":1", "").split()) 120 | pred = 0.0 121 | for i in range(1, len(data)): 122 | feat = data[i] 123 | if feat in featWeight: 124 | pred += featWeight[feat] 125 | pred = sigmoid(pred) 126 | fo.write(str(pred) + '\n') 127 | fo.close() 128 | fi.close() 129 | 130 | 131 | 132 | -------------------------------------------------------------------------------- /python/rtb-test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | import sys 3 | import random 4 | import math 5 | 6 | random.seed(10) 7 | 8 | def bidding_const(bid): 9 | return bid 10 | 11 | def bidding_rand(upper): 12 | return int(random.random() * upper) 13 | 14 | def bidding_mcpc(ecpc, pctr): 15 | return int(ecpc * pctr) 16 | 17 | def bidding_lin(pctr, base_ctr, base_bid): 18 | return int(pctr * base_bid / base_ctr) 19 | 20 | def win_auction(case, bid): 21 | return bid > case[1] # bid > winning price 22 | 23 | # budgetProportion clk cnv bid imp budget spend para 24 | def simulate_one_bidding_strategy_with_parameter(cases, ctrs, tcost, proportion, algo, para): 25 | budget = int(tcost / proportion) # intialise the budget 26 | cost = 0 27 | clks = 0 28 | bids = 0 29 | imps = 0 30 | for idx in range(0, len(cases)): 31 | bid = 0 32 | pctr = ctrs[idx] 33 | if algo == "const": 34 | bid = bidding_const(para) 35 | elif algo == "rand": 36 | bid = bidding_rand(para) 37 | elif algo == "mcpc": 38 | bid = bidding_mcpc(original_ecpc, pctr) 39 | elif algo == "lin": 40 | bid = bidding_lin(pctr, original_ctr, para) 41 | else: 42 | print 'wrong bidding strategy name' 43 | sys.exit(-1) 44 | bids += 1 45 | case = cases[idx] 46 | if win_auction(case, bid): 47 | imps += 1 48 | clks += case[0] 49 | cost += case[1] 50 | if cost > budget: 51 | break 52 | return str(proportion) + '\t' + str(clks) + '\t' + str(bids) + '\t' + \ 53 | str(imps) + '\t' + str(budget) + '\t' + str(cost) + '\t' + algo + '\t' + str(para) 54 | 55 | def simulate_one_bidding_strategy(cases, ctrs, tcost, proportion, algo, writer): 56 | paras = algo_paras[algo] 57 | for para in paras: 58 | res = simulate_one_bidding_strategy_with_parameter(cases, ctrs, tcost, proportion, algo, para) 59 | print res 60 | writer.write(res + '\n') 61 | 62 | 63 | if len(sys.argv) < 5: 64 | print 'Usage: python rtb-test.py train.yzx.txt test.yzx.txt test.yzx.txt.lr.pred rtb.result.txt' 65 | exit(-1) 66 | 67 | clicks_prices = [] # clk and price 68 | pctrs = [] # pCTR from logistic regression prediciton 69 | total_cost = 0 # total original cost during the test data 70 | original_ecpc = 0. # original eCPC from train data 71 | original_ctr = 0. # original ctr from train data 72 | 73 | # read in train data for original_ecpc and original_ctr 74 | fi = open(sys.argv[1], 'r') # train.yzx.txt 75 | first = True 76 | imp_num = 0 77 | for line in fi: 78 | s = line.split(' ') 79 | if first: 80 | first = False 81 | continue 82 | click = int(s[0]) # y 83 | cost = int(s[1]) # z 84 | imp_num += 1 85 | original_ctr += click 86 | original_ecpc += cost 87 | fi.close() 88 | original_ecpc /= original_ctr 89 | original_ctr /= imp_num 90 | 91 | # read in test data 92 | fi = open(sys.argv[2], 'r') # test.yzx.txt 93 | for line in fi: 94 | s = line.split(' ') 95 | click = int(s[0]) 96 | winning_price = int(s[1]) 97 | clicks_prices.append((click, winning_price)) 98 | total_cost += winning_price 99 | fi.close() 100 | 101 | # read in pctr from logistic regression 102 | fi = open(sys.argv[3], 'r') # test.yzx.txt.lr.pred 103 | for line in fi: 104 | pctrs.append(float(line.strip())) 105 | fi.close() 106 | 107 | # parameters setting for each bidding strategy 108 | budget_proportions = [64, 16] # , 32, 8] 109 | const_paras = range(2, 20, 2) + range(20, 100, 5) + range(100, 301, 10) 110 | rand_paras = range(2, 20, 2) + range(20, 100, 5) + range(100, 501, 10) 111 | mcpc_paras = [1] 112 | lin_paras = range(2, 20, 2) + range(20, 100, 5) + range(100, 400, 10) + range(400, 800, 50) 113 | 114 | algo_paras = {"const":const_paras, "rand":rand_paras, "mcpc":mcpc_paras, "lin":lin_paras} 115 | 116 | # initalisation finished 117 | # rock! 118 | 119 | fo = open(sys.argv[4], 'w') # rtb.results.txt 120 | #header = "proportion\tclicks\tbids\timpressions\tbudget\tspend\tstrategy\tparameter" 121 | header = "prop\tclks\tbids\timps\tbudget\tspend\talgo\tpara" 122 | fo.write(header + "\n") 123 | print header 124 | for proportion in budget_proportions: 125 | for algo in algo_paras: 126 | simulate_one_bidding_strategy(clicks_prices, pctrs, total_cost, proportion, algo, fo) 127 | fo.close() 128 | -------------------------------------------------------------------------------- /results/rtb.results.1458.best.perf.tsv: -------------------------------------------------------------------------------- 1 | prop clks bids imps budget spend algo para 2 | 16 51 227472 97784 2826028 2826032 const 55 3 | 16 482 614638 89408 2826028 2804895 lin 130 4 | 16 473 614638 38858 2826028 816763 mcpc 1 5 | 16 50 199209 77751 2826028 2826041 rand 100 6 | 64 16 472727 57030 706507 706508 const 20 7 | 64 471 614638 31908 706507 628971 lin 60 8 | 64 330 424901 32372 706507 706520 mcpc 1 9 | 64 14 614638 52079 706507 689611 rand 30 10 | -------------------------------------------------------------------------------- /results/rtb.results.1458.tsv: -------------------------------------------------------------------------------- 1 | prop clks bids imps budget spend algo para 2 | 64 0 614638 0 706507 0 rand 2 3 | 64 0 614638 1 706507 1 rand 4 4 | 64 0 614638 547 706507 2180 rand 6 5 | 64 0 614638 3417 706507 16127 rand 8 6 | 64 2 614638 6125 706507 31389 rand 10 7 | 64 1 614638 9047 706507 52134 rand 12 8 | 64 4 614638 12329 706507 79775 rand 14 9 | 64 5 614638 16062 706507 115668 rand 16 10 | 64 6 614638 20119 706507 162255 rand 18 11 | 64 5 614638 24158 706507 214621 rand 20 12 | 64 9 614638 38765 706507 449953 rand 25 13 | 64 14 614638 52079 706507 689611 rand 30 14 | 64 12 427206 47597 706507 706513 rand 35 15 | 64 10 328205 44102 706507 706509 rand 40 16 | 64 12 264211 41413 706507 706522 rand 45 17 | 64 9 223711 39101 706507 706513 rand 50 18 | 64 9 188058 36138 706507 706535 rand 55 19 | 64 11 160097 33267 706507 706523 rand 60 20 | 64 12 136076 31502 706507 706515 rand 65 21 | 64 10 117180 29745 706507 706525 rand 70 22 | 64 10 94625 26858 706507 706541 rand 75 23 | 64 11 79862 24934 706507 706558 rand 80 24 | 64 12 67459 23021 706507 706533 rand 85 25 | 64 9 59706 21962 706507 706534 rand 90 26 | 64 6 53323 21228 706507 706519 rand 95 27 | 64 11 49320 20613 706507 706559 rand 100 28 | 64 8 42584 19389 706507 706518 rand 110 29 | 64 9 38516 18891 706507 706545 rand 120 30 | 64 11 34938 18213 706507 706526 rand 130 31 | 64 11 33181 18003 706507 706522 rand 140 32 | 64 8 31412 17486 706507 706526 rand 150 33 | 64 12 30006 17311 706507 706531 rand 160 34 | 64 9 27697 16815 706507 706549 rand 170 35 | 64 11 26043 16304 706507 706540 rand 180 36 | 64 9 24960 15957 706507 706562 rand 190 37 | 64 10 24088 15778 706507 706509 rand 200 38 | 64 10 23123 15419 706507 706510 rand 210 39 | 64 9 22058 15105 706507 706544 rand 220 40 | 64 12 21536 14934 706507 706564 rand 230 41 | 64 8 20605 14632 706507 706531 rand 240 42 | 64 8 20083 14448 706507 706565 rand 250 43 | 64 11 19368 14215 706507 706577 rand 260 44 | 64 7 18735 14002 706507 706508 rand 270 45 | 64 9 18280 13790 706507 706513 rand 280 46 | 64 9 17964 13768 706507 706575 rand 290 47 | 64 9 17781 13697 706507 706548 rand 300 48 | 64 10 17359 13534 706507 706508 rand 310 49 | 64 9 17148 13497 706507 706560 rand 320 50 | 64 9 16774 13333 706507 706513 rand 330 51 | 64 12 16657 13277 706507 706739 rand 340 52 | 64 7 16411 13234 706507 706518 rand 350 53 | 64 12 16212 13157 706507 706638 rand 360 54 | 64 6 16020 13069 706507 706510 rand 370 55 | 64 11 15610 12875 706507 706511 rand 380 56 | 64 7 15366 12760 706507 706552 rand 390 57 | 64 8 15315 12802 706507 706605 rand 400 58 | 64 8 15220 12687 706507 706587 rand 410 59 | 64 9 15029 12665 706507 706547 rand 420 60 | 64 8 14762 12587 706507 706733 rand 430 61 | 64 10 14821 12525 706507 706601 rand 440 62 | 64 4 14699 12551 706507 706542 rand 450 63 | 64 7 14664 12577 706507 706556 rand 460 64 | 64 7 14550 12560 706507 706592 rand 470 65 | 64 7 14363 12407 706507 706522 rand 480 66 | 64 9 14552 12533 706507 706641 rand 490 67 | 64 8 14138 12362 706507 706606 rand 500 68 | 64 467 614638 966 706507 84238 lin 2 69 | 64 468 614638 1003 706507 86382 lin 4 70 | 64 468 614638 1072 706507 87792 lin 6 71 | 64 468 614638 1196 706507 88942 lin 8 72 | 64 468 614638 1436 706507 91166 lin 10 73 | 64 468 614638 1846 706507 94437 lin 12 74 | 64 468 614638 2382 706507 98791 lin 14 75 | 64 468 614638 3095 706507 105251 lin 16 76 | 64 468 614638 3932 706507 112894 lin 18 77 | 64 468 614638 4915 706507 122472 lin 20 78 | 64 468 614638 7532 706507 151887 lin 25 79 | 64 468 614638 10557 706507 190201 lin 30 80 | 64 468 614638 13799 706507 238598 lin 35 81 | 64 469 614638 17235 706507 300096 lin 40 82 | 64 470 614638 20737 706507 366210 lin 45 83 | 64 470 614638 24305 706507 444340 lin 50 84 | 64 470 614638 28092 706507 531024 lin 55 85 | 64 471 614638 31908 706507 628971 lin 60 86 | 64 445 581403 34157 706507 706527 lin 65 87 | 64 311 400665 31954 706507 706529 lin 70 88 | 64 256 314845 29842 706507 706685 lin 75 89 | 64 224 250196 28284 706507 706515 lin 80 90 | 64 185 211622 27221 706507 706557 lin 85 91 | 64 166 187009 25835 706507 706537 lin 90 92 | 64 153 170745 24736 706507 706544 lin 95 93 | 64 141 155614 23739 706507 706510 lin 100 94 | 64 118 126549 22425 706507 706566 lin 110 95 | 64 106 114204 21857 706507 706633 lin 120 96 | 64 100 104190 21594 706507 706584 lin 130 97 | 64 83 92587 21014 706507 706543 lin 140 98 | 64 72 81691 20234 706507 706569 lin 150 99 | 64 66 74631 19829 706507 706560 lin 160 100 | 64 59 68343 19232 706507 706526 lin 170 101 | 64 56 62653 18616 706507 706534 lin 180 102 | 64 45 57767 18255 706507 706514 lin 190 103 | 64 41 52546 17916 706507 706538 lin 200 104 | 64 39 48057 17379 706507 706570 lin 210 105 | 64 35 44840 17089 706507 706515 lin 220 106 | 64 34 41914 16747 706507 706519 lin 230 107 | 64 32 39648 16560 706507 706568 lin 240 108 | 64 32 37534 16311 706507 706543 lin 250 109 | 64 31 35753 15959 706507 706575 lin 260 110 | 64 30 33914 15687 706507 706532 lin 270 111 | 64 29 32610 15588 706507 706542 lin 280 112 | 64 28 31074 15257 706507 706516 lin 290 113 | 64 27 29990 15180 706507 706541 lin 300 114 | 64 25 28970 15046 706507 706664 lin 310 115 | 64 23 27925 14824 706507 706516 lin 320 116 | 64 23 27055 14599 706507 706583 lin 330 117 | 64 23 26185 14353 706507 706566 lin 340 118 | 64 22 25766 14397 706507 706557 lin 350 119 | 64 22 25185 14264 706507 706592 lin 360 120 | 64 21 24388 14045 706507 706540 lin 370 121 | 64 21 23701 13928 706507 706586 lin 380 122 | 64 19 23128 13825 706507 706518 lin 390 123 | 64 18 22481 13694 706507 706660 lin 400 124 | 64 16 21135 13487 706507 706555 lin 450 125 | 64 16 20167 13384 706507 706567 lin 500 126 | 64 15 19165 13224 706507 706529 lin 550 127 | 64 15 18350 13086 706507 706740 lin 600 128 | 64 14 17806 13044 706507 706563 lin 650 129 | 64 14 17297 12974 706507 706535 lin 700 130 | 64 14 16775 12884 706507 706560 lin 750 131 | 64 330 424901 32372 706507 706520 mcpc 1 132 | 64 0 614638 2 706507 2 const 2 133 | 64 0 614638 4 706507 7 const 4 134 | 64 3 614638 11130 706507 52394 const 6 135 | 64 3 614638 15785 706507 82965 const 8 136 | 64 5 614638 20922 706507 126505 const 10 137 | 64 5 614638 30335 706507 223802 const 12 138 | 64 9 614638 37758 706507 316741 const 14 139 | 64 12 614638 49115 706507 481174 const 16 140 | 64 14 614638 60480 706507 668270 const 18 141 | 64 16 472727 57030 706507 706508 const 20 142 | 64 12 227830 46755 706507 706525 const 25 143 | 64 12 191633 43328 706507 706517 const 30 144 | 64 11 151644 38989 706507 706530 const 35 145 | 64 11 129712 36720 706507 706518 const 40 146 | 64 12 96857 32242 706507 706523 const 45 147 | 64 14 83623 30108 706507 706512 const 50 148 | 64 8 57279 25399 706507 706521 const 55 149 | 64 7 50653 24173 706507 706520 const 60 150 | 64 8 42035 22267 706507 706537 const 65 151 | 64 8 38405 21488 706507 706514 const 70 152 | 64 9 27588 18721 706507 706524 const 75 153 | 64 9 25674 18078 706507 706571 const 80 154 | 64 11 22510 17003 706507 706550 const 85 155 | 64 10 20784 16343 706507 706525 const 90 156 | 64 10 20222 16189 706507 706553 const 95 157 | 64 10 19576 15928 706507 706543 const 100 158 | 64 9 18647 15571 706507 706551 const 110 159 | 64 9 18275 15403 706507 706542 const 120 160 | 64 9 17955 15281 706507 706539 const 130 161 | 64 9 17346 15005 706507 706554 const 140 162 | 64 10 16074 14362 706507 706560 const 150 163 | 64 8 14955 13711 706507 706517 const 160 164 | 64 8 14397 13410 706507 706544 const 170 165 | 64 7 14107 13234 706507 706527 const 180 166 | 64 7 13870 13073 706507 706515 const 190 167 | 64 6 13369 12809 706507 706527 const 200 168 | 64 5 13086 12606 706507 706527 const 210 169 | 64 5 12824 12418 706507 706515 const 220 170 | 64 5 12565 12231 706507 706508 const 230 171 | 64 5 12308 12048 706507 706546 const 240 172 | 64 4 12100 11895 706507 706631 const 250 173 | 64 4 11890 11745 706507 706522 const 260 174 | 64 5 11757 11644 706507 706566 const 270 175 | 64 5 11670 11581 706507 706522 const 280 176 | 64 5 11536 11486 706507 706655 const 290 177 | 64 5 11427 11400 706507 706516 const 300 178 | 16 0 614638 0 2826028 0 rand 2 179 | 16 0 614638 1 2826028 1 rand 4 180 | 16 0 614638 541 2826028 2157 rand 6 181 | 16 0 614638 3480 2826028 16462 rand 8 182 | 16 1 614638 6221 2826028 32142 rand 10 183 | 16 3 614638 9359 2826028 54063 rand 12 184 | 16 2 614638 12432 2826028 80297 rand 14 185 | 16 7 614638 15868 2826028 114057 rand 16 186 | 16 5 614638 20020 2826028 161041 rand 18 187 | 16 5 614638 24351 2826028 216081 rand 20 188 | 16 13 614638 38950 2826028 451611 rand 25 189 | 16 17 614638 51806 2826028 684011 rand 30 190 | 16 20 614638 65142 2826028 966131 rand 35 191 | 16 16 614638 77189 2826028 1243933 rand 40 192 | 16 27 614638 88209 2826028 1530112 rand 45 193 | 16 30 614638 98560 2826028 1829446 rand 50 194 | 16 39 614638 111099 2826028 2274542 rand 55 195 | 16 47 614638 123658 2826028 2751025 rand 60 196 | 16 36 530779 117416 2826028 2826029 rand 65 197 | 16 45 466188 112431 2826028 2826044 rand 70 198 | 16 42 377035 102357 2826028 2826043 rand 75 199 | 16 42 317438 95567 2826028 2826058 rand 80 200 | 16 47 274020 89888 2826028 2826074 rand 85 201 | 16 40 241387 85328 2826028 2826032 rand 90 202 | 16 44 218582 81485 2826028 2826034 rand 95 203 | 16 50 199209 77751 2826028 2826041 rand 100 204 | 16 45 171111 72491 2826028 2826073 rand 110 205 | 16 48 152123 69361 2826028 2826093 rand 120 206 | 16 42 138036 67102 2826028 2826048 rand 130 207 | 16 39 127797 65306 2826028 2826038 rand 140 208 | 16 41 117669 63609 2826028 2826047 rand 150 209 | 16 44 109682 62157 2826028 2826048 rand 160 210 | 16 36 103245 60524 2826028 2826067 rand 170 211 | 16 37 97563 59227 2826028 2826086 rand 180 212 | 16 40 93352 58231 2826028 2826076 rand 190 213 | 16 47 89341 57031 2826028 2826044 rand 200 214 | 16 39 86235 56494 2826028 2826033 rand 210 215 | 16 43 82822 55333 2826028 2826065 rand 220 216 | 16 42 80487 54578 2826028 2826067 rand 230 217 | 16 42 77499 53750 2826028 2826106 rand 240 218 | 16 39 74926 52956 2826028 2826034 rand 250 219 | 16 38 72696 52168 2826028 2826036 rand 260 220 | 16 46 70690 51374 2826028 2826039 rand 270 221 | 16 41 68963 50813 2826028 2826094 rand 280 222 | 16 38 67321 50269 2826028 2826247 rand 290 223 | 16 32 65591 49687 2826028 2826029 rand 300 224 | 16 42 64045 48906 2826028 2826060 rand 310 225 | 16 38 63234 48657 2826028 2826038 rand 320 226 | 16 43 61934 48152 2826028 2826049 rand 330 227 | 16 38 61667 48103 2826028 2826033 rand 340 228 | 16 38 59626 47225 2826028 2826122 rand 350 229 | 16 32 59039 47075 2826028 2826128 rand 360 230 | 16 32 58230 46785 2826028 2826030 rand 370 231 | 16 32 57735 46447 2826028 2826073 rand 380 232 | 16 30 57309 46388 2826028 2826147 rand 390 233 | 16 31 56635 46148 2826028 2826046 rand 400 234 | 16 32 55757 45752 2826028 2826066 rand 410 235 | 16 32 55275 45572 2826028 2826072 rand 420 236 | 16 27 54910 45573 2826028 2826075 rand 430 237 | 16 33 54413 45464 2826028 2826046 rand 440 238 | 16 38 54126 45386 2826028 2826061 rand 450 239 | 16 35 53707 45213 2826028 2826059 rand 460 240 | 16 34 53440 45167 2826028 2826077 rand 470 241 | 16 34 52666 44812 2826028 2826081 rand 480 242 | 16 33 52668 44769 2826028 2826044 rand 490 243 | 16 23 51871 44451 2826028 2826060 rand 500 244 | 16 467 614638 966 2826028 84238 lin 2 245 | 16 468 614638 1003 2826028 86382 lin 4 246 | 16 468 614638 1072 2826028 87792 lin 6 247 | 16 468 614638 1196 2826028 88942 lin 8 248 | 16 468 614638 1436 2826028 91166 lin 10 249 | 16 468 614638 1846 2826028 94437 lin 12 250 | 16 468 614638 2382 2826028 98791 lin 14 251 | 16 468 614638 3095 2826028 105251 lin 16 252 | 16 468 614638 3932 2826028 112894 lin 18 253 | 16 468 614638 4915 2826028 122472 lin 20 254 | 16 468 614638 7532 2826028 151887 lin 25 255 | 16 468 614638 10557 2826028 190201 lin 30 256 | 16 468 614638 13799 2826028 238598 lin 35 257 | 16 469 614638 17235 2826028 300096 lin 40 258 | 16 470 614638 20737 2826028 366210 lin 45 259 | 16 470 614638 24305 2826028 444340 lin 50 260 | 16 470 614638 28092 2826028 531024 lin 55 261 | 16 471 614638 31908 2826028 628971 lin 60 262 | 16 473 614638 35777 2826028 730160 lin 65 263 | 16 473 614638 39751 2826028 841999 lin 70 264 | 16 474 614638 43614 2826028 960612 lin 75 265 | 16 474 614638 47597 2826028 1085723 lin 80 266 | 16 476 614638 51638 2826028 1223842 lin 85 267 | 16 476 614638 55714 2826028 1371649 lin 90 268 | 16 476 614638 59734 2826028 1524368 lin 95 269 | 16 477 614638 63964 2826028 1687772 lin 100 270 | 16 479 614638 72367 2826028 2036667 lin 110 271 | 16 479 614638 80915 2826028 2412613 lin 120 272 | 16 482 614638 89408 2826028 2804895 lin 130 273 | 16 367 461968 82882 2826028 2826031 lin 140 274 | 16 294 365929 77596 2826028 2826029 lin 150 275 | 16 261 307033 73870 2826028 2826088 lin 160 276 | 16 239 263610 70582 2826028 2826045 lin 170 277 | 16 209 232254 68633 2826028 2826089 lin 180 278 | 16 194 208358 65620 2826028 2826040 lin 190 279 | 16 179 191279 63387 2826028 2826193 lin 200 280 | 16 169 179420 61597 2826028 2826081 lin 210 281 | 16 161 169496 60032 2826028 2826051 lin 220 282 | 16 153 160806 59007 2826028 2826038 lin 230 283 | 16 151 153678 58097 2826028 2826102 lin 240 284 | 16 146 146289 57412 2826028 2826268 lin 250 285 | 16 140 139407 56724 2826028 2826114 lin 260 286 | 16 130 132653 55950 2826028 2826063 lin 270 287 | 16 125 128033 55613 2826028 2826040 lin 280 288 | 16 125 123809 54949 2826028 2826102 lin 290 289 | 16 121 120645 54629 2826028 2826091 lin 300 290 | 16 116 117301 54196 2826028 2826032 lin 310 291 | 16 114 114214 53706 2826028 2826169 lin 320 292 | 16 113 111804 53591 2826028 2826053 lin 330 293 | 16 112 109676 53611 2826028 2826077 lin 340 294 | 16 113 107498 53470 2826028 2826066 lin 350 295 | 16 109 104842 52878 2826028 2826076 lin 360 296 | 16 105 102845 52785 2826028 2826061 lin 370 297 | 16 101 100940 52548 2826028 2826075 lin 380 298 | 16 96 98798 52343 2826028 2826093 lin 390 299 | 16 95 96493 51966 2826028 2826051 lin 400 300 | 16 86 86903 50652 2826028 2826050 lin 450 301 | 16 79 80529 49478 2826028 2826032 lin 500 302 | 16 73 75531 48664 2826028 2826095 lin 550 303 | 16 71 71482 47866 2826028 2826049 lin 600 304 | 16 65 68553 47487 2826028 2826030 lin 650 305 | 16 64 66111 47056 2826028 2826050 lin 700 306 | 16 60 63480 46409 2826028 2826044 lin 750 307 | 16 473 614638 38858 2826028 816763 mcpc 1 308 | 16 0 614638 2 2826028 2 const 2 309 | 16 0 614638 4 2826028 7 const 4 310 | 16 3 614638 11130 2826028 52394 const 6 311 | 16 3 614638 15785 2826028 82965 const 8 312 | 16 5 614638 20922 2826028 126505 const 10 313 | 16 5 614638 30335 2826028 223802 const 12 314 | 16 9 614638 37758 2826028 316741 const 14 315 | 16 12 614638 49115 2826028 481174 const 16 316 | 16 14 614638 60480 2826028 668270 const 18 317 | 16 19 614638 74342 2826028 926827 const 20 318 | 16 31 614638 110642 2826028 1682897 const 25 319 | 16 39 614638 128698 2826028 2168756 const 30 320 | 16 37 584376 146179 2826028 2826051 const 35 321 | 16 40 498861 137011 2826028 2826030 const 40 322 | 16 42 382131 123073 2826028 2826037 const 45 323 | 16 40 334771 116242 2826028 2826048 const 50 324 | 16 51 227472 97784 2826028 2826032 const 55 325 | 16 47 200255 91184 2826028 2826035 const 60 326 | 16 43 180816 86265 2826028 2826057 const 65 327 | 16 42 168668 82809 2826028 2826032 const 70 328 | 16 43 100865 67056 2826028 2826050 const 75 329 | 16 44 93622 65016 2826028 2826094 const 80 330 | 16 43 82115 61545 2826028 2826030 const 85 331 | 16 43 77290 60004 2826028 2826078 const 90 332 | 16 41 75777 59527 2826028 2826045 const 95 333 | 16 41 72586 58282 2826028 2826033 const 100 334 | 16 40 68140 56578 2826028 2826046 const 110 335 | 16 40 65479 55426 2826028 2826076 const 120 336 | 16 38 63934 54632 2826028 2826049 const 130 337 | 16 37 61608 53400 2826028 2826098 const 140 338 | 16 35 58437 51719 2826028 2826062 const 150 339 | 16 30 55555 50123 2826028 2826031 const 160 340 | 16 31 53778 49235 2826028 2826054 const 170 341 | 16 31 52588 48490 2826028 2826070 const 180 342 | 16 31 51831 48027 2826028 2826033 const 190 343 | 16 31 50719 47280 2826028 2826075 const 200 344 | 16 32 48983 46131 2826028 2826096 const 210 345 | 16 32 47491 45124 2826028 2826048 const 220 346 | 16 32 46106 44198 2826028 2826080 const 230 347 | 16 31 44800 43302 2826028 2826064 const 240 348 | 16 32 42464 41661 2826028 2826052 const 250 349 | 16 31 41508 40987 2826028 2826121 const 260 350 | 16 32 41030 40649 2826028 2826077 const 270 351 | 16 33 40617 40330 2826028 2826036 const 280 352 | 16 33 40018 39869 2826028 2826035 const 290 353 | 16 33 39614 39553 2826028 2826050 const 300 354 | -------------------------------------------------------------------------------- /scripts/check-best-perf.sh: -------------------------------------------------------------------------------- 1 | campaigns="1458" # "1458 2259 2261 2821 2997 3358 3386 3427 3476" 2 | resultsfolder=../results 3 | for campaign in $campaigns; do 4 | python ../python/check-best-perf.py $resultsfolder/rtb.results.$campaign.tsv 5 | done 6 | -------------------------------------------------------------------------------- /scripts/demo.sh: -------------------------------------------------------------------------------- 1 | campaigns="1458" 2 | folder=../../make-ipinyou-data 3 | resultsfolder=../results 4 | 5 | if [ ! -d $resultsfolder ]; then 6 | mkdir $resultsfolder 7 | fi 8 | 9 | for campaign in $campaigns; do 10 | echo $campaign 11 | # run logistc regression 12 | python ../python/lryzx.py $folder/$campaign/train.yzx.txt $folder/$campaign/test.yzx.txt 13 | 14 | # run real-time bidding test 15 | python ../python/rtb-test.py $folder/$campaign/train.yzx.txt $folder/$campaign/test.yzx.txt $folder/$campaign/test.yzx.txt.lr.pred $resultsfolder/rtb.results.$campaign.tsv 16 | 17 | # check the best perf 18 | python ../python/check-best-perf.py $resultsfolder/rtb.results.$campaign.tsv 19 | done 20 | -------------------------------------------------------------------------------- /scripts/run-lr.sh: -------------------------------------------------------------------------------- 1 | advs="1458 2259 2261 2821 2997 3358 3386 3427 3476" 2 | folder=../../make-ipinyou-data 3 | for adv in $advs; do 4 | echo $adv 5 | python ../python/lryzx.py $folder/$adv/train.yzx.txt $folder/$adv/test.yzx.txt 6 | done 7 | -------------------------------------------------------------------------------- /scripts/run-rtb-test.sh: -------------------------------------------------------------------------------- 1 | campaigns="1458 2259 2261 2821 2997 3358 3386 3427 3476" 2 | folder=../../make-ipinyou-data 3 | resultsfolder=../results 4 | 5 | if [ ! -d $resultsfolder ]; then 6 | mkdir $resultsfolder 7 | fi 8 | 9 | for campaign in $campaigns; do 10 | echo $campaign 11 | python ../python/rtb-test.py $folder/$campaign/train.yzx.txt $folder/$campaign/test.yzx.txt $folder/$campaign/test.yzx.txt.lr.pred $resultsfolder/rtb.results.$campaign.tsv 12 | python ../python/check-best-perf.py $resultsfolder/rtb.results.$campaign.tsv 13 | done 14 | --------------------------------------------------------------------------------