├── .gitignore ├── LICENSE ├── README.md └── fast-ftrl-proximal ├── pom.xml └── src ├── main ├── java │ └── io │ │ └── scaledml │ │ ├── core │ │ ├── BaseDisruptorRunner.java │ │ ├── SparseItem.java │ │ ├── TwoPhaseEvent.java │ │ ├── inputformats │ │ │ ├── AbstractDelimiterSeparatedValuesFormat.java │ │ │ ├── BinaryInputFormat.java │ │ │ ├── CSVFormat.java │ │ │ ├── ColumnsMask.java │ │ │ ├── InputFormat.java │ │ │ └── VowpalWabbitFormat.java │ │ ├── outputformats │ │ │ ├── NullOutputFormat.java │ │ │ ├── OutputFormat.java │ │ │ └── PrintStreamOutputFormat.java │ │ └── util │ │ │ ├── LineBytesBuffer.java │ │ │ ├── MultiListsViewLongList.java │ │ │ └── Util.java │ │ ├── features │ │ ├── Binning.java │ │ ├── BinningWorkHandler.java │ │ ├── FeatureEngineeringModule.java │ │ ├── FeatureEngineeringRunner.java │ │ ├── FirstPassRunner.java │ │ ├── NumericalFeaturesStatistics.java │ │ ├── OutputWriterEventHandler.java │ │ ├── SecondPassRunner.java │ │ └── StatisticsWorkHandler.java │ │ └── ftrl │ │ ├── AbstractParallelModule.java │ │ ├── FTRLProximalAlgorithm.java │ │ ├── FtrlProximalModel.java │ │ ├── FtrlProximalRunner.java │ │ ├── Increment.java │ │ ├── Main.java │ │ ├── featuresprocessors │ │ └── FeaturesProcessor.java │ │ ├── options │ │ ├── FtrlOptions.java │ │ ├── FtrlOptionsObject.java │ │ └── InputFormatType.java │ │ ├── outputformats │ │ ├── FinishCollectStatisticsListener.java │ │ └── StatisticsCalculator.java │ │ ├── parallel │ │ ├── LearnWorkHandler.java │ │ ├── ParallelModule.java │ │ └── WriteUpdatesEventHandler.java │ │ └── semiparallel │ │ ├── LearnEventHandler.java │ │ ├── ParseInputWorkHandler.java │ │ └── SemiParallelModule.java └── resources │ └── simplelogger.properties └── test ├── java ├── integration │ ├── BaseIntegrationTest.java │ ├── FTRLProximalAlgorithmITest.java │ └── FeatureEngineeringITest.java └── io │ └── scaledml │ ├── core │ ├── SparseItemTest.java │ ├── inputformats │ │ ├── CSVFormatTest.java │ │ ├── ColumnsMaskTest.java │ │ └── VowpalWabbitFormatTest.java │ └── util │ │ └── LineBytesBufferTest.java │ ├── features │ ├── BinningTest.java │ └── NumericalFeaturesStatisticsTest.java │ └── ftrl │ ├── MainTest.java │ └── options │ └── ColumnsInfoTest.java └── resources ├── ruslan-CAT-test-small.vw ├── ruslan-CAT-train-small.vw ├── ruslan-test-small.csv ├── ruslan-test-small.vw ├── ruslan-train-small.csv ├── ruslan-train-small.vw ├── test-small.vw └── train-small.vw /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | *.iml 3 | target 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 scaled-ml 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Java FTRL-Proximal implementation 2 | 3 | 4 | Here is java FTRL-Proximal implementation. 5 | 6 | This is a machine learning algorithm that predicts the probability of some kind of events such as clicks on ad. 7 | 8 | Main advantage of this implementation that it gets benifits from multi-core hardware and can __scale up to 32 cores__. 9 | 10 | Build 11 | ----- 12 | 13 | To build a project [JDK8](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) and [maven3](http://maven.apache.org/download.cgi) must be installed. 14 | 15 | Build command: 16 | ``` 17 | $ cd fast-ftrl-proximal 18 | $ mvn clean install 19 | ``` 20 | 21 | Runnable jar ftrl-proximal.jar will be found in fast-ftrl-proximal/target/ directory. 22 | 23 | Input format 24 | ------------ 25 | Input format is similar to [vowpal wabbit's one](http://github.com/JohnLangford/vowpal_wabbit/wiki/Input-format) except that example's weight nor feature's weight are not supported. 26 | 27 | Only ascii symbols are supported yet. 28 | Run 29 | --- 30 | Train: 31 | ``` 32 | $ java -Xmx2G -jar ftrl-proximal.jar -d train.slit.logit.vw -b 22 -f model1 33 | ``` 34 | Apply: 35 | ``` 36 | $ java -Xmx2G -jar ftrl-proximal.jar -d test.slit.logit.vw -i model1 -p predictions 37 | ``` 38 | 39 | Run from java 40 | ------------- 41 | You can train or apply the algorithm from java by specifing options via FtrlOptionsObject: 42 | ``` 43 | Main.runFtrlProximal(new FtrlOptionsObject() 44 | .finalRegressor("model") 45 | .data("train-small.vw") 46 | .alfa(0.01) 47 | .lambda1(1.) 48 | .lambda2(2.))); 49 | ``` 50 | Options 51 | ------- 52 | 53 | ``` 54 | The options available are: 55 | [--ftrl_alpha value] : ftrl alpha parameter (option in ftrl) 56 | [--ftrl_beta value] : ftrl beta patameter (option in ftrl) 57 | [--data -d value] : Example Set 58 | [--final_regressor -f value] : Final regressor to save (arg inputStream filename) 59 | [--format value] : Input file format.'vw' or 'csv' are currently supported 60 | [--bit_precision -b value] : number of bits in the feature table 61 | [--help -h] : Show this help 62 | [--initial_regressor -i value] : Initial regressor(s) to load into memory (arg inputStream filename) 63 | [--l1 value] : l_1 lambda (L1 regularization) 64 | [--l2 value] : l_2 lambda (L2 regularization) 65 | [--predictions -p value] : File to output predictions to 66 | [--quadratic -q] : Add quadratic features 67 | [--testonly -t] : Ignore label information and just test 68 | [--threads value] : Parallelization level 69 | ``` 70 | 71 | References 72 | ---------- 73 | For more information see ["Ad Click Prediction: a View from the Trenches"](http://research.google.com/pubs/pub41159.html) paper. 74 | 75 | [ ![Codeship Status for scaled-ml/Scaled-ML](https://codeship.com/projects/55b7e1c0-bfe9-0132-f5cd-7eb09717a41c/status?branch=master)](https://codeship.com/projects/73069) 76 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 5 | 4.0.0 6 | 7 | io.scaled-ml 8 | fast-ftrl-proximal 9 | 1.0-SNAPSHOT 10 | 11 | 12 | internal.repo 13 | Temporary Staging Repository 14 | file://${project.build.directory}/mvn-repo 15 | 16 | 17 | 18 | 19 | github 20 | 21 | 22 | 23 | 24 | org.apache.maven.plugins 25 | maven-compiler-plugin 26 | 3.2 27 | 28 | 1.8 29 | 1.8 30 | 31 | 32 | 33 | maven-deploy-plugin 34 | 2.8.1 35 | 36 | internal.repo::default::file://${project.build.directory}/mvn-repo 37 | 38 | 39 | 40 | 41 | com.github.github 42 | site-maven-plugin 43 | 0.11 44 | 45 | Maven artifacts for ${project.version} 46 | 47 | true 48 | 49 | ${project.build.directory}/mvn-repo 50 | 51 | refs/heads/mvn-repo 52 | 53 | 54 | **/* 55 | 56 | Scaled-ML 57 | 58 | scaled-ml 59 | 60 | 61 | 62 | 63 | 64 | 65 | site 66 | 67 | deploy 68 | 69 | 70 | 71 | 72 | maven-assembly-plugin 73 | 74 | 75 | 76 | io.scaledml.ftrl.Main 77 | 78 | 79 | 80 | jar-with-dependencies 81 | 82 | ftrl-proximal 83 | false 84 | 85 | 86 | 87 | make-assembly 88 | package 89 | 90 | single 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | junit 100 | junit 101 | 4.11 102 | test 103 | 104 | 105 | com.google.inject 106 | guice 107 | 4.0 108 | 109 | 110 | com.google.inject.extensions 111 | guice-throwingproviders 112 | 4.0 113 | 114 | 115 | com.google.guava 116 | guava 117 | 18.0 118 | 119 | 120 | com.lexicalscope.jewelcli 121 | jewelcli 122 | 0.8.9 123 | 124 | 125 | it.unimi.dsi 126 | fastutil 127 | 6.5.15 128 | 129 | 130 | com.lmax 131 | disruptor 132 | 3.3.2 133 | 134 | 135 | commons-io 136 | commons-io 137 | 2.4 138 | test 139 | 140 | 141 | org.slf4j 142 | slf4j-simple 143 | 1.7.10 144 | 145 | 146 | com.clearspring.analytics 147 | stream 148 | 2.7.0 149 | 150 | 151 | 152 | 153 | 154 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/BaseDisruptorRunner.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core; 2 | 3 | import com.clearspring.analytics.util.Preconditions; 4 | import com.google.inject.Inject; 5 | import com.google.inject.name.Named; 6 | import com.lmax.disruptor.RingBuffer; 7 | import com.lmax.disruptor.dsl.Disruptor; 8 | import io.scaledml.core.util.LineBytesBuffer; 9 | import it.unimi.dsi.fastutil.io.FastBufferedInputStream; 10 | 11 | import java.io.IOException; 12 | import java.io.InputStream; 13 | import java.util.concurrent.Phaser; 14 | import java.util.function.Supplier; 15 | 16 | 17 | public abstract class BaseDisruptorRunner { 18 | private Disruptor> disruptor; 19 | private Supplier inputStreamFactory; 20 | private boolean skipFirst; 21 | private Phaser phaser; 22 | 23 | public void process() throws IOException { 24 | try (FastBufferedInputStream stream = new FastBufferedInputStream(inputStreamFactory.get())) { 25 | Preconditions.checkArgument(phaser.getRegisteredParties() == 0); 26 | phaser.register(); 27 | disruptor.start(); 28 | RingBuffer ringBuffer = disruptor.getRingBuffer(); 29 | long cursor = ringBuffer.next(); 30 | LineBytesBuffer buffer = ringBuffer.get(cursor).input(); 31 | long lineNo = 0; 32 | ringBuffer.get(cursor).lineNo(lineNo); 33 | boolean needToSkipNext = skipFirst; 34 | while (buffer.readLineFrom(stream)) { 35 | if (needToSkipNext) { 36 | needToSkipNext = false; 37 | continue; 38 | } 39 | lineNo++; 40 | ringBuffer.publish(cursor); 41 | cursor = ringBuffer.next(); 42 | buffer = ringBuffer.get(cursor).input(); 43 | ringBuffer.get(cursor).lineNo(lineNo); 44 | } 45 | disruptor.shutdown(); 46 | phaser.arriveAndAwaitAdvance(); 47 | phaser.arriveAndDeregister(); 48 | } finally { 49 | afterDisruptorProcessed(); 50 | } 51 | } 52 | 53 | protected abstract void afterDisruptorProcessed() throws IOException; 54 | 55 | protected void setDisruptor(Disruptor> disruptor) { 56 | this.disruptor = disruptor; 57 | } 58 | 59 | @Inject 60 | public void setInputStreamFactory(Supplier inputStreamFactory) { 61 | this.inputStreamFactory = inputStreamFactory; 62 | } 63 | 64 | @Inject 65 | public void setSkipFirst(@Named("skipFirst") boolean skipFirst) { 66 | this.skipFirst = skipFirst; 67 | } 68 | 69 | @Inject 70 | public void setPhaser(Phaser phaser) { 71 | this.phaser = phaser; 72 | } 73 | } 74 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/SparseItem.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core; 2 | 3 | import com.google.common.base.MoreObjects; 4 | import io.scaledml.core.util.LineBytesBuffer; 5 | import io.scaledml.core.util.MultiListsViewLongList; 6 | import io.scaledml.core.util.Util; 7 | import it.unimi.dsi.fastutil.doubles.DoubleArrayList; 8 | import it.unimi.dsi.fastutil.doubles.DoubleList; 9 | import it.unimi.dsi.fastutil.longs.LongArrayList; 10 | import it.unimi.dsi.fastutil.longs.LongList; 11 | 12 | import java.util.Objects; 13 | import java.util.concurrent.atomic.AtomicInteger; 14 | 15 | public class SparseItem { 16 | private final LongList categoricalIndexes = new LongArrayList(); 17 | private final LongList numericalIndexes = new LongArrayList(); 18 | private final DoubleList numericalValues = new DoubleArrayList(); 19 | private String id; 20 | private double label; 21 | 22 | private final LongList indexesView = new MultiListsViewLongList(categoricalIndexes, numericalIndexes); 23 | 24 | public SparseItem addCategoricalIndex(long index) { 25 | categoricalIndexes.add(normalizeIndex(index)); 26 | return this; 27 | } 28 | 29 | public SparseItem addNumericalIndex(long index, double value) { 30 | numericalIndexes.add(normalizeIndex(index)); 31 | numericalValues.add(value); 32 | return this; 33 | } 34 | 35 | private long normalizeIndex(long ind) { 36 | return Math.abs(ind) % (1L << 40); 37 | } 38 | 39 | public LongList indexes() { 40 | return indexesView; 41 | } 42 | 43 | public double getValue(int i) { 44 | return i < categoricalIndexes.size() ? 1 : numericalValues.getDouble(i - categoricalIndexes.size()); 45 | } 46 | 47 | public SparseItem label(double label) { 48 | this.label = Util.doublesEqual(1., label) ? 1. : 0.; 49 | return this; 50 | } 51 | 52 | public LongList categoricalIndexes() { 53 | return categoricalIndexes; 54 | } 55 | 56 | public LongList numericalIndexes() { 57 | return numericalIndexes; 58 | } 59 | 60 | public DoubleList numericalValues() { 61 | return numericalValues; 62 | } 63 | 64 | public SparseItem id(String id) { 65 | this.id = id; 66 | return this; 67 | } 68 | 69 | public double label() { 70 | return label; 71 | } 72 | 73 | public void clear() { 74 | label = 0.; 75 | id = null; 76 | numericalIndexes.clear(); 77 | numericalValues.clear(); 78 | categoricalIndexes.clear(); 79 | } 80 | 81 | public String id() { 82 | return id; 83 | } 84 | 85 | public void write(LineBytesBuffer line) { 86 | line.putByte((byte) Math.signum(label)); 87 | line.putString(id); 88 | line.putShort((short) categoricalIndexes.size()); 89 | for (long index : categoricalIndexes) { 90 | line.putLong(index); 91 | } 92 | line.putShort((short) numericalIndexes.size()); 93 | for (int i = 0; i < numericalIndexes.size(); i++) { 94 | line.putLong(numericalIndexes.getLong(i)); 95 | line.putFloat((float) numericalValues.getDouble(i)); 96 | } 97 | } 98 | 99 | public void read(LineBytesBuffer line) { 100 | clear(); 101 | AtomicInteger cursor = new AtomicInteger(0); 102 | label = line.readByte(cursor) > 0 ? 1. : 0; 103 | id = line.readString(cursor); 104 | short factorsSize = line.readShort(cursor); 105 | for (short i = 0; i < factorsSize; i++) { 106 | categoricalIndexes.add(line.readLong(cursor)); 107 | } 108 | short numericalSize = line.readShort(cursor); 109 | for (short i = 0; i < numericalSize; i++) { 110 | numericalIndexes.add(line.readLong(cursor)); 111 | numericalValues.add(line.readFloat(cursor)); 112 | } 113 | } 114 | 115 | @Override 116 | public boolean equals(Object o) { 117 | if (this == o) { 118 | return true; 119 | } 120 | if (o == null || getClass() != o.getClass()) { 121 | return false; 122 | } 123 | SparseItem that = (SparseItem) o; 124 | return Objects.equals(id, that.id) && 125 | Objects.equals(label, that.label) && 126 | Objects.equals(categoricalIndexes, that.categoricalIndexes) && 127 | Objects.equals(numericalIndexes, that.numericalIndexes) && 128 | equalNumericalValues(that); 129 | } 130 | 131 | private boolean equalNumericalValues(SparseItem that) { 132 | if (numericalValues.size() != that.numericalValues.size()) { 133 | return false; 134 | } 135 | for (int i = 0; i < numericalValues.size(); i++) { 136 | if (!Util.doublesEqual(numericalValues.getDouble(i), that.numericalValues.getDouble(i), 0.00001)) { 137 | return false; 138 | } 139 | } 140 | return true; 141 | } 142 | 143 | @Override 144 | public int hashCode() { 145 | return Objects.hash(id, label, categoricalIndexes, numericalIndexes); 146 | } 147 | 148 | @Override 149 | public String toString() { 150 | return MoreObjects.toStringHelper(getClass()) 151 | .add("id", id) 152 | .add("label", label) 153 | .add("categorialIndexes", categoricalIndexes) 154 | .add("numericalIndexes", numericalIndexes) 155 | .add("numericalValues", numericalValues).toString(); 156 | } 157 | 158 | public double scalarMultiply(DoubleArrayList weights) { 159 | double sum = 0.; 160 | for (int i = 0; i < categoricalIndexes.size(); i++) { 161 | sum += weights.getDouble(i); 162 | } 163 | for (int i = 0; i < numericalIndexes.size(); i++) { 164 | sum += numericalValues.getDouble(i) * weights.getDouble(i + categoricalIndexes.size()); 165 | } 166 | return sum; 167 | } 168 | 169 | public SparseItem copyCategoricalFeaturesFrom(SparseItem item) { 170 | categoricalIndexes.addAll(item.categoricalIndexes); 171 | return this; 172 | } 173 | } 174 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/TwoPhaseEvent.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core; 2 | 3 | import com.lmax.disruptor.EventFactory; 4 | import io.scaledml.core.util.LineBytesBuffer; 5 | 6 | public class TwoPhaseEvent { 7 | public static EventFactory> factory(EventFactory outputFactory) { 8 | return () -> new TwoPhaseEvent<>(outputFactory.newInstance()); 9 | } 10 | 11 | private final LineBytesBuffer input = new LineBytesBuffer(); 12 | private long lineNo; 13 | private final T output; 14 | 15 | public TwoPhaseEvent(T output) { 16 | this.output = output; 17 | } 18 | 19 | public LineBytesBuffer input() { 20 | return input; 21 | } 22 | 23 | public T output() { 24 | return output; 25 | } 26 | 27 | public TwoPhaseEvent lineNo(long lineNo) { 28 | this.lineNo = lineNo; 29 | return this; 30 | } 31 | 32 | public long lineNo() { 33 | return lineNo; 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/inputformats/AbstractDelimiterSeparatedValuesFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import com.google.inject.Inject; 4 | import com.google.inject.name.Named; 5 | import io.scaledml.core.SparseItem; 6 | import io.scaledml.core.util.LineBytesBuffer; 7 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 8 | 9 | /** 10 | * Created by artem on 7/8/15. 11 | */ 12 | public abstract class AbstractDelimiterSeparatedValuesFormat implements InputFormat { 13 | private final LineBytesBuffer valueBuffer = new LineBytesBuffer(); 14 | protected final LineBytesBuffer namespaceBuffer = new LineBytesBuffer(); 15 | protected FeaturesProcessor featuresProcessor; 16 | 17 | 18 | @Override 19 | public void parse(LineBytesBuffer line, SparseItem item, long lineNo) { 20 | item.clear(); 21 | valueBuffer.clear(); 22 | int colNum = 0; 23 | for (int i = 0; i < line.size(); i++) { 24 | byte b = line.get(i); 25 | if (((char) b) != csvDelimiter()) { 26 | valueBuffer.append(b); 27 | } else { 28 | processColumn(item, colNum, valueBuffer); 29 | colNum++; 30 | valueBuffer.clear(); 31 | } 32 | } 33 | processColumn(item, colNum, valueBuffer); 34 | if (item.id() == null) { 35 | item.id(Long.toString(lineNo)); 36 | } 37 | finalize(item); 38 | featuresProcessor.finalize(item); 39 | } 40 | 41 | protected void finalize(SparseItem item) { 42 | 43 | } 44 | 45 | protected abstract char csvDelimiter(); 46 | 47 | protected abstract void processColumn(SparseItem item, int colNum, LineBytesBuffer valueBuffer); 48 | 49 | @Inject 50 | public AbstractDelimiterSeparatedValuesFormat featruresProcessor(FeaturesProcessor featuresProcessor) { 51 | this.featuresProcessor = featuresProcessor; 52 | return this; 53 | } 54 | } 55 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/inputformats/BinaryInputFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | 4 | import io.scaledml.core.SparseItem; 5 | import io.scaledml.core.util.LineBytesBuffer; 6 | 7 | public class BinaryInputFormat implements InputFormat { 8 | 9 | @Override 10 | public void parse(LineBytesBuffer line, SparseItem item, long lineNo) { 11 | item.read(line); 12 | } 13 | } 14 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/inputformats/CSVFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import com.google.inject.Inject; 4 | import com.google.inject.name.Named; 5 | import io.scaledml.core.SparseItem; 6 | import io.scaledml.core.util.LineBytesBuffer; 7 | 8 | /** 9 | * @author Ilya Smagin ilya-sm@yandex-team.ru on 4/2/15. 10 | */ 11 | public class CSVFormat extends AbstractDelimiterSeparatedValuesFormat { 12 | 13 | private ColumnsMask columnsMask; 14 | private char csvDelimiter = ','; 15 | 16 | @Override 17 | protected void processColumn(SparseItem item, int colNum, LineBytesBuffer valueBuffer) { 18 | if (valueBuffer.empty()) { 19 | return; 20 | } 21 | namespaceBuffer.clear(); 22 | namespaceBuffer.putShort((short) colNum); 23 | ColumnsMask.ColumnType columnType = columnsMask.getCategory(colNum); 24 | switch (columnType) { 25 | case LABEL: 26 | double label = Double.parseDouble(valueBuffer.toAsciiString()); 27 | item.label(label); 28 | break; 29 | case ID: 30 | item.id(valueBuffer.toString()); 31 | break; 32 | case NUMERICAL: 33 | double value = Double.parseDouble(valueBuffer.toAsciiString()); 34 | featuresProcessor.addNumericalFeature(item, namespaceBuffer, namespaceBuffer, value); 35 | break; 36 | case CATEGORICAL: 37 | featuresProcessor.addCategoricalFeature(item, namespaceBuffer, valueBuffer); 38 | break; 39 | } 40 | } 41 | 42 | @Override 43 | protected char csvDelimiter() { 44 | return csvDelimiter; 45 | } 46 | 47 | @Inject 48 | public CSVFormat csvMask(@Named("csvMask") ColumnsMask columnsMask) { 49 | this.columnsMask = columnsMask; 50 | return this; 51 | } 52 | 53 | @Inject 54 | public CSVFormat csvDelimiter(@Named("csvDelimiter") char csvDelimiter) { 55 | this.csvDelimiter = csvDelimiter; 56 | return this; 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/inputformats/ColumnsMask.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import java.util.ArrayList; 4 | 5 | public class ColumnsMask { 6 | private final String str; 7 | private ArrayList columns = new ArrayList<>(); 8 | 9 | public ColumnsMask(String str) { 10 | this.str = str; 11 | char[] arr = str.toCharArray(); 12 | State state = State.OUT_OF_BRACKET; 13 | StringBuilder sb = new StringBuilder(); 14 | for (char c : arr) { 15 | switch (state) { 16 | case OUT_OF_BRACKET: 17 | switch (c) { 18 | case 'l': 19 | case 'i': 20 | case 'n': 21 | case 'c': 22 | columns.add(charToColumnType(c)); 23 | break; 24 | case '[': 25 | state = State.IN_BRACKET; 26 | break; 27 | default: 28 | throw new IllegalArgumentException(); 29 | } 30 | break; 31 | case IN_BRACKET: 32 | switch (c) { 33 | case '1': 34 | case '2': 35 | case '3': 36 | case '4': 37 | case '5': 38 | case '6': 39 | case '7': 40 | case '8': 41 | case '9': 42 | case '0': 43 | sb.append(c); 44 | break; 45 | case ']': 46 | int repeats = Integer.parseInt(sb.toString()) - 1; 47 | ColumnType last = columns.get(columns.size() - 1); 48 | for (int i = 0; i < repeats; i++) { 49 | columns.add(last); 50 | } 51 | sb.setLength(0); 52 | state = State.OUT_OF_BRACKET; 53 | break; 54 | default: 55 | throw new IllegalArgumentException(); 56 | } 57 | break; 58 | } 59 | 60 | } 61 | } 62 | 63 | private static ColumnType charToColumnType(char c) { 64 | switch (c) { 65 | case 'l': 66 | return ColumnType.LABEL; 67 | case 'i': 68 | return ColumnType.ID; 69 | case 'n': 70 | return ColumnType.NUMERICAL; 71 | case 'c': 72 | return ColumnType.CATEGORICAL; 73 | default: 74 | throw new IllegalArgumentException("" + c); 75 | } 76 | } 77 | 78 | @Override 79 | public String toString() { 80 | return str; 81 | } 82 | 83 | public ColumnType getCategory(int colNumber) { 84 | if (colNumber < columns.size()) { 85 | return columns.get(colNumber); 86 | } 87 | return columns.get(columns.size() - 1); 88 | } 89 | 90 | public static enum ColumnType { 91 | LABEL, 92 | ID, 93 | NUMERICAL, 94 | CATEGORICAL 95 | } 96 | 97 | private static enum State { 98 | IN_BRACKET, OUT_OF_BRACKET 99 | } 100 | 101 | 102 | } 103 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/inputformats/InputFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import io.scaledml.core.SparseItem; 4 | import io.scaledml.core.util.LineBytesBuffer; 5 | 6 | public interface InputFormat { 7 | void parse(LineBytesBuffer line, SparseItem item, long lineNo); 8 | } 9 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/inputformats/VowpalWabbitFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import com.google.common.base.CharMatcher; 4 | import com.google.inject.Inject; 5 | import io.scaledml.core.SparseItem; 6 | import io.scaledml.core.util.LineBytesBuffer; 7 | import io.scaledml.core.util.Util; 8 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 9 | 10 | public class VowpalWabbitFormat implements InputFormat { 11 | private static final char NAME_CHAR = 'z'; 12 | private final LineBytesBuffer feature = new LineBytesBuffer(); 13 | private final LineBytesBuffer namespace = new LineBytesBuffer(); 14 | private final LineBytesBuffer number = new LineBytesBuffer(); 15 | 16 | private FeaturesProcessor featuresProcessor; 17 | 18 | private enum State { 19 | BEFORE_LABEL, LABEL, AFTER_LABEL, BEFORE_NAMESPACE, NAMESPACE, BEFORE_FEATURE, FEATURE, FEATURE_VALUE 20 | } 21 | 22 | private final CharMatcher NUMBER_MATCHER = CharMatcher.DIGIT.or(CharMatcher.anyOf(".-")).precomputed(); 23 | private final CharMatcher NAME_MATCHER = CharMatcher.JAVA_LETTER_OR_DIGIT.or(CharMatcher.anyOf("_=-")).precomputed(); 24 | private final CharMatcher PIPE_MATCHER = CharMatcher.anyOf("|").precomputed(); 25 | private final CharMatcher COLON_MATCHER = CharMatcher.anyOf(":").precomputed(); 26 | 27 | 28 | @Override 29 | public void parse(LineBytesBuffer line, SparseItem item, long lineNo) { 30 | item.clear(); 31 | feature.clear(); 32 | namespace.clear(); 33 | number.clear(); 34 | item.id(Long.toString(lineNo)); 35 | State state = State.BEFORE_LABEL; 36 | for (int i = 0; i < line.size(); i++) { 37 | byte b = line.get(i); 38 | char c = b > 0 ? (char) b : NAME_CHAR; 39 | switch (state) { 40 | case BEFORE_LABEL: 41 | if (NUMBER_MATCHER.matches(c)) { 42 | number.append(b); 43 | state = State.LABEL; 44 | } 45 | break; 46 | case LABEL: 47 | if (NUMBER_MATCHER.matches(c)) { 48 | number.append(b); 49 | } else { 50 | item.label(Util.doublesEqual(1., Double.parseDouble(number.toAsciiString())) ? 1. : 0.); 51 | number.clear(); 52 | state = State.AFTER_LABEL; 53 | } 54 | break; 55 | case AFTER_LABEL: 56 | if (PIPE_MATCHER.matches(c)) { 57 | state = State.BEFORE_NAMESPACE; 58 | } 59 | break; 60 | case BEFORE_NAMESPACE: 61 | if (NAME_MATCHER.matches(c)) { 62 | namespace.append(b); 63 | state = State.NAMESPACE; 64 | } 65 | break; 66 | case NAMESPACE: 67 | if (NAME_MATCHER.matches(c)) { 68 | namespace.append(b); 69 | } else { 70 | state = State.BEFORE_FEATURE; 71 | } 72 | break; 73 | case BEFORE_FEATURE: 74 | if (NAME_MATCHER.matches(c)) { 75 | feature.append(b); 76 | state = State.FEATURE; 77 | } else if (PIPE_MATCHER.matches(c)) { 78 | state = State.BEFORE_NAMESPACE; 79 | namespace.clear(); 80 | } 81 | break; 82 | case FEATURE: 83 | if (NAME_MATCHER.matches(c)) { 84 | feature.append(b); 85 | } else if (PIPE_MATCHER.matches(c)) { 86 | addCategoricalIndex(item); 87 | state = State.BEFORE_NAMESPACE; 88 | namespace.clear(); 89 | } else if (COLON_MATCHER.matches(c)) { 90 | state = State.FEATURE_VALUE; 91 | } else { 92 | addCategoricalIndex(item); 93 | state = State.BEFORE_FEATURE; 94 | } 95 | break; 96 | case FEATURE_VALUE: 97 | if (NUMBER_MATCHER.matches(c)) { 98 | number.append(b); 99 | } else if (PIPE_MATCHER.matches(c)) { 100 | addNumericalIndex(item, Double.parseDouble(number.toAsciiString())); 101 | state = State.BEFORE_NAMESPACE; 102 | namespace.clear(); 103 | } else { 104 | addNumericalIndex(item, Double.parseDouble(number.toAsciiString())); 105 | state = State.BEFORE_FEATURE; 106 | } 107 | } 108 | } 109 | if (state == State.FEATURE) { 110 | addCategoricalIndex(item); 111 | } 112 | if (state == State.FEATURE_VALUE) { 113 | addNumericalIndex(item, Double.parseDouble(number.toAsciiString())); 114 | } 115 | featuresProcessor.finalize(item); 116 | } 117 | 118 | private void addNumericalIndex(SparseItem item, double value) { 119 | featuresProcessor.addNumericalFeature(item, namespace, feature, value); 120 | feature.clear(); 121 | number.clear(); 122 | } 123 | 124 | private void addCategoricalIndex(SparseItem item) { 125 | featuresProcessor.addCategoricalFeature(item, namespace, feature); 126 | feature.clear(); 127 | number.clear(); 128 | } 129 | 130 | @Inject 131 | public VowpalWabbitFormat featruresProcessor(FeaturesProcessor featuresProcessor) { 132 | this.featuresProcessor = featuresProcessor; 133 | return this; 134 | } 135 | } 136 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/outputformats/NullOutputFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.outputformats; 2 | 3 | 4 | import io.scaledml.core.SparseItem; 5 | 6 | import java.io.IOException; 7 | 8 | public class NullOutputFormat implements OutputFormat { 9 | @Override 10 | public void emit(SparseItem item, double prediction) { 11 | } 12 | 13 | @Override 14 | public void close() throws IOException { 15 | } 16 | } 17 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/outputformats/OutputFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.outputformats; 2 | 3 | import io.scaledml.core.SparseItem; 4 | 5 | import java.io.Closeable; 6 | 7 | public interface OutputFormat extends Closeable { 8 | void emit(SparseItem item, double prediction); 9 | } 10 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/outputformats/PrintStreamOutputFormat.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.outputformats; 2 | 3 | import io.scaledml.core.SparseItem; 4 | 5 | import java.io.IOException; 6 | import java.io.PrintStream; 7 | 8 | public class PrintStreamOutputFormat implements OutputFormat { 9 | private PrintStream outputStream; 10 | 11 | @Override 12 | public void emit(SparseItem item, double prediction) { 13 | outputStream.print(item.id() + "\t" + prediction + "\n"); 14 | } 15 | 16 | @Override 17 | public void close() throws IOException { 18 | outputStream.close(); 19 | } 20 | 21 | public PrintStreamOutputFormat outputStream(PrintStream outputStream) { 22 | this.outputStream = outputStream; 23 | return this; 24 | } 25 | } 26 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/util/LineBytesBuffer.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.util; 2 | 3 | import com.google.common.base.Charsets; 4 | import it.unimi.dsi.fastutil.bytes.ByteArrays; 5 | import it.unimi.dsi.fastutil.io.FastBufferedInputStream; 6 | 7 | import java.io.IOException; 8 | import java.util.Arrays; 9 | import java.util.concurrent.atomic.AtomicInteger; 10 | 11 | public class LineBytesBuffer implements Comparable { 12 | private static final byte MINUS_CODE = 0; 13 | private static final byte NL_CODE = 1; 14 | private static final byte RETURN_CODE = 2; 15 | private byte[] bytes; 16 | private int size; 17 | 18 | public LineBytesBuffer() { 19 | bytes = new byte[1024]; 20 | size = 0; 21 | } 22 | 23 | public LineBytesBuffer(String str) { 24 | this.bytes = str.getBytes(Charsets.UTF_8); 25 | size = bytes.length; 26 | } 27 | 28 | public LineBytesBuffer(LineBytesBuffer other) { 29 | this.bytes = Arrays.copyOf(other.bytes, other.size); 30 | size = other.size; 31 | } 32 | 33 | public void drainTo(LineBytesBuffer other) { 34 | byte[] tmp = other.bytes; 35 | other.bytes = bytes; 36 | other.size = size; 37 | bytes = tmp; 38 | size = 0; 39 | } 40 | 41 | public void setContentOf(LineBytesBuffer feature) { 42 | size = feature.size; 43 | bytes = ByteArrays.ensureCapacity(bytes, size); 44 | System.arraycopy(feature.bytes, 0, bytes, 0, size); 45 | } 46 | 47 | public int size() { 48 | return size; 49 | } 50 | 51 | public byte get(int i) { 52 | return bytes[i]; 53 | } 54 | 55 | public boolean readLineFrom(FastBufferedInputStream stream) throws IOException { 56 | int start = 0, len; 57 | while ((len = stream.readLine(bytes, start, bytes.length - start)) == bytes.length - start) { 58 | start += len; 59 | bytes = ByteArrays.grow(bytes, bytes.length + 1024); 60 | } 61 | size = start + Math.max(len, 0); 62 | return !(size == 0 && len < 0); 63 | } 64 | 65 | public String toAsciiString() { 66 | return new String(bytes, 0, size, Charsets.US_ASCII); 67 | } 68 | 69 | @Override 70 | public String toString() { 71 | return toAsciiString(); 72 | } 73 | 74 | public byte[] bytes() { 75 | return bytes; 76 | } 77 | 78 | public void append(byte b) { 79 | if (size >= bytes.length - 1) { 80 | bytes = ByteArrays.grow(bytes, bytes.length + 1024); 81 | } 82 | switch (b) { 83 | case -1: 84 | bytes[size++] = -1; 85 | bytes[size++] = MINUS_CODE; 86 | break; 87 | case (byte) '\n': 88 | bytes[size++] = -1; 89 | bytes[size++] = NL_CODE; 90 | break; 91 | case (byte) '\r': 92 | bytes[size++] = -1; 93 | bytes[size++] = RETURN_CODE; 94 | break; 95 | default: 96 | bytes[size++] = b; 97 | } 98 | 99 | } 100 | 101 | public void clear() { 102 | size = 0; 103 | } 104 | 105 | @Override 106 | public int compareTo(LineBytesBuffer o) { 107 | int minSize = Math.min(size, o.size); 108 | for (int i = 0; i < minSize; i++) { 109 | int byteCompare = Byte.compare(get(i), o.get(i)); 110 | if (byteCompare != 0) { 111 | return byteCompare; 112 | } 113 | } 114 | return Integer.compare(size, o.size); 115 | } 116 | 117 | public boolean empty() { 118 | return size == 0; 119 | } 120 | 121 | public int putByte(byte b) { 122 | append(b); 123 | return 1; 124 | } 125 | 126 | public int putShort(short num) { 127 | append((byte) (num)); 128 | append((byte) (num >> 8)); 129 | return 2; 130 | } 131 | public int putString(String str) { 132 | byte[] strBytes = str.getBytes(Charsets.US_ASCII); 133 | assert strBytes.length < Short.MAX_VALUE; 134 | putShort((short) strBytes.length); 135 | bytes = ByteArrays.ensureCapacity(bytes, size + strBytes.length); 136 | System.arraycopy(strBytes, 0, bytes, size, strBytes.length); 137 | size += strBytes.length; 138 | return strBytes.length + 2; 139 | } 140 | 141 | public int putLong(long num) { 142 | assert num >= 0 && num < (1L << 40); 143 | append((byte) (num)); 144 | append((byte) (num >> 8)); 145 | append((byte) (num >> 16)); 146 | append((byte) (num >> 24)); 147 | append((byte) (num >> 32)); 148 | return 5; 149 | } 150 | 151 | public int putInteger(int num) { 152 | append((byte) (num)); 153 | append((byte) (num >> 8)); 154 | append((byte) (num >> 16)); 155 | append((byte) (num >> 24)); 156 | return 4; 157 | } 158 | 159 | public int putFloat(float num) { 160 | return putInteger(Float.floatToIntBits(num)); 161 | } 162 | 163 | public short readShort(AtomicInteger cursor) { 164 | return (short) (((readByte(cursor) & 0xff)) | 165 | ((readByte(cursor) & 0xff) << 8)); 166 | } 167 | 168 | public long readLong(AtomicInteger cursor) { 169 | return ((readByte(cursor) & 0xffL) | 170 | ((readByte(cursor) & 0xffL) << 8) | 171 | ((readByte(cursor) & 0xffL) << 16) | 172 | ((readByte(cursor) & 0xffL) << 24) | 173 | ((readByte(cursor) & 0xffL) << 32)); 174 | } 175 | 176 | 177 | 178 | public String readString(AtomicInteger cursor) { 179 | short size = readShort(cursor); 180 | int start = cursor.get(); 181 | return new String(bytes, start, cursor.addAndGet(size) - start, Charsets.US_ASCII); 182 | } 183 | 184 | public byte readByte(AtomicInteger cursor) { 185 | byte b = get(cursor.getAndIncrement()); 186 | if (b != -1) { 187 | return b; 188 | } 189 | byte next = get(cursor.getAndIncrement()); 190 | switch (next) { 191 | case MINUS_CODE: 192 | return -1; 193 | case NL_CODE: 194 | return '\n'; 195 | case RETURN_CODE: 196 | return '\r'; 197 | default: 198 | throw new IllegalStateException(); 199 | } 200 | } 201 | 202 | 203 | public float readFloat(AtomicInteger cursor) { 204 | return Float.intBitsToFloat(readInt(cursor)); 205 | } 206 | 207 | public int readInt(AtomicInteger cursor) { 208 | return ((readByte(cursor) & 0xff) | 209 | ((readByte(cursor) & 0xff) << 8) | 210 | ((readByte(cursor) & 0xff) << 16) | 211 | ((readByte(cursor)) << 24)); 212 | } 213 | 214 | @Override 215 | public boolean equals(Object o) { 216 | if (this == o) { 217 | return true; 218 | } 219 | if (o == null || getClass() != o.getClass()) { 220 | return false; 221 | } 222 | LineBytesBuffer that = (LineBytesBuffer) o; 223 | if (size != that.size) { 224 | return false; 225 | } 226 | for (int i = 0; i < size; i++) { 227 | if (get(i) != that.get(i)) { 228 | return false; 229 | } 230 | } 231 | return true; 232 | } 233 | 234 | @Override 235 | public int hashCode() { 236 | return Util.murmur32().hashBytes(bytes, 0, size).hashCode(); 237 | } 238 | } 239 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/util/MultiListsViewLongList.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.util; 2 | 3 | 4 | import it.unimi.dsi.fastutil.longs.AbstractLongList; 5 | import it.unimi.dsi.fastutil.longs.LongList; 6 | 7 | public class MultiListsViewLongList extends AbstractLongList { 8 | private final LongList[] lists; 9 | 10 | public MultiListsViewLongList(LongList... lists) { 11 | this.lists = lists; 12 | } 13 | 14 | @Override 15 | public int size() { 16 | int size = 0; 17 | for (LongList list : lists) { 18 | size += list.size(); 19 | } 20 | return size; 21 | } 22 | 23 | @Override 24 | public long getLong(int i) { 25 | for (LongList list : lists) { 26 | if (i < list.size()) { 27 | return list.getLong(i); 28 | } 29 | i -= list.size(); 30 | } 31 | throw new ArrayIndexOutOfBoundsException(); 32 | } 33 | } 34 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/core/util/Util.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.util; 2 | 3 | import com.google.common.hash.HashFunction; 4 | import com.google.common.hash.Hashing; 5 | import com.google.common.primitives.Doubles; 6 | 7 | public class Util { 8 | private final static HashFunction murmur = Hashing.murmur3_128(42); 9 | 10 | private final static HashFunction murmur32 = Hashing.murmur3_32(17); 11 | public static final double EPSILON = 0.0000001; 12 | 13 | public static boolean doublesEqual(double d1, double d2, double precision) { 14 | if (!Doubles.isFinite(d1) || !Doubles.isFinite(d2)) { 15 | return false; 16 | } 17 | return Math.abs(d1 - d2) < precision; 18 | } 19 | 20 | public static boolean doublesEqual(double d1, double d2) { 21 | return doublesEqual(d1, d2, EPSILON); 22 | } 23 | 24 | public static HashFunction murmur() { 25 | return murmur; 26 | } 27 | 28 | public static HashFunction murmur32() { 29 | return murmur32; 30 | } 31 | 32 | public static boolean doublesLess(double v1, double v2) { 33 | return v1 < v2 - EPSILON; 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/Binning.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import io.scaledml.core.util.Util; 4 | import it.unimi.dsi.fastutil.doubles.*; 5 | import java.util.Collections; 6 | 7 | public class Binning { 8 | private final DoubleList percentiles = new DoubleArrayList(); 9 | 10 | public Binning addPercentile(double percentile) { 11 | percentiles.add(percentile); 12 | return this; 13 | } 14 | 15 | public Binning finishBuild() { 16 | percentiles.sort(Double::compare); 17 | for (int i = 0; i < percentiles.size() - 1; i++) { 18 | if (Util.doublesEqual(percentiles.getDouble(i), percentiles.getDouble(i + 1), 0.0001)) { 19 | percentiles.set(i + 1, percentiles.getDouble(i)); 20 | } 21 | } 22 | return this; 23 | } 24 | 25 | 26 | public double roundToPercentile(double value) { 27 | int i = getInsertionPoint(value); 28 | if (i >= 0) { 29 | return percentiles.getDouble(i); 30 | } 31 | throw new IllegalArgumentException(value + " is not a part of sample"); 32 | } 33 | 34 | int getInsertionPoint(double value) { 35 | int i = Collections.binarySearch(percentiles, value); 36 | if (i >= 0) { 37 | return i; 38 | } 39 | return -i - 2; 40 | } 41 | 42 | public int getNumberOfValuesBetween(double from, double to) { 43 | return getInsertionPoint(to - Util.EPSILON) - 44 | getInsertionPoint(from - Util.EPSILON); 45 | } 46 | } 47 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/BinningWorkHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.lmax.disruptor.WorkHandler; 6 | import io.scaledml.core.SparseItem; 7 | import io.scaledml.core.inputformats.InputFormat; 8 | import io.scaledml.core.util.Util; 9 | import io.scaledml.core.TwoPhaseEvent; 10 | import it.unimi.dsi.fastutil.longs.Long2ObjectMap; 11 | 12 | public class BinningWorkHandler implements WorkHandler> { 13 | private final SparseItem item = new SparseItem(); 14 | private NumericalFeaturesStatistics statistics; 15 | private InputFormat format; 16 | 17 | 18 | @Override 19 | public void onEvent(TwoPhaseEvent event) throws Exception { 20 | SparseItem outputItem = event.output(); 21 | item.clear(); 22 | outputItem.clear(); 23 | format.parse(event.input(), item, event.lineNo()); 24 | outputItem 25 | .copyCategoricalFeaturesFrom(item) 26 | .label(item.label()) 27 | .id(item.id()); 28 | Long2ObjectMap binnings = statistics.binnings(); 29 | for (int i = 0; i < item.numericalIndexes().size(); i++) { 30 | long numericalIndex = item.numericalIndexes().getLong(i); 31 | double numericalValue = item.numericalValues().getDouble(i); 32 | long binningIndex = Util.murmur().newHasher() 33 | .putLong(numericalIndex) 34 | .putDouble(binnings.get(numericalIndex).roundToPercentile(numericalValue)) 35 | .hash().asLong(); 36 | outputItem.addCategoricalIndex(binningIndex); 37 | } 38 | } 39 | 40 | @Inject 41 | public BinningWorkHandler statistics(NumericalFeaturesStatistics statistics) { 42 | this.statistics = statistics; 43 | return this; 44 | } 45 | 46 | @Inject 47 | public BinningWorkHandler format(InputFormat format) { 48 | this.format = format; 49 | return this; 50 | } 51 | } 52 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/FeatureEngineeringModule.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | 4 | import com.google.inject.*; 5 | import com.google.inject.name.Named; 6 | import com.google.inject.name.Names; 7 | import com.google.inject.throwingproviders.ThrowingProviderBinder; 8 | import com.lmax.disruptor.*; 9 | import com.lmax.disruptor.dsl.Disruptor; 10 | import com.lmax.disruptor.dsl.ProducerType; 11 | import com.lmax.disruptor.util.DaemonThreadFactory; 12 | import io.scaledml.core.SparseItem; 13 | import io.scaledml.core.inputformats.*; 14 | import io.scaledml.core.TwoPhaseEvent; 15 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 16 | import io.scaledml.ftrl.options.FtrlOptions; 17 | import it.unimi.dsi.fastutil.io.FastBufferedOutputStream; 18 | import java.io.IOException; 19 | import java.io.InputStream; 20 | import java.nio.file.Files; 21 | import java.nio.file.Paths; 22 | import java.util.concurrent.ExecutorService; 23 | import java.util.concurrent.Executors; 24 | import java.util.concurrent.Phaser; 25 | import java.util.function.Supplier; 26 | 27 | public class FeatureEngineeringModule extends AbstractModule { 28 | protected final ExecutorService threadsProvider = Executors.newCachedThreadPool(DaemonThreadFactory.INSTANCE); 29 | protected final FtrlOptions options; 30 | 31 | public FeatureEngineeringModule(FtrlOptions options) { 32 | this.options = options; 33 | } 34 | 35 | @Override 36 | protected void configure() { 37 | ThrowingProviderBinder.forModule(this); 38 | bindConstant().annotatedWith(Names.named("testOnly")).to(options.testOnly()); 39 | bindConstant().annotatedWith(Names.named("skipFirst")).to(options.skipFirst()); 40 | bindConstant().annotatedWith(Names.named("percentsBinningStep")).to(0.01); 41 | switch (options.format()) { 42 | case vw: 43 | bind(InputFormat.class).to(VowpalWabbitFormat.class); 44 | break; 45 | case csv: 46 | ColumnsMask columnsMask = new ColumnsMask(options.csvMask()); 47 | bindConstant().annotatedWith(Names.named("csvDelimiter")).to(options.csvDelimiter()); 48 | bind(new TypeLiteral() { 49 | }).annotatedWith(Names.named("csvMask")).toInstance(columnsMask); 50 | bind(InputFormat.class).to(CSVFormat.class); 51 | break; 52 | case binary: 53 | bind(InputFormat.class).to(BinaryInputFormat.class); 54 | break; 55 | default: 56 | throw new IllegalArgumentException(options.format().toString()); 57 | } 58 | bind(FirstPassRunner.class); 59 | bind(SecondPassRunner.class); 60 | bind(FeatureEngineeringRunner.class); 61 | bind(Phaser.class).asEagerSingleton(); 62 | bind(FeaturesProcessor.class); 63 | try { 64 | bind(new TypeLiteral>() {}).toInstance(this::openInputFile); 65 | bind(FastBufferedOutputStream.class).toInstance(new FastBufferedOutputStream( 66 | Files.newOutputStream(Paths.get(options.predictions())))); 67 | 68 | } catch (IOException e) { 69 | throw new IllegalArgumentException(e); 70 | } 71 | bind(NumericalFeaturesStatistics.class).asEagerSingleton(); 72 | bind(new TypeLiteral>>() { 73 | }) 74 | .to(StatisticsWorkHandler.class); 75 | bind(new TypeLiteral>>() { 76 | }) 77 | .to(BinningWorkHandler.class); 78 | bind(new TypeLiteral>>() {}) 79 | .to(OutputWriterEventHandler.class); 80 | } 81 | 82 | @Provides 83 | @Singleton 84 | @Named("firstPassDisruptor") 85 | public Disruptor> firstPassDisruptor( 86 | Provider>> statisticsHandlerProvider) { 87 | Disruptor> disruptor = new Disruptor<>( 88 | TwoPhaseEvent.factory(() -> null), 89 | options.ringSize(), threadsProvider, 90 | ProducerType.SINGLE, new SleepingWaitStrategy()); 91 | WorkHandler>[] parsers = new WorkHandler[options.threads()]; 92 | for (int i = 0; i < options.threads(); i++) { 93 | parsers[i] = statisticsHandlerProvider.get(); 94 | } 95 | disruptor.handleExceptionsWith(new FatalExceptionHandler()); 96 | disruptor.handleEventsWithWorkerPool(parsers); 97 | return disruptor; 98 | } 99 | 100 | @Provides 101 | @Singleton 102 | @Named("secondPassDisruptor") 103 | public Disruptor> secondPassDisruptor( 104 | Provider>> binningHandlerProvider, 105 | Provider>> outputHandlerProvider) { 106 | Disruptor> disruptor = new Disruptor<>( 107 | TwoPhaseEvent.factory(SparseItem::new), 108 | options.ringSize(), threadsProvider, 109 | ProducerType.SINGLE, new SleepingWaitStrategy()); 110 | WorkHandler>[] parsers = new WorkHandler[options.threads()]; 111 | for (int i = 0; i < options.threads(); i++) { 112 | parsers[i] = binningHandlerProvider.get(); 113 | } 114 | disruptor.handleExceptionsWith(new FatalExceptionHandler()); 115 | disruptor.handleEventsWithWorkerPool(parsers) 116 | .then(outputHandlerProvider.get()); 117 | return disruptor; 118 | } 119 | 120 | private InputStream openInputFile() { 121 | try { 122 | return Files.newInputStream(Paths.get(options.data())); 123 | } catch (IOException e) { 124 | throw new RuntimeException(e); 125 | } 126 | } 127 | } 128 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/FeatureEngineeringRunner.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import com.google.inject.Inject; 4 | 5 | import java.io.IOException; 6 | 7 | public class FeatureEngineeringRunner { 8 | private FirstPassRunner firstPassRunner; 9 | private SecondPassRunner secondPassRunner; 10 | private NumericalFeaturesStatistics statistics; 11 | 12 | public void process() throws IOException { 13 | firstPassRunner.process(); 14 | statistics.logFeaturesStatistics(); 15 | secondPassRunner.process(); 16 | } 17 | 18 | @Inject 19 | public FeatureEngineeringRunner firstPassRunner(FirstPassRunner firstPassRunner) { 20 | this.firstPassRunner = firstPassRunner; 21 | return this; 22 | } 23 | 24 | @Inject 25 | public FeatureEngineeringRunner secondPassRunner(SecondPassRunner secondPassRunner) { 26 | this.secondPassRunner = secondPassRunner; 27 | return this; 28 | } 29 | 30 | @Inject 31 | public FeatureEngineeringRunner statistics(NumericalFeaturesStatistics statistics) { 32 | this.statistics = statistics; 33 | return this; 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/FirstPassRunner.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.google.inject.name.Named; 6 | import com.lmax.disruptor.dsl.Disruptor; 7 | import io.scaledml.core.BaseDisruptorRunner; 8 | import io.scaledml.core.TwoPhaseEvent; 9 | 10 | import java.io.IOException; 11 | 12 | public class FirstPassRunner extends BaseDisruptorRunner { 13 | @Override 14 | protected void afterDisruptorProcessed() throws IOException { 15 | } 16 | 17 | @Inject 18 | public FirstPassRunner disruptor(@Named("firstPassDisruptor") Disruptor> disruptor) { 19 | setDisruptor(disruptor); 20 | return this; 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/NumericalFeaturesStatistics.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import com.clearspring.analytics.stream.quantile.TDigest; 4 | import com.clearspring.analytics.util.Preconditions; 5 | import com.google.inject.Inject; 6 | import com.google.inject.name.Named; 7 | import io.scaledml.core.util.Util; 8 | import it.unimi.dsi.fastutil.doubles.Double2DoubleArrayMap; 9 | import it.unimi.dsi.fastutil.doubles.Double2DoubleMap; 10 | import it.unimi.dsi.fastutil.longs.*; 11 | import org.slf4j.Logger; 12 | import org.slf4j.LoggerFactory; 13 | 14 | public class NumericalFeaturesStatistics { 15 | private static final Logger statisticsLogger = LoggerFactory.getLogger("statistics-logger"); 16 | private final Long2ObjectMap digests = new Long2ObjectLinkedOpenHashMap<>(); 17 | private Long2ObjectMap binnings = null; 18 | private Long2ObjectMap histograms = null; 19 | private double percentsBinningStep; 20 | private final Long2DoubleMap minimums = new Long2DoubleLinkedOpenHashMap(); 21 | private final Long2DoubleMap maximums = new Long2DoubleLinkedOpenHashMap(); 22 | private final Long2LongMap counts = new Long2LongLinkedOpenHashMap(); 23 | 24 | public NumericalFeaturesStatistics() { 25 | maximums.defaultReturnValue(Double.MIN_VALUE); 26 | minimums.defaultReturnValue(Double.MAX_VALUE); 27 | counts.defaultReturnValue(0); 28 | } 29 | 30 | public synchronized void finishCalculateDigests(StatisticsWorkHandler handler) { 31 | for (long index : handler.digests().keySet()) { 32 | if (!digests.containsKey(index)) { 33 | digests.put(index, handler.digests().get(index)); 34 | } else { 35 | digests.get(index).add(handler.digests().get(index)); 36 | } 37 | } 38 | for (long index : handler.counts().keySet()) { 39 | long count = handler.counts().get(index); 40 | counts.put(index, counts.get(index) + count); 41 | } 42 | for (long index : handler.minimums().keySet()) { 43 | double min = handler.minimums().get(index); 44 | minimums.put(index, Math.min(minimums.get(index), min)); 45 | } 46 | for (long index : handler.maximums().keySet()) { 47 | double max = handler.maximums().get(index); 48 | maximums.put(index, Math.max(maximums.get(index), max)); 49 | } 50 | } 51 | 52 | private synchronized Long2ObjectMap buildBinning() { 53 | Long2ObjectMap newBinnings = new Long2ObjectLinkedOpenHashMap<>(); 54 | for (long index : digests.keySet()) { 55 | TDigest digest = digests.get(index); 56 | double min = minimums.get(index); 57 | Binning binning = buildBinning(digest, min); 58 | newBinnings.put(index, binning); 59 | } 60 | return newBinnings; 61 | } 62 | 63 | Binning buildBinning(TDigest digest, double min) { 64 | Binning binning = new Binning().addPercentile(min); 65 | for (double p = percentsBinningStep; Util.doublesLess(p, 1.); p += percentsBinningStep) { 66 | binning.addPercentile(digest.quantile(p)); 67 | } 68 | binning.finishBuild(); 69 | return binning; 70 | } 71 | 72 | public Long2ObjectMap binnings() { 73 | if (binnings == null) { 74 | binnings = buildBinning(); 75 | } 76 | return binnings; 77 | } 78 | 79 | public synchronized Long2ObjectMap buildHistograms() { 80 | Long2ObjectMap histograms = new Long2ObjectLinkedOpenHashMap<>(); 81 | for (long index : binnings().keySet()) { 82 | Binning binning = binnings().get(index); 83 | double max = maximums.get(index); 84 | double min = minimums.get(index); 85 | Double2DoubleMap histogram = buildHistogram(binning, min, max); 86 | histograms.put(index, histogram); 87 | } 88 | return histograms; 89 | } 90 | 91 | Double2DoubleMap buildHistogram(Binning binning, double min, double max) { 92 | double step = (max - min) / 100; 93 | Double2DoubleMap histogram = new Double2DoubleArrayMap(); 94 | for (double bucketLower = min - step; bucketLower < max + step; bucketLower += step) { 95 | int v = binning.getNumberOfValuesBetween(bucketLower, bucketLower + step); 96 | histogram.put(bucketLower, v * percentsBinningStep); 97 | } 98 | return histogram; 99 | } 100 | 101 | public Long2ObjectMap histograms() { 102 | if (histograms == null) { 103 | histograms = buildHistograms(); 104 | } 105 | return histograms; 106 | } 107 | 108 | public void logFeaturesStatistics() { 109 | for (long index : counts.keySet()) { 110 | statisticsLogger.info("Column index: " + index); 111 | statisticsLogger.info( 112 | index + "\tCount=" + counts.get(index) + 113 | "\tMax=" + maximums.get(index) + 114 | "\tMin=" + minimums.get(index)); 115 | Double2DoubleMap histogram = histograms().get(index); 116 | statisticsLogger.info("Statistics_type\tindex\tvalue\tdensity"); 117 | for (double value : histogram.keySet()) { 118 | statisticsLogger.info("histogram\t" + index + "\t" + value + "\t" + histogram.get(value)); 119 | } 120 | } 121 | } 122 | 123 | @Inject 124 | public NumericalFeaturesStatistics percentsHistogramStep( 125 | @Named("percentsBinningStep") double percentsHistogramStep) { 126 | this.percentsBinningStep = percentsHistogramStep; 127 | return this; 128 | } 129 | } 130 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/OutputWriterEventHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.lmax.disruptor.EventHandler; 6 | import com.lmax.disruptor.LifecycleAware; 7 | import io.scaledml.core.SparseItem; 8 | import io.scaledml.core.util.LineBytesBuffer; 9 | import io.scaledml.core.TwoPhaseEvent; 10 | import it.unimi.dsi.fastutil.io.FastBufferedOutputStream; 11 | 12 | import java.io.IOException; 13 | import java.util.concurrent.Phaser; 14 | 15 | public class OutputWriterEventHandler implements EventHandler>, LifecycleAware { 16 | private Phaser phaser; 17 | private FastBufferedOutputStream outputStream; 18 | private final LineBytesBuffer buffer = new LineBytesBuffer(); 19 | 20 | @Override 21 | public void onEvent(TwoPhaseEvent event, long sequence, boolean endOfBatch) throws Exception { 22 | buffer.clear(); 23 | SparseItem item = event.output(); 24 | item.write(buffer); 25 | outputStream.write(buffer.bytes(), 0, buffer.size()); 26 | outputStream.write('\n'); 27 | } 28 | 29 | @Override 30 | public void onStart() { 31 | phaser.register(); 32 | } 33 | 34 | @Override 35 | public void onShutdown() { 36 | try { 37 | outputStream.close(); 38 | } catch (IOException e) { 39 | throw new RuntimeException(e); 40 | } finally { 41 | phaser.arriveAndDeregister(); 42 | } 43 | } 44 | 45 | @Inject 46 | public OutputWriterEventHandler phaser(Phaser phaser) { 47 | this.phaser = phaser; 48 | return this; 49 | } 50 | 51 | @Inject 52 | public OutputWriterEventHandler outputStream(FastBufferedOutputStream outputStream) { 53 | this.outputStream = outputStream; 54 | return this; 55 | } 56 | } 57 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/SecondPassRunner.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import com.google.inject.Inject; 4 | import com.google.inject.name.Named; 5 | import com.lmax.disruptor.dsl.Disruptor; 6 | import io.scaledml.core.BaseDisruptorRunner; 7 | import io.scaledml.core.SparseItem; 8 | import io.scaledml.core.TwoPhaseEvent; 9 | 10 | import java.io.IOException; 11 | 12 | public class SecondPassRunner extends BaseDisruptorRunner { 13 | @Override 14 | protected void afterDisruptorProcessed() throws IOException { 15 | } 16 | 17 | @Inject 18 | public SecondPassRunner disruptor(@Named("secondPassDisruptor") Disruptor> disruptor) { 19 | super.setDisruptor(disruptor); 20 | return this; 21 | } 22 | } 23 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/features/StatisticsWorkHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import com.clearspring.analytics.stream.quantile.TDigest; 4 | import com.google.inject.Inject; 5 | import com.lmax.disruptor.LifecycleAware; 6 | import com.lmax.disruptor.WorkHandler; 7 | import io.scaledml.core.SparseItem; 8 | import io.scaledml.core.inputformats.InputFormat; 9 | import io.scaledml.core.TwoPhaseEvent; 10 | import it.unimi.dsi.fastutil.longs.*; 11 | 12 | import java.util.concurrent.Phaser; 13 | 14 | public class StatisticsWorkHandler implements WorkHandler>, LifecycleAware { 15 | private final Long2ObjectMap digests = new Long2ObjectLinkedOpenHashMap<>(); 16 | private final Long2DoubleMap minimums = new Long2DoubleOpenHashMap(); 17 | private final Long2DoubleMap maximums = new Long2DoubleOpenHashMap(); 18 | private final Long2LongMap counts = new Long2LongOpenHashMap(); 19 | private final SparseItem item = new SparseItem(); 20 | private NumericalFeaturesStatistics listener; 21 | private InputFormat format; 22 | private Phaser phaser; 23 | 24 | public StatisticsWorkHandler() { 25 | minimums.defaultReturnValue(Double.MAX_VALUE); 26 | maximums.defaultReturnValue(Double.MIN_VALUE); 27 | counts.defaultReturnValue(0); 28 | } 29 | 30 | @Override 31 | public void onEvent(TwoPhaseEvent event) throws Exception { 32 | item.clear(); 33 | format.parse(event.input(), item, event.lineNo()); 34 | for (int i = 0; i < item.numericalIndexes().size(); i++) { 35 | long index = item.numericalIndexes().getLong(i); 36 | double value = item.numericalValues().getDouble(i); 37 | if (!digests.containsKey(index)) { 38 | digests.put(index, new TDigest(100)); 39 | } 40 | counts.put(index, counts.get(index) + 1); 41 | minimums.put(index, Math.min(minimums.get(index), value)); 42 | maximums.put(index, Math.max(maximums.get(index), value)); 43 | digests.get(index).add(value); 44 | } 45 | } 46 | 47 | @Override 48 | public void onStart() { 49 | phaser.register(); 50 | } 51 | 52 | @Override 53 | public void onShutdown() { 54 | try { 55 | listener.finishCalculateDigests(this); 56 | } finally { 57 | phaser.arriveAndDeregister(); 58 | } 59 | } 60 | 61 | @Inject 62 | public StatisticsWorkHandler listener(NumericalFeaturesStatistics listener) { 63 | this.listener = listener; 64 | return this; 65 | } 66 | 67 | @Inject 68 | public StatisticsWorkHandler phaser(Phaser phaser) { 69 | this.phaser = phaser; 70 | return this; 71 | } 72 | 73 | @Inject 74 | public StatisticsWorkHandler format(InputFormat format) { 75 | this.format = format; 76 | return this; 77 | } 78 | 79 | public Long2ObjectMap digests() { 80 | return digests; 81 | } 82 | 83 | public Long2DoubleMap minimums() { 84 | return minimums; 85 | } 86 | 87 | public Long2DoubleMap maximums() { 88 | return maximums; 89 | } 90 | 91 | public Long2LongMap counts() { 92 | return counts; 93 | } 94 | } 95 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/AbstractParallelModule.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | import com.google.common.base.Throwables; 4 | import com.google.inject.*; 5 | import com.google.inject.name.Named; 6 | import com.google.inject.name.Names; 7 | import com.google.inject.throwingproviders.ThrowingProviderBinder; 8 | import com.lmax.disruptor.*; 9 | import com.lmax.disruptor.dsl.Disruptor; 10 | import com.lmax.disruptor.dsl.ProducerType; 11 | import com.lmax.disruptor.util.DaemonThreadFactory; 12 | import io.scaledml.core.TwoPhaseEvent; 13 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 14 | import io.scaledml.core.inputformats.*; 15 | import io.scaledml.core.inputformats.ColumnsMask; 16 | import io.scaledml.ftrl.options.FtrlOptions; 17 | import io.scaledml.ftrl.options.InputFormatType; 18 | import io.scaledml.ftrl.outputformats.FinishCollectStatisticsListener; 19 | import io.scaledml.core.outputformats.NullOutputFormat; 20 | import io.scaledml.core.outputformats.OutputFormat; 21 | import io.scaledml.core.outputformats.PrintStreamOutputFormat; 22 | import it.unimi.dsi.fastutil.io.FastBufferedOutputStream; 23 | 24 | import java.io.IOException; 25 | import java.io.InputStream; 26 | import java.io.PrintStream; 27 | import java.nio.file.Files; 28 | import java.nio.file.Path; 29 | import java.nio.file.Paths; 30 | import java.util.Optional; 31 | import java.util.concurrent.ExecutorService; 32 | import java.util.concurrent.Executors; 33 | import java.util.concurrent.Phaser; 34 | import java.util.function.Supplier; 35 | 36 | 37 | public abstract class AbstractParallelModule extends AbstractModule { 38 | protected final ExecutorService threadsProvider = Executors.newCachedThreadPool(DaemonThreadFactory.INSTANCE); 39 | protected final FtrlOptions options; 40 | 41 | public AbstractParallelModule(FtrlOptions options) { 42 | this.options = options; 43 | } 44 | 45 | @Provides 46 | @Singleton 47 | FtrlProximalModel model() throws Exception { 48 | if (options.initialRegressor() != null) { 49 | return FtrlProximalModel.loadModel(Paths.get(options.initialRegressor())); 50 | } 51 | return new FtrlProximalModel() 52 | .alfa(options.alfa()) 53 | .beta(options.beta()) 54 | .lambda1(options.l1()) 55 | .lambda2(options.l2()) 56 | .featuresNumber(1L << options.hashcodeBits()); 57 | } 58 | 59 | @Provides 60 | @Named("delegate") 61 | @Singleton 62 | OutputFormat delegateOutputFormat() throws IOException { 63 | if (options.predictions() == null) { 64 | return new NullOutputFormat(); 65 | } else if (options.predictions().equals("/dev/stdout")) { 66 | return new PrintStreamOutputFormat().outputStream(System.out); 67 | } 68 | return new PrintStreamOutputFormat() 69 | .outputStream(new PrintStream(new FastBufferedOutputStream( 70 | Files.newOutputStream(Paths.get(options.predictions()))))); 71 | } 72 | 73 | @Provides 74 | @Singleton 75 | Supplier inputStream() throws IOException { 76 | if (options.data() == null) { 77 | return () -> System.in; 78 | } 79 | return this::openIputFile; 80 | } 81 | 82 | private InputStream openIputFile() { 83 | try { 84 | return Files.newInputStream(Paths.get(options.data())); 85 | } catch (IOException e) { 86 | throw new RuntimeException(e); 87 | } 88 | } 89 | 90 | @Provides 91 | @Singleton 92 | Optional outputForModelPath() { 93 | return options.finalRegressor() == null ? Optional.empty() : Optional.of(Paths.get(options.finalRegressor())); 94 | } 95 | 96 | @Provides 97 | @Singleton 98 | @Named("disruptor") 99 | protected Disruptor> inputDisruptor(@Named("disruptor") Disruptor> disruptor) { 100 | return disruptor; 101 | } 102 | 103 | @Provides 104 | @Singleton 105 | @Named("disruptor") 106 | @SuppressWarnings("Unchecked") 107 | protected Disruptor> inputDisruptor(Provider>> workHandlerProvider, 108 | Provider>> evenHandlerProvider) { 109 | Disruptor> disruptor = new Disruptor<>( 110 | TwoPhaseEvent.factory(outputEventFactory()), 111 | options.ringSize(), threadsProvider, 112 | ProducerType.SINGLE, new SleepingWaitStrategy()); 113 | WorkHandler>[] parsers = new WorkHandler[options.threads()]; 114 | for (int i = 0; i < options.threads(); i++) { 115 | parsers[i] = workHandlerProvider.get(); 116 | } 117 | disruptor.handleExceptionsWith(new FatalExceptionHandler()); 118 | disruptor.handleEventsWithWorkerPool(parsers).then(evenHandlerProvider.get()); 119 | return disruptor; 120 | } 121 | 122 | protected abstract EventFactory outputEventFactory(); 123 | 124 | 125 | protected void configureCommonBeans() { 126 | ThrowingProviderBinder.forModule(this); 127 | bindConstant().annotatedWith(Names.named("testOnly")).to(options.testOnly()); 128 | bindConstant().annotatedWith(Names.named("skipFirst")).to(options.skipFirst()); 129 | try { 130 | bindInputFormat(); 131 | } catch (Exception e) { 132 | Throwables.propagate(e); 133 | } 134 | bind(FeaturesProcessor.class); 135 | bind(FtrlProximalRunner.class).asEagerSingleton(); 136 | bind(FinishCollectStatisticsListener.class).asEagerSingleton(); 137 | bind(Phaser.class).asEagerSingleton(); 138 | bind(FtrlOptions.class).toInstance(options); 139 | } 140 | 141 | private void bindInputFormat() throws ClassNotFoundException { 142 | if (options.customInputFormatClass() != null) { 143 | bind(InputFormat.class).to(Class.forName( 144 | options.customInputFormatClass()) 145 | .asSubclass(InputFormat.class)); 146 | } else { 147 | if (options.format() == InputFormatType.csv) { 148 | ColumnsMask columnsMask = new ColumnsMask(options.csvMask()); 149 | bindConstant().annotatedWith(Names.named("csvDelimiter")).to(options.csvDelimiter()); 150 | bind(new TypeLiteral() { 151 | }).annotatedWith(Names.named("csvMask")).toInstance(columnsMask); 152 | } 153 | bind(InputFormat.class).to(options.format().formatClass); 154 | } 155 | } 156 | } 157 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/FTRLProximalAlgorithm.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.google.inject.name.Named; 6 | import io.scaledml.core.SparseItem; 7 | import it.unimi.dsi.fastutil.doubles.DoubleArrayList; 8 | 9 | public class FTRLProximalAlgorithm { 10 | private FtrlProximalModel model; 11 | private boolean testOnly; 12 | private final DoubleArrayList currentN = new DoubleArrayList(); 13 | private final DoubleArrayList currentZ = new DoubleArrayList(); 14 | private final DoubleArrayList currentWeights = new DoubleArrayList(); 15 | 16 | public double learn(SparseItem item, Increment increment) { 17 | calculateWeights(item); 18 | double predict = predict(item); 19 | double gradient = item.label() - predict; 20 | if (!testOnly) { 21 | increment.clear(); 22 | for (int i = 0; i < item.indexes().size(); i++) { 23 | long index = item.indexes().getLong(i) % model.featuresNumber(); 24 | double nDelta = gradient * gradient; 25 | double n = currentN.getDouble(i); 26 | double w = currentWeights.getDouble(i); 27 | double learning_rate = 1. / model.alfa() * (Math.sqrt(n + nDelta) - Math.sqrt(n)); 28 | double zDelta = gradient - w * learning_rate; 29 | increment.addIncrement(index, nDelta, zDelta); 30 | } 31 | } 32 | return predict; 33 | } 34 | 35 | private double predict(SparseItem item) { 36 | double product = item.scalarMultiply(currentWeights); 37 | return 1. / (1. + Math.exp(product)); 38 | } 39 | 40 | private void calculateWeights(SparseItem item) { 41 | currentWeights.clear(); 42 | model.readVectors(item.indexes(), currentN, currentZ); 43 | for (int i = 0; i < item.indexes().size(); i++) { 44 | double z = currentZ.getDouble(i); 45 | double n = currentN.getDouble(i); 46 | if (Math.abs(z) > model.lambda1()) { 47 | currentWeights.add(-1. / ((model.beta() + Math.sqrt(n)) / model.alfa() + model.lambda2()) * (z - 48 | Math.signum(z) * model.lambda1())); 49 | } else { 50 | currentWeights.add(0.); 51 | } 52 | } 53 | } 54 | 55 | @Inject 56 | public FTRLProximalAlgorithm model(FtrlProximalModel model) { 57 | this.model = model; 58 | return this; 59 | } 60 | 61 | @Inject 62 | public FTRLProximalAlgorithm testOnly(@Named("testOnly") boolean testOnly) { 63 | this.testOnly = testOnly; 64 | return this; 65 | } 66 | } 67 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/FtrlProximalModel.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | import it.unimi.dsi.fastutil.doubles.DoubleList; 4 | import it.unimi.dsi.fastutil.floats.FloatBigArrayBigList; 5 | import it.unimi.dsi.fastutil.floats.FloatBigList; 6 | import it.unimi.dsi.fastutil.io.FastBufferedInputStream; 7 | import it.unimi.dsi.fastutil.io.FastBufferedOutputStream; 8 | import it.unimi.dsi.fastutil.longs.LongList; 9 | 10 | import java.io.IOException; 11 | import java.io.ObjectInputStream; 12 | import java.io.ObjectOutputStream; 13 | import java.io.Serializable; 14 | import java.nio.file.Files; 15 | import java.nio.file.Path; 16 | 17 | public class FtrlProximalModel implements Serializable { 18 | private FloatBigList n; 19 | private FloatBigList z; 20 | private double lambda1; 21 | private double lambda2; 22 | private double alfa; 23 | private double beta; 24 | 25 | public double lambda1() { 26 | return lambda1; 27 | } 28 | 29 | public FtrlProximalModel lambda1(double lambda1) { 30 | this.lambda1 = lambda1; 31 | return this; 32 | } 33 | 34 | public double lambda2() { 35 | return lambda2; 36 | } 37 | 38 | public FtrlProximalModel lambda2(double lambda2) { 39 | this.lambda2 = lambda2; 40 | return this; 41 | } 42 | 43 | public double alfa() { 44 | return alfa; 45 | } 46 | 47 | public FtrlProximalModel alfa(double alfa) { 48 | this.alfa = alfa; 49 | return this; 50 | } 51 | 52 | public double beta() { 53 | return beta; 54 | } 55 | 56 | public FtrlProximalModel beta(double beta) { 57 | this.beta = beta; 58 | return this; 59 | } 60 | 61 | public FtrlProximalModel featuresNumber(long featuresNumber) { 62 | n = new FloatBigArrayBigList(featuresNumber); 63 | z = new FloatBigArrayBigList(featuresNumber); 64 | n.size(featuresNumber); 65 | z.size(featuresNumber); 66 | return this; 67 | } 68 | 69 | public void readVectors(LongList indexes, DoubleList currentN, DoubleList currentZ) { 70 | currentN.clear(); 71 | currentZ.clear(); 72 | for (long ind : indexes) { 73 | long index = ind % featuresNumber(); 74 | currentN.add(n.get(index)); 75 | currentZ.add(z.get(index)); 76 | } 77 | } 78 | 79 | public static void saveModel(FtrlProximalModel model, Path output) throws IOException { 80 | try (ObjectOutputStream os = new ObjectOutputStream(new FastBufferedOutputStream(Files.newOutputStream(output)))) { 81 | os.writeObject(model); 82 | } 83 | } 84 | 85 | public static FtrlProximalModel loadModel(Path input) throws Exception { 86 | try (ObjectInputStream is = new ObjectInputStream(new FastBufferedInputStream(Files.newInputStream(input)))) { 87 | return (FtrlProximalModel) is.readObject(); 88 | } 89 | } 90 | 91 | public long featuresNumber() { 92 | return n.size64(); 93 | } 94 | 95 | public void writeToModel(Increment increment) { 96 | for (int i = 0; i < increment.indexes().size(); i++) { 97 | long index = increment.indexes().getLong(i); 98 | double nDelta = increment.incrementOfN().getDouble(i); 99 | double zDelta = increment.incrementOfZ().getDouble(i); 100 | n.set(index, (float) (n.get(index) + nDelta)); 101 | z.set(index, (float) (z.get(index) + zDelta)); 102 | } 103 | } 104 | } 105 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/FtrlProximalRunner.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.google.inject.name.Named; 6 | import com.lmax.disruptor.dsl.Disruptor; 7 | import io.scaledml.core.BaseDisruptorRunner; 8 | import io.scaledml.core.outputformats.OutputFormat; 9 | import io.scaledml.core.TwoPhaseEvent; 10 | 11 | import java.io.IOException; 12 | import java.nio.file.Path; 13 | import java.util.Optional; 14 | 15 | public class FtrlProximalRunner extends BaseDisruptorRunner { 16 | private FtrlProximalModel model; 17 | private Path outputForModelPath; 18 | private OutputFormat outputFormat; 19 | 20 | protected void afterDisruptorProcessed() throws IOException { 21 | outputFormat.close(); 22 | if (outputForModelPath != null) { 23 | FtrlProximalModel.saveModel(model, outputForModelPath); 24 | } 25 | } 26 | 27 | @Inject 28 | public FtrlProximalRunner disruptor(@Named("disruptor") Disruptor> disruptor) { 29 | setDisruptor(disruptor); 30 | return this; 31 | } 32 | 33 | @Inject 34 | public FtrlProximalRunner model(FtrlProximalModel model) { 35 | this.model = model; 36 | return this; 37 | } 38 | 39 | @Inject 40 | public FtrlProximalRunner outputFormat(@Named("delegate") OutputFormat outputFormat) { 41 | this.outputFormat = outputFormat; 42 | return this; 43 | } 44 | 45 | @Inject 46 | public FtrlProximalRunner outputForModelPath(Optional outputForModelPath) { 47 | return outputForModelPath(outputForModelPath.orElse(null)); 48 | } 49 | 50 | public FtrlProximalRunner outputForModelPath(Path outputForModelPath) { 51 | this.outputForModelPath = outputForModelPath; 52 | return this; 53 | } 54 | 55 | } 56 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/Increment.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | import it.unimi.dsi.fastutil.doubles.DoubleArrayList; 4 | import it.unimi.dsi.fastutil.doubles.DoubleList; 5 | import it.unimi.dsi.fastutil.longs.LongArrayList; 6 | import it.unimi.dsi.fastutil.longs.LongList; 7 | 8 | 9 | public class Increment { 10 | private final LongList indexes = new LongArrayList(); 11 | private final DoubleList incrementOfN = new DoubleArrayList(); 12 | private final DoubleList incrementOfZ = new DoubleArrayList(); 13 | 14 | public void addIncrement(long index, double nDelta, double zDelta) { 15 | indexes.add(index); 16 | incrementOfN.add(nDelta); 17 | incrementOfZ.add(zDelta); 18 | } 19 | 20 | public void clear() { 21 | indexes.clear(); 22 | incrementOfN.clear(); 23 | incrementOfZ.clear(); 24 | } 25 | 26 | public DoubleList incrementOfZ() { 27 | return incrementOfZ; 28 | } 29 | 30 | public DoubleList incrementOfN() { 31 | return incrementOfN; 32 | } 33 | 34 | public LongList indexes() { 35 | return indexes; 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/Main.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | import com.google.inject.Guice; 4 | import com.google.inject.Injector; 5 | import com.lexicalscope.jewel.cli.ArgumentValidationException; 6 | import com.lexicalscope.jewel.cli.CliFactory; 7 | import io.scaledml.features.FeatureEngineeringModule; 8 | import io.scaledml.features.FeatureEngineeringRunner; 9 | import io.scaledml.ftrl.options.FtrlOptions; 10 | import io.scaledml.ftrl.outputformats.FinishCollectStatisticsListener; 11 | import io.scaledml.ftrl.parallel.ParallelModule; 12 | import io.scaledml.ftrl.semiparallel.SemiParallelModule; 13 | 14 | import java.io.IOException; 15 | 16 | public class Main { 17 | 18 | public static void main(String... args) throws Exception { 19 | FtrlOptions ftrlOptions; 20 | try { 21 | ftrlOptions = CliFactory.parseArguments(FtrlOptions.class, args); 22 | } catch (ArgumentValidationException e) { 23 | System.out.println(e.getMessage()); 24 | return; 25 | } 26 | if (ftrlOptions.featureEngineering()) { 27 | runFeatureEngineering(ftrlOptions); 28 | } else { 29 | runFtrlProximal(ftrlOptions); 30 | } 31 | } 32 | 33 | public static void runFeatureEngineering(FtrlOptions ftrlOptions) throws IOException { 34 | Injector injector = Guice.createInjector(new FeatureEngineeringModule(ftrlOptions)); 35 | FeatureEngineeringRunner runner = injector.getInstance(FeatureEngineeringRunner.class); 36 | runner.process(); 37 | } 38 | 39 | public static double runFtrlProximal(FtrlOptions ftrlOptions) throws Exception { 40 | Injector injector = createInjector(ftrlOptions); 41 | FtrlProximalRunner runner = injector.getInstance(FtrlProximalRunner.class); 42 | runner.process(); 43 | return injector.getInstance(FinishCollectStatisticsListener.class).logLoss(); 44 | } 45 | 46 | private static Injector createInjector(FtrlOptions ftrlOptions) { 47 | if (ftrlOptions.parallelLearn()) { 48 | return Guice.createInjector(new ParallelModule(ftrlOptions)); 49 | } 50 | return Guice.createInjector(new SemiParallelModule(ftrlOptions)); 51 | } 52 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/featuresprocessors/FeaturesProcessor.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.featuresprocessors; 2 | 3 | import io.scaledml.core.SparseItem; 4 | import io.scaledml.core.util.LineBytesBuffer; 5 | import io.scaledml.core.util.Util; 6 | 7 | public class FeaturesProcessor { 8 | 9 | public void addCategoricalFeature(SparseItem item, LineBytesBuffer namespace, LineBytesBuffer feature) { 10 | item.addCategoricalIndex(index(namespace, feature)); 11 | } 12 | 13 | private long index(LineBytesBuffer namespace, LineBytesBuffer feature) { 14 | return Util.murmur().newHasher() 15 | .putBytes(namespace.bytes(), 0, namespace.size()) 16 | .putBytes(feature.bytes(), 0, feature.size()).hash().asLong(); 17 | } 18 | 19 | public void addNumericalFeature(SparseItem item, LineBytesBuffer namespace, LineBytesBuffer feature, double value) { 20 | item.addNumericalIndex(index(namespace, feature), value); 21 | } 22 | 23 | public void finalize(SparseItem item) { 24 | 25 | } 26 | } 27 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/options/FtrlOptions.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.options; 2 | 3 | import com.google.common.base.Joiner; 4 | import com.lexicalscope.jewel.cli.Option; 5 | import io.scaledml.core.inputformats.InputFormat; 6 | 7 | import java.util.Arrays; 8 | 9 | public interface FtrlOptions { 10 | @Option(longName = "feature_engineering", 11 | description = "Launch feature engineering program", 12 | hidden = true 13 | ) 14 | boolean featureEngineering(); 15 | @Option(shortName = "b", longName = "bit_precision", defaultValue = "18", 16 | maximum = 40, minimum = 1, 17 | description = "number of bits in the feature table") 18 | int hashcodeBits(); 19 | 20 | @Option(longName = "ftrl_alpha", defaultValue = "0.005", 21 | description = "ftrl alpha parameter (option in ftrl)") 22 | double alfa(); 23 | 24 | @Option(longName = "ftrl_beta", defaultValue = "0.1", 25 | description = "ftrl beta patameter (option in ftrl)") 26 | double beta(); 27 | 28 | @Option(longName = "l1", defaultValue = "0.0", 29 | description = "l_1 lambda (L1 regularization)") 30 | double l1(); 31 | 32 | @Option(longName = "l2", defaultValue = "0.0", 33 | description = "l_2 lambda (L2 regularization)") 34 | double l2(); 35 | 36 | @Option(shortName = "f", longName = "final_regressor", defaultToNull = true, 37 | description = "Final regressor to save (arg setInputStream filename)") 38 | String finalRegressor(); 39 | 40 | @Option(shortName = "i", longName = "initial_regressor", defaultToNull = true, 41 | description = "Initial regressor(s) to load into memory (arg setInputStream filename)") 42 | String initialRegressor(); 43 | 44 | @Option(shortName = "t", longName = "testonly", 45 | description = "Ignore label information and just test") 46 | boolean testOnly(); 47 | 48 | @Option(shortName = "d", longName = "data", defaultToNull = true, 49 | description = "Example Set") 50 | String data(); 51 | 52 | @Option(shortName = "p", longName = "predictions", defaultToNull = true, 53 | description = "File to output predictions to") 54 | String predictions(); 55 | 56 | @Option(shortName = "h", longName = "help", helpRequest = true, 57 | description = "Show this help") 58 | boolean help(); 59 | 60 | @Option(longName = "threads", defaultValue = "1", 61 | description = "Parallelization level") 62 | int threads(); 63 | 64 | @Option(longName = "parallel-learn", hidden = true, 65 | description = "Make algorithm learn parallelLearn may be in cost of some quality loss. " + 66 | "You should not use that property with threads < 8") 67 | boolean parallelLearn(); 68 | 69 | @Option(longName = "format", defaultValue = "vw", description = "Input file format." + 70 | "vw, csv, binary are currently supported") 71 | InputFormatType format(); 72 | 73 | @Option(longName = "custom-format-class", defaultToNull = true, description = "Input file format." + 74 | "vw, csv, binary are currently supported") 75 | String customInputFormatClass(); 76 | 77 | @Option(longName = "ring_size", defaultValue = "2048", hidden = true) 78 | int ringSize(); 79 | 80 | @Option(longName = "skip_first", description = "Skip first string of file(usually if it is a csv header", hidden = true) 81 | boolean skipFirst(); 82 | 83 | @Option(longName = "csv_mask", description = "Csv columns information. It could contain " + 84 | "(l)Label, (i)d, (n)umeric or (c)categorical marks with amount of columns on the same type" + 85 | " in brackets[]." + 86 | "Some valid examples are: 'ilcccccnnnnn', 'lc[10]n", 87 | defaultValue = "lc") 88 | String csvMask(); 89 | 90 | @Option(longName = "csv_delimiter", 91 | description ="csv columns delimiter. Must be only one character", 92 | defaultValue = ",") 93 | char csvDelimiter(); 94 | } 95 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/options/FtrlOptionsObject.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.options; 2 | 3 | 4 | import io.scaledml.core.inputformats.InputFormat; 5 | 6 | public class FtrlOptionsObject implements FtrlOptions { 7 | private int hashcodeBits = 18; 8 | private double alfa = 0.005; 9 | private double beta = 0.1; 10 | private double l1 = 0.; 11 | private double l2 = 0.; 12 | private String predictions; 13 | private String data; 14 | private String finalRegressor; 15 | private String initialRegressor; 16 | private boolean testOnly = false; 17 | private int threads = 1; 18 | private boolean parallelLearn = false; 19 | private boolean quadratic = false; 20 | private InputFormatType format = InputFormatType.vw; 21 | private boolean skipFirst = false; 22 | private String csvMask = "lc"; 23 | private char csvDelimiter = ','; 24 | private int ringSize = 2048; 25 | private boolean featureEngineering; 26 | private String customInputFormatClass; 27 | 28 | @Override 29 | public boolean featureEngineering() { 30 | return featureEngineering; 31 | } 32 | 33 | @Override 34 | public int hashcodeBits() { 35 | return hashcodeBits; 36 | } 37 | 38 | @Override 39 | public double alfa() { 40 | return alfa; 41 | } 42 | 43 | @Override 44 | public double beta() { 45 | return beta; 46 | } 47 | 48 | @Override 49 | public double l1() { 50 | return l1; 51 | } 52 | 53 | @Override 54 | public double l2() { 55 | return l2; 56 | } 57 | 58 | @Override 59 | public String finalRegressor() { 60 | return finalRegressor; 61 | } 62 | 63 | @Override 64 | public String initialRegressor() { 65 | return initialRegressor; 66 | } 67 | 68 | @Override 69 | public boolean testOnly() { 70 | return testOnly; 71 | } 72 | 73 | @Override 74 | public String data() { 75 | return data; 76 | } 77 | 78 | @Override 79 | public String predictions() { 80 | return predictions; 81 | } 82 | 83 | @Override 84 | public boolean help() { 85 | return false; 86 | } 87 | 88 | @Override 89 | public int threads() { 90 | return threads; 91 | } 92 | 93 | @Override 94 | public boolean parallelLearn() { 95 | return parallelLearn; 96 | } 97 | 98 | @Override 99 | public InputFormatType format() { 100 | return format; 101 | } 102 | 103 | @Override 104 | public String customInputFormatClass() { 105 | return customInputFormatClass; 106 | } 107 | 108 | 109 | public FtrlOptionsObject customInputFormatClass(String customInputFormatClass) { 110 | this.customInputFormatClass = customInputFormatClass; 111 | return this; 112 | } 113 | 114 | public FtrlOptionsObject hashcodeBits(int hashcodeBits) { 115 | this.hashcodeBits = hashcodeBits; 116 | return this; 117 | } 118 | 119 | public FtrlOptionsObject format(InputFormatType format) { 120 | this.format = format; 121 | return this; 122 | } 123 | 124 | public FtrlOptionsObject alfa(double alfa) { 125 | this.alfa = alfa; 126 | return this; 127 | } 128 | 129 | public FtrlOptionsObject beta(double beta) { 130 | this.beta = beta; 131 | return this; 132 | } 133 | 134 | public FtrlOptionsObject skipFirst(boolean skipFirst) { 135 | this.skipFirst = skipFirst; 136 | return this; 137 | } 138 | 139 | public FtrlOptionsObject l1(double l1) { 140 | this.l1 = l1; 141 | return this; 142 | } 143 | 144 | public FtrlOptionsObject l2(double l2) { 145 | this.l2 = l2; 146 | return this; 147 | } 148 | 149 | public FtrlOptionsObject finalRegressor(String finalRegressor) { 150 | this.finalRegressor = finalRegressor; 151 | return this; 152 | } 153 | 154 | public FtrlOptionsObject initialRegressor(String initialRegressor) { 155 | this.initialRegressor = initialRegressor; 156 | return this; 157 | } 158 | 159 | public FtrlOptionsObject testOnly(boolean testOnly) { 160 | this.testOnly = testOnly; 161 | return this; 162 | } 163 | 164 | public FtrlOptionsObject data(String data) { 165 | this.data = data; 166 | return this; 167 | } 168 | 169 | public FtrlOptionsObject predictions(String predictions) { 170 | this.predictions = predictions; 171 | return this; 172 | } 173 | 174 | public FtrlOptionsObject threads(int threads) { 175 | this.threads = threads; 176 | return this; 177 | } 178 | 179 | public FtrlOptionsObject scalable(boolean scalable) { 180 | this.parallelLearn = scalable; 181 | return this; 182 | } 183 | 184 | public FtrlOptionsObject csvMask(String csvMask) { 185 | this.csvMask = csvMask; 186 | return this; 187 | } 188 | 189 | public FtrlOptionsObject featureEngineering(boolean featureEngineering) { 190 | this.featureEngineering = featureEngineering; 191 | return this; 192 | } 193 | @Override 194 | public int ringSize() { 195 | return ringSize; 196 | } 197 | 198 | public FtrlOptionsObject quadratic(boolean quadratic) { 199 | this.quadratic = quadratic; 200 | return this; 201 | } 202 | 203 | @Override 204 | public boolean skipFirst() { 205 | return skipFirst; 206 | } 207 | 208 | @Override 209 | public String csvMask() { 210 | return csvMask; 211 | } 212 | 213 | 214 | @Override 215 | public char csvDelimiter() { 216 | return csvDelimiter; 217 | } 218 | 219 | public FtrlOptionsObject csvDelimiter(char csvDelimiter) { 220 | this.csvDelimiter = csvDelimiter; 221 | return this; 222 | } 223 | 224 | public FtrlOptionsObject ringSize(int ringSize) { 225 | this.ringSize = ringSize; 226 | return this; 227 | } 228 | } 229 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/options/InputFormatType.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.options; 2 | 3 | 4 | import io.scaledml.core.inputformats.BinaryInputFormat; 5 | import io.scaledml.core.inputformats.CSVFormat; 6 | import io.scaledml.core.inputformats.InputFormat; 7 | import io.scaledml.core.inputformats.VowpalWabbitFormat; 8 | 9 | public enum InputFormatType { 10 | vw(VowpalWabbitFormat.class), 11 | csv(CSVFormat.class), 12 | binary(BinaryInputFormat.class); 13 | public final Class formatClass; 14 | 15 | InputFormatType(Class formatClass) { 16 | this.formatClass = formatClass; 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/outputformats/FinishCollectStatisticsListener.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.outputformats; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.google.inject.name.Named; 6 | import org.slf4j.Logger; 7 | import org.slf4j.LoggerFactory; 8 | 9 | public class FinishCollectStatisticsListener { 10 | private static final Logger logger = LoggerFactory.getLogger(FinishCollectStatisticsListener.class); 11 | 12 | private int expectedFinishCollectEventsNum; 13 | private int finishCollectEvents = 0; 14 | 15 | private double totalLogLikelyhood = 0.; 16 | private long totalItems = 0; 17 | 18 | public FinishCollectStatisticsListener() { 19 | logger.info("mean_logloss\tsmooth_logloss\titems\tcurrent_label\tcurrent_prediction\tfeatures_number"); 20 | } 21 | 22 | public synchronized void finishedCollectingStatistics(StatisticsCalculator collector) { 23 | totalItems += collector.itemNo(); 24 | totalLogLikelyhood += collector.logLikelihood(); 25 | finishCollectEvents++; 26 | if (finishCollectEvents >= expectedFinishCollectEventsNum) { 27 | logger.info("Total mean logloss: " + logLoss() + " Total items: " + totalItems); 28 | } 29 | } 30 | 31 | public synchronized double logLoss() { 32 | return -totalLogLikelyhood / totalItems; 33 | } 34 | 35 | @Inject 36 | public FinishCollectStatisticsListener expectedFinishCollectEventsNum(@Named("statsCollectors") int expectedFinishCollectEventsNum) { 37 | this.expectedFinishCollectEventsNum = expectedFinishCollectEventsNum; 38 | return this; 39 | } 40 | } 41 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/outputformats/StatisticsCalculator.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.outputformats; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.google.inject.name.Named; 6 | import io.scaledml.core.outputformats.OutputFormat; 7 | import io.scaledml.core.SparseItem; 8 | import io.scaledml.core.util.Util; 9 | import org.slf4j.Logger; 10 | import org.slf4j.LoggerFactory; 11 | 12 | import java.io.IOException; 13 | import java.text.DecimalFormat; 14 | 15 | public class StatisticsCalculator implements OutputFormat { 16 | private static final Logger logger = LoggerFactory.getLogger(StatisticsCalculator.class); 17 | private static final DecimalFormat df = new DecimalFormat("0.0000"); 18 | private OutputFormat delegate; 19 | private FinishCollectStatisticsListener finishListener; 20 | private double logLikelihood = 0.; 21 | private double smoothLogLikelihood = 0.; 22 | private double alfa = 1. / 10000.; 23 | private long itemNo = 0; 24 | private long nextItemNoToPrint = 1; 25 | 26 | private static String f(double v){ 27 | return df.format(v); 28 | } 29 | 30 | @Override 31 | public void emit(SparseItem item, double prediction) { 32 | delegate.emit(item, prediction); 33 | itemNo++; 34 | double itemLogLikelihood = Math.log(Util.doublesEqual(1., item.label()) ? prediction : 1 - prediction); 35 | logLikelihood += itemLogLikelihood; 36 | smoothLogLikelihood = smoothLogLikelihood * (1. - alfa) + itemLogLikelihood * alfa; 37 | if (itemNo == nextItemNoToPrint) { 38 | nextItemNoToPrint *= 2; 39 | double meanLogLoss = -logLikelihood / itemNo; 40 | logger.info(f(meanLogLoss) + "\t" + f(-smoothLogLikelihood) + "\t" + 41 | itemNo + "\t" + f(item.label()) + "\t" + f(prediction) + "\t" + item.indexes().size()); 42 | } 43 | } 44 | 45 | @Override 46 | public void close() throws IOException { 47 | finishListener.finishedCollectingStatistics(this); 48 | 49 | // delegate can be shared among threads so it must be closed separately 50 | } 51 | 52 | public double logLikelihood() { 53 | return logLikelihood; 54 | } 55 | 56 | public long itemNo() { 57 | return itemNo; 58 | } 59 | 60 | @Inject 61 | public StatisticsCalculator delegate(@Named("delegate") OutputFormat delegate) { 62 | this.delegate = delegate; 63 | return this; 64 | } 65 | 66 | @Inject 67 | public StatisticsCalculator finishListener(FinishCollectStatisticsListener finishListener) { 68 | this.finishListener = finishListener; 69 | return this; 70 | } 71 | } 72 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/parallel/LearnWorkHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.parallel; 2 | 3 | import com.google.inject.Inject; 4 | import com.lmax.disruptor.LifecycleAware; 5 | import com.lmax.disruptor.WorkHandler; 6 | import io.scaledml.ftrl.FTRLProximalAlgorithm; 7 | import io.scaledml.ftrl.Increment; 8 | import io.scaledml.core.SparseItem; 9 | import io.scaledml.core.TwoPhaseEvent; 10 | import io.scaledml.core.inputformats.InputFormat; 11 | import io.scaledml.core.outputformats.OutputFormat; 12 | import org.slf4j.Logger; 13 | import org.slf4j.LoggerFactory; 14 | 15 | import java.io.IOException; 16 | import java.util.concurrent.Phaser; 17 | 18 | public class LearnWorkHandler implements WorkHandler>, LifecycleAware { 19 | private static final Logger logger = LoggerFactory.getLogger(LearnWorkHandler.class); 20 | private InputFormat inputFormat; 21 | private FTRLProximalAlgorithm algorithm; 22 | private OutputFormat outputFormat; 23 | private final SparseItem item = new SparseItem(); 24 | private Phaser phaser; 25 | 26 | @Override 27 | public void onEvent(TwoPhaseEvent event) throws Exception { 28 | item.clear(); 29 | inputFormat.parse(event.input(), item, event.lineNo()); 30 | outputFormat.emit(item, algorithm.learn(item, event.output())); 31 | } 32 | 33 | @Override 34 | public void onStart() { 35 | phaser.register(); 36 | } 37 | 38 | @Override 39 | public void onShutdown() { 40 | try { 41 | outputFormat.close(); 42 | } catch (IOException e) { 43 | logger.error("failed to close", e); 44 | } finally { 45 | phaser.arrive(); 46 | } 47 | } 48 | 49 | @Inject 50 | public LearnWorkHandler inputFormat(InputFormat inputFormat) { 51 | this.inputFormat = inputFormat; 52 | return this; 53 | } 54 | 55 | @Inject 56 | public LearnWorkHandler algorithm(FTRLProximalAlgorithm algorithm) { 57 | this.algorithm = algorithm; 58 | return this; 59 | } 60 | 61 | @Inject 62 | public LearnWorkHandler outputFormat(OutputFormat outputFormat) { 63 | this.outputFormat = outputFormat; 64 | return this; 65 | } 66 | 67 | @Inject 68 | public LearnWorkHandler phaser(Phaser phaser) { 69 | this.phaser = phaser; 70 | return this; 71 | } 72 | } 73 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/parallel/ParallelModule.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.parallel; 2 | 3 | import com.google.inject.TypeLiteral; 4 | import com.google.inject.name.Names; 5 | import com.lmax.disruptor.EventFactory; 6 | import com.lmax.disruptor.EventHandler; 7 | import com.lmax.disruptor.WorkHandler; 8 | import io.scaledml.ftrl.Increment; 9 | import io.scaledml.ftrl.AbstractParallelModule; 10 | import io.scaledml.core.TwoPhaseEvent; 11 | import io.scaledml.ftrl.options.FtrlOptions; 12 | import io.scaledml.core.outputformats.OutputFormat; 13 | import io.scaledml.ftrl.outputformats.StatisticsCalculator; 14 | 15 | 16 | public class ParallelModule extends AbstractParallelModule { 17 | 18 | public ParallelModule(FtrlOptions options) { 19 | super(options); 20 | } 21 | 22 | @Override 23 | protected void configure() { 24 | configureCommonBeans(); 25 | bindConstant().annotatedWith(Names.named("statsCollectors")).to(options.threads()); 26 | bind(new TypeLiteral>>() { 27 | }).to(WriteUpdatesEventHandler.class).asEagerSingleton(); 28 | bind(new TypeLiteral>>() { 29 | }).to(LearnWorkHandler.class); 30 | bind(OutputFormat.class).to(StatisticsCalculator.class); 31 | } 32 | 33 | @Override 34 | protected EventFactory outputEventFactory() { 35 | return Increment::new; 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/parallel/WriteUpdatesEventHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.parallel; 2 | 3 | 4 | import com.google.inject.Inject; 5 | import com.lmax.disruptor.EventHandler; 6 | import io.scaledml.ftrl.FtrlProximalModel; 7 | import io.scaledml.ftrl.Increment; 8 | import io.scaledml.core.TwoPhaseEvent; 9 | 10 | public class WriteUpdatesEventHandler implements EventHandler> { 11 | private FtrlProximalModel model; 12 | 13 | @Override 14 | public void onEvent(TwoPhaseEvent event, long sequence, boolean endOfBatch) throws Exception { 15 | model.writeToModel(event.output()); 16 | } 17 | 18 | @Inject 19 | public WriteUpdatesEventHandler model(FtrlProximalModel model) { 20 | this.model = model; 21 | return this; 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/semiparallel/LearnEventHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.semiparallel; 2 | 3 | import com.google.inject.Inject; 4 | import com.lmax.disruptor.EventHandler; 5 | import com.lmax.disruptor.LifecycleAware; 6 | import io.scaledml.ftrl.FTRLProximalAlgorithm; 7 | import io.scaledml.ftrl.FtrlProximalModel; 8 | import io.scaledml.ftrl.Increment; 9 | import io.scaledml.core.SparseItem; 10 | import io.scaledml.core.TwoPhaseEvent; 11 | import io.scaledml.core.outputformats.OutputFormat; 12 | import org.slf4j.Logger; 13 | import org.slf4j.LoggerFactory; 14 | 15 | import java.io.IOException; 16 | import java.util.concurrent.Phaser; 17 | 18 | public class LearnEventHandler implements EventHandler>, LifecycleAware { 19 | private static final Logger logger = LoggerFactory.getLogger(LearnEventHandler.class); 20 | private OutputFormat outputFormat; 21 | private FTRLProximalAlgorithm algorithm; 22 | private FtrlProximalModel model; 23 | private Increment increment = new Increment(); 24 | private Phaser phaser; 25 | 26 | @Override 27 | public void onEvent(TwoPhaseEvent event, long sequence, boolean endOfBatch) throws Exception { 28 | double prediction = algorithm.learn(event.output(), increment); 29 | model.writeToModel(increment); 30 | outputFormat.emit(event.output(), prediction); 31 | } 32 | 33 | @Override 34 | public void onStart() { 35 | phaser.register(); 36 | } 37 | 38 | @Override 39 | public void onShutdown() { 40 | try { 41 | outputFormat.close(); 42 | } catch (IOException e) { 43 | logger.error("Failed to close", e); 44 | } finally { 45 | phaser.arriveAndDeregister(); 46 | } 47 | } 48 | 49 | @Inject 50 | public LearnEventHandler outputFormat(OutputFormat format) { 51 | this.outputFormat = format; 52 | return this; 53 | } 54 | 55 | @Inject 56 | public LearnEventHandler algorithm(FTRLProximalAlgorithm algorithm) { 57 | this.algorithm = algorithm; 58 | return this; 59 | } 60 | 61 | @Inject 62 | public LearnEventHandler model(FtrlProximalModel model) { 63 | this.model = model; 64 | return this; 65 | } 66 | 67 | @Inject 68 | public LearnEventHandler phaser(Phaser phaser) { 69 | this.phaser = phaser; 70 | return this; 71 | } 72 | } 73 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/semiparallel/ParseInputWorkHandler.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.semiparallel; 2 | 3 | import com.google.inject.Inject; 4 | import com.lmax.disruptor.WorkHandler; 5 | import io.scaledml.core.SparseItem; 6 | import io.scaledml.core.TwoPhaseEvent; 7 | import io.scaledml.core.inputformats.InputFormat; 8 | 9 | 10 | public class ParseInputWorkHandler implements WorkHandler> { 11 | InputFormat inputFormat; 12 | 13 | @Override 14 | public void onEvent(TwoPhaseEvent event) throws Exception { 15 | inputFormat.parse(event.input(), event.output(), event.lineNo()); 16 | } 17 | 18 | @Inject 19 | public ParseInputWorkHandler inputFormat(InputFormat format) { 20 | this.inputFormat = format; 21 | return this; 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/java/io/scaledml/ftrl/semiparallel/SemiParallelModule.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.semiparallel; 2 | 3 | 4 | import com.google.inject.TypeLiteral; 5 | import com.google.inject.name.Names; 6 | import com.lmax.disruptor.EventFactory; 7 | import com.lmax.disruptor.EventHandler; 8 | import com.lmax.disruptor.WorkHandler; 9 | import io.scaledml.core.SparseItem; 10 | import io.scaledml.ftrl.AbstractParallelModule; 11 | import io.scaledml.core.TwoPhaseEvent; 12 | import io.scaledml.ftrl.options.FtrlOptions; 13 | import io.scaledml.core.outputformats.OutputFormat; 14 | import io.scaledml.ftrl.outputformats.StatisticsCalculator; 15 | 16 | public class SemiParallelModule extends AbstractParallelModule { 17 | 18 | public SemiParallelModule(FtrlOptions options) { 19 | super(options); 20 | } 21 | 22 | @Override 23 | protected void configure() { 24 | configureCommonBeans(); 25 | bindConstant().annotatedWith(Names.named("statsCollectors")).to(1); 26 | bind(new TypeLiteral>>() { 27 | }).to(LearnEventHandler.class).asEagerSingleton(); 28 | bind(new TypeLiteral>>() { 29 | }).to(ParseInputWorkHandler.class); 30 | bind(OutputFormat.class).to(StatisticsCalculator.class).asEagerSingleton(); 31 | } 32 | 33 | @Override 34 | protected EventFactory outputEventFactory() { 35 | return SparseItem::new; 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/main/resources/simplelogger.properties: -------------------------------------------------------------------------------- 1 | org.slf4j.simpleLogger.showDateTime=true 2 | org.slf4j.simpleLogger.showLogName=false 3 | org.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss.SSS 4 | org.slf4j.simpleLogger.levelInBrackets=true -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/integration/BaseIntegrationTest.java: -------------------------------------------------------------------------------- 1 | package integration; 2 | 3 | import org.apache.commons.io.FileUtils; 4 | import org.junit.After; 5 | import org.junit.Before; 6 | 7 | import java.io.IOException; 8 | import java.io.RandomAccessFile; 9 | import java.nio.file.Files; 10 | import java.nio.file.Path; 11 | 12 | 13 | public class BaseIntegrationTest { 14 | protected Path tempDirectory; 15 | 16 | @Before 17 | public void setup() throws IOException { 18 | tempDirectory = Files.createTempDirectory("csv-ftrl-test-" + getClass().getSimpleName()); 19 | } 20 | 21 | protected void syncFS() throws Exception { 22 | try (RandomAccessFile rws = new RandomAccessFile(tempDirectory + "/sync", "rws")) { 23 | Thread.sleep(500); 24 | rws.getFD().sync(); 25 | } 26 | } 27 | 28 | @After 29 | public void teardown() throws IOException { 30 | FileUtils.deleteDirectory(tempDirectory.toFile()); 31 | } 32 | 33 | protected String resourcePath(String name) { 34 | return getClass().getResource(name).getPath(); 35 | } 36 | } 37 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/integration/FTRLProximalAlgorithmITest.java: -------------------------------------------------------------------------------- 1 | package integration; 2 | 3 | 4 | import io.scaledml.core.util.Util; 5 | import io.scaledml.ftrl.Main; 6 | import io.scaledml.ftrl.options.FtrlOptionsObject; 7 | import io.scaledml.ftrl.options.InputFormatType; 8 | import org.apache.commons.io.FileUtils; 9 | import org.junit.After; 10 | import org.junit.Before; 11 | import org.junit.Test; 12 | 13 | import java.io.IOException; 14 | import java.nio.file.Files; 15 | import java.nio.file.Path; 16 | import java.nio.file.Paths; 17 | import java.util.Arrays; 18 | 19 | import static org.junit.Assert.assertEquals; 20 | import static org.junit.Assert.assertTrue; 21 | 22 | public class FTRLProximalAlgorithmITest extends BaseIntegrationTest { 23 | 24 | @Test 25 | public void testRunWvFtrlProximal() throws Exception { 26 | Main.runFtrlProximal(new FtrlOptionsObject() 27 | .finalRegressor(tempDirectory + "/model") 28 | .threads(3) 29 | .data(resourcePath("/train-small.vw"))); 30 | syncFS(); 31 | double logLoss = Main.runFtrlProximal(new FtrlOptionsObject() 32 | .initialRegressor(tempDirectory + "/model") 33 | .testOnly(true) 34 | .predictions(tempDirectory + "/predictions") 35 | .data(resourcePath("/test-small.vw"))); 36 | syncFS(); 37 | assertEquals(0.47427705769071893, logLoss, Util.EPSILON); 38 | double[] predictions = Files.readAllLines(Paths.get(tempDirectory.toString(), "predictions")) 39 | .stream().map(s -> s.split("\t")[1]).mapToDouble(Double::parseDouble).toArray(); 40 | int predictionsNum = predictions.length; 41 | assertEquals(predictionsNum, 100); 42 | 43 | assertTrue(Arrays.stream(predictions).allMatch(p -> p < 0.5)); 44 | assertEquals(0.2355821069092084, predictions[0], 0.001); 45 | assertEquals(0.2495902538274775, predictions[63], 0.001); 46 | } 47 | 48 | @Test 49 | public void testRunParallelFtrlProximal() throws Exception { 50 | Main.runFtrlProximal(new FtrlOptionsObject() 51 | .finalRegressor(tempDirectory + "/model") 52 | .threads(3) 53 | .scalable(true) 54 | .data(resourcePath("/train-small.vw"))); 55 | syncFS(); 56 | double logLoss = Main.runFtrlProximal(new FtrlOptionsObject() 57 | .initialRegressor(tempDirectory + "/model") 58 | .testOnly(true) 59 | .threads(3) 60 | .scalable(true) 61 | .predictions(tempDirectory + "/predictions") 62 | .data(resourcePath("/test-small.vw"))); 63 | syncFS(); 64 | assertEquals(0.4716154011659849, logLoss, 0.01); 65 | double[] predictions = Files.readAllLines(Paths.get(tempDirectory.toString(), "predictions")) 66 | .stream().map(s -> s.split("\t")[1]).mapToDouble(Double::parseDouble).toArray(); 67 | int predictionsNum = predictions.length; 68 | assertEquals(predictionsNum, 100); 69 | assertTrue(Arrays.stream(predictions).allMatch(p -> p < 0.5)); 70 | } 71 | 72 | @Test 73 | public void testRunCsvFtrlProximal() throws Exception { 74 | Main.runFtrlProximal(new FtrlOptionsObject() 75 | .finalRegressor(tempDirectory + "/model") 76 | .threads(3) 77 | .data(resourcePath("/ruslan-train-small.csv")) 78 | .format(InputFormatType.csv) 79 | .skipFirst(true)); 80 | syncFS(); 81 | double logLoss = Main.runFtrlProximal(new FtrlOptionsObject() 82 | .initialRegressor(tempDirectory + "/model") 83 | .testOnly(true) 84 | .predictions(tempDirectory + "/predictions") 85 | .data(resourcePath("/ruslan-test-small.csv")) 86 | .format(InputFormatType.csv) 87 | .skipFirst(true)); 88 | syncFS(); 89 | assertEquals(0.5183230180345785, logLoss, Util.EPSILON); 90 | 91 | double[] predictions = Files.readAllLines(Paths.get(tempDirectory.toString(), "predictions")) 92 | .stream().map(s -> s.split("\t")[1]).mapToDouble(Double::parseDouble).toArray(); 93 | int predictionsNum = predictions.length; 94 | assertEquals(predictionsNum, 100); 95 | } 96 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/integration/FeatureEngineeringITest.java: -------------------------------------------------------------------------------- 1 | package integration; 2 | 3 | import io.scaledml.core.util.Util; 4 | import io.scaledml.ftrl.Main; 5 | import io.scaledml.ftrl.options.FtrlOptionsObject; 6 | import io.scaledml.ftrl.options.InputFormatType; 7 | import org.junit.Ignore; 8 | import org.junit.Test; 9 | import static org.junit.Assert.*; 10 | 11 | import java.io.IOException; 12 | import java.nio.file.Files; 13 | import java.nio.file.Paths; 14 | 15 | @Ignore 16 | public class FeatureEngineeringITest extends BaseIntegrationTest { 17 | 18 | @Test 19 | public void testFeatureEngineering() throws Exception { 20 | Main.runFeatureEngineering( 21 | new FtrlOptionsObject() 22 | .data(resourcePath("/ruslan-train-small.csv")) 23 | .format(InputFormatType.csv) 24 | .skipFirst(true) 25 | .csvMask("lc[37]n") 26 | .predictions(tempDirectory + "/input")); 27 | syncFS(); 28 | double logLoss = Main.runFtrlProximal(new FtrlOptionsObject() 29 | .threads(3) 30 | .data(tempDirectory + "/input") 31 | .format(InputFormatType.binary) 32 | .predictions(tempDirectory + "/output")); 33 | syncFS(); 34 | assertEquals(0.5125, logLoss, 0.01); 35 | assertEquals(1000, Files.readAllLines(Paths.get(tempDirectory + "/output")).size()); 36 | } 37 | } 38 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/core/SparseItemTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core; 2 | 3 | import io.scaledml.core.inputformats.InputFormat; 4 | import io.scaledml.core.inputformats.VowpalWabbitFormat; 5 | import io.scaledml.core.util.LineBytesBuffer; 6 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 7 | import org.junit.Test; 8 | 9 | import static org.junit.Assert.*; 10 | 11 | public class SparseItemTest { 12 | 13 | @Test 14 | public void testWriteRead() throws Exception { 15 | SparseItem item = createSparseItem("-1 |C1 1005 |banner_pos 1 |site_id 0a742914 |site_domain 510bd839 |site_category f028772b " + 16 | "|app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 0cff710f " + 17 | "|device_model 76dc4769 |device_type 1 |device_conn_type 0 |C14 8330 |C15 320 |C16 50 |C17 761 |C18 3 " + 18 | "|C19 175 |C20 100075"); 19 | assertEquals(20, item.indexes().stream().distinct().count()); 20 | testWriteRead(item); 21 | } 22 | 23 | @Test 24 | public void testWriteRead2() throws Exception { 25 | String line1 = "1 |сat1 feature1:2 |сat2 feature2:100.55 фича3:-123.4 | cat3 feature4 feature5:-17"; 26 | SparseItem item = createSparseItem(line1); 27 | assertEquals(5, item.indexes().stream().distinct().count()); 28 | testWriteRead(item); 29 | } 30 | 31 | private void testWriteRead(SparseItem item) { 32 | LineBytesBuffer bb = new LineBytesBuffer(); 33 | item.write(bb); 34 | SparseItem other = new SparseItem(); 35 | other.read(bb); 36 | assertEquals(item, other); 37 | } 38 | 39 | private SparseItem createSparseItem(String line1) { 40 | InputFormat format = new VowpalWabbitFormat() 41 | .featruresProcessor(new FeaturesProcessor()); 42 | LineBytesBuffer line = new LineBytesBuffer(line1); 43 | SparseItem item = new SparseItem(); 44 | format.parse(line, item, 0); 45 | return item; 46 | } 47 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/core/inputformats/CSVFormatTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import io.scaledml.core.SparseItem; 4 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 5 | import io.scaledml.core.util.LineBytesBuffer; 6 | import org.junit.Test; 7 | 8 | import static org.junit.Assert.assertEquals; 9 | import static org.junit.Assert.assertNotNull; 10 | 11 | /** 12 | * @author Ilya Smagin ilya-sm@yandex-team.ru on 4/5/15. 13 | */ 14 | public class CSVFormatTest { 15 | @Test 16 | public void testParse() throws Exception { 17 | String line1 = "0,e6c5b5cd,68fd1e64,0b153874,c92f3b61,8e407662,2b53e5fb,1f6f0bb6,21ddcdc9,4f94c62a,7d1526c6," + 18 | "606b0dda,f0cf0024,b04e4670,454bf5f0,60f6221e,fe6b92e5,3a171ecb,25c83c98,07c540c4,41274cd7,623049e6," + 19 | "5840adea,922afcc0,731c3655,2f532987,0850bcd9,f9b4759b,d7020589,6f67f7e5,4f1b46f3,b28479f6,e8b83407," + 20 | "d7497e30,5fca948f,43f13e8b,,a73ee510,0.5,1.0,4,1,4.0,2,2,44.0,1,204.0,2,,0,110.0,3.0,4,8,440.0,46.0," + 21 | "44.0,1,44,102"; 22 | 23 | InputFormat format = new CSVFormat() 24 | .csvMask(new ColumnsMask("lc[37]n")) 25 | .csvDelimiter(',') 26 | .featruresProcessor(new FeaturesProcessor()); 27 | LineBytesBuffer line = new LineBytesBuffer(line1); 28 | SparseItem item = new SparseItem(); 29 | format.parse(line, item, 0); 30 | assertEquals(0., item.label(), 0.000001); 31 | assertEquals(58, item.indexes().stream().distinct().count()); 32 | } 33 | } 34 | -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/core/inputformats/ColumnsMaskTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import io.scaledml.core.inputformats.ColumnsMask.ColumnType; 4 | import org.junit.Test; 5 | 6 | import static org.junit.Assert.*; 7 | 8 | public class ColumnsMaskTest { 9 | 10 | @Test 11 | public void testParse1() { 12 | ColumnsMask mask = new ColumnsMask("lc[37]n"); 13 | assertEquals(ColumnType.LABEL, mask.getCategory(0)); 14 | assertEquals(ColumnType.CATEGORICAL, mask.getCategory(30)); 15 | assertEquals(ColumnType.NUMERICAL, mask.getCategory(47)); 16 | assertEquals(ColumnType.NUMERICAL, mask.getCategory(1024)); 17 | } 18 | 19 | @Test 20 | public void testParse2() { 21 | ColumnsMask mask = new ColumnsMask("lc"); 22 | assertEquals(ColumnType.LABEL, mask.getCategory(0)); 23 | assertEquals(ColumnType.CATEGORICAL, mask.getCategory(30)); 24 | assertEquals(ColumnType.CATEGORICAL, mask.getCategory(47)); 25 | assertEquals(ColumnType.CATEGORICAL, mask.getCategory(1024)); 26 | } 27 | 28 | @Test 29 | public void testParse3() { 30 | ColumnsMask mask = new ColumnsMask("ilcccccnnnnn"); 31 | assertEquals(ColumnType.ID, mask.getCategory(0)); 32 | assertEquals(ColumnType.LABEL, mask.getCategory(1)); 33 | assertEquals(ColumnType.CATEGORICAL, mask.getCategory(4)); 34 | assertEquals(ColumnType.NUMERICAL, mask.getCategory(1024)); 35 | } 36 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/core/inputformats/VowpalWabbitFormatTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.inputformats; 2 | 3 | import io.scaledml.core.SparseItem; 4 | import io.scaledml.ftrl.featuresprocessors.FeaturesProcessor; 5 | import io.scaledml.core.util.LineBytesBuffer; 6 | import org.junit.Test; 7 | 8 | import static org.junit.Assert.assertEquals; 9 | import static org.junit.Assert.assertNotNull; 10 | 11 | public class VowpalWabbitFormatTest { 12 | 13 | @Test 14 | public void testParse() throws Exception { 15 | String line1 = "-1 |C1 1005 |banner_pos 1 |site_id 0a742914 |site_domain 510bd839 |site_category f028772b " + 16 | "|app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 0cff710f " + 17 | "|device_model 76dc4769 |device_type 1 |device_conn_type 0 |C14 8330 |C15 320 |C16 50 |C17 761 |C18 3 " + 18 | "|C19 175 |C20 100075"; 19 | InputFormat format = new VowpalWabbitFormat() 20 | .featruresProcessor(new FeaturesProcessor()); 21 | LineBytesBuffer line = new LineBytesBuffer(line1); 22 | SparseItem item = new SparseItem(); 23 | format.parse(line, item, 0); 24 | assertEquals(0., item.label(), 0.000001); 25 | assertEquals(20, item.indexes().stream().distinct().count()); 26 | } 27 | 28 | @Test 29 | public void testParseUtf8() throws Exception { 30 | String line1 = "1 |КАТ1 ФИЧА1 |кат2 фича2 фича3 |запрос у попа была собака он ее любил "; 31 | InputFormat format = new VowpalWabbitFormat() 32 | .featruresProcessor(new FeaturesProcessor()); 33 | LineBytesBuffer line = new LineBytesBuffer(line1); 34 | SparseItem item = new SparseItem(); 35 | format.parse(line, item, 0); 36 | assertEquals(1., item.label(), 0.000001); 37 | assertEquals(10, item.indexes().stream().distinct().count()); 38 | } 39 | 40 | @Test 41 | public void testParseNumerical() throws Exception { 42 | String line1 = "1 |сat1 feature1:2 |сat2 feature2:100.55 фича3:-123.4 | cat3 feature4 feature5:-17"; 43 | InputFormat format = new VowpalWabbitFormat() 44 | .featruresProcessor(new FeaturesProcessor()); 45 | LineBytesBuffer line = new LineBytesBuffer(line1); 46 | SparseItem item = new SparseItem(); 47 | format.parse(line, item, 0); 48 | assertEquals(1., item.label(), 0.000001); 49 | assertEquals(5, item.indexes().stream().distinct().count()); 50 | assertEquals(1., item.getValue(0), 0.000001); 51 | assertEquals(2., item.getValue(1), 0.000001); 52 | assertEquals(100.55, item.getValue(2), 0.000001); 53 | assertEquals(-123.4, item.getValue(3), 0.000001); 54 | assertEquals(-17., item.getValue(4), 0.000001); 55 | 56 | } 57 | 58 | @Test 59 | public void testBigNamespace() { 60 | String line1 = "-1 |cat CAT01=8ba8b39a CAT02=68fd1e64 CAT03=1f89b562 CAT04=891b62e7 CAT05=e7e2fcab " + 61 | "CAT06=a8cd5504 CAT07=5b56befb CAT08=21ddcdc9 CAT09=fc055e07 CAT10=7d1526c6 CAT11=606b0dda " + 62 | "CAT12=80e26c9b CAT13=f54016b9 CAT14=8d51ec69 CAT15=07b5194c CA16=7e0ccccf CAT17=3a171ecb " + 63 | "CAT18=25c83c98 CAT19=e5ba7672 CAT20=7b4723c4 CAT21=37c9c164 CAT22=b1252a9d CAT23=de7995b8 " + 64 | "CAT24=9727dd16 CAT25=8d51ec69 CAT26=581e8232 CAT27=4918af02 CAT28=2824a5f6 CAT29=fb936136 " + 65 | "CAT30=b2cb9c98 CAT31=1adce6ef CAT32=cc651ac8 AT33=8a544033 CAT34=860584cc CAT35=fd323779 " + 66 | "CAT37=a73ee510 |num NUM01:0.0 NUM02:1.0 NUM03:2 NUM04:2 NUM05:16.0 NUM06:15 NUM07:2 NUM08:6.0 " + 67 | "NUM09:1 NUM10:1382.0 NUM11:1 NUM13:1 NUM14:1386.0 NUM15:1.0 NUM16:181 NUM17:4 NUM18:6.0 NUM19:7.0 " + 68 | "NUM20:0.0 NUM21:0 NUM22:5 NUM23:1382"; 69 | InputFormat format = new VowpalWabbitFormat() 70 | .featruresProcessor(new FeaturesProcessor()); 71 | LineBytesBuffer line = new LineBytesBuffer(line1); 72 | SparseItem item = new SparseItem(); 73 | format.parse(line, item, 0); 74 | assertEquals(0., item.label(), 0.000001); 75 | assertEquals(58, item.indexes().stream().distinct().count()); 76 | } 77 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/core/util/LineBytesBufferTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.core.util; 2 | 3 | 4 | import com.google.common.base.Charsets; 5 | import it.unimi.dsi.fastutil.io.FastBufferedInputStream; 6 | import org.apache.commons.io.FileUtils; 7 | import org.junit.After; 8 | import org.junit.Before; 9 | import org.junit.Test; 10 | 11 | import java.io.BufferedWriter; 12 | import java.io.IOException; 13 | import java.nio.file.Files; 14 | import java.nio.file.Path; 15 | import java.nio.file.Paths; 16 | import java.util.concurrent.atomic.AtomicInteger; 17 | 18 | import static org.junit.Assert.assertEquals; 19 | import static org.junit.Assert.assertTrue; 20 | 21 | public class LineBytesBufferTest { 22 | private Path tempDirectory; 23 | 24 | @Before 25 | public void setup() throws IOException { 26 | tempDirectory = Files.createTempDirectory("ftrl-test"); 27 | } 28 | 29 | @Test 30 | public void testReadLine() throws Exception { 31 | Path fileExpected = Paths.get(tempDirectory.toString(), "expected"); 32 | try (BufferedWriter writer = Files.newBufferedWriter(fileExpected, Charsets.US_ASCII)) { 33 | StringBuilder sb = new StringBuilder(); 34 | for (int i = 0; i < 2000; i++) { 35 | for (int j = 0; j < i; j++) { 36 | sb.append(j); 37 | sb.append(' '); 38 | } 39 | sb.append('\n'); 40 | } 41 | for (int i = 2000; i > 0; i--) { 42 | for (int j = 0; j < i; j++) { 43 | sb.append(j); 44 | sb.append(' '); 45 | } 46 | sb.append('\n'); 47 | writer.write(sb.toString()); 48 | sb.setLength(0); 49 | } 50 | } 51 | Path fileActual = Paths.get(tempDirectory.toString(), "actual"); 52 | try (FastBufferedInputStream stream = new FastBufferedInputStream(Files.newInputStream(fileExpected))) { 53 | try (BufferedWriter writer = Files.newBufferedWriter(fileActual, Charsets.US_ASCII)) { 54 | LineBytesBuffer buffer = new LineBytesBuffer(); 55 | LineBytesBuffer line = new LineBytesBuffer(); 56 | while (buffer.readLineFrom(stream)) { 57 | buffer.drainTo(line); 58 | writer.write(line.toAsciiString()); 59 | writer.newLine(); 60 | } 61 | } 62 | } 63 | assertTrue(Files.readAllLines(fileExpected, Charsets.US_ASCII).equals(Files.readAllLines(fileActual, Charsets.US_ASCII))); 64 | } 65 | 66 | @Test 67 | public void testCompare() throws Exception { 68 | LineBytesBuffer b1 = new LineBytesBuffer(); 69 | LineBytesBuffer b2 = new LineBytesBuffer(); 70 | assertTrue(b1.compareTo(b2) == 0); 71 | b1.append((byte) 1); 72 | b1.append((byte) 1); 73 | b2.append((byte) 1); 74 | b2.append((byte) 2); 75 | assertTrue(b1.compareTo(b2) < 0); 76 | b1.clear(); 77 | b2.clear(); 78 | b1.append((byte) 1); 79 | b1.append((byte) 1); 80 | b2.append((byte) 1); 81 | b2.append((byte) 1); 82 | b2.append((byte) 1); 83 | assertTrue(b1.compareTo(b2) < 0); 84 | } 85 | 86 | @Test 87 | public void testPutReadShort() { 88 | LineBytesBuffer bb = new LineBytesBuffer(); 89 | for (short num = -1024; num < 1024; num++) { 90 | assertEquals(2, bb.putShort(num)); 91 | } 92 | bb.putShort(Short.MIN_VALUE); 93 | bb.putShort(Short.MAX_VALUE); 94 | AtomicInteger cursor = new AtomicInteger(); 95 | for (short num = -1024; num < 1024; num++) { 96 | assertEquals(num, bb.readShort(cursor)); 97 | } 98 | assertEquals(Short.MIN_VALUE, bb.readShort(cursor)); 99 | assertEquals(Short.MAX_VALUE, bb.readShort(cursor)); 100 | } 101 | 102 | @Test 103 | public void testPutReadString() { 104 | LineBytesBuffer bb = new LineBytesBuffer(); 105 | StringBuilder sb = new StringBuilder(); 106 | for (int i = 0; i < 1024; i++) { 107 | sb.append(Integer.toBinaryString(i)); 108 | sb.append(" oh "); 109 | } 110 | int writtenBytes = bb.putString(sb.toString()); 111 | AtomicInteger cursor = new AtomicInteger(0); 112 | assertEquals(sb.toString(), bb.readString(cursor)); 113 | assertEquals(writtenBytes, cursor.get()); 114 | } 115 | 116 | @Test 117 | public void testPutReadLong() { 118 | LineBytesBuffer bb = new LineBytesBuffer(); 119 | AtomicInteger cursor = new AtomicInteger(0); 120 | assertEquals(5, bb.putLong(0)); 121 | assertEquals(0, bb.readLong(cursor)); 122 | assertEquals(5, cursor.get()); 123 | long bigNum = 7L * Integer.MAX_VALUE; 124 | assertTrue(bigNum > Integer.MAX_VALUE); 125 | for (long num = bigNum - 5000; num <= bigNum; num++) { 126 | assertEquals(5, bb.putLong(num)); 127 | } 128 | for (long num = bigNum - 5000; num <= bigNum; num++) { 129 | assertEquals(num, bb.readLong(cursor)); 130 | } 131 | bb.clear(); 132 | cursor.set(0); 133 | long l1 = 338592878955L; 134 | assertTrue(bigNum < l1); 135 | assertEquals(5, bb.putLong(l1)); 136 | assertEquals(l1, bb.readLong(cursor)); 137 | } 138 | 139 | @Test 140 | public void testPutReadFloat() { 141 | LineBytesBuffer bb = new LineBytesBuffer(); 142 | AtomicInteger cursor = new AtomicInteger(0); 143 | bb.putFloat((float) Math.PI); 144 | assertEquals((float) Math.PI, bb.readFloat(cursor), 0.0000001); 145 | bb.putFloat((float) Math.E); 146 | assertEquals((float) Math.E, bb.readFloat(cursor), 0.0000001); 147 | } 148 | 149 | @After 150 | public void teardown() throws IOException { 151 | FileUtils.deleteDirectory(tempDirectory.toFile()); 152 | } 153 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/features/BinningTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import org.junit.Test; 4 | 5 | import static org.junit.Assert.*; 6 | 7 | public class BinningTest { 8 | 9 | @Test 10 | public void testGetInsertionPoint() throws Exception { 11 | Binning binning = new Binning(); 12 | for (int i = 0; i < 100; i++) { 13 | binning.addPercentile(i * 0.1); 14 | } 15 | binning.finishBuild(); 16 | for (int i = 0; i < 100; i++) { 17 | assertEquals(i, binning.getInsertionPoint(i * 0.1)); 18 | assertEquals(i, binning.getInsertionPoint(i * 0.1 + 0.005)); 19 | } 20 | } 21 | 22 | @Test 23 | public void testGetNumberOfValuesBetween1() throws Exception { 24 | Binning binning = new Binning(); 25 | for (int i = 0; i < 100; i++) { 26 | binning.addPercentile(i * 0.1); 27 | } 28 | binning.finishBuild(); 29 | for (int i = 0; i < 99; i++) { 30 | assertEquals("i=" + i, 1, binning.getNumberOfValuesBetween(i * 0.1, i * 0.1 + 0.1)); 31 | assertEquals("i=" + i, 1, binning.getNumberOfValuesBetween(i * 0.1 + 0.005, i * 0.1 + 0.105)); 32 | } 33 | assertEquals(1, binning.getNumberOfValuesBetween(9.9, 10.)); 34 | assertEquals(0, binning.getNumberOfValuesBetween(10., 10.1)); 35 | assertEquals(0, binning.getNumberOfValuesBetween(-0.1, 0.)); 36 | } 37 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/features/NumericalFeaturesStatisticsTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.features; 2 | 3 | import com.clearspring.analytics.stream.quantile.TDigest; 4 | import com.clearspring.analytics.util.Preconditions; 5 | import io.scaledml.core.util.Util; 6 | import it.unimi.dsi.fastutil.doubles.Double2DoubleMap; 7 | import it.unimi.dsi.fastutil.doubles.DoubleArrayList; 8 | import it.unimi.dsi.fastutil.doubles.DoubleList; 9 | import org.junit.Test; 10 | 11 | import java.util.concurrent.ThreadLocalRandom; 12 | 13 | import static org.junit.Assert.*; 14 | 15 | public class NumericalFeaturesStatisticsTest { 16 | 17 | @Test 18 | public void testBuildBinning() { 19 | TDigest digest = new TDigest(100); 20 | for (int i = 0; i < 1000; i++) { 21 | digest.add(ThreadLocalRandom.current().nextDouble()); 22 | } 23 | NumericalFeaturesStatistics st = new NumericalFeaturesStatistics() 24 | .percentsHistogramStep(0.01); 25 | Binning binning = st.buildBinning(digest, 0.); 26 | assertEquals(0, binning.getInsertionPoint(0.)); 27 | assertEquals(-1, binning.getInsertionPoint(-0.1)); 28 | assertEquals(99, binning.getInsertionPoint(1.)); 29 | assertEquals(99, binning.getInsertionPoint(Double.MAX_VALUE)); 30 | int middleInsertion = binning.getInsertionPoint(0.5); 31 | assertTrue("middleInsertion is " + middleInsertion, middleInsertion > 45 && middleInsertion < 55); 32 | int quarterInsertion = binning.getInsertionPoint(0.25); 33 | assertTrue("quarterInsertion is " + quarterInsertion, quarterInsertion > 20 && quarterInsertion < 30); 34 | int thirdQuarterInsertion = binning.getInsertionPoint(0.75); 35 | assertTrue("thirdQuarterInsertion is " + thirdQuarterInsertion, thirdQuarterInsertion > 70 && thirdQuarterInsertion < 80); 36 | } 37 | 38 | @Test 39 | public void testBuildHistogram() { 40 | Binning binning = new Binning(); 41 | for (int i = 0; i < 99; i++) { 42 | binning.addPercentile(ThreadLocalRandom.current().nextDouble()); 43 | } 44 | binning.addPercentile(0.).finishBuild(); 45 | NumericalFeaturesStatistics st = new NumericalFeaturesStatistics() 46 | .percentsHistogramStep(0.01); 47 | Double2DoubleMap histogram = st.buildHistogram(binning, 0, 1); 48 | assertEquals(102, histogram.size()); 49 | double sum = histogram.values() 50 | .stream().mapToDouble(Double::doubleValue).sum(); 51 | assertEquals(1., sum, Util.EPSILON); 52 | } 53 | 54 | @Test 55 | public void testBuildHistogram2() { 56 | DoubleList sample = new DoubleArrayList(); 57 | for (int i = 0; i < 1000; i++) { 58 | sample.add(ThreadLocalRandom.current().nextGaussian()); 59 | } 60 | double min = sample.stream().mapToDouble(Double::doubleValue).min().getAsDouble(); 61 | double max = sample.stream().mapToDouble(Double::doubleValue).max().getAsDouble(); 62 | Binning binning = new Binning(); 63 | sample.forEach(binning::addPercentile); 64 | binning.finishBuild(); 65 | NumericalFeaturesStatistics st = new NumericalFeaturesStatistics() 66 | .percentsHistogramStep(0.001); 67 | Double2DoubleMap histogram = st.buildHistogram(binning, min, max); 68 | double sum = histogram.values() 69 | .stream().mapToDouble(Double::doubleValue).sum(); 70 | assertEquals(1., sum, Util.EPSILON); 71 | } 72 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/ftrl/MainTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl; 2 | 3 | import org.junit.Test; 4 | 5 | import static org.junit.Assert.*; 6 | 7 | public class MainTest { 8 | @Test 9 | public void testMain() throws Exception { 10 | //test parse options 11 | Main.main("--help"); 12 | } 13 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/java/io/scaledml/ftrl/options/ColumnsInfoTest.java: -------------------------------------------------------------------------------- 1 | package io.scaledml.ftrl.options; 2 | 3 | import io.scaledml.core.inputformats.ColumnsMask; 4 | import org.junit.Test; 5 | 6 | import static junit.framework.TestCase.assertEquals; 7 | 8 | public class ColumnsInfoTest { 9 | 10 | @Test 11 | public void parsesLongMask() { 12 | ColumnsMask ilcnn = new ColumnsMask("ilcnn"); 13 | assertEquals(ColumnsMask.ColumnType.ID, ilcnn.getCategory(0)); 14 | assertEquals(ColumnsMask.ColumnType.LABEL, ilcnn.getCategory(1)); 15 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(2)); 16 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(3)); 17 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(4)); 18 | } 19 | 20 | @Test 21 | public void parsesWithBrackets1() { 22 | ColumnsMask ilcnn = new ColumnsMask("ic[1]n"); 23 | assertEquals(ColumnsMask.ColumnType.ID, ilcnn.getCategory(0)); 24 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(1)); 25 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(2)); 26 | } 27 | 28 | @Test 29 | public void parsesWithBrackets2() { 30 | ColumnsMask ilcnn = new ColumnsMask("ic[2]n"); 31 | assertEquals(ColumnsMask.ColumnType.ID, ilcnn.getCategory(0)); 32 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(1)); 33 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(2)); 34 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(3)); 35 | } 36 | 37 | @Test 38 | public void parsesWithBracketsAndTheRest() { 39 | ColumnsMask ilcnn = new ColumnsMask("lc[2]n[4]c"); 40 | assertEquals(ColumnsMask.ColumnType.LABEL, ilcnn.getCategory(0)); 41 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(1)); 42 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(2)); 43 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(3)); 44 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(4)); 45 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(5)); 46 | assertEquals(ColumnsMask.ColumnType.NUMERICAL, ilcnn.getCategory(6)); 47 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(7)); 48 | assertEquals(ColumnsMask.ColumnType.CATEGORICAL, ilcnn.getCategory(100)); 49 | } 50 | 51 | } -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/resources/ruslan-test-small.csv: -------------------------------------------------------------------------------- 1 | CLICK,CAT01,CAT02,CAT03,CAT04,CAT05,CAT06,CAT07,CAT08,CAT09,CAT10,CAT11,CAT12,CAT13,CAT14,CAT15,CA16,CAT17,CAT18,CAT19,CAT20,CAT21,CAT22,CAT23,CAT24,CAT25,CAT26,CAT27,CAT28,CAT29,CAT30,CAT31,CAT32,AT33,CAT34,CAT35,CAT36,CAT37,NUM01,NUM02,NUM03,NUM04,NUM05,NUM06,NUM07,NUM08,NUM09,NUM10,NUM11,NUM12,NUM13,NUM14,NUM15,NUM16,NUM17,NUM18,NUM19,NUM20,NUM21,NUM22,NUM23 2 | 0,5d922427,05db9164,d6b9e7c1,36103458,26eef6f3,3b08e48b,2ad7df96,,9b17ab79,c8afb369,c8afb369,8084ee93,003d4f4f,27d1baee,3b6c1f5d,fe6b92e5,2c892c66,43b19349,ea2ea347,c18be181,8fe001f4,202a4552,41d09bb1,8e108113,bf1389ab,e79c0f81,73f94f3b,b6a5c8d9,02cf9876,73a0431c,0bf4d2c2,,d42eb6f6,6a8d8998,fd323779,dc6773ae,a79d9eee,0.0,0.0,2,0,0.0,0,20,1.0,,29660.0,,,1,2966.0,2.0,85,,1.0,20.0,20.0,2,,2966 3 | 0,5d922427,7d35c0db,5b392875,36103458,26eef6f3,4fee36e2,1f6f0bb6,5c4693e8,949ae048,c8afb369,c8afb369,8084ee93,003d4f4f,454bf5f0,3b6c1f5d,fe6b92e5,32c7478e,43b19349,1e88c74f,169de313,8fe001f4,,41d09bb1,,e30839e1,e79c0f81,73f94f3b,b6a5c8d9,60bc0f35,73a0431c,1adce6ef,cc651ac8,d42eb6f6,6a8d8998,3b183c5c,,a73ee510,0.0,0.0,2,0,0.0,0,20,1.0,,29660.0,,,1,2966.0,2.0,85,,1.0,20.0,20.0,2,,2966 4 | 1,06373944,5a9ed9b0,0b153874,1195884a,4453d02c,4fee36e2,80a17bc7,21ddcdc9,c73a43c9,39e026c9,606b0dda,38d50e09,e30eff7f,8d51ec69,872c22d6,6f6d9be8,2c892c66,25c83c98,ea2ea347,88e439d9,6984f1a5,b1252a9d,d50ef305,8e108113,bf1389ab,b74dd19b,432000b0,d3802338,92eb3174,7ca25fd2,0bf4d2c2,001f3601,3fea8d91,f96c5841,df487a73,,a73ee510,0.0,0.0,,1,3.0,3,0,-1.0,0,0.0,0,,-1,3605.0,0.0,17,33,-10.0,0.0,0.0,,,3572 5 | 1,06373944,5a9ed9b0,d6b9e7c1,1195884a,4453d02c,1b46e71c,0b1925b7,21ddcdc9,ba10a31d,39e026c9,606b0dda,38d50e09,fffe2a63,8d51ec69,872c22d6,7f809d5c,32c7478e,25c83c98,07c540c4,88e439d9,604f499b,b1252a9d,4e784388,c27f155b,386be15c,b74dd19b,432000b0,830c881e,60bc0f35,7ca25fd2,b28479f6,001f3601,3fea8d91,f96c5841,df487a73,,a79d9eee,0.0,0.0,,1,3.0,3,0,-1.0,0,35720.0,0,,-1,3605.0,0.0,17,33,1.0,0.0,0.0,,,3572 6 | 0,a4462113,17f69355,d6b9e7c1,b041b04a,3b6328cb,47913bfa,4e661096,,b22a7d4a,c8afb369,c8afb369,421b43cd,2804effd,454bf5f0,723b4dfd,7e0ccccf,2c892c66,25c83c98,d4bb7bd8,29998ed1,6aaba33c,,83be4a6a,,23eaa4a0,ba626733,855d9575,cd98af01,97a09517,d9b1e3ff,0bf4d2c2,cc651ac8,50867cdd,96b830aa,fd323779,,a79d9eee,2.0,0.0,2,1,1.0,1,2,222.0,0,35390.0,0,,181,3689.0,2.0,23,150,1.2265193370165746,43.0,82.0,2,41,3539 7 | 0,2d0bb053,17f69355,d6b9e7c1,b041b04a,3b6328cb,47913bfa,1f6f0bb6,,b22a7d4a,c8afb369,c8afb369,421b43cd,2804effd,f0cba4ae,723b4dfd,7e0ccccf,2c892c66,25c83c98,ea2ea347,29998ed1,6aaba33c,,83be4a6a,,dcc19010,0dd1dd57,855d9575,cd98af01,97a09517,3e5f0580,b28479f6,,50867cdd,96b830aa,b34f3128,dc6773ae,a79d9eee,2.0,0.0,2,1,1.0,1,2,222.0,0,35390.0,0,,181,3689.0,2.0,23,150,40182.0,43.0,0.04878048780487805,2,41,3539 8 | 0,916e9a2c,05db9164,0b153874,fa48817d,41848647,b173a655,3f099307,21ddcdc9,b22a7d4a,51ef04f7,606b0dda,c1384774,8e8b535e,0e056148,864d8c0b,,32c7478e,25c83c98,ea2ea347,784ab0b0,6984f1a5,b1252a9d,f2487bdb,89ba17d9,5d36cd8e,c5229266,4a64999b,740c210d,51037f2e,5874c9c9,b28479f6,ea9a246c,8c2d2040,38100ca1,089c0ea9,,7cc72ec2,0.0,0.0,1,0,0.0,0,2,3.0,,0.0,,0,0,82739.0,1.0,48,276,0.0,5.0,0.3333333333333333,1,3,82463 9 | 0,916e9a2c,05db9164,0b153874,fa48817d,41848647,4fee36e2,f0cba4ae,21ddcdc9,667797fb,51ef04f7,606b0dda,c1384774,8e8b535e,0e056148,3b6c1f5d,7f809d5c,32c7478e,25c83c98,27c07bd6,784ab0b0,6984f1a5,b1252a9d,f2487bdb,89ba17d9,f4a4b035,c5229266,855d9575,740c210d,51037f2e,5874c9c9,b28479f6,ea9a246c,8c2d2040,38100ca1,089c0ea9,,7cc72ec2,0.0,0.0,1,0,0.0,0,2,3.0,,824630.0,,0,0,82739.0,1.0,48,276,30.0,5.0,3.0,1,3,82463 10 | 0,a4462113,7d35c0db,0b153874,154aedde,520e4d6a,cce8f055,80a17bc7,5c4693e8,2e48855c,51ef04f7,a51c2f41,287130e0,e30eff7f,8d51ec69,e2f749f7,7f809d5c,32c7478e,982fa226,ea2ea347,d89ea181,614e3e67,202a4552,ab34a858,74b7a7e2,1d08f649,17c5b10d,22fc1738,34cbb1bc,af16409d,364e8b48,b28479f6,ea9a246c,479165d1,4ff020d0,023a2077,,a73ee510,0.75,0.0,3,3,4.0,4,2,2.0,,0.0,,0,1,2911.0,3.0,7,4,2.0,3.0,3.0,3,1,2907 11 | 0,9efd8b77,7d35c0db,d6b9e7c1,154aedde,520e4d6a,cce8f055,454bf5f0,87d578e3,4e045f5a,d53dc20a,a51c2f41,287130e0,891589e7,0e056148,e2f749f7,7f809d5c,32c7478e,982fa226,07c540c4,d89ea181,614e3e67,a458ea53,ab34a858,74b7a7e2,c2f02320,17c5b10d,22fc1738,34cbb1bc,af16409d,3e5f0580,b28479f6,ea9a246c,479165d1,4ff020d0,fd323779,,a79d9eee,12.0,0.0,3,3,4.0,4,2,2.0,,29070.0,,0,1,2911.0,3.0,7,4,2.0,3.0,3.0,3,1,2907 12 | 0,9efd8b77,05db9164,0b153874,154aedde,520e4d6a,cce8f055,3f95d319,87d578e3,13285044,51ef04f7,a51c2f41,287130e0,891589e7,8d51ec69,e2f749f7,7e0ccccf,2c892c66,25c83c98,07c540c4,d89ea181,614e3e67,a458ea53,ab34a858,74b7a7e2,80a17bc7,17c5b10d,22fc1738,34cbb1bc,af16409d,3e5f0580,b28479f6,ea9a246c,479165d1,4ff020d0,023a2077,,a73ee510,12.0,0.0,3,3,4.0,4,2,2.0,,29070.0,,0,1,2911.0,3.0,7,4,2.0,3.0,3.0,3,1,2907 13 | 0,f5baaaa1,68fd1e64,0b153874,72aaec00,60d289ab,dc790dda,bff9df05,,944d305d,d53dc20a,c8afb369,b961056b,c24ac50d,e30839e1,97008b21,7e0ccccf,32c7478e,25c83c98,e5ba7672,0fc80044,02eb2161,,f4ae27b8,,2ad7df96,ffd9bea8,8a924736,b688506c,0668ec3a,3e5f0580,07d13a8f,,2f5ab93a,cfc221a3,60a197ae,,a79d9eee,60.0,0.0,6,0,0.0,0,6,2.0,,0.0,,,1,23077.0,6.0,1,,2.0,7.0,6.0,6,1,23077 14 | 0,f5baaaa1,7d35c0db,0b153874,72aaec00,60d289ab,4fee36e2,a7dd2259,5c4693e8,b6c6a379,c8afb369,c8afb369,5f71999f,c24ac50d,0e056148,97008b21,7e0ccccf,32c7478e,25c83c98,e5ba7672,0fc80044,02eb2161,,f4ae27b8,,47b48dcf,ffd9bea8,8a924736,b688506c,0668ec3a,e3205ff0,07d13a8f,,2f5ab93a,38100ca1,60a197ae,dc6773ae,a73ee510,60.0,0.0,6,0,0.0,0,6,2.0,,230770.0,,,1,23077.0,6.0,1,,2.0,7.0,6.0,6,1,23077 15 | 0,f1e1df0a,7d35c0db,5b392875,ba0eeab9,02aa6b08,0461dac0,1f6f0bb6,,4e5852ea,d53dc20a,ba8fd52a,e112a9de,e30eff7f,27d1baee,8dbce355,7f809d5c,c3dc6cef,982fa226,3486227d,169de313,21161865,202a4552,838f50c6,,31b3e683,cb8f2d27,855d9575,d7ccce7c,6b80d7c0,e2274ca7,0bf4d2c2,,60261742,0d1a7317,8f079aa5,,a73ee510,0.0,0.0,,,0.0,,0,19.0,,100690.0,,,19,10069.0,0.0,,,361.0,0.0,0.0,,,10069 16 | 0,f1e1df0a,05db9164,5b392875,1195884a,02aa6b08,0461dac0,21763d48,,2ef00afe,c8afb369,c8afb369,e112a9de,fdbdefe6,0e056148,8dbce355,fbad5c96,c3dc6cef,307e775a,3486227d,81e71114,6984f1a5,,838f50c6,8e108113,bf1389ab,cb8f2d27,55f8f502,d7ccce7c,6b80d7c0,e2274ca7,ad1cc976,,60261742,38100ca1,8f079aa5,,a73ee510,0.0,0.0,,,0.0,,0,19.0,,0.0,,,19,10069.0,0.0,,,361.0,0.0,0.0,,,10069 17 | 0,f1e1df0a,05db9164,d6b9e7c1,ba0eeab9,02aa6b08,0461dac0,5c6c77b7,,63a67935,c8afb369,c8afb369,5f71999f,e30eff7f,27d1baee,8dbce355,fbad5c96,c3dc6cef,307e775a,ea2ea347,81e71114,21161865,,838f50c6,8e108113,218891af,cb8f2d27,55f8f502,830c881e,6b80d7c0,3e5f0580,ad1cc976,,60261742,0d1a7317,fd323779,,a73ee510,0.0,0.0,,,0.0,,0,19.0,,0.0,,,19,10069.0,0.0,,,361.0,0.0,0.0,,,10069 18 | 0,55dc357b,05db9164,062b5529,c92f3b61,38537aaf,3b08e48b,21763d48,21ddcdc9,ef9b1422,51ef04f7,606b0dda,f0cf0024,b04e4670,8d51ec69,60f6221e,fbad5c96,c7dc6720,25c83c98,776ce399,41274cd7,623049e6,b1252a9d,072f3aba,731c3655,b35eaacc,0850bcd9,af4ae3a4,b657eb7f,6f67f7e5,149170d2,1adce6ef,ea9a246c,d7497e30,5fca948f,43f13e8b,,a73ee510,0.0,0.0,4,0,0.0,0,4,14.0,,70.0,,,0,7.0,4.0,4,,140.0,18.0,56.0,4,14,7 19 | 0,55dc357b,05db9164,062b5529,c92f3b61,38537aaf,3b08e48b,5b56befb,21ddcdc9,080055f1,51ef04f7,606b0dda,f0cf0024,b04e4670,f0cba4ae,60f6221e,fbad5c96,c7dc6720,25c83c98,776ce399,41274cd7,623049e6,b1252a9d,072f3aba,731c3655,5f19ed16,0850bcd9,af4ae3a4,b657eb7f,6f67f7e5,149170d2,1adce6ef,ea9a246c,d7497e30,5fca948f,43f13e8b,,a73ee510,40.0,0.0,4,0,0.0,0,4,14.0,,0.0,,,0,7.0,4.0,4,,140.0,18.0,56.0,4,14,7 20 | 0,55dc357b,05db9164,062b5529,1195884a,38537aaf,3b08e48b,f0cba4ae,21ddcdc9,d8546b0e,d53dc20a,606b0dda,f0cf0024,b04e4670,e30839e1,3b6c1f5d,fbad5c96,2c892c66,25c83c98,776ce399,41274cd7,623049e6,b1252a9d,072f3aba,731c3655,bf1389ab,0850bcd9,855d9575,830c881e,6f67f7e5,3e5f0580,1adce6ef,ea9a246c,d7497e30,38100ca1,43f13e8b,dc6773ae,a79d9eee,0.0,0.0,4,0,0.0,0,4,14.0,,0.0,,,0,7.0,4.0,4,,140.0,18.0,0.2857142857142857,4,14,7 21 | 0,4f7854df,05db9164,0b153874,,c8afb369,877bfc40,f0cba4ae,,b22a7d4a,c8afb369,c8afb369,5f71999f,cdfa8259,0e056148,,,32c7478e,982fa226,e5ba7672,,,202a4552,dc1a35f3,,3f099307,8a3d7e2d,855d9575,345cf499,60bc0f35,d9466be4,0bf4d2c2,,830368a7,bdda1023,,,a73ee510,0.0,0.0,,1,6.0,4,2,2.0,0,2608.0,2,,2,1304.0,2.0,2,0,4.0,2.0,0.0,,,1304 22 | 0,4f7854df,7d35c0db,0b153874,,c8afb369,877bfc40,c29d241e,,378dd116,c8afb369,c8afb369,5f71999f,cdfa8259,8d51ec69,,7f809d5c,2c892c66,25c83c98,e5ba7672,,6984f1a5,,dc1a35f3,,bf1389ab,8a3d7e2d,8cc41fbc,345cf499,,d9466be4,07d13a8f,cc651ac8,830368a7,bdda1023,,,a73ee510,0.0,0.0,,1,6.0,4,2,2.0,0,652.0,2,,2,1304.0,2.0,2,0,1.0,2.0,0.0,,,1304 23 | 0,a4462113,05db9164,0b153874,1195884a,c8afb369,877bfc40,d5a87b62,,1c54cac5,d53dc20a,ba8fd52a,ad4527a2,e30eff7f,f0cba4ae,,,2c892c66,982fa226,ea2ea347,,,,dc1a35f3,,bff9df05,ba626733,8cc41fbc,345cf499,60bc0f35,d9466be4,0bf4d2c2,,830368a7,bdda1023,,dc6773ae,a73ee510,0.0,0.0,,1,6.0,4,2,2.0,0,652.0,2,,2,1304.0,2.0,2,0,4.0,2.0,0.0,,,1304 24 | 0,a4462113,ae82ea21,0b153874,8a1f421d,23077b13,3b08e48b,145d36f0,5c4693e8,efa7666e,c8afb369,c8afb369,8db5bc37,181879d3,f0cba4ae,4559c9a5,7f809d5c,be7c41b4,25c83c98,ea2ea347,a6a40ee4,6984f1a5,202a4552,e2ec9176,,bf1389ab,3d5408c3,bfe399e4,b0c30eeb,60bc0f35,a50ef3e5,07d13a8f,cc651ac8,aa0c1468,38100ca1,6ba379d8,dc6773ae,7cc72ec2,0.0,0.0,2,0,0.0,0,2,2.0,,0.0,,,1,33121.0,2.0,2,,2.0,3.0,2.0,2,1,33121 25 | 0,a4462113,ae82ea21,0b153874,8a1f421d,23077b13,3b08e48b,1f6f0bb6,5c4693e8,4dfac9c9,c8afb369,ba8fd52a,5f71999f,e30eff7f,f0cba4ae,4559c9a5,fe6b92e5,be7c41b4,25c83c98,e5ba7672,a6a40ee4,ece8e513,,e2ec9176,,d5a87b62,ba626733,bfe399e4,b0c30eeb,fbc43fd8,a50ef3e5,0bf4d2c2,,2304acb2,8ebee9f6,6ba379d8,,7cc72ec2,0.0,0.0,2,0,0.0,0,2,2.0,,331210.0,,,1,33121.0,2.0,2,,2.0,3.0,2.0,2,1,33121 26 | 0,a4462113,ae82ea21,0b153874,8a1f421d,23077b13,3b08e48b,b21f6f64,5c4693e8,2f28d158,c8afb369,c8afb369,8db5bc37,e30eff7f,f0cba4ae,3b6c1f5d,fe6b92e5,be7c41b4,25c83c98,e5ba7672,a6a40ee4,ece8e513,,e2ec9176,,964b9b5c,ba626733,bfe399e4,b0c30eeb,fbc43fd8,a50ef3e5,07d13a8f,,2304acb2,8ebee9f6,fd323779,,a79d9eee,0.0,0.0,2,0,0.0,0,2,2.0,,331210.0,,,1,33121.0,2.0,2,,2.0,3.0,2.0,2,1,33121 27 | 0,37f2f6dc,ae82ea21,0b153874,8a1f421d,23077b13,3b08e48b,292ffaec,,b22a7d4a,c8afb369,ba8fd52a,8db5bc37,181879d3,27d1baee,4559c9a5,7f809d5c,2c892c66,25c83c98,e5ba7672,a6a40ee4,ece8e513,,d50ef305,,7a27b5f3,ba626733,855d9575,830c881e,fbc43fd8,a50ef3e5,07d13a8f,cc651ac8,2304acb2,8ebee9f6,6ba379d8,,7cc72ec2,20.0,0.0,2,0,0.0,0,2,2.0,,331210.0,,,1,33121.0,2.0,2,,2.0,3.0,2.0,2,1,33121 28 | 1,c6ceab4d,68fd1e64,37e4aa92,b1f3cb49,e7e2fcab,1212582a,1f6f0bb6,,8d4b5d20,c8afb369,c8afb369,ed7b1c58,167708ca,0e056148,723ce5bc,,32c7478e,25c83c98,e5ba7672,342ddc8b,85f7511a,,341fa342,,728c5454,918d843d,8113d08b,e0a5ea97,5d0ca0f6,fef0266d,b28479f6,,2c049bbc,1661686d,2fd70e1c,dc6773ae,a79d9eee,0.0,1.0,,4,22.0,14,0,-1.0,1,3048.0,8,,-1,381.0,8.0,6,0,-10.0,0.0,0.0,,,381 29 | 1,c6ceab4d,68fd1e64,d6b9e7c1,b1f3cb49,cb3ddd08,1212582a,c9cd0b01,,1e7f83c3,c8afb369,c8afb369,ed7b1c58,167708ca,e30839e1,723ce5bc,7f809d5c,32c7478e,25c83c98,e5ba7672,342ddc8b,85f7511a,,341fa342,,0b1925b7,918d843d,8113d08b,e0a5ea97,5d0ca0f6,fef0266d,b28479f6,,2c049bbc,38100ca1,2fd70e1c,,a79d9eee,0.0,1.0,,4,22.0,14,0,-1.0,1,47.625,8,,-1,381.0,8.0,6,0,-10.0,0.0,0.0,,,381 30 | 1,c6ceab4d,68fd1e64,37e4aa92,b1f3cb49,cb3ddd08,1212582a,292ffaec,,2558b80e,c8afb369,c8afb369,ed7b1c58,167708ca,454bf5f0,723ce5bc,,32c7478e,25c83c98,e5ba7672,342ddc8b,85f7511a,,341fa342,,964b9b5c,918d843d,8113d08b,e0a5ea97,5d0ca0f6,fef0266d,b28479f6,,2c049bbc,1661686d,2fd70e1c,,a73ee510,0.0,1.0,,4,22.0,14,0,-1.0,1,3048.0,8,,-1,381.0,8.0,6,0,-10.0,0.0,0.0,,,381 31 | 0,c6438ddb,7d35c0db,0b153874,d925fa94,bad4ee45,4fee36e2,df6115d8,21ddcdc9,b22a7d4a,39e026c9,606b0dda,5f71999f,fa0643ee,0e056148,c8297264,fe6b92e5,32c7478e,25c83c98,d4bb7bd8,4e1c036b,795646b5,b1252a9d,d50ef305,ec1e3f9e,23eaa4a0,0ecd456e,855d9575,830c881e,60bc0f35,54e3c2f7,b28479f6,001f3601,e36715e8,2dc93905,fd323779,,a73ee510,0.0,0.0,1,,0.0,,0,15.0,0,70600.0,0,,0,7060.0,0.0,,,150.0,15.0,0.0,0,15,7060 32 | 0,a4462113,05db9164,0b153874,d925fa94,bad4ee45,3b08e48b,973b5580,21ddcdc9,de78cbb7,39e026c9,606b0dda,207b2d81,fa0643ee,454bf5f0,c8297264,7f809d5c,32c7478e,982fa226,ea2ea347,4e1c036b,795646b5,b1252a9d,45fccddd,8e108113,21763d48,ba626733,29999e76,baf9aeb5,b3770b31,54e3c2f7,b28479f6,001f3601,e36715e8,2dc93905,2896ad66,,a73ee510,0.0,0.0,1,,0.0,,0,15.0,0,70600.0,0,,0,7060.0,0.0,,,150.0,15.0,0.0,0,15,7060 33 | 0,c6438ddb,05db9164,0b153874,d925fa94,bad4ee45,3b08e48b,44fae045,21ddcdc9,8fa2ca78,39e026c9,606b0dda,207b2d81,e30eff7f,27d1baee,3b6c1f5d,fe6b92e5,32c7478e,25c83c98,d4bb7bd8,4e1c036b,795646b5,b1252a9d,45fccddd,ec1e3f9e,cbb85a30,ba626733,29999e76,830c881e,b3770b31,54e3c2f7,b28479f6,001f3601,e36715e8,2dc93905,2896ad66,,a73ee510,0.0,0.0,1,,0.0,,0,15.0,0,70600.0,0,,0,7060.0,0.0,,,0.0,15.0,0.0,0,15,7060 34 | 0,06373944,7d35c0db,d6b9e7c1,8e662061,4453d02c,4fee36e2,2ad7df96,5c4693e8,b22a7d4a,acf70d0f,ba8fd52a,38d50e09,fffe2a63,27d1baee,872c22d6,fe6b92e5,32c7478e,b2241560,3486227d,88e439d9,604f499b,b1252a9d,be6ddaca,c27f155b,bf1389ab,b74dd19b,67f6d65b,addc3db7,60bc0f35,3e5f0580,b28479f6,001f3601,3fea8d91,38100ca1,fd323779,78e2e389,a73ee510,5.0,1.0,1,1,5.0,5,1,35.0,,0.0,,1,30,8927.0,1.0,1,5,1.1666666666666667,6.0,0.2,1,5,8922 35 | 0,06373944,05db9164,0b153874,8e662061,4453d02c,4ad94725,a7dd2259,21ddcdc9,4cbb2e74,acf70d0f,83703e25,38d50e09,fffe2a63,e30839e1,872c22d6,fe6b92e5,32c7478e,b2241560,3486227d,88e439d9,604f499b,b1252a9d,be6ddaca,c27f155b,46f78cba,b74dd19b,855d9575,addc3db7,92eb3174,585a8b28,0bf4d2c2,cc651ac8,3fea8d91,7bc397ae,df487a73,78e2e389,a73ee510,0.2,1.0,1,1,5.0,5,1,35.0,,0.0,,1,30,8927.0,1.0,1,5,1050.0,6.0,5.0,1,5,8922 36 | 0,06373944,7d35c0db,0b153874,8e662061,4453d02c,4ad94725,2f532987,21ddcdc9,e16f60f8,acf70d0f,83703e25,38d50e09,fffe2a63,e30839e1,3b6c1f5d,fe6b92e5,32c7478e,b2241560,ea2ea347,88e439d9,604f499b,b1252a9d,be6ddaca,8e108113,41f77f93,ba626733,67f6d65b,addc3db7,92eb3174,585a8b28,0bf4d2c2,001f3601,aa0c1468,7bc397ae,df487a73,dc6773ae,a73ee510,5.0,1.0,1,1,5.0,5,1,35.0,,0.0,,1,30,8927.0,1.0,1,5,1.1666666666666667,6.0,5.0,1,5,8922 37 | 0,b62ec7c9,7d35c0db,37e4aa92,1195884a,e7e2fcab,4fee36e2,1f6f0bb6,21ddcdc9,32f7c942,2a92602b,f7c440b4,b06f9574,e30eff7f,0e056148,695514c6,7e0ccccf,be7c41b4,4cf72387,ea2ea347,5ec469f9,c94a041d,b1252a9d,e2ec9176,77152453,bf1389ab,0793018f,cf5fc6c4,b0c30eeb,60bc0f35,ef121faa,1adce6ef,cc651ac8,0e9b0809,ca95dc2e,dadea544,ad3062eb,7cc72ec2,10.0,0.0,1,0,0.0,0,1,3.0,,0.0,,,0,246255.0,1.0,28,,0.0,4.0,3.0,1,3,246255 38 | 0,b62ec7c9,68fd1e64,d6b9e7c1,1195884a,1af78cd7,3b08e48b,44fae045,21ddcdc9,b22a7d4a,2a92602b,ba8fd52a,b06f9574,e9a3d86d,f0cba4ae,3b6c1f5d,7e0ccccf,2c892c66,982fa226,1e88c74f,5ec469f9,c94a041d,b1252a9d,d50ef305,77152453,dc4f0e11,ba626733,855d9575,830c881e,d8dd6f6f,ef121faa,1adce6ef,001f3601,0e9b0809,38100ca1,dadea544,ad3062eb,7cc72ec2,0.0,0.0,1,0,0.0,0,1,3.0,,2462550.0,,,0,246255.0,1.0,28,,0.0,4.0,0.3333333333333333,1,3,246255 39 | 0,b62ec7c9,68fd1e64,37e4aa92,18fbd7af,1af78cd7,3b08e48b,44fae045,21ddcdc9,7101e4b7,2a92602b,f7c440b4,b06f9574,e9a3d86d,454bf5f0,695514c6,7e0ccccf,2c892c66,4cf72387,1e88c74f,5ec469f9,c94a041d,b1252a9d,e2ec9176,77152453,740f6cf6,0793018f,cf5fc6c4,b0c30eeb,d8dd6f6f,ef121faa,1adce6ef,001f3601,0e9b0809,ca95dc2e,dadea544,ad3062eb,7cc72ec2,0.0,0.0,1,0,0.0,0,1,3.0,,2462550.0,,,0,246255.0,1.0,28,,0.0,4.0,3.0,1,3,246255 40 | 0,b62ec7c9,68fd1e64,37e4aa92,18fbd7af,1af78cd7,3b08e48b,d5a87b62,21ddcdc9,74c5cf9c,2a92602b,f7c440b4,b06f9574,e9a3d86d,0e056148,695514c6,7e0ccccf,be7c41b4,4cf72387,1e88c74f,5ec469f9,c94a041d,202a4552,e2ec9176,77152453,bf1389ab,0793018f,cf5fc6c4,b0c30eeb,d8dd6f6f,ef121faa,1adce6ef,001f3601,0e9b0809,ca95dc2e,dadea544,ad3062eb,7cc72ec2,10.0,0.0,1,0,0.0,0,1,3.0,,0.0,,,0,246255.0,1.0,28,,0.0,4.0,3.0,1,3,246255 41 | 1,2b8f96c1,05db9164,37e4aa92,cada85da,41689d43,3b08e48b,e30839e1,21ddcdc9,0b7e5fc7,c6956305,ba8fd52a,8cc9c66e,a6f5dd38,27d1baee,6bbf758a,fe6b92e5,32c7478e,982fa226,3486227d,b387ab8b,d71cad82,b1252a9d,468a0854,6df850a8,691f5737,ba626733,0949cc78,605bbc24,60bc0f35,3e5f0580,07d13a8f,cc651ac8,9f523c93,736dbc04,c9e6e726,c9d4222a,a79d9eee,0.0,2.0,0,1,2.0,1,0,121.0,1,1.0,1,1,121,2.0,1.0,0,1,14641.0,0.0,0.0,0,,1 42 | 1,2b8f96c1,05db9164,37e4aa92,cada85da,41689d43,4fee36e2,973b5580,21ddcdc9,b22a7d4a,c6956305,ef2fa723,5f71999f,e30eff7f,454bf5f0,6bbf758a,fe6b92e5,32c7478e,25c83c98,3486227d,169de313,d71cad82,b1252a9d,d50ef305,8e108113,20a4b55f,ba626733,855d9575,830c881e,c27c1bb0,3e5f0580,07d13a8f,2bf691b1,aa0c1468,736dbc04,c9e6e726,c9d4222a,a73ee510,0.0,2.0,0,1,2.0,1,0,121.0,1,1.0,1,1,121,2.0,1.0,0,1,1.0,0.0,0.0,0,,1 43 | 1,2b8f96c1,05db9164,d6b9e7c1,cada85da,e7e2fcab,3b08e48b,c9cd0b01,5c4693e8,e88ca123,c6956305,ef2fa723,8cc9c66e,a6f5dd38,454bf5f0,6bbf758a,7f809d5c,32c7478e,25c83c98,3486227d,b387ab8b,d71cad82,202a4552,468a0854,8e108113,bcc5d651,ca952fbf,0949cc78,605bbc24,c27c1bb0,a60de4e5,07d13a8f,cc651ac8,9f523c93,38100ca1,c9e6e726,c9d4222a,a73ee510,0.0,2.0,0,1,2.0,1,0,121.0,1,1.0,1,1,121,2.0,1.0,0,1,14641.0,0.0,0.0,0,,1 44 | 0,ee569ce2,fbc55dae,37e4aa92,,39e026c9,3b08e48b,fdd449cf,21ddcdc9,8e6bf04c,39e026c9,606b0dda,38d50e09,582152eb,e30839e1,,7e0ccccf,32c7478e,4cf72387,776ce399,,,5840adea,4c397c64,56be3401,f4a4b035,9c6fdd57,de76ec6f,32569bcf,,3e5f0580,07d13a8f,001f3601,3fea8d91,39bce589,,,a73ee510,10.0,0.0,33,0,0.0,0,33,277.0,,0.0,,,-1,11796.0,1.0,33,,-277.0,311.0,0.0035971223021582736,1,278,11796 45 | 0,ee569ce2,fbc55dae,37e4aa92,1195884a,39e026c9,3b08e48b,d5a87b62,5c4693e8,c59cc787,39e026c9,606b0dda,5f71999f,582152eb,8d51ec69,,7f809d5c,32c7478e,4cf72387,776ce399,169de313,,202a4552,4c397c64,56be3401,b35eaacc,ba626733,de76ec6f,32569bcf,,494bd436,07d13a8f,001f3601,3fea8d91,39bce589,,,a73ee510,0.0,0.0,33,0,0.0,0,33,277.0,,0.0,,,-1,11796.0,1.0,33,,-277.0,311.0,278.0,1,278,11796 46 | 0,ee569ce2,fbc55dae,37e4aa92,,39e026c9,3b08e48b,21763d48,21ddcdc9,33224b37,39e026c9,606b0dda,38d50e09,582152eb,8d51ec69,,7e0ccccf,32c7478e,4cf72387,776ce399,,,5840adea,4c397c64,56be3401,740f6cf6,9c6fdd57,de76ec6f,32569bcf,,494bd436,07d13a8f,001f3601,3fea8d91,39bce589,,,a73ee510,10.0,0.0,33,0,0.0,0,33,277.0,,117960.0,,,-1,11796.0,1.0,33,,-277.0,311.0,278.0,1,278,11796 47 | 0,ee569ce2,fbc55dae,37e4aa92,,39e026c9,3b08e48b,91916258,5c4693e8,b22a7d4a,39e026c9,606b0dda,38d50e09,582152eb,e30839e1,,7e0ccccf,2c892c66,4cf72387,776ce399,,6984f1a5,5840adea,4c397c64,56be3401,bf1389ab,9c6fdd57,de76ec6f,830c881e,,3e5f0580,07d13a8f,cc651ac8,aa0c1468,39bce589,,,a79d9eee,10.0,0.0,33,0,0.0,0,33,277.0,,0.0,,,-1,11796.0,1.0,33,,-277.0,311.0,0.0035971223021582736,1,278,11796 48 | 0,0f942372,7d35c0db,d6b9e7c1,87acb535,a935dd04,465779f0,3f099307,21ddcdc9,b22a7d4a,7d1526c6,606b0dda,80e26c9b,005c6740,0e056148,3b6c1f5d,fbad5c96,3a171ecb,4cf72387,07c540c4,f922efad,6984f1a5,5840adea,493a2762,9904c656,b4d1fa42,ba626733,c92f1559,ac44a253,60bc0f35,358dd1ee,1adce6ef,e8b83407,aa0c1468,70897b90,b34f3128,,a73ee510,1.0,0.0,2,1,2.0,2,3,85.0,0,29510.0,0,,0,3120.0,2.0,51,169,0.0,88.0,170.0,2,85,2951 49 | 0,0f942372,05db9164,0b153874,87acb535,a935dd04,4fee36e2,2f532987,21ddcdc9,cce5919a,7d1526c6,606b0dda,80e26c9b,005c6740,8d51ec69,a4b7004c,fbad5c96,3a171ecb,4cf72387,07c540c4,f922efad,d326786e,5840adea,493a2762,9904c656,1d08f649,7196923f,c92f1559,ac44a253,256e0878,358dd1ee,1adce6ef,e8b83407,8a544033,70897b90,b34f3128,,a73ee510,4.0,0.0,2,1,2.0,2,3,85.0,0,0.0,0,,0,3120.0,2.0,51,169,850.0,88.0,170.0,2,85,2951 50 | 0,a4462113,05db9164,0b153874,87acb535,a935dd04,465779f0,9e8e4808,5c4693e8,2046394c,7d1526c6,606b0dda,5f71999f,005c6740,8d51ec69,a4b7004c,fbad5c96,3a171ecb,4cf72387,07c540c4,f922efad,d326786e,5840adea,493a2762,9904c656,292ffaec,7196923f,c92f1559,ac44a253,256e0878,358dd1ee,1adce6ef,e8b83407,aa0c1468,70897b90,b34f3128,,a73ee510,4.0,0.0,2,1,2.0,2,3,85.0,0,29510.0,0,,0,3120.0,2.0,51,169,0.0,88.0,0.023529411764705882,2,85,2951 51 | 0,42b3012c,68fd1e64,062b5529,db741cd7,e7e2fcab,e7d1c6a1,4e661096,21ddcdc9,53f55c3a,d53dc20a,606b0dda,38d50e09,582152eb,f0cba4ae,c61ef534,7e0ccccf,3a171ecb,25c83c98,e5ba7672,11622d5f,2263e94f,202a4552,800354a7,aa5f0a15,292ffaec,ffc3da20,cf4fb6b3,a98bc52f,8bd4620d,c851b930,b28479f6,001f3601,aa0c1468,38100ca1,fbd032d8,,7cc72ec2,10.0,0.0,1,0,0.0,0,1,23.0,,7443700.0,,,23,744370.0,1.0,1,,1.0,1.0,0.0,1,,744370 52 | 0,42b3012c,68fd1e64,062b5529,db741cd7,cfa87f80,e7d1c6a1,c29d241e,21ddcdc9,12a9e903,39e026c9,606b0dda,38d50e09,582152eb,f0cba4ae,3b6c1f5d,7e0ccccf,3a171ecb,25c83c98,e5ba7672,11622d5f,2263e94f,5840adea,800354a7,aa5f0a15,454bf5f0,ffc3da20,cf4fb6b3,a98bc52f,60bc0f35,c851b930,b28479f6,001f3601,07a8c672,cb518868,fbd032d8,,7cc72ec2,0.0,0.0,1,0,0.0,0,1,23.0,,7443700.0,,,23,744370.0,1.0,1,,1.0,1.0,0.0,1,,744370 53 | 0,42b3012c,68fd1e64,062b5529,db741cd7,cfa87f80,e7d1c6a1,4e661096,21ddcdc9,b22a7d4a,39e026c9,606b0dda,38d50e09,e30eff7f,0e056148,c61ef534,7e0ccccf,3a171ecb,25c83c98,e5ba7672,169de313,2263e94f,5840adea,d50ef305,8e108113,bff9df05,ffc3da20,cf4fb6b3,a98bc52f,8bd4620d,c851b930,b28479f6,001f3601,07a8c672,cb518868,fbd032d8,,7cc72ec2,0.0,0.0,1,0,0.0,0,1,23.0,,7443700.0,,,23,744370.0,1.0,1,,1.0,1.0,10.0,1,,744370 54 | 0,42b3012c,7d35c0db,062b5529,1195884a,cfa87f80,4fee36e2,7f7c244b,5c4693e8,b22a7d4a,39e026c9,ba8fd52a,38d50e09,582152eb,f0cba4ae,c61ef534,7e0ccccf,3a171ecb,25c83c98,e5ba7672,11622d5f,2263e94f,5840adea,d50ef305,aa5f0a15,23eaa4a0,ffc3da20,855d9575,a98bc52f,8bd4620d,c851b930,b28479f6,001f3601,aa0c1468,cb518868,fbd032d8,dc6773ae,7cc72ec2,10.0,0.0,1,0,0.0,0,1,23.0,,7443700.0,,,23,744370.0,1.0,1,,529.0,1.0,10.0,1,,744370 55 | 0,a4462113,7d35c0db,0b153874,729b1c08,51d11566,99d2fb3c,16928c0a,5c4693e8,d91160b5,d53dc20a,ca6210e5,5f71999f,0a106e05,27d1baee,8622388b,fe6b92e5,c7dc6720,25c83c98,27c07bd6,1f54337f,6984f1a5,5840adea,3e1aecd1,f5422ec9,bf1389ab,02180dd0,b56dc09e,837d93f2,b576f089,eb9e7931,07d13a8f,ea9a246c,aa0c1468,38100ca1,8c882225,,7cc72ec2,3.0,0.0,3,1,1.0,1,3,35.0,,0.0,,0,2,28036.0,3.0,103,69,70.0,36.0,0.09090909090909091,3,33,27967 56 | 0,8b556421,68fd1e64,0b153874,729b1c08,51d11566,99d2fb3c,2d85ec63,5c4693e8,0d6bbd01,51ef04f7,ca6210e5,287130e0,0a106e05,0e056148,8622388b,7f809d5c,2c892c66,25c83c98,27c07bd6,1f54337f,727d8baf,5840adea,3e1aecd1,f5422ec9,2f532987,02180dd0,855d9575,830c881e,60bc0f35,eb9e7931,07d13a8f,ea9a246c,55e7dbfa,42ba86e2,8c882225,,7cc72ec2,3.0,0.0,3,1,1.0,1,3,35.0,,0.0,,0,2,28036.0,3.0,103,69,70.0,36.0,0.09090909090909091,3,33,27967 57 | 0,e6c5b5cd,05db9164,0b153874,c6b1e1b2,df77d795,3b08e48b,f35f84c1,21ddcdc9,f089a093,51ef04f7,606b0dda,f0cf0024,e30eff7f,8d51ec69,99c09e97,fbad5c96,32c7478e,4cf72387,ea2ea347,169de313,fb8fab62,5840adea,3ce91fbf,3135c89d,de626a4a,d3ab08b1,7295b1d6,df5886ca,60bc0f35,3e5f0580,b28479f6,ea9a246c,5422407f,1ae89522,335a6a1e,dc6773ae,a79d9eee,0.0,0.0,,0,0.0,0,3,9.0,,465580.0,,,0,46558.0,0.0,86,,0.0,12.0,0.0,,9,46558 58 | 0,e6c5b5cd,05db9164,0b153874,c6b1e1b2,df77d795,4fee36e2,454bf5f0,21ddcdc9,b38d0d51,51ef04f7,606b0dda,f0cf0024,b04e4670,8d51ec69,99c09e97,fbad5c96,32c7478e,982fa226,776ce399,169de313,fb8fab62,5840adea,3ce91fbf,3135c89d,92e5d191,d3ab08b1,7295b1d6,df5886ca,74e1a23a,b9ec9192,b28479f6,ea9a246c,aa0c1468,1ae89522,335a6a1e,,7cc72ec2,0.0,0.0,,0,0.0,0,3,9.0,,0.0,,,0,46558.0,0.0,86,,90.0,12.0,0.0,,9,46558 59 | 0,e6c5b5cd,7d35c0db,d6b9e7c1,1195884a,df77d795,3b08e48b,1f6f0bb6,5c4693e8,b22a7d4a,51ef04f7,606b0dda,5f71999f,b04e4670,e30839e1,99c09e97,fbad5c96,2c892c66,4cf72387,776ce399,9a6888fb,6984f1a5,5840adea,3ce91fbf,8e108113,964b9b5c,d3ab08b1,7295b1d6,df5886ca,74e1a23a,b9ec9192,0bf4d2c2,ea9a246c,aa0c1468,1ae89522,335a6a1e,,7cc72ec2,0.0,0.0,,0,0.0,0,3,9.0,,0.0,,,0,46558.0,0.0,86,,90.0,12.0,0.0,,9,46558 60 | 0,7e319349,05db9164,062b5529,58ba3baa,e7e2fcab,cbc7bc74,2d85ec63,21ddcdc9,6744fd81,51ef04f7,606b0dda,287130e0,a10c0817,e30839e1,a1d8444c,7f809d5c,2c892c66,4cf72387,8efede7f,1e61a878,6984f1a5,a458ea53,1e7a094c,da623bae,b21f6f64,59411f40,065f1d5a,7db6a946,de56deb0,9f423ce8,07d13a8f,ea9a246c,55e7dbfa,53b061eb,fd323779,,a79d9eee,0.0,0.0,2,0,0.0,0,2,12.0,,0.0,,,10,282350.0,2.0,4,,1.2,4.0,1.0,2,2,282350 61 | 0,7e319349,05db9164,062b5529,58ba3baa,eca13bf1,cbc7bc74,4e661096,21ddcdc9,a0715032,51ef04f7,606b0dda,287130e0,e30eff7f,8d51ec69,a1d8444c,fbad5c96,32c7478e,4cf72387,8efede7f,1e61a878,45eb0bf4,a458ea53,d50ef305,da623bae,47b48dcf,59411f40,855d9575,7db6a946,de56deb0,9f423ce8,07d13a8f,ea9a246c,55e7dbfa,53b061eb,5d6b89d0,,7cc72ec2,0.0,0.0,2,0,0.0,0,2,12.0,,0.0,,,10,282350.0,2.0,4,,1.2,4.0,4.0,2,2,282350 62 | 1,2c14c412,05db9164,c8ddd494,1195884a,e7e2fcab,3b08e48b,44fae045,338f20de,30a21a4a,12c8037e,7bc22f00,8947f767,bd17c3da,0e056148,7557c505,7e0ccccf,c7dc6720,982fa226,3486227d,169de313,a92f76a2,b1252a9d,bb3b7ab9,968f2b7d,80a17bc7,71779e59,d7e60bf8,830c881e,77d8f55a,3e5f0580,07d13a8f,cc651ac8,aa0c1468,d343fc4c,fd323779,,a73ee510,0.0,0.0,,8,95.0,95,1,9.0,,0.0,,0,3,7547.0,0.0,489,54,3.0,7.0,0.0,,6,7493 63 | 1,2c14c412,05db9164,c8ddd494,75212aaa,ed1f9dbe,3b08e48b,16928c0a,338f20de,2f117690,12c8037e,7bc22f00,8947f767,bd17c3da,8d51ec69,7557c505,7f809d5c,c7dc6720,25c83c98,3486227d,f85c6e4a,a92f76a2,b1252a9d,bb3b7ab9,968f2b7d,2f532987,71779e59,855d9575,3a9dafb8,77d8f55a,90b202b5,07d13a8f,010f6491,bc523842,38100ca1,786a0db5,,a73ee510,0.0,0.0,,8,95.0,95,1,9.0,,74930.0,,0,3,7547.0,0.0,489,54,27.0,7.0,0.0,,6,7493 64 | 1,2eef01bc,05db9164,d6b9e7c1,e2e2fcd9,eb403f2b,3b08e48b,1f6f0bb6,,c9a2b10e,c8afb369,ba8fd52a,270cc1b8,e30eff7f,f0cba4ae,30bb890d,7e0ccccf,2c892c66,982fa226,8efede7f,169de313,f256936e,,468a0854,,bf1389ab,9fbf259b,a5a24cda,830c881e,60bc0f35,a60de4e5,0bf4d2c2,,a5957347,69f391ec,b34f3128,,a79d9eee,0.2,6.0,0,2,10.0,5,2,49.0,2,5.0,5,4,47,2.0,6.0,1,1,2303.0,4.0,0.5,1,2,1 65 | 1,2eef01bc,05db9164,5b392875,e2e2fcd9,e7e2fcab,3b08e48b,f35f84c1,,1ae2ab99,c8afb369,c8afb369,270cc1b8,86b4c7aa,0e056148,30bb890d,7e0ccccf,32c7478e,4cf72387,8efede7f,f922efad,f256936e,,d50ef305,,691f5737,9fbf259b,a5a24cda,605bbc24,0e0c81f5,a60de4e5,b28479f6,,a5957347,69f391ec,fd323779,,a73ee510,5.0,6.0,0,2,10.0,5,2,49.0,2,0.2,5,4,47,2.0,6.0,1,1,2303.0,4.0,0.5,1,2,1 66 | 1,a4462113,05db9164,d6b9e7c1,1195884a,eb403f2b,3b08e48b,a7dd2259,,7f4b7766,c8afb369,c8afb369,270cc1b8,86b4c7aa,27d1baee,3b6c1f5d,7f809d5c,32c7478e,4cf72387,8efede7f,f922efad,f256936e,,468a0854,,a7dd2259,9fbf259b,a5a24cda,605bbc24,0e0c81f5,a60de4e5,b28479f6,,a5957347,38100ca1,b34f3128,,a73ee510,0.2,6.0,0,2,10.0,5,2,49.0,2,5.0,5,4,47,2.0,6.0,1,1,2303.0,4.0,0.5,1,2,1 67 | 1,2eef01bc,05db9164,5b392875,e2e2fcd9,e7e2fcab,4fee36e2,145d36f0,5c4693e8,4d1ce2d4,c8afb369,ba8fd52a,270cc1b8,86b4c7aa,454bf5f0,30bb890d,7e0ccccf,32c7478e,4cf72387,8efede7f,f922efad,f256936e,,d50ef305,8e108113,3f95d319,9fbf259b,a5a24cda,830c881e,0e0c81f5,3e5f0580,b28479f6,,a5957347,69f391ec,fd323779,dc6773ae,a73ee510,0.2,6.0,0,2,10.0,5,2,49.0,2,5.0,5,4,47,2.0,6.0,1,1,1.0425531914893618,4.0,2.0,1,2,1 68 | 0,d83fb924,241546e0,0b153874,e35e50d2,68956606,77c84e7c,1f6f0bb6,21ddcdc9,ea8f1bdb,ea144390,ba8fd52a,5f71999f,c21c3e4c,f0cba4ae,3b6c1f5d,7f809d5c,3a171ecb,982fa226,e5ba7672,280d8784,6984f1a5,a458ea53,5cfda1ab,9726a061,bf1389ab,8a71c3b2,855d9575,034e5f3b,60bc0f35,3e5f0580,051219e6,9b3e8820,18cc11b4,d4d03c10,d1c0e82c,,a73ee510,2508.0,1.0,42,4,41.0,38,1,11.0,1,120.0,3,,1,83.0,69.0,8778,43,11.0,11.0,6.6,66,10,40 69 | 0,d83fb924,241546e0,0b153874,e35e50d2,e7e2fcab,4fee36e2,5b56befb,21ddcdc9,5cf1878b,d53dc20a,606b0dda,58e67aaf,c21c3e4c,0e056148,74cdd32f,,2c892c66,25c83c98,ea2ea347,280d8784,33f3e185,202a4552,5cfda1ab,9726a061,3f95d319,8a71c3b2,8bb1fff1,034e5f3b,346c8a7c,720446f5,0bf4d2c2,9b3e8820,18cc11b4,d4d03c10,fd323779,,a73ee510,1.736842105263158,1.0,42,4,41.0,38,1,11.0,1,13.333333333333334,3,,1,83.0,69.0,8778,43,11.0,11.0,6.6,66,10,40 70 | 0,bffbd637,5bfa8ab5,37e4aa92,0cf93790,ee2535a7,451bd4e4,0b1925b7,5c4693e8,38485e06,c8afb369,c8afb369,08d6d899,bbf70d82,f0cba4ae,19c4255f,7e0ccccf,c7dc6720,f281d2a7,e5ba7672,cbe73052,0a8dfe71,,d50ef305,,c1520d76,ba626733,a7f6a7c4,39e37558,60bc0f35,e92f89be,b28479f6,cc651ac8,970d1f97,253d17f9,23f3a8f5,,a73ee510,0.0,0.0,17,,0.0,,24,60.0,,0.0,,,0,21916.0,17.0,,,0.0,84.0,1020.0,17,60,21916 71 | 0,a4462113,5bfa8ab5,37e4aa92,0cf93790,ee2535a7,451bd4e4,292ffaec,,b22a7d4a,d53dc20a,ba8fd52a,08d6d899,e30eff7f,27d1baee,19c4255f,7e0ccccf,c7dc6720,982fa226,ea2ea347,cbe73052,6984f1a5,,54a5ff49,8e108113,e5e1285c,dfaa9d42,a7f6a7c4,39e37558,60bc0f35,e92f89be,b28479f6,,970d1f97,38100ca1,23f3a8f5,,a73ee510,170.0,0.0,17,,0.0,,24,60.0,,219160.0,,,0,21916.0,17.0,,,600.0,84.0,1020.0,17,60,21916 72 | 0,bffbd637,7d35c0db,d6b9e7c1,0cf93790,ee2535a7,451bd4e4,2f532987,5c4693e8,8af1ee39,c8afb369,c8afb369,08d6d899,bbf70d82,8d51ec69,19c4255f,7e0ccccf,c7dc6720,982fa226,e5ba7672,169de313,0a8dfe71,202a4552,54a5ff49,8e108113,4e661096,dfaa9d42,a7f6a7c4,830c881e,c7d75642,e92f89be,b28479f6,,aa0c1468,253d17f9,23f3a8f5,,a73ee510,0.0,0.0,17,,0.0,,24,60.0,,0.0,,,0,21916.0,17.0,,,600.0,84.0,1020.0,17,60,21916 73 | 0,bffbd637,5bfa8ab5,d6b9e7c1,0cf93790,ee2535a7,451bd4e4,f0cba4ae,5c4693e8,3a3c7c97,c8afb369,c8afb369,08d6d899,bbf70d82,e30839e1,19c4255f,7f809d5c,2c892c66,f281d2a7,e5ba7672,cbe73052,0a8dfe71,,54a5ff49,8e108113,740f6cf6,dfaa9d42,a7f6a7c4,39e37558,c7d75642,e92f89be,b28479f6,,970d1f97,253d17f9,23f3a8f5,dc6773ae,a73ee510,0.0,0.0,17,,0.0,,24,60.0,,219160.0,,,0,21916.0,17.0,,,0.0,84.0,1020.0,17,60,21916 74 | 0,a4462113,05db9164,0b153874,776f5665,9b6c1c1d,4fee36e2,b7fc5f30,,f0dff54c,c8afb369,c8afb369,e112a9de,a7cf409e,454bf5f0,5c7c443c,fbad5c96,32c7478e,4cf72387,e5ba7672,169de313,252162ec,202a4552,c823e712,,5c6c77b7,e25984c4,ba5e4f10,062065a8,2a7fa4bc,9ad95d23,1adce6ef,,60261742,6d37db5a,8f079aa5,,a73ee510,168.0,0.0,3,2,56.0,56,9,6.0,,31960.0,,0,6,3293.0,3.0,140,97,1.0,9.0,30.0,3,,3196 75 | 0,11da3cff,7d35c0db,d6b9e7c1,776f5665,9b6c1c1d,2562fe66,2f532987,5c4693e8,b8a3bd43,c8afb369,ba8fd52a,e112a9de,a7cf409e,27d1baee,5c7c443c,fbad5c96,32c7478e,982fa226,e5ba7672,169de313,6984f1a5,202a4552,c823e712,,bf1389ab,e25984c4,ba5e4f10,062065a8,2a7fa4bc,9ad95d23,0bf4d2c2,,aa0c1468,6d37db5a,8f079aa5,,a73ee510,168.0,0.0,3,2,56.0,56,9,6.0,,0.0,,0,6,3293.0,3.0,140,97,1.0,9.0,0.0,3,,3196 76 | 1,03259d67,05db9164,d6b9e7c1,4a09aba8,53ebcd54,3b08e48b,e30839e1,,b22a7d4a,c8afb369,c8afb369,7008ef6d,d1c83925,e30839e1,e9ba6c16,7e0ccccf,55dd3565,4cf72387,e5ba7672,169de313,b35573cd,,42f629ee,,6d4e1c2b,59d37952,a3bfe8d4,c2e887fc,68d09113,ab60c4de,0bf4d2c2,,bf07ea52,5951b775,cf004f46,,a73ee510,90.0,0.0,9,0,0.0,0,5,40.0,0,86150.0,0,,35,8615.0,9.0,6,,1400.0,10.0,45.0,9,5,8615 77 | 1,03259d67,05db9164,37e4aa92,1195884a,53ebcd54,3b08e48b,e30839e1,,8b1ee53f,d53dc20a,c8afb369,7008ef6d,d1c83925,27d1baee,e9ba6c16,7f809d5c,55dd3565,982fa226,ea2ea347,169de313,b35573cd,,42f629ee,8e108113,bf1389ab,59d37952,a3bfe8d4,830c881e,68d09113,3e5f0580,0bf4d2c2,,bf07ea52,5951b775,cf004f46,,a73ee510,90.0,0.0,9,0,0.0,0,5,40.0,0,86150.0,0,,35,8615.0,9.0,6,,1.1428571428571428,10.0,1.8,9,5,8615 78 | 1,a4462113,f473b8dc,5b392875,9fc13a66,e7e2fcab,3b08e48b,1f6f0bb6,5c4693e8,f0a600dd,d53dc20a,606b0dda,a5b69ae3,e30eff7f,27d1baee,fb6ba62c,fbad5c96,55dd3565,25c83c98,1e88c74f,edf9805c,6984f1a5,5840adea,276c620c,074bb89f,bf1389ab,c1b3ee11,ef33c488,a0874a81,60bc0f35,9ffb3655,0bf4d2c2,cc651ac8,a1893fb8,b1605555,a3bf6bd4,,7cc72ec2,1.0,0.0,1,1,1.0,1,1,20.0,,0.0,,,17,10324.0,1.0,1,37,1.1764705882352942,4.0,0.3333333333333333,1,3,10287 79 | 1,64748fbe,f473b8dc,5b392875,9fc13a66,119ad24d,3b08e48b,66606852,21ddcdc9,568fc120,a2f90ea3,ba8fd52a,a5b69ae3,a1654f4f,27d1baee,fb6ba62c,fbad5c96,2c892c66,25c83c98,1e88c74f,edf9805c,f4ff6477,5840adea,276c620c,074bb89f,2f532987,c1b3ee11,ef33c488,a0874a81,60bc0f35,9ffb3655,07d13a8f,2bf691b1,aa0c1468,b1605555,a3bf6bd4,,7cc72ec2,1.0,0.0,1,1,1.0,1,1,20.0,,102870.0,,,17,10324.0,1.0,1,37,1.1764705882352942,4.0,3.0,1,3,10287 80 | 0,9538b54b,be589b51,d6b9e7c1,1195884a,28235b62,c5df96d3,3f099307,21ddcdc9,c1734d80,51ef04f7,606b0dda,d7988e72,0f2f9850,454bf5f0,65a56438,fe6b92e5,32c7478e,25c83c98,e5ba7672,e5d4c5ff,43994d95,b1252a9d,22fd2464,2a4caf7c,61ceaaa1,530f4d82,19175895,830c881e,f09d675d,d9085127,1adce6ef,ea9a246c,81b63bac,a8548b17,6c1cdd05,,a73ee510,0.0,0.0,,2,4.0,4,8,39.0,0,0.0,0,,38,2851.0,0.0,39,35,1.0263157894736843,9.0,0.0,,1,2816 81 | 0,9538b54b,be589b51,5b392875,f327e411,28235b62,c5df96d3,c2f02320,21ddcdc9,6dd8e619,51ef04f7,606b0dda,d7988e72,0f2f9850,0e056148,65a56438,fe6b92e5,32c7478e,25c83c98,e5ba7672,e5d4c5ff,43994d95,202a4552,22fd2464,2a4caf7c,4b190d01,530f4d82,19175895,ef7e2c01,f09d675d,d9085127,1adce6ef,ea9a246c,81b63bac,a8548b17,6c1cdd05,,a73ee510,0.0,0.0,,2,4.0,4,8,39.0,0,0.0,0,,38,2851.0,0.0,39,35,1.0263157894736843,9.0,0.0,,1,2816 82 | 0,9538b54b,be589b51,5b392875,1195884a,28235b62,c5df96d3,b7fc5f30,21ddcdc9,3a6ffbe2,51ef04f7,606b0dda,d7988e72,0f2f9850,27d1baee,3b6c1f5d,fe6b92e5,2c892c66,25c83c98,ea2ea347,e5d4c5ff,43994d95,b1252a9d,d50ef305,2a4caf7c,fdd449cf,ba626733,19175895,ef7e2c01,f09d675d,3e5f0580,1adce6ef,cc651ac8,aa0c1468,38100ca1,6c1cdd05,dc6773ae,a73ee510,0.0,0.0,,2,4.0,4,8,39.0,0,28160.0,0,,38,2851.0,0.0,39,35,1482.0,9.0,0.0,,1,2816 83 | 0,bfef54b3,05db9164,0b153874,bad5ee18,ff610a82,2462946f,2ad7df96,,b22a7d4a,c8afb369,c8afb369,08d6d899,87c6f83c,e30839e1,0429f84b,7e0ccccf,32c7478e,25c83c98,e5ba7672,f56b7dd5,ae1bb660,,38eb9cf4,,80a17bc7,03036cd1,f2bec40d,46f42a63,9143c832,7f8ffe57,b28479f6,,970d1f97,0aae6782,c0d61a5c,,a73ee510,0.8,0.0,19,1,5.0,5,25,18.0,,27900.0,,0,5,3445.0,4.0,230,655,3.6,38.0,52.0,4,13,2790 84 | 0,bfef54b3,05db9164,0b153874,bad5ee18,ff610a82,2462946f,c9cd0b01,,4229bcd7,c8afb369,c8afb369,08d6d899,87c6f83c,0e056148,0429f84b,7e0ccccf,32c7478e,25c83c98,e5ba7672,f56b7dd5,6984f1a5,202a4552,38eb9cf4,,52cb4ca3,03036cd1,f2bec40d,46f42a63,60bc0f35,7f8ffe57,b28479f6,,970d1f97,0aae6782,c0d61a5c,dc6773ae,a73ee510,0.8,0.0,19,1,5.0,5,25,18.0,,27900.0,,0,5,3445.0,4.0,230,655,3.6,38.0,52.0,4,13,2790 85 | 0,bfef54b3,05db9164,0b153874,bad5ee18,e7e2fcab,2462946f,2f532987,,b22a7d4a,c8afb369,c8afb369,08d6d899,87c6f83c,454bf5f0,0429f84b,7e0ccccf,32c7478e,25c83c98,e5ba7672,f56b7dd5,ae1bb660,,38eb9cf4,,2ac95885,03036cd1,855d9575,830c881e,9143c832,7f8ffe57,0bf4d2c2,cc651ac8,aa0c1468,0aae6782,c0d61a5c,dc6773ae,a73ee510,20.0,0.0,19,1,5.0,5,25,18.0,,27900.0,,0,5,3445.0,4.0,230,655,90.0,38.0,0.3076923076923077,4,13,2790 86 | 0,856048a0,7d35c0db,0b153874,1195884a,38537aaf,ede207dc,b7fc5f30,5c4693e8,1f463f5a,d53dc20a,606b0dda,6496eea0,fba30a05,0e056148,60f6221e,fbad5c96,3a171ecb,982fa226,e5ba7672,41274cd7,623049e6,202a4552,c1225605,8e108113,7bcc0d31,a9be9f4c,8b2591a7,7eaf6f1a,60bc0f35,3e5f0580,0bf4d2c2,cc651ac8,4b53e449,5fca948f,43f13e8b,,a73ee510,0.7142857142857143,0.0,10,2,14.0,14,42,54.0,0,0.0,0,,1,1551.0,10.0,63,63,54.0,95.0,0.18867924528301888,10,53,1488 87 | 0,856048a0,05db9164,0b153874,c92f3b61,38537aaf,ede207dc,b490f28f,21ddcdc9,f505b1f3,51ef04f7,606b0dda,6496eea0,fba30a05,8d51ec69,60f6221e,fbad5c96,3a171ecb,25c83c98,ea2ea347,41274cd7,623049e6,5840adea,c1225605,731c3655,47b48dcf,a9be9f4c,8b2591a7,7eaf6f1a,6f67f7e5,f29b9ed2,b28479f6,ea9a246c,4b53e449,5fca948f,43f13e8b,,a73ee510,0.7142857142857143,0.0,10,2,14.0,14,42,54.0,0,0.0,0,,1,1551.0,10.0,63,63,54.0,95.0,0.18867924528301888,10,53,1488 88 | 0,a4462113,87552397,0b153874,3273d44f,e7e2fcab,7cb8561e,145d36f0,21ddcdc9,b22a7d4a,d53dc20a,606b0dda,5f71999f,642f2610,27d1baee,7439aef4,7e0ccccf,423fab69,25c83c98,d4bb7bd8,13508380,d57d87da,202a4552,7fcb1ce3,c84c4aec,bf1389ab,a0be16e2,aed6d6e2,158b69db,cd894e99,13429d76,07d13a8f,2bf691b1,aa0c1468,38100ca1,fd323779,,a73ee510,3.0,0.0,9,1,1.0,1,12,228.0,0,41180.0,0,,227,4200.0,3.0,30,82,1.0044052863436124,13.0,3.0,3,1,4118 89 | 0,3b2d8705,7d35c0db,0b153874,1195884a,141495af,7cb8561e,292ffaec,5c4693e8,b22a7d4a,a2f90ea3,ba8fd52a,2a69d406,642f2610,27d1baee,7439aef4,7f809d5c,423fab69,25c83c98,d4bb7bd8,13508380,d57d87da,202a4552,7fcb1ce3,c84c4aec,c2e7b0c7,a0be16e2,aed6d6e2,158b69db,60bc0f35,13429d76,07d13a8f,2bf691b1,5ac1608d,e147d174,fd323779,dc6773ae,a79d9eee,3.0,0.0,9,1,1.0,1,12,228.0,0,41180.0,0,,227,4200.0,3.0,30,82,51756.0,13.0,3.0,3,1,4118 90 | 1,a4462113,05db9164,0b153874,1195884a,58b33448,4fee36e2,1f6f0bb6,,13d6964e,c8afb369,c8afb369,5f71999f,1f8f8372,8d51ec69,c8aa5128,7e0ccccf,423fab69,25c83c98,e5ba7672,76196e95,6984f1a5,,9f525672,,2f532987,519aa7f5,0ce78127,9cab1003,feabcd68,843d8639,b28479f6,,c9e3eb0c,2b6c9bee,b258af68,dc6773ae,a73ee510,0.0,0.0,,4,16.0,16,4,-1.0,0,57480.0,0,,-1,5784.0,0.0,122,36,-10.0,4.0,0.0,,,5748 91 | 1,17194aa4,05db9164,0b153874,96de5913,58b33448,1e2ab9fa,0e056148,,b22a7d4a,c8afb369,c8afb369,38a947a1,1f8f8372,0e056148,c8aa5128,7e0ccccf,423fab69,25c83c98,e5ba7672,76196e95,0eed3d39,,9f525672,,1d08f649,519aa7f5,855d9575,9cab1003,feabcd68,843d8639,b28479f6,,c9e3eb0c,2b6c9bee,fd323779,,a73ee510,0.0,0.0,,4,16.0,16,4,-1.0,0,0.0,0,,-1,5784.0,0.0,122,36,1.0,4.0,0.0,,,5748 92 | 1,17194aa4,05db9164,0b153874,96de5913,58b33448,1e2ab9fa,2ad7df96,,31815209,c8afb369,c8afb369,38a947a1,1f8f8372,0e056148,c8aa5128,7e0ccccf,423fab69,25c83c98,e5ba7672,76196e95,0eed3d39,,9f525672,,2ac95885,519aa7f5,0ce78127,9cab1003,feabcd68,843d8639,b28479f6,,aa0c1468,2b6c9bee,b258af68,,a73ee510,0.0,0.0,,4,16.0,16,4,-1.0,0,0.0,0,,-1,5784.0,0.0,122,36,-10.0,4.0,0.0,,,5748 93 | 0,cb7f4e45,87552397,51d76abe,f8b34416,4505748d,3b08e48b,2f532987,,b22a7d4a,ec9909f0,ec9909f0,38a947a1,e5f8f18f,27d1baee,3b6c1f5d,fbad5c96,3a171ecb,25c83c98,776ce399,6a14f9b9,37c82b83,,c7ab64f5,,5ea79608,ba626733,3f85ec9d,31b42deb,60bc0f35,3e5f0580,07d13a8f,,c9e3eb0c,38100ca1,b34f3128,c9d4222a,a79d9eee,1.0,0.0,1,1,1.0,1,2,13.0,0,2790.0,0,,12,400.0,1.0,12,121,1.0833333333333333,3.0,1.0,1,1,279 94 | 0,cb7f4e45,87552397,51d76abe,f8b34416,4505748d,3b08e48b,145d36f0,,50325762,ec9909f0,ba8fd52a,38a947a1,e5f8f18f,454bf5f0,f3ddd519,fbad5c96,3a171ecb,25c83c98,776ce399,6a14f9b9,37c82b83,,c7ab64f5,,f014ef14,ba626733,3f85ec9d,31b42deb,6e7bf45e,636405ac,07d13a8f,,c9e3eb0c,c32f5cfe,b34f3128,c9d4222a,a73ee510,1.0,0.0,1,1,1.0,1,2,13.0,0,2790.0,0,,12,400.0,1.0,12,121,1.0833333333333333,3.0,1.0,1,1,279 95 | 0,cb7f4e45,87552397,51d76abe,1195884a,e7e2fcab,3b08e48b,145d36f0,,4defb42f,ec9909f0,ba8fd52a,38a947a1,e30eff7f,27d1baee,f3ddd519,fbad5c96,3a171ecb,25c83c98,776ce399,169de313,6984f1a5,,d50ef305,,03cd6fcf,207c5585,3f85ec9d,31b42deb,6e7bf45e,636405ac,07d13a8f,cc651ac8,c9e3eb0c,c32f5cfe,b34f3128,c9d4222a,a73ee510,1.0,0.0,1,1,1.0,1,2,13.0,0,0.0,0,,12,400.0,1.0,12,121,1.0833333333333333,3.0,1.0,1,1,279 96 | 0,cb7f4e45,87552397,51d76abe,1195884a,4505748d,4fee36e2,454bf5f0,,3c95a558,d53dc20a,ba8fd52a,38a947a1,e5f8f18f,8d51ec69,f3ddd519,fbad5c96,3a171ecb,982fa226,776ce399,169de313,6984f1a5,,d50ef305,,0d1b81b1,207c5585,855d9575,830c881e,60bc0f35,636405ac,07d13a8f,,aa0c1468,c32f5cfe,b34f3128,c9d4222a,a73ee510,1.0,0.0,1,1,1.0,1,2,13.0,0,0.0,0,,12,400.0,1.0,12,121,1.0833333333333333,3.0,1.0,1,1,279 97 | 0,df2f73e9,75ac2fe6,0b153874,7e53905f,500e47d8,1fcfcb5a,e30839e1,5b885066,4302449c,39e026c9,db46b16e,4f25e98b,bc5a0ff7,0e056148,52a58c2a,7e0ccccf,c7dc6720,25c83c98,e5ba7672,169de313,34402deb,a458ea53,dc2b40a4,aee7bde0,23eaa4a0,ba626733,855d9575,7edc047a,693641ff,6685ea28,b28479f6,001f3601,9b869d9e,ed361a81,fd323779,dc6773ae,a79d9eee,180.0,0.0,5,3,58.0,36,5,11.0,0,66.36363636363636,22,,3,1497.0,27.0,77,37,3.6666666666666665,13.0,40.0,5,8,1460 98 | 0,df2f73e9,75ac2fe6,0b153874,7e53905f,500e47d8,1fcfcb5a,16928c0a,5b885066,450c3775,d53dc20a,db46b16e,4f25e98b,bc5a0ff7,27d1baee,52a58c2a,7e0ccccf,c7dc6720,25c83c98,e5ba7672,386335c9,34402deb,a458ea53,dc2b40a4,aee7bde0,16928c0a,77b47863,5c7cbfe3,7edc047a,693641ff,6685ea28,b28479f6,001f3601,9b869d9e,ed361a81,e20d69d0,,a73ee510,0.1388888888888889,0.0,5,3,58.0,36,5,11.0,0,66.36363636363636,22,,3,1497.0,27.0,77,37,33.0,13.0,40.0,5,8,1460 99 | 1,2547a09c,05db9164,0b153874,1195884a,16830cfc,3b08e48b,1f6f0bb6,5c4693e8,437a3cd1,c8afb369,c8afb369,fc1fa80d,f68751cd,27d1baee,e27d6c43,,32c7478e,25c83c98,e5ba7672,169de313,902ac8b1,,d57680ee,,5d36cd8e,133b246d,855d9575,c7b0ae82,bdcbd9e1,b0a049a6,07d13a8f,cc651ac8,ff266d62,9c52ea07,fd323779,,a79d9eee,60.0,1.0,12,2,6.0,5,12,13.0,1,23.0,1,0,1,35.0,13.0,21,12,13.0,24.0,144.0,12,12,23 100 | 1,2547a09c,05db9164,0b153874,ff5a027e,16830cfc,3b08e48b,1f6f0bb6,,4c1df573,c8afb369,c8afb369,fc1fa80d,f68751cd,8d51ec69,e27d6c43,,32c7478e,25c83c98,e5ba7672,45e7b9c6,902ac8b1,,d57680ee,,b35eaacc,133b246d,5bb92b91,c7b0ae82,bdcbd9e1,b0a049a6,07d13a8f,,ff266d62,9c52ea07,1793a828,,a73ee510,60.0,1.0,12,2,6.0,5,12,13.0,1,23.0,1,0,1,35.0,13.0,21,12,13.0,24.0,1.0,12,12,23 101 | 0,310d155b,7d35c0db,5b392875,1195884a,e7e2fcab,a567fd47,4e661096,315ba0e1,8d312796,f503ed62,ba8fd52a,5f71999f,891589e7,0e056148,3b6c1f5d,7f809d5c,bcdee96c,982fa226,e5ba7672,c1c01bb6,8215e1d6,a458ea53,e798d858,49d68486,21763d48,55779fb8,ee70ec24,bc1e82c6,4131161a,e3baf8d4,1adce6ef,cc651ac8,aa0c1468,c10a095d,3fdb382b,8ec974f4,a79d9eee,1.0,1.0,8,3,6.0,6,21,20.0,0,0.0,0,1,2,1968.0,6.0,29,26,40.0,39.0,108.0,6,18,1942 -------------------------------------------------------------------------------- /fast-ftrl-proximal/src/test/resources/test-small.vw: -------------------------------------------------------------------------------- 1 | -1 |C1 1005 |site_id f86ed51c |site_domain 68e3e4e4 |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip aabf5136 |device_model 9c7b6730 |device_type 1 |device_conn_type 0 |C14 22561 |C15 320 |C16 50 |C17 1863 |C18 3 |C19 39 |C20 -1 2 | 1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 30b38828 |device_model 6569eeb3 |device_type 1 |device_conn_type 0 |C14 23171 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 -1 3 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 3005106b |device_model cbb77256 |device_type 1 |device_conn_type 3 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100177 4 | -1 |C1 1005 |site_id ee8b8550 |site_domain f415c8a8 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 9dc3139c |device_model 5096d134 |device_type 1 |device_conn_type 0 |C14 23440 |C15 728 |C16 90 |C17 2685 |C18 1 |C19 33 |C20 -1 5 | -1 |C1 1005 |site_id 6c5b482c |site_domain 7687a86e |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 94f75d17 |device_model e9b8d8d7 |device_type 1 |device_conn_type 0 |C14 17653 |C15 300 |C16 250 |C17 1994 |C18 2 |C19 39 |C20 100084 6 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id e2fcccd2 |app_domain 5c5a694b |app_category 0f2161f8 |device_id a99f214a |device_ip 25eb4b24 |device_model 034b811d |device_type 1 |device_conn_type 0 |C14 20632 |C15 320 |C16 50 |C17 2374 |C18 3 |C19 39 |C20 -1 7 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id e2fcccd2 |app_domain 5c5a694b |app_category 0f2161f8 |device_id a99f214a |device_ip 5e6c2516 |device_model 8f5c9827 |device_type 1 |device_conn_type 0 |C14 20633 |C15 320 |C16 50 |C17 2374 |C18 3 |C19 39 |C20 -1 8 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 92f5800b |app_domain ae637522 |app_category 0f2161f8 |device_id a99f214a |device_ip 554a294a |device_model 4d1ad389 |device_type 1 |device_conn_type 2 |C14 21191 |C15 320 |C16 50 |C17 2424 |C18 1 |C19 161 |C20 100193 9 | 1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 855c8a48 |device_model 8a4875bd |device_type 1 |device_conn_type 0 |C14 23163 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 -1 10 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id ce183bbd |app_domain ae637522 |app_category cef3e649 |device_id a99f214a |device_ip 72759c19 |device_model 36b67a2a |device_type 1 |device_conn_type 0 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100105 11 | -1 |C1 1002 |site_id acc6cd52 |site_domain 219720a5 |site_category 50e219e0 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip b8e94e8d |device_model 8a332ed1 |device_type 0 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 12 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id fb7c70a3 |app_domain d9b5648e |app_category 0f2161f8 |device_id 015057b5 |device_ip 431b3174 |device_model aad45b01 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 13 | -1 |C1 1005 |site_id 5ee41ff2 |site_domain 17d996e6 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 11d4ee83 |device_model 8a4875bd |device_type 1 |device_conn_type 0 |C14 19950 |C15 320 |C16 50 |C17 1800 |C18 3 |C19 167 |C20 100077 14 | -1 |C1 1005 |site_id 6c5b482c |site_domain 7687a86e |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip e45d4907 |device_model 1f0bc64f |device_type 1 |device_conn_type 0 |C14 17654 |C15 300 |C16 250 |C17 1994 |C18 2 |C19 39 |C20 -1 15 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 194300d7 |device_model 462f0bdf |device_type 1 |device_conn_type 2 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 -1 16 | -1 |C1 1002 |site_id 9550c324 |site_domain 61db8349 |site_category 50e219e0 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id 119f55d4 |device_ip 0f32af7c |device_model 43e7b962 |device_type 0 |device_conn_type 0 |C14 23204 |C15 300 |C16 50 |C17 2672 |C18 0 |C19 35 |C20 100075 17 | 1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 77f32a19 |device_model a5bce124 |device_type 1 |device_conn_type 0 |C14 23165 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 -1 18 | -1 |C1 1005 |site_id b99a2c43 |site_domain cc962a1f |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip dadb4fb8 |device_model 158e4944 |device_type 1 |device_conn_type 0 |C14 23369 |C15 320 |C16 50 |C17 2680 |C18 3 |C19 815 |C20 100155 19 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 98ae2fed |device_model edead9f4 |device_type 1 |device_conn_type 0 |C14 23173 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 100084 20 | -1 |C1 1005 |site_id 5bcf81a2 |site_domain 9d54950b |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 61a7dc19 |device_model a5bce124 |device_type 1 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 21 | -1 |C1 1005 |site_id 0a742914 |site_domain 510bd839 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 0cff710f |device_model 76dc4769 |device_type 1 |device_conn_type 0 |C14 8330 |C15 320 |C16 50 |C17 761 |C18 3 |C19 175 |C20 100075 22 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 73206397 |app_domain 2347f47a |app_category 8ded1f7a |device_id 4a50a3bd |device_ip bdbd9785 |device_model 5db079b5 |device_type 1 |device_conn_type 2 |C14 23143 |C15 320 |C16 50 |C17 2665 |C18 0 |C19 35 |C20 -1 23 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 73206397 |app_domain 2347f47a |app_category 8ded1f7a |device_id e3ea52e4 |device_ip 1792aa05 |device_model e4a6a646 |device_type 1 |device_conn_type 0 |C14 23143 |C15 320 |C16 50 |C17 2665 |C18 0 |C19 35 |C20 -1 24 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 707d41a7 |device_model 1f0bc64f |device_type 1 |device_conn_type 2 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 25 | -1 |C1 1005 |site_id e8692a75 |site_domain 77c8e77d |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 9094077a |device_model 1f0bc64f |device_type 1 |device_conn_type 0 |C14 22742 |C15 320 |C16 50 |C17 2637 |C18 3 |C19 35 |C20 100188 26 | -1 |C1 1005 |site_id fec6e632 |site_domain ace5b8fd |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 2d4a0d55 |device_model c6263d8a |device_type 1 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 27 | -1 |C1 1005 |site_id 17caea14 |site_domain 0dde25ec |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip eaa563b2 |device_model 5ef191d3 |device_type 1 |device_conn_type 0 |C14 20251 |C15 320 |C16 50 |C17 2323 |C18 0 |C19 687 |C20 100081 28 | -1 |C1 1005 |site_id fec6e632 |site_domain ace5b8fd |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 9b4c02b8 |device_model 1ccc7835 |device_type 1 |device_conn_type 0 |C14 17239 |C15 320 |C16 50 |C17 1973 |C18 3 |C19 39 |C20 -1 29 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 0fb82f6b |device_model 8b65b455 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 30 | -1 |C1 1005 |site_id 92c7cbe7 |site_domain 6e882918 |site_category 75fa27f6 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 79a1617b |device_model 84ebbcd4 |device_type 1 |device_conn_type 0 |C14 23424 |C15 216 |C16 36 |C17 2683 |C18 1 |C19 163 |C20 -1 31 | -1 |C1 1005 |site_id 12fb4121 |site_domain 6b59f079 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 2bebab0c |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 100083 32 | -1 |C1 1005 |site_id 235ba823 |site_domain f6ebf28e |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip b1730759 |device_model e0ced982 |device_type 1 |device_conn_type 0 |C14 22741 |C15 320 |C16 50 |C17 2637 |C18 3 |C19 35 |C20 100188 33 | -1 |C1 1005 |site_id b7e9786d |site_domain b12b9f85 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 65767ebe |device_model 78d9bd10 |device_type 1 |device_conn_type 0 |C14 19771 |C15 320 |C16 50 |C17 2227 |C18 0 |C19 935 |C20 100079 34 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 6ed7183f |device_model 4ea23a13 |device_type 1 |device_conn_type 2 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100176 35 | -1 |C1 1005 |site_id b99a2c43 |site_domain cc962a1f |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 1f58b397 |device_model cbb77256 |device_type 1 |device_conn_type 0 |C14 22268 |C15 320 |C16 50 |C17 2566 |C18 3 |C19 815 |C20 100155 36 | -1 |C1 1005 |site_id a7853007 |site_domain 7e091613 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 2bb4af67 |device_model 3bb1ddd7 |device_type 1 |device_conn_type 0 |C14 22268 |C15 320 |C16 50 |C17 2566 |C18 3 |C19 815 |C20 100156 37 | -1 |C1 1005 |site_id 4b6c0684 |site_domain d65b1643 |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 0875a0ef |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 38 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 3f2a6cbb |app_domain 33da2e74 |app_category cef3e649 |device_id a99f214a |device_ip b8959ded |device_model 5b799d60 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 39 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 10e93435 |device_model ecb851b2 |device_type 1 |device_conn_type 3 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100176 40 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 75807b96 |app_domain d9b5648e |app_category cef3e649 |device_id a99f214a |device_ip 431b3174 |device_model 72a00661 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 41 | -1 |C1 1005 |site_id 9aef3962 |site_domain d47b862d |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip cc9784df |device_model 051263ee |device_type 1 |device_conn_type 0 |C14 19771 |C15 320 |C16 50 |C17 2227 |C18 0 |C19 935 |C20 100081 42 | -1 |C1 1005 |site_id 450d6400 |site_domain bea33b9a |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip a3247264 |device_model a0f5f879 |device_type 1 |device_conn_type 0 |C14 8330 |C15 320 |C16 50 |C17 761 |C18 3 |C19 175 |C20 100077 43 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id ce183bbd |app_domain ae637522 |app_category cef3e649 |device_id a99f214a |device_ip bf98ece1 |device_model 36b67a2a |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 100084 44 | 1 |C1 1005 |site_id 38217daf |site_domain 449497bc |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 3746391d |device_model 41c87f95 |device_type 1 |device_conn_type 0 |C14 20346 |C15 300 |C16 250 |C17 2331 |C18 2 |C19 39 |C20 100156 45 | 1 |C1 1005 |site_id e151e245 |site_domain 7e091613 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 7bb6ffd8 |device_model 00b1f3a7 |device_type 1 |device_conn_type 0 |C14 17264 |C15 320 |C16 50 |C17 1872 |C18 3 |C19 39 |C20 -1 46 | -1 |C1 1005 |site_id 4e430aa3 |site_domain 964a3bd1 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 514f9df0 |device_model d1cbe61f |device_type 1 |device_conn_type 0 |C14 22850 |C15 320 |C16 50 |C17 2645 |C18 3 |C19 35 |C20 -1 47 | -1 |C1 1005 |site_id 4407d527 |site_domain c1e9c6cf |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 8a014cbb |device_model 816484ae |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 48 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip bbf176fa |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 23166 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 -1 49 | -1 |C1 1005 |site_id 12fb4121 |site_domain 6b59f079 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 028b608f |device_model 6332421a |device_type 1 |device_conn_type 0 |C14 23401 |C15 320 |C16 50 |C17 2682 |C18 1 |C19 419 |C20 100084 50 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id ce183bbd |app_domain ae637522 |app_category cef3e649 |device_id a99f214a |device_ip a160e776 |device_model 36b67a2a |device_type 1 |device_conn_type 0 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100176 51 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 41b9522b |device_model 8a4875bd |device_type 1 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 52 | 1 |C1 1005 |site_id a7853007 |site_domain 7e091613 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip ec5b686a |device_model bfdc1fc9 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 53 | 1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip e35f42c2 |device_model a5bce124 |device_type 1 |device_conn_type 0 |C14 22258 |C15 320 |C16 50 |C17 2545 |C18 0 |C19 167 |C20 100083 54 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 51527e4a |device_model 9ea0eb04 |device_type 1 |device_conn_type 2 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 -1 55 | -1 |C1 1010 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 2e402503 |app_domain 7801e8d9 |app_category 0f2161f8 |device_id 3088019c |device_ip 2e33e84b |device_model 908d96b8 |device_type 4 |device_conn_type 0 |C14 18987 |C15 320 |C16 50 |C17 2158 |C18 3 |C19 291 |C20 -1 56 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id a4869716 |app_domain 2347f47a |app_category d1327cf5 |device_id a99f214a |device_ip 639adf50 |device_model 1cfbf412 |device_type 1 |device_conn_type 2 |C14 377 |C15 320 |C16 50 |C17 112 |C18 3 |C19 1319 |C20 -1 57 | -1 |C1 1005 |site_id 9e8cf15d |site_domain 0d3cb7be |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 65860f59 |device_model a5bce124 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 100084 58 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip b7750909 |device_model c6263d8a |device_type 1 |device_conn_type 3 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 59 | 1 |C1 1005 |site_id f84e52b6 |site_domain d7e2f29b |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip ac6c5a0c |device_model e9eda42b |device_type 1 |device_conn_type 0 |C14 20346 |C15 300 |C16 250 |C17 2331 |C18 2 |C19 39 |C20 -1 60 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 73206397 |app_domain 2347f47a |app_category 8ded1f7a |device_id 732b28ed |device_ip 7b9b8d78 |device_model 24f6b932 |device_type 1 |device_conn_type 0 |C14 23143 |C15 320 |C16 50 |C17 2665 |C18 0 |C19 35 |C20 -1 61 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 92f5800b |app_domain ae637522 |app_category 0f2161f8 |device_id a99f214a |device_ip f5179c21 |device_model 981edffc |device_type 1 |device_conn_type 3 |C14 21189 |C15 320 |C16 50 |C17 2424 |C18 1 |C19 161 |C20 100193 62 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 51cedd4e |app_domain aefc06bd |app_category 0f2161f8 |device_id a99f214a |device_ip a7fb5777 |device_model e3bb76b5 |device_type 1 |device_conn_type 0 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100176 63 | -1 |C1 1005 |site_id 57ef2c87 |site_domain bd6d812f |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip f92f36bc |device_model b72d1c6e |device_type 1 |device_conn_type 0 |C14 19772 |C15 320 |C16 50 |C17 2227 |C18 0 |C19 935 |C20 100079 64 | -1 |C1 1005 |site_id 89a490f5 |site_domain ce307e01 |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 90f55d3e |device_model 684581ce |device_type 1 |device_conn_type 0 |C14 20008 |C15 320 |C16 50 |C17 2283 |C18 0 |C19 163 |C20 100076 65 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 755da49e |device_model 8a4875bd |device_type 1 |device_conn_type 0 |C14 23173 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 100084 66 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 15e363b0 |device_model 4c8aeb60 |device_type 1 |device_conn_type 0 |C14 23166 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 100083 67 | 1 |C1 1005 |site_id 5ee41ff2 |site_domain 17d996e6 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 9b34f448 |device_model 64a7b95b |device_type 1 |device_conn_type 0 |C14 8994 |C15 320 |C16 50 |C17 827 |C18 3 |C19 1071 |C20 100075 68 | -1 |C1 1005 |site_id 57fe1b20 |site_domain 5b626596 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 828a2ae9 |device_model 8441e1f3 |device_type 1 |device_conn_type 0 |C14 22682 |C15 320 |C16 50 |C17 2528 |C18 0 |C19 39 |C20 100075 69 | -1 |C1 1005 |site_id 57fe1b20 |site_domain 5b626596 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip a4c3ae54 |device_model 1f0bc64f |device_type 1 |device_conn_type 0 |C14 21413 |C15 320 |C16 50 |C17 2467 |C18 2 |C19 167 |C20 100077 70 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id a4869716 |app_domain 2347f47a |app_category d1327cf5 |device_id a99f214a |device_ip e45666a8 |device_model 5b1faad5 |device_type 1 |device_conn_type 2 |C14 380 |C15 320 |C16 50 |C17 112 |C18 3 |C19 1319 |C20 -1 71 | 1 |C1 1005 |site_id 57fe1b20 |site_domain 5b626596 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip d55e0e38 |device_model be6db1d7 |device_type 1 |device_conn_type 0 |C14 19950 |C15 320 |C16 50 |C17 1800 |C18 3 |C19 167 |C20 -1 72 | -1 |C1 1002 |site_id 4307b3a7 |site_domain 3a0cd494 |site_category 50e219e0 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id 3a344b09 |device_ip 7ccb451e |device_model 0e68325a |device_type 0 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 73 | 1 |C1 1005 |site_id e151e245 |site_domain 7e091613 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip aeda4e4d |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 22742 |C15 320 |C16 50 |C17 2637 |C18 3 |C19 35 |C20 100188 74 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 43ffbf74 |device_model d5db24c3 |device_type 1 |device_conn_type 0 |C14 22104 |C15 320 |C16 50 |C17 2545 |C18 0 |C19 167 |C20 -1 75 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id e2a1ca37 |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip c6444a0a |device_model cd5b1559 |device_type 1 |device_conn_type 2 |C14 16688 |C15 320 |C16 50 |C17 1873 |C18 3 |C19 39 |C20 -1 76 | -1 |C1 1005 |site_id 26fa1946 |site_domain e2a5dc06 |site_category 3e814130 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip ef7d41ba |device_model 0ea40c5b |device_type 1 |device_conn_type 0 |C14 17239 |C15 320 |C16 50 |C17 1973 |C18 3 |C19 39 |C20 -1 77 | -1 |C1 1005 |site_id 70ed6df4 |site_domain 907f1c1a |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 7fccb7d7 |device_model 0eb711ec |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 78 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 37f152dd |device_model 711ee120 |device_type 1 |device_conn_type 0 |C14 22261 |C15 320 |C16 50 |C17 2545 |C18 0 |C19 167 |C20 100084 79 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 6b451e87 |app_domain ae637522 |app_category 0f2161f8 |device_id a99f214a |device_ip a7818588 |device_model e1eae715 |device_type 1 |device_conn_type 2 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100105 80 | 1 |C1 1005 |site_id 17caea14 |site_domain 0dde25ec |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 677b37d8 |device_model 6ac9eb05 |device_type 1 |device_conn_type 0 |C14 21882 |C15 320 |C16 50 |C17 2526 |C18 0 |C19 167 |C20 100075 81 | -1 |C1 1005 |site_id 57fe1b20 |site_domain 5b626596 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 9161b52a |device_model ecb851b2 |device_type 1 |device_conn_type 0 |C14 8330 |C15 320 |C16 50 |C17 761 |C18 3 |C19 175 |C20 -1 82 | 1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 9cb8abc9 |device_model 5096d134 |device_type 1 |device_conn_type 0 |C14 22254 |C15 320 |C16 50 |C17 2545 |C18 0 |C19 167 |C20 100084 83 | -1 |C1 1005 |site_id 1d8321a3 |site_domain 31eace6b |site_category f66779e6 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 4add4074 |device_model c6263d8a |device_type 1 |device_conn_type 0 |C14 23430 |C15 320 |C16 50 |C17 2683 |C18 1 |C19 163 |C20 -1 84 | -1 |C1 1002 |site_id c70e6525 |site_domain 03cc62db |site_category 50e219e0 |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id def0a0a4 |device_ip 494a27a2 |device_model be8e9571 |device_type 0 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 100188 85 | -1 |C1 1005 |site_id e8f79e60 |site_domain c4342784 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip a8e56bc1 |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 21593 |C15 320 |C16 50 |C17 2478 |C18 3 |C19 167 |C20 100075 86 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id a2b190d4 |app_domain 33da2e74 |app_category dc97ec06 |device_id a99f214a |device_ip 91927b58 |device_model cad4c01d |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 87 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip b362e06b |device_model 36d749e5 |device_type 1 |device_conn_type 0 |C14 23165 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 -1 88 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id a2b190d4 |app_domain 33da2e74 |app_category dc97ec06 |device_id a99f214a |device_ip aae69350 |device_model 8b65b455 |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 -1 89 | -1 |C1 1005 |site_id 54e6fb06 |site_domain 91dd76e0 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip d9df405e |device_model 1f0bc64f |device_type 1 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 -1 90 | -1 |C1 1005 |site_id 12fb4121 |site_domain 6b59f079 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 2198d47c |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 23441 |C15 320 |C16 50 |C17 2685 |C18 1 |C19 33 |C20 100084 91 | -1 |C1 1005 |site_id 57fe1b20 |site_domain 5b626596 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip f3e30a19 |device_model 1f0bc64f |device_type 1 |device_conn_type 0 |C14 21413 |C15 320 |C16 50 |C17 2467 |C18 2 |C19 167 |C20 -1 92 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 685d1c4c |app_domain 2347f47a |app_category 8ded1f7a |device_id a99f214a |device_ip 164050b8 |device_model d787e91b |device_type 1 |device_conn_type 2 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 -1 93 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id febd1138 |app_domain 82e27996 |app_category 0f2161f8 |device_id a99f214a |device_ip 99ac1101 |device_model 2ea4f8ba |device_type 1 |device_conn_type 0 |C14 23454 |C15 320 |C16 50 |C17 2688 |C18 1 |C19 33 |C20 100105 94 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip bf2f7567 |device_model 897aafd7 |device_type 1 |device_conn_type 0 |C14 23165 |C15 320 |C16 50 |C17 2668 |C18 0 |C19 35 |C20 100083 95 | -1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 5580a95c |device_model 76dc4769 |device_type 1 |device_conn_type 0 |C14 23438 |C15 320 |C16 50 |C17 2684 |C18 2 |C19 291 |C20 100148 96 | 1 |C1 1005 |site_id 1fbe01fe |site_domain f3845767 |site_category 28905ebd |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip d0e9d4d1 |device_model d787e91b |device_type 1 |device_conn_type 0 |C14 22260 |C15 320 |C16 50 |C17 2545 |C18 0 |C19 167 |C20 100084 97 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 92f5800b |app_domain ae637522 |app_category 0f2161f8 |device_id a99f214a |device_ip e36a90ce |device_model e1eae715 |device_type 1 |device_conn_type 3 |C14 21191 |C15 320 |C16 50 |C17 2424 |C18 1 |C19 161 |C20 100192 98 | 1 |C1 1005 |site_id e151e245 |site_domain 7e091613 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 08b74884 |device_model 7abbbd5c |device_type 1 |device_conn_type 0 |C14 22813 |C15 320 |C16 50 |C17 2647 |C18 2 |C19 35 |C20 100148 99 | -1 |C1 1005 |site_id 856e6d3f |site_domain 58a89a43 |site_category f028772b |app_id ecad2386 |app_domain 7801e8d9 |app_category 07d7df22 |device_id a99f214a |device_ip 920ee471 |device_model ce576f14 |device_type 1 |device_conn_type 0 |C14 22682 |C15 320 |C16 50 |C17 2528 |C18 0 |C19 39 |C20 -1 100 | -1 |C1 1005 |site_id 85f751fd |site_domain c4e18dd6 |site_category 50e219e0 |app_id 7358e05e |app_domain b9528b13 |app_category cef3e649 |device_id 1105fa12 |device_ip e88a37f9 |device_model 9c7b6730 |device_type 1 |device_conn_type 0 |C14 16615 |C15 320 |C16 50 |C17 1863 |C18 3 |C19 39 |C20 -1 101 | --------------------------------------------------------------------------------