├── code ├── one_second_window │ ├── extractFromPcap.sh │ ├── README.md │ ├── extract_features_chronological.py │ └── testClass.py ├── raw_flows │ ├── processPcap.sh │ ├── README.md │ ├── devices.txt │ ├── PcapToInput.py │ └── ModelBuilder.py ├── tcp_upd_flows │ ├── README.md │ ├── testRF.py │ └── testNN.py └── two_stage │ ├── README.md │ ├── deviceMap.py │ ├── testRF.py │ ├── getDictionary.py │ ├── testMNB.py │ └── getFeatures.py ├── README.md ├── data └── README.md ├── .gitignore └── LICENSE /code/one_second_window/extractFromPcap.sh: -------------------------------------------------------------------------------- 1 | dir=$1 2 | 3 | for f in $(find $dir -name "*.pcap"); do 4 | tshark -r $f -T fields -e eth.src -e frame.time_epoch -e frame.len -E separator=, >> data.csv; 5 | done 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # revisiting-iot-device-identification 2 | 3 | Data and code for TMA 2021 paper [Revisiting IoT Device Identification](https://tma.ifip.org/2021/wp-content/uploads/sites/10/2021/08/tma2021-paper6.pdf) 4 | 5 | `code` directory contains code for four different approaches to train models 6 | `data` directory contains link to the datasets used for training and evaluation of models 7 | -------------------------------------------------------------------------------- /code/raw_flows/processPcap.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | pcap=$1 4 | pcapFile=$(basename $pcap) 5 | mac=$2 6 | devId=$3 7 | 8 | destDir="flows/$pcapFile" 9 | 10 | # this script requires that you have mono installed (framework for executing .NET executables on Linux) 11 | # and SpltiCap.exe which splits input pcap into separate TCP/UDP flows 12 | 13 | mono SplitCap.exe -r $pcap -p 1014 -o $destDir 14 | 15 | -------------------------------------------------------------------------------- /code/tcp_upd_flows/README.md: -------------------------------------------------------------------------------- 1 | This set of scripts trains and evaluates a neural network with dense layers and a random forest classifier on data extracted from TCP/UDP flows. 2 | 3 | The flows were extracted from PCAP files using a [joy](https://github.com/cisco/joy) utility from Cisco. 4 | 5 | Description of the files: 6 | 7 | - `testNN.py` creates and evaluates models using neural network 8 | - `testRF.py` creates and evaluates models using Random Forest Classifier 9 | -------------------------------------------------------------------------------- /code/raw_flows/README.md: -------------------------------------------------------------------------------- 1 | This set of scripts trains model using raw packets from TCP/UDP flows. 2 | 3 | They work as follows: 4 | 5 | - `processPcap.sh` takes pcap file as an input and uses SplitCap to extract independent TCP/UDP flows 6 | - `PcapToInput.py` takes a directory containing separate TCP/UDP flows as an input and procudes binary files containing formatted data (250x10) that can be used as input for a 2D Convolutional layer. 7 | - `ModelBuilder.py` creates and evaluates models using data produced by PcapToInput.py 8 | -------------------------------------------------------------------------------- /code/one_second_window/README.md: -------------------------------------------------------------------------------- 1 | This set of scripts trains and evaluates models for a statistical data created from one second window of packets. 2 | 3 | Description of files: 4 | 5 | - `extractFromPcap.sh` takes a directory with pcap files as an input and extract mac address, time, and packet length using tshark 6 | - `extract_features_chronological.py` takes a file produced by previous script as an input and for each one second window computes statistical information (mean, sum, and standard deviation) 7 | - `testClass.py` trains and evaluates models on data produced by previous script 8 | -------------------------------------------------------------------------------- /code/two_stage/README.md: -------------------------------------------------------------------------------- 1 | This is a set of scripts using a two stage classifier. 2 | 3 | When extracting features from pcap files a joy utility from Cisco needs to be installed [https://github.com/cisco/joy](https://github.com/cisco/joy) 4 | 5 | Description of files: 6 | 7 | - `deviceMap.py` maps MAC addresses to device IDs 8 | - `getDictionary.py` extracts data from a Multinomial Naive Bayes Classifier 9 | - `getFeatures.py` extracts features from pcap files. Separate pcap file for each 1 hour window needs to be created. 10 | - `testiMNB.py` classfies data using Naive Bayes Multinomial (NBM) 11 | - `testRF.py` classifies data using Random Forest (RF) 12 | -------------------------------------------------------------------------------- /data/README.md: -------------------------------------------------------------------------------- 1 | Due to the file constraints, the data files used for training and evaluating of various models can be found at: 2 | 3 | [https://drive.google.com/drive/folders/1Qbafa7DsEM5rhUwA\_-oM79FaKe2mpGzt?usp=sharing](https://drive.google.com/drive/folders/1Qbafa7DsEM5rhUwA_-oM79FaKe2mpGzt?usp=sharing) 4 | 5 | In particular: 6 | 7 | - `devices.txt` is a list of devices with their IDs and MAC addresses 8 | - `features\_nov-apr.tar.gz` data for [tcp\_udp\_flows](../code/tcp_udp_flows) models 9 | - `unsw\_features.tar.gz` data for [two\_stage](../code/two_stage) model 10 | - `stats\_nov-apr.tar.gz` data for [one\_second\_window](../code/one_second_window) models 11 | - `raw\_flows.tar.gz` data for [raw\_flows](../code/raw_flows) model 12 | -------------------------------------------------------------------------------- /code/two_stage/deviceMap.py: -------------------------------------------------------------------------------- 1 | dm = {'0:26:29:0:77:ce': 1, 2 | '0:3:7f:96:d8:ec': 2, 3 | '0:c:43:3:51:be': 3, 4 | '0:e0:4c:c9:46:31': 4, 5 | '0:e:f3:2c:d4:4': 5, 6 | '0:fc:8b:84:22:10': 6, 7 | '34:29:8f:1c:f3:9c': 7, 8 | '3c:71:bf:25:c5:60': 8, 9 | '40:31:3c:e6:77:c2': 9, 10 | '48:46:c1:1c:46:a5': 10, 11 | '50:32:37:b8:c7:f': 11, 12 | '50:c7:bf:ca:3f:9d': 12, 13 | '54:60:9:6f:32:84': 13, 14 | '58:b3:fc:5e:ca:74': 14, 15 | '58:ef:68:99:7d:ed': 15, 16 | '5c:41:5a:29:ad:97': 16, 17 | '64:16:66:2a:98:62': 17, 18 | '68:c6:3a:ba:c2:6b': 18, 19 | '68:c6:3a:e4:85:61': 19, 20 | '70:ee:50:36:98:da': 20, 21 | '74:40:be:cd:21:a4': 21, 22 | '78:a5:dd:28:a1:b7': 22, 23 | '7c:49:eb:22:30:9c': 23, 24 | '7c:49:eb:88:da:82': 24, 25 | '84:18:26:7c:1a:56': 25, 26 | 'ae:ca:6:e:ec:89': 26, 27 | 'b0:be:76:be:f2:aa': 27, 28 | 'b0:f1:ec:d4:26:ae': 28, 29 | 'b8:2c:a0:28:3e:6b': 29, 30 | 'c:2a:69:11:1:ba': 30, 31 | 'c8:3a:6b:fa:1c:0': 31, 32 | 'c:8c:24:b:be:fb': 32, 33 | 'cc:f7:35:25:af:4d': 33, 34 | 'cc:f7:35:49:f4:5': 34, 35 | 'd0:52:a8:a4:e6:46': 35, 36 | 'ec:71:db:49:af:ee': 36, 37 | 'ec:b5:fa:0:98:da': 37, 38 | 'ec:fa:bc:2e:85:5b': 38, 39 | 'f0:45:da:36:e6:23': 39, 40 | 'f4:b8:5e:68:8f:35': 40, 41 | 'fc:3:9f:93:22:62': 41 42 | } 43 | -------------------------------------------------------------------------------- /code/raw_flows/devices.txt: -------------------------------------------------------------------------------- 1 | 0:26:29:0:77:ce,appkettle,1,6 2 | 0:3:7f:96:d8:ec,blink-security-hub,2,2 3 | 0:c:43:3:51:be,bosiwo-camera-wifi,3,1 4 | 0:e0:4c:c9:46:31,lefun-cam-wired,4,1 5 | 0:e:f3:2c:d4:4,insteon-hub,5,2, 6 | 0:fc:8b:84:22:10,echoplus,6,5 7 | 34:29:8f:1c:f3:9c,meross-dooropener,7,3 8 | 3c:71:bf:25:c5:60,smartlife-bulb,8,3 9 | 40:31:3c:e6:77:c2,xiaomi-ricecooker,9,6 10 | 48:46:c1:1c:46:a5,ubell-doorbell,10,1 11 | 50:32:37:b8:c7:f,appletv,11,4 12 | 50:c7:bf:ca:3f:9d,tplink-bulb,12,3 13 | 54:60:9:6f:32:84,google-home,13,5 14 | 58:b3:fc:5e:ca:74,icsee-doorbell,14,1 15 | 58:ef:68:99:7d:ed,t-wemo-plug,15,3 16 | 5c:41:5a:29:ad:97,echospot,16,5 17 | 64:16:66:2a:98:62,nest-tstat,17,3 18 | 68:c6:3a:ba:c2:6b,sousvide,18,6 19 | 68:c6:3a:e4:85:61,smartlife-remote,19,3 20 | 70:ee:50:36:98:da,netatmo-weather-station,20,6 21 | 74:40:be:cd:21:a4,lgtv-wifi,21,4 22 | 78:a5:dd:28:a1:b7,wansview-cam-wired,22,1 23 | 7c:49:eb:22:30:9c,xiaomi-plug,23,3 24 | 7c:49:eb:88:da:82,xiaomi-hub,24,2 25 | 84:18:26:7c:1a:56,lightify-hub,25,2 26 | ae:ca:6:e:ec:89,bosiwo-camera-wired,26,1 27 | b0:be:76:be:f2:aa,tplink-plug2,27,3 28 | b0:f1:ec:d4:26:ae,allure-speaker,28,5 29 | b8:2c:a0:28:3e:6b,honeywell-thermostat,29,3 30 | c:2a:69:11:1:ba,smarter-coffee-mach,30,6 31 | c8:3a:6b:fa:1c:0,roku-tv,31,4 32 | c:8c:24:b:be:fb,yi-camera,32,1 33 | cc:f7:35:25:af:4d,firetv,33,4 34 | cc:f7:35:49:f4:5,echodot,34,5 35 | d0:52:a8:a4:e6:46,smartthings-hub,35,2 36 | ec:71:db:49:af:ee,reolink-cam-wired,36,1 37 | ec:b5:fa:0:98:da,t-philips-hub,37,2 38 | ec:fa:bc:2e:85:5b,switchbot-hub,38,2 39 | f0:45:da:36:e6:23,ring-doorbell,39,1 40 | f4:b8:5e:68:8f:35,blink-camera,40,1 41 | fc:3:9f:93:22:62,samsungtv-wired,41,4 42 | -------------------------------------------------------------------------------- /code/two_stage/testRF.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import numpy as np 3 | from sklearn.model_selection import train_test_split 4 | from sklearn.ensemble import RandomForestClassifier 5 | from sklearn import metrics 6 | import sys 7 | from getDictionary import InputVector 8 | from testMNB import MNBs 9 | 10 | 11 | if __name__ == '__main__': 12 | df = pd.read_csv(sys.argv[1], parse_dates=['time_start']) 13 | 14 | mnbs = MNBs(df, ['ports', 'dns', 'cs'], 'mac') 15 | 16 | clf = RandomForestClassifier(n_estimators=20, n_jobs=50) 17 | 18 | weeks = [list(range(44,53)), list(range(1,10)), list(range(10,19))] 19 | 20 | #trainCon = df['time_start'] >= pd.Timestamp('2020-03-01') 21 | trainCon = df['time_start'].dt.week.isin(weeks[2]) 22 | #trainCon = df['time_start'] < pd.Timestamp('2020-01-01') 23 | #testCon = df['time_start'] > pd.Timestamp('2020-02-29') 24 | testCons = pd.date_range('2019-11-01', '2020-05-01', freq = '1W').tolist() 25 | #print(testCons) 26 | 27 | #mnbs.extractDictionary(trainCon) 28 | mnbs.extractDictionary(df['time_start'] < pd.Timestamp('2021-01-01')) 29 | mnbs.fit(trainCon) 30 | 31 | trainDF = mnbs.updateDFWithProba(trainCon) 32 | 33 | #print(trainDF) 34 | 35 | features = ['volume_mean','flow_durations','ratio','sleep_time','dns_interval','ports_id','ports_proba','dns_id','dns_proba','cs_id','cs_proba'] 36 | X = trainDF[features] 37 | y = trainDF['mac'] 38 | 39 | clf.fit(X, y) 40 | 41 | for date in testCons: 42 | testCon = df['time_start'].dt.week == pd.Timestamp(date).week 43 | testDF = mnbs.updateDFWithProba(testCon) 44 | 45 | X = testDF[features] 46 | y = testDF['mac'] 47 | 48 | y_pred = clf.predict(X) 49 | print(metrics.classification_report(y, y_pred)) 50 | print(f"Accuracy {date.week}:", metrics.accuracy_score(y, y_pred), metrics.f1_score(y, y_pred, average="micro")) 51 | 52 | 53 | -------------------------------------------------------------------------------- /code/two_stage/getDictionary.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import numpy as np 3 | import ast, sys 4 | from collections import Counter 5 | 6 | class InputVector(object): 7 | 8 | def __init__(self): 9 | self.input = {} 10 | self.wholeDict = [] 11 | 12 | 13 | def extractDataFromDF(self, df, indexCol, dataCol): 14 | for index, rec in df[(df[dataCol] != "{}")].groupby(indexCol)[dataCol].unique().iteritems(): 15 | self.input[index] = self._extractData(rec) 16 | self.wholeDict += self.input[index] 17 | 18 | self.wholeDict = list(set(self.wholeDict)) 19 | 20 | def _extractData(self, data): 21 | flatten = lambda t: [item for sublist in t for item in sublist] 22 | return list(set(flatten([list(ast.literal_eval(x).keys()) for x in data]))) 23 | 24 | def _getEmptyInputVector(self, deviceId = None): 25 | input = self.wholeDict if deviceId is None else self.input[deviceId] 26 | 27 | vector = dict(zip(input, [0]*len(input))) 28 | vector['other'] = 0 29 | return vector 30 | 31 | def getFinalInputVector(self, inputData: dict, deviceId = None): 32 | vector = self._getEmptyInputVector(deviceId) 33 | for k, v in inputData.items(): 34 | if k in vector: 35 | vector[k] = v 36 | else: 37 | vector['other'] += v 38 | 39 | #return vector 40 | return list(vector.values()) 41 | 42 | def getFinalInputMatrix(self, df: pd.DataFrame, dataCol: str): 43 | X = [] 44 | for index, rec in df[dataCol].iteritems(): 45 | X.append(self.getFinalInputVector(ast.literal_eval(rec))) 46 | 47 | return X 48 | 49 | def getXy(self, df: pd.DataFrame, dataCol: str, indexCol: str): 50 | X = self.getFinalInputMatrix(df, dataCol) 51 | y = df[indexCol] 52 | 53 | return np.array(X), np.array(y) 54 | 55 | 56 | if __name__ == "__main__": 57 | df = pd.read_csv(sys.argv[1], parse_dates=['time_start']) 58 | dns = InputVector() 59 | dns.extractDataFromDF(df[df['time_start'] < pd.Timestamp("2020-03-01")], 'mac', 'dns') 60 | #print(dns.input) 61 | mar = df[df['time_start'] > pd.Timestamp("2020-02-29")] 62 | X,y = dns.getXy(mar, 'dns', 'mac') 63 | print(X) 64 | print(y) 65 | #print(dns.getFinalInputVector(ast.literal_eval(mar.iloc[20750]['dns'])))#, mar.iloc[20750]['mac'])) 66 | 67 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | target/ 76 | 77 | # Jupyter Notebook 78 | .ipynb_checkpoints 79 | 80 | # IPython 81 | profile_default/ 82 | ipython_config.py 83 | 84 | # pyenv 85 | .python-version 86 | 87 | # pipenv 88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 91 | # install all needed dependencies. 92 | #Pipfile.lock 93 | 94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 95 | __pypackages__/ 96 | 97 | # Celery stuff 98 | celerybeat-schedule 99 | celerybeat.pid 100 | 101 | # SageMath parsed files 102 | *.sage.py 103 | 104 | # Environments 105 | .env 106 | .venv 107 | env/ 108 | venv/ 109 | ENV/ 110 | env.bak/ 111 | venv.bak/ 112 | 113 | # Spyder project settings 114 | .spyderproject 115 | .spyproject 116 | 117 | # Rope project settings 118 | .ropeproject 119 | 120 | # mkdocs documentation 121 | /site 122 | 123 | # mypy 124 | .mypy_cache/ 125 | .dmypy.json 126 | dmypy.json 127 | 128 | # Pyre type checker 129 | .pyre/ 130 | -------------------------------------------------------------------------------- /code/raw_flows/PcapToInput.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | from scapy.all import load_layer, PcapReader, PcapWriter 3 | from scapy.layers.inet import Ether, IP 4 | from scapy.packet import raw 5 | import sys, os 6 | import pandas as pd 7 | 8 | 9 | class PcapToInput(object): 10 | 11 | def __init__(self, filePath, numPackets=10, packetLength=250): 12 | self.filePath = filePath 13 | self.numPackets = numPackets 14 | self.packetLength = packetLength 15 | 16 | def processFile(self): 17 | pcap = PcapReader(self.filePath.as_posix()) 18 | i = 0 19 | rawPackets = [] 20 | 21 | try: 22 | for p in pcap: 23 | #print(p) 24 | if len(rawPackets) >= self.numPackets: 25 | break 26 | p = self._removeFields(p) 27 | p = self._trimPacket(p) 28 | #print(p) 29 | rawPackets.append(p) 30 | #rawPackets[i] = raw(p) 31 | 32 | i+=1 33 | 34 | except Exception as e: 35 | print(f"Error: {e}") 36 | finally: 37 | if len(rawPackets) < self.numPackets: 38 | for i in range(len(rawPackets), self.numPackets): 39 | rawPackets.append(bytes([0]*self.packetLength)) 40 | 41 | return b''.join(rawPackets) 42 | 43 | def _removeFields(self, packet): 44 | packet[Ether].src = 0 45 | packet[Ether].dst = 0 46 | packet[IP].src = 0 47 | 48 | return packet 49 | 50 | 51 | def _trimPacket(self, packet): 52 | packet = raw(packet) 53 | if len(packet) > self.packetLength: 54 | return packet[0:self.packetLength] 55 | 56 | return packet + bytes([0]*(self.packetLength - len(packet))) 57 | 58 | def saveRawPackets(self, saveFile, packets): 59 | df = pd.DataFrame({'deviceId': 1, 'packets': packets}) 60 | df.to_csv('tmp.csv', index=False) 61 | #return 62 | with open(saveFile.as_posix(), 'wb') as f: 63 | for p in packets: 64 | f.write(p) 65 | 66 | 67 | class PcapsToInput(object): 68 | 69 | def __init__(self, deviceId: int, numPackets=10, packetLength=250): 70 | self.deviceId = deviceId 71 | self.rawPackets = [] 72 | 73 | self.flowCounter = 0 74 | self.metadata = {} 75 | 76 | def processDir(self, dirPath: Path): 77 | for path in dirPath.iterdir(): 78 | #print(path) 79 | pti = PcapToInput(path) 80 | self.rawPackets.append(pti.processFile()) 81 | self.flowCounter+= 1 82 | 83 | def saveRawPackets(self, saveFile): 84 | with open(saveFile.as_posix(), 'wb') as f: 85 | f.write(b''.join(self.rawPackets)) 86 | 87 | if __name__ == "__main__": 88 | #pti = PcapToInput(Path(sys.argv[1])) 89 | #rp = pti.processFile() 90 | #print(len(rp),rp[99]) 91 | #pti.saveRawPackets(Path('tmp.raw'), rp) 92 | 93 | ptis = PcapsToInput(sys.argv[1]) 94 | ptis.processDir(Path(sys.argv[2])) 95 | ptis.saveRawPackets(Path(f"/data/roman/flows/{sys.argv[1]}_{ptis.flowCounter}_{sys.argv[3]}.bin")) 96 | -------------------------------------------------------------------------------- /code/two_stage/testMNB.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import ast, sys 3 | import numpy as np 4 | from getDictionary import InputVector 5 | from sklearn.naive_bayes import MultinomialNB 6 | from sklearn.ensemble import RandomForestClassifier 7 | from sklearn import metrics 8 | 9 | class MNB(object): 10 | 11 | def __init__(self, df: pd.DataFrame, dataCol: str, indexCol: str): 12 | self.df = df 13 | self.dataCol = dataCol 14 | self.indexCol = indexCol 15 | self.clf = MultinomialNB() 16 | self.iv = InputVector() 17 | 18 | def extractDictionary(self, cond): 19 | self.iv.extractDataFromDF(self.df[cond], self.indexCol, self.dataCol) 20 | 21 | def fit(self, cond): 22 | X, y = self.iv.getXy(self.df[cond], self.dataCol, self.indexCol) 23 | 24 | self.clf.fit(X, y) 25 | 26 | def predictProba(self, cond): 27 | X, y = self.iv.getXy(self.df[cond], self.dataCol, self.indexCol) 28 | 29 | predy = self.clf.predict_proba(X) 30 | indices = np.argmax(predy, axis=1) 31 | vals = np.amax(predy, axis=-1) 32 | 33 | return np.array(list(zip(indices, vals))) 34 | 35 | class MNBs(object): 36 | 37 | def __init__(self, df: pd.DataFrame, dataCol: list, indexCol: str): 38 | self.df = df 39 | self.mnb = {} 40 | for col in dataCol: 41 | self.mnb[col] = MNB(df, col, indexCol) 42 | 43 | def extractDictionary(self, cond): 44 | for mnb in self.mnb.values(): 45 | mnb.extractDictionary(cond) 46 | 47 | def fit(self, cond): 48 | for mnb in self.mnb.values(): 49 | mnb.fit(cond) 50 | 51 | def predictProba(self, cond): 52 | probs = {} 53 | for col, mnb in self.mnb.items(): 54 | probs[col] = mnb.predictProba(cond) 55 | 56 | return probs 57 | 58 | def updateDFWithProba(self, cond) -> pd.DataFrame: 59 | probs = self.predictProba(cond) 60 | 61 | df = self.df[cond].copy() 62 | 63 | for col, proba in probs.items(): 64 | df[f'{col}_id'], df[f'{col}_proba'] = proba[:, 0], proba[:, 1] 65 | 66 | return df 67 | 68 | 69 | 70 | 71 | if __name__ == "__main__": 72 | df = pd.read_csv(sys.argv[1], parse_dates=['time_start']) 73 | 74 | weeks = [list(range(44,53)), list(range(1,10)), list(range(10,19))] 75 | 76 | mnb = MNB(df, 'dns', 'mac') 77 | feb = df['time_start'] < pd.Timestamp("2020-03-01") 78 | mar = df['time_start'] > pd.Timestamp("2020-02-29") 79 | mnb.extractDictionary(feb) 80 | mnb.fit(feb) 81 | 82 | pred = mnb.predictProba(mar) 83 | print(pred) 84 | sys.exit() 85 | 86 | 87 | 88 | dns = InputVector() 89 | cs = InputVector() 90 | 91 | dnsClf = MultinomialNB() 92 | csClf = MultinomialNB() 93 | 94 | #dnsClf = RandomForestClassifier(n_estimators=20, n_jobs=50) 95 | #csClf = RandomForestClassifier(n_estimators=20, n_jobs=50) 96 | 97 | 98 | feb = df[df['time_start'] < pd.Timestamp("2020-03-01")] 99 | mar = df[df['time_start'] > pd.Timestamp("2020-02-29")] 100 | 101 | dns.extractDataFromDF(feb, 'mac', 'dns') 102 | cs.extractDataFromDF(feb, 'mac', 'cs') 103 | 104 | dnsX, dnsy = dns.getXy(feb, 'dns', 'mac') 105 | csX, csy = cs.getXy(feb, 'cs', 'mac') 106 | 107 | dnsClf.fit(dnsX, dnsy) 108 | csClf.fit(csX, csy) 109 | 110 | 111 | dnsTestX, dnsTesty = dns.getXy(mar, 'dns', 'mac') 112 | csTestX, csTesty = cs.getXy(mar, 'cs', 'mac') 113 | 114 | #print(f"dnsX shape {dnsX.shape} dnsy {dnsy.shape} test x: {dnsTestX.shape} test y: {dnsTesty.shape}") 115 | #print(f"csX shape {csX.shape} csy {csy.shape} test x: {csTestX.shape} test y: {csTesty.shape}") 116 | 117 | dnsPredicty = dnsClf.predict_proba(dnsTestX) 118 | indices = np.argmax(dnsPredicty, axis=1) 119 | vals = np.amax(dnsPredicty, axis=-1) 120 | print(np.array(list(zip(indices, vals)))) 121 | sys.exit() 122 | csPredicty = csClf.predict(csTestX) 123 | 124 | print("Accuracy next month DNS:", metrics.accuracy_score(dnsTesty, dnsPredicty)) 125 | print("Accuracy next month CS:", metrics.accuracy_score(csTesty, csPredicty)) 126 | -------------------------------------------------------------------------------- /code/one_second_window/extract_features_chronological.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import os 3 | import sys 4 | 5 | ''' 6 | The network traffic made available by the authors of 'Classifying IoT Devices in Smart Environments Using Network Traffic Characteristics' is composed of IoT and non-IoT devices. 7 | 8 | We implemented this Python script to extract IoT devices from datasets and assign labels to them based on their mac addresses. 9 | ''' 10 | 11 | #The folder containing the datasets. 12 | #folder = str(sys.argv[1]) 13 | 14 | #Maps the MAC address of an IoT device to a label (integer). 15 | #devices ={"d0:52:a8:00:67:5e":1,"44:65:0d:56:cc:d3":2,"70:ee:50:18:34:43":3,"f4:f2:6d:93:51:f1":4,"00:16:6c:ab:6b:88":5,"30:8c:fb:2f:e4:b2":6,"00:62:6e:51:27:2e":7,"00:24:e4:11:18:a8":8,"ec:1a:59:79:f4:89":9,"50:c7:bf:00:56:39":10,"74:c6:3b:29:d7:1d":11,"ec:1a:59:83:28:11":12,"18:b4:30:25:be:e4":13,"70:ee:50:03:b8:ac":14,"00:24:e4:1b:6f:96":15,"74:6a:89:00:2e:25":16,"00:24:e4:20:28:c6":17,"d0:73:d5:01:83:08":18,"18:b7:9e:02:20:44":19,"e0:76:d0:33:bb:85":20,"70:5a:0f:e4:9b:c0":21} 16 | #devices ={"d0:52:a8:a4:e6:46":1,"00:fc:8b:84:22:10":2,"70:ee:50:18:34:43":3,"f4:f2:6d:93:51:f1":4,"00:16:6c:ab:6b:88":5,"30:8c:fb:2f:e4:b2":6,"00:62:6e:51:27:2e":7,"00:24:e4:11:18:a8":8,"ec:1a:59:79:f4:89":9,"50:c7:bf:b1:d2:78":10,"74:c6:3b:29:d7:1d":11,"ec:1a:59:83:28:11":12,"18:b4:30:25:be:e4":13,"70:ee:50:36:98:da":14,"00:24:e4:1b:6f:96":15,"74:6a:89:00:2e:25":16,"00:24:e4:20:28:c6":17,"d0:73:d5:01:83:08":18,"18:b7:9e:02:20:44":19,"e0:76:d0:33:bb:85":20,"70:5a:0f:e4:9b:c0":21} 17 | 18 | dm = {'0:26:29:0:77:ce': 1, 19 | '0:3:7f:96:d8:ec': 2, 20 | '0:c:43:3:51:be': 3, 21 | '0:e0:4c:c9:46:31': 4, 22 | '0:e:f3:2c:d4:4': 5, 23 | '0:fc:8b:84:22:10': 6, 24 | '34:29:8f:1c:f3:9c': 7, 25 | '3c:71:bf:25:c5:60': 8, 26 | '40:31:3c:e6:77:c2': 9, 27 | '48:46:c1:1c:46:a5': 10, 28 | '50:32:37:b8:c7:f': 11, 29 | '50:c7:bf:ca:3f:9d': 12, 30 | '54:60:9:6f:32:84': 13, 31 | '58:b3:fc:5e:ca:74': 14, 32 | '58:ef:68:99:7d:ed': 15, 33 | '5c:41:5a:29:ad:97': 16, 34 | '64:16:66:2a:98:62': 17, 35 | '68:c6:3a:ba:c2:6b': 18, 36 | '68:c6:3a:e4:85:61': 19, 37 | '70:ee:50:36:98:da': 20, 38 | '74:40:be:cd:21:a4': 21, 39 | '78:a5:dd:28:a1:b7': 22, 40 | '7c:49:eb:22:30:9c': 23, 41 | '7c:49:eb:88:da:82': 24, 42 | '84:18:26:7c:1a:56': 25, 43 | 'ae:ca:6:e:ec:89': 26, 44 | 'b0:be:76:be:f2:aa': 27, 45 | 'b0:f1:ec:d4:26:ae': 28, 46 | 'b8:2c:a0:28:3e:6b': 29, 47 | 'c:2a:69:11:1:ba': 30, 48 | 'c8:3a:6b:fa:1c:0': 31, 49 | 'c:8c:24:b:be:fb': 32, 50 | 'cc:f7:35:25:af:4d': 33, 51 | 'cc:f7:35:49:f4:5': 34, 52 | 'd0:52:a8:a4:e6:46': 35, 53 | 'ec:71:db:49:af:ee': 36, 54 | 'ec:b5:fa:0:98:da': 37, 55 | 'ec:fa:bc:2e:85:5b': 38, 56 | 'f0:45:da:36:e6:23': 39, 57 | 'f4:b8:5e:68:8f:35': 40, 58 | 'fc:3:9f:93:22:62': 41 59 | } 60 | 61 | #for file in os.listdir(folder): 62 | df = pd.read_csv(sys.argv[1], header=None, names=["src","Time","Length"])[["src","Time","Length"]] 63 | df["Time"] = df["Time"].apply(lambda x: int(x)) 64 | 65 | #Replaces the MAC address of IoT devices with labels. 66 | #for d in devices: 67 | df["src"] = df.replace({"src": dm}) 68 | 69 | #Extracts IoT devices from the original dataset. 70 | df = df[df['src'].astype(str).str.isdigit()] 71 | 72 | #Groups packets into one-second windows for each IoT device. 73 | df = df.groupby(["Time","src"]).agg(['mean', 'sum', 'std']) 74 | df.fillna(0, inplace=True) 75 | 76 | df.to_csv(f"{sys.argv[2]}", header=False) 77 | sys.exit() 78 | #Computes the statistical features for each one-second window and saves it to temporary CSV files. 79 | g.mean().to_csv("length_avg.csv",sep=",") 80 | g.sum().to_csv("length_sum.csv",sep=",") 81 | g.std().to_csv("length_std.csv",sep=",") 82 | 83 | #Creates a new data frame to store the statistical features. 84 | df_final = pd.DataFrame() 85 | 86 | #Populates the new data frame with statistical features and labels. 87 | df_final["avg"] = pd.read_csv("length_avg.csv")["Length"] 88 | df_final["n_bytes"] = pd.read_csv("length_sum.csv")["Length"] 89 | df_final["std"] = pd.read_csv("length_std.csv")["Length"] 90 | df_final["label"] = pd.read_csv("length_avg.csv")["src"] 91 | 92 | #Discard NaN values. 93 | df_final = df_final.dropna() 94 | 95 | #Save the statistical features to a new CSV file 96 | df_final.to_csv(str(file)+"_statistics.csv",sep=",",mode='a',index=False,header=False) 97 | 98 | -------------------------------------------------------------------------------- /code/tcp_upd_flows/testRF.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import numpy as np 3 | from sklearn.model_selection import train_test_split 4 | from sklearn.ensemble import RandomForestClassifier 5 | from sklearn.svm import SVC 6 | from sklearn.neighbors import KNeighborsClassifier 7 | from sklearn.tree import DecisionTreeClassifier 8 | from sklearn.ensemble import VotingClassifier 9 | import sys 10 | from sklearn import metrics 11 | from joblib import dump, load 12 | from pathlib import Path 13 | 14 | features = ['srcPort', 'destPort', 15 | 'bytes_out', 'num_pkts_out', 'bytes_in', 'num_pkts_in', 'f_ipt_mean', 16 | 'f_ipt_std', 'f_ipt_var', 'f_ipt_skew', 'f_ipt_kurtosis', 'f_b_mean', 17 | 'f_b_std', 'f_b_var', 'f_b_skew', 'f_b_kurtosis', 'duration', 'pr', 18 | 'domainId'] 19 | label = 'deviceId' 20 | 21 | fieldTypes = {'deviceId': 'int16', 22 | #'time_start': 'datetime64[ns]', 'time_end': 'datetime64[ns]', 23 | 'srcPort': 'int32', 'destPort': 'int32', 24 | 'bytes_out': 'int32', 'num_pkts_out': 'int32', 25 | 'bytes_in': 'int32', 'num_pkts_in': 'int32', 26 | 'f_ipt_mean': 'float64', 'f_ipt_std': 'float64', 27 | 'f_ipt_var': 'float64', 'f_ipt_skew': 'float64', 28 | 'f_ipt_kurtosis': 'float64', 29 | 'f_b_mean': 'float64', 'f_b_std': 'float64', 30 | 'f_b_var': 'float64', 'f_b_skew': 'float64', 31 | 'f_b_kurtosis': 'float64', 32 | 'duration': 'float64', 'pr': 'int8', 33 | 'expire_type': 'object', 34 | 'domain': 'object', 35 | 'domain2': 'object', 36 | 'domainId': 'int16'} 37 | 38 | 39 | #df = pd.read_csv(sys.argv[1], dtype = fieldTypes, parse_dates = ['time_start']) 40 | df = pd.read_csv(sys.argv[1], parse_dates = ['time_start']) 41 | df.fillna(0, inplace=True) 42 | 43 | weeks = [list(range(44,53)), list(range(1,10)), list(range(10,19))] 44 | 45 | #trainCon = df['time'] >= pd.Timestamp('2020-03-01') 46 | #trainCon = df['time'] < pd.Timestamp('2020-01-01') 47 | #trainCon = ((df['time'] >= pd.Timestamp('2020-01-01')) & (df['time'] < pd.Timestamp('2020-03-01'))) 48 | trainCon = df['time_start'].dt.week.isin(weeks[int(sys.argv[2])]) 49 | #trainCon = df['time'].dt.week.isin([44,45]) 50 | 51 | 52 | #testCon = df['time'] > pd.Timestamp('2020-02-29') 53 | #testCons = pd.date_range('2019-11-01', '2020-05-01', freq = '1W').tolist() 54 | testCons = pd.date_range('2019-12-01', '2020-05-01', freq = '1W').tolist() 55 | #testCons = pd.date_range('2019-11-01', '2019-12-01', freq = '1W').tolist() 56 | 57 | #X = dfTrain.iloc[:, 2:5] 58 | #y = dfTrain.iloc[:, 1] 59 | X = df[trainCon][features] 60 | y = df[trainCon][label] 61 | 62 | X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0, stratify=y) 63 | #clf = RandomForestClassifier(n_estimators=20, n_jobs=50) 64 | rfc = RandomForestClassifier(n_jobs=-1) 65 | #svc = SVC(verbose=True) 66 | #dtc = DecisionTreeClassifier() 67 | #knn = KNeighborsClassifier(n_jobs=54) 68 | #mv = VotingClassifier(estimators=[('knn', knn), ('dt', dtc), ('rf', rfc), ('svm',svc)], voting='hard', n_jobs=54) 69 | 70 | modelFile = Path(f"model/rfc_{sys.argv[2]}") 71 | if modelFile.is_file(): 72 | print(f"loading model {modelFile}") 73 | rfc = load(modelFile) 74 | else: 75 | print(f"start training for weeks {weeks[int(sys.argv[2])]}") 76 | rfc.fit(X_train, y_train) 77 | print("RFC trained") 78 | #dump(rfc, f"model/rfc_{sys.argv[2]}") 79 | dump(rfc, modelFile) 80 | 81 | #svc.fit(X, y) 82 | #print("SVC trained") 83 | #dump(SVC, f"model/svc_{sys.argv[2]}") 84 | # 85 | #dtc.fit(X, y) 86 | #print("DTC trained") 87 | #dump(dtc, f"model/dtc_{sys.argv[2]}") 88 | # 89 | #knn.fit(X,y) 90 | #print("KNN Trained") 91 | #dump(knn, f"model/knn_{sys.argv[2]}") 92 | # 93 | #mv.fit(X, y) 94 | #print("MV Trained") 95 | #dump(mv, f"model/mv_{sys.argv[2]}") 96 | 97 | for date in testCons: 98 | testCon = df['time_start'].dt.week == pd.Timestamp(date).week 99 | #testCon = df['time'].dt.date == pd.Timestamp(date).date 100 | testDF = df[testCon] 101 | 102 | X = testDF[features] 103 | y = testDF[label] 104 | 105 | if len(X) == 0: 106 | continue 107 | 108 | #y_pred = clf.predict(X) 109 | y_rfc = rfc.predict(X) 110 | #print("rfc predict") 111 | #y_svc = svc.predict(X) 112 | #print("svc predict") 113 | #y_dtc = dtc.predict(X) 114 | #print("dtc predict") 115 | #y_knn = knn.predict(X) 116 | #print("knn predict") 117 | #y_mv = mv.predict(X) 118 | #print("mv predict") 119 | 120 | print(f"{date.week}", metrics.f1_score(y, y_rfc, average='micro')) 121 | #metrics.f1_score(y, y_svc, average='micro'), 122 | #metrics.f1_score(y, y_dtc, average='micro'), 123 | #metrics.f1_score(y, y_knn, average='micro'), 124 | #metrics.f1_score(y, y_mv, average='micro') 125 | #) 126 | #print(f"Accuracy {date.date}:", metrics.accuracy_score(y, y_pred)) 127 | 128 | sys.exit() 129 | 130 | 131 | -------------------------------------------------------------------------------- /code/one_second_window/testClass.py: -------------------------------------------------------------------------------- 1 | from sklearn.ensemble import RandomForestClassifier 2 | from sklearn.svm import SVC 3 | from sklearn.neighbors import KNeighborsClassifier 4 | from sklearn.tree import DecisionTreeClassifier 5 | from sklearn.metrics import accuracy_score, recall_score, f1_score 6 | from imblearn.metrics import geometric_mean_score, specificity_score 7 | from sklearn.model_selection import StratifiedKFold 8 | from sklearn.ensemble import VotingClassifier 9 | 10 | import pandas as pd 11 | import numpy as np 12 | import sys, os 13 | import warnings 14 | import datetime 15 | #import pickle 16 | from joblib import dump, load 17 | 18 | 19 | warnings.simplefilter("ignore", category=FutureWarning) 20 | 21 | class Evaluator(object): 22 | 23 | def __init__(self, modelType = ""): 24 | self.modelType = modelType 25 | self.model = None 26 | self.testFile = "" 27 | self.trainFile = "" 28 | self.metrics = {} 29 | 30 | self.modelBaseDir = "model" 31 | 32 | #self.XCols = ['avg', 'std', 'n_bytes'] 33 | #self.label = 'label' 34 | 35 | self.XCols = ['destPort', 'bytes_out', 'bytes_in', 'num_pkts_out', 'num_pkts_in', 36 | 'f_ipt_mean', 'f_ipt_std', 'f_b_mean', 'f_b_std', 'duration', 'pr']#, 'domain2'] 37 | self.label = 'deviceId' 38 | 39 | def loadModel(self, fileName): 40 | self.model = load(os.path.join(self.modelBaseDir, fileName)) 41 | 42 | def incrementalTrainEval(self, startDate, endDate, step = 1): 43 | dates = pd.date_range(startDate, endDate, freq = "D").tolist() 44 | self.model = self.getModel() 45 | 46 | for i, date in enumerate(dates): 47 | try: 48 | trainData = self.filterByDate(date, dates[i+1]) 49 | #print(trainData) #print(trainData['time_start'][0], trainData['time_start'][-1]) 50 | testData = self.filterByDate(dates[i+1], dates[i+2]) 51 | #print(testData[0]['time_start'], testData[-1]['time_start']) 52 | 53 | trainX, trainY = self.getXandY(trainData) 54 | if not trainX.empty: 55 | #print (trainX, trainY) 56 | self.model.fit(trainX, trainY) 57 | 58 | testX, testY = self.getXandY(testData) 59 | 60 | if not testX.empty: 61 | self.yPredict = self.model.predict(testX) 62 | self.y = testY 63 | self.evaluateModel() 64 | 65 | except (IndexError, KeyError): 66 | print (date, i, len(dates)) 67 | 68 | 69 | 70 | 71 | def trainModel(self, trainFile): 72 | self.trainFile = trainFile 73 | 74 | self.model = self.getModel() 75 | self.loadData(trainFile) 76 | X, y = self.getXandY() 77 | #print(X) 78 | self.model.fit(X, y) 79 | 80 | def testModel(self, testFile): 81 | self.loadData(testFile) 82 | X, y = self.getXandY() 83 | 84 | self.yPredict = self.model.predict(self.X) 85 | 86 | def evaluateModel(self): 87 | self.metrics['acc'] = accuracy_score(self.y, self.yPredict) 88 | self.metrics['recall'] = recall_score(self.y, self.yPredict, average="micro") 89 | self.metrics['f1'] = f1_score(self.y, self.yPredict, average="micro") 90 | self.metrics['spec'] = specificity_score(self.y, self.yPredict, average="micro") 91 | self.metrics['mean'] = geometric_mean_score(self.y, self.yPredict, average="micro") 92 | 93 | print(self.metrics) 94 | 95 | def loadData(self, fileName): 96 | #df = pd.read_csv(fileName, names = self.XCols + [self.label], low_memory = False) 97 | self.df = pd.read_csv(fileName, low_memory = False) 98 | for col in self.XCols: 99 | self.df[col].fillna(0, inplace = True) 100 | 101 | self.df['time_start'] = self.df['time_start'].apply(lambda x: pd.Timestamp(x, unit='s')) 102 | self.df['time_end'] = self.df['time_end'].apply(lambda x: pd.Timestamp(x, unit='s')) 103 | 104 | def getXandY(self, df = None): 105 | #if df == None: 106 | # df = self.df 107 | return df[self.XCols], df[self.label] 108 | 109 | def filterByDate(self, timeStart, timeEnd, df = None): 110 | if df is None: 111 | df = self.df 112 | mask = (df['time_start'] >= timeStart) & (df['time_start'] < timeEnd) 113 | 114 | return df[mask] 115 | 116 | def saveModel(self): 117 | dump(self.model, os.path.join(self.modelBaseDir, "{}_{}.joblib".format(self.modelType, os.path.basename(self.trainFile)))) 118 | 119 | def getModel(self): 120 | if self.modelType == "rfc": 121 | return RandomForestClassifier() 122 | elif self.modelType == "svc": 123 | return SVC() 124 | elif self.modelType == "dtc": 125 | return DecisionTreeClassifier() 126 | elif self.modelType == "knn": 127 | return KNeighborsClassifier(n_neighbors=3) 128 | else: 129 | print("No model defined") 130 | return None 131 | 132 | if __name__ == "__main__": 133 | e = Evaluator("rfc") 134 | startDate = '2019-04-28' 135 | endDate = '2019-05-01' 136 | e.loadData(sys.argv[1]) 137 | e.incrementalTrainEval(startDate, endDate) 138 | 139 | sys.exit() 140 | if sys.argv[1] == "train": 141 | e = Evaluator(sys.argv[2]) 142 | e.trainModel(sys.argv[3]) 143 | e.saveModel() 144 | elif sys.argv[1] == "test": 145 | e = Evaluator() 146 | e.loadModel(sys.argv[2]) 147 | e.testModel(sys.argv[3]) 148 | e.evaluateModel() 149 | -------------------------------------------------------------------------------- /code/tcp_upd_flows/testNN.py: -------------------------------------------------------------------------------- 1 | import keras 2 | from keras.models import Sequential 3 | from keras.layers import Dense, Dropout, Activation, Flatten 4 | from keras.layers import Conv2D, Conv1D, MaxPooling2D, MaxPooling1D, LSTM, Bidirectional 5 | from keras.wrappers.scikit_learn import KerasClassifier 6 | from keras.callbacks import EarlyStopping 7 | from keras import backend as K 8 | 9 | from sklearn.preprocessing import LabelBinarizer 10 | from sklearn.model_selection import train_test_split 11 | from sklearn.metrics import classification_report 12 | from sklearn import metrics 13 | from sklearn.preprocessing import StandardScaler 14 | 15 | import sys, os, time, json, datetime, glob 16 | 17 | from logzero import logger 18 | import warnings 19 | import tensorflow as tf 20 | 21 | import numpy as np 22 | import pandas as pd 23 | from pathlib import Path 24 | 25 | os.environ["CUDA_VISIBLE_DEVICES"] = "-1" 26 | 27 | warnings.simplefilter("ignore", category=FutureWarning) 28 | 29 | 30 | if __name__ == '__main__': 31 | model = Sequential() 32 | model.add(Dense(64, input_dim = 19, activation='relu')) 33 | model.add(Dense(128, activation='relu')) 34 | model.add(Dense(256, activation='relu')) 35 | model.add(Dense(41, activation='softmax')) 36 | 37 | model.compile(loss="categorical_crossentropy", optimizer='adam', 38 | metrics=["accuracy"]) 39 | 40 | y = [] 41 | packets = [] 42 | i = 0 43 | 44 | devs = ['appkettle','blink-security-hub','bosiwo-camera-wifi','lefun-cam-wired','insteon-hub', 45 | 'echoplus','meross-dooropener','smartlife-bulb','xiaomi-ricecooker','ubell-doorbell', 46 | 'appletv','tplink-bulb','google-home','icsee-doorbell','t-wemo-plug','echospot', 47 | 'nest-tstat','sousvide','smartlife-remote','netatmo-weather-station','lgtv-wifi', 48 | 'wansview-cam-wired','xiaomi-plug','xiaomi-hub','lightify-hub','bosiwo-camera-wired', 49 | 'tplink-plug2','allure-speaker','honeywell-thermostat','smarter-coffee-mach','roku-tv', 50 | 'yi-camera','firetv','echodot','smartthings-hub','reolink-cam-wired','t-philips-hub', 51 | 'switchbot-hub','ring-doorbell','blink-camera','samsungtv-wired'] 52 | 53 | features = ['srcPort', 'destPort', 54 | 'bytes_out', 'num_pkts_out', 'bytes_in', 'num_pkts_in', 'f_ipt_mean', 55 | 'f_ipt_std', 'f_ipt_var', 'f_ipt_skew', 'f_ipt_kurtosis', 'f_b_mean', 56 | 'f_b_std', 'f_b_var', 'f_b_skew', 'f_b_kurtosis', 'duration', 'pr', 57 | 'domainId'] 58 | label = 'deviceId' 59 | 60 | 61 | #weeksToProcess = list(range(44,53)) + list(range(1,19)) 62 | weeks = [list(range(44,53)), list(range(1,10)), list(range(10,19))] 63 | weeksToProcess = weeks[0] + weeks[1] + weeks[2] 64 | #weeksToProcess = weeks[0] 65 | test = False 66 | #test = True 67 | 68 | modelName = "model_mar-apr/model.50-0.07.h5" 69 | 70 | df = pd.read_csv(sys.argv[1], parse_dates = ['time_start'], low_memory=False) 71 | df.fillna(0, inplace=True) 72 | 73 | 74 | trainCon = df['time_start'].dt.week.isin(weeks[int(sys.argv[2])]) 75 | 76 | testCons = pd.date_range('2019-11-01', '2020-05-01', freq = '1Y').tolist() 77 | lb = LabelBinarizer() 78 | lb.fit(range(1,42)) 79 | 80 | if test: 81 | model = keras.models.load_model(sys.argv[3]) 82 | 83 | for date in testCons: 84 | testCon = df['time_start'].dt.week == pd.Timestamp(date).week 85 | #testCon = df['time'].dt.date == pd.Timestamp(date).date 86 | #testDF = df[testCon] 87 | testDF = df 88 | 89 | X = testDF[features] 90 | y = testDF[label] 91 | 92 | if len(X) == 0: 93 | continue 94 | 95 | scaler = StandardScaler().fit(df[trainCon][features]) 96 | X = scaler.transform(X) 97 | 98 | labels = lb.transform(y) 99 | 100 | 101 | y_predict = model.predict(x=X, batch_size=128) 102 | np.save("yPred.npy", y_predict) 103 | np.save("yReal.npy", y.to_numpy()) 104 | 105 | #print(y_predict.shape, labels.shape, len(lb.classes_)) 106 | 107 | print(classification_report(labels.argmax(axis=1), y_predict.argmax(axis=1)))#, labels=devs)) 108 | print("Week", pd.Timestamp(date).week, 109 | "Accuracy: ", metrics.accuracy_score(labels.argmax(axis=1), y_predict.argmax(axis=1)), 110 | "F1: ", metrics.f1_score(labels.argmax(axis=1), y_predict.argmax(axis=1), average='micro')) 111 | 112 | else: 113 | X = df[trainCon][features] 114 | y = df[trainCon][label] 115 | 116 | scaler = StandardScaler().fit(X) 117 | X = scaler.transform(X) 118 | 119 | #labels = lb.transform(y) 120 | labels = y - 1#lb.transform(y) 121 | 122 | X_train, X_test, y_train, y_test = train_test_split(X, labels, test_size=0.2, stratify=labels) 123 | 124 | my_callbacks = [ 125 | #tf.keras.callbacks.EarlyStopping(patience=2), 126 | keras.callbacks.ModelCheckpoint(filepath='model.{epoch:02d}-{val_loss:.2f}.h5', monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1), 127 | #tf.keras.callbacks.TensorBoard(log_dir='./logs'), 128 | ] 129 | 130 | H = model.fit(X_train, y_train, validation_data=(X_test, y_test), batch_size=128, epochs=15, verbose=1, callbacks=my_callbacks) 131 | 132 | 133 | -------------------------------------------------------------------------------- /code/two_stage/getFeatures.py: -------------------------------------------------------------------------------- 1 | import os 2 | import subprocess 3 | import statistics 4 | import json 5 | from sklearn.feature_extraction.text import CountVectorizer 6 | from collections import Counter 7 | import pandas as pd 8 | from datetime import datetime 9 | import sys 10 | 11 | mac_to_device = {} 12 | mac_to_ip = {} 13 | with open("devices.txt", "r") as devices_list: 14 | for line in devices_list.readlines(): 15 | line = line.strip().split(",") 16 | mac_to_device[line[0].strip()] = (line[1].strip()) 17 | #mac_to_ip[line[1].strip()] = (line[2].strip()) 18 | 19 | # separate pcap file per each one hour window 20 | hourly_pcaps_dir = "pcaps/hourly/" 21 | 22 | all_ports_ever = set() 23 | all_domains_ever = set() 24 | all_cs_ever = set() 25 | 26 | #ports = [] 27 | #domains = [] 28 | #css = [] 29 | 30 | records = []] 31 | instances = {} 32 | #for mac in mac_to_device.keys(): 33 | for mac in [sys.argv[1]]: 34 | instances[mac] = [] 35 | print(f"Processing {mac}") 36 | if not os.path.exists(hourly_pcaps_dir + mac): 37 | print(hourly_pcaps_dir + mac + "does not exists") 38 | continue 39 | for hourly_instnace in os.listdir(hourly_pcaps_dir + mac): 40 | time_start = 0 41 | remote_ports = [] 42 | dns_names = [] 43 | cs = [] 44 | 45 | flow_volumes = [] 46 | flow_durations = [] 47 | dns_times = [] 48 | 49 | sleep_time = 3600 50 | latest_end = 0 51 | 52 | # joy tool from Cisco needs to be installed 53 | task = subprocess.Popen("joy bidir=1 tls=1 dns=1 " + hourly_pcaps_dir + mac + "/" + hourly_instnace, shell=True, stdout=subprocess.PIPE) 54 | data = task.stdout.read().decode() 55 | #assert task.wait() == 0 56 | if task.wait() != 0: 57 | print(f"Error processing {mac}/{hourly_instnace}") 58 | for line in data.split("\n"): 59 | if not len(line) > 0: 60 | continue 61 | json_obj = json.loads(line) 62 | if "sa" in json_obj: 63 | if time_start == 0: 64 | time_start = json_obj["time_start"] 65 | # Add remote port 66 | if "192.168" in json_obj["sa"] and "192.168" not in json_obj["da"]: 67 | if type(json_obj["dp"]) == type(0): 68 | remote_ports.append(str(json_obj["dp"])) 69 | all_ports_ever.add(str(json_obj["dp"])) 70 | 71 | # Calculate sleep time for the 3600 seconds over which pcap file is generated 72 | if json_obj["time_start"] >= latest_end: 73 | sleep_time -= json_obj["time_end"] - json_obj["time_start"] 74 | latest_end = json_obj["time_end"] 75 | else: 76 | if json_obj["time_end"] <= latest_end: 77 | pass 78 | else: 79 | sleep_time -= json_obj["time_end"] - latest_end 80 | latest_end = json_obj["time_end"] 81 | 82 | # Add DNS name 83 | if "dns" in json_obj: 84 | for item in json_obj["dns"]: 85 | try: 86 | dns_names.append(item["qn"]) 87 | all_domains_ever.add(item["qn"]) 88 | except: 89 | dns_names.append(item["rn"]) 90 | all_domains_ever.add(item["rn"]) 91 | dns_times.append(json_obj["time_start"]) 92 | 93 | # Add CS 94 | if "tls" in json_obj: 95 | if "cs" in json_obj["tls"]: 96 | cs.append(''.join(json_obj["tls"]["cs"])) 97 | all_cs_ever.add(''.join(json_obj["tls"]["cs"])) 98 | 99 | # Add flow information if a biflow 100 | if "bytes_in" in json_obj and "bytes_out" in json_obj: 101 | flow_volumes.append(json_obj["bytes_in"] + json_obj["bytes_out"]) 102 | flow_durations.append(json_obj["time_end"] - json_obj["time_start"]) 103 | 104 | if len(dns_times) > 2: 105 | temp = [] 106 | for i in range(1, len(dns_times)): 107 | temp.append(dns_times[i] - dns_times[i-1]) 108 | dns_interval = statistics.mean(temp) 109 | else: 110 | dns_interval = 0 111 | 112 | # print(mac, remote_ports) 113 | 114 | # Ensure we have read at least one flow from the pcap 115 | if len(flow_volumes) >= 1 and statistics.mean(flow_durations) > 0: 116 | instances[mac].append([remote_ports, dns_names, cs, statistics.mean(flow_volumes), 117 | statistics.mean(flow_durations), statistics.mean(flow_volumes) / statistics.mean(flow_durations), 118 | sleep_time, dns_interval]) 119 | record = {'filename': hourly_instnace, 'time_start': datetime.fromtimestamp(time_start), 'mac': mac, 'ports': dict(Counter(remote_ports)), 'dns': dict(Counter(dns_names)), 'cs': dict(Counter(cs)), 120 | 'volume_mean': statistics.mean(flow_volumes), 121 | 'flow_durations': statistics.mean(flow_durations), 'ratio': statistics.mean(flow_volumes) / statistics.mean(flow_durations), 122 | 'sleep_time': sleep_time, 'dns_interval': dns_interval} 123 | records.append(record) 124 | else: 125 | print("NOTIFY: possibly zero flows in the file ", hourly_instnace, " for ", mac) 126 | 127 | df = pd.DataFrame(records) 128 | df.to_csv(f'unsw_features_{mac}.csv', index=False) 129 | 130 | sys.exit() 131 | 132 | -------------------------------------------------------------------------------- /code/raw_flows/ModelBuilder.py: -------------------------------------------------------------------------------- 1 | import keras 2 | from keras.models import Sequential 3 | from keras.layers import Dense, Dropout, Activation, Flatten 4 | from keras.layers import Conv2D, Conv1D, MaxPooling2D, MaxPooling1D, LSTM, Bidirectional 5 | from keras.wrappers.scikit_learn import KerasClassifier 6 | from keras.callbacks import EarlyStopping 7 | from keras import backend as K 8 | 9 | from sklearn.preprocessing import LabelBinarizer 10 | from sklearn.model_selection import train_test_split 11 | from sklearn.metrics import classification_report 12 | from sklearn import metrics 13 | 14 | import sys, os, time, json, datetime, glob 15 | 16 | from logzero import logger 17 | import warnings 18 | import tensorflow as tf 19 | 20 | import numpy as np 21 | import pandas as pd 22 | from pathlib import Path 23 | 24 | os.environ["CUDA_VISIBLE_DEVICES"] = "-1" 25 | 26 | warnings.simplefilter("ignore", category=FutureWarning) 27 | 28 | 29 | class ModelBuilder(object): 30 | 31 | def __init__(self, options): 32 | self.options = options 33 | 34 | def inferParameters(self): 35 | if self.options.label == "deviceId": 36 | self.outputSize = 59 37 | else: 38 | self.outputSize = 7 39 | 40 | if self.options.perDeviceModel: 41 | self.outputSize = 1 42 | self.activation = "sigmoid" 43 | self.loss = keras.losses.binary_crossentropy 44 | else: 45 | self.activation = "softmax" 46 | self.loss = keras.losses.categorical_crossentropy 47 | 48 | if self.options.modelType == "nn": 49 | self.input = 19 50 | elif self.options.modelType == "lstm": 51 | self.input = (1, 19) 52 | elif self.options.modelType == "conv1d": 53 | self.input = (19, 1) 54 | 55 | self.optimizer = 'adam' 56 | self.metrics = ['accuracy'] 57 | 58 | def getSciKitModel(self): 59 | return KerasClassifier(build_fn=self.getModel) 60 | 61 | def getModel(self): 62 | self.model = Sequential() 63 | 64 | if self.options.modelType == "nn": 65 | self._getNNModel() 66 | elif self.options.modelType == "lstm": 67 | self._getLSTMModel() 68 | elif self.options.modelType == "conv1d": 69 | self._getConv1DModel() 70 | 71 | self._addOutputLayer() 72 | self._compileModel() 73 | 74 | return self.model 75 | 76 | def _addOutputLayer(self): 77 | self.model.add(Dense(self.outputSize, activation = self.activation)) 78 | 79 | def _compileModel(self): 80 | self.model.compile(loss = self.loss, optimizer = self.optimizer, 81 | metrics = self.metrics) 82 | 83 | def _getNNModel(self): 84 | # create model 85 | self.model.add(Dense(32, input_dim = self.input, activation = 'relu')) 86 | self.model.add(Dense(64, activation = 'relu')) 87 | self.model.add(Dense(128, activation = 'relu')) 88 | self.model.add(Dense(256, activation = 'relu')) 89 | 90 | def _getReverseNNModel(self): 91 | self.model.add(Dense(256, input_dim = self.input, activation = 'relu')) 92 | self.model.add(Dense(128, activation = 'relu')) 93 | self.model.add(Dense(64, activation = 'relu')) 94 | self.model.add(Dense(256, activation = 'relu')) 95 | 96 | 97 | 98 | def _getLSTMModel(self): 99 | self.model.add(LSTM(200, activation = 'relu', return_sequences = True, input_shape = self.input)) 100 | self.model.add(LSTM(100, activation = 'relu', return_sequences = True)) 101 | self.model.add(LSTM(50, activation = 'relu', return_sequences = True)) 102 | self.model.add(LSTM(25, activation = 'relu')) 103 | self.model.add(Dropout(0.2)) 104 | 105 | def _getConv1DModel(self): 106 | self.model.add(Conv1D(64, 3, activation = 'relu', input_shape = self.input)) 107 | self.model.add(Conv1D(64, 3, activation = 'relu')) 108 | self.model.add(Dropout(0.2)) 109 | self.model.add(MaxPooling1D()) 110 | self.model.add(Flatten()) 111 | self.model.add(Dense(100, activation = 'relu')) 112 | 113 | def freezeLayers(self, numOfLayers): 114 | frozenLayers = 0 115 | for layer in self.model.layers: 116 | if layer.name.startswith(("dense", "lstm", "conv")): 117 | layer.trainable = False 118 | frozenLayers+= 1 119 | 120 | if frozenLayers == numOfLayers: 121 | break 122 | 123 | self._compileModel() 124 | 125 | 126 | if __name__ == '__main__': 127 | model = Sequential() 128 | model.add(Conv2D(64, (3,3), activation='relu', strides=(2,2), input_shape=(10,250,1))) 129 | model.add(MaxPooling2D((2, 2), padding='same')) 130 | model.add(Conv2D(32, (2, 2), activation='relu', strides=(2, 2))) 131 | model.add(MaxPooling2D((2, 2), padding='same')) 132 | model.add(Flatten()) 133 | model.add(Dropout(0.5)) 134 | #model.add(Dense(64, activation='relu')) 135 | #model.add(Bidirectional(LSTM(32, activation = 'relu', return_sequences=True), input_shape=(64,1))) 136 | #model.add(Bidirectional(LSTM(32, activation = 'relu', return_sequences=False))) 137 | #model.add(Dropout(0.5)) 138 | model.add(Dense(41, activation='softmax')) 139 | 140 | model.compile(loss="categorical_crossentropy", optimizer='adam', 141 | metrics=["accuracy"]) 142 | 143 | y = [] 144 | packets = [] 145 | i = 0 146 | 147 | devs = ['appkettle','blink-security-hub','bosiwo-camera-wifi','lefun-cam-wired','insteon-hub', 148 | 'echoplus','meross-dooropener','smartlife-bulb','xiaomi-ricecooker','ubell-doorbell', 149 | 'appletv','tplink-bulb','google-home','icsee-doorbell','t-wemo-plug','echospot', 150 | 'nest-tstat','sousvide','smartlife-remote','netatmo-weather-station','lgtv-wifi', 151 | 'wansview-cam-wired','xiaomi-plug','xiaomi-hub','lightify-hub','bosiwo-camera-wired', 152 | 'tplink-plug2','allure-speaker','honeywell-thermostat','smarter-coffee-mach','roku-tv', 153 | 'yi-camera','firetv','echodot','smartthings-hub','reolink-cam-wired','t-philips-hub', 154 | 'switchbot-hub','ring-doorbell','blink-camera','samsungtv-wired'] 155 | 156 | 157 | #weeksToProcess = list(range(44,53)) + list(range(1,19)) 158 | weeks = [list(range(44,53)), list(range(1,10)), list(range(10,19))] 159 | weeksToProcess = weeks[0] + weeks[1] + weeks[2] 160 | #weeksToProcess = weeks[2] 161 | #test = False 162 | test = True 163 | 164 | modelName = "model_mar-apr/model.50-0.07.h5" 165 | 166 | #a = np.fromfile('/data/roman/flows/10_3_2020-03-12_21.06.19_192.168.20.192.pcap.bin', dtype='uint8') 167 | for week in weeksToProcess: 168 | print(f"Week {week}") 169 | y = [] 170 | packets = [] 171 | i = 0 172 | 173 | for filename in Path(sys.argv[1]).iterdir(): 174 | devId, numRec, date, time, ip = filename.name.split('_') 175 | #if date.startswith("2019-12-"): # or date.startswith("2019-12-"): 176 | if pd.Timestamp(date).week == week: #in weeksToProcess: 177 | #if pd.Timestamp(date).week in weeksToProcess: 178 | y.extend([int(devId)]*int(numRec)) 179 | 180 | #print(devId, numRec, date, time, ip, filename.name) 181 | x = np.fromfile(filename, dtype='uint8') 182 | x = np.reshape(x, (int(numRec), 10, 250, 1)) 183 | packets.append(x) 184 | 185 | i+=1 186 | 187 | #if i > 100: 188 | # break 189 | 190 | X = np.concatenate(packets) 191 | print(X.shape) 192 | 193 | X = np.array(X, dtype='float') / 255.0 194 | 195 | lb = LabelBinarizer() 196 | lb.fit(range(1,42)) 197 | labels = lb.transform(y) 198 | #labels = keras.utils.to_categorical(y, num_classes = 41) 199 | 200 | if test: 201 | model = keras.models.load_model(modelName) 202 | y_predict = model.predict(x=X, batch_size=128) 203 | 204 | print(y_predict.shape, labels.shape, len(lb.classes_)) 205 | 206 | print(classification_report(labels.argmax(axis=1), y_predict.argmax(axis=1)))#, labels=devs)) 207 | print("Accuracy: ", metrics.accuracy_score(labels.argmax(axis=1), y_predict.argmax(axis=1)), 208 | "F1: ", metrics.f1_score(labels.argmax(axis=1), y_predict.argmax(axis=1))) 209 | 210 | else: # train 211 | model = keras.models.load_model('model_jan-part-feb/model.50-0.05.h5') 212 | (trainX, testX, trainY, testY) = train_test_split(X, labels, 213 | test_size=0.25, stratify=labels, random_state=42) 214 | 215 | my_callbacks = [ 216 | #tf.keras.callbacks.EarlyStopping(patience=2), 217 | keras.callbacks.ModelCheckpoint(filepath='model.{epoch:02d}-{val_loss:.2f}.h5', monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1), 218 | #tf.keras.callbacks.TensorBoard(log_dir='./logs'), 219 | ] 220 | 221 | H = model.fit(trainX, trainY, validation_data=(testX, testY), batch_size=128, epochs=50, verbose=1, callbacks=my_callbacks) 222 | #H = model.fit(X, labels, epochs=10, verbose=2) 223 | break 224 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | --------------------------------------------------------------------------------