├── COPYING ├── README └── ct-honeybee /COPYING: -------------------------------------------------------------------------------- 1 | Creative Commons Legal Code 2 | 3 | CC0 1.0 Universal 4 | 5 | CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE 6 | LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN 7 | ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS 8 | INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES 9 | REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS 10 | PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM 11 | THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED 12 | HEREUNDER. 13 | 14 | Statement of Purpose 15 | 16 | The laws of most jurisdictions throughout the world automatically confer 17 | exclusive Copyright and Related Rights (defined below) upon the creator 18 | and subsequent owner(s) (each and all, an "owner") of an original work of 19 | authorship and/or a database (each, a "Work"). 20 | 21 | Certain owners wish to permanently relinquish those rights to a Work for 22 | the purpose of contributing to a commons of creative, cultural and 23 | scientific works ("Commons") that the public can reliably and without fear 24 | of later claims of infringement build upon, modify, incorporate in other 25 | works, reuse and redistribute as freely as possible in any form whatsoever 26 | and for any purposes, including without limitation commercial purposes. 27 | These owners may contribute to the Commons to promote the ideal of a free 28 | culture and the further production of creative, cultural and scientific 29 | works, or to gain reputation or greater distribution for their Work in 30 | part through the use and efforts of others. 31 | 32 | For these and/or other purposes and motivations, and without any 33 | expectation of additional consideration or compensation, the person 34 | associating CC0 with a Work (the "Affirmer"), to the extent that he or she 35 | is an owner of Copyright and Related Rights in the Work, voluntarily 36 | elects to apply CC0 to the Work and publicly distribute the Work under its 37 | terms, with knowledge of his or her Copyright and Related Rights in the 38 | Work and the meaning and intended legal effect of CC0 on those rights. 39 | 40 | 1. Copyright and Related Rights. A Work made available under CC0 may be 41 | protected by copyright and related or neighboring rights ("Copyright and 42 | Related Rights"). Copyright and Related Rights include, but are not 43 | limited to, the following: 44 | 45 | i. the right to reproduce, adapt, distribute, perform, display, 46 | communicate, and translate a Work; 47 | ii. moral rights retained by the original author(s) and/or performer(s); 48 | iii. publicity and privacy rights pertaining to a person's image or 49 | likeness depicted in a Work; 50 | iv. rights protecting against unfair competition in regards to a Work, 51 | subject to the limitations in paragraph 4(a), below; 52 | v. rights protecting the extraction, dissemination, use and reuse of data 53 | in a Work; 54 | vi. database rights (such as those arising under Directive 96/9/EC of the 55 | European Parliament and of the Council of 11 March 1996 on the legal 56 | protection of databases, and under any national implementation 57 | thereof, including any amended or successor version of such 58 | directive); and 59 | vii. other similar, equivalent or corresponding rights throughout the 60 | world based on applicable law or treaty, and any national 61 | implementations thereof. 62 | 63 | 2. Waiver. To the greatest extent permitted by, but not in contravention 64 | of, applicable law, Affirmer hereby overtly, fully, permanently, 65 | irrevocably and unconditionally waives, abandons, and surrenders all of 66 | Affirmer's Copyright and Related Rights and associated claims and causes 67 | of action, whether now known or unknown (including existing as well as 68 | future claims and causes of action), in the Work (i) in all territories 69 | worldwide, (ii) for the maximum duration provided by applicable law or 70 | treaty (including future time extensions), (iii) in any current or future 71 | medium and for any number of copies, and (iv) for any purpose whatsoever, 72 | including without limitation commercial, advertising or promotional 73 | purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each 74 | member of the public at large and to the detriment of Affirmer's heirs and 75 | successors, fully intending that such Waiver shall not be subject to 76 | revocation, rescission, cancellation, termination, or any other legal or 77 | equitable action to disrupt the quiet enjoyment of the Work by the public 78 | as contemplated by Affirmer's express Statement of Purpose. 79 | 80 | 3. Public License Fallback. Should any part of the Waiver for any reason 81 | be judged legally invalid or ineffective under applicable law, then the 82 | Waiver shall be preserved to the maximum extent permitted taking into 83 | account Affirmer's express Statement of Purpose. In addition, to the 84 | extent the Waiver is so judged Affirmer hereby grants to each affected 85 | person a royalty-free, non transferable, non sublicensable, non exclusive, 86 | irrevocable and unconditional license to exercise Affirmer's Copyright and 87 | Related Rights in the Work (i) in all territories worldwide, (ii) for the 88 | maximum duration provided by applicable law or treaty (including future 89 | time extensions), (iii) in any current or future medium and for any number 90 | of copies, and (iv) for any purpose whatsoever, including without 91 | limitation commercial, advertising or promotional purposes (the 92 | "License"). The License shall be deemed effective as of the date CC0 was 93 | applied by Affirmer to the Work. Should any part of the License for any 94 | reason be judged legally invalid or ineffective under applicable law, such 95 | partial invalidity or ineffectiveness shall not invalidate the remainder 96 | of the License, and in such case Affirmer hereby affirms that he or she 97 | will not (i) exercise any of his or her remaining Copyright and Related 98 | Rights in the Work or (ii) assert any associated claims and causes of 99 | action with respect to the Work, in either case contrary to Affirmer's 100 | express Statement of Purpose. 101 | 102 | 4. Limitations and Disclaimers. 103 | 104 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 105 | surrendered, licensed or otherwise affected by this document. 106 | b. Affirmer offers the Work as-is and makes no representations or 107 | warranties of any kind concerning the Work, express, implied, 108 | statutory or otherwise, including without limitation warranties of 109 | title, merchantability, fitness for a particular purpose, non 110 | infringement, or the absence of latent or other defects, accuracy, or 111 | the present or absence of errors, whether or not discoverable, all to 112 | the greatest extent permissible under applicable law. 113 | c. Affirmer disclaims responsibility for clearing rights of other persons 114 | that may apply to the Work or any use thereof, including without 115 | limitation any person's Copyright and Related Rights in the Work. 116 | Further, Affirmer disclaims responsibility for obtaining any necessary 117 | consents, permissions or other rights required for any use of the 118 | Work. 119 | d. Affirmer understands and acknowledges that Creative Commons is not a 120 | party to this document and has no duty or obligation with respect to 121 | this CC0 or use of the Work. 122 | -------------------------------------------------------------------------------- /README: -------------------------------------------------------------------------------- 1 | The Certificate Transparency Honeybee (ct-honeybee) is a lightweight 2 | program that retrieves signed tree heads (STHs) from Certificate 3 | Transparency logs and uploads them to auditors. 4 | 5 | You can help strengthen the integrity of the Certificate Transparency 6 | ecosystem by running ct-honeybee on your workstation/server/toaster every 7 | hour or so (pick a random minute so that not everyone runs ct-honeybee 8 | at the same time). Running ct-honeybee from many different Internet 9 | vantage points increases the likelihood of detecting a misbehaving log 10 | which has presented a different view of the log to different clients. 11 | 12 | 13 | INSTALLATION 14 | 15 | Python 3 is required. 16 | 17 | Install ct-honeybee and put it in a cron job to run once an hour or so 18 | (pick a random minute so that not everyone runs ct-honeybee at the 19 | same time). 20 | 21 | ct-honeybee is stateless and won't write to your filesystem. 22 | 23 | 24 | LOGS 25 | 26 | All logs trusted or pending inclusion by Chrome are audited by 27 | ct-honeybee. Currently the list is hard-coded in the source code. 28 | 29 | 30 | AUDITORS 31 | 32 | ct-honeybee uploads STHs to the following auditors: 33 | 34 | certspotter.com 35 | ct.grahamedgecombe.com 36 | 37 | If you run an auditor that implements the sth-pollination endpoint 38 | described in Section 8.2 of draft-ietf-trans-gossip-00, please get in 39 | touch and we will add you to ct-honeybee. 40 | 41 | 42 | TECHNICAL OPERATION 43 | 44 | 1. For each log: fetch the latest STH and add it to the list of STHs. 45 | For simplicity, signatures are not checked; we leave this job to the 46 | auditors. 47 | 48 | 2. Shuffle the list of auditors. 49 | 50 | 3. For each auditor: upload the list of STHs to the auditor using the 51 | protocol described in Section 8.2 of draft-ietf-trans-gossip-00. 52 | Add each returned STH to the list of STHs so they get pollinated 53 | to subsequent auditors. Since we shuffle the list of auditors, 54 | we will pollinate in a different order each time ct-honeybee is run. 55 | 56 | 57 | LEGALESE 58 | 59 | Written in 2017 by Opsmate, Inc. d/b/a SSLMate 60 | 61 | To the extent possible under law, the author(s) have dedicated all 62 | copyright and related and neighboring rights to this software to the 63 | public domain worldwide. This software is distributed without any 64 | warranty. 65 | 66 | You should have received a copy of the CC0 Public 67 | Domain Dedication along with this software. If not, see 68 | . 69 | -------------------------------------------------------------------------------- /ct-honeybee: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # 4 | # The Certificate Transparency Honeybee (ct-honeybee) is a lightweight 5 | # program that retrieves signed tree heads (STHs) from Certificate 6 | # Transparency logs and uploads them to auditors. 7 | # 8 | # You can help strengthen the integrity of the Certificate Transparency 9 | # ecosystem by running ct-honeybee on your workstation/server/toaster every 10 | # hour or so (pick a random minute so that not everyone runs ct-honeybee 11 | # at the same time). Running ct-honeybee from many different Internet 12 | # vantage points increases the likelihood of detecting a misbehaving log 13 | # which has presented a different view of the log to different clients. 14 | # 15 | # Written in 2017 by Opsmate, Inc. d/b/a SSLMate 16 | # 17 | # To the extent possible under law, the author(s) have dedicated all 18 | # copyright and related and neighboring rights to this software to the 19 | # public domain worldwide. This software is distributed without any 20 | # warranty. 21 | # 22 | # You should have received a copy of the CC0 Public 23 | # Domain Dedication along with this software. If not, see 24 | # . 25 | # 26 | 27 | import json 28 | import random 29 | import re 30 | import socket 31 | import ssl 32 | import sys 33 | import time 34 | import urllib.request 35 | 36 | version = "2021-09-14" 37 | 38 | log_lists = [ 39 | "https://loglist.certspotter.org/honeybee.json", 40 | ] 41 | 42 | auditors = [ 43 | "certspotter.com", 44 | "ct.grahamedgecombe.com", 45 | ] 46 | 47 | user_agent = "ct-honeybee/" + version + " (https://github.com/SSLMate/ct-honeybee/)" 48 | log_list_timeout = 60 49 | log_timeout = 15 50 | auditor_timeout = 60 51 | 52 | time_format = '%Y-%m-%d %H:%M:%S %z' 53 | base64_re = re.compile('^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{4}|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)$') 54 | 55 | def is_base64(obj): 56 | return isinstance(obj, str) and base64_re.search(obj) is not None 57 | 58 | def is_sth(obj): 59 | return isinstance(obj, dict) \ 60 | and "sth_version" in obj and isinstance(obj["sth_version"], int) \ 61 | and "tree_size" in obj and isinstance(obj["tree_size"], int) \ 62 | and "timestamp" in obj and isinstance(obj["timestamp"], int) \ 63 | and "sha256_root_hash" in obj and is_base64(obj["sha256_root_hash"]) \ 64 | and "tree_head_signature" in obj and is_base64(obj["tree_head_signature"]) \ 65 | and "log_id" in obj and is_base64(obj["log_id"]) 66 | 67 | def is_pollen(obj): 68 | return isinstance(obj, dict) \ 69 | and "sths" in obj and isinstance(obj["sths"], list) 70 | 71 | def is_known_log(arg): 72 | for _, log_id in logs: 73 | if arg == log_id: 74 | return True 75 | return False 76 | 77 | def is_same_sth(a, b): 78 | return a["log_id"] == b["log_id"] \ 79 | and a["tree_size"] == b["tree_size"] \ 80 | and a["timestamp"] == b["timestamp"] \ 81 | and a["sha256_root_hash"] == b["sha256_root_hash"] 82 | 83 | def has_sth(sths, target_sth): 84 | return any(sth for sth in sths if is_same_sth(sth, target_sth)) 85 | 86 | pollen = { "sths": [] } 87 | logs = [] 88 | 89 | for log_list_url in log_lists: 90 | try: 91 | req = urllib.request.Request(log_list_url, headers={"User-Agent": user_agent}) 92 | with urllib.request.urlopen(req, timeout=log_list_timeout) as response: 93 | log_list = json.loads(response.read().decode("utf-8")) 94 | for operator in log_list["operators"]: 95 | for log in operator["logs"]: 96 | logs.append([ log["url"], log["log_id"] ]) 97 | except Exception as err: 98 | print("[%s] ct-honeybee: log list error: %s: %s: %s" % (time.strftime(time_format), log_list_url, type(err).__name__, err), file=sys.stderr) 99 | 100 | # Disable certificate validation. Unfortunately, there is no guarantee 101 | # that logs use a certificate from a widely-trusted CA. Fortunately, 102 | # all responses are signed by logs and verified by auditors, so there 103 | # is technically no need for certificate validation. 104 | try: 105 | _create_unverified_https_context = ssl._create_unverified_context 106 | except AttributeError: 107 | pass 108 | else: 109 | ssl._create_default_https_context = _create_unverified_https_context 110 | 111 | for log_url, log_id in logs: 112 | try: 113 | req = urllib.request.Request(log_url + "ct/v1/get-sth", 114 | data=None, headers={"User-Agent": ""}) 115 | with urllib.request.urlopen(req, timeout=log_timeout) as response: 116 | sth = json.loads(response.read().decode("utf-8")) 117 | if isinstance(sth, dict): 118 | sth["sth_version"] = 0 119 | sth["log_id"] = log_id 120 | if is_sth(sth): 121 | pollen["sths"].append(sth) 122 | except Exception as err: 123 | print("[%s] ct-honeybee: Log error: %s: %s: %s" % (time.strftime(time_format), log_url, type(err).__name__, err), file=sys.stderr) 124 | 125 | random.shuffle(auditors) 126 | 127 | for auditor_domain in auditors: 128 | try: 129 | req = urllib.request.Request("https://" + auditor_domain + "/.well-known/ct/v1/sth-pollination", 130 | data=json.dumps(pollen).encode("utf8"), 131 | headers={"Content-Type": "application/json", "User-Agent": user_agent}) 132 | with urllib.request.urlopen(req, timeout=auditor_timeout) as response: 133 | more_pollen = json.loads(response.read().decode("utf-8")) 134 | if is_pollen(more_pollen): 135 | for sth in more_pollen["sths"]: 136 | if is_sth(sth) and is_known_log(sth["log_id"]) and not has_sth(pollen["sths"], sth): 137 | pollen["sths"].append(sth) 138 | except Exception as err: 139 | print("[%s] ct-honeybee: Auditor error: %s: %s: %s" % (time.strftime(time_format), auditor_domain, type(err).__name__, err), file=sys.stderr) 140 | 141 | #print(json.dumps(pollen)) 142 | --------------------------------------------------------------------------------