├── BCC.md ├── LICENSE ├── METADATA ├── MODULE_LICENSE_APACHE2 ├── README.md ├── README.version ├── TODO ├── addons ├── bashrc ├── bashrc.common ├── bashrc.silent ├── build-debian-tar ├── device-umount-all ├── device-unpack ├── get_kvers.sh ├── run ├── run-command └── run.common ├── adeb ├── androdeb ├── bcc ├── build-bcc.sh └── misc │ ├── 0001-trace-file-path-inode-number.patch │ ├── android-futex-contention-record │ ├── android-futex-contention-report │ ├── android-futex-contention.py │ ├── build-bcc-minimal │ ├── build-on-target.sh │ └── build.sh │ ├── build-gcc.sh │ ├── futex-contention.py │ ├── inodemap │ └── lockstat.py ├── buildimage ├── buildstrap ├── packages ├── bcc ├── compilers ├── editors ├── others ├── scheduler └── tracers └── utils ├── banners ├── remote └── support /BCC.md: -------------------------------------------------------------------------------- 1 | BCC (BPF compiler collection) for Android 2 | ========================================= 3 | 4 | Introduction 5 | ------------ 6 | BCC is a compiler and a toolkit, containing powerful kernel tracing tools that 7 | trace at the lowest levels, including adding hooks to functions in kernel space 8 | and user space to deeply understand system behavior while being low in 9 | overhead. [Here's a presentation with an 10 | overview](http://www.joelfernandes.org/resources/bcc-ospm.pdf) and visit [BCC's 11 | project page](https://github.com/iovisor/bcc) for the official BCC 12 | documentation. 13 | 14 | Quick Start 15 | ----------- 16 | adeb is the primary vehicle for running BCC on Android. It supports preparing 17 | the target Android device, cloning and building BCC on device, and other setup. 18 | Take a look a quick look at [adeb 19 | README](https://github.com/joelagnel/adeb/blob/master/README.md) so that you're 20 | familiar with what it is. 21 | 22 | To download a prebuilt filesystem with BCC already built/installed for an ARM 23 | 64-bit device, you can just run: 24 | ``` 25 | adeb prepare --full 26 | ``` 27 | 28 | This downloads a prebuilt filesystem for ARM 64-bit and sets it up on your device. 29 | 30 | If your device is an architecture other than ARM64, see the [Other 31 | Architectures 32 | section](https://github.com/joelagnel/adeb/blob/master/BCC.md#other-architectures-other-than-arm64) 33 | 34 | Now to run BCC, just start an adeb shell: `adeb shell`. This uses adb 35 | in the background to start a shell into your adeb environment. Try running 36 | `opensnoop` or any of the other BCC tracers to confirm that the setup worked 37 | correctly. 38 | 39 | If building your own kernel, following are the kernel requirements: 40 | 41 | You need kernel 4.9 or newer. Anything less needs backports. Your kernel also 42 | needs to be built with the following config options at the minimum: 43 | ``` 44 | CONFIG_KPROBES=y 45 | CONFIG_KPROBE_EVENT=y 46 | CONFIG_BPF_SYSCALL=y 47 | CONFIG_IKHEADERS=m 48 | ``` 49 | Optionally, 50 | ``` 51 | CONFIG_UPROBES=y 52 | CONFIG_UPROBE_EVENT=y 53 | ``` 54 | 55 | Build BCC during adeb install (Optional) 56 | -------------------------------------------- 57 | If you would like the latest upstream BCC built and installed on your Android 58 | device, you can run: 59 | ``` 60 | adeb prepare --build --bcc 61 | ``` 62 | NOTE: This is a slow process and can take a long time. Since it not only builds 63 | BCC but also installs all non-BCC debian packages onto the filesystem and configures them. 64 | 65 | Other Architectures (other than ARM64) 66 | ----------------------- 67 | By default adeb assumes the target Android device is based on ARM64 68 | processor architecture. For other architectures, use the --arch option. For 69 | example for x86_64 architecture, run: 70 | ``` 71 | adeb prepare --arch amd64 --build --bcc 72 | ``` 73 | Note: The --download option ignores the --arch flag. This is because we only 74 | provide pre-built filesystems for ARM64 at the moment. 75 | Note: For arch other than 64-bit ARM, you have to pass --build, because 76 | prebuilt filesystems are not available for other non-ARM64 architectures at the 77 | moment. 78 | 79 | Common Issues 80 | ------------- 81 | Here are some common issues you may face when running different BCC tools. 82 | 83 | * Issue 1: Headers are missing on the target device. 84 | 85 | Symptom: This will usually result in an error like the following: 86 | ``` 87 | root@localhost:/# criticalstat 88 | 89 | In file included from :2 90 | In file included from /virtual/include/bcc/bpf.h:12: 91 | In file included from include/linux/types.h:5: 92 | include/uapi/linux/types.h:4:10: fatal error: 'asm/types.h' file not found 93 | 94 | #include 95 | 96 | ^~~~~~~~~~~~~ 97 | 1 error generated. 98 | Traceback (most recent call last): 99 | 100 | File "./criticalstat.py", line 138, in 101 | b = BPF(text=bpf_text) 102 | File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__ 103 | raise Exception("Failed to compile BPF text:\n%s" % text) 104 | Exception: Failed to compile BPF text: 105 | 106 | #include 107 | #include 108 | #include 109 | 110 | extern char _stext[]; 111 | ``` 112 | 113 | * Issue 2: `CONFIG_KPROBES` isn't enabled. 114 | 115 | Symptom: This will result in an error like the following: 116 | ``` 117 | Traceback (most recent call last): 118 | File "/usr/share/bcc/tools/cachetop", line 263, in 119 | curses.wrapper(handle_loop, args) 120 | File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper 121 | return func(stdscr, *args, **kwds) 122 | File "/usr/share/bcc/tools/cachetop", line 172, in handle_loop 123 | b.attach_kprobe(event="add_to_page_cache_lru", fn_name="do_count") 124 | File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 543, in 125 | attach_kprobe 126 | fn = self.load_func(fn_name, BPF.KPROBE) 127 | File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 355, in 128 | load_func 129 | (func_name, errstr)) 130 | Exception: Failed to load BPF program do_count: Invalid argument 131 | ``` 132 | 133 | * Issue 3: `CONFIG_BPF_SYSCALL` isn't enabled. 134 | 135 | Symptom: This may result in a compilation error like the following: 136 | ``` 137 | root@localhost:/# cachetop 138 | Traceback (most recent call last): 139 | File "/usr/share/bcc/tools/cachetop", line 263, in 140 | curses.wrapper(handle_loop, args) 141 | File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper 142 | return func(stdscr, *args, **kwds) 143 | File "/usr/share/bcc/tools/cachetop", line 171, in handle_loop 144 | b = BPF(text=bpf_text) 145 | File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__ 146 | raise Exception("Failed to compile BPF text:\n%s" % text) 147 | Exception: Failed to compile BPF text: 148 | 149 | 150 | #include 151 | struct key_t { 152 | u64 ip; 153 | u32 pid; 154 | u32 uid; 155 | char comm[16]; 156 | }; 157 | 158 | BPF_HASH(counts, struct key_t); 159 | 160 | int do_count(struct pt_regs *ctx) { 161 | struct key_t key = {}; 162 | u64 zero = 0 , *val; 163 | u64 pid = bpf_get_current_pid_tgid(); 164 | u32 uid = bpf_get_current_uid_gid(); 165 | 166 | key.ip = PT_REGS_IP(ctx); 167 | key.pid = pid & 0xFFFFFFFF; 168 | key.uid = uid & 0xFFFFFFFF; 169 | bpf_get_current_comm(&(key.comm), 16); 170 | 171 | val = counts.lookup_or_init(&key, &zero); // update counter 172 | (*val)++; 173 | return 0; 174 | } 175 | ``` 176 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright [yyyy] [name of copyright owner] 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | -------------------------------------------------------------------------------- /METADATA: -------------------------------------------------------------------------------- 1 | name: "adeb" 2 | description: "adeb (also known as androdeb) provides a powerful Linux shell 3 | environment where one can run popular and mainstream Linux tracing, compiling, 4 | editing and other development tools on an existing Android device. All the 5 | commands typically available on a modern Linux system are supported in adeb." 6 | 7 | third_party { 8 | url { 9 | type: HOMEPAGE 10 | value: "https://github.com/joelagnel/adeb/README.md" 11 | } 12 | url { 13 | type: GIT 14 | value: "https://github.com/joelagnel/adeb.git" 15 | } 16 | version: "v0.99f" 17 | last_upgrade_date { year: 2018 month: 7 day: 25 } 18 | license_type: NOTICE 19 | } 20 | -------------------------------------------------------------------------------- /MODULE_LICENSE_APACHE2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/joelagnel/adeb/3671fa28fb8084714b5710da3d3eaa2c1bff4f5a/MODULE_LICENSE_APACHE2 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | > Note: The project 2 | is not maintained for new Android and Debian 3 | releases, but likely works with little effort. 4 | I suggest reaching out to me if you are 5 | interested in maintaining it and I will decide. 6 | 7 | adeb 8 | -------- 9 | 10 | **adeb** (also known as **androdeb**) provides a powerful Linux shell 11 | environment where one can run popular and mainstream Linux tracing, compiling, 12 | editing and other development tools on an existing Android device. All the 13 | commands typically available on a modern Linux system are supported in 14 | adeb. 15 | 16 | Usecases 17 | -------- 18 | 1. Powerful development environment with all tools ready to go (editors, 19 | compilers, tracers, perl/python etc) for your on-device development. 20 | 21 | 2. No more cross-compiler needed: Because it comes with gcc and clang, one can 22 | build target packages natively without needing to do any cross compilation. We even 23 | ship git, and have support to run apt-get to get any missing development packages 24 | from the web. 25 | 26 | 3. Using these one can run popular tools such as BCC that are difficult to run 27 | in an Android environment due to lack of packages, dependencies and 28 | cross-compilation needed for their operation. [Check BCC on Android using 29 | adeb](https://github.com/joelagnel/adeb/blob/master/BCC.md) for more 30 | information on that. 31 | 32 | 4. No more crippled tools: Its often a theme to build a static binary with 33 | features disabled, because you couldn't cross-compile the feature's dependencies. One 34 | classic example is perf. However, thanks to adeb, we can build perf natively 35 | on device without having to cripple it. 36 | 37 | Requirements for running 38 | ------------------------ 39 | Target: 40 | An ARM64 android N or later device which has "adb root" supported. Typically 41 | this is a build in a userdebug configuration. Device should have atleast 2 GB 42 | free space in the data partition. If you would like to use other architectures, 43 | see the [Other Architectures](https://github.com/joelagnel/adeb/blob/master/README.md#how-to-use-adeb-for-other-architectures-other-than-arm64) section. 44 | 45 | You can also use ssh to run on non-android systems. The system must still be 46 | rooted and has 2 GB of free space. 47 | 48 | Host: 49 | A machine running recent Ubuntu or Debian, with 4GB of memory and 4GB free space. 50 | Host needs debootstrap and qemu-debootstrap packages. 51 | To install it, run `sudo apt-get install qemu-user-static debootstrap`. 52 | Other distributions may work but they are not tested. 53 | 54 | Quick Start Instructions 55 | ------------------------ 56 | * First clone this repository into adeb and cd into it. 57 | ``` 58 | cd adeb 59 | 60 | # Add some short cuts: 61 | sudo ln -s $(pwd)/adeb /usr/bin/adeb 62 | 63 | # Cached image downloads result in a huge speed-up. These are automatic if you 64 | # cloned the repository using git. However, if you downloaded the repository 65 | # as a zip file (or you want to host images elsewere), you could set the 66 | # ADEB_REPO_URL environment variable in your bashrc file. 67 | # Disclaimer: Google is not liable for the below URL and this 68 | # is just an example. 69 | export ADEB_REPO_URL="github.com/joelagnel/adeb/" 70 | ``` 71 | 72 | * Installing adeb onto your device: 73 | First make sure device is connected to system 74 | Then run, for the base image: 75 | ``` 76 | adeb prepare 77 | ``` 78 | The previous command only downloads and installs the base image. 79 | Instead if you want to download and install the full image, do: 80 | ``` 81 | adeb prepare --full 82 | ``` 83 | 84 | * Now run adeb shell to enter your new environment!: 85 | ``` 86 | adeb shell 87 | ``` 88 | 89 | * Once done, hit `CTRL + D` and you will exit out of the shell. 90 | To remove adeb from the device, run: 91 | ``` 92 | adeb remove 93 | ``` 94 | If you have multiple devices connected, please add `-s `. 95 | Serial numbers of all devices connected can be obtained by `adb devices`. 96 | 97 | * To update an existing adeb clone on your host, run: 98 | ``` 99 | adeb git-pull 100 | ``` 101 | 102 | * To use ssh instead of adb to communicate with the target 103 | ``` 104 | adeb --ssh --sshpass 105 | ``` 106 | If you use keys to authenticate then you can omit --sshpass option. 107 | If you don't use keys you can still omit --sshpass option but you'd need to 108 | keep an eye to enter the password at the right moments when prompted or it'll 109 | timeout. 110 | 111 | The first time you connect to the target make sure to ssh outside of adeb first 112 | to add it to your known_hosts. 113 | 114 | 115 | More advanced usage instructions 116 | -------------------------------- 117 | ### Build and prepare device with a custom rootfs locally: 118 | 119 | The adeb fs will be prepared locally by downloading packages as needed: 120 | ``` 121 | adeb prepare --build 122 | ``` 123 | This is unlike the default behavior, where the adeb rootfs is itself pulled from the web. 124 | 125 | If you wish to do a full build (that is locally prepare a rootfs with all packages, including bcc, then do): 126 | ``` 127 | adeb prepare --full --build 128 | ``` 129 | 130 | ### Build/install a base image with BCC: 131 | ``` 132 | adeb prepare --bcc --build 133 | ``` 134 | Note: BCC is built from source. 135 | 136 | ### Extract the FS from the device, after its prepared: 137 | ``` 138 | adeb prepare --full --buildtar /path/ 139 | ``` 140 | After device is prepared, it will extract the root fs from it 141 | and store it as a tar archive at `/path/adeb-fs.tgz`. This 142 | can be used later. 143 | 144 | ### Use a previously prepared adeb rootfs tar from local: 145 | ``` 146 | adeb prepare --archive /path/adeb-fs.tgz 147 | ``` 148 | 149 | ### Build a standalone raw EXT4 image out of the FS: 150 | ``` 151 | adeb prepare --build-image /path/to/image.img 152 | ``` 153 | This can then be passed to Qemu as -hda. Note: This option doesn't need a 154 | device connected. 155 | 156 | ### How to use adeb for other Architectures (other than ARM64) 157 | By default adeb assumes the target Android device is based on ARM64 158 | processor architecture. For other architectures, use the --arch and --build option. 159 | For example for x86_64 architecture, run: 160 | ``` 161 | adeb prepare --build --arch amd64 162 | ``` 163 | Note: For arch other than ARM 64-bit, you have to pass the --build option to 164 | adeb. Without this, adeb tries to download an ARM image and will not work. 165 | TODO: We should auto detect this issue and provide an informative error. This 166 | is because we only provide pre-built filesystems for ARM 64-bit at the moment. 167 | 168 | Common Trouble shooting 169 | ----------------- 170 | 1. Installing g++ with `apt-get install g++` fails. 171 | 172 | Solution: Run `adeb shell apt-get update` after the `adeb prepare` stage. 173 | 174 | 2. It's too slow to use debootstrap to create debian fs 175 | 176 | Solution: Use a local mirror, for example in China you could use 177 | https://mirror.tuna.tsinghua.edu.cn/debian/ instead of debian official website 178 | http://deb.debian.org/debian/ 179 | -------------------------------------------------------------------------------- /README.version: -------------------------------------------------------------------------------- 1 | URL: https://github.com/joelagnel/adeb/ 2 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | TODO: 2 | - update androdeb with latest bcc (which will include criticalstat) 3 | - symlink whichever perf is installed to /usr/bin/perf 4 | - patch perf with the new futex contention script 5 | (probably it can just be dropped into a path) 6 | -------------------------------------------------------------------------------- /addons/bashrc: -------------------------------------------------------------------------------- 1 | # The bannered bashrc 2 | source .bashrc.common 3 | 4 | if [ ! -f .banner.shown ]; then 5 | 6 | echo "" 7 | echo "##########################################################" 8 | echo "# Welcome to androdeb environment running on Android! #" 9 | echo "# Questions to: Joel Fernandes #" 10 | echo " #" 11 | echo " Try running vim, gcc, clang, git, make, perf, filetop #" 12 | echo " ..etc or apt-get install something. #" 13 | echo "##########################################################" 14 | echo "" 15 | 16 | touch .banner.shown 17 | 18 | fi 19 | 20 | homedir=$( getent passwd "$USER" | cut -d: -f6 ) 21 | export HOME=$homedir 22 | -------------------------------------------------------------------------------- /addons/bashrc.common: -------------------------------------------------------------------------------- 1 | export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/share/bcc/tools/ 2 | export TMPDIR=/tmp/ 3 | 4 | psk=/proc/sys/kernel/kptr_restrict 5 | if [ -f $psk ]; then echo 0 > $psk; fi 6 | -------------------------------------------------------------------------------- /addons/bashrc.silent: -------------------------------------------------------------------------------- 1 | source .bashrc.common 2 | 3 | homedir=$( getent passwd "$USER" | cut -d: -f6 ) 4 | export HOME=$homedir 5 | -------------------------------------------------------------------------------- /addons/build-debian-tar: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -e 3 | # Build a tarball out of a android environment which I can then 4 | # upload to places for people who want to expedite the install 5 | 6 | # This script runs on the device 7 | 8 | spath=$( cd "$(dirname "$0")" ; pwd -P ) 9 | cd $spath 10 | 11 | ./device-umount-all 12 | 13 | if [ ! -d debian ]; then echo "Error: environment to tar doesn't exist"; exit 1; fi 14 | 15 | rm -rf debian-tar; cp -r debian debian-tar; 16 | rm -rf debian/debian; mv debian-tar debian/debian 17 | ./run-command "tar -zcf androdeb-fs.tgz --exclude='debian/kernel-headers' debian" 18 | mv debian/androdeb-fs.tgz . 19 | rm -rf debian/debian-tar 20 | -------------------------------------------------------------------------------- /addons/device-umount-all: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | if [ "$1x" == "--debugx" ]; then 4 | set -x 5 | fi 6 | 7 | umount_all() { 8 | mpoints=$(mount|cut -d ' ' -f3|grep debian) 9 | for m in $mpoints; 10 | do umount $m 2>&1 > /dev/null 11 | done 12 | } 13 | 14 | for i in $(seq 0 6); do 15 | umount_all 2>&1 > /dev/null 16 | done 17 | -------------------------------------------------------------------------------- /addons/device-unpack: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | 5 | # Script to do unpack of rootfs, ensures proper tear down 6 | # of existing environment. Expects debian rootfs in 7 | # /data/deb.tar.gz which it will delete after successful 8 | # unpack of rootfs. 9 | 10 | spath=$( cd "$(dirname "$0")" ; pwd -P ) 11 | 12 | if [ ! -f /data/androdeb/deb.tar.gz ]; then 13 | echo "Debian rootfs tar doesn't existing at /data/androdeb/deb.tar.gz" 14 | echo "Run androdeb with device connected first" 15 | exit 1 16 | fi 17 | 18 | if [ -d /data/androdeb/debian ]; then 19 | echo "androdeb environment already exists, doing a tear down" 20 | /data/androdeb/device-umount-all 21 | rm -rf /data/androdeb/debian 22 | fi 23 | 24 | 25 | gunzip /data/androdeb/deb.tar.gz 26 | tar -xf /data/androdeb/deb.tar -C /data/androdeb/ 27 | rm /data/androdeb/deb.tar 28 | 29 | echo "Unpack of rootfs successful!" 30 | -------------------------------------------------------------------------------- /addons/get_kvers.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | kvers=$(uname -r) 4 | 5 | MAJOR=$(echo $kvers | awk -F. '{ print $1 }') 6 | MINOR=$(echo $kvers | awk -F. '{ print $2 }') 7 | SUBVR=$(echo $kvers | awk -F. '{ print $3 }' | awk -F- '{ print $1 }' | sed 's/[^0-9]*//g') 8 | 9 | maj_num=$(($MAJOR * 65536)) 10 | min_num=$(($MINOR * 256)) 11 | 12 | echo $(($maj_num + $min_num + $SUBVR)) 13 | -------------------------------------------------------------------------------- /addons/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | spath=$( cd "$(dirname "$0")" ; pwd -P ) 3 | cd $spath 4 | 5 | source $spath/run.common 6 | 7 | chroot debian/ /bin/bash --rcfile '.bashrc' 8 | -------------------------------------------------------------------------------- /addons/run-command: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | spath=$( cd "$(dirname "$0")" ; pwd -P ) 3 | cd $spath 4 | 5 | source $spath/run.common 6 | 7 | # Directly execute a command within the chroot of an Android device 8 | CMD="$*" 9 | 10 | chroot debian /bin/bash --rcfile '.bashrc.silent' -i -c "$CMD" 11 | -------------------------------------------------------------------------------- /addons/run.common: -------------------------------------------------------------------------------- 1 | do_mounts() 2 | { 3 | mount --bind /proc debian/proc/ > /dev/null 4 | 5 | mount --bind /dev debian/dev/ > /dev/null 6 | mount --bind /dev/pts debian/dev/pts > /dev/null 7 | 8 | mount --bind /sys debian/sys/ > /dev/null 9 | mount --bind /sys/fs/bpf/ debian/sys/fs/bpf/ > /dev/null 10 | mount --bind /sys/kernel/debug/ debian/sys/kernel/debug/ > /dev/null 11 | mount --bind /sys/kernel/debug/tracing/ debian/sys/kernel/debug/tracing/ 12 | 13 | # Fix up weirdness with debugfs permission changing because of 14 | # above mounts. 15 | chmod 0777 /sys/kernel/debug > /dev/null 16 | chmod 0777 debian/sys/kernel/debug > /dev/null 17 | chmod 0777 /sys/kernel/debug/tracing > /dev/null 18 | chmod 0777 debian/sys/kernel/debug/tracing > /dev/null 19 | 20 | # Mount Android partitions 21 | if [ -d /d/ ]; then 22 | if [ ! -d debian/d ]; then ln -s /sys/kernel/debug debian/d; fi 23 | fi 24 | 25 | if [ -d /data/ ]; then 26 | mkdir -p debian/data/ 27 | mount --bind /data debian/data/ 28 | fi 29 | 30 | if [ -d /system/ ]; then 31 | mkdir -p debian/system/ 32 | mount --bind /system debian/system/ 33 | fi 34 | 35 | if [ -d /vendor/ ]; then 36 | mkdir -p debian/vendor/ 37 | mount --bind /vendor debian/vendor/ 38 | fi 39 | 40 | if [ -d /dev/binderfs/ ]; then 41 | mkdir -p debian/dev/binderfs 42 | mount --bind /dev/binderfs/ debian/dev/binderfs 43 | fi 44 | 45 | } 46 | 47 | mount | grep debian > /dev/null 48 | if [ $? -ne 0 ]; then do_mounts; fi 49 | -------------------------------------------------------------------------------- /adeb: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | # 3 | # (c) Joel Fernandes 4 | 5 | spath="$(dirname "$(readlink -f "$0")")" 6 | source $spath/androdeb 7 | -------------------------------------------------------------------------------- /androdeb: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | # 3 | # (c) Joel Fernandes 4 | 5 | VERSION=v0.99h 6 | 7 | spath="$(dirname "$(readlink -f "$0")")" 8 | # spath=$( cd "$(dirname "$0")" ; pwd -P ) 9 | curdir=$( pwd -P ) 10 | source $spath/utils/support 11 | source $spath/utils/banners 12 | source $spath/utils/remote 13 | 14 | # Set default vars 15 | DISTRO=buster; ARCH=arm64 16 | ADB="adb" 17 | REMOTE="adb" 18 | FULL=0 # Default to a minimal install 19 | DOWNLOAD=1 # Default to downloading from web 20 | SKIP_DEVICE=0 # Skip device preparation 21 | INSTALL_BCC=0 # Decide if BCC is to be installed 22 | 23 | # Default packages 24 | PACKAGES="" 25 | DEFAULT_PACKAGES="bash ca-certificates apt net-tools iputils-ping procps vim" 26 | 27 | EXTRA_FILES="none" 28 | 29 | config_full_build() { 30 | for f in $(ls $spath/packages); do source $spath/packages/$f; done; 31 | } 32 | 33 | # Parse command line parameters 34 | if [ $# -lt 1 ]; then usage; fi; POSITIONAL=() 35 | while [[ $# -gt 0 ]]; do key="$1"; 36 | 37 | # If its shell mode, any future args become shell's args 38 | if [ "x$ASHELL" == "x1" ]; then 39 | if [ -z "$SHELL_ARGS" ]; then 40 | SHELL_ARGS=$key 41 | else 42 | SHELL_ARGS="$SHELL_ARGS $key" 43 | fi 44 | shift || true; continue 45 | fi 46 | 47 | case $key in 48 | shell) ASHELL=1; shift || true; ;; 49 | remove) REMOVE=1; shift || true; ;; 50 | git-pull) GIT_PULL=1; shift || true; ;; 51 | pull) PULL=1; shift || true; break ;; 52 | push) PUSH=1; shift || true; break ;; 53 | prepare) PREPARE=1; shift || true; ;; 54 | --full) FULL=1; config_full_build; shift || true; ;; 55 | --arch) ARCH=$2; shift || true; shift || true; ;; 56 | --archive) DOWNLOAD=0; TARF=$2; shift || true; shift || true; ;; 57 | --bcc) FULL=1; source $spath/packages/bcc; shift || true; ;; 58 | --tempdir) TDIR="$2"; shift || true; shift || true; ;; 59 | --build) DOWNLOAD=0; shift || true; ;; 60 | --buildtar) BTAR=1; DOWNLOAD=0; TARDIR="$2"; shift || true; shift || true; ;; 61 | --device|-s) ADB="$ADB -s $2"; shift || true; shift || true; ;; 62 | --ssh) REMOTE="ssh"; SSH_URI="$2"; shift || true; shift || true; ;; 63 | --sshpass) SSHPASS="sshpass -p$2"; shift || true; shift || true; ;; 64 | --build-image) BI=1; BUILD_IMAGEF=$2; SKIP_DEVICE=1; DOWNLOAD=0; shift || true; shift || true; ;; 65 | --debug) set -x; shift || true; ;; 66 | *) c_error "Unknown option ($1)"; usage; ;; 67 | esac 68 | done 69 | 70 | [ -z $ASHELL ] && box_out "adeb: $VERSION" 71 | 72 | if [ $FULL -eq 1 ]; then 73 | FNAME=androdeb-fs.tgz.zip 74 | FNAME_UZ=androdeb-fs.tgz 75 | else 76 | FNAME=androdeb-fs-minimal.tgz.zip 77 | FNAME_UZ=androdeb-fs-minimal.tgz 78 | fi 79 | 80 | if [ ! -z $BTAR ] && [ -z $TARDIR ]; then 81 | TARDIR=$spath 82 | fi 83 | 84 | if [ ! -z "$GIT_PULL" ]; then 85 | c_info "Updating androdeb by git pull" 86 | cd $spath 87 | git pull 88 | c_info "Done." 89 | exit 0 90 | fi 91 | 92 | if [ ! -z "$PULL" ]; then 93 | remote_pull "$@" 94 | exit 0 95 | fi 96 | 97 | if [ ! -z "$PUSH" ]; then 98 | remote_push "$@" 99 | exit 0 100 | fi 101 | 102 | if [[ ! -z ${TARDIR+x} ]] && [[ ! -d $TARDIR ]]; then die 7 "Tar dir specified doesn't exist"; fi 103 | 104 | if [ -z $BI ]; then 105 | [ -z $ASHELL ] && c_info "Looking for device.." 106 | set +e 107 | remote_root 108 | 109 | if [ $? -ne 0 ]; then 110 | c_error "adb root failed, make sure:" 111 | c_error " * If multiple devices connected, provide --device (or -s )" 112 | c_error " * Try to run \"adb root\" manually and see if it works. Typically this needs a userdebug build." 113 | c_error "" 114 | c_error "Note: adb can be typically obtained using the android-tools-adb or the adb" 115 | c_error "packages on your distro, or by installing the Android SDK." 116 | die 3 "Exiting." 117 | fi 118 | set -e 119 | else 120 | [ ! -z $BUILD_IMAGEF ] || die 8 "--build-image passed but no image file provided" 121 | fi 122 | 123 | if [ ! -z "$REMOVE" ]; then 124 | die_if_no_androdeb "Nothing to remove." 125 | remote_shell /data/androdeb/device-umount-all || true; 126 | remote_shell rm -rf /data/androdeb; exit 0; fi 127 | 128 | ########################################################## 129 | # SHELL 130 | ########################################################## 131 | if [ ! -z ${ASHELL+x} ]; then 132 | set +e; remote_shell ls /data/androdeb/debian/.bashrc > /dev/null 2>&1 133 | if [ $? -ne 0 ]; then 134 | die 2 "Device doesn't have an androdeb environment, run \"./androdeb prepare\" first"; 135 | fi; set -e 136 | 137 | if [ ! -z ${SHELL_ARGS+x} ]; then 138 | # Explanation of quotes: 139 | # Outer quote is so that androdeb's bash passes the SHELL_ARGS as a single 140 | # argument to $ADB shell. Inner quotes is so that run-command can receive all 141 | # the args even though they may be separated by spaces. \m/ 142 | remote_shell -t /data/androdeb/run-command "\"$SHELL_ARGS\"" 143 | else 144 | remote_shell -t /data/androdeb/run 145 | fi 146 | 147 | exit 0 148 | fi 149 | 150 | ########################################################## 151 | # PREPARE 152 | ########################################################## 153 | 154 | function do_cleanup() { 155 | rm -rf $TDIR/*; if [ $MKTEMP -eq 1 ]; then rm -rf $TDIR; fi 156 | } 157 | 158 | function all_done_banner() { 159 | c_info "All done! Run \"adeb shell\" to enter environment" 160 | } 161 | 162 | function detect_repo_url() { 163 | ADEB_REPO_URL=`cd $spath && git config -l | grep -m1 remote | grep url | sed -e "s/.*url=//" \ 164 | -e "s/.*@//" \ 165 | -e "s/https:\/\///" \ 166 | -e "s/:/\//" \ 167 | -e "s/\.git$//"`"/" 168 | c_info "Detected URL: $ADEB_REPO_URL" 169 | } 170 | 171 | function check_repo_url () { 172 | if [ -z $ADEB_REPO_URL ]; then 173 | c_info "No repository URL provided in enviromnent. Attempting to auto-detect it" 174 | detect_repo_url 175 | fi 176 | 177 | if [ -z $ADEB_REPO_URL ]; then 178 | c_warning "Automatic download is disabled. To enable it, please set the \$ADEB_REPO_URL" 179 | c_warning "environment variable as recommended in the setup instructions in the README.md" 180 | do_cleanup 181 | exit 0 182 | fi 183 | } 184 | 185 | # Prepare is the last command checked 186 | if [ -z "$PREPARE" ]; then usage; fi 187 | 188 | if [ ! -z "$TARF" ] && [ ! -f $TARF ] && [ -z "$DOWNLOAD" ]; then die 5 "archive provided doesn't exist"; fi 189 | 190 | print_prepare_banner 191 | 192 | # Where do we want to store temporary files 193 | MKTEMP=0; if [[ -z ${TDIR+x} ]] || [[ ! -d "${TDIR}" ]]; then 194 | TDIR=`mktemp -d`; MKTEMP=1; fi 195 | rm -rf $TDIR/* 196 | TDIR_ABS=$( cd "$TDIR" ; pwd -P ) 197 | 198 | if [ $DOWNLOAD -eq 1 ]; then 199 | c_info "Downloading Androdeb from the web..."; c_info "" 200 | 201 | # Github dropped tar gz support! ##?#??#! Now we've to zip everything. 202 | check_repo_url 203 | 204 | curl -L https://$ADEB_REPO_URL/releases/download/$VERSION/$FNAME --output $TDIR_ABS/$FNAME || 205 | die 9 "Failed to download adeb release." 206 | 207 | unzip -e $TDIR_ABS/$FNAME -d $TDIR_ABS/ || 208 | die 10 "Failed to download adeb release. Double check the ADEB_REPO_URL value." 209 | TARF=$TDIR_ABS/$FNAME_UZ 210 | fi 211 | 212 | OUT_TMP=$TDIR/debian; rm -rf $OUT_TMP; mkdir -p $OUT_TMP 213 | 214 | # Build FS from existing tar, very simple. 215 | if [ ! -z "$TARF" ]; then 216 | c_info "Using archive at $TARF for filesystem preparation" 217 | remote_shell mkdir -p /data/androdeb/ 218 | 219 | c_info "Pushing filesystem to device.." 220 | run_quiet remote_copy $TARF /data/androdeb/deb.tar.gz 221 | 222 | c_info "Pushing addons to device.." 223 | run_quiet remote_copy $spath/addons/* /data/androdeb/ 224 | 225 | c_info "Unpacking filesystem in device.." 226 | run_quiet remote_shell /data/androdeb/device-unpack 227 | 228 | do_cleanup; all_done_banner; exit 0 229 | fi 230 | 231 | PACKAGES+="$DEFAULT_PACKAGES" 232 | c_info "Using temporary directory: $TDIR" 233 | 234 | if [[ $EUID -ne 0 ]]; then c_info "The next stage runs as sudo, please enter password if asked."; fi 235 | 236 | ex_files=$(mktemp); echo $EXTRA_FILES > $ex_files 237 | 238 | sudo $spath/buildstrap $ARCH $DISTRO $TDIR $OUT_TMP \ 239 | "$(make_csv "$PACKAGES")"\ 240 | $ex_files $INSTALL_BCC $SKIP_DEVICE 241 | rm $ex_files 242 | 243 | # If we only wanted to prepare a rootfs and don't have 244 | # a device connected, then just echo that and skip cleanup 245 | if [ $SKIP_DEVICE -eq 1 ]; then 246 | c_info "Device preparation is being skipped for the selected options" 247 | c_info "any builds that need to happen on device may be cloned but not built." 248 | 249 | if [ ! -z $BI ]; then 250 | sudo $spath/buildimage $OUT_TMP $(dirname $BUILD_IMAGEF)/$(basename $BUILD_IMAGEF) 251 | sudo chmod a+rw $(dirname $BUILD_IMAGEF)/$(basename $BUILD_IMAGEF) 252 | c_info "Your .img has been built! Enjoy!" 253 | fi 254 | 255 | do_cleanup 256 | exit 0 257 | fi 258 | 259 | # Push tar to device and start unpack 260 | remote_shell mkdir -p /data/androdeb/ 261 | remote_copy $TDIR/deb.tar.gz /data/androdeb/ 262 | remote_copy $spath/addons/* /data/androdeb/ 263 | remote_shell /data/androdeb/device-unpack 264 | 265 | # Extract a tar of the built, compiled and installed androdeb env 266 | if [[ ! -z ${TARDIR+x} ]]; then 267 | c_info "Creating tarball" 268 | pushd $TARDIR 269 | if [ $INSTALL_BCC -eq 0 ]; then 270 | mv $TDIR/deb.tar.gz $FNAME_UZ 271 | else 272 | remote_shell /data/androdeb/build-debian-tar 273 | remote_pull /data/androdeb/androdeb-fs.tgz $FNAME_UZ 274 | remote_shell rm /data/androdeb/androdeb-fs.tgz; 275 | fi 276 | zip -r $FNAME $FNAME_UZ 277 | popd 278 | fi 279 | 280 | do_cleanup 281 | 282 | all_done_banner 283 | -------------------------------------------------------------------------------- /bcc/build-bcc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script should run within a bcc checkout 3 | 4 | spath=$( cd "$(dirname "$0")" ; pwd -P ) 5 | cd $spath 6 | 7 | rm -rf build && mkdir -p build && cd build 8 | cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_C_COMPILER=clang-7 -DCMAKE_CXX_COMPILER=clang++-7 9 | make -j4 10 | make install 11 | cmake -DPYTHON_CMD=python3 .. 12 | pushd src/python/ 13 | make 14 | make install 15 | popd 16 | cd .. 17 | rm -rf build 18 | -------------------------------------------------------------------------------- /bcc/misc/0001-trace-file-path-inode-number.patch: -------------------------------------------------------------------------------- 1 | From 4484b9f102e4d226df06e8e12f860bd54422e429 Mon Sep 17 00:00:00 2001 2 | From: "Joel Fernandes (Google)" 3 | Date: Mon, 11 Mar 2019 21:20:41 -0400 4 | Subject: [PATCH] trace file-path -> inode number 5 | 6 | Change-Id: I464821bb5696f0d51259d9c1a31eb62e6c1b76da 7 | Signed-off-by: Joel Fernandes (Google) 8 | --- 9 | fs/namei.c | 36 +++++++++++++++++++++++++++++++++--- 10 | include/trace/events/namei.h | 31 +++++++++++++++++++++++++++++++ 11 | 2 files changed, 64 insertions(+), 3 deletions(-) 12 | create mode 100644 include/trace/events/namei.h 13 | 14 | diff --git a/fs/namei.c b/fs/namei.c 15 | index d33064a40420..e0d6f4441357 100644 16 | --- a/fs/namei.c 17 | +++ b/fs/namei.c 18 | @@ -15,6 +15,9 @@ 19 | /* [Feb-Apr 2000, AV] Rewrite to the new namespace architecture. 20 | */ 21 | 22 | +#define CREATE_TRACE_POINTS 23 | +#include 24 | + 25 | #include 26 | #include 27 | #include 28 | @@ -801,6 +804,33 @@ static inline int d_revalidate(struct dentry *dentry, unsigned int flags) 29 | return 1; 30 | } 31 | 32 | +static int success_walk(struct nameidata *nd) 33 | +{ 34 | + struct path *pt = &nd->path; 35 | + struct inode *i = nd->inode; 36 | + char buf[200]; 37 | + int n = 200; 38 | + char *p; 39 | + 40 | + p = d_path(pt, buf, n); 41 | + 42 | + if (!IS_ERR(p)) { 43 | + char *end = mangle_path(buf, p, "\n"); 44 | + if (end) { 45 | + buf[end - buf] = 0; 46 | + } else { 47 | + buf[n-1] = 0; 48 | + } 49 | + } else { 50 | + buf[0] = 0; 51 | + } 52 | + 53 | + buf[30] = 0; 54 | + 55 | + trace_namei_inode_path(i->i_ino, buf); 56 | + return 0; 57 | +} 58 | + 59 | /** 60 | * complete_walk - successful completion of path walk 61 | * @nd: pointer nameidata 62 | @@ -824,14 +854,14 @@ static int complete_walk(struct nameidata *nd) 63 | } 64 | 65 | if (likely(!(nd->flags & LOOKUP_JUMPED))) 66 | - return 0; 67 | + return success_walk(nd); 68 | 69 | if (likely(!(dentry->d_flags & DCACHE_OP_WEAK_REVALIDATE))) 70 | - return 0; 71 | + return success_walk(nd); 72 | 73 | status = dentry->d_op->d_weak_revalidate(dentry, nd->flags); 74 | if (status > 0) 75 | - return 0; 76 | + return success_walk(nd); 77 | 78 | if (!status) 79 | status = -ESTALE; 80 | diff --git a/include/trace/events/namei.h b/include/trace/events/namei.h 81 | new file mode 100644 82 | index 000000000000..9eed50e37190 83 | --- /dev/null 84 | +++ b/include/trace/events/namei.h 85 | @@ -0,0 +1,31 @@ 86 | +/* SPDX-License-Identifier: GPL-2.0 */ 87 | + 88 | +#undef TRACE_SYSTEM 89 | +#define TRACE_SYSTEM namei 90 | + 91 | +#if !defined(_TRACE_NAMEI_H) || defined(TRACE_HEADER_MULTI_READ) 92 | +#define _TRACE_NAMEI_H 93 | + 94 | +#include 95 | + 96 | +TRACE_EVENT(namei_inode_path, 97 | + TP_PROTO(int ino, char *path_param), 98 | + TP_ARGS(ino, path_param), 99 | + TP_STRUCT__entry( 100 | + __field(int, inode_no) 101 | + __array(char, path, 32) 102 | + ), 103 | + TP_fast_assign( 104 | + __entry->inode_no = ino; 105 | + memcpy(__entry->path, path_param, 32); 106 | + __entry->path[31] = 0; 107 | + ), 108 | + TP_printk("inode_no: %d path: %s", 109 | + __entry->inode_no, 110 | + __entry->path 111 | + ) 112 | +); 113 | + 114 | +#endif /* _TRACE_NAMEI_H */ 115 | + 116 | +#include 117 | -- 118 | 2.21.0.360.g471c308f928-goog 119 | 120 | -------------------------------------------------------------------------------- /bcc/misc/android-futex-contention-record: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | perf record -a -g -e syscalls:sys_enter_futex -e syscalls:sys_exit_futex -e sched:sched_waking $@ 3 | -------------------------------------------------------------------------------- /bcc/misc/android-futex-contention-report: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # description: futext contention measurement 3 | 4 | perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/android-futex-contention.py 5 | -------------------------------------------------------------------------------- /bcc/misc/android-futex-contention.py: -------------------------------------------------------------------------------- 1 | # futex contention 2 | # (c) 2010, Arnaldo Carvalho de Melo 3 | # Licensed under the terms of the GNU GPL License version 2 4 | # 5 | # Translation of: 6 | # 7 | # http://sourceware.org/systemtap/wiki/WSFutexContention 8 | # 9 | # to perf python scripting. 10 | # 11 | # Measures futex contention 12 | 13 | import os, sys 14 | sys.path.append(os.environ['PERF_EXEC_PATH'] + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') 15 | from Util import * 16 | 17 | process_names = {} 18 | thread_thislock = {} 19 | thread_blocktime = {} 20 | 21 | lock_waits = {} # long-lived stats on (tid,lock) blockage elapsed time 22 | waker_wakee = {} # maps the futex waker to wakee 23 | max_waits = {} # Details about a maximum contention like owner, owner chain 24 | process_names = {} # long-lived pid-to-execname mapping 25 | 26 | def android_lock(callchain): 27 | for c in callchain: 28 | if 'sym' in c and 'name' in c['sym']: 29 | name = c['sym']['name'] 30 | else: 31 | continue 32 | 33 | if 'art::Monitor::Lock' in name: 34 | return True 35 | return False 36 | 37 | def print_callchain(callchain): 38 | for c in callchain: 39 | if 'sym' in c and 'name' in c['sym']: 40 | name = c['sym']['name'] 41 | else: 42 | continue 43 | 44 | print(" %s" % (name)) 45 | 46 | def sched__sched_waking(event_name, context, common_cpu, 47 | common_secs, common_nsecs, common_pid, common_comm, 48 | common_callchain, comm, pid, prio, success, 49 | target_cpu): 50 | waker_wakee[pid] = [common_pid, common_callchain] 51 | 52 | def syscalls__sys_enter_futex(event, ctxt, cpu, s, ns, tid, comm, callchain, 53 | nr, uaddr, op, val, utime, uaddr2, val3): 54 | 55 | cmd = op & FUTEX_CMD_MASK 56 | if cmd != FUTEX_WAIT or android_lock(callchain) == False: 57 | return # we don't care about originators of WAKE events 58 | # or futex uses that aren't android locks. 59 | 60 | process_names[tid] = comm 61 | thread_thislock[tid] = uaddr 62 | thread_blocktime[tid] = nsecs(s, ns) 63 | 64 | def syscalls__sys_exit_futex(event, ctxt, cpu, s, ns, tid, comm, callchain, 65 | nr, ret): 66 | 67 | waker_pid = -1 68 | waker_chain = "[no call chain]" 69 | 70 | if thread_blocktime.has_key(tid): 71 | # Gather stats about the contention (sum, min, max) 72 | elapsed = nsecs(s, ns) - thread_blocktime[tid] 73 | add_stats(lock_waits, (tid, thread_thislock[tid]), elapsed) 74 | 75 | # Track details about the maximum contention seen 76 | # including owner and its callchain 77 | if (tid, thread_thislock[tid]) in max_waits: 78 | prev_wait = max_waits[(tid, thread_thislock[tid])][0] 79 | else: 80 | prev_wait = 0 81 | 82 | if elapsed > prev_wait: 83 | if tid in waker_wakee: 84 | waker_pid = waker_wakee[tid][0] 85 | waker_chain = waker_wakee[tid][1] 86 | 87 | max_waits[(tid, thread_thislock[tid])] = [elapsed, waker_pid, waker_chain, callchain] 88 | 89 | del thread_blocktime[tid] 90 | del thread_thislock[tid] 91 | 92 | def trace_begin(): 93 | print "Press control+C to stop and show the summary" 94 | 95 | def trace_end(): 96 | for (tid, lock) in lock_waits: 97 | print("\n==============================================================\n") 98 | min, max, avg, count = lock_waits[tid, lock] 99 | print "%s[%d] lock %x contended %d times, %d avg ns, %d max ns" % \ 100 | (process_names[tid], tid, lock, count, avg, max) 101 | print "" 102 | 103 | if not (tid, lock) in max_waits: 104 | print"Max contention info not available" 105 | continue 106 | 107 | print "Callstack of suffering task:" 108 | print_callchain(max_waits[tid, lock][3]) 109 | print "" 110 | 111 | waker_pid = max_waits[tid, lock][1] 112 | waker_name = process_names[waker_pid] if waker_pid in process_names else "nameless-owner" 113 | print "Owner %s caused this contention of %d ns. Owner's Call stack below:" % (waker_name, max_waits[tid, lock][0]) 114 | print_callchain(max_waits[tid, lock][2]) 115 | 116 | -------------------------------------------------------------------------------- /bcc/misc/build-bcc-minimal/build-on-target.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This is run in the bcc directory of the chroot 3 | 4 | cd bcc-master 5 | rm -rf build && mkdir -p build && cd build 6 | cmake .. -DCMAKE_INSTALL_PREFIX=./future-usr -DCMAKE_C_COMPILER=clang-7 -DCMAKE_CXX_COMPILER=clang++-7 7 | make -j90 8 | make install 9 | -------------------------------------------------------------------------------- /bcc/misc/build-bcc-minimal/build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | set -e 3 | 4 | if [[ $EUID -ne 0 ]]; then 5 | echo "This script must be run as root" 6 | exit 1 7 | fi 8 | 9 | # This is a WIP script to be build a minimal BCC rootfs image It works by using 10 | # a 2-chroot technique. The first chroot is used to build BCC, The second 11 | # chroot is used to install BCC in. The whole image should take around 35MB 12 | # after compression. 13 | 14 | SPATH="$(dirname "$(readlink -f "$0")")" 15 | 16 | DIST=buster 17 | RUN_DIR=$SPATH/bcc.run 18 | BLD_DIR=$SPATH/bcc.bld 19 | 20 | ############## FIRST STAGE - Build the BCC build chroot 21 | rm -rf $RUN_DIR && mkdir -p $RUN_DIR 22 | rm -rf $BLD_DIR && mkdir -p $BLD_DIR 23 | 24 | # Build a chroot for the build system to build BCC into 25 | sudo qemu-debootstrap \ 26 | --arch arm64 \ 27 | --include=procps,git,clang-7,cmake,llvm-7-dev,libclang-7-dev,libelf-dev,libfl-dev,libunwind-dev,libdw-dev,git,libtool,autoconf,make,cmake,iperf,arping,ethtool,flex,bison,python,clang-7,python-netaddr,python-pyroute2 \ 28 | --variant=minbase $DIST $BLD_DIR http://ftp.us.debian.org/debian 29 | 30 | pushd $BLD_DIR 31 | git clone --recurse-submodules https://github.com/iovisor/bcc.git bcc-master 32 | popd 33 | 34 | cp $SPATH/build-on-target.sh $BLD_DIR/ 35 | sudo chroot $BLD_DIR /build-on-target.sh 36 | 37 | ################ SECOND STAGE - BUILD the run chroot 38 | qemu-debootstrap \ 39 | --arch arm64 \ 40 | --include=libelf1,python,python-netaddr,python-pyroute2 \ 41 | --variant=minbase $DIST $RUN_DIR http://ftp.us.debian.org/debian 42 | 43 | # Sync the built BCC from the build chroot onto the run chroot 44 | rsync -rav $BLD_DIR/bcc-master/build/future-usr/ $RUN_DIR/usr/ 45 | 46 | # Clean up the chroot for whatever is not needed 47 | find $RUN_DIR -name "*.a" |xargs rm -rf 48 | rm -rf $RUN_DIR/lib/udev/* 49 | rm -rf $RUN_DIR/var/lib/apt/lists/* 50 | rm -rf $RUN_DIR/var/cache/apt/archives/*deb 51 | 52 | rm -rf $RUN_DIR/var/cache/debconf/* 53 | rm -rf $RUN_DIR/var/cache/apt/*.bin 54 | 55 | rm -rf $RUN_DIR/usr/share/locale/* 56 | rm -rf $RUN_DIR/usr/lib/share/locale/* 57 | rm -rf $RUN_DIR/usr/share/doc/* 58 | rm -rf $RUN_DIR/usr/lib/share/doc/* 59 | rm -rf $RUN_DIR/usr/share/ieee-data/* 60 | rm -rf $RUN_DIR/usr/lib/share/ieee-data/* 61 | rm -rf $RUN_DIR/usr/share/man/* 62 | rm -rf $RUN_DIR/usr/lib/share/man/* 63 | 64 | rm -rf $RUN_DIR/usr/lib/*/perl 65 | rm -rf $RUN_DIR/usr/bin/perl* 66 | rm -rf $RUN_DIR/usr/lib/perl* 67 | rm -rf $RUN_DIR/usr/share/*perl* 68 | 69 | rm -rf $RUN_DIR/usr/bin/qemu-aarch64-static 70 | rm -rf $RUN_DIR/var/cache/apt/*bin 71 | rm -rf $RUN_DIR/usr/lib/file/magic.mgc 72 | 73 | tar -Jc -C $RUN_DIR -f bcc-rootfs.txz . 74 | -------------------------------------------------------------------------------- /bcc/misc/build-gcc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | set -e 3 | 4 | if [[ $EUID -ne 0 ]]; then 5 | echo "This script must be run as root" 6 | exit 1 7 | fi 8 | 9 | # This is a WIP script to be build a minimal GCC rootfs image 10 | SPATH="$(dirname "$(readlink -f "$0")")" 11 | 12 | DIST=buster 13 | BLD_DIR=$SPATH/gcc.bld 14 | 15 | rm -rf $BLD_DIR && mkdir -p $BLD_DIR 16 | 17 | # Build a chroot for the build system to build BCC into 18 | sudo qemu-debootstrap \ 19 | --arch amd64 \ 20 | --include=gcc,make,libc6-dev \ 21 | --variant=minbase $DIST $BLD_DIR http://ftp.us.debian.org/debian 22 | 23 | # Clean up the chroot for whatever is not needed 24 | # find $BLD_DIR -name "*.a" |xargs rm -rf 25 | rm -rf $BLD_DIR/lib/udev/* 26 | rm -rf $BLD_DIR/var/lib/apt/lists/* 27 | rm -rf $BLD_DIR/var/cache/apt/archives/*deb 28 | 29 | rm -rf $BLD_DIR/var/cache/debconf/* 30 | rm -rf $BLD_DIR/var/cache/apt/*.bin 31 | 32 | rm -rf $BLD_DIR/usr/share/locale/* 33 | rm -rf $BLD_DIR/usr/lib/share/locale/* 34 | rm -rf $BLD_DIR/usr/share/doc/* 35 | rm -rf $BLD_DIR/usr/lib/share/doc/* 36 | rm -rf $BLD_DIR/usr/share/ieee-data/* 37 | rm -rf $BLD_DIR/usr/lib/share/ieee-data/* 38 | rm -rf $BLD_DIR/usr/share/man/* 39 | rm -rf $BLD_DIR/usr/lib/share/man/* 40 | 41 | rm -rf $BLD_DIR/usr/lib/*/perl 42 | rm -rf $BLD_DIR/usr/bin/perl* 43 | rm -rf $BLD_DIR/usr/lib/perl* 44 | rm -rf $BLD_DIR/usr/share/*perl* 45 | 46 | rm -rf $BLD_DIR/usr/bin/qemu-aarch64-static 47 | rm -rf $BLD_DIR/var/cache/apt/*bin 48 | rm -rf $BLD_DIR/usr/lib/file/magic.mgc 49 | 50 | tar -Jc -C $BLD_DIR -f gcc-rootfs.txz . 51 | -------------------------------------------------------------------------------- /bcc/misc/futex-contention.py: -------------------------------------------------------------------------------- 1 | # futex contention 2 | # (c) 2010, Arnaldo Carvalho de Melo 3 | # Licensed under the terms of the GNU GPL License version 2 4 | # 5 | # Translation of: 6 | # 7 | # http://sourceware.org/systemtap/wiki/WSFutexContention 8 | # 9 | # to perf python scripting. 10 | # 11 | # Measures futex contention 12 | 13 | import os, sys 14 | sys.path.append(os.environ['PERF_EXEC_PATH'] + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') 15 | from Util import * 16 | 17 | process_names = {} 18 | thread_thislock = {} 19 | thread_blocktime = {} 20 | 21 | lock_waits = {} # long-lived stats on (tid,lock) blockage elapsed time 22 | process_names = {} # long-lived pid-to-execname mapping 23 | 24 | def android_lock(callchain): 25 | for c in callchain: 26 | if 'sym' in c and 'name' in c['sym']: 27 | name = c['sym']['name'] 28 | else: 29 | continue 30 | 31 | if 'art::Monitor::Lock' in name: 32 | return True 33 | return False 34 | 35 | def syscalls__sys_enter_futex(event, ctxt, cpu, s, ns, tid, comm, callchain, 36 | nr, uaddr, op, val, utime, uaddr2, val3): 37 | 38 | cmd = op & FUTEX_CMD_MASK 39 | if cmd != FUTEX_WAIT or android_lock(callchain) == False: 40 | return # we don't care about originators of WAKE events 41 | # or futex uses that aren't android locks. 42 | 43 | process_names[tid] = comm 44 | thread_thislock[tid] = uaddr 45 | thread_blocktime[tid] = nsecs(s, ns) 46 | 47 | def syscalls__sys_exit_futex(event, ctxt, cpu, s, ns, tid, comm, callchain, 48 | nr, ret): 49 | if thread_blocktime.has_key(tid): 50 | elapsed = nsecs(s, ns) - thread_blocktime[tid] 51 | add_stats(lock_waits, (tid, thread_thislock[tid]), elapsed) 52 | del thread_blocktime[tid] 53 | del thread_thislock[tid] 54 | 55 | def trace_begin(): 56 | print "Press control+C to stop and show the summary" 57 | 58 | def trace_end(): 59 | for (tid, lock) in lock_waits: 60 | min, max, avg, count = lock_waits[tid, lock] 61 | print "%s[%d] lock %x contended %d times, %d avg ns" % \ 62 | (process_names[tid], tid, lock, count, avg) 63 | 64 | -------------------------------------------------------------------------------- /bcc/misc/inodemap: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # Map inode numbers to names. 3 | # Corresponding kernel patch is in: 4 | # 0001-trace-file-path-inode-number.patch 5 | 6 | from __future__ import print_function 7 | from bcc import BPF, USDT, DEBUG_PREPROCESSOR 8 | from time import sleep, strftime 9 | import argparse 10 | import re 11 | import signal 12 | import sys 13 | import traceback 14 | from subprocess import call 15 | 16 | bpf_text = """ 17 | 18 | #define MAX_PATH_LENGTH 32 19 | 20 | struct val_t { 21 | char path[MAX_PATH_LENGTH]; 22 | }; 23 | 24 | BPF_HASH(inode_map, u64, struct val_t); 25 | 26 | /* 27 | struct namei_args { 28 | unsigned long long ignore; 29 | int inode_no; 30 | char path[MAX_PATH_LENGTH]; 31 | }; 32 | */ 33 | 34 | TRACEPOINT_PROBE(namei, namei_inode_path) { 35 | u64 inode_no = args->inode_no; 36 | struct val_t val = {}; 37 | 38 | /* bug, we may be over reading */ 39 | bpf_probe_read(&val.path, sizeof(val.path), args->path); 40 | 41 | inode_map.update(&inode_no, &val); 42 | return 0; 43 | } 44 | """ 45 | 46 | b = BPF(text=bpf_text, debug=DEBUG_PREPROCESSOR) 47 | 48 | exiting = 0 49 | 50 | inode_map = b.get_table("inode_map") 51 | while 1: 52 | try: 53 | sleep(1) 54 | except KeyboardInterrupt: 55 | exiting = 1 56 | call("clear") 57 | 58 | for k, v in inode_map.items(): 59 | print("%-6d %s" % (k.value, v.path.decode('utf-8', 'replace'))) 60 | 61 | inode_map.clear() 62 | 63 | if exiting: 64 | print("Detaching...") 65 | exit() 66 | -------------------------------------------------------------------------------- /bcc/misc/lockstat.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # 3 | # lockstat Trace and display lock contention stats 4 | # 5 | # USAGE: lockstat 6 | 7 | # Licensed under the Apache License, Version 2.0 (the "License") 8 | # 28-Jul-2017 Gisle Dankel Created this. 9 | 10 | from bcc import BPF 11 | from ctypes import c_int 12 | from time import sleep 13 | from datetime import datetime 14 | import argparse 15 | import subprocess 16 | import os 17 | 18 | # One Lock object per TGID and uaddr. 19 | class Lock(object): 20 | def __init__(self): 21 | self.contention_count = 0 22 | self.elapsed_blocked = 0 23 | self.thread_count = 0 24 | self.last_stack_syms = [] 25 | 26 | def update(self, count, block_time, last_stack_syms): 27 | self.contention_count += count 28 | self.elapsed_blocked += block_time 29 | self.thread_count += 1 30 | self.last_stack_syms = last_stack_syms 31 | 32 | def run_command_get_pid(command): 33 | p = subprocess.Popen(command.split()) 34 | return p.pid 35 | 36 | examples = """ 37 | EXAMPLES: 38 | 39 | ./lockstat 40 | Trace calls to sys_futex and display contented locks every 5 seconds 41 | for all processes running on the system 42 | ./lockstat -p 43 | Trace only for the specified pid and display contended locks 44 | every 5 seconds 45 | ./lockstat -p -t 46 | Trace for a specified pid and print a message on each entry and exit to 47 | sys_futex until interrupted or killed 48 | ./lockstat -p 10 49 | Trace the specified pid and show a message every 10 seconds 50 | ./lockstat -c 1 30 51 | Run the specified command and display contended locks every 1 second 52 | for a total of 30 times 53 | """ 54 | 55 | description = """ 56 | Trace kernel futex events. 57 | These often occur because of lock contention, e.g. involving a pthread_mutex. 58 | This script resemblers the following SystemTap example: 59 | https://sourceware.org/systemtap/SystemTap_Beginners_Guide/futexcontentionsect.html 60 | """ 61 | 62 | parser = argparse.ArgumentParser(description=description, 63 | formatter_class=argparse.RawDescriptionHelpFormatter, 64 | epilog=examples) 65 | parser.add_argument("-p", "--pid", type=int, default=-1, 66 | help="the PID to trace; if not specified, trace all") 67 | parser.add_argument("-t", "--trace", action="store_true", 68 | help="print trace messages for each futex enter/exit") 69 | parser.add_argument("interval", nargs="?", default=5, type=int, 70 | help="interval in seconds to print summary") 71 | parser.add_argument("count", nargs="?", type=int, 72 | help="number of times to print the report before exiting") 73 | parser.add_argument("-c", "--command", 74 | help="execute and trace the specified command") 75 | 76 | args = parser.parse_args() 77 | 78 | pid = args.pid 79 | command = args.command 80 | interval = args.interval 81 | num_prints = args.count 82 | trace_all = args.trace 83 | 84 | if command is not None: 85 | print("Executing '%s' and tracing the resulting process." % command) 86 | pid = run_command_get_pid(command) 87 | 88 | bpf_source = """ 89 | #include 90 | #include 91 | #include 92 | #include 93 | 94 | struct comm_t { 95 | char name[TASK_COMM_LEN]; 96 | }; 97 | 98 | struct lock_key_t { 99 | u64 uaddr; 100 | u32 pid; 101 | u32 tgid; 102 | }; 103 | 104 | struct lock_info_t { 105 | u64 elapsed_blocked; 106 | u64 contention_count; 107 | u64 sid; 108 | }; 109 | 110 | BPF_HASH(pid_lock, u32, u64); 111 | BPF_HASH(pid_blocktime, u32, u64); 112 | BPF_HASH(tgid_comm, u32, struct comm_t); 113 | BPF_HASH(lock_stats, struct lock_key_t, struct lock_info_t, 1000000); 114 | BPF_STACK_TRACE(stack_traces, 16384); 115 | 116 | static inline int update_stats(u32 pid, u32 tgid, u64 uaddr, u64 block_time, u64 sid) { 117 | struct lock_key_t key = {}; 118 | struct lock_info_t zero = {}; 119 | struct lock_info_t *info; 120 | 121 | key.pid = pid; 122 | key.tgid = tgid; 123 | key.uaddr = uaddr; 124 | info = lock_stats.lookup_or_init(&key, &zero); 125 | info->elapsed_blocked += block_time; 126 | info->contention_count++; 127 | info->sid = sid; 128 | 129 | if (0 == tgid_comm.lookup(&tgid)) { 130 | struct comm_t comm; 131 | bpf_get_current_comm(&comm.name, sizeof(comm.name)); 132 | tgid_comm.update(&tgid, &comm); 133 | } 134 | return 0; 135 | } 136 | 137 | // FIXME: Should attach to sys_enter_futex and sys_exit_futex tracepoints here, 138 | // but that does not currently work 139 | int sys_futex_enter(struct pt_regs *ctx, u32 *uaddr, int op, u32 val, 140 | struct timespec *utime, u32 *uaddr2, u32 val3) { 141 | int cmd = op & FUTEX_CMD_MASK; 142 | if (cmd != FUTEX_WAIT) 143 | return 0; 144 | 145 | u64 pid_tgid = bpf_get_current_pid_tgid(); 146 | u32 pid = pid_tgid; 147 | u32 tgid = pid_tgid >> 32; 148 | 149 | if (!(THREAD_FILTER)) 150 | return 0; 151 | 152 | u64 timestamp = bpf_ktime_get_ns(); 153 | u64 uaddr64 = (u64) uaddr; 154 | pid_lock.update(&pid, &uaddr64); 155 | pid_blocktime.update(&pid, ×tamp); 156 | 157 | if (SHOULD_PRINT) 158 | bpf_trace_printk("enter sys_futex, pid = %u, uaddr = %x, " 159 | "cmd = %u\\n", pid, uaddr64, cmd); 160 | return 0; 161 | } 162 | 163 | int sys_futex_exit(struct pt_regs *ctx) { 164 | u64 pid_tgid = bpf_get_current_pid_tgid(); 165 | u32 pid = pid_tgid; 166 | u32 tgid = pid_tgid >> 32; 167 | if (!(THREAD_FILTER)) 168 | return 0; 169 | 170 | u64 *blocktime = pid_blocktime.lookup(&pid); 171 | u64 *uaddr = pid_lock.lookup(&pid); 172 | u64 timestamp = bpf_ktime_get_ns(); 173 | u64 elapsed; 174 | u64 sid; 175 | 176 | if (blocktime == 0 || uaddr == 0) 177 | return 0; // not FUTEX_WAIT, or (less likely) missed futex_enter 178 | 179 | elapsed = timestamp - *blocktime; 180 | 181 | sid = stack_traces.get_stackid(ctx, BPF_F_USER_STACK); 182 | update_stats(pid, tgid, *uaddr, elapsed, sid); 183 | pid_lock.delete(&pid); 184 | pid_blocktime.delete(&pid); 185 | 186 | if (SHOULD_PRINT) { 187 | bpf_trace_printk("exit sys_futex, uaddr = %x, elapsed = %uns\\n", 188 | uaddr == 0 ? 0 : *uaddr, elapsed); 189 | } 190 | return 0; 191 | } 192 | 193 | """ 194 | 195 | bpf_source = bpf_source.replace("SHOULD_PRINT", "1" if trace_all else "0") 196 | 197 | thread_filter = '1' 198 | if pid != -1: 199 | print("Tracing pid %d, Ctrl+C to quit." % pid) 200 | # 'tgid' in kernel space is what people thin of as 'pid' in userspace 201 | thread_filter = "tgid == %d" % pid 202 | else: 203 | print("Tracing all processes, Ctrl+C to quit.") 204 | 205 | bpf_source = bpf_source.replace("THREAD_FILTER", thread_filter) 206 | 207 | bpf_program = BPF(text=bpf_source) 208 | bpf_program.attach_kprobe(event="SyS_futex", fn_name="sys_futex_enter") 209 | bpf_program.attach_kretprobe(event="SyS_futex", fn_name="sys_futex_exit") 210 | 211 | def get_syms(stack, pid): 212 | global bpf_program 213 | syms = [] 214 | for addr in stack: 215 | s = bpf_program.sym(addr, pid, show_offset=True) 216 | syms.append(s) 217 | return syms 218 | 219 | def print_syms(syms): 220 | print("=========") 221 | for f in syms: 222 | print(f) 223 | print("=========") 224 | 225 | def is_android_monitor_lock(syms): 226 | for s in syms: 227 | if 'art::Monitor::Lock' in s: 228 | return True 229 | return False 230 | 231 | def disp_stack(stack, pid): 232 | for addr in stack: 233 | s = bpf_program.sym(addr, pid, show_offset=True) 234 | 235 | def create_tgid_stats(): 236 | global bpf_program 237 | stats = bpf_program["lock_stats"] 238 | res = {} 239 | stack_traces = bpf_program['stack_traces'] 240 | for key, val in stats.items(): 241 | # Only display Android monitor locks 242 | if val.sid >= 0: 243 | ust = stack_traces.walk(val.sid) 244 | syms = get_syms(ust, key.pid) 245 | if not is_android_monitor_lock(syms): 246 | continue 247 | else: 248 | continue 249 | 250 | if not key.tgid in res: 251 | res[key.tgid] = {} 252 | if not key.uaddr in res[key.tgid]: 253 | res[key.tgid][key.uaddr] = Lock() 254 | 255 | lock = res[key.tgid][key.uaddr] 256 | lock.update(val.contention_count, val.elapsed_blocked, syms) 257 | return res 258 | 259 | def print_comm_stats(stats): 260 | if stats == {}: 261 | return 262 | 263 | comms = bpf_program["tgid_comm"] 264 | print("\n%s:" % (datetime.now().strftime("%H:%M:%S"))) 265 | for tgid, locks in stats.items(): 266 | comm = comms[c_int(tgid)].name 267 | print("\n %s (%d):" % (comm, tgid)) 268 | sorted_locks = sorted(locks.items(), 269 | key=lambda x: x[1].elapsed_blocked, 270 | reverse=True) 271 | for addr, stats in sorted_locks: 272 | print(" %x: %dms (%d contentions involving %d threads, avg %dus)" % 273 | (addr, stats.elapsed_blocked / 1000000, 274 | stats.contention_count, stats.thread_count, 275 | stats.elapsed_blocked / stats.contention_count / 1000)) 276 | 277 | # No use of displaying lock stacks since we're only 278 | # filtering for Android monitor locks. 279 | # 280 | # print("Last stack for this lock:") 281 | # print_syms(stats.last_stack_syms) 282 | 283 | count_so_far = 0 284 | while True: 285 | if trace_all: 286 | print(bpf_program.trace_fields()) 287 | else: 288 | try: 289 | sleep(interval) 290 | except KeyboardInterrupt: 291 | exit() 292 | print_comm_stats(create_tgid_stats()) 293 | count_so_far += 1 294 | bpf_program['tgid_comm'].clear() 295 | bpf_program['lock_stats'].clear() 296 | bpf_program['pid_lock'].clear() 297 | bpf_program['pid_blocktime'].clear() 298 | 299 | if num_prints is not None and count_so_far >= num_prints: 300 | exit() 301 | -------------------------------------------------------------------------------- /buildimage: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # Given a path, make an ext4 image out of it, store it in $2 5 | dd if=/dev/zero of=$2 bs=4k count=1 seek=$((256 * 1024 * 7)) 6 | mkfs.ext4 $2 7 | 8 | OUT=`mktemp -d` 9 | mount -o loop $2 $OUT/ 10 | rsync -ra $1/ $OUT/ 11 | umount $OUT/ 12 | -------------------------------------------------------------------------------- /buildstrap: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | spath="$(dirname "$(readlink -f "$0")")" 4 | source $spath/utils/support 5 | source $spath/utils/banners 6 | 7 | ARCH=$1 8 | DISTRO=$2 9 | TDIR=$3 10 | OUT_TMP=$4 11 | PACKAGES=$5 12 | EXTRA_FILES="$(cat $6)" 13 | INSTALL_BCC=$7 14 | SKIP_DEVICE=$8 # Skip any device-specific stages 15 | VARIANT="--variant=minbase" 16 | 17 | time qemu-debootstrap --arch $ARCH --include=$PACKAGES $VARIANT \ 18 | $DISTRO $OUT_TMP http://ftp.us.debian.org/debian/ 19 | 20 | # Some reason debootstrap leaves these mounted 21 | umount $OUT_TMP/proc/sys/fs/binfmt_misc || true 22 | umount $OUT_TMP/proc || true 23 | 24 | # Make bash the default shell 25 | chroot $OUT_TMP rm /bin/sh || true 26 | chroot $OUT_TMP ln -s /bin/bash /bin/sh || true 27 | cp $spath/addons/bashrc $OUT_TMP/.bashrc 28 | cp $spath/addons/bashrc.common $OUT_TMP/.bashrc.common 29 | cp $spath/addons/bashrc.silent $OUT_TMP/.bashrc.silent 30 | cp $spath/addons/get_kvers.sh $OUT_TMP/ 31 | 32 | for f in $EXTRA_FILES; do 33 | if [ $f == "none" ]; then continue; fi 34 | cp $f $OUT_TMP/ 35 | done 36 | 37 | # Cleanup 38 | rm -rf $OUT_TMP/lib/udev/* 39 | rm -rf $OUT_TMP/var/lib/apt/lists/* 40 | rm -rf $OUT_TMP/var/cache/apt/archives/*deb 41 | rm -rf $OUT_TMP/usr/share/locale/* 42 | rm -rf $OUT_TMP/usr/lib/share/locale/* 43 | rm -rf $OUT_TMP/usr/share/doc/* 44 | rm -rf $OUT_TMP/usr/lib/share/doc/* 45 | rm -rf $OUT_TMP/usr/share/ieee-data/* 46 | rm -rf $OUT_TMP/usr/lib/share/ieee-data/* 47 | rm -rf $OUT_TMP/usr/share/man/* 48 | rm -rf $OUT_TMP/usr/lib/share/man/* 49 | 50 | # Fix apt-get issue: Android requires _apt user to be in the 51 | # AID_INET group which is also android specific. 52 | grep -ri _apt:x:100:65534 $OUT_TMP/etc/passwd > /dev/null 2>&1 53 | if [ $? -ne 0 ]; then 54 | c_warning "_apt user cannot be added to AID_INET group" 55 | else 56 | sed -i -e 's/_apt:x:100:65534/_apt:x:100:3003/' $OUT_TMP/etc/passwd 57 | fi 58 | 59 | # Add a default DNS server 60 | echo "nameserver 4.2.2.2" > $OUT_TMP/etc/resolv.conf 61 | 62 | # Clone BCC if needed 63 | if [ $INSTALL_BCC -eq 1 ]; then 64 | git clone https://github.com/iovisor/bcc.git $TDIR/debian/bcc-master 65 | cp $spath/bcc/build-bcc.sh $TDIR/debian/bcc-master/ 66 | chroot $OUT_TMP /bcc-master/build-bcc.sh 67 | fi 68 | 69 | # Should we really do this? 70 | chmod -R 0777 $TDIR/ 71 | 72 | [ $SKIP_DEVICE -eq 0 ] || exit 0 73 | 74 | c_info "Compressing new filesystem to prepare to push to Android /data/androdeb/" 75 | tar -zcf $TDIR/deb.tar.gz -C $TDIR debian 76 | 77 | chmod 0777 $TDIR/deb.tar.gz 78 | -------------------------------------------------------------------------------- /packages/bcc: -------------------------------------------------------------------------------- 1 | PACKAGES+=" 2 | llvm-7-dev 3 | libclang-7-dev 4 | libelf-dev 5 | libfl-dev 6 | libunwind-dev 7 | libdw-dev 8 | git 9 | gcc 10 | libtool 11 | autoconf 12 | make 13 | cmake 14 | iperf 15 | arping 16 | ethtool 17 | flex 18 | bison 19 | python 20 | clang-7 21 | python-netaddr 22 | python-pyroute2 23 | python3-distutils 24 | " 25 | 26 | INSTALL_BCC=1 27 | -------------------------------------------------------------------------------- /packages/compilers: -------------------------------------------------------------------------------- 1 | PACKAGES+="\ 2 | git 3 | clang-7 4 | gcc 5 | libtool 6 | autoconf 7 | make 8 | cmake 9 | " 10 | -------------------------------------------------------------------------------- /packages/editors: -------------------------------------------------------------------------------- 1 | PACKAGES+="\ 2 | vim 3 | nano 4 | git 5 | " 6 | -------------------------------------------------------------------------------- /packages/others: -------------------------------------------------------------------------------- 1 | PACKAGES+="\ 2 | xz-utils 3 | " 4 | -------------------------------------------------------------------------------- /packages/scheduler: -------------------------------------------------------------------------------- 1 | PACKAGES+="\ 2 | git 3 | rt-app 4 | " 5 | -------------------------------------------------------------------------------- /packages/tracers: -------------------------------------------------------------------------------- 1 | PACKAGES+="\ 2 | linux-perf 3 | trace-cmd 4 | strace 5 | " 6 | -------------------------------------------------------------------------------- /utils/banners: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | 3 | print_prepare_banner() { 4 | c_info "Preparing device..." 5 | if [ $FULL -eq 1 ]; then 6 | c_info "Doing a full install." 7 | else 8 | c_info "Doing a base install." 9 | fi 10 | c_info "" 11 | } 12 | 13 | usage() { 14 | c_info "USAGE:" 15 | c_info "adeb" 16 | c_info " shell Enter the androdeb shell environment and get to work!" 17 | c_info " remove Remove androdeb from the device" 18 | c_info " git-pull Git pull androdeb to update it on your host" 19 | c_info " pull Copy files from the androdeb filesystem in the device" 20 | c_info " push Copy files to the androdeb filesystem in the device" 21 | c_info "" 22 | c_info " prepare Prepare the device (when running for the first time)" 23 | c_info " By default, this will download and install a base image." 24 | c_info " ** Folowing are the prepare options **" 25 | c_info " --full Pass this to prepare to download and install the full image which" 26 | c_info " contains compilers, editors, tracers etc." 27 | c_info "" 28 | c_info " --build Instead of download, build and install the image onto the device" 29 | c_info "" 30 | c_info " --archive Use archive for root fs (overrides all other prepare options)" 31 | c_info "" 32 | c_info " --buildtar While preparing, also build a tar.gz.zip file of the filesystem," 33 | c_info " this is how images that are downloaded by prepare are built" 34 | c_info "" 35 | c_info " --build-image Build an ext4 .img with the base image and BCC (useful for Qemu)" 36 | c_info "" 37 | c_info " ** Folowing are misc build options **" 38 | c_info " --tempdir Use a specific temporary directory for build operation" 39 | c_info " --arch Specify an ARCH to build for (default arm64)" 40 | c_info " --distro Debian distro to base on (default is buster)" 41 | c_info "" 42 | c_info " ** Folowing are the options for BCC **" 43 | c_info " --bcc Build and install BCC onto the device, from source" 44 | c_info " BCC is already included in 'prepare --full'" 45 | c_info "" 46 | c_info " ** Folowing are device specific options ** " 47 | c_info " --device Serial number of adb device." 48 | c_info " -s Serial number of adb device." 49 | c_info "" 50 | c_info " --ssh Use ssh instead of adb to talk to the device" 51 | c_info " The uri of the device is expected to be passed with this option" 52 | c_info " ie: user@ip_address" 53 | c_info "" 54 | c_info " --sshpass The ssh password to use" 55 | c_info "" 56 | c_info " --debug Debug all execution." 57 | exit 1 58 | } 59 | 60 | 61 | -------------------------------------------------------------------------------- /utils/remote: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | # Utilities to talk to the device more easily using adb or ssh 3 | 4 | function validate_remote() { 5 | if [ "$REMOTE" == "adb" ]; then 6 | 7 | if [ -z "$ADB" ]; then 8 | c_error "ADB is not defined" 9 | exit -1 10 | fi 11 | elif [ "$REMOTE" == "ssh" ]; then 12 | 13 | if [ -z "$SSH_URI" ]; then 14 | c_error "SSH_URI is not defined" 15 | exit -1 16 | fi 17 | else 18 | c_error "Unknown remote: $REMOTE" 19 | exit -1 20 | fi 21 | } 22 | 23 | # 24 | # Adb Commands 25 | # 26 | function remote_adb_shell() { 27 | $ADB shell "$@" 28 | } 29 | 30 | function remote_adb_pull() { 31 | if [ "x$1" == "x-a" ]; then 32 | PRESERVE="-a" 33 | c_info "Preserving filestamps and mode" 34 | shift || true 35 | fi 36 | file_count=`count_sources $@` 37 | i=0 38 | while [ $i -lt $file_count ]; do 39 | files["$i"]=/data/androdeb/debian/$1 40 | shift || true 41 | i=$((i + 1)) 42 | done 43 | $ADB pull $PRESERVE "${files[@]}" "$@" 44 | } 45 | 46 | function remote_adb_push() { 47 | file_count=`count_sources $@` 48 | i=0 49 | while [ $i -lt $file_count ]; do 50 | files["$i"]=$1 51 | shift || true 52 | i=$((i + 1)) 53 | done 54 | dest=/data/androdeb/debian/$1 55 | $ADB push $sync "${files[@]}" $dest 56 | } 57 | 58 | function remote_adb_copy() { 59 | $ADB push "$@" 60 | } 61 | 62 | function remote_adb_root() { 63 | do_adb_root "$ADB" 64 | } 65 | 66 | # 67 | # Shell Commands 68 | # 69 | function remote_ssh_shell() { 70 | $SSHPASS ssh $SSH_URI "$@" 71 | } 72 | 73 | function remote_ssh_pull() { 74 | src="${@:1:$#-1}" 75 | DEST="${@:$#}" 76 | 77 | for file in $src 78 | do 79 | SRC="$SRC /data/androdeb/debian/$file" 80 | done 81 | $SSHPASS scp -r $SSH_URI:"$SRC" $DEST 82 | } 83 | 84 | function remote_ssh_push() { 85 | SRC="${@:1:$#-1}" 86 | DEST="/data/androdeb/debian/${@:$#}" 87 | $SSHPASS scp -r $SRC $SSH_URI:$DEST 88 | } 89 | 90 | function remote_ssh_copy() { 91 | SRC="${@:1:$#-1}" 92 | DEST="${@:$#}" 93 | $SSHPASS scp -r $SRC $SSH_URI:$DEST 94 | } 95 | 96 | function remote_ssh_root() { 97 | # We always assume the ssh user has root access 98 | true 99 | } 100 | 101 | # 102 | # Abstraction layer 103 | # 104 | function remote_shell() { 105 | # Sanity everything we need is defined correctly 106 | validate_remote 107 | 108 | remote_${REMOTE}_shell "$@" 109 | } 110 | 111 | function remote_pull() { 112 | # Sanity everything we need is defined correctly 113 | validate_remote 114 | 115 | remote_${REMOTE}_pull "$@" 116 | } 117 | 118 | function remote_push() { 119 | # Sanity everything we need is defined correctly 120 | validate_remote 121 | 122 | remote_${REMOTE}_push "$@" 123 | } 124 | 125 | # Like push but for copying files outsisde deb-fs 126 | # ie: full paths are expected here 127 | function remote_copy() { 128 | # Sanity everything we need is defined correctly 129 | validate_remote 130 | # 131 | remote_${REMOTE}_copy "$@" 132 | } 133 | 134 | function remote_root() { 135 | # Sanity everything we need is defined correctly 136 | validate_remote 137 | 138 | remote_${REMOTE}_root 139 | } 140 | -------------------------------------------------------------------------------- /utils/support: -------------------------------------------------------------------------------- 1 | #!/bin/bash -x 2 | # Utilities to interact with android more easily 3 | 4 | function run_quiet() { eval "$* >/dev/null 2>&1"; } 5 | 6 | make_csv() { 7 | out="" 8 | in=$1 9 | for p in $in; do 10 | if [ "x$out" == "x" ]; then 11 | out=$p 12 | else 13 | out="$out,$p" 14 | fi 15 | done 16 | echo $out 17 | } 18 | 19 | make_spaces() { 20 | out="" 21 | in=$1 22 | for p in $in; do 23 | if [ "x$out" == "x" ]; then 24 | out=$p 25 | else 26 | out="$out $p" 27 | fi 28 | done 29 | echo $out 30 | } 31 | 32 | cmd_exists() { 33 | which $1 > /dev/null 34 | return $? 35 | } 36 | 37 | do_adb_root() { 38 | ADB="$1" 39 | $ADB root > /dev/null 2>&1 40 | return $? 41 | } 42 | 43 | die() { 44 | exit_code=$1 45 | msg=$2 46 | c_error "$msg" 47 | exit $exit_code 48 | } 49 | 50 | die_if_no_androdeb() { 51 | set +e 52 | remote_shell ls /data/androdeb/debian > /dev/null 2>&1 53 | if [ $? -ne 0 ]; then die 8 "Existing androdeb env not found on device. $1"; fi 54 | set -e 55 | } 56 | 57 | # Helper function to count number of source arguments in a list 58 | # when more than one argument is supplied, it is assumed the last argument 59 | # is a destination 60 | count_sources() { 61 | local source_count=$# 62 | if [ $source_count -gt 1 ]; then 63 | source_count=$((source_count - 1)) 64 | fi 65 | echo "$source_count" 66 | } 67 | 68 | # Borrowed from project LISA. 69 | ################################################################################ 70 | # Logging functions 71 | ################################################################################ 72 | c_error() { 73 | NOW=$(date +"%H:%m:%S") 74 | # If there is only one parameter, let's assume it's just the message 75 | if [ $# -gt 1 ]; then 76 | local parent_lineno="$1" 77 | local message="$2" 78 | echo -e "${red}$NOW - ERROR: on or near line ${parent_lineno}: ${message}${nocol}" 79 | return 80 | fi 81 | 82 | local message="$1" 83 | echo -e "${red}$NOW - ERROR : ${message}${nocol}" 84 | } 85 | 86 | c_warning() { 87 | NOW=$(date +"%H:%m:%S") 88 | # If there is only one parameter, let's assume it's just the message 89 | if [ $# -gt 1 ]; then 90 | local parent_lineno="$1" 91 | local message="$2" 92 | echo -e "${yellow}$NOW - WARNING: on or near line ${parent_lineno}: ${message}${nocol}" 93 | return 94 | fi 95 | local message="$1" 96 | echo -e "${yellow}$NOW - WARNING : ${message}${nocol}" 97 | } 98 | 99 | c_info() { 100 | NOW=$(date +"%H:%m:%S") 101 | # If there is only one parameter, let's assume it's just the message 102 | if [ $# -gt 1 ]; then 103 | local parent_lineno="$1" 104 | local message="$2" 105 | echo -e "${blue}$NOW - INFO: on or near line ${parent_lineno}: ${message}${nocol}" 106 | return 107 | fi 108 | local message="$1" 109 | echo -e "${blue}$NOW - INFO : ${message}${nocol}" 110 | } 111 | 112 | d_notify() { 113 | MESSAGE=$1 114 | ICON=$2 115 | # Let's try to send a desktop notification, 116 | # silently fails if there is not support. 117 | notify-send \ 118 | --icon=$ICON \ 119 | --urgency=critical \ 120 | --expire-time=1500 \ 121 | "Test Series" \ 122 | "$MESSAGE" \ 123 | 2>/dev/null 124 | } 125 | 126 | my_tput() { 127 | if [ "${TERM-dumb}" == dumb ]; then 128 | return 129 | fi 130 | tput $* 131 | } 132 | 133 | box_out() 134 | { 135 | local s=("$@") b w 136 | for l in "${s[@]}"; do 137 | ((w<${#l})) && { b="$l"; w="${#l}"; } 138 | done 139 | my_tput setaf 3 140 | echo -e "|-${b//?/-}-|" 141 | for l in "${s[@]}"; do 142 | printf '| %s%*s%s |\n' "$(my_tput setaf 4)" "-$w" "$l" "$(my_tput setaf 3)" 143 | # echo "|-${b//?/-}-|" 144 | done 145 | echo "|-${b//?/-}-|" 146 | my_tput sgr 0 147 | } 148 | 149 | 150 | ################################################################################ 151 | # Colors 152 | ################################################################################ 153 | 154 | if [ -t 1 ]; then 155 | ncolors=$(my_tput colors) 156 | if [ -n "${ncolors}" ] && [ ${ncolors} -ge 8 ]; then 157 | nocol='\e[0m' # No Color 158 | white='\e[1;37m' 159 | black='\e[0;30m' 160 | blue='\e[0;34m' 161 | lblue='\e[1;34m' 162 | green='\e[0;32m' 163 | lgreen='\e[1;32m' 164 | cyan='\e[0;36m' 165 | lcyan='\e[1;36m' 166 | red='\e[0;31m' 167 | lred='\e[1;31m' 168 | purple='\e[0;35m' 169 | lpurple='\e[1;35m' 170 | brown='\e[0;33m' 171 | yellow='\e[1;33m' 172 | grey='\e[0;30m' 173 | lgrey='\e[0;37m' 174 | fi 175 | fi 176 | --------------------------------------------------------------------------------