├── Linux Kernel.md ├── README.md └── chromium.md /Linux Kernel.md: -------------------------------------------------------------------------------- 1 | # Linux Kernel 2 | 3 | Linux kernel compilation is complex. The following link maybe helpful: 4 | 5 | [building linux kernel into a bitcode file](https://github.com/ssl-tud/k-miner) 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Build whole programs into LLVM IR 2 | 3 | Here is some notes for how to build the projects into a single LLVM IR bitcode file. Generally, get a integrate LLVM IR bitcode has several purposes such as whole program analysis or optimization. Based on information on the network, there are two ways, named gold plugin and whole-program-llvm respectively, to achieve this goal. (Update : LLD is also an alternative solution but it is not stable enough compared to gold plugin in earlier versions of LLVM.) 4 | 5 | LLVM official organization introduced the gold plugin to support LTO (link time optimization) and further be used to build the whole project into LLVM bitcodes. Some useful opensource tools has also been published on github and the most outstanding in those tools is wllvm (whole-program-llvm, write in python and support Linux and Mac). 6 | 7 | The mainly differences between the gold plugin and wllvm are : 8 | * the gold plugin only support ELF files while wllvm can also support MACHO files. 9 | * wllvm unsupports share libraries and does not need the targets are position independent (i.e. '-fPIC'). 10 | * the gold plugin performs a real link process (with LTO, i.e. '-flto') while wllvm simply uses llvm-link (with less optimizations) to connect all intermediate bitcodes produced in compile time. 11 | 12 | More information can be found in reference but the bitcodes (of executables) produced by above methods are extremly similar in practice. 13 | 14 | ## The gold plugin 15 | 16 | ### download and build binutils. 17 | ```sh 18 | # some necessary pre-requisite 19 | sudo apt install bison flex libncurses5-dev texinfo 20 | # get the trunk version of binutils 21 | git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils 22 | # create a build directory 23 | mkdir binutils_build && mkdir binutils_install && cd binutils_build 24 | # configuration 25 | ../binutils/configure --enable-gold --enable-plugins --disable-werror --prefix=/path/to/binutils_install 26 | # build/compile 27 | make -j4 28 | make install 29 | ``` 30 | ### build LLVM with binutils header. 31 | ```sh 32 | # some necessary pre-requisite 33 | sudo apt install subversion cmake zlib1g zlib1g-dev 34 | # get the trunk version of LLVM and clang 35 | cd /where/you/want/llvm/to/live 36 | svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm 37 | cd llvm/tools 38 | svn co http://llvm.org/svn/llvm-project/cfe/trunk clang 39 | cd /where/you/want/to/build/llvm 40 | # create build and install directories 41 | mkdir llvm_build && mkdir llvm_install && cd llvm_build 42 | # configuration with binutils header 43 | cmake ../llvm -DLLVM_BINUTILS_INCDIR=/path/to/binutils/include -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_INSTALL_PREFIX=/path/to/llvm_install 44 | # build/compile 45 | make -j4 46 | ``` 47 | the following cmake flags are optional : 48 | ```sh 49 | -DLLVM_ENABLE_ASSERTIONS=On 50 | -DLLVM_ENABLE_RTTI=On (LLVM Release disables RTTI by default) 51 | ``` 52 | 53 | ### add newest binutils and newest LLVM to envirnoment variables. 54 | ```sh 55 | vim ~/.bashrc 56 | # use (ESC + a) to insert sentences to current file in vi/vim 57 | export PATH=/path/to/binutils_install/bin:$PATH # add this line to bashrc 58 | export PATH=/path/to/llvm/build/bin:$PATH # add this line to bashrc 59 | # use (ESC + :wq) to quit vi/vim 60 | ``` 61 | then type the command to shell 62 | ```sh 63 | source ~/.bashrc 64 | ``` 65 | to evaluate the envirnoment variables, reopen shell is also workful. 66 | 67 | ### test (optional) 68 | 69 | we use sed as a benchmark to test whether the gold plugins work correctly in LLVM. 70 | ```sh 71 | # check the envirnoment variables are correct. 72 | clang --version 73 | ld --version 74 | # switch to workspace where to build sed. 75 | cd /path/to/workspace 76 | # create a build directory. 77 | mkdir sed_build && cd sed_build 78 | # configuration 79 | /path/to/sed/source/configure CC=clang CFLAGS='-flto' LDFLAGS='-flto -fuse-ld=gold -Wl,-plugin-opt=save-temps' RANLIB=llvm-ranlib 80 | # build/compile 81 | make 82 | ``` 83 | If everything is ok, sed.0.0.preopt.bc can be found under the build directory. (sepecifically in /path/to/sed_build/sed) 84 | 85 | ## whole-program-llvm 86 | 87 | ### local python environment (optional) 88 | we create isolated python environments for wllvm. Make sure virtualenv already be installed: 89 | ```sh 90 | sudo pip install virtualenv 91 | ``` 92 | Now we create and activate a new isolated python envirnoment: 93 | ```sh 94 | cd /where/we/want/to/live/python/envirnoments 95 | virtualenv pyenv --no-site-packages 96 | source pyenv/bin/activate 97 | ``` 98 | ### install wllvm 99 | fetch wllvm and setup: 100 | ```sh 101 | git clone https://github.com/travitch/whole-program-llvm wllvm 102 | cd wllvm && python setup.py install 103 | ``` 104 | note wllvm is also available on pip. 105 | ```sh 106 | pip install wllvm 107 | ``` 108 | ### test (optinal) 109 | build sed and generate the bitcodes like: 110 | ```sh 111 | mkdir sed_build && cd sed_build 112 | /path/to/sed/source/configure CC=wllvm 113 | make -j4 114 | cd sed && extract-bc sed 115 | ``` 116 | Then you can find sed.bc in the same folder as the sed executable. 117 | 118 | ## Reference 119 | 120 | [Getting Started with the LLVM System](http://llvm.org/docs/GettingStarted.html) 121 |
[The LLVM gold plugin](https://llvm.org/docs/GoldPlugin.html) 122 |
[Compiling Autotooled projects to LLVM Bitcode](http://gbalats.github.io/2015/12/10/compiling-autotooled-projects-to-LLVM-bitcode.html) 123 |
[Install LLVM Gold plugin on Ubuntu](https://github.com/SVF-tools/SVF/wiki/Install-LLVM-Gold-Plugin-on-Ubuntu) 124 |
[whole-program-llvm](https://github.com/travitch/whole-program-llvm) 125 | -------------------------------------------------------------------------------- /chromium.md: -------------------------------------------------------------------------------- 1 | # Build chromium into LLVM IR 2 | 3 | Here is some notes for how to build the chromium project into a single LLVM IR bitcode file (ONLY on Linux). Chromium official homepage detailed records how to checkout and build the chromium project. To avoid unnecessary compatibility problems, here we prefer Ubuntu 16.04 (64 bit) to build the newest chromium. 4 | 5 | ## check out the chromium project 6 | 7 | The first step of check out is clone depot_tools and configure depot_tools according to offical website of chromium. 8 | ``` 9 | cd where-to-live-depot_tools 10 | git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git 11 | export PATH="$PATH:/path/to/depot_tools" 12 | ``` 13 | ### 1. from googlesource 14 | 15 | ```sh 16 | mkdir ~/chromium && cd ~/chromium 17 | fetch --nohooks chromium 18 | cd src && ./build/install-build-deps.sh 19 | gclient runhooks 20 | ``` 21 | 22 | ### 2. from github 23 | 24 | For chinese mainland users, we perfer to clone code from a mirror site instead of googlesource. the mirror of chromium can be found easily on github 25 | ```sh 26 | git clone https://github.com/chromium/chromium.git src 27 | gclient config --spec 'solutions = [ 28 | { 29 | "url" : "https://github.com/chromium/chromium.git", 30 | "managed" : False, 31 | "name" : "src", 32 | "custom_deps" : {}, 33 | }, 34 | ]' 35 | gclient sync --nohooks 36 | cd src && ./build/install-build-deps.sh 37 | gclient runhooks 38 | ``` 39 | Once ./build/install-build-deps.sh be executed, the build envirnoment of chromium is complete. The operating system is running in low graphics mode when we reboot Ubuntu. Switch to terminal (Ctrl + F1) and type 40 | ```sh 41 | sudo apt update && sudo apt upgrade 42 | ``` 43 | to fix it. 44 | 45 | ## generate build system files into the build directory 46 | 47 | assume we are undering the src directory (the root of chromium project). 48 | ```sh 49 | gn args out/mybuild 50 | ``` 51 | this command will bring users to a vi/vim editor, type the following configuration for compiling: 52 | ```sh 53 | # use (ESC + a) to insert sentences to current file in vi/vim 54 | clang_base_path = "/path/to/llvm_build" 55 | binutils_path = "/path/to/binutils_install/bin" 56 | clang_use_chrome_plugins = false 57 | is_component_build = true 58 | enable_nacl = false 59 | is_debug = false 60 | symbol_level = 0 61 | use_lld = false 62 | use_gold = true 63 | treat_warnings_as_errors = false 64 | # use (ESC + :wq) to quit vi/vim 65 | ``` 66 | disable NACL (native client) is necessary because its built-in compiler does not support '-flto'. 67 | 68 | ## modify build system files 69 | 70 | 71 | c/cxx/link flags should be changed because we will save temporary bitcodes in compile time. 72 | Use your favorite editor to open directory out/mybuild then 73 | ```sh 74 | replace all appears of "${ldflags}" to "${ldflags} -flto -fuse-ld=gold -Wl,-plugin-opt=save-temps" 75 | replace all appears of "${cflags_c}" to "${cflags_c} -flto" 76 | replace all appears of "${cflags_cc}" to "${clags_cc} -flto" 77 | replace all appears of "../../third_party/binutils/Linux_x64/Release/bin" to your binutils path "/path/to/binutils_install/bin" 78 | ``` 79 | If you want to add some flags yourself, you can change the build command as below, for example 80 | ```sh 81 | replace all appears of "${cflags_c}" to "${cflags_c} -flto -g3 -O0" 82 | ``` 83 | 84 | ## building 85 | 86 | ```sh 87 | ninja -C out/mybuild chrome -j 16 88 | ``` 89 | use -v to obvious the building flows. 90 | 91 | ## some issues 92 | 93 | There are some shared libraries can not be transformed to LLVM IR normally. 94 | ```sh 95 | ./libcontent.so (llvm-dis : error : Malformed block) 96 | ``` 97 | 98 | ## Reference 99 | 100 |
[offical : build chromium on linux](https://chromium.googlesource.com/chromium/src/+/master/docs/linux_build_instructions.md) 101 |
[offical : build chromium by clang](https://chromium.googlesource.com/chromium/src/+/master/docs/clang.md) 102 |
[build chromium into LLVM IR](https://github.com/SVF-tools/SVF/wiki/Compiling-Chrome-using-flto) 103 | 104 | --------------------------------------------------------------------------------