├── .gitmodules ├── .travis.yml ├── README.md ├── deploy.sh ├── deploy_rsa.enc ├── generate_analysis.sh └── img └── opt_viewer_sample.png /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "cpython"] 2 | path = cpython 3 | url = https://github.com/python/cpython/ 4 | [submodule "llvm"] 5 | path = llvm 6 | url = https://github.com/llvm-mirror/llvm 7 | [submodule "thrift"] 8 | path = thrift 9 | url = https://github.com/apache/thrift 10 | [submodule "kfr"] 11 | path = kfr 12 | url = https://github.com/kfrlib/kfr 13 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | dist: trusty 2 | compiler: clang 3 | sudo: required 4 | 5 | # for kfr: 6 | addons: 7 | apt: 8 | sources: 9 | - ubuntu-toolchain-r-test 10 | packages: 11 | - libmpfr-dev 12 | # ^^ for kfr 13 | 14 | before_install: 15 | - wget -nv -O - http://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - 16 | - sudo apt-add-repository -y 'deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-4.0 main' 17 | - sudo apt-get update -qq 18 | - virtualenv optviewer 19 | - source optviewer/bin/activate ; pip install pyyaml pygments 20 | install: 21 | - sudo apt-get install -qq -y clang-4.0 binutils-gold 22 | script: 23 | - ./generate_analysis.sh 24 | - ./deploy.sh 25 | 26 | branches: 27 | only: 28 | - master 29 | - dev 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | ## What is opt-viewer? 3 | 4 | opt-viewer can tell you more about missed optimization opportunities. The 5 | optimizers that `clang` uses can explain the rationale for why their specific 6 | optimization method couldn't be leveraged in given parts of your source. 7 | 8 | ![Build status](https://travis-ci.org/androm3da/optviewer-demo.svg?branch=master) 9 | 10 | ## Output Examples 11 | * [CPython](https://androm3da.github.io/optviewer-demo/output_analysis/cpython/) 12 | * [KFR](https://androm3da.github.io/optviewer-demo/output_analysis/kfr/) DSP lib ([repo](https://github.com/kfrlib/kfr/), [website](https://www.kfrlib.com/)) 13 | 14 | # Usage 15 | 16 | ## Prep 17 | 18 | Grab a very recent [clang, 4.0 or later](http://releases.llvm.org) for your 19 | architecture/OS distro and install or unpack it on your machine. 20 | 21 | The example below is for ARM7a linux, there are several other releases available. 22 | 23 | curl -O http://releases.llvm.org/4.0.0/clang+llvm-4.0.0-armv7a-linux-gnueabihf.tar.xz 24 | tar xf clang+llvm-4.0.0-rc1-armv7a-linux-gnueabihf.tar.xz 25 | 26 | Or instead you might have Ubuntu. Below is how you would do it for Ubuntu 27 | Trusty. If you're not sure which Ubuntu release you have, see ["Checking Your 28 | Ubuntu Version"](https://help.ubuntu.com/community/CheckingYourUbuntuVersion). 29 | 30 | wget -nv -O - http://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - 31 | sudo apt-add-repository -y 'deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-4.0 main' 32 | sudo apt-get update -qq 33 | sudo apt-get install -qq -y clang-4.0 llvm-4.0 34 | 35 | Next, you must get a copy of opt-viewer. For now, it's not distributed with 36 | clang+llvm releases, so a simple way to get it is to just get a copy of the 37 | llvm project [source from github](https://github.com/llvm-mirror/llvm). Unpack 38 | or clone it somewhere convenient and note where you put it. 39 | 40 | Once you have a new clang installed, build your software with it. You must 41 | include "`-fsave-optimization-record`" among the arguments. Imagine we have 42 | the following sample C/C++ project: 43 | 44 | sample/ 45 | └── src 46 | ├── bar.cpp 47 | ├── baz.cpp 48 | ├── foo.c 49 | └── Makefile 50 | 51 | 52 | If you're using `make`, you could add these lines to your `Makefile`: 53 | 54 | CXXFLAGS+=-fsave-optimization-record 55 | CLAGS+=-fsave-optimization-record 56 | 57 | and then clang will build your program or library like so: 58 | 59 | clang-4.0 -fsave-optimization-record -c -o foo.o foo.cpp 60 | ... 61 | 62 | `clang` will create build annotation artifacts in the YAML format. Now 63 | you can use `opt-viewer` to see these in relation to your project. 64 | 65 | 66 | ## Generating output 67 | 68 | First, gather the necessary dependencies for opt-viewer: 69 | 70 | virtualenv optviewer_env 71 | source optviewer_env/bin/activate 72 | pip install pyyaml pygments 73 | 74 | Let's assume your `sample` project is in the same directory as 75 | the one where you unpacked/cloned `llvm`. Then you should invoke `opt-viewer` 76 | like so: 77 | 78 | llvm/utils/opt-viewer/opt-viewer.py -source-dir sample/src/ \ 79 | sample/src/baz.opt.yaml \ 80 | sample/src/bar.opt.yaml \ 81 | sample/src/foo.opt.yaml \ 82 | ./output_report/ 83 | 84 | In the `output_report` dir, we'll have some output files that could look like 85 | below: 86 | 87 | ![opt-viewer screenshot](https://github.com/androm3da/optviewer-demo/raw/master/img/opt_viewer_sample.png) 88 | 89 | ## Advanced usage: PGO 90 | 91 | If you are able to leverage profile-guided-optimization (PGO), opt-viewer will 92 | produce an index that's sorted by the most-frequently executed code 93 | ("hotness"). This makes it easy to see which functions and methods would 94 | benefit the most from improved optimization. 95 | 96 | For more info on PGO, see [the `clang` manual on 97 | PGO](https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization). 98 | 99 | 100 | ## Appendix 101 | 102 | * `opt-viewer` was introduced in "Compiler-assisted Performance Analysis", 2016 Bay Area LLVM Developer's Meeting ([slides](http://llvm.org/devmtg/2016-11/Slides/Nemet-Compiler-assistedPerformanceAnalysis.pdf), [video](https://youtu.be/qq0q1hfzidg)) 103 | -------------------------------------------------------------------------------- /deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | set -euo pipefail 4 | 5 | configure_identity() { 6 | ENCRYPTION_LABEL=d87c07fd1a5b 7 | ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key" 8 | ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv" 9 | ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR} 10 | ENCRYPTED_IV=${!ENCRYPTED_IV_VAR} 11 | 12 | # Decrypt the key we'll use to publish this change to 13 | # github: 14 | openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in deploy_rsa.enc -out deploy_key -d 15 | chmod 600 deploy_key 16 | eval $(ssh-agent -s) 17 | ssh-add deploy_key 18 | } 19 | 20 | THIS_REPO_SHA=$(git rev-parse --verify HEAD) 21 | 22 | configure_identity 23 | 24 | REPO=git@github.com:androm3da/optviewer-demo.git 25 | git clone $REPO ${HOME}/output_repo 26 | cd ${HOME}/output_repo 27 | 28 | SOURCE_BRANCH="master" 29 | TARGET_BRANCH="gh-pages" 30 | 31 | export GENERATED_OUTPUT=${HOME}/output_analysis/ 32 | #find ${HOME}/output_repo/output_analysis/ 33 | git fetch --all 34 | git checkout ${TARGET_BRANCH} 35 | 36 | rm -rf ${HOME}/output_repo/output_analysis/ 37 | mv ${GENERATED_OUTPUT} ${HOME}/output_repo/ 38 | 39 | git add output_analysis/ 40 | git commit -m "Deploy output for '${THIS_REPO_SHA}'" 41 | 42 | # Publish to the gh-pages branch 43 | 44 | git push origin ${TARGET_BRANCH} 45 | git push origin ${TARGET_BRANCH} 46 | -------------------------------------------------------------------------------- /deploy_rsa.enc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/androm3da/optviewer-demo/0d46bbb23420a756a03bea48259d1a54fefe93c0/deploy_rsa.enc -------------------------------------------------------------------------------- /generate_analysis.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash -ex 2 | 3 | 4 | source optviewer/bin/activate 5 | 6 | set -euo pipefail 7 | export OUTPUT=${HOME}/output_analysis/ 8 | 9 | mkdir -p ${OUTPUT}/cpython 10 | mkdir -p ${OUTPUT}/kfr 11 | 12 | # CPython 13 | build_cpython() 14 | { 15 | cd cpython; CC=clang-4.0 CFLAGS="-O3 -fsave-optimization-record" ./configure --with-optimizations && make ; cd - 16 | } 17 | 18 | build_cpython 2>&1 | tee ${OUTPUT}/cpython/build.log 19 | CPY_YAML=$(find cpython -name '*.opt.yaml') 20 | ./llvm/utils/opt-viewer/opt-viewer.py -source-dir cpython/ ${CPY_YAML} ${OUTPUT}/cpython/ 21 | 22 | build_kfr() 23 | { 24 | mkdir kfr_build 25 | cd kfr_build 26 | cmake \ 27 | -DCMAKE_BUILD_TYPE=Release \ 28 | -DCMAKE_C_COMPILER=clang-4.0 \ 29 | -DCMAKE_CXX_COMPILER=clang++-4.0 \ 30 | -DCMAKE_C_FLAGS="-O3 -fsave-optimization-record" \ 31 | -DCMAKE_CXX_FLAGS="-O3 -fsave-optimization-record" \ 32 | ../kfr 33 | make -j4 34 | cd - 35 | } 36 | 37 | 38 | build_kfr 2>&1 | tee ${OUTPUT}/kfr/build.log 39 | #KFR_YAML=$(find kfr_build -name '*.opt.yaml') 40 | # FIXME: hack until we get git LFS figured out (index.html 41 | # is greater than allowed limit) 42 | KFR_YAML=$(find kfr_build -name '*.opt.yaml' | head -4) 43 | ./llvm/utils/opt-viewer/opt-viewer.py -source-dir kfr/ ${KFR_YAML} ${OUTPUT}/kfr/ 44 | -------------------------------------------------------------------------------- /img/opt_viewer_sample.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/androm3da/optviewer-demo/0d46bbb23420a756a03bea48259d1a54fefe93c0/img/opt_viewer_sample.png --------------------------------------------------------------------------------