├── README.md ├── assets ├── README.md ├── hpp-skel-badge_blue.svg ├── hpp-skel-badge_red.svg ├── hpp-skel-readme_blue.png ├── hpp-skel-readme_red.png ├── node-cpp-skel-badge_blue.svg ├── node-cpp-skel-badge_red.svg ├── node-cpp-skel-readme_blue.png ├── node-cpp-skel-readme_red.png └── skel-readme.ai ├── docs ├── coverage.md ├── faq.md ├── general.md ├── learning-resources.md ├── makefiles.md ├── packaging.md ├── strong_types.md ├── structural-metaprogramming.md ├── troubleshooting.md └── user_defined_literals.md ├── glossary.md ├── node-cpp.md └── package.json /README.md: -------------------------------------------------------------------------------- 1 | # C++ @ Mapbox 2 | 3 | The C++ language and projects built on top of it are essential to the Mapbox platform. This is the place to start for developing and deploying C++ at Mapbox. You'll find information about best-practices, gotchas, and helpful libraries/resources for beginning to develop in C++. 4 | 5 | ### Table of contents 6 | 7 | - [General C++ Setup @ Mapbox](docs/general.md) (standard compilers & libraries) 8 | - [Packaging](docs/packaging.md) (Mason, deploying code) 9 | - [Node.js addons](node-cpp.md) (writing C++ for Node.js modules) 10 | - [Code coverage](docs/coverage.md) 11 | - [Troubleshooting](docs/troubleshooting.md) (profiling, debugging) 12 | - [External learning resources](docs/learning-resources.md) (skeleton libraries, reading, videos) 13 | - [Glossary](glossary.md) 14 | - [FAQ](docs/faq.md) (and things that have no home) 15 | 16 | ### Guides 17 | 18 | - [Structural metaprogramming](docs/structural-metaprogramming.md) 19 | - [Strong types](docs/strong_types.md) 20 | 21 | ### Active C++ projects at Mapbox 22 | 23 | - Map rendering with [Mapbox GL Native](https://github.com/mapbox/mapbox-gl-native), the core codebase shared between all of the native frontend SDKs, some of which are also in C++ like [Qt](https://github.com/mapbox/mapbox-gl-native/tree/master/platform/qt/), and the [macOS SDK](https://github.com/mapbox/mapbox-gl-native/tree/master/platform/macos/) or [iOS](https://github.com/mapbox/mapbox-gl-native/blob/master/platform/ios/), which is in Objective-C++ (a hybrid language). 24 | - Mapbox directions is based on [OSRM](https://github.com/Project-OSRM/osrm-backend) 25 | - Mapbox geocoding uses [carmen-cache](https://github.com/mapbox/carmen-cache) for indexes 26 | - Mapbox tile services use [Mapnik](https://github.com/mapnik/mapnik) and [node-mapbox-gl-native](https://github.com/mapbox/mapbox-gl-native/tree/master/platform/node) 27 | - Mapbox scripts for processing OSM are often based on [libosmium](https://github.com/osmcode/libosmium). [Minjur](https://github.com/mapbox/minjur) creates geojson and [tippecanoe](https://github.com/mapbox/tippecanoe) creates vector tiles for [OSM QA tiles](https://osmlab.github.io/osm-qa-tiles/). 28 | - Mapbox Vector Tile encoding, decoding, and analysis tools: [lib-vector-tile](https://github.com/mapbox/vector-tile), [protozero](https://github.com/mapbox/protozero), [vtzero](https://github.com/mapbox/vtzero), [mapnik-vector-tile](https://github.com/mapbox/mapnik-vector-tile), [vtvalidate](https://github.com/mapbox/vtvalidate), [vtquery](https://github.com/mapbox/vtquery) 29 | -------------------------------------------------------------------------------- /assets/README.md: -------------------------------------------------------------------------------- 1 | ![](https://travis-ci.org/mapbox/node-cpp-skel.svg?branch=master) 2 | ![](node-cpp-skel-badge_blue.svg) 3 | ![](node-cpp-skel-badge_red.svg) 4 | ![](hpp-skel-badge_blue.svg) 5 | ![](hpp-skel-badge_red.svg) 6 | ![](hpp-skel-readme_blue.png) 7 | ![](hpp-skel-readme_red.png) 8 | ![](node-cpp-skel-readme_blue.png) 9 | ![](node-cpp-skel-readme_red.png) 10 | -------------------------------------------------------------------------------- /assets/hpp-skel-badge_blue.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 12 | 13 | 14 | 25 | hpp-skel 26 | hpp-skel 27 | 28 | -------------------------------------------------------------------------------- /assets/hpp-skel-badge_red.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 12 | 13 | 14 | 25 | hpp-skel 26 | hpp-skel 27 | 28 | -------------------------------------------------------------------------------- /assets/hpp-skel-readme_blue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mapbox/cpp/1bb519ef25edd6169f1d6d8a65414044616590a9/assets/hpp-skel-readme_blue.png -------------------------------------------------------------------------------- /assets/hpp-skel-readme_red.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mapbox/cpp/1bb519ef25edd6169f1d6d8a65414044616590a9/assets/hpp-skel-readme_red.png -------------------------------------------------------------------------------- /assets/node-cpp-skel-badge_blue.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 13 | 14 | 15 | 26 | node-cpp-skel 27 | node-cpp-skel 28 | 29 | -------------------------------------------------------------------------------- /assets/node-cpp-skel-badge_red.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 12 | 13 | 14 | 25 | node-cpp-skel 26 | node-cpp-skel 27 | 28 | -------------------------------------------------------------------------------- /assets/node-cpp-skel-readme_blue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mapbox/cpp/1bb519ef25edd6169f1d6d8a65414044616590a9/assets/node-cpp-skel-readme_blue.png -------------------------------------------------------------------------------- /assets/node-cpp-skel-readme_red.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mapbox/cpp/1bb519ef25edd6169f1d6d8a65414044616590a9/assets/node-cpp-skel-readme_red.png -------------------------------------------------------------------------------- /assets/skel-readme.ai: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mapbox/cpp/1bb519ef25edd6169f1d6d8a65414044616590a9/assets/skel-readme.ai -------------------------------------------------------------------------------- /docs/coverage.md: -------------------------------------------------------------------------------- 1 | # Code coverage 2 | 3 | To enable code coverage for C/C++ with clang++ you first need to compile your code in Debug mode and with specific flags to tell the clang++ compiler to instrument the binaries with coverage information. 4 | 5 | The specific flags you use will depend on the tools you wish to use to display the results. There are two options: `gcov` format and `llvm-cov profile` format. We recommend using the `gcov` format to interact with services like coveralls and codecov and using the `llvm-cov profile` format for displaying code coverage results locally. 6 | 7 | ##### Using profile coverage format 8 | 9 | 1) First set the key compiler and linker flags: 10 | 11 | ``` 12 | export CXXFLAGS="-fprofile-instr-generate -fcoverage-mapping" 13 | export LDFLAGS="-fprofile-instr-generate" 14 | ``` 15 | 16 | Set an environment variable to ask for the files generated during testing to be named by their `pid`. This allows multiple test runs to contribute to the final results. 17 | 18 | ``` 19 | export LLVM_PROFILE_FILE="code-%p.profraw" 20 | ``` 21 | 22 | 2) Build your application in Debug mode. 23 | 24 | 3) Run the tests 25 | 26 | 4) Run llvm-profdata and llvm-cov to display results 27 | 28 | The `llvm-profdata` tool is able to get all `.profraw` files (from one or more test runs) and merge them into a format suitable for `llvm-cov`. 29 | 30 | ``` 31 | llvm-profdata merge -output=code.profdata code-*.profraw 32 | ``` 33 | 34 | Then with `llvm-cov` you can either a) report of summary of coverage for all tracked files, 2) display the line-by-line coverage in your terminal, or 3) display line-by-line coverage in an html file 35 | 36 | Summary: 37 | 38 | ``` 39 | llvm-cov report /path/to/your/binary -instr-profile=code.profdata -use-color 40 | ``` 41 | 42 | Terminal output: 43 | 44 | ``` 45 | llvm-cov show /path/to/your/binary -instr-profile=code.profdata /path/to/source_files/*.cpp -filename-equivalence -use-color 46 | ``` 47 | 48 | HTML output: 49 | 50 | ``` 51 | llvm-cov show /path/to/your/binary -instr-profile=code.profdata /path/to/source_files/*.cpp -filename-equivalence -use-color --format html > /tmp/coverage.html 52 | ``` 53 | 54 | Troubleshooting: 55 | 56 | More details on the `llvm-cov show` command at http://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show. 57 | 58 | If you hit an error like `Unsupported instrumentation profile format version` or `Unsupported profiling format version` or `Failed to load coverage: Unsupported coverage format version` running `llvm-profdata` or `llvm-cov` that means you have a mismatch in the versions between the clang++ compiler and the llvm-cov or llvm-profdata tools. Make sure you are using all of these tools from mason and with consistent versions. You may need to set `CXX=/path/to/mason-provided/clang++` to solve this error. 59 | 60 | If you hit an error like `error: B??? : No such file or directory` or `warning: The file 'B??? isn't covered.` that indicates that inside the directory of source files you pointed `llvm-cov` to there is some invalid file. This can occur if a file was not compiled into your binary or if the file completely lacks coverage. To workaround this, remove any files that completely lack coverage from the arguments you pass to `llvm-cov show`. I consider this a glitch/bug of llvm-cov and have reported the issue upstream: https://llvm.org/bugs/show_bug.cgi?id=30659 61 | 62 | 63 | ##### Using gcov coverage format 64 | 65 | 1) First set the key compiler and linker flags: 66 | 67 | ``` 68 | export CXXFLAGS="--coverage" 69 | export LDFLAGS="--coverage" 70 | ``` 71 | 72 | 2) Build your application in Debug mode. 73 | 74 | 3) Run the tests 75 | 76 | 4) Post results to codecov.io 77 | 78 | Using `llvm-cov gcov` command. 79 | 80 | ``` 81 | curl -S -f https://codecov.io/bash -o codecov 82 | chmod +x codecov 83 | ./codecov -x "llvm-cov gcov" -Z 84 | ``` 85 | 86 | TODO: test, document how to use the results of `llvm-cov gcov` locally with `gcovr` 87 | 88 | Troubleshooting: 89 | 90 | More details on the `llvm-cov gcov` command at http://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-gcov. 91 | -------------------------------------------------------------------------------- /docs/faq.md: -------------------------------------------------------------------------------- 1 | ## Should I install `libstdc++6` instead of `libstdc++-5-dev`? 2 | 3 | The `tl;dr` is yes you can - there is no harm - but don't please. Please install `libstdc++-5-dev` instead - it is more explicit. 4 | 5 | The longer version is that installing the `libstdc++6` package is functionally the same as installing `libstdc++-5-dev`. The difference is that `libstdc++-5-dev`: 6 | 7 | - Brings in a few more headers (nothing so big to worry about) 8 | - If you need the ability to compile c++14 programs you need these headers 9 | - The `libstdc++-5-dev` package is wonderfully named! 10 | 11 | Wonderfully named you say? Yes, it is named logically and that is a wonderful thing. The name clearly indicates that `libstdc++` is being installed. The version `5` indicates that you are getting the version compatible with g++-5 and its respective headers. And it is the development version so the version you use if you need libstdc++ headers (the files put into {prefix}/include and the right libstdc++ library to be installed (the dynamic (aka shared) library put into `{prefix}/lib`). 12 | 13 | The `libstdc++6` package name can be confusing. Note that there is no dash between the `++` and the `6` like there is between the `++` and the `5` in the dev package. 14 | 15 | So why the `6`? It is the soname 6. Sonames are a linux way of speaking about binary library compatibility (ABI). The soname is not the version! The soname also does not really capture all you need to know about binary capability because libstdc++ also uses [versioned symbols](https://github.com/springmeyer/glibcxx-symbol-versioning). Because of versioned symbols you can live a free and exciting life always running the latest libc or libstdc++ without fear because it is forward compatible (binaries only use the versioned symbols they were compiled against). 16 | 17 | **So what version do you actually get if you install `libstdc++6` or `libstdc++-5-dev`?** 18 | 19 | It depends :) The important thing to remember is that unless you know better you want the latest version. If you really need to know then you can do: 20 | 21 | ``` 22 | apt-cache show libstdc++6 23 | ``` 24 | 25 | You will get the latest version packaged in https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+packages for ubuntu precise and trusty. At the time of this writing the version is `v6.1.1`. Again, that `6` is not related to the soname. 26 | 27 | Btw, if you ran `apt-get show libstdc++6` on ubuntu precise without the `ubuntu-toolchain-r` PPA installed then `apt-get install libstdc++6` would give libstdc++ `v4.6.3`. 28 | 29 | If you hit a runtime error like `/usr/lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.20' not found` it means you forgot to upgrade libstdc++6 to at least `v6.1.1`. 30 | 31 | One caveat: To upgrade libstdc++ for binary programs that don't need to be compiled, just installing `libstdc++6` is enough. You don't need the extra headers brought in by `libstdc++-5-dev`. But there is no harm having the headers installed and being explicit will help make clear in the future when we require the g++-6 version of libstdc++ and that we've upgraded to it by requiring `libstdc++-6-dev`. 32 | 33 | For much more detail about which libstdc++ versions are available on which ubuntu versions (and upgradable via which packages) see https://github.com/springmeyer/glibcxx-symbol-versioning 34 | 35 | ## I'm seeing a compiler error like "no template named 'foo' in namespace 'std'". What is that about? 36 | 37 | If you hit an error like: 38 | 39 | ``` 40 | deps/mapbox/variant/include/mapbox/variant.hpp:558:44: error: no template named 'is_nothrow_default_constructible' in namespace 'std'; did you mean 'has_nothrow_default_constructor'? 41 | VARIANT_INLINE variant() noexcept(std::is_nothrow_default_constructible::value) 42 | ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 43 | has_nothrow_default_constructor 44 | ``` 45 | 46 | look a few lines down and you will see: 47 | 48 | ``` 49 | /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/type_traits:737:12: note: 'has_nothrow_default_constructor' declared here 50 | struct has_nothrow_default_constructor 51 | ``` 52 | 53 | Notice that the headers from `g++` 4.6 are being used. This means that clang++ is looking for C++11 features but could not find them in the headers available from g++. Therefore that indicates that clang++ could not find headers to use from more g++ version more recent than 4.6 (Note: at least 4.8 or ideally 5 is needed for c++11 support). This is an indication that you need to upgrade your libstdc++-5-dev package. See above. 54 | -------------------------------------------------------------------------------- /docs/general.md: -------------------------------------------------------------------------------- 1 | * [Standard compiler](#standard-compiler) 2 | * [Standard C++ Library](#standard-c-library) 3 | 4 | # Standard compiler 5 | 6 | Clang++ is the primary compiler we use to build binaries (in mason) and test C++ code at Mapbox. Clang++ v7.0.0 is the latest stable version that we should be targeting. 7 | 8 | ## Installing clang++ 9 | 10 | ### Debian/Ubuntu 11 | 12 | Because clang++/llvm development moves fast it is impractical to depend on the packaged versions in Ubuntu LTS releases (Ubuntu Precise only has clang 3.4 and Ubuntu Trusty only has clang 3.5). 13 | 14 | Recommendation: install with mason (see below). 15 | 16 | ### OS X 17 | 18 | On OS X Apple ships frequent XCode updates, so staying up to date with latest clang++ is as easy as following the XCode updates in the app store. 19 | 20 | ### Mason 21 | 22 | You can install clang++ v7.0.0 using [mason](https://github.com/mapbox/mason), which works the same on both OS X and Linux: 23 | 24 | ``` 25 | CLANG_VERSION="7.0.0" 26 | mason install clang++ ${CLANG_VERSION} 27 | export PATH=$(mason prefix clang++ ${CLANG_VERSION})/bin:${PATH} 28 | which clang++ 29 | ``` 30 | 31 | On older linux versions (like `Ubuntu Trusty` and older) to compile C++ programs with this version of clang you'll need to have the libstdc++ headers available. These can be fetched by installing `libstdc++-5-dev (which are a sub-package of g++)`. 32 | 33 | ### TravisCI 34 | 35 | **OSX** 36 | 37 | On Travis OS X machines you can upgrade xcode by supplying the most recent `xcode` value for the `osx_image` option. At the time of writing the latest is installable like: 38 | 39 | ```yml 40 | matrix: 41 | include: 42 | - os: osx 43 | # https://docs.travis-ci.com/user/languages/objective-c/#Supported-OS-X-iOS-SDK-versions 44 | osx_image: xcode9.2 45 | compiler: clang 46 | ``` 47 | 48 | **Installing via Mason** 49 | 50 | Installing on travis specifically looks like: 51 | 52 | ```yml 53 | language: generic 54 | 55 | cache: 56 | directories: 57 | - $HOME/.ccache 58 | 59 | matrix: 60 | include: 61 | - os: linux 62 | sudo: false 63 | compiler: "clang-release" 64 | env: CXX=clang++ CLANG_VERSION="7.0.0" 65 | addons: 66 | apt: 67 | sources: [ 'ubuntu-toolchain-r-test' ] 68 | packages: [ 'libstdc++-5-dev' ] 69 | - os: linux 70 | sudo: false 71 | compiler: "clang-debug" 72 | env: CXX=clang++ CLANG_VERSION="7.0.0" 73 | addons: 74 | apt: 75 | sources: [ 'ubuntu-toolchain-r-test' ] 76 | packages: [ 'libstdc++-5-dev' ] 77 | 78 | install: 79 | - mkdir ./mason 80 | curl -sSfL https://github.com/mapbox/mason/archive/v0.19.0.tar.gz | tar --gunzip --extract --strip-components=1 --exclude="*md" --exclude="test*" --directory=./mason 81 | - ./mason/mason install clang++ ${CLANG_VERSION} 82 | - export PATH=$(./mason/mason prefix clang++ ${CLANG_VERSION})/bin:${PATH} 83 | - which clang++ 84 | ``` 85 | 86 | Note: see all the versions of clang++ available in Mason at https://github.com/mapbox/mason/tree/master/scripts/clang%2B%2B 87 | 88 | ### Installing other llvm tools 89 | 90 | You may also wish to install other parts of llvm like llvm-cov, clang-format, or clang-tidy. 91 | 92 | You can either install the `llvm` package that contains all the llvm tools, or for faster downloads you can install individual packages. 93 | 94 | Installing clang-format via mason can be done like: 95 | 96 | ```bash 97 | mkdir ./mason 98 | curl -sSfL https://github.com/mapbox/mason/archive/v0.18.0.tar.gz | tar --gunzip --extract --strip-components=1 --exclude="*md" --exclude="test*" --directory=./mason 99 | CLANG_VERSION="7.0.0" 100 | ./mason/mason install clang-format ${CLANG_VERSION} 101 | export PATH=$(./mason/mason prefix clang-format ${CLANG_VERSION})/bin:${PATH} 102 | which clang-format 103 | ``` 104 | 105 | # Standard C++ Library 106 | 107 | ### OS X 108 | 109 | On OS X we use libc++: http://libcxx.llvm.org 110 | 111 | - This is the default C++ standard library on OS X since 10.9. 112 | - But libstdc++ also still exists on OS X (at least as of 10.12 where it is deprecated) 113 | - It is safe to dlopen a module linked to libc++ library from executable linked to libstdc++ 114 | - It is not a good idea to link a module to both libraries 115 | 116 | Apple did some incredible work to make both libc++ and libstdc++ live harmoniously on OS X. This would lead to crashes or undefined behavior on linux, but on OS X it works fine. So, for example, you are able to use node.js binaries that link to `libstdc++` with node c++ addons that link to `libc++`. We do this commonly so that a node c++ addon can use the latest c++11 or c++14 features even though the node binary does not. The node binary provided by https://nodejs.org at the time of this writing still links to libstdc++. This is because they want the binary to support versions as old as OS X 10.7 that predate the existence of `libc++` on the system. This is achieved by passing the `-mmacosx-version-min` which automatically adds `-stdlib=libstdc++` to the linking flags. See [this line](https://github.com/nodejs/node/blob/88323e874473d18cce22d6ae134a056919c457e4/common.gypi#L355). If `-mmacosx-version-min=10.7` is not passed then the default linking would use `-stdlib=libc++`. 117 | 118 | ### Linux 119 | 120 | On Linux we use libstdc++: https://gcc.gnu.org/libstdc++ 121 | 122 | ### libstdc++ details 123 | 124 | - The apt packages are tightly coupled with g++ version 125 | - It is forward compatible and uses "versioned" symbols: this means that you can build against an older version and still safely run against a newer version. This means that it is: 126 | - Always safe to constantly upgrade to the latest version in production/runtime 127 | - If you upgrade the version used to compile/link binaries then you MUST also use at least that minimum version at runtime. Otherwise you will see runtime errors like `version GLIBCXX_3.4.20 not found`. 128 | -------------------------------------------------------------------------------- /docs/learning-resources.md: -------------------------------------------------------------------------------- 1 | # Learning Resources 2 | 3 | - Writing header-only library: https://github.com/mapbox/hpp-skel 4 | - Writing node c++ module: https://github.com/mapbox/node-cpp-skel 5 | - Writing c++ cli application: https://github.com/mapbox/cpp-cli-skel 6 | - Effective C++ by Scott Meyers: [Amazon](https://amzn.com/0321334876) 7 | - Effective Modern C++ by Scott Meyers: [Amazon](https://amzn.com/1491903996) 8 | - Basic examples of using the compiler and linker directly [learn-c](https://github.com/springmeyer/learn-c) 9 | - [Everything you wanted to know about move semantics](http://www.slideshare.net/ripplelabs/howard-hinnant-accu2014) 10 | - [Efficient Optional Values](https://akrzemi1.wordpress.com/2015/07/15/efficient-optional-values/) 11 | - [Eli Bendersky's C/C++ posts](http://eli.thegreenplace.net/tag/c-c) 12 | - [Howard Hinnants blog](https://howardhinnant.github.io) 13 | - [The overhead of abstraction in C/C++ vs. Python/Ruby](http://blog.reverberate.org/2014/10/the-overhead-of-abstraction-in-cc-vs.html) 14 | - [Competitive Programmer's Handbook](https://cses.fi/book.html) 15 | - [Great runthrough of thread behavior in Node, and how to properly use UV_THREADPOOL_SIZE with some examples](https://www.future-processing.pl/blog/on-problems-with-threads-in-node-js/) 16 | - [How compilers and linkers work](http://www.lurklurk.org/linkers/linkers.html) 17 | - [Linear C++ tutorial](https://github.com/jesyspa/linear-cpp) 18 | - [Kate Gregory “10 Core Guidelines You Need to Start Using Now” - CppCon 2017](https://www.youtube.com/watch?v=XkDEzfpdcSg) 19 | -------------------------------------------------------------------------------- /docs/makefiles.md: -------------------------------------------------------------------------------- 1 | Make tips 2 | ========= 3 | 4 | author: @ericfischer 5 | 6 | The basic idea behind Makefiles is pretty simple: each rule says what file it makes (the "target"), what other files have to exist before it can (the "prerequisites"), and what shell commands it has to run to in order to make it happen. When you run `make`, you say what file you want, and it figures out the sequence of rules that have to happen to get that result. 7 | 8 | Rules 9 | ----- 10 | 11 | A Makefile that builds a program typically has a specific rule that links that program together from a series of object files and libraries, like this: 12 | 13 | ``` 14 | program: main.o util.o other.o 15 | c++ -o $@ $^ -lm -lsqlite3 16 | ``` 17 | 18 | and a generic rule that compiles an object file from a source file: 19 | 20 | ``` 21 | %.o: %.cpp 22 | c++ -c -o $@ -O3 $< 23 | ``` 24 | 25 | In this case, since `program` wants `main.o`, `util.o`, and `other.o`, and there is a general rule to make `.o` from `.cpp` files, the two rules together will produce `program` from `main.cpp`, `util.cpp`, and `other.cpp`. 26 | 27 | You tell `make` to run the rule that produces `program` with the shell command 28 | 29 | ``` 30 | make program 31 | ``` 32 | 33 | If you run `make` without specifying what rule to run, it makes whatever rule target is the first one in the Makefile. In a Makefile that can produce many targets, it is conventional for the first target to be named `all`, and to have its prequisites be all the programs are part of the project. So this Makefile should probably begin with 34 | 35 | ``` 36 | all: program 37 | ``` 38 | 39 | to make it explicit that making `program` is what this Makefile does. 40 | 41 | Note that there is no shell command associated with this rule. It is entirely a target (`all`) and a prerequisite (`program`), and doesn't do anything on its own after ensuring that the prequisites exist. It does not actually create a file called `all`, so if you run `make` again, it will perform the rule again. 42 | 43 | Magic special characters 44 | ------------------------ 45 | 46 | There are already a lot of mysterious special characters in this tiny Makefile. You can read the list of many more of them in the [GNU Make Automatic Variables](https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html) page, but the important ones here are: 47 | 48 | * `$@` is the name of the file that is being made. So in the first rule, `$@` is `program`, and in the second, `$@` is the name of the `.o` file that is being made from whatever `.cpp` file is being compiled. 49 | * `$^` is the list of all the prerequisites for the file that is being made. So in the first rule, `$^` is `main.o util.o other.o`, since these are the three object files that go into that program. 50 | * `$<` is the *first* prequisite for the file that is being made. So in the second, rule, `$<` is the name of the single `.cpp` source file that is being compiled. 51 | * `%` is a wildcard in generic rules. If you had a rule that made PDF files from PostScript files, you would specify it as `%.pdf: %.ps` in the same way that this rule generically makes `.o` files from `.cpp` files. 52 | 53 | The reason the generic rule uses `$<` instead of `$^` is that the object file might be specified elsewhere in the Makefile to have other, indirect prerequisites, for instance some header files that the `.cpp` file #includes. These are specified as prerequisites so that the `.o` file gets rebuilt when they change, but they are included indirectly and shouldn't be given as direct command line arguments to `c++`. So the rule uses `$<` to refer to the one specific source file that is being compiled. 54 | 55 | There is also an almost invisible special character: the ` ` that precedes each shell command in a rule is a *tab* character, not a series of spaces. If you use spaces instead of a tab, your Makefile will not work. 56 | 57 | Variables 58 | --------- 59 | 60 | It is often useful to use variables in your Makefile for configuration options. For example, the rules above explicitly refer to the C++ compiler as `c++`, which leaves you without an easy way to specify that you want to use a specific version of a specific compiler instead of the operating system default. It also doesn't give you a way to specify compiler options for optimization or libraries or debugging. 61 | 62 | 63 | See also 64 | -------- 65 | 66 | https://bost.ocks.org/mike/make -------------------------------------------------------------------------------- /docs/packaging.md: -------------------------------------------------------------------------------- 1 | # Mason 2 | 3 | TODO 4 | 5 | # Deploying code 6 | 7 | #### Packaging binaries 8 | 9 | - Use [node-pre-gyp](https://github.com/mapbox/node-pre-gyp) for node C++ modules 10 | - Use [mason](https://github.com/mapbox/mason) for C++ libraries and command line (cli) applications 11 | 12 | #### Upgrade libstdc++ 13 | 14 | For binaries to run properly they will need the latest libstdc++ version installed. 15 | 16 | #### Using apt-get directly 17 | 18 | ``` 19 | sudo apt-get install -y software-properties-common python-software-properties || true 20 | sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test 21 | sudo apt-get update -y 22 | sudo apt-get install -y libstdc++-5-dev 23 | ``` 24 | 25 | ##### Update vs upgrade? 26 | 27 | Note that the above lines use `update` and `install ` not `upgrade`. 28 | 29 | The `apt-get upgrade` command will upgrade every package on your system and does not accept arguments. 30 | 31 | Usually what you want is to upgrade a single package. The way to do that is using `update + install` like: 32 | 33 | ``` 34 | apt-get update -y 35 | apt-get install -y 36 | ``` 37 | 38 | See https://linux.die.net/man/8/apt-get for more details. 39 | 40 | 41 | ##### Upgrade libstdc++ On Travis: 42 | 43 | ```yml 44 | addons: 45 | apt: 46 | sources: [ 'ubuntu-toolchain-r-test' ] 47 | packages: [ 'libstdc++-5-dev' ] 48 | ``` 49 | 50 | ##### Upgrade libstdc++ On Circleci: 51 | 52 | ```yml 53 | dependencies: 54 | pre: 55 | - sudo apt-add-repository -y ppa:ubuntu-toolchain-r/test 56 | - sudo apt-get -y update 57 | - sudo apt-get -y install libstdc++-5-dev 58 | ``` 59 | -------------------------------------------------------------------------------- /docs/strong_types.md: -------------------------------------------------------------------------------- 1 | ## Using strong types in c++ 2 | 3 | Author: https://github.com/artemp 4 | 5 | ##### Bite-sized c++ design patterns 6 | 7 | Here is a `bite-size` c++ design pattern - "Phantom types" or also known as ghost types, strong types, BOOST_STRONG_TYPEDEF etc. 8 | 9 | #### Introduction: what's it all about? 10 | 11 | The easiest way to describe this pattern is by example from basic science. Consider Newton's second law: 12 | 13 | ` 14 | F = m * a 15 | ` 16 | 17 | Where `F` is force, `m` is mass and `a` is acceleration. All these parameters can be represented by floating point types. 18 | 19 | ```c++ 20 | double m = 100.0; // mass 21 | double a = 9.8; // acceleration 22 | 23 | double calc_force(double m, double a) 24 | { 25 | return m * a; 26 | } 27 | 28 | double F = calc_force(m, a); // unbalanced force 29 | ``` 30 | 31 | All good and sound as far as relations between these types and units are preserved. This means variable `m` is representing mass, `a` is acceleration and so on. This is a simplistic example, but in larger, more complex code the problem arises when operations on unrelated types can lead to hard to spot logical bugs. Think about a nuclear plant and a careless programmer who caused a meltdown by plucking wrong types into equation. There's nothing in the code above tells compiler that `m` is mass - it's just `double`. Luckily, c++ being a strongly typed programming language, allows us to be explicit about types involved. When we write c++, compiler is our main target audience!! 32 | 33 | 34 | ```c++ 35 | mass m = 100.0; 36 | acceleration = 9.8; 37 | force calc_force(mass m, acceleration a) 38 | { 39 | return m * a; 40 | } 41 | force F = calc_force(m, a); 42 | ``` 43 | 44 | Note that there's no need for comments anymore:). But more importantly we're communicating directly and explicitly to the compiler that to get `force` we need to multiply `mass` and `acceleration` and not anything else. Following code, for example, would result in compile error: 45 | 46 | ```c++ 47 | mass m = 100.0; 48 | mass a = 10.0; 49 | force = F = calc_force(m, a); // Compile ERROR! 50 | ``` 51 | 52 | Jumping into our domain: 53 | 54 | ```c++ 55 | 56 | struct vector_tile 57 | { 58 | // some internal state 59 | // ... 60 | // 61 | bool processed; 62 | }; 63 | 64 | vector_tile create_vector_tile(...); 65 | 66 | void do_something_with_processed_vector_tile(vector_tile const& tile); 67 | // this method expects `processed' vector tile but there's nothing stopping someone making a mistake 68 | ``` 69 | 70 | Ok, we can write validation code inside methods body : 71 | 72 | ```c++ 73 | void do_something_with_processed_vector_tile(vector_tile const& tile) 74 | { 75 | if (!tile.processed) return; 76 | // Do some stuff with the tile 77 | } 78 | ``` 79 | 80 | But `c++` provides much better way to enforce correct symantics at COMPILE time!!! 81 | 82 | ```c++ 83 | struct vector_tile {}; 84 | struct processed_vector_tile {}; 85 | 86 | processed_vector_tile do_some_processing(vector_tile const& tile); 87 | void do_something_with_processed_vector_tile(processed_vector_tile const& tile); 88 | 89 | ``` 90 | 91 | Note how methods signatures changed ^. It's now a compile error to pass anything but `processed_vector_tile` to `do_something_with_processed_vector_tile`. 92 | 93 | #### Now to the main point 94 | 95 | Everyone should be convinced by now of the usefulness of this pattern, hopefully. 96 | 97 | Strong types in c++ are useful for enforcing correct units, validation, deterministic calculations. 98 | 99 | Let's discuss possible implementations. Let's try making using this pattern as easy as possible. Should we use existing `BOOST_STRONG_TYPEDEF` ? 100 | Or should we implement our own simple `header` only library. This is the main topic of this workshop. 101 | 102 | 103 | #### Using BOOST_STRONG_TYPEDEF 104 | 105 | ```c++ 106 | #include 107 | #include 108 | 109 | BOOST_STRONG_TYPEDEF(int, apples); 110 | BOOST_STRONG_TYPEDEF(int, oranges); 111 | 112 | void print_apples( apples const& v) 113 | { 114 | std::cerr << v << std::endl; 115 | } 116 | 117 | int main() 118 | { 119 | 120 | apples v0 {123}; 121 | oranges v1 {456}; 122 | 123 | print_apples(v0); // OK 124 | //print_apples(v1); // Compile ERROR 125 | return 0; 126 | } 127 | ``` 128 | 129 | 130 | #### Custom implementation 131 | 132 | ```c++ 133 | #ifndef PHANTOM_TYPE_HPP 134 | #define PHANTOM_TYPE_HPP 135 | 136 | #include 137 | 138 | 139 | namespace util { 140 | 141 | template 142 | struct phantom_type 143 | { 144 | explicit constexpr phantom_type(BaseType const& value) : value(value) {} 145 | explicit constexpr phantom_type(BaseType&& value) : value(std::move(value)) {} 146 | operator BaseType& () noexcept { return value; } 147 | constexpr operator BaseType const& () const noexcept { return value; } 148 | BaseType value; 149 | }; 150 | } // namespace 151 | 152 | #define PHANTOM_TYPE(BASE_TYPE, TYPE_NAME) \ 153 | struct TYPE_NAME : util::phantom_type \ 154 | { \ 155 | using phantom_type::phantom_type; \ 156 | }; 157 | 158 | #endif // PHANTOM_TYPE_HPP 159 | 160 | ``` 161 | 162 | ##### Using it 163 | 164 | ```c++ 165 | #include "phantom_type.hpp" 166 | 167 | #include 168 | 169 | PHANTOM_TYPE(int, apples); 170 | PHANTOM_TYPE(int, oranges); 171 | 172 | void print_apples( apples const& v) 173 | { 174 | std::cerr << v << std::endl; 175 | } 176 | 177 | int main() 178 | { 179 | 180 | apples v0{123}; 181 | oranges v1{456}; 182 | 183 | print_apples(v0); // OK 184 | //print_apples(v1); // Compile ERROR 185 | return 0; 186 | } 187 | ``` 188 | 189 | #### Performance considerations 190 | 191 | -------------------------------------------------------------------------------- /docs/structural-metaprogramming.md: -------------------------------------------------------------------------------- 1 | # C++ Structural Metaprogramming 2 | 3 | Have you ever wished you could: 4 | 5 | * Iterate over the member variables of a class or structure? 6 | * Apply a transform to the type of each member, to produce a new list of members? 7 | * Define a family of classes, each with a different set of members, but behaving in a common way? 8 | 9 | If so, you probably ran into the fact that compile-time metaprogramming over member variables has historically been a weak point of C++. I thought so too. But recently I've discovered a few techniques for dealing with these kind of situations that work pretty well in practice, and I wanted to share them. 10 | 11 | ### Super `std::tuple` 12 | 13 | The basis for this technique is `std::tuple`, and the essential idea is that instead of using a `struct` or `class` type with regular members, we use a type that derives from `std::tuple`, where what would normally be member variables are instead `std::tuple` template parameters. 14 | 15 | To make this concrete, let's take [an example from Mapbox GL](https://www.mapbox.com/mapbox-gl-style-spec/#layers-circle). Suppose we have a circle layer type with properties such as `circle-color`, `circle-opacity`, and `circle-radius`. The value of each property is a function returning a type appropriate to that property, evaluated at a particular zoom level. We'd also like to define a function that evaluates _all_ the properties of a layer at a particular zoom level. 16 | 17 | Traditionally, we would probably reach for a solution like the following: 18 | 19 | ```C++ 20 | struct EvaluatedCircleProperties { 21 | Color color; 22 | float opacity; 23 | float radius; 24 | }; 25 | 26 | struct UnevaluatedCircleProperties { 27 | std::function color; 28 | std::function opacity; 29 | std::function radius; 30 | 31 | EvaluatedCircleProperties evaluate(float zoom) const { 32 | return { 33 | color(zoom), 34 | opacity(zoom), 35 | radius(zoom) 36 | }; 37 | } 38 | }; 39 | ``` 40 | 41 | This works well enough for a simple three-property example, but it has a few disadvantages: 42 | 43 | * We repeat the property types in three places. If we were to add a property, we'd have to update them all. 44 | * When you have multiple layer types, you also have to repeat the "superstructure" -- the knowledge that each layer has `Evaluated` and `Unevaluated` structs with corresponding members, and that in each case `evaluate` is defined by evaluating member-wise and aggregate-constructing the result. 45 | * These costs increase as the number of properties and layer types grow. 46 | 47 | Here's an alternative: 48 | 49 | ```C++ 50 | template 51 | struct Properties { 52 | struct Evaluated : std::tuple {}; 53 | struct Unevaluated : std::tuple...> { 54 | Evaluated evaluate(float zoom) const; 55 | }; 56 | }; 57 | 58 | struct ColorProperties : Properties< 59 | Color, 60 | float, 61 | float> {}; 62 | ``` 63 | 64 | Instead of using a member variable for each property, we use a template parameter. Now adding a new property requires a code change in only one place. We've also defined the structure and relationship between `Evaluated` and `Unevaluated` sets of properties in a single place; no need to repeat this for other layer types. 65 | 66 | But I've left out some details. First, how do we access an individual value in the `Evaluated` and `Unevaluated` structs -- radius, say? With member variables, we can use member variable syntax: `properties.radius`. But tuple members are unnamed; the only way to access them is with [`std::get`](http://en.cppreference.com/w/cpp/utility/tuple/get). And because multiple properties may have the same type, we can't even use the type-based `std::get`; `std::get(properties)` could mean either opacity or radius. We're stuck with `std::get<2>(properties)`, which has terrible readability and would break if we reordered the properties or inserted a new one before radius. 67 | 68 | We can solve this problem if we ensure that each template parameter has a unique type: 69 | 70 | ```C++ 71 | template 72 | struct Property { 73 | using EvaluatedType = T; 74 | using UnevaluatedType = std::function; 75 | }; 76 | 77 | template 78 | struct Properties { 79 | struct Evaluated : std::tuple { 80 | template auto get(); 81 | }; 82 | 83 | struct Unevaluated : std::tuple { 84 | template auto get(); 85 | Evaluated evaluate(float zoom); 86 | }; 87 | }; 88 | 89 | struct CircleColor : Property {}; 90 | struct CircleOpacity : Property {}; 91 | struct CircleRadius : Property {}; 92 | 93 | struct ColorProperties : Properties< 94 | CircleColor, 95 | CircleOpacity, 96 | CircleRadius> {}; 97 | ``` 98 | 99 | We've added a `get` member function to `Evaluated` and `Unevaluated`, which provides a much nicer syntax: `properties.get()`. The cost is that we now need to define a new type for each property, touching two places rather than one. But I think it's worth it -- and in fact this case having a distinct type is convenient, as it provides a place to hang other per-property data such as the default value. 100 | 101 | ### More `tuple` tricks 102 | 103 | I haven't shown the implementation of `get` or `evaluate` yet. They involve a few more tricks. First, `get`: 104 | 105 | ```C++ 106 | template 107 | struct TypeIndex; 108 | 109 | template 110 | struct TypeIndex : std::integral_constant {}; 111 | 112 | template 113 | struct TypeIndex : std::integral_constant::value> {}; 114 | 115 | template 116 | struct Properties { 117 | template 118 | static constexpr std::size_t Index = TypeIndex::value; 119 | 120 | struct Evaluated : std::tuple { 121 | template auto get() { 122 | return std::get>(*this); 123 | } 124 | }; 125 | 126 | ... 127 | }; 128 | ``` 129 | 130 | Why this implementation? Well, we want to allow `properties.get()` where `properties` is either `Evaluated` or `Unevaluated`. However, `CircleRadius` is not itself one of the `Evaluated` or `Unevaluated` tuple template parameters. What `get` needs to do is obtain the _index_ of `CircleRadius` in the `Ps` template parameter pack, and then use that index as the template parameter for `std::get`. That's what `TypeIndex` and `Properties::Index` accomplish. (`Properties::Index` uses a [variable template](http://en.cppreference.com/w/cpp/language/variable_template), a C++14 feature, but you could replace its uses with `TypeIndex::value` for a pure C++11 solution.) 131 | 132 | Now, what about `evaluate`? Here's its definition: 133 | 134 | ```C++ 135 | template 136 | struct Properties { 137 | ... 138 | 139 | struct Unevaluated : std::tuple { 140 | template auto get() { ... } 141 | 142 | Evaluated evaluate(float zoom) { 143 | return { get()(zoom)... }; 144 | } 145 | }; 146 | 147 | ... 148 | }; 149 | ``` 150 | 151 | 🤗 Our definition of `get` turns out to be very useful, as we can now use template parameter pack expansion over `Ps` to automatically evaluate all the properties, no matter how many of them there are. 152 | 153 | ### How to ignore all the things 154 | 155 | One last trick. We were able to take advantage of parameter pack expansion in the above definition because we conveniently had a context -- aggregate construction -- in which expansion with `...` is allowed. What if we wanted to define a member-wise operation where we wouldn't normally have such a context? 156 | 157 | For example, suppose we need to check if any of the unevaluated `std::function`s are [empty](http://en.cppreference.com/w/cpp/utility/functional/function/operator_bool). We need to calculate `bool(color) || bool(opacity) || bool(radius) || ...`, but defined generically over the parameter pack `Ps`. In C++17 we'll be getting [fold expressions](http://en.cppreference.com/w/cpp/language/fold) for this sort of thing, but we can simulate them in C++ today: 158 | 159 | ```C++ 160 | template void ignore(Ts&&...) {} 161 | 162 | template 163 | bool any_empty(const Properties::Unevaluated& properties) { 164 | bool result = false; 165 | ignore((result |= bool(properties.get()))...); 166 | return result; 167 | } 168 | ``` 169 | 170 | In order to give ourselves a context in which the `|=` expression can be expanded, we define the `ignore` template function, which takes any number of arguments of any types and does nothing with them. We expand the parameter pack into arguments to `ignore`, relying on the expression's side effect to compute the desired `result`. 171 | 172 | Using `ignore`, you can expand pretty much any expression. For expressions that would normally be typed `void`, which isn't allowed as a function parameter type, use a comma expression instead: e.g. `ignore((fnWithVoidResultType(get()), 0)...)`. 173 | 174 | In summary: `std::tuple` and C++ template parameter packs are super powerful, and with this one weird trick (ok, several weird tricks), very useful for structural metaprogramming. -------------------------------------------------------------------------------- /docs/troubleshooting.md: -------------------------------------------------------------------------------- 1 | # Debugging crashes 2 | 3 | The first step is to view the [backtrace](glossary.md#backtrace) for the crash. 4 | 5 | ##### OS X 6 | 7 | On OS X [backtraces](glossary.md#backtrace) are automatically generated after a crash and dropped into `~/Library/Logs/DiagnosticReports/`. OS X can also be configured to display a notification on any crash in native code with: 8 | 9 | ```bash 10 | defaults write com.apple.CrashReporter UseUNC 1 11 | ``` 12 | 13 | ##### Linux 14 | 15 | On linux backtraces [backtraces](glossary.md#backtrace) are not automatically generated. 16 | 17 | To enable them, use [logbt](https://github.com/mapbox/logbt/). This will ensure that backtraces are displayed for any crashes right when the program exits. You can use `logbt` on both Linux and OS X by launching your program with `logbt`: 18 | 19 | ```bash 20 | logbt -- node index.js 21 | ``` 22 | 23 | If your backtraces don't have enough detail, then try running [debug binaries](glossary.md#debug-build) in production. 24 | 25 | If backtraces don't provide enough detail to clearly identify the problem consider running [sanitized binaries](glossary.md#sanitized-build). 26 | 27 | #### Sanitizers 28 | 29 | Sanitizers work at runtime to watch your code for common mistakes and warn or throw when those mistakes are encountered. As opposed to static analysis (which operates at compile time) they are not designed catch errors that never happen. So test coverage is essential. Or running your code in real-world scenarios. The latter case is where the sanitizers really shine compared to other debugging tools because they can work with `-O1` optimization and don't slow down the runtime very much. Other tools, like valgrind, may slow down your program execution so much that actually reaching the bug in a long running process might take so long as to be untenable. 30 | 31 | The Address sanitizer is one of the most useful because it can catch memory leaks (only on Linux however) and invalid use of memory that might cause crashes or undefined behavior. 32 | 33 | To leverage `-fsanitize=address` on OS X you'll need a custom built clang because the apple/xcode provided command line clang++ does not support `-fsanitize=address` at the time of this writing (though it does work inside XCode). Also on OS X, if using `-fsanitize=address` outside of XCode, it is important to define `-D_LIBCPP_HAS_NO_ASAN` in your CXXFLAGS otherwise you might receive false positives for container overflows (see http://stackoverflow.com/a/38858905 for an explanation). 34 | 35 | To leverage `-fsanitize=address` on Linux you'll want a very recent clang to ensure you a getting the best sanitizer results. 36 | 37 | So, for both platforms it is recommended to [install clang++ via mason packages](#mason). 38 | 39 | After installing your custom clang, for `make`-based build systems, do: 40 | 41 | ``` 42 | export CXXFLAGS="-fsanitize=address -D_LIBCPP_HAS_NO_ASAN" 43 | export LDFLAGS="-fsanitize=address" 44 | ``` 45 | 46 | Then inspect the build output to ensure that the `-fsanitize=address` flag is showing up. You may need to pass `V=1` or `VERBOSE=1` to make systems to get the build output to show up. 47 | 48 | For `cmake`-based systems it may be harder to pass these flags, so consult with a developer on the project that can help you. 49 | 50 | # Errors 51 | 52 | A listing of errors and solutions. 53 | 54 | ### Linking error of 'file not recognized: File format not recognized' 55 | 56 | Error: 57 | 58 | ``` 59 | Debug/obj.target/module/src/module.o: file not recognized: File format not recognized 60 | clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation) 61 | ``` 62 | 63 | #### Solution 64 | 65 | Install binutils from mason and put its `bin` directory on your `PATH` such that the `ld` command is found. 66 | 67 | #### Context 68 | 69 | You are building C++ code with [Link Time Optimization, aka LTO](https://github.com/mapbox/cpp/blob/master/glossary.md#link-time-optimization) on Linux. Your build compiles okay but fails a linking with this obscure error about `File format not recognized`. This is because the default [linker](https://github.com/mapbox/cpp/blob/master/glossary.md#linker) on linux does not support LTO object files. But the linker from the latest binutils does, so upgrading the linker fixes the problem. 70 | 71 | ### This version of node/NAN/v8 requires a C++11 compiler 72 | 73 | Error: 74 | 75 | ``` 76 | ../node_modules/nan/nan.h:45:3: error: #error This version of node/NAN/v8 requires a C++11 compiler 77 | ``` 78 | 79 | #### Solution 80 | 81 | Upgrade your compiler using this method: https://github.com/mapbox/cpp#standard-compiler 82 | 83 | #### Context: 84 | 85 | You are building a node c++ module. That error above was happening because node itself (node v4 and greater) needs a compiler that supports C++11. You are likely building on a linux system with an old default compiler like g++ 4.8 or older. 86 | 87 | ### clang: warning: libstdc++ is deprecated 88 | 89 | On OSX you hit an error like: 90 | 91 | ``` 92 | clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 93 | ``` 94 | 95 | #### Solution: 96 | 97 | This is just a warning - it is harmless. However to fix the warning: 98 | 99 | - If you are the developer of the module: add [these lines](https://github.com/mapbox/node-cpp-skel/blob/bf5fa91961c4027e709b557f5d3acd7eef8a7894/binding.gyp#L20-L21) to your `binding.gyp` 100 | - If you are the user of the module, file a bug report upstream. 101 | 102 | #### Context 103 | 104 | You are compiling a node c++ module on OS X. This warning is expected if you are running OS X >= 10.12 and: 105 | 106 | - You are building a node c++ addon against the nodejs provided binary 107 | - That c++ addon is not explicitly requesting linking to libc++ or; 108 | - Overriding the `-mmacosx-version-min` flag. 109 | 110 | This warning is harmless until Apple decides to remove `libstdc++` from an OS X version. For modules that don't explicitly link to libc++ (like recommended in the solution above) this warning will persist until either: 111 | 112 | - the nodejs project starts linking to libc++ explicitly (refs https://github.com/nodejs/node/commit/7292a1e954e0db348ff78c704df9e105ba5667ad) 113 | - or the nodejs project changes [this line](https://github.com/nodejs/node/blob/88323e874473d18cce22d6ae134a056919c457e4/common.gypi#L355) to `10.8` 114 | -------------------------------------------------------------------------------- /docs/user_defined_literals.md: -------------------------------------------------------------------------------- 1 | # UDL (user defined literals in c++) 2 | 3 | ## Background 4 | 5 | ### What is a literal ? 6 | 7 | You should know the answer already, right ? ;) 8 | 9 | via Wikipedia: 10 | 11 | "In computer science, a literal is a notation for representing a fixed value in source code. Almost all programming languages have notations for atomic values such as integers, floating-point numbers, and strings, and usually for booleans and characters; some also have notations for elements of enumerated types and compound values such as arrays, records, and objects. An anonymous function is a literal for the function type." 12 | 13 | 14 | ### Integer literals in c++ 15 | Allows values of integer type to be used in expressions directly. 16 | 17 | ```c++ 18 | int d = 42; 19 | int o = 052; 20 | int x = 0x2a; 21 | int X = 0X2A; 22 | int b = 0b101010; // c++14 23 | ``` 24 | 25 | 26 | ### string literals 27 | 28 | * " (unescaped_character|escaped_character)* " (1) 29 | * L " (unescaped_character|escaped_character)* " (2) 30 | * u8 " (unescaped_character|escaped_character)* " (3) (since C++11) 31 | * u " (unescaped_character|escaped_character)* " (4) (since C++11) 32 | * U " (unescaped_character|escaped_character)* " (5) (since C++11) 33 | * prefix(optional) R "delimiter( raw_characters )delimiter" (6) (since C++11) 34 | 35 | ### Floating point literals 36 | 37 | ``` 38 | 1.23e-12 39 | ``` 40 | More in your favourite c++ reference... 41 | 42 | ### User defined literals UDL `*NEW from c++11!*` 43 | 44 | UDLs have been part of the standard from c++11: 45 | http://en.cppreference.com/w/cpp/language/user_literal 46 | 47 | "Allows integer, floating-point, character, and string literals to produce objects of user-defined type by defining a user-defined suffix." 48 | 49 | 50 | ## Contrived example 51 | 52 | Let's say we'd like to craft a data structure for representing geographical points. Something like this: 53 | 54 | ```c++ 55 | template 56 | struct geographic_coordinate 57 | { 58 | using type = T; 59 | type value; 60 | explicit constexpr geographic_coordinate(T value_) 61 | : value(value_) 62 | {} 63 | // operator T() const { return value;} 64 | // geographic_coordinate& operator=(geographic_coordinate && other) = default; 65 | }; 66 | 67 | ``` 68 | 69 | We also would like to clearly differentiate between longitude and latitude. Both are geographical coordinates and both share the same base class. 70 | 71 | ```c++ 72 | 73 | template 74 | struct longitude : geographic_coordinate 75 | { 76 | using geographic_coordinate::geographic_coordinate; 77 | }; 78 | 79 | template 80 | struct latitude : geographic_coordinate 81 | { 82 | using geographic_coordinate::geographic_coordinate; 83 | }; 84 | 85 | ``` 86 | 87 | To reduce verbosity we add a couple of type definitions aliases. Also we add streaming operator to be able to check underlying values: 88 | 89 | ```c++ 90 | using LON = longitude; 91 | using LAT = latitude; 92 | 93 | template 94 | std::ostream & operator<<(std::ostream & out, geographic_coordinate const& p) 95 | { 96 | out << p.value; 97 | return out; 98 | } 99 | 100 | ``` 101 | 102 | With this framework in hand we can now go ahead and implement `geographic_point` structure which can be constructed from longitude and latitude. We'd like to be able to write `geographic_point(lan, lon)` or `geographic_point(lon, lat)`, both resulting in the same object. To accomplish this we define two specialised constructors: 103 | 104 | ```c++ 105 | 106 | template 107 | struct geographic_point 108 | { 109 | constexpr geographic_point(longitude lon, latitude lat) 110 | : lon_(lon.value), 111 | lat_(lat.value) {} 112 | 113 | constexpr geographic_point(latitude lat, longitude lon) 114 | : lon_(lon.value), 115 | lat_(lat.value) {} 116 | 117 | T lon_; 118 | T lat_; 119 | }; 120 | 121 | ``` 122 | 123 | Almost, done! We will also need an output streaming operator and `equal/not-equal` operators 124 | 125 | ```c++ 126 | template 127 | constexpr bool operator==(geographic_point const& p0, geographic_point const& p1) 128 | { 129 | return (p0.lon_ == p1.lon_) && (p0.lat_ == p1.lat_); 130 | } 131 | 132 | template 133 | constexpr bool operator!=(geographic_point const& p0, geographic_point const& p1) 134 | { 135 | //return !(p0.lon_ == p1.lon_) && (p0.lat_ == p1.lat_); 136 | //return (p0.lon_ != p1.lon_) || (p0.lat_ != p1.lat_); 137 | return !(p0 == p1); 138 | } 139 | 140 | template 141 | std::ostream & operator<<(std::ostream & out, geographic_point const& p) 142 | { 143 | out << "GeoPoint(" << p.lon_ << " lon," << p.lat_ << " lat)"; 144 | return out; 145 | } 146 | 147 | ``` 148 | 149 | We can now create our points like this: 150 | 151 | ```c++ 152 | constexpr geographic_point p(LON(-1.2), LAT(51)); 153 | ``` 154 | 155 | or 156 | 157 | ```c++ 158 | constexpr geographic_point p(LAT(51), LON(-1.2)); 159 | ``` 160 | 161 | Following attempt to construct a point will result in compile error as expected: 162 | 163 | ```c++ 164 | //constexpr geographic_point p(52.0, -2.0); // compile error 165 | ``` 166 | 167 | Ok, but what about UDLs :) ? Let's say we'd like to have a concise way to define geographical coordinates at compile time. Something like `-1.2_lon` `51_lat` for example. 168 | 169 | To achieve this we define: 170 | 171 | ```c++ 172 | 173 | constexpr longitude operator ""_lon(long double v) 174 | { 175 | return longitude{static_cast(v)}; 176 | } 177 | 178 | constexpr latitude operator ""_lat(long double v) 179 | { 180 | return latitude{static_cast(v)}; 181 | } 182 | 183 | ``` 184 | 185 | Note, that we use `long double` for the argument type. This is because a numeric argument type must have the largest possible representation. In case of a floating point type it's `long double`. So far so good, but there are more things to do. Firstly, we also need to add overloads for integer arguments. Without them the compiler won't accept `51_lat`. 186 | 187 | 188 | ```c++ 189 | constexpr longitude operator ""_lon(unsigned long long v) 190 | { 191 | return longitude{static_cast(v)}; 192 | } 193 | 194 | constexpr latitude operator ""_lat(unsigned long long v) 195 | { 196 | return latitude{static_cast(v)}; 197 | } 198 | 199 | ``` 200 | 201 | Secondly, `-` minus sign is not a part of a literal and to be able to construct negative longitude and latitide we must provide unary `operator-()`. Here are our updated definitions: 202 | 203 | 204 | ```c++ 205 | 206 | template 207 | struct longitude : geographic_coordinate 208 | { 209 | using geographic_coordinate::geographic_coordinate; 210 | constexpr longitude operator-() const 211 | { 212 | return longitude{-this->value}; 213 | } 214 | }; 215 | 216 | template 217 | struct latitude : geographic_coordinate 218 | { 219 | using geographic_coordinate::geographic_coordinate; 220 | constexpr latitude operator-() const 221 | { 222 | return latitude{-this->value}; 223 | } 224 | }; 225 | 226 | ``` 227 | 228 | 229 | Now we have everything in place to start playing with our mini framework for geographical points: 230 | 231 | ```c++ 232 | 233 | 234 | int main() 235 | { 236 | constexpr auto lon = -2_lon; 237 | constexpr auto lat = 51.8_lat; 238 | 239 | { 240 | constexpr geographic_point p(lon, lat); 241 | std::cerr << p << std::endl; 242 | } 243 | { 244 | constexpr auto p = geographic_point(lat, lon); 245 | std::cerr << p << std::endl; 246 | } 247 | { 248 | //constexpr geographic_point p(52.0, -2.0); // compile error 249 | //constexpr geographic_point p(52_lon, -2_lon); // compile error 250 | //constexpr geographic_point p(52_lat, -2_lat); // compile error 251 | //constexpr geographic_point p(52_lat, -2); // compile error 252 | //constexpr geographic_point p(52, -2_lon); // compile error 253 | 254 | constexpr geographic_point p0(LON(30.0), -2.0_lat); // OK 255 | constexpr geographic_point p1(LON(30.0), LAT(-2.0)); // OK 256 | constexpr geographic_point p2(LAT(-2.0), LON(30.0)); // OK 257 | constexpr geographic_point p3(-2.0_lat, 30.4_lon); // OK 258 | std::cerr << p0 << std::endl; 259 | std::cerr << p1 << std::endl; 260 | std::cerr << p2 << std::endl; 261 | std::cerr << p3 << std::endl; 262 | 263 | static_assert(p0 == p1, "FAIL"); 264 | static_assert(p0 == p2, "FAIL"); 265 | static_assert(p0 != p3, "FAIL"); 266 | } 267 | } 268 | ``` 269 | 270 | 271 | ## Motivating example 272 | 273 | The example above, as its title suggests, is rather contrived. Here is a more practical, IMHO, use-case for UDLs. In Mapnik, in order to deal with dispatching on string tags (retuned by parser) we use following recursive function to calculate `hash` value at compile time: https://github.com/mapnik/mapnik/blob/master/include/mapnik/util/name_to_int.hpp 274 | 275 | This technique came about after having to deal with endless `if/else if/else` spaghetti statements in various input parsers. 276 | 277 | ```c++ 278 | 279 | if (name == "stroke") 280 | { 281 | // 282 | } 283 | else if (name == "stroke-width") 284 | { 285 | // 286 | } 287 | else if (name == "fill") 288 | { 289 | /// 290 | } 291 | // add many more!!! 292 | 293 | ``` 294 | 295 | The code above is not efficient and is not-concise either. Specially for the large number of cases. We can do better by introducing compile time `name_to_int` function and using `switch` statement instead : 296 | 297 | ```c++ 298 | 299 | constexpr unsigned name_to_int(char const* str, unsigned off = 0) 300 | { 301 | return !str[off] ? 5381 : (name_to_int(str, off + 1) * 33) ^ static_cast(str[off]); 302 | } 303 | 304 | 305 | ..... 306 | 307 | 308 | auto val = name_to_int(name); 309 | switch(val) 310 | { 311 | case name_to_int("stroke"): 312 | break; 313 | case name_to_int("stroke-width"): 314 | break; 315 | case name_to_int("fill"): 316 | break 317 | ... 318 | ... 319 | default: 320 | break; 321 | } 322 | 323 | ``` 324 | 325 | The `name_to_int` function feels like a great candidate for UDL. It might make code even more expressive. Let's give it a go: 326 | 327 | ```c++ 328 | 329 | constexpr unsigned operator"" _hash(char const* str, std::size_t) 330 | { 331 | return name_to_int(str); 332 | } 333 | 334 | ``` 335 | 336 | 337 | And now we can write: 338 | 339 | ```c++ 340 | 341 | 342 | auto val = name_to_int(name); 343 | switch(val) 344 | { 345 | case "stroke"_hash: 346 | break; 347 | case "stroke-width"_hash: 348 | break; 349 | case "fill"_hash: 350 | break 351 | ... 352 | ... 353 | default: 354 | break; 355 | } 356 | 357 | ``` 358 | 359 | I hope you're convinced by now that UDLs can be a useful technique when coding in c++! 360 | -------------------------------------------------------------------------------- /glossary.md: -------------------------------------------------------------------------------- 1 | # C++ Glossary 2 | 3 | This is an opinionated glossary of terms developed by the Mapbox team. 4 | 5 | The assumptions are: 6 | 7 | **_performance matters_** 8 | 9 | You are choosing to use C++ to write the most efficient code possible 10 | 11 | **_robustness matters_** 12 | 13 | You are choosing to use C++ to unlock the benefits of a statically typed language 14 | 15 | **_multiple platforms matter_** 16 | 17 | Your code will need to run across multiple linux or os x versions 18 | 19 | **_binary distribution matters_** 20 | 21 | You seek to distribute your code to users and clients via easily installable binaries (without them needing to know how to compile it) 22 | 23 | * * * 24 | 25 | The glossary covers these primary areas: 26 | 27 | - packaging 28 | - build systems 29 | - compilers and linkers 30 | - memory allocation 31 | - library design 32 | - binary distribution 33 | - profiling 34 | - debugging 35 | 36 | It is not intended to be a complete reference on language or syntax terms. Learn more about language terms at: 37 | 38 | - A glossary by the creator of C++: 39 | - A keyword glossary: 40 | 41 | Contributions are welcome. To contribute, please: 42 | 43 | - Add a new section with `##` 44 | - Add a new term with `###` 45 | - If you add a new term or section, rebuild the table of contents by running `npm install && npm run toc` 46 | - If you add new terms and want review please create a PR and /cc @springmeyer 47 | 48 | ## Table of Contents 49 | 50 | - [Core concepts](#core-concepts) 51 | 52 | - [posix](#posix) 53 | - [API](#api) 54 | - [ABI](#abi) 55 | - [signal](#signal) 56 | - [I/O](#io) 57 | - [crash](#crash) 58 | - [libstdc++](#libstdc) 59 | - [libc++](#libc) 60 | - [source code](#source-code) 61 | 62 | - [Packaging](#packaging) 63 | 64 | - [mason](#mason) 65 | - [node-pre-gyp](#node-pre-gyp) 66 | 67 | - [Environment variables](#environment-variables) 68 | 69 | - [CPP](#cpp) 70 | - [CC](#cc) 71 | - [CXX](#cxx) 72 | - [LD](#ld) 73 | - [CFLAGS](#cflags) 74 | - [CXXFLAGS](#cxxflags) 75 | - [LDFLAGS](#ldflags) 76 | 77 | - [Builds](#builds) 78 | 79 | - [make](#make) 80 | - [cmake](#cmake) 81 | - [gyp](#gyp) 82 | - [bazel](#bazel) 83 | - [visual studio](#visual-studio) 84 | - [ninja](#ninja) 85 | - [out of source build](#out-of-source-build) 86 | - [in-tree build](#in-tree-build) 87 | 88 | - [Compilers and linkers](#compilers-and-linkers) 89 | 90 | - [development toolchain](#development-toolchain) 91 | - [compiler](#compiler) 92 | - [Front-end](#front-end) 93 | - [linker](#linker) 94 | - [dynamic linker](#dynamic-linker) 95 | - [linked](#linked) 96 | - [linking](#linking) 97 | - [linking order](#linking-order) 98 | - [link time optimization](#link-time-optimization) 99 | - [translation unit](#translation-unit) 100 | - [object file](#object-file) 101 | - [symbol](#symbol) 102 | - [executable](#executable) 103 | - [loadable module](#loadable-module) 104 | - [library](#library) 105 | - [precompiled library](#precompiled-library) 106 | - [precompiled](#precompiled) 107 | - [shared library](#shared-library) 108 | - [static library](#static-library) 109 | - [statically linked](#statically-linked) 110 | - [dynamically linked](#dynamically-linked) 111 | - [dynamically loaded](#dynamically-loaded) 112 | - [dlopen](#dlopen) 113 | - [header-only library](#header-only-library) 114 | - [zero cost design](#zero-cost-design) 115 | - [LLVM](#llvm) 116 | - [gcc](#gcc) 117 | - [clang](#clang) 118 | - [g++](#g) 119 | - [clang++](#clang-1) 120 | - [calling application](#calling-application) 121 | - [header](#header) 122 | - [include](#include) 123 | 124 | - [Performance](#performance) 125 | 126 | - [performance](#performance-1) 127 | - [efficiency](#efficiency) 128 | - [expressive](#expressive) 129 | - [optimization technique](#optimization-technique) 130 | - [memoization](#memoization) 131 | - [small size optimization](#small-size-optimization) 132 | - [performant](#performant) 133 | - [responsive](#responsive) 134 | - [latency](#latency) 135 | - [response time](#response-time) 136 | - [scalable](#scalable) 137 | - [scalability](#scalability) 138 | - [compiler optimization level](#compiler-optimization-level) 139 | - [concurrency](#concurrency) 140 | 141 | - [Build modes](#build-modes) 142 | 143 | - [DNDEBUG](#dndebug) 144 | - [DDEBUG](#ddebug) 145 | - [release mode](#release-mode) 146 | - [release build](#release-build) 147 | - [debug build](#debug-build) 148 | - [debuggable release build](#debuggable-release-build) 149 | - [sanitized build](#sanitized-build) 150 | - [sanitizers](#sanitizers) 151 | - [profiling build](#profiling-build) 152 | - [problem of debugging release-crashes](#problem-of-debugging-release-crashes) 153 | - [problem of profiling and compiler optimization levels](#problem-of-profiling-and-compiler-optimization-levels) 154 | 155 | - [Debugging and Profiling](#debugging-and-profiling) 156 | 157 | - [Profiling](#profiling) 158 | - [debug symbols](#debug-symbols) 159 | - [Debugger](#debugger) 160 | - [Tracer](#tracer) 161 | - [core file](#core-file) 162 | - [callstack](#callstack) 163 | - [calltree](#calltree) 164 | - [backtrace](#backtrace) 165 | - [core_pattern](#core_pattern) 166 | - [undefined behavior](#undefined-behavior) 167 | - [static analysis](#static-analysis) 168 | - [runtime analysis](#runtime-analysis) 169 | 170 | - [Memory concepts](#memory-concepts) 171 | 172 | - [allocation](#allocation) 173 | - [deallocation](#deallocation) 174 | - [reallocation](#reallocation) 175 | - [instantiate](#instantiate) 176 | - [memory address](#memory-address) 177 | - [address space](#address-space) 178 | - [free store](#free-store) 179 | - [new keyword](#new-keyword) 180 | - [allocator](#allocator) 181 | - [stack](#stack) 182 | - [stack allocation](#stack-allocation) 183 | - [heap allocation](#heap-allocation) 184 | - [heap](#heap) 185 | - [custom allocator](#custom-allocator) 186 | - [memory locality](#memory-locality) 187 | 188 | - [Language concepts and keywords](#language-concepts-and-keywords) 189 | 190 | - [inline keyword](#inline-keyword) 191 | - [one-definition rule](#one-definition-rule) 192 | - [inline-expands](#inline-expands) 193 | - [inlined](#inlined) 194 | - [declaration](#declaration) 195 | - [definition](#definition) 196 | - [Forward declaration](#forward-declaration) 197 | - [implementation](#implementation) 198 | - [pointer](#pointer) 199 | - [reference](#reference) 200 | - [noexcept](#noexcept) 201 | - [bytes](#bytes) 202 | - [bits](#bits) 203 | - [structure packing](#structure-packing) 204 | - [sizeof](#sizeof) 205 | - [memory alignment](#memory-alignment) 206 | - [statically typed](#statically-typed) 207 | - [dynamically typed](#dynamically-typed) 208 | - [public member](#public-member) 209 | - [private member](#private-member) 210 | - [protected member](#protected-member) 211 | - [class](#class) 212 | - [struct](#struct) 213 | - [assembly](#assembly) 214 | - [machine code](#machine-code) 215 | - [implicit conversion](#implicit-conversion) 216 | - [explicit](#explicit) 217 | - [explicit conversion](#explicit-conversion) 218 | - [strong type](#strong-type) 219 | - [deterministic](#deterministic) 220 | - [fits into register](#fits-into-register) 221 | - [constexpr](#constexpr) 222 | - [singleton](#singleton) 223 | - [static initialization](#static-initialization) 224 | - [global static](#global-static) 225 | - [static member](#static-member) 226 | - [static method](#static-method) 227 | - [static data](#static-data) 228 | - [macro](#macro) 229 | 230 | - [Node.js & C++](#nodejs--c) 231 | 232 | - [Node](#node) 233 | - [V8](#v8) 234 | - [Node Addon](#node-addon) 235 | - [Event loop](#event-loop) 236 | - [Threadpool](#threadpool) 237 | - [libuv](#libuv) 238 | 239 | ## Core concepts 240 | 241 | ### posix 242 | 243 | A set of standards for maintaining compatibility between unix-like operating systems. Generally posix is synonymous with unix when it comes to what system APIs exist in C/C++ code to interface with the operating system. With minor exceptions POSIX APIs work the same on linux and osx. 244 | 245 | ### API 246 | 247 | Application programming interface. This term is used broadly to describe how software and code can and should interact. In C++ programs an API describes a public set of functions that can be called, classes that can be [instantiated](#instantiate), or [static data](#static-data) that can be used. 248 | 249 | ### ABI 250 | 251 | Application binary interface. This term refers to specific details of how an [API](#API) defintion is compiled by the [compiler](#compiler): for instance the size, order, alignment, and padding of structure members, or the definition of constants. Identical source code that uses identical APIs is not compatible at the ABI level when these definitions or implementation-defined details change. 252 | 253 | ### signal 254 | 255 | Signals are a method of interprocess communication. When a program exits it returns an integer code. That code, if the program crashed, will encode an integer id of a signal. This offers a way for monitoring programs that crashed and why. The signals that result from a [crash](#crash) are listed at . 256 | 257 | To know the exit code of a signal you add the ` + 128`. You can find the id of each signal at . 258 | 259 | More info at 260 | 261 | ### I/O 262 | 263 | An I/O operation that "calls out" from the process to the underlying system. For example, accessing the file system, reading/writing to memory, or sending something over the network 264 | 265 | I/O stands for "input/output". 266 | 267 | ### crash 268 | 269 | A crash is a general term to describe when execution of the program exits in an unintended and unrecoverable way. There are a variety of reasons a program may crash, ranging from a bug that created a fatal error condition to another program killing your program. Each crash can be described by one of a known set of [signals](#signal) which map to a return code. 270 | 271 | For example, a segfault (segmentation fault or violation) leads to a `SIGSEGV` signal, which is id `11`, and an exit code of `128+11` == 139. 272 | 273 | ### libstdc++ 274 | 275 | The [GNU implementation of the C++ STL](https://gcc.gnu.org/onlinedocs/libstdc++/). 276 | 277 | Note that on OS X, `libstdc++` does not support C++11 or C++14, and so we use `libc++` instead. ([Read more](https://github.com/mapbox/cpp#standard-c-library).) 278 | 279 | ### libc++ 280 | 281 | The [LLVM implementation of the C++ STL](https://libcxx.llvm.org/) 282 | 283 | ### source code 284 | 285 | In C++ source code describes `.hpp` or `.cpp` files before they are compiled. Groups of source code passed to the compiler are called [translation units](#translation-unit). 286 | 287 | ## Packaging 288 | 289 | ### mason 290 | 291 | Mason is a package manager for C++. It is able to install [precompiled libraries](#precompiled-library) and [executables](#executable) from binaries. And it is able to install the source code for [header-only libraries](#header-only-library). All packages end up in the `./mason_packages` folder in the current working directory. 292 | 293 | Learn more [at the mason homepage](https://github.com/mapbox/mason). 294 | 295 | ### node-pre-gyp 296 | 297 | node-pre-gyp is a package manager for [node C++ addons](https://github.com/mapbox/cpp/blob/master/node-cpp.md). It is able to install [pre-compiled loadable modules](#loadable-modules) (aka `.node` files) from binaries. 298 | 299 | It is called `node-pre-gyp` because it stands in-front of [gyp](#gyp), the default build system used for node core and node c++ addons. When `node-pre-gyp` is used to download a pre-compiled module it skips needing to invoke [gyp](#gyp) for a source compile. 300 | 301 | Learn more [at the node-pre-gyp homepage](https://github.com/mapbox/node-pre-gyp). 302 | 303 | ## Environment variables 304 | 305 | Build tools on [posix](#posix) systems such as [make](#make), [cmake](#cmake) and [gyp](#gyp) pick up many of these environment variables and use them for compiling and linking your code. If you leave them blank it depends on the build system and your setup what you will get. 306 | 307 | ### CPP 308 | 309 | The C Pre-Processor - the program reading your code and replacing header includes with their content and macros with their values 310 | 311 | ### CC 312 | 313 | C Compiler - the program understanding the C language, taking a .c source code file and generating a .o object file out of it 314 | 315 | Ignored by [cmake](#cmake) if `-DCMAKE_C_COMPILER` is passed. 316 | 317 | ### CXX 318 | 319 | C++ Compiler - the program understanding the C++ language, taking a [translation unit](#translation-unit) and generating an [object file](#object-file) out of it 320 | 321 | Ignored by [cmake](#cmake) if `-DCMAKE_CXX_COMPILER` is passed. 322 | 323 | ### LD 324 | 325 | linker - the program taking multiple [object files](#object-file) and [libraries](#library) and generating an ELF file out of it on linux or a Mach-O file on OS X - the final binary 326 | 327 | ### CFLAGS 328 | 329 | Compilation flags for the C compiler, think `-Wall -Wextra -pedantic -O2 -std=c99` to enable warnings, optimize the code and work with C99 source code 330 | 331 | ### CXXFLAGS 332 | 333 | Same for C++, `-Wall -Wextra -pedantic -O2 -std=c++14`. 334 | 335 | Ignored by [cmake](#cmake) if `-DCMAKE_CXX_FLAGS` is passed. 336 | 337 | ### LDFLAGS 338 | 339 | linker flags, for more advanced linker tuning. 340 | 341 | Note: not respected with [cmake](#cmake) which uses different flags depending on the type of binary being created: 342 | 343 | - CMAKE_EXE_LINKER_FLAGS (for [executables](#executable)) 344 | - CMAKE_SHARED_LINKER_FLAGS (for [shared library](#dynamicshared-library)) 345 | - CMAKE_MODULE_LINKER_FLAGS (for [loadable module](#loadable-module)) 346 | 347 | More info [here](https://stackoverflow.com/a/3544343). 348 | 349 | ## Builds 350 | 351 | ### make 352 | 353 | A venerable build system only compatible with unix systems. Usually available by default on most unix systems without needing to be installed or upgraded. In this sense `make` is a bit like `bash`. It is the first tool many developers reach for when starting a project. 354 | 355 | The configuration for make is a `Makefile`. It uses `tab` indentation. It is rare that you will ever encounter a `Makefile` that needs a specific version of `make`. This is likely because what `make` does is pretty simple and was hammered out a long time ago and has not changed much in recent versions. (But note that most `Makefile`s rely on [GNU Make](https://www.gnu.org/software/make/) features and will not work with a standard [POSIX Make](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html).) 356 | 357 | For a project that only needs to build a few files `make` can be a good choice. For projects that need to build libraries or many files, [cmake](#cmake) is the better choice. 358 | 359 | At Mapbox `make` is commonly used as the entry point for building a complex application. So, you might have a `Makefile` that actually calls out to another build system like [cmake](#cmake) or [gyp](#gyp). This allows the documentation to simply say: type `make` to build this project, while more complex things actually happen under the hood. 360 | 361 | Learn more [at the make homepage](https://www.gnu.org/software/make). 362 | 363 | ### cmake 364 | 365 | A modern, very sophisticated, build system that is cross-platform. 366 | 367 | But cmake is not a build system in the traditional sense: it does not directly call out to the compiler or linker. Instead it is designed to generate build files in a variety of formats. For example, it is common to use `cmake` to generate configuration files for [make](#make) on unix systems and [Visual Studio](#visual-studio) on windows systems. Frequently the desired target is for `cmake` to generate files for [ninja](#ninja) 368 | 369 | The configuration for cmake is `CMakeLists.txt`. It uses a very custom syntax that takes a while to learn, may bring about anger, but works and will scale to large projects. 370 | 371 | The `cmake` project is written in C++ so it needs to be compiled itself. Often needs to be upgraded since you will encounter projects using `CMakeLists.txt` that need specific `cmake` features only available in recent versions. 372 | 373 | Learn more [at the cmake homepage](https://cmake.org/). 374 | 375 | ### gyp 376 | 377 | Also modern and cross-platform like [cmake](#cmake). The advantage is that `gyp` is written in `python` and is therefore easier to install and upgrade. The disadvantage is that google, the original maintainer of `gyp`, is no longer maintaining `gyp` actively. For this reason we recommend porting most projects written in `gyp` to [cmake](#cmake). Google, in the meantime, is porting projects to [bazel](#bazel). 378 | 379 | The `gyp` build system is currently the default build system used by node.js (and therefore node-addons). See also [node-pre-gyp](#node-pre-gyp) which provides binary distribution support in front of `gyp`. 380 | 381 | Learn more about gyp [here](https://gyp.gsrc.io) 382 | 383 | ### bazel 384 | 385 | A build system, written in java, from Google: . At this time (mid 2018) it appears rarely used outside Google, however one projects using it is [envoy](https://github.com/envoyproxy/envoy). 386 | 387 | ### visual studio 388 | 389 | The windows GUI that supports "project" files that `cmake` can generate in order to build C++ files. 390 | 391 | ### ninja 392 | 393 | A cross platform build system that is designed to be fast. Written by google and the chromium project, but now a great, general open source project. 394 | 395 | ### out of source build 396 | 397 | A common build convention is to put into a custom directory structure all the results of all compiling and linking [source code](#source-code). For example, all [object files](#object-files) might be put at a path like `./build/Release/obj/`, all [libraries](#library) at `./build/Release/lib/`, or all [executables](#executable) at `./build/Release/bin`. 398 | 399 | Each build system may use a different directory name and structure, but the general goal is to: 400 | 401 | - Keep the binary outputs separate from the source code 402 | - Version the path of the build results based on the build type (e.g. `Release` or `Debug`) 403 | 404 | This approach is in contrast to an [in-tree build](#in-tree-build). It is advantageous to [in-tree build](#in-tree-build) because: 405 | 406 | - It makes it easy to clean up all compiled files just by removing a single directly. (no complex cleanup scripts are needed to track down all the .o files) 407 | - It makes it easy to create multiple binaries, potentially build with different build options, without overwriting previous binaries. 408 | 409 | ### in-tree build 410 | 411 | An in-tree build describes when a build system compiles all [object files](#object-files) and writes them to the same directory structure where the [source code](#source-code) exists. In this case you might have [source code](#source-code) at `src/foo/bar.cpp` and the object file would end up at `src/foo/bar.o`. 412 | 413 | ## Compilers and linkers 414 | 415 | ### development toolchain 416 | 417 | The toolset used to generate C++ code into [executables](#executable) for a computer. Typically consists of a [compiler](#compiler) and [linker](#linker). 418 | 419 | All C++ code falls into the ["ahead-of-time" compilation](https://en.wikipedia.org/wiki/Ahead-of-time_compilation) category, which means code is compiled and linked separately from execution, unlike ["just-in-time"](https://en.wikipedia.org/wiki/Just-in-time_compilation) compilation languages, like JavaScript. 420 | 421 | ### compiler 422 | 423 | A program that converts C++ [source code](#source-code) into machine code (binary) files readable by a computer. 424 | 425 | Compilers turn `.cpp` code into `.o` [object files](#object-file) and are often a [front-end](#front-end) to the [linker](#linker) 426 | 427 | Example open source compilers are [clang](#clang) or [gcc](#gcc). 428 | 429 | It is common to specify the [CC](#CC) and [CXX](#CXX) environment variables and many build systems ([make](#make) in particular) to customize the default C and C++ compiler that should be used. 430 | 431 | ### Front-end 432 | 433 | The term front-end refers to when a command line tool `A` can be used in place of another command line tool `B` such that you call `B` through the interface of `A`. 434 | 435 | ### linker 436 | 437 | A program that combines [object files](#object-file) into a single [executable](#executable) or [library](#library). Typically a step enacted by the [compiler](#compiler). The linker on unix systems is usually the `ld` command. Type `man ld` for more info or visit [the ld linux man page](https://linux.die.net/man/1/ld). 438 | 439 | ### dynamic linker 440 | 441 | The dynamic linker, aka the loader, is responsible for performing symbol resolution during runtime, either when a program starts (load-time linking) or when an undefined symbol is used (lazy linking). 442 | 443 | `ld.so`, or `dyld` if you're on macos, is the dynamic linker that runs at runtime. Type `man ld.so`, or `man dyld` if you're on macos, for more info. 444 | 445 | You can specify when you want the dynamic linker to resolve symbols by supplying `-z,`. There are a couple different keywords you can use: 446 | 447 | * `lazy` - when you want the dynamic linker to to defer symbol resolution to the point when the symbol is used. This is called **lazy linking**. 448 | 449 | * `now` - when you want the dynamic linker to resolve symbols when the program is started or when the shared library is linked to using `dlopen`. This is called **load-time linking**. 450 | 451 | ### linked 452 | 453 | This term is used to describe the process when an [executable](#executable) or [library](#library) is told to be [dynamically linked](#dynamically-linked) to [library](#library). 454 | 455 | ### linking 456 | 457 | A general term to describe the process of combining all [translation units](#translation-unit) and resolving the undefined [symbols](#symbol) with the defined ones. 458 | 459 | Generally linking occurs when [object files](#object-file) are combined into a [library](#library). 460 | 461 | ### linking order 462 | 463 | The process of [linking](#linking) involves resolving dependencies that your code might depend on, including both across the [translation units](#translation-unit) of your project and external dependencies. When dependencies are libraries, the linker handles resolving [symbols](#symbol) differently depending on a variety of factors including: 464 | 465 | - The order of the libraries in the command sent to the linker on Linux. 466 | - Whether those libraries are [static libraries](#static-library) or [shared libraries](#dynamicshared-library) 467 | - Custom linker flags like `-undefined dynamic` on OSX or `--start-group` on Linux 468 | 469 | This matters when you are linking your code to [static libraries](#static-library) on Linux. Imagine that: 470 | 471 | - Your program is called `convert` and it depends on `libpng` 472 | - You are building against a static `libpng.a` from [mason](#mason) 473 | - `libpng.a` also depends on zlib (could be `libz.so` from the system or libz.a from [mason](#mason)). 474 | - Your build system correctly asks for `convert` to be linked to both libpng and zlib 475 | - But you hit a linking error on defined symbols for zlib 476 | 477 | Why did you hit undefined symbols? Most likely it is because your build system invoked the linker (through clang+++ in this example) with the order of libraries like: 478 | 479 | clang++ -o convert -lz -lpng 480 | 481 | This will fail on Linux, when `libpng` is a static library, because the linker on linux requires `-lz` to be listed **after** `libpng` because `libpng` depends on `zlib`. 482 | 483 | This would fix the linking: 484 | 485 | clang++ -o convert -lpng -lz 486 | 487 | Hence: libraries depended upon by other libraries need to come after the library that needs them. 488 | 489 | Yes, this is really quite tedious and brittle on Linux. If you don't have control of your build system such that you can change it to fix the linking order, you may be stuck. So, another workaround is to disable this behavior by asking the Linux linker to recurse to look for dependencies across all listed libraries. 490 | 491 | This would work: 492 | 493 | clang++ -o convert -Wl,--start-group -lz -lpng 494 | 495 | Because it would tell the compiler to try looking for undefined symbols across the entire group rather than just in order. Note: technically you are supposed to denote the end of the group of libraries with `-Wl,--end-group`, but I've found that is optional. 496 | 497 | See the `Static Libraries` section of for another mention of how linking order matters. 498 | 499 | ### link time optimization 500 | 501 | Link Time Optimization (LTO) describes a method for "intermodular" optimizations on code: which is when optimizations are applied across [translation units](#translation-unit) through coordination between the compiler and the linker. You'll also hear the term "interprocedural", "cross program", or "whole program" optimization which generally refer to the same concept: being able to optimize across [translation units](#translation-unit). 502 | 503 | To understand why this is important we need to know that the compiler, by default, only applies [optimizations](#compiler-optimization-level) on code it sees and normally this is a single [translation unit](#translation-unit). So, if you have all your code in a single `cpp` file (and you are not statically linking any third-party libraries) you are essentially already benefiting from "whole program optimization". But we want modular code for best maintainability and readability, so when we split a single `.cpp` into multiple `.cpp` and effectively spread our code across multiple [translation units](#translation-unit), LTO is the way we can keep our code running as fast as possible. 504 | 505 | Enabling LTO is done by: 506 | 507 | - adding `-flto` to your compiler (aka [cxxflags](#cxxflags)) and link flags (aka [ldflags](#ldflags)) 508 | - ensuring that your linker supports the compilers LTO format (on linux this means installing `binutils-gold`) 509 | - ensuring you use the same compiler to compile all your code (no mixing of compiler versions) 510 | 511 | Then your code should run as fast as possible, despite modularization into separate `.cpp` files. If you are linking third-party static libraries then you'll also want to compile them with `-flto`. 512 | 513 | One downside exists however: LTO requires both your compiler and linker to do more work and this may slow down your build. For small projects this is unimportant. But when building complex projects this might cause your machine to run out of memory or your builds to take forever. This is the reason that significant effort has gone into a variety of different implementations for LTO in compilers. In particular, Google engineers have worked hard on a [technology called "Thin LTO"](https://clang.llvm.org/docs/ThinLTO.html) in [LLVM](#llvm). Thin LTO is enabled with `-flto=thin` and is designed to unlock intermodular optimizations without slowing down builds as much as other LTO implementations. 514 | 515 | But LTO is a complex subject and there is a lot more happening under the hood with `-flto=thin`. Learn all about Thin LTO via Teresa Johnson's presentation at cppcon: 516 | 517 | Learn more about LTO in LLVM at 518 | 519 | ### translation unit 520 | 521 | The input to a compiler from which an [object file](#object-file) is created. Normally this is one or more files with the extension of `.cpp`, `c++`, or `.cc`. 522 | 523 | ### object file 524 | 525 | An object file contains object code. Object code is what a [compiler](#compiler) produces. It contains symbols, compiled code, external symbols, and other [static data](#static-data). If you are compiling three C++ files (aka [translation units](#translation-unit)) – `one.cpp`, `two.cpp`, `three.cpp` – they will be compiled into three object files – `one.o`, `two.o`, `three.o`. These can subsequently be combined by a [linker](#linker) and turned into a [library](#library) or [executable](#executable). 526 | 527 | Note: The file extension of object files is usually `.o`. You may also encounter `.os`. 528 | 529 | ### symbol 530 | 531 | In C/C++ a symbol is the name embedded inside a binary for a function or class. For example, every [executable](#executable) will have a function in the code named `main()`. Once this function is compiled it will be available in the binary as a public symbol. For example, we can confirm this for the `python` binary using the `nm` command: 532 | 533 | ```bash 534 | $ nm $(which python) | grep main 535 | 0000000100000f60 T _main 536 | ``` 537 | 538 | `nm` shows you the symbol table for an [object file](#object-files). [symbols](#symbol) are generally either: 539 | 540 | - `T` = this object file has the implementation of this symbol 541 | - `U` = this object file will need the symbol to be dynamically loaded before use 542 | 543 | Including a header file that only had definitions of functions or classes gets you to the `U` state - this lets the [translation unit](#translation-unit) compile, but you can’t execute it because there will be undefined symbols. It must be [linked](#linked). 544 | 545 | ### executable 546 | 547 | A binary file and/or program that can be run by a computer. This is the outcome of [compiling](#compiler) and [linking](#linker). Specifically, an executable has a `main()` function or _entry point_. A binary that does not have a `main()` entry point is likely a # 548 | 549 | ### loadable module 550 | 551 | A loadable module is similar to a [shared library](#dynamicshared-library). Just like a [shared library](#dynamicshared-library) a loadable module is: 552 | 553 | - created by [linking](#linking) 554 | - [pre-compiled](#precompiled) 555 | - [dynamically loaded](#dynamically-loaded). 556 | 557 | But, whereas a [shared library](#dynamicshared-library) is [dynamically linked](#dynamically-linked) at startup, a loadable module is designed to be loaded sometime after startup, by the [calling application](#calling-application), with [dlopen](#dlopen). 558 | 559 | Most plugin type interfaces in C/C++ are implemented as loadable modules. These loadable modules can have any extension, but most commonly use the `.so` extension like a [shared library](#dynamicshared-library) does on linux. 560 | 561 | Advanced note: A loadable module may depend on [symbols](#symbol) from external libraries. When those symbols are also needed by the [calling application](#calling-application) that will load the loadable module, then it is common for the loadable module to be created without resolving (aka linking it to) those external symbols. This means that the loadable module is [linked](#linked) without [linking](#linking) all of its dependencies. It will have undefined symbols when it is loaded unless they are available already in the [address space](#address-space) of the [calling application](#calling-application) already. This will be possible if the the [calling application](#calling-application) has itself been either [statically linked](#statically-linked) or [dynamically linked](#dynamically-linked) to external libraries that provide those [symbols](#symbol). 562 | 563 | ### library 564 | 565 | A set of reusable C++ code that can be shared across projects. Libraries can be organized very differently, but typically contain a set of header files (static library) or [pre-compiled binaries (dynamic/shared library)](#precompiled-library). 566 | 567 | As opposed to an [executable](#executable), a library does not have a `main()` entry point and, instead, provide a set of functions and classes that can be called by other C/C++ libraries or executables. It must include at least a single [header file](#header) so the _calling application_ knows the definition of the interfaces provided. A library can be as static archives ([static library](#static-library)) or as a [shared library](#dynamicshared-library). 568 | 569 | ### precompiled library 570 | 571 | A general term to describe a [library](#library) created from [precompiled](#precompiled) code. All libraries are created this way, but may be turned into different types of libraries depending on how they should be used. Most commonly a precompiled library is either a [shared library](#shared-library) or [static library](#static-library). 572 | 573 | ### precompiled 574 | 575 | A term used to describe something created from compiling code that defines an [API](#API). Something precompiled is assembled from a set of one or more C++ files (aka [translation unit](#translation-unit)) turned into [object files](#object-files). 576 | 577 | ### shared library 578 | 579 | A shared library is a type of [precompiled library](#precompiled-library). It required at link time and runtime by applications that depend on it. This means it has already been [compiled](#compiler) into machine-readable code and is just required alongside your code when it executes. 580 | 581 | Also known as a dynamic library. 582 | 583 | ### static library 584 | 585 | A static library is a type of [precompiled library](#precompiled-library). It is only required at link time. When a static library is linked to another application all of its [object files](#object-file) are put into that application's executable. This step is often called "static linking". The [source code](#source-code) of a static library looks identical to a shared library (both contain headers and .cpp files). The only difference is the way it is built. 586 | 587 | ### statically linked 588 | 589 | A program can statically link a [static library](#static-library). This means that no external library is needed at runtime. Instead all the [symbols](#symbol) needed are available in the binary already. 590 | 591 | ### dynamically linked 592 | 593 | A [shared library](#shared-library) can be dynamically linked to another binary (either an [executable](#executable) or another [shared library](#shared-library)). The link happens during a build and is done by the system linker (usually the `ld` program or a compiler). This creates a dependence on this [shared library](#shared-library) that is resolved at startup of the program linking it. This runtime resolution at startup is done by the [operating systems loader](https://en.wikipedia.org/wiki/Loader_(computing)). 594 | 595 | ### dynamically loaded 596 | 597 | A general term to describe a binary that is loaded either of these two cases: 598 | 599 | - A [library](#library) loaded into the [address space](#address-space) of a program by [dynamic linking](#dynamically-linked) 600 | - A [loadable module](#loadable-module) loaded into the [address space](#address-space) of a program by [dlopen](#dlopen). 601 | 602 | ### dlopen 603 | 604 | The `dlopen` api is a [POSIX api](#posix) API available on linux and osx (on windows a similar API is `LoadLibraryA`) that is able to dynamically load a [shared library](#dynamicshared-library) or [loadable module](#loadable-module). 605 | 606 | It is often used with [loadable modules](#loadable-module) to implement plugin systems to C++ programs. 607 | 608 | It is often used with pure C [shared libraries](#dynamicshared-library) and [the libffi foreign function interface library](https://en.wikipedia.org/wiki/Libffi) to implement language bindings to C libraries. 609 | 610 | More info at [this man7 page](http://man7.org/linux/man-pages/man3/dlopen.3.html) 611 | 612 | ### header-only library 613 | 614 | Used to describe when code is organized such that all of the [source code](#source-code) is in the .hpp file such that: 615 | 616 | - No cpp files need to be compiled 617 | - To use the library, no library needs to be linked (just [using `#include `](#include) is enough 618 | 619 | A well organized header-only library is rarely in a single header, but rather can be accessed by both: 620 | 621 | - A single header that includes all other headers 622 | - Public access to each individual header that implements discrete functionality with the aim of [zero cost design](#zero-cost-design). 623 | 624 | ### zero cost design 625 | 626 | When library headers are organized in discrete parts such that downstream users can include just the functionality they need and nothing more. Zero cost design indicates there is no extra cost beyond the cost that is unavoidable. Zero cost design is [separation of concerns](https://en.wikipedia.org/wiki/Separation_of_concerns) applied to C++ libraries. It is particularly important for [header-only libraries](#header-only-library) but also is relevant to [precompiled libraries](#precompiled-library). 627 | 628 | But really, what do we mean by "cost"? 629 | 630 | The motivation is to avoid two types of cost: 631 | 632 | - adding extra compile time to downstream builds because major chunks of code are present in a header even when not used by consumers. 633 | - having headers pull in other headers that might have external dependencies which are not needed. 634 | 635 | ### LLVM 636 | 637 | Low level virtual machine - a suite of tools that [clang++](#clang++) depends on. 638 | 639 | ### gcc 640 | 641 | The gnu C compiler. See also [g++](#g++). 642 | 643 | ### clang 644 | 645 | The `clang` is the same as [clang++](clang++) except compiles C code instead of C++. Usually you should be using `clang++` even if you are compiling pure C. 646 | 647 | ### g++ 648 | 649 | The gnu C++ compiler. Very similar to bu licensed differently and slower to compile C++ templates. 650 | 651 | ### clang++ 652 | 653 | The recommended [C++ compiler](#compiler) used a Mapbox. It is open source and packaged in [mason](#mason). 654 | 655 | ### calling application 656 | 657 | A term used to describe the application that depends on the [API](#API) of another application. 658 | 659 | ### header 660 | 661 | A file with the `.hpp` or `.h` file extension. 662 | 663 | ### include 664 | 665 | Pronounced "hash include". This is the line of source code that looks like `#include ` at the top of a C++ file. This syntax includes a header and allows you to utilize the code defined in the included header file. 666 | 667 | Literally speaking, [the compiler will "replace" the #include line with the actual contents of the file you're including when it compiles the file.](http://www.cplusplus.com/forum/articles/10627/). 668 | 669 | There are [a couple ways](https://stackoverflow.com/questions/21593/what-is-the-difference-between-include-filename-and-include-filename/21594#21594) to include headers: 670 | 671 | - Using carrots, ex: `` --> look for header in the compiler search paths 672 | - Using quotes, ex: `"hello.hpp"` --> look for header in location relative to this file 673 | 674 | ## Performance 675 | 676 | ### performance 677 | 678 | How quickly a program does its work. Improving performance involves doing work faster by optimizing code to be more [efficient](#efficiency) through code changes or [compiler optimizations](#compiler-optimization-level). But ultimately how fast a program runs is also constrained by the system resources and how long certain operations take on a computer: . 679 | 680 | Performance in concurrent systems usually refers to [response time](#response-time) for a single request. When we describe the performance of systems under load we usually are referring to [scalability](#scalability) 681 | 682 | ### efficiency 683 | 684 | How much work is required by a task. Improving efficiency requires doing less work and results in [performance improvements](#performance). An efficient program is one which does the minimum (that we're aware of) amount of work to accomplish a given task. 685 | 686 | - See also: CppCon 2014: Chandler Carruth "Efficiency with Algorithms, Performance with Data Structures" 687 | 688 | ### expressive 689 | 690 | Exposing intent clearly through programming language constructs. Not resorting to boiler-plate code and/or over-commenting to achieve it. 691 | 692 | ### optimization technique 693 | 694 | Changing code to increase [efficiency](#efficiency) or [performance](#performance). 695 | 696 | ### memoization 697 | 698 | An [optimization technique](#optimization-technique) that reduces the amount of work needed and increases [efficiency](#efficiency) by storing the results of expensive function calls or data access. 699 | 700 | ### small size optimization 701 | 702 | Also known as SSO, this refers to when [stack allocation](#stack-allocation) is used to make data structures extremely fast. This strategy generally works by writing a custom class which: 703 | 704 | - Has a data member is used to store data. 705 | - That data member is not dynamically allocated unless the container needs to grow to a large size. 706 | - This saves on many allocations which often end up being a bottleneck. 707 | 708 | This can be described as using "hybrid data structures". 709 | 710 | This is also sometimes referred to as SBO (Small Buffer Optimization). 711 | 712 | This is such an important optimization it is central to how the llvm compiler is written. See Chandler Carruth's talk at CppCon 2016 that covers this: “High Performance Code 201: Hybrid Data Structures" . 713 | 714 | This optimization was applied to `std::string` in libstdc++ () 715 | 716 | It is also central to why Javascript is fast in node.js and chrome. V8 uses SSO for objects like [SmallOrderedHashTable and more](https://github.com/v8/v8/blob/fe598532ec1317e8b85343133be9fb708e07bd2e/src/objects/hash-table.h#L613-L649). 717 | 718 | See also [this](https://stackoverflow.com/questions/10315041/meaning-of-acronym-sso-in-the-context-of-stdstring/10319672#10319672) and [this](https://akrzemi1.wordpress.com/2014/04/14/common-optimizations) for more about SSO. 719 | 720 | ### performant 721 | 722 | Not yet an official word in English. Usually meant to refer to a program that performs well enough by some measure. Prefer describing a program as either having acceptable [performance](#performance) or acceptable [efficiency](#efficiency). Or consider if your program is better described as fast, [responsive](#responsive), or [low latency](#latency). 723 | 724 | See also 725 | 726 | ### responsive 727 | 728 | A term used to describe when a program or API has a quick [response time](#response-time). 729 | 730 | See also 731 | 732 | ### latency 733 | 734 | A term used to describe [response time](#response-time). A low latency system responds quickly. Systems that are low latency even under increased load (more requests) have the potential to be highly [scalable](#scalable). 735 | 736 | See also 737 | 738 | ### response time 739 | 740 | The time taken between a request and response. 741 | 742 | ### scalable 743 | 744 | Systems where [performance](#performance) is maintained under increasing load. 745 | 746 | ### scalability 747 | 748 | How [scalable](#scalable) a system is by some measure of load. 749 | 750 | ### compiler optimization level 751 | 752 | [Compilers](#compiler) transform code into binaries that can run on various platforms. But they also optimize that code in specific ways depending on the `optimization level`. 753 | 754 | The `optimization level` impacts both how fast the resulting binaries will run, their size, and how much diagnostic information we can learn about the running binary through profiling. 755 | 756 | The most important reason to understand `optimization levels` is to understand the larger concept of [build modes](#build-modes) including: 757 | 758 | - [release builds](#release-build) 759 | - [debuggable release builds](#debuggable-release-build) 760 | - [debug builds](#debug-build) 761 | - [profiling builds](#profiling-build) 762 | - [sanitized builds](#sanitized-build) 763 | 764 | There are hundreds of different optimization options inside compilers. Luckily we don't need to know about the details because the common open source compilers (`clang++` and `g++`) provide a simplified "O" interface. You simply pass `-ON` where `O` is a capitol O and `N` is a number between `0` and `3`. 765 | 766 | - `-O0` - "oh zero" disables optimization. 767 | 768 | - The program will run very very slowly and the resulting binary will likely be much larger (no dead code removal). 769 | - The advantage is that [backtraces](#backtrace) to be much more detailed because inlining and other optimizations are disabled that would otherwise shrink the size of the [callstack](#callstack). 770 | - A [backtrace](#backtrace) for a crash will likely show the exact line number where a crash happened when also built with `-g`. 771 | 772 | - `-O1` - "oh one" is moderately optimized. 773 | 774 | - Not all functions will be inlined, but the program should still run reasonably fast and dead code will be removed. 775 | - This level is a good compromise when you want good [backtraces](#backtrace)(when also built with -g) but not awful speed. 776 | 777 | - `-O2` - "oh two" is optimized, but should not explode code size 778 | 779 | - The program will run nearly as fast as possible and the resulting binary will be small 780 | - [Backtraces](#backtrace) will be less detailed due to inlining 781 | 782 | - `-O3` - "oh three" enables most all optimizations inside the compiler. 783 | 784 | - The program will run as fast as possible and the resulting binary will be smaller (except where inlining might increase its size somewhat). 785 | - [Backtraces](#backtrace) will be less detailed due to inlining 786 | 787 | - `-Os` - "oh s" enables optimizations for speed, but not at the cost of code size. 788 | - The program may run slower than `-O2` or `-O3`, but how much slower is hard to determine without extensive testing. 789 | 790 | Compilers are rapidly adding more and more optimizations and shifting around which internal optimizations are enabled in which "O" level. See also: 791 | 792 | - [clang](https://clang.llvm.org/docs/CommandGuide/clang.html#cmdoption-o0) 793 | - [gcc](https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html) 794 | - [discussion by clang devs](http://clang-developers.42468.n3.nabble.com/Meaning-of-LLVM-optimization-levels-td4032493.html) 795 | - [a listing](http://llvm.org/docs/Passes.html) of all the internal compiler optimizations in clang++ which you don't need to know specifically, but which are under-the-hood of the "O" levels. 796 | - [a detailed summary](http://stackoverflow.com/a/15548189) of which internal optimizations are grouped in which "O" level across clang++ releases. 797 | 798 | ### concurrency 799 | 800 | Concurrency is the process of executing different pieces of the same process to allow for parallel execution of these pieces \[[wikipedia](https://en.wikipedia.org/wiki/Concurrency_(computer_science))]. See [more info here](https://github.com/mapbox/cpp/blob/master/node-cpp.md#why-create-a-node-addon). 801 | 802 | ## Build modes 803 | 804 | ### DNDEBUG 805 | 806 | `DNDEBUG` stands for `do not debug` or `define no debug` and is a [preprocessor macro](#macro) or `#define`. It is passed as an option to a [compiler](#compiler). 807 | 808 | To define it pass `-DNDEBUG` to the compiler. It is the opposite of [`-DDEBUG`](#DDEBUG) which stands for `yes, please, do debug`. 809 | 810 | The `-DNDEBUG` flag internally tells the compiler to remove [assert](http://en.cppreference.com/w/cpp/error/assert)s from the code. This allows the program to run faster (less expensive checks) at the cost of no longer preventing the program from stopping on `assert`s that might protect from [undefined behavior](#undefined-behavior). 811 | 812 | :warning: Do not be tempted to write unit tests using `assert` unless you are 100% confident your build system sets `-DDEBUG`. Otherwise your tests will not be testing what you think because the `assert`s may be gone! 813 | 814 | Also, sometimes programs hook into the `DNDEBUG` `#define` to apply their own custom behavior. For example `protozero` changes behavior depending if `-DNDEBUG` is set. If it is set then C++ exception are thrown only when something terrible goes wrong parsing. If `-DNDEBUG` is not set then `protozero` assumes you want `-DDEBUG` and enables `asserts` that help catch bugs that might not ever show up with `-DNDEBUG` (aka [release mode](#release-mode)). More details on this example at . 815 | 816 | See also: 817 | 818 | ### DDEBUG 819 | 820 | `DDEBUG` stands for `yes, please, do debug` or `define debug` and is a [preprocessor macro](#macro) or `#define`. It is passed as an option to a [compiler](#compiler). 821 | 822 | To define it pass `-DDEBUG` to the compiler. It is the opposite of [`-DNDEBUG`](#DNDEBUG) which stands for `do not debug`. 823 | 824 | When defined [assert](http://en.cppreference.com/w/cpp/error/assert) in the code will be kept and enabled. 825 | 826 | ### release mode 827 | 828 | Release mode is when a [release build](#release-build) is run. 829 | 830 | ### release build 831 | 832 | A release build describes a C++ binary built with a high [compiler optimization level](#compiler-optimization-level) and the `-DNDEBUG`flag. A release build is often built without flags enabling [debug symbols](#debug-symbols) (when developers want the smallest possible binaries). But because [debug symbols](#debug-symbols) do not impact program speed, some projects and developers prefer to include flags enabling [debug symbols](#debug-symbols) even in release builds. 833 | 834 | ### debug build 835 | 836 | A debug build describes a C++ binary built with the [`-DDEBUG`](#DDEBUG) flag and with lower [compiler optimization levels](#compiler-optimization-level). A debug build is also usually built with flags enabling [debug symbols](#debug-symbols) 837 | 838 | Debug builds run slower and may assert on hidden bugs in the code. They are very useful for testing and finding problems in the code. 839 | 840 | ### debuggable release build 841 | 842 | Like a [profiling build](#profiling-build), a debuggable release build is a hybrid between a [release build](#release-build) and a [debug build](#debug-build). A debuggable release build tries to address the [problem of debugging release-crashes](#problem-of-debugging-release-crashes). 843 | 844 | A build like this should: 845 | 846 | - Use the [highest compiler optimization level](#compiler-optimization-level) 847 | - Enable [debug symbols](#debug-symbols) by passing `-g`. 848 | 849 | This is similar to a [profiling build](#profiling-build) but without any extra flags that might hurt [performance](#performance). 850 | 851 | Note: Developers that use cmake can automatically build this way by passing the `RelWithDebInfo` value to the [CMAKE_BUILD_TYPE](https://cmake.org/cmake/help/v3.0/variable/CMAKE_BUILD_TYPE.html) variable like: `-DCMAKE_BUILD_TYPE=RelWithDebInfo` 852 | 853 | ### sanitized build 854 | 855 | Sanitized builds include the `-fsanitize` option and produce special binaries. This option accepts one or more named [sanitizers](#sanitizers) that instrument your code to help catch problems at runtime. 856 | 857 | The sanitizers are designed to be efficient and generally low overhead compared to other tools for catching C++ bugs. This makes them viable to run in production or staging environments to catch bugs that: 858 | 859 | - might only occur under high load 860 | - might never occur in unit tests due to lacking coverage or absence of memory or cpu pressure 861 | - might never occur if the binaries were [debug builds](#debug-build) because [debug builds](#debug-build) slow down execution so much that race conditions or undefined behavior may vanish. 862 | 863 | For unit tests a sanitized build should likely be a [debug build](#debug-build) with sanitizers enabled. This way invalid code execution will stop abruptly and present a detailed stack trace of the problem, for quick fixing. 864 | 865 | For production a sanitized should be a [profiling build](#profiling-build) with sanitizers enabled such that the binary still runs fast and therefore more closely emulates a [release build](#release-build) that is running in production. 866 | 867 | ### sanitizers 868 | 869 | Sanitizers provide runtime checks for various forms of undefined or suspicious behavior. When enabled they make C++ more akin to other ["safe" languages that trap errors as they happen](http://blog.regehr.org/archives/213). 870 | 871 | They are available as part of clang++: 872 | 873 | ### profiling build 874 | 875 | Like a [debuggable release build](#debuggable-release-build), a profiling build is a hybrid between a [release build](#release-build) and a [debug build](#debug-build). A profiling build tries to address the [problem of profiling and optimization levels](#problem-of-profiling-and-optimization-levels). 876 | 877 | So, a profiling build generally should: 878 | 879 | - Use the [highest compiler optimization level](#compiler-optimization-level) 880 | - Disable `assert`s by enabling [DNDEBUG](#DNDEBUG) 881 | - Disable key compiler optimizations or options that make [callstacks](#callstack) more detailed: 882 | - Add `-fno-omit-frame-pointer` (no significant [performance](#performance) cost) 883 | - Consider adding `-fno-inline-functions` (potentially significant [performance](#performance) cost, so test before adding this to an -O3 build) 884 | - Enable [debug symbols](#debug-symbols) by passing `-g`. Except when binary size is important and you want to keep it at a minimum the ideal solution is to only request the compiler to add debug metadata need for better [callstacks](#callstack). (This can be done when building with `clang++` by passing `-gline-tables-only` instead of `-g`) 885 | 886 | This boils down to: 887 | 888 | ```cpp 889 | clang++ -O3 -DNDEBUG -fno-omit-frame-pointer -gline-tables-only ... 890 | ``` 891 | 892 | ### problem of debugging release-crashes 893 | 894 | When production binaries crash the [callstacks](#callstack) in the [backtraces](#backtrace) will not likely contain line numbers for where, exactly, the code crashed. Instead the only clue for what happened is the name of last function called (that was not inlined by the compiler). If the function is simple and only a few lines of code, perhaps this will give you the programmer enough of the lead to see the potential problem and fix the bug that caused the crash. But more likely the function will be composed of many lines of code and/or it may call other functions that have been inlined away. 895 | 896 | So really you want the line number for where the crash happened. An ideal solution might be to run the [debug builds](#debug-build), replicate the crash, to get a more detailed backtrace. But more often than not it is difficult to impossible to replicate a crash with [debug builds](#debug-build). An example of this would be a rare race condition that only happens under load: it can't be replicated with a [debug build](#debug-build) because the code runs too slow. Or it can only be replicated under production load and [debug builds](#debug-build) can't be pushed into production since it would hurt [performance](#performance) too much. 897 | 898 | The solution to this problem then is to build your [release builds](#release-build) with just enough debug information to get line numbers, without hurting [performance](#performance). For details on how to do this see [debuggable release build](#debuggable-release-build). 899 | 900 | ### problem of profiling and compiler optimization levels 901 | 902 | [Profiling](#profiling) is a great strategy for learning why and where a program is slow. But caution must be used in interpreting the results when profiling because: 903 | 904 | - If you are profiling [release builds](#release-build) then your profiling output is likely not showing inline functions which may mislead you about where time is spent in the program. Profilers generally aggregate timing results in a [calltree](#calltree) composed of multiple times sampling the [callstack](#callstack). If the [callstack](#callstack) is not showing you all the functions because some have been inlined away, you may not be able to see the whole picture. 905 | 906 | - If you are profiling [debug builds](#debug-build) then you should be able to see the entire [callstack](#callstack) and therefore [calltree](#calltree) should be very detailed. However you can't trust the timing results because they represent how much time was spent for unoptimized code. 907 | 908 | There is no perfect solution, other than only relying on profiling output to guide your understanding of a program and not letting it be the end-all word. But consider profiling a [profile build](#profiling-build) for a binary that should give more trustworthy results than a debug builds](#debug-build) and more detailed [callstacks](#callstack) than a [release builds](#release-build). 909 | 910 | ## Debugging and Profiling 911 | 912 | ### Profiling 913 | 914 | Profiling is a way to analyze or measure the [efficiency] of a program: the work it is doing when run. Profiling can help measure things like memory usage, how often a particular function is called within a process, or how many threads are running. It is especially useful for determining how to best optimize a program. 915 | 916 | Check out [this guide](https://github.com/springmeyer/profiling-guide) for more information on how to profile. 917 | 918 | ### debug symbols 919 | 920 | Extra stuff the compiler encodes in a binary to help [debuggers](#debugger) inspect details about a running or crashed program and [tracers](#tracer) collect detailed information about program execution. Normally to enable debug [symbols](#symbol) you pass `-g` to a compiler. This can be done in either a [release build](#release-build) or a [debug build](#debug-build). 921 | 922 | ### Debugger 923 | 924 | A program like `gdb` or `lldb` that is able to run or "hook" into executables, pause them, inspect variables, print backtraces, and see what threads are doing. 925 | 926 | ### Tracer 927 | 928 | A program like `perf` or `Activity Monitor` that is able to observe program behavior and generate summary reports. 929 | 930 | ### core file 931 | 932 | Core files, also known as core dumps, are a file generated by the system after a program [crash](#crash). The system however must be ask, before a crash happens, to enable them. This is usually done by calling `ulimit -c unlimited` inside a shell. But where is this file saved and how can you locate them? It depends on how the system is configured. The way to find out is to look at the system's [core_pattern](#core_pattern). 933 | 934 | Core files contain a snapshot of information about the program at the moment before the program exited and at the time of the crash. Core files can be read by [debugger](#debugger). A common use of [debugger](#debugger) is to generate a [backtrace](#backtrace) from a [core file](#core-file). 935 | 936 | More info at 937 | 938 | ### callstack 939 | 940 | The `callstack` is the list of functions that are being called at a given moment, as reported in a backtrace. Each thread in a backtrace will have its own `callstack`. Usually a `callstack` from a [debugger](#debugger) is read from bottom to top: the function at the bottom was called first (the parent) and the function at the top was called last (final child). 941 | 942 | See also [calltree](#calltree) 943 | 944 | ### calltree 945 | 946 | When sampling a program over time multiple pathways through function calls will be observed. While a [calltree](#calltree) captures what functions are being called at a given moment, a `calltree` is one way to represent a summary of the branches of code that executed over a given time interval. This is a common display format used by [tracers](#tracer). 947 | 948 | ### backtrace 949 | 950 | A backtrace, also known as a stack trace, "is a report of the active stack frames at a certain point in time during the execution of a program". (from ). 951 | 952 | The C/C++ language has a method called [backtrace and backtrace_symbols](http://man7.org/linux/man-pages/man3/backtrace_symbols.3.html) that can be used to print out the active stack frames. However, because C/C++ crashes might result from invalid use of memory, or out of memory conditions, it is dangerous to try to run more code, even these functions, after a [crash](#crash). Therefore the convention to get backtraces for crashes is to enable [core file](#core-file) generation and use a [debugger](#debugger) to generate a backtrace after the [crash](#crash) has happened. 953 | 954 | Have a look at [logbt](https://github.com/mapbox/logbt) for a project that makes generating backtraces easier, on both Linux and OS X. 955 | 956 | ### core_pattern 957 | 958 | The core pattern describes where the system should create a [core file](#core-file). Therefore it usually consists of a path on the filesystem and one or more tokens that are populated to dynamically construct the filename. 959 | 960 | On Linux, the default core_pattern can be read by doing: 961 | 962 | ```bash 963 | cat /proc/sys/kernel/core_pattern 964 | ``` 965 | 966 | More info at 967 | 968 | And on OS X, the default can be read by doing: 969 | 970 | ```bash 971 | sysctl -n kern.corefile 972 | ``` 973 | 974 | More info at 975 | 976 | ### undefined behavior 977 | 978 | See and 979 | 980 | ### static analysis 981 | 982 | Static analysis of code is checks or assertions that are done on the code without needing to execute or run the code. The advantage of this is that these checks may find bugs which [runtime analysis](#runtime-analysis) alone would not see. Ideally you implement both methods in your project. 983 | 984 | ### runtime analysis 985 | 986 | Runtime analysis of code is checks or assertions in the code that happen when your code runs. Usually these checks, like the [santizers](#sanitizers), will only find problems in code that has tests or that is actually used. To find problems in code that you don't have tests for, consider [static analysis](#static-analysis). Ideally you implement both methods in your project. 987 | 988 | ## Memory concepts 989 | 990 | ### allocation 991 | 992 | A term used to describe when memory is requested to create space to hold a variable. This request is answered by the [allocator](#allocator) for [dynamically allocated memory](#allocator). Usually we are talking about [dynamically allocated memory](#allocator) when we speak about allocation, unless the object was allocated on the [stack](#stack-allocation). The [dynamic memory allocator](#allocator) is often the bottleneck for programs that have been optimized in all the obvious ways. In other words, once you have optimized the easy things and looked at what still takes time in a program, a likely culprit is allocation. Therefore the key to unlocking the highest levels of [efficiency](#efficiency) is often reducing allocations. 993 | 994 | ### deallocation 995 | 996 | A term used to describe when dynamically allocated memory is released. This is when the space previously allocated to hold a variable is let go such that it can be used for allocating new variables. This release is answered by the [allocator](#allocator), and can take the same amount of time to service as [allocation](#allocation). 997 | 998 | ### reallocation 999 | 1000 | A term used to describe when the space previously allocated to hold a variable is discarded and re-created at a new size, usually larger. Reallocation is expensive because it triggers both [deallocation](#deallocation) and [allocation](#allocation). 1001 | 1002 | ### instantiate 1003 | 1004 | To instantiate a C++ type is to create a variable that refers to an instance of that type. That instance may point to [dynamically allocated memory](#allocator) on the [heap](#heap) if the [new keyword](#new-keyword) was used. Or if the [new keyword](#new-keyword) was not used then [stack allocation](#stack-allocation) is how the instance was created and the object will be temporary and go out of scope. 1005 | 1006 | For example, to instantiate a temporary variable `a` that is type `std::int64_t` and value `1`, on the stack, you would do: 1007 | 1008 | ```c++ 1009 | { 1010 | std::int64_t a = 1; 1011 | } 1012 | ``` 1013 | 1014 | After program control passes `}` the the variable has gone out of scope and will no longer exist. 1015 | 1016 | If you instantiate the same type but with the [new keyword](#new-keyword), then: 1017 | 1018 | - the type returned will be a pointer to the type rather than just the type 1019 | - the pointer will point to a dynamically allocated [memory address](#memory-address) 1020 | - the pointer must be deleted to avoid a memory leak 1021 | 1022 | Here is an example of dynamical allocation: 1023 | 1024 | ```c++ 1025 | { 1026 | std::int64_t * a = new std::int64_t(1); 1027 | delete a; 1028 | } 1029 | ``` 1030 | 1031 | Note: when instantiating classes without the [new keyword](#new-keyword) you might think that only [stack allocation](#stack-allocation) is being used. For example when creating a `std::string` like: 1032 | 1033 | ```c++ 1034 | std::string a("hello"); 1035 | ``` 1036 | 1037 | That is using [stack allocation](#stack-allocation) so you might thing that no dynamic memory is being allocated. However, dynamic memory is being allocated. This is because `std::string` is a container class that holds an arbitrary length array of characters inside. So to create space for this arbitrary array of characters it uses [dynamically allocated memory](#allocator) internally. It also cleans up this memory automatically by [deallocating](#deallcation) when the `std::string` goes out of scope. Both allocation and deallocation take time. So when [performance](#performance) is critical and you want to avoid dynamic allocations, always give consideration to how the class is implemented that you are instantiating. 1038 | 1039 | Note: We say above that `std::string` uses [dynamically allocated memory](#allocator) internally. This is true except when the `std::string` implementation itself has been optimized to reduce allocations. Many implementations are clever enough and care about [performance](#performance) enough to do this using an optimization internally that uses [stack allocation](#stack-allocation) for the array of characters. This only works for small strings and as such is called the [short/small size optimization](#small-size-optimization) or SSO. 1040 | 1041 | ### memory address 1042 | 1043 | A value like `0x7fb20ac02680` that points to a location in the program's [address space](#address-space). Various things are "Addressable" in this space. See for more details. 1044 | 1045 | ### address space 1046 | 1047 | In C++ the term address space usually refers to the scope of memory the program has access to. You can refer to something in code by referencing its [memory address](#memory-address) if it is in the same address space as your code. 1048 | 1049 | ### free store 1050 | 1051 | A term use to describe free memory or [address space](#address-space) not yet requested. 1052 | 1053 | ### new keyword 1054 | 1055 | In C++ the `new` keyword is used to allocate an object dynamically, on the [heap](#heap). The result is a pointer to a class instance. This object must be cleaned up by calling `delete` on the pointer. 1056 | 1057 | ### allocator 1058 | 1059 | C and C++ programs all depend upon a single dynamic memory allocator to implement dynamic management functions like `malloc` and related functions (`free`, `calloc`, and `realloc`). 1060 | 1061 | We call this "dynamic" memory because the size of memory requested is determined at runtime by your C++ code: what objects it creates and how much space they require. 1062 | 1063 | Dynamic memory allocation requires able [RAM](https://en.wikipedia.org/wiki/Random-access_memory) available on your machine. 1064 | 1065 | The default allocator is usually tuned to perform well under varied conditions and provided as part of the implementation of the C library. [Custom allocators](#custom-allocator) can be swapped in and may perform better. 1066 | 1067 | ### stack 1068 | 1069 | In C++ when you write code like: 1070 | 1071 | ```c++ 1072 | int num = 1; 1073 | MyClass klass = MyClass(); 1074 | ``` 1075 | 1076 | You are depending on pre-allocated [address space](#address-space) reserved for your program for placing those temporary objects. 1077 | 1078 | This pre-allocated [address space](#address-space) is the stack. Your program has access to it automatically. It is local to a thread of execution. 1079 | 1080 | When a program runs out of stack space (too much is allocated on the stack) you get a [stack overflow](https://en.wikipedia.org/wiki/Stack_overflow). 1081 | 1082 | Note: there is also a `std::stack` in the C++ STL. This is different - this is a [data structure](http://en.cppreference.com/w/cpp/container/stack). 1083 | 1084 | ### stack allocation 1085 | 1086 | When an object is created without the `new` keyword, it is allocated on the [stack](#stack) and does not need to be manually deleted. 1087 | 1088 | Generally stack allocation is faster than [heap allocation](#heap-allocation) because it does not require a call to the [dynamic memory allocator](#allocator). 1089 | 1090 | Often very tuned C++ programs find their remaining bottlenecks are memory allocation. In this case, to further increase [performance](#performance), developers often look for ways to utilize stack allocation without the fear of [overflow](https://en.wikipedia.org/wiki/Stack_overflow). See [stack alloc](https://howardhinnant.github.io/stack_alloc.html) as well as "SmallVector" inside llvm which both allocate on the stack up for a limite number of elements. See [this talk](https://youtu.be/vElZc6zSIXM?t=135) or [this stack overflow (pun not intended) mention here](https://stackoverflow.com/a/42122275). 1091 | 1092 | Read more [here](https://stackoverflow.com/a/80113) 1093 | 1094 | ### heap allocation 1095 | 1096 | When an object is created with the `new` keyword it is allocated on the [heap](#heap) and must be deleted otherwise your program will have a memory leak. 1097 | 1098 | When a chunk of memory is initialized by calling `malloc`, this is also heap allocation. The `new` keyword used with a custom object calls `malloc` under the hood with the right arguments to request memory for the custom object. 1099 | 1100 | The advantage of heap allocation is: 1101 | 1102 | - The stack is limited in size and might [overflow](https://en.wikipedia.org/wiki/Stack_overflow) 1103 | - The result of heap allocation is a pointer that can be passed around and is not limited to the local scope. 1104 | 1105 | Note: as of C++11, now that you can move stack allocated objects using `std::move` you can treat these objects more like heap allocated pointers (they can be passed around and, once moved, are not limited to the local scope). 1106 | 1107 | Read more [here](https://stackoverflow.com/a/80113) 1108 | 1109 | ### heap 1110 | 1111 | The heap is [address space](#addres-space) in RAM. 1112 | 1113 | ### custom allocator 1114 | 1115 | A custom allocator may be able to allocate memory faster, or use memory more efficiently. An example of such an allocator is [jemalloc](https://github.com/jemalloc/jemalloc). 1116 | 1117 | ### memory locality 1118 | 1119 | A term used to describe when the location of data ([memory address](#memory-address)) is coincident enough (often because layout is contiguous) such that repeated access of that data results in a high cpu cache hit ratio. 1120 | 1121 | Also known as [`locality of reference`](https://en.wikipedia.org/wiki/Locality_of_reference). 1122 | 1123 | See for an example of how memory locality relates to performance. 1124 | 1125 | ## Language concepts and keywords 1126 | 1127 | ### inline keyword 1128 | 1129 | A standalone function or a class method can be marked with the `inline` keyword like: 1130 | 1131 | ```c++ 1132 | inline int get() { 1133 | return 1; 1134 | } 1135 | ``` 1136 | 1137 | This asks the compiler to [inline-expand](#inline-expands) your code. 1138 | 1139 | In addition the `inline` keyword has to be used for [definitions](#definition) in headers to not violate the [One-Definition Rule (ODR)](#one-definition-rule): 1140 | 1141 | Learn more at 1142 | 1143 | ### one-definition rule 1144 | 1145 | The `inline` keyword has to be used for [definitions](#definition) in headers to not violate the One-Definition Rule (ODR): 1146 | 1147 | ```c++ 1148 | // myheader.hpp 1149 | 1150 | inline int get() { 1151 | return 1; 1152 | } 1153 | 1154 | template int put(int x) { 1155 | return x + 1; 1156 | } 1157 | ``` 1158 | 1159 | The definition for `get` in the header needs `inline` - not for [performance](#performance) reasons - but to make sure multiple translation units do not clash exposing the same definition. 1160 | 1161 | Except for fully specialized templates (`template <>`) templates are `inline` by default and do not require the `inline` keyword when defining them in headers. 1162 | It's a good practice to just always specify the `inline` keyword for definitions in headers. 1163 | 1164 | Learn more at 1165 | 1166 | ### inline-expands 1167 | 1168 | Also know as "inlined", this describes when a compiler moves code such that "the function’s code gets inserted into the caller’s code stream" from 1169 | 1170 | Inlining functions allows the compiler's optimizer to "see" across [translation unit](#translation-unit) boundaries and optimize the code in its larger context. 1171 | 1172 | This expansion happens during compilation so the impact of inlining is on the resultant [object files](#object-files) and binaries produced. The compiler does not rewrite [source code](#source-code). 1173 | 1174 | However, let's use [source code](#source-code) as a simple example of what inlining is about. 1175 | 1176 | Given the following code: 1177 | 1178 | ```c++ 1179 | inline int get() { 1180 | return 1; 1181 | } 1182 | 1183 | int main() { 1184 | int result = get(); 1185 | return result; 1186 | } 1187 | ``` 1188 | 1189 | The inline-expanded version would likely be seen/re-written internally by the compiler to be more like: 1190 | 1191 | ```c++ 1192 | int main() { 1193 | int result = 1; 1194 | return result; 1195 | } 1196 | ``` 1197 | 1198 | What the compiler does is hard to predict (its job is to optimize code in innovative ways!), so this is an incomplete example, but hints at the start of what may be happening behind the scenes. 1199 | You can use Clang's [`-Rpass`](https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports) switch to inspect what the inliner is doing and especially where it fails to inline. 1200 | 1201 | You could also imagine the compiler taking this example a step further and doing: 1202 | 1203 | ```c++ 1204 | int main() { 1205 | return 1; 1206 | } 1207 | ``` 1208 | 1209 | Because in our example it knows that `result` will always be `1`. 1210 | 1211 | ### inlined 1212 | 1213 | When a compiler [inline-expands](#inline-expands) a function. 1214 | 1215 | ### declaration 1216 | 1217 | A declaration represents signatures but not the actual implementation. 1218 | 1219 | A declaration for a function looks like: 1220 | 1221 | ```c++ 1222 | int get(); 1223 | ``` 1224 | 1225 | A declaration for a class looks like: 1226 | 1227 | ```c++ 1228 | class C; 1229 | ``` 1230 | 1231 | A class declaration (also called [forward declaration](#forward-declaration)) is an [incomplete type](http://en.cppreference.com/w/cpp/language/type#Incomplete_type). 1232 | 1233 | The primary use-case for incomplete types is [forward declaring](#forward-declaration) classes in header files to speed up compilation by hiding implementation details from users including the header. 1234 | 1235 | In contrast to a declaration a [definition](#definition) consists of the implementation. 1236 | 1237 | Lean more at 1238 | 1239 | ### definition 1240 | 1241 | A definition is code that describes the types an object like a function, class, or template. 1242 | 1243 | A definition for a function looks like: 1244 | 1245 | ```c++ 1246 | int get() { 1247 | return 1; 1248 | } 1249 | ``` 1250 | 1251 | Each definition is a [declaration](#declaration). 1252 | 1253 | When code is designed to be a [precompiled library](#precompiled-library) the definition is often included on its own in a header file (.hpp) and the [implementation](#implementation) is put in a `.cpp` file. 1254 | 1255 | It is also valid to have the entire [implementation](#implementation) in the header file. When only this is done then the library is called a [header-only library](#header-only-library) 1256 | 1257 | Learn more at: 1258 | 1259 | - 1260 | - 1261 | 1262 | ### Forward declaration 1263 | 1264 | Refers to when you provide a [declaration](#declaration) of a [symbol](#symbol) without a definition. 1265 | 1266 | Learn more at: 1267 | 1268 | - 1269 | 1270 | ### implementation 1271 | 1272 | An implementation is the code for a function inside the `{ ... }`. So, for example, an implementation of a function looks like: 1273 | 1274 | ```c++ 1275 | int get() { 1276 | return 1; 1277 | } 1278 | ``` 1279 | 1280 | When the implementation is in a .cpp file it will be compiled into an [executable](#executable) or a [precompiled library](#precompiled-library). 1281 | 1282 | When the implementation is in a .hpp file completely it is considered a [header-only library](#header-only-library). 1283 | 1284 | Read more at 1285 | 1286 | ### pointer 1287 | 1288 | . 1289 | 1290 | ### reference 1291 | 1292 | A reference in C++ is denoted by the `&` keyword when applied to a variable. When you dereference a [pointer](#pointer) you get a reference. Also when you pass an object "by reference" you are asking the object to not be copied. When you pass an object "by value" you are asking for the object to be copied. 1293 | 1294 | Learn more at . 1295 | 1296 | ### noexcept 1297 | 1298 | See 1299 | 1300 | ### bytes 1301 | 1302 | 1303 | [c++17] 1304 | 1305 | ### bits 1306 | 1307 | 1308 | 1309 | ### structure packing 1310 | 1311 | Learn more at 1312 | 1313 | ### sizeof 1314 | 1315 | Function built-in to C++ that can display the size, in [bytes](#bytes) of a type. 1316 | 1317 | 1318 | ### memory alignment 1319 | 1320 | [C++ objects and alignment](http://en.cppreference.com/w/c/language/object) 1321 | 1322 | See also 1323 | 1324 | ### statically typed 1325 | 1326 | When a type is enforced at compile time. 1327 | 1328 | Statically typed languages allows us to be much more descriptive. An example is [strong types](#strong-type). 1329 | 1330 | ### dynamically typed 1331 | 1332 | Languages is dynamically typed (or just dynamic) when type checking happens at run time. Example languages: Python, JavaScript. 1333 | 1334 | ### public member 1335 | 1336 | 1337 | 1338 | ### private member 1339 | 1340 | 1341 | 1342 | ### protected member 1343 | 1344 | 1345 | 1346 | ### class 1347 | 1348 | Same as a [struct](#struct) except all members are default private. 1349 | 1350 | ### struct 1351 | 1352 | Same as a [class](#class) except all members are default public. 1353 | 1354 | ### assembly 1355 | 1356 | The ASCII textual representation of [machine code](#machine-code). 1357 | It can be 1-to-1 translated to machine code, but is in text form 1358 | to provide a way to read the output of the [compiler](#compiler), after optimization, 1359 | to understand what compiler optimizations were made. 1360 | 1361 | (It can also be written manually instead of produced by a compiler, 1362 | but this is now rare.) 1363 | 1364 | ### machine code 1365 | 1366 | 1367 | 1368 | ### implicit conversion 1369 | 1370 | When a type is automatically converted to another type. This is often [a cause of bugs](https://github.com/mapbox/mapbox-gl-native/pull/5754) and a surprise to developers. 1371 | 1372 | To prevent this behavior use the [explicit keyword](#explicit). 1373 | 1374 | 1375 | 1376 | ### explicit 1377 | 1378 | 1379 | 1380 | ### explicit conversion 1381 | 1382 | 1383 | 1384 | ### strong type 1385 | 1386 | Creating a custom type to enforce correct usage of a function or class. 1387 | 1388 | When we write in a compiled language, like C++, our main target audience is the compiler. This is a very important concept to bear in mind. Second most important is fellow developers trying to understand what you have written years later and users downstream. If we can work well with the compiler, we can serve all audiences better. 1389 | 1390 | ### deterministic 1391 | 1392 | Definition of deterministic in English: 1393 | 1394 | "Relating to the philosophical doctrine that all events, including human action, are ultimately determined by causes regarded as external to the will. 1395 | ‘a deterministic theory’" 1396 | 1397 | In computer science, a deterministic algorithm is an algorithm which, given a particular input, will always produce the same output, with the underlying machine always passing through the same sequence of states. 1398 | 1399 | 1400 | 1401 | ### fits into register 1402 | 1403 | The "registers" are storage directly within the CPU rather than in main memory. They are faster than memory, 1404 | but few in number, so they are usually used (by the compiler's optimizer) for local variables and for passing 1405 | parameters when calling functions. 1406 | 1407 | If a data type fits into a register (i.e., it is a single number or a very small structure) it is as efficient 1408 | to pass it directly by value as it is to pass a pointer/reference to it, so it is usually okay and encouraged 1409 | to pass such data by value. 1410 | 1411 | Larger data structures that don't fit into a register must be copied into memory on the stack if they are 1412 | to be passed by value, so it is usually much faster to pass a reference to the structure rather than passing 1413 | the structure itself. 1414 | 1415 | ### constexpr 1416 | 1417 | The constexpr specifier declares that it is possible to evaluate the value of the function or variable at compile time. 1418 | 1419 | ### singleton 1420 | 1421 | A singleton is a instance of something that is created only once. When accessed the same instance is always returned. 1422 | 1423 | There are a variety of ways to implement a singleton. None of them are recommended unless you know what you are doing. 1424 | 1425 | Learn more [here](https://en.wikipedia.org/wiki/Singleton_pattern). 1426 | 1427 | ### static initialization 1428 | 1429 | Read about this at . 1430 | 1431 | ### global static 1432 | 1433 | A global static is a variable created in the global scope that points to [static data](#static-data). 1434 | 1435 | Generally it is bad practice to create global statics due to the [possibility of the static initialization fiasco](https://isocpp.org/wiki/faq/ctors#static-init-order), so only do this if you know what you are doing. 1436 | 1437 | They can be created like: 1438 | 1439 | ```c++ 1440 | #include 1441 | 1442 | static int one = 1; 1443 | 1444 | int main() { 1445 | std::cout << one << "\n"; 1446 | } 1447 | ``` 1448 | 1449 | ### static member 1450 | 1451 | A static member is [static data](#static-data) declared on a class that uses the `static` keyword. For example: 1452 | 1453 | #include 1454 | #include 1455 | 1456 | class HelloWorld { 1457 | public: 1458 | static int one_; 1459 | }; 1460 | 1461 | int HelloWorld::one_ = 1; 1462 | 1463 | int main() 1464 | { 1465 | std::cout << "One: " << HelloWorld::one_ << "!\n"; 1466 | } 1467 | 1468 | Using the `static` keyword on a class member allows the member to be accessed without creating an instance of that class. So, that member is not "bound" to a class instance. 1469 | 1470 | Read more at . 1471 | 1472 | ### static method 1473 | 1474 | A static method is a function defined on a class that uses the `static` keyword. For example: 1475 | 1476 | ```cpp 1477 | #include 1478 | #include 1479 | 1480 | class HelloWorld { 1481 | public: 1482 | static std::string hello() { 1483 | return "world"; 1484 | } 1485 | }; 1486 | 1487 | int main() 1488 | { 1489 | std::cout << "Hello " << HelloWorld::hello() << "!\n"; 1490 | } 1491 | ``` 1492 | 1493 | Using the `static` keyword on a class method allows the method to be called without creating an instance of that class. So, that method is not "bound" to a class instance. 1494 | 1495 | Read more at . 1496 | 1497 | ### static data 1498 | 1499 | Data created statically. In short this means a variable created with the static keyword like: 1500 | 1501 | ```c++ 1502 | static int one = 1; 1503 | ``` 1504 | 1505 | The longer explanation is that it is data created by [static initialization](#static-initialization). This basically means that the variable is created at the time the program starts and therefore is available for the whole time the program is running (aka "static storage duration"). 1506 | 1507 | Often static is used in combination with [constexpr](#constexpr) to ask for a variable to have static storage duration and have its value computed at compile time. 1508 | 1509 | ### macro 1510 | 1511 | 1512 | 1513 | ## Node.js & C++ 1514 | 1515 | ### Node 1516 | 1517 | Node.js is a JavaScript runtime built on Chrome's [V8](#v8) JavaScript engine. 1518 | 1519 | It is a command line tool that can execute Javascript files and C++ code compiled as [a node addon](#node-addon). 1520 | 1521 | Node builds on top of [V8](#v8) by providing ways to do evented, non-blocking [I/O](#io). What this means in practice is that node runs your code on an [event loop](#event-loop) and provides an API for deferring work through asynchronous function callbacks which execute in a [threadpool](#threadpool). This "work" might be traditional [I/O](#io) like accessing the filesystem or a data on a network, or it could be any kind of intensive computations. 1522 | 1523 | This functionality in node is internally using a C library called [libuv](#libuv). 1524 | 1525 | ### V8 1526 | 1527 | V8 is a Javascript "engine", or a Javascript interpreter. It translates Javascript into more efficient machine code (native assembly code), then executes it. V8 gives developers access to functionality (networking, DOM handling, external events, HTML5 video, canvas and data storage) needed to control the web browser. V8 does not allow access to the filesystem (or provide other features you'd normally associate with a scripting language used on the serverside). Therefore [Node](#node) implements features like filesystem access on top of V8. 1528 | 1529 | V8 is [open source and written in C++](https://github.com/v8/v8). 1530 | 1531 | ### Node Addon 1532 | 1533 | Node provides some "addons" built in, like the `zlib` module, which is implemented [in C++ inside node core](https://github.com/nodejs/node/blob/df63e534584a54dcf02b37446e1e821382e3cef3/src/node_zlib.cc). 1534 | 1535 | You can also write your own addon and include it dynamically within your application. 1536 | 1537 | See [node addon docs for more details](https://github.com/mapbox/cpp/blob/master/node-cpp.md) 1538 | 1539 | ### Event loop 1540 | 1541 | The event loop is what allows Node.js to perform non-blocking [I/O](#io) operations — despite the fact that JavaScript is single-threaded — by offloading operations to the system kernel whenever possible. 1542 | 1543 | The event loop runs in the main process (or main thread) and responds to an event queue: grabs the top item in the event queue, executes it, and then grabs the next item. 1544 | 1545 | Learn more at 1546 | 1547 | ### Threadpool 1548 | 1549 | In Node.js this is a queue of threads - of fixed size - that are launched at started and ready to execute work. 1550 | 1551 | To learn how to control the threadpool size with `UV_THREADPOOL_SIZE` and much more see the [node addon docs](https://github.com/mapbox/cpp/blob/master/node-cpp.md) 1552 | 1553 | ### libuv 1554 | 1555 | A library that provides a cross-platform API for asynchronous [I/O](#io). Most importantly libuv provides access, for [node addons](#node-addon), to the [Node Threadpool](#threadpool). 1556 | 1557 | Libuv stands for "lib-ultra-event" and [is open source and written in C](http://libuv.org) 1558 | 1559 | See [node addon docs for more details](https://github.com/mapbox/cpp/blob/master/node-cpp.md) 1560 | -------------------------------------------------------------------------------- /node-cpp.md: -------------------------------------------------------------------------------- 1 | # Node.js C++ Addons 2 | 3 | * [Why create a node addon?](#why-create-a-node-addon) 4 | * [How is a Node addon different than a C++ project?](#how-is-a-node-addon-different-than-a-c-project) 5 | * [Native Abstractions for Node.js (NAN)](#native-abstractions-for-nodejs-nan) 6 | * [Examples](#examples) 7 | * [Developing addons](#developing-addons) 8 | * [Where do I include other C++ libraries?](#where-do-i-include-other-c-libraries) 9 | * [Versioning](#versioning) 10 | * [Additional Resources](#additional-resources) 11 | 12 | The following document outlines Mapbox's general approach to writing C++ modules for [Node.js](https://github.com/mapbox/cpp/blob/master/glossary.md#node) (often referred to as _addons_), and the _why_. Check out [node-cpp-skel](https://github.com/mapbox/node-cpp-skel), a skeleton library for creating a Node.js addon, to learn more about _how_ to create an addon. When we hit bottlenecks in JS/Node.js scripts that can be solved by high concurrency we will consider porting node modules to C++ to leverage the threadpool - https://www.joyent.com/blog/node-js-on-the-road-dc-young-hahn 13 | 14 | Node is integral to the Mapbox APIs. Sometimes at scale, though, Node becomes a bottleneck for performance. Node is single-threaded, which blocks execution. C++ on the other hand allows you to execute operations without clogging up the event loop (learn more about the node event loop [here](https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/)). Passing heavy operations into C++ and subsequently into C++ workers can greatly improve the overall runtime of the code. 15 | 16 | ### Why create a node addon? 17 | 18 | 1. To port a C++ project to Node to expose a new interface for the tool (like Mapnik & Node Mapnik) 19 | 1. Improve performance at scale where Node becomes the bottleneck (i.e. concurrency) 20 | 21 | **Concurrency** 22 | 23 | Concurrency is the process of executing different pieces of the same process to allow for parallel execution of these pieces [[wikipedia](https://en.wikipedia.org/wiki/Concurrency_(computer_science))]. Node.js addons allow us to take advantage of concurrent operations within our node applications by reaching into more than just the [v8](https://github.com/mapbox/cpp/blob/master/glossary.md#v8) thread, but this can result in a few surprises. In the example below, we’ve passed data from our node application, into the v8 thread, and subsequently into our worker threadpool. Here there are multiple workers executing code at the same time. In our worker, we have the following line: 24 | 25 | ``` 26 | std::cout << "royal with cheese" << std::endl; 27 | ``` 28 | 29 | This will print `royal with cheese` to the terminal. In an application running this multiple times we’d expect the output to look like this: 30 | 31 | ``` 32 | royal with cheese 33 | royal with cheese 34 | royal with cheese 35 | royal with cheese 36 | // ... and so on 37 | ``` 38 | 39 | ![not concurrent](https://mapbox.s3.amazonaws.com/cpp-assets/addon-hey-nonconcurrent.gif) 40 | 41 | But when we start printing from within the threadpool all of these can start executing simultaneously. 42 | 43 | ![concurrent](https://mapbox.s3.amazonaws.com/cpp-assets/addon-hey-concurrent.gif) 44 | 45 | When we run the script again from within the threadpool, here's the output: 46 | 47 | ``` 48 | royal with cheese 49 | royraolry roawoyliya talwhl i wtcwihhit ethceh hs ceeche 50 | heseeeres 51 | oseye 52 | r 53 | aolry oawrylioa tylwha i lwtc ihhwt eihcet hshcee he 54 | cesheeres 55 | oeeys 56 | raeol 57 | ry oawrylioa tylwha i lwtc ihhwt eihcet hshcee he 58 | cesheeres 59 | oeeys 60 | raeol 61 | ry oawrylioa tylwha i lwtc ihhwt eihcet hshcee he 62 | cesheeres 63 | oeeys 64 | raeol 65 | ``` 66 | 67 | That’s a messy burger! 🍔 std::cout is logging at the same time and space is being filled concurrently. We’re literally seeing concurrency happen here–awesome! This makes testing pretty hard though. There are a few options to get your logs in order, such as adding a [mutex](http://en.cppreference.com/w/cpp/thread/mutex) to keep things straight. Since we’re just testing, we can tell our computer to only run this script with a single thread - forcing our application to run non-concurrently. We pass the following before running our script: 68 | 69 | ```shell 70 | UV_THREADPOOL_SIZE=1 node index.js 71 | ``` 72 | 73 | And the output is 74 | 75 | ``` 76 | royal with cheese 77 | royal with cheese 78 | royal with cheese 79 | royal with cheese 80 | ``` 81 | 82 | ### How is a Node addon different than a C++ project? 83 | 84 | A Node.js addon is still a Node module. Users still interact with it as if they are writing Javascript (i.e. `var awesome = require('awesome')`), but the library will tend to pass much of the logic into C++ workers, which are highly performant, then return information back into a javascript interface. Bottom line, the user of your library never has to write or interact with C++. 85 | 86 | ### Native Abstractions for Node.js (NAN) 87 | 88 | To swing between Node and C++, the Node community maintains a project called [_NAN_](https://github.com/nodejs/nan) (Native Abstractions for Node.js) that simplifies running different versions of Node and, subsequently, v8. NAN is a header-only C++ library that provides a set of Macros for developing Node.js addons. Check out the [usage](https://github.com/nodejs/nan#usage) guidelines. 89 | 90 | More examples of how to port C++ libraries to node can be found at [nodejs.org/api/addons](https://nodejs.org/api/addons.html). See https://nodesource.com/blog/c-add-ons-for-nodejs-v4/ for a detailed summary of the origins of Nan. And see http://blog.reverberate.org/2016/10/17/native-extensions-memory-management-part2-javascript-v8.html for a good introduction to v8 that is not specific to node.js. 91 | 92 | ### Examples 93 | 94 | All of the following libraries are installable in a Node.js environment, but execute much of their logic in C++: 95 | 96 | * [Node Mapnik](https://github.com/mapnik/node-mapnik) - creating map tiles 97 | * [Node OSRM](https://github.com/Project-OSRM/node-osrm) - directions & routing 98 | * [sqlite](https://github.com/mapbox/node-sqlite3) - asynchronous, non-blocking SQLite3 bindings 99 | * [Node GDAL](https://github.com/naturalatlas/node-gdal) - geographic operations 100 | * [vtquery](https://github.com/mapbox/vtquery) - query vector tiles 101 | * [vtinfo](https://github.com/mapbox/vtinfo) - reads and returns general information about a vector tile buffer 102 | 103 | ### Developing addons 104 | 105 | Developing an addon requires Node.js, NPM, and a C++ compiler. Check out node-cpp-skel's ["extended tour"](https://github.com/mapbox/node-cpp-skel/blob/master/docs/extended-tour.md) for an up-to-date and opinionated approach to developing an addon. Additinally, the repository has docs on [running benchmarks](https://github.com/mapbox/node-cpp-skel/blob/master/docs/benchmarking.md), [publishing binaries](https://github.com/mapbox/node-cpp-skel/blob/master/docs/publishing-binaries.md), and a breakdown of all the [necessary components](https://github.com/mapbox/node-cpp-skel/blob/master/docs/extended-tour.md#configuration-files) to make it run. 106 | 107 | ### Where do I include other C++ libraries? 108 | 109 | One big bonus of developing a Node.js addon is that you can include other C++ code in your project, even if this code wasn't intended to be used via a Node.js interface. C++ headers can be installed in a few ways: 110 | 111 | * Installed via [Mason](https://github.com/mapbox/cpp/blob/master/glossary.md#mason): use Mason to install a project, these can be installed into whichever folder you choose to host dependencies. Best practice is a `/deps` directory. 112 | * Copied/pasted into a `/deps` directory (this is also referred to as "vendoring") 113 | * Installed via NPM: Publishing headers to NPM allows them to be included in addons easily, since we are already using the NPM ecosystem. Header paths will point to the `/node_modules` folder or can include dynamically with an [`include_dirs.js`](https://github.com/mapbox/protozero/blob/master/include_dirs.js) file. **Note: this practice is no longer recommended.** 114 | 115 | Depending on how a project is installed, the path to the header files will be different. These paths can be added to the `binding.gyp` file and will look like this: 116 | 117 | ```javascript 118 | { 119 | 'includes': [ 'common.gypi' ], 120 | 'targets': [ 121 | { 122 | 'include_dirs': [ 123 | '