├── .gitignore ├── LICENSE ├── README.md ├── include └── llcpp │ ├── detail │ ├── config.hpp │ ├── format_parser.hpp │ ├── log_line.hpp │ ├── logging.hpp │ ├── prefix.hpp │ ├── string_format.hpp │ ├── terminators.hpp │ ├── udl.hpp │ └── utils.hpp │ └── llcpp.hpp └── scripts └── llcpp_parse.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Prerequisites 2 | *.d 3 | 4 | # Compiled Object files 5 | *.slo 6 | *.lo 7 | *.o 8 | *.obj 9 | 10 | # Precompiled Headers 11 | *.gch 12 | *.pch 13 | 14 | # Compiled Dynamic libraries 15 | *.so 16 | *.dylib 17 | *.dll 18 | 19 | # Fortran module files 20 | *.mod 21 | *.smod 22 | 23 | # Compiled Static libraries 24 | *.lai 25 | *.la 26 | *.a 27 | *.lib 28 | 29 | # Executables 30 | *.exe 31 | *.out 32 | *.app 33 | 34 | # VSCode 35 | .vscode 36 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 blapid 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # llcpp 2 | 3 | Literal Logging for C++ 4 | 5 | ## TL;DR 6 | 7 | If the requirement of outputting a textual log from the C++ application is relaxed, we can create a faster logging framework. Enter `llcpp` (literal logging for c++). This framework uses C++17 template (and constexpr) meta-programming magic to reduce the number of allocations needed to zero and eliminates the need to format the data to human readable form. Later, a simple script can be used to transform the log into human readable form. Oh, and we get type safety for free. 8 | 9 | Intrigued? skeptical? excited? angry? Read on! 10 | 11 | ## Status 12 | 13 | This library is still in PoC mode. Feel free to try it and hack on it, but I can't promise a smooth ride just yet! 14 | 15 | Also, I'll appreciate any feedback you may have :) 16 | 17 | ## Contents 18 | 19 | - [Features](#features) 20 | - [Not textual?!](#not-textual) 21 | - [Benchmarks](#benchmarks) 22 | - [Example](#example) 23 | - [Design Overview](#design-overview) 24 | - [Extending](#extending) 25 | - [Blog post](#blog-post) 26 | 27 | ## Features 28 | 29 | - Fast: Low call site latency, low total CPU usage. See [benchmarks](#benchmarks) 30 | - `printf`-like format syntax. 31 | - Type safe: Mixing format type specifiers and arguments result in compilation error. 32 | - No allocation for any log line. 33 | - Header only. 34 | - Available format specifiers: %d, %u, %x, %s. More coming soon. 35 | - Prefixes available for line structure: Log level indication, Date/Time, Nanosecond timestamp. 36 | - Synchronous: No independant state, when call returns the line has been written. May add async features in the future. 37 | - Caveat: Output is *not* textual. See [this](#not-textual). Parser written in python is implemented and will output the textual log. 38 | - Caveat: Requires C++17. Tested on Clang 5.0.1 (Ubuntu) and Clang 900 (MacOS). 39 | - Caveat: Requires a user-defined string literal GNU extension (`template operator""`). Can be substituted with macro magic such as [this](https://github.com/irrequietus/typestring) if needed. 40 | 41 | ## Not Textual?! 42 | 43 | Yeah! Outputting textual logs is one of those "hidden" constrainsts/assumptions that no one talks about. If you think about it, you hardly ever read your logs directly. They're usually pulled and aggregated from your servers and go through a whole pipeline, adding a parser is no big deal. 44 | 45 | Once you free yourself from this constraint, it's possible to implement a logging framework thats even faster than the current state of the art. How much faster you ask? See the [benchmarks](#benchmarks). 46 | 47 | ## Benchmarks 48 | 49 | Make sure you read [my post]() on benchmarking logging frameworks to get the whole story. 50 | 51 | The following benchmark were run on Ubuntu 16.04.03. Compiled with Clang 5.0.1 with `--std=c++1z -flto -O3` flags. `/tmp` is mounted on an SSD. llcpp is presented with both timestamp prefix and date/time prefix (which is slower due to `localtime()`). 52 | 53 | (time in **nanoseconds** unless stated otherwise) 54 | 55 | ### 1 Thread Logging To `/tmp` 56 | 57 | |Logger| 50th| 75th| 90th| 99th| 99.9th| Worst| Average | Total Time (seconds)| 58 | |:----:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:---------------:| 59 | |llcpp (timestamp) | 165| 171| 211| 2058| 6313| 149432|230.360544| 0.42 | 60 | |llcpp (date/time) | 184| 195| 216| 2153| 3718| 1099523|260.166040| 0.53 | 61 | |mini-async-logger | 286| 335| 361| 1348| 2884| 6204357|324.418935| 1.32 | 62 | |NanoLog | 211| 248| 291| 1099| 3809| 2493347|798.895457| 3.01 | 63 | |spdlog | 729| 751| 780| 973| 3277| 1057879|744.825963| 0.95 | 64 | |g3log | 2300| 2819| 3230| 5603| 28176| 2249642|2490.365190| 3.86 | 65 | 66 | ### 4 Threads Logging To `/tmp` 67 | 68 | |Logger| 50th| 75th| 90th| 99th| 99.9th| Worst| Average | Total Time (seconds)| 69 | |:----:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:---------------:| 70 | | llcpp (timestamp) | 173| 191| 242| 7501| 36031| 2668978|458.101174| 1.61 | 71 | |llcpp (date/time) | 220| 227| 270| 15515| 46522| 2162221|567.410498| 1.69 | 72 | |mini-async-logger | 414| 581| 863| 2872| 24384| 5858754|612.069366| 5.52 | 73 | |NanoLog | 338| 404| 1120| 1859| 24906| 1599487|845.320061| 12.81 | 74 | |spdlog | 1100| 1193| 1238| 1373| 14022| 1312104|1032.874598| 1.91 | 75 | |g3log | 1904| 2452| 3171| 9600| 37283| 6174296|2428.927793| 16.29 | 76 | 77 | #### Worst case measurements 78 | 79 | See my blog post for some discussion over worst-case measurements. 80 | 81 | ## Example 82 | 83 | ```c++ 84 | int main(int argc, char* argv[]) 85 | { 86 | using conf_t = llcpp::default_config; 87 | using prefix_tuple_t = std::tuple; 88 | using logger_t = llcpp::file_logger; 89 | auto logger = logger_t("./log.txt"); 90 | // Or: auto logger = llcpp::default_logger("./log.txt"); 91 | 92 | logger.info("llcpp message #%d : This is some text for your pleasure. %s"_log, 0, "/dev/null"); 93 | logger.info("llcpp message #%d : This is some text for your pleasure. %d %s %ld %llx. end."_log, 2, 42, "asdf", 24, 50); 94 | return 0; 95 | } 96 | ``` 97 | 98 | ## Binary Log Format 99 | 100 | The binary format is pretty straightforward: write the null-terminated format string, then write the in-memory binary representation of each of the arguments. For arithmetic types (ints, floats, etc.), the size of each binary representation is derived from the format string (`%d` will be 4 bytes, `%lld` will be 8 bytes, etc.). For strings, we have two options: writing the length and then the string or passing information about the maximum length in the format string and then copying the minimum between the given maximum and the string's actual length. 101 | 102 | Examples: 103 | 104 | - `logger.info("Hello world"_log)` - would simply be written as the null terminated string, as you'd expect. 105 | - `logger.info("Hello world %d"_log, 42)` - would write the null terminated string, followed by 4 bytes representing `42` (little endian). 106 | - `logger.info("Hello world %lld"_log, 42)` - would write the null terminated string, followed by 8 bytes representing `42` (little endian). 107 | - `logger.info("Hello world %s"_log, "42")` - would write the null terminated string, followed by 4 bytes representing `2` (the length of `"42"`) and 2 bytes with the actual string `"42"` 108 | - `logger.info("Hello world %8s"_log, "42")` - would write the null terminated string, followed by 8 bytes, of which the first two would hold the string `"42"`. If the string given in the argument is bigger, it would be truncated. 109 | 110 | ## Design Overview 111 | 112 | - User defined string literals are used to capture log format into variadic template parameters. 113 | - `string_format` - Is specialized and holds a static array of characters representing the format string. 114 | - `format_parser` - Parses the format string in compile time, generating a tuple of `argument_parser`'s which correspond to the deduced arguments from the format. 115 | - `terminators` - Are a collection of types that correspond to printf format type identifiers (such as `'d'`, `'s'`, `'u'`, etc.). These are used by the `format_parser` to deduce the types of the escape sequences in the format string. These also provide a specific `argument_parser` which will be aggregated by the `format_parser`. 116 | - `log_line` - Uses the `string_format` and `format_parser` to serialize the arguments to the log file. Exposes a `operator()` function which takes variadic arguments which are validated with the `argument_parser`'s tuple provided by the `format_parser`. 117 | - `logging` - Provide an interface for writing log parts to an output sink (file, network, etc.). Also handle `prefix`'es - a way to add structured data to your log lines (log level indication, timestamp, etc.). Also provide the high level (`info()`, `warn()`, etc.) functions. 118 | - `config` - Allows the user to override internal classes such as `string_format`, `format_parser` and built-in `terminator` list for easy customization. 119 | 120 | ## Extending 121 | 122 | ### Adding loggers: 123 | 124 | - Loggers *must* use the [CRTP](https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern) and inherit from `logger_base`. 125 | - Loggers *must* implement the following functions: 126 | - `void line_hint_impl()` - Called by `logger_base` when an entire log line was written. 127 | - `void write_impl(const std::uint8_t *data, const std::size_t len)` - Called by the serialization code for each log line part. 128 | - Loggers *should* let the user pass a PrefixTuple and Config by template parameters, and pass it to `logger_base`. 129 | 130 | Example: 131 | 132 | ```c++ 133 | template 134 | struct vector_logger : public logger_base, Config> { 135 | 136 | vector_logger(std::vector& vec, PrefixTuple&& prefix_tuple) : m_vec(vec), 137 | logger_base>(std::forward(prefix_tuple)) 138 | { 139 | } 140 | 141 | void line_hint_impl() { 142 | } 143 | 144 | void write_impl(const std::uint8_t *data, const std::size_t len) { 145 | m_vec.insert(m_vec.end(), data, data+len); 146 | } 147 | 148 | private: 149 | std::vector& m_vec; 150 | }; 151 | ``` 152 | 153 | ### Adding terminators 154 | 155 | - Terminators *must* inherit from `terminator` where X is the character representing the type identifier in the format string (i.e. `'d'`, `'s'`, etc.). 156 | - Terminators *must* not interfere with other terminators. For example: Adding an `'l'` terminator would interfere with the `'l'` options of the `'d'` terminator (think about `'%lld'`). 157 | - Terminators must declare `template 158 | struct argument_parser` with the following static members and functions: 159 | - `static constexpr std::size_t argument_size` - The size used when serializing this argument 160 | - `using argument_type` - The C++ type that this `argument_parser` expect to receive from the caller when serializing. 161 | - `static constexpr bool is_fixed_size` - False if this `argument_parser` cannot know in compile-time the size that will be needed during serialization. 162 | - `template static void apply(Logger& logger, const Arg arg)` - The serialization function. It *should* use `logger.write()` to write the serialized data. 163 | - `template static void apply_variable(Logger& logger, const char *arg)` - An auxiliary function that will be called if `is_fixed_size` is false. It *should* be used to write auxiliary serialization data if necessary. 164 | - Terminators *should* be declared in the `llcpp::user_terminators` namespace. 165 | 166 | Example: 167 | 168 | ```c++ 169 | namespace llcpp::user_terminators { 170 | struct d : public terminator 171 | { 172 | template 173 | struct argument_parser 174 | { 175 | static constexpr bool is_fixed_size = true; 176 | static constexpr std::size_t num_of_l = tuple_counter::count; 177 | static_assert(num_of_l < 3, "Invalid conversion specifier"); 178 | static constexpr std::size_t argument_size = ((num_of_l == 0) ? (sizeof(std::uint32_t)) : (sizeof(std::uint32_t) * num_of_l)); 179 | using argument_type = typename std::conditional< 180 | num_of_l == 0, 181 | int, 182 | typename std::conditional< 183 | num_of_l == 1, 184 | long, 185 | long long>::type>::type; 186 | 187 | template 188 | static void apply(Logger& logger, const Arg arg) 189 | { 190 | static_assert(std::is_integral::value, 191 | "Integral argument's apply function called with non-integral value"); 192 | static_assert(argument_size >= sizeof(Arg), 193 | "Discrepency detected between parsed argument_size and size of given arg, " 194 | "you may need a specialized argument_parser"); 195 | logger.write((std::uint8_t *)&arg, argument_size); 196 | } 197 | 198 | template 199 | static void apply_variable(Logger& logger, const char *arg) {} 200 | }; 201 | }; 202 | } 203 | ``` 204 | 205 | ### Adding prefixes 206 | 207 | - Prefixes *must* implement `template 208 | void apply(logging::level::level_enum level, Logger& logger)`. This function should use the given logger to write the prefix as needed. 209 | 210 | Example: 211 | 212 | ```c++ 213 | struct nanosec_time_prefix : public prefix_base { 214 | template 215 | void apply(typename logging::level::level_enum level, Logger& logger) { 216 | auto now = std::chrono::system_clock::now(); 217 | auto now_ns = std::chrono::duration_cast(now.time_since_epoch()).count(); 218 | "[%llu]: "_log(logger, now_ns); 219 | } 220 | }; 221 | ``` 222 | 223 | ### Changing the configuration 224 | 225 | You may extend the default configuration using the `config_with_XXX` declarations. 226 | Example: 227 | 228 | ```c++ 229 | using conf_t = llcpp::default_config::config_with_additional_terminators>; 230 | 231 | using prefix_tuple_t = std::tuple>; 232 | using logger_t = llcpp::file_logger; 233 | ``` 234 | 235 | ## Blog post 236 | 237 | For more information and background, head over to my blog posts [here](https://blapid.github.io/cpp/2017/10/31/llcpp-a-quest-for-faster-logging-intro.html)! 238 | -------------------------------------------------------------------------------- /include/llcpp/detail/config.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | #include 3 | #include "utils.hpp" 4 | #include "terminators.hpp" 5 | #include "log_line.hpp" 6 | 7 | namespace llcpp::detail::config { 8 | struct base_config { 9 | // void or type to override 10 | template 11 | using format_parser = format_parser::format_parser; 12 | template 13 | using string_format = string_format::string_format; 14 | /* 15 | * 1. Be careful when overriding or adding terminators. The parser makes decision as it passes the format character by character. 16 | * Certain terminator combination may lead to unwanted results. For example, using both the 'd' terminator and a new terminator 17 | * that expects to end with 'l' will not handle "%lld" correctly. 18 | * 2. Make sure that the `terminator tuple` or `additional terminators` are std::tuples 19 | */ 20 | using terminator_tuple = terminators::builtin_terminator_tuple; 21 | using additional_terminators = std::tuple<>; 22 | 23 | }; 24 | 25 | template 26 | struct _config_with_format_parser : public Base { 27 | using format_parser = FormatParser; 28 | }; 29 | template 30 | struct _config_with_string_format : public Base { 31 | using string_format = StringFormat; 32 | }; 33 | template 34 | struct _config_with_terminator_tuple : public Base { 35 | using terminator_tuple = TerminatorTuple; 36 | }; 37 | template 38 | struct _config_with_additional_terminators : public Base { 39 | using additional_terminators = AdditionalTerminators; 40 | }; 41 | 42 | template 43 | struct config : public Base { 44 | template 45 | using config_with_format_parser = config<_config_with_format_parser>; 46 | template 47 | using config_with_string_format = config<_config_with_string_format>; 48 | template 49 | using config_with_terminator_tuple = config<_config_with_terminator_tuple>; 50 | template 51 | using config_with_additional_terminators = config<_config_with_additional_terminators>; 52 | }; 53 | 54 | using default_config = config; 55 | } -------------------------------------------------------------------------------- /include/llcpp/detail/format_parser.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "terminators.hpp" 4 | #include "utils.hpp" 5 | 6 | namespace llcpp::detail::format_parser { 7 | 8 | template 9 | struct format_parser_state { 10 | using is_escaped = std::integral_constant; 11 | using escape_idx = std::integral_constant; 12 | }; 13 | 14 | template 15 | struct format_parser_impl { 16 | static_assert(utils::is_specialization_of::value, "CharTuple must be a tuple"); 17 | using config_t = Config; 18 | using cur_char = typename std::tuple_element::type; 19 | using is_escape_char = typename std::conditional< 20 | cur_char::value == '%', 21 | std::true_type, 22 | std::false_type>::type; 23 | 24 | using escape_termination_t = typename terminators::terminator_container::template search_terminators_t; 25 | using is_escape_termination = typename terminators::terminator_container::template is_terminator; 26 | using is_next_state_escaped = typename std::conditional< 27 | (State::is_escaped::value && !is_escape_termination::value) || 28 | (!State::is_escaped::value && is_escape_char::value), 29 | std::true_type, 30 | std::false_type>::type; 31 | using escape_idx = typename std::conditional< 32 | !State::is_escaped::value && is_escape_char::value, 33 | std::integral_constant, 34 | typename State::escape_idx>::type; 35 | 36 | 37 | using current_argument_parser = typename utils::type_if_or< 38 | State::is_escaped::value, 39 | typename escape_termination_t::template argument_parser, 40 | typename escape_termination_t::empty_parser 41 | >::type; 42 | 43 | using current_argument_tuple = typename std::conditional_t< 44 | (current_argument_parser::argument_size > 0), 45 | std::tuple, 46 | typename std::tuple<> 47 | >; 48 | 49 | using arguments_tail = 50 | typename format_parser_impl< 51 | config_t, 52 | CharTuple, 53 | CharTupleSize, 54 | Idx + 1, 55 | format_parser_state< 56 | is_next_state_escaped::value, 57 | escape_idx::value 58 | > 59 | >::argument_tuple; 60 | using argument_tuple = utils::tuple_cat_t; 61 | }; 62 | template 63 | struct format_parser_impl { 64 | using argument_tuple = std::tuple<>; 65 | }; 66 | 67 | template 68 | struct format_parser { 69 | using config_t = Config; 70 | using format_size = std::integral_constant; 71 | using format_tuple = std::tuple::type...>; 72 | using parser_impl = format_parser_impl>; 73 | 74 | using argument_tuple = typename parser_impl::argument_tuple; 75 | 76 | 77 | template 78 | struct sum_arguments_size_impl { 79 | static constexpr std::size_t value = 80 | std::tuple_element::type::argument_size + 81 | sum_arguments_size_impl::value; 82 | }; 83 | template 84 | struct sum_arguments_size_impl { 85 | static constexpr std::size_t value = 0; 86 | }; 87 | 88 | struct sum_arguments_size { 89 | static constexpr std::size_t value = sum_arguments_size_impl<0, std::tuple_size::value>::value; 90 | }; 91 | }; 92 | 93 | template 94 | struct accumulate_argument_parser_size { 95 | static_assert(Idx <= std::tuple_size_v, "Idx is out of bounds"); 96 | using argument_parser = typename std::tuple_element_t; 97 | static constexpr std::size_t value = argument_parser::argument_size + 98 | accumulate_argument_parser_size::value; 99 | }; 100 | template 101 | struct accumulate_argument_parser_size { 102 | static constexpr std::size_t value = 0; 103 | }; 104 | 105 | template 106 | struct count_variable_arguments_impl { 107 | static_assert(Idx < std::tuple_size_v, "Idx is out of bounds"); 108 | using argument_parser = typename std::tuple_element_t; 109 | static constexpr std::size_t value = (!argument_parser::is_fixed_size) ? (1) : (0) + 110 | count_variable_arguments_impl::value; 111 | }; 112 | template 113 | struct count_variable_arguments_impl { 114 | using argument_parser = typename std::tuple_element_t<0, ArgumentTuple>; 115 | static constexpr std::size_t value = (!argument_parser::is_fixed_size) ? (1) : (0); 116 | }; 117 | template> 118 | struct count_variable_arguments { 119 | static constexpr std::size_t value = 120 | count_variable_arguments_impl::value; 121 | }; 122 | template<> 123 | struct count_variable_arguments> { 124 | static constexpr std::size_t value = 0; 125 | }; 126 | 127 | } -------------------------------------------------------------------------------- /include/llcpp/detail/log_line.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "string_format.hpp" 4 | #include "format_parser.hpp" 5 | #include "config.hpp" 6 | 7 | namespace llcpp::detail::log_line { 8 | struct log_line_base {}; 9 | 10 | template 11 | struct log_line : public log_line_base { 12 | using config_t = Config; 13 | using string_format = typename config_t::template string_format; 14 | 15 | template 16 | using log_line_with_suffix = log_line; 17 | template 18 | using log_line_with_prefix = log_line; 19 | template 20 | using log_line_with_config = log_line<_Config, CharT, Chars...>; 21 | 22 | constexpr std::size_t size() const { 23 | return string_format::size(); 24 | } 25 | static constexpr std::size_t fmt_size() { 26 | return string_format::fmt_size(); 27 | } 28 | static constexpr std::size_t args_size() { 29 | return string_format::args_size(); 30 | } 31 | 32 | template 33 | void operator()(Logger& logger, Args&&... args) { 34 | using fmt_argument_tuple_size = std::tuple_size; 35 | static_assert(sizeof...(Args) == fmt_argument_tuple_size::value, 36 | "Discrepency between number of arguments in format and number of arguments in call"); 37 | logger.write((std::uint8_t *)string_format::_chars, fmt_size()); 38 | apply_args(logger, std::forward_as_tuple(args...)); 39 | } 40 | 41 | private: 42 | using argument_tuple = typename string_format::format_parser::argument_tuple; 43 | using num_variable_args = format_parser::count_variable_arguments; 44 | 45 | template::type> 46 | void apply_arg(Logger& logger, const Arg arg) { 47 | using argument_parser = typename std::tuple_element_t; 48 | using _expected_arg_type_from_format = typename argument_parser::argument_type; 49 | //Add const to allow const arguments to be passed 50 | using expected_arg_type_from_format = 51 | typename std::conditional_t< 52 | std::is_pointer_v<_expected_arg_type_from_format>, 53 | //HACK: for char pointers 54 | std::add_pointer_t>>, 55 | typename std::conditional_t< 56 | std::is_array_v<_expected_arg_type_from_format>, 57 | //HACK for char arrays 58 | std::add_pointer_t>>, 59 | std::add_const_t<_expected_arg_type_from_format> 60 | > 61 | >; 62 | 63 | static_assert(std::is_convertible::value, 64 | "Discrepency between argument type deduced from format and the given argument's type"); 65 | 66 | argument_parser::apply(logger, arg); 67 | }; 68 | 69 | template::type> 70 | void apply_arg_variable(Logger& logger, const Arg arg) { 71 | //No-Op for anything other than strings... 72 | } 73 | template::type> 74 | void apply_arg_variable(Logger& logger, const char * arg) { 75 | using argument_parser = typename std::tuple_element_t; 76 | 77 | if constexpr(!argument_parser::is_fixed_size) { 78 | argument_parser::apply_variable(logger, arg); 79 | } 80 | }; 81 | template 82 | void _apply_args(Logger& logger, ArgTuple args, std::index_sequence) { 83 | (apply_arg(logger, std::get(args)), ...); 84 | if constexpr(num_variable_args::value > 0) { 85 | (apply_arg_variable(logger, std::get(args)), ...); 86 | } 87 | } 88 | template 89 | void apply_args(Logger& logger, std::tuple args) { 90 | _apply_args(logger, std::move(args), std::index_sequence_for{}); 91 | } 92 | }; 93 | } -------------------------------------------------------------------------------- /include/llcpp/detail/logging.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #include "log_line.hpp" 9 | #include "config.hpp" 10 | 11 | namespace llcpp::detail::logging { 12 | struct level { 13 | enum level_enum : unsigned int { 14 | trace = 0, 15 | debug = 1, 16 | info = 2, 17 | warn = 3, 18 | err = 4, 19 | critical = 5, 20 | }; 21 | }; 22 | 23 | template 24 | struct logger_base { 25 | static_assert(utils::is_specialization_of::value, "PrefixTuple must be a tuple"); 26 | using config_t = Config; 27 | 28 | logger_base(PrefixTuple&& prefix_tuple = {}) : m_prefix_tuple(std::forward(prefix_tuple)) {} 29 | virtual ~logger_base() {} 30 | 31 | template 32 | void trace(LogLine&& line, Args&&... args) { 33 | log(level::trace, std::forward(line), std::forward(args)...); 34 | } 35 | template 36 | void debug(LogLine&& line, Args&&... args) { 37 | log(level::debug, std::forward(line), std::forward(args)...); 38 | } 39 | template 40 | void info(LogLine&& line, Args&&... args) { 41 | log(level::info, std::forward(line), std::forward(args)...); 42 | } 43 | template 44 | void warn(LogLine&& line, Args&&... args) { 45 | log(level::warn, std::forward(line), std::forward(args)...); 46 | } 47 | template 48 | void err(LogLine&& line, Args&&... args) { 49 | log(level::err, std::forward(line), std::forward(args)...); 50 | } 51 | template 52 | void critical(LogLine&& line, Args&&... args) { 53 | log(level::critical, std::forward(line), std::forward(args)...); 54 | } 55 | 56 | void write(const std::uint8_t *data, const std::size_t len) { 57 | static_cast(this)->write_impl(data,len); 58 | } 59 | void line_hint() { 60 | static_cast(this)->line_hint_impl(); 61 | } 62 | 63 | protected: 64 | template 65 | void apply_prefix_tuples(typename level::level_enum level, std::index_sequence) { 66 | (std::get(m_prefix_tuple).apply(level, *this), ...); 67 | } 68 | 69 | template 70 | void log(typename level::level_enum level, LogLine&& line, Args&&... args) { 71 | static_assert(std::is_base_of_v, "LogLine type error, did you mean to use _log?"); 72 | apply_prefix_tuples(level, std::make_index_sequence>{}); 73 | using log_line_t = typename LogLine::template log_line_with_config::template log_line_with_suffix<'\n'>; 74 | log_line_t _line; 75 | _line(*this, args...); 76 | line_hint(); 77 | } 78 | 79 | //Override these two functions in your derived logger 80 | void write_impl(const std::uint8_t *data, const std::size_t len) { 81 | } 82 | void line_hint_impl() { 83 | } 84 | 85 | PrefixTuple m_prefix_tuple; 86 | }; 87 | 88 | template 89 | struct file_logger : public detail::logging::logger_base, Config> { 90 | friend class detail::logging::logger_base, Config>; 91 | 92 | file_logger(std::string_view path, PrefixTuple&& prefix_tuple = {}) : m_fp(std::fopen(path.data(), "w"), std::fclose), 93 | logger_base, Config>(std::forward(prefix_tuple)) 94 | { 95 | //TODO: Check errors etc... 96 | } 97 | file_logger(std::FILE *fp, PrefixTuple&& prefix_tuple = {}) : m_fp(fp, {}), 98 | logger_base>(std::forward(prefix_tuple)) 99 | { 100 | //TODO: Check errors etc... 101 | } 102 | ~file_logger() { 103 | line_hint_impl(true); 104 | } 105 | 106 | void line_hint_impl(bool force_flush = false) { 107 | if (force_flush) { 108 | std::fwrite(m_cache, m_cache_count, 1, m_fp.get()); 109 | m_cache_count = 0; 110 | } 111 | } 112 | protected: 113 | void write_impl(const std::uint8_t *data, const std::size_t len) { 114 | if (len > sizeof(m_cache)) { 115 | // Flush current cache 116 | line_hint_impl(true); 117 | // Write everything else directly 118 | std::fwrite(data, len, 1, m_fp.get()); 119 | return; 120 | } 121 | if (m_cache_count + len > sizeof(m_cache)) { 122 | // TODO: Will probably be faster if we fill the cache, flush and then copy the rest. 123 | line_hint_impl(true); 124 | } 125 | std::memcpy(m_cache + m_cache_count, data, len); 126 | m_cache_count += len; 127 | } 128 | 129 | std::unique_ptr> m_fp; 130 | static constexpr std::size_t m_cache_size = 4 * 1024; 131 | static thread_local std::uint8_t m_cache[m_cache_size]; 132 | static thread_local std::size_t m_cache_count; 133 | }; 134 | 135 | template 136 | thread_local std::uint8_t file_logger::m_cache[m_cache_size] = {0}; 137 | template 138 | thread_local std::size_t file_logger::m_cache_count = 0; 139 | 140 | //XXX: Hack... 141 | template 142 | struct stdout_logger : public file_logger { 143 | stdout_logger(PrefixTuple&& prefix_tuple = {}) : 144 | file_logger(stdout, std::forward(prefix_tuple)) 145 | { 146 | } 147 | }; 148 | 149 | template 150 | struct vector_logger : public logger_base, Config> { 151 | 152 | vector_logger(std::vector& vec, PrefixTuple&& prefix_tuple = {}) : m_vec(vec), 153 | logger_base>(std::forward(prefix_tuple)) 154 | { 155 | } 156 | 157 | void line_hint_impl() { 158 | } 159 | 160 | void write_impl(const std::uint8_t *data, const std::size_t len) { 161 | m_vec.insert(m_vec.end(), data, data+len); 162 | } 163 | 164 | private: 165 | std::vector& m_vec; 166 | }; 167 | } -------------------------------------------------------------------------------- /include/llcpp/detail/prefix.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "udl.hpp" 7 | #include "logging.hpp" 8 | 9 | namespace llcpp::detail::prefix { 10 | 11 | struct prefix_base { 12 | virtual ~prefix_base() {} 13 | 14 | template 15 | void apply(logging::level::level_enum level, Logger& logger) {} 16 | }; 17 | 18 | template 19 | struct time_format_prefix : public prefix_base { 20 | std::tm localtime() 21 | { 22 | //TODO check errors 23 | std::time_t now_t = std::time(nullptr); 24 | std::tm tm; 25 | if constexpr (UseLocalTime) { 26 | localtime_r(&now_t, &tm); 27 | } else { 28 | gmtime_r(&now_t, &tm); 29 | } 30 | 31 | return tm; 32 | } 33 | 34 | template 35 | void apply(typename logging::level::level_enum level, Logger& logger) { 36 | auto lt = localtime(); 37 | "[%d-%3s-%d %d:%d:%d]: "_log(logger, lt.tm_year + 1900, m_month_strings[lt.tm_mon], lt.tm_mday, lt.tm_hour, lt.tm_min, lt.tm_sec); 38 | } 39 | 40 | static constexpr std::array m_month_strings = { 41 | "Jan", "Feb", "Mar", "Apr", "May", "Jun", 42 | "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" 43 | }; 44 | }; 45 | struct nanosec_time_prefix : public prefix_base { 46 | template 47 | void apply(typename logging::level::level_enum level, Logger& logger) { 48 | auto now = std::chrono::system_clock::now(); 49 | auto now_ns = std::chrono::duration_cast(now.time_since_epoch()).count(); 50 | "[%llu]: "_log(logger, now_ns); 51 | } 52 | }; 53 | struct log_level_prefix : public prefix_base { 54 | 55 | template 56 | void apply(typename logging::level::level_enum level, Logger& logger) { 57 | "[%8s]"_log(logger, m_level_strings[level]); 58 | } 59 | 60 | static constexpr std::array m_level_strings = { 61 | "TRACE", 62 | "DEBUG", 63 | "INFO", 64 | "WARN", 65 | "ERR", 66 | "CRITICAL" 67 | }; 68 | }; 69 | } -------------------------------------------------------------------------------- /include/llcpp/detail/string_format.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "format_parser.hpp" 4 | 5 | namespace llcpp::detail::string_format { 6 | template 7 | struct string_format { 8 | using config_t = Config; 9 | using format_parser = typename config_t::template format_parser; 10 | constexpr string_format() {} 11 | 12 | static constexpr std::size_t size() { 13 | return _size; 14 | } 15 | 16 | static constexpr std::size_t fmt_size() { 17 | return _fmt_size; 18 | } 19 | static constexpr std::size_t args_size() { 20 | return _args_size; 21 | } 22 | 23 | static constexpr std::size_t _fmt_size = sizeof...(Chars) + 1; 24 | static constexpr std::size_t _args_size = format_parser::sum_arguments_size::value; 25 | static constexpr std::size_t _size = _fmt_size + _args_size; 26 | 27 | static constexpr CharT _chars[string_format::fmt_size()] = {Chars..., '\0'}; 28 | }; 29 | } -------------------------------------------------------------------------------- /include/llcpp/detail/terminators.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | #include 3 | #include 4 | 5 | #include "utils.hpp" 6 | 7 | namespace llcpp::detail::terminators 8 | { 9 | template 10 | struct terminator 11 | { 12 | using char_type = CharT; 13 | static constexpr CharT terminator_value = Char; 14 | 15 | template 16 | struct argument_parser 17 | { 18 | static constexpr std::size_t argument_size = 0; 19 | using argument_type = ArgumentType; 20 | static constexpr bool is_fixed_size = true; 21 | 22 | template 23 | static void apply(Logger& logger, const Arg arg) {} 24 | template 25 | static void apply_variable(Logger& logger, const char *arg) {} 26 | }; 27 | 28 | using empty_parser = argument_parser, 0, 0>; 29 | }; 30 | 31 | struct void_terminator : public terminator 32 | { 33 | }; 34 | 35 | struct pct_terminator : public terminator 36 | { 37 | template 38 | struct argument_parser : public terminator::argument_parser 39 | { 40 | static_assert(EscapeIdx == TerminatorIdx - 1, "Invalid %% escape. Either the format is wrong " 41 | "or you are using a terminator we don't support"); 42 | }; 43 | }; 44 | 45 | template 46 | struct tuple_counter 47 | { 48 | using cur_char = typename std::tuple_element::type; 49 | static constexpr std::size_t count = ((cur_char::value == Char) ? (1) : (0)) + 50 | tuple_counter::count; 51 | }; 52 | template 53 | struct tuple_counter 54 | { 55 | static constexpr std::size_t count = 0; 56 | }; 57 | 58 | struct d : public terminator 59 | { 60 | template 61 | struct argument_parser 62 | { 63 | static constexpr bool is_fixed_size = true; 64 | static constexpr std::size_t num_of_l = tuple_counter::count; 65 | static_assert(num_of_l < 3, "Invalid conversion specifier"); 66 | static constexpr std::size_t argument_size = ((num_of_l == 0) ? (sizeof(std::uint32_t)) : (sizeof(std::uint32_t) * num_of_l)); 67 | using argument_type = typename std::conditional< 68 | num_of_l == 0, 69 | int, 70 | typename std::conditional< 71 | num_of_l == 1, 72 | long, 73 | long long>::type>::type; 74 | 75 | template 76 | static void apply(Logger& logger, const Arg arg) 77 | { 78 | static_assert(std::is_integral::value, 79 | "Integral argument's apply function called with non-integral value"); 80 | static_assert(argument_size >= sizeof(Arg), 81 | "Discrepency detected between parsed argument_size and size of given arg, " 82 | "you may need a specialized argument_parser"); 83 | argument_type tmp = static_cast(arg); 84 | logger.write((std::uint8_t *)&tmp, sizeof(argument_type)); 85 | } 86 | 87 | template 88 | static void apply_variable(Logger& logger, const char *arg) {} 89 | }; 90 | }; 91 | struct u : public terminator 92 | { 93 | template 94 | struct argument_parser : public d::argument_parser 95 | { 96 | using base_argument_type = typename d::argument_parser::argument_type; 97 | using argument_type = typename std::make_unsigned_t; 98 | }; 99 | 100 | }; 101 | struct x : public terminator 102 | { 103 | template 104 | using argument_parser = d::argument_parser; 105 | }; 106 | 107 | //HACK: There's probably a better way to do this 108 | constexpr std::size_t pow(std::size_t base, std::size_t exp) 109 | { 110 | std::size_t tmp = 1; 111 | for (std::size_t i = 0; i < exp; i++) 112 | { 113 | tmp *= base; 114 | } 115 | return tmp; 116 | } 117 | 118 | //HACK: Should probably switch this to a constexpr function, just need to convert the tuple to a char array 119 | template 120 | struct tuple_atoi 121 | { 122 | using cur_char = typename std::tuple_element::type; 123 | static_assert(cur_char::value >= '0' && cur_char::value <= '9', "tuple_atoi encountered a non-digit character"); 124 | static constexpr std::size_t exp = EndIdx - Idx; 125 | static constexpr std::size_t value = pow(10, exp - 1) * (cur_char::value - '0') + tuple_atoi::value; 126 | }; 127 | template 128 | struct tuple_atoi 129 | { 130 | static constexpr std::size_t value = 0; 131 | }; 132 | 133 | struct s : public terminator 134 | { 135 | using variable_string_length_type = std::uint32_t; 136 | 137 | template 138 | struct argument_parser 139 | { 140 | static constexpr std::size_t argument_size = ((EscapeIdx + 1 == TerminatorIdx) ? (sizeof(variable_string_length_type)) : (tuple_atoi::value)); 141 | static constexpr bool is_fixed_size = (EscapeIdx + 1 != TerminatorIdx); 142 | 143 | using argument_type = typename std::conditional< 144 | is_fixed_size, 145 | char[argument_size], 146 | char *>::type; 147 | 148 | template 149 | static void apply(Logger& logger, const char *arg) 150 | { 151 | if 152 | constexpr(is_fixed_size) 153 | { 154 | auto len = std::strlen(arg); 155 | if (len < argument_size) 156 | { 157 | logger.write((std::uint8_t *)arg, len); 158 | // fill the rest with zeroes 159 | static std::uint8_t zeroes[1024] = {0}; 160 | std::size_t diff = argument_size - len; 161 | std::size_t zeroes_written = 0; 162 | while (zeroes_written < diff) 163 | { 164 | std::size_t zeroes_to_write = std::min(diff - zeroes_written, sizeof(zeroes)); 165 | logger.write(zeroes, zeroes_to_write); 166 | zeroes_written += zeroes_to_write; 167 | } 168 | } 169 | else if (len >= argument_size) 170 | { 171 | logger.write((std::uint8_t *)arg, argument_size); 172 | } 173 | } 174 | else 175 | { 176 | variable_string_length_type size = static_cast(std::strlen(arg)); 177 | logger.write((std::uint8_t *)&size, sizeof(size)); 178 | } 179 | } 180 | 181 | template 182 | static void apply_variable(Logger& logger, const char *arg) 183 | { 184 | if 185 | constexpr(!is_fixed_size) 186 | { 187 | logger.write((std::uint8_t *)arg, std::strlen(arg)); 188 | } 189 | } 190 | }; 191 | }; 192 | 193 | template 194 | struct search_terminators_impl 195 | { 196 | static_assert(std::is_same::type::char_type, CharT>::value, 197 | "Discrepency between the format's element type and terminator element type"); 198 | using type = typename std::conditional< 199 | std::tuple_element::type::terminator_value == Char, 200 | typename std::tuple_element::type, 201 | typename search_terminators_impl::type>::type; 202 | }; 203 | template 204 | struct search_terminators_impl 205 | { 206 | using type = void_terminator; 207 | }; 208 | 209 | template 210 | struct search_terminators 211 | { 212 | using type = typename search_terminators_impl>::type; 213 | }; 214 | 215 | template 216 | struct terminator_container_impl 217 | { 218 | static_assert(utils::is_specialization_of::value, "TerminatorTuple must be a tuple"); 219 | 220 | template 221 | using is_terminator = typename std::conditional< 222 | std::is_same::type, void_terminator>::value, 223 | std::false_type, 224 | std::true_type>::type; 225 | 226 | template 227 | using search_terminators_t = typename search_terminators::type; 228 | }; 229 | 230 | using builtin_terminator_tuple = std::tuple; 231 | 232 | template 233 | using terminator_tuple_from_config = utils::tuple_cat_t; 234 | template 235 | struct terminator_container : public terminator_container_impl>{}; 236 | } 237 | 238 | namespace llcpp::user_terminators { 239 | using namespace llcpp::detail::terminators; 240 | } -------------------------------------------------------------------------------- /include/llcpp/detail/udl.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "log_line.hpp" 4 | #include "config.hpp" 5 | 6 | template 7 | constexpr auto operator"" _log() noexcept 8 | { 9 | using log_line_t = typename llcpp::detail::log_line::log_line; 10 | return log_line_t(); 11 | } -------------------------------------------------------------------------------- /include/llcpp/detail/utils.hpp: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | namespace llcpp::detail::utils { 4 | /* 5 | * A little static_assert debug utility. 6 | * The extra `Args...` are shown pretty close to the static_assert in the compilation 7 | * log which allows us to give us hints on the issue. 8 | */ 9 | template 10 | struct StaticAssert { 11 | static constexpr bool value = Bool; 12 | static_assert(Bool, "DONT PANIC"); 13 | }; 14 | 15 | /* 16 | * A little hack to create a std::conditional like struct that won't evaluate 17 | * both branches (which sometimes lead to errors on the other branch) 18 | */ 19 | template 20 | struct type_if_or { 21 | using type = Else; 22 | }; 23 | template 24 | struct type_if_or { 25 | using type = T; 26 | }; 27 | 28 | //https://bitbucket.org/martinhofernandes/wheels/src/default/include/wheels/meta/type_traits.h%2B%2B?fileviewer=file-view-default#cl-161 29 | //Example: static_assert(utils::is_specialization_of::value, "CharTuple must be a tuple"); 30 | template class Template> 31 | struct is_specialization_of : std::integral_constant {}; 32 | template