├── .clang-format ├── .gitignore ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── build.sh ├── build_static.sh └── src ├── histogram.h ├── ipv4_ranges.cc ├── ipv4_ranges.h ├── metrics_labels.h ├── metrics_page.cc ├── metrics_page.h ├── packet_counter.cc ├── packet_counter.h ├── packet_parser.cc ├── packet_parser.h ├── parsed_packet_processor.h ├── pcap.cc ├── pcap.h ├── promacct.cc ├── protocol_histogram.h ├── raw_packet_processor.h ├── webserver.cc ├── webserver.h └── webserver_request_handler.h /.clang-format: -------------------------------------------------------------------------------- 1 | BasedOnStyle: Google 2 | AllowShortIfStatementsOnASingleLine: false 3 | AllowShortLoopsOnASingleLine: false 4 | AllowShortFunctionsOnASingleLine: None 5 | DerivePointerBinding: false 6 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /promacct 2 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of experience, 9 | nationality, personal appearance, race, religion, or sexual identity and 10 | orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at mgmt@kumina.nl. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at [http://contributor-covenant.org/version/1/4][version] 72 | 73 | [homepage]: http://contributor-covenant.org 74 | [version]: http://contributor-covenant.org/version/1/4/ 75 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | All code in the src/ directory is distributed under the following license: 2 | 3 | Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 4 | 5 | Redistribution and use in source and binary forms, with or without 6 | modification, are permitted provided that the following conditions 7 | are met: 8 | 1. Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer. 10 | 2. Redistributions in binary form must reproduce the above copyright 11 | notice, this list of conditions and the following disclaimer in the 12 | documentation and/or other materials provided with the distribution. 13 | 14 | THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17 | ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20 | OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21 | HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22 | LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23 | OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24 | SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # promacct: a pcap-based traffic accounting tool for Prometheus 2 | 3 | The utility provided by this repository is similar to 4 | [pmacct](http://www.pmacct.net/). It can use 5 | [libpcap](http://www.tcpdump.org/) to inspect traffic on a network 6 | interface and store statistics regarding the number of packets and their 7 | size. In addition to storing the total amount of traffic, it also has 8 | support for aggregating by source/destination IPv4 address. 9 | 10 | Where this utility differs from pmacct is that it doesn't store its 11 | results in a database. Instead, it binds a HTTP server that exports a 12 | metrics page that can be scraped by Prometheus. By default, this HTTP 13 | server listens on port 9112. 14 | 15 | # Building promacct 16 | 17 | Right now promacct is still simple enough that it can easily be built by 18 | hand. Be sure to take a look at `build.sh` to see how. The script 19 | `build_static.sh` builds a statically linked executable for Linux-based 20 | systems. 21 | 22 | Promacct has very few dependencies. It's written in C++, making use of 23 | certain C++17 features. It makes use of libpcap. 24 | 25 | # Using promacct 26 | 27 | After building promacct, it can be started as follows: 28 | 29 | ``` 30 | promacct -i eth0 -i eth1 -r 192.168.1.100-192.168.1.200:customer=acmecorp:environment=production 31 | ``` 32 | 33 | This makes promacct sniff for traffic on eth0 and eth1, storing the 34 | total amount of traffic in separate histograms. It also creates 35 | histograms for the aggregated amount of network traffic for every 36 | individual IPv4 address between 192.168.1.100 and 192.168.1.200. To each 37 | of these entries, it also attaches the labels `customer` and 38 | `environment`, having the values `acmecorp` and `production`, 39 | respectively. 40 | 41 | # Useful Prometheus rules 42 | 43 | The following rule can be used to compute a five-minute rate of all 44 | traffic per host and network interface: 45 | 46 | ``` 47 | instance_interface:promacct_packet_size_bytes_all:rate5m = 48 | sum(rate(promacct_packet_size_bytes_all_sum{job="promacct"}[5m])) 49 | by (instance, interface) 50 | ``` 51 | 52 | This metric can be used to compute a monthly 95th percentile as follows: 53 | 54 | ``` 55 | instance_interface:promacct_packet_size_bytes_all:quantile31d{quantile="0.95"} = 56 | quantile_over_time(0.95, instance_interface:promacct_packet_size_bytes_all:rate5m[31d]) 57 | ``` 58 | -------------------------------------------------------------------------------- /build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | c++ -O2 -std=gnu++1z -o promacct src/*.cc -Wall -Werror -lpcap 5 | clang-format -i src/* 6 | -------------------------------------------------------------------------------- /build_static.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | docker run -i -v `pwd`:/promacct alpine:edge /bin/sh << 'EOF' 4 | set -ex 5 | 6 | # Install prerequisites for the build process. 7 | apk update 8 | apk add g++ libpcap-dev 9 | 10 | # Compatibility with C++17. This can be removed when Alpine Linux 11 | # switches to GCC 7, which will include these headers by default. 12 | cat > /usr/include/optional << EOF2 13 | #include 14 | namespace std { 15 | using std::experimental::optional; 16 | } 17 | EOF2 18 | cat > /usr/include/string_view << EOF2 19 | #include 20 | namespace std { 21 | using std::experimental::basic_string_view; 22 | using std::experimental::string_view; 23 | } 24 | EOF2 25 | 26 | cd /promacct 27 | c++ -O2 -static -std=gnu++1z -o promacct src/*.cc -Wall -Werror -lpcap 28 | strip promacct 29 | EOF 30 | -------------------------------------------------------------------------------- /src/histogram.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef HISTOGRAM_H 7 | #define HISTOGRAM_H 8 | 9 | #include 10 | #include 11 | #include 12 | 13 | #include "metrics_labels.h" 14 | #include "metrics_page.h" 15 | 16 | // Prometheus-style histogram. 17 | // 18 | // Prometheus has a standard convention for processing histograms. This 19 | // templated class provides a convenient way of creating such objects. 20 | // The template arguments can be used to specify the bucket cutoffs. 21 | // 22 | // A histogram containing four buckets ("32", "64", "128" and "+Inf") 23 | // can be created as follows: 24 | // 25 | // Histogram<32, 64, 128> my_histogram; 26 | template 27 | class Histogram { 28 | public: 29 | Histogram() : count_(), sum_(), buckets_() { 30 | } 31 | 32 | // Stores a new sample value in the histogram object. 33 | void Record(std::uint64_t value) { 34 | // Update scalar values. 35 | ++count_; 36 | sum_ += value; 37 | 38 | // Adjust bucket counters. 39 | const unsigned int boundaries[] = {Buckets...}; 40 | for (unsigned int i = sizeof...(Buckets); 41 | i > 0 && value <= boundaries[i - 1]; --i) 42 | ++buckets_[i - 1]; 43 | } 44 | 45 | // Prints all values stored in the histogram object. 46 | void PrintMetrics(const std::string& name, const MetricsLabels& labels, 47 | MetricsPage* output) const { 48 | if (count_ > 0) { 49 | // Print scalar values. 50 | output->PrintMetric(name + "_count", labels, count_); 51 | output->PrintMetric(name + "_sum", labels, sum_); 52 | 53 | // Print bucket counters. 54 | const unsigned int boundaries[] = {Buckets...}; 55 | for (unsigned int i = 0; i < sizeof...(Buckets); ++i) { 56 | std::string boundary = std::to_string(boundaries[i]); 57 | MetricsLabel le("le", boundary); 58 | MetricsLabelsJoiner joiner(&labels, &le); 59 | output->PrintMetric(name + "_bucket", joiner, buckets_[i]); 60 | } 61 | MetricsLabel le("le", "+Inf"); 62 | MetricsLabelsJoiner joiner(&labels, &le); 63 | output->PrintMetric(name + "_bucket", joiner, count_); 64 | } 65 | } 66 | 67 | private: 68 | std::uint64_t count_; // Number of samples. 69 | std::uint64_t sum_; // Sum of all samples. 70 | std::array buckets_; // Histogram buckets. 71 | }; 72 | 73 | #endif 74 | -------------------------------------------------------------------------------- /src/ipv4_ranges.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | #include 8 | #include 9 | 10 | #include "ipv4_ranges.h" 11 | 12 | void IPv4Ranges::AddRange(const MetricsLabels* labels, std::uint32_t first, 13 | std::uint32_t last) { 14 | assert(first <= last && "Not a valid IPv4 range."); 15 | ranges_.push_back({labels, first, last}); 16 | } 17 | 18 | std::size_t IPv4Ranges::GetLength() const { 19 | std::size_t length = 0; 20 | for (const auto& range : ranges_) 21 | length += range.last - range.first + 1; 22 | return length; 23 | } 24 | 25 | std::optional IPv4Ranges::GetIndexByAddress( 26 | std::uint32_t address) const { 27 | std::size_t offset = 0; 28 | for (const auto& range : ranges_) { 29 | if (address >= range.first && address <= range.last) 30 | return offset + address - range.first; 31 | offset += range.last - range.first + 1; 32 | } 33 | return {}; 34 | } 35 | 36 | std::pair IPv4Ranges::GetAddressByIndex( 37 | std::size_t idx) const { 38 | for (const auto& range : ranges_) { 39 | if (idx <= range.last - range.first) 40 | return {range.labels, range.first + idx}; 41 | idx -= range.last - range.first + 1; 42 | } 43 | assert(0 && "GetAddressByIndex() out of bounds"); 44 | } 45 | -------------------------------------------------------------------------------- /src/ipv4_ranges.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef IPV4_RANGES_H 7 | #define IPV4_RANGES_H 8 | 9 | #include 10 | #include 11 | #include 12 | 13 | class MetricsLabels; 14 | 15 | // Set of IPv4 address ranges. 16 | // 17 | // We want to be able to aggregate traffic based on IPv4 address ranges. 18 | // This class implements a storage for such ranges, keeping track of 19 | // them as pairs of first and last IPv4 address. 20 | class IPv4Ranges { 21 | public: 22 | // Adds a new IPv4 address range to the store. 23 | void AddRange(const MetricsLabels* labels, std::uint32_t first, 24 | std::uint32_t last); 25 | 26 | // Returns the total number of IPv4 addresses stored within. 27 | std::size_t GetLength() const; 28 | 29 | // Given an IPv4 address, return a unique identifier. This is used by 30 | // ParsedPacketCounter to store all histograms contiguously. 31 | std::optional GetIndexByAddress(std::uint32_t address) const; 32 | 33 | // Given a unique identifier, return the IPv4 address. This is used by 34 | // ParsedPacketCounter to reobtain the IPv4 address associated with a 35 | // histogram, used for printing metrics. 36 | std::pair GetAddressByIndex( 37 | std::size_t) const; 38 | 39 | private: 40 | struct IPv4Range { 41 | const MetricsLabels* labels; 42 | std::uint32_t first; 43 | std::uint32_t last; 44 | }; 45 | 46 | std::vector ranges_; 47 | }; 48 | 49 | #endif 50 | -------------------------------------------------------------------------------- /src/metrics_labels.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef METRICS_LABELS_H 7 | #define METRICS_LABELS_H 8 | 9 | #include 10 | #include 11 | 12 | // Rope/cord-like class for holding a set of metrics labels. 13 | // 14 | // These classes can be used to temporarily construct metrics labels on 15 | // the stack, so that they can be printed by class MetricsPage. To 16 | // prevent any heap allocations, these classes use a tree-like 17 | // structure, so labels from multiple sources can be combined. 18 | class MetricsLabels { 19 | public: 20 | MetricsLabels() { 21 | } 22 | virtual ~MetricsLabels() { 23 | } 24 | 25 | virtual void Print(std::ostream* output, bool* needs_comma) const = 0; 26 | 27 | MetricsLabels& operator=(const MetricsLabels&) = delete; 28 | MetricsLabels(const MetricsLabels&) = delete; 29 | }; 30 | 31 | // Singleton key-value pair, representing a single label. 32 | class MetricsLabel : public MetricsLabels { 33 | public: 34 | MetricsLabel(std::string_view key, std::string_view value) 35 | : key_(key), value_(value) { 36 | } 37 | 38 | void Print(std::ostream* output, bool* needs_comma) const override { 39 | if (*needs_comma) 40 | *output << ','; 41 | *needs_comma = true; 42 | *output << key_ << "=\"" << value_ << '"'; 43 | } 44 | 45 | private: 46 | std::string_view key_; 47 | std::string_view value_; 48 | }; 49 | 50 | // The empty set: no labels. 51 | class MetricsLabelsTerminator : public MetricsLabels { 52 | public: 53 | void Print(std::ostream* output, bool* needs_comma) const override { 54 | } 55 | }; 56 | 57 | // Union of two sets of labels. 58 | class MetricsLabelsJoiner : public MetricsLabels { 59 | public: 60 | MetricsLabelsJoiner(const MetricsLabels* left, const MetricsLabels* right) 61 | : left_(left), right_(right) { 62 | } 63 | 64 | void Print(std::ostream* output, bool* needs_comma) const override { 65 | left_->Print(output, needs_comma); 66 | right_->Print(output, needs_comma); 67 | } 68 | 69 | private: 70 | const MetricsLabels* const left_; 71 | const MetricsLabels* const right_; 72 | }; 73 | 74 | #endif 75 | -------------------------------------------------------------------------------- /src/metrics_page.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | 8 | #include "metrics_labels.h" 9 | #include "metrics_page.h" 10 | 11 | void MetricsPage::PrintMetric(std::string_view name, 12 | const MetricsLabels& labels, 13 | std::uint64_t value) { 14 | // Print metric name, labels and its value. 15 | *output_ << prefix_ << name << '{'; 16 | bool needs_comma = false; 17 | labels.Print(output_, &needs_comma); 18 | *output_ << "} " << value << std::endl; 19 | } 20 | -------------------------------------------------------------------------------- /src/metrics_page.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef METRICS_PAGE_H 7 | #define METRICS_PAGE_H 8 | 9 | #include 10 | #include 11 | #include 12 | 13 | class MetricsLabels; 14 | 15 | // Adapter for rendering pages containing metrics. 16 | // 17 | // Instead of letting all of the code print metrics straight to an 18 | // ostream, this class acts like an adapter. Callers can simply call 19 | // PrintMetric(), providing it a name, a set of labels and a value. 20 | class MetricsPage { 21 | public: 22 | MetricsPage(std::string_view prefix, std::ostream* output) 23 | : prefix_(prefix), output_(output) { 24 | } 25 | 26 | // Writes a metric to the output in the form '$prefix$name{$labels} $value'. 27 | void PrintMetric(std::string_view name, const MetricsLabels& labels, 28 | std::uint64_t value); 29 | 30 | private: 31 | const std::string prefix_; 32 | std::ostream* const output_; 33 | }; 34 | 35 | #endif 36 | -------------------------------------------------------------------------------- /src/packet_counter.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #include "ipv4_ranges.h" 13 | #include "packet_counter.h" 14 | 15 | PacketCounter::PacketCounter(const IPv4Ranges* aggregation_ipv4) 16 | : aggregation_ipv4_(aggregation_ipv4), 17 | packet_size_bytes_ipv4_tx_(aggregation_ipv4_->GetLength()), 18 | packet_size_bytes_ipv4_rx_(aggregation_ipv4_->GetLength()) { 19 | // Histograms for IPv4 address aggregation are already preallocated, 20 | // so that ProcessIPv4Packet() doesn't need to do any resizing. 21 | } 22 | 23 | void PacketCounter::ProcessIPv4Packet(std::uint32_t src, std::uint32_t dst, 24 | std::uint8_t protocol, 25 | std::size_t original_length) { 26 | packet_size_bytes_all_.Record(original_length); 27 | 28 | // Aggregation on source IPv4 address. 29 | { 30 | std::optional index = 31 | aggregation_ipv4_->GetIndexByAddress(src); 32 | if (index) 33 | packet_size_bytes_ipv4_tx_[*index].Record(protocol, original_length); 34 | } 35 | 36 | // Aggregation on destination IPv4 address. 37 | { 38 | std::optional index = 39 | aggregation_ipv4_->GetIndexByAddress(dst); 40 | if (index) 41 | packet_size_bytes_ipv4_rx_[*index].Record(protocol, original_length); 42 | } 43 | } 44 | 45 | void PacketCounter::ProcessUnknownPacket(std::size_t original_length) { 46 | packet_size_bytes_all_.Record(original_length); 47 | } 48 | 49 | void PacketCounter::PrintMetrics(const MetricsLabels& labels, 50 | MetricsPage* output) { 51 | packet_size_bytes_all_.PrintMetrics("packet_size_bytes_all", labels, output); 52 | 53 | for (std::size_t i = 0; i < aggregation_ipv4_->GetLength(); ++i) { 54 | // Combine the labels of the packet counter and the per-address 55 | // histogram. 56 | std::pair addr = 57 | aggregation_ipv4_->GetAddressByIndex(i); 58 | MetricsLabelsJoiner joiner1(&labels, addr.first); 59 | 60 | // Add the IPv4 address as a label. 61 | std::stringstream addr_ss; 62 | addr_ss << (addr.second >> 24) << '.' << (addr.second >> 16 & 0xff) << '.' 63 | << (addr.second >> 8 & 0xff) << '.' << (addr.second & 0xff); 64 | std::string addr_str = addr_ss.str(); 65 | MetricsLabel ip("ip", addr_str); 66 | MetricsLabelsJoiner joiner2(&joiner1, &ip); 67 | 68 | // Print aggregated TX/RX statistics. 69 | packet_size_bytes_ipv4_tx_[i].PrintMetrics("packet_size_bytes_ipv4_tx", 70 | joiner2, output); 71 | packet_size_bytes_ipv4_rx_[i].PrintMetrics("packet_size_bytes_ipv4_rx", 72 | joiner2, output); 73 | } 74 | } 75 | -------------------------------------------------------------------------------- /src/packet_counter.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef PACKET_COUNTER_H 7 | #define PACKET_COUNTER_H 8 | 9 | #include 10 | #include 11 | 12 | #include "histogram.h" 13 | #include "parsed_packet_processor.h" 14 | #include "protocol_histogram.h" 15 | 16 | class IPv4Ranges; 17 | class MetricsLabels; 18 | class MetricsPage; 19 | 20 | // Counts network packets, aggregating them by IPv4 source/destination 21 | // address. 22 | class PacketCounter : public ParsedPacketProcessor { 23 | public: 24 | explicit PacketCounter(const IPv4Ranges* aggregation_ipv4); 25 | 26 | // Counts an IPv4 packet. 27 | void ProcessIPv4Packet(std::uint32_t src, std::uint32_t dst, 28 | std::uint8_t protocol, 29 | std::size_t original_length) override; 30 | // Counts a network packet of an unknown type. 31 | void ProcessUnknownPacket(std::size_t original_length) override; 32 | 33 | // Prints all of the stored histograms to the metrics page output. 34 | void PrintMetrics(const MetricsLabels& labels, MetricsPage* output); 35 | 36 | private: 37 | typedef Histogram<64, 128, 256, 512, 1024, 2048> PacketSizeHistogram; 38 | typedef ProtocolHistogram ProtocolPacketSizeHistogram; 39 | 40 | const IPv4Ranges* const aggregation_ipv4_; 41 | 42 | // Histogram storing the sizes of all incoming network packets. 43 | PacketSizeHistogram packet_size_bytes_all_; 44 | 45 | // Per-IPv4 address histogram storing the sizes of all packets having 46 | // the address as its source. 47 | std::vector packet_size_bytes_ipv4_tx_; 48 | 49 | // Per-IPv4 address histogram storing the sizes of all packets having 50 | // the address as its destination. 51 | std::vector packet_size_bytes_ipv4_rx_; 52 | }; 53 | 54 | #endif 55 | -------------------------------------------------------------------------------- /src/packet_parser.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | #include 8 | #include 9 | 10 | #include "packet_parser.h" 11 | #include "parsed_packet_processor.h" 12 | 13 | void PacketParser::ProcessPacket(std::basic_string_view bytes, 14 | std::size_t original_length) { 15 | // Strip off the ethernet header and don't account for it in the 16 | // histograms. We're not interested in accounting the link layer. 17 | assert(bytes.size() >= BytesNeededEthernetHeader); 18 | assert(original_length >= bytes.size()); 19 | bytes.remove_prefix(BytesNeededEthernetHeader); 20 | original_length -= BytesNeededEthernetHeader; 21 | 22 | if (bytes.size() >= 20 && (bytes[0] & 0xf0) == 0x40) { 23 | // Proper IPv4 packet. Extract source and destination addresses. 24 | std::uint32_t src = (std::uint32_t)bytes[12] << 24 | 25 | (std::uint32_t)bytes[13] << 16 | 26 | (std::uint32_t)bytes[14] << 8 | bytes[15]; 27 | std::uint32_t dst = (std::uint32_t)bytes[16] << 24 | 28 | (std::uint32_t)bytes[17] << 16 | 29 | (std::uint32_t)bytes[18] << 8 | bytes[19]; 30 | processor_->ProcessIPv4Packet(src, dst, bytes[9], original_length); 31 | } else { 32 | // Unknown packet type. 33 | processor_->ProcessUnknownPacket(original_length); 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /src/packet_parser.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef PACKET_PARSER_H 7 | #define PACKET_PARSER_H 8 | 9 | #include 10 | #include 11 | 12 | #include "raw_packet_processor.h" 13 | 14 | class ParsedPacketProcessor; 15 | 16 | // Adapter for parsing Ethernet packets. 17 | // 18 | // The Pcap class invokes ProcessPacket(), providing it access to the 19 | // raw packet data. This class attempts to extract the IPv4 source and 20 | // destination addresses and invokes the methods of the 21 | // ParsedPacketProcessor. 22 | class PacketParser : public RawPacketProcessor { 23 | public: 24 | explicit PacketParser(ParsedPacketProcessor* processor) 25 | : processor_(processor) { 26 | } 27 | 28 | // Parses raw packets. 29 | void ProcessPacket(std::basic_string_view bytes, 30 | std::size_t original_length) override; 31 | 32 | // Minimum snapshot length needed in order to properly parse the 33 | // ethernet header containing the MAC addresses. 34 | static constexpr std::size_t BytesNeededEthernetHeader = 14; 35 | 36 | // Minimum snapshot length needed in order to properly parse IPv4 37 | // packet headers. 38 | static constexpr std::size_t BytesNeededIPv4 = BytesNeededEthernetHeader + 20; 39 | 40 | private: 41 | ParsedPacketProcessor* const processor_; 42 | }; 43 | 44 | #endif 45 | -------------------------------------------------------------------------------- /src/parsed_packet_processor.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef PARSED_PACKET_PROCESSOR_H 7 | #define PARSED_PACKET_PROCESSOR_H 8 | 9 | #include 10 | 11 | // Interface for processing parsed network packets. 12 | class ParsedPacketProcessor { 13 | public: 14 | // IPv4 packet detected. 15 | virtual void ProcessIPv4Packet(std::uint32_t src, std::uint32_t dst, 16 | std::uint8_t protocol, 17 | std::size_t original_length) = 0; 18 | 19 | // Network packet of a different format detected. 20 | virtual void ProcessUnknownPacket(std::size_t original_length) = 0; 21 | }; 22 | 23 | #endif 24 | -------------------------------------------------------------------------------- /src/pcap.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | 8 | #include 9 | #include 10 | #include 11 | #include 12 | #include 13 | #include 14 | 15 | #include "pcap.h" 16 | #include "raw_packet_processor.h" 17 | 18 | std::optional Pcap::Activate(const std::string& device, 19 | std::size_t snapshot_length, 20 | std::size_t buffer_length) { 21 | char errbuf[PCAP_ERRBUF_SIZE]; 22 | std::unique_ptr pcap( 23 | pcap_create(device.c_str(), errbuf)); 24 | if (!pcap) 25 | return std::string(errbuf); 26 | 27 | pcap_set_promisc(pcap.get(), false); 28 | pcap_set_snaplen(pcap.get(), snapshot_length); 29 | pcap_set_buffer_size(pcap.get(), buffer_length); 30 | 31 | if (pcap_activate(pcap.get()) != 0) 32 | return std::string(pcap_geterr(pcap.get())); 33 | 34 | int ret = pcap_setnonblock(pcap.get(), true, errbuf); 35 | if (ret != 0) 36 | return std::string(errbuf); 37 | 38 | pcap_ = std::move(pcap); 39 | return {}; 40 | } 41 | 42 | void Pcap::Callback_(unsigned char* user, const struct pcap_pkthdr* header, 43 | const unsigned char* bytes) { 44 | RawPacketProcessor* processor = (RawPacketProcessor*)user; 45 | processor->ProcessPacket({bytes, header->caplen}, header->len); 46 | } 47 | 48 | unsigned int Pcap::Dispatch(RawPacketProcessor* processor) { 49 | assert(pcap_ && "Cannot dispatch before activating."); 50 | int count = pcap_dispatch(pcap_.get(), 0, &Pcap::Callback_, 51 | (unsigned char*)processor); 52 | assert(count >= 0 && "pcap_dispatch() failed."); 53 | return count; 54 | } 55 | -------------------------------------------------------------------------------- /src/pcap.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef PCAP_H 7 | #define PCAP_H 8 | 9 | #include 10 | 11 | #include 12 | #include 13 | #include 14 | #include 15 | 16 | class RawPacketProcessor; 17 | 18 | // Automatic deleter for pcap handles to be used with std::unique_ptr. 19 | class PcapDeleter { 20 | public: 21 | void operator()(pcap_t* pcap) { 22 | pcap_close(pcap); 23 | } 24 | }; 25 | 26 | // Network packet capturer based on libpcap. 27 | class Pcap { 28 | public: 29 | // Create a pcap handle for a network device and start capturing. 30 | std::optional Activate(const std::string& device, 31 | std::size_t snapshot_length, 32 | std::size_t buffer_length); 33 | 34 | // Read the next batch of packets from the pcap handle and forward its 35 | // contents over to the raw packet processor. 36 | unsigned int Dispatch(RawPacketProcessor* processor); 37 | 38 | private: 39 | std::unique_ptr pcap_; 40 | 41 | // Callback invoked by libpcap. 42 | static void Callback_(unsigned char* user, const struct pcap_pkthdr* header, 43 | const unsigned char* bytes); 44 | }; 45 | 46 | #endif 47 | -------------------------------------------------------------------------------- /src/promacct.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | 8 | #include 9 | 10 | #include 11 | #include 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | #include "ipv4_ranges.h" 22 | #include "metrics_page.h" 23 | #include "packet_counter.h" 24 | #include "packet_parser.h" 25 | #include "pcap.h" 26 | #include "webserver.h" 27 | #include "webserver_request_handler.h" 28 | 29 | namespace { 30 | class PacketCounterServer : public WebserverRequestHandler { 31 | public: 32 | PacketCounterServer(const std::vector* interfaces, 33 | std::vector* packet_counters) 34 | : interfaces_(interfaces), packet_counters_(packet_counters) { 35 | } 36 | 37 | void HandleRequest(std::ostream* output) override { 38 | MetricsPage page("promacct_", output); 39 | for (size_t i = 0; i < interfaces_->size(); ++i) { 40 | MetricsLabel interface("interface", (*interfaces_)[i]); 41 | (*packet_counters_)[i].PrintMetrics(interface, &page); 42 | } 43 | } 44 | 45 | private: 46 | const std::vector* const interfaces_; 47 | std::vector* const packet_counters_; 48 | }; 49 | 50 | void usage() { 51 | std::cerr << "usage: promacct -i interface ... [-p httpport] " 52 | "[-r startaddr-endaddr[:key=value...] ...]" 53 | << std::endl; 54 | std::exit(1); 55 | } 56 | 57 | std::uint32_t parse_ipv4_address(const std::string& str) { 58 | struct in_addr addr; 59 | if (inet_pton(AF_INET, str.c_str(), &addr) != 1) 60 | usage(); 61 | return ntohl(addr.s_addr); 62 | } 63 | } // namespace 64 | 65 | int main(int argc, char* argv[]) { 66 | // Parse command line arguments. 67 | int ch; 68 | std::vector interfaces; 69 | std::uint16_t httpport = 9112; 70 | IPv4Ranges ranges; 71 | MetricsLabelsTerminator no_labels; 72 | std::forward_list labels; 73 | std::forward_list joiners; 74 | while ((ch = getopt(argc, argv, "i:p:r:")) != -1) { 75 | switch (ch) { 76 | case 'i': 77 | // Network interface. 78 | interfaces.push_back(optarg); 79 | break; 80 | case 'p': 81 | // Port number on which to bind the HTTP server. 82 | httpport = std::stoi(optarg); 83 | break; 84 | case 'r': { 85 | // IP range: startaddr-endaddr[:key=value...]. 86 | // Extract start address and end address. 87 | std::string_view arg(optarg); 88 | auto endaddr = std::find(arg.begin(), arg.end(), '-'); 89 | if (endaddr == arg.end()) 90 | usage(); 91 | auto kvs = std::find(endaddr, arg.end(), ':'); 92 | 93 | // Extract labels. 94 | const MetricsLabels* range_labels = &no_labels; 95 | for (auto key = kvs; key != arg.end();) { 96 | auto value = std::find(key, arg.end(), '='); 97 | if (value == arg.end()) 98 | usage(); 99 | auto next = std::find(value, arg.end(), ':'); 100 | // TODO(ed): Use C++17 emplace_front(). 101 | labels.emplace_front(std::string_view(key + 1, value - (key + 1)), 102 | std::string_view(value + 1, next - (value + 1))); 103 | joiners.emplace_front(range_labels, &labels.front()); 104 | range_labels = &joiners.front(); 105 | key = next; 106 | } 107 | ranges.AddRange(range_labels, 108 | parse_ipv4_address(std::string(arg.begin(), endaddr)), 109 | parse_ipv4_address(std::string(endaddr + 1, kvs))); 110 | break; 111 | } 112 | default: 113 | usage(); 114 | } 115 | } 116 | argc -= optind; 117 | argv += optind; 118 | if (argc != 0 || interfaces.empty()) 119 | usage(); 120 | 121 | // Create pcap handles and allocate histograms. 122 | std::vector pcaps; 123 | std::vector packet_counters; 124 | for (const std::string& interface : interfaces) { 125 | // TODO(ed): Use C++17 emplace_back(). 126 | // Pcap& pcap = pcaps.emplace_back(); 127 | pcaps.emplace_back(); 128 | Pcap& pcap = pcaps.back(); 129 | std::optional error = 130 | pcap.Activate(interface, PacketParser::BytesNeededIPv4, 1 << 24); 131 | if (error) { 132 | std::cerr << "Failed to activate pcap for interface " << interface << ": " 133 | << *error << std::endl; 134 | std::exit(1); 135 | } 136 | packet_counters.emplace_back(&ranges); 137 | } 138 | 139 | // Create HTTP server that returns metrics for all interfaces. 140 | PacketCounterServer packet_counter_server(&interfaces, &packet_counters); 141 | Webserver webserver(&packet_counter_server); 142 | webserver.BindAndListen(httpport); 143 | 144 | // Spawn a small number of worker threads for HTTP GET requests. 145 | std::vector webserver_workers; 146 | for (int i = 0; i < 5; ++i) { 147 | webserver_workers.push_back(std::thread([&webserver]() { 148 | for (;;) 149 | webserver.Dispatch(); 150 | })); 151 | } 152 | 153 | // Count incoming network packets in the main thread at a fixed 10 Hz 154 | // rate. This has the advantage of reducing CPU load significantly, as 155 | // libpcap tends to already unblock when a very small number of 156 | // packets are available for processing. 157 | for (;;) { 158 | for (std::size_t i = 0; i < pcaps.size(); ++i) { 159 | PacketParser parser(&packet_counters[i]); 160 | pcaps[i].Dispatch(&parser); 161 | } 162 | std::this_thread::sleep_for(std::chrono::milliseconds(100)); 163 | } 164 | } 165 | -------------------------------------------------------------------------------- /src/protocol_histogram.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef PROTOCOL_HISTOGRAM_H 7 | #define PROTOCOL_HISTOGRAM_H 8 | 9 | #include 10 | #include 11 | #include 12 | 13 | #include "metrics_labels.h" 14 | 15 | namespace { 16 | constexpr std::pair kIanaProtocolNumbers[] = { 17 | {1, "icmp"}, {2, "igmp"}, {6, "tcp"}, {17, "udp"}, {41, "encap"}, 18 | {47, "gre"}, {50, "esp"}, {88, "eigrp"}, {89, "ospf"}, {132, "sctp"}, 19 | }; 20 | } 21 | 22 | // Multiplexer for storing a histogram per transport layer protocol. 23 | // 24 | // This template can be wrapped around the Histogram class to create 25 | // multiple instances, each keeping track of stats of a value per 26 | // transport layer protocol. 27 | template 28 | class ProtocolHistogram { 29 | public: 30 | // Stores a new sample value in one of its histogram objects. 31 | template 32 | void Record(std::uint8_t protocol, Args&&... args) { 33 | for (std::size_t i = 0; i < std::size(kIanaProtocolNumbers); ++i) { 34 | if (protocol == kIanaProtocolNumbers[i].first) { 35 | known_[i].Record(std::forward(args)...); 36 | return; 37 | } 38 | } 39 | unknown_.Record(std::forward(args)...); 40 | } 41 | 42 | // Prints all values stored in all of its histogram objects. 43 | void PrintMetrics(const std::string& name, const MetricsLabels& labels, 44 | MetricsPage* output) const { 45 | for (std::size_t i = 0; i < std::size(kIanaProtocolNumbers); ++i) { 46 | MetricsLabel protocol("protocol", kIanaProtocolNumbers[i].second); 47 | MetricsLabelsJoiner joiner(&labels, &protocol); 48 | known_[i].PrintMetrics(name, joiner, output); 49 | } 50 | unknown_.PrintMetrics(name, labels, output); 51 | } 52 | 53 | private: 54 | T known_[std::size(kIanaProtocolNumbers)]; // Stats for all known protocols. 55 | T unknown_; // Stats for all other traffic. 56 | }; 57 | 58 | #endif 59 | -------------------------------------------------------------------------------- /src/raw_packet_processor.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016-2017 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef RAW_PACKET_PROCESSOR_H 7 | #define RAW_PACKET_PROCESSOR_H 8 | 9 | #include 10 | #include 11 | 12 | // Interface for handling raw network packets. 13 | class RawPacketProcessor { 14 | public: 15 | virtual void ProcessPacket(std::basic_string_view bytes, 16 | std::size_t original_length) = 0; 17 | }; 18 | 19 | #endif 20 | -------------------------------------------------------------------------------- /src/webserver.cc: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #include 7 | #include 8 | 9 | #include 10 | 11 | #include 12 | 13 | #include 14 | #include 15 | #include 16 | 17 | #include "webserver.h" 18 | #include "webserver_request_handler.h" 19 | 20 | Webserver::~Webserver() { 21 | if (fd_ >= 0) 22 | close(fd_); 23 | } 24 | 25 | void Webserver::BindAndListen(std::uint16_t port) { 26 | // Create socket. 27 | assert(fd_ == -1 && "Webserver is already listening."); 28 | fd_ = socket(AF_INET, SOCK_STREAM, 0); 29 | assert(fd_ >= 0 && "Failed to create socket."); 30 | 31 | // Allow binding if there are still connections in TIME_WAIT. 32 | { 33 | int set = 1; 34 | setsockopt(fd_, SOL_SOCKET, SO_REUSEADDR, (void*)&set, sizeof(int)); 35 | } 36 | 37 | // Bind socket. 38 | { 39 | struct sockaddr_in sin = {}; 40 | sin.sin_family = AF_INET; 41 | sin.sin_addr.s_addr = htonl(INADDR_ANY); 42 | sin.sin_port = htons(port); 43 | int ret = bind(fd_, (struct sockaddr*)&sin, sizeof(sin)); 44 | assert(ret == 0 && "Failed to bind socket."); 45 | } 46 | 47 | // Listen for incoming connections. 48 | { 49 | int ret = listen(fd_, 0); 50 | assert(ret == 0 && "Failed to listen on socket."); 51 | } 52 | } 53 | 54 | void Webserver::Dispatch() { 55 | // Accept new incoming connection. 56 | assert(fd_ >= 0 && "Cannot dispatch on unbound socket."); 57 | int conn = accept(fd_, nullptr, nullptr); 58 | assert(conn >= 0 && "Failed to accept incoming connection."); 59 | 60 | // Disable signalling on socket, so we don't get SIGPIPE. 61 | { 62 | int set = 1; 63 | #ifdef MSG_NOSIGNAL 64 | setsockopt(conn, SOL_SOCKET, MSG_NOSIGNAL, (void*)&set, sizeof(int)); 65 | #else 66 | setsockopt(conn, SOL_SOCKET, SO_NOSIGPIPE, (void*)&set, sizeof(int)); 67 | #endif 68 | } 69 | 70 | { 71 | // Compute response body. 72 | std::ostringstream body; 73 | handler_->HandleRequest(&body); 74 | std::string body_str = body.str(); 75 | 76 | // Compute response headers. 77 | std::ostringstream headers; 78 | headers << "HTTP/1.1 200 OK\r\n" 79 | << "Connection: close\r\n" 80 | << "Content-Length: " << body_str.size() << "\r\n" 81 | << "Content-Type: text/plain\r\n" 82 | << "\r\n"; 83 | std::string headers_str = headers.str(); 84 | 85 | // Send response headers and body over the socket. 86 | struct iovec iov[2]; 87 | iov[0].iov_base = (void*)headers_str.data(); 88 | iov[0].iov_len = headers_str.size(); 89 | iov[1].iov_base = (void*)body_str.data(); 90 | iov[1].iov_len = body_str.size(); 91 | writev(conn, iov, 2); 92 | } 93 | 94 | // Initiate shutdown and drain until the peer has disconnected. 95 | shutdown(conn, SHUT_WR); 96 | char discard[1024]; 97 | while (read(conn, discard, sizeof(discard)) > 0) { 98 | } 99 | close(conn); 100 | } 101 | -------------------------------------------------------------------------------- /src/webserver.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef WEBSERVER_H 7 | #define WEBSERVER_H 8 | 9 | #include 10 | 11 | class WebserverRequestHandler; 12 | 13 | // A simple HTTP web server. 14 | // 15 | // This web server is not capable of parsing actual HTTP requests. It 16 | // invokes WebserverRequestHandler::HandleRequest() for every incoming 17 | // request and returns the string generated, serving it back with MIME 18 | // type 'text/plain'. 19 | class Webserver { 20 | public: 21 | Webserver(WebserverRequestHandler* handler) : handler_(handler), fd_(-1) { 22 | } 23 | ~Webserver(); 24 | 25 | // Binds the web server to a given port number and starts listening 26 | // for incoming requests. 27 | void BindAndListen(std::uint16_t port); 28 | 29 | // Blocks and processes a single incoming HTTP request. 30 | void Dispatch(); 31 | 32 | private: 33 | WebserverRequestHandler* const handler_; 34 | int fd_; 35 | }; 36 | 37 | #endif 38 | -------------------------------------------------------------------------------- /src/webserver_request_handler.h: -------------------------------------------------------------------------------- 1 | // Copyright (c) 2016 Kumina, https://kumina.nl/ 2 | // 3 | // This file is distributed under a 2-clause BSD license. 4 | // See the LICENSE file for details. 5 | 6 | #ifndef WEBSERVER_REQUEST_HANDLER_H 7 | #define WEBSERVER_REQUEST_HANDLER_H 8 | 9 | #include 10 | 11 | // Interface for handlers for HTTP requests. 12 | // 13 | // This implementation does not yet allow you to distinguish between 14 | // URLs, HTTP methods, extract headers, etc. It can only be used to 15 | // return a single response, which is good enough for what we need. 16 | class WebserverRequestHandler { 17 | public: 18 | // Processes an incoming HTTP request, by writing the response body to 19 | // the output stream. 20 | virtual void HandleRequest(std::ostream* output) = 0; 21 | }; 22 | 23 | #endif 24 | --------------------------------------------------------------------------------