diff --git a/docs/spectator/lang/cpp/meters/age-gauge.md b/docs/spectator/lang/cpp/meters/age-gauge.md new file mode 100644 index 00000000..30872656 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/age-gauge.md @@ -0,0 +1,67 @@ +The value is the time in seconds since the epoch at which an event has successfully occurred, or +`0` to use the current time in epoch seconds. After an Age Gauge has been set, it will continue +reporting the number of seconds since the last time recorded, for as long as the SpectatorD +process runs. The purpose of this metric type is to enable users to more easily implement the +Time Since Last Success alerting pattern. + +To set a specific time as the last success: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create an Age Gauge + auto successAgeGauge = r.CreateAgeGauge("time.sinceLastSuccess"); + successAgeGauge.Set(1611081000); + + // Option 2: Create an Age Gauge from a MeterID + auto successMeter = r.CreateNewId("time.sinceLastSuccess"); + r.CreateAgeGauge(successMeter).Set(1611081000); +} +``` + +To set `Now()` as the last success: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create an Age Gauge + auto successAgeGauge = r.CreateAgeGauge("time.sinceLastSuccess"); + successAgeGauge.Now(); + + // Option 2: Create an Age Gauge from a MeterID + auto successMeter = r.CreateNewId("time.sinceLastSuccess"); + r.CreateAgeGauge(successMeter).Now(); +} +``` + +By default, a maximum of `1000` Age Gauges are allowed per `spectatord` process, because there is no +mechanism for cleaning them up. This value may be tuned with the `--age_gauge_limit` flag on the +`spectatord` binary. + +Since Age Gauges are long-lived entities that reside in the memory of the SpectatorD process, if +you need to delete and re-create them for any reason, then you can use the [SpectatorD admin server] +to accomplish this task. You can delete all Age Gauges or a single Age Gauge. + +**Example:** + +``` +curl -X DELETE \ +http://localhost:1234/metrics/A +``` + +``` +curl -X DELETE \ +http://localhost:1234/metrics/A/fooIsTheName,some.tag=val1,some.otherTag=val2 +``` + +[SpectatorD admin server]: ../../../agent/usage.md#admin-server diff --git a/docs/spectator/lang/cpp/meters/counter.md b/docs/spectator/lang/cpp/meters/counter.md new file mode 100644 index 00000000..cb4c1343 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/counter.md @@ -0,0 +1,46 @@ +A Counter is used to measure the rate at which an event is occurring. Considering an API endpoint, +a Counter could be used to measure the rate at which it is being accessed. + +Counters are reported to the backend as a rate-per-second. In Atlas, the `:per-step` operator can +be used to convert them back into a value-per-step on a graph. + +Call `Increment()` when an event occurs: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Counter + auto serverRequestCounter = r.CreateCounter("server.numRequests"); + serverRequestCounter.Increment(); + + // Option 2: Create a Counter from a MeterID + auto serverRequestMeter = r.CreateNewId("server.numRequests"); + r.CreateCounter(serverRequestMeter).Increment(); +} +``` + +You can also pass a value to `Increment()`. This is useful when a collection of events happens +together: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Counter + auto serverRequestCounter = r.CreateCounter("server.numRequests"); + serverRequestCounter.Increment(10); + + // Option 2: Create a Counter from a MeterID + auto serverRequestMeter = r.CreateNewId("server.numRequests"); + r.CreateCounter(serverRequestMeter).Increment(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/dist-summary.md b/docs/spectator/lang/cpp/meters/dist-summary.md new file mode 100644 index 00000000..4ae31c9e --- /dev/null +++ b/docs/spectator/lang/cpp/meters/dist-summary.md @@ -0,0 +1,29 @@ +A Distribution Summary is used to track the distribution of events. It is similar to a Timer, but +more general, in that the size does not have to be a period of time. For example, a Distribution +Summary could be used to measure the payload sizes of requests hitting a server. Note that the C++ +implementation of Distribution Summary allows for the recording of floating point values, which the +other thin clients do not allow. + +Always use base units when recording data, to ensure that the tick labels presented on Atlas graphs +are readable. If you are measuring payload size, then use bytes, not kilobytes (or some other unit). +This means that a `4K` tick label will represent 4 kilobytes, rather than 4 kilo-kilobytes. + +Call `Record()` with a value: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Distribution Summary + auto serverRequestSize = r.CreateDistributionSummary("server.requestSize"); + serverRequestSize.Record(42); + + // Option 2: Create a Distribution Summary from a MeterID + auto serverRequestMeter = r.CreateNewId("server.requestSize"); + r.CreateDistributionSummary(serverRequestMeter).Record(42); +} +``` diff --git a/docs/spectator/lang/cpp/meters/gauge.md b/docs/spectator/lang/cpp/meters/gauge.md new file mode 100644 index 00000000..bc2c38d3 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/gauge.md @@ -0,0 +1,49 @@ +A gauge is a value that is sampled at some point in time. Typical examples for gauges would be +the size of a queue or number of threads in a running state. Since gauges are not updated inline +when a state change occurs, there is no information about what might have occurred between samples. + +Consider monitoring the behavior of a queue of tasks. If the data is being collected once a minute, +then a gauge for the size will show the size when it was sampled. The size may have been much +higher or lower at some point during interval, but that is not known. + +Call `Set()` with a value: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Gauge + auto serverQueueSize = r.CreateGauge("server.queueSize"); + serverQueueSize.Set(10); + + // Option 2: Create a Gauge from a MeterID + auto serverQueueMeter = r.CreateNewId("server.queueSize"); + r.CreateGauge(serverQueueMeter).Set(10); +} +``` + +Gauges will report the last set value for 15 minutes. This done so that updates to the values do +not need to be collected on a tight 1-minute schedule to ensure that Atlas shows unbroken lines in +graphs. A custom TTL may be configured for gauges. SpectatorD enforces a minimum TTL of 5 seconds. + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Gauge + auto serverQueueSize = r.CreateGauge("server.queueSize", {}, 120); + serverQueueSize.Set(10); + + // Option 2: Create a Gauge from a MeterID + auto serverQueueMeter = r.CreateNewId("server.queueSize"); + r.CreateGauge(serverQueueMeter, 120).Set(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/max-gauge.md b/docs/spectator/lang/cpp/meters/max-gauge.md new file mode 100644 index 00000000..8ad17a9e --- /dev/null +++ b/docs/spectator/lang/cpp/meters/max-gauge.md @@ -0,0 +1,24 @@ +The value is a number that is sampled at a point in time, but it is reported as a maximum Gauge +value to the backend. This ensures that only the maximum value observed during a reporting interval +is sent to the backend, thus over-riding the last-write-wins semantics of standard Gauges. Unlike +standard Gauges, Max Gauges do not continue to report to the backend, and there is no TTL. + +Call `Set()` with a value: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Max Gauge + auto serverQueueSize = r.CreateMaxGauge("server.queueSize"); + serverQueueSize.Set(10); + + // Option 2: Create a Gauge from a MeterID + auto serverQueueMeter = r.CreateNewId("server.queueSize"); + r.CreateMaxGauge(serverQueueMeter).Set(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md new file mode 100644 index 00000000..5284a99e --- /dev/null +++ b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md @@ -0,0 +1,25 @@ +A Monotonic Counter (uint64) is used to measure the rate at which an event is occurring, when the +source data is a monotonically increasing number. A minimum of two samples must be sent, in order to +calculate a delta value and report it to the backend as a rate-per-second. A variety of networking +metrics may be reported monotonically, and this metric type provides a convenient means of recording +these values, at the expense of a slower time-to-first metric. + +Call `Set()` when an event occurs: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Monotonic Counter uint64_t + auto interfaceBytes = r.CreateMonotonicCounterUint("iface.bytes"); + interfaceBytes.Set(10); + + // Option 2: Create a Monotonic Counter uint64_t from a MeterID + auto interfaceBytesMeter = r.CreateNewId("iface.bytes"); + r.CreateMonotonicCounterUint(interfaceBytesMeter).Set(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter.md b/docs/spectator/lang/cpp/meters/monotonic-counter.md new file mode 100644 index 00000000..06d3580d --- /dev/null +++ b/docs/spectator/lang/cpp/meters/monotonic-counter.md @@ -0,0 +1,25 @@ +A Monotonic Counter (float) is used to measure the rate at which an event is occurring, when the +source data is a monotonically increasing number. A minimum of two samples must be sent, in order to +calculate a delta value and report it to the backend as a rate-per-second. A variety of networking +metrics may be reported monotonically, and this metric type provides a convenient means of recording +these values, at the expense of a slower time-to-first metric. + +Call `Set()` when an event occurs: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Monotonic Counter + auto interfaceBytes = r.CreateMonotonicCounter("iface.bytes"); + interfaceBytes.Set(10); + + // Option 2: Create a Monotonic Counter from a MeterID + auto interfaceBytesMeter = r.CreateNewId("iface.bytes"); + r.CreateMonotonicCounter(interfaceBytesMeter).Set(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md new file mode 100644 index 00000000..0bc4aae2 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md @@ -0,0 +1,31 @@ +The value tracks the distribution of events, with percentile estimates. It is similar to a +`PercentileTimer`, but more general, because the size does not have to be a period of time. + +For example, it can be used to measure the payload sizes of requests hitting a server or the +number of records returned from a query. Note that the C++ implementation of Percentile Distribution +Summary allows for the recording of floating point values, which the other thin clients do not +allow. + +In order to maintain the data distribution, they have a higher storage cost, with a worst-case of +up to 300X that of a standard Distribution Summary. Be diligent about any additional dimensions +added to Percentile Distribution Summaries and ensure that they have a small bounded cardinality. + +Call `Record()` with a value: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Percentile Distribution Summary + auto serverSize = r.CreatePercentDistributionSummary("server.requestSize"); + serverSize.Record(10); + + // Option 2: Create a Percentile Distribution Summary from a MeterID + auto requestSizeMeter = r.CreateNewId("server.requestSize"); + r.CreatePercentDistributionSummary(requestSizeMeter).Record(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/percentile-timer.md b/docs/spectator/lang/cpp/meters/percentile-timer.md new file mode 100644 index 00000000..ded4221e --- /dev/null +++ b/docs/spectator/lang/cpp/meters/percentile-timer.md @@ -0,0 +1,29 @@ +The value is the number of seconds that have elapsed for an event, with percentile estimates. + +This metric type will track the data distribution by maintaining a set of Counters. The +distribution can then be used on the server side to estimate percentiles, while still +allowing for arbitrary slicing and dicing based on dimensions. + +In order to maintain the data distribution, they have a higher storage cost, with a worst-case of +up to 300X that of a standard Timer. Be diligent about any additional dimensions added to Percentile +Timers and ensure that they have a small bounded cardinality. + +Call `Record()` with a value: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Percentile Timer + auto serverLatency = r.CreatePercentTimer("server.requestLatency"); + serverLatency.Record(10); + + // Option 2: Create a Percentile Timer from a MeterID + auto requestLatencyMeter = r.CreateNewId("server.requestLatency"); + r.CreatePercentTimer(requestLatencyMeter).Record(10); +} +``` diff --git a/docs/spectator/lang/cpp/meters/timer.md b/docs/spectator/lang/cpp/meters/timer.md new file mode 100644 index 00000000..e770bf6c --- /dev/null +++ b/docs/spectator/lang/cpp/meters/timer.md @@ -0,0 +1,21 @@ +A Timer is used to measure how long (in seconds) some event is taking. + +Call `Record()` with a value: + +```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::UDP)); + auto r = Registry(config); + + // Option 1: Directly create a Timer + auto serverLatency = r.CreateTimer("server.requestLatency"); + serverLatency.Record(10); + + // Option 2: Create a Timer from a MeterID + auto requestLatencyMeter = r.CreateNewId("server.requestLatency"); + r.CreateTimer(requestLatencyMeter).Record(10); +} +``` diff --git a/docs/spectator/lang/cpp/migrations.md b/docs/spectator/lang/cpp/migrations.md new file mode 100644 index 00000000..2ab24987 --- /dev/null +++ b/docs/spectator/lang/cpp/migrations.md @@ -0,0 +1,42 @@ +## Migrating to 2.X + +Version 2.X consists of a major rewrite that greatly simplifies spectator-cpp and the process in +which it sends metrics to SpectatorD. + +### New + +#### Writers + +`spectator.Registry` now supports 3 different writers. The WriterType is specified through a WriterConfig object. + +See [Usage > Output Location](usage.md#output-location) for more details. + +#### Common Tags + +A few local environment common tags are now automatically added to all Meters. Their values are read +from the environment variables. + +| Tag | Environment Variable | +|--------------|----------------------| +| nf.container | TITUS_CONTAINER_NAME | +| nf.process | NETFLIX_PROCESS_NAME | + +Tags from environment variables take precedence over tags passed on code when creating the `Config`. + +Note that common tags sourced by [spectatord](https://github.com/Netflix-Skunkworks/spectatord) can't be overwritten. + +#### Registry, Config, and Writer Config + +* `Config` is now created through a constructor which throws an error, if the passed in parameters are not valid. +* `WriterConfig` now specifies which writer type the thin client uses. +* `WriterConfig` allows line buffering for all writer types. +* `Registry` is instantiated by passing only a `Config` object to it. + +### Migration Steps + +1. Remove old references to the old spectator library implementation. +2. Utilize the `Config` & `WriterConfig` to initialize the `Registry`. +3. Currently there is no support for collecting runtime metrics, using the spectator-cpp library. +4. If you need to configure a `Registry` that doesn't emit metrics, for testing purposes, you can +use the `WriterConfig` to configure a `MemoryWriter`. This will emit metrics to a vector, so make +sure to clear the vector every so often. diff --git a/docs/spectator/lang/cpp/perf-test.md b/docs/spectator/lang/cpp/perf-test.md new file mode 100644 index 00000000..ebe2f7f7 --- /dev/null +++ b/docs/spectator/lang/cpp/perf-test.md @@ -0,0 +1,64 @@ +# Performance + +## Test Script + +Test maximum single-threaded throughput for two minutes. + +```cpp +#include +#include +#include +#include +#include +#include + + +int main() +{ + Logger::info("Starting UDP performance test..."); + + //auto r = Registry(Config(WriterConfig(WriterTypes::UDP))); + auto r = Registry(Config(WriterConfig(WriterTypes::Unix))); + + std::unordered_map tags = { {"location", "udp"}, {"version", "correct-horse-battery-staple"}}; + + // Set maximum duration to 2 minutes + constexpr int max_duration_seconds = 2 * 60; + + // Track iterations and timing + unsigned long long iterations = 0; + auto start_time = std::chrono::steady_clock::now(); + + // Helper function to get elapsed time in seconds + auto elapsed = [&start_time]() -> double { + auto now = std::chrono::steady_clock::now(); + return std::chrono::duration(now - start_time).count(); + }; + + while (true) + { + r.CreateCounter("udp_test_counter", tags).Increment(); + iterations++; + + if (iterations % 500000 == 0) + { + if (elapsed() > max_duration_seconds) + { + break; + } + } + } + + double total_elapsed = elapsed(); + double rate_per_second = iterations / total_elapsed; + + Logger::info("Iterations completed: {}", iterations); + Logger::info("Total elapsed time: {:.2f} seconds", total_elapsed); + Logger::info("Rate: {:.2f} iterations/second", rate_per_second); + return 0; +} +``` + +## Results + +See [Usage > Performance](usage.md#performance). diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index b15e3e2b..5054d6c0 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -1,112 +1,287 @@ # spectator-cpp Usage -C++ thin-client [metrics library] for use with [Atlas] and [SpectatorD]. +CPP thin-client [metrics library] for use with [Atlas] and [SpectatorD]. [metrics library]: https://github.com/Netflix/spectator-cpp [Atlas]: ../../../overview.md [SpectatorD]: ../../agent/usage.md +## Supported CPP Versions + +This library currently utilizes C++ 20. + +## Installing & Building + +If your project uses CMake, you can easily integrate this library by calling `add_subdirectory()` +on the root folder. To build the Spectator-CPP thin client independently, follow the Docker +container instructions at [Dockerfiles](https://github.com/Netflix/spectator-cpp/tree/main/Dockerfiles). +The container provides a minimal build environment with g++-13, python3, and conan. Spectator-CPP +relies on just three external dependencies (spdlog, gtest, and boost), which are managed automatically +via conan. + ## Instrumenting Code -```C++ -#include - -// use default values -static constexpr auto kDefault = 0; - -struct Request { - std::string country; -}; - -struct Response { - int status; - int size; -}; - -class Server { - public: - explicit Server(spectator::Registry* registry) - : registry_{registry}, - request_count_id_{registry->CreateId("server.requestCount", spectator::Tags{})}, - request_latency_{registry->GetTimer("server.requestLatency")}, - response_size_{registry->GetDistributionSummary("server.responseSizes")} {} - - Response Handle(const Request& request) { - auto start = std::chrono::steady_clock::now(); - - // do some work and obtain a response... - Response res{200, 64}; - - // Update the Counter id with dimensions, based on information in the request. The Counter - // will be looked up in the Registry, which is a fairly cheap operation, about the same as - // the lookup of an id object in a map. However, it is more expensive than having a local - // variable set to the Counter. - auto cnt_id = request_count_id_ - ->WithTag("country", request.country) - ->WithTag("status", std::to_string(res.status)); - registry_->GetCounter(std::move(cnt_id))->Increment(); - request_latency_->Record(std::chrono::steady_clock::now() - start); - response_size_->Record(res.size); - return res; - } - - private: - spectator::Registry* registry_; - std::shared_ptr request_count_id_; - std::shared_ptr request_latency_; - std::shared_ptr response_size_; -}; - -Request get_next_request() { - return Request{"US"}; +{% raw %} + +```cpp +#include + +int main() +{ + // Create common tags to be applied to all metrics sent to Atlas + std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; + + // Create a config which defines the way you send metrics to SpectatorD + auto config = Config(WriterConfig(WriterTypes::UDP), commonTags); + + // Initialize the Registry with the Config (Always required before sending metrics) + auto r = Registry(config); + + // Create some meters + auto threadGauge = r.CreateGauge("threads"); + auto queueGauge = r.CreateGauge("queue-size", {{"my-tags", "bar"}}); + + threadGauge.Set(GetNumThreads()); + queueGauge.Set(GetQueueSize()); + + /* Metrics Sent: + "g:threads,platform=my-platform,process=my-process:5.000000\n" + "g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n" + */ } +``` + +{% endraw %} + +## Logging + +Logging uses the `spdlog` library and outputs to standard output by default, with a default log +level of `spdlog::level::info`. The `Logger` class is a singleton and provides the +`Logger::GetLogger()` function to access the logger instance. To change the log level, call +`Logger::GetLogger()->set_level(spdlog::level::debug);` after the logger has been successfully +created. + +## Working with MeterId Objects + +Each metric stored in Atlas is uniquely identified by the combination of the name and the tags +associated with it. In `spectator-cpp`, this data is represented with `MeterId` objects, created +by the `Registry`. The `CreateNewId()` method returns new a `MeterId` object, which has extra +common tags applied, and which can be further customized by calling the `WithTag()` and +`WithTags()` methods. Each `MeterId` will create and store a validated subset of the `spectatord` +protocol line to be written for each `Meter`, when it is instantiated. Manipulating the tags with +the provided methods will create new `MeterId` objects. + +Note that **all tag keys and values must be strings.** Tags are represented as +`std::unordered_map`. For example, to track the number of successful +requests, ensure your tags use string values, such as `{"statusCode": std::to_string(200)}`. The +`MeterId` class validates all provided tag keys and values: if either is empty or contains only +whitespace, it will be dropped, and any invalid characters will be replaced with an underscore. + +{% raw %} -int main() { - auto logger = spdlog::stdout_color_mt("console"); - std::unordered_map common_tags{{"xatlas.process", "some-sidecar"}}; - spectator::Config cfg{"unix:/run/spectatord/spectatord.unix", common_tags}; - spectator::Registry registry{std::move(cfg), logger); +```cpp +#include - Server server{®istry}; +int main() +{ + // Create common tags + std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; - for (auto i = 1; i <= 3; ++i) { - // get a request - auto req = get_next_request(); - server.Handle(req); - } + // Initialize the Registry + auto config = Config(WriterConfig(WriterTypes::UDP), commonTags); + auto registry = Registry(config); + + + // Option 1: Using the registry to create a MeterId & creating a Counter from the MeterId + auto numRequestsId = registry.CreateNewId("server.numRequests", {{"statusCode", std::to_string(200)}}); + registry.CreateCounter(numRequestsId).Increment(); + + // Option 2: Directly creating a Counter + auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}}); + numRequestsCounter.Increment(); } ``` -## Usage +{% endraw %} + +Atlas metrics will be consumed by users many times after the data has been reported, so they should +be chosen thoughtfully, while considering how they will be used. See the [naming conventions] page +for general guidelines on metrics naming and restrictions. + +[naming conventions]: ../../../concepts/naming.md + +## Meter Types + +* [Age Gauge](./meters/age-gauge.md) +* [Counter](./meters/counter.md) +* [Distribution Summary](./meters/dist-summary.md) +* [Gauge](./meters/gauge.md) +* [Max Gauge](./meters/max-gauge.md) +* [Monotonic Counter](./meters/monotonic-counter.md) +* [Monotonic Counter Uint](./meters/monotonic-counter-uint.md) +* [Percentile Distribution Summary](./meters/percentile-dist-summary.md) +* [Percentile Timer](./meters/percentile-timer.md) +* [Timer](./meters/timer.md) + +## Output Locations + +`spectator.Registry` supports three output writer types: Memory Writer, UDP Writer, and Unix Domain +Socket (UDS) Writer. To specify the writer type, initialize the Registry with a `Config` object. A +`Config` requires a `WriterConfig` (which defines the writer type and location) and can optionally +include extra tags to be applied to all metrics. The `WriterConfig` also accepts an optional +buffer size parameter, enabling buffering for all writer types. + +### Writer Config Constructors + +```cpp +// Constructor 1: Define a location and no buffering +WriterConfig(const std::string& type) + +// Constructor 2: Define a location with buffering +WriterConfig(const std::string& type, unsigned int bufferSize); +``` + +### Writer Config Examples + +```cpp +/* Default Writer Config Examples */ + +// Write metrics to memory for testing +WriterConfig wConfig(WriterTypes::Memory); + +// Default UDP address for spectatord +WriterConfig wConfig(WriterTypes::UDP); -We do not publish this library as a binary artifact, because it can be used across a variety of CPU -and OS platforms, and we do not want to incur this support overheard for a library that is not on -the Paved Path. However, this is a Conan 2 and CMake project, so you can pull the latest code, and -add some build configuration to use it in your project. +// Default UDS address for spectatord with buffering +WriterConfig wConfig(WriterTypes::Unix, 4096); -As an example of how this is done, see the [atlas-system-agent] project. +/* Custom Writer Config Location Examples */ -* Download the latest `spectator-cpp` code ([conanfile.py#L39-L57]). -* Add the library to your CMake build ([lib/CMakeLists.txt#L1-L32]). +// Custom UDP writer location +std::string udpUrl = std::string(WriterTypes::UDPURL) + "192.168.1.100:8125"; +WriterConfig wConfig(udpUrl); -This library has a few dependencies ([conanfile.py#L6-L13]), including a recent `abseil`. +// Custom UDS writer location +std::string unixUrl = std::string(WriterTypes::UnixURL) + "/var/run/custom/socket.sock"; +WriterConfig wConfig(unixUrl, 4096); +``` + +### Config Constructor + +```cpp +// Constructor: WriterConfig & optional extraTags for all metrics +Config(const WriterConfig& writerConfig, const std::unordered_map& extraTags = {}); +``` + +### Config Examples + +{% raw %} + +```cpp +/* Config Examples */ + +// Config with a WriterConfig & no extra tags +Config config = Config(WriterConfig(WriterTypes::Memory)); + +// Config with a WriterConfig & extra tags +std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; +Config config = Config(WriterConfig(WriterTypes::Memory), commonTags); + +/* Registry Initialization */ +WriterConfig wConfig(WriterTypes::Memory); +Config config = Config(wConfig); +Registry registry(config); +``` + +{% endraw %} + +Location can also be set through the environment variable `SPECTATOR_OUTPUT_LOCATION`. If both are +set, the environment variable takes precedence over the value passed to the WriterConfig. If either +values provided to the WriterConfig are invalid, then a runtime exception will be thrown. + +## Line Buffer + +The `WriterConfig` allows you to set an optional `bufferSize` parameter. If `bufferSize` is not +set, each metric is sent immediately to `spectatord` using the configured writer type. If +`bufferSize` is set, metrics are buffered locally and only flushed to `spectatord` when the buffer +exceeds the specified size. For high-performance scenarios, a buffer size of 60KB is recommended. +The maximum buffer size for UDP and Unix Domain Socket writers on Linux is 64KB, so ensure your +buffer size does not exceed this limit. + +## Batch Usage + +When using `spectator-cpp` to report metrics from a batch job, ensure that the batch job runs for at +least five (5), if not ten (10) seconds in duration. This is necessary in order to allow sufficient +time for `spectatord` to publish metrics to the Atlas backend; it publishes every five seconds. If +your job does not run this long, or you find you are missing metrics that were reported at the end +of your job run, then add a five-second sleep before exiting. This will allow time for the metrics +to be sent. + +## Debug Metrics Delivery to `spectatord` + +In order to see debug log messages from `spectatord`, create an `/etc/default/spectatord` file with +the following contents: + +```shell +SPECTATORD_OPTIONS="--verbose" +``` + +This will report all metrics that are sent to the Atlas backend in the `spectatord` logs, which will +provide an opportunity to correlate metrics publishing events from your client code. + +## Design Considerations - Reporting Intervals + +This client is stateless, and sends a UDP packet (or unixgram) to `spectatord` each time a meter is +updated. If you are performing high-volume operations, on the order of tens-of-thousands or millions +of operations per second, then you should pre-aggregate your metrics and report them at a cadence +closer to the `spectatord` publish interval of 5 seconds. This will keep the CPU usage related to +`spectator-cpp` and `spectatord` low (around 1% or less), as compared to up to 40% for high-volume +scenarios. If you choose to use the `WriterConfig` with the buffering feature enabled, metrics will +only be sent when the buffer exceeds its size. The buffer is protected by a mutex, allowing +multiple threads to safely write metrics concurrently. + +## Writing Tests + +To write tests against this library, instantiate a test instance of the `Registry` and configure it +to use the `MemoryWriter`, which stores all updates in a `Vector`. Maintain a handle to the +`MemoryWriter`, then inspect the protocol lines with `GetMessages()` to verify your metric updates. +See the source code for more testing examples. + +{% raw %} + +```cpp +int main() +{ + // Initialize Registry + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto registry = Registry(config); + + // Directly create a Counter + auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}}); + + // Create a handle to the Writer + auto memoryWriter = static_cast(WriterTestHelper::GetImpl()); + + numRequestsCounter.Increment(); + + auto messages = memoryWriter->GetMessages(); + for (const auto& message : messages) { + std::cout << message; // Print all messages sent to SpectatorD + } +} +``` -[atlas-system-agent]: https://github.com/Netflix-Skunkworks/atlas-system-agent/tree/main -[conanfile.py#L39-L57]: https://github.com/Netflix-Skunkworks/atlas-system-agent/blob/main/conanfile.py#L39-L57 -[lib/CMakeLists.txt#L1-L32]: https://github.com/Netflix-Skunkworks/atlas-system-agent/blob/main/lib/CMakeLists.txt#L1-L32 -[conanfile.py#L6-L13]: https://github.com/Netflix/spectator-cpp/blob/main/conanfile.py#L6-L13 +{% endraw %} -### High-Volume Publishing +## Performance -By default, the library sends every meter change to the `spectatord` sidecar immediately. This -involves a blocking `send` call and underlying system calls, and may not be the most efficient way -to publish metrics in high-volume use cases. +On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and +`github.com/Netflix/spectator-cpp/v2 v2.0.0`, we have observed the following single-threaded +performance numbers across a two-minute test window (unbuffered scenario): -For this purpose, a simple buffering functionality in `Publisher` is implemented, and it can be -turned on by passing a buffer size to the `spectator::Config` constructor ([config.h#L8-L12]). It -is important to note that, until this buffer fills up, the `Publisher` will not send any meters to -the sidecar. Therefore, if your application doesn't emit meters at a high rate, you should either -keep the buffer very small, or do not configure a buffer size at all, which will fall back to the -"publish immediately" mode of operation. +* 113,655.11 requests/second over `udp` +* 132,490.97 requests/second over `unix` -[config.h#L8-L12]: https://github.com/Netflix/spectator-cpp/blob/main/spectator/config.h#L8-L12 +The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world +tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol +line was `74` characters in length. diff --git a/mkdocs.yml b/mkdocs.yml index bed46ede..c1b9bba6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -228,6 +228,19 @@ nav: - Overview: spectator/lang/overview.md - C++: - Usage: spectator/lang/cpp/usage.md + - Meters: + - Age Gauge: spectator/lang/cpp/meters/age-gauge.md + - Counter: spectator/lang/cpp/meters/counter.md + - Distribution Summary: spectator/lang/cpp/meters/dist-summary.md + - Gauges: spectator/lang/cpp/meters/gauge.md + - Max Gauge: spectator/lang/cpp/meters/max-gauge.md + - Monotonic Counter: spectator/lang/cpp/meters/monotonic-counter.md + - Monotonic Counter Uint: spectator/lang/cpp/meters/monotonic-counter-uint.md + - Percentile Distribution Summary: spectator/lang/cpp/meters/percentile-dist-summary.md + - Percentile Timer: spectator/lang/cpp/meters/percentile-timer.md + - Timer: spectator/lang/cpp/meters/timer.md + - Migrations: spectator/lang/cpp/migrations.md + - Performance: spectator/lang/cpp/perf-test.md - Go: - Usage: spectator/lang/go/usage.md - Meters: @@ -235,7 +248,7 @@ nav: - Counter: spectator/lang/go/meters/counter.md - Distribution Summary: spectator/lang/go/meters/dist-summary.md - Gauges: spectator/lang/go/meters/gauge.md - - Max Gauge: spectator/lang/py/meters/max-gauge.md + - Max Gauge: spectator/lang/go/meters/max-gauge.md - Monotonic Counter: spectator/lang/go/meters/monotonic-counter.md - Monotonic Counter Uint: spectator/lang/go/meters/monotonic-counter-uint.md - Percentile Distribution Summary: spectator/lang/go/meters/percentile-dist-summary.md